Chad Rosier
649326ab15
No case stmt for BUILD_VECTOR in PerformDAGCombine(), so I assume this isn't
...
necessary. Please chime in if I'm mistaken.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147065 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-21 19:14:52 +00:00
Chandler Carruth
f2d7693fbb
Begin teaching the X86 target how to efficiently codegen patterns that
...
use the zero-undefined variants of CTTZ and CTLZ. These are just simple
patterns for now, there is more to be done to make real world code using
these constructs be optimized and codegen'ed properly on X86.
The existing tests are spiffed up to check that we no longer generate
unnecessary cmov instructions, and that we generate the very important
'xor' to transform bsr which counts the index of the most significant
one bit to the number of leading (most significant) zero bits. Also they
now check that when the variant with defined zero result is used, the
cmov is still produced.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146974 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-20 11:19:37 +00:00
Benjamin Kramer
0581ed792b
Another variadics tweak.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146852 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-18 20:51:31 +00:00
Benjamin Kramer
2ea4cdb81f
Use the fancy new VariadicFunction template instead of a plain variadic function.
...
Some compilers were complaining about passing StringRef to it.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146850 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-18 19:59:20 +00:00
Craig Topper
ab44d3cf49
Remove an unused X86ISD node type.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146833 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-17 19:16:44 +00:00
Benjamin Kramer
e6cddb77dc
X86: Factor the bswap asm matching to be slightly less horrible to read.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146831 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-17 14:36:05 +00:00
Lang Hames
8b99c1e42c
Make sure that the lower bits on the VSELECT condition are properly set.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146800 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-17 01:08:46 +00:00
Craig Topper
94438ba538
Don't try to match 'unpackl/h v, v' for 32xi8 and 16xi16 when only AVX1 is supported. Fix 'unpackh v, v' for 256-bit types to understand 128-bit lanes.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146726 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-16 08:06:31 +00:00
Chad Rosier
0660cfe3c8
Fix assert in LowerBUILD_VECTOR for v16i16 type on AVX.
...
Patch by Elena Demikhovsky <elena.demikhovsky@intel.com>!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146684 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-15 21:34:44 +00:00
Lang Hames
a0a251372f
Fix VSELECT operand order. Was previously backwards, causing bogus vector shift results - <rdar://problem/10559581>.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146671 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-15 18:57:27 +00:00
Chad Rosier
01d426e0e1
Use SmallVector/assign(), rather than std::vector/push_back().
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146627 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-15 01:16:09 +00:00
Chad Rosier
a860b189e4
Add support for lowering fneg when AVX is enabled.
...
rdar://10566486
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146625 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-15 01:02:25 +00:00
Chandler Carruth
63974b2144
Initial CodeGen support for CTTZ/CTLZ where a zero input produces an
...
undefined result. This adds new ISD nodes for the new semantics,
selecting them when the LLVM intrinsic indicates that the undef behavior
is desired. The new nodes expand trivially to the old nodes, so targets
don't actually need to do anything to support these new nodes besides
indicating that they should be expanded. I've done this for all the
operand types that I could figure out for all the targets. Owners of
various targets, please review and let me know if any of these are
incorrect.
Note that the expand behavior is *conservatively correct*, and exactly
matches LLVM's current behavior with these operations. Ideally this
patch will not change behavior in any way. For example the regtest suite
finds the exact same instruction sequences coming out of the code
generator. That's why there are no new tests here -- all of this is
being exercised by the existing test suite.
Thanks to Duncan Sands for reviewing the various bits of this patch and
helping me get the wrinkles ironed out with expanding for each target.
Also thanks to Chris for clarifying through all the discussions that
this is indeed the approach he was looking for. That said, there are
likely still rough spots. Further review much appreciated.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146466 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-13 01:56:10 +00:00
Craig Topper
d93e4c3496
Remove some remants of the old palign pattern fragment that were still hanging around. Also remove a cast from inside getShuffleVPERM2X128Immediate and getShuffleVPERMILPImmediate since the only caller already had done the cast.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146344 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-11 19:12:35 +00:00
Benjamin Kramer
b653397dcd
X86: Add patterns for the various rounding ops for SSE4.1 and AVX.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146257 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-09 15:44:03 +00:00
Owen Anderson
587e34065f
Don't explicitly marked libm rounding ops as legal on SSE4.1/AVX. There don't seem to be patterns for these, so I don't know why they were marked legal in the first place.
...
Fixes failures caused by r146171.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146180 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-08 20:51:38 +00:00
Owen Anderson
4a4fdf3476
Teach SelectionDAG to match more calls to libm functions onto existing SDNodes. Mark these nodes as illegal by default, unless the target declares otherwise.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146171 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-08 19:32:14 +00:00
Craig Topper
fe03315925
Add X86ISD::HADD/HSUB to getTargetNodeName
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145929 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-06 09:31:36 +00:00
Craig Topper
34671b812a
Merge floating point and integer UNPCK X86ISD node types.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145926 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-06 08:21:25 +00:00
Craig Topper
3d8c2ce3e4
Clean up some of the shuffle decoding code for UNPCK instructions. Add instruction commenting for AVX/AVX2 forms for integer UNPCKs.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145924 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-06 05:31:16 +00:00
Craig Topper
1ff73d7a67
Merge isSHUFPMask and isCommutedSHUFPMask into single function that can do both. Do the same for the 256-bit version. Use loops to reduce size of isVSHUFPYMask. Fix test cases that were incorrectly passing due to isCommutedSHUFPMask not checking for the vector being 128-bit. This caused some 256-bit shuffles to be incorrectly commuted.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145921 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-06 04:59:07 +00:00
Jakob Stoklund Olesen
8c741b8064
Use logarithmic units for basic block alignment.
...
This was actually a bit of a mess. TLI.setPrefLoopAlignment was clearly
documented as taking log2(bytes) units, but the x86 target would still
set a preferred loop alignment of '16'.
CodePlacementOpt passed this number on to the basic block, and
AsmPrinter interpreted it as bytes.
Now both MachineFunction and MachineBasicBlock use logarithmic
alignments.
Obviously, MachineConstantPool still measures alignments in bytes, so we
can emulate the thrill of using as.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145889 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-06 01:26:19 +00:00
Craig Topper
1dc0fbc168
Remove some leftover remnants that once tried to create 64-bit MMX PALIGNR instructions.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145804 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-05 07:27:14 +00:00
Craig Topper
beabc6cc6d
Clean up and optimizations to the X86 shuffle lowering code. No functional change.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145803 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-05 06:56:46 +00:00
Nick Lewycky
8a8d479214
Move global variables in TargetMachine into new TargetOptions class. As an API
...
change, now you need a TargetOptions object to create a TargetMachine. Clang
patch to follow.
One small functionality change in PTX. PTX had commented out the machine
verifier parts in their copy of printAndVerify. That now calls the version in
LLVMTargetMachine. Users of PTX who need verification disabled should rely on
not passing the command-line flag to enable it.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145714 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-02 22:16:29 +00:00
Craig Topper
f8363305eb
Reduce duplicate code in isHorizontalBinOp and add some asserts to protect assumptions
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145681 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-02 08:18:41 +00:00
Craig Topper
138a5c66b9
Add instruction selection support for horizontal add/sub of 256-bit floating point vectors. Also add the test case for 256-bit integer vectors.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145680 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-02 07:16:01 +00:00
Nadav Rotem
18197d7425
X86: PerformOrCombine introduced a vselect node with a wrong order of operands. This bug was introduced when a dedicated blend sdnode was replaced with the vselect node (in 139479).
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145488 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-30 10:13:37 +00:00
Craig Topper
b72039cf52
Add instruction selection support for AVX2 horizontal add/sub instructions.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145487 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-30 09:10:50 +00:00
Craig Topper
ec24e61ab0
Merge VPERM2F128/VPERM2I128 ISD node types.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145485 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-30 07:47:51 +00:00
Craig Topper
316cd2a2c5
Merge decoding of VPERMILPD and VPERMILPS shuffle masks. Merge X86ISD node type for VPERMILPD/PS. Add instruction selection support for VINSERTI128/VEXTRACTI128.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145483 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-30 06:25:25 +00:00
Craig Topper
36e36ace77
Fix issues in shuffle decoding around VPERM* instructions. Fix shuffle decoding for VSHUFPS/D for 256-bit types. Add pattern matching for memory forms of VPERMILPS/VPERMILPD.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145390 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-29 07:49:05 +00:00
Craig Topper
70b883b3a7
Add X86 instruction selection for VPERM2I128 when AVX2 is enabled. Merge VPERMILPS/VPERMILPD detection since they are pretty similar.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145238 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-28 10:14:51 +00:00
Craig Topper
71c4c12232
Make isCommutedVSHUFP more like the way isCommutedSHUFP is handled.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145218 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-28 01:14:24 +00:00
Craig Topper
9d7025b56b
Merge detecting and handling for VSHUFPSY and VSHUFPDY since a lot of the code was similar for both.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145199 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-27 21:41:12 +00:00
Craig Topper
38034c568c
Merge 128-bit and 256-bit X86ISD node types for VPERMILPS and VPERMILPD. Simplify some shuffle lowering code since V1 can never be UNDEF due to canonalizing that occurs when shuffle nodes are created.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145153 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-26 22:55:48 +00:00
Craig Topper
06cb680779
Collapse X86ISD node types for PUNPCKH*, PUNPCKL*, UNPCKLP*, and UNPCKHP* to not be type specific. Now we just have integer high and low and floating point high and low. Pattern matching will choose the correct instruction based on the vector type.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145148 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-26 20:47:44 +00:00
Craig Topper
705f2431a0
Remove 256-bit specific node types for UNPCKHPS/D and instead use the 128-bit versions and let the operand type disinquish. Also fix the load form of the v8i32 patterns for these to realize that the load would be promoted to v4i64.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145126 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-24 22:57:10 +00:00
Craig Topper
f475a55bd4
Remove AVX2 specific X86ISD node types for PUNPCKH/L and instead just reuse the 128-bit versions and let the vector type distinguish.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145125 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-24 22:20:08 +00:00
Benjamin Kramer
f238f50aaf
X86: Use btq for bit tests if the immediate can't be encoded in 32 bits.
...
Before:
movabsq $4294967296, %rax ## encoding: [0x48,0xb8,0x00,0x00,0x00,0x00,0x01,0x00,0x00,0x00]
testq %rax, %rdi ## encoding: [0x48,0x85,0xf8]
jne LBB0_2 ## encoding: [0x75,A]
After:
btq $32, %rdi ## encoding: [0x48,0x0f,0xba,0xe7,0x20]
jb LBB0_2 ## encoding: [0x72,A]
btq is usually slower than testq because it doesn't fuse with the jump, but here we're better off
saving one register and a giant movabsq.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145103 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-23 13:54:17 +00:00
Elena Demikhovsky
52a35a89e6
I added several lines in X86 code generator that allow to choose
...
VSHUFPS/VSHUFPD instructions while lowering VECTOR_SHUFFLE node. I check a commuted VSHUFP mask.
The patch was reviewed by Bruno.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145099 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-23 10:23:16 +00:00
Craig Topper
f7de577a08
Fix shuffle decoding logic to handle UNPCKLPS/UNPCKLPD on 256-bit vectors correctly. Add support for decoding UNPCKHPS/UNPCKHPD for AVX 128-bit and 256-bit forms.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145055 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-22 01:57:35 +00:00
Craig Topper
c0d82857e0
Add methods for querying minimum SSE version along with AVX. Simplifies all the places that had to check a version of SSE and AVX.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145053 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-22 00:44:41 +00:00
Craig Topper
6fa583d787
Lowering for v32i8 to VPUNPCKLBW/VPUNPCKHBW when AVX2 is enabled.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145028 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-21 08:26:50 +00:00
Craig Topper
6347e8662c
Add support for lowering 256-bit shuffles to VPUNPCKL/H for i16, i32, i64 if AVX2 is enabled.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145026 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-21 06:57:39 +00:00
Craig Topper
a124f94952
Make LowerSIGN_EXTEND_INREG split 256-bit vectors when AVX1 is enabled and use AVX2 shifts when AVX2 is enabled.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145022 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-21 01:12:36 +00:00
Craig Topper
0d86d462f8
Add code for lowering v32i8 shifts by a splat to AVX2 immediate shift instructions. Remove 256-bit splat handling from LowerShift as it was already handled by PerformShiftCombine.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145005 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-20 00:12:05 +00:00
Craig Topper
745a86bac9
Use 256-bit vcmpeqd for creating an all ones vector when AVX2 is enabled.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145004 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-19 22:34:59 +00:00
Craig Topper
98fc72940b
Custom lower AVX2 variable shift intrinsics to shl/srl/sra nodes and remove the intrinsic patterns.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@144999 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-19 17:46:46 +00:00
Craig Topper
54f952afac
Synthesize SSSE3/AVX 128-bit horizontal integer add/sub instructions from add/sub of appropriate shuffle vectors.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@144989 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-19 09:02:40 +00:00