Craig Topper
b7ab7fe053
Give priority to AVX over SSE for 128-bit floating point unpck instructions.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148233 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-16 09:56:42 +00:00
Craig Topper
d07ef50ca1
Fix the memop type on a couple 256-bit AVX instructions that were using f128mem instead of f256mem.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148196 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-14 18:29:57 +00:00
Chad Rosier
d32d3b758f
Fix pasto from r146196.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148167 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-14 01:50:21 +00:00
Craig Topper
0518970dc8
Convert SHUFPD with the same register for both sources to PSHUFD if it would prevent a register copy. Similar to SHUFPS, but requires the mask to be converted.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148112 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-13 09:21:41 +00:00
Craig Topper
12216172c0
Make X86 instruction selection use 256-bit VPXOR for build_vector of all ones if AVX2 is enabled. This gives the ExeDepsFix pass a chance to choose FP vs int as appropriate. Also use v8i32 as the type for getZeroVector if AVX2 is enabled. This is consistent with SSE2 using prefering v4i32.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148108 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-13 08:12:35 +00:00
Craig Topper
c30432ab57
Add patterns for v16i16 and v32i8 immAllZerosV to select VPXOR to match v4i64 and v8i32.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148106 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-13 06:59:47 +00:00
Chad Rosier
1b2983bb23
Add missing VEX predicates to VMOVSDto64rr/VMOVSDto64mr. This fixes a few
...
failing test cases on our internal AVX nightly tester.
rdar://10663637
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147881 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-10 22:14:06 +00:00
Craig Topper
c6d59954d8
Instruction selection priority fixes to remove the XMM/XMMInt/orAVX predicates. Another commit will remove orAVX functions from X86SubTarget.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147841 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-10 06:30:56 +00:00
Craig Topper
8ffc964582
Add HasAVX predicate to some of the AVX patterns.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147769 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-09 08:34:00 +00:00
Craig Topper
47cf1003fa
Reorder a bunch of patterns to put the AVX version first thus giving it priority over the SSE version. Another step towards trying to remove the AVX hack that disables SSE from X86Subtarget.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147768 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-09 08:10:38 +00:00
Craig Topper
5feb5dae93
Clean up patterns for MOVNT*. Not sure why there were floating point types on MOVNTPS and MOVNTDQ. And v4i64 was completely missing.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147767 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-09 06:52:46 +00:00
Craig Topper
8974cd85cc
Mark MOVNTI as being supported in SSE2 OR AVX mode. This instruction has no AVX equivalent so we should use the SSE version.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147766 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-09 06:38:55 +00:00
Craig Topper
dfa5f573e7
Move SSE2 logical operations PAND/POR/PXOR/PANDN above SSE1 logical operations ANDPS/ORPS/XORPS/ANDNPS. This fixes a pattern ordering issue that meant that the SSE2 instructions could never be directly selected since the SSE1 patterns would always match first. This is largely moot with the ExeDepsFix pass, but I'm trying to audit for all such ordering issues.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147765 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-09 05:07:01 +00:00
Chad Rosier
3d1161e9ae
Enhance DAGCombine for transforming 128->256 casts into a vmovaps, rather
...
then a vxorps + vinsertf128 pair if the original vector came from a load.
rdar://10594409
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147481 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-03 21:05:52 +00:00
Craig Topper
a51bb3aa75
Make CanXFormVExtractWithShuffleIntoLoad reject loads with multiple uses. Also make it return false if there's not even a load at all. This makes the code better match the code in DAGCombiner that it tries to match. These two changes prevent some cases where vector_shuffles were making it to instruction selection and causing the older shuffle selection code to be triggered. Also needed to fix a bad pattern that this change exposed. This is the first step towards getting rid of the old shuffle selection support. No test cases yet because there's no way to tell whether a shuffle was handled in the legalize stage or at instruction selection.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147428 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-02 08:46:48 +00:00
Craig Topper
de9e4c728e
Fix sfence, lfence, mfence, and clflush to be able to be selected when AVX is enabled. Fix monitor and mwait to require SSE3 or AVX, previously they worked even if SSE3 was disabled. Make prefetch instructions not set the execution domain since they don't use XMM registers.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147409 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-01 19:40:22 +00:00
Craig Topper
b3982da7d2
Merge X86 SHUFPS and SHUFPD node types.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147394 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-31 23:50:21 +00:00
Craig Topper
3ee6d22c78
Add patterns for integer forms of SHUFPD/VSHUFPD with a memory load.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147393 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-31 23:24:49 +00:00
Craig Topper
e00805d52f
Fix typo in a SHUFPD and VSHUFPD pattern that prevented SHUFPD/VSHUFPD with a load from being selected.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147392 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-31 23:15:11 +00:00
Craig Topper
d65c7da5b0
Remove the separate explicit AES instruction patterns. They are equivalent to the patterns specified by the instructions. Also remove unnecessary bitconverts from the AES patterns.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147342 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-29 17:41:56 +00:00
Chad Rosier
5c0d761d63
Fix 80-column violations.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147095 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-21 20:59:09 +00:00
Elena Demikhovsky
ba4f83b4e9
This is the second fix related to VZEXT_MOVL node.
...
The failure that I see in the current version is:
LLVM ERROR: Cannot select: 0x18b8f70: v4i64 = X86ISD::VZEXT_MOVL 0x18beee0 [ID=14]
0x18beee0: v4i64 = insert_subvector 0x18b8c70, 0x18b9170, 0x18b9570 [ID=13]
0x18b8c70: v4i64 = insert_subvector 0x18b9870, 0x18bf4e0, 0x18b9970 [ID=12]
0x18b9870: v4i64 = undef [ID=4]
0x18bf4e0: v2i64 = bitcast 0x18bf3e0 [ID=10]
0x18bf3e0: v4i32 = BUILD_VECTOR 0x18b9770, 0x18b9770, 0x18b9770, 0x18b9770 [ID=8]
0x18b9770: i32 = TargetConstant<0> [ID=6]
0x18b9770: i32 = TargetConstant<0> [ID=6]
0x18b9770: i32 = TargetConstant<0> [ID=6]
0x18b9770: i32 = TargetConstant<0> [ID=6]
0x18b9970: i32 = Constant<0> [ID=3]
0x18b9170: v2i64 = undef [ORD=1] [ID=1]
0x18b9570: i32 = Constant<2> [ID=5]
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146975 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-20 13:34:28 +00:00
Eli Friedman
7e840efc23
Make sure we correctly note the existence of an i8 immediate for vblendvps and friends, so we compute fixups correctly. PR11586.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146709 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-15 23:46:18 +00:00
Chad Rosier
c8dd20170e
Add missing zmovl AVX patterns which were causing crashes.
...
Patch by Elena Demikhovsky <elena.demikhovsky@intel.com>!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146689 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-15 22:11:31 +00:00
Benjamin Kramer
b653397dcd
X86: Add patterns for the various rounding ops for SSE4.1 and AVX.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146257 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-09 15:44:03 +00:00
Benjamin Kramer
a73fb9adbb
X86: Split (v)rounds[sd] into a normal and an intrinsic version.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146256 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-09 15:43:55 +00:00
Evan Cheng
e955726a0e
Add 256-bit variant vmovss and vmovsd patterns. rdar://10538417
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146196 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-08 22:30:45 +00:00
Evan Cheng
13d2ba34f2
Add various missing AVX patterns which was causing crashes. Sadly, the generated
...
code looks pretty bad compared to SSE.
rdar://10538793
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146191 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-08 22:05:28 +00:00
Evan Cheng
2f435511e9
Many of the SSE patterns should not be selected when AVX is available. This led to the following code in X86Subtarget.cpp
...
if (HasAVX)
X86SSELevel = NoMMXSSE;
This is so patterns that are predicated on hasSSE3, etc. would not be selected when avx is available. Instead, the AVX variant is selected.
However, this breaks instructions which do not have AVX variants.
The right way to fix this is for the SSE but not-AVX patterns to predicate on something like hasSSE3() && !hasAVX().
Then we can take out the hack in X86Subtarget.cpp. Patterns which do not have AVX variants do not need to change.
However, we need to audit all the patterns before we make the change. This patch is workaround that fixes one specific case,
the prefetch instructions. rdar://10538297
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146163 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-08 19:00:42 +00:00
Craig Topper
d802326335
Fix a bunch of SSE/AVX patterns to use proper memop types. In particular, not using integer loads other than v2i64/v4i64 since the others are all promoted.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146031 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-07 08:30:53 +00:00
Craig Topper
cb6bd11bd6
Fix a bunch of SSE/AVX patterns to use v2i64/v4i64 loads since all other integer vector loads are promoted to those.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145927 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-06 09:04:59 +00:00
Craig Topper
34671b812a
Merge floating point and integer UNPCK X86ISD node types.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145926 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-06 08:21:25 +00:00
Craig Topper
ec24e61ab0
Merge VPERM2F128/VPERM2I128 ISD node types.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145485 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-30 07:47:51 +00:00
Craig Topper
316cd2a2c5
Merge decoding of VPERMILPD and VPERMILPS shuffle masks. Merge X86ISD node type for VPERMILPD/PS. Add instruction selection support for VINSERTI128/VEXTRACTI128.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145483 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-30 06:25:25 +00:00
Evan Cheng
a3438cf48b
Add another missing pattern. llvm-gcc likes f64 but clang likes i64 so it was generating poor code for some SSE builtins.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145448 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-29 22:48:34 +00:00
Jakob Stoklund Olesen
0edd83bfff
Make X86::FsFLD0SS / FsFLD0SD real pseudo-instructions.
...
Like V_SET0, these instructions are expanded by ExpandPostRA to xorps /
vxorps so they can participate in execution domain swizzling.
This also makes the AVX variants redundant.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145440 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-29 22:27:25 +00:00
Elena Demikhovsky
f68b214e2d
Fixed vsqrt.ss intrinsic usage - order of input operands was wrong.
...
Added a test.
Thanks Bruno for reviewing the patch.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145403 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-29 15:00:45 +00:00
Craig Topper
36e36ace77
Fix issues in shuffle decoding around VPERM* instructions. Fix shuffle decoding for VSHUFPS/D for 256-bit types. Add pattern matching for memory forms of VPERMILPS/VPERMILPD.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145390 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-29 07:49:05 +00:00
Craig Topper
fe2a6c584a
Fix VINSERTF128/VEXTRACTF128 to be marked as FP instructions. Allow execution dependency fix pass to convert them to their integer equivalents when AVX2 is enabled.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145376 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-29 05:37:58 +00:00
Craig Topper
108126cfc6
Correctly mark VPERM2F128 as being an FP instruction and add execution domain fixing support to convert it to VPERM2I128 for AVX2.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145370 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-29 03:57:34 +00:00
Evan Cheng
678cda052c
Add missing avx pattern.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145272 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-28 20:27:23 +00:00
Craig Topper
70b883b3a7
Add X86 instruction selection for VPERM2I128 when AVX2 is enabled. Merge VPERMILPS/VPERMILPD detection since they are pretty similar.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145238 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-28 10:14:51 +00:00
Craig Topper
38034c568c
Merge 128-bit and 256-bit X86ISD node types for VPERMILPS and VPERMILPD. Simplify some shuffle lowering code since V1 can never be UNDEF due to canonalizing that occurs when shuffle nodes are created.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145153 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-26 22:55:48 +00:00
Craig Topper
06cb680779
Collapse X86ISD node types for PUNPCKH*, PUNPCKL*, UNPCKLP*, and UNPCKHP* to not be type specific. Now we just have integer high and low and floating point high and low. Pattern matching will choose the correct instruction based on the vector type.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145148 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-26 20:47:44 +00:00
Craig Topper
705f2431a0
Remove 256-bit specific node types for UNPCKHPS/D and instead use the 128-bit versions and let the operand type disinquish. Also fix the load form of the v8i32 patterns for these to realize that the load would be promoted to v4i64.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145126 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-24 22:57:10 +00:00
Craig Topper
f475a55bd4
Remove AVX2 specific X86ISD node types for PUNPCKH/L and instead just reuse the 128-bit versions and let the vector type distinguish.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145125 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-24 22:20:08 +00:00
Craig Topper
6fa583d787
Lowering for v32i8 to VPUNPCKLBW/VPUNPCKHBW when AVX2 is enabled.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145028 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-21 08:26:50 +00:00
Craig Topper
6347e8662c
Add support for lowering 256-bit shuffles to VPUNPCKL/H for i16, i32, i64 if AVX2 is enabled.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145026 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-21 06:57:39 +00:00
Craig Topper
0d86d462f8
Add code for lowering v32i8 shifts by a splat to AVX2 immediate shift instructions. Remove 256-bit splat handling from LowerShift as it was already handled by PerformShiftCombine.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145005 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-20 00:12:05 +00:00
Craig Topper
745a86bac9
Use 256-bit vcmpeqd for creating an all ones vector when AVX2 is enabled.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145004 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-19 22:34:59 +00:00