Craig Topper
cc30006391
Fix pattern for memory form of PSHUFD for use with FP vectors to remove bitcast to an integer vector that normal code wouldn't have. Also remove bitcasts from code that turns splat vector loads into a shuffle as it was making the broken pattern necessary.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149232 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-30 07:50:31 +00:00
Craig Topper
3982b3cc7b
Move some patterns back near their instructions and use AddedComplexity to fix priority. Merge some patterns into their instruction definition.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149122 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-27 07:09:40 +00:00
Victor Umansky
668f7ac9e4
Fix for the following bug in AVX codegen for double-to-int conversions:
...
. "fptosi" and "fptoui" IR instructions are defined with round-to-zero rounding mode.
. Currently for AVX mode for <4xdouble> and <8xdouble> the "VCVTPD2DQ.128" and "VCVTPD2DQ.256" instructions are selected (for .fp_to_sint. DAG node operation ) by AVX codegen. However they use round-to-nearest-even rounding mode.
. Consequently, the conversion produces incorrect numbers.
The fix is to replace selection of VCVTPD2DQ instructions with VCVTTPD2DQ instructions. The latter use truncate (i.e. round-to-zero) rounding mode.
As .fp_to_sint. DAG node operation is used only for lowering of "fptosi" and "fptoui" IR instructions, the fix in X86InstrSSE.td definition file doesn.t have an impact on other LLVM flows.
The patch includes changes in the .td file, LIT test for the changes and a fix in a legacy LIT test (which produced asm code conflicting with LLVN IR spec).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149056 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-26 08:51:39 +00:00
Craig Topper
15388c4666
Fix AVX vs SSE patterns ordering issue for VPCMPESTRM and VPCMPISTRM.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149053 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-26 07:31:30 +00:00
Craig Topper
e566cd0f4d
Remove some more patterns by custom lowering intrinsics to target specific nodes.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149052 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-26 07:18:03 +00:00
Craig Topper
969ba287cd
Custom lower PSIGN and PSHUFB intrinsics to their corresponding target specific nodes so we can remove the isel patterns.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148933 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-25 06:43:11 +00:00
Craig Topper
4bb3f34b22
Custom lower phadd and phsub intrinsics to target specific nodes. Remove the patterns that are no longer necessary.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148927 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-25 05:37:32 +00:00
Craig Topper
bce73e0a8c
Remove AVX 256-bit unaligned load intrinsics. 128-bit versions had been removed a while ago.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148922 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-25 04:42:03 +00:00
Craig Topper
042883f5da
Merge intrinsic pattern and no pattern versions of VCVTSD2SI intruction definitions. Matches non-AVX version of same instructions.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148914 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-25 03:52:09 +00:00
Craig Topper
7925e2555d
Custom lower PCMPEQ/PCMPGT intrinsics to target specific nodes and remove the intrinsic patterns.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148687 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-23 08:18:28 +00:00
Craig Topper
80e46360e9
Custom lower vector shift intrinsics to target specific nodes and remove the patterns that are no longer needed.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148684 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-23 06:16:53 +00:00
Craig Topper
2b21fbaf11
Remove pattern fragments for v32i8, v16i16, v8i32, v16i8, v8i16, and v4i32 loads. All integer vector loads are promoted to v2i64 or v4i64 so these pattern fragments can never match. Fix or remove patterns that used these fragments.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148672 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-23 00:06:44 +00:00
Craig Topper
1906d32e55
Combine X86 CMPPD and CMPPS node types. Simplifies selection code and pattern matching.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148670 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-22 23:36:02 +00:00
Craig Topper
67609fd0eb
Merge PCMPEQB/PCMPEQW/PCMPEQD/PCMPEQQ and PCMPGTB/PCMPGTW/PCMPGTD/PCMPGTQ X86 ISD node types into only two node types. Simplifying opcode selection and pattern matching.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148667 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-22 22:42:16 +00:00
Craig Topper
ed2e13d667
Add target specific ISD node types for SSE/AVX vector shuffle instructions and change all the code that used to create intrinsic nodes to create the new nodes instead.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148664 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-22 19:15:14 +00:00
Craig Topper
6fdf3d54d2
Move some vector shift patterns into their instruction definitions.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148643 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-22 00:41:20 +00:00
Craig Topper
babb1459f3
Add memory patterns for some of the fp<->integer conversion instructions. Fold some patterns into instruction definitions.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148641 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-21 18:37:15 +00:00
Craig Topper
0e2037ba2b
Add support for selecting 256-bit PALIGNR.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148532 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-20 05:53:00 +00:00
Craig Topper
b7ab7fe053
Give priority to AVX over SSE for 128-bit floating point unpck instructions.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148233 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-16 09:56:42 +00:00
Craig Topper
d07ef50ca1
Fix the memop type on a couple 256-bit AVX instructions that were using f128mem instead of f256mem.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148196 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-14 18:29:57 +00:00
Chad Rosier
d32d3b758f
Fix pasto from r146196.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148167 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-14 01:50:21 +00:00
Craig Topper
0518970dc8
Convert SHUFPD with the same register for both sources to PSHUFD if it would prevent a register copy. Similar to SHUFPS, but requires the mask to be converted.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148112 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-13 09:21:41 +00:00
Craig Topper
12216172c0
Make X86 instruction selection use 256-bit VPXOR for build_vector of all ones if AVX2 is enabled. This gives the ExeDepsFix pass a chance to choose FP vs int as appropriate. Also use v8i32 as the type for getZeroVector if AVX2 is enabled. This is consistent with SSE2 using prefering v4i32.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148108 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-13 08:12:35 +00:00
Craig Topper
c30432ab57
Add patterns for v16i16 and v32i8 immAllZerosV to select VPXOR to match v4i64 and v8i32.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148106 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-13 06:59:47 +00:00
Chad Rosier
1b2983bb23
Add missing VEX predicates to VMOVSDto64rr/VMOVSDto64mr. This fixes a few
...
failing test cases on our internal AVX nightly tester.
rdar://10663637
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147881 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-10 22:14:06 +00:00
Craig Topper
c6d59954d8
Instruction selection priority fixes to remove the XMM/XMMInt/orAVX predicates. Another commit will remove orAVX functions from X86SubTarget.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147841 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-10 06:30:56 +00:00
Craig Topper
8ffc964582
Add HasAVX predicate to some of the AVX patterns.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147769 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-09 08:34:00 +00:00
Craig Topper
47cf1003fa
Reorder a bunch of patterns to put the AVX version first thus giving it priority over the SSE version. Another step towards trying to remove the AVX hack that disables SSE from X86Subtarget.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147768 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-09 08:10:38 +00:00
Craig Topper
5feb5dae93
Clean up patterns for MOVNT*. Not sure why there were floating point types on MOVNTPS and MOVNTDQ. And v4i64 was completely missing.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147767 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-09 06:52:46 +00:00
Craig Topper
8974cd85cc
Mark MOVNTI as being supported in SSE2 OR AVX mode. This instruction has no AVX equivalent so we should use the SSE version.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147766 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-09 06:38:55 +00:00
Craig Topper
dfa5f573e7
Move SSE2 logical operations PAND/POR/PXOR/PANDN above SSE1 logical operations ANDPS/ORPS/XORPS/ANDNPS. This fixes a pattern ordering issue that meant that the SSE2 instructions could never be directly selected since the SSE1 patterns would always match first. This is largely moot with the ExeDepsFix pass, but I'm trying to audit for all such ordering issues.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147765 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-09 05:07:01 +00:00
Chad Rosier
3d1161e9ae
Enhance DAGCombine for transforming 128->256 casts into a vmovaps, rather
...
then a vxorps + vinsertf128 pair if the original vector came from a load.
rdar://10594409
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147481 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-03 21:05:52 +00:00
Craig Topper
a51bb3aa75
Make CanXFormVExtractWithShuffleIntoLoad reject loads with multiple uses. Also make it return false if there's not even a load at all. This makes the code better match the code in DAGCombiner that it tries to match. These two changes prevent some cases where vector_shuffles were making it to instruction selection and causing the older shuffle selection code to be triggered. Also needed to fix a bad pattern that this change exposed. This is the first step towards getting rid of the old shuffle selection support. No test cases yet because there's no way to tell whether a shuffle was handled in the legalize stage or at instruction selection.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147428 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-02 08:46:48 +00:00
Craig Topper
de9e4c728e
Fix sfence, lfence, mfence, and clflush to be able to be selected when AVX is enabled. Fix monitor and mwait to require SSE3 or AVX, previously they worked even if SSE3 was disabled. Make prefetch instructions not set the execution domain since they don't use XMM registers.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147409 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-01 19:40:22 +00:00
Craig Topper
b3982da7d2
Merge X86 SHUFPS and SHUFPD node types.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147394 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-31 23:50:21 +00:00
Craig Topper
3ee6d22c78
Add patterns for integer forms of SHUFPD/VSHUFPD with a memory load.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147393 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-31 23:24:49 +00:00
Craig Topper
e00805d52f
Fix typo in a SHUFPD and VSHUFPD pattern that prevented SHUFPD/VSHUFPD with a load from being selected.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147392 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-31 23:15:11 +00:00
Craig Topper
d65c7da5b0
Remove the separate explicit AES instruction patterns. They are equivalent to the patterns specified by the instructions. Also remove unnecessary bitconverts from the AES patterns.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147342 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-29 17:41:56 +00:00
Chad Rosier
5c0d761d63
Fix 80-column violations.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147095 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-21 20:59:09 +00:00
Elena Demikhovsky
ba4f83b4e9
This is the second fix related to VZEXT_MOVL node.
...
The failure that I see in the current version is:
LLVM ERROR: Cannot select: 0x18b8f70: v4i64 = X86ISD::VZEXT_MOVL 0x18beee0 [ID=14]
0x18beee0: v4i64 = insert_subvector 0x18b8c70, 0x18b9170, 0x18b9570 [ID=13]
0x18b8c70: v4i64 = insert_subvector 0x18b9870, 0x18bf4e0, 0x18b9970 [ID=12]
0x18b9870: v4i64 = undef [ID=4]
0x18bf4e0: v2i64 = bitcast 0x18bf3e0 [ID=10]
0x18bf3e0: v4i32 = BUILD_VECTOR 0x18b9770, 0x18b9770, 0x18b9770, 0x18b9770 [ID=8]
0x18b9770: i32 = TargetConstant<0> [ID=6]
0x18b9770: i32 = TargetConstant<0> [ID=6]
0x18b9770: i32 = TargetConstant<0> [ID=6]
0x18b9770: i32 = TargetConstant<0> [ID=6]
0x18b9970: i32 = Constant<0> [ID=3]
0x18b9170: v2i64 = undef [ORD=1] [ID=1]
0x18b9570: i32 = Constant<2> [ID=5]
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146975 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-20 13:34:28 +00:00
Eli Friedman
7e840efc23
Make sure we correctly note the existence of an i8 immediate for vblendvps and friends, so we compute fixups correctly. PR11586.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146709 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-15 23:46:18 +00:00
Chad Rosier
c8dd20170e
Add missing zmovl AVX patterns which were causing crashes.
...
Patch by Elena Demikhovsky <elena.demikhovsky@intel.com>!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146689 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-15 22:11:31 +00:00
Benjamin Kramer
b653397dcd
X86: Add patterns for the various rounding ops for SSE4.1 and AVX.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146257 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-09 15:44:03 +00:00
Benjamin Kramer
a73fb9adbb
X86: Split (v)rounds[sd] into a normal and an intrinsic version.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146256 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-09 15:43:55 +00:00
Evan Cheng
e955726a0e
Add 256-bit variant vmovss and vmovsd patterns. rdar://10538417
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146196 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-08 22:30:45 +00:00
Evan Cheng
13d2ba34f2
Add various missing AVX patterns which was causing crashes. Sadly, the generated
...
code looks pretty bad compared to SSE.
rdar://10538793
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146191 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-08 22:05:28 +00:00
Evan Cheng
2f435511e9
Many of the SSE patterns should not be selected when AVX is available. This led to the following code in X86Subtarget.cpp
...
if (HasAVX)
X86SSELevel = NoMMXSSE;
This is so patterns that are predicated on hasSSE3, etc. would not be selected when avx is available. Instead, the AVX variant is selected.
However, this breaks instructions which do not have AVX variants.
The right way to fix this is for the SSE but not-AVX patterns to predicate on something like hasSSE3() && !hasAVX().
Then we can take out the hack in X86Subtarget.cpp. Patterns which do not have AVX variants do not need to change.
However, we need to audit all the patterns before we make the change. This patch is workaround that fixes one specific case,
the prefetch instructions. rdar://10538297
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146163 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-08 19:00:42 +00:00
Craig Topper
d802326335
Fix a bunch of SSE/AVX patterns to use proper memop types. In particular, not using integer loads other than v2i64/v4i64 since the others are all promoted.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146031 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-07 08:30:53 +00:00
Craig Topper
cb6bd11bd6
Fix a bunch of SSE/AVX patterns to use v2i64/v4i64 loads since all other integer vector loads are promoted to those.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145927 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-06 09:04:59 +00:00
Craig Topper
34671b812a
Merge floating point and integer UNPCK X86ISD node types.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@145926 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-06 08:21:25 +00:00