Commit Graph

1715 Commits

Author SHA1 Message Date
Eli Friedman
7d3e2b78c7 PR11033: Make sure we don't generate PCMPGTQ and PCMPEQQ if the target CPU does not support them.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@140723 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-28 21:00:25 +00:00
Duncan Sands
04aa4aee89 Implement Chris's suggestion of legalizing the various SSE and AVX
hadd/hsub intrinsics into the new fhadd/fhsub X86 node.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@140383 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-23 16:10:22 +00:00
Duncan Sands
17470bee5f Synthesize SSE3/AVX 128 bit horizontal add/sub instructions from
floating point add/sub of appropriate shuffle vectors.  Does not
synthesize the 256 bit AVX versions because they work differently.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@140332 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-22 20:15:48 +00:00
Benjamin Kramer
2c2ccbf108 The SSE version differences for fmin/fmax are more involved than I thought.
- x87: no min or max.
- SSE1: min/max for single precision scalars and vectors.
- SSE2: min/max for single and double precision scalars and vectors.
- AVX: as SSE2, but also supports the wider ymm vectors. (this is covered by the isTypeLegal check)

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@140296 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-22 03:27:22 +00:00
Benjamin Kramer
74f3501d15 X86: Don't form min/max nodes if the target is missing SSE.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@140294 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-22 03:01:42 +00:00
Nadav Rotem
64ac73bb15 fix comment
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@140258 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-21 17:14:40 +00:00
Nadav Rotem
9c6cdf4c1c Insert a sanity check on the combining of x86 truncing-store nodes. This comes to replace the problematic check that was removed in r139995.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@140246 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-21 08:45:10 +00:00
Richard Trieu
23946fcaae Change:
assert(!"error message");

To:

  assert(0 && "error message");

which is more consistant across the code base.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@140234 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-21 03:09:09 +00:00
Bruno Cardoso Lopes
149f29f1fd Simplify max/minp[s|d] dagcombine matching
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@140199 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-20 22:34:45 +00:00
Craig Topper
3699261d3f Extend changes from r139986 to produce 256-bit AVX minps/minpd/maxps/maxpd.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@140140 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-20 07:38:59 +00:00
Nadav Rotem
ca6f296b48 Fix typos in my prev commit, found by Tobi.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@140003 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-18 19:00:23 +00:00
Nadav Rotem
354efd88db setOperationAction should be done on the return value of the type, not the operands.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@140001 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-18 14:57:03 +00:00
Nadav Rotem
91e43fd17a When promoting integer vectors we often create ext-loads. This patch adds a
dag-combine optimization to implement the ext-load efficiently (using shuffles).

For example the type <4 x i8> is stored in memory as i32, but it needs to
find its way into a <4 x i32> register. Previously we scalarized the memory
access, now we use shuffles.




git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139995 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-18 10:39:32 +00:00
Craig Topper
89af15ee11 Fix typo by changing Lower256IntVETCC to Lower256IntVSETCC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139993 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-18 08:03:58 +00:00
Duncan Sands
6bcd2196e5 Synthesize x86 max/min instructions also for vectors (i.e. produce
maxps and maxpd).  This broke the sse41-blend.ll testcase by causing
maxpd to be produced rather than a cmp+blend pair, which is the reason
I tweaked it.  Gives a small speedup on doduc with dragonegg when the
GCC vectorizer is used.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139986 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-17 16:49:39 +00:00
Bruno Cardoso Lopes
0c4b9ff077 Change all checks regarding the presence of any SSE level to always
take into consideration the presence of AVX. This change, together with
the SSEDomainFix enabled for AVX, makes AVX codegen to always (hopefully)
emit the same code as SSE for 128-bit vector ops. I don't
have a testcase for this, but AVX now beats SSE in performance for
128-bit ops in the majority of programas in the llvm testsuite

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139817 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-15 18:27:36 +00:00
Eli Friedman
322ea080ad Fix the code creating VZEXT_LOAD so that it creates the right memoperand. Issue spotted in -debug output. I can't think of any practical effects at the moment, but it might matter if we start doing more aggressive alias analysis in CodeGen.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139758 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-14 23:42:45 +00:00
Bruno Cardoso Lopes
5ca0d14915 Vector shuffle mask <i32 4, i32 5, i32 2, i32 3> should yield "movsd", not "movss".
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139686 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-14 02:36:14 +00:00
Bruno Cardoso Lopes
809f17fbb1 Revert the remaining part of r139528. According to PR10907 the bug seems
to be in the VSELECT operands order, so I'll leave the fix for Nadav.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139624 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-13 19:33:00 +00:00
Nadav Rotem
aec5861bb6 Add vselect target support for targets that do not support blend but do support
xor/and/or (For example SSE2).



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139623 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-13 19:17:42 +00:00
Bruno Cardoso Lopes
457d53d9ce Revert the wrong part of r139528, and fix testcases.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139541 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-12 21:24:07 +00:00
Bruno Cardoso Lopes
8e03a821f9 Not sure how CMPPS and CMPPD had already ever worked, I guess it didn't.
However with this fix it does now.

Basically the operand order for the x86 target specific node
is not the same as the instruction, but since the intrinsic need that
specific order at the instruction definition, just change the order
during legalization. Also, there were some wrong invertions of condition
codes, such as GE => LE, GT => LT, fix that too. Fix PR10907.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139528 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-12 19:30:40 +00:00
Nadav Rotem
fbad25e120 CR fixes per Bruno's request.
Undo the changes from r139285 which added custom lowering to vselect.
Add tablegen lowering for vselect.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139479 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-11 15:02:23 +00:00
Eli Friedman
106f6e7a27 r139454 activates an assert in a case where we were doing the right thing anyway. Make that explicit, and un-XFAIL the testcase.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139458 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-10 02:01:42 +00:00
Richard Trieu
2db8628085 Fixed an assert from:
assert("not implemented for target shuffle node");

to:

  assert(0 && "not implemented for target shuffle node");

This causes a test failure in CodeGen/X86/palignr.ll which has
been marked as XFAIL for the time being.
Test failure filed at PR10901.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139454 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-10 01:26:21 +00:00
Nadav Rotem
8ffad56f8e Implement vector-select support for avx256. Refactor the vblend implementation to have tablegen match the instruction by the node type
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139400 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-09 20:29:17 +00:00
Nadav Rotem
ee64be9c17 Dix the 80-columns and remove unsupported v8i16 type from the list of legal vselect types.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139324 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-08 22:17:35 +00:00
Bruno Cardoso Lopes
814c6ced85 Add AVX versions of blend vector operations and fix some issues noticed
in Nadav's r139285 and r139287 commits.

1) Rename vsel.ll to a more descriptive name
2) Change the order of BLEND operands to "Op1, Op2, Cond", this is
necessary because PBLENDVB is already used in different places with
this order, and it was being emitted in the wrong way for vselect
3) Add AVX patterns and tests for the same SSE41 instructions

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139305 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-08 18:05:08 +00:00
Nadav Rotem
ffe3e7da84 Add X86-SSE4 codegen support for vector-select.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139285 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-08 08:11:19 +00:00
Duncan Sands
28b77e968d Add codegen support for vector select (in the IR this means a select
with a vector condition); such selects become VSELECT codegen nodes.
This patch also removes VSETCC codegen nodes, unifying them with SETCC
nodes (codegen was actually often using SETCC for vector SETCC already).
This ensures that various DAG combiner optimizations kick in for vector
comparisons.  Passes dragonegg bootstrap with no testsuite regressions
(nightly testsuite as well as "make check-all").  Patch mostly by
Nadav Rotem.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139159 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-06 19:07:46 +00:00
Rafael Espindola
96428cea3d Fix style issues and typos found by Duncan.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139154 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-06 18:43:08 +00:00
Duncan Sands
4a544a79bd Split the init.trampoline intrinsic, which currently combines GCC's
init.trampoline and adjust.trampoline intrinsics, into two intrinsics
like in GCC.  While having one combined intrinsic is tempting, it is
not natural because typically the trampoline initialization needs to
be done in one function, and the result of adjust trampoline is needed
in a different (nested) function.  To get around this llvm-gcc hacks the
nested function lowering code to insert an additional parent variable
holding the adjust.trampoline result that can be accessed from the child
function.  Dragonegg doesn't have the luxury of tweaking GCC code, so it
stored the result of adjust.trampoline in the memory GCC set aside for
the trampoline itself (this is always available in the child function),
and set up some new memory (using an alloca) to hold the trampoline.
Unfortunately this breaks Go which allocates trampoline memory on the
heap and wants to use it even after the parent has exited (!).  Rather
than doing even more hacks to get Go working, it seemed best to just use
two intrinsics like in GCC.  Patch mostly by Sanjoy Das.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139140 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-06 13:37:06 +00:00
Jakob Stoklund Olesen
4a1b9d82a4 Use existing function.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139055 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-02 23:52:49 +00:00
Jakob Stoklund Olesen
439f71eb30 Remove unused variables.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139047 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-02 22:41:25 +00:00
Bruno Cardoso Lopes
a39ccdb9d4 Fix vbroadcast matching logic to early unmatch if the node doesn't have
only one use. Fix PR10825.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@138951 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-01 18:15:06 +00:00
Eric Christopher
c967ad8c88 Rework this conditional a bit.
Patch by Sanjoy Das


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@138853 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-31 04:17:21 +00:00
Bruno Cardoso Lopes
57d6a5e491 - Move all MOVSS and MOVSD patterns close to their definitions
- Duplicate some store patterns to their AVX forms!
- Catched a bug while restricting the patterns subtarget, fix it
  and update a testcase to check it properly

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@138851 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-31 03:04:20 +00:00
Bruno Cardoso Lopes
5affa5196f Teach more places to use VMOVAPS,VMOVUPS instead of MOVAPS,MOVUPS,
whenever AVX is enabled.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@138849 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-31 03:04:09 +00:00
Evan Cheng
0899f5c62d Fix (movhps load) lowering / pattern to match more cases. rdar://10050549
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@138848 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-31 02:05:24 +00:00
Rafael Espindola
151ab3e2f7 Adds support for variable sized allocas. For a variable sized alloca,
code is inserted to first check if the current stacklet has enough
space. If so, space is allocated by simply decrementing the stack
pointer. Otherwise a runtime routine (__morestack_allocate_stack_space
in libgcc) is called which allocates the required memory from the
heap.

Patch by Sanjoy Das.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@138818 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-30 19:47:04 +00:00
Rafael Espindola
d07b7ec772 Adds a SelectionDAG node X86SegAlloca which will be custom lowered
from DYNAMIC_STACKALLOC.

Two new pseudo instructions (SEG_ALLOCA_32 and SEG_ALLOCA_64) which
will match X86SegAlloca (based on word size) are also added.  They
will be custom emitted to inject the actual stack handling code.

Patch by Sanjoy Das.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@138814 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-30 19:43:21 +00:00
Rafael Espindola
76927d7586 Emit segmented-stack specific code into function prologues for
X86. Modify the pass added in the previous patch to call this new
code.

This new prologues generated will call a libgcc routine (__morestack)
to allocate more stack space from the heap when required

Patch by Sanjoy Das.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@138812 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-30 19:39:58 +00:00
Eli Friedman
f3704769bb Explicitly zero out parts of a vector which are required to be zero by the algorithm in LowerUINT_TO_FP_i32. This only has a substantial effect on the generated code when the input is extracted from a vector register; other ways of loading an i32 do the appropriate zeroing implicitly. Fixes PR10802.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@138768 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-29 21:15:46 +00:00
Benjamin Kramer
2753ae314f Silence GCC warnings and make an array const.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@138706 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-27 17:36:14 +00:00
Eli Friedman
43f51aeca8 Add support for generating CMPXCHG16B on x86-64 for the cmpxchg IR instruction.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@138660 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-26 21:21:21 +00:00
Bruno Cardoso Lopes
6292eceea0 Add support for AVX 256-bit version of MOVDDUP!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@138588 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-25 21:40:37 +00:00
Bruno Cardoso Lopes
06ef923d14 Make isMOVDDUP mask check more strict and update comments!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@138587 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-25 21:40:34 +00:00
Bruno Cardoso Lopes
07b7f672a0 Add support for 256-bit versions of VSHUFPD and VSHUFPS.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@138546 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-25 02:58:26 +00:00
Bruno Cardoso Lopes
af002d8405 Move all SHUFP* patterns close to the SHUFP* definitions. Also be
explicit about which subtarget they refer to, and add AVX versions of
the ones we currently don't. Make the mask check more strict, to be
clear it won't be used to match to 256-bit versions!

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@138514 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-24 23:17:55 +00:00
Eli Friedman
f8f90f0174 Hook up 64-bit atomic load/store on x86-32. I plan to write more efficient implementations eventually.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@138505 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-24 22:33:28 +00:00