2597 Commits

Author SHA1 Message Date
Juergen Ributzka
1941431f8a SelectionDAG: Teach the legalizer to split SETCC if VSELECT needs splitting too.
The Type Legalizer recognizes that VSELECT needs to be split, because the type
is to wide for the given target. The same does not always apply to SETCC,
because less space is required to encode the result of a comparison. As a result
VSELECT is split and SETCC is unrolled into scalar comparisons.

This commit fixes the issue by checking for VSELECT-SETCC patterns in the DAG
Combiner. If a matching pattern is found, then the result mask of SETCC is
promoted to the expected vector mask for the given target. This mask has usually
te same size as the VSELECT return type (except for Intel KNL). Now the type
legalizer will split both VSELECT and SETCC.

This allows the following X86 DAG Combine code to sucessfully detect the MIN/MAX
pattern. This fixes PR16695, PR17002, and <rdar://problem/14594431>.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191130 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-21 04:55:18 +00:00
Elena Demikhovsky
f9d2d2dc89 AVX-512: implemented extractelement with variable index.
Added parsing of mask register and "zeroing" semantic, like {%k1} {z}.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190595 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-12 08:55:00 +00:00
Juergen Ributzka
d7174719a9 [X86] Perform VSELECT DAG combines also before DAG type legalization.
If the DAG already has only legal types, then the second round of DAG combines
is skipped. In this case VSELECT+SETCC patterns that match a more efficient
instruction (e.g. min/max) are never recognized.

This fix allows VSELECT+SETCC combines if the types are already legal before DAG
type legalization.

Reviewer: Nadav

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190105 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-05 23:02:56 +00:00
Craig Topper
69c474ffa8 Create BEXTR instructions for (and ((sra or srl) x, imm), (2**size - 1)). Fixes PR17028.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189742 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-02 07:53:17 +00:00
Elena Demikhovsky
6adcd58d3c AVX-512: Added GATHER and SCATTER instructions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189729 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-01 14:24:41 +00:00
Craig Topper
a9080653c2 Fixup BZHI selection to remove an unneeded zero extension.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189656 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-30 07:16:16 +00:00
Craig Topper
b6ac30a155 Teach X86 backend to create BMI2 BZHI instructions from (and X, (add (shl 1, Y), -1)). Fixes PR17038.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189653 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-30 06:52:21 +00:00
Elena Demikhovsky
4edfa2278a AVX-512: added extend and truncate instructions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189580 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-29 11:56:53 +00:00
Elena Demikhovsky
1567abe74f AVX-512: Added FMA instructions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189326 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-27 08:39:25 +00:00
Elena Demikhovsky
92bfb54770 AVX-512: Added shuffle instructions -
VPSHUFD, VPERMILPS, VMOVDDUP, VMOVLHPS, VMOVHLPS, VSHUFPS, VALIGN
 single and double forms.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189215 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-26 12:45:35 +00:00
Elena Demikhovsky
41f7baf181 AVX-512: added UNPACK instructions and tests for all-zero/all-ones vectors
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189189 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-25 12:54:30 +00:00
Elena Demikhovsky
8ba76daba0 AVX-512: Added SHIFT instructions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188899 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-21 09:36:02 +00:00
Elena Demikhovsky
f12df0ad50 AVX-512: added arithmetic and logical operations.
ADD, SUB, MUL integer and FP types. OR, AND, XOR.
Added embeded broadcast form for these instructions.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188673 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-19 13:26:14 +00:00
Elena Demikhovsky
3491d67d3a AVX-512: Added VMOVD, VMOVQ, VMOVSS, VMOVSD instructions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188637 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-18 13:08:57 +00:00
Craig Topper
ff79bc6e18 Make more of the lowering helpers static. Also use MVT instead of EVT in a couple places.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188629 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-18 08:53:01 +00:00
Craig Topper
d36e1efa4b Revert r188449 as it turns out we're just missing the instructions that need the v16i32/v16f32 matching.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188454 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-15 08:38:25 +00:00
Craig Topper
46ceaf4ba6 Don't let isPermImmMask handle v16i32 since VPERMI doesn't match on that type. Remove 128-bit vector handling from isPermImmMask too, it's covered by isPSHUFDMask.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188449 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-15 07:30:51 +00:00
Craig Topper
8d725b9a03 Use MVT in place of EVT in more X86 operation lowering functions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188445 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-15 05:33:45 +00:00
Craig Topper
5a0910b349 Replace getValueType().getSimpleVT() with getSimpleValueType(). Also remove one weird cast from MVT->EVT just to call getSimpleVT().
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188441 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-15 02:33:50 +00:00
Craig Topper
35e194fbad Make more helper methods into static functions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188366 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-14 07:53:41 +00:00
Craig Topper
8971717313 Remove tab characters.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188365 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-14 07:35:18 +00:00
Craig Topper
158ec07008 Make some helper methods static.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188364 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-14 07:34:43 +00:00
Craig Topper
f3d98a882e Use MVT in more lowering code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188363 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-14 07:04:42 +00:00
Craig Topper
d36b53e764 Replace EVT with MVT in isVectorShift. Keeps compiler from generating unneeded checks and handling for extended types.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188362 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-14 06:21:10 +00:00
Craig Topper
cc60bbc739 Replace EVT with MVT in many of the shuffle lowering functions. Keeps compiler from generating unneeded checks and handling for extended types.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188361 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-14 05:58:39 +00:00
Evgeniy Stepanov
4c857225ec Fix compiler warnings.
../lib/Target/X86/X86ISelLowering.cpp:9715:7: error: unused variable 'OpVT' [-Werror,-Wunused-variable]
  EVT OpVT = Op0.getValueType();
      ^
../lib/Target/X86/X86ISelLowering.cpp:9763:14: error: unused variable 'NumElems' [-Werror,-Wunused-variable]
    unsigned NumElems = VT.getVectorNumElements();



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188269 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-13 14:04:20 +00:00
Elena Demikhovsky
4d36bd80e6 AVX-512: Added CMP and BLEND instructions.
Lowering for SETCC.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188265 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-13 13:24:07 +00:00
Elena Demikhovsky
55db69c97e AVX-512: Added more tests for BROADCAST
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188148 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-11 12:29:16 +00:00
Elena Demikhovsky
fac4a4eb7d AVX-512: Added VPERM* instructons and MOV* zmm-to-zmm instructions.
Added a test for shuffles using VPERM.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188147 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-11 07:55:09 +00:00
Elena Demikhovsky
207600d2cf AVX-512 set: Added BROADCAST instructions
with lowering logic and a test.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187884 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-07 12:34:55 +00:00
Craig Topper
b19982c42b Simplify code. No functional change intended.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187870 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-07 08:16:07 +00:00
Tim Northover
d113448c1d Refactor isInTailCallPosition handling
This change came about primarily because of two issues in the existing code.
Niether of:

define i64 @test1(i64 %val) {
  %in = trunc i64 %val to i32
  tail call i32 @ret32(i32 returned %in)
  ret i64 %val
}

define i64 @test2(i64 %val) {
  tail call i32 @ret32(i32 returned undef)
  ret i32 42
}

should be tail calls, and the function sameNoopInput is responsible. The main
problem is that it is completely symmetric in the "tail call" and "ret" value,
but in reality different things are allowed on each side.

For these cases:
1. Any truncation should lead to a larger value being generated by "tail call"
   than needed by "ret".
2. Undef should only be allowed as a source for ret, not as a result of the
   call.

Along the way I noticed that a mismatch between what this function treats as a
valid truncation and what the backends see can lead to invalid calls as well
(see x86-32 test case).

This patch refactors the code so that instead of being based primarily on
values which it recurses into when necessary, it starts by inspecting the type
and considers each fundamental slot that the backend will see in turn. For
example, given a pathological function that returned {{}, {{}, i32, {}}, i32}
we would consider each "real" i32 in turn, and ask if it passes through
unchanged. This is much closer to what the backend sees as a result of
ComputeValueVTs.

Aside from the bug fixes, this eliminates the recursion that's going on and, I
believe, makes the bulk of the code significantly easier to understand. The
trade-off is the nasty iterators needed to find the real types inside a
returned value.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187787 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-06 09:12:35 +00:00
Craig Topper
5923547e2a Simplify vector lane handling math a bit. No functional change intended.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187783 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-06 07:23:12 +00:00
Craig Topper
57bc5a0b5b Simplify math a little bit.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187781 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-06 06:54:25 +00:00
Craig Topper
aa0f420567 Replace EVT with MVT in isHorizontalBinOp as it is only called with legal types.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187779 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-06 06:05:05 +00:00
Craig Topper
20b46b0733 Simplify code slightly. No functional change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187771 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-06 04:12:40 +00:00
Aaron Ballman
2a37c7e9e6 Silencing an MSVC11 type conversion warning.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187727 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-05 13:47:03 +00:00
Elena Demikhovsky
13e6e9171f AVX-512 set: added mask operations, lowering BUILD_VECTOR for i1 vector types.
Added intrinsics and tests.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187717 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-05 08:52:21 +00:00
Benjamin Kramer
75311b7b4d X86: Turn fp selects into mask operations.
double test(double a, double b, double c, double d) { return a<b ? c : d; }

before:
_test:
	ucomisd	%xmm0, %xmm1
	ja	LBB0_2
	movaps	%xmm3, %xmm2
LBB0_2:
	movaps	%xmm2, %xmm0

after:
_test:
	cmpltsd	%xmm1, %xmm0
	andpd	%xmm0, %xmm2
	andnpd	%xmm3, %xmm0
	orpd	%xmm2, %xmm0

Small speedup on Benchmarks/SmallPT

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187706 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-04 12:05:16 +00:00
Tim Northover
a54b66277e X86: correct tail return address calculation
Due to the weird and wondeful usual arithmetic conversions, some
calculations involving negative values were getting performed in
uint32_t and then promoted to int64_t, which is really not a good
idea.

Patch by Katsuhiro Ueno.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187703 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-04 09:35:57 +00:00
Elena Demikhovsky
0780179d53 EVEX and compressed displacement encoding for AVX512
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187576 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-01 13:34:06 +00:00
Elena Demikhovsky
093043ce11 Fixed assertion in Extract128BitVector()
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187493 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-31 12:03:08 +00:00
Elena Demikhovsky
8395251c0a Added INSERT and EXTRACT intructions from AVX-512 ISA.
All insertf*/extractf* functions replaced with insert/extract since we have insertf and inserti forms.
Added lowering for INSERT_VECTOR_ELT / EXTRACT_VECTOR_ELT for 512-bit vectors.
Added lowering for EXTRACT/INSERT subvector for 512-bit vectors.
Added a test.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187491 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-31 11:35:14 +00:00
Nico Rieck
944061c4e1 Proper va_arg/va_copy lowering on win64
Win64 uses CharPtrBuiltinVaList instead of X86_64ABIBuiltinVaList like
other 64-bit targets.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187355 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-29 13:07:06 +00:00
Justin Holewinski
320185fa5f Add a target legalize hook for SplitVectorOperand (again)
CustomLowerNode was not being called during SplitVectorOperand,
meaning custom legalization could not be used by targets.

This also adds a test case for NVPTX that depends on this custom
legalization.

Differential Revision: http://llvm-reviews.chandlerc.com/D1195

Attempt to fix the buildbots by making the X86 test I just added platform independent

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187202 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-26 13:28:29 +00:00
Rafael Espindola
c735c1c2ae Revert "Add a target legalize hook for SplitVectorOperand"
This reverts commit 187198. It broke the bots.

The soft float test probably needs a -triple because of name differences.
On the hard float test I am getting a "roundss $1, %xmm0, %xmm0", instead of
"vroundss $1, %xmm0, %xmm0, %xmm0".

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187201 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-26 13:18:16 +00:00
Justin Holewinski
5a24ed951b Add a target legalize hook for SplitVectorOperand
CustomLowerNode was not being called during SplitVectorOperand,
meaning custom legalization could not be used by targets.

This also adds a test case for NVPTX that depends on this custom
legalization.

Differential Revision: http://llvm-reviews.chandlerc.com/D1195

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187198 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-26 12:46:39 +00:00
Elena Demikhovsky
e3809eed34 I'm starting to commit KNL backend. I'll push patches one-by-one. This patch includes support for the extended register set XMM16-31, YMM16-31, ZMM0-31.
The full ISA you can see here: http://software.intel.com/en-us/intel-isa-extensions


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187030 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-24 11:02:47 +00:00
Juergen Ributzka
b95e0f6f2f [X86] Use min/max to optimze unsigend vector comparison on X86
Use PMIN/PMAX for UGE/ULE vector comparions to reduce the number of required
instructions. This trick also works for UGT/ULT, but there is no advantage in
doing so. It wouldn't reduce the number of instructions and it would actually
reduce performance.

Reviewer: Ben

radar:5972691



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186432 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-16 18:20:45 +00:00
Craig Topper
da129a2eb3 Add 'static' keyword to some const arrays for consistency.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186308 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-15 06:54:12 +00:00