18 Commits

Author SHA1 Message Date
Adam Nemet
49f31255be [AVX512] Fix miscompile for unpack
r189189 implemented AVX512 unpack by essentially performing a 256-bit unpack
between the low and the high 256 bits of src1 into the low part of the
destination and another unpack of the low and high 256 bits of src2 into the
high part of the destination.

I don't think that's how unpack works.  AVX512 unpack simply has more 128-bit
lanes but other than it works the same way as AVX.  So in each 128-bit lane,
we're always interleaving certain parts of both operands rather different
parts of one of the operands.

E.g. for this:
__v16sf a = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 };
__v16sf b = { 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31 };
__v16sf c = __builtin_shufflevector(a, b, 0, 8, 1, 9, 4, 12, 5, 13, 16,
	    			       	     24, 17, 25, 20, 28, 21, 29);

we generated punpcklps (notice how the elements of a and b are not interleaved
in the shuffle).  In turn, c was set to this:

  0 16 1 17 4 20 5 21 8 24 9 25 12 28 13 29

Obviously this should have just returned the mask vector of the shuffle
vector.

I mostly reverted this change and made sure the original AVX code worked
for 512-bit vectors as well.

Also updated the tests because they matched the logic from the code.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217602 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-11 16:51:10 +00:00
Elena Demikhovsky
4c97c1420b AVX-512: Fixed a bug in shufflevector lowering.
PALIGNR instruction does not exist in AVX-512F set.
Added a test.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215526 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-13 07:58:43 +00:00
Adam Nemet
a8e1cda622 [AVX512] Add zero-masking variant to AVX512_masking multiclass
This completes one item from the todo-list of r215125 "Generate masking
instruction variants with tablegen".

The AddedComplexity is needed just like for the k variant.

Added a codegen test based on valignq.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215173 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-07 23:53:38 +00:00
Adam Nemet
690499ed49 [AVX512] Add codegen test for the masking variant of valign
The AddedComplexity is needed just like in avx512_perm_3src.  There may be a
bug in the complexity computation...

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215168 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-07 23:18:18 +00:00
Adam Nemet
df5d431084 [X86] AVX512: Add writemask variants for vperm*2*
This includes assembler and codegen support (see the new tests in
avx512-encodings.s and avx512-shuffle.ll).

<rdar://problem/17492620>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212221 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-02 21:25:54 +00:00
Elena Demikhovsky
e3e08acd09 AVX-512: optimized a shuffle pattern to VINSERTI64x4.
Added intrinsics for VPERMT2PS/PD/D/Q instructions.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207513 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-29 09:09:15 +00:00
Elena Demikhovsky
002683abc7 AVX-512: Added intrinsic for cvtph2ps.
Added VPTESTNM instruction.
Added a pattern to vselect (lit tests will follow).


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200823 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-05 07:05:03 +00:00
Elena Demikhovsky
e1a621d84f AVX-512: added VPERM2D VPERM2Q VPERM2PS VPERM2PD instructions,
they give better sequences than VPERMI


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199893 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-23 14:27:26 +00:00
Elena Demikhovsky
3bf51cf302 AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC.
I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll
I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions).


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@197041 91177308-0d34-0410-b5e6-96231b3b80d8
2013-12-11 14:31:04 +00:00
Elena Demikhovsky
5cd32afac4 AVX-512: Concat 4 128-bit vectors in one 512-bit vector.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195229 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-20 09:10:40 +00:00
Elena Demikhovsky
f58e414405 AVX-512: Handled extractelement from mask vector;
Added VMOSHDUP/VMOVSLDUP shuffle instructions.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194691 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-14 11:29:27 +00:00
Elena Demikhovsky
a6269ee5fb AVX-512: fixed shuffle lowering
in case of BLEND and added VSHUFPS patterns.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192055 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-06 06:11:18 +00:00
Elena Demikhovsky
92bfb54770 AVX-512: Added shuffle instructions -
VPSHUFD, VPERMILPS, VMOVDDUP, VMOVLHPS, VMOVHLPS, VSHUFPS, VALIGN
 single and double forms.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189215 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-26 12:45:35 +00:00
Elena Demikhovsky
41f7baf181 AVX-512: added UNPACK instructions and tests for all-zero/all-ones vectors
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189189 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-25 12:54:30 +00:00
Craig Topper
0163356ad1 Don't use v16i32 for load pattern matching. All 512-bit loads are cated to v8i64.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188534 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-16 06:07:34 +00:00
Craig Topper
d36e1efa4b Revert r188449 as it turns out we're just missing the instructions that need the v16i32/v16f32 matching.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188454 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-15 08:38:25 +00:00
Craig Topper
46ceaf4ba6 Don't let isPermImmMask handle v16i32 since VPERMI doesn't match on that type. Remove 128-bit vector handling from isPermImmMask too, it's covered by isPSHUFDMask.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188449 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-15 07:30:51 +00:00
Elena Demikhovsky
fac4a4eb7d AVX-512: Added VPERM* instructons and MOV* zmm-to-zmm instructions.
Added a test for shuffles using VPERM.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188147 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-11 07:55:09 +00:00