232 Commits

Author SHA1 Message Date
Tom Stellard
11580ee62d R600/SI: Use a ComplexPattern for MUBUF stores
Now that non-leaf ComplexPatterns are allowed we can fold all the MUBUF
store patterns into the instruction definition.  We will also be able to
reuse this new ComplexPattern for MUBUF loads and atomic operations.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211644 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-24 23:33:07 +00:00
Tom Stellard
78d1e95201 R600: Promote i64 stores to v2i32
Now we need only one 64-bit pattern for stores.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211643 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-24 23:33:04 +00:00
Matt Arsenault
95eb45c5d9 R600: Fix inconsistency in rsq instructions.
R600 was using a clamped version of rsq, but SI was not. Add a
new rsq_clamped intrinsic and use them consistently.

It's unclear to me from the documentation what behavior
the R600 instructions have, so I assume they have the legacy behavior
described by the SI documents. For R600, use RECIPSQRT_IEEE
for both llvm.AMDGPU.rsq.legacy and llvm.AMDGPU.rsq. R600 also
has RECIPSQRT_FF, which I'm not sure how it fits in here.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211637 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-24 22:13:39 +00:00
Matt Arsenault
26da337208 R600/SI: Move pattern to instruction definition
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211614 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-24 17:17:06 +00:00
Matt Arsenault
ed143b7c0c R600/SI: Fix div_scale intrinsic.
The operand that must match one of the others does matter,
and implement selecting for it.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211523 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-23 18:28:28 +00:00
Tom Stellard
c0bf939e80 R600/SI: Add patterns for ctpop inside a branch
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211378 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-20 17:06:11 +00:00
Tom Stellard
61d64acd0c R600/SI: Add a pattern for f32 ftrunc
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211377 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-20 17:06:09 +00:00
Tom Stellard
538c95179c R600/SI: Add a VALU pattern for i64 xor
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211373 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-20 17:05:57 +00:00
Matt Arsenault
d9b35435b8 R600/SI: Add intrinsics for various math instructions.
These will be used for custom lowering and for library
implementations of various math functions, so it's useful
to expose these as builtins.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211247 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-19 01:19:19 +00:00
Marek Olsak
f286d63757 R600/SI: add gather4 and getlod intrinsics (v3)
This contains all the previous patches + getlod support on top of it.
It doesn't use SDNodes anymore, so it's quite small.
It also adds v16i8 to SReg_128, which is used for the sampler descriptor.

Reviewed-by: Tom Stellard

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211228 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 22:00:29 +00:00
Matt Arsenault
2b6e6fc1a8 R600/SI: Add intrinsics for brev instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211187 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 17:13:57 +00:00
Matt Arsenault
3de1bed8f4 R600/SI: Comparisons set vcc.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211178 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 16:53:48 +00:00
Matt Arsenault
3f1f259c22 R600/SI: Match cttz_zero_undef
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211116 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-17 17:36:27 +00:00
Matt Arsenault
62e378b057 R600/SI: Match ctlz_zero_undef
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211115 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-17 17:36:24 +00:00
Tom Stellard
f56e7678d1 R600: Use LDS and vectors for private memory
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211110 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-17 16:53:14 +00:00
Tom Stellard
ff8dc48da3 R600/SI: Add a pattern for llvm.AMDGPU.barrier.global
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211109 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-17 16:53:09 +00:00
Tom Stellard
61bc72e9ae R600: Remove AMDIL instruction and register definitions
Most of these are no longer used any more.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210915 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-13 16:38:59 +00:00
Matt Arsenault
00c3986254 R600: Mostly remove remaining AMDIL intrinsics.
Delete all unused ones, and add new AMDGPU named intrinsics for
the ones that are. Handle the old AMDIL names for comptability (although
remove their GCCBuiltin names) and add tests since there weren't any
for these before.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210827 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-12 21:15:44 +00:00
Matt Arsenault
0b87955888 R600/SI: Use a register set to -1 for data0 on ds_inc*/ds_dec*
There is not such thing as a 0-data ds instruction, and the data
operand needs to be a vgpr set to something meaningful.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210756 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-12 08:21:54 +00:00
Matt Arsenault
7fa80b45eb R600/SI: Fix bitcast between v2i32 and f64
This is the same problem fixed in r210664 for more types.

The test passes without this fix. For some reason
I'm only hitting this when creating selects lowered
to v2i32 selects.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210692 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 19:31:13 +00:00
Matt Arsenault
4600175c7f R600/SI: Update place using old subtarget predicate
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210683 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 18:11:34 +00:00
Matt Arsenault
c9dbd0da7a R600/SI: Add common 64-bit LDS atomics
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210680 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 18:08:54 +00:00
Matt Arsenault
481a071a8b R600/SI: Add instruction definitions for 64-bit LDS atomics
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210679 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 18:08:50 +00:00
Matt Arsenault
6b19a3a474 R600/SI: Add 32-bit LDS atomic cmpxchg
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210678 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 18:08:48 +00:00
Matt Arsenault
a396a70c1d R600/SI: Use LDS atomic inc / dec
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210677 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 18:08:45 +00:00
Matt Arsenault
2da1a85cbb R600/SI: Add other LDS atomic operations
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210676 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 18:08:42 +00:00
Matt Arsenault
507d8cad7b R600/SI: Add instruction definitions for more LDS ops
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210675 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 18:08:39 +00:00
Matt Arsenault
4a19dd468d R600/SI: Fix backwards names for local atomic instructions.
The manual lists them as *_RTN_U32, not *_U32_RTN, which is more
consistent with how every other sized instruction is named.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210674 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 18:08:37 +00:00
Matt Arsenault
b97095b94f R600/SI: Refactor local atomics.
Use patterns that will also match the immediate offset to
match the normal read / writes.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210673 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 18:08:34 +00:00
Matt Arsenault
8a9df8f92c R600/SI: Use v_cvt_f32_ubyte* instructions
This eliminates extra extract instructions when loading an i8 vector to
a float vector.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210666 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 17:50:44 +00:00
Matt Arsenault
a2dca4cc04 R600/SI: Fix selection failure on scalar_to_vector
There seem to be only 2 places that produce these,
and it's kind of tricky to hit them.

Also fixes failure to bitcast between i64 and v2f32,
although this for some reason wasn't actually broken in the
simple bitcast testcase, but did in the scalar_to_vector one.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210664 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 17:40:32 +00:00
Tom Stellard
48d11b0228 R600/SI: Fix a crash when spilling SGPRs
We need to make sure only one new instruction is added when spilling
otherwise the register allocator may crash.

This fixes a crash in the game Antichamber.

https://bugs.freedesktop.org/show_bug.cgi?id=75276

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210587 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-10 21:20:38 +00:00
Matt Arsenault
69891c0115 R600/SI: Implement i64 ctpop
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210568 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-10 19:18:24 +00:00
Matt Arsenault
ee9772d9dd R600/SI: Use bcnt instruction for ctpop
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210567 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-10 19:18:21 +00:00
Matt Arsenault
bfd00e21b7 R600: Handle fcopysign
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210564 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-10 19:00:20 +00:00
Matt Arsenault
0ba78a9121 R600/SI: Handle sign_extend and zero_extend to i64 with patterns.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210563 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-10 18:54:59 +00:00
Matt Arsenault
405ed284b7 R600/SI: Rename VOP3 helper class to be more general
It has other uses besides shift instructions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210478 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-09 17:00:46 +00:00
Matt Arsenault
1968da36ed R600/SI: Keep 64-bit not on SALU
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210476 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-09 16:36:31 +00:00
Matt Arsenault
798a829eca R600/SI: Match rsq instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210226 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-05 00:15:55 +00:00
Matt Arsenault
606b80c8c4 R600/SI: Remove redundant patterns
These patterns are already handled in the instruction definition.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209979 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-31 19:25:17 +00:00
Matt Arsenault
da5824d073 R600/SI: Fix [s|u]int_to_fp for i1
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209971 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-31 06:47:42 +00:00
Matt Arsenault
bf933548ba R600/SI: Fix pattern variable names.
These are confusing enough since the order swaps,
so give them more useful names.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209787 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-29 01:18:01 +00:00
Matt Arsenault
f49da4338a R600: Add intrinsics for mad24
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209456 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-22 18:00:15 +00:00
Matt Arsenault
351c658fc2 R600/SI: Move instruction pattern to instruction definition
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209454 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-22 17:45:20 +00:00
Matt Arsenault
bce7d05ba9 R600/SI: Match fp_to_uint / uint_to_fp for f64
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209388 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-22 03:20:30 +00:00
Tom Stellard
17200e3bb3 R600/SI: Refactor the VOP3_32 tablegen class
This will allow us to use a single MachineInstr to represent
instructions which behave the same but have different encodings
on some subtargets.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209028 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-16 20:56:47 +00:00
Tom Stellard
9d99d7a185 R600/SI: Add a PredicateControl class for managing TableGen predicates
This was inspired by the PredicateControl class in the MIPS backend.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209027 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-16 20:56:45 +00:00
Tom Stellard
9b24e0c6f2 R600/SI: Move tablegen patterns away from instruction defs
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209026 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-16 20:56:44 +00:00
Tom Stellard
1c073c3954 R600/SI: Remove unused instruction
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209025 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-16 20:56:43 +00:00
Tom Stellard
3bd63cd655 R600/SI: Promote f32 SELECT to i32
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209024 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-16 20:56:41 +00:00