Tom Stellard
11580ee62d
R600/SI: Use a ComplexPattern for MUBUF stores
...
Now that non-leaf ComplexPatterns are allowed we can fold all the MUBUF
store patterns into the instruction definition. We will also be able to
reuse this new ComplexPattern for MUBUF loads and atomic operations.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211644 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-24 23:33:07 +00:00
Tom Stellard
78d1e95201
R600: Promote i64 stores to v2i32
...
Now we need only one 64-bit pattern for stores.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211643 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-24 23:33:04 +00:00
Matt Arsenault
95eb45c5d9
R600: Fix inconsistency in rsq instructions.
...
R600 was using a clamped version of rsq, but SI was not. Add a
new rsq_clamped intrinsic and use them consistently.
It's unclear to me from the documentation what behavior
the R600 instructions have, so I assume they have the legacy behavior
described by the SI documents. For R600, use RECIPSQRT_IEEE
for both llvm.AMDGPU.rsq.legacy and llvm.AMDGPU.rsq. R600 also
has RECIPSQRT_FF, which I'm not sure how it fits in here.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211637 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-24 22:13:39 +00:00
Matt Arsenault
26da337208
R600/SI: Move pattern to instruction definition
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211614 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-24 17:17:06 +00:00
Matt Arsenault
ed143b7c0c
R600/SI: Fix div_scale intrinsic.
...
The operand that must match one of the others does matter,
and implement selecting for it.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211523 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-23 18:28:28 +00:00
Tom Stellard
c0bf939e80
R600/SI: Add patterns for ctpop inside a branch
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211378 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-20 17:06:11 +00:00
Tom Stellard
61d64acd0c
R600/SI: Add a pattern for f32 ftrunc
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211377 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-20 17:06:09 +00:00
Tom Stellard
538c95179c
R600/SI: Add a VALU pattern for i64 xor
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211373 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-20 17:05:57 +00:00
Matt Arsenault
d9b35435b8
R600/SI: Add intrinsics for various math instructions.
...
These will be used for custom lowering and for library
implementations of various math functions, so it's useful
to expose these as builtins.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211247 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-19 01:19:19 +00:00
Marek Olsak
f286d63757
R600/SI: add gather4 and getlod intrinsics (v3)
...
This contains all the previous patches + getlod support on top of it.
It doesn't use SDNodes anymore, so it's quite small.
It also adds v16i8 to SReg_128, which is used for the sampler descriptor.
Reviewed-by: Tom Stellard
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211228 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 22:00:29 +00:00
Matt Arsenault
2b6e6fc1a8
R600/SI: Add intrinsics for brev instructions
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211187 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 17:13:57 +00:00
Matt Arsenault
3de1bed8f4
R600/SI: Comparisons set vcc.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211178 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 16:53:48 +00:00
Matt Arsenault
3f1f259c22
R600/SI: Match cttz_zero_undef
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211116 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-17 17:36:27 +00:00
Matt Arsenault
62e378b057
R600/SI: Match ctlz_zero_undef
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211115 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-17 17:36:24 +00:00
Tom Stellard
f56e7678d1
R600: Use LDS and vectors for private memory
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211110 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-17 16:53:14 +00:00
Tom Stellard
ff8dc48da3
R600/SI: Add a pattern for llvm.AMDGPU.barrier.global
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211109 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-17 16:53:09 +00:00
Tom Stellard
61bc72e9ae
R600: Remove AMDIL instruction and register definitions
...
Most of these are no longer used any more.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210915 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-13 16:38:59 +00:00
Matt Arsenault
00c3986254
R600: Mostly remove remaining AMDIL intrinsics.
...
Delete all unused ones, and add new AMDGPU named intrinsics for
the ones that are. Handle the old AMDIL names for comptability (although
remove their GCCBuiltin names) and add tests since there weren't any
for these before.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210827 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-12 21:15:44 +00:00
Matt Arsenault
0b87955888
R600/SI: Use a register set to -1 for data0 on ds_inc*/ds_dec*
...
There is not such thing as a 0-data ds instruction, and the data
operand needs to be a vgpr set to something meaningful.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210756 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-12 08:21:54 +00:00
Matt Arsenault
7fa80b45eb
R600/SI: Fix bitcast between v2i32 and f64
...
This is the same problem fixed in r210664 for more types.
The test passes without this fix. For some reason
I'm only hitting this when creating selects lowered
to v2i32 selects.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210692 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 19:31:13 +00:00
Matt Arsenault
4600175c7f
R600/SI: Update place using old subtarget predicate
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210683 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 18:11:34 +00:00
Matt Arsenault
c9dbd0da7a
R600/SI: Add common 64-bit LDS atomics
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210680 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 18:08:54 +00:00
Matt Arsenault
481a071a8b
R600/SI: Add instruction definitions for 64-bit LDS atomics
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210679 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 18:08:50 +00:00
Matt Arsenault
6b19a3a474
R600/SI: Add 32-bit LDS atomic cmpxchg
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210678 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 18:08:48 +00:00
Matt Arsenault
a396a70c1d
R600/SI: Use LDS atomic inc / dec
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210677 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 18:08:45 +00:00
Matt Arsenault
2da1a85cbb
R600/SI: Add other LDS atomic operations
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210676 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 18:08:42 +00:00
Matt Arsenault
507d8cad7b
R600/SI: Add instruction definitions for more LDS ops
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210675 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 18:08:39 +00:00
Matt Arsenault
4a19dd468d
R600/SI: Fix backwards names for local atomic instructions.
...
The manual lists them as *_RTN_U32, not *_U32_RTN, which is more
consistent with how every other sized instruction is named.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210674 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 18:08:37 +00:00
Matt Arsenault
b97095b94f
R600/SI: Refactor local atomics.
...
Use patterns that will also match the immediate offset to
match the normal read / writes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210673 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 18:08:34 +00:00
Matt Arsenault
8a9df8f92c
R600/SI: Use v_cvt_f32_ubyte* instructions
...
This eliminates extra extract instructions when loading an i8 vector to
a float vector.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210666 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 17:50:44 +00:00
Matt Arsenault
a2dca4cc04
R600/SI: Fix selection failure on scalar_to_vector
...
There seem to be only 2 places that produce these,
and it's kind of tricky to hit them.
Also fixes failure to bitcast between i64 and v2f32,
although this for some reason wasn't actually broken in the
simple bitcast testcase, but did in the scalar_to_vector one.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210664 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 17:40:32 +00:00
Tom Stellard
48d11b0228
R600/SI: Fix a crash when spilling SGPRs
...
We need to make sure only one new instruction is added when spilling
otherwise the register allocator may crash.
This fixes a crash in the game Antichamber.
https://bugs.freedesktop.org/show_bug.cgi?id=75276
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210587 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-10 21:20:38 +00:00
Matt Arsenault
69891c0115
R600/SI: Implement i64 ctpop
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210568 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-10 19:18:24 +00:00
Matt Arsenault
ee9772d9dd
R600/SI: Use bcnt instruction for ctpop
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210567 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-10 19:18:21 +00:00
Matt Arsenault
bfd00e21b7
R600: Handle fcopysign
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210564 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-10 19:00:20 +00:00
Matt Arsenault
0ba78a9121
R600/SI: Handle sign_extend and zero_extend to i64 with patterns.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210563 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-10 18:54:59 +00:00
Matt Arsenault
405ed284b7
R600/SI: Rename VOP3 helper class to be more general
...
It has other uses besides shift instructions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210478 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-09 17:00:46 +00:00
Matt Arsenault
1968da36ed
R600/SI: Keep 64-bit not on SALU
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210476 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-09 16:36:31 +00:00
Matt Arsenault
798a829eca
R600/SI: Match rsq instructions
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210226 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-05 00:15:55 +00:00
Matt Arsenault
606b80c8c4
R600/SI: Remove redundant patterns
...
These patterns are already handled in the instruction definition.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209979 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-31 19:25:17 +00:00
Matt Arsenault
da5824d073
R600/SI: Fix [s|u]int_to_fp for i1
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209971 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-31 06:47:42 +00:00
Matt Arsenault
bf933548ba
R600/SI: Fix pattern variable names.
...
These are confusing enough since the order swaps,
so give them more useful names.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209787 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-29 01:18:01 +00:00
Matt Arsenault
f49da4338a
R600: Add intrinsics for mad24
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209456 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-22 18:00:15 +00:00
Matt Arsenault
351c658fc2
R600/SI: Move instruction pattern to instruction definition
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209454 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-22 17:45:20 +00:00
Matt Arsenault
bce7d05ba9
R600/SI: Match fp_to_uint / uint_to_fp for f64
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209388 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-22 03:20:30 +00:00
Tom Stellard
17200e3bb3
R600/SI: Refactor the VOP3_32 tablegen class
...
This will allow us to use a single MachineInstr to represent
instructions which behave the same but have different encodings
on some subtargets.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209028 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-16 20:56:47 +00:00
Tom Stellard
9d99d7a185
R600/SI: Add a PredicateControl class for managing TableGen predicates
...
This was inspired by the PredicateControl class in the MIPS backend.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209027 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-16 20:56:45 +00:00
Tom Stellard
9b24e0c6f2
R600/SI: Move tablegen patterns away from instruction defs
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209026 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-16 20:56:44 +00:00
Tom Stellard
1c073c3954
R600/SI: Remove unused instruction
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209025 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-16 20:56:43 +00:00
Tom Stellard
3bd63cd655
R600/SI: Promote f32 SELECT to i32
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209024 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-16 20:56:41 +00:00