146 Commits

Author SHA1 Message Date
Matt Arsenault
c9cc488dfe R600/SI: Try to keep i32 mul on SALU
Also fix bug this exposed where when legalizing an immediate
operand, a v_mov_b32 would be created with a VSrc dest register.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217108 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-03 23:24:35 +00:00
Tom Stellard
2381dfeab5 R600/SI: Fix typos in SIInstrInfo::areLoadsFromSameBasePtr()
This fixes a crash in the OpenCV test:
ImgprocWarpResizeArea/Resize.Mat/16

There is no test case for this, because this failure depends on a
specific ordering of the loads, which could easily change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217040 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-03 15:22:39 +00:00
Tom Stellard
fdbf61d00d R600/SI: Teach moveToVALU how to handle more S_LOAD_* instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@216220 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-21 20:41:00 +00:00
Tom Stellard
9b60cb102a R600/SI: Use eliminateFrameIndex() to expand SGPR spill pseudos
This will simplify the SGPR spilling and also allow us to use
MachineFrameInfo for calculating offsets, which should be more
reliable than our custom code.

This fixes a crash in some cases where a register would be spilled
in a branch such that the VGPR defined for spilling did not dominate
all the uses when restoring.

This fixes a crash in an ocl conformance test.  The test requries
register spilling and is too big to include.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@216217 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-21 20:40:54 +00:00
Matt Arsenault
34ef4cd65b R600/SI: Fix offset folding in some cases with shifted pointers.
Ordinarily (shl (add x, c1), c2) -> (add (shl x, c2), c1 << c2)
is only done if the add has one use. If the resulting constant
add can be folded into an addressing mode, force this to happen
for the pointer operand.

This ends up happening a lot because of how LDS objects are allocated.
Since the globals are allocated next to each other, acessing the first
element of the second object is directly indexed by a shifted pointer.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215739 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-15 17:49:05 +00:00
NAKAMURA Takumi
61b3abb685 R600/SIInstrInfo.cpp: Suppress an warning. [-Wunused-variable]
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215406 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-11 23:03:38 +00:00
Tom Stellard
13f4476c55 R600/SI: Add a ComplexPattern for selecting MUBUF _OFFSET variant
This saves us from having to copy a 64-bit 0 value into VGPRs for
BUFFER_* instruction which only have a 12-bit immediate offset.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215399 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-11 22:18:17 +00:00
Matt Arsenault
c9c70b1651 R600/SI: Implement areLoadsFromSameBasePtr
This currently has a noticable effect on the kernel argument loads.
LDS and global loads are more problematic, I think because of how copies
are currently inserted to ensure that the address is a VGPR.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214942 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-06 00:29:43 +00:00
Tom Stellard
17c8fefc9f R600/SI: Do abs/neg folding with ComplexPatterns
Abs/neg folding has moved out of foldOperands and into the instruction
selection phase using complex patterns.  As a consequence of this
change, we now prefer to select the 64-bit encoding for most
instructions and the modifier operands have been dropped from
integer VOP3 instructions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214467 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-01 00:32:39 +00:00
Tom Stellard
5494792536 R600/SI: Simplify and fix handling of VOP2 in SIInstrInfo::legalizeOperands
We were incorrectly assuming that all VOP2 instructions can read SGPRs
in Src0, but this is not true for instructions that read carry-in from
VCC.

The old logic has been replaced with new logic which checks the defined
register classes of the VOP2 instruction to determine whether or not to
legalize the operands.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214465 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-01 00:32:35 +00:00
Tom Stellard
635fb9fb59 R600/SI: Fold immediates when shrinking instructions
This will prevent us from using extra MOV instructions once we prefer
selecting 64-bit instructions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214464 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-01 00:32:33 +00:00
Tom Stellard
e5fc4208eb R600/SI: Fix incorrect commute operation in shrink instructions pass
We were commuting the instruction by still shrinking it using the
original opcode.

NOTE: This is a candidate for the 3.5 branch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214463 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-01 00:32:28 +00:00
Matt Arsenault
8ad24437bd R600/SI: Consider adjacent offsets in getLdStBaseRegImmOfs
We can treat ds_read2_* as a single offset if the offsets are adjacent.

No test since emission of read2 instructions for partially
aligned loads isn't implemented yet.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214269 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-30 01:01:10 +00:00
Matt Arsenault
37467aeaf2 R600/SI: Implement getLdStBaseRegImmOfs
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214225 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-29 21:34:55 +00:00
Matt Arsenault
b33d6c412d R600/SI: Add isMUBUF / isMTBUF
Also add missing comments about how the flags work.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214195 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-29 18:51:56 +00:00
Matt Arsenault
3bd14877eb Fix typos / grammar.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214147 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-29 00:02:40 +00:00
Matt Arsenault
7505602266 R600/SI: Fix return type for isMIMG / isSMRD
All the others use bool, so these should too.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214106 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-28 17:59:38 +00:00
Tom Stellard
9787e8c76b R600/SI: Add instruction shrinking pass
This pass converts 64-bit instructions to 32-bit when possible.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213561 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-21 16:55:33 +00:00
Tom Stellard
3280804237 R600/SI: Use scratch memory for large private arrays
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213551 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-21 15:45:01 +00:00
Tom Stellard
b664d47cb0 R600/SI: Store constant initializer data in constant memory
This implements a solution for constant initializers suggested
by Vadim Girlin, where we store the data after the shader code
and then use the S_GETPC instruction to compute its address.

This saves use the trouble of creating a new buffer for constant data
and then having to pass the pointer to the kernel via user SGPRs or the
input buffer.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213530 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-21 14:01:14 +00:00
Tom Stellard
a75d388f18 R600/SI: Add verifier check for immediates in register operands.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212214 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-02 20:53:44 +00:00
Matt Arsenault
140c6ae970 R600/SI: Verify restrictions on div_scale operands.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211524 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-23 18:28:31 +00:00
Matt Arsenault
2b6e6fc1a8 R600/SI: Add intrinsics for brev instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211187 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 17:13:57 +00:00
Matt Arsenault
3f1f259c22 R600/SI: Match cttz_zero_undef
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211116 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-17 17:36:27 +00:00
Matt Arsenault
62e378b057 R600/SI: Match ctlz_zero_undef
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211115 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-17 17:36:24 +00:00
Tom Stellard
6b0a08b15b R600: Move AMDGPUInstrInfo from AMDGPUTargetMachine into AMDGPUSubtarget
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210869 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-13 01:32:00 +00:00
Tom Stellard
cd0b8d6cbf R600/SI: Emit an error when attempting to spill VGPRs v4
I can't get VGPR spilling to work reliable, so for now just emit
an error when the register allocator tries to spill VGPRs.

v2:
  - Fix build
v3:
  - Added crash fix when spilling SPGRs
v4:
  - Use V_MOV_B32 as a dummy instruction instead of S_NOP

Patch by: Darren Powell

https://bugs.freedesktop.org/show_bug.cgi?id=75276

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210588 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-10 21:20:41 +00:00
Tom Stellard
48d11b0228 R600/SI: Fix a crash when spilling SGPRs
We need to make sure only one new instruction is added when spilling
otherwise the register allocator may crash.

This fixes a crash in the game Antichamber.

https://bugs.freedesktop.org/show_bug.cgi?id=75276

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210587 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-10 21:20:38 +00:00
Matt Arsenault
69891c0115 R600/SI: Implement i64 ctpop
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210568 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-10 19:18:24 +00:00
Matt Arsenault
ee9772d9dd R600/SI: Use bcnt instruction for ctpop
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210567 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-10 19:18:21 +00:00
Matt Arsenault
1968da36ed R600/SI: Keep 64-bit not on SALU
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210476 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-09 16:36:31 +00:00
Matt Arsenault
1b68686365 Fix typos
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210135 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-03 23:06:13 +00:00
Tom Stellard
704fe35d88 R600/SI: Fix a bug with handling of INSERT_SUBREG in SIFixSGPRCopies
This prevents a future commit from regressing the load-i1.ll test.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@208887 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-15 14:41:55 +00:00
Matt Arsenault
dda22295e4 R600/SI: Try to fix BFE operands when moving to VALU
This was broken by r208479

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@208740 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-13 23:45:50 +00:00
Vincent Lejeune
3378ca7d5c R600/SI: Prettier display of input modifiers
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@208479 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-10 19:18:33 +00:00
Tom Stellard
3f26d366a4 R600/SI: Teach SIInstrInfo::moveToVALU() how to move S_LOAD_*_IMM instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@208432 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-09 16:42:22 +00:00
Tom Stellard
9b22626068 R600/SI: Only create one instruction when spilling/restoring register v3
The register spiller assumes that only one new instruction is created
when spilling and restoring registers, so we need to emit pseudo
instructions for vector register spills and lower them after
register allocation.

v2:
  - Fix calculation of lane index
  - Extend VGPR liveness to end of program.

v3:
  - Use SIMM16 field of S_NOP to specify multiple NOPs.

https://bugs.freedesktop.org/show_bug.cgi?id=75005

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207843 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-02 15:41:42 +00:00
Tom Stellard
1d8e31fc7a R600/SI: Teach moveToVALU how to handle some SMRD instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207660 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-30 15:31:29 +00:00
Craig Topper
c848b1bbcf [C++] Use 'nullptr'. Target edition.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207197 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-25 05:30:21 +00:00
Matt Arsenault
746734df1a R600/SI: Try to use scalar BFE.
Use scalar BFE with constant shift and offset when possible.
This is complicated by the fact that the scalar version packs
the two operands of the vector version into one.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206558 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-18 05:19:26 +00:00
Matt Arsenault
6834a55df3 R600/SI: Match sign_extend_inreg to s_sext_i32_i8 and s_sext_i32_i16
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206547 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-18 01:53:18 +00:00
Tom Stellard
c2bab04829 R600/SI: Teach SIInstrInfo::moveToVALU() how to handle PHI instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206498 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-17 21:00:07 +00:00
Tom Stellard
1b011b51ea R600/SI: Legalize operands after changing dst reg in FixSGPRCopies
Otherwise we may not legalize some illegal REG_SEQUENCE instructions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206497 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-17 21:00:01 +00:00
Matt Arsenault
6e6126ac4c R600/SI: Refactor SOPC classes slightly.
Better match what is done for VOPC to eventually
prefer selecting these.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206048 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-11 19:25:18 +00:00
Matt Arsenault
d4786ed1de R600/SI: Match not instruction.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205837 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-09 07:16:16 +00:00
Tom Stellard
1d8c7eb225 R600/SI: Handle INSERT_SUBREG in SIFixSGPRCopies
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205732 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-07 19:45:45 +00:00
Matt Arsenault
4065ed4eb0 R600/SI: Implement shouldConvertConstantLoadToIntImm
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205244 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-31 19:54:27 +00:00
Tom Stellard
aa23732edd R600/SI: Implement SIInstrInfo::isTriviallyRematerializable()
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205188 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-31 14:01:56 +00:00
Matt Arsenault
add2e2ec8f R600/SI: Fix extra mov from legalizing 64-bit SALU ops.
Check the register class of each operand individually
to avoid an extra copy to a vgpr.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204662 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-24 20:08:13 +00:00
Matt Arsenault
3a96e61469 R600/SI: Sub-optimial fix for 64-bit immediates with SALU ops.
No longer asserts, but now you get moves loading legal immediates
into the split 32-bit operations.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204661 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-24 20:08:09 +00:00