79 Commits

Author SHA1 Message Date
Tom Stellard
9787e8c76b R600/SI: Add instruction shrinking pass
This pass converts 64-bit instructions to 32-bit when possible.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213561 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-21 16:55:33 +00:00
Tom Stellard
3280804237 R600/SI: Use scratch memory for large private arrays
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213551 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-21 15:45:01 +00:00
Tom Stellard
b664d47cb0 R600/SI: Store constant initializer data in constant memory
This implements a solution for constant initializers suggested
by Vadim Girlin, where we store the data after the shader code
and then use the S_GETPC instruction to compute its address.

This saves use the trouble of creating a new buffer for constant data
and then having to pass the pointer to the kernel via user SGPRs or the
input buffer.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213530 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-21 14:01:14 +00:00
Tom Stellard
a75d388f18 R600/SI: Add verifier check for immediates in register operands.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212214 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-02 20:53:44 +00:00
Matt Arsenault
140c6ae970 R600/SI: Verify restrictions on div_scale operands.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211524 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-23 18:28:31 +00:00
Matt Arsenault
2b6e6fc1a8 R600/SI: Add intrinsics for brev instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211187 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 17:13:57 +00:00
Matt Arsenault
3f1f259c22 R600/SI: Match cttz_zero_undef
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211116 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-17 17:36:27 +00:00
Matt Arsenault
62e378b057 R600/SI: Match ctlz_zero_undef
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211115 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-17 17:36:24 +00:00
Tom Stellard
6b0a08b15b R600: Move AMDGPUInstrInfo from AMDGPUTargetMachine into AMDGPUSubtarget
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210869 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-13 01:32:00 +00:00
Tom Stellard
cd0b8d6cbf R600/SI: Emit an error when attempting to spill VGPRs v4
I can't get VGPR spilling to work reliable, so for now just emit
an error when the register allocator tries to spill VGPRs.

v2:
  - Fix build
v3:
  - Added crash fix when spilling SPGRs
v4:
  - Use V_MOV_B32 as a dummy instruction instead of S_NOP

Patch by: Darren Powell

https://bugs.freedesktop.org/show_bug.cgi?id=75276

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210588 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-10 21:20:41 +00:00
Tom Stellard
48d11b0228 R600/SI: Fix a crash when spilling SGPRs
We need to make sure only one new instruction is added when spilling
otherwise the register allocator may crash.

This fixes a crash in the game Antichamber.

https://bugs.freedesktop.org/show_bug.cgi?id=75276

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210587 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-10 21:20:38 +00:00
Matt Arsenault
69891c0115 R600/SI: Implement i64 ctpop
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210568 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-10 19:18:24 +00:00
Matt Arsenault
ee9772d9dd R600/SI: Use bcnt instruction for ctpop
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210567 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-10 19:18:21 +00:00
Matt Arsenault
1968da36ed R600/SI: Keep 64-bit not on SALU
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210476 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-09 16:36:31 +00:00
Matt Arsenault
1b68686365 Fix typos
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210135 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-03 23:06:13 +00:00
Tom Stellard
704fe35d88 R600/SI: Fix a bug with handling of INSERT_SUBREG in SIFixSGPRCopies
This prevents a future commit from regressing the load-i1.ll test.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@208887 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-15 14:41:55 +00:00
Matt Arsenault
dda22295e4 R600/SI: Try to fix BFE operands when moving to VALU
This was broken by r208479

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@208740 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-13 23:45:50 +00:00
Vincent Lejeune
3378ca7d5c R600/SI: Prettier display of input modifiers
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@208479 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-10 19:18:33 +00:00
Tom Stellard
3f26d366a4 R600/SI: Teach SIInstrInfo::moveToVALU() how to move S_LOAD_*_IMM instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@208432 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-09 16:42:22 +00:00
Tom Stellard
9b22626068 R600/SI: Only create one instruction when spilling/restoring register v3
The register spiller assumes that only one new instruction is created
when spilling and restoring registers, so we need to emit pseudo
instructions for vector register spills and lower them after
register allocation.

v2:
  - Fix calculation of lane index
  - Extend VGPR liveness to end of program.

v3:
  - Use SIMM16 field of S_NOP to specify multiple NOPs.

https://bugs.freedesktop.org/show_bug.cgi?id=75005

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207843 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-02 15:41:42 +00:00
Tom Stellard
1d8e31fc7a R600/SI: Teach moveToVALU how to handle some SMRD instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207660 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-30 15:31:29 +00:00
Craig Topper
c848b1bbcf [C++] Use 'nullptr'. Target edition.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207197 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-25 05:30:21 +00:00
Matt Arsenault
746734df1a R600/SI: Try to use scalar BFE.
Use scalar BFE with constant shift and offset when possible.
This is complicated by the fact that the scalar version packs
the two operands of the vector version into one.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206558 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-18 05:19:26 +00:00
Matt Arsenault
6834a55df3 R600/SI: Match sign_extend_inreg to s_sext_i32_i8 and s_sext_i32_i16
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206547 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-18 01:53:18 +00:00
Tom Stellard
c2bab04829 R600/SI: Teach SIInstrInfo::moveToVALU() how to handle PHI instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206498 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-17 21:00:07 +00:00
Tom Stellard
1b011b51ea R600/SI: Legalize operands after changing dst reg in FixSGPRCopies
Otherwise we may not legalize some illegal REG_SEQUENCE instructions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206497 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-17 21:00:01 +00:00
Matt Arsenault
6e6126ac4c R600/SI: Refactor SOPC classes slightly.
Better match what is done for VOPC to eventually
prefer selecting these.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206048 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-11 19:25:18 +00:00
Matt Arsenault
d4786ed1de R600/SI: Match not instruction.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205837 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-09 07:16:16 +00:00
Tom Stellard
1d8c7eb225 R600/SI: Handle INSERT_SUBREG in SIFixSGPRCopies
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205732 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-07 19:45:45 +00:00
Matt Arsenault
4065ed4eb0 R600/SI: Implement shouldConvertConstantLoadToIntImm
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205244 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-31 19:54:27 +00:00
Tom Stellard
aa23732edd R600/SI: Implement SIInstrInfo::isTriviallyRematerializable()
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205188 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-31 14:01:56 +00:00
Matt Arsenault
add2e2ec8f R600/SI: Fix extra mov from legalizing 64-bit SALU ops.
Check the register class of each operand individually
to avoid an extra copy to a vgpr.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204662 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-24 20:08:13 +00:00
Matt Arsenault
3a96e61469 R600/SI: Sub-optimial fix for 64-bit immediates with SALU ops.
No longer asserts, but now you get moves loading legal immediates
into the split 32-bit operations.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204661 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-24 20:08:09 +00:00
Matt Arsenault
db1807144a R600/SI: Fix 64-bit bit ops that require the VALU.
Try to match scalar and first like the other instructions.
Expand 64-bit ands to a pair of 32-bit ands since that is not
available on the VALU.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204660 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-24 20:08:05 +00:00
Matt Arsenault
03cd663eb1 R600/SI: Move splitting 64-bit immediates to separate function.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204651 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-24 18:26:52 +00:00
Tom Stellard
4ddee6a5da R600/SI: Fix warning with gcc 4.8.2
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204618 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-24 16:12:34 +00:00
Matt Arsenault
55d17f4842 R600/SI: Move instruction patterns to scalar versions.
Some of them also had the pattern on both, so this removes the
duplication.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204492 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-21 18:01:18 +00:00
Tom Stellard
a1d28f6dd7 R600/SI: Handle MUBUF instructions in SIInstrInfo::moveToVALU()
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204476 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-21 15:51:57 +00:00
Tom Stellard
1f1c0495d0 R600/SI: Handle S_MOV_B64 in SIInstrInfo::moveToVALU()
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204475 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-21 15:51:54 +00:00
Tom Stellard
ad52f4f70c R600/SI: Fix implementation of isInlineConstant() used by the verifier
The type of the immediates should not matter as long as the encoding is
equivalent to the encoding of one of the legal inline constants.

Tested-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204056 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-17 17:03:52 +00:00
Tom Stellard
27b614fcd3 R600/SI: Add generic checks to SIInstrInfo::verifyInstruction()
Added checks for number of operands and operand register classes.

Tested-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204054 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-17 17:03:49 +00:00
Owen Anderson
bf63022492 Phase 2 of the great MachineRegisterInfo cleanup. This time, we're changing
operator* on the by-operand iterators to return a MachineOperand& rather than
a MachineInstr&.  At this point they almost behave like normal iterators!

Again, this requires making some existing loops more verbose, but should pave
the way for the big range-based for-loop cleanups in the future.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@203865 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-13 23:12:04 +00:00
Matt Arsenault
6cf9da3d85 Move trivial getter into header.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@203517 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-11 00:01:34 +00:00
Tom Stellard
42faffddb9 R600/SI: Initialize M0 and emit S_WQM_B64 whenever DS instructions are used
DS instructions that access local memory can only uses addresses that
are less than or equal to the value of M0.  When M0 is uninitialized,
then we experience undefined behavior.

This patch also changes the behavior to emit S_WQM_B64 on pixel shaders
no matter what kind of DS instruction is used.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201097 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-10 16:58:30 +00:00
Andrew Trick
c4c5a1d526 Allow MachineCSE to coalesce trivial subregister copies the same way that it coalesces normal copies.
Without this, MachineCSE is powerless to handle redundant operations with truncated source operands.

This required fixing the 2-addr pass to handle tied subregisters. It isn't clear what combinations of subregisters can legally be tied, but the simple case of truncated source operands is now safely handled:

     %vreg11<def> = COPY %vreg1:sub_32bit; GR32:%vreg11 GR64:%vreg1
     %vreg12<def> = COPY %vreg2:sub_32bit; GR32:%vreg12 GR64:%vreg2
     %vreg13<def,tied1> = ADD32rr %vreg11<tied0>, %vreg12<kill>, %EFLAGS<imp-def>

Test case: cse-add-with-overflow.ll.

This exposed an existing bug in
PPCInstrInfo::commuteInstruction. Thanks to Rafael for the test case:
PowerPC/crash.ll.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@197465 91177308-0d34-0410-b5e6-96231b3b80d8
2013-12-17 04:50:45 +00:00
Tom Stellard
aa6ec15caf R600/SI: Implement spilling of SGPRs v5
SGPRs are spilled into VGPRs using the {READ,WRITE}LANE_B32 instructions.

v2:
  - Fix encoding of Lane Mask
  - Use correct register flags, so we don't overwrite the low dword
    when restoring multi-dword registers.

v3:
  - Register spilling seems to hang the GPU, so replace all shaders
    that need spilling with a dummy shader.

v4:
  - Fix *LANE definitions
  - Change destination reg class for 32-bit SMRD instructions

v5:
  - Remove small optimization that was crashing Serious Sam 3.

https://bugs.freedesktop.org/show_bug.cgi?id=68224
https://bugs.freedesktop.org/show_bug.cgi?id=71285

NOTE: This is a candidate for the 3.4 branch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195880 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-27 21:23:35 +00:00
Matt Arsenault
3560dd2dcd R600/SI: Fix moveToVALU when the first operand is VSrc.
Moving into a VSrc doesn't always work, since it could be
replaced with an SGPR later.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195042 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-18 20:09:55 +00:00
Matt Arsenault
9bc4b2c0da R600/SI: Fix multiple SGPR reads when using VCC.
No other SGPR operands are allowed, so if VCC is
used, move the other to a VGPR.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195041 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-18 20:09:50 +00:00
Matt Arsenault
3e38856f04 R600/SI: Move patterns to match add / sub to scalar instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195034 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-18 20:09:29 +00:00
Matt Arsenault
836c5133c6 R600/SI: Fix extra defs of VCC / SCC.
When replacing scalar operations with vector,
the wrong implicit output register was used.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195033 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-18 20:09:21 +00:00