Commit Graph

289 Commits

Author SHA1 Message Date
Matt Arsenault
aa796b99bb R600/SI: Use complex pattern for MUBUF load patterns.
This eliminates a use of the SI_ADDR64_RSRC pseudo

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@220057 91177308-0d34-0410-b5e6-96231b3b80d8
2014-10-17 17:43:00 +00:00
Matt Arsenault
46b53c9e4b R600/SI: Remove SI_BUFFER_RSRC pseudo
Just use REG_SEQUENCE directly, so there are fewer
instructions to need to deal with later.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@220056 91177308-0d34-0410-b5e6-96231b3b80d8
2014-10-17 17:42:56 +00:00
Matt Arsenault
61bf4bf2c3 R600/SI: Remove another VALU pattern
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@219988 91177308-0d34-0410-b5e6-96231b3b80d8
2014-10-16 23:33:37 +00:00
Matt Arsenault
ceb4f4907d R600/SI: Remove unnecessary VALU patterns
These haven't been necessary since allowing
selecting SALU instructions in non-entry blocks
was enabled.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@219956 91177308-0d34-0410-b5e6-96231b3b80d8
2014-10-16 20:31:50 +00:00
Tom Stellard
806e08c5a0 R600/SI: Refactor VOP3 instruction defs
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@219256 91177308-0d34-0410-b5e6-96231b3b80d8
2014-10-07 23:51:41 +00:00
Tom Stellard
96c9ebbd04 R600/SI: Refactor VOPC instruction defs
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@219255 91177308-0d34-0410-b5e6-96231b3b80d8
2014-10-07 23:51:39 +00:00
Tom Stellard
40949737c4 R600/SI: Refactor VOP2 instruction defs
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@219254 91177308-0d34-0410-b5e6-96231b3b80d8
2014-10-07 23:51:38 +00:00
Tom Stellard
6bf4c41242 R600/SI: Refactor VOP1 instruction defs
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@219253 91177308-0d34-0410-b5e6-96231b3b80d8
2014-10-07 23:51:34 +00:00
Tom Stellard
f7082f9bd7 R600/SI: Add a generic pseudo EXP instruction
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218767 91177308-0d34-0410-b5e6-96231b3b80d8
2014-10-01 14:44:45 +00:00
Tom Stellard
cbb63311cd R600/SI: Add generic pseudo MTBUF instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218766 91177308-0d34-0410-b5e6-96231b3b80d8
2014-10-01 14:44:43 +00:00
Tom Stellard
29d48e6a49 R600/SI: Add support for global atomic add
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218457 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-25 18:30:26 +00:00
Tom Stellard
81c6c9690a R600/SI: Enable selecting SALU inside branches
We can do this now that the FixSGPRLiveRanges pass is working.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218353 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-24 01:33:28 +00:00
Tom Stellard
36ba7962a4 R600/SI: Fix the FixSGPRLiveRanges pass
The previous implementation was extending the live range of SGPRs
by modifying the live intervals directly.  This was causing a lot
of machine verification errors when the machine scheduler was enabled.

The new implementation adds pseudo instructions with implicit uses to
extend the live ranges of SGPRs, which works much better.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218351 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-24 01:33:24 +00:00
Tom Stellard
860cabe1e6 R600/SI: Implement VGPR register spilling for compute at -O0 v3
VGPRs are spilled to LDS.  This still needs more testing, but
we need to at least enable it at -O0, because the fast register
allocator spills all registers that are live at the end of blocks
and without this some future commits will break the
flat-address-space.ll test.

v2: Only calculate thread id once

v3: Move insertion of spill instructions to
    SIRegisterInfo::eliminateFrameIndex()

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218348 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-24 01:33:17 +00:00
Tom Stellard
e1bc40b1e6 Revert "R600/SI: Add support for global atomic add"
This reverts commit r218254.

The global_atomics.ll test fails with asserts disabled.  For some reason,
the compiler fails to produce the atomic no return variants.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218257 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-22 16:44:04 +00:00
Tom Stellard
6d625ad495 R600/SI: Add support for global atomic add
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218254 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-22 15:35:35 +00:00
Tom Stellard
2de7c388a6 R600/SI: Remove modifier operands from V_CNDMASK_B32_e64
Modifiers don't work for this instruction.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218253 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-22 15:35:34 +00:00
Matt Arsenault
f1b16047b7 R600/SI: Prefer selecting more e64 instruction forms.
Add some more tests to make sure better operand
choices are still made. Leave some cases that seem
to have no reason to ever be e64 alone.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217789 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-15 17:15:02 +00:00
Matt Arsenault
d189a0407d R600/SI: Add preliminary support for flat address space
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217777 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-15 15:41:53 +00:00
Matt Arsenault
ef4bb30475 R600/SI: Replace LDS atomics with no return versions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217379 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-08 15:07:31 +00:00
Matt Arsenault
f1cd7ce098 R600/SI: Add InstrMapping for noret atomics.
Only handles LDS atomics for now, and will be used
to replace atomics with no uses with the no return
versions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217378 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-08 15:07:27 +00:00
Matt Arsenault
324a7cd8be R600/SI: Fix register class for some 64-bit atomics
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217323 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-07 00:46:20 +00:00
Matt Arsenault
89a7e3ec3e R600/SI: Use same complex patterns for DS atomics
This fixes hitting the same negative base offset problem
that was already fixed for regular loads and stores.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217256 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-05 16:24:58 +00:00
Tom Stellard
7cda2d0666 R600/SI: Use S_ADD_U32 and S_SUB_U32 for low half of 64-bit operations
https://bugs.freedesktop.org/show_bug.cgi?id=83416

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217248 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-05 14:07:59 +00:00
Matt Arsenault
fa2e31c394 R600/SI: Un-move pattern I forgot to remove in last commit
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217109 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-03 23:28:57 +00:00
Matt Arsenault
c9cc488dfe R600/SI: Try to keep i32 mul on SALU
Also fix bug this exposed where when legalizing an immediate
operand, a v_mov_b32 would be created with a VSrc dest register.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217108 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-03 23:24:35 +00:00
Tom Stellard
ce4caf146f R600/SI: Add a pattern for i64 and in a branch
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217041 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-03 15:22:41 +00:00
Matt Arsenault
f4d57e7874 R600/SI: Use mad for fsub + fmul
We can use a negate source modifier to match
this for fsub.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@216735 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-29 16:01:14 +00:00
Tom Stellard
f50f927d65 R600/SI: Use READ2/WRITE2 instructions for 64-bit mem ops with 32-bit alignment
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@216279 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-22 18:49:35 +00:00
Tom Stellard
ec4cb3346d R600/SI: Use a ComplexPattern for DS loads and stores
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@216278 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-22 18:49:33 +00:00
Tom Stellard
9b60cb102a R600/SI: Use eliminateFrameIndex() to expand SGPR spill pseudos
This will simplify the SGPR spilling and also allow us to use
MachineFrameInfo for calculating offsets, which should be more
reliable than our custom code.

This fixes a crash in some cases where a register would be spilled
in a branch such that the VGPR defined for spilling did not dominate
all the uses when restoring.

This fixes a crash in an ocl conformance test.  The test requries
register spilling and is too big to include.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@216217 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-21 20:40:54 +00:00
Matt Arsenault
c86e55eb6e R600/SI: Move all fabs / fneg handling to patterns
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215749 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-15 18:42:22 +00:00
Matt Arsenault
0498d07255 R600/SI: Use source modifiers for f64 fneg
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215748 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-15 18:42:18 +00:00
Matt Arsenault
b5cb5e29a7 R600/SI: Refactor fneg / fabs patterns
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215746 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-15 18:42:11 +00:00
Matt Arsenault
5bc44c7603 R600/SI: Add intrinsic for ldexp
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215734 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-15 17:30:25 +00:00
Tom Stellard
f56c55d003 R600/SI: Add an _OFFEN variant MUBUF_STORE_* and use it for scratch writes
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215398 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-11 22:18:14 +00:00
Matt Arsenault
f1ca953dc6 R600/SI: Add definitions for ds_read2st64_ / ds_write2st64_
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214936 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-05 23:53:20 +00:00
Matt Arsenault
fa76ba82f2 R600/SI: Fix definitions for ds_read2 / ds_write2 instructions.
These were just wrong, using the wrong register classes
and store2 was missing an operand.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214756 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-04 18:49:22 +00:00
Tom Stellard
17c8fefc9f R600/SI: Do abs/neg folding with ComplexPatterns
Abs/neg folding has moved out of foldOperands and into the instruction
selection phase using complex patterns.  As a consequence of this
change, we now prefer to select the 64-bit encoding for most
instructions and the modifier operands have been dropped from
integer VOP3 instructions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214467 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-01 00:32:39 +00:00
Matt Arsenault
f3e7778454 R600/SI: Remove redundant setting of bits on instructions.
neverHasSideEffects is deprecated, and hasSideEffects = 0 is already
set on the base classes of the basic ALU instruction classes. The
base classes also already set mayLoad = 0 and mayStore = 0

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214283 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-30 03:18:57 +00:00
Tom Stellard
3280804237 R600/SI: Use scratch memory for large private arrays
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213551 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-21 15:45:01 +00:00
Tom Stellard
59b8363f8a R600/SI: Remove vaddr operand from BUFFER_LOAD_*_OFFSET instructions
This operand is never used.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213549 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-21 15:44:55 +00:00
Tom Stellard
b664d47cb0 R600/SI: Store constant initializer data in constant memory
This implements a solution for constant initializers suggested
by Vadim Girlin, where we store the data after the shader code
and then use the S_GETPC instruction to compute its address.

This saves use the trouble of creating a new buffer for constant data
and then having to pass the pointer to the kernel via user SGPRs or the
input buffer.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213530 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-21 14:01:14 +00:00
Tom Stellard
b97240dba8 R600/SI: Add isCFDepth0 Predicate to SALU addc pattern
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213529 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-21 14:01:12 +00:00
Tom Stellard
54a2540fee R600/SI: Use VALU for i1 XOR
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213528 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-21 14:01:10 +00:00
Tom Stellard
ad9769a118 R600/SI: Use a custom encoding method for simm16 in SOPP branch instructions
This allows us to explicitly define the type of fixup that is needed,
so we can distinguish this from future fixup types.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213527 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-21 14:01:08 +00:00
Tom Stellard
efb733cbba R600/SI: Rename SOPP operands to match the encoding fields
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213526 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-21 14:01:05 +00:00
Matt Arsenault
18ecf3fff3 R600/SI: implement range reduction for sin/cos
These instructions can only take a limited input range, and return
the constant value 1 out of range. We should do range reduction to
be able to process arbitrary values. Use a FRACT instruction after
normalization to achieve this. Also add a test for constant folding
with the lowered code with unsafe-fp-math enabled.

v2: use DAG lowering instead of intrinsic, adapt test
v3: calculate constant, fold pattern into instruction definition
v4: misc style fixes, add sin-fold testcase, cosmetics

Patch by Grigori Goronzy

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213458 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-19 18:44:39 +00:00
Tim Northover
3e61ccdded CodeGen: extend f16 conversions to permit types > float.
This makes the two intrinsics @llvm.convert.from.f16 and
@llvm.convert.to.f16 accept types other than simple "float". This is
only strictly needed for the truncate operation, since otherwise
double rounding occurs and there's no way to represent the strict IEEE
conversion. However, for symmetry we allow larger types in the extend
too.

During legalization, we can expand an "fp16_to_double" operation into
two extends for convenience, but abort when the truncate isn't legal. A new
libcall is probably needed here.

Even after this commit, various target tweaks are needed to actually use the
extended intrinsics. I've put these into separate commits for clarity, so there
are no actual tests of f64 conversion here.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213248 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-17 10:51:23 +00:00
Matt Arsenault
a27bf373d9 R600/SI: Allow using f32 rcp / rsq when denormals not handled.
These are precise enough to use for OpenCL unless denormals
are handled.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213107 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-15 23:50:10 +00:00