Commit Graph

568 Commits

Author SHA1 Message Date
Matt Arsenault
add2e2ec8f R600/SI: Fix extra mov from legalizing 64-bit SALU ops.
Check the register class of each operand individually
to avoid an extra copy to a vgpr.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204662 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-24 20:08:13 +00:00
Matt Arsenault
3a96e61469 R600/SI: Sub-optimial fix for 64-bit immediates with SALU ops.
No longer asserts, but now you get moves loading legal immediates
into the split 32-bit operations.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204661 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-24 20:08:09 +00:00
Matt Arsenault
db1807144a R600/SI: Fix 64-bit bit ops that require the VALU.
Try to match scalar and first like the other instructions.
Expand 64-bit ands to a pair of 32-bit ands since that is not
available on the VALU.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204660 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-24 20:08:05 +00:00
Matt Arsenault
6c199d8212 R600: Implement isNarrowingProfitable.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204658 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-24 19:43:31 +00:00
Matt Arsenault
875870fdb4 R600/SI: Fix 64-bit private loads.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204630 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-24 17:50:46 +00:00
Matt Arsenault
55d17f4842 R600/SI: Move instruction patterns to scalar versions.
Some of them also had the pattern on both, so this removes the
duplication.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204492 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-21 18:01:18 +00:00
Tom Stellard
a1d28f6dd7 R600/SI: Handle MUBUF instructions in SIInstrInfo::moveToVALU()
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204476 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-21 15:51:57 +00:00
Tom Stellard
1f1c0495d0 R600/SI: Handle S_MOV_B64 in SIInstrInfo::moveToVALU()
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204475 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-21 15:51:54 +00:00
Matt Arsenault
e3620da269 R600/SI: Add support for 64-bit LDS writes
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204274 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-19 22:19:54 +00:00
Matt Arsenault
62b3e22092 R600/SI: Add support for 64-bit LDS loads.
v2:
  -Use correct opcode for DS_READ_64

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204273 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-19 22:19:52 +00:00
Matt Arsenault
6eaa49233f R600/SI: Match i16 immediate offset of LDS instructions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204272 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-19 22:19:49 +00:00
Matt Arsenault
adf5141ecd R600/SI: Fix test checking wrong instruction operand.
The source and destination happen to be the same register.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204271 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-19 22:19:45 +00:00
Matt Arsenault
9c0b2d08d3 R600/SI: Don't display the GDS bit.
It isn't actually used now, and probably never will be, plus it makes
tests less annoying. I also think SC prints GDS instructions as a
separate instruction name.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204270 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-19 22:19:43 +00:00
NAKAMURA Takumi
3bdef4b6dc CodeGen/R600/v_cndmask.ll: Relax an expression to unbreak msvcrt.
V_CNDMASK_B32_e64 v0, v0, -1.#QNAN0e+00, s[2:3], 0, 0, 0, 0

FIXME: We really need to implement our formatter...

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204118 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-18 06:17:22 +00:00
Kevin Enderby
c153e49aa9 Making a guess to fix the test case with r204056 to get the build bot working.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204073 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-17 19:00:03 +00:00
Matt Arsenault
2683baa8ac R600: Match sign_extend_inreg to BFE instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204072 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-17 18:58:11 +00:00
Tom Stellard
ad52f4f70c R600/SI: Fix implementation of isInlineConstant() used by the verifier
The type of the immediates should not matter as long as the encoding is
equivalent to the encoding of one of the legal inline constants.

Tested-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204056 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-17 17:03:52 +00:00
Tom Stellard
eb7876083d R600/SI: Use correct dest register class for V_READFIRSTLANE_B32
This instructions writes to an 32-bit SGPR.  This change required adding
the 32-bit VCC_LO and VCC_HI registers, because the full VCC register
is 64 bits.

This fixes verifier errors on several of the indirect addressing piglit
tests.

Tested-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204055 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-17 17:03:51 +00:00
Tom Stellard
47feea0802 R600: LDS instructions shouldn't implicitly define OQAP
LDS instructions are pseudo instructions which model
the OQAP defs and uses within a single instruction.

This fixes a hang in the opencv MedianFilter tests.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@203818 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-13 17:13:04 +00:00
Matt Arsenault
054f4eccd2 R600: Fix trunc store from i64 to i1
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@203695 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-12 18:45:52 +00:00
Tom Stellard
6cadd406cc R600/SI: Using SGPRs is illegal for instructions that read carry-out from VCC
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@203281 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-07 20:12:39 +00:00
Tom Stellard
7e06370873 R600/SI: Custom lower i1 stores
These are sometimes created by the shrink to boolean optimization in the
globalopt pass.

Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@203280 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-07 20:12:33 +00:00
Matt Arsenault
161e3a80b2 R600: Fix extloads from i8 / i16 to i64.
This appears to only be working for global loads. Private
and local break for other reasons.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@203135 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-06 17:34:12 +00:00
Matt Arsenault
b4cd160bb9 R600/SI: Expand selects on vectors.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@203134 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-06 17:34:03 +00:00
Matt Arsenault
c59a9f09fb R600: Add failing control flow tests.
Simple cases hit a variety of problems at -O0.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202601 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-01 21:45:41 +00:00
Tom Stellard
9f0d68f522 R600/SI: Expand all v16[if]32 operations
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202543 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-28 21:36:37 +00:00
Michel Danzer
644aecfc97 R600/SI: Optimize SI_KILL for constant operands
If the SI_KILL operand is constant, we can either clear the exec mask if
the operand is negative, or do nothing otherwise.

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202337 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-27 01:47:09 +00:00
Michel Danzer
a5fbf24716 R600/SI: Allow SI_KILL for geometry shaders
Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202336 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-27 01:47:02 +00:00
Tom Stellard
d8c31046a9 R600/SI: Custom select 64-bit ADD
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202194 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-25 21:36:18 +00:00
Matt Arsenault
bc247e4afd R600/SI - Add new CI arithmetic instructions.
Does not yet include larger part required
to match v_mad_i64_i32 / v_mad_u64_u32.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202077 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-24 21:01:28 +00:00
Quentin Colombet
a6e734d2ed [CodeGenPrepare] Fix the check of the legality of an instruction.
The API expects an ISD opcode, not an IR opcode.
Fixes a regression for R600.

Related to <rdar://problem/15519855>.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201923 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-22 01:06:41 +00:00
Nico Rieck
1800766f5a Fix more broken CHECK lines
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201493 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-16 13:28:39 +00:00
Quentin Colombet
038600b42e [CodeGenPrepare][AddressingModeMatcher] Give up on type promotion if the
transformation does not bring any immediate benefits and introduce an illegal
operation. 


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201439 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-14 22:23:22 +00:00
Tom Stellard
5f80594797 TargetLowering: n * r where n > 2 should be an illegal addressing mode
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201433 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-14 21:10:34 +00:00
Tom Stellard
0f2e653317 R600/SI: Expand all v8[if]32 operations
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201371 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-13 23:34:15 +00:00
Tom Stellard
9757ba1206 R600/SI: Add a pattern for i32 anyext
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201370 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-13 23:34:13 +00:00
Tom Stellard
130f271259 R600/SI: Completely Disable TypeRewriter on compute
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201369 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-13 23:34:12 +00:00
Tom Stellard
1a52c2b25a R600/SI: Split global vector loads with more than 4 elements
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201368 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-13 23:34:10 +00:00
Tom Stellard
c4fea3594f R600/SI: Add ShaderType attribute to some tests
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201367 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-13 23:34:07 +00:00
Matt Arsenault
69bc4ddf10 R600/SI: Fix assertion on infinite loops.
This isn't the most useful case to fix in the real world,
but bugpoint runs into this.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201177 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-11 21:12:38 +00:00
Tom Stellard
42faffddb9 R600/SI: Initialize M0 and emit S_WQM_B64 whenever DS instructions are used
DS instructions that access local memory can only uses addresses that
are less than or equal to the value of M0.  When M0 is uninitialized,
then we experience undefined behavior.

This patch also changes the behavior to emit S_WQM_B64 on pixel shaders
no matter what kind of DS instruction is used.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201097 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-10 16:58:30 +00:00
Matt Arsenault
26bc2c8eed R600/SI: Add failing test for 3 x i64 vectors.
Stores of <4 x i64> do work (although they do expand to 4 stores
instead of 2), but 3 x i64 vectors fail to select.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200989 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-07 20:29:40 +00:00
Tom Stellard
226bd876c6 R600/SI: Add a MUBUF store pattern for Reg+Imm offsets
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200935 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-06 18:36:41 +00:00
Tom Stellard
603cd56372 R600/SI: Add a MUBUF store pattern for Imm offsets
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200934 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-06 18:36:39 +00:00
Tom Stellard
becac0f183 R600/SI: Add a MUBUF load pattern for Reg+Imm offsets
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200933 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-06 18:36:38 +00:00
Tom Stellard
22274378d5 R600/SI: Use immediates offsets for SMRD instructions whenever possible
There was a problem with the old pattern, so we were copying some
larger immediates into registers when we could have been encoding
them in the instruction.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200932 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-06 18:36:34 +00:00
Michel Danzer
cf4061a601 R600/SI: Add pattern for zero-extending i1 to i32
Fixes opencl-example if_* tests with radeonsi.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=74469

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200830 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-05 09:48:05 +00:00
Tom Stellard
83918a2ad2 R600/SI: Custom lower i64 ISD::SELECT
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200774 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-04 17:18:40 +00:00
Tom Stellard
4ee42eaec9 R600: Enable vector fpow.
The OpenCL specs say: "The vector versions of the math functions operate
component-wise. The description is per-component."

Patch by: Jan Vesely

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>
Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200773 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-04 17:18:37 +00:00
Michel Danzer
de2e0bc366 R600/SI: Fix fneg for 0.0
V_ADD_F32 with source modifier does not produce -0.0 for this. Just
manipulate the sign bit directly instead.

Also add a pattern for (fneg (fabs ...)).

Fixes a bunch of bit encoding piglit tests with radeonsi.

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200743 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-04 07:12:38 +00:00
Matt Arsenault
1bdadf38ef Add some xfailed R600 tests for 64-bit private accesses.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200620 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-02 00:13:12 +00:00
Matt Arsenault
b2abb9752e R600/SI: Fix insertelement with dynamic indices.
This didn't work for any integer vectors, and didn't
work with some sizes of float vectors. This should now
work with all sizes of float and i32 vectors.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200619 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-02 00:05:35 +00:00
Michel Danzer
f555fb410e R600/SI: Add pattern for truncating i32 to i1
Fixes half a dozen piglit tests with radeonsi.

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200283 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-28 03:01:16 +00:00
Michel Danzer
7018cd5af7 R600/SI: Add intrinsic for BUFFER_LOAD_DWORD* instructions
Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200196 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-27 07:20:51 +00:00
Michel Danzer
b3bfe7f18c R600/SI: Add intrinsic for S_SENDMSG instruction
Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200195 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-27 07:20:44 +00:00
Tom Stellard
35f321dde5 R600: Disable the BFE pattern
This pattern uses an SDNodeXForm, which isn't being emitted for some
reason.  I can get it to work by attaching the PatLeaf that has the
XForm to the argument in the output pattern, but this results in an
immediate being used in a register operand, which the backend can't
handle yet.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199918 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-23 18:49:33 +00:00
Tom Stellard
cbf79028c3 R600: Correctly handle vertex fetch clauses the precede ENDIFs
The control flow finalizer would sometimes use an ALU_POP_AFTER
instruction before the vetex fetch clause instead of using a POP
instruction after it.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199917 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-23 18:49:31 +00:00
Tom Stellard
01df2fa3c4 R600: Unconditionally unroll loops that contain GEPs with alloca pointers
Implement the getUnrollingPreferences() function for
AMDGPUTargetTransformInfo so that loops that do address calculations
on pointers derived from alloca are unconditionally unrolled.

Unrolling these loops makes it more likely that SROA will be able to
eliminate the allocas, which is a big win for R600 since memory
allocated by alloca (private memory) is really slow.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199916 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-23 18:49:28 +00:00
Tom Stellard
7d3b9d96b6 R600: Recommit 199842: Add work-around for the CF stack entry HW bug
The unit test is now disabled on non-asserts builds.

The CF stack can be corrupted if you use CF_ALU_PUSH_BEFORE,
CF_ALU_ELSE_AFTER, CF_ALU_BREAK, or CF_ALU_CONTINUE when the number of
sub-entries on the stack is greater than or equal to the stack entry
size and sub-entries modulo 4 is either 0 or 3 (on cedar the bug is
present when number of sub-entries module 8 is either 7 or 0)

We choose to be conservative and always apply the work-around when the
number of sub-enries is greater than or equal to the stack entry size,
so that we can safely over-allocate the stack when we are unsure of the
stack allocation rules.

reviewed-by: Vincent Lejeune <vljn at ovi.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199905 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-23 16:18:02 +00:00
Tom Stellard
8a4f11e3b6 Revert "R600: Add work-around for the CF stack entry HW bug"
This reverts commit 35b8331cad6eb512a2506adbc394201181da94ba.

The -debug-only flag for llc doesn't appear to be available in
all build configurations.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199845 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-22 22:20:54 +00:00
Tom Stellard
efa1355495 R600: Add work-around for the CF stack entry HW bug
The CF stack can be corrupted if you use CF_ALU_PUSH_BEFORE,
CF_ALU_ELSE_AFTER, CF_ALU_BREAK, or CF_ALU_CONTINUE when the number of
sub-entries on the stack is greater than or equal to the stack entry
size and sub-entries modulo 4 is either 0 or 3 (on cedar the bug is
present when number of sub-entries module 8 is either 7 or 0)

We choose to be conservative and always apply the work-around when the
number of sub-enries is greater than or equal to the stack entry size,
so that we can safely over-allocate the stack when we are unsure of the
stack allocation rules.

reviewed-by: Vincent Lejeune <vljn at ovi.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199842 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-22 21:55:46 +00:00
Tom Stellard
5c0c884e42 R600: Refactor stack size calculation
reviewed-by: Vincent Lejeune <vljn at ovi.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199840 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-22 21:55:43 +00:00
Tom Stellard
b1d24c51fc R600: MOVA is vector only
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199827 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-22 19:24:24 +00:00
Tom Stellard
e7d4e83702 R600: Take alignment into account when calculating the stack offset
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199826 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-22 19:24:23 +00:00
Tom Stellard
9c3e0ede1d R600: Add support for global addresses with constant initializers
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199825 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-22 19:24:21 +00:00
Tom Stellard
655ba251b5 R600: Begin private memory at the second GPR.
This way private memory does not over-write work group information
stored in GPRs 0 and 1.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199824 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-22 19:24:19 +00:00
Tom Stellard
7dd37ae57a R600/SI: Add support for i8 and i16 private loads/stores
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199823 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-22 19:24:14 +00:00
Benjamin Kramer
ccdb9c9483 Fix broken CHECK lines.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199016 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-11 21:06:00 +00:00
Tom Stellard
37ee312f77 R600: Allow ftrunc
v2: Add ftrunc->TRUNC pattern instead of replacing int_AMDGPU_trunc
v3: move ftrunc pattern next to TRUNC definition, it's available since R600

Patch By: Jan Vesely

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>
Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@197783 91177308-0d34-0410-b5e6-96231b3b80d8
2013-12-20 05:11:55 +00:00
NAKAMURA Takumi
a2e3b51422 Add REQUIRES:asserts to 3 tests in llvm/test/CodeGen/R600 added in r192212.
They are failing in assertions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@197669 91177308-0d34-0410-b5e6-96231b3b80d8
2013-12-19 10:41:12 +00:00
Matt Arsenault
e2c28ad84f R600/SI: Make private pointers be 32-bit.
Different sized address spaces should theoretically work
most of the time now, and since 64-bit add is currently
disabled, using more 32-bit pointers fixes some cases.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@197659 91177308-0d34-0410-b5e6-96231b3b80d8
2013-12-19 05:32:55 +00:00
Matt Arsenault
530c0b8862 R600/SI: Minor improvements to test.
Use CHECK-LABEL, add an i64 version, check store instructions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@197293 91177308-0d34-0410-b5e6-96231b3b80d8
2013-12-14 00:38:04 +00:00
Matt Arsenault
bc54a09d20 R600/SI: Add i64 cmp tests
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@196960 91177308-0d34-0410-b5e6-96231b3b80d8
2013-12-10 21:11:55 +00:00
Vincent Lejeune
a563c91840 R600: Fix an infinite loop when trying to reorganize export/tex vector input
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@196923 91177308-0d34-0410-b5e6-96231b3b80d8
2013-12-10 14:43:31 +00:00
Vincent Lejeune
8ff689b443 R600: Fix input modifiers lost for Cayman
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@196922 91177308-0d34-0410-b5e6-96231b3b80d8
2013-12-10 14:43:27 +00:00
Vincent Lejeune
d254d3111e Add a RequireStructuredCFG Field to TargetMachine.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@196634 91177308-0d34-0410-b5e6-96231b3b80d8
2013-12-07 01:49:19 +00:00
Matt Arsenault
87234703e8 R600/SI: Add comments for number of used registers.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@196467 91177308-0d34-0410-b5e6-96231b3b80d8
2013-12-05 05:15:35 +00:00
Vincent Lejeune
7043c7a35e R600: Workaround for cayman loop bug
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@196121 91177308-0d34-0410-b5e6-96231b3b80d8
2013-12-02 17:29:37 +00:00
Tom Stellard
8a6b7df6f8 R600: Expand vector FABS
NOTE: This is a candidate for the 3.4 branch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195881 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-27 21:23:39 +00:00
Tom Stellard
aa6ec15caf R600/SI: Implement spilling of SGPRs v5
SGPRs are spilled into VGPRs using the {READ,WRITE}LANE_B32 instructions.

v2:
  - Fix encoding of Lane Mask
  - Use correct register flags, so we don't overwrite the low dword
    when restoring multi-dword registers.

v3:
  - Register spilling seems to hang the GPU, so replace all shaders
    that need spilling with a dummy shader.

v4:
  - Fix *LANE definitions
  - Change destination reg class for 32-bit SMRD instructions

v5:
  - Remove small optimization that was crashing Serious Sam 3.

https://bugs.freedesktop.org/show_bug.cgi?id=68224
https://bugs.freedesktop.org/show_bug.cgi?id=71285

NOTE: This is a candidate for the 3.4 branch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195880 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-27 21:23:35 +00:00
Tom Stellard
0cbf943733 R600/SI: Use SGPR_32 register class for 32-bit SMRD outputs
Writing to the M0 register from an SMRD instruction hangs the GPU, so
we need to use the SGPR_32 register class, which does not include M0.

NOTE: This is a candidate for the 3.4 branch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195879 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-27 21:23:29 +00:00
Tom Stellard
496dbfe7b9 R600: Add support for ISD::FROUND
NOTE: This is a candidate for the 3.4 branch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195878 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-27 21:23:20 +00:00
Tom Stellard
0f39827340 R600/SI: Fixing handling of condition codes
We were ignoring the ordered/onordered bits and also the signed/unsigned
bits of condition codes when lowering the DAG to MachineInstrs.

NOTE: This is a candidate for the 3.4 branch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195514 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-22 23:07:58 +00:00
Tom Stellard
0ffcaa0d54 SelectionDAG: Optimize expansion of vec_type = BITCAST scalar_type
The legalizer can now do this type of expansion for more
type combinations without loading and storing to and
from the stack.

NOTE: This is a candidate for the 3.4 branch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195398 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-22 00:41:05 +00:00
Matt Arsenault
3560dd2dcd R600/SI: Fix moveToVALU when the first operand is VSrc.
Moving into a VSrc doesn't always work, since it could be
replaced with an SGPR later.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195042 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-18 20:09:55 +00:00
Matt Arsenault
9bc4b2c0da R600/SI: Fix multiple SGPR reads when using VCC.
No other SGPR operands are allowed, so if VCC is
used, move the other to a VGPR.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195041 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-18 20:09:50 +00:00
Matt Arsenault
e64a289609 R600/SI: Implement add i64, but do not yet enable.
Test doesn't actually check the output. I need
to fix add i64 being matched for the addressing
calculations.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195040 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-18 20:09:47 +00:00
Matt Arsenault
3e38856f04 R600/SI: Move patterns to match add / sub to scalar instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195034 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-18 20:09:29 +00:00
Tom Stellard
b570599c8f R600: Enable the IR structurizer by default
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195031 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-18 19:43:44 +00:00
Tom Stellard
44248e0f63 R600: Fix a crash in the AMDILCFGStrucurizer
The ifPatternMatch() function was not correctly reporting the number
of matches in some cases.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195030 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-18 19:43:38 +00:00
Tom Stellard
204c953cd5 R600/SI: Fix illegal VGPR->SGPR copy inside of loop
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195026 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-18 18:50:20 +00:00
Tom Stellard
bf9ddd5e8f R600/SI: Fix another case of illegal VGPR->SGPR copy
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195025 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-18 18:50:15 +00:00
Matt Arsenault
94437c9691 Use right address space pointer size
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194940 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-17 00:06:39 +00:00
Matt Arsenault
e6e811277f Fix assert on unaligned access to global with different address space size.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194934 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-16 20:50:54 +00:00
Matt Arsenault
4fe5b640ee Fix codegen for null different sized pointer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194932 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-16 20:24:41 +00:00
Vincent Lejeune
4110797853 R600: Make dot_4 instructions predicable
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194927 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-16 16:24:41 +00:00
Tom Stellard
5cddda6d13 R600/SI: Add VReg_96 register class to SIRegisterInfo::hasVGPRs()
This fixes a crash with GNOME settings manager.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194836 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-15 18:26:45 +00:00
Matt Arsenault
509a492442 Add target hook to prevent folding some bitcasted loads.
This is to avoid this transformation in some cases:
fold (conv (load x)) -> (load (conv*)x)

On architectures that don't natively support some vector
loads efficiently casting the load to a smaller vector of
larger types and loading is more efficient.

Patch by Micah Villmow.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194783 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-15 04:42:23 +00:00
Tom Stellard
19a99df130 R600: Fix scheduling of instructions that use the LDS output queue
The LDS output queue is accessed via the OQAP register.  The OQAP
register cannot be live across clauses, so if value is written to the
output queue, it must be retrieved before the end of the clause.
With the machine scheduler, we cannot statisfy this constraint, because
it lacks proper alias analysis and it will mark some LDS accesses as
having a chain dependency on vertex fetches.  Since vertex fetches
require a new clauses, the dependency may end up spiltting OQAP uses and
defs so the end up in different clauses.  See the lds-output-queue.ll
test for a more detailed explanation.

To work around this issue, we now combine the LDS read and the OQAP
copy into one instruction and expand it after register allocation.

This patch also adds some checks to the EmitClauseMarker pass, so that
it doesn't end a clause with a value still in the output queue and
removes AR.X and OQAP handling from the scheduler (AR.X uses and defs
were already being expanded post-RA, so the scheduler will never see
them).

Reviewed-by: Vincent Lejeune <vljn at ovi.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194755 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-15 00:12:45 +00:00
Matt Arsenault
e0375295c5 R600/SI: Add testcase for problem I ran into
with the older version of the moveToVALU changes.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194682 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-14 07:57:29 +00:00
Tom Stellard
a2b4eb6d15 R600/SI: Add support for private address space load/store
Private address space is emulated using the register file with
MOVRELS and MOVRELD instructions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194626 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-13 23:36:50 +00:00
Tom Stellard
b52bf6a3b3 R600/SI: Prefer SALU instructions for bit shift operations
All shift operations will be selected as SALU instructions and then
if necessary lowered to VALU instructions in the SIFixSGPRCopies pass.

This allows us to do more operations on the SALU which will improve
performance and is also required for implementing private memory
using indirect addressing, since the private memory pointers must stay
in the scalar registers.

This patch includes some fixes from Matt Arsenault.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194625 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-13 23:36:37 +00:00
Matt Arsenault
29f1788de9 R600: Fix selection failure on EXTLOAD
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194547 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-13 02:39:07 +00:00
Matt Arsenault
86245071b5 R600/SI: Change formatting of printed registers.
Print the range of registers used with a single letter prefix.
This better matches what the shader compiler produces and
is overall less obnoxious than concatenating all of the
subregister names together.

Instead of SGPR0, it will print s0. Instead of SGPR0_SGPR1,
it will print s[0:1] and so on.

There doesn't appear to be a straightforward way
to get the actual register info in the InstPrinter,
so this parses the generated name to print with the
new syntax.

The required test changes are pretty nasty, and register
matching regexes are now worse. Since there isn't a way to
add to a variable in FileCheck, some of the tests now don't
check the exact number of registers used, but I don't think that
will be a real problem.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194443 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-12 02:35:51 +00:00
Matt Arsenault
e57a8a3d10 R600/SI: Add test that fails due to requiring i64 mul for pointers
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194433 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-11 23:31:02 +00:00
Vincent Lejeune
70a7d5ddb4 R600: Use function inputs to represent data stored in gpr
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194425 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-11 22:10:24 +00:00
Vincent Lejeune
69239a98b6 R600: Fix LowerUDIVREM
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194153 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-06 17:36:04 +00:00
Matt Arsenault
4f17f88071 Fix CodeGen for unaligned loads with address spaces
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193721 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-30 23:30:05 +00:00
Tom Stellard
aa1d078e7f R600: Custom lower f32 = uint_to_fp i64
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193701 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-30 17:22:05 +00:00
Tom Stellard
54328c772c R600/SI: Add compute support for CI v2
v2:
  - Fix LDS size calculation

Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193621 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-29 16:37:28 +00:00
Tom Stellard
f54a8409f9 R600: Expand vector FSQRT ops
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193620 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-29 16:37:20 +00:00
Tom Stellard
96b5670cf4 R600/SI: fix MIMG writemask adjustement
This fixes piglit:
- shaders/glsl-fs-texture2d-masked
- shaders/glsl-fs-texture2d-masked-4

Patch by: Marek Olšák

Signed-off-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193222 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-23 02:53:47 +00:00
Tom Stellard
f95b162188 R600: Fix handling of vector kernel arguments
The SelectionDAGBuilder was promoting vector kernel arguments to legal
types, but this won't work for R600 and SI since kernel arguments are
stored in memory and can't be promoted.  In order to handle vector
arguments correctly we need to look at the original types from the LLVM IR
function.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193215 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-23 00:44:32 +00:00
Tom Stellard
6a2f9b9137 R600/SI: Add support for i64 bitwise or
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193213 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-23 00:44:19 +00:00
Tom Stellard
f9e5c39811 R600/SI: Use S_LOAD_DWORD instructions for v8i32 and v16i32
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193212 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-23 00:44:12 +00:00
Tom Stellard
04c559569f R600: Simplify handling of private address space
The AMDGPUIndirectAddressing pass was previously responsible for
lowering private loads and stores to indirect addressing instructions.
However, this pass was buggy and way too complicated.  The only
advantage it had over the new simplified code was that it saved one
instruction per direct write to private memory.  This optimization
likely has a minimal impact on performance, and we may be able
to duplicate it using some other transformation.

For the private address space, we now:
1. Lower private loads/store to Register(Load|Store) instructions
2. Reserve part of the register file as 'private memory'
3. After regalloc lower the Register(Load|Store) instructions to
   MOV instructions that use indirect addressing.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193179 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-22 18:19:10 +00:00
Matt Arsenault
2d7d477d94 Fix CodeGen for vectors of pointers with address spaces.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193112 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-21 20:03:58 +00:00
Matt Arsenault
ff71812dfa Fix CodeGen for different size address space GEPs
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193111 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-21 20:03:54 +00:00
Tom Stellard
a4f468f245 R600: Fix a crash in the AMDILCFGStructurizer
We were calling llvm_unreachable() when failing to optimize the
branch into if case.  However, it is still possible for us
to structurize the CFG by duplicating blocks even if this optimization
fails.

Reviewed-by: Vincent Lejeune<vljn at ovi.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192813 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-16 17:06:02 +00:00
Vincent Lejeune
cf1f4c7dd1 R600: improve dump of S_WAITCNT
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192557 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-13 17:56:28 +00:00
Vincent Lejeune
f2b3a569ae R600: Use masked read sel for texture instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192554 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-13 17:56:10 +00:00
Vincent Lejeune
91ec4b0cac R600: fix swizzle export
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192553 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-13 17:56:04 +00:00
Matt Arsenault
b19b474de9 R600: Add scalar i32 add test
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192501 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-11 21:03:41 +00:00
Matt Arsenault
2ad612a9e7 Use CHECK-LABEL
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192500 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-11 21:03:39 +00:00
Matt Arsenault
1cc41bf63c R600: Fix trunc i64 to i32 on SI
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192375 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-10 18:04:16 +00:00
Tom Stellard
3986785046 R600/SI: Use -verify-machineinstrs for most tests
We can't enable the verifier for tests with SI_IF and SI_ELSE, because
these instructions are always followed by a COPY which copies their
result to the next basic block.  This violates the machine verifier's
rule that non-terminators can not folow terminators.

Reviewed-by: Vincent Lejeune<vljn at ovi.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192366 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-10 17:11:46 +00:00
Matt Arsenault
194d437f11 Add some xfaild R600 tests.
These are bugs to fix later.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192212 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-08 18:06:36 +00:00
Vincent Lejeune
dfef7cbfc6 R600: add a pass that merges clauses.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191790 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-01 19:32:58 +00:00
Vincent Lejeune
c6c37d74a2 R600: Put PRED_X instruction in its own clause
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191789 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-01 19:32:49 +00:00
Vincent Lejeune
5b00e833fa R600: Enable -verify-machineinstrs in some tests.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191788 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-01 19:32:38 +00:00
Manman Ren
e267f04ef5 TBAA: update tbaa format from scalar format to struct-path aware format.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191690 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-30 18:17:55 +00:00
Manman Ren
aef1b37824 TBAA: remove !tbaa from testing cases when they are not needed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191689 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-30 18:17:35 +00:00
Tom Stellard
9c598cfebc R600: Fix handling of NAN in comparison instructions
We were completely ignoring the unorder/ordered attributes of condition
codes and also incorrectly lowering seto and setuo.

Reviewed-by: Vincent Lejeune<vljn at ovi.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191603 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-28 02:50:50 +00:00
Vincent Lejeune
f57d692c11 R600: Move code handling literal folding into R600ISelLowering.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190644 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-12 23:44:53 +00:00
Vincent Lejeune
fe78318614 R600: Move fabs/fneg/sel folding logic into PostProcessIsel
This move makes possible to correctly handle multiples instructions
from a single pattern.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190643 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-12 23:44:44 +00:00
Tom Stellard
a3c2bcf0ee R600/SI: expose TBUFFER_STORE_FORMAT_* for OpenGL transform feedback
For _XYZ, the type of VDATA is v4i32, because v3i32 doesn't exist.

The ADDR64 bit is not exposed. A simpler intrinsic that doesn't take
a resource descriptor might be nicer.

The maximum number of input SGPRs is bumped to 17.

Signed-off-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190575 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-12 02:55:14 +00:00
Aaron Watry
546dcc5ddc R600: Add support for LDS atomic subtract
Signed-off-by: Aaron Watry <awatry@gmail.com>
Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190200 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-06 20:17:42 +00:00
Matt Arsenault
ce8e4647bf Teach CodeGenPrepare about address spaces
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190112 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-06 00:18:43 +00:00
Matt Arsenault
3987e532f7 R600: Fix i64 to i32 trunc on SI
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190091 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-05 19:41:10 +00:00
Tom Stellard
79916948e1 R600: Add support for local memory atomic add
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190080 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-05 18:38:09 +00:00
Tom Stellard
756f382ac1 R600: Expand SELECT nodes rather than custom lowering them
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190079 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-05 18:38:03 +00:00
Tom Stellard
470c451574 R600: Fix incorrect LDS size calculation
GlobalAdderss nodes that appeared in more than one basic block were
being counted twice.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190078 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-05 18:37:57 +00:00
Tom Stellard
402b8e2175 R600/SI: Don't emit S_WQM_B64 instruction for compute shaders
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190077 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-05 18:37:52 +00:00
Vincent Lejeune
f94eea9e11 R600: Use shared op optimization when checking cycle compatibility
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189981 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-04 19:53:54 +00:00
Vincent Lejeune
bb25a01d23 R600: Non vector only instruction can be scheduled on trans unit
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189980 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-04 19:53:46 +00:00
Vincent Lejeune
09c52189ce R600: Remove fmul.v4f32.ll test which is redundant with fmul.ll
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189978 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-04 19:53:22 +00:00
Michel Danzer
845ad06efb R600/SI: Enable local-memory-two-objects lit test
Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189334 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-27 10:28:26 +00:00
Tom Stellard
edd08f7428 SelectionDAG: Remove unnecessary uses of TargetLowering::getPointerTy()
If we have a binary operation like ISD:ADD, we can set the result type
equal to the result type of one of its operands rather than using
TargetLowering::getPointerTy().

Also, any use of DAG.getIntPtrConstant(C) as an operand for a binary
operation can be replaced with:
DAG.getConstant(C, OtherOperand.getValueType());

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189227 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-26 15:06:10 +00:00
Tom Stellard
d08a930361 R600: Add support for vector local memory loads
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189226 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-26 15:06:04 +00:00
Tom Stellard
a01cdea9c6 R600: Add support for i8 and i16 local memory loads
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189225 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-26 15:05:59 +00:00
Tom Stellard
c52565157d SelectionDAG: Use correct pointer size when splitting vector stores
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189224 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-26 15:05:55 +00:00
Tom Stellard
8e78012457 R600: Add support for i8 and i16 local memory stores
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189223 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-26 15:05:49 +00:00
Tom Stellard
7a0282daeb R600: Add support for v4i32 and v2i32 local stores
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189222 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-26 15:05:44 +00:00
Tom Stellard
da25cd3e6d SelectionDAG: Use correct pointer size when lowering function arguments v2
This adds minimal support to the SelectionDAG for handling address spaces
with different pointer sizes.  The SelectionDAG should now correctly
lower pointer function arguments to the correct size as well as generate
the correct code when lowering getelementptr.

This patch also updates the R600 DataLayout to use 32-bit pointers for
the local address space.

v2:
  - Add more helper functions to TargetLoweringBase
  - Use CHECK-LABEL for tests

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189221 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-26 15:05:36 +00:00
Bill Wendling
80075c482c Update to remove the no-frame-pointer-elim-non-leaf flag if it was set to 'false'.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189068 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-22 21:28:54 +00:00
Tom Stellard
344cfb4db4 R600/SI: Fix another case of illegal VGPR to SGPR copy
This fixes a crash in Unigine Tropics.

https://bugs.freedesktop.org/show_bug.cgi?id=68389

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189057 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-22 20:21:02 +00:00
Tom Stellard
d00968a7a5 SelectionDAG: Make sure stores are always added to the LegalizedNodes list
When truncated vector stores were being custom lowered in
VectorLegalizer::LegalizeOp(), the old (illegal) and new (legal) node pair
was not being added to LegalizedNodes list.  Instead of the legalized
result being passed to VectorLegalizer::TranslateLegalizeResult(),
the result was being passed back into VectorLegalizer::LegalizeOp(),
which ended up adding a (new, new) pair to the list instead.

This was causing an assertion failure when a custom lowered truncated
vector store was the last instruction a basic block and the VectorLegalizer
was unable to find it in the LegalizedNodes list when updating the
DAG root.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188953 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-21 22:42:58 +00:00
Tom Stellard
3cae823f69 R600: Expand vector FRINT ops
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188598 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-16 23:51:33 +00:00
Tom Stellard
84c0bd9803 R600: Expand vector FFLOOR ops
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188597 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-16 23:51:29 +00:00
Tom Stellard
0991c314d7 R600: Expand vector float operations for both SI and R600
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188596 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-16 23:51:24 +00:00
Michel Danzer
b89f197713 R600/SI: Add pattern for xor of i1
Fixes two recent piglit regressions with radeonsi.

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188559 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-16 16:19:31 +00:00
Michel Danzer
ebd4eec538 R600/SI: Fix broken encoding of DS_WRITE_B32
The logic in SIInsertWaits::getHwCounts() only really made sense for SMRD
instructions, and trying to shoehorn it into handling DS_WRITE_B32 caused
it to corrupt the encoding of that by clobbering the first operand with
the second one.

Undo that damage and only apply the SMRD logic to that.

Fixes some derivates related piglit regressions with radeonsi.

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188558 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-16 16:19:24 +00:00
Tom Stellard
b07ec96068 Revert "R600/SI: Fix incorrect encoding of DS_WRITE_B32 instructions"
This reverts commit a6a39ced09.
This is the wrong version of this fix.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188523 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-16 01:18:43 +00:00
Tom Stellard
a6a39ced09 R600/SI: Fix incorrect encoding of DS_WRITE_B32 instructions
The SIInsertWaits pass was overwriting the first operand (gds bit) of
DS_WRITE_B32 with the second operand (value to write).  This meant that
any time the value to write was stored in an odd number VGPR, the gds
bit would be set causing the instruction to write to GDS instead of LDS.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188522 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-16 01:12:20 +00:00
Tom Stellard
30d84d8dfa R600: Add support for global vector loads with element types less than 32-bits
Tested-by: Aaron Watry <awatry@gmail.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188521 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-16 01:12:16 +00:00
Tom Stellard
4c52d450dc R600: Add support for global vector stores with elements less than 32-bits
Tested-by: Aaron Watry <awatry@gmail.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188520 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-16 01:12:11 +00:00
Tom Stellard
ec484277dd R600: Add support for i16 and i8 global stores
Tested-by: Aaron Watry <awatry@gmail.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188519 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-16 01:12:06 +00:00
Tom Stellard
4781d314b7 R600: Add support for v4i32 stores on Cayman
Tested-by: Aaron Watry <awatry@gmail.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188518 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-16 01:12:00 +00:00
Tom Stellard
6b88cdb34c R600: Enable folding of inline literals into REQ_SEQUENCE instructions
Tested-by: Aaron Watry <awatry@gmail.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188517 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-16 01:11:55 +00:00
Tom Stellard
e560d526a1 R600: Change the RAT instruction assembly names so they match the docs
Tested-by: Aaron Watry <awatry@gmail.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188515 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-16 01:11:46 +00:00
Daniel Dunbar
24ec2e5a72 [tests] Cleanup initialization of test suffixes.
- Instead of setting the suffixes in a bunch of places, just set one master
   list in the top-level config. We now only modify the suffix list in a few
   suites that have one particular unique suffix (.ml, .mc, .yaml, .td, .py).

 - Aside from removing the need for a bunch of lit.local.cfg files, this enables
   4 tests that were inadvertently being skipped (one in
   Transforms/BranchFolding, a .s file each in DebugInfo/AArch64 and
   CodeGen/PowerPC, and one in CodeGen/SI which is now failing and has been
   XFAILED).

 - This commit also fixes a bunch of config files to use config.root instead of
   older copy-pasted code.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188513 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-16 00:37:11 +00:00
Tom Stellard
a41520cf9b R600/SI: Improve legalization of vector operations
This should fix hangs in the OpenCL piglit tests.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188431 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-14 23:25:00 +00:00
Tom Stellard
e8e33f448e R600/SI: Replace v1i32 type with i32 in imageload and sample intrinsics
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188430 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-14 23:24:53 +00:00
Tom Stellard
68db37b952 R600/SI: Convert v16i8 resource descriptors to i128
Now that compute support is better on SI, we can't continue using v16i8
for descriptors since this is also a legal type in OpenCL.

This patch fixes numerous hangs with the piglit OpenCL test and since
we now use a target specific DAG node for LOAD_CONSTANT with the
correct MemOperandFlags, this should also fix:

https://bugs.freedesktop.org/show_bug.cgi?id=66805

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188429 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-14 23:24:45 +00:00
Tom Stellard
34f505e227 R600/SI: Use i8 types for resource descriptors in tests
We switched from i32 to i8 types a while ago and the tests were never
updated.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188428 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-14 23:24:37 +00:00
Tom Stellard
38d5e1c36d R600/SI: Lower BUILD_VECTOR to REG_SEQUENCE v2
Using REG_SEQUENCE for BUILD_VECTOR rather than a series of INSERT_SUBREG
instructions should make it easier for the register allocator to coalasce
unnecessary copies.

v2:
  - Use an SGPR register class if all the operands of BUILD_VECTOR are
    SGPRs.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188427 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-14 23:24:32 +00:00
Tom Stellard
df4626ef15 R600/SI: Assign a register class to the $vaddr operand for MIMG instructions
The previous code declared the operand as unknown:$vaddr, which made
it possible for scalar registers to be used instead of vector registers.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188425 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-14 23:24:17 +00:00
Tom Stellard
67ca7b1bb0 R600/SI: Handle MSAA texture targets
Patch by: Marek Olšák

Signed-off-by: Marek Olšák <marek.olsak@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188421 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-14 22:22:14 +00:00
Tom Stellard
a7b7ab3299 R600/SI: Allow conversion between v32i8 and v8i32
Patch by: Marek Olšák

Signed-off-by: Marek Olšák <marek.olsak@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188420 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-14 22:22:09 +00:00
Tom Stellard
2760387d86 R600/SI: Add pattern for fp_to_uint
This fixes the F2U opcode for the Mesa driver.

Patch by: Marek Olšák

Signed-off-by: Marek Olšák <marek.olsak@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188418 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-14 22:21:57 +00:00
Tom Stellard
f15dfe4eb4 R600: Set scheduling preference to Sched::Source
R600 doesn't need to do any scheduling on the SelectionDAG now that it
has a very good MachineScheduler.  Also, using the VLIW SelectionDAG
scheduler was having a major impact on compile times. For example with
the phatk kernel here are the LLVM IR to machine code compile times:

With Sched::VLIW

Total Compile Time:                  1.4890 Seconds (User + System)
SelectionDAG Instruction Scheduling: 1.1670 Seconds (User + System)

With Sched::Source

Total Compile Time:                  0.3330 Seconds (User + System)
SelectionDAG Instruction Scheduling: 0.0070 Seconds (User + System)

The code ouput was identical with both schedulers.  This may not be true
for all programs, but it gives me confidence that there won't be much
reduction, if any, in code quality by using Sched::Source.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188215 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-12 22:33:21 +00:00
Niels Ole Salscheider
6e4dbcd115 R600/SI: FMA is faster than fmul and fadd for f64
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188136 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-10 10:38:54 +00:00
Niels Ole Salscheider
4ab92e4d4b R600/SI: Add FMA pattern
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188135 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-10 10:38:47 +00:00
Niels Ole Salscheider
014773626d R600/SI: Implement fp32<->fp64 conversions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187988 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-08 16:06:15 +00:00
Niels Ole Salscheider
7114e2e7cf R600/SI: Implement sint<->fp64 conversions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187987 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-08 16:06:08 +00:00
Tom Stellard
3492eefa4b R600/SI: Use VSrc_* register classes as the default classes for types
Since the VSrc_* register classes contain both VGPRs and SGPRs, copies
that used be emitted by isel like this:

SGPR = COPY VGPR

Will now be emitted like this:

VSrC = COPY VGPR

This patch also adds a pass that tries to identify and fix situations where
a VGPR to SGPR copy may occur.  Hopefully, these changes will make it
impossible for the compiler to generate illegal VGPR to SGPR copies.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187831 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-06 23:08:28 +00:00
Tom Stellard
3406d882c0 R600/SI: Add more special cases for opcodes to ensureSRegLimit()
Also factor out the register class lookup to its own function.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187830 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-06 23:08:18 +00:00
Tom Stellard
01d7203ef8 Factor FlattenCFG out from SimplifyCFG
Patch by: Mei Ye

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187764 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-06 02:43:45 +00:00
Tom Stellard
c1ad37e5cd R600/SI: Add missing test for r187749
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187754 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-05 22:45:56 +00:00
Tom Stellard
692ee102eb R600: Add 64-bit float load/store support
* Added R600_Reg64 class
* Added T#Index#.XY registers definition
* Added v2i32 register reads from parameter and global space
* Added f32 and i32 elements extraction from v2f32 and v2i32
* Added v2i32 -> v2f32 conversions

Tom Stellard:
  - Mark vec2 operations as expand.  The addition of a vec2 register
    class made them all legal.

Patch by: Dmitry Cherkassov

Signed-off-by: Dmitry Cherkassov <dcherkassov@gmail.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187582 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-01 15:23:42 +00:00
Tom Stellard
98b357e1cd R600: Use 64-bit alignment for 64-bit kernel arguments
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187581 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-01 15:23:31 +00:00
Tom Stellard
8cd70d3a5b R600/SI: Custom lower i64 ZERO_EXTEND
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187580 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-01 15:23:26 +00:00
Tom Stellard
6b3f6a744a Revert "R600: Non vector only instruction can be scheduled on trans unit"
This reverts commit 98ce62780e.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187526 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-31 20:43:27 +00:00
Vincent Lejeune
8e37705a73 R600: Avoid more than 4 literals in the same instruction group at scheduling
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187515 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-31 19:32:07 +00:00
Vincent Lejeune
98ce62780e R600: Non vector only instruction can be scheduled on trans unit
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187514 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-31 19:31:56 +00:00
Tom Stellard
e3d60ac334 R600/SI: Expand vector fp <-> int conversions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187421 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-30 14:31:03 +00:00
Quentin Colombet
15d1b85094 [R600] Replicate old DAGCombiner behavior in target specific DAG combine.
build_vector is lowered to REG_SEQUENCE, which is something the register
allocator does a good job at optimizing.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187397 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-30 00:27:16 +00:00
Quentin Colombet
75c9433b49 [DAGCombiner] insert_vector_elt: Avoid building a vector twice.
This patch prevents the following combine when the input vector is used more
than once.
insert_vector_elt (build_vector elt0, ..., eltN), NewEltIdx, idx
=>
build_vector elt0, ..., NewEltIdx, ..., eltN 

The reasons are:
- Building a vector may be expensive, so try to reuse the existing part of a
  vector instead of creating a new one (think big vectors).
- elt0 to eltN now have two users instead of one. This may prevent some other
  optimizations.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187396 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-30 00:24:09 +00:00
Tom Stellard
1f67c63cb2 DAGCombiner: Pass the correct type to TargetLowering::isF(Abs|Neg)Free
This commit also implements these functions for R600 and removes a test
case that was relying on the buggy behavior.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187007 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-23 23:55:03 +00:00
Tom Stellard
8ea83d4999 R600: Treat CONSTANT_ADDRESS loads like GLOBAL_ADDRESS loads when necessary
These are really the same address space in hardware.  The only
difference is that CONSTANT_ADDRESS uses a special cache for faster
access.  When we are unable to use the constant kcache for some reason
(e.g. smaller types or lack of indirect addressing) then the instruction
selector must use GLOBAL_ADDRESS loads instead.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187006 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-23 23:54:56 +00:00
Tom Stellard
10205d5a73 R600: Add support for 24-bit MAD instructions
Reviewed-by: Vincent Lejeune <vljn at ovi.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186923 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-23 01:48:49 +00:00
Tom Stellard
3f5d63b956 R600: Add support for 24-bit MUL instructions
Reviewed-by: Vincent Lejeune <vljn at ovi.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186922 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-23 01:48:42 +00:00
Tom Stellard
eb643b9b37 R600: Improve support for < 32-bit loads
Reviewed-by: Vincent Lejeune <vljn at ovi.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186921 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-23 01:48:35 +00:00
Tom Stellard
58d3335cb9 R600: Move CONST_ADDRESS folding into AMDGPUDAGToDAGISel::Select()
This increases the number of opportunites we have for folding.  With the
previous implementation we were unable to fold into any instructions
other than the first when multiple instructions were selected from a
single SDNode.

Reviewed-by: Vincent Lejeune <vljn at ovi.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186919 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-23 01:48:24 +00:00
Tom Stellard
a7eea0568c R600: Use KCache for kernel arguments
Reviewed-by: Vincent Lejeune <vljn at ovi.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186918 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-23 01:48:18 +00:00
Tom Stellard
f502c292f6 R600: Use the same compute kernel calling convention for all GPUs
A side-effect of this is that now the compiler expects kernel arguments
to be 4-byte aligned.

Reviewed-by: Vincent Lejeune <vljn at ovi.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186916 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-23 01:48:05 +00:00
Tom Stellard
5864284d71 R600: Use correct LoadExtType when lowering kernel arguments
Reviewed-by: Vincent Lejeune <vljn at ovi.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186915 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-23 01:47:58 +00:00
Tom Stellard
2bb20fd2bf R600: Clean up extended load patterns
Reviewed-by: Vincent Lejeune <vljn at ovi.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186914 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-23 01:47:52 +00:00
Tom Stellard
d7a472c9c6 R600: Expand vector FNEG
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186913 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-23 01:47:46 +00:00
Vincent Lejeune
272458bd06 R600: Don't emit empty then clause and use alu_pop_after
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186725 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-19 21:45:15 +00:00
Tom Stellard
4e518fd941 R600/SI: Fix crash with VSELECT
https://bugs.freedesktop.org/show_bug.cgi?id=66175

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186616 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-18 21:43:53 +00:00
Tom Stellard
ac85f3f65c R600/SI: Add support for v2f32 loads
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186615 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-18 21:43:48 +00:00
Tom Stellard
fc047278c1 R600/SI: Add support for v2f32 stores
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186614 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-18 21:43:42 +00:00
Tom Stellard
f5660aab41 R600: Expand VSELECT for all types
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186613 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-18 21:43:35 +00:00
Tom Stellard
cd0f245864 R600/SI: Add support for 64-bit loads
https://bugs.freedesktop.org/show_bug.cgi?id=65873

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186339 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-15 19:00:09 +00:00
Benjamin Kramer
5343cfe5c8 R600: Reapply testcase from r186178, the big endian issue should be fixed by r186196.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186209 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-12 21:54:43 +00:00
Tom Stellard
2a4ac07d21 R600: Remove the fpconst64.ll test which was failing on non-x86 buildbots
I'm guessing the failure had something to do with the double precision
floating point constant used in the test.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186191 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-12 19:29:54 +00:00
Tom Stellard
c9c322cc39 R600/SI: Add support for f64 kernel arguments
Patch by: Niels Ole Salscheider

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186182 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-12 18:15:26 +00:00
Tom Stellard
97781281ca R600/SI: Implement select and compares for SI
Patch by: Niels Ole Salscheider

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186181 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-12 18:15:19 +00:00
Tom Stellard
513fc45629 R600/SI: Add fsqrt pattern for SI
Patch by: Niels Ole Salscheider

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186180 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-12 18:15:13 +00:00
Tom Stellard
d2442c10f9 R600/SI: Add double precision fsub pattern for SI
Patch by: Niels Ole Salscheider

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186179 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-12 18:15:08 +00:00
Tom Stellard
54453c11b4 R600/SI: SI support for 64bit ConstantFP
Patch by: Niels Ole Salscheider

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186178 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-12 18:15:02 +00:00
Tom Stellard
68e1328662 R600/SI: Add initial double precision support for SI
Patch by: Niels Ole Salscheider

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186177 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-12 18:14:56 +00:00
Michel Danzer
a3e39dc705 R600/SI: Initial local memory support
Enough for the radeonsi driver to use it for calculating derivatives.

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186012 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-10 16:37:07 +00:00
Michel Danzer
0a9e22b863 R600/SI: Add intrinsic for retrieving the current thread ID
Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186010 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-10 16:36:52 +00:00
Michel Danzer
0a9aaacd72 R600/SI: Add intrinsics for texture sampling with user derivatives
Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186008 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-10 16:36:36 +00:00
Vincent Lejeune
f2cfef8172 R600: Do not predicated basic block with multiple alu clause
Test is not included as it is several 1000 lines long.
To test this functionnality, a test case must generate at least 2 ALU clauses,
where an ALU clause is ~110 instructions long.

NOTE: This is a candidate for the stable branch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185943 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-09 15:03:33 +00:00
Vincent Lejeune
f4bdec2ebe R600: Fix a rare bug where swizzle optimization returns wrong values
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185942 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-09 15:03:25 +00:00
Vincent Lejeune
07bb3f1d0a R600: Fix wrong export reswizzling
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185941 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-09 15:03:19 +00:00
Vincent Lejeune
c6f13db656 R600: Use DAG lowering pass to handle fcos/fsin
NOTE: This is a candidate for the stable branch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185940 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-09 15:03:11 +00:00
Rafael Espindola
73477b9f32 Prefix failing commands with not to make clear they are expected to fail.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185554 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-03 16:41:29 +00:00
Vincent Lejeune
8f9fbd67c3 R600: Support schedule and packetization of trans-only inst
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185268 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-29 19:32:43 +00:00
Tom Stellard
e3d4cbc7d2 R600: Add local memory support via LDS
Reviewed-by: Vincent Lejeune<vljn at ovi.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185162 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-28 15:47:08 +00:00
Tom Stellard
cedcfee405 R600: Add support for GROUP_BARRIER instruction
Reviewed-by: Vincent Lejeune<vljn at ovi.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185161 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-28 15:46:59 +00:00
Tom Stellard
5f72fa5701 R600: Remove alu-split.ll test
The purpose of this test was to check boundary conditions for the size
of an ALU clause.  This test is very sensitive to changes to the
optimizer or scheduler, because it requires an exact number of ALU
instructions in order to remain valid.  It's not good to have a test
this sensitive, because it is confusing to developers who implement
optimizations and then 'break' the test.

I'm not sure if there is a good way to test these limits using lit, but
if I can come up with replacement test that isn't as sensitive I'll add
it back to the tree.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185084 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-27 17:00:38 +00:00
Tom Stellard
5e48a0e9ae R600: Use new getNamedOperandIdx function generated by TableGen
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184880 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-25 21:22:18 +00:00
Aaron Watry
69d3a40298 R600: Add v2i32 test for vselect
Note: Only adding test for evergreen, not SI yet.

When I attempted to expand vselect for SI, I got the following:
llc: /home/awatry/src/llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp:522:
llvm::SDValue llvm::DAGTypeLegalizer::PromoteIntRes_SETCC(llvm::SDNode*):
Assertion `SVT.isVector() == N->getOperand(0).getValueType().isVector() &&
"Vector compare must return a vector result!"' failed.

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184847 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-25 13:55:54 +00:00
Aaron Watry
1842ec4d9f R600/SI: Expand xor v2i32/v4i32
Add test cases for both vector sizes on SI and also add v2i32 test for EG.

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184846 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-25 13:55:52 +00:00
Aaron Watry
50c1c9102e R600: Add v2i32 test for setcc on evergreen
No test/expansion for SI has been added yet. Attempts to expand this
operation for SI resulted in a stacktrace in (IIRC) LegalizeIntegerTypes
which was complaining about vector comparisons being required to return
a vector type.

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184845 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-25 13:55:49 +00:00
Aaron Watry
41f3db569c R600/SI: Expand urem of v2i32/v4i32 for SI
Also add lit test for both cases on SI, and v2i32 for evergreen.

Note: I followed the guidance of the v4i32 EG check... UREM produces really
complex code, so let's just check that the instruction was lowered
successfully.

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184844 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-25 13:55:46 +00:00
Aaron Watry
e80978f9dd R600/SI: Expand udiv v[24]i32 for SI and v2i32 for EG
Also add lit test for both cases on SI, and v2i32 for evergreen.

Note: I followed the guidance of the v4i32 EG check... UDIV produces really
complex code, so let's just check that the instruction was lowered
successfully.

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184843 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-25 13:55:43 +00:00
Aaron Watry
c1fcf01d7f R600/SI: Expand ashr of v2i32/v4i32 for SI
Also add lit test for both cases on SI, and v2i32 for evergreen.

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184842 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-25 13:55:40 +00:00
Aaron Watry
61de9f83cb R600/SI: Expand srl of v2i32/v4i32 for SI
Also add lit test for both cases on SI, and v2i32 for evergreen.

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184841 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-25 13:55:37 +00:00
Aaron Watry
60e6dacd1c R600/SI: Expand shl of v2i32/v4i32 for SI
Also add lit test for both cases on SI, and v2i32 for evergreen.

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184840 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-25 13:55:32 +00:00
Aaron Watry
b8ce77752b R600/SI: Expand or of v2i32/v4i32 for SI
Also add lit test for both cases on SI, and v2i32 for evergreen.

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184839 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-25 13:55:29 +00:00
Aaron Watry
02ed261f7f R600/SI: Expand mul of v2i32/v4i32 for SI
Also add lit test for both cases on SI, and v2i32 for evergreen.

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184838 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-25 13:55:26 +00:00
Aaron Watry
e3599ca191 R600/SI: Expand and of v2i32/v4i32 for SI
Also add lit test for both cases on SI, and v2i32 for evergreen.

Reviewed-by: Tom Stellard <thomas.stellard@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184837 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-25 13:55:23 +00:00
Tom Stellard
73e44d8ae4 R600/SI: Report unaligned memory accesses as legal for > 32-bit types
In reality, some unaligned memory accesses are legal for 32-bit types and
smaller too, but it all depends on the address space.  Allowing
unaligned loads/stores for > 32-bit types is mainly to prevent the
legalizer from splitting one load into multiple loads of smaller types.

https://bugs.freedesktop.org/show_bug.cgi?id=65873

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184822 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-25 02:39:35 +00:00
Tom Stellard
d67d029b6d R600: Add support for i32 loads from the constant address space on Cayman
Tested-By: Aaron Watry <awatry@gmail.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184821 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-25 02:39:30 +00:00
Tom Stellard
fa57da12f7 R600/SI: Add support for v4i32 and v4f32 kernel args
Tested-By: Aaron Watry <awatry@gmail.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184820 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-25 02:39:25 +00:00