804 Commits

Author SHA1 Message Date
Tom Stellard
6d625ad495 R600/SI: Add support for global atomic add
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218254 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-22 15:35:35 +00:00
Matt Arsenault
1a505ebae4 R600: Un-xfail a test which passes with pass disabled
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218165 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-19 23:02:20 +00:00
Matt Arsenault
ea3a0242f4 R600/SI: Un-xfail tests which work now
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218164 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-19 23:02:18 +00:00
Matt Arsenault
c58ab80f78 R600/SI: Un xfail a test that works now
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218162 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-19 22:42:40 +00:00
Matt Arsenault
c14f7630e0 R600/SI: Fix test to prepare for scheduler
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218131 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-19 18:11:16 +00:00
Matt Arsenault
bd2b96a12d R600: Better fix for bug 20982
Just do the left shift as unsigned to avoid the UB.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218092 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-19 00:42:06 +00:00
Matt Arsenault
e08e52528b R600: Bug 20982 - Avoid undefined left shift of negative value
I'm not sure what the hardware actually does, so don't
bother trying to fold it for now.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218057 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-18 15:52:26 +00:00
Alexey Samsonov
dc4eb3d6dc Exclude known and bugzilled failures from UBSan bootstrap
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217979 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-17 20:17:52 +00:00
Matt Arsenault
507636288f R600/SI: Change formatting of printed FP immediates
Only 1 decimal place should be printed for inline immediates.
Other constants should be hex constants.

Does not include f64 tests because folding those inline
immediates currently does not work.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217964 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-17 17:32:13 +00:00
Matt Arsenault
f1b16047b7 R600/SI: Prefer selecting more e64 instruction forms.
Add some more tests to make sure better operand
choices are still made. Leave some cases that seem
to have no reason to ever be e64 alone.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217789 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-15 17:15:02 +00:00
Matt Arsenault
6fc71a0cfc R600/SI: Make sure double vector fmul is tested
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217787 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-15 17:04:54 +00:00
Matt Arsenault
e626ee51b6 R600/SI: Add some mubuf testcases.
I noticed some odd looking cases where addr64 wasn't set
when storing to a pointer in an SGPR. This seems to be intentional,
and partially tested already.

The documentation seems to describe addr64 in terms of which registers
addressing modifiers come from, but I would expect to always need
addr64 when using 64-bit pointers. If no offset is applied,
it makes sense to not need to worry about doing a 64-bit add
for the final address. A small immediate offset can be applied,
so is it OK to not have addr64 set if a carry is necessary when adding
the base pointer in the resource to the offset?

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217785 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-15 16:48:01 +00:00
Matt Arsenault
d189a0407d R600/SI: Add preliminary support for flat address space
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217777 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-15 15:41:53 +00:00
Matt Arsenault
a0ba49844c R600/SI: Fix broken check lines
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217736 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-14 18:32:05 +00:00
Matt Arsenault
86ffcddf42 R600/SI: Fix off by 1 error in used register count
The register numbers start at 0, so if only 1 register
was used, this was reported as 0.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217636 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-11 22:51:37 +00:00
Matt Arsenault
31b1bdbd95 Add DAG combine for shl + add of constants.
Do
 (shl (add x, c1), c2) -> (add (shl x, c2), c1 << c2)

This is already done for multiplies, but since multiplies
by powers of two are turned into shifts, we also need
to handle it here.

This might want checks for isLegalAddImmediate to avoid
transforming an add of a legal immediate with one that isn't.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217610 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-11 17:34:19 +00:00
Aaron Watry
1ff44854f6 R600: Test local atomics for evergreen
Now that the operations are all implemented, we can test this sub-arch here.

Signed-off-by: Aaron Watry <awatry@gmail.com>
Reviewed-by: Matt Arsenault <matthew.arsenault@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217595 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-11 15:02:52 +00:00
Matt Arsenault
5ee5d45e7e R600/SI: Fix losing chain when fixing reg class of loads.
The lost chain resulting in earlier side effecting nodes
being deleted.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217561 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-10 23:26:19 +00:00
Matt Arsenault
257e85e7c2 R600: Custom lower frem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217553 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-10 21:44:27 +00:00
Matt Arsenault
ef4bb30475 R600/SI: Replace LDS atomics with no return versions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217379 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-08 15:07:31 +00:00
Matt Arsenault
6a712e709d R600/SI: Relax a few tests to help enable scheduler
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217320 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-06 20:44:41 +00:00
Matt Arsenault
360ed46f68 R600/SI: Fix broken check lines.
Fix missing check, and hardcoded register numbers.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217318 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-06 20:37:56 +00:00
Matt Arsenault
89a7e3ec3e R600/SI: Use same complex patterns for DS atomics
This fixes hitting the same negative base offset problem
that was already fixed for regular loads and stores.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217256 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-05 16:24:58 +00:00
Jan Vesely
286f644bce R600: Fix FROUND
round halfway cases away from zero

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
Reviewed-by: Tom Stellard <tom@stellard.net>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217250 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-05 14:26:54 +00:00
Tom Stellard
7cda2d0666 R600/SI: Use S_ADD_U32 and S_SUB_U32 for low half of 64-bit operations
https://bugs.freedesktop.org/show_bug.cgi?id=83416

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217248 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-05 14:07:59 +00:00
Matt Arsenault
c9cc488dfe R600/SI: Try to keep i32 mul on SALU
Also fix bug this exposed where when legalizing an immediate
operand, a v_mov_b32 would be created with a VSrc dest register.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217108 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-03 23:24:35 +00:00
Tom Stellard
ce4caf146f R600/SI: Add a pattern for i64 and in a branch
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217041 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-03 15:22:41 +00:00
Matt Arsenault
2aab51a118 R600/SI: Relax some ordering in tests.
This will help with enabling misched

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@216971 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-02 21:45:50 +00:00
Matt Arsenault
9c21df64a4 R600/SI: Fix hardcoded register numbers in test
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@216944 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-02 20:43:07 +00:00
Matt Arsenault
f471c483e6 R600/SI: Add failing testcase.
This is broken when 64-bit add is only partially
moved to the VALU.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@216933 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-02 19:12:31 +00:00
Matt Arsenault
f7a3c7e705 Fix interference caused by fmul 2, x -> fadd x, x
If an fmul was introduced by lowering, it wouldn't be folded
into a multiply by a constant since the earlier combine would
have replaced the fmul with the fadd.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@216932 91177308-0d34-0410-b5e6-96231b3b80d8
2014-09-02 19:02:53 +00:00
Matt Arsenault
f4d57e7874 R600/SI: Use mad for fsub + fmul
We can use a negate source modifier to match
this for fsub.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@216735 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-29 16:01:14 +00:00
Tom Stellard
f50f927d65 R600/SI: Use READ2/WRITE2 instructions for 64-bit mem ops with 32-bit alignment
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@216279 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-22 18:49:35 +00:00
Tom Stellard
ec4cb3346d R600/SI: Use a ComplexPattern for DS loads and stores
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@216278 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-22 18:49:33 +00:00
Tom Stellard
fdbf61d00d R600/SI: Teach moveToVALU how to handle more S_LOAD_* instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@216220 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-21 20:41:00 +00:00
Tom Stellard
5f52739370 R600/SI: Make sure SCRATCH_WAVE_OFFSET is added as Live-In to the function
This fixes a crash in an ocl conformance test.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@216219 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-21 20:40:58 +00:00
Matt Arsenault
c86e55eb6e R600/SI: Move all fabs / fneg handling to patterns
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215749 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-15 18:42:22 +00:00
Matt Arsenault
0498d07255 R600/SI: Use source modifiers for f64 fneg
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215748 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-15 18:42:18 +00:00
Matt Arsenault
c882fc78fe R600/SI: Use source modifier for f64 fabs
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215747 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-15 18:42:15 +00:00
Matt Arsenault
34ef4cd65b R600/SI: Fix offset folding in some cases with shifted pointers.
Ordinarily (shl (add x, c1), c2) -> (add (shl x, c2), c1 << c2)
is only done if the add has one use. If the resulting constant
add can be folded into an addressing mode, force this to happen
for the pointer operand.

This ends up happening a lot because of how LDS objects are allocated.
Since the globals are allocated next to each other, acessing the first
element of the second object is directly indexed by a shifted pointer.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215739 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-15 17:49:05 +00:00
Matt Arsenault
5bc44c7603 R600/SI: Add intrinsic for ldexp
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215734 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-15 17:30:25 +00:00
Matt Arsenault
ed76ca720b R600/SI: Implement isLegalAddressingMode
The default assumes that a 16-bit signed offset is used.
LDS instruction use a 16-bit unsigned offset, so it wasn't
being used in some cases where it was assumed a negative offset
could be used.

More should be done here, but first isLegalAddressingMode needs
to gain an addressing mode argument. For now, copy most of the rest
of the default implementation with the immediate offset change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215732 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-15 17:17:07 +00:00
Matt Arsenault
bd949eea85 R600: Correctly set the src value offset for scalarized kernel args
This for some reason fixes v1i64 kernel arguments on pre-SI. This
currently breaks some other cases in the kernel-args.ll test for R600,
but I'm not particularly confident in the new output. VTX_READ_* are not
used for some of the scalarized cases, and the code reading from the
constant buffer doesn't make much sense to me.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215564 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-13 18:14:11 +00:00
Hal Finkel
695e914c03 Fix classof for ISD::INTRINSIC_W_CHAIN and INTRINSIC_VOID
Unfortunately, our use of the SDNode class hierarchy for INTRINSIC_W_CHAIN and
INTRINSIC_VOID nodes is somewhat broken right now. These nodes sometimes are
used for memory intrinsics (those with MachineMemOperands), and sometimes not.
When not, the nodes are not created as instances of MemIntrinsicSDNode, but
rather created as some other subclass of SDNode using DAG::getNode. When they
are memory intrinsics, they are created using DAG::getMemIntrinsicNode as
instances of MemIntrinsicSDNode. MemIntrinsicSDNode is a subclass of
MemSDNode, but prior to r214452, we had a non-self-consistent setup whereby
MemIntrinsicSDNode::classof on INTRINSIC_W_CHAIN and INTRINSIC_VOID would
return true but MemSDNode::classof on INTRINSIC_W_CHAIN and INTRINSIC_VOID
would return false. In r214452, MemSDNode::classof was changed to return true
for INTRINSIC_W_CHAIN and INTRINSIC_VOID, which is now self-consistent. The
problem is that neither the pre-r214452 logic and the post-r214452 logic are
really right. The truth is that not all INTRINSIC_W_CHAIN and INTRINSIC_VOID
nodes are instances of MemIntrinsicSDNode (or MemSDNode for that matter), and
the return value from classof needs to reflect that. This was broken before
r214452 (because MemIntrinsicSDNode::classof always returned true), and was
broken afterward (because MemSDNode::classof also always returned true), and
will now be correct.

The minimal solution is to grab one of the SubclassData bits (there is one left
for MemIntrinsicSDNode nodes) and use it to store whether or not a particular
INTRINSIC_W_CHAIN or INTRINSIC_VOID is really an instance of
MemIntrinsicSDNode or not. Doing this allows both MemIntrinsicSDNode::classof
and MemSDNode::classof to return the correct answer for the underlying object
for both the memory-intrinsic and non-memory-intrinsic cases.

This fixes the problem that r214452 created in the SelectionDAGDumper (thanks
to Matt Arsenault for pointing it out).

Because PowerPC does not implement getTgtMemIntrinsic, this change breaks
test/CodeGen/PowerPC/unal-altivec-wint.ll. I've XFAILed it for now, and will
fix it in a follow-up commit.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215511 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-13 01:15:37 +00:00
Jan Vesely
3c57820bbb R600: Use optimized 24bit path in udivrem
v2: drop enum keyword
    use correct extension mode
    don't bother computing the sign in unsinged case

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215462 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-12 17:31:20 +00:00
Jan Vesely
b40562c0ec R600: Use i24 optimized path for SREM
v2: add tests
    rename LowerSDIV24 to LowerSDIVREM24
    handle the rem part in this function

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215460 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-12 17:31:17 +00:00
Tom Stellard
13f4476c55 R600/SI: Add a ComplexPattern for selecting MUBUF _OFFSET variant
This saves us from having to copy a 64-bit 0 value into VGPRs for
BUFFER_* instruction which only have a 12-bit immediate offset.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215399 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-11 22:18:17 +00:00
Tom Stellard
68e9ebbe44 R600/SI: Add check for low 32 bits of encoding to mubuf tests
There are no variable values like registers encoded in the low 32 bits of MUBUF
instructions, so it is relatively easy to check these bits, and it will
help prevent us from introducing encoding bugs.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215397 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-11 22:18:11 +00:00
Tom Stellard
728d0e4218 R600/SI: Clear lds bit on MUBUF instructions used for private stores
This bit was left uninitialized, which was causing some random failures
of piglit tests.

NOTE: This is a candidate for the 3.5 branch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215396 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-11 22:18:09 +00:00
Tom Stellard
0df264a0fd R600/SI: Fix broken test
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215395 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-11 22:18:05 +00:00