Commit Graph

762 Commits

Author SHA1 Message Date
Marek Olsak
e1a8ca95be R600/SI: Fix B64 VALU shifts on VI
SI only has standard versions. VI only has REV versions.

Tested-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228037 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-03 21:53:01 +00:00
Marek Olsak
a95296a86e R600/SI: Don't generate non-existent LSHL, LSHR, ASHR B32 variants on VI
This can happen when a REV instruction is commuted.

The trick is not to define the _vi versions of instructions, which has these
consequences:
- code generation will always fail if a pseudo cannot be lowered
  (very useful to catch bugs where an unsupported instruction somehow makes
   it to the printer)
- ability to query if a pseudo can be lowered, which is done in commuteOpcode
  to prevent REV from commuting to non-REV on VI

Tested-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227990 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-03 17:38:12 +00:00
Marek Olsak
b19dbd9eb3 R600/SI: Fix dependency between instruction writing M0 and S_SENDMSG on VI (v2)
This fixes a hang when using an empty geometry shader.

v2: - don't add s_nop when followed by s_waitcnt
    - comestic changes

Tested-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227986 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-03 17:37:52 +00:00
Tom Stellard
d73d1062fe R600/SI: 64-bit and larger memory access must be at least 4-byte aligned
This is true for SI only. CI+ supports unaligned memory accesses,
but this requires driver support, so for now we disallow unaligned
accesses for all GCN targets.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227822 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-02 18:02:28 +00:00
Tom Stellard
80e70ee18e R600/SI: Merge two test files
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227821 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-02 18:02:23 +00:00
Matt Arsenault
9061eb6d2e R600/SI: Only select cvt_flr/cvt_rpi with no NaNs.
These have different behavior from cvt_i32_f32 on NaN.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227693 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-31 21:28:13 +00:00
Matt Arsenault
fa711758df R600/SI: Implement enableAggressiveFMAFusion
Add tests for the various combines. This should
always be at least cycle neutral on all subtargets for f64,
and faster on some. For f32 we should prefer selecting
v_mad_f32 over v_fma_f32.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227484 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-29 19:34:32 +00:00
Tom Stellard
51a3c27d6e R600/SI: Define a schedule model and enable the generic machine scheduler
The schedule model is not complete yet, and could be improved.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227461 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-29 16:55:25 +00:00
Tom Stellard
ff340f98e3 R600: Move DataLayout to AMDGPUTargetMachine
This is a follow up to r227113.

It is now required to use the amdgcn target for SI and newer GPUs.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227316 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-28 16:04:26 +00:00
Marek Olsak
fd55bcd060 R600/SI: Enable all tests that pass on VI without changes
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227214 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-27 17:27:15 +00:00
Matt Arsenault
b33118d503 R600: Cleanup or test
Fix broken check lines, use multiple check prefixes,
add an additional test for i1 or.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227137 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-26 21:16:10 +00:00
Tom Stellard
5b37a2e5ff R600/SI: Emit .hsa.version section for amdhsa OS
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226970 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-23 23:59:08 +00:00
Tom Stellard
511a3c71fc R600/SI: Move i64 -> v2i32 load promotion into AMDGPUDAGToDAGISel::Select()
We used to do this promotion during DAG legalization, but this
caused an infinite loop in ExpandUnalignedLoad() because it assumed
that i64 loads were legal if i64 was a legal type.

It also seems better to report i64 loads as legal, since they actually
are and we were just promoting them to simplify our tablegen files.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226945 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-23 22:05:45 +00:00
Jan Vesely
1d07592ec7 R600: Try to use lower types for 64bit division if possible
v2: add and enable tests for SI

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
Reviewed-by: Matt Arsenault <Matthew.Arsenault@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226881 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-22 23:42:43 +00:00
Tim Northover
f5f8a3e6a6 DAGCombine: fold (or (and X, M), (and X, N)) -> (and X, (or M, N))
It can help with argument juggling on some targets, and is generally a good
idea.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226740 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-21 23:17:19 +00:00
Matt Arsenault
85661f76e3 R600: Add checks for urem/srem by a constant
Make sure this uses the faster expansion using magic constants
to avoid the full division path.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226734 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-21 22:56:15 +00:00
Matt Arsenault
50c3bc9956 R600: Add missing tests for i64 srem
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226713 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-21 22:43:19 +00:00
Matt Arsenault
305228cc0b R600/SI: Custom lower fround
This fixes it for SI. It also removes the pattern
used previously for Evergreen for f32. I'm not sure
if the the new R600 output is better or not, but it uses
1 fewer instructions if BFI is available.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226682 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-21 18:18:25 +00:00
Tim Northover
c49e57ade1 Revert "DAGCombine: fold (or (and X, M), (and X, N)) -> (and X, (or M, N))"
It hadn't gone through review yet, but was still on my local copy.

This reverts commit r226663

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226665 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-21 15:48:52 +00:00
Tim Northover
47f47f5d2a DAGCombine: fold (or (and X, M), (and X, N)) -> (and X, (or M, N))
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226663 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-21 15:43:28 +00:00
Tom Stellard
5d96beaab5 R600/SI: Fix simple-loop.ll test
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226596 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-20 19:33:02 +00:00
Tom Stellard
ad7a884efe R600/SI: Add kill flag when copying scratch offset to a register
This allows us to re-use the same register for the scratch offset
when accessing large private arrays.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226585 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-20 17:49:45 +00:00
Tom Stellard
a978a481bb R600/SI: Don't store scratch buffer frame index in MUBUF offset field
We don't have a good way of legalizing this if the frame index offset
is more than the 12-bits, which is size of MUBUF's offset field, so
now we store the frame index in the vaddr field.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226584 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-20 17:49:43 +00:00
Matt Arsenault
676db0a373 R600: Remove redundant test
This is already covered in ftrunc.ll

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226412 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-18 19:30:32 +00:00
Matt Arsenault
b2bb846f17 R600: Clean up floor tests
These were using different naming schemes,
not using multiple check prefixes and not using
-LABEL.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226333 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-16 22:11:00 +00:00
Matt Arsenault
ab2315014e R600/SI: Add patterns for v_cvt_{flr|rpi}_i32_f32
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226230 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-15 23:58:35 +00:00
Matt Arsenault
c204f47feb R600/SI: Fix trailing comma with modifiers
Instructions with 1 operand can still use source modifiers,
so make sure we don't print an extra comma afterwards.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226226 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-15 23:17:03 +00:00
Matt Arsenault
ecbec418bd R600/SI: Improve fpext / fptrunc test coverage
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226197 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-15 19:39:42 +00:00
Marek Olsak
232d5fa02c R600/SI: Use 64-bit encoding by default for opcodes that are VOP3-only on VI
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226190 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-15 18:43:01 +00:00
Matt Arsenault
140c2ece1e R600/SI: Remove some redudant load testcases.
This reduces coverage for Evergreen, since the more
complete tests have those run lines disabled.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225927 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-14 01:35:26 +00:00
Matt Arsenault
781f7ee502 R600/SI: Fix bad code with unaligned byte vector loads
Don't do the v4i8 -> v4f32 combine if the load will need to
be expanded due to alignment. This stops adding instructions
to repack into a single register that the v_cvt_ubyteN_f32
instructions read.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225926 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-14 01:35:22 +00:00
Matt Arsenault
8b6a26ca85 Implement new way of expanding extloads.
Now that the source and destination types can be specified,
allow doing an expansion that doesn't use an EXTLOAD of the
result type. Try to do a legal extload to an intermediate type
and extend that if possible.

This generalizes the special case custom lowering of extloads
R600 has been using to work around this problem.

This also happens to fix a bug that would incorrectly use more
aligned loads than should be used.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225925 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-14 01:35:17 +00:00
Matt Arsenault
8603a3d1c5 R600: Implement getRsqrtEstimate
Only do for f32 since I'm unclear on both what this is expecting
for the refinement steps in terms of accuracy, and what
f64 instruction actually provides.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225827 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-13 20:53:18 +00:00
Matt Arsenault
9e495c518c R600: Make cttz / ctlz cheap to speculate
Speculating things is generally good. SI+ has instructions for these
for 32-bit values. This is still probably better even with the expansion
for 64-bit values, although it is odd that this callback doesn't have
the size as a parameter.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225822 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-13 19:46:48 +00:00
Matt Arsenault
29ad7506e1 Combine fcmp + select to fminnum / fmaxnum if no nans and legal
Also require unsafe FP math for no since there isn't a way to
test for signed zeros.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225744 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-13 00:43:00 +00:00
Tom Stellard
d275e025d2 R600/SI: Use RegisterOperands to specify which operands can accept immediates
There are some operands which can take either immediates or registers
and we were previously using different register class to distinguish
between operands that could take immediates and those that could not.

This patch switches to using RegisterOperands which should simplify the
backend by reducing the number of register classes and also make it
easier to implement the assembler.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225662 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-12 19:33:18 +00:00
Tom Stellard
9a6e4f08fe R600/SI: Remove SIISelLowering::legalizeOperands()
Its functionality has been replaced by calling
SIInstrInfo::legalizeOperands() from
SIISelLowering::AdjstInstrPostInstrSelection() and running the
SIFoldOperands and SIShrinkInstructions passes.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225445 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-08 15:08:17 +00:00
Matthias Braun
a065cf13cd RegisterCoalescer: Fix valuesIdentical() in some subrange merge cases.
I got confused and assumed SrcIdx/DstIdx of the CoalescerPair is a
subregister index in SrcReg/DstReg, but they are actually subregister
indices of the coalesced register that get you back to SrcReg/DstReg
when applied.

Fixed the bug, improved comments and simplified code accordingly.

Testcase by Tom Stellard!

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225415 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-07 23:58:38 +00:00
Tom Stellard
a36b682c17 R600/SI: Commute instructions to enable more folding opportunities
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225410 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-07 22:44:19 +00:00
Tom Stellard
a3ee583339 R600/SI: Only fold immediates that have one use
Folding the same immediate into multiple instruction will increase
program size, which can hurt performance.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225405 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-07 22:18:27 +00:00
Tom Stellard
f7587043ef R600/SI: Add a V_MOV_B64 pseudo instruction
This is used to simplify the SIFoldOperands pass and make it easier to
fold immediates.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225373 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-07 20:27:25 +00:00
Tom Stellard
546520a727 R600/SI: Teach SIFoldOperands to split 64-bit constants when folding
This allows folding of sequences like:

s[0:1] = s_mov_b64 4
v_add_i32 v0, s0, v0
v_addc_u32 v1, s1, v1

into

v_add_i32 v0, 4, v0
v_add_i32 v1, 0, v1

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225369 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-07 19:56:17 +00:00
Matt Arsenault
6a72b20325 R600/SI: Add combine for isinfinite pattern
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225310 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-06 23:00:46 +00:00
Matt Arsenault
42d9f7cf0a R600/SI: Pattern match isinf to v_cmp_class instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225307 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-06 23:00:41 +00:00
Matt Arsenault
a5b2b64292 R600/SI: Add basic DAG combines for fp_class
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225306 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-06 23:00:39 +00:00
Matt Arsenault
b6520ab625 R600/SI: Add class intrinsic
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225305 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-06 23:00:37 +00:00
Tom Stellard
bac89f3dd2 R600/SI: Insert s_waitcnt before s_barrier instructions.
This ensures that all memory operations are complete when all threads
reach the barrier.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225290 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-06 19:52:07 +00:00
Tom Stellard
1f996fa36b R600/SI: Add a stub GCNTargetMachine
This is equivalent to the AMDGPUTargetMachine now, but it is the
starting point for separating R600 and GCN functionality into separate
targets.

It is recommened that users start using the gcn triple for GCN-based
GPUs, because using the r600 triple for these GPUs will be deprecated in
the future.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225277 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-06 18:00:21 +00:00
Matt Arsenault
d796cf2e01 Enable (sext x) == C --> x == (trunc C) combine
Extend the existing code which handles this for zext. This makes this
more useful for targets with ZeroOrNegativeOne BooleanContent and
obsoletes a custom combine SI uses for i1 setcc (sext(i1), 0, setne)
since the constant will now be shrunk to i1.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@224691 91177308-0d34-0410-b5e6-96231b3b80d8
2014-12-21 16:48:42 +00:00
Matt Arsenault
7fc3bdab6a R600/SI: Only form min/max with 1 use.
If the condition is used for something else, this increases
the number of instructions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@224646 91177308-0d34-0410-b5e6-96231b3b80d8
2014-12-19 23:15:30 +00:00