387 Commits

Author SHA1 Message Date
Tom Stellard
a787066317 R600/SI: Initial support for assembler and inline assembly
This is currently considered experimental, but most of the more
commonly used instructions should work.

So far only SI has been extensively tested, CI and VI probably work too,
but may be buggy.  The current set of tests cases do not give complete
coverage, but I think it is sufficient for an experimental assembler.

See the documentation in R600Usage for more information.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@234381 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-08 01:09:26 +00:00
Tom Stellard
c0578c36bb R600/SI: Add missing SOPK instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@234380 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-08 01:09:22 +00:00
Marek Olsak
a2705bbd42 R600/SI: Select V_BFE_U32 for and+shift with a non-literal offset
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@233079 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-24 13:40:34 +00:00
Marek Olsak
945fab3447 R600/SI: Improve BFM support
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@233077 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-24 13:40:21 +00:00
Marek Olsak
3f05a5e0ad R600/SI: Use V_FRACT_F64 for faster 64-bit floor on SI
Other f64 opcodes not supported on SI can be lowered in a similar way.

v2: use complex VOP3 patterns

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@233076 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-24 13:40:15 +00:00
Marek Olsak
91c066ae15 R600/SI: Expand fract to floor, then only select V_FRACT on CI
V_FRACT is buggy on SI.

R600-specific code is left intact.

v2: drop the multiclass, use complex VOP3 patterns

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@233075 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-24 13:40:08 +00:00
Matt Arsenault
16ef3632cf R600/SI: Only use one range of isCommutable for compares
Also don't count the class instructions as isCompare anymore.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@232991 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-23 18:45:38 +00:00
Matt Arsenault
88ed640930 R600/SI: Remove redundant unsetting of hasSideEffects
These are already set in the base class for the instruction.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@232990 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-23 18:45:36 +00:00
Matt Arsenault
7ec044c97b R600/SI: Move hasSideEffects setting into VOPCX classes
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@232989 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-23 18:45:35 +00:00
Matt Arsenault
59a5e979b5 R600/SI: Allow commuting compares
This enables very common cases to switch to the
smaller encoding.

All of the standard LLVM canonicalizations of comparisons
are the opposite of what we want. Compares with constants
are moved to the RHS, but the first operand can be an inline
immediate, literal constant, or SGPR using the 32-bit VOPC
encoding.

There are additional bad canonicalizations that should
also be fixed, such as canonicalizing ge x, k to gt x, (k + 1)
if this makes k no longer an inline immediate value.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@232988 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-23 18:45:30 +00:00
Matt Arsenault
b53594d1cb R600/SI: Use right class for cmpsx f64 instructions
Use VOPCX_F64 to not need the let Defs = [EXEC]

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@232987 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-23 18:45:23 +00:00
Craig Topper
3220d112fa [Tablegen] Attempt to add support for patterns containing nodes with multiple results.
This is needed for AVX512 masked scatter/gather support.

The R600 change is necessary to remove a hack that was working around the lack of multiple results.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@232798 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-20 05:09:06 +00:00
Tom Stellard
3d712a6373 R600/SI: Remove _e32 and _e64 suffixes from mnemonics
Instead print them as part of the $dst operand.  The AsmMatcher
requires the 32-bit and 64-bit encodings have the same mnemonic in
order to parse them correctly.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@232105 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-12 21:34:22 +00:00
Tom Stellard
29e7485cda R600/SI: Re-order MUBUF operands to match asm strings.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@231797 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-10 16:16:49 +00:00
Tom Stellard
92811fa2c7 R600/SI: Add 32-bit encoding of v_cndmask_b32
This was done by refactoring the v_cndmask_b32 tablegen definition
to use inherit from VOP2Inst.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@231795 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-10 16:16:44 +00:00
Tom Stellard
3db921673e R600/SI: Move gds operand to the end of operand list
Also print it in the assembly string.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@231684 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-09 18:49:54 +00:00
Tom Stellard
4c5aebe89d R600/SI: Refactor DS instruction defs
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@231683 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-09 18:49:45 +00:00
Tom Stellard
2ffe261575 R600/SI: Fix DS definitions and add missing instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@231663 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-09 16:03:45 +00:00
Tom Stellard
cc9a0ba0bf R600/SI: Fix opcode for ds_read2_b64 and ds_read2st64_b64
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@231662 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-09 16:03:39 +00:00
Marek Olsak
506d4b2cb4 R600/SI: Add an intrinsic for S_FLBIT_I32 / V_FFBH_I32
Required by OpenGL (ARB_gpu_shader5).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@231259 91177308-0d34-0410-b5e6-96231b3b80d8
2015-03-04 17:33:45 +00:00
Tom Stellard
27eab65932 R600/SI: Add missing mubuf instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@230759 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-27 14:59:46 +00:00
Tom Stellard
a9c15883ba R600/SI: Consistently put soffset before the offset operand for mubuf instructions
This matches the assembly syntax.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@230758 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-27 14:59:44 +00:00
Tom Stellard
4ed3bf29cd R600/SI: Add slc, glc, and tfe to non-atomic _ADDR64 instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@230757 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-27 14:59:41 +00:00
Matt Arsenault
dc9d5dcdd7 R600/SI: Fix mad*k definitions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@230146 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-21 21:29:00 +00:00
Matt Arsenault
16fc5e9c0f R600/SI: Remove v_sub_f64 pseudo
The expansion code does the same thing. Since
the operands were not defined with the correct
types, this has the side effect of fixing operand
folding since the expanded pseudo would never use
SGPRs or inline immediates.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@230072 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-20 22:10:45 +00:00
Matt Arsenault
bbb748eece R600: Use new fmad node.
This enables a few useful combines that used to only
use fma.

Also since v_mad_f32 apparently does not support denormals,
disable the existing cases that are custom handled if they are
requested.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@230071 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-20 22:10:41 +00:00
Marek Olsak
4f5a891372 R600/SI: Fix READLANE and WRITELANE lane select for VI
VOP2 declares vsrc1, but VOP3 declares src1.
We can't use the same "ins" if the operands have different names in VOP2
and VOP3 encodings.

This fixes a hang in geometry shaders which spill M0 on VI.
(BTW it doesn't look like M0 needs spilling and the spilling seems
duplicated 3 times)

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229752 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-18 22:12:45 +00:00
Tom Stellard
d1971ae8ba R600/SI: Add missing VOP1 instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229688 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-18 16:08:15 +00:00
Tom Stellard
16b731f19f R600/SI: Add missing VOP2 instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229687 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-18 16:08:14 +00:00
Tom Stellard
585290a32a R600/SI: Add definition for S_CBRANCH_G_FORK
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229686 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-18 16:08:13 +00:00
Tom Stellard
fc0347ac9c R600/SI: Add missing SOP1 instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229685 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-18 16:08:11 +00:00
Matt Arsenault
5177e9551c R600/SI: Fix not setting clamp / omod for v_cndmask_b32_e64
Rename the multiclass since it now applies to the output
modifiers as well.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229610 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-18 02:15:27 +00:00
Matt Arsenault
2422768a8a R600/SI: Add missing offset operand to buffer bothen
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229605 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-18 02:04:38 +00:00
Matt Arsenault
8a44761afe R600/SI: Implement correct f64 fdiv
This version passes the OpenCL conformance test.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229239 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-14 04:30:08 +00:00
Matt Arsenault
9295d69bea R600/SI: Fix implicit vcc operand to v_div_fmas_*
This should allow finally fixing the f64 fdiv implementation.

Test is disabled for VI since there seems to be a problem with one
of the buffer load instructions on it.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229236 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-14 04:22:00 +00:00
Matt Arsenault
f1b6c1f287 R600/SI: Fix schedule model for v_div_scale_{f32|f64}
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229235 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-14 04:03:18 +00:00
Tom Stellard
6378a7cb0b R600/SI: Add soffset operand to mubuf addr64 instruction
We were previously hard-coding soffset to 0.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228775 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-11 00:34:32 +00:00
Michel Danzer
a7879dcf33 R600/SI: Also enable WQM for image opcodes which calculate LOD v3
If whole quad mode isn't enabled for these, the level of detail is
calculated incorrectly for pixels along diagonal triangle edges, causing
artifacts.

v2: Use a TSFlag instead of lots of switch cases
v3: Add test coverage

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=88642
Reviewed-by: Tom Stellard <tom@stellard.net>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228372 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-06 02:51:20 +00:00
Matt Arsenault
81eb6ca158 R600/SI: Fix i64 truncate to i1
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228273 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-05 06:05:13 +00:00
Marek Olsak
f34625adbd R600/SI: Remove useless patterns in VALU which are already covered by SALU
Also remove hasPostISelHook=1 from V_LSHL_B32. It's defined by InstSI already.

Tested-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228039 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-03 21:53:08 +00:00
Marek Olsak
e1a8ca95be R600/SI: Fix B64 VALU shifts on VI
SI only has standard versions. VI only has REV versions.

Tested-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228037 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-03 21:53:01 +00:00
Marek Olsak
a95296a86e R600/SI: Don't generate non-existent LSHL, LSHR, ASHR B32 variants on VI
This can happen when a REV instruction is commuted.

The trick is not to define the _vi versions of instructions, which has these
consequences:
- code generation will always fail if a pseudo cannot be lowered
  (very useful to catch bugs where an unsupported instruction somehow makes
   it to the printer)
- ability to query if a pseudo can be lowered, which is done in commuteOpcode
  to prevent REV from commuting to non-REV on VI

Tested-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227990 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-03 17:38:12 +00:00
Marek Olsak
2ea95bd471 R600/SI: Remove VOP2_REV definitions from target-specific instructions
The getCommute* functions are only used with pseudos, so this commit doesn't
change anything.

The issue with missing non-rev versions of shift instructions on VI will fixed
separately.

Tested-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227989 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-03 17:38:05 +00:00
Marek Olsak
5e58a1bc29 R600/SI: Trivial instruction definition corrections for VI (v2)
- V_MAC_LEGACY_F32 exists on VI, but it's VOP3-only.

- Define CVT_PK opcodes which are different between SI and VI. These are
  unused. The idea is to define all chip differences.

v2: keep V_MUL_LO_U32

Tested-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227988 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-03 17:38:01 +00:00
Eric Christopher
0def30471a Reuse a bunch of cached subtargets and remove getSubtarget calls
without a Function argument.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227638 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-30 23:24:40 +00:00
Marek Olsak
d922eb0be9 R600/SI: Fix MIN3/MAX3 on VI, define MED3
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227213 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-27 17:25:15 +00:00
Marek Olsak
9f824da16d R600/SI: Add VI versions of LDS atomics
Each class is split into two: one adds let statements around non-pseudos,
and the other one specifies the parameters.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227211 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-27 17:25:07 +00:00
Marek Olsak
37788ca85a R600/SI: Add VI versions of MUBUF atomics
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227210 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-27 17:25:02 +00:00
Marek Olsak
058ab6001b R600/SI: Add VI versions of MUBUF loads and stores
This enables a lot of existing patterns for VI.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227209 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-27 17:24:58 +00:00
Tom Stellard
6dc4d8f0ed R600/SI: Use external symbols for scratch buffer
We were passing the scratch buffer address to the shaders via user sgprs,
but now we use external symbols and have the driver patch the shader
using reloc information.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226586 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-20 17:49:47 +00:00