Commit Graph

1598 Commits

Author SHA1 Message Date
Matt Arsenault
2d925cbe10 R600/SI: Fix size of VReg_1
This is really a 32-bit register, if we try to check the size of it,
we want 32-bits.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229223 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-14 02:51:44 +00:00
Duncan P. N. Exon Smith
20d6ad9de2 R600: Canonicalize access to function attributes, NFC
Canonicalize access to function attributes to use the simpler API.

getAttributes().getAttribute(AttributeSet::FunctionIndex, Kind)
  => getFnAttribute(Kind)

getAttributes().hasAttribute(AttributeSet::FunctionIndex, Kind)
  => hasFnAttribute(Kind)

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229222 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-14 02:45:45 +00:00
Tom Stellard
193679b6ae R600/SI: Refactor SOP1 classes
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229152 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-13 21:02:37 +00:00
Tom Stellard
ad4bb96a12 R600/SI: Lowercase register names
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229151 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-13 21:02:36 +00:00
Tom Stellard
1f85e3b090 R600/SI: Remove some unused TableGen classes
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229150 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-13 21:02:33 +00:00
Matt Arsenault
941454f17f R600/SI: Remove handling of fpimm
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229136 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-13 19:05:07 +00:00
Matt Arsenault
1751616522 R600/SI: Allow f64 inline immediates in i64 operands
This requires considering the size of the operand when
checking immediate legality.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229135 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-13 19:05:03 +00:00
Chandler Carruth
417c5c172c [PM] Remove the old 'PassManager.h' header file at the top level of
LLVM's include tree and the use of using declarations to hide the
'legacy' namespace for the old pass manager.

This undoes the primary modules-hostile change I made to keep
out-of-tree targets building. I sent an email inquiring about whether
this would be reasonable to do at this phase and people seemed fine with
it, so making it a reality. This should allow us to start bootstrapping
with modules to a certain extent along with making it easier to mix and
match headers in general.

The updates to any code for users of LLVM are very mechanical. Switch
from including "llvm/PassManager.h" to "llvm/IR/LegacyPassManager.h".
Qualify the types which now produce compile errors with "legacy::". The
most common ones are "PassManager", "PassManagerBase", and
"FunctionPassManager".

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229094 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-13 10:01:29 +00:00
Matt Arsenault
c8055a1f5e R600/SI: Remove unnecessary check for fpimm
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229034 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-13 02:47:22 +00:00
Benjamin Kramer
d913d9d2c3 MathExtras: Bring Count(Trailing|Leading)Ones and CountPopulation in line with countTrailingZeros
Update all callers.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228930 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-12 15:35:40 +00:00
Tom Stellard
293dfe59a5 R600/SI: Disable subreg liveness
This is temporary while we try to fix a crash in the register coalescer.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228861 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-11 18:24:53 +00:00
Tom Stellard
28f550eed1 R600: Split AMDGPUPassConfig into R600PassConfig and GCNPassConfig
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228850 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-11 17:11:51 +00:00
Tom Stellard
5b98db607b R600: Create an R600TargetMachine for pre-gcn GPUs
No functinality change. R600TargetMachine inherits from
AMDGPUTargetMachine.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228849 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-11 17:11:50 +00:00
Tom Stellard
9f5d593c1f R600/SI: Store immediate offsets > 12-bits in soffset
This will save us from having to extend these offsets to 64-bits
and storing them in a pair of vgprs.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228776 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-11 00:34:35 +00:00
Tom Stellard
6378a7cb0b R600/SI: Add soffset operand to mubuf addr64 instruction
We were previously hard-coding soffset to 0.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228775 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-11 00:34:32 +00:00
Benjamin Kramer
e003f1ac8c Make helper functions/classes/globals static. NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228410 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-06 17:51:54 +00:00
Michel Danzer
971f0f0071 R600/SI: Don't enable WQM for V_INTERP_* instructions v2
Doesn't seem necessary anymore. I think this was mostly compensating for
not enabling WQM for texture sampling instructions.

v2: Add test coverage
Reviewed-by: Tom Stellard <tom@stellard.net>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228373 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-06 02:51:25 +00:00
Michel Danzer
a7879dcf33 R600/SI: Also enable WQM for image opcodes which calculate LOD v3
If whole quad mode isn't enabled for these, the level of detail is
calculated incorrectly for pixels along diagonal triangle edges, causing
artifacts.

v2: Use a TSFlag instead of lots of switch cases
v3: Add test coverage

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=88642
Reviewed-by: Tom Stellard <tom@stellard.net>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228372 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-06 02:51:20 +00:00
Tom Stellard
c7198528eb R600/SI: Fix bug in TTI loop unrolling preferences
We should be setting UnrollingPreferences::MaxCount to MAX_UINT instead
of UnrollingPreferences::Count.

Count is a 'forced unrolling factor', while MaxCount sets an upper
limit to the unrolling factor.

Setting Count to MAX_UINT was causing the loop in the testcase to be
unrolled 15 times, when it only had a maximum of 4 iterations.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228303 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-05 15:32:18 +00:00
Tom Stellard
041211cd79 R600/SI: Fix bug from insertion of llvm.SI.end.cf into loop headers
The llvm.SI.end.cf intrinsic is used to mark the end of if-then blocks,
if-then-else blocks, and loops.  It is responsible for updating the
exec mask to re-enable threads that had been masked during the preceding
control flow block.  For example:

s_mov_b64 exec, 0x3                 ; Initial exec mask
s_mov_b64 s[0:1], exec              ; Saved exec mask
v_cmpx_gt_u32 exec, s[2:3], v0, 0   ; llvm.SI.if
do_stuff()
s_or_b64 exec, exec, s[0:1]         ; llvm.SI.end.cf

The bug fixed by this patch was one where the llvm.SI.end.cf intrinsic
was being inserted into the header of loops.  This would happen when
an if block terminated in a loop header and we would end up with
code like this:

s_mov_b64 exec, 0x3                 ; Initial exec mask
s_mov_b64 s[0:1], exec              ; Saved exec mask
v_cmpx_gt_u32 exec, s[2:3], v0, 0   ; llvm.SI.if
do_stuff()

LOOP:                       ; Start of loop header
s_or_b64 exec, exec, s[0:1] ; llvm.SI.end.cf <-BUG: The exec mask has the
                              same value at the beginning of each loop
			      iteration.
do_stuff();
s_cbranch_execnz LOOP

The fix is to create a new basic block before the loop and insert the
llvm.SI.end.cf there.  This way the exec mask is restored before the
start of the loop instead of at the beginning of each iteration.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228302 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-05 15:32:15 +00:00
Matt Arsenault
81eb6ca158 R600/SI: Fix i64 truncate to i1
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228273 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-05 06:05:13 +00:00
Tom Stellard
26bfda9dd3 R600/SI: Enable subreg liveness by default
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228228 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-04 23:14:18 +00:00
Tom Stellard
89c96b1cd0 R600/SI: Expand misaligned 16-bit memory accesses
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228190 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-04 20:49:52 +00:00
Tom Stellard
fd4c349de2 R600/SI: Make more store operations legal
v2i32, i32, trunc i32 to i16, and truc i32 to i8 stores are legal for
all address spaces.  We had marked them as custom in order to lower
them for the private address space, but this is no longer necessary.

This enables lowering of misaligned stores of these types in the
DAGLegalizer.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228189 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-04 20:49:51 +00:00
Tom Stellard
056a34916a R600: Don't promote i64 stores to v2i32 during DAG legalization
We take care of this during instruction selection now.  This
fixes a potential infinite loop when lowering misaligned stores.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228188 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-04 20:49:49 +00:00
Marek Olsak
f34625adbd R600/SI: Remove useless patterns in VALU which are already covered by SALU
Also remove hasPostISelHook=1 from V_LSHL_B32. It's defined by InstSI already.

Tested-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228039 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-03 21:53:08 +00:00
Marek Olsak
f8cf57cb0c R600/SI: Rewrite VOP1InstSI to contain a pseudo and _si opcode
What this does is that if you accidentally select these instructions on VI,
the code generation will fail, because the pseudo -> _vi mapping will be
undefined.

The idea is to be able to catch possible future bugs easily.

Tested-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228038 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-03 21:53:05 +00:00
Marek Olsak
e1a8ca95be R600/SI: Fix B64 VALU shifts on VI
SI only has standard versions. VI only has REV versions.

Tested-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228037 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-03 21:53:01 +00:00
Marek Olsak
a95296a86e R600/SI: Don't generate non-existent LSHL, LSHR, ASHR B32 variants on VI
This can happen when a REV instruction is commuted.

The trick is not to define the _vi versions of instructions, which has these
consequences:
- code generation will always fail if a pseudo cannot be lowered
  (very useful to catch bugs where an unsupported instruction somehow makes
   it to the printer)
- ability to query if a pseudo can be lowered, which is done in commuteOpcode
  to prevent REV from commuting to non-REV on VI

Tested-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227990 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-03 17:38:12 +00:00
Marek Olsak
2ea95bd471 R600/SI: Remove VOP2_REV definitions from target-specific instructions
The getCommute* functions are only used with pseudos, so this commit doesn't
change anything.

The issue with missing non-rev versions of shift instructions on VI will fixed
separately.

Tested-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227989 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-03 17:38:05 +00:00
Marek Olsak
5e58a1bc29 R600/SI: Trivial instruction definition corrections for VI (v2)
- V_MAC_LEGACY_F32 exists on VI, but it's VOP3-only.

- Define CVT_PK opcodes which are different between SI and VI. These are
  unused. The idea is to define all chip differences.

v2: keep V_MUL_LO_U32

Tested-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227988 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-03 17:38:01 +00:00
Marek Olsak
f426f1db6e R600/SI: Determine target-specific encoding of READLANE and WRITELANE early v2
These are VOP2 on SI and VOP3 on VI, and their pseudos are neither, which can
be a problem. In order to make isVOP2 and isVOP3 queries behave as expected,
the encoding must be determined first.

This doesn't fix any known issue, but better safe than sorry.

v2: add and use getMCOpcodeFromPseudo

Tested-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227987 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-03 17:37:57 +00:00
Marek Olsak
b19dbd9eb3 R600/SI: Fix dependency between instruction writing M0 and S_SENDMSG on VI (v2)
This fixes a hang when using an empty geometry shader.

v2: - don't add s_nop when followed by s_waitcnt
    - comestic changes

Tested-by: Michel Dänzer <michel.daenzer@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227986 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-03 17:37:52 +00:00
Tom Stellard
d73d1062fe R600/SI: 64-bit and larger memory access must be at least 4-byte aligned
This is true for SI only. CI+ supports unaligned memory accesses,
but this requires driver support, so for now we disallow unaligned
accesses for all GCN targets.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227822 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-02 18:02:28 +00:00
Chandler Carruth
26bc071088 [multiversion] Remove the function parameter from the unrolling
preferences interface on TTI now that all of TTI is per-function.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227741 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-01 14:31:23 +00:00
Chandler Carruth
b71d385494 [multiversion] Switch the TTI queries from TargetMachine to Subtarget
now that we have a correct and cached subtarget specific to the
function.

Also, finish providing a cached per-function subtarget in the core
LLVMTargetMachine -- that layer hadn't switched over yet.

The only use of the TargetMachine was to re-lookup a subtarget for
a particular function to work around the fact that TTI was immutable.
Now that it is per-function and we haved a cached subtarget, use it.

This still leaves a few interfaces with real warts on them where we were
passing Function objects through the TTI interface. I'll remove these
and clean their usage up in subsequent commits now that this isn't
necessary.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227738 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-01 14:22:17 +00:00
Chandler Carruth
d0bfb83efb [multiversion] Remove the cached TargetMachine pointer from the
intermediate TTI implementation template and instead query up to the
derived class for both the TargetMachine and the TargetLowering.

Most of the derived types had a TLI cached already and there is no need
to store a less precisely typed target machine pointer.

This will in turn make it much cleaner to look up the TLI via
a per-function subtarget instead of the generic subtarget, and it will
pave the way toward pulling the subtarget used for unroll preferences
into the same form once we are *always* using the function to look up
the correct subtarget.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227737 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-01 14:01:15 +00:00
Chandler Carruth
6e89e1316a [multiversion] Switch all of the targets over to use the
TargetIRAnalysis access path directly rather than implementing getTTI.

This even removes getTTI from the interface. It's more efficient for
each target to just register a precise callback that creates their
specific TTI.

As part of this, all of the targets which are building their subtargets
individually per-function now build their TTI instance with the function
and thus look up the correct subtarget and cache it. NVPTX, R600, and
XCore currently don't leverage this functionality, but its trivial for
them to add it now.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227735 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-01 13:20:00 +00:00
Chandler Carruth
d12af8754e [multiversion] Remove a false freedom to leave the TargetMachine pointer
null.

For some reason some of the original TTI code supported a null target
machine. This seems to have been legacy, and I made matters worse when
refactoring this code by spreading that pattern further through the
various targets.

The TargetMachine can't actually be null, and it doesn't make sense to
support that use case. I've now consistently removed it and removed all
of the code trying to cope with that situation. This is probably good,
as several targets *didn't* cope with it being null despite the null
default argument in their constructors. =]

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227734 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-01 12:38:24 +00:00
Chandler Carruth
685c2add65 [PM] Remove a bunch of stale TTI creation method declarations. I nuked
their definitions, but forgot to clean up all the declarations which are
in different files.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227698 91177308-0d34-0410-b5e6-96231b3b80d8
2015-02-01 00:22:15 +00:00
Matt Arsenault
ea3e75f4d4 Fix typo
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227697 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-31 23:37:27 +00:00
Matt Arsenault
9061eb6d2e R600/SI: Only select cvt_flr/cvt_rpi with no NaNs.
These have different behavior from cvt_i32_f32 on NaN.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227693 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-31 21:28:13 +00:00
Chandler Carruth
1937233a22 [PM] Switch the TargetMachine interface from accepting a pass manager
base which it adds a single analysis pass to, to instead return the type
erased TargetTransformInfo object constructed for that TargetMachine.

This removes all of the pass variants for TTI. There is now a single TTI
*pass* in the Analysis layer. All of the Analysis <-> Target
communication is through the TTI's type erased interface itself. While
the diff is large here, it is nothing more that code motion to make
types available in a header file for use in a different source file
within each target.

I've tried to keep all the doxygen comments and file boilerplate in line
with this move, but let me know if I missed anything.

With this in place, the next step to making TTI work with the new pass
manager is to introduce a really simple new-style analysis that produces
a TTI object via a callback into this routine on the target machine.
Once we have that, we'll have the building blocks necessary to accept
a function argument as well.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227685 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-31 11:17:59 +00:00
Chandler Carruth
a6a87b595d [PM] Change the core design of the TTI analysis to use a polymorphic
type erased interface and a single analysis pass rather than an
extremely complex analysis group.

The end result is that the TTI analysis can contain a type erased
implementation that supports the polymorphic TTI interface. We can build
one from a target-specific implementation or from a dummy one in the IR.

I've also factored all of the code into "mix-in"-able base classes,
including CRTP base classes to facilitate calling back up to the most
specialized form when delegating horizontally across the surface. These
aren't as clean as I would like and I'm planning to work on cleaning
some of this up, but I wanted to start by putting into the right form.

There are a number of reasons for this change, and this particular
design. The first and foremost reason is that an analysis group is
complete overkill, and the chaining delegation strategy was so opaque,
confusing, and high overhead that TTI was suffering greatly for it.
Several of the TTI functions had failed to be implemented in all places
because of the chaining-based delegation making there be no checking of
this. A few other functions were implemented with incorrect delegation.
The message to me was very clear working on this -- the delegation and
analysis group structure was too confusing to be useful here.

The other reason of course is that this is *much* more natural fit for
the new pass manager. This will lay the ground work for a type-erased
per-function info object that can look up the correct subtarget and even
cache it.

Yet another benefit is that this will significantly simplify the
interaction of the pass managers and the TargetMachine. See the future
work below.

The downside of this change is that it is very, very verbose. I'm going
to work to improve that, but it is somewhat an implementation necessity
in C++ to do type erasure. =/ I discussed this design really extensively
with Eric and Hal prior to going down this path, and afterward showed
them the result. No one was really thrilled with it, but there doesn't
seem to be a substantially better alternative. Using a base class and
virtual method dispatch would make the code much shorter, but as
discussed in the update to the programmer's manual and elsewhere,
a polymorphic interface feels like the more principled approach even if
this is perhaps the least compelling example of it. ;]

Ultimately, there is still a lot more to be done here, but this was the
huge chunk that I couldn't really split things out of because this was
the interface change to TTI. I've tried to minimize all the other parts
of this. The follow up work should include at least:

1) Improving the TargetMachine interface by having it directly return
   a TTI object. Because we have a non-pass object with value semantics
   and an internal type erasure mechanism, we can narrow the interface
   of the TargetMachine to *just* do what we need: build and return
   a TTI object that we can then insert into the pass pipeline.
2) Make the TTI object be fully specialized for a particular function.
   This will include splitting off a minimal form of it which is
   sufficient for the inliner and the old pass manager.
3) Add a new pass manager analysis which produces TTI objects from the
   target machine for each function. This may actually be done as part
   of #2 in order to use the new analysis to implement #2.
4) Work on narrowing the API between TTI and the targets so that it is
   easier to understand and less verbose to type erase.
5) Work on narrowing the API between TTI and its clients so that it is
   easier to understand and less verbose to forward.
6) Try to improve the CRTP-based delegation. I feel like this code is
   just a bit messy and exacerbating the complexity of implementing
   the TTI in each target.

Many thanks to Eric and Hal for their help here. I ended up blocked on
this somewhat more abruptly than I expected, and so I appreciate getting
it sorted out very quickly.

Differential Revision: http://reviews.llvm.org/D7293

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227669 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-31 03:43:40 +00:00
Eric Christopher
0def30471a Reuse a bunch of cached subtargets and remove getSubtarget calls
without a Function argument.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227638 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-30 23:24:40 +00:00
Tom Stellard
24b999d7d0 R600/SI: Handle SI_SPILL_V96_RESTORE in SIRegisterInfo::eliminateFrameIndex()
This fixes a crash in Unigine Heaven.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227618 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-30 21:51:51 +00:00
Matt Arsenault
fa711758df R600/SI: Implement enableAggressiveFMAFusion
Add tests for the various combines. This should
always be at least cycle neutral on all subtargets for f64,
and faster on some. For f32 we should prefer selecting
v_mad_f32 over v_fma_f32.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227484 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-29 19:34:32 +00:00
Matt Arsenault
c416d94735 R600/SI: Add subtarget feature for if f32 fma is fast
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227483 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-29 19:34:25 +00:00
Matt Arsenault
30a5d21457 R600/SI: Fix tonga's basic scheduling model
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227482 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-29 19:34:18 +00:00
Rafael Espindola
9936b80df5 Compute the ELF SectionKind from the flags.
Any code creating an MCSectionELF knows ELF and already provides the flags.

SectionKind is an abstraction used by common code that uses a plain
MCSection.

Use the flags to compute the SectionKind. This removes a lot of
guessing and boilerplate from the MCSectionELF construction.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227476 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-29 17:33:21 +00:00
Tom Stellard
882d1b71e0 R600/SI: Remove stray debug statements
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227462 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-29 16:55:28 +00:00
Tom Stellard
51a3c27d6e R600/SI: Define a schedule model and enable the generic machine scheduler
The schedule model is not complete yet, and could be improved.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227461 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-29 16:55:25 +00:00
Tom Stellard
ff340f98e3 R600: Move DataLayout to AMDGPUTargetMachine
This is a follow up to r227113.

It is now required to use the amdgcn target for SI and newer GPUs.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227316 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-28 16:04:26 +00:00
Tom Stellard
9813f8be5d R600: Use a Southern Islands GPU as the default for the amdgcn target
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227314 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-28 15:38:42 +00:00
Marek Olsak
d922eb0be9 R600/SI: Fix MIN3/MAX3 on VI, define MED3
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227213 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-27 17:25:15 +00:00
Marek Olsak
2ba0c13c26 R600/SI: Don't set patterns for chip-specific instructions while having pseudos
Only pseudos have patterns on them.

Also don't set the asm string for VINTRP_Pseudo. All pseudos should have empty
asm.

This matches what all other multiclasses do.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227212 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-27 17:25:11 +00:00
Marek Olsak
9f824da16d R600/SI: Add VI versions of LDS atomics
Each class is split into two: one adds let statements around non-pseudos,
and the other one specifies the parameters.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227211 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-27 17:25:07 +00:00
Marek Olsak
37788ca85a R600/SI: Add VI versions of MUBUF atomics
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227210 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-27 17:25:02 +00:00
Marek Olsak
058ab6001b R600/SI: Add VI versions of MUBUF loads and stores
This enables a lot of existing patterns for VI.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227209 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-27 17:24:58 +00:00
Marek Olsak
3111d0ae12 R600/SI: Add pseudos for MUBUF loads and stores
This defines the SI versions only, so it shouldn't change anything.

There are no changes other than using the new multiclasses, adding missing
mayLoad/mayStore, and formatting fixes.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227208 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-27 17:24:54 +00:00
Eric Christopher
04bcc11905 Move DataLayout back to the TargetMachine from TargetSubtargetInfo
derived classes.

Since global data alignment, layout, and mangling is often based on the
DataLayout, move it to the TargetMachine. This ensures that global
data is going to be layed out and mangled consistently if the subtarget
changes on a per function basis. Prior to this all targets(*) have
had subtarget dependent code moved out and onto the TargetMachine.

*One target hasn't been migrated as part of this change: R600. The
R600 port has, as a subtarget feature, the size of pointers and
this affects global data layout. I've currently hacked in a FIXME
to enable progress, but the port needs to be updated to either pass
the 64-bitness to the TargetMachine, or fix the DataLayout to
avoid subtarget dependent features.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227113 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-26 19:03:15 +00:00
Tom Stellard
5b37a2e5ff R600/SI: Emit .hsa.version section for amdhsa OS
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226970 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-23 23:59:08 +00:00
Tom Stellard
511a3c71fc R600/SI: Move i64 -> v2i32 load promotion into AMDGPUDAGToDAGISel::Select()
We used to do this promotion during DAG legalization, but this
caused an infinite loop in ExpandUnalignedLoad() because it assumed
that i64 loads were legal if i64 was a legal type.

It also seems better to report i64 loads as legal, since they actually
are and we were just promoting them to simplify our tablegen files.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226945 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-23 22:05:45 +00:00
Jan Vesely
1d07592ec7 R600: Try to use lower types for 64bit division if possible
v2: add and enable tests for SI

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
Reviewed-by: Matt Arsenault <Matthew.Arsenault@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226881 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-22 23:42:43 +00:00
Jan Vesely
cd0d34f1a3 R600: Simplify LowerUDIVREM
optimizations can handle removing the Hi part operations.
The generated code is identical for R600, ~10% icount reduction for SI

v2: rebase

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
Reviewed-by: Matt Arsenault <Matthew.Arsenault@amd.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226879 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-22 23:42:39 +00:00
Matt Arsenault
305228cc0b R600/SI: Custom lower fround
This fixes it for SI. It also removes the pattern
used previously for Evergreen for f32. I'm not sure
if the the new R600 output is better or not, but it uses
1 fewer instructions if BFI is available.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226682 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-21 18:18:25 +00:00
Tom Stellard
46846844ee R600/SI: Add subtarget feature to enable VGPR spilling for all shader types
This is disabled by default, but can be enabled with the subtarget
feature: 'vgpr-spilling'

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226597 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-20 19:33:04 +00:00
Tom Stellard
5d96beaab5 R600/SI: Fix simple-loop.ll test
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226596 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-20 19:33:02 +00:00
Tom Stellard
daa414fd39 R600/SI: Remove stray debugging code from r226586
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226591 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-20 19:24:31 +00:00
Tom Stellard
6dc4d8f0ed R600/SI: Use external symbols for scratch buffer
We were passing the scratch buffer address to the shaders via user sgprs,
but now we use external symbols and have the driver patch the shader
using reloc information.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226586 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-20 17:49:47 +00:00
Tom Stellard
ad7a884efe R600/SI: Add kill flag when copying scratch offset to a register
This allows us to re-use the same register for the scratch offset
when accessing large private arrays.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226585 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-20 17:49:45 +00:00
Tom Stellard
a978a481bb R600/SI: Don't store scratch buffer frame index in MUBUF offset field
We don't have a good way of legalizing this if the frame index offset
is more than the 12-bits, which is size of MUBUF's offset field, so
now we store the frame index in the vaddr field.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226584 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-20 17:49:43 +00:00
Tom Stellard
2a2bde40ab R600/SI: Update SIInstrInfo:verifyInstruction() after r225662
Now that we have our own custom register operand types, we need
to handle them in the verifiier.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226583 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-20 17:49:41 +00:00
Rafael Espindola
a23cc6a1ea Add r224985 back with fixes.
The fixes are to note that AArch64 has additional restrictions on when local
relocations can be used. In particular, ld64 requires that relocations to
cstring/cfstrings use linker visible symbols.

Original message:

In an assembly expression like

bar:
  .long L0 + 1

the intended semantics is that bar will contain a pointer one byte past L0.

In sections that are merged by content (strings, 4 byte constants, etc), a
single position in the section doesn't give the linker enough information.
For example, it would not be able to tell a relocation must point to the
end of a string, since that would look just like the start of the next.

The solution used in ELF to use relocation with symbols if there is a non-zero
addend.

In MachO before this patch we would just keep all symbols in some sections.

This would miss some cases (only cstrings on x86_64 were implemented) and was
inefficient since most relocations have an addend of 0 and can be represented
without the symbol.

This patch implements the non-zero addend logic for MachO too.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226503 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-19 21:11:14 +00:00
David Blaikie
341a7e245e std::unique_ptrify the MCStreamer argument to createAsmPrinter
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226414 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-18 20:29:04 +00:00
Matt Arsenault
ab2315014e R600/SI: Add patterns for v_cvt_{flr|rpi}_i32_f32
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226230 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-15 23:58:35 +00:00
Matt Arsenault
c204f47feb R600/SI: Fix trailing comma with modifiers
Instructions with 1 operand can still use source modifiers,
so make sure we don't print an extra comma afterwards.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226226 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-15 23:17:03 +00:00
Marek Olsak
3d7ab06c38 R600/SI: Unify VOP2 instructions which are VOP3-only on VI
This removes some duplicated classes and definitions.

These instructions are defined:
  _e32 // pseudo
  _e32_si
  _e64 // pseudo
  _e64_si
  _e64_vi

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226191 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-15 18:43:06 +00:00
Marek Olsak
232d5fa02c R600/SI: Use 64-bit encoding by default for opcodes that are VOP3-only on VI
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226190 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-15 18:43:01 +00:00
Marek Olsak
352fa59ec3 R600/SI: Add V_READLANE_B32 and V_WRITELANE_B32 for VI
These are VOP3-only on VI.

The new multiclass doesn't define VOP3 versions of VOP2 instructions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226189 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-15 18:42:55 +00:00
Marek Olsak
6589ea14e7 R600/SI: Don't shrink instructions whose e32 encoding doesn't exist
v2: modify hasVALU32BitEncoding instead
v3: - add pseudoToMCOpcode helper to AMDGPUInstInfo, which is used by both
      hasVALU32BitEncoding and AMDGPUMCInstLower::lower
    - report an error if a pseudo can't be lowered

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226188 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-15 18:42:51 +00:00
Marek Olsak
cbb4ac578d R600/SI: Add common class VOPAnyCommon
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226187 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-15 18:42:44 +00:00
Marek Olsak
a32c8ca9eb R600/SI: Don't select SI-only VOP3 opcodes on VI
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226186 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-15 18:42:40 +00:00
Rafael Espindola
8327f0bca1 Revert "Add r224985 back with two fixes."
This reverts commit r225644 while I debug a regression.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226022 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-14 19:07:23 +00:00
Tom Stellard
65e0a4d4da R600/SI: Use IMPLICIT_DEF and KILL when failing to spill VGPRs
This helps us avoid 'invalid register class for operand' verifier
errors.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225989 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-14 15:42:34 +00:00
Tom Stellard
33040cf56e R600/SI: Spill VGPRs to scratch space for compute shaders
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225988 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-14 15:42:31 +00:00
Chandler Carruth
1b279144ec [cleanup] Re-sort all the #include lines in LLVM using
utils/sort_includes.py.

I clearly haven't done this in a while, so more changed than usual. This
even uncovered a missing include from the InstrProf library that I've
added. No functionality changed here, just mechanical cleanup of the
include order.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225974 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-14 11:23:27 +00:00
Matt Arsenault
781f7ee502 R600/SI: Fix bad code with unaligned byte vector loads
Don't do the v4i8 -> v4f32 combine if the load will need to
be expanded due to alignment. This stops adding instructions
to repack into a single register that the v_cvt_ubyteN_f32
instructions read.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225926 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-14 01:35:22 +00:00
Matt Arsenault
8b6a26ca85 Implement new way of expanding extloads.
Now that the source and destination types can be specified,
allow doing an expansion that doesn't use an EXTLOAD of the
result type. Try to do a legal extload to an intermediate type
and extend that if possible.

This generalizes the special case custom lowering of extloads
R600 has been using to work around this problem.

This also happens to fix a bug that would incorrectly use more
aligned loads than should be used.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225925 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-14 01:35:17 +00:00
Tom Stellard
aafca11ff9 R600/SI: Define a schedule model
The machine scheduler is still disabled by default.

The schedule model is not complete yet, and could be improved.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225913 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-14 01:13:19 +00:00
Tom Stellard
d90e5063ca R600/SI: Add pattern for bitcasting fp immediates to integers
The backend now assumes that all immediates are integers.  This allows
us to simplify immediate handling code, becasue we no longer need to
handle fp and integer immediates differently.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225844 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-13 22:59:41 +00:00
Matt Arsenault
7c06364dc0 R600: Implement getRecipEstimate
This requires a new hook to prevent expanding sqrt in terms
of rsqrt and reciprocal. v_rcp_f32, v_rsq_f32, and v_sqrt_f32 are
all the same rate, so this expansion would just double the number
of instructions and cycles.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225828 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-13 20:53:23 +00:00
Matt Arsenault
8603a3d1c5 R600: Implement getRsqrtEstimate
Only do for f32 since I'm unclear on both what this is expecting
for the refinement steps in terms of accuracy, and what
f64 instruction actually provides.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225827 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-13 20:53:18 +00:00
Matt Arsenault
9e495c518c R600: Make cttz / ctlz cheap to speculate
Speculating things is generally good. SI+ has instructions for these
for 32-bit values. This is still probably better even with the expansion
for 64-bit values, although it is odd that this callback doesn't have
the size as a parameter.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225822 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-13 19:46:48 +00:00
Matt Arsenault
549b6dbbb7 R600/SI: Remove redundant setting expand on f64 vectors
None of these are legal types already, so they default to
Expand.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225728 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-12 23:13:00 +00:00
Tom Stellard
d275e025d2 R600/SI: Use RegisterOperands to specify which operands can accept immediates
There are some operands which can take either immediates or registers
and we were previously using different register class to distinguish
between operands that could take immediates and those that could not.

This patch switches to using RegisterOperands which should simplify the
backend by reducing the number of register classes and also make it
easier to implement the assembler.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225662 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-12 19:33:18 +00:00
Rafael Espindola
5512415ade Add r224985 back with two fixes.
One is that AArch64 has additional restrictions on when local relocations can
be used. We have to take those into consideration when deciding to put a L
symbol in the symbol table or not.

The other is that ld64 requires the relocations to cstring to use linker
visible symbols on AArch64.

Thanks to Michael Zolotukhin for testing this!

Remove doesSectionRequireSymbols.

In an assembly expression like

bar:
.long L0 + 1

the intended semantics is that bar will contain a pointer one byte past L0.

In sections that are merged by content (strings, 4 byte constants, etc), a
single position in the section doesn't give the linker enough information.
For example, it would not be able to tell a relocation must point to the
end of a string, since that would look just like the start of the next.

The solution used in ELF to use relocation with symbols if there is a non-zero
addend.

In MachO before this patch we would just keep all symbols in some sections.

This would miss some cases (only cstrings on x86_64 were implemented) and was
inefficient since most relocations have an addend of 0 and can be represented
without the symbol.

This patch implements the non-zero addend logic for MachO too.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225644 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-12 18:13:07 +00:00
Tom Stellard
9a6e4f08fe R600/SI: Remove SIISelLowering::legalizeOperands()
Its functionality has been replaced by calling
SIInstrInfo::legalizeOperands() from
SIISelLowering::AdjstInstrPostInstrSelection() and running the
SIFoldOperands and SIShrinkInstructions passes.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225445 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-08 15:08:17 +00:00
Ahmed Bougacha
7fac1d945f [SelectionDAG] Allow targets to specify legality of extloads' result
type (in addition to the memory type).

The *LoadExt* legalization handling used to only have one type, the
memory type.  This forced users to assume that as long as the extload
for the memory type was declared legal, and the result type was legal,
the whole extload was legal.

However, this isn't always the case.  For instance, on X86, with AVX,
this is legal:
    v4i32 load, zext from v4i8
but this isn't:
    v4i64 load, zext from v4i8
Whereas v4i64 is (arguably) legal, even without AVX2.

Note that the same thing was done a while ago for truncstores (r46140),
but I assume no one needed it yet for extloads, so here we go.

Calls to getLoadExtAction were changed to add the value type, found
manually in the surrounding code.

Calls to setLoadExtAction were mechanically changed, by wrapping the
call in a loop, to match previous behavior.  The loop iterates over
the MVT subrange corresponding to the memory type (FP vectors, etc...).
I also pulled neighboring setTruncStoreActions into some of the loops;
those shouldn't make a difference, as the additional types are illegal.
(e.g., i128->i1 truncstores on PPC.)

No functional change intended.

Differential Revision: http://reviews.llvm.org/D6532


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225421 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-08 00:51:32 +00:00
Tom Stellard
a36b682c17 R600/SI: Commute instructions to enable more folding opportunities
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225410 91177308-0d34-0410-b5e6-96231b3b80d8
2015-01-07 22:44:19 +00:00