Commit Graph

228 Commits

Author SHA1 Message Date
Matt Arsenault
cb1ac70623 R600: Add new functions for splitting vector loads and stores.
These will be used in future patches and shouldn't change anything yet.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213877 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-24 17:10:35 +00:00
Matt Arsenault
f303d037f2 R600: Fix LowerSDIV24
Use ComputeNumSignBits instead of checking for i8 / i16 which only
worked when AMDIL was lying about having legal i8 / i16.

If an integer is known to fit in 24-bits, we can
do division faster with float ops.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213843 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-24 06:59:20 +00:00
Tom Stellard
b664d47cb0 R600/SI: Store constant initializer data in constant memory
This implements a solution for constant initializers suggested
by Vadim Girlin, where we store the data after the shader code
and then use the S_GETPC instruction to compute its address.

This saves use the trouble of creating a new buffer for constant data
and then having to pass the pointer to the kernel via user SGPRs or the
input buffer.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213530 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-21 14:01:14 +00:00
Tim Northover
7714a60ed1 R600: support fpext/fptrunc operations to and from f16.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213376 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-18 13:01:37 +00:00
Tim Northover
cc03227446 R600: support f16 -> f64 conversion intrinsic.
Unfortunately, we don't seem to have a direct truncation, but the
extension can be legally split into two operations so we should
support that.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213357 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-18 08:43:24 +00:00
Jan Vesely
7b5f8a525a R600: Implement zero undef variants of ctlz/cttz
v2: use ffbh/l if available
v3: Rebase on top of Matt's SI patches

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
Reviewed-by: Tom Stellard <tom@stellard.net>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213072 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-15 15:51:09 +00:00
Matt Arsenault
5fbf09a69f R600: Add dag combine for copy of an illegal type.
This helps avoid redundant instructions to unpack, and repack
the vectors. Ideally we could recognize that pattern and eliminate
it. Currently v4i8 and other small element type vectors are scalarized,
so this has the added bonus of avoiding that.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213031 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-15 02:06:31 +00:00
Jan Vesely
865527a09b R600: Implement float to long/ulong
Use alg. from LegalizeDAG.cpp
Move Expand setting to SIISellowering

v2: Extend existing tests instead of creating new ones
v3: use separate LowerFPTOSINT function
v4: use TargetLowering::expandFP_TO_SINT
    add comment about using FP_TO_SINT for uints

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
Reviewed-by: Tom Stellard <tom@stellard.net>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212773 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-10 22:40:21 +00:00
Matt Arsenault
0e1619e77c R600: Fix mishandling of load / store chains.
Fixes various bugs with reordering loads and stores.
Scalarized vector loads weren't collecting the chains
at all.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212473 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-07 18:34:45 +00:00
Tom Stellard
fad0de390b R600: Add a comment that llvm.AMDGPU.trunc is a legacy intrinsic
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212218 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-02 20:53:57 +00:00
Tom Stellard
1378871b1a R600: Promote i64 loads to v2i32
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212216 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-02 20:53:54 +00:00
Matt Arsenault
b1fb2ba24d R600: Fix crashes when an illegal type load or store is not handled.
I don't think anything hits this now, but will be exposed in future
patches.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212197 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-02 17:44:53 +00:00
Matt Arsenault
97fb702886 R600: Move mul combine to separate function
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212052 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-30 17:55:48 +00:00
Matt Arsenault
556a7c0f08 R600: Move load/store ReplaceNodeResults to common code.
Future patches will want to custom lower loads on SI.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211848 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-27 02:33:47 +00:00
Aaron Ballman
2711c0a68b Silencing a warning about isZExtFree hiding an inherited virtual function. No functional change intended.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211783 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-26 13:45:47 +00:00
Matt Arsenault
b0f5a0e7e7 R600: Fix vector FMA
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211757 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-26 01:28:05 +00:00
Tom Stellard
78d1e95201 R600: Promote i64 stores to v2i32
Now we need only one 64-bit pattern for stores.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211643 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-24 23:33:04 +00:00
Matt Arsenault
95eb45c5d9 R600: Fix inconsistency in rsq instructions.
R600 was using a clamped version of rsq, but SI was not. Add a
new rsq_clamped intrinsic and use them consistently.

It's unclear to me from the documentation what behavior
the R600 instructions have, so I assume they have the legacy behavior
described by the SI documents. For R600, use RECIPSQRT_IEEE
for both llvm.AMDGPU.rsq.legacy and llvm.AMDGPU.rsq. R600 also
has RECIPSQRT_FF, which I'm not sure how it fits in here.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211637 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-24 22:13:39 +00:00
Matt Arsenault
a91ff54e43 R600: Remove DIV_INF
This corresponded to an amdil instruction which there is
a 2 instruction equivalent for.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211616 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-24 17:42:16 +00:00
Matt Arsenault
ed143b7c0c R600/SI: Fix div_scale intrinsic.
The operand that must match one of the others does matter,
and implement selecting for it.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211523 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-23 18:28:28 +00:00
Matt Arsenault
791c054391 R600: Remove AMDILISelLowering
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211519 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-23 18:00:55 +00:00
Matt Arsenault
6272302394 R600: Select is not expensive.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211518 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-23 18:00:52 +00:00
Matt Arsenault
9ad2c7ef92 R600: Move add/sub with overflow out of AMDILISelLowering
Add more tests for these.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211517 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-23 18:00:49 +00:00
Matt Arsenault
5f607c0b39 R600: Move more out of AMDILISelLowering
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211516 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-23 18:00:44 +00:00
Matt Arsenault
c4471e9248 R600/SI: Handle i64 sub.
We can handle it the same way as add

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211514 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-23 18:00:38 +00:00
Matt Arsenault
b170a8042a R600: Rename AMDIL file
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211512 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-23 18:00:31 +00:00
Jan Vesely
ddf2a7902a R600: Use LowerSDIVREM for i64 node replace
v2: move div/rem node replacement to R600ISelLowering
    make lowerSDIVREM protected

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211478 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-22 21:43:01 +00:00
Jan Vesely
cd88535ab9 R600: Implement custom SDIVREM.
Instead of separate SDIV/SREM. SDIV used UDIV which in turn used UDIVREM anyway.
SREM used SDIV(UDIV->UDIVREM)+MUL+SUB, using UDIVREM directly is more efficient.

v2: Don't use all caps names

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211477 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-22 21:43:00 +00:00
Tom Stellard
61d64acd0c R600/SI: Add a pattern for f32 ftrunc
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211377 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-20 17:06:09 +00:00
Tom Stellard
2cda6e8ca6 R600: Expand vector flog2
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211376 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-20 17:06:07 +00:00
Tom Stellard
2d245e2da4 R600: Expand vector fexp2
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211375 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-20 17:06:05 +00:00
Matt Arsenault
d9b35435b8 R600/SI: Add intrinsics for various math instructions.
These will be used for custom lowering and for library
implementations of various math functions, so it's useful
to expose these as builtins.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211247 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-19 01:19:19 +00:00
Matt Arsenault
2ea6d93c5e Use stdint macros for specifying size of constants
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211231 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 22:11:03 +00:00
Matt Arsenault
ce09bda96e R600: Handle fnearbyint
The difference from rint isn't really relevant here,
so treat them as equivalent. OpenCL doesn't have nearbyint,
so this is sort of pointless other than for completeness.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211229 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 22:03:45 +00:00
Matt Arsenault
311ea12506 Use LL suffix for literal that should be 64-bits.
This hopefully fixes Windows

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211225 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 21:40:43 +00:00
Jan Vesely
52b6c2d6ef R600: Expand vector fceil
Move fp64 fceil tests to fceil64.ll

v2: rebase

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211194 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 17:57:29 +00:00
Matt Arsenault
4380c61415 Work around ridiculous warning.
Apparently C++ doesn't really have hex floating point constants.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211192 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 17:45:58 +00:00
Matt Arsenault
2b6e6fc1a8 R600/SI: Add intrinsics for brev instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211187 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 17:13:57 +00:00
Matt Arsenault
debd831223 R600: Implement f64 ftrunc, ffloor and fceil.
CI has instructions for these, so this fixes them for older hardware.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211183 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 17:05:30 +00:00
Matt Arsenault
a5395c03f0 R600: Custom lower f64 frint for pre-CI
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211182 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 17:05:26 +00:00
Matt Arsenault
62e378b057 R600/SI: Match ctlz_zero_undef
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211115 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-17 17:36:24 +00:00
Tom Stellard
f56e7678d1 R600: Use LDS and vectors for private memory
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211110 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-17 16:53:14 +00:00
Tom Stellard
bae98b1b45 SelectionDAG: Expand i64 = FP_TO_SINT i32
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211108 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-17 16:53:07 +00:00
Matt Arsenault
f9ef37a986 Fix copy paste error
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211003 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-15 21:22:52 +00:00
Matt Arsenault
fa848ccd09 R600: Remove a few more things from AMDILISelLowering
Try to keep all the setOperationActions for integer ops
together.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211001 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-15 21:08:58 +00:00
Matt Arsenault
e2480a202f R600: Fix assert on vector sdiv
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211000 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-15 21:08:54 +00:00
Matt Arsenault
62f6ab7a6d R600: Move / cleanup more leftover AMDIL stuff.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210998 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-15 20:23:38 +00:00
Matt Arsenault
57177e3361 R600: Move division custom lowering out of AMDILISelLowering
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210997 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-15 20:08:02 +00:00
Matt Arsenault
61bfbc4d96 R600: Report that integer division is expensive.
Divides by weird constants now emit much better code.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210995 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-15 19:48:16 +00:00
Matt Arsenault
aac455af99 R600: Fix asserts related to constant initializers
This would assert if a constant address space was extern
and therefore didn't have an initializer. If the initializer
was undef, it would hit the unreachable unhandled initializer case.

An extern global should never really occur since we don't have
machine linking, but bugpoint likes to remove initializers.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210967 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-14 04:26:05 +00:00