173 Commits

Author SHA1 Message Date
Jan Vesely
7b5f8a525a R600: Implement zero undef variants of ctlz/cttz
v2: use ffbh/l if available
v3: Rebase on top of Matt's SI patches

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
Reviewed-by: Tom Stellard <tom@stellard.net>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213072 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-15 15:51:09 +00:00
Matt Arsenault
5fbf09a69f R600: Add dag combine for copy of an illegal type.
This helps avoid redundant instructions to unpack, and repack
the vectors. Ideally we could recognize that pattern and eliminate
it. Currently v4i8 and other small element type vectors are scalarized,
so this has the added bonus of avoiding that.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213031 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-15 02:06:31 +00:00
Jan Vesely
865527a09b R600: Implement float to long/ulong
Use alg. from LegalizeDAG.cpp
Move Expand setting to SIISellowering

v2: Extend existing tests instead of creating new ones
v3: use separate LowerFPTOSINT function
v4: use TargetLowering::expandFP_TO_SINT
    add comment about using FP_TO_SINT for uints

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
Reviewed-by: Tom Stellard <tom@stellard.net>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212773 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-10 22:40:21 +00:00
Matt Arsenault
0e1619e77c R600: Fix mishandling of load / store chains.
Fixes various bugs with reordering loads and stores.
Scalarized vector loads weren't collecting the chains
at all.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212473 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-07 18:34:45 +00:00
Tom Stellard
fad0de390b R600: Add a comment that llvm.AMDGPU.trunc is a legacy intrinsic
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212218 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-02 20:53:57 +00:00
Tom Stellard
1378871b1a R600: Promote i64 loads to v2i32
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212216 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-02 20:53:54 +00:00
Matt Arsenault
b1fb2ba24d R600: Fix crashes when an illegal type load or store is not handled.
I don't think anything hits this now, but will be exposed in future
patches.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212197 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-02 17:44:53 +00:00
Matt Arsenault
97fb702886 R600: Move mul combine to separate function
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212052 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-30 17:55:48 +00:00
Matt Arsenault
556a7c0f08 R600: Move load/store ReplaceNodeResults to common code.
Future patches will want to custom lower loads on SI.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211848 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-27 02:33:47 +00:00
Aaron Ballman
2711c0a68b Silencing a warning about isZExtFree hiding an inherited virtual function. No functional change intended.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211783 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-26 13:45:47 +00:00
Matt Arsenault
b0f5a0e7e7 R600: Fix vector FMA
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211757 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-26 01:28:05 +00:00
Tom Stellard
78d1e95201 R600: Promote i64 stores to v2i32
Now we need only one 64-bit pattern for stores.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211643 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-24 23:33:04 +00:00
Matt Arsenault
95eb45c5d9 R600: Fix inconsistency in rsq instructions.
R600 was using a clamped version of rsq, but SI was not. Add a
new rsq_clamped intrinsic and use them consistently.

It's unclear to me from the documentation what behavior
the R600 instructions have, so I assume they have the legacy behavior
described by the SI documents. For R600, use RECIPSQRT_IEEE
for both llvm.AMDGPU.rsq.legacy and llvm.AMDGPU.rsq. R600 also
has RECIPSQRT_FF, which I'm not sure how it fits in here.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211637 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-24 22:13:39 +00:00
Matt Arsenault
a91ff54e43 R600: Remove DIV_INF
This corresponded to an amdil instruction which there is
a 2 instruction equivalent for.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211616 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-24 17:42:16 +00:00
Matt Arsenault
ed143b7c0c R600/SI: Fix div_scale intrinsic.
The operand that must match one of the others does matter,
and implement selecting for it.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211523 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-23 18:28:28 +00:00
Matt Arsenault
791c054391 R600: Remove AMDILISelLowering
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211519 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-23 18:00:55 +00:00
Matt Arsenault
6272302394 R600: Select is not expensive.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211518 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-23 18:00:52 +00:00
Matt Arsenault
9ad2c7ef92 R600: Move add/sub with overflow out of AMDILISelLowering
Add more tests for these.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211517 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-23 18:00:49 +00:00
Matt Arsenault
5f607c0b39 R600: Move more out of AMDILISelLowering
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211516 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-23 18:00:44 +00:00
Matt Arsenault
c4471e9248 R600/SI: Handle i64 sub.
We can handle it the same way as add

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211514 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-23 18:00:38 +00:00
Matt Arsenault
b170a8042a R600: Rename AMDIL file
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211512 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-23 18:00:31 +00:00
Jan Vesely
ddf2a7902a R600: Use LowerSDIVREM for i64 node replace
v2: move div/rem node replacement to R600ISelLowering
    make lowerSDIVREM protected

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211478 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-22 21:43:01 +00:00
Jan Vesely
cd88535ab9 R600: Implement custom SDIVREM.
Instead of separate SDIV/SREM. SDIV used UDIV which in turn used UDIVREM anyway.
SREM used SDIV(UDIV->UDIVREM)+MUL+SUB, using UDIVREM directly is more efficient.

v2: Don't use all caps names

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211477 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-22 21:43:00 +00:00
Tom Stellard
61d64acd0c R600/SI: Add a pattern for f32 ftrunc
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211377 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-20 17:06:09 +00:00
Tom Stellard
2cda6e8ca6 R600: Expand vector flog2
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211376 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-20 17:06:07 +00:00
Tom Stellard
2d245e2da4 R600: Expand vector fexp2
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211375 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-20 17:06:05 +00:00
Matt Arsenault
d9b35435b8 R600/SI: Add intrinsics for various math instructions.
These will be used for custom lowering and for library
implementations of various math functions, so it's useful
to expose these as builtins.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211247 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-19 01:19:19 +00:00
Matt Arsenault
2ea6d93c5e Use stdint macros for specifying size of constants
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211231 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 22:11:03 +00:00
Matt Arsenault
ce09bda96e R600: Handle fnearbyint
The difference from rint isn't really relevant here,
so treat them as equivalent. OpenCL doesn't have nearbyint,
so this is sort of pointless other than for completeness.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211229 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 22:03:45 +00:00
Matt Arsenault
311ea12506 Use LL suffix for literal that should be 64-bits.
This hopefully fixes Windows

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211225 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 21:40:43 +00:00
Jan Vesely
52b6c2d6ef R600: Expand vector fceil
Move fp64 fceil tests to fceil64.ll

v2: rebase

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211194 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 17:57:29 +00:00
Matt Arsenault
4380c61415 Work around ridiculous warning.
Apparently C++ doesn't really have hex floating point constants.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211192 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 17:45:58 +00:00
Matt Arsenault
2b6e6fc1a8 R600/SI: Add intrinsics for brev instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211187 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 17:13:57 +00:00
Matt Arsenault
debd831223 R600: Implement f64 ftrunc, ffloor and fceil.
CI has instructions for these, so this fixes them for older hardware.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211183 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 17:05:30 +00:00
Matt Arsenault
a5395c03f0 R600: Custom lower f64 frint for pre-CI
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211182 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-18 17:05:26 +00:00
Matt Arsenault
62e378b057 R600/SI: Match ctlz_zero_undef
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211115 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-17 17:36:24 +00:00
Tom Stellard
f56e7678d1 R600: Use LDS and vectors for private memory
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211110 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-17 16:53:14 +00:00
Tom Stellard
bae98b1b45 SelectionDAG: Expand i64 = FP_TO_SINT i32
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211108 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-17 16:53:07 +00:00
Matt Arsenault
f9ef37a986 Fix copy paste error
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211003 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-15 21:22:52 +00:00
Matt Arsenault
fa848ccd09 R600: Remove a few more things from AMDILISelLowering
Try to keep all the setOperationActions for integer ops
together.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211001 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-15 21:08:58 +00:00
Matt Arsenault
e2480a202f R600: Fix assert on vector sdiv
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211000 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-15 21:08:54 +00:00
Matt Arsenault
62f6ab7a6d R600: Move / cleanup more leftover AMDIL stuff.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210998 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-15 20:23:38 +00:00
Matt Arsenault
57177e3361 R600: Move division custom lowering out of AMDILISelLowering
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210997 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-15 20:08:02 +00:00
Matt Arsenault
61bfbc4d96 R600: Report that integer division is expensive.
Divides by weird constants now emit much better code.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210995 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-15 19:48:16 +00:00
Matt Arsenault
aac455af99 R600: Fix asserts related to constant initializers
This would assert if a constant address space was extern
and therefore didn't have an initializer. If the initializer
was undef, it would hit the unreachable unhandled initializer case.

An extern global should never really occur since we don't have
machine linking, but bugpoint likes to remove initializers.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210967 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-14 04:26:05 +00:00
Matt Arsenault
15f474c925 R600: Use address space enum instead of value
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210966 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-14 04:26:01 +00:00
Matt Arsenault
f317d86f8f R600: Cleanup some old AMDIL stuff.
Move / delete some of the more obviously wrong
setOperationAction calls. Most of these are setting Expand
for types that aren't legal which is the default anyway.
Leave stuff that might require more thought on whether it's
junk or not as it is.

No functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210922 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-13 17:20:53 +00:00
Matt Arsenault
d344c6bcf9 R600/SI: Fix selection error on i64 rotl / rotr.
Evergreen is still broken due to missing shl_parts.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210885 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-13 04:00:30 +00:00
Matt Arsenault
00c3986254 R600: Mostly remove remaining AMDIL intrinsics.
Delete all unused ones, and add new AMDGPU named intrinsics for
the ones that are. Handle the old AMDIL names for comptability (although
remove their GCCBuiltin names) and add tests since there weren't any
for these before.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210827 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-12 21:15:44 +00:00
Matt Arsenault
8a9df8f92c R600/SI: Use v_cvt_f32_ubyte* instructions
This eliminates extra extract instructions when loading an i8 vector to
a float vector.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210666 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-11 17:50:44 +00:00