Commit Graph

102 Commits

Author SHA1 Message Date
Justin Holewinski
2941802bad [NVPTX] Add some extra tests for mul.wide to test non-power-of-two source types
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213794 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-23 20:23:49 +00:00
Justin Holewinski
03f160f9d3 [NVPTX] mul.wide generation works for any smaller integer source types, not just the next smaller power of two
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213784 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-23 18:46:03 +00:00
Justin Holewinski
6b50a7d179 [NVPTX] Make sure we do not generate MULWIDE ISD nodes when optimizations are disabled
With optimizations disabled, we disable the isel patterns for mul.wide; but we
were still generating MULWIDE ISD nodes.  Now, we only try to generate MULWIDE
ISD nodes in DAGCombine if the optimization level is not zero.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213773 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-23 17:40:45 +00:00
Eli Bendersky
07efe87198 Add some tests for NVPTX lowering of cmpxchg
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213586 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-21 22:54:44 +00:00
Eli Bendersky
bd9c6c0773 Add tests for atomic adds on floats.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213406 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-18 20:11:26 +00:00
Eli Bendersky
6a33571ba9 Use CHECK-LABEL where appropriate in this test.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213398 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-18 19:32:09 +00:00
Tim Northover
b41b1d4bac NVPTX: support fpext/fptrunc to and from f16.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213377 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-18 13:01:43 +00:00
Tim Northover
0afed03229 CodeGen: soften f16 type by default instead of marking legal.
Actual support for softening f16 operations is still limited, and can be added
when it's needed.  But Soften is much closer to being a useful thing to try
than keeping it Legal when no registers can actually hold such values.

Longer term, we probably want something between Soften and Promote semantics
for most targets, it'll be more efficient to promote the 4 basic operations to
f32 than libcall them.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213372 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-18 12:41:46 +00:00
Tim Northover
7bbf5786d7 NVPTX: support direct f16 <-> f64 conversions via intrinsics.
Clang may well start emitting these soon, and while it may not be
directly relevant for OpenCL or GLSL, the instructions were just
sitting there waiting to be used.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213356 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-18 08:30:10 +00:00
Justin Holewinski
11ae250ec9 [NVPTX] Improve handling of FP fusion
We now consider the FPOpFusion flag when determining whether
to fuse ops.  We also explicitly emit add.rn when fusion is
disabled to prevent ptxas from fusing the operations on its
own.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213287 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-17 18:10:09 +00:00
Justin Holewinski
07bc0b6ae6 [NVPTX] Add missing .v4 qualifier on vector store instruction
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213276 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-17 16:58:56 +00:00
Justin Holewinski
b26ede07ad [NVPTX] Flag surface/texture query instructions with IsTexSurfQuery
Also, add some tests to make sure we can handle surface/texture
queries on both Fermi and Kepler+.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213268 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-17 14:51:33 +00:00
Justin Holewinski
d6663f565c [NVPTX] Add more surface/texture intrinsics, including CUDA unified texture fetch
This also uses TSFlags to mark machine instructions that are surface/texture
accesses, as well as the vector width for surface operations.  This is used
to simplify some of the switch statements that need to detect surface/texture
instructions

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213256 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-17 11:59:04 +00:00
Justin Holewinski
a1535e3b9b [NVPTX] Honor alignment on vector loads/stores
We were not considering the stated alignment on vector loads/stores,
leading us to generate vector instructions even when we do not have
sufficient alignment.

Now, for IR like:

  %1 = load <4 x float>, <4 x float>* %ptr, align 4

we will generate correct, conservative PTX like:

  ld.f32 ... [%ptr]
  ld.f32 ... [%ptr+4]
  ld.f32 ... [%ptr+8]
  ld.f32 ... [%ptr+12]

Or if we have an alignment of 8 (for example), we can
generate code like:

  ld.v2.f32 ... [%ptr]
  ld.v2.f32 ... [%ptr+8]

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213186 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-16 19:45:35 +00:00
Justin Holewinski
7e6565112b [NVPTX] Rename registers %fl -> %fd and %rl -> %rd
This matches the internal behavior of NVIDIA tools like libnvvm.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213168 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-16 16:26:58 +00:00
Justin Holewinski
7a28de08f3 [NVPTX] Add reflect intrinsic (better than matching by function name)
Also clean up some of the logic in NVVMReflect.cpp while we're messing around in there.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211948 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-27 18:36:11 +00:00
Justin Holewinski
c95d327874 [NVPTX] Add 'b' asm constraint
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211946 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-27 18:36:06 +00:00
Justin Holewinski
3c81367a5d [NVPTX] Error out if initializer is given for variable in an address space that does not support initialization
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211943 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-27 18:36:01 +00:00
Justin Holewinski
0ded57ccc5 [NVPTX] Add support for .managed variables for UVM
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211942 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-27 18:35:58 +00:00
Justin Holewinski
2a8dc35cca [NVPTX] Emit .weak linkage for link_once, weak, available_externally, and common linkage
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211941 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-27 18:35:56 +00:00
Justin Holewinski
cb8f98382b [NVPTX] Fix handling of ldg/ldu intrinsics.
The address space of the pointer must be global (1) for these intrinsics.  There must also be alignment metadata attached to the intrinsic calls, e.g.

%val = tail call i32 @llvm.nvvm.ldu.i.global.i32.p1i32(i32 addrspace(1)* %ptr), !align !0

!0 = metadata !{i32 4}

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211939 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-27 18:35:51 +00:00
Justin Holewinski
8992274412 [NVPTX] Clean up argument lowering code and properly handle alignment for structs and vectors
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211938 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-27 18:35:44 +00:00
Justin Holewinski
863b0d45a5 [NVPTX] Add support for [SHL,SRA,SRL]_PARTS
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211936 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-27 18:35:40 +00:00
Justin Holewinski
10da1651ed [NVPTX] Implement fma and imad contraction as target DAGCombiner patterns
This also introduces DAGCombiner patterns for mul.wide to multiply two smaller integers and produce a larger integer

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211935 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-27 18:35:37 +00:00
Justin Holewinski
508c80f11f [NVPTX] Add support for efficient rotate instructions on SM 3.2+
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211934 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-27 18:35:33 +00:00
Justin Holewinski
1f75f4a0ee [NVPTX] Add missing isel patterns for 64-bit atomics
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211933 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-27 18:35:30 +00:00
Justin Holewinski
ef92cf50d6 [NVPTX] Add isel patterns for bit-field extract (bfe)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211932 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-27 18:35:27 +00:00
Justin Holewinski
de7bbdff33 [NVPTX] Add support for isspacep instruction
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211931 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-27 18:35:24 +00:00
Justin Holewinski
1571d272c8 [NVPTX] Add support for envreg reads
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211930 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-27 18:35:21 +00:00
Justin Holewinski
a54609ed93 [NVPTX] Emit .weak when linkage is not external, internal, or private
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211926 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-27 18:35:10 +00:00
Jingyue Wu
f6eb7e3175 Canonicalize addrspacecast ConstExpr between different pointer types
As a follow-up to r210375 which canonicalizes addrspacecast
instructions, this patch canonicalizes addrspacecast constant
expressions.

Given clang uses ConstantExpr::getAddrSpaceCast to emit addrspacecast
cosntant expressions, this patch is also a step towards having the
frontend emit canonicalized addrspacecasts.

Piggyback a minor refactor in InstCombineCasts.cpp

Update three affected tests in addrspacecast-alias.ll,
access-non-generic.ll and constant-fold-gep.ll and added one new test in
constant-fold-address-space-pointer.ll


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211004 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-15 21:40:57 +00:00
Alp Toker
8aeca44558 Reduce verbiage of lit.local.cfg files
We can just split targets_to_build in one place and make it immutable.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210496 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-09 22:42:55 +00:00
Rafael Espindola
6fd1b8ee48 Allow aliases to be unnamed_addr.
Alias with unnamed_addr were in a strange state. It is stored in GlobalValue,
the language reference talks about "unnamed_addr aliases" but the verifier
was rejecting them.

It seems natural to allow unnamed_addr in aliases:

* It is a property of how it is accessed, not of the data itself.
* It is perfectly possible to write code that depends on the address
of an alias.

This patch then makes unname_addr legal for aliases. One side effect is that
the syntax changes for a corner case: In globals, unnamed_addr is now printed
before the address space.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210302 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-06 01:20:28 +00:00
Eli Bendersky
85af3e7445 Fix the test: DCE optimized away everything.
Use volatile store to protect the generated PTX from DCE.

Patch by Jingyue Wu.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206763 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-21 17:23:12 +00:00
Justin Holewinski
77f268945e [NVPTX] Add preliminary intrinsics and codegen support for textures/surfaces
This commit adds intrinsics and codegen support for the surface read/write and texture read instructions that take an explicit sampler parameter. Codegen operates on image handles at the PTX level, but falls back to direct replacement of handles with kernel arguments if image handles are not enabled. Note that image handles are explicitly disabled for all target architectures in this change (to be enabled later).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205907 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-09 15:39:15 +00:00
Justin Holewinski
ac4c131de6 [NVPTX] Add support for addrspacecast in global variable initializers, including emitting generic() when casting to address space 0.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205906 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-09 15:39:11 +00:00
Eli Bendersky
25540a7f39 Optimize away unnecessary address casts.
Removes unnecessary casts from non-generic address spaces to the generic address
space for certain code patterns.

Patch by Jingyue Wu.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205571 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-03 21:18:25 +00:00
Eli Bendersky
416a0e993f Fix for PR19099 - NVPTX produces invalid symbol names.
This is a more thorough fix for the issue than r203483. An IR pass will run
before NVPTX codegen to make sure there are no invalid symbol names that can't
be consumed by the ptxas assembler.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205212 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-31 15:56:26 +00:00
Eli Bendersky
b6f02c13da Add test to test/CodeGen/NVPTX for "alloca buffer" arguments.
Make sure such IR gets properly lowered to PTX.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204624 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-24 16:52:30 +00:00
Justin Holewinski
b8cb709858 [NVPTX] Add isel patterns for addrspacecast
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204600 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-24 11:17:53 +00:00
Eli Bendersky
21354ec60d Expose "noduplicate" attribute as a property for intrinsics.
The "noduplicate" function attribute exists to prevent certain optimizations
from duplicating calls to the function. This is important on platforms where
certain function call duplications are unsafe (for example execution barriers
for CUDA and OpenCL).

This patch makes it possible to specify intrinsics as "noduplicate" and
translates that to the appropriate function attribute.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204200 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-18 23:51:07 +00:00
Rafael Espindola
1f21e0dd0d Remove the linker_private and linker_private_weak linkages.
These linkages were introduced some time ago, but it was never very
clear what exactly their semantics were or what they should be used
for. Some investigation found these uses:

* utf-16 strings in clang.
* non-unnamed_addr strings produced by the sanitizers.

It turns out they were just working around a more fundamental problem.
For some sections a MachO linker needs a symbol in order to split the
section into atoms, and llvm had no idea that was the case. I fixed
that in r201700 and it is now safe to use the private linkage. When
the object ends up in a section that requires symbols, llvm will use a
'l' prefix instead of a 'L' prefix and things just work.

With that, these linkages were already dead, but there was a potential
future user in the objc metadata information. I am still looking at
CGObjcMac.cpp, but at this point I am convinced that linker_private
and linker_private_weak are not what they need.

The objc uses are currently split in

* Regular symbols (no '\01' prefix). LLVM already directly provides
whatever semantics they need.
* Uses of a private name (start with "\01L" or "\01l") and private
linkage. We can drop the "\01L" and "\01l" prefixes as soon as llvm
agrees with clang on L being ok or not for a given section. I have two
patches in code review for this.
* Uses of private name and weak linkage.

The last case is the one that one could think would fit one of these
linkages. That is not the case. The semantics are

* the linker will merge these symbol by *name*.
* the linker will hide them in the final DSO.

Given that the merging is done by name, any of the private (or
internal) linkages would be a bad match. They allow llvm to rename the
symbols, and that is really not what we want. From the llvm point of
view, these objects should really be (linkonce|weak)(_odr)?.

For now, just keeping the "\01l" prefix is probably the best for these
symbols. If we one day want to have a more direct support in llvm,
IMHO what we should add is not a linkage, it is just a hidden_symbol
attribute. It would be applicable to multiple linkages. For example,
on weak it would produce the current behavior we have for objc
metadata. On internal, it would be equivalent to private (and we
should then remove private).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@203866 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-13 23:18:37 +00:00
Eli Bendersky
428b609de3 Followup to r203483 - add test.
[forgot to 'svn add' before committing r203483]


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@203485 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-10 20:36:04 +00:00
Gautam Chakrabarti
3dcb2a2d92 [NVPTX] Fix emitting aggregate parameters
The code was missing the case for aggregate parameters and
hence was emitting them as .b0 type. Also fixed a couple
of comments.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200325 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-28 18:35:29 +00:00
Justin Holewinski
c93ed1707a [NVPTX] Add missing patterns for div.approx with immediate denominator
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199746 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-21 14:40:05 +00:00
Nico Rieck
3c1dc3cac8 Fix non-deterministic SDNodeOrder-dependent codegen
Reset SelectionDAGBuilder's SDNodeOrder to ensure deterministic code
generation.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199050 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-12 14:09:17 +00:00
Justin Holewinski
7add5421a6 [NVPTX] Fix off-by-one error when creating the VT list for an SDNode
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@196503 91177308-0d34-0410-b5e6-96231b3b80d8
2013-12-05 12:58:00 +00:00
Justin Holewinski
4d748eb0e4 [NVPTX] Fix handling of indirect calls
Using a special machine node is cleaner than an InlineAsm node, and fixes an assertion failure in InstrEmitter

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194810 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-15 12:30:04 +00:00
Justin Holewinski
07a5d3dc3d [NVPTX] Properly handle bitcast ConstantExpr when checking for the alignment of function parameters
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194410 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-11 19:28:19 +00:00
Justin Holewinski
0df3ce333b [NVPTX] Fix logic error in loading vector parameters of more than 4 components
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194409 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-11 19:28:16 +00:00