Upcoming SLP vectorization improvements will want to be able to estimate costs
of horizontal reductions. Add infrastructure to support this.
We model reductions as a series of (shufflevector,add) tuples ultimately
followed by an extractelement. For example, for an add-reduction of <4 x float>
we could generate the following sequence:
(v0, v1, v2, v3)
\ \ / /
\ \ /
+ +
(v0+v2, v1+v3, undef, undef)
\ /
((v0+v2) + (v1+v3), undef, undef)
%rdx.shuf = shufflevector <4 x float> %rdx, <4 x float> undef,
<4 x i32> <i32 2, i32 3, i32 undef, i32 undef>
%bin.rdx = fadd <4 x float> %rdx, %rdx.shuf
%rdx.shuf7 = shufflevector <4 x float> %bin.rdx, <4 x float> undef,
<4 x i32> <i32 1, i32 undef, i32 undef, i32 undef>
%bin.rdx8 = fadd <4 x float> %bin.rdx, %rdx.shuf7
%r = extractelement <4 x float> %bin.rdx8, i32 0
This commit adds a cost model interface "getReductionCost(Opcode, Ty, Pairwise)"
that will allow clients to ask for the cost of such a reduction (as backends
might generate more efficient code than the cost of the individual instructions
summed up). This interface is excercised by the CostModel analysis pass which
looks for reduction patterns like the one above - starting at extractelements -
and if it sees a matching sequence will call the cost model interface.
We will also support a second form of pairwise reduction that is well supported
on common architectures (haddps, vpadd, faddp).
(v0, v1, v2, v3)
\ / \ /
(v0+v1, v2+v3, undef, undef)
\ /
((v0+v1)+(v2+v3), undef, undef, undef)
%rdx.shuf.0.0 = shufflevector <4 x float> %rdx, <4 x float> undef,
<4 x i32> <i32 0, i32 2 , i32 undef, i32 undef>
%rdx.shuf.0.1 = shufflevector <4 x float> %rdx, <4 x float> undef,
<4 x i32> <i32 1, i32 3, i32 undef, i32 undef>
%bin.rdx.0 = fadd <4 x float> %rdx.shuf.0.0, %rdx.shuf.0.1
%rdx.shuf.1.0 = shufflevector <4 x float> %bin.rdx.0, <4 x float> undef,
<4 x i32> <i32 0, i32 undef, i32 undef, i32 undef>
%rdx.shuf.1.1 = shufflevector <4 x float> %bin.rdx.0, <4 x float> undef,
<4 x i32> <i32 1, i32 undef, i32 undef, i32 undef>
%bin.rdx.1 = fadd <4 x float> %rdx.shuf.1.0, %rdx.shuf.1.1
%r = extractelement <4 x float> %bin.rdx.1, i32 0
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190876 91177308-0d34-0410-b5e6-96231b3b80d8
We can't insert an insertelement after an invoke. We would have to split a
critical edge. So when we see a phi node that uses an invoke we just give up.
radar://14990770
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190871 91177308-0d34-0410-b5e6-96231b3b80d8
other in memory.
The motivation was to get rid of truncate and shift right instructions that get
in the way of paired load or floating point load.
E.g.,
Consider the following example:
struct Complex {
float real;
float imm;
};
When accessing a complex, llvm was generating a 64-bits load and the imm field
was obtained by a trunc(lshr) sequence, resulting in poor code generation, at
least for x86.
The idea is to declare that two load instructions is the canonical form for
loading two arithmetic type, which are next to each other in memory.
Two scalar loads at a constant offset from each other are pretty
easy to detect for the sorts of passes that like to mess with loads.
<rdar://problem/14477220>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190870 91177308-0d34-0410-b5e6-96231b3b80d8
Add llvm.x86.* intrinsics for all of the Intel SHA Extensions instructions, as
well as tests. Also remove mayLoad and hasSideEffects, which can be inferred
from the instruction patterns.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190864 91177308-0d34-0410-b5e6-96231b3b80d8
When a truncate node defines a legal vector type but uses an illegal
vector type, the legalization process was splitting the vector until
<1 x vector> type, but then it was failing to scalarize the node because
it did not know how to handle TRUNCATE.
<rdar://problem/14989896>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190830 91177308-0d34-0410-b5e6-96231b3b80d8
A DBG_VALUE is register-indirect iff the first operand is a register
_and_ the second operand is an immediate.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190821 91177308-0d34-0410-b5e6-96231b3b80d8
If there are no legal integers, assume 1 byte.
This makes more sense than using the pointer size as
a guess for the maximum GPR width.
It is conceivable to want to use some 64-bit pointers
on a target where 64-bit integers aren't legal.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190817 91177308-0d34-0410-b5e6-96231b3b80d8
- check that -mcpu=slm uses the call register indirect optimization
- check that -mcpu=slm runs the scheduler
- check that -mcpu=slm supports the movbe instruction
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190814 91177308-0d34-0410-b5e6-96231b3b80d8
I got a report of a hang in git's helper functions trying to figure out
how to display results of "git svn info" when run inside ninja, even though
the result is immediately piped to grep. This seems to avoid that.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190808 91177308-0d34-0410-b5e6-96231b3b80d8
Fast-isel generates a COPY_TO_REGCLASS for widening f32 to f64, which
is a nop on PPC64. This is needed to keep the register class system
happy, but on the fast-isel path it is not removed before emit as it
is for DAG select. Ignore this op when emitting instructions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190795 91177308-0d34-0410-b5e6-96231b3b80d8
TableGen was sorting the entries in some of its internal data
structures by pointer. This order filtered through to the final
matching table and affected the diagnostics produced on bad assembly
occasionally.
It also turns out STL algorithms are ridiculously easy to misuse on
containers with custom order methods. (No bugs before, or now that I
know of, but plenty in the middle).
This should fix the sanitizer bot, which ends up with weird pointers.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190793 91177308-0d34-0410-b5e6-96231b3b80d8
We would have to compute the pre increment value, either by computing it on
every loop iteration or by splitting the edge out of the loop and inserting a
computation for it there.
For now, just give up vectorizing such loops.
Fixes PR17179.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190790 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
When a git repository had multiple remotes, ${repository} will be set to a multiline string. This causes compilation errors in SVNVersion.inc.
Fix this by limiting the output of utils/GetRepositoryPath to the first remote (which is reasonably likely to be 'origin').
Reviewers: jordan_rose
CC: llvm-commits, t.p.northover
Differential Revision: http://llvm-reviews.chandlerc.com/D1659
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190778 91177308-0d34-0410-b5e6-96231b3b80d8
The port originally had special patterns for extload, mapping them to the
same instructions as sextload. It seemed neater to have patterns that
match "an extension that is allowed to be signed" and "an extension that
is allowed to be unsigned".
This was originally meant to be a clean-up, but it does improve the handling
of promoted integers a little, as shown by args-06.ll.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190777 91177308-0d34-0410-b5e6-96231b3b80d8
This is a re-commit of r190764, with an extra check to make sure that we're not
performing the transformation on illegal types (a small test case has been
added for this as well).
Original commit message:
The PPC backend uses a target-specific DAG combine to turn unaligned Altivec
loads into a permutation-based sequence when possible. Unfortunately, the
target-specific DAG combine is not always called on all loads of interest
(sometimes the routines in DAGCombine call CombineTo such that the new node and
users are not added to the worklist); allowing the combine to trigger early
(before type legalization) mitigates this problem. Because the autovectorizers
only create legal vector types, I don't expect a lot of cases where this
optimization is enabled by type legalization in practice.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190771 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
The '?' flag uses the last section group if the last had a section
group. We treat combining an explicit section group and the '?' as a
hard error.
This fixes PR17198.
Reviewers: rafael, bkramer
Reviewed By: bkramer
CC: llvm-commits
Differential Revision: http://llvm-reviews.chandlerc.com/D1686
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190768 91177308-0d34-0410-b5e6-96231b3b80d8
For alignment purposes, the instruction array will always have an even
number of entries, with the final entry potentially unused (in which
case the array will be one longer than indicated by the count of unwind
codes field).
Reviewed by Anton Korobeynikov, Charles Davis and Nico Rieck.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190767 91177308-0d34-0410-b5e6-96231b3b80d8
data structures.
The Win64 EH data structures must be of type IMAGE_REL_AMD64_ADDR32NB
instead of IMAGE_REL_AMD64_ADDR32. This is easiely achieved by adding
the VK_COFF_IMGREL32 modifier to the symbol reference.
Change also references to start and end of the SEH range of a function
as offsets to start of the function.
Reviewed by Jim Grosbach, Charles Davis and Nico Rieck.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190766 91177308-0d34-0410-b5e6-96231b3b80d8
This is causing test-suite failures.
Original commit message:
The PPC backend uses a target-specific DAG combine to turn unaligned Altivec
loads into a permutation-based sequence when possible. Unfortunately, the
target-specific DAG combine is not always called on all loads of interest
(sometimes the routines in DAGCombine call CombineTo such that the new node and
users are not added to the worklist); allowing the combine to trigger early
(before type legalization) mitigates this problem. Because the autovectorizers
only create legal vector types, I don't expect a lot of cases where this
optimization is enabled by type legalization in practice.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190765 91177308-0d34-0410-b5e6-96231b3b80d8
The PPC backend uses a target-specific DAG combine to turn unaligned Altivec
loads into a permutation-based sequence when possible. Unfortunately, the
target-specific DAG combine is not always called on all loads of interest
(sometimes the routines in DAGCombine call CombineTo such that the new node and
users are not added to the worklist); allowing the combine to trigger early
(before type legalization) mitigates this problem. Because the autovectorizers
only create legal vector types, I don't expect a lot of cases where this
optimization is enabled by type legalization in practice.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190764 91177308-0d34-0410-b5e6-96231b3b80d8
DAGCombiner::isAlias can be called with SrcValue1 or SrcValue2 null, and we
can't use AA in this case (if we try, then the casting code in AA will assert).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190763 91177308-0d34-0410-b5e6-96231b3b80d8
