Commit Graph

25 Commits

Author SHA1 Message Date
Andrea Di Biagio
60e9a53c21 [CostModel][x86] Improved cost model for alternate shuffles.
This patch:
 1) Improves the cost model for x86 alternate shuffles (originally
added at revision 211339);
 2) Teaches the Cost Model Analysis pass how to analyze alternate shuffles.

Alternate shuffles are a special kind of blend; on x86, we can often
easily lowered alternate shuffled into single blend
instruction (depending on the subtarget features).

The existing cost model didn't take into account subtarget features.
Also, it had a couple of "dead" entries for vector types that are never
legal (example: on x86 types v2i32 and v2f32 are not legal; those are
always either promoted or widened to 128-bit vector types).

The new x86 cost model takes into account what target features we have
before returning the shuffle cost (i.e. the number of instructions
after the blend is lowered/expanded).

This patch also teaches the Cost Model Analysis how to identify and analyze
alternate shuffles (i.e. 'SK_Alternate' shufflevector instructions):
 - added function 'isAlternateVectorMask';
 - added some logic to check if an instruction is a alternate shuffle and, in
   case, call the target specific TTI to get the corresponding shuffle cost;
 - added a test to verify the cost model analysis on alternate shuffles.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212296 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-03 22:24:18 +00:00
Chandler Carruth
4da253756d [Modules] Fix potential ODR violations by sinking the DEBUG_TYPE
definition below all the header #include lines, lib/Analysis/...
edition.

This one has a bit extra as there were *other* #define's before #include
lines in addition to DEBUG_TYPE. I've sunk all of them as a block.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206843 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-22 02:48:03 +00:00
Benjamin Kramer
561edae834 Remove a couple of redundant copies of SmallVector::operator==.
No functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206635 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-18 19:48:03 +00:00
Craig Topper
570e52c6f1 [C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206243 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-15 04:59:12 +00:00
Craig Topper
c37e6c0734 [C++11] Add 'override' keyword to virtual methods that override their base class.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202945 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-05 07:30:04 +00:00
Benjamin Kramer
a4f0aad951 [C++11] Replace llvm::tie with std::tie.
The old implementation is no longer needed in C++11.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202644 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-02 13:30:33 +00:00
Benjamin Kramer
8ed2c8e99f Reduce code duplication resulting from the ConstantVector/ConstantDataVector split.
No intended functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201344 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-13 16:48:38 +00:00
Andrea Di Biagio
029a76b0a2 [Vectorizer] Add a new 'OperandValueKind' in TargetTransformInfo called
'OK_NonUniformConstValue' to identify operands which are constants but
not constant splats.

The cost model now allows returning 'OK_NonUniformConstValue'
for non splat operands that are instances of ConstantVector or
ConstantDataVector.

With this change, targets are now able to compute different costs
for instructions with non-uniform constant operands.
For example, On X86 the cost of a vector shift may vary depending on whether
the second operand is a uniform or non-uniform constant.

This patch applies the following changes:
 - The cost model computation now takes into account non-uniform constants;
 - The cost of vector shift instructions has been improved in
   X86TargetTransformInfo analysis pass;
 - BBVectorize, SLPVectorizer and LoopVectorize now know how to distinguish
   between non-uniform and uniform constant operands.

Added a new test to verify that the output of opt
'-cost-model -analyze' is valid in the following configurations: SSE2,
SSE4.1, AVX, AVX2.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201272 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-12 23:43:47 +00:00
Matt Arsenault
4bcb85658d Get right cost for addrspacecast in cost model
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199833 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-22 20:30:16 +00:00
Eric Christopher
2bc3cd83b3 Move variable into assert to avoid unused variable warning.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190886 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-17 21:13:57 +00:00
Arnold Schwaighofer
65457b679a Costmodel: Add support for horizontal vector reductions
Upcoming SLP vectorization improvements will want to be able to estimate costs
of horizontal reductions. Add infrastructure to support this.

We model reductions as a series of (shufflevector,add) tuples ultimately
followed by an extractelement. For example, for an add-reduction of <4 x float>
we could generate the following sequence:

 (v0, v1, v2, v3)
   \   \  /  /
     \  \  /
       +  +

 (v0+v2, v1+v3, undef, undef)
    \      /
 ((v0+v2) + (v1+v3), undef, undef)

 %rdx.shuf = shufflevector <4 x float> %rdx, <4 x float> undef,
                           <4 x i32> <i32 2, i32 3, i32 undef, i32 undef>
 %bin.rdx = fadd <4 x float> %rdx, %rdx.shuf
 %rdx.shuf7 = shufflevector <4 x float> %bin.rdx, <4 x float> undef,
                          <4 x i32> <i32 1, i32 undef, i32 undef, i32 undef>
 %bin.rdx8 = fadd <4 x float> %bin.rdx, %rdx.shuf7
 %r = extractelement <4 x float> %bin.rdx8, i32 0

This commit adds a cost model interface "getReductionCost(Opcode, Ty, Pairwise)"
that will allow clients to ask for the cost of such a reduction (as backends
might generate more efficient code than the cost of the individual instructions
summed up). This interface is excercised by the CostModel analysis pass which
looks for reduction patterns like the one above - starting at extractelements -
and if it sees a matching sequence will call the cost model interface.

We will also support a second form of pairwise reduction that is well supported
on common architectures (haddps, vpadd, faddp).

 (v0, v1, v2, v3)
  \   /    \  /
 (v0+v1, v2+v3, undef, undef)
    \     /
 ((v0+v1)+(v2+v3), undef, undef, undef)

  %rdx.shuf.0.0 = shufflevector <4 x float> %rdx, <4 x float> undef,
        <4 x i32> <i32 0, i32 2 , i32 undef, i32 undef>
  %rdx.shuf.0.1 = shufflevector <4 x float> %rdx, <4 x float> undef,
        <4 x i32> <i32 1, i32 3, i32 undef, i32 undef>
  %bin.rdx.0 = fadd <4 x float> %rdx.shuf.0.0, %rdx.shuf.0.1
  %rdx.shuf.1.0 = shufflevector <4 x float> %bin.rdx.0, <4 x float> undef,
        <4 x i32> <i32 0, i32 undef, i32 undef, i32 undef>
  %rdx.shuf.1.1 = shufflevector <4 x float> %bin.rdx.0, <4 x float> undef,
        <4 x i32> <i32 1, i32 undef, i32 undef, i32 undef>
  %bin.rdx.1 = fadd <4 x float> %rdx.shuf.1.0, %rdx.shuf.1.1
  %r = extractelement <4 x float> %bin.rdx.1, i32 0

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190876 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-17 18:06:50 +00:00
Craig Topper
9e639e8fd9 Use SmallVectorImpl& instead of SmallVector to avoid repeating small vector size.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186098 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-11 16:22:38 +00:00
Craig Topper
2ebba647ea Fix indentation. No functional change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186065 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-11 05:39:44 +00:00
Arnold Schwaighofer
6bf4f67641 CostModel: Add parameter to instruction cost to further classify operand values
On certain architectures we can support efficient vectorized version of
instructions if the operand value is uniform (splat) or a constant scalar.
An example of this is a vector shift on x86.

We can efficiently support

for (i = 0 ; i < ; i += 4)
  w[0:3] = v[0:3] << <2, 2, 2, 2>

but not

for (i = 0; i < ; i += 4)
  w[0:3] = v[0:3] << x[0:3]

This patch adds a parameter to getArithmeticInstrCost to further qualify operand
values as uniform or uniform constant.

Targets can then choose to return a different cost for instructions with such
operand values.

A follow-up commit will test this feature on x86.

radar://13576547

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@178807 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-04 23:26:21 +00:00
Benjamin Kramer
8611d4449a Cost model support for lowered math builtins.
We make the cost for calling libm functions extremely high as emitting the
calls is expensive and causes spills (on x86) so performance suffers. We still
vectorize important calls like ceilf and friends on SSE4.1. and fabs.

Differential Revision: http://llvm-reviews.chandlerc.com/D466

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176287 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-28 19:09:33 +00:00
Arnold Schwaighofer
4fc6484ee2 Cost model: Add check for reverse shuffles to CostModel analysis
Check for reverse shuffles in the CostModel analysis pass and query
TargetTransform info accordingly. This allows us we can write test cases for
reverse shuffles.

radar://13171406

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174932 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-12 02:40:37 +00:00
Arnold Schwaighofer
fb55a8fd7c ARM cost model: Address computation in vector mem ops not free
Adds a function to target transform info to query for the cost of address
computation. The cost model analysis pass now also queries this interface.
The code in LoopVectorize adds the cost of address computation as part of the
memory instruction cost calculation. Only there, we know whether the instruction
will be scalarized or not.
Increase the penality for inserting in to D registers on swift. This becomes
necessary because we now always assume that address computation has a cost and
three is a closer value to the architecture.

radar://13097204

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174713 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-08 14:50:48 +00:00
Chandler Carruth
be04929f7f Move TargetTransformInfo to live under the Analysis library. This no
longer would violate any dependency layering and it is in fact an
analysis. =]

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171686 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-07 03:08:10 +00:00
Chandler Carruth
194bd71b06 Switch the cost model analysis over to just the TTI interface.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171619 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-05 10:09:33 +00:00
Chandler Carruth
0b8c9a80f2 Move all of the header files which are involved in modelling the LLVM IR
into their new header subdirectory: include/llvm/IR. This matches the
directory structure of lib, and begins to correct a long standing point
of file layout clutter in LLVM.

There are still more header files to move here, but I wanted to handle
them in separate commits to make tracking what files make sense at each
layer easier.

The only really questionable files here are the target intrinsic
tablegen files. But that's a battle I'd rather not fight today.

I've updated both CMake and Makefile build systems (I think, and my
tests think, but I may have missed something).

I've also re-sorted the includes throughout the project. I'll be
committing updates to Clang, DragonEgg, and Polly momentarily.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171366 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-02 11:36:10 +00:00
Nadav Rotem
99b7a99376 Update the docs of the cost model.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171016 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-24 05:51:12 +00:00
Nadav Rotem
e70b2680a8 constify the cost API
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169172 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-03 22:47:12 +00:00
Chandler Carruth
d04a8d4b33 Use the new script to sort the includes of every file under lib.
Sooooo many of these had incorrect or strange main module includes.
I have manually inspected all of these, and fixed the main module
include to be the nearest plausible thing I could find. If you own or
care about any of these source files, I encourage you to take some time
and check that these edits were sensible. I can't have broken anything
(I strictly added headers, and reordered them, never removed), but they
may not be the headers you'd really like to identify as containing the
API being implemented.

Many forward declarations and missing includes were added to a header
files to allow them to parse cleanly when included first. The main
module rule does in fact have its merits. =]

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169131 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-03 16:50:05 +00:00
Nadav Rotem
f149567160 CostModel: add support for Vector Insert and Extract.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167329 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-02 22:31:56 +00:00
Nadav Rotem
6bed58ef24 Add a cost model analysis that allows us to estimate the cost of IR-level instructions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167324 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-02 21:48:17 +00:00