dd8b1015c8
Add initial support for IF-conversion. This patch implements the first 1/3,
...
which is the legality of the if-conversion transformation. The next step is to
implement the cost-model for the if-converted code as well as the
vectorization itself.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169152 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-03 21:06:35 +00:00
d04a8d4b33
Use the new script to sort the includes of every file under lib.
...
Sooooo many of these had incorrect or strange main module includes.
I have manually inspected all of these, and fixed the main module
include to be the nearest plausible thing I could find. If you own or
care about any of these source files, I encourage you to take some time
and check that these edits were sensible. I can't have broken anything
(I strictly added headers, and reordered them, never removed), but they
may not be the headers you'd really like to identify as containing the
API being implemented.
Many forward declarations and missing includes were added to a header
files to allow them to parse cleanly when included first. The main
module rule does in fact have its merits. =]
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169131 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-03 16:50:05 +00:00
d6964741f5
minor cleanups
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169048 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-30 22:37:11 +00:00
f735a7f88d
Remove the use of LPPassManager. We can remove LPM because we dont need to run any additional loop passes on the new vector loop.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169016 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-30 17:27:53 +00:00
dddaad624e
When broadcasting invariant scalars into vectors, place the broadcast code in the preheader.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168927 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-29 19:25:41 +00:00
22689b6034
Move the code that uses SCEVs prior to creating the new loops.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168601 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-26 19:51:46 +00:00
6bfc3481bd
Move the max vector width to a constant parameter. No functionality change.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168570 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-25 16:48:08 +00:00
00e8074ee2
Fix the document style.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168569 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-25 16:39:01 +00:00
8c6b73666b
Refactor the ptr runtime check generation code. No functionality change.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168568 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-25 16:27:16 +00:00
a8e2b2b68f
Rename method. No functionality change.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168560 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-25 09:13:57 +00:00
d7e0b7cdc5
The induction-pointer work is inspired by a research paper. This commit adds a reference.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168559 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-25 09:09:26 +00:00
0af63ac245
Add support for pointer induction variables even when there is no integer induction variable.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168558 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-25 08:41:35 +00:00
9a6823516f
LoopVectorizer: Add initial support for pointer induction variables (for example: *dst++ = *src++).
...
At the moment we still require to have an integer induction variable (for example: i++).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168231 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-17 00:27:03 +00:00
77d6ae1b01
LoopVectorize: Division reductions generate incorrect code. Remove the part of the code that deals with divs.
...
Thanks to Paul Redmond for catching this while reviewing the code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168142 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-16 06:51:17 +00:00
e123fd9c59
use the getSplat API. Patch by Paul Redmond.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167892 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-14 00:02:13 +00:00
9005b8d27d
Fix a comment typo and add comments.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167684 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-11 05:15:00 +00:00
b1bf1eeede
Add support for memory runtime check. When we can, we calculate array bounds.
...
If the arrays are found to be disjoint then we run the vectorized version of
the loop. If they are not, we run the scalar code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167608 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-09 07:09:44 +00:00
6e48f03077
Fix sign compare warning. Patch by Mahesha HS.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167282 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-02 05:24:00 +00:00
e57b2cbce6
LoopVectorize: Preserve NSW, NUW and IsExact flags.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167174 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31 21:40:39 +00:00
4c1b4b1fe7
Put the threshold magic number in a variable.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167134 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31 16:22:16 +00:00
2a5401cc75
Remove enum values since they are not used anymore.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167131 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31 16:14:06 +00:00
5b2c4dc5f8
LoopVectorize: Do not vectorize loops with tiny constant trip counts.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167101 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31 03:31:07 +00:00
462d1ca428
Add support for loops that don't start with Zero.
...
This is important for loops in the LAPACK test-suite.
These loops start at 1 because they are auto-converted from fortran.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167084 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31 00:45:26 +00:00
a368b89f28
Add documentation.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167055 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30 22:06:26 +00:00
e709f5b600
LoopVectorize: Add support for write-only loops when the write destination is a single pointer.
...
Speedup SciMark by 1%
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167035 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30 18:36:45 +00:00
cc03331caa
LoopVectorize: Fix a bug in the initialization of reduction variables. AND needs to start at all-one
...
while XOR, and OR need to start at zero.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167032 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30 18:12:36 +00:00
803ea1c40a
LoopVectorizer: change debug prints: Print the module identifier when deciding to vectorize. When deciding not to vectorize do not print the called function name because it can be null.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166989 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30 00:40:39 +00:00
369ff7b740
LoopVectorize: Update and preserve the dominator tree info.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166970 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-29 21:52:38 +00:00
f065a84677
1. Fix a bug in getTypeConversion. When a *simple* type is split, we need to return the type of the split result.
...
2. Change the maximum vectorization width from 4 to 8.
3. A test for both.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166864 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-27 04:11:32 +00:00
a5a3a61c5f
Refactor the VectorTargetTransformInfo interface.
...
Add getCostXXX calls for different families of opcodes, such as casts, arithmetic, cmp, etc.
Port the LoopVectorizer to the new API.
The LoopVectorizer now finds instructions which will remain uniform after vectorization. It uses this information when calculating the cost of these instructions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166836 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-26 23:49:28 +00:00
3ef9dfa685
LoopVectorize: Teach the cost model to query scalar costs as scalar types and not vectors of 1.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166715 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25 21:03:48 +00:00
8dbac7b529
Add support for additional reduction variables: AND, OR, XOR.
...
Patch by Paul Redmond <paul.redmond@intel.com >.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166649 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25 00:08:41 +00:00
2652c50f74
Implement a basic cost model for vector and scalar instructions.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166642 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24 23:47:38 +00:00
7c6de1d6e5
whitespace
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166622 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24 20:58:40 +00:00
50bec6f8c4
LoopVectorizer: Add a basic cost model which uses the VTTI interface.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166620 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24 20:36:32 +00:00
b8bce928f4
Back out r166591, not sure why this made it through since I cancelled the command. Bleh, sorry about this!
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166596 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24 17:25:11 +00:00
2f87640b86
Delete a directory that wasn't supposed to be checked in yet.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166591 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24 17:20:04 +00:00
6457001f31
Use the AliasAnalysis isIdentifiedObj because it also understands mallocs and c++ news.
...
PR14158.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166491 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-23 18:44:18 +00:00
782090aa02
Don't crash if the load/store pointer is not a GEP.
...
Fix by Shivarama Rao <Shivarama.Rao@amd.com >
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166427 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22 18:27:56 +00:00
5f7d810223
Rename a variable.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166410 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22 04:53:05 +00:00
565048e78a
Vectorizer: optimize the generation of selects. If the condition is uniform, generate a scalar-cond select (i1 as selector).
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166409 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22 04:38:00 +00:00
e0fa403e23
Update the loop vectorizer docs.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166408 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22 03:52:53 +00:00
5bb5a75b1e
Avoid an extra hash lookup when inserting a value into the widen map.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166395 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21 16:26:35 +00:00
ffe5008fd3
Simplify code. No functionality change.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166393 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21 15:36:03 +00:00
b75209cb66
Simplify code. No functionality change.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166392 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21 15:29:19 +00:00
bb950854ac
Fix a bug in the vectorization of wide load/store operations.
...
We used a SCEV to detect that A[X] is consecutive. We assumed that X was
the induction variable. But X can be any expression that uses the induction
for example: X = i + 2;
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166388 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21 06:49:10 +00:00
c847872629
Add support for reduction variables that do not start at zero.
...
This is important for nested-loop reductions such as :
In the innermost loop, the induction variable does not start with zero:
for (i = 0 .. n)
for (j = 0 .. m)
sum += ...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166387 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21 05:52:51 +00:00
f01cad69c1
Document change. Describe the pass and some papers that inspired the design of the pass.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166386 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21 04:04:25 +00:00
5a418ba5f5
Vectorizer: fix a bug in the classification of induction/reduction phis.
...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166384 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21 02:38:01 +00:00
ccaccfa8bf
Fix an infinite loop in the loop-vectorizer.
...
PR14134.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166379 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-20 20:45:01 +00:00
Nadav Rotem
Chandler Carruth
Micah Villmow
Anders Carlsson
Jakub Staszak