Commit Graph

91 Commits

Author SHA1 Message Date
Nadav Rotem
d0b144c04c Fix indentation.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170005 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-12 19:39:36 +00:00
Nadav Rotem
ae3b652f5c LoopVectorizer: Use the "optsize" attribute to decide if we are allowed to increase the function size.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170004 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-12 19:29:45 +00:00
Nadav Rotem
db65ff39fa Fix the ascii drawing that was ruined when I split the H and CPP
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169955 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-12 01:33:47 +00:00
Nadav Rotem
6c645a35cc fix a typo.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169953 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-12 01:31:10 +00:00
Nadav Rotem
5da804150d LoopVectorizer: When -Os is used, vectorize only loops that dont require a tail loop. There is no testcase because I dont know of a way to initialize the loop vectorizer pass without adding an additional hidden flag.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169950 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-12 01:11:46 +00:00
Nadav Rotem
655d2c5354 PR14574. Fix a bug in the code that calculates the mask the converted PHIs in if-conversion.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169916 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-11 21:30:14 +00:00
Nadav Rotem
5e9efa10fc Loop Vectorize: optimize the vectorization of trunc(induction_var). The truncation is now done on scalars.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169904 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-11 18:58:10 +00:00
Nadav Rotem
cfb6285fdb Fix PR14565. Don't if-convert loops that have switch statements in them.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169813 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-11 04:55:10 +00:00
Nadav Rotem
d1d92bf953 Split the LoopVectorizer into H and CPP.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169771 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-10 21:39:02 +00:00
Nadav Rotem
f0d19bd129 Add support for reverse induction variables. For example:
while (i--)
 sum+=A[i];



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169752 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-10 19:25:06 +00:00
Paul Redmond
880166684e LoopVectorize: support vectorizing intrinsic calls
- added function to VectorTargetTransformInfo to query cost of intrinsics
- vectorize trivially vectorizable intrinsic calls such as sin, cos, log, etc.

Reviewed by: Nadav


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169711 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-09 20:42:17 +00:00
Paul Redmond
b48fdbc811 test commit.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169709 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-09 19:46:31 +00:00
Nadav Rotem
46c5f79789 LoopVectorizer: Increase the number of pointers that can be tested at runtime. If we cant prove statically that the pointers are disjoint then we add the runtime check.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169334 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-04 23:25:24 +00:00
Nadav Rotem
dc5eff5b98 Enable if-conversion during vectorization.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169331 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-04 22:59:52 +00:00
Nadav Rotem
e570dee4b0 Fix a bug in vectorization of if-converted reduction variables. If the
reduction variable is not used outside the loop then we ran into an
endless loop. This change checks if we found the original PHI.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169324 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-04 22:40:22 +00:00
Nadav Rotem
f6088d126e Add support for reduction variables when IF-conversion is enabled.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169288 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-04 18:17:33 +00:00
Nadav Rotem
e6f2df9610 Give scalar if-converted blocks half the score because they are not always executed due to CF.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169223 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-04 07:11:52 +00:00
Nadav Rotem
319d594e22 Add the last part that is needed for vectorization of if-converted code.
Added the code that actually performs the if-conversion during vectorization.

We can now vectorize this code:

for (int i=0; i<n; ++i) {
  unsigned k = 0;

  if (a[i] > b[i])   <------ IF inside the loop.
    k = k * 5 + 3;

  a[i] = k;          <---- K is a phi node that becomes vector-select.
}



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169217 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-04 06:15:11 +00:00
NAKAMURA Takumi
915584ed13 LoopVectorize.cpp: Suppress a warning. [-Wunused-variable]
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169195 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-04 00:49:34 +00:00
NAKAMURA Takumi
36422d8876 Fix whitespace.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169194 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-04 00:49:28 +00:00
Nadav Rotem
b8f842dce4 minor renaming, documentation and cleanups.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169175 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-03 22:57:09 +00:00
Nadav Rotem
fa72ee729a IF-conversion: teach the cost-model how to grade if-converted loops.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169171 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-03 22:46:31 +00:00
Nadav Rotem
56f1ce22dc Now that we have a basic if-conversion infrastructure we can rename the
"single basic block loop vectorizer" to "innermost loop vectorizer".



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169158 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-03 21:33:08 +00:00
Nadav Rotem
dd8b1015c8 Add initial support for IF-conversion. This patch implements the first 1/3,
which is the legality of the if-conversion transformation. The next step is to
implement the cost-model for the if-converted code as well as the
vectorization itself.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169152 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-03 21:06:35 +00:00
Chandler Carruth
d04a8d4b33 Use the new script to sort the includes of every file under lib.
Sooooo many of these had incorrect or strange main module includes.
I have manually inspected all of these, and fixed the main module
include to be the nearest plausible thing I could find. If you own or
care about any of these source files, I encourage you to take some time
and check that these edits were sensible. I can't have broken anything
(I strictly added headers, and reordered them, never removed), but they
may not be the headers you'd really like to identify as containing the
API being implemented.

Many forward declarations and missing includes were added to a header
files to allow them to parse cleanly when included first. The main
module rule does in fact have its merits. =]

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169131 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-03 16:50:05 +00:00
Nadav Rotem
d6964741f5 minor cleanups
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169048 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-30 22:37:11 +00:00
Nadav Rotem
f735a7f88d Remove the use of LPPassManager. We can remove LPM because we dont need to run any additional loop passes on the new vector loop.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169016 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-30 17:27:53 +00:00
Nadav Rotem
dddaad624e When broadcasting invariant scalars into vectors, place the broadcast code in the preheader.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168927 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-29 19:25:41 +00:00
Nadav Rotem
22689b6034 Move the code that uses SCEVs prior to creating the new loops.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168601 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-26 19:51:46 +00:00
Nadav Rotem
6bfc3481bd Move the max vector width to a constant parameter. No functionality change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168570 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-25 16:48:08 +00:00
Nadav Rotem
00e8074ee2 Fix the document style.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168569 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-25 16:39:01 +00:00
Nadav Rotem
8c6b73666b Refactor the ptr runtime check generation code. No functionality change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168568 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-25 16:27:16 +00:00
Nadav Rotem
a8e2b2b68f Rename method. No functionality change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168560 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-25 09:13:57 +00:00
Nadav Rotem
d7e0b7cdc5 The induction-pointer work is inspired by a research paper. This commit adds a reference.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168559 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-25 09:09:26 +00:00
Nadav Rotem
0af63ac245 Add support for pointer induction variables even when there is no integer induction variable.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168558 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-25 08:41:35 +00:00
Nadav Rotem
9a6823516f LoopVectorizer: Add initial support for pointer induction variables (for example: *dst++ = *src++).
At the moment we still require to have an integer induction variable (for example: i++).



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168231 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-17 00:27:03 +00:00
Nadav Rotem
77d6ae1b01 LoopVectorize: Division reductions generate incorrect code. Remove the part of the code that deals with divs.
Thanks to Paul Redmond for catching this while reviewing the code.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168142 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-16 06:51:17 +00:00
Nadav Rotem
e123fd9c59 use the getSplat API. Patch by Paul Redmond.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167892 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-14 00:02:13 +00:00
Nadav Rotem
9005b8d27d Fix a comment typo and add comments.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167684 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-11 05:15:00 +00:00
Nadav Rotem
b1bf1eeede Add support for memory runtime check. When we can, we calculate array bounds.
If the arrays are found to be disjoint then we run the vectorized version of
the loop. If they are not, we run the scalar code.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167608 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-09 07:09:44 +00:00
Chandler Carruth
6e48f03077 Fix sign compare warning. Patch by Mahesha HS.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167282 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-02 05:24:00 +00:00
Nadav Rotem
e57b2cbce6 LoopVectorize: Preserve NSW, NUW and IsExact flags.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167174 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31 21:40:39 +00:00
Nadav Rotem
4c1b4b1fe7 Put the threshold magic number in a variable.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167134 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31 16:22:16 +00:00
Nadav Rotem
2a5401cc75 Remove enum values since they are not used anymore.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167131 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31 16:14:06 +00:00
Nadav Rotem
5b2c4dc5f8 LoopVectorize: Do not vectorize loops with tiny constant trip counts.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167101 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31 03:31:07 +00:00
Nadav Rotem
462d1ca428 Add support for loops that don't start with Zero.
This is important for loops in the LAPACK test-suite.
These loops start at 1 because they are auto-converted from fortran.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167084 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-31 00:45:26 +00:00
Nadav Rotem
a368b89f28 Add documentation.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167055 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30 22:06:26 +00:00
Nadav Rotem
e709f5b600 LoopVectorize: Add support for write-only loops when the write destination is a single pointer.
Speedup SciMark by 1%



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167035 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30 18:36:45 +00:00
Nadav Rotem
cc03331caa LoopVectorize: Fix a bug in the initialization of reduction variables. AND needs to start at all-one
while XOR, and OR need to start at zero.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167032 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30 18:12:36 +00:00
Nadav Rotem
803ea1c40a LoopVectorizer: change debug prints: Print the module identifier when deciding to vectorize. When deciding not to vectorize do not print the called function name because it can be null.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166989 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30 00:40:39 +00:00