Commit Graph

413 Commits

Author SHA1 Message Date
6c30749583 Add IRBuilder::CreateVectorSplat and use it to simplify code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171349 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-01 19:55:16 +00:00
831737d329 Remove the Function::getFnAttributes method in favor of using the AttributeSet
directly.

This is in preparation for removing the use of the 'Attribute' class as a
collection of attributes. That will shift to the AttributeSet class instead.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171253 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-30 10:32:01 +00:00
db2367512e LoopVectorizer: Fix a bug in the code that updates the loop exiting block.
LCSSA PHIs may have undef values. The vectorizer updates values that are used by outside users such as PHIs.
The bug happened because undefs are not loop values. This patch handles these PHIs.

PR14725



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171251 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-30 07:47:00 +00:00
5dd839430c If all of the write objects are identified then we can vectorize the loop even if the read objects are unidentified.
PR14719.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171124 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-26 23:30:53 +00:00
13eb1e7817 LoopVectorizer: Optimize the vectorization of consecutive memory access when the iteration step is -1
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171114 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-26 19:08:17 +00:00
1d59f5fa53 LoopVectorize: Enable vectorization of the fmuladd intrinsic
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171076 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-25 23:21:29 +00:00
9e5329d77e LoopVectorizer: When checking for vectorizable types, also check
the StoreInst operands.

PR14705.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171023 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-24 09:14:18 +00:00
470ea9b72f LoopVectorizer: Fix an endless loop in the code that looks for reductions.
The bug was in the code that detects PHIs in if-then-else block sequence.

PR14701.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171008 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-24 01:22:06 +00:00
a1acf55738 LoopVectorize: Fix accidentaly inverted condition.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171001 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-23 13:21:41 +00:00
417872ed08 LoopVectorize: For scalars and void types there is no need to compute vector insert/extract costs.
Fixes an assert during the build of oggenc in the test suite.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171000 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-23 13:19:18 +00:00
d54fed2786 Loop Vectorizer: Update the cost model of scatter/gather operations and make
them more expensive.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170995 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-23 07:23:55 +00:00
629fb82419 Change 'AttrVal' to 'AttrKind' to better reflect that it's a kind of attribute instead of the value of the attribute.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170972 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-22 00:37:52 +00:00
38b06020db Remove duplicate includes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170902 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-21 17:06:44 +00:00
ebf395d39e Enable if-conversion.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170841 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-21 04:47:54 +00:00
55306bdea5 Fix a bug in the code that checks if we can vectorize loops while using dynamic
memory bound checks.  Before the fix we were able to vectorize this loop from
the Livermore Loops benchmark:

for ( k=1 ; k<n ; k++ )
  x[k] = x[k-1] + y[k];



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170811 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-21 00:07:35 +00:00
8386acd734 LoopVectorize: Fix a bug in the scalarization of instructions.
Before if-conversion we could check if a value is loop invariant
if it was declared inside the basic block. Now that loops have
multiple blocks this check is incorrect.

This fixes External/SPEC/CINT95/099_go/099_go



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170756 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-20 20:24:40 +00:00
d5d46ace89 Loop Vectorizer: turn-off if-conversion.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170708 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-20 17:42:53 +00:00
d3071bb042 Loop Vectorizer: Enable if-conversion.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170632 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-20 02:00:02 +00:00
034b94b170 Rename the 'Attributes' class to 'Attribute'. It's going to represent a single attribute in the future.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170502 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-19 07:18:57 +00:00
0ef0e2e6d0 LoopVectorize: Emit reductions as log2(vectorsize) shuffles + vector ops instead of scalar operations.
For example on x86 with SSE4.2 a <8 x i8> add reduction becomes
	movdqa	%xmm0, %xmm1
	movhlps	%xmm1, %xmm1            ## xmm1 = xmm1[1,1]
	paddw	%xmm0, %xmm1
	pshufd	$1, %xmm1, %xmm0        ## xmm0 = xmm1[1,0,0,0]
	paddw	%xmm1, %xmm0
	phaddw	%xmm0, %xmm0
	pextrb	$0, %xmm0, %edx

instead of
	pextrb	$2, %xmm0, %esi
	pextrb	$0, %xmm0, %edx
	addb	%sil, %dl
	pextrb	$4, %xmm0, %esi
	addb	%dl, %sil
	pextrb	$6, %xmm0, %edx
	addb	%sil, %dl
	pextrb	$8, %xmm0, %esi
	addb	%dl, %sil
	pextrb	$10, %xmm0, %edi
	pextrb	$14, %xmm0, %edx
	addb	%sil, %dil
	pextrb	$12, %xmm0, %esi
	addb	%dil, %sil
	addb	%sil, %dl

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170439 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-18 18:40:20 +00:00
9ad73e93a5 Enable the Loop Vectorizer by default for O2 and O3. Disable if-conversion by default. I plan to revert this patch later today.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170157 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-13 23:11:54 +00:00
807dad62a0 Teach the cost model about the optimization in r169904: Truncation of induction variables costs the same as scalar trunc.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170051 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-13 00:21:03 +00:00
d0b144c04c Fix indentation.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170005 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-12 19:39:36 +00:00
ae3b652f5c LoopVectorizer: Use the "optsize" attribute to decide if we are allowed to increase the function size.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170004 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-12 19:29:45 +00:00
db65ff39fa Fix the ascii drawing that was ruined when I split the H and CPP
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169955 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-12 01:33:47 +00:00
6c645a35cc fix a typo.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169953 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-12 01:31:10 +00:00
5da804150d LoopVectorizer: When -Os is used, vectorize only loops that dont require a tail loop. There is no testcase because I dont know of a way to initialize the loop vectorizer pass without adding an additional hidden flag.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169950 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-12 01:11:46 +00:00
655d2c5354 PR14574. Fix a bug in the code that calculates the mask the converted PHIs in if-conversion.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169916 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-11 21:30:14 +00:00
5e9efa10fc Loop Vectorize: optimize the vectorization of trunc(induction_var). The truncation is now done on scalars.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169904 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-11 18:58:10 +00:00
cfb6285fdb Fix PR14565. Don't if-convert loops that have switch statements in them.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169813 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-11 04:55:10 +00:00
d1d92bf953 Split the LoopVectorizer into H and CPP.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169771 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-10 21:39:02 +00:00
f0d19bd129 Add support for reverse induction variables. For example:
while (i--)
 sum+=A[i];



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169752 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-10 19:25:06 +00:00
880166684e LoopVectorize: support vectorizing intrinsic calls
- added function to VectorTargetTransformInfo to query cost of intrinsics
- vectorize trivially vectorizable intrinsic calls such as sin, cos, log, etc.

Reviewed by: Nadav


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169711 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-09 20:42:17 +00:00
b48fdbc811 test commit.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169709 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-09 19:46:31 +00:00
46c5f79789 LoopVectorizer: Increase the number of pointers that can be tested at runtime. If we cant prove statically that the pointers are disjoint then we add the runtime check.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169334 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-04 23:25:24 +00:00
dc5eff5b98 Enable if-conversion during vectorization.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169331 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-04 22:59:52 +00:00
e570dee4b0 Fix a bug in vectorization of if-converted reduction variables. If the
reduction variable is not used outside the loop then we ran into an
endless loop. This change checks if we found the original PHI.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169324 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-04 22:40:22 +00:00
f6088d126e Add support for reduction variables when IF-conversion is enabled.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169288 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-04 18:17:33 +00:00
e6f2df9610 Give scalar if-converted blocks half the score because they are not always executed due to CF.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169223 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-04 07:11:52 +00:00
319d594e22 Add the last part that is needed for vectorization of if-converted code.
Added the code that actually performs the if-conversion during vectorization.

We can now vectorize this code:

for (int i=0; i<n; ++i) {
  unsigned k = 0;

  if (a[i] > b[i])   <------ IF inside the loop.
    k = k * 5 + 3;

  a[i] = k;          <---- K is a phi node that becomes vector-select.
}



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169217 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-04 06:15:11 +00:00
915584ed13 LoopVectorize.cpp: Suppress a warning. [-Wunused-variable]
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169195 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-04 00:49:34 +00:00
36422d8876 Fix whitespace.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169194 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-04 00:49:28 +00:00
b8f842dce4 minor renaming, documentation and cleanups.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169175 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-03 22:57:09 +00:00
fa72ee729a IF-conversion: teach the cost-model how to grade if-converted loops.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169171 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-03 22:46:31 +00:00
56f1ce22dc Now that we have a basic if-conversion infrastructure we can rename the
"single basic block loop vectorizer" to "innermost loop vectorizer".



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169158 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-03 21:33:08 +00:00
dd8b1015c8 Add initial support for IF-conversion. This patch implements the first 1/3,
which is the legality of the if-conversion transformation. The next step is to
implement the cost-model for the if-converted code as well as the
vectorization itself.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169152 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-03 21:06:35 +00:00
d04a8d4b33 Use the new script to sort the includes of every file under lib.
Sooooo many of these had incorrect or strange main module includes.
I have manually inspected all of these, and fixed the main module
include to be the nearest plausible thing I could find. If you own or
care about any of these source files, I encourage you to take some time
and check that these edits were sensible. I can't have broken anything
(I strictly added headers, and reordered them, never removed), but they
may not be the headers you'd really like to identify as containing the
API being implemented.

Many forward declarations and missing includes were added to a header
files to allow them to parse cleanly when included first. The main
module rule does in fact have its merits. =]

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169131 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-03 16:50:05 +00:00
d6964741f5 minor cleanups
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169048 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-30 22:37:11 +00:00
f735a7f88d Remove the use of LPPassManager. We can remove LPM because we dont need to run any additional loop passes on the new vector loop.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169016 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-30 17:27:53 +00:00
dddaad624e When broadcasting invariant scalars into vectors, place the broadcast code in the preheader.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168927 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-29 19:25:41 +00:00