Commit Graph

671 Commits

Author SHA1 Message Date
Craig Topper
f06c7072c2 Repace SmallPtrSet with SmallPtrSetImpl in function arguments to avoid needing to mention the size.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215868 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-17 23:47:00 +00:00
Rafael Espindola
d37ec47374 Introduce a helper to combine instruction metadata.
Replace the old code in GVN and BBVectorize with it. Update SimplifyCFG to use
it.

Patch by Björn Steinbrink!

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215723 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-15 15:46:38 +00:00
James Molloy
414df79b80 [LoopVectorizer] Enable support for floating-point subtraction reductions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215200 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-08 12:41:08 +00:00
Arnold Schwaighofer
2158dec965 SLPVectorizer: Use the type of the value loaded/stored to get the ABI alignment
We were using the pointer type which is incorrect.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215162 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-07 22:47:27 +00:00
James Molloy
72035e9a8e Teach the SLP Vectorizer that keeping some values live over a callsite can have a cost.
Some types, such as 128-bit vector types on AArch64, don't have any callee-saved registers. So if a value needs to stay live over a callsite, it must be spilled and refilled. This cost is now taken into account.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214859 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-05 12:30:34 +00:00
Erik Eckstein
8624519c0c fix bug 20513 - Crash in SLP Vectorizer
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214638 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-02 19:39:42 +00:00
Tyler Nowicki
842a06e8dd Add diagnostics to the vectorizer cost model.
When the cost model determines vectorization is not possible/profitable these remarks print an analysis of that decision.

Note that in selectVectorizationFactor() we can assume that OptForSize and ForceVectorization are mutually exclusive.

Reviewed by Arnold Schwaighofer


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214599 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-02 00:14:03 +00:00
Erik Eckstein
5e2214cca6 SLPVectorizer: fix build problem in Release configuration
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214496 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-01 09:47:38 +00:00
Erik Eckstein
956268f9dc SLPVectorizer: improved scheduling algorithm.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214494 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-01 09:20:42 +00:00
Erik Eckstein
12f7582cf0 SLP Vectorizer: added statistics counter
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214487 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-01 08:14:28 +00:00
Erik Eckstein
e47fce60e4 SLP Vectorizer: improve canonicalize tree operands of commutitive binary operands.
This reverts r214338 (except the test file) and replaces it with a more general algorithm.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214485 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-01 08:05:55 +00:00
Tyler Nowicki
f7be7f15c1 Improve the remark generated for -Rpass-missed.
The current remark is ambiguous and makes it sounds like explicitly specifying vectorization will allow the loop to be vectorized. This is not the case. The improved remark directs the user to -Rpass-analysis=loop-vectorize to determine the cause of the pass-miss.

Reviewed by Arnold Schwaighofer`


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214445 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-31 21:22:22 +00:00
Tyler Nowicki
88212074a8 Improve the remark generated when a variable that is used outside the loop is not a reduction or induction variable.
Reviewed by Arnold Schwaighofer


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214440 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-31 21:02:40 +00:00
Chad Rosier
7f6a685444 SLP Vectorizer: Canonicalize tree operands of commutitive binary operands.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214338 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-30 21:07:56 +00:00
Hal Finkel
16fd27b2c3 Add scoped-noalias metadata
This commit adds scoped noalias metadata. The primary motivations for this
feature are:
  1. To preserve noalias function attribute information when inlining
  2. To provide the ability to model block-scope C99 restrict pointers

Neither of these two abilities are added here, only the necessary
infrastructure. In fact, there should be no change to existing functionality,
only the addition of new features. The logic that converts noalias function
parameters into this metadata during inlining will come in a follow-up commit.

What is added here is the ability to generally specify noalias memory-access
sets. Regarding the metadata, alias-analysis scopes are defined similar to TBAA
nodes:

!scope0 = metadata !{ metadata !"scope of foo()" }
!scope1 = metadata !{ metadata !"scope 1", metadata !scope0 }
!scope2 = metadata !{ metadata !"scope 2", metadata !scope0 }
!scope3 = metadata !{ metadata !"scope 2.1", metadata !scope2 }
!scope4 = metadata !{ metadata !"scope 2.2", metadata !scope2 }

Loads and stores can be tagged with an alias-analysis scope, and also, with a
noalias tag for a specific scope:

... = load %ptr1, !alias.scope !{ !scope1 }
... = load %ptr2, !alias.scope !{ !scope1, !scope2 }, !noalias !{ !scope1 }

When evaluating an aliasing query, if one of the instructions is associated
with an alias.scope id that is identical to the noalias scope associated with
the other instruction, or is a descendant (in the scope hierarchy) of the
noalias scope associated with the other instruction, then the two memory
accesses are assumed not to alias.

Note that is the first element of the scope metadata is a string, then it can
be combined accross functions and translation units. The string can be replaced
by a self-reference to create globally unqiue scope identifiers.

[Note: This overview is slightly stylized, since the metadata nodes really need
to just be numbers (!0 instead of !scope0), and the scope lists are also global
unnamed metadata.]

Existing noalias metadata in a callee is "cloned" for use by the inlined code.
This is necessary because the aliasing scopes are unique to each call site
(because of possible control dependencies on the aliasing properties). For
example, consider a function: foo(noalias a, noalias b) { *a = *b; } that gets
inlined into bar() { ... if (...) foo(a1, b1); ... if (...) foo(a2, b2); } --
now just because we know that a1 does not alias with b1 at the first call site,
and a2 does not alias with b2 at the second call site, we cannot let inlining
these functons have the metadata imply that a1 does not alias with b2.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213864 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-24 14:25:39 +00:00
Hal Finkel
2c7c54c86c AA metadata refactoring (introduce AAMDNodes)
In order to enable the preservation of noalias function parameter information
after inlining, and the representation of block-level __restrict__ pointer
information (etc.), additional kinds of aliasing metadata will be introduced.
This metadata needs to be carried around in AliasAnalysis::Location objects
(and MMOs at the SDAG level), and so we need to generalize the current scheme
(which is hard-coded to just one TBAA MDNode*).

This commit introduces only the necessary refactoring to allow for the
introduction of other aliasing metadata types, but does not actually introduce
any (that will come in a follow-up commit). What it does introduce is a new
AAMDNodes structure to hold all of the aliasing metadata nodes associated with
a particular memory-accessing instruction, and uses that structure instead of
the raw MDNode* in AliasAnalysis::Location, etc.

No functionality change intended.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213859 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-24 12:16:19 +00:00
Mark Heffernan
bc7f1aba2d Rename metadata llvm.loop.vectorize.unroll to llvm.loop.vectorize.interleave.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213588 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-21 23:11:03 +00:00
Duncan P. N. Exon Smith
facdfc6781 Revert "[C++11] Add predecessors(BasicBlock *) / successors(BasicBlock *) iterator ranges."
This reverts commit r213474 (and r213475), which causes a miscompile on
a stage2 LTO build.  I'll reply on the list in a moment.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213562 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-21 17:06:51 +00:00
Hal Finkel
07d4e558b3 [LoopVectorize] Remove an unused private AA pointer
Thanks to the lld-x86_64-darwin13 builder for catching this first.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213488 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-20 23:28:25 +00:00
Hal Finkel
160f9b9c10 [LoopVectorize] Use AA to partition potential dependency checks
Prior to this change, the loop vectorizer did not make use of the alias
analysis infrastructure. Instead, it performed memory dependence analysis using
ScalarEvolution-based linear dependence checks within equivalence classes
derived from the results of ValueTracking's GetUnderlyingObjects.

Unfortunately, this meant that:
  1. The loop vectorizer had logic that essentially duplicated that in BasicAA
     for aliasing based on identified objects.
  2. The loop vectorizer could not partition the space of dependency checks
     based on information only easily available from within AA (TBAA metadata is
     currently the prime example).

This means, for example, regardless of whether -fno-strict-aliasing was
provided, the vectorizer would only vectorize this loop with a runtime
memory-overlap check:

void foo(int *a, float *b) {
  for (int i = 0; i < 1600; ++i)
    a[i] = b[i];
}

This is suboptimal because the TBAA metadata already provides the information
necessary to show that this check unnecessary. Of course, the vectorizer has a
limit on the number of such checks it will insert, so in practice, ignoring
TBAA means not vectorizing more-complicated loops that we should.

This change causes the vectorizer to use an AliasSetTracker to keep track of
the pointers in the loop. The resulting alias sets are then used to partition
the space of dependency checks, and potential runtime checks; this results in
more-efficient vectorizations.

When pointer locations are added to the AliasSetTracker, two things are done:
  1. The location size is set to UnknownSize (otherwise you'd not catch
     inter-iteration dependencies)
  2. For instructions in blocks that would need to be predicated, TBAA is
     removed (because the metadata might have a control dependency on the condition
     being speculated).

For non-predicated blocks, you can leave the TBAA metadata. This is safe
because you can't have an iteration dependency on the TBAA metadata (if you
did, and you unrolled sufficiently, you'd end up with the same pointer value
used by two accesses that TBAA says should not alias, and that would yield
undefined behavior).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213486 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-20 23:07:52 +00:00
Manuel Jacob
a4697dad19 [C++11] Add predecessors(BasicBlock *) / successors(BasicBlock *) iterator ranges.
Summary: This patch introduces two new iterator ranges and updates existing code to use it.  No functional change intended.

Test Plan: All tests (make check-all) still pass.

Reviewers: dblaikie

Reviewed By: dblaikie

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D4481

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213474 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-20 09:10:11 +00:00
Hal Finkel
4df656952a [LoopVectorize] Use CreateAligned(Load|Store)
IRBuilder has CreateAligned(Load|Store) functions; use them and we don't need
to make a second call to setAlignment.

No functionality change intended.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213453 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-19 13:39:45 +00:00
Hal Finkel
2350e9f6b7 [LoopVectorize] Propagate known metadata to vectorized instructions
There are some kinds of metadata that are safe to propagate from the scalar
instructions to the vector instructions (fpmath and tbaa currently).

Regarding TBAA, one might worry about propagating it on if-converted loads and
stores, because the metadata might have had a control dependency on the
condition, and thus actually aliased with some other non-speculated memory
access when the condition was false. However, this would be caught by the
runtime overlap checks.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213452 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-19 13:33:16 +00:00
Tyler Nowicki
77f03040d5 Emit warnings if vectorization is forced and fails.
This patch modifies the existing DiagnosticInfo system to create a generic base
class that is inherited to produce diagnostic-based warnings. This is used by
the loop vectorizer to trigger a warning when vectorization is forced and
fails. Several tests have been added to verify this behavior.

Reviewed by: Arnold Schwaighofer


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@213110 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-16 00:36:00 +00:00
Alp Toker
8dd8d5c2b2 Revert "Introduce a string_ostream string builder facilty"
Temporarily back out commits r211749, r211752 and r211754.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211814 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-26 22:52:05 +00:00
Alp Toker
2559070422 Introduce a string_ostream string builder facilty
string_ostream is a safe and efficient string builder that combines opaque
stack storage with a built-in ostream interface.

small_string_ostream<bytes> additionally permits an explicit stack storage size
other than the default 128 bytes to be provided. Beyond that, storage is
transferred to the heap.

This convenient class can be used in most places an
std::string+raw_string_ostream pair or SmallString<>+raw_svector_ostream pair
would previously have been used, in order to guarantee consistent access
without byte truncation.

The patch also converts much of LLVM to use the new facility. These changes
include several probable bug fixes for truncated output, a programming error
that's no longer possible with the new interface.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211749 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-26 00:00:48 +00:00
Tyler Nowicki
d5a8fa72bb Add Rpass-missed and Rpass-analysis reports to the loop vectorizer. The remarks give the vector width of vectorized loops and a brief analysis of loops that fail to be vectorized. For example, an analysis will be generated for loops containing control flow that cannot be simplified to a select. The optimization remarks also give the debug location of expressions that cannot be vectorized, for example the location of an unvectorizable call.
Reviewed by: Arnold Schwaighofer


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211721 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-25 17:50:15 +00:00
Eli Bendersky
bb167336b3 Rename loop unrolling and loop vectorizer metadata to have a common prefix.
[LLVM part]

These patches rename the loop unrolling and loop vectorizer metadata
such that they have a common 'llvm.loop.' prefix.  Metadata name
changes:

llvm.vectorizer.* => llvm.loop.vectorizer.*
llvm.loopunroll.* => llvm.loop.unroll.*

This was a suggestion from an earlier review
(http://reviews.llvm.org/D4090) which added the loop unrolling
metadata. 

Patch by Mark Heffernan.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211710 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-25 15:41:00 +00:00
Arnold Schwaighofer
78f173a22d LoopVectorizer: Fix a dominance issue
The induction variables start value needs to be defined before we branch
(overflow check) to the scalar preheader where we used it.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211460 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-22 03:38:59 +00:00
Karthik Bhat
d2ce9392dc Add Support to Recognize and Vectorize NON SIMD instructions in SLPVectorizer.
This patch adds support to recognize patterns such as fadd,fsub,fadd,fsub.../add,sub,add,sub... and
vectorizes them as vector shuffles if they are profitable.
These patterns of vector shuffle can later be converted to instructions such as addsubpd etc on X86.
Thanks to Arnold and Hal for the reviews. http://reviews.llvm.org/D4015 


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211339 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-20 04:32:48 +00:00
Michael Zolotukhin
afbe82d590 [SLP] Enable vectorization of GEP expressions.
The use cases look like the following:
    x->a = y->a + 10
    x->b = y->b + 12

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210342 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-06 15:34:24 +00:00
Karthik Bhat
b73a9b384e Fix PR19657 (scalar loads not combined into vector load)
If we have common uses on separate paths in the tree; process the one with greater common depth first.
This makes sure that we do not assume we need to extract a load when it is actually going to be part of a vectorized tree.

Review: http://reviews.llvm.org/D3800


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210310 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-06 06:20:08 +00:00
Karthik Bhat
8ffc96a163 Allow vectorization of intrinsics such as powi,cttz and ctlz in Loop and SLP Vectorizer.
This patch adds support to vectorize intrinsics such as powi, cttz and ctlz in Vectorizer. These intrinsics are different from other
intrinsics as second argument to these function must be same in order to vectorize them and it should be represented as a scalar.
Review: http://reviews.llvm.org/D3851#inline-32769 and http://reviews.llvm.org/D3937#inline-32857


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209873 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-30 04:31:24 +00:00
Arnold Schwaighofer
06413cd0f0 LoopVectorizer: Add a check that the backedge taken count + 1 does not overflow
The loop vectorizer instantiates be-taken-count + 1 as the loop iteration count.
If this expression overflows the generated code was invalid.

In case of overflow the code now jumps to the scalar loop.

Fixes PR17288.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209854 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-29 22:10:01 +00:00
Diego Novillo
d16404a0e7 Add support for missed and analysis optimization remarks.
Summary:
This adds two new diagnostics: -pass-remarks-missed and
-pass-remarks-analysis. They take the same values as -pass-remarks but
are intended to be triggered in different contexts.

-pass-remarks-missed is used by LLVMContext::emitOptimizationRemarkMissed,
which passes call when they tried to apply a transformation but
couldn't.

-pass-remarks-analysis is used by LLVMContext::emitOptimizationRemarkAnalysis,
which passes call when they want to inform the user about analysis
results.

The patch also:

1- Adds support in the inliner for the two new remarks and a
   test case.

2- Moves emitOptimizationRemark* functions to the llvm namespace.

3- Adds an LLVMContext argument instead of making them member functions
   of LLVMContext.

Reviewers: qcolombet

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D3682

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209442 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-22 14:19:46 +00:00
Eric Christopher
68c7a1cb98 Clean up language and grammar.
Based on a patch by jfcaron3@gmail.com!
PR19806

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209216 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-20 17:11:11 +00:00
Zinovy Nis
6f242c93ec [LV][REFACTOR] One more tiny fix for printing debug locations in loop vectorizer. Now consistent with the remarks emitter.
Differential Revision: http://reviews.llvm.org/D3821



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@209197 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-20 08:26:20 +00:00
Benjamin Kramer
4db3f41012 SLPVectorizer: Instead of just performing CSE on dead blocks ignore them completely.
Turns out that there is a very cheap way of testing whether a block is dead,
just look it up in the DomTree. We have to do this anyways so just ignore
unreachable blocks before sorting by domination. This restores a proper
ordering for std::stable_sort when dead code is present.

Covered by existing tests & buildbots running in STL debug mode (MSVC).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@208492 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-11 10:28:58 +00:00
Benjamin Kramer
3ae331d81a SLPVectorizer: When sorting by domination for CSE don't assert on unreachable code.
There is no total ordering if the CFG is disconnected. We don't care if we
catch all CSE opportunities in dead code either so just exclude ignore them in
the assert.

PR19646

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@208461 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-09 23:28:49 +00:00
Zinovy Nis
6a48f1c271 [BUG][REFACTOR]
1) Fix for printing debug locations for absolute paths.
2) Location printing is moved into public method DebugLoc::print() to avoid re-inventing the wheel.

Differential Revision: http://reviews.llvm.org/D3513




git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@208177 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-07 09:51:22 +00:00
Yi Jiang
606660f1f3 Always set alignment of vectorized LD/ST in SLP-Vectorizer. <rdar://problem/16812145>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207983 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-05 17:59:14 +00:00
Arnold Schwaighofer
28a739b4dc SLPVectorizer: Bring back the insertelement patch (r205965) with fixes
When can't assume a vectorized tree is rooted in an instruction. The IRBuilder
could have constant folded it. When we rebuild the build_vector (the series of
InsertElement instructions) use the last original InsertElement instruction. The
vectorized tree root is guaranteed to be before it.

Also, we can't assume that the n-th InsertElement inserts the n-th element into
a vector.

This reverts r207746 which reverted the revert of the revert of r205018 or so.

Fixes the test case in PR19621.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207939 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-04 17:10:15 +00:00
Benjamin Kramer
81f28f603a SLPVectorizer: Lazily allocate the map for block numbering.
There is no point in creating it if we're not going to vectorize
anything. Creating the map is expensive as it creates large values.
No functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207916 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-03 15:50:37 +00:00
Karthik Bhat
486ad6262e Vectorize intrinsic math function calls in SLPVectorizer.
This patch adds support to recognize and vectorize intrinsic math functions in SLPVectorizer.
Review: http://reviews.llvm.org/D3560 and http://reviews.llvm.org/D3559


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207901 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-03 09:59:54 +00:00
Eric Christopher
f41cc33836 Clean up constructor logic and member access for LoopVectorizeHints.
There are public functions that mutate various members as well as
another private member already, so make all the members private to
avoid the discontinuity and add accessors for the values. Should
be no functional change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207868 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-02 20:40:04 +00:00
Chandler Carruth
06ace3c869 Revert r205965, which essentially reverts r205018 for the second time.
=[

Turns out that this was the root cause of PR19621. We found a crasher
only recently (likely due to improvements elsewhere in the SLP
vectorizer) but the reduced test case failed all the way back to here.
I've confirmed that reverting this patch both fixes the reduced test
case in PR19621 and the actual source file that led to it, so it seems
to really be rooted here. I've replied to the commit thread with
discussion of my (feeble) attempts to debug this. Didn't make it very
far, so reverting now that we have a good test case so that things can
get back to healthy while the debugging carries on.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207746 91177308-0d34-0410-b5e6-96231b3b80d8
2014-05-01 11:24:11 +00:00
Benjamin Kramer
28ea58283b Add a <tuple> include to more files that aren't getting it transitively on MSVC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207617 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-30 07:21:01 +00:00
Diego Novillo
55deff895d Fix vectorization remarks.
This patch changes the vectorization remarks to also inform when
vectorization is possible but not beneficial.

Added tests to exercise some loop remarks.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207574 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-29 20:06:10 +00:00
Yi Jiang
bbea6143f2 Continue slp vectorization even the BB already has vectorized store radar://16641956
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207572 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-29 19:37:20 +00:00
Diego Novillo
44180390f8 Add optimization remarks to the loop unroller and vectorizer.
Summary:
This calls emitOptimizationRemark from the loop unroller and vectorizer
at the point where they make a positive transformation. For the
vectorizer, it reports vectorization and interleave factors. For the
loop unroller, it reports all the different supported types of
unrolling.

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D3456

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207528 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-29 14:27:31 +00:00
Zinovy Nis
0f79399dee [BUG] Fix -Wunused-variable warning in Release mode. Thnx to Kostya Serebryany for pointing.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207516 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-29 09:45:08 +00:00
Kostya Serebryany
0f99bc0606 fix -Wunused-variable warning in Release mode
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207514 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-29 09:33:02 +00:00
Zinovy Nis
c5e41aed09 [OPENMP][LV][D3423] Respect Hints.Force meta-data for loops in LoopVectorizer
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207512 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-29 08:55:11 +00:00
Craig Topper
c34a25d59d [C++] Use 'nullptr'.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207394 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-28 04:05:08 +00:00
Craig Topper
8d7221ccf5 [C++] Use 'nullptr'. Transforms edition.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207196 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-25 05:29:35 +00:00
Karthik Bhat
ac16f0e024 Allow vectorization of bit intrinsics in BB Vectorizer.
This patch adds support for vectorization of  bit intrinsics such as bswap,ctpop,ctlz,cttz.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207174 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-25 03:33:48 +00:00
Karthik Bhat
0698b2b6cc Allow vectorization of few missed llvm intrinsic calls in BBVectorizor by handling them in isVectorizableIntrinsic function.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207085 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-24 07:29:55 +00:00
Alexander Musman
bf255f5d5a [LV] Statistics numbers for LoopVectorize introduced: a number of analyzed loops & a number of vectorized loops.
Use -stats to see how many loops were analyzed for possible vectorization and how many of them were actually vectorized.
Patch by Zinovy Nis

Differential Revision: http://reviews.llvm.org/D3438



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206956 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-23 08:40:37 +00:00
Chandler Carruth
7962dbdc65 [Modules] Fix potential ODR violations by sinking the DEBUG_TYPE
definition below all of the header #include lines, lib/Transforms/...
edition.

This one is tricky for two reasons. We again have a couple of passes
that define something else before the includes as well. I've sunk their
name macros with the DEBUG_TYPE.

Also, InstCombine contains headers that need DEBUG_TYPE, so now those
headers #define and #undef DEBUG_TYPE around their code, leaving them
well formed modular headers. Fixing these headers was a large motivation
for all of these changes, as "leaky" macros of this form are hard on the
modules implementation.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206844 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-22 02:55:47 +00:00
Alexey Bataev
15cbb64eb4 D3348 - [BUG] "Rotate Loop" pass kills "llvm.vectorizer.enable" metadata
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206266 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-15 09:37:30 +00:00
Arnold Schwaighofer
e2d124d396 Reapply "SLPVectorizer: Ignore users that are insertelements we can reschedule them"
This commit reapplies 205018. After 205855 we should correctly vectorize
intrinsics.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205965 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-10 13:41:35 +00:00
Arnold Schwaighofer
b0ee2374ce SLPVectorizer: Only vectorize intrinsics whose operands are widened equally
The vectorizer only knows how to vectorize intrinics by widening all operands by
the same factor.

Patch by Tyler Nowicki!

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205855 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-09 14:20:47 +00:00
Eric Christopher
231ed9dd9c Add NDEBUG markers around debug only function.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205706 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-07 12:46:30 +00:00
Eric Christopher
c75aecc473 Add debug location information to the vectorizer debug statements.
Patch by Zinovy Nis.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205705 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-07 12:32:17 +00:00
David Blaikie
2c5f72b629 Fixing typo.
Differential Revision: http://reviews.llvm.org/D3154

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205674 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-05 20:30:31 +00:00
Tim Northover
24e78e0125 SLPVectorizer: compare entire intrinsic for SLP compatibility.
Some Intrinsics are overloaded to the extent that return type equality (all
that's been checked up to now) does not guarantee that the arguments are the
same. In these cases SLP vectorizer should not recurse into the operands, which
can be achieved by comparing them as "Function *" rather than simply the ID.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205424 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-02 14:39:02 +00:00
Hal Finkel
081e6fcd17 [LoopVectorizer] Count dependencies of consecutive pointers as uniforms
For the purpose of calculating the cost of the loop at various vectorization
factors, we need to count dependencies of consecutive pointers as uniforms
(which means that the VF = 1 cost is used for all overall VF values).

For example, the TSVC benchmark function s173 has:
  ...
  %3 = add nsw i64 %indvars.iv, 16000
  %arrayidx8 = getelementptr inbounds %struct.GlobalData* @global_data, i64 0, i32 0, i64 %3
  ...
and we must realize that the add will be a scalar in order to correctly deduce
it to be profitable to vectorize this on PowerPC with VSX enabled. In fact, all
dependencies of a consecutive pointer must be a scalar (uniform), and so we
simply need to add all consecutive pointers to the worklist that currently
detects collects uniforms.

Fixes PR19296.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205387 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-02 02:34:49 +00:00
Arnold Schwaighofer
694365f955 Revert "SLPVectorizer: Ignore users that are insertelements we can reschedule them"
This reverts commit r205018.

Conflicts:
	lib/Transforms/Vectorize/SLPVectorizer.cpp
	test/Transforms/SLPVectorizer/X86/insert-element-build-vector.ll

This is breaking libclc build.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205260 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-31 23:05:56 +00:00
Arnold Schwaighofer
ce36237826 SLPVectorizer: Take credit for free extractelement instructions
Extract element instructions that will be removed when vectorzing lower the
cost.

Patch by Arch D. Robison!

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205020 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-28 17:21:32 +00:00
Arnold Schwaighofer
aa0a2a35f8 SLPVectorizer: Fix typos
Patch by Arch D. Robison!

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205019 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-28 17:21:27 +00:00
Arnold Schwaighofer
b48eb2cdaa SLPVectorizer: Ignore users that are insertelements we can reschedule them
Patch by Arch D. Robison!

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205018 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-28 17:21:22 +00:00
Andrew Trick
f5bf687cf7 SLP vectorizer: Don't hoist vector extracts of phis.
Extracts coming from phis were being hoisted, while all others were
sunk to their uses. This was inconsistent and didn't seem to serve a
purpose. Changing all extracts to be sunk to uses is a prerequisite
for adding block frequency to the SLP vectorizer's cost model.

I benchmarked the change in isolation (without block frequency). I
only saw noise on x86 and some potentially significant improvements on
ARM. No major regressions is good enough for me.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204699 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-25 02:18:47 +00:00
Chandler Carruth
3c1eb9903b [LV] While I'm here, use range based for loops which are so much cleaner
for this kind of walk.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204188 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-18 22:00:32 +00:00
Chandler Carruth
633eb08d31 [LV] The actual change I intended to commit in r204148. Sorry for the
noise.

Original commit log:
Replace some dead code with an assert. When I first ported this pass
from a loop pass to a function pass I did so in the naive, recursive
way. It doesn't actually work, we need a worklist instead. When
I switched to the worklist I didn't delete the naive recursion. That
recursion was also buggy because it was dead and never really exercised.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204187 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-18 21:58:38 +00:00
Chandler Carruth
a22773cd45 [LV] Replace some dead code with an assert. When I first ported this
pass from a loop pass to a function pass I did so in the naive,
recursive way. It doesn't actually work, we need a worklist instead.
When I switched to the worklist I didn't delete the naive recursion.
That recursion was also buggy because it was dead and never really
exercised.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204184 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-18 21:51:46 +00:00
Raul E. Silvera
230eda4bdf Resubmit "[SLPV] Recognize vectorizable intrinsics during SLP vectorization ..."
This reverts commit 86cb795388.
The problems previously found have been resolved through other CLs.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@203707 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-12 20:21:50 +00:00
Ahmed Charles
9c3328fc7f Fix build break.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@203366 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-09 03:50:36 +00:00
Chandler Carruth
36b699f2b1 [C++11] Add range based accessors for the Use-Def chain of a Value.
This requires a number of steps.
1) Move value_use_iterator into the Value class as an implementation
   detail
2) Change it to actually be a *Use* iterator rather than a *User*
   iterator.
3) Add an adaptor which is a User iterator that always looks through the
   Use to the User.
4) Wrap these in Value::use_iterator and Value::user_iterator typedefs.
5) Add the range adaptors as Value::uses() and Value::users().
6) Update *all* of the callers to correctly distinguish between whether
   they wanted a use_iterator (and to explicitly dig out the User when
   needed), or a user_iterator which makes the Use itself totally
   opaque.

Because #6 requires churning essentially everything that walked the
Use-Def chains, I went ahead and added all of the range adaptors and
switched them to range-based loops where appropriate. Also because the
renaming requires at least churning every line of code, it didn't make
any sense to split these up into multiple commits -- all of which would
touch all of the same lies of code.

The result is still not quite optimal. The Value::use_iterator is a nice
regular iterator, but Value::user_iterator is an iterator over User*s
rather than over the User objects themselves. As a consequence, it fits
a bit awkwardly into the range-based world and it has the weird
extra-dereferencing 'operator->' that so many of our iterators have.
I think this could be fixed by providing something which transforms
a range of T&s into a range of T*s, but that *can* be separated into
another patch, and it isn't yet 100% clear whether this is the right
move.

However, this change gets us most of the benefit and cleans up
a substantial amount of code around Use and User. =]

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@203364 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-09 03:16:01 +00:00
Arnold Schwaighofer
9d84b4d70c LoopVectorizer: Preserve fast-math flags
Fixes PR19045.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@203008 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-05 21:10:47 +00:00
Craig Topper
7b62be28cb [C++11] Add 'override' keyword to virtual methods that override their base class.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202953 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-05 09:10:37 +00:00
Chandler Carruth
eb3d76da81 [Modules] Move ValueHandle into the IR library where Value itself lives.
Move the test for this class into the IR unittests as well.

This uncovers that ValueMap too is in the IR library. Ironically, the
unittest for ValueMap is useless in the Support library (honestly, so
was the ValueHandle test) and so it already lives in the IR unittests.
Mmmm, tasty layering.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202821 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-04 11:17:44 +00:00
Chandler Carruth
df3d8e8b4d [Modules] Move the LLVM IR pattern match header into the IR library, it
obviously is coupled to the IR.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202818 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-04 11:08:18 +00:00
Benjamin Kramer
a4f0aad951 [C++11] Replace llvm::tie with std::tie.
The old implementation is no longer needed in C++11.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202644 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-02 13:30:33 +00:00
Benjamin Kramer
d628f19f5d [C++11] Replace llvm::next and llvm::prior with std::next and std::prev.
Remove the old functions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202636 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-02 12:27:27 +00:00
Benjamin Kramer
ee5e607355 Now that we have C++11, turn simple functors into lambdas and remove a ton of boilerplate.
No intended functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202588 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-01 11:47:00 +00:00
Rafael Espindola
57edc9d4ff Make DataLayout a plain object, not a pass.
Instead, have a DataLayoutPass that holds one. This will allow parts of LLVM
don't don't handle passes to also use DataLayout.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202168 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-25 17:30:31 +00:00
Rafael Espindola
37ecf69cbf Make a few more DataLayout variables const.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202155 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-25 14:24:11 +00:00
Rafael Espindola
ec89b9fb9e Make some DataLayout pointers const.
No functionality change. Just reduces the noise of an upcoming patch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202087 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-24 23:12:18 +00:00
Arnold Schwaighofer
68085c7bef SLPVectorizer: Try vectorizing 'splat' stores
Vectorize sequential stores of a broadcasted value.
5% on eon.

radar://16124699

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202067 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-24 19:52:29 +00:00
Rafael Espindola
f116e5308d Rename many DataLayout variables from TD to DL.
I am really sorry for the noise, but the current state where some parts of the
code use TD (from the old name: TargetData) and other parts use DL makes it
hard to write a patch that changes where those variables come from and how
they are passed along.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201827 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-21 00:06:31 +00:00
Gerolf Hoflehner
3bc859466b fix for null VectorizedValue assertion in the SLP Vectorizer (in function vectorizeTree()). radar://16064178
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201501 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-17 03:06:16 +00:00
Gerolf Hoflehner
8e810aeec3 fixed typo in comment as my test commit
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201486 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-16 10:43:25 +00:00
Benjamin Kramer
8ed2c8e99f Reduce code duplication resulting from the ConstantVector/ConstantDataVector split.
No intended functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201344 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-13 16:48:38 +00:00
Andrea Di Biagio
029a76b0a2 [Vectorizer] Add a new 'OperandValueKind' in TargetTransformInfo called
'OK_NonUniformConstValue' to identify operands which are constants but
not constant splats.

The cost model now allows returning 'OK_NonUniformConstValue'
for non splat operands that are instances of ConstantVector or
ConstantDataVector.

With this change, targets are now able to compute different costs
for instructions with non-uniform constant operands.
For example, On X86 the cost of a vector shift may vary depending on whether
the second operand is a uniform or non-uniform constant.

This patch applies the following changes:
 - The cost model computation now takes into account non-uniform constants;
 - The cost of vector shift instructions has been improved in
   X86TargetTransformInfo analysis pass;
 - BBVectorize, SLPVectorizer and LoopVectorize now know how to distinguish
   between non-uniform and uniform constant operands.

Added a new test to verify that the output of opt
'-cost-model -analyze' is valid in the following configurations: SSE2,
SSE4.1, AVX, AVX2.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201272 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-12 23:43:47 +00:00
Arnold Schwaighofer
846acbeef1 LoopVectorizer: Keep track of conditional store basic blocks
Before conditional store vectorization/unrolling we had only one
vectorized/unrolled basic block. After adding support for conditional store
vectorization this will not only be one block but multiple basic blocks. The
last block would have the back-edge. I updated the code to use a vector of basic
blocks instead of a single basic block and fixed the users to use the last entry
in this vector. But, I forgot to add the basic blocks to this vector!

Fixes PR18724.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201028 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-08 20:41:13 +00:00
Paul Robinson
2684ddd72e Disable most IR-level transform passes on functions marked 'optnone'.
Ideally only those transform passes that run at -O0 remain enabled,
in reality we get as close as we reasonably can.
Passes are responsible for disabling themselves, it's not the job of
the pass manager to do it for them.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200892 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-06 00:07:05 +00:00
Arnold Schwaighofer
a16c1b55e2 LoopVectorizer: Enable unrolling of conditional stores and the load/store
unrolling heuristic per default

Benchmarking on x86_64 (thanks Chandler!) and ARM has shown those options speed
up some benchmarks while not causing any interesting regressions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200621 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-02 03:12:34 +00:00
Reid Kleckner
86cb795388 Revert "[SLPV] Recognize vectorizable intrinsics during SLP vectorization ..."
This reverts commit r200576.  It broke 32-bit self-host builds by
vectorizing two calls to @llvm.bswap.i64, which we then fail to expand.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200602 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-01 01:37:30 +00:00
Chandler Carruth
093b0413fe [SLPV] Recognize vectorizable intrinsics during SLP vectorization and
transform accordingly. Based on similar code from Loop vectorization.
Subsequent commits will include vectorization of function calls to
vector intrinsics and form function calls to vector library calls.

Patch by Raul Silvera! (Much delayed due to my not running dcommit)

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200576 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-31 21:14:40 +00:00
Chandler Carruth
93228f6199 [vectorizer] Tweak the way we do small loop runtime unrolling in the
loop vectorizer to not do so when runtime pointer checks are needed and
share code with the new (not yet enabled) load/store saturation runtime
unrolling. Also ensure that we only consider the runtime checks when the
loop hasn't already been vectorized. If it has, the runtime check cost
has already been paid.

I've fleshed out a test case to cover the scalar unrolling as well as
the vector unrolling and comment clearly why we are or aren't following
the pattern.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200530 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-31 10:51:08 +00:00