Commit Graph

4979 Commits

Author SHA1 Message Date
Michael Gottesman
2253a2f52f Added support for the Builtin attribute.
The Builtin attribute is an attribute that can be placed on function call site that signal that even though a function is declared as being a builtin,

rdar://problem/13727199

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185049 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-27 00:25:01 +00:00
Nadav Rotem
e26fa2dd73 Erase all of the instructions that we RAUWed
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184969 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-26 17:16:09 +00:00
Nadav Rotem
29acf7e03a Do not add cse-ed instructions into the visited map because we dont want to consider them as a candidate for replacement of instructions to be visited.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184966 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-26 16:54:53 +00:00
Nadav Rotem
805e8a01fe SLPVectorizer: support slp-vectorization of PHINodes between basic blocks
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184888 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-25 23:04:09 +00:00
Bob Wilson
a1fe2948ed Fix SROA to avoid unnecessary scalar conversions for 1-element vectors.
When a 1-element vector alloca is promoted, a store instruction can often be
rewritten without converting the value to a scalar and using an insertelement
instruction to stuff it into the new alloca.  This patch just adds a check
to skip that conversion when it is unnecessary.  This turns out to be really
important for some ARM Neon operations where <1 x i64> is used to get around
the fact that i64 is not a legal type.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184870 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-25 19:09:50 +00:00
Arnold Schwaighofer
bc7c58d2b1 Reapply 184685 after the SetVector iteration order fix.
This should hopefully have fixed the stage2/stage3 miscompare on the dragonegg
testers.

"LoopVectorize: Use the dependence test utility class

We now no longer need alias analysis - the cases that alias analysis would
handle are now handled as accesses with a large dependence distance.

We can now vectorize loops with simple constant dependence distances.

  for (i = 8; i < 256; ++i) {
    a[i] = a[i+4] * a[i+8];
  }

  for (i = 8; i < 256; ++i) {
    a[i] = a[i-4] * a[i-8];
  }

We would be able to vectorize about 200 more loops (in many cases the cost model
instructs us no to) in the test suite now. Results on x86-64 are a wash.

I have seen one degradation in ammp. Interestingly, the function in which we
now vectorize a loop is never executed so we probably see some instruction
cache effects. There is a 2% improvement in h264ref. There is one or the other
TSCV loop kernel that speeds up.

radar://13681598"

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184724 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-24 12:09:15 +00:00
Arnold Schwaighofer
ec677e2a64 Revert "LoopVectorize: Use the dependence test utility class"
This reverts commit cbfa1ca993.

We are seeing a stage2 and stage3 miscompare on some dragonegg bots.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184690 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-24 06:10:41 +00:00
Arnold Schwaighofer
cbfa1ca993 LoopVectorize: Use the dependence test utility class
We now no longer need alias analysis - the cases that alias analysis would
handle are now handled as accesses with a large dependence distance.

We can now vectorize loops with simple constant dependence distances.

  for (i = 8; i < 256; ++i) {
    a[i] = a[i+4] * a[i+8];
  }

  for (i = 8; i < 256; ++i) {
    a[i] = a[i-4] * a[i-8];
  }

We would be able to vectorize about 200 more loops (in many cases the cost model
instructs us no to) in the test suite now. Results on x86-64 are a wash.

I have seen one degradation in ammp. Interestingly, the function in which we
now vectorize a loop is never executed so we probably see some instruction
cache effects. There is a 2% improvement in h264ref. There is one or the other
TSCV loop kernel that speeds up.

radar://13681598

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184685 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-24 03:55:48 +00:00
Nadav Rotem
25961b469a SLP Vectorizer: Add support for vectorizing parts of the tree.
Untill now we detected the vectorizable tree and evaluated the cost of the
entire tree.  With this patch we can decide to trim-out branches of the tree
that are not profitable to vectorizer.

Also, increase the max depth from 6 to 12. In the worse possible case where all
of the code is made of diamond-shaped graph this can bring the cost to 2**10,
but diamonds are not very common.




git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184681 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-24 02:52:43 +00:00
Nadav Rotem
722b0a4d29 SLP Vectorizer: Fix a bug in the code that does CSE on the generated gather sequences.
Make sure that we don't replace and RAUW two sequences if one does not dominate the other.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184674 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-23 21:57:27 +00:00
Nadav Rotem
6959f08f44 SLP Vectorizer: Implement a simple CSE optimization for the gather sequences.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184660 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-23 06:15:46 +00:00
Nadav Rotem
53a0552b06 SLP Vectorizer: Implement multi-block slp-vectorization.
Rewrote the SLP-vectorization as a whole-function vectorization pass. It is now able to vectorize chains across multiple basic blocks.
It still does not vectorize PHIs, but this should be easy to do now that we scan the entire function.
I removed the support for extracting values from trees.
We are now able to vectorize more programs, but there are some serious regressions in many workloads (such as flops-6 and mandel-2).



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184647 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-22 21:34:10 +00:00
Nadav Rotem
d69d9f20bc SLPVectorization: Add a basic support for cross-basic block slp vectorization.
We collect gather sequences when we vectorize basic blocks. Gather sequences are excellent
hints for vectorization of other basic blocks.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184444 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-20 17:41:45 +00:00
Matt Arsenault
ad966ea7a8 Move StructurizeCFG out of R600 to generic Transforms.
Register it with PassManager

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184343 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-19 20:18:24 +00:00
Quentin Colombet
5a2fb058d3 LSR: Fix the parameters used to compute the scaling factor cost.
Prior to this change, the considered addressing modes may be invalid since the
maximum and minimum offsets were not taking into account.
This was causing an assertion failure.

The added test case exercices that behavior.

<rdar://problem/14199725> Assertion failed: (CurScaleCost >= 0 && "Legal
addressing mode has an illegal cost!")


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184341 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-19 19:59:41 +00:00
Nadav Rotem
80a6d6526c SLPVectorizer: handle scalars that are extracted from vectors (using ExtractElementInst).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184325 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-19 17:33:16 +00:00
Nadav Rotem
7d180ac7b6 SLPVectorizer: start constructing chains at stores that are not power of two.
The type <3 x i8> is a common in graphics and we want to be able to vectorize it.

This changes accelerates bullet by 12% and 471_omnetpp by 5%.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184317 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-19 15:57:29 +00:00
Nadav Rotem
eb30e5115e SLPVectorizer: vectorize compares and selects.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184282 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-19 05:49:52 +00:00
Pekka Jaaskelainen
a8a04380c5 Fix for a regression caused by the LoopVectorizer when
vectorizing loops with memory accesses to non-zero address spaces. It
simply dropped the AS info. Fixes PR16306.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184103 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-17 18:49:06 +00:00
Derek Schuff
c06e5cf2e3 Fix DeleteDeadVarargs not to crash on functions referenced by BlockAddresses
This pass was assuming that if hasAddressTaken() returns false for a
function, the function's only uses are call sites.  That's not true
because there can be references by BlockAddresses too.

Fix the pass to handle this case.  Fix
BlockAddress::replaceUsesOfWithOnConstant() to allow a function's type
to be changed by RAUW'ing the function with a bitcast of the recreated
function.

Patch by Mark Seaborn.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183933 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-13 19:51:17 +00:00
Rafael Espindola
100fbdd06b Always remove an alias when we rename the target.
Should fix the dragonegg build bots.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183845 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-12 16:45:47 +00:00
Rafael Espindola
6d6cbe3a3a Convert test to FileCheck.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183843 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-12 16:35:53 +00:00
Rafael Espindola
d1b6ca23b4 Change how globalopt handles aliases in llvm.used.
Instead of a custom implementation of replaceAllUsesWith, we just call
replaceAllUsesWith and recreate llvm.used and llvm.compiler-used.

This change is particularity interesting because it makes llvm see
through what clang is doing with static used functions in extern "C"
contexts. With this change, running clang -O2 in

extern "C" {
  __attribute__((used)) static void foo() {}
}

produces

@llvm.used = appending global [1 x i8*] [i8* bitcast (void ()* @foo to
i8*)], section "llvm.metadata"
define internal void @foo() #0 {
entry:
  ret void
}

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183756 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-11 17:48:06 +00:00
Tim Northover
d84b17e157 Make DeadArgumentElimination more conservative on variadic functions
Variadic functions are particularly fragile in the face of ABI changes, so this
limits how much the pass changes them

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183625 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-09 02:17:27 +00:00
Shuxin Yang
9792b646c6 Fix a potential bug in r183584.
r183584 tries to derive some info from the code *AFTER* a call and apply
these derived info to the code *BEFORE* the call, which is not always safe
as the call in question may never return, and in this case, the derived
info is invalid.
  
  Thank Duncan for pointing out this potential bug.

rdar://14073661 


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183606 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-08 04:56:05 +00:00
Shuxin Yang
1c2b03aae9 Fix an assertion in MemCpyOpt pass.
The MemCpyOpt pass is capable of optimizing:
      callee(&S); copy N bytes from S to D.
    into:
      callee(&D);
subject to some legality constraints. 

  Assertion is triggered when the compiler tries to evalute "sizeof(typeof(D))",
while D is an opaque-typed, 'sret' formal argument of function being compiled.
i.e. the signature of the func being compiled is something like this:
  T caller(...,%opaque* noalias nocapture sret %D, ...)

  The fix is that when come across such situation, instead of calling some
utility functions to get the size of D's type (which will crash), we simply
assume D has at least N bytes as implified by the copy-instruction.

rdar://14073661 


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183584 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-07 22:45:21 +00:00
Michael Gottesman
9eb856bc29 [objc-arc] Ensure that the cfg path count does not overflow when we multiply TopDownPathCount/BottomUpPathCount.
rdar://12480535

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183489 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-07 06:16:49 +00:00
Rafael Espindola
ab9ba5321d Add a testcase from pr16244.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183433 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-06 19:15:23 +00:00
David Majnemer
5a57dbef33 IndVarSimplify: check if loop invariant expansion can trap
IndVarSimplify is willing to move divide instructions outside of their
loop bodies if they are invariant of the loop.  However, it may not be
safe to expand them if we do not know if they can trap.

Instead, check to see if it is not safe to expand the instruction and
skip the expansion.

This fixes PR16041.

Testcase by Rafael Ávila de Espíndola.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183239 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-04 17:51:58 +00:00
Rafael Espindola
babae05237 Second part of pr16069
The problem this time seems to be a thinko. We were assuming that in the CFG

A
| \
|  B
| /
C

speculating the basic block B would cause only the phi value for the B->C edge
to be speculated. That is not true, the phi's are semantically in the edges, so
if the A->B->C path is taken, any code needed for A->C is not executed and we
have to consider it too when deciding to speculate B.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183226 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-04 14:11:59 +00:00
David Majnemer
3931bdb788 SimplifyCFG: Do not transform PHI to select if doing so would be unsafe
PR16069 is an interesting case where an incoming value to a PHI is a
trap value while also being a 'ConstantExpr'.

We do not consider this case when performing the 'HoistThenElseCodeToIf'
optimization.

Instead, make our modifications more conservative if we detect that we
cannot transform the PHI to a select.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183152 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-03 20:43:12 +00:00
Nick Lewycky
e4546cb71e When determining the new index for an insertelement, we may not assume that an
index greater than the size of the vector is invalid. The shuffle may be
shrinking the size of the vector. Fixes a crash!

Also drop the maximum recursion depth of the safety check for this
optimization to five.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183080 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-01 20:51:31 +00:00
Andrew Trick
cd8e3c4dcf Prevent loop-unroll from making assumptions about undefined behavior.
Fixes rdar:14036816, PR16130.

There is an opportunity to compute precise trip counts for 'or'
expressions and multi-exit loops.
rdar:14038809: Optimize trip count computation for multi-exit loops.

To do this we need to record the fact that ExitLimit assumes NSW. When
it does not we can safely assume that the loop trip count is the
minimum ExitLimt across all subexpressions and loop exits.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183060 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-31 23:34:46 +00:00
Arnold Schwaighofer
47afc19625 LoopVectorize: PHIs with only outside users should prevent vectorization
We check that instructions in the loop don't have outside users (except if
they are reduction values). Unfortunately, we skipped this check for
if-convertable PHIs.

Fixes PR16184.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183035 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-31 19:53:50 +00:00
Quentin Colombet
5b00f4edcb Modify how the formulae are rated in Loop Strength Reduce.
Namely, check if the target allows to fold more that one register in the
addressing mode and if yes, adjust the cost accordingly.

Prior to this commit, reg1 + scale * reg2 accesses were artificially preferred
to reg1 + reg2 accesses. Indeed, the cost model wrongly assumed that reg1 + reg2
needs a temporary register for the computation, whereas it was correctly
estimated for reg1 + scale * reg2.

<rdar://problem/13973908>


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183021 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-31 17:20:29 +00:00
Rafael Espindola
4f3d7eea04 Simplify multiplications by vectors whose elements are powers of 2.
Patch by Andrea Di Biagio.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183005 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-31 14:27:15 +00:00
Nick Lewycky
903f26d904 Reapply with r182909 with a fix to the calculation of the new indices for
insertelement instructions.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182976 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-31 00:59:42 +00:00
Evgeniy Stepanov
ccb7bd9d84 Revert r182909.
PR/16177


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182919 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-30 09:40:17 +00:00
Nick Lewycky
e97b102e2b Swizzle vector inputs if it helps us eliminate shuffles.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182909 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-30 04:33:38 +00:00
Paul Redmond
ee21b6f7b4 Add support for llvm.vectorizer metadata
- llvm.loop.parallel metadata has been renamed to llvm.loop to be more generic
  by making the root of additional loop metadata.
  - Loop::isAnnotatedParallel now looks for llvm.loop and associated
    llvm.mem.parallel_loop_access
  - document llvm.loop and update llvm.mem.parallel_loop_access
- add support for llvm.vectorizer.width and llvm.vectorizer.unroll
  - document llvm.vectorizer.* metadata
  - add utility class LoopVectorizerHints for getting/setting loop metadata
  - use llvm.vectorizer.width=1 to indicate already vectorized instead of
    already_vectorized
- update existing tests that used llvm.loop.parallel and
  llvm.vectorizer.already_vectorized

Reviewed by: Nadav Rotem


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182802 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-28 20:00:34 +00:00
Andrew Trick
dd0fb018a7 Track IR ordering of SelectionDAG nodes 3/4.
Remove the old IR ordering mechanism and switch to new one.  Fix unit
test failures.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182704 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-25 03:08:10 +00:00
Michael Gottesman
7932798384 [objc-arc] KnownSafe does not imply that it is safe to perform code motion across CFG edges since even if it is safe to remove RR pairs, we may still be able to move a retain/release into a loop.
rdar://13949644

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182670 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-24 20:44:05 +00:00
Michael Gottesman
35e88e57ea [objc-arc] Make sure that multiple owners is propogated correctly through the pass via the usage of a global data structure.
rdar://13750319

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182669 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-24 20:44:02 +00:00
Benjamin Kramer
959ecb2eec LoopVectorize: LoopSimplify can't canonicalize loops with an indirectbr in it, don't assert on those cases.
Fixes PR16139.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182656 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-24 18:05:35 +00:00
Joey Gouly
4a941316cd scalarizePHI needs to insert the next ExtractElement in the same block
as the BinaryOperator, *not* in the block where the IRBuilder is currently
inserting into. Fixes a bug where scalarizePHI would create instructions
that would not dominate all uses.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182639 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-24 12:29:54 +00:00
Nadav Rotem
3f75c6cfb5 SLPVectorizer: Change the order in which new instructions are added to the function.
We are not working on a DAG and I ran into a number of problems when I enabled the vectorizations of 'diamond-trees' (trees that share leafs).
* Imroved the numbering API.
* Changed the placement of new instructions to the last root.
* Fixed a bug with external tree users with non-zero lane.
* Fixed a bug in the placement of in-tree users.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182508 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-22 19:47:32 +00:00
Jean-Luc Duprat
5e6cabd163 This is an update to a previous commit (r181216).
The earlier change list introduced the following inst combines:
B * (uitofp i1 C) —> select C, B, 0
A * (1 - uitofp i1 C) —> select C, 0, A
select C, 0, B + select C, A, 0 —> select C, A, B

Together these 3 changes would simplify :
A * (1 - uitofp i1 C) + B * uitofp i1 C 
down to :
select C, B, A

In practice we found that the first two substitutions can have a
negative effect on performance, because they reduce opportunities to
use FMA contractions; between the two options FMAs are often the
better choice.  This change list amends the previous one to enable
just these inst combines:

select C, B, 0 + select C, 0, A —> select C, B, A
A * (1 - uitofp i1 C) + B * uitofp i1 C —> select C, B, A




git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182499 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-22 18:29:31 +00:00
Arnold Schwaighofer
6e4a9c14f6 LoopVectorize: Make Value pointers that could be RAUW'ed a VH
The Value pointers we store in the induction variable list can be RAUW'ed by a
call to SCEVExpander::expandCodeFor, use a TrackingVH instead. Do the same thing
in some other places where we store pointers that could potentially be RAUW'ed.

Fixes PR16073.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182485 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-22 16:54:56 +00:00
Benjamin Kramer
ebe7a52a17 Move the remaining simplify-libcalls tests to instcombine, merging most of them into a single file.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182211 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-19 13:28:39 +00:00
David Majnemer
cb9d4667b7 isKnownToBeAPowerOfTwo: (X & Y) + Y is a power of 2 or zero if y is also.
This is useful if something that looks like (x & (1 << y)) ? 64 : 32 is
the divisor in a modulo operation.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182200 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-18 19:30:37 +00:00
Arnold Schwaighofer
688b5103eb LoopVectorize: Handle single edge PHIs
We might encouter single edge PHIs - handle them with an identity select.

Fixes PR15990.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182199 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-18 18:38:34 +00:00
Richard Smith
eb351eb849 Respect the 'nobuiltin' attribute when determining if a call is to a memory builtin.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181978 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-16 04:12:04 +00:00
Arnold Schwaighofer
1386692ef6 LoopVectorize: Hoist conditional loads if possible
InstCombine can be uncooperative to vectorization and sink loads into
conditional blocks. This prevents vectorization.

Undo this optimization if there are unconditional memory accesses to the same
addresses in the loop.

radar://13815763

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181860 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-15 01:44:30 +00:00
Manman Ren
5150270d07 GlobalOpt: fix an issue where CXAAtExitFn points to a deleted function.
CXAAtExitFn was set outside a loop and before optimizations where functions
can be deleted. This patch will set CXAAtExitFn inside the loop and after
optimizations.

Seg fault when running LTO because of accesses to a deleted function.
rdar://problem/13838828


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181838 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-14 21:52:44 +00:00
Arnold Schwaighofer
123f18bcb9 LoopVectorize: Handle loops with multiple forward inductions
We used to give up if we saw two integer inductions. After this patch, we base
further induction variables on the chosen one like we do in the reverse
induction and pointer induction case.

Fixes PR15720.

radar://13851975

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181746 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-14 00:21:18 +00:00
Michael Gottesman
acfb3584c5 [objc-arc-opts] In the presense of an alloca unconditionally remove RR pairs if and only if we are both KnownSafeBU/KnownSafeTD rather than just either or.
In the presense of a block being initialized, the frontend will emit the
objc_retain on the original pointer and the release on the pointer loaded from
the alloca. The optimizer will through the provenance analysis realize that the
two are related (albiet different), but since we only require KnownSafe in one
direction, will match the inner retain on the original pointer with the guard
release on the original pointer. This is fixed by ensuring that in the presense
of allocas we only unconditionally remove pointers if both our retain and our
release are KnownSafe (i.e. we are KnownSafe in both directions) since we must
deal with the possibility that the frontend will emit what (to the optimizer)
appears to be unbalanced retain/releases.

An example of the miscompile is:

  %A = alloca
  retain(%x)
  retain(%x) <--- Inner Retain
  store %x, %A
  %y = load %A
  ... DO STUFF ...
  release(%y)
  call void @use(%x)
  release(%x) <--- Guarding Release

getting optimized to:

  %A = alloca
  retain(%x)
  store %x, %A
  %y = load %A
  ... DO STUFF ...
  release(%y)
  call void @use(%x)

rdar://13750319

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181743 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-13 23:49:42 +00:00
Nadav Rotem
507b9242ed SLPVectorizer: Fix a bug in the code that generates extracts for values with multiple users.
The external user does not have to be in lane #0. We have to save the lane for each scalar so that we know which vector lane to extract.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181674 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-12 22:58:45 +00:00
David Majnemer
fa49d7d6e4 InstCombine: Flip the order of two urem transforms
There are two transforms in visitUrem that conflict with each other.

*) One, if a divisor is a power of two, subtracts one from the divisor
   and turns it into a bitwise-and.
*) The other unwraps both operands if they are surrounded by zext
   instructions.

Flipping the order allows the subtraction to go beneath the sign
extension.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181668 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-12 00:07:05 +00:00
Arnold Schwaighofer
9b5d70f076 LoopVectorize: Use the widest induction variable type
Use the widest induction type encountered for the cannonical induction variable.

We used to turn the following loop into an empty loop because we used i8 as
induction variable type and truncated 1024 to 0 as trip count.

int a[1024];
void fail() {
  int reverse_induction = 1023;
  unsigned char forward_induction = 0;
  while ((reverse_induction) >= 0) {
    forward_induction++;
    a[reverse_induction] = forward_induction;
    --reverse_induction;
  }
}

radar://13862901

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181667 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-11 23:04:28 +00:00
David Majnemer
a8ccefc0a3 InstCombine: Turn urem to bitwise-and more often
Use isKnownToBeAPowerOfTwo in visitUrem so that we may more aggressively
fold away urem instructions.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181661 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-11 09:01:28 +00:00
Nadav Rotem
9bba9f6300 SLPVectorizer: Add support for trees with external users.
For example:
bar() {
  int a = A[i];
  int b = A[i+1];
  B[i] = a;
  B[i+1] = b;
  foo(a);  <--- a is used outside the vectorized expression.
}



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181648 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-10 22:59:33 +00:00
Nadav Rotem
ef3e423e23 Add an additional testcase for PR15882.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181646 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-10 22:55:44 +00:00
Benjamin Kramer
7159a307b9 InstCombine: Don't claim to be able to evaluate any shl in a zexted type.
The shift amount may be larger than the type leading to undefined behavior.
Limit the transform to constant shift amounts. While there update the bits to
clear in the result which may enable additional optimizations.

PR15959.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181604 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-10 16:26:37 +00:00
Benjamin Kramer
51dab6e394 InstCombine: Verify the type before transforming uitofp into select.
PR15952.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181586 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-10 09:16:52 +00:00
Rafael Espindola
95f885390b Don't replace an alias in llvm.used with its target.
When we replace an internal alias with its target, be careful not to
replace the entry in llvm.used (and llvm.compiler_used).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181524 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-09 17:22:59 +00:00
Benjamin Kramer
a6ff92a975 InstCombine: Don't just copy known bits from the first operand of an srem.
That's obviously wrong. Conservatively restrict it to the sign bit, which
matches the original intention of this analysis. Fixes PR15940.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181518 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-09 16:32:32 +00:00
Arnold Schwaighofer
c121f5dc26 LoopVectorizer: Don't assert on the absence of induction variables
A computable loop exit count does not imply the presence of an induction
variable. Scalar evolution can return a value for an infinite loop.

Fixes PR15926.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181495 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-09 00:32:18 +00:00
Daniel Malea
f7f4184710 Revert 181475 as the DebugIR tests are breaking (automake) buildbots that re-use build dirs
- the temporaries "-debug.ll" files generated by DebugIR pass are considered tests, even though they are not



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181476 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-08 21:55:31 +00:00
Daniel Malea
0fd09cd99b DebugIR tests -- lit tests for the line number transform
- simple one-function case
- function-calling case
- external function calling case
- exception throwing case
- vector case

Note: these tests are somewhat coupled to the current format of debug metadata.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181469 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-08 21:03:00 +00:00
Arnold Schwaighofer
280e1df858 LoopVectorizer: Improve reduction variable identification
The two nested loops were confusing and also conservative in identifying
reduction variables. This patch replaces them by a worklist based approach.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181369 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-07 21:55:37 +00:00
Arnold Schwaighofer
eb95cec176 LoopVectorize: getConsecutiveVector must respect signed arithmetic
We were passing an i32 to ConstantInt::get where an i64 was needed and we must
also pass the sign if we pass negatives numbers. The start index passed to
getConsecutiveVector must also be signed.

Should fix PR15882.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181286 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-07 04:37:05 +00:00
David Majnemer
8ec23cb07e InstCombine: (X ^ signbit) + C -> X + (signbit ^ C)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181249 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-06 21:21:31 +00:00
Jean-Luc Duprat
0aa2b8800c Test results verified using FileCheck rather than grep | count
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181234 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-06 18:45:16 +00:00
Andrew Trick
fcf79528da Rotate multi-exit loops even if the latch was simplified.
Test case by Michele Scandale!

Fixes PR10293: Load not hoisted out of loop with multiple exits.

There are few regressions with this patch, now tracked by
rdar:13817079, and a roughly equal number of improvements. The
regressions are almost certainly back luck because LoopRotate has very
little idea of whether rotation is profitable. Doing better requires a
more comprehensive solution.

This checkin is a quick fix that lacks generality (PR10293 has
a counter-example). But it trivially fixes the case in PR10293 without
interfering with other cases, and it does satify the criteria that
LoopRotate is a loop canonicalization pass that should avoid
heuristics and special cases.

I can think of two approaches that would probably be better in
the long run. Ultimately they may both make sense.

(1) LoopRotate should check that the current header would make a good
loop guard, and that the loop does not already has a sufficient
guard. The artifical SimplifiedLoopLatch check would be unnecessary,
and the design would be more general and canonical. Two difficulties:

- We need a strong guarantee that we won't endlessly rotate, so the
  analysis would need to be precise in order to avoid the
  SimplifiedLoopLatch precondition.

- Analysis like this are usually based on SCEV, which we don't want to
  rely on.

(2) Rotate on-demand in late loop passes. This could even be done by
shoving the loop back on the queue after the optimization that needs
it. This could work well when we find LICM opportunities in
multi-branch loops. This requires some work, and it doesn't really
solve the problem of SCEV wanting a loop guard before the analysis.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181230 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-06 17:58:18 +00:00
Jean-Luc Duprat
81fa2abc85 Fix add4.ll test cmdline so that it passes
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181219 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-06 17:18:47 +00:00
Jean-Luc Duprat
c5cf6e5365 Provide InstCombines for the following 3 cases:
A * (1 - (uitofp i1 C)) -> select C, 0, A
B * (uitofp i1 C) -> select C, B, 0
select C, 0, A + select C, B, 0 -> select C, B, A

These come up in code that has been hand-optimized from a select to a linear blend, 
on platforms where that may have mattered. We want to undo such changes 
with the following transform:
A*(1 - uitofp i1 C) + B*(uitofp i1 C) -> select C, A, B



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181216 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-06 16:55:50 +00:00
Nadav Rotem
4ee312bac1 Revert r164763 because it introduces new shuffles.
Thanks Nick Lewycky for pointing this out.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181177 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-06 02:39:09 +00:00
Matt Arsenault
8b9dc21d6f Fix unchecked uses of DominatorTree in MemoryDependenceAnalysis.
Use unknown results for places where it would be needed

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181176 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-06 02:07:24 +00:00
Rafael Espindola
eaf14786ca Fix const merging when an alias of a const is llvm.used.
We used to disable constant merging not only if a constant is llvm.used, but
also if an alias of a constant is llvm.used. This change fixes that.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181175 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-06 01:48:55 +00:00
Arnold Schwaighofer
87defd0924 LoopVectorize: Add support for floating point min/max reductions
Add support for min/max reductions when "no-nans-float-math" is enabled. This
allows us to assume we have ordered floating point math and treat ordered and
unordered predicates equally.

radar://13723044

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181144 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-05 01:54:48 +00:00
Arnold Schwaighofer
c1738fdadd LoopVectorize: We don't need an identity element for min/max reductions
We can just use the initial element that feeds the reduction.

  max(max(x, y), z) == max(max(x,y), max(x,z))

radar://13723044

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181141 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-05 01:54:42 +00:00
Nadav Rotem
4bcd5f888f LoopVectorizer: Add support for if-conversion of PHINodes with 3+ incoming values.
By supporting the vectorization of PHINodes with more than two incoming values we can increase the complexity of nested if statements.

We can now vectorize this loop:

int foo(int *A, int *B, int n) {
  for (int i=0; i < n; i++) {
    int x = 9;
    if (A[i] > B[i]) {
      if (A[i] > 19) {
        x = 3;
      } else if (B[i] < 4 ) {
        x = 4;
      } else {
        x = 5;
      }
    }
    A[i] = x;
  }
}



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181037 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-03 17:42:55 +00:00
Manman Ren
436849be6a TBAA: remove !tbaa from testing cases if not used.
This will make it easier to turn on struct-path aware TBAA since the metadata
format will change.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180935 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-02 18:11:35 +00:00
David Majnemer
304099a56d Add a test for the foldSelectICmpAndOr fix committed in r180779.
This tests a case where C1 and C2 were the same but X and Y were different
widths.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180907 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-02 02:44:23 +00:00
Nadav Rotem
fee6969463 SROA: Generate selects instead of shuffles when blending values because this is the cannonical form.
Shuffles are more difficult to lower and we usually don't touch them, while we do optimize selects more often.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180875 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-01 19:53:30 +00:00
Jim Grosbach
10cc563bfe Revert "InstCombine: Fold more shuffles of shuffles."
This reverts commit r180802

There's ongoing discussion about whether this is the right place to make
this transformation. Reverting for now while we figure it out.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180834 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-01 00:25:27 +00:00
Jim Grosbach
6548096a2e InstCombine: Fold more shuffles of shuffles.
Always fold a shuffle-of-shuffle into a single shuffle when there's only one
input vector in the first place. Continue to be more conservative when there's
multiple inputs.

rdar://13402653
PR15866

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180802 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-30 20:43:52 +00:00
Manman Ren
2dc50d3067 TBAA: remove !tbaa from testing cases if not used.
This will make it easier to turn on struct-path aware TBAA since the metadata
format will change.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180796 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-30 17:52:57 +00:00
David Majnemer
defce4cfd6 Fix "Combine bit test + conditional or into simple math"
This fixes the optimization introduced in r179748 and reverted in r179750.

While the optimization was sound, it did not properly respect differences in
bit-width.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180777 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-30 08:57:58 +00:00
Arnold Schwaighofer
8228ffe72d SimplifyCFG: If convert single conditional stores
This resurrects r179957, but adds code that makes sure we don't touch
atomic/volatile stores:

This transformation will transform a conditional store with a preceeding
uncondtional store to the same location:

 a[i] =
 may-alias with a[i] load
 if (cond)
   a[i] = Y

into an unconditional store.

 a[i] = X
 may-alias with a[i] load
 tmp = cond ? Y : X;
 a[i] = tmp

We assume that on average the cost of a mispredicted branch is going to be
higher than the cost of a second store to the same location, and that the
secondary benefits of creating a bigger basic block for other optimizations to
work on outway the potential case where the branch would be correctly predicted
and the cost of the executing the second store would be noticably reflected in
performance.

hmmer's execution time improves by 30% on an imac12,2 on ref data sets. With
this change we are on par with gcc's performance (gcc also performs this
transformation). There was a 1.2 % performance improvement on a ARM swift chip.
Other tests in the test-suite+external seem to be mostly uninfluenced in my
experiments:
This optimization was triggered on 41 tests such that the executable was
different before/after the patch. Only 1 out of the 40 tests (dealII) was
reproducable below 100% (by about .4%). Given that hmmer benefits so much I
believe this to be a fair trade off.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180731 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-29 21:28:24 +00:00
Michael Gottesman
f11a6856cc [objc-arc] Apply the RV optimization to retains next to calls in ObjCARCContract instead of ObjCARCOpts.
Turning retains into retainRV calls disrupts the data flow analysis in
ObjCARCOpts. Thus we move it as late as we can by moving it into
ObjCARCContract.

We leave in the conversion from retainRV -> retain in ObjCARCOpt since
it enables the dataflow analysis.

rdar://10813093

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180698 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-29 06:53:53 +00:00
Shuxin Yang
4d4c54d29f Fix a XOR reassociation bug.
When Reassociator optimize "(x | C1)" ^ "(X & C2)", it may swap the two
subexpressions, however, it forgot to swap cached constants (of C1 and C2)
accordingly.

rdar://13739160


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180676 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-27 18:02:12 +00:00
Michael Gottesman
2c2e85536d [objc-arc] Test cleanups.
Mainly adding paranoid checks for the closing brace of a function to
help with FileCheck error readability. Also some other minor changes.

No actual CHECK changes.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180668 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-27 05:25:54 +00:00
Nadav Rotem
7557e521e5 LoopVectorizer: Calculate the number of pointers to disambiguate at runtime based on the numbers of reads and writes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180593 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-26 05:08:59 +00:00
Nadav Rotem
975b1ddf60 LoopVectorizer: No need to generate pointer disambiguation checks between readonly pointers.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180570 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-25 19:55:03 +00:00
Arnold Schwaighofer
a4b8b4ccc9 LoopVectorize: Scalarize padded types
This patch disables memory-instruction vectorization for types that need padding
bytes, e.g., x86_fp80 has 10 bytes store size with 6 bytes padding in darwin on
x86_64. Because the load/store vectorization is performed by the bit casting to
a packed vector, which has incompatible memory layout due to the lack of padding
bytes, the present vectorizer produces inconsistent result for memory
instructions of those types.
This patch checks an equality of the AllocSize of a scalar type and allocated
size for each vector element, to ensure that there is no padding bytes and the
array can be read/written using vector operations.

Patch by Daisuke Takahashi!

Fixes PR15758.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180196 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-24 16:16:01 +00:00
Arnold Schwaighofer
b03ad17536 LoopVectorizer: Bail out if we don't have datalayout we need it
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180195 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-24 16:15:58 +00:00
Nadav Rotem
a7d9a6ee63 LoopVectorizer: Fix 15830. When scalarizing and unrolling stores make sure that the order in which the elements are scalarized is the same as the original order.
This fixes a miscompilation in FreeBSD's regex library.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180121 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-23 17:12:42 +00:00
Pekka Jaaskelainen
2e59a125fc Call the potentially costly isAnnotatedParallel() only once.
Made the uniform write test's checks a bit stricter.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180119 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-23 16:44:43 +00:00
Pekka Jaaskelainen
a8958769ea Refuse to (even try to) vectorize loops which have uniform writes,
even if erroneously annotated with the parallel loop metadata.

Fixes Bug 15794: 
"Loop Vectorizer: Crashes with the use of llvm.loop.parallel metadata"



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180081 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-23 08:08:51 +00:00