Commit Graph

4947 Commits

Author SHA1 Message Date
Arnold Schwaighofer
c14380d195 LoopVectorize: Math functions only read rounding mode
Math functions are mark as readonly because they read the floating point
rounding mode. Because we don't vectorize loops that would contain function
calls that set the rounding mode it is safe to ignore this memory read.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185299 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-01 00:54:44 +00:00
Stephen Lin
3f4385a0b6 DeadArgumentElimination: keep return value on functions that have a live argument with the 'returned' attribute (rather than generate invalid IR); however, if both can be eliminated, both will be
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185290 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-30 20:26:21 +00:00
Benjamin Kramer
80c070493d ConstantFold: Check that truncating the other side is safe under a sext when trying to remove a sext from a compare.
Fixes PR16462.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185284 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-30 13:47:43 +00:00
David Majnemer
58b6dc032a ValueTracking: Teach isKnownToBeAPowerOfTwo about (ADD X, (XOR X, Y)) where X is a power of two
This allows us to simplify urem instructions involving the add+xor to
turn into simpler math.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185272 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-29 23:44:53 +00:00
Benjamin Kramer
edac9151fd InstCombine: Also turn selects fed by an and into arithmetic when the types don't match.
Inserting a zext or trunc is sufficient. This pattern is somewhat common in
LLVM's pointer mangling code.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185270 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-29 21:17:04 +00:00
David Majnemer
c22a4eeec6 InstCombine: FoldGEPICmp shouldn't change sign of base pointer comparison
Changing the sign when comparing the base pointer would introduce all
sorts of unexpected things like:
  %gep.i = getelementptr inbounds [1 x i8]* %a, i32 0, i32 0
  %gep2.i = getelementptr inbounds [1 x i8]* %b, i32 0, i32 0
  %cmp.i = icmp ult i8* %gep.i, %gep2.i
  %cmp.i1 = icmp ult [1 x i8]* %a, %b
  %cmp = icmp ne i1 %cmp.i, %cmp.i1
  ret i1 %cmp

into:
  %cmp.i = icmp slt [1 x i8]* %a, %b
  %cmp.i1 = icmp ult [1 x i8]* %a, %b
  %cmp = xor i1 %cmp.i, %cmp.i1
  ret i1 %cmp

By preserving the original sign, we now get:
  ret i1 false

This fixes PR16483.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185259 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-29 10:28:04 +00:00
David Majnemer
f723e5d1c2 InstCombine: Be more agressive optimizing 'udiv' instrs with 'select' denoms
Real world code sometimes has the denominator of a 'udiv' be a
'select'.  LLVM can handle such cases but only when the 'select'
operands are symmetric in structure (both select operands are a constant
power of two or a left shift, etc.).  This falls apart if we are dealt a
'udiv' where the code is not symetric or if the select operands lead us
to more select instructions.

Instead, we should treat the LHS and each select operand as a distinct
divide operation and try to optimize them independently.  If we can
to simplify each operation, then we can replace the 'udiv' with, say, a
'lshr' that has a new select with a bunch of new operands for the
select.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185257 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-29 08:40:07 +00:00
David Majnemer
b41f4bbfbd InstCombine: Optimize (1 << X) Pred CstP2 to X Pred Log2(CstP2)
We may, after other optimizations, find ourselves with IR that looks
like:

  %shl = shl i32 1, %y
  %cmp = icmp ult i32 %shl, 32

Instead, we should just compare the shift count:

  %cmp = icmp ult i32 %y, 5


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185242 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-28 23:42:03 +00:00
Nadav Rotem
ac26786846 SLP Vectorizer: Add support for trees with external users.
To support this we have to insert 'extractelement' instructions to pick the right lane.
We had this functionality before but I removed it when we moved to the multi-block design because it was too complicated.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185230 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-28 22:07:09 +00:00
Daniel Malea
5fa8186b8d Adding tests for DebugIR pass
- lit tests verify that each line of input LLVM IR gets a !dbg node and a
  corresponding entry of metadata that contains the line number 
- unit tests verify that DebugIR works as advertised in the interface
- refactored some useful IR generation functionality from the MCJIT unit tests
  so it can be reused



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185212 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-28 20:37:20 +00:00
Manman Ren
cbafae6d33 Debug Info: clean up usage of Verify.
No functionality change.
It should suffice to check the type of a debug info metadata, instead of
calling Verify. For cases where we know the type of a DI metadata, use
assert.

Also update testing cases to make them conform to the format of DI classes.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185135 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-28 05:43:10 +00:00
Matt Arsenault
1300638d50 Convert tests to FileCheck
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185124 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-28 01:29:35 +00:00
Arnold Schwaighofer
57a7da8b23 LoopVectorize: Preserve debug location info
radar://14169017

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185122 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-28 00:38:54 +00:00
Arnold Schwaighofer
0bbbf7cbb0 LoopVectorize: Cache edge masks created during if-conversion
Otherwise, we end up with an exponential IR blowup.
Fixes PR16472.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185097 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-27 20:31:06 +00:00
Arnold Schwaighofer
0862d589ee LoopVectorize: Use vectorized loop invariant gep index anchored in loop
Use vectorized instruction instead of original instruction anchored in the
original loop.

Fixes PR16452 and t2075.c of PR16455.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185081 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-27 15:11:55 +00:00
Manman Ren
084476a3ee Update testing case to make DI nodes have the correct format.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185061 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-27 06:40:18 +00:00
Arnold Schwaighofer
e2b9912a78 Fix spelling.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185052 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-27 01:01:11 +00:00
Arnold Schwaighofer
45ef457b8f LoopVectorize: Don't store a reversed value in the vectorized value map
When we store values for reversed induction stores we must not store the
reversed value in the vectorized value map. Another instruction might use this
value.

This fixes 3 test cases of PR16455.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185051 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-27 00:45:41 +00:00
Michael Gottesman
2253a2f52f Added support for the Builtin attribute.
The Builtin attribute is an attribute that can be placed on function call site that signal that even though a function is declared as being a builtin,

rdar://problem/13727199

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185049 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-27 00:25:01 +00:00
Nadav Rotem
e26fa2dd73 Erase all of the instructions that we RAUWed
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184969 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-26 17:16:09 +00:00
Nadav Rotem
29acf7e03a Do not add cse-ed instructions into the visited map because we dont want to consider them as a candidate for replacement of instructions to be visited.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184966 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-26 16:54:53 +00:00
Nadav Rotem
805e8a01fe SLPVectorizer: support slp-vectorization of PHINodes between basic blocks
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184888 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-25 23:04:09 +00:00
Bob Wilson
a1fe2948ed Fix SROA to avoid unnecessary scalar conversions for 1-element vectors.
When a 1-element vector alloca is promoted, a store instruction can often be
rewritten without converting the value to a scalar and using an insertelement
instruction to stuff it into the new alloca.  This patch just adds a check
to skip that conversion when it is unnecessary.  This turns out to be really
important for some ARM Neon operations where <1 x i64> is used to get around
the fact that i64 is not a legal type.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184870 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-25 19:09:50 +00:00
Arnold Schwaighofer
bc7c58d2b1 Reapply 184685 after the SetVector iteration order fix.
This should hopefully have fixed the stage2/stage3 miscompare on the dragonegg
testers.

"LoopVectorize: Use the dependence test utility class

We now no longer need alias analysis - the cases that alias analysis would
handle are now handled as accesses with a large dependence distance.

We can now vectorize loops with simple constant dependence distances.

  for (i = 8; i < 256; ++i) {
    a[i] = a[i+4] * a[i+8];
  }

  for (i = 8; i < 256; ++i) {
    a[i] = a[i-4] * a[i-8];
  }

We would be able to vectorize about 200 more loops (in many cases the cost model
instructs us no to) in the test suite now. Results on x86-64 are a wash.

I have seen one degradation in ammp. Interestingly, the function in which we
now vectorize a loop is never executed so we probably see some instruction
cache effects. There is a 2% improvement in h264ref. There is one or the other
TSCV loop kernel that speeds up.

radar://13681598"

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184724 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-24 12:09:15 +00:00
Arnold Schwaighofer
ec677e2a64 Revert "LoopVectorize: Use the dependence test utility class"
This reverts commit cbfa1ca993.

We are seeing a stage2 and stage3 miscompare on some dragonegg bots.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184690 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-24 06:10:41 +00:00
Arnold Schwaighofer
cbfa1ca993 LoopVectorize: Use the dependence test utility class
We now no longer need alias analysis - the cases that alias analysis would
handle are now handled as accesses with a large dependence distance.

We can now vectorize loops with simple constant dependence distances.

  for (i = 8; i < 256; ++i) {
    a[i] = a[i+4] * a[i+8];
  }

  for (i = 8; i < 256; ++i) {
    a[i] = a[i-4] * a[i-8];
  }

We would be able to vectorize about 200 more loops (in many cases the cost model
instructs us no to) in the test suite now. Results on x86-64 are a wash.

I have seen one degradation in ammp. Interestingly, the function in which we
now vectorize a loop is never executed so we probably see some instruction
cache effects. There is a 2% improvement in h264ref. There is one or the other
TSCV loop kernel that speeds up.

radar://13681598

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184685 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-24 03:55:48 +00:00
Nadav Rotem
25961b469a SLP Vectorizer: Add support for vectorizing parts of the tree.
Untill now we detected the vectorizable tree and evaluated the cost of the
entire tree.  With this patch we can decide to trim-out branches of the tree
that are not profitable to vectorizer.

Also, increase the max depth from 6 to 12. In the worse possible case where all
of the code is made of diamond-shaped graph this can bring the cost to 2**10,
but diamonds are not very common.




git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184681 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-24 02:52:43 +00:00
Nadav Rotem
722b0a4d29 SLP Vectorizer: Fix a bug in the code that does CSE on the generated gather sequences.
Make sure that we don't replace and RAUW two sequences if one does not dominate the other.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184674 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-23 21:57:27 +00:00
Nadav Rotem
6959f08f44 SLP Vectorizer: Implement a simple CSE optimization for the gather sequences.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184660 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-23 06:15:46 +00:00
Nadav Rotem
53a0552b06 SLP Vectorizer: Implement multi-block slp-vectorization.
Rewrote the SLP-vectorization as a whole-function vectorization pass. It is now able to vectorize chains across multiple basic blocks.
It still does not vectorize PHIs, but this should be easy to do now that we scan the entire function.
I removed the support for extracting values from trees.
We are now able to vectorize more programs, but there are some serious regressions in many workloads (such as flops-6 and mandel-2).



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184647 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-22 21:34:10 +00:00
Nadav Rotem
d69d9f20bc SLPVectorization: Add a basic support for cross-basic block slp vectorization.
We collect gather sequences when we vectorize basic blocks. Gather sequences are excellent
hints for vectorization of other basic blocks.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184444 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-20 17:41:45 +00:00
Matt Arsenault
ad966ea7a8 Move StructurizeCFG out of R600 to generic Transforms.
Register it with PassManager

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184343 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-19 20:18:24 +00:00
Quentin Colombet
5a2fb058d3 LSR: Fix the parameters used to compute the scaling factor cost.
Prior to this change, the considered addressing modes may be invalid since the
maximum and minimum offsets were not taking into account.
This was causing an assertion failure.

The added test case exercices that behavior.

<rdar://problem/14199725> Assertion failed: (CurScaleCost >= 0 && "Legal
addressing mode has an illegal cost!")


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184341 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-19 19:59:41 +00:00
Nadav Rotem
80a6d6526c SLPVectorizer: handle scalars that are extracted from vectors (using ExtractElementInst).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184325 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-19 17:33:16 +00:00
Nadav Rotem
7d180ac7b6 SLPVectorizer: start constructing chains at stores that are not power of two.
The type <3 x i8> is a common in graphics and we want to be able to vectorize it.

This changes accelerates bullet by 12% and 471_omnetpp by 5%.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184317 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-19 15:57:29 +00:00
Nadav Rotem
eb30e5115e SLPVectorizer: vectorize compares and selects.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184282 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-19 05:49:52 +00:00
Pekka Jaaskelainen
a8a04380c5 Fix for a regression caused by the LoopVectorizer when
vectorizing loops with memory accesses to non-zero address spaces. It
simply dropped the AS info. Fixes PR16306.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184103 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-17 18:49:06 +00:00
Derek Schuff
c06e5cf2e3 Fix DeleteDeadVarargs not to crash on functions referenced by BlockAddresses
This pass was assuming that if hasAddressTaken() returns false for a
function, the function's only uses are call sites.  That's not true
because there can be references by BlockAddresses too.

Fix the pass to handle this case.  Fix
BlockAddress::replaceUsesOfWithOnConstant() to allow a function's type
to be changed by RAUW'ing the function with a bitcast of the recreated
function.

Patch by Mark Seaborn.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183933 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-13 19:51:17 +00:00
Rafael Espindola
100fbdd06b Always remove an alias when we rename the target.
Should fix the dragonegg build bots.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183845 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-12 16:45:47 +00:00
Rafael Espindola
6d6cbe3a3a Convert test to FileCheck.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183843 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-12 16:35:53 +00:00
Rafael Espindola
d1b6ca23b4 Change how globalopt handles aliases in llvm.used.
Instead of a custom implementation of replaceAllUsesWith, we just call
replaceAllUsesWith and recreate llvm.used and llvm.compiler-used.

This change is particularity interesting because it makes llvm see
through what clang is doing with static used functions in extern "C"
contexts. With this change, running clang -O2 in

extern "C" {
  __attribute__((used)) static void foo() {}
}

produces

@llvm.used = appending global [1 x i8*] [i8* bitcast (void ()* @foo to
i8*)], section "llvm.metadata"
define internal void @foo() #0 {
entry:
  ret void
}

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183756 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-11 17:48:06 +00:00
Tim Northover
d84b17e157 Make DeadArgumentElimination more conservative on variadic functions
Variadic functions are particularly fragile in the face of ABI changes, so this
limits how much the pass changes them

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183625 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-09 02:17:27 +00:00
Shuxin Yang
9792b646c6 Fix a potential bug in r183584.
r183584 tries to derive some info from the code *AFTER* a call and apply
these derived info to the code *BEFORE* the call, which is not always safe
as the call in question may never return, and in this case, the derived
info is invalid.
  
  Thank Duncan for pointing out this potential bug.

rdar://14073661 


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183606 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-08 04:56:05 +00:00
Shuxin Yang
1c2b03aae9 Fix an assertion in MemCpyOpt pass.
The MemCpyOpt pass is capable of optimizing:
      callee(&S); copy N bytes from S to D.
    into:
      callee(&D);
subject to some legality constraints. 

  Assertion is triggered when the compiler tries to evalute "sizeof(typeof(D))",
while D is an opaque-typed, 'sret' formal argument of function being compiled.
i.e. the signature of the func being compiled is something like this:
  T caller(...,%opaque* noalias nocapture sret %D, ...)

  The fix is that when come across such situation, instead of calling some
utility functions to get the size of D's type (which will crash), we simply
assume D has at least N bytes as implified by the copy-instruction.

rdar://14073661 


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183584 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-07 22:45:21 +00:00
Michael Gottesman
9eb856bc29 [objc-arc] Ensure that the cfg path count does not overflow when we multiply TopDownPathCount/BottomUpPathCount.
rdar://12480535

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183489 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-07 06:16:49 +00:00
Rafael Espindola
ab9ba5321d Add a testcase from pr16244.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183433 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-06 19:15:23 +00:00
David Majnemer
5a57dbef33 IndVarSimplify: check if loop invariant expansion can trap
IndVarSimplify is willing to move divide instructions outside of their
loop bodies if they are invariant of the loop.  However, it may not be
safe to expand them if we do not know if they can trap.

Instead, check to see if it is not safe to expand the instruction and
skip the expansion.

This fixes PR16041.

Testcase by Rafael Ávila de Espíndola.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183239 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-04 17:51:58 +00:00
Rafael Espindola
babae05237 Second part of pr16069
The problem this time seems to be a thinko. We were assuming that in the CFG

A
| \
|  B
| /
C

speculating the basic block B would cause only the phi value for the B->C edge
to be speculated. That is not true, the phi's are semantically in the edges, so
if the A->B->C path is taken, any code needed for A->C is not executed and we
have to consider it too when deciding to speculate B.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183226 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-04 14:11:59 +00:00
David Majnemer
3931bdb788 SimplifyCFG: Do not transform PHI to select if doing so would be unsafe
PR16069 is an interesting case where an incoming value to a PHI is a
trap value while also being a 'ConstantExpr'.

We do not consider this case when performing the 'HoistThenElseCodeToIf'
optimization.

Instead, make our modifications more conservative if we detect that we
cannot transform the PHI to a select.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183152 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-03 20:43:12 +00:00
Nick Lewycky
e4546cb71e When determining the new index for an insertelement, we may not assume that an
index greater than the size of the vector is invalid. The shuffle may be
shrinking the size of the vector. Fixes a crash!

Also drop the maximum recursion depth of the safety check for this
optimization to five.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183080 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-01 20:51:31 +00:00