Commit Graph

4541 Commits

Author SHA1 Message Date
Chandler Carruth
70dace3052 Fix PR14212: For some strange reason I treated vectors differently from
integers in that the code to handle split alloca-wide integer loads or
stores doesn't come first. It should, for the same reasons as with
integers, and the PR attests to that. Also had to fix a busted assert in
that this test case also covers.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167051 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30 20:52:40 +00:00
Benjamin Kramer
415f869cf3 LoopIdiom: Fix a serious missed optimization: we only turned top-level loops into memmove.
Thanks to Preston Briggs for catching this!

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167045 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30 19:49:39 +00:00
Hal Finkel
8a69a95f91 BBVectorize: Fix a small bug introduced in r167042.
We need to make sure that we take the correct load/store alignment
when the inputs are flipped.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167044 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30 19:47:37 +00:00
Nadav Rotem
e709f5b600 LoopVectorize: Add support for write-only loops when the write destination is a single pointer.
Speedup SciMark by 1%



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167035 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30 18:36:45 +00:00
Nadav Rotem
cc03331caa LoopVectorize: Fix a bug in the initialization of reduction variables. AND needs to start at all-one
while XOR, and OR need to start at zero.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167032 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30 18:12:36 +00:00
Ulrich Weigand
c197a55019 Set %defaultjit to use MCJIT for PowerPC targets.
Update Transforms/LICM/2003-12-11-SinkingToPHI.ll test to use
%defaultjit as well.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167031 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30 18:07:58 +00:00
Hans Wennborg
92b469971e switch_to_lookup_table.ll: Remove some unnecessary lines, comments,
function attributes, etc.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167016 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30 15:11:52 +00:00
Ulrich Weigand
3467b9fd5d Enable some additional constant folding for PPCDoubleDouble.
This fixes Clang :: CodeGen/complex-builtints.c on PowerPC.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167013 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30 12:33:18 +00:00
Hans Wennborg
04d7d13d30 Use TargetTransformInfo to control switch-to-lookup table transformation
When the switch-to-lookup tables transform landed in SimplifyCFG, it
was pointed out that this could be inappropriate for some targets.
Since there was no way at the time for the pass to know anything about
the target, an awkward reverse-transform was added in CodeGenPrepare
that turned lookup tables back into switches for some targets.

This patch uses the new TargetTransformInfo to determine if a
switch should be transformed, and removes
CodeGenPrepare::ConvertLoadToSwitch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167011 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30 11:23:25 +00:00
Hal Finkel
c588e0e162 Remove an invalid assert in TargetTransformImpl
getCastInstrCost had an assert prohibiting scalar to vector casts. Such casts,
however, are allowed. This should make the vectorizer buildbot happier.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166998 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30 02:41:57 +00:00
Benjamin Kramer
badffcf8fd LoopIdiom: Add checks to avoid turning memmove into an infinite loop.
I don't think this is possible with the current implementation but that may change eventually.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166877 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-27 15:18:28 +00:00
Benjamin Kramer
d11c5d08a5 LoopIdiom: Recognize memmove loops.
This turns loops like
  for (unsigned i = 0; i != n; ++i)
    p[i] = p[i+1];
into memmove, which has a highly optimized implementation in most libcs.

This was really easy with the new DependenceAnalysis :)

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166875 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-27 14:25:51 +00:00
Benjamin Kramer
96c8735e28 LoopIdiom: Replace custom dependence analysis with DependenceAnalysis.
Requires a lot less code and complexity on loop-idiom's side and the more
precise analysis can catch more cases, like the one I included as a test case.
This also fixes the edge-case miscompilation from PR9481.

Compile time performance seems to be slightly worse, but this is mostly due
to an extra LCSSA run scheduled by the PassManager and should be fixed there.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166874 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-27 14:25:44 +00:00
Nadav Rotem
f065a84677 1. Fix a bug in getTypeConversion. When a *simple* type is split, we need to return the type of the split result.
2. Change the maximum vectorization width from 4 to 8.
3. A test for both.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166864 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-27 04:11:32 +00:00
Nadav Rotem
a5a3a61c5f Refactor the VectorTargetTransformInfo interface.
Add getCostXXX calls for different families of opcodes, such as casts, arithmetic, cmp, etc.

Port the LoopVectorizer to the new API.

The LoopVectorizer now finds instructions which will remain uniform after vectorization. It uses this information when calculating the cost of these instructions.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166836 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-26 23:49:28 +00:00
Hal Finkel
ecc69a1d99 Move target-specific BBVectorize tests into a separate directory.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166802 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-26 19:38:09 +00:00
Nadav Rotem
f563801144 Move the target-specific tests, which require specific backends, to dirs that only run if the target is present.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166796 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-26 18:52:01 +00:00
Rafael Espindola
e5551ed9ce Change the internalize pass to internalize all symbols when given an empty
list of externals. This makes sense since a shared library with no symbols
can still be useful if it has static constructors.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166795 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-26 18:47:48 +00:00
Benjamin Kramer
b2b2273ef4 Fix SCEV cache invalidation in LCSSA and LoopSimplify.
The LoopSimplify bug is pretty harmless because the loop goes from unanalyzable
to analyzable but the LCSSA bug is very nasty. It only comes into play with a
specific order of the LoopPassManager worklist and can cause actual
miscompilations, when a SCEV refers to a value that has been replaced with PHI
node. SCEVExpander may then insert code into the wrong place, either violating
domination or randomly miscompiling stuff.

Comes with an extensive test case reduced from the test-suite with
bugpoint+SCEVValidator.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166787 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-26 17:31:43 +00:00
Nadav Rotem
12145f0339 Fix a crash in SimpliftDemandedBits of vectors of pointers.
PR14183.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166785 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-26 17:17:05 +00:00
Rafael Espindola
cadc38a4b1 Port testcase to FileCheck.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166742 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-26 00:14:11 +00:00
Hal Finkel
822ab00847 Disable generation of pointer vectors by BBVectorize.
Once vector-of-pointer support works, then this can be reverted.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166741 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-26 00:05:26 +00:00
Nadav Rotem
0636291137 Revert 166726 because it may have broken a number of SPEC tests. PR14183.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166739 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25 23:51:48 +00:00
Nadav Rotem
4f3c676102 Fix a crash in ValueTracking. Add support for vectors of pointers.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166726 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25 21:52:52 +00:00
Nadav Rotem
b6cbb6215c Fix the cost-model test.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166722 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25 21:42:50 +00:00
Hal Finkel
32ff623c8b Add CPU model to BBVectorize cost-model tests.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166720 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25 21:31:51 +00:00
Nadav Rotem
402a4d616f Add the cpu model to the test.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166718 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25 21:18:42 +00:00
Hal Finkel
65309660fa Begin incorporating target information into BBVectorize.
This is the first of several steps to incorporate information from the new
TargetTransformInfo infrastructure into BBVectorize. Two things are done here:

 1. Target information is used to determine if it is profitable to fuse two
    instructions. This means that the cost of the vector operation must not
    be more expensive than the cost of the two original operations. Pairs that
    are not profitable are no longer considered (because current cost information
    is incomplete, for intrinsics for example, equal-cost pairs are still
    considered).

 2. The 'cost savings' computed for the profitability check are also used to
    rank the DAGs that represent the potential vectorization plans. Specifically,
    for nodes of non-trivial depth, the cost savings is used as the node
    weight.

The next step will be to incorporate the shuffle costs into the DAG weighting;
this will give the edges of the DAG weights as well. Once that is done, when
target information is available, we should be able to dispense with the
depth heuristic.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166716 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25 21:12:23 +00:00
Jakob Stoklund Olesen
e5a7a68dfa Also optimize large switch statements.
The isValueEqualityComparison() guard at the top of SimplifySwitch()
only applies to some of the possible transformations.

The newer transformations work just fine on large switches, and the
check on predecessor count is nonsensical.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166710 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25 18:51:15 +00:00
Chandler Carruth
a2b88163af Teach SROA how to split whole-alloca integer loads and stores into
smaller integer loads and stores.

The high-level motivation is that the frontend sometimes generates
a single whole-alloca integer load or store during ABI lowering of
splittable allocas. We need to be able to break this apart in order to
see the underlying elements and properly promote them to SSA values. The
hope is that this fixes some performance regressions on x86-32 with the
new SROA pass.

Unfortunately, this causes quite a bit of churn in the test cases, and
bloats some IR that comes out. When we see an alloca that consists soley
of bits and bytes being extracted and re-inserted, we now do some
splitting first, before building widened integer "bucket of bits"
representations. These are always well folded by instcombine however, so
this shouldn't actually result in missed opportunities.

If this splitting of all-integer allocas does cause problems (perhaps
due to smaller SSA values going into the RA), we could potentially go to
some extreme measures to only do this integer splitting trick when there
are non-integer component accesses of an alloca, but discovering this is
quite expensive: it adds yet another complete walk of the recursive use
tree of the alloca.

Either way, I will be watching build bots and LNT bots to see what
fallout there is here. If anyone gets x86-32 numbers before & after this
change, I would be very interested.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166662 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25 04:37:07 +00:00
Nadav Rotem
8dbac7b529 Add support for additional reduction variables: AND, OR, XOR.
Patch by Paul Redmond <paul.redmond@intel.com>.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166649 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25 00:08:41 +00:00
Nadav Rotem
2652c50f74 Implement a basic cost model for vector and scalar instructions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166642 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24 23:47:38 +00:00
Hal Finkel
aacb68806f Update GVN to support vectors of pointers.
GVN will now generate ptrtoint instructions for vectors of pointers.
Fixes PR14166.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166624 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24 21:22:30 +00:00
Nadav Rotem
50bec6f8c4 LoopVectorizer: Add a basic cost model which uses the VTTI interface.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166620 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24 20:36:32 +00:00
Hal Finkel
8c65549318 getSmallConstantTripMultiple should never return zero.
When the trip count is -1, getSmallConstantTripMultiple could return zero,
and this would cause runtime loop unrolling to assert. Instead of returning
zero, one is now returned (consistent with the existing overflow cases).
Fixes PR14167.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166612 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24 19:46:44 +00:00
Micah Villmow
aa76e9e2cf Add in support for getIntPtrType to get the pointer type based on the address space.
This checkin also adds in some tests that utilize these paths and updates some of the
clients.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166578 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24 15:52:52 +00:00
Duncan Sands
747fcd58bc Add a testcase that would have noticed the typo fixed in commit 166475.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166547 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24 07:17:20 +00:00
Nadav Rotem
6457001f31 Use the AliasAnalysis isIdentifiedObj because it also understands mallocs and c++ news.
PR14158.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166491 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-23 18:44:18 +00:00
Bill Wendling
b846719663 Ignore unreachable blocks when doing memory dependence analysis on non-local
loads. It's not really profitable and may result in GVN going into an infinite
loop when it hits constructs like this:

     %x = gep %some.type %x, ...

Found via an LTO build of LLVM.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166490 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-23 18:37:11 +00:00
Duncan Sands
bbc7016c60 Transform code like this
%V = mul i64 %N, 4
 %t = getelementptr i8* bitcast (i32* %arr to i8*), i32 %V
into
 %t1 = getelementptr i32* %arr, i32 %N
 %t = bitcast i32* %t1 to i8*
incorporating the multiplication into the getelementptr.
This happens all the time in dragonegg, for example for
  int foo(int *A, int N) {
    return A[N];
  }
because gcc turns this into byte pointer arithmetic before it hits the plugin:
  D.1590_2 = (long unsigned int) N_1(D);
  D.1591_3 = D.1590_2 * 4;
  D.1592_5 = A_4(D) + D.1591_3;
  D.1589_6 = *D.1592_5;
  return D.1589_6;
The D.1592_5 line is a POINTER_PLUS_EXPR, which is turned into a getelementptr
on a bitcast of A_4 to i8*, so this becomes exactly the kind of IR that the
transform fires on.

An analogous transform (with no testcases!) already existed for bitcasts of
arrays, so I rewrote it to share code with this one.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166474 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-23 08:28:26 +00:00
Nadav Rotem
782090aa02 Don't crash if the load/store pointer is not a GEP.
Fix by Shivarama Rao <Shivarama.Rao@amd.com>



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166427 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22 18:27:56 +00:00
Argyrios Kyrtzidis
0b06e2331e Revert r166407 because it caused analyzer tests to crash and broke self-host bots.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166424 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22 18:16:14 +00:00
Hal Finkel
e29c19091c BBVectorize should ignore unreachable blocks.
Unreachable blocks can have invalid instructions. For example,
jump threading can produce self-referential instructions in
unreachable blocks. Also, we should not be spending time
optimizing unreachable code. Fixes PR14133.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166423 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22 18:00:55 +00:00
Nadav Rotem
565048e78a Vectorizer: optimize the generation of selects. If the condition is uniform, generate a scalar-cond select (i1 as selector).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166409 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22 04:38:00 +00:00
Nick Lewycky
18b1f4e769 Reapply r166405, teaching tailcallelim to be smarter about nocapture, with a
very small but very important bugfix:
  bool shouldExplore(Use *U) {
    Value *V = U->get();
    if (isa<CallInst>(V) || isa<InvokeInst>(V))
    [...]
should have read:
  bool shouldExplore(Use *U) {
    Value *V = U->getUser();
    if (isa<CallInst>(V) || isa<InvokeInst>(V))
Fixes PR14143!



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166407 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22 03:03:52 +00:00
NAKAMURA Takumi
d581b9e61f Revert r166405, "Teach TailRecursionElimination to consider 'nocapture' when deciding whether"
It broke selfhosting stage2 in several builders.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166406 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22 00:48:51 +00:00
Nick Lewycky
241d1398e0 Teach TailRecursionElimination to consider 'nocapture' when deciding whether
calls can be marked tail.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166405 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21 23:51:22 +00:00
Hal Finkel
3d39fb8a3f DataLayout should use itself when calculating the size of a vector.
This is important for vectors of pointers because only DataLayout,
not the underlying vector type, knows how to calculate the size
of the pointers in the vector. Fixes PR14138.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166401 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21 20:38:03 +00:00
Benjamin Kramer
3740e798bc Revert r166390 "LoopIdiom: Replace custom dependence analysis with LoopDependenceAnalysis."
It passes all tests, produces better results than the old code but uses the
wrong pass, LoopDependenceAnalysis, which is old and unmaintained. "Why is it
still in tree?", you might ask. The answer is obviously: "To confuse developers."

Just swapping in the new dependency pass sends the pass manager into an infinte
loop, I'll try to figure out why tomorrow.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166399 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21 19:31:16 +00:00
Benjamin Kramer
5c6e9ae14e LoopIdiom: Replace custom dependence analysis with LoopDependenceAnalysis.
Requires a lot less code and complexity on loop-idiom's side and the more
precise analysis can catch more cases, like the one I included as a test case.
This also fixes the edge-case miscompilation from PR9481. I'm not entirely
sure that all cases are handled that the old checks handled but LDA will
certainly become smarter in the future.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166390 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21 15:03:07 +00:00
Nadav Rotem
bb950854ac Fix a bug in the vectorization of wide load/store operations.
We used a SCEV to detect that A[X] is consecutive. We assumed that X was
the induction variable. But X can be any expression that uses the induction
for example: X = i + 2;



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166388 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21 06:49:10 +00:00
Nadav Rotem
c847872629 Add support for reduction variables that do not start at zero.
This is important for nested-loop reductions such as :

In the innermost loop, the induction variable does not start with zero:

for (i = 0 .. n)
 for (j = 0 .. m)
  sum += ...



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166387 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21 05:52:51 +00:00
Nadav Rotem
5a418ba5f5 Vectorizer: fix a bug in the classification of induction/reduction phis.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166384 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21 02:38:01 +00:00
Nadav Rotem
ccaccfa8bf Fix an infinite loop in the loop-vectorizer.
PR14134.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166379 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-20 20:45:01 +00:00
Benjamin Kramer
82a1833865 InstCombine: Fix an edge case where constant icmps could sneak into ConstantFoldInstOperands and crash.
Have to refactor the ConstantFolder interface one day to define bugs like this away. Fixes PR14131.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166374 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-20 08:43:52 +00:00
Nadav Rotem
bf8772ed2c Vectorize: teach cavVectorizeMemory to distinguish between A[i]+=x and A[B[i]]+=x.
If the pointer is consecutive then it is safe to read and write. If the pointer is non-loop-consecutive then
it is unsafe to vectorize it because we may hit an ordering issue.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166371 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-20 08:26:33 +00:00
Nadav Rotem
5dbe64e2bc Vectorizer: Add support for loop reductions.
For example:

  for (i=0; i<n; i++)
   sum += A[i] +  B[i] + i;



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166351 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-19 23:05:40 +00:00
Benjamin Kramer
0aae4bd0fc SimplifyLibcalls: The return value of ffsll is always i32, even when the input is zero.
Fixes PR13028.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166313 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-19 20:43:44 +00:00
Benjamin Kramer
7182126b0f Indvars: Don't recursively delete instruction during BB iteration.
This can invalidate the iterators leading to use after frees and crashes.
Fixes PR12536.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166291 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-19 17:53:54 +00:00
Benjamin Kramer
239fd44f7a SCEVExpander: Don't crash when trying to merge two constant phis.
Just constant fold them so they can't cause any trouble. Fixes PR12627.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166286 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-19 16:37:30 +00:00
Nadav Rotem
89e7b356f2 vectorizer: Add support for reading and writing from the same memory location.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166255 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-19 01:24:18 +00:00
Meador Inge
0c41d57b09 instcombine: Migrate strcpy optimizations
This patch migrates the strcpy optimizations from the simplify-libcalls pass
into the instcombine library call simplifier.  Note also that StrCpyChkOpt
has been updated with a few simplifications that were being done in the
simplify-libcalls version of StrCpyOpt, but not in the migrated implementation
of StrCpyOpt.  There is no reason to overload StrCpyOpt with fortified and
regular simplifications in the new model since there is already a dedicated
simplifier for __strcpy_chk.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166198 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-18 18:12:40 +00:00
Nadav Rotem
1953ace81d Vectorizer: Add support for loops with an unknown count. For example:
for (i=0; i<n; i++){
        a[i] = b[i+1] + c[i+3];
     }



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166165 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-18 05:29:12 +00:00
Nadav Rotem
d15c0c7ac1 Add a loop vectorizer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166112 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-17 18:25:06 +00:00
Chandler Carruth
02bf98ab38 This just in, it is a *bad idea* to use 'udiv' on an offset of
a pointer. A very bad idea. Let's not do that. Fixes PR14105.

Note that this wasn't *that* glaring of an oversight. Originally, these
routines were only called on offsets within an alloca, which are
intrinsically positive. But over the evolution of the pass, they ended
up being called for arbitrary offsets, and things went downhill...

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166095 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-17 09:23:48 +00:00
Michael Gottesman
4932bbe20c [InstCombine] Teach InstCombine how to handle an obfuscated splat.
An obfuscated splat is where the frontend poorly generates code for a splat
using several different shuffles to create the splat, i.e.,

  %A = load <4 x float>* %in_ptr, align 16
  %B = shufflevector <4 x float> %A, <4 x float> undef, <4 x i32> <i32 0, i32 0, i32 undef, i32 undef>
  %C = shufflevector <4 x float> %B, <4 x float> %A, <4 x i32> <i32 0, i32 1, i32 4, i32 undef>
  %D = shufflevector <4 x float> %C, <4 x float> %A, <4 x i32> <i32 0, i32 1, i32 2, i32 4>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166061 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-16 21:29:38 +00:00
Chandler Carruth
d2cd73f6a5 Update the memcpy rewriting to fully support widened int rewriting. This
includes extracting ints for copying elsewhere and inserting ints when
copying into the alloca. This should fix the CanSROA assertion coming
out of Clang's regression test suite.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165931 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-15 10:24:43 +00:00
Chandler Carruth
94fc64c42f Follow-up fix to r165928: handle memset rewriting for widened integers,
and generally clean up the memset handling. It had rotted a bit as the
other rewriting logic got polished more.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165930 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-15 10:24:40 +00:00
Chandler Carruth
81ff90db44 First major step toward addressing PR14059. This teaches SROA to handle
cases where we have partial integer loads and stores to an otherwise
promotable alloca to widen[1] those loads and stores to cover the entire
alloca and bitcast them into the appropriate type such that promotion
can proceed.

These partial loads and stores stem from an annoying confluence of ARM's
calling convention and ABI lowering and the FCA pre-splitting which
takes place in SROA. Clang lowers a { double, double } in-register
function argument as a [4 x i32] function argument to ensure it is
placed into integer 32-bit registers (a really unnerving implicit
contract between Clang and the ARM backend I would add). This results in
a FCA load of [4 x i32]* from the { double, double } alloca, and SROA
decomposes this into a sequence of i32 loads and stores. Inlining
proceeds, code gets folded, but at the end of the day, we still have i32
stores to the low and high halves of a double alloca. Widening these to
be i64 operations, and bitcasting them to double prior to loading or
storing allows promotion to proceed for these allocas.

I looked quite a bit changing the IR which Clang produces for this case
to be more friendly, but small changes seem unlikely to help. I think
the best representation we could use currently would be to pass 4 i32
arguments thereby avoiding any FCAs, but that would still require this
fix. It seems like it might eventually be nice to somehow encode the ABI
register selection choices outside of the parameter type system so that
the parameter can be a { double, double }, but the CC register
annotations indicate that this should be passed via 4 integer registers.

This patch does not address the second problem in PR14059, which is the
reverse: when a struct alloca is loaded as a *larger* single integer.

This patch also does not address some of the code quality issues with
the FCA-splitting. Those don't actually impede any optimizations really,
but they're on my list to clean up.

[1]: Pedantic footnote: for those concerned about memory model issues
here, this is safe. For the alloca to be promotable, it cannot escape or
have any use of its address that could allow these loads or stores to be
racing. Thus, widening is always safe.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165928 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-15 08:40:30 +00:00
Meador Inge
a239c2e6a7 instcombine: Migrate strcmp and strncmp optimizations
This patch migrates the strcmp and strncmp optimizations from the
simplify-libcalls pass into the instcombine library call simplifier.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165915 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-15 03:47:37 +00:00
Meador Inge
186f8d90df instcombine: Migrate strchr and strrchr optimizations
This patch migrates the strchr and strrchr optimizations from the
simplify-libcalls pass into the instcombine library call simplifier.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165875 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-13 16:45:37 +00:00
Meador Inge
73d8a5864f instcombine: Migrate strcat and strncat optimizations
This patch migrates the strcat and strncat optimizations from the
simplify-libcalls pass into the instcombine library call simplifier.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165874 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-13 16:45:32 +00:00
Chandler Carruth
07525a6be6 Teach SROA to cope with wrapper aggregates. These show up a lot in ABI
type coercion code, especially when targetting ARM. Things like [1
x i32] instead of i32 are very common there.

The goal of this logic is to ensure that when we are picking an alloca
type, we look through such wrapper aggregates and across any zero-length
aggregate elements to find the simplest type possible to form a type
partition.

This logic should (generally speaking) rarely fire. It only ends up
kicking in when an alloca is accessed using two different types (for
instance, i32 and float), and the underlying alloca type has wrapper
aggregates around it. I noticed a significant amount of this occurring
looking at stepanov_abstraction generated code for arm, and suspect it
happens elsewhere as well.

Note that this doesn't yet address truly heinous IR productions such as
PR14059 is concerning. Those result in mismatched *sizes* of types in
addition to mismatched access and alloca types.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165870 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-13 10:49:33 +00:00
Nick Lewycky
5e01f80bf8 Don't crash when !tbaa.struct contents is invalid.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165693 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-11 02:05:23 +00:00
Duncan Sands
ea46827d6c Add the testcase from pr13254 (the old scalarreply pass handles this wrong;
the new sroa pass handles it right).


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165644 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-10 18:41:19 +00:00
Michael Ilseman
440ae6800e New EarlyCSE tests for CSE-ing across commutativity.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165510 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-09 16:58:13 +00:00
Alexey Samsonov
e97a3a4b4f Fix PR14016.
DeadArgumentElimination pass can replace one LLVM function with another,
invalidating a pointer stored in debug info metadata entry for this function.
To fix this, we collect debug info descriptors for functions before
running a DeadArgumentElimination pass and "patch" pointers in metadata nodes
if we replace a function.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165490 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-09 08:13:15 +00:00
Chandler Carruth
2fdb25b5a9 Fix PR14034, an infloop / heap corruption / crash bug in the new SROA.
Thanks to Benjamin for the raw test case. This one took about 50 times
longer to reduce than to fix. =/

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165476 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-09 01:58:35 +00:00
Micah Villmow
791cfc211a Move TargetData to DataLayout.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165403 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-08 16:39:34 +00:00
Chandler Carruth
fca3f4021a Teach the new SROA a new trick. Now we zap any memcpy or memmoves which
are in fact identity operations. We detect these and kill their
partitions so that even splitting is unaffected by them. This is
particularly important because Clang relies on emitting identity memcpy
operations for struct copies, and these fold away to constants very
often after inlining.

Fixes the last big performance FIXME I have on my plate.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165285 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-05 01:29:09 +00:00
Benjamin Kramer
1e21db6e83 SimplifyCFG: Enhance the "remove CFG edge that leads to null pointer dereference" optimization to also handle instructions with multiple uses.
We conservatively only check the first use to avoid walking long use chains.
This catches the common case of having both a load and a store to a pointer
supplied by a PHI node.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165232 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-04 16:11:49 +00:00
Duncan Sands
7508f946bc In my recent change to avoid use of underaligned memory I didn't notice that
cpyDest can be mutated in some cases, which would then cause a crash later if
indeed the memory was underaligned.  This brought down several buildbots, so
I guess the underaligned case is much more common than I thought!


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165228 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-04 13:53:21 +00:00
Duncan Sands
ffcf6dffee The alignment of an sret parameter is known: it must be at least the
alignment of the return type.  Teach the optimizers this.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165226 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-04 13:36:31 +00:00
Chandler Carruth
b2d98c2917 Fix PR13969, a mini-phase-ordering issue with the new SROA pass.
Currently, we re-visit allocas when something changes about the way they
might be *split* to allow better scalarization to take place. However,
we weren't handling the case when the *promotion* is what would change
the behavior of SROA. When an address derived from an alloca is stored
into another alloca, we consider the first to have escaped. If the
second is ever promoted to an SSA value, we will suddenly be able to run
the SROA pass on the first alloca.

This patch adds explicit support for this form if iteration. When we
detect a store of a pointer derived from an alloca, we flag the
underlying alloca for reprocessing after promotion. The logic works hard
to only do this when there is definitely going to be promotion and it
might remove impediments to the analysis of the alloca.

Thanks to Nick for the great test case and Benjamin for some sanity
check review.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165223 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-04 12:33:50 +00:00
Duncan Sands
f58747517c The memcpy optimizer was happily doing call slot forwarding when the new memory
was less aligned than the old.  In the testcase this results in an overaligned
memset: the memset alignment was correct for the original memory but is too much
for the new memory.  Fix this by either increasing the alignment of the new
memory or bailing out if that isn't possible.  Should fix the gcc-4.7 self-host
buildbot failure.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165220 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-04 10:54:40 +00:00
Chandler Carruth
aa3cb334af Teach the integer-promotion rewrite strategy to be endianness aware.
Sorry for this being broken so long. =/

As part of this, switch all of the existing tests to be Little Endian,
which is the behavior I was asserting in them anyways! Add in a new
big-endian test that checks the interesting behavior there.

Another part of this is to tighten the rules abotu when we perform the
full-integer promotion. This logic now rejects cases where there fully
promoted integer is a non-multiple-of-8 bitwidth or cases where the
loads or stores touch bits which are in the allocated space of the
alloca but are not loaded or stored when accessing the integer. Sadly,
these aren't really observable today as the rest of the pass will
already ensure the invariants hold. However, the latter situation is
likely to become a potential concern in the future.

Thanks to Benjamin and Duncan for early review of this patch. I'm still
looking into whether there are further endianness issues, please let me
know if anyone sees BE failures persisting past this.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165219 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-04 10:39:28 +00:00
Jakub Staszak
395c1502a7 Fix PR13967.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165187 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-03 23:59:47 +00:00
Chandler Carruth
322e9ba2cb Fix an issue where we failed to adjust the alignment constraint on
a memcpy to reflect that '0' has a different meaning when applied to
a load or store. Now we correctly use underaligned loads and stores for
the test case added.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165101 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-03 08:26:28 +00:00
Chandler Carruth
f710fb14ee Try to use a better set of abstractions for computing the alignment
necessary during rewriting. As part of this, fix a real think-o here
where we might have left off an alignment specification when the address
is in fact underaligned. I haven't come up with any way to trigger this,
as there is always some other factor that reduces the alignment, but it
certainly might have been an observable bug in some way I can't think
of. This also slightly changes the strategy for placing explicit
alignments on loads and stores to only do so when the alignment does not
match that required by the ABI. This causes a few redundant alignments
to go away from test cases.

I've also added a couple of tests that really push on the alignment that
we end up with on loads and stores. More to come here as I try to fix an
underlying bug I have conjectured and produced test cases for, although
it's not clear if this bug is the one currently hitting dragonegg's
gcc47 bootstrap.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165100 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-03 08:14:02 +00:00
Chandler Carruth
5c5b3cf5b8 Teach the new SROA to handle cases where an alloca that has already been
scheduled for processing on the worklist eventually gets deleted while
we are processing another alloca, fixing the original test case in
PR13990.

To facilitate this, add a remove_if helper to the SetVector abstraction.
It's not easy to use the standard abstractions for this because of the
specifics of SetVectors types and implementation.

Finally, a nice small test case is included. Thanks to Benjamin for the
fantastic reduced test case here! All I had to do was delete some empty
basic blocks!

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165065 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-02 22:46:45 +00:00
Benjamin Kramer
fba80d9e97 Fix broken tests.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165019 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-02 15:49:34 +00:00
Chandler Carruth
cfb65293c3 Fix more misspellings found by Duncan during review.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164940 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-01 12:30:45 +00:00
Chandler Carruth
673850aa2d Fix several issues with alignment. We weren't always accounting for type
alignment requirements of the new alloca. As one consequence which was
reported as a bug by Duncan, we overaligned memcpy calls to ranges of
allocas after they were rewritten to types with lower alignment
requirements. Other consquences are possible, but I don't have any test
cases for them.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164937 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-01 12:16:54 +00:00
Benjamin Kramer
64f27e78c4 SimplifyCFG: Don't crash when forming a switch bitmap with an undef default value.
Fixes PR13985.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164934 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-01 11:31:48 +00:00
Chandler Carruth
77c1270033 Refactor the PartitionUse structure to actually use the Use* instead of
a pair of instructions, one for the used pointer and the second for the
user. This simplifies the representation and also makes it more dense.

This was noticed because of the miscompile in PR13926. In that case, we
were running up against a fundamental "bad idea" in the speculation of
PHI and select instructions: the speculation and rewriting are
interleaved, which requires phi speculation to also perform load
rewriting! This is bad, and causes us to miss opportunities to do (for
example) vector rewriting only exposed after PHI speculation, etc etc.
It also, in the old system, required us to insert *new* load uses into
the current partition's use list, which would then be ignored during
rewriting because we had already extracted an end iterator for the use
list. The appending behavior (and much of the other oddities) stem from
the strange de-duplication strategy in the PartitionUse builder.
Amusingly, all this went without notice for so long because it could
only be triggered by having *different* GEPs into the same partition of
the same alloca, where both different GEPs were operands of a single
PHI, and where the GEP which was not encountered first also had multiple
uses within that same PHI node... Hence the insane steps required to
reproduce.

So, step one in fixing this fundamental bad idea is to make the
PartitionUse actually contain a Use*, and to make the builder do proper
deduplication instead of funky de-duplication. This is enough to remove
the appending behavior, and fix the miscompile in PR13926, but there is
more work to be done here. Subsequent commits will lift the speculation
into its own visitor. It'll be a useful step toward potentially
extracting all of the speculation logic into a generic utility
transform.

The existing PHI test case for repeated operands has been made more
extreme to catch even these issues. This test case, run through the old
pass, will exactly reproduce the miscompile from PR13926. ;] We were so
close here!

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164925 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-01 01:49:22 +00:00
Chandler Carruth
b67c9a5b02 Fix a somewhat surprising miscompile where code relying on an ABI
alignment could lose it due to the alloca type moving down to a much
smaller alignment guarantee.

Now SROA will actively compute a proper alignment, factoring the target
data, any explicit alignment, and the offset within the struct. This
will in some cases lower the alignment requirements, but when we lower
them below those of the type, we drop the alignment entirely to give
freedom to the code generator to align it however is convenient.

Thanks to Duncan for the lovely test case that pinned this down. =]

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164891 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-29 10:41:21 +00:00
Evan Cheng
aa7e47416a Add test case for r164850.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164867 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-29 00:12:08 +00:00
Benjamin Kramer
8be16fe703 CorrelatedPropagation: BasicBlock::removePredecessor can simplify PHI nodes. If the it's the condition of a SwitchInst, reload it.
Fixes PR13972.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164818 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-28 10:42:50 +00:00
Benjamin Kramer
ab16423c91 GlobalOpt: non-constexpr bitcasts or GEPs can occur even if the global value is only stored once.
Fixes PR13968.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164815 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-28 10:01:27 +00:00
Nick Lewycky
7e0e166d17 Surprisingly, we missed a trivial case here. Fix that!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164814 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-28 09:33:53 +00:00
Meador Inge
ce71822d38 instcombine: Add more test cases for __strncpy_chk simplification
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164800 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-27 21:21:31 +00:00
Meador Inge
a968582978 instcombine: Add more test cases for __strcpy_chk simplification
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164799 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-27 21:21:28 +00:00
Meador Inge
c8613c9089 instcombine: Add more test cases for __memmove_chk simplification
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164798 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-27 21:21:25 +00:00
Meador Inge
012ffd251b instcombine: Add more test cases for __memcpy_chk simplification
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164797 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-27 21:21:21 +00:00
Meador Inge
910c564d3b instcombine: Add more test cases for __memset_chk simplification
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164796 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-27 21:21:18 +00:00
Benjamin Kramer
465251a7ce Fix a integer overflow in SimplifyCFG's look up table formation logic.
If the width is very large it gets truncated from uint64_t to uint32_t when
passed to TD->fitsInLegalInteger. The truncated value can fit in a register.
This manifested in massive memory usage or crashes (PR13946).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164784 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-27 18:29:58 +00:00
Sylvestre Ledru
94c22716d6 Revert 'Fix a typo 'iff' => 'if''. iff is an abreviation of if and only if. See: http://en.wikipedia.org/wiki/If_and_only_if Commit 164767
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164768 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-27 10:14:43 +00:00
Sylvestre Ledru
7e2c793a2b Fix a typo 'iff' => 'if'
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164767 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-27 09:59:43 +00:00
Nick Lewycky
466e0f38d3 Prefer shuffles to selects. Backends love shuffles!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164763 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-27 08:33:56 +00:00
Hans Wennborg
50b7d70707 Address Duncan's comments on r164684:
- Put statistics in alphabetical order
- Don't use getZextValue when building TableInt, just use APInts
- Introduce Create{Z,S}ExtOrTrunc in IRBuilder.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164696 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-26 14:01:53 +00:00
Chandler Carruth
238fd1591e When rewriting the pointer operand to a load or store which has
alignment guarantees attached, re-compute the alignment so that we
consider offsets which impact alignment.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164690 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-26 10:45:28 +00:00
Chandler Carruth
81b001a220 Teach all of the loads, stores, memsets and memcpys created by the
rewriter in SROA to carry a proper alignment. This involves
interrogating various sources of alignment, etc. This is a more complete
and principled fix to PR13920 as well as related bugs pointed out by Eli
in review and by inspection in the area.

Also by inspection fix the integer and vector promotion paths to create
aligned loads and stores. I still need to work up test cases for
these... Sorry for the delay, they were found purely by inspection.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164689 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-26 10:27:46 +00:00
Benjamin Kramer
e7381cc99c Fix tests that didn't test anything.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164686 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-26 09:51:39 +00:00
Hans Wennborg
d72271cd84 SimplifyCFG: Make the switch-to-lookup table transformation store the
tables in bitmaps when they fit in a target-legal register.

This saves some space, and it also allows for building tables that would
otherwise be deemed too sparse.

One interesting case that this hits is example 7 from
http://blog.regehr.org/archives/320. We currently generate good code
for this when lowering the switch to the selection DAG: we build a
bitmask to decide whether to jump to one block or the other. My patch
will result in the same bitmask, but it removes the need for the jump,
as the return value can just be retrieved from the mask.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164684 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-26 09:44:49 +00:00
Chandler Carruth
b3dca3f50e Revert the business end of r164636 and try again. I'll come in again. ;]
This should really, really fix PR13916. For real this time. The
underlying bug is... a bit more subtle than I had imagined.

The setup is a code pattern that leads to an @llvm.memcpy call with two
equal pointers to an alloca in the source and dest. Now, not any pattern
will do. The alloca needs to be formed just so, and both pointers should
be wrapped in different bitcasts etc. When this precise pattern hits,
a funny sequence of events transpires. First, we correctly detect the
potential for overlap, and correctly optimize the memcpy. The first
time. However, we do simplify the set of users of the alloca, and that
causes us to run the alloca back through the SROA pass in case there are
knock-on simplifications. At this point, a curious thing has happened.
If we happen to have an i8 alloca, we have direct i8 pointer values. So
we don't bother creating a cast, we rewrite the arguments to the memcpy
to dircetly refer to the alloca.

Now, in an unrelated area of the pass, we have clever logic which
ensures that when visiting each User of a particular pointer derived
from an alloca, we only visit that User once, and directly inspect all
of its operands which refer to that particular pointer value. However,
the mechanism used to detect memcpy's with the potential to overlap
relied upon getting visited once per *Use*, not once per *User*. This is
always true *unless* the same exact value is both source and dest. It
turns out that almost nothing actually produces that pattern though.

We can hand craft test cases that more directly test this behavior of
course, and those are included. Also, note that there is a significant
missed optimization here -- we prove in many cases that there is
a non-volatile memcpy call with identical source and dest addresses. We
shouldn't prevent splitting the alloca in that case, and in fact we
should just remove such memcpy calls eagerly. I'll address that in
a subsequent commit.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164669 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-26 07:41:40 +00:00
Nick Lewycky
051a318e67 Don't drop the alignment on a memcpy intrinsic when producing a store. This is
only a missed optimization opportunity if the store is over-aligned, but a
miscompile if the store's new type has a higher natural alignment than the
memcpy did. Fixes PR13920!


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164641 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-25 22:46:21 +00:00
Nick Lewycky
c3f10e43fc Don't try to promote the same alloca twice. Fixes PR13916!
Chandler, it's not obvious that it's okay that this alloca gets into the list
twice to begin with. Please review and see whether this is the fix you really
want, but I wanted to get a fix checked in quickly.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164634 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-25 21:15:50 +00:00
Nick Lewycky
fe4a778f6e Make this test check the transforms it's actually doing. Also add a test that it
doesn't transform the trivially unsafe case.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164617 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-25 18:17:38 +00:00
Chandler Carruth
c303463dcc Fix a case where SROA did not correctly detect dead PHI or selects due
to chains or cycles between PHIs and/or selects. Also add a couple of
really nice test cases reduced from Kostya's reports in PR13905 and
PR13906. Both are fixed by this patch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164596 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-25 10:03:40 +00:00
Nick Lewycky
24ab21c691 Don't forget that strcpy and friends return a pointer to the destination, so
it's not a dead store if that pointer is used. Whoops!


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164583 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-25 01:55:59 +00:00
Nick Lewycky
3dbefbd9bb Teach DSE that strcpy, strncpy, strcat and strncat are all stores which may be
dead.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164561 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-24 22:09:10 +00:00
Richard Osborne
47bd469b8a Add missing : in CHECK line.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164540 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-24 17:22:43 +00:00
Richard Osborne
bdc76fc75a Add missing check for presence of target data.
This avoids a crash in visitAllocaInst when target data isn't available.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164539 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-24 17:10:03 +00:00
Chandler Carruth
bc4021f31e Address one of the original FIXMEs for the new SROA pass by implementing
integer promotion analogous to vector promotion. When there is an
integer alloca being accessed both as its integer type and as a narrower
integer type, promote the narrower access to "insert" and "extract" the
smaller integer from the larger one, and make the integer alloca
a candidate for promotion.

In the new formulation, we don't care about target legal integer or use
thresholds to control things. Instead, we only perform this promotion to
an integer type which the frontend has already emitted a load or store
for. This bounds the scope and prevents optimization passes from
coalescing larger and larger entities into a single integer.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164479 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-24 00:34:20 +00:00
Chandler Carruth
02e92a0b5d Switch to a signed representation for the dynamic offsets while walking
across the uses of the alloca. It's entirely possible for negative
numbers to come up here, and in some rare cases simply doing the 2's
complement arithmetic isn't the correct decision. Notably, we can't zext
the index of the GEP. The definition of GEP is that these offsets are
sign extended or truncated to the size of the pointer, and then wrapping
2's complement arithmetic used.

This patch fixes an issue that comes up with *no* input from the
buildbots or bootstrap afaict. The only place where it manifested,
disturbingly, is Clang's own regression test suite. A reduced and
targeted collection of tests are added to cope with this. Note that I've
tried to pin down the potential cases of overflow, but may have missed
some cases. I've tried to add a few cases to test this, but its hard
because LLVM has quite limited support for >64bit constructs.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164475 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-23 11:43:14 +00:00
Chandler Carruth
d54a6b56a9 Fix a case where the new SROA pass failed to zap dead operands to
selects with a constant condition. This resulted in the operands
remaining live through the SROA rewriter. Most of the time, this just
caused some dead allocas to persist and get zapped by later passes, but
in one case found by Joerg, it caused a crash when we tried to *promote*
the alloca despite it having this dead use. We already have the
mechanisms in place to handle this, just wire select up to them.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164427 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-21 23:36:40 +00:00
Benjamin Kramer
28aff84ceb LoopIdiom: Give up when the loop is not in canonical form.
We rely on it when doing the transforms. This can happen when there is an
indirectbr in  the loop.

Fixes PR13892.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164383 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-21 17:27:23 +00:00
Benjamin Kramer
e5bd3cf000 InstCombine: Make sure we use the pre-zext type when creating a constant of a value that is zext'd.
Fixes PR13250.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164377 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-21 16:26:41 +00:00
Manman Ren
554da1a222 SimplifyCFG: sink common codes from IF, ELSE blocks down to END block.
We already have HoistThenElseCodeToIf, this patch implements
SinkThenElseCodeToEnd. When END block has only two predecessors and each
predecessor terminates with unconditional branches, we compare instructions in
IF and ELSE blocks backwards and check whether we can sink the common
instructions down.

rdar://12191395


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164325 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-20 22:37:36 +00:00
Hans Wennborg
57933e3481 SimplifyCFG: Don't generate invalid code for switch used to initialize
two variables where the first variable is returned and the second
ignored.

I don't think this occurs in practice (other passes should have cleaned
up the unused phi node), but it should still be handled correctly.

Also make the logic for determining if we should return early less
sketchy.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164225 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-19 14:24:21 +00:00
Hans Wennborg
c4ba62ca1d Move load_to_switch.ll to test/CodeGen/SPARC/
Because the test invokes llc -march=sparc, it needs to be in a directory
which is only run when the sparc target is built.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164211 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-19 09:25:03 +00:00
Nadav Rotem
f252441239 rename test
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164210 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-19 09:22:17 +00:00
Nadav Rotem
92df026f0d Prevent inlining of callees which allocate lots of memory into a recursive caller.
Example:

void foo() {
 ... foo();   // I'm recursive!

  bar();
}

bar() {  int a[1000];  // large stack size }

rdar://10853263



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164207 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-19 08:08:04 +00:00
Hans Wennborg
93ba133906 CodeGenPrep: turn lookup tables into switches for some targets.
This is a follow-up from r163302, which added a transformation to
SimplifyCFG that turns some switches into loads from lookup tables.

It was pointed out that some targets, such as GPUs and deeply embedded
targets, might not find this appropriate, but SimplifyCFG doesn't have
enough information about the target to decide this.

This patch adds the reverse transformation to CodeGenPrep: it turns
loads from lookup tables back into switches for targets where we do not
build jump tables (assuming these are also the targets where lookup
tables are inappropriate).

Hopefully we will eventually get to have target information in
SimplifyCFG, and then this CodeGenPrep transformation can be removed.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164206 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-19 07:48:16 +00:00
Chandler Carruth
38f35fd3b7 Fix the last crasher I've gotten a reproduction for in SROA. This one
from the dragonegg build bots when we turned on the full version of the
pass. Included a much reduced test case for this pesky bug, despite
bugpoint's uncooperative behavior.

Also, I audited all the similar code I could find and didn't spot any
other cases where this mistake cropped up.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164178 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-18 22:37:19 +00:00
Andrew Trick
f08c115e6c LSR critical edge splitting fix for PR13756.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164147 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-18 17:51:33 +00:00
Chandler Carruth
7c8df7aa0c Fix getCommonType in a different way from the way I fixed it when
working on FCA splitting. Instead of refusing to form a common type when
there are uses of a subsection of the alloca as well as a use of the
entire alloca, just skip the subsection uses and continue looking for
a whole-alloca use with a type that we can use.

This produces slightly prettier IR I think, and also fixes the other
failure in the test.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164146 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-18 17:49:37 +00:00
Benjamin Kramer
6ab4bd7145 XFAIL SROA test until Chandler can get to it.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164128 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-18 14:27:53 +00:00
Chandler Carruth
9e3f639579 Fix a warning in release builds and a test case I forgot to update with
a fix to getCommonType in the previous patch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164120 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-18 13:02:06 +00:00
Chandler Carruth
c370acdf96 Add a major missing piece to the new SROA pass: aggressive splitting of
FCAs. This is essential in order to promote allocas that are used in
struct returns by frontends like Clang. The FCA load would block the
rest of the pass from firing, resulting is significant regressions with
the bullet benchmark in the nightly test suite.

Thanks to Duncan for repeated discussions about how best to do this, and
to both him and Benjamin for review.

This appears to have blocked many places where the pass tries to fire,
and so I'm expect somewhat different results with this fix added.

As with the last big patch, I'm including a change to enable the SROA by
default *temporarily*. Ben is going to remove this as soon as the LNT
bots pick up the patch. I'm just trying to get a round of LNT numbers
from the stable machines in the lab.

NOTE: Four clang tests are expected to fail in the brief window where
this is enabled. Sorry for the noise!

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164119 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-18 12:57:43 +00:00
Richard Osborne
d7cc8b839c Fix instcombine to obey requested alignment when merging allocas.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164117 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-18 09:31:44 +00:00
Manman Ren
222d6192ad PGO: preserve branch-weight metadata when simplifying Switch to a sub, an icmp
and a conditional branch; also when removing dead cases from a switch.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164084 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-18 00:47:33 +00:00
Manman Ren
b010277b59 PGO: preserve branch-weight metadata when simplifying Switch
Hanlde the case when we split the default edge if the default target has "icmp"
and unconditinal branch.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164076 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-17 23:07:43 +00:00
Manman Ren
b11cbe6b23 PGO: preserve branch-weight metadata when simplifying SwitchOnSelect.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164068 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-17 22:28:55 +00:00
Manman Ren
566540332f PGO: preserve branch-weight metadata when simplifying two branches with a common
destination in SimplifyCondBranchToCondBranch.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164054 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-17 21:30:40 +00:00
Chandler Carruth
1c8db50a9a Port the SSAUpdater-based promotion logic from the old SROA pass to the
new one, and add support for running the new pass in that mode and in
that slot of the pass manager. With this the new pass can completely
replace the old one within the pipeline.

The strategy for enabling or disabling the SSAUpdater logic is to do it
by making the requirement of the domtree analysis optional. By default,
it is required and we get the standard mem2reg approach. This is usually
the desired strategy when run in stand-alone situations. Within the
CGSCC pass manager, we disable requiring of the domtree analysis and
consequentially trigger fallback to the SSAUpdater promotion.

In theory this would allow the pass to re-use a domtree if one happened
to be available even when run in a mode that doesn't require it. In
practice, it lets us have a single pass rather than two which was
simpler for me to wrap my head around.

There is a hidden flag to force the use of the SSAUpdater code path for
the purpose of testing. The primary testing strategy is just to run the
existing tests through that path. One notable difference is that it has
custom code to handle lifetime markers, and one of the tests has been
enhanced to exercise that code.

This has survived a bootstrap and the test suite without serious
correctness issues, however my run of the test suite produced *very*
alarming performance numbers. I don't entirely understand or trust them
though, so more investigation is on-going.

To aid my understanding of the performance impact of the new SROA now
that it runs throughout the optimization pipeline, I'm enabling it by
default in this commit, and will disable it again once the LNT bots have
picked up one iteration with it. I want to get those bots (which are
much more stable) to evaluate the impact of the change before I jump to
any conclusions.

NOTE: Several Clang tests will fail because they run -O3 and check the
result's order of output. They'll go back to passing once I disable it
again.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163965 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-15 11:43:14 +00:00
Manman Ren
062986c2f0 PGO: preserve branch-weight metadata when simplifying two branches with a common
destination.

Updated previous implementation to fix a case not covered:
// PBI: br i1 %x, TrueDest, BB
// BI:  br i1 %y, TrueDest, FalseDest
The other case was handled correctly.
// PBI: br i1 %x, BB, FalseDest
// BI:  br i1 %y, TrueDest, FalseDest

Also tried to use 64-bit arithmetic instead of APInt with scale to simplify the
computation. Let me know if you have other opinions about this.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163954 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-15 00:39:57 +00:00
Manman Ren
ad2890760f PGO: preserve branch-weight metadata when simplifying a switch with a single
case to a conditional branch and when removing dead cases.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163942 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-14 21:53:06 +00:00
Alex Rosenberg
eee94b3432 Review feedback from Duncan Sands. Alphabetize includes and simplify
lit config.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163928 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-14 19:19:57 +00:00
Manman Ren
a8a2b99aec PGO: preserve branch-weight metadata when merging two switches where
the default target of the first switch is not the basic block the second switch
is in (PredDefault != BB).


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163916 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-14 17:29:56 +00:00
Chandler Carruth
713aa9431d Introduce a new SROA implementation.
This is essentially a ground up re-think of the SROA pass in LLVM. It
was initially inspired by a few problems with the existing pass:
- It is subject to the bane of my existence in optimizations: arbitrary
  thresholds.
- It is overly conservative about which constructs can be split and
  promoted.
- The vector value replacement aspect is separated from the splitting
  logic, missing many opportunities where splitting and vector value
  formation can work together.
- The splitting is entirely based around the underlying type of the
  alloca, despite this type often having little to do with the reality
  of how that memory is used. This is especially prevelant with unions
  and base classes where we tail-pack derived members.
- When splitting fails (often due to the thresholds), the vector value
  replacement (again because it is separate) can kick in for
  preposterous cases where we simply should have split the value. This
  results in forming i1024 and i2048 integer "bit vectors" that
  tremendously slow down subsequnet IR optimizations (due to large
  APInts) and impede the backend's lowering.

The new design takes an approach that fundamentally is not susceptible
to many of these problems. It is the result of a discusison between
myself and Duncan Sands over IRC about how to premptively avoid these
types of problems and how to do SROA in a more principled way. Since
then, it has evolved and grown, but this remains an important aspect: it
fixes real world problems with the SROA process today.

First, the transform of SROA actually has little to do with replacement.
It has more to do with splitting. The goal is to take an aggregate
alloca and form a composition of scalar allocas which can replace it and
will be most suitable to the eventual replacement by scalar SSA values.
The actual replacement is performed by mem2reg (and in the future
SSAUpdater).

The splitting is divided into four phases. The first phase is an
analysis of the uses of the alloca. This phase recursively walks uses,
building up a dense datastructure representing the ranges of the
alloca's memory actually used and checking for uses which inhibit any
aspects of the transform such as the escape of a pointer.

Once we have a mapping of the ranges of the alloca used by individual
operations, we compute a partitioning of the used ranges. Some uses are
inherently splittable (such as memcpy and memset), while scalar uses are
not splittable. The goal is to build a partitioning that has the minimum
number of splits while placing each unsplittable use in its own
partition. Overlapping unsplittable uses belong to the same partition.
This is the target split of the aggregate alloca, and it maximizes the
number of scalar accesses which become accesses to their own alloca and
candidates for promotion.

Third, we re-walk the uses of the alloca and assign each specific memory
access to all the partitions touched so that we have dense use-lists for
each partition.

Finally, we build a new, smaller alloca for each partition and rewrite
each use of that partition to use the new alloca. During this phase the
pass will also work very hard to transform uses of an alloca into a form
suitable for promotion, including forming vector operations, speculating
loads throguh PHI nodes and selects, etc.

After splitting is complete, each newly refined alloca that is
a candidate for promotion to a scalar SSA value is run through mem2reg.

There are lots of reasonably detailed comments in the source code about
the design and algorithms, and I'm going to be trying to improve them in
subsequent commits to ensure this is well documented, as the new pass is
in many ways more complex than the old one.

Some of this is still a WIP, but the current state is reasonbly stable.
It has passed bootstrap, the nightly test suite, and Duncan has run it
successfully through the ACATS and DragonEgg test suites. That said, it
remains behind a default-off flag until the last few pieces are in
place, and full testing can be done.

Specific areas I'm looking at next:
- Improved comments and some code cleanup from reviews.
- SSAUpdater and enabling this pass inside the CGSCC pass manager.
- Some datastructure tuning and compile-time measurements.
- More aggressive FCA splitting and vector formation.

Many thanks to Duncan Sands for the thorough final review, as well as
Benjamin Kramer for lots of review during the process of writing this
pass, and Daniel Berlin for reviewing the data structures and algorithms
and general theory of the pass. Also, several other people on IRC, over
lunch tables, etc for lots of feedback and advice.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163883 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-14 09:22:59 +00:00
Dan Gohman
b998913ff4 Handle the new !tbaa.struct metadata tags when converting a memcpy into scalar
loads and stores.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163844 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-13 21:51:01 +00:00
Benjamin Kramer
39acdb0200 MemCpyOpt: When forming a memset from stores also take GEP constexprs into account.
This is common when storing to global variables.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163809 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-13 16:29:49 +00:00
Dan Gohman
0d1bc5f916 Detect overflow in the path count computation. rdar://12277446.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163739 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-12 20:45:17 +00:00
Manman Ren
ee99c7f1bf PGO: preserve branch-weight metadata when removing a case which jumps
to the default target.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163724 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-12 17:04:11 +00:00
Manman Ren
020aba0c3b SimplifyCFG: preserve branch-weight metadata when creating a new switch from
a pair of switch/branch where both depend on the value of the same variable and
the default case of the first switch/branch goes to the second switch/branch.

Code clean up and fixed a few issues:
1> handling the case where some cases of the 2nd switch are invalidated
2> correctly calculate the weight for the 2nd switch when it is a conditional eq

Testing case is modified from Alastair's original patch.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163635 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-11 17:43:35 +00:00
Alex Rosenberg
95f1ef4ac7 Add a pass that renames everything with metasyntatic names. This works well after using bugpoint to reduce the confusion presented by the original names, which no longer mean what they used to.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163592 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-11 02:46:18 +00:00
Andrew Trick
a344341849 Remove an incorrect assert during branch weight propagation.
Patch and test case by Alastair Murray!

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163437 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-08 00:07:26 +00:00
Hans Wennborg
3bd51b8df3 Fix switch_to_lookup_table.ll test from r163302.
The lookup tables did not get built in a deterministic order.
This makes them get built in the order that the corresponding phi nodes
were found.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163305 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-06 10:10:35 +00:00
Hans Wennborg
486270aee6 Build lookup tables for switches (PR884)
This adds a transformation to SimplifyCFG that attemps to turn switch
instructions into loads from lookup tables. It works on switches that
are only used to initialize one or more phi nodes in a common successor
basic block, for example:

  int f(int x) {
    switch (x) {
    case 0: return 5;
    case 1: return 4;
    case 2: return -2;
    case 5: return 7;
    case 6: return 9;
    default: return 42;
  }

This speeds up the code by removing the hard-to-predict jump, and
reduces code size by removing the code for the jump targets.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163302 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-06 09:43:28 +00:00
Manman Ren
408853ea16 JumpThreading: when default destination is the destination of some cases in a
switch, make sure we include the value for the cases when calculating edge
value from switch to the default destination.

rdar://12241132


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163270 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-05 23:45:58 +00:00
Dan Gohman
230768bd13 Make provenance checking conservative in cases when
pointers-to-strong-pointers may be in play. These can lead to retains and
releases happening in unstructured ways, foiling the optimizer. This fixes
rdar://12150909.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163180 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-04 23:16:20 +00:00
Nadav Rotem
7765492a7a LICM may hoist an instruction with undefined behavior above a trap.
Scan the body of the loop and find instructions that may trap.
Use this information when deciding if it is safe to hoist or sink instructions.
Notice that we can optimize the search of instructions that may throw in the case of nested loops.

rdar://11518836



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163132 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-04 10:25:04 +00:00
Bob Wilson
84451a110d Fix more fallout from r158919, similar to PR13547.
This code used to only handle malloc-like calls, which do not read memory.
r158919 changed it to check isNoAliasFn(), which includes strdup-like and
realloc-like calls, but it was not checking for dependencies on the memory
read by those calls.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163106 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-03 05:15:15 +00:00
Benjamin Kramer
7de7078933 LoopRotation: Make the brute force DomTree update more brute force.
We update until we hit a fixpoint. This is probably slow but also
slightly simplifies the code. It should also fix the occasional
invalid domtrees observed when building with expensive checking.

I couldn't find a case where this had a measurable slowdown, but
if someone finds a pathological case where it does we may have
to find a cleverer way of updating dominators here.

Thanks to Duncan for the test case.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163091 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-02 11:57:22 +00:00
Michael Gottesman
cee2f72bfe [llvm] Updated the test fold-vector-select so that we test the vector selects exhaustively.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162953 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-30 23:11:49 +00:00
Benjamin Kramer
86c0d8c92f Fix test case.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162913 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-30 15:42:45 +00:00
Benjamin Kramer
d70846ec1b LoopRotate: Also rotate loops with multiple exits.
The old PHI updating code in loop-rotate was replaced with SSAUpdater a while
ago, it has no problems with comples PHIs. What had to be fixed is detecting
whether a loop was already rotated and updating dominators when multiple exits
were present.

This change increases overall code size a bit, mostly due to additional loop
unrolling opportunities. Passes test-suite and selfhost with -verify-dom-info.
Fixes PR7447.

Thanks to Andy for the input on the domtree updating code.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162912 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-30 15:39:42 +00:00
Nadav Rotem
639570c311 It is illegal to transform (sdiv (ashr X c1) c2) -> (sdiv x (2^c1 * c2)),
because C always rounds towards zero.

Thanks Dirk and Ben.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162899 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-30 11:23:20 +00:00
Benjamin Kramer
8e0d1c03ca Make MemoryBuiltins aware of TargetLibraryInfo.
This disables malloc-specific optimization when -fno-builtin (or -ffreestanding)
is specified. This has been a problem for a long time but became more severe
with the recent memory builtin improvements.

Since the memory builtin functions are used everywhere, this required passing
TLI in many places. This means that functions that now have an optional TLI
argument, like RecursivelyDeleteTriviallyDeadFunctions, won't remove dead
mallocs anymore if the TLI argument is missing. I've updated most passes to do
the right thing.

Fixes PR13694 and probably others.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162841 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-29 15:32:21 +00:00
Benjamin Kramer
aac7c650a6 InstCombine: Guard the transform introduced in r162743 against large ints and non-const shifts.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162751 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-28 13:08:13 +00:00
Nadav Rotem
a694e2a691 Make sure that we don't call getZExtValue on values > 64 bits.
Thanks Benjamin for noticing this.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162749 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-28 12:23:22 +00:00
Nadav Rotem
9753f0b9b4 Teach InstCombine to canonicalize [SU]div+[AL]shl patterns.
For example:
  %1 = lshr i32 %x, 2
  %2 = udiv i32 %1, 100

rdar://12182093




git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162743 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-28 10:01:43 +00:00
Benjamin Kramer
7f07d2fbcf SimplifyLibCalls: Give all safely-shrinkable libcalls the same treatment.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162383 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-22 19:39:15 +00:00
Chad Rosier
5f676fe139 Whitespace.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162370 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-22 17:34:11 +00:00
Chad Rosier
69b8e6295b Add test case for r162368.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162369 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-22 17:31:04 +00:00
Chandler Carruth
ec68f552f2 Port the global copy optimization from the SROA pass to InstCombine.
This optimization is really just replacing allocas wholesale with
globals, there is no scalarization.

The underlying motivation for this patch is to simplify the SROA pass
and focus it on splitting and promoting allocas.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162271 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-21 08:39:44 +00:00
Benjamin Kramer
a4b5717ea7 InstCombine: Fix a crasher when encountering a function pointer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162180 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-18 22:04:34 +00:00
Benjamin Kramer
00abcd3da0 InstCombine: Add a couple of fabs identities for comparing with 0.0.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162174 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-18 20:06:47 +00:00
Benjamin Kramer
2b74957106 SimplifyLibcalls: Add fabs and trunc to the list of libcalls that are safe to shrink from double to float.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162173 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-18 19:27:32 +00:00
Benjamin Kramer
168843c013 MemoryBuiltins: Properly guard ObjectSizeOffsetVisitor against cycles in the IR.
The previous fix only checked for simple cycles, use a set to catch longer
cycles too.

Drop the broken check from the ObjectSizeOffsetEvaluator. The BoundsChecking
pass doesn't have to deal with invalid IR like InstCombine does.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162120 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-17 19:26:41 +00:00
Benjamin Kramer
823573a381 Guard MemoryBuiltins against self-looping GEPs, which can occur in unreachable code due to constant propagation.
Fixes PR13621.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162098 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-17 14:16:37 +00:00
Benjamin Kramer
4e81d40545 Fix broken check lines.
I really need to find a way to automate this, but I can't come up with a regex
that has no false positives while handling tricky cases like custom check
prefixes.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162097 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-17 12:28:26 +00:00
Rafael Espindola
0513059726 Teach GVN to reason about edges dominating uses. This allows it to handle cases
where some fact lake a=b dominates a use in a phi, but doesn't dominate the
basic block itself.

This feature could also be implemented by splitting critical edges, but at least
with the current algorithm reasoning about the dominance directly is faster.

The time for running "opt -O2" in the testcase in pr10584 is 1.003 times slower
and on gcc as a single file it is 1.0007 times faster.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162023 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-16 15:09:43 +00:00
Michael Liao
9441ad0b6e fix infinite loop in instcombine with more than 4GB memcpy
- memcpy size is wrongly truncated into 32-bit and treat 8GB memcpy is
  0-sized memcpy
- as 0-sized memcpy/memset is already removed before SimplifyMemTransfer
  and SimplifyMemSet in visitCallInst, replace 0 checking with
  assertions.
- replace getZExtValue() with getLimitedValue() according to
  Eli Friedman



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@161923 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-15 03:49:59 +00:00
Craig Topper
750d7616c6 Change greater than to greater than or equal so that an identical sized store to the same offset is treated as completing overwriting.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@161857 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-14 07:32:05 +00:00
Nadav Rotem
3e883734fa During the CodeGenPrepare we often lower intrinsics (such as objsize)
and allow some optimizations to turn conditional branches into unconditional.
This commit adds a simple control-flow optimization which merges two consecutive
basic blocks which are connected by a single edge. This allows the codegen to
operate on larger basic blocks.

rdar://11973998



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@161852 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-14 05:19:07 +00:00
Eli Friedman
6b951b25c3 The normal edge of an invoke is not allowed to branch to a block with a
landingpad.  Enforce it in the verifier, and fix the regression tests to match.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@161697 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-10 20:55:20 +00:00
Pete Cooper
3eeba88631 Fix crash when when do lto on Bullet. Dynamic GEPs in SROA were incorrectly being applied to all accesses to an alloca, not just the ones which read from the GEP. Thanks to Evan for reducing the test. rdar://11861001
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@161654 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-10 03:26:36 +00:00
Eli Friedman
1b88fc0122 isAllocLikeFn is allowed to return true for functions which read memory; make
sure we account for that correctly in DeadStoreElimination.  Fixes a regression
from r158919.  PR13547.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@161468 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-08 02:17:32 +00:00
Dan Gohman
9d1747c3cd Avoid recomputing the unique exit blocks and their insert points when doing
multiple scalar promotions on a single loop. This also has the effect of
preserving the order of stores sunk out of loops, which is aesthetically
pleasing, and it happens to fix the testcase in PR13542, though it doesn't
fix the underlying problem.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@161459 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-08 00:00:26 +00:00
Bob Wilson
f12c95a8af Fix a serious typo in InstCombine's optimization of comparisons.
An unsigned value converted to floating-point will always be greater than
a negative constant.  Unfortunately InstCombine reversed the check so that
unsigned values were being optimized to always be greater than all positive
floating-point constants.  <rdar://problem/12029145>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@161452 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-07 22:35:16 +00:00
Benjamin Kramer
b6fdd022b7 PR13095: Give an inline cost bonus to functions using byval arguments.
We give a bonus for every argument because the argument setup is not needed
anymore when the function is inlined. With this patch we interpret byval
arguments as a compact representation of many arguments. The byval argument
setup is implemented in the backend as an inline memcpy, so to model the
cost as accurately as possible we take the number of pointer-sized elements
in the byval argument and give a bonus of 2 instructions for every one of
those. The bonus is capped at 8 elements, which is the number of stores
at which the x86 backend switches from an expanded inline memcpy to a real
memcpy. It would be better to use the real memcpy threshold from the backend,
but it's not available via TargetData.

This change brings the performance of c-ray in line with gcc 4.7. The included
test case tries to reproduce the c-ray problem to catch regressions for this
benchmark early, its performance is dominated by the inline decision of a
specific call.

This only has a small impact on most code, more on x86 and arm than on x86_64
due to the way the ABI works. When building LLVM for x86 it gives a small
inline cost boost to virtually any function using StringRef or STL allocators,
but only a 0.01% increase in overall binary size. The size of gcc compiled by
clang actually shrunk by a couple bytes with this patch applied, but not
significantly.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@161413 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-07 11:13:19 +00:00
Chandler Carruth
961e1acfb2 Fix PR13412, a nasty miscompile due to the interleaved
instsimplify+inline strategy.

The crux of the problem is that instsimplify was reasonably relying on
an invariant that is true within any single function, but is no longer
true mid-inline the way we use it. This invariant is that an argument
pointer != a local (alloca) pointer.

The fix is really light weight though, and allows instsimplify to be
resiliant to these situations: when checking the relation ships to
function arguments, ensure that the argumets come from the same
function. If they come from different functions, then none of these
assumptions hold. All credit to Benjamin Kramer for coming up with this
clever solution to the problem.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@161410 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-07 10:59:59 +00:00
Chandler Carruth
e6450dc2af Add a much more conservative strategy for aligning branch targets.
Previously, MBP essentially aligned every branch target it could. This
bloats code quite a bit, especially non-looping code which has no real
reason to prefer aligned branch targets so heavily.

As Andy said in review, it's still a bit odd to do this without a real
cost model, but this at least has much more plausible heuristics.

Fixes PR13265.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@161409 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-07 09:45:24 +00:00
Nadav Rotem
97baaeaeb2 When constant folding GEP expressions, keep the address space information of pointers.
Together with Ran Chachick <ran.chachick@intel.com>



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@160954 91177308-0d34-0410-b5e6-96231b3b80d8
2012-07-30 07:25:20 +00:00
Nick Lewycky
d64cb165d7 Add testcases for GlobalOpt changes in r160693 and r160757.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@160925 91177308-0d34-0410-b5e6-96231b3b80d8
2012-07-29 01:15:37 +00:00
Nuno Lopes
e982de7be9 fix PR13390: do not loop forever with self-referencing self instructions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@160876 91177308-0d34-0410-b5e6-96231b3b80d8
2012-07-27 18:21:15 +00:00
Nuno Lopes
75564e3514 fix infinite loop in instcombine in the presence of a (malformed) self-referencing select inst.
This can happen as long as the instruction is not reachable. Instcombine does generate these unreachable malformed selects when doing RAUW

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@160874 91177308-0d34-0410-b5e6-96231b3b80d8
2012-07-27 18:03:57 +00:00
Pete Cooper
7971de4178 Simplify demanded bits of select sources where the condition is a constant vector
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@160835 91177308-0d34-0410-b5e6-96231b3b80d8
2012-07-26 23:10:24 +00:00