Commit Graph

4369 Commits

Author SHA1 Message Date
Nadav Rotem
0636291137 Revert 166726 because it may have broken a number of SPEC tests. PR14183.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166739 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25 23:51:48 +00:00
Nadav Rotem
4f3c676102 Fix a crash in ValueTracking. Add support for vectors of pointers.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166726 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25 21:52:52 +00:00
Nadav Rotem
b6cbb6215c Fix the cost-model test.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166722 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25 21:42:50 +00:00
Hal Finkel
32ff623c8b Add CPU model to BBVectorize cost-model tests.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166720 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25 21:31:51 +00:00
Nadav Rotem
402a4d616f Add the cpu model to the test.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166718 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25 21:18:42 +00:00
Hal Finkel
65309660fa Begin incorporating target information into BBVectorize.
This is the first of several steps to incorporate information from the new
TargetTransformInfo infrastructure into BBVectorize. Two things are done here:

 1. Target information is used to determine if it is profitable to fuse two
    instructions. This means that the cost of the vector operation must not
    be more expensive than the cost of the two original operations. Pairs that
    are not profitable are no longer considered (because current cost information
    is incomplete, for intrinsics for example, equal-cost pairs are still
    considered).

 2. The 'cost savings' computed for the profitability check are also used to
    rank the DAGs that represent the potential vectorization plans. Specifically,
    for nodes of non-trivial depth, the cost savings is used as the node
    weight.

The next step will be to incorporate the shuffle costs into the DAG weighting;
this will give the edges of the DAG weights as well. Once that is done, when
target information is available, we should be able to dispense with the
depth heuristic.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166716 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25 21:12:23 +00:00
Jakob Stoklund Olesen
e5a7a68dfa Also optimize large switch statements.
The isValueEqualityComparison() guard at the top of SimplifySwitch()
only applies to some of the possible transformations.

The newer transformations work just fine on large switches, and the
check on predecessor count is nonsensical.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166710 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25 18:51:15 +00:00
Chandler Carruth
a2b88163af Teach SROA how to split whole-alloca integer loads and stores into
smaller integer loads and stores.

The high-level motivation is that the frontend sometimes generates
a single whole-alloca integer load or store during ABI lowering of
splittable allocas. We need to be able to break this apart in order to
see the underlying elements and properly promote them to SSA values. The
hope is that this fixes some performance regressions on x86-32 with the
new SROA pass.

Unfortunately, this causes quite a bit of churn in the test cases, and
bloats some IR that comes out. When we see an alloca that consists soley
of bits and bytes being extracted and re-inserted, we now do some
splitting first, before building widened integer "bucket of bits"
representations. These are always well folded by instcombine however, so
this shouldn't actually result in missed opportunities.

If this splitting of all-integer allocas does cause problems (perhaps
due to smaller SSA values going into the RA), we could potentially go to
some extreme measures to only do this integer splitting trick when there
are non-integer component accesses of an alloca, but discovering this is
quite expensive: it adds yet another complete walk of the recursive use
tree of the alloca.

Either way, I will be watching build bots and LNT bots to see what
fallout there is here. If anyone gets x86-32 numbers before & after this
change, I would be very interested.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166662 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25 04:37:07 +00:00
Nadav Rotem
8dbac7b529 Add support for additional reduction variables: AND, OR, XOR.
Patch by Paul Redmond <paul.redmond@intel.com>.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166649 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-25 00:08:41 +00:00
Nadav Rotem
2652c50f74 Implement a basic cost model for vector and scalar instructions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166642 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24 23:47:38 +00:00
Hal Finkel
aacb68806f Update GVN to support vectors of pointers.
GVN will now generate ptrtoint instructions for vectors of pointers.
Fixes PR14166.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166624 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24 21:22:30 +00:00
Nadav Rotem
50bec6f8c4 LoopVectorizer: Add a basic cost model which uses the VTTI interface.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166620 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24 20:36:32 +00:00
Hal Finkel
8c65549318 getSmallConstantTripMultiple should never return zero.
When the trip count is -1, getSmallConstantTripMultiple could return zero,
and this would cause runtime loop unrolling to assert. Instead of returning
zero, one is now returned (consistent with the existing overflow cases).
Fixes PR14167.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166612 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24 19:46:44 +00:00
Micah Villmow
aa76e9e2cf Add in support for getIntPtrType to get the pointer type based on the address space.
This checkin also adds in some tests that utilize these paths and updates some of the
clients.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166578 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24 15:52:52 +00:00
Duncan Sands
747fcd58bc Add a testcase that would have noticed the typo fixed in commit 166475.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166547 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24 07:17:20 +00:00
Nadav Rotem
6457001f31 Use the AliasAnalysis isIdentifiedObj because it also understands mallocs and c++ news.
PR14158.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166491 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-23 18:44:18 +00:00
Bill Wendling
b846719663 Ignore unreachable blocks when doing memory dependence analysis on non-local
loads. It's not really profitable and may result in GVN going into an infinite
loop when it hits constructs like this:

     %x = gep %some.type %x, ...

Found via an LTO build of LLVM.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166490 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-23 18:37:11 +00:00
Duncan Sands
bbc7016c60 Transform code like this
%V = mul i64 %N, 4
 %t = getelementptr i8* bitcast (i32* %arr to i8*), i32 %V
into
 %t1 = getelementptr i32* %arr, i32 %N
 %t = bitcast i32* %t1 to i8*
incorporating the multiplication into the getelementptr.
This happens all the time in dragonegg, for example for
  int foo(int *A, int N) {
    return A[N];
  }
because gcc turns this into byte pointer arithmetic before it hits the plugin:
  D.1590_2 = (long unsigned int) N_1(D);
  D.1591_3 = D.1590_2 * 4;
  D.1592_5 = A_4(D) + D.1591_3;
  D.1589_6 = *D.1592_5;
  return D.1589_6;
The D.1592_5 line is a POINTER_PLUS_EXPR, which is turned into a getelementptr
on a bitcast of A_4 to i8*, so this becomes exactly the kind of IR that the
transform fires on.

An analogous transform (with no testcases!) already existed for bitcasts of
arrays, so I rewrote it to share code with this one.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166474 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-23 08:28:26 +00:00
Nadav Rotem
782090aa02 Don't crash if the load/store pointer is not a GEP.
Fix by Shivarama Rao <Shivarama.Rao@amd.com>



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166427 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22 18:27:56 +00:00
Argyrios Kyrtzidis
0b06e2331e Revert r166407 because it caused analyzer tests to crash and broke self-host bots.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166424 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22 18:16:14 +00:00
Hal Finkel
e29c19091c BBVectorize should ignore unreachable blocks.
Unreachable blocks can have invalid instructions. For example,
jump threading can produce self-referential instructions in
unreachable blocks. Also, we should not be spending time
optimizing unreachable code. Fixes PR14133.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166423 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22 18:00:55 +00:00
Nadav Rotem
565048e78a Vectorizer: optimize the generation of selects. If the condition is uniform, generate a scalar-cond select (i1 as selector).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166409 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22 04:38:00 +00:00
Nick Lewycky
18b1f4e769 Reapply r166405, teaching tailcallelim to be smarter about nocapture, with a
very small but very important bugfix:
  bool shouldExplore(Use *U) {
    Value *V = U->get();
    if (isa<CallInst>(V) || isa<InvokeInst>(V))
    [...]
should have read:
  bool shouldExplore(Use *U) {
    Value *V = U->getUser();
    if (isa<CallInst>(V) || isa<InvokeInst>(V))
Fixes PR14143!



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166407 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22 03:03:52 +00:00
NAKAMURA Takumi
d581b9e61f Revert r166405, "Teach TailRecursionElimination to consider 'nocapture' when deciding whether"
It broke selfhosting stage2 in several builders.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166406 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-22 00:48:51 +00:00
Nick Lewycky
241d1398e0 Teach TailRecursionElimination to consider 'nocapture' when deciding whether
calls can be marked tail.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166405 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21 23:51:22 +00:00
Hal Finkel
3d39fb8a3f DataLayout should use itself when calculating the size of a vector.
This is important for vectors of pointers because only DataLayout,
not the underlying vector type, knows how to calculate the size
of the pointers in the vector. Fixes PR14138.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166401 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21 20:38:03 +00:00
Benjamin Kramer
3740e798bc Revert r166390 "LoopIdiom: Replace custom dependence analysis with LoopDependenceAnalysis."
It passes all tests, produces better results than the old code but uses the
wrong pass, LoopDependenceAnalysis, which is old and unmaintained. "Why is it
still in tree?", you might ask. The answer is obviously: "To confuse developers."

Just swapping in the new dependency pass sends the pass manager into an infinte
loop, I'll try to figure out why tomorrow.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166399 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21 19:31:16 +00:00
Benjamin Kramer
5c6e9ae14e LoopIdiom: Replace custom dependence analysis with LoopDependenceAnalysis.
Requires a lot less code and complexity on loop-idiom's side and the more
precise analysis can catch more cases, like the one I included as a test case.
This also fixes the edge-case miscompilation from PR9481. I'm not entirely
sure that all cases are handled that the old checks handled but LDA will
certainly become smarter in the future.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166390 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21 15:03:07 +00:00
Nadav Rotem
bb950854ac Fix a bug in the vectorization of wide load/store operations.
We used a SCEV to detect that A[X] is consecutive. We assumed that X was
the induction variable. But X can be any expression that uses the induction
for example: X = i + 2;



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166388 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21 06:49:10 +00:00
Nadav Rotem
c847872629 Add support for reduction variables that do not start at zero.
This is important for nested-loop reductions such as :

In the innermost loop, the induction variable does not start with zero:

for (i = 0 .. n)
 for (j = 0 .. m)
  sum += ...



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166387 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21 05:52:51 +00:00
Nadav Rotem
5a418ba5f5 Vectorizer: fix a bug in the classification of induction/reduction phis.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166384 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-21 02:38:01 +00:00
Nadav Rotem
ccaccfa8bf Fix an infinite loop in the loop-vectorizer.
PR14134.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166379 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-20 20:45:01 +00:00
Benjamin Kramer
82a1833865 InstCombine: Fix an edge case where constant icmps could sneak into ConstantFoldInstOperands and crash.
Have to refactor the ConstantFolder interface one day to define bugs like this away. Fixes PR14131.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166374 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-20 08:43:52 +00:00
Nadav Rotem
bf8772ed2c Vectorize: teach cavVectorizeMemory to distinguish between A[i]+=x and A[B[i]]+=x.
If the pointer is consecutive then it is safe to read and write. If the pointer is non-loop-consecutive then
it is unsafe to vectorize it because we may hit an ordering issue.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166371 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-20 08:26:33 +00:00
Nadav Rotem
5dbe64e2bc Vectorizer: Add support for loop reductions.
For example:

  for (i=0; i<n; i++)
   sum += A[i] +  B[i] + i;



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166351 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-19 23:05:40 +00:00
Benjamin Kramer
0aae4bd0fc SimplifyLibcalls: The return value of ffsll is always i32, even when the input is zero.
Fixes PR13028.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166313 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-19 20:43:44 +00:00
Benjamin Kramer
7182126b0f Indvars: Don't recursively delete instruction during BB iteration.
This can invalidate the iterators leading to use after frees and crashes.
Fixes PR12536.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166291 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-19 17:53:54 +00:00
Benjamin Kramer
239fd44f7a SCEVExpander: Don't crash when trying to merge two constant phis.
Just constant fold them so they can't cause any trouble. Fixes PR12627.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166286 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-19 16:37:30 +00:00
Nadav Rotem
89e7b356f2 vectorizer: Add support for reading and writing from the same memory location.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166255 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-19 01:24:18 +00:00
Meador Inge
0c41d57b09 instcombine: Migrate strcpy optimizations
This patch migrates the strcpy optimizations from the simplify-libcalls pass
into the instcombine library call simplifier.  Note also that StrCpyChkOpt
has been updated with a few simplifications that were being done in the
simplify-libcalls version of StrCpyOpt, but not in the migrated implementation
of StrCpyOpt.  There is no reason to overload StrCpyOpt with fortified and
regular simplifications in the new model since there is already a dedicated
simplifier for __strcpy_chk.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166198 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-18 18:12:40 +00:00
Nadav Rotem
1953ace81d Vectorizer: Add support for loops with an unknown count. For example:
for (i=0; i<n; i++){
        a[i] = b[i+1] + c[i+3];
     }



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166165 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-18 05:29:12 +00:00
Nadav Rotem
d15c0c7ac1 Add a loop vectorizer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166112 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-17 18:25:06 +00:00
Chandler Carruth
02bf98ab38 This just in, it is a *bad idea* to use 'udiv' on an offset of
a pointer. A very bad idea. Let's not do that. Fixes PR14105.

Note that this wasn't *that* glaring of an oversight. Originally, these
routines were only called on offsets within an alloca, which are
intrinsically positive. But over the evolution of the pass, they ended
up being called for arbitrary offsets, and things went downhill...

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166095 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-17 09:23:48 +00:00
Michael Gottesman
4932bbe20c [InstCombine] Teach InstCombine how to handle an obfuscated splat.
An obfuscated splat is where the frontend poorly generates code for a splat
using several different shuffles to create the splat, i.e.,

  %A = load <4 x float>* %in_ptr, align 16
  %B = shufflevector <4 x float> %A, <4 x float> undef, <4 x i32> <i32 0, i32 0, i32 undef, i32 undef>
  %C = shufflevector <4 x float> %B, <4 x float> %A, <4 x i32> <i32 0, i32 1, i32 4, i32 undef>
  %D = shufflevector <4 x float> %C, <4 x float> %A, <4 x i32> <i32 0, i32 1, i32 2, i32 4>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166061 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-16 21:29:38 +00:00
Chandler Carruth
d2cd73f6a5 Update the memcpy rewriting to fully support widened int rewriting. This
includes extracting ints for copying elsewhere and inserting ints when
copying into the alloca. This should fix the CanSROA assertion coming
out of Clang's regression test suite.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165931 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-15 10:24:43 +00:00
Chandler Carruth
94fc64c42f Follow-up fix to r165928: handle memset rewriting for widened integers,
and generally clean up the memset handling. It had rotted a bit as the
other rewriting logic got polished more.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165930 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-15 10:24:40 +00:00
Chandler Carruth
81ff90db44 First major step toward addressing PR14059. This teaches SROA to handle
cases where we have partial integer loads and stores to an otherwise
promotable alloca to widen[1] those loads and stores to cover the entire
alloca and bitcast them into the appropriate type such that promotion
can proceed.

These partial loads and stores stem from an annoying confluence of ARM's
calling convention and ABI lowering and the FCA pre-splitting which
takes place in SROA. Clang lowers a { double, double } in-register
function argument as a [4 x i32] function argument to ensure it is
placed into integer 32-bit registers (a really unnerving implicit
contract between Clang and the ARM backend I would add). This results in
a FCA load of [4 x i32]* from the { double, double } alloca, and SROA
decomposes this into a sequence of i32 loads and stores. Inlining
proceeds, code gets folded, but at the end of the day, we still have i32
stores to the low and high halves of a double alloca. Widening these to
be i64 operations, and bitcasting them to double prior to loading or
storing allows promotion to proceed for these allocas.

I looked quite a bit changing the IR which Clang produces for this case
to be more friendly, but small changes seem unlikely to help. I think
the best representation we could use currently would be to pass 4 i32
arguments thereby avoiding any FCAs, but that would still require this
fix. It seems like it might eventually be nice to somehow encode the ABI
register selection choices outside of the parameter type system so that
the parameter can be a { double, double }, but the CC register
annotations indicate that this should be passed via 4 integer registers.

This patch does not address the second problem in PR14059, which is the
reverse: when a struct alloca is loaded as a *larger* single integer.

This patch also does not address some of the code quality issues with
the FCA-splitting. Those don't actually impede any optimizations really,
but they're on my list to clean up.

[1]: Pedantic footnote: for those concerned about memory model issues
here, this is safe. For the alloca to be promotable, it cannot escape or
have any use of its address that could allow these loads or stores to be
racing. Thus, widening is always safe.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165928 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-15 08:40:30 +00:00
Meador Inge
a239c2e6a7 instcombine: Migrate strcmp and strncmp optimizations
This patch migrates the strcmp and strncmp optimizations from the
simplify-libcalls pass into the instcombine library call simplifier.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165915 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-15 03:47:37 +00:00
Meador Inge
186f8d90df instcombine: Migrate strchr and strrchr optimizations
This patch migrates the strchr and strrchr optimizations from the
simplify-libcalls pass into the instcombine library call simplifier.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165875 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-13 16:45:37 +00:00
Meador Inge
73d8a5864f instcombine: Migrate strcat and strncat optimizations
This patch migrates the strcat and strncat optimizations from the
simplify-libcalls pass into the instcombine library call simplifier.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165874 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-13 16:45:32 +00:00