refactored out of ScalarEvolution::isImpliedCond, which will be updated
to use this new utility routine soon.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@102229 91177308-0d34-0410-b5e6-96231b3b80d8
they're used a lot by getNodeForGEP, which can be called a lot.
This speeds up -iv-users by around 15% on several testcases.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@101083 91177308-0d34-0410-b5e6-96231b3b80d8
AddRecs so that it checks for overflow in the computation that it is
performing, rather than just checking hasNo{Signed,Unsigned}Wrap, since
those flags are for a different computation. This fixes a bug that
impacts an upcoming change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@101028 91177308-0d34-0410-b5e6-96231b3b80d8
BumpPtrAllocator-allocated region to allow it to be stored in a more
compact form and to avoid the need for a non-trivial destructor call.
Use this new mechanism in ScalarEvolution instead of
FastFoldingSetNode to avoid leaking memory in the case where a
FoldingSetNodeID uses heap storage, and to reduce overall memory
usage.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@98829 91177308-0d34-0410-b5e6-96231b3b80d8
pointer and length, and allocate the arrays in ScalarEvolution's
BumpPtrAllocator, so that they get released when their owning
SCEV gets released. SCEVs are immutable, so they don't need to worry
about operand array resizing. This fixes a memory leak reported
in PR6637.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@98755 91177308-0d34-0410-b5e6-96231b3b80d8
which branch on undef to branch on a boolean constant for the edge
exiting the loop. This helps ScalarEvolution compute trip counts for
loops.
Teach ScalarEvolution to recognize single-value PHIs, when safe, and
ForgetSymbolicName to forget such single-value PHI nodes as apprpriate
in ForgetSymbolicName.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@97126 91177308-0d34-0410-b5e6-96231b3b80d8
true or false as its exit condition. These are usually eliminated by
SimplifyCFG, but the may be left around during a pass which wishes
to preserve the CFG.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@96683 91177308-0d34-0410-b5e6-96231b3b80d8
a loop exit value, so that if a loop gets deleted, ScalarEvolution
isn't stick holding on to dangling SCEVAddRecExprs for that loop. This
fixes PR6339.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@96626 91177308-0d34-0410-b5e6-96231b3b80d8
and T->isPointerTy(). Convert most instances of the first form to the second form.
Requested by Chris.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@96344 91177308-0d34-0410-b5e6-96231b3b80d8
name, test whether the SCEV itself is that temporary symbolic name,
in addition to checking whether the symbolic name appears as a
possibly-indirect operand.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@96216 91177308-0d34-0410-b5e6-96231b3b80d8
SCEVAddRecExpr doesn't necessarily dominate blocks merely dominated
by all of its operands. This fixes an abort compiling 403.gcc.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@96056 91177308-0d34-0410-b5e6-96231b3b80d8
than int. This will make it more convenient for LSR, which does
a lot of things with int64_t offsets.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@95281 91177308-0d34-0410-b5e6-96231b3b80d8
cases, and implement target-independent folding rules for alignof and
offsetof. Also, reassociate reassociative operators when it leads to
more folding.
Generalize ScalarEvolution's isOffsetOf to recognize offsetof on
arrays. Rename getAllocSizeExpr to getSizeOfExpr, and getFieldOffsetExpr
to getOffsetOfExpr, for consistency with analagous ConstantExpr routines.
Make the target-dependent folder promote GEP array indices to
pointer-sized integers, to make implicit casting explicit and exposed
to subsequent folding.
And add a bunch of testcases for this new functionality, and a bunch
of related existing functionality.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@94987 91177308-0d34-0410-b5e6-96231b3b80d8
use plain SCEVUnknowns with ConstantExpr::getSizeOf and
ConstantExpr::getOffsetOf constants. This eliminates a bunch of
special-case code.
Also add code for pattern-matching these expressions, for clients that
want to recognize them.
Move ScalarEvolution's logic for expanding array and vector sizeof
expressions into an element count times the element size, to expose
the multiplication to subsequent folding, into the regular constant
folder.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@94737 91177308-0d34-0410-b5e6-96231b3b80d8
have trouble with an intermediate add overflowing. Also, be more conservative
about the case where the induction variable in an SLT loop exit can step past
the RHS of the SLT and overflow in a single step.
Make getSignedRange more aggressive, to recover for some common cases which
the above fixes pessimized.
This addresses rdar://7561161.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@94512 91177308-0d34-0410-b5e6-96231b3b80d8
This new version is much more aggressive about doing "full" reduction in
cases where it reduces register pressure, and also more aggressive about
rewriting induction variables to count down (or up) to zero when doing so
reduces register pressure.
It currently uses fairly simplistic algorithms for finding reuse
opportunities, but it introduces a new framework allows it to combine
multiple strategies at once to form hybrid solutions, instead of doing
all full-reduction or all base+index.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@94061 91177308-0d34-0410-b5e6-96231b3b80d8
contains another loop, or an instruction. The loop form is
substantially more efficient on large loops than the typical
code it replaces.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@91654 91177308-0d34-0410-b5e6-96231b3b80d8
Chris claims we should never have visibility_hidden inside any .cpp file but
that's still not true even after this commit.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@85042 91177308-0d34-0410-b5e6-96231b3b80d8
where the induction variable has a non-unit stride, such as {0,+,2}, and
there are expressions such as {1,+,2} inside the loop formed with
or or add nsw operators.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@82151 91177308-0d34-0410-b5e6-96231b3b80d8
SCEVUnknowns, as the non-SCEVUnknown cases in the getSCEVAtScope code
can also end up repeatedly climing through the same expression trees,
which can be unusably slow when the trees are very tall.
Also, add a quick check for SCEV pointer equality to the main
SCEV comparison routine, as the full comparison code can be expensive
in the case of large expression trees.
These fix compile-time problems in some pathlogical cases.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@80623 91177308-0d34-0410-b5e6-96231b3b80d8
will always return the same value. This isn't currently necessary,
since this code doesn't currently ever get called under circumstances
where it would matter, but it may some day.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@80017 91177308-0d34-0410-b5e6-96231b3b80d8
by hand, and fix a few places that were using hasInitializer() that
appear to depend on the initializer value.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@79441 91177308-0d34-0410-b5e6-96231b3b80d8
TargetData is not present. It still uses TargetData when available.
This generalization also fixed some limitations in the TargetData
case; the attached testcase covers this.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@79344 91177308-0d34-0410-b5e6-96231b3b80d8
the step value as unsigned, the start value and the addrec
itself still need to be treated as signed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@77078 91177308-0d34-0410-b5e6-96231b3b80d8
affected after a PHI node has been analyzed, just remove affected
SCEVs from the Scalars map, so that they'll be (lazily) recreated as
needed. This avoids creating SCEV objects that aren't actually needed.
Also, rewrite the associated def-use walking code to be non-recursive
and to continue traversing past Instructions that don't have an
entry in the Scalars map.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@77032 91177308-0d34-0410-b5e6-96231b3b80d8
(x pred y) with more thorough code that does more complete canonicalization
before resorting to range checks. This helps it find more cases where
the canonicalized expressions match.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@76671 91177308-0d34-0410-b5e6-96231b3b80d8
Getelementptrs that are defined to wrap are virtually useless to
optimization, and getelementptrs that are undefined on any kind
of overflow are too restrictive -- it's difficult to ensure that
all intermediate addresses are within bounds. I'm going to take
a different approach.
Remove a few optimizations that depended on this flag.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@76437 91177308-0d34-0410-b5e6-96231b3b80d8
in a convenient manner, factoring out some common code from
InstructionCombining and ValueTracking. Move the contents of
BinaryOperators.h into Operator.h and use Operator to generalize them
to support ConstantExprs as well as Instructions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@76232 91177308-0d34-0410-b5e6-96231b3b80d8
This adds location info for all llvm_unreachable calls (which is a macro now) in
!NDEBUG builds.
In NDEBUG builds location info and the message is off (it only prints
"UREACHABLE executed").
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@75640 91177308-0d34-0410-b5e6-96231b3b80d8
check for avoiding re-analyzing a widening cast needed to happen
earlier, as getSCEV itself may result in a isLoopGuardedByCond query.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@75511 91177308-0d34-0410-b5e6-96231b3b80d8
SCEVZeroExtendExpr ahead of the most expensive analysis. This
speeds up analysis and helps avoid pathologically bad behavior
on the testcase in PR4534.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@75496 91177308-0d34-0410-b5e6-96231b3b80d8
a bunch of redundent code in Profile methods, and prepares for upcoming
changes to do improved memoization.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@75494 91177308-0d34-0410-b5e6-96231b3b80d8
This involves temporarily hard wiring some parts to use the global context. This isn't ideal, but it's
the only way I could figure out to make this process vaguely incremental.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@75445 91177308-0d34-0410-b5e6-96231b3b80d8
Make llvm_unreachable take an optional string, thus moving the cerr<< out of
line.
LLVM_UNREACHABLE is now a simple wrapper that makes the message go away for
NDEBUG builds.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@75379 91177308-0d34-0410-b5e6-96231b3b80d8
of loops. Add several new functions to for working with ScalarEvolution's
add-hoc value-range analysis functionality.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@75252 91177308-0d34-0410-b5e6-96231b3b80d8
to a loop deletion more thorough. Don't prune the def-use tree search at
instructions that don't have SCEVs computed, because an instruction with
a user that has a computed SCEV may itself lack a computed SCEV. Also,
remove loop-related values from the ValuesAtScopes and
ConstantEvolutionLoopExitValues maps as well.
This fixes a regression in 483.xalancbmk.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@75030 91177308-0d34-0410-b5e6-96231b3b80d8
than a wider one, before trying to compare their contents which will crash
if their sizes are different.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@74792 91177308-0d34-0410-b5e6-96231b3b80d8
Constant. This lets ConstantInts be handled as SCEVConstant instead
of SCEVUnknown, as getUnknown no longer has special-case code for
ConstantInt and friends. This usually doesn't affect the final
output, since the constants end up getting folded later, but it
does make intermediate expressions more obvious in many cases.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@74459 91177308-0d34-0410-b5e6-96231b3b80d8
an individual exhaustive evaluation reflects only the exit value
implied by an individual exit, which may differ from the actual
exit value of the loop if there are other exits. This fixes PR4477.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@74447 91177308-0d34-0410-b5e6-96231b3b80d8
nesting order of nested AddRec expressions to skip the transformation
if it would introduce an AddRec with operands not loop-invariant
with respect to its loop.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@74343 91177308-0d34-0410-b5e6-96231b3b80d8
computations in loops with multiple exits.
Adjust the testcase for PR4436 so that the relevant portion isn't
optimized away.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@74073 91177308-0d34-0410-b5e6-96231b3b80d8
trip counts in more cases.
Generalize ScalarEvolution's isLoopGuardedByCond code to recognize
And and Or conditions, splitting the code out into an
isNecessaryCond helper function so that it can evaluate Ands and Ors
recursively, and make SCEVExpander be much more aggressive about
hoisting instructions out of loops.
test/CodeGen/X86/pr3495.ll has an additional instruction now, but
it appears to be due to an arbitrary register allocation difference.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@74048 91177308-0d34-0410-b5e6-96231b3b80d8
createSCEV. Also, recognize UndefValue in createSCEV.
Change getIntegerSCEV's comment to avoid mentioning FP types,
and re-implement it in terms of getConstant instead of getUnknown.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@74041 91177308-0d34-0410-b5e6-96231b3b80d8
This also throws out the SCEV reference counting scheme, as the the SCEVs now have a lifetime controlled by the
ScalarEvolution pass.
Note that SCEVHandle is now a no-op, and will be remove in a future commit.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73892 91177308-0d34-0410-b5e6-96231b3b80d8
counts for loops with multiple exits, replacing more conservative code
which only handled constants. This is derived from a patch by
Nick Lewycky.
This also fixes llc aborts in ClamAV and others, as
getUMinFromMismatchedTypes takes care of balancing the types before
working with them.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73884 91177308-0d34-0410-b5e6-96231b3b80d8
blocks, and also exit blocks with multiple conditions (combined
with (bitwise) ands and ors). It's often infeasible to compute an
exact trip count in such cases, but a useful upper bound can often
be found.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73866 91177308-0d34-0410-b5e6-96231b3b80d8
SCEVUnknowns with identical Instructions to be equal. This allows
it to analze cases such as the attached testcase, where the front-end
has cloned the loop controlling expression. Along with r73805, this
lets IndVarSimplify eliminate all the sign-extend casts in the
loop in the attached testcase.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73807 91177308-0d34-0410-b5e6-96231b3b80d8
so that it can access the TargetData member (when available) and
use ValueTracking.h information to compute information for
SCEVUnknown Values.
Also add GetMinLeadingZeros and GetMinSignBits functions,
with minimal implementations.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73794 91177308-0d34-0410-b5e6-96231b3b80d8
comparisons were recognized for umax, but instcombine canonicalizes
unsigned comparisons with zero to this simpler form.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73717 91177308-0d34-0410-b5e6-96231b3b80d8
If C is a single bit and the and gets analyzed as a truncate and
zero-extend, the xor can be represnted as an add.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73664 91177308-0d34-0410-b5e6-96231b3b80d8
that gets recognized with a SCEVZeroExtendExpr must be an And
with a low-bits mask. With r73540, this is no longer the case.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73594 91177308-0d34-0410-b5e6-96231b3b80d8
obscuring what would otherwise be a low-bits mask. Use ComputeMaskedBits
to compute what ShrinkDemandedConstant knew about to reconstruct a
low-bits mask value.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73540 91177308-0d34-0410-b5e6-96231b3b80d8
failures.
To support this, add some utility functions to Type to help support
vector/scalar-independent code. Change ConstantInt::get and
ConstantFP::get to support vector types, and add an overload to
ConstantInt::get that uses a static IntegerType type, for
convenience.
Introduce a new getConstant method for ScalarEvolution, to simplify
common use cases.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73431 91177308-0d34-0410-b5e6-96231b3b80d8
they contain multiplications of constants with add operations.
This helps simplify several kinds of things; in particular it
helps simplify expressions like ((-1 * (%a + %b)) + %a) to %b,
as expressions like this often come up in loop trip count
computations.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73361 91177308-0d34-0410-b5e6-96231b3b80d8
even though the order doesn't matter at the top level of an expression,
it does matter when the constant is a subexpression of an n-ary
expression, because n-ary expressions are sorted lexicographically.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73358 91177308-0d34-0410-b5e6-96231b3b80d8
that of the LHS. It doesn't matter for correctness, but the LHS
is more likely than the RHS to be a pointer type in exotic cases,
and it's more tidy to have it return the integer type.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@72424 91177308-0d34-0410-b5e6-96231b3b80d8
in the case where a loop exit value cannot be computed, instead of only in
some cases while using SCEVCouldNotCompute in others. This simplifies
getSCEVAtScope's callers.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@72375 91177308-0d34-0410-b5e6-96231b3b80d8
instructions. It attempts to create high-level multi-operand GEPs,
though in cases where this isn't possible it falls back to casting
the pointer to i8* and emitting a GEP with that. Using GEP instructions
instead of ptrtoint+arithmetic+inttoptr helps pointer analyses that
don't use ScalarEvolution, such as BasicAliasAnalysis.
Also, make the AddrModeMatcher more aggressive in handling GEPs.
Previously it assumed that operand 0 of a GEP would require a register
in almost all cases. It now does extra checking and can do more
matching if operand 0 of the GEP is foldable. This fixes a problem
that was exposed by SCEVExpander using GEPs.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@72093 91177308-0d34-0410-b5e6-96231b3b80d8
