SCEVUnknowns, as the non-SCEVUnknown cases in the getSCEVAtScope code
can also end up repeatedly climing through the same expression trees,
which can be unusably slow when the trees are very tall.
Also, add a quick check for SCEV pointer equality to the main
SCEV comparison routine, as the full comparison code can be expensive
in the case of large expression trees.
These fix compile-time problems in some pathlogical cases.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@80623 91177308-0d34-0410-b5e6-96231b3b80d8
will always return the same value. This isn't currently necessary,
since this code doesn't currently ever get called under circumstances
where it would matter, but it may some day.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@80017 91177308-0d34-0410-b5e6-96231b3b80d8
by hand, and fix a few places that were using hasInitializer() that
appear to depend on the initializer value.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@79441 91177308-0d34-0410-b5e6-96231b3b80d8
TargetData is not present. It still uses TargetData when available.
This generalization also fixed some limitations in the TargetData
case; the attached testcase covers this.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@79344 91177308-0d34-0410-b5e6-96231b3b80d8
the step value as unsigned, the start value and the addrec
itself still need to be treated as signed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@77078 91177308-0d34-0410-b5e6-96231b3b80d8
affected after a PHI node has been analyzed, just remove affected
SCEVs from the Scalars map, so that they'll be (lazily) recreated as
needed. This avoids creating SCEV objects that aren't actually needed.
Also, rewrite the associated def-use walking code to be non-recursive
and to continue traversing past Instructions that don't have an
entry in the Scalars map.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@77032 91177308-0d34-0410-b5e6-96231b3b80d8
(x pred y) with more thorough code that does more complete canonicalization
before resorting to range checks. This helps it find more cases where
the canonicalized expressions match.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@76671 91177308-0d34-0410-b5e6-96231b3b80d8
Getelementptrs that are defined to wrap are virtually useless to
optimization, and getelementptrs that are undefined on any kind
of overflow are too restrictive -- it's difficult to ensure that
all intermediate addresses are within bounds. I'm going to take
a different approach.
Remove a few optimizations that depended on this flag.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@76437 91177308-0d34-0410-b5e6-96231b3b80d8
in a convenient manner, factoring out some common code from
InstructionCombining and ValueTracking. Move the contents of
BinaryOperators.h into Operator.h and use Operator to generalize them
to support ConstantExprs as well as Instructions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@76232 91177308-0d34-0410-b5e6-96231b3b80d8
This adds location info for all llvm_unreachable calls (which is a macro now) in
!NDEBUG builds.
In NDEBUG builds location info and the message is off (it only prints
"UREACHABLE executed").
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@75640 91177308-0d34-0410-b5e6-96231b3b80d8
check for avoiding re-analyzing a widening cast needed to happen
earlier, as getSCEV itself may result in a isLoopGuardedByCond query.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@75511 91177308-0d34-0410-b5e6-96231b3b80d8
SCEVZeroExtendExpr ahead of the most expensive analysis. This
speeds up analysis and helps avoid pathologically bad behavior
on the testcase in PR4534.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@75496 91177308-0d34-0410-b5e6-96231b3b80d8
a bunch of redundent code in Profile methods, and prepares for upcoming
changes to do improved memoization.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@75494 91177308-0d34-0410-b5e6-96231b3b80d8
This involves temporarily hard wiring some parts to use the global context. This isn't ideal, but it's
the only way I could figure out to make this process vaguely incremental.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@75445 91177308-0d34-0410-b5e6-96231b3b80d8
Make llvm_unreachable take an optional string, thus moving the cerr<< out of
line.
LLVM_UNREACHABLE is now a simple wrapper that makes the message go away for
NDEBUG builds.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@75379 91177308-0d34-0410-b5e6-96231b3b80d8
of loops. Add several new functions to for working with ScalarEvolution's
add-hoc value-range analysis functionality.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@75252 91177308-0d34-0410-b5e6-96231b3b80d8
to a loop deletion more thorough. Don't prune the def-use tree search at
instructions that don't have SCEVs computed, because an instruction with
a user that has a computed SCEV may itself lack a computed SCEV. Also,
remove loop-related values from the ValuesAtScopes and
ConstantEvolutionLoopExitValues maps as well.
This fixes a regression in 483.xalancbmk.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@75030 91177308-0d34-0410-b5e6-96231b3b80d8
than a wider one, before trying to compare their contents which will crash
if their sizes are different.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@74792 91177308-0d34-0410-b5e6-96231b3b80d8
Constant. This lets ConstantInts be handled as SCEVConstant instead
of SCEVUnknown, as getUnknown no longer has special-case code for
ConstantInt and friends. This usually doesn't affect the final
output, since the constants end up getting folded later, but it
does make intermediate expressions more obvious in many cases.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@74459 91177308-0d34-0410-b5e6-96231b3b80d8
an individual exhaustive evaluation reflects only the exit value
implied by an individual exit, which may differ from the actual
exit value of the loop if there are other exits. This fixes PR4477.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@74447 91177308-0d34-0410-b5e6-96231b3b80d8
nesting order of nested AddRec expressions to skip the transformation
if it would introduce an AddRec with operands not loop-invariant
with respect to its loop.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@74343 91177308-0d34-0410-b5e6-96231b3b80d8
computations in loops with multiple exits.
Adjust the testcase for PR4436 so that the relevant portion isn't
optimized away.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@74073 91177308-0d34-0410-b5e6-96231b3b80d8
trip counts in more cases.
Generalize ScalarEvolution's isLoopGuardedByCond code to recognize
And and Or conditions, splitting the code out into an
isNecessaryCond helper function so that it can evaluate Ands and Ors
recursively, and make SCEVExpander be much more aggressive about
hoisting instructions out of loops.
test/CodeGen/X86/pr3495.ll has an additional instruction now, but
it appears to be due to an arbitrary register allocation difference.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@74048 91177308-0d34-0410-b5e6-96231b3b80d8
createSCEV. Also, recognize UndefValue in createSCEV.
Change getIntegerSCEV's comment to avoid mentioning FP types,
and re-implement it in terms of getConstant instead of getUnknown.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@74041 91177308-0d34-0410-b5e6-96231b3b80d8
This also throws out the SCEV reference counting scheme, as the the SCEVs now have a lifetime controlled by the
ScalarEvolution pass.
Note that SCEVHandle is now a no-op, and will be remove in a future commit.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73892 91177308-0d34-0410-b5e6-96231b3b80d8
counts for loops with multiple exits, replacing more conservative code
which only handled constants. This is derived from a patch by
Nick Lewycky.
This also fixes llc aborts in ClamAV and others, as
getUMinFromMismatchedTypes takes care of balancing the types before
working with them.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73884 91177308-0d34-0410-b5e6-96231b3b80d8
blocks, and also exit blocks with multiple conditions (combined
with (bitwise) ands and ors). It's often infeasible to compute an
exact trip count in such cases, but a useful upper bound can often
be found.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73866 91177308-0d34-0410-b5e6-96231b3b80d8
SCEVUnknowns with identical Instructions to be equal. This allows
it to analze cases such as the attached testcase, where the front-end
has cloned the loop controlling expression. Along with r73805, this
lets IndVarSimplify eliminate all the sign-extend casts in the
loop in the attached testcase.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73807 91177308-0d34-0410-b5e6-96231b3b80d8
so that it can access the TargetData member (when available) and
use ValueTracking.h information to compute information for
SCEVUnknown Values.
Also add GetMinLeadingZeros and GetMinSignBits functions,
with minimal implementations.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73794 91177308-0d34-0410-b5e6-96231b3b80d8
comparisons were recognized for umax, but instcombine canonicalizes
unsigned comparisons with zero to this simpler form.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73717 91177308-0d34-0410-b5e6-96231b3b80d8
If C is a single bit and the and gets analyzed as a truncate and
zero-extend, the xor can be represnted as an add.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73664 91177308-0d34-0410-b5e6-96231b3b80d8
that gets recognized with a SCEVZeroExtendExpr must be an And
with a low-bits mask. With r73540, this is no longer the case.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73594 91177308-0d34-0410-b5e6-96231b3b80d8
obscuring what would otherwise be a low-bits mask. Use ComputeMaskedBits
to compute what ShrinkDemandedConstant knew about to reconstruct a
low-bits mask value.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73540 91177308-0d34-0410-b5e6-96231b3b80d8
failures.
To support this, add some utility functions to Type to help support
vector/scalar-independent code. Change ConstantInt::get and
ConstantFP::get to support vector types, and add an overload to
ConstantInt::get that uses a static IntegerType type, for
convenience.
Introduce a new getConstant method for ScalarEvolution, to simplify
common use cases.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73431 91177308-0d34-0410-b5e6-96231b3b80d8
they contain multiplications of constants with add operations.
This helps simplify several kinds of things; in particular it
helps simplify expressions like ((-1 * (%a + %b)) + %a) to %b,
as expressions like this often come up in loop trip count
computations.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73361 91177308-0d34-0410-b5e6-96231b3b80d8
even though the order doesn't matter at the top level of an expression,
it does matter when the constant is a subexpression of an n-ary
expression, because n-ary expressions are sorted lexicographically.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@73358 91177308-0d34-0410-b5e6-96231b3b80d8
that of the LHS. It doesn't matter for correctness, but the LHS
is more likely than the RHS to be a pointer type in exotic cases,
and it's more tidy to have it return the integer type.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@72424 91177308-0d34-0410-b5e6-96231b3b80d8
in the case where a loop exit value cannot be computed, instead of only in
some cases while using SCEVCouldNotCompute in others. This simplifies
getSCEVAtScope's callers.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@72375 91177308-0d34-0410-b5e6-96231b3b80d8
instructions. It attempts to create high-level multi-operand GEPs,
though in cases where this isn't possible it falls back to casting
the pointer to i8* and emitting a GEP with that. Using GEP instructions
instead of ptrtoint+arithmetic+inttoptr helps pointer analyses that
don't use ScalarEvolution, such as BasicAliasAnalysis.
Also, make the AddrModeMatcher more aggressive in handling GEPs.
Previously it assumed that operand 0 of a GEP would require a register
in almost all cases. It now does extra checking and can do more
matching if operand 0 of the GEP is foldable. This fixes a problem
that was exposed by SCEVExpander using GEPs.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@72093 91177308-0d34-0410-b5e6-96231b3b80d8
getNoopOrSignExtend, and getTruncateOrNoop. These are similar
to getTruncateOrZeroExtend etc., except that they assert that
the conversion is either not widening or narrowing, as
appropriate. These will be used in some upcoming fixes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@71632 91177308-0d34-0410-b5e6-96231b3b80d8
These values aren't analyzable, so they don't care if more information
about the loop trip count can be had. Also, SCEVUnknown is used for
a PHI while the PHI itself is being analyzed, so it needs to be left
in the Scalars map. This fixes a variety of subtle issues.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@71533 91177308-0d34-0410-b5e6-96231b3b80d8
return the correct value when the cast operand is all zeros. This ought
to be pretty rare, because it would mean that the regular SCEV folding
routines missed a case, though there are cases they might legitimately
miss. Also, it's unlikely anything currently using GetMinTrailingZeros
cares about this case.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@71532 91177308-0d34-0410-b5e6-96231b3b80d8
add-recurrence to be exposed. Add a new SCEV folding rule to
help simplify expressions in the presence of these extra truncs.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@71264 91177308-0d34-0410-b5e6-96231b3b80d8
which are not analyzed with SCEV techniques, which can require
brute-forcing through a large number of instructions. This
fixes a massive compile-time issue on 400.perlbench (in
particular, the loop in MD5Transform).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@71259 91177308-0d34-0410-b5e6-96231b3b80d8
to sorting SCEVs by their kind, sort SCEVs of the same kind according
to their operands. This helps avoid things like (a+b) being a distinct
expression from (b+a).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@71160 91177308-0d34-0410-b5e6-96231b3b80d8
CallbackVH, with fixes. allUsesReplacedWith need to
walk the def-use chains and invalidate all users of a
value that is replaced. SCEVs of users need to be
recalcualted even if the new value is equivalent. Also,
make forgetLoopPHIs walk def-use chains, since any
SCEV that depends on a PHI should be recalculated when
more information about that PHI becomes available.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@70927 91177308-0d34-0410-b5e6-96231b3b80d8
makes ScalarEvolution::deleteValueFromRecords, and it's code that
subtly needed to be called before ReplaceAllUsesWith, unnecessary.
It also makes ValueDeletionListener unnecessary.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@70645 91177308-0d34-0410-b5e6-96231b3b80d8
it also forget any SCEVs associated with loop-header PHIs in the loop,
as they may be dependent on trip count information.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@70633 91177308-0d34-0410-b5e6-96231b3b80d8
artificial "ptrtoint", as it tends to clutter up complicated
expressions. The cast operators now print both source and
destination types, which is usually sufficient.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@70554 91177308-0d34-0410-b5e6-96231b3b80d8
compute an upper-bound value for the trip count, in addition to
the actual trip count. Use this to allow getZeroExtendExpr and
getSignExtendExpr to fold casts in more cases.
This may eventually morph into a more general value-range
analysis capability; there are certainly plenty of places where
more complete value-range information would allow more folding.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@70509 91177308-0d34-0410-b5e6-96231b3b80d8
(sext i8 {-128,+,1} to i64) to i64 {-128,+,1}, where the iteration
crosses from negative to positive, but is still safe if the trip
count is within range.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@70421 91177308-0d34-0410-b5e6-96231b3b80d8
print sext, zext, and trunc, instead of signextend, zeroextend,
and truncate, respectively, for consistency with the main IR.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@70405 91177308-0d34-0410-b5e6-96231b3b80d8
information to simplify [sz]ext({a,+,b}) to {zext(a),+,[zs]ext(b)},
as appropriate.
These functions and the trip count code each call into the other, so
this requires careful handling to avoid infinite recursion. During
the initial trip count computation, conservative SCEVs are used,
which are subsequently discarded once the trip count is actually
known.
Among other benefits, this change lets LSR automatically eliminate
some unnecessary zext-inreg and sext-inreg operation where the
operand is an induction variable.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@70241 91177308-0d34-0410-b5e6-96231b3b80d8
type to truncate to should be the number of bits of the value that are
preserved, not the number that are clobbered with sign-extension.
This fixes regressions in ldecod.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@69704 91177308-0d34-0410-b5e6-96231b3b80d8
as they appear in LLVM IR. This isn't particularly interesting
on its own; this is just setting up some infrastructure.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@69655 91177308-0d34-0410-b5e6-96231b3b80d8
size from the integer, requiring zero extension or truncation. Don't
create ZExtInsts with pointer types. This fixes a regression in
consumer-jpeg.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@69307 91177308-0d34-0410-b5e6-96231b3b80d8
have pointer types, though in contrast to C pointer types, SCEV
addition is never implicitly scaled. This not only eliminates the
need for special code like IndVars' EliminatePointerRecurrence
and LSR's own GEP expansion code, it also does a better job because
it lets the normal optimizations handle pointer expressions just
like integer expressions.
Also, since LLVM IR GEPs can't directly index into multi-dimensional
VLAs, moving the GEP analysis out of client code and into the SCEV
framework makes it easier for clients to handle multi-dimensional
VLAs the same way as other arrays.
Some existing regression tests show improved optimization.
test/CodeGen/ARM/2007-03-13-InstrSched.ll in particular improved to
the point where if-conversion started kicking in; I turned it off
for this test to preserve the intent of the test.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@69258 91177308-0d34-0410-b5e6-96231b3b80d8
to more accurately describe what it does. Expand its doxygen comment
to describe what the backedge-taken count is and how it differs
from the actual iteration count of the loop. Adjust names and
comments in associated code accordingly.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@65382 91177308-0d34-0410-b5e6-96231b3b80d8
trip count value when the original loop iteration condition is
signed and the canonical induction variable won't undergo signed
overflow. This isn't required for correctness; it just preserves
more information about original loop iteration values.
Add a getTruncateOrSignExtend method to ScalarEvolution,
following getTruncateOrZeroExtend.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@64918 91177308-0d34-0410-b5e6-96231b3b80d8
modified in a way that may effect the trip count calculation. Change
IndVars to use this method when it rewrites pointer or floating-point
induction variables instead of using a doInitialization method to
sneak these changes in before ScalarEvolution has a chance to see
the loop. This eliminates the need for LoopPass to depend on
ScalarEvolution.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@64810 91177308-0d34-0410-b5e6-96231b3b80d8
loop induction on LP64 targets. When the induction variable is
used in addressing, IndVars now is usually able to inserst a
64-bit induction variable and eliminates the sign-extending cast.
This is also useful for code using C "short" types for
induction variables on targets with 32-bit addressing.
Inserting a wider induction variable is easy; the tricky part is
determining when trunc(sext(i)) expressions are no-ops. This
requires range analysis of the loop trip count. A common case is
when the original loop iteration starts at 0 and exits when the
induction variable is signed-less-than a fixed value; this case
is now handled.
This replaces IndVarSimplify's OptimizeCanonicalIVType. It was
doing the same optimization, but it was limited to loops with
constant trip counts, because it was running after the loop
rewrite, and the information about the original induction
variable is lost by that point.
Rename ScalarEvolution's executesAtLeastOnce to
isLoopGuardedByCond, generalize it to be able to test for
ICMP_NE conditions, and move it to be a public function so that
IndVars can use it.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@64407 91177308-0d34-0410-b5e6-96231b3b80d8
Use it to safely handle less-than-or-equals-to exit conditions in loops. These
also occur when the loop exit branch is exit on true because SCEV inverses the
icmp predicate.
Use it again to handle non-zero strides, but only with an unsigned comparison
in the exit condition.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@59528 91177308-0d34-0410-b5e6-96231b3b80d8
If this patch causes a performance regression for anyone, please let me know,
and it can be fixed in a different way with much more effort.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@59384 91177308-0d34-0410-b5e6-96231b3b80d8
