Chris claims we should never have visibility_hidden inside any .cpp file but
that's still not true even after this commit.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@85042 91177308-0d34-0410-b5e6-96231b3b80d8
basic blocks that are so long that their size overflows a short.
Also assert that overflow does not happen in the future, as requested by Evan.
This fixes PR4401.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@83159 91177308-0d34-0410-b5e6-96231b3b80d8
This adds location info for all llvm_unreachable calls (which is a macro now) in
!NDEBUG builds.
In NDEBUG builds location info and the message is off (it only prints
"UREACHABLE executed").
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@75640 91177308-0d34-0410-b5e6-96231b3b80d8
Make llvm_unreachable take an optional string, thus moving the cerr<< out of
line.
LLVM_UNREACHABLE is now a simple wrapper that makes the message go away for
NDEBUG builds.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@75379 91177308-0d34-0410-b5e6-96231b3b80d8
Massive check in. This changes the "-fast" flag to "-O#" in llc. If you want to
use the old behavior, the flag is -O0. This change allows for finer-grained
control over which optimizations are run at different -O levels.
Most of this work was pretty mechanical. The majority of the fixes came from
verifying that a "fast" variable wasn't used anymore. The JIT still uses a
"Fast" flag. I'll change the JIT with a follow-up patch.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@70343 91177308-0d34-0410-b5e6-96231b3b80d8
use the old behavior, the flag is -O0. This change allows for finer-grained
control over which optimizations are run at different -O levels.
Most of this work was pretty mechanical. The majority of the fixes came from
verifying that a "fast" variable wasn't used anymore. The JIT still uses a
"Fast" flag. I'm not 100% sure if it's necessary to change it there...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@70270 91177308-0d34-0410-b5e6-96231b3b80d8
add dependencies on nodes with exactly one successor which is a
COPY_TO_REGCLASS node. In the case that the copy is coalesced
away, the dependence should be on the user of the copy, rather
than the copy itself.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@69309 91177308-0d34-0410-b5e6-96231b3b80d8
help out the register pressure reduction heuristics in the case of
nodes with multiple uses. Currently this uses very conservative
heuristics, so it doesn't have a broad impact, but in cases where it
does help it can make a big difference.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@67586 91177308-0d34-0410-b5e6-96231b3b80d8
a data dependency on the load node, so it really needs a
data-dependence edge to the load node, even if the load previously
existed.
And add a few comments.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@67554 91177308-0d34-0410-b5e6-96231b3b80d8
with multiple chain operands. This can occur when the scheduler
has added chain operands to a node that already has a chain
operand, in order to handle physical register dependencies.
This fixes an llvm-gcc bootstrap failure on x86-64 introduced
in r66058.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@66240 91177308-0d34-0410-b5e6-96231b3b80d8
unless they actually have data successors, and likewise for nodes
with no data successors unless they actually have data precessors.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@64327 91177308-0d34-0410-b5e6-96231b3b80d8
is determined by whether the node has a Flag operand. However, if the
node does have a Flag operand, it will be glued to its register's
def, so the heuristic would end up spuriously applying to whatever
node is the def.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@64319 91177308-0d34-0410-b5e6-96231b3b80d8
instruction index across each part. Instruction indices are used
to make live range queries, and live ranges can extend beyond
scheduling region boundaries.
Refactor the ScheduleDAGSDNodes class some more so that it
doesn't have to worry about this additional information.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@64288 91177308-0d34-0410-b5e6-96231b3b80d8
scheduling, and generalize is so that preserves state across
scheduling regions. This fixes incorrect live-range information around
terminators and labels, which are effective region boundaries.
In place of looking for terminators to anchor inter-block dependencies,
introduce special entry and exit scheduling units for this purpose.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@64254 91177308-0d34-0410-b5e6-96231b3b80d8
and into the ScheduleDAGInstrs class, so that they don't get
destructed and re-constructed for each block. This fixes a
compile-time hot spot in the post-pass scheduler.
To help facilitate this, tidy and do some minor reorganization
in the scheduler constructor functions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@62275 91177308-0d34-0410-b5e6-96231b3b80d8
scheduling dependencies. Add assertion checks to help catch
this.
It appears the Mips target defaults to list-td, and it has a
regression test that uses a physreg dependence. Such code was
liable to be miscompiled, and now evokes an assertion failure.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@62177 91177308-0d34-0410-b5e6-96231b3b80d8
Also future proof the scheduler to handle "normal" physical register dependencies. The code is not exercised yet.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@62074 91177308-0d34-0410-b5e6-96231b3b80d8
AddPseudoTwoAddrDeps. This lets the scheduling infrastructure
avoid recalculating node heights. In very large testcases this
was a major bottleneck. Thanks to Roman Levenstein for finding
this!
As a side effect, fold-pcmpeqd-0.ll is now scheduled better
and it no longer requires spilling on x86-32.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@61778 91177308-0d34-0410-b5e6-96231b3b80d8
instructions to avoid copies, because TwoAddressInstructionPass
also does this optimization. The scheduler's version didn't
account for live-out values, which resulted in spurious commutes
and missed opportunities.
Now, TwoAddressInstructionPass handles all the opportunities,
instead of just those that the scheduler missed. The result is
usually the same, though there are occasional trivial differences
resulting from the avoidance of spurious commutes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@61611 91177308-0d34-0410-b5e6-96231b3b80d8
computation code. Also, avoid adding output-depenency edges when both
defs are dead, which frequently happens with EFLAGS defs.
Compute Depth and Height lazily, and always in terms of edge latency
values. For the schedulers that don't care about latency, edge latencies
are set to 1.
Eliminate Cycle and CycleBound, and LatencyPriorityQueue's Latencies array.
These are all subsumed by the Depth and Height fields.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@61073 91177308-0d34-0410-b5e6-96231b3b80d8
The Cost field is removed. It was only being used in a very limited way,
to indicate when the scheduler should attempt to protect a live register,
and it isn't really needed to do that. If we ever want the scheduler to
start inserting copies in non-prohibitive situations, we'll have to
rethink some things anyway.
A Latency field is added. Instead of giving each node a single
fixed latency, each edge can have its own latency. This will eventually
be used to model various micro-architecture properties more accurately.
The PointerIntPair class and an internal union are now used, which
reduce the overall size.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@60806 91177308-0d34-0410-b5e6-96231b3b80d8
introduce any new spilling; it just uses unused registers.
Refactor the SUnit topological sort code out of the RRList scheduler and
make use of it to help with the post-pass scheduler.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@59999 91177308-0d34-0410-b5e6-96231b3b80d8
schedulers. This doesn't have much immediate impact because
targets that use these schedulers by default don't yet provide
pipeline information.
This code also didn't have the benefit of register pressure
information. Also, removing it will avoid problems with list-burr
suddenly starting to do latency-oriented scheduling on x86 when we
start providing pipeline data, which would increase spilling.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@59775 91177308-0d34-0410-b5e6-96231b3b80d8
the RR scheduler actually does look at latency values, but it
doesn't use a hazard recognizer so it has no way to know when
a no-op is needed, as opposed to just stalling and incrementing
the cycle count.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@59759 91177308-0d34-0410-b5e6-96231b3b80d8
dedicated "fast" scheduler in -fast mode instead, which is
faster. This speeds up llc -fast by a few percent on some
testcases -- the speedup only happens for code not handled by
fast-isel.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@59700 91177308-0d34-0410-b5e6-96231b3b80d8
is currently off by default, and can be enabled with
-disable-post-RA-scheduler=false.
This doesn't have a significant impact on most code yet because it doesn't
yet do anything to address anti-dependencies and it doesn't attempt to
disambiguate memory references. Also, several popular targets
don't have pipeline descriptions yet.
The majority of the changes here are splitting the SelectionDAG-specific
code out of ScheduleDAG, so that ScheduleDAG can be moved to
libLLVMCodeGen.a. The interface between ScheduleDAG-using code and
the rest of the scheduling code is somewhat rough and will evolve.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@59676 91177308-0d34-0410-b5e6-96231b3b80d8
new CycleBound value. Instead, just update CycleBound on each call.
Also, make ReleasePred and ReleaseSucc methods more consistent accross
the various schedulers.
This also happens to make ScheduleDAGRRList's CycleBound computation
somewhat more interesting, though it still doesn't have any noticeable
effect, because no current targets that use the register-pressure
reduction scheduler provide pipeline models.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@59475 91177308-0d34-0410-b5e6-96231b3b80d8
to carry a SmallVector of flagged nodes, just calculate the flagged nodes
dynamically when they are needed.
The local-liveness change is due to a trivial scheduling change where
the scheduler arbitrary decision differently.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@59273 91177308-0d34-0410-b5e6-96231b3b80d8
before creating the SUnit for the operation that it was unfolded from. This
allows each SUnit to have all of its predecessor SUnits available at the time
it is created. I don't know yet if this will be absolutely required, but it
is a little tidier to do it this way.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@59083 91177308-0d34-0410-b5e6-96231b3b80d8
argument instead of taking the SelectionDAG's TargetMachine. This is
needed for some upcoming scheduler changes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@59055 91177308-0d34-0410-b5e6-96231b3b80d8
instead of requiring all "short description" strings to begin with
two spaces. This makes these strings less mysterious, and it fixes
some cases where short description strings mistakenly did not
begin with two spaces.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@57521 91177308-0d34-0410-b5e6-96231b3b80d8
of two, and to not need a scratch std::vector. Also, compute the ordering
immediately in the result array, instead of in another scratch std::vector
that is copied to the result array.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@55421 91177308-0d34-0410-b5e6-96231b3b80d8
replacement of multiple values. This is slightly more efficient
than doing multiple ReplaceAllUsesOfValueWith calls, and theoretically
could be optimized even further. However, an important property of this
new function is that it handles the case where the source value set and
destination value set overlap. This makes it feasible for isel to use
SelectNodeTo in many very common cases, which is advantageous because
SelectNodeTo avoids a temporary node and it doesn't require CSEMap
updates for users of values that don't change position.
Revamp MorphNodeTo, which is what does all the work of SelectNodeTo, to
handle operand lists more efficiently, and to correctly handle a number
of corner cases to which its new wider use exposes it.
This commit also includes a change to the encoding of post-isel opcodes
in SDNodes; now instead of being sandwiched between the target-independent
pre-isel opcodes and the target-dependent pre-isel opcodes, post-isel
opcodes are now represented as negative values. This makes it possible
to test if an opcode is pre-isel or post-isel without having to know
the size of the current target's post-isel instruction set.
These changes speed up llc overall by 3% and reduce memory usage by 10%
on the InstructionCombining.cpp testcase with -fast and -regalloc=local.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@53728 91177308-0d34-0410-b5e6-96231b3b80d8
use a timer group for the timers in SelectionDAGISel. Also,
Split scheduling out from emitting, to give each their own
timer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@53476 91177308-0d34-0410-b5e6-96231b3b80d8
clear() on each iteration. This avoids allocating and deallocating
all of DenseMap's memory on each iteration.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@52642 91177308-0d34-0410-b5e6-96231b3b80d8
and provides fairly efficient removal of arbitrary elements. Switch
ScheduleDAGRRList from std::set to this new priority queue.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@52582 91177308-0d34-0410-b5e6-96231b3b80d8
to DenseMap<SDNode*, SUnit*>, and adjust the way cloned SUnit nodes are
handled so that only the original node needs to be in the map.
This speeds up llc on 447.dealII.llvm.bc by about 2%.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@52576 91177308-0d34-0410-b5e6-96231b3b80d8
and better control the abstraction. Rename the type
to MVT. To update out-of-tree patches, the main
thing to do is to rename MVT::ValueType to MVT, and
rewrite expressions like MVT::getSizeInBits(VT) in
the form VT.getSizeInBits(). Use VT.getSimpleVT()
to extract a MVT::SimpleValueType for use in switch
statements (you will get an assert failure if VT is
an extended value type - these shouldn't exist after
type legalization).
This results in a small speedup of codegen and no
new testsuite failures (x86-64 linux).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@52044 91177308-0d34-0410-b5e6-96231b3b80d8
This removes the existing bottleneck related to the removal of elements from
the middle of the queue.
Also fixes a subtle bug in ScheduleDAGRRList::CapturePred:
It was updating the state of the SUnit before removing it. As a result, the
comparison operators were working incorrectly and this SUnit could not be removed
from the queue properly.
Reviewed by Evan and Dan. Approved by Dan.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@50412 91177308-0d34-0410-b5e6-96231b3b80d8
ScheduleDAG; they don't correspond to any actual instructions so they
don't need to be scheduled.
This fixes a bug where the EntryToken was being scheduled multiple
times in some cases, though it ended up not causing any trouble because
EntryToken doesn't expand into anything. With this fixed the schedulers
reliably schedule the expected number of units, so we can check this
with an assertion.
This requires a tweak to test/CodeGen/X86/loop-hoist.ll because it
ends up getting scheduled differently in a trivial way, though it was
enough to fool the prcontext+grep that the test does.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@49701 91177308-0d34-0410-b5e6-96231b3b80d8
called LimitedSumOfUnscheduledPredsOfSuccs. It terminates the computation
after a given treshold is reached. This new function is always faster, but
brings real wins only on bigger test-cases.
The old function SumOfUnscheduledPredsOfSuccs is left in-place for now and therefore a warning about an unused static function is produced.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@48872 91177308-0d34-0410-b5e6-96231b3b80d8
This fixes Bugzilla #1835 (http://llvm.org/bugs/show_bug.cgi?id=1835).
This patched is reviewed by Tanya and Dan. Dan tested and approved it.
The reason for the bad performance of the old algorithm is that it is very naive and scans every
time all nodes of the DAG in the worst case.
This patch introduces a new algorithm based on the paper "Online algorithms
for maintaining the topological order of a directed acyclic graph" by
David J.Pearce and Paul H.J.Kelly. This is the MNR algorithm. It has a
linear time worst-case and performs much better in most situations.
The paper can be found here:
http://fano.ics.uci.edu/cites/Document/Online-algorithms-for-maintaining-the-topological-order-of-a-directed-acyclic-graph.html
The main idea of the new algorithm is to compute the topological ordering of the SNodes in the
DAG and to maintain it even after DAG modifications. The topological ordering allows for very fast
node reachability checks.
Tests on very big input files with tens of thousands of instructions in a BB indicate huge
speed-ups (up to 10x compilation time improvement) compared to the old version.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@48817 91177308-0d34-0410-b5e6-96231b3b80d8
all clients over to using predicates instead of these flags directly.
These are now private values which are only to be used to statically
initialize the tables.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@45692 91177308-0d34-0410-b5e6-96231b3b80d8
that "machine" classes are used to represent the current state of
the code being compiled. Given this expanded name, we can start
moving other stuff into it. For now, move the UsedPhysRegs and
LiveIn/LoveOuts vectors from MachineFunction into it.
Update all the clients to match.
This also reduces some needless #includes, such as MachineModuleInfo
from MachineFunction.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@45467 91177308-0d34-0410-b5e6-96231b3b80d8
take a deleted nodes vector, instead of requiring it.
One more significant change: Implement the start of a legalizer that
just works on types. This legalizer is designed to run before the
operation legalizer and ensure just that the input dag is transformed
into an output dag whose operand and result types are all legal, even
if the operations on those types are not.
This design/impl has the following advantages:
1. When finished, this will *significantly* reduce the amount of code in
LegalizeDAG.cpp. It will remove all the code related to promotion and
expansion as well as splitting and scalarizing vectors.
2. The new code is very simple, idiomatic, and modular: unlike
LegalizeDAG.cpp, it has no 3000 line long functions. :)
3. The implementation is completely iterative instead of recursive, good
for hacking on large dags without blowing out your stack.
4. The implementation updates nodes in place when possible instead of
deallocating and reallocating the entire graph that points to some
mutated node.
5. The code nicely separates out handling of operations with invalid
results from operations with invalid operands, making some cases
simpler and easier to understand.
6. The new -debug-only=legalize-types option is very very handy :),
allowing you to easily understand what legalize types is doing.
This is not yet done. Until the ifdef added to SelectionDAGISel.cpp is
enabled, this does nothing. However, this code is sufficient to legalize
all of the code in 186.crafty, olden and freebench on an x86 machine. The
biggest issues are:
1. Vectors aren't implemented at all yet
2. SoftFP is a mess, I need to talk to Evan about it.
3. No lowering to libcalls is implemented yet.
4. Various operations are missing etc.
5. There are FIXME's for stuff I hax0r'd out, like softfp.
Hey, at least it is a step in the right direction :). If you'd like to help,
just enable the #ifdef in SelectionDAGISel.cpp and compile code with it. If
this explodes it will tell you what needs to be implemented. Help is
certainly appreciated.
Once this goes in, we can do three things:
1. Add a new pass of dag combine between the "type legalizer" and "operation
legalizer" passes. This will let us catch some long-standing isel issues
that we miss because operation legalization often obfuscates the dag with
target-specific nodes.
2. We can rip out all of the type legalization code from LegalizeDAG.cpp,
making it much smaller and simpler. When that happens we can then
reimplement the core functionality left in it in a much more efficient and
non-recursive way.
3. Once the whole legalizer is non-recursive, we can implement whole-function
selectiondags maybe...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@42981 91177308-0d34-0410-b5e6-96231b3b80d8
(almost) a register copy. However, it always coalesced to the register of the
RHS (the super-register). All uses of the result of a EXTRACT_SUBREG are sub-
register uses which adds subtle complications to load folding, spiller rewrite,
etc.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@42899 91177308-0d34-0410-b5e6-96231b3b80d8
scheduler will try a number of tricks in order to avoid generating the
copies. This may not be possible in case the node produces a chain value
that prevent movement. Try unfolding the load from the node before to allow
it to be moved / cloned.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@42625 91177308-0d34-0410-b5e6-96231b3b80d8
terminator) the one that has a CopyToReg use. This fixes
2006-05-11-InstrSched.ll with -new-cc-modeling-scheme.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@42453 91177308-0d34-0410-b5e6-96231b3b80d8
- Added ability to emit cross class register copies to the BBRU scheduler.
- More aggressive backtracking.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@42375 91177308-0d34-0410-b5e6-96231b3b80d8
ordering and thus violated the strict weak ordering requirement of
priority_queue. Uncovered by _GLIBCXX_DEBUG.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@37794 91177308-0d34-0410-b5e6-96231b3b80d8
it as a late BURR scheduling tie-breaker.
Intuitively, it's good to push down instructions whose results are liveout so
their long live ranges won't conflict with other values which are needed inside
the BB. Further prioritize liveout instructions by the number of operands which
are calculated within the BB.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@35109 91177308-0d34-0410-b5e6-96231b3b80d8
e.g.
t1 = op t2, c1
t3 = op t4, c2
and the following instructions are both ready.
t2 = op c3
t4 = op c4
Then schedule t2 = op first.
i.e.
t4 = op c4
t2 = op c3
t1 = op t2, c1
t3 = op t4, c2
This creates more short live intervals which work better with the register
allocator.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@35089 91177308-0d34-0410-b5e6-96231b3b80d8
- Fixed bugs in sethi-ullman number computation and priority queue comparison
functions.
- Separate code that handles priority computation special cases from SU number computation.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@33025 91177308-0d34-0410-b5e6-96231b3b80d8
Turn on -Wunused and -Wno-unused-parameter. Clean up most of the resulting
fall out by removing unused variables. Remaining warnings have to do with
unused functions (I didn't want to delete code without review) and unused
variables in generated code. Maintainers should clean up the remaining
issues when they see them. All changes pass DejaGnu tests and Olden.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@31380 91177308-0d34-0410-b5e6-96231b3b80d8
This reduces selectiondag time on kc++ from 5.43s to 4.98s (9%). More
significantly, this speeds up the default ppc scheduler from ~1571ms to 1063ms,
a 33% speedup.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@29743 91177308-0d34-0410-b5e6-96231b3b80d8
2. Added argument to instruction scheduler creators so the creators can do
special things.
3. Repaired target hazard code.
4. Misc.
More to follow.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@29450 91177308-0d34-0410-b5e6-96231b3b80d8
separate file. Added an initial implementation of top-down register pressure
reduction list scheduler.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@28226 91177308-0d34-0410-b5e6-96231b3b80d8
