When a live interval is being spilled, rather than creating short, non-spillable
intervals for every def / use, split the interval at BB boundaries. That is, for
every BB where the live interval is defined or used, create a new interval that
covers all the defs and uses in the BB.
This is designed to eliminate one common problem: multiple reloads of the same
value in a single basic block. Note, it does *not* decrease the number of spills
since no copies are inserted so the split intervals are *connected* through
spill and reloads (or rematerialization). The newly created intervals can be
spilled again, in that case, since it does not span multiple basic blocks, it's
spilled in the usual manner. However, it can reuse the same stack slot as the
previously split interval.
This is currently controlled by -split-intervals-at-bb.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@44198 91177308-0d34-0410-b5e6-96231b3b80d8
MachineOperand auxInfo. Previous clunky implementation uses an external map
to track sub-register uses. That works because register allocator uses
a new virtual register for each spilled use. With interval splitting (coming
soon), we may have multiple uses of the same register some of which are
of using different sub-registers from others. It's too fragile to constantly
update the information.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@44104 91177308-0d34-0410-b5e6-96231b3b80d8
Turn a store folding instruction into a load folding instruction. e.g.
xorl %edi, %eax
movl %eax, -32(%ebp)
movl -36(%ebp), %eax
orl %eax, -32(%ebp)
=>
xorl %edi, %eax
orl -36(%ebp), %eax
mov %eax, -32(%ebp)
This enables the unfolding optimization for a subsequent instruction which will
also eliminate the newly introduced store instruction.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@43192 91177308-0d34-0410-b5e6-96231b3b80d8
Turn this:
movswl %ax, %eax
movl %eax, -36(%ebp)
xorl %edi, -36(%ebp)
into
movswl %ax, %eax
xorl %edi, %eax
movl %eax, -36(%ebp)
by unfolding the load / store xorl into an xorl and a store when we know the
value in the spill slot is available in a register. This doesn't change the
number of instructions but reduce the number of times memory is accessed.
Also unfold some load folding instructions and reuse the value when similar
situation presents itself.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@42947 91177308-0d34-0410-b5e6-96231b3b80d8
(almost) a register copy. However, it always coalesced to the register of the
RHS (the super-register). All uses of the result of a EXTRACT_SUBREG are sub-
register uses which adds subtle complications to load folding, spiller rewrite,
etc.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@42899 91177308-0d34-0410-b5e6-96231b3b80d8
- If the defs of a spilled rematerializable MI are dead after the spill store is deleted, delete
the def MI as well.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@41086 91177308-0d34-0410-b5e6-96231b3b80d8
no more uses within the MBB and the spilled value isn't live out of the MBB.
Then it's safe to delete the spill store.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@41069 91177308-0d34-0410-b5e6-96231b3b80d8
with a general target hook to identify rematerializable instructions. Some
instructions are only rematerializable with specific operands, such as loads
from constant pools, while others are always rematerializable. This hook
allows both to be identified as being rematerializable with the same
mechanism.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@37644 91177308-0d34-0410-b5e6-96231b3b80d8
The code sequence before the spiller is something like:
= tMOVrr
%reg1117 = tMOVrr
%reg1078 = tLSLri %reg1117, 2
The it starts spilling:
%r0 = tRestore <fi#5>, 0
%r1 = tRestore <fi#7>, 0
%r1 = tMOVrr %r1<kill>
tSpill %r1, <fi#5>, 0
%reg1078 = tLSLri %reg1117, 2
It restores the value while processing the first tMOVrr. At this point, the
spiller remembers fi#5 is available in %r0. Next it processes the second move.
It restores the source before the move and spills the result afterwards. The
move becomes a noop and is deleted. However, a spill has been inserted and that
should invalidate reuse of %r0 for fi#5 and add reuse of %r1 for fi#5.
Therefore, %reg1117 (which is also assigned fi#5) should get %r1, not %r0.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@34039 91177308-0d34-0410-b5e6-96231b3b80d8
t1 := op t2, t3
t2 <- assigned r0 for use by the reload but ended up reuse r1
t3 <- assigned r1 for use by the reload but ended up reuse r0
t1 <- desires r1
sees r1 is taken by t2, tries t2's reload register r0
sees r0 is taken by t3, tries t3's reload register r1
sees r1 is taken by t2, tries t2's reload register r0 ...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@33382 91177308-0d34-0410-b5e6-96231b3b80d8
rework the hacks that had us passing OStream in. We pass in std::ostream*
instead, check for null, and then dispatch to the correct print() method.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@32636 91177308-0d34-0410-b5e6-96231b3b80d8
If a spillslot value is available in a register, and there is a noop copy that
targets that register, the spiller correctly decide not to invalidate the
spillslot register.
However, even though the noop copy does not clobbers the value. It does start a
new intersecting live range. That means the spillslot register is available for
use but should not be reused for a two-address instruction modref operand which
would clobber the new live range.
When we remove the noop copy, update the available information by clearing the
canClobber bit.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@32576 91177308-0d34-0410-b5e6-96231b3b80d8
tied to another oeprand, 2) whether is is being tied to by another operand. So
the destination operand of a two-address MI can be correctly identified.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@32354 91177308-0d34-0410-b5e6-96231b3b80d8
there may be other def(s) apart from the use&def two-address operand. We need
to check if the register reuse for a use&def operand may conflicts with another
def. Provide a mean to recover from the conflict if it is detected when the
defs are processed later.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@31439 91177308-0d34-0410-b5e6-96231b3b80d8
actually *removes* one of the operands, instead of just assigning both operands
the same register. This make reasoning about instructions unnecessarily complex,
because you need to know if you are before or after register allocation to match
up operand #'s with the target description file.
Changing this also gets rid of a bunch of hacky code in various places.
This patch also includes changes to fold loads into cmp/test instructions in
the X86 backend, along with a significant simplification to the X86 spill
folding code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@30108 91177308-0d34-0410-b5e6-96231b3b80d8
instructions which define each value#) to simplify and improve the coallescer.
In particular, this patch:
1. Implements iterative coallescing.
2. Reverts an unsafe hack from handlePhysRegDef, superceeding it with a
better solution.
3. Implements PR865, "coallescing" away the second copy in code like:
A = B
...
B = A
This also includes changes to symbolically print registers in intervals
when possible.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@29862 91177308-0d34-0410-b5e6-96231b3b80d8
instructions in the virtregfolded map that were deleted. Because they
were deleted, newly allocated instructions could end up at the same address,
magically finding themselves in the map. The solution is to remove entries
from the map when we delete the instructions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@28041 91177308-0d34-0410-b5e6-96231b3b80d8
But this is incorrect if the spilled value live range extends beyond the
current BB.
It is currently controlled by a temporary option -spiller-check-liveout.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@28024 91177308-0d34-0410-b5e6-96231b3b80d8
and is already available, instead of falling back to emitting a load, fall
back to emitting a reg-reg copy. This generates significantly better code
for some SSE testcases, as SSE has lots of two-address instructions and
none of them are read/modify/write. As one example, this change does:
pshufd %XMM5, XMMWORD PTR [%ESP + 84], 255
xorps %XMM2, %XMM5
cmpltps %XMM1, %XMM0
- movaps XMMWORD PTR [%ESP + 52], %XMM0
- movapd %XMM6, XMMWORD PTR [%ESP + 52]
+ movaps %XMM6, %XMM0
cmpltps %XMM6, XMMWORD PTR [%ESP + 68]
movapd XMMWORD PTR [%ESP + 52], %XMM6
movaps %XMM6, %XMM0
cmpltps %XMM6, XMMWORD PTR [%ESP + 36]
cmpltps %XMM3, %XMM0
- movaps XMMWORD PTR [%ESP + 20], %XMM0
- movapd %XMM7, XMMWORD PTR [%ESP + 20]
+ movaps %XMM7, %XMM0
cmpltps %XMM7, XMMWORD PTR [%ESP + 4]
movapd XMMWORD PTR [%ESP + 20], %XMM7
cmpltps %XMM4, %XMM0
... which is far better than a store followed by a load!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@28001 91177308-0d34-0410-b5e6-96231b3b80d8
exposed with a fastcc problem (breaking pcompress2 on x86 with -enable-x86-fastcc).
When reloading a reused reg, make sure to invalidate the reloaded reg, and
check to see if there are any other pending uses of the same register.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@26369 91177308-0d34-0410-b5e6-96231b3b80d8
This gets rid of two gotos, which is always nice, and also adds some comments.
No functionality change, this is just a refactor.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@26367 91177308-0d34-0410-b5e6-96231b3b80d8
store EAX -> [ss#0]
[ss#0] += 1
...
use(EAX)
In this case, it is not valid to rewrite this as:
store EAX -> [ss#0]
EAX += 1
store EAX -> [ss#0] ;;; this would also delete the store above
...
use(EAX)
... because EAX is not a dead at that point. Keep track of which registers
we are allowed to clobber, and which ones we aren't, and don't clobber the
ones we're not supposed to. :)
This should resolve the issues on X86 last night.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@25948 91177308-0d34-0410-b5e6-96231b3b80d8
and PhysRegsAvailable maps out into a new AvailableSpills struct. No
functionality change.
This paves the way for a bugfix, coming up next.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@25947 91177308-0d34-0410-b5e6-96231b3b80d8
1. a target doesn't know how to fold load/stores into copies, or
2. the spiller rewrites the input to a copy to the same register as the dest
instead of to the reloaded reg.
This will be moved/improved in the near future, but allows elimination of
some ancient x86 hacks. This eliminates 92 copies from SMG2000 on X86 and
163 copies from 252.eon.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@25922 91177308-0d34-0410-b5e6-96231b3b80d8
of this, and use it to our advantage (bwahahah). This allows us to eliminate another
60 instructions from smg2000 on PPC (probably significantly more on X86). A common
old-new diff looks like this:
stw r2, 3304(r1)
- lwz r2, 3192(r1)
stw r2, 3300(r1)
- lwz r2, 3192(r1)
stw r2, 3296(r1)
- lwz r2, 3192(r1)
stw r2, 3200(r1)
- lwz r2, 3192(r1)
stw r2, 3196(r1)
- lwz r2, 3192(r1)
+ or r2, r2, r2
stw r2, 3188(r1)
and
- lwz r31, 604(r1)
- lwz r13, 604(r1)
- lwz r14, 604(r1)
- lwz r15, 604(r1)
- lwz r16, 604(r1)
- lwz r30, 604(r1)
+ or r31, r30, r30
+ or r13, r30, r30
+ or r14, r30, r30
+ or r15, r30, r30
+ or r16, r30, r30
+ or r30, r30, r30
Removal of the R = R copies is coming next...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@25919 91177308-0d34-0410-b5e6-96231b3b80d8
this code:
store [stack slot #0], R10
= add R14, [stack slot #0]
The spiller didn't know that the store made the value of [stackslot#0] available
in R10 *IF* the store came from a copy instruction with the store folded into it.
This patch teaches VirtRegMap to look at these stores and recognize the values
they make available. In one case Evan provided, this code:
divsd %XMM0, %XMM1
movsd %XMM1, QWORD PTR [%ESP + 40]
1) movsd QWORD PTR [%ESP + 48], %XMM1
2) movsd %XMM1, QWORD PTR [%ESP + 48]
addsd %XMM1, %XMM0
3) movsd QWORD PTR [%ESP + 48], %XMM1
movsd QWORD PTR [%ESP + 4], %XMM0
turns into:
divsd %XMM0, %XMM1
movsd %XMM1, QWORD PTR [%ESP + 40]
addsd %XMM1, %XMM0
3) movsd QWORD PTR [%ESP + 48], %XMM1
movsd QWORD PTR [%ESP + 4], %XMM0
In this case, instruction #2 was removed because of the value made
available by #1, and inst #1 was later deleted because it is now
never used before the stack slot is redefined by #3.
This occurs here and there in a lot of code with high spilling, on PPC
most of the removed loads/stores are LSU-reject-causing loads, which is
nice.
On X86, things are much better (because it spills more), where we nuke
about 1% of the instructions from SMG2000 and several hundred from eon.
More improvements to come...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@25917 91177308-0d34-0410-b5e6-96231b3b80d8
previous copy elisions and we discover we need to reload a register, make
sure to use the regclass of the original register for the reload, not the
class of the current register. This avoid using 16-bit loads to reload 32-bit
values.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@23645 91177308-0d34-0410-b5e6-96231b3b80d8
store r12 -> [ss#2]
R3 = load [ss#1]
use R3
R3 = load [ss#2]
R4 = load [ss#1]
and turn it into this code:
store R12 -> [ss#2]
R3 = load [ss#1]
use R3
R3 = R12
R4 = R3 <- oops!
The problem was that promoting R3 = load[ss#2] to a copy missed the fact that
the instruction invalidated R3 at that point.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@23638 91177308-0d34-0410-b5e6-96231b3b80d8
when possible, avoiding the load (and avoiding the copy if the value is already
in the right register).
This patch came about when I noticed code like the following being generated:
store R17 -> [SS1]
...blah...
R4 = load [SS1]
This was causing an LSU reject on the G5. This problem was due to the register
allocator folding spill code into a reg-reg copy (producing the load), which
prevented the spiller from being able to rewrite the load into a copy, despite
the fact that the value was already available in a register. In the case
above, we now rip out the R4 load and replace it with a R4 = R17 copy.
This speeds up several programs on X86 (which spills a lot :) ), e.g.
smg2k from 22.39->20.60s, povray from 12.93->12.66s, 168.wupwise from
68.54->53.83s (!), 197.parser from 7.33->6.62s (!), etc. This may have a larger
impact in some cases on the G5 (by avoiding LSU rejects), though it probably
won't trigger as often (less spilling in general).
Targets that implement folding of loads/stores into copies should implement
the isLoadFromStackSlot hook to get this.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@23388 91177308-0d34-0410-b5e6-96231b3b80d8
