Commit Graph

256 Commits

Author SHA1 Message Date
Chandler Carruth
8677f2ff9a [Modules] Remove potential ODR violations by sinking the DEBUG_TYPE
define below all header includes in the lib/CodeGen/... tree. While the
current modules implementation doesn't check for this kind of ODR
violation yet, it is likely to grow support for it in the future. It
also removes one layer of macro pollution across all the included
headers.

Other sub-trees will follow.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206837 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-22 02:02:50 +00:00
Craig Topper
4ba844388c [C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206142 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-14 00:51:57 +00:00
Quentin Colombet
010311b104 [RegAllocGreedy][Last Chance Recoloring] Change the name of the exhaustive search option.
fexhaustive-register-search => exhaustive-register-search
'f' is a Clang thing!

This is related to PR18747.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206075 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-11 21:51:09 +00:00
Quentin Colombet
92a892e8f9 [RegAllocGreedy][Last Chance Recoloring] Addition of
-fexhaustive-register-search option to allow an exhaustive search during last
chance recoloring.

This is related to PR18747

Patch by MAYUR PANDEY <mayur.p@samsung.com>. 


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@206072 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-11 21:39:44 +00:00
Duncan P. N. Exon Smith
861e4db905 RegAlloc: Account for a variable entry block frequency
Until r197284, the entry frequency was constant -- i.e., set to 2^14.
Although current ToT still has a constant entry frequency, since r197284
that has been an implementation detail (which is soon going to change).

  - r204690 made the wrong assumption for the CSRCost metric.  Adjust
    callee-saved register cost based on entry frequency.

  - r185393 made the wrong assumption (although it was valid at the
    time).  Update SpillPlacement.cpp::Threshold to be relative to the
    entry frequency.

Since ToT still has 2^14 entry frequency, this should have no observable
functionality change.

<rdar://problem/14292693>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205789 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-08 19:18:56 +00:00
Quentin Colombet
3b2b5dfa9b [RegAllocGreedy][Last Chance Recoloring] Emit diagnostics when last chance
recoloring cut-offs are encountered and register allocation failed.

This is related to PR18747

Patch by MAYUR PANDEY <mayur.p@samsung.com>.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205601 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-04 02:05:21 +00:00
Quentin Colombet
cc99615837 Revert r205599, the commit was not intended to have so many changes
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205600 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-04 02:02:49 +00:00
Quentin Colombet
c65a77b92d [RegAllocGreedy][Last Chance Recoloring] Emit diagnostics when last chance
recoloring cut-offs are hit.

This is related to PR18747.

Patch by MAYUR PANDEY <mayur.p@samsung.com>


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@205599 91177308-0d34-0410-b5e6-96231b3b80d8
2014-04-04 01:58:57 +00:00
Manman Ren
d9524d66cd Provide a target override for the cost of using a callee-saved register
for the first time.

Thanks Andy for the discussion.
rdar://16162005


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204979 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-27 23:10:04 +00:00
Manman Ren
b5e6ddca56 Register Allocator: refactoring and add comments.
No functionality change. Thanks Andy for reviewing.

rdar://16162005


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204962 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-27 21:21:57 +00:00
Manman Ren
3d7d0bc71a Add comments. Addressing review comments from Evan on r204690.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204864 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-26 22:14:09 +00:00
Manman Ren
692d1830f3 Register Allocator: check other options before using a CSR for the first time.
When register allocator's stage is RS_Spill, we choose spill over using the CSR
for the first time, if the spill cost is lower than CSRCost. 
When register allocator's stage is < RS_Split, we choose pre-splitting over
using the CSR for the first time, if the cost of splitting is lower than
CSRCost.

CSRCost is set with command-line option "regalloc-csr-first-time-cost". The
default value is 0 to generate the same codes as before this commit.

With a value of 15 (1 << 14 is the entry frequency), I measured performance
gain of 3% on 253.perlbmk and 1.7% on 197.parser, with instrumented PGO,
on an arm device.

rdar://16162005


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204690 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-25 00:16:25 +00:00
Manman Ren
66124f9e89 Register Allocator: refactoring (no functionality change).
Factor out two functions calculateRegionSplitCost and doRegionSplit
from tryRegionSplit. These two functions will be used in coming patches.

rdar://16162005


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@204684 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-24 23:23:42 +00:00
Craig Topper
9f998de891 [C++11] Add 'override' keyword to virtual methods that override their base class.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@203220 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-07 09:26:03 +00:00
Ahmed Charles
f4ccd11075 Replace OwningPtr<T> with std::unique_ptr<T>.
This compiles with no changes to clang/lld/lldb with MSVC and includes
overloads to various functions which are used by those projects and llvm
which have OwningPtr's as parameters. This should allow out of tree
projects some time to move. There are also no changes to libs/Target,
which should help out of tree targets have time to move, if necessary.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@203083 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-06 05:51:42 +00:00
Benjamin Kramer
9efaf2f2da [C++11] Use std::tie to simplify compare operators.
No functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202751 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-03 19:58:30 +00:00
Alp Toker
18510b7e43 [C++11] Expand and eliminate the LLVM_ENUM_INT_TYPE() macro
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202607 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-02 03:20:38 +00:00
Andrew Trick
4c34f71b81 Provide a target override for the latest regalloc heuristic.
This is a temporary workaround for native arm linux builds:
PR18996: Changing regalloc order breaks "lencod" on native arm linux builds.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202433 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-27 21:37:33 +00:00
Andrew Trick
eaf8a32859 Add a limit to the heuristic that register allocates instructions in local order.
This handles pathological cases in which we see 2x increase in spill
code for large blocks (~50k instructions). I don't have a unit test
for this behavior.

Fixes rdar://16072279.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202304 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-26 22:07:26 +00:00
Manman Ren
d225511ea7 Remove outdated comments.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202186 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-25 19:47:15 +00:00
Alp Toker
bf930d5c1f Fix typos
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202107 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-25 04:21:15 +00:00
Quentin Colombet
126099ea23 [RegAlloc] Fix the assertion in the last chance recoloring to match the
condition at the call site.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201296 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-13 05:17:37 +00:00
Quentin Colombet
1a10a51431 [RegAlloc] Add a last chance recoloring mechanism when everything else failed to
find a register.

The idea is to choose a color for the variable that cannot be allocated and
recolor its interferences around. Unlike the current register allocation scheme,
it is allowed to change the color of an already assigned (but maybe not
splittable or spillable) live interval while propagating this change to its
neighbors.
In other word, there are two things that may help finding an available color:
- Already assigned variables (RS_Done) can be recolored to different color.
- The recoloring allows to catch solutions that needs to touch more that just
  the neighbors of the current allocated variable.

E.g.,
vA can use {R1, R2    }
vB can use {    R2, R3}
vC can use {R1        }
Where vA, vB, and vC cannot be split anymore (they are reloads for instance) and
they all interfere.

vA is assigned R1
vB is assigned R2
vC tries to evict vA but vA is already done.
=> Regular register allocation heuristic fails.

Last chance recoloring kicks in:
vC does as if vA was evicted => vC uses R1.
vC is marked as fixed.
vA needs to find a color.
None are available.
vA cannot evict vC: vC is a fixed virtual register now.
vA does as if vB was evicted => vA uses R2.
vB needs to find a color.
R3 is available.
Recoloring => vC = R1, vA = R2, vB = R3.

<rdar://problem/15947839>


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200883 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-05 22:13:59 +00:00
NAKAMURA Takumi
1a3b25343f RegAllocGreedy.cpp: Use more simple value as Hysteresis, to suppress -mfpmath-dependent behavior.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@200738 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-04 06:29:38 +00:00
Quentin Colombet
fb57392a8d [RegAlloc] Make tryInstructionSplit less aggressive.
The greedy register allocator tries to split a live-range around each
instruction where it is used or defined to relax the constraints on the entire
live-range (this is a last chance split before falling back to spill).
The goal is to have a big live-range that is unconstrained (i.e., that can use
the largest legal register class) and several small local live-range that carry
the constraints implied by each instruction.
E.g.,
Let csti be the constraints on operation i.

V1=
op1 V1(cst1)
op2 V1(cst2)

V1 live-range is constrained on the intersection of cst1 and cst2.

tryInstructionSplit relaxes those constraints by aggressively splitting each
def/use point:
V1=
V2 = V1
V3 = V2
op1 V3(cst1)
V4 = V2
op2 V4(cst2)

Because of how the coalescer infrastructure works, each new variable (V3, V4)
that is alive at the same time as V1 (or its copy, here V2) interfere with V1.
Thus, we end up with an uncoalescable copy for each split point.

To make tryInstructionSplit less aggressive, we check if the split point
actually relaxes the constraints on the whole live-range. If it does not, we do
not insert it.
Indeed, it will not help the global allocation problem:
- V1 will have the same constraints.
- V1 will have the same interference + possibly the newly added split variable
  VS.
- VS will produce an uncoalesceable copy if alive at the same time as V1.

<rdar://problem/15570057>


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@198369 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-02 22:47:22 +00:00
Michael Gottesman
523823b897 [block-freq] Rename getEntryFrequency() -> getEntryFreq() to match getBlockFreq() in all *BlockFrequencyInfo*.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@197304 91177308-0d34-0410-b5e6-96231b3b80d8
2013-12-14 02:37:38 +00:00
Michael Gottesman
c1b4dfd345 [block-freq] Update MachineBlockPlacement and RegAllocGreedy to use the new MachineBlockFrequencyInfo methods.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@197290 91177308-0d34-0410-b5e6-96231b3b80d8
2013-12-14 00:25:45 +00:00
Andrew Trick
2e3f799f01 Add TargetRegisterInfo::reverseLocalAssignment hook.
This hook reverses the order of assignment for local live ranges. This
will generally allocate shorter local live ranges first. For targets with
many registers, this could reduce regalloc compile time by a large
factor. It should still achieve optimal coloring; however, it can change
register eviction decisions. It is disabled by default for two reasons:
(1) Top-down allocation is simpler and easier to debug for targets that
don't benefit from reversing the order.
(2) Bottom-up allocation could result in poor evicition decisions on some
targets affecting the performance of compiled code.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@197001 91177308-0d34-0410-b5e6-96231b3b80d8
2013-12-11 03:40:15 +00:00
Aditya Nandakumar
226e3eab9b Check hint registers for interference only once before evictions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@196536 91177308-0d34-0410-b5e6-96231b3b80d8
2013-12-05 21:18:40 +00:00
Andrew Trick
4dc7377b20 Reverse the order of eviction checks for possible compile time savings. No functionality.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195969 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-29 23:49:38 +00:00
Andrew Trick
09f1b252af DEBUG shouldEvict decisions
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195490 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-22 19:07:42 +00:00
Andrew Trick
833a9e9395 Minor cleanup. EvictionCost ctor was confusing relative to the other costs floating around in the code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195489 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-22 19:07:38 +00:00
Aditya Nandakumar
3da011f561 Fixed an extra for(typo) in the comments
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195171 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-19 23:51:32 +00:00
Aaron Ballman
eb36024720 Replacing HUGE_VALF with llvm::huge_valf in order to work around a warning triggered in MSVC 12.
Patch reviewed by Reid Kleckner and Jim Grosbach.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194533 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-13 00:15:44 +00:00
Arnaud A. de Grandmaison
095f994ba6 CalcSpillWeights: give a better describing name to calculateSpillWeights
Besides, this relates it more obviously to the VirtRegAuxInfo::calculateSpillWeightAndHint.

No functionnal change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194404 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-11 19:04:45 +00:00
Arnaud A. de Grandmaison
a77da0579b CalculateSpillWeights does not need to be a pass
Based on discussions with Lang Hames and Jakob Stoklund Olesen at the hacker's lab, and in the light of upcoming work on the PBQP register allocator, it was though that CalcSpillWeights does not need to be a pass. This change will enable to customize / tune the spill weight computation depending on the allocator.

Update the documentation style while there.

No functionnal change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194356 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-10 17:46:31 +00:00
Arnaud A. de Grandmaison
d241fa7a61 Revert "CalculateSpillWeights does not need to be a pass"
Temporarily revert my previous commit until I understand why it breaks 3 target tests.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194272 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-08 18:19:19 +00:00
Arnaud A. de Grandmaison
663fcde3d3 CalculateSpillWeights does not need to be a pass
Based on discussions with Lang Hames and Jakob Stoklund Olesen at the hacker's lab, and in the light of upcoming work on the PBQP register allocator, it was though that CalcSpillWeights does not need to be a pass. This change will enable to customize / tune the spill weight computation depending on the allocator.

Update the documentation style while there.

No functionnal change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194269 91177308-0d34-0410-b5e6-96231b3b80d8
2013-11-08 17:56:29 +00:00
Matthias Braun
4f3b5e8c92 Represent RegUnit liveness with LiveRange instance
Previously LiveInterval has been used, but having a spill weight and
register number is unnecessary for a register unit.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192397 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-10 21:29:02 +00:00
Reid Kleckner
73f615b0bd Explicitly request unsigned enum types when desired
This fixes repeated -Wmicrosoft warnings when self-hosting clang on
Windows, and gets us real unsigned enum types with MSVC.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192227 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-08 20:15:11 +00:00
Eli Friedman
ae43dac300 Fix unused variables.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190448 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-10 23:18:14 +00:00
Mark Lacey
1feb5854ae Track new virtual registers by register number.
Track new virtual registers by register number, rather than by the live
interval created for them. This is the first step in separating the
creation of new virtual registers and new live intervals.  Eventually
live intervals will be created and populated on demand after the virtual
registers have been created and used in instructions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188434 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-14 23:50:04 +00:00
Andrew Trick
c0173e6f9f Down-scale slot index distance to save bits.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187438 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-30 19:59:19 +00:00
Andrew Trick
bef4c3e069 RegAllocGreedy comment.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187141 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-25 18:35:22 +00:00
Andrew Trick
8adae96fd9 Evict local live ranges if they can be reassigned.
The previous change to local live range allocation also suppressed
eviction of local ranges. In rare cases, this could result in more
expensive register choices. This commit actually revives a feature
that I added long ago: check if live ranges can be reassigned before
eviction. But now it only happens in rare cases of evicting a local
live range because another local live range wants a cheaper register.

The benefit is improved code size for some benchmarks on x86 and armv7.

I measured no significant compile time increase and performance
changes are noise.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187140 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-25 18:35:19 +00:00
Andrew Trick
6ea2b9608a Allocate local registers in order for optimal coloring.
Also avoid locals evicting locals just because they want a cheaper register.

Problem: MI Sched knows exactly how many registers we have and assumes
they can be colored. In cases where we have large blocks, usually from
unrolled loops, greedy coloring fails. This is a source of
"regressions" from the MI Scheduler on x86. I noticed this issue on
x86 where we have long chains of two-address defs in the same live
range. It's easy to see this in matrix multiplication benchmarks like
IRSmk and even the unit test misched-matmul.ll.

A fundamental difference between the LLVM register allocator and
conventional graph coloring is that in our model a live range can't
discover its neighbors, it can only verify its neighbors. That's why
we initially went for greedy coloring and added eviction to deal with
the hard cases. However, for singly defined and two-address live
ranges, we can optimally color without visiting neighbors simply by
processing the live ranges in instruction order.

Other beneficial side effects:

It is much easier to understand and debug regalloc for large blocks
when the live ranges are allocated in order. Yes, global allocation is
still very confusing, but it's nice to be able to comprehend what
happened locally.

Heuristics could be added to bias register assignment based on
instruction locality (think late register pairing, banks...).

Intuituvely this will make some test cases that are on the threshold
of register pressure more stable.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187139 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-25 18:35:14 +00:00
Andrew Trick
5dca613978 Dump LIS before regalloc. MI sched changes them.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187107 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-25 07:26:26 +00:00
Jakob Stoklund Olesen
03ef600623 Remove floats from live range splitting costs.
These floats all represented block frequencies anyway, so just use the
BlockFrequency class directly.

Some floating point computations remain in tryLocalSplit(). They are
estimating spill weights which are still floats.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186435 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-16 18:26:18 +00:00
Benjamin Kramer
4eed756153 Switch spill weights from a basic loop depth estimation to BlockFrequencyInfo.
The main advantages here are way better heuristics, taking into account not
just loop depth but also __builtin_expect and other static heuristics and will
eventually learn how to use profile info. Most of the work in this patch is
pushing the MachineBlockFrequencyInfo analysis into the right places.

This is good for a 5% speedup on zlib's deflate (x86_64), there were some very
unfortunate spilling decisions in its hottest loop in longest_match(). Other
benchmarks I tried were mostly neutral.

This changes register allocation in subtle ways, update the tests for it.
2012-02-20-MachineCPBug.ll was deleted as it's very fragile and the instruction
it looked for was gone already (but the FileCheck pattern picked up unrelated
stuff).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184105 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-17 19:00:36 +00:00
David Blaikie
453f4f0130 Use only explicit bool conversion operators
BitVector/SmallBitVector::reference::operator bool remain implicit since
they model more exactly a bool, rather than something else that can be
boolean tested.

The most common (non-buggy) case are where such objects are used as
return expressions in bool-returning functions or as boolean function
arguments. In those cases I've used (& added if necessary) a named
function to provide the equivalent (or sometimes negative, depending on
convenient wording) test.

One behavior change (YAMLParser) was made, though no test case is
included as I'm not sure how to reach that code path. Essentially any
comparison of llvm::yaml::document_iterators would be invalid if neither
iterator was at the end.

This helped uncover a couple of bugs in Clang - test cases provided for
those in a separate commit along with similar changes to `operator bool`
instances in Clang.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@181868 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-15 07:36:59 +00:00