Commit Graph

41 Commits

Author SHA1 Message Date
Craig Topper
6227d5c690 Use SmallVectorImpl::iterator/const_iterator instead of SmallVector to avoid specifying the vector size.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@185606 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-04 01:31:24 +00:00
Diego Novillo
fe808927ac Do not reserve space for the ColdEdges and NormalEdges vectors.
Discussion and rationale at
http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20130520/175698.html

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182653 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-24 17:00:22 +00:00
Diego Novillo
77226a03dc Add a new function attribute 'cold' to functions.
Other than recognizing the attribute, the patch does little else.
It changes the branch probability analyzer so that edges into
blocks postdominated by a cold function are given low weight.

Added analysis and code generation tests.  Added documentation for the
new attribute.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@182638 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-24 12:26:52 +00:00
Chandler Carruth
0b8c9a80f2 Move all of the header files which are involved in modelling the LLVM IR
into their new header subdirectory: include/llvm/IR. This matches the
directory structure of lib, and begins to correct a long standing point
of file layout clutter in LLVM.

There are still more header files to move here, but I wanted to handle
them in separate commits to make tracking what files make sense at each
layer easier.

The only really questionable files here are the target intrinsic
tablegen files. But that's a battle I'd rather not fight today.

I've updated both CMake and Makefile build systems (I think, and my
tests think, but I may have missed something).

I've also re-sorted the includes throughout the project. I'll be
committing updates to Clang, DragonEgg, and Polly momentarily.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171366 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-02 11:36:10 +00:00
Chandler Carruth
d04a8d4b33 Use the new script to sort the includes of every file under lib.
Sooooo many of these had incorrect or strange main module includes.
I have manually inspected all of these, and fixed the main module
include to be the nearest plausible thing I could find. If you own or
care about any of these source files, I encourage you to take some time
and check that these edits were sensible. I can't have broken anything
(I strictly added headers, and reordered them, never removed), but they
may not be the headers you'd really like to identify as containing the
API being implemented.

Many forward declarations and missing includes were added to a header
files to allow them to parse cleanly when included first. The main
module rule does in fact have its merits. =]

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169131 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-03 16:50:05 +00:00
Manman Ren
1a710fdde1 BranchProb: modify the definition of an edge in BranchProbabilityInfo to handle
the case of multiple edges from one block to another.

A simple example is a switch statement with multiple values to the same
destination. The definition of an edge is modified from a pair of blocks to
a pair of PredBlock and an index into the successors.

Also set the weight correctly when building SelectionDAG from LLVM IR,
especially when converting a Switch.
IntegersSubsetMapping is updated to calculate the weight for each cluster.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162572 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-24 18:14:27 +00:00
Bill Wendling
0c34ae88bf Set the branch probability of branching to the 'normal' destination of an invoke
instruction to something absurdly high, while setting the probability of
branching to the 'unwind' destination to the bare minimum. This should set cause
the normal destination's invoke blocks to be moved closer to the invoke.

PR13612


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@161944 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-15 12:22:35 +00:00
Chandler Carruth
51f40a725b Make the unreachable probability much much heavier. The previous
probability wouldn't be considered "hot" in some weird loop structures
or other compounding probability patterns. This makes it much harder to
confuse, but isn't really a principled fix. I'd actually like it if we
could model a zero probability, as it would make this much easier to
reason about. Suggestions for how to do this better are welcome.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147142 91177308-0d34-0410-b5e6-96231b3b80d8
2011-12-22 09:26:37 +00:00
Benjamin Kramer
a7b0cb7594 Remove all remaining uses of Value::getNameStr().
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@144648 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-15 16:27:03 +00:00
Chandler Carruth
45baf6bb85 Fix the API usage in loop probability heuristics. It was incorrectly
classifying many edges as exiting which were in fact not. These mainly
formed edges into sub-loops. It was also not correctly classifying all
returning edges out of loops as leaving the loop. With this match most
of the loop heuristics are more rational.

Several serious regressions on loop-intesive benchmarks like perlbench's
loop tests when built with -enable-block-placement are fixed by these
updated heuristics. Unfortunately they in turn uncover some other
regressions. There are still several improvemenst that should be made to
loop heuristics including trip-count, and early back-edge management.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@142917 91177308-0d34-0410-b5e6-96231b3b80d8
2011-10-25 09:47:41 +00:00
Chandler Carruth
de1c9bb450 Remove return heuristics from the static branch probabilities, and
introduce no-return or unreachable heuristics.

The return heuristics from the Ball and Larus paper don't work well in
practice as they pessimize early return paths. The only good hitrate
return heuristics are those for:
 - NULL return
 - Constant return
 - negative integer return

Only the last of these three can possibly require significant code for
the returning block, and even the last is fairly rare and usually also
a constant. As a consequence, even for the cold return paths, there is
little code on that return path, and so little code density to be gained
by sinking it. The places where sinking these blocks is valuable (inner
loops) will already be weighted appropriately as the edge is a loop-exit
branch.

All of this aside, early returns are nearly as common as all three of
these return categories, and should actually be predicted as taken!
Rather than muddy the waters of the static predictions, just remain
silent on returns and let the CFG itself dictate any layout or other
issues.

However, the return heuristic was flagging one very important case:
unreachable. Unfortunately it still gave a 1/4 chance of the
branch-to-unreachable occuring. It also didn't do a rigorous job of
finding those blocks which post-dominate an unreachable block.

This patch builds a more powerful analysis that should flag all branches
to blocks known to then reach unreachable. It also has better worst-case
runtime complexity by not looping through successors for each block. The
previous code would perform an N^2 walk in the event of a single entry
block branching to N successors with a switch where each successor falls
through to the next and they finally fall through to a return.

Test case added for noreturn heuristics. Also doxygen comments improved
along the way.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@142793 91177308-0d34-0410-b5e6-96231b3b80d8
2011-10-24 12:01:08 +00:00
Chandler Carruth
b068bbbaec Simplify the design of BranchProbabilityInfo by collapsing it into
a single class. Previously it was split between two classes, one
internal and one external. The concern seemed to center around exposing
the weights used, but those can remain confined to the implementation
file.

Having a single class to maintain the state and analyses in use will
also simplify several of the enhancements I want to make to our static
heuristics.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@142783 91177308-0d34-0410-b5e6-96231b3b80d8
2011-10-24 01:40:45 +00:00
Chandler Carruth
22c8946239 Tidy up a loop to be more idiomatic for LLVM's codebase, and remove some
extraneous whitespace. Trying to clean-up this pass as much as I can
before I start making functional changes.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@142780 91177308-0d34-0410-b5e6-96231b3b80d8
2011-10-23 22:40:13 +00:00
Chandler Carruth
14edd314af Teach the BranchProbabilityInfo pass to print its results, and use that
to bring it under direct test instead of merely indirectly testing it in
the BlockFrequencyInfo pass.

The next step is to start adding tests for the various heuristics
employed, and to start fixing those heuristics once they're under test.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@142778 91177308-0d34-0410-b5e6-96231b3b80d8
2011-10-23 21:21:50 +00:00
Benjamin Kramer
341473c86d Add compare operators to BranchProbability and use it to determine if an edge is hot.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@142751 91177308-0d34-0410-b5e6-96231b3b80d8
2011-10-23 11:19:14 +00:00
Benjamin Kramer
675c02b0b9 Extend the floating point heuristic to consider NaN checks unlikely.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@142687 91177308-0d34-0410-b5e6-96231b3b80d8
2011-10-21 21:13:47 +00:00
Benjamin Kramer
c888aa47bd BranchProbabilityInfo: floating point equality is unlikely.
This is from the same paper from Ball and Larus as the rest of the currently implemented heuristics.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@142677 91177308-0d34-0410-b5e6-96231b3b80d8
2011-10-21 20:12:47 +00:00
Chandler Carruth
941aa7b1e8 Generalize the reading of probability metadata to work for both branches
and switches, with arbitrary numbers of successors. Still optimized for
the common case of 2 successors for a conditional branch.

Add a test case for switch metadata showing up in the BlockFrequencyInfo pass.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@142493 91177308-0d34-0410-b5e6-96231b3b80d8
2011-10-19 10:32:19 +00:00
Chandler Carruth
99d01c54a0 Teach the BranchProbabilityInfo analysis pass to read any metadata
encoding of probabilities. In the absense of metadata, it continues to
fall back on static heuristics.

This allows __builtin_expect, after lowering through llvm.expect
a branch instruction's metadata, to actually enter the branch
probability model. This is one component of resolving PR2577.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@142492 91177308-0d34-0410-b5e6-96231b3b80d8
2011-10-19 10:30:30 +00:00
Chandler Carruth
7a34c8b602 Delete a dead member. Dunno if this was ever used, but the current code
directly manipulates the weights inside of the BranchProbabilityInfo
that is passed in.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@142163 91177308-0d34-0410-b5e6-96231b3b80d8
2011-10-16 22:27:54 +00:00
Benjamin Kramer
26eb870d7e Use canonical forms for the branch probability zero heutistic.
- Drop support for X >u 0, it's equivalent to X != 0 and should be canonicalized into the latter.
- Add X < 1 -> unlikely, which is what instcombine canonicalizes X <= 0 into.
- Add X > -1 -> likely, which is what instcombine canonicalizes X >= 0 into.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@139110 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-04 23:53:04 +00:00
Jakub Staszak
b137f16936 Change SmallVector to SmallPtrSet in BranchProbabilityInfo. Handle cases where
one than one successor goes to the same block.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@136638 91177308-0d34-0410-b5e6-96231b3b80d8
2011-08-01 19:16:26 +00:00
Jakub Staszak
6762dc1fb3 Do not handle cases with >= and <= predicates.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@136588 91177308-0d34-0410-b5e6-96231b3b80d8
2011-07-31 05:54:04 +00:00
Jakub Staszak
4faf553d50 Remove untrue comment.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@136587 91177308-0d34-0410-b5e6-96231b3b80d8
2011-07-31 04:51:14 +00:00
Jakub Staszak
a385c20d42 Do not handle case where LHS is equal to zero, because InstCombiner always moves
it to RHS anyway.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@136586 91177308-0d34-0410-b5e6-96231b3b80d8
2011-07-31 04:47:20 +00:00
Jakub Staszak
a5dd550588 Add Zero Heurestics to BranchProbabilityInfo. If we compare value to zero we
decide whether condition is likely to be true this way:

x == 0  ->  false
x <  0  ->  false
x <= 0  ->  false
x != 0  ->  true
x >  0  ->  true
x >= 0  ->  true


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@136583 91177308-0d34-0410-b5e6-96231b3b80d8
2011-07-31 03:27:24 +00:00
Jakub Staszak
6f6baf1bdd Add more constantness in BranchProbabilityInfo.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@136502 91177308-0d34-0410-b5e6-96231b3b80d8
2011-07-29 19:30:00 +00:00
Jakub Staszak
e0058b4b0c Remove incEdgeWeight and decEdgeWeight. Set edge weight directly to avoid
rounding errors.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@136456 91177308-0d34-0410-b5e6-96231b3b80d8
2011-07-29 02:36:53 +00:00
Jakub Staszak
3d8b15ea70 Change LBH_TAKEN_WEIGHT to 124 (from 128). Right now, sum of
LBH_TAKEN_WEIGHT + LBH_NONTAKEN_WEIGHT = 128 which in _most_ cases reduce
number of rounding errors.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@136428 91177308-0d34-0410-b5e6-96231b3b80d8
2011-07-28 23:42:08 +00:00
Jakub Staszak
7241caf7b8 Heuristics are in descending priority now. If we use one of them, skip the rest.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@136402 91177308-0d34-0410-b5e6-96231b3b80d8
2011-07-28 21:45:07 +00:00
Jakub Staszak
fa44725233 Add InEdges (edges from header to the loop) in Loop Branch Heuristics, so
there is no frequency difference whether condition is in the header or in
the latch.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@136398 91177308-0d34-0410-b5e6-96231b3b80d8
2011-07-28 21:33:46 +00:00
Jakub Staszak
12af93ae86 Remove "LoopInfo.h" include from BranchProbabilityInfo.h.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@135353 91177308-0d34-0410-b5e6-96231b3b80d8
2011-07-16 20:31:15 +00:00
Jakub Staszak
d7932ca962 Fix pointer heuristic. Check whether predicator is ICMP_NE instead of if it is
not isEquality().


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@135296 91177308-0d34-0410-b5e6-96231b3b80d8
2011-07-15 20:51:06 +00:00
Jakub Staszak
66dddd1da3 Calculate backedge probability correctly.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@133776 91177308-0d34-0410-b5e6-96231b3b80d8
2011-06-23 23:52:11 +00:00
Jakub Staszak
44eb49c2a1 Introduce BlockFrequency analysis for BasicBlocks.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@133766 91177308-0d34-0410-b5e6-96231b3b80d8
2011-06-23 21:45:20 +00:00
Jakub Staszak
7cc2b07437 Introduce MachineBranchProbabilityInfo class, which has similar API to
BranchProbabilityInfo (expect setEdgeWeight which is not available here).
Branch Weights are kept in MachineBasicBlocks. To turn off this analysis
set -use-mbpi=false.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@133184 91177308-0d34-0410-b5e6-96231b3b80d8
2011-06-16 20:22:37 +00:00
Benjamin Kramer
afa88eaf65 Move class into an anonymous namespace.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@132925 91177308-0d34-0410-b5e6-96231b3b80d8
2011-06-13 18:38:56 +00:00
Andrew Trick
f289df2d95 Branch profiling: floating-point avoidance.
Patch by: Jakub Staszak!

Introduces BranchProbability. Changes unsigned to uint32_t all over and
uint64_t only when overflow is expected.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@132867 91177308-0d34-0410-b5e6-96231b3b80d8
2011-06-11 01:05:22 +00:00
Nick Lewycky
404b53e38c Fold assert-only-used variable into the assert.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@132620 91177308-0d34-0410-b5e6-96231b3b80d8
2011-06-04 02:07:10 +00:00
Andrew Trick
b4528c3d7e Missing include of climits in the new BranchProbability pass.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@132616 91177308-0d34-0410-b5e6-96231b3b80d8
2011-06-04 01:30:52 +00:00
Andrew Trick
9e76422b96 New BranchProbabilityInfo analysis. Patch by Jakub Staszak!
BranchProbabilityInfo provides an interface for IR passes to query the
likelihood that control follows a CFG edge. This patch provides an
initial implementation of static branch predication that will populate
BranchProbabilityInfo for branches with no external profile
information using very simple heuristics. It currently isn't hooked up
to any external profile data, so static prediction does all the work.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@132613 91177308-0d34-0410-b5e6-96231b3b80d8
2011-06-04 01:16:30 +00:00