Commit Graph
100 Commits
Author SHA1 Message Date
Chandler Carruth 8bc3434e68 Teach the 'opt' tool about '-Os' and '-Oz', corresponding to the Clang
options, to enable easier testing of the innards of LLVM that are
enabled by such optimization strategies.

Note that this doesn't provide the (much needed) function attribute
support for -Oz (as opposed to -Os), but still seems like a positive
step to better test the logic that Clang currently relies on.

Patch by Patrik Hägglund.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@156913 91177308-0d34-0410-b5e6-96231b3b80d8
2012-05-16 08:32:49 +00:00
Chandler Carruth 917644d0c3 Fix a Clang warning in the new NVPTX backend:
In file included from ../lib/Target/NVPTX/VectorElementize.cpp:53:
../lib/Target/NVPTX/NVPTX.h:44:3: warning: default label in switch which covers all enumeration values [-Wcovered-switch-default]
  default: assert(0 && "Unknown condition code");
  ^
1 warning generated.

The prevailing pattern in LLVM is to not use a default label, and instead to
use llvm_unreachable to denote that the switch in fact covers all return paths
from the function.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@156209 91177308-0d34-0410-b5e6-96231b3b80d8
2012-05-04 21:35:49 +00:00
Chandler Carruth 30ba82933c Teach the code extractor how to extract a sequence of blocks from
RegionInfo's RegionNode. This mirrors the logic for automating the
extraction from a Loop.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@156208 91177308-0d34-0410-b5e6-96231b3b80d8
2012-05-04 21:33:30 +00:00
Chandler Carruth 7c52c97a22 Rename the Region::block_iterator to Region::block_node_iterator, and
add a new Region::block_iterator which actually iterates over the basic
blocks of the region.

The old iterator, now call 'block_node_iterator' iterates over
RegionNodes which contain a single basic block. This works well with the
GraphTraits-based iterator design, however most users actually want an
iterator over the BasicBlocks inside these RegionNodes. Now the
'block_iterator' is a wrapper which exposes exactly this interface.
Internally it uses the block_node_iterator to walk all nodes which are
single basic blocks, but transparently unwraps the basic block to make
user code simpler.

While this patch is a bit of a wash, most of the updates are to internal
users, not external users of the RegionInfo. I have an accompanying
patch to Polly that is a strict simplification of every user of this
interface, and I'm working on a pass that also wants the same simplified
interface.

This patch alone should have no functional impact.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@156202 91177308-0d34-0410-b5e6-96231b3b80d8
2012-05-04 20:55:23 +00:00
Chandler Carruth 90cb7089e3 Factor the computation of input and output sets into a public interface
of the CodeExtractor utility. This allows speculatively computing input
and output sets to measure the likely size impact of the code
extraction.

These sets cannot be reused sadly -- we mutate the function prior to
forming the final sets used by the actual extraction.

The interface has been revamped slightly to make it easier to use
correctly by making the interface const and sinking the computation of
the number of exit blocks into the full extraction function and away
from the rest of this logic which just computed two output parameters.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@156168 91177308-0d34-0410-b5e6-96231b3b80d8
2012-05-04 11:20:27 +00:00
Chandler Carruth 50955031b8 Rather than trying to gracefully handle input sequences with repeated
blocks, assert that this doesn't happen. We don't want to bother trying
to support this call pattern as it isn't necessary.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@156167 91177308-0d34-0410-b5e6-96231b3b80d8
2012-05-04 11:17:06 +00:00
Chandler Carruth 6a81f64ab6 Fix a goof with my previous commit by completely returning when we
detect an in-eligible block rather than just breaking out of the loop.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@156166 91177308-0d34-0410-b5e6-96231b3b80d8
2012-05-04 11:14:19 +00:00
Chandler Carruth 27742c1a76 Hoist a safety assert from the extraction method into the construction
of the extractor itself.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@156164 91177308-0d34-0410-b5e6-96231b3b80d8
2012-05-04 10:26:45 +00:00
Chandler Carruth 99650c9088 Move the CodeExtractor utility to a dedicated header file / source file,
and expose it as a utility class rather than as free function wrappers.

The simple free-function interface works well for the bugpoint-specific
pass's uses of code extraction, but in an upcoming patch for more
advanced code extraction, they simply don't expose a rich enough
interface. I need to expose various stages of the process of doing the
code extraction and query information to decide whether or not to
actually complete the extraction or give up.

Rather than build up a new predicate model and pass that into these
functions, just take the class that was actually implementing the
functions and lift it up into a proper interface that can be used to
perform code extraction. The interface is cleaned up and re-documented
to work better in a header. It also is now setup to accept the blocks to
be extracted in the constructor rather than in a method.

In passing this essentially reverts my previous commit here exposing
a block-level query for eligibility of extraction. That is no longer
necessary with the more rich interface as clients can query the
extraction object for eligibility directly. This will reduce the number
of walks of the input basic block sequence by quite a bit which is
useful if this enters the normal optimization pipeline.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@156163 91177308-0d34-0410-b5e6-96231b3b80d8
2012-05-04 10:18:49 +00:00
Chandler Carruth d5003cafd6 A pile of long over-due refactorings here. There are some very, *very*
minor behavior changes with this, but nothing I have seen evidence of in
the wild or expect to be meaningful. The real goal is unifying our logic
and simplifying the interfaces. A summary of the changes follows:

- Make 'callIsSmall' actually accept a callsite so it can handle
  intrinsics, and simplify callers appropriately.
- Nuke a completely bogus declaration of 'callIsSmall' that was still
  lurking in InlineCost.h... No idea how this got missed.
- Teach the 'isInstructionFree' about the various more intelligent
  'free' heuristics that got added to the inline cost analysis during
  review and testing. This mostly surrounds int->ptr and ptr->int casts.
- Switch most of the interesting parts of the inline cost analysis that
  were essentially computing 'is this instruction free?' to use the code
  metrics routine instead. This way we won't keep duplicating logic.

All of this is motivated by the desire to allow other passes to compute
a roughly equivalent 'cost' metric for a particular basic block as the
inline cost analysis. Sadly, re-using the same analysis for both is
really messy because only the actual inline cost analysis is ever going
to go to the contortions required for simplification, SROA analysis,
etc.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@156140 91177308-0d34-0410-b5e6-96231b3b80d8
2012-05-04 00:58:03 +00:00
Chandler Carruth a83a6d3725 Add a FoldingSetVector datastructure which is analogous to a SetVector,
but using a FoldingSet underneath and with a largely compatible
interface to that of FoldingSet. This can be used anywhere a FoldingSet
would be natural, but iteration order is significant. The initial
intended use case is in Clang's template specialization lists to
preserve instantiation order iteration.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@156131 91177308-0d34-0410-b5e6-96231b3b80d8
2012-05-03 23:38:34 +00:00
Chandler Carruth 9f7af7b748 Factor the logic for testing whether a basic block is viable for code
extraction into a public interface. Also clean it up and apply it more
consistently such that we check for landing pads *anywhere* in the
extracted code, not just in single-block extraction.

This will be used to guide decisions in passes that are planning to
eventually perform a round of code extraction.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@156114 91177308-0d34-0410-b5e6-96231b3b80d8
2012-05-03 22:26:53 +00:00
Chandler Carruth 464bda3a16 Teach the reassociate pass to fold chains of multiplies with repeated
elements to minimize the number of multiplies required to compute the
final result. This uses a heuristic to attempt to form near-optimal
binary exponentiation-style multiply chains. While there are some cases
it misses, it seems to at least a decent job on a very diverse range of
inputs.

Initial benchmarks show no interesting regressions, and an 8%
improvement on SPASS. Let me know if any other interesting results (in
either direction) crop up!

Credit to Richard Smith for the core algorithm, and helping code the
patch itself.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@155616 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-26 05:30:30 +00:00
Chandler Carruth 3ef91f575e Actually delete now-empty file.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@155532 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-25 02:30:00 +00:00
Chandler Carruth 7362ac7f8c Fix a crash on valid (if UB) bitcode that is produced for some global
constants in C++11 mode. I have no idea why it required such particular
circumstances to get here, the code seems clearly to rely upon unchecked
assumptions.

Specifically, when we decide to form an index into a struct type, we may
have gone through (at least one) zero-length array indexing round, which
would have left the offset un-adjusted, and thus not necessarily valid
for use when indexing the struct type.

This is just an canonicalization step, so the correct thing is to refuse
to canonicalize nonsensical GEPs of this form. Implemented, and test
case added.

Fixes PR12642. Pair debugged and coded with Richard Smith. =] I credit
him with most of the debugging, and preventing me from writing the wrong
code.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@155466 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-24 18:42:47 +00:00
Chandler Carruth e3fd2a36d9 Temporarily revert r155364 until the upstream review can complete, per
the stated developer policy.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@155373 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-23 18:28:57 +00:00
Chandler Carruth d410eaba04 Revert r155365, r155366, and r155367. All three of these have regression
test suite failures. The failures occur at each stage, and only get
worse, so I'm reverting all of them.

Please resubmit these patches, one at a time, after verifying that the
regression test suite passes. Never submit a patch without running the
regression test suite.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@155372 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-23 18:25:57 +00:00
Chandler Carruth a3e3481c57 Tidy up this test more:
1) Make the checked assertions a bit more precise. We really want the
   canonical forms coming out of reassociate to be exactly what is
   expected.
2) Remove other passes, and switch the test to actually directly check
   that reassociate makes the important transforms and
   canonicalizations.
3) Fold in a related test case now that we're using FileCheck. Make the
   same tidying changes to it.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@155311 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-22 10:11:26 +00:00
Chandler Carruth 71f8bc37f2 FileCheck-ize a test, and tidy it up a touch.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@155310 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-22 10:11:23 +00:00
Chandler Carruth 37097623bb This reverts a long string of commits to the Hexagon backend. These
commits have had several major issues pointed out in review, and those
issues are not being addressed in a timely fashion. Furthermore, this
was all committed leading up to the v3.1 branch, and we don't need piles
of code with outstanding issues in the branch.

It is possible that not all of these commits were necessary to revert to
get us back to a green state, but I'm going to let the Hexagon
maintainer sort that out. They can recommit, in order, after addressing
the feedback.

Reverted commits, with some notes:

Primary commit r154616: HexagonPacketizer
  - There are lots of review comments here. This is the primary reason
    for reverting. In particular, it introduced large amount of warnings
    due to a bad construct in tablegen.
  - Follow-up commits that should be folded back into this when
    reposting:
    - r154622: CMake fixes
    - r154660: Fix numerous build warnings in release builds.
  - Please don't resubmit this until the three commits above are
    included, and the issues in review addressed.

Primary commit r154695: Pass to replace transfer/copy ...
  - Reverted to minimize merge conflicts. I'm not aware of specific
    issues with this patch.

Primary commit r154703: New Value Jump.
  - Primarily reverted due to merge conflicts.
  - Follow-up commits that should be folded back into this when
    reposting:
    - r154703: Remove iostream usage
    - r154758: Fix CMake builds
    - r154759: Fix build warnings in release builds
  - Please incorporate these fixes and and review feedback before
    resubmitting.

Primary commit r154829: Hexagon V5 (floating point) support.
  - Primarily reverted due to merge conflicts.
  - Follow-up commits that should be folded back into this when
    reposting:
    - r154841: Remove unused variable (fixing build warnings)

There are also accompanying Clang commits that will be reverted for
consistency.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@155047 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-18 21:31:19 +00:00
Chandler Carruth 03565edea6 Update woefully stale credits entry. Everything listed there had been
re-done by wiser souls.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154894 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-17 01:19:37 +00:00
Chandler Carruth c76c94f9bf Add a stub about the inline cost rewrite.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154892 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-17 01:13:53 +00:00
Chandler Carruth 268fde43cf Add a stub for block placement pass. This needs fleshing out w/ details
about what folks should expect w.r.t. the new algorithm.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154891 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-17 01:10:35 +00:00
Chandler Carruth fd2e4e65f7 Disable the atom scheduling test after r154874 broke it.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154877 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-16 23:11:39 +00:00
Chandler Carruth 177bea5330 Relax this test a touch to cope with different assembly variants.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154870 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-16 22:20:48 +00:00
Chandler Carruth f1a60c734c Fix updateTerminator to be resiliant to degenerate terminators where
both fallthrough and a conditional branch target the same successor.
Gracefully delete the conditional branch and introduce any unconditional
branch needed to reach the actual successor. This fixes memory
corruption in 2009-06-15-RegScavengerAssert.ll and possibly other tests.

Also, while I'm here fix a latent bug I spotted by inspection. I never
applied the same fundamental fix to this fallthrough successor finding
logic that I did to the logic used when there are no conditional
branches. As a consequence it would have selected landing pads had they
be aligned in just the right way here. I don't have a test case as
I spotted this by inspection, and the previous time I found this
required have of TableGen's source code to produce it. =/ I hate backend
bugs. ;]

Thanks to Jim Grosbach for helping me reason through this and reviewing
the fix.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154867 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-16 22:03:00 +00:00
Chandler Carruth 9e67db4af1 Flip the new block-placement pass to be on by default.
This is mostly to test the waters. I'd like to get results from FNT
build bots and other bots running on non-x86 platforms.

This feature has been pretty heavily tested over the last few months by
me, and it fixes several of the execution time regressions caused by the
inlining work by preventing inlining decisions from radically impacting
block layout.

I've seen very large improvements in yacr2 and ackermann benchmarks,
along with the expected noise across all of the benchmark suite whenever
code layout changes. I've analyzed all of the regressions and fixed
them, or found them to be impossible to fix. See my email to llvmdev for
more details.

I'd like for this to be in 3.1 as it complements the inliner changes,
but if any failures are showing up or anyone has concerns, it is just
a flag flip and so can be easily turned off.

I'm switching it on tonight to try and get at least one run through
various folks' performance suites in case SPEC or something else has
serious issues with it. I'll watch bots and revert if anything shows up.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154816 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-16 13:49:17 +00:00
Chandler Carruth 0de089a4d0 Remove an overly brittle test. This test will no longer be interesting
once we start changing the block layout, so just nuke it. If anyone has
ideas about how to craft a code layout agnostic form of the test please
let me know.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154815 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-16 13:49:09 +00:00
Chandler Carruth e773e8c3e5 Add a somewhat hacky heuristic to do something different from whole-loop
rotation. When there is a loop backedge which is an unconditional
branch, we will end up with a branch somewhere no matter what. Try
placing this backedge in a fallthrough position above the loop header as
that will definitely remove at least one branch from the loop iteration,
where whole loop rotation may not.

I haven't seen any benchmarks where this is important but loop-blocks.ll
tests for it, and so this will be covered when I flip the default.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154812 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-16 13:33:36 +00:00
Chandler Carruth 16295fc20b Tweak the loop rotation logic to check whether the loop is naturally
laid out in a form with a fallthrough into the header and a fallthrough
out of the bottom. In that case, leave the loop alone because any
rotation will introduce unnecessary branches. If either side looks like
it will require an explicit branch, then the rotation won't add any, do
it to ensure the branch occurs outside of the loop (if possible) and
maximize the benefit of the fallthrough in the bottom.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154806 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-16 09:31:23 +00:00
Chandler Carruth 70daea90af Rewrite how machine block placement handles loop rotation.
This is a complex change that resulted from a great deal of
experimentation with several different benchmarks. The one which proved
the most useful is included as a test case, but I don't know that it
captures all of the relevant changes, as I didn't have specific
regression tests for each, they were more the result of reasoning about
what the old algorithm would possibly do wrong. I'm also failing at the
moment to craft more targeted regression tests for these changes, if
anyone has ideas, it would be welcome.

The first big thing broken with the old algorithm is the idea that we
can take a basic block which has a loop-exiting successor and a looping
successor and use the looping successor as the layout top in order to
get that particular block to be the bottom of the loop after layout.
This happens to work in many cases, but not in all.

The second big thing broken was that we didn't try to select the exit
which fell into the nearest enclosing loop (to which we exit at all). As
a consequence, even if the rotation worked perfectly, it would result in
one of two bad layouts. Either the bottom of the loop would get
fallthrough, skipping across a nearer enclosing loop and thereby making
it discontiguous, or it would be forced to take an explicit jump over
the nearest enclosing loop to earch its successor. The point of the
rotation is to get fallthrough, so we need it to fallthrough to the
nearest loop it can.

The fix to the first issue is to actually layout the loop from the loop
header, and then rotate the loop such that the correct exiting edge can
be a fallthrough edge. This is actually much easier than I anticipated
because we can handle all the hard parts of finding a viable rotation
before we do the layout. We just store that, and then rotate after
layout is finished. No inner loops get split across the post-rotation
backedge because we check for them when selecting the rotation.

That fix exposed a latent problem with our exitting block selection --
we should allow the backedge to point into the middle of some inner-loop
chain as there is no real penalty to it, the whole point is that it
*won't* be a fallthrough edge. This may have blocked the rotation at all
in some cases, I have no idea and no test case as I've never seen it in
practice, it was just noticed by inspection.

Finally, all of these fixes, and studying the loops they produce,
highlighted another problem: in rotating loops like this, we sometimes
fail to align the destination of these backwards jumping edges. Fix this
by actually walking the backwards edges rather than relying on loopinfo.

This fixes regressions on heapsort if block placement is enabled as well
as lots of other cases where the previous logic would introduce an
abundance of unnecessary branches into the execution.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154783 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-16 01:12:56 +00:00
Chandler Carruth d6fc26217e Add two statistics to help track how we are computing the inline cost.
Yea, 'NumCallerCallersAnalyzed' isn't a great name, suggestions welcome.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154492 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-11 10:15:10 +00:00
Chandler Carruth 45fb79bc54 Make a somewhat subtle change in the logic of block placement. Sometimes
the loop header has a non-loop predecessor which has been pre-fused into
its chain due to unanalyzable branches. In this case, rotating the
header into the body of the loop in order to place a loop exit at the
bottom of the loop is a Very Bad Idea as it makes the loop
non-contiguous.

I'm working on a good test case for this, but it's a bit annoynig to
craft. I should get one shortly, but I'm submitting this now so I can
begin the (lengthy) performance analysis process. An initial run of LNT
looks really, really good, but there is too much noise there for me to
trust it much.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154395 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-10 13:35:57 +00:00
Chandler Carruth ab5a55e118 Cleanup and relax a restriction on the matching of global offsets into
x86 addressing modes. This allows PIE-based TLS offsets to fit directly
into an addressing mode immediate offset, which is the last remaining
code quality issue from PR12380. With this patch, that PR is completely
fixed.

To understand why this patch is correct to match these offsets into
addressing mode immediates, break it down by cases:
1) 32-bit is trivially correct, and unmodified here.
2) 64-bit non-small mode is unchanged and never matches.
3) 64-bit small PIC code which is RIP-relative is handled specially in
   the match to try to fit RIP into the base register. If it fails, it
   now early exits. This behavior is unchanged by the patch.
4) 64-bit small non-PIC code which is not RIP-relative continues to work
   as it did before. The reason these immediates are safe is because the
   ABI ensures they fit in small mode. This behavior is unchanged.
5) 64-bit small PIC code which is *not* using RIP-relative addressing.
   This is the only case changed by the patch, and the primary place you
   see it is in TLS, either the win64 section offset TLS or Linux
   local-exec TLS model in a PIC compilation. Here the ABI again ensures
   that the immediates fit because we are in small mode, and any other
   operations required due to the PIC relocation model have been handled
   externally to the Wrapper node (extra loads etc are made around the
   wrapper node in ISelLowering).

I've tested this as much as I can comparing it with GCC's output, and
everything appears safe. I discussed this with Anton and it made sense
to him at least at face value. That said, if there are issues with PIC
code after this patch, yell and we can revert it.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154304 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-09 02:13:06 +00:00
Chandler Carruth 6916a2375a Fold 15 tiny test cases into a single file that implements the
comprehensive testing of TLS codegen for x86. Convert all of the ones
that were still using grep to use FileCheck. Remove some redundancies
between them.

Perhaps most interestingly expand the test cases so that they actually
fully list the instruction snippet being tested. TLS operations are
*very* narrowly defined, and so these seem reasonably stable. More
importantly, the existing test cases already were crazy fine grained,
expecting specific registers to be allocated. This just clarifies that
no *other* instructions are expected, and fills in some crucial gaps
that weren't being tested at all.

This will make any subsequent changes to TLS much more clear during
review.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154303 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-09 01:43:17 +00:00
Chandler Carruth 253933ee9e Teach LLVM about a PIE option which, when enabled on top of PIC, makes
optimizations which are valid for position independent code being linked
into a single executable, but not for such code being linked into
a shared library.

I discussed the design of this with Eric Christopher, and the decision
was to support an optional bit rather than a completely separate
relocation model. Fundamentally, this is still PIC relocation, its just
that certain optimizations are only valid under a PIC relocation model
when the resulting code won't be in a shared library. The simplest path
to here is to expose a single bit option in the TargetOptions. If folks
have different/better designs, I'm all ears. =]

I've included the first optimization based upon this: changing TLS
models to the *Exec models when PIE is enabled. This is the LLVM
component of PR12380 and is all of the hard work.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154294 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-08 17:51:45 +00:00
Chandler Carruth 34797136cb Move the TLSModel information into the TargetMachine rather than hiding
in TargetLowering. There was already a FIXME about this location being
odd. The interface is simplified as a consequence. This will also make
it easier to change TLS models when compiling with PIE.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154292 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-08 17:20:55 +00:00
Chandler Carruth 5e93a76722 Remove an over zealous assert. The assert was trying to catch places
where a chain outside of the loop block-set ended up in the worklist for
scheduling as part of the contiguous loop. However, asserting the first
block in the chain is in the loop-set isn't a valid check -- we may be
forced to drag a chain into the worklist due to one block in the chain
being part of the loop even though the first block is *not* in the loop.
This occurs when we have been forced to form a chain early due to
un-analyzable branches.

No test case here as I have no idea how to even begin reducing one, and
it will be hopelessly fragile. We have to somehow end up with a loop
header of an inner loop which is a successor of a basic block with an
unanalyzable pair of branch instructions. Ow. Self-host triggers it so
it is unlikely it will regress.

This at least gets block placement back to passing selfhost and the test
suite. There are still a lot of slowdown that I don't like coming out of
block placement, although there are now also a lot of speedups. =[ I'm
seeing swings in both directions up to 10%. I'm going to try to find
time to dig into this and see if we can turn this on for 3.1 as it does
a really good job of cleaning up after some loops that degraded with the
inliner changes.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154287 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-08 14:37:02 +00:00
Chandler Carruth 6313d941d2 Add a debug-only 'dump' method to the BlockChain structure to ease
debugging.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154286 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-08 14:37:01 +00:00
Chandler Carruth 2450eca960 Teach InstCombine to nuke a common alloca pattern -- an alloca which has
GEPs, bit casts, and stores reaching it but no other instructions. These
often show up during the iterative processing of the inliner, SROA, and
DCE. Once we hit this point, we can completely remove the alloca. These
were actually showing up in the final, fully optimized code in a bunch
of inliner tests I've been working on, and notably they show up after
LLVM finishes optimizing away all function calls involved in
hash_combine(a, b).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154285 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-08 14:36:56 +00:00
Chandler Carruth 5cd79bc14c Perform partial SROA on the helper hashing structure. I really wish the
optimizers could do this for us, but expecting partial SROA of classes
with template methods through cloning is probably expecting too much
heroics. With this change, the begin/end pointer pairs which indicate
the status of each loop iteration are actually passed directly into each
layer of the combine_data calls, and the inliner has a chance to see
when most of the combine_data function could be deleted by inlining.
Similarly for 'length'.

We have to be careful to limit the places where in/out reference
parameters are used as those will also defeat the inliner / optimizers
from properly propagating constants.

With this change, LLVM is able to fully inline and unroll the hash
computation of small sets of values, such as two or three pointers.
These now decompose into essentially straight-line code with no loops or
function calls.

There is still one code quality problem to be solved with the hashing --
LLVM is failing to nuke the alloca. It removes all loads from the
alloca, leaving only lifetime intrinsics and dead(!!) stores to the
alloca. =/ Very unfortunate.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154264 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-07 20:01:31 +00:00
Chandler Carruth c0d18b6696 Fix ValueTracking to conclude that debug intrinsics are safe to
speculate. Without this, loop rotate (among many other places) would
suddenly stop working in the presence of debug info. I found this
looking at loop rotate, and have augmented its tests with a reduction
out of a very hot loop in yacr2 where failing to do this rotation costs
sometimes more than 10% in runtime performance, perturbing numerous
downstream optimizations.

This should have no impact on performance without debug info, but the
change in performance when debug info is enabled can be extreme. As
a consequence (and this how I got to this yak) any profiling of
performance problems should be treated with deep suspicion -- they may
have been wildly innacurate of debug info was enabled for profiling. =/
Just a heads up.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154263 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-07 19:22:18 +00:00
Chandler Carruth ba4d45737f There is no portable std::abs overload for int64_t, use the llvm::abs64
which exists for this purpose.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154199 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-06 20:10:52 +00:00
Chandler Carruth 9ceebb7e92 Sink the collection of return instructions until after *all*
simplification has been performed. This is a bit less efficient
(requires another ilist walk of the basic blocks) but shouldn't matter
in practice. More importantly, it's just too much work to keep track of
all the various ways the return instructions can be mutated while
simplifying them. This fixes yet another crasher, reported by Daniel
Dunbar.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154179 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-06 17:21:31 +00:00
Chandler Carruth be2df1675d Tweak this test to ensure the inliner did indeed fire. Thanks to Richard
Smith for pointing this out in review.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154178 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-06 17:21:28 +00:00
Chandler Carruth c0a7a1280c Actually finish this sentence in the comment the way I intended. Thanks
Matt for pointing this out.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154158 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-06 01:19:38 +00:00
Chandler Carruth 6bbab86af9 Sink the return instruction collection until after we're done deleting
dead code, including dead return instructions in some cases. Otherwise,
we end up having a bogus poniter to a return instruction that blows up
much further down the road.

It turns out that this pattern is both simpler to code, easier to update
in the face of enhancements to the inliner cleanup, and likely cheaper
given that it won't add dead instructions to the list.

Thanks to John Regehr's numerous test cases for teasing this out.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@154157 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-06 01:11:52 +00:00
Chandler Carruth caa2c40a57 Start cleaning up the InlineCost class. This switches to sentinel values
rather than a bitfield, a great suggestion by Chris during code review.

There is still quite a bit of cruft in the interface, but that requires
sorting out some awkward uses of the cost inside the actual inliner.

No functionality changed intended here.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153853 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-01 22:44:09 +00:00
Chandler Carruth dafe48e230 Belatedly address some code review from Chris.
As a side note, I really dislike array_pod_sort... Do we really still
care about any STL implementations that get this so wrong? Does libc++?

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153834 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-01 10:41:24 +00:00
Chandler Carruth 48ec3b50e7 Add some more testing to cover the remaining two cases where
always-inlining is disabled: recursive functions and indirectbr.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153833 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-01 10:36:17 +00:00
Chandler Carruth 6052eef8bd Fix a pretty scary bug I introduced into the always inliner with
a single missing character. Somehow, this had gone untested. I've added
tests for returns-twice logic specifically with the always-inliner that
would have caught this, and fixed the bug.

Thanks to Matt for the careful review and spotting this!!! =D

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153832 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-01 10:21:05 +00:00
Chandler Carruth 0b42f9dd2f Replace four tiny tests with various uses of grep and not with a single
test and FileCheck.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153831 91177308-0d34-0410-b5e6-96231b3b80d8
2012-04-01 10:11:17 +00:00
Chandler Carruth f5f256cffd Fix a typo reported in IRC by someone reviewing this code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153815 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-31 13:18:09 +00:00
Chandler Carruth b594a84df5 Give the always-inliner its own custom filter. It shouldn't have to pay
the very high overhead of the complex inline cost analysis when all it
wants to do is detect three patterns which must not be inlined. Comment
the code, clean it up, and leave some hints about possible performance
improvements if this ever shows up on a profile.

Moving this off of the (now more expensive) inline cost analysis is
particularly important because we have to run this inliner even at -O0.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153814 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-31 13:17:18 +00:00
Chandler Carruth 45de584b4f Remove a bunch of empty, dead, and no-op methods from all of these
interfaces. These methods were used in the old inline cost system where
there was a persistent cache that had to be updated, invalidated, and
cleared. We're now doing more direct computations that don't require
this intricate dance. Even if we resume some level of caching, it would
almost certainly have a simpler and more narrow interface than this.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153813 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-31 12:48:08 +00:00
Chandler Carruth f2286b0152 Initial commit for the rewrite of the inline cost analysis to operate
on a per-callsite walk of the called function's instructions, in
breadth-first order over the potentially reachable set of basic blocks.

This is a major shift in how inline cost analysis works to improve the
accuracy and rationality of inlining decisions. A brief outline of the
algorithm this moves to:

- Build a simplification mapping based on the callsite arguments to the
  function arguments.
- Push the entry block onto a worklist of potentially-live basic blocks.
- Pop the first block off of the *front* of the worklist (for
  breadth-first ordering) and walk its instructions using a custom
  InstVisitor.
- For each instruction's operands, re-map them based on the
  simplification mappings available for the given callsite.
- Compute any simplification possible of the instruction after
  re-mapping, and store that back int othe simplification mapping.
- Compute any bonuses, costs, or other impacts of the instruction on the
  cost metric.
- When the terminator is reached, replace any conditional value in the
  terminator with any simplifications from the mapping we have, and add
  any successors which are not proven to be dead from these
  simplifications to the worklist.
- Pop the next block off of the front of the worklist, and repeat.
- As soon as the cost of inlining exceeds the threshold for the
  callsite, stop analyzing the function in order to bound cost.

The primary goal of this algorithm is to perfectly handle dead code
paths. We do not want any code in trivially dead code paths to impact
inlining decisions. The previous metric was *extremely* flawed here, and
would always subtract the average cost of two successors of
a conditional branch when it was proven to become an unconditional
branch at the callsite. There was no handling of wildly different costs
between the two successors, which would cause inlining when the path
actually taken was too large, and no inlining when the path actually
taken was trivially simple. There was also no handling of the code
*path*, only the immediate successors. These problems vanish completely
now. See the added regression tests for the shiny new features -- we
skip recursive function calls, SROA-killing instructions, and high cost
complex CFG structures when dead at the callsite being analyzed.

Switching to this algorithm required refactoring the inline cost
interface to accept the actual threshold rather than simply returning
a single cost. The resulting interface is pretty bad, and I'm planning
to do lots of interface cleanup after this patch.

Several other refactorings fell out of this, but I've tried to minimize
them for this patch. =/ There is still more cleanup that can be done
here. Please point out anything that you see in review.

I've worked really hard to try to mirror at least the spirit of all of
the previous heuristics in the new model. It's not clear that they are
all correct any more, but I wanted to minimize the change in this single
patch, it's already a bit ridiculous. One heuristic that is *not* yet
mirrored is to allow inlining of functions with a dynamic alloca *if*
the caller has a dynamic alloca. I will add this back, but I think the
most reasonable way requires changes to the inliner itself rather than
just the cost metric, and so I've deferred this for a subsequent patch.
The test case is XFAIL-ed until then.

As mentioned in the review mail, this seems to make Clang run about 1%
to 2% faster in -O0, but makes its binary size grow by just under 4%.
I've looked into the 4% growth, and it can be fixed, but requires
changes to other parts of the inliner.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153812 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-31 12:42:41 +00:00
Chandler Carruth 7384530c7c Add support to the InstVisitor for visiting a generic callsite. The
visitor will now visit a CallInst and an InvokeInst with
instruction-specific visitors, then visit a generic CallSite visitor,
then delegate back to the Instruction visitor and the TerminatorInst
visitors depending on whether a call or an invoke originally. This will
be used in the soon-to-land inline cost rewrite.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153811 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-31 11:31:24 +00:00
Chandler Carruth 426d5715b1 Clean up the naming in this test. Someone pointed this out in review at
one point, and I forgot to go back and clean it up. Sorry about that. =/

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153801 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-31 10:38:48 +00:00
Chandler Carruth c3e955927f FileCheck-ize this test, and generally tidy it up prior to changing
things around.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153799 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-31 09:22:33 +00:00
Chandler Carruth 803d134007 Filecheck-ize this test so that it actually tests something reasonable.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153697 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-29 22:01:41 +00:00
Chandler Carruth afff33001a Switch to WeakVHs in the value mapper, and aggressively prune dead basic
blocks in the function cloner. This removes the last case of trivially
dead code that I've been seeing in the wild getting inlined, analyzed,
re-inlined, optimized, only to be deleted. Nukes a FIXME from the
cleanup tests.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153572 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-28 08:38:27 +00:00
Chandler Carruth dacffb6679 Make a seemingly tiny change to the inliner and fix the generated code
size bloat. Unfortunately, I expect this to disable the majority of the
benefit from r152737. I'm hopeful at least that it will fix PR12345. To
explain this requires... quite a bit of backstory I'm afraid.

TL;DR: The change in r152737 actually did The Wrong Thing for
linkonce-odr functions. This change makes it do the right thing. The
benefits we saw were simple luck, not any actual strategy. Benchmark
numbers after a mini-blog-post so that I've written down my thoughts on
why all of this works and doesn't work...

To understand what's going on here, you have to understand how the
"bottom-up" inliner actually works. There are two fundamental modes to
the inliner:

1) Standard fixed-cost bottom-up inlining. This is the mode we usually
   think about. It walks from the bottom of the CFG up to the top,
   looking at callsites, taking information about the callsite and the
   called function and computing th expected cost of inlining into that
   callsite. If the cost is under a fixed threshold, it inlines. It's
   a touch more complicated than that due to all the bonuses, weights,
   etc. Inlining the last callsite to an internal function gets higher
   weighth, etc. But essentially, this is the mode of operation.

2) Deferred bottom-up inlining (a term I just made up). This is the
   interesting mode for this patch an r152737. Initially, this works
   just like mode #1, but once we have the cost of inlining into the
   callsite, we don't just compare it with a fixed threshold. First, we
   check something else. Let's give some names to the entities at this
   point, or we'll end up hopelessly confused. We're considering
   inlining a function 'A' into its callsite within a function 'B'. We
   want to check whether 'B' has any callers, and whether it might be
   inlined into those callers. If so, we also check whether inlining 'A'
   into 'B' would block any of the opportunities for inlining 'B' into
   its callers. We take the sum of the costs of inlining 'B' into its
   callers where that inlining would be blocked by inlining 'A' into
   'B', and if that cost is less than the cost of inlining 'A' into 'B',
   then we skip inlining 'A' into 'B'.

Now, in order for #2 to make sense, we have to have some confidence that
we will actually have the opportunity to inline 'B' into its callers
when cheaper, *and* that we'll be able to revisit the decision and
inline 'A' into 'B' if that ever becomes the correct tradeoff. This
often isn't true for external functions -- we can see very few of their
callers, and we won't be able to re-consider inlining 'A' into 'B' if
'B' is external when we finally see more callers of 'B'. There are two
cases where we believe this to be true for C/C++ code: functions local
to a translation unit, and functions with an inline definition in every
translation unit which uses them. These are represented as internal
linkage and linkonce-odr (resp.) in LLVM. I enabled this logic for
linkonce-odr in r152737.

Unfortunately, when I did that, I also introduced a subtle bug. There
was an implicit assumption that the last caller of the function within
the TU was the last caller of the function in the program. We want to
bonus the last caller of the function in the program by a huge amount
for inlining because inlining that callsite has very little cost.
Unfortunately, the last caller in the TU of a linkonce-odr function is
*not* the last caller in the program, and so we don't want to apply this
bonus. If we do, we can apply it to one callsite *per-TU*. Because of
the way deferred inlining works, when it sees this bonus applied to one
callsite in the TU for 'B', it decides that inlining 'B' is of the
*utmost* importance just so we can get that final bonus. It then
proceeds to essentially force deferred inlining regardless of the actual
cost tradeoff.

The result? PR12345: code bloat, code bloat, code bloat. Another result
is getting *damn* lucky on a few benchmarks, and the over-inlining
exposing critically important optimizations. I would very much like
a list of benchmarks that regress after this change goes in, with
bitcode before and after. This will help me greatly understand what
opportunities the current cost analysis is missing.

Initial benchmark numbers look very good. WebKit files that exhibited
the worst of PR12345 went from growing to shrinking compared to Clang
with r152737 reverted.

- Bootstrapped Clang is 3% smaller with this change.
- Bootstrapped Clang -O0 over a single-source-file of lib/Lex is 4%
  faster with this change.

Please let me know about any other performance impact you see. Thanks to
Nico for reporting and urging me to actually fix, Richard Smith, Duncan
Sands, Manuel Klimek, and Benjamin Kramer for talking through the issues
today.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153506 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-27 10:48:28 +00:00
Chandler Carruth 58725a66c0 Teach instsimplify how to simplify comparisons of pointers which are
constant-offsets of a common base using the generic GEP-walking logic
I added for computing pointer differences in the same situation.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153419 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-25 21:28:14 +00:00
Chandler Carruth 9d9e29b4a8 Switch the pointer-difference simplification logic to only work with
inbounds GEPs. This isn't really necessary for simplifying pointer
differences, but I'm planning to re-use the same code to simplify
pointer comparisons where it is necessary. Since real code almost
exclusively uses inbounds GEPs, it doesn't seem worth it to support the
extra complexity of turning it on and off. If anyone would like that
back, feel free to shout. Note that instcombine will still catch any of
these patterns.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153418 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-25 20:43:07 +00:00
Chandler Carruth f8c8a9cbb4 Teach the function cloner (and thus the inliner) to simplify PHINodes
aggressively. There are lots of dire warnings about this being expensive
that seem to predate switching to the TrackingVH-based value remapper
that is automatically updated on RAUW. This makes it easy to not just
prune single-entry PHIs, but to fully simplify PHIs, and to recursively
simplify the newly inlined code to propagate PHINode simplifications.

This introduces a bit of a thorny problem though. We may end up
simplifying a branch condition to a constant when we fold PHINodes, and
we would like to nuke any dead blocks resulting from this so that time
isn't wasted continually analyzing them, but this isn't easy. Deleting
basic blocks *after* they are fully cloned and mapped into the new
function currently requires manually updating the value map. The last
piece of the simplification-during-inlining puzzle will require either
switching to WeakVH mappings or some other piece of refactoring. I've
left a FIXME in the testcase about this.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153410 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-25 10:34:54 +00:00
Chandler Carruth d54f9a4c3b Move the instruction simplification of callsite arguments in the inliner
to instead rely on much more generic and powerful instruction
simplification in the function cloner (and thus inliner).

This teaches the pruning function cloner to use instsimplify rather than
just the constant folder to fold values during cloning. This can
simplify a large number of things that constant folding alone cannot
begin to touch. For example, it will realize that 'or' and 'and'
instructions with certain constant operands actually become constants
regardless of what their other operand is. It also can thread back
through the caller to perform simplifications that are only possible by
looking up a few levels. In particular, GEPs and pointer testing tend to
fold much more heavily with this change.

This should (in some cases) have a positive impact on compile times with
optimizations on because the inliner itself will simply avoid cloning
a great deal of code. It already attempted to prune proven-dead code,
but now it will be use the stronger simplifications to prove more code
dead.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153403 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-25 04:03:40 +00:00
Chandler Carruth acdae3e25a Add an asserting ValueHandle to the block simplification code which will
fire if anything ever invalidates the assumption of a terminator
instruction being unchanged throughout the routine.

I've convinced myself that the current definition of simplification
precludes such a transformation, so I think getting some asserts
coverage that we don't violate this agreement is sufficient to make this
code safe for the foreseeable future.

Comments to the contrary or other suggestions are of course welcome. =]
The bots are now happy with this code though, so it appears the bug here
has indeed been fixed.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153401 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-25 03:29:25 +00:00
Chandler Carruth 858cd1c33c Don't form a WeakVH around the sentinel node in the instructions BB
list. This is a bad idea. ;] I'm hopeful this is the bug that's showing
up with the MSVC bots, but we'll see.

It is definitely unnecessary. InstSimplify won't do anything to
a terminator instruction, we don't need to even include it in the
iteration range. We can also skip the now dead terminator check,
although I've made it an assert to help document that this is an
important invariant.

I'm still a bit queasy about this because there is an implicit
assumption that the terminator instruction cannot be RAUW'ed by the
simplification code. While that appears to be true at the moment, I see
no guarantee that would ensure it remains true in the future. I'm
looking at the cleanest way to solve that...

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153399 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-24 23:03:27 +00:00
Chandler Carruth 6231d5be41 Try to harden the recursive simplification still further. This is again
spotted by inspection, and I've crafted no test case that triggers it on
my machine, but some of the windows builders are hitting what looks like
memory corruption, so *something* is amiss here.

This patch takes a more generalized approach to eliminating
double-visits. Imagine code such as:

  %x = ...
  %y = add %x, 1
  %z = add %x, %y

You can imagine that if we simplify %x, we would add %y and %z to the
list. If the use-chain order happens to cause us to add them in reverse
order, we could pull %y off first, and simplify it, adding %z to the
list. We now have %z on the list twice, and will reference it after it
is deleted.

Currently, all my test cases happen to not trigger this, likely due to
the use-chain ordering, but there seems no guarantee that such
a situation could not occur, so we should handle it correctly.

Again, if anyone knows how to craft a testcase that actually triggers
this, please let me know.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153397 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-24 22:34:26 +00:00
Chandler Carruth c5b785b91c Don't add the instruction about to be RAUW'ed and erased to the
worklist. This can happen in theory when an instruction uses itself,
such as a PHI node. This was spotted by inspection, and unfortunately
I've not been able to come up with a test case that would trigger it. If
anyone has ideas, let me know...

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153396 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-24 22:34:23 +00:00
Chandler Carruth b8095464d6 FileCheck-ize this test. Note the FIXME I've introduced here: we've
regressed seriously here, we are no longer removing allocas during
inline cleanup. This appears to be because of lifetime markers "using"
them. =/ I'll look into this shortly.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153394 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-24 21:24:19 +00:00
Chandler Carruth 6b980541df Refactor the interface to recursively simplifying instructions to be tad
bit simpler by handling a common case explicitly.

Also, refactor the implementation to use a worklist based walk of the
recursive users, rather than trying to use value handles to detect and
recover from RAUWs during the recursive descent. This fixes a very
subtle bug in the previous implementation where degenerate control flow
structures could cause mutually recursive instructions (PHI nodes) to
collapse in just such a way that From became equal to To after some
amount of recursion. At that point, we hit the inf-loop that the assert
at the top attempted to guard against. This problem is defined away when
not using value handles in this manner. There are lots of comments
claiming that the WeakVH will protect against just this sort of error,
but they're not accurate about the actual implementation of WeakVHs,
which do still track RAUWs.

I don't have any test case for the bug this fixes because it requires
running the recursive simplification on unreachable phi nodes. I've no
way to either run this or easily write an input that triggers it. It was
found when using instruction simplification inside the inliner when
running over the nightly test-suite.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153393 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-24 21:11:24 +00:00
Chandler Carruth 3e29671cca Revert a series of commits to MCJIT to get the build working in CMake
(and hopefully on Windows). The bots have been down most of the day
because of this, and it's not clear to me what all will be required to
fix it.

The commits started with r153205, then r153207, r153208, and r153221.
The first commit seems to be the real culprit, but I couldn't revert
a smaller number of patches.

When resubmitting, r153207 and r153208 should be folded into r153205,
they were simple build fixes.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153241 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-22 05:44:06 +00:00
Chandler Carruth ff739c1575 Teach instsimplify to gracefully degrade in the presence of instructions
not attched to a basic block or function. There are conservatively
correct answers in these cases, and this makes the analysis more useful
in contexts where we have a partially formed bit of IR.

I don't have any way to test this directly... suggestions welcome here,
but I'm not seeing anything sadly. I only found this using a subsequent
patch to the inliner which runs instsimplify on partially inlined
instructions, and even then only on a quite large program. I never got
a reasonable testcase out of it, and anything I do get is likely to be
quite fragile due to requiring an interaction of two different passes,
and the only result being a segfault if it goes wrong.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153176 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-21 10:58:47 +00:00
Chandler Carruth a87a75fd80 Make the formatting of this file more consistent, and fix the 80-columns
violations I introduced. Also sort some of the instructions to get
a more consistent ordering.

Suggestions on still better / more consistent formatting would be
welcome. I'm actually tempted to use a macro to define all of the
delegate methods...

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153030 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-18 23:45:14 +00:00
Chandler Carruth 080b862e67 Teach InstVisitor about the UnaryInstruction layer in the instruction
type hierarchy. I wanted to use this for the inline cost rewrite, and
found it was missing.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153029 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-18 23:31:28 +00:00
Chandler Carruth b2442fc760 Rip out support for 'llvm.noinline'. This thing has a strange history...
It was added in 2007 as the first cut at supporting no-inline
attributes, but we didn't have function attributes of any form at the
time. However, it was added without any mention in the LangRef or other
documentation.

Later on, in 2008, Devang added function notes for 'inline=never' and
then turned them into proper function attributes. From that point
onward, as far as I can tell, the world moved on, and no one has touched
'llvm.noinline' in any meaningful way since.

It's time has now come. We have had better mechanisms for doing this for
a long time, all the frontends I'm aware of use them, and this is just
holding back progress. Given that it was never a documented feature of
the IR, I've provided no auto-upgrade support. If people know of real,
in-the-wild bitcode that relies on this, yell at me and I'll add it, but
I *seriously* doubt anyone cares.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152904 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-16 06:10:15 +00:00
Chandler Carruth f91f5af802 Start removing the use of an ad-hoc 'never inline' set and instead
directly query the function information which this set was representing.
This simplifies the interface of the inline cost analysis, and makes the
always-inline pass significantly more efficient.

Previously, always-inline would first make a single set of every
function in the module *except* those marked with the always-inline
attribute. It would then query this set at every call site to see if the
function was a member of the set, and if so, refuse to inline it. This
is quite wasteful. Instead, simply check the function attribute directly
when looking at the callsite.

The normal inliner also had similar redundancy. It added every function
in the module with the noinline attribute to its set to ignore, even
though inside the cost analysis function we *already tested* the
noinline attribute and produced the same result.

The only tricky part of removing this is that we have to be able to
correctly remove only the functions inlined by the always-inline pass
when finalizing, which requires a bit of a hack. Still, much less of
a hack than the set of all non-always-inline functions was. While I was
touching this function, I switched a heavy-weight set to a vector with
sort+unique. The algorithm already had a two-phase insert and removal
pattern, we were just needlessly paying the uniquing cost on every
insert.

This probably speeds up some compiles by a small amount (-O0 compiles
with lots of always-inline, so potentially heavy libc++ users), but I've
not tried to measure it.

I believe there is no functional change here, but yell if you spot one.
None are intended.

Finally, the direction this is going in is to greatly simplify the
inline cost query interface so that we can replace its implementation
with a much more clever one. Along the way, all the APIs get simplified,
so it seems incrementally good.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152903 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-16 06:10:13 +00:00
Chandler Carruth 9b081d9691 Pull the implementation of the code metrics out of the inline cost
analysis implementation. The header was already separated. Also cleanup
all the comments in the header to follow a nice modern doxygen form.

There is still plenty of cruft here, but some of that will fall out in
subsequent refactorings and this was an easy step in the right
direction. No functionality changed here.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152898 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-16 05:51:52 +00:00
Chandler Carruth 59c5d7bf2c Remove the basic inliner. This was added in 2007, and hasn't really
changed since. No one was using it. It is yet another consumer of the
InlineCost interface that I'd like to change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152769 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-15 01:37:56 +00:00
Chandler Carruth 377c7f049d Make the swap code here a bit more obvious what its doing... We're
essentially sorting the pair's arguments. I'd love to actually call sort
here, but I'm just not that crazy. ;]

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152764 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-15 00:55:51 +00:00
Chandler Carruth 0e33d9fea2 Don't assume that the arguments are processed in some particular order.
This appears to not be the case with dragonegg at least in some
contexts. Hopefully will fix the bootstrap assert failure there.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152763 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-15 00:50:21 +00:00
Chandler Carruth ae029098a5 This pass didn't want the inline cost per-se, it just wants generic code
metrics.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152760 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-15 00:29:10 +00:00
Chandler Carruth 220d2d7b50 Remove all remnants of partial specialization in the cost computation
side of things. This is all dead code.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152759 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-15 00:29:08 +00:00
Chandler Carruth 274d377ea6 Extend the inline cost calculation to account for bonuses due to
correlated pairs of pointer arguments at the callsite. This is designed
to recognize the common C++ idiom of begin/end pointer pairs when the
end pointer is a constant offset from the begin pointer. With the
C-based idiom of a pointer and size, the inline cost saw the constant
size calculation, and this provides the same level of information for
begin/end pairs.

In order to propagate this information we have to search for candidate
operations on a pair of pointer function arguments (or derived from
them) which would be simplified if the pointers had a known constant
offset. Then the callsite analysis looks for such pointer pairs in the
argument list, and applies the appropriate bonus.

This helps LLVM detect that half of bounds-checked STL algorithms
(such as hash_combine_range, and some hybrid sort implementations)
disappear when inlined with a constant size input. However, it's not
a complete fix due the inaccuracy of our cost metric for constants in
general. I'm looking into that next.

Benchmarks showed no significant code size change, and very minor
performance changes. However, specific code such as hashing is showing
significantly cleaner inlining decisions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152752 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-14 23:19:53 +00:00
Chandler Carruth b16117c368 Change where we enable the heuristic that delays inlining into functions
which are small enough to themselves be inlined. Delaying in this manner
can be harmful if the function is inelligible for inlining in some (or
many) contexts as it pessimizes the code of the function itself in the
event that inlining does not eventually happen.

Previously the check was written to only do this delaying of inlining
for static functions in the hope that they could be entirely deleted and
in the knowledge that all callers of static functions will have the
opportunity to inline if it is in fact profitable. However, with C++ we
get two other important sources of functions where the definition is
always available for inlining: inline functions and templated functions.
This patch generalizes the inliner to allow linkonce-ODR (the linkage
such C++ routines receive) to also qualify for this delay-based
inlining.

Benchmarking across a range of large real-world applications shows
roughly 2% size increase across the board, but an average speedup of
about 0.5%. Some benhcmarks improved over 2%, and the 'clang' binary
itself (when bootstrapped with this feature) shows a 1% -O0 performance
improvement when run over all Sema, Lex, and Parse source code smashed
into a single file. A clean re-build of Clang+LLVM with a bootstrapped
Clang shows approximately 2% improvement, but that measurement is often
noisy.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152737 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-14 20:16:41 +00:00
Chandler Carruth 3d1d895c86 Refactor the inline cost bonus calculation for constants to use
a worklist rather than a recursive call.

No functionality changed.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152706 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-14 07:32:53 +00:00
Chandler Carruth 90c14fcb7e Address some review comments from Duncan. This moves the iterative
offset accumulation to use a boring APInt instead of ConstantExprs.
I didn't go all the way to an 'int64_t' because I wanted APInt to handle
any magic required to properly wrap the arithmetic when the pointer
width is <64 bits. If there is a significant penalty from using APInt
here, first off WTF, and secondly let me know and I'll do the math by
hand.

I've left one layer still operating w/ ConstantExpr because it makes the
interface quite a bit simpler, and that one isn't iterative so has much
lower cost.

I suppose this may potentially speed up some strang compilation
situations, but I don't really expect much. It should have no functional
impact either way.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152590 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-13 00:06:15 +00:00
Chandler Carruth 6c0b3ac8ea When inlining a function and adding its inner call sites to the
candidate set for subsequent inlining, try to simplify the arguments to
the inner call site now that inlining has been performed.

The goal here is to propagate and fold constants through deeply nested
call chains. Without doing this, we loose the inliner bonus that should
be applied because the arguments don't match the exact pattern the cost
estimator uses.

Reviewed on IRC by Benjamin Kramer.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152556 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-12 11:19:33 +00:00
Chandler Carruth fc72ae613a Teach instsimplify how to constant fold pointer differences.
Typically instcombine has handled this, but pointer differences show up
in several contexts where we would like to get constant folding, and
cannot afford to run instcombine. Specifically, I'm working on improving
the constant folding of arguments used in inline cost analysis with
instsimplify.

Doing this in instsimplify implies some algorithm changes. We have to
handle multiple layers of all-constant GEPs because instsimplify cannot
fold them into a single GEP the way instcombine can. Also, we're only
interested in all-constant GEPs. The result is that this doesn't really
replace the instcombine logic, it's just complimentary and focused on
constant folding.

Reviewed on IRC by Benjamin Kramer.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152555 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-12 11:19:31 +00:00
Chandler Carruth 747cccf0dc FileCheck-ize this test.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152554 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-12 11:19:28 +00:00
Chandler Carruth 84dfc32ff9 Refactor some methods to look through bitcasts and GEPs on pointers into
a common collection of methods on Value, and share their implementation.
We had two variations in two different places already, and I need the
third variation for inline cost estimation.

Reviewed by Duncan Sands on IRC, but further comments here welcome.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152490 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-10 08:39:09 +00:00
Chandler Carruth 005874056e Fix a silly restriction on the fast-path for hash_combine_range. This
caused several clients to select the slow variation. =[ This is extra
annoying because we don't have any realistic way of testing this -- by
design, these two functions *must* compute the same value.

Found while inspecting the output of some benchmarks I'm working on.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152369 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-09 02:49:38 +00:00
Chandler Carruth e8187e0294 Undo a previous restriction on the inline cost calculation which Nick
introduced. Specifically, there are cost reductions for all
constant-operand icmp instructions against an alloca, regardless of
whether the alloca will in fact be elligible for SROA. That means we
don't want to abort the icmp reduction computation when we abort the
SROA reduction computation. That in turn frees us from the need to keep
a separate worklist and defer the ICmp calculations.

Use this new-found freedom and some judicious function boundaries to
factor the innards of computing the cost factor of any given instruction
out of the loop over the instructions and into static helper functions.
This greatly simplifies the code, and hopefully makes it more clear what
is happening here.

Reviewed by Eric Christopher. There is some concern that we'd like to
ensure this doesn't get out of hand, and I plan to benchmark the effects
of this change over the next few days along with some further fixes to
the inline cost.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152368 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-09 02:49:36 +00:00
Chandler Carruth 6f130bf368 Rotate two of the functions used to count bonuses for the inline cost
analysis to be methods on the cost analysis's function info object
instead of the code metrics object. These really are just users of the
code metrics, they're building the information for the function's
analysis.

This is the first step of growing the amount of information we collect
about a function in order to cope with pair-wise simplifications due to
allocas.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152283 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-08 02:04:19 +00:00
Chandler Carruth cbfc117674 Try a completely different approach to this type trait to appease older
compilers. It seems that GCC 4.3 (and likely older) simply aren't going
to do SFINAE on non-type template parameters the way Clang and modern
GCCs do...

Now we detect the implicit conversion to an integer type, and then
blacklist classes, pointers, and floating point types. This seems to
work well enough, and I'm hopeful will return the bots to life.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152227 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-07 12:27:35 +00:00
Chandler Carruth ff12877a6f Attempt #2 at appeasing GCC 4.3. This compiler really doesn't like these
traits.

With this change, the pattern used here is *extremely* close to the
pattern used elsewhere in the file, so I'm hoping it survives the
build-bots.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152225 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-07 11:31:11 +00:00
Chandler Carruth b53a1d67b1 Try to clarify this comment some.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152221 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-07 10:13:40 +00:00
Chandler Carruth a1eb50fe3d Switch the is_integral_or_enum trait machinery to use an explicit
template argument and an *implicit* conversion from '0' to a null
pointer. For some bizarre reason, GCC 4.3.2 thinks that the cast to
'(T*)' is invalid inside of an enumerator's value... which it isn't but
whatever. ;] This pattern is used elsewhere in the type_traits header
and so hopefully will survive the wrath of the build bots.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152220 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-07 10:05:35 +00:00
Chandler Carruth 4e5b0f9943 What's better than fixing and simplifying broken hash functions?
Deleting them because they aren't used. =D

Yell if you need these, I'm happy to instead replace them with nice uses
of the new infrastructure.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@152219 91177308-0d34-0410-b5e6-96231b3b80d8
2012-03-07 09:54:06 +00:00