being present. Make a member of one of the helper classes a reference as
part of this.
Reformatting goodness brought to you by clang-format.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171726 91177308-0d34-0410-b5e6-96231b3b80d8
This makes the loop vectorizer match the pattern followed by roughly all
other passses. =]
Notably, this header file was braken in several regards: it contained
a using namespace directive, global #define's that aren't globaly
appropriate, and global constants defined directly in the header file.
As a side benefit, lots of the types in this file become internal, which
will cause the optimizer to chew on this pass more effectively.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171723 91177308-0d34-0410-b5e6-96231b3b80d8
This could be simplified further, but Hal has a specific feature for
ignoring TTI, and so I preserved that.
Also, I needed to use it because a number of tests fail when switching
from a null TTI to the NoTTI nonce implementation. That seems suspicious
to me and so may be something that you need to look into Hal. I worked
it by preserving the old behavior for these tests with the flag that
ignores all target info.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171722 91177308-0d34-0410-b5e6-96231b3b80d8
this patch brought to you by the tool clang-format.
I wanted to fix up the names of constructor parameters because they
followed a bit of an anti-pattern by naming initialisms with CamelCase:
'Tti', 'Se', etc. This appears to have been in an attempt to not overlap
with the names of member variables 'TTI', 'SE', etc. However,
constructor arguments can very safely alias members, and in fact that's
the conventional way to pass in members. I've fixed all of these I saw,
along with making some strang abbreviations such as 'Lp' be simpler 'L',
or 'Lgl' be the word 'Legal'.
However, the code I was touching had indentation and formatting somewhat
all over the map. So I ran clang-format and fixed them.
I also fixed a few other formatting or doxygen formatting issues such as
using ///< on trailing comments so they are associated with the correct
entry.
There is still a lot of room for improvement of the formating and
cleanliness of this code. ;] At least a few parts of the coding
standards or common practices in LLVM's code aren't followed, the enum
naming rules jumped out at me. I may mix some of these while I'm here,
but not all of them.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171719 91177308-0d34-0410-b5e6-96231b3b80d8
I'm sorry for duplicating bad style here, but I wanted to keep
consistency. I've pinged the code review thread where this style was
reviewed and changes were requested.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171714 91177308-0d34-0410-b5e6-96231b3b80d8
through as a reference rather than a pointer. There is always *some*
implementation of this available, so this simplifies code by not having
to test for whether it is available or not.
Further, it turns out there were piles of places where SimplifyCFG was
recursing and not passing down either TD or TTI. These are fixed to be
more pedantically consistent even though I don't have any particular
cases where it would matter.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171691 91177308-0d34-0410-b5e6-96231b3b80d8
The reason that there are two cases is that the first case handles the unary cases and the second the binary cases.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171672 91177308-0d34-0410-b5e6-96231b3b80d8
next to its only user. This helper relies on TargetLowering information
that shouldn't be generally used throughout the Transfoms library, and
so it made little sense as a generic utility.
This also consolidates the file where we need to remove the remaining
uses of TargetLowering in favor of the IR-layer abstract interface in
TargetTransformInfo.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171590 91177308-0d34-0410-b5e6-96231b3b80d8
Since subtraction does not commute the loop vectorizer incorrectly vectorizes
reductions such as x = A[i] - x.
Disabling for now.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171537 91177308-0d34-0410-b5e6-96231b3b80d8
1. Add code to estimate register pressure.
2. Add code to select the unroll factor based on register pressure.
3. Add bits to TargetTransformInfo to provide the number of registers.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171469 91177308-0d34-0410-b5e6-96231b3b80d8
code that includes Intrinsics.gen directly.
This never showed up in my testing because the old Intrinsics.gen was
still kicking around in the make build system and was correct there. =[
Thankfully, some of the bots to clean rebuilds and that caught this.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171373 91177308-0d34-0410-b5e6-96231b3b80d8
into their new header subdirectory: include/llvm/IR. This matches the
directory structure of lib, and begins to correct a long standing point
of file layout clutter in LLVM.
There are still more header files to move here, but I wanted to handle
them in separate commits to make tracking what files make sense at each
layer easier.
The only really questionable files here are the target intrinsic
tablegen files. But that's a battle I'd rather not fight today.
I've updated both CMake and Makefile build systems (I think, and my
tests think, but I may have missed something).
I've also re-sorted the includes throughout the project. I'll be
committing updates to Clang, DragonEgg, and Polly momentarily.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171366 91177308-0d34-0410-b5e6-96231b3b80d8
utils/sort_includes.py script.
Most of these are updating the new R600 target and fixing up a few
regressions that have creeped in since the last time I sorted the
includes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171362 91177308-0d34-0410-b5e6-96231b3b80d8
Specifically these calls return their argument verbatim, as a low-level
optimization. However, this makes high-level optimizations
harder. We undo any uses of this optimization that the front-end
emitted. We redo them later in the contract pass.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171346 91177308-0d34-0410-b5e6-96231b3b80d8
The later API is nicer than the former, and is correct regarding wrap-around offsets (if anyone cares).
There are a few more places left with duplicated code, which I'll remove soon.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171259 91177308-0d34-0410-b5e6-96231b3b80d8
directly.
This is in preparation for removing the use of the 'Attribute' class as a
collection of attributes. That will shift to the AttributeSet class instead.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171253 91177308-0d34-0410-b5e6-96231b3b80d8
LCSSA PHIs may have undef values. The vectorizer updates values that are used by outside users such as PHIs.
The bug happened because undefs are not loop values. This patch handles these PHIs.
PR14725
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171251 91177308-0d34-0410-b5e6-96231b3b80d8
such as by a compiler warning, a check in clang -fsanitizer=undefined, being
optimized to unreachable, or a combination of the above. PR14722.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171119 91177308-0d34-0410-b5e6-96231b3b80d8
Origin alignment is as high as the alignment of the corresponding application
location, but never less than 4.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171110 91177308-0d34-0410-b5e6-96231b3b80d8
For the time being this includes only some dummy test cases. Once the
generic implementation of the intrinsics cost function does something other
than assuming scalarization in all cases, or some target specializes the
interface, some real test cases can be added.
Also, for consistency, I changed the type of IID from unsigned to Intrinsic::ID
in a few other places.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171079 91177308-0d34-0410-b5e6-96231b3b80d8
When the backend is used from clang, it should produce proper diagnostics
instead of just printing messages to errs(). Other clients may also want to
register their own error handlers with the LLVMContext, and the same handler
should work for warnings in the same way as the existing emitError methods.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171041 91177308-0d34-0410-b5e6-96231b3b80d8
and not the return type, which is void. A number of test
cases fail after adding the assertion in TTImpl.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170828 91177308-0d34-0410-b5e6-96231b3b80d8
memory bound checks. Before the fix we were able to vectorize this loop from
the Livermore Loops benchmark:
for ( k=1 ; k<n ; k++ )
x[k] = x[k-1] + y[k];
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170811 91177308-0d34-0410-b5e6-96231b3b80d8
Before if-conversion we could check if a value is loop invariant
if it was declared inside the basic block. Now that loops have
multiple blocks this check is incorrect.
This fixes External/SPEC/CINT95/099_go/099_go
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170756 91177308-0d34-0410-b5e6-96231b3b80d8
Similarly inlining of the function is inhibited, if that would duplicate the call (in particular inlining is still allowed when there is only one callsite and the function has internal linkage).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170704 91177308-0d34-0410-b5e6-96231b3b80d8
When the least bit of C is greater than V, (x&C) must be greater than V
if it is not zero, so the comparison can be simplified.
Although this was suggested in Target/X86/README.txt, it benefits any
architecture with a directly testable form of AND.
Patch by Kevin Schoedel
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170576 91177308-0d34-0410-b5e6-96231b3b80d8
This changes adds shadow and origin propagation for unknown intrinsics
by examining the arguments and ModRef behaviour. For now, only 3 classes
of intrinsics are handled:
- those that look like simple SIMD store
- those that look like simple SIMD load
- those that don't have memory effects and look like arithmetic/logic/whatever
operation on simple types.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170530 91177308-0d34-0410-b5e6-96231b3b80d8
MapVector is a bit heavyweight, but I don't see a simpler way. Also the
InductionList is unlikely to be large. This should help 3-stage selfhost
compares (PR14647).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170528 91177308-0d34-0410-b5e6-96231b3b80d8
This was a silly oversight, we weren't pruning allocas which were used
by variable-length memory intrinsics from the set that could be widened
and promoted as integers. Fix that.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170353 91177308-0d34-0410-b5e6-96231b3b80d8
This also cleans up a bit of the memcpy call rewriting by sinking some
irrelevant code further down and making the call-emitting code a bit
more concrete.
Previously, memcpy of a subvector would actually miscompile (!!!) the
copy into a single vector element copy. I have no idea how this ever
worked. =/ This is the memcpy half of PR14478 which we probably weren't
noticing previously because it didn't actually assert.
The rewrite relies on the newly refactored insert- and extractVector
functions to do the heavy lifting, and those are the same as used for
loads and stores which makes the test coverage a bit more meaningful
here.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170338 91177308-0d34-0410-b5e6-96231b3b80d8
Check whether a BB is known as reachable before adding it to the worklist.
This way BB's with multiple predecessors are added to the list no more than
once.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170335 91177308-0d34-0410-b5e6-96231b3b80d8
The first half of fixing this bug was actually in r170328, but was
entirely coincidental. It did however get me to realize the nature of
the bug, and adapt the test case to test more interesting behavior. In
turn, that uncovered the rest of the bug which I've fixed here.
This should fix two new asserts that showed up in the vectorize nightly
tester.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170333 91177308-0d34-0410-b5e6-96231b3b80d8
I noticed this while looking at r170328. We only ever do a vector
rewrite when the alloca *is* the vector type, so it's good to not paper
over bugs here by doing a convertValue that isn't needed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170331 91177308-0d34-0410-b5e6-96231b3b80d8
This will allow its use inside of memcpy rewriting as well. This routine
is more complex than extractVector, and some of its uses are not 100%
where I want them to be so there is still some work to do here.
While this can technically change the output in some cases, it shouldn't
be a change that matters -- IE, it can leave some dead code lying around
that prior versions did not, etc.
Yet another step in the refactorings leading up to the solution to the
last component of PR14478.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170328 91177308-0d34-0410-b5e6-96231b3b80d8
The method helpers all implicitly act upon the alloca, and what we
really want is a fully generic helper. Doing memcpy rewrites is more
special than all other rewrites because we are at times rewriting
instructions which touch pointers *other* than the alloca. As
a consequence all of the helpers needed by memcpy rewriting of
sub-vector copies will need to be generalized fully.
Note that all of these helpers ({insert,extract}{Integer,Vector}) are
woefully uncommented. I'm going to go back through and document them
once I get the factoring correct.
No functionality changed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170325 91177308-0d34-0410-b5e6-96231b3b80d8
This makes it suitable for use in rewriting memcpy in the presence of
subvector memcpy intrinsics.
No functionality changed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170324 91177308-0d34-0410-b5e6-96231b3b80d8
PR14478 highlights a serious problem in SROA that simply wasn't being
exercised due to a lack of vector input code mixed with C-library
function calls. Part of SROA was written carefully to handle subvector
accesses via memset and memcpy, but the rewriter never grew support for
this. Fixing it required refactoring the subvector access code in other
parts of SROA so it could be shared, and then fixing the splat formation
logic and using subvector insertion (this patch).
The PR isn't quite fixed yet, as memcpy is still broken in the same way.
I'm starting on that series of patches now.
Hopefully this will be enough to bring the bullet benchmark back to life
with the bb-vectorizer enabled, but that may require fixing memcpy as
well.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170301 91177308-0d34-0410-b5e6-96231b3b80d8
No functionality changed. Refactoring leading up to the fix for PR14478
which requires some significant changes to the memset and memcpy
rewriting.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170299 91177308-0d34-0410-b5e6-96231b3b80d8
This change moves the code for default shadow propagaition (handleShadowOr)
and origin tracking (setOriginForNaryOp) into a new builder-like class. Also
gets rid of handleShadowOrBinary.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170192 91177308-0d34-0410-b5e6-96231b3b80d8
This assumes (1 << n) is always not zero. Consider n is greater than word size.
Although I know it is undefined, this transforms undefined behavior hidden.
This led clang unexpected behavior with some failures. I will investigate to fix undefined shl in clang.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170128 91177308-0d34-0410-b5e6-96231b3b80d8
In a previous thread it was pointed out that isPowerOfTwo is not a very precise
name since it can return false for powers of two if it is unable to show that
they are powers of two.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170093 91177308-0d34-0410-b5e6-96231b3b80d8
Provides m_Argument that allows matching against a CallSite's specified argument. Provides m_Intrinsic pattern that can be templatized over the intrinsic id and bind/match arguments similarly to other pattern matchers. Implementations provided for 0 to 4 arguments, though it's very simple to extend for more. Also provides example template specialization for bswap (m_BSwap) and example of code cleanup for its use.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170091 91177308-0d34-0410-b5e6-96231b3b80d8
Better controls the inlining of functions when the caller function has MinSize attribute.
Basically, when the caller function has this attribute, we do not "force" the inlining
of callee functions carrying the InlineHint attribute (i.e., functions defined with
inline keyword)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170065 91177308-0d34-0410-b5e6-96231b3b80d8
been used in the first place. It simply was passed to the function and to the
recursive invocations. Simply drop the parameter and update the callers for the
new signature.
Patch by Saleem Abdulrasool!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169988 91177308-0d34-0410-b5e6-96231b3b80d8
When ASan replaces <alloca instruction> with
<offset into a common large alloca>, it should also patch
llvm.dbg.declare calls and replace debug info descriptors to mark
that we've replaced alloca with a value that stores an address
of the user variable, not the user variable itself.
See PR11818 for more context.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169984 91177308-0d34-0410-b5e6-96231b3b80d8
