Commit Graph

9687 Commits

Author SHA1 Message Date
Nadav Rotem
d3071bb042 Loop Vectorizer: Enable if-conversion.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170632 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-20 02:00:02 +00:00
Nadav Rotem
a6603b35a9 whitespace
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170626 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-20 00:49:56 +00:00
Paul Redmond
6da2e22dff Transform (x&C)>V into (x&C)!=0 where possible
When the least bit of C is greater than V, (x&C) must be greater than V
if it is not zero, so the comparison can be simplified.

Although this was suggested in Target/X86/README.txt, it benefits any
architecture with a directly testable form of AND.

Patch by Kevin Schoedel


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170576 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-19 19:47:13 +00:00
Evgeniy Stepanov
33660cdfbd [msan] Add track-origins argument to the pass constructor.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170544 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-19 13:55:51 +00:00
Evgeniy Stepanov
b8837ab8fc [msan] Heuristically instrument unknown intrinsics.
This changes adds shadow and origin propagation for unknown intrinsics
by examining the arguments and ModRef behaviour. For now, only 3 classes
of intrinsics are handled:
- those that look like simple SIMD store
- those that look like simple SIMD load
- those that don't have memory effects and look like arithmetic/logic/whatever
  operation on simple types.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170530 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-19 11:22:04 +00:00
Benjamin Kramer
8b7c89f2d4 LoopVectorize: Make iteration over induction variables not depend on pointer values.
MapVector is a bit heavyweight, but I don't see a simpler way. Also the
InductionList is unlikely to be large. This should help 3-stage selfhost
compares (PR14647).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170528 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-19 11:09:15 +00:00
Bill Wendling
1d3dcfe424 Inline the 'hasIncompatibleWithVarArgsAttrs' method into its only uses. And some minor comment reformatting.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170516 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-19 08:57:40 +00:00
Bill Wendling
034b94b170 Rename the 'Attributes' class to 'Attribute'. It's going to represent a single attribute in the future.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170502 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-19 07:18:57 +00:00
Shuxin Yang
d6b51d1dc1 Make sure the buffer, which containas an instance of APFloat, has proper alignment.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170486 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-19 01:10:17 +00:00
Shuxin Yang
1a3150098c rdar://12801297
InstCombine for unsafe floating-point add/sub.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170471 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-18 23:10:12 +00:00
Nadav Rotem
62570c2813 Enable the loop vectorizer in clang and not in the pass manager, so that we can disable it in clang.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170470 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-18 23:09:44 +00:00
Benjamin Kramer
0ef0e2e6d0 LoopVectorize: Emit reductions as log2(vectorsize) shuffles + vector ops instead of scalar operations.
For example on x86 with SSE4.2 a <8 x i8> add reduction becomes
	movdqa	%xmm0, %xmm1
	movhlps	%xmm1, %xmm1            ## xmm1 = xmm1[1,1]
	paddw	%xmm0, %xmm1
	pshufd	$1, %xmm1, %xmm0        ## xmm0 = xmm1[1,0,0,0]
	paddw	%xmm1, %xmm0
	phaddw	%xmm0, %xmm0
	pextrb	$0, %xmm0, %edx

instead of
	pextrb	$2, %xmm0, %esi
	pextrb	$0, %xmm0, %edx
	addb	%sil, %dl
	pextrb	$4, %xmm0, %esi
	addb	%dl, %sil
	pextrb	$6, %xmm0, %edx
	addb	%sil, %dl
	pextrb	$8, %xmm0, %esi
	addb	%dl, %sil
	pextrb	$10, %xmm0, %edi
	pextrb	$14, %xmm0, %edx
	addb	%sil, %dil
	pextrb	$12, %xmm0, %esi
	addb	%dil, %sil
	addb	%sil, %dl

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170439 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-18 18:40:20 +00:00
Nadav Rotem
04b116e08e Enable the loop vectorizer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170416 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-18 06:37:12 +00:00
Nadav Rotem
e21708e4aa SROA: Replace calls to getScalarSizeInBits to DataLayout's API because
getScalarSizeInBits could not handle vectors of pointers.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170412 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-18 05:23:31 +00:00
Rafael Espindola
f209dea96a Initialize NoRedZone and remove unused default values.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170404 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-18 03:35:05 +00:00
Chandler Carruth
b0de1e31d1 Fix another SROA crasher, PR14601.
This was a silly oversight, we weren't pruning allocas which were used
by variable-length memory intrinsics from the set that could be widened
and promoted as integers. Fix that.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170353 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-17 18:48:07 +00:00
Evgeniy Stepanov
2dfa3eb566 [msan] Fix lint warning.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170347 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-17 16:30:05 +00:00
Chandler Carruth
99a54942ae Teach the rewriting of memcpy calls to support subvector copies.
This also cleans up a bit of the memcpy call rewriting by sinking some
irrelevant code further down and making the call-emitting code a bit
more concrete.

Previously, memcpy of a subvector would actually miscompile (!!!) the
copy into a single vector element copy. I have no idea how this ever
worked. =/ This is the memcpy half of PR14478 which we probably weren't
noticing previously because it didn't actually assert.

The rewrite relies on the newly refactored insert- and extractVector
functions to do the heavy lifting, and those are the same as used for
loads and stores which makes the test coverage a bit more meaningful
here.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170338 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-17 14:51:24 +00:00
Evgeniy Stepanov
1794814198 Optimize tree walking in markAliveBlocks.
Check whether a BB is known as reachable before adding it to the worklist.
This way BB's with multiple predecessors are added to the list no more than
once.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170335 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-17 14:28:00 +00:00
Chandler Carruth
8bbff2348d Fix a secondary bug I introduced while fixing the first part of PR14478.
The first half of fixing this bug was actually in r170328, but was
entirely coincidental. It did however get me to realize the nature of
the bug, and adapt the test case to test more interesting behavior. In
turn, that uncovered the rest of the bug which I've fixed here.

This should fix two new asserts that showed up in the vectorize nightly
tester.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170333 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-17 14:03:01 +00:00
Chandler Carruth
5814a88ce9 Hoist a convertValue call to the two paths where it is needed.
I noticed this while looking at r170328. We only ever do a vector
rewrite when the alloca *is* the vector type, so it's good to not paper
over bugs here by doing a convertValue that isn't needed.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170331 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-17 13:51:03 +00:00
Chandler Carruth
7f7d201d73 Hoist the insertVector helper to be a static helper.
This will allow its use inside of memcpy rewriting as well. This routine
is more complex than extractVector, and some of its uses are not 100%
where I want them to be so there is still some work to do here.

While this can technically change the output in some cases, it shouldn't
be a change that matters -- IE, it can leave some dead code lying around
that prior versions did not, etc.

Yet another step in the refactorings leading up to the solution to the
last component of PR14478.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170328 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-17 13:41:21 +00:00
Chandler Carruth
5c531eb8b1 Lift the extractVector helper all the way out to a static helper function.
The method helpers all implicitly act upon the alloca, and what we
really want is a fully generic helper. Doing memcpy rewrites is more
special than all other rewrites because we are at times rewriting
instructions which touch pointers *other* than the alloca. As
a consequence all of the helpers needed by memcpy rewriting of
sub-vector copies will need to be generalized fully.

Note that all of these helpers ({insert,extract}{Integer,Vector}) are
woefully uncommented. I'm going to go back through and document them
once I get the factoring correct.

No functionality changed.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170325 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-17 13:07:30 +00:00
Chandler Carruth
8ab1efd5fc Factor the vector load rewriting into a more generic form.
This makes it suitable for use in rewriting memcpy in the presence of
subvector memcpy intrinsics.

No functionality changed.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170324 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-17 12:50:21 +00:00
Chandler Carruth
17c84ea594 Fix the first part of PR14478: memset now works.
PR14478 highlights a serious problem in SROA that simply wasn't being
exercised due to a lack of vector input code mixed with C-library
function calls. Part of SROA was written carefully to handle subvector
accesses via memset and memcpy, but the rewriter never grew support for
this. Fixing it required refactoring the subvector access code in other
parts of SROA so it could be shared, and then fixing the splat formation
logic and using subvector insertion (this patch).

The PR isn't quite fixed yet, as memcpy is still broken in the same way.
I'm starting on that series of patches now.

Hopefully this will be enough to bring the bullet benchmark back to life
with the bb-vectorizer enabled, but that may require fixing memcpy as
well.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170301 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-17 04:07:37 +00:00
Chandler Carruth
d6e4397a5b Extract the logic for inserting a subvector into a vector alloca.
No functionality changed. Another step of refactoring toward solving
PR14487.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170300 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-17 04:07:35 +00:00
Chandler Carruth
225d25de49 Lift the integer splat computation into a helper function.
No functionality changed. Refactoring leading up to the fix for PR14478
which requires some significant changes to the memset and memcpy
rewriting.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170299 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-17 04:07:30 +00:00
Chandler Carruth
19820053fe Relax an overly aggressive assert to fix PR14572.
The alloca width is based on the alloc size, not the type size.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170270 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-15 09:26:06 +00:00
NAKAMURA Takumi
0005093db6 Revert r170246, "Enable the loop vectorizer by default."
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170267 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-15 06:11:13 +00:00
Michael Ilseman
07acee7a09 Add back FoldOpIntoPhi optimizations with fix. Included test cases to help catch these errors and to test the presence of the optimization itself
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170248 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-14 22:08:26 +00:00
Nadav Rotem
f53e7cb9b9 Enable the loop vectorizer by default.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170246 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-14 21:30:23 +00:00
Shuxin Yang
a5ed031fbc rdar://12753946
Implement rule : "x * (select cond 1.0, 0.0) -> select cond x, 0.0"


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170226 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-14 18:46:06 +00:00
Evgeniy Stepanov
79c3742620 Fix lint warnings in MemorySanitizer.cpp.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170203 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-14 13:48:31 +00:00
Evgeniy Stepanov
63cca4e2fd [msan] Origin stores and loads do not need explicit alignment.
Origin address is always 4 byte aligned, and the access type is always i32.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170199 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-14 13:43:11 +00:00
Evgeniy Stepanov
e08878efa3 [msan] Refactor default shadow propagation and origin tracking.
This change moves the code for default shadow propagaition (handleShadowOr)
and origin tracking (setOriginForNaryOp) into a new builder-like class. Also
gets rid of handleShadowOrBinary.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170192 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-14 12:54:18 +00:00
Nadav Rotem
31bc9e00d1 revert r170166 - disable the loop vectorizer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170172 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-14 01:57:00 +00:00
Nadav Rotem
44a71c4575 Enable the loop vectorizer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170166 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-14 00:30:34 +00:00
Nadav Rotem
a97fd7f1a8 Disable the loop vectorizer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170162 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-14 00:02:07 +00:00
Nadav Rotem
9ad73e93a5 Enable the Loop Vectorizer by default for O2 and O3. Disable if-conversion by default. I plan to revert this patch later today.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170157 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-13 23:11:54 +00:00
NAKAMURA Takumi
bb9004e0dc Revert r170020, "Simplify negated bit test", for now.
This assumes (1 << n) is always not zero. Consider n is greater than word size.
Although I know it is undefined, this transforms undefined behavior hidden.

This led clang unexpected behavior with some failures. I will investigate to fix undefined shl in clang.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170128 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-13 14:28:16 +00:00
Eric Christopher
dcb024d27c Revert "Restore the PHI optimization I accidently removed" temporarily since
it seems to be breaking self-host for a few people and is PR14592.

This reverts commit r170024.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170106 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-13 06:48:05 +00:00
Rafael Espindola
99c86c72b2 Missed these calls from the previous rename somehow.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170094 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-13 03:42:31 +00:00
Rafael Espindola
dbaa2376f7 Rename isPowerOfTwo to isKnownToBeAPowerOfTwo.
In a previous thread it was pointed out that isPowerOfTwo is not a very precise
name since it can return false for powers of two if it is unable to show that
they are powers of two.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170093 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-13 03:37:24 +00:00
Michael Ilseman
8ad435fa48 Pattern matching code for intrinsics.
Provides m_Argument that allows matching against a CallSite's specified argument. Provides m_Intrinsic pattern that can be templatized over the intrinsic id and bind/match arguments similarly to other pattern matchers. Implementations provided for 0 to 4 arguments, though it's very simple to extend for more. Also provides example template specialization for bswap (m_BSwap) and example of code cleanup for its use.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170091 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-13 03:13:36 +00:00
Quentin Colombet
fcc934322b Take into account minimize size attribute in the inliner.
Better controls the inlining of functions when the caller function has MinSize attribute.
Basically, when the caller function has this attribute, we do not "force" the inlining
of callee functions carrying the InlineHint attribute (i.e., functions defined with
inline keyword)



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170065 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-13 01:05:25 +00:00
Nadav Rotem
807dad62a0 Teach the cost model about the optimization in r169904: Truncation of induction variables costs the same as scalar trunc.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170051 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-13 00:21:03 +00:00
Chad Rosier
0fabd08066 Typo.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170050 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-13 00:18:46 +00:00
Michael Ilseman
94993e37c6 Restore the PHI optimization I accidently removed
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170024 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-12 20:59:36 +00:00
Michael Ilseman
4d96e6f0d1 Remove trailing whitespace
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170022 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-12 20:57:53 +00:00
David Majnemer
63522b1998 Simplify negated bit test
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170020 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-12 20:48:54 +00:00