Commit Graph

9717 Commits

Author SHA1 Message Date
Nadav Rotem
13eb1e7817 LoopVectorizer: Optimize the vectorization of consecutive memory access when the iteration step is -1
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171114 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-26 19:08:17 +00:00
Evgeniy Stepanov
b53be53c72 [msan] Raise alignment of origin stores/loads when possible.
Origin alignment is as high as the alignment of the corresponding application
location, but never less than 4.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171110 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-26 11:55:09 +00:00
Evgeniy Stepanov
ab29644a33 [msan] Expand the file comment with track-origins info.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171109 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-26 10:59:00 +00:00
Hal Finkel
a777284158 BBVectorize: Use VTTI to compute costs for intrinsics vectorization
For the time being this includes only some dummy test cases. Once the
generic implementation of the intrinsics cost function does something other
than assuming scalarization in all cases, or some target specializes the
interface, some real test cases can be added.

Also, for consistency, I changed the type of IID from unsigned to Intrinsic::ID
in a few other places.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171079 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-26 01:36:57 +00:00
Hal Finkel
1d59f5fa53 LoopVectorize: Enable vectorization of the fmuladd intrinsic
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171076 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-25 23:21:29 +00:00
Hal Finkel
64a7a24edf BBVectorize: Enable vectorization of the fmuladd intrinsic
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171075 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-25 22:36:08 +00:00
Evgeniy Stepanov
59a65f7b24 [msan] Fix handling of vectors of pointers.
VectorType::getInteger() can not be used with them, because pointer size
depends on the target.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171070 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-25 16:04:38 +00:00
Evgeniy Stepanov
6607716368 [msan] Fix handling of select with vector condition.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171069 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-25 14:56:21 +00:00
Alexey Samsonov
4684858624 ASan: initialize callbacks from ASan module pass in a separate function for consistency
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171061 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-25 12:28:20 +00:00
Alexey Samsonov
59cca13a80 ASan: move stack poisoning logic into FunctionStackPoisoner struct
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171060 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-25 12:04:36 +00:00
Bob Wilson
a0be09f511 Add LLVMContext::emitWarning methods and use them. <rdar://problem/12867368>
When the backend is used from clang, it should produce proper diagnostics
instead of just printing messages to errs(). Other clients may also want to
register their own error handlers with the LLVMContext, and the same handler
should work for warnings in the same way as the existing emitError methods.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171041 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-24 18:15:21 +00:00
Nadav Rotem
9e5329d77e LoopVectorizer: When checking for vectorizable types, also check
the StoreInst operands.

PR14705.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171023 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-24 09:14:18 +00:00
Alexey Samsonov
3a19999413 Fix typo in comments
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171021 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-24 08:52:53 +00:00
Nadav Rotem
470ea9b72f LoopVectorizer: Fix an endless loop in the code that looks for reductions.
The bug was in the code that detects PHIs in if-then-else block sequence.

PR14701.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171008 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-24 01:22:06 +00:00
Benjamin Kramer
a1acf55738 LoopVectorize: Fix accidentaly inverted condition.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171001 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-23 13:21:41 +00:00
Benjamin Kramer
417872ed08 LoopVectorize: For scalars and void types there is no need to compute vector insert/extract costs.
Fixes an assert during the build of oggenc in the test suite.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171000 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-23 13:19:18 +00:00
Nadav Rotem
d54fed2786 Loop Vectorizer: Update the cost model of scatter/gather operations and make
them more expensive.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170995 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-23 07:23:55 +00:00
Craig Topper
8a8413d75c Remove trailing whitespace
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170990 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-22 18:09:02 +00:00
Bill Wendling
629fb82419 Change 'AttrVal' to 'AttrKind' to better reflect that it's a kind of attribute instead of the value of the attribute.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170972 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-22 00:37:52 +00:00
Roman Divacky
38b06020db Remove duplicate includes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170902 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-21 17:06:44 +00:00
Evgeniy Stepanov
3333e66822 [msan] Remove unreachable blocks before instrumenting a function.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170883 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-21 11:18:49 +00:00
Nadav Rotem
ebf395d39e Enable if-conversion.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170841 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-21 04:47:54 +00:00
Evan Cheng
a16422f11b Every pass deserves a name, even codegenprep.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170831 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-21 01:48:14 +00:00
Nadav Rotem
c2a537bd08 BB-Vectorizer: Check the cost of the store pointer type
and not the return type, which is void. A number of test
cases fail after adding the assertion in TTImpl.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170828 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-21 01:24:36 +00:00
Nadav Rotem
55306bdea5 Fix a bug in the code that checks if we can vectorize loops while using dynamic
memory bound checks.  Before the fix we were able to vectorize this loop from
the Livermore Loops benchmark:

for ( k=1 ; k<n ; k++ )
  x[k] = x[k-1] + y[k];



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170811 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-21 00:07:35 +00:00
Nadav Rotem
8386acd734 LoopVectorize: Fix a bug in the scalarization of instructions.
Before if-conversion we could check if a value is loop invariant
if it was declared inside the basic block. Now that loops have
multiple blocks this check is incorrect.

This fixes External/SPEC/CINT95/099_go/099_go



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170756 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-20 20:24:40 +00:00
Nadav Rotem
d5d46ace89 Loop Vectorizer: turn-off if-conversion.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170708 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-20 17:42:53 +00:00
James Molloy
67ae135759 Add a new attribute, 'noduplicate'. If a function contains a noduplicate call, the call cannot be duplicated - Jump threading, loop unrolling, loop unswitching, and loop rotation are inhibited if they would duplicate the call.
Similarly inlining of the function is inhibited, if that would duplicate the call (in particular inlining is still allowed when there is only one callsite and the function has internal linkage).



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170704 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-20 16:04:27 +00:00
Craig Topper
00f8102758 Formatting fixes. Remove some unnecessary 'else' after 'return'. No functional change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170676 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-20 07:15:54 +00:00
Craig Topper
41a05dc59c Removing trailing whitespace
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170675 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-20 07:09:41 +00:00
Nadav Rotem
d3071bb042 Loop Vectorizer: Enable if-conversion.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170632 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-20 02:00:02 +00:00
Nadav Rotem
a6603b35a9 whitespace
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170626 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-20 00:49:56 +00:00
Paul Redmond
6da2e22dff Transform (x&C)>V into (x&C)!=0 where possible
When the least bit of C is greater than V, (x&C) must be greater than V
if it is not zero, so the comparison can be simplified.

Although this was suggested in Target/X86/README.txt, it benefits any
architecture with a directly testable form of AND.

Patch by Kevin Schoedel


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170576 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-19 19:47:13 +00:00
Evgeniy Stepanov
33660cdfbd [msan] Add track-origins argument to the pass constructor.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170544 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-19 13:55:51 +00:00
Evgeniy Stepanov
b8837ab8fc [msan] Heuristically instrument unknown intrinsics.
This changes adds shadow and origin propagation for unknown intrinsics
by examining the arguments and ModRef behaviour. For now, only 3 classes
of intrinsics are handled:
- those that look like simple SIMD store
- those that look like simple SIMD load
- those that don't have memory effects and look like arithmetic/logic/whatever
  operation on simple types.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170530 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-19 11:22:04 +00:00
Benjamin Kramer
8b7c89f2d4 LoopVectorize: Make iteration over induction variables not depend on pointer values.
MapVector is a bit heavyweight, but I don't see a simpler way. Also the
InductionList is unlikely to be large. This should help 3-stage selfhost
compares (PR14647).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170528 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-19 11:09:15 +00:00
Bill Wendling
1d3dcfe424 Inline the 'hasIncompatibleWithVarArgsAttrs' method into its only uses. And some minor comment reformatting.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170516 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-19 08:57:40 +00:00
Bill Wendling
034b94b170 Rename the 'Attributes' class to 'Attribute'. It's going to represent a single attribute in the future.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170502 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-19 07:18:57 +00:00
Shuxin Yang
d6b51d1dc1 Make sure the buffer, which containas an instance of APFloat, has proper alignment.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170486 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-19 01:10:17 +00:00
Shuxin Yang
1a3150098c rdar://12801297
InstCombine for unsafe floating-point add/sub.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170471 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-18 23:10:12 +00:00
Nadav Rotem
62570c2813 Enable the loop vectorizer in clang and not in the pass manager, so that we can disable it in clang.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170470 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-18 23:09:44 +00:00
Benjamin Kramer
0ef0e2e6d0 LoopVectorize: Emit reductions as log2(vectorsize) shuffles + vector ops instead of scalar operations.
For example on x86 with SSE4.2 a <8 x i8> add reduction becomes
	movdqa	%xmm0, %xmm1
	movhlps	%xmm1, %xmm1            ## xmm1 = xmm1[1,1]
	paddw	%xmm0, %xmm1
	pshufd	$1, %xmm1, %xmm0        ## xmm0 = xmm1[1,0,0,0]
	paddw	%xmm1, %xmm0
	phaddw	%xmm0, %xmm0
	pextrb	$0, %xmm0, %edx

instead of
	pextrb	$2, %xmm0, %esi
	pextrb	$0, %xmm0, %edx
	addb	%sil, %dl
	pextrb	$4, %xmm0, %esi
	addb	%dl, %sil
	pextrb	$6, %xmm0, %edx
	addb	%sil, %dl
	pextrb	$8, %xmm0, %esi
	addb	%dl, %sil
	pextrb	$10, %xmm0, %edi
	pextrb	$14, %xmm0, %edx
	addb	%sil, %dil
	pextrb	$12, %xmm0, %esi
	addb	%dil, %sil
	addb	%sil, %dl

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170439 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-18 18:40:20 +00:00
Nadav Rotem
04b116e08e Enable the loop vectorizer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170416 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-18 06:37:12 +00:00
Nadav Rotem
e21708e4aa SROA: Replace calls to getScalarSizeInBits to DataLayout's API because
getScalarSizeInBits could not handle vectors of pointers.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170412 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-18 05:23:31 +00:00
Rafael Espindola
f209dea96a Initialize NoRedZone and remove unused default values.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170404 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-18 03:35:05 +00:00
Chandler Carruth
b0de1e31d1 Fix another SROA crasher, PR14601.
This was a silly oversight, we weren't pruning allocas which were used
by variable-length memory intrinsics from the set that could be widened
and promoted as integers. Fix that.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170353 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-17 18:48:07 +00:00
Evgeniy Stepanov
2dfa3eb566 [msan] Fix lint warning.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170347 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-17 16:30:05 +00:00
Chandler Carruth
99a54942ae Teach the rewriting of memcpy calls to support subvector copies.
This also cleans up a bit of the memcpy call rewriting by sinking some
irrelevant code further down and making the call-emitting code a bit
more concrete.

Previously, memcpy of a subvector would actually miscompile (!!!) the
copy into a single vector element copy. I have no idea how this ever
worked. =/ This is the memcpy half of PR14478 which we probably weren't
noticing previously because it didn't actually assert.

The rewrite relies on the newly refactored insert- and extractVector
functions to do the heavy lifting, and those are the same as used for
loads and stores which makes the test coverage a bit more meaningful
here.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170338 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-17 14:51:24 +00:00
Evgeniy Stepanov
1794814198 Optimize tree walking in markAliveBlocks.
Check whether a BB is known as reachable before adding it to the worklist.
This way BB's with multiple predecessors are added to the list no more than
once.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170335 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-17 14:28:00 +00:00
Chandler Carruth
8bbff2348d Fix a secondary bug I introduced while fixing the first part of PR14478.
The first half of fixing this bug was actually in r170328, but was
entirely coincidental. It did however get me to realize the nature of
the bug, and adapt the test case to test more interesting behavior. In
turn, that uncovered the rest of the bug which I've fixed here.

This should fix two new asserts that showed up in the vectorize nightly
tester.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170333 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-17 14:03:01 +00:00