Commit Graph

10036 Commits

Author SHA1 Message Date
Nadav Rotem
5290baacb8 PR14448 - prevent the loop vectorizer from vectorizing the same loop twice.
The LoopVectorizer often runs multiple times on the same function due to inlining.
When this happens the loop vectorizer often vectorizes the same loops multiple times, increasing code size and adding unneeded branches.
With this patch, the vectorizer during vectorization puts metadata on scalar loops and marks them as 'already vectorized' so that it knows to ignore them when it sees them a second time.

PR14448.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176399 91177308-0d34-0410-b5e6-96231b3b80d8
2013-03-02 01:33:49 +00:00
Peter Collingbourne
328d1b6500 Modify {Call,Invoke}Inst::addAttribute to take an AttrKind.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176397 91177308-0d34-0410-b5e6-96231b3b80d8
2013-03-02 01:20:18 +00:00
Benjamin Kramer
5d79bb8770 LoopVectorize: Don't hang forever if a PHI only has skipped PHI uses.
Fixes PR15384.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176366 91177308-0d34-0410-b5e6-96231b3b80d8
2013-03-01 19:07:31 +00:00
Quentin Colombet
c5a4c25b87 Fix a bug in instcombine for fmul in fast math mode.
The instcombine recognized pattern looks like:
a = b * c
d = a +/- Cst
or
a = b * c
d = Cst +/- a

When creating the new operands for fadd or fsub instruction following the related fmul, the first operand was created with the second original operand (M0 was created with C1) and the second with the first (M1 with Opnd0).

The fix consists in creating the new operands with the appropriate original operand, i.e., M0 with Opnd0 and M1 with C1.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176300 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-28 21:12:40 +00:00
Evgeniy Stepanov
5e81213969 [msan] Implement sanitize_memory attribute.
Shadow checks are disabled and memory loads always produce fully initialized
values in functions that don't have a sanitize_memory attribute. Value and
argument shadow is propagated as usual.

This change also updates blacklist behaviour to match the above.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176247 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-28 11:25:14 +00:00
Evgeniy Stepanov
14eadeda00 Remove unused leftover declarations.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176240 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-28 08:42:11 +00:00
Benjamin Kramer
2fe71f8f7e LoopVectorize: Vectorize math builtin calls.
This properly asks TargetLibraryInfo if a call is available and if it is, it
can be translated into the corresponding LLVM builtin. We don't vectorize sqrt()
yet because I'm not sure about the semantics for negative numbers. The other
intrinsic should be exact equivalents to the libm functions.

Differential Revision: http://llvm-reviews.chandlerc.com/D465

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176188 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-27 15:24:19 +00:00
Nick Lewycky
52b4edf6a1 In GCC 4.7, function names are now forbidden from .gcda files. Support this by
passing a null pointer to the function name in to GCDAProfiling, and add another
switch onto GCOVProfiling.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176173 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-27 06:22:56 +00:00
Nick Lewycky
58591b1647 Doh, fix behaviour change introduced in r176168 which is tested in clang,
not llvm.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176172 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-27 06:21:30 +00:00
Nadav Rotem
f26b4f05b3 For each function that we optimize we initialize a new list of lib functions. For each function name we malloc memory. This patch changes the Libcall map to use BumpPtrAllocator. Now we malloc only once. This speeds up instcombine by a few % on a large c++ program.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176170 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-27 05:53:43 +00:00
Nick Lewycky
bd2d1245e7 IRBuilder has grown all sorts of useful utility functions. Make use of them to
clean up this code a tiny bit. No functionality change.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176168 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-27 05:46:30 +00:00
Pedro Artigas
b3201c5cf1 Enhance integer division emulation support to handle types smaller than 32 bits,
enhancement done the trivial way; by extending inputs and truncating outputs 
which is addequate for targets with little or no support for integer arithmetic
on integer types less than 32 bits.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176139 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-26 23:33:20 +00:00
Kostya Serebryany
8eec41fc77 Unify clang/llvm attributes for asan/tsan/msan (LLVM part)
These are two related changes (one in llvm, one in clang).
LLVM: 
- rename address_safety => sanitize_address (the enum value is the same, so we preserve binary compatibility with old bitcode)
- rename thread_safety => sanitize_thread
- rename no_uninitialized_checks -> sanitize_memory

CLANG: 
- add __attribute__((no_sanitize_address)) as a synonym for __attribute__((no_address_safety_analysis))
- add __attribute__((no_sanitize_thread))
- add __attribute__((no_sanitize_memory))

for S in address thread memory
If -fsanitize=S is present and __attribute__((no_sanitize_S)) is not
set llvm attribute sanitize_S


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176075 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-26 06:58:09 +00:00
Benjamin Kramer
e8aa36a4af CVP: If we have a PHI with an incoming select, try to skip the select.
This is a common pattern with dyn_cast and similar constructs, when the
PHI no longer depends on the select it can often be turned into a simpler
construct or even get hoisted out of the loop.

PR15340.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175995 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-24 15:34:43 +00:00
Michael Gottesman
3603e9aa5e Fixed a careless mistake.
rdar://13273675.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175939 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-23 00:31:32 +00:00
Bill Wendling
143d46476c Implement the NoBuiltin attribute.
The 'nobuiltin' attribute is applied to call sites to indicate that LLVM should
not treat the callee function as a built-in function. I.e., it shouldn't try to
replace that function with different code.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175835 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-22 00:12:35 +00:00
Renato Golin
e18bce5317 Allow GlobalValues to vectorize with AliasAnalysis
Storing the load/store instructions with the values
and inspect them using Alias Analysis to make sure
they don't alias, since the GEP pointer operand doesn't
take the offset into account.

Trying hard to not add any extra cost to loads and stores
that don't overlap on global values, AA is *only* calculated
if all of the previous attempts failed.

Using biggest vector register size as the stride for the
vectorization access, as we're being conservative and
the cost model (which calculates the real vectorization
factor) is only run after the legalization phase.

We might re-think this relationship in the future, but
for now, I'd rather be safe than sorry.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175818 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-21 22:39:03 +00:00
Chad Rosier
5e5529c74c Remove dead code and whitespace.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175804 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-21 21:40:51 +00:00
Chad Rosier
6aecad1074 Update a comment that looks to have been accidentally deleted many moons ago.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175658 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-20 20:15:55 +00:00
Kostya Serebryany
1479c9bb39 [asan] instrument invoke insns with noreturn attribute (as well as call insns)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175617 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-20 12:35:15 +00:00
Jakub Staszak
bcff7b7734 Remove unused variable.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175568 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-19 22:17:58 +00:00
Jakub Staszak
50573b1c27 Minor cleanups. No functionality change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175567 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-19 22:14:45 +00:00
Jakub Staszak
4263ed33a7 Remove unneeded #includes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175565 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-19 22:06:38 +00:00
Jakub Staszak
ba6f722d6a Fix typos.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175562 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-19 22:02:21 +00:00
Kostya Serebryany
6ecccdbb2b [asan] instrument memory accesses with unusual sizes
This patch makes asan instrument memory accesses with unusual sizes (e.g. 5 bytes or 10 bytes), e.g. long double or
packed structures.
Instrumentation is done with two 1-byte checks
(first and last bytes) and if the error is found
__asan_report_load_n(addr, real_size) or
__asan_report_store_n(addr, real_size)
is called.

Also, call these two new functions in memset/memcpy
instrumentation.

asan-rt part will follow.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175507 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-19 11:29:21 +00:00
Bill Wendling
df8a668e3f Temporarily revert r175470 for more review.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175476 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-19 00:52:45 +00:00
Bill Wendling
2af5035a1d Check to see if the 'no-builtin' attribute is set before simplifying a library call.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175470 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-18 23:17:16 +00:00
Kostya Serebryany
605ff6655b [asan] revert r175266 as it breaks code with packed structures. supporting long double will require a more general solution
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175442 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-18 13:47:02 +00:00
Hal Finkel
f79f136cc6 BBVectorize: Fix an invalid reference bug
This fixes PR15289. This bug was introduced (recently) in r175215; collecting
all std::vector references for candidate pairs to delete at once is invalid
because subsequent lookups in the owning DenseMap could invalidate the
references.

bugpoint was able to reduce a useful test case. Unfortunately, because whether
or not this asserts depends on memory layout, this test case will sometimes
appear to produce valid output. Nevertheless, running under valgrind will
reveal the error.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175397 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-17 15:59:26 +00:00
Bill Wendling
cc54889cd5 The transform is:
(or (bool?A:B),(bool?C:D)) --> (bool?(or A,C):(or B,D))

By the time the OR is visited, both the SELECTs have been visited and not
optimized and the OR itself hasn't been transformed so we do this transform in
the hopes that the new ORs will be optimized.

The transform is explicitly disabled for vector-selects until "codegen matures
to handle them better".

Patch by Muhammad Tauqir!


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175380 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-16 23:41:36 +00:00
Jakub Staszak
71d6a79ada Reduce indents in LSRInstance::NarrowSearchSpaceByCollapsingUnrolledCode method.
No functionality change.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175364 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-16 16:08:15 +00:00
Hal Finkel
f64a7a83be BBVectorize: Call a DAG and DAG instead of a tree
Several functions and variable names used the term 'tree' to refer
to what is actually a DAG. Correcting this mistake will, hopefully,
prevent confusion in the future.

No functionality change intended.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175278 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-15 17:20:54 +00:00
Arnaud A. de Grandmaison
ad079b274d Fix refactoring mistake in "Teach InstCombine to work with smaller legal types..."
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175273 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-15 15:18:17 +00:00
Arnaud A. de Grandmaison
7c5c9b39c9 Teach InstCombine to work with smaller legal types in icmp (shl %v, C1), C2
It enables to work with a smaller constant, which is target friendly for those which can compare to immediates.
It also avoids inserting a shift in favor of a trunc, which can be free on some targets.

This used to work until LLVM-3.1, but regressed with the 3.2 release.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175270 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-15 14:35:47 +00:00
Kostya Serebryany
74ecd214e2 [asan] support long double on 64-bit. See https://code.google.com/p/address-sanitizer/issues/detail?id=151
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175266 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-15 12:46:06 +00:00
Benjamin Kramer
74b3c8da48 Make helpers static. Add missing include so LLVMInitializeObjCARCOpts gets C linkage.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175264 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-15 12:30:38 +00:00
Hal Finkel
ab90084bca BBVectorize: Cap the number of candidate pairs in each instruction group
For some basic blocks, it is possible to generate many candidate pairs for
relatively few pairable instructions. When many (tens of thousands) of these pairs
are generated for a single instruction group, the time taken to generate and
rank the different vectorization plans can become quite large. As a result, we now
cap the number of candidate pairs within each instruction group. This is done by
closing out the group once the threshold is reached (set now at 3000 pairs).

Although this will limit the overall compile-time impact, this may not be the best
way to achieve this result. It might be better, for example, to prune excessive
candidate pairs after the fact the prevent the generation of short, but highly-connected
groups. We can experiment with this in the future.

This change reduces the overall compile-time slowdown of the csa.ll test case in
PR15222 to ~5x. If 5x is still considered too large, a lower limit can be
used as the default.

This represents a functionality change, but only for very large inputs
(thus, there is no regression test).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175251 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-15 04:28:42 +00:00
Hal Finkel
97a241b173 BBVectorize: Remove the remaining instances of std::multimap
All instances of std::multimap have now been replaced by
DenseMap<K, std::vector<V> >, and this yields a speedup of 5% on the
csa.ll test case from PR15222.

No functionality change intended.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175216 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-14 22:38:04 +00:00
Hal Finkel
6ca6d3b1ea BBVectorize: Don't store candidate pairs in a std::multimap
This is another commit on the road to removing std::multimap from
BBVectorize. This gives an ~1% speedup on the csa.ll test case
in PR15222.

No functionality change intended.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175215 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-14 22:37:09 +00:00
Bill Wendling
17fe48ce6e Retain the name of the new internal global that's been shrunk.
It's possible (e.g. after an LTO build) that an internal global may be used for
debugging purposes. If that's the case appending a '.b' to it makes it hard to
find that variable. Steal the name from the old GV before deleting it so that
they can find that variable again.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175104 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-13 23:00:51 +00:00
Benjamin Kramer
c0a6e070fc LoopVectorize: Simplify code for clarity.
No functionality change.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175076 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-13 21:12:29 +00:00
Pekka Jaaskelainen
5d0ce79e26 Metadata for annotating loops as parallel. The first consumer for this
metadata is the loop vectorizer.

See the documentation update for more info.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175060 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-13 18:08:57 +00:00
Kostya Serebryany
39f02940ba [asan] fix confusing indentation
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175033 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-13 05:14:12 +00:00
Arnaud A. de Grandmaison
66bff1eec9 Fix comment
visitSExt is an adapted copy of the related visitZExt method, so adapt the comment accordingly.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175019 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-13 00:19:19 +00:00
Michael Gottesman
49b5bb4cb4 Changed isStoredObjCPointer => IsStoredObjCPointer. No functionality change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175017 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-12 23:35:08 +00:00
Dan Gohman
8a8de9889d Actually delete this code, since it's really not clear what it's
trying to do.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175014 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-12 22:26:41 +00:00
Dan Gohman
5f3c4a3910 Record PRE predecessors with a SmallVector instead of a DenseMap, and
avoid a second pred_iterator traversal.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@175001 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-12 19:49:10 +00:00
Dan Gohman
8c0d29fee9 When disabling PRE for a value is directly redundant with itself
(through a loop), don't continue to iterate through the reamining
predecessors.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174994 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-12 19:05:10 +00:00
Dan Gohman
c73b96a99f Check that pointers are removed from maps before calling delete on the pointers,
for tidiness' sake.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174988 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-12 18:44:43 +00:00
Dan Gohman
67cd669f7b Minor code simplification.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174985 91177308-0d34-0410-b5e6-96231b3b80d8
2013-02-12 18:38:36 +00:00