69 Commits

Author SHA1 Message Date
Nadav Rotem
7b15c0afc1 Enable late-vectorization by default.
This patch changes the default setting for the LateVectorization flag that controls where the loop-vectorizer is ran.

Perf gains:
SingleSource/Benchmarks/Shootout/matrix -37.33%
MultiSource/Benchmarks/PAQ8p/paq8p  -22.83%
SingleSource/Benchmarks/Linpack/linpack-pc  -16.22%
SingleSource/Benchmarks/Shootout-C++/ary3 -15.16%
MultiSource/Benchmarks/TSVC/NodeSplitting-flt/NodeSplitting-flt -10.34%
MultiSource/Benchmarks/TSVC/NodeSplitting-dbl/NodeSplitting-dbl -7.12%

Regressions:
SingleSource/Benchmarks/Misc/lowercase  15.10%
MultiSource/Benchmarks/TSVC/Equivalencing-flt/Equivalencing-flt 13.18%
SingleSource/Benchmarks/Shootout-C++/matrix 8.27%
SingleSource/Benchmarks/CoyoteBench/lpbench 7.30%



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189858 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-03 21:33:17 +00:00
Bill Wendling
6e4d93b232 Random cleanup: No need to use a std::vector here, since createInternalizePass uses an ArrayRef.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189632 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-30 00:48:37 +00:00
Nadav Rotem
f1cd7983b1 Vectorizer/PassManager: I am working on moving the vectorizer out of the SCC passes. This patch moves the SLP-vectorizer and BB-vectorizer back into SCC passes for two reasons:
1. They are a kind of cannonicalization.
2. The performance measurements show that it is better to keep them in.

There should be no functional change if you are not enabling the LateVectorization mode.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189539 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-28 23:40:29 +00:00
Hal Finkel
435798e96a Disable unrolling in the loop vectorizer when disabled in the pass manager
When unrolling is disabled in the pass manager, the loop vectorizer should also
not unroll loops. This will allow the -fno-unroll-loops option in Clang to
behave as expected (even for vectorizable loops). The loop vectorizer's
-force-vector-unroll option will (continue to) override the pass-manager
setting (including -force-vector-unroll=0 to force use of the internal
auto-selection logic).

In order to test this, I added a flag to opt (-disable-loop-unrolling) to force
disable unrolling through opt (the analog of -fno-unroll-loops in Clang). Also,
this fixes a small bug in opt where the loop vectorizer was enabled only after
the pass manager populated the queue of passes (the global_alias.ll test needed
a slight update to the RUN line as a result of this fix).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189499 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-28 18:33:10 +00:00
Arnold Schwaighofer
ceda68fe24 Also remove logic in LateVectorize
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188285 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-13 16:12:04 +00:00
Arnold Schwaighofer
b6171c5296 Remove logic that decides whether to vectorize or not depending on O-levels
I have moved this logic into clang and opt.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188281 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-13 15:51:25 +00:00
Tom Stellard
01d7203ef8 Factor FlattenCFG out from SimplifyCFG
Patch by: Mei Ye

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187764 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-06 02:43:45 +00:00
Nadav Rotem
72bc423e7f Move the optlevel check to the frontend.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187628 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-01 22:41:58 +00:00
Nadav Rotem
a4cbd78af8 Only enable SLP-vectorization on O3 builds.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187595 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-01 18:28:15 +00:00
Tom Stellard
57e6b2d1f3 SimplifyCFG: Use parallel-and and parallel-or mode to consolidate branch conditions
Merge consecutive if-regions if they contain identical statements.
Both transformations reduce number of branches.  The transformation
is guarded by a target-hook, and is currently enabled only for +R600,
but the correctness has been tested on X86 target using a variety of
CPU benchmarks.

Patch by: Mei Ye

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187278 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-27 00:01:07 +00:00
Chandler Carruth
6bf3a05235 Add a flag to defer vectorization into a phase after the inliner and its
CGSCC pass manager. This should insulate the inlining decisions from the
vectorization decisions, however it may have both compile time and code
size problems so it is just an experimental option right now.

Adding this based on a discussion with Arnold and it seems at least
worth having this flag for us to both run some experiments to see if
this strategy is workable. It may solve some of the regressions seen
with the loop vectorizer.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184698 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-24 07:21:47 +00:00
Meador Inge
be87bce32b Remove the simplify-libcalls pass (finally)
This commit completely removes what is left of the simplify-libcalls
pass.  All of the functionality has now been migrated to the instcombine
and functionattrs passes.  The following C API functions are now NOPs:

  1. LLVMAddSimplifyLibCallsPass
  2. LLVMPassManagerBuilderSetDisableSimplifyLibCalls

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184459 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-20 19:48:07 +00:00
Nadav Rotem
7459ff4be2 Disable vectorization for -Oz.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184089 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-17 17:22:40 +00:00
Nadav Rotem
f9f8681286 Enable the loop vectorizer by default for -Os and -O2.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@184084 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-17 16:23:34 +00:00
Nadav Rotem
9342b9ccdd Jeffrey Yasskin volunteered to benchmark the vectorizer on -O2 or -Os when compiling chrome. This patch adds a new flag to enable vectorization on all levels and not only on -O3. It should go away once we make a decision.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@183456 91177308-0d34-0410-b5e6-96231b3b80d8
2013-06-06 22:35:47 +00:00
Filip Pizlo
40be1e8566 This patch breaks up Wrap.h so that it does not have to include all of
the things, and renames it to CBindingWrapping.h.  I also moved 
CBindingWrapping.h into Support/.

This new file just contains the macros for defining different wrap/unwrap 
methods.

The calls to those macros, as well as any custom wrap/unwrap definitions 
(like for array of Values for example), are put into corresponding C++ 
headers.

Doing this required some #include surgery, since some .cpp files relied 
on the fact that including Wrap.h implicitly caused the inclusion of a 
bunch of other things.

This also now means that the C++ headers will include their corresponding 
C API headers; for example Value.h must include llvm-c/Core.h.  I think 
this is harmless, since the C API headers contain just external function 
declarations and some C types, so I don't believe there should be any 
nasty dependency issues here.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180881 91177308-0d34-0410-b5e6-96231b3b80d8
2013-05-01 20:59:00 +00:00
Eric Christopher
3e39731e88 Move C++ code out of the C headers and into either C++ headers
or the C++ files themselves. This enables people to use
just a C compiler to interoperate with LLVM.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@180063 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-22 22:47:22 +00:00
Nadav Rotem
e9a4411db4 SLPVectorizer: Make it a function pass and add code for hoisting the vector-gather sequence out of loops.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179562 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-15 22:00:26 +00:00
Nadav Rotem
1129a832e6 Add an option -vectorize-slp-aggressive for running the BB vectorizer. Make -fslp-vectorize run the slp-vectorizer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179508 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-15 05:39:58 +00:00
Nadav Rotem
8849838965 Rename the slp-vectorizer clang/llvm flags. No functionality change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179505 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-15 04:54:42 +00:00
Nick Lewycky
0ebc084132 Use LLVMBool instead of 'bool' in the C API. Based on a patch by Peter Zotov!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176793 91177308-0d34-0410-b5e6-96231b3b80d8
2013-03-10 21:58:22 +00:00
Andrew Trick
76c25dc2bf Generalize my previous fix for -print-options.
Always print options that differ from their implicit default. At least
for simple option types.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176572 91177308-0d34-0410-b5e6-96231b3b80d8
2013-03-06 19:04:56 +00:00
Andrew Trick
122136b119 Give -loop-vectorize an explicit default.
This way, clang -mllvm -print-options shows that the driver is overriding it.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@176569 91177308-0d34-0410-b5e6-96231b3b80d8
2013-03-06 18:22:22 +00:00
Hal Finkel
c0b3d4c9c2 Unroll again after running BBVectorize
Because BBVectorize may significantly shorten a loop body, unroll
again after vectorization. This is especially important when using
runtime or partial unrolling.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@173730 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-29 00:22:49 +00:00
Chandler Carruth
1ada2ada3c Remove the long defunct 'DefaultPasses' header. We have a pass manager
builder these days, and this thing hasn't seen updates for a very long
time.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171741 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-07 15:16:50 +00:00
Nadav Rotem
b7c772922e Move the loop vectorizer from O2 to O3. It looks like the increase in code size actually hurts the performance on many programs.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171471 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-04 17:57:44 +00:00
Roman Divacky
38b06020db Remove duplicate includes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170902 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-21 17:06:44 +00:00
Nadav Rotem
62570c2813 Enable the loop vectorizer in clang and not in the pass manager, so that we can disable it in clang.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170470 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-18 23:09:44 +00:00
Nadav Rotem
04b116e08e Enable the loop vectorizer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170416 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-18 06:37:12 +00:00
NAKAMURA Takumi
0005093db6 Revert r170246, "Enable the loop vectorizer by default."
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170267 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-15 06:11:13 +00:00
Nadav Rotem
f53e7cb9b9 Enable the loop vectorizer by default.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170246 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-14 21:30:23 +00:00
Nadav Rotem
31bc9e00d1 revert r170166 - disable the loop vectorizer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170172 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-14 01:57:00 +00:00
Nadav Rotem
44a71c4575 Enable the loop vectorizer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170166 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-14 00:30:34 +00:00
Nadav Rotem
a97fd7f1a8 Disable the loop vectorizer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170162 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-14 00:02:07 +00:00
Nadav Rotem
9ad73e93a5 Enable the Loop Vectorizer by default for O2 and O3. Disable if-conversion by default. I plan to revert this patch later today.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170157 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-13 23:11:54 +00:00
Nadav Rotem
ae3b652f5c LoopVectorizer: Use the "optsize" attribute to decide if we are allowed to increase the function size.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170004 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-12 19:29:45 +00:00
Nadav Rotem
5da804150d LoopVectorizer: When -Os is used, vectorize only loops that dont require a tail loop. There is no testcase because I dont know of a way to initialize the loop vectorizer pass without adding an additional hidden flag.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169950 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-12 01:11:46 +00:00
Nadav Rotem
9e81a440f5 Enable the loop vectorizer only on O2 and above. (Still disabled by default)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169774 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-10 21:45:01 +00:00
Chandler Carruth
d04a8d4b33 Use the new script to sort the includes of every file under lib.
Sooooo many of these had incorrect or strange main module includes.
I have manually inspected all of these, and fixed the main module
include to be the nearest plausible thing I could find. If you own or
care about any of these source files, I encourage you to take some time
and check that these edits were sensible. I can't have broken anything
(I strictly added headers, and reordered them, never removed), but they
may not be the headers you'd really like to identify as containing the
API being implemented.

Many forward declarations and missing includes were added to a header
files to allow them to parse cleanly when included first. The main
module rule does in fact have its merits. =]

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169131 91177308-0d34-0410-b5e6-96231b3b80d8
2012-12-03 16:50:05 +00:00
Nadav Rotem
3d1a975026 No need to run LICM after loop vectorization because we dont generate invariant code any more.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168928 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-29 19:28:29 +00:00
Dmitri Gribenko
79c07d2a36 Use empty parens for empty function parameter list instead of '(void)'.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@168049 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-15 16:51:49 +00:00
Nadav Rotem
9210cc3a17 80-col
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@167036 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-30 18:37:43 +00:00
Nadav Rotem
2459bf2427 Rename the BB-vectorize flag to match the dragonegg name
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166948 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-29 18:01:14 +00:00
Nadav Rotem
d233b78760 Change the PassManagerBuilder (used by -O3) loop vectorizer flag from -vectorize to -vectorize-loops because we dont want to share the same flag as the bb-vectorizer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166937 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-29 16:36:25 +00:00
Rafael Espindola
e5551ed9ce Change the internalize pass to internalize all symbols when given an empty
list of externals. This makes sense since a shared library with no symbols
can still be useful if it has static constructors.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166795 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-26 18:47:48 +00:00
Nadav Rotem
c58b799da5 revert accidental change
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166643 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24 23:48:57 +00:00
Nadav Rotem
2652c50f74 Implement a basic cost model for vector and scalar instructions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166642 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-24 23:47:38 +00:00
Chandler Carruth
63a1eb62e4 Introduce a BarrierNoop pass, a hack designed to allow *some* control
over the implicitly-formed-and-nesting CGSCC pass manager and function
pass managers, especially when using them on the opt commandline or
using extension points in the module builder. The '-barrier' opt flag
(or the pass itself) will create a no-op module pass in the pipeline,
resetting the pass manager stack, and allowing the creation of a new
pipeline of function passes or CGSCC passes to be created that is
independent from any previous pipelines.

For example, this can be used to test running two CGSCC passes in
independent CGSCC pass managers as opposed to in the same CGSCC pass
manager. It also allows us to introduce a further hack into the
PassManagerBuilder to separate the O0 pipeline extension passes from the
always-inliner's CGSCC pass manager, which they likely do not want to
participate in... At the very least none of the Sanitizer passes want
this behavior.

This fixes a bug with ASan at O0 currently, and I'll commit the ASan
test which covers this pass. I'm happy to add a test case that this pass
exists and works, but not sure how much time folks would like me to
spend adding test cases for the details of its behavior of partition
pass managers.... The whole thing is just vile, and mostly intended to
unblock ASan, so I'm hoping to rip this all out in a brave new pass
manager world.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166172 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-18 08:05:46 +00:00
Nadav Rotem
d15c0c7ac1 Add a loop vectorizer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166112 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-17 18:25:06 +00:00
Chandler Carruth
1fe4fae463 Turn the new SROA pass back on. Let's see if it sticks this time. =]
Again, let me know if anything breaks due to this!

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164986 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-02 04:24:01 +00:00