Commit Graph

18617 Commits

Author SHA1 Message Date
Elena Demikhovsky
4a25fbea03 Masked gather and scatter - added DAGCombine visitors
and AVX-512 instruction selection patterns.
All other patches, including tests will follow.

http://reviews.llvm.org/D7665



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236211 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-30 08:38:48 +00:00
Owen Anderson
36a398fe70 Semantically revert r236031, which is not a good idea for in-order targets.
At the least it should be guarded by some kind of target hook.
It also introduced catastrophic compile time and code quality
regressions on some out of tree targets (test case still being
reduced/sanitized).

Sanjay agreed with reverting this patch until these issues can be
resolved.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236199 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-30 04:06:32 +00:00
Hans Wennborg
49baa9f896 Switch lowering: use profile info to build weight-balanced binary search trees
This will cause hot nodes to appear closer to the root.

The literature says building the tree like this makes it a near-optimal (in
terms of search time given key frequencies) binary search tree. In LLVM's case,
we can do up to 3 comparisons in each leaf node, so it might be better to opt
for lower tree height in some cases; that's something to look into in the
future.

Differential Revision: http://reviews.llvm.org/D9318

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236192 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-30 00:57:37 +00:00
Reid Kleckner
85b9ebb7e8 [WinEH] Start EH preparation for 32-bit x86, it uses no arguments
32-bit x86 MSVC-style exceptions are functionaly similar to 64-bit, but
they take no arguments. Instead, they implicitly use the value of EBP
passed in by the caller as a pointer to the parent's frame. In LLVM, we
can represent this as llvm.frameaddress(1), and feed that into all of
our calls to llvm.framerecover.

The next steps are:
- Add an alloca to the fs:00 linked list of handlers
- Add something like llvm.sjlj.lsda or generalize it to store in the
  alloca
- Move state number calculation to WinEHPrepare, arrange for
  FunctionLoweringInfo to call it
- Use the state numbers to insert explicit loads and stores in the IR

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236172 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-29 22:49:54 +00:00
Sanjay Patel
3c2da524a6 generalize binop reassociation; NFC
Move the fold introduced in r236031:
http://reviews.llvm.org/rL236031

to its own helper function, so we can use it for other binops.

This is a preliminary step before partially solving:
https://llvm.org/bugs/show_bug.cgi?id=21768
https://llvm.org/bugs/show_bug.cgi?id=23116



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236171 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-29 22:30:02 +00:00
Pat Gavlin
ec4a389039 Run StatepointLowering.{cpp,h} through clang-format.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236166 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-29 21:52:45 +00:00
David Blaikie
39e7388a19 [opaque pointer type] Pass GlobalAlias the actual pointer type rather than decomposing it into pointee type + address space
Many of the callers already have the pointer type anyway, and for the
couple of callers that don't it's pretty easy to call PointerType::get
on the pointee type and address space.

This avoids LLParser from using PointerType::getElementType when parsing
GlobalAliases from IR.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236160 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-29 21:22:39 +00:00
Sanjay Patel
90d98d86c2 tidy up; NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236156 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-29 21:01:41 +00:00
Sanjay Patel
35684a13d4 too much space again; NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236150 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-29 20:38:02 +00:00
Sanjay Patel
d08b65a38f too much space; NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236147 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-29 20:32:57 +00:00
Andrew Kaylor
a147b2575b [WinEH] Fix minor bug in begincatch block splitting
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236129 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-29 17:21:26 +00:00
Duncan P. N. Exon Smith
e56023a059 IR: Give 'DI' prefix to debug info metadata
Finish off PR23080 by renaming the debug info IR constructs from `MD*`
to `DI*`.  The last of the `DIDescriptor` classes were deleted in
r235356, and the last of the related typedefs removed in r235413, so
this has all baked for about a week.

Note: If you have out-of-tree code (like a frontend), I recommend that
you get everything compiling and tests passing with the *previous*
commit before updating to this one.  It'll be easier to keep track of
what code is using the `DIDescriptor` hierarchy and what you've already
updated, and I think you're extremely unlikely to insert bugs.  YMMV of
course.

Back to *this* commit: I did this using the rename-md-di-nodes.sh
upgrade script I've attached to PR23080 (both code and testcases) and
filtered through clang-format-diff.py.  I edited the tests for
test/Assembler/invalid-generic-debug-node-*.ll by hand since the columns
were off-by-three.  It should work on your out-of-tree testcases (and
code, if you've followed the advice in the previous paragraph).

Some of the tests are in badly named files now (e.g.,
test/Assembler/invalid-mdcompositetype-missing-tag.ll should be
'dicompositetype'); I'll come back and move the files in a follow-up
commit.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236120 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-29 16:38:44 +00:00
Jan Vesely
42eeb1d91f CodeGen: Default overflow operations to expand so we don't have to assume targets are lying
Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
Reviewed-by: ab
Differential Revision: http://reviews.llvm.org/D9265

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236119 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-29 16:30:46 +00:00
Elena Demikhovsky
5470fb0338 Fixed masked gather/scatter switch-case
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236092 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-29 08:38:53 +00:00
Elena Demikhovsky
a2b90dc078 fixed comments, blanks, nullptr; NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236086 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-29 06:49:50 +00:00
Matthias Braun
2d6f83dac9 RegisterCoalescer: hide terminal rule option by default
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236062 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-28 23:55:11 +00:00
Andrew Kaylor
63ca5d6589 Style updates
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236048 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-28 22:01:51 +00:00
Andrew Kaylor
0595a97817 [WinEH] Split blocks at calls to llvm.eh.begincatch
Differential Revision: http://reviews.llvm.org/D9311



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236046 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-28 21:54:14 +00:00
Sanjay Patel
959b276771 transform fadd chains to increase parallelism
This is a compromise: with this simple patch, we should always handle a chain of exactly 3
operations optimally, but we're not generating the optimal balanced binary tree for a longer
sequence.

In general, this transform will reduce the dependency chain for a sequence of instructions
using N operands from a worst case N-1 dependent operations to N/2 dependent operations. 
The optimal balanced binary tree would reduce the chain to log2(N).

The trade-off for not dealing with longer sequences is: (1) we have less complexity in the
compiler, (2) we avoid unknown compile-time blowup calculating a balanced tree, and (3) we
don't need to worry about the increased register pressure required to parallelize longer
sequences. It also seems unlikely that we would ever encounter really long strings of
dependent ops like that in the wild, but I'm not sure how to verify that speculation.
FWIW, I see no perf difference for test-suite running on btver2 (x86-64) with -ffast-math
and this patch.

We can extend this patch to cover other associative operations such as fmul, fmax, fmin, 
integer add, integer mul.

This is a partial fix for:
https://llvm.org/bugs/show_bug.cgi?id=17305

and if extended:
https://llvm.org/bugs/show_bug.cgi?id=21768
https://llvm.org/bugs/show_bug.cgi?id=23116

The issue also came up in:
http://reviews.llvm.org/D8941

Differential Revision: http://reviews.llvm.org/D9232



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236031 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-28 21:03:22 +00:00
Sanjay Patel
2a7841dd4d move IR-level optimization flags into their own struct
This is a preliminary step to using the IR-level floating-point fast-math-flags in the SDAG (D8900).

In this patch, we introduce the optimization flags as their own struct. As noted in the TODO comment, 
we should eventually share this data between the IR passes and the backend.

We also switch the existing nsw / nuw / exact bit functionality of the BinaryWithFlagsSDNode class to
use the new struct.

The tradeoff is that instead of using the free but limited space of SDNode's SubclassData, we add a
data member to the subclass. This means we don't have to repeat all of the get/set methods per flag,
but we're potentially adding size to all nodes of this subclassi type.

In practice on 64-bit systems (measured on Linux and MacOS X), there is no size difference between an
SDNode and BinaryWithFlagsSDNode after this change: they're both 80 bytes. This means that we had at
least one free byte to play with due to struct alignment.

Differential Revision: http://reviews.llvm.org/D9325



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235997 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-28 16:39:12 +00:00
Sergey Dmitrouk
1f7a90d793 Reapply r235977 "[DebugInfo] Add debug locations to constant SD nodes"
[DebugInfo] Add debug locations to constant SD nodes

This adds debug location to constant nodes of Selection DAG and updates
all places that create constants to pass debug locations
(see PR13269).

Can't guarantee that all locations are correct, but in a lot of cases choice
is obvious, so most of them should be. At least all tests pass.

Tests for these changes do not cover everything, instead just check it for
SDNodes, ARM and AArch64 where it's easy to get incorrect locations on
constants.

This is not complete fix as FastISel contains workaround for wrong debug
locations, which drops locations from instructions on processing constants,
but there isn't currently a way to use debug locations from constants there
as llvm::Constant doesn't cache it (yet). Although this is a bit different
issue, not directly related to these changes.

Differential Revision: http://reviews.llvm.org/D9084

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235989 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-28 14:05:47 +00:00
Daniel Jasper
515cc265c9 Revert "[DebugInfo] Add debug locations to constant SD nodes"
This breaks a test:
http://bb.pgr.jp/builders/cmake-llvm-x86_64-linux/builds/23870

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235987 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-28 13:38:35 +00:00
Sergey Dmitrouk
716c5d8a30 [DebugInfo] Add debug locations to constant SD nodes
This adds debug location to constant nodes of Selection DAG and updates
all places that create constants to pass debug locations
(see PR13269).

Can't guarantee that all locations are correct, but in a lot of cases choice
is obvious, so most of them should be. At least all tests pass.

Tests for these changes do not cover everything, instead just check it for
SDNodes, ARM and AArch64 where it's easy to get incorrect locations on
constants.

This is not complete fix as FastISel contains workaround for wrong debug
locations, which drops locations from instructions on processing constants,
but there isn't currently a way to use debug locations from constants there
as llvm::Constant doesn't cache it (yet). Although this is a bit different
issue, not directly related to these changes.

Differential Revision: http://reviews.llvm.org/D9084

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235977 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-28 11:56:37 +00:00
Elena Demikhovsky
8dfda019a0 Masked gather and scatter: Added code for SelectionDAG.
All other patches, including tests will follow.

http://reviews.llvm.org/D7665



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235970 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-28 07:57:37 +00:00
Hans Wennborg
5d004276ec Switch lowering: use uint32_t for weights everywhere
I previously thought switch clusters would need to use uint64_t in case
the weights of multiple cases overflowed a 32-bit int. It turns
out that the weights on a terminator instruction are capped to allow for
being added together, so using a uint32_t should be safe.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235945 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-27 23:52:19 +00:00
Hans Wennborg
b176a4f2e4 Switch lowering: Take branch weight into account when ordering for fall-through
Previously, the code would try to put a fall-through case last,
even if that meant moving a case with much higher branch weight
further down the chain.

Ordering by branch weight is most important, putting a fall-through
block last is secondary.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235942 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-27 23:35:22 +00:00
Hans Wennborg
84145dcd08 Switch lowering: order bit tests by branch weight.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235912 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-27 20:21:17 +00:00
Philip Reames
c4ab95d900 Make the message associated with a fatal error slightly more helpful
Looking into 23095, my best guess is that the CodeGen library itself isn't getting linked and initialized properly.  To make this slightly more obvious to consumers of LLVM, emit a different error message if we can tell that the registry is empty vs you've simply happened to name a collector which hasn't been registered.  



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235824 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-26 22:00:34 +00:00
Andrew Kaylor
87dc11bb4c Fix build error from accidental change
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235792 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-24 23:34:46 +00:00
Andrew Kaylor
e0905685aa [WinEH] Find correct cloned entry block for outlined handler functions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235791 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-24 23:27:32 +00:00
Andrew Kaylor
7d3dd38582 [WinEH] Find correct cloned entry block for outlined handler functions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235789 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-24 23:10:38 +00:00
Quentin Colombet
1615d1720e [DAGCombiner] Fix the type used in canFoldInAddressingMode to account for the
right scaling.

In the function canFoldInAddressingMode, VT is computed as the type of the
destination/source of a LOAD/STORE operations, instead of the memory type of the
operation.
On targets with a scaling factor on the offset of the LOAD/STORE operations, the
function may return false for actually valid cases. This may then prevent the
selection of profitable pre or post indexed load/store operations, and instead
select pre or post indexed load/store for unprofitable cases.

Patch by Francois de Ferriere <francois.de-ferriere@st.com>!

Differential Revision: http://reviews.llvm.org/D9146


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235780 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-24 21:28:00 +00:00
Kaelyn Takata
5f10af2b53 Remove an unused variable to prevent -Werror build failures.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235773 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-24 21:02:18 +00:00
Reid Kleckner
39789f81ab [SEH] Implement GetExceptionCode in __except blocks
This introduces an intrinsic called llvm.eh.exceptioncode. It is lowered
by copying the EAX value live into whatever basic block it is called
from. Obviously, this only works if you insert it late during codegen,
because otherwise mid-level passes might reschedule it.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235768 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-24 20:25:05 +00:00
Lang Hames
579cebfb15 [AsmPrinter] Make AsmPrinter's OutStreamer member a unique_ptr.
AsmPrinter owns the OutStreamer, so an owning pointer makes sense here. Using a
reference for this is crufty.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235752 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-24 19:11:51 +00:00
Hans Wennborg
794cd3418b Switch lowering: fix APInt overflow causing infinite loop / OOM
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235729 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-24 16:53:55 +00:00
Reid Kleckner
8410b22f91 [WinEH] Split the landingpad BB instead of cloning it
This means we don't have to RAUW the landingpad instruction and
landingpad BB, which is a nice win.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235725 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-24 16:22:19 +00:00
Matthias Braun
37f93d4cbf RegisterCoalescer: implicit phsreg uses are fine when rematerializing
The target hooks should have already checked them. This change is
necessary to enable the remateriailzation on R600.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235673 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-24 00:01:37 +00:00
Matthias Braun
589d8fd65d RegisterCoalescer: Avoid unnecessary register class widening for some rematerializations
I couldn't provide a testcase as none of the public targets has wide
register classes with alot of subregisters and at the same time an
instruction which "ReMaterializable" and "AsCheapAsAMove" (could
probably be added for R600).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235668 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-23 23:24:36 +00:00
Reid Kleckner
f87165820d Re-commit "[SEH] Remove the old __C_specific_handler code now that WinEHPrepare works"
This reverts commit r235617.

r235649 should have addressed the problems.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235667 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-23 23:22:33 +00:00
Andrew Kaylor
3ad4bc1445 [WinEH] Ignore filter clauses while mapping landing pad blocks.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235656 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-23 22:38:36 +00:00
Reid Kleckner
173a68d4a2 Remove trivial assert to fix NDEBUG Werror builds
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235652 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-23 21:36:32 +00:00
Reid Kleckner
d1807ff318 [WinEH] Replace more lpad value uses with undef
We were asserting on code like this:
  extern "C" unsigned long _exception_code();
  void might_crash(unsigned long);
  void foo() {
    __try {
      might_crash(0);
    } __except(1) {
      might_crash(_exception_code());
    }
  }

Gtest and many other libraries get the exception code from the __except
block. What's supposed to happen here is that EAX is live into the
__except block, and it contains the exception code. Eventually we'll
represent that as a use of the landingpad ehptr value, but for now we
can replace it with undef.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235649 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-23 21:22:30 +00:00
Quentin Colombet
c364314ec3 [MachineCopyPropagation] Handle undef flags conservatively so that we do not
remove copies that are useful after breaking some hardware dependencies.
In other words, handle this kind of situations conservatively by assuming reg2
is redefined by the undef flag.
reg1 = copy reg2
= inst reg2<undef>
reg2 = copy reg1
Copy propagation used to remove the last copy.
This is incorrect because the undef flag on reg2 in inst, allows next
passes to put whatever trashed value in reg2 that may help.
In practice we end up with this code:
reg1 = copy reg2
reg2 = 0
= inst reg2<undef>
reg2 = copy reg1

This fixes PR21743.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235647 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-23 21:17:39 +00:00
Andrew Kaylor
71ed98da5c [WinEH] Handle stubs for outlined functions that have only unreached terminators.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235618 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-23 18:37:39 +00:00
Reid Kleckner
70e56ae6b3 Revert "[SEH] Remove the old __C_specific_handler code now that WinEHPrepare works"
We still have some "uses remain after removal" issues in -O0 builds.

This reverts commit r235557.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235617 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-23 18:34:01 +00:00
Hans Wennborg
defaf830f9 Re-commit r235560: Switch lowering: extract jump tables and bit tests before building binary tree (PR22262)
Third time's the charm. The previous commit was reverted as a
reverse for-loop in SelectionDAGBuilder::lowerWorkItem did 'I--'
on an iterator at the beginning of a vector, causing asserts
when using debugging iterators. This commit fixes that.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235608 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-23 16:45:24 +00:00
Aaron Ballman
5d538f71c2 Revert r235560; this commit was causing several failed assertions in Debug builds using MSVC's STL. The iterator is being used outside of its valid range.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235597 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-23 13:41:59 +00:00
Simon Pilgrim
77aa4a8c4d [DAGCombiner] Remove extra bitcasts surrounding vector shuffles
Patch to remove extra bitcasts from shuffles, this is often a legacy of XformToShuffleWithZero being used to combine bitmaskings (of float vectors bitcast to integer vectors) into shuffles: bitcast(shuffle(bitcast(s0),bitcast(s1))) -> shuffle(s0,s1)

Differential Revision: http://reviews.llvm.org/D9097

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235578 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-23 08:43:13 +00:00
Andrew Kaylor
a1df0a3120 [WinEH] Don't skip landing pads that end with an unreachable instruction.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@235563 91177308-0d34-0410-b5e6-96231b3b80d8
2015-04-23 00:20:44 +00:00