Commit Graph

117888 Commits

Author SHA1 Message Date
Elena Demikhovsky
9f63519857 AVX-512: Fixed a bug in compress and expand intrinsics.
By Igor Breger (igor.breger@intel.com)



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238724 91177308-0d34-0410-b5e6-96231b3b80d8
2015-06-01 06:30:13 +00:00
Matt Arsenault
5f3a6430d6 Add address space argument to isLegalAddressingMode
This is important because of different addressing modes
depending on the address space for GPU targets.

This only adds the argument, and does not update
any of the uses to provide the correct address space.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238723 91177308-0d34-0410-b5e6-96231b3b80d8
2015-06-01 05:31:59 +00:00
David Blaikie
d84c8ef1be [opaque pointer type] Explicitly store the pointee type of the result of a GEP
Alternatively, this type could be derived on-demand whenever
getResultElementType is called - if someone thinks that's the better
choice (simple time/space tradeoff), I'm happy to give it a go.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238716 91177308-0d34-0410-b5e6-96231b3b80d8
2015-06-01 03:09:34 +00:00
Rafael Espindola
dd2e39c453 Try to fix the build of IntelJITEventListener.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238709 91177308-0d34-0410-b5e6-96231b3b80d8
2015-06-01 02:18:14 +00:00
Rafael Espindola
223e58b67d Rename HasData to IsRegistered.
There is no MCSectionData, so the old name is now meaningless.

Also remove some asserts/checks that were there just because the information
they used was in MCSectionData.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238708 91177308-0d34-0410-b5e6-96231b3b80d8
2015-06-01 01:52:18 +00:00
Rafael Espindola
d1678240e5 Remove trivial forwarding function.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238707 91177308-0d34-0410-b5e6-96231b3b80d8
2015-06-01 01:39:15 +00:00
Rafael Espindola
2dd8a67317 Store a bit in MCSection saying if it was registered with MCAssembler.
With this we can replace a SetVector with a plain std::vector.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238706 91177308-0d34-0410-b5e6-96231b3b80d8
2015-06-01 01:30:01 +00:00
Rafael Espindola
13950e506d Use a bitfield. NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238705 91177308-0d34-0410-b5e6-96231b3b80d8
2015-06-01 01:05:07 +00:00
Rafael Espindola
2e20fdbfcb Use a 32 bit field for the symbol index.
Even 64 ELF uses a 32 bit field to refer to symbols.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238704 91177308-0d34-0410-b5e6-96231b3b80d8
2015-06-01 00:58:31 +00:00
Rafael Espindola
481f35f113 Simplify another function that doesn't fail.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238703 91177308-0d34-0410-b5e6-96231b3b80d8
2015-06-01 00:27:26 +00:00
David Majnemer
75ce6bba4e [PHITransAddr] Don't translate unreachable values
Unreachable values may use themselves in strange ways due to their
dominance property.  Attempting to translate through them can lead to
infinite recursion, crashing LLVM.  Instead, claim that we weren't able
to translate the value.

This fixes PR23096.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238702 91177308-0d34-0410-b5e6-96231b3b80d8
2015-06-01 00:15:08 +00:00
David Majnemer
5a13050628 [PHITransAddr] Use std::find instead of std::count
There is no need to visit all the elements if we are merely performing a
membership check.  NFCI.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238701 91177308-0d34-0410-b5e6-96231b3b80d8
2015-06-01 00:15:04 +00:00
Rafael Espindola
64afb8375d Simplify interface of function that doesn't fail.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238700 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-31 23:52:50 +00:00
Keno Fischer
23125c500f [DWARF] Fix a bug in line info handling
This fixes a bug in the line info handling in the dwarf code, based on a
problem I when implementing RelocVisitor support for MachO.
Since addr+size will give the first address past the end of the function,
we need to back up one line table entry. Fix this by looking up the
end_addr-1, which is the last address in the range. Note that this also
removes a duplicate output from the llvm-rtdyld line table dump. The
relevant line is the end_sequence one in the line table and has an offset
of the first address part the end of the range and hence should not be
included.
Also factor out the common functionality into a separate function.
This comes up on MachO much more than on ELF, since MachO
doesn't store the symbol size separately, hence making
said situation always occur.

Differential Revision: http://reviews.llvm.org/D9925

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238699 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-31 23:37:04 +00:00
Rafael Espindola
f8f9476a1b For COFF and MachO, compute the gap between to symbols.
Before r238028 we used to do this in O(N^2), now we do it in O(N log N).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238698 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-31 23:15:35 +00:00
NAKAMURA Takumi
7b72baf44a ARMConstantIslandPass.cpp: Prune an empty \brief. [-Wdocumentation]
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238697 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-31 23:05:35 +00:00
Colin LeMahieu
d41d04784c [Hexagon] Including raw_ostream for debug builds.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238695 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-31 22:29:33 +00:00
Colin LeMahieu
b3d23d2f42 [Hexagon] classes are actually structs.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238694 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-31 22:18:42 +00:00
Rafael Espindola
67635abeaf Use a range loop. NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238693 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-31 22:13:51 +00:00
Colin LeMahieu
37041b8d58 [Hexagon] Adding MC packet shuffler.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238692 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-31 21:57:09 +00:00
Tim Northover
876dd978b8 ARM: recommit r237590: allow jump tables to be placed as constant islands.
The original version didn't properly account for the base register
being modified before the final jump, so caused miscompilations in
Chromium and LLVM. I've fixed this and tested with an LLVM self-host
(I don't have the means to build & test Chromium).

The general idea remains the same: in pathological cases jump tables
can be too far away from the instructions referencing them (like other
constants) so they need to be movable.

Should fix PR23627.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238680 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-31 19:22:07 +00:00
Benjamin Kramer
d79915c6fe [MC] Simplify code. No functionality change intended.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238676 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-31 18:49:28 +00:00
Davide Italiano
a651b2236f Clarify how the binary file checked in was generated.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238665 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 22:43:36 +00:00
Colin LeMahieu
dd25aab4f0 [Hexagon] Adding override specifier and removing erroneous assertion
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238664 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 20:03:07 +00:00
Keno Fischer
dbdf667725 Add RelocVisitor support for MachO
This commit adds partial support for MachO relocations to RelocVisitor.
A simple test case is added to show that relocations are indeed being
applied and that using llvm-dwarfdump on MachO files no longer errors.
Correctness is not yet tested, due to an unrelated bug in DebugInfo,
which will be fixed with appropriate testcase in a followup commit.

Differential Revision: http://reviews.llvm.org/D8148

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238663 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 19:44:53 +00:00
Colin LeMahieu
27420dd231 [Hexagon] Adding basic relaxation functionality.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238660 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 18:55:47 +00:00
Colin LeMahieu
32de7d791e [MC] Allow backends to decide relaxation for unresolved fixups.
Differential Revision: http://reviews.llvm.org/D8217

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238659 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 18:42:22 +00:00
Kostya Serebryany
63da212749 [lib/Fuzzer] make assertions more informative and update comments for the user-supplied mutator
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238658 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 17:33:13 +00:00
Benjamin Kramer
bfe3365795 [MC] Reorder MCSymbol members to reduce padding.
sizeof(MCSymbol) goes from 72 to 64 bytes on x86_64.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238655 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 13:52:30 +00:00
Simon Pilgrim
08786dc314 Stripped trailing whitespace. NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238654 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 13:01:42 +00:00
Renato Golin
9d945aa625 Comment change. NFC
That comment misleads the current discussions in mentioned bug. Leave
the discussions to the bug. Also, adding a future change FIXME.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238653 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 10:44:07 +00:00
Chandler Carruth
fa68750e54 [x86] Unify the horizontal adding used for popcount lowering taking the
best approach of each.

For vNi16, we use SHL + ADD + SRL pattern that seem easily the best.

For vNi32, we use the PUNPCK + PSADBW + PACKUSWB pattern. In some cases
there is a huge improvement with this in IACA's estimated throughput --
over 2x higher throughput!!!! -- but the measurements are too good to be
true. In one narrow case, the SHL + ADD + SHL + ADD + SRL pattern looks
slightly faster, but I'm not sure I believe any of the measurements at
this point. Both are the exact same uops though. Hard to be confident of
anything past that.

If anyone wants to collect very detailed (Agner-level) timings with the
result of this patch, or with the i32 case replaced with SHL + ADD + SHl
+ ADD + SRL, I'd be very interested. Note that you'll need to test it on
both Ivybridge and Haswell, with both SSE3, SSSE3, and AVX selected as
I saw unique behavior in each of these buckets with IACA all of which
should be checked against measured performance.

But this patch is still a useful improvement by dropping duplicate work
and getting the much nicer PSADBW lowering for v2i64.

I'd still like to rephrase this in terms of generic horizontal sum. It's
a bit lame to have a special case of that just for popcount.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238652 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 10:35:03 +00:00
Renato Golin
215bfbf9ea [ARMTargetParser] Move IAS arch ext parser. NFC
The plan was to move the whole table into the already existing ArchExtNames
but some fields depend on a table-generated file, and we don't yet have this
feature in the generic lib/Support side.

Once the minimum target-specific table-generated files are available in a
generic fashion to these libraries, we'll have to keep it in the ASM parser.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238651 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 10:30:02 +00:00
Chandler Carruth
da8bb20158 [x86] Split out the horizontal byte sum lowering component of the LUT
lowering into a helper function.

NFC.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238650 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 09:46:16 +00:00
Craig Topper
be43b88fae [TableGen] Merge RecTy::typeIsConvertibleTo and RecTy::baseClassOf. NFC
typeIsConvertibleTo was just calling baseClassOf(this) on the argument passed to it, but there weren't different signatures for baseClassOf so passing 'this' didn't really do anything interesting. typeIsConvertibleTo could have just been a non-virtual method in RecTy. But since that would be kind of a silly method, I instead re-distributed the logic from baseClassOf into typeIsConvertibleTo.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238648 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 07:36:01 +00:00
Craig Topper
15617cdf57 Fix indentation. NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238647 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 07:35:21 +00:00
Craig Topper
17246b12cf [TableGen] Remove all the variations of RecTy::convertValue and just handle the conversions in convertInitializerTo directly. This saves a bunch of vtable entries. NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238646 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 07:34:51 +00:00
Chandler Carruth
60dbe0fd0d [x86] Update the order of instructions after I switched to a bitcast
helper that skips creating a cast when it isn't necessary.

It's really somewhat concerning that this was caused by the the presence
of a no-op bitcast, but...

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238642 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 06:02:37 +00:00
David Majnemer
e0d2a2939d [WinCOFF] Add support for the .safeseh directive
.safeseh adds an entry to the .sxdata section to register all the
appropriate functions which may handle an exception.  This entry is not
a relocation to the symbol but instead the symbol table index of the
function.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238641 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 04:56:02 +00:00
Chandler Carruth
3279f2381b [x86] Replace the long spelling of getting a bitcast with the *much*
shorter one. NFC.

In addition to being much shorter to type and requiring fewer arguments,
this change saves over 30 lines from this one file, all wasted on total
boilerplate...

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238640 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 04:23:13 +00:00
Chandler Carruth
b26a073acb [x86] Replace the long spelling of getting a bitcast with the new short
spelling. NFC.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238639 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 04:19:57 +00:00
Chandler Carruth
89a133960b [sdag] Add the helper I most want to the DAG -- building a bitcast
around a value using its existing SDLoc.

Start using this in just one function to save omg lines of code.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238638 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 04:14:10 +00:00
Chandler Carruth
d8018eeac9 [x86] Restore the bitcasts I removed when refactoring this to avoid
shifting vectors of bytes as x86 doesn't have direct support for that.

This removes a bunch of redundant masking in the generated code for SSE2
and SSE3.

In order to avoid the really significant code size growth this would
have triggered, I also factored the completely repeatative logic for
shifting and masking into two lambdas which in turn makes all of this
much easier to read IMO.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238637 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 04:05:11 +00:00
Chandler Carruth
828f5b807c [x86] Implement a faster vector population count based on the PSHUFB
in-register LUT technique.

Summary:
A description of this technique can be found here:
http://wm.ite.pl/articles/sse-popcount.html

The core of the idea is to use an in-register lookup table and the
PSHUFB instruction to compute the population count for the low and high
nibbles of each byte, and then to use horizontal sums to aggregate these
into vector population counts with wider element types.

On x86 there is an instruction that will directly compute the horizontal
sum for the low 8 and high 8 bytes, giving vNi64 popcount very easily.
Various tricks are used to get vNi32 and vNi16 from the vNi8 that the
LUT computes.

The base implemantion of this, and most of the work, was done by Bruno
in a follow up to D6531. See Bruno's detailed post there for lots of
timing information about these changes.

I have extended Bruno's patch in the following ways:

0) I committed the new tests with baseline sequences so this shows
   a diff, and regenerated the tests using the update scripts.

1) Bruno had noticed and mentioned in IRC a redundant mask that
   I removed.

2) I introduced a particular optimization for the i32 vector cases where
   we use PSHL + PSADBW to compute the the low i32 popcounts, and PSHUFD
   + PSADBW to compute doubled high i32 popcounts. This takes advantage
   of the fact that to line up the high i32 popcounts we have to shift
   them anyways, and we can shift them by one fewer bit to effectively
   divide the count by two. While the PSHUFD based horizontal add is no
   faster, it doesn't require registers or load traffic the way a mask
   would, and provides more ILP as it happens on different ports with
   high throughput.

3) I did some code cleanups throughout to simplify the implementation
   logic.

4) I refactored it to continue to use the parallel bitmath lowering when
   SSSE3 is not available to preserve the performance of that version on
   SSE2 targets where it is still much better than scalarizing as we'll
   still do a bitmath implementation of popcount even in scalar code
   there.

With #1 and #2 above, I analyzed the result in IACA for sandybridge,
ivybridge, and haswell. In every case I measured, the throughput is the
same or better using the LUT lowering, even v2i64 and v4i64, and even
compared with using the native popcnt instruction! The latency of the
LUT lowering is often higher than the latency of the scalarized popcnt
instruction sequence, but I think those latency measurements are deeply
misleading. Keeping the operation fully in the vector unit and having
many chances for increased throughput seems much more likely to win.

With this, we can lower every integer vector popcount implementation
using the LUT strategy if we have SSSE3 or better (and thus have
PSHUFB). I've updated the operation lowering to reflect this. This also
fixes an issue where we were scalarizing horribly some AVX lowerings.

Finally, there are some remaining cleanups. There is duplication between
the two techniques in how they perform the horizontal sum once the byte
population count is computed. I'm going to factor and merge those two in
a separate follow-up commit.

Differential Revision: http://reviews.llvm.org/D10084

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238636 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 03:20:59 +00:00
Chandler Carruth
43d1e87d73 [x86] Restructure the parallel bitmath lowering of popcount into
a separate routine, generalize it to work for all the integer vector
sizes, and do general code cleanups.

This dramatically improves lowerings of byte and short element vector
popcount, but more importantly it will make the introduction of the
LUT-approach much cleaner.

The biggest cleanup I've done is to just force the legalizer to do the
bitcasting we need. We run these iteratively now and it makes the code
much simpler IMO. Other changes were minor, and mostly naming and
splitting things up in a way that makes it more clear what is going on.

The other significant change is to use a different final horizontal sum
approach. This is the same number of instructions as the old method, but
shifts left instead of right so that we can clear everything but the
final sum with a single shift right. This seems likely better than
a mask which will usually have to read the mask from memory. It is
certaily fewer u-ops. Also, this will be temporary. This and the LUT
approach share the need of horizontal adds to finish the computation,
and we have more clever approaches than this one that I'll switch over
to.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238635 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 03:20:55 +00:00
Jim Grosbach
586c0042da MC: Clean up MCExpr naming. NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238634 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 01:25:56 +00:00
Filipe Cabecinhas
3b821159da [BitcodeReader] Change an assert to a call to a call to Error()
It's reachable from user input.

Bug found with AFL fuzz.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238633 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-30 00:17:20 +00:00
Fiona Glaser
dbf84c95fe SelectionDAG: fix logic for promoting shift types
r238503 fixed the problem of too-small shift types by promoting them
during legalization, but the correct solution is to promote only the
operands that actually demand promotion.

This fixes a crash on an out-of-tree target caused by trying to
promote an operand that can't be promoted.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238632 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-29 23:37:22 +00:00
Reid Kleckner
bfa311df8c [WinEH] Adjust the 32-bit SEH prologue to better match reality
It turns out that _except_handler3 and _except_handler4 really use the
same stack allocation layout, at least today. They just make different
choices about encoding the LSDA.

This is in preparation for lowering the llvm.eh.exceptioninfo().

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238627 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-29 22:57:46 +00:00
Jingyue Wu
bb06a52fa7 [docs] fix the declarations of the llvm.nvvm.ptr.gen.to.* intrinsics
Summary:
These intrinsics should take a generic input address space and outputs a
non-generic address space.

Test Plan: no

Reviewers: jholewinski, eliben

Reviewed By: eliben

Subscribers: eliben, jholewinski, llvm-commits

Differential Revision: http://reviews.llvm.org/D10132

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238620 91177308-0d34-0410-b5e6-96231b3b80d8
2015-05-29 22:18:03 +00:00