hash_value overload for MachineOperands. This addresses a FIXME
sufficient for me to remove it, and cleans up the code nicely too.
The important changes to the hashing logic:
- TargetFlags are now included in all of the hashes. These were complete
missed.
- Register operands have their subregisters and whether they are a def
included in the hash.
- We now actually hash all of the operand types. Previously, many
operand types were simply *dropped on the floor*. For example:
- Floating point immediates
- Large integer immediates (>64-bit)
- External globals!
- Register masks
- Metadata operands
- It removes the offset from the block-address hash; I'm a bit
suspicious of this, but isIdenticalTo doesn't consider the offset for
black addresses.
Any patterns involving these entities could have triggered extreme
slowdowns in MachineCSE or PHIElimination. Let me know if there are PRs
you think might be closed now... I'm looking myself, but I may miss
them.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159743 91177308-0d34-0410-b5e6-96231b3b80d8
IntegersSubsetMapping
- Replaced type of Items field from std::list with std::map. In neares future I'll test it with DenseMap and do the correspond replacement
if possible.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159703 91177308-0d34-0410-b5e6-96231b3b80d8
This pass performs if-conversion on SSA form machine code by
speculatively executing both sides of the branch and using a cmov
instruction to select the result. This can help lower the number of
branch mispredictions on architectures like x86 that don't have
predicable instructions.
The current implementation is very aggressive, and causes regressions on
mosts tests. It needs good heuristics that have yet to be implemented.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159694 91177308-0d34-0410-b5e6-96231b3b80d8
IntegersSubsetMapping
- Replaced type of Items field from std::list with std::map. In neares future I'll test it with DenseMap and do the correspond replacement
if possible.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159659 91177308-0d34-0410-b5e6-96231b3b80d8
some, and allows the routine to be inlined into common callers. The
various bits that hit this code in their hotpath seem slightly lower on
the profile, but I can't really measure a performance improvement as
everything seems to still be bottlenecked on likely cache misses. =/
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159648 91177308-0d34-0410-b5e6-96231b3b80d8
This is still a work in progress but I believe it is currently good enough
to fix PR13122 "Need unit test driver for codegen IR passes". For example,
you can run llc with -stop-after=loop-reduce to have it dump out the IR after
running LSR. Serializing machine-level IR is not yet supported but we have
some patches in progress for that.
The plan is to serialize the IR to a YAML file, containing separate sections
for the LLVM IR, machine-level IR, and whatever other info is needed. Chad
suggested that we stash the stop-after pass in the YAML file and use that
instead of the start-after option to figure out where to restart the
compilation. I think that's a great idea, but since it's not implemented yet
I put the -start-after option into this patch for testing purposes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159570 91177308-0d34-0410-b5e6-96231b3b80d8
This makes it possible to just use a zero value to represent "no pass", so
the phony NoPassID global variable is no longer needed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159568 91177308-0d34-0410-b5e6-96231b3b80d8
This is a preliminary step toward having TargetPassConfig be able to
start and stop the compilation at specified passes for unit testing
and debugging. No functionality change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159567 91177308-0d34-0410-b5e6-96231b3b80d8
1) DIContext is now able to return function name for a given instruction address (besides file/line info).
2) llvm-dwarfdump accepts flag --functions that prints the function name (if address is specified by --address flag).
3) test case that checks the basic functionality of llvm-dwarfdump added
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159512 91177308-0d34-0410-b5e6-96231b3b80d8
This was always part of the VMCore library out of necessity -- it deals
entirely in the IR. The .cpp file in fact was already part of the VMCore
library. This is just a mechanical move.
I've tried to go through and re-apply the coding standard's preferred
header sort, but at 40-ish files, I may have gotten some wrong. Please
let me know if so.
I'll be committing the corresponding updates to Clang and Polly, and
Duncan has DragonEgg.
Thanks to Bill and Eric for giving the green light for this bit of cleanup.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159421 91177308-0d34-0410-b5e6-96231b3b80d8
The TargetInstrInfo::getNumMicroOps API does not change, but soon it
will be used by MachineScheduler. Now each subtarget can specify the
number of micro-ops per itinerary class. For ARM, this is currently
always dynamic (-1), because it is used for load/store multiple which
depends on the number of register operands.
Zero is now a valid number of micro-ops. This can be used for
nop pseudo-instructions or instructions that the hardware can squash
during dispatch.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159406 91177308-0d34-0410-b5e6-96231b3b80d8
Corrected type for index of llvm.x86.avx2.gather.d.pd.256
from 256-bit to 128-bit.
Corrected types for src|dst|mask of llvm.x86.avx2.gather.q.ps.256
from 256-bit to 128-bit.
Support the following intrinsics:
llvm.x86.avx2.gather.d.q, llvm.x86.avx2.gather.q.q
llvm.x86.avx2.gather.d.q.256, llvm.x86.avx2.gather.q.q.256
llvm.x86.avx2.gather.d.d, llvm.x86.avx2.gather.q.d
llvm.x86.avx2.gather.d.d.256, llvm.x86.avx2.gather.q.d.256
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159402 91177308-0d34-0410-b5e6-96231b3b80d8
- recognize C++ new(std::nothrow) friends
- ignore ExtractElement and ExtractValue instructions in size/offset analysis (all easy cases are probably folded away before we get here)
- also recognize realloc as noalias
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159356 91177308-0d34-0410-b5e6-96231b3b80d8
The original algorithm only used recursive pair fusion of equal-length
types. This is now extended to allow pairing of any types that share
the same underlying scalar type. Because we would still generally
prefer the 2^n-length types, those are formed first. Then a second
set of iterations form the non-2^n-length types.
Also, a call to SimplifyInstructionsInBlock has been added after each
pairing iteration. This takes care of DCE (and a few other things)
that make the following iterations execute somewhat faster. For the
same reason, some of the simple shuffle-combination cases are now
handled internally.
There is some additional refactoring work to be done, but I've had
many requests for this feature, so additional refactoring will come
soon in future commits (as will additional test cases).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159330 91177308-0d34-0410-b5e6-96231b3b80d8
Maintaining this kind of checking in different places is dangerous, extending
Instruction::isSameOperationAs consolidates this logic into one place. Here
I've added an optional flags parameter and two flags that are important for
vectorization: CompareIgnoringAlignment and CompareUsingScalarTypes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159329 91177308-0d34-0410-b5e6-96231b3b80d8
include/llvm/Analysis/DebugInfo.h to include/llvm/DebugInfo.h.
The reasoning is because the DebugInfo module is simply an interface to the
debug info MDNodes and has nothing to do with analysis.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159312 91177308-0d34-0410-b5e6-96231b3b80d8
which many Mips 64 ABIs use than for O64 which many
if not all other target ABIs use.
Most architectures have the following 64 bit relocation record format:
typedef struct
{
Elf64_Addr r_offset; /* Address of reference */
Elf64_Xword r_info; /* Symbol index and type of relocation */
} Elf64_Rel;
typedef struct
{
Elf64_Addr r_offset;
Elf64_Xword r_info;
Elf64_Sxword r_addend;
} Elf64_Rela;
Whereas N64 has the following format:
typedef struct
{
Elf64_Addr r_offset;/* Address of reference */
Elf64_Word r_sym; /* Symbol index */
Elf64_Byte r_ssym; /* Special symbol */
Elf64_Byte r_type3; /* Relocation type */
Elf64_Byte r_type2; /* Relocation type */
Elf64_Byte r_type; /* Relocation type */
} Elf64_Rel;
typedef struct
{
Elf64_Addr r_offset;/* Address of reference */
Elf64_Word r_sym; /* Symbol index */
Elf64_Byte r_ssym; /* Special symbol */
Elf64_Byte r_type3; /* Relocation type */
Elf64_Byte r_type2; /* Relocation type */
Elf64_Byte r_type; /* Relocation type */
Elf64_Sxword r_addend;
} Elf64_Rela;
The structure is the same size, but the r_info data element
is now 5 separate elements. Besides the content aspects,
endian byte reordering will be different for the area with
each element being endianized separately.
I treat this as generic and continue to pass r_type as
an integer masking and unmasking the byte sized N64
values for N64 mode. I've implemented this and it causes no
affect on other current targets.
This passes make check.
Jack
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159299 91177308-0d34-0410-b5e6-96231b3b80d8
Original commit message:
If a constant or a function has linkonce_odr linkage and unnamed_addr, mark it
hidden. Being linkonce_odr guarantees that it is available in every dso that
needs it. Being a constant/function with unnamed_addr guarantees that the
copies don't have to be merged.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159272 91177308-0d34-0410-b5e6-96231b3b80d8
It's not necessary for each DI class to have its own copy of `print' and
`dump'. Instead, just give DIDescriptor those methods and have it call the
appropriate debugging printing routine based on the type of the debug
information.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159237 91177308-0d34-0410-b5e6-96231b3b80d8
Such passes can be used to tweak the register assignments in a
target-dependent way, for example to avoid write-after-write
dependencies.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159209 91177308-0d34-0410-b5e6-96231b3b80d8
- LHS exclude RHS
- LHS intersect RHS (LHS successors will keeped)
- RHS exclude LHS
The complexity is N+M, where
N is size of LHS
M is size of RHS.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159201 91177308-0d34-0410-b5e6-96231b3b80d8
The primary advantage is that loop optimizations will be applied in a
stable order. This helps debugging and unit test creation. It is also
a better overall implementation without pathologically bad performance
on deep functions.
On large functions (llvm-stress --size=200000 | opt -loops)
Before: 0.1263s
After: 0.0225s
On deep functions (after tweaking llvm-stress, thanks Nadav):
Before: 0.2281s
After: 0.0227s
See r158790 for more comments.
The loop tree is now consistently generated in forward order, but loop
passes are applied in reverse order over the program. If we have a
loop optimization that prefers forward order, that can easily be
achieved by adding a different type of LoopPassManager.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159183 91177308-0d34-0410-b5e6-96231b3b80d8
hidden. Being linkonce_odr guarantees that it is available in every dso that
needs it. Being a constant/function with unnamed_addr guarantees that the
copies don't have to be merged.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159136 91177308-0d34-0410-b5e6-96231b3b80d8
advertising complete support w/o alignas implemented, and its
implementation of alignas in the latest versions is so convoluted as to
be unusable.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159125 91177308-0d34-0410-b5e6-96231b3b80d8
This allows the user/front-end to specify a model that is better
than what LLVM would choose by default. For example, a variable
might be declared as
@x = thread_local(initialexec) global i32 42
if it will not be used in a shared library that is dlopen'ed.
If the specified model isn't supported by the target, or if LLVM can
make a better choice, a different model may be used.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159077 91177308-0d34-0410-b5e6-96231b3b80d8
"Invalid operand" may be a completely correct diagnostic, but it's often
insufficiently specific to really help identify and fix the problem in
assembly source. Allow a target to specify a more-specific diagnostic kind
for each AsmOperandClass derived definition and use that to provide
more detailed diagnostics when an operant of that class resulted in a
match failure.
rdar://8987109
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159050 91177308-0d34-0410-b5e6-96231b3b80d8
Original commit message:
Allow up to 64 functional units per processor itinerary.
This patch changes the type used to hold the FU bitset from unsigned to uint64_t.
This will be needed for some upcoming PowerPC itineraries.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159027 91177308-0d34-0410-b5e6-96231b3b80d8
With regunit liveness permanently enabled, this function would always
return true.
Also remove now obsolete code for checking physreg interference.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@159006 91177308-0d34-0410-b5e6-96231b3b80d8
Original message:
Performance optimizations:
- SwitchInst: case values stored separately from Operands List. It allows to make faster access to individual case value numbers or ranges.
- Optimized IntItem, added APInt value caching.
- Optimized IntegersSubsetGeneric: added optimizations for cases when subset is single number or when subset consists from single numbers only.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158997 91177308-0d34-0410-b5e6-96231b3b80d8
fail. Original commit message:
Performance optimizations:
- SwitchInst: case values stored separately from Operands List. It allows to make faster access to individual case value numbers or ranges.
- Optimized IntItem, added APInt value caching.
- Optimized IntegersSubsetGeneric: added optimizations for cases when subset is single number or when subset consists from single numbers only.
On my machine these optimizations gave about 4-6% of compile-time improvement.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158986 91177308-0d34-0410-b5e6-96231b3b80d8
- SwitchInst: case values stored separately from Operands List. It allows to make faster access to individual case value numbers or ranges.
- Optimized IntItem, added APInt value caching.
- Optimized IntegersSubsetGeneric: added optimizations for cases when subset is single number or when subset consists from single numbers only.
On my machine these optimizations gave about 4-6% of compile-time improvement.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158979 91177308-0d34-0410-b5e6-96231b3b80d8
This makes it explicit when ScoreboardHazardRecognizer will be used.
"GenericItineraries" would only make sense if it contained real
itinerary values and still required ScoreboardHazardRecognizer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158963 91177308-0d34-0410-b5e6-96231b3b80d8
boolean flag to an enum: { Fast, Standard, Strict } (default = Standard).
This option controls the creation by optimizations of fused FP ops that store
intermediate results in higher precision than IEEE allows (E.g. FMAs). The
behavior of this option is intended to match the behaviour specified by a
soon-to-be-introduced frontend flag: '-ffuse-fp-ops'.
Fast mode - allows formation of fused FP ops whenever they're profitable.
Standard mode - allow fusion only for 'blessed' FP ops. At present the only
blessed op is the fmuladd intrinsic. In the future more blessed ops may be
added.
Strict mode - allow fusion only if/when it can be proven that the excess
precision won't effect the result.
Note: This option only controls formation of fused ops by the optimizers. Fused
operations that are explicitly requested (e.g. FMA via the llvm.fma.* intrinsic)
will always be honored, regardless of the value of this option.
Internally TargetOptions::AllowExcessFPPrecision has been replaced by
TargetOptions::AllowFPOpFusion.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158956 91177308-0d34-0410-b5e6-96231b3b80d8
- provide more extensive set of functions to detect library allocation functions (e.g., malloc, calloc, strdup, etc)
- provide an API to compute the size and offset of an object pointed by
Move a few clients (GVN, AA, instcombine, ...) to the new API.
This implementation is a lot more aggressive than each of the custom implementations being replaced.
Patch reviewed by Nick Lewycky and Chandler Carruth, thanks.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158919 91177308-0d34-0410-b5e6-96231b3b80d8
Live intervals for regunits and virtual registers are stored separately,
and physreg live intervals are going away.
To visit the live ranges of all virtual registers, use this pattern
instead:
for (unsigned i = 0, e = MRI->getNumVirtRegs(); i != e; ++i) {
unsigned Reg = TargetRegisterInfo::index2VirtReg(i);
if (MRI->reg_nodbg_empty(Reg))
continue;
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158879 91177308-0d34-0410-b5e6-96231b3b80d8
I don't think anyone has been using this functionality for a while, and
it is getting in the way of refactoring now.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158876 91177308-0d34-0410-b5e6-96231b3b80d8
This is supported by gcc and clang, but guarded by a macro for MSVC 2008.
The extern template declaration is not necessary but generally good
form. It can avoid extra instantiations of the template methods
defined inline.
The EXTERN_TEMPLATE_INSTANTIATION macro could probably be generalized to
handle multiple template parameters if someone thinks it's worthwhile.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158840 91177308-0d34-0410-b5e6-96231b3b80d8
Regunit live ranges are computed on demand, so when mi-sched calls
handleMove, some regunits may not have live ranges yet.
That makes updating them easier: Just skip the non-existing ranges. They
will be computed correctly from the rescheduled machine code when they
are needed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158831 91177308-0d34-0410-b5e6-96231b3b80d8
There is a pretty staggering amount of this in LLVM's header files, this
is not all of the instances I'm afraid. These include all of the
functions that (in my build) are used by a non-static inline (or
external) function. Specifically, these issues were caught by the new
'-Winternal-linkage-in-inline' warning.
I'll try to just clean up the remainder of the clearly redundant "static
inline" cases on functions (not methods!) defined within headers if
I can do so in a reliable way.
There were even several cases of a missing 'inline' altogether, or my
personal favorite "static bool inline". Go figure. ;]
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158800 91177308-0d34-0410-b5e6-96231b3b80d8
-stable-loops enables a new algorithm for generating the Loop
forest. It differs from the original algorithm in a few respects:
- Not determined by use-list order.
- Initially guarantees RPO order of block and subloops.
- Linear in the number of CFG edges.
- Nonrecursive.
I didn't want to change the LoopInfo API yet, so the block lists are
still inclusive. This seems strange to me, and it means that building
LoopInfo is not strictly linear, but it may not be a problem in
practice. At least the block lists start out in RPO order now. In the
future we may add an attribute or wrapper analysis that allows other
passes to assume RPO order.
The primary motivation of this work was not to optimize LoopInfo, but
to allow reproducing performance issues by decomposing the compilation
stages. I'm often unable to do this with the current LoopInfo, because
the loop tree order determines Loop pass order. Serializing the IR
tends to invert the order, which reverses the optimization order. This
makes it nearly impossible to debug interdependent loop optimizations
such as LSR.
I also believe this will provide more stable performance results across time.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158790 91177308-0d34-0410-b5e6-96231b3b80d8
The implementation only needs inclusion from LoopInfo.cpp and
MachineLoopInfo.cpp. Clients of the interface should only include the
interface. This makes the interface readable and speeds up rebuilds
after modifying the implementation.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158787 91177308-0d34-0410-b5e6-96231b3b80d8
ensureAlignment() in MachineFunction). Also, drop setMaxAlignment() in
favor of this new function. This creates a main entry point to setting
MaxAlignment, which will be helpful for future work. No functionality
change intended.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158758 91177308-0d34-0410-b5e6-96231b3b80d8
This patch adds DAG combines to form FMAs from pairs of FADD + FMUL or
FSUB + FMUL. The combines are performed when:
(a) Either
AllowExcessFPPrecision option (-enable-excess-fp-precision for llc)
OR
UnsafeFPMath option (-enable-unsafe-fp-math)
are set, and
(b) TargetLoweringInfo::isFMAFasterThanMulAndAdd(VT) is true for the type of
the FADD/FSUB, and
(c) The FMUL only has one user (the FADD/FSUB).
If your target has fast FMA instructions you can make use of these combines by
overriding TargetLoweringInfo::isFMAFasterThanMulAndAdd(VT) to return true for
types supported by your FMA instruction, and adding patterns to match ISD::FMA
to your FMA instructions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158757 91177308-0d34-0410-b5e6-96231b3b80d8
TargetLoweringObjectFileELF. Use this to support it on X86. Unlike ARM,
on X86 it is not easy to find out if .init_array should be used or not, so
the decision is made via TargetOptions and defaults to off.
Add a command line option to llc that enables it.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158692 91177308-0d34-0410-b5e6-96231b3b80d8
Original commit msg:
add the 'alloc' metadata node to represent the size of offset of buffers pointed to by pointers.
This metadata can be attached to any instruction returning a pointer
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158688 91177308-0d34-0410-b5e6-96231b3b80d8
Based on review discussion of r158638 with Chandler Carruth, Tobias von Koch, and Duncan Sands and a -Wmaybe-uninitialized warning from GCC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158685 91177308-0d34-0410-b5e6-96231b3b80d8
This patch changes the type used to hold the FU bitset from unsigned to uint64_t.
This will be needed for some upcoming PowerPC itineraries.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158679 91177308-0d34-0410-b5e6-96231b3b80d8
It always returns the iterator for the first inserted element, or the passed in
iterator if the inserted range was empty. Flesh out the unit test more and fix
all the cases it uncovered so far.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158645 91177308-0d34-0410-b5e6-96231b3b80d8
We have SmallDenseMap now that has more correct and predictable
semantics, even though it is a more narrow abstraction.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158644 91177308-0d34-0410-b5e6-96231b3b80d8
SmallDenseMap::swap.
First, make it parse cleanly. Yay for uninstantiated methods.
Second, make the inline-buckets case work correctly. This is way
trickier than it should be due to the uninitialized values in empty and
tombstone buckets.
Finally fix a few typos that caused construction/destruction mismatches
in the counting unittest.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158641 91177308-0d34-0410-b5e6-96231b3b80d8
destruction and fix a bug in SmallDenseMap they caught.
This is kind of a poor-man's version of the testing that just adds the
addresses to a set on construction and removes them on destruction. We
check that double construction and double destruction don't occur.
Amusingly enough, this is enough to catch a lot of SmallDenseMap issues
because we spend a lot of time with fixed stable addresses in the inline
buffer.
The SmallDenseMap bug fix included makes grow() not double-destroy in
some cases. It also fixes a FIXME there, the code was pretty crappy. We
now don't have any wasted initialization, but we do move the entries in
inline bucket array an extra time. It's probably a better tradeoff, and
is much easier to get correct.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158639 91177308-0d34-0410-b5e6-96231b3b80d8
implementation.
This type includes an inline bucket array which is used initially. Once
it is exceeded, an array of 64 buckets is allocated on the heap. The
bucket count grows from there as needed. Some highlights of this
implementation:
- The inline buffer is very carefully aligned, and so supports types
with alignment constraints.
- It works hard to avoid aliasing issues.
- Supports types with non-trivial constructors, destructors, copy
constructions, etc. It works reasonably hard to minimize copies and
unnecessary initialization. The most common initialization is to set
keys to the empty key, and so that should be fast if at all possible.
This class has a performance / space trade-off. It tries to optimize for
relatively small maps, and so packs the inline bucket array densely into
the object. It will be marginally slower than a normal DenseMap in a few
use patterns, so it isn't appropriate everywhere.
The unit tests for DenseMap have been generalized a bit to support
running over different map implementations in addition to different
key/value types. They've then been automatically extended to cover the
new container through the magic of GoogleTest's typed tests.
All of this is still a bit rough though. I'm going to be cleaning up
some aspects of the implementation, documenting things better, and
adding tests which include non-trivial types. As soon as I'm comfortable
with the correctness, I plan to switch existing users of SmallMap over
to this class as it is already more correct w.r.t. construction and
destruction of objects iin the map.
Thanks to Benjamin Kramer for all the reviews of this and the lead-up
patches. That said, more review on this would really be appreciated. As
I've noted a few times, I'm quite surprised how hard it is to get the
semantics for a hashtable-based map container with a small buffer
optimization correct. =]
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158638 91177308-0d34-0410-b5e6-96231b3b80d8
array of a suitable size and alignment for any of a number of different
types to be stored into the character array.
The mechanisms for producing an explicitly aligned type are fairly
complex because this operation is poorly supported on all compilers.
We've spent a fairly significant amount of time experimenting with
different implementations inside of Google, and the one using explicitly
expanded templates has been the most robust.
Credit goes to Nick Lewycky for writing the first 20 versions or so of
this logic we had inside of Google. I based this on the only one to
actually survive. In case anyone is worried, yes we are both explicitly
re-contributing and re-licensing it for LLVM. =]
Once the issues with actually specifying the alignment are finished, it
turns out that most compilers don't in turn align anything the way they
are instructed. Testing of this logic against both Clang and GCC
indicate that the alignment constraints are largely ignored by both
compilers! I've come up with and used a work-around by wrapping each
alignment-hinted type directly in a struct, and using that struct to
align the character array through a union. This elaborate hackery is
terrifying, but I've included testing that caught a terrifying number of
bugs in every other technique I've tried.
All of this in order to implement a poor C++98 programmers emulation of
C++11 unrestricted unions in classes such as SmallDenseMap.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158597 91177308-0d34-0410-b5e6-96231b3b80d8
rather than the base class. Add a pile of boilerplate to indirect around
this.
This is pretty ugly, but it allows the super class to change the
representation of these values, which will be key for doing
a SmallDenseMap.
Suggestions on better method structuring / naming are welcome, but keep
in mind that SmallDenseMap won't have an 'unsigned' member to expose
a reference to... =/
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158586 91177308-0d34-0410-b5e6-96231b3b80d8
and a derived class that provides the allocation and growth strategy.
This is the first (and biggest) step toward building a SmallDenseMap
that actually behaves exactly the same as DenseMap, and supports all the
same types and interface points with the same semantics.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158585 91177308-0d34-0410-b5e6-96231b3b80d8
since then the entire expression must equal zero (similarly for other operations
with an absorbing element). With this in place a bunch of reassociate code for
handling constants is dead since it is all taken care of when linearizing. No
intended functionality change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158398 91177308-0d34-0410-b5e6-96231b3b80d8
topologies, it is quite possible for a leaf node to have huge multiplicity, for
example: x0 = x*x, x1 = x0*x0, x2 = x1*x1, ... rapidly gives a value which is x
raised to a vast power (the multiplicity, or weight, of x). This patch fixes
the computation of weights by correctly computing them no matter how big they
are, rather than just overflowing and getting a wrong value. It turns out that
the weight for a value never needs more bits to represent than the value itself,
so it is enough to represent weights as APInts of the same bitwidth and do the
right overflow-avoiding dance steps when computing weights. As a side-effect it
reduces the number of multiplies needed in some cases of large powers. While
there, in view of external uses (eg by the vectorizer) I made LinearizeExprTree
static, pushing the rank computation out into users. This is progress towards
fixing PR13021.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158358 91177308-0d34-0410-b5e6-96231b3b80d8
thread local data, embed them in the class using a uint64_t and make sure
we get compiler errors if there's a platform where this is not big enough.
This makes ThreadLocal more safe for using it in conjunction with CrashRecoveryContext.
Related to crash in rdar://11434201.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158342 91177308-0d34-0410-b5e6-96231b3b80d8
The TableGenBackend base class doesn't do much, and will be removed
completely soon.
Patch by Sean Silva!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158311 91177308-0d34-0410-b5e6-96231b3b80d8
This showed up the first time rend() was called on a bundled instruction
in the Mips backend.
Also avoid dereferencing end() in bundle_iterator::operator++().
We still don't have a place to put unit tests for this stuff.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158310 91177308-0d34-0410-b5e6-96231b3b80d8
The LiveRegMatrix represents the live range of assigned virtual
registers in a Live interval union per register unit. This is not
fundamentally different from the interference tracking in RegAllocBase
that both RABasic and RAGreedy use.
The important differences are:
- LiveRegMatrix tracks interference per register unit instead of per
physical register. This makes interference checks cheaper and
assignments slightly more expensive. For example, the ARM D7 reigster
has 24 aliases, so we would check 24 physregs before assigning to one.
With unit-based interference, we check 2 units before assigning to 2
units.
- LiveRegMatrix caches regmask interference checks. That is currently
duplicated functionality in RABasic and RAGreedy.
- LiveRegMatrix is a pass which makes it possible to insert
target-dependent passes between register allocation and rewriting.
Such passes could tweak the register assignments with interference
checking support from LiveRegMatrix.
Eventually, RABasic and RAGreedy will be switched to LiveRegMatrix.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158255 91177308-0d34-0410-b5e6-96231b3b80d8
OK, not really. We don't want to reintroduce the old rewriter hacks.
This patch extracts virtual register rewriting as a separate pass that
runs after the register allocator. This is possible now that
CodeGen/Passes.cpp can configure the full optimizing register allocator
pipeline.
The rewriter pass uses register assignments in VirtRegMap to rewrite
virtual registers to physical registers, and it inserts kill flags based
on live intervals.
These finalization steps are the same for the optimizing register
allocators: RABasic, RAGreedy, and PBQP.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158244 91177308-0d34-0410-b5e6-96231b3b80d8
This fixes an accidental dependence on static initialization order that I introduced yesterday.
Thank you Lang!!!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158215 91177308-0d34-0410-b5e6-96231b3b80d8
The commit is intended to fix rdar://11540023.
It is implemented as part of peephole optimization. We can actually implement
this in the SelectionDAG lowering phase.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158122 91177308-0d34-0410-b5e6-96231b3b80d8
LLVM is now -Wunused-private-field clean except for
- lib/MC/MCDisassembler/Disassembler.h. Not sure why it keeps all those unaccessible fields.
- gtest.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158096 91177308-0d34-0410-b5e6-96231b3b80d8
There are some that I didn't remove this round because they looked like
obvious stubs. There are dead variables in gtest too, they should be
fixed upstream.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158090 91177308-0d34-0410-b5e6-96231b3b80d8
Don't print out the register number and spill weight, making the TRI
argument unnecessary.
This allows callers to interpret the reg field. It can currently be a
virtual register, a physical register, a spill slot, or a register unit.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158031 91177308-0d34-0410-b5e6-96231b3b80d8
Instead of computing a live interval per physreg, LiveIntervals can
compute live intervals per register unit. This makes impossible the
confusing situation where aliasing registers could have overlapping live
intervals. It should also make fixed interferernce checking cheaper
since registers have fewer register units than aliases.
Live intervals for regunits are computed on demand, using MRI use-def
chains and the new LiveRangeCalc class. Only regunits live in to ABI
blocks are precomputed during LiveIntervals::runOnMachineFunction().
The regunit liveness computations don't depend on LiveVariables.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158029 91177308-0d34-0410-b5e6-96231b3b80d8
expression (a * b + c) that can be implemented as a fused multiply-add (fma)
if the target determines that this will be more efficient. This intrinsic
will be used to implement FP_CONTRACT support and an aggressive FMA formation
mode.
If your target has a fast FMA instruction you should override the
isFMAFasterThanMulAndAdd method in TargetLowering to return true.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@158014 91177308-0d34-0410-b5e6-96231b3b80d8
Changed type of Items collection: from std::vector to std::list.
Also some small fixes made in IntegersSubset.h, IntegersSubsetMapping.h and IntegersSubsetTest.cpp.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157987 91177308-0d34-0410-b5e6-96231b3b80d8
This allows a subtarget to explicitly specify the issue width and
other properties without providing pipeline stage details for every
instruction.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157979 91177308-0d34-0410-b5e6-96231b3b80d8
valid itinerary but no pipeline stages.
An itinerary can contain useful scheduling information without specifying pipeline stages for each instruction.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157977 91177308-0d34-0410-b5e6-96231b3b80d8
It is an old function that does a lot more than required by
CalcSpillWeights, which was the only remaining caller.
The isRematerializable() function never actually sets the isLoad
argument, so don't try to compute that.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157973 91177308-0d34-0410-b5e6-96231b3b80d8
IntRange converted from struct to class. So main change everywhere is replacement of ".Low/High" with ".getLow/getHigh()"
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157884 91177308-0d34-0410-b5e6-96231b3b80d8
IntegersSubsetGeneric, IntegersSubsetMapping: added IntTy template parameter, that allows use either APInt or IntItem. This change allows to write unittest for these classes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157880 91177308-0d34-0410-b5e6-96231b3b80d8
These functions exposed the layout of the underlying data tables as
null-terminated uint16_t arrays.
Use the new MCSubRegIterator, MCSuperRegIterator, and MCRegAliasIterator
classes instead.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157855 91177308-0d34-0410-b5e6-96231b3b80d8
No functional change intended.
Sorry for the churn. The iterator classes are supposed to help avoid
giant commits like this one in the future. The TableGen-produced
register lists are getting quite large, and it may be necessary to
change the table representation.
This makes it possible to do so without changing all clients (again).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157854 91177308-0d34-0410-b5e6-96231b3b80d8
IntegersSubset devided into IntegersSubsetGeneric and into IntegersSubset itself. The first has no references to ConstantInt and works with IntItem only.
IntegersSubsetMapping also made generic. Here added second template parameter "IntegersSubsetTy" that allows to use on of two IntegersSubset types described below.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157815 91177308-0d34-0410-b5e6-96231b3b80d8
IntItem cleanup. IntItemBase, IntItemConstantIntImp and IntItem merged into IntItem. All arithmetic operators was propogated from APInt. Also added comparison operators <,>,<=,>=. Currently you will find set of macros that propogates operators from APInt to IntItem in the beginning of IntegerSubset. Note that THESE MACROS WILL REMOVED after all passes will case-ranges compatible. Also note that these macros much smaller pain that something like this:
if (V->getValue().ugt(AnotherV->getValue()) { ... }
These changes made IntItem full featured integer object. It allows to make IntegerSubset class generic (move out all ConstantInt references inside and add unit-tests) in next commits.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157810 91177308-0d34-0410-b5e6-96231b3b80d8
This patch will optimize the following
movq %rdi, %rax
subq %rsi, %rax
cmovsq %rsi, %rdi
movq %rdi, %rax
to
cmpq %rsi, %rdi
cmovsq %rsi, %rdi
movq %rdi, %rax
Perform this optimization if the actual result of SUB is not used.
rdar: 11540023
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157755 91177308-0d34-0410-b5e6-96231b3b80d8
Reg-units are named after their root registers, and most units have a
single root, so they simply print as 'AL', 'XMM0', etc. The rare dual
root reg-units print as FPSCR~FPSCR_NZCV, FP0~ST7, ...
The printing piggybacks on the existing register name tables, so no
extra const data space is required.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157754 91177308-0d34-0410-b5e6-96231b3b80d8
Each register unit has one or two root registers. The full set of
registers containing a given register unit can be computed as the union
of the root registers and their super-registers.
Provide an MCRegUnitRootIterator class to enumerate the roots.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157753 91177308-0d34-0410-b5e6-96231b3b80d8
This also required making recursive simplifications until
nothing changes or a hard limit (currently 3) is hit.
With the simplification in place indvars can canonicalize
loops of the form
for (unsigned i = 0; i < a-b; ++i)
into
for (unsigned i = 0; i != a-b; ++i)
which used to fail because SCEV created a weird umax expr
for the backedge taken count.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157701 91177308-0d34-0410-b5e6-96231b3b80d8
Also add subclasses MCSubRegIterator, MCSuperRegIterator, and
MCRegAliasIterator.
These iterators provide an abstract interface to the MCRegisterInfo
register lists so the internal representation can be changed without
changing all clients.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157695 91177308-0d34-0410-b5e6-96231b3b80d8
Besides adding the new insertPass function, this patch uses it to
enhance the existing -print-machineinstrs so that the MachineInstrs
after a specific pass can be printed.
Patch by Bin Zeng!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157655 91177308-0d34-0410-b5e6-96231b3b80d8
The register unit lists are typically much shorter than the register
overlap lists, and the backing table for register units has better cache
locality because it is smaller.
This makes llc about 0.5% faster. The regsOverlap() function isn't that hot.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157651 91177308-0d34-0410-b5e6-96231b3b80d8
Register units are already used internally in TableGen to compute
register pressure sets and overlapping registers. This patch makes them
available to the code generators.
The register unit lists are differentially encoded so they can be reused
for many related registers. This keeps the total size of the lists below
200 bytes for most targets. ARM has the largest table at 560 bytes.
Add an MCRegUnitIterator for traversing the register unit lists. It
provides an abstract interface so the representation can be changed in
the future without changing all clients.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157650 91177308-0d34-0410-b5e6-96231b3b80d8
This required light surgery on the assembler and disassembler
because the instructions use an uncommon encoding. They are
the only two instructions in x86 that use register operands
and two immediates.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157634 91177308-0d34-0410-b5e6-96231b3b80d8
Attribute bits above 1<<30 are now encoded correctly. Additionally,
the encoding/decoding functionality has been hoisted to helper functions
in Attributes.h in an effort to help the encoding/decoding to stay in
sync with the Attribute bitcode definitions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157581 91177308-0d34-0410-b5e6-96231b3b80d8
Implemented IntItem - the wrapper around APInt. Why not to use APInt item directly right now?
1. It will very difficult to implement case ranges as series of small patches. We got several large and heavy patches. Each patch will about 90-120 kb. If you replace ConstantInt with APInt in SwitchInst you will need to changes at the same time all Readers,Writers and absolutely all passes that uses SwitchInst.
2. We can implement APInt pool inside and save memory space. E.g. we use several switches that works with 256 bit items (switch on signatures, or strings). We can avoid value duplicates in this case.
3. IntItem can be easyly easily replaced with APInt.
4. Currenly we can interpret IntItem both as ConstantInt and as APInt. It allows to provide SwitchInst methods that works with ConstantInt for non-updated passes.
Why I need it right now? Currently I need to update SimplifyCFG pass (EqualityComparisons). I need to work with APInts directly a lot, so peaces of code
ConstantInt *V = ...;
if (V->getValue().ugt(AnotherV->getValue()) {
...
}
will look awful. Much more better this way:
IntItem V = ConstantIntVal->getValue();
if (AnotherV < V) {
}
Of course any reviews are welcome.
P.S.: I'm also going to rename ConstantRangesSet to IntegersSubset, and CRSBuilder to IntegersSubsetMapping (allows to map individual subsets of integers to the BasicBlocks).
Since in future these classes will founded on APInt, it will possible to use them in more generic ways.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157576 91177308-0d34-0410-b5e6-96231b3b80d8
The only missing part is insert(), which uses a pair of parameters and I haven't
figured out how to convert it to rvalue references. It's now possible to use a
DenseMap with std::unique_ptr values :)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157539 91177308-0d34-0410-b5e6-96231b3b80d8
to pass around a struct instead of a large set of individual values. This
cleans up the interface and allows more information to be added to the struct
for future targets without requiring changes to each and every target.
NV_CONTRIB
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157479 91177308-0d34-0410-b5e6-96231b3b80d8
The Hazard checker implements in-order contraints, or interlocked
resources. Ready instructions with hazards do not enter the available
queue and are not visible to other heuristics.
The major code change is the addition of SchedBoundary to encapsulate
the state at the top or bottom of the schedule, including both a
pending and available queue.
The scheduler now counts cycles in sync with the hazard checker. These
are minimum cycle counts based on known hazards.
Targets with no itinerary (x86_64) currently remain at cycle 0. To fix
this, we need to provide some maximum issue width for all targets. We
also need to add the concept of expected latency vs. minimum latency.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157427 91177308-0d34-0410-b5e6-96231b3b80d8
move EmitGEPOffset from InstCombine to Transforms/Utils/Local.h
(a draft of this) patch reviewed by Andrew, thanks.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157261 91177308-0d34-0410-b5e6-96231b3b80d8
This helps compile time when the greedy register allocator splits live
ranges in giant functions. Without the bias, we would try to grow
regions through the giant edge bundles, usually to find out that the
region became too big and expensive.
If a live range has many uses in blocks near the giant bundle, the small
negative bias doesn't make a big difference, and we still consider
regions including the giant edge bundle.
Giant edge bundles are usually connected to landing pads or indirect
branches.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157174 91177308-0d34-0410-b5e6-96231b3b80d8
This class is meant to be the primary interface for examining a live
range in the vicinity on a given instruction. It avoids all the messy
dealings with iterators and early clobbers.
This is a more abstract interface to live ranges, hiding the
implementation as a vector of segments.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157141 91177308-0d34-0410-b5e6-96231b3b80d8
Otherwise just looking up a value in the map requires creating a VH, adding it to the use lists and destroying it again.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157124 91177308-0d34-0410-b5e6-96231b3b80d8
getUDivExpr attempts to simplify by checking for overflow.
isLoopEntryGuardedByCond then evaluates the loop predicate which
may lead to the same getUDivExpr causing endless recursion.
Fixes PR12868: clang 3.2 segmentation fault.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157092 91177308-0d34-0410-b5e6-96231b3b80d8
Use a dedicated MachO load command to annotate data-in-code regions.
This is the same format the linker produces for final executable images,
allowing consistency of representation and use of introspection tools
for both object and executable files.
Data-in-code regions are annotated via ".data_region"/".end_data_region"
directive pairs, with an optional region type.
data_region_directive := ".data_region" { region_type }
region_type := "jt8" | "jt16" | "jt32" | "jta32"
end_data_region_directive := ".end_data_region"
The previous handling of ARM-style "$d.*" labels was broken and has
been removed. Specifically, it didn't handle ARM vs. Thumb mode when
marking the end of the section.
rdar://11459456
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@157062 91177308-0d34-0410-b5e6-96231b3b80d8
named metadata list, unlike all the other global objects (global variables,
functions, aliases), so add that for consistency.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@156915 91177308-0d34-0410-b5e6-96231b3b80d8
Many targets always use the same bitwise encoding value for physical
registers in all (or most) instructions. Add this mapping to the
.td files and TableGen'erate the information and expose an accessor
in MCRegisterInfo.
patch by Tom Stellard.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@156829 91177308-0d34-0410-b5e6-96231b3b80d8
Add the MCRegisterInfo to the factories and constructors.
Patch by Tom Stellard <Tom.Stellard@amd.com>.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@156828 91177308-0d34-0410-b5e6-96231b3b80d8
