The powi intrinsic requires special handling because it always takes a single
integer power regardless of the result type. As a result, we can vectorize
only if the powers are equal. Fixes PR12364.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153797 91177308-0d34-0410-b5e6-96231b3b80d8
ARMConstantIslandPass still has bugs where jump table compression can
cause constant pool entries to go out of range.
Add a safety margin of 2 bytes when placing constant islands, but use
the real max displacement for verification.
<rdar://problem/11156595>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153789 91177308-0d34-0410-b5e6-96231b3b80d8
The 8-bit payload is not contiguous in the opcode. Move the upper nibble
over 4 bits into the correct place.
rdar://11158641
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153780 91177308-0d34-0410-b5e6-96231b3b80d8
When an immediate is both a value [t2_]so_imm and a [t2_]so_imm_neg,
we want to use the non-negated form to make sure we prefer the normal
encoding, not the aliased encoding via the negation of, e.g., 'cmp.w'.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153770 91177308-0d34-0410-b5e6-96231b3b80d8
Make the non-tied register operand names line up with what the base
class encoding handler expects.
rdar://11157236
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153766 91177308-0d34-0410-b5e6-96231b3b80d8
For 'adds r2, r2, #56' outside of an IT block, the 16-bit encoding T2
can be used for this syntax. Prefer the narrow encoding when possible.
rdar://11156277
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153759 91177308-0d34-0410-b5e6-96231b3b80d8
1. The main works will made in the RuntimeDyLdImpl with uses the ObjectFile class. RuntimeDyLdMachO and RuntimeDyLdELF now only parses relocations and resolve it. This is allows to make improvements of the RuntimeDyLd more easily. In addition the support for COFF can be easily added.
2. Added ARM relocations to RuntimeDyLdELF.
3. Added support for stub functions for the ARM, allowing to do a long branch.
4. Added support for external functions that are not loaded from the object files, but can be loaded from external libraries. Now MCJIT can correctly execute the code containing the printf, putc, and etc.
5. The sections emitted instead functions, thanks Jim Grosbach. MemoryManager.startFunctionBody() and MemoryManager.endFunctionBody() have been removed.
6. MCJITMemoryManager.allocateDataSection() and MCJITMemoryManager. allocateCodeSection() used JMM->allocateSpace() instead of JMM->allocateCodeSection() and JMM->allocateDataSection(), because I got an error: "Cannot allocate an allocated block!" with object file contains more than one code or data sections.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153754 91177308-0d34-0410-b5e6-96231b3b80d8
here but it has no other uses, then we have a problem. E.g.,
int foo (const int *x) {
char a[*x];
return 0;
}
If we assign 'a' a vreg and fast isel later on has to use the selection
DAG isel, it will want to copy the value to the vreg. However, there are
no uses, which goes counter to what selection DAG isel expects.
<rdar://problem/11134152>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153705 91177308-0d34-0410-b5e6-96231b3b80d8
This pass splits basic blocks to insert constant islands, and it
doesn't recompute the live-in lists. No later passes depend on accurate
liveness information.
This fixes PR12410 where the machine code verifier was complaining.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153700 91177308-0d34-0410-b5e6-96231b3b80d8
We are sometimes allocatinog from the DPair register class which
contains odd-even pairs in addition to the Q registers.
Place the Q registers first in the DPair allocation order as they can be
copied with a single instruction. The odd-even pairs should only be
allocated as a last resort.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153699 91177308-0d34-0410-b5e6-96231b3b80d8
The CMP->CMN alias was matching for an immediate of zero when it
should only match for negative values.
rdar://11129224
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153689 91177308-0d34-0410-b5e6-96231b3b80d8
ARM recently gained DPair, DTriple, and DQuad register classes.
Update copyPhysReg() to handle copies in these register classes.
No test case, it is difficult to make the register allocator emit the
odd copies reliably. The missing DPair copy caused a failure on
partialsums in the nightly test suite.
<rdar://problem/11147997>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153686 91177308-0d34-0410-b5e6-96231b3b80d8
CodeGenPrepare sinks compare instructions down to their uses to prevent
live flags and predicate registers across basic blocks.
PRE of a compare instruction prevents that, forcing the i1 compare
result into a general purpose register. That is usually more expensive
than the redundant compare PRE was trying to eliminate in the first
place.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153657 91177308-0d34-0410-b5e6-96231b3b80d8
This is a code change to add support for changing instruction sequences of the form:
load
inc/dec of 8/16/32/64 bits
store
into the appropriate X86 inc/dec through memory instruction:
inc[qlwb] / dec[qlwb]
The checks that were in X86DAGToDAGISel::Select(SDNode *Node)>>ISD::STORE have been extracted to isLoadIncOrDecStore and reworked to use the better
named wrappers for getOperand(unsigned) (e.g. getOffset()) and replaced Chain.getNode() with LoadNode. The comments have also been expanded.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153635 91177308-0d34-0410-b5e6-96231b3b80d8
This is a code change to add support for changing instruction sequences of the form:
load
inc/dec of 8/16/32/64 bits
store
into the appropriate X86 inc/dec through memory instruction:
inc[qlwb] / dec[qlwb]
The checks that were in X86DAGToDAGISel::Select(SDNode *Node)>>ISD::STORE have been extracted to isLoadIncOrDecStore and reworked to use the better
named wrappers for getOperand(unsigned) (e.g. getOffset()) and replaced Chain.getNode() with LoadNode. The comments have also been expanded.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153617 91177308-0d34-0410-b5e6-96231b3b80d8
Some targets still mess up the liveness information, but that isn't
verified after MRI->invalidateLiveness().
The verifier can still check other useful things like register classes
and CFG, so it should be enabled after all passes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153615 91177308-0d34-0410-b5e6-96231b3b80d8
The late scheduler depends on accurate liveness information if it is
breaking anti-dependencies, so we should be able to verify it.
Relax the terminator checking in the machine code verifier so it can
handle the basic blocks created by if conversion.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153614 91177308-0d34-0410-b5e6-96231b3b80d8
When an strd instruction doesn't get the registers it wants, it can be
expanded into two str instructions. Make sure the first str doesn't kill
the base register in the case where the base and data registers are
identical:
t2STRi12 %R0<kill>, %R0, 4, pred:14, pred:%noreg
t2STRi12 %R2<kill>, %R0, 8, pred:14, pred:%noreg
<rdar://problem/11101911>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153611 91177308-0d34-0410-b5e6-96231b3b80d8
When a number of sub-register VLRDS instructions are combined into a
VLDM, preserve any super-register implicit defs. This is required to
keep the register scavenger and machine code verifier happy.
Enable machine code verification after ARMLoadStoreOptimizer.
ARM/2012-01-26-CopyPropKills.ll was failing because of this.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153610 91177308-0d34-0410-b5e6-96231b3b80d8
The arm_neon intrinsics can create virtual registers from the DPair
register class which allows both even-odd and odd-even D-register pairs.
This fixes PR12389.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153603 91177308-0d34-0410-b5e6-96231b3b80d8
Extract the liveness verification into its own method.
This makes it possible to run the machine code verifier after liveness
information is no longer required to be valid.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153596 91177308-0d34-0410-b5e6-96231b3b80d8
Revert r153519: "ARMLoadStoreOptimizer invalidates register liveness."
These patches caused miscompilations in povray by turning off branch
folding's updating of live-in lists.
It turns out the the late scheduler depends on the live-in lists, even
if it doesn't need correct kill flags.
<rdar://problem/11139228>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153593 91177308-0d34-0410-b5e6-96231b3b80d8
Original commit message for r153521 (aka r153423):
Use the new range metadata in computeMaskedBits and add a new optimization to
instruction simplify that lets us remove an and when loding a boolean value.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153587 91177308-0d34-0410-b5e6-96231b3b80d8
blocks in the function cloner. This removes the last case of trivially
dead code that I've been seeing in the wild getting inlined, analyzed,
re-inlined, optimized, only to be deleted. Nukes a FIXME from the
cleanup tests.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153572 91177308-0d34-0410-b5e6-96231b3b80d8
imposes a constraint that GOT16 referring to a local symbol or HI16 has to be
followed immediately by a matching LO16 relocation.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153553 91177308-0d34-0410-b5e6-96231b3b80d8
them as machine instructions. Directives ".set noat" and ".set at" are now
emitted only at the beginning and end of a function except in the case where
they are emitted to enclose .cpload with an immediate operand that doesn't fit
in 16-bit field or unaligned load/stores.
Also, make the following changes:
- Remove function isUnalignedLoadStore and use a switch-case statement to
determine whether an instruction is an unaligned load or store.
- Define helper function CreateMCInst which generates an instance of an MCInst
from an opcode and a list of operands.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153552 91177308-0d34-0410-b5e6-96231b3b80d8
If EmitNOAT is true, directives ".set noat" and ".set at" are emitted at the
beginning and end of a function.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153528 91177308-0d34-0410-b5e6-96231b3b80d8
undefined behavior, which Rafael was kind enough to fix.
Original commit message for r153423:
Use the new range metadata in computeMaskedBits and add a new optimization to
instruction simplify that lets us remove an and when loding a boolean value.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153521 91177308-0d34-0410-b5e6-96231b3b80d8
This pass tries to update kill flags, but there are still many bugs.
Passes after the load/store optimizer don't need accurate liveness, so
don't even try.
<rdar://problem/11101911>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153519 91177308-0d34-0410-b5e6-96231b3b80d8
Branch folding can use a register scavenger to update liveness
information when required. Don't do that if liveness information is
already invalid.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153517 91177308-0d34-0410-b5e6-96231b3b80d8
Late optimization passes like branch folding and tail duplication can
transform the machine code in a way that makes it expensive to keep the
register liveness information up to date. There is a fuzzy line between
register allocation and late scheduling where the liveness information
degrades.
The MRI::tracksLiveness() flag makes the line clear: While true,
liveness information is accurate, and can be used for register
scavenging. Once the flag is false, liveness information is not
accurate, and can only be used as a hint.
Late passes generally don't need the liveness information, but they will
sometimes use the register scavenger to help update it. The scavenger
enforces strict correctness, and we have to spend a lot of code to
update register liveness that may never be used.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153511 91177308-0d34-0410-b5e6-96231b3b80d8
size bloat. Unfortunately, I expect this to disable the majority of the
benefit from r152737. I'm hopeful at least that it will fix PR12345. To
explain this requires... quite a bit of backstory I'm afraid.
TL;DR: The change in r152737 actually did The Wrong Thing for
linkonce-odr functions. This change makes it do the right thing. The
benefits we saw were simple luck, not any actual strategy. Benchmark
numbers after a mini-blog-post so that I've written down my thoughts on
why all of this works and doesn't work...
To understand what's going on here, you have to understand how the
"bottom-up" inliner actually works. There are two fundamental modes to
the inliner:
1) Standard fixed-cost bottom-up inlining. This is the mode we usually
think about. It walks from the bottom of the CFG up to the top,
looking at callsites, taking information about the callsite and the
called function and computing th expected cost of inlining into that
callsite. If the cost is under a fixed threshold, it inlines. It's
a touch more complicated than that due to all the bonuses, weights,
etc. Inlining the last callsite to an internal function gets higher
weighth, etc. But essentially, this is the mode of operation.
2) Deferred bottom-up inlining (a term I just made up). This is the
interesting mode for this patch an r152737. Initially, this works
just like mode #1, but once we have the cost of inlining into the
callsite, we don't just compare it with a fixed threshold. First, we
check something else. Let's give some names to the entities at this
point, or we'll end up hopelessly confused. We're considering
inlining a function 'A' into its callsite within a function 'B'. We
want to check whether 'B' has any callers, and whether it might be
inlined into those callers. If so, we also check whether inlining 'A'
into 'B' would block any of the opportunities for inlining 'B' into
its callers. We take the sum of the costs of inlining 'B' into its
callers where that inlining would be blocked by inlining 'A' into
'B', and if that cost is less than the cost of inlining 'A' into 'B',
then we skip inlining 'A' into 'B'.
Now, in order for #2 to make sense, we have to have some confidence that
we will actually have the opportunity to inline 'B' into its callers
when cheaper, *and* that we'll be able to revisit the decision and
inline 'A' into 'B' if that ever becomes the correct tradeoff. This
often isn't true for external functions -- we can see very few of their
callers, and we won't be able to re-consider inlining 'A' into 'B' if
'B' is external when we finally see more callers of 'B'. There are two
cases where we believe this to be true for C/C++ code: functions local
to a translation unit, and functions with an inline definition in every
translation unit which uses them. These are represented as internal
linkage and linkonce-odr (resp.) in LLVM. I enabled this logic for
linkonce-odr in r152737.
Unfortunately, when I did that, I also introduced a subtle bug. There
was an implicit assumption that the last caller of the function within
the TU was the last caller of the function in the program. We want to
bonus the last caller of the function in the program by a huge amount
for inlining because inlining that callsite has very little cost.
Unfortunately, the last caller in the TU of a linkonce-odr function is
*not* the last caller in the program, and so we don't want to apply this
bonus. If we do, we can apply it to one callsite *per-TU*. Because of
the way deferred inlining works, when it sees this bonus applied to one
callsite in the TU for 'B', it decides that inlining 'B' is of the
*utmost* importance just so we can get that final bonus. It then
proceeds to essentially force deferred inlining regardless of the actual
cost tradeoff.
The result? PR12345: code bloat, code bloat, code bloat. Another result
is getting *damn* lucky on a few benchmarks, and the over-inlining
exposing critically important optimizations. I would very much like
a list of benchmarks that regress after this change goes in, with
bitcode before and after. This will help me greatly understand what
opportunities the current cost analysis is missing.
Initial benchmark numbers look very good. WebKit files that exhibited
the worst of PR12345 went from growing to shrinking compared to Clang
with r152737 reverted.
- Bootstrapped Clang is 3% smaller with this change.
- Bootstrapped Clang -O0 over a single-source-file of lib/Lex is 4%
faster with this change.
Please let me know about any other performance impact you see. Thanks to
Nico for reporting and urging me to actually fix, Richard Smith, Duncan
Sands, Manuel Klimek, and Benjamin Kramer for talking through the issues
today.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153506 91177308-0d34-0410-b5e6-96231b3b80d8
MachinePointerInfo when getStore is called to create a node that stores an
argument passed in register to the stack. Without this change, the post RA
scheduler will fail to discover the dependencies between the stores
instructions and the instructions that load from a structure passed by value.
The link to the related discussion is here:
http://lists.cs.uiuc.edu/pipermail/llvmdev/2012-March/048055.html
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153499 91177308-0d34-0410-b5e6-96231b3b80d8
set it in MipsMCCodeEmitter::getMachineOpValue. Assert in getMachineOpValue if
MachineOperand MO is of an unexpected type.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153494 91177308-0d34-0410-b5e6-96231b3b80d8
copies being considered for removal. Make sure to track all of the copies,
rather than just the most recent encountered, by holding a DenseSet instead of
an unsigned in SrcMap.
No test case - couldn't reduce something with a sane size.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153487 91177308-0d34-0410-b5e6-96231b3b80d8
produces a 32-bit immediate which is consumed by the use. It tries to
fold the immediate by breaking it into two parts and fold them into the
immmediate fields of two uses. e.g
movw r2, #40885
movt r3, #46540
add r0, r0, r3
=>
add.w r0, r0, #3019898880
add.w r0, r0, #30146560
;
However, this transformation is incorrect if the user produces a flag. e.g.
movw r2, #40885
movt r3, #46540
adds r0, r0, r3
=>
add.w r0, r0, #3019898880
adds.w r0, r0, #30146560
Note the adds.w may not set the carry flag even if the original sequence
would.
rdar://11116189
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153484 91177308-0d34-0410-b5e6-96231b3b80d8
backtrace locations.
Testcase forthcoming, but I wanted to get some testing here.
Should fix:
PR12323
PR12314
rdar://11091100
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153471 91177308-0d34-0410-b5e6-96231b3b80d8
relocations. The algorithm is the same as
that for x86_64. Scattered relocations, a
feature present in i386 but not on x86_64,
are not yet supported.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153466 91177308-0d34-0410-b5e6-96231b3b80d8
Original commit message:
Use the new range metadata in computeMaskedBits and add a new optimization to
instruction simplify that lets us remove an and when loading a boolean value.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@153452 91177308-0d34-0410-b5e6-96231b3b80d8
