This is the initial checkin of the basic-block autovectorization pass along with some supporting vectorization infrastructure.
Special thanks to everyone who helped review this code over the last several months (especially Tobias Grosser).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149468 91177308-0d34-0410-b5e6-96231b3b80d8
Changing arguments from being passed as fixed to varargs is unsafe, as
the ABI may require they be handled differently (stack vs. register, for
example).
Remove two tests which rely on the bitcast being folded into the direct
call, which is exactly the transformation that's unsafe.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149457 91177308-0d34-0410-b5e6-96231b3b80d8
symbol from an assignment. In this case the symbol did not have a fragment so
MCObjectWriter::IsSymbolRefDifferenceFullyResolved() should not have been
calling IsSymbolRefDifferenceFullyResolvedImpl() with a NULL fragment and should
just have returned false in that case.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149442 91177308-0d34-0410-b5e6-96231b3b80d8
This new function provides a way to get the Mac OS X version number from
either generic "darwin" triples of macosx triples.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149438 91177308-0d34-0410-b5e6-96231b3b80d8
This removes implicit assumption about the form of MI coming into regalloc. In particular, it should be independent of ProcessImplicitDefs which will eventually become a standard part of coming out of SSA--unless we simply can eliminate IMPLICIT_DEF completely. Current unit tests expose this once I remove incidental pass ordering restrictions.
This is not a final fix. Just a temporary workaround until I figure out the right way.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149360 91177308-0d34-0410-b5e6-96231b3b80d8
kicking in the big win of ConstantDataArray. As part of this, change
the implementation of GetConstantStringInfo in ValueTracking to work
with ConstantDataArray (and not ConstantArray) making it dramatically,
amazingly, more efficient in the process and renaming it to
getConstantStringInfo.
This keeps around a GetConstantStringInfo entrypoint that (grossly)
forwards to getConstantStringInfo and constructs the std::string
required, but existing clients should move over to
getConstantStringInfo instead.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149351 91177308-0d34-0410-b5e6-96231b3b80d8
vectors of all one bits to be printed more cleverly in the AsmPrinter.
Unfortunately, the byte value for all one bits is the same with
-fsigned-char as the error return of '-1'. Force this to be the unsigned
byte value when returning it to avoid this problem, and update the test
case for the shiny new behavior.
Yay for building LLVM and Clang with -funsigned-char.
Chris, please review, and let me know if there is any reason to not
desire this change. It seems good on the surface, and certainly intended
based on the code written.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149299 91177308-0d34-0410-b5e6-96231b3b80d8
The redzones emitted by AddressSanitizer for CFString instances confuse the linker and are of little use, so we shouldn't add them.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149243 91177308-0d34-0410-b5e6-96231b3b80d8
- Don't call malloc+free in the very hot forward().
- Don't call isTiedToDefOperand().
- Don't create BitVector temporaries.
- Merge DeadRegs into KillRegs.
- Eliminate the early clobber checks, they were irrelevant to scavenging.
- Remove unnecessary code from -Asserts builds.
This speeds up ARM PEI by 3.4x and overall llc -O0 codegen time by 11%.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149189 91177308-0d34-0410-b5e6-96231b3b80d8
Sometimes there is only one 'resume' instruction per function. In those
situations, we don't need a separate block for the call to _Unwind_Resume. In
fact, it adds a lot of overhead to code-gen if we do that -- especially at -O0.
If we have a single 'resume' instruction, just generate the call within that
block.
<rdar://problem/10694814>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149159 91177308-0d34-0410-b5e6-96231b3b80d8
Unfortunately I also had to disable constant-pool-sharing.ll the code it tests has been
updated to use the IL logic.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149148 91177308-0d34-0410-b5e6-96231b3b80d8
around within a basic block while maintaining live-intervals.
Updated ScheduleTopDownLive in MachineScheduler.cpp to use the moveInstr API
when reordering MIs.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149147 91177308-0d34-0410-b5e6-96231b3b80d8
GEP instructions are there for the compiler and shouldn't really output much
code (if any at all). When a GEP is stored in the entry block, Fast ISel (for
one) will not know that it could fold it into further uses. For instance, inside
of the EH handling code. This results in a lot of unnecessary spills and loads
which bloat code and slows down pretty much everything.
<rdar://problem/10694814>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149114 91177308-0d34-0410-b5e6-96231b3b80d8
mid-level constant folding APIs instead of doing its own analysis.
This makes it more general (e.g. can now share a <2 x i64> with a
<4 x i32>) and avoid duplicating a bunch of logic.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149111 91177308-0d34-0410-b5e6-96231b3b80d8
Adjust an example MachObjectWriter diagnostic to use the information
to issue a better message.
Before:
LLVM ERROR: unknown ARM fixup kind!
After:
x.s:6:5: error: unsupported relocation on symbol
beq bar
^
rdar://9800182
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149093 91177308-0d34-0410-b5e6-96231b3b80d8
The Win64 calling convention has xmm6-15 as callee-saved while still
clobbering all ymm registers.
Add a YMM_HI_6_15 pseudo-register that aliases the clobbered part of the
ymm registers, and mark that as call-clobbered. This allows live xmm
registers across calls.
This hack wouldn't be necessary with RegisterMask operands representing
the call clobbers, but they are not quite operational yet.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149088 91177308-0d34-0410-b5e6-96231b3b80d8
we're at it, allow PatternMatch's "neg" pattern to match integer
vector negations, and enhance ComputeNumSigned bits to handle
shl of vectors.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149082 91177308-0d34-0410-b5e6-96231b3b80d8
ConstantExpr::getWithOperandReplaced and ConstantExpr::replaceUsesOfWithOnConstant
in terms of ConstantExpr::getWithOperands. While we're at it,
make sure that ConstantExpr::getWithOperands covers all instructions: it was
missing insert/extractvalue.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149076 91177308-0d34-0410-b5e6-96231b3b80d8
MachineBasicBlock::canFallThrough(). We're interested in the state of the
instruction (i.e., is this a barrier or not?), not if the instruction is
predicable or not.
rdar://10501092
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149070 91177308-0d34-0410-b5e6-96231b3b80d8
The live range of the source register may be extended when a redundant
copy is eliminated. Make sure any kill flags between the two copies are
cleared.
This fixes PR11765.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149069 91177308-0d34-0410-b5e6-96231b3b80d8
This enables the linker to match concrete relocation types (absolute or relative) with whatever library or C++ support code is being linked against.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149057 91177308-0d34-0410-b5e6-96231b3b80d8
. "fptosi" and "fptoui" IR instructions are defined with round-to-zero rounding mode.
. Currently for AVX mode for <4xdouble> and <8xdouble> the "VCVTPD2DQ.128" and "VCVTPD2DQ.256" instructions are selected (for .fp_to_sint. DAG node operation ) by AVX codegen. However they use round-to-nearest-even rounding mode.
. Consequently, the conversion produces incorrect numbers.
The fix is to replace selection of VCVTPD2DQ instructions with VCVTTPD2DQ instructions. The latter use truncate (i.e. round-to-zero) rounding mode.
As .fp_to_sint. DAG node operation is used only for lowering of "fptosi" and "fptoui" IR instructions, the fix in X86InstrSSE.td definition file doesn.t have an impact on other LLVM flows.
The patch includes changes in the .td file, LIT test for the changes and a fix in a legacy LIT test (which produced asm code conflicting with LLVN IR spec).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149056 91177308-0d34-0410-b5e6-96231b3b80d8
more robust) ways to do what it was doing now. Also, add static methods
for decoding a ShuffleVector mask.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149028 91177308-0d34-0410-b5e6-96231b3b80d8
ConstantVector. Fix some outright bugs in the implementation of
ConstantArray and Constant struct, which would cause us to not make
one big UndefValue when asking for an array/struct with all undef
elements. Enhance Constant::isAllOnesValue to work with
ConstantDataVector.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149021 91177308-0d34-0410-b5e6-96231b3b80d8
to reduce the number of cast<>'s we have. This allows someone to use
things like Ty->getVectorNumElements() instead of
cast<VectorType>(Ty)->getNumElements() when you know that a type is a
vector.
It would be a great general cleanup to move the codebase to use these,
I will do so in the code I'm touching.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148999 91177308-0d34-0410-b5e6-96231b3b80d8
This boils down to using MachineOperand::readsReg() more.
This fixes PR11829 where a use ended up after the first def when
lowering REG_SEQUENCE instructions involving IMPLICIT_DEFs.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148996 91177308-0d34-0410-b5e6-96231b3b80d8
function. They don't appear to be used, and are inconsistent with handling of
other physreg intervals (i.e. intervals that are not live-in) where ranges are
not inserted for aliases.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148986 91177308-0d34-0410-b5e6-96231b3b80d8
"Although a Thumb2 instruction, the IT mnemonic shall be permitted in
ARM mode, and the condition verified to match the condition code(s)
on the following instruction(s)."
PR11853
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148969 91177308-0d34-0410-b5e6-96231b3b80d8
savings from a pointer argument becoming an alloca. Sometimes callees will even
compare a pointer to null and then branch to an otherwise unreachable block!
Detect these cases and compute the number of saved instructions, instead of
bailing out and reporting no savings.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148941 91177308-0d34-0410-b5e6-96231b3b80d8
- Use MipsAnalyzeImmediate to expand immediates that do not fit in 16-bit.
- Change the types of variables so that they are sufficiently large to handle
64-bit pointers.
- Emit instructions to set register $28 in a function prologue after
instructions which store callee-saved registers have been emitted.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148917 91177308-0d34-0410-b5e6-96231b3b80d8
expand offsets that do not fit in the 16-bit immediate field of load and store
instructions. Also change the types of variables so that they are sufficiently
large to handle 64-bit pointers.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148916 91177308-0d34-0410-b5e6-96231b3b80d8
Add a test case to show fewer instructions are needed to load an immediate
with the new way of loading immediates.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148908 91177308-0d34-0410-b5e6-96231b3b80d8
Original log:
Introduce a new ConstantVector::getSplat constructor function to
simplify a really common case.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148906 91177308-0d34-0410-b5e6-96231b3b80d8
did something extremely surprising, and shadowed actually useful
implementations that had completely different behavior.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148898 91177308-0d34-0410-b5e6-96231b3b80d8
A REG_SEQUENCE instruction is lowered into a sequence of partial defs:
%vreg7:ssub_0<def,undef> = COPY %vreg20:ssub_0
%vreg7:ssub_1<def> = COPY %vreg2
%vreg7:ssub_2<def> = COPY %vreg2
%vreg7:ssub_3<def> = COPY %vreg2
The first def needs an <undef> flag to indicate it is the beginning of
the live range, while the other defs are read-modify-write. Previously,
we depended on LiveIntervalAnalysis to notice and fix the missing
<def,undef>, but that solution was never robust, it was causing problems
with ProcessImplicitDefs and the lowering of chained REG_SEQUENCE
instructions.
This fixes PR11841.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148879 91177308-0d34-0410-b5e6-96231b3b80d8
When not using subsections via symbols, the assembler can resolve
symbol differences (including pcrel references) to non-local
labels at assembly time, not just those in the same atom.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148865 91177308-0d34-0410-b5e6-96231b3b80d8
add a ConstantDataArray::getString method that corresponds to the (to be
removed) StringRef version of ConstantArray::get, but is dramatically more
efficient.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148804 91177308-0d34-0410-b5e6-96231b3b80d8
This change adds an new option --arm-enable-ehabi-descriptors that
enables emitting unwinding descriptors. This provides a mode with a
working backtrace() without the (currently broken) exception support.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148800 91177308-0d34-0410-b5e6-96231b3b80d8
and clean up some other misc stuff. Unlike ConstantArray, we will
prefer to emit .fill directives for "String" arrays that all have
the same value, since they are denser than emitting a .ascii
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148793 91177308-0d34-0410-b5e6-96231b3b80d8
same semantics as ConstantArray's but much more efficient because they
don't have to return std::string's. The ConstantArray methods will
eventually be removed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148792 91177308-0d34-0410-b5e6-96231b3b80d8
instead of its own hard coded thing, allowing it to handle
ConstantDataSequential and fixing some obscure bugs (e.g. it would
previously crash on a CAZ of vector type).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148788 91177308-0d34-0410-b5e6-96231b3b80d8
out into a new ConstantFoldLoadThroughGEPIndices (more useful) function
and rewrite it to be simpler, more efficient, and to handle the new
ConstantDataSequential type.
Enhance ConstantFoldLoadFromConstPtr to handle ConstantDataSequential.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148786 91177308-0d34-0410-b5e6-96231b3b80d8
violation -- MC cannot depend on CodeGen.
Specifically, the MCTargetDesc component of each target is actually
a subcomponent of the MC library. As such, it cannot depend on the
target-independent code generator, because MC itself cannot depend on
the target-independent code generator. This change moved a flag from the
ARM MCTargetDesc file ARMMCAsmInfo.cpp to the CodeGen layer in
ARMException.cpp, leaving behind an 'extern' to refer back to it. That
layering order isn't viable givin the constraints outlined above.
Commandline flags are designed to be static specifically to avoid these
types of bugs.
Fixing this is likely going to require some non-trivial refactoring.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148759 91177308-0d34-0410-b5e6-96231b3b80d8
