Call clobbers are now represented with register mask operands. The
regmask can easily represent the fact that xmm6 is call-preserved while
ymm6 isn't. This is automatically computed by TableGen from the
CalleeSavedRegs containing xmm6.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@150709 91177308-0d34-0410-b5e6-96231b3b80d8
The different calling conventions and call-preserved registers are
represented with regmask operands that are added dynamically.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@150708 91177308-0d34-0410-b5e6-96231b3b80d8
Call instructions no longer have a list of 43 call-clobbered registers.
Instead, they get a single register mask operand with a bit vector of
call-preserved registers.
This saves a lot of memory, 42 x 32 bytes = 1344 bytes per call
instruction, and it speeds up building call instructions because those
43 imp-def operands no longer need to be added to use-def lists. (And
removed and shifted and re-added for every explicit call operand).
Passes like LiveVariables, LiveIntervals, RAGreedy, PEI, and
BranchFolding are significantly faster because they can deal with call
clobbers in bulk.
Overall, clang -O2 is between 0% and 8% faster, uniformly distributed
depending on call density in the compiled code. Debug builds using
clang -O0 are 0% - 3% faster.
I have verified that this patch doesn't change the assembly generated
for the LLVM nightly test suite when building with -disable-copyprop
and -disable-branch-fold.
Branch folding behaves slightly differently in a few cases because call
instructions have different hash values now.
Copy propagation flushes its data structures when it crosses a register
mask operand. This causes it to leave a few dead copies behind, on the
order of 20 instruction across the entire nightly test suite, including
SPEC. Fixing this properly would require the pass to use different data
structures.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@150638 91177308-0d34-0410-b5e6-96231b3b80d8
The c'tor list is stored as a list of 'void ()*'s, so all of the functions are
bitcast to that. However, the dyn_cast doesn't automagically look through
bitcasts. Do that for it.
<rdar://problem/10813350>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@150572 91177308-0d34-0410-b5e6-96231b3b80d8
- Use unsigned literals when the desired result is unsigned. This mostly allows unsigned/signed mismatch warnings to be less noisy even if they aren't on by default.
- Remove misplaced llvm_unreachable.
- Add static to a declaration of a function on MSVC x86 only.
- Change some instances of calling a static function through a variable to simply calling that function while removing the unused variable.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@150364 91177308-0d34-0410-b5e6-96231b3b80d8
If the DEC node had more than one user, it was doing this lowering but
leaving the original DEC node around and so decrementing twice.
Fixes PR11964.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@150356 91177308-0d34-0410-b5e6-96231b3b80d8
This requires some gymnastics to make it available for C code. Remove the names
from the disassembler tables, making them relocation free.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@150303 91177308-0d34-0410-b5e6-96231b3b80d8
Now that the clang driver passes the CPU and feature information to
the backend when processing assembly files (150273), this isn't necessary.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@150274 91177308-0d34-0410-b5e6-96231b3b80d8
Creates a configurable regalloc pipeline.
Ensure specific llc options do what they say and nothing more: -reglloc=... has no effect other than selecting the allocator pass itself. This patch introduces a new umbrella flag, "-optimize-regalloc", to enable/disable the optimizing regalloc "superpass". This allows for example testing coalscing and scheduling under -O0 or vice-versa.
When a CodeGen pass requires the MachineFunction to have a particular property, we need to explicitly define that property so it can be directly queried rather than naming a specific Pass. For example, to check for SSA, use MRI->isSSA, not addRequired<PHIElimination>.
CodeGen transformation passes are never "required" as an analysis
ProcessImplicitDefs does not require LiveVariables.
We have a plan to massively simplify some of the early passes within the regalloc superpass.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@150226 91177308-0d34-0410-b5e6-96231b3b80d8
Moving toward a uniform style of pass definition to allow easier target configuration.
Globally declare Pass ID.
Globally declare pass initializer.
Use INITIALIZE_PASS consistently.
Add a call to the initializer from CodeGen.cpp.
Remove redundant "createPass" functions and "getPassName" methods.
While cleaning up declarations, cleaned up comments (sorry for large diff).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@150100 91177308-0d34-0410-b5e6-96231b3b80d8
Creating the isPredicated TSFlag enables the code
to use the property defined in the instruction format
instead of using a large switch statement.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@150078 91177308-0d34-0410-b5e6-96231b3b80d8
load / store) if the ADD / SUB has a live definition of CPSR.
Bug reported by David Meyer. Alas, no test case.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149970 91177308-0d34-0410-b5e6-96231b3b80d8
This CL delays reading of function bodies from initial parse until
materialization, allowing overlap of compilation with bitcode download.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149918 91177308-0d34-0410-b5e6-96231b3b80d8
convert at least one client over to use them. Subsequent patches both to
LLVM and Clang will try to convert more people over to a common set of
predicates.
This round of predicates is focused on OS-categorization predicates.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149815 91177308-0d34-0410-b5e6-96231b3b80d8
but with a critical fix to the SelectionDAG code that optimizes copies
from strings into immediate stores: the previous code was stopping reading
string data at the first nul. Address this by adding a new argument to
llvm::getConstantStringInfo, preserving the behavior before the patch.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149800 91177308-0d34-0410-b5e6-96231b3b80d8
Passes prior to instructon selection are now split into separate configurable stages.
Header dependencies are simplified.
The bulk of this diff is simply removal of the silly DisableVerify flags.
Sorry for the target header churn. Attempting to stabilize them.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149754 91177308-0d34-0410-b5e6-96231b3b80d8
Allows command line overrides to be centralized in LLVMTargetMachine.cpp.
LLVMTargetMachine can intercept common passes and give precedence to command line overrides.
Allows adding "internal" target configuration options without touching TargetOptions.
Encapsulates the PassManager.
Provides a good point to initialize all CodeGen passes so that Pass ID's can be used in APIs.
Allows modifying the target configuration hooks without rebuilding the world.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149672 91177308-0d34-0410-b5e6-96231b3b80d8
needed to emit a 64-bit gp-relative relocation entry. Make changes necessary
for emitting jump tables which have entries with directive .gpdword. This patch
does not implement the parts needed for direct object emission or JIT.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149668 91177308-0d34-0410-b5e6-96231b3b80d8
It doesn't seem worthwhile to give meaning to a NULL register mask
pointer. It complicates all the code using register mask operands.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149646 91177308-0d34-0410-b5e6-96231b3b80d8
NEON loads and stores accept single and double spaced pairs, triples,
and quads of D registers. This patch adds new register classes to
accurately model those constraints:
Dn, Dn+1 Dn, Dn+2
----------------------
DPair DPairSpc
DTriple DTripleSpc
DQuad DQuadSpc
Also extend the existing QQ and QQQQ register classes to contains all Q
pairs and quads instead of just the aligned ones.
These new register classes will make it possible to accurately model
constraints on NEON loads and stores, and we can get rid of all the NEON
pseudo-instructions. The late scheduler will be able to accurately
model instruction dependencies from the explicit operands.
This more than doubles the number of ARM registers, but the backend
passes are quite good at handling this. The llc -O0 compile time only
regresses by 1.5%. Future work on register mask operands will recover
this regression.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149640 91177308-0d34-0410-b5e6-96231b3b80d8
Adds an instruction itinerary to all x86 instructions, giving each a default latency of 1, using the InstrItinClass IIC_DEFAULT.
Sets specific latencies for Atom for the instructions in files X86InstrCMovSetCC.td, X86InstrArithmetic.td, X86InstrControl.td, and X86InstrShiftRotate.td. The Atom latencies for the remainder of the x86 instructions will be set in subsequent patches.
Adds a test to verify that the scheduler is working.
Also changes the scheduling preference to "Hybrid" for i386 Atom, while leaving x86_64 as ILP.
Patch by Preston Gurd!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149558 91177308-0d34-0410-b5e6-96231b3b80d8
This new scheduler plugs into the existing selection DAG scheduling framework. It is a top-down critical path scheduler that tracks register pressure and uses a DFA for pipeline modeling.
Patch by Sergei Larin!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149547 91177308-0d34-0410-b5e6-96231b3b80d8
The purpose of refactoring is to hide operand roles from SwitchInst user (programmer). If you want to play with operands directly, probably you will need lower level methods than SwitchInst ones (TerminatorInst or may be User). After this patch we can reorganize SwitchInst operands and successors as we want.
What was done:
1. Changed semantics of index inside the getCaseValue method:
getCaseValue(0) means "get first case", not a condition. Use getCondition() if you want to resolve the condition. I propose don't mix SwitchInst case indexing with low level indexing (TI successors indexing, User's operands indexing), since it may be dangerous.
2. By the same reason findCaseValue(ConstantInt*) returns actual number of case value. 0 means first case, not default. If there is no case with given value, ErrorIndex will returned.
3. Added getCaseSuccessor method. I propose to avoid usage of TerminatorInst::getSuccessor if you want to resolve case successor BB. Use getCaseSuccessor instead, since internal SwitchInst organization of operands/successors is hidden and may be changed in any moment.
4. Added resolveSuccessorIndex and resolveCaseIndex. The main purpose of these methods is to see how case successors are really mapped in TerminatorInst.
4.1 "resolveSuccessorIndex" was created if you need to level down from SwitchInst to TerminatorInst. It returns TerminatorInst's successor index for given case successor.
4.2 "resolveCaseIndex" converts low level successors index to case index that curresponds to the given successor.
Note: There are also related compatability fix patches for dragonegg, klee, llvm-gcc-4.0, llvm-gcc-4.2, safecode, clang.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149481 91177308-0d34-0410-b5e6-96231b3b80d8
Adjust an example MachObjectWriter diagnostic to use the information
to issue a better message.
Before:
LLVM ERROR: unknown ARM fixup kind!
After:
x.s:6:5: error: unsupported relocation on symbol
beq bar
^
rdar://9800182
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149093 91177308-0d34-0410-b5e6-96231b3b80d8
The Win64 calling convention has xmm6-15 as callee-saved while still
clobbering all ymm registers.
Add a YMM_HI_6_15 pseudo-register that aliases the clobbered part of the
ymm registers, and mark that as call-clobbered. This allows live xmm
registers across calls.
This hack wouldn't be necessary with RegisterMask operands representing
the call clobbers, but they are not quite operational yet.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149088 91177308-0d34-0410-b5e6-96231b3b80d8
This enables the linker to match concrete relocation types (absolute or relative) with whatever library or C++ support code is being linked against.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149057 91177308-0d34-0410-b5e6-96231b3b80d8
. "fptosi" and "fptoui" IR instructions are defined with round-to-zero rounding mode.
. Currently for AVX mode for <4xdouble> and <8xdouble> the "VCVTPD2DQ.128" and "VCVTPD2DQ.256" instructions are selected (for .fp_to_sint. DAG node operation ) by AVX codegen. However they use round-to-nearest-even rounding mode.
. Consequently, the conversion produces incorrect numbers.
The fix is to replace selection of VCVTPD2DQ instructions with VCVTTPD2DQ instructions. The latter use truncate (i.e. round-to-zero) rounding mode.
As .fp_to_sint. DAG node operation is used only for lowering of "fptosi" and "fptoui" IR instructions, the fix in X86InstrSSE.td definition file doesn.t have an impact on other LLVM flows.
The patch includes changes in the .td file, LIT test for the changes and a fix in a legacy LIT test (which produced asm code conflicting with LLVN IR spec).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@149056 91177308-0d34-0410-b5e6-96231b3b80d8
"Although a Thumb2 instruction, the IT mnemonic shall be permitted in
ARM mode, and the condition verified to match the condition code(s)
on the following instruction(s)."
PR11853
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148969 91177308-0d34-0410-b5e6-96231b3b80d8
- Use MipsAnalyzeImmediate to expand immediates that do not fit in 16-bit.
- Change the types of variables so that they are sufficiently large to handle
64-bit pointers.
- Emit instructions to set register $28 in a function prologue after
instructions which store callee-saved registers have been emitted.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148917 91177308-0d34-0410-b5e6-96231b3b80d8
expand offsets that do not fit in the 16-bit immediate field of load and store
instructions. Also change the types of variables so that they are sufficiently
large to handle 64-bit pointers.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148916 91177308-0d34-0410-b5e6-96231b3b80d8
Add a test case to show fewer instructions are needed to load an immediate
with the new way of loading immediates.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148908 91177308-0d34-0410-b5e6-96231b3b80d8
violation -- MC cannot depend on CodeGen.
Specifically, the MCTargetDesc component of each target is actually
a subcomponent of the MC library. As such, it cannot depend on the
target-independent code generator, because MC itself cannot depend on
the target-independent code generator. This change moved a flag from the
ARM MCTargetDesc file ARMMCAsmInfo.cpp to the CodeGen layer in
ARMException.cpp, leaving behind an 'extern' to refer back to it. That
layering order isn't viable givin the constraints outlined above.
Commandline flags are designed to be static specifically to avoid these
types of bugs.
Fixing this is likely going to require some non-trivial refactoring.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148759 91177308-0d34-0410-b5e6-96231b3b80d8
Let the generic token alias definitions handle the data subtype
suffices. We don't need explicit versions for each.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148718 91177308-0d34-0410-b5e6-96231b3b80d8
This change adds an new value to the --arm-enable-ehabi option that
disables emitting unwinding descriptors. This mode gives a working
backtrace() without the (currently broken) exception support.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148686 91177308-0d34-0410-b5e6-96231b3b80d8
We have patterns for vector sext and zext operations but were missing
anyext. Without those patterns, codegen will fail when the selection DAG
has any_extend nodes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148568 91177308-0d34-0410-b5e6-96231b3b80d8
For bit patterns that aren't representable using the 8-bit floating point
representation for vmov.f32, but are representable via vmov.i32, treat
the .f32 syntax as an alias. Most importantly, this covers the case
'vmov.f32 Vd, #0.0'.
rdar://10616677
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148556 91177308-0d34-0410-b5e6-96231b3b80d8
to instruction right after the last instruction in the bundle.
- Add a finalizeBundle() variant that doesn't specify LastMI. Instead, the code
will find the last instruction in the bundle by following the 'InsideBundle'
marker. This is useful in case bundles are formed early (i.e. during MI
scheduling) but finalized later (i.e. after register allocator has finished
rewriting virtual registers with physical registers).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148444 91177308-0d34-0410-b5e6-96231b3b80d8
It adds register mask operands to x86 call instructions. Once all the
backend passes support register mask operands, this will be permanently
enabled.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148438 91177308-0d34-0410-b5e6-96231b3b80d8
This is similar to implicit register operands. MC doesn't understand
register liveness and call clobbers.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148437 91177308-0d34-0410-b5e6-96231b3b80d8
If the fixup is out of range for the Thumb1 instruction, relax it
to the Thumb2 encoding instead.
rdar://10711829
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148424 91177308-0d34-0410-b5e6-96231b3b80d8
Load/store instructions w/ a fixup to be relative a function marked as thumb
don't use the low bit to specify thumb vs. non-thumb like interworking
branches do, so don't set it when dealing with those fixups.
rdar://10348687.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148366 91177308-0d34-0410-b5e6-96231b3b80d8
When set, this bit indicates that a register is completely defined by
the value of its sub-registers.
Use the CoveredBySubRegs property to infer which super-registers are
call-preserved given a list of callee-saved registers. For example, the
ARM registers D8-D15 are callee-saved. This now automatically implies
that Q4-Q7 are call-preserved.
Conversely, Win64 callees save XMM6-XMM15, but the corresponding
YMM6-YMM15 registers are not call-preserved because they are not fully
defined by their sub-registers.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148363 91177308-0d34-0410-b5e6-96231b3b80d8
In CanXFormVExtractWithShuffleIntoLoad we assumed that EXTRACT_VECTOR_ELT can be later handled by the DAGCombiner.
However, in some cases on AVX, the EXTRACT_VECTOR_ELT is legalized to EXTRACT_SUBVECTOR + EXTRACT_VECTOR_ELT, which
currently is not handled by the DAGCombiner. In this patch I added a check that we only extract from the XMM part.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148298 91177308-0d34-0410-b5e6-96231b3b80d8
(This time I believe I've checked all the -Wreturn-type warnings from GCC & added the couple of llvm_unreachables necessary to silence them. If I've missed any, I'll happily fix them as soon as I know about them)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148262 91177308-0d34-0410-b5e6-96231b3b80d8
We know that the blend instructions only use the MSB, so if the mask is
sign-extended then we can convert it into a SHL instruction. This is a
common pattern because the type-legalizer sign-extends the i1 type which
is used by the LLVM-IR for the condition.
Added a new optimization in SimplifyDemandedBits for SIGN_EXTEND_INREG -> SHL.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148225 91177308-0d34-0410-b5e6-96231b3b80d8
live across BBs before register allocation. This miscompiled 197.parser
when a cmp + b are optimized to a cbnz instruction even though the CPSR def
is live-in a successor.
cbnz r6, LBB89_12
...
LBB89_12:
ble LBB89_1
The fix consists of two parts. 1) Teach LiveVariables that some unallocatable
registers might be liveouts so don't mark their last use as kill if they are.
2) ARM constantpool island pass shouldn't form cbz / cbnz if the conditional
branch does not kill CPSR.
rdar://10676853
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148168 91177308-0d34-0410-b5e6-96231b3b80d8
The QQ and QQQQ registers are not 'real', they are pseudo-registers used
to model some vld and vst instructions.
This makes the call clobber lists longer, but I intend to get rid of
those soon.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148151 91177308-0d34-0410-b5e6-96231b3b80d8
The registers are placed into the saved registers list in the reverse order,
which is why the original loop was written to loop backwards.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148064 91177308-0d34-0410-b5e6-96231b3b80d8
lc: X86ISelLowering.cpp:6480: llvm::SDValue llvm::X86TargetLowering::LowerVECTOR_SHUFFLE(llvm::SDValue, llvm::SelectionDAG&) const: Assertion `V1.getOpcode() != ISD::UNDEF&& "Op 1 of shuffle should not be undef"' failed.
Added a test.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148044 91177308-0d34-0410-b5e6-96231b3b80d8
Uses the pvArbitrary slot of the TIB, which is reserved for applications. We
only support frames with a static size.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148040 91177308-0d34-0410-b5e6-96231b3b80d8
Restore the (obviously wrong) behavior from before r147938 without relying on
undefined behavior. Add a fat FIXME note.
This should fix nightly tester failures.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148030 91177308-0d34-0410-b5e6-96231b3b80d8
In att style asm syntax memory operand size is derived from suffix attached with mnemonic. In intel style asm syntax it is part of memory operand hence predicate method check is required to select appropriate instruction.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148006 91177308-0d34-0410-b5e6-96231b3b80d8
same pattern. We already had this pattern is a few places, but others
tried to make a rough approximation of an actual DAG structure. As not
everywhere went to this trouble, nothing could rely on this being done.
In fact, I've checked all references to these node Ids, and the ones
that are using the topo-sort properties are actually satisfied with
a strict-weak-ordering. The requirement appears to be that Use >= Def.
I've added a big blurb of comments to this bit of the transform to
clarify why the order is so important for the next reader of the code.
I'm starting with this change as it is very small, and trivially
reverted if something breaks or the >= above really does need to be >.
If that proves the case, we can hide the problem by reverting this
patch, but the problem exists elsewhere as well, and so a more
comprehensive solution will be needed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@148001 91177308-0d34-0410-b5e6-96231b3b80d8
This uses TLS slot 90, which actually belongs to JavaScriptCore. We only support
frames with static size
Patch by Brian Anderson.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147960 91177308-0d34-0410-b5e6-96231b3b80d8
hoped this would revive one of the llvm-gcc selfhost build bots, but it
didn't so it doesn't appear that my transform is the culprit.
If anyone else is seeing failures, please let me know!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147957 91177308-0d34-0410-b5e6-96231b3b80d8
strange build bot failures that look like a miscompile into an infloop.
I'll investigate this tomorrow, but I'd both like to know whether my
patch is the culprit, and get the bots back to green.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147945 91177308-0d34-0410-b5e6-96231b3b80d8
factor the differences that were hiding in one of them into its other
caller, the SRL handling code. No change in behavior.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147940 91177308-0d34-0410-b5e6-96231b3b80d8
mask+shift pairs at the beginning of the ISD::AND case block, and then
hoist the final pattern into a helper function, simplifying and
reflowing it appropriately. This should have no observable behavior
change, but several simplifications fell out of this such as directly
computing the new mask constant, etc.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147939 91177308-0d34-0410-b5e6-96231b3b80d8
extracts and scaled addressing modes into its own helper function. No
functionality changed here, just hoisting and layout fixes falling out
of that hoisting.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147937 91177308-0d34-0410-b5e6-96231b3b80d8
detect a pattern which can be implemented with a small 'shl' embedded in
the addressing mode scale. This happens in real code as follows:
unsigned x = my_accelerator_table[input >> 11];
Here we have some lookup table that we look into using the high bits of
'input'. Each entity in the table is 4-bytes, which means this
implicitly gets turned into (once lowered out of a GEP):
*(unsigned*)((char*)my_accelerator_table + ((input >> 11) << 2));
The shift right followed by a shift left is canonicalized to a smaller
shift right and masking off the low bits. That hides the shift right
which x86 has an addressing mode designed to support. We now detect
masks of this form, and produce the longer shift right followed by the
proper addressing mode. In addition to saving a (rather large)
instruction, this also reduces stalls in Intel chips on benchmarks I've
measured.
In order for all of this to work, one part of the DAG needs to be
canonicalized *still further* than it currently is. This involves
removing pointless 'trunc' nodes between a zextload and a zext. Without
that, we end up generating spurious masks and hiding the pattern.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147936 91177308-0d34-0410-b5e6-96231b3b80d8
Allow LDRD to be formed from pairs with different LDR encodings. This was the original intention of the pass. Somewhere along the way, the LDR opcodes were refined which broke the optimization. We really don't care what the original opcodes are as long as they both map to the same LDRD and the immediate still fits.
Fixes rdar://10435045 ARMLoadStoreOptimization cannot handle mixed LDRi8/LDRi12
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147922 91177308-0d34-0410-b5e6-96231b3b80d8
Add a test that checks the stack alignment of a simple function for
Darwin, Linux and NetBSD for 32bit and 64bit mode.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147888 91177308-0d34-0410-b5e6-96231b3b80d8
This function runs after all constant islands have been placed, and may
shrink some instructions to their 2-byte forms. This can actually cause
some constant pool entries to move out of range because of growing
alignment padding.
Treat instructions that may be shrunk the same as inline asm - they
erode the known alignment bits.
Also reinstate an old assertion in verify(). It is correct now that
basic block offsets include alignments.
Add a single large test case that will hopefully exercise many parts of
the constant island pass.
<rdar://problem/10670199>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147885 91177308-0d34-0410-b5e6-96231b3b80d8
As the comment around 7746 says, it's better to use the x87 extended precision
here than SSE. And the generic code doesn't know how to do that. It also regains
the speed lost for the uint64_to_float.c testcase.
<rdar://problem/10669858>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147869 91177308-0d34-0410-b5e6-96231b3b80d8
of several newly un-defaulted switches. This also helps optimizers
(including LLVM's) recognize that every case is covered, and we should
assume as much.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147861 91177308-0d34-0410-b5e6-96231b3b80d8
On Thumb, the displacement computation hardware uses the address of the
current instruction rouned down to a multiple of 4. Include this
rounding in the UserOffset we compute for each instruction.
When inline asm is present, the instruction alignment may not be known.
Constrain the maximum displacement instead in that case.
This makes it possible for CreateNewWater() and OffsetIsInRange() to
agree about the valid displacements. When they disagree, infinite
looping happens.
As always, test cases for this stuff are insane.
<rdar://problem/10660175>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147825 91177308-0d34-0410-b5e6-96231b3b80d8
The pass is prone to looping, and it is better to crash than loop
forever, even in a -Asserts build.
<rdar://problem/10660175>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147806 91177308-0d34-0410-b5e6-96231b3b80d8
AsmParser holds info specific to target parser.
AsmParserVariant holds info specific to asm variants supported by the target.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147787 91177308-0d34-0410-b5e6-96231b3b80d8
this substraction will result in small negative numbers at worst which
become very large positive numbers on assignment and are thus caught by
the <=4 check on the next line. The >0 check clearly intended to catch
these as negative numbers.
Spotted by inspection, and impossible to trigger given the shift widths
that can be used.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147773 91177308-0d34-0410-b5e6-96231b3b80d8
This enables basic local CSE, giving us 20% smaller code for
consumer-typeset in -O0 builds.
<rdar://problem/10658692>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147720 91177308-0d34-0410-b5e6-96231b3b80d8
file error checking. Use that to error on an unfinished cfi_startproc.
The error is not nice, but is already better than a segmentation fault.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147717 91177308-0d34-0410-b5e6-96231b3b80d8
exposed with an upcoming change will would delete the copy to return register
because there is no use! It's amazing anything works.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147715 91177308-0d34-0410-b5e6-96231b3b80d8
This eliminates a lot of constant pool entries for -O0 builds of code
with many global variable accesses.
This speeds up -O0 codegen of consumer-typeset by 2x because the
constant island pass no longer has to look at thousands of constant pool
entries.
<rdar://problem/10629774>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147712 91177308-0d34-0410-b5e6-96231b3b80d8
Testing: passed 'make check' including LIT tests for all sequences being handled (both SSE and AVX)
Reviewers: Evan Cheng, David Blaikie, Bruno Lopes, Elena Demikhovsky, Chad Rosier, Anton Korobeynikov
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147601 91177308-0d34-0410-b5e6-96231b3b80d8
This small bit of ASM code is sufficient to do what the old algorithm did:
movq %rax, %xmm0
punpckldq (c0), %xmm0 // c0: (uint4){ 0x43300000U, 0x45300000U, 0U, 0U }
subpd (c1), %xmm0 // c1: (double2){ 0x1.0p52, 0x1.0p52 * 0x1.0p32 }
#ifdef __SSE3__
haddpd %xmm0, %xmm0
#else
pshufd $0x4e, %xmm0, %xmm1
addpd %xmm1, %xmm0
#endif
It's arguably faster. One caveat, the 'haddpd' instruction isn't very fast on
all processors.
<rdar://problem/7719814>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147593 91177308-0d34-0410-b5e6-96231b3b80d8
Now that canRealignStack() understands frozen reserved registers, it is
safe to use it for aligned spill instructions.
It will only return true if the registers reserved at the beginning of
register allocation allow for dynamic stack realignment.
<rdar://problem/10625436>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147579 91177308-0d34-0410-b5e6-96231b3b80d8
Once register allocation has started the reserved registers are frozen.
Fix the ARM canRealignStack() hook to respect the frozen register state.
Now the hook returns false if register allocation was started with frame
pointer elimination enabled.
It also returns false if register allocation started without a reserved
base pointer, and stack realignment would require a base pointer. This
bug was breaking oggenc on armv6.
No test case, an upcoming patch will use this functionality to realign
the stack for spill slots when possible.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147578 91177308-0d34-0410-b5e6-96231b3b80d8
versions derive from them.
- JALR64 is not needed since N64 does not emit jal.
- Add template parameter to BranchLink that sets the rt field.
- Fix the set of temporary registers for O32 and N64.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147518 91177308-0d34-0410-b5e6-96231b3b80d8
(x > y) ? x : y
=>
(x >= y) ? x : y
So for something like
(x - y) > 0 : (x - y) ? 0
It will be
(x - y) >= 0 : (x - y) ? 0
This makes is possible to test sign-bit and eliminate a comparison against
zero. e.g.
subl %esi, %edi
testl %edi, %edi
movl $0, %eax
cmovgl %edi, %eax
=>
xorl %eax, %eax
subl %esi, $edi
cmovsl %eax, %edi
rdar://10633221
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147512 91177308-0d34-0410-b5e6-96231b3b80d8
This patch caused a miscompilation of oggenc because a frame pointer was
suddenly needed halfway through register allocation.
<rdar://problem/10625436>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147487 91177308-0d34-0410-b5e6-96231b3b80d8
If anybody has strong feelings about 'default: assert(0 && "blah")' vs
'default: llvm_unreachable("blah")', feel free to regularize the instances of
each in this file.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147459 91177308-0d34-0410-b5e6-96231b3b80d8
Implement encoder methods getJumpTargetOpValue and getBranchTargetOpValue
for jmptarget and brtarget Mips tablegen operand types in the code emitter
for old-style JIT. Rename the pc relative relocation for branches - new
name is Mips::reloc_mips_pc16.
Patch by Sasa Stankovic
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147382 91177308-0d34-0410-b5e6-96231b3b80d8
1. The ST*UX instructions that store and update the stack pointer did not set define/kill on R1. This became a problem when I activated post-RA scheduling (and had incorrectly adjusted the Frames-large test).
2. eliminateFrameIndex did not kill its scavenged temporary register, and this could cause the scavenger to exhaust all available registers (and its emergency spill slot) when there were a lot of CR values to spill. The 2010-02-12-saveCR test has been adjusted to check for this.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147359 91177308-0d34-0410-b5e6-96231b3b80d8
Matching MOVLP mask for AVX (265-bit vectors) was wrong.
The failure was detected by conformance tests.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147308 91177308-0d34-0410-b5e6-96231b3b80d8
LZCNT instructions are available. Force promotion to i32 to get
a smaller encoding since the fix-ups necessary are just as complex for
either promoted type
We can't do standard promotion for CTLZ when lowering through BSR
because it results in poor code surrounding the 'xor' at the end of this
instruction. Essentially, if we promote the entire CTLZ node to i32, we
end up doing the xor on a 32-bit CTLZ implementation, and then
subtracting appropriately to get back to an i8 value. Instead, our
custom logic just uses the knowledge of the incoming size to compute
a perfect xor. I'd love to know of a way to fix this, but so far I'm
drawing a blank. I suspect the legalizer could be more clever and/or it
could collude with the DAG combiner, but how... ;]
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147251 91177308-0d34-0410-b5e6-96231b3b80d8
'bsf' instructions here.
This one is actually debatable to my eyes. It's not clear that any chip
implementing 'tzcnt' would have a slow 'bsf' for any reason, and unless
EFLAGS or a zero input matters, 'tzcnt' is just a longer encoding.
Still, this restores the old behavior with 'tzcnt' enabled for now.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147246 91177308-0d34-0410-b5e6-96231b3b80d8
X86ISelLowering C++ code. Because this is lowered via an xor wrapped
around a bsr, we want the dagcombine which runs after isel lowering to
have a chance to clean things up. In particular, it is very common to
see code which looks like:
(sizeof(x)*8 - 1) ^ __builtin_clz(x)
Which is trying to compute the most significant bit of 'x'. That's
actually the value computed directly by the 'bsr' instruction, but if we
match it too late, we'll get completely redundant xor instructions.
The more naive code for the above (subtracting rather than using an xor)
still isn't handled correctly due to the dagcombine getting confused.
Also, while here fix an issue spotted by inspection: we should have been
expanding the zero-undef variants to the normal variants when there is
an 'lzcnt' instruction. Do so, and test for this. We don't want to
generate unnecessary 'bsr' instructions.
These two changes fix some regressions in encoding and decoding
benchmarks. However, there is still a *lot* to be improve on in this
type of code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147244 91177308-0d34-0410-b5e6-96231b3b80d8
ARM targets with NEON units have access to aligned vector loads and
stores that are potentially faster than unaligned operations.
Add support for spilling the callee-saved NEON registers to an aligned
stack area using 16-byte aligned NEON loads and store.
This feature is off by default, controlled by an -align-neon-spills
command line option.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147211 91177308-0d34-0410-b5e6-96231b3b80d8
My change r146949 added register clobbers to the eh_sjlj_dispatchsetup pseudo
instruction, but on Thumb1 some of those registers cannot be used. This
caused massive failures on the testsuite when compiling for Thumb1. While
fixing that, I noticed that the eh_sjlj_setjmp instruction has a "nofp"
variant, and I realized that dispatchsetup needs the same thing, so I have
added that as well.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147204 91177308-0d34-0410-b5e6-96231b3b80d8
The value from the operands isn't right yet, but we weren't encoding it at
all previously. The parser needs to twiddle the values when building the
instruction.
Partial for: rdar://10558523
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147170 91177308-0d34-0410-b5e6-96231b3b80d8
Rather than require the symbol to be explicitly an argument of the directive,
allow it to look ahead and grab the symbol from the next non-whitespace
line.
rdar://10611140
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147100 91177308-0d34-0410-b5e6-96231b3b80d8
DSHD (Double Swap Halfwords within Doublewords). Add a pattern which replaces
64-bit bswap with a DSBH and DSHD pair.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147017 91177308-0d34-0410-b5e6-96231b3b80d8
instruction supported by mips32r2, and add a pattern which replaces bswap with
a ROTR and WSBH pair.
WSBW is removed since it is not an instruction the current architectures
support.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@147015 91177308-0d34-0410-b5e6-96231b3b80d8
Use the spill slot alignment as well as the local variable alignment to
determine when the stack needs to be realigned. This works now that the
ARM target can always realign the stack by using a base pointer.
Still respect the ARMBaseRegisterInfo::canRealignStack() function
vetoing a realigned stack. Don't use aligned spill code in that case.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146997 91177308-0d34-0410-b5e6-96231b3b80d8
use the zero-undefined variants of CTTZ and CTLZ. These are just simple
patterns for now, there is more to be done to make real world code using
these constructs be optimized and codegen'ed properly on X86.
The existing tests are spiffed up to check that we no longer generate
unnecessary cmov instructions, and that we generate the very important
'xor' to transform bsr which counts the index of the most significant
one bit to the number of leading (most significant) zero bits. Also they
now check that when the variant with defined zero result is used, the
cmov is still produced.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146974 91177308-0d34-0410-b5e6-96231b3b80d8
We used to rely on the *eh_sjlj_setjmp instructions to mark that a function
with setjmp/longjmp exception handling clobbers all the registers. But with
the recent reorganization of ARM EH, those eh_sjlj_setjmp instructions are
expanded away earlier, before PEI can see them to determine what registers to
save and restore. Mark the dispatchsetup instruction in the same way, since
that instruction cannot be expanded early. This also more accurately reflects
when the registers are clobbered.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146949 91177308-0d34-0410-b5e6-96231b3b80d8
"mov r1, r2, lsl #0" should assemble as "mov r1, r2" even though it's
not strictly legal UAL syntax. It's a common extension and the friendly
thing to do.
rdar://10604663
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146937 91177308-0d34-0410-b5e6-96231b3b80d8
This change reduces the number of instructions generated.
For example,
(load (add (sub $n0, $n1), (MipsLo got(s))))
results in the following sequence of instructions:
1. sub $n2, $n0, $n1
2. lw got(s)($n2)
Previously, three instructions were needed.
1. sub $n2, $n0, $n1
2. addiu $n3, $n2, got(s)
3. lw 0($n3)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146888 91177308-0d34-0410-b5e6-96231b3b80d8
Use information computed while inferring new register classes to emit
accurate, table-driven implementations of getMatchingSuperRegClass().
Delete the old manual, error-prone implementations in the targets.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146873 91177308-0d34-0410-b5e6-96231b3b80d8
The bad sorting caused a misaligned basic block when building 176.vpr in
ARM mode.
<rdar://problem/10594653>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146767 91177308-0d34-0410-b5e6-96231b3b80d8
This adjustment is already included in the block offsets computed by
BasicBlockInfo, and adjusting again here can cause the pass to loop.
When CreateNewWater splits a basic block, OffsetIsInRange would reject
the new CPE on the next pass because of the too conservative alignment
adjustment. This caused the block to be split again, and so on.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146751 91177308-0d34-0410-b5e6-96231b3b80d8
The command line option should be removed, but not until the feature has
gotten a lot of testing. The ARMConstantIslandPass tends to have subtle
bugs that only show up after a while.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@146739 91177308-0d34-0410-b5e6-96231b3b80d8
