There were a number of issues to fix up here:
* The "device" argument of the llvm.memory.barrier intrinsic should be
used to distinguish the "Full System" domain from the "Inner Shareable"
domain. It has nothing to do with using DMB vs. DSB instructions.
* The compiler should never need to emit DSB instructions. Remove the
ARMISD::SYNCBARRIER node and also remove the instruction patterns for DSB.
* Merge the separate DMB/DSB instructions for options only used for the
disassembler with the default DMB/DSB instructions. Add the default
"full system" option ARM_MB::SY to the ARM_MB::MemBOpt enum.
* Add a separate ARMISD::MEMBARRIER_MCR node for subtargets that implement
a data memory barrier using the MCR instruction.
* Fix up encodings for these instructions (except MCR).
I also updated the tests and added a few new ones to check for DMB options
that were not currently being exercised.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@117756 91177308-0d34-0410-b5e6-96231b3b80d8
This code had previously used 2*N, where N is the mask length, to represent
undef. That is not safe because the shufflevector operands may have more
than N elements -- they don't have to match the result type.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@117721 91177308-0d34-0410-b5e6-96231b3b80d8
Allow splats even if they don't match either of the original shuffles,
possibly due to undef entries in the shuffles masks. Radar 8597790.
Also fix some 80-column violations.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@117719 91177308-0d34-0410-b5e6-96231b3b80d8
elements than the result vector type. So, when an instruction like:
%8 = shufflevector <2 x float> %4, <2 x float> %7, <4 x i32> <i32 1, i32 0, i32 3, i32 2>
is translated to a DAG, each operand is changed to a concat_vectors node that appends 2 undef elements. That is:
shuffle [a,b], [c,d] is changed to:
shuffle [a,b,u,u], [c,d,u,u]
That's probably the right thing for x86 but for NEON, we'd much rather have:
shuffle [a,b,c,d], undef
Teach the DAG combiner how to do that transformation for ARM. Radar 8597007.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@117482 91177308-0d34-0410-b5e6-96231b3b80d8
do not double-count the duplicate instructions by counting once from the
beginning and again from the end. Keep track of where the duplicates from
the beginning ended and don't go past that point when counting duplicates
at the end. Radar 8589805.
This change causes one of the MC/ARM/simple-fp-encoding tests to produce
different (better!) code without the vmovne instruction being tested.
I changed the test to produce vmovne and vmoveq instructions but moving
between register files in the opposite direction. That's not quite the same
but predicated versions of those instructions weren't being tested before,
so at least the test coverage is not any worse, just different.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@117333 91177308-0d34-0410-b5e6-96231b3b80d8
instructions separately from the count of non-predicated instructions. The
instruction count is used in places to determine how many instructions to
copy, predicate, etc. and things get confused if that count includes the
extra cost for microcoded ops.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@117332 91177308-0d34-0410-b5e6-96231b3b80d8
have been printed with the "S" modifier after the predicate. With ARM's
unified syntax, they are supposed to go in the other order. We fixed this
for Thumb when we switched to unified syntax but missed changing it for
ARM. Apparently we don't generate these instructions often because no one
noticed until now. Thanks to Bill Wendling for the testcase!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@116563 91177308-0d34-0410-b5e6-96231b3b80d8
"-mattr=+vfp3" is specified. However, this will not work for hardware that
only supports 16 registers. Add a new flag to support -"mattr=+vfp3,+d16".
Patch by Jan Voung!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@116310 91177308-0d34-0410-b5e6-96231b3b80d8
callee-saved registers at the end of the lists. Also prefer to avoid using
the low registers that are in register subclasses required by certain
instructions, so that those registers will more likely be available when needed.
This change makes a huge improvement in spilling in some cases. Thanks to
Jakob for helping me realize the problem.
Most of this patch is fixing the testsuite. There are quite a few places
where we're checking for specific registers. I changed those to wildcards
in places where that doesn't weaken the tests. The spill-q.ll and
thumb2-spill-q.ll tests stopped spilling with this change, so I added a bunch
of live values to force spills on those tests.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@116055 91177308-0d34-0410-b5e6-96231b3b80d8
scheduling change in svn 115121. The CriticalAntiDepBreaker had bad
liveness information. It was calculating the KillIndices for one scheduling
region in a basic block, rescheduling that region so the KillIndices were
no longer valid, and then using those wrong KillIndices to make decisions
for the next scheduling region. I've not been able to reduce a small
testcase for this. Radar 8502534.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@115400 91177308-0d34-0410-b5e6-96231b3b80d8
LDM/STM instructions can run one cycle faster on some ARM processors if the
memory address is 64-bit aligned. Radar 8489376.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@115047 91177308-0d34-0410-b5e6-96231b3b80d8
"external" even when doing lazy bitcode loading. This was broken because
a function that is not materialized fails the !isDeclaration() test.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@114666 91177308-0d34-0410-b5e6-96231b3b80d8
CombineTo to avoid putting the result on the worklist. I don't think it makes
much difference for now, but it might help someday as we add more DAG
combine optimizations.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@114595 91177308-0d34-0410-b5e6-96231b3b80d8
of those. Refactor to share code for handling BUILD_VECTOR(VMOVRRD).
I don't have a testcase that exercises this, but it seems like an obvious
good thing to do.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@114589 91177308-0d34-0410-b5e6-96231b3b80d8
truncates are free only in the case where the extended type is legal but the
load type is not. If both types are illegal, such as when they are too big,
the load may not be legalized into an extended load.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@114568 91177308-0d34-0410-b5e6-96231b3b80d8
ARM cross-compiler on x86, because the MMO size did not match the type size.
This fixes the MMO size and also the size of the stack object to match the
type size.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@114554 91177308-0d34-0410-b5e6-96231b3b80d8
load when the type of the load is not legal, even if truncates are not free.
The load is going to be legalized to an extending load anyway.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@114488 91177308-0d34-0410-b5e6-96231b3b80d8
value should be in GPRs when it's going to be used as a scalar, and we use
VMOVRRD to make that happen, but if the value is converted back to a vector
we need to fold to a simple bit_convert. Radar 8407927.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@114233 91177308-0d34-0410-b5e6-96231b3b80d8
encountered while building llvm-gcc for arm. This is probably the same issue
that the ppc buildbot hit. llvm::prior works on a MachineBasicBlock::iterator,
not a plain MachineInstr.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@113983 91177308-0d34-0410-b5e6-96231b3b80d8
an argument, so that we can distinguish instructions with the same register
classes but different numbers of registers (e.g., vld3 and vld4). Fix some
of the non-pseudo NEON ld/st instruction itineraries to reflect the number
of registers loaded or stored, not just the opcode name.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@113854 91177308-0d34-0410-b5e6-96231b3b80d8
pseudo-instruction approach. Change ARMExpandPseudoInsts to use a table
to record all the NEON load/store information.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@113812 91177308-0d34-0410-b5e6-96231b3b80d8
to use AddrMode4, there was a count of the registers stored in one of the
operands. I changed that to just count the operands but forgot to adjust for
the size of D registers. This was noticed by Evan as a performance problem
but it is a potential correctness bug as well, since it is possible that this
could merge a base update with a non-matching immediate.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@113576 91177308-0d34-0410-b5e6-96231b3b80d8
the VST pseudos. The VLD/VST scheduling still needs work (see pr6722), but
at least we shouldn't confuse the loads with the stores.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@113473 91177308-0d34-0410-b5e6-96231b3b80d8
operand from the pseudo instruction to the new instruction as an implicit use.
This will preserve any other flags (e.g., kill) on the operand.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@113456 91177308-0d34-0410-b5e6-96231b3b80d8
For VLD3/VLD4 with double-spaced registers, add the implicit use of the
super register for both the instruction loading the even registers and the
instruction loading the odd registers.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@113452 91177308-0d34-0410-b5e6-96231b3b80d8
solve the root problem, but it corrects the bug in the code I added to
support legalizing in the case where the non-extended type is also legal.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@112997 91177308-0d34-0410-b5e6-96231b3b80d8
instructions prior to regalloc. Since it's getting a little close to
the 2.8 branch deadline, I'll have to leave the rest of the instructions
handled by the NEONPreAllocPass for now, but I didn't want to leave half
of the VLD instructions converted and the other half not.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@112983 91177308-0d34-0410-b5e6-96231b3b80d8
vabd intrinsic and add and/or zext operations. In the case of vaba, this
also avoids the need for a DAG combine pattern to combine vabd with add.
Update tests. Auto-upgrade the old intrinsics.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@112941 91177308-0d34-0410-b5e6-96231b3b80d8
add, and subtract operations with zero-extended or sign-extended vectors.
Update tests. Add auto-upgrade support for the old intrinsics.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@112773 91177308-0d34-0410-b5e6-96231b3b80d8
operand is killed, add it to the expanded instruction as an implicit kill
operand instead of marking the individual subregs with kill flags. This
should work better in general and also handles the case for VST3 where one
of the subregs was not referenced in the expanded instruction and so was
not marked killed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@112494 91177308-0d34-0410-b5e6-96231b3b80d8
IR add/sub operations with one or both operands sign- or zero-extended.
Auto-upgrade the old intrinsics.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@112416 91177308-0d34-0410-b5e6-96231b3b80d8
the special values that for ARM would be used with IB or DA modes. Fall
through and consider materializing a new base address is it would be
profitable.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@112329 91177308-0d34-0410-b5e6-96231b3b80d8
all the other LDM/STM instructions. This fixes asm printer crashes when
compiling with -O0. I've changed one of the NEON tests (vst3.ll) to run
with -O0 to check this in the future.
Prior to this change VLDM/VSTM used addressing mode #5, but not really.
The offset field was used to hold a count of the number of registers being
loaded or stored, and the AM5 opcode field was expanded to specify the IA
or DB mode, instead of the standard ADD/SUB specifier. Much of the backend
was not aware of these special cases. The crashes occured when rewriting
a frameindex caused the AM5 offset field to be changed so that it did not
have a valid submode. I don't know exactly what changed to expose this now.
Maybe we've never done much with -O0 and NEON. Regardless, there's no longer
any reason to keep a count of the VLDM/VSTM registers, so we can use
addressing mode #4 and clean things up in a lot of places.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@112322 91177308-0d34-0410-b5e6-96231b3b80d8
Update all the tests using those intrinsics and add support for
auto-upgrading bitcode files with the old versions of the intrinsics.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@112271 91177308-0d34-0410-b5e6-96231b3b80d8
with the VST4 instructions. Until after register allocation, we want to
represent sets of adjacent registers by a single super-register. These
VST4 pseudo instructions have a single QQ or QQQQ source register operand.
They get expanded to the real VST4 instructions with 4 separate D register
operands. Once this conversion is complete, we'll be able to remove the
NEONPreAllocPass and avoid some fragile and hacky code elsewhere.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@112108 91177308-0d34-0410-b5e6-96231b3b80d8
it involves specific floating-point types, legalize should expand an
extending load to a non-extending load followed by a separate extend operation.
For example, we currently expand SEXTLOAD to EXTLOAD+SIGN_EXTEND_INREG (and
assert that EXTLOAD should always be supported). Now we can expand that to
LOAD+SIGN_EXTEND. This is needed to allow vector SIGN_EXTEND and ZERO_EXTEND
to be used for NEON.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@111586 91177308-0d34-0410-b5e6-96231b3b80d8
extending vector load should extend each element in the same way as the
corresponding scalar extending load.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@111577 91177308-0d34-0410-b5e6-96231b3b80d8
printing "lsl #0". This fixes the remaining parts of pr7792. Make
corresponding changes for encoding/decoding these instructions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@111251 91177308-0d34-0410-b5e6-96231b3b80d8
Bob Wilson