Commit Graph
100 Commits
Author SHA1 Message Date
Jakob Stoklund Olesen cdcdfd2cab Revert r166046 "Switch back to the old coalescer for now to fix the 32 bit bit"
A fix for PR14098, including the test case is in the next commit.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@166067 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-16 22:51:55 +00:00
Jakob Stoklund Olesen feab72c20a Remove unused BitVectors from getAllocatableSet().
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165999 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-16 00:05:06 +00:00
Jakob Stoklund Olesen 14d1dd95c7 Remove RegisterClassInfo::isReserved() and isAllocatable().
Clients can use the equivalent functions in MRI.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165990 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-15 22:41:03 +00:00
Jakob Stoklund Olesen 790047620a Remove LIS::isAllocatable() and isReserved() helpers.
All callers can simply use the corresponding MRI functions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165985 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-15 22:14:34 +00:00
Jakob Stoklund Olesen fb9ebbf236 Switch most getReservedRegs() clients to the MRI equivalent.
Using the cached bit vector in MRI avoids comstantly allocating and
recomputing the reserved register bit vector.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165983 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-15 21:57:41 +00:00
Jakob Stoklund Olesen e4f273908b Freeze the reserved registers as soon as isel is complete.
Also provide an MRI::getReservedRegs() function to access the frozen
register set, and isReserved() and isAllocatable() methods to test
individual registers.

The various implementations of TRI::getReservedRegs() are quite
complicated, and many passes need to look at the reserved register set.
This patch makes it possible for these passes to use the cached copy in
MRI, avoiding a lot of malloc traffic and repeated calculations.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165982 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-15 21:33:06 +00:00
Jakob Stoklund Olesen d86296a4ae Drop <def,dead> flags when merging into an unused lane.
The new coalescer can merge a dead def into an unused lane of an
otherwise live vector register.

Clear the <dead> flag when that happens since the flag refers to the
full virtual register which is still live after the partial dead def.

This fixes PR14079.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165877 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-13 17:26:47 +00:00
Jakob Stoklund Olesen af89690760 Allow for loops in LiveIntervals::pruneValue().
It is possible that the live range of the value being pruned loops back
into the kill MBB where the search started. When that happens, make sure
that the beginning of KillMBB is also pruned.

Instead of starting a DFS at KillMBB and skipping the root of the
search, start a DFS at each KillMBB successor, and allow the search to
loop back to KillMBB.

This fixes PR14078.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165872 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-13 16:15:31 +00:00
Jakob Stoklund Olesen 2bbb07d13c Fix buildbots: -misched=shuffle is only available in +Asserts builds.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165846 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-12 23:01:33 +00:00
Jakob Stoklund Olesen ad5e969ba9 Use a transposed algorithm for handleMove().
Completely update one interval at a time instead of collecting live
range fragments to be updated. This avoids building data structures,
except for a single SmallPtrSet of updated intervals.

Also share code between handleMove() and handleMoveIntoBundle().

Add support for moving dead defs across other live values in the
interval. The MI scheduler can do that.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165824 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-12 21:31:57 +00:00
Jakob Stoklund Olesen 795f951c6d Fix coalescing with IMPLICIT_DEF values.
PHIElimination inserts IMPLICIT_DEF instructions to guarantee that all
PHI predecessors have a live-out value. These IMPLICIT_DEF values are
not considered to be real interference when coalescing virtual
registers:

  %vreg1 = IMPLICIT_DEF
  %vreg2 = MOV32r0

When joining %vreg1 and %vreg2, the IMPLICIT_DEF instruction and its
value number should simply be erased since the %vreg2 value number now
provides a live-out value for the PHI predecesor block.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165813 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-12 18:03:04 +00:00
Jakob Stoklund Olesen ebba49395c Pass an explicit operand number to addLiveIns.
Not all instructions define a virtual register in their first operand.
Specifically, INLINEASM has a different format.

<rdar://problem/12472811>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165721 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-11 16:46:07 +00:00
Jakob Stoklund Olesen 6be75ae196 Don't crash on extra evil irreducible control flow.
When the CFG contains a loop with multiple entry blocks, the traces
computed by MachineTraceMetrics don't always have the same nice
properties. Loop back-edges are normally excluded from traces, but
MachineLoopInfo doesn't recognize loops with multiple entry blocks, so
those back-edges may be included.

Avoid asserting when that happens by adding an isEarlierInSameTrace()
function that accurately determines if a dominating block is part of the
same trace AND is above the currrent block in the trace.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165434 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-08 22:06:44 +00:00
Jakob Stoklund Olesen e6521b57cc Get MCSchedModel directly from the subtarget.
Not all targets have itineraries, but the subtarget always has an
MCSchedModel.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165236 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-04 17:30:43 +00:00
Jakob Stoklund Olesen f43fe1d163 Switch MachineTraceMetrics to the new TargetSchedModel interface.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165235 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-04 17:30:40 +00:00
Jakob Stoklund Olesen 0d141f867d The early if conversion pass is ready to be used as an opt-in.
Enable the pass by default for targets that request it, and change the
-enable-early-ifcvt to the opposite -disable-early-ifcvt.

There are still some x86 regressions when enabling early if-conversion
because of the missing machine models. Disable the pass for x86 until
machine models are added.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165075 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-03 00:51:32 +00:00
Jakob Stoklund Olesen 6c0c6d613a Remove the old coalescer algorithm.
The new algorithm has been enabled by default for almost a week now and
seems to be stable.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165062 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-02 22:45:03 +00:00
Jakob Stoklund Olesen d4919a988d Handle reserved registers more accurately in handleMove().
Reserved register live ranges look like a set of dead defs - any uses of
reserved registers are ignored.

Instead of skipping the updating of reserved register operands entirely,
just ignore the use operands and treat the def operands normally.

No test case, handleMove() is not commonly used yet.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165060 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-02 22:08:36 +00:00
Jakob Stoklund Olesen 27cb347d0e Make sure the whole live range is covered when values are pruned twice.
JoinVals::pruneValues() calls LIS->pruneValue() to avoid conflicts when
overlapping two different values. This produces a set of live range end
points that are used to reconstruct the live range (with SSA update)
after joining the two registers.

When a value is pruned twice, the set of end points was insufficient:

  v1 = DEF
  v1 = REPLACE1
  v1 = REPLACE2
  KILL v1

The end point at KILL would only reconstruct the live range from
REPLACE2 to KILL, leaving the range REPLACE1-REPLACE2 dead.

Add REPLACE2 as an end point in this case so the full live range is
reconstructed.

This fixes PR13999.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@165056 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-02 21:46:39 +00:00
Jakob Stoklund Olesen 83ef63efce Remove <def,read-undef> flags from partial redefinitions.
The new coalescer can turn a full virtual register definition into a
partial redef by merging another value into an unused vector lane.

Make sure to clear the <read-undef> flag on such defs.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164807 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-27 23:31:32 +00:00
Jakob Stoklund Olesen 5cf178f281 Enable the new coalescer algorithm by default.
The new coalescer is better at merging values into unused vector lanes,
improving NEON code.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164794 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-27 21:06:02 +00:00
Jakob Stoklund Olesen 657720bc6e Don't dereference begin() on an empty vector.
The fix is obvious and the only test case I have is horrible, so I am
not including it. The problem shows up when self-hosting clang on i386
with -new-coalescer enabled.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164793 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-27 21:05:59 +00:00
Jakob Stoklund Olesen ddc26d8936 Avoid dereferencing a NULL pointer.
Fixes PR13943.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164778 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-27 16:34:19 +00:00
Jakob Stoklund Olesen e6e2d8cd90 Ignore PHI-defs for -new-coalescer interference checks.
A PHI can't create interference on its own. If two live ranges interfere
at a PHI, they must also interfere when leaving one of the PHI
predecessors.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164330 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-20 23:08:42 +00:00
Jakob Stoklund Olesen 2df8ac84ae Extend -new-coalescer SSA update to handle mapped values as well.
The old-fashioned many-to-one value mapping doesn't always work when
merging vector lanes. A value can map to multiple different values, and
it can even be necessary to insert new PHIs.

When a value number is defined by a copy from a value number that
required SSa update, include the live range of the copied value number
in the SSA update as well. It is not necessarily a copy of the original
value number any longer.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164329 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-20 23:08:39 +00:00
Jakob Stoklund Olesen d40d4c34f7 Resolve conflicts involving dead vector lanes for -new-coalescer.
A common coalescing conflict in vector code is lane insertion:

  %dst = FOO
  %src = BAR
  %dst:ssub0 = COPY %src

The live range of %src interferes with the ssub0 lane of %dst, but that
lane is never read after %src would have clobbered it. That makes it
safe to merge the live ranges and eliminate the COPY:

  %dst = FOO
  %dst:ssub0 = BAR

This patch teaches the new coalescer to resolve conflicts where dead
vector lanes would be clobbered, at least as long as the clobbered
vector lanes don't escape the basic block.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164250 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-19 21:29:18 +00:00
Jakob Stoklund Olesen 87f7864c6d Merge into undefined lanes under -new-coalescer.
Add LIS::pruneValue() and extendToIndices(). These two functions are
used by the register coalescer when merging two live ranges requires
more than a trivial value mapping as supported by LiveInterval::join().

The pruneValue() function can remove the part of a value number that is
going to conflict in join(). Afterwards, extendToIndices can restore the
live range, using any new dominating value numbers and updating the SSA
form.

Use this complex value mapping to support merging a register into a
vector lane that has a conflicting value, but the clobbered lane is
undef.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164074 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-17 23:03:25 +00:00
Jakob Stoklund Olesen 98279e8d65 Stop adding <imp-def> operands when expanding REG_SEQUENCE.
These extra operands are not needed by register allocators using
VirtRegRewriter, and RAFast don't need them any longer.

By omitting the <imp-def> operands, it becomes possible for the new
register coalescer to track which lanes are valid and which are undef.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164073 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-17 23:03:21 +00:00
Jakob Stoklund Olesen c722ae7a2b Add alternative coalescing algorithm under a flag.
The live range of an SSA value forms a sub-tree of the dominator tree.
That means the live ranges of two values overlap if and only if the def
of one value lies within the live range of the other.

This can be used to simplify the interference checking a bit: Visit each
def in the two registers about to be joined. Check for interference
against the value that is live in the other register at the def point
only. It is not necessary to scan the set of overlapping live ranges,
this interference check can be done while computing the value mapping
required for the final live range join.

The new algorithm is prepared to handle more complicated conflict
resolution - We can allow overlapping live ranges with different values
as long as the differing lanes are undef or unused in the other
register.

The implementation in this patch doesn't do that yet, it creates code
that is nearly identical to the old algorithm's, except:

- The new stripCopies() function sees through multiple copies while
  the old RegistersDefinedFromSameValue() only can handle one.

- There are a few rare cases where the new algorithm can erase an
  IMPLICIT_DEF instuction that RegistersDefinedFromSameValue() couldn't
  handle.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163991 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-16 02:15:36 +00:00
Jakob Stoklund Olesen 2131eb4887 Fix problem when using LiveRangeQuery with block entries.
A value that is live in to a basic block should be returned by valueIn()
in LiveRangeQuery(getMBBStartIdx(MBB)), unless it is a PHI-def which
should be returned by valueDefined() instead.

Current code isn't using this functionality. Future code will.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163990 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-16 02:15:33 +00:00
Jakob Stoklund Olesen ab9baf7ff4 Don't depend on kill flags in removeCopyByCommutingDef().
Kill flags are removed more and more aggressively during the register
allocation passes, it is better to get information from LiveIntervals.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163972 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-15 16:32:11 +00:00
Jakob Stoklund Olesen 9ae03fa69f Make LiveRangeQuery work for PHIDefs as well.
If a PHI value happens to be live out from the layout predecessor of its
def block, the def slot index will be in the middle of the segment:

%vreg11 = [192r,240B:0)[352r,416B:2)[416B,496r:1)  0@192r 1@480B-phi %2@352r

A LiveRangeQuery for 480 should return NULL from valueIn() since the
PHI value is defined at the block entry, not live in to the block.

No test case, future code depends on this functionality.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163971 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-15 16:29:49 +00:00
Jakob Stoklund Olesen da0e8219b7 Fix test case to avoid PIC magic.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163827 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-13 19:47:45 +00:00
Jakob Stoklund Olesen 7bba7d0efc Fix the TCRETURNmi64 bug differently.
Add a PatFrag to match X86tcret using 6 fixed registers or less. This
avoids folding loads into TCRETURNmi64 using 7 or more volatile
registers.

<rdar://problem/12282281>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163819 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-13 18:31:27 +00:00
Jakob Stoklund Olesen 0767dc546e Revert r163761 "Don't fold indexed loads into TCRETURNmi64."
The patch caused "Wrong topological sorting" assertions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163810 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-13 16:52:17 +00:00
Jakob Stoklund Olesen aa0cfea9a4 Don't fold indexed loads into TCRETURNmi64.
We don't have enough GR64_TC registers when calling a varargs function
with 6 arguments. Since %al holds the number of vector registers used,
only %r11 is available as a scratch register.

This means that addressing modes using both base and index registers
can't be folded into TCRETURNmi64.

<rdar://problem/12282281>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163761 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-13 00:25:00 +00:00
Jakob Stoklund Olesen 01b50ca5b3 Delete dead code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163735 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-12 20:04:17 +00:00
Jakob Stoklund Olesen a6035773d8 Add TRI::getSubRegIndexLaneMask().
Sub-register lane masks are bitmasks that can be used to determine if
two sub-registers of a virtual register will overlap. For example, ARM's
ssub0 and ssub1 sub-register indices don't overlap each other, but both
overlap dsub0 and qsub0.

The lane masks will be accurate on most targets, but on targets that use
sub-register indexes in an irregular way, the masks may conservatively
report that two sub-register indices overlap when the eventually
allocated physregs don't.

Irregular register banks also mean that the bits in a lane mask can't be
mapped onto register units, but the concept is similar.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163630 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-11 16:34:08 +00:00
Jakob Stoklund Olesen de0250728b Clean the sub-reg index composition maps at emission.
Preserve the Composites map in the CodeGenSubRegIndex class so it can be
used to determine which sub-register indices can actually be composed.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163629 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-11 16:34:05 +00:00
Jakob Stoklund Olesen 59f45e4610 Add MCRI::getNumSubRegIndices() and start checking SubRegIndex ranges.
Apparently, NumSubRegIndices was completely unused before. Adjust it by
one to include the null subreg index, just like getNumRegs() includes
the null register.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163628 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-11 16:34:02 +00:00
Jakob Stoklund Olesen 519daf5d2d Don't attempt to use flags from predicated instructions.
The ARM backend can eliminate cmp instructions by reusing flags from a
nearby sub instruction with similar arguments.

Don't do that if the sub is predicated - the flags are not written
unconditionally.

<rdar://problem/12263428>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163535 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-10 19:17:25 +00:00
Jakob Stoklund Olesen a7390fadba Custom DAGCombine for and/or/xor are for all ARMs.
The 'select' transformations apply to all ARM architectures and don't
require hasV6T2Ops.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163396 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-07 17:34:15 +00:00
Jakob Stoklund Olesen c8ecb401ca TiedTo is an integer, not a bool.
Thanks, Andy.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163343 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-06 19:51:21 +00:00
Jakob Stoklund Olesen 45c5c57179 Allow overlaps between virtreg and physreg live ranges.
The RegisterCoalescer understands overlapping live ranges where one
register is defined as a copy of the other. With this change, register
allocators using LiveRegMatrix can do the same, at least for copies
between physical and virtual registers.

When a physreg is defined by a copy from a virtreg, allow those live
ranges to overlap:

  %CL<def> = COPY %vreg11:sub_8bit; GR32_ABCD:%vreg11
  %vreg13<def,tied1> = SAR32rCL %vreg13<tied0>, %CL<imp-use,kill>

We can assign %vreg11 to %ECX, overlapping the live range of %CL.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163336 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-06 18:15:23 +00:00
Jakob Stoklund Olesen e617ccb80d Handle overlapping regunit intervals in LiveIntervals::addKillFlags().
We will soon allow virtual register live ranges to overlap regunit live
ranges when the physreg is defined as a copy of the virtreg:

  %EAX = COPY %vreg5
  FOO %vreg5
  BAR %EAX<kill>

There is no real interference since %vreg5 and %EAX have the same value
where they overlap.

This patch prevents addKillFlags from adding virtreg kill flags to FOO
where the assigned physreg is overlapping the virtual register live
range.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163335 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-06 18:15:18 +00:00
Jakob Stoklund Olesen f9dff0e0e4 Clear kill flags while computing live ranges.
Kill flags are difficult to maintain, and liveness queries are better
handled by live intervals.

Kill flags are reinserted after register allocation by addKillFlags().

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163334 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-06 18:15:15 +00:00
Jakob Stoklund Olesen f632d80e0c Remove predicated pseudo-instructions.
These pseudos are no longer needed now that it is possible to represent
predicated instructions in SSA form.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163275 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-05 23:58:04 +00:00
Jakob Stoklund Olesen 098c6a547f Use predication instead of pseudo-opcodes when folding into MOVCC.
Now that it is possible to dynamically tie MachineInstr operands,
predicated instructions are possible in SSA form:

  %vreg3<def> = SUBri %vreg1, -2147483647, pred:14, pred:%noreg, %opt:%noreg
  %vreg4<def,tied1> = MOVCCr %vreg3<tied0>, %vreg1, %pred:12, pred:%CPSR

Becomes a predicated SUBri with a tied imp-use:

  SUBri %vreg1, -2147483647, pred:13, pred:%CPSR, opt:%noreg, %vreg1<imp-use,tied0>

This means that any instruction that is safe to move can be folded into
a MOVCC, and the *CC pseudo-instructions are no longer needed.

The test case changes reflect that Thumb2SizeReduce recognizes the
predicated instructions. It didn't understand the pseudos.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163274 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-05 23:58:02 +00:00
Jakob Stoklund Olesen f363ebdb03 Search the whole instruction for tied operands.
Implicit uses can be dynamically tied to defs. This will soon be used
for predicated instructions on ARM.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163177 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-04 22:59:30 +00:00
Jakob Stoklund Olesen 4f1a56ce3c Typo.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163154 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-04 18:44:43 +00:00
Jakob Stoklund Olesen 99c416c9f7 Actually use the MachineOperand field for isRegTiedToDefOperand().
The MachineOperand::TiedTo field was maintained, but not used.

This patch enables it in isRegTiedToDefOperand() and
isRegTiedToUseOperand() which are the actual functions use by the
register allocator.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163153 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-04 18:43:25 +00:00
Jakob Stoklund Olesen daddf07497 Move tie checks into MachineVerifier::visitMachineOperand.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163152 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-04 18:38:28 +00:00
Jakob Stoklund Olesen 9c13067276 Allow tied uses and defs in different orders.
After much agonizing, use a full 4 bits of precious MachineOperand space
to encode this. This uses existing padding, and doesn't grow
MachineOperand beyond its current 32 bytes.

This allows tied defs among the first 15 operands on a normal
instruction, just like the current MCInstrDesc constraint encoding.
Inline assembly needs to be able to tie more than the first 15 operands,
and gets special treatment.

Tied uses can appear beyond 15 operands, as long as they are tied to a
def that's in range.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163151 91177308-0d34-0410-b5e6-96231b3b80d8
2012-09-04 18:36:28 +00:00
Jakob Stoklund Olesen 94083149fd Add MachineInstr::tieOperands, remove setIsTied().
Manage tied operands entirely internally to MachineInstr. This makes it
possible to change the representation of tied operands, as I will do
shortly.

The constraint that tied uses and defs must be in the same order was too
restrictive.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@163021 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-31 20:50:53 +00:00
Jakob Stoklund Olesen 908c0c01f6 Don't enforce ordered inline asm operands.
I was too optimistic, inline asm can have tied operands that don't
follow the def order.

Fixes PR13742.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162998 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-31 15:34:59 +00:00
Jakob Stoklund Olesen 05e80f2714 Fix a couple of typos in EmitAtomic.
Thumb2 instructions are mostly constrained to rGPR, not tGPR which is
for Thumb1.

rdar://problem/12203728

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162968 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-31 02:08:34 +00:00
Jakob Stoklund Olesen e941df5e35 Don't use MCInstrDesc flags for implicit operands.
When a MachineInstr is constructed, its implicit operands are added
first, then the explicit operands are inserted before the implicits.

MCInstrDesc has oprand flags like early clobber and operand ties that
apply to the explicit operands.

Don't look at those flags when the implicit operands are first added in
the explicit operands's positions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162910 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-30 14:39:06 +00:00
Jakob Stoklund Olesen b4e090dffc Add a bit of documentation to copyPhysReg.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162879 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-29 23:52:55 +00:00
Jakob Stoklund Olesen 492d76b02e Verify the order of tied operands in inline asm.
When there are multiple tied use-def pairs on an inline asm instruction,
the tied uses must appear in the same order as the defs.

It is possible to write an LLVM IR inline asm instruction that breaks
this constraint, but there is no reason for a front end to emit the
operands out of order.

The gnu inline asm syntax specifies tied operands as a single read/write
constraint "+r", so ouf of order operands are not possible.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162878 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-29 23:52:52 +00:00
Jakob Stoklund Olesen 66390805ad Set the isTied flags when building INLINEASM MachineInstrs.
For normal instructions, isTied() is set automatically by addOperand(),
based on MCInstrDesc, but inline asm has tied operands outside the
descriptor.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162869 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-29 22:02:00 +00:00
Jakob Stoklund Olesen f036f7a1e7 Rename hasVolatileMemoryRef() to hasOrderedMemoryRef().
Ordered memory operations are more constrained than volatile loads and
stores because they must be ordered with respect to all other memory
operations.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162861 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-29 21:19:21 +00:00
Jakob Stoklund Olesen ad7ebc2aeb Add MachineMemOperand::isUnordered().
This means the same as LoadInst/StoreInst::isUnordered(), and implies
!isVolatile().

Atomic loads and stored are also ordered, and this is the right method
to check if it is safe to reorder memory operations. Ordered atomics
can't be reordered wrt normal loads and stores, which is a stronger
constraint than volatile.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162859 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-29 21:08:52 +00:00
Jakob Stoklund Olesen 0d75858395 Don't move normal loads across volatile/atomic loads.
It is technically allowed to move a normal load across a volatile load,
but probably not a good idea.

It is not allowed to move a load across an atomic load with
Ordering > Monotonic, and we model those with MOVolatile as well.

I recently removed the mayStore flag from atomic load instructions, so
they don't need a pseudo-opcode. This patch makes up for the difference.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162857 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-29 20:48:45 +00:00
Jakob Stoklund Olesen 90a4f78d3b Verify the consistency of inline asm operands.
The operands on an INLINEASM machine instruction are divided into groups
headed by immediate flag operands. Verify this structure.

Extract verifyTiedOperands(), and only call it for non-inlineasm
instructions.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162849 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-29 18:11:05 +00:00
Jakob Stoklund Olesen ca71c5debe Verify the tied operand flags.
WHen running with -verify-machineinstrs, check that tied operands come
in matching use/def pairs, and that they are consistent with MCInstrDesc
when it applies.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162816 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-29 00:38:03 +00:00
Jakob Stoklund Olesen 699ac04951 Maintain a vaild isTied bit as operands are added and removed.
The isTied bit is set automatically when a tied use is added and
MCInstrDesc indicates a tied operand. The tie is broken when one of the
tied operands is removed.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162814 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-29 00:37:58 +00:00
Jakob Stoklund Olesen 4ba6916a98 Add a MachineOperand::isTied() flag.
While in SSA form, a MachineInstr can have pairs of tied defs and uses.
The tied operands are used to represent read-modify-write operands that
must be assigned the same physical register.

Previously, tied operand pairs were computed from fixed MCInstrDesc
fields, or by using black magic on inline assembly instructions.

The isTied flag makes it possible to add tied operands to any
instruction while getting rid of (some of) the inlineasm magic.

Tied operands on normal instructions are needed to represent predicated
individual instructions in SSA form. An extra <tied,imp-use> operand is
required to represent the output value when the instruction predicate is
false.

Adding a predicate to:

  %vreg0<def> = ADD %vreg1, %vreg2

Will look like:

  %vreg0<tied,def> = ADD %vreg1, %vreg2, pred:3, %vreg7<tied,imp-use>

The virtual register %vreg7 is the value given to %vreg0 when the
predicate is false. It will be assigned the same physreg as %vreg0.

This commit adds the isTied flag and sets it based on MCInstrDesc when
building an instruction. The flag is not used for anything yet.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162774 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-28 18:34:41 +00:00
Jakob Stoklund Olesen 190e342cbc Don't allow TargetFlags on MO_Register MachineOperands.
Register operands are manipulated by a lot of target-independent code,
and it is not always possible to preserve target flags. That means it is
not safe to use target flags on register operands.

None of the targets in the tree are using register operand target flags.
External targets should be using immediate operands to annotate
instructions with operand modifiers.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162770 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-28 18:05:48 +00:00
Jakob Stoklund Olesen 325907d086 Check all patterns for missing instruction flags.
Both single-instruction and multi-instruction patterns can be checked
for missing mayLoad / mayStore, and hasSideEffects flags.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162734 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-28 03:26:49 +00:00
Jakob Stoklund Olesen 36d29bc723 Remove extra MayLoad/MayStore flags from atomic_load/store.
These extra flags are not required to properly order the atomic
load/store instructions. SelectionDAGBuilder chains atomics as if they
were volatile, and SelectionDAG::getAtomic() sets the isVolatile bit on
the memory operands of all atomic operations.

The volatile bit is enough to order atomic loads and stores during and
after SelectionDAG.

This means we set mayLoad on atomic_load, mayStore on atomic_store, and
mayLoad+mayStore on the remaining atomic read-modify-write operations.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162733 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-28 03:11:32 +00:00
Jakob Stoklund Olesen cff9baa952 Revert r162713: "Add ATOMIC_LDR* pseudo-instructions to model atomic_load on ARM."
This wasn't the right way to enforce ordering of atomics.

We are already setting the isVolatile bit on memory operands of atomic
operations which is good enough to enforce the correct ordering.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162732 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-28 03:11:27 +00:00
Jakob Stoklund Olesen 2f1c6f52bd More missing mayLoad flags on AVX multiclasses.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162714 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-28 00:02:01 +00:00
Jakob Stoklund Olesen dd364419ee Add ATOMIC_LDR* pseudo-instructions to model atomic_load on ARM.
It is not safe to use normal LDR instructions because they may be
reordered by the scheduler. The ATOMIC_LDR pseudos have a mayStore flag
that prevents reordering.

Atomic loads are also prevented from participating in rematerialization
and load folding.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162713 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-27 23:58:52 +00:00
Jakob Stoklund Olesen cac59d8ae8 Add missing mayLoad flags to a large class of AVX *_Int instructions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162622 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-24 23:29:07 +00:00
Jakob Stoklund Olesen 36ff8f25c5 Missed tLEApcrelJT.
ARMConstantIslandPass expects this instruction to stay in the same basic
block as the jump table branch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162615 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-24 22:46:55 +00:00
Jakob Stoklund Olesen 4ad27eda29 Infer instruction properties from single-instruction patterns.
Previously, instructions without a primary patterns wouldn't get their
properties inferred. Now, we use all single-instruction patterns for
inference, including 'def : Pat<>' instances.

This causes a lot of instruction flags to change.

- Many instructions no longer have the UnmodeledSideEffects flag because
  their flags are now inferred from a pattern.

- Instructions with intrinsics will get a mayStore flag if they already
  have UnmodeledSideEffects and a mayLoad flag if they already have
  mayStore. This is because intrinsics properties are linear.

- Instructions with atomic_load patterns get a mayStore flag because
  atomic loads can't be reordered. The correct workaround is to create
  pseudo-instructions instead of using normal loads. PR13693.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162614 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-24 22:46:53 +00:00
Jakob Stoklund Olesen 7778ee1ed9 Explicitly mark LEApcrel pseudos with hasSideEffects.
It's not clear that they should be marked as such, but tbb formation
fails if t2LEApcrelJT is hoisted of of a loop.

This doesn't change the flags on these instructions,
UnmodeledSideEffects was already inferred from the missing pattern.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162603 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-24 21:44:11 +00:00
Jakob Stoklund Olesen aaaecfce70 Stop inferring isVariadic from instruction patterns.
Instructions are now only marked as variadic if they use variable_ops in
their ins list.

A variadic SDNode is typically used for call nodes that have the call
arguments as operands.

A variadic MachineInstr can actually encode a variable number of
operands, for example ARM's stm/ldm instructions. A call instruction
does not have to be variadic. The call argument registers are added as
implicit operands.

This change remove the MCID::Variadic flags from most call and return
instructions, allowing us to better verify their operands.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162599 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-24 21:08:09 +00:00
Jakob Stoklund Olesen 0745b649ed Fix call instruction operands in ARMFastISel.
The ARM BL and BLX instructions don't have predicate operands, but the
thumb variants tBL and tBLX do.

The argument registers should be added as implicit uses.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162593 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-24 20:52:46 +00:00
Jakob Stoklund Olesen 9511a460d8 Mark X86::RET and RETI instructions as variadic.
There is special magic happening when returning floating point values on
the x87 stack. The RET instructions get extra f80 operands.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162592 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-24 20:52:44 +00:00
Jakob Stoklund Olesen baa74e4b35 Avoid including explicit uses when counting SDNode imp-uses.
It is legal to have a register node as an explicit operand, it shouldn't
be counted as an implicit use.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162591 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-24 20:52:42 +00:00
Jakob Stoklund Olesen 91f8dc9bae Verify explicit instruction properties when they can be inferred.
It is now allowed to explicitly set hasSideEffects, mayStore, and
mayLoad on instructions with patterns.

Verify that the patterns are consistent with the explicit flags.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162569 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-24 17:08:41 +00:00
Jakob Stoklund Olesen 4d320db5d1 Fix load/store SDNode flags.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162558 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-24 14:43:30 +00:00
Jakob Stoklund Olesen ea47628cba Add missing SDNPSideEffect flags.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162557 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-24 14:43:27 +00:00
Jakob Stoklund Olesen 3d2a2d1217 Remove more mayLoad workarounds.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162556 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-24 14:43:22 +00:00
Jakob Stoklund Olesen 6211386799 Remove some spurious mayLoad = 0 flags.
They were inserted to silence TableGen's warning about
redundant properties. That warning is now gone.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162517 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-24 00:31:20 +00:00
Jakob Stoklund Olesen 912519a72e Heed guessInstructionProperties, and stop warning on redundant flags.
Emit TableGen errors if guessInstructionProperties is 0 and
instruction properties can't be inferred from patterns.

Allow explicit instruction properties even when they can be inferred.

This patch doesn't change the TableGen output. Redundant properties
are not yet verified because the tree has errors.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162516 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-24 00:31:16 +00:00
Jakob Stoklund Olesen 02c63803e5 Add missing SDNP properties on the flushw node.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162515 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-24 00:31:13 +00:00
Jakob Stoklund Olesen cfe8a9695b X86MemBarrier has unmodeled side effects.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162514 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-24 00:31:10 +00:00
Jakob Stoklund Olesen 91f3a6cfd9 Preserve operand flags in convertToThreeAddress() by copying operands.
No test case, this is a generalization of r160260.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162485 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-23 22:36:31 +00:00
Jakob Stoklund Olesen c1f10fd5b9 Tristate mayLoad, mayStore, and hasSideEffects.
Keep track of the set/unset state of these bits along with their
true/false values, but treat '?' as '0' for now.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162461 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-23 19:34:46 +00:00
Jakob Stoklund Olesen f104bf65b9 Add CodeGenTarget::guessInstructionProperties.
Currently, TableGen just guesses instruction properties when it can't
infer them form patterns.

This adds a guessInstructionProperties flag to the instruction set
definition that will be used to disable guessing. The flag is intended
as a migration aid. It will be removed again when no more targets need
their properties guessed.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162460 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-23 19:34:41 +00:00
Jakob Stoklund Olesen 55f1ef0010 Fix attributes on X86 store intrinsics.
These intrinsics don't have unmodeled side effects, they are just
stores.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162459 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-23 19:21:38 +00:00
Jakob Stoklund Olesen 376a8a773e Print out the location of expanded multiclass defs in TableGen errors.
When reporting an error for a defm, we would previously only report the
location of the outer defm, which is not always where the error is.

Now we also print the location of the expanded multiclass defs:

lib/Target/X86/X86InstrSSE.td:2902:12: error: foo
  defm ADD : basic_sse12_fp_binop_s<0x58, "add", fadd, SSE_ALU_ITINS_S>,
             ^
lib/Target/X86/X86InstrSSE.td:2801:11: note: instantiated from multiclass
  defm PD : sse12_fp_packed<opc, !strconcat(OpcodeStr, "pd"), OpNode, VR128,
            ^
lib/Target/X86/X86InstrSSE.td:194:5: note: instantiated from multiclass
    def rm : PI<opc, MRMSrcMem, (outs RC:$dst), (ins RC:$src1, x86memop:$src2),
        ^

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162409 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-22 23:33:58 +00:00
Jakob Stoklund Olesen 5379904807 Add a missing def flag.
*** Bad machine code: Explicit definition marked as use ***
- function:    test_cos
- basic block: BB#0 L.entry (0x7ff2a2024fd0)
- instruction: VSETLNi32 %D11, %D11<undef>, %R0, 0, pred:14, pred:%noreg, %Q5<imp-use,kill>, %Q5<imp-def>
- operand 0:   %D11

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162247 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-21 00:34:53 +00:00
Jakob Stoklund Olesen 990ca5517f Fix a quadratic algorithm in MachineBranchProbabilityInfo.
The getSumForBlock function was quadratic in the number of successors
because getSuccWeight would perform a linear search for an already known
iterator.

This patch was originally committed as r161460, but reverted again
because of assertion failures. Now that duplicate Machine CFG edges have
been eliminated, this works properly.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162233 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-20 22:01:38 +00:00
Jakob Stoklund Olesen e7fdef420d Don't add CFG edges for redundant conditional branches.
IR that hasn't been through SimplifyCFG can look like this:

  br i1 %b, label %r, label %r

Make sure we don't create duplicate Machine CFG edges in this case.

Fix the machine code verifier to accept conditional branches with a
single CFG edge.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162230 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-20 21:39:52 +00:00
Jakob Stoklund Olesen 2c5a960b0e Clarify that duplicate edges are not allowed in the Machine CFG.
LLVM IR has labeled duplicate CFG edges, but since Machine CFG edges
don't have labels, it doesn't make sense to allow duplicates. There is
no way of telling what the edges mean.

Duplicate CFG edges cause confusion when dealing with edge weights. It
seems that code producing duplicate CFG edges usually does the wrong
thing with edge weights.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162227 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-20 21:05:58 +00:00
Jakob Stoklund Olesen 228e3f5fac Add a verification pass after ExpandISelPseudos.
This pass often has weird CFG hacks and hand-written MI building code
that can go wrong in many ways.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162224 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-20 20:52:08 +00:00
Jakob Stoklund Olesen b254c6d77a Add CFG checks to MachineVerifier.
Verify that the predecessor and successor lists are consistent and free
of duplicates.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@162223 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-20 20:52:06 +00:00