Commit Graph

1327 Commits

Author SHA1 Message Date
045573ce21 Add AVX matching patterns to Packed Bit Test intrinsics.
Apply the same approach of SSE4.1 ptest intrinsics but
create a new x86 node "testp" since AVX introduces
vtest{ps}{pd} instructions which set ZF and CF depending
on sign bit AND and ANDN of packed floating-point sources.

This is slightly different from what the "ptest" does.
Tests comming with the other 256 intrinsics tests.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@110744 91177308-0d34-0410-b5e6-96231b3b80d8
2010-08-10 23:25:42 +00:00
405f11b300 Support AVX 256-bit load and store intrinsics
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@110645 91177308-0d34-0410-b5e6-96231b3b80d8
2010-08-10 01:43:16 +00:00
ac09835a22 Support very basic (doesn't include ABI support in the front-end, varags, ...) 256-bit argument passing and return for AVX
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@110394 91177308-0d34-0410-b5e6-96231b3b80d8
2010-08-05 23:35:51 +00:00
b6729dc0ef Make x86-64 membarriers work without sse and clean up some of the
uses.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@110274 91177308-0d34-0410-b5e6-96231b3b80d8
2010-08-04 23:03:04 +00:00
98f985607b Support all 128-bit AVX vector intrinsics. Most part of them I already
declared during the addition of the assembler support, the additional
changes are:
- Add missing intrinsics
- Move all SSE conversion instructions in X86InstInfo64.td to the SSE.td file.
- Duplicate some patterns to AVX mode.
- Step into PCMPEST/PCMPIST custom inserter and add AVX versions.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@109878 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-30 19:54:33 +00:00
b2eeed7464 Revert r109652, and remove the offending assert in loadRegFromStackSlot instead.
We do sometimes load from a too small stack slot when dealing with x86 arguments
(varargs and smaller-than-32-bit args). It looks like we know what we are doing
in those cases, so I am going to remove the assert instead of artifically
enlarging stack slot sizes.

The assert in storeRegToStackSlot stays in. We don't want to write beyond the
bounds of a stack slot.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@109764 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-29 17:42:27 +00:00
4c010ec851 Create a fixed stack object for varargs that is as large as any register.
The size of this object isn't used for anything - technically it is of variable
size.

This avoids a false positive from the assert in
X86InstrInfo::loadRegFromStackSlot, and fixes PR7735.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@109652 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-28 20:55:38 +00:00
51409214d7 Implement a vectorized algorithm for <16 x i8> << <16 x i8>
This is about 4x faster and smaller than the existing scalarization.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@109566 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-28 00:21:48 +00:00
bdcb5afb77 ~40% faster vector shl <4 x i32> on SSE 4.1 Larger improvements for smaller types coming in future patches.
For:

define <2 x i64> @shl(<4 x i32> %r, <4 x i32> %a) nounwind readnone ssp {
entry:
  %shl = shl <4 x i32> %r, %a                     ; <<4 x i32>> [#uses=1]
  %tmp2 = bitcast <4 x i32> %shl to <2 x i64>     ; <<2 x i64>> [#uses=1]
  ret <2 x i64> %tmp2
}

We get:

_shl:                                   ## @shl
	pslld	$23, %xmm1
	paddd	LCPI0_0, %xmm1
	cvttps2dq	%xmm1, %xmm1
	pmulld	%xmm1, %xmm0
	ret

Instead of:

_shl:                                   ## @shl
	pshufd	$3, %xmm0, %xmm2
	movd	%xmm2, %eax
	pshufd	$3, %xmm1, %xmm2
	movd	%xmm2, %ecx
	shll	%cl, %eax
	movd	%eax, %xmm2
	pshufd	$1, %xmm0, %xmm3
	movd	%xmm3, %eax
	pshufd	$1, %xmm1, %xmm3
	movd	%xmm3, %ecx
	shll	%cl, %eax
	movd	%eax, %xmm3
	punpckldq	%xmm2, %xmm3
	movd	%xmm0, %eax
	movd	%xmm1, %ecx
	shll	%cl, %eax
	movd	%eax, %xmm2
	movhlps	%xmm0, %xmm0
	movd	%xmm0, %eax
	movhlps	%xmm1, %xmm1
	movd	%xmm1, %ecx
	shll	%cl, %eax
	movd	%eax, %xmm0
	punpckldq	%xmm0, %xmm2
	movdqa	%xmm2, %xmm0
	punpckldq	%xmm3, %xmm0
	ret


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@109549 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-27 22:37:06 +00:00
dee81010eb On x86, f32 / f64 nodes share the same registers as 128-bit vector values.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@109450 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-26 21:50:05 +00:00
70017e44cd Add an ILP scheduler. This is a register pressure aware scheduler that's
appropriate for targets without detailed instruction iterineries.
The scheduler schedules for increased instruction level parallelism in
low register pressure situation; it schedules to reduce register pressure
when the register pressure becomes high.

On x86_64, this is a win for all tests in CFP2000. It also sped up 256.bzip2
by 16%.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@109300 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-24 00:39:05 +00:00
c76d23f2e2 The only supported calling convention for X86-64 uses
SSE, so we can't return floating point values if this
is disabled.  Detect this error for clang.

With SSE1 only, f64 is a problem; it can be done, but
neither llvm-gcc nor clang has ever generated correct
code for it.  Since nobody noticed this I think it's
OK to treat it as an error for now.

This also handles SSE-sized vectors of floating point.
8207686, 8204109.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@109201 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-23 00:30:35 +00:00
9a9d275dc7 Custom lower the memory barrier instructions and add support
for lowering without sse2.  Add a couple of new testcases.

Fixes a few libgomp tests and latent bugs.  Remove a few todos.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@109078 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-22 02:48:34 +00:00
90eb4024ba 80-columns.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@109070 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-22 00:26:08 +00:00
c8ea673bc0 Fix a couple issues with Win64 ABI
1) all registers were spilled as xmm, regardless of actual size
2) win64 abi doesn't do the varargs-size-in-%al thing

Still to look into:

xmm6-15 are marked as clobbered by call instructions on win64 even though they aren't.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@109035 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-21 20:49:52 +00:00
dab4dac2a0 Pulling out previous patch, must've run the tests in
the wrong directory.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@109005 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-21 09:23:56 +00:00
87f41370a8 Lower MEMBARRIER on x86 and support processors without SSE2.
Fixes a pile of libgomp failures in the llvm-gcc testsuite due
to the libcall not existing.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@109004 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-21 09:05:23 +00:00
60108e96bb Split -enable-finite-only-fp-math to two options:
-enable-no-nans-fp-math and -enable-no-infs-fp-math. All of the current codegen fp math optimizations only care whether the fp arithmetics arguments and results can never be NaN.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@108465 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-15 22:07:12 +00:00
b5378ea12e Use TargetOpcode::COPY instead of X86-native register copy instructions when
lowering atomics. This will allow those copies to still be coalesced after
TII::isMoveInstr is removed.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@108385 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-14 23:50:27 +00:00
dedd974e7e Fix for PR7193 was overly conservative. The only case where sibcall callee
address cannot be allocated a register is in 32-bit mode where the first
three arguments are marked inreg. In that case EAX, EDX, and ECX will be
used for argument passing.

This fixes PR7610.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@108327 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-14 06:44:01 +00:00
84023e0fbe Reapply bottom-up fast-isel, with several fixes for x86-32:
- Check getBytesToPopOnReturn().
 - Eschew ST0 and ST1 for return values.
 - Fix the PIC base register initialization so that it doesn't ever
   fail to end up the top of the entry block.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@108039 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-10 09:00:22 +00:00
d737fcafc4 An x86 function returns a floating point value in st(0), and we must make sure
it is popped, even if it is ununsed. A CopyFromReg node is too weak to represent
the required sideeffect, so insert an FpGET_ST0 instruction directly instead.

This will matter when CopyFromReg gets lowered to a generic COPY instruction.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@108037 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-10 04:04:25 +00:00
02266e29f9 --- Reverse-merging r107947 into '.':
U    utils/TableGen/FastISelEmitter.cpp
--- Reverse-merging r107943 into '.':
U    test/CodeGen/X86/fast-isel.ll
U    test/CodeGen/X86/fast-isel-loads.ll
U    include/llvm/Target/TargetLowering.h
U    include/llvm/Support/PassNameParser.h
U    include/llvm/CodeGen/FunctionLoweringInfo.h
U    include/llvm/CodeGen/CallingConvLower.h
U    include/llvm/CodeGen/FastISel.h
U    include/llvm/CodeGen/SelectionDAGISel.h
U    lib/CodeGen/LLVMTargetMachine.cpp
U    lib/CodeGen/CallingConvLower.cpp
U    lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
U    lib/CodeGen/SelectionDAG/FunctionLoweringInfo.cpp
U    lib/CodeGen/SelectionDAG/FastISel.cpp
U    lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp
U    lib/CodeGen/SelectionDAG/ScheduleDAGSDNodes.cpp
U    lib/CodeGen/SelectionDAG/InstrEmitter.cpp
U    lib/CodeGen/SelectionDAG/TargetLowering.cpp
U    lib/Target/XCore/XCoreISelLowering.cpp
U    lib/Target/XCore/XCoreISelLowering.h
U    lib/Target/X86/X86ISelLowering.cpp
U    lib/Target/X86/X86FastISel.cpp
U    lib/Target/X86/X86ISelLowering.h


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107987 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-09 16:37:18 +00:00
01dcb18cf3 Fix the memoperand offsets in code generated for va_start.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107948 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-09 01:06:48 +00:00
bf87e24917 Re-apply bottom-up fast-isel, with fixes. Be very careful to avoid emitting
a DBG_VALUE after a terminator, or emitting any instructions before an EH_LABEL.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107943 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-09 00:39:23 +00:00
599b531a96 Change LEA to have 5 operands for its memory operand, just
like all other instructions, even though a segment is not
allowed.  This resolves a bunch of gross hacks in the 
encoder and makes LEA more consistent with the rest of the
instruction set.

No functionality change.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107934 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-08 23:46:44 +00:00
ac0ed5dc08 add some long-overdue enums to refer to the parts of the 5-operand
X86 memory operand.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107925 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-08 22:41:28 +00:00
f595141525 Revert 107840 107839 107813 107804 107800 107797 107791.
Debug info intrinsics win for now.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107850 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-08 01:00:56 +00:00
bcc8017c73 Move getExtLoad() and (some) getLoad() DebugLoc argument after EVT argument for consistency sake.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107820 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-07 22:15:37 +00:00
f423a69839 Add X86FastISel support for return statements. This entails refactoring
a bunch of stuff, to allow the target-independent calling convention
logic to be employed.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107800 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-07 18:32:53 +00:00
a4160c3434 Simplify FastISel's constructor by giving it a FunctionLoweringInfo
instance, rather than pointers to all of FunctionLoweringInfo's
members.

This eliminates an NDEBUG ABI sensitivity.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107789 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-07 16:29:44 +00:00
c9403659a9 Split the SDValue out of OutputArg so that SelectionDAG-independent
code can do calling-convention queries. This obviates OutputArgReg.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107786 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-07 15:54:55 +00:00
e2b448c208 Accept RIP-relative symbols with 'i' constraint, and
print the (%rip) only if the 'a' modifier is present.
PR 7528.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107727 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-06 23:27:00 +00:00
c9af33c685 CanLowerReturn doesn't need a SelectionDAG; it just needs an LLVMContext.
SelectBasicBlock doesn't needs its BasicBlock argument.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107712 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-06 22:19:37 +00:00
0d881dabc1 Propagate debug loc.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107710 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-06 22:08:15 +00:00
14152b480d Reapply r107655 with fixes; insert the pseudo instruction into
the block before calling the expansion hook. And don't
put EFLAGS in a mbb's live-in list twice.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107691 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-06 20:24:04 +00:00
258c58cc62 Revert r107655.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107668 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-06 15:49:48 +00:00
b81c771c0d Fix a bunch of custom-inserter functions to handle the case where
the pseudo instruction is not at the end of the block.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107655 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-06 15:18:19 +00:00
f7a0c7bf8b Fix up -fstack-protector on linux to use the segment
registers.  Split out testcases per architecture and os
now.

Patch from Nelson Elhage.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107640 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-06 05:18:56 +00:00
62f35a2c13 Have the X86 backend use Triple instead of a string and some enums.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107625 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-05 19:26:33 +00:00
32b4b5aea6 more tidying.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107615 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-05 05:53:14 +00:00
f172ecd964 Just rip v2f32 support completely out of the X86 backend. In
the example in the testcase, we now generate:

_test1:                                 ## @test1
	movss	4(%esp), %xmm0
	addss	8(%esp), %xmm0
	movl	12(%esp), %eax
	movss	%xmm0, (%eax)
	ret

instead of:

_test1:                                                     ## @test1
	subl	$20, %esp
	movl	24(%esp), %eax
	movq	%mm0, (%esp)
	movq	%mm0, 8(%esp)
	movss	(%esp), %xmm0
	addss	12(%esp), %xmm0
	movss	%xmm0, (%eax)
	addl	$20, %esp
	ret

v2f32 support did not work reliably because most of the X86
backend didn't know it was legal.  It was apparently only added
to support returning source-level v2f32 values in MMX registers
in x86-32 mode.  If ABI compatibility is important on this
GCC-extended-vector type for some reason, then the frontend
should generate IR that returns v2i32 instead of v2f32.  However,
we generally don't try very hard to be abi compatible on gcc
extended vectors. 



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107601 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-04 23:07:25 +00:00
e35d9842f7 fix PR7518 - terrible codegen of <2 x float>, by only marking
v2f32 as legal in 32-bit mode.  It is just as terrible there,
but I just care about x86-64 and noone claims it is valuable
in 64-bit mode.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107600 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-04 22:57:10 +00:00
ed2ae136d2 Remove isSS argument from CreateFixedObject. Fixed objects cannot be spill slots so it's always false.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107550 91177308-0d34-0410-b5e6-96231b3b80d8
2010-07-03 00:40:23 +00:00
e1c2b9cc3d use ArgOperand API
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107280 91177308-0d34-0410-b5e6-96231b3b80d8
2010-06-30 13:03:37 +00:00
78337b4d4d Remove pointless and unused variables.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107130 91177308-0d34-0410-b5e6-96231b3b80d8
2010-06-29 12:48:49 +00:00
c25ccf85e5 Reduce indentation via early exit. NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107067 91177308-0d34-0410-b5e6-96231b3b80d8
2010-06-28 21:08:32 +00:00
1cfe44a460 use ArgOperand API
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@106944 91177308-0d34-0410-b5e6-96231b3b80d8
2010-06-26 11:51:52 +00:00
1784d160e4 The hasMemory argument is irrelevant to how the argument
for an "i" constraint should get lowered; PR 6309.  While
this argument was passed around a lot, this is the only
place it was used, so it goes away from a lot of other
places.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@106893 91177308-0d34-0410-b5e6-96231b3b80d8
2010-06-25 21:55:36 +00:00
730c07e50d - Reapply r106066 now that the bzip2 build regression has been fixed.
- 2010-06-25-CoalescerSubRegDefDead.ll is the testcase for r106878.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@106880 91177308-0d34-0410-b5e6-96231b3b80d8
2010-06-25 20:48:10 +00:00