Port `DIExpression::Operand` over to `MDExpression::ExprOperand`. The
logic is needed directly in `MDExpression` to support printing in
assembly.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@229002 91177308-0d34-0410-b5e6-96231b3b80d8
The sub-arrays for compile units have for a long time been initialized
to distinct temporary nodes with the `DW_TAG_base_type` tag, with no
other operands. These invalid `DIBasicType`s are later replaced with
appropriate arrays.
This seems like a poor man's assertion that the arrays do eventually get
replaced. These days, temporaries in the graph will cause assertions
when writing bitcode or assembly, so this isn't necessary. Use
temporary empty tuples instead.
Note that the whole idea of using temporaries and then replacing them
later is wasteful here. We never actually want to merge compile units
by uniquing based on content. Compile units should use `getDistinct()`
instead of `get()`, and then their operands can be freely replaced later
on.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228967 91177308-0d34-0410-b5e6-96231b3b80d8
Apparently some code finally started to tickle this after my
canonicalization changes to instcombine.
The bug stems from trying to form a vector type out of scalars that
aren't compatible at all. In this example, from x86_mmx values. The code
in the vectorizer that checks for reasonable types whas checking for
aggregates or vectors, but there are lots of other types that should
just never reach the vectorizer.
Debugging this was made more confusing by the lie in an assert in
VectorType::get() -- it isn't that the types are *primitive*. The types
must be integer, pointer, or floating point types. No other types are
allowed.
I've improved the assert and added a helper to the vectorizer to handle
the element type validity checks. It now re-uses the VectorType static
function and then further excludes weird target-specific types that we
probably shouldn't be touching here (x86_fp80 and ppc_fp128). Neither of
these are really reachable anyways (neither 80-bit nor 128-bit things
will get vectorized) but it seems better to just eagerly exclude such
nonesense.
I've added a test case, but while it definitely covers two of the paths
through this code there may be more paths that would benefit from test
coverage. I'm not familiar enough with the SLP vectorizer to synthesize
test cases for all of these, but was able to update the code itself by
inspection.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228899 91177308-0d34-0410-b5e6-96231b3b80d8
createReplaceableCompositeType() that allows to create non-forward-declared
temporary nodes.
Paired commit with CFE.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228852 91177308-0d34-0410-b5e6-96231b3b80d8
Add new API for converting temporaries that may self-reference.
Self-referencing nodes are not allowed to be uniqued, so sending them
into `replaceWithUniqued()` is dangerous (and this commit adds
assertions that prevent it).
`replaceWithPermanent()` has similar semantics to `get()` followed by
calls to `replaceOperandWith()`. In particular, if there's a
self-reference, it returns a distinct node; otherwise, it returns a
uniqued one. Like `replaceWithUniqued()` and `replaceWithDistinct()`
(well, it calls out to them) it mutates the temporary node in place if
possible, only calling `replaceAllUsesWith()` on a uniquing collision.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228726 91177308-0d34-0410-b5e6-96231b3b80d8
I noticed this fields were never used in r228607, but I neglected to
propagate that into `MDTemplateParameter` until now. This really should
have been done before commit in r228640; sorry for the churn.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228652 91177308-0d34-0410-b5e6-96231b3b80d8
Add specialized debug info metadata nodes that match the `DIDescriptor`
wrappers (used by `DIBuilder`) closely. Assembly and bitcode support to
follow soon (it'll mostly just be obvious), but this sketches in today's
schema. This is the first big commit (well, the only *big* one aside
from the testcase changes that'll come when I move this into place) for
PR22464.
I've marked a bunch of obvious changes as `TODO`s in the source; I plan
to make those changes promptly after this hierarchy is moved underneath
`DIDescriptor`, but for now I'm aiming mostly to match the status quo.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228640 91177308-0d34-0410-b5e6-96231b3b80d8
intermediate representation. This
- increases consistency by using the same granularity everywhere
- allows for pieces < 1 byte
- DW_OP_piece didn't actually allow storing an offset.
Part of PR22495.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228631 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
It's important that our users immediately know what gc.safepoint_poll
is. Also fix the style of the declaration of CreateGCStatepoint, in
preparation for another change that will wrap it.
Reviewers: reames
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D7517
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228626 91177308-0d34-0410-b5e6-96231b3b80d8
Remove handling for DW_TAG_constant. We started producing it in
r110656, but reverted that in r110876 without dropping the support.
Finish the job.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228623 91177308-0d34-0410-b5e6-96231b3b80d8
`DIExpression` deals with `uint64_t`, so it doesn't make sense that
`createExpression()` is created from `int64_t`. Switch to `uint64_t` to
unify them.
I've temporarily left in the `int64_t` version, which forwards to the
`uint64_t` version. I'll delete it once I've updated the callers.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228619 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
The alias.scope metadata represents sets of things an instruction might
alias with. When generically combining the metadata from two
instructions the result must be the union of the original sets, because
the new instruction might alias with anything any of the original
instructions aliased with.
Reviewers: hfinkel
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D7490
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228525 91177308-0d34-0410-b5e6-96231b3b80d8
Gather and Scatter are new introduced intrinsics, comming after recently implemented masked load and store.
This is the first patch for Gather and Scatter intrinsics. It includes only the syntax, parsing and verification.
Gather and Scatter intrinsics allow to perform multiple memory accesses (read/write) in one vector instruction.
The intrinsics are not target specific and will have the following syntax:
Gather:
declare <16 x i32> @llvm.masked.gather.v16i32(<16 x i32*> <vector of ptrs>, i32 <alignment>, <16 x i1> <mask>, <16 x i32> <passthru>)
declare <8 x float> @llvm.masked.gather.v8f32(<8 x float*><vector of ptrs>, i32 <alignment>, <8 x i1> <mask>, <8 x float><passthru>)
Scatter:
declare void @llvm.masked.scatter.v8i32(<8 x i32><vector value to be stored> , <8 x i32*><vector of ptrs> , i32 <alignment>, <8 x i1> <mask>)
declare void @llvm.masked.scatter.v16i32(<16 x i32> <vector value to be stored> , <16 x i32*> <vector of ptrs>, i32 <alignment>, <16 x i1><mask> )
Vector of ptrs - a set of source/destination addresses, to load/store the value.
Mask - switches on/off vector lanes to prevent memory access for switched-off lanes
vector of ptrs, value and mask should have the same vector width.
These are code examples where gather / scatter should be used and will allow function vectorization
;void foo1(int * restrict A, int * restrict B, int * restrict C) {
; for (int i=0; i<SIZE; i++) {
; A[i] = B[C[i]];
; }
;}
;void foo3(int * restrict A, int * restrict B) {
; for (int i=0; i<SIZE; i++) {
; A[B[i]] = i+5;
; }
;}
Tests will come in the following patches, with CodeGen and Vectorizer.
http://reviews.llvm.org/D7433
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228521 91177308-0d34-0410-b5e6-96231b3b80d8
Remove unnecessary restriction of 24-bits for line numbers in
`MDLocation`.
The rest of the debug info schema (with the exception of local
variables) uses 32-bits for line numbers. As I introduce the
specialized nodes, it makes sense to canonicalize on one size or the
other.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228455 91177308-0d34-0410-b5e6-96231b3b80d8
Minimize the boilerplate required for the `MDNode` subclass
`DenseMapInfo<>` overrides in `LLVMContextImpl`.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228212 91177308-0d34-0410-b5e6-96231b3b80d8
This pass is responsible for figuring out where to place call safepoints and safepoint polls. It doesn't actually make the relocations explicit; that's the job of the RewriteStatepointsForGC pass (http://reviews.llvm.org/D6975).
Note that this code is not yet finalized. Its moving in tree for incremental development, but further cleanup is needed and will happen over the next few days. It is not yet part of the standard pass order.
Planned changes in the near future:
- I plan on restructuring the statepoint rewrite to use the functions add to the IRBuilder a while back.
- In the current pass, the function "gc.safepoint_poll" is treated specially but is not an intrinsic. I plan to make identifying the poll function a property of the GCStrategy at some point in the near future.
- As follow on patches, I will be separating a collection of test cases we have out of tree and submitting them upstream.
- It's not explicit in the code, but these two patches are introducing a new state for a statepoint which looks a lot like a patchpoint. There's no a transient form which doesn't yet have the relocations explicitly represented, but does prevent reordering of memory operations. Once this is in, I need to update actually make this explicit by reserving the 'unused' argument of the statepoint as a flag, updating the docs, and making the code explicitly check for such a thing. This wasn't really planned, but once I split the two passes - which was done for other reasons - the intermediate state fell out. Just reminds us once again that we need to merge statepoints and patchpoints at some point in the not that distant future.
Future directions planned:
- Identifying more cases where a backedge safepoint isn't required to ensure timely execution of a safepoint poll.
- Tweaking the insertion process to generate easier to optimize IR. (For example, investigating making SplitBackedge) the default.
- Adding opt-in flags for a GCStrategy to use this pass. Once done, add this pass to the actual pass ordering.
Differential Revision: http://reviews.llvm.org/D6981
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228090 91177308-0d34-0410-b5e6-96231b3b80d8
Patch by: Igor Laevsky
"This change generalizes statepoint verification to use ImmutableCallSite instead of CallInst. This will allow to easily implement invoke statepoint verification (in a following change)."
Differential Revision: http://reviews.llvm.org/D7308
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@228064 91177308-0d34-0410-b5e6-96231b3b80d8
Allow `GenericDebugNode` construction directly from `MDString`, rather
than requiring `StringRef`s. I've refactored the `StringRef`
constructors to use these. There's no real functionality change here,
except for exposing the lower-level API.
The purpose of this is to simplify construction of string operands when
reading bitcode. It's unnecessarily indirect to parse an `MDString` ID,
lookup the `MDString` in the bitcode reader list, get the `StringRef`
out of that, and then have `GenericDebugNode::getImpl()` use
`MDString::get()` to acquire the original `MDString`. Instead, this
allows the bitcode reader to directly pass in the `MDString`.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227848 91177308-0d34-0410-b5e6-96231b3b80d8
Move debug-info-centred `Metadata` subclasses into their own
header/source file. A couple of private template functions are needed
from both `Metadata.cpp` and `DebugInfoMetadata.cpp`, so I've moved them
to `lib/IR/MetadataImpl.h`.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227835 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
MetadataAsValue uses a canonical format that strips the MDNode if it
contains only a single constant value. This triggers an assertion when
trying to cast the value to a MDNode.
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D7165
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227319 91177308-0d34-0410-b5e6-96231b3b80d8
querying of the pass registry.
The pass manager relies on the static registry of PassInfo objects to
perform all manner of its functionality. I don't understand why it does
much of this. My very vague understanding is that this registry is
touched both during static initialization *and* while each pass is being
constructed. As a consequence it is hard to make accessing it not
require a acquiring some lock. This lock ends up in the hot path of
setting up, tearing down, and invaliditing analyses in the legacy pass
manager.
On most systems you can observe this as a non-trivial % of the time
spent in 'ninja check-llvm'. However, I haven't really seen it be more
than 1% in extreme cases of compiling more real-world software,
including LTO.
Unfortunately, some of the GPU JITs are seeing this taking essentially
all of their time because they have very small IR running through
a small pass pipeline very many times (at least, this is the vague
understanding I have of it).
This patch tries to minimize the cost of looking up PassInfo objects by
leveraging the fact that the objects themselves are immutable and they
are allocated separately on the heap and so don't have their address
change. It also requires a change I made the last time I tried to debug
this problem which removed the ability to de-register a pass from the
registry. This patch creates a single access path to these objects
inside the PMTopLevelManager which memoizes the result of querying the
registry. This is somewhat gross as I don't really know if
PMTopLevelManager is the *right* place to put it, and I dislike using
a mutable member to memoize things, but it seems to work.
For long-lived pass managers this should completely eliminate
the cost of acquiring locks to look into the pass registry once the
memoized cache is warm. For 'ninja check' I measured about 1.5%
reduction in CPU time and in total time on a machine with 32 hardware
threads. For normal compilation, I don't know how much this will help,
sadly. We will still pay the cost while we populate the memoized cache.
I don't think it will hurt though, and for LTO or compiles with many
small functions it should still be a win. However, for tight loops
around a pass manager with many passes and small modules, this will help
tremendously. On the AArch64 backend I saw nearly 50% reductions in time
to complete 2000 cycles of spinning up and tearing down the pipeline.
Measurements from Owen of an actual long-lived pass manager show more
along the lines of 10% improvements.
Differential Revision: http://reviews.llvm.org/D7213
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227299 91177308-0d34-0410-b5e6-96231b3b80d8
This change reverts the interesting parts of 226311 (and 227046). This change introduced two problems, and I've been convinced that an alternate approach is preferrable anyways.
The bugs were:
- Registery appears to require all users be within the same linkage unit. After this change, asking for "statepoint-example" in Transform/ would sometimes get you nullptr, whereas asking the same question in CodeGen would return the right GCStrategy. The correct long term fix is to get rid of the utter hack which is Registry, but I don't have time for that right now. 227046 appears to have been an attempt to fix this, but I don't believe it does so completely.
- GCMetadataPrinter::finishAssembly was being called more than once per GCStrategy. Each Strategy was being added to the GCModuleInfo multiple times.
Once I get time again, I'm going to split GCModuleInfo into the gc.root specific part and a GCStrategy owning Analysis pass. I'm probably also going to kill off the Registry. Once that's done, I'll move the new GCStrategyAnalysis and all built in GCStrategies into Analysis. (As original suggested by Chandler.) This will accomplish my original goal of being able to access GCStrategy from Transform/ without adding all of the builtin GCs to IR/.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@227109 91177308-0d34-0410-b5e6-96231b3b80d8
These things are potentially used for non-DWARF data (see the discussion
in PR22235), so take the `Dwarf` out of the name. Since the new name
gives fewer clues, update the doxygen to properly describe what they
are.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226874 91177308-0d34-0410-b5e6-96231b3b80d8
During `MDNode::deleteTemporary()`, call `replaceAllUsesWith(nullptr)`
to update all tracking references to `nullptr`.
This fixes PR22280, where inverted destruction order between tracking
references and the temporaries themselves caused a use-after-free in
`LLParser`.
An alternative fix would be to add an assertion that there are no users,
and continue to fix inverted destruction order in clients (like
`LLParser`), but instead I decided to make getting-teardown-right easy.
(If someone disagrees let me know.)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226866 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Some parsers need references back to the option they are members of. This is used for handling the argument string as well as by the various pass name parsers for making pass names into flags.
Making parsers that need to refer back to the option have a reference to the option eliminates some of the members of various parsers, and enables further code cleanup.
Reviewers: dexonsmith
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D7131
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226864 91177308-0d34-0410-b5e6-96231b3b80d8
Specifically, gc.result benefits from this greatly. Instead of:
gc.result.int.*
gc.result.float.*
gc.result.ptr.*
...
We now have a gc.result.* that can specialize to literally any type.
Differential Revision: http://reviews.llvm.org/D7020
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226857 91177308-0d34-0410-b5e6-96231b3b80d8
With the appropriate Verifier changes, exactracting the result out of a
statepoint wrapping a vararg function crashes. However, a void vararg
function works fine: commit this first step.
Differential Revision: http://reviews.llvm.org/D7071
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226599 91177308-0d34-0410-b5e6-96231b3b80d8
ConstantArrays constructed during linking can cause quadratic memory
explosion. An example is the ConstantArrays constructed when linking in
GlobalVariables with appending linkage.
Releasing all unused constants can cause a 20% LTO compile-time
slowdown for a large application. So this commit releases unused ConstantArrays
only.
rdar://19040716. It reduces memory footprint from 20+G to 6+G.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226592 91177308-0d34-0410-b5e6-96231b3b80d8
This reverts commit r226542, effectively reapplying r226540. This time,
initialize `IsEmpty` in the copy and move constructors as well.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226545 91177308-0d34-0410-b5e6-96231b3b80d8
Now that the clone methods used by `MapMetadata()` don't do any
remapping (and return a temporary), they make more sense as member
functions on `MDNode` (and subclasses).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226541 91177308-0d34-0410-b5e6-96231b3b80d8
Change `HeaderBuilder` API to work well even when it's not starting with
a tag. There's already one case like this, and the tag is moving
elsewhere as part of PR22235.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226540 91177308-0d34-0410-b5e6-96231b3b80d8
Rather than relying on updating switch statements correctly, detect
whether `setHash()` exists in the subclass. If so, call
`recalculateHash()` and `setHash(0)` appropriately.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226531 91177308-0d34-0410-b5e6-96231b3b80d8
As part of PR22235, introduce `DwarfNode` and `GenericDwarfNode`. The
former is a metadata node with a DWARF tag. The latter matches our
current (generic) schema of a header with string (and stringified
integer) data and an arbitrary number of operands.
This doesn't move it into place yet; that change will require a large
number of testcase updates.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226529 91177308-0d34-0410-b5e6-96231b3b80d8
Swap usage of `SubclassData32` and `MDNodeSubclassData`, and rename
`MDNodeSubclassData` to `NumUnresolved`. Small drive-by cleanup to
`countUnresolvedOperands()` since otherwise the name clash with local
vars named `NumUnresolved` would be confusing.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226523 91177308-0d34-0410-b5e6-96231b3b80d8
As pointed out in r226501, the distinction between `MDNode` and
`UniquableMDNode` is confusing. When we need subclasses of `MDNode`
that don't use all its functionality it might make sense to break it
apart again, but until then this makes the code clearer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226520 91177308-0d34-0410-b5e6-96231b3b80d8
Add `MDNode::replaceWithUniqued()` and `MDNode::replaceWithDistinct()`,
which mutate temporary nodes to become uniqued or distinct. On uniquing
collisions, the unique version is returned and the node is deleted.
This takes advantage of temporary nodes being folded back in, and should
let me clean up some awkward logic in `MapMetadata()`.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226510 91177308-0d34-0410-b5e6-96231b3b80d8
Change `MDTuple::getTemporary()` and `MDLocation::getTemporary()` to
return (effectively) `std::unique_ptr<T, MDNode::deleteTemporary>`, and
clean up call sites. (For now, `DIBuilder` call sites just call
`release()` immediately.)
There's an accompanying change in each of clang and polly to use the new
API.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@226504 91177308-0d34-0410-b5e6-96231b3b80d8
