- Use a DenseSet instead of a FoldingSet to cache
canonicalized nodes. This reduces the overhead
of double-hashing.
- Use reference counts in ImutAVLTree to much
more aggressively recover tree nodes that are
no longer usable. We can generate many
transient nodes while using add() and remove()
on ImmutableSet/ImmutableMaps to generate a final
set/map.
For the clang static analyzer (the main client
of these data structures), this results in
a slight speedup (0.5%) when analyzing sqlite3,
but much more importantly results in a 30-60%
reduction in peak memory usage when the analyzer
is analyzing a given function in a file. On
average that's about a ** 44% reduction ** in the
memory footprint of the static analyzer.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@120459 91177308-0d34-0410-b5e6-96231b3b80d8
Sometimes std::copy can become a memmove call, and that is not a good idea when
copying relatively few bytes as we are doing. We also get a small win by
changing two loops into one.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@120265 91177308-0d34-0410-b5e6-96231b3b80d8
We always disallowed overlapping inserts with different values, and this makes
the insertion code smaller and faster.
If an overwriting insert is needed, it can be added as a separate method that
trims any existing intervals before inserting. The immediate use cases for
IntervalMap don't need this - they only use disjoint insertions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@120264 91177308-0d34-0410-b5e6-96231b3b80d8
These iterators don't point anywhere, and they can't be compared to anything.
They are only good for assigning to.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@120239 91177308-0d34-0410-b5e6-96231b3b80d8
Implement iterator::erase() in a simple version that erases nodes when they
become empty, but doesn't try to redistribute elements among siblings for better
packing.
Handle coalescing across leaf nodes which may require erasing entries.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@120226 91177308-0d34-0410-b5e6-96231b3b80d8
situations where on the common path an API can return a pointer to some direct
memory, but which on an exceptional path may need to return a pointer to a
temporary struct.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@120201 91177308-0d34-0410-b5e6-96231b3b80d8
- Double the vector's capacity when growing to avoid unneeccesary reallocation.
- Do the reallocation with realloc(3) which can expand the memory in place.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@120183 91177308-0d34-0410-b5e6-96231b3b80d8
The path also holds a reference to the root node, and that allows important
iterator accessors like start() and stop() to have no conditional code. (When
the compiler is clever enough to remove it.)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@120165 91177308-0d34-0410-b5e6-96231b3b80d8
to use lowercase letters for the start of most
method names and to replace some method names
with more descriptive names (e.g., "getLeft()"
instead of "Left()"). No real functionality
change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@120070 91177308-0d34-0410-b5e6-96231b3b80d8
This doesn't quite work yet because the calls to treeDecrement and treeIncrement
operate at the leaf level, not on pathNode(Level) as required.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@120068 91177308-0d34-0410-b5e6-96231b3b80d8
It is now possible to navigate the B+-tree using NodeRef::subtree() and
NodeRef::size() without knowing the key and value template types used in the
tree.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@119880 91177308-0d34-0410-b5e6-96231b3b80d8
that the noderefs are the first member in the object.
This is in preparation of detemplatization of tree navigation.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@119879 91177308-0d34-0410-b5e6-96231b3b80d8
Key and value objects may not be destructed instantly when they are erased from
the container, but they will be destructed eventually by the IntervalMap
destructor.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@119873 91177308-0d34-0410-b5e6-96231b3b80d8
llvm/include/llvm/ADT/IntervalMap.h:334: error: '((llvm::IntervalMapImpl::DesiredNodeBytes / static_cast<unsigned int>(((2 * sizeof (KeyT)) + sizeof (ValT)))) >? 3u)' is not a valid template argument for type 'unsigned int' because it is a non-constant expression
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@119790 91177308-0d34-0410-b5e6-96231b3b80d8
This is a sorted interval map data structure for small keys and values with
automatic coalescing and bidirectional iteration over coalesced intervals.
Except for coalescing intervals, it provides similar functionality to std::map.
It is however much more compact for small keys and values, and hopefully faster
too.
The container object itself can hold the first few intervals without any
allocations, then it switches to a cache conscious B+-tree representation. A
recycling allocator can be shared between many containers, even between
containers holding different types.
The IntervalMap is initially intended to be used with SlotIndex intervals for:
- Backing store for LiveIntervalUnion that is smaller and faster than std::set.
- Backing store for LiveInterval with less overhead than std::vector for typical
intervals and O(N log N) merging of large intervals. 99% of virtual registers
need 4 entries or less and would benefit from the small object optimization.
- Backing store for LiveDebugVariable which doesn't exist yet, but will track
debug variables during register allocation.
This is a work in progress. Missing items are:
- Performance metrics.
- erase().
- insert() shrinkage.
- clear().
- More performance metrics.
- Simplification and detemplatization.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@119787 91177308-0d34-0410-b5e6-96231b3b80d8
This is a sorted interval map data structure for small keys and values with
automatic coalescing and bidirectional iteration over coalesced intervals.
Except for coalescing intervals, it provides similar functionality to std::map.
It is however much more compact for small keys and values, and hopefully faster
too.
The container object itself can hold the first few intervals without any
allocations, then it switches to a cache conscious B+-tree representation. A
recycling allocator can be shared between many containers, even between
containers holding different types.
The IntervalMap is initially intended to be used with SlotIndex intervals for:
- Backing store for LiveIntervalUnion that is smaller and faster than std::set.
- Backing store for LiveInterval with less overhead than std::vector for typical
intervals and O(N log N) merging of large intervals. 99% of virtual registers
need 4 entries or less and would benefit from the small object optimization.
- Backing store for LiveDebugVariable which doesn't exist yet, but will track
debug variables during register allocation.
This is a work in progress. Missing items are:
- Performance metrics.
- erase().
- insert() shrinkage.
- clear().
- More performance metrics.
- Simplification and detemplatization.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@119772 91177308-0d34-0410-b5e6-96231b3b80d8
specializations provided here. This is a little annoying because its size
changes from platform to platform. If possible, I may follow up with a patch
that uses standard constants to simplify much of this, but assuming for now
that was avoided for a reason.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@117880 91177308-0d34-0410-b5e6-96231b3b80d8
logic to use the new APInt methods. Among other things this
implements rdar://8501501 - llvm.smul.with.overflow.i32 should constant fold
which comes from "clang -ftrapv", originally brought to my attention from PR8221.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@116457 91177308-0d34-0410-b5e6-96231b3b80d8
is partly because this attribute caused trouble in the past (the
SmallVector one had to be changed from aligned to aligned(8) due
to causing crashes on i386 for example; in theory the same might
be needed in the Allocator case...). But it's mostly because
there seems to be no point in special casing gcc here. Using the
same implementation for all compilers results in better testing.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@115462 91177308-0d34-0410-b5e6-96231b3b80d8
capacity and remove the workaround in SmallVector<T,0>. There are some
theoretical benefits to a N->2N+1 growth policy anyway.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@112870 91177308-0d34-0410-b5e6-96231b3b80d8
I think there are good reasons to change this, but in the interests
of short-term stability, make SmallVector<...,0> reserve non-zero
capacity in its constructors. This means that SmallVector<...,0>
uses more memory than SmallVector<...,1> and should really only be
used (unless/until this workaround is removed) by clients that
care about using SmallVector with an incomplete type.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@112147 91177308-0d34-0410-b5e6-96231b3b80d8
- Cache used characters in a bitset to reduce memory overhead to just 32 bytes.
- On my core2 this code is faster except when the checked string was very short
(smaller than the list of delimiters).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@111817 91177308-0d34-0410-b5e6-96231b3b80d8
- Respect find_first_of(char's From parameter instead of silently dropping it.
- Prefer std::string() to std::string("")
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@111814 91177308-0d34-0410-b5e6-96231b3b80d8
We must complete the DFS, otherwise we might miss needed phi-defs, and
prematurely color live ranges with a non-dominating value.
This is not a big deal since we get to color more of the CFG and the next
mapValue call will be faster.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@111397 91177308-0d34-0410-b5e6-96231b3b80d8
implementations of equality comparison and hash computation. This
can be used to optimize node lookup by avoiding creating lots of
temporary ID values just for hashing and comparison purposes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@111130 91177308-0d34-0410-b5e6-96231b3b80d8
target triple and straightens it out. This does less than gcc's script
config.sub, for example it turns i386-mingw32 into i386--mingw32 not
i386-pc-mingw32, but it does a decent job of turning funky triples into
something that the rest of the Triple class can understand. The plan
is to use this to canonicalize triple's when they are first provided
by users, and have the rest of LLVM only deal with canonical triples.
Once this is done the special case workarounds in the Triple constructor
can be removed, making the class more regular and easier to use. The
comments and unittests for the Triple class are already adjusted in this
patch appropriately for this brave new world of increased uniformity.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@110909 91177308-0d34-0410-b5e6-96231b3b80d8
handles with a pointer to the containing map. When a map is copied, these
pointers need to be corrected to point to the new map. If not, then consider
the case of a map M1 which maps a value V to something. Create a copy M2 of
M1. At this point there are two value handles on V, one representing V as a
key in M1, the other representing V as a key in M2. But both value handles
point to M1 as the containing map. Now delete V. The value handles remove
themselves from their containing map (which destroys them), but only the first
value handle is successful: the second one cannot remove itself from M1 as
(once the first one has removed itself) there is nothing there to remove; it
is therefore not destroyed. This causes an assertion failure "All references
to V were not removed?".
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@109851 91177308-0d34-0410-b5e6-96231b3b80d8
with an existing allocator. The interesting use case of this
is that it allows "StringMap<whatever, BumpPtrAllocator&>" for
when you want to allocate out of a preexisting bump pointer
allocator owned by someone else.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@109213 91177308-0d34-0410-b5e6-96231b3b80d8
replaced by a bigger array in SmallPtrSet (by overridding it), instead just use a
pointer to the start of the storage, and have SmallPtrSet pass in the value to use.
This has the disadvantage that SmallPtrSet becomes bigger by one pointer. It has
the advantage that it no longer uses tricky C++ rules, and is clearly correct while
I'm not sure the previous version was. This was inspired by g++-4.6 pointing out
that SmallPtrSetImpl was writing off the end of SmallArray, which it was. Since
SmallArray is replaced with a bigger array in SmallPtrSet, the write was still to
valid memory. But it was writing off the end of the declared array type - sounds
kind of dubious to me, like it sounded dubious to g++-4.6. Maybe g++-4.6 is wrong
and this construct is perfectly valid and correctly compiled by all compilers, but
I think it is better to avoid the whole can of worms by avoiding this construct.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107285 91177308-0d34-0410-b5e6-96231b3b80d8
SmallArray[SmallSize] in the SmallPtrSetIteratorImpl, and this is
one off the end of the array. For those who care, right now gcc
warns about writing off the end because it is confused about the
declaration of SmallArray as having length 1 in the parent class
SmallPtrSetIteratorImpl. However if you tweak code to unconfuse
it, then it still warns about writing off the end of the array,
because of this buffer overflow. In short, even with this fix
gcc-4.6 will warn about writing off the end of the array, but now
that is only because it is confused.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@107200 91177308-0d34-0410-b5e6-96231b3b80d8
the Profile method. Currently this only works with the default FoldingSetTraits
implementation.
The point of this is to allow nodes to not store context values which are only
used during profiling. A better solution would thread this value through the
folding algorithms, but then those would need to be (1) templated and
(2) non-opaque.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@105819 91177308-0d34-0410-b5e6-96231b3b80d8
'class llvm::DAGDeltaAlgorithm' has virtual functions and accessible non-virtual destructor
Not sure if this is the best solution, but this class has state and some of the
classes that inherit from it also do, so it looks appropriate.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@105675 91177308-0d34-0410-b5e6-96231b3b80d8
- This can give substantial speedups in the delta process for inputs we can construct dependency information for.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@105612 91177308-0d34-0410-b5e6-96231b3b80d8
This means that our Registers are now ordered R7, R8, R9, R10, R12, ...
Not R1, R10, R11, R12, R2, R3, ...
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@104745 91177308-0d34-0410-b5e6-96231b3b80d8
- This provides a convenient alternative to using something llvm::prior or
manual iterator access, for example::
if (T *Prev = foo->getPrevNode())
...
instead of::
iterator it(foo);
if (it != begin()) {
--it;
...
}
- Chris, please review.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@103647 91177308-0d34-0410-b5e6-96231b3b80d8
Microoptimize Twine's with unsigned and int to not pin their value to
the stack. This saves stack space in common cases and allows mem2reg
in the caller. A simple example is:
void foo(const Twine &);
void bar(int x) {
foo("xyz: " + Twine(x));
}
Before:
__Z3bari:
subq $40, %rsp
movl %edi, 36(%rsp)
leaq L_.str3(%rip), %rax
leaq 36(%rsp), %rcx
leaq 8(%rsp), %rdi
movq %rax, 8(%rsp)
movq %rcx, 16(%rsp)
movb $3, 24(%rsp)
movb $7, 25(%rsp)
callq __Z3fooRKN4llvm5TwineE
addq $40, %rsp
ret
After:
__Z3bari:
subq $24, %rsp
leaq L_.str3(%rip), %rax
movq %rax, (%rsp)
movslq %edi, %rax
movq %rax, 8(%rsp)
movb $3, 16(%rsp)
movb $7, 17(%rsp)
leaq (%rsp), %rdi
callq __Z3fooRKN4llvm5TwineE
addq $24, %rsp
ret
It saves 16 bytes of stack and one instruction in this case.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@103107 91177308-0d34-0410-b5e6-96231b3b80d8
Limit alignment in SmallVector 8, otherwise GCC assumes 16 byte alignment.
opetaror new, and malloc only return 8-byte aligned memory on 32-bit Linux,
which cause a crash if code is compiled with -O3 (or -ftree-vectorize) and some
SmallVector code is vectorized.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@102604 91177308-0d34-0410-b5e6-96231b3b80d8
CGSCC can delete nodes in regions of the callgraph that
have already been visited. If new CG nodes are allocated
to the same pointer, we shouldn't abort, just handle it
correctly by assigning a new number. This should restore
stability by removing invalidated pointers that *will* be
reused from the densemap in the iterator.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@101628 91177308-0d34-0410-b5e6-96231b3b80d8
to keep the node entries in scc_iterator up to date instead of dangling as
the SCC mutates.
This is a really terrible problem which was causing -g to affect codegen
because it would permute the memory image of the compiler process.
Thanks to Dale for expertly hunting it down.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@101565 91177308-0d34-0410-b5e6-96231b3b80d8
ownership over the pointer it contains. Useful when we want to
communicate ownership while still having several clients holding on to
the same pointer *without* introducing reference counting.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@100463 91177308-0d34-0410-b5e6-96231b3b80d8
passing the command-line parameter "-stats" and to print the resulting
statistics without calling llvm_shutdown().
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@99893 91177308-0d34-0410-b5e6-96231b3b80d8
doesn't apply to the type, only to the variable, so subsequent uses
of U which expect it to be aligned weren't actually aligned.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@98843 91177308-0d34-0410-b5e6-96231b3b80d8
BumpPtrAllocator-allocated region to allow it to be stored in a more
compact form and to avoid the need for a non-trivial destructor call.
Use this new mechanism in ScalarEvolution instead of
FastFoldingSetNode to avoid leaking memory in the case where a
FoldingSetNodeID uses heap storage, and to reduce overall memory
usage.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@98829 91177308-0d34-0410-b5e6-96231b3b80d8
It gets its own implementation totally divorced from the (presumably
performance-sensitive) routines which parse into a uint64_t.
Add APInt::operator|=(uint64_t), which is situationally much better than
using a full APInt.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@97381 91177308-0d34-0410-b5e6-96231b3b80d8
payloads. APFloat's internal folding routines always make QNaNs now,
instead of sometimes making QNaNs and sometimes SNaNs depending on the
type.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@97364 91177308-0d34-0410-b5e6-96231b3b80d8
and use them to avoid a copy of a string in getNameWithPrefix in
the common case. It seems like Value::setName and other places
should use this as well?
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@93301 91177308-0d34-0410-b5e6-96231b3b80d8
- getToken is modeled after StringRef::split but it can split on multiple
separator chars and skips leading seperators.
- SplitString is a StringRef::split variant for more than 2 elements with the
same behaviour as getToken.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@93161 91177308-0d34-0410-b5e6-96231b3b80d8
a single pointer (PointerIntPair) member. In "small" mode, the
pointer field is reinterpreted as a set of bits. In "large" mode,
the pointer points to a heap-allocated object.
Also, give BitVector empty and swap functions.
And, add some simple unittests for BitVector and SmallBitVector.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@92730 91177308-0d34-0410-b5e6-96231b3b80d8
smallest-normalized-magnitude values in a given FP semantics.
Provide an APFloat-to-string conversion which I am quite ready to admit could
be much more efficient.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@92126 91177308-0d34-0410-b5e6-96231b3b80d8
SmallVectorTemplateBase class, which allows us to statically
dispatch on isPodLike instead of dynamically.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@91523 91177308-0d34-0410-b5e6-96231b3b80d8
down into SmallVectorImpl. This requires sprinking a ton of this->'s in,
but gives us a place to factor.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@91522 91177308-0d34-0410-b5e6-96231b3b80d8
and there is a new SmallVectorTemplateBase class in between it and SmallVectorImpl.
SmallVectorTemplateBase can be specialized based on isPodLike.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@91518 91177308-0d34-0410-b5e6-96231b3b80d8
1. Use std::equal instead of reinventing it.
2. don't run dtors in destroy_range if element is pod-like.
3. Use isPodLike to decide between memcpy/uninitialized_copy
instead of is_class. isPodLike is more generous in some cases.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@91427 91177308-0d34-0410-b5e6-96231b3b80d8
isPodLike type trait. This is a generally useful type trait for
more than just DenseMap, and we really care about whether something
acts like a pod, not whether it really is a pod.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@91421 91177308-0d34-0410-b5e6-96231b3b80d8
This change removes the DefaultConstructible
and CopyAssignable constraints on the template
parameter T (the first one).
The second template parameter (R) is defaulted to be
identical to the first and controls the result type.
By specifying it to be (const T&) additionally the
CopyConstructible constraint on T can be removed.
This allows to use StringSwitch e.g. for llvm::Constant
instances.
Regarding the other review feedback regarding performance
because of taking pointers, this class should be completely
optimizable like before, since all methods are inline and
the pointer dereferencing and result value caching should be
possible behind the scenes by the "as-if" rule.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@91123 91177308-0d34-0410-b5e6-96231b3b80d8
- This is a pretty slow / memory intensive implementation, and I will likely
change it to an iterative model, but it works.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@90447 91177308-0d34-0410-b5e6-96231b3b80d8
4.2.4, 4.3.4, 4.4.2.
The workaround is to use a local min/max implementation that takes an integer
param, and not a reference to integer param (like std::min does).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@89352 91177308-0d34-0410-b5e6-96231b3b80d8
2. Allow SCCIterator to work with inverse graphs.
3. Fix an incorrect comment in GraphTraits.h (the type in the comment
was given as GraphType* when it is actually const GraphType &).
Patch by Patrick Alexander Simmons.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@89091 91177308-0d34-0410-b5e6-96231b3b80d8
This patch forbids implicit conversion of DenseMap::const_iterator to
DenseMap::iterator which was possible because DenseMapIterator inherited
(publicly) from DenseMapConstIterator. Conversion the other way around is now
allowed as one may expect.
The template DenseMapConstIterator is removed and the template parameter
IsConst which specifies whether the iterator is constant is added to
DenseMapIterator.
Actually IsConst parameter is not necessary since the constness can be
determined from KeyT but this is not relevant to the fix and can be addressed
later.
Patch by Victor Zverovich!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@86636 91177308-0d34-0410-b5e6-96231b3b80d8
