Add API for specifying which `LLVMContext` each `lto_module_t` and
`lto_code_gen_t` is in.
In particular, this enables the following flow:
for (auto &File : Files) {
lto_module_t M = lto_module_create_in_local_context(File...);
querySymbols(M);
lto_module_dispose(M);
}
lto_code_gen_t CG = lto_codegen_create_in_local_context();
for (auto &File : FilesToLink) {
lto_module_t M = lto_module_create_in_codegen_context(File..., CG);
lto_codegen_add_module(CG, M);
lto_module_dispose(M);
}
lto_codegen_compile(CG);
lto_codegen_write_merged_modules(CG, ...);
lto_codegen_dispose(CG);
This flow has a few benefits.
- Only one module (two if you count the combined module in the code
generator) is in memory at a time.
- Metadata (and constants) from files that are parsed to query symbols
but not linked into the code generator don't pollute the global
context.
- The first for loop can be parallelized, since each module is in its
own context.
- When the code generator is disposed, the memory from LTO gets freed.
rdar://problem/18767512
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@221733 91177308-0d34-0410-b5e6-96231b3b80d8
We used to always vectorize (slp and loop vectorize) in the LTO pass pipeline.
r220345 changed it so that we used the PassManager's fields 'LoopVectorize' and
'SLPVectorize' out of the desire to be able to disable vectorization using the
cl::opt flags 'vectorize-loops'/'slp-vectorize' which the before mentioned
fields default to.
Unfortunately, this turns off vectorization because those fields
default to false.
This commit adds flags to the LTO library to disable lto vectorization which
reconciles the desire to optionally disable vectorization during LTO and
the desired behavior of defaulting to enabled vectorization.
We really want tools to set PassManager flags directly to enable/disable
vectorization and not go the route via cl::opt flags *in*
PassManagerBuilder.cpp.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@220652 91177308-0d34-0410-b5e6-96231b3b80d8
files in this directory. If it should be defined anywhere, it should be defined
when building lib/LTO/LTOCodeGenerator.cpp, but we've not had it defined there
for quite some time, so that doesn't really seem to be very important. (It also
would slow down the modules build by creating extra module variants.)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@218544 91177308-0d34-0410-b5e6-96231b3b80d8
We want to encourage users of the C++ LTO API to reuse memory buffers instead
of repeatedly opening and reading the same file contents.
This reverts commit r212305 and implements a tidier scheme.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212308 91177308-0d34-0410-b5e6-96231b3b80d8
Any uses of tools/lto as a static lib should probably move to lib/LTO.
This was also never implemented in the configure build, so this reduces
the differences among the two.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211852 91177308-0d34-0410-b5e6-96231b3b80d8
This adds support for an -mattr option to the gold plugin and to llvm-lto. This
allows the caller to specify details of the subtarget architecture, like +aes,
or +ssse3 on x86. Note that this requires a change to the include/llvm-c/lto.h
interface: it adds a function lto_codegen_set_attr and it increments the
version of the interface.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@207279 91177308-0d34-0410-b5e6-96231b3b80d8
This centralizes the Makefile handling of -install_name and -rpath. It also
moves the cmake build to using @rpath. The reason being that libclang needs it,
and it works for everything else.
A followup patch will move clang to using this and then there will be a single
point to edit to support other systems.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202499 91177308-0d34-0410-b5e6-96231b3b80d8
The same code (~20 lines) for initializing a TargetOptions object from CodeGen
cmdline flags is duplicated 4 times in 4 different tools. This patch moves it
into a utility function.
Since the CodeGen/CommandFlags.h file defines cl::opt flags in a header, it's
a bit of a touchy situation because we should only link them into tools. So this
patch puts the init function in the header.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201699 91177308-0d34-0410-b5e6-96231b3b80d8
This function adds an extra path argument to lto_module_create_from_memory.
The path argument will be passed to makeBuffer to make sure the MemoryBuffer
has a name and the created module has a module identifier.
This is mainly for emitting warning messages from the linker. When we emit
warning message on a module, we can use the module identifier.
rdar://15985737
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201114 91177308-0d34-0410-b5e6-96231b3b80d8
Add a hook in the C API of LTO so that clients of the code generator can set
their own handler for the LLVM diagnostics.
The handler is defined like this:
typedef void (*lto_diagnostic_handler_t)(lto_codegen_diagnostic_severity_t
severity, const char *diag, void *ctxt)
- severity says how bad this is.
- diag is a string that contains the diagnostic message.
- ctxt is the registered context for this handler.
This hook is more general than the lto_get_error_message, since this function
keeps only the latest message and can only be queried when something went wrong
(no warning for instance).
<rdar://problem/15517596>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199338 91177308-0d34-0410-b5e6-96231b3b80d8
Reapply r199191, reverted in r199197 because it carelessly broke
Other/link-opts.ll. The problem was that calling
createInternalizePass("main") would select
createInternalizePass(bool("main")) instead of
createInternalizePass(ArrayRef<const char *>("main")). This commit
fixes the bug.
The original commit message follows.
Add API to LTOCodeGenerator to specify a strategy for the -internalize
pass.
This is a new attempt at Bill's change in r185882, which he reverted in
r188029 due to problems with the gold linker. This puts the onus on the
linker to decide whether (and what) to internalize.
In particular, running internalize before outputting an object file may
change a 'weak' symbol into an internal one, even though that symbol
could be needed by an external object file --- e.g., with arclite.
This patch enables three strategies:
- LTO_INTERNALIZE_FULL: the default (and the old behaviour).
- LTO_INTERNALIZE_NONE: skip -internalize.
- LTO_INTERNALIZE_HIDDEN: only -internalize symbols with hidden
visibility.
LTO_INTERNALIZE_FULL should be used when linking an executable.
Outputting an object file (e.g., via ld -r) is more complicated, and
depends on whether hidden symbols should be internalized. E.g., for
ld -r, LTO_INTERNALIZE_NONE can be used when -keep_private_externs, and
LTO_INTERNALIZE_HIDDEN can be used otherwise. However,
LTO_INTERNALIZE_FULL is inappropriate, since the output object file will
eventually need to link with others.
lto_codegen_set_internalize_strategy() sets the strategy for subsequent
calls to lto_codegen_write_merged_modules() and lto_codegen_compile*().
<rdar://problem/14334895>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199244 91177308-0d34-0410-b5e6-96231b3b80d8
Add API to LTOCodeGenerator to specify a strategy for the -internalize
pass.
This is a new attempt at Bill's change in r185882, which he reverted in
r188029 due to problems with the gold linker. This puts the onus on the
linker to decide whether (and what) to internalize.
In particular, running internalize before outputting an object file may
change a 'weak' symbol into an internal one, even though that symbol
could be needed by an external object file --- e.g., with arclite.
This patch enables three strategies:
- LTO_INTERNALIZE_FULL: the default (and the old behaviour).
- LTO_INTERNALIZE_NONE: skip -internalize.
- LTO_INTERNALIZE_HIDDEN: only -internalize symbols with hidden
visibility.
LTO_INTERNALIZE_FULL should be used when linking an executable.
Outputting an object file (e.g., via ld -r) is more complicated, and
depends on whether hidden symbols should be internalized. E.g., for
ld -r, LTO_INTERNALIZE_NONE can be used when -keep_private_externs, and
LTO_INTERNALIZE_HIDDEN can be used otherwise. However,
LTO_INTERNALIZE_FULL is inappropriate, since the output object file will
eventually need to link with others.
lto_codegen_set_internalize_strategy() sets the strategy for subsequent
calls to lto_codegen_write_merged_modules() and lto_codegen_compile*().
<rdar://problem/14334895>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199191 91177308-0d34-0410-b5e6-96231b3b80d8
The cmake build didn't support EXPORTED_SYMBOL_FILE. Instead, it had a
Windows-only implementation in tools/lto/CMakeLists.txt, a linux-only
implementation in tools/gold/CMakeLists.txt, and a darwin-only implementation
in tools/clang/tools/libclang/CMakeLists.txt.
This attempts to consolidate these one-offs into a single place. Clients can now
just set LLVM_EXPORTED_SYMBOL_FILE and things (hopefully) Just Work, like in
the make build.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@198136 91177308-0d34-0410-b5e6-96231b3b80d8
This reduces the size of clang-format from 22 MB to 1.8 MB, diagtool goes from
21 MB to 2.8 MB, libclang.so goes from 29 MB to 20 MB, etc. The size of the
bin/ folder shrinks from 270 MB to 200 MB.
Targets that support plugins and don't already use EXPORTED_SYMBOL_FILE
(which libclang and libLTO already do) can set NO_DEAD_STRIP to opt out.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@198087 91177308-0d34-0410-b5e6-96231b3b80d8
There are two ways one could implement hiding of linkonce_odr symbols in LTO:
* LLVM tells the linker which symbols can be hidden if not used from native
files.
* The linker tells LLVM which symbols are not used from other object files,
but will be put in the dso symbol table if present.
GOLD's API is the second option. It was implemented almost 1:1 in llvm by
passing the list down to internalize.
LLVM already had partial support for the first option. It is also very similar
to how ld64 handles hiding these symbols when *not* doing LTO.
This patch then
* removes the APIs for the DSO list.
* marks LTO_SYMBOL_SCOPE_DEFAULT_CAN_BE_HIDDEN all linkonce_odr unnamed_addr
global values and other linkonce_odr whose address is not used.
* makes the gold plugin responsible for handling the API mismatch.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193800 91177308-0d34-0410-b5e6-96231b3b80d8
This reverts commit r193255 and instead creates an lto_bool_t typedef
that points to bool, _Bool, or unsigned char depending on what is
available. Only recent versions of MSVC provide a stdbool.h header.
Reviewers: rafael.espindola
Differential Revision: http://llvm-reviews.chandlerc.com/D2019
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193377 91177308-0d34-0410-b5e6-96231b3b80d8
Generalize the API so we can distinguish symbols that are needed just for a DSO
symbol table from those that are used from some native .o.
The symbols that are only wanted for the dso symbol table can be dropped if
llvm can prove every other dso has a copy (linkonce_odr) and the address is not
important (unnamed_addr).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191922 91177308-0d34-0410-b5e6-96231b3b80d8
This was broken when options were moved up in r191680. No test because this is
specific LLVMgold.so/libLTO.so.
Patch by Tom Roeder!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191829 91177308-0d34-0410-b5e6-96231b3b80d8
Enable building the LTO library (.lib and.dll) and llvm-lto.exe on Windows with
MSVC and Mingw as well as re-enabling the associated test.
Patch by Greg Bedwell!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191823 91177308-0d34-0410-b5e6-96231b3b80d8
Enable building the LTO library (.lib and.dll) and llvm-lto.exe on Windows with
MSVC and Mingw as well as re-enabling the associated test.
Patch by Greg Bedwell!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191670 91177308-0d34-0410-b5e6-96231b3b80d8
