Commit Graph

144 Commits

Author SHA1 Message Date
Rafael Espindola
3fd7e37b34 Rewrite the gold plugin to fix pr19901.
There is a fundamental difference between how the gold API and lib/LTO view
the LTO process.

The gold API talks about a particular symbol in a particular file. The lib/LTO
API talks about a symbol in the merged module.

The merged module is then defined in terms of the IR semantics. In particular,
a linkonce_odr GV is only copied if it is used, since it is valid to drop
unused linkonce_odr GVs.

In the testcase in pr19901 both properties collide. What happens is that gold
asks us to keep a particular linkonce_odr symbol, but the IR linker doesn't
copy it to the merged module and we never have a chance to ask lib/LTO to keep
it.

This patch fixes it by having a more direct implementation of the gold API. If
it asks us to keep a symbol, we change the linkage so it is not linkonce. If it
says we can drop a symbol, we do so. All of this before we even send the module
to lib/Linker.

Since now we don't have to produce LTO_SYMBOL_SCOPE_DEFAULT_CAN_BE_HIDDEN,
during symbol resolution we can use a temporary LLVMContext and do lazy
module loading. This allows us to keep the minimum possible amount of
allocated memory around. This should also allow as much parallelism as
we want, since there is no shared context.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@216215 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-21 20:28:55 +00:00
Rafael Espindola
b2d2f28351 Fix using -plugin-opt=apiflie when also using -plugin-opt=emit-llvm.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215378 91177308-0d34-0410-b5e6-96231b3b80d8
2014-08-11 19:06:54 +00:00
Rafael Espindola
72c5355bbe Remove unused argument.
Thanks to Justin Bogner for noticing it.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214426 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-31 19:32:04 +00:00
Rafael Espindola
562fd7a830 Use range loops.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214280 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-30 01:52:40 +00:00
Rafael Espindola
3bff0d2b95 Convert a few more function pointer calls to just "f()".
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214276 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-30 01:36:32 +00:00
Rafael Espindola
fe580aaa2b Don't duplicate the function name in a comment.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214272 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-30 01:23:45 +00:00
Rafael Espindola
601bb093ef Use function pointers with just f(1,2) instead of (*f)(1,2).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214265 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-30 00:38:58 +00:00
Rafael Espindola
3c08473236 Use nullptr instead of NULL.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214229 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-29 21:46:05 +00:00
Rafael Espindola
9a3b1ebcb1 Remove unused includes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214226 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-29 21:38:05 +00:00
Rafael Espindola
ee7f2db4d0 Delete dead code.
The gold plugin doesn't call lseek or read directly any more.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214221 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-29 21:09:43 +00:00
Rafael Espindola
df7568e9f3 gold plugin: Fix handling of corrupted bitcode files.
We should still claim them and tell gold about the error.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214214 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-29 20:46:19 +00:00
Rafael Espindola
d7a5d9444d Small gold plugin simplifications.
* Use a range loop.
* Store the extra options as "const char *".

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214199 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-29 19:17:44 +00:00
Rafael Espindola
b6b12e59a2 Use raw_fd_ostream instead of std::ofstream.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212483 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-07 20:34:51 +00:00
Rafael Espindola
7cba2a973f Update the MemoryBuffer API to use ErrorOr.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212405 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-06 17:43:13 +00:00
Rafael Espindola
04e43e64a9 Declare variable on first use.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212403 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-06 14:31:22 +00:00
Peter Collingbourne
59349ad8ad gold plugin: move target initialization to the top of the onload function.
On at least my machine, ar does not register an all symbols read hook (which
previously triggered target initialization), but it does register a claim
files hook, which depends on the targets being initialized.

Differential Revision: http://reviews.llvm.org/D4372

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212303 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-03 23:28:03 +00:00
Peter Collingbourne
1e64b30a9b LTO: rename the various makeLTOModule overloads.
This rename makes it easier to identify the specific overload being called
in each particular case and makes future refactorings easier.

Differential Revision: http://reviews.llvm.org/D4370

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@212302 91177308-0d34-0410-b5e6-96231b3b80d8
2014-07-03 23:28:00 +00:00
Tom Roeder
9ad9c96755 Fix the new LTOCodeGenerator setup in gold to parse options before using MAttrs.
This fixes a regression that made clang -flto -Wl,--plugin-opt=-mattr=+aes not
pass the "+aes" option to the LTOCodeGenerator attributes.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211804 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-26 20:43:27 +00:00
Alexey Volkov
9316a78fdd Fix unresolved symbols when loading gold plugin
Differential Revision: http://reviews.llvm.org/D4275


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211675 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-25 08:04:58 +00:00
Rafael Espindola
8483dacfd8 Small clanups:
Use static instead of anonymous namespace.
Delete write only variables.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211335 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-20 01:37:35 +00:00
Rafael Espindola
2ae686a2ab The gold plugin doesn't need disassemblers.
Back in r128440 tools/LTO started exporting the disassembler interface. It
was never clear why, but whatever the reason I am pretty sure it doesn't hold
for tools/gold.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211325 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-19 23:06:53 +00:00
Rafael Espindola
f38fdf1b61 Set gold plugin options in a sane order.
This fixes the  processing of --plugin-opt=-jump-table-type=arity.

Nice properties:
 * We call InitTargetOptionsFromCodeGenFlags once.
 * We call parseCodeGenDebugOptions once.
 * It works :-)

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211322 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-19 22:54:47 +00:00
Rafael Espindola
7c9dcfb29e Simplify. No functionality change.
Thanks to Alp Toker for noticing it.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211320 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-19 22:33:23 +00:00
Rafael Espindola
474f223860 Reduce indentation. No functionality change.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211318 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-19 22:20:07 +00:00
Rafael Espindola
0b5745abd8 Use lib/LTO directly in the gold plugin.
The tools/lto API is not the best choice for implementing a gold plugin. Among
other issues:

* It is an stable ABI. Old errors stay and we have to be really careful
  before adding new features.
* It has to support two fairly different linkers: gold and ld64.
* We end up with a plugin that depends on a shared lib, something quiet
  unusual in LLVM land.
* It hides LLVM. For some features in the gold plugin it would be really
  nice to be able to just get a Module or a GlobalValue.

This change is intended to be a very direct translation from the C API. It
will just enable other fixes and cleanups.

Tested with a LTO bootstrap on linux.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211315 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-19 21:14:13 +00:00
Rafael Espindola
d308c35617 Remove an incorrect fixme.
dynamic-no-pic is just another output type. If gnu ld gets support for MachO,
it should also add something like LDPO_DYN_NO_PIC to the plugin interface.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211305 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-19 19:45:25 +00:00
Rafael Espindola
1ad45020ec Remove 'using std::error_code' from tools.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210876 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-13 03:07:50 +00:00
Rafael Espindola
5c792faa0e Don't use 'using std::error_code' in include/llvm.
This should make sure that most new uses use the std prefix.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210835 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-12 21:46:39 +00:00
Rafael Espindola
d5132f9073 Remove system_error.h.
This is a minimal change to remove the header. I will remove the occurrences
of "using std::error_code" in a followup patch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210803 91177308-0d34-0410-b5e6-96231b3b80d8
2014-06-12 17:38:55 +00:00
Ahmed Charles
f4ccd11075 Replace OwningPtr<T> with std::unique_ptr<T>.
This compiles with no changes to clang/lld/lldb with MSVC and includes
overloads to various functions which are used by those projects and llvm
which have OwningPtr's as parameters. This should allow out of tree
projects some time to move. There are also no changes to libs/Target,
which should help out of tree targets have time to move, if necessary.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@203083 91177308-0d34-0410-b5e6-96231b3b80d8
2014-03-06 05:51:42 +00:00
Rafael Espindola
c540920a24 Don't mix the plain and keyword signatures of target_link_libraries.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@202299 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-26 20:39:28 +00:00
Sylvestre Ledru
6a089db385 Improve the declaration when LDPO_PIE is not available.
Thanks to İsmail Dönmez for the better declaration.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201163 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-11 17:30:18 +00:00
Sylvestre Ledru
31575e7fec If LDPO_PIE is not defined (before binutils 2.23 version),
use the hardcoded declaration 3
See r201110 for the initial change



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201161 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-11 17:11:32 +00:00
Rafael Espindola
256d839eed Fix an old FIXME. LDPO_PIE is available since 2.23, realeased 2012-10-22.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@201110 91177308-0d34-0410-b5e6-96231b3b80d8
2014-02-10 20:38:38 +00:00
Chandler Carruth
9f20a4c6ce Re-sort #include lines again, prior to moving headers around.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199080 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-13 08:04:33 +00:00
Tom Roeder
69051578c7 Space formatting fix for r198966.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@198971 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-10 23:17:39 +00:00
Tom Roeder
48324ba825 Fixing build break: should be in the if statement, not outside.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@198966 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-10 22:55:25 +00:00
Tom Roeder
b2c6a6c0bd Restore the library dependency of LLVMgold on LTO; this was removed recently but
is needed for LLVMgold to load in ld.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@198965 91177308-0d34-0410-b5e6-96231b3b80d8
2014-01-10 22:48:35 +00:00
Nico Weber
b6fe25c476 EXPORTED_SYMBOL_FILE support for cmake
The cmake build didn't support EXPORTED_SYMBOL_FILE. Instead, it had a
Windows-only implementation in tools/lto/CMakeLists.txt, a linux-only
implementation in tools/gold/CMakeLists.txt, and a darwin-only implementation
in tools/clang/tools/libclang/CMakeLists.txt.

This attempts to consolidate these one-offs into a single place. Clients can now
just set LLVM_EXPORTED_SYMBOL_FILE and things (hopefully) Just Work, like in
the make build.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@198136 91177308-0d34-0410-b5e6-96231b3b80d8
2013-12-28 23:31:44 +00:00
NAKAMURA Takumi
64cd49e624 gold-plugin.cpp: Use form of <plugin-api.h>, since it is external header.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@197997 91177308-0d34-0410-b5e6-96231b3b80d8
2013-12-25 02:24:38 +00:00
Alp Toker
c7cf4869a7 Cut the gold plugin README down to size
This file hasn't been updated in years. Remove old information and point to
the current documentation at GoldPlugin.rst.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@196100 91177308-0d34-0410-b5e6-96231b3b80d8
2013-12-02 14:17:47 +00:00
Rafael Espindola
7e667c56cf Use LTO_SYMBOL_SCOPE_DEFAULT_CAN_BE_HIDDEN instead of the "dso list".
There are two ways one could implement hiding of linkonce_odr symbols in LTO:
* LLVM tells the linker which symbols can be hidden if not used from native
  files.
* The linker tells LLVM which symbols are not used from other object files,
  but will be put in the dso symbol table if present.

GOLD's API is the second option. It was implemented almost 1:1 in llvm by
passing the list down to internalize.

LLVM already had partial support for the first option. It is also very similar
to how ld64 handles hiding these symbols when *not* doing LTO.

This patch then
* removes the APIs for the DSO list.
* marks LTO_SYMBOL_SCOPE_DEFAULT_CAN_BE_HIDDEN all linkonce_odr unnamed_addr
  global values and other linkonce_odr whose address is not used.
* makes the gold plugin responsible for handling the API mismatch.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193800 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-31 20:51:58 +00:00
Rafael Espindola
2d8a1d677c Check for errors when calling lto_codegen_add_module in the gold plugin.
Thanks to Milan Lenčo for noticing it.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192996 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-18 19:32:06 +00:00
Rafael Espindola
648a2e6714 [pr17595] Fix a use after free.
Destroying the codegen also frees the path of the created object. Copy the
path to a std::string.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192787 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-16 12:47:04 +00:00
Rafael Espindola
438900938c Optimize linkonce_odr unnamed_addr functions during LTO.
Generalize the API so we can distinguish symbols that are needed just for a DSO
symbol table from those that are used from some native .o.

The symbols that are only wanted for the dso symbol table can be dropped if
llvm can prove every other dso has a copy (linkonce_odr) and the address is not
important (unnamed_addr).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191922 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-03 18:29:09 +00:00
Rafael Espindola
75a641c910 Dispose the codegen even when just writing the bitcode file.
This makes it possible to add timers to the code generator and still use them
with -plugin-opt=emit-llvm.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191866 91177308-0d34-0410-b5e6-96231b3b80d8
2013-10-03 00:07:30 +00:00
Shuxin Yang
cfaa636a1d Revert r188188 and r188200.
In order to appease people (in Apple) who accuse me for committing "huge change" (?) without proper review. 

Thank Eric for fixing a compile-warning. 


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188204 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-12 21:07:31 +00:00
Shuxin Yang
67d135ae40 Misc enhancements to LTO:
1. Add some helper classes for partitions. They are designed in a
     way such that the top-level LTO driver will not see much difference 
     with or without partitioning.

  2. Introduce work-dir. Now all intermediate files generated during 
     LTO phases will be saved under work-dir. User can specify the workdir
     via -lto-workdir=/path/to/dir. By default the work-dir will be 
     erased before linker exit. To keep the workdir, do -lto-keep, or -lto-keep=1.

    TODO: Erase the workdir, if the linker exit prematurely.  
      We are currently not able to remove directory on signal. The support 
      routines simply ignore directory.

  3. Add one new API lto_codegen_get_files_need_remove().
     Linker and LTO plugin will communicate via this API about which files
    (including directories) need to removed before linker exit.


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188188 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-12 18:29:43 +00:00
Rafael Espindola
70c7e48545 Split getOpenFile into getOpenFile and getOpenFileSlice.
The main observation is that we never need both the filesize and the map size.
When mapping a slice of a file, it doesn't make sense to request a null
terminator and that would be the only case where the filesize would be used.

There are other cleanups that should be done in this area:

* A client should not have to pass the size (even an explicit -1) to say if
  it wants a null terminator or not, so we should probably swap the argument
  order.
* The default should be to not require a null terminator. Very few clients
  require this, but many end up asking for it just because it is the default.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186984 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-23 20:25:01 +00:00
Rafael Espindola
963cf75e1c Fix use of the getOpenFile api.
The gold plugin was passing the desired map size as the file size. This was
working for two reasons:
* Recent version of gold provide the get_view callback, so this code was not
  used.
* In older versions, getOpenFile was called, but the file size is never used
  if we don't require null terminated buffers and map size defaults to the
  file size.

Thanks to Eli Bendersky for noticing this.

I will try to make this api a bit less error prone.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186978 91177308-0d34-0410-b5e6-96231b3b80d8
2013-07-23 18:44:31 +00:00