mirror of
https://github.com/c64scene-ar/llvm-6502.git
synced 2024-11-01 15:11:24 +00:00
89064930a0
Cleanup some bitrot in SourceLevelDebugging.rst. - Pull the still-relevant details about individual descriptors into LangRef.rst. Cut a lot of it to avoid over-describing the fields, as the C++ classes and assembly format are mostly self-describing now. If there's anything specific that I shouldn't have cut, let me know and I'll add it back. - Rewrite the remaining sections to refer to the new debug info hierarchy in LangRef.rst. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@232566 91177308-0d34-0410-b5e6-96231b3b80d8
1338 lines
54 KiB
ReStructuredText
1338 lines
54 KiB
ReStructuredText
================================
|
|
Source Level Debugging with LLVM
|
|
================================
|
|
|
|
.. contents::
|
|
:local:
|
|
|
|
Introduction
|
|
============
|
|
|
|
This document is the central repository for all information pertaining to debug
|
|
information in LLVM. It describes the :ref:`actual format that the LLVM debug
|
|
information takes <format>`, which is useful for those interested in creating
|
|
front-ends or dealing directly with the information. Further, this document
|
|
provides specific examples of what debug information for C/C++ looks like.
|
|
|
|
Philosophy behind LLVM debugging information
|
|
--------------------------------------------
|
|
|
|
The idea of the LLVM debugging information is to capture how the important
|
|
pieces of the source-language's Abstract Syntax Tree map onto LLVM code.
|
|
Several design aspects have shaped the solution that appears here. The
|
|
important ones are:
|
|
|
|
* Debugging information should have very little impact on the rest of the
|
|
compiler. No transformations, analyses, or code generators should need to
|
|
be modified because of debugging information.
|
|
|
|
* LLVM optimizations should interact in :ref:`well-defined and easily described
|
|
ways <intro_debugopt>` with the debugging information.
|
|
|
|
* Because LLVM is designed to support arbitrary programming languages,
|
|
LLVM-to-LLVM tools should not need to know anything about the semantics of
|
|
the source-level-language.
|
|
|
|
* Source-level languages are often **widely** different from one another.
|
|
LLVM should not put any restrictions of the flavor of the source-language,
|
|
and the debugging information should work with any language.
|
|
|
|
* With code generator support, it should be possible to use an LLVM compiler
|
|
to compile a program to native machine code and standard debugging
|
|
formats. This allows compatibility with traditional machine-code level
|
|
debuggers, like GDB or DBX.
|
|
|
|
The approach used by the LLVM implementation is to use a small set of
|
|
:ref:`intrinsic functions <format_common_intrinsics>` to define a mapping
|
|
between LLVM program objects and the source-level objects. The description of
|
|
the source-level program is maintained in LLVM metadata in an
|
|
:ref:`implementation-defined format <ccxx_frontend>` (the C/C++ front-end
|
|
currently uses working draft 7 of the `DWARF 3 standard
|
|
<http://www.eagercon.com/dwarf/dwarf3std.htm>`_).
|
|
|
|
When a program is being debugged, a debugger interacts with the user and turns
|
|
the stored debug information into source-language specific information. As
|
|
such, a debugger must be aware of the source-language, and is thus tied to a
|
|
specific language or family of languages.
|
|
|
|
Debug information consumers
|
|
---------------------------
|
|
|
|
The role of debug information is to provide meta information normally stripped
|
|
away during the compilation process. This meta information provides an LLVM
|
|
user a relationship between generated code and the original program source
|
|
code.
|
|
|
|
Currently, debug information is consumed by DwarfDebug to produce dwarf
|
|
information used by the gdb debugger. Other targets could use the same
|
|
information to produce stabs or other debug forms.
|
|
|
|
It would also be reasonable to use debug information to feed profiling tools
|
|
for analysis of generated code, or, tools for reconstructing the original
|
|
source from generated code.
|
|
|
|
TODO - expound a bit more.
|
|
|
|
.. _intro_debugopt:
|
|
|
|
Debugging optimized code
|
|
------------------------
|
|
|
|
An extremely high priority of LLVM debugging information is to make it interact
|
|
well with optimizations and analysis. In particular, the LLVM debug
|
|
information provides the following guarantees:
|
|
|
|
* LLVM debug information **always provides information to accurately read
|
|
the source-level state of the program**, regardless of which LLVM
|
|
optimizations have been run, and without any modification to the
|
|
optimizations themselves. However, some optimizations may impact the
|
|
ability to modify the current state of the program with a debugger, such
|
|
as setting program variables, or calling functions that have been
|
|
deleted.
|
|
|
|
* As desired, LLVM optimizations can be upgraded to be aware of the LLVM
|
|
debugging information, allowing them to update the debugging information
|
|
as they perform aggressive optimizations. This means that, with effort,
|
|
the LLVM optimizers could optimize debug code just as well as non-debug
|
|
code.
|
|
|
|
* LLVM debug information does not prevent optimizations from
|
|
happening (for example inlining, basic block reordering/merging/cleanup,
|
|
tail duplication, etc).
|
|
|
|
* LLVM debug information is automatically optimized along with the rest of
|
|
the program, using existing facilities. For example, duplicate
|
|
information is automatically merged by the linker, and unused information
|
|
is automatically removed.
|
|
|
|
Basically, the debug information allows you to compile a program with
|
|
"``-O0 -g``" and get full debug information, allowing you to arbitrarily modify
|
|
the program as it executes from a debugger. Compiling a program with
|
|
"``-O3 -g``" gives you full debug information that is always available and
|
|
accurate for reading (e.g., you get accurate stack traces despite tail call
|
|
elimination and inlining), but you might lose the ability to modify the program
|
|
and call functions where were optimized out of the program, or inlined away
|
|
completely.
|
|
|
|
:ref:`LLVM test suite <test-suite-quickstart>` provides a framework to test
|
|
optimizer's handling of debugging information. It can be run like this:
|
|
|
|
.. code-block:: bash
|
|
|
|
% cd llvm/projects/test-suite/MultiSource/Benchmarks # or some other level
|
|
% make TEST=dbgopt
|
|
|
|
This will test impact of debugging information on optimization passes. If
|
|
debugging information influences optimization passes then it will be reported
|
|
as a failure. See :doc:`TestingGuide` for more information on LLVM test
|
|
infrastructure and how to run various tests.
|
|
|
|
.. _format:
|
|
|
|
Debugging information format
|
|
============================
|
|
|
|
LLVM debugging information has been carefully designed to make it possible for
|
|
the optimizer to optimize the program and debugging information without
|
|
necessarily having to know anything about debugging information. In
|
|
particular, the use of metadata avoids duplicated debugging information from
|
|
the beginning, and the global dead code elimination pass automatically deletes
|
|
debugging information for a function if it decides to delete the function.
|
|
|
|
To do this, most of the debugging information (descriptors for types,
|
|
variables, functions, source files, etc) is inserted by the language front-end
|
|
in the form of LLVM metadata.
|
|
|
|
Debug information is designed to be agnostic about the target debugger and
|
|
debugging information representation (e.g. DWARF/Stabs/etc). It uses a generic
|
|
pass to decode the information that represents variables, types, functions,
|
|
namespaces, etc: this allows for arbitrary source-language semantics and
|
|
type-systems to be used, as long as there is a module written for the target
|
|
debugger to interpret the information.
|
|
|
|
To provide basic functionality, the LLVM debugger does have to make some
|
|
assumptions about the source-level language being debugged, though it keeps
|
|
these to a minimum. The only common features that the LLVM debugger assumes
|
|
exist are `source files <LangRef.html#MDFile>`_, and `program objects
|
|
<LangRef.html#MDGlobalVariable>`_. These abstract objects are used by a
|
|
debugger to form stack traces, show information about local variables, etc.
|
|
|
|
This section of the documentation first describes the representation aspects
|
|
common to any source-language. :ref:`ccxx_frontend` describes the data layout
|
|
conventions used by the C and C++ front-ends.
|
|
|
|
Debug information descriptors are `specialized metadata nodes
|
|
<LangRef.html#specialized-metadata>`_, first-class subclasses of ``Metadata``.
|
|
|
|
.. _format_common_intrinsics:
|
|
|
|
Debugger intrinsic functions
|
|
----------------------------
|
|
|
|
LLVM uses several intrinsic functions (name prefixed with "``llvm.dbg``") to
|
|
provide debug information at various points in generated code.
|
|
|
|
``llvm.dbg.declare``
|
|
^^^^^^^^^^^^^^^^^^^^
|
|
|
|
.. code-block:: llvm
|
|
|
|
void %llvm.dbg.declare(metadata, metadata, metadata)
|
|
|
|
This intrinsic provides information about a local element (e.g., variable).
|
|
The first argument is metadata holding the alloca for the variable. The second
|
|
argument is a `local variable <LangRef.html#MDLocalVariable>`_ containing a
|
|
description of the variable. The third argument is a `complex expression
|
|
<LangRef.html#MDExpression>`_.
|
|
|
|
``llvm.dbg.value``
|
|
^^^^^^^^^^^^^^^^^^
|
|
|
|
.. code-block:: llvm
|
|
|
|
void %llvm.dbg.value(metadata, i64, metadata, metadata)
|
|
|
|
This intrinsic provides information when a user source variable is set to a new
|
|
value. The first argument is the new value (wrapped as metadata). The second
|
|
argument is the offset in the user source variable where the new value is
|
|
written. The third argument is a `local variable
|
|
<LangRef.html#MDLocalVariable>`_ containing a description of the variable. The
|
|
third argument is a `complex expression <LangRef.html#MDExpression>`_.
|
|
|
|
Object lifetimes and scoping
|
|
============================
|
|
|
|
In many languages, the local variables in functions can have their lifetimes or
|
|
scopes limited to a subset of a function. In the C family of languages, for
|
|
example, variables are only live (readable and writable) within the source
|
|
block that they are defined in. In functional languages, values are only
|
|
readable after they have been defined. Though this is a very obvious concept,
|
|
it is non-trivial to model in LLVM, because it has no notion of scoping in this
|
|
sense, and does not want to be tied to a language's scoping rules.
|
|
|
|
In order to handle this, the LLVM debug format uses the metadata attached to
|
|
llvm instructions to encode line number and scoping information. Consider the
|
|
following C fragment, for example:
|
|
|
|
.. code-block:: c
|
|
|
|
1. void foo() {
|
|
2. int X = 21;
|
|
3. int Y = 22;
|
|
4. {
|
|
5. int Z = 23;
|
|
6. Z = X;
|
|
7. }
|
|
8. X = Y;
|
|
9. }
|
|
|
|
Compiled to LLVM, this function would be represented like this:
|
|
|
|
.. code-block:: llvm
|
|
|
|
; Function Attrs: nounwind ssp uwtable
|
|
define void @foo() #0 {
|
|
entry:
|
|
%X = alloca i32, align 4
|
|
%Y = alloca i32, align 4
|
|
%Z = alloca i32, align 4
|
|
call void @llvm.dbg.declare(metadata i32* %X, metadata !11, metadata !13), !dbg !14
|
|
store i32 21, i32* %X, align 4, !dbg !14
|
|
call void @llvm.dbg.declare(metadata i32* %Y, metadata !15, metadata !13), !dbg !16
|
|
store i32 22, i32* %Y, align 4, !dbg !16
|
|
call void @llvm.dbg.declare(metadata i32* %Z, metadata !17, metadata !13), !dbg !19
|
|
store i32 23, i32* %Z, align 4, !dbg !19
|
|
%0 = load i32, i32* %X, align 4, !dbg !20
|
|
store i32 %0, i32* %Z, align 4, !dbg !21
|
|
%1 = load i32, i32* %Y, align 4, !dbg !22
|
|
store i32 %1, i32* %X, align 4, !dbg !23
|
|
ret void, !dbg !24
|
|
}
|
|
|
|
; Function Attrs: nounwind readnone
|
|
declare void @llvm.dbg.declare(metadata, metadata, metadata) #1
|
|
|
|
attributes #0 = { nounwind ssp uwtable "less-precise-fpmad"="false" "no-frame-pointer-elim"="true" "no-frame-pointer-elim-non-leaf" "no-infs-fp-math"="false" "no-nans-fp-math"="false" "stack-protector-buffer-size"="8" "unsafe-fp-math"="false" "use-soft-float"="false" }
|
|
attributes #1 = { nounwind readnone }
|
|
|
|
!llvm.dbg.cu = !{!0}
|
|
!llvm.module.flags = !{!7, !8, !9}
|
|
!llvm.ident = !{!10}
|
|
|
|
!0 = !MDCompileUnit(language: DW_LANG_C99, file: !1, producer: "clang version 3.7.0 (trunk 231150) (llvm/trunk 231154)", isOptimized: false, runtimeVersion: 0, emissionKind: 1, enums: !2, retainedTypes: !2, subprograms: !3, globals: !2, imports: !2)
|
|
!1 = !MDFile(filename: "/dev/stdin", directory: "/Users/dexonsmith/data/llvm/debug-info")
|
|
!2 = !{}
|
|
!3 = !{!4}
|
|
!4 = !MDSubprogram(name: "foo", scope: !1, file: !1, line: 1, type: !5, isLocal: false, isDefinition: true, scopeLine: 1, isOptimized: false, function: void ()* @foo, variables: !2)
|
|
!5 = !MDSubroutineType(types: !6)
|
|
!6 = !{null}
|
|
!7 = !{i32 2, !"Dwarf Version", i32 2}
|
|
!8 = !{i32 2, !"Debug Info Version", i32 3}
|
|
!9 = !{i32 1, !"PIC Level", i32 2}
|
|
!10 = !{!"clang version 3.7.0 (trunk 231150) (llvm/trunk 231154)"}
|
|
!11 = !MDLocalVariable(tag: DW_TAG_auto_variable, name: "X", scope: !4, file: !1, line: 2, type: !12)
|
|
!12 = !MDBasicType(name: "int", size: 32, align: 32, encoding: DW_ATE_signed)
|
|
!13 = !MDExpression()
|
|
!14 = !MDLocation(line: 2, column: 9, scope: !4)
|
|
!15 = !MDLocalVariable(tag: DW_TAG_auto_variable, name: "Y", scope: !4, file: !1, line: 3, type: !12)
|
|
!16 = !MDLocation(line: 3, column: 9, scope: !4)
|
|
!17 = !MDLocalVariable(tag: DW_TAG_auto_variable, name: "Z", scope: !18, file: !1, line: 5, type: !12)
|
|
!18 = distinct !MDLexicalBlock(scope: !4, file: !1, line: 4, column: 5)
|
|
!19 = !MDLocation(line: 5, column: 11, scope: !18)
|
|
!20 = !MDLocation(line: 6, column: 11, scope: !18)
|
|
!21 = !MDLocation(line: 6, column: 9, scope: !18)
|
|
!22 = !MDLocation(line: 8, column: 9, scope: !4)
|
|
!23 = !MDLocation(line: 8, column: 7, scope: !4)
|
|
!24 = !MDLocation(line: 9, column: 3, scope: !4)
|
|
|
|
|
|
This example illustrates a few important details about LLVM debugging
|
|
information. In particular, it shows how the ``llvm.dbg.declare`` intrinsic and
|
|
location information, which are attached to an instruction, are applied
|
|
together to allow a debugger to analyze the relationship between statements,
|
|
variable definitions, and the code used to implement the function.
|
|
|
|
.. code-block:: llvm
|
|
|
|
call void @llvm.dbg.declare(metadata i32* %X, metadata !11, metadata !13), !dbg !14
|
|
; [debug line = 2:7] [debug variable = X]
|
|
|
|
The first intrinsic ``%llvm.dbg.declare`` encodes debugging information for the
|
|
variable ``X``. The metadata ``!dbg !14`` attached to the intrinsic provides
|
|
scope information for the variable ``X``.
|
|
|
|
.. code-block:: llvm
|
|
|
|
!14 = !MDLocation(line: 2, column: 9, scope: !4)
|
|
!4 = !MDSubprogram(name: "foo", scope: !1, file: !1, line: 1, type: !5,
|
|
isLocal: false, isDefinition: true, scopeLine: 1,
|
|
isOptimized: false, function: void ()* @foo,
|
|
variables: !2)
|
|
|
|
Here ``!14`` is metadata providing `location information
|
|
<LangRef.html#MDLocation>`_. In this example, scope is encoded by ``!4``, a
|
|
`subprogram descriptor <LangRef.html#MDSubprogram>`_. This way the location
|
|
information attached to the intrinsics indicates that the variable ``X`` is
|
|
declared at line number 2 at a function level scope in function ``foo``.
|
|
|
|
Now lets take another example.
|
|
|
|
.. code-block:: llvm
|
|
|
|
call void @llvm.dbg.declare(metadata i32* %Z, metadata !17, metadata !13), !dbg !19
|
|
; [debug line = 5:9] [debug variable = Z]
|
|
|
|
The third intrinsic ``%llvm.dbg.declare`` encodes debugging information for
|
|
variable ``Z``. The metadata ``!dbg !19`` attached to the intrinsic provides
|
|
scope information for the variable ``Z``.
|
|
|
|
.. code-block:: llvm
|
|
|
|
!18 = distinct !MDLexicalBlock(scope: !4, file: !1, line: 4, column: 5)
|
|
!19 = !MDLocation(line: 5, column: 11, scope: !18)
|
|
|
|
Here ``!19`` indicates that ``Z`` is declared at line number 5 and column
|
|
number 0 inside of lexical scope ``!18``. The lexical scope itself resides
|
|
inside of subprogram ``!4`` described above.
|
|
|
|
The scope information attached with each instruction provides a straightforward
|
|
way to find instructions covered by a scope.
|
|
|
|
.. _ccxx_frontend:
|
|
|
|
C/C++ front-end specific debug information
|
|
==========================================
|
|
|
|
The C and C++ front-ends represent information about the program in a format
|
|
that is effectively identical to `DWARF 3.0
|
|
<http://www.eagercon.com/dwarf/dwarf3std.htm>`_ in terms of information
|
|
content. This allows code generators to trivially support native debuggers by
|
|
generating standard dwarf information, and contains enough information for
|
|
non-dwarf targets to translate it as needed.
|
|
|
|
This section describes the forms used to represent C and C++ programs. Other
|
|
languages could pattern themselves after this (which itself is tuned to
|
|
representing programs in the same way that DWARF 3 does), or they could choose
|
|
to provide completely different forms if they don't fit into the DWARF model.
|
|
As support for debugging information gets added to the various LLVM
|
|
source-language front-ends, the information used should be documented here.
|
|
|
|
The following sections provide examples of a few C/C++ constructs and the debug
|
|
information that would best describe those constructs. The canonical
|
|
references are the ``DIDescriptor`` classes defined in
|
|
``include/llvm/IR/DebugInfo.h`` and the implementations of the helper functions
|
|
in ``lib/IR/DIBuilder.cpp``.
|
|
|
|
C/C++ source file information
|
|
-----------------------------
|
|
|
|
``llvm::Instruction`` provides easy access to metadata attached with an
|
|
instruction. One can extract line number information encoded in LLVM IR using
|
|
``Instruction::getMetadata()`` and ``DILocation::getLineNumber()``.
|
|
|
|
.. code-block:: c++
|
|
|
|
if (MDNode *N = I->getMetadata("dbg")) { // Here I is an LLVM instruction
|
|
DILocation Loc(N); // DILocation is in DebugInfo.h
|
|
unsigned Line = Loc.getLineNumber();
|
|
StringRef File = Loc.getFilename();
|
|
StringRef Dir = Loc.getDirectory();
|
|
}
|
|
|
|
C/C++ global variable information
|
|
---------------------------------
|
|
|
|
Given an integer global variable declared as follows:
|
|
|
|
.. code-block:: c
|
|
|
|
int MyGlobal = 100;
|
|
|
|
a C/C++ front-end would generate the following descriptors:
|
|
|
|
.. code-block:: llvm
|
|
|
|
;;
|
|
;; Define the global itself.
|
|
;;
|
|
@MyGlobal = global i32 100, align 4
|
|
|
|
;;
|
|
;; List of debug info of globals
|
|
;;
|
|
!llvm.dbg.cu = !{!0}
|
|
|
|
;; Some unrelated metadata.
|
|
!llvm.module.flags = !{!6, !7}
|
|
|
|
;; Define the compile unit.
|
|
!0 = !MDCompileUnit(language: DW_LANG_C99, file: !1,
|
|
producer:
|
|
"clang version 3.7.0 (trunk 231150) (llvm/trunk 231154)",
|
|
isOptimized: false, runtimeVersion: 0, emissionKind: 1,
|
|
enums: !2, retainedTypes: !2, subprograms: !2, globals:
|
|
!3, imports: !2)
|
|
|
|
;;
|
|
;; Define the file
|
|
;;
|
|
!1 = !MDFile(filename: "/dev/stdin",
|
|
directory: "/Users/dexonsmith/data/llvm/debug-info")
|
|
|
|
;; An empty array.
|
|
!2 = !{}
|
|
|
|
;; The Array of Global Variables
|
|
!3 = !{!4}
|
|
|
|
;;
|
|
;; Define the global variable itself.
|
|
;;
|
|
!4 = !MDGlobalVariable(name: "MyGlobal", scope: !0, file: !1, line: 1,
|
|
type: !5, isLocal: false, isDefinition: true,
|
|
variable: i32* @MyGlobal)
|
|
|
|
;;
|
|
;; Define the type
|
|
;;
|
|
!5 = !MDBasicType(name: "int", size: 32, align: 32, encoding: DW_ATE_signed)
|
|
|
|
;; Dwarf version to output.
|
|
!6 = !{i32 2, !"Dwarf Version", i32 2}
|
|
|
|
;; Debug info schema version.
|
|
!7 = !{i32 2, !"Debug Info Version", i32 3}
|
|
|
|
C/C++ function information
|
|
--------------------------
|
|
|
|
Given a function declared as follows:
|
|
|
|
.. code-block:: c
|
|
|
|
int main(int argc, char *argv[]) {
|
|
return 0;
|
|
}
|
|
|
|
a C/C++ front-end would generate the following descriptors:
|
|
|
|
.. code-block:: llvm
|
|
|
|
;;
|
|
;; Define the anchor for subprograms.
|
|
;;
|
|
!4 = !MDSubprogram(name: "main", scope: !1, file: !1, line: 1, type: !5,
|
|
isLocal: false, isDefinition: true, scopeLine: 1,
|
|
flags: DIFlagPrototyped, isOptimized: false,
|
|
function: i32 (i32, i8**)* @main, variables: !2)
|
|
|
|
;;
|
|
;; Define the subprogram itself.
|
|
;;
|
|
define i32 @main(i32 %argc, i8** %argv) {
|
|
...
|
|
}
|
|
|
|
Debugging information format
|
|
============================
|
|
|
|
Debugging Information Extension for Objective C Properties
|
|
----------------------------------------------------------
|
|
|
|
Introduction
|
|
^^^^^^^^^^^^
|
|
|
|
Objective C provides a simpler way to declare and define accessor methods using
|
|
declared properties. The language provides features to declare a property and
|
|
to let compiler synthesize accessor methods.
|
|
|
|
The debugger lets developer inspect Objective C interfaces and their instance
|
|
variables and class variables. However, the debugger does not know anything
|
|
about the properties defined in Objective C interfaces. The debugger consumes
|
|
information generated by compiler in DWARF format. The format does not support
|
|
encoding of Objective C properties. This proposal describes DWARF extensions to
|
|
encode Objective C properties, which the debugger can use to let developers
|
|
inspect Objective C properties.
|
|
|
|
Proposal
|
|
^^^^^^^^
|
|
|
|
Objective C properties exist separately from class members. A property can be
|
|
defined only by "setter" and "getter" selectors, and be calculated anew on each
|
|
access. Or a property can just be a direct access to some declared ivar.
|
|
Finally it can have an ivar "automatically synthesized" for it by the compiler,
|
|
in which case the property can be referred to in user code directly using the
|
|
standard C dereference syntax as well as through the property "dot" syntax, but
|
|
there is no entry in the ``@interface`` declaration corresponding to this ivar.
|
|
|
|
To facilitate debugging, these properties we will add a new DWARF TAG into the
|
|
``DW_TAG_structure_type`` definition for the class to hold the description of a
|
|
given property, and a set of DWARF attributes that provide said description.
|
|
The property tag will also contain the name and declared type of the property.
|
|
|
|
If there is a related ivar, there will also be a DWARF property attribute placed
|
|
in the ``DW_TAG_member`` DIE for that ivar referring back to the property TAG
|
|
for that property. And in the case where the compiler synthesizes the ivar
|
|
directly, the compiler is expected to generate a ``DW_TAG_member`` for that
|
|
ivar (with the ``DW_AT_artificial`` set to 1), whose name will be the name used
|
|
to access this ivar directly in code, and with the property attribute pointing
|
|
back to the property it is backing.
|
|
|
|
The following examples will serve as illustration for our discussion:
|
|
|
|
.. code-block:: objc
|
|
|
|
@interface I1 {
|
|
int n2;
|
|
}
|
|
|
|
@property int p1;
|
|
@property int p2;
|
|
@end
|
|
|
|
@implementation I1
|
|
@synthesize p1;
|
|
@synthesize p2 = n2;
|
|
@end
|
|
|
|
This produces the following DWARF (this is a "pseudo dwarfdump" output):
|
|
|
|
.. code-block:: none
|
|
|
|
0x00000100: TAG_structure_type [7] *
|
|
AT_APPLE_runtime_class( 0x10 )
|
|
AT_name( "I1" )
|
|
AT_decl_file( "Objc_Property.m" )
|
|
AT_decl_line( 3 )
|
|
|
|
0x00000110 TAG_APPLE_property
|
|
AT_name ( "p1" )
|
|
AT_type ( {0x00000150} ( int ) )
|
|
|
|
0x00000120: TAG_APPLE_property
|
|
AT_name ( "p2" )
|
|
AT_type ( {0x00000150} ( int ) )
|
|
|
|
0x00000130: TAG_member [8]
|
|
AT_name( "_p1" )
|
|
AT_APPLE_property ( {0x00000110} "p1" )
|
|
AT_type( {0x00000150} ( int ) )
|
|
AT_artificial ( 0x1 )
|
|
|
|
0x00000140: TAG_member [8]
|
|
AT_name( "n2" )
|
|
AT_APPLE_property ( {0x00000120} "p2" )
|
|
AT_type( {0x00000150} ( int ) )
|
|
|
|
0x00000150: AT_type( ( int ) )
|
|
|
|
Note, the current convention is that the name of the ivar for an
|
|
auto-synthesized property is the name of the property from which it derives
|
|
with an underscore prepended, as is shown in the example. But we actually
|
|
don't need to know this convention, since we are given the name of the ivar
|
|
directly.
|
|
|
|
Also, it is common practice in ObjC to have different property declarations in
|
|
the @interface and @implementation - e.g. to provide a read-only property in
|
|
the interface,and a read-write interface in the implementation. In that case,
|
|
the compiler should emit whichever property declaration will be in force in the
|
|
current translation unit.
|
|
|
|
Developers can decorate a property with attributes which are encoded using
|
|
``DW_AT_APPLE_property_attribute``.
|
|
|
|
.. code-block:: objc
|
|
|
|
@property (readonly, nonatomic) int pr;
|
|
|
|
.. code-block:: none
|
|
|
|
TAG_APPLE_property [8]
|
|
AT_name( "pr" )
|
|
AT_type ( {0x00000147} (int) )
|
|
AT_APPLE_property_attribute (DW_APPLE_PROPERTY_readonly, DW_APPLE_PROPERTY_nonatomic)
|
|
|
|
The setter and getter method names are attached to the property using
|
|
``DW_AT_APPLE_property_setter`` and ``DW_AT_APPLE_property_getter`` attributes.
|
|
|
|
.. code-block:: objc
|
|
|
|
@interface I1
|
|
@property (setter=myOwnP3Setter:) int p3;
|
|
-(void)myOwnP3Setter:(int)a;
|
|
@end
|
|
|
|
@implementation I1
|
|
@synthesize p3;
|
|
-(void)myOwnP3Setter:(int)a{ }
|
|
@end
|
|
|
|
The DWARF for this would be:
|
|
|
|
.. code-block:: none
|
|
|
|
0x000003bd: TAG_structure_type [7] *
|
|
AT_APPLE_runtime_class( 0x10 )
|
|
AT_name( "I1" )
|
|
AT_decl_file( "Objc_Property.m" )
|
|
AT_decl_line( 3 )
|
|
|
|
0x000003cd TAG_APPLE_property
|
|
AT_name ( "p3" )
|
|
AT_APPLE_property_setter ( "myOwnP3Setter:" )
|
|
AT_type( {0x00000147} ( int ) )
|
|
|
|
0x000003f3: TAG_member [8]
|
|
AT_name( "_p3" )
|
|
AT_type ( {0x00000147} ( int ) )
|
|
AT_APPLE_property ( {0x000003cd} )
|
|
AT_artificial ( 0x1 )
|
|
|
|
New DWARF Tags
|
|
^^^^^^^^^^^^^^
|
|
|
|
+-----------------------+--------+
|
|
| TAG | Value |
|
|
+=======================+========+
|
|
| DW_TAG_APPLE_property | 0x4200 |
|
|
+-----------------------+--------+
|
|
|
|
New DWARF Attributes
|
|
^^^^^^^^^^^^^^^^^^^^
|
|
|
|
+--------------------------------+--------+-----------+
|
|
| Attribute | Value | Classes |
|
|
+================================+========+===========+
|
|
| DW_AT_APPLE_property | 0x3fed | Reference |
|
|
+--------------------------------+--------+-----------+
|
|
| DW_AT_APPLE_property_getter | 0x3fe9 | String |
|
|
+--------------------------------+--------+-----------+
|
|
| DW_AT_APPLE_property_setter | 0x3fea | String |
|
|
+--------------------------------+--------+-----------+
|
|
| DW_AT_APPLE_property_attribute | 0x3feb | Constant |
|
|
+--------------------------------+--------+-----------+
|
|
|
|
New DWARF Constants
|
|
^^^^^^^^^^^^^^^^^^^
|
|
|
|
+--------------------------------------+-------+
|
|
| Name | Value |
|
|
+======================================+=======+
|
|
| DW_APPLE_PROPERTY_readonly | 0x01 |
|
|
+--------------------------------------+-------+
|
|
| DW_APPLE_PROPERTY_getter | 0x02 |
|
|
+--------------------------------------+-------+
|
|
| DW_APPLE_PROPERTY_assign | 0x04 |
|
|
+--------------------------------------+-------+
|
|
| DW_APPLE_PROPERTY_readwrite | 0x08 |
|
|
+--------------------------------------+-------+
|
|
| DW_APPLE_PROPERTY_retain | 0x10 |
|
|
+--------------------------------------+-------+
|
|
| DW_APPLE_PROPERTY_copy | 0x20 |
|
|
+--------------------------------------+-------+
|
|
| DW_APPLE_PROPERTY_nonatomic | 0x40 |
|
|
+--------------------------------------+-------+
|
|
| DW_APPLE_PROPERTY_setter | 0x80 |
|
|
+--------------------------------------+-------+
|
|
| DW_APPLE_PROPERTY_atomic | 0x100 |
|
|
+--------------------------------------+-------+
|
|
| DW_APPLE_PROPERTY_weak | 0x200 |
|
|
+--------------------------------------+-------+
|
|
| DW_APPLE_PROPERTY_strong | 0x400 |
|
|
+--------------------------------------+-------+
|
|
| DW_APPLE_PROPERTY_unsafe_unretained | 0x800 |
|
|
+--------------------------------+-----+-------+
|
|
|
|
Name Accelerator Tables
|
|
-----------------------
|
|
|
|
Introduction
|
|
^^^^^^^^^^^^
|
|
|
|
The "``.debug_pubnames``" and "``.debug_pubtypes``" formats are not what a
|
|
debugger needs. The "``pub``" in the section name indicates that the entries
|
|
in the table are publicly visible names only. This means no static or hidden
|
|
functions show up in the "``.debug_pubnames``". No static variables or private
|
|
class variables are in the "``.debug_pubtypes``". Many compilers add different
|
|
things to these tables, so we can't rely upon the contents between gcc, icc, or
|
|
clang.
|
|
|
|
The typical query given by users tends not to match up with the contents of
|
|
these tables. For example, the DWARF spec states that "In the case of the name
|
|
of a function member or static data member of a C++ structure, class or union,
|
|
the name presented in the "``.debug_pubnames``" section is not the simple name
|
|
given by the ``DW_AT_name attribute`` of the referenced debugging information
|
|
entry, but rather the fully qualified name of the data or function member."
|
|
So the only names in these tables for complex C++ entries is a fully
|
|
qualified name. Debugger users tend not to enter their search strings as
|
|
"``a::b::c(int,const Foo&) const``", but rather as "``c``", "``b::c``" , or
|
|
"``a::b::c``". So the name entered in the name table must be demangled in
|
|
order to chop it up appropriately and additional names must be manually entered
|
|
into the table to make it effective as a name lookup table for debuggers to
|
|
se.
|
|
|
|
All debuggers currently ignore the "``.debug_pubnames``" table as a result of
|
|
its inconsistent and useless public-only name content making it a waste of
|
|
space in the object file. These tables, when they are written to disk, are not
|
|
sorted in any way, leaving every debugger to do its own parsing and sorting.
|
|
These tables also include an inlined copy of the string values in the table
|
|
itself making the tables much larger than they need to be on disk, especially
|
|
for large C++ programs.
|
|
|
|
Can't we just fix the sections by adding all of the names we need to this
|
|
table? No, because that is not what the tables are defined to contain and we
|
|
won't know the difference between the old bad tables and the new good tables.
|
|
At best we could make our own renamed sections that contain all of the data we
|
|
need.
|
|
|
|
These tables are also insufficient for what a debugger like LLDB needs. LLDB
|
|
uses clang for its expression parsing where LLDB acts as a PCH. LLDB is then
|
|
often asked to look for type "``foo``" or namespace "``bar``", or list items in
|
|
namespace "``baz``". Namespaces are not included in the pubnames or pubtypes
|
|
tables. Since clang asks a lot of questions when it is parsing an expression,
|
|
we need to be very fast when looking up names, as it happens a lot. Having new
|
|
accelerator tables that are optimized for very quick lookups will benefit this
|
|
type of debugging experience greatly.
|
|
|
|
We would like to generate name lookup tables that can be mapped into memory
|
|
from disk, and used as is, with little or no up-front parsing. We would also
|
|
be able to control the exact content of these different tables so they contain
|
|
exactly what we need. The Name Accelerator Tables were designed to fix these
|
|
issues. In order to solve these issues we need to:
|
|
|
|
* Have a format that can be mapped into memory from disk and used as is
|
|
* Lookups should be very fast
|
|
* Extensible table format so these tables can be made by many producers
|
|
* Contain all of the names needed for typical lookups out of the box
|
|
* Strict rules for the contents of tables
|
|
|
|
Table size is important and the accelerator table format should allow the reuse
|
|
of strings from common string tables so the strings for the names are not
|
|
duplicated. We also want to make sure the table is ready to be used as-is by
|
|
simply mapping the table into memory with minimal header parsing.
|
|
|
|
The name lookups need to be fast and optimized for the kinds of lookups that
|
|
debuggers tend to do. Optimally we would like to touch as few parts of the
|
|
mapped table as possible when doing a name lookup and be able to quickly find
|
|
the name entry we are looking for, or discover there are no matches. In the
|
|
case of debuggers we optimized for lookups that fail most of the time.
|
|
|
|
Each table that is defined should have strict rules on exactly what is in the
|
|
accelerator tables and documented so clients can rely on the content.
|
|
|
|
Hash Tables
|
|
^^^^^^^^^^^
|
|
|
|
Standard Hash Tables
|
|
""""""""""""""""""""
|
|
|
|
Typical hash tables have a header, buckets, and each bucket points to the
|
|
bucket contents:
|
|
|
|
.. code-block:: none
|
|
|
|
.------------.
|
|
| HEADER |
|
|
|------------|
|
|
| BUCKETS |
|
|
|------------|
|
|
| DATA |
|
|
`------------'
|
|
|
|
The BUCKETS are an array of offsets to DATA for each hash:
|
|
|
|
.. code-block:: none
|
|
|
|
.------------.
|
|
| 0x00001000 | BUCKETS[0]
|
|
| 0x00002000 | BUCKETS[1]
|
|
| 0x00002200 | BUCKETS[2]
|
|
| 0x000034f0 | BUCKETS[3]
|
|
| | ...
|
|
| 0xXXXXXXXX | BUCKETS[n_buckets]
|
|
'------------'
|
|
|
|
So for ``bucket[3]`` in the example above, we have an offset into the table
|
|
0x000034f0 which points to a chain of entries for the bucket. Each bucket must
|
|
contain a next pointer, full 32 bit hash value, the string itself, and the data
|
|
for the current string value.
|
|
|
|
.. code-block:: none
|
|
|
|
.------------.
|
|
0x000034f0: | 0x00003500 | next pointer
|
|
| 0x12345678 | 32 bit hash
|
|
| "erase" | string value
|
|
| data[n] | HashData for this bucket
|
|
|------------|
|
|
0x00003500: | 0x00003550 | next pointer
|
|
| 0x29273623 | 32 bit hash
|
|
| "dump" | string value
|
|
| data[n] | HashData for this bucket
|
|
|------------|
|
|
0x00003550: | 0x00000000 | next pointer
|
|
| 0x82638293 | 32 bit hash
|
|
| "main" | string value
|
|
| data[n] | HashData for this bucket
|
|
`------------'
|
|
|
|
The problem with this layout for debuggers is that we need to optimize for the
|
|
negative lookup case where the symbol we're searching for is not present. So
|
|
if we were to lookup "``printf``" in the table above, we would make a 32 hash
|
|
for "``printf``", it might match ``bucket[3]``. We would need to go to the
|
|
offset 0x000034f0 and start looking to see if our 32 bit hash matches. To do
|
|
so, we need to read the next pointer, then read the hash, compare it, and skip
|
|
to the next bucket. Each time we are skipping many bytes in memory and
|
|
touching new cache pages just to do the compare on the full 32 bit hash. All
|
|
of these accesses then tell us that we didn't have a match.
|
|
|
|
Name Hash Tables
|
|
""""""""""""""""
|
|
|
|
To solve the issues mentioned above we have structured the hash tables a bit
|
|
differently: a header, buckets, an array of all unique 32 bit hash values,
|
|
followed by an array of hash value data offsets, one for each hash value, then
|
|
the data for all hash values:
|
|
|
|
.. code-block:: none
|
|
|
|
.-------------.
|
|
| HEADER |
|
|
|-------------|
|
|
| BUCKETS |
|
|
|-------------|
|
|
| HASHES |
|
|
|-------------|
|
|
| OFFSETS |
|
|
|-------------|
|
|
| DATA |
|
|
`-------------'
|
|
|
|
The ``BUCKETS`` in the name tables are an index into the ``HASHES`` array. By
|
|
making all of the full 32 bit hash values contiguous in memory, we allow
|
|
ourselves to efficiently check for a match while touching as little memory as
|
|
possible. Most often checking the 32 bit hash values is as far as the lookup
|
|
goes. If it does match, it usually is a match with no collisions. So for a
|
|
table with "``n_buckets``" buckets, and "``n_hashes``" unique 32 bit hash
|
|
values, we can clarify the contents of the ``BUCKETS``, ``HASHES`` and
|
|
``OFFSETS`` as:
|
|
|
|
.. code-block:: none
|
|
|
|
.-------------------------.
|
|
| HEADER.magic | uint32_t
|
|
| HEADER.version | uint16_t
|
|
| HEADER.hash_function | uint16_t
|
|
| HEADER.bucket_count | uint32_t
|
|
| HEADER.hashes_count | uint32_t
|
|
| HEADER.header_data_len | uint32_t
|
|
| HEADER_DATA | HeaderData
|
|
|-------------------------|
|
|
| BUCKETS | uint32_t[n_buckets] // 32 bit hash indexes
|
|
|-------------------------|
|
|
| HASHES | uint32_t[n_hashes] // 32 bit hash values
|
|
|-------------------------|
|
|
| OFFSETS | uint32_t[n_hashes] // 32 bit offsets to hash value data
|
|
|-------------------------|
|
|
| ALL HASH DATA |
|
|
`-------------------------'
|
|
|
|
So taking the exact same data from the standard hash example above we end up
|
|
with:
|
|
|
|
.. code-block:: none
|
|
|
|
.------------.
|
|
| HEADER |
|
|
|------------|
|
|
| 0 | BUCKETS[0]
|
|
| 2 | BUCKETS[1]
|
|
| 5 | BUCKETS[2]
|
|
| 6 | BUCKETS[3]
|
|
| | ...
|
|
| ... | BUCKETS[n_buckets]
|
|
|------------|
|
|
| 0x........ | HASHES[0]
|
|
| 0x........ | HASHES[1]
|
|
| 0x........ | HASHES[2]
|
|
| 0x........ | HASHES[3]
|
|
| 0x........ | HASHES[4]
|
|
| 0x........ | HASHES[5]
|
|
| 0x12345678 | HASHES[6] hash for BUCKETS[3]
|
|
| 0x29273623 | HASHES[7] hash for BUCKETS[3]
|
|
| 0x82638293 | HASHES[8] hash for BUCKETS[3]
|
|
| 0x........ | HASHES[9]
|
|
| 0x........ | HASHES[10]
|
|
| 0x........ | HASHES[11]
|
|
| 0x........ | HASHES[12]
|
|
| 0x........ | HASHES[13]
|
|
| 0x........ | HASHES[n_hashes]
|
|
|------------|
|
|
| 0x........ | OFFSETS[0]
|
|
| 0x........ | OFFSETS[1]
|
|
| 0x........ | OFFSETS[2]
|
|
| 0x........ | OFFSETS[3]
|
|
| 0x........ | OFFSETS[4]
|
|
| 0x........ | OFFSETS[5]
|
|
| 0x000034f0 | OFFSETS[6] offset for BUCKETS[3]
|
|
| 0x00003500 | OFFSETS[7] offset for BUCKETS[3]
|
|
| 0x00003550 | OFFSETS[8] offset for BUCKETS[3]
|
|
| 0x........ | OFFSETS[9]
|
|
| 0x........ | OFFSETS[10]
|
|
| 0x........ | OFFSETS[11]
|
|
| 0x........ | OFFSETS[12]
|
|
| 0x........ | OFFSETS[13]
|
|
| 0x........ | OFFSETS[n_hashes]
|
|
|------------|
|
|
| |
|
|
| |
|
|
| |
|
|
| |
|
|
| |
|
|
|------------|
|
|
0x000034f0: | 0x00001203 | .debug_str ("erase")
|
|
| 0x00000004 | A 32 bit array count - number of HashData with name "erase"
|
|
| 0x........ | HashData[0]
|
|
| 0x........ | HashData[1]
|
|
| 0x........ | HashData[2]
|
|
| 0x........ | HashData[3]
|
|
| 0x00000000 | String offset into .debug_str (terminate data for hash)
|
|
|------------|
|
|
0x00003500: | 0x00001203 | String offset into .debug_str ("collision")
|
|
| 0x00000002 | A 32 bit array count - number of HashData with name "collision"
|
|
| 0x........ | HashData[0]
|
|
| 0x........ | HashData[1]
|
|
| 0x00001203 | String offset into .debug_str ("dump")
|
|
| 0x00000003 | A 32 bit array count - number of HashData with name "dump"
|
|
| 0x........ | HashData[0]
|
|
| 0x........ | HashData[1]
|
|
| 0x........ | HashData[2]
|
|
| 0x00000000 | String offset into .debug_str (terminate data for hash)
|
|
|------------|
|
|
0x00003550: | 0x00001203 | String offset into .debug_str ("main")
|
|
| 0x00000009 | A 32 bit array count - number of HashData with name "main"
|
|
| 0x........ | HashData[0]
|
|
| 0x........ | HashData[1]
|
|
| 0x........ | HashData[2]
|
|
| 0x........ | HashData[3]
|
|
| 0x........ | HashData[4]
|
|
| 0x........ | HashData[5]
|
|
| 0x........ | HashData[6]
|
|
| 0x........ | HashData[7]
|
|
| 0x........ | HashData[8]
|
|
| 0x00000000 | String offset into .debug_str (terminate data for hash)
|
|
`------------'
|
|
|
|
So we still have all of the same data, we just organize it more efficiently for
|
|
debugger lookup. If we repeat the same "``printf``" lookup from above, we
|
|
would hash "``printf``" and find it matches ``BUCKETS[3]`` by taking the 32 bit
|
|
hash value and modulo it by ``n_buckets``. ``BUCKETS[3]`` contains "6" which
|
|
is the index into the ``HASHES`` table. We would then compare any consecutive
|
|
32 bit hashes values in the ``HASHES`` array as long as the hashes would be in
|
|
``BUCKETS[3]``. We do this by verifying that each subsequent hash value modulo
|
|
``n_buckets`` is still 3. In the case of a failed lookup we would access the
|
|
memory for ``BUCKETS[3]``, and then compare a few consecutive 32 bit hashes
|
|
before we know that we have no match. We don't end up marching through
|
|
multiple words of memory and we really keep the number of processor data cache
|
|
lines being accessed as small as possible.
|
|
|
|
The string hash that is used for these lookup tables is the Daniel J.
|
|
Bernstein hash which is also used in the ELF ``GNU_HASH`` sections. It is a
|
|
very good hash for all kinds of names in programs with very few hash
|
|
collisions.
|
|
|
|
Empty buckets are designated by using an invalid hash index of ``UINT32_MAX``.
|
|
|
|
Details
|
|
^^^^^^^
|
|
|
|
These name hash tables are designed to be generic where specializations of the
|
|
table get to define additional data that goes into the header ("``HeaderData``"),
|
|
how the string value is stored ("``KeyType``") and the content of the data for each
|
|
hash value.
|
|
|
|
Header Layout
|
|
"""""""""""""
|
|
|
|
The header has a fixed part, and the specialized part. The exact format of the
|
|
header is:
|
|
|
|
.. code-block:: c
|
|
|
|
struct Header
|
|
{
|
|
uint32_t magic; // 'HASH' magic value to allow endian detection
|
|
uint16_t version; // Version number
|
|
uint16_t hash_function; // The hash function enumeration that was used
|
|
uint32_t bucket_count; // The number of buckets in this hash table
|
|
uint32_t hashes_count; // The total number of unique hash values and hash data offsets in this table
|
|
uint32_t header_data_len; // The bytes to skip to get to the hash indexes (buckets) for correct alignment
|
|
// Specifically the length of the following HeaderData field - this does not
|
|
// include the size of the preceding fields
|
|
HeaderData header_data; // Implementation specific header data
|
|
};
|
|
|
|
The header starts with a 32 bit "``magic``" value which must be ``'HASH'``
|
|
encoded as an ASCII integer. This allows the detection of the start of the
|
|
hash table and also allows the table's byte order to be determined so the table
|
|
can be correctly extracted. The "``magic``" value is followed by a 16 bit
|
|
``version`` number which allows the table to be revised and modified in the
|
|
future. The current version number is 1. ``hash_function`` is a ``uint16_t``
|
|
enumeration that specifies which hash function was used to produce this table.
|
|
The current values for the hash function enumerations include:
|
|
|
|
.. code-block:: c
|
|
|
|
enum HashFunctionType
|
|
{
|
|
eHashFunctionDJB = 0u, // Daniel J Bernstein hash function
|
|
};
|
|
|
|
``bucket_count`` is a 32 bit unsigned integer that represents how many buckets
|
|
are in the ``BUCKETS`` array. ``hashes_count`` is the number of unique 32 bit
|
|
hash values that are in the ``HASHES`` array, and is the same number of offsets
|
|
are contained in the ``OFFSETS`` array. ``header_data_len`` specifies the size
|
|
in bytes of the ``HeaderData`` that is filled in by specialized versions of
|
|
this table.
|
|
|
|
Fixed Lookup
|
|
""""""""""""
|
|
|
|
The header is followed by the buckets, hashes, offsets, and hash value data.
|
|
|
|
.. code-block:: c
|
|
|
|
struct FixedTable
|
|
{
|
|
uint32_t buckets[Header.bucket_count]; // An array of hash indexes into the "hashes[]" array below
|
|
uint32_t hashes [Header.hashes_count]; // Every unique 32 bit hash for the entire table is in this table
|
|
uint32_t offsets[Header.hashes_count]; // An offset that corresponds to each item in the "hashes[]" array above
|
|
};
|
|
|
|
``buckets`` is an array of 32 bit indexes into the ``hashes`` array. The
|
|
``hashes`` array contains all of the 32 bit hash values for all names in the
|
|
hash table. Each hash in the ``hashes`` table has an offset in the ``offsets``
|
|
array that points to the data for the hash value.
|
|
|
|
This table setup makes it very easy to repurpose these tables to contain
|
|
different data, while keeping the lookup mechanism the same for all tables.
|
|
This layout also makes it possible to save the table to disk and map it in
|
|
later and do very efficient name lookups with little or no parsing.
|
|
|
|
DWARF lookup tables can be implemented in a variety of ways and can store a lot
|
|
of information for each name. We want to make the DWARF tables extensible and
|
|
able to store the data efficiently so we have used some of the DWARF features
|
|
that enable efficient data storage to define exactly what kind of data we store
|
|
for each name.
|
|
|
|
The ``HeaderData`` contains a definition of the contents of each HashData chunk.
|
|
We might want to store an offset to all of the debug information entries (DIEs)
|
|
for each name. To keep things extensible, we create a list of items, or
|
|
Atoms, that are contained in the data for each name. First comes the type of
|
|
the data in each atom:
|
|
|
|
.. code-block:: c
|
|
|
|
enum AtomType
|
|
{
|
|
eAtomTypeNULL = 0u,
|
|
eAtomTypeDIEOffset = 1u, // DIE offset, check form for encoding
|
|
eAtomTypeCUOffset = 2u, // DIE offset of the compiler unit header that contains the item in question
|
|
eAtomTypeTag = 3u, // DW_TAG_xxx value, should be encoded as DW_FORM_data1 (if no tags exceed 255) or DW_FORM_data2
|
|
eAtomTypeNameFlags = 4u, // Flags from enum NameFlags
|
|
eAtomTypeTypeFlags = 5u, // Flags from enum TypeFlags
|
|
};
|
|
|
|
The enumeration values and their meanings are:
|
|
|
|
.. code-block:: none
|
|
|
|
eAtomTypeNULL - a termination atom that specifies the end of the atom list
|
|
eAtomTypeDIEOffset - an offset into the .debug_info section for the DWARF DIE for this name
|
|
eAtomTypeCUOffset - an offset into the .debug_info section for the CU that contains the DIE
|
|
eAtomTypeDIETag - The DW_TAG_XXX enumeration value so you don't have to parse the DWARF to see what it is
|
|
eAtomTypeNameFlags - Flags for functions and global variables (isFunction, isInlined, isExternal...)
|
|
eAtomTypeTypeFlags - Flags for types (isCXXClass, isObjCClass, ...)
|
|
|
|
Then we allow each atom type to define the atom type and how the data for each
|
|
atom type data is encoded:
|
|
|
|
.. code-block:: c
|
|
|
|
struct Atom
|
|
{
|
|
uint16_t type; // AtomType enum value
|
|
uint16_t form; // DWARF DW_FORM_XXX defines
|
|
};
|
|
|
|
The ``form`` type above is from the DWARF specification and defines the exact
|
|
encoding of the data for the Atom type. See the DWARF specification for the
|
|
``DW_FORM_`` definitions.
|
|
|
|
.. code-block:: c
|
|
|
|
struct HeaderData
|
|
{
|
|
uint32_t die_offset_base;
|
|
uint32_t atom_count;
|
|
Atoms atoms[atom_count0];
|
|
};
|
|
|
|
``HeaderData`` defines the base DIE offset that should be added to any atoms
|
|
that are encoded using the ``DW_FORM_ref1``, ``DW_FORM_ref2``,
|
|
``DW_FORM_ref4``, ``DW_FORM_ref8`` or ``DW_FORM_ref_udata``. It also defines
|
|
what is contained in each ``HashData`` object -- ``Atom.form`` tells us how large
|
|
each field will be in the ``HashData`` and the ``Atom.type`` tells us how this data
|
|
should be interpreted.
|
|
|
|
For the current implementations of the "``.apple_names``" (all functions +
|
|
globals), the "``.apple_types``" (names of all types that are defined), and
|
|
the "``.apple_namespaces``" (all namespaces), we currently set the ``Atom``
|
|
array to be:
|
|
|
|
.. code-block:: c
|
|
|
|
HeaderData.atom_count = 1;
|
|
HeaderData.atoms[0].type = eAtomTypeDIEOffset;
|
|
HeaderData.atoms[0].form = DW_FORM_data4;
|
|
|
|
This defines the contents to be the DIE offset (eAtomTypeDIEOffset) that is
|
|
encoded as a 32 bit value (DW_FORM_data4). This allows a single name to have
|
|
multiple matching DIEs in a single file, which could come up with an inlined
|
|
function for instance. Future tables could include more information about the
|
|
DIE such as flags indicating if the DIE is a function, method, block,
|
|
or inlined.
|
|
|
|
The KeyType for the DWARF table is a 32 bit string table offset into the
|
|
".debug_str" table. The ".debug_str" is the string table for the DWARF which
|
|
may already contain copies of all of the strings. This helps make sure, with
|
|
help from the compiler, that we reuse the strings between all of the DWARF
|
|
sections and keeps the hash table size down. Another benefit to having the
|
|
compiler generate all strings as DW_FORM_strp in the debug info, is that
|
|
DWARF parsing can be made much faster.
|
|
|
|
After a lookup is made, we get an offset into the hash data. The hash data
|
|
needs to be able to deal with 32 bit hash collisions, so the chunk of data
|
|
at the offset in the hash data consists of a triple:
|
|
|
|
.. code-block:: c
|
|
|
|
uint32_t str_offset
|
|
uint32_t hash_data_count
|
|
HashData[hash_data_count]
|
|
|
|
If "str_offset" is zero, then the bucket contents are done. 99.9% of the
|
|
hash data chunks contain a single item (no 32 bit hash collision):
|
|
|
|
.. code-block:: none
|
|
|
|
.------------.
|
|
| 0x00001023 | uint32_t KeyType (.debug_str[0x0001023] => "main")
|
|
| 0x00000004 | uint32_t HashData count
|
|
| 0x........ | uint32_t HashData[0] DIE offset
|
|
| 0x........ | uint32_t HashData[1] DIE offset
|
|
| 0x........ | uint32_t HashData[2] DIE offset
|
|
| 0x........ | uint32_t HashData[3] DIE offset
|
|
| 0x00000000 | uint32_t KeyType (end of hash chain)
|
|
`------------'
|
|
|
|
If there are collisions, you will have multiple valid string offsets:
|
|
|
|
.. code-block:: none
|
|
|
|
.------------.
|
|
| 0x00001023 | uint32_t KeyType (.debug_str[0x0001023] => "main")
|
|
| 0x00000004 | uint32_t HashData count
|
|
| 0x........ | uint32_t HashData[0] DIE offset
|
|
| 0x........ | uint32_t HashData[1] DIE offset
|
|
| 0x........ | uint32_t HashData[2] DIE offset
|
|
| 0x........ | uint32_t HashData[3] DIE offset
|
|
| 0x00002023 | uint32_t KeyType (.debug_str[0x0002023] => "print")
|
|
| 0x00000002 | uint32_t HashData count
|
|
| 0x........ | uint32_t HashData[0] DIE offset
|
|
| 0x........ | uint32_t HashData[1] DIE offset
|
|
| 0x00000000 | uint32_t KeyType (end of hash chain)
|
|
`------------'
|
|
|
|
Current testing with real world C++ binaries has shown that there is around 1
|
|
32 bit hash collision per 100,000 name entries.
|
|
|
|
Contents
|
|
^^^^^^^^
|
|
|
|
As we said, we want to strictly define exactly what is included in the
|
|
different tables. For DWARF, we have 3 tables: "``.apple_names``",
|
|
"``.apple_types``", and "``.apple_namespaces``".
|
|
|
|
"``.apple_names``" sections should contain an entry for each DWARF DIE whose
|
|
``DW_TAG`` is a ``DW_TAG_label``, ``DW_TAG_inlined_subroutine``, or
|
|
``DW_TAG_subprogram`` that has address attributes: ``DW_AT_low_pc``,
|
|
``DW_AT_high_pc``, ``DW_AT_ranges`` or ``DW_AT_entry_pc``. It also contains
|
|
``DW_TAG_variable`` DIEs that have a ``DW_OP_addr`` in the location (global and
|
|
static variables). All global and static variables should be included,
|
|
including those scoped within functions and classes. For example using the
|
|
following code:
|
|
|
|
.. code-block:: c
|
|
|
|
static int var = 0;
|
|
|
|
void f ()
|
|
{
|
|
static int var = 0;
|
|
}
|
|
|
|
Both of the static ``var`` variables would be included in the table. All
|
|
functions should emit both their full names and their basenames. For C or C++,
|
|
the full name is the mangled name (if available) which is usually in the
|
|
``DW_AT_MIPS_linkage_name`` attribute, and the ``DW_AT_name`` contains the
|
|
function basename. If global or static variables have a mangled name in a
|
|
``DW_AT_MIPS_linkage_name`` attribute, this should be emitted along with the
|
|
simple name found in the ``DW_AT_name`` attribute.
|
|
|
|
"``.apple_types``" sections should contain an entry for each DWARF DIE whose
|
|
tag is one of:
|
|
|
|
* DW_TAG_array_type
|
|
* DW_TAG_class_type
|
|
* DW_TAG_enumeration_type
|
|
* DW_TAG_pointer_type
|
|
* DW_TAG_reference_type
|
|
* DW_TAG_string_type
|
|
* DW_TAG_structure_type
|
|
* DW_TAG_subroutine_type
|
|
* DW_TAG_typedef
|
|
* DW_TAG_union_type
|
|
* DW_TAG_ptr_to_member_type
|
|
* DW_TAG_set_type
|
|
* DW_TAG_subrange_type
|
|
* DW_TAG_base_type
|
|
* DW_TAG_const_type
|
|
* DW_TAG_file_type
|
|
* DW_TAG_namelist
|
|
* DW_TAG_packed_type
|
|
* DW_TAG_volatile_type
|
|
* DW_TAG_restrict_type
|
|
* DW_TAG_interface_type
|
|
* DW_TAG_unspecified_type
|
|
* DW_TAG_shared_type
|
|
|
|
Only entries with a ``DW_AT_name`` attribute are included, and the entry must
|
|
not be a forward declaration (``DW_AT_declaration`` attribute with a non-zero
|
|
value). For example, using the following code:
|
|
|
|
.. code-block:: c
|
|
|
|
int main ()
|
|
{
|
|
int *b = 0;
|
|
return *b;
|
|
}
|
|
|
|
We get a few type DIEs:
|
|
|
|
.. code-block:: none
|
|
|
|
0x00000067: TAG_base_type [5]
|
|
AT_encoding( DW_ATE_signed )
|
|
AT_name( "int" )
|
|
AT_byte_size( 0x04 )
|
|
|
|
0x0000006e: TAG_pointer_type [6]
|
|
AT_type( {0x00000067} ( int ) )
|
|
AT_byte_size( 0x08 )
|
|
|
|
The DW_TAG_pointer_type is not included because it does not have a ``DW_AT_name``.
|
|
|
|
"``.apple_namespaces``" section should contain all ``DW_TAG_namespace`` DIEs.
|
|
If we run into a namespace that has no name this is an anonymous namespace, and
|
|
the name should be output as "``(anonymous namespace)``" (without the quotes).
|
|
Why? This matches the output of the ``abi::cxa_demangle()`` that is in the
|
|
standard C++ library that demangles mangled names.
|
|
|
|
|
|
Language Extensions and File Format Changes
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Objective-C Extensions
|
|
""""""""""""""""""""""
|
|
|
|
"``.apple_objc``" section should contain all ``DW_TAG_subprogram`` DIEs for an
|
|
Objective-C class. The name used in the hash table is the name of the
|
|
Objective-C class itself. If the Objective-C class has a category, then an
|
|
entry is made for both the class name without the category, and for the class
|
|
name with the category. So if we have a DIE at offset 0x1234 with a name of
|
|
method "``-[NSString(my_additions) stringWithSpecialString:]``", we would add
|
|
an entry for "``NSString``" that points to DIE 0x1234, and an entry for
|
|
"``NSString(my_additions)``" that points to 0x1234. This allows us to quickly
|
|
track down all Objective-C methods for an Objective-C class when doing
|
|
expressions. It is needed because of the dynamic nature of Objective-C where
|
|
anyone can add methods to a class. The DWARF for Objective-C methods is also
|
|
emitted differently from C++ classes where the methods are not usually
|
|
contained in the class definition, they are scattered about across one or more
|
|
compile units. Categories can also be defined in different shared libraries.
|
|
So we need to be able to quickly find all of the methods and class functions
|
|
given the Objective-C class name, or quickly find all methods and class
|
|
functions for a class + category name. This table does not contain any
|
|
selector names, it just maps Objective-C class names (or class names +
|
|
category) to all of the methods and class functions. The selectors are added
|
|
as function basenames in the "``.debug_names``" section.
|
|
|
|
In the "``.apple_names``" section for Objective-C functions, the full name is
|
|
the entire function name with the brackets ("``-[NSString
|
|
stringWithCString:]``") and the basename is the selector only
|
|
("``stringWithCString:``").
|
|
|
|
Mach-O Changes
|
|
""""""""""""""
|
|
|
|
The sections names for the apple hash tables are for non-mach-o files. For
|
|
mach-o files, the sections should be contained in the ``__DWARF`` segment with
|
|
names as follows:
|
|
|
|
* "``.apple_names``" -> "``__apple_names``"
|
|
* "``.apple_types``" -> "``__apple_types``"
|
|
* "``.apple_namespaces``" -> "``__apple_namespac``" (16 character limit)
|
|
* "``.apple_objc``" -> "``__apple_objc``"
|
|
|