From 82c4671f86ec80eae2da6962eef7f48c961a161f Mon Sep 17 00:00:00 2001 From: Reid Spencer Date: Wed, 7 Jul 2004 13:34:26 +0000 Subject: [PATCH] An update with corrections to content as well as using a regex style notation that Chris' suggested to make the specification more compact and succinct. Added a section to Describe the notation, made the VBR description its own section, and otherwise generally cleaned things up. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@14668 91177308-0d34-0410-b5e6-96231b3b80d8 --- docs/BytecodeFormat.html | 481 +++++++++++++++++++++------------------ 1 file changed, 261 insertions(+), 220 deletions(-) diff --git a/docs/BytecodeFormat.html b/docs/BytecodeFormat.html index 05742fd6468..72afd5b4eb9 100644 --- a/docs/BytecodeFormat.html +++ b/docs/BytecodeFormat.html @@ -22,6 +22,7 @@
  • Lists
  • Fields
  • Alignment
    • +
  • Variable Bit-Rate Encoding
  • Encoding Primitives
  • Slots
    • @@ -57,11 +58,15 @@
    Abstract
    -

    This document describes the LLVM bytecode file format as of version 1.3. -It specifies the binary encoding rules of the bytecode file format -so that equivalent systems can encode bytecode files correctly. The LLVM -bytecode representation is used to store the intermediate representation on -disk in compacted form. +

    This document describes the LLVM bytecode file format. It specifies the + binary encoding rules of the bytecode file format so that equivalent systems + can encode bytecode files correctly. The LLVM bytecode representation is + used to store the intermediate representation on disk in compacted form.

    +

    The LLVM bytecode format may change in the future, but LLVM will always be + backwards compatible with older formats. This document will only describe + the most current version of the bytecode format. See + Version Differences for the details on how the + current version is different from previous versions.

    @@ -69,28 +74,26 @@ disk in compacted form.
    Concepts
    -

    This section describes the general concepts of the bytecode file format -without getting into bit and byte level specifics. Note that the LLVM bytecode -format may change in the future, but will always be backwards compatible with -older formats. This document only describes the most current version of the -bytecode format.

    +

    This section describes the general concepts of the bytecode file format + without getting into specific layout details. It is recommended that you read + this section thoroughly before interpreting the detailed descriptions.

    Blocks
    -

    LLVM bytecode files consist simply of a sequence of blocks of bytes. -Each block begins with an header of two unsigned integers. The first value -identifies the type of block and the second value provides the size of the -block in bytes. The block identifier is used because it is possible for entire -blocks to be omitted from the file if they are empty. The block identifier helps -the reader determine which kind of block is next in the file. Note that blocks -can be nested within other blocks.

    -

    All blocks are variable length, and the block header specifies the size of -the block. All blocks begin on a byte index that is aligned to an even 32-bit -boundary. That is, the first block is 32-bit aligned because it starts at offset -0. Each block is padded with zero fill bytes to ensure that the next block also -starts on a 32-bit boundary.

    +

    LLVM bytecode files consist simply of a sequence of blocks of bytes using + a binary encoding Each block begins with an header of two unsigned integers. + The first value identifies the type of block and the second value provides + the size of the block in bytes. The block identifier is used because it is + possible for entire blocks to be omitted from the file if they are empty. + The block identifier helps the reader determine which kind of block is next + in the file. Note that blocks can be nested within other blocks.

    +

    All blocks are variable length, and the block header specifies the size + of the block. All blocks begin on a byte index that is aligned to an even + 32-bit boundary. That is, the first block is 32-bit aligned because it + starts at offset 0. Each block is padded with zero fill bytes to ensure that + the next block also starts on a 32-bit boundary.

    @@ -99,16 +102,9 @@ starts on a 32-bit boundary.

    LLVM Bytecode blocks often contain lists of things of a similar type. For example, a function contains a list of instructions and a function type contains a list of argument types. There are two basic types of lists: - length lists, and null terminated lists, as described here:

    - + length lists (llist), and null terminated lists + (zlist), as described below in the + Encoding Primitives.

    @@ -136,11 +132,8 @@ written and how the bits are to be interpreted.

    -
    Encoding Primitives
    +
    Variable Bit-Rate Encoding
    -

    Each field that can be put out is encoded into the file using a small set -of primitives. The rules for these primitives are described below.

    -

    Variable Bit Rate Encoding

    Most of the values written to LLVM bytecode files are small integers. To minimize the number of bytes written for these quantities, an encoding scheme similar to UTF-8 is used to write integer data. The scheme is known as @@ -177,52 +170,74 @@ with the sign bit as the low order bit instead of the high order bit. This allows small negative quantities to be encoded efficiently. For example, -3 is encoded as "((3 << 1) | 1)" and 3 is encoded as "(3 << 1) | 0)", emitted with the standard vbr encoding above.

    +
    -

    The table below defines the encoding rules for type names used in the -descriptions of blocks and fields in the next section. Any type name with -the suffix _vbr indicate a quantity that is encoded using -variable bit rate encoding as described above.

    + +
    Encoding Primitives
    +
    +

    Each field in the bytecode format is encoded into the file using a small + set of primitive formats. The table below defines the encoding rules for the + various primitives used and gives them each a type name. The type names used + in the descriptions of blocks and fields in the Detailed + Layoutnext section. Any type name with the suffix _vbr indicates + a quantity that is encoded using variable bit rate encoding as described + above.

    - + - + - + - + - + - - + + - + - + - + + + + + + +
    Type Rule
    unsignedunsigned A 32-bit unsigned integer that always occupies four consecutive bytes. The unsigned integer is encoded using LSB first ordering. That is bits 20 through 27 are in the byte with the lowest file offset (little endian).
    uint32_vbruint32_vbr A 32-bit unsigned integer that occupies from one to five bytes using variable bit rate encoding.
    uint64_vbruint64_vbr A 64-bit unsigned integer that occupies from one to ten bytes using variable bit rate encoding.
    int64_vbrint64_vbr A 64-bit signed integer that occupies from one to ten bytes using the signed variable bit rate encoding.
    charchar A single unsigned character encoded into one byte
    bitA single bit within some larger integer field.bit(n-m)A set of bit within some larger integer field. The + values of n and m specify the inclusive range + of bits that define the subfield. The value for m may be + omitted if its the same as n.
    stringstring A uint32_vbr indicating the type of the constant string which also includes its length, immediately followed by the characters of the string. There is no terminating null byte in the string.
    datadata An arbitrarily long segment of data to which no interpretation is implied. This is used for float, double, and constant initializers.
    blockllist(x)A length list of x. This means the list is encoded as + an uint32_vbr providing the length of the list, + followed by a sequence of that many "x" items. This implies that the reader + should iterate the number of times provided by the length.
    zlist(x)A zero-terminated list of x. This means the list is encoded + as a sequence of an indeterminate number of "x" items, followed by an + uint32_vbr terminating value. This implies that none + of the "x" items can have a zero value (or else the list terminates).
    block A block of data that is logically related. A block begins with an unsigned that provides the block identifier (constant value) and an unsigned that @@ -232,6 +247,56 @@ variable bit rate encoding as described above.

    + +
    Field Notation
    +
    +

    In the detailed block and field descriptions that follow, a regex like + notation is used to describe optional and repeated fields. A very limited + subset of regex is used to describe these, as given in the following table: +

    + + + + + + + + + + + + + + + + + + + + +
    CharacterMeaning
    ?The question mark indicates 0 or 1 occurrences of + the thing preceding it.
    *The asterisk indicates 0 or more occurrences of the + thing preceding it.
    +The plus sign indicates 1 or more occurrences of the + thing preceding it.
    ()Parentheses are used for grouping.
    ,The comma separates sequential fields.
    +

    So, for example, consider the following specifications:

    +
    +
      +
    1. string?
      2. +
    2. (uint32_vbr,uin32_vbr)+
      3. +
    3. (unsigned?,uint32_vbr)*
      4. +
    4. (llist(unsigned))?
      5. +
    +
    +

    with the following interpretations:

    +
      +
    1. An optional string. Matches either nothing or a single string
      2. +
    2. One or more pairs of uint32_vbr.
      3. +
    3. Zero or more occurrences of either an unsigned followed by a uint32_vbr + or just a uint32_vbr.
      4. +
    4. An optional length list of unsigned values.
      5. +
    +
    +
    Slots
    @@ -240,10 +305,10 @@ Values. Since the bytecode file is a direct representation of LLVM's intermediate representation, there is a need to represent pointers in the file. Slots are used for this purpose. For example, if one has the following assembly:

    -
    +
    %MyType = type { int, sbyte }
    %MyVar = external global %MyType -
    +

    there are two definitions. The definition of %MyVar uses %MyType. In the C++ IR this linkage between %MyVar and %MyType is @@ -276,7 +341,7 @@ This is exactly what the compaction table does.

    General Structure
    -

    This section provides the general structur of the LLVM bytecode file +

    This section provides the general structure of the LLVM bytecode file format. The bytecode file format requires blocks to be in a certain order and nested in a particular way so that an LLVM module can be constructed efficiently from the contents of the file. This ordering defines a general @@ -321,7 +386,7 @@ This is exactly what the compaction table does.

    except function arguments, global values and constant strings. 0x11ModuleYesYes1 -    Function Definitions +    Function Definitions* One function block is written for each function in the module. The function block contains the instructions, compaction table, type constant pool, and symbol table for the function. @@ -356,8 +421,7 @@ This is exactly what the compaction table does.

    functions mostly). -

    Use the links in the table or see Block Types for -details about the contents of each of the block types.

    +

    Use the links in the table for details about the contents of each of the block types.

    @@ -427,7 +491,7 @@ sections.

    block Module Constant Pool - block + block* Function Definitions block @@ -443,24 +507,23 @@ sections.

    integer as shown in the following table.

    - - - + + - - + + - - + + - - + + - - + +
    Bit(s) Type Description
    0bitBig Endian?bit(0)Target is big endian?
    1bitPointers Are 64-bit?bit(1)On target pointers are 64-bit?
    2bitHas No Endianess?bit(2)Target has no endianess?
    3bitHas No Pointer Size?bit(3)Target has no pointer size?
    4-31bitBytecode Format Versionbit(4-31)Bytecode format version

    @@ -503,24 +566,16 @@ below.

    unsigned Size in bytes of the type pool block. - uint32_vbr - Number of type definitions that follow in the next - field. - - type - Each of the type definitions (see below)1 + llist(type) + A length list of type definitions. -Notes: -
      -
    1. Repeated field.
      2. -
    Type Definitions
    -

    Types in the type pool are defined using a different format for each -basic type of type as given in the following sections.

    +

    Types in the type pool are defined using a different format for each kind +of type, as given in the following sections.

    Primitive Types

    The primitive types encompass the basic integer and floating point types

    @@ -528,14 +583,29 @@ basic type of type as given in the following sections.

    - - + +
    Type Description
    uint32_vbrType ID For The Primitive (1-11)1uint32_vbrType ID for the primitive types (values 1 to 11) + 1
    Notes:
      -
    1. See the definition of Type::TypeID in Type.h for the numeric equivalents - of the primitive type ids.
      2. +
    2. The values for the Type IDs for the primitive types are provided by the + definition of the llvm::Type::TypeID enumeration in + include/llvm/Type.h. The enumeration gives the following + mapping:
        +
      1. bool
        2. +
      2. ubyte
        3. +
      3. sbyte
        4. +
      4. ushort
        5. +
      5. short
        6. +
      6. uint
        7. +
      7. int
        8. +
      8. ulong
        9. +
      9. long
        10. +
      10. float
        11. +
      11. double
        12. +

    Function Types

    @@ -543,60 +613,45 @@ Notes: - + - + - - + + - - - - - + +
    Type Description
    uint32_vbruint32_vbr Type ID for function types (13)
    uint32_vbruint32_vbr Slot number of function's return type.
    uint32_vbrThe number of arguments in the function.llist(uint32_vbr)Slot number of each argument's type.
    uint32_vbrSlot number of each argument's type.1
    uint32_vbrValue 0 if this is a varargs function.2uint32_vbr?Value 0 if this is a varargs function, missing otherwise.
    -Notes: -
      -
    1. Repeated field.
      2. -
    2. Optional field.
      3. -

    Structure Types

    - + - - - - - + +
    Type Description
    uint32_vbruint32_vbr Type ID for structure types (14)
    uint32_vbrSlot number of each of the element's fields.1
    uint32_vbrNull Terminator (VoidTy type id)zlist(uint32_vbr)Slot number of each of the element's fields.
    -Notes: -
      -
    1. Repeatable field.
      2. -

    Array Types

    - + - + - +
    Type Description
    uint32_vbruint32_vbr Type ID for Array Types (15)
    uint32_vbruint32_vbr Slot number of array's element type.
    uint32_vbruint32_vbr The number of elements in the array.
    @@ -606,10 +661,10 @@ Notes: Type Description - uint32_vbr + uint32_vbr Type ID For Pointer Types (16) - uint32_vbr + uint32_vbr Slot number of pointer's element type. @@ -619,7 +674,7 @@ Notes: Type Description - uint32_vbr + uint32_vbr Type ID For Opaque Types (17) @@ -641,70 +696,60 @@ Notes: unsigned Size in bytes of the module global info block. - globalvar - Definition of the global variable (see below). - 1 - + zlist(globalvar) + A zero terminated list of global var definitions + occuring in the module. - uint32_vbr - Slot number of the global variable's constant - initializer.1,2 - - - uint32_vbr - Zero. This terminates the list of global variables. - - - uint32_vbr - Type slot number of a function defined in this - bytecode file.3 - - - uint32_vbr - Zero. This terminates the list of function - declarations. + zlist(uint32_vbr) + A zero terminated list of function types occuring in + the module. - Notes:
      -
    1. Both these fields are repeatable but in pairs.
      2. -
    2. Optional field.
      3. -
    3. Repeatable field.
      4. -
    Global Variable Field
    -

    Global variables are written using a single - uint32_vbr that encodes information about the global - variable. The table below provides the bit layout of the value written for - each global variable.

    +

    Global variables are written using an uint32_vbr + that encodes information about the global variable and a list of the constant + initializers for the global var, if any.

    +

    The table below provides the bit layout of the first + uint32_vbr that describes the global variable.

    - - + - - + + - + - +
    Bit(s) Type Description
    0bitbit(0) Is constant?
    1bitHas initializer?1bit(1)Has initializer? Note that this bit determines whether + the constant initializer field (described below) follows.
    2-4enumerationbit(2-4) Linkage type: 0=External, 1=Weak, 2=Appending, 3=Internal, 4=LinkOnce
    5-31type slotbit(5-31) Slot number of type for the global variable.
    - Notes: -
      -
    1. This bit determines whether the constant initializer field follows - immediately after this field
      2. -
    +

    The table below provides the format of the constant initializers for the + global variable field, if it has one.

    + + + + + + + + +
    TypeDescription
    (zlist(uint32_vbr))? + + An optional zero-terminated list of slot numbers of + the global variable's constant initializer.
    @@ -714,7 +759,7 @@ Notes: types of constant pool blocks: one for modules and one for functions. For modules, the block begins with the constant strings encountered anywhere in the module. For functions, the block begins with types only encountered in - the function. In both cases the header is identical. The tables the follow, + the function. In both cases the header is identical. The tables that follow, show the header, module constant pool preamble, function constant pool preamble, and the part common to both function and module constant pools.

    Common Block Header

    @@ -725,6 +770,9 @@ Notes: unsigned Constant pool identifier (0x12) + + unsigned + Size in bytes of the constant pool block.

    Module Constant Pool Preamble (constant strings)

    @@ -738,19 +786,17 @@ Notes: uint32_vbr Zero. This identifies the following "plane" as - containing the constant strings. + containing the constant strings. This is needed to identify it + uniquely from other constant planes that follow. - string - Slot number of the constant string's type which - includes the length of the string.1 + uint32_vbr+ + Slot number of the constant string's type. Note + that the constant string's type implicitly defines the length of + the string. - Notes: -
      -
    1. Repeated field.
      2. -

    Function Constant Pool Preamble (function types)

    The structure of the types for functions is identical to the Global Type Pool. Please refer to that section @@ -767,7 +813,7 @@ Notes: uint32_vbr Type slot number of this plane. - constant + constant+ The definition of a constant (see below). @@ -825,33 +871,40 @@ Notes:

    Function Definition
    -

    To be determined.

    +

    Function definitions contain the linkage, constant pool or compaction + table, instruction list, and symbol table for a function. The following table + shows the structure of a function definition.

    + + + + + + - - + + - - + + - - + + - - + +
    Type Field Description
    unsignedFunction definition block identifier (0x11)
    unsignedSize in bytes of the function definition block.
    uint32_vbr The linkage type of the function: 0=External, 1=Weak, 2=Appending, 3=Internal, 4=LinkOnce1
    constant poolThe constant pool block for this function. - 2 - blockThe constant pool block + for this function.2
    compaction tableThe compaction table block for the function. - 2 - blockThe compaction table + block for the function.2
    instruction listThe list of instructions in the function.blockThe instruction list + for the function.
    symbol tableThe function's slot table containing only those - symbols pertinent to the function (mostly block labels). - blockThe function's symbol table + containing only those symbols pertinent to the function (mostly + block labels).
    Notes:
      @@ -869,13 +922,14 @@ Notes: device for reducing the size of bytecode files. The size of a bytecode file is dependent on the value of the slot numbers used because larger values use more bytes in the variable bit rate encoding scheme. - Furthermore, the compresses instruction format reserves only six bits for + Furthermore, the compressed instruction format reserves only six bits for the type of the instruction. In large modules, declaring hundreds or thousands of types, the values of the slot numbers can be quite large. However, functions may use only a small fraction of the global types. In such cases a compaction table is created that maps the global type and value slot - numbers to smaller values used by a function. Compaction tables have the - format shown in the table below.

      + numbers to smaller values used by a function. Functions will contain either + a function-specific constant pool or a compaction table but not + both. Compaction tables have the format shown in the table below.

      @@ -884,26 +938,23 @@ Notes: - + + this entry in the compaction table. + follow. This field's encoding varies depending on the size of + the type plane. See Type and Length for + further details. - + + the compaction table
      Typeuint32_vbr The number of types that follow
      uint32_vbruint32_vbr+ The slot number in the global type plane of the type that will be referenced in the function with the index of - this entry in the compaction table.1
      type_len An encoding of the type and number of values that - follow.2
      uint32_vbruint32_vbr+ The slot number in the globals of the value that will be referenced in the function with the index of this entry in - the compaction table1
      - Notes:
        -
      1. Repeated field.
        2. -
      2. This field's encoding varies depending on the size of the type plane. - See Type and Length for further details. -
    @@ -935,15 +986,11 @@ Notes: unsigned Size in bytes of the instruction list. - instruction - An instruction.1 + instruction+ + An instruction. Instructions have a variety of formats. + See Instructions for details. - Notes: -
      -
    1. A repeated field with a variety of formats. See - Instructions for details.
      2. -
    @@ -975,13 +1022,12 @@ Notes: uint32_vbr The number of operands that follow. - uint32_vbr - The slot number of the value for the operand(s). - 1,2 + uint32_vbr+ + The slot number of the value(s) for the operand(s). + 1 Notes:
      -
    1. Repeatable field (limit given by previous field).
    2. Note that if the instruction is a getelementptr and the type of the operand is a sequential type (array or pointer) then the slot number is shifted up two bits and the low order bits will encode the type of index @@ -1003,7 +1049,7 @@ Notes: 2-7opcode Specifies the opcode of the instruction. Note that - the maximum opcode value si 63. + the maximum opcode value is 63. 8-19unsigned Specifies the slot number of the type for this @@ -1031,7 +1077,7 @@ Notes: 2-7opcode Specifies the opcode of the instruction. Note that - the maximum opcode value si 63. + the maximum opcode value is 63. 8-15unsigned Specifies the slot number of the type for this @@ -1062,7 +1108,7 @@ Notes: 2-7opcode Specifies the opcode of the instruction. Note that - the maximum opcode value si 63. + the maximum opcode value is 63. 8-13unsigned Specifies the slot number of the type for this @@ -1106,19 +1152,14 @@ format is given in the table below.

      uint32_vbr Number of entries in type plane - symtab_entry - Provides the slot number of the type and its name. - 1 + symtab_entry* + Provides the slot number of the type and its name. - symtab_plane + symtab_plane* A type plane containing value slot number and name - for all values of the same type.1 + for all values of the same type. -Notes: -
        -
      1. Repeated field.
        2. -
      @@ -1138,8 +1179,8 @@ Notes: uint32_vbr Slot number of type for this plane. - symtab_entry - The symbol table entries for this plane (repeated). + symtab_entry+ + The symbol table entries for this plane. @@ -1163,8 +1204,8 @@ Notes: uint32_vbr Length of the character array that follows. - char - The characters of the name (repeated). + char+ + The characters of the name.