## Binary ][ Specification -- Initial Release

This is an older version of the specification. Please use the File Type Note instead.

Back to nulib.com library

                               Binary ][ protocol

                                  developed by
                                 Gary B. Little

     Version History
     ---------------
     November 24, 1986  : Initial release.

     Background
     ----------
     Transferring Apple II files in binary form to commercial information
     services like CompuServe, Delphi, GEnie, and The Source is, to put it
     mildly, a frustrating exercise. (For convenience, I'll refer to such
     services, and any other non-Apple II systems, as "hosts.") Although
     most hosts are able to receive a file's *data* in binary form (using
     the Xmodem protocol, for example), they don't receive the file's all-
     important attribute bytes. All the common Apple II operating systems,
     notably ProDOS, store the attributes inside the disk directory, not
     inside the file itself.

     The ProDOS attributes are the access code, file type code, auxiliary
     type code, storage type code, date of creation and last modification,
     time of creation and last modification, the file size, and the name of
     the file itself. (All these terms are defined in Apple's "ProDOS
     Technical Reference Manual" or in the book "Apple ProDOS: Advanced
     Features for Programmers" by Gary Little.) It is usually not possible
     to use a ProDOS file's data without knowing what the file's attributes
     are (particularly the file type code, auxiliary type code, and size).
     This means ProDOS files uploaded in binary form to a host are useless
     to those who download them. The same is true for DOS 3.3 and Pascal
     files.

     Most Apple II communications programs use special protocols for
     transferring file attributes during a binary file transfer, but none
     of these protocols have been implemented by hosts. These programs are
     only useful for exchanging files with another Apple II running the
     same program.

     At present, the only acceptable way to transfer an Apple II file to a
     host is to convert it into lines of text and send it as a textfile.
     Such a textfile would contain a listing of an Applesoft program, or a
     series of Apple II system monitor "enter" commands (e.g., 0300:A4 32
     etc.). Someone downloading such a file can convert it to binary form
     using the Applesoft EXEC command.

     The main disadvantage of this technique is that the text version of
     the file is over three times the size of the original binary file,
     making it expensive (in terms of time and $$$) to upload and download.
     It is also awkward, and sometimes impossible, to perform the binary-
     to-text or text-to-binary conversion.

     The solution to the problem is to upload an encoded binary file which
     contains not just the file's data, but the file's attributes as well.
     Someone downloading such a file, say using Xmodem, can then use a
     conversion program to strip the attributes from the file and create a
     file with the required attributes.

     To make this technique truly useful, however, the Apple II community
     must agree on a format for this encoded binary file. A variety of
     incompatible formats, all achieving the same general result, cannot be
     allowed to appear.

     It is proposed that the Binary II format described in this document be
     adopted. What follows is a description of the Binary II format in
     sufficient detail to allow software developers to implement it in
     Apple II communications programs.

     The Binary II File Format
     -------------------------
     The Binary II form of a standard file consists of a 128-byte file
     information header followed by the file's data. The data portion of
     the file is padded with nulls ($00 bytes), if necessary, to ensure the
     data length is an even multiple of 128. As a result, the Binary II
     form of a file is never more than 255 bytes longer than the original
     file.

     The file information header contains four ID bytes, the attributes of
     the file (in ProDOS 8 form), and some control information. Here is the
     structure of the header:

           Offset  Length                  Contents
           ------  ------   ---------------------------------------
            +0       1      ID byte: always $0A
            +1       1      ID byte: always $47
            +2       1      ID byte: always $4C
            +3       1      access code
            +4       1      file type code
            +5       2      auxiliary type code
            +7       1      storage type code
            +8       2      size of file in 512-byte blocks
            +10      2      date of modification
            +12      2      time of modification
            +14      2      date of creation
            +16      2      time of creation
            +18      1      ID byte: always $02
            +19      1      [reserved]
            +20      3      end-of-file (EOF) position
            +23      1      length of filename/partial pathname
            +24      64     ASCII filename or partial pathname
            +88      23     [reserved, must be zero]
            +111     1      ProDOS 16 access code (high)
            +112     1      ProDOS 16 file type code (high)
            +113     1      ProDOS 16 storage type code (high)
            +114     2      ProDOS 16 size of file in blocks (high)
            +116     1      ProDOS 16 end-of-file position (high)
            +117     4      disk space needed
            +121     1      operating system type
            +122     2      native file type code
            +124     1      phantom file flag
            +125     1      data flags
            +126     1      Binary II version number
            +127     1      number of files to follow

     Multi-byte numeric quantities are stored with their low-order bytes
     first, the same order expected by ProDOS. All reserved bytes must be
     set to zero; they may be used in future versions of the protocol.

     To determine the values of the attributes to be put into a file
     information header for a ProDOS file, you can use the ProDOS
     GET_FILE_INFO and GET_EOF MLI commands.

        Note: Some file attributes returned by ProDOS 16 commands
              are one or two bytes longer than the attributes
              returned by the corresponding ProDOS 8 commands. At
              present, these extra bytes are always zero, and
              probably will remain zero forever. In any event,
              place the extra bytes returned by ProDOS 16 in the
              header at +114 to +119. ProDOS 8 communications
              programs should zero these header locations.

     The "disk space needed" bytes contain the number of 512-byte disk
     blocks the files inside the Binary II file will occupy after they've
     been removed from the Binary II file. (The format of a Binary II file
     containing multiple files is described below.) If the number is zero,
     the person uploading the file did not bother to calculate the space
     needed. The "disk space needed" must be placed in the file information
     header for the first file inside the Binary II file; it can be set to
     zero in subsequent headers. A downloading program can inspect "disk
     space needed" and abort the transfer immediately if there isn't enough
     disk free space.

     The value of the "operating system type" byte indicates the native
     operating system of the file:

             $00 = ProDOS 8, ProDOS 16, or SOS
             $01 = DOS 3.3
             $02 = Pascal
             $03 = CP/M
             $04 = MS-DOS

     Note that even if a file is not a ProDOS file, the attributes in the
     file information header, including the name, must be inserted in
     ProDOS form. Instructions on how to do this for DOS 3.3 files are
     given later in this document. Similar considerations apply for the
     files of other operating systems.

     The "native file type code" has meaning only if the "operating system
     type" is non-zero. It is set to the actual file type code assigned to
     the file by its native operating system. (Some operating systems, such
     as CP/M and MS-DOS, do not use file type codes, however.) Contrast
     this with the file type code at +4, which is the closest equivalent
     ProDOS file type code. The "native file type code" is needed to
     distinguish files which have the same *ProDOS* file type, but which
     may have different file types in their native operating system. Note
     that if the file type code is only byte long (the usual case), the
     high-order byte of "native file type code" is set to zero.

     The "phantom file flag" byte indicates whether a receiver of the
     Binary II file should save the file which follows (flag is zero) or
     ignore it (flag is non-zero). It is anticipated that some
     communications programs will use phantom files to pass non-essential
     explanatory notes or encoded information which would be understood
     only by a receiver using the same communications program. Such
     programs must not rely on receiving a phantom file, however, since
     this would mean they couldn't handle Binary II files created by other
     communications programs.

     The first two bytes in a phantom file *must* contain an ID code unique
     to the communications program. Developers must obtain ID codes from
     Gary Little to ensure uniqueness (see below for his address). Here is
     a current list of approved ID codes for phantom files used by Apple II
     communications programs:

             $00 $00  =  [generic]
             $00 $01  =  Point-to-Point
             $00 $02  =  Tele-Master Communications System

     Developers of communications programs are responsible for defining and
     publishing the structures of their phantom files.

     The ID bytes appear in the first two bytes of the phantom file.
     Phantom files having a generic ID code of zero must contain lines of
     text terminated by a $00 byte. The text must begin at the third byte
     in the file.

     The "data flags" byte is a bit vector indicating whether the data
     portion of the Binary II file has been compressed, encrypted, or
     packed. If bit 7 (the high-order bit) is set to 1, the file is
     compressed. If bit 6 is 1, the file is encrypted. If bit 0 is 1, the
     file is a sparse file that is packed. A Binary II downloading program
     can examine this byte and warn the user, when necessary, that the file
     must be expanded, decrypted, or unpacked. The person uploading a
     Binary II file may use any convenient method for compressing,
     encrypting, or packing the file but is responsible for providing
     instructions on how to restore the file to its original state.

     This initial release of Binary II has a "Binary II version number" of
     $00.

     Handling Multiple Files
     -----------------------
     An appealing feature of Binary II is that a single Binary II file can
     hold multiple disk files, making it easy to keep a group of related
     files "glued" together when they're sent to a host.

     The structure of a Binary II file containing multiple disk files is
     what you might expect: it is a series of images of individual Binary
     II files. For example, here is the general structure of a Binary II
     file containing three disk files:

      start                                                           end
      -------------------------------------------------------------------
      | Header #1 | #1 Data | Header #2 | #2 Data | Header #3 | #3 Data |
      -------------------------------------------------------------------
        +127 = 2              +127 = 1              +127 = 0

     The data areas following each header end on a 128-byte boundary.

     The "number of files to follow" byte (at offset 127) in the file
     information header for each disk file contains the number of disk
     files that follow it in the Binary II file. It will be zero in the
     header for the last disk file in the group.

     Filenames and Partial Pathnames
     -------------------------------
     Notice that you can put a standard ProDOS filename or a partial
     pathname in the file information header (but never a complete
     pathname). *Beware!* Don't use a partial pathname unless you've
     included, earlier on in the Binary II file, file information headers
     for each of the directories referred to in the partial pathname. Such
     a header must have its "end of file position" bytes set to zero, and
     no data blocks for the subdirectory file must follow it.

     For example, if you want to send a file whose partial pathname is
     HELP/GS/READ.ME, first send a file information header defining the
     HELP/ subdirectory, then one defining the HELP/GS/ subdirectory. If
     you don't, someone downloading the Binary II file won't be able to
     convert it because the necessary subdirectories will not exist.

     Filename Convention
     -------------------
     Whenever a file is sent to a host, the host asks the sender to provide
     a name for it. If it's a Binary II file, the name provided should end
     in .BNY so that its special form will be apparent to anyone viewing a
     list of filenames.

     Identifying Binary II Files
     ---------------------------
     ose the ProDOS
     file. You would repeat this for each file contained inside the Binary
     II file.

        Note: The number of 128-byte data blocks following the
              file information header must be derived from the
              "end-of-file position" attribute (EOF) not the "size
              of file in blocks" attribute. Calculate the number
              by dividing EOF by 128 and adding one to the result
              if EOF is not 0 or an exact multiple of 128.

     Exception: If the file information header defines a subdirectory (the
     file type code is 15), simply CREATE the subdirectory file. Do not
     OPEN it and do not set its size with SET_EOF.

     Ideally, all this conversion work will be done automatically by a
     communications program during an Xmodem (or other binary protocol)
     download. If not, a separate conversion program will have to be run
     after the Binary II file has been received and saved to disk. Gary
     Little has published a public domain program, called BINARY.DWN, that
     will do this for you. (A related program, BINARY.UP, combines multiple
     ProDOS files into one Binary II file which can then be uploaded to a
     host.)

     DOS 3.3 Considerations
     ----------------------
     With a little extra effort, you can also convert DOS 3.3 files to
     Binary II form. This involves translating the DOS 3.3 file attributes
     to the corresponding ProDOS attributes so that you can build a proper
     file information header. Here is how to do this:

        (1) Set the name to one that adheres to the stricter ProDOS naming
            rules.

        (2) Set the ProDOS file type code, auxiliary type code, and access
            code to values which correspond to the DOS 3.3 file type:

               DOS 3.3  |   ProDOS     ProDOS    ProDOS
              file type | file type   aux type   access
             -----------|----------- ---------- --------
              $00 ( T)  | $04 (TXT)    $0000      $E3
              $80 (*T)  | $04 (TXT)    $0000      $21
              $01 ( I)  | $FA (INT)    $0C00      $E3
              $81 (*I)  | $FA (INT)    $0C00      $21
              $02 ( A)  | $FC (BAS)    $0801      $E3
              $82 (*A)  | $FC (BAS)    $0801      $21
              $04 ( B)  | $06 (BIN)     (*)       $E3
              $84 (*B)  | $06 (BIN)     (*)       $21
              $08 ( S)  | $06 (BIN)    $0000      $E3
              $88 (*S)  | $06 (BIN)    $0000      $21
              $10 ( R)  | $FE (REL)    $0000      $E3
              $90 (*R)  | $FE (REL)    $0000      $21
              $20 ( A)  | $06 (BIN)    $0000      $E3
              $A0 (*A)  | $06 (BIN)    $0000      $21
              $40 ( B)  | $06 (BIN)    $0000      $E3
              $C0 (*B)  | $06 (BIN)    $0000      $21

              (*) Set the aux type for a B file to the
                  value stored in the first two bytes
                  of the file (this is the default load
                  address).

         (3) Set the storage type code to $01.

         (4) Set the size of file in blocks, date of creation, date of
             modification, time of creation, and time of modification to
             $0000.

         (5) Set the end-of-file position to the length of the DOS 3.3
             file, in bytes. For a B file (code $04 or $84), this number is
             stored in the third and fourth bytes of the file. For an I
             file (code $01 or $81) or an A file (code $02 or $82), this
             number is stored in the first and second bytes of the file.

         (6) Set the operating system type to $01.

         (7) Set the native file type code to the value of the DOS 3.3 file
             type code.

     Attribute bytes inside a DOS 3.3 file (if any) must *not* be included
     in the data portion of the Binary II file. This includes the first
     four bytes of a B (Binary) file, and the first two bytes of an A
     (Applesoft) or I (Integer BASIC) file.

     Acknowledgements
     ----------------
     Thanks to Glen Bredon for suggesting that partial pathnames be allowed
     in file information headers. Thanks also to Shawn Quick for suggesting
     the "phantom file" byte, to Scott McMahan for suggesting the
     compression and encryption bits in the "data flags" byte, and to
     William Bond for suggesting the "disk space needed" bytes. Finally, a
     big thank you to Neil Shapiro, Chief Sysop of MAUG, for supporting the
     development of the Binary II format and helping it become a true
     standard.

     Feedback and Support
     --------------------
     Send any comments or questions concerning the Binary II file format
     to:

        Gary B. Little
        #210 - 131 Water Street
        Vancouver, British Columbia
        Canada  V6B 4M3
        (604) 681-3371

        CompuServe : 70135,1007
        Delphi     : GBL
        MCI Mail   : 658L6

     Gary developed the Point-to-Point telecommunications program published
     by Pinpoint Publishing. He has also written several books on how to
     program Apple computers: "Inside the Apple IIe," "Inside the Apple
     IIc," "Apple ProDOS: Advanced Features for Programmers," and "Mac
     Assembly Language: A Guide for Programmers." He is currently a
     Contributing Editor for A+ magazine and writes A+'s monthly Rescue
     Squad column. Gary has also published articles in Nibble, Micro, Call
     -A.P.P.L.E, and Softalk.