add E7 detection, refactor, add some comments

seek() returns flux data if needed

README.md

@@ -1,32 +1,152 @@
+# `wozardry`
+
+`wozardry` is a multi-purpose tool for manipulating `.woz` disk images. It can
+validate file structure, edit metadata, import and export metadata in `JSON`
+format, remove unused tracks, and provides a programmatic interface to "read"
+bits and nibbles from a disk image. It supports both WOZ 1.0 and WOZ 2.0 files
+and can convert files from one version to the other.
+
+* [Installation](#installation)
+* [Command line interface](#command-line-interface)
+  * [`verify` command](#verify-command)
+  * [`dump` command](#dump-command)
+  * [`edit` command](#edit-command)
+  * [How to convert WOZ1 to WOZ2 files](#how-to-convert-woz1-to-woz2-files)
+  * [`import` and `export` commands](#import-and-export-commands)
+  * [`remove` command](#remove-command)
+* [Python interface](#python-interface)
+  * [`WozDiskImage` interface](#wozdiskimage-interface)
+  * [How to load and save files on disk](#how-to-load-and-save-files-on-disk)
+  * [`Track` interface](#track-interface)
+
+## Installation
+
+`wozardry` is written in [Python 3](https://www.python.org).
+
+It requires [bitarray](https://pypi.org/project/bitarray/), which can be
+installed thusly:
+
+```
+$ pip3 install -U bitarray
 ```
-$ ./wozardry.py verify -h
-usage: wozardry verify [-h] file
 
-Verify file structure and metadata of a .woz disk image (produces no output
-unless a problem is found)
+(Developers who wish to run the test suite should also install the `pytest`
+module with `pip3 install -U pytest`)
 
-positional arguments:
-  file        .woz disk image
+## Command line interface
 
-optional arguments:
-  -h, --help  show this help message and exit
+wozardry is primarily designed to be used on the command line to directly
+manipulate `.woz` disk images in place. It supports multiple commands, which are
+listed in the `wozardry -h` output.
 
+### `verify` command
 
+This command verifies the file structure and metadata of a `.woz` disk image.
+It produces no output unless a problem is found.
 
-$ ./wozardry.py dump -h
-usage: wozardry dump [-h] file
+Sample usage:
 
-Print all available information and metadata in a .woz disk image
+```
+$ wozardry verify "WOZ 2.0/DOS 3.3 System Master.woz"
+```
 
-positional arguments:
-  file        .woz disk image
+**Tip**: you can [download a collection of .woz test
+images](http://evolutioninteractive.com/applesauce/woz_images.zip).
 
-optional arguments:
-  -h, --help  show this help message and exit
+The `verify` command does not "read" the data on the disk like an emulator
+would. It merely verifies the structure of the `.woz` file itself and applies a
+few sanity checks on the embedded metadata (if any). The disk may or may not
+boot in an emulator. It may not pass its own copy protection checks. It may not
+have the data you expected, or any data at all. The `verify` command can not
+answer those questions.
 
+### `dump` command
 
+Prints all available information and metadata in a `.woz` disk image.
 
-$ ./wozardry edit -h
+Sample usage:
+
+```
+$ wozardry dump "WOZ 2.0/Wings of Fury - Disk 1, Side A.woz"
+TMAP:  Track 0.00              TRKS 0
+TMAP:  Track 0.25              TRKS 0
+TMAP:  Track 0.75              TRKS 1
+TMAP:  Track 1.00              TRKS 1
+.
+. [many lines elided]
+.
+TMAP:  Track 33.75             TRKS 34
+TMAP:  Track 34.00             TRKS 34
+TMAP:  Track 34.25             TRKS 34
+META:  language:               English
+META:  publisher:              Broderbund
+META:  developer:
+META:  side:                   Disk 1, Side A
+META:  copyright:              1987
+META:  requires_ram:           128K
+META:  subtitle:
+META:  image_date:             2018-01-15T01:30:53.025Z
+META:  title:                  Wings of Fury
+META:  version:
+META:  contributor:            DiskBlitz
+META:  notes:
+META:  side_name:
+META:  requires_machine:       2e
+META:                          2c
+META:                          2e+
+META:                          2gs
+INFO:  File format version:    2
+INFO:  Disk type:              5.25-inch (140K)
+INFO:  Write protected:        yes
+INFO:  Tracks synchronized:    yes
+INFO:  Weakbits cleaned:       yes
+INFO:  Creator:                Applesauce v1.1
+INFO:  Boot sector format:     1 (16-sector)
+INFO:  Optimal bit timing:     32 (standard)
+INFO:  Compatible hardware:    2e
+INFO:                          2c
+INFO:                          2e+
+INFO:                          2gs
+INFO:  Required RAM:           128K
+INFO:  Largest track:          13 blocks
+```
+
+The `TMAP` section (stands for "track map") shows which tracks are included in
+the disk image. As you can see from the above sample, the same bitstream data
+can be assigned to multiple tracks, usually adjacent quarter tracks. Each
+bitstream is stored only once in the `.woz` file.
+
+The `META` section shows any embedded metadata, such as copyright and
+version. This section is optional; not all `.woz` files will have the same
+metadata fields, and some may have none at all.
+
+The `INFO` section shows information that emulators or other programs might need
+to know, such as the boot sector format (13- or 16-sector, or both) and whether
+the disk is write protected. All `INFO` fields are required and are included in
+every `.woz` file.
+
+The output of the `dump` command is designed to by grep-able, if you're into
+that kind of thing.
+
+```
+$ wozardry dump "WOZ 2.0/Wings of Fury - Disk 1, Side A.woz" | grep "^INFO"
+```
+
+will show just the `INFO` section.
+
+**Tip**: [the .woz specification](https://applesaucefdc.com/woz/reference2/)
+lists the standard metadata fields and the acceptable values of all info fields.
+
+### `edit` command
+
+This command lets you modify any information or metadata field in a `.woz`
+file. This is where the fun(\*) starts.
+
+(\*) not guaranteed, actual fun may vary
+
+The inline help is a good overview.
+
+```
 usage: wozardry edit [-h] [-i INFO] [-m META] file
 
 Edit information and metadata in a .woz disk image
@@ -38,8 +158,12 @@ optional arguments:
   -h, --help            show this help message and exit
   -i INFO, --info INFO  change information field. INFO format is "key:value".
                         Acceptable keys are disk_type, write_protected,
-                        synchronized, cleaned, creator, version. Other keys
-                        are ignored.
+                        synchronized, cleaned, creator, version. Additional
+                        keys for WOZ2 files are disk_sides, required_ram,
+                        boot_sector_format, compatible_hardware,
+                        optimal_bit_timing. Other keys are ignored. For
+                        boolean fields, use "1" or "true" or "yes" for true,
+                        "0" or "false" or "no" for false.
   -m META, --meta META  change metadata field. META format is "key:value".
                         Standard keys are title, subtitle, publisher,
                         developer, copyright, version, language, requires_ram,
@@ -52,26 +176,377 @@ Tips:
  - Use "key:" with no value to delete a metadata field.
  - Keys are case-sensitive.
  - Some values have format restrictions; read the .woz specification.
-
-$ ./wozardry export -h
-usage: wozardry export [-h] file
-
-Export (as JSON) all information and metadata from a .woz disk image
-
-positional arguments:
-  file        .woz disk image
-
-optional arguments:
-  -h, --help  show this help message and exit
-
-$ ./wozardry import -h
-usage: wozardry import [-h] file
-
-Import JSON file to update information and metadata in a .woz disk image
-
-positional arguments:
-  file        .woz disk image
-
-optional arguments:
-  -h, --help  show this help message and exit
+```
+
+Let's look at some examples.
+
+Working with this same "Wings of Fury" disk image, let's give the game author
+his due by adding the `developer` metadata field:
+
+```
+$ wozardry edit -m "developer:Steve Waldo" "WOZ 2.0/Wings of Fury - Disk 1, Side A.woz"
+```
+
+Metadata fields are arbitrary; there is a standard set listed in [the .woz
+specification](https://applesaucefdc.com/woz/reference2/), but you can add your
+own.
+
+```
+$ wozardry edit -m "genre:action" -m "perspective:side view" "WOZ 2.0/Wings of Fury - Disk 1, Side A.woz"
+```
+
+You can use a similar syntax to remove metadata fields that don't apply to this
+disk.
+
+```
+$ wozardry edit -m "version:" -m "notes:" -m "side_name:" -m "subtitle:" "WOZ 2.0/Wings of Fury - Disk 1, Side A.woz"
+```
+
+Now let's look at that metadata section again:
+
+```
+$ wozardry dump "WOZ 2.0/Wings of Fury - Disk 1, Side A.woz" | grep "^META"
+META:  language:               English
+META:  publisher:              Broderbund
+META:  developer:              Steve Waldo
+META:  side:                   Disk 1, Side A
+META:  copyright:              1987
+META:  requires_ram:           128K
+META:  image_date:             2018-01-15T01:30:53.025Z
+META:  title:                  Wings of Fury
+META:  contributor:            DiskBlitz
+META:  requires_machine:       2e
+META:                          2c
+META:                          2e+
+META:                          2gs
+META:  genre:                  action
+META:  perspective:            side view
+```
+
+You can modify `INFO` fields using a similar syntax (`-i` instead of `-m`), but
+be aware that `INFO` fields are highly constrained, and incorrect values can
+have noticeable effects in emulators. `wozardry` will reject any values that are
+nonsensical or out of range, but even in-range values can render the disk image
+unbootable. For example, the "optimal bit timing" field specifies the rate at
+which bits appear in the floppy drive data latch; if the rate is not what the
+disk's low-level RWTS code is expecting, the disk may be unable to read itself.
+
+Nonetheless, here are some examples of changing `INFO` fields. To tell emulators
+that a disk is not write-protected, set the `write_protected` field to `no`,
+`false`, or `0`.
+
+```
+$ wozardry edit -i "write_protected:no" "WOZ 2.0/Wings of Fury - Disk 1, Side A.woz"
+```
+
+To tell emulators that the disk only runs on certain Apple II models, set the
+`compatible_hardware` field with a pipe-separated list. (Values may appear in
+any order. See `kRequiresMachine` in the `wozardry` source code for all the
+acceptable values.)
+
+```
+$ wozardry edit -i "compatible_hardware:2e|2e+|2c|2gs" "WOZ 2.0/Wings of Fury - Disk 1, Side A.woz"
+```
+
+### How to convert WOZ1 to WOZ2 files
+
+As of this writing, the `.woz` specification has undergone one major revision,
+which changed the internal structure of a `.woz` file and added several new
+`INFO` fields. Both file formats use the `.woz` file extension; they are
+distinguished by magic bytes (`WOZ1` vs. `WOZ2`) within the file.
+
+Let's say you have an older `WOZ1` file, like this one from the `WOZ 1.0`
+directory of the official test images collection:
+
+```
+$ wozardry dump "WOZ 1.0/Wings of Fury - Disk 1, Side A.woz" | grep "^INFO"
+INFO:  File format version:    1
+INFO:  Disk type:              5.25-inch (140K)
+INFO:  Write protected:        yes
+INFO:  Tracks synchronized:    yes
+INFO:  Weakbits cleaned:       yes
+INFO:  Creator:                Applesauce v0.29
+```
+
+The "file format version" confirms this is a `WOZ1` file. To convert it to a
+`WOZ2` file, set the `version` field to `2`.
+
+```
+$ wozardry edit -i "version:2" "WOZ 1.0/Wings of Fury - Disk 1, Side A.woz"
+
+$ wozardry dump "WOZ 1.0/Wings of Fury - Disk 1, Side A.woz" | grep "^INFO"
+INFO:  File format version:    2
+INFO:  Disk type:              5.25-inch (140K)
+INFO:  Write protected:        yes
+INFO:  Tracks synchronized:    yes
+INFO:  Weakbits cleaned:       yes
+INFO:  Creator:                Applesauce v0.29
+INFO:  Boot sector format:     0 (unknown)
+INFO:  Optimal bit timing:     32 (standard)
+INFO:  Compatible hardware:    unknown
+INFO:  Required RAM:           unknown
+INFO:  Largest track:          13 blocks
+```
+
+All the new (v2-specific) `INFO` fields are initialized with default
+values. Existing fields like the write-protected flag are retained. ("Largest
+track" is a calculated field and can not be set directly.)
+
+### `import` and `export` commands
+
+These commands allow you to access the information and metadata in a `.woz` file
+in `JSON` format.
+
+```
+$ wozardry export "WOZ 2.0/Wings of Fury - Disk 1, Side A.woz"
+{
+  "woz": {
+    "info": {
+      "version": 2,
+      "disk_type": 1,
+      "write_protected": true,
+      "synchronized": true,
+      "cleaned": true,
+      "creator": "Applesauce v1.1",
+      "disk_sides": 1,
+      "boot_sector_format": 1,
+      "optimal_bit_timing": 32,
+      "compatible_hardware": [
+        "2e",
+        "2c",
+        "2e+",
+        "2gs"
+      ],
+      "required_ram": 128,
+      "largest_track": 13
+    },
+    "meta": {
+      "language": "English",
+      "publisher": "Broderbund",
+      "developer": [
+        ""
+      ],
+      "side": "Disk 1, Side A",
+      "copyright": "1987",
+      "requires_ram": "128K",
+      "subtitle": [
+        ""
+      ],
+      "image_date": "2018-01-15T01:30:53.025Z",
+      "title": "Wings of Fury",
+      "version": [
+        ""
+      ],
+      "contributor": "DiskBlitz",
+      "notes": [
+        ""
+      ],
+      "side_name": [
+        ""
+      ],
+      "requires_machine": [
+        "2e",
+        "2c",
+        "2e+",
+        "2gs"
+      ]
+    }
+  }
+}
+```
+
+You can pipe the output of the `export` command to the `import` command to copy
+metadata from one `.woz` file to another.
+
+```
+$ wozardry export game-side-a.woz | wozardry import game-side-b.woz
+```
+
+Technically, this merges metadata. All metadata fields in `game-side-a.woz` will
+be copied to `game-side-b.woz`, overwriting any existing values for those
+fields. But if `game-side-b.woz` already had additional metadata fields that
+were not present in `game-side-a.woz`, those will be retained.
+
+**Tip**: [a2rchery](https://github.com/a2-4am/a2rchery) is a tool to manipulate
+`.a2r` flux images. These `.a2r` files can also have embedded metadata, just
+like `.woz` files. And guess what! `a2rchery` also has `import` and `export`
+commands, just like `wozardry`. You see where this is going.
+
+```
+$ wozardry export game.woz | a2rchery import game.a2r
+```
+
+### `remove` command
+
+This command allow you to remove one or more tracks from a `.woz` disk image.
+
+Tracks are specified in quarter tracks, in base 10 (not base 16). Multiple
+tracks can be removed at once.
+
+```
+$ wozardry remove -t0.25 -t0.5 -t0.75 -t1 -t1.25 -t35 "Gamma Goblins.woz"
+```
+
+**Note**: tracks are stored as indices in the `TMAP` chunk, and multiple tracks
+can refer to the same bitstream (stored in the `TRKS` chunk). If you remove all
+tracks that refer to a bitstream, the bitstream will be removed from the `TRKS`
+chunk and all the indices in the `TMAP` chunk will be adjusted accordingly.
+
+## Python interface
+
+### `WozDiskImage` interface
+
+This represents a single WOZ disk image. You can create it from scratch, load it
+from a file on disk, or parse it from a bytestream in memory.
+
+```
+>>> import wozardry
+>>> woz_image = wozardry.WozDiskImage()
+>>> woz_image.woz_version
+2
+```
+
+This newly created `woz_image` already has an `info` dictionary with all the
+required fields in the `INFO` chunk.
+
+```
+>>> from pprint import pprint
+>>> pprint(woz_image.info)
+OrderedDict([('version', 2),
+             ('disk_type', 1),
+             ('write_protected', False),
+             ('synchronized', False),
+             ('cleaned', False),
+             ('creator', 'wozardry 2.0-beta'),
+             ('disk_sides', 1),
+             ('boot_sector_format', 0),
+             ('optimal_bit_timing', 32),
+             ('compatible_hardware', []),
+             ('required_ram', 0)])
+>>> woz_image.info["compatible_hardware"] = ["2", "2+"]
+>>> woz_image.info["compatible_hardware"]
+['2', '2+']
+```
+
+It also has an empty `meta` dictionary for metadata.
+
+```
+>>> pprint(woz_image.meta)
+OrderedDict()
+>>> woz_image.meta["copyright"] = "1981"
+>>> woz_image.meta["developer"] = "Chuckles"
+>>> pprint(woz_image.meta)
+OrderedDict([('copyright', '1981'),
+             ('developer', 'Chuckles')])
+```
+
+### How to load and save files on disk
+
+To load a `.woz` disk image from a file (or any file-like object), open the file
+and pass it to the `WozDiskImage` constructor. Be sure to open files in binary
+mode.
+
+```
+>>> with open("Wings of Fury.woz", "rb") as fp:
+...     woz_image = wozardry.WozDiskImage(fp)
+```
+
+To save a file, serialize the `WozDiskImage` object with the `bytes()` method
+and write that to disk. Be sure to open files in binary mode.
+
+```
+>>> with open("Wings of Fury.woz", "wb") as fp:
+...     fp.write(bytes(woz_image))
+```
+
+### `Track` interface
+
+A `.woz` disk image usually contains multiple tracks of data, otherwise what's
+the point, right? Each track is accessed by the `Track` interface.
+
+The `WozDiskImage.seek()` returns a `Track` object that contains that track's
+data (or `None` if that track is not in the disk image).
+
+**Tip**: the `seek()` method takes a logical track number, which could be a
+quarter track or half track. To get the data on track 1.5, call `seek(1.5)`.
+
+In this example, we load a `.woz` image from disk and seek to track 0:
+
+```
+>>> with open("Wings of Fury.woz", "rb") as fp:
+...     woz_image = wozardry.WozDiskImage(fp)
+>>> tr = woz_image.seek(0)
+>>> tr
+<wozardry.Track object at 0x108ccf3c8>
+```
+
+Now we can access the bitstream of the track. The raw bitstream is in `tr.bits`,
+but you probably want to use one of these convenience methods instead.
+
+To search the track for a specific nibble sequence, use the `find()` method. It
+returns `True` if the nibble sequence was found, or `False` otherwise.
+
+```
+>>> tr.find(bytes.fromhex("D5 AA 96"))
+True
+```
+
+The `Track` object maintains state of where it is within the bitstream
+(`tr.bit_index`), including wrapping around to the beginning if it reaches the
+end (`tr.revolutions`). After finding that `D5 AA 96` nibble sequence with the
+`find()` method, we can read the next nibbles in the bitstream with the
+`nibble()` generator.
+
+```
+>>> hex(next(tr.nibble()))
+'0xff'
+>>> hex(next(tr.nibble()))
+'0xfe'
+>>> hex(next(tr.nibble()))
+'0xaa'
+>>> hex(next(tr.nibble()))
+'0xaa'
+>>> hex(next(tr.nibble()))
+'0xab'
+>>> hex(next(tr.nibble()))
+'0xaa'
+```
+
+**Tip**: the `nibble()` generator returns nibbles like a real disk controller.
+`0` bits between nibbles are ignored, so the high bit of the returned nibble is
+always `1`. The `find()` method uses the `nibble()` generator internally, so it
+also ignores `0` bits between nibbles.
+
+If you want to read individual bits from the current position in the bitstream,
+use the `bit()` generator.
+
+```
+>>> next(tr.bit())
+1
+>>> next(tr.bit())
+1
+>>> next(tr.bit())
+1
+>>> next(tr.bit())
+1
+>>> next(tr.bit())
+1
+>>> next(tr.bit())
+1
+>>> next(tr.bit())
+1
+>>> next(tr.bit())
+0
+```
+
+Unlike a real disk controller, you can move backwards in the bitstream, allowing
+you to speculatively look at bits then rewind as if you hadn't seen them yet.
+
+Let's rewind as if we hadn't just read those 8 individual bits, then read them
+as a nibble:
+
+```
+>>> tr.rewind(8)
+>>> hex(next(tr.nibble()))
+'0xfe'
 ```

moofimage.py

@@ -0,0 +1,388 @@
+#!/usr/bin/env python3
+
+#(c) 2022 by 4am
+#license:MIT
+
+import wozardry # https://github.com/a2-4am/wozardry
+import bitarray # https://pypi.org/project/bitarray/
+import sys
+
+def myhex(b):
+    return hex(b)[2:].rjust(2, "0").upper()
+
+class MoofTrack(wozardry.Track):
+    def __init__(self, raw_bytes, raw_count):
+        wozardry.Track.__init__(self, raw_bytes, raw_count)
+        self.bit_index = 0
+        self.revolutions = 0
+        self.bits = bitarray.bitarray(endian="big")
+        self.bits.frombytes(self.raw_bytes)
+        while len(self.bits) > raw_count:
+            self.bits.pop()
+
+    def rewind(self, bit_count=1):
+        self.bit_index -= bit_count
+        while self.bit_index < 0:
+            self.bit_index += self.raw_count
+            self.revolutions -= 1
+
+    def forward(self, bit_count=1):
+        self.bit_index += bit_count
+        while self.bit_index >= self.raw_count:
+            self.bit_index -= self.raw_count
+            self.revolutions += 1
+
+    def bit(self):
+        b = self.bits[self.bit_index]
+        self.forward()
+        yield b
+
+    def nibble(self):
+        while not next(self.bit()): pass
+        n = 0b10000000
+        for bit_index in range(6, -1, -1):
+            b = next(self.bit())
+            n |= b << bit_index
+        yield n
+
+    def find(self, sequence):
+        starting_revolutions = self.revolutions
+        seen = [0] * len(sequence)
+        while (self.revolutions < starting_revolutions + 2):
+            del seen[0]
+            seen.append(next(self.nibble()))
+            if tuple(seen) == tuple(sequence): return True
+        return False
+
+    def find_this_not_that(self, good, bad):
+        starting_revolutions = self.revolutions
+        good = tuple(good)
+        bad = tuple(bad)
+        seen_good = [0] * len(good)
+        seen_bad = [0] * len(bad)
+        while (self.revolutions < starting_revolutions + 2):
+            del seen_good[0]
+            del seen_bad[0]
+            n = next(self.nibble())
+            seen_good.append(n)
+            seen_bad.append(n)
+            if tuple(seen_bad) == bad: return False
+            if tuple(seen_good) == good: return True
+        return False
+
+class MoofAddressField:
+    def __init__(self, valid, volume, track_id, sector_id):
+        self.valid = valid
+        self.volume = volume
+        self.track_id = track_id
+        self.sector_id = sector_id
+
+class MoofDataField:
+    def __init__(self, valid, sector_id, tags, data):
+        self.valid = valid
+        self.sector_id = sector_id
+        self.tags = tags
+        self.data = data
+
+class MoofBlock:
+    def __init__(self, address_field, data_field):
+        self.address_field = address_field
+        self.data_field = data_field
+
+class MoofRWTS:
+    kDefaultAddressPrologue16 = (0xD5, 0xAA, 0x96)
+    kDefaultAddressEpilogue16 = (0xDE, 0xAA)
+    kDefaultDataPrologue16 =    (0xD5, 0xAA, 0xAD)
+    kDefaultDataEpilogue16 =    (0xDE, 0xAA)
+    kDefaultNibbleTranslationTable16 = {
+        0x96: 0x00, 0x97: 0x01, 0x9A: 0x02, 0x9B: 0x03, 0x9D: 0x04, 0x9E: 0x05, 0x9F: 0x06, 0xA6: 0x07,
+        0xA7: 0x08, 0xAB: 0x09, 0xAC: 0x0A, 0xAD: 0x0B, 0xAE: 0x0C, 0xAF: 0x0D, 0xB2: 0x0E, 0xB3: 0x0F,
+        0xB4: 0x10, 0xB5: 0x11, 0xB6: 0x12, 0xB7: 0x13, 0xB9: 0x14, 0xBA: 0x15, 0xBB: 0x16, 0xBC: 0x17,
+        0xBD: 0x18, 0xBE: 0x19, 0xBF: 0x1A, 0xCB: 0x1B, 0xCD: 0x1C, 0xCE: 0x1D, 0xCF: 0x1E, 0xD3: 0x1F,
+        0xD6: 0x20, 0xD7: 0x21, 0xD9: 0x22, 0xDA: 0x23, 0xDB: 0x24, 0xDC: 0x25, 0xDD: 0x26, 0xDE: 0x27,
+        0xDF: 0x28, 0xE5: 0x29, 0xE6: 0x2A, 0xE7: 0x2B, 0xE9: 0x2C, 0xEA: 0x2D, 0xEB: 0x2E, 0xEC: 0x2F,
+        0xED: 0x30, 0xEE: 0x31, 0xEF: 0x32, 0xF2: 0x33, 0xF3: 0x34, 0xF4: 0x35, 0xF5: 0x36, 0xF6: 0x37,
+        0xF7: 0x38, 0xF9: 0x39, 0xFA: 0x3A, 0xFB: 0x3B, 0xFC: 0x3C, 0xFD: 0x3D, 0xFE: 0x3E, 0xFF: 0x3F,
+    }
+
+    def __init__(self,
+                 address_prologue = kDefaultAddressPrologue16,
+                 address_epilogue = kDefaultAddressEpilogue16,
+                 data_prologue = kDefaultDataPrologue16,
+                 data_epilogue = kDefaultDataEpilogue16,
+                 nibble_translate_table = kDefaultNibbleTranslationTable16):
+        self.address_prologue = address_prologue
+        self.address_epilogue = address_epilogue
+        self.data_prologue = data_prologue
+        self.data_epilogue = data_epilogue
+        self.nibble_translate_table = nibble_translate_table
+        self.sectors_per_track = dict(zip(range(0xA0), (i for i in range(0x0C,0x07,-1) for j in range(0x20))))
+
+    def _(self, track):
+        return self.nibble_translate_table[next(track.nibble())]
+
+    def find_address_prologue(self, track):
+        return track.find(self.address_prologue)
+
+    def address_field_at_point(self, track):
+        h0 = self._(track)
+        sector_id = self._(track)
+        h2 = self._(track)
+        volume = self._(track)
+        checksum = self._(track)
+        valid = h0 ^ sector_id ^ h2 ^ volume == checksum
+        track_id = (h0 << 1) | ((h2 & 0b00000001) << 7) | ((h2 & 0b00100000) >> 5)
+        return MoofAddressField(valid, volume, track_id, sector_id)
+
+    def verify_nibbles_at_point(self, track, nibbles):
+        found = []
+        for i in nibbles:
+            found.append(next(track.nibble()))
+        return tuple(found) == tuple(nibbles)
+
+    def verify_address_epilogue_at_point(self, track):
+        return self.verify_nibbles_at_point(track, self.address_epilogue)
+
+    def find_data_prologue(self, track):
+        return track.find_this_not_that(self.data_prologue, self.address_prologue)
+
+    def data_field_at_point(self, track):
+        # three checksums
+        c1 = c2 = c3 = 0
+
+        # generator to decode grouped bytes while juggling three checksums
+        def gcr_generator(byte_groups):
+            nonlocal c1, c2, c3
+            for d0, d1, d2 in byte_groups:
+                c1 = (c1 & 0b11111111) << 1
+                if c1 > 0xFF:
+                    c1 -= 0xFF
+                    c3 += 1
+                b = d0 ^ c1
+                c3 += b
+                yield b
+
+                if c3 > 0xFF:
+                    c3 &= 0b11111111
+                    c2 += 1
+                b = d1 ^ c3
+                c2 += b
+                yield b
+
+                if c2 > 0xFF:
+                    c2 &= 0b11111111
+                    c1 += 1
+                b = d2 ^ c2
+                c1 += b
+                yield b
+
+        # first nibble is sector number
+        sector_id = self._(track)
+
+        # read 700 nibbles, decode each against nibble translate table, store in 175 groups of 4
+        nibble_groups = [(self._(track), self._(track), self._(track), self._(track))
+                         for i in range(175)]
+
+        # convert each group of 4 nibbles into a group of 3 bytes to pass into the decoder
+        gcr_byte = gcr_generator((((n[1] & 0b00111111) | ((n[0] << 2) & 0b11000000),
+                                   (n[2] & 0b00111111) | ((n[0] << 4) & 0b11000000),
+                                   (n[3] & 0b00111111) | ((n[0] << 6) & 0b11000000))
+                                  for n in nibble_groups))
+
+        # decode 524 bytes (12 tag bytes + 512 data bytes)
+        tags = [next(gcr_byte) for i in range(12)]
+        data = [next(gcr_byte) for i in range(512)]
+
+        # validate checksums against last data field nibble and three epilogue nibbles
+        valid = nibble_groups[-1][-1] == ((c1 & 0b11000000) >> 6) | ((c2 & 0b11000000) >> 4) | ((c3 & 0b11000000) >> 2)
+        valid &= self._(track) == (c3 & 0b00111111)
+        valid &= self._(track) == (c2 & 0b00111111)
+        valid &= self._(track) == (c1 & 0b00111111)
+
+        return MoofDataField(valid, sector_id, tags, data)
+
+    def verify_data_epilogue_at_point(self, track):
+        return self.verify_nibbles_at_point(track, self.data_epilogue)
+
+class DefaultLogger:
+    def warn(self, message, T=None, S=None, X=None, Y=None):
+        if T: T = myhex(T)
+        if S: S = myhex(S)
+        message = message.format(**locals())
+        sys.stderr.write(message)
+        sys.stderr.write('\n')
+    info=warn
+    error=warn
+
+class MoofDiskImage(wozardry.WozDiskImage):
+    kE7Bytestream = (0x2B, 0x00, 0x2B, 0xFD, 0x83, 0x6F, 0x20, 0xE2,
+                     0x8D, 0x99, 0x49, 0x44, 0x47, 0x82, 0xD9, 0x26,
+                     0xFB, 0xC6, 0x3, 0xF8)
+    kPACEPrologue = (0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
+                     0xFF, 0xFF, 0xFF, 0xFF, 0xAB, 0xCD, 0xEF, 0xEF)
+
+    def __init__(self, iostream=None, rwtsclass=MoofRWTS, loggerclass=DefaultLogger):
+        wozardry.WozDiskImage.__init__(self, iostream)
+        self.logger = loggerclass()
+        self.rwts = rwtsclass()
+        for i,t in zip(range(len(self.tracks)), self.tracks):
+            self.tracks[i] = MoofTrack(t.raw_bytes, t.raw_count)
+        self.parse()
+
+    def get_pace_key_at_point(self, track, bit_index):
+        # save bitstream position
+        track.bit_index, bit_index = bit_index, track.bit_index
+        key = []
+        if self.rwts.verify_nibbles_at_point(track, self.kPACEPrologue):
+            for i in range(4):
+                next(track.nibble())
+            for i in range(4):
+                x = (next(track.nibble()) << 8) + next(track.nibble())
+                x = x & 0x5555
+                x = (x | (x >> 1)) & 0x3333
+                x = (x | (x >> 2)) & 0x0f0f
+                x = (x | (x >> 4)) & 0x00ff
+                x = (x | (x >> 8)) & 0xffff
+                key.append(x)
+            key.reverse()
+        # restore bitstream position
+        track.bit_index, bit_index = bit_index, track.bit_index
+        return "".join(map(myhex, key))
+
+    def parse(self):
+        self.blocks = []
+        # only 400K and 800K disks supported at the moment
+        if not self.info["disk_type"] in (1,2): return
+        for track_index in self.tmap:
+            if track_index == 0xFF: continue
+            track = self.tracks[track_index]
+            seen_sectors = []
+            track_id = -1
+            while self.rwts.find_address_prologue(track):
+                address_field = self.rwts.address_field_at_point(track)
+
+                # log if address field checksum doesn't match
+                if not address_field.valid:
+                    self.logger.warn(
+                        'T{T},S{S} Address field checksum invalid',
+                        T=address_field.track_id,
+                        S=address_field.sector_id
+                    )
+                    continue
+
+                # log if track ID is ridiculous
+                if not (0x00 <= address_field.track_id <= 0x9F):
+                    self.logger.warn(
+                        'Address field track ID {T} invalid',
+                        T=address_field.track_id
+                    )
+                    continue
+
+                # log if sector ID is ridiculous
+                if not (0x00 <= address_field.sector_id < self.rwts.sectors_per_track[address_field.track_id]):
+                    self.logger.warn(
+                        'Address field sector ID {S} invalid',
+                        S=address_field.sector_id
+                    )
+                    continue
+
+                # log if address field epilogue isn't next
+                if not self.rwts.verify_address_epilogue_at_point(track):
+                    self.logger.warn(
+                        'T{T},S{S} Address field epilogue invalid',
+                        T=address_field.track_id,
+                        S=address_field.sector_id
+                    )
+                    continue
+
+                # if we see duplicate sector IDs, assume we're done
+                if address_field.sector_id in seen_sectors: break
+                seen_sectors.append(address_field.sector_id)
+
+                old_bit_index = track.bit_index
+                if not self.rwts.find_data_prologue(track):
+                    # if we didn't find any data field prologue at all
+                    # before the next address field prologue, then check
+                    # if this is a specially formatted protection sector
+                    # from which we can extract some useful information
+                    decryption_key = self.get_pace_key_at_point(track, old_bit_index)
+                    if decryption_key:
+                        self.logger.info(
+                            'T{T},S{S} Found PACE protection, key={X}',
+                            T=address_field.track_id,
+                            S=address_field.sector_id,
+                            X=decryption_key
+                        )
+                    continue
+
+                try:
+                    data_field = self.rwts.data_field_at_point(track)
+                except KeyError:
+                    # log if GCR decoding failed
+                    self.logger.warn(
+                        'T{T},S{S} Data field contains invalid nibble',
+                        T=address_field.track_id,
+                        S=address_field.sector_id
+                    )
+                    continue
+
+                # log if checksums didn't match after GCR decoding
+                if not data_field.valid:
+                    self.logger.warn(
+                        'T{T},S{S} Data field checksums invalid',
+                        T=address_field.track_id,
+                        S=address_field.sector_id
+                    )
+                    continue
+
+                # address and data fields are supposed to contain
+                # matching sector IDs, so log if they don't match
+                if address_field.sector_id != data_field.sector_id:
+                    self.logger.warn(
+                        'T{T},S{S} Address and data field sector IDs do not match',
+                        T=address_field.track_id,
+                        S=address_field.sector_id
+                    )
+                    continue
+
+                # log if sector data contains E7 bitstream
+                if (sum(data_field.data[:0x18E]) == 0) and (tuple(data_field.data[0x18F:0x1A3]) == self.kE7Bytestream):
+                    #print(data_field.data[0x18F:0x1A3])
+                    self.logger.warn(
+                        'T{T},S{S} Found E7 bitstream',
+                        T=address_field.track_id,
+                        S=address_field.sector_id
+                    )
+
+                # log if data field epilogue isn't next
+                if not self.rwts.verify_data_epilogue_at_point(track):
+                    self.logger.warn(
+                        'T{T},S{S} Data field epilogue invalid',
+                        T=address_field.track_id,
+                        S=address_field.sector_id
+                    )
+                    continue
+
+                track_id = address_field.track_id
+                self.blocks.append(MoofBlock(address_field, data_field))
+
+            # move on if we didn't find any valid sectors
+            if track_id == -1: continue
+
+            # log if we didn't find enough sectors
+            sector_count = len(seen_sectors)
+            expected_count = self.rwts.sectors_per_track[track_id]
+            if sector_count < expected_count:
+                self.logger.warn(
+                    'T{T} Found {X} sectors (expected {Y})',
+                    T=track_id,
+                    X=sector_count,
+                    Y=expected_count
+                )
+
+def driver(filename):
+    with open(filename, 'rb') as f:
+        mdisk = MoofDiskImage(f)
+
+if __name__ == '__main__':
+    driver(sys.argv[1])

test_wozardry.py

@@ -0,0 +1,536 @@
+#!/usr/bin/env python3
+
+# Sources of all truth:
+#
+# - WOZ1 specification <https://applesaucefdc.com/woz/reference1/>
+# - WOZ2 specification <https://applesaucefdc.com/woz/reference2/>
+#
+# There is no spec but the spec itself.
+
+import wozardry
+import pytest # https://pypi.org/project/pytest/
+import argparse
+import tempfile
+import shutil
+import io
+
+# two valid .woz files in the repository
+kValid1 = "test/valid1.woz"
+kValid2 = "test/valid2.woz"
+
+# valid WOZ1 header as string of hex
+kHeader1 = "57 4F 5A 31 FF 0A 0D 0A 00 00 00 00 "
+
+# valid WOZ2 header as string of hex
+kHeader2 = "57 4F 5A 32 FF 0A 0D 0A 00 00 00 00 "
+
+def bfh(s):
+    """utility function to convert string of hex into a BytesIO stream"""
+    return io.BytesIO(bytes.fromhex(s))
+
+#----- test file parser -----
+
+def test_parse_header():
+    # incomplete header
+    with pytest.raises(wozardry.WozEOFError):
+        wozardry.WozDiskImage(bfh("57 4F 5A 32"))
+
+    with pytest.raises(wozardry.WozEOFError):
+        wozardry.WozDiskImage(bfh("57 4F 5A 32 FF 0A 0D"))
+
+    with pytest.raises(wozardry.WozEOFError):
+        wozardry.WozDiskImage(bfh("57 4F 5A 32 FF 0A 0D 0A 00 00 00"))
+
+    # invalid signature at offset 0
+    with pytest.raises(wozardry.WozHeaderError_NoWOZMarker):
+        wozardry.WozDiskImage(bfh("57 4F 5A 30 00 00 00 00"))
+
+    # invalid signature at offset 0
+    with pytest.raises(wozardry.WozHeaderError_NoWOZMarker):
+        wozardry.WozDiskImage(bfh("57 4F 5A 33 00 00 00 00"))
+
+    # missing FF byte at offset 4
+    with pytest.raises(wozardry.WozHeaderError_NoFF):
+        wozardry.WozDiskImage(bfh("57 4F 5A 32 00 0A 0D 0A"))
+
+    # missing 0A byte at offset 5
+    with pytest.raises(wozardry.WozHeaderError_NoLF):
+        wozardry.WozDiskImage(bfh("57 4F 5A 32 FF 0D 0D 0D"))
+
+    # missing 0D byte at offset 6
+    with pytest.raises(wozardry.WozHeaderError_NoLF):
+        wozardry.WozDiskImage(bfh("57 4F 5A 32 FF 0A 0A 0A"))
+
+    # missing 0A byte at offset 7
+    with pytest.raises(wozardry.WozHeaderError_NoLF):
+        wozardry.WozDiskImage(bfh("57 4F 5A 32 FF 0A 0D 0D"))
+
+def test_parse_info():
+    # TMAP chunk before INFO chunk
+    with pytest.raises(wozardry.WozINFOFormatError_MissingINFOChunk):
+        wozardry.WozDiskImage(bfh(kHeader2 + "54 4D 41 50 A0 00 00 00 " + "FF "*160))
+
+    # wrong INFO chunk size (too small)
+    with pytest.raises(wozardry.WozINFOFormatError):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3B 00 00 00 " + "00 "*59))
+
+    # wrong INFO chunk size (too big)
+    with pytest.raises(wozardry.WozINFOFormatError):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3D 00 00 00 " + "00 "*61))
+
+    # invalid version (0) in a WOZ1 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadVersion):
+        wozardry.WozDiskImage(bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 00" + "00 "*59))
+
+    # invalid version (0) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadVersion):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 00" + "00 "*59))
+
+    # invalid version (1) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadVersion):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 01" + "00 "*59))
+
+    # invalid version (2) in a WOZ1 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadVersion):
+        wozardry.WozDiskImage(bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 02" + "00 "*59))
+
+    # invalid disk type (0) in a WOZ1 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadDiskType):
+        wozardry.WozDiskImage(bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 01 00 " + "00 "*58))
+
+    # invalid disk type (0) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadDiskType):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 00 " + "00 "*58))
+
+    # invalid disk type (3) in a WOZ1 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadDiskType):
+        wozardry.WozDiskImage(bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 01 03 " + "00 "*58))
+
+    # invalid disk type (3) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadDiskType):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 03 " + "00 "*58))
+
+    # invalid write protected flag (2) in a WOZ1 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadWriteProtected):
+        wozardry.WozDiskImage(bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 01 01 02 " + "00 "*57))
+
+    # invalid write protected flag (2) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadWriteProtected):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 02 " + "00 "*57))
+
+    # invalid synchronized flag (2) in a WOZ1 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadSynchronized):
+        wozardry.WozDiskImage(bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 01 01 00 02 " + "00 "*56))
+
+    # invalid synchronized flag (2) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadSynchronized):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 02 " + "00 "*56))
+
+    # invalid cleaned flag (2) in a WOZ1 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadCleaned):
+        wozardry.WozDiskImage(bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 01 01 00 00 02 " + "00 "*55))
+
+    # invalid cleaned flag (2) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadCleaned):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 02 " + "00 "*55))
+
+    # invalid creator (bad UTF-8 bytes) in a WOZ1 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadCreator):
+        wozardry.WozDiskImage(bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 01 01 00 00 00 E0 80 80 " + "00 "*52))
+
+    # invalid creator (bad UTF-8 bytes) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadCreator):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 E0 80 80 " + "00 "*52))
+
+    # invalid disk sides (0) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadDiskSides):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "00 " + "00 "*22))
+
+    # invalid disk sides (3) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadDiskSides):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "03 " + "00 "*22))
+
+    # invalid disk sides (2, when disk type = 1) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadDiskSides):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "02 " + "00 "*22))
+
+    # invalid boot sector format (4) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadBootSectorFormat):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 04 " + "00 "*21))
+
+    # invalid boot sector format (1, when disk type = 2) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadBootSectorFormat):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 02 00 00 00 " + "20 "*32 + "01 01 " + "00 "*21))
+
+    # invalid boot sector format (2, when disk type = 2) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadBootSectorFormat):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 02 00 00 00 " + "20 "*32 + "01 02 " + "00 "*21))
+
+    # invalid boot sector format (3, when disk type = 2) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadBootSectorFormat):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 02 00 00 00 " + "20 "*32 + "01 02 " + "00 "*21))
+
+    # invalid optimal bit timing (23, when disk type = 1) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadOptimalBitTiming):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 17 " + "00 "*20))
+
+    # invalid optimal bit timing (41, when disk type = 1) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadOptimalBitTiming):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 29 " + "00 "*20))
+
+    # invalid optimal bit timing (7, when disk type = 2) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadOptimalBitTiming):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 02 00 00 00 " + "20 "*32 + "01 00 07 " + "00 "*20))
+
+    # invalid optimal bit timing (25, when disk type = 2) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadOptimalBitTiming):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 02 00 00 00 " + "20 "*32 + "01 00 19 " + "00 "*20))
+
+    # invalid optimal bit timing (0, when disk type = 1) in a WOZ2 file
+    # unlike other fields, this does not allow a 0 value to mean "unknown"
+    with pytest.raises(wozardry.WozINFOFormatError_BadOptimalBitTiming):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 00 " + "00 "*20))
+
+    # invalid optimal bit timing (0, when disk type = 2) in a WOZ2 file
+    # unlike other fields, this does not allow a 0 value to mean "unknown"
+    with pytest.raises(wozardry.WozINFOFormatError_BadOptimalBitTiming):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 02 00 00 00 " + "20 "*32 + "01 00 00 " + "00 "*20))
+
+    # invalid compatible hardware (00000010 00000000) in a WOZ2 file
+    # this field only uses the lower 9 bits (for 9 hardware models), so the 7 high bits must all be 0
+    with pytest.raises(wozardry.WozINFOFormatError_BadCompatibleHardware):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 02 " + "00 "*18))
+
+    # invalid compatible hardware (00000100 00000000) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadCompatibleHardware):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 04 " + "00 "*18))
+
+    # invalid compatible hardware (00001000 00000000) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadCompatibleHardware):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 08 " + "00 "*18))
+
+    # invalid compatible hardware (00010000 00000000) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadCompatibleHardware):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 10 " + "00 "*18))
+
+    # invalid compatible hardware (00100000 00000000) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadCompatibleHardware):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 20 " + "00 "*18))
+
+    # invalid compatible hardware (01000000 00000000) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadCompatibleHardware):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 40 " + "00 "*18))
+
+    # invalid compatible hardware (10000000 00000000) in a WOZ2 file
+    with pytest.raises(wozardry.WozINFOFormatError_BadCompatibleHardware):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 80 " + "00 "*18))
+
+def test_parse_tmap():
+    # missing TMAP chunk
+    with pytest.raises(wozardry.WozTMAPFormatError_MissingTMAPChunk):
+        wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 00 " + "00 "*18))
+
+    # TRKS chunk before TMAP chunk
+    with pytest.raises(wozardry.WozTMAPFormatError_MissingTMAPChunk):
+        wozardry.WozDiskImage(
+            bfh(
+                kHeader2 + \
+                "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 00 " + "00 "*18 + \
+                "54 52 4B 53 00 00 00 00 "))
+
+    # TMAP points to non-existent TRK in TRKS chunk
+    with pytest.raises(wozardry.WozTMAPFormatError_BadTRKS):
+        wozardry.WozDiskImage(
+            bfh(
+                kHeader2 + \
+                "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 00 " + "00 "*18 + \
+                "54 4D 41 50 A0 00 00 00 00 " + "FF "*159 + \
+                "54 52 4B 53 00 00 00 00 "))
+
+def test_parse_trks():
+    # this constitutes a valid WOZ2 file header, valid INFO chunk,
+    # valid TMAP chunk with 1 entry pointing to TRK 0 in TRKS chunk,
+    # and the 4-byte TRKS chunk ID
+    prefix = kHeader2 + \
+        "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 00 " + "00 "*18 + \
+        "54 4D 41 50 A0 00 00 00 00 " + "FF "*159 + \
+        "54 52 4B 53 "
+
+    # invalid TRKS chunk with 1 TRK entry whose starting block = 1 (must be 3+)
+    with pytest.raises(wozardry.WozTRKSFormatError_BadStartingBlock):
+        wozardry.WozDiskImage(bfh(prefix + "00 05 00 00 01 00 01 00 01 00 00 00  " + "00 "*1272))
+
+    # invalid TRKS chunk with 1 TRK entry whose starting block = 2 (must be 3+)
+    with pytest.raises(wozardry.WozTRKSFormatError_BadStartingBlock):
+        wozardry.WozDiskImage(bfh(prefix + "00 05 00 00 02 00 01 00 01 00 00 00  " + "00 "*1272))
+
+    # invalid TRKS chunk with 1 TRK entry whose block count = 1 but has no BITS data for the block
+    with pytest.raises(wozardry.WozTRKSFormatError_BadStartingBlock):
+        wozardry.WozDiskImage(bfh(prefix + "00 05 00 00 03 00 01 00 01 00 00 00  " + "FF "*1272))
+
+    # invalid TRKS chunk with 1 TRK entry whose block count = 1 but has only partial BITS data for the block
+    with pytest.raises(wozardry.WozTRKSFormatError_BadBlockCount):
+        wozardry.WozDiskImage(bfh(prefix + "FF 06 00 00 03 00 01 00 01 00 00 00  " + "00 "*1272 + "FF "*511))
+
+def test_parse_meta():
+    def build_meta_chunk(key, value):
+        """|key| and |value| are strings, returns string of hex bytes to feed into bfh()"""
+        bkey = key.encode("utf-8")
+        bvalue = value.encode("utf-8")
+        return (wozardry.to_uint32(len(bkey) + len(bvalue) + 2) + bkey + b'\x09' + bvalue + b'\x0A').hex()
+
+    # this constitutes a valid WOZ2 header, valid INFO chunk,
+    # valid TMAP chunk with 0 entries, and the 4-byte META chunk ID
+    prefix = kHeader2 + \
+        "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 00 " + "00 "*18 + \
+        "54 4D 41 50 A0 00 00 00 " + "FF "*160 + \
+        "4D 45 54 41 "
+
+    # valid META chunk with 0 length
+    wozardry.WozDiskImage(bfh(prefix + "00 00 00 00 "))
+
+    # invalid UTF-8
+    with pytest.raises(wozardry.WozMETAFormatError_EncodingError):
+        wozardry.WozDiskImage(bfh(prefix + "03 00 00 00 E0 80 80"))
+
+    # valid language values
+    for lang in wozardry.kLanguages:
+        wozardry.WozDiskImage(bfh(prefix + build_meta_chunk("language", lang)))
+
+    # invalid language value
+    with pytest.raises(wozardry.WozMETAFormatError_BadLanguage):
+        wozardry.WozDiskImage(bfh(prefix + build_meta_chunk("language", "Englush")))
+
+    # valid requires_ram values
+    for ram in wozardry.kRequiresRAM:
+        wozardry.WozDiskImage(bfh(prefix + build_meta_chunk("requires_ram", ram)))
+
+    # invalid requires_ram value
+    with pytest.raises(wozardry.WozMETAFormatError_BadRAM):
+        wozardry.WozDiskImage(bfh(prefix + build_meta_chunk("requires_ram", "0")))
+
+    # valid requires_machine values
+    for machine in wozardry.kRequiresMachine:
+        wozardry.WozDiskImage(bfh(prefix + build_meta_chunk("requires_machine", machine)))
+
+    # invalid requires_machine value
+    with pytest.raises(wozardry.WozMETAFormatError_BadMachine):
+        wozardry.WozDiskImage(bfh(prefix + build_meta_chunk("requires_machine", "4")))
+
+    # invalid format (duplicate key)
+    bk = "language".encode("utf-8")
+    bv = "English".encode("utf-8")
+    chunk = bk + b"\x09" + bv + b"\x0A" + bk + b"\x09" + bv + b"\0x0A"
+    chunk = (wozardry.to_uint32(len(chunk)) + chunk).hex()
+    with pytest.raises(wozardry.WozMETAFormatError_DuplicateKey):
+        wozardry.WozDiskImage(bfh(prefix + chunk))
+
+    # invalid format (no tab separator between key an dvalue)
+    chunk = bk + bv + b"\x0A"
+    chunk = (wozardry.to_uint32(len(chunk)) + chunk).hex()
+    with pytest.raises(wozardry.WozMETAFormatError_NotEnoughTabs):
+        wozardry.WozDiskImage(bfh(prefix + chunk))
+
+    # invalid format (too many tabs between key and value)
+    chunk = bk + b"\x09"*2 + bv + b"\x0A"
+    chunk = (wozardry.to_uint32(len(chunk)) + chunk).hex()
+    with pytest.raises(wozardry.WozMETAFormatError_TooManyTabs):
+        wozardry.WozDiskImage(bfh(prefix + chunk))
+
+#----- test command-line interface -----
+
+def test_command_verify():
+    """verify a valid WOZ1/WOZ2 file and exit cleanly"""
+    wozardry.parse_args(["verify", kValid1])
+    wozardry.parse_args(["verify", kValid2])
+
+def test_command_dump_woz1(capsys):
+    """dump a WOZ1 file and ensure it prints expected output"""
+    wozardry.parse_args(["dump", kValid1])
+    captured = capsys.readouterr()
+    assert "INFO:  File format version:    1" in captured.out
+    assert "INFO:  Disk type:              5.25-inch (140K)" in captured.out
+    assert "INFO:  Write protected:        no" in captured.out
+    assert "INFO:  Tracks synchronized:    no" in captured.out
+    assert "INFO:  Weakbits cleaned:       no" in captured.out
+    assert "INFO:  Creator:                wozardry" in captured.out
+
+def test_command_dump_woz2(capsys):
+    """dump a WOZ2 file and ensure it prints expected output"""
+    wozardry.parse_args(["dump", kValid2])
+    captured = capsys.readouterr()
+    assert "INFO:  File format version:    2" in captured.out
+    assert "INFO:  Disk type:              5.25-inch (140K)" in captured.out
+    assert "INFO:  Write protected:        no" in captured.out
+    assert "INFO:  Tracks synchronized:    no" in captured.out
+    assert "INFO:  Weakbits cleaned:       no" in captured.out
+    assert "INFO:  Creator:                wozardry" in captured.out
+    assert "INFO:  Boot sector format:     0 (unknown)" in captured.out
+    assert "INFO:  Optimal bit timing:     32 (standard)" in captured.out
+    assert "INFO:  Compatible hardware:    unknown" in captured.out
+    assert "INFO:  Required RAM:           unknown" in captured.out
+    assert "INFO:  Largest track:          0 blocks" in captured.out
+
+def test_command_edit_info_version_1_to_2():
+    """convert a WOZ1 file to WOZ2 and ensure new info fields are set to default values"""
+    with tempfile.NamedTemporaryFile() as tmp:
+        shutil.copy(kValid1, tmp.name)
+
+        wozardry.parse_args(["edit", "-i", "version:2", tmp.name])
+        with open(tmp.name, "rb") as tmpstream:
+            woz = wozardry.WozDiskImage(tmpstream)
+        assert woz.image_type == wozardry.kWOZ2
+        assert woz.info["version"] == 2
+        assert woz.info["boot_sector_format"] == 0
+        assert woz.info["optimal_bit_timing"] == 32
+        assert woz.info["compatible_hardware"] == []
+        assert woz.info["required_ram"] == 0
+
+def test_command_edit_info_disk_type():
+    """edit a WOZ1/WOZ2 file to change the disk type"""
+    # this is pathological, don't do this in real life
+    def f(inputfile):
+        with tempfile.NamedTemporaryFile() as tmp:
+            shutil.copy(inputfile, tmp.name)
+
+            # disk_type = 1 is ok
+            wozardry.parse_args(["edit", "-i", "disk_type:1", tmp.name])
+            with open(tmp.name, "rb") as tmpstream:
+                woz = wozardry.WozDiskImage(tmpstream)
+            assert woz.info["disk_type"] == 1
+
+            # disk_type = 2 is ok
+            wozardry.parse_args(["edit", "-i", "disk_type:2", tmp.name])
+            with open(tmp.name, "rb") as tmpstream:
+                woz = wozardry.WozDiskImage(tmpstream)
+            assert woz.info["disk_type"] == 2
+
+            # disk_type = 0 is not ok
+            with pytest.raises(wozardry.WozINFOFormatError_BadDiskType):
+                wozardry.parse_args(["edit", "-i", "disk_type:0", tmp.name])
+
+            # disk_type = 3 is not ok
+            with pytest.raises(wozardry.WozINFOFormatError_BadDiskType):
+                wozardry.parse_args(["edit", "-i", "disk_type:3", tmp.name])
+    f(kValid1)
+    f(kValid2)
+
+def test_command_edit_info_changing_disk_type_resets_optimal_bit_timing():
+    """edit a WOZ2 file to change the disk type and ensure optimal bit timing is reset to default value"""
+    # this is pathological, don't do this in real life
+    with tempfile.NamedTemporaryFile() as tmp:
+        shutil.copy(kValid2, tmp.name)
+        wozardry.parse_args(["edit", "-i", "disk_type:2", tmp.name])
+        with open(tmp.name, "rb") as tmpstream:
+            woz = wozardry.WozDiskImage(tmpstream)
+        assert woz.info["optimal_bit_timing"] == wozardry.kDefaultBitTiming[2]
+
+def test_command_edit_info_boolean_flags():
+    """edit a WOZ1/WOZ2 file to change Boolean flags in a variety of ways and ensure they change"""
+    def f(inputfile):
+        with tempfile.NamedTemporaryFile() as tmp:
+            shutil.copy(inputfile, tmp.name)
+
+            for flag in ("write_protected", "synchronized", "cleaned"):
+                for true_value, false_value in (("1", "0"),
+                                                ("yes", "no"),
+                                                ("YES", "No"),
+                                                ("true", "false"),
+                                                ("tRuE", "FaLsE")):
+                    wozardry.parse_args(["edit", "-i", "%s:%s" % (flag, true_value), tmp.name])
+                    with open(tmp.name, "rb") as tmpstream:
+                        woz = wozardry.WozDiskImage(tmpstream)
+                    assert woz.info[flag] == True
+                    wozardry.parse_args(["edit", "-i", "%s:%s" % (flag, false_value), tmp.name])
+                    with open(tmp.name, "rb") as tmpstream:
+                        woz = wozardry.WozDiskImage(tmpstream)
+                    assert woz.info[flag] == False
+    f(kValid1)
+    f(kValid2)
+
+def test_command_edit_disk_sides():
+    """edit a WOZ2 file to change disk sides"""
+    with tempfile.NamedTemporaryFile() as tmp:
+        shutil.copy(kValid2, tmp.name)
+
+        # this file is a 5.25-inch disk image
+        with open(tmp.name, "rb") as tmpstream:
+            woz = wozardry.WozDiskImage(tmpstream)
+        assert woz.info["disk_type"] == 1
+        assert woz.info["disk_sides"] == 1
+
+        # 5.25-inch disk images can only be "1-sided"
+        with pytest.raises(wozardry.WozINFOFormatError_BadDiskSides):
+            wozardry.parse_args(["edit", "-i", "disk_sides:2", tmp.name])
+
+        # now change it to a 3.5-inch disk image
+        wozardry.parse_args(["edit", "-i", "disk_type:2", tmp.name])
+        with open(tmp.name, "rb") as tmpstream:
+            woz = wozardry.WozDiskImage(tmpstream)
+        assert woz.info["disk_type"] == 2
+
+        # 3.5-inch disk images can be 1- or 2-sided
+        wozardry.parse_args(["edit", "-i", "disk_sides:2", tmp.name])
+        with open(tmp.name, "rb") as tmpstream:
+            woz = wozardry.WozDiskImage(tmpstream)
+        assert woz.info["disk_sides"] == 2
+        wozardry.parse_args(["edit", "-i", "disk_sides:1", tmp.name])
+        with open(tmp.name, "rb") as tmpstream:
+            woz = wozardry.WozDiskImage(tmpstream)
+        assert woz.info["disk_sides"] == 1
+
+        # ...but not 3-sided, that's silly
+        with pytest.raises(wozardry.WozINFOFormatError_BadDiskSides):
+            wozardry.parse_args(["edit", "-i", "disk_sides:3", tmp.name])
+
+def test_command_edit_language():
+    """edit a WOZ1/WOZ2 file to change the metadata language field"""
+    def f(inputfile):
+        with tempfile.NamedTemporaryFile() as tmp:
+            shutil.copy(inputfile, tmp.name)
+
+            for lang in wozardry.kLanguages:
+                wozardry.parse_args(["edit", "-m", "language:%s" % lang, tmp.name])
+                with open(tmp.name, "rb") as tmpstream:
+                    woz = wozardry.WozDiskImage(tmpstream)
+                assert woz.meta["language"] == lang
+    f(kValid1)
+    f(kValid2)
+
+def test_command_edit_requires_ram():
+    """edit a WOZ1/WOZ2 file to change the metadata requires_ram field"""
+    def f(inputfile):
+        with tempfile.NamedTemporaryFile() as tmp:
+            shutil.copy(inputfile, tmp.name)
+
+            for ram in wozardry.kRequiresRAM:
+                wozardry.parse_args(["edit", "-m", "requires_ram:%s" % ram, tmp.name])
+                with open(tmp.name, "rb") as tmpstream:
+                    woz = wozardry.WozDiskImage(tmpstream)
+                assert woz.meta["requires_ram"] == ram
+
+            # invalid required RAM (must be one of enumerated values)
+            with pytest.raises(wozardry.WozMETAFormatError_BadRAM):
+                wozardry.parse_args(["edit", "-m", "requires_ram:65K", tmp.name])
+
+    f(kValid1)
+    f(kValid2)
+
+def test_command_edit_requires_machine():
+    """edit a WOZ1/WOZ2 file to change the metadata requires_machine field"""
+    def f(inputfile):
+        with tempfile.NamedTemporaryFile() as tmp:
+            shutil.copy(inputfile, tmp.name)
+
+            for model in wozardry.kRequiresMachine:
+                wozardry.parse_args(["edit", "-m", "requires_machine:%s" % model, tmp.name])
+                with open(tmp.name, "rb") as tmpstream:
+                    woz = wozardry.WozDiskImage(tmpstream)
+                assert woz.meta["requires_machine"] == model
+
+            # invalid machine (Apple IV)
+            with pytest.raises(wozardry.WozMETAFormatError_BadMachine):
+                wozardry.parse_args(["edit", "-m", "requires_machine:4", tmp.name])
+
+    f(kValid1)
+    f(kValid2)