diff --git a/README.md b/README.md index f6eeb6d..0700167 100644 --- a/README.md +++ b/README.md @@ -1,41 +1,72 @@ -# Installation +# `wozardry` + +`wozardry` is a multi-purpose tool for manipulating `.woz` disk images. It can +validate file structure, edit metadata, import and export metadata in `JSON` +format, remove unused tracks, and provides a programmatic interface to "read" +bits and nibbles from a disk image. It supports both WOZ 1.0 and WOZ 2.0 files +and can convert files from one version to the other. + +* [Installation](#installation) +* [Command line interface](#command-line-interface) + * [`verify` command](#verify-command) + * [`dump` command](#dump-command) + * [`edit` command](#edit-command) + * [How to convert WOZ1 to WOZ2 files](#how-to-convert-woz1-to-woz2-files) + * [`import` and `export` commands](#import-and-export-commands) + * [`remove` command](#remove-command) +* [Python interface](#python-interface) + * [`WozDiskImage`](#wozdiskimage) + * [Loading and saving files on disk](#loading-and-saving-files-on-disk) + +## Installation `wozardry` is written in [Python 3](https://www.python.org). -It requires [bitarray](https://pypi.org/project/bitarray/), which can be installed thusly: +It requires [bitarray](https://pypi.org/project/bitarray/), which can be +installed thusly: ``` $ pip3 install -U bitarray ``` -(Developers who wish to run the test suite should also install the `pytest` module with `pip3 install -U pytest`) +(Developers who wish to run the test suite should also install the `pytest` +module with `pip3 install -U pytest`) -# Command line usage +## Command line interface -wozardry is primarily designed to be used on the command line to directly manipulate `.woz` disk images. It supports multiple commands, which are listed in the `wozardry -h` output. +wozardry is primarily designed to be used on the command line to directly +manipulate `.woz` disk images in place. It supports multiple commands, which are +listed in the `wozardry -h` output. -## `verify` command +### `verify` command -This command verifies the file structure and metadata of a `.woz` disk image. It produces no output unless a problem is found. +This command verifies the file structure and metadata of a `.woz` disk image. +It produces no output unless a problem is found. Sample usage: ``` -$ wozardry verify "woz test images/WOZ 2.0/DOS 3.3 System Master.woz" +$ wozardry verify "WOZ 2.0/DOS 3.3 System Master.woz" ``` -**Tip**: you can [download a collection of .woz test images](http://evolutioninteractive.com/applesauce/woz_images.zip). +**Tip**: you can [download a collection of .woz test +images](http://evolutioninteractive.com/applesauce/woz_images.zip). -The `verify` command does not "read" the data on the disk like an emulator would. It merely verifies the structure of the `.woz` file itself and applies a few sanity checks on the embedded metadata (if any). The disk may or may not boot in an emulator. It may not pass its own copy protection checks. It may not have the data you expected, or any data at all. `wozardry` can't answer those questions. +The `verify` command does not "read" the data on the disk like an emulator +would. It merely verifies the structure of the `.woz` file itself and applies a +few sanity checks on the embedded metadata (if any). The disk may or may not +boot in an emulator. It may not pass its own copy protection checks. It may not +have the data you expected, or any data at all. `wozardry` can't answer those +questions. -## `dump` command +### `dump` command Prints all available information and metadata in a `.woz` disk image. Sample usage: ``` -$ wozardry dump "woz test images/WOZ 2.0/Wings of Fury - Disk 1, Side A.woz" +$ wozardry dump "WOZ 2.0/Wings of Fury - Disk 1, Side A.woz" TMAP: Track 0.00 TRKS 0 TMAP: Track 0.25 TRKS 0 TMAP: Track 0.75 TRKS 1 @@ -79,25 +110,36 @@ INFO: Required RAM: 128K INFO: Largest track: 13 blocks ``` -The `TMAP` section (stands for "track map") shows which tracks are included in the disk image. As you can see from the above sample, the same bitstream data can be assigned to multiple tracks, usually adjacent quarter tracks. Each bitstream is stored only once in the `.woz` file. +The `TMAP` section (stands for "track map") shows which tracks are included in +the disk image. As you can see from the above sample, the same bitstream data +can be assigned to multiple tracks, usually adjacent quarter tracks. Each +bitstream is stored only once in the `.woz` file. -The `META` section shows any embedded metadata, such as copyright and version. This section is optional; not all `.woz` files will have the same metadata fields, and some may have none at all. +The `META` section shows any embedded metadata, such as copyright and +version. This section is optional; not all `.woz` files will have the same +metadata fields, and some may have none at all. -The `INFO` section shows information that emulators or other programs might need to know, such as the boot sector format (13- or 16-sector, or both) and whether the disk is write protected. All `INFO` fields are required and are included in every `.woz` file. +The `INFO` section shows information that emulators or other programs might need +to know, such as the boot sector format (13- or 16-sector, or both) and whether +the disk is write protected. All `INFO` fields are required and are included in +every `.woz` file. -The output of the `dump` command is designed to by grep-able, if you're into that kind of thing. +The output of the `dump` command is designed to by grep-able, if you're into +that kind of thing. ``` -$ wozardry dump "woz test images/WOZ 2.0/Wings of Fury - Disk 1, Side A.woz" | grep "^INFO" +$ wozardry dump "WOZ 2.0/Wings of Fury - Disk 1, Side A.woz" | grep "^INFO" ``` will show just the `INFO` section. -**Tip**: [the .woz specification](https://applesaucefdc.com/woz/reference2/) lists the standard metadata fields and the acceptable values of all info fields. +**Tip**: [the .woz specification](https://applesaucefdc.com/woz/reference2/) +lists the standard metadata fields and the acceptable values of all info fields. -## `edit` command +### `edit` command -This command lets you modify any information or metadata field in a `.woz` file. This is where the fun(\*) starts. +This command lets you modify any information or metadata field in a `.woz` +file. This is where the fun(\*) starts. (\*) not guaranteed, actual fun may vary @@ -133,32 +175,36 @@ Tips: - Use "key:" with no value to delete a metadata field. - Keys are case-sensitive. - Some values have format restrictions; read the .woz specification. - ``` - - Let's look at some examples. - - Working with this same "Wings of Fury" disk image, let's give the game author his due by adding the `developer` metadata field: - - ``` - $ wozardry edit -m "developer:Steve Waldo" "woz test images/WOZ 2.0/Wings of Fury - Disk 1, Side A.woz" - ``` - -Metadata fields are arbitrary; there is a standard set listed in [the .woz specification](https://applesaucefdc.com/woz/reference2/), but you can add your own. - -``` -$ wozardry edit -m "genre:action" -m "perspective:side view" "woz test images/WOZ 2.0/Wings of Fury - Disk 1, Side A.woz" ``` -You can use a similar syntax to remove metadata fields that don't apply to this disk. +Let's look at some examples. + +Working with this same "Wings of Fury" disk image, let's give the game author +his due by adding the `developer` metadata field: ``` -$ wozardry edit -m "version:" -m "notes:" -m "side_name:" -m "subtitle:" "woz test images/WOZ 2.0/Wings of Fury - Disk 1, Side A.woz" +$ wozardry edit -m "developer:Steve Waldo" "WOZ 2.0/Wings of Fury - Disk 1, Side A.woz" +``` + +Metadata fields are arbitrary; there is a standard set listed in [the .woz +specification](https://applesaucefdc.com/woz/reference2/), but you can add your +own. + +``` +$ wozardry edit -m "genre:action" -m "perspective:side view" "WOZ 2.0/Wings of Fury - Disk 1, Side A.woz" +``` + +You can use a similar syntax to remove metadata fields that don't apply to this +disk. + +``` +$ wozardry edit -m "version:" -m "notes:" -m "side_name:" -m "subtitle:" "WOZ 2.0/Wings of Fury - Disk 1, Side A.woz" ``` Now let's look at that metadata section again: ``` -$ wozardry dump "woz test images/WOZ 2.0/Wings of Fury - Disk 1, Side A.woz" | grep "^META" +$ wozardry dump "WOZ 2.0/Wings of Fury - Disk 1, Side A.woz" | grep "^META" META: language: English META: publisher: Broderbund META: developer: Steve Waldo @@ -176,28 +222,43 @@ META: genre: action META: perspective: side view ``` -You can modify `INFO` fields using a similar syntax (`-i` instead of `-m`), but be aware that `INFO` fields are highly constrained, and incorrect values can have noticeable effects in emulators. `wozardry` will reject any values that are nonsensical or out of range, but even in-range values can render the disk image unbootable. For example, the "optimal bit timing" field specifies the rate at which bits appear in the floppy drive data latch; if the rate is not what the disk's low-level RWTS code is expecting, the disk may be unable to read itself. +You can modify `INFO` fields using a similar syntax (`-i` instead of `-m`), but +be aware that `INFO` fields are highly constrained, and incorrect values can +have noticeable effects in emulators. `wozardry` will reject any values that are +nonsensical or out of range, but even in-range values can render the disk image +unbootable. For example, the "optimal bit timing" field specifies the rate at +which bits appear in the floppy drive data latch; if the rate is not what the +disk's low-level RWTS code is expecting, the disk may be unable to read itself. -Nonetheless, here are some examples of changing `INFO` fields. To tell emulators that a disk is not write-protected, set the `write_protected` field to `no`, `false`, or `0`. +Nonetheless, here are some examples of changing `INFO` fields. To tell emulators +that a disk is not write-protected, set the `write_protected` field to `no`, +`false`, or `0`. ``` -$ wozardry edit -i "write_protected:no" "woz test images/WOZ 2.0/Wings of Fury - Disk 1, Side A.woz" +$ wozardry edit -i "write_protected:no" "WOZ 2.0/Wings of Fury - Disk 1, Side A.woz" ``` -To tell emulators that the disk only runs on certain Apple II models, set the `compatible_hardware` field with a pipe-separated list. (Values may appear in any order. See `kRequiresMachine` in the `wozardry` source code for all the acceptable values.) +To tell emulators that the disk only runs on certain Apple II models, set the +`compatible_hardware` field with a pipe-separated list. (Values may appear in +any order. See `kRequiresMachine` in the `wozardry` source code for all the +acceptable values.) ``` -$ wozardry edit -i "compatible_hardware:2e|2e+|2c|2gs" "woz test images/WOZ 2.0/Wings of Fury - Disk 1, Side A.woz" +$ wozardry edit -i "compatible_hardware:2e|2e+|2c|2gs" "WOZ 2.0/Wings of Fury - Disk 1, Side A.woz" ``` ### How to convert WOZ1 to WOZ2 files -As of this writing, the `.woz` specification has undergone one major revision, which changed the internal structure of a `.woz` file and added several new `INFO` fields. Both file formats use the `.woz` file extension; they are distinguished by magic bytes (`WOZ1` vs. `WOZ2`) within the file. +As of this writing, the `.woz` specification has undergone one major revision, +which changed the internal structure of a `.woz` file and added several new +`INFO` fields. Both file formats use the `.woz` file extension; they are +distinguished by magic bytes (`WOZ1` vs. `WOZ2`) within the file. -Let's say you have an older `WOZ1` file, like this one from the `WOZ 1.0` directory of the official test images collection: +Let's say you have an older `WOZ1` file, like this one from the `WOZ 1.0` +directory of the official test images collection: ``` -$ wozardry dump "woz test images/WOZ 1.0/Wings of Fury - Disk 1, Side A.woz" | grep "^INFO" +$ wozardry dump "WOZ 1.0/Wings of Fury - Disk 1, Side A.woz" | grep "^INFO" INFO: File format version: 1 INFO: Disk type: 5.25-inch (140K) INFO: Write protected: yes @@ -206,12 +267,13 @@ INFO: Weakbits cleaned: yes INFO: Creator: Applesauce v0.29 ``` -The "file format version" confirms this is a `WOZ1` file. To convert it to a `WOZ2` file, set the `version` field to `2`. +The "file format version" confirms this is a `WOZ1` file. To convert it to a +`WOZ2` file, set the `version` field to `2`. ``` -$ wozardry -i "version:2" "woz test images/WOZ 1.0/Wings of Fury - Disk 1, Side A.woz" +$ wozardry -i "version:2" "WOZ 1.0/Wings of Fury - Disk 1, Side A.woz" -$ wozardry dump "woz test images/WOZ 1.0/Wings of Fury - Disk 1, Side A.woz" | grep "^INFO" +$ wozardry dump "WOZ 1.0/Wings of Fury - Disk 1, Side A.woz" | grep "^INFO" INFO: File format version: 2 INFO: Disk type: 5.25-inch (140K) INFO: Write protected: yes @@ -225,14 +287,17 @@ INFO: Required RAM: unknown INFO: Largest track: 13 blocks ``` -All the new (v2-specific) `INFO` fields are initialized with default values. Existing fields like the write-protected flag are retained. ("Largest track" is a calculated field and can not be set directly.) +All the new (v2-specific) `INFO` fields are initialized with default +values. Existing fields like the write-protected flag are retained. ("Largest +track" is a calculated field and can not be set directly.) -## `import` and `export` commands +### `import` and `export` commands -These commands allow you to access the information and metadata in a `.woz` file in `JSON` format. +These commands allow you to access the information and metadata in a `.woz` file +in `JSON` format. ``` -$ wozardry export "woz test images/WOZ 2.0/Wings of Fury - Disk 1, Side A.woz" +$ wozardry export "WOZ 2.0/Wings of Fury - Disk 1, Side A.woz" { "woz": { "info": { @@ -289,16 +354,106 @@ $ wozardry export "woz test images/WOZ 2.0/Wings of Fury - Disk 1, Side A.woz" } ``` -You can pipe the output of the `export` command to the `import` command to copy metadata from one `.woz` file to another. +You can pipe the output of the `export` command to the `import` command to copy +metadata from one `.woz` file to another. ``` $ wozardry export game-side-a.woz | wozardry import game-side-b.woz ``` -Technically, this merges metadata. All metadata fields in `game-side-a.woz` will be copied to `game-side-b.woz`, overwriting any existing values for those fields. But if `game-side-b.woz` already had additional metadata fields that were not present in `game-side-a.woz`, those will be retained. +Technically, this merges metadata. All metadata fields in `game-side-a.woz` will +be copied to `game-side-b.woz`, overwriting any existing values for those +fields. But if `game-side-b.woz` already had additional metadata fields that +were not present in `game-side-a.woz`, those will be retained. -**Tip**: [a2rchery](https://github.com/a2-4am/a2rchery) is a tool to manipulate `.a2r` flux images. These `.a2r` files can also have embedded metadata, just like `.woz` files. And guess what! `a2rchery` also has `import` and `export` commands, just like `wozardry`. You see where this is going. +**Tip**: [a2rchery](https://github.com/a2-4am/a2rchery) is a tool to manipulate +`.a2r` flux images. These `.a2r` files can also have embedded metadata, just +like `.woz` files. And guess what! `a2rchery` also has `import` and `export` +commands, just like `wozardry`. You see where this is going. ``` $ wozardry export game.woz | a2rchery import game.a2r ``` + +### `remove` command + +This command allow you to remove one or more tracks from a `.woz` disk image. + +Tracks are specified in quarter tracks, in base 10 (not base 16). Multiple +tracks can be removed at once. + +``` +$ wozardry remove -t0.25 -t0.5 -t0.75 -t1 -t1.25 -t35 "Gamma Goblins.woz" +``` + +**Note**: tracks are stored as indices in the `TMAP` chunk, and multiple tracks +can refer to the same bitstream (stored in the `TRKS` chunk). If you remove all +tracks that refer to a bitstream, the bitstream will be removed from the `TRKS` +chunk and all the indices in the `TMAP` chunk will be adjusted accordingly. + +## Python interface + +### `WozDiskImage` + +This represents a single WOZ disk image. You can create it from scratch, load it +from a file on disk, or parse it from a bytestream in memory. + +``` +>>> import wozardry +>>> woz_image = wozardry.WozDiskImage() +>>> woz_image.woz_version +2 +``` + +This newly created `woz_image` already has an `info` dictionary with all the +required fields in the `INFO` chunk. + +``` +>>> from pprint import pprint +>>> pprint(woz_image.info) +OrderedDict([('version', 2), + ('disk_type', 1), + ('write_protected', False), + ('synchronized', False), + ('cleaned', False), + ('creator', 'wozardry 2.0-beta'), + ('disk_sides', 1), + ('boot_sector_format', 0), + ('optimal_bit_timing', 32), + ('compatible_hardware', []), + ('required_ram', 0)]) +>>> woz_image.info["compatible_hardware"] = ["2", "2+"] +>>> woz_image.info["compatible_hardware"] +['2', '2+'] +``` + +It also has an empty `meta` dictionary for metadata. + +``` +>>> pprint(woz_image.meta) +OrderedDict() +>>> woz_image.meta["copyright"] = "1981" +>>> woz_image.meta["developer"] = "Chuckles" +>>> pprint(woz_image.meta) +OrderedDict([('copyright', '1981'), + ('developer', 'Chuckles')]) +``` + +### Loading and saving files on disk + +To load a `.woz` disk image from a file (or any file-like object), open the file +and pass it to the `WozDiskImage` constructor. Be sure to open files in binary +mode. + +``` +>>> with open("Wings of Fury.woz", "rb") as fp: +... w = wozardry.WozDiskImage(fp) +``` + +To save a file, serialize the `WozDiskImage` object with the `bytes()` method +and write that to disk. Be sure to open files in binary mode. + +``` +>>> with open("Wings of Fury.woz", "wb") as fp: +... fp.write(bytes(w)) +``` diff --git a/test_wozardry.py b/test_wozardry.py index 69a4ddd..9223d9d 100755 --- a/test_wozardry.py +++ b/test_wozardry.py @@ -33,223 +33,309 @@ def bfh(s): def test_parse_header(): # incomplete header with pytest.raises(wozardry.WozEOFError): - wozardry.WozReader(stream=bfh("57 4F 5A 32")) + wozardry.WozDiskImage(bfh("57 4F 5A 32")) with pytest.raises(wozardry.WozEOFError): - wozardry.WozReader(stream=bfh("57 4F 5A 32 FF 0A 0D")) + wozardry.WozDiskImage(bfh("57 4F 5A 32 FF 0A 0D")) with pytest.raises(wozardry.WozEOFError): - wozardry.WozReader(stream=bfh("57 4F 5A 32 FF 0A 0D 0A 00 00 00")) + wozardry.WozDiskImage(bfh("57 4F 5A 32 FF 0A 0D 0A 00 00 00")) # invalid signature at offset 0 with pytest.raises(wozardry.WozHeaderError_NoWOZMarker): - wozardry.WozReader(stream=bfh("57 4F 5A 30 00 00 00 00")) + wozardry.WozDiskImage(bfh("57 4F 5A 30 00 00 00 00")) # invalid signature at offset 0 with pytest.raises(wozardry.WozHeaderError_NoWOZMarker): - wozardry.WozReader(stream=bfh("57 4F 5A 33 00 00 00 00")) + wozardry.WozDiskImage(bfh("57 4F 5A 33 00 00 00 00")) # missing FF byte at offset 4 with pytest.raises(wozardry.WozHeaderError_NoFF): - wozardry.WozReader(stream=bfh("57 4F 5A 32 00 0A 0D 0A")) + wozardry.WozDiskImage(bfh("57 4F 5A 32 00 0A 0D 0A")) # missing 0A byte at offset 5 with pytest.raises(wozardry.WozHeaderError_NoLF): - wozardry.WozReader(stream=bfh("57 4F 5A 32 FF 0D 0D 0D")) + wozardry.WozDiskImage(bfh("57 4F 5A 32 FF 0D 0D 0D")) # missing 0D byte at offset 6 with pytest.raises(wozardry.WozHeaderError_NoLF): - wozardry.WozReader(stream=bfh("57 4F 5A 32 FF 0A 0A 0A")) + wozardry.WozDiskImage(bfh("57 4F 5A 32 FF 0A 0A 0A")) # missing 0A byte at offset 7 with pytest.raises(wozardry.WozHeaderError_NoLF): - wozardry.WozReader(stream=bfh("57 4F 5A 32 FF 0A 0D 0D")) + wozardry.WozDiskImage(bfh("57 4F 5A 32 FF 0A 0D 0D")) def test_parse_info(): # TMAP chunk before INFO chunk with pytest.raises(wozardry.WozINFOFormatError_MissingINFOChunk): - wozardry.WozReader(stream=bfh(kHeader2 + "54 4D 41 50 A0 00 00 00 " + "FF "*160)) + wozardry.WozDiskImage(bfh(kHeader2 + "54 4D 41 50 A0 00 00 00 " + "FF "*160)) # wrong INFO chunk size (too small) with pytest.raises(wozardry.WozINFOFormatError): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3B 00 00 00 " + "00 "*59)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3B 00 00 00 " + "00 "*59)) # wrong INFO chunk size (too big) with pytest.raises(wozardry.WozINFOFormatError): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3D 00 00 00 " + "00 "*61)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3D 00 00 00 " + "00 "*61)) # invalid version (0) in a WOZ1 file with pytest.raises(wozardry.WozINFOFormatError_BadVersion): - wozardry.WozReader(stream=bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 00" + "00 "*59)) + wozardry.WozDiskImage(bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 00" + "00 "*59)) # invalid version (0) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadVersion): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 00" + "00 "*59)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 00" + "00 "*59)) # invalid version (1) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadVersion): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 01" + "00 "*59)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 01" + "00 "*59)) # invalid version (2) in a WOZ1 file with pytest.raises(wozardry.WozINFOFormatError_BadVersion): - wozardry.WozReader(stream=bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 02" + "00 "*59)) + wozardry.WozDiskImage(bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 02" + "00 "*59)) # invalid disk type (0) in a WOZ1 file with pytest.raises(wozardry.WozINFOFormatError_BadDiskType): - wozardry.WozReader(stream=bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 01 00 " + "00 "*58)) + wozardry.WozDiskImage(bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 01 00 " + "00 "*58)) # invalid disk type (0) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadDiskType): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 00 " + "00 "*58)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 00 " + "00 "*58)) # invalid disk type (3) in a WOZ1 file with pytest.raises(wozardry.WozINFOFormatError_BadDiskType): - wozardry.WozReader(stream=bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 01 03 " + "00 "*58)) + wozardry.WozDiskImage(bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 01 03 " + "00 "*58)) # invalid disk type (3) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadDiskType): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 03 " + "00 "*58)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 03 " + "00 "*58)) # invalid write protected flag (2) in a WOZ1 file with pytest.raises(wozardry.WozINFOFormatError_BadWriteProtected): - wozardry.WozReader(stream=bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 01 01 02 " + "00 "*57)) + wozardry.WozDiskImage(bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 01 01 02 " + "00 "*57)) # invalid write protected flag (2) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadWriteProtected): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 02 " + "00 "*57)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 02 " + "00 "*57)) # invalid synchronized flag (2) in a WOZ1 file with pytest.raises(wozardry.WozINFOFormatError_BadSynchronized): - wozardry.WozReader(stream=bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 01 01 00 02 " + "00 "*56)) + wozardry.WozDiskImage(bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 01 01 00 02 " + "00 "*56)) # invalid synchronized flag (2) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadSynchronized): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 02 " + "00 "*56)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 02 " + "00 "*56)) # invalid cleaned flag (2) in a WOZ1 file with pytest.raises(wozardry.WozINFOFormatError_BadCleaned): - wozardry.WozReader(stream=bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 01 01 00 00 02 " + "00 "*55)) + wozardry.WozDiskImage(bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 01 01 00 00 02 " + "00 "*55)) # invalid cleaned flag (2) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadCleaned): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 02 " + "00 "*55)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 02 " + "00 "*55)) # invalid creator (bad UTF-8 bytes) in a WOZ1 file with pytest.raises(wozardry.WozINFOFormatError_BadCreator): - wozardry.WozReader(stream=bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 01 01 00 00 00 E0 80 80 " + "00 "*52)) + wozardry.WozDiskImage(bfh(kHeader1 + "49 4E 46 4F 3C 00 00 00 01 01 00 00 00 E0 80 80 " + "00 "*52)) # invalid creator (bad UTF-8 bytes) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadCreator): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 E0 80 80 " + "00 "*52)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 E0 80 80 " + "00 "*52)) # invalid disk sides (0) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadDiskSides): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "00 " + "00 "*22)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "00 " + "00 "*22)) # invalid disk sides (3) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadDiskSides): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "03 " + "00 "*22)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "03 " + "00 "*22)) # invalid disk sides (2, when disk type = 1) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadDiskSides): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "02 " + "00 "*22)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "02 " + "00 "*22)) # invalid boot sector format (4) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadBootSectorFormat): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 04 " + "00 "*21)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 04 " + "00 "*21)) # invalid boot sector format (1, when disk type = 2) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadBootSectorFormat): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 02 00 00 00 " + "20 "*32 + "01 01 " + "00 "*21)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 02 00 00 00 " + "20 "*32 + "01 01 " + "00 "*21)) # invalid boot sector format (2, when disk type = 2) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadBootSectorFormat): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 02 00 00 00 " + "20 "*32 + "01 02 " + "00 "*21)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 02 00 00 00 " + "20 "*32 + "01 02 " + "00 "*21)) # invalid boot sector format (3, when disk type = 2) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadBootSectorFormat): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 02 00 00 00 " + "20 "*32 + "01 02 " + "00 "*21)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 02 00 00 00 " + "20 "*32 + "01 02 " + "00 "*21)) # invalid optimal bit timing (23, when disk type = 1) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadOptimalBitTiming): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 17 " + "00 "*20)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 17 " + "00 "*20)) # invalid optimal bit timing (41, when disk type = 1) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadOptimalBitTiming): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 29 " + "00 "*20)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 29 " + "00 "*20)) # invalid optimal bit timing (7, when disk type = 2) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadOptimalBitTiming): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 02 00 00 00 " + "20 "*32 + "01 00 07 " + "00 "*20)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 02 00 00 00 " + "20 "*32 + "01 00 07 " + "00 "*20)) # invalid optimal bit timing (25, when disk type = 2) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadOptimalBitTiming): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 02 00 00 00 " + "20 "*32 + "01 00 19 " + "00 "*20)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 02 00 00 00 " + "20 "*32 + "01 00 19 " + "00 "*20)) # invalid optimal bit timing (0, when disk type = 1) in a WOZ2 file # unlike other fields, this does not allow a 0 value to mean "unknown" with pytest.raises(wozardry.WozINFOFormatError_BadOptimalBitTiming): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 00 " + "00 "*20)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 00 " + "00 "*20)) # invalid optimal bit timing (0, when disk type = 2) in a WOZ2 file # unlike other fields, this does not allow a 0 value to mean "unknown" with pytest.raises(wozardry.WozINFOFormatError_BadOptimalBitTiming): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 02 00 00 00 " + "20 "*32 + "01 00 00 " + "00 "*20)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 02 00 00 00 " + "20 "*32 + "01 00 00 " + "00 "*20)) # invalid compatible hardware (00000010 00000000) in a WOZ2 file # this field only uses the lower 9 bits (for 9 hardware models), so the 7 high bits must all be 0 with pytest.raises(wozardry.WozINFOFormatError_BadCompatibleHardware): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 02 " + "00 "*18)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 02 " + "00 "*18)) # invalid compatible hardware (00000100 00000000) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadCompatibleHardware): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 04 " + "00 "*18)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 04 " + "00 "*18)) # invalid compatible hardware (00001000 00000000) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadCompatibleHardware): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 08 " + "00 "*18)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 08 " + "00 "*18)) # invalid compatible hardware (00010000 00000000) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadCompatibleHardware): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 10 " + "00 "*18)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 10 " + "00 "*18)) # invalid compatible hardware (00100000 00000000) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadCompatibleHardware): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 20 " + "00 "*18)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 20 " + "00 "*18)) # invalid compatible hardware (01000000 00000000) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadCompatibleHardware): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 40 " + "00 "*18)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 40 " + "00 "*18)) # invalid compatible hardware (10000000 00000000) in a WOZ2 file with pytest.raises(wozardry.WozINFOFormatError_BadCompatibleHardware): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 80 " + "00 "*18)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 80 " + "00 "*18)) def test_parse_tmap(): # missing TMAP chunk with pytest.raises(wozardry.WozTMAPFormatError_MissingTMAPChunk): - wozardry.WozReader(stream=bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 00 " + "00 "*18)) + wozardry.WozDiskImage(bfh(kHeader2 + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 00 " + "00 "*18)) # TRKS chunk before TMAP chunk with pytest.raises(wozardry.WozTMAPFormatError_MissingTMAPChunk): - wozardry.WozReader( - stream=bfh( + wozardry.WozDiskImage( + bfh( kHeader2 + \ "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 00 " + "00 "*18 + \ "54 52 4B 53 00 00 00 00 ")) # TMAP points to non-existent TRK in TRKS chunk with pytest.raises(wozardry.WozTMAPFormatError_BadTRKS): - wozardry.WozReader( - stream=bfh( + wozardry.WozDiskImage( + bfh( kHeader2 + \ "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 00 " + "00 "*18 + \ "54 4D 41 50 A0 00 00 00 00 " + "FF "*159 + \ "54 52 4B 53 00 00 00 00 ")) -def test_parse_meta(): - +def test_parse_trks(): + # this constitutes a valid WOZ2 file header, valid INFO chunk, + # valid TMAP chunk with 1 entry pointing to TRK 0 in TRKS chunk, + # and the 4-byte TRKS chunk ID + prefix = kHeader2 + \ + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 00 " + "00 "*18 + \ + "54 4D 41 50 A0 00 00 00 00 " + "FF "*159 + \ + "54 52 4B 53 " + # invalid TRKS chunk with 1 TRK entry whose starting block = 1 (must be 3+) + with pytest.raises(wozardry.WozTRKSFormatError_BadStartingBlock): + wozardry.WozDiskImage(bfh(prefix + "00 05 00 00 01 00 01 00 01 00 00 00 " + "00 "*1272)) + + # invalid TRKS chunk with 1 TRK entry whose starting block = 2 (must be 3+) + with pytest.raises(wozardry.WozTRKSFormatError_BadStartingBlock): + wozardry.WozDiskImage(bfh(prefix + "00 05 00 00 02 00 01 00 01 00 00 00 " + "00 "*1272)) + + # invalid TRKS chunk with 1 TRK entry whose block count = 1 but has no BITS data for the block + with pytest.raises(wozardry.WozTRKSFormatError_BadStartingBlock): + wozardry.WozDiskImage(bfh(prefix + "00 05 00 00 03 00 01 00 01 00 00 00 " + "FF "*1272)) + + # invalid TRKS chunk with 1 TRK entry whose block count = 1 but has only partial BITS data for the block + with pytest.raises(wozardry.WozTRKSFormatError_BadBlockCount): + wozardry.WozDiskImage(bfh(prefix + "FF 06 00 00 03 00 01 00 01 00 00 00 " + "00 "*1272 + "FF "*511)) + +def test_parse_meta(): + def build_meta_chunk(key, value): + """|key| and |value| are strings, returns string of hex bytes to feed into bfh()""" + bkey = key.encode("utf-8") + bvalue = value.encode("utf-8") + return (wozardry.to_uint32(len(bkey) + len(bvalue) + 2) + bkey + b'\x09' + bvalue + b'\x0A').hex() + + # this constitutes a valid WOZ2 header, valid INFO chunk, + # valid TMAP chunk with 0 entries, and the 4-byte META chunk ID + prefix = kHeader2 + \ + "49 4E 46 4F 3C 00 00 00 02 01 00 00 00 " + "20 "*32 + "01 00 20 00 00 " + "00 "*18 + \ + "54 4D 41 50 A0 00 00 00 " + "FF "*160 + \ + "4D 45 54 41 " + + # valid META chunk with 0 length + wozardry.WozDiskImage(bfh(prefix + "00 00 00 00 ")) + + # invalid UTF-8 + with pytest.raises(wozardry.WozMETAFormatError_EncodingError): + wozardry.WozDiskImage(bfh(prefix + "03 00 00 00 E0 80 80")) + + # valid language values + for lang in wozardry.kLanguages: + wozardry.WozDiskImage(bfh(prefix + build_meta_chunk("language", lang))) + + # invalid language value + with pytest.raises(wozardry.WozMETAFormatError_BadLanguage): + wozardry.WozDiskImage(bfh(prefix + build_meta_chunk("language", "Englush"))) + + # valid requires_ram values + for ram in wozardry.kRequiresRAM: + wozardry.WozDiskImage(bfh(prefix + build_meta_chunk("requires_ram", ram))) + + # invalid requires_ram value + with pytest.raises(wozardry.WozMETAFormatError_BadRAM): + wozardry.WozDiskImage(bfh(prefix + build_meta_chunk("requires_ram", "0"))) + + # valid requires_machine values + for machine in wozardry.kRequiresMachine: + wozardry.WozDiskImage(bfh(prefix + build_meta_chunk("requires_machine", machine))) + + # invalid requires_machine value + with pytest.raises(wozardry.WozMETAFormatError_BadMachine): + wozardry.WozDiskImage(bfh(prefix + build_meta_chunk("requires_machine", "4"))) + + # invalid format (duplicate key) + bk = "language".encode("utf-8") + bv = "English".encode("utf-8") + chunk = bk + b"\x09" + bv + b"\x0A" + bk + b"\x09" + bv + b"\0x0A" + chunk = (wozardry.to_uint32(len(chunk)) + chunk).hex() + with pytest.raises(wozardry.WozMETAFormatError_DuplicateKey): + wozardry.WozDiskImage(bfh(prefix + chunk)) + + # invalid format (no tab separator between key an dvalue) + chunk = bk + bv + b"\x0A" + chunk = (wozardry.to_uint32(len(chunk)) + chunk).hex() + with pytest.raises(wozardry.WozMETAFormatError_NotEnoughTabs): + wozardry.WozDiskImage(bfh(prefix + chunk)) + + # invalid format (too many tabs between key and value) + chunk = bk + b"\x09"*2 + bv + b"\x0A" + chunk = (wozardry.to_uint32(len(chunk)) + chunk).hex() + with pytest.raises(wozardry.WozMETAFormatError_TooManyTabs): + wozardry.WozDiskImage(bfh(prefix + chunk)) #----- test command-line interface ----- @@ -291,7 +377,8 @@ def test_command_edit_info_version_1_to_2(): shutil.copy(kValid1, tmp.name) wozardry.parse_args(["edit", "-i", "version:2", tmp.name]) - woz = wozardry.WozReader(tmp.name) + with open(tmp.name, "rb") as tmpstream: + woz = wozardry.WozDiskImage(tmpstream) assert woz.woz_version == 2 assert woz.info["version"] == 2 assert woz.info["boot_sector_format"] == 0 @@ -308,12 +395,14 @@ def test_command_edit_info_disk_type(): # disk_type = 1 is ok wozardry.parse_args(["edit", "-i", "disk_type:1", tmp.name]) - woz = wozardry.WozReader(tmp.name) + with open(tmp.name, "rb") as tmpstream: + woz = wozardry.WozDiskImage(tmpstream) assert woz.info["disk_type"] == 1 # disk_type = 2 is ok wozardry.parse_args(["edit", "-i", "disk_type:2", tmp.name]) - woz = wozardry.WozReader(tmp.name) + with open(tmp.name, "rb") as tmpstream: + woz = wozardry.WozDiskImage(tmpstream) assert woz.info["disk_type"] == 2 # disk_type = 0 is not ok @@ -332,7 +421,8 @@ def test_command_edit_info_changing_disk_type_resets_optimal_bit_timing(): with tempfile.NamedTemporaryFile() as tmp: shutil.copy(kValid2, tmp.name) wozardry.parse_args(["edit", "-i", "disk_type:2", tmp.name]) - woz = wozardry.WozReader(tmp.name) + with open(tmp.name, "rb") as tmpstream: + woz = wozardry.WozDiskImage(tmpstream) assert woz.info["optimal_bit_timing"] == wozardry.kDefaultBitTiming[2] def test_command_edit_info_boolean_flags(): @@ -348,10 +438,12 @@ def test_command_edit_info_boolean_flags(): ("true", "false"), ("tRuE", "FaLsE")): wozardry.parse_args(["edit", "-i", "%s:%s" % (flag, true_value), tmp.name]) - woz = wozardry.WozReader(tmp.name) + with open(tmp.name, "rb") as tmpstream: + woz = wozardry.WozDiskImage(tmpstream) assert woz.info[flag] == True wozardry.parse_args(["edit", "-i", "%s:%s" % (flag, false_value), tmp.name]) - woz = wozardry.WozReader(tmp.name) + with open(tmp.name, "rb") as tmpstream: + woz = wozardry.WozDiskImage(tmpstream) assert woz.info[flag] == False f(kValid1) f(kValid2) @@ -362,7 +454,8 @@ def test_command_edit_disk_sides(): shutil.copy(kValid2, tmp.name) # this file is a 5.25-inch disk image - woz = wozardry.WozReader(tmp.name) + with open(tmp.name, "rb") as tmpstream: + woz = wozardry.WozDiskImage(tmpstream) assert woz.info["disk_type"] == 1 assert woz.info["disk_sides"] == 1 @@ -372,15 +465,18 @@ def test_command_edit_disk_sides(): # now change it to a 3.5-inch disk image wozardry.parse_args(["edit", "-i", "disk_type:2", tmp.name]) - woz = wozardry.WozReader(tmp.name) + with open(tmp.name, "rb") as tmpstream: + woz = wozardry.WozDiskImage(tmpstream) assert woz.info["disk_type"] == 2 # 3.5-inch disk images can be 1- or 2-sided wozardry.parse_args(["edit", "-i", "disk_sides:2", tmp.name]) - woz = wozardry.WozReader(tmp.name) + with open(tmp.name, "rb") as tmpstream: + woz = wozardry.WozDiskImage(tmpstream) assert woz.info["disk_sides"] == 2 wozardry.parse_args(["edit", "-i", "disk_sides:1", tmp.name]) - woz = wozardry.WozReader(tmp.name) + with open(tmp.name, "rb") as tmpstream: + woz = wozardry.WozDiskImage(tmpstream) assert woz.info["disk_sides"] == 1 # ...but not 3-sided, that's silly @@ -395,7 +491,8 @@ def test_command_edit_language(): for lang in wozardry.kLanguages: wozardry.parse_args(["edit", "-m", "language:%s" % lang, tmp.name]) - woz = wozardry.WozReader(tmp.name) + with open(tmp.name, "rb") as tmpstream: + woz = wozardry.WozDiskImage(tmpstream) assert woz.meta["language"] == lang f(kValid1) f(kValid2) @@ -408,7 +505,8 @@ def test_command_edit_requires_ram(): for ram in wozardry.kRequiresRAM: wozardry.parse_args(["edit", "-m", "requires_ram:%s" % ram, tmp.name]) - woz = wozardry.WozReader(tmp.name) + with open(tmp.name, "rb") as tmpstream: + woz = wozardry.WozDiskImage(tmpstream) assert woz.meta["requires_ram"] == ram # invalid required RAM (must be one of enumerated values) @@ -426,7 +524,8 @@ def test_command_edit_requires_machine(): for model in wozardry.kRequiresMachine: wozardry.parse_args(["edit", "-m", "requires_machine:%s" % model, tmp.name]) - woz = wozardry.WozReader(tmp.name) + with open(tmp.name, "rb") as tmpstream: + woz = wozardry.WozDiskImage(tmpstream) assert woz.meta["requires_machine"] == model # invalid machine (Apple IV) diff --git a/wozardry.py b/wozardry.py index bb6fea1..6469a12 100755 --- a/wozardry.py +++ b/wozardry.py @@ -7,29 +7,30 @@ import argparse import binascii import bitarray # https://pypi.org/project/bitarray/ import collections +import io import json import itertools import os import sys __version__ = "2.0-beta" # https://semver.org -__date__ = "2019-02-23" +__date__ = "2019-03-02" __progname__ = "wozardry" __displayname__ = __progname__ + " " + __version__ + " by 4am (" + __date__ + ")" -# domain-specific constants defined in .woz specification +# domain-specific constants defined in .woz specifications kWOZ1 = b"WOZ1" kWOZ2 = b"WOZ2" kINFO = b"INFO" kTMAP = b"TMAP" kTRKS = b"TRKS" -kWRIT = b"WRIT" +kWRIT = b"WRIT" # WOZ2 only kMETA = b"META" -kBitstreamLengthInBytes = 6646 +kBitstreamLengthInBytes = 6646 # WOZ1 only kLanguages = ("English","Spanish","French","German","Chinese","Japanese","Italian","Dutch","Portuguese","Danish","Finnish","Norwegian","Swedish","Russian","Polish","Turkish","Arabic","Thai","Czech","Hungarian","Catalan","Croatian","Greek","Hebrew","Romanian","Slovak","Ukrainian","Indonesian","Malay","Vietnamese","Other") kRequiresRAM = ("16K","24K","32K","48K","64K","128K","256K","512K","768K","1M","1.25M","1.5M+","Unknown") kRequiresMachine = ("2","2+","2e","2c","2e+","2gs","2c+","3","3+") -kDefaultBitTiming = (0, 32, 16) +kDefaultBitTiming = (0, 32, 16) # WOZ2 only # strings and things, for print routines and error messages sEOF = "Unexpected EOF" @@ -41,6 +42,7 @@ tDiskType = {(1,1,False): "5.25-inch (140K)", (2,2,False): "3.5-inch (800K)", (2,2,True): "3.5-inch (1.44MB)"} tBootSectorFormat = ("unknown", "16-sector", "13-sector", "hybrid 13- and 16-sector") +tDefaultCreator = (__progname__ + " " + __version__)[:32] # errors that may be raised class WozError(Exception): pass # base class @@ -67,7 +69,13 @@ class WozTMAPFormatError(WozFormatError): pass class WozTMAPFormatError_MissingTMAPChunk(WozTMAPFormatError): pass class WozTMAPFormatError_BadTRKS(WozTMAPFormatError): pass class WozTRKSFormatError(WozFormatError): pass +class WozTRKSFormatError_BadStartingBlock(WozTRKSFormatError): pass +class WozTRKSFormatError_BadBlockCount(WozTRKSFormatError): pass +class WozTRKSFormatError_BadBitCount(WozTRKSFormatError): pass class WozMETAFormatError(WozFormatError): pass +class WozMETAFormatError_EncodingError(WozFormatError): pass +class WozMETAFormatError_NotEnoughTabs(WozFormatError): pass +class WozMETAFormatError_TooManyTabs(WozFormatError): pass class WozMETAFormatError_DuplicateKey(WozFormatError): pass class WozMETAFormatError_BadValue(WozFormatError): pass class WozMETAFormatError_BadLanguage(WozFormatError): pass @@ -177,82 +185,186 @@ class Track: if tuple(seen) == tuple(sequence): return True return False -class WozTrack(Track): - def __init__(self, bits, bit_count, splice_point = 0xFFFF, splice_nibble = 0, splice_bit_count = 0): - Track.__init__(self, bits, bit_count) - self.splice_point = splice_point - self.splice_nibble = splice_nibble - self.splice_bit_count = splice_bit_count - class WozDiskImage: - def __init__(self): - self.woz_version = None + def __init__(self, iostream=None): + if iostream: + self.load(iostream) + else: + self.reset() + + def reset(self): + self.info = collections.OrderedDict() self.tmap = [0xFF]*160 self.tracks = [] self.writ = None - self.info = collections.OrderedDict() self.meta = collections.OrderedDict() + self.woz_version = 2 + self.info["version"] = self.woz_version + self.info["disk_type"] = 1 + self.info["write_protected"] = False + self.info["synchronized"] = False + self.info["cleaned"] = False + self.info["creator"] = tDefaultCreator + self.info["disk_sides"] = 1 + self.info["boot_sector_format"] = 0 + self.info["optimal_bit_timing"] = 32 + self.info["compatible_hardware"] = [] + self.info["required_ram"] = 0 - def track_num_to_half_phase(self, track_num): - if type(track_num) != float: - track_num = float(track_num) - if track_num < 0.0 or \ - track_num > 40.0 or \ - track_num.as_integer_ratio()[1] not in (1,2,4): - raise WozError("Invalid track %s" % track_num) - return int(track_num * 4) + def load(self, iostream): + self.reset() + seen_info = False + seen_tmap = False + header_raw = iostream.read(8) + raise_if(len(header_raw) != 8, WozEOFError, sEOF) + self._load_header(header_raw) + crc_raw = iostream.read(4) + raise_if(len(crc_raw) != 4, WozEOFError, sEOF) + crc = from_uint32(crc_raw) + all_data = [] + while True: + chunk_id = iostream.read(4) + if not chunk_id: break + raise_if(len(chunk_id) != 4, WozEOFError, sEOF) + all_data.append(chunk_id) + chunk_size_raw = iostream.read(4) + raise_if(len(chunk_size_raw) != 4, WozEOFError, sEOF) + all_data.append(chunk_size_raw) + chunk_size = from_uint32(chunk_size_raw) + data = iostream.read(chunk_size) + raise_if(len(data) != chunk_size, WozEOFError, sEOF) + all_data.append(data) + if chunk_id == kINFO: + raise_if(chunk_size != 60, WozINFOFormatError, sBadChunkSize) + self._load_info(data) + seen_info = True + continue + raise_if(not seen_info, WozINFOFormatError_MissingINFOChunk, "Expected INFO chunk at offset 20") + if chunk_id == kTMAP: + raise_if(chunk_size != 160, WozTMAPFormatError, sBadChunkSize) + self._load_tmap(data) + seen_tmap = True + continue + raise_if(not seen_tmap, WozTMAPFormatError_MissingTMAPChunk, "Expected TMAP chunk at offset 88") + if chunk_id == kTRKS: + self._load_trks(data) + elif chunk_id == kWRIT: + self._load_writ(data) + elif chunk_id == kMETA: + self._load_meta(data) + raise_if(not seen_info, WozINFOFormatError_MissingINFOChunk, "Expected INFO chunk at offset 20") + raise_if(not seen_tmap, WozTMAPFormatError_MissingTMAPChunk, "Expected TMAP chunk at offset 88") + if crc: + raise_if(crc != binascii.crc32(b"".join(all_data)) & 0xffffffff, WozCRCError, "Bad CRC") - def seek(self, track_num): - """returns Track object for the given track, or None if the track is not part of this disk image. track_num can be 0..40 in 0.25 increments (0, 0.25, 0.5, 0.75, 1, &c.)""" - half_phase = self.track_num_to_half_phase(track_num) - trk_id = self.tmap[half_phase] - if trk_id == 0xFF: return None - return self.tracks[trk_id] + def _load_header(self, data): + raise_if(data[:4] not in (kWOZ1, kWOZ2), WozHeaderError_NoWOZMarker, "Magic string 'WOZ1' or 'WOZ2' not present at offset 0") + self.woz_version = int(data[3]) - 0x30 + raise_if(data[4] != 0xFF, WozHeaderError_NoFF, "Magic byte 0xFF not present at offset 4") + raise_if(data[5:8] != b"\x0A\x0D\x0A", WozHeaderError_NoLF, "Magic bytes 0x0A0D0A not present at offset 5") - def clean(self): - """removes tracks from self.tracks that are not referenced from self.tmap, and adjusts remaining self.tmap indices""" + def _load_info(self, data): + self.info["version"] = self.validate_info_version(data[0]) # int + self.info["disk_type"] = self.validate_info_disk_type(data[1]) # int + self.info["write_protected"] = self.validate_info_write_protected(data[2]) # boolean + self.info["synchronized"] = self.validate_info_synchronized(data[3]) # boolean + self.info["cleaned"] = self.validate_info_cleaned(data[4]) # boolean + self.info["creator"] = self.validate_info_creator(data[5:37]) # string + if self.info["version"] >= 2: + self.info["disk_sides"] = self.validate_info_disk_sides(data[37]) # int + self.info["boot_sector_format"] = self.validate_info_boot_sector_format(data[38]) # int + self.info["optimal_bit_timing"] = self.validate_info_optimal_bit_timing(data[39]) # int + compatible_hardware_bitfield = self.validate_info_compatible_hardware(data[40:42]) # int + compatible_hardware_list = [] + for offset in range(9): + if compatible_hardware_bitfield & (1 << offset): + compatible_hardware_list.append(kRequiresMachine[offset]) + self.info["compatible_hardware"] = compatible_hardware_list + self.info["required_ram"] = self.validate_info_required_ram(data[42:44]) + self.info["largest_track"] = from_uint16(data[44:46]) + + def _load_tmap(self, data): + self.tmap = list(data) + + def _load_trks(self, data): + if self.info["version"] == 1: + self._load_trks_v1(data) + else: + self._load_trks_v2(data) + for trk, i in zip(self.tmap, itertools.count()): + raise_if(trk != 0xFF and trk >= len(self.tracks), WozTMAPFormatError_BadTRKS, "Invalid TMAP entry: track %d%s points to non-existent TRKS chunk %d" % (i/4, tQuarters[i%4], trk)) + + def _load_trks_v1(self, data): i = 0 - while i < len(self.tracks): - if i not in self.tmap: - del self.tracks[i] - for adjust in range(len(self.tmap)): - if (self.tmap[adjust] >= i) and (self.tmap[adjust] != 0xFF): - self.tmap[adjust] -= 1 - else: - i += 1 + while i < len(data): + raw_bytes = data[i:i+kBitstreamLengthInBytes] + raise_if(len(raw_bytes) != kBitstreamLengthInBytes, WozEOFError, sEOF) + i += kBitstreamLengthInBytes + bytes_used_raw = data[i:i+2] + raise_if(len(bytes_used_raw) != 2, WozEOFError, sEOF) + bytes_used = from_uint16(bytes_used_raw) + raise_if(bytes_used > kBitstreamLengthInBytes, WozTRKSFormatError, "TRKS chunk %d bytes_used is out of range" % len(self.tracks)) + i += 2 + bit_count_raw = data[i:i+2] + raise_if(len(bit_count_raw) != 2, WozEOFError, sEOF) + bit_count = from_uint16(bit_count_raw) + i += 2 + splice_point_raw = data[i:i+2] + raise_if(len(splice_point_raw) != 2, WozEOFError, sEOF) + splice_point = from_uint16(splice_point_raw) + if splice_point != 0xFFFF: + raise_if(splice_point > bit_count, WozTRKSFormatError, "TRKS chunk %d splice_point is out of range" % len(self.tracks)) + i += 2 + splice_nibble = data[i] + i += 1 + splice_bit_count = data[i] + if splice_point != 0xFFFF: + raise_if(splice_bit_count not in (8,9,10), WozTRKSFormatError, "TRKS chunk %d splice_bit_count is out of range" % len(self.tracks)) + i += 3 + bits = bitarray.bitarray(endian="big") + bits.frombytes(raw_bytes) + self.tracks.append(Track(bits, bit_count)) - def add(self, half_phase, track): - trk_id = len(self.tracks) - self.tracks.append(track) - self.tmap[half_phase] = trk_id - if half_phase: - self.tmap[half_phase - 1] = trk_id - if half_phase < 159: - self.tmap[half_phase + 1] = trk_id + def _load_trks_v2(self, data): + for trk in range(160): + i = trk * 8 + starting_block = from_uint16(data[i:i+2]) + raise_if(starting_block in (1,2), WozTRKSFormatError_BadStartingBlock, "TRKS TRK %d starting_block out of range (expected 3+ or 0, found %s)" % (trk, starting_block)) + block_count = from_uint16(data[i+2:i+4]) + bit_count = from_uint32(data[i+4:i+8]) + if starting_block == 0: + raise_if(block_count != 0, WozTRKSFormatError_BadBlockCount, "TRKS unused TRK %d block_count must be 0 (found %s)" % (trk, block_count)) + raise_if(bit_count != 0, WozTRKSFormatError_BadBitCount, "TRKS unused TRK %d bit_count must be 0 (found %s)" % (trk, bit_count)) + break + bits_index_into_data = 1280 + (starting_block-3)*512 + raise_if(len(data) <= bits_index_into_data, WozTRKSFormatError_BadStartingBlock, sEOF) + raw_bytes = data[bits_index_into_data : bits_index_into_data + block_count*512] + raise_if(len(raw_bytes) != block_count*512, WozTRKSFormatError_BadBlockCount, sEOF) + bits = bitarray.bitarray(endian="big") + bits.frombytes(raw_bytes) + self.tracks.append(Track(bits, bit_count)) - def add_track(self, track_num, track): - self.add(self.track_num_to_half_phase(track_num), track) + def _load_writ(self, data): + self.writ = data - def remove(self, half_phase): - if self.tmap[half_phase] == 0xFF: return False - self.tmap[half_phase] = 0xFF - self.clean() - return True + def _load_meta(self, metadata_as_bytes): + metadata = self.decode_metadata(metadata_as_bytes) + for line in metadata.split("\n"): + if not line: continue + columns_raw = line.split("\t") + raise_if(len(columns_raw) < 2, WozMETAFormatError_NotEnoughTabs, "Malformed metadata") + raise_if(len(columns_raw) > 2, WozMETAFormatError_TooManyTabs, "Malformed metadata") + key, value_raw = columns_raw + raise_if(key in self.meta, WozMETAFormatError_DuplicateKey, "Duplicate metadata key %s" % key) + values = value_raw.split("|") + if key == "language": + list(map(self.validate_metadata_language, values)) + elif key == "requires_ram": + list(map(self.validate_metadata_requires_ram, values)) + elif key == "requires_machine": + list(map(self.validate_metadata_requires_machine, values)) + self.meta[key] = len(values) == 1 and values[0] or tuple(values) - def remove_track(self, track_num): - """removes given track, returns True if anything was actually removed, or False if track wasn't found. track_num can be 0..40 in 0.25 increments (0, 0.25, 0.5, 0.75, 1, &c.)""" - return self.remove(self.track_num_to_half_phase(track_num)) - - def from_json(self, json_string): - j = json.loads(json_string) - root = [x for x in j.keys()].pop() - self.meta.update(j[root]["meta"]) - - def to_json(self): - j = {"woz": {"info":self.info, "meta":self.meta}} - return json.dumps(j, indent=2) - -class WozValidator: def validate_info_version(self, version): """ |version| can be str, bytes, or int. returns same value as int""" version = from_intish(version, WozINFOFormatError_BadVersion, "Unknown version (expected numeric value, found %s)") @@ -339,7 +451,7 @@ class WozValidator: try: metadata = metadata_as_bytes.decode("UTF-8") except: - raise WozMETAFormatError("Metadata is not valid UTF-8") + raise WozMETAFormatError_EncodingError("Metadata is not valid UTF-8") def decode_metadata(self, metadata_as_bytes): self.validate_metadata(metadata_as_bytes) @@ -359,179 +471,21 @@ class WozValidator: def validate_metadata_requires_machine(self, requires_machine): raise_if(requires_machine and (requires_machine not in kRequiresMachine), WozMETAFormatError_BadMachine, "Invalid metadata requires_machine") -class WozReader(WozDiskImage, WozValidator): - def __init__(self, filename=None, stream=None): - WozDiskImage.__init__(self) - self.filename = filename - seen_info = False - seen_tmap = False - with stream or open(filename, "rb") as f: - header_raw = f.read(8) - raise_if(len(header_raw) != 8, WozEOFError, sEOF) - self.__process_header(header_raw) - crc_raw = f.read(4) - raise_if(len(crc_raw) != 4, WozEOFError, sEOF) - crc = from_uint32(crc_raw) - all_data = [] - while True: - chunk_id = f.read(4) - if not chunk_id: break - raise_if(len(chunk_id) != 4, WozEOFError, sEOF) - all_data.append(chunk_id) - chunk_size_raw = f.read(4) - raise_if(len(chunk_size_raw) != 4, WozEOFError, sEOF) - all_data.append(chunk_size_raw) - chunk_size = from_uint32(chunk_size_raw) - data = f.read(chunk_size) - raise_if(len(data) != chunk_size, WozEOFError, sEOF) - all_data.append(data) - if chunk_id == kINFO: - raise_if(chunk_size != 60, WozINFOFormatError, sBadChunkSize) - self.__process_info(data) - seen_info = True - continue - raise_if(not seen_info, WozINFOFormatError_MissingINFOChunk, "Expected INFO chunk at offset 20") - if chunk_id == kTMAP: - raise_if(chunk_size != 160, WozTMAPFormatError, sBadChunkSize) - self.__process_tmap(data) - seen_tmap = True - continue - raise_if(not seen_tmap, WozTMAPFormatError_MissingTMAPChunk, "Expected TMAP chunk at offset 88") - if chunk_id == kTRKS: - self.__process_trks(data) - elif chunk_id == kWRIT: - self.__process_writ(data) - elif chunk_id == kMETA: - self.__process_meta(data) - raise_if(not seen_info, WozINFOFormatError_MissingINFOChunk, "Expected INFO chunk at offset 20") - raise_if(not seen_tmap, WozTMAPFormatError_MissingTMAPChunk, "Expected TMAP chunk at offset 88") - if crc: - raise_if(crc != binascii.crc32(b"".join(all_data)) & 0xffffffff, WozCRCError, "Bad CRC") + def __bytes__(self): + return self.dump() - def __process_header(self, data): - raise_if(data[:4] not in (kWOZ1, kWOZ2), WozHeaderError_NoWOZMarker, "Magic string 'WOZ1' or 'WOZ2' not present at offset 0") - self.woz_version = int(data[3]) - 0x30 - raise_if(data[4] != 0xFF, WozHeaderError_NoFF, "Magic byte 0xFF not present at offset 4") - raise_if(data[5:8] != b"\x0A\x0D\x0A", WozHeaderError_NoLF, "Magic bytes 0x0A0D0A not present at offset 5") + def dump(self): + """returns serialization of the disk image in bytes, suitable for writing to disk""" + info = self._dump_info() + tmap = self._dump_tmap() + trks = self._dump_trks() + writ = self._dump_writ() # will be zero-length if no WRIT chunk + meta = self._dump_meta() # will be zero-length if no META chunk + crc = binascii.crc32(info + tmap + trks + writ + meta) + head = self._dump_head(crc) + return bytes(head + info + tmap + trks + writ + meta) - def __process_info(self, data): - self.info["version"] = self.validate_info_version(data[0]) # int - self.info["disk_type"] = self.validate_info_disk_type(data[1]) # int - self.info["write_protected"] = self.validate_info_write_protected(data[2]) # boolean - self.info["synchronized"] = self.validate_info_synchronized(data[3]) # boolean - self.info["cleaned"] = self.validate_info_cleaned(data[4]) # boolean - self.info["creator"] = self.validate_info_creator(data[5:37]) # string - if self.info["version"] >= 2: - self.info["disk_sides"] = self.validate_info_disk_sides(data[37]) # int - self.info["boot_sector_format"] = self.validate_info_boot_sector_format(data[38]) # int - self.info["optimal_bit_timing"] = self.validate_info_optimal_bit_timing(data[39]) # int - compatible_hardware_bitfield = self.validate_info_compatible_hardware(data[40:42]) # int - compatible_hardware_list = [] - for offset in range(9): - if compatible_hardware_bitfield & (1 << offset): - compatible_hardware_list.append(kRequiresMachine[offset]) - self.info["compatible_hardware"] = compatible_hardware_list - self.info["required_ram"] = self.validate_info_required_ram(data[42:44]) - self.info["largest_track"] = from_uint16(data[44:46]) - - def __process_tmap(self, data): - self.tmap = list(data) - - def __process_trks(self, data): - if self.info["version"] == 1: - self.__process_trks_v1(data) - else: - self.__process_trks_v2(data) - for trk, i in zip(self.tmap, itertools.count()): - raise_if(trk != 0xFF and trk >= len(self.tracks), WozTMAPFormatError_BadTRKS, "Invalid TMAP entry: track %d%s points to non-existent TRKS chunk %d" % (i/4, tQuarters[i%4], trk)) - - def __process_trks_v1(self, data): - i = 0 - while i < len(data): - raw_bytes = data[i:i+kBitstreamLengthInBytes] - raise_if(len(raw_bytes) != kBitstreamLengthInBytes, WozEOFError, sEOF) - i += kBitstreamLengthInBytes - bytes_used_raw = data[i:i+2] - raise_if(len(bytes_used_raw) != 2, WozEOFError, sEOF) - bytes_used = from_uint16(bytes_used_raw) - raise_if(bytes_used > kBitstreamLengthInBytes, WozTRKSFormatError, "TRKS chunk %d bytes_used is out of range" % len(self.tracks)) - i += 2 - bit_count_raw = data[i:i+2] - raise_if(len(bit_count_raw) != 2, WozEOFError, sEOF) - bit_count = from_uint16(bit_count_raw) - i += 2 - splice_point_raw = data[i:i+2] - raise_if(len(splice_point_raw) != 2, WozEOFError, sEOF) - splice_point = from_uint16(splice_point_raw) - if splice_point != 0xFFFF: - raise_if(splice_point > bit_count, WozTRKSFormatError, "TRKS chunk %d splice_point is out of range" % len(self.tracks)) - i += 2 - splice_nibble = data[i] - i += 1 - splice_bit_count = data[i] - if splice_point != 0xFFFF: - raise_if(splice_bit_count not in (8,9,10), WozTRKSFormatError, "TRKS chunk %d splice_bit_count is out of range" % len(self.tracks)) - i += 3 - bits = bitarray.bitarray(endian="big") - bits.frombytes(raw_bytes) - self.tracks.append(WozTrack(bits, bit_count, splice_point, splice_nibble, splice_bit_count)) - - def __process_trks_v2(self, data): - for trk in range(160): - i = trk * 8 - starting_block = from_uint16(data[i:i+2]) - raise_if(starting_block in (1,2), WozTRKSFormatError, "TRKS TRK %d starting_block out of range (expected 3+ or 0, found %s)" % (trk, starting_block)) - block_count = from_uint16(data[i+2:i+4]) - bit_count = from_uint32(data[i+4:i+8]) - if starting_block == 0: - raise_if(block_count != 0, WozTRKSFormatError, "TRKS unused TRK %d block_count must be 0 (found %s)" % (trk, block_count)) - raise_if(bit_count != 0, WozTRKSFormatError, "TRKS unused TRK %d bit_count must be 0 (found %s)" % (trk, bit_count)) - break - bits_index_into_data = 1280 + (starting_block-3)*512 - raw_bytes = data[bits_index_into_data : bits_index_into_data + block_count*512] - bits = bitarray.bitarray(endian="big") - bits.frombytes(raw_bytes) - self.tracks.append(WozTrack(bits, bit_count)) - - def __process_writ(self, data): - self.writ = data - - def __process_meta(self, metadata_as_bytes): - metadata = self.decode_metadata(metadata_as_bytes) - for line in metadata.split("\n"): - if not line: continue - columns_raw = line.split("\t") - raise_if(len(columns_raw) != 2, WozMETAFormatError, "Malformed metadata") - key, value_raw = columns_raw - raise_if(key in self.meta, WozMETAFormatError_DuplicateKey, "Duplicate metadata key %s" % key) - values = value_raw.split("|") - if key == "language": - list(map(self.validate_metadata_language, values)) - elif key == "requires_ram": - list(map(self.validate_metadata_requires_ram, values)) - elif key == "requires_machine": - list(map(self.validate_metadata_requires_machine, values)) - self.meta[key] = len(values) == 1 and values[0] or tuple(values) - -class WozWriter(WozDiskImage, WozValidator): - def __init__(self, woz_version, creator): - WozDiskImage.__init__(self) - self.woz_version = woz_version - self.info["version"] = woz_version - self.validate_info_version(woz_version) - self.info["disk_type"] = 1 - self.info["write_protected"] = False - self.info["synchronized"] = False - self.info["cleaned"] = False - self.info["creator"] = creator - self.encode_info_creator(creator) - self.info["disk_sides"] = 1 - self.info["boot_sector_format"] = 0 - self.info["optimal_bit_timing"] = 32 - self.info["compatible_hardware"] = [] - self.info["required_ram"] = 0 - - def build_info(self): + def _dump_info(self): chunk = bytearray() chunk.extend(kINFO) # chunk ID chunk.extend(to_uint32(60)) # chunk size (constant) @@ -582,20 +536,20 @@ class WozWriter(WozDiskImage, WozValidator): chunk.extend(b"\x00" * 14) # 14 bytes of unused space return chunk - def build_tmap(self): + def _dump_tmap(self): chunk = bytearray() chunk.extend(kTMAP) # chunk ID chunk.extend(to_uint32(160)) # chunk size chunk.extend(bytes(self.tmap)) return chunk - def build_trks(self): + def _dump_trks(self): if self.woz_version == 1: - return self.build_trks_v1() + return self._dump_trks_v1() else: - return self.build_trks_v2() + return self._dump_trks_v2() - def build_trks_v1(self): + def _dump_trks_v1(self): chunk = bytearray() chunk.extend(kTRKS) # chunk ID chunk_size = len(self.tracks)*6656 @@ -612,7 +566,7 @@ class WozWriter(WozDiskImage, WozValidator): chunk.extend(b"\x00\x00") # reserved return chunk - def build_trks_v2(self): + def _dump_trks_v2(self): starting_block = 3 trk_chunk = bytearray() bits_chunk = bytearray() @@ -637,7 +591,7 @@ class WozWriter(WozDiskImage, WozValidator): chunk.extend(bits_chunk) return chunk - def build_writ(self): + def _dump_writ(self): chunk = bytearray() if self.writ: chunk.extend(kWRIT) # chunk ID @@ -645,7 +599,7 @@ class WozWriter(WozDiskImage, WozValidator): chunk.extend(self.writ) return chunk - def build_meta(self): + def _dump_meta(self): if not self.meta: return b"" meta_tmp = {} for key, value_raw in self.meta.items(): @@ -672,7 +626,7 @@ class WozWriter(WozDiskImage, WozValidator): chunk.extend(data) return chunk - def build_head(self, crc): + def _dump_head(self, crc): chunk = bytearray() if self.woz_version == 1: chunk.extend(kWOZ1) # magic bytes @@ -682,24 +636,68 @@ class WozWriter(WozDiskImage, WozValidator): chunk.extend(to_uint32(crc)) # CRC32 of rest of file (calculated in caller) return chunk - def write(self, stream): - info = self.build_info() - tmap = self.build_tmap() - trks = self.build_trks() - writ = self.build_writ() # will be zero-length if no WRIT chunk - meta = self.build_meta() # will be zero-length if no META chunk - crc = binascii.crc32(info + tmap + trks + writ + meta) - head = self.build_head(crc) - stream.write(head) - stream.write(info) - stream.write(tmap) - stream.write(trks) - stream.write(writ) - stream.write(meta) + def track_num_to_half_phase(self, track_num): + if type(track_num) != float: + track_num = float(track_num) + if track_num < 0.0 or \ + track_num > 40.0 or \ + track_num.as_integer_ratio()[1] not in (1,2,4): + raise WozError("Invalid track %s" % track_num) + return int(track_num * 4) + + def add(self, half_phase, track): + trk_id = len(self.tracks) + self.tracks.append(track) + self.tmap[half_phase] = trk_id + if half_phase: + self.tmap[half_phase - 1] = trk_id + if half_phase < 159: + self.tmap[half_phase + 1] = trk_id + + def add_track(self, track_num, track): + self.add(self.track_num_to_half_phase(track_num), track) + + def remove(self, half_phase): + if self.tmap[half_phase] == 0xFF: return False + self.tmap[half_phase] = 0xFF + self.clean() + return True + + def remove_track(self, track_num): + """removes given track, returns True if anything was actually removed, or False if track wasn't found. track_num can be 0..40 in 0.25 increments (0, 0.25, 0.5, 0.75, 1, &c.)""" + return self.remove(self.track_num_to_half_phase(track_num)) + + def clean(self): + """removes tracks from self.tracks that are not referenced from self.tmap, and adjusts remaining self.tmap indices""" + i = 0 + while i < len(self.tracks): + if i not in self.tmap: + del self.tracks[i] + for adjust in range(len(self.tmap)): + if (self.tmap[adjust] >= i) and (self.tmap[adjust] != 0xFF): + self.tmap[adjust] -= 1 + else: + i += 1 + + def seek(self, track_num): + """returns Track object for the given track, or None if the track is not part of this disk image. track_num can be 0..40 in 0.25 increments (0, 0.25, 0.5, 0.75, 1, &c.)""" + half_phase = self.track_num_to_half_phase(track_num) + trk_id = self.tmap[half_phase] + if trk_id == 0xFF: return None + return self.tracks[trk_id] + + def from_json(self, json_string): + j = json.loads(json_string) + root = [x for x in j.keys()].pop() + self.meta.update(j[root]["meta"]) + + def to_json(self): + j = {"woz": {"info":self.info, "meta":self.meta}} + return json.dumps(j, indent=2) #---------- command line interface ---------- -class BaseCommand: +class _BaseCommand: def __init__(self, name): self.name = name @@ -709,28 +707,29 @@ class BaseCommand: self.parser.set_defaults(action=self) def __call__(self, args): - self.woz_image = WozReader(args.file) + with open(args.file, "rb") as f: + self.woz_image = WozDiskImage(f) -class CommandVerify(BaseCommand): +class _CommandVerify(_BaseCommand): def __init__(self): - BaseCommand.__init__(self, "verify") + _BaseCommand.__init__(self, "verify") def setup(self, subparser): - BaseCommand.setup(self, subparser, + _BaseCommand.setup(self, subparser, description="Verify file structure and metadata of a .woz disk image (produces no output unless a problem is found)") -class CommandDump(BaseCommand): +class _CommandDump(_BaseCommand): kWidth = 30 def __init__(self): - BaseCommand.__init__(self, "dump") + _BaseCommand.__init__(self, "dump") def setup(self, subparser): - BaseCommand.setup(self, subparser, + _BaseCommand.setup(self, subparser, description="Print all available information and metadata in a .woz disk image") def __call__(self, args): - BaseCommand.__call__(self, args) + _BaseCommand.__call__(self, args) self.print_tmap() self.print_meta() self.print_info() @@ -800,34 +799,24 @@ class CommandDump(BaseCommand): for value in values[1:]: print("META: ".ljust(self.kWidth), value) -class CommandExport(BaseCommand): +class _CommandExport(_BaseCommand): def __init__(self): - BaseCommand.__init__(self, "export") + _BaseCommand.__init__(self, "export") def setup(self, subparser): - BaseCommand.setup(self, subparser, + _BaseCommand.setup(self, subparser, description="Export (as JSON) all information and metadata from a .woz disk image") def __call__(self, args): - BaseCommand.__call__(self, args) + _BaseCommand.__call__(self, args) print(self.woz_image.to_json()) -class WriterBaseCommand(BaseCommand): +class _WriterBaseCommand(_BaseCommand): def __call__(self, args): - BaseCommand.__call__(self, args) - self.args = args - self.output = WozWriter(self.woz_image.info.get("version", 2), - self.woz_image.info.get("creator", __displayname__)) - self.output.tmap = self.woz_image.tmap - self.output.tracks = self.woz_image.tracks - self.output.info.update(self.woz_image.info) - self.output.writ = self.woz_image.writ - self.output.meta = self.woz_image.meta.copy() - self.update() - tmpfile = args.file + ".ardry" - with open(tmpfile, "wb") as f: - self.output.write(f) - # as a final sanity check, load and parse the temporary file we just created + _BaseCommand.__call__(self, args) + self.update(args) + output_as_bytes = bytes(self.woz_image) + # as a final sanity check, load and parse the output we just created # to help ensure we never create invalid .woz files try: global raise_if @@ -835,19 +824,21 @@ class WriterBaseCommand(BaseCommand): except: pass try: - WozReader(tmpfile) + WozDiskImage(io.BytesIO(output_as_bytes)) except Exception as e: sys.stderr.write("WozInternalError: refusing to write an invalid .woz file (this is the developer's fault)\n") - os.remove(tmpfile) raise Exception from e + tmpfile = args.file + ".ardry" + with open(tmpfile, "wb") as tmp: + tmp.write(output_as_bytes) os.rename(tmpfile, args.file) -class CommandEdit(WriterBaseCommand): +class _CommandEdit(_WriterBaseCommand): def __init__(self): - WriterBaseCommand.__init__(self, "edit") + _WriterBaseCommand.__init__(self, "edit") def setup(self, subparser): - WriterBaseCommand.setup(self, + _WriterBaseCommand.setup(self, subparser, description="Edit information and metadata in a .woz disk image", epilog="""Tips: @@ -871,76 +862,76 @@ META format is "key:value". Standard keys are title, subtitle, publisher, developer, copyright, version, language, requires_ram, requires_machine, notes, side, side_name, contributor, image_date. Other keys are allowed.""") - def update(self): + def update(self, args): # 1st update version info field - for i in self.args.info or (): + for i in args.info or (): k, v = i.split(":", 1) if k == "version": v = from_intish(v, WozINFOFormatError_BadVersion, "Unknown version (expected numeric value, found %s)") raise_if(v not in (1,2), WozINFOFormatError_BadVersion, "Unknown version (expected 1 or 2, found %s) % v") - self.output.woz_version = v - self.output.info["version"] = v + self.woz_image.woz_version = v + self.woz_image.info["version"] = v # 2nd update disk_type info field - for i in self.args.info or (): + for i in args.info or (): k, v = i.split(":", 1) if k == "disk_type": - old_disk_type = self.output.info["disk_type"] - new_disk_type = self.output.validate_info_disk_type(v) + old_disk_type = self.woz_image.info["disk_type"] + new_disk_type = self.woz_image.validate_info_disk_type(v) if old_disk_type != new_disk_type: - self.output.info["disk_type"] = new_disk_type - self.output.info["optimal_bit_timing"] = kDefaultBitTiming[new_disk_type] + self.woz_image.info["disk_type"] = new_disk_type + self.woz_image.info["optimal_bit_timing"] = kDefaultBitTiming[new_disk_type] # then update all other info fields - for i in self.args.info or (): + for i in args.info or (): k, v = i.split(":", 1) if k == "version": continue if k == "disk_type": continue if k == "write_protected": - self.output.info[k] = self.output.validate_info_write_protected(v) + self.woz_image.info[k] = self.woz_image.validate_info_write_protected(v) elif k == "synchronized": - self.output.info[k] = self.output.validate_info_synchronized(v) + self.woz_image.info[k] = self.woz_image.validate_info_synchronized(v) elif k == "cleaned": - self.output.info[k] = self.output.validate_info_cleaned(v) + self.woz_image.info[k] = self.woz_image.validate_info_cleaned(v) elif k == "creator": - self.output.info[k] = self.output.validate_info_creator(self.output.encode_info_creator(v)) - if self.output.info["version"] == 1: continue + self.woz_image.info[k] = self.woz_image.validate_info_creator(self.woz_image.encode_info_creator(v)) + if self.woz_image.info["version"] == 1: continue # remaining fields are only recognized in WOZ2 files (v2+ INFO chunk) if k == "disk_sides": - self.output.info[k] = self.output.validate_info_disk_sides(v) + self.woz_image.info[k] = self.woz_image.validate_info_disk_sides(v) elif k == "boot_sector_format": - self.output.info[k] = self.output.validate_info_boot_sector_format(v) + self.woz_image.info[k] = self.woz_image.validate_info_boot_sector_format(v) elif k == "optimal_bit_timing": - self.output.info[k] = self.output.validate_info_optimal_bit_timing(v) + self.woz_image.info[k] = self.woz_image.validate_info_optimal_bit_timing(v) elif k == "required_ram": if v.lower().endswith("k"): # forgive user for typing "128K" instead of "128" v = v[:-1] - self.output.info[k] = self.output.validate_info_required_ram(v) + self.woz_image.info[k] = self.woz_image.validate_info_required_ram(v) elif k == "compatible_hardware": machines = v.split("|") for machine in machines: - self.output.validate_metadata_requires_machine(machine) - self.output.info[k] = machines + self.woz_image.validate_metadata_requires_machine(machine) + self.woz_image.info[k] = machines # add all new metadata fields, and delete empty ones - for m in self.args.meta or (): + for m in args.meta or (): k, v = m.split(":", 1) v = v.split("|") if len(v) == 1: v = v[0] if v: - self.output.meta[k] = v - elif k in self.output.meta.keys(): - del self.output.meta[k] + self.woz_image.meta[k] = v + elif k in self.woz_image.meta.keys(): + del self.woz_image.meta[k] -class CommandRemove(WriterBaseCommand): +class _CommandRemove(_WriterBaseCommand): def __init__(self): - WriterBaseCommand.__init__(self, "remove") + _WriterBaseCommand.__init__(self, "remove") def setup(self, subparser): - WriterBaseCommand.setup(self, + _WriterBaseCommand.setup(self, subparser, description="Remove tracks from a 5.25-inch .woz disk image", epilog="""Tips: @@ -953,24 +944,24 @@ class CommandRemove(WriterBaseCommand): self.parser.add_argument("-t", "--track", type=str, action="append", help="""track to remove""") - def update(self): - raise_if(self.output.info["disk_type"] != 1, WozINFOFormatError_BadDiskType, "Can not remove tracks from 3.5-inch disks") - for i in self.args.track or (): - self.output.remove_track(float(i)) + def update(self, args): + raise_if(self.woz_image.info["disk_type"] != 1, WozINFOFormatError_BadDiskType, "Can not remove tracks from 3.5-inch disks") + for i in args.track or (): + self.woz_image.remove_track(float(i)) -class CommandImport(WriterBaseCommand): +class _CommandImport(_WriterBaseCommand): def __init__(self): - WriterBaseCommand.__init__(self, "import") + _WriterBaseCommand.__init__(self, "import") def setup(self, subparser): - WriterBaseCommand.setup(self, subparser, + _WriterBaseCommand.setup(self, subparser, description="Import JSON file to update metadata in a .woz disk image") - def update(self): - self.output.from_json(sys.stdin.read()) + def update(self, args): + self.woz_image.from_json(sys.stdin.read()) def parse_args(args): - cmds = [CommandDump(), CommandVerify(), CommandEdit(), CommandRemove(), CommandExport(), CommandImport()] + cmds = [_CommandDump(), _CommandVerify(), _CommandEdit(), _CommandRemove(), _CommandExport(), _CommandImport()] parser = argparse.ArgumentParser(prog=__progname__, description="""A multi-purpose tool for manipulating .woz disk images.