2023-09-24 18:56:36 +00:00
# Multipurpose Chunked File
## Goals
- meant for the Commander X16 system, depends on working MACPTR kernal call for fast loading.
- single file, possibly megabytes in size, that contains various kinds of data to be loaded or streamed into different locations.
- simple file format with few special cases.
- no random access, only sequential reading/streaming.
- simple user code that doesn't have to bother with the I/O at all.
2023-11-14 17:23:37 +00:00
- a few chunk types that can be handled automatically to load data into system ram, banked ram, or video ram.
2023-09-24 18:56:36 +00:00
- custom chunk types for flexibility or extensibility.
Theoretical optimal chunk size is 512 bytes but actual size may be different. (In practice there seems to be no significant speed impact)
MCF files are meant to be be created using a tool on PC, and only being read on the X16.
A Python tool is provided to create a demo MCF file.
A proof of concept Prog8 library module and example program is provided to consume that demo MCF file on the X16.
## File Format
This is the MCF file format:
| Chunk | size (bytes) |
|------------|--------------|
| LoadList | 256 |
| Data | 1 - 65535 |
| Data | 1 - 65535 |
| ... | |
| LoadList | 256 |
| Data | 1 - 65535 |
| Data | 1 - 65535 |
| ... | |
and so on.
There is no limit to the number of chunks and the size of the file, as long as it fits on the disk.
2023-09-25 20:14:03 +00:00
### LoadList chunk
2023-09-24 18:56:36 +00:00
This chunk is a list of what kinds of chunks occur in the file after it.
It starts with a small identification header:
| data | meaning |
|---------|---------------------|
| 2 bytes | 'LL' (76, 76) |
| byte | version (1 for now) |
| byte | reserved (0) |
Then a sequence of 1 or more chunk specs (6 bytes each), as long as it still fits in 256 bytes:
| data | meaning |
|----------------------|--------------------------------------------|
| byte | chunk type |
2023-09-25 20:14:03 +00:00
| word (little-endian) | chunk size |
2023-09-24 18:56:36 +00:00
| byte | bank number (used for some chunk types) |
| word (little-endian) | memory address (used for some chunk types) |
Total 6 bytes per occurrence. Any remaining unused bytes in the 256 bytes LoadList chunk are to be padded with byte 255.
If there are more chunks in the file than fit in a single loadlist, we simply add another loadlist block and continue there.
(The file only ends once an End Of File chunk type is encountered in a loadlist.)
2023-09-25 20:14:03 +00:00
### Chunk types
2023-09-24 18:56:36 +00:00
2023-09-25 20:14:03 +00:00
| chunk type | meaning |
|------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 0 - 239 | custom chunk types. See below. |
| 240 - 248 | reserved for future system chunk types. |
| 249 | dummy chunk: read a chunk of the specified number of bytes but don't do anything with it. Useful to realign the file I/O on disk block size. |
| 250 | bonk ram load: use banknumber + address to set the Cartridge RAM ('bonk' RAM) bank and load address and loads the chunk there, then continue streaming. Note this is slower than other types of ram. Rquires 1 page of user chosen buffer area. |
| 251 | system ram load: use banknumber + address to set the RAM bank and load address and loads the chunk there, then continue streaming. |
| 252 | video ram load: use banknumber + address to set the Vera VRAM bank (hi byte) and load address (mid+lo byte) and loads the chunk into video ram there, then continue streaming. |
| 253 | pause streaming. Returns from stream routine with pause status: Carry=clear. And reg.r0=size. until perhaps the program calls the stream routine again to resume. |
| 254 | end of file. Closes the file and stops streaming: returns from stream routine with exit status: Carry=set. |
| 255 | ignore this byte. Used to pad out the loadlist chunk to 256 bytes. |
2023-09-24 18:56:36 +00:00
### Custom chunk types (0 - 239)
When such a custom chunk type is encountered, a user routine is called to get the load address for the chunk data,
then the chunk is loaded into that buffer, and finally a user routine is called to process the chunk data.
The size can be zero, so that the chunk type acts like a simple notification flag for the main program to do something.
The first routine has the following signature:
**get_buffer()**:
Arguments: reg.A = chunk type, reg.XY = chunksize.
2023-09-25 20:14:03 +00:00
Returns: Carry flag=success (set = fail, clear = ok), ram bank in reg.A, memory address in reg.XY.
2023-09-24 18:56:36 +00:00
The second routine has the following signature:
2023-09-25 20:14:03 +00:00
2023-09-24 18:56:36 +00:00
**process_chunk()**:
Arguments: none (save them from the get_buffer call if needed).
Returns: Carry flag=success (set = fail, clear = ok).
These routines are provided to the streaming routine as callback addresses (ram bank number + address to call).
If any of these routines returns Carry set (error status) the streaming routine halts, otherwise it keeps on going.
The streaming continues until a End of File chunk type is encountered in the loadlist.
