History

Dietrich Epp c058e68d93 Add README for table generator GitOrigin-RevId: 8062aa48890eb74a92b3322fdf306df9e5ecdee2		2021-03-17 21:05:19 -04:00
..
README.md	Add README for table generator	2021-03-17 21:05:19 -04:00
go.mod	Add generator for UTF-8 to Macintosh tables	2021-03-17 21:01:04 -04:00
go.sum	Add generator for UTF-8 to Macintosh tables	2021-03-17 21:01:04 -04:00
macroman.go	Add flags for debugging charset conversion tables	2021-03-17 21:05:18 -04:00

README.md

Character Conversion Tables

Used by SyncFiles.

This program generates the tables necessary to convert from UTF-8 to Mac OS Roman.

The conversion process is entirely table-driven. The table maps a (state, input) pair to a (state, output) pair. The initial state is 0. A transition to state 0 is considered invalid.

A transition may have both a state and output. This means that the input may be translated in different ways depending on the bytes that follow. The translation code prefers the longest path through the state table that results in an output.

The table is compressed with PackBits to reduce its size by a factor of 22x.