Macintosh-SW/syncfiles
1
0
Fork 0
mirror of https://github.com/depp/syncfiles.git synced 2024-11-22 03:30:57 +00:00
Code Issues Projects Releases Wiki Activity
db4187b65b
syncfiles/gen
History
Dietrich Epp db4187b65b Refactor table generator, handle line breaks 
Extract table generation to its own file, table.go, and refactor the
interface. This exposed an inconsistency in the way that line breaks
were handled: both CR and LF on the Mac side were mapped to LF on the
UTF-8 side, but when the conversion table was inverted, the reverse
mappings would conflict. Previously, there was no explicit handling for
it, and whichever Mac charecter had a higher byte value would take
precedence. Conflicts are now detected and return an error, so line
breaks must be mapped explicitly. The new code maps CR, LF, and CRLF to
CR when converting UTF-8 to Mac.
2021-12-14 13:12:52 -05:00
..
.gitignore Ignore conversion table generator binary 2021-12-14 11:01:37 -05:00
go.mod Extract packbits into separate repository 2021-12-14 12:23:14 -05:00
go.sum Extract packbits into separate repository 2021-12-14 12:23:14 -05:00
macroman.go Refactor table generator, handle line breaks 2021-12-14 13:12:52 -05:00
README.md Add README for table generator 2021-03-17 21:05:19 -04:00
table.go Refactor table generator, handle line breaks 2021-12-14 13:12:52 -05:00

README.md

Character Conversion Tables

Used by SyncFiles.

This program generates the tables necessary to convert from UTF-8 to Mac OS Roman.

The conversion process is entirely table-driven. The table maps a (state, input) pair to a (state, output) pair. The initial state is 0. A transition to state 0 is considered invalid.

A transition may have both a state and output. This means that the input may be translated in different ways depending on the bytes that follow. The translation code prefers the longest path through the state table that results in an output.

The table is compressed with PackBits to reduce its size by a factor of 22x.