Commit Graph

11 Commits

Author SHA1 Message Date
kris
97403ab431 Create data dir before writing to it 2023-01-25 22:57:18 +00:00
kris
1d5bcfd74e Optimize make_data_tables and use numpy.save instead of pickling. The
file sizes are a bit larger but it unblocks updating to python 3.8.
2023-01-24 21:49:33 +00:00
kris
5c728e7ff1 Allow edit transposes again, there doesn't seem to be a good reason to prevent it and I'm not sure it can even happen in practice 2023-01-17 21:10:49 +00:00
kris
4f6d7a796a Fix some numpy deprecation warnings 2023-01-17 12:26:25 +00:00
kris
f3d03a1b87 Update comments and fix some bugs
make_edit_distance
- use MASKED_DOTS since it does not have a simple relationship to the
  HEADER_BITS/BODY_BITS for HGR
- try disabling transposition distances for Damerau-Levenshtein, this
  may give better quality

screen
- introduce separate notion of MASKED_DOTS which is the number of
  (coloured) pixels we can extract from MASKED_BITS.  For HGR this is
  not the same.
- fix bug in _fix_array_neighbours that was not fixing headers for HGR
- don't cache everything in byte_pair_differences, it's effectively
  unbounded.  Using 1M for LRU size seems to work just as well in
  practise, without leaking memory.
- fix bug in _diff_weights when comparing content, we want to evaluate
  the effect of storing content byte in each offset separately, not
  cumulatively.
- add a consistency check function (not currently wired up) to assert
  that headers/footers are in sync across columns
- HGR should have 16 body bits, this was causing headers not to
  propagate correctly to/from neighbouring column
- add test case for this bug

video
- Use 8 random bits consistently, using 16 in some places may have
  introduced bias
- ignore palette bit when comparing 0x00 and 0x7f in sanity check
2019-07-11 23:40:00 +01:00
kris
b05c3bec1e - Add a progress bar while computing the edit distance matrices
- Only compute the upper triangle since the matrix is symmetrical and
  we can reconstruct the rest cheaply at load time

- Compute edit distances for HGR as well by making use of the fact that
  the masked representation is 14-bit but still sparse.

- Add sanity checks for various invariants of the edit distance matrices.
2019-07-07 21:25:07 +01:00
kris
666272a8fc Checkpoint WIP for easier comparison to dhgr branch:
- naive version of NTSC artifacting, it uses a sliding 4-bit window to
  assign a nominal (D)HGR colour to each dot position.  A more
  sophisticated/correct implementation would model the YIQ signal
  directly.

- Switch DHGRBitmap implementation to use a 34-bit representation of
  the 4-byte tuple, comprised of a 3-bit header and footer, plus
  4*7=28-bit body.  The headers/footers account for the influence on
  neighbouring tuples from the 4-bit NTSC window.

- With this model each screen byte influences 13 pixels, so we need to
  precompute 2^26 edit distances for all possible (source, target)
  13-bit sequences.

- Checkpointing not-yet-working HGR implementation.

- Add new unit tests but not yet all passing due to refactoring
2019-07-02 22:40:50 +01:00
kris
b2c00784b0 Minor code cleanups 2019-06-21 22:08:22 +01:00
kris
ca817999f6 Parametrize the RGB palette to encode with, and support both NTSC and
IIGS palettes.

Move the palette diff_matrix generation into make_data_tables.py since
that is the only place it is used.

Demand-load the edit distance matrices when transcoding.
2019-06-15 21:02:00 +01:00
kris
824154fd39 Compress the precomputed edit_distance arrays 2019-06-14 22:08:34 +01:00
kris
fb14a20380 Precompute the edit distance between the 8-bit (2-pixel) and 12-bit
(3-pixel) sequences that may be modified when storing bytes to the
DHGR display.

This relies on producing an efficient linear representation of the
DHGR framebuffer in terms of a packed 28-bit representation of (Aux,
Main, Aux, Main) screen bytes.
2019-06-13 23:27:30 +01:00