Commit Graph

209 Commits

Author SHA1 Message Date
KrisKennaway
cb35ead5f3
Merge pull request #13 from KrisKennaway/python3.10
Update to python 3.10 now that blocker (weighted-levenshetin) is resolved.  3.11 is still blocked by llvmlite.
2023-02-09 23:58:19 +00:00
kris
08526ed190 Update to python 3.10 now that blocker (weighted-levenshtein) is
resolved.  3.11 is still blocked by llvmlite.

Also refresh other dependencies while I'm here.
2023-02-09 23:57:30 +00:00
KrisKennaway
ee66fa0bcf
Merge pull request #12 from KrisKennaway/fix-encoder
Modernize and fix some image quality bugs
2023-01-28 21:42:58 +00:00
kris
a925a897a7 Update README to note emulation options 2023-01-28 21:37:41 +00:00
kris
97403ab431 Create data dir before writing to it 2023-01-25 22:57:18 +00:00
kris
4529bc3c74 Add a TODO to upload more youtube videos showing quality improvements 2023-01-24 22:38:56 +00:00
kris
f055920dc8 Point to b2d fork instead of the original bmp2dhr source, it's somewhat newer and seems to have made some improvements. 2023-01-24 22:34:28 +00:00
kris
bbb4d14db8 Link to sample videos 2023-01-24 22:31:28 +00:00
kris
53dac6a47c Disable debugging assertions now they've served their purpose 2023-01-24 21:49:33 +00:00
kris
6a8d49bd97 - Stop using the 5x weighting for diff values, it was preventing from
consistently finding enough additional offsets (~2.5x avg instead of
  >2.9)

- Remove instrumentation for fill rate now that it's served its purpose
2023-01-24 21:49:33 +00:00
kris
990e1c9d74 - have Bitmap.apply() update the memory representation instead of
requiring callers to keep track of it

- stop trying to cache content_deltas, I think it results in losing
  deltas.  Instead just recompute the deltas for each page as we need
  it.  This is fast enough in practice.

- track the average fill rate for the additional offsets we emit.
  This should be close to 3 if we're succeeding in finding enough
  collateral work

- overhaul how we pass in the target memory maps.  The previous way
  didn't make sense: we weren't actually encoding for the target video
  frame, but were using an inconsistent mix of old and new frames.  I
  think this was causing image artifacting because we were aiming for
  the wrong thing.

- Add some debugging assertions that were used to track this down.
2023-01-24 21:49:33 +00:00
kris
6b612ffb0a Normalize audio at 0.5/99.5%iles to clip less 2023-01-24 21:49:33 +00:00
kris
157d7596d7 Downgrade numpy until colormath supports 1.23+ 2023-01-24 21:49:33 +00:00
kris
9cea6f7d18 Update to python 3.8 2023-01-24 21:49:33 +00:00
kris
1d5bcfd74e Optimize make_data_tables and use numpy.save instead of pickling. The
file sizes are a bit larger but it unblocks updating to python 3.8.
2023-01-24 21:49:33 +00:00
kris
89633aa845 Add release notes for v0.3 2023-01-17 21:47:26 +00:00
kris
7384878ecb Disable debugging assertions now they've served their purpose 2023-01-17 21:44:43 +00:00
kris
f7f5950637 - Stop using the 5x weighting for diff values, it was preventing from
consistently finding enough additional offsets (~2.5x avg instead of
  >2.9)

- Remove instrumentation for fill rate now that it's served its purpose
2023-01-17 21:42:28 +00:00
kris
0a3c81c8c6 - have Bitmap.apply() update the memory representation instead of
requiring callers to keep track of it

- stop trying to cache content_deltas, I think it results in losing
  deltas.  Instead just recompute the deltas for each page as we need
  it.  This is fast enough in practice.

- track the average fill rate for the additional offsets we emit.
  This should be close to 3 if we're succeeding in finding enough
  collateral work

- overhaul how we pass in the target memory maps.  The previous way
  didn't make sense: we weren't actually encoding for the target video
  frame, but were using an inconsistent mix of old and new frames.  I
  think this was causing image artifacting because we were aiming for
  the wrong thing.

- Add some debugging assertions that were used to track this down.
2023-01-17 21:39:05 +00:00
kris
5c728e7ff1 Allow edit transposes again, there doesn't seem to be a good reason to prevent it and I'm not sure it can even happen in practice 2023-01-17 21:10:49 +00:00
kris
efe821d215 Fix warning 2023-01-17 21:09:34 +00:00
kris
b1132e97f1 Remove pregenerated data files, they are using up all my LFS quota 2023-01-17 21:09:18 +00:00
kris
4f6d7a796a Fix some numpy deprecation warnings 2023-01-17 12:26:25 +00:00
kris
d603445862 Add basic installation instructions and requirements.txt 2023-01-17 12:26:00 +00:00
kris
45ce98d6ef Generate iivision.dbg file which is read by transcoder 2023-01-17 12:25:01 +00:00
KrisKennaway
a885ec1305
Move sample videos up front 2019-07-30 00:07:08 +01:00
KrisKennaway
3709ff295e
Fix header levels 2019-07-30 00:06:12 +01:00
KrisKennaway
7696d5bd0f
Update README.md
Update videos
2019-07-29 23:51:22 +01:00
kris
75e98ce7e0 Update with release notes for v0.2 and revise content 2019-07-17 14:26:29 +01:00
KrisKennaway
142dbe02fe
Merge pull request #3 from KrisKennaway/audio
Improvements to audio processing (quality, support better quality video stream on //gs)
2019-07-14 22:08:04 +01:00
kris
451523bdef Support for custom output bitrate, e.g. to target //gs in 2.8MHz mode.
For some reason playback speed is only about 1.6x (probably due to
slowing down accesses to the I/O page to 1MHz, so as not to mess up
hardware timings), but happily this comes within 3% of being 44100/2.
2019-07-14 22:05:20 +01:00
kris
35f999bc8a Ignore .a2m files in the root directory 2019-07-14 22:05:08 +01:00
kris
6c2f8278ce Oops, 34 ticks is the "neutral" position, not 36. Update duty cycles
during slow path.  This removes the slight buzz audible during periods
of silence.
2019-07-14 22:03:26 +01:00
kris
ce3bed0d38 Try resampling with scipy and scaling audio power 2019-07-13 14:32:21 +01:00
KrisKennaway
326ca62075
Merge pull request #2 from KrisKennaway/dhgr
Dhgr
2019-07-11 23:56:51 +01:00
KrisKennaway
2dcd3dadab
Merge pull request #1 from KrisKennaway/dhgr2
Dhgr2
2019-07-11 23:56:06 +01:00
kris
34f2663448 Initial checkin of edit distance matrices for HGR/DHGR palettes. 2019-07-11 23:44:32 +01:00
kris
1b20cb8f63 Remove obsolete files 2019-07-11 23:43:59 +01:00
kris
f3d03a1b87 Update comments and fix some bugs
make_edit_distance
- use MASKED_DOTS since it does not have a simple relationship to the
  HEADER_BITS/BODY_BITS for HGR
- try disabling transposition distances for Damerau-Levenshtein, this
  may give better quality

screen
- introduce separate notion of MASKED_DOTS which is the number of
  (coloured) pixels we can extract from MASKED_BITS.  For HGR this is
  not the same.
- fix bug in _fix_array_neighbours that was not fixing headers for HGR
- don't cache everything in byte_pair_differences, it's effectively
  unbounded.  Using 1M for LRU size seems to work just as well in
  practise, without leaking memory.
- fix bug in _diff_weights when comparing content, we want to evaluate
  the effect of storing content byte in each offset separately, not
  cumulatively.
- add a consistency check function (not currently wired up) to assert
  that headers/footers are in sync across columns
- HGR should have 16 body bits, this was causing headers not to
  propagate correctly to/from neighbouring column
- add test case for this bug

video
- Use 8 random bits consistently, using 16 in some places may have
  introduced bias
- ignore palette bit when comparing 0x00 and 0x7f in sanity check
2019-07-11 23:40:00 +01:00
kris
722e9c5d70 s/BIT TICK/STA TICK/
It doesn't functionally matter since I'm not using any conditional
opcodes in the fast path anyway, but STA removes these side-effects.

Earlier I had mistakenly thought that STA absolute did a
read-before-write on 6502, but this is not the case
(verified with visual6502.org)
2019-07-11 14:40:05 +01:00
kris
b05c3bec1e - Add a progress bar while computing the edit distance matrices
- Only compute the upper triangle since the matrix is symmetrical and
  we can reconstruct the rest cheaply at load time

- Compute edit distances for HGR as well by making use of the fact that
  the masked representation is 14-bit but still sparse.

- Add sanity checks for various invariants of the edit distance matrices.
2019-07-07 21:25:07 +01:00
kris
ab29b01d0f Finish implementing HGRBitmap support.
- For HGRBitmap introduce a packed representation in the form
  hhHaaaaaaaABbbbbbbbFff where capitals indicate the location of the
  palette bit.  i.e. for header and footer we include the neighbouring
  2 data bits as in DHGR but also the palette bit from that byte, which
  is necessary to understand how these data bits unpack into dots.

  The nonstandard ordering of the palette bit for the odd data byte (B)
  is so that the masking by byte offset produces a contiguous sequence
  of bits, i.e. the 14-bit masked representation is still dense.

- Introduce a to_dots() classmethod that converts from the masked bit
  representation of dots influenced by a screen byte to the actual
  sequence of screen dots.  For DHGR this is the identity map since
  there are no shenanigans with palette bits causing dots to shift
  around.

- Add a bunch more unit tests, and add back the Sather tests for HGR
  artifact colours from palette bit interference, which now all pass!

- Reduce the size of the precomputed edit distance matrix by half by
  exploiting the fact that it is symmetrical under
  i << N + j <-> j << N + i where N is the size of the masked bit
  representation (i.e. transposing the original (i, j) -> dist
  metric matrix).
2019-07-07 21:22:44 +01:00
kris
16c4faa66d Support decoding arbitrary number of dots as nominal colour sequence.
This is needed for HGR support.
2019-07-07 21:13:28 +01:00
kris
fc4a63fffe Support HGR mode again 2019-07-07 21:12:10 +01:00
kris
740dafbd74 Remove commented out code 2019-07-04 15:54:05 +01:00
kris
5c550d8524 Separate the details of the bitmap packing from operations on the
packed representation (diff, apply etc).  This allows the (D)HGRBitmap
classes to focus on the bitmap packing and share common logic.

Numpy has unfortunate long-standing bugs to do with type coercion of
np.uint64, which leads to spurious "incompatible type" warnings when
e.g. operating on a np.uint64 and some other integer type.  To work
around this we cast explicitly to np.uint64 everywhere.

Get tests working again - for now HGR tests in screen_test.py are
disabled until I finish implementing new packing.

HGRBitmap is still incomplete although closer.
2019-07-04 15:21:20 +01:00
kris
666272a8fc Checkpoint WIP for easier comparison to dhgr branch:
- naive version of NTSC artifacting, it uses a sliding 4-bit window to
  assign a nominal (D)HGR colour to each dot position.  A more
  sophisticated/correct implementation would model the YIQ signal
  directly.

- Switch DHGRBitmap implementation to use a 34-bit representation of
  the 4-byte tuple, comprised of a 3-bit header and footer, plus
  4*7=28-bit body.  The headers/footers account for the influence on
  neighbouring tuples from the 4-bit NTSC window.

- With this model each screen byte influences 13 pixels, so we need to
  precompute 2^26 edit distances for all possible (source, target)
  13-bit sequences.

- Checkpointing not-yet-working HGR implementation.

- Add new unit tests but not yet all passing due to refactoring
2019-07-02 22:40:50 +01:00
kris
e2a8bd9b4d Update IP addresses to 10.0.0.0/8 network 2019-06-26 11:24:07 +01:00
kris
2b00a09d11 Update IP addresses to 10.0.0.0/8 network 2019-06-26 11:22:58 +01:00
kris
b2c00784b0 Minor code cleanups 2019-06-21 22:08:22 +01:00