Apple-2-SW/ii-vision
1
0
Fork 0
mirror of https://github.com/KrisKennaway/ii-vision.git synced 2024-06-26 00:29:29 +00:00
Code Issues Projects Releases Wiki Activity
d4a27444c6
ii-vision/transcoder/video.py

308 lines
10 KiB
Python
Raw Normal View History

Add module docstrings
2019-03-21 16:42:47 +00:00
"""Encode a sequence of images as an optimized stream of screen changes."""
Integrated audio + video player! - Introduce a new Movie() class that multiplexes audio and video. - Every N audio frames we grab a new video frame and begin pulling opcodes from the audio and video streams - Grab frames from the input video using bmp2dhr if the .BIN file does not already exist. Run bmp2dhr in a background thread to not block encoding - move the output byte streaming from Video to Movie - For now, manually clip updates to pages > 56 since the client doesn't support them yet The way we encode video is now: - iterate in descending order over update_priority - begin a new (page, content) opcode - for all of the other offset bytes in that page, compute the error between the candidate content byte and the target content byte - iterate over offsets in order of increasing error and decreasing update_priority to fill out the remaining opcode
2019-03-07 23:07:24 +00:00
import heapq
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
import os
import queue
Fix imports and point to player debug symbols
2019-03-21 16:24:40 +00:00
import random
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
import subprocess
Fix imports and point to player debug symbols
2019-03-21 16:24:40 +00:00
import threading
Vectorize the computation of diff weights, by precomputing a map of all possible weights. We encode the two 8-bit inputs into a single 16 bit value instead of dealing with an array of tuples. Fix an important bug in _compute_delta: old and is_odd were transposed so we weren't actually subtracting the old deltas! Surprisingly, when I accidentally fixed this bug in the vectorized version, the video encoding was much worse! This turned out to be because the edit distance metric allowed reducing diffs by turning on pixels, which meant it would tend to do this when "minimizing error" in a way that was visually unappealing. To remedy this, introduce a separate notion of substitution cost for errors, and weight pixel colour changes more highly to discourage them unless absolutely necessary. This gives very good quality results! Also vectorize the selection of page offsets and priorities having a negative error delta, instead of heapifying the entire page. Also it turns out to be a bit faster to compute (and memoize) the delta between a proposed content byte and the entire target screen at once, since we'll end up recomputing the same content diffs multiple times. (Committing messy version in case I want to revisit some of those interim versions)
2019-03-14 22:08:50 +00:00
from typing import List, Iterator, Tuple
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
- Don't hardcode clock speed - Commented-out experimentation with using hitherdither library to use yliluoma dithering (see https://bisqwit.iki.fi/story/howto/dither/jy/) instead of the (error-diffusion) based BMP2DHR which introduces a lot of noise between frames since it is easily perturbed. Unfortunately apart from being extremely slow, it also doesn't give good results, even for (simulated) DHGR palette. There's a lot of banding and for HGR the available colours are just too far apart in colour space. This is even without (somehow) applying the HGR colour constraints. - Also return the priority from _compute_error as preparation for reinserting the offset back into the priority heap, in case we can do a better job later. In order to do this properly we need to compute both the error edit distance and the "true" edit distance and only insert the priority of the latter.
2019-03-21 15:57:09 +00:00
# import hitherdither
Introduction more general notion of update priority used to increase weight of diffs that persist across multiple frames. For each frame, zero out update priority of bytes that no longer have a pending diff, and add the edit distance of the remaining diffs. Zero these out as opcodes are retired. Replace hamming distance with Damerau-Levenshtein distance of the encoded pixel colours in the byte, e.g. 0x2A --> GGG0 (taking into account the half-pixel) This has a couple of benefits over hamming distance of the bit patterns: - transposed pixels are weighted less (edit distance 1, not 2+ for Hamming) - coloured pixels are weighted equally as white pixels (not half as much) - weighting changes in palette bit that flip multiple pixel colours While I'm here, the RLE opcode should emit run_length - 1 so that we can encode runs of 256 bytes.
2019-03-04 23:09:00 +00:00
import numpy as np
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
import skvideo.io
Fix imports and point to player debug symbols
2019-03-21 16:24:40 +00:00
from PIL import Image
Introduction more general notion of update priority used to increase weight of diffs that persist across multiple frames. For each frame, zero out update priority of bytes that no longer have a pending diff, and add the edit distance of the remaining diffs. Zero these out as opcodes are retired. Replace hamming distance with Damerau-Levenshtein distance of the encoded pixel colours in the byte, e.g. 0x2A --> GGG0 (taking into account the half-pixel) This has a couple of benefits over hamming distance of the bit patterns: - transposed pixels are weighted less (edit distance 1, not 2+ for Hamming) - coloured pixels are weighted equally as white pixels (not half as much) - weighting changes in palette bit that flip multiple pixel colours While I'm here, the RLE opcode should emit run_length - 1 so that we can encode runs of 256 bytes.
2019-03-04 23:09:00 +00:00
Fix imports and point to player debug symbols
2019-03-21 16:24:40 +00:00
import edit_distance
import opcodes
import screen
Refactor the world
2019-01-05 23:31:56 +00:00
class Video:
"""Apple II screen memory map encoding a bitmapped frame."""
Style cleanups
2019-03-14 23:05:15 +00:00
CLOCK_SPEED = 1024 * 1024 # type: int
Video() is now aware of target frame rate, and will continue to emit opcodes until the cycle budget for the frame is exhausted. Output stream is also now aware of TCP framing, and schedules an ACK opcode every 2048 output bytes to instruct the client to perform TCP ACK and buffer management. Fixes several serious bugs in RLE encoding, including: - we were emitting the RLE opcode with the next content byte after the run completed! - we were looking at the wrong field for the start offset! - handle the case where the entire page is a single run - stop trying to allow accumulating error when RLE -- this does not respect the Apple II colour encoding, i.e. may introduce colour fringing. - also because of this we're unlikely to actually be able to find many runs because odd and even columns are encoded differently. In a followup we should start encoding odd and even columns separately Optimize after profiling -- encoder is now about 2x faster Add tests.
2019-02-23 23:52:25 +00:00
Add some docstrings Clean up naming in edit_distance In video encoder, when we emit additional offsets as part of an opcode, reinsert back into the priority heapq if the new edit distance is nonzero, in case we get the chance to fix it up later in the frame. Also make sure to zero out the diff_weights and content_deltas so we don't consider the offset again as a side-effect of some other opcode. Instead of prioritizing side-effect offsets by their previous update priority, prioritize by those with the lowest (error - edit) delta i.e. not introducing too much error relative to their edit distance.
2019-03-21 22:56:45 +00:00
def __init__(
self,
filename: str,
):
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
self.filename = filename # type: str
self._reader = skvideo.io.FFmpegReader(filename)
# Compute frame rate from input video
Style cleanups
2019-03-14 23:05:15 +00:00
# TODO: possible to compute time offset for each frame instead?
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
data = skvideo.io.ffprobe(self.filename)['video']
rate_data = data['@r_frame_rate'].split("/") # e.g. 12000/1001
Print input frame rate, and default to every_n_video_frames=2 which gives better quality without compromising frame rate too much.
2019-03-21 23:25:51 +00:00
self.input_frame_rate = float(
Style cleanups
2019-03-14 23:05:15 +00:00
rate_data[0]) / float(rate_data[1]) # type: float
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
Style cleanups
2019-03-14 23:05:15 +00:00
self.cycles_per_frame = (
Print input frame rate, and default to every_n_video_frames=2 which gives better quality without compromising frame rate too much.
2019-03-21 23:25:51 +00:00
self.CLOCK_SPEED / self.input_frame_rate) # type: float
Style cleanups
2019-03-14 23:05:15 +00:00
self.frame_number = 0 # type: int
Refactor the world
2019-01-05 23:31:56 +00:00
Style cleanups
2019-03-14 23:05:15 +00:00
# Initialize empty screen
Refactor the various representations of screen memory (bitmap, (x,y) bytemap, (page,offset) memory map) - add a FlatMemoryMap that is a linear 8K array - add converter methods and default constructors that allow converting between them - use MemoryMap as the central representation used by the video encoder
2019-03-03 22:21:28 +00:00
self.memory_map = screen.MemoryMap(
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
screen_page=1) # type: screen.MemoryMap
Video() is now aware of target frame rate, and will continue to emit opcodes until the cycle budget for the frame is exhausted. Output stream is also now aware of TCP framing, and schedules an ACK opcode every 2048 output bytes to instruct the client to perform TCP ACK and buffer management. Fixes several serious bugs in RLE encoding, including: - we were emitting the RLE opcode with the next content byte after the run completed! - we were looking at the wrong field for the start offset! - handle the case where the entire page is a single run - stop trying to allow accumulating error when RLE -- this does not respect the Apple II colour encoding, i.e. may introduce colour fringing. - also because of this we're unlikely to actually be able to find many runs because odd and even columns are encoded differently. In a followup we should start encoding odd and even columns separately Optimize after profiling -- encoder is now about 2x faster Add tests.
2019-02-23 23:52:25 +00:00
Introduction more general notion of update priority used to increase weight of diffs that persist across multiple frames. For each frame, zero out update priority of bytes that no longer have a pending diff, and add the edit distance of the remaining diffs. Zero these out as opcodes are retired. Replace hamming distance with Damerau-Levenshtein distance of the encoded pixel colours in the byte, e.g. 0x2A --> GGG0 (taking into account the half-pixel) This has a couple of benefits over hamming distance of the bit patterns: - transposed pixels are weighted less (edit distance 1, not 2+ for Hamming) - coloured pixels are weighted equally as white pixels (not half as much) - weighting changes in palette bit that flip multiple pixel colours While I'm here, the RLE opcode should emit run_length - 1 so that we can encode runs of 256 bytes.
2019-03-04 23:09:00 +00:00
# Accumulates pending edit weights across frames
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
self.update_priority = np.zeros((32, 256), dtype=np.int64)
Style cleanups
2019-03-14 23:05:15 +00:00
def tick(self, cycles: int) -> bool:
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
if cycles > (self.cycles_per_frame * self.frame_number):
self.frame_number += 1
return True
return False
Introduction more general notion of update priority used to increase weight of diffs that persist across multiple frames. For each frame, zero out update priority of bytes that no longer have a pending diff, and add the edit distance of the remaining diffs. Zero these out as opcodes are retired. Replace hamming distance with Damerau-Levenshtein distance of the encoded pixel colours in the byte, e.g. 0x2A --> GGG0 (taking into account the half-pixel) This has a couple of benefits over hamming distance of the bit patterns: - transposed pixels are weighted less (edit distance 1, not 2+ for Hamming) - coloured pixels are weighted equally as white pixels (not half as much) - weighting changes in palette bit that flip multiple pixel colours While I'm here, the RLE opcode should emit run_length - 1 so that we can encode runs of 256 bytes.
2019-03-04 23:09:00 +00:00
- Don't hardcode clock speed - Commented-out experimentation with using hitherdither library to use yliluoma dithering (see https://bisqwit.iki.fi/story/howto/dither/jy/) instead of the (error-diffusion) based BMP2DHR which introduces a lot of noise between frames since it is easily perturbed. Unfortunately apart from being extremely slow, it also doesn't give good results, even for (simulated) DHGR palette. There's a lot of banding and for HGR the available colours are just too far apart in colour space. This is even without (somehow) applying the HGR colour constraints. - Also return the priority from _compute_error as preparation for reinserting the offset back into the priority heap, in case we can do a better job later. In order to do this properly we need to compute both the error edit distance and the "true" edit distance and only insert the priority of the latter.
2019-03-21 15:57:09 +00:00
def _frame_grabber(self) -> Iterator[Image.Image]:
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
for frame_array in self._reader.nextFrame():
yield Image.fromarray(frame_array)
Refactor the world
2019-01-05 23:31:56 +00:00
- Don't hardcode clock speed - Commented-out experimentation with using hitherdither library to use yliluoma dithering (see https://bisqwit.iki.fi/story/howto/dither/jy/) instead of the (error-diffusion) based BMP2DHR which introduces a lot of noise between frames since it is easily perturbed. Unfortunately apart from being extremely slow, it also doesn't give good results, even for (simulated) DHGR palette. There's a lot of banding and for HGR the available colours are just too far apart in colour space. This is even without (somehow) applying the HGR colour constraints. - Also return the priority from _compute_error as preparation for reinserting the offset back into the priority heap, in case we can do a better job later. In order to do this properly we need to compute both the error edit distance and the "true" edit distance and only insert the priority of the latter.
2019-03-21 15:57:09 +00:00
@staticmethod
Style
2019-03-21 16:04:26 +00:00
def _rgb(r, g, b):
- Don't hardcode clock speed - Commented-out experimentation with using hitherdither library to use yliluoma dithering (see https://bisqwit.iki.fi/story/howto/dither/jy/) instead of the (error-diffusion) based BMP2DHR which introduces a lot of noise between frames since it is easily perturbed. Unfortunately apart from being extremely slow, it also doesn't give good results, even for (simulated) DHGR palette. There's a lot of banding and for HGR the available colours are just too far apart in colour space. This is even without (somehow) applying the HGR colour constraints. - Also return the priority from _compute_error as preparation for reinserting the offset back into the priority heap, in case we can do a better job later. In order to do this properly we need to compute both the error edit distance and the "true" edit distance and only insert the priority of the latter.
2019-03-21 15:57:09 +00:00
return (r << 16) + (g << 8) + b
# def dither_framesframes(self) -> Iterator[screen.MemoryMap]:
# palette = hitherdither.palette.Palette(
# [
# self._rgb(0,0,0), # black */
# self._rgb(148,12,125), # red - hgr 0*/
# self._rgb(32,54,212), # dk blue - hgr 0 */
# self._rgb(188,55,255), # purple - default HGR overlay color */
# self._rgb(51,111,0), # dk green - hgr 0 */
# self._rgb(126,126,126), # gray - hgr 0 */
# self._rgb(7,168,225), # med blue - alternate HGR overlay
# # color */
# self._rgb(158,172,255), # lt blue - hgr 0 */
# self._rgb(99,77,0), # brown - hgr 0 */
# self._rgb(249,86,29), # orange */
# self._rgb(126,126,126), # grey - hgr 0 */
# self._rgb(255,129,236), # pink - hgr 0 */
# self._rgb(67,200,0), # lt green */
# self._rgb(221,206,23), # yellow - hgr 0 */
# self._rgb(93,248,133), # aqua - hgr 0 */
# self._rgb(255,255,255) # white
# ]
# )
# for _idx, _frame in enumerate(self._frame_grabber()):
# if _idx % 60 == 0:
# img_dithered = hitherdither.ordered.yliluoma.yliluomas_1_ordered_dithering(
# _frame.resize((280,192), resample=Image.NEAREST),
# palette, order=8)
#
# yield img_dithered
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
def frames(self) -> Iterator[screen.MemoryMap]:
"""Encode frame to HGR using bmp2dhr.
Refactor the world
2019-01-05 23:31:56 +00:00
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
We do the encoding in a background thread to parallelize.
Refactor the world
2019-01-05 23:31:56 +00:00
"""
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
frame_dir = self.filename.split(".")[0]
try:
os.mkdir(frame_dir)
except FileExistsError:
pass
q = queue.Queue(maxsize=10)
def worker():
Style cleanups
2019-03-14 23:05:15 +00:00
"""Invoke bmp2dhr to encode input image frames and push to queue."""
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
for _idx, _frame in enumerate(self._frame_grabber()):
outfile = "%s/%08dC.BIN" % (frame_dir, _idx)
bmpfile = "%s/%08d.bmp" % (frame_dir, _idx)
try:
os.stat(outfile)
except FileNotFoundError:
Downsize image using LANCZOS resampler.
2019-03-22 00:06:29 +00:00
_frame = _frame.resize((280, 192), resample=Image.LANCZOS)
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
_frame.save(bmpfile)
subprocess.call(
["/usr/local/bin/bmp2dhr", bmpfile, "hgr", "D9"])
os.remove(bmpfile)
_frame = np.fromfile(outfile, dtype=np.uint8)
q.put(_frame)
q.put(None)
Style cleanups
2019-03-14 23:05:15 +00:00
t = threading.Thread(target=worker, daemon=True)
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
t.start()
while True:
frame = q.get()
if frame is None:
break
yield screen.FlatMemoryMap(
screen_page=1, data=frame).to_memory_map()
q.task_done()
t.join()
def encode_frame(
self, target: screen.MemoryMap
) -> Iterator[opcodes.Opcode]:
"""Update to match content of frame within provided budget."""
print("Similarity %f" % (self.update_priority.mean()))
Integrated audio + video player! - Introduce a new Movie() class that multiplexes audio and video. - Every N audio frames we grab a new video frame and begin pulling opcodes from the audio and video streams - Grab frames from the input video using bmp2dhr if the .BIN file does not already exist. Run bmp2dhr in a background thread to not block encoding - move the output byte streaming from Video to Movie - For now, manually clip updates to pages > 56 since the client doesn't support them yet The way we encode video is now: - iterate in descending order over update_priority - begin a new (page, content) opcode - for all of the other offset bytes in that page, compute the error between the candidate content byte and the target content byte - iterate over offsets in order of increasing error and decreasing update_priority to fill out the remaining opcode
2019-03-07 23:07:24 +00:00
yield from self._index_changes(self.memory_map, target)
Video() is now aware of target frame rate, and will continue to emit opcodes until the cycle budget for the frame is exhausted. Output stream is also now aware of TCP framing, and schedules an ACK opcode every 2048 output bytes to instruct the client to perform TCP ACK and buffer management. Fixes several serious bugs in RLE encoding, including: - we were emitting the RLE opcode with the next content byte after the run completed! - we were looking at the wrong field for the start offset! - handle the case where the entire page is a single run - stop trying to allow accumulating error when RLE -- this does not respect the Apple II colour encoding, i.e. may introduce colour fringing. - also because of this we're unlikely to actually be able to find many runs because odd and even columns are encoded differently. In a followup we should start encoding odd and even columns separately Optimize after profiling -- encoder is now about 2x faster Add tests.
2019-02-23 23:52:25 +00:00
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
def _index_changes(
self,
source: screen.MemoryMap,
target: screen.MemoryMap
Move edit distance functions into separate module and clean up partially. Slight optimization to not heapppush() many times, instead build a regular list and then heapify.
2019-03-14 22:32:52 +00:00
) -> Iterator[Tuple[int, int, List[int]]]:
"""Transform encoded screen to sequence of change tuples."""
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
Move edit distance functions into separate module and clean up partially. Slight optimization to not heapppush() many times, instead build a regular list and then heapify.
2019-03-14 22:32:52 +00:00
diff_weights = self._diff_weights(source, target)
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
Move edit distance functions into separate module and clean up partially. Slight optimization to not heapppush() many times, instead build a regular list and then heapify.
2019-03-14 22:32:52 +00:00
# Clear any update priority entries that have resolved themselves
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
# with new frame
self.update_priority[diff_weights == 0] = 0
Vectorize the computation of diff weights, by precomputing a map of all possible weights. We encode the two 8-bit inputs into a single 16 bit value instead of dealing with an array of tuples. Fix an important bug in _compute_delta: old and is_odd were transposed so we weren't actually subtracting the old deltas! Surprisingly, when I accidentally fixed this bug in the vectorized version, the video encoding was much worse! This turned out to be because the edit distance metric allowed reducing diffs by turning on pixels, which meant it would tend to do this when "minimizing error" in a way that was visually unappealing. To remedy this, introduce a separate notion of substitution cost for errors, and weight pixel colour changes more highly to discourage them unless absolutely necessary. This gives very good quality results! Also vectorize the selection of page offsets and priorities having a negative error delta, instead of heapifying the entire page. Also it turns out to be a bit faster to compute (and memoize) the delta between a proposed content byte and the entire target screen at once, since we'll end up recomputing the same content diffs multiple times. (Committing messy version in case I want to revisit some of those interim versions)
2019-03-14 22:08:50 +00:00
# Halve existing weights to increase bias to new diffs.
# In particular this means that existing updates with diff 1 will
# become diff 0, i.e. will only be prioritized if they are still
# diffs in the new frame.
# self.update_priority >>= 1
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
self.update_priority += diff_weights
priorities = self._heapify_priorities()
Vectorize the computation of diff weights, by precomputing a map of all possible weights. We encode the two 8-bit inputs into a single 16 bit value instead of dealing with an array of tuples. Fix an important bug in _compute_delta: old and is_odd were transposed so we weren't actually subtracting the old deltas! Surprisingly, when I accidentally fixed this bug in the vectorized version, the video encoding was much worse! This turned out to be because the edit distance metric allowed reducing diffs by turning on pixels, which meant it would tend to do this when "minimizing error" in a way that was visually unappealing. To remedy this, introduce a separate notion of substitution cost for errors, and weight pixel colour changes more highly to discourage them unless absolutely necessary. This gives very good quality results! Also vectorize the selection of page offsets and priorities having a negative error delta, instead of heapifying the entire page. Also it turns out to be a bit faster to compute (and memoize) the delta between a proposed content byte and the entire target screen at once, since we'll end up recomputing the same content diffs multiple times. (Committing messy version in case I want to revisit some of those interim versions)
2019-03-14 22:08:50 +00:00
content_deltas = {}
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
while priorities:
_, _, page, offset = heapq.heappop(priorities)
# Check whether we've already cleared this diff while processing
# an earlier opcode
if self.update_priority[page, offset] == 0:
Integrated audio + video player! - Introduce a new Movie() class that multiplexes audio and video. - Every N audio frames we grab a new video frame and begin pulling opcodes from the audio and video streams - Grab frames from the input video using bmp2dhr if the .BIN file does not already exist. Run bmp2dhr in a background thread to not block encoding - move the output byte streaming from Video to Movie - For now, manually clip updates to pages > 56 since the client doesn't support them yet The way we encode video is now: - iterate in descending order over update_priority - begin a new (page, content) opcode - for all of the other offset bytes in that page, compute the error between the candidate content byte and the target content byte - iterate over offsets in order of increasing error and decreasing update_priority to fill out the remaining opcode
2019-03-07 23:07:24 +00:00
continue
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
Integrated audio + video player! - Introduce a new Movie() class that multiplexes audio and video. - Every N audio frames we grab a new video frame and begin pulling opcodes from the audio and video streams - Grab frames from the input video using bmp2dhr if the .BIN file does not already exist. Run bmp2dhr in a background thread to not block encoding - move the output byte streaming from Video to Movie - For now, manually clip updates to pages > 56 since the client doesn't support them yet The way we encode video is now: - iterate in descending order over update_priority - begin a new (page, content) opcode - for all of the other offset bytes in that page, compute the error between the candidate content byte and the target content byte - iterate over offsets in order of increasing error and decreasing update_priority to fill out the remaining opcode
2019-03-07 23:07:24 +00:00
offsets = [offset]
content = target.page_offset[page, offset]
# Clear priority for the offset we're emitting
self.update_priority[page, offset] = 0
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
self.memory_map.page_offset[page, offset] = content
Add some docstrings Clean up naming in edit_distance In video encoder, when we emit additional offsets as part of an opcode, reinsert back into the priority heapq if the new edit distance is nonzero, in case we get the chance to fix it up later in the frame. Also make sure to zero out the diff_weights and content_deltas so we don't consider the offset again as a side-effect of some other opcode. Instead of prioritizing side-effect offsets by their previous update priority, prioritize by those with the lowest (error - edit) delta i.e. not introducing too much error relative to their edit distance.
2019-03-21 22:56:45 +00:00
diff_weights[page, offset] = 0
# Make sure we don't emit this offset as a side-effect of some
# other offset later.
for cd in content_deltas.values():
cd[page, offset] = 0
Integrated audio + video player! - Introduce a new Movie() class that multiplexes audio and video. - Every N audio frames we grab a new video frame and begin pulling opcodes from the audio and video streams - Grab frames from the input video using bmp2dhr if the .BIN file does not already exist. Run bmp2dhr in a background thread to not block encoding - move the output byte streaming from Video to Movie - For now, manually clip updates to pages > 56 since the client doesn't support them yet The way we encode video is now: - iterate in descending order over update_priority - begin a new (page, content) opcode - for all of the other offset bytes in that page, compute the error between the candidate content byte and the target content byte - iterate over offsets in order of increasing error and decreasing update_priority to fill out the remaining opcode
2019-03-07 23:07:24 +00:00
# Need to find 3 more offsets to fill this opcode
Add some docstrings Clean up naming in edit_distance In video encoder, when we emit additional offsets as part of an opcode, reinsert back into the priority heapq if the new edit distance is nonzero, in case we get the chance to fix it up later in the frame. Also make sure to zero out the diff_weights and content_deltas so we don't consider the offset again as a side-effect of some other opcode. Instead of prioritizing side-effect offsets by their previous update priority, prioritize by those with the lowest (error - edit) delta i.e. not introducing too much error relative to their edit distance.
2019-03-21 22:56:45 +00:00
for o in self._compute_error(
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
page,
Integrated audio + video player! - Introduce a new Movie() class that multiplexes audio and video. - Every N audio frames we grab a new video frame and begin pulling opcodes from the audio and video streams - Grab frames from the input video using bmp2dhr if the .BIN file does not already exist. Run bmp2dhr in a background thread to not block encoding - move the output byte streaming from Video to Movie - For now, manually clip updates to pages > 56 since the client doesn't support them yet The way we encode video is now: - iterate in descending order over update_priority - begin a new (page, content) opcode - for all of the other offset bytes in that page, compute the error between the candidate content byte and the target content byte - iterate over offsets in order of increasing error and decreasing update_priority to fill out the remaining opcode
2019-03-07 23:07:24 +00:00
content,
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
target,
Vectorize the computation of diff weights, by precomputing a map of all possible weights. We encode the two 8-bit inputs into a single 16 bit value instead of dealing with an array of tuples. Fix an important bug in _compute_delta: old and is_odd were transposed so we weren't actually subtracting the old deltas! Surprisingly, when I accidentally fixed this bug in the vectorized version, the video encoding was much worse! This turned out to be because the edit distance metric allowed reducing diffs by turning on pixels, which meant it would tend to do this when "minimizing error" in a way that was visually unappealing. To remedy this, introduce a separate notion of substitution cost for errors, and weight pixel colour changes more highly to discourage them unless absolutely necessary. This gives very good quality results! Also vectorize the selection of page offsets and priorities having a negative error delta, instead of heapifying the entire page. Also it turns out to be a bit faster to compute (and memoize) the delta between a proposed content byte and the entire target screen at once, since we'll end up recomputing the same content diffs multiple times. (Committing messy version in case I want to revisit some of those interim versions)
2019-03-14 22:08:50 +00:00
diff_weights,
content_deltas
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
):
offsets.append(o)
Add some docstrings Clean up naming in edit_distance In video encoder, when we emit additional offsets as part of an opcode, reinsert back into the priority heapq if the new edit distance is nonzero, in case we get the chance to fix it up later in the frame. Also make sure to zero out the diff_weights and content_deltas so we don't consider the offset again as a side-effect of some other opcode. Instead of prioritizing side-effect offsets by their previous update priority, prioritize by those with the lowest (error - edit) delta i.e. not introducing too much error relative to their edit distance.
2019-03-21 22:56:45 +00:00
# Compute new edit distance between new content and target
# byte, so we can reinsert with this value
p = edit_distance.edit_weight(
content, target.page_offset[page, o], o % 2 == 1,
error=False)
- Don't hardcode clock speed - Commented-out experimentation with using hitherdither library to use yliluoma dithering (see https://bisqwit.iki.fi/story/howto/dither/jy/) instead of the (error-diffusion) based BMP2DHR which introduces a lot of noise between frames since it is easily perturbed. Unfortunately apart from being extremely slow, it also doesn't give good results, even for (simulated) DHGR palette. There's a lot of banding and for HGR the available colours are just too far apart in colour space. This is even without (somehow) applying the HGR colour constraints. - Also return the priority from _compute_error as preparation for reinserting the offset back into the priority heap, in case we can do a better job later. In order to do this properly we need to compute both the error edit distance and the "true" edit distance and only insert the priority of the latter.
2019-03-21 15:57:09 +00:00
# Update priority for the offset we're emitting
Add some docstrings Clean up naming in edit_distance In video encoder, when we emit additional offsets as part of an opcode, reinsert back into the priority heapq if the new edit distance is nonzero, in case we get the chance to fix it up later in the frame. Also make sure to zero out the diff_weights and content_deltas so we don't consider the offset again as a side-effect of some other opcode. Instead of prioritizing side-effect offsets by their previous update priority, prioritize by those with the lowest (error - edit) delta i.e. not introducing too much error relative to their edit distance.
2019-03-21 22:56:45 +00:00
self.update_priority[page, o] = p # 0
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
self.memory_map.page_offset[page, o] = content
Add some docstrings Clean up naming in edit_distance In video encoder, when we emit additional offsets as part of an opcode, reinsert back into the priority heapq if the new edit distance is nonzero, in case we get the chance to fix it up later in the frame. Also make sure to zero out the diff_weights and content_deltas so we don't consider the offset again as a side-effect of some other opcode. Instead of prioritizing side-effect offsets by their previous update priority, prioritize by those with the lowest (error - edit) delta i.e. not introducing too much error relative to their edit distance.
2019-03-21 22:56:45 +00:00
if p:
# This content byte introduced an error, so put back on the
# heap in case we can get back to fixing it exactly
# during this frame. Otherwise we'll get to it later.
heapq.heappush(
priorities, (-p, random.random(), page, offset))
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
# Pad to 4 if we didn't find enough
Integrated audio + video player! - Introduce a new Movie() class that multiplexes audio and video. - Every N audio frames we grab a new video frame and begin pulling opcodes from the audio and video streams - Grab frames from the input video using bmp2dhr if the .BIN file does not already exist. Run bmp2dhr in a background thread to not block encoding - move the output byte streaming from Video to Movie - For now, manually clip updates to pages > 56 since the client doesn't support them yet The way we encode video is now: - iterate in descending order over update_priority - begin a new (page, content) opcode - for all of the other offset bytes in that page, compute the error between the candidate content byte and the target content byte - iterate over offsets in order of increasing error and decreasing update_priority to fill out the remaining opcode
2019-03-07 23:07:24 +00:00
for _ in range(len(offsets), 4):
offsets.append(offsets[0])
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
yield (page + 32, content, offsets)
Video() is now aware of target frame rate, and will continue to emit opcodes until the cycle budget for the frame is exhausted. Output stream is also now aware of TCP framing, and schedules an ACK opcode every 2048 output bytes to instruct the client to perform TCP ACK and buffer management. Fixes several serious bugs in RLE encoding, including: - we were emitting the RLE opcode with the next content byte after the run completed! - we were looking at the wrong field for the start offset! - handle the case where the entire page is a single run - stop trying to allow accumulating error when RLE -- this does not respect the Apple II colour encoding, i.e. may introduce colour fringing. - also because of this we're unlikely to actually be able to find many runs because odd and even columns are encoded differently. In a followup we should start encoding odd and even columns separately Optimize after profiling -- encoder is now about 2x faster Add tests.
2019-02-23 23:52:25 +00:00
Read video frame rate and encode a new frame when the cycle count has ticked past the appropriate time. - optimize the frame encoding a bit - use int64 consistently to avoid casting Fix a bug - when retiring an offset, also update our memory map with the new content, oops If we run out of changes to index, keep emitting stores for content at page=32,offset=0 forever Switch to a weighted D-L implementation so we can weight e.g. different substitutions differently (e.g. weighting diffs to/from black pixels differently than color errors)
2019-03-10 22:42:31 +00:00
# If we run out of things to do, pad forever
content = target.page_offset[(0, 0)]
while True:
yield (32, content, [0, 0, 0, 0])
Move edit distance functions into separate module and clean up partially. Slight optimization to not heapppush() many times, instead build a regular list and then heapify.
2019-03-14 22:32:52 +00:00
@staticmethod
def _diff_weights(
source: screen.MemoryMap,
target: screen.MemoryMap
):
Add some docstrings Clean up naming in edit_distance In video encoder, when we emit additional offsets as part of an opcode, reinsert back into the priority heapq if the new edit distance is nonzero, in case we get the chance to fix it up later in the frame. Also make sure to zero out the diff_weights and content_deltas so we don't consider the offset again as a side-effect of some other opcode. Instead of prioritizing side-effect offsets by their previous update priority, prioritize by those with the lowest (error - edit) delta i.e. not introducing too much error relative to their edit distance.
2019-03-21 22:56:45 +00:00
return edit_distance.screen_edit_distance(
Move edit distance functions into separate module and clean up partially. Slight optimization to not heapppush() many times, instead build a regular list and then heapify.
2019-03-14 22:32:52 +00:00
source.page_offset, target.page_offset)
def _heapify_priorities(self) -> List:
priorities = []
it = np.nditer(self.update_priority, flags=['multi_index'])
while not it.finished:
priority = it[0]
if not priority:
it.iternext()
continue
page, offset = it.multi_index
# Don't use deterministic order for page, offset
nonce = random.random()
priorities.append((-priority, nonce, page, offset))
it.iternext()
heapq.heapify(priorities)
return priorities
@staticmethod
def _compute_delta(content, target, old):
"""
This function is the critical path for the video encoding.
"""
Add some docstrings Clean up naming in edit_distance In video encoder, when we emit additional offsets as part of an opcode, reinsert back into the priority heapq if the new edit distance is nonzero, in case we get the chance to fix it up later in the frame. Also make sure to zero out the diff_weights and content_deltas so we don't consider the offset again as a side-effect of some other opcode. Instead of prioritizing side-effect offsets by their previous update priority, prioritize by those with the lowest (error - edit) delta i.e. not introducing too much error relative to their edit distance.
2019-03-21 22:56:45 +00:00
return edit_distance.byte_screen_error_distance(content, target) - old
Move edit distance functions into separate module and clean up partially. Slight optimization to not heapppush() many times, instead build a regular list and then heapify.
2019-03-14 22:32:52 +00:00
_OFFSETS = np.arange(256)
def _compute_error(self, page, content, target, old_error, content_deltas):
offsets = []
Add some docstrings Clean up naming in edit_distance In video encoder, when we emit additional offsets as part of an opcode, reinsert back into the priority heapq if the new edit distance is nonzero, in case we get the chance to fix it up later in the frame. Also make sure to zero out the diff_weights and content_deltas so we don't consider the offset again as a side-effect of some other opcode. Instead of prioritizing side-effect offsets by their previous update priority, prioritize by those with the lowest (error - edit) delta i.e. not introducing too much error relative to their edit distance.
2019-03-21 22:56:45 +00:00
# TODO: move this up into parent
Move edit distance functions into separate module and clean up partially. Slight optimization to not heapppush() many times, instead build a regular list and then heapify.
2019-03-14 22:32:52 +00:00
delta_screen = content_deltas.get(content)
if delta_screen is None:
delta_screen = self._compute_delta(
content, target.page_offset, old_error)
content_deltas[content] = delta_screen
delta_page = delta_screen[page]
cond = delta_page < 0
candidate_offsets = self._OFFSETS[cond]
Add some docstrings Clean up naming in edit_distance In video encoder, when we emit additional offsets as part of an opcode, reinsert back into the priority heapq if the new edit distance is nonzero, in case we get the chance to fix it up later in the frame. Also make sure to zero out the diff_weights and content_deltas so we don't consider the offset again as a side-effect of some other opcode. Instead of prioritizing side-effect offsets by their previous update priority, prioritize by those with the lowest (error - edit) delta i.e. not introducing too much error relative to their edit distance.
2019-03-21 22:56:45 +00:00
priorities = delta_page[cond]
Move edit distance functions into separate module and clean up partially. Slight optimization to not heapppush() many times, instead build a regular list and then heapify.
2019-03-14 22:32:52 +00:00
l = [
Add some docstrings Clean up naming in edit_distance In video encoder, when we emit additional offsets as part of an opcode, reinsert back into the priority heapq if the new edit distance is nonzero, in case we get the chance to fix it up later in the frame. Also make sure to zero out the diff_weights and content_deltas so we don't consider the offset again as a side-effect of some other opcode. Instead of prioritizing side-effect offsets by their previous update priority, prioritize by those with the lowest (error - edit) delta i.e. not introducing too much error relative to their edit distance.
2019-03-21 22:56:45 +00:00
(priorities[i], random.random(), candidate_offsets[i])
Move edit distance functions into separate module and clean up partially. Slight optimization to not heapppush() many times, instead build a regular list and then heapify.
2019-03-14 22:32:52 +00:00
for i in range(len(candidate_offsets))
]
heapq.heapify(l)
while l:
Add some docstrings Clean up naming in edit_distance In video encoder, when we emit additional offsets as part of an opcode, reinsert back into the priority heapq if the new edit distance is nonzero, in case we get the chance to fix it up later in the frame. Also make sure to zero out the diff_weights and content_deltas so we don't consider the offset again as a side-effect of some other opcode. Instead of prioritizing side-effect offsets by their previous update priority, prioritize by those with the lowest (error - edit) delta i.e. not introducing too much error relative to their edit distance.
2019-03-21 22:56:45 +00:00
_, _, o = heapq.heappop(l)
offsets.append(o)
# Make sure we don't end up considering this (page, offset) again
# until the next image frame. Even if a better match comes along,
# it's probably better to fix up some other byte.
for cd in content_deltas.values():
cd[page, o] = 0
Move edit distance functions into separate module and clean up partially. Slight optimization to not heapppush() many times, instead build a regular list and then heapify.
2019-03-14 22:32:52 +00:00
if len(offsets) == 3:
break
return offsets
Reference in New Issue Copy Permalink