Update to describe NTSC emulation
This commit is contained in:
kris
parent
e0b732cdaa
commit
d5bd173345
121
README.md
121
README.md
|
|
@ -28,11 +28,12 @@ Then, to convert an image, the simplest usage is:
|
|||
python convert.py <input> <output.dhr>
|
||||
```
|
||||
|
||||
By default preview image will be shown after conversion, and saved as `<output>_prefix.png`, and `<output.dhr>` contains the double-hires image data in a form suitable for transfer to an Apple II disk image.
|
||||
The 16k output consists of 8k MAIN data first, 8K AUX data second. i.e. if loaded at 0x2000, the contents of 0x4000.0x5fff should be moved to 0x2000..0x3fff in AUX memory.
|
||||
By default preview image will be shown after conversion, and saved as `<output>-preview.png`, and `<output.dhr>` contains the double-hires image data in a form suitable for transfer to an Apple II disk image. The 16k output consists of 8k AUX data first, 8K MAIN data second (this matches the output format of other DHGR image converters). i.e. if loaded at 0x2000, the contents of 0x2000..0x3fff should be moved to 0x4000..0x5fff in AUX memory, and the image can be viewed on DHGR page 2.
|
||||
|
||||
For other available options, use `python convert.py --help`
|
||||
|
||||
TODO: document flags
|
||||
|
||||
## Examples
|
||||
|
||||
See [here](examples/gallery.md) for more sample image conversions.
|
||||
|
|
@ -61,7 +62,14 @@ This is a screenshot taken from OpenEmulator when viewing the Double Hi-res imag
|
|||
|
||||
|
||||
|
||||
Some difference in colour tone is visible due to blending of colours across pixels (e.g. brown blending into grey, in the background). This is due to NTSC chroma subsampling, which OpenEmulator implements but which ][-pix does not yet account for.
|
||||
Some difference in colour tone is visible due to blending of colours across pixels (e.g. brown blending into grey, in the background). This is due to the fact that OpenEmulator simulates the reduced chroma bandwidth of the NTSC signal.
|
||||
|
||||
][-pix also allows modeling this NTSC signal behaviour, which effectively allows access to more than 16 DHGR colours, through carefully chosen sequences of pixels (see below for more details). The resulting images have much higher quality, but only when viewed on a suitable target (e.g. OpenEmulator, or real hardware). On other targets the colour balance tends to be skewed, though image detail is still good.
|
||||
|
||||
This is an OpenEmulator screenshot of the same image converted with `--resolution=ntsc` instead of `--resolution=560`. Colour match to the original is substantially improved, and more colour detail is visible, e.g. in the shading of the background.
|
||||
|
||||
|
||||
|
||||
|
||||
## Some background on Apple II Double Hi-Res graphics
|
||||
|
||||
|
|
@ -69,21 +77,24 @@ Like other (pre-//gs) Apple II graphics modes, Double Hi-Res relies on [NTSC Art
|
|||
|
||||
In Double Hi-Res mode, the 560 horizontal pixels per line are individually addressable. This is an improvement over the (single) Hi-Res mode, which also has 560 horizontal pixels, but which can only be addressed in groups of two (with an option to shift blocks of 7 pixels each by one dot). See _Assembly Lines: The Complete Book_ (Wagner) for a detailed introduction to this, or _Understanding the Apple IIe_ (Sather) for a deep technical discussion.
|
||||
|
||||
Double Hi-Res is capable of producing 16 display colours, but with heavy restrictions on how these colours can be arranged horizontally.
|
||||
Double Hi-Res is usually characterized as being capable of producing 16 display colours, but with heavy restrictions on how these colours can be arranged horizontally.
|
||||
|
||||
One simple model is to only treat the display in groups of 4 horizontal pixels, which gives an effective resolution of 140x192 in 16 colours. These 140 pixel colours can be chosen independently, but they exhibit (sometimes severe) interference/fringing effects when two colours are next to one another. This is the approach used by the [bmp2dhr](http://www.appleoldies.ca/bmp2dhr/) image converter.
|
||||
### Naive model: 140x192x16
|
||||
|
||||
One simple model for Double Hi-Res graphics is to only treat the display in groups of 4 horizontal pixels, which gives an effective resolution of 140x192 in 16 colours (=2^4). These 140 pixel colours can be chosen independently, which makes this model easy to think about and to work with (e.g. when creating images by hand). However the resulting images will exhibit (sometimes severe) colour interference/fringing effects when two colours are next to one another, because the underlying hardware does not actually work this way. See below for an example image conversion, showing the unwanted colour fringing that results.
|
||||
|
||||
### Simplest realistic model: 560 pixels, 4-pixel colour
|
||||
|
||||
A more complete model for thinking about DHGR comes from looking at how the NTSC signal produces colour on the display.
|
||||
The [NTSC colour-burst signal](https://en.wikipedia.org/wiki/Colorburst) completes one complete phase in the time taken to draw 4 horizontal dots. The colours produced are due to the interactions of the pixel luminosity (on/off) relative to this NTSC phase.
|
||||
The [NTSC chrominance subcarrier](https://en.wikipedia.org/wiki/Chrominance_subcarrier) completes one complete phase cycle in the time taken to draw 4 horizontal dots. The colours produced are due to the interactions of the pixel luminosity (on/off) relative to this NTSC chroma phase.
|
||||
|
||||
What this means is that the colour of each of the 560 horizontal pixels is determined by the current pixel value (on/off), the current X-coordinate modulo 4 (X coordinate relative to NTSC phase), as well as the on-off status of the pixels to the left of it.
|
||||
|
||||
The simplest approximation is to only look at the current pixel value and the 3 pixels to the left, i.e. to consider a sliding window of 4 horizontal pixels moving across the screen from left to right. Within this window, we have one pixel for each of the 4 values of NTSC phase (x % 4, ranging from 0 .. 3). The
|
||||
on-off values for these 4 values of NTSC phase determine the colour of the pixel. See [here](https://docs.google.com/presentation/d/1_eqBknG-4-llQw3oAOmPO3FlawUeWCeRPYpr_mh2iRU/edit) for more details.
|
||||
The simplest approximation is to only look at the current pixel value and the 3 pixels to the left, i.e. to consider a sliding window of 4 horizontal pixels moving across the screen from left to right. Within this window, we have one pixel for each of the 4 values of NTSC phase (x % 4, ranging from 0 .. 3). The on-off values for these 4 values of NTSC phase determine the colour of the pixel. See [here](https://docs.google.com/presentation/d/1_eqBknG-4-llQw3oAOmPO3FlawUeWCeRPYpr_mh2iRU/edit) for more details.
|
||||
|
||||
This model allows us to understand and predict the interference behaviour when two "140px" colours are next to each other, and to go beyond this "140px" model to take maximum advantage of the 560px horizontal resolution.
|
||||
This model allows us to understand and predict the interference behaviour when two "140px" colours are next to each other, and to go beyond this "140px" model to take more advantage of the true 560px horizontal resolution.
|
||||
|
||||
If we imagine drawing pixels from left to right across the screen, at each pixel we only have *two* accessible choices of colour: those resulting from turning the current pixel on, or off. Which of these two colours we can obtain are determined by the pixels already drawn to the left (the immediate 3 neighbours, in our model). One of these will always be the same as the pixel colour to the left (the on/off pixel choice corresponding to the value that just "fell off the left side" of the sliding window), and the other choice is some other colour from our palette of 16.
|
||||
If we imagine drawing pixels from left to right across the screen, at each pixel we only have *two* accessible choices of colour: those resulting from turning the current pixel on, or off. Which two particular colours are produced are determined by the pixels already drawn to the left (the immediate 3 neighbours, in our model). One of these possibilities will always be the same as the pixel colour to the left (the on/off pixel choice corresponding to the value that just "fell off the left side" of the sliding window), and the other choice is some other colour from our palette of 16.
|
||||
|
||||
This can be summarized in a chart, showing the possible colour transitions depending on the colour of the pixel to the immediate left, and the value of x%4.
|
||||
|
||||
|
|
@ -93,22 +104,59 @@ So, if we want to transition from one colour to a particular new colour, it may
|
|||
|
||||
These constraints are difficult to work with when constructing DHGR graphics "by hand", but we can account for them programmatically in our image conversion to take full advantage of the "true" 560px resolution while accounting for colour interference effects.
|
||||
|
||||
### Limitations of this colour model
|
||||
#### Limitations of this colour model
|
||||
|
||||
In practise the above description of the Apple II colour model is only a discrete approximation. On real hardware, the video signal is a continuous analogue signal, and colour is continuously modulated rather than producing discrete coloured pixels with fixed colour values.
|
||||
In practise the above description of the Apple II colour model is still only an approximation. On real hardware, the video signal is a continuous analogue signal, and colour is continuously modulated rather than producing discretely-coloured pixels with fixed colour values.
|
||||
|
||||
Furthermore, in an NTSC video signal the colour (chroma) signal has a lower bandwidth than the luma (brightness) signal ([Chroma sub-sampling](https://en.wikipedia.org/wiki/Chroma_subsampling)),
|
||||
which means that colours will tend to bleed across multiple pixels. i.e. the colour of a pixel is determined by that of more than its immediate neighbour to the left. A more accurate model of double-hires colour would take this into account.
|
||||
More importantly, in an NTSC video signal the colour (chroma) signal has a lower bandwidth than the luma (brightness) signal ([Chroma sub-sampling](https://en.wikipedia.org/wiki/Chroma_subsampling)), which means that colours will tend to bleed across more than 4 pixels. However our simple "4-pixel chroma bleed" model already produces good results, and exactly matches the implementation behaviour of some emulators, e.g. Virtual II.
|
||||
|
||||
This means that images produced by ][-pix do not always quite match colours produced on real hardware (or high-fidelity emulators, like OpenEmulator) due to this colour bleeding effect. In effect, the set of available colours is larger (since adjacent colours blend together), but the effective resolution is lower (since each colour bleeds over multiple pixels). In principle, it would be possible to simulate the NTSC video signal more completely to account for this during image processing.
|
||||
### NTSC emulation and 8-pixel colour
|
||||
|
||||
However our simple model already produces good results, and exactly matches the behaviour of some emulators, e.g. Virtual II.
|
||||
By simulating the NTSC (Y'UV) signal directly we are able to recover the Apple II colour output from "first principles". Here are the 16 "basic" DHGR colours, obtained using saturation/hue parameters tuned to match OpenEmulator's NTSC implementation, and allowing chroma to bleed across 4 pixels.
|
||||
|
||||
|
||||
|
||||
However in real NTSC, chroma bleeds over more than 4 pixels, which means that we actually have more than 2^4 colours available to work with.
|
||||
|
||||
**When viewed on a composite colour display, Double Hi-Res graphics is not just a 16-colour graphics mode!**
|
||||
|
||||
If we allow the NTSC chroma signal to bleed over 8 pixels instead of 4, then the resulting colour is determined by sequences of 8 pixels instead of 4 pixels, i.e. there are 2^8 = 256 possibilities. In practise many of these result in the same output colour, and (with this approximation) there are only 85 unique colours available. However this is still a marked improvement on the 16 "basic" DHGR colours:
|
||||
|
||||
|
||||
|
||||
The "extra" DHGR colours are only available on real hardware, or an emulator that implements NTSC chroma sub-sampling (such as OpenEmulator). But the result is that on such targets a much larger range of colours is available for use in image conversion. However the restriction still exists that any given pixel only has a choice of 2 colours available (as determined by the on/off state of pixels to the left).
|
||||
|
||||
In practise this gives much better image quality, especially when shading areas of similar colour. The Apple II is still unable to directly modulate the luma (brightness) NTSC signal component, so areas of low or high brightness still tend to be heavily dithered. This is because there are more bit sequences that have the number of '1' bits close to the average than there are at the extremes, so there are correspondingly few available colours that are very bright or very dark.
|
||||
|
||||
These 85 unique double hi-res colours produced by the ][-pix NTSC emulation are not the definitive story - though they're closer to it than the usual story that double hi-res is a 16-colour graphics mode. The implementation used by ][-pix is the simplest one: the Y'UV signal is averaged with a sliding window of 4 pixels for the Y' (luma) component and 8 pixels for the UV (chroma) component.
|
||||
|
||||
The choice of 8 pixels is not strictly correct - e.g. the chroma bandwidth (~0.6MHz) is much less than half of luma bandwidth (~2Mhz) so the signal bleeds over more than twice as many pixels; but also decays in a more complex way than the simple step function sliding window chosen here. In practise using 8 pixels is a good compromise between ease of implementation, runtime performance and fidelity.
|
||||
|
||||
By contrast, OpenEmulator uses a more complex (and realistic) band-pass filtering to produce its colour output, which presumably allows even more possible colours (physical hardware will also produce its own unique results, depending on the hardware implementation of the signal decoding, and other physical characteristics). I expect that most of these will be small variations on the above though; and in practise the ][-pix NTSC implementation already produces a close colour match for the OpenEmulator behaviour.
|
||||
|
||||
#### Examples of NTSC images
|
||||
|
||||
(Source: [Reinhold Möller](https://commons.wikimedia.org/wiki/File:Nymphaea_caerulea-20091014-RM-115245.jpg), [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0), via Wikimedia Commons)
|
||||
|
||||
|
||||
|
||||
OpenEmulator screenshot of image produced with `--resolution=560 --palette=openemulator --lookahead=8`. The distorted background colour compared to the original is particularly noticeable.
|
||||
|
||||
|
||||
|
||||
OpenEmulator screenshot of image produced with `--resolution=ntsc --lookahead=8`. Not only is the background colour a much better match, the image shading and detail is markedly improved.
|
||||
|
||||
|
||||
|
||||
Rendering the same .dhr image with 4-pixel colour shows the reason for the difference. For example the background shading is due to pixel sequences that with this simpler (and less hardware-accurate) rendering scheme appear as sequences of grey and dark green, with a lot of blue and red sprinkled in. With NTSC these pixel sequences combine to produce various shades of green. Note also that the dark green (1 pixel set; low luma) is brightened by the grey (2 pixels set; medium luma) to produce a green of medium intensity.
|
||||
|
||||
|
||||
|
||||
# Dithering and Double Hi-Res
|
||||
|
||||
Dithering an image to produce an approximation with fewer image colours is a well-known technique. The basic idea is to pick a "best colour match" for a pixel from our limited palette, then to compute the difference between the true and selected colour values and diffuse this error to nearby pixels (using some pattern).
|
||||
|
||||
In the particular case of DHGR this algorithm runs into difficulties, because each pixel only has two possible colour choices (from a total of 16). If we only consider the two possibilities for the immediate next pixel then neither may be a particularly good match. However it may be more beneficial to make a suboptimal choice now (deliberately introduce more error), if it allows us access to a better colour for a subsequent pixel. "Classical" dithering algorithms do not account for these palette constraints, and produce suboptimal image quality for DHGR conversions.
|
||||
In the particular case of DHGR this algorithm runs into difficulties, because each pixel only has two possible colour choices (from a total of 16+). If we only consider the two possibilities for the immediate next pixel then neither may be a particularly good match. However it may be more beneficial to make a suboptimal choice now (deliberately introduce more error), if it allows us access to a better colour for a subsequent pixel. "Classical" dithering algorithms do not account for these palette constraints, and produce suboptimal image quality for DHGR conversions.
|
||||
|
||||
We can deal with this by looking ahead N pixels (6 by default) for each image position (x,y), and computing the effect of choosing all 2^N combinations of these N-pixel states on the dithered source image.
|
||||
|
||||
|
|
@ -157,7 +205,9 @@ Existing conversion tools (see below) tend to support a variety of RGB palette v
|
|||
|
||||
## Precomputing distance matrix
|
||||
|
||||
Computing the CIE2000 distance between two RGB colour values is fairly expensive, since the [formula](https://en.wikipedia.org/wiki/Color_difference#CIEDE2000) is complex. We deal with this by precomputing a matrix from all 256^3 integer RGB values to the 16 RGB values in a palette. This 256MB matrix is generated on disk by the `precompute_distance.py` utility, and is mmapped at runtime for efficient access.
|
||||
Computing the CIE2000 distance between two RGB colour values is fairly expensive, since the [formula](https://en.wikipedia.org/wiki/Color_difference#CIEDE2000) is complex. We deal with this by precomputing a matrix from all 256^3 integer RGB values to the 16 RGB values in a palette. This matrix is generated on disk by the `precompute_distance.py` utility, and is mmapped at runtime for efficient access.
|
||||
|
||||
For a 4-bit colour palette the file is 256MB; for the 8-bit NTSC colour palette it is 4GB! Image dithering is also correspondingly slower (especially if the file cannot be mmapped completely into memory and must be demand-paged).
|
||||
|
||||
# Comparison to other DHGR image converters
|
||||
|
||||
|
|
@ -177,7 +227,6 @@ Computing the CIE2000 distance between two RGB colour values is fairly expensive
|
|||
|
||||
* It does not perform RGB colour space conversions before dithering, i.e. if the input image is in sRGB colour space (as most digital images will be) then the dithering is also performed in sRGB. Since sRGB is not a linear colour space, the effect of dithering is to distribute errors non-linearly, which distorts the brightness of the resulting image.
|
||||
|
||||
|
||||
## a2bestpix
|
||||
|
||||
* Like ][-pix, [a2bestpix](http://lukazi.blogspot.com/2017/03/double-high-resolution-graphics-dhgr.html) only supports DHGR conversion. Overall quality is usually fairly good, although colours and brightness are slightly distorted (for reasons described below), and the generated preview images do not quite give a faithful representation of the native image rendering.
|
||||
|
|
@ -214,13 +263,20 @@ These three images were converted using the same target (openemulator) palette,
|
|||
|
||||
The following images were generated with a palette approximating OpenEmulator's colours (`--palette=openemulator` for ][-pix)
|
||||
|
||||
### ][-pix (Preview image)
|
||||
### ][-pix 4-pixel colour
|
||||
|
||||
Preview image and OpenEmulator screenshot
|
||||
|
||||
|
||||
|
||||
### ][-pix (OpenEmulator screenshot)
|
||||
|
||||
|
||||
### ][-pix NTSC 8-pixel colour (Preview image)
|
||||
|
||||
Preview image and OpenEmulator screenshot
|
||||
|
||||
|
||||
|
||||
|
||||
### bmp2dhr (OpenEmulator screenshot)
|
||||
|
||||
|
||||
|
|
@ -246,14 +302,17 @@ The following images were generated with a palette matching the one used by Virt
|
|||
|
||||
### ][-pix (Preview image)
|
||||
|
||||
|
||||
|
||||
|
||||
### ][-pix (Virtual II screenshot)
|
||||
|
||||
|
||||
|
||||
|
||||
### bmp2dhr
|
||||
|
||||
|
||||
|
||||
|
||||
The image is heavily impacted by colour fringing, which bmp2dhr does not account for at all. The difference in brightness of the groundhog's flank is also because bmp2dhr does not gamma-correct the image, so shadows/highlights tend to get blown out.
|
||||
|
|
@ -262,21 +321,29 @@ The image is heavily impacted by colour fringing, which bmp2dhr does not account
|
|||
|
||||
These next two images were generated with a palette approximating OpenEmulator's colours (`--palette=openemulator` for ][-pix), i.e. not the same image files as above.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
On OpenEmulator, which simulates NTSC chroma sub-sampling, the fringing is not pronounced but changes the colour balance of the image, e.g. creates a greenish tinge. Though it should be noted that the ][-pix image below also doesn't closely match the original image colours, because it does not account for the chroma blending across pixels.
|
||||
On OpenEmulator, which simulates NTSC chroma sub-sampling, the fringing is not pronounced but changes the colour balance of the image, e.g. creates a greenish tinge.
|
||||
|
||||
### ][-pix (OpenEmulator)
|
||||
### ][-pix, 4-pixel colour (OpenEmulator)
|
||||
|
||||
|
||||
|
||||
|
||||
Colour balance here is also slightly distorted due to not fully accounting for chroma blending.
|
||||
|
||||
### ][-pix, NTSC 8-pixel colour (OpenEmulator)
|
||||
|
||||
|
||||
|
||||
|
||||
Detail and colour balance is much improved.
|
||||
|
||||
# Future work
|
||||
|
||||
* Supporting lo-res and double lo-res graphics modes would be straightforward.
|
||||
|
||||
* Hi-res will require more care, since the 560 pixel display is not individually dot addressible. In particular the behaviour of the "palette bit" (which shifts a group of 7 dots to the right by 1) is another optimization constraint. In practise a similar lookahead algorithm should work well though.
|
||||
|
||||
* With more work to model the NTSC video signal it should be possible to produce images that better model the NTSC output (e.g. chroma subsampling). For example, I think it is still true that at each horizontal dot position there is a choice of two possible "output colours", but these are influenced by the previous pixel values in a more complex way and do not come from a fixed palette of 16 choices.
|
||||
|
||||
* I would like to be able to find an ordered dithering algorithm that works well for Apple II graphics. Ordered dithering specifically avoids diffusing errors arbitrarily across the image, which produces visual noise (and unnecessary deltas) when combined with animation. For example such a thing may work well with my II-Vision video streamer. However the properties of NTSC artifact colour seem to be in conflict with these requirements, i.e. pixel changes *always* propagate colour to some extent.
|
||||
|
|
|
|||
Loading…
Reference in New Issue