Previously, we used the default character encoding from the project
properties to determine how strings and character constants in the
entire source file should be encoded. Now we switch between
encodings as needed. The default character encoding is no longer
relevant.
High ASCII is now an actual encoding, rather than acting like ASCII
that sometimes doesn't work. Because we can do high ASCII character
operands with "| $80", we don't output a .enc to switch from ASCII
to high ASCII unless we need to generate a string. (If we're
already in high ASCII mode, the "| $80" isn't required but won't
hurt anything.)
We now do a scan up front to see if ASCII or high ASCII is needed,
and only output the .cdefs for the encodings that are actually used.
The only gap in the matrix is high ASCII DCI strings -- the ".shift"
pseudo-op rejects text if the string doesn't start with the high
bit clear.
The documentation for 64tass says you're required to pass "--ascii"
when the source file is ASCII (as opposed to PETSCII). We were
ignoring this, but it turns out that everything works a bit better
if we don't.
So we now pass "--ascii" on the command line, and add a two-line
character encoding definition to every file that is generated with
ASCII as the default encoding. The sg_petscii and sg_screen
encodings go away, as PETSCII is now the default, and we can use the
built-in "screen" encoding.
We weren't checking to see if character operands matched their
delimiters, so bad code like "LDA #'''" was being generated.
There wasn't a test for this in 2006-operand-formats, so the test
has been updated with single and double quotes in low and high ASCII.
The previous code output a character in single-quotes if it was
standard ASCII, double-quotes if high ASCII, or hex if it was neither
of those. If a flag was set, high ASCII would also be output as
hex.
The new system takes the character value and an encoding identifier.
The identifier selects the character converter and delimiter
pattern, and puts the two together to generate the operand.
While doing this I realized that I could trivially support high
ASCII character arguments in all assemblers by setting the delimiter
pattern to "'#' | $80".
In FormatDescriptor, I had previously renamed the "Ascii" sub-type
"LowAscii" so it wouldn't be confused, but I dislike filling the
project file with "LowAscii" when "Ascii" is more accurate and less
confusing. So I switched it back, and we now check the project
file version number when deciding what to do with an ASCII item.
The CharEncoding tests/converters were also renamed.
Moved the default delimiter patterns to the string table.
Widened the delimiter pattern input fields slightly. Added a read-
only TextBox with assorted non-typewriter quotes and things so
people have something to copy text from.
Most tests pass, but 2007-labels-and-symbols fails because the
expressions recognized by 64tass don't match up with either of the
other assemblers.
This is currently using a workaround for the local label syntax.
64tass uses '_' as the prefix, which is unfortunate since SourceGen
explicitly allowed underscores in labels. (So does 64tass for that
matter, but it treats labels specially when the '_' comes first.)
We will need to rename any non-local user labels that start with '_'.
(issue #16)
