mirror of
https://github.com/irmen/prog8.git
synced 2024-08-20 05:29:02 +00:00
3.8 KiB
3.8 KiB
Just a few remarks upfront:
- There is the (gradle/IDEA) module
parser
: that's the parser generated by ANTLR4, in Java. The only file to be edited here is the grammar,prog8.g4
. - Then we have the module
compilerAst
- in Kotlin - which usesparser
and adds AST nodes. Here we put our additions to the generated thing, including any tests of the parsing stage.- the name is a bit misleading, as this module isn't (or, resp. shouldn't be; see below) about compiling, only the parsing stage
- also, the tree that comes out isn't much of an abstraction, but rather still more or less a parse tree (this might very well change).
- However, let's not yet rename the module. We'll find a good name during refactoring.
Problems with compilerAst
:
ModuleImporter.kt
, doing (Prog8-) module resolution. That's not the parser's job.ParsingFailedError
(inModuleParsing.kt
): this exception (it is actually not ajava.lang.Error
...) is thrown in a number of places, where other exceptions would make more sense. For example: not finding a file should just yield aNoSuchFileException
, not this one. The other problem with it is that it does not provide any additional information about the source of parsing error, in particular aPosition
.- During parsing, character literals are turned into UBYTEs (since there is no basic type e.g. CHAR). That's bad because it depends on a specific character encoding (
IStringEncoding
incompilerAst/src/prog8/ast/AstToplevel.kt
) of/for some target platform. Note that strings are indeed encoded later, in thecompiler
module. - The same argument applies to
IMemSizer
, and - not entirely sure about that -IBuiltinFunctions
.
Steps to take, in conceptual (!) order:
- introduce an abstraction
SourceCode
that encapsulates the origin and actual loading of Prog8 source code- from the local file system (use case: user programs)
- from resources (prog8lib)
- from plain strings (for testing)
- add subclass
ParseError : ParsingFailedError
which adds information about the source of parsing error (SourceCode
andPosition
). We cannot just replaceParsingFailedError
right away because it is so widely used (even in thecompiler
module). Therefore we'll just subclass for the time being, add more and more tests requiring the new one to be thrown (or, resp., NOT to be thrown), and gradually transition. - introduce a minimal interface to the outside, input:
SourceCode
, output: a tree with aModule
node as the root- this will be the Kotlin singleton
Prog8Parser
with the main methodparseModule
- plus, optionally, method's for registering/unregistering a listener with the parser
- the only exception ever thrown / reported to listeners (TBD) will be
ParseError
- anything related to the lexer, error strategies, character/token streams is hidden from the outside
- to make a clear distinction between the generated parser (and lexer) vs.
Prog8Parser
, and to discourage directly using the generated stuff, we'll rename the existingprog8Parser
/prog8Lexer
toProg8ANTLRParser
andProg8ANTLRLexer
and move them to packageprog8.parser.generated
- this will be the Kotlin singleton
- introduce AST node
CharLiteral
and keep them until after identifier resolution and type checking; insert there an AST transformation step that turns them in UBYTE constants (literals) - remove uses of
IStringEncoding
from modulecompilerAst
- none should be necessary anymore - move
IStringEncoding
to modulecompiler
- same with
ModuleImporter
, then rewrite that (addressing #46) - refactor AST nodes and grammar: less generated parse tree nodes (
XyzContext
), less intermediary stuff (private classes inAntrl2Kotlin.kt
), more compact code. Also: nicer names such as simplyStringLiteral
instead ofStringLiteralValue
- re-think
IStringEncoding
to address #38