prog8/AGENTS.md

CRITICAL RULES - MUST FOLLOW

RESPONSES

• Keep responses concise and to the point - unless the user asks otherwise

PLANNING MODE

• Always ask clarifying questions
• Never assume design, tech stack or features
• Use deep-dive sub-agents to assist with research
• Use deep-dive sub-agents to review the different aspects of your plan before presenting to the user

CHANGE / EDIT MODE

• Use a subagent for larger chunks of work
• Identify changes from the plan that can be implemented in parallel, and use sub-agents to implement the features efficiently
• When using sub-agents to implement features, act as a coordinator only
• Use the best model for the task - premium models for complex tasks (like coding) and mid-tier models for simpler tasks, like documentation
• After completing features (large or small), run the relevant quality commands (lint, type check, build) to check code quality. Only compiler/standard-library changes require a build: that means Kotlin sources (.kt), the grammar (.g4), or embedded standard library files (compiler/res/prog8lib/**/*.p8 and **/*.asm). For changes limited to examples/**, docs/**, or other non-compiler files, verify with the target compiler directly instead (for example prog8c -target <target> <file>), and skip the build entirely (exception: if the change is only whitespace or code comments, a build or test run is not necessary)
• gradle build already runs the full test suite; don't run a separate full gradle test before it, to avoid lengthy double test runs (targeted --tests filters to find failing tests are fine)

Project Overview

• This project is a compiler for the Prog8 programming language.
• Prog8 is a programming language primarily targeting 8-bit retro systems with the 6502 CPU, such as the Commodore 64, Commodore 128, and Commander X16.
• The compiler has a 6502 code generator backend, and an IR code generator.
• The IR code is used by machine-specific code generator backends, including 6502 and M68K.
• The compiler includes a simple 'virtual machine' that can execute the IR code directly via interpretation.
• Prog8 source files have .p8 extension – these are not LUA or PICO-8 source files in this case!
• Prog8 source files are a "module" that can contain one or more "blocks". They can also import other modules, from internal library files or from source files on the filesystem.
• The prog8 compiler is written mostly in Kotlin, those files have the .kt extension.
• The standard library is mostly written in Prog8 and assembly code, and can be found in the "compiler" module, in the 'res/prog8lib' directory.
• Library Reference: For a quick overview of all available modules, routines, and their signatures, consult the skeleton files (symboldumps) in docs/source/_static/symboldumps/. They are named skeletons-<target>.txt.
• Kotlin version 2.4 is used for the compiler implementation.
• Java 17 is used as Java runtime version.
• ANTLR4 version 4.13 is used for the parser implementation.
• Dependent library versions can be found in 'build.gradle.kts' and in the IntelliJ IDEA configuration files in .idea/libraries
• The compiler main entrypoint is in the "compiler" module, in src/prog8/CompilerMain.kt
• Codebase knowledge graph: For a structured overview of modules, their dependencies, key entities, the end-to-end compilation pipeline, the standard library layout, and the test structure, consult docs/CODEBASE-KNOWLEDGE-GRAPH.md. Read it when you need to understand how the compiler fits together before diving into specific files.

Compilation Flow (High-Level)

Source → parseMainModule() → processAst() → optimizeAst() → postprocessAst()
       → Simple AST → Code Generator → Assembly → .prg

Key phases in compiler/src/prog8/compiler/Compiler.kt:

• parseMainModule() - Import modules, parse to Compiler AST
• processAst() - Semantic analysis, constant folding, type casting, validation
• optimizeAst() - Dead code elimination, inlining, statement optimization
• postprocessAst() - Memory layout, final transformations
• Code generation – Simple AST → IR or 6502 assembly

Understanding this order is important: For example, ConstantIdentifierReplacer runs during processAst(), so by the time AstChecker runs at the end of that phase, identifier references have already been replaced with their actual values.

Target Differences

• CPU instruction set differences: Only the CommanderX16 target (cx16) can use 65C02 instructions such as STZ. The other targets (C64, C128, PET32) can only use original 6502 instructions.
• When writing or understanding 6502/65C02 assembly code, load the asm6502-coder skill for 64tass syntax and conventions.
• When writing or understanding M68K assembly code (amiga500, amiga1200, qemu68k targets), load the m68k-coder skill for vasm mot syntax and conventions.
• M68K ordinary subroutine locals and parameters use A5-based stack frames. Normal calls pass word-packed argument slots on the stack (2-byte slots for byte/word args, 4-byte for wide args, byte values at the slot base), and the caller cleans up; see docs/source/technical.rst for the complete convention and frame eligibility rules.

DEBUGGING TIP: Use -noopt to isolate problems

When investigating a possible code generation problem (both IR and 6502), FIRST try compiling the program with the -noopt switch to disable most of the compiler optimizations.

• Problem gone with -noopt: Issue is in optimization phases (optimizeAst(), UnusedCodeRemover, Inliner, etc.)
• Problem persists with -noopt: Issue is in parsing, semantic analysis, symboltable, or the regular code generation path.

This way you can determine if the problem is caused by a faulty optimization step, or just occurs in the regular code generation path.

DEBUGGING TIP: Use -compareir to see what changed

When investigating optimization-related issues or tracking regressions:

# Compile without optimizations (baseline)
prog8c -target virtual -noopt -out dir program.p8

# Compile with optimizations and compare
prog8c -target virtual -compareir dir/program_noopt.p8ir program.p8

Output shows:

• Instruction/chunk/register count changes with percentages
• First 10 actual instruction differences
• Helps identify which optimization transformed the code

DEBUGGING TIP: Use -vmtrace to trace execution

When debugging VM execution or control flow issues:

prog8c -target virtual -vm program.p8ir -vmtrace
prog8c -target virtual -emu -vmtrace program.p8

Output format: [chunkName:instructionIndex] instruction

• Shows each executed IR instruction with location
• Useful for understanding control flow and finding where execution diverges
• Only works on virtual target

Typical debugging workflow:

1. Quick check: -check for syntax errors
2. Isolate: -noopt to determine if problem is optimizer-related
3. Compare: -compareir to see what instructions changed
4. Trace: -vmtrace to watch actual execution flow
5. Deep dive: -printast1 / -printast2 for compiler internals

IMPORTANT: SymbolTable cachedFlat cache

The SymbolTable class has a cached flat property for fast symbol lookups.

CRITICAL: If you modify the AST after SymbolTable construction, you MUST call symbolTable.resetCachedFlat() or lookups will fail!

IMPORTANT: Virtual Target (IR Codegen) Issues

For problems that ONLY occur with the 'virtual' target, ONLY modify these modules:

• codeGenIntermediate - IR code generator
• intermediate - IR representation and file I/O
• virtualmachine - VM that executes IR code

DO NOT modify compilerAst, simpleAst, codeGenCpu6502, etc. The IR codegen has its own separate handling for symbol tables, AST transformations, and unused code removal.

IMPORTANT: Compiler Crashes Must Be Fixed in the Compiler

A Kotlin/Java crash in the compiler itself (as opposed to a compilation error in the user's program) indicates a bug in the compiler code. Such crashes should NEVER be worked around by modifying the Prog8 source file that triggered them.

Instead, always fix the bug in the compiler's Kotlin code. The user's program is correct - the compiler needs to handle it properly.

Steps when encountering a compiler crash:

1. Create a minimal reproduction case of the Prog8 source that triggers the crash
2. Debug the compiler to find the root cause in the Kotlin code
3. Fix the compiler bug, not the user program

CRITICAL: NO FORMATTING

• DO NOT change indentation and formatting of lines that are not being modified. NEVER run formatters (black, ruff, prettier, etc.) after edits,
• .editorconfig handles basic formatting (indentation, line endings, whitespace)
• Make ONLY the requested changes, touch nothing else
• NO EMOJI in user documentation. Do not use emoji or decorative unicode symbols in documentation files. Functional unicode symbols are acceptable when they serve a clear purpose (e.g., → for arrows, ± for plus-minus, × for multiplication, ° for degrees). Avoid decorative emoji like ❌ ✅ ⚠️ 🎉 etc.

Code Style Guidelines

Code comments: Comments are the exception, not the norm. A comment earns its place only by recording something the code itself cannot say: the reason for a non-obvious choice, or a constraint that a well-meaning future edit would otherwise violate.

• Inline comment inside a function body: 1 line, never more.
• KDoc on a declaration: 3 lines, unless a genuine non-obvious contract cannot be stated shorter.
• Never restate the identifier, the type, or what the following line does.
• Prefer a good name or a well-named function over a comment that explains it.

Before keeping any comment, delete it mentally and ask what the reader loses. If nothing, it goes.

Keep existing comments as they are; do not expand, reword, or reformat them while editing nearby lines - unless that would cause inconsistency.

Kotlin string $ handling: The $ character is special in Kotlin string literals (string interpolation). If you need a literal $ character, use Kotlin's $$"..." interpolation prefix instead of backslash escaping (\$) or ${'$'}. The $$"..." syntax is a fairly recent Kotlin feature that avoids nasty escaping. For example:

val bill = $$"The value is $3.99"

Prog8 language information

When the task involves writing or understanding .p8 (Prog8 source) or .p8ir (Prog8 intermediate representation) files, load the prog8-coder skill — it contains the full language reference, syntax rules, and standard library guidance.

Project Module Descriptions

• compiler - Main compiler entrypoint (src/prog8/CompilerMain.kt)
• compilerAst - Complex AST where most optimizations run
• simpleAst - Simplified AST used by code generator backends
• parser - ANTLR4 parser implementation
• codeCore - Core code generation utilities
• codeGenCpu6502 - 6502 assembly code generator backend
• codeGenIntermediate - Intermediate representation (IR) code generator
• codeGenVirtual - Virtual machine code generator backend
• codeOptimizers - Optimization passes
• intermediate - IR components
• virtualmachine - VM that executes IR code
• languageServer - Language Server Protocol implementation
• docs - Documentation files
• examples - Example Prog8 programs

Key Information

• never read the files and directories that are ignored via the .aiignore and .gitignore files (EXCEPT when it is a folder for agent config such as .junie, .opencode, .qwen or .agent)
• never perform any git source control write/update/add/commit/branch operations. Read and status operations are allowed.
• git log/history queries can be useful for understanding when/why a feature was added or tracking down when a bug was introduced, but for locating code use grep_search or glob instead.
• Architecture decisions: separation of frontend/parser, IR intermediate representation, multiple backends

Dev environment tips

Development Workflows

1. Testing your own Prog8 programs (no compiler changes):

• No rebuild needed - just run prog8c directly
• Edit your .p8 file → compile/run → check stdout output
• Example: prog8c -target virtual -emu myprogram.p8

2. Testing compiler or standard library changes:

• Rebuild required after every change to .kt, .p8, or .asm files in the compiler project
• Workflow: edit source → gradle installdist installshadowdist → test with prog8c
• Example: fix bug in CodeGen6502.kt → rebuild → prog8c -target cx16 test.p8

Building and Installing the Compiler

• use the system installed gradle command instead of the gradle wrapper.

Quick compile check (NO tests)

• After changing compiler Kotlin source (.kt files): Use gradle :compiler:compileKotlin --console=plain to quickly check for syntax/compile errors
• This compiles the compiler but skips running tests, this is much faster than gradle build
• Note: This does NOT install the compiler - use gradle installdist installshadowdist --console=plain after to actually use your changes

When you MUST rebuild AND reinstall the compiler

After ANY change to:

1. Kotlin compiler source code (.kt files in any module, or the .g4 grammar file)
2. Standard library Prog8 files (compiler/res/prog8lib/**/*.p8)
3. Standard library assembly files (compiler/res/prog8lib/**/*.asm)

Run: gradle installdist installshadowdist --console=plain

Why: The standard library files (.p8 and .asm) are embedded into the compiler JAR during the build. Changes to these files are NOT picked up by simply running prog8c - you MUST rebuild and reinstall for your changes to take effect.

Without this step, your changes will NOT be reflected when running prog8c!

• This compiles AND installs, but still skips running tests (faster than gradle build)

When to run full build with tests

• Before final verification: Run gradle build to ensure all tests pass - but only if the change actually touched the compiler or standard library (.kt, .g4, or compiler/res/prog8lib/**). For examples/** or docs/**-only changes, verify with prog8c directly instead and skip the build
• After major refactoring: Run gradle build to catch regressions
• When debugging test failures: Use gradle test --tests "*TestName*" for specific tests
• When working on a standalone code gen module (one that reads .p8ir files): Run gradle :<module>:test --console=plain instead of full gradle build. Compiler unit tests are irrelevant for IR-level changes. Do run full gradle build before final verification if the change touched compiler sources.

Build command summary

Command	When to use	Time
gradle :compiler:compileKotlin --console=plain	Quick syntax check (no tests)	~10-20s
gradle installdist installshadowdist --console=plain	After compiler changes (to use them)	~10-20s
gradle build --console=plain	Before commits, after major changes	~45-60s
gradle test --tests "*Name*" --console=plain	Run specific tests	~5-30s

Note: Use system gradle command, not wrapper. Run gradle clean if you suspect stale artifacts.

Using the Compiler (prog8c)

• the prog8c compiler executable can be found in the compiler/build/install/prog8c/bin folder (this is already added to the shell's path)
• the -check switch performs a quick syntax/semantic check only; it will NOT produce any output files (no .prg, .asm, etc.). Use it only for fast error checking during development.
• the -noopt switch DISABLES all optimizations - useful for debugging to determine if a problem is caused by the optimizer. Optimizations are ENABLED by default (no flag needed).
• prog8c options accept single-dash or double-dash (e.g., -target and --target are equivalent).
• the -printast1 switch prints out the internal Compiler AST after parsing and semantic analysis.
• the -printast2 switch prints out the optimized Simple AST just before it goes to the code generator. This is useful for debugging optimizer issues.
• the -out outdir switch sets an alternative output directory for compiled files (.prg, .asm, .list, etc.). By default, output files are written to the current working directory.
• the -o <file> switch names the exact final program artifact (also spelled --output); all other output files go in the same directory with the same stem. Cannot be combined with -out or with multiple source files.
• Other useful flags: -quiet (suppress messages), -warnimplicitcasts (warn on implicit type widening), -daemon (keep a background compiler process alive — must be passed on every invocation)

Compilation Output Files

Names follow the source file name, or the stem of the -o file when that option is used.

• *.prg - The final compiled program file for the target system (e.g., Commander X16)
• *.asm - Generated assembly code from the Prog8 source
• *.list - Generated full assembly listing file from the Prog8 source
• *.p8ir - Intermediate representation file, can be executed in the Virtual Machine
• *.vice-mon-list - Vice emulator monitor list file for debugging

Execution Examples

• prog8c -target targetname input.p8 - Compile a Prog8 source file "input.p8" for the given target (cx16, c64, pet32, c128, virtual)
• prog8c -target targetname -emu input.p8 - Compile and execute a prog8 file in the emulator for the given target (cx16, c64, pet32, c128, virtual)
• prog8c -vm input.p8ir - Execute an existing prog8 program, compiled in IR form, in the Virtual Machine

Library Tools

• Library search: prog8c -libsearch <regex> — search for a regex pattern in the embedded stdlib files. Extremely useful to quickly find library routines, variables, or signatures (e.g., prog8c -libsearch "txt\." lists all textio routines)
• Library dump: prog8c -libdump <dir> — extract all embedded library source files into a directory for direct inspection

Testing and Verification

• Never assume your changes simply work, always test!
• If the project does not have any testing tools, scripts, MCP tools, skills, etc. available for testing, ask the user whether testing should be skipped.
• in prog8 code being tested, use '@shared' variables to avoid const-folding

Automated Tests (gradle)

Always use the --console=plain flag with these commands to avoid messing up the agent's text interface.

• gradle build --console=plain - Full build including all tests (about 50s)
• gradle :compiler:test --console=plain - Run only compiler tests (faster)
• gradle :compiler:test --tests "prog8tests.compiler.TestOptimization" --console=plain - Run specific test class

Note: By default, Gradle only shows failed tests. Passed and skipped tests are silent.

A full gradle build is CPU- and IO-heavy; on Unix-like systems prefix it with nice (e.g. nice gradle build --console=plain) to lower its priority.

⚠️ CRITICAL: Test Filtering Patterns - Read This First!

Gradle's --tests filter has strict rules that are easy to get wrong:

Pattern	Example	Works?
Full class name	--tests "prog8tests.compiler.TestOptimization"	USE THIS
Wildcard at END	--tests "prog8tests.compiler.Test*"	Yes
Wildcard on package	--tests "prog8tests.compiler.*"	Yes
Wildcard at START	--tests "*TestOptimization"	FAILS
Test description	--tests "*Optimization*inline*"	FAILS

Why the restrictions? Gradle's --tests filter matches fully qualified class names only. Wildcards work as suffixes (e.g., Test*) but NOT as prefixes (e.g., *Test). KoTest test names like "inline multi-value void" are descriptions, not method names, so you cannot filter by them.

To find failing tests after a run:

# Check XML results (reliable)
grep "<failure" compiler/build/test-results/test/*.xml

# Or check HTML report (most reliable)
cat compiler/build/reports/tests/test/index.html

# For test compilation errors
gradle :compiler:compileTestKotlin --info 2>&1 | grep "^e:"

Additional notes:

• Unit tests use KoTest (FunSpec style)
• Tests are in the "compiler" module's test directory (and some other modules)
• Test config is in root build.gradle.kts; tests run in parallel
• When writing test programs, add at the top: %zeropage basicsafe and %option no_sysinit - the latter ONLY when NOT writing for the amiga500 or qemu68k targets
• When a test fails, the output shows "There were failing tests. See the report at:" - read that HTML report

TestCompilerOnExamples expected sizes

TestCompilerOnExamples asserts exact output sizes per example. A code-generator change shifts them, so expect a handful of expected:<N> but was:<M> failures with no stack trace pointing at the cause. Attribute each one before editing: rebuild with the change reverted and compare the two generated .asm files, so the size delta is demonstrably yours and not a side effect. Then update the expected size. A delta of a few bytes from removed ldy/lda instructions is normal; a large or erratic delta means something else changed and needs investigating.

Compiler Unit Test Snippets

Prog8 code snippets embedded in the compiler's own Kotlin unit tests (e.g., in TestAstChecks.kt, TestOptimization.kt) follow different rules from standalone programs:

• Keep self-contained — avoid %import directives (test setup may not have library search path configured)
• Use cx16 target (not virtual, unless specifically testing the VM)
• Set writeAssembly=false if the test only checks generated AST (much faster)
• Set optimize=false by default for these snippets
• Do NOT use the %encoding iso / txt.iso() / sys.poweroff_system() pattern — that's only for real CX16 emulator runs
• Do NOT write compiler tests that depend on the vasm assembler: it is not always installed (CI only installs 64tass). For m68k codegen tests (amiga500, qemu68k targets) pass writeAssembly = true, assemble = false to the compileText/compileFile helpers. This still generates the codegen AST to assert on but skips assembling, so no vasm is needed. If a test's assertions cannot be made from the AST alone, drop the test.

Testing Assembly Output

When testing generated assembly in Kotlin tests:

• NEVER dump raw assembly in string assertions (e.g., asm shouldContain "jsr label"). On failure, this dumps the ENTIRE assembly listing into test logs.
• Instead, parse into lines and check specific properties:

  val lines = asm.lines().map { it.trim() }
  lines.any { it == "lda #42" } shouldBe true
  lines.count { it.startsWith("jsr") } shouldBe 0
  

• Prefer lines.any { ... } for existence checks and lines.count { ... } shouldBe 0 for absence checks.

Manual Verification & Emulators

Testing tip: When writing and testing Prog8 programs, use the virtual target (e.g., prog8c -target virtual -emu input.p8 or prog8c -vm input.p8ir). This is the preferred way to test because the virtual target can easily write output to stdout, making it simple to verify program behavior and check results.

CX16 output verification: Use x16emu -echo iso -run -prg input.prg to echo screen output to stdout and auto-start the program. The -run flag is critical: without it, the program loads but doesn't execute, so you'll see no output. Pipe through strings or grep to filter: x16emu -echo iso -run -prg input.prg 2>&1 | grep -E "(PASS|FAIL)". IMPORTANT: Always add %encoding iso at the top of your source file and call txt.iso() in start(). This prevents PETSCII→ISO charset translation errors that garble uppercase/special characters and make output unreadable:

%encoding iso
%import textio
main { sub start() { txt.iso(); txt.print("PASS\n") } }

IMPORTANT: Always use sys.poweroff_system() to exit the CX16 emulator cleanly! Add sys.poweroff_system() at the end of your main program block - this exits x16emu automatically in most cases. Note: The sys module is always available, there is no need to import it ever.

Commodore 64 (x64sc): x64sc input.prg - run an existing compiled program in the Commodore-64 emulator. Ignore any errors and warnings, because the emulator doesn't produce any output on STDOUT.

actual 6502 CPU simulation tests

For high-fidelity functional verification of generated 6502 code without a full emulator, you can use the ksim65 simulator in your unit tests. This allows executing the compiled machine code and asserting on memory or register states.

• Use the simulate() extension function on a CompilationResult.
• See prog8tests.codegeneration.TestExecution6502 for examples.
• Both 6502 and 65C02 CPUs can be simulated.
• The simulator supports capturing serial output and handling hardware reset/poweroff signals.
• There are various assertion helper methods to test on the state of CPU registers, memory contents, etc.
• Using the simulator is much faster and way more controllable than using a full emulator to run the code.
• The simulator DOES NOT represent an actual real world machine and has VERY LIMITED hardware devices so it can generally not be used to run full example prorams on.

Git Operations for File Moves/Deletes

When renaming or moving git-tracked files, ALWAYS use git mv:

# CORRECT - preserves git history
git mv old/path/File.kt new/path/File.kt

# WRONG - git sees this as delete + add (loses history)
mv old/path/File.kt new/path/File.kt

When deleting git-tracked files, ALWAYS use git rm:

# CORRECT - properly stages the deletion
git rm path/to/File.kt

# WRONG - git sees this as unstaged deletion
rm path/to/File.kt

Why this matters: git mv and git rm properly stage the changes and preserve file history. Plain mv/rm requires git to detect renames heuristically, which may not always work correctly.

TODO Items

The file docs/source/todo.rst contains a comprehensive list of things that still have to be fixed, implemented, or optimized. Use this to understand what features are NOT yet available in the compiler or Prog8 language: if a user asks for something that's on the TODO list, you'll know it's not implemented yet and can explain the limitation.