From ce06b20c6c696f781332c45958a082b5a7b1983e Mon Sep 17 00:00:00 2001 From: mrdudz Date: Wed, 22 Jul 2020 15:31:39 +0200 Subject: [PATCH] add some details to the readme --- test/readme.txt | 29 ++++++++++++++++++++--------- 1 file changed, 20 insertions(+), 9 deletions(-) diff --git a/test/readme.txt b/test/readme.txt index 57ebfdd23..21b13fc62 100644 --- a/test/readme.txt +++ b/test/readme.txt @@ -2,29 +2,40 @@ This directory contains test code for automatic regression testing of the CC65 compiler. -/val - the bulk of tests are contained here, individual tests should exit with - an exit code of EXIT_SUCCESS when they pass, or EXIT_FAILURE on error +/val - The bulk of tests are contained here, individual tests should exit with + an exit code of EXIT_SUCCESS when they pass, or EXIT_FAILURE on error. -/ref - these tests produce output that must be compared with reference output +/ref - These tests produce output that must be compared with reference output. /err - contains tests that MUST NOT compile -/todo - these tests fail due to open compiler issues. - when an issue was fixed, the test should get moved to /var +/todo - These tests fail due to open compiler issues. + + The makefile in this directory _expects_ the tests to fail, because of + that when an issue was fixed it will break the CI. The test should get + moved to /var in the PR fixing the issue, which will make CI pass again. + No changes to makefiles are required! /asm - contains the assembler regression tests /dasm - contains the disassembler regression tests /misc - a few tests that need special care of some sort - tests that (incorrectly) fail to compile and other tests that fail and - do NOT return an exit code are collected here. + + Tests that (incorrectly) fail to compile and other tests that fail and + do NOT return an exit code are collected here. The makefile _expects_ + those tests to fail, so when an issue is fixed it will break the CI. + When this happens, the PR fixing the issue should also "invert" the + failing condition in the makefile by removing the $(NOT) before the + offending line (or removing it when it is already there), which will + make the CI pass again. The test should then be moved elsewhere later, + which will require additional changes to the makefile(s). -to run the tests use "make" in this (top) directory, the makefile should exit +To run the tests use "make" in this (top) directory, the makefile should exit with no error. -when a test failed you can use "make continue" to run further tests +When a test failed you can use "make continue" to run further tests. --------------------------------------------------------------------------------