mirror of
https://github.com/StewBC/cc65-Chess.git
synced 2026-10-25 01:02:25 +00:00
Stefan WesselsandClaude Opus 5
e5b640eb4f
Everything published against Sargon II was level 1, with one six-game look at level 6. Levels 2-5 had never been played, so the competitive band for cc65's upper levels was unmeasured and no match could be sized. sargon/levelcost.py times a few opening replies per level: six minutes for all seven. Level 3 is 1.5 seconds a Sargon move and level 6 is 91, so a 64-game screen is 1.6 hours at level 3, 5 to 11 at level 4 and 93 at level 6. The wall between an afternoon and a fortnight is between levels 4 and 6. Two numbers here were wrong. "45 seconds per Sargon move" at level 6 was seconds per *ply* - half those plies are cc65's, which answers in 8 ms - so it is 91 a move, out by two in the direction that matters when sizing a match. And "an early move is the expensive case" holds at the shallow levels and inverts at level 6, where a whole game costs 1.9x its opening per move. A one-factor model fitted the two known anchors within 4% across a range of 150 and then over-predicted level 3 by 87%. Three measured levels are in the table and levels 4 and 5 are ranges, because two anchors and a line through them is not a model. The first screen off it - cc65 Harder vs level 3, 64 games, 1.6 hours - scores 73.4%, above the band, so level 3 is not the pairing. The numbers worth reading are 27 distinct games in 64 against 16-17 before, worst repeat down from fourteen to six, and one fifty-move draw where the baseline had fifteen. The levels also refuse to sit on one scale: interpolation predicted 58%, and reading back down from 73.4% puts level 6 somewhere cc65 Very Hard should have scored 90% rather than the 41.7% it did. And the queen table, recorded as the most promising untested lead in the project, is closed. EVAL_QUEENHOME sets sc_pstQueen's d1 from -5 to +5, which zeroes the ten centipawns the evaluation pays the queen to leave home; the shipping binaries hash identically with it present. Over 6144 games a side it measures -1.09 sigma, the wrong way. Tripling the incentive instead, which had to be worse if the square mattered, measures +0.40 sigma. Both directions run backwards: the square is inert and the mechanism is not where the queen's opening cost comes from. Replicating the measurement that motivated it on 64 fresh games then found the statistic itself was fragile. Mate scores clamp at +-10000, and a plain mean over evaluation deltas is not robust to that: a middlegame king move reads as -289.3 against a true -23.5, and a pawn move reads as a *gain*. With the clamps dropped the queen ties with the bishop at about twice a pawn, not four times - and a developing bishop move is correct chess, which says this statistic was never measuring what it was read as. sargon/piececost.py is that computation with the clamps dropped and a median printed beside every mean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
57 lines
2.1 KiB
C
57 lines
2.1 KiB
C
/*
|
|
* testutil.h
|
|
* cc65 Chess - test support
|
|
*
|
|
* Shared helpers for the native test build. Nothing in tests/ is ever
|
|
* compiled into a cc65 target - it uses stdio, longs and other things the
|
|
* 8-bit builds cannot afford.
|
|
*/
|
|
|
|
#ifndef _TESTUTIL_H_
|
|
#define _TESTUTIL_H_
|
|
|
|
/*-----------------------------------------------------------------------*/
|
|
// Tile <-> algebraic, in the UI's 0..63 numbering where tile 0 is a8
|
|
char test_Square(const char *algebraic);
|
|
void test_TileName(char tile, char *out3);
|
|
|
|
/*-----------------------------------------------------------------------*/
|
|
// Print the engine board
|
|
void test_DumpBoard(const char *tag);
|
|
|
|
/*-----------------------------------------------------------------------*/
|
|
// Set the engine from a FEN; returns the side to move
|
|
char test_EngineSetFEN(const char *fen);
|
|
// The other direction, for writing an opening book or lifting a position out
|
|
// of a game to look at. "out" needs 90 bytes
|
|
void test_EngineGetFEN(char side, char *out);
|
|
// Per-move node counts, for tracking down a perft mismatch
|
|
void test_EnginePerftDivide(const char *fen, int depth);
|
|
|
|
/*-----------------------------------------------------------------------*/
|
|
// The suites. Each returns the number of failures
|
|
int test_RunEnginePerft(int maxDepth, int verbose);
|
|
int test_RunQuiescenceGen(int verbose);
|
|
int test_RunBudgetSurvey(int verbose);
|
|
int test_RunGameFuzz(int seed, int games, int verbose);
|
|
int test_RunCastle(int verbose);
|
|
int test_RunRepetition(int verbose);
|
|
int test_RunOpening(int verbose);
|
|
int test_RunSelfPlay(int games, int maxPlies, int verbose);
|
|
int test_RunSearchTactics(int verbose);
|
|
int test_RunSearchAlwaysMoves(int verbose);
|
|
int test_RunSearchMateInOne(int verbose);
|
|
int test_RunSearchConversion(int verbose);
|
|
int test_RunSearchBench(int verbose);
|
|
int test_RunMatchSanity(int verbose);
|
|
int test_RunMatchTerms(int verbose);
|
|
int test_RunMatchDepth(int verbose);
|
|
int test_RunMatchEqualTime(int verbose);
|
|
int test_RunMatchEndgame(int verbose);
|
|
int test_RunMatchLadder(int verbose);
|
|
int test_RunMatchRepetition(int verbose);
|
|
int test_RunMatchMateDrive(int verbose);
|
|
int test_RunMatchQueen(int verbose);
|
|
|
|
#endif //_TESTUTIL_H_
|