Skip position history in quiescence

doc/next-engine.md

@@ -196,6 +196,13 @@ full `eng_Make` behavior for actual game moves and callers that rely on it.
 This is an API refactor with a silent-failure mode. Add a fuzzer assertion that the board,
 evaluation, phase, key, ring top and valid count are identical after every rejected probe.
 
+**Phase 22 B2 result: rejected.**  A split probe/commit API passed the state fuzzer and all
+1,024 whole-book searches, but the fixed C64 replay was 47,350 jiffies against 47,324 for the
+same refactored code using the old full probes.  Both were slower than the 46,450-jiffy Phase A
+engine because cc65's extra C calls cost more than the illegal probes saved.  Enabling B2 beside
+B3 changed its final 42,300-jiffy replay by less than 0.1% and reversed direction across pairs.
+The shipped `eng_Make` interface and ordinary legality path therefore remain unchanged.
+
 ### B3. Do not maintain history in quiescence
 
 `quiesce` never calls `eng_IsRepetition`. Check evasions mean it can now contain quiet moves,
@@ -213,6 +220,16 @@ Give quiescence a no-history make/unmake path. Prove:
 This is likely the largest hash experiment because quiescence owns most generating nodes. It
 must be measured rather than inferred from that sentence.
 
+**Phase 22 B3 result: kept.**  History maintenance is disabled once at the quiescence boundary
+and restored on every return, while board and all three evaluation totals continue to use the
+ordinary make/unmake path.  All 1,024 whole-book records were identical.  Explicit tests cover
+ordinary, budget-abort, arena-exhaustion and mate returns and compare the parent board, totals,
+key, entire ring digest, EP, castling, halfmove and king state.  The final fixed C64 replay fell
+from 46,558 to **42,234 jiffies, 9.3%**, with identical 14,400 nodes and result digest.
+The compact boundary-mode implementation costs 46 bytes: Atari `DATA` ends `$7BEF`, leaving 16
+bytes before `DLIST` without taking its page jump and retaining 716 bytes below the framebuffer;
+Apple II MAIN ends `$B29F`, leaving 1,120 bytes, and BSS is unchanged.
+
 ### B4. Restore running state instead of recomputing it on unmake
 
 A quiet make/unmake calculates the same two-piece hash delta twice. It also calls the same

doc/rework-log.md

@@ -2474,6 +2474,52 @@ entry still describes the board.  B1 is rejected as a speed change.
 
 ---
 
+## Phase 22 - history stops at the quiescence boundary
+
+B2 and B3 share an attractive operation: move the board and running evaluation without
+maintaining a position signature no caller will read.  They did not share a result.
+
+The first implementation split make into probe and commit calls.  The fuzzer compared all 128
+board bytes, middlegame and endgame totals, phase, piece hash, en-passant, castling, halfmove,
+both king trackers and a digest covering the entire history ring plus top and valid around
+every legality probe.  It stayed exact, as did all 1,024 `book.epd` searches.  But B2's fixed
+C64 result was 47,350 jiffies against 47,324 for the refactored old path: nothing.  Worse, both
+were about 1.9% slower than Phase 20's roughly 46,450 because the additional cc65 C calls cost
+more than the rejected probes' hash work.  With B3 also active, toggling B2 produced 42,244
+against 42,268 in one pair and 42,301 against 42,268 in another.  The sub-tenth-percent result
+reversed direction.  B2 is rejected and the ordinary legality path keeps the full `eng_Make`
+contract.
+
+B3 owns enough calls to pay.  Quiescence never calls `eng_IsRepetition`; the key and ring may
+therefore stay at the parent position throughout that subtree.  The compact implementation
+sets one engine history mode at the depth-zero boundary, runs ordinary recursive quiescence
+make/unmake with only hash and ring maintenance skipped, and restores the mode on return.  It
+does not infer safety from whole-book equality alone.  A test-only quiescence entry snapshots
+and compares the board, three totals, key, full ring digest, EP, castling, halfmove and kings
+after ordinary completion, budget abort, arena exhaustion and mate.
+
+The final target pair, windowed c64m, NTSC, retained six-position replay in opposite order:
+
+| build | first pass | second pass | nodes / digest |
+|---|---:|---:|---|
+| compact control, history on | 46,558 | 46,558 | 14,400 / identical |
+| B3 compact boundary mode | 42,234 | 42,235 | 14,400 / identical |
+
+That is **9.3% less C64 time at the same nodes**.  Baseline and B3 also returned identical
+move, score, depth and node records for all 256 opening positions at every shipped budget.
+
+The first split API grew common code by 259 bytes and pushed Atari's display list from `$7C00`
+to `$7D00`, spending a 256-byte page for no chess.  Passing a history flag to every make/unmake
+cut the growth to 102 bytes but still crossed the page.  Setting the mode once at the
+quiescence boundary is 46 bytes over Phase 20: Atari DATA ends `$7BEF`, leaving 16 bytes before
+the `$7C00` display list and the full 716 bytes below the framebuffer; Apple II MAIN ends
+`$B29F`, leaving 1,120 bytes, and its BSS still ends `$17B5`, leaving 2,122.  All seven targets
+build at `optsize`.
+
+This saving is banked as faster fixed-budget search only.  No node budget changes here.
+
+---
+
 ## Decisions on record
 
 Kept here so they do not get relitigated.

src/engine.c

@@ -49,6 +49,7 @@ unsigned int geHashKey;
 static unsigned int	su_hashRing[HASH_RING];
 static char			sc_hashTop;			// next slot to write, wraps
 static char			sc_hashValid;		// entries that can be trusted
+static char			sc_historyEnabled = 1;
 
 #ifdef SEARCH_PROFILE
 // The sink stops cc65 deleting a deliberately discarded duplicate result.
@@ -752,6 +753,7 @@ void eng_HashReset(void)
 	sc_hashValid = 0;
 	su_hashRing[sc_hashTop++] = positionKey();
 	sc_hashValid = 1;
+	sc_historyEnabled = 1;
 }
 
 /*-----------------------------------------------------------------------*/
@@ -787,6 +789,22 @@ char eng_HistoryMatchesPosition(void)
 	return sc_hashValid &&
 	       su_hashRing[(char)(sc_hashTop - 1) & HASH_MASK] == positionKey();
 }
+
+/*-----------------------------------------------------------------------*/
+unsigned int eng_HistoryStateDigest(void)
+{
+	unsigned int digest = sc_hashTop | ((unsigned int)sc_hashValid << 8);
+	char i;
+
+	// Include stale slots too.  A no-history move must not write the ring, even
+	// if it leaves top and valid unchanged and the write is not visible yet.
+	for(i = 0; i < HASH_RING; ++i)
+	{
+		digest = (digest << 1) | (digest >> 15);
+		digest ^= su_hashRing[i];
+	}
+	return digest;
+}
 #endif
 
 /*-----------------------------------------------------------------------*/
@@ -799,6 +817,12 @@ static void revokeRights(char square)
 	else if(square == sc_rookQ[SIDE_BLACK]) geCastle &= ~ENG_CASTLE_BQ;
 }
 
+/*-----------------------------------------------------------------------*/
+void eng_HistoryEnable(char enabled)
+{
+	sc_historyEnabled = enabled;
+}
+
 /*-----------------------------------------------------------------------*/
 void eng_Make(const t_engMove *move, t_engUndo *undo)
 {
@@ -871,8 +895,6 @@ void eng_Make(const t_engMove *move, t_engUndo *undo)
 			su_profileSink = eval_PhaseDelta(move, piece, undo->m_captured);
 		if(PROFILE_EVAL_END == geSearchProfile)
 			su_profileSink = eval_EndDelta(move, piece, undo->m_captured);
-		if(PROFILE_HASH_DELTA == geSearchProfile)
-			su_profileSink = hashDelta(move, piece, undo->m_captured);
 #endif
 
 		// keep the running evaluation with the pieces.  eng_Unmake subtracts
@@ -880,11 +902,18 @@ void eng_Make(const t_engMove *move, t_engUndo *undo)
 		geEvalScore += eval_MoveDelta(move, piece, undo->m_captured);
 		gePhase += eval_PhaseDelta(move, piece, undo->m_captured);
 		geEvalEnd += eval_EndDelta(move, piece, undo->m_captured);
+	}
+
+	if(sc_historyEnabled)
+	{
+#ifdef SEARCH_PROFILE
+		if(PROFILE_HASH_DELTA == geSearchProfile)
+			su_profileSink = hashDelta(move, piece, undo->m_captured);
+#endif
 		geHashKey ^= hashDelta(move, piece, undo->m_captured);
 
-		// and record where we now are.  This runs for search moves as well as
-		// real ones, which is the point: the ring is the game's history and the
-		// current search line at the same time
+		// Search lines and played moves share the ring.  Quiescence is absent
+		// because nothing below it asks about repetition.
 #ifdef SEARCH_PROFILE
 		if(PROFILE_HISTORY == geSearchProfile)
 		{
@@ -953,15 +982,20 @@ void eng_Unmake(const t_engMove *move, const t_engUndo *undo)
 			su_profileSink = eval_PhaseDelta(move, moved, undo->m_captured);
 		if(PROFILE_EVAL_END == geSearchProfile)
 			su_profileSink = eval_EndDelta(move, moved, undo->m_captured);
-		if(PROFILE_HASH_DELTA == geSearchProfile)
-			su_profileSink = hashDelta(move, moved, undo->m_captured);
 #endif
 
 		geEvalScore -= eval_MoveDelta(move, moved, undo->m_captured);
 		gePhase -= eval_PhaseDelta(move, moved, undo->m_captured);
 		geEvalEnd -= eval_EndDelta(move, moved, undo->m_captured);
-		geHashKey ^= hashDelta(move, moved, undo->m_captured);
+	}
 
+	if(sc_historyEnabled)
+	{
+#ifdef SEARCH_PROFILE
+		if(PROFILE_HASH_DELTA == geSearchProfile)
+			su_profileSink = hashDelta(move, moved, undo->m_captured);
+#endif
+		geHashKey ^= hashDelta(move, moved, undo->m_captured);
 		--sc_hashTop;
 		if(sc_hashValid)
 			--sc_hashValid;
@@ -973,10 +1007,13 @@ void eng_Unmake(const t_engMove *move, const t_engUndo *undo)
 void eng_ProfileBoardPair(const t_engMove *move)
 {
 	t_engUndo undo;
+	char history = sc_historyEnabled;
 
 	sc_profileBoardOnly = 1;
+	eng_HistoryEnable(0);
 	eng_Make(move, &undo);
 	eng_Unmake(move, &undo);
+	eng_HistoryEnable(history);
 	sc_profileBoardOnly = 0;
 }
 #endif

src/engine.h

@@ -122,6 +122,7 @@ char eng_IsRepetition(char needed);
 #ifdef EVAL_TUNING
 // Native-test view of the invariant used by ENGINE_REPETITION_RING_KEY.
 char eng_HistoryMatchesPosition(void);
+unsigned int eng_HistoryStateDigest(void);
 #endif
 
 /*-----------------------------------------------------------------------*/
@@ -172,4 +173,9 @@ char eng_GenLegalMoves(char side, t_engMove *moves);
 void eng_Make(const t_engMove *move, t_engUndo *undo);
 void eng_Unmake(const t_engMove *move, const t_engUndo *undo);
 
+// Quiescence disables position-history maintenance for its whole subtree.
+// Nothing there reads the key or repetition ring, and setting the mode once
+// at the boundary is much cheaper on cc65 than another make/unmake argument.
+void eng_HistoryEnable(char enabled);
+
 #endif //_ENGINE_H_

src/search.c

@@ -436,7 +436,18 @@ static int negamax(char side, char depth, int alpha, int beta, char ply)
 		return 0;
 
 	if(0 == depth)
+	{
+#if SEARCH_QUIESCE_HISTORY
 		return quiesce(side, alpha, beta, ply);
+#else
+		int quietScore;
+
+		eng_HistoryEnable(0);
+		quietScore = quiesce(side, alpha, beta, ply);
+		eng_HistoryEnable(1);
+		return quietScore;
+#endif
+	}
 
 	inCheck = eng_InCheck(side);
 
@@ -761,3 +772,45 @@ void search_Best(char side, char maxDepth, unsigned int nodeBudget, t_searchResu
 	if(sc_randMoves < SEARCH_RANDOM_MOVES)
 		++sc_randMoves;
 }
+
+#ifdef EVAL_TUNING
+/*-----------------------------------------------------------------------*/
+char search_TestQuiesceState(char side, unsigned int budget, char exhaustArena)
+{
+	static char board[128];
+	unsigned int hash, history;
+	int score, end, phase;
+	char ep, castle, halfmove, kingBlack, kingWhite, sq, same = 1;
+
+	eval_Refresh();
+	hash = geHashKey;
+	history = eng_HistoryStateDigest();
+	score = geEvalScore;
+	end = geEvalEnd;
+	phase = gePhase;
+	ep = geEP;
+	castle = geCastle;
+	halfmove = geHalfmove;
+	kingBlack = geKing[SIDE_BLACK];
+	kingWhite = geKing[SIDE_WHITE];
+	for(sq = 0; sq < 128; ++sq)
+		board[sq] = geBoard[sq];
+
+	si_nodes = 0;
+	si_budget = budget;
+	si_arenaTop = exhaustArena ? SEARCH_ARENA - 4 : 0;
+	sc_abort = 0;
+	eng_HistoryEnable(0);
+	(void)quiesce(side, -EVAL_INFINITY, EVAL_INFINITY, 0);
+	eng_HistoryEnable(1);
+	si_arenaTop = 0;
+
+	for(sq = 0; sq < 128; ++sq)
+		if(board[sq] != geBoard[sq])
+			same = 0;
+	return same && hash == geHashKey && history == eng_HistoryStateDigest() &&
+	       score == geEvalScore && end == geEvalEnd && phase == gePhase &&
+	       ep == geEP && castle == geCastle && halfmove == geHalfmove &&
+	       kingBlack == geKing[SIDE_BLACK] && kingWhite == geKing[SIDE_WHITE];
+}
+#endif

src/search.h

@@ -20,6 +20,11 @@
 // Deepest the quiescence search may run past the main search
 #define SEARCH_MAX_PLY		12
 
+// Exact-state switch keeps the old quiescence path for target A/B measurement.
+#ifndef SEARCH_QUIESCE_HISTORY
+#define SEARCH_QUIESCE_HISTORY	0
+#endif
+
 /*-----------------------------------------------------------------------*/
 // Scoring a repeated position as a draw, switchable the same way and for the
 // same reason as the evaluation terms: measuring a change means playing it
@@ -136,4 +141,10 @@ char search_Seeded(void);
 // there legal moves, and am I in check" - no special-case machinery needed
 char search_Outcome(char side);
 
+#ifdef EVAL_TUNING
+// Test-only entry to exercise quiescence return paths and prove its no-history
+// subtree restores the parent position key and ring exactly.
+char search_TestQuiesceState(char side, unsigned int budget, char exhaustArena);
+#endif
+
 #endif //_SEARCH_H_

tests/gamefuzz.c

@@ -35,6 +35,7 @@
 
 static char sc_snapshots[MAX_PLIES][128];
 static t_engMove st_played[MAX_PLIES];
+static char sc_probeBoard[128];
 
 /*-----------------------------------------------------------------------*/
 static int compareBoard(const char *want, int ply, const char *tag, int game)
@@ -188,6 +189,41 @@ static int checkHistoryKey(int game, int ply, const char *tag)
 	return 0;
 }
 
+/*-----------------------------------------------------------------------*/
+// A no-history move must be exactly self-reversing and must never touch the
+// hash or any ring slot.  Return -1 for a broken round trip, else whether the
+// move was legal.
+static int probeLegal(const t_engMove *move, char side, int game, int ply)
+{
+	t_engUndo undo;
+	unsigned int hash = geHashKey;
+	unsigned int history = eng_HistoryStateDigest();
+	int score = geEvalScore, end = geEvalEnd, phase = gePhase;
+	char ep = geEP, castle = geCastle, halfmove = geHalfmove;
+	char kingBlack = geKing[SIDE_BLACK], kingWhite = geKing[SIDE_WHITE];
+	char legal;
+
+	memcpy(sc_probeBoard, geBoard, 128);
+	eng_HistoryEnable(0);
+	eng_Make(move, &undo);
+	legal = !eng_IsAttacked(geKing[side], 1 - side);
+	eng_Unmake(move, &undo);
+	eng_HistoryEnable(1);
+
+	if(memcmp(sc_probeBoard, geBoard, 128) ||
+	   hash != geHashKey || history != eng_HistoryStateDigest() ||
+	   score != geEvalScore || end != geEvalEnd || phase != gePhase ||
+	   ep != geEP || castle != geCastle || halfmove != geHalfmove ||
+	   kingBlack != geKing[SIDE_BLACK] || kingWhite != geKing[SIDE_WHITE])
+	{
+		printf("    game %d probe ply %d: no-history make/unmake changed state\n",
+		       game, ply);
+		return -1;
+	}
+
+	return legal;
+}
+
 /*-----------------------------------------------------------------------*/
 int test_RunGameFuzz(int seed, int games, int verbose)
 {
@@ -218,21 +254,15 @@ int test_RunGameFuzz(int seed, int games, int verbose)
 				if((moves[i].m_flags & (ENG_MF_CASTLE | ENG_MF_ENPASSANT | ENG_MF_PROMO)) &&
 				   (rand() % 10) < 8)
 				{
-					t_engUndo probe;
-					eng_Make(&moves[i], &probe);
-					if(checkHistoryKey(game, ply, "probe make"))
+					int legal = probeLegal(&moves[i], side, game, ply);
+
+					if(legal < 0)
 					{
 						++failures;
 						goto nextGame;
 					}
-					if(!eng_IsAttacked(geKing[side], 1 - side))
+					if(legal)
 						chosen = i;
-					eng_Unmake(&moves[i], &probe);
-					if(checkHistoryKey(game, ply, "probe unmake"))
-					{
-						++failures;
-						goto nextGame;
-					}
 					if(0xFF != chosen)
 						++specials;
 				}
@@ -241,22 +271,15 @@ int test_RunGameFuzz(int seed, int games, int verbose)
 			for(i = 0; i < count * 2 && 0xFF == chosen; ++i)
 			{
 				char pick = rand() % count;
-				t_engUndo probe;
+				int legal = probeLegal(&moves[pick], side, game, ply);
 
-				eng_Make(&moves[pick], &probe);
-				if(checkHistoryKey(game, ply, "probe make"))
+				if(legal < 0)
 				{
 					++failures;
 					goto nextGame;
 				}
-				if(!eng_IsAttacked(geKing[side], 1 - side))
+				if(legal)
 					chosen = pick;
-				eng_Unmake(&moves[pick], &probe);
-				if(checkHistoryKey(game, ply, "probe unmake"))
-				{
-					++failures;
-					goto nextGame;
-				}
 			}
 
 			if(0xFF == chosen)

tests/quiescence.c

@@ -20,6 +20,7 @@
 #include <stdio.h>
 #include "types.h"
 #include "engine.h"
+#include "search.h"
 #include "testutil.h"
 
 #define QT_MAX_PLY	6
@@ -146,6 +147,41 @@ static void walk(char side, int depth, int ply)
 	}
 }
 
+/*-----------------------------------------------------------------------*/
+static void checkSearchStateReturns(void)
+{
+	static const struct
+	{
+		const char *fen;
+		unsigned int budget;
+		char exhaustArena;
+		const char *name;
+	} cases[] =
+	{
+		{ "r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 1",
+		  60000, 0, "ordinary return" },
+		{ "r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 1",
+		  2, 0, "budget abort" },
+		{ "r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 1",
+		  60000, 1, "arena exhaustion" },
+		{ "7k/6Q1/5K2/8/8/8/8/8 b - - 0 1",
+		  60000, 0, "mate return" },
+	};
+	int i;
+
+	for(i = 0; i < (int)(sizeof(cases)/sizeof(cases[0])); ++i)
+	{
+		char side = test_EngineSetFEN(cases[i].fen);
+
+		if(!search_TestQuiesceState(side, cases[i].budget,
+		                              cases[i].exhaustArena))
+		{
+			printf("    quiescence changed parent state on %s\n", cases[i].name);
+			++si_failures;
+		}
+	}
+}
+
 /*-----------------------------------------------------------------------*/
 int test_RunQuiescenceGen(int verbose)
 {
@@ -175,6 +211,7 @@ int test_RunQuiescenceGen(int verbose)
 		if(si_failures)
 			printf("    position %d failed\n", i);
 	}
+	checkSearchStateReturns();
 
 	if(verbose || !si_failures)
 		printf("  %ld positions, %ld capture moves against %ld full moves (%ld%%)\n",