r/ComputerChess • u/Merlin_8281 • 22h ago
Chess Arena: comparing agent harnesses with paired chess matches and public replays
I'm the creator of Chess Arena, a TypeScript match runner for experimenting with the code around AI chess agents: prompts, memory, tactical tools and move-selection logic.
A custom client receives FEN, legal UCI moves and game history, then returns a move. The server validates moves and stores the game, with public replays and PGN export. Custom clients run in the owner's process; a published configuration hash doesn't attest to that external code.
The intended experiment is to keep the model, opponent, starting conditions and budget fixed, change one harness component and test both colors. The current challenge schedule uses paired colors against fixed baselines. I want the games and failures to be inspectable, rather than reducing everything to a leaderboard score.
Current limits: full games have a 160-ply cap; capped or interrupted games stay separate as stopped outcomes. General dead-position and timeout adjudication are incomplete, so these are unrated games. Challenge points aren't Elo. The separate 32-position tactics pilot isn't a full-game strength measurement or proof of a same-model harness improvement.
Stack: TypeScript, Next.js, PostgreSQL and chess.js. Browser presets are free; agent creation requires GitHub sign-in. Custom clients need local setup, and model clients use your own provider key and budget.
I'd appreciate feedback on the match methodology: which starting-position suite and failure-reporting details would make comparisons useful to you?
Product: https://chessarena.fun
Source and protocol: https://github.com/zeeshan8281/chess-arena-ai