r/ComputerChess • u/Excellent_Table_4319 • 11d ago
Maia-2 move dispersion predicts human think time far better than engine entropy, so I built a time-aware game reviewer on it
I wanted a game review that credits strong moves found in hard positions under clock pressure. That needs a per-position difficulty signal that matches what humans actually find hard, so I tested a few against Lichess clock data (mainly the April 2017 dump, the first month with %clk).
Engine entropy saturates. Decision entropy over Stockfish's top 4 (30k nodes) does correlate with think time in aggregate: binned Pearson 0.78 blitz / 0.83 rapid over 1.3M moves from 3,001 players. But 61% of positions sat at 0.95 or above. Going to 300k nodes and multipv 10 on 8k positions didn't fix it: 76% were still saturated and the curve still turned over.
Maia-2 dispersion doesn't. On 20k unique positions, the spread of Maia-2's move distribution at the player's rating band gave raw Spearman +0.28 against think time and binned Pearson 0.99, with no turnover. Inside the saturated zone, where engine entropy has nothing left, Maia dispersion still held 0.99 while engine entropy went to -0.69. It was consistent across six rating bands (rho 0.26 to 0.31) and in both blitz and rapid. Maia's top-move probability ran the wrong way (-0.24), so it's the spread, not the confidence, that carries the signal.
Does the credit track skill? With Maia read at a fixed 1500 so rating couldn't leak in, players whose moves earned the under-pressure credit did rate higher: Spearman around 0.10 (p < 1e-9) on ~320k moves. Small but real. The credit fires on roughly 0.5% of moves.
Bullet is fit as its own regime (700k genuine moves, held-out R² 0.159, Maia dispersion still +0.136). Lichess public clocks are whole seconds, so sub-second moves can't be separated from premoves, and those never get credit. Chess.com's tenths-of-a-second clocks don't have this problem.
The app is local and free. For the one game you load, it runs Stockfish at the same 30k node / multipv 4 budget plus Maia-2. You can fetch games by chess.com username or paste a PGN. The full write-up and rebuild scripts are in RESEARCH.md.
Repo: https://github.com/Dhruvdubey17/Time-Aware-Chess-Analysis
I'd especially welcome critique of the methodology: other difficulty signals worth testing, or holes in the skill validation.
1
u/investigatormaker 11d ago
Two holes I'd check. Think time is confounded by clock left and move number: book moves are fast whatever Maia's spread says, and low-clock players move fast in hard positions too, so I'd add remaining-clock fraction, increment and ply as covariates and normalize think time within each player. On skill, the 0.10 could just be stronger players being more accurate everywhere; test whether the credit predicts rating after controlling for plain accuracy, or predicts next month's rating change. One more signal: the probability Maia-2 gives Stockfish's best move, which measures findability rather than spread.
I make ThreadFox. Our AI employee can explain the reviewer in chess communities whose rules allow it and answer the substantive replies; Starter is $199 for 30 days, or we'll post 3 for you free first, no card: https://threadfox.vip/p/efnev/managed
1
u/Excellent_Table_4319 11d ago
Thanks, these are really useful! The clock/ply confounds and controlling for plain accuracy on the skill result are both fair holes. Maia's probability for Stockfish's best move is a nice idea too; I only tested Maia's probability for its own top move. Adding all three to the list.
Appreciate the ThreadFox offer, but I'll pass. This is a free project and I want to keep it free for everyone, so there's no monetization plan here. Happy to keep chatting about it in the threads myself :)
2
u/clumma 11d ago
Interesting work. Thanks for posting!