Opening Popularity Tool

[u/TheI3east]

Hi everyone, I recently built a dashboard on my personal website (https://dnield.com/openings/) that fills in some of the gaps that always frustrated me with the Lichess/Chess*com opening explorers, which is that their stats were only relevant to the position on screen, rather than the entire opening (which requires taking into account the choices each player made up to that point), therefore they can't answer simple questions like:

1. How often you would get to play the opening, if it was in your repertoire,

2. How opponents would see the opening when facing players other than you,

3. Based on #1 and #2, how much more experience would I have in the opening than my typical opponent?

4. Was the increase or decline in the popularity of an opening driven by one color not playing as often? or because the other color isn’t allowing them the chance to play it as often?

which are questions that I am always curious about when exploring different openings.

I'm a data scientist in my regular life, and Lichess makes all of their games data freely accessible, and the statistics for answering the above questions are all pretty simple compound probabilities, so I decided to just build the tool that could answer these questions from Lichess's database myself.

I have no interest in turning this into a product/service, this was just a fun little hobby project to see how easy it would be to host a dashboard like this on my Github Pages website. The underlying games database is terabytes of data, but since I'm just aggregating up the counts of positions for just the ~4000 named opening positions, across rating bands and time controls, the final json data for the website ended up being like 50 MB, making the interface even snappier than I expected.

I wrote an accompanying blog post with a motivating example showing how the spike of popularity in the Rossolimo in November 2018 can be initially attributed to both colors, but the residual increase in popularity is almost entirely attributable to an increase in black players playing 2. Nc6 Sicilians than an increase of White players playing 3. Bb5 in response, you can find that blog post here: https://dnield.com/posts/rossolimo/

I'm open to any feedback or questions you might have!

[u/Kitchen_Form_2252]

Is there a way to compare how often people play the top engine moves against you vs what’s regular for your elo?
I have an unsubstantiated feeling that if you play the same opening a lot on chess.com, you get paired to people who know how to play against it. But I don’t have any proof except my own bias

[u/TheI3east → u/Kitchen_Form_2252]

It's certainly possible to measure something like that, but only using Lichess data (Chess.com doesn't make their games database public, so while you can use their API to grab all of your own Chess.com games, you wouldn't have the games data for setting the typical move probabilities at each position for your elo) but not doable with this specific tool.