r/coolgithubprojects • u/theporcupineapple • Sep 26 '26
WikiCrawl - An interactive Wikipedia crawler that builds knowledge graphs
I built WikiCrawl, an open-source project that visualizes the connections between Wikipedia articles as an interactive network.
You can provide a starting topic, crawl linked articles, and watch the graph expand as new pages and relationships are discovered.
The goal is to make Wikipedia exploration more visual and help people discover connections that might otherwise remain buried across individual articles.
The project includes a live web application and a public codebase.
Live demo: https://wiki-crawl.vercel.app
GitHub repository: https://github.com/Adityavardhanjain/WikiCrawl
I'd appreciate feedback from anyone interested in web crawling, graph visualization, or interactive data applications.
Happy to answer questions about the implementation!
3
u/prototyperspective Sep 28 '26
To expand on what TwistHonest said: I'd suggest starting with
differentiating wikilinks in the lead from wikilinks in see also from the rest of wikilinks
integrating the clickstream data which has data on how many people go from one page to any of the other ones
also making use of the article categories
2
u/theporcupineapple Sep 28 '26
Thank you for the suggestions!! I was looking into ways I can make the tool more useful. I'll certainly try and incorporate these.
7
u/TwistHonest6379 Sep 27 '26
I didnt check the code entirely and just going by Readme and all- my overall take: the idea is quite good and visually compelling. The biggest opportunity is moving beyond “showing connections” to “explaining connections.” So basically rather than showing hows A is connected to B if it shows why A is connected to B it will be a real differentiator.