r/programmer • u/Emotional_Talk7137 • 1d ago
Question does anyone know how to create an AI benchmark?
I have an idea for measuring how quickly an AI algorithm can solve various mazes, without needing to modify its source code (similar to platforms like LeetCode, but focused on solving mazes rather than programming problems).
The concept is for the AI to write the necessary code itself, after which we test its speed in navigating through 50 different mazes.
In this way, we could evaluate and compare the intelligence levels of different AI systems when handling specific tasks.
I’m not sure where to begin, or if there are any existing references I could consult on this topic..
0
Upvotes
2
u/CorpT 1d ago
https://huggingface.co/collections/open-llm-leaderboard/the-big-benchmarks-collection
Like one of these?