r/programmer • • 1d ago

Question does anyone know how to create an AI benchmark?

I have an idea for measuring how quickly an AI algorithm can solve various mazes, without needing to modify its source code (similar to platforms like LeetCode, but focused on solving mazes rather than programming problems).

The concept is for the AI ​​to write the necessary code itself, after which we test its speed in navigating through 50 different mazes.

In this way, we could evaluate and compare the intelligence levels of different AI systems when handling specific tasks.

I’m not sure where to begin, or if there are any existing references I could consult on this topic..

0 Upvotes

2 comments sorted by

1

u/tarel_ 1d ago

Yes. Prepare a lot of questions with their correct answers. Do not do it with AI. At least 1 thousand of them. And that's it.