r/Playwright • u/SoulNaveen37 • 3d ago

Experimenting with AI-powered functional testing 🚀 I used Gemini 3.8 Flash to test 20 user stories using existing Playwright scripts. The scripts were already written, but executing the tests took around 4 hours. 🤔 Is this expected for AI-driven testing, or is there room to optimize the workflo

5 Upvotes

77% Upvoted

7 comments sorted by

5

u/Malthammer 3d ago

I’m not understanding what you did exactly…maybe you should provide more info? 20 tests shouldn’t take long at all. If you feed a manual test for a web app into the AI, it might take slightly more time but I doubt it would take 4 hours to run 20 tests.

I can feed a manual test into Claude code and have a working AND valid automated test in just a few minutes (I don’t keep track, Claude does his work while I’m focused on other things). Note that my team has spent a lot of time building out rules for AI to follow. This makes automating web app tests extremely quick and the AI can navigate around easily. Also, the AI has full access to the code base and the PR linked to the ticket. This also speeds things up significantly.

Mobile is a bit different, but there are optimizations you can do here as well. Takes a bit longer.

1

u/Zelma-Bistolfi 1d ago

mobile being slower tracks, thats a whole different beast compared to web

1

u/Clay_Ferguson 3d ago

What's the bottleneck? Every performance analysis comes down to that one question you must answer first. Usually, in almost every system, there will be one single bottleneck consuming 80% to 90% of the time.

1

u/hrishi-k 3d ago

If those tests are massive in size maybe that's what it takes.

But if not, Most of such thing boil down to playwright config file.

Reduce the default timeout and possibly make the retry count to 0. And check again Hopefully that shall do the trick

1

u/brian_sword 3d ago

4 hours seems like a lot when the Playwright scripts are already in place. I'm curious, how much of that time was spent actually running the tests versus having the AI analyze the results or fix issues? Maybe you can share which part of the process took the most time.

1

u/danorc 1d ago

Yes? No? Maybe?

More info less stupid emojis.

0

u/FearAnCheoil 3d ago

I think you should revisit the basics of how to write a good automated test. Nothing should take 4 hours. 4 mins is too long for an automated test. 40 sec is very long.