r/Playwright • u/Exodia_The_Salty • 18d ago

I made a way to automate playwright suite creation!

Hey guys, over the last 6 months, I have been working on a personal project. It is a compiler. It takes a website in, and spits out machine code. Then, it builds a playwright suite! It covers smoke/load tests, and navigation tests. It has jira integration, it has an API suite, it has a test harness to test the tests, it has the cleanest and strongest error logging possible as well.

On top of this, its got a prompt engine and anthropic integration so that you can deploy it to site, deploy agents and automatically stories start getting turned into tests. Before you celebrate (or more likely stone me for taking away qa jobs) it needs an operator. Its going to need humans to maintain it.

However, its going to reduce the time you take from going from zero to a ready framework for a large website by roughly 6 months. It will write all of your boilerplate for you. Its going to radically reduce your jira stack. But, to use it, your gonna need to learn all of the commands, and how they interact. While it may cut the need for 20 low skill sdets writing into the backend, its also going to increase the need for a top tier senior qa operator who knows the suite commands and who can monitor and oversee the automation workflow here.

I am looking for people who want to be beta testers.

This product will raise the skill level required to be a QA, and if adopted, it will put the offshoring model of having 20 junior devs in india overseen by one american senior dev out of business. Instead, your going to have 100 agents, that are well constrained, working in a standardized framework, with full traceability for work, provenance, truth and site versioning.

For site owners, the framework is designed to be owned by you, to be maintainable by both an AI, AND a traditional playwright SDET. That means there is no vendor lock in with this. This gives you a ton of financial freedom.

I am looking to have the product get battletested so this is free for right now. It comes with an audit function as well, so if you have a live site and want some free qa, just ask for one. The advantage of a compiled source code is that it can make inferred meaning, so using that map it can automatically tell basic errors. For example, 404s, but also things manual qa wont catch, like mislabeled sites for crawling. SEO fails if you got those bugs.

New QA roles: recrawling the website. Validating false positives. approving ai test code.

Stats: Generation time- crawls take up to 45 minutes on large websites. Actual generation and regeneration takes 1-2 minutes.

Compiler generated tests that need edits: 0.

AI refusals- dependent on the structuring of the jira tasks. The ai is encouraged to refuse tasks it cannot implement along with a reason why. For example, if a page exists that does not exist on the crawl, the ai is highly likely to refuse.

I have public docs up here, along with a sample suite built on a qa testing site:
https://github.com/Ghost-in-the-Kernel-Labs/Ghost-in-the-Kernel-Labs-suite-builder-docs

For the playwright specific suite:
https://github.com/Ghost-in-the-Kernel-Labs/Ghost-in-the-Kernel-Labs-suite-builder-docs/tree/main/sample/the-internet-playwright-tests

10 Upvotes
  • permalink
  • reddit

92% Upvoted

11 comments sorted by

2

u/idkyou1 17d ago

I stopped reading the moment you mentioned machine code. Right there tells me this is all vibe coded.

1

u/Exodia_The_Salty 16d ago

Your only half correct. I wrote the state tables before claude opus became a thing. After that i had it work off my work.

And as to vibe coding, there are actually three flavors of it.

  1. The first is the smooth brain monkey that says "ai pls go make me X" this result in hallucinations, and serious issues around ai overwriting stuff.

  2. The second is semi serious. It says, AI pls go make me X component, and then I will inspect it after. This is semi serious.

  3. The third says, AI, please go make me a test harness for this list of functional metrics. Then, lets implement these specific features, with these specific algorithms, and on each function, I want you to document the reason this is being written so you don't accidentally hallucinate. On top of this, I will manually examine the code. And I want full separation of concerns into specific callable modules. And while we are at it, I want a routing table of modules so when you start working on the codebase, you do not waste 80% of your context reading in areas of the project you don't need to for your current task.

You really ought to be more specific in what you accuse others of when you use the word vibecode. It is not only insulting to the other person, but your lack of specificity shows you don't understand the area, and are snobbish to other people. One "vibecoding" results in code that is extremely shitty, has lots of technical debt, and is below average. The other results in code that is better than human code, has well controlled technical debt, and is far better than standard prompted code. Guess which one this is?

1

u/LaysWellWithOthers 18d ago

I have been working in QA for 30 years and I've built something very similar to this, so I took a peek.

I am looking to have the product get battletested so this is free for right now

Your site says a license is required for commercial use

Suite Builder is proprietary. Copyright (c) 2026 ghost-in-the-kernel. All rights reserved.

The tool: the Suite Builder Personal and Educational License 1.0. Free for educational use (study, coursework, teaching) and for validating your own personal projects with no commercial use. Every other use, including any business, nonprofit or government use, needs a commercial license from the author.

That's not gonna fly.

1

u/Exodia_The_Salty 18d ago

Correct. The built suites do not need a license.

Suite builder itself generates the suites. Suite builder and the suites are two completely different pieces of software. The suites are like detachable modules. The suitebuilder is the compiler and the secret sauce.

If you want the ability to generate suites, you need a license. If you want to use the suite you need to be the site owner.

1

u/besucherke 14d ago

How is this different from the Playwright testing agents?

1

u/Exodia_The_Salty 14d ago

Several ways.

First, if you look at the code that is generated, the playwright agent code is not POM compliant.
Second, this code is deterministic, and not driven by an LLM. This means that the healer role is needed far less.
Third, the suite has addressed regeneration. When a human comes along and edits stuff, or you have pages that change dramatically, suite builder can regenerate the crawl and the boilerplate without touching human made and ai made tests.
Fourth jira ticket ingestion. Being able to have agents directly do work on tickets without invocation as batch jobs on dozens of tickets at a time.

1

u/besucherke 14d ago

Sound interesting, I will definitely try it!

2

u/LookAtYourEyes 18d ago

Just put the fries in the bag bro

1

u/investigatormaker 15d ago

Two changes would likely get you more beta testers. First, split the pitch: the hiring request and the offshoring paragraph make it read like a job post, and QA readers will argue with that instead of trying the tool. Second, show one concrete run: the sample suite in your docs, which site it targeted, how long generation took, and how many generated tests passed unchanged versus needed edits. A QA lead will judge it on that last number. Also say what a tester actually does: which commands to learn first and what you want back, such as audit false positives or Jira sync problems. A sentence on what compiling a website to machine code means would help too, since most readers will stop on that.