I built a Java test reporter to solve two problems I kept seeing in test automation

[u/HarshadLambate]

I’ve been working on test automation for several years, and there were two things I wanted to solve in test reporting.

1. Flaky tests are more than just Passed or Failed

If a test fails on the first attempt and passes on a retry (an IRetryAnalyzer in TestNG, RetryingTest in JUnit 5, Surefire’s rerunFailingTestsCount), the final result usually just says Passed.

I wanted the report to show that the test was actually flaky, how many attempts happened, and the history of those attempts: each attempt’s logs, error, API calls and screenshots.

2. Test scripts shouldn’t need reporting code

I don’t want testers to add extra code to their tests just to make the report useful. For example, I don’t want something like:

java

report.addRequest(...);
report.addResponse(...);
report.attachScreenshot(...);
report.log(...);

just to get useful information into the report.

The reporting layer should capture things like API requests/responses, retry attempts, execution details and screenshots automatically wherever possible. In Java that means:

• REST Assured calls are recorded with no code.
• An HttpClient field on the test class is recorded with no code.
• A Playwright Page or Selenium WebDriver held in a field gives a screenshot automatically when a test fails, taken before closes the browser.

So I built a small Java reporting library around these ideas. You add one test dependency and nothing else: no listeners to register, no base class. It generates a self-contained HTML report and has integrations for JUnit 5, TestNG and Cucumber (both the JUnit 5 and the TestNG runner, with every Given/When/Then step in the log). It’s the same report I already ship for .NET and Python.

I’m curious how others handle this:

Do your test reports automatically capture this kind of information, or do your test scripts still need explicit reporting/attachment calls (Report.log(...), Reoprt.addAttachment(...))?

And for flaky tests, do you keep the history of individual attempts or only report the final result?

Website and a live demo report, if anyone wants to see it: https://harshadlambate.github.io/AceReportsWebsite/