Disabled tests (class or method level) not showing in report (JUnit 5)

[dberna2]

Hi @rieckpil,

I’ve been playing around with it and noticed something: disabled tests (either at the class level or method level) don’t show up in the generated report (I’m running tests with JUnit 5).

Looking into the code, I saw that the check happens in generateHtmlWithThymeleaf:

long disabledTests = TemplateHelpers.countTestsByStatus(classMetrics, "DISABLED");

But since disabled tests never actually run, I guess they never make it into the data that gets reported — which could explain why they’re missing.

So I’m wondering if Is this how it’s supposed to work or could it be a small bug?

If it is a bug, do you think it makes sense to show separate counts (one for disabled classes, one for disabled methods), or just one count that combines both?

Happy to open a PR with a fix once we figure out what makes the most sense. 🛠️

[rieckpil]

Good catch 🦅

I'm currently not 100% happy with the tracking statistic of the test in general, e.g. like 20 test classes and 100 test methods have been executed. You can see there are already some discrepancies reported as issues.

To make this test statistic work, we'd have to (I guess) use a JUnit/TestNG extension to also keep track of parameterized tests/display names, etc. or parse existing XML reports of e.g. Surefire/Failsafe.

When I've started this tool, I thought these statistic would be nice to have but the more I've tried to cover all cases, the more I think this would be duplicate work that other tools already provide.

What I'm currently thinking is: Get rid of this general test execution tracking and focus solely on the cache statistic tracking and the related test classes.

WDYT?

[dberna2]

You mean removing only the Test Execution Summary or also the Test Execution Details?

I agree that other tools already cover generic test stats, so dropping them makes sense to keep this project focused.

Maybe we could still provide some actionable insights to improve test execution times, like:

• Highlighting slow tests or classes that reload the context too often
• Detecting expensive patterns like @DirtiesContext or using this annotation with classMode = AFTER_EACH_TEST_METHOD and suggesting alternatives (e.g., reusing context where possible)
• Suggesting best practices like Build Time Optimization Recommendations (e.g. using test slices, @MockBean, etc.)

This could make the report more of a performance advisor for integration tests instead of just a test counter.

WDYT?

[rieckpil]

Yes, my plan was to get rid of Test Execution Details and Test Execution Summary, at least for the basic KPIs that handle things like: number of tests, number of test classes, etc.

The focus on your proposed actionable insights is exactly where I want to drive this profiler to.