feat(cli): add wheels coverage command with CRAP reporting - #3434
Merged
Conversation
Adds a ERROR: command: - instruments app/ with function-level coverage counters (server.__wheels_cov), backing up originals and reverting automatically - runs the app test suite against the running server (?coverage=true) - combines collected coverage with per-file cyclomatic complexity into a CRAP ranking (complexity^2 x (1 - coverage)^3 + complexity) - app-runner.cfm dumps the counter map to /tmp/wheels-app-coverage.json when ?coverage=true (reset + best-effort dump so it never breaks test output) Pairs with the debug bar Complexity panel (static) to complete the CRAP loop. Signed-off-by: Peter Amiri <peter@alurium.com>
The gate looked baseline entries up by exact file:line:function id, so any edit that shifted line numbers in a file (e.g. adding a new CLI command to Module.cfc) made unchanged over-threshold functions look 'new' and failed CI. Match by file:function now, with a 250-line window fallback when a file defines several same-named functions. Line shifts from unrelated edits no longer trip the gate; real regressions and genuinely new hotspots still fail. Signed-off-by: Peter Amiri <peter@alurium.com>
Returning the error text made ERROR: exit 0 on failure. Throw Wheels.CoverageFailed so the runtime maps it to a non-zero exit, matching the typed-error convention used by every other write-side command. The finally block still reverts instrumentation. Signed-off-by: Peter Amiri <peter@alurium.com>
ERROR: deliberately builds its runner URL without reload — Lucee recompiles instrumented files from their disk timestamps anyway, and an application restart mid-test-run is an unneeded side effect. Signed-off-by: Peter Amiri <peter@alurium.com>
parseCoverageArgs read parsed.testDb, but ArgSpec stores dashed flags under their literal name (parseTestArgs uses parsed["test-db"]). Live e2e run of ERROR: surfaced it as: key [testDb] doesn't exist. Signed-off-by: Peter Amiri <peter@alurium.com>
Lucee routes to its ORO fallback, which throws "Sequence (?<...) not recognized" — surfaced by a live ERROR: run at $analyze. The debug-bar CodeComplexity.cfc carried the same latent pattern. Both now mask (so URL strings cannot be eaten) and strip line comments with a plain — valid in ORO, Java, and RustCFML engines. Verified on Lucee 7 against a sample with URLs, tag comments, block comments and code. Signed-off-by: Peter Amiri <peter@alurium.com>
Live run on the pmx app showed FileRead/FileWrite round-trips drop the UTF-8 BOM, leaving BOM'd files (e.g. app/lib/oauth2/oauth2.cfc) with a one-line diff after revert. Backups now store the raw bytes and revert restores them byte-for-byte; instrumented writes re-attach the BOM via a ByteArrayOutputStream. Verified on Lucee 7 with BOM'd and BOM-less files: hashes identical after instrument+revert, backup dir removed. Signed-off-by: Peter Amiri <peter@alurium.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
wheels coveragecommand that closes the CRAP loop started with the complexity gate and the debug-bar Complexity panel.What it does
wheels coverage(CLI, from an app root with a running server):app/with function-level coverage counters (server.__wheels_cov), backing up originals to.coverage-backup/and reverting automatically when done (even on error). Backups are byte-exact — UTF-8 BOMs survive the round-trip (verified against pmx'sapp/lib/oauth2/oauth2.cfc).?coverage=true.vendor/wheels/tests/app-runner.cfmresets the counter map and dumps it to/tmp/wheels-app-coverage.jsonin afinallyso it never breaks test output.complexity^2 x (1 - coverage)^3 + complexity), sorted worst-first, with--top Ncontrol (default 15). Failures throwWheels.CoverageFailed(non-zero exit).Details
cli/lucli/services/coverage/CoverageService.cfc(instrument/revert/run/collect/analyze/report)cli/lucli/Module.cfc: publiccoverage()+parseCoverageArgs()(ArgSpec--top,--test-db); hidden from MCP tools (stateful — mutates app files while running)vendor/wheels/tests/app-runner.cfm: coverage reset + best-effort JSON dumpAlso fixed along the way (all found by live runs)
tools/code-quality/cfml-complexity.py): matched by exactfile:line:function, so inserting lines in a file made unchanged over-threshold functions look "new" and failed CI. Now matches byfile:functionwith a 250-line window for same-named functions; regression + new-hotspot detection verified.(?m)(?<!:)//throwsSequence (?<...) not recognizedon Lucee's ORO fallback. Replaced with://masking + plain//[^\r\n]*in both the new service and the shipped debug-barCodeComplexity.cfc(which carried the same latent pattern).Verified
wheels coveragee2e on the wheels repo demo app (3,717 counters instrumented and reverted on pmx, working-tree verified byte-identical afterwards)