test(cli): fail-closed on directoryRejected / empty / unloadable + browserTest exit - #3395
Merged
Merged
Conversation
Harness for #3083: $cliTestResultFailed currently mirrors runTests' totalFail+totalError check, and $browserTestResultFailed mirrors browserTest's always-success return. Specs expect directoryRejected, bundlesDiscovered=0, unloadable, and browser Fail/Error to fail-closed. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Peter Amiri <peter@alurium.com>
…r Fail Expand $cliTestResultFailed to match test-local.sh / run-tests.sh honesty flags, pass the unloadable-spec probe into the exit path, and throw Wheels.TestsFailed from browserTest when Fail/Error > 0. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Peter Amiri <peter@alurium.com>
$throwIfCliTestsFailed and $throwIfBrowserTestsFailed are no-ops so the Wheels.TestsFailed locks for directoryRejected, empty, unloadable, and browser Fail/Error go red. Helper boolean specs stay green. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Peter Amiri <peter@alurium.com>
Compose $throwIfCliTestsFailed / $throwIfBrowserTestsFailed from the existing boolean helpers and wire runTests / browserTest to call them after the report flushes. No parallel bare throw left beside the seams. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Peter Amiri <peter@alurium.com>
bpamiri
marked this pull request as ready for review
August 23, 2026 23:55
cursor Bot
pushed a commit
that referenced
this pull request
Aug 24, 2026
Tip already fail-closes compile-skip via $countSpecsFailedToLoad and $throwIfCliTestsFailed. Restore those helpers and the WARN copy. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Peter Amiri <peter@alurium.com>
bpamiri
added a commit
that referenced
this pull request
Aug 24, 2026
* test(cli): prove-red Hardener SHOULDs 3-6 (unloadable / docs / CommandBox / orphan) Lock residual gaps on tip ebb48e5: browser unloadable ignore, WARN soft-fail copy, guides that still teach vacuous exit 0, CommandBox test runners that swallow failing exits, and unrouted Tests.cfc. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Peter Amiri <peter@alurium.com> * fix(cli): fail-closed browser unloadable, deprecate CommandBox test, drop orphan Tests.cfc SHOULD 3 residual: $browserTestResultFailed takes specsFailedToLoad; WARN no longer says silently skipped. SHOULD 4: guides name Wheels.TestsFailed. SHOULD 5: CommandBox test runners error() instead of swallowing TestBox exits. SHOULD 6: delete unrouted Tests.cfc. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Peter Amiri <peter@alurium.com> * revert(cli): keep #3395 SHOULD 3 seams; do not reopen C1 Tip already fail-closes compile-skip via $countSpecsFailedToLoad and $throwIfCliTestsFailed. Restore those helpers and the WARN copy. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Peter Amiri <peter@alurium.com> * test(cli): prove-red C5 — CommandBox error() must not reach testbox run CommandBox error() does not abort. Specs now require no testbox run and return immediately after the deprecation error(). Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Peter Amiri <peter@alurium.com> * fix(cli): return after CommandBox test deprecation; drop testbox run error() does not abort. Each frozen test runner now returns immediately and no longer invokes testbox run (C5). Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Peter Amiri <peter@alurium.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #3083 (does not close).
wheels testexited 0 on runner-honesty failures thattools/test-local.shandtools/ci/run-tests.shalready treat as exit 1 (directoryRejected,bundlesDiscovered == 0). Unloadable*Spec.cfcfiles got a WARN indisplayTestResultsbut did not settestsFailed.wheels browser testprinted Fail/Error then alwaysreturn ""(LuCLI success).TDD on this branch:
$throwIf*(red)$throwIf*from the helpers;runTests/browserTestcall them as the onlyWheels.TestsFailedsites (green)Desk accepted
$throwIf*seam specs as process-exit PROVEN. Specs call the seams with stub JSON; no liverunTests/browserTestHTTP.BLOCKER 1 —
wheels testfail-closed$cliTestResultFailedowns the boolean.$throwIfCliTestsFailedthrowsWheels.TestsFailedwhen that helper is true.runTestsstashes the JSON + unloadable probe, then calls the seam after the report flushes (so the throw is not swallowed as a crashed run).BLOCKER 2 —
wheels browser testfail-closed$browserTestResultFailedowns the boolean.$throwIfBrowserTestsFailedthrowsWheels.TestsFailedwhen Fail/Error > 0. Both the text path and--format=jsoncall this seam; missing Playwright / unreachable server stillreturn "".Red → green
Filter / URL:
GET http://localhost:8080/wheels/cli/tests?format=json. Isolatecli.lucli.tests.specs.commands.TestExitFailClosedSpecinbundleStats.f63ddd719cecba650eae6d7abe0toThrowred (did not throw … Type=[Wheels.TestsFailed]); helper specs + clean-passnotToThrowgreen8320573b9Throw-lock its that went red then green:
$throwIfCliTestsFailed— directoryRejected / bundlesDiscovered=0 / specsFailedToLoad>0 / totalFail>0 / totalError>0$throwIfBrowserTestsFailed— totalFail>0 / totalError>0notToThrowon both seams stayed green through both SHAsHEAD:
8320573b9a0cdb503dc07f5522b6c79c3c73a578Required CI checks
cli/lucli/**)