Break into the debugger upon error

doc/org.melusina.confidence.texinfo

@@ -11,7 +11,7 @@
 @end direntry
 
 @copying
-Confidence software and associated documentation is distributed
+@b{Confidence} software and associated documentation is distributed
 under the terms of the MIT License.
 
 @quotation
@@ -106,21 +106,21 @@ function. A special kind of function, but a function.
 
 @menu
 * Define Assertions::
-* Basic Assertions::
+* Comparison Assertions::
 * Condition Assertions::
 * Character Assertions::
 * String Assertions::
 * List Assertions::
 * Vector Assertions::
-* Floating Numbers Assertions::
+* Floating Number Assertions::
 @end menu
 
-@node Define Assertions, Basic Assertions, Assertions, Assertions
+@node Define Assertions, Comparison Assertions, Assertions, Assertions
 @section Define Assertions
 @include include/macro-org.melusina.confidence-define-assertion.texinfo
 
-@node Basic Assertions, Condition Assertions, Define Assertions, Assertions
-@section Basic Assertions
+@node Comparison Assertions, Condition Assertions, Define Assertions, Assertions
+@section Comparison Assertions
 
 @include include/fun-org.melusina.confidence-assert-t.texinfo
 @include include/fun-org.melusina.confidence-assert-t-star.texinfo
@@ -136,7 +136,7 @@ function. A special kind of function, but a function.
 @include include/fun-org.melusina.confidence-assert-lt-equals.texinfo
 @include include/fun-org.melusina.confidence-assert-gt-equals.texinfo
 
-@node Condition Assertions, Character Assertions, Basic Assertions, Assertions
+@node Condition Assertions, Character Assertions, Comparison Assertions, Assertions
 @section Condition Assertions
 
 @include include/macro-org.melusina.confidence-assert-condition.texinfo
@@ -169,13 +169,13 @@ function. A special kind of function, but a function.
 @include include/fun-org.melusina.confidence-assert-set-equal.texinfo
 @include include/fun-org.melusina.confidence-assert-subsetp.texinfo
 
-@node Vector Assertions, Floating Numbers Assertions, List Assertions, Assertions
+@node Vector Assertions, Floating Number Assertions, List Assertions, Assertions
 @section Vector Assertions
 
 @include include/fun-org.melusina.confidence-assert-vector-equal.texinfo
 
-@node Floating Numbers Assertions, Testcases, Vector Assertions, Assertions
-@section Floating Numbers Assertions
+@node Floating Number Assertions, Testcases, Vector Assertions, Assertions
+@section Floating Number Assertions
 
 @include include/var-org.melusina.confidence-star-double-float-precision-star.texinfo
 @include include/var-org.melusina.confidence-star-single-float-precision-star.texinfo
@@ -184,24 +184,83 @@ function. A special kind of function, but a function.
 @include include/fun-org.melusina.confidence-assert-float-is-definitely-greater-than.texinfo
 @include include/fun-org.melusina.confidence-assert-float-is-definitely-less-than.texinfo
 
-@node Testcases, Specialities, Floating Numbers Assertions, Top
+@node Testcases, Specialities, Floating Number Assertions, Top
 @chapter Testcases
 
-The @i{define-testcase} macro allows to define testcases, which are
-functions specially crafted to run tests.  There are two differences
-with regular functions. The first difference is that calls to
-testcases and assertions in a testcase are instrumented, so that a
-global success report is produced. The second difference is that when
-a testcase is run as a toplevel form in a batch environment (not at a
-REPL) the testcase prints a summary of results when done and exits the
-program with a success status reflecting the testcase result.
+The @i{define-testcase} macro allows to define @i{testcases}, which
+are functions specially crafted to run @i{assertions} and build a
+testsuite.  There are three important differences with regular
+functions. The first one is that calls to assertions in a testcase are
+instrumented, so that a global success report is produced. The second
+difference is that when a testcase is run as a toplevel form the
+testcase prints a summary of results when done. The third difference
+is that some specific @i{restarts} are available in this context.
+
+@section Define a Testcase
+
+@include include/macro-org.melusina.confidence-define-testcase.texinfo
+@include include/class-org.melusina.confidence-testcase-outcome.texinfo
+
+@section Describe Failed Assertions
+
+When an assertion fails, it prints a description on the
+@code{*error-output*}, unless the following configuration variable
+is set to @code{nil}.
+
+@include include/var-org.melusina.confidence-star-testcase-describe-failed-assertions-star.texinfo
+
+
+@section Run Testcases in Batch
+
+The @b{Confidence} system normally runs test cases in batch, which is
+appropriate when it is used for fully automated tests, in continuous
+integration and delivery pipelines.
+
+@section Run Testcases Interactively
+
+The @b{Confidence} system can be configured so that when a testcase
+meets an unexpected condition, such as an error or an unsatisfied
+assertion, the debugger is invoked. To do so, modify the variable
+@code{*testcase-break-into-the-debugger-on-errors*} either directly
+or with the function @code{testcase-break-into-the-debugger-on-errors}.
+
+@include include/var-org.melusina.confidence-star-testcase-break-into-the-debugger-on-errors-star.texinfo
+@include include/fun-org.melusina.confidence-testcase-break-into-the-debugger-on-errors.texinfo
+
+When this configuration is active, the following @i{restarts} are
+available in the debugger:
+
+@itemize
+@item
+@b{ASSERTION-RETRY} Which retries to evaluate the arguments of the
+failing assertion and the assertion itself.
+@item
+@b{TESTCASE-RETRY} Which retries the current testcase.
+@item
+@b{TESTCASE-CONTINUE} Which registers an error and continues the
+current testcase.
+@item
+@b{TESTCASE-RETURN} Which registers an error and immediately returns
+from the current testcase.
+@item
+@b{TESTCASE-STEP-UP} Which registers an error and continue the current
+testcase without breaking into the debugger.
+@item
+@b{TESTCASE-SCROLL} Which registers an error and continue the current
+testcase and other testcases higher in the call stack without breaking
+into the debugger.
+@end itemize
+
+@section Add Testcases to the Export List of a Package
 
 Defined testcases are automatically exported, which makes it easy to
 call them from the REPL, the testsuite tool or to add them to
-generated documentation.
+generated documentation.  It could however be desirable to explciitly
+add them to the export list of a package.  To support this, the
+@b{Confidence} system exports the following function:
+
+@include include/fun-org.melusina.confidence-print-export-list-for-testcases.texinfo
 
-@include include/var-org.melusina.confidence-star-testcase-interactive-p-star.texinfo
-@include include/macro-org.melusina.confidence-define-testcase.texinfo
 
 @node Specialities, , Testcases, Top
 @chapter Specialities
@@ -218,7 +277,7 @@ test resources is just working as expected.
 
 It is simple enough to define a with-macro in Common Lisp so that
 @b{Confidence} does not provide an extra feature to do so. Just mind
-the `unwind-protect` operator.
+the @code{unwind-protect} operator.
 
 
 @section Test Hierarchies
@@ -240,31 +299,30 @@ functional tests journeys, adding fixtures to the defined testsuite.
 Having a testcase explicitly listing the other testcases it must run
 is a simple and reliable organisation that never lies to the
 programmer.  It also makes the test hierarchy navigable in IDEs
-without extra tooling, which is a good thing.
+without supplementary tooling.
 
 
 @section Test Tags
 
 Some test frameworks provide a tag feature so that it possible to run
-tests according to some tags attached to them.
-
-Using tags to decorate tests that need to be run in such or such
-context, similarly to what popular Java test frameworks propose,
-obfuscates the test workflow and nobody wants to discover after
-several weeks of operation that some tests were not run by the
-deployment pipeline.
+tests according to some tags attached to them.  Some popular Java
+testing frameworks offer such a tag feature. Relying on this to decide
+which tests to run obfuscates the test workflow and makes it hard to
+understand which tests are run in a specific context.
 
 Having a testcase explicitly listing the other testcases it must run
 is a simple and reliable organisation that never lies to the
 programmer.  It also makes the test hierarchy navigable in IDEs
-without extra tooling, which is a good thing.
+without extra tooling.
 
 
 @section Test Selection
 
 When some tests are only relevant to some platform, or must be be
 controlled by a feature, it is very easy to control their inclusion in
 the testsuite using the feature-related reader macros.  It does not
-seem necessary for @b{Confidence} to provide extra tools here.
+seem necessary for @b{Confidence} to provide special features as the
+usual Common Lisp operstors @code{when}, @code{unless} and
+@code{return-from} can be used here.
 
 @bye

src/package.lisp

@@ -23,6 +23,7 @@
    #:testcase-success
    #:testcase-failure
    #:testcase-condition
+   #:testcase-argument-values
    ;; Testcases
    #:assertion-path
    #:assertion-name
@@ -39,7 +40,10 @@
    #:*testsuite-name*
    #:*testsuite-id*
    #:*testsuite-last-result*
+   #:*testcase-break-into-the-debugger-on-errors*
+   #:testcase-break-into-the-debugger-on-errors
    #:list-testcases
+   #:print-export-list-for-testcases
    ;; Assertions
    #:define-assertion
    #:list-assertions