Skip to content

Commit

Permalink
Print a form suitable for exporting testcases defined by a package
Browse files Browse the repository at this point in the history
  • Loading branch information
foretspaisibles committed Apr 13, 2024
1 parent 98b0b68 commit d33f8cf
Show file tree
Hide file tree
Showing 4 changed files with 64 additions and 24 deletions.
38 changes: 20 additions & 18 deletions doc/org.melusina.confidence.texinfo
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ function. A special kind of function, but a function.

@menu
* Define Assertions::
* Basic Assertions::
* Comparison Assertions::
* Condition Assertions::
* Character Assertions::
* String Assertions::
Expand All @@ -115,12 +115,12 @@ function. A special kind of function, but a function.
* Floating Number Assertions::
@end menu

@node Define Assertions, Basic Assertions, Assertions, Assertions
@node Define Assertions, Comparison Assertions, Assertions, Assertions
@section Define Assertions
@include include/macro-org.melusina.confidence-define-assertion.texinfo

@node Basic Assertions, Condition Assertions, Define Assertions, Assertions
@section Basic Assertions
@node Comparison Assertions, Condition Assertions, Define Assertions, Assertions
@section Comparison Assertions

@include include/fun-org.melusina.confidence-assert-t.texinfo
@include include/fun-org.melusina.confidence-assert-t-star.texinfo
Expand All @@ -136,7 +136,7 @@ function. A special kind of function, but a function.
@include include/fun-org.melusina.confidence-assert-lt-equals.texinfo
@include include/fun-org.melusina.confidence-assert-gt-equals.texinfo

@node Condition Assertions, Character Assertions, Basic Assertions, Assertions
@node Condition Assertions, Character Assertions, Comparison Assertions, Assertions
@section Condition Assertions

@include include/macro-org.melusina.confidence-assert-condition.texinfo
Expand Down Expand Up @@ -255,9 +255,12 @@ into the debugger.

Defined testcases are automatically exported, which makes it easy to
call them from the REPL, the testsuite tool or to add them to
generated documentation.
generated documentation. It could however be desirable to explciitly
add them to the export list of a package. To support this, the
@b{Confidence} system exports the following function:

@include include/fun-org.melusina.confidence-print-export-list-for-testcases.texinfo

XXXX

@node Specialities, , Testcases, Top
@chapter Specialities
Expand All @@ -274,7 +277,7 @@ test resources is just working as expected.

It is simple enough to define a with-macro in Common Lisp so that
@b{Confidence} does not provide an extra feature to do so. Just mind
the `unwind-protect` operator.
the @code{unwind-protect} operator.


@section Test Hierarchies
Expand All @@ -296,31 +299,30 @@ functional tests journeys, adding fixtures to the defined testsuite.
Having a testcase explicitly listing the other testcases it must run
is a simple and reliable organisation that never lies to the
programmer. It also makes the test hierarchy navigable in IDEs
without extra tooling, which is a good thing.
without supplementary tooling.


@section Test Tags

Some test frameworks provide a tag feature so that it possible to run
tests according to some tags attached to them.

Using tags to decorate tests that need to be run in such or such
context, similarly to what popular Java test frameworks propose,
obfuscates the test workflow and nobody wants to discover after
several weeks of operation that some tests were not run by the
deployment pipeline.
tests according to some tags attached to them. Some popular Java
testing frameworks offer such a tag feature. Relying on this to decide
which tests to run obfuscates the test workflow and makes it hard to
understand which tests are run in a specific context.

Having a testcase explicitly listing the other testcases it must run
is a simple and reliable organisation that never lies to the
programmer. It also makes the test hierarchy navigable in IDEs
without extra tooling, which is a good thing.
without extra tooling.


@section Test Selection

When some tests are only relevant to some platform, or must be be
controlled by a feature, it is very easy to control their inclusion in
the testsuite using the feature-related reader macros. It does not
seem necessary for @b{Confidence} to provide extra tools here.
seem necessary for @b{Confidence} to provide special features as the
usual Common Lisp operstors @code{when}, @code{unless} and
@code{return-from} can be used here.

@bye
5 changes: 3 additions & 2 deletions src/package.lisp
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,10 @@
#:*testsuite-name*
#:*testsuite-id*
#:*testsuite-last-result*
#:list-testcases
#:testcase-break-into-the-debugger-on-errors
#:*testcase-break-into-the-debugger-on-errors*
#:testcase-break-into-the-debugger-on-errors
#:list-testcases
#:print-export-list-for-testcases
;; Assertions
#:define-assertion
#:list-assertions
Expand Down
10 changes: 8 additions & 2 deletions src/testcase.lisp
Original file line number Diff line number Diff line change
Expand Up @@ -639,11 +639,17 @@ of tests that ran and their outcomes."
(export (quote ,testcase-name))
(set-testcase-properties ',testcase-name))))

(defun list-testcases (&optional package-designator)
(defun list-testcases (&optional (package-designator *package*))
"List testcases exported by PACKAGE-DESIGNATOR."
(loop :for s :being :the :external-symbols :of (find-package package-designator)
:when (get s :org.melusina.confidence/testcase)
:collect s))
:collect s :into testcases
:finally (return (sort testcases #'string< :key #'symbol-name))))

(defun print-export-list-for-testcases (&optional (package-designator *package*))
"Print a form suitable for exporting TESTCASES defiend by PACKAGE-DESIGNATOR."
(loop :for testcase :in (list-testcases package-designator)
:do (format t "~& #:~A~%" (string-downcase (symbol-name testcase)))))

(defun quit ()
"Quit the SBCL lisp image and set exit status accordingly."
Expand Down
35 changes: 33 additions & 2 deletions testsuite/package.lisp
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,39 @@
(:local-nicknames (#:confidence #:org.melusina.confidence))
(:use #:common-lisp #:org.melusina.confidence)
(:export
#:list-available-tests
))
#:a-compound-failing-testsuite
#:a-failing-argument-testsuite
#:a-failing-testcase-testsuite
#:a-simple-failure
#:a-successful-testsuite
#:a-successful-testsuite-with-function-calls
#:ensure-that-define-testcase-recognises-sharpsign-single-quote-in-function-names
#:ensure-that-testcase-is-reported-when-wrapped-in-flet
#:interactive-assertion-count
#:interactive-testcase-extensivity
#:interactive-testcase-extensivity-1
#:interactive-testcase-extensivity-2
#:perform-many-assertions
#:perform-many-assertions-wrapped-with-flet
#:run-all-tests
#:run-interactive-tests
#:testsuite-assert-char*
#:testsuite-assert-condition
#:testsuite-assert-float*
#:testsuite-assert-list*
#:testsuite-assert-string*
#:testsuite-assert-vector*
#:testsuite-assertion
#:testsuite-basic-assertions
#:testsuite-define-assertion
#:testsuite-list-as-set
#:testsuite-result
#:testsuite-string-match
#:testsuite-testcase
#:testsuite-utilities
#:validate-define-testcase
#:validate-outcome-can-be-described
#:validate-supervise-assertion))

(in-package #:org.melusina.confidence/testsuite)

Expand Down

0 comments on commit d33f8cf

Please sign in to comment.