diff --git a/docs/assert/expect.md b/docs/assert/expect.md index 6819584ed..a6fbd02c9 100644 --- a/docs/assert/expect.md +++ b/docs/assert/expect.md @@ -1,7 +1,7 @@ --- layout: page-api title: assert.expect() -excerpt: Specify how many assertions are expected to run within a test. +excerpt: Specify how many assertions are expected in a test. groups: - assert redirect_from: @@ -11,20 +11,61 @@ version_added: "1.0.0" `expect( amount )` -Specify how many assertions are expected to run within a test. +Specify how many assertions are expected in a test. | name | description | |------|-------------| -| `amount` | Number of assertions in this test. | +| `amount` | Number of expected assertions in this test. | -To ensure that an explicit number of assertions are run within any test, use `assert.expect()` to register an expected count. If the number of assertions run does not match the expected count, the test will fail. +This is most commonly used as `assert.expect(0)`, which indicates that a test may pass making any assertions. This means the test is only used to verify that the code to completion without any uncaught errors. This is is essentially the inverse of [`assert.throws()`](./throws.md). + +It can also be used to explicitly require a certain number of assertions to be recorded in a given test. If afterwards the number of assertions does not match the expected count, the test will fail. + +It is recommended to test asynchronous code with [`assert.step()`](./step.md) or [`assert.async()`](./async.md) instead. ## Examples -Establish an expected assertion count +### Example: No assertions + +A test without any assertions: + +```js +QUnit.test('example', function (assert) { + assert.expect(0); + + var android = new Robot(); + android.up(2); + android.down(2); + android.left(); + android.right(); + android.left(); + android.right(); + android.attack(); + android.jump(); +}); +``` + +### Example: Custom assert + +If you use a generic assertion library that throws when an expectation is not met, you can use `assert.expect(0)` if there are no other assertions needed in the test. + +```js +QUnit.test('example', function (assert) { + assert.expect(0); + + var android = new Robot(database); + android.run(); + + database.assertNoOpenConnections(); +}); +``` + +### Example: Explicit count + +Require an explicit assertion count. ```js -QUnit.test('a test', function (assert) { +QUnit.test('example', function (assert) { assert.expect(2); function calc (x, operation) { diff --git a/docs/callbacks/QUnit.begin.md b/docs/callbacks/QUnit.begin.md index 70a556420..2b8cc9c91 100644 --- a/docs/callbacks/QUnit.begin.md +++ b/docs/callbacks/QUnit.begin.md @@ -1,7 +1,7 @@ --- layout: page-api title: QUnit.begin() -excerpt: Register a callback to fire whenever the test suite begins. +excerpt: Register a callback to fire when the test run begins. groups: - callbacks redirect_from: @@ -11,21 +11,25 @@ version_added: "1.0.0" `QUnit.begin( callback )` -Register a callback to fire whenever the test suite begins. The callback may be an async function, or a function that returns a promise, which will be waited for before the next callback is handled. +Register a callback to fire when the test run begins. The callback may be an async function, or a function that returns a Promise, which will be waited for before the next callback is handled. The callback will be called once, before QUnit runs any tests. | parameter | description | |-----------|-------------| -| callback (function) | Callback to execute. Provides a single argument with the callback Details object | +| `callback` (function) | Callback to execute, called with a `details` object. | ### Details object -Passed to the callback: - | property | description | |-----------|-------------| -| `totalTests` | The number of total tests in the test suite | +| `totalTests` (number) | Number of registered tests | +| `modules` (array) | List of registered modules,
as `{ name: string }` objects. | + +## Changelog + +| [QUnit 1.16](https://github.com/qunitjs/qunit/releases/tag/1.16.0) | Added `details.modules` property, containing `{ name: string }` objects. +| [QUnit 1.15](https://github.com/qunitjs/qunit/releases/tag/1.15.0) | Added `details.totalTests` property. ## Examples diff --git a/docs/callbacks/QUnit.done.md b/docs/callbacks/QUnit.done.md index 1281706e2..550ee9f75 100644 --- a/docs/callbacks/QUnit.done.md +++ b/docs/callbacks/QUnit.done.md @@ -1,7 +1,7 @@ --- layout: page-api title: QUnit.done() -excerpt: Register a callback to fire whenever the test suite ends. +excerpt: Register a callback to fire when the test run has ended. groups: - callbacks redirect_from: @@ -11,37 +11,42 @@ version_added: "1.0.0" `QUnit.done( callback )` -Register a callback to fire whenever the test suite ends. The callback may be an async function, or a function that return a promise which will be waited for before the next callback is handled. +Register a callback to fire when the test run has ended. The callback may be an async function, or a function that return a Promise which will be waited for before the next callback is handled. | parameter | description | |-----------|-------------| -| callback (function) | Callback to execute. Provides a single argument with the callback Details object | +| `callback` (function) | Callback to execute, called with a `details` object: ### Details object -Passed to the callback: - | property | description | |-----------|-------------| -| `failed` (number) | The number of failed assertions | -| `passed` (number) | The number of passed assertions | -| `total` (number) | The total number of assertions | -| `runtime` (number) | The time in milliseconds it took tests to run from start to finish. | +| `failed` (number) | Number of failed assertions | +| `passed` (number) | Number of passed assertions | +| `total` (number) | Total number of assertions | +| `runtime` (number) | Duration of the test run in milliseconds | -## Examples +
-Register a callback that logs test results to the console. +Use of `details` is __deprecated__ and it's recommended to use [`QUnit.on('runEnd')`](./QUnit.on.md) instead. -```js -QUnit.done(details => { - console.log( - `Total: ${details.total} Failed: ${details.failed} ` + - `Passed: ${details.passed} Runtime: ${details.runtime}` - ); -}); -``` +Caveats: + +* This callback reports the **internal assertion count**. + +* The default browser and CLI interfaces for QUnit and other popular test frameworks, and most CI integrations, report the number of tests. Reporting the number _assertions_ may be confusing to developers. + +* Failed assertions of a [`test.todo()`](../QUnit/test.todo.md) test are reported exactly as such. While rare, this means that a test run and all tests within it may be reported as passing, while internally there were some failed assertions. Unfortunately, this internal detail is exposed for compatibility reasons. + +
+ +## Changelog + +| [QUnit 2.2](https://github.com/qunitjs/qunit/releases/tag/2.2.0) | Deprecate `details` parameter in favour of `QUnit.on('runEnd')`. + +## Examples -Using classic ES5 syntax: +Register a callback that logs internal assertion counts. ```js QUnit.done(function (details) { diff --git a/docs/callbacks/QUnit.log.md b/docs/callbacks/QUnit.log.md index fc3fab450..cba185f1c 100644 --- a/docs/callbacks/QUnit.log.md +++ b/docs/callbacks/QUnit.log.md @@ -13,8 +13,6 @@ version_added: "1.0.0" Register a callback to fire whenever an assertion completes. -This is one of several callbacks QUnit provides. It's intended for continuous integration scenarios. - **NOTE: The QUnit.log() callback does not handle promises and MUST be synchronous.** | parameter | description | diff --git a/docs/callbacks/QUnit.on.md b/docs/callbacks/QUnit.on.md index 2e48424d7..cdc7cba61 100644 --- a/docs/callbacks/QUnit.on.md +++ b/docs/callbacks/QUnit.on.md @@ -9,20 +9,127 @@ version_added: "2.2.0" `QUnit.on( eventName, callback )` -Register a callback to fire whenever the specified event is emitted. Conforms to the [js-reporters standard](https://github.com/js-reporters/js-reporters). +Register a callback to fire whenever a specified event is emitted. -Use this to listen for events related to the test suite's execution. Available event names and corresponding data payloads are defined in the [js-reporters specification](https://github.com/js-reporters/js-reporters). +This API implements the [js-reporters CRI standard](https://github.com/js-reporters/js-reporters/blob/v2.1.0/spec/cri-draft.adoc), and is the primary interface for use by continuous integration plugins and other reporting software. -**NOTE: The QUnit.on() callback does not handle promises and MUST be synchronous.** +| type | parameter | description +|--|--|-- +| `string` | `eventName` | Name of an event. +| `Function` | `callback`| A callback function. -| parameter | description | -|-----------|-------------| -| eventName (string) | The name of the event for which to execute the provided callback. | -| callback (function) | Callback to execute. Receives a single argument representing the data for the event. | +## The `runStart` event -## Examples +The `runStart` event indicates the beginning of a test run. It is emitted exactly once, and before any other events. -Printing results of a test suite. +| `Object` | `testCounts` | Aggregate counts about tests. +| `number` | `testCounts.total` | Total number of registered tests. + +```js +QUnit.on('runStart', runStart => { + console.log(`Test plan: ${runStart.testCounts.total}`); +}); +``` +## The `suiteStart` event + +The `suiteStart` event indicates the beginning of a module. It is eventually be followed by a corresponding `suiteEnd` event. + +| `string` | `name` | Name of the module. +| `Array` | `fullName`| List of one or more strings, containing (in order) the names of any ancestor modules and the name of the current module. + +```js +QUnit.on('suiteStart', suiteStart => { + console.log('suiteStart', suiteStart); + // name: 'my module' + // fullName: ['grandparent', 'parent', 'my module'] +}); +``` + +## The `suiteEnd` event + +The `suiteEnd` event indicates the end of a module. It is emitted after its corresponding `suiteStart` event. + +| `string` | `name` | Name of the module. +| `Array` | `fullName`| List of one or more strings, containing (in order) the names of any ancestor modules and the name of the current module. +| `string` | `status` | Aggregate result of tests in this module, one of:
`failed`: at least one test has failed;
`passed`: there were no failing tests, which means there were only tests with a passed, skipped, or todo status. +| `number` | `runtime` | Duration of the module in milliseconds. + +```js +QUnit.on('suiteEnd', suiteEnd => { + console.log(suiteEnd); + // … +}); +``` + +## The `testStart` event + +The `testStart` event indicates the beginning of a test. It is eventually followed by a corresponding `testEnd` event. + +| `string` | `name` | Name of the test. +| `string|null` | `moduleName` | The module the test belongs to, or null for a global test. +| `Array` | `fullName` | List (in order) of the names of any ancestor modules and the name of the test itself. + +```js +QUnit.on('testStart', testStart => { + console.log(testStart); + // name: 'my test' + // moduleName: 'my module' + // fullName: ['parent', 'my module', 'my test'] + + // name: 'global test' + // moduleName: null + // fullName: ['global test'] +}); +``` + +## The `testEnd` event + +The `testEnd` event indicates the end of a test. It is emitted after its corresponding `testStart` event. + +Properties of a testEnd object: + +| `string` | `name` | Name of the test. +| `string|null` | `moduleName` | The module the test belongs to, or null for a global test. +| `Array` | `fullName` | List (in order) of the names of any ancestor modules and the name of the test itself. +| `string` | `status` | Result of the test, one of:
`passed`: all assertions passed or no assertions found;
`failed`: at least one assertion failed or it is a [todo test](../QUnit/test.todo.md) that no longer has any failing assertions;
`skipped`: the test was intentionally not run; or
`todo`: the test is "todo" and still has a failing assertion. +| `number` | `runtime` | Duration of the test in milliseconds. +| `Array` | `errors` | For tests with status `failed` or `todo`, there will be at least one failed assertion. However, the list may be empty if the status is `failed` due to a "todo" test having no failed assertions.

Note that all negative test outcome communicate their details in this manner. For example, timeouts, uncaught errors, and [global pollution](../config/noglobals.md) also synthesize a failed assertion. + +Properties of a FailedAssertion object: + +| `boolean` | `passed` | False for a failed assertion. +| `string|undefined` | `message` | Description of what the assertion checked. +| `any` | `actual` | The actual value passed to the assertion. +| `any` | `expected` | The expected value passed to the assertion. +| `string|undefined` | `stack` | Stack trace, may be undefined if the result came from an old web browsers. + +```js +QUnit.on('testEnd', testEnd => { + if (testEnd.status === 'failed') { + console.error('Failed! ' + testEnd.fullName.join(' > ')); + testEnd.errors.forEach(assertion => { + console.error(assertion); + // message: speedometer + // actual: 75 + // expected: 88 + // stack: at dmc.test.js:12 + }); + } +}); +``` + +## The `runEnd` event + +The `runEnd` event indicates the end of a test run. It is emitted exactly once. + +| `string` | `status` | Aggregate result of all tests, one of:
`failed`: at least one test failed or a global error ocurred;
`passed`: there were no failed tests, which means there were only tests with a passed, skipped, or todo status. If [`QUnit.config.failOnZeroTests`](../config/failOnZeroTests.md) is disabled, then the run may also pass if there were no tests. +| `Object` | `testCounts` | Aggregate counts about tests: +| `number` | `testCounts.passed` | Number of passed tests. +| `number` | `testCounts.failed` | Number of failed tests. +| `number` | `testCounts.skipped` | Number of skipped tests. +| `number` | `testCounts.todo` | Number of todo tests. +| `number` | `testCounts.total` | Total number of tests, equal to the sum of the above properties. +| `number` | `runtime` | Total duration of the run in milliseconds. ```js QUnit.on('runEnd', runEnd => { diff --git a/docs/extension/QUnit.dump.parse.md b/docs/extension/QUnit.dump.parse.md index 7666eaf5c..c5cf61bfc 100644 --- a/docs/extension/QUnit.dump.parse.md +++ b/docs/extension/QUnit.dump.parse.md @@ -3,7 +3,7 @@ layout: page-api title: QUnit.dump.parse() excerpt: Extensible data dumping and string serialization. groups: - - extension + - extension redirect_from: - "/QUnit.dump.parse/" - "/QUnit.jsDump.parse/" diff --git a/docs/extension/QUnit.extend.md b/docs/extension/QUnit.extend.md index 8e5a42d9d..dad6fac8e 100644 --- a/docs/extension/QUnit.extend.md +++ b/docs/extension/QUnit.extend.md @@ -3,8 +3,8 @@ layout: page-api title: QUnit.extend() excerpt: Copy the properties from one object into a target object. groups: -- extension -- deprecated + - extension + - deprecated redirect_from: - "/config/QUnit.extend/" version_added: "1.0.0" @@ -15,13 +15,13 @@ version_deprecated: "2.12.0" Copy the properties defined by a mixin object into a target object. +

This method is __deprecated__ and it's recommended to use [`Object.assign()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/assign) instead.

+ | name | description | |------|-------------| | `target` | An object whose properties are to be modified | | `mixin` | An object describing which properties should be modified | -

This method is __deprecated__ and it's recommended to use [`Object.assign()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/assign) instead.

- This method will modify the `target` object to contain the "own" properties defined by the `mixin`. If the `mixin` object specifies the value of any attribute as `undefined`, this property will instead be removed from the `target` object. ## Examples diff --git a/docs/extension/QUnit.push.md b/docs/extension/QUnit.push.md index 517a6e80b..8e28f16d5 100644 --- a/docs/extension/QUnit.push.md +++ b/docs/extension/QUnit.push.md @@ -3,8 +3,8 @@ layout: page-api title: QUnit.push() excerpt: Report the result of a custom assertion. groups: -- extension -- deprecated + - extension + - deprecated redirect_from: - "/config/QUnit.push/" version_added: "1.0.0" @@ -15,6 +15,8 @@ version_deprecated: "2.1.0" Report the result of a custom assertion. +

This method is __deprecated__ and it's recommended to use [`pushResult`](../assert/pushResult.md) in the assertion context instead.

+ | name | description | |------|-------------| | `result` (boolean) | Result of the assertion | @@ -22,8 +24,6 @@ Report the result of a custom assertion. | `expected` | Known comparison value | | `message` (string) | A short description of the assertion | -

This method is __deprecated__ and it's recommended to use [`pushResult`](../assert/pushResult.md) in the assertion context instead.

- `QUnit.push` reflects to the current running test, and it may leak assertions in asynchronous mode. Checkout [`assert.pushResult()`](../assert/pushResult.md) to set a proper custom assertion. Invoking `QUnit.push` allows to create a readable expectation that is not defined by any of QUnit's built-in assertions. diff --git a/docs/extension/QUnit.stack.md b/docs/extension/QUnit.stack.md index 972527003..9c0e10986 100644 --- a/docs/extension/QUnit.stack.md +++ b/docs/extension/QUnit.stack.md @@ -3,7 +3,7 @@ layout: page-api title: QUnit.stack() excerpt: Return a single line string representing the stacktrace. groups: -- extension + - extension redirect_from: - "/config/QUnit.stack/" version_added: "1.19.0" diff --git a/src/assert.js b/src/assert.js index 2bc73c06e..8685a9a94 100644 --- a/src/assert.js +++ b/src/assert.js @@ -58,8 +58,6 @@ class Assert { this.test.steps.length = 0; } - // Specify the number of expected assertions to guarantee that failed test - // (no assertions are run at all) don't slip through. expect (asserts) { if (arguments.length === 1) { this.test.expected = asserts; diff --git a/src/core.js b/src/core.js index aed626e66..35336c789 100644 --- a/src/core.js +++ b/src/core.js @@ -179,6 +179,10 @@ export function begin () { for (let i = 0; i < l; i++) { modulesLog.push({ name: config.modules[i].name, + + // Added in QUnit 1.16.0 for internal use by html-reporter, + // but no longer used since QUnit 2.7.0. + // @deprecated Kept unofficially to be removed in QUnit 3.0. tests: config.modules[i].tests }); } diff --git a/src/core/processing-queue.js b/src/core/processing-queue.js index bf23da93d..4e15ba16c 100644 --- a/src/core/processing-queue.js +++ b/src/core/processing-queue.js @@ -190,6 +190,9 @@ function done () { emit('runEnd', runSuite.end(true)); runLoggingCallbacks('done', { + // @deprecated since 2.19.0 Use done() without `details` parameter, + // or use `QUnit.on('runEnd')` instead. Parameter to be replaced in + // QUnit 3.0 with test counts. passed, failed: config.stats.bad, total: config.stats.all,