Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
211 commits
Select commit Hold shift + click to select a range
c8d933f
chore(deps-dev): bump playwright from 1.53.0 to 1.54.1 (#5043)
dependabot[bot] Jul 16, 2025
4ef395f
chore(deps-dev): bump @eslint/js from 9.30.0 to 9.31.0 (#5044)
dependabot[bot] Jul 16, 2025
7c764fc
chore(deps-dev): bump expect from 29.7.0 to 30.0.4 (#5036)
dependabot[bot] Jul 16, 2025
6b45b27
chore(deps-dev): bump graphql from 16.10.0 to 16.11.0 (#5049)
dependabot[bot] Jul 22, 2025
a130ba3
chore(deps-dev): bump electron from 37.1.0 to 37.2.3 (#5047)
dependabot[bot] Jul 22, 2025
2aef872
chore(deps-dev): bump puppeteer from 24.8.0 to 24.15.0 (#5054)
dependabot[bot] Jul 30, 2025
af27fc9
chore(deps): bump axios from 1.8.4 to 1.11.0 (#5055)
dependabot[bot] Aug 7, 2025
2ad213a
chore(deps-dev): bump typedoc-plugin-markdown from 4.7.0 to 4.7.1 (#5…
dependabot[bot] Aug 7, 2025
4cc3c84
chore(deps): bump browser-actions/setup-chrome from 1 to 2 (#5041)
dependabot[bot] Aug 7, 2025
e090eb9
chore(deps-dev): bump expect from 30.0.4 to 30.0.5 (#5064)
dependabot[bot] Aug 13, 2025
220b73f
fix: 5070 issue (#5071)
kobenguyent Aug 19, 2025
5c14df4
chore(deps-dev): bump typedoc from 0.28.7 to 0.28.10 (#5059)
dependabot[bot] Aug 19, 2025
a27b99e
fix: hook exit code (#5058)
kobenguyent Aug 19, 2025
3cf5fe2
5066 unable to inject data between workers because of proxy object (#…
kobenguyent Aug 19, 2025
5476a9d
chore(deps): bump actions/checkout from 4 to 5 (#5062)
dependabot[bot] Aug 19, 2025
3408489
chore(deps-dev): bump tsd from 0.32.0 to 0.33.0 (#5063)
dependabot[bot] Aug 19, 2025
eceb2a3
chore(deps-dev): bump typedoc-plugin-markdown from 4.7.1 to 4.8.1 (#5…
dependabot[bot] Aug 19, 2025
4daabc8
feat: shuffle test suite order in run command (#5051)
NivYarmus Aug 19, 2025
ff63f70
fix: sessions playwright traces - naming convention and error handlin…
kobenguyent Aug 19, 2025
a483106
release 3.7.4 (#5075)
kobenguyent Aug 19, 2025
2e4d0bb
update docs
kobenguyent Aug 19, 2025
376c758
fix: Update inquirer (#5076)
kobenguyent Aug 19, 2025
f0759ca
add docs
kobenguyent Aug 20, 2025
04ec768
feat(cli): make test file hyperlink (#5078)
kobenguyent Aug 20, 2025
e3a195d
Playwright: I.waitForText() causes unexpected delay equal to `waitFor…
Copilot Aug 20, 2025
93ab75d
3.7.3 I.seeResponseContainsJson not working (#5081)
Copilot Aug 20, 2025
58f83a1
Fix JUnit XML test case name inconsistency when using scenario retrie…
Copilot Aug 20, 2025
f295302
fix: testcafe workflow failed (#5085)
kobenguyent Aug 21, 2025
ce16e92
[FR] - Support feature.only like Scenario.only (#5087)
Copilot Aug 21, 2025
2047cf2
Fix tryTo steps appearing in test failure traces (#5088)
Copilot Aug 22, 2025
0797716
feat: Introduce CodeceptJS WebElement Class to mirror chosen helpers'…
Copilot Aug 22, 2025
75d98de
DOC: Autogenerate and update documentation
actions-user Aug 22, 2025
b27e9cf
Fix waitForText timeout regression in Playwright helper (#5093)
Copilot Aug 22, 2025
c95f78d
fix: missing module 'codeceptjs/effects' (#5094)
kobenguyent Aug 22, 2025
de21279
[WIP] [FEATURE REQUEST](puppeteer) migrate locators from ElementHandl…
Copilot Aug 23, 2025
cd6fec7
DOC: Autogenerate and update documentation
actions-user Aug 23, 2025
273a63e
Fix test statistics reporting issue in pool mode - consolidate result…
Copilot Aug 23, 2025
5598d39
Fix mocha retries losing CodeceptJS-specific properties (opts, tags, …
Copilot Aug 23, 2025
a52bba7
Test Sharding for CI Matrix Purposes with GitHub Workflows (#5098)
Copilot Aug 23, 2025
5535d16
API test server to run unit tests, acceptance tests for codeceptjs wi…
Copilot Aug 23, 2025
0a0067f
Enable HTML reporter by default in new CodeceptJS projects with compr…
Copilot Aug 24, 2025
f02280c
fix: use platformName for mobile click detection instead of isW3C (An…
mirao Aug 24, 2025
21cf8b6
[WIP] Bug: built in HTML reporter missing test stats when running run…
Copilot Aug 26, 2025
e94fc98
Fix: JSONResponse helper to preserve original onResponse behavior (Fi…
myrepojuly Aug 26, 2025
88509d1
feat: Add configurable sensitive data masking with custom patterns (#…
Copilot Aug 26, 2025
663be8d
chore(deps-dev): bump @eslint/js from 9.31.0 to 9.34.0 (#5112)
dependabot[bot] Aug 26, 2025
2e814fd
Add Custom Strategy Locators support to Playwright helper (#5090)
Copilot Aug 27, 2025
07a58e5
DOC: Autogenerate and update documentation
actions-user Aug 27, 2025
773ba14
fix(playwright): relaunch browser correctly with `restart: 'session'`…
Samuel-StO Aug 28, 2025
4270372
[WIP] CodeceptJS 3.7.x: "tryTo" and "session" do not work together (#…
Copilot Sep 1, 2025
bc1c500
[WIP] Popups triggered during a test execution are not isolated betwe…
Copilot Sep 1, 2025
73960e8
[WIP] fix Circular reference in objects (#5123)
Copilot Sep 1, 2025
4f7a936
[WIP] Retries mechanism improvements (#5103)
Copilot Sep 1, 2025
15a288e
chore(deps-dev): bump playwright from 1.54.1 to 1.55.0 (#5114)
dependabot[bot] Sep 1, 2025
a396c5b
Bugfix: Properly stop network traffic recording (#5127)
Samuel-StO Sep 2, 2025
fc955e6
Bump electron to 38.0.0 (#5141)
thomashohn Sep 12, 2025
ddf2a4c
Update axios to 1.12.1 (#5147)
thomashohn Sep 15, 2025
8e521ce
[SECURITY] Fix tmp package to 0.25.0 (#5151)
thomashohn Sep 15, 2025
cd062e6
Bump @codeceptjs/detox-helper (#5153)
thomashohn Sep 15, 2025
d833e6e
Update of minor versions (#5149)
thomashohn Sep 15, 2025
d1261df
Bump apollo/server ^5 (#5145)
thomashohn Sep 15, 2025
75386a1
Bump actions/stale to 10 and actions/setup-node to 5 (#5155)
thomashohn Sep 15, 2025
7da6b79
Bump @cucumber/gherkin to 35.0.0 and @cucumber/messages to 29.0.1 (#5…
thomashohn Sep 15, 2025
8be396e
Bump electron to 38.1.0 (#5168)
thomashohn Sep 15, 2025
f29a311
Update more minors (#5167)
thomashohn Sep 15, 2025
c6946ae
Update @eslint/js to 9.35.0 (#5161)
thomashohn Sep 15, 2025
0cdb842
Bump globals to 16.3.0 (#5157)
thomashohn Sep 15, 2025
e2bb7e3
Bump mocha to 11.7.2 (#5159)
thomashohn Sep 15, 2025
ba053f0
Bump acorn to 8.15.0 (#5169)
thomashohn Sep 15, 2025
c417bd3
Bump globals to 16.4.0 (#5177)
thomashohn Sep 15, 2025
94e73ea
chore(deps): bump joi from 17.13.3 to 18.0.1 (#5178)
dependabot[bot] Sep 15, 2025
f8ef112
Bump expect to 30.1.2 (#5182)
thomashohn Sep 16, 2025
5c38989
Bump some minors before 3.7.5 (#5187)
thomashohn Sep 16, 2025
b004ca8
bugfix: prevent this error when running WebDriver without Bidi protoc…
ngraf Sep 16, 2025
6ff0dc6
fix(utils): resolve command injection vulnerability in `emptyFolder` …
mhassan1 Sep 21, 2025
5a440e2
Bump versions (#5197)
thomashohn Sep 21, 2025
3ba96d1
release 3.7.5 (#5198)
kobenguyent Sep 22, 2025
6b7eb17
chore(deps-dev): bump @wdio/utils from 9.15.0 to 9.19.2 (#5199)
dependabot[bot] Sep 23, 2025
a2f5da3
Revise contributing guidelines for clarity and updates
kobenguyent Sep 23, 2025
05396d5
Bump @codeceptjs/detox-helper and playwright (#5210)
thomashohn Sep 24, 2025
6e9ae95
fix(Playwright): removes a redundant asynchronous initialization call…
Samuel-StO Sep 24, 2025
9cf209c
fixed: typo in example of "retryTo" in lib/effects.js (#5203)
danielrentz Sep 24, 2025
f4da7e9
fix: add a correct signatures for the package types exports (#5204)
epszaw Sep 24, 2025
37cd6cb
fix(Appium): should not override sauce:options cap with prefix - iOS …
mikhail-yesipchuk-zenitech Sep 24, 2025
bad14d1
feat(REST): add HEAD request (#5212)
kobenguyent Sep 24, 2025
cef6e01
DOC: Autogenerate and update documentation
actions-user Sep 24, 2025
f5a9157
fix: platformName does not work in Appium helper (#5214)
mirao Sep 24, 2025
db7d0fc
fix: wrong stats when running with workers (#5215)
kobenguyent Sep 26, 2025
aed3123
fixed: TS typings for Codecept and MainConfig (#5216)
danielrentz Sep 29, 2025
2857e96
Bump various deps to keep them in sync (#5220)
thomashohn Sep 30, 2025
50eb7f4
fixed: remove deprecated fs.rmdirSync with recursive option (#5218)
danielrentz Sep 30, 2025
2505ac7
fix: flaky UTs (#5221)
kobenguyent Sep 30, 2025
0345fc7
feat: Add support for Playwright storageState configuration (#5192)
Samuel-StO Sep 30, 2025
3e9b799
fix(docs): update some deprecated links of puppeteer and playwright (…
danielrentz Oct 1, 2025
2c4a7d9
DOC: Autogenerate and update documentation
actions-user Oct 1, 2025
dcbcc81
fix: flaky tests (#5225)
kobenguyent Oct 1, 2025
5826298
fix: max listeners exceeded warning (#5227)
kobenguyent Oct 1, 2025
1075552
fix: show only verbose or debug mode (#5232)
kobenguyent Oct 1, 2025
743d0a1
fix: appium tests (#5230)
kobenguyent Oct 2, 2025
5defdf9
improvement: workers cli log (#5235)
kobenguyent Oct 2, 2025
3db47c8
fixed: add typings for class Result (#5236)
danielrentz Oct 2, 2025
c1c0a51
fix: mock server for internal tests (#5238)
kobenguyent Oct 2, 2025
0db01c6
fix: address HTML reporter issues (#5240)
kobenguyent Oct 2, 2025
1584113
fix: html reporter improvements (#5242)
kobenguyent Oct 3, 2025
9085269
Update dependabot.yml
kobenguyent Oct 7, 2025
2418a1e
fix: handle missing opts in retryFailedStep plugin (Shell: Command fa…
mirao Oct 29, 2025
aaf4fef
fix: global timeout before suite (#5275)
mirao Oct 29, 2025
af28305
docs: update example for "event.all.result" handler (receives a singl…
danielrentz Oct 29, 2025
05c5d06
fix(docs): Update reports.md: use "reporter" option instead of CLI (#…
danielrentz Oct 29, 2025
f55a991
clean up files (#5281)
kobenguyent Oct 30, 2025
8d37a99
Merge origin/3.x into rebase-3.x
kobenguyent Oct 30, 2025
d3b1228
chore: Start ESM conversion - Phase 1
kobenguyent Oct 30, 2025
d277565
chore: ESM conversion - Phase 2
kobenguyent Oct 30, 2025
934e0fd
chore: ESM conversion - Phase 3
kobenguyent Oct 30, 2025
aa01456
feat(esm): Phase 4 - Convert remaining lib infrastructure files to ESM
kobenguyent Oct 30, 2025
5f35314
feat(esm): Convert heal.js template to ESM
kobenguyent Oct 30, 2025
7e7987a
docs: Update ESM conversion progress - lib/ directory 100% complete
kobenguyent Oct 30, 2025
c264148
feat(esm): Convert bin/test-server.js to ESM
kobenguyent Oct 30, 2025
62a9464
feat(esm): Convert test infrastructure files to ESM
kobenguyent Oct 30, 2025
b5c1519
feat(esm): Update tsconfig.json for ES2022 modules
kobenguyent Oct 30, 2025
a9e7e85
fix(esm): Update tsconfig module to Node16
kobenguyent Oct 30, 2025
663d0cc
fix(esm): Replace require() with dynamic import() in codecept.js
kobenguyent Oct 30, 2025
fdf4fab
docs: Final ESM conversion progress update
kobenguyent Oct 30, 2025
bcb1f38
refactor: Rename test config files from .js to .cjs
kobenguyent Oct 30, 2025
3e25f76
fix: Update test references to use .cjs config files
kobenguyent Oct 30, 2025
1f8eb8b
docs: Update progress with test config fixes
kobenguyent Oct 30, 2025
470333a
fix: Correct container support test expectation
kobenguyent Oct 30, 2025
90990a2
docs: Update with 100% unit test pass rate
kobenguyent Oct 30, 2025
c83903d
fix: Add .js extensions to module imports in requireModules() and fix…
kobenguyent Oct 30, 2025
a6ac292
fix: Multiple critical ESM-related fixes for runner tests
kobenguyent Oct 31, 2025
e0bf8d6
fix: Major pool mode and worker infrastructure improvements
kobenguyent Oct 31, 2025
ba977ef
fix: Restore missing PlaywrightReactVueLocator.js and convert to ESM
kobenguyent Oct 31, 2025
b2dfe68
fix: update timeout tests to reference .cjs config files
kobenguyent Oct 31, 2025
30ee679
wip: investigate timeout test discovery issue
kobenguyent Oct 31, 2025
7685e12
fix: resolve test file discovery issue for ESM configs
kobenguyent Oct 31, 2025
e0224c8
fix: resolve remaining runner test failures
kobenguyent Oct 31, 2025
b01e79e
rebase 3.x
kobenguyent Oct 31, 2025
33d2c7b
Update Appium_platformNameNormalization_test.js
kobenguyent Nov 1, 2025
b58d9f4
Update WebElement_integration_test.js
kobenguyent Nov 1, 2025
a7c8710
Update WebElement_test.js
kobenguyent Nov 1, 2025
0956be6
Update Appium_platformNameNormalization_test.js
kobenguyent Nov 1, 2025
99451ee
Update WebElement_integration_test.js
kobenguyent Nov 1, 2025
5316cd4
Update WebElement_test.js
kobenguyent Nov 1, 2025
65a0ce8
Update Appium_platformNameNormalization_test.js
kobenguyent Nov 1, 2025
1109936
Update WebElement_integration_test.js
kobenguyent Nov 1, 2025
32bec2f
Update WebElement_test.js
kobenguyent Nov 1, 2025
29ceddd
Update custom_masking_test.js
kobenguyent Nov 1, 2025
68787f6
Update custom_masking_test.js
kobenguyent Nov 1, 2025
4a1346a
Update circular_reference_test.js
kobenguyent Nov 1, 2025
7605730
Update html-reporter-plugin_test.js
kobenguyent Nov 1, 2025
87d7d09
fix failed tests
kobenguyent Nov 3, 2025
6a54204
fix more tests
kobenguyent Nov 3, 2025
00f374d
fix types tests
kobenguyent Nov 3, 2025
c8f95ad
remove testcafe like in 4.x branch
kobenguyent Nov 3, 2025
c0646ea
fix: Puppeteer JavaScript world protocol errors
kobenguyent Nov 3, 2025
2ff912e
fix: add missing return statement for XPath in findElements
kobenguyent Nov 4, 2025
b6523ae
fix: handle XPath queries within ElementHandles
kobenguyent Nov 4, 2025
f6ec8a9
fix: use ::-p-xpath selector for Puppeteer 24.x+ instead of removed $…
kobenguyent Nov 4, 2025
f99b964
fix: React selector detection in findElements for Locator objects
kobenguyent Nov 4, 2025
b59142f
fix: React/Vue/Playwright locator detection for Locator objects
kobenguyent Nov 4, 2025
27f56ca
fix: convert Playwright timeout errors to ElementNotFound for consist…
kobenguyent Nov 4, 2025
3a61c6f
fix: add missing crypto import and correct test expectation in WebDriver
kobenguyent Nov 4, 2025
bb425c2
fix: Puppeteer grabAttributeFrom and XPath click issues
kobenguyent Nov 4, 2025
d41da19
fix more tests
kobenguyent Nov 4, 2025
ab84a94
fix: add role locator support and XPath handling to Puppeteer findEle…
kobenguyent Nov 4, 2025
4968282
feat(Playwright): Add browser restart strategy support
kobenguyent Nov 4, 2025
282e58a
fix(Playwright): Properly clean up event listeners to prevent hanging
kobenguyent Nov 4, 2025
dd909f1
fix: Prevent process from hanging after tests complete
kobenguyent Nov 4, 2025
42a975b
fix(Playwright): Handle selector registration in worker processes
kobenguyent Nov 4, 2025
112ce88
fix(Playwright): Guard _init() to prevent duplicate selector registra…
kobenguyent Nov 4, 2025
d6a4c02
fix(workers): Use dynamic imports to work around Node 22.x ES Module …
kobenguyent Nov 4, 2025
3fec548
ci: update workflows to use Node 20.x instead of 22.x
kobenguyent Nov 4, 2025
808bba3
Update Dockerfile
kobenguyent Nov 5, 2025
8ee6c69
fix: Properly preserve exit codes in run and run-workers commands
kobenguyent Nov 5, 2025
9f4382a
fix(ApiDataFactory): Improve cleanup reliability in _after() method
kobenguyent Nov 5, 2025
fa36608
fix(ApiDataFactory): Add delay after cleanup for Docker file system p…
kobenguyent Nov 5, 2025
0498937
fix(ApiDataFactory): Ensure uppercase DELETE method and add better lo…
kobenguyent Nov 5, 2025
7bb7e62
Add healthcheck to json_server and wait for it in test-rest
kobenguyent Nov 5, 2025
0193a46
Reset db.json before each json-server start in Docker
kobenguyent Nov 5, 2025
9d7b653
Add read-only mode to test-server to prevent data persistence
kobenguyent Nov 5, 2025
4af0224
Force Docker rebuild in CI to include latest code changes
kobenguyent Nov 5, 2025
f85830c
Fix docker-compose command - build separately before running tests
kobenguyent Nov 5, 2025
2e0c410
Fix DELETE method uppercase and test-server fallback data
kobenguyent Nov 5, 2025
8cbb511
tryout the working one from 4.x
kobenguyent Nov 5, 2025
7dc82ba
Fix Playwright hanging - always stop browser in _afterSuite
kobenguyent Nov 5, 2025
4654f83
Fix Playwright hanging in check command - respect dry-run mode
kobenguyent Nov 5, 2025
16a61e4
Fix coverage plugin not writing output - use recorder for async gener…
kobenguyent Nov 5, 2025
0ef76fd
Fix coverage output - await recorder tasks after event.all.after
kobenguyent Nov 5, 2025
c4b3090
Fix coverage output - properly await async done() callback
kobenguyent Nov 5, 2025
5699dff
Fix REST tests - add missing setup.js import for chai.should()
kobenguyent Nov 6, 2025
c9f62ad
Fix Playwright process hanging - add timeout to browser.close()
kobenguyent Nov 6, 2025
0cdc49a
Fix Playwright test assertions - initialize chai.should()
kobenguyent Nov 6, 2025
680c183
fix: add expect import to REST_test.js for jest-style assertions
kobenguyent Nov 6, 2025
746333b
refactor: move REST tests from Docker to regular test suite
kobenguyent Nov 6, 2025
8f5d316
fix: prevent Playwright tests from hanging when BROWSER_RESTART=browser
kobenguyent Nov 6, 2025
b7db169
refactor: remove redundant acceptance-tests workflow and fix webdrive…
kobenguyent Nov 6, 2025
68b1dfd
Fix Playwright worker selector registration and test:runner exit code…
kobenguyent Nov 6, 2025
48a7ddf
Fix Playwright worker tests and async config loading
kobenguyent Nov 6, 2025
ec2579c
Fix amOnPage auto-initialization with BROWSER_RESTART=browser
kobenguyent Nov 6, 2025
190ae7b
Update documentation with browser auto-initialization fix
kobenguyent Nov 6, 2025
9c50929
Fix process hanging after tests complete with Playwright
kobenguyent Nov 6, 2025
bbbe770
Update documentation with process hang fix
kobenguyent Nov 6, 2025
c212a18
Add auto-exit to _afterSuite for BROWSER_RESTART=browser mode
kobenguyent Nov 6, 2025
a2bccda
Fix force exit to work unconditionally after cleanup
kobenguyent Nov 7, 2025
f743fbf
Update docs with tested process exit fix
kobenguyent Nov 7, 2025
857c3af
Remove worker tests to avoid selector registration conflicts
kobenguyent Nov 7, 2025
fc20104
fix: Add role locator support with text/exact matching and fix drag-a…
kobenguyent Nov 7, 2025
79da6b7
refactor: DRY role locator handling with helper functions
kobenguyent Nov 7, 2025
b1a609a
chore: Remove 27 truly unused test app view files
kobenguyent Nov 7, 2025
6bd2953
Fix: Remove premature process.exit() that prevented test summary from…
kobenguyent Nov 7, 2025
7043f22
Reduce auto-exit delay from 5s to 1s to prevent workflow timeouts
kobenguyent Nov 7, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
136 changes: 63 additions & 73 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,16 @@

Thanks for getting here. If you have a good will to improve CodeceptJS we are always glad to help. Ask questions, raise issues, ping in Twitter.

Go over the steps in [this](https://github.com/firstcontributions/first-contributions) guide as first contributions

To start you need:

1. Fork and clone the repo.
2. Run `npm i --force` to install all required libraries
3. Do the changes.
4. Add/Update Test (if possible)
5. Update documentation
6. Run `npm run def` to generate types
7. Run `npm run docs` if you change the documentation
8. Commit and Push to your fork
9. Make Pull Request
1. Fork the repo.
2. Run `npm install` to install all required libraries
3. Run `npm run def`to generate types
4. Do the changes.
5. Add/Update Test (if possible)
6. Update documentation
7. Commit and Push to your fork
8. Make Pull Request

To run codeceptjs from this repo use:

Expand All @@ -28,28 +25,21 @@ To run examples:
node bin/codecept.js run -c examples
```

Depending on a type of change you should do the following.

## Debugging

To see recorder queue in logs enable NodeJS debug output by passing `DEBUG=codeceptjs:*` env variable:

```
DEBUG=codeceptjs:* npx codeceptjs run
```
Depending on a type of a change you should do the following.

## Helpers

Please keep in mind that CodeceptJS have **unified API** for Playwright, WebDriverIO, Appium, Puppeteer, TestCafe. Tests written using those helpers should be compatible at syntax level. However, some helpers may contain unique methods. That happens. If, for instance, WebDriverIO has method XXX and Playwright doesn't, you can implement XXX inside Playwright using the same method signature.
Please keep in mind that CodeceptJS have **unified API** for WebDriver, Appium, Playwright, and Puppeteer. Tests written using those helpers should be compatible at syntax level. However, some of helpers may contain unique methods. That happens. If, for instance, WebDriver has method XXX and Playwright doesn't, you can implement XXX inside Playwright using the same method signature.

### Updating Playwright | Puppeteer | WebDriver
### Updating a WebDriver | Playwright | Puppeteer

_Whenever a new method or new behavior is added it should be documented in a docblock. Valid JS-example is required! Do **not edit** `docs/helpers/`, those files are generated from docblocks in corresponding helpers! _
*Whenever a new method or new behavior is added it should be documented in a docblock. Valid JS-example is required! Do **not edit** `docs/helpers/`, those files are generated from docblocks in corresponding helpers! *

Working test is highly appreciated. To run the test suite you need:

- selenium server + chromedriver
- PHP installed
* selenium server + chromedriver
* PHP installed

To launch PHP demo application run:

Expand All @@ -60,8 +50,9 @@ php -S 127.0.0.1:8000 -t test/data/app
Execute test suite:

```sh
mocha test/helper/WebDriver_test.js
mocha test/helper/WebDriverIO_test.js
mocha test/helper/Puppeteer_test.js
mocha test/helper/Nightmare_test.js
```

Use `--grep` to execute tests only for changed parts.
Expand All @@ -80,9 +71,21 @@ Then is should be accessible at:
http://localhost:8000/form/myexample
```

### Updating Protractor

*Whenever a new method or new behavior is added it should be documented in a docblock. Valid JS-example is required! Do **not edit** `docs/helpers/`, those files are generated from docblocks in corresponding helpers! *

Protractor helper is based on [Protractor library](http://www.protractortest.org)

In case you do Protractor-specific change, please add a test:To run the test suite you need:

* selenium server + chromedriver

Demo application is located at: [http://davertmik.github.io/angular-demo-app](http://davertmik.github.io/angular-demo-app)

### Updating REST | ApiDataFactory

_Whenever a new method or new behavior is added it should be documented in a docblock. Valid JS-example is required!_
*Whenever a new method or new behavior is added it should be documented in a docblock. Valid JS-example is required!*

Adding a test is highly appreciated.

Expand All @@ -96,20 +99,30 @@ Edit a test at `test/rest/REST_test.js` or `test/rest/ApiDataFactory_test.js`

## Appium

_Whenever a new method or new behavior is added it should be documented in a docblock. Valid JS-example is required! Do **not edit** `docs/helpers/`, those files are generated from docblocks in corresponding helpers! _
*Whenever a new method or new behavior is added it should be documented in a docblock. Valid JS-example is required! Do **not edit** `docs/helpers/`, those files are generated from docblocks in corresponding helpers! *

It is recommended to run mobile tests on CI.
So do the changes, make pull request, see the CI status.
Appium tests are executed at **Saucelabs**.
Appium tests are executed at **Semaphore CI**.

## Core Changes

Before applying any Core changes please raise an issue to discuss that change with core team.
Please try to add corresponding testcase to runner or unit.

## Typings

Typings is generated in `typings/` directory via `jsdoc`

After you updated docblock in JS file, generate typing files with next command:

```
npm run def
```

## Documentation

Documentation is stored in `/docs` directory in Markdown format.
Documentation is stored in `/docs` directory in markdown format.

**Documentation for helpers is a part of a source code**.

Expand All @@ -121,52 +134,17 @@ After you updated docblock in JS file, generate markdown files with next command
npm run docs
```

Documentation parts can be shared across helpers. Those parts are located in `docs/webapi/*.mustache`. Inside a docblock those files can be included like this:

```js
/**
* {{> click }}
*/
click() {
// ...
}
```

_Note:_ Due to the (lib)[https://documentation.js.org/] that we are using to generate docs, the fast and cheap way to fix format issue that text after the mustache template is appended without formatting is moving the texts to above the mustache template.

```js
/**
* // Before
* Click action
* {{> click }}
* Click action
*/
click() {
// ...
}
```
Documentation parts can be shared accross helpers. Those parts are located in `docs/webapi/*.mustache`. Inside a docblock those files can be included like this:

```js
/**
* // After
* Click action
* {{> click }}
*/
click() {
// ...
}
```

## Typings

Typings are generated in `typings/` directory via `jsdoc`

After you updated docblock in JS file, generate typing files with next command:

```
npm run def
```

## Testing

Whenever you implemented a feature/bugfix
Expand All @@ -188,7 +166,7 @@ mocha test/runner
Instead of manually running php, json_server and selenium for before tests you
can use `docker-compose` to run those automatically.
You can find `docker-compose.yml` file in `test` directory and run all commands
from this directory. Currently, we provide following commands to run tests with
from this directory. Currently we provide following commands to run tests with
respective dependencies:

#### Run unit tests
Expand All @@ -203,25 +181,28 @@ docker-compose run --rm test-unit
docker-compose run --rm test-helpers

# or pass path to helper test to run specific helper,
# for example to run only WebDriver tests:
docker-compose run --rm test-helpers test/helper/WebDriver_test.js
# for example to run only WebDriverIO tests:
docker-compose run --rm test-helpers test/helper/WebDriverIO_test.js

# Or to run only rest and ApiDataFactory tests
docker-compose run --rm test-helpers test/rest
```

#### Run acceptance tests

To that we provide three separate services respectively for WebDriver, Nightmare and Puppeteer tests:
To that we provide three separate services respectively for WebDriverIO, Nightmare, Puppeteer and
Protractor tests:

```sh
docker-compose run --rm test-acceptance.webdriverio
docker-compose run --rm test-acceptance.nightmare
docker-compose run --rm test-acceptance.puppeteer
docker-compose run --rm test-acceptance.protractor
```

#### Running against specific Node version

By default, dockerized tests are run against node 12.10.0, you can run it against
By default dockerized tests are run against node 12.10.0, you can run it against
specific version as long as there is Docker container available for such
version. To do that you need to build codecept's Docker image prior to running
tests and pass `NODE_VERSION` as build argument.
Expand All @@ -236,17 +217,26 @@ And now every command based on `test-helpers` service will use node 9.4.0. The
same argument can be passed when building unit and acceptance tests services.

### CI flow

We're currently using a bunch of CI services to build and test codecept in
We're currently using bunch of CI services to build and test codecept in
different environments. Here's short summary of what are differences between
separate services

#### CircleCI
#### TravisCI
Travis CI uses runs tests against Node 8 and Node 9. In total it uses 8 jobs to
build each helper against both Node versions. For every job it runs unit tests
first, then `ApiDataFactory` and `REST` tests present in `test/rest` directory.
Finally if those pass we run specific helper tests found in `test/helper`
directory. It doesn't run acceptance tests.
Config is present in `.travis.yml` file.

#### CircleCI
Here we use CodeceptJS docker image to build and execute tests inside it. We
start with building Docker container based on Dockerfile present in main project
directory. Then we run (in this order) unit tests, all helpers present in
`test/helpers`, then we go with `test/rest` directory and finally if everything
passed so far it executes acceptance tests. For easier maintenance and local
debugging CircleCI uses `docker-compose.yml` file from `test` directory.
You can find Circle config in `.circleci` directory.

#### Semaphore
Currently Semaphore runs only Appium helper tests.
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
* CodeceptJS version:
* NodeJS Version:
* Operating System:
* puppeteer || webdriverio || testcafe version (if related)
* puppeteer || webdriverio || playwright version (if related)
* Configuration file:

```js
Expand Down
1 change: 0 additions & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ Applicable helpers:
- [ ] REST
- [ ] FileHelper
- [ ] Appium
- [ ] TestCafe

Applicable plugins:

Expand Down
96 changes: 96 additions & 0 deletions .github/SHARDING_WORKFLOWS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
# Test Sharding Workflows

This document explains the GitHub Actions workflows that demonstrate the new test sharding functionality in CodeceptJS.

## Updated/Created Workflows

### 1. `acceptance-tests.yml` (Updated)

**Purpose**: Demonstrates sharding with acceptance tests across multiple browser configurations.

**Key Features**:

- Runs traditional docker-compose tests (for backward compatibility)
- Adds new sharded acceptance tests using CodeceptJS directly
- Tests across multiple browser configurations (Puppeteer, Playwright)
- Uses 2x2 matrix: 2 configs × 2 shards = 4 parallel jobs

**Example Output**:

```
- Sharded Tests: codecept.Puppeteer.js (Shard 1/2)
- Sharded Tests: codecept.Puppeteer.js (Shard 2/2)
- Sharded Tests: codecept.Playwright.js (Shard 1/2)
- Sharded Tests: codecept.Playwright.js (Shard 2/2)
```

### 2. `sharding-demo.yml` (New)

**Purpose**: Comprehensive demonstration of sharding features with larger test suite.

**Key Features**:

- Uses sandbox tests (2 main test files) for sharding demonstration
- Shows basic sharding with 2-way split (`1/2`, `2/2`)
- Demonstrates combination of `--shuffle` + `--shard` options
- Uses `DONT_FAIL_ON_EMPTY_RUN=true` to handle cases where some shards may be empty

### 3. `test.yml` (Updated)

**Purpose**: Clarifies which tests support sharding.

**Changes**:

- Added comment explaining that runner tests are mocha-based and don't support sharding
- Points to sharding-demo.yml for examples of CodeceptJS-based sharding

## Sharding Commands Used

### Basic Sharding

```bash
npx codeceptjs run --config ./codecept.js --shard 1/2
npx codeceptjs run --config ./codecept.js --shard 2/2
```

### Combined with Other Options

```bash
npx codeceptjs run --config ./codecept.js --shuffle --shard 1/2 --verbose
```

## Test Distribution

The sharding algorithm distributes tests evenly:

- **38 tests across 4 shards**: ~9-10 tests per shard
- **6 acceptance tests across 2 shards**: 3 tests per shard
- **Uneven splits handled gracefully**: Earlier shards get extra tests when needed

## Benefits Demonstrated

1. **Parallel Execution**: Tests run simultaneously across multiple CI workers
2. **No Manual Configuration**: Automatic test distribution without maintaining test lists
3. **Load Balancing**: Even distribution ensures balanced execution times
4. **Flexibility**: Works with any number of shards and test configurations
5. **Integration**: Compatible with existing CodeceptJS features (`--shuffle`, `--verbose`, etc.)

## CI Matrix Integration

The workflows show practical CI matrix usage:

```yaml
strategy:
matrix:
config: ['codecept.Puppeteer.js', 'codecept.Playwright.js']
shard: ['1/2', '2/2']
```

This creates 4 parallel jobs:

- Config A, Shard 1/2
- Config A, Shard 2/2
- Config B, Shard 1/2
- Config B, Shard 2/2

Perfect for scaling test execution across multiple machines and configurations.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Acceptance Tests using docker compose
name: Acceptance Tests

on:
push:
Expand Down Expand Up @@ -32,9 +32,9 @@ jobs:
sudo apt-get update --allow-releaseinfo-change
sudo apt-get install -y docker-compose

# Run rest tests using docker-compose
- name: Run REST Tests
run: docker-compose run --rm test-rest
# Build Docker images to include latest code changes
- name: Build Docker Images
run: docker-compose build
working-directory: test

# Run WebDriverIO acceptance tests using docker-compose
Expand Down
Loading