Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement: Declare test code in separate directory #5

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
/.github/ export-ignore
/.phive/ export-ignore
/test/ export-ignore
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you work on a package and declare test code in a separate directory, adding exclusions for test code in .gitattributes is less work.

/.editorconfig export-ignore
/.gitattributes export-ignore
/.gitignore export-ignore
Expand All @@ -9,6 +10,8 @@
/composer-require-checker.json export-ignore
/composer.lock export-ignore
/Makefile export-ignore
/phpbench.json export-ignore
/phpunit.xml export-ignore
/psalm-baseline.xml export-ignore
/psalm.xml export-ignore
/rector.php export-ignore
35 changes: 34 additions & 1 deletion .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,28 @@ make dependency-analysis

to run a dependency analysis.

## Mutation Tests

We are using [`infection/infection`](https://github.com/infection/infection) to ensure a minimum quality of the tests.

Enable `Xdebug` and run

```sh
make mutation-tests
```

to run mutation tests.

## Performance Tests

We are using [`phpbench/phpbench`](https://github.com/phpbench/phpbench) to control the performance of our production code.

```sh
make performance-tests
```

to run performance tests.

## Refactoring

We are using [`rector/rector`](https://github.com/rectorphp/rector) to automatically refactor code.
Expand Down Expand Up @@ -94,6 +116,17 @@ to regenerate the baseline in [`../psalm-baseline.xml`](../psalm-baseline.xml).

:exclamation: Ideally, the baseline should shrink over time.

## Tests

We are using [`phpunit/phpunit`](https://github.com/sebastianbergmann/phpunit) to drive the development.

Run

```sh
make tests
```

to run all the tests.

## Extra lazy?

Expand All @@ -103,7 +136,7 @@ Run
make
```

to automatically refactor code, enforce coding standards, and run a static code analysis!
to automatically refactor code, enforce coding standards, run a static code analysis, run tests, run performance tests, and run mutation tests!

## Help

Expand Down
4 changes: 4 additions & 0 deletions .github/settings.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,15 @@ branches:
required_status_checks:
checks:
- context: "Autoloader (8.3, locked)"
- context: "Code Coverage (8.3, locked)"
- context: "Coding Standards (8.3, locked)"
- context: "Dependency Analysis (8.3, locked)"
- context: "Mutation Tests (8.3, locked)"
- context: "Performance Tests (8.3, locked)"
- context: "Refactoring (8.3, locked)"
- context: "Security Analysis (8.3, locked)"
- context: "Static Code Analysis (8.3, locked)"
- context: "Tests (8.3, locked)"
strict: false
restrictions: null

Expand Down
198 changes: 198 additions & 0 deletions .github/workflows/integrate.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,62 @@ jobs:
- name: "Test autoloader for production"
run: "php autoloader.php"

code-coverage:
name: "Code Coverage"

runs-on: "ubuntu-latest"

strategy:
matrix:
php-version:
- "8.3"

dependencies:
- "locked"

steps:
- name: "Checkout"
uses: "actions/[email protected]"

- name: "Set up PHP"
uses: "shivammathur/[email protected]"
with:
coverage: "xdebug"
extensions: "none, ctype, dom, json, mbstring, phar, simplexml, tokenizer, xml, xmlwriter"
php-version: "${{ matrix.php-version }}"

- name: "Set up problem matchers for PHP"
run: "echo \"::add-matcher::${{ runner.tool_cache }}/php.json\""

- name: "Set up problem matchers for phpunit/phpunit"
run: "echo \"::add-matcher::${{ runner.tool_cache }}/phpunit.json\""

- name: "Determine composer cache directory"
uses: "ergebnis/.github/actions/composer/[email protected]"

- name: "Cache dependencies installed with composer"
uses: "actions/[email protected]"
with:
path: "${{ env.COMPOSER_CACHE_DIR }}"
key: "php-${{ matrix.php-version }}-composer-${{ matrix.dependencies }}-${{ hashFiles('composer.lock') }}"
restore-keys: "php-${{ matrix.php-version }}-composer-${{ matrix.dependencies }}-"

- name: "Install ${{ matrix.dependencies }} dependencies with composer"
uses: "ergebnis/.github/actions/composer/[email protected]"
with:
dependencies: "${{ matrix.dependencies }}"

- name: "Collect code coverage with Xdebug and phpunit/phpunit"
env:
XDEBUG_MODE: "coverage"
run: "vendor/bin/phpunit --colors=always --configuration=phpunit.xml --coverage-clover=.build/phpunit/logs/clover.xml"

- name: "Send code coverage report to codecov.io"
uses: "codecov/[email protected]"
with:
files: ".build/phpunit/logs/clover.xml"
token: "${{ secrets.CODECOV_TOKEN }}"

coding-standards:
name: "Coding Standards"

Expand Down Expand Up @@ -190,6 +246,98 @@ jobs:
- name: "Run maglnet/composer-require-checker"
run: ".phive/composer-require-checker check --ansi --config-file=$(pwd)/composer-require-checker.json"

mutation-tests:
name: "Mutation Tests"

runs-on: "ubuntu-latest"

strategy:
matrix:
php-version:
- "8.3"

dependencies:
- "locked"

steps:
- name: "Checkout"
uses: "actions/[email protected]"

- name: "Set up PHP"
uses: "shivammathur/[email protected]"
with:
coverage: "xdebug"
extensions: "none, ctype, dom, json, mbstring, phar, simplexml, tokenizer, xml, xmlwriter"
php-version: "${{ matrix.php-version }}"

- name: "Set up problem matchers for PHP"
run: "echo \"::add-matcher::${{ runner.tool_cache }}/php.json\""

- name: "Determine composer cache directory"
uses: "ergebnis/.github/actions/composer/[email protected]"

- name: "Cache dependencies installed with composer"
uses: "actions/[email protected]"
with:
path: "${{ env.COMPOSER_CACHE_DIR }}"
key: "php-${{ matrix.php-version }}-composer-${{ matrix.dependencies }}-${{ hashFiles('composer.lock') }}"
restore-keys: "php-${{ matrix.php-version }}-composer-${{ matrix.dependencies }}-"

- name: "Install ${{ matrix.dependencies }} dependencies with composer"
uses: "ergebnis/.github/actions/composer/[email protected]"
with:
dependencies: "${{ matrix.dependencies }}"

- name: "Run mutation tests with Xdebug and infection/infection"
env:
XDEBUG_MODE: "coverage"
run: "vendor/bin/infection --ansi --configuration=infection.json --logger-github"

performance-tests:
name: "Performance Tests"

runs-on: "ubuntu-latest"

strategy:
matrix:
php-version:
- "8.3"

dependencies:
- "locked"

steps:
- name: "Checkout"
uses: "actions/[email protected]"

- name: "Set up PHP"
uses: "shivammathur/[email protected]"
with:
coverage: "none"
extensions: "none, ctype, dom, json, mbstring, phar, simplexml, tokenizer, xml, xmlwriter"
php-version: "${{ matrix.php-version }}"

- name: "Set up problem matchers for PHP"
run: "echo \"::add-matcher::${{ runner.tool_cache }}/php.json\""

- name: "Determine composer cache directory"
uses: "ergebnis/.github/actions/composer/[email protected]"

- name: "Cache dependencies installed with composer"
uses: "actions/[email protected]"
with:
path: "${{ env.COMPOSER_CACHE_DIR }}"
key: "php-${{ matrix.php-version }}-composer-${{ matrix.dependencies }}-${{ hashFiles('composer.lock') }}"
restore-keys: "php-${{ matrix.php-version }}-composer-${{ matrix.dependencies }}-"

- name: "Install ${{ matrix.dependencies }} dependencies with composer"
uses: "ergebnis/.github/actions/composer/[email protected]"
with:
dependencies: "${{ matrix.dependencies }}"

- name: "Run performance tests with phpbench/phpbench"
run: "vendor/bin/phpbench run --config=phpbench.json test/"

refactoring:
name: "Refactoring"

Expand Down Expand Up @@ -354,3 +502,53 @@ jobs:

- name: "Run vimeo/psalm"
run: "vendor/bin/psalm --config=psalm.xml --output-format=github --shepherd --show-info=false --stats --threads=4"

tests:
name: "Tests"

runs-on: "ubuntu-latest"

strategy:
matrix:
php-version:
- "8.3"

dependencies:
- "lowest"
- "locked"
- "highest"

steps:
- name: "Checkout"
uses: "actions/[email protected]"

- name: "Set up PHP"
uses: "shivammathur/[email protected]"
with:
coverage: "none"
extensions: "none, ctype, dom, json, mbstring, phar, simplexml, tokenizer, xml, xmlwriter"
php-version: "${{ matrix.php-version }}"

- name: "Set up problem matchers for PHP"
run: "echo \"::add-matcher::${{ runner.tool_cache }}/php.json\""

- name: "Set up problem matchers for phpunit/phpunit"
run: "echo \"::add-matcher::${{ runner.tool_cache }}/phpunit.json\""

- name: "Determine composer cache directory"
uses: "ergebnis/.github/actions/composer/[email protected]"

- name: "Cache dependencies installed with composer"
uses: "actions/[email protected]"
with:
path: "${{ env.COMPOSER_CACHE_DIR }}"
key: "php-${{ matrix.php-version }}-composer-${{ matrix.dependencies }}-${{ hashFiles('composer.lock') }}"
restore-keys: "php-${{ matrix.php-version }}-composer-${{ matrix.dependencies }}-"

- name: "Install ${{ matrix.dependencies }} dependencies with composer"
uses: "ergebnis/.github/actions/composer/[email protected]"
with:
dependencies: "${{ matrix.dependencies }}"

- name: "Run tests with phpunit/phpunit"
run: "vendor/bin/phpunit --colors=always --configuration=phpunit.xml"
21 changes: 20 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,11 +1,16 @@
.PHONY: it
it: refactoring coding-standards security-analysis static-code-analysis autoloader ## Runs the refactoring, coding-standards, security-analysis, static-code-analysis, and autoloader targets
it: refactoring coding-standards security-analysis static-code-analysis tests performance-tests mutation-tests autoloader ## Runs the refactoring, coding-standards, security-analysis, static-code-analysis, tests, performance-tests, mutation-tests, and autoloader targets

.PHONY: autoloader
autoloader: ## Dumps the autoloader for production and verifies that it does not include classes not intended for production
composer dump-autoload --no-dev --optimize
php autoloader.php

.PHONY: code-coverage
code-coverage: vendor ## Collects coverage from running tests with phpunit/phpunit
mkdir -p .build/phpunit
vendor/bin/phpunit --configuration=phpunit.xml --coverage-text

.PHONY: coding-standards
coding-standards: vendor ## Lints YAML files with yamllint, normalizes composer.json with ergebnis/composer-normalize, and fixes code style issues with friendsofphp/php-cs-fixer
yamllint -c .yamllint.yaml --strict .
Expand All @@ -21,6 +26,15 @@ dependency-analysis: phive vendor ## Runs a dependency analysis with maglnet/com
help: ## Displays this list of targets with descriptions
@grep -E '^[a-zA-Z0-9_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[32m%-30s\033[0m %s\n", $$1, $$2}'

.PHONY: performance-tests
performance-tests: vendor ## Runs performance tests with phpbench/phpbench
vendor/bin/phpbench run --config=phpbench.json test/

.PHONY: mutation-tests
mutation-tests: vendor ## Runs mutation tests with infection/infection
mkdir -p .build/infection
vendor/bin/infection --configuration=infection.json

.PHONY: phive
phive: .phive ## Installs dependencies with phive
mkdir -p .build/phive/
Expand All @@ -47,6 +61,11 @@ static-code-analysis-baseline: vendor ## Generates a baseline for static code an
vendor/bin/psalm --config=psalm.xml --clear-cache
vendor/bin/psalm --config=psalm.xml --set-baseline=psalm-baseline.xml

.PHONY: tests
tests: vendor ## Runs tests with phpunit/phpunit
mkdir -p .build/phpunit
vendor/bin/phpunit --configuration=phpunit.xml

vendor: composer.json composer.lock
composer validate --strict
composer install --no-interaction --no-progress
5 changes: 5 additions & 0 deletions composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,11 @@
"Localheinz\\OrganizingTestCodeInPhp\\": "src/"
}
},
"autoload-dev": {
"psr-4": {
"Localheinz\\OrganizingTestCodeInPhp\\Test\\": "test/"
}
},
Comment on lines +39 to +43
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you declare test code in a separate directory, managing exclusions for the composer autoloader in composer.json is unnecessary. Instead, you configure and document namespaces for test code via an autoloader configuration in the autoload-dev section.

"config": {
"allow-plugins": {
"composer/package-versions-deprecated": true,
Expand Down
17 changes: 17 additions & 0 deletions infection.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"ignoreMsiWithNoMutations": true,
"logs": {
"text": ".build/infection/infection-log.txt"
},
"minCoveredMsi": 100,
"minMsi": 100,
"phpUnit": {
"configDir": "."
},
"source": {
"directories": [
"src"
]
Comment on lines +12 to +14
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you declare test code in a separate directory, managing exclusions for infection/infection in infection.json is unnecessary.

},
"timeout": 10
}
5 changes: 5 additions & 0 deletions phpbench.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"$schema":"vendor/phpbench/phpbench/phpbench.schema.json",
"runner.bootstrap": "vendor/autoload.php",
"runner.file_pattern": "*.php"
}
Loading