Skip to content

Commit

Permalink
End to end tests (#23)
Browse files Browse the repository at this point in the history
* Add end-to-end tests

The tests run the actual GitHub Action and then verify that Hoverfly and
Hoverctl have been successfully installed, or not.

This provides great end-to-end black box testing[1] coverage :-)

GitHub Actions cannot be run locally[2], so that means that these tests
cannot be run locally either.  Instead, they run automatically as a
GitHub Action themselves[3], triggered on every push.

There is no need for a separate language for the tests - as we
are running the actual GitHub Action we are able use the GitHub Action
workflow syntax[4], which gives us what we need (e.g. expressions)[5] to
write clean tests.

[1] http://softwaretestingfundamentals.com/black-box-testing
[2] https://github.community/t/can-i-run-github-actions-on-my-laptop/17019/2
[3] https://github.com/agilepathway/hoverfly-github-action/actions?query=workflow%3ATest
[4] https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions
[5] https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#about-contexts-and-expressions

* Allow yaml lines of any length

Lines of more than 80 characters were being flagged as an error.
Sometimes longer lines are necessary (e.g. long URLs), so bumped the
limit up to 120 characters and downgraded the alerts from error to
warning.
  • Loading branch information
johnboyes committed Aug 8, 2020
1 parent 9d48e60 commit 009a06b
Show file tree
Hide file tree
Showing 4 changed files with 114 additions and 0 deletions.
90 changes: 90 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
---
name: Test
on: push # yamllint disable-line rule:truthy
env:
ASSERT_VERSION: "| grep -q $HOVERFLY_VERSION"
ASSERT_HOVERFLY_NOT_INSTALLED: "! hoverfly -version"
ASSERT_HOVERCTL_NOT_INSTALLED: "! hoverctl version"

jobs:


install_latest_version_by_default:
name: Install latest version by default
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install Hoverfly
uses: ./
with:
runner_github_workspace_path: ${{ github.workspace }}
- name: Assert latest version installed
env:
HOVERFLY_VERSION: "v1.3.0"
run: |
hoverfly -version ${{ env.ASSERT_VERSION }}
hoverctl version ${{ env.ASSERT_VERSION }}
install_specific_version:
name: Install specific version
runs-on: ubuntu-latest
env:
HOVERFLY_VERSION: v1.2.0
steps:
- uses: actions/checkout@v2
- name: Install Hoverfly
uses: ./
with:
version: ${{ env.HOVERFLY_VERSION }}
runner_github_workspace_path: ${{ github.workspace }}
- name: Assert latest version installed
run: |
hoverfly -version ${{ env.ASSERT_VERSION }}
hoverctl version ${{ env.ASSERT_VERSION }}
install_fails_if_version_does_not_begin_with_v:
name: Install fails if version does not begin with v
runs-on: ubuntu-latest
env:
HOVERFLY_VERSION: "1.2.0"
steps:
- uses: actions/checkout@v2
- name: Install Hoverfly
uses: ./
with:
version: ${{ env.HOVERFLY_VERSION }}
runner_github_workspace_path: ${{ github.workspace }}
- name: Assert Hoverfly not installed
run: |
${{ env.ASSERT_HOVERFLY_NOT_INSTALLED }}
${{ env.ASSERT_HOVERCTL_NOT_INSTALLED }}
install_fails_if_no_runner_github_workspace_path:
name: Install fails when no runner GitHub workspace path provided
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install Hoverfly
uses: ./
- name: Assert Hoverfly not installed
run: |
${{ env.ASSERT_HOVERFLY_NOT_INSTALLED }}
${{ env.ASSERT_HOVERCTL_NOT_INSTALLED }}
install_fails_if_incorrect_runner_github_workspace_path:
name: Install fails when incorrect runner GitHub workspace path provided
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install Hoverfly
uses: ./
with: # Invalid runner_github_workspace_path (must be <dollarsign>{{ github.workspace}})
runner_github_workspace_path: /tmp
- name: Assert Hoverfly not installed
run: |
${{ env.ASSERT_HOVERFLY_NOT_INSTALLED }}
${{ env.ASSERT_HOVERCTL_NOT_INSTALLED }}
8 changes: 8 additions & 0 deletions .yamllint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
extends: default

rules:
# 120 chars should be enough, but don't fail if a line is longer
line-length:
max: 120
level: warning
11 changes: 11 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,14 @@ Firstly thanks for thinking of contributing - the project is [open source](https
* Make your changes on your fork
* Write a [good commit message(s)](https://chris.beams.io/posts/git-commit/) for your changes
* [Create the pull request for your changes](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/proposing-changes-to-your-work-with-pull-requests)
* [Update the tests or add new tests](#running-the-tests) to cover the new behaviour.

## Running the tests

The [tests](.github/workflows/tests.yml) are [end-to-end black box tests](http://softwaretestingfundamentals.com/black-box-testing), to verify that the GitHub Action installs [Hoverfly](https://docs.hoverfly.io) and [Hoverctl](https://docs.hoverfly.io/en/latest/pages/keyconcepts/hoverctl.html) successfully.

[GitHub Actions cannot be run locally](https://github.community/t/can-i-run-github-actions-on-my-laptop/17019/2), so that means that these tests cannot be run locally either. Instead, they run automatically as a [GitHub Action themselves](https://github.com/agilepathway/hoverfly-github-action/actions?query=workflow%3ATest), triggered on every push.

There is no need for a separate language for the tests - as we are running the actual GitHub Action we are able to use the [GitHub Action workflow syntax](https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions), which gives us what we need (e.g. [expressions](https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#about-contexts-and-expressions)) to write clean tests.


5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# Hoverfly GitHub Action

[![tests](https://github.com/agilepathway/hoverfly-github-action/workflows/Test/badge.svg?branch=main&event=push)](https://github.com/agilepathway/hoverfly-github-action/actions?query=workflow%3ATest+event%3Apush+branch%3Amain)
[![reviewdog](https://github.com/agilepathway/hoverfly-github-action/workflows/reviewdog/badge.svg?branch=main&event=push)](https://github.com/agilepathway/hoverfly-github-action/actions?query=workflow%3Areviewdog+event%3Apush+branch%3Amain)
[![License](https://img.shields.io/badge/license-MIT-blue.svg?maxAge=43200)](LICENSE)

Expand Down Expand Up @@ -99,6 +100,10 @@ The Hoverfly binaries are installed at `${{ github.workspace }}/bin`

(The [GitHub workspace directory is persistent throughout the GitHub Action workflow](https://docs.github.com/en/actions/reference/virtual-environments-for-github-hosted-runners#filesystems-on-github-hosted-runners), which means that the binaries are available to any subsequent workflow steps.)

## Tests / examples

The [tests](.github/workflows/tests.yml) contain further configuration examples.


## Suggestions / bug reports / contributions

Expand Down

0 comments on commit 009a06b

Please sign in to comment.