Serenity + Cucumber-JVM Study

A study in Maven, Cucumber and Serenity
serenity-core v. 1.2.2-rc.6
serenity-cucumber v. 1.1.20

This demo project demonstrates:

• Serenity @driver tag to force use of specifc browser
• Pending/Skipping Scenarios in Serenity Cucumber (Outstanding)
• Discover Naming Conventions
• Running Cucumber Tests in Parallel
• Running Cucumber Tests from a test-jar Test Project

To run (the easy, straight forward way):
Maven commands to generate reports against different versions for comparison:
mvn clean install -PSerenity
mvn clean install -PSerenity -Dtestrunner.prefixes=DoNotRunTags|Naming

Serenity @driver tag to force use of specifc browser

Test Runner Tag: @verify_driver_tag
Serenity recognizes the @driver tag used to force Cucumber Features/Scenarios to use a specific browser:
@driver:chrome, @driver:firefox, @driver:iexplorer, @driver:phantomjs

Pending/Skipping Scenarios in Serenity Cucumber

Test Runner Perfix: DoNotRunTags Test Runner Tag: @verify_norun_tags
Serenity recognizes the following tags, and should report on Cucumber Features/Scenarios without actually executing them:
@ignore, @pending, @skip, @wip, @manual v 1.1.40 - Didn't skip @ignore, @ignored; Also didn't skip code in Cucumber Step Definitions, only code in Serenity Step Libraires. v 1.2.2-rc.2 - Handles @ignore, @ignored; Skips Cucumber Step Definitions code now too!

Discover Naming Conventions

Test Runner Tag: @verify_naming_convention
This example organizes requirments into Themes, Epics and Features. Within each category, Serenity requires unique names.
Note: Serenity treats names delimited with spaces, dashes, underscores or camelcase as the same name.

Running Cucumber Tests in Parallel

This project uses the maven-failsafe-plugin to run tests in parallel, as opposed to the sereinity.properties configuration. The highlights:

• Need multiple test runners in order to run tests in parallel, use a standard naming convention
  (this project uses *FeatureTest.java)
• The pom files uses a regular expression to tell failsafe what runners to include:

<testrunner.prefixes></testrunner.prefixes>
<failsafe.testrunner>%regex[.*(${testrunner.prefixes})FeatureTest.class]</failsafe.testrunner>

• testrunner.prefixes defaults to empty (All), but can be a piped list of test runner prefixes.
• %regex[.*()FeatureTest.class] - Runs all test runners
• %regex[.*(Driver)FeatureTest.class] - Runs only the *DriverFeatureTest runners
  (tests @driver tag to force specific browsers)
• %regex[.*(Driver|DoNotRunTags)FeatureTest.class] - Runs both Driver and DoNotRunTags test runners

Running Cucumber Tests from a test-jar Test project

From the start, I've kept acceptance tests in their own project. For various reasons, the team initially kept this acceptance 'test code' in src/test/java. This forced the issue of creating and working with a test-jar. We've changed our minds, and are now following Maven's recommendations for test projects to put the code in src\main\java. This has forced other issues, like needing to tell failsafe where to find the code, but overall has been easier to work with. All of this is taken care of in the pom files.

This project includes two pom files:

• pom.xml - main pom used for CI at check-in time; builds the test project jar
• pomIT.xml - test runner pom; pulls the test project jar from artifactory, unpacks it, and launches the tests

This requires a two step process to first build the test project jar, then launch the tests. This is useful for setting up a test runner so business and testers can run on demand.

1. mvn clean install (pom.xml; builds the jar and in CI deploys to artifactory)
2. mvn -f pomIT.xml verify (pomIT.xml; launches the tests)