Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide a useful implementation of something compatible with testing.T #571

Merged
merged 9 commits into from
Apr 29, 2024
4 changes: 2 additions & 2 deletions features/formatter/events.feature
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Feature: event stream formatter
"""

Scenario: should process simple scenario
Given a feature path "features/load.feature:26"
Given a feature path "features/load.feature:27"
When I run feature suite with formatter "events"
Then the following events should be fired:
"""
Expand All @@ -34,7 +34,7 @@ Feature: event stream formatter
"""

Scenario: should process outline scenario
Given a feature path "features/load.feature:34"
Given a feature path "features/load.feature:35"
When I run feature suite with formatter "events"
Then the following events should be fired:
"""
Expand Down
7 changes: 4 additions & 3 deletions features/formatter/pretty.feature
Original file line number Diff line number Diff line change
Expand Up @@ -350,7 +350,7 @@ Feature: pretty formatter
Scenario: Should scenarios identified with path:line and preserve the order.
Given a feature path "features/load.feature:6"
And a feature path "features/multistep.feature:6"
And a feature path "features/load.feature:26"
And a feature path "features/load.feature:27"
And a feature path "features/multistep.feature:23"
When I run feature suite with formatter "pretty"
Then the rendered output will be as follows:
Expand All @@ -363,7 +363,7 @@ Feature: pretty formatter
Scenario: load features within path # features/load.feature:6
Given a feature path "features" # suite_context_test.go:0 -> *godogFeaturesScenario
When I parse features # suite_context_test.go:0 -> *godogFeaturesScenario
Then I should have 13 feature files: # suite_context_test.go:0 -> *godogFeaturesScenario
Then I should have 14 feature files: # suite_context_test.go:0 -> *godogFeaturesScenario
\"\"\"
features/background.feature
features/events.feature
Expand All @@ -378,6 +378,7 @@ Feature: pretty formatter
features/run.feature
features/snippets.feature
features/tags.feature
features/testingt.feature
\"\"\"

Feature: run features with nested steps
Expand Down Expand Up @@ -407,7 +408,7 @@ Feature: pretty formatter
As a test suite
I need to be able to load features

Scenario: load a specific feature file # features/load.feature:26
Scenario: load a specific feature file # features/load.feature:27
Given a feature path "features/load.feature" # suite_context_test.go:0 -> *godogFeaturesScenario
When I parse features # suite_context_test.go:0 -> *godogFeaturesScenario
Then I should have 1 feature file: # suite_context_test.go:0 -> *godogFeaturesScenario
Expand Down
3 changes: 2 additions & 1 deletion features/lang.feature
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Savybė: užkrauti savybes
Scenarijus: savybių užkrovimas iš aplanko
Duota savybių aplankas "features"
Kai aš išskaitau savybes
Tada aš turėčiau turėti 13 savybių failus:
Tada aš turėčiau turėti 14 savybių failus:
"""
features/background.feature
features/events.feature
Expand All @@ -23,4 +23,5 @@ Savybė: užkrauti savybes
features/run.feature
features/snippets.feature
features/tags.feature
features/testingt.feature
"""
3 changes: 2 additions & 1 deletion features/load.feature
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Feature: load features
Scenario: load features within path
Given a feature path "features"
When I parse features
Then I should have 13 feature files:
Then I should have 14 feature files:
"""
features/background.feature
features/events.feature
Expand All @@ -21,6 +21,7 @@ Feature: load features
features/run.feature
features/snippets.feature
features/tags.feature
features/testingt.feature
"""

Scenario: load a specific feature file
Expand Down
144 changes: 144 additions & 0 deletions features/testingt.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
Feature: providing testingT compatibility
In order to test application behavior using standard go assertion techniques
As a test suite
I need to be able to provide a testing.T compatible interface

Scenario: should fail test if FailNow called on testing T
Given a feature "failed.feature" file:
"""
Feature: failed feature

Scenario: fail a scenario
Given passing step
When I fail the test by calling FailNow on testing T
"""
When I run feature suite
Then the suite should have failed
And the following steps should be passed:
"""
passing step
"""
And the following step should be failed:
"""
I fail the test by calling FailNow on testing T
"""

Scenario: should pass test when testify assertions pass
Given a feature "testify.feature" file:
"""
Feature: passed feature

Scenario: pass a scenario
Given passing step
When I call testify's assert.Equal with expected "exp" and actual "exp"
When I call testify's require.Equal with expected "exp" and actual "exp"
"""
When I run feature suite
Then the suite should have passed
And the following steps should be passed:
"""
passing step
I call testify's assert.Equal with expected "exp" and actual "exp"
I call testify's require.Equal with expected "exp" and actual "exp"
"""

Scenario: should fail test when testify assertions do not pass
Given a feature "testify.feature" file:
"""
Feature: failed feature

Scenario: fail a scenario
Given passing step
When I call testify's assert.Equal with expected "exp" and actual "not"
And I call testify's assert.Equal with expected "exp2" and actual "not"
"""
When I run feature suite
Then the suite should have failed
And the following steps should be passed:
"""
passing step
"""
And the following steps should be failed:
"""
I call testify's assert.Equal with expected "exp" and actual "not"
"""
And the following steps should be skipped:
"""
I call testify's assert.Equal with expected "exp2" and actual "not"
"""

Scenario: should fail test when multiple testify assertions are used in a step
Given a feature "testify.feature" file:
"""
Feature: failed feature

Scenario: fail a scenario
Given passing step
When I call testify's assert.Equal 3 times
"""
When I run feature suite
Then the suite should have failed
And the following steps should be passed:
"""
passing step
"""
And the following steps should be failed:
"""
I call testify's assert.Equal 3 times
"""

Scenario: should pass test when multiple testify assertions are used successfully in a step
Given a feature "testify.feature" file:
"""
Feature: passed feature

Scenario: pass a scenario
Given passing step
When I call testify's assert.Equal 3 times with match
"""
When I run feature suite
Then the suite should have passed
And the following steps should be passed:
"""
passing step
I call testify's assert.Equal 3 times with match
"""

Scenario: should skip test when skip is called on the testing.T
Given a feature "testify.feature" file:
"""
Feature: skipped feature

Scenario: skip a scenario
Given passing step
When I skip the test by calling Skip on testing T
"""
When I run feature suite
Then the suite should have passed
And the following steps should be passed:
"""
passing step
"""
And the following steps should be skipped:
"""
I skip the test by calling Skip on testing T
"""

Scenario: should log to testing.T
Given a feature "logging.feature" file:
"""
Feature: logged feature

Scenario: logged scenario
Given passing step
When I call Logf on testing T with message "format this %s" and argument "formatparam1"
And I call Log on testing T with message "log this message"
"""
When I run feature suite
Then the suite should have passed
And the following steps should be passed:
"""
passing step
I call Logf on testing T with message "format this %s" and argument "formatparam1"
I call Log on testing T with message "log this message"
"""
12 changes: 6 additions & 6 deletions run_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -525,11 +525,11 @@ func Test_AllFeaturesRun(t *testing.T) {
...................................................................... 210
...................................................................... 280
...................................................................... 350
...... 356
....................................... 389


94 scenarios (94 passed)
356 steps (356 passed)
101 scenarios (101 passed)
389 steps (389 passed)
0s
`

Expand All @@ -553,11 +553,11 @@ func Test_AllFeaturesRunAsSubtests(t *testing.T) {
...................................................................... 210
...................................................................... 280
...................................................................... 350
...... 356
....................................... 389


94 scenarios (94 passed)
356 steps (356 passed)
101 scenarios (101 passed)
389 steps (389 passed)
0s
`

Expand Down
20 changes: 18 additions & 2 deletions suite.go
Original file line number Diff line number Diff line change
Expand Up @@ -85,12 +85,18 @@ func (s *suite) runStep(ctx context.Context, pickle *Scenario, step *Step, scena
// user multistep definitions may panic
defer func() {
if e := recover(); e != nil {
if err != nil {
pe, isErr := e.(error)
switch {
case isErr && errors.Is(pe, errStopNow):
// FailNow or SkipNow called on dogTestingT, so clear the error to let the normal
// below getTestingT(ctx).isFailed() call handle the reasons.
err = nil
case err != nil:
err = &traceError{
msg: fmt.Sprintf("%s: %v", err.Error(), e),
stack: callStack(),
}
} else {
default:
err = &traceError{
msg: fmt.Sprintf("%v", e),
stack: callStack(),
Expand All @@ -100,6 +106,11 @@ func (s *suite) runStep(ctx context.Context, pickle *Scenario, step *Step, scena

earlyReturn := scenarioErr != nil || errors.Is(err, ErrUndefined)

// Check for any calls to Fail on dogT
if err == nil {
err = getTestingT(ctx).isFailed()
}

switch {
case errors.Is(err, ErrPending):
status = StepPending
Expand Down Expand Up @@ -509,10 +520,15 @@ func (s *suite) runPickle(pickle *messages.Pickle) (err error) {

s.fmt.Pickle(pickle)

dt := &testingT{
name: pickle.Name,
}
ctx = setContextTestingT(ctx, dt)
// scenario
if s.testingT != nil {
// Running scenario as a subtest.
s.testingT.Run(pickle.Name, func(t *testing.T) {
dt.t = t
ctx, err = s.runSteps(ctx, pickle, pickle.Steps)
if s.shouldFail(err) {
t.Errorf("%+v", err)
Expand Down
Loading