Skip to content

Reqnroll: inconclusive tests are reported as broken #557

@delatrie

Description

@delatrie

When marking a test as inconclusive (e.g., with Assert.Inconclusive), Reqnroll reports it as skipped:

S /start-specflow-net8.0-dotnet-dotnet> dotnet test 
  Determining projects to restore...
  All projects are up-to-date for restore.
  Examples.Allure -> /start-specflow-net8.0-dotnet-dotnet/tests/Examples.Allure/bin/Debug/net8.0/Examples.Allure.dll
Test run for /start-specflow-net8.0-dotnet-dotnet/tests/Examples.Allure/bin/Debug/net8.0/Examples.Allure.dll (.NETCoreApp,Version=v8.0)
VSTest version 17.11.1 (arm64)

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
  Skipped TestScenario [50 ms]

None     - Failed:     0, Passed:     0, Skipped:     0, Total:     0, Duration: 50 ms - Examples.Allure.dll (net8.0)

However, Allure Report shows such tests as broken:

Some frameworks (NUnit) allow using TestIgnore:

using Reqnroll;
using System;
using Reqnroll.UnitTestProvider;

namespace Examples.Allure
{
    [Binding]
    public class Steps
    {
        private readonly  IUnitTestRuntimeProvider runtimeProvider;

        public Steps(IUnitTestRuntimeProvider runtimeProvider)
        {
            this.runtimeProvider = runtimeProvider;
        }

        [When("the second step passes")]
        public void Step2()
        {
            this.runtimeProvider.TestIgnore("Can't run here");
        }

        /* the rest of the steps */
    }
}

But in this case, the step is still reported as broken:

Other frameworks (MsTest) don't support even that.

Expected behavior

Both the step and the test should be reported as skipped to match what Reqnroll reports by itself.

Metadata

Metadata

Assignees

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions