Skip to content

dotnet test --filter parameter doesn't affect tests with Platform attribute #1257

@melanchall

Description

@melanchall

Hi,

I've updated packages references to the newest versions of NUnit (4.3.2) and NUnit3TestAdapter (5.0.0). Now if I execute following command

dotnet test DryWetMidi.Tests/Melanchall.DryWetMidi.Tests.csproj --filter "FullyQualifiedName!~.Multimedia" --blame --verbosity normal --logger "console;verbosity=normal" --framework net9.0

I'll see attempts to execute tests which fully qualified name doesn't contain .Multimedia (!~ means "doesn't contain" in filter of dotnet test command):

  Passed SetValue_NonDefault_AtMiddle_NonDefault_AtStart(True) [< 1 ms]
  Passed SetValue_NonDefault_AtMiddle_NonDefault_AtStart(False) [< 1 ms]
  Passed SetValue_NonDefault_AtStart [< 1 ms]
CheckDeviceAddedRemoved: OneTimeSetUp: Only supported on MacOsX
CheckDeviceAddedRemoved: OneTimeSetUp: Only supported on MacOsX
  Passed SetValue_NonDefault_AtStart_Default_AtMiddle(True) [< 1 ms]
  Passed SetValue_NonDefault_AtStart_Default_AtMiddle(False) [< 1 ms]
  Passed SetValue_NonDefault_AtStart_Default_AtStart(True) [< 1 ms]
  Passed SetValue_NonDefault_AtStart_Default_AtStart(False) [< 1 ms]
  Passed SetValue_NonDefault_AtStart_NonDefault_AtMiddle(True) [< 1 ms]
  Passed SetValue_NonDefault_AtStart_NonDefault_AtMiddle(False) [< 1 ms]
  Passed SetValue_NonDefault_AtStart_NonDefault_AtStart(True) [< 1 ms]
  Passed SetValue_NonDefault_AtStart_NonDefault_AtStart(False) [< 1 ms]
CheckInputDevicesEquality_ViaEquals_DifferentDevices_Mac: Only supported on MacOsX
CheckInputDevicesEquality_ViaEquals_DifferentDevices_Mac: Only supported on MacOsX

Those tests with text "Only supported on ..." have .Multimedia in their fully qualified name. For example, CheckDeviceAddedRemoved:

namespace Melanchall.DryWetMidi.Tests.Multimedia
{
    [TestFixture]
    [Platform("MacOsX")]
    public sealed class DevicesWatcherTests
    {
        #region Test methods

        [Test]
        public void CheckDeviceAddedRemoved()

What I see:

  1. Filter is ignored for tests that have Platform attribute and only for such tests. Tests with .Multimedia in namespace but without Platform attribute are ignored.
  2. Issue happens on all operating systems and I have fails due to this bug on macOS for example.

You can easily reproduce the issue using my repo on pack-update branch: https://github.com/melanchall/drywetmidi/tree/pack-update. Here you can find instructions on building the project: https://melanchall.github.io/drywetmidi/articles/dev/Manual-build.html.

I can't update to the latest version due to the bug :-(

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions