On Release dashboard a multi-run test (e.g. rolling upgrade) is marked as not planned, even though it was scheduled and started, but all runs are failed

[dimakr]

A multi-run test (e.g. rolling-upgrade) is marked as not-run on Release dashboard, if all runs of the test are failed.

When investigating test results of a release it may seem that such a test was not planned. The assignee would notice that the test was actually executed (and failed) only when checking My Jobs page.

Example of such a not-run test on a release dashboard:

And actual rub for the release in question:

[k0machi]

The reason here is that the submitted test has the version set as 2024.1.12, but your dash version is set to 2024.1.13. If this behaviour should be handled as part of a set of tests I can set it aside for rolling upgrade improvements/fixes we have lined up in the future.

[dimakr]

This screenshot is for one of 2 failed runs in this multi-run #32 in Argus. The run is for 2024.1.13 release but 2024.1.12 version is the base version before upgrade here. It is just not clear why it is marked as Scylla version on Details tab, as on a successful run it would be marked as Base scylla version.
For instance the second failed run in #32 is marked with Scylla version: 5.4.9-20240703.fdcbbb85adcd on Details tab.

[dimakr]

This is the overview of the main pipeline of that run:

Pipeline tasks for versions 2022.2 and 2023.1 failed, so that the Argus runs were not even created for them. But Argus runs for 5.4 and 2024.1 were successfully created

[k0machi]

I see, in this case it is a planned feature to support base versions without a succesful upgrade, currently there's no way to tell which version is the base and upgraded one without upgraded one being submitted

[dimakr]

in this case it is a planned feature to support base versions without a succesful upgrade,

@k0machi
Do we have maybe a ticket raised already for this?
Or maybe we can label current one as an enhancement