Revise exit codes for dotnet test

[Youssef1313]

Just throwing ideas here 😄

For the new dotnet test experience for MTP, we should revise what final exit code of dotnet test should.

dotnet test will be invoking test executables. Each executable may exit with a different exit code.

See https://learn.microsoft.com/dotnet/core/testing/microsoft-testing-platform-exit-codes

Should we define some order of importance for exit codes, and return the exit code that's "most" important?
I would love if MTP exit codes were powers of two, dotnet test can simply bitwise-OR the values together to be able to more accurately model the exit codes, but that's currently not the case.

Another question that arises is, what if MTP in future adds a new exit code that SDK doesn't know about (yet)? What should we do?

Or, should we simply keep using exit code 1, as it's today, to denote "generic failure"?

Another idea is to check, if all executables are exiting with the same exit code, dotnet test should use that. Otherwise, dotnet test can exit with 1 denoting generic failure.

Related to #45927

[Evangelink]

Another idea is to check, if all executables are exiting with the same exit code, dotnet test should use that. Otherwise, dotnet test can exit with 1 denoting generic failure.

This seems to me like the more logical choice and also the easiest to understand as final user.

[mariam-abdulla]

Now, if at least 1 executable returned non-zero exit code, the exit code will be a generic failure.

Another idea is to check, if all executables are exiting with the same exit code, dotnet test should use that. Otherwise, dotnet test can exit with 1 denoting generic failure.

I also agree with @Evangelink, this seems like the most logical.

[Youssef1313]

I agree as well!

[Youssef1313]

Another question here is if we should explicitly print out the exit code in case of failure, with a link to https://learn.microsoft.com/en-us/dotnet/core/testing/microsoft-testing-platform-exit-codes (probably via aka.ms link)

[Evangelink]

I like the idea of the aka.ms link to exit code doc page.