How to debug flaky BadImageFormatExceptions?

[Dimension4]

Hey guys,

I'm currently trying to figure out a bug that happens occasionally on the macOS CI pipeline that I have at work. Basically, at seemingly random CI runs, the dotnet tests fail in one specific test project (all tests in that project fail) with a BadImageFormatException. The detailed reason seems to vary a bit. Sometimes it's "corrupt metadata", sometimes "bad token", and one or two other ones I can't remember at the moment (I'm citing from memory here so the quotes are probably not 100% accurate).

We did quite a bit of debugging on different CI agents and local machines and here is what we found out:

1. It doesn't seem related to the build itself. We can get the tests to throw with builds that were previously successful, as well as builds that failed the tests right away.
2. We test with dotnet test --no-build, so testing shouldn't change the binary. When I run dotnet test --no-build in a loop with the exact same command line arguments and environment, the tests will sometimes pass and sometimes throw (throw happens roughly 20% of the time).
3. The tests do not do any IO, everything IO is mocked and in-memory.
4. It only happens if we execute the tests on a solution. Running dotnet test directly on the dll (or group of dlls) will always pass the tests.
5. We managed to reproduce the issue on three different versions of .net 9 (but we also haven't tested any other).
6. The failures only seem to happen on the CI agents, so far we weren't able reproduce them on local machines, be it Windows or macOS.
7. The projects themselves use the generic net9.0 target, no platform specific one.
8. It's always one specific dll (or project) that fails and only if we execute at least two tests.
9. The exceptions seem to appear always in the same place in the call stack but their details can vary slightly (tho not in a single run, one test run will throw the same exception in all cases).
10. Each test method is executed and throws somewhere down the call stack.
11. There's no unmanaged or multithreaded code involved (at least not from our side)
12. We use NUnit and the dotnet cli to run the tests.

Has anyone an idea how to debug this further?

I will try to get a debugger attached to the process when it happens but I haven't managed to do so yet.

Are there any environment variables or flags for the runtime that I could use to get more information about what exactly failed? Maybe an unmanaged stack trace of the runtime itself when it generates the managed exceptions? Some extra logging?

Any help or idea is appreciated.