Test run shows results for responses with error codes endpoints

[nicolai-rhesis]

Summary

When running test sets against non-functional endpoints, the test results section shows results (and a error 405) at the same time.

Steps to Reproduce

1. Create a non-functional endpoint
2. Run any test set against the endpoint
3. Go into test run
4. See results and a 405 error

Expected Behavior

Only an error is shown that the test run could not be completed. No actual metrics results are shown.

Screenshots / Recordings

[harry-rhesis]

Errors being displayed for endpoints is a feature, to be able to give insight for users into the reason why the endpoint might be failing.

[nicolai-rhesis]

This does not seem logical to me. Why do we apply the metrics on an error message. I even get an email that says x / y tests passed. This is easily confusing to the user. If the endpoint returns an error, the metrics should not be applied and the email not been send. Or another email be send "your endpoint return an error".

[nicolai-rhesis]

I just ran into this again, you have a results list, wondering why 86% failed just to find out loads of 503 errors actually happened.