Generalize test infrastructure a bit

Previously we had logic in place that for every tests tried to guess which
exceptions were expected, given the test configuration. This was a bit limiting
however; it's easier to just put this logic in the tests themselves. We make
that change in this PR, which will make the _next_ PR a bit easier.