Use snapshot testing for json values?

[kellpossible]

@juhaku I was writing some new tests in https://github.com/juhaku/utoipa/blob/master/tests/component_derive_test.rs, and I realised that it seems like a good candidate to use snapshot testing with something like https://github.com/mitsuhiko/insta or https://github.com/rust-analyzer/expect-test It's a lot easier to create and update new tests, and the format using json or ron is easier to read (in my humble opinion). Would you be open to the idea of using insta for tests?

[juhaku]

@kellpossible Wow.. Didn't know that Rust had such library.. Actually both of them looks pretty good and could be adopted IMO. Yes JSON syntax is good for comparing output and is preferred indeed. Currently in component_derive_test.rs I rely on json path of variables which is less writing of expectations.

Those insta and expect_test are kind of similar wha is actually already done in https://github.com/juhaku/utoipa/blob/master/src/openapi/security.rs#L931 just without any library. There also I am comparing the actual value to value from json.

The reason I am relying on json value comparison instead of actual json string comparison is that the order of certain components in the OpenApi structure tree can change and are not certain (HashMaps). When comparing against json value the order or components does not matter but the content matters.

This is also a catch of writing pretty much any test in the utoipa. The test need to be written so that it tests the expected value against how it sould be in the OpenApi doc itself that's why the OpenApi struct is used in most of the tests there. And the other thing to note is the one reasoned above that tests cannot rely on json string output but to json value instead.

Assumably the expect_test would not be able to provide such feature. But if insta would be able to compare against the json value instead of json string then I think it could be adopted.

On the other hand the macro here: https://github.com/juhaku/utoipa/blob/master/src/openapi/security.rs#L910 could be refactored so that it would fit to the purpose as well as it now creates test fn but it could be refactored so that it would accept 2 arguments one is the expected type instantiation and the another is the expected value. Then this could be used in any other test function as well.

[kellpossible]

The reason I am relying on json value comparison instead of actual json string comparison is that the order of certain components in the OpenApi structure tree can change and are not certain (HashMaps). When comparing against json value the order or components does not matter but the content matters.

From what I can tell, insta can sort unordered maps before storing and comparing them if it proves to be a concern: https://docs.rs/insta/latest/insta/struct.Settings.html#method.set_sort_maps

Assumably the expect_test would not be able to provide such feature. But if insta would be able to compare against the json value instead of json string then I think it could be adopted.

The other alternative could be to use something like https://github.com/davidpdrsn/assert-json-diff which compares json values, and also can provide decent failure messages, but more manual work than snapshot tests to update tests if required output changes (which I'm guessing won't happen much for the kinds of tests these will mostly get used for?).

[kellpossible]

I decided to try using assert-json-diff in this work in progress PR https://github.com/juhaku/utoipa/pull/146/files#diff-a845bc10f67ab496e75fdbaabf725efbb33898f5ee7fc7b2bf49d8ec65bae696R557

[juhaku]

Okay, the assert json diff looks fine too and probably good enough for the purpose indeed.