-
Notifications
You must be signed in to change notification settings - Fork 0
Description
As of the time of writing, there are two unittests(encoder_unittest.py, verify_unittest.py) that depend on test_dataset_utils, a utility component that creates randomized structures such as datasets and lists.
There are two main issues with this approach:
A) Unittests are no longer repeatable
B) On test failure, it is currently not possible to recreate the test enviornment
C) The value of randomization is not neccesarily good in these contexts
D) The type of randomization does not help cover unseen edge cases
To expand on C), the main use case for randomization is not neccesarily to test for the randomized values itself, but to rather quickly instantiate objects of a certain type that is required by the function being tested, for example when a decorator function @requires nedes a dataset object, but the test case does not require the values of the dataset object to be relevant. In the times when randomized values are directly compared, it is still possible to mundanely rewrite the tests to include static and well-purposed values to succeed the tests.
Lastly, the randomzed values are litteraly just a list of random letters and digits, a monotonous data format that isn't productive for any possible edge cases.
The conclusion is that randomized input values for unit tests aren't the right tools in this context, and some refactoring will be required to make test_dataset_utils generate arbitrary, but not random, input and test values.