val_data_size and test_data_size have no effect.

[hyeok9855]

The configs val_data_size and test_data_size (see here) have no effect. It seems that the default env.val_file and env.test_file are used.

You can simply check this by

1. change those configs to some value
2. python run.py with the changed config file.

[fedebotu]

Right, that is working as intended though.

There are two cases:

1. User provides val and/or test files. Then, we should use them
2. User does not provide them OR they cannot be loaded for some reason. Then, we should generate a validation and test dataset

I agree though that it can be confusing 🤔

How about having a log.warning that says something , e.g. during val: "env.val_file has been loaded. val_data_size will have no effect"? Or do you have better ideas?

[hyeok9855]

How about slicing the dataset according to the size when both val/test_file and val/test_data_size are provided?

[fedebotu]

That sounds like a good idea!