Description
neuralbench --help shows the datasets you can use for each task. For the EEG image task it shows:
image: Gifford2022Large, Grootswagers2022Human, Xu2024Alljoined, Xu2025Alljoined
None of these names works with --dataset:
- Gifford2022Large is the task's default dataset. It fails, and the error doesn't even list it as a choice.
- Grootswagers2022Human also fails. Only the lowercase grootswagers2022human works.
This happens because --help shows each dataset's study name, but --dataset only accepts the name of a file in the task's datasets/ folder, and those file names are lowercase. The default dataset is set in the task's main config.yaml and has no file of its own under datasets/, so --dataset can't select it by name at all.
Steps to reproduce
1. Run neuralbench --help and find the image: line under eeg. It lists Gifford2022Large first.
2. Run neuralbench eeg image --dataset Gifford2022Large
3. It fails with:
ValueError: Unknown dataset 'Gifford2022Large' for eeg/image. Choose from: ['grootswagers2022human', 'xu2024alljoined', 'xu2025alljoined']
4. Run neuralbench eeg image --dataset Grootswagers2022Human. It fails the same way.
Expected: every name that --help lists works with --dataset, and naming the default dataset runs the task with its normal config.
Environment
- OS: Manjaro Linux
- Python version: 3.13
- neuralbench 0.3.1
Description
neuralbench --help shows the datasets you can use for each task. For the EEG image task it shows:
image: Gifford2022Large, Grootswagers2022Human, Xu2024Alljoined, Xu2025Alljoined
None of these names works with --dataset:
This happens because --help shows each dataset's study name, but --dataset only accepts the name of a file in the task's datasets/ folder, and those file names are lowercase. The default dataset is set in the task's main config.yaml and has no file of its own under datasets/, so --dataset can't select it by name at all.
Steps to reproduce
Environment