How to prepare a datalist with limited data?

[Minxiangliu]

Currently, the 3D medical imaging data I want to analyze is quite limited, with only around 100 records. Previously, I used sklearn train_test_split to create 5 folds and trained custom models using monai. Now, I'm considering using auto3Dseg for training. However, I found that the datalist_generator in the tutorials divides the entire dataset into multiple folds. If applied to my data, the training data would be scarce.

1. Can auto3Dseg be used with a small amount of data?
2. Are there any suggestions for generating a datalist with a small amount of data?

[KumoLiu]

Hi @Minxiangliu, thanks for your interest here.

I don't think it should have much to do with how you set the datalist and the size of the dataset, you can try to set a small fold then to make each fold more data.

If I misunderstood, feel free to correct me, thanks!

[Minxiangliu]

Thank you for your response. Your explanation is one solution, but I'd like to know if auto3Dseg can perform cross-validation as shown in the following code:

import numpy as np
from sklearn.model_selection import KFold
kfold = KFold(n_splits=5)

kf_data = kfold.split(np.zeros((100)))
for i, (train_index, test_index) in enumerate(kf_data):
    print(f'Fold {i}:')
    print(f'  Train len={len(train_index)}')
    print(f'  Test  len={len(test_index)}')

Output:

Fold 0:
  Train len=80
  Test  len=20
Fold 1:
  Train len=80
  Test  len=20
Fold 2:
  Train len=80
  Test  len=20
Fold 3:
  Train len=80
  Test  len=20
Fold 4:
  Train len=80
  Test  len=20

If I only have 100 records, even with 2 folds, it might be challenging to train the data.

[KumoLiu]

If you only have 100 pieces of data, deep learning may not be appropriate, but if it is a particularly simple task, you can try to train it. After all, splitting a dataset does not increase the total amount of data. :)