Skip to content

Correct once dataset default path and paths for data generation #1207

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Correct once dataset default path and paths for data generation #1207

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
70e2408
Set pyyaml requirements version to supported version 5.4.1(yaml.load(…
kun-woo-park Dec 1, 2022
b174f67
Correct default info option for once dataset and components inside ig…
kun-woo-park Dec 1, 2022
7b1b08d
Make sources to generate only existing datssets
kun-woo-park Dec 1, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 11 additions & 4 deletions pcdet/datasets/once/once_dataset.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
from ...utils import box_utils
from .once_toolkits import Octopus


class ONCEDataset(DatasetTemplate):
def __init__(self, dataset_cfg, class_names, training=True, root_path=None, logger=None):
"""
Expand Down Expand Up @@ -388,8 +389,14 @@ def evaluation(self, det_annos, class_names, **kwargs):
def create_once_infos(dataset_cfg, class_names, data_path, save_path, workers=4):
dataset = ONCEDataset(dataset_cfg=dataset_cfg, class_names=class_names, root_path=data_path, training=False)

splits = ['train', 'val', 'test', 'raw_small', 'raw_medium', 'raw_large']
ignore = ['test']
image_sets = save_path / Path("ImageSets")
exist_sets = set(list(map(lambda p: p.stem, list(image_sets.glob("*.txt")))))

splits = set(['train', 'val', 'test', 'raw_small', 'raw_medium', 'raw_large'])
ignore = splits - exist_sets

splits = list(splits)
ignore = list(ignore)

print('---------------Start to generate data infos---------------')
for split in splits:
Expand All @@ -416,7 +423,7 @@ def create_once_infos(dataset_cfg, class_names, data_path, save_path, workers=4)

parser = argparse.ArgumentParser(description='arg parser')
parser.add_argument('--cfg_file', type=str, default=None, help='specify the config of dataset')
parser.add_argument('--func', type=str, default='create_waymo_infos', help='')
parser.add_argument('--func', type=str, default='create_once_infos', help='')
parser.add_argument('--runs_on', type=str, default='server', help='')
args = parser.parse_args()

Expand All @@ -441,4 +448,4 @@ def create_once_infos(dataset_cfg, class_names, data_path, save_path, workers=4)
class_names=['Car', 'Bus', 'Truck', 'Pedestrian', 'Bicycle'],
data_path=once_data_path,
save_path=once_save_path
)
)
2 changes: 1 addition & 1 deletion requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ numba
torch>=1.1
tensorboardX
easydict
pyyaml
pyyaml==5.4.1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May I know why need to fix the pyyaml version?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on source, it use yaml.load() function for loading yaml scripts, but it is deprecated in latest version and causing error : "load() missing 1 required positional argument 'loader'".

So, it might need to fix the pyyaml version supporting yaml.load() function.
(Another option i found is replacing it to yaml.full_load() in latest version of pyyaml)

scikit-image
tqdm
torchvision
Expand Down