KeyError: Pretrain

[TheNoliOliandPea]

Hello. I'm hoping I'm overlooking something obvious from this repo. I'm trying to run:

python train_chart.py --cfg_file CornerNetLine --data_dir "linedata(1028)"

But the error I'm receiving is:

(DeepRule) PS C:\Users\lemon\OneDrive\Documents\GitHub\ChartOCR> python train_chart.py --cfg_file CornerNetLine --data_dir "linedata(1028)"
{'system': {'snapshot_name': 'CornerNetLine', 'dataset': 'Line', 'batch_size': 26, 'sampling_function': 'kp_detection', 'train_split': 'trainchart', 'val_split': 'valchart', 'test_split': 'testchart', 'learning_rate': 0.00025, 'decay_rate': 10, 'val_iter': 100, 'opt_algo': 'adam', 'prefetch_size': 5, 'max_iter': 50000, 'stepsize': 45000, 'snapshot': 5000, 'chunk_sizes': [5, 7, 7, 7], 'data_dir': '$AZUREML_DATAREFERENCE_workspaceblobstore/chartdata(0610)/', 'cache_dir': './cache_pure/', 'display': 5}, 'db': {'rand_scale_min': 0.6, 'rand_scale_max': 1.4, 'rand_scale_step': 0.1, 'rand_scales': None, 'rand_crop': True, 'rand_color': True, 'border': 128, 'gaussian_bump': True, 'gaussian_iou': 0.3, 'input_size': [511, 511], 'output_sizes': [[128, 128]], 'test_scales': [1], 'top_k': 200, 'categories': 1, 'ae_threshold': 0.5, 'nms_threshold': 0.5, 'max_per_image': 100, 'annotations': 'line/annotations/instancesLine(1023)_train2019.json'}}
['cache', 'line']
loading all datasets...
using 1 threads
loading from cache file: linedata(1028)\cache\line_train2019.pkl
loading annotations into memory...
Path to file: C:\Users\lemon\OneDrive\Documents\GitHub\ChartOCR\linedata(1028)\line\annotations\instancesLine(1023)_train2019.json
Done (t=3.71s)
creating index...
index created!
loading from cache file: linedata(1028)\cache\line_val2019.pkl
loading annotations into memory...
Path to file: C:\Users\lemon\OneDrive\Documents\GitHub\ChartOCR\linedata(1028)\line\annotations\instancesLine(1023)_val2019.json
Done (t=0.12s)
creating index...
index created!
system config...
{'batch_size': 26,
'cache_dir': 'linedata(1028)\cache',
'chunk_sizes': [5, 7, 7, 7],
'config_dir': 'config',
'data_dir': 'linedata(1028)',
'data_rng': RandomState(MT19937) at 0x11108D6D140,
'dataset': 'Line',
'decay_rate': 10,
'display': 5,
'learning_rate': 0.00025,
'max_iter': 50000,
'nnet_rng': RandomState(MT19937) at 0x11108D6D240,
'opt_algo': 'adam',
'prefetch_size': 5,
'result_dir': 'results',
'sampling_function': 'kp_detection',
'snapshot': 5000,
'snapshot_name': 'CornerNetLine',
'stepsize': 45000,
'tar_data_dir': 'cls',
'test_split': 'testchart',
'train_split': 'trainchart',
'val_iter': 100,
'val_split': 'valchart'}
db config...
{'ae_threshold': 0.5,
'border': 128,
'categories': 1,
'data_aug': True,
'gaussian_bump': True,
'gaussian_iou': 0.3,
'gaussian_radius': -1,
'input_size': [511, 511],
'lighting': True,
'max_per_image': 100,
'merge_bbox': False,
'nms_algorithm': 'exp_soft_nms',
'nms_kernel': 3,
'nms_threshold': 0.5,
'output_sizes': [[128, 128]],
'rand_color': True,
'rand_crop': True,
'rand_pushes': False,
'rand_samples': False,
'rand_scale_max': 1.4,
'rand_scale_min': 0.6,
'rand_scale_step': 0.1,
'rand_scales': array([0.6, 0.7, 0.8, 0.9, 1. , 1.1, 1.2, 1.3]),
'special_crop': False,
'test_scales': [1],
'top_k': 200,
'weight_exp': 8}
len of db: 116745
Traceback (most recent call last):
File "C:\Users\lemon\OneDrive\Documents\GitHub\ChartOCR\train_chart.py", line 206, in
train(training_dbs, validation_db, args.start_iter)
File "C:\Users\lemon\OneDrive\Documents\GitHub\ChartOCR\train_chart.py", line 76, in train
pretrained_model = system_configs.pretrain
^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\lemon\OneDrive\Documents\GitHub\ChartOCR\config.py", line 99, in pretrain
return self._configs["pretrain"]
~~~~~~~~~~~~~^^^^^^^^^^^^
KeyError: 'pretrain'

My inkling is that it's not loading a pretrained model in config, however if that's the case, I have no idea which file is the pretrained file in this repo. Or when I go to comment it out to run without the pretrained parameters, it'll give me the same error as well. I'm a bit new to all of this so any help is much appreciate it.

[soap117]

See the dataset link, included.

[TheNoliOliandPea]

See the dataset link, included.

So I have that included in the directory. It's loading the line(1028) (or whatever the actual folder name is). Then to clarify, it's the one under chart pipeline model that I'm probably not loading correctly perhaps? Thanks.

[soap117]

You can debug to track the system_config instance. I am not sure, the default setting will set that para to None. Probably it is because you loaded some cached data on training mode, which is not expected. Junyu Luo, Informatics Ph.D. Student The Pennsylvania State University

…

________________________________ From: TheNoliOliandPea ***@***.***> Sent: Sunday, June 9, 2024 1:43 To: soap117/DeepRule ***@***.***> Cc: Luo, Junyu ***@***.***>; Comment ***@***.***> Subject: Re: [soap117/DeepRule] KeyError: Pretrain (Issue #37) See the dataset link, included. So I have that included in the directory. It's loading the line(1028) (or whatever the actual folder name is). Then to clarify, it's the one under chart pipeline model that I'm probably not loading correctly perhaps? Thanks. — Reply to this email directly, view it on GitHub<#37 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AGXQFDWMQZWPMCNC3ULK65TZGPTR7AVCNFSM6AAAAABI5UGRNWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJWGMZTCNZYHA>. You are receiving this because you commented.Message ID: ***@***.***>

[TheNoliOliandPea]

I see! I'll check there, thank you for pointing that out.