Get errors when running at 26

[vtddggg]

Hello Shashank Kotyan,
Nice work and helpful code for me!
It seems that some error happens when run run_evolution.py
I got AttributeError: 'dict' object has no attribute 'graph', when the algorithm goes at GPU 2 running 26
Thanks if you can give some advices.
there is the log:

(ras) root@iZuf63vsuvldhpds99q9crZ:~/project/RobustArchitectureSearch# python -u run_evolution.py --use_adaptive_mutations --use_augmentation --use_limited_data

Namespace(dataset=2, epochs=50, log_dir='logs', num_mutated_child=2, num_mutations=5, population_dir='population', population_size=25, test=True, use_adaptive_mutations=True, use_augmentation=True, use_cycles=True, use_limited_data=True, use_non_squares=False, use_random_params=False)
Searching for previous geneartions
Did Not Found Any Last Generation

GPU 2 running 1
GPU 2 running 2
GPU 2 running 3
GPU 2 running 4
GPU 2 running 5
GPU 2 running 6
GPU 2 running 7
GPU 2 running 8
GPU 2 running 9
GPU 2 running 10
GPU 2 running 11
GPU 2 running 12
GPU 2 running 13
GPU 2 running 14
GPU 2 running 15
GPU 2 running 16
GPU 2 running 17
GPU 2 running 18
GPU 2 running 19
GPU 2 running 20
GPU 2 running 21
GPU 2 running 22
GPU 2 running 23
GPU 2 running 24
GPU 2 running 25
GPU 2 running 26
Traceback (most recent call last):
  File "run_evolution.py", line 315, in <module>
    NeuroEvolution(args).run()
  File "run_evolution.py", line 127, in run
    self.evolve()
  File "run_evolution.py", line 251, in evolve
    self.evolve_ras()        
  File "run_evolution.py", line 199, in evolve_ras
    self.parallel_run(1, [0, parent_dna, self.num_mutations, child_individual, self.population, self.store_individual, self.gen])
  File "run_evolution.py", line 62, in parallel_run
    if self.jobs[deviceID].exception is not None: raise Exception(f"{self.jobs[deviceID].exception[0]}, {self.jobs[deviceID].exception[1]}")
Exception: 'dict' object has no attribute 'graph', Traceback (most recent call last):
  File "/root/project/RobustArchitectureSearch/process.py", line 26, in run
    mp.Process.run(self)
  File "/root/anaconda3/envs/ras/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/root/project/RobustArchitectureSearch/worker.py", line 156, in create_child
    while test_dna is None: mutation, test_dna = population.mutate(dna)
  File "<string>", line 2, in mutate
  File "/root/anaconda3/envs/ras/lib/python3.7/multiprocessing/managers.py", line 834, in _callmethod
    raise convert_to_error(kind, result)
AttributeError: 'dict' object has no attribute 'graph'

[shashankkotyan]

Hi @vtddggg,

Sorry for my late reply.

The error seems strange to me.
I am familiar with the dictionary having a graph element in it.

Can you check the dna.pkl file for 26th architecture? The dictionary should have a 'graph' key in it.

Also, it would be beneficial if you send a traceback of the error.