Object detection API

[dbrazey]

Hello all,

I was wondering if your pruning tools could be used on my object detection models (SSD / Faster RCNN, ...) trained with the tensorflow object detection API.

When training, I don't use directly Keras, but I follow tutorials of the object detection API.

Thanks for your help :)

[alanchiao]

There is nothing preventing integrating and experimenting with pruning with the object detection API. The object detection project internally has been mostly migrated to work with Keras (e.g. this sample usage). The said, there are probably ways we can make the integration easier.

Someone is experimenting with applying pruning to the object detection API. Will bring him into this thread.

[Xhark]

Hi, I'm experimenting pruning for object detection. I'll write more information on this thread when I make some progress. Thank you for your attention.

[dbrazey]

Thanks for your answer.

[nicobar97]

@alanchiao any updates?

[alanchiao]

@Xhark has been continuing to run the experiments. We'll give an update when we see clear good or bad signs. He has been working on other efforts at the same time (e.g. #133).

If getting object detection models with smaller storage space usage is a need of yours (see #173 for considerations), please do feel free to give it try.

There are ongoing efforts that will make it easier (#155, #133) as they are finished.

[panchgonzalez]

I ran into this same problem a while ago. I eventually wrote a quick patch to the TF Object Detection API and the old tf.contrib.model_pruning library to prune FasterRCNN models (or any tf.slim-based object detection models).

I pushed an example to github here: https://github.com/panchgonzalez/tf_object_detection_pruning

Using Keras and tensorflow_model_optimization is definitely the way to go from now on, but for older models this should do the trick.

[anshkumar]

@panchgonzalez your patch is not complete. It's missing changes in model_main.py, model_lib.py, model_builder.py and also in the maskrccn config file. Anyway I've added these changes object_detection slim. One more important thing it's missing is that the pruning works while training but when the model is saved for evaluation and restored again, it's not able to restore all the variable. It's giving me following error:

Traceback (most recent call last):
  File "object_detection/model_main.py", line 131, in <module>
    tf.app.run()
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/home/sort/.local/lib/python3.6/site-packages/absl/app.py", line 299, in run
    _run_main(main, args)
  File "/home/sort/.local/lib/python3.6/site-packages/absl/app.py", line 250, in _run_main
    sys.exit(main(argv))
  File "object_detection/model_main.py", line 127, in main
    tf.estimator.train_and_evaluate(estimator, train_spec, eval_specs[0])
  File "/home/sort/.local/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/training.py", line 473, in train_and_evaluate
    return executor.run()
  File "/home/sort/.local/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/training.py", line 613, in run
    return self.run_local()
  File "/home/sort/.local/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/training.py", line 714, in run_local
    saving_listeners=saving_listeners)
  File "/home/sort/.local/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 370, in train
    loss = self._train_model(input_fn, hooks, saving_listeners)
  File "/home/sort/.local/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1161, in _train_model
    return self._train_model_default(input_fn, hooks, saving_listeners)
  File "/home/sort/.local/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1195, in _train_model_default
    saving_listeners)
  File "/home/sort/.local/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1490, in _train_with_estimator_spec
    log_step_count_steps=log_step_count_steps) as mon_sess:
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/monitored_session.py", line 584, in MonitoredTrainingSession
    stop_grace_period_secs=stop_grace_period_secs)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/monitored_session.py", line 1014, in __init__
    stop_grace_period_secs=stop_grace_period_secs)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/monitored_session.py", line 725, in __init__
    self._sess = _RecoverableSession(self._coordinated_creator)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/monitored_session.py", line 1207, in __init__
    _WrappedSession.__init__(self, self._create_session())
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/monitored_session.py", line 1212, in _create_session
    return self._sess_creator.create_session()
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/monitored_session.py", line 878, in create_session
    self.tf_sess = self._session_creator.create_session()
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/monitored_session.py", line 647, in create_session
    init_fn=self._scaffold.init_fn)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/session_manager.py", line 306, in prepare_session
    msg))
RuntimeError: Init operations did not make model ready for local_init.  Init op: group_deps, init fn: None, error: Variables not initialized: model_pruning/last_mask_update_step

It's looks like the variable model_pruning/last_mask_update_step is not getting initialized after restoring the graph. The Scaffold is not handling it. It looks like we can pass local_init_op to scaffold (link), but not very sure if we have to reinitialize the variable or restore it from the previous.

[panchgonzalez]

Hi @anshkumar, thanks for pointing this out. It looks like I didn't include the changes to model_lib.py and model_hparams.py in the patch. I've updated these in the repo.

Re: the runtime error you mentioned above. It looks like some changes to TF1.15 are causing this error, but I haven't quite figured out what exactly. The error goes away when downgrading to TF1.14.

I did a quick test after these changes and was able to successfully prune a MaskRCNN-InceptionV2 model

[anshkumar]

@panchgonzalez I got it somehow working. One major thing was downgrading the version of numpy to 1.16.

Also, did you find any improvement in speed or the size of model after pruning ?

EDIT

It's mentioned here that:

This tensor is sparse, has the same size as the weight tensor, and the sparsity is as set by the target_sparsity or the weight_sparsity_map hyperparameters above.
For now, it is assumed that the underlying hardware platform will provide mechanisms for compressing the sparse tensors and/or accelerating the sparse tensor computations.

In my case I'm not finding any improvement in speed. I'm using NVIDIA V100 for inference. Is there any special package requirement for inferencing a sparse tensor ?

[panchgonzalez]

@anshkumar, glad to see you got it to work.

Re: speed improvements -- that statement from tf.contrib.model_pruning library is consistent with what I've seen. Unless there's additional mechanisms to compress or otherwise take advantage of the sparse tensors, the inference speed is about the same.

Optimized inference engines (e.g., Nvidia's TensorRT framework) might be your best bet.

[anshkumar]

@panchgonzalez I thought so. Anyway thanks for the help.
Also, I'm currently working on the addition of Mask head in the SSD_meta_architecture (issue). Would you like to work along ?

[teijeong]

@Xhark can you update?