Added mmdet smqtk plugin implementation, tests, and config files#39
Added mmdet smqtk plugin implementation, tests, and config files#39alexander-lynch wants to merge 2 commits intoKitware:masterfrom
Conversation
|
Going to give this a rebase poke since I needed to reactivate the actions workflows. |
Purg
force-pushed
the
mmdetection_plugin
branch
from
e8b4737 to
30cb555
Compare
| import mmcv | ||
|
|
||
| import torch.nn | ||
| from torch.utils.data import DataLoader, Dataset, IterableDataset |
There was a problem hiding this comment.
This implementation is "optional" in this package as these dependencies are not requirements on the base package (they should continue to not be base package requirements). We will need to add protection to imports of these implementation-specific requirements. See other impls here like smqtk_detection/impls/detect_image_objects/resnet_frcnn.py.
We should also update the pyproject.yaml to include optional dependencies and an extra specification for this mmdet plugin.
| It is expected that classes will be derived from this base that concretely | ||
| defines the data augmentation to appropriate transform input imagery for | ||
| the configured network. |
There was a problem hiding this comment.
I don't think this applies anymore for this implementation (copy/paste from detectron2 plugin).
| if len(batch) < self._batch_size: | ||
| continue |
There was a problem hiding this comment.
This will only handle when the input and batch size are evenly divisible. If there is a partial batch left over it will not currently be processed. A post-loop partial-batch condition will need to be added after this top-level for-loop.
| This plugin attempts to be intelligent in how it handles different kinds of | ||
| iterable inputs. When given a ``Dataset`` or countable sequence (has | ||
| ``__len__`` and ``__getitem__``), any valid value may be provided to | ||
| ``num_workers`` as ``DataLoader`` might accept. | ||
| However, when the input to ``detect_objects`` is an uncountable iterable, | ||
| like a generic generator or stream source, the ``num_workers`` value should | ||
| usually be either 0 or 1. | ||
| This is due to the input iterable being copied for each worker, which may | ||
| not result in desired behavior. | ||
| For example: | ||
| * when the input iterable involves non-trivial operations per yield, | ||
| these operations are duplicated for each copy of the iterable as | ||
| traversed on each worker, probably resulting in excessive use of | ||
| resources. E.g. if the iterable is loading images from disk, each | ||
| worker is loading every image as it traverses their copy of the | ||
| iterable, even though each worker may only operate on a minority of | ||
| elements traversed. | ||
| * when the input iterable yields real-time data or is otherwise **not** | ||
| idempotent, like an iterable that yields images from a webcam stream, | ||
| each traversal of a copy of that iterable will produce different values | ||
| for equivalent "indices" since what is returned is conditional on when | ||
| ``next()`` is requested. Since iterators are copied to N separate | ||
| workers, each making independent next requests, the e.g. 64th | ||
| ``next()`` for each worker might yield a different image matrix. |
There was a problem hiding this comment.
This whole section of the doc-string doesn't seem to apply anymore.
| self._batch_size = batch_size | ||
| self._weights_uri = weights_uri | ||
| self._model_lazy_load = model_lazy_load | ||
| self._num_workers = num_workers |
There was a problem hiding this comment.
Num workers does not need to apply any more. We should remove this constructor variable/doc and the use in config return.
| return torch.as_tensor(aug_image) | ||
|
|
||
|
|
||
| def _aug_one_image(image: np.ndarray, aug: AutoAugment, gt_bboxes=[]) -> Dict[str, Union[torch.Tensor, int]]: |
There was a problem hiding this comment.
This function does not seem to be used in this implementation.
| } | ||
|
|
||
|
|
||
| def _trivial_batch_collator(batch: Any) -> Any: |
There was a problem hiding this comment.
This function does not seem to be used in this implementation.
| return batch | ||
|
|
||
|
|
||
| def _to_tensor(np_image: np.ndarray) -> torch.Tensor: |
There was a problem hiding this comment.
This function does not seem to be used in this implementation.
| def _format_detections( | ||
| self, | ||
| preds | ||
| ): |
There was a problem hiding this comment.
This could use a doc-string for what its purpose is in this context, e.g. what preds is expected to be (also needs type annotations).
| for pred in preds: | ||
| a_bboxes = [] | ||
| score_dicts = [] | ||
| for i, bbox in enumerate(pred): | ||
| a_bboxes.append(AxisAlignedBoundingBox( | ||
| [bbox[0], bbox[1]], [bbox[2], bbox[3]])) | ||
| class_dict = zero_dict.copy() | ||
| class_dict[self._classes[i]] = bbox[4] | ||
| score_dicts.append(class_dict) | ||
| break |
There was a problem hiding this comment.
Given the single-prediction context of this method compared to Brian's original example, I think these loops should potentially be reworked.
| @@ -0,0 +1,275 @@ | |||
| import logging | |||
There was a problem hiding this comment.
This file probably doesn't need to be called "mmdet_base.py": we can drop the "base" part since it doesn't seem that we need to subclass this like the detectron2 impl needed.
2 participants
Moved this work to smqtk instead of learn https://gitlab.kitware.com/darpa_learn/learn/-/merge_requests/179