[DRAFT] Two shot pipeline

[MinhxNguyen7]

Summary

• Implement the new two-shot pipeline, where the first shot gives bounding boxes, and the second gives (all) classifications.
  • The first shot is YOLO detection on the "target" class, encompassing all shapes.
  • The second shot is a custom classifier.
• Extracts YoloDetector base class for use in old ShapeDetector (should we want to go back) and OneDetector (new).

TODOs

• Data converters to format the data for the new models.
  • All shapes to "target" Yolo Image Pipeline  imaging_training_2024#10, All Shapes to "Target" imaging_training_2024#11
  • Crop shapes according to bounding boxes and give classification. feat: BBoxToCropTransformer and test imaging_training_2024#14
• Refactor pipeline to use the new architecture.
• Switch the test over to the new ImageProcessor.
• Train first-shot YOLO model.
• Develop Resnet-34 classifier.
  • Model implementation.
  • Dataset and train loop.
  • Train the model.
• Integrate new models in.

Test Plan

• Run the image processor tests on main and this pipeline to confirm that they have the same failures and errors.

Issues

• Closes Two-shot pipeline #179

Future Follow-Up

• Consider filtering out targets that have low classification confidences.
• If non-max suppression can take a generic iterable instead of a list, we can stream the whole pipeline, which should optimize memory usage and even inference time. @SuvirBajaj is this possible?
  • Answer: No

[MinhxNguyen7]

@EricPedley Everything is done and works, but the YOLO detector trained on Isaac data is ass, so you can't tell the performance of classification. I anticipate that we won't have time to train it to be better than what we had, so we should just stick with that. However, according to PyTorch, the weights on my PC are corrupted, so I can't even load the old model. Do you mind running the tests on your machine? With the OneDetector, all the relevant tests pass on my PC.