spatial_embedding.md

SpatialEmbedding: Instance Segmentation by Jointly Optimizing Spatial Embeddings and Clustering Bandwidth

July 2020

tl;dr: Single stage instance segmentation with bottom-up approach.

Overall impression

Overall performance is not that great compared to other approaches. This forms the foundation of PointTrack.

PointTrack uses a single stage instance segmentation method with a seeding location. This makes it compatible with many instance segmentation method, such as CenterMask or BlendMask.

The visualization of instance distance map looks great.

• 

Key ideas

• SpatialEmbedding predicts
  • a seed map (similar to the heatmap in CenterNet or FCOS
  • a sigma map to predict clustering bandwith (learned, largely proportional to bbox size)
  • offset map for each pixel pointing to the instance center

Technical details

• Summary of technical details

Notes

• Questions and notes on how to improve/revise the current work