Recent Advances in Video Object Segmentation (VOS). VOS works before 2022 can be found in our survey paper:
Deep Learning for Video Object Segmentation: A Review / paper / project page
BibTex
@article{gao2023deep, title={Deep learning for video object segmentation: a review}, author={Gao, Mingqi and Zheng, Feng and Yu, James JQ and Shan, Caifeng and Ding, Guiguang and Han, Jungong}, journal={Artificial Intelligence Review}, volume={56}, number={1}, pages={457--531}, year={2023}, publisher={Springer} }
🧸 We mark different VOS tasks with coloured squares:
🟦SVOS |
 |
🟧RVOS |
 |
🟩UVOS |
 |
🟥AVOS |
 |
💠VMAT |
 |
⬜XVOS |
Other types of VOS |
🧸 Please feel free to send us pull requests to add VOS works.
Links for a quick jump: ArXiv 2024, 🔥NeurIPS 2024🔥, ECCV 2024, CVPR 2024, AAAI 2024, Journals 2024, EMNLP 2023, NeurIPS 2023, ACMMM 2023, ICCV 2023, CVPR 2023, IJCAI 2023, AAAI 2023, Journals 2023, Earlier ArXiv 2023, NeurIPS 2022, ECCV 2022, CVPR 2022, AAAI 2022, Journals 2022
No Active Workshops/Challenges.
Finished
🟦 SVOS 🟧 RVOS - LSVOS @ECCV 2024 (Finished)
🟦 SVOS - VOTS @ECCV 2024 (Finished)
🟦 SVOS 🟧 RVOS - PVUW @CVPR 2024 (Finished)
🟦 SVOS: SA-V (2024), LVOS (2023), MOSE (2023), VOST (2023), VISOR (2022), YouTube-VOS (2018/2019), DAVIS (2016/2017)
🟧 RVOS: ReVOS (2024), MeViS (2023), Ref-YouTube-VOS (2020), Ref-DAVIS (2018), J-HMDB-Sentences (2018), A2D-Sentences (2018)
🟩 UVOS: DAVIS (2016)
🟥 AVOS: AVSBench (2022)
💠 VMAT: VideoMatte240K (2021), CRGNN (2021)
🟧 RVOS Dec - paper / project page - ViCaS: A Dataset for Combining Holistic and Pixel-level Video Understanding using Captions with Grounded Segmentation
🟥 AVOS Dec - paper / code - Collaborative Hybrid Propagator for Temporal Misalignment in Audio-Visual Segmentation
⬜ XVOS Dec - paper / code - Using Diffusion Priors for Video Amodal Segmentation (segment both visible and invisible (e.g., occluded) video objects)
🟩 UVOS Dec - paper / project page - Video Decomposition Prior: A Methodology to Decompose Videos into Layers (with applications in UVOS)
🟦 SVOS Dec - paper / project page - Multi-Granularity Video Object Segmentation
🟧 RVOS Dec - paper / project page - Referring Video Object Segmentation via Language-aligned Track Selection
🟩 UVOS Nov - paper / code - Det-SAM2: Technical Report on the Self-Prompting Segmentation Framework Based on Segment Anything Model 2
🟩 UVOS Nov - paper / code - On Moving Object Segmentation from Monocular Video with Transformers
🟦 SVOS Nov - paper / code - Track Anything Behind Everything: Zero-Shot Amodal Video Object Segmentation
🟦 SVOS Nov - paper / code - Efficient Track Anything
🟦 SVOS Nov - paper / code - A Distractor-Aware Memory for Visual Object Tracking with SAM2
🟧 RVOS Nov - paper / code - SAMWISE: Infusing wisdom in SAM2 for Text-Driven Video Segmentation
🟧 RVOS Nov - paper / project page - Motion-Grounded Video Reasoning: Understanding and Perceiving Motion at Pixel Level
🟦 SVOS Nov - paper / code - SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memory
🟦 SVOS Nov - paper / code - LiVOS: Light Video Object Segmentation with Gated Linear Matching
🟧 RVOS Oct - paper / project page - ReferEverything: Towards Segmenting Everything We Can Speak of in Videos
🟦 SVOS Oct - paper / code - SMITE: Segment Me In TimE
🟦 SVOS Oct - paper / code - SAM2Long: Enhancing SAM 2 for Long Video Segmentation with a Training-Free Memory Tree
🟦 SVOS Sep - paper / code - X-Prompt: Multi-modal Visual Prompt for Video Object Segmentation
🟩 UVOS Sep - paper / code - When SAM2 Meets Video Camouflaged Object Segmentation: A Comprehensive Evaluation and Adaptation
🟦 SVOS Sep - paper / code - Memory Matching is not Enough: Jointly Improving Memory Matching and Decoding for Video Object Segmentation
🟥 AVOS 🟧 RVOS Aug - paper / code - Unleashing the Temporal-Spatial Reasoning Capacity of GPT for Training-Free Audio and Language Referenced Video Object Segmentation
🟥 AVOS Aug - paper / code - AVESFormer: Efficient Transformer Design for Real-Time Audio-Visual Segmentation
⬜ XVOS Jul - paper / code - Strike the Balance: On-the-Fly Uncertainty based User Interactions for Long-Term Video Object Segmentation
🟦 SVOS Jul - paper / code - SAM 2: Segment Anything in Images and Videos
🟧 RVOS Jul - paper / code - ViLLa: Video Reasoning Segmentation with Large Language Model
🟩 UVOS Jul - paper / code - Improving Unsupervised Video Object Segmentation via Fake Flow Generation
🟦 SVOS Jul - paper / code - Learning Spatial-Semantic Features for Robust Video Object Segmentation
🟥 AVOS Jun - paper / code - Progressive Confident Masking Attention Network for Audio-Visual Segmentation
🟥 AVOS Jun - paper / code - Extending Segment Anything Model into Auditory and Temporal Dimensions for Audio-Visual Segmentation
🟧 RVOS Jun - paper / code - GroPrompt: Efficient Grounded Prompting and Adaptation for Referring Video Object Segmentation
🟧 RVOS - paper / code - One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videos
🟦 SVOS - paper / code - Spatial-Temporal Multi-level Association for Video Object Segmentation
🟦 SVOS - paper / code - OneVOS: Unifying Video Object Segmentation with All-in-One Transformer Framework
🟧 RVOS - paper / code - VISA: Reasoning Video Object Segmentation via Large Language Model
🟧 RVOS - paper / code - Exploring Pre-trained Text-to-Video Diffusion Models for Referring Video Object Segmentation
🟧 RVOS - paper / code - ActionVOS: Actions as Prompts for Video Object Segmentation
🟧 RVOS 🟥 AVOS - paper / code & dataset - R2-Bench: Benchmarking the Robustness of Referring Perception Models under Perturbations
🟧 RVOS 🟥 AVOS - paper / code - Ref-AVS: Refer and Segment Objects in Audio-Visual Scenes
🟥 AVOS - paper / code - Stepping Stones: A Progressive Training Strategy for Audio-Visual Semantic Segmentation
🟩 UVOS - paper / code - Betrayed by Attention: A Simple yet Effective Approach for Self-supervised Video Object Segmentation
💠 VMAT - paper / code - MaGGIe: Masked Guided Gradual Human Instance Matting
⬜ XVOS - paper / code - Point-VOS: Pointing Up Video Object Segmentation
🟥 AVOS - paper / code - Towards Robust Audiovisual Segmentation in Complex Environments with Quantization-based Semantic Decomposition
🟥 AVOS - paper / code - Cooperation Does Matter: Exploring Multi-Order Bilateral Relations for Audio-Visual Segmentation
🟥 AVOS - paper / code - A Closer Look at Audio-Visual Segmentation
🟩 UVOS - paper / code - Depth-aware Test-Time Training for Zero-shot Video Object Segmentation
🟩 UVOS - paper / code - Dual Prototype Attention for Unsupervised Video Object Segmentation
🟩 UVOS - paper / code - Guided Slot Attention for Unsupervised Video Object Segmentation
🟧 RVOS - paper / code - Decoupling Static and Hierarchical Motion Perception for Referring Video Segmentation
🟧 RVOS - paper / code - LoSh: Long-Short Text Joint Prediction Network for Referring Video Object Segmentation
🟧 RVOS - paper / code - Universal Segmentation at Arbitrary Granularity with Language Instruction
🟦 SVOS 🟧 RVOS - paper / code - UniVS: Unified and Universal Video Segmentation with Prompts as Queries
🟦 SVOS 🟩 UVOS - paper / code - Matching Anything By Segmenting Anything
🟦 SVOS - paper / code - RMem: Restricted Memory Banks Improve Video Object Segmentation
🟦 SVOS - paper / code - Event-assisted Low-Light Video Object Segmentation
🟦 SVOS - paper / code - Putting the Object Back into Video Object Segmentation
🟧 RVOS 🟥 AVOS - paper / code - Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentation
🟩 UVOS - paper / code - Generalizable Fourier Augmentation for Unsupervised Video Object Segmentation
🟧 RVOS - paper / code - TPAMI Language-Aware Vision Transformer for Referring Segmentation
🟦 SVOS - paper / code - TPAMI Continuous-time Object Segmentation using High Temporal Resolution Event Camera
🟧 RVOS - paper / code - Towards Noise-Tolerant Speech-Referring Video Object Segmentation: Bridging Speech and Text (Spoken language as referring guidance)
🟧 RVOS - paper / code - SOC: Semantic-Assisted Object Cluster for Referring Video Object Segmentation
🟦 SVOS - paper / code - Test-time Training for Matching-based Video Object Segmentation
🟦 SVOS - paper / code - From ViT Features to Training-free Video Object Segmentation via Streaming-data Mixture Models
🟩 UVOS - paper / code - SimulFlow: Simultaneously Extracting Feature and Identifying Target for Unsupervised Video Object Segmentation
🟩 UVOS - paper / code - Temporally Efficient Gabor Transformer for Unsupervised Video Object Segmentation
🟦 SVOS - paper / code - Exploring the Adversarial Robustness of Video Object Segmentation via One-shot Adversarial Attacks
🟥 AVOS - paper / code - CATR: Combinatorial-Dependence Audio-Queried Transformer for Audio-Visual Video Segmentation
🟥 AVOS - paper / code - Audio-Visual Segmentation by Exploring Cross-Modal Mutual Semantics
⬜ XVOS - paper / code - Multi-grained Temporal Prototype Learning for Few-shot Video Object Segmentation
🟩 UVOS - paper / code - Time Does Tell: Self-Supervised Time-Tuning of Dense Image Representations (self-supervised learning for UVOS)
🟩 UVOS - paper / code - Isomer: Isomerous Transformer for Zero-Shot Video Object Segmentation
🟩 UVOS - paper / code - Unsupervised Video Object Segmentation with Online Adversarial Self-Tuning
🟩 UVOS 🟧 RVOS - paper / code - DEVA: Tracking Anything with Decoupled Video Segmentation
🟧 RVOS - paper / code - Temporal Collection and Distribution for Referring Video Object Segmentation
🟧 RVOS - paper / code - Robust Referring Video Object Segmentation with Cyclic Structural Consensus
🟧 RVOS - paper / code - Spectrum-guided Multi-granularity Referring Video Object Segmentation
🟧 RVOS - paper / code - OnlineRefer: A Simple Online Baseline for Referring Video Object Segmentation
🟧 RVOS - paper / code - Learning Cross-Modal Affinity for Referring Video Object Segmentation Targeting Limited Samples
🟧 RVOS - paper / code - HTML: Hybrid Temporal-scale Multimodal Learning Framework for Referring Video Object Segmentation
🟧 RVOS - paper / code & dataset - MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressions
🟦 SVOS - paper / code - Integrating Boxes and Masks: A Multi-Object Framework for Unified Visual Tracking and Segmentation
🟦 SVOS - paper / code - XMem++: Production-level Video Segmentation From Few Annotated Frames
🟦 SVOS - paper / code - Scalable Video Object Segmentation with Simplified Framework
🟦 SVOS - paper / code - Alignment Before Aggregation: Trajectory Memory Retrieval Network for Video Object Segmentation
🟦 SVOS - paper / code - SegGPT: Segmenting Everything In Context
🟦 SVOS - paper / code & dataset - LVOS: A Benchmark for Long-term Video Object Segmentation
🟦 SVOS - paper / code & dataset - MOSE: A New Dataset for Video Object Segmentation in Complex Scenes
💠 VMAT - paper / code - Adaptive Human Matting for Dynamic Videos
🟩 UVOS - paper / code - MED-VT: Multiscale Encoder-Decoder Video Transformer with Application to Object Segmentation
🟦 SVOS - paper / code - Boosting Video Object Segmentation via Space-time Correspondence Learning
🟦 SVOS 🟧 RVOS - paper / code - Universal Instance Perception as Object Discovery and Retrieval
🟦 SVOS - paper / code - TarViS: A Unified Approach for Target-Based Video Segmentation
🟦 SVOS - paper / code - Two-shot Video Object Segmetnation
🟦 SVOS - paper / code - MobileVOS: Real-Time Video Object Segmentation Contrastive Learning meets Knowledge Distillation
🟦 SVOS - paper / code - Look Before You Match: Instance Understanding Matters in Video Object Segmentation
⬜ XVOS - paper / code & dataset - Breaking the “Object” in Video Object Segmentation
🟥 AVOS - paper / code - Discovering Sounding Objects by Audio Queries for Audio Visual Segmentation
🟦 SVOS - paper / code & dataset - Video Object Segmentation in Panoptic Wild Scenes
🟦 SVOS - paper / code - Learning to Learn Better for Video Object Segmentation
🟩 UVOS - paper / code - TIP Hierarchical Graph Pattern Understanding for Zero-Shot Video Object Segmentation
🟩 UVOS - paper / code - TCSVT Online Unsupervised Video Object Segmentation via Contrastive Motion Clustering
🟦 SVOS - paper / code - TIP Hierarchical Co-Attention Propagation Network for Zero-Shot Video Object Segmentation
🟧 RVOS - paper / code - TPAMI VLT: Vision-Language Transformer and Query Generation for Referring Segmentation
🟧 RVOS - paper / code - TPAMI Local-Global Context Aware Transformer for Language-Guided Video Segmentation
🟦 SVOS - paper / code and dataset - LVOS (v2, with more data): A Benchmark for Large-scale Long-term Video Object Segmentation
🟧 RVOS - paper / code - Driving Referring Video Object Segmentation with Vision-Language Pre-trained Models
🟦 SVOS - paper / code - One-shot Training for Video Object Segmentation
🟦 SVOS - paper / code - Global Motion Understanding in Large-Scale Video Object Segmentation
🟦 SVOS - paper / code - DeVOS: Flow-Guided Deformable Transformer for Video Object Segmentation
⬜ XVOS - paper / code - Dynamic in Static: Hybrid Visual Correspondence for Self-Supervised Video Object Segmentation
🟩 UVOS - paper / code - Moving Object Segmentation: All You Need Is SAM (and Flow)
🟧 RVOS - paper / code - Towards Temporally Consistent Referring Video Object Segmentation
🟥 AVOS - paper / code - Unsupervised Audio-Visual Segmentation with Modality Alignment
🟦 SVOS - paper / code - Efficient Video Object Segmentation via Modulated Cross-Attention Memory
⬜ XVOS - paper / code - ClickVOS: Click Video Object Segmentation
🟥 AVOS - paper / code - Bootstrapping Audio-Visual Segmentation by Strengthening Audio Cues
⬜ XVOS - paper / code - Vivim: a Video Vision Mamba for Medical Video Object Segmentation
⬜ XVOS - paper / code - Self-supervised Video Object Segmentation with Distillation Learning of Deformable Attention
⬜ XVOS - paper / code - Explore Synergistic Interaction Across Frames for Interactive Video Object Segmentation
🟧 RVOS - paper / code - Tracking with Human-Intent Reasoning
🟦 SVOS 🟧 RVOS - paper / code - UniRef++: Segment Every Reference Object in Spatial and Temporal Spaces
🟩 UVOS Dec - paper / code - Appearance-based Refinement for Object-Centric Motion Segmentation
🟦 SVOS 🟧 RVOS - paper / code - General Object Foundation Model for Images and Videos at Scale
🟦 SVOS - paper / code - M3T: Multi-Scale Memory Matching for Video Object Segmentation and Tracking
🟥 AVOS - paper / code - Cooperation Does Matter: Exploring Multi-Order Bilateral Relations for Audio-Visual Segmentation
🟧 RVOS - paper / code - VIDiff: Translating Videos via Multi-Modal Instructions with Diffusion Models
⬜ XVOS - paper / code - Sketch-based Video Object Segmentation: Benchmark and Analysis
⬜ XVOS - paper / code - Learning the What and How of Annotation in Video Object Segmentation
🟦 SVOS - paper / code - Sub-token ViT Embedding via Stochastic Resonance Transformers (support svos)
🟦 SVOS - paper / code & dataset - PanoVOS: Bridging Non-panoramic and Panoramic Views with Transformer for Video Segmentation
🟩 UVOS - paper / code - Treating Motion as Option with Output Selection for Unsupervised Video Object Segmentation
🟥 AVOS - paper / code - Rethinking Audiovisual Segmentation with Semantic Quantization and Decomposition
🟦 SVOS - paper / code - Joint Modeling of Feature, Correspondence, and a Compressed Memory for Video Object Segmentation
🟧 RVOS 🟥 AVOS - paper / code - EPCFormer: Expression Prompt Collaboration Transformer for Universal Referring Video Object Segmentation
🟧 RVOS - paper / code - Learning Referring Video Object Segmentation from Weak Annotation
🟩 UVOS - paper / code - UVOSAM: A Mask-free Paradigm for Unsupervised Video Object Segmentation via Segment Anything Model
⬜ XVOS - paper / code - Segment and Track Anything
⬜ XVOS - paper / code - Track Anything: Segment Anything Meets Videos
⬜ XVOS - paper / code - Reliability-Hierarchical Memory Network for Scribble-Supervised Video Object Segmentation
🟦 SVOS - paper / code - Tracking Anything in High Quality
🟧 RVOS - paper / code - Referring Video Object Segmentation with Inter-Frame Interaction and Cross-Modal Correlation
🟧 RVOS - paper / code - RefSAM: Efficiently Adapting Segmenting Anything Model for Referring Video Object Segmentation
⬜ XVOS - paper / code - Segment Anything Meets Point Tracking
🟦 SVOS - paper / code - Decoupling Features in Hierarchical Propagation for Video Object Segmentation
⬜ XVOS - paper / code - Self-supervised Amodal Video Object Segmentation
🟦 SVOS - paper / code - XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Model
🟦 SVOS - paper / code - BATMAN: Bilateral Attention Transformer in Motion-Appearance Neighboring Space for Video Object Segmentation
🟦 SVOS - paper / code - Learning Quality-aware Dynamic Memory for Video Object Segmentation
🟦 SVOS - paper / code - Tackling Background Distraction in Video Object Segmentation
🟦 SVOS - paper / code - Global Spectral Filter Memory Network for Video Object Segmentation
🟩 UVOS - paper / code - Hierarchical Feature Alignment Network for Unsupervised Video Object Segmentation
🟧 RVOS - paper / code - End-to-End Referring Video Object Segmentation With Multimodal Transformers
🟧 RVOS - paper / code - Language As Queries for Referring Video Object Segmentation
🟧 RVOS - paper / code - Language-Bridged Spatial-Temporal Interaction for Referring Video Object Segmentation
🟧 RVOS - paper / code - Multi-Level Representation Learning With Semantic Alignment for Referring Video Object Segmentation
🟦 SVOS - paper / code - Recurrent Dynamic Embedding for Video Object Segmentation
🟦 SVOS - paper / code - Accelerating Video Object Segmentation With Compressed Video
🟦 SVOS - paper / code - SWEM: Towards Real-Time Video Object Segmentation With Sequential Weighted Expectation-Maximization
🟦 SVOS - paper / code - Per-Clip Video Object Segmentation
⬜ XVOS - paper / code - Wnet: Audio-Guided Video Object Segmentation via Wavelet-Based Cross-Modal Denoising Networks
⬜ XVOS - paper / code & dataset - YouMVOS: An Actor-Centric Multi-Shot Video Object Segmentation Dataset
🟦 SVOS - paper / code - Siamese Network with Interactive Transformer for Video Object Segmentation
🟦 SVOS - paper / code - Reliable Propagation-Correction Modulation for Video Object Segmentation
🟧 RVOS - paper / code - You Only Infer Once: Cross-Modal Meta-Transfer for Referring Video Object Segmentation
🟩 UVOS - paper / code - Iteratively Selecting an Easy Reference Frame Makes Unsupervised Video Object Segmentation Easier
🟦 SVOS - paper / code - TPAMI Video Object Segmentation Using Kernelized Memory Network With Multiple Kernels
🟦 SVOS - paper / code - TIP From Pixels to Semantics: Self-Supervised Video Object Segmentation With Multiperspective Feature Mining
🟦 SVOS - paper / code - TIP Delving Deeper Into Mask Utilization in Video Object Segmentation
🟦 SVOS - paper / code - TIP Adaptive Online Mutual Learning Bi-Decoders for Video Object Segmentation
End of the list. 🌱
VOS papers and datasets before 2022 could be found below:
Deep Learning for Video Object Segmentation: A Review / paper / project page