This repository collects the common datasets and paper list related to the research on Sign Language🤟
This repository is continuously updating🎉
If this repository brings you some inspiration, I would be very honored😊
If you have any suggestions, feel free to contact me with: [email protected]📮
Additionally, if you could consider giving my repository a star🌟, that would motivate me a lot!
- Datasets
- Isolated sign language recognition
- Continue sign language recognition
- Sign language translation
- Sign language production
- Sign language retrieval
- Pre-training
-
Isolated sign language recognition datasets:
- WLASL: 14,289, 3,916, and 2,878 video segments in the train, dev, and test splits, respectively. [Link]
- MSASL: 16,054, 5,287, and 4,172 video segments in the train, dev, and test splits, respectively. [Link]
- NMFs-CSL: 25,608 and 6,402 video segments in the train and test splits, respectively. [Link]
- SLR500: 90,000 and 35,000 video segments in the train and test splits, respectively. [Link]
- Slovo: 15,300 and 5,100 video segments in the train and test splits, respectively. [Link]
- GSL: 34,995 and 3,500 video segments in the train and test splits, respectively. [Link]
- BOBSL: 993,000, 20,000, 165,000 video segments in train, val and test splits, respectively. [Link]
- ASL Citizen: 40,154, 10,304, 32,941 video segments in train, val and test splits, respectively. [Link]
- Auslan-Daily: 1,800, 600, 600 video segments in train, val and test splits, respectively. [Link]
-
Continue sign language recognition datasets:
- Phoenix-2014: 5,672, 540 and 629 video segments in the train, dev, and test splits, respectively. [Link]
- Phoenix-2014T: 7,096, 519 and 642 video segments in train, dev and test splits, respectively. [Link]
- CSL-Daily: 18,401, 1,077 and 1,176 video segments in train, dev and test splits, respectively. [Link]
- GSL: 8,189, 1,063 and 1,043 video segments in train, dev and test splits, respectively. [Link]
- TVB-HKSL-News: 6,516, 322 and 322 video segments in train, dev and test splits, respectively. [Link]
-
Sign language translation datasets:
- Phoenix-2014T: 7,096, 519 and 642 video segments in train, dev and test splits, respectively. [Link]
- TVB-HKSL-News: 6,516, 322 and 322 video segments in train, dev and test splits, respectively. [Link]
- CSL-Daily: 18,401, 1,077 and 1,176 video segments in train, dev and test splits, respectively. [Link]
- OpenASL: 96,476, 966 and 975 video segments in train, val and test splits, respectively. [Link]
- How2Sign: 31,128, 1,741, 2,322 video segments in train, val and test splits, respectively. [Link]
- BOBSL: 993,000, 20,000, 165,000 video segments in train, val and test splits, respectively. [Link]
- Auslan-Daily Communication: 12,441, 800, 800 video segments in train, val and test splits, respectively. [Link]
- Auslan-Daily News: 9,665, 700, 700 video segments in train, val and test splits, respectively. [Link]
- Iterative Reference Driven Metric Learning for Signer Independent Isolated Sign. ECCV 2016. [Paper]
- Skeleton-Based Gesture Recognition Using Several Fully Connected Layers with Path Signature Features and Temporal Transformer Module. AAAI 2019. [Paper]
- Transferring Cross-Domain Knowledge for Video Sign Language Recognition. CVPR 2020. [Paper]
- BSL-1K: Scaling up co-articulated sign language recognition using mouthing cues. ECCV 2020. [Paper]
- Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison. WACV 2020. [Paper][Code]
- FineHand: Learning Hand Shapes for American Sign Language Recognition. FG 2020. [Paper]
- Hand-Model-Aware Sign Language Recognition. AAAI 2021. [Paper]
- Global-Local Enhancement Network for NMF-Aware Sign Language Recognition. TOMM 2021. [Paper]
- Hand Pose Guided 3D Pooling for Word-level Sign Language Recognition. WACV 2021. [Paper]
- Pose-based Sign Language Recognition using GCN and BERT. WACVW 2021. [Paper]
- Skeleton Aware Multi-modal Sign Language Recognition. CVPRW 2021. [Paper][Code]
- Sign Language Recognition via Skeleton-Aware Multi-Model Ensemble. Arxiv 2021. [Paper][Code]
- Isolated Sign Language Recognition based on Tree Structure Skeleton Images. CVPRW 2023. [Paper][Code]
- Natural Language-Assisted Sign Language Recognition. CVPR 2023. [Paper][Code]
- Human Part-wise 3D Motion Context Learning for Sign Language Recognition. ICCV 2023. [Paper]
- Deep Sign: Hybrid CNN-HMM for Continuous Sign Language Recognition. BMVC 2016. [Paper]
- SubUNets: End-To-End Hand Shape and Continuous Sign Language Recognition. ICCV 2017. [Paper]
- Recurrent Convolutional Neural Networks for Continuous Sign Language Recognition by Staged Optimization. CVPR 2017. [Paper]
- Deep Sign: Enabling Robust Statistical Continuous Sign Language Recognition via Hybrid CNN-HMMs. IJCV 2018. [Paper]
- Iterative Alignment Network for Continuous Sign Language Recognition. CVPR 2019. [Paper]
- Weakly Supervised Learning with Multi-Stream CNN-LSTM-HMMs to Discover Sequential Parallelism in Sign Language Videos. TPAMI 2019. [Paper]
- Boosting Continuous Sign Language Recognition via Cross Modality Augmentation. ACM MM 2020. [Paper]
- Stochastic Fine-grained Labeling of Multi-state Sign Glosses for Continuous Sign Language Recognition. ECCV 2020. [Paper]
- Fully Convolutional Networks for Continuous Sign Language Recognition. ECCV 2020. [Paper]
- Spatial-Temporal Multi-Cue Network for Continuous Sign Language Recognition. AAAI 2020. [Paper]
- Visual Alignment Constraint for Continuous Sign Language Recognition. ICCV 2021. [Paper][Code]
- Self-Mutual Distillation Learning for Continuous Sign Language Recognition. ICCV 2021. [Paper]
- Signing Outside the Studio: Benchmarking Background Robustness for Continuous Sign Language Recognition. BMVC 2022. [Paper][Code]
- Temporal Lift Pooling for Continuous Sign Language Recognition. ECCV 2022. [Paper][Code]
- Deep Radial Embedding for Visual Sequence Learning. ECCV 2022. [Paper]
- C2SLR: Consistency-Enhanced Continuous Sign Language Recognition. CVPR 2022. [Paper]
- AdaBrowse: Adaptive Video Browser for Efficient Continuous Sign Language Recognition. ACM MM 2023. [Paper][Code]
- CoSign: Exploring Co-occurrence Signals in Skeleton-based Continuous Sign Language Recognition. ICCV 2023. [Paper]
- Improving Continuous Sign Language Recognition with Cross-Lingual Signs. ICCV 2023. [Paper]
- C2ST: Cross-modal Contextualized Sequence Transduction for Continuous Sign Language Recognition. ICCV 2023. [Paper]
- CVT-SLR: Contrastive Visual-Textual Transformation for Sign Language Recognition with Variational Alignment. CVPR 2023. [Paper][Code]
- Continuous Sign Language Recognition with Correlation Network. CVPR 2023. [Paper][Code]
- Distilling Cross-Temporal Contexts for Continuous Sign Language Recognition. CVPR 2023. [Paper]
- Self-Emphasizing Network for Continuous Sign Language Recognition. AAAI 2023. [Paper][Code]
- Prior-Aware Cross Modality Augmentation Learning for Continuous Sign Language Recognition. TMM 2023. [Paper]
- Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation. CVPR 2020. [Paper][Code]
- TSPNet: Hierarchical Feature Learning via Temporal Semantic Pyramid for Sign Language Translation. NeurIPS 2020. [Paper][Code]
- Neural Sign Language Translation by Learning Tokenization. FG 2020. [Paper]
- Spatial-Temporal Multi-Cue Network for Sign Language Recognition and Translation. TMM 2021. [Paper]
- Conditional Sentence Generation and Cross-Modal Reranking for Sign Language Translation. TMM 2021. [Paper]
- How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Language. CVPR 2021. [Paper][Project]
- Improving Sign Language Translation with Monolingual Data by Sign Back-Translation. CVPR 2021. [Paper]
- Skeleton-Aware Neural Sign Language Translation. ACM MM 2021. [Paper][Code]
- SimulSLT: End-to-End Simultaneous Sign Language Translation. ACM MM 2021. [Paper][Code]
- Prior Knowledge and Memory Enriched Transformer for Sign Language Translation. ACL 2022. [Paper][Code]
- Open-Domain Sign Language Translation Learned from Online Video. EMNLP 2022. [Paper][Code]
- Automatic Gloss-level Data Augmentation for Sign Language Translation. LREC 2022. [Paper]
- A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translation. CVPR 2022. [Paper][Code]
- MLSLT: Towards Multilingual Sign Language Translation. CVPR 2022. [Paper][Code]
- Two-Stream Network for Sign Language Recognition and Translation. NeurIPS 2022. [Paper][Code]
- Sign Language Translation With Hierarchical Spatio-Temporal Graph Neural Network. WACV 2022. [Paper]
- Sign Language Translation based on Transformers for the How2Sign Dataset. Report 2022. [Paper]
- Gloss-Free End-to-End Sign Language Translation. ACL 2023. [Paper][Code]
- Neural Machine Translation Methods for Translating Text to Sign Language Glosses. ACL 2023. [Paper]
- Considerations for meaningful sign language machine translation based on glosses. ACL 2023. [Paper]
- ISLTranslate: Dataset for Translating Indian Sign Language. ACL 2023. [Paper][Code]
- Sign Language Translation from Instructional Videos. CVPRW 2023. [Paper][Project][Code]
- Gloss Attention for Gloss-free Sign Language Translation. CVPR 2023. [Paper][Code]
- Sign Language Translation with Iterative Prototype. ICCV 2023. [Paper]
- Gloss-free Sign Language Translation: Improving from Visual-Language Pretraining. ICCV 2023. [paper][Code]
- SLTUNET: A Simple Unified Model for Sign Language Translation. ICLR 2023. [paper][Code]
- Cross-modality Data Augmentation for End-to-End Sign Language Translation. EMNLP 2023. [paper][Code]
- Sign2GPT: Leveraging Large Language Models for Gloss-Free Sign Language Translation. ICLR 2024. [paper]
- Conditional Variational Autoencoder for Sign Language Translation with Cross-Modal Alignment. AAAI 2024. [paper][Code]
- Factorized Learning Assisted with Large Language Model for Gloss-free Sign Language Translation. LREC-COLING 2024. [paper]
- LLMs are Good Sign Language Translators. CVPR 2024. [paper]
- GestureGAN for Hand Gesture-to-Gesture Translation in the Wild. ACM MM 2018. [Paper]
- Neural Sign Language Synthesis: Words Are Our Glosses. WACV 2020. [Paper]
- Adversarial Training for Multi-Channel Sign Language Production. BMVC 2020. [Paper][Code]
- Progressive Transformers for End-to-End Sign Language Production. ECCV 2020. [Paper][Code]
- Text2Sign: Towards Sign Language Production Using Neural Machine Translation and Generative Adversarial Networks. IJCV 2020. [Paper]
- Towards Fast and High-Quality Sign Language Production. ACM MM 2021. [Paper]
- Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitives. ICCV 2021. [Paper]
- Model-Aware Gesture-to-Gesture Translation. CVPR 2021. [Paper]
- Continuous 3D Multi-Channel Sign Language Production via Progressive Transformers and Mixture Density Networks. IJCV 2021. [Paper][Code]
- Signing at Scale: Learning to Co-Articulate Signs for Large-Scale Photo-Realistic Sign Language Production. CVPR 2022. [Paper]
- Sign Language Production with Latent Motion Transformer. WACV 2024. [Paper]
- SignAvatar: Sign Language 3D Motion Reconstruction and Generation. FG 2024. [Paper][Project]
- Select and Reorder: A Novel Approach for Neural Sign Language Production. LREC-COLING 2024. [Paper][Project]
- T2S-GPT: Dynamic Vector Quantization for Autoregressive Sign Language Production from Text. ACL 2024. [Paper][Project]
- SignGen: End-to-End Sign Language Video Generation with Latent Diffusion. ECCV 2024. [Paper][Code]
- A Simple Baseline for Spoken Language to Sign Language Translation with 3D Avatars. ECCV 2024. [Paper][Code]
- CiCo: Domain-Aware Sign Language Retrieval via Cross-Lingual Contrastive Learning. CVPR 2023. [paper][Code]
- SEDS: Semantically Enhanced Dual-Stream Encoder for Sign Language Retrieval. ACM MM 2024. [paper][Code]
- Uncertainty-aware Sign Language Video Retrieval with Probability Distribution Modeling. ECCV 2024. [Paper][Code]
- SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognition. ICCV 2021. [Paper]
- BEST: BERT Pre-Training for Sign Language Recognition with Coupling Tokenization. AAAI 2023. [Paper]
- SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understanding. TPAMI 2023. [Paper][Project]
- Self-Supervised Representation Learning with Spatial-Temporal Consistency for Sign Language Recognition. TIP 2023. [Paper][Code]