Skip to content

Latest commit

 

History

History
231 lines (217 loc) · 25.6 KB

README.md

File metadata and controls

231 lines (217 loc) · 25.6 KB
Raw

Awesome Sign Language Processing

Awesome

A curated list of sign language procesing (e.g., sign language recognition, sign language translation) and related area (e.g., speech translation, motion generation) resources. The old version can be found here.

Contents

Conference Paper

Sign Language Recognition

  • [NeurIPS 2024] MM-WLAuslan: Multi-View Multi-Modal Word-Level Australian Sign Language Recognition Dataset. [paper]
    Xin Shen, Heming Du, Hongwei Sheng, Shuyun Wang, Hui Chen, Huiqiang Chen, Zhuojie Wu, Xiaobiao Du, Jiaying Ying, Ruihan Lu, Qingzheng Xu, Xin Yu
  • [ICLR 2025 Submission] Representing Signs as Signs: One-Shot ISLR to Facilitate Functional Sign Language Technologies. [paper]
    Anonymous authors
  • [EMNLP 2024] SignCLIP: Connecting Text and Sign Language by Contrastive Learning. [paper]
    Zifan Jiang, Gerard Sant, Amit Moryossef, Mathias Müller, Rico Sennrich, Sarah Ebling
  • [EMNLP 2024] Towards Online Continuous Sign Language Recognition and Translation. [paper]
    Ronglai Zuo, Fangyun Wei, Brian Mak
  • [CVPR 2024] SignGraph: A Sign Sequence is Worth Graphs of Nodes. [paper] [code]
    Shiwei Gan, Yafeng Yin, Zhiwei Jiang, Hongkai Wen, Kang Xia, Lei Xie, Sanglu Lu
  • [EMNLP Findings 2023] Handshape-Aware Sign Language Recognition: Extended Datasets and Exploration of Handshape-Inclusive Methods. [paper]
    Xuan Zhang, Kevin Duh
  • [EMNLP Findings 2023] Making Body Movement in Sign Language Corpus Accessible for Linguists and Machines with Three-Dimensional Normalization of MediaPipe. [paper]
    Houda Bouamor, Juan Pino, Kalika Bali
  • [EMNLP Findings 2023] Linguistically Motivated Sign Language Segmentation. [paper]
    Amit Moryossef, Zifan Jiang, Mathias Müller, Sarah Ebling, Yoav Goldberg
  • [NeurIPS 2023 Dataset] ASL Citizen: A Community-Sourced Dataset for Advancing Isolated Sign Language Recognition. [paper]
    Aashaka Desai, Lauren Berger, Fyodor O Minakov, Vanessa Milan, Chinmay Singh, Kriston L Pumphrey, Richard Ladner, Hal Daumé III, Alex Xijie Lu, Naomi Caselli, Danielle Bragg
  • [NeurIPS 2023 Dataset] PopSign ASL v1.0: An Isolated American Sign Language Dataset Collected via Smartphones. [paper]
    Thad Starner, Sean Forbes, Matthew So, David Martin, Rohit Sridhar, Gururaj Deshpande, Sam Sepah, Sahir Shahryar, Khushi Bhardwaj, Tyler Kwok, Daksh Sehgal, Saad Hassan, Bill Neubauer, Sofia Anandi Vempala, Alec Tan, Jocelyn Heath, Unnathi Utpal Kumar, Priyanka Vijayaraghavan Mosur, Tavenner M. Hall, Rajandeep Singh et al.
  • [ICCV 2023] CoSign: Exploring Co-occurrence Signals in Skeleton-based Continuous Sign Language Recognition. [paper]
    Peiqi Jiao, Yuecong Min, Yanan Li, Xiaotao Wang, Lei Lei, Xilin Chen
  • [ICCV 2023] Human Part-wise 3D Motion Context Learning for Sign Language Recognition. [paper]
    Taeryung Lee, Yeonguk Oh and Kyoung Mu Lee
  • [ICCV 2023] Improving Continuous Sign Language Recognition with Cross-Lingual Signs. [paper]
    Fangyun Wei, Yutong Chen
  • [ICCV 2023] C2ST: Cross-modal Contextualized Sequence Transduction for Continuous Sign Language Recognition. [paper]
    Huaiwen Zhang, Zihang Guo, Yang Yang, Xin Liu, De Hu
  • Complete List of Sign Language Recognition

Sign Language Translation

  • [Neurips 2024] Scaling Sign Language Translation.[paper]
    Biao Zhang, Garrett Tanzer, Orhan Firat
  • [Neurips 2024] Improving Gloss-free Sign Language Translation by Reducing Representation Density.[paper]
    Jinhui Ye, Xing Wang, Wenxiang Jiao, Junwei Liang, Hui Xiong
  • [ICLR 2025 Submission] YouTube-SL-25: A Large-Scale, Open-Domain Multilingual Sign Language Parallel Corpus. [paper]
    Anonymous authors
  • [ICLR 2025 Submission] Uni-Sign: Toward Unified Sign Language Understanding at Scale. [paper]
    Anonymous authors
  • [ICLR 2025 Submission] Hybrid Model Collaboration for Sign Language Translation with VQ-VAE and RAG Enhanced LLMS. [paper]
    Anonymous authors
  • [EMNLP 2024] Reconsidering Sentence-Level Sign Language Translation. [paper]
    Garrett Tanzer, Maximus Shengelia, Ken Harrenstien, David Uthus
  • [ECCV 2024] Visual Alignment Pre-training for Sign Language Translation. [paper]
    Peiqi Jiao, Yuecong Min, Xilin Chen
  • [ECCV 2024] EvSign: Sign Language Recognition and Translation with Streaming Events. [paper]
    Pengyu Zhang, Hao Yin, Zeren Wang, Wenyue Chen, Sheng Ming Li, Dong Wang, Huchuan Lu, Xu Jia
  • [ACL 2024] Towards Privacy-Aware Sign Language Translation at Scale. [paper]
    Phillip Rust, Bowen Shi, Skyler Wang, Necati Cihan Camgoz, Jean Maillard
  • [ACL 2024] Sign Language Translation with Sentence Embedding Supervision. [paper]
    HAMIDULLAH Yasser, Josef van Genabith, Cristina España-Bonet
  • [ACL Findings 2024] iSign: A Benchmark for Indian Sign Language Processing. [paper]
    Abhinav Joshi, Romit Mohanty, Mounika Kanakanti, Andesha Mangla, Sudeep Choudhary, Monali Barbate, Ashutosh Modi
  • [ACL Findings 2024] Unsupervised Sign Language Translation and Generation. [paper]
    Zhengsheng Guo, Zhiwei He, Wenxiang Jiao, Xing Wang, Rui Wang, Kehai Chen, Zhaopeng Tu, Yong Xu, Min Zhang
  • [CVPR 2024] LLMs are Good Sign Language Translators. [paper]
    Jia Gong, Lin Geng Foo, Yixuan He, Hossein Rahmani, Jun Liu
  • [ICLR 2024] Sign2GPT: Leveraging Large Language Models for Gloss-Free Sign Language Translation. [paper] Ryan Wong, Necati Cihan Camgoz, Richard Bowden
  • [EMNLP Findings 2023] JWSign: A Highly Multilingual Corpus of Bible Translations for more Diversity in Sign Language Processing. [paper]
    Shester Gueuwou, Sophie Siake, Colin Leong, Mathias Müller
  • [EMNLP Findings 2023] Cross-modality Data Augmentation for End-to-End Sign Language Translation. [paper]
    Jinhui Ye, Wenxiang Jiao, Xing Wang, Zhaopeng Tu, Hui Xiong
  • [OpenReview] Towards Faithful Sign Language Translation. [paper]
    Yuecong Min, Xilin Chen
  • [NeurIPS 2023 Dataset] YouTube-ASL: A Large-Scale, Open-Domain American Sign Language-English Parallel Corpus. [paper]
    David Uthus, Garrett Tanzer, Manfred Georg
  • [NeurIPS 2023 Dataset] Auslan-Daily: Australian Sign Language Translation for Daily Communication and News. [paper]
    Xin Shen, Shaozu Yuan, Hongwei Sheng, Heming Du, Xin Yu
  • [ICCV 2023] Sign Language Translation with Iterative Prototype. [paper]
    Huijie Yao, Wengang Zhou, Hao Feng, Hezhen Hu, Hao Zhou, Houqiang Li
  • [ICCV 2023] Gloss-free Sign Language Translation: Improving from Visual-Language Pretraining. [paper]
    Benjia Zhou, Zhigang Chen, Albert Clapés, Jun Wan, Yanyan Liang, Sergio Escalera, Zhen Lei, Du Zhang
  • Complete List of Sign Language Translation

Sign Language Production

  • [EMNLP Findings 2024] Word-Conditioned 3D American Sign Language Motion Generation. [paper]
    Lu Dong, Xiao Wang, Ifeoma Nwogu
  • [ECCV 2024] A Simple Baseline for Spoken Language to Sign Language Translation with 3D Avatars. [paper]
    Ronglai Zuo, Fangyun Wei, Zenggui Chen, Brian Mak, Jiaolong Yang, Xin Tong
  • [ECCV 2024] Pose Guided Fine-Grained Sign Language Video Generation. [paper]
    Tongkai Shi, Lianyu Hu, Fanhua Shang, Jichao Feng, liu peidong, Wei Feng
  • [ECCV 2024] SignGen: End-to-End Sign Language Video Generation with Latent Diffusion. [paper]
    Fan Qi, Yu Duan, Changsheng Xu, Huaiwen Zhang
  • [ACL 2024] T2S-GPT: Dynamic Vector Quantization for Autoregressive Sign Language Production from Text. [paper]
    Aoxiong Yin, Haoyuan Li, Kai Shen, Siliang Tang, Yueting Zhuang
  • [ACL Findings 2024] MS2SL: Multimodal Spoken Data-Driven Continuous Sign Language Production. [paper]
    Jian Ma, Wenguan Wang, Yi Yang, Feng Zheng
  • [CVPR 2024] Neural Sign Actors: A diffusion model for 3D sign language production from text. [paper]
    Vasileios Baltatzis, Rolandos Alexandros Potamias, Evangelos Ververas, Guanxiong Sun, Jiankang Deng, Stefanos Zafeiriou
  • [WACV 2024] Sign Language Production With Latent Motion Transformer. [paper]
    Pan Xie, Taiying Peng, Yao Du, Qipeng Zhang
  • [CVPR 2022] Signing at Scale: Learning to Co-Articulate Signs for Large-Scale Photo-Realistic Sign Language Production. [paper]
    Ben Saunders, Necati Cihan Camgoz, Richard Bowden
  • [ICCV 2021] Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitives. [paper]
    Ben Saunders; Necati Cihan Camgoz; Richard Bowden
  • Complete List of Sign Language Production

Other Sign Language Topic

  • [EMNLP Findings 2024] Gloss2Text: Sign Language Gloss translation using LLMs and Semantically Aware Label Smoothing. [paper]
    Pooya Fayyazsanavi, Antonios Anastasopoulos, Jana Kosecka
  • [EMNLP 2024] Unsupervised Discrete Representations of American Sign Language. [paper]
    Artem Abzaliev, Rada Mihalcea
  • [EMNLP 2024] Studying and Mitigating Biases in Sign Language Understanding Models. [paper]
    Katherine Atwell, Danielle Bragg, Malihe Alikhani
  • [ECCV 2024] Uncertainty-aware sign language video retrieval with probability distribution modeling. [paper]
    Xuan Wu, Hongxiang Li, yuanjiang luo, Xuxin Cheng, Xianwei Zhuang, Meng Cao, Keren Fu
  • [ECCV 2024] SignAvatars: A Large-scale 3D Sign Language Holistic Motion Dataset and Benchmark. [paper]
    Zhengdi Yu, Shaoli Huang, yongkang cheng, Tolga Birdal
  • [ACL 2024] Semi-Supervised Spoken Language Glossification. [paper]
    Huijie Yao, Wengang Zhou, Hao Zhou, Houqiang Li
  • [ACL 2024] American Sign Language Handshapes Reflect Pressures for Communicative Efficiency. [paper]
    Kayo Yin, Terry Regier, Dan Klein
  • [EMNLP Findings 2023] Enhancing Accessible Communication: from European Portuguese to Portuguese Sign Language. [paper]
    Catarina Sousa, Luisa Coheur, Mara Moita
  • [CVPR 2023] CiCo: Domain-Aware Sign Language Retrieval via Cross-Lingual Contrastive Learning. [paper][code]
    Yiting Cheng, Fangyun Wei, Jianmin Bao, Dong Chen, Wenqiang Zhang
  • [CVPR 2023] Ham2Pose: Animating Sign Language Notation into Pose Sequences. [paper]
    Rotem Shalev Arkushin, Amit Moryossef, Ohad Fried

Co-speech Gesture Generation

  • [ICLR 2025 Submission] Realistic-Gesture: Co-Speech Gesture Video Generation through Semantic-aware Gesture Representation. [paper]
    Anonymous authors
  • [ICLR 2025 Submission] Co3Gesture: Towards Coherent Concurrent Co-speech 3D Gesture Generation with Interactive Diffusion. [paper]
    Anonymous authors
  • [ICLR 2025 Submission] TANGO: Co-Speech Gesture Video Reenactment with Hierarchical Audio Motion Embedding and Diffusion Interpolation. [paper]
    Anonymous authors
  • [ECCV 2024] Co-speech Gesture Video Generation with 3D Human Meshes Gestures. [paper]
    Aniruddha Mahapatra, Richa Mishra, Ziyi Chen, Boyang Ding, Renda Li, Shoulei Wang, Jun-Yan Zhu, Peng Chang, Mei Han, Jing Xiao
  • [ACL 2024] LLM Knows Body Language, Too: Translating Speech Voices into Human Gestures. [paper]
    Chenghao Xu, Guangtao Lyu, Jiexi Yan, Muli Yang, Cheng Deng
  • [CVPR 2024] EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modeling. [paper]
    Haiyang Liu, Zihao Zhu, Giorgio Becherini, YICHEN PENG, Mingyang Su, YOU ZHOU, Xuefei Zhe, Naoya Iwamoto, Bo Zheng, Michael J. Black
  • [CVPR 2024] Weakly-Supervised Emotion Transition Learning for Diverse 3D Co-speech Gesture Generation. [paper]
    Xingqun Qi, Jiahao Pan, Peng Li, Ruibin Yuan, Xiaowei Chi, Mengfei Li, Wenhan Luo, Wei Xue, Shanghang Zhang, Qifeng Liu, Yike Guo
  • [CVPR 2024] Towards Variable and Coordinated Holistic Co-Speech Motion Generation. [paper]
    Yifei Liu, Qiong Cao, Yandong Wen, Huaiguang Jiang, Changxing Ding
  • [CVPR 2024] ConvoFusion: Multi-Modal Conversational Diffusion for Co-Speech Gesture Synthesis. [paper]
    Muhammad Hamza Mughal, Rishabh Dabral, Ikhsanul Habibie, Lucia Donatelli, Marc Habermann, Christian Theobalt
  • [CVPR 2024] Co-Speech Gesture Video Generation via Motion-Decoupled Diffusion Model. [paper]
    Xu He, Qiaochu Huang, Zhensong Zhang, Zhiwei Lin, Zhiyong Wu, Sicheng Yang, Minglei Li, Zhiyi Chen, Songcen Xu, Xiaofei Wu
  • [ICCV 2023] Continual Learning for Personalized Co-speech Gesture Generation. [paper]
    Chaitanya Ahuja, Pratik Joshi, Ryo Ishii, Louis-Philippe Morency
  • [ICCV 2023] LivelySpeaker: Towards Semantic-Aware Co-Speech Gesture Generation. [paper]
    Yihao Zhi, Xiaodong Cun, Xuelin Chen, Xi Shen, Wen Guo, Shaoli Huang, Shenghua Gao
  • [CVPR 2023] QPGesture: Quantization-Based and Phase-Guided Motion Matching for Natural Speech-Driven Gesture Generation. [paper]
    SiCheng Yang, Zhiyong Wu, Minglei Li, Zhensong Zhang, Lei Hao, Weihong Bao, Haolin Zhuang
  • [CVPR 2023] Taming Diffusion Models for Audio-Driven Co-Speech Gesture Generation. [paper]
    Lingting Zhu, Xian Liu, Xuanyu Liu, Rui Qian, Ziwei Liu, Lequan Yu
  • [CVPR 2023] Diverse 3D Hand Gesture Prediction from Body Dynamics by Bilateral Hand Disentanglement. [paper]
    Xingqun Qi, Chen Liu, Muyi Sun, Lincheng Li, Changjie Fan, Xin Yu
  • [CVPR 2023] Co-speech Gesture Synthesis by Reinforcement Learning with Contrastive Pre-trained Rewards. [paper]
    Mingyang Sun, Mengchen Zhao, Yaqing Hou, Minglei Li, Huang Xu, Songcen Xu, Jianye HAO
  • Complete List of Co-speech Gesture Generation

Gesture Recognition

  • [ICLR 2025 Submission] Revisiting Noise Resilience Strategies in Gesture Recognition: Short-Term Enhancement in Surface Electromyographic Signal Analysis. [paper]
    Anonymous authors
  • [ICCV 2023] Learning Robust Representations with Information Bottleneck and Memory Network for RGB-D-based Gesture Recognition. [paper]
    Yunan Li, Huizhou Chen, Guanwen Feng, Qiguang Miao
  • [ICCV 2023] Data-Free Class-Incremental Hand Gesture Recognition. [paper]
    Shubhra Aich, Jesus Ruiz-Santaquiteria, Zhenyu Lu, Prachi Garg, K J Joseph, Alvaro Fernandez Garcia, Vineeth N Balasubramanian, Kenrick Kin, Chengde Wan, Necati Cihan Camgoz, Shugao Ma, Fernando De la Torre
  • [BMVC 2022] Towards more efficient few-shot learning based human gesture recognition via dynamic vision sensors. [paper]
    Linglin Jing, Yifan Wang, Tailin Chen, Shirin Dora, Zhigang Ji, Hui Fang
  • [BMVC 2022] Continuous Hand Gesture Recognition using Deep Coarse and Fine Hand Features. [paper]
    Hazem Wannous, Jean-Philippe Vandeborre
  • [CVPR 2022] LD-ConGR: A Large RGB-D Video Dataset for Long-Distance Continuous Gesture Recognition. [paper]
    Dan Liu; Libo Zhang; Yanjun Wu
  • [AAAI 2022] Learning Unseen Emotions from Gestures via Semantically-Conditioned Zero-Shot Perception with Adversarial Autoencoders. [paper]
    Abhishek Banerjee, Uttaran Bhattacharya, Aniket Bera
  • Complete List of Gesture Recognition

Sign Language Workshop

Dataset

Dataset Task Modality Statistic
Name Language Isolated Continuous RGB Depth #Signers #Vocab #Sequences
DGS Kinect DGS √ √ √ 15 40 3,000
ASLLVD ASL √ √ 6 2,742 9,794
DEVISIGN-L CSL √ √ √ 8 2,000 24,000
CSL-500 CSL √ √ 50 500 125,000
MSASL ASL √ √ 222 1,000 25,513
WLASL2000 ASL √ √ 119 2,000 21,083
AUTSL TSL √ √ √ 43 226 38,336
SIGNUM DGS √ √ √ 9 455 2,340
Phoenix14 DGS √ √ 9 1,200 6,841
KETI KSL √ √ 14 524 15,672
GSL GSL √ √ √ 7 310 10,290
CSL-100 CSL √ √ 50 2,000 25,000
CSL-Daily CSL √ √ 10 2,000 20,654
BSL-1K BSL √ √ 40 1,064 273K
VRT-NEWS VGT √ √ 9 6,325 7,174
How2Sign BSL √ √ √ 11 15,686 35,191
BOBSL BSL √ √ 39 2,281 452K
OpenASL ASL √ √ 220 33,549 98,417
YouTube-ASL ASL √ √ >2519 60K 11,093