ChatEMG: Synthetic Data Generation to Control a Robotic Hand Orthosis for Stroke

Jingxi Xu$^{* , 1}$, Runsheng Wang$^{* , 1}$, Siqi Shang$^{* ,1}$, Ava Chen$^1$, Lauren Winterbottom$^2$, To-Liang Hsu$^1$, Wenxi Chen$^1$, Khondoker Ahmed$^1$, Pedro Leandro La Rotta$^1$, Xinyue Zhu$^1$, Dawn M. Nilsen$^2$, Joel Stein$^2$, Matei Ciocarlie $^1$

$^*$ Equal contribution, $^1$ Columbia University, $^2$ Columbia University Irving Medical Center

Project Page | Arxiv | Video

ChatEMG is an autoregressive generative model that can generate synthetic EMG signals conditioned on prompts (i.e., a given sequence of EMG signals). ChatEMG enables us to collect only a small dataset from the new condition, session, or subject and expand it with synthetic samples conditioned on prompts from this new context.

This repository contains code for model training and synthetic EMG data generation for ChatEMG.

If you find this codebase useful, consider citing:

@article{xu2024chatemg,
  title={ChatEMG: Synthetic Data Generation to Control a Robotic Hand Orthosis for Stroke},
  author={Xu, Jingxi and Wang, Runsheng and Shang, Siqi and Chen, Ava and Winterbottom, Lauren and Hsu, To-Liang and Chen, Wenxi and Ahmed, Khondoker and La Rotta, Pedro Leandro and Zhu, Xinyue and others},
  journal={arXiv preprint arXiv:2406.12123},
  year={2024}
}

If you have any questions, please contact Jingxi at jxu [at] cs [dot] columbia [dot] edu.

Table of Contents

• ⚙️ Setup
• 🚶 Codebase Walkthrough
  • 💾 EMG Data
  • ✌️ Two Branches
  • 🔄 Channel Rotation
• 🔬 Reproducing
  • 🧠 Training
  • 📊 Evaluation

Acknowledgements

This work was supported in part by the National Institutes of Health (R01NS115652, F31HD111301) and the CU Data Science Institute.

Code

• NanoGPT: The codebase is based on the NanoGPT repository.