Skip to content

Fine-tuned motions cannot be replayed: no motion_lib to deploy reference export step #241

Description

@Harshita-82

Summary

There is no documented path from a training motion_lib motion to the deploy
reference format, so a motion a policy was fine tuned on cannot be replayed in
sim2sim or on hardware.

1. No export step from motion_lib to deploy reference format

gear_sonic_deploy/reference/convert_motions.py consumes a PKL keyed by:

joint_pos       (T, 29)      joint_vel       (T, 29)
body_pos_w      (T, 14, 3)   body_quat_w     (T, 14, 4)
body_lin_vel_w  (T, 14, 3)   body_ang_vel_w  (T, 14, 3)

The PKLs produced by the documented data pipeline
(gear_sonic/data_process/convert_soma_csv_to_motion_lib.py) instead contain
root_trans_offset, pose_aa, dof, root_rot, smpl_joints, fps.
There is no key overlap, and no shipped script bridges them.

To be precise about where the gap is: MotionLibBase already computes every
array the deploy format needs.
fk_batch resamples to target_fps via
interploate_pose and derives velocities, and motion_lib_base.py (~L1599-1619)
converts quaternions xyzw -> wxyz and slices to the selected body subset at load
time. The arrays exist in memory during training and are used for tracking
rewards. The only missing piece is a step that writes them to disk.

deploy.sh defaults to MOTION_DATA_DEFAULT="reference/example/", so when no
converted motion exists the stack loads the bundled examples and emits no
warning:

Motion Data:  reference/example/
Found 13 motion folders
✓ Loaded dance_in_da_party_001__A464_M (497 timesteps)
✓ Loaded macarena_001__A545 (1375 timesteps)
...

Proposed fix

PR #239 adds gear_sonic/data_process/export_motion_to_deploy.py, which loads a
motion through MotionLibRobot, reads the arrays the library already computes,
and reuses convert_motions.py for CSV writing. It deliberately does not
reimplement forward kinematics, resampling or differentiation.

Verified against MuJoCo's independent forward kinematics across 7 motions
(~3900 frames): exported body poses agree to 2.3e-06 m and 2.1e-04 deg,
which is the float32 / 6-decimal-CSV noise floor. The exported motions were also
replayed end-to-end in sim2sim with a fine-tuned checkpoint and track correctly.

If there is an intended path for this that I have missed, I am happy to close
this out — I could not find one in the docs or scripts.

Environment

Repo        main @ aa263a8
OS          Ubuntu 24.04.4, kernel 6.17.0-35
GPU         RTX PRO 6000 Blackwell Server Edition, driver 595.58.03
CUDA        12.6
Isaac Sim   5.1.0.0
Isaac Lab   2.3.0
Python      3.11.15
PyTorch     2.7.0+cu128
MuJoCo      3.11.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions