Skip to content

Low-latency model loses yaw responsiveness in Planner VR3PT mode #231

Description

@LEECHOONGHO

Hellow,

We're seeing a difference in yaw turning behavior between the standard model
and the low-latency SONIC model:

  • Standard model: yaw works well in both Pose and Planner, VR3PT mode
  • Low-latency model: yaw works well in Other mode, but barely responds in
    Planner VR3PT mode — only exaggerated direction inputs** produce visible turning

** facing = self.yaw_accumulator.update(rx * 3, self.dt)

Looking at GetObservationRegistry() in g1_deploy_onnx_ref.cpp, the anchor
orientation observation has three modes: full base quaternion (mode 0),
heading-only/yaw-isolated (mode 1), and refheading (mode 2). The 10-frame and
2-frame variants register all three, but the 4-frame (low-latency) variant
only registers mode 0:

{"smpl_anchor_orientation_4frame_step1", ...}  // no heading/refheading version

So the low-latency model can only observe the full quaternion (roll+pitch+yaw
combined), not a yaw-isolated signal. Since Planner mode's planner_motion_
carries more roll/pitch noise than a fixed Pose-mode reference, this may be
diluting the yaw signal enough to cause the weak turning response.

Could you confirm whether this is intentional, and whether adding
heading/refheading variants for the 4-frame observation set would need
retraining, or could be tested against the existing checkpoint?

Happy to share logs or a repro video if helpful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions