Skip to content

fix(engine): stop passing -avoid_negative_ts make_zero in muxVideoWithAudio - #3488

Open
mincua wants to merge 1 commit into
heygen-com:mainfrom
mincua:fix/mux-avoid-negative-ts
Open

fix(engine): stop passing -avoid_negative_ts make_zero in muxVideoWithAudio#3488
mincua wants to merge 1 commit into
heygen-com:mainfrom
mincua:fix/mux-avoid-negative-ts

Conversation

@mincua

@mincua mincua commented Aug 25, 2026

Copy link
Copy Markdown

Fixes #3487

What

muxVideoWithAudio no longer passes -avoid_negative_ts make_zero — in any mode, for any container. One flag removed from the mux path; the preserveAudioPrimingEditList option that used to gate it is deprecated and ignored rather than deleted.

Why

The flag is actively harmful on the dominant mux path (an AAC sidecar copied into mp4), and it is not needed on any of the others.

The defect. ffmpeg's default is -avoid_negative_ts auto, i.e. "enabled when required by the target format". The mp4/mov muxers set AVFMT_TS_NEGATIVE (libavformat/movenc.c), so auto already resolves to disabled for the containers this function writes — mp4 expresses an offset start natively through an edit list. Passing make_zero explicitly overrides that deliberate per-format opt-out. On the copy path it discards the AAC priming edit list the sidecar encode created, shifts the copied video forward by one AAC frame (~21 ms), and makes the muxer write an empty video edit at t=0 (media time: -1). Players and thumbnailers that honor edit lists — QuickTime, Safari, WhatsApp previews, requestVideoFrameCallback — find no video sample at t=0 and render a black first frame.

The regression chain. The flag was added in v0.4.45 as belt-and-suspenders against negative DTS, a problem the same commit had already fixed at the source with -bf 0 on the encoder. #1615 (v0.6.116) diagnosed the black-first-frame symptom and fixed it by dropping a second AAC encode. Then:

  1. v0.7.72 narrowed the skip to config?.preserveAudioPrimingEditList === true, so the copy path — dominant, because the mixer already pads every input to the exact composition duration — ran make_zero again. Harmless at the time: the mixer wrote raw ADTS audio.aac, which has no edit list to destroy.
  2. v0.7.107 (fix(producer): mix audio into a container that can record encoder delay #3200) changed the mixer artifact from audio.aac to audio.m4a, precisely so the container would carry the priming edit list — but the v0.7.72 gate was not revisited. The callers still passed preserveAudioPrimingEditList: operation !== "copy", so make_zero ran on virtually every render and destroyed the very edit list fix(producer): mix audio into a container that can record encoder delay #3200 had introduced.

Why omitting the flag is correct rather than merely convenient. For mp4/mov, omitting it is the flag's own documented behavior — auto → disabled — so this restores ffmpeg's per-format default instead of imposing a new policy. Genuine leading negative DTS is still handled: the priming interval is carried by the audio track's edit list, and the encoder keeps -bf 0, which removes negative DTS at the source. make_zero also isn't idempotent — every re-encode pass through it stacks another priming interval (our delivery upscale turned 21 ms into 42 ms, a full frame at 24 fps). And the scenario it was guarding against does not materialize: muxing B-frame video with real leading negative DTS into mp4 comes out correct with the default and broken with make_zero.

How

  • packages/engine/src/services/chunkEncoder.ts — drop the copiesContainerizedAac condition and the args.push("-avoid_negative_ts", "make_zero") it guarded. Replaced by a comment recording why the flag must not be reintroduced here, so the next person reading the file doesn't re-derive it as a missing safety net.
  • MuxVideoWithAudioOptions.preserveAudioPrimingEditList is part of the exported engine API, so it stays, marked @deprecated and documented as a no-op, rather than being removed. No breaking change for external callers; removal can happen in a future major.
  • packages/producer/.../render/stages/assembleStage.ts and packages/producer/.../distributed/assemble.ts — the two internal callers stop passing the option. In distributed/assemble.ts the normalizedAudio object existed only to carry that boolean alongside the path, so it collapses back to a plain normalizedAudioPath: string | null.
  • Tests: the case that pinned the buggy behavior ("keeps negative-timestamp repair for an M4A without a known priming edit list", asserting the flag is passed) is inverted into "never repairs negative timestamps for an M4A sidecar (regression #3487)". A not.toContain("-avoid_negative_ts") assertion is added to the re-encode case and to the per-container loop, so the flag can't come back for any container without a test failing.

Deliberately left alone: buildEncoderArgs and streamingEncoder still pass -avoid_negative_ts make_zero for video-only output. Those chunks are intermediates consumed by the concat/mux stages, not delivered mp4/mov, they carry no audio priming edit list to destroy, and the flag's original negative-DTS rationale still applies there. Changing them is out of scope for this fix and would need its own evidence.

Test plan

1. Caller-shaped ffmpeg repro — a 30 fps h264 mp4 plus a 48 kHz AAC .m4a sidecar, muxed with exactly the argv muxVideoWithAudio builds, with and without the flag. ffprobe:

with make_zero (before) without (this PR)
video start_time 0.021029 0.000000
video elst media time: -1, dur 323 + media time: 0, dur 30720 media time: 0, dur 30720
audio elst media time: 0 (priming trim destroyed) media time: 1024 (priming preserved)
audio duration 2.021333 2.000000

The empty leading video edit and the offset both disappear, and the audio keeps its 1024-sample priming edit.

2. Minimal repro from the issue (24 fps, both 44.1 kHz and 48 kHz) — start_time 0.023031 / 0.020996 with the flag, 0.000000 without; the media time: -1 empty edit is present in each broken case and absent in each fixed one.

3. Full render through the fixed pipeline — a real 24 fps mp4 with audio out of the producer: video start_time=0.000000, audio start_time=0.000000, one edit per track (media time: 0), no empty edit.

4. Unit + static checks (ffmpeg 8.1.2, bun 1.3.14, macOS):

bun run --filter @hyperframes/engine test          68 files passed, 1 skipped — 1638 passed, 3 skipped
  └ src/services/chunkEncoder.test.ts              85 passed
bun run --filter @hyperframes/producer test:unit   40 files — 598 passed, 0 failed
bun run --filter @hyperframes/engine typecheck     exit 0
bun run --filter @hyperframes/producer typecheck   exit 0
bun run lint                                       0 warnings, 0 errors (2930 files)
bun run format:check                               all matched files correctly formatted
  • Unit tests added/updated
  • Manual testing performed
  • Documentation updated (if applicable) — no docs page covers this flag; the deprecation note lives in the JSDoc on MuxVideoWithAudioOptions.

Investigated, written, and tested with Claude Code (Claude Fable 5).

`muxVideoWithAudio` passed `-avoid_negative_ts make_zero` unless the caller
set `preserveAudioPrimingEditList`. In practice the dominant path is an AAC
sidecar copied into mp4, where that flag is actively harmful: ffmpeg's
default is `auto`, which the mp4/mov muxers (AVFMT_TS_NEGATIVE) already
resolve to `disabled`. Forcing `make_zero` overrides the correct default,
discards the priming edit list the sidecar encode created, shifts the video
start_time forward by one AAC frame and writes an empty video edit at t=0 —
which edit-list-honoring players (QuickTime/Safari) render as a black first
frame.

Verified with ffprobe on a copy mux of a 30fps h264 mp4 and an AAC sidecar:

  with `make_zero`   video start_time 0.066000, elst: [media time -1,
                     dur 5940] + [media time 6000, dur 180000]
                     audio start_time 0.042993, elst: [media time -1, ...]
  without (this fix) video start_time 0.000000, elst: [media time 6000,
                     dur 180000]
                     audio start_time 0.000000, elst: [media time 1024, ...]

The empty leading edit and the offset both disappear, and the audio keeps
its 1024-sample priming edit.

The flag is now never passed for a mux, in any mode. `preserveAudioPrimingEditList`
is part of the exported engine API, so it stays on `MuxVideoWithAudioOptions`
as `@deprecated` and no-op rather than being removed; the two internal callers
that set it (`assembleStage`, distributed `assemble`) drop it.

`buildEncoderArgs` and `streamingEncoder` still pass the flag for video-only
output and are deliberately left alone — those chunks are consumed as
intermediates, not as a delivered mp4/mov.

Fixes heygen-com#3487

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

-avoid_negative_ts make_zero in muxVideoWithAudio shifts video +21ms and re-introduces the black first frame from #1615 (regression since v0.7.107)

1 participant