Skip to content

-avoid_negative_ts make_zero in muxVideoWithAudio shifts video +21ms and re-introduces the black first frame from #1615 (regression since v0.7.107) #3487

Description

@mincua

Summary

Since v0.7.107, MP4 renders with audio take the -avoid_negative_ts make_zero branch in muxVideoWithAudio again on the common path (audio needs no pad/trim → operation === "copy"). The flag destroys the AAC priming edit list on the audio track, shifts the entire video timeline forward by one AAC frame (~21 ms), and writes an empty edit (media_time = -1) on the video track. Players and thumbnailers that honor edit lists (QuickTime, Safari, WhatsApp previews, requestVideoFrameCallback) find no video sample at t=0 and show a black first frame — the exact symptom #1615 fixed in v0.6.116.

video  start_time=0.020996   <- should be 0.000000
audio  start_time=0.000000
elst (video): media_time=-1, then media_time=0   <- empty edit: nothing presented at t=0
elst (audio): media_time=0                        <- priming trim destroyed

How the regression happened

  1. 0e541673e (fix: render robustness — sub-comp src paths, alpha tag case, encoder + matter improvements #627, shipped v0.4.45) added -avoid_negative_ts make_zero to the mux path — described in the commit itself as "belt-and-suspenders" against negative DTS sneaking back in. The concrete bug reported there (negative DTS from B-frames freezing some players) was fixed in the same commit at the source, with -bf 0 on the encoder.
  2. fix(engine): preserve AAC start time during MP4 mux #1615 (v0.6.116, title: "fix(engine): preserve AAC start time during MP4 mux") diagnosed the black-first-frame symptom (video start_time=0.021029, empty video edit / media time: -1) and fixed it by removing the second AAC encode.
  3. 59c56d325 (fix(render): trim AAC packet padding exactly #2531) added the comment that is still in chunkEncoder.ts today: "make_zero discards that edit and shifts copied video forward by one AAC frame (~21ms), creating a visible first-frame offset."
  4. 9289551e9 (v0.7.72, subject: "scope M4A priming preservation to trims") narrowed the skip to config?.preserveAudioPrimingEditList === true. Callers originally set that only for operation === "trim"; a same-release follow-up (4b116b988) broadened them to operation !== "copy" so pad also preserves. Either way, the copy path — which is dominant because the mixer already pads every input to the exact composition duration — still runs make_zero. Safe at the time: the mixer wrote raw ADTS audio.aac, which has no edit list to destroy.
  5. fix(producer): mix audio into a container that can record encoder delay #3200 / dc4383113 (v0.7.107) changed the mixer artifact from audio.aac to audio.m4a, precisely so the container would carry the priming edit list — but the v0.7.72 condition was not revisited. Callers (assembleStage.ts, distributed/assemble.ts) still set preserveAudioPrimingEditList: normalizeResult.operation !== "copy", so make_zero runs on virtually every render and discards the very edit list fix(producer): mix audio into a container that can record encoder delay #3200 introduced.

A unit test currently pins the buggy behavior (the "keeps negative-timestamp repair for an M4A without a known priming edit list" case in packages/engine/src/services/chunkEncoder.test.ts asserts the flag IS passed).

Why make_zero is never needed for MP4

ffmpeg's default -avoid_negative_ts auto means "enabled when required by the target format" (libavformat/mux.c). The mp4/mov muxers set AVFMT_TS_NEGATIVE (libavformat/movenc.c), so auto resolves to disabled: MP4 expresses offset starts natively via edit lists. Passing make_zero overrides that deliberate per-format opt-out. (mpegts does not set the flag — that, along with -ss -c copy cuts and concat-demuxer prep, is where make_zero belongs.)

It is also not idempotent: every re-encode + make_zero pass stacks another priming interval. Our delivery upscale turned 21 ms into 42 ms — a full frame at 24 fps.

And the scenario the flag guards against doesn't materialize: muxing B-frame video with genuine leading negative DTS into mp4 comes out correct with the default (start_time 0.000, clean edit list) and broken with make_zero (both tracks shifted off zero plus the empty edit). Related ffmpeg-side report of this bug class: https://trac.ffmpeg.org/ticket/10380

Minimal reproduction

(ffmpeg 8.1.2)

# video-only mp4, no B-frames (like the hyperframes encoder output)
ffmpeg -f lavfi -i testsrc2=duration=2:rate=24 -c:v libx264 -bf 0 -pix_fmt yuv420p video.mp4
# AAC in m4a at 48 kHz — carries a priming edit list, like the mixer artifact since v0.7.107
ffmpeg -f lavfi -i sine=frequency=440:duration=2:sample_rate=48000 -c:a aac -ar 48000 audio.m4a

# the copy path, as muxVideoWithAudio builds it today
ffmpeg -i video.mp4 -i audio.m4a -c:v copy -c:a copy -movflags +faststart -avoid_negative_ts make_zero broken.mp4
# identical, flag omitted (ffmpeg default `auto` = disabled for mp4)
ffmpeg -i video.mp4 -i audio.m4a -c:v copy -c:a copy -movflags +faststart ok.mp4

ffprobe -v error -select_streams v:0 -show_entries stream=start_time -of default=nw=1 broken.mp4   # start_time=0.020996
ffprobe -v error -select_streams v:0 -show_entries stream=start_time -of default=nw=1 ok.mp4       # start_time=0.000000

(ffprobe -v trace broken.mp4 also shows video media time: -1 then media time: 0, and audio media time: 0 — priming edit destroyed. The source audio.m4a itself has media time: 1024.)

Suggested fix

Drop -avoid_negative_ts from muxVideoWithAudio for mp4/mov outputs entirely (keep -bf 0 at the encoder, which addresses negative DTS at its source) — or at minimum invert the gate's polarity: skip the flag by default and apply it only for output formats that actually require timestamp shifting. Happy to open a PR if that helps.

Impact

Every mp4 render with audio on the no-pad/trim path ships with the offset (v0.4.45–v0.6.115, and again since v0.7.107). In our production corpus (rendered on 0.6.13), 121 of 121 finals probe at video start_time=0.021 and all of them show a black poster frame when shared to WhatsApp.


Investigated and written with Claude Code (Claude Fable 5); the reproduction and the ffmpeg-source claims were verified by hand before filing.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions