Summary
AudioTimelineMixer stores a session's audio as a dense array where the index carries the time — sample 16,000 is second 1. That is what keeps two capture sources in step. The cost is that when no source is writing, the gap still has to exist in the array for later indices to mean what they say, so it is allocated and zeroed the moment recording resumes.
For ordinary mutes this is a few megabytes and invisible. For a long mute in an in-person recording it is hundreds of megabytes of silence, memset on the main actor, charged against the memory budget meant for audio, and then fed through both models.
Follow-up from #48, which introduced the time-addressed timeline. Not a regression against main — the old buffer only ever held real audio because it was append-ordered, which is the bug #48 fixed.
Where it happens
The gap is materialised in Logue/Engine/AudioTimelineMixer.swift:147-157, when a source resumes at a cursor set by beginSource — called on unmute at RecordingSessionManager.swift:644. DiarizationManager is @MainActor (DiarizationManager.swift:12-13), so the allocation lands on the main actor.
Cost
| mute |
zeroed samples |
memory |
| 30 s |
0.5 M |
2 MB |
| 5 min |
4.8 M |
18 MB |
| 20 min |
19.2 M |
73 MB |
| 1 h |
57.6 M |
220 MB |
| 2 h |
115.2 M |
439 MB |
Silence counts against capacity, so it displaces real audio. On an 8 GB Mac (134 M samples ≈ 2.33 h):
| mute |
real audio still accepted afterwards |
| 20 min |
~120 min |
| 1 h |
~80 min |
| 2 h |
~20 min |
When free memory is low the budget falls to the audioBufferMinBytes floor (AppConstants.swift:151) of 256 MB ≈ 1.16 h, and a 2-hour mute exhausts it outright — every sample after the break is dropped, and didDropAudio sends the session down the disk route (AudioTimelineMixer.swift:129).
When it is reachable
Only when no source is writing at all — in practice an in-person recording muted for a long stretch. During an online meeting the system tap keeps writing, so no gap forms.
Gaps over a minute are logged today (AudioTimelineMixer.swift:153), so the cost is visible rather than silent, but it is not avoided.
Suggested approach
The activations are already known: CaptureSegmentTimeline.Placement (CaptureSegmentTimeline.swift:16) records where each contiguous run belongs on the meeting. Feeding the models each run separately and offsetting its results by that run's sessionStart would need no silence in memory and none through the models, while keeping timestamps exact.
Two things to establish before committing to it:
- Sortformer carries speaker identity in state across
process() calls. Feeding runs back-to-back should preserve it — arguably better than making it listen to hours of digital silence — but that needs checking against real audio, not assuming.
- Its output would then be in gap-free time and would need mapping back through the placements.
Acceptance criteria
Summary
AudioTimelineMixerstores a session's audio as a dense array where the index carries the time — sample 16,000 is second 1. That is what keeps two capture sources in step. The cost is that when no source is writing, the gap still has to exist in the array for later indices to mean what they say, so it is allocated and zeroed the moment recording resumes.For ordinary mutes this is a few megabytes and invisible. For a long mute in an in-person recording it is hundreds of megabytes of silence, memset on the main actor, charged against the memory budget meant for audio, and then fed through both models.
Follow-up from #48, which introduced the time-addressed timeline. Not a regression against
main— the old buffer only ever held real audio because it was append-ordered, which is the bug #48 fixed.Where it happens
The gap is materialised in Logue/Engine/AudioTimelineMixer.swift:147-157, when a source resumes at a cursor set by
beginSource— called on unmute at RecordingSessionManager.swift:644.DiarizationManageris@MainActor(DiarizationManager.swift:12-13), so the allocation lands on the main actor.Cost
Silence counts against capacity, so it displaces real audio. On an 8 GB Mac (134 M samples ≈ 2.33 h):
When free memory is low the budget falls to the
audioBufferMinBytesfloor (AppConstants.swift:151) of 256 MB ≈ 1.16 h, and a 2-hour mute exhausts it outright — every sample after the break is dropped, anddidDropAudiosends the session down the disk route (AudioTimelineMixer.swift:129).When it is reachable
Only when no source is writing at all — in practice an in-person recording muted for a long stretch. During an online meeting the system tap keeps writing, so no gap forms.
Gaps over a minute are logged today (AudioTimelineMixer.swift:153), so the cost is visible rather than silent, but it is not avoided.
Suggested approach
The activations are already known:
CaptureSegmentTimeline.Placement(CaptureSegmentTimeline.swift:16) records where each contiguous run belongs on the meeting. Feeding the models each run separately and offsetting its results by that run'ssessionStartwould need no silence in memory and none through the models, while keeping timestamps exact.Two things to establish before committing to it:
process()calls. Feeding runs back-to-back should preserve it — arguably better than making it listen to hours of digital silence — but that needs checking against real audio, not assuming.Acceptance criteria