Dreamio turn document
Audio Warm Resume
Dreamio now keeps the iOS playback audio session warm while native VLC playback is paused and explicitly primes it before resuming, so video no longer races ahead while audio wakes up.
Summary
Dreamio now keeps the iOS playback audio session warm while native VLC playback is paused and explicitly primes it before resuming, so video no longer races ahead while audio wakes up.
Changes Made
- Added a native audio-session policy for deciding when playback should prime the audio route before starting.
- Prepared
AVAudioSessionwith.playbackand.moviePlaybackbefore initial VLC playback and before resuming from inactive states. - Kept the audio session active immediately after pause, including a short follow-up activation, so the route is less likely to go cold before the next play tap.
- Added lightweight policy coverage to the existing Swift test harness.
Context
Pausing and resuming a native video could let the video frame clock restart immediately while the audio route took a moment to become audible. Earlier low-latency and audio-delay attempts did not address the route warm-up itself, so this change targets the iOS audio session around pause/resume.
Important Implementation Details
VLCNativePlaybackBackend.play(request:)primes the audio session before assigning and starting new media.VLCNativePlaybackBackend.play()only primes on paused, stopped, ended, or error states, avoiding repeated session churn while already opening, buffering, or playing.pause()no longer leaves the route entirely to VLC after the pause command; it explicitly keeps the app audio session warm and repeats once after 80 ms.- Debug logs include the audio-session preparation reason to help confirm whether a resume path warmed the session before VLC starts rendering again.
Relevant Diff Snippets
Dreamio/NativePlaybackBackend.swift ยท adds a testable audio-session resume policy
62 unmodified lines636465666768697062 unmodified lines}}}enum NativePlaybackError: LocalizedError {case backendUnavailablecase startupTimedOutcase playbackFailed62 unmodified lines6364656667686970717273747576777879808162 unmodified lines}}}enum NativePlaybackAudioSessionPolicy {static func shouldPrepareBeforePlayback(from state: NativePlaybackToggleState) -> Bool {switch state {case .paused, .stopped, .ended, .error:return truecase .opening, .buffering, .playing, .unknown:return false}}}enum NativePlaybackError: LocalizedError {case backendUnavailablecase startupTimedOutcase playbackFailed
Dreamio/VLCNativePlaybackBackend.swift ยท prepares and keeps AVAudioSession active around pause/resume
123467 unmodified lines727374757677787910 unmodified lines909192939495969715 unmodified lines113114115116117118119120220 unmodified lines341342343344345346347348import UIKit#if canImport(MobileVLCKit)import MobileVLCKit67 unmodified linesif !headerValue.isEmpty {media.addOption(":http-header=\(headerValue)")}addConservativePlaybackOptions(to: media)mediaPlayer.currentAudioPlaybackDelay = 0mediaPlayer.media = media#if DEBUG10 unmodified lines}func play() {#if canImport(MobileVLCKit)#if DEBUGlogPlaybackSnapshot(reason: "before-play")#endifmediaPlayer.play()15 unmodified lines#endifreturn}mediaPlayer.pause()#if DEBUGlogPlaybackSnapshot(reason: "after-pause-command")DispatchQueue.main.asyncAfter(deadline: .now() + 0.25) { [weak self] inself?.logPlaybackSnapshot(reason: "pause-follow-up-250ms")220 unmodified lines":clock-jitter=0"].forEach { media.addOption($0) }}private func isPauseableState(_ state: VLCMediaPlayerState) -> Bool {switch state {case .playing, .buffering:return true1234567 unmodified lines73747576777879808110 unmodified lines929394959697989910010110210315 unmodified lines119120121122123124125126127220 unmodified lines348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377import AVFoundationimport UIKit#if canImport(MobileVLCKit)import MobileVLCKit67 unmodified linesif !headerValue.isEmpty {media.addOption(":http-header=\(headerValue)")}addConservativePlaybackOptions(to: media)prepareAudioSessionForPlayback(reason: "initial-play")mediaPlayer.currentAudioPlaybackDelay = 0mediaPlayer.media = media#if DEBUG10 unmodified lines}func play() {#if canImport(MobileVLCKit)let toggleState = playbackToggleState(for: mediaPlayer.state)if NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: toggleState) {prepareAudioSessionForPlayback(reason: "resume-from-\(toggleState)")}#if DEBUGlogPlaybackSnapshot(reason: "before-play")#endifmediaPlayer.play()15 unmodified lines#endifreturn}mediaPlayer.pause()keepAudioSessionWarmAfterPause()#if DEBUGlogPlaybackSnapshot(reason: "after-pause-command")DispatchQueue.main.asyncAfter(deadline: .now() + 0.25) { [weak self] inself?.logPlaybackSnapshot(reason: "pause-follow-up-250ms")220 unmodified lines":clock-jitter=0"].forEach { media.addOption($0) }}private func prepareAudioSessionForPlayback(reason: String) {do {let session = AVAudioSession.sharedInstance()try session.setCategory(.playback, mode: .moviePlayback, options: [])try session.setActive(true)#if DEBUGprint("[DreamioVLC] audio-session prepared reason=\(reason) category=\(session.category.rawValue) mode=\(session.mode.rawValue)")#endif} catch {#if DEBUGprint("[DreamioVLC] audio-session prepare failed reason=\(reason) error=\(error.localizedDescription)")#endif}}private func keepAudioSessionWarmAfterPause() {prepareAudioSessionForPlayback(reason: "pause-keep-warm")DispatchQueue.main.asyncAfter(deadline: .now() + 0.08) { [weak self] inself?.prepareAudioSessionForPlayback(reason: "pause-keep-warm-follow-up")}}private func isPauseableState(_ state: VLCMediaPlayerState) -> Bool {switch state {case .playing, .buffering:return true
Tests/StreamResolverTests.swift ยท covers the audio-session resume policy
23 unmodified lines2425262728293031483 unmodified lines51551651751851952052152223 unmodified linestestSubtitleDisplayNameNormalization()testSubtitleDisplayNameUsesPreservedNamesForGenericVLCTracks()testSubtitleOptionMappingIncludesNone()testNativePlaybackTogglePolicy()print("StreamResolverTests passed")}private static func testClassifierPrefersObservedDirectFile() {483 unmodified linesassertEqual(NativePlaybackTogglePolicy.action(for: .opening), .waitForTransition)assertEqual(NativePlaybackTogglePolicy.action(for: .unknown), .waitForTransition)}private static func assertEqual<T: Equatable>(_ actual: T?, _ expected: T, file: StaticString = #file, line: UInt = #line) {assert(actual == expected, "Expected \(String(describing: expected)), got \(String(describing: actual))", file: file, line: line)}23 unmodified lines242526272829303132483 unmodified lines51651751851952052152252352452552652752852953053153253353423 unmodified linestestSubtitleDisplayNameNormalization()testSubtitleDisplayNameUsesPreservedNamesForGenericVLCTracks()testSubtitleOptionMappingIncludesNone()testNativePlaybackTogglePolicy()testNativePlaybackAudioSessionPolicy()print("StreamResolverTests passed")}private static func testClassifierPrefersObservedDirectFile() {483 unmodified linesassertEqual(NativePlaybackTogglePolicy.action(for: .opening), .waitForTransition)assertEqual(NativePlaybackTogglePolicy.action(for: .unknown), .waitForTransition)}private static func testNativePlaybackAudioSessionPolicy() {assertEqual(NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: .paused), true)assertEqual(NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: .stopped), true)assertEqual(NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: .ended), true)assertEqual(NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: .error), true)assertEqual(NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: .playing), false)assertEqual(NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: .buffering), false)assertEqual(NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: .opening), false)assertEqual(NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: .unknown), false)}private static func assertEqual<T: Equatable>(_ actual: T?, _ expected: T, file: StaticString = #file, line: UInt = #line) {assert(actual == expected, "Expected \(String(describing: expected)), got \(String(describing: actual))", file: file, line: line)}
Diffs are generated with @pierre/diffs/ssr at documentation time. Each file diff stays inside its own shell so the page remains readable as a static HTML artifact.
Expected Impact for End-Users
After pausing and pressing play, audio should be ready before VLC restarts visible playback, reducing the perceived gap where the video moves but audio has not yet caught up.
New Changes as of 2026-05-27 00:46
Summary of changes
Follow-up work now treats the recurring resume lag as a VLC resume-clock issue rather than only an iOS audio-session warm-up issue. The resume path captures the paused timestamp, removes the aggressive :clock-jitter=0 streaming option, and briefly holds VLC at the paused media time if video advances before MobileVLCKit reports audio loudness.
Why this change was made
The lag happened on every resume, which means the previous audio-session-only mitigation was not sufficient. MobileVLCKit exposes time and loudness callbacks, and its headers note that loudness timestamps can be affected by audio output buffering. The new path uses those signals to prevent silent video advancement during resume.
Code diffs
Dreamio/NativePlaybackBackend.swift ยท adds testable resume-hold policy
74 unmodified lines75767778798074 unmodified lines}}enum NativePlaybackError: LocalizedError {case backendUnavailablecase startupTimedOut74 unmodified lines7576777879808182838485868788899091929394959697989910074 unmodified lines}}enum NativePlaybackResumePolicy {static let freezeInterval: TimeInterval = 0.08static let maximumFreezeDuration: TimeInterval = 1.2static let maximumAllowedSilentAdvance: Int32 = 120static func shouldHoldVideoAtPausedTime(elapsedSinceResume: TimeInterval,hasObservedAudioOutput: Bool,mediaAdvanceMilliseconds: Int32) -> Bool {guard !hasObservedAudioOutput else {return false}guard elapsedSinceResume < maximumFreezeDuration else {return false}return mediaAdvanceMilliseconds > maximumAllowedSilentAdvance}}enum NativePlaybackError: LocalizedError {case backendUnavailablecase startupTimedOut
Dreamio/VLCNativePlaybackBackend.swift ยท removes clock-jitter override and holds video until audio output is observed
32 unmodified lines33343536373822 unmodified lines61626364656627 unmodified lines9495969798991 unmodified line10110210310410510612 unmodified lines119120121122123124111 unmodified lines236237238239240241102 unmodified lines34434534634734834935035119 unmodified lines37137237337437537627 unmodified lines404405406407408409410156 unmodified lines56756856957057157232 unmodified linesprivate var pendingExternalSubtitleDisplayNames: [String] = []private var externalSubtitleDisplayNamesByTrackID: [Int32: String] = [:]private var didReportReadyForCurrentMedia = falseprivate var lastToggleDate = Date.distantPastprivate let minimumToggleInterval: TimeInterval = 0.3522 unmodified linespendingExternalSubtitleDisplayNames.removeAll()externalSubtitleDisplayNamesByTrackID.removeAll()didReportReadyForCurrentMedia = falselastToggleDate = .distantPastlet media = VLCMedia(url: request.playbackURL)let headerValue = request.headers27 unmodified linesfunc play() {#if canImport(MobileVLCKit)let toggleState = playbackToggleState(for: mediaPlayer.state)if NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: toggleState) {prepareAudioSessionForPlayback(reason: "resume-from-\(toggleState)")}1 unmodified linelogPlaybackSnapshot(reason: "before-play")#endifmediaPlayer.play()#if DEBUGlogPlaybackSnapshot(reason: "after-play-command")#endif12 unmodified lines#endifreturn}mediaPlayer.pause()keepAudioSessionWarmAfterPause()#if DEBUG111 unmodified lineslogPlaybackSnapshot(reason: "before-stop")#endifdidReportReadyForCurrentMedia = falsemediaPlayer.stop()mediaPlayer.drawable = nilmediaPlayer.media = nil102 unmodified lines[":network-caching=1000",":file-caching=1000",":live-caching=1000",":clock-jitter=0"].forEach { media.addOption($0) }}19 unmodified lines}}private func isPauseableState(_ state: VLCMediaPlayerState) -> Bool {switch state {case .playing, .buffering:27 unmodified lines#if DEBUGprivate func logPlaybackSnapshot(reason: String) {let mediaLength = mediaPlayer.media?.length.intValue ?? 0print("[DreamioVLC] snapshot reason=\(reason) state=\(stateName(mediaPlayer.state)) isPlaying=\(mediaPlayer.isPlaying) canPause=\(mediaPlayer.canPause) seekable=\(mediaPlayer.isSeekable) currentTime=\(String(format: "%.3f", currentTime)) duration=\(String(format: "%.3f", TimeInterval(max(0, mediaLength)) / 1000)) position=\(String(format: "%.4f", mediaPlayer.position)) audioDelay=\(mediaPlayer.currentAudioPlaybackDelay) readyReported=\(didReportReadyForCurrentMedia)")}#endif156 unmodified lines#if canImport(MobileVLCKit)extension VLCNativePlaybackBackend: VLCMediaPlayerDelegate {func mediaPlayerStateChanged(_ aNotification: Notification) {#if DEBUGlogPlaybackSnapshot(reason: "state-change")32 unmodified lines3334353637383940414222 unmodified lines6566676869707127 unmodified lines991001011021031041051 unmodified line10710810911011111211311411511611712 unmodified lines130131132133134135136137111 unmodified lines249250251252253254255102 unmodified lines35835936036136236336419 unmodified lines38438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146227 unmodified lines490491492493494495496156 unmodified lines65365465565665765865966066166266366466566666766866967032 unmodified linesprivate var pendingExternalSubtitleDisplayNames: [String] = []private var externalSubtitleDisplayNamesByTrackID: [Int32: String] = [:]private var didReportReadyForCurrentMedia = falseprivate var pausedTimeMilliseconds: Int32?private var resumeCorrectionGeneration = 0private var resumeCorrectionStartDate: Date?private var hasObservedResumeAudioOutput = falseprivate var lastToggleDate = Date.distantPastprivate let minimumToggleInterval: TimeInterval = 0.3522 unmodified linespendingExternalSubtitleDisplayNames.removeAll()externalSubtitleDisplayNamesByTrackID.removeAll()didReportReadyForCurrentMedia = falseresetResumeCorrectionState()lastToggleDate = .distantPastlet media = VLCMedia(url: request.playbackURL)let headerValue = request.headers27 unmodified linesfunc play() {#if canImport(MobileVLCKit)let toggleState = playbackToggleState(for: mediaPlayer.state)let isResumingFromPause = toggleState == .pausedif NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: toggleState) {prepareAudioSessionForPlayback(reason: "resume-from-\(toggleState)")}1 unmodified linelogPlaybackSnapshot(reason: "before-play")#endifmediaPlayer.play()if isResumingFromPause {beginResumeCorrection()} else {resetResumeCorrectionState()}#if DEBUGlogPlaybackSnapshot(reason: "after-play-command")#endif12 unmodified lines#endifreturn}pausedTimeMilliseconds = mediaPlayer.time.intValueresetResumeCorrectionRuntimeState()mediaPlayer.pause()keepAudioSessionWarmAfterPause()#if DEBUG111 unmodified lineslogPlaybackSnapshot(reason: "before-stop")#endifdidReportReadyForCurrentMedia = falseresetResumeCorrectionState()mediaPlayer.stop()mediaPlayer.drawable = nilmediaPlayer.media = nil102 unmodified lines[":network-caching=1000",":file-caching=1000",":live-caching=1000"].forEach { media.addOption($0) }}19 unmodified lines}}private func beginResumeCorrection() {guard let pausedTimeMilliseconds else {return}resetResumeCorrectionRuntimeState()resumeCorrectionGeneration += 1let generation = resumeCorrectionGenerationresumeCorrectionStartDate = Date()#if DEBUGprint("[DreamioVLC] resume-correction begin pausedTimeMS=\(pausedTimeMilliseconds)")#endifscheduleResumeCorrectionTick(generation: generation)}private func scheduleResumeCorrectionTick(generation: Int) {DispatchQueue.main.asyncAfter(deadline: .now() + NativePlaybackResumePolicy.freezeInterval) { [weak self] inself?.performResumeCorrectionTick(generation: generation)}}private func performResumeCorrectionTick(generation: Int) {guard generation == resumeCorrectionGeneration,let pausedTimeMilliseconds,let resumeCorrectionStartDate else {return}let elapsed = Date().timeIntervalSince(resumeCorrectionStartDate)let currentTimeMilliseconds = mediaPlayer.time.intValuelet advance = max(0, currentTimeMilliseconds - pausedTimeMilliseconds)let shouldHold = NativePlaybackResumePolicy.shouldHoldVideoAtPausedTime(elapsedSinceResume: elapsed,hasObservedAudioOutput: hasObservedResumeAudioOutput,mediaAdvanceMilliseconds: advance)guard shouldHold else {#if DEBUGprint("[DreamioVLC] resume-correction release elapsed=\(String(format: "%.3f", elapsed)) audioObserved=\(hasObservedResumeAudioOutput) advanceMS=\(advance)")#endifresetResumeCorrectionRuntimeState()return}mediaPlayer.time = VLCTime(int: pausedTimeMilliseconds)#if DEBUGprint("[DreamioVLC] resume-correction hold elapsed=\(String(format: "%.3f", elapsed)) advanceMS=\(advance) resetToMS=\(pausedTimeMilliseconds)")#endifscheduleResumeCorrectionTick(generation: generation)}private func noteResumeAudioOutputIfNeeded(reason: String) {guard resumeCorrectionStartDate != nil else {return}hasObservedResumeAudioOutput = true#if DEBUGlet loudness = mediaPlayer.momentaryLoudnessprint("[DreamioVLC] resume-correction audio-observed reason=\(reason) loudness=\(loudness?.loudnessValue ?? 0) date=\(loudness?.date ?? 0)")#endif}private func resetResumeCorrectionState() {pausedTimeMilliseconds = nilresetResumeCorrectionRuntimeState()}private func resetResumeCorrectionRuntimeState() {resumeCorrectionGeneration += 1resumeCorrectionStartDate = nilhasObservedResumeAudioOutput = false}private func isPauseableState(_ state: VLCMediaPlayerState) -> Bool {switch state {case .playing, .buffering:27 unmodified lines#if DEBUGprivate func logPlaybackSnapshot(reason: String) {let mediaLength = mediaPlayer.media?.length.intValue ?? 0print("[DreamioVLC] snapshot reason=\(reason) state=\(stateName(mediaPlayer.state)) isPlaying=\(mediaPlayer.isPlaying) canPause=\(mediaPlayer.canPause) seekable=\(mediaPlayer.isSeekable) currentTime=\(String(format: "%.3f", currentTime)) duration=\(String(format: "%.3f", TimeInterval(max(0, mediaLength)) / 1000)) position=\(String(format: "%.4f", mediaPlayer.position)) audioDelay=\(mediaPlayer.currentAudioPlaybackDelay) pausedTimeMS=\(pausedTimeMilliseconds.map(String.init) ?? "nil") resumeActive=\(resumeCorrectionStartDate != nil) audioObserved=\(hasObservedResumeAudioOutput) readyReported=\(didReportReadyForCurrentMedia)")}#endif156 unmodified lines#if canImport(MobileVLCKit)extension VLCNativePlaybackBackend: VLCMediaPlayerDelegate {func mediaPlayerTimeChanged(_ aNotification: Notification) {#if DEBUGif resumeCorrectionStartDate != nil {logPlaybackSnapshot(reason: "time-change-during-resume")}#endif}func mediaPlayerLoudnessChanged(_ aNotification: Notification) {noteResumeAudioOutputIfNeeded(reason: "loudness-changed")}func mediaPlayerStateChanged(_ aNotification: Notification) {#if DEBUGlogPlaybackSnapshot(reason: "state-change")
Tests/StreamResolverTests.swift ยท covers resume-hold policy decisions
25 unmodified lines262728293031496 unmodified lines52852953053153253325 unmodified linestestSubtitleOptionMappingIncludesNone()testNativePlaybackTogglePolicy()testNativePlaybackAudioSessionPolicy()print("StreamResolverTests passed")}496 unmodified linesassertEqual(NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: .unknown), false)}private static func assertEqual<T: Equatable>(_ actual: T?, _ expected: T, file: StaticString = #file, line: UInt = #line) {assert(actual == expected, "Expected \(String(describing: expected)), got \(String(describing: actual))", file: file, line: line)}25 unmodified lines26272829303132496 unmodified lines52953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856925 unmodified linestestSubtitleOptionMappingIncludesNone()testNativePlaybackTogglePolicy()testNativePlaybackAudioSessionPolicy()testNativePlaybackResumePolicy()print("StreamResolverTests passed")}496 unmodified linesassertEqual(NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: .unknown), false)}private static func testNativePlaybackResumePolicy() {assertEqual(NativePlaybackResumePolicy.shouldHoldVideoAtPausedTime(elapsedSinceResume: 0.4,hasObservedAudioOutput: false,mediaAdvanceMilliseconds: 500),true)assertEqual(NativePlaybackResumePolicy.shouldHoldVideoAtPausedTime(elapsedSinceResume: 0.4,hasObservedAudioOutput: true,mediaAdvanceMilliseconds: 500),false)assertEqual(NativePlaybackResumePolicy.shouldHoldVideoAtPausedTime(elapsedSinceResume: 1.3,hasObservedAudioOutput: false,mediaAdvanceMilliseconds: 500),false)assertEqual(NativePlaybackResumePolicy.shouldHoldVideoAtPausedTime(elapsedSinceResume: 0.4,hasObservedAudioOutput: false,mediaAdvanceMilliseconds: 80),false)}private static func assertEqual<T: Equatable>(_ actual: T?, _ expected: T, file: StaticString = #file, line: UInt = #line) {assert(actual == expected, "Expected \(String(describing: expected)), got \(String(describing: actual))", file: file, line: line)}
Related issues or PRs
No Beads issue was provided for this follow-up. Manual device validation remains the key confirmation step for the original timing-sensitive symptom.
Validation
DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer xcodebuild -workspace Dreamio.xcworkspace -scheme Dreamio -configuration Debug -sdk iphonesimulator buildsucceeded for the original change.xcodebuild -workspace Dreamio.xcworkspace -scheme Dreamio -destination 'generic/platform=iOS Simulator' buildsucceeded for the follow-up resume correction.- The build still reports the existing MobileVLCKit run-script output warning and AppIntents metadata warning.
Issues, Limitations, and Mitigations
- This is a native route-warmth fix; it still needs device playback confirmation because the original symptom is timing-sensitive and hardware/audio-route dependent.
- If a specific Bluetooth or AirPlay route still wakes slowly, further mitigation may need route-specific handling or a brief visual resume hold.
Follow-up Work
- Manually validate pause/resume on device speakers, Bluetooth, and AirPlay if available.
- If lag remains, add measured resume diagnostics for state-change time, first audible audio route availability, and video clock movement.