Dreamio turn document

Audio Warm Resume

Dreamio now keeps the iOS playback audio session warm while native VLC playback is paused and explicitly primes it before resuming, so video no longer races ahead while audio wakes up.

2026-05-27 Beads issue dreamio-69r Native playback

Summary

Dreamio now keeps the iOS playback audio session warm while native VLC playback is paused and explicitly primes it before resuming, so video no longer races ahead while audio wakes up.

Changes Made

  • Added a native audio-session policy for deciding when playback should prime the audio route before starting.
  • Prepared AVAudioSession with .playback and .moviePlayback before initial VLC playback and before resuming from inactive states.
  • Kept the audio session active immediately after pause, including a short follow-up activation, so the route is less likely to go cold before the next play tap.
  • Added lightweight policy coverage to the existing Swift test harness.

Context

Pausing and resuming a native video could let the video frame clock restart immediately while the audio route took a moment to become audible. Earlier low-latency and audio-delay attempts did not address the route warm-up itself, so this change targets the iOS audio session around pause/resume.

Important Implementation Details

  • VLCNativePlaybackBackend.play(request:) primes the audio session before assigning and starting new media.
  • VLCNativePlaybackBackend.play() only primes on paused, stopped, ended, or error states, avoiding repeated session churn while already opening, buffering, or playing.
  • pause() no longer leaves the route entirely to VLC after the pause command; it explicitly keeps the app audio session warm and repeats once after 80 ms.
  • Debug logs include the audio-session preparation reason to help confirm whether a resume path warmed the session before VLC starts rendering again.

Relevant Diff Snippets

Dreamio/NativePlaybackBackend.swift ยท adds a testable audio-session resume policy

Dreamio/NativePlaybackBackend.swift
+11
62 unmodified lines
63
64
65
66
67
68
69
70
62 unmodified lines
        }
    }
}


enum NativePlaybackError: LocalizedError {
    case backendUnavailable
    case startupTimedOut
    case playbackFailed
62 unmodified lines
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
62 unmodified lines
        }
    }
}


enum NativePlaybackAudioSessionPolicy {
    static func shouldPrepareBeforePlayback(from state: NativePlaybackToggleState) -> Bool {
        switch state {
        case .paused, .stopped, .ended, .error:
            return true
        case .opening, .buffering, .playing, .unknown:
            return false
        }
    }
}


enum NativePlaybackError: LocalizedError {
    case backendUnavailable
    case startupTimedOut
    case playbackFailed

Dreamio/VLCNativePlaybackBackend.swift ยท prepares and keeps AVAudioSession active around pause/resume

Dreamio/VLCNativePlaybackBackend.swift
+29
1
2
3
4
67 unmodified lines
72
73
74
75
76
77
78
79
10 unmodified lines
90
91
92
93
94
95
96
97
15 unmodified lines
113
114
115
116
117
118
119
120
220 unmodified lines
341
342
343
344
345
346
347
348
import UIKit


#if canImport(MobileVLCKit)
import MobileVLCKit
67 unmodified lines
        if !headerValue.isEmpty {
            media.addOption(":http-header=\(headerValue)")
        }
        addConservativePlaybackOptions(to: media)
        mediaPlayer.currentAudioPlaybackDelay = 0


        mediaPlayer.media = media
#if DEBUG
10 unmodified lines
    }


    func play() {
#if canImport(MobileVLCKit)
#if DEBUG
        logPlaybackSnapshot(reason: "before-play")
#endif
        mediaPlayer.play()
15 unmodified lines
#endif
            return
        }
        mediaPlayer.pause()
#if DEBUG
        logPlaybackSnapshot(reason: "after-pause-command")
        DispatchQueue.main.asyncAfter(deadline: .now() + 0.25) { [weak self] in
            self?.logPlaybackSnapshot(reason: "pause-follow-up-250ms")
220 unmodified lines
            ":clock-jitter=0"
        ].forEach { media.addOption($0) }
    }


    private func isPauseableState(_ state: VLCMediaPlayerState) -> Bool {
        switch state {
        case .playing, .buffering:
            return true
1
2
3
4
5
67 unmodified lines
73
74
75
76
77
78
79
80
81
10 unmodified lines
92
93
94
95
96
97
98
99
100
101
102
103
15 unmodified lines
119
120
121
122
123
124
125
126
127
220 unmodified lines
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
import AVFoundation
import UIKit


#if canImport(MobileVLCKit)
import MobileVLCKit
67 unmodified lines
        if !headerValue.isEmpty {
            media.addOption(":http-header=\(headerValue)")
        }
        addConservativePlaybackOptions(to: media)
        prepareAudioSessionForPlayback(reason: "initial-play")
        mediaPlayer.currentAudioPlaybackDelay = 0


        mediaPlayer.media = media
#if DEBUG
10 unmodified lines
    }


    func play() {
#if canImport(MobileVLCKit)
        let toggleState = playbackToggleState(for: mediaPlayer.state)
        if NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: toggleState) {
            prepareAudioSessionForPlayback(reason: "resume-from-\(toggleState)")
        }
#if DEBUG
        logPlaybackSnapshot(reason: "before-play")
#endif
        mediaPlayer.play()
15 unmodified lines
#endif
            return
        }
        mediaPlayer.pause()
        keepAudioSessionWarmAfterPause()
#if DEBUG
        logPlaybackSnapshot(reason: "after-pause-command")
        DispatchQueue.main.asyncAfter(deadline: .now() + 0.25) { [weak self] in
            self?.logPlaybackSnapshot(reason: "pause-follow-up-250ms")
220 unmodified lines
            ":clock-jitter=0"
        ].forEach { media.addOption($0) }
    }


    private func prepareAudioSessionForPlayback(reason: String) {
        do {
            let session = AVAudioSession.sharedInstance()
            try session.setCategory(.playback, mode: .moviePlayback, options: [])
            try session.setActive(true)
#if DEBUG
            print("[DreamioVLC] audio-session prepared reason=\(reason) category=\(session.category.rawValue) mode=\(session.mode.rawValue)")
#endif
        } catch {
#if DEBUG
            print("[DreamioVLC] audio-session prepare failed reason=\(reason) error=\(error.localizedDescription)")
#endif
        }
    }


    private func keepAudioSessionWarmAfterPause() {
        prepareAudioSessionForPlayback(reason: "pause-keep-warm")
        DispatchQueue.main.asyncAfter(deadline: .now() + 0.08) { [weak self] in
            self?.prepareAudioSessionForPlayback(reason: "pause-keep-warm-follow-up")
        }
    }


    private func isPauseableState(_ state: VLCMediaPlayerState) -> Bool {
        switch state {
        case .playing, .buffering:
            return true

Tests/StreamResolverTests.swift ยท covers the audio-session resume policy

Tests/StreamResolverTests.swift
+12
23 unmodified lines
24
25
26
27
28
29
30
31
483 unmodified lines
515
516
517
518
519
520
521
522
23 unmodified lines
        testSubtitleDisplayNameNormalization()
        testSubtitleDisplayNameUsesPreservedNamesForGenericVLCTracks()
        testSubtitleOptionMappingIncludesNone()
        testNativePlaybackTogglePolicy()
        print("StreamResolverTests passed")
    }


    private static func testClassifierPrefersObservedDirectFile() {
483 unmodified lines
        assertEqual(NativePlaybackTogglePolicy.action(for: .opening), .waitForTransition)
        assertEqual(NativePlaybackTogglePolicy.action(for: .unknown), .waitForTransition)
    }


    private static func assertEqual<T: Equatable>(_ actual: T?, _ expected: T, file: StaticString = #file, line: UInt = #line) {
        assert(actual == expected, "Expected \(String(describing: expected)), got \(String(describing: actual))", file: file, line: line)
    }


23 unmodified lines
24
25
26
27
28
29
30
31
32
483 unmodified lines
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
23 unmodified lines
        testSubtitleDisplayNameNormalization()
        testSubtitleDisplayNameUsesPreservedNamesForGenericVLCTracks()
        testSubtitleOptionMappingIncludesNone()
        testNativePlaybackTogglePolicy()
        testNativePlaybackAudioSessionPolicy()
        print("StreamResolverTests passed")
    }


    private static func testClassifierPrefersObservedDirectFile() {
483 unmodified lines
        assertEqual(NativePlaybackTogglePolicy.action(for: .opening), .waitForTransition)
        assertEqual(NativePlaybackTogglePolicy.action(for: .unknown), .waitForTransition)
    }


    private static func testNativePlaybackAudioSessionPolicy() {
        assertEqual(NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: .paused), true)
        assertEqual(NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: .stopped), true)
        assertEqual(NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: .ended), true)
        assertEqual(NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: .error), true)
        assertEqual(NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: .playing), false)
        assertEqual(NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: .buffering), false)
        assertEqual(NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: .opening), false)
        assertEqual(NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: .unknown), false)
    }


    private static func assertEqual<T: Equatable>(_ actual: T?, _ expected: T, file: StaticString = #file, line: UInt = #line) {
        assert(actual == expected, "Expected \(String(describing: expected)), got \(String(describing: actual))", file: file, line: line)
    }

Diffs are generated with @pierre/diffs/ssr at documentation time. Each file diff stays inside its own shell so the page remains readable as a static HTML artifact.

Expected Impact for End-Users

After pausing and pressing play, audio should be ready before VLC restarts visible playback, reducing the perceived gap where the video moves but audio has not yet caught up.

Validation

  • DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer xcodebuild -workspace Dreamio.xcworkspace -scheme Dreamio -configuration Debug -sdk iphonesimulator build succeeded.
  • The build still reports the existing MobileVLCKit run-script output warning and AppIntents metadata warning.

Issues, Limitations, and Mitigations

  • This is a native route-warmth fix; it still needs device playback confirmation because the original symptom is timing-sensitive and hardware/audio-route dependent.
  • If a specific Bluetooth or AirPlay route still wakes slowly, further mitigation may need route-specific handling or a brief visual resume hold.

Follow-up Work

  • Manually validate pause/resume on device speakers, Bluetooth, and AirPlay if available.
  • If lag remains, add measured resume diagnostics for state-change time, first audible audio route availability, and video clock movement.