Dreamio turn document

Fix VLC resume audio sync

Removed the repeated resume seek-hold loop and aligned MobileVLCKit streaming options with VLC-iOS so paused streams can resume without app-driven video rewinds fighting libVLC buffering.

2026-05-27Beads issue dreamio-0piMobileVLCKit streaming

Summary

The VLC native backend now trusts MobileVLCKit resume semantics instead of repeatedly resetting playback time after every resume. DEBUG logging still records resume timing, media advance, and loudness callbacks so device testing can confirm whether audio output catches up naturally.

Changes Made

Removed the old NativePlaybackResumePolicy seek-hold loop that rewound video every 80 ms until loudness changed.
Added NativePlaybackStreamingOptionsPolicy with one conservative :network-caching=1000 option.
Dropped always-on :file-caching=1000 and :live-caching=1000 for direct streaming playback.
Stopped re-preparing the audio session after pause; resume still prepares it before calling play().
Updated focused Swift tests to cover the streaming options policy.

Context

Research checked MobileVLCKit and VLC-iOS sources. VLCKit documents play() as resuming at the paused position. VLC-iOS uses per-media options and defaults network caching to roughly 999 ms, without adding file and live caching to every open network stream. This made the repeated app-level time reset the riskiest part of the previous workaround.

Important Implementation Details

Resume is now an observation path, not a correction path. It logs what happens but does not seek during resume.
The paused millisecond value is preserved only for DEBUG comparison against later media advance.
The streaming option policy is isolated in NativePlaybackBackend.swift so future HLS, live, RTSP, or direct-file profiles can be tested deliberately.
Manual device testing remains essential because simulator builds cannot validate MobileVLCKit audio output timing.

Relevant Diff Snippets

Dreamio/NativePlaybackBackend.swift · streaming option policy

Dreamio/NativePlaybackBackend.swift

-16+4

74 unmodified lines
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
74 unmodified lines
    }
}

enum NativePlaybackResumePolicy {
    static let freezeInterval: TimeInterval = 0.08
    static let maximumFreezeDuration: TimeInterval = 1.2
    static let maximumAllowedSilentAdvance: Int32 = 120

    static func shouldHoldVideoAtPausedTime(
        elapsedSinceResume: TimeInterval,
        hasObservedAudioOutput: Bool,
        mediaAdvanceMilliseconds: Int32
    ) -> Bool {
        guard !hasObservedAudioOutput else {
            return false
        }
        guard elapsedSinceResume < maximumFreezeDuration else {
            return false
        }
        return mediaAdvanceMilliseconds > maximumAllowedSilentAdvance
    }
}

74 unmodified lines
75
76
77
78
79
80
81
82
83
84
85
74 unmodified lines
    }
}

enum NativePlaybackStreamingOptionsPolicy {
    static let networkCachingMilliseconds = 1000

    static func mediaOptions() -> [String] {
        [":network-caching=\(networkCachingMilliseconds)"]
    }
}

Dreamio/VLCNativePlaybackBackend.swift · remove resume seek-holding

Dreamio/VLCNativePlaybackBackend.swift

-68+37

33 unmodified lines
34
35
36
37
38
39
40
41
23 unmodified lines
65
66
67
68
69
70
71
36 unmodified lines
108
109
110
111
112
113
114
115
116
14 unmodified lines
131
132
133
134
135
136
137
138
139
109 unmodified lines
249
250
251
252
253
254
255
99 unmodified lines
355
356
357
358
359
360
361
362
363
364
365
11 unmodified lines
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
30 unmodified lines
490
491
492
493
494
495
496
158 unmodified lines
655
656
657
658
659
660
661
33 unmodified lines
    private var externalSubtitleDisplayNamesByTrackID: [Int32: String] = [:]
    private var didReportReadyForCurrentMedia = false
    private var pausedTimeMilliseconds: Int32?
    private var resumeCorrectionGeneration = 0
    private var resumeCorrectionStartDate: Date?
    private var hasObservedResumeAudioOutput = false
    private var lastToggleDate = Date.distantPast
    private let minimumToggleInterval: TimeInterval = 0.35
23 unmodified lines
        pendingExternalSubtitleDisplayNames.removeAll()
        externalSubtitleDisplayNamesByTrackID.removeAll()
        didReportReadyForCurrentMedia = false
        resetResumeCorrectionState()
        lastToggleDate = .distantPast
        let media = VLCMedia(url: request.playbackURL)
        let headerValue = request.headers
36 unmodified lines
#endif
        mediaPlayer.play()
        if isResumingFromPause {
            beginResumeCorrection()
        } else {
            resetResumeCorrectionState()
        }
#if DEBUG
        logPlaybackSnapshot(reason: "after-play-command")
14 unmodified lines
            return
        }
        pausedTimeMilliseconds = mediaPlayer.time.intValue
        resetResumeCorrectionRuntimeState()
        mediaPlayer.pause()
        keepAudioSessionWarmAfterPause()
#if DEBUG
        logPlaybackSnapshot(reason: "after-pause-command")
        DispatchQueue.main.asyncAfter(deadline: .now() + 0.25) { [weak self] in
109 unmodified lines
        logPlaybackSnapshot(reason: "before-stop")
#endif
        didReportReadyForCurrentMedia = false
        resetResumeCorrectionState()
        mediaPlayer.stop()
        mediaPlayer.drawable = nil
        mediaPlayer.media = nil
99 unmodified lines

#if canImport(MobileVLCKit)
    private func addConservativePlaybackOptions(to media: VLCMedia) {
        [
            ":network-caching=1000",
            ":file-caching=1000",
            ":live-caching=1000"
        ].forEach { media.addOption($0) }
    }

    private func prepareAudioSessionForPlayback(reason: String) {
11 unmodified lines
        }
    }

    private func keepAudioSessionWarmAfterPause() {
        prepareAudioSessionForPlayback(reason: "pause-keep-warm")
        DispatchQueue.main.asyncAfter(deadline: .now() + 0.08) { [weak self] in
            self?.prepareAudioSessionForPlayback(reason: "pause-keep-warm-follow-up")
        }
    }

    private func beginResumeCorrection() {
        guard let pausedTimeMilliseconds else {
            return
        }
        resetResumeCorrectionRuntimeState()
        resumeCorrectionGeneration += 1
        let generation = resumeCorrectionGeneration
        resumeCorrectionStartDate = Date()
#if DEBUG
        print("[DreamioVLC] resume-correction begin pausedTimeMS=\(pausedTimeMilliseconds)")
#endif
        scheduleResumeCorrectionTick(generation: generation)
    }

    private func scheduleResumeCorrectionTick(generation: Int) {
        DispatchQueue.main.asyncAfter(deadline: .now() + NativePlaybackResumePolicy.freezeInterval) { [weak self] in
            self?.performResumeCorrectionTick(generation: generation)
        }
    }

    private func performResumeCorrectionTick(generation: Int) {
        guard generation == resumeCorrectionGeneration,
              let pausedTimeMilliseconds,
              let resumeCorrectionStartDate else {
            return
        }

        let elapsed = Date().timeIntervalSince(resumeCorrectionStartDate)
        let currentTimeMilliseconds = mediaPlayer.time.intValue
        let advance = max(0, currentTimeMilliseconds - pausedTimeMilliseconds)
        let shouldHold = NativePlaybackResumePolicy.shouldHoldVideoAtPausedTime(
            elapsedSinceResume: elapsed,
            hasObservedAudioOutput: hasObservedResumeAudioOutput,
            mediaAdvanceMilliseconds: advance
        )

        guard shouldHold else {
#if DEBUG
            print("[DreamioVLC] resume-correction release elapsed=\(String(format: "%.3f", elapsed)) audioObserved=\(hasObservedResumeAudioOutput) advanceMS=\(advance)")
#endif
            resetResumeCorrectionRuntimeState()
            return
        }

        mediaPlayer.time = VLCTime(int: pausedTimeMilliseconds)
#if DEBUG
        print("[DreamioVLC] resume-correction hold elapsed=\(String(format: "%.3f", elapsed)) advanceMS=\(advance) resetToMS=\(pausedTimeMilliseconds)")
#endif
        scheduleResumeCorrectionTick(generation: generation)
    }

    private func noteResumeAudioOutputIfNeeded(reason: String) {
        guard resumeCorrectionStartDate != nil else {
            return
        }
        hasObservedResumeAudioOutput = true
#if DEBUG
        let loudness = mediaPlayer.momentaryLoudness
        print("[DreamioVLC] resume-correction audio-observed reason=\(reason) loudness=\(loudness?.loudnessValue ?? 0) date=\(loudness?.date ?? 0)")
#endif
    }

    private func resetResumeCorrectionState() {
        pausedTimeMilliseconds = nil
        resetResumeCorrectionRuntimeState()
    }

    private func resetResumeCorrectionRuntimeState() {
        resumeCorrectionGeneration += 1
        resumeCorrectionStartDate = nil
        hasObservedResumeAudioOutput = false
    }

30 unmodified lines
#if DEBUG
    private func logPlaybackSnapshot(reason: String) {
        let mediaLength = mediaPlayer.media?.length.intValue ?? 0
        print("[DreamioVLC] snapshot reason=\(reason) state=\(stateName(mediaPlayer.state)) isPlaying=\(mediaPlayer.isPlaying) canPause=\(mediaPlayer.canPause) seekable=\(mediaPlayer.isSeekable) currentTime=\(String(format: "%.3f", currentTime)) duration=\(String(format: "%.3f", TimeInterval(max(0, mediaLength)) / 1000)) position=\(String(format: "%.4f", mediaPlayer.position)) audioDelay=\(mediaPlayer.currentAudioPlaybackDelay) pausedTimeMS=\(pausedTimeMilliseconds.map(String.init) ?? "nil") resumeActive=\(resumeCorrectionStartDate != nil) audioObserved=\(hasObservedResumeAudioOutput) readyReported=\(didReportReadyForCurrentMedia)")
    }
#endif

158 unmodified lines
extension VLCNativePlaybackBackend: VLCMediaPlayerDelegate {
    func mediaPlayerTimeChanged(_ aNotification: Notification) {
#if DEBUG
        if resumeCorrectionStartDate != nil {
            logPlaybackSnapshot(reason: "time-change-during-resume")
        }
#endif33 unmodified lines
34
35
36
37
38
39
40
41
23 unmodified lines
65
66
67
68
69
70
71
36 unmodified lines
108
109
110
111
112
113
114
115
116
14 unmodified lines
131
132
133
134
135
136
137
138
109 unmodified lines
248
249
250
251
252
253
254
99 unmodified lines
354
355
356
357
358
359
360
11 unmodified lines
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
30 unmodified lines
459
460
461
462
463
464
465
158 unmodified lines
624
625
626
627
628
629
630
33 unmodified lines
    private var externalSubtitleDisplayNamesByTrackID: [Int32: String] = [:]
    private var didReportReadyForCurrentMedia = false
    private var pausedTimeMilliseconds: Int32?
    private var resumeObservationGeneration = 0
    private var resumeObservationStartDate: Date?
    private var hasObservedResumeAudioOutput = false
    private var lastToggleDate = Date.distantPast
    private let minimumToggleInterval: TimeInterval = 0.35
23 unmodified lines
        pendingExternalSubtitleDisplayNames.removeAll()
        externalSubtitleDisplayNamesByTrackID.removeAll()
        didReportReadyForCurrentMedia = false
        resetResumeObservationState()
        lastToggleDate = .distantPast
        let media = VLCMedia(url: request.playbackURL)
        let headerValue = request.headers
36 unmodified lines
#endif
        mediaPlayer.play()
        if isResumingFromPause {
            beginResumeObservation()
        } else {
            resetResumeObservationState()
        }
#if DEBUG
        logPlaybackSnapshot(reason: "after-play-command")
14 unmodified lines
            return
        }
        pausedTimeMilliseconds = mediaPlayer.time.intValue
        resetResumeObservationRuntimeState()
        mediaPlayer.pause()
#if DEBUG
        logPlaybackSnapshot(reason: "after-pause-command")
        DispatchQueue.main.asyncAfter(deadline: .now() + 0.25) { [weak self] in
109 unmodified lines
        logPlaybackSnapshot(reason: "before-stop")
#endif
        didReportReadyForCurrentMedia = false
        resetResumeObservationState()
        mediaPlayer.stop()
        mediaPlayer.drawable = nil
        mediaPlayer.media = nil
99 unmodified lines

#if canImport(MobileVLCKit)
    private func addConservativePlaybackOptions(to media: VLCMedia) {
        NativePlaybackStreamingOptionsPolicy.mediaOptions().forEach { media.addOption($0) }
    }

    private func prepareAudioSessionForPlayback(reason: String) {
11 unmodified lines
        }
    }

    private func beginResumeObservation() {
        guard let pausedTimeMilliseconds else {
            return
        }
        resetResumeObservationRuntimeState()
        resumeObservationGeneration += 1
        let generation = resumeObservationGeneration
        resumeObservationStartDate = Date()
#if DEBUG
        print("[DreamioVLC] resume-observation begin pausedTimeMS=\(pausedTimeMilliseconds)")
        [0.25, 0.75, 1.5].forEach { delay in
            DispatchQueue.main.asyncAfter(deadline: .now() + delay) { [weak self] in
                self?.logResumeObservation(generation: generation, delay: delay)
            }
        }
#endif
    }

#if DEBUG
    private func logResumeObservation(generation: Int, delay: TimeInterval) {
        guard generation == resumeObservationGeneration,
              let pausedTimeMilliseconds,
              let resumeObservationStartDate else {
            return
        }
        let elapsed = Date().timeIntervalSince(resumeObservationStartDate)
        let advance = max(0, mediaPlayer.time.intValue - pausedTimeMilliseconds)
        print("[DreamioVLC] resume-observation tick delay=\(String(format: "%.2f", delay)) elapsed=\(String(format: "%.3f", elapsed)) audioObserved=\(hasObservedResumeAudioOutput) advanceMS=\(advance)")
        logPlaybackSnapshot(reason: "resume-observation-\(String(format: "%.2f", delay))")
    }
#endif

    private func noteResumeAudioOutputIfNeeded(reason: String) {
        guard resumeObservationStartDate != nil else {
            return
        }
        hasObservedResumeAudioOutput = true
#if DEBUG
        let loudness = mediaPlayer.momentaryLoudness
        print("[DreamioVLC] resume-observation audio-observed reason=\(reason) loudness=\(loudness?.loudnessValue ?? 0) date=\(loudness?.date ?? 0)")
#endif
    }

    private func resetResumeObservationState() {
        pausedTimeMilliseconds = nil
        resetResumeObservationRuntimeState()
    }

    private func resetResumeObservationRuntimeState() {
        resumeObservationGeneration += 1
        resumeObservationStartDate = nil
        hasObservedResumeAudioOutput = false
    }

30 unmodified lines
#if DEBUG
    private func logPlaybackSnapshot(reason: String) {
        let mediaLength = mediaPlayer.media?.length.intValue ?? 0
        print("[DreamioVLC] snapshot reason=\(reason) state=\(stateName(mediaPlayer.state)) isPlaying=\(mediaPlayer.isPlaying) canPause=\(mediaPlayer.canPause) seekable=\(mediaPlayer.isSeekable) currentTime=\(String(format: "%.3f", currentTime)) duration=\(String(format: "%.3f", TimeInterval(max(0, mediaLength)) / 1000)) position=\(String(format: "%.4f", mediaPlayer.position)) audioDelay=\(mediaPlayer.currentAudioPlaybackDelay) pausedTimeMS=\(pausedTimeMilliseconds.map(String.init) ?? "nil") resumeActive=\(resumeObservationStartDate != nil) audioObserved=\(hasObservedResumeAudioOutput) readyReported=\(didReportReadyForCurrentMedia)")
    }
#endif

158 unmodified lines
extension VLCNativePlaybackBackend: VLCMediaPlayerDelegate {
    func mediaPlayerTimeChanged(_ aNotification: Notification) {
#if DEBUG
        if resumeObservationStartDate != nil {
            logPlaybackSnapshot(reason: "time-change-during-resume")
        }
#endif

Tests/StreamResolverTests.swift · policy coverage

Tests/StreamResolverTests.swift

-34+6

25 unmodified lines
26
27
28
29
30
31
32
496 unmodified lines
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
25 unmodified lines
        testSubtitleOptionMappingIncludesNone()
        testNativePlaybackTogglePolicy()
        testNativePlaybackAudioSessionPolicy()
        testNativePlaybackResumePolicy()
        print("StreamResolverTests passed")
    }

496 unmodified lines
        assertEqual(NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: .unknown), false)
    }

    private static func testNativePlaybackResumePolicy() {
        assertEqual(
            NativePlaybackResumePolicy.shouldHoldVideoAtPausedTime(
                elapsedSinceResume: 0.4,
                hasObservedAudioOutput: false,
                mediaAdvanceMilliseconds: 500
            ),
            true
        )
        assertEqual(
            NativePlaybackResumePolicy.shouldHoldVideoAtPausedTime(
                elapsedSinceResume: 0.4,
                hasObservedAudioOutput: true,
                mediaAdvanceMilliseconds: 500
            ),
            false
        )
        assertEqual(
            NativePlaybackResumePolicy.shouldHoldVideoAtPausedTime(
                elapsedSinceResume: 1.3,
                hasObservedAudioOutput: false,
                mediaAdvanceMilliseconds: 500
            ),
            false
        )
        assertEqual(
            NativePlaybackResumePolicy.shouldHoldVideoAtPausedTime(
                elapsedSinceResume: 0.4,
                hasObservedAudioOutput: false,
                mediaAdvanceMilliseconds: 80
            ),
            false
        )
    }

    private static func assertEqual<T: Equatable>(_ actual: T?, _ expected: T, file: StaticString = #file, line: UInt = #line) {25 unmodified lines
26
27
28
29
30
31
32
496 unmodified lines
529
530
531
532
533
534
535
536
537
538
539
25 unmodified lines
        testSubtitleOptionMappingIncludesNone()
        testNativePlaybackTogglePolicy()
        testNativePlaybackAudioSessionPolicy()
        testNativePlaybackStreamingOptionsPolicy()
        print("StreamResolverTests passed")
    }

496 unmodified lines
        assertEqual(NativePlaybackAudioSessionPolicy.shouldPrepareBeforePlayback(from: .unknown), false)
    }

    private static func testNativePlaybackStreamingOptionsPolicy() {
        assertEqual(NativePlaybackStreamingOptionsPolicy.networkCachingMilliseconds, 1000)
        assertEqual(NativePlaybackStreamingOptionsPolicy.mediaOptions(), [":network-caching=1000"])
        assertEqual(NativePlaybackStreamingOptionsPolicy.mediaOptions().contains(":file-caching=1000"), false)
        assertEqual(NativePlaybackStreamingOptionsPolicy.mediaOptions().contains(":live-caching=1000"), false)
    }

    private static func assertEqual<T: Equatable>(_ actual: T?, _ expected: T, file: StaticString = #file, line: UInt = #line) {

Diffs were rendered with @pierre/diffs/ssr. Each rendered file diff is contained in its own shell.

Expected Impact for End-Users

Every resume should have less opportunity for video to race ahead because Dreamio no longer forces repeated seeks while libVLC is restoring stream buffers and audio output. The user-visible expectation is smoother resume behavior for direct-file streams in the native player.

Validation

Passed: DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer xcrun --sdk iphonesimulator swiftc -target arm64-apple-ios18.0-simulator Dreamio/StreamCandidate.swift Dreamio/StreamResolver.swift Dreamio/NativePlaybackBackend.swift Tests/StreamResolverTests.swift -o /tmp/dreamio-stream-tests-ios.
Passed: DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer xcodebuild -workspace Dreamio.xcworkspace -scheme Dreamio -configuration Debug -destination 'generic/platform=iOS Simulator' CODE_SIGNING_ALLOWED=NO build.
Not run: on-device streaming playback validation, which is required to confirm real MobileVLCKit audio and video timing.

Issues, Limitations, and Mitigations

This change removes the highest-risk workaround but does not prove the underlying MobileVLCKit stream behavior is fixed until a real device plays and resumes the problematic streams.
DEBUG resume-observation logs remain available to compare audio-observed timing against media advance during manual validation.
The app still uses one network cache value for all direct streams; protocol-specific profiles can be added if device logs show one stream family needs different treatment.

Follow-up Work

Validate on a real iPhone or iPad with the exact streams that previously showed video-first resume.
If audio still lags, capture DEBUG logs around resume-observation ticks and test a protocol-specific cache profile rather than reintroducing repeated seeks.