Dreamio turn document · 2026-05-25

# Fix OpenSubtitles Native Captions

OpenSubtitles candidates now survive more Stremio and OpenSubtitles payload shapes, resolve through nested download responses, attach to VLC with clearer diagnostics, and get preferred when external tracks become visible after playback has already started.

Beads: dreamio-hzj Branch: lavender/opensubtitles Native player captions

## Summary

Hardened the external subtitle path so OpenSubtitles tracks are more likely to appear as selectable VLC caption tracks alongside embedded MKV subtitles. The change focuses on discovery, candidate parsing, resolver compatibility, VLC visibility timing, and debug output.

## Changes Made

- Added Beads bug `dreamio-hzj` before implementation.
- Expanded the web bridge to recognize subtitle objects with `attributes`, `files`, `file_id`, `download`, `link`, `file_name`, and language metadata.
- Extended Swift subtitle candidate parsing for nested OpenSubtitles payloads and `file_id` download candidates.
- Kept parent label and language metadata when nested subtitle URLs are selected during resolution.
- Changed VLC attachment behavior so newly visible external tracks are preferred over an earlier automatic embedded-track selection when the user has not manually chosen a track.
- Added focused tests for nested OpenSubtitles attributes/files payloads and nested API download responses.

## Context

The native captions menu was already able to show embedded VLC subtitle tracks, which narrowed the problem to external subtitle handoff. OpenSubtitles data can arrive as direct file URLs, API download URLs, nested file objects, or delayed network payloads after native playback has started. VLC also exposes subtitle slaves asynchronously, so the first menu refresh can happen before an external track exists.

## Important Implementation Details

- The browser bridge now posts likely subtitle-shaped objects even when they are not immediately reduced to a direct URL. Swift performs the final recursive parsing.
- `SubtitleCandidateParser` now walks `attributes`, `data`, and `results` early, which better matches OpenSubtitles API structures.
- `file_id` values are converted into OpenSubtitles download API candidates so the existing resolver path can try to turn them into direct subtitle files.
- VLC records the subtitle track IDs visible before external subtitle attachment. Delayed refreshes use that baseline to identify a newly visible external track.
- If VLC accepts a subtitle slave but no new track appears by the final delayed refresh, the debug log now says so explicitly with baseline and visible track counts.

## Relevant Diff Snippets

Rendered with `@pierre/diffs/ssr` from the working tree diff.

Dreamio/DreamioWebViewController.swift
-5+35

    83 unmodified lines84858687888942 unmodified lines13213313413513613713813914014140 unmodified lines1821831841851861871881891907 unmodified lines19819920020120220320459 unmodified lines26426526626726826983 unmodified lines          const postedSubtitleURLs = new Set();          const subtitleURLPattern = /https?:\/\/[^\s"'<>]+(?:\.srt|\.vtt|\.ass|\.ssa|\.sub|opensubtitles|subtitle)[^\s"'<>]*/ig;          const subtitleSignalPattern = /subtitle|subtitles|opensubtitles|vtt|srt|ass|ssa/i;
              const looksNative = (url) => {            if (!url || typeof url !== "string") {42 unmodified lines          const postSubtitleCandidates = (candidates, debug = {}) => {            const discoveredCount = candidates.length;            const fresh = candidates.filter((candidate) => {              if (postedSubtitleURLs.has(candidate.url)) {                return false;              }              postedSubtitleURLs.add(candidate.url);              return true;            });            if (fresh.length === 0) {40 unmodified lines                entry.fileUrl ||                entry.fileURL              );            const url = absoluteURL(rawURL);            subtitleURLPattern.lastIndex = 0;            if (!url || !subtitleURLPattern.test(url)) {              subtitleURLPattern.lastIndex = 0;              return;            }7 unmodified lines              language: entry && (entry.lang || entry.language) || ""            };            subtitleCandidates.push(candidate);            postSubtitleCandidates([candidate]);          };
              const inspectTrack = (track) => {59 unmodified lines            }            if (typeof payload === "object") {              addSubtitleCandidate(payload);              Object.values(payload).forEach(inspectSubtitlePayload);            }          };83 unmodified lines8485868788899091929394959697989910010110242 unmodified lines14514614714814915015115215315415515615715840 unmodified lines1992002012022032042052062072082092107 unmodified lines21821922022122222322422522622759 unmodified lines28728828929029129229329429529629729829983 unmodified lines          const postedSubtitleURLs = new Set();          const subtitleURLPattern = /https?:\/\/[^\s"'<>]+(?:\.srt|\.vtt|\.ass|\.ssa|\.sub|opensubtitles|subtitle)[^\s"'<>]*/ig;          const subtitleSignalPattern = /subtitle|subtitles|opensubtitles|vtt|srt|ass|ssa/i;          const subtitleObjectKeys = [            "attributes",            "files",            "file_id",            "url",            "download",            "link",            "file",            "file_name",            "filename",            "language",            "lang"          ];
              const looksNative = (url) => {            if (!url || typeof url !== "string") {42 unmodified lines          const postSubtitleCandidates = (candidates, debug = {}) => {            const discoveredCount = candidates.length;            const fresh = candidates.filter((candidate) => {              const key = candidate && (candidate.url || candidate.link || candidate.download || candidate.file || candidate.file_id);              if (!key) {                return false;              }              if (postedSubtitleURLs.has(String(key))) {                return false;              }              postedSubtitleURLs.add(String(key));              return true;            });            if (fresh.length === 0) {40 unmodified lines                entry.fileUrl ||                entry.fileURL              );            let url = absoluteURL(rawURL);            if (!url && entry && entry.file_id) {              url = `https://api.opensubtitles.com/api/v1/download/${encodeURIComponent(String(entry.file_id))}`;            }            subtitleURLPattern.lastIndex = 0;            if (!url || (!subtitleURLPattern.test(url) && !/api\.opensubtitles\.com\/api\/v1\/download/i.test(url))) {              subtitleURLPattern.lastIndex = 0;              return;            }7 unmodified lines              language: entry && (entry.lang || entry.language) || ""            };            subtitleCandidates.push(candidate);            postSubtitleCandidates([candidate], {              discovered: 1,              totalKnown: subtitleCandidates.length            });          };
              const inspectTrack = (track) => {59 unmodified lines            }            if (typeof payload === "object") {              addSubtitleCandidate(payload);              const likelySubtitlePayload = subtitleObjectKeys.some((key) => Object.prototype.hasOwnProperty.call(payload, key));              if (likelySubtitlePayload) {                postSubtitleCandidates([payload], {                  source: "payload-object",                  totalKnown: subtitleCandidates.length                });              }              Object.values(payload).forEach(inspectSubtitlePayload);            }          };

Dreamio/StreamCandidate.swift
-4+22

    131 unmodified lines13213313413513613713813954 unmodified lines19419519619719819920013 unmodified lines21421521621721821922033 unmodified lines254255256257258259131 unmodified lines
    enum SubtitleCandidateParser {    private static let supportedExtensions = ["srt", "vtt", "ass", "ssa", "sub"]    private static let urlFields = ["url", "href", "src", "link", "subtitles", "subtitle", "subtitleUrl", "subtitleURL", "file", "download"]    private static let labelFields = ["label", "name", "title", "file_name", "lang", "language", "id"]    private struct CandidateContext {        let label: String?        let language: String?54 unmodified lines    }
        private static func candidate(from dictionary: [String: Any], context: CandidateContext) -> SubtitleCandidate? {        guard let url = urlFields.lazy.compactMap({ subtitleURL(from: dictionary[$0] as? String) }).first else {            return nil        }
    13 unmodified lines    }
        private static func orderedNestedValues(in dictionary: [String: Any]) -> [Any] {        let preferredKeys = ["subtitles", "subtitle", "files", "downloads", "download"]        var visitedKeys = Set<String>()        var values: [Any] = []
    33 unmodified lines        return url    }
        private static func defaultLabel(for url: URL) -> String {        let lastPathComponent = url.deletingPathExtension().lastPathComponent        return lastPathComponent.isEmpty ? "External Subtitle" : lastPathComponent131 unmodified lines13213313413513613713813954 unmodified lines19419519619719819920020120213 unmodified lines21621721821922022122233 unmodified lines256257258259260261262263264265266267268269270271272273274275276277131 unmodified lines
    enum SubtitleCandidateParser {    private static let supportedExtensions = ["srt", "vtt", "ass", "ssa", "sub"]    private static let urlFields = ["url", "href", "src", "link", "subtitles", "subtitle", "subtitleUrl", "subtitleURL", "file", "download", "fileUrl", "fileURL"]    private static let labelFields = ["label", "name", "title", "file_name", "filename", "lang", "language", "id"]    private struct CandidateContext {        let label: String?        let language: String?54 unmodified lines    }
        private static func candidate(from dictionary: [String: Any], context: CandidateContext) -> SubtitleCandidate? {        guard let url = urlFields.lazy.compactMap({ subtitleURL(from: dictionary[$0] as? String) }).first            ?? openSubtitlesDownloadURL(from: dictionary["file_id"])        else {            return nil        }
    13 unmodified lines    }
        private static func orderedNestedValues(in dictionary: [String: Any]) -> [Any] {        let preferredKeys = ["attributes", "subtitles", "subtitle", "files", "downloads", "download", "data", "results"]        var visitedKeys = Set<String>()        var values: [Any] = []
    33 unmodified lines        return url    }
        private static func openSubtitlesDownloadURL(from value: Any?) -> URL? {        let id: String?        if let string = value as? String, !string.isEmpty {            id = string        } else if let number = value as? NSNumber {            id = number.stringValue        } else {            id = nil        }
            guard let id else {            return nil        }        return URL(string: "https://api.opensubtitles.com/api/v1/download/\(id)")    }
        private static func defaultLabel(for url: URL) -> String {        let lastPathComponent = url.deletingPathExtension().lastPathComponent        return lastPathComponent.isEmpty ? "External Subtitle" : lastPathComponent

Dreamio/VLCNativePlaybackBackend.swift
-7+39

    25 unmodified lines26272829303115 unmodified lines474849505152172 unmodified lines2252262272282292302312322332342352362372382392402412422432442452461 unmodified line2482492502512522532542552569 unmodified lines2662672682692702712722732742752762772782793 unmodified lines28328428528628728844 unmodified lines33333433533633733833925 unmodified lines    private var didAutoSelectSubtitleTrack = false    private var didUserSelectSubtitleTrack = false    private var autoSelectedSubtitleTrackID: Int32?
        override init() {        super.init()15 unmodified lines        didAutoSelectSubtitleTrack = false        didUserSelectSubtitleTrack = false        autoSelectedSubtitleTrackID = nil        let media = VLCMedia(url: request.playbackURL)        let headerValue = request.headers            .map { "\($0.key): \($0.value)" }172 unmodified lines    private func attachSubtitles(_ candidates: [SubtitleCandidate]) -> Int {        var attachedCount = 0        var duplicateCount = 0        candidates.forEach { candidate in            guard !attachedSubtitleURLs.contains(candidate.url) else {                duplicateCount += 1                return            }            attachedSubtitleURLs.insert(candidate.url)            mediaPlayer.addPlaybackSlave(candidate.url, type: .subtitle, enforce: false)            attachedCount += 1#if DEBUG            print("[DreamioVLC] addPlaybackSlave subtitle=\(URLRedactor.redactedURLString(candidate.url.absoluteString)) label=\(candidate.label) language=\(candidate.language ?? "unknown") ext=\(candidate.url.pathExtension.lowercased())")            logSubtitleTracks(reason: "after-addPlaybackSlave")#endif        }#if DEBUG        if !candidates.isEmpty {            print("[DreamioVLC] subtitle candidates=\(candidates.count) attached=\(attachedCount) duplicates=\(duplicateCount)")        }#endif        guard attachedCount > 0 else {1 unmodified line        }        [0.2, 0.6, 1.0, 2.0, 4.0].forEach { delay in            DispatchQueue.main.asyncAfter(deadline: .now() + delay) { [weak self] in                self?.selectInitialSubtitleTrackIfNeeded(reason: "delayed-refresh-\(String(format: "%.1f", delay))")#if DEBUG                self?.logSubtitleTracks(reason: "delayed-refresh-\(String(format: "%.1f", delay))")#endif                self?.onSubtitleTracksChange?()            }9 unmodified lines    }#endif
        private func selectInitialSubtitleTrackIfNeeded(reason: String) {        guard !didUserSelectSubtitleTrack,              !didAutoSelectSubtitleTrack,              mediaPlayer.currentVideoSubTitleIndex < 0,              let track = subtitleTracks.first(where: { $0.id >= 0 }) else {            return        }
            didAutoSelectSubtitleTrack = true        autoSelectedSubtitleTrackID = track.id#if DEBUG3 unmodified lines        scheduleAutoSubtitleSelectionReapply(trackID: track.id)    }
        private func scheduleAutoSubtitleSelectionReapply(trackID: Int32) {        [0.3, 1.0, 2.0, 4.0].forEach { delay in            DispatchQueue.main.asyncAfter(deadline: .now() + delay) { [weak self] in44 unmodified lines        case .paused, .stopped, .ended:            onStateChange?()        case .esAdded:            selectInitialSubtitleTrackIfNeeded(reason: "esAdded")#if DEBUG            logSubtitleTracks(reason: "esAdded")#endif25 unmodified lines262728293031323315 unmodified lines4950515253545556172 unmodified lines2292302312322332342352362372382392402412422432442452462472482492502512522531 unmodified line2552562572582592602612622632642652669 unmodified lines2762772782792802812822832842852862872882892902912922932942952962972982993003013023 unmodified lines30630730830931031131231331431531631731831932044 unmodified lines36536636736836937037125 unmodified lines    private var didAutoSelectSubtitleTrack = false    private var didUserSelectSubtitleTrack = false    private var autoSelectedSubtitleTrackID: Int32?    private var externalSubtitleBaselineTrackIDs = Set<Int32>()    private var hasPendingExternalSubtitleSelection = false
        override init() {        super.init()15 unmodified lines        didAutoSelectSubtitleTrack = false        didUserSelectSubtitleTrack = false        autoSelectedSubtitleTrackID = nil        externalSubtitleBaselineTrackIDs.removeAll()        hasPendingExternalSubtitleSelection = false        let media = VLCMedia(url: request.playbackURL)        let headerValue = request.headers            .map { "\($0.key): \($0.value)" }172 unmodified lines    private func attachSubtitles(_ candidates: [SubtitleCandidate]) -> Int {        var attachedCount = 0        var duplicateCount = 0        let baselineTrackIDs = Set(subtitleTracks.filter { $0.id >= 0 }.map(\.id))        candidates.forEach { candidate in            guard !attachedSubtitleURLs.contains(candidate.url) else {                duplicateCount += 1                return            }            attachedSubtitleURLs.insert(candidate.url)            externalSubtitleBaselineTrackIDs.formUnion(baselineTrackIDs)            hasPendingExternalSubtitleSelection = true            mediaPlayer.addPlaybackSlave(candidate.url, type: .subtitle, enforce: false)            attachedCount += 1#if DEBUG            print("[DreamioVLC] attach accepted subtitle=\(URLRedactor.redactedURLString(candidate.url.absoluteString)) label=\(candidate.label) language=\(candidate.language ?? "unknown") ext=\(candidate.url.pathExtension.lowercased()) visibleBefore=\(baselineTrackIDs.count)")            logSubtitleTracks(reason: "after-addPlaybackSlave")#endif        }#if DEBUG        if !candidates.isEmpty {            print("[DreamioVLC] subtitle candidates=\(candidates.count) attached=\(attachedCount) duplicates=\(duplicateCount) visible=\(subtitleTracks.filter { $0.id >= 0 }.count)")        }#endif        guard attachedCount > 0 else {1 unmodified line        }        [0.2, 0.6, 1.0, 2.0, 4.0].forEach { delay in            DispatchQueue.main.asyncAfter(deadline: .now() + delay) { [weak self] in                self?.selectPreferredSubtitleTrackIfNeeded(reason: "delayed-refresh-\(String(format: "%.1f", delay))")#if DEBUG                self?.logSubtitleTracks(reason: "delayed-refresh-\(String(format: "%.1f", delay))")                if delay == 4.0 {                    self?.logMissingExternalSubtitleTrackIfNeeded()                }#endif                self?.onSubtitleTracksChange?()            }9 unmodified lines    }#endif
        private func selectPreferredSubtitleTrackIfNeeded(reason: String) {        guard !didUserSelectSubtitleTrack else {            return        }
            if hasPendingExternalSubtitleSelection,           let externalTrack = subtitleTracks.first(where: { $0.id >= 0 && !externalSubtitleBaselineTrackIDs.contains($0.id) }) {            selectAutoSubtitleTrack(externalTrack, reason: "\(reason)-external")            hasPendingExternalSubtitleSelection = false            return        }
            guard !didAutoSelectSubtitleTrack,              mediaPlayer.currentVideoSubTitleIndex < 0,              let track = subtitleTracks.first(where: { $0.id >= 0 }) else {            return        }        selectAutoSubtitleTrack(track, reason: reason)    }
        private func selectAutoSubtitleTrack(_ track: SubtitleTrack, reason: String) {        didAutoSelectSubtitleTrack = true        autoSelectedSubtitleTrackID = track.id#if DEBUG3 unmodified lines        scheduleAutoSubtitleSelectionReapply(trackID: track.id)    }
    #if DEBUG    private func logMissingExternalSubtitleTrackIfNeeded() {        guard hasPendingExternalSubtitleSelection else {            return        }        print("[DreamioVLC] attach accepted but no new external subtitle track visible baseline=\(externalSubtitleBaselineTrackIDs.sorted()) visible=\(subtitleTracks.filter { $0.id >= 0 }.map(\.id))")    }#endif
        private func scheduleAutoSubtitleSelectionReapply(trackID: Int32) {        [0.3, 1.0, 2.0, 4.0].forEach { delay in            DispatchQueue.main.asyncAfter(deadline: .now() + delay) { [weak self] in44 unmodified lines        case .paused, .stopped, .ended:            onStateChange?()        case .esAdded:            selectPreferredSubtitleTrackIfNeeded(reason: "esAdded")#if DEBUG            logSubtitleTracks(reason: "esAdded")#endif

Tests/StreamResolverTests.swift
+73

    9 unmodified lines10111213141516137 unmodified lines15415515615715815919 unmodified lines1791801811821831849 unmodified lines        testPlaybackTimeFormatting()        testSubtitleCandidateParsing()        testOpenSubtitlesV3CandidateParsing()        testOpenSubtitlesV3DownloadResponseResolution()        await testSubtitleResolverDownloadJSONReturningLink()        await testSubtitleResolverRedirectToDirectSubtitle()        await testSubtitleResolverRejectsNonSubtitleAPIResponse()137 unmodified lines        assertEqual(candidates[3].url.absoluteString, "https://cdn.example.test/from-string.ass?source=opensubtitles")    }
        private static func testOpenSubtitlesV3DownloadResponseResolution() {        let payload = """        {19 unmodified lines        assertEqual(candidate?.language, "eng")    }
        private static func testSubtitleResolverDownloadJSONReturningLink() async {        MockURLProtocol.handlers = [            "https://api.opensubtitles.com/api/v1/download/123": (9 unmodified lines101112131415161718137 unmodified lines15615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419 unmodified lines2142152162172182192202212222232242252262272282292302312322332342352362372382392402412422432442452462472482492502512522532542552562579 unmodified lines        testPlaybackTimeFormatting()        testSubtitleCandidateParsing()        testOpenSubtitlesV3CandidateParsing()        testOpenSubtitlesNestedAttributesFilesParsing()        testOpenSubtitlesV3DownloadResponseResolution()        testOpenSubtitlesNestedDownloadResponseResolution()        await testSubtitleResolverDownloadJSONReturningLink()        await testSubtitleResolverRedirectToDirectSubtitle()        await testSubtitleResolverRejectsNonSubtitleAPIResponse()137 unmodified lines        assertEqual(candidates[3].url.absoluteString, "https://cdn.example.test/from-string.ass?source=opensubtitles")    }
        private static func testOpenSubtitlesNestedAttributesFilesParsing() {        let payload: [String: Any] = [            "data": [                [                    "attributes": [                        "language": "English",                        "file_name": "episode.en.srt",                        "files": [                            [                                "file_id": 12345,                                "file_name": "nested.en.srt"                            ],                            [                                "link": "https://dl.opensubtitles.org/en/download/nested.vtt?token=secret",                                "language": "eng"                            ]                        ]                    ]                ]            ]        ]
            let candidates = SubtitleCandidateParser.candidates(in: payload)
            assertEqual(candidates.count, 2)        assertEqual(candidates[0].url.absoluteString, "https://api.opensubtitles.com/api/v1/download/12345")        assertEqual(candidates[0].label, "nested.en.srt")        assertEqual(candidates[0].language, "English")        assertEqual(candidates[1].url.absoluteString, "https://dl.opensubtitles.org/en/download/nested.vtt?token=secret")        assertEqual(candidates[1].label, "eng")        assertEqual(candidates[1].language, "eng")    }
        private static func testOpenSubtitlesV3DownloadResponseResolution() {        let payload = """        {19 unmodified lines        assertEqual(candidate?.language, "eng")    }
        private static func testOpenSubtitlesNestedDownloadResponseResolution() {        let payload = """        {          "data": {            "attributes": {              "files": [                {                  "file_name": "ignored.txt",                  "link": "https://cdn.example.test/ignored.txt"                },                {                  "file_name": "episode.en.ass",                  "download": {                    "link": "https://dl.opensubtitles.org/en/download/episode.en.ass?token=secret"                  }                }              ]            }          }        }        """.data(using: .utf8)!        let original = SubtitleCandidate(            url: URL(string: "https://api.opensubtitles.com/api/v1/download/987")!,            label: "English SDH",            language: "eng"        )
            let candidate = SubtitleResolver.bestPlayableCandidate(            from: payload,            responseURL: original.url,            original: original        )
            assertEqual(candidate?.url.absoluteString, "https://dl.opensubtitles.org/en/download/episode.en.ass?token=secret")        assertEqual(candidate?.label, "English SDH")        assertEqual(candidate?.language, "eng")    }
        private static func testSubtitleResolverDownloadJSONReturningLink() async {        MockURLProtocol.handlers = [            "https://api.opensubtitles.com/api/v1/download/123": (

## Expected Impact for End-Users

When OpenSubtitles provides usable captions, the native captions menu should show external OpenSubtitles options in addition to `None` and embedded subtitle tracks. If an embedded track appears first, Dreamio can still switch to the external track automatically once VLC surfaces it, unless the user already made a manual caption choice.

## Validation

- Passed `swiftc Dreamio/StreamCandidate.swift Dreamio/StreamResolver.swift Tests/StreamResolverTests.swift -o /tmp/dreamio-stream-tests && /tmp/dreamio-stream-tests`.
- Passed `swiftc -typecheck Dreamio/StreamCandidate.swift Dreamio/StreamResolver.swift`.
- Passed `DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer xcodebuild -workspace Dreamio.xcworkspace -scheme Dreamio -configuration Debug -sdk iphonesimulator build`.
- Manual device validation was not performed in this turn. The next device run should verify that OpenSubtitles options appear in the captions menu for a title with OpenSubtitles enabled.

## Issues, Limitations, and Mitigations

- OpenSubtitles API download URLs may still require provider-specific authorization. This change preserves and resolves more candidate shapes, but it does not add new API credentials.
- VLC subtitle slave exposure is asynchronous and backend-dependent. Delayed refreshes and explicit missing-track logs make that timing easier to diagnose.
- The existing dirty Xcode workspace user-state file was present before this work and was not intentionally edited.

## Follow-up Work

- Run on a physical device with OpenSubtitles enabled and confirm the menu shows external OpenSubtitles tracks plus embedded tracks.
- If device logs still show accepted slaves with no visible VLC track, capture the URL shape and VLC track arrays from the new diagnostics.
- Consider exposing external track provenance in the menu if VLC track names remain too generic.