diff --git a/.agents/skills/liquid-glass-design/SKILL.md b/.agents/skills/liquid-glass-design/SKILL.md new file mode 100644 index 0000000..60551c2 --- /dev/null +++ b/.agents/skills/liquid-glass-design/SKILL.md @@ -0,0 +1,279 @@ +--- +name: liquid-glass-design +description: iOS 26 Liquid Glass design system — dynamic glass material with blur, reflection, and interactive morphing for SwiftUI, UIKit, and WidgetKit. +--- + +# Liquid Glass Design System (iOS 26) + +Patterns for implementing Apple's Liquid Glass — a dynamic material that blurs content behind it, reflects color and light from surrounding content, and reacts to touch and pointer interactions. Covers SwiftUI, UIKit, and WidgetKit integration. + +## When to Activate + +- Building or updating apps for iOS 26+ with the new design language +- Implementing glass-style buttons, cards, toolbars, or containers +- Creating morphing transitions between glass elements +- Applying Liquid Glass effects to widgets +- Migrating existing blur/material effects to the new Liquid Glass API + +## Core Pattern — SwiftUI + +### Basic Glass Effect + +The simplest way to add Liquid Glass to any view: + +```swift +Text("Hello, World!") + .font(.title) + .padding() + .glassEffect() // Default: regular variant, capsule shape +``` + +### Customizing Shape and Tint + +```swift +Text("Hello, World!") + .font(.title) + .padding() + .glassEffect(.regular.tint(.orange).interactive(), in: .rect(cornerRadius: 16.0)) +``` + +Key customization options: +- `.regular` — standard glass effect +- `.tint(Color)` — add color tint for prominence +- `.interactive()` — react to touch and pointer interactions +- Shape: `.capsule` (default), `.rect(cornerRadius:)`, `.circle` + +### Glass Button Styles + +```swift +Button("Click Me") { /* action */ } + .buttonStyle(.glass) + +Button("Important") { /* action */ } + .buttonStyle(.glassProminent) +``` + +### GlassEffectContainer for Multiple Elements + +Always wrap multiple glass views in a container for performance and morphing: + +```swift +GlassEffectContainer(spacing: 40.0) { + HStack(spacing: 40.0) { + Image(systemName: "scribble.variable") + .frame(width: 80.0, height: 80.0) + .font(.system(size: 36)) + .glassEffect() + + Image(systemName: "eraser.fill") + .frame(width: 80.0, height: 80.0) + .font(.system(size: 36)) + .glassEffect() + } +} +``` + +The `spacing` parameter controls merge distance — closer elements blend their glass shapes together. + +### Uniting Glass Effects + +Combine multiple views into a single glass shape with `glassEffectUnion`: + +```swift +@Namespace private var namespace + +GlassEffectContainer(spacing: 20.0) { + HStack(spacing: 20.0) { + ForEach(symbolSet.indices, id: \.self) { item in + Image(systemName: symbolSet[item]) + .frame(width: 80.0, height: 80.0) + .glassEffect() + .glassEffectUnion(id: item < 2 ? "group1" : "group2", namespace: namespace) + } + } +} +``` + +### Morphing Transitions + +Create smooth morphing when glass elements appear/disappear: + +```swift +@State private var isExpanded = false +@Namespace private var namespace + +GlassEffectContainer(spacing: 40.0) { + HStack(spacing: 40.0) { + Image(systemName: "scribble.variable") + .frame(width: 80.0, height: 80.0) + .glassEffect() + .glassEffectID("pencil", in: namespace) + + if isExpanded { + Image(systemName: "eraser.fill") + .frame(width: 80.0, height: 80.0) + .glassEffect() + .glassEffectID("eraser", in: namespace) + } + } +} + +Button("Toggle") { + withAnimation { isExpanded.toggle() } +} +.buttonStyle(.glass) +``` + +### Extending Horizontal Scrolling Under Sidebar + +To allow horizontal scroll content to extend under a sidebar or inspector, ensure the `ScrollView` content reaches the leading/trailing edges of the container. The system automatically handles the under-sidebar scrolling behavior when the layout extends to the edges — no additional modifier is needed. + +## Core Pattern — UIKit + +### Basic UIGlassEffect + +```swift +let glassEffect = UIGlassEffect() +glassEffect.tintColor = UIColor.systemBlue.withAlphaComponent(0.3) +glassEffect.isInteractive = true + +let visualEffectView = UIVisualEffectView(effect: glassEffect) +visualEffectView.translatesAutoresizingMaskIntoConstraints = false +visualEffectView.layer.cornerRadius = 20 +visualEffectView.clipsToBounds = true + +view.addSubview(visualEffectView) +NSLayoutConstraint.activate([ + visualEffectView.centerXAnchor.constraint(equalTo: view.centerXAnchor), + visualEffectView.centerYAnchor.constraint(equalTo: view.centerYAnchor), + visualEffectView.widthAnchor.constraint(equalToConstant: 200), + visualEffectView.heightAnchor.constraint(equalToConstant: 120) +]) + +// Add content to contentView +let label = UILabel() +label.text = "Liquid Glass" +label.translatesAutoresizingMaskIntoConstraints = false +visualEffectView.contentView.addSubview(label) +NSLayoutConstraint.activate([ + label.centerXAnchor.constraint(equalTo: visualEffectView.contentView.centerXAnchor), + label.centerYAnchor.constraint(equalTo: visualEffectView.contentView.centerYAnchor) +]) +``` + +### UIGlassContainerEffect for Multiple Elements + +```swift +let containerEffect = UIGlassContainerEffect() +containerEffect.spacing = 40.0 + +let containerView = UIVisualEffectView(effect: containerEffect) + +let firstGlass = UIVisualEffectView(effect: UIGlassEffect()) +let secondGlass = UIVisualEffectView(effect: UIGlassEffect()) + +containerView.contentView.addSubview(firstGlass) +containerView.contentView.addSubview(secondGlass) +``` + +### Scroll Edge Effects + +```swift +scrollView.topEdgeEffect.style = .automatic +scrollView.bottomEdgeEffect.style = .hard +scrollView.leftEdgeEffect.isHidden = true +``` + +### Toolbar Glass Integration + +```swift +let favoriteButton = UIBarButtonItem(image: UIImage(systemName: "heart"), style: .plain, target: self, action: #selector(favoriteAction)) +favoriteButton.hidesSharedBackground = true // Opt out of shared glass background +``` + +## Core Pattern — WidgetKit + +### Rendering Mode Detection + +```swift +struct MyWidgetView: View { + @Environment(\.widgetRenderingMode) var renderingMode + + var body: some View { + if renderingMode == .accented { + // Tinted mode: white-tinted, themed glass background + } else { + // Full color mode: standard appearance + } + } +} +``` + +### Accent Groups for Visual Hierarchy + +```swift +HStack { + VStack(alignment: .leading) { + Text("Title") + .widgetAccentable() // Accent group + Text("Subtitle") + // Primary group (default) + } + Image(systemName: "star.fill") + .widgetAccentable() // Accent group +} +``` + +### Image Rendering in Accented Mode + +```swift +Image("myImage") + .widgetAccentedRenderingMode(.monochrome) +``` + +### Container Background + +```swift +VStack { /* content */ } + .containerBackground(for: .widget) { + Color.blue.opacity(0.2) + } +``` + +## Key Design Decisions + +| Decision | Rationale | +|----------|-----------| +| GlassEffectContainer wrapping | Performance optimization, enables morphing between glass elements | +| `spacing` parameter | Controls merge distance — fine-tune how close elements must be to blend | +| `@Namespace` + `glassEffectID` | Enables smooth morphing transitions on view hierarchy changes | +| `interactive()` modifier | Explicit opt-in for touch/pointer reactions — not all glass should respond | +| UIGlassContainerEffect in UIKit | Same container pattern as SwiftUI for consistency | +| Accented rendering mode in widgets | System applies tinted glass when user selects tinted Home Screen | + +## Best Practices + +- **Always use GlassEffectContainer** when applying glass to multiple sibling views — it enables morphing and improves rendering performance +- **Apply `.glassEffect()` after** other appearance modifiers (frame, font, padding) +- **Use `.interactive()`** only on elements that respond to user interaction (buttons, toggleable items) +- **Choose spacing carefully** in containers to control when glass effects merge +- **Use `withAnimation`** when changing view hierarchies to enable smooth morphing transitions +- **Test across appearances** — light mode, dark mode, and accented/tinted modes +- **Ensure accessibility contrast** — text on glass must remain readable + +## Anti-Patterns to Avoid + +- Using multiple standalone `.glassEffect()` views without a GlassEffectContainer +- Nesting too many glass effects — degrades performance and visual clarity +- Applying glass to every view — reserve for interactive elements, toolbars, and cards +- Forgetting `clipsToBounds = true` in UIKit when using corner radii +- Ignoring accented rendering mode in widgets — breaks tinted Home Screen appearance +- Using opaque backgrounds behind glass — defeats the translucency effect + +## When to Use + +- Navigation bars, toolbars, and tab bars with the new iOS 26 design +- Floating action buttons and card-style containers +- Interactive controls that need visual depth and touch feedback +- Widgets that should integrate with the system's Liquid Glass appearance +- Morphing transitions between related UI states diff --git a/.beads/interactions.jsonl b/.beads/interactions.jsonl index df78ee8..7975793 100644 --- a/.beads/interactions.jsonl +++ b/.beads/interactions.jsonl @@ -13,3 +13,14 @@ {"id":"int-5d355e9b","kind":"field_change","created_at":"2026-05-25T09:51:17.04306Z","actor":"dirtydishes","issue_id":"dreamio-wgk","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Closed"}} {"id":"int-9ddb7b1a","kind":"field_change","created_at":"2026-05-25T10:18:30.826897Z","actor":"dirtydishes","issue_id":"dreamio-7w6","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Streamlined native player controls into a compact bottom overlay and validated the simulator build."}} {"id":"int-2a84633f","kind":"field_change","created_at":"2026-05-25T10:25:22.649574Z","actor":"dirtydishes","issue_id":"dreamio-88m","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Implemented captions as a single-choice menu with None and selected loaded tracks, updated tests and turn documentation."}} +{"id":"int-38a97132","kind":"field_change","created_at":"2026-05-25T10:43:21.805452Z","actor":"dirtydishes","issue_id":"dreamio-lw6","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Implemented late subtitle forwarding into active native playback, added VLC append path and parser tests."}} +{"id":"int-ddab585f","kind":"field_change","created_at":"2026-05-25T11:07:34.849628Z","actor":"dirtydishes","issue_id":"dreamio-8cz","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Hardened subtitle bridge network observers so non-text Stremio subtitle loads are not touched, and made parser traversal deterministic for metadata preservation."}} +{"id":"int-e07aeefe","kind":"field_change","created_at":"2026-05-25T13:50:43.373777Z","actor":"dirtydishes","issue_id":"dreamio-h5q","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Resolved OpenSubtitles V3 API-style subtitle download URLs to direct subtitle files before VLC attachment; added parser/resolver coverage and simulator build validation."}} +{"id":"int-c7246990","kind":"field_change","created_at":"2026-05-25T14:07:13.774172Z","actor":"dirtydishes","issue_id":"dreamio-e9p","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Added DEBUG-only subtitle pipeline proof logging and documented validation."}} +{"id":"int-45781aa3","kind":"field_change","created_at":"2026-05-25T14:19:19.141163Z","actor":"dirtydishes","issue_id":"dreamio-c1m","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Added DEBUG-only logs for captions menu actions and VLC subtitle selection results."}} +{"id":"int-6343b773","kind":"field_change","created_at":"2026-05-25T14:25:59.50764Z","actor":"dirtydishes","issue_id":"dreamio-bd9","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Stopped rebuilding the captions menu on every progress refresh and validated the build."}} +{"id":"int-26b872a1","kind":"field_change","created_at":"2026-05-25T14:31:46.83464Z","actor":"dirtydishes","issue_id":"dreamio-ese","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Added subtitle-shaped fetch/XHR inspection diagnostics and validated the build."}} +{"id":"int-4e095d3f","kind":"field_change","created_at":"2026-05-25T14:38:21.968713Z","actor":"dirtydishes","issue_id":"dreamio-djc","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Auto-select the first discovered VLC subtitle track when playback is still disabled, while preserving manual caption choices."}} +{"id":"int-96629c65","kind":"field_change","created_at":"2026-05-25T14:45:38.521113Z","actor":"dirtydishes","issue_id":"dreamio-ppj","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Re-applied the auto-selected VLC subtitle track after stream discovery and playback state changes to harden rendering timing."}} +{"id":"int-027cec57","kind":"field_change","created_at":"2026-05-25T14:51:44.599319Z","actor":"dirtydishes","issue_id":"dreamio-3xi","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Captured OpenSubtitles V3 subtitle URLs from browser track elements and textTracks so they can be forwarded to native playback."}} +{"id":"int-8f943c34","kind":"field_change","created_at":"2026-05-25T15:01:35.610049Z","actor":"dirtydishes","issue_id":"dreamio-bao","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"Implemented native audio track discovery and selection with a far-left audio menu in the VLC-backed player."}} diff --git a/.beads/issues.jsonl b/.beads/issues.jsonl index 5ad5342..6445066 100644 --- a/.beads/issues.jsonl +++ b/.beads/issues.jsonl @@ -1,3 +1,12 @@ +{"_type":"issue","id":"dreamio-8cz","title":"fix stremio external subtitle loading regression","description":"After adding late subtitle forwarding for native playback, Stremio external subtitle loading is failing. Investigate the injected bridge and native subtitle forwarding path, then adjust behavior so Stremio can still load external subtitles while native playback receives late candidates.","status":"closed","priority":0,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T11:05:42Z","created_by":"dirtydishes","updated_at":"2026-05-25T11:07:35Z","started_at":"2026-05-25T11:05:55Z","closed_at":"2026-05-25T11:07:35Z","close_reason":"Hardened subtitle bridge network observers so non-text Stremio subtitle loads are not touched, and made parser traversal deterministic for metadata preservation.","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"dreamio-bao","title":"add native player audio track selection","description":"Add audio track discovery and selection to the native VLC-backed player so multi-language files can be filtered from the player controls.","status":"closed","priority":1,"issue_type":"feature","owner":"dishes@dpdrm.com","created_at":"2026-05-25T14:57:14Z","created_by":"dirtydishes","updated_at":"2026-05-25T15:01:36Z","closed_at":"2026-05-25T15:01:36Z","close_reason":"Implemented native audio track discovery and selection with a far-left audio menu in the VLC-backed player.","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"dreamio-3xi","title":"Capture browser text tracks for OpenSubtitles V3","description":"OpenSubtitles V3 subtitles can be attached to the Stremio web player as HTML track/textTrack entries rather than appearing in the initial stream candidate. Extend the web bridge to inspect track elements and textTracks so external subtitles can be forwarded to native playback.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T14:49:50Z","created_by":"dirtydishes","updated_at":"2026-05-25T14:51:45Z","started_at":"2026-05-25T14:49:52Z","closed_at":"2026-05-25T14:51:45Z","close_reason":"Captured OpenSubtitles V3 subtitle URLs from browser track elements and textTracks so they can be forwarded to native playback.","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"dreamio-ppj","title":"Reapply VLC embedded subtitle selection after track discovery","description":"Device logs show VLC eventually exposes and selects the embedded English SDH subtitle track, but subtitles still do not render. Investigate and harden the VLC selection timing so embedded tracks are selected after discovery is stable.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T14:44:08Z","created_by":"dirtydishes","updated_at":"2026-05-25T14:45:38Z","started_at":"2026-05-25T14:44:18Z","closed_at":"2026-05-25T14:45:38Z","close_reason":"Re-applied the auto-selected VLC subtitle track after stream discovery and playback state changes to harden rendering timing.","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"dreamio-djc","title":"Auto-select embedded VLC subtitle tracks","description":"VLC discovers embedded MKV subtitle tracks after playback starts, but Dreamio leaves subtitles disabled when no external candidates were provided. Add automatic selection for the first selectable VLC subtitle track while preserving manual caption choices.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T14:36:11Z","created_by":"dirtydishes","updated_at":"2026-05-25T14:38:22Z","started_at":"2026-05-25T14:36:17Z","closed_at":"2026-05-25T14:38:22Z","close_reason":"Auto-select the first discovered VLC subtitle track when playback is still disabled, while preserving manual caption choices.","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"dreamio-ese","title":"Discover Stremio external subtitle payloads","description":"Extend and instrument the injected web subtitle discovery path so Stremio/OpenSubtitles addon responses can be captured when native playback only sees embedded VLC subtitle tracks.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T14:29:57Z","created_by":"dirtydishes","updated_at":"2026-05-25T14:31:47Z","started_at":"2026-05-25T14:30:03Z","closed_at":"2026-05-25T14:31:47Z","close_reason":"Added subtitle-shaped fetch/XHR inspection diagnostics and validated the build.","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"dreamio-bd9","title":"Stabilize captions menu refresh","description":"Stop rebuilding the captions UIMenu on every playback progress refresh so embedded subtitle actions can remain stable long enough to fire, while keeping DEBUG logs for menu state and selection.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T14:24:45Z","created_by":"dirtydishes","updated_at":"2026-05-25T14:25:59Z","started_at":"2026-05-25T14:24:50Z","closed_at":"2026-05-25T14:25:59Z","close_reason":"Stopped rebuilding the captions menu on every progress refresh and validated the build.","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"dreamio-h5q","title":"Resolve OpenSubtitles API subtitle URLs before VLC attachment","description":"OpenSubtitles V3 can surface API/download endpoints that are not subtitle files themselves. Dreamio should resolve those endpoints to playable subtitle file URLs before handing them to VLC so Stremio does not show failed subtitle loads after native playback opens.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T13:47:17Z","created_by":"dirtydishes","updated_at":"2026-05-25T13:50:43Z","started_at":"2026-05-25T13:47:21Z","closed_at":"2026-05-25T13:50:43Z","close_reason":"Resolved OpenSubtitles V3 API-style subtitle download URLs to direct subtitle files before VLC attachment; added parser/resolver coverage and simulator build validation.","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"dreamio-lw6","title":"forward late opensubtitles subtitles to native player","description":"Native playback only receives subtitle candidates discovered before the stream candidate is posted. OpenSubtitles V3 candidates can arrive later through addon/network responses, so the active native player needs an append path for newly discovered external subtitles.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T10:40:28Z","created_by":"dirtydishes","updated_at":"2026-05-25T10:43:22Z","started_at":"2026-05-25T10:40:36Z","closed_at":"2026-05-25T10:43:22Z","close_reason":"Implemented late subtitle forwarding into active native playback, added VLC append path and parser tests.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"dreamio-poo","title":"Native player controls captions and close flow","description":"Add and validate VLC-backed native playback transport controls, subtitle track controls, external subtitle discovery, and Stremio Web close cleanup after native playback dismisses.","status":"closed","priority":1,"issue_type":"feature","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T09:47:56Z","created_by":"dirtydishes","updated_at":"2026-05-25T09:49:40Z","started_at":"2026-05-25T09:48:00Z","closed_at":"2026-05-25T09:49:40Z","close_reason":"Implemented and validated native player controls, subtitle handling refinements, and close-flow cleanup.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"dreamio-wgk","title":"Fix native player controls tap-to-show","description":"Native player controls can be hidden by tapping, but subsequent taps on the player do not bring them back. Investigate the overlay gesture handling and restore reliable tap-to-show/tap-to-hide behavior.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T09:27:58Z","created_by":"dirtydishes","updated_at":"2026-05-25T09:51:17Z","started_at":"2026-05-25T09:28:11Z","closed_at":"2026-05-25T09:51:17Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"dreamio-ija","title":"Fix MobileVLCKit linker dependency","description":"Dreamio fails to link because the MobileVLCKit framework is not found. Investigate how the dependency is configured and update the repository so the framework is available to Xcode builds.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T04:40:49Z","created_by":"dirtydishes","updated_at":"2026-05-25T04:44:36Z","started_at":"2026-05-25T04:40:57Z","closed_at":"2026-05-25T04:44:36Z","close_reason":"Fixed MobileVLCKit linker failures by preparing the XCFramework slice before app linking and preserving the integration through pod install.","dependency_count":0,"dependent_count":0,"comment_count":0} @@ -8,6 +17,8 @@ {"_type":"issue","id":"dreamio-l68","title":"Add native playback for direct debrid streams","description":"Implement a WKWebView JavaScript bridge that detects direct-file debrid media URLs and routes unsupported containers to a native player backend, initially MobileVLCKit, while preserving normal Stremio Web playback for compatible streams.","status":"closed","priority":1,"issue_type":"feature","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T03:13:19Z","created_by":"dirtydishes","updated_at":"2026-05-25T03:20:17Z","started_at":"2026-05-25T03:13:28Z","closed_at":"2026-05-25T03:20:17Z","close_reason":"Implemented native direct-stream bridge, classification, MobileVLCKit backend wiring, CocoaPods workflow docs, and turn documentation. Full iOS build is blocked locally by missing CocoaPods and iPhoneOS SDK.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"dreamio-tnv","title":"Fix iOS bundle identifier install failure","description":"Xcode built Dreamio.app without a valid CFBundleIdentifier, causing device install to fail with CoreDeviceError 3000/3002. Investigate project bundle settings, fix the source configuration, validate the app bundle Info.plist, and document the change.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T01:23:00Z","created_by":"dirtydishes","updated_at":"2026-05-25T01:25:36Z","started_at":"2026-05-25T01:23:07Z","closed_at":"2026-05-25T01:25:36Z","close_reason":"Added bundle metadata to Info.plist and validated processed app bundle identifier.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"dreamio-4yn","title":"Build WKWebView MVP shell","description":"Create the first Dreamio MVP implementation: a minimal iOS WKWebView wrapper around hosted Stremio Web, with configuration, launch behavior, diagnostics, and documentation for real-device viability testing.","acceptance_criteria":"App project exists; WKWebView loads hosted Stremio Web; external/new-window navigation is handled; basic diagnostics and manual test documentation exist; quality gates are run or documented.","status":"closed","priority":1,"issue_type":"feature","owner":"dishes@dpdrm.com","created_at":"2026-05-24T14:55:12Z","created_by":"dirtydishes","updated_at":"2026-05-24T14:59:44Z","closed_at":"2026-05-24T14:59:44Z","close_reason":"Implemented the MVP WKWebView iOS shell, added run and validation documentation, and recorded current validation limits.","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"dreamio-c1m","title":"Add captions selection proof logging","description":"Add DEBUG-only logs around the native captions menu and VLC subtitle selection path so subtitle tap actions prove whether the UI fires and whether VLC accepts the selected embedded track index.","status":"closed","priority":2,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T14:18:06Z","created_by":"dirtydishes","updated_at":"2026-05-25T14:19:19Z","started_at":"2026-05-25T14:18:11Z","closed_at":"2026-05-25T14:19:19Z","close_reason":"Added DEBUG-only logs for captions menu actions and VLC subtitle selection results.","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"dreamio-e9p","title":"Add native subtitle pipeline proof logging","description":"Add DEBUG-only logs across the web bridge, native player, subtitle resolution, and VLC attachment points so the next Xcode run can identify where external subtitles disappear without changing playback behavior.","status":"closed","priority":2,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T14:03:18Z","created_by":"dirtydishes","updated_at":"2026-05-25T14:07:14Z","started_at":"2026-05-25T14:03:22Z","closed_at":"2026-05-25T14:07:14Z","close_reason":"Added DEBUG-only subtitle pipeline proof logging and documented validation.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"dreamio-88m","title":"Make caption selection states clearer","description":"The native player caption menu should behave like a simple single-choice menu with None and loaded caption tracks, making the current caption state visually obvious.","status":"closed","priority":2,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T10:22:12Z","created_by":"dirtydishes","updated_at":"2026-05-25T10:25:23Z","started_at":"2026-05-25T10:22:48Z","closed_at":"2026-05-25T10:25:23Z","close_reason":"Implemented captions as a single-choice menu with None and selected loaded tracks, updated tests and turn documentation.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"dreamio-7w6","title":"Streamline native player controls","description":"Make the native playback controls take up less screen space while preserving play, seek, jump, captions, and close actions.","status":"closed","priority":2,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T10:15:49Z","created_by":"dirtydishes","updated_at":"2026-05-25T10:18:31Z","started_at":"2026-05-25T10:15:59Z","closed_at":"2026-05-25T10:18:31Z","close_reason":"Streamlined native player controls into a compact bottom overlay and validated the simulator build.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"dreamio-mj8","title":"Add native player controls and captions","description":"Implement a fuller VLC-backed native playback surface with transport controls, caption controls, external subtitle discovery, and a clean close flow back to Stremio episode selection.","status":"closed","priority":2,"issue_type":"feature","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T04:57:53Z","created_by":"dirtydishes","updated_at":"2026-05-25T05:04:55Z","started_at":"2026-05-25T04:57:57Z","closed_at":"2026-05-25T05:04:55Z","close_reason":"Implemented native VLC player controls, caption controls, subtitle candidate discovery, and close-flow cleanup.","dependency_count":0,"dependent_count":0,"comment_count":0} diff --git a/Dreamio.xcworkspace/xcuserdata/kell.xcuserdatad/UserInterfaceState.xcuserstate b/Dreamio.xcworkspace/xcuserdata/kell.xcuserdatad/UserInterfaceState.xcuserstate index d9f17eb..dfff5d4 100644 Binary files a/Dreamio.xcworkspace/xcuserdata/kell.xcuserdatad/UserInterfaceState.xcuserstate and b/Dreamio.xcworkspace/xcuserdata/kell.xcuserdatad/UserInterfaceState.xcuserstate differ diff --git a/Dreamio/DreamioWebViewController.swift b/Dreamio/DreamioWebViewController.swift index 301ef99..53d417f 100644 --- a/Dreamio/DreamioWebViewController.swift +++ b/Dreamio/DreamioWebViewController.swift @@ -6,6 +6,7 @@ final class DreamioWebViewController: UIViewController { static let stremioWebURL = URL(string: "https://web.stremio.com/")! static let diagnosticsMessageHandler = "dreamioDiagnostics" static let streamCandidateMessageHandler = "dreamioStreamCandidate" + static let subtitleCandidateMessageHandler = "dreamioSubtitleCandidate" } private lazy var webView: WKWebView = { @@ -18,6 +19,10 @@ final class DreamioWebViewController: UIViewController { WeakScriptMessageHandler(delegate: self), name: Constants.streamCandidateMessageHandler ) + configuration.userContentController.add( + WeakScriptMessageHandler(delegate: self), + name: Constants.subtitleCandidateMessageHandler + ) configuration.userContentController.addUserScript(Self.streamCandidateScript) #if DEBUG configuration.userContentController.add( @@ -52,6 +57,7 @@ final class DreamioWebViewController: UIViewController { private var progressObservation: NSKeyValueObservation? private var userAgent: String? private var lastNativePlaybackURL: URL? + private weak var currentNativePlayer: NativePlayerViewController? private let streamResolver: StreamResolving = StremioStreamResolver() private static let streamCandidateScript = WKUserScript( @@ -73,7 +79,9 @@ final class DreamioWebViewController: UIViewController { /\.mp4(?:[?#]|$)/i ]; const subtitleCandidates = []; + const postedSubtitleURLs = new Set(); const subtitleURLPattern = /https?:\/\/[^\s"'<>]+(?:\.srt|\.vtt|\.ass|\.ssa|\.sub|opensubtitles|subtitle)[^\s"'<>]*/ig; + const subtitleSignalPattern = /subtitle|subtitles|opensubtitles|vtt|srt|ass|ssa/i; const looksNative = (url) => { if (!url || typeof url !== "string") { @@ -119,8 +127,59 @@ final class DreamioWebViewController: UIViewController { } catch (_) {} }; + const postSubtitleCandidates = (candidates, debug = {}) => { + const discoveredCount = candidates.length; + const fresh = candidates.filter((candidate) => { + if (postedSubtitleURLs.has(candidate.url)) { + return false; + } + postedSubtitleURLs.add(candidate.url); + return true; + }); + if (fresh.length === 0) { + try { + window.webkit.messageHandlers.dreamioSubtitleCandidate.postMessage({ + pageUrl: window.location.href, + subtitles: [], + debug: { + discovered: discoveredCount, + deduped: 0, + forwarded: 0, + ...debug + } + }); + } catch (_) {} + return; + } + try { + window.webkit.messageHandlers.dreamioSubtitleCandidate.postMessage({ + pageUrl: window.location.href, + subtitles: fresh, + debug: { + discovered: discoveredCount, + deduped: fresh.length, + forwarded: fresh.length, + ...debug + } + }); + } catch (_) {} + }; + const addSubtitleCandidate = (entry) => { - const rawURL = typeof entry === "string" ? entry : entry && (entry.url || entry.href || entry.src || entry.file || entry.download); + const rawURL = typeof entry === "string" + ? entry + : entry && ( + entry.url || + entry.href || + entry.src || + entry.link || + entry.file || + entry.download || + entry.externalUrl || + entry.externalURL || + entry.fileUrl || + entry.fileURL + ); const url = absoluteURL(rawURL); subtitleURLPattern.lastIndex = 0; if (!url || !subtitleURLPattern.test(url)) { @@ -131,10 +190,56 @@ final class DreamioWebViewController: UIViewController { if (subtitleCandidates.some((candidate) => candidate.url === url)) { return; } - subtitleCandidates.push({ + const candidate = { url, label: entry && (entry.label || entry.name || entry.title || entry.lang || entry.language) || "External Subtitle", language: entry && (entry.lang || entry.language) || "" + }; + subtitleCandidates.push(candidate); + postSubtitleCandidates([candidate]); + }; + + const inspectTrack = (track) => { + if (!track) { + return; + } + if (track instanceof HTMLTrackElement) { + addSubtitleCandidate({ + url: track.src || track.getAttribute("src") || "", + label: track.label || track.srclang || "External Subtitle", + language: track.srclang || "" + }); + return; + } + const source = track.src || track.url || ""; + if (source) { + addSubtitleCandidate({ + url: source, + label: track.label || track.language || track.kind || "External Subtitle", + language: track.language || "" + }); + } + }; + + const inspectTextTracks = (media) => { + try { + Array.from(media.textTracks || []).forEach(inspectTrack); + } catch (_) {} + try { + media.querySelectorAll("track").forEach(inspectTrack); + } catch (_) {} + }; + + const postSubtitleInspection = (source, url, beforeCount, afterCount, payloadLength) => { + if (afterCount > beforeCount) { + return; + } + postSubtitleCandidates([], { + source, + inspected: true, + url: url || "", + payloadLength: payloadLength || 0, + totalKnown: subtitleCandidates.length }); }; @@ -161,12 +266,30 @@ final class DreamioWebViewController: UIViewController { } }; + const inspectSubtitleText = (source, url, text) => { + const beforeCount = subtitleCandidates.length; + inspectSubtitlePayload(text); + postSubtitleInspection(source, url, beforeCount, subtitleCandidates.length, text ? text.length : 0); + }; + const originalFetch = window.fetch; if (originalFetch) { window.fetch = async (...args) => { const response = await originalFetch(...args); try { - response.clone().text().then(inspectSubtitlePayload).catch(() => {}); + const contentType = response.headers && response.headers.get("content-type") || ""; + const url = response.url || ""; + subtitleURLPattern.lastIndex = 0; + const shouldInspect = !contentType + || /json|text|javascript|xml|subtitle|vtt|srt/i.test(contentType) + || subtitleURLPattern.test(url) + || subtitleSignalPattern.test(url); + if (shouldInspect) { + subtitleURLPattern.lastIndex = 0; + response.clone().text().then((text) => { + inspectSubtitleText("fetch", url, text); + }).catch(() => {}); + } } catch (_) {} return response; }; @@ -175,7 +298,21 @@ final class DreamioWebViewController: UIViewController { const originalXHRSend = XMLHttpRequest.prototype.send; XMLHttpRequest.prototype.send = function(...args) { try { - this.addEventListener("load", () => inspectSubtitlePayload(this.responseText)); + this.addEventListener("load", () => { + try { + const responseType = this.responseType || ""; + if (responseType && responseType !== "text") { + return; + } + const url = this.responseURL || ""; + const text = this.responseText || ""; + if (subtitleSignalPattern.test(url) || subtitleSignalPattern.test(text)) { + inspectSubtitleText("xhr", url, text); + } else { + inspectSubtitlePayload(text); + } + } catch (_) {} + }); } catch (_) {} return originalXHRSend.apply(this, args); }; @@ -201,11 +338,17 @@ final class DreamioWebViewController: UIViewController { if (!node) { return; } + if (node instanceof HTMLTrackElement) { + inspectTrack(node); + } if (node instanceof HTMLVideoElement || node instanceof HTMLSourceElement) { postCandidate(node.currentSrc || node.src || node.getAttribute("src"), node); } if (node.querySelectorAll) { - node.querySelectorAll("video, source").forEach(inspectMedia); + node.querySelectorAll("video, source, track").forEach(inspectMedia); + } + if (node instanceof HTMLVideoElement) { + inspectTextTracks(node); } }; @@ -231,10 +374,34 @@ final class DreamioWebViewController: UIViewController { }); } + const trackSrcDescriptor = Object.getOwnPropertyDescriptor(HTMLTrackElement.prototype, "src"); + if (trackSrcDescriptor && trackSrcDescriptor.set) { + Object.defineProperty(HTMLTrackElement.prototype, "src", { + get: trackSrcDescriptor.get, + set(value) { + addSubtitleCandidate({ + url: value, + label: this.label || this.srclang || "External Subtitle", + language: this.srclang || "" + }); + return trackSrcDescriptor.set.call(this, value); + } + }); + } + const originalSetAttribute = Element.prototype.setAttribute; Element.prototype.setAttribute = function(name, value) { - if (String(name).toLowerCase() === "src" && (this instanceof HTMLVideoElement || this instanceof HTMLSourceElement)) { - postCandidate(value, this); + if (String(name).toLowerCase() === "src") { + if (this instanceof HTMLVideoElement || this instanceof HTMLSourceElement) { + postCandidate(value, this); + } + if (this instanceof HTMLTrackElement) { + addSubtitleCandidate({ + url: value, + label: this.label || this.srclang || "External Subtitle", + language: this.srclang || "" + }); + } } return originalSetAttribute.call(this, name, value); }; @@ -243,9 +410,13 @@ final class DreamioWebViewController: UIViewController { HTMLMediaElement.prototype.load = function() { inspectMedia(this); this.querySelectorAll("source").forEach(inspectMedia); + inspectTextTracks(this); return originalLoad.call(this); }; + document.addEventListener("addtrack", (event) => { + inspectTrack(event.track || event.target); + }, true); document.addEventListener("loadedmetadata", (event) => inspectMedia(event.target), true); document.addEventListener("error", (event) => inspectMedia(event.target), true); new MutationObserver((mutations) => { @@ -259,7 +430,7 @@ final class DreamioWebViewController: UIViewController { childList: true, subtree: true, attributes: true, - attributeFilter: ["src"] + attributeFilter: ["src", "label", "srclang"] }); inspectMedia(document); @@ -422,7 +593,7 @@ final class DreamioWebViewController: UIViewController { #if DEBUG let classification = request.classification - print("[DreamioStream] class=\(classification.sourceKind.rawValue) container=\(classification.containerGuess.rawValue) reason=\(classification.reason) observed=\(classification.sanitizedObservedURL) resolver=\(classification.sanitizedResolverURL ?? "none")") + print("[DreamioStream] class=\(classification.sourceKind.rawValue) container=\(classification.containerGuess.rawValue) reason=\(classification.reason) subtitles=\(request.subtitleCandidates.count) observed=\(classification.sanitizedObservedURL) resolver=\(classification.sanitizedResolverURL ?? "none")") #endif Task { [weak self] in @@ -430,6 +601,27 @@ final class DreamioWebViewController: UIViewController { } } + private func handleSubtitleCandidates(_ candidates: [SubtitleCandidate]) { + guard !candidates.isEmpty else { + return + } + +#if DEBUG + print("[DreamioSubtitles] native discovered=\(candidates.count) playerActive=\(currentNativePlayer != nil) candidates=\(SubtitleDebugFormatter.candidateSummary(candidates))") +#endif + guard let currentNativePlayer else { +#if DEBUG + print("[DreamioSubtitles] discovered=\(candidates.count) forwarded=0 reason=no-active-native-player") +#endif + return + } + + let forwarded = currentNativePlayer.addSubtitleCandidates(candidates) +#if DEBUG + print("[DreamioSubtitles] discovered=\(candidates.count) forwarded=\(forwarded) reason=active-native-player") +#endif + } + @MainActor private func resolveAndPresentNativePlayback(_ request: NativePlaybackRequest) async { guard VLCNativePlaybackBackend.isAvailable else { @@ -455,8 +647,10 @@ final class DreamioWebViewController: UIViewController { subtitleCandidates: request.subtitleCandidates ) let player = NativePlayerViewController(request: resolvedRequest) + currentNativePlayer = player player.onDismiss = { [weak self] in self?.lastNativePlaybackURL = nil + self?.currentNativePlayer = nil self?.cleanUpStremioPlayerAfterNativeDismiss() } present(player, animated: true) @@ -593,6 +787,21 @@ final class DreamioWebViewController: UIViewController { private func redactedURLString(_ value: String) -> String { URLRedactor.redactedURLString(value) } + + private func logSubtitleBridgeMessage(_ body: Any, parsedCandidates: [SubtitleCandidate]) { + let dictionary = body as? [String: Any] + let debug = dictionary?["debug"] as? [String: Any] + let discovered = debug?["discovered"] as? Int ?? parsedCandidates.count + let deduped = debug?["deduped"] as? Int ?? parsedCandidates.count + let posted = debug?["forwarded"] as? Int ?? parsedCandidates.count + let source = debug?["source"] as? String ?? "bridge" + let inspected = debug?["inspected"] as? Bool ?? false + let inspectedURL = (debug?["url"] as? String).map(redactedURLString) ?? "none" + let payloadLength = debug?["payloadLength"] as? Int ?? 0 + let totalKnown = debug?["totalKnown"] as? Int ?? parsedCandidates.count + let pageURL = dictionary?["pageUrl"] as? String + print("[DreamioSubtitles] bridge source=\(source) inspected=\(inspected) discovered=\(discovered) deduped=\(deduped) posted=\(posted) parsed=\(parsedCandidates.count) totalKnown=\(totalKnown) payloadLength=\(payloadLength) playerActive=\(currentNativePlayer != nil) inspectedURL=\(inspectedURL) page=\(pageURL.map(redactedURLString) ?? "unknown") candidates=\(SubtitleDebugFormatter.candidateSummary(parsedCandidates))") + } #endif } @@ -669,6 +878,15 @@ extension DreamioWebViewController: WKScriptMessageHandler { return } + if message.name == Constants.subtitleCandidateMessageHandler { + let candidates = SubtitleCandidateParser.candidates(in: message.body) +#if DEBUG + logSubtitleBridgeMessage(message.body, parsedCandidates: candidates) +#endif + handleSubtitleCandidates(candidates) + return + } + #if DEBUG guard message.name == Constants.diagnosticsMessageHandler, let body = message.body as? [String: Any], diff --git a/Dreamio/NativePlaybackBackend.swift b/Dreamio/NativePlaybackBackend.swift index 57ec708..4f278a0 100644 --- a/Dreamio/NativePlaybackBackend.swift +++ b/Dreamio/NativePlaybackBackend.swift @@ -6,12 +6,15 @@ protocol NativePlaybackBackend: AnyObject { var onFailure: ((Error) -> Void)? { get set } var onStateChange: (() -> Void)? { get set } var onSubtitleTracksChange: (() -> Void)? { get set } + var onAudioTracksChange: (() -> Void)? { get set } var isPlaying: Bool { get } var isSeekable: Bool { get } var duration: TimeInterval { get } var currentTime: TimeInterval { get } var remainingTime: TimeInterval { get } var position: Float { get } + var audioTracks: [AudioTrack] { get } + var selectedAudioTrackID: Int32 { get } var subtitleTracks: [SubtitleTrack] { get } var selectedSubtitleTrackID: Int32 { get } var subtitleDelay: TimeInterval { get } @@ -23,11 +26,18 @@ protocol NativePlaybackBackend: AnyObject { func togglePlayPause() func seek(to position: Float) func jump(by seconds: TimeInterval) + func selectAudioTrack(id: Int32) func selectSubtitleTrack(id: Int32) func adjustSubtitleDelay(by seconds: TimeInterval) + @discardableResult + func addSubtitleCandidates(_ candidates: [SubtitleCandidate]) -> Int func stop() } +protocol SubtitleResolving { + func resolve(_ candidate: SubtitleCandidate) async -> SubtitleCandidate? +} + enum NativePlaybackError: LocalizedError { case backendUnavailable case startupTimedOut diff --git a/Dreamio/NativePlayerViewController.swift b/Dreamio/NativePlayerViewController.swift index 6c30810..1679f1f 100644 --- a/Dreamio/NativePlayerViewController.swift +++ b/Dreamio/NativePlayerViewController.swift @@ -3,10 +3,14 @@ import UIKit final class NativePlayerViewController: UIViewController { private let request: NativePlaybackRequest private var backend: NativePlaybackBackend + private let subtitleResolver: SubtitleResolving private var startupTimer: Timer? private var controlsTimer: Timer? private var progressTimer: Timer? private var isScrubbing = false + private var attachedSubtitleURLs: Set + private var audioMenuSignature: String? + private var captionsMenuSignature: String? var onDismiss: (() -> Void)? private let loadingView: UIActivityIndicatorView = { @@ -31,8 +35,11 @@ final class NativePlayerViewController: UIViewController { private let controlsContainer: UIVisualEffectView = { let view = UIVisualEffectView(effect: UIBlurEffect(style: .systemUltraThinMaterialDark)) view.translatesAutoresizingMaskIntoConstraints = false - view.layer.cornerRadius = 16 + view.layer.cornerRadius = 22 view.clipsToBounds = true + view.backgroundColor = UIColor.white.withAlphaComponent(0.08) + view.layer.borderColor = UIColor.white.withAlphaComponent(0.18).cgColor + view.layer.borderWidth = 1 return view }() @@ -46,6 +53,7 @@ final class NativePlayerViewController: UIViewController { private let playPauseButton = NativePlayerViewController.iconButton(systemName: "pause.fill", label: "Play or Pause") private let jumpBackButton = NativePlayerViewController.iconButton(systemName: "gobackward.15", label: "Jump Back 15 Seconds") private let jumpForwardButton = NativePlayerViewController.iconButton(systemName: "goforward.15", label: "Jump Forward 15 Seconds") + private let audioButton = NativePlayerViewController.iconButton(systemName: "waveform.circle", label: "Audio Tracks") private let captionsButton = NativePlayerViewController.iconButton(systemName: "captions.bubble", label: "Captions") private let elapsedLabel: UILabel = { @@ -91,9 +99,15 @@ final class NativePlayerViewController: UIViewController { return label }() - init(request: NativePlaybackRequest, backend: NativePlaybackBackend = VLCNativePlaybackBackend()) { + init( + request: NativePlaybackRequest, + backend: NativePlaybackBackend = VLCNativePlaybackBackend(), + subtitleResolver: SubtitleResolving = SubtitleResolver() + ) { self.request = request self.backend = backend + self.subtitleResolver = subtitleResolver + self.attachedSubtitleURLs = [] super.init(nibName: nil, bundle: nil) modalPresentationStyle = .fullScreen modalTransitionStyle = .crossDissolve @@ -124,6 +138,52 @@ final class NativePlayerViewController: UIViewController { configureLayout() startStartupTimer() backend.play(request: request) + addSubtitleCandidates(request.subtitleCandidates) + } + + @discardableResult + func addSubtitleCandidates(_ candidates: [SubtitleCandidate]) -> Int { + let pendingCandidates = candidates.filter { !attachedSubtitleURLs.contains($0.url) } + guard !pendingCandidates.isEmpty else { +#if DEBUG + print("[DreamioNativePlayer] subtitle candidates=\(candidates.count) pending=0 duplicates=\(candidates.count) resolved=0 attached=0 tracks=\(SubtitleDebugFormatter.trackSummary(backend.subtitleTracks)) selected=\(backend.selectedSubtitleTrackID)") +#endif + return 0 + } + + pendingCandidates.forEach { attachedSubtitleURLs.insert($0.url) } + + Task { [weak self] in + guard let self else { + return + } + let resolvedCandidates = await self.resolveSubtitleCandidates(pendingCandidates) + await MainActor.run { + guard !resolvedCandidates.isEmpty else { +#if DEBUG + print("[DreamioNativePlayer] subtitle candidates=\(candidates.count) pending=\(pendingCandidates.count) resolved=0 attached=0 tracks=\(SubtitleDebugFormatter.trackSummary(self.backend.subtitleTracks)) selected=\(self.backend.selectedSubtitleTrackID) candidates=\(SubtitleDebugFormatter.candidateSummary(pendingCandidates))") +#endif + return + } + let attachableCandidates = resolvedCandidates.filter { candidate in + guard !self.attachedSubtitleURLs.contains(candidate.url) || pendingCandidates.contains(where: { $0.url == candidate.url }) else { + return false + } + self.attachedSubtitleURLs.insert(candidate.url) + return true + } + let attachedCount = self.backend.addSubtitleCandidates(attachableCandidates) + if attachedCount > 0 { + self.refreshControls() + } +#if DEBUG + let duplicateCount = candidates.count - pendingCandidates.count + resolvedCandidates.count - attachableCandidates.count + print("[DreamioNativePlayer] subtitle candidates=\(candidates.count) pending=\(pendingCandidates.count) resolved=\(resolvedCandidates.count) attachable=\(attachableCandidates.count) attached=\(attachedCount) duplicates=\(duplicateCount) tracks=\(SubtitleDebugFormatter.trackSummary(self.backend.subtitleTracks)) selected=\(self.backend.selectedSubtitleTrackID) resolvedCandidates=\(SubtitleDebugFormatter.candidateSummary(resolvedCandidates))") +#endif + } + } + + return pendingCandidates.count } override func viewDidDisappear(_ animated: Bool) { @@ -135,6 +195,16 @@ final class NativePlayerViewController: UIViewController { onDismiss?() } + private func resolveSubtitleCandidates(_ candidates: [SubtitleCandidate]) async -> [SubtitleCandidate] { + var resolved: [SubtitleCandidate] = [] + for candidate in candidates { + if let playableCandidate = await subtitleResolver.resolve(candidate) { + resolved.append(playableCandidate) + } + } + return resolved + } + private func configureBackend() { backend.prepare(in: self) backend.view.translatesAutoresizingMaskIntoConstraints = false @@ -164,6 +234,11 @@ final class NativePlayerViewController: UIViewController { self?.refreshControls() } } + backend.onAudioTracksChange = { [weak self] in + DispatchQueue.main.async { + self?.refreshControls() + } + } } private func startStartupTimer() { @@ -185,8 +260,9 @@ final class NativePlayerViewController: UIViewController { playPauseButton.addTarget(self, action: #selector(togglePlayPause), for: .touchUpInside) jumpBackButton.addTarget(self, action: #selector(jumpBack), for: .touchUpInside) jumpForwardButton.addTarget(self, action: #selector(jumpForward), for: .touchUpInside) + audioButton.showsMenuAsPrimaryAction = true captionsButton.showsMenuAsPrimaryAction = true - playPauseButton.layer.cornerRadius = 21 + playPauseButton.layer.cornerRadius = 24 scrubber.addTarget(self, action: #selector(scrubbingStarted), for: .touchDown) scrubber.addTarget(self, action: #selector(scrubberChanged), for: .valueChanged) scrubber.addTarget(self, action: #selector(scrubbingEnded), for: [.touchUpInside, .touchUpOutside, .touchCancel]) @@ -201,12 +277,19 @@ final class NativePlayerViewController: UIViewController { timeAndScrubRow.alignment = .center timeAndScrubRow.spacing = 8 - let controlRow = UIStackView(arrangedSubviews: [jumpBackButton, playPauseButton, jumpForwardButton, captionsButton]) + let playbackCluster = UIStackView(arrangedSubviews: [jumpBackButton, playPauseButton, jumpForwardButton]) + playbackCluster.translatesAutoresizingMaskIntoConstraints = false + playbackCluster.axis = .horizontal + playbackCluster.alignment = .center + playbackCluster.distribution = .equalCentering + playbackCluster.spacing = 14 + + let controlRow = UIStackView(arrangedSubviews: [audioButton, playbackCluster, captionsButton]) controlRow.translatesAutoresizingMaskIntoConstraints = false controlRow.axis = .horizontal controlRow.alignment = .center - controlRow.distribution = .equalSpacing - controlRow.spacing = 14 + controlRow.distribution = .equalCentering + controlRow.spacing = 18 let stack = UIStackView(arrangedSubviews: [timeAndScrubRow, controlRow]) stack.translatesAutoresizingMaskIntoConstraints = false @@ -257,6 +340,9 @@ final class NativePlayerViewController: UIViewController { playPauseButton.heightAnchor.constraint(equalToConstant: 42), jumpForwardButton.widthAnchor.constraint(equalToConstant: 36), jumpForwardButton.heightAnchor.constraint(equalToConstant: 36), + audioButton.widthAnchor.constraint(equalToConstant: 36), + audioButton.heightAnchor.constraint(equalToConstant: 36), + playbackCluster.centerXAnchor.constraint(equalTo: controlRow.centerXAnchor), captionsButton.widthAnchor.constraint(equalToConstant: 36), captionsButton.heightAnchor.constraint(equalToConstant: 36) ]) @@ -316,13 +402,28 @@ final class NativePlayerViewController: UIViewController { private func captionsMenu() -> UIMenu { let selectedTrackID = backend.selectedSubtitleTrackID - let trackActions = SubtitleOptionMapper.options(from: backend.subtitleTracks).map { track in + let tracks = backend.subtitleTracks + let options = SubtitleOptionMapper.options(from: tracks) +#if DEBUG + print("[DreamioCaptions] build-menu tracks=\(SubtitleDebugFormatter.trackSummary(tracks)) options=\(SubtitleDebugFormatter.trackSummary(options)) selected=\(selectedTrackID)") +#endif + let trackActions = options.map { track in UIAction( title: track.name, state: track.id == selectedTrackID ? .on : .off ) { [weak self] _ in - self?.backend.selectSubtitleTrack(id: track.id) - self?.refreshControls() + guard let self else { + return + } +#if DEBUG + print("[DreamioCaptions] select-request id=\(track.id) name=\(track.name) before=\(self.backend.selectedSubtitleTrackID)") +#endif + self.backend.selectSubtitleTrack(id: track.id) +#if DEBUG + print("[DreamioCaptions] select-result id=\(track.id) after=\(self.backend.selectedSubtitleTrackID) tracks=\(SubtitleDebugFormatter.trackSummary(self.backend.subtitleTracks))") +#endif + self.captionsMenuSignature = nil + self.refreshControls() } } @@ -332,10 +433,12 @@ final class NativePlayerViewController: UIViewController { children: [ UIAction(title: "Decrease 0.5s") { [weak self] _ in self?.backend.adjustSubtitleDelay(by: -0.5) + self?.captionsMenuSignature = nil self?.refreshControls() }, UIAction(title: "Increase 0.5s") { [weak self] _ in self?.backend.adjustSubtitleDelay(by: 0.5) + self?.captionsMenuSignature = nil self?.refreshControls() }, UIAction( @@ -348,6 +451,36 @@ final class NativePlayerViewController: UIViewController { return UIMenu(title: "Captions", children: trackActions + [delayActions]) } + private func audioMenu() -> UIMenu { + let selectedTrackID = backend.selectedAudioTrackID + let tracks = backend.audioTracks + let options = AudioOptionMapper.options(from: tracks) +#if DEBUG + print("[DreamioAudio] build-menu tracks=\(SubtitleDebugFormatter.trackSummary(tracks)) selected=\(selectedTrackID)") +#endif + let trackActions = options.map { track in + UIAction( + title: track.name, + state: track.id == selectedTrackID ? .on : .off + ) { [weak self] _ in + guard let self else { + return + } +#if DEBUG + print("[DreamioAudio] select-request id=\(track.id) name=\(track.name) before=\(self.backend.selectedAudioTrackID)") +#endif + self.backend.selectAudioTrack(id: track.id) +#if DEBUG + print("[DreamioAudio] select-result id=\(track.id) after=\(self.backend.selectedAudioTrackID) tracks=\(SubtitleDebugFormatter.trackSummary(self.backend.audioTracks))") +#endif + self.audioMenuSignature = nil + self.refreshControls() + } + } + + return UIMenu(title: "Audio", children: trackActions) + } + private func startProgressUpdates() { progressTimer?.invalidate() progressTimer = Timer.scheduledTimer(withTimeInterval: 0.5, repeats: true) { [weak self] _ in @@ -356,12 +489,14 @@ final class NativePlayerViewController: UIViewController { } private func refreshControls() { + let audioTracks = backend.audioTracks + let subtitleTracks = backend.subtitleTracks playPauseButton.setImage(UIImage(systemName: backend.isPlaying ? "pause.fill" : "play.fill"), for: .normal) scrubber.isEnabled = backend.isSeekable jumpBackButton.isEnabled = backend.isSeekable jumpForwardButton.isEnabled = backend.isSeekable - captionsButton.isEnabled = !SubtitleOptionMapper.options(from: backend.subtitleTracks).isEmpty - captionsButton.menu = captionsMenu() + updateAudioMenuIfNeeded(audioTracks: audioTracks) + updateCaptionsMenuIfNeeded(subtitleTracks: subtitleTracks) elapsedLabel.text = PlaybackTimeFormatter.label(for: backend.currentTime) remainingLabel.text = "-\(PlaybackTimeFormatter.label(for: backend.remainingTime))" if !isScrubbing { @@ -370,6 +505,65 @@ final class NativePlayerViewController: UIViewController { [scrubber, jumpBackButton, jumpForwardButton].forEach { $0.alpha = backend.isSeekable ? 1 : 0.45 } } + private func updateAudioMenuIfNeeded(audioTracks: [AudioTrack]) { + let selectedTrackID = backend.selectedAudioTrackID + let signature = trackMenuSignatureValue( + tracks: audioTracks, + selectedTrackID: selectedTrackID + ) + let hasSelectableTrack = AudioOptionMapper.options(from: audioTracks).count > 1 + audioButton.isEnabled = hasSelectableTrack + audioButton.alpha = hasSelectableTrack ? 1 : 0.45 + guard signature != audioMenuSignature else { + return + } + + audioMenuSignature = signature + audioButton.menu = audioMenu() +#if DEBUG + print("[DreamioAudio] refresh-menu enabled=\(audioButton.isEnabled) tracks=\(SubtitleDebugFormatter.trackSummary(audioTracks)) selected=\(selectedTrackID)") +#endif + } + + private func updateCaptionsMenuIfNeeded(subtitleTracks: [SubtitleTrack]) { + let selectedTrackID = backend.selectedSubtitleTrackID + let signature = captionsMenuSignatureValue( + tracks: subtitleTracks, + selectedTrackID: selectedTrackID, + delay: backend.subtitleDelay + ) + let hasSelectableTrack = subtitleTracks.contains { $0.id >= 0 } + captionsButton.isEnabled = hasSelectableTrack + guard signature != captionsMenuSignature else { + return + } + + captionsMenuSignature = signature + captionsButton.menu = captionsMenu() +#if DEBUG + print("[DreamioCaptions] refresh-menu enabled=\(captionsButton.isEnabled) tracks=\(SubtitleDebugFormatter.trackSummary(subtitleTracks)) selected=\(selectedTrackID)") +#endif + } + + private func captionsMenuSignatureValue( + tracks: [SubtitleTrack], + selectedTrackID: Int32, + delay: TimeInterval + ) -> String { + let trackSignature = trackMenuSignatureValue(tracks: tracks, selectedTrackID: selectedTrackID) + return "\(trackSignature)#delay=\(String(format: "%.1f", delay))" + } + + private func trackMenuSignatureValue( + tracks: [SubtitleTrack], + selectedTrackID: Int32 + ) -> String { + let trackSignature = tracks + .map { "\($0.id):\($0.name)" } + .joined(separator: "|") + return "\(trackSignature)#selected=\(selectedTrackID)" + } + private func revealControls() { controlsContainer.isUserInteractionEnabled = true closeButton.isUserInteractionEnabled = true @@ -404,8 +598,10 @@ final class NativePlayerViewController: UIViewController { button.translatesAutoresizingMaskIntoConstraints = false button.setImage(UIImage(systemName: systemName), for: .normal) button.tintColor = .white - button.backgroundColor = UIColor.black.withAlphaComponent(0.35) + button.backgroundColor = UIColor.white.withAlphaComponent(0.12) button.layer.cornerRadius = 18 + button.layer.borderColor = UIColor.white.withAlphaComponent(0.16).cgColor + button.layer.borderWidth = 1 button.accessibilityLabel = label return button } diff --git a/Dreamio/StreamCandidate.swift b/Dreamio/StreamCandidate.swift index 3371b54..f372e84 100644 --- a/Dreamio/StreamCandidate.swift +++ b/Dreamio/StreamCandidate.swift @@ -40,6 +40,35 @@ struct SubtitleTrack: Equatable { let name: String } +typealias AudioTrack = SubtitleTrack + +#if DEBUG +enum SubtitleDebugFormatter { + static func candidateSummary(_ candidates: [SubtitleCandidate]) -> String { + guard !candidates.isEmpty else { + return "[]" + } + + return candidates.map { candidate in + let extensionLabel = candidate.url.pathExtension.isEmpty ? "none" : candidate.url.pathExtension.lowercased() + let language = candidate.language?.isEmpty == false ? candidate.language! : "unknown" + let label = candidate.label.isEmpty ? "External Subtitle" : candidate.label + return "{label=\(label), language=\(language), ext=\(extensionLabel)}" + }.joined(separator: ", ") + } + + static func trackSummary(_ tracks: [SubtitleTrack]) -> String { + guard !tracks.isEmpty else { + return "[]" + } + + return tracks.map { track in + "{id=\(track.id), name=\(track.name)}" + }.joined(separator: ", ") + } +} +#endif + enum PlaybackTimeFormatter { static func label(for seconds: TimeInterval) -> String { guard seconds.isFinite, seconds > 0 else { @@ -66,6 +95,12 @@ enum SubtitleOptionMapper { } } +enum AudioOptionMapper { + static func options(from tracks: [AudioTrack]) -> [AudioTrack] { + tracks.filter { $0.id >= 0 } + } +} + struct StreamClassification { let sourceKind: StreamSourceKind let containerGuess: StreamContainerGuess @@ -105,8 +140,8 @@ struct StreamCandidate { enum SubtitleCandidateParser { private static let supportedExtensions = ["srt", "vtt", "ass", "ssa", "sub"] - private static let urlFields = ["url", "href", "src", "subtitles", "subtitle", "subtitleUrl", "subtitleURL", "file", "download"] - private static let labelFields = ["label", "name", "title", "lang", "language", "id"] + private static let urlFields = ["url", "href", "src", "link", "subtitles", "subtitle", "subtitleUrl", "subtitleURL", "file", "download"] + private static let labelFields = ["label", "name", "title", "file_name", "lang", "language", "id"] static func candidates(in payload: Any?) -> [SubtitleCandidate] { var results: [SubtitleCandidate] = [] @@ -129,7 +164,7 @@ enum SubtitleCandidateParser { if let candidate = candidate(from: dictionary) { results.append(candidate) } - dictionary.values.forEach { collect(from: $0, into: &results) } + orderedNestedValues(in: dictionary).forEach { collect(from: $0, into: &results) } case let array as [Any]: array.forEach { collect(from: $0, into: &results) } case let string as String: @@ -159,6 +194,27 @@ enum SubtitleCandidateParser { ) } + private static func orderedNestedValues(in dictionary: [String: Any]) -> [Any] { + let preferredKeys = ["subtitles", "subtitle", "files", "downloads", "download"] + var visitedKeys = Set() + var values: [Any] = [] + + preferredKeys.forEach { key in + if let value = dictionary[key] { + values.append(value) + visitedKeys.insert(key) + } + } + + dictionary.keys + .filter { !visitedKeys.contains($0) && !urlFields.contains($0) } + .sorted() + .compactMap { dictionary[$0] } + .forEach { values.append($0) } + + return values + } + private static func subtitleURL(from string: String?) -> URL? { guard let string, let url = URL(string: string), diff --git a/Dreamio/StreamResolver.swift b/Dreamio/StreamResolver.swift index 1943dea..c342cfd 100644 --- a/Dreamio/StreamResolver.swift +++ b/Dreamio/StreamResolver.swift @@ -29,6 +29,106 @@ enum StreamResolverError: LocalizedError { } } +final class SubtitleResolver: SubtitleResolving { + private let session: URLSession + + init(session: URLSession = .shared) { + self.session = session + } + + func resolve(_ candidate: SubtitleCandidate) async -> SubtitleCandidate? { + if Self.isDirectSubtitleFile(candidate.url) { + return candidate + } + + guard Self.shouldResolve(candidate.url) else { + return nil + } + + var request = URLRequest(url: candidate.url) + request.setValue("application/json, text/plain, text/vtt, application/x-subrip, */*", forHTTPHeaderField: "Accept") + + do { + let (data, response) = try await session.data(for: request) + if let httpResponse = response as? HTTPURLResponse, + !(200...299).contains(httpResponse.statusCode) { +#if DEBUG + print("[DreamioSubtitles] resolve status=\(httpResponse.statusCode) url=\(URLRedactor.redactedURLString(candidate.url.absoluteString))") +#endif + return nil + } + + if let finalURL = response.url, Self.isDirectSubtitleFile(finalURL) { + return SubtitleCandidate(url: finalURL, label: candidate.label, language: candidate.language) + } + + return Self.bestPlayableCandidate( + from: data, + responseURL: response.url, + original: candidate + ) + } catch { +#if DEBUG + print("[DreamioSubtitles] resolve failure=\(URLRedactor.redactedURLString(error.localizedDescription)) url=\(URLRedactor.redactedURLString(candidate.url.absoluteString))") +#endif + return nil + } + } + + static func bestPlayableCandidate( + from data: Data, + responseURL: URL?, + original: SubtitleCandidate + ) -> SubtitleCandidate? { + if let responseURL, isDirectSubtitleFile(responseURL) { + return SubtitleCandidate(url: responseURL, label: original.label, language: original.language) + } + + guard !data.isEmpty else { + return nil + } + + if let payload = try? JSONSerialization.jsonObject(with: data) { + return SubtitleCandidateParser.candidates(in: payload) + .first(where: { isDirectSubtitleFile($0.url) }) + .map { playable in + SubtitleCandidate( + url: playable.url, + label: original.label.isEmpty ? playable.label : original.label, + language: playable.language ?? original.language + ) + } + } + + if let text = String(data: data, encoding: .utf8) { + return SubtitleCandidateParser.candidates(in: text) + .first(where: { isDirectSubtitleFile($0.url) }) + .map { playable in + SubtitleCandidate( + url: playable.url, + label: original.label.isEmpty ? playable.label : original.label, + language: playable.language ?? original.language + ) + } + } + + return nil + } + + static func isDirectSubtitleFile(_ url: URL) -> Bool { + let lowercased = url.absoluteString.lowercased() + return ["srt", "vtt", "ass", "ssa", "sub"].contains(url.pathExtension.lowercased()) + || [".srt?", ".vtt?", ".ass?", ".ssa?", ".sub?", ".srt&", ".vtt&", ".ass&", ".ssa&", ".sub&"].contains(where: lowercased.contains) + } + + private static func shouldResolve(_ url: URL) -> Bool { + let lowercased = url.absoluteString.lowercased() + return lowercased.contains("opensubtitles") + || lowercased.contains("/subtitle") + || lowercased.contains("subtitle") + } +} + protocol StreamResolving { func resolve(request: NativePlaybackRequest) async throws -> ResolvedNativeStream } diff --git a/Dreamio/VLCNativePlaybackBackend.swift b/Dreamio/VLCNativePlaybackBackend.swift index d891c6f..f860366 100644 --- a/Dreamio/VLCNativePlaybackBackend.swift +++ b/Dreamio/VLCNativePlaybackBackend.swift @@ -18,11 +18,15 @@ final class VLCNativePlaybackBackend: NSObject, NativePlaybackBackend { var onFailure: ((Error) -> Void)? var onStateChange: (() -> Void)? var onSubtitleTracksChange: (() -> Void)? + var onAudioTracksChange: (() -> Void)? #if canImport(MobileVLCKit) private let mediaPlayer = VLCMediaPlayer() #endif private var attachedSubtitleURLs = Set() + private var didAutoSelectSubtitleTrack = false + private var didUserSelectSubtitleTrack = false + private var autoSelectedSubtitleTrackID: Int32? override init() { super.init() @@ -41,6 +45,9 @@ final class VLCNativePlaybackBackend: NSObject, NativePlaybackBackend { func play(request: NativePlaybackRequest) { #if canImport(MobileVLCKit) attachedSubtitleURLs.removeAll() + didAutoSelectSubtitleTrack = false + didUserSelectSubtitleTrack = false + autoSelectedSubtitleTrackID = nil let media = VLCMedia(url: request.playbackURL) let headerValue = request.headers .map { "\($0.key): \($0.value)" } @@ -58,7 +65,6 @@ final class VLCNativePlaybackBackend: NSObject, NativePlaybackBackend { print("[DreamioVLC] opening url=\(URLRedactor.redactedURLString(request.playbackURL.absoluteString))") #endif mediaPlayer.play() - attachSubtitles(request.subtitleCandidates) #else onFailure?(NativePlaybackError.backendUnavailable) #endif @@ -99,20 +105,56 @@ final class VLCNativePlaybackBackend: NSObject, NativePlaybackBackend { #endif } + func selectAudioTrack(id: Int32) { +#if canImport(MobileVLCKit) +#if DEBUG + logAudioTracks(reason: "before-select-\(id)") +#endif + mediaPlayer.currentAudioTrackIndex = id +#if DEBUG + logAudioTracks(reason: "after-select-\(id)") +#endif + onAudioTracksChange?() +#endif + } + func selectSubtitleTrack(id: Int32) { #if canImport(MobileVLCKit) + didUserSelectSubtitleTrack = true + autoSelectedSubtitleTrackID = nil +#if DEBUG + logSubtitleTracks(reason: "before-select-\(id)") +#endif mediaPlayer.currentVideoSubTitleIndex = id +#if DEBUG + logSubtitleTracks(reason: "after-select-\(id)") +#endif onSubtitleTracksChange?() #endif } func adjustSubtitleDelay(by seconds: TimeInterval) { #if canImport(MobileVLCKit) +#if DEBUG + print("[DreamioVLC] subtitle delay before=\(subtitleDelay) delta=\(seconds)") +#endif mediaPlayer.currentVideoSubTitleDelay += Int(seconds * 1_000_000) +#if DEBUG + print("[DreamioVLC] subtitle delay after=\(subtitleDelay)") +#endif onSubtitleTracksChange?() #endif } + @discardableResult + func addSubtitleCandidates(_ candidates: [SubtitleCandidate]) -> Int { +#if canImport(MobileVLCKit) + return attachSubtitles(candidates) +#else + return 0 +#endif + } + func stop() { #if canImport(MobileVLCKit) mediaPlayer.stop() @@ -165,6 +207,26 @@ final class VLCNativePlaybackBackend: NSObject, NativePlaybackBackend { #endif } + var audioTracks: [AudioTrack] { +#if canImport(MobileVLCKit) + let names = mediaPlayer.audioTrackNames as? [String] ?? [] + let indexes = mediaPlayer.audioTrackIndexes as? [NSNumber] ?? [] + return zip(indexes, names).map { index, name in + AudioTrack(id: index.int32Value, name: name) + } +#else + [] +#endif + } + + var selectedAudioTrackID: Int32 { +#if canImport(MobileVLCKit) + mediaPlayer.currentAudioTrackIndex +#else + -1 +#endif + } + var subtitleTracks: [SubtitleTrack] { #if canImport(MobileVLCKit) let names = mediaPlayer.videoSubTitlesNames as? [String] ?? [] @@ -194,23 +256,90 @@ final class VLCNativePlaybackBackend: NSObject, NativePlaybackBackend { } #if canImport(MobileVLCKit) - private func attachSubtitles(_ candidates: [SubtitleCandidate]) { + private func attachSubtitles(_ candidates: [SubtitleCandidate]) -> Int { + var attachedCount = 0 + var duplicateCount = 0 candidates.forEach { candidate in guard !attachedSubtitleURLs.contains(candidate.url) else { + duplicateCount += 1 return } attachedSubtitleURLs.insert(candidate.url) mediaPlayer.addPlaybackSlave(candidate.url, type: .subtitle, enforce: false) + attachedCount += 1 #if DEBUG - print("[DreamioVLC] attached subtitle=\(URLRedactor.redactedURLString(candidate.url.absoluteString))") + print("[DreamioVLC] addPlaybackSlave subtitle=\(URLRedactor.redactedURLString(candidate.url.absoluteString)) label=\(candidate.label) language=\(candidate.language ?? "unknown") ext=\(candidate.url.pathExtension.lowercased())") + logSubtitleTracks(reason: "after-addPlaybackSlave") #endif } - guard !candidates.isEmpty else { - return +#if DEBUG + if !candidates.isEmpty { + print("[DreamioVLC] subtitle candidates=\(candidates.count) attached=\(attachedCount) duplicates=\(duplicateCount)") + } +#endif + guard attachedCount > 0 else { + return attachedCount } DispatchQueue.main.asyncAfter(deadline: .now() + 1) { [weak self] in + self?.selectInitialSubtitleTrackIfNeeded(reason: "delayed-refresh") + #if DEBUG + self?.logSubtitleTracks(reason: "delayed-refresh") + #endif self?.onSubtitleTracksChange?() } + return attachedCount + } + +#if DEBUG + private func logAudioTracks(reason: String) { + let names = mediaPlayer.audioTrackNames as? [String] ?? [] + let indexes = mediaPlayer.audioTrackIndexes as? [NSNumber] ?? [] + print("[DreamioVLC] audio tracks reason=\(reason) names=\(names) indexes=\(indexes.map { $0.int32Value }) selected=\(mediaPlayer.currentAudioTrackIndex)") + } + + private func logSubtitleTracks(reason: String) { + let names = mediaPlayer.videoSubTitlesNames as? [String] ?? [] + let indexes = mediaPlayer.videoSubTitlesIndexes as? [NSNumber] ?? [] + print("[DreamioVLC] subtitle tracks reason=\(reason) names=\(names) indexes=\(indexes.map { $0.int32Value }) selected=\(mediaPlayer.currentVideoSubTitleIndex)") + } +#endif + + private func selectInitialSubtitleTrackIfNeeded(reason: String) { + guard !didUserSelectSubtitleTrack, + !didAutoSelectSubtitleTrack, + mediaPlayer.currentVideoSubTitleIndex < 0, + let track = subtitleTracks.first(where: { $0.id >= 0 }) else { + return + } + + didAutoSelectSubtitleTrack = true + autoSelectedSubtitleTrackID = track.id +#if DEBUG + print("[DreamioVLC] auto-select subtitle id=\(track.id) name=\(track.name) reason=\(reason)") +#endif + mediaPlayer.currentVideoSubTitleIndex = track.id + scheduleAutoSubtitleSelectionReapply(trackID: track.id) + } + + private func scheduleAutoSubtitleSelectionReapply(trackID: Int32) { + [0.3, 1.0, 2.0, 4.0].forEach { delay in + DispatchQueue.main.asyncAfter(deadline: .now() + delay) { [weak self] in + self?.reapplyAutoSelectedSubtitleTrackIfNeeded(reason: "delayed-\(String(format: "%.1f", delay))") + } + } + } + + private func reapplyAutoSelectedSubtitleTrackIfNeeded(reason: String) { + guard !didUserSelectSubtitleTrack, + let trackID = autoSelectedSubtitleTrackID, + subtitleTracks.contains(where: { $0.id == trackID }) else { + return + } + + mediaPlayer.currentVideoSubTitleIndex = trackID +#if DEBUG + print("[DreamioVLC] reapply subtitle id=\(trackID) reason=\(reason) selected=\(mediaPlayer.currentVideoSubTitleIndex)") +#endif } #endif } @@ -223,13 +352,21 @@ extension VLCNativePlaybackBackend: VLCMediaPlayerDelegate { #endif switch mediaPlayer.state { case .buffering, .playing: + reapplyAutoSelectedSubtitleTrackIfNeeded(reason: stateName(mediaPlayer.state)) onReady?() onStateChange?() + onAudioTracksChange?() case .error: onFailure?(NativePlaybackError.playbackFailed) case .paused, .stopped, .ended: onStateChange?() case .esAdded: + selectInitialSubtitleTrackIfNeeded(reason: "esAdded") +#if DEBUG + logAudioTracks(reason: "esAdded") + logSubtitleTracks(reason: "esAdded") +#endif + onAudioTracksChange?() onSubtitleTracksChange?() default: break diff --git a/Tests/StreamResolverTests.swift b/Tests/StreamResolverTests.swift index e70fc2b..c14ddc8 100644 --- a/Tests/StreamResolverTests.swift +++ b/Tests/StreamResolverTests.swift @@ -9,6 +9,9 @@ struct StreamResolverTests { testRedactorHandlesPercentEncodedPath() testPlaybackTimeFormatting() testSubtitleCandidateParsing() + testOpenSubtitlesV3CandidateParsing() + testOpenSubtitlesV3DownloadResponseResolution() + testSubtitleCandidateDeduplicationPreservesLabels() testSubtitleOptionMappingIncludesNone() print("StreamResolverTests passed") } @@ -110,6 +113,90 @@ struct StreamResolverTests { assertEqual(candidates[2].url.absoluteString, "https://cdn.example.test/movie.fr.ass?download=1") } + private static func testOpenSubtitlesV3CandidateParsing() { + let payload: [String: Any] = [ + "subtitles": [ + [ + "language": "English", + "download": "https://api.opensubtitles.com/api/v1/download/subtitle-file", + "nested": [ + [ + "file": "https://dl.opensubtitles.org/en/subtitle.vtt?download=1" + ] + ] + ], + [ + "lang": "spa", + "url": "https://opensubtitles.example.test/download/episode.srt" + ] + ], + "body": "alternate https://cdn.example.test/from-string.ass?source=opensubtitles", + "ignored": [ + "https://cdn.example.test/poster.jpg", + ["file": "https://cdn.example.test/video.mkv"] + ] + ] + + let candidates = SubtitleCandidateParser.candidates(in: payload) + + assertEqual(candidates.count, 4) + assertEqual(candidates[0].label, "English") + assertEqual(candidates[0].language, "English") + assertEqual(candidates[1].url.absoluteString, "https://dl.opensubtitles.org/en/subtitle.vtt?download=1") + assertEqual(candidates[2].label, "spa") + assertEqual(candidates[2].language, "spa") + assertEqual(candidates[3].url.absoluteString, "https://cdn.example.test/from-string.ass?source=opensubtitles") + } + + private static func testOpenSubtitlesV3DownloadResponseResolution() { + let payload = """ + { + "link": "https://dl.opensubtitles.org/en/download/subtitle.srt?token=secret", + "file_name": "episode.srt", + "requests": 1 + } + """.data(using: .utf8)! + let original = SubtitleCandidate( + url: URL(string: "https://api.opensubtitles.com/api/v1/download")!, + label: "English", + language: "eng" + ) + + let candidate = SubtitleResolver.bestPlayableCandidate( + from: payload, + responseURL: original.url, + original: original + ) + + assertEqual(candidate?.url.absoluteString, "https://dl.opensubtitles.org/en/download/subtitle.srt?token=secret") + assertEqual(candidate?.label, "English") + assertEqual(candidate?.language, "eng") + } + + private static func testSubtitleCandidateDeduplicationPreservesLabels() { + let payload: [String: Any] = [ + "subtitles": [ + [ + "label": "English SDH", + "lang": "eng", + "url": "https://opensubtitles.example.test/download/duplicate.srt" + ], + [ + "label": "Duplicate", + "language": "English", + "download": "https://opensubtitles.example.test/download/duplicate.srt" + ], + "https://opensubtitles.example.test/download/duplicate.srt" + ] + ] + + let candidates = SubtitleCandidateParser.candidates(in: payload) + + assertEqual(candidates.count, 1) + assertEqual(candidates[0].label, "English SDH") + assertEqual(candidates[0].language, "eng") + } + private static func testSubtitleOptionMappingIncludesNone() { let options = SubtitleOptionMapper.options(from: [ SubtitleTrack(id: 2, name: "English"), diff --git a/docs/turns/2026-05-25-auto-select-vlc-subtitle-tracks.html b/docs/turns/2026-05-25-auto-select-vlc-subtitle-tracks.html new file mode 100644 index 0000000..2abc22c --- /dev/null +++ b/docs/turns/2026-05-25-auto-select-vlc-subtitle-tracks.html @@ -0,0 +1,395 @@ + + + + + + Auto-select VLC subtitle tracks + + + +
+
+

Auto-select VLC subtitle tracks

+

Turn document for dreamio-djc, created May 25, 2026.

+
+

Dreamio now asks MobileVLCKit to enable the first real subtitle track when VLC discovers embedded MKV subtitles and playback is still on “Disable.” This targets the log pattern where VLC sees English (SDH) but leaves selected=-1.

+
+
+ +
+

Summary

+

Fixed the remaining native playback subtitle issue for streams that provide no external subtitle candidates but do contain embedded subtitle tracks. VLC can discover those tracks after playback starts; Dreamio now auto-selects the first selectable one once it appears.

+
+ +
+

Changes Made

+
    +
  • Added per-playback state to track whether Dreamio has already auto-selected subtitles.
    • +
  • Added per-playback state to detect when the user manually chooses a caption option, including disabling captions.
    • +
  • Auto-selects the first subtitle track with a non-negative VLC track id when VLC reports .esAdded or after an external subtitle attach refresh.
    • +
  • Added a debug log line for automatic subtitle selection so future device logs should show the selected track id and reason.
    • +
+
+ +
+

Context

+

The reported logs showed subtitle candidates=0 from the native player, followed by VLC reporting subtitle tracks named Disable and English (SDH) - [English] with selected track -1. That means stream parsing and VLC track discovery were both working; the remaining gap was that Dreamio never changed VLC away from the disabled subtitle track.

+
+ +
+

Important Implementation Details

+
    +
  • The selection guard only runs when currentVideoSubTitleIndex is below zero, so it will not replace an already active subtitle track.
    • +
  • The auto-selection runs only once per playback item.
    • +
  • User interaction wins: once selectSubtitleTrack(id:) is called from the captions menu, Dreamio stops automatic caption selection for that playback item.
    • +
  • The state resets in play(request:), alongside the existing attached subtitle URL reset.
    • +
+
+ +
+

Relevant Diff Snippets

+

Rendered with @pierre/diffs/ssr.

+
Dreamio/VLCNativePlaybackBackend.swift
+22
22 unmodified lines
23
24
25
26
27
28
12 unmodified lines
41
42
43
44
45
46
53 unmodified lines
100
101
102
103
104
105
133 unmodified lines
239
240
241
242
243
244
9 unmodified lines
254
255
256
257
258
259
12 unmodified lines
272
273
274
275
276
277
22 unmodified lines
    private let mediaPlayer = VLCMediaPlayer()
#endif
    private var attachedSubtitleURLs = Set<URL>()

+
    override init() {
        super.init()
12 unmodified lines
    func play(request: NativePlaybackRequest) {
#if canImport(MobileVLCKit)
        attachedSubtitleURLs.removeAll()
        let media = VLCMedia(url: request.playbackURL)
        let headerValue = request.headers
            .map { "\($0.key): \($0.value)" }
53 unmodified lines

+
    func selectSubtitleTrack(id: Int32) {
#if canImport(MobileVLCKit)
#if DEBUG
        logSubtitleTracks(reason: "before-select-\(id)")
#endif
133 unmodified lines
            return attachedCount
        }
        DispatchQueue.main.asyncAfter(deadline: .now() + 1) { [weak self] in
            #if DEBUG
            self?.logSubtitleTracks(reason: "delayed-refresh")
            #endif
9 unmodified lines
        print("[DreamioVLC] subtitle tracks reason=\(reason) names=\(names) indexes=\(indexes.map { $0.int32Value }) selected=\(mediaPlayer.currentVideoSubTitleIndex)")
    }
#endif
#endif
}

+
12 unmodified lines
        case .paused, .stopped, .ended:
            onStateChange?()
        case .esAdded:
#if DEBUG
            logSubtitleTracks(reason: "esAdded")
#endif
22 unmodified lines
23
24
25
26
27
28
29
30
12 unmodified lines
43
44
45
46
47
48
49
50
53 unmodified lines
104
105
106
107
108
109
110
133 unmodified lines
244
245
246
247
248
249
250
9 unmodified lines
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
12 unmodified lines
293
294
295
296
297
298
299
22 unmodified lines
    private let mediaPlayer = VLCMediaPlayer()
#endif
    private var attachedSubtitleURLs = Set<URL>()
    private var didAutoSelectSubtitleTrack = false
    private var didUserSelectSubtitleTrack = false

+
    override init() {
        super.init()
12 unmodified lines
    func play(request: NativePlaybackRequest) {
#if canImport(MobileVLCKit)
        attachedSubtitleURLs.removeAll()
        didAutoSelectSubtitleTrack = false
        didUserSelectSubtitleTrack = false
        let media = VLCMedia(url: request.playbackURL)
        let headerValue = request.headers
            .map { "\($0.key): \($0.value)" }
53 unmodified lines

+
    func selectSubtitleTrack(id: Int32) {
#if canImport(MobileVLCKit)
        didUserSelectSubtitleTrack = true
#if DEBUG
        logSubtitleTracks(reason: "before-select-\(id)")
#endif
133 unmodified lines
            return attachedCount
        }
        DispatchQueue.main.asyncAfter(deadline: .now() + 1) { [weak self] in
            self?.selectInitialSubtitleTrackIfNeeded(reason: "delayed-refresh")
            #if DEBUG
            self?.logSubtitleTracks(reason: "delayed-refresh")
            #endif
9 unmodified lines
        print("[DreamioVLC] subtitle tracks reason=\(reason) names=\(names) indexes=\(indexes.map { $0.int32Value }) selected=\(mediaPlayer.currentVideoSubTitleIndex)")
    }
#endif

+
    private func selectInitialSubtitleTrackIfNeeded(reason: String) {
        guard !didUserSelectSubtitleTrack,
              !didAutoSelectSubtitleTrack,
              mediaPlayer.currentVideoSubTitleIndex < 0,
              let track = subtitleTracks.first(where: { $0.id >= 0 }) else {
            return
        }

+
        didAutoSelectSubtitleTrack = true
#if DEBUG
        print("[DreamioVLC] auto-select subtitle id=\(track.id) name=\(track.name) reason=\(reason)")
#endif
        mediaPlayer.currentVideoSubTitleIndex = track.id
    }
#endif
}

+
12 unmodified lines
        case .paused, .stopped, .ended:
            onStateChange?()
        case .esAdded:
            selectInitialSubtitleTrackIfNeeded(reason: "esAdded")
#if DEBUG
            logSubtitleTracks(reason: "esAdded")
#endif
+
+ +
+

Expected Impact for End-Users

+

MKV streams with embedded subtitles should show captions automatically instead of requiring the user to open the captions menu and pick the embedded track manually. Users can still disable captions or switch tracks afterward.

+
+ +
+

Validation

+
    +
  • Ran xcodebuild -scheme Dreamio -project Dreamio.xcodeproj -destination 'generic/platform=iOS' build.
    • +
  • The build succeeded against MobileVLCKit.
    • +
+
+ +
+

Issues, Limitations, and Mitigations

+
    +
  • This was validated by build, not by replaying the exact Real-Debrid stream on device in this turn.
    • +
  • If a file has multiple embedded subtitle tracks, Dreamio chooses the first selectable track VLC exposes. The captions menu remains available for manual switching.
    • +
  • If a stream intentionally starts with subtitles disabled and the user never touches the captions menu, Dreamio will now enable the first discovered track by default.
    • +
+
+ +
+

Follow-up Work

+
    +
  • Test the same South Park MKV on device and confirm the logs include [DreamioVLC] auto-select subtitle followed by a non-negative selected subtitle id.
    • +
  • Consider a future setting for “auto-enable embedded subtitles” if users want control over the default behavior.
    • +
+
+
+

New Changes as of May 25, 2026 at 10:45 AM EDT

+

Summary of changes: Added a timed re-apply loop for the automatically selected VLC subtitle track. Dreamio now remembers the auto-selected track id and re-sends it after short delays and when VLC reports buffering or playing.

+

Why this change was made: Follow-up device logs showed VLC selected track 3, but subtitles still did not render. That points to a MobileVLCKit timing issue after elementary stream discovery, so the selected embedded track is re-applied after playback settles.

+

Code diffs:

+
Dreamio/VLCNativePlaybackBackend.swift
+27
24 unmodified lines
25
26
27
28
29
30
14 unmodified lines
45
46
47
48
49
50
54 unmodified lines
105
106
107
108
109
110
159 unmodified lines
270
271
272
273
274
275
276
277
278
279
6 unmodified lines
286
287
288
289
290
291
24 unmodified lines
    private var attachedSubtitleURLs = Set<URL>()
    private var didAutoSelectSubtitleTrack = false
    private var didUserSelectSubtitleTrack = false

+
    override init() {
        super.init()
14 unmodified lines
        attachedSubtitleURLs.removeAll()
        didAutoSelectSubtitleTrack = false
        didUserSelectSubtitleTrack = false
        let media = VLCMedia(url: request.playbackURL)
        let headerValue = request.headers
            .map { "\($0.key): \($0.value)" }
54 unmodified lines
    func selectSubtitleTrack(id: Int32) {
#if canImport(MobileVLCKit)
        didUserSelectSubtitleTrack = true
#if DEBUG
        logSubtitleTracks(reason: "before-select-\(id)")
#endif
159 unmodified lines
        }

+
        didAutoSelectSubtitleTrack = true
#if DEBUG
        print("[DreamioVLC] auto-select subtitle id=\(track.id) name=\(track.name) reason=\(reason)")
#endif
        mediaPlayer.currentVideoSubTitleIndex = track.id
    }
#endif
}
6 unmodified lines
#endif
        switch mediaPlayer.state {
        case .buffering, .playing:
            onReady?()
            onStateChange?()
        case .error:
24 unmodified lines
25
26
27
28
29
30
31
14 unmodified lines
46
47
48
49
50
51
52
54 unmodified lines
107
108
109
110
111
112
113
159 unmodified lines
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
6 unmodified lines
312
313
314
315
316
317
318
24 unmodified lines
    private var attachedSubtitleURLs = Set<URL>()
    private var didAutoSelectSubtitleTrack = false
    private var didUserSelectSubtitleTrack = false
    private var autoSelectedSubtitleTrackID: Int32?

+
    override init() {
        super.init()
14 unmodified lines
        attachedSubtitleURLs.removeAll()
        didAutoSelectSubtitleTrack = false
        didUserSelectSubtitleTrack = false
        autoSelectedSubtitleTrackID = nil
        let media = VLCMedia(url: request.playbackURL)
        let headerValue = request.headers
            .map { "\($0.key): \($0.value)" }
54 unmodified lines
    func selectSubtitleTrack(id: Int32) {
#if canImport(MobileVLCKit)
        didUserSelectSubtitleTrack = true
        autoSelectedSubtitleTrackID = nil
#if DEBUG
        logSubtitleTracks(reason: "before-select-\(id)")
#endif
159 unmodified lines
        }

+
        didAutoSelectSubtitleTrack = true
        autoSelectedSubtitleTrackID = track.id
#if DEBUG
        print("[DreamioVLC] auto-select subtitle id=\(track.id) name=\(track.name) reason=\(reason)")
#endif
        mediaPlayer.currentVideoSubTitleIndex = track.id
        scheduleAutoSubtitleSelectionReapply(trackID: track.id)
    }

+
    private func scheduleAutoSubtitleSelectionReapply(trackID: Int32) {
        [0.3, 1.0, 2.0, 4.0].forEach { delay in
            DispatchQueue.main.asyncAfter(deadline: .now() + delay) { [weak self] in
                self?.reapplyAutoSelectedSubtitleTrackIfNeeded(reason: "delayed-\(String(format: "%.1f", delay))")
            }
        }
    }

+
    private func reapplyAutoSelectedSubtitleTrackIfNeeded(reason: String) {
        guard !didUserSelectSubtitleTrack,
              let trackID = autoSelectedSubtitleTrackID,
              subtitleTracks.contains(where: { $0.id == trackID }) else {
            return
        }

+
        mediaPlayer.currentVideoSubTitleIndex = trackID
#if DEBUG
        print("[DreamioVLC] reapply subtitle id=\(trackID) reason=\(reason) selected=\(mediaPlayer.currentVideoSubTitleIndex)")
#endif
    }
#endif
}
6 unmodified lines
#endif
        switch mediaPlayer.state {
        case .buffering, .playing:
            reapplyAutoSelectedSubtitleTrackIfNeeded(reason: stateName(mediaPlayer.state))
            onReady?()
            onStateChange?()
        case .error:
+

Related issues or PRs: Beads issue dreamio-ppj.

+
+ +
+

New Changes as of May 25, 2026 at 10:51 AM EDT

+

Summary of changes: Added browser text-track capture for OpenSubtitles V3 subtitles. The web bridge now inspects <track> elements, HTMLTrackElement.src assignments, setAttribute("src", ...) calls, video.textTracks, and late addtrack events.

+

Why this change was made: The device logs showed embedded subtitles were visible, but OpenSubtitles V3 options still never reached native playback. That means the external subtitle candidates were being missed before VLC, likely because Stremio attached them as browser text tracks rather than including them in the original stream candidate.

+

Code diffs:

+
Dreamio/DreamioWebViewController.swift
-4+69
198 unmodified lines
199
200
201
202
203
204
102 unmodified lines
307
308
309
310
311
312
313
314
315
316
317
19 unmodified lines
337
338
339
340
341
342
343
344
345
346
2 unmodified lines
349
350
351
352
353
354
355
356
357
7 unmodified lines
365
366
367
368
369
370
371
198 unmodified lines
            postSubtitleCandidates([candidate]);
          };

+
          const postSubtitleInspection = (source, url, beforeCount, afterCount, payloadLength) => {
            if (afterCount > beforeCount) {
              return;
102 unmodified lines
            if (!node) {
              return;
            }
            if (node instanceof HTMLVideoElement || node instanceof HTMLSourceElement) {
              postCandidate(node.currentSrc || node.src || node.getAttribute("src"), node);
            }
            if (node.querySelectorAll) {
              node.querySelectorAll("video, source").forEach(inspectMedia);
            }
          };

+
19 unmodified lines
            });
          }

+
          const originalSetAttribute = Element.prototype.setAttribute;
          Element.prototype.setAttribute = function(name, value) {
            if (String(name).toLowerCase() === "src" && (this instanceof HTMLVideoElement || this instanceof HTMLSourceElement)) {
              postCandidate(value, this);
            }
            return originalSetAttribute.call(this, name, value);
          };
2 unmodified lines
          HTMLMediaElement.prototype.load = function() {
            inspectMedia(this);
            this.querySelectorAll("source").forEach(inspectMedia);
            return originalLoad.call(this);
          };

+
          document.addEventListener("loadedmetadata", (event) => inspectMedia(event.target), true);
          document.addEventListener("error", (event) => inspectMedia(event.target), true);
          new MutationObserver((mutations) => {
7 unmodified lines
            childList: true,
            subtree: true,
            attributes: true,
            attributeFilter: ["src"]
          });

+
          inspectMedia(document);
198 unmodified lines
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
102 unmodified lines
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
19 unmodified lines
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
2 unmodified lines
410
411
412
413
414
415
416
417
418
419
420
421
422
7 unmodified lines
430
431
432
433
434
435
436
198 unmodified lines
            postSubtitleCandidates([candidate]);
          };

+
          const inspectTrack = (track) => {
            if (!track) {
              return;
            }
            if (track instanceof HTMLTrackElement) {
              addSubtitleCandidate({
                url: track.src || track.getAttribute("src") || "",
                label: track.label || track.srclang || "External Subtitle",
                language: track.srclang || ""
              });
              return;
            }
            const source = track.src || track.url || "";
            if (source) {
              addSubtitleCandidate({
                url: source,
                label: track.label || track.language || track.kind || "External Subtitle",
                language: track.language || ""
              });
            }
          };

+
          const inspectTextTracks = (media) => {
            try {
              Array.from(media.textTracks || []).forEach(inspectTrack);
            } catch (_) {}
            try {
              media.querySelectorAll("track").forEach(inspectTrack);
            } catch (_) {}
          };

+
          const postSubtitleInspection = (source, url, beforeCount, afterCount, payloadLength) => {
            if (afterCount > beforeCount) {
              return;
102 unmodified lines
            if (!node) {
              return;
            }
            if (node instanceof HTMLTrackElement) {
              inspectTrack(node);
            }
            if (node instanceof HTMLVideoElement || node instanceof HTMLSourceElement) {
              postCandidate(node.currentSrc || node.src || node.getAttribute("src"), node);
            }
            if (node.querySelectorAll) {
              node.querySelectorAll("video, source, track").forEach(inspectMedia);
            }
            if (node instanceof HTMLVideoElement) {
              inspectTextTracks(node);
            }
          };

+
19 unmodified lines
            });
          }

+
          const trackSrcDescriptor = Object.getOwnPropertyDescriptor(HTMLTrackElement.prototype, "src");
          if (trackSrcDescriptor && trackSrcDescriptor.set) {
            Object.defineProperty(HTMLTrackElement.prototype, "src", {
              get: trackSrcDescriptor.get,
              set(value) {
                addSubtitleCandidate({
                  url: value,
                  label: this.label || this.srclang || "External Subtitle",
                  language: this.srclang || ""
                });
                return trackSrcDescriptor.set.call(this, value);
              }
            });
          }

+
          const originalSetAttribute = Element.prototype.setAttribute;
          Element.prototype.setAttribute = function(name, value) {
            if (String(name).toLowerCase() === "src") {
              if (this instanceof HTMLVideoElement || this instanceof HTMLSourceElement) {
                postCandidate(value, this);
              }
              if (this instanceof HTMLTrackElement) {
                addSubtitleCandidate({
                  url: value,
                  label: this.label || this.srclang || "External Subtitle",
                  language: this.srclang || ""
                });
              }
            }
            return originalSetAttribute.call(this, name, value);
          };
2 unmodified lines
          HTMLMediaElement.prototype.load = function() {
            inspectMedia(this);
            this.querySelectorAll("source").forEach(inspectMedia);
            inspectTextTracks(this);
            return originalLoad.call(this);
          };

+
          document.addEventListener("addtrack", (event) => {
            inspectTrack(event.track || event.target);
          }, true);
          document.addEventListener("loadedmetadata", (event) => inspectMedia(event.target), true);
          document.addEventListener("error", (event) => inspectMedia(event.target), true);
          new MutationObserver((mutations) => {
7 unmodified lines
            childList: true,
            subtree: true,
            attributes: true,
            attributeFilter: ["src", "label", "srclang"]
          });

+
          inspectMedia(document);
+

Related issues or PRs: Beads issue dreamio-3xi.

+
+ +
+ + \ No newline at end of file diff --git a/docs/turns/2026-05-25-forward-late-opensubtitles-subtitles.html b/docs/turns/2026-05-25-forward-late-opensubtitles-subtitles.html new file mode 100644 index 0000000..07ff922 --- /dev/null +++ b/docs/turns/2026-05-25-forward-late-opensubtitles-subtitles.html @@ -0,0 +1,711 @@ + + + + + + Forward Late OpenSubtitles Subtitles to Native Player + + + +
+
+

Forward Late OpenSubtitles Subtitles to Native Player

+

Native playback now keeps listening for subtitle discoveries after VLC opens, so OpenSubtitles V3 tracks that arrive late can be attached to the active player session without duplicating captions.

+
+ Date: 2026-05-25 + Beads: dreamio-lw6 + Scope: native playback subtitles +
+
+ +
+

Summary

+

Fixed the timing gap where subtitle candidates were only passed to native playback during the initial stream-candidate message. The web bridge now emits dedicated subtitle-candidate messages as new URLs are discovered, and the active native player can append those subtitles to VLC while playback is already running.

+
+ +
+

Changes Made

+
    +
  • Added a dedicated dreamioSubtitleCandidate web bridge message for newly discovered subtitle URLs.
    • +
  • Tracked the currently presented NativePlayerViewController so late subtitle candidates can be routed into the active VLC session.
    • +
  • Added addSubtitleCandidates(_:) to the native player and playback backend protocols.
    • +
  • Updated the VLC backend to append subtitles through addPlaybackSlave(..., type: .subtitle, enforce: false) after playback has started.
    • +
  • Kept de-duplication in JavaScript, the native player, and the VLC backend.
    • +
  • Added parser tests for OpenSubtitles V3-like payloads and duplicate preservation behavior.
    • +
+
+ +
+

Context

+

The old bridge stored subtitle candidates in JavaScript and included the last batch only when posting a native stream candidate. OpenSubtitles V3 addon responses can arrive after the stream has already been handed to VLC, which meant the native player opened with no chance to learn about those later tracks.

+
+ +
+

Important Implementation Details

+
    +
  • The bridge posts each new subtitle URL once through postedSubtitleURLs, while retaining the existing subtitle list for stream-candidate startup payloads.
    • +
  • The native player keeps its own Set<URL> for late candidates, preventing duplicate menu entries before they reach the backend.
    • +
  • The VLC backend still maintains attachedSubtitleURLs, so backend-level calls remain idempotent even if callers repeat URLs.
    • +
  • DEBUG logs now report discovered, forwarded, attached, and duplicate counts with redacted URLs for individual VLC attachments.
    • +
+
+ +
+

Relevant Diff Snippets

+

Dreamio/DreamioWebViewController.swift

Dreamio/DreamioWebViewController.swift
-3+56
5 unmodified lines
6
7
8
9
10
11
6 unmodified lines
18
19
20
21
22
23
28 unmodified lines
52
53
54
55
56
57
15 unmodified lines
73
74
75
76
77
78
40 unmodified lines
119
120
121
122
123
124
6 unmodified lines
131
132
133
134
135
136
137
138
139
140
141
280 unmodified lines
422
423
424
425
426
427
428
1 unmodified line
430
431
432
433
434
435
19 unmodified lines
455
456
457
458
459
460
461
462
206 unmodified lines
669
670
671
672
673
674
5 unmodified lines
        static let stremioWebURL = URL(string: "https://web.stremio.com/")!
        static let diagnosticsMessageHandler = "dreamioDiagnostics"
        static let streamCandidateMessageHandler = "dreamioStreamCandidate"
    }

+
    private lazy var webView: WKWebView = {
6 unmodified lines
            WeakScriptMessageHandler(delegate: self),
            name: Constants.streamCandidateMessageHandler
        )
        configuration.userContentController.addUserScript(Self.streamCandidateScript)
#if DEBUG
        configuration.userContentController.add(
28 unmodified lines
    private var progressObservation: NSKeyValueObservation?
    private var userAgent: String?
    private var lastNativePlaybackURL: URL?
    private let streamResolver: StreamResolving = StremioStreamResolver()

+
    private static let streamCandidateScript = WKUserScript(
15 unmodified lines
            /\.mp4(?:[?#]|$)/i
          ];
          const subtitleCandidates = [];
          const subtitleURLPattern = /https?:\/\/[^\s"'<>]+(?:\.srt|\.vtt|\.ass|\.ssa|\.sub|opensubtitles|subtitle)[^\s"'<>]*/ig;

+
          const looksNative = (url) => {
40 unmodified lines
            } catch (_) {}
          };

+
          const addSubtitleCandidate = (entry) => {
            const rawURL = typeof entry === "string" ? entry : entry && (entry.url || entry.href || entry.src || entry.file || entry.download);
            const url = absoluteURL(rawURL);
6 unmodified lines
            if (subtitleCandidates.some((candidate) => candidate.url === url)) {
              return;
            }
            subtitleCandidates.push({
              url,
              label: entry && (entry.label || entry.name || entry.title || entry.lang || entry.language) || "External Subtitle",
              language: entry && (entry.lang || entry.language) || ""
            });
          };

+
          const inspectSubtitlePayload = (payload) => {
280 unmodified lines

+
#if DEBUG
        let classification = request.classification
        print("[DreamioStream] class=\(classification.sourceKind.rawValue) container=\(classification.containerGuess.rawValue) reason=\(classification.reason) observed=\(classification.sanitizedObservedURL) resolver=\(classification.sanitizedResolverURL ?? "none")")
#endif

+
        Task { [weak self] in
1 unmodified line
        }
    }

+
    @MainActor
    private func resolveAndPresentNativePlayback(_ request: NativePlaybackRequest) async {
        guard VLCNativePlaybackBackend.isAvailable else {
19 unmodified lines
                subtitleCandidates: request.subtitleCandidates
            )
            let player = NativePlayerViewController(request: resolvedRequest)
            player.onDismiss = { [weak self] in
                self?.lastNativePlaybackURL = nil
                self?.cleanUpStremioPlayerAfterNativeDismiss()
            }
            present(player, animated: true)
206 unmodified lines
            return
        }

+
#if DEBUG
        guard message.name == Constants.diagnosticsMessageHandler,
              let body = message.body as? [String: Any],
5 unmodified lines
6
7
8
9
10
11
12
6 unmodified lines
19
20
21
22
23
24
25
26
27
28
28 unmodified lines
57
58
59
60
61
62
63
15 unmodified lines
79
80
81
82
83
84
85
40 unmodified lines
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
6 unmodified lines
157
158
159
160
161
162
163
164
165
166
167
168
169
280 unmodified lines
450
451
452
453
454
455
456
1 unmodified line
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
19 unmodified lines
501
502
503
504
505
506
507
508
509
510
206 unmodified lines
717
718
719
720
721
722
723
724
725
726
727
5 unmodified lines
        static let stremioWebURL = URL(string: "https://web.stremio.com/")!
        static let diagnosticsMessageHandler = "dreamioDiagnostics"
        static let streamCandidateMessageHandler = "dreamioStreamCandidate"
        static let subtitleCandidateMessageHandler = "dreamioSubtitleCandidate"
    }

+
    private lazy var webView: WKWebView = {
6 unmodified lines
            WeakScriptMessageHandler(delegate: self),
            name: Constants.streamCandidateMessageHandler
        )
        configuration.userContentController.add(
            WeakScriptMessageHandler(delegate: self),
            name: Constants.subtitleCandidateMessageHandler
        )
        configuration.userContentController.addUserScript(Self.streamCandidateScript)
#if DEBUG
        configuration.userContentController.add(
28 unmodified lines
    private var progressObservation: NSKeyValueObservation?
    private var userAgent: String?
    private var lastNativePlaybackURL: URL?
    private weak var currentNativePlayer: NativePlayerViewController?
    private let streamResolver: StreamResolving = StremioStreamResolver()

+
    private static let streamCandidateScript = WKUserScript(
15 unmodified lines
            /\.mp4(?:[?#]|$)/i
          ];
          const subtitleCandidates = [];
          const postedSubtitleURLs = new Set();
          const subtitleURLPattern = /https?:\/\/[^\s"'<>]+(?:\.srt|\.vtt|\.ass|\.ssa|\.sub|opensubtitles|subtitle)[^\s"'<>]*/ig;

+
          const looksNative = (url) => {
40 unmodified lines
            } catch (_) {}
          };

+
          const postSubtitleCandidates = (candidates) => {
            const fresh = candidates.filter((candidate) => {
              if (postedSubtitleURLs.has(candidate.url)) {
                return false;
              }
              postedSubtitleURLs.add(candidate.url);
              return true;
            });
            if (fresh.length === 0) {
              return;
            }
            try {
              window.webkit.messageHandlers.dreamioSubtitleCandidate.postMessage({
                pageUrl: window.location.href,
                subtitles: fresh
              });
            } catch (_) {}
          };

+
          const addSubtitleCandidate = (entry) => {
            const rawURL = typeof entry === "string" ? entry : entry && (entry.url || entry.href || entry.src || entry.file || entry.download);
            const url = absoluteURL(rawURL);
6 unmodified lines
            if (subtitleCandidates.some((candidate) => candidate.url === url)) {
              return;
            }
            const candidate = {
              url,
              label: entry && (entry.label || entry.name || entry.title || entry.lang || entry.language) || "External Subtitle",
              language: entry && (entry.lang || entry.language) || ""
            };
            subtitleCandidates.push(candidate);
            postSubtitleCandidates([candidate]);
          };

+
          const inspectSubtitlePayload = (payload) => {
280 unmodified lines

+
#if DEBUG
        let classification = request.classification
        print("[DreamioStream] class=\(classification.sourceKind.rawValue) container=\(classification.containerGuess.rawValue) reason=\(classification.reason) subtitles=\(request.subtitleCandidates.count) observed=\(classification.sanitizedObservedURL) resolver=\(classification.sanitizedResolverURL ?? "none")")
#endif

+
        Task { [weak self] in
1 unmodified line
        }
    }

+
    private func handleSubtitleCandidates(_ candidates: [SubtitleCandidate]) {
        guard !candidates.isEmpty else {
            return
        }

+
        guard let currentNativePlayer else {
#if DEBUG
            print("[DreamioSubtitles] discovered=\(candidates.count) forwarded=0 reason=no-active-native-player")
#endif
            return
        }

+
        let forwarded = currentNativePlayer.addSubtitleCandidates(candidates)
#if DEBUG
        print("[DreamioSubtitles] discovered=\(candidates.count) forwarded=\(forwarded)")
#endif
    }

+
    @MainActor
    private func resolveAndPresentNativePlayback(_ request: NativePlaybackRequest) async {
        guard VLCNativePlaybackBackend.isAvailable else {
19 unmodified lines
                subtitleCandidates: request.subtitleCandidates
            )
            let player = NativePlayerViewController(request: resolvedRequest)
            currentNativePlayer = player
            player.onDismiss = { [weak self] in
                self?.lastNativePlaybackURL = nil
                self?.currentNativePlayer = nil
                self?.cleanUpStremioPlayerAfterNativeDismiss()
            }
            present(player, animated: true)
206 unmodified lines
            return
        }

+
        if message.name == Constants.subtitleCandidateMessageHandler {
            handleSubtitleCandidates(SubtitleCandidateParser.candidates(in: message.body))
            return
        }

+
#if DEBUG
        guard message.name == Constants.diagnosticsMessageHandler,
              let body = message.body as? [String: Any],
+

Dreamio/NativePlayerViewController.swift

Dreamio/NativePlayerViewController.swift
+22
6 unmodified lines
7
8
9
10
11
12
81 unmodified lines
94
95
96
97
98
99
26 unmodified lines
126
127
128
129
130
131
6 unmodified lines
    private var controlsTimer: Timer?
    private var progressTimer: Timer?
    private var isScrubbing = false
    var onDismiss: (() -> Void)?

+
    private let loadingView: UIActivityIndicatorView = {
81 unmodified lines
    init(request: NativePlaybackRequest, backend: NativePlaybackBackend = VLCNativePlaybackBackend()) {
        self.request = request
        self.backend = backend
        super.init(nibName: nil, bundle: nil)
        modalPresentationStyle = .fullScreen
        modalTransitionStyle = .crossDissolve
26 unmodified lines
        backend.play(request: request)
    }

+
    override func viewDidDisappear(_ animated: Bool) {
        super.viewDidDisappear(animated)
        startupTimer?.invalidate()
6 unmodified lines
7
8
9
10
11
12
13
81 unmodified lines
95
96
97
98
99
100
101
26 unmodified lines
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
6 unmodified lines
    private var controlsTimer: Timer?
    private var progressTimer: Timer?
    private var isScrubbing = false
    private var attachedSubtitleURLs: Set<URL>
    var onDismiss: (() -> Void)?

+
    private let loadingView: UIActivityIndicatorView = {
81 unmodified lines
    init(request: NativePlaybackRequest, backend: NativePlaybackBackend = VLCNativePlaybackBackend()) {
        self.request = request
        self.backend = backend
        self.attachedSubtitleURLs = Set(request.subtitleCandidates.map(\.url))
        super.init(nibName: nil, bundle: nil)
        modalPresentationStyle = .fullScreen
        modalTransitionStyle = .crossDissolve
26 unmodified lines
        backend.play(request: request)
    }

+
    @discardableResult
    func addSubtitleCandidates(_ candidates: [SubtitleCandidate]) -> Int {
        let newCandidates = candidates.filter { candidate in
            guard !attachedSubtitleURLs.contains(candidate.url) else {
                return false
            }
            attachedSubtitleURLs.insert(candidate.url)
            return true
        }
        let attachedCount = backend.addSubtitleCandidates(newCandidates)
        if attachedCount > 0 {
            refreshControls()
        }
#if DEBUG
        let duplicateCount = candidates.count - newCandidates.count
        print("[DreamioNativePlayer] subtitle candidates=\(candidates.count) forwarded=\(newCandidates.count) attached=\(attachedCount) duplicates=\(duplicateCount)")
#endif
        return attachedCount
    }

+
    override func viewDidDisappear(_ animated: Bool) {
        super.viewDidDisappear(animated)
        startupTimer?.invalidate()
+

Dreamio/NativePlaybackBackend.swift

Dreamio/NativePlaybackBackend.swift
+2
24 unmodified lines
25
26
27
28
29
30
24 unmodified lines
    func jump(by seconds: TimeInterval)
    func selectSubtitleTrack(id: Int32)
    func adjustSubtitleDelay(by seconds: TimeInterval)
    func stop()
}

+
24 unmodified lines
25
26
27
28
29
30
31
32
24 unmodified lines
    func jump(by seconds: TimeInterval)
    func selectSubtitleTrack(id: Int32)
    func adjustSubtitleDelay(by seconds: TimeInterval)
    @discardableResult
    func addSubtitleCandidates(_ candidates: [SubtitleCandidate]) -> Int
    func stop()
}

+
+

Dreamio/VLCNativePlaybackBackend.swift

Dreamio/VLCNativePlaybackBackend.swift
-4+23
57 unmodified lines
58
59
60
61
62
63
64
48 unmodified lines
113
114
115
116
117
118
75 unmodified lines
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
57 unmodified lines
        print("[DreamioVLC] opening url=\(URLRedactor.redactedURLString(request.playbackURL.absoluteString))")
#endif
        mediaPlayer.play()
        attachSubtitles(request.subtitleCandidates)
#else
        onFailure?(NativePlaybackError.backendUnavailable)
#endif
48 unmodified lines
#endif
    }

+
    func stop() {
#if canImport(MobileVLCKit)
        mediaPlayer.stop()
75 unmodified lines
    }

+
#if canImport(MobileVLCKit)
    private func attachSubtitles(_ candidates: [SubtitleCandidate]) {
        candidates.forEach { candidate in
            guard !attachedSubtitleURLs.contains(candidate.url) else {
                return
            }
            attachedSubtitleURLs.insert(candidate.url)
            mediaPlayer.addPlaybackSlave(candidate.url, type: .subtitle, enforce: false)
#if DEBUG
            print("[DreamioVLC] attached subtitle=\(URLRedactor.redactedURLString(candidate.url.absoluteString))")
#endif
        }
        guard !candidates.isEmpty else {
            return
        }
        DispatchQueue.main.asyncAfter(deadline: .now() + 1) { [weak self] in
            self?.onSubtitleTracksChange?()
        }
    }
#endif
}
57 unmodified lines
58
59
60
61
62
63
64
48 unmodified lines
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
75 unmodified lines
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
57 unmodified lines
        print("[DreamioVLC] opening url=\(URLRedactor.redactedURLString(request.playbackURL.absoluteString))")
#endif
        mediaPlayer.play()
        addSubtitleCandidates(request.subtitleCandidates)
#else
        onFailure?(NativePlaybackError.backendUnavailable)
#endif
48 unmodified lines
#endif
    }

+
    @discardableResult
    func addSubtitleCandidates(_ candidates: [SubtitleCandidate]) -> Int {
#if canImport(MobileVLCKit)
        return attachSubtitles(candidates)
#else
        return 0
#endif
    }

+
    func stop() {
#if canImport(MobileVLCKit)
        mediaPlayer.stop()
75 unmodified lines
    }

+
#if canImport(MobileVLCKit)
    private func attachSubtitles(_ candidates: [SubtitleCandidate]) -> Int {
        var attachedCount = 0
        var duplicateCount = 0
        candidates.forEach { candidate in
            guard !attachedSubtitleURLs.contains(candidate.url) else {
                duplicateCount += 1
                return
            }
            attachedSubtitleURLs.insert(candidate.url)
            mediaPlayer.addPlaybackSlave(candidate.url, type: .subtitle, enforce: false)
            attachedCount += 1
#if DEBUG
            print("[DreamioVLC] attached subtitle=\(URLRedactor.redactedURLString(candidate.url.absoluteString))")
#endif
        }
#if DEBUG
        if !candidates.isEmpty {
            print("[DreamioVLC] subtitle candidates=\(candidates.count) attached=\(attachedCount) duplicates=\(duplicateCount)")
        }
#endif
        guard attachedCount > 0 else {
            return attachedCount
        }
        DispatchQueue.main.asyncAfter(deadline: .now() + 1) { [weak self] in
            self?.onSubtitleTracksChange?()
        }
        return attachedCount
    }
#endif
}
+

Tests/StreamResolverTests.swift

Tests/StreamResolverTests.swift
+61
8 unmodified lines
9
10
11
12
13
14
95 unmodified lines
110
111
112
113
114
115
8 unmodified lines
        testRedactorHandlesPercentEncodedPath()
        testPlaybackTimeFormatting()
        testSubtitleCandidateParsing()
        testSubtitleOptionMappingIncludesNone()
        print("StreamResolverTests passed")
    }
95 unmodified lines
        assertEqual(candidates[2].url.absoluteString, "https://cdn.example.test/movie.fr.ass?download=1")
    }

+
    private static func testSubtitleOptionMappingIncludesNone() {
        let options = SubtitleOptionMapper.options(from: [
            SubtitleTrack(id: 2, name: "English"),
8 unmodified lines
9
10
11
12
13
14
15
16
95 unmodified lines
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
8 unmodified lines
        testRedactorHandlesPercentEncodedPath()
        testPlaybackTimeFormatting()
        testSubtitleCandidateParsing()
        testOpenSubtitlesV3CandidateParsing()
        testSubtitleCandidateDeduplicationPreservesLabels()
        testSubtitleOptionMappingIncludesNone()
        print("StreamResolverTests passed")
    }
95 unmodified lines
        assertEqual(candidates[2].url.absoluteString, "https://cdn.example.test/movie.fr.ass?download=1")
    }

+
    private static func testOpenSubtitlesV3CandidateParsing() {
        let payload: [String: Any] = [
            "subtitles": [
                [
                    "language": "English",
                    "download": "https://api.opensubtitles.com/api/v1/download/subtitle-file",
                    "nested": [
                        [
                            "file": "https://dl.opensubtitles.org/en/subtitle.vtt?download=1"
                        ]
                    ]
                ],
                [
                    "lang": "spa",
                    "url": "https://opensubtitles.example.test/download/episode.srt"
                ]
            ],
            "body": "alternate https://cdn.example.test/from-string.ass?source=opensubtitles",
            "ignored": [
                "https://cdn.example.test/poster.jpg",
                ["file": "https://cdn.example.test/video.mkv"]
            ]
        ]

+
        let candidates = SubtitleCandidateParser.candidates(in: payload)

+
        assertEqual(candidates.count, 4)
        assertEqual(candidates[0].label, "English")
        assertEqual(candidates[0].language, "English")
        assertEqual(candidates[1].url.absoluteString, "https://dl.opensubtitles.org/en/subtitle.vtt?download=1")
        assertEqual(candidates[2].label, "spa")
        assertEqual(candidates[2].language, "spa")
        assertEqual(candidates[3].url.absoluteString, "https://cdn.example.test/from-string.ass?source=opensubtitles")
    }

+
    private static func testSubtitleCandidateDeduplicationPreservesLabels() {
        let payload: [String: Any] = [
            "subtitles": [
                [
                    "label": "English SDH",
                    "lang": "eng",
                    "url": "https://opensubtitles.example.test/download/duplicate.srt"
                ],
                [
                    "label": "Duplicate",
                    "language": "English",
                    "download": "https://opensubtitles.example.test/download/duplicate.srt"
                ],
                "https://opensubtitles.example.test/download/duplicate.srt"
            ]
        ]

+
        let candidates = SubtitleCandidateParser.candidates(in: payload)

+
        assertEqual(candidates.count, 1)
        assertEqual(candidates[0].label, "English SDH")
        assertEqual(candidates[0].language, "eng")
    }

+
    private static func testSubtitleOptionMappingIncludesNone() {
        let options = SubtitleOptionMapper.options(from: [
            SubtitleTrack(id: 2, name: "English"),
+
+ +
+

Expected Impact for End-Users

+

When a Stremio stream opens in the native player, OpenSubtitles tracks that arrive after VLC starts should still appear in the captions menu. Repeated addon/network responses should not pile up duplicate entries.

+
+ +
+

Validation

+
+
Passed: swiftc Dreamio/StreamCandidate.swift Dreamio/StreamResolver.swift Tests/StreamResolverTests.swift -o /tmp/dreamio-stream-tests && /tmp/dreamio-stream-tests
+
Passed: xcodebuild -workspace Dreamio.xcworkspace -scheme Dreamio -sdk iphonesimulator -configuration Debug build CODE_SIGNING_ALLOWED=NO
+
Completed with limitation: Beads issue dreamio-lw6 was closed locally. bd dolt pull reported no remote, and bd dolt push skipped because no Dolt remote is configured.
+
+
+ +
+

Issues, Limitations, and Mitigations

+
    +
  • Manual validation with a live OpenSubtitles V3-enabled title was not performed in this environment. The code path is covered by parser tests and a simulator build.
    • +
  • This keeps the assumption that discovered OpenSubtitles URLs are directly consumable by VLC. If a provider returns short-lived API URLs that require authenticated download, a cache/download layer may still be needed later.
    • +
  • Caption menu refresh relies on VLC reporting track changes or the delayed refresh after subtitle attachment.
    • +
  • Beads remote sync could not be performed because this workspace has no Dolt remote configured.
    • +
+
+ +
+

Follow-up Work

+
    +
  • Manually test on a real OpenSubtitles V3-enabled title and verify subtitle selection displays text in VLC.
    • +
  • Create a future Beads issue if live validation shows VLC cannot consume any discovered OpenSubtitles URLs directly.
    • +
+
+ +
+

New Changes as of May 25, 2026 at 7:07 AM EDT

+

Summary of changes

+

Hardened the injected network observers so Stremio can continue loading external subtitle files through binary or blob-style requests while Dreamio still observes text and JSON payloads for late subtitle candidates.

+

Why this change was made

+

After the late-subtitle bridge landed, Stremio external subtitle loading could fail when the XHR observer touched responseText for a non-text response. The observer now ignores non-text XHR payloads and catches inspection errors inside the load callback.

+

Code diffs

+

Dreamio/DreamioWebViewController.swift

Dreamio/DreamioWebViewController.swift
-2+19
193 unmodified lines
194
195
196
197
198
199
200
2 unmodified lines
203
204
205
206
207
208
209
193 unmodified lines
            window.fetch = async (...args) => {
              const response = await originalFetch(...args);
              try {
                response.clone().text().then(inspectSubtitlePayload).catch(() => {});
              } catch (_) {}
              return response;
            };
2 unmodified lines
          const originalXHRSend = XMLHttpRequest.prototype.send;
          XMLHttpRequest.prototype.send = function(...args) {
            try {
              this.addEventListener("load", () => inspectSubtitlePayload(this.responseText));
            } catch (_) {}
            return originalXHRSend.apply(this, args);
          };
193 unmodified lines
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
2 unmodified lines
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
193 unmodified lines
            window.fetch = async (...args) => {
              const response = await originalFetch(...args);
              try {
                const contentType = response.headers && response.headers.get("content-type") || "";
                const url = response.url || "";
                subtitleURLPattern.lastIndex = 0;
                const shouldInspect = !contentType
                  || /json|text|javascript|xml|subtitle|vtt|srt/i.test(contentType)
                  || subtitleURLPattern.test(url);
                if (shouldInspect) {
                  subtitleURLPattern.lastIndex = 0;
                  response.clone().text().then(inspectSubtitlePayload).catch(() => {});
                }
              } catch (_) {}
              return response;
            };
2 unmodified lines
          const originalXHRSend = XMLHttpRequest.prototype.send;
          XMLHttpRequest.prototype.send = function(...args) {
            try {
              this.addEventListener("load", () => {
                try {
                  const responseType = this.responseType || "";
                  if (responseType && responseType !== "text") {
                    return;
                  }
                  inspectSubtitlePayload(this.responseText);
                } catch (_) {}
              });
            } catch (_) {}
            return originalXHRSend.apply(this, args);
          };
+

Dreamio/StreamCandidate.swift

Dreamio/StreamCandidate.swift
-1+22
128 unmodified lines
129
130
131
132
133
134
135
23 unmodified lines
159
160
161
162
163
164
128 unmodified lines
            if let candidate = candidate(from: dictionary) {
                results.append(candidate)
            }
            dictionary.values.forEach { collect(from: $0, into: &results) }
        case let array as [Any]:
            array.forEach { collect(from: $0, into: &results) }
        case let string as String:
23 unmodified lines
        )
    }

+
    private static func subtitleURL(from string: String?) -> URL? {
        guard let string,
              let url = URL(string: string),
128 unmodified lines
129
130
131
132
133
134
135
23 unmodified lines
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
128 unmodified lines
            if let candidate = candidate(from: dictionary) {
                results.append(candidate)
            }
            orderedNestedValues(in: dictionary).forEach { collect(from: $0, into: &results) }
        case let array as [Any]:
            array.forEach { collect(from: $0, into: &results) }
        case let string as String:
23 unmodified lines
        )
    }

+
    private static func orderedNestedValues(in dictionary: [String: Any]) -> [Any] {
        let preferredKeys = ["subtitles", "subtitle", "files", "downloads", "download"]
        var visitedKeys = Set<String>()
        var values: [Any] = []

+
        preferredKeys.forEach { key in
            if let value = dictionary[key] {
                values.append(value)
                visitedKeys.insert(key)
            }
        }

+
        dictionary.keys
            .filter { !visitedKeys.contains($0) && !urlFields.contains($0) }
            .sorted()
            .compactMap { dictionary[$0] }
            .forEach { values.append($0) }

+
        return values
    }

+
    private static func subtitleURL(from string: String?) -> URL? {
        guard let string,
              let url = URL(string: string),
+

Related issues or PRs

+

Beads issue: dreamio-8cz.

+

Validation

+
+
Passed: swiftc Dreamio/StreamCandidate.swift Dreamio/StreamResolver.swift Tests/StreamResolverTests.swift -o /tmp/dreamio-stream-tests && /tmp/dreamio-stream-tests
+
Passed: xcodebuild -workspace Dreamio.xcworkspace -scheme Dreamio -sdk iphonesimulator -configuration Debug build CODE_SIGNING_ALLOWED=NO
+
+
+ +
Generated for repository turn documentation. Rendered diffs use @pierre/diffs/ssr.
+
+ + diff --git a/docs/turns/2026-05-25-native-player-audio-tracks.html b/docs/turns/2026-05-25-native-player-audio-tracks.html new file mode 100644 index 0000000..2fbb9bc --- /dev/null +++ b/docs/turns/2026-05-25-native-player-audio-tracks.html @@ -0,0 +1,421 @@ + + + + + + Native Player Audio Track Selection + + + +
+
+
Dreamio turn document · May 25, 2026
+

Native player audio track selection

+

Dreamio's VLC-backed native player now exposes embedded audio tracks, adds a far-left audio menu to the control bar, and refreshes the player chrome with a more iOS-native glass treatment.

+
+ +
+

Summary

+

Added audio track discovery and selection to native playback so multi-language MKV and similar files can be switched without leaving the player.

+
+ +
+

Changes Made

+
    +
  • Extended NativePlaybackBackend with audio track state, a selection API, and an audio-track-change callback.
    • +
  • Read MobileVLCKit audio track names, indexes, and current selection from VLCMediaPlayer.
    • +
  • Added an audio menu button on the far left side of the native controls using the waveform.circle symbol.
    • +
  • Grouped skip/play/skip into a centered playback cluster so the play button stays visually centered.
    • +
  • Updated the control surface and buttons with translucent material, softer radius, subtle borders, and lighter glass-like control wells.
    • +
+
+ +
+

Context

+

The player already exposed embedded subtitle tracks through MobileVLCKit. The same streams often include multiple audio tracks, such as alternate languages or commentary, but the native player had no way to inspect or switch them.

+

The user-provided diagnostics showed VLC discovering embedded subtitle tracks while the UI still lacked track filtering for audio. This change follows the existing subtitle menu pattern instead of creating a separate player path.

+
+ +
+

Important Implementation Details

+
    +
  • AudioTrack currently aliases the existing SubtitleTrack value shape because both VLC APIs expose an integer id and display name.
    • +
  • The audio button is enabled only when VLC reports more than one selectable audio track, keeping single-track files quiet.
    • +
  • The backend fires onAudioTracksChange when VLC enters playback/buffering and when elementary streams are added.
    • +
  • Selection is applied with mediaPlayer.currentAudioTrackIndex, matching the existing subtitle selection style.
    • +
+
+ +
+

Relevant Diff Snippets

+

Rendered with @pierre/diffs/ssr from the current git diff.

+
+

Dreamio/NativePlaybackBackend.swift

Dreamio/NativePlaybackBackend.swift
+4
5 unmodified lines
6
7
8
9
10
11
12
13
14
15
16
17
5 unmodified lines
23
24
25
26
27
28
5 unmodified lines
    var onFailure: ((Error) -> Void)? { get set }
    var onStateChange: (() -> Void)? { get set }
    var onSubtitleTracksChange: (() -> Void)? { get set }
    var isPlaying: Bool { get }
    var isSeekable: Bool { get }
    var duration: TimeInterval { get }
    var currentTime: TimeInterval { get }
    var remainingTime: TimeInterval { get }
    var position: Float { get }
    var subtitleTracks: [SubtitleTrack] { get }
    var selectedSubtitleTrackID: Int32 { get }
    var subtitleDelay: TimeInterval { get }
5 unmodified lines
    func togglePlayPause()
    func seek(to position: Float)
    func jump(by seconds: TimeInterval)
    func selectSubtitleTrack(id: Int32)
    func adjustSubtitleDelay(by seconds: TimeInterval)
    @discardableResult
5 unmodified lines
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
5 unmodified lines
26
27
28
29
30
31
32
5 unmodified lines
    var onFailure: ((Error) -> Void)? { get set }
    var onStateChange: (() -> Void)? { get set }
    var onSubtitleTracksChange: (() -> Void)? { get set }
    var onAudioTracksChange: (() -> Void)? { get set }
    var isPlaying: Bool { get }
    var isSeekable: Bool { get }
    var duration: TimeInterval { get }
    var currentTime: TimeInterval { get }
    var remainingTime: TimeInterval { get }
    var position: Float { get }
    var audioTracks: [AudioTrack] { get }
    var selectedAudioTrackID: Int32 { get }
    var subtitleTracks: [SubtitleTrack] { get }
    var selectedSubtitleTrackID: Int32 { get }
    var subtitleDelay: TimeInterval { get }
5 unmodified lines
    func togglePlayPause()
    func seek(to position: Float)
    func jump(by seconds: TimeInterval)
    func selectAudioTrack(id: Int32)
    func selectSubtitleTrack(id: Int32)
    func adjustSubtitleDelay(by seconds: TimeInterval)
    @discardableResult
+

Dreamio/StreamCandidate.swift

Dreamio/StreamCandidate.swift
+8
39 unmodified lines
40
41
42
43
44
45
47 unmodified lines
93
94
95
96
97
98
39 unmodified lines
    let name: String
}

+
#if DEBUG
enum SubtitleDebugFormatter {
    static func candidateSummary(_ candidates: [SubtitleCandidate]) -> String {
47 unmodified lines
    }
}

+
struct StreamClassification {
    let sourceKind: StreamSourceKind
    let containerGuess: StreamContainerGuess
39 unmodified lines
40
41
42
43
44
45
46
47
47 unmodified lines
95
96
97
98
99
100
101
102
103
104
105
106
39 unmodified lines
    let name: String
}

+
typealias AudioTrack = SubtitleTrack

+
#if DEBUG
enum SubtitleDebugFormatter {
    static func candidateSummary(_ candidates: [SubtitleCandidate]) -> String {
47 unmodified lines
    }
}

+
enum AudioOptionMapper {
    static func options(from tracks: [AudioTrack]) -> [AudioTrack] {
        tracks.filter { $0.id >= 0 }
    }
}

+
struct StreamClassification {
    let sourceKind: StreamSourceKind
    let containerGuess: StreamContainerGuess
+

Dreamio/VLCNativePlaybackBackend.swift

Dreamio/VLCNativePlaybackBackend.swift
+43
17 unmodified lines
18
19
20
21
22
23
80 unmodified lines
104
105
106
107
108
109
83 unmodified lines
193
194
195
196
197
198
58 unmodified lines
257
258
259
260
261
262
52 unmodified lines
315
316
317
318
319
320
1 unmodified line
322
323
324
325
326
327
328
329
17 unmodified lines
    var onFailure: ((Error) -> Void)?
    var onStateChange: (() -> Void)?
    var onSubtitleTracksChange: (() -> Void)?

+
#if canImport(MobileVLCKit)
    private let mediaPlayer = VLCMediaPlayer()
80 unmodified lines
#endif
    }

+
    func selectSubtitleTrack(id: Int32) {
#if canImport(MobileVLCKit)
        didUserSelectSubtitleTrack = true
83 unmodified lines
#endif
    }

+
    var subtitleTracks: [SubtitleTrack] {
#if canImport(MobileVLCKit)
        let names = mediaPlayer.videoSubTitlesNames as? [String] ?? []
58 unmodified lines
    }

+
#if DEBUG
    private func logSubtitleTracks(reason: String) {
        let names = mediaPlayer.videoSubTitlesNames as? [String] ?? []
        let indexes = mediaPlayer.videoSubTitlesIndexes as? [NSNumber] ?? []
52 unmodified lines
            reapplyAutoSelectedSubtitleTrackIfNeeded(reason: stateName(mediaPlayer.state))
            onReady?()
            onStateChange?()
        case .error:
            onFailure?(NativePlaybackError.playbackFailed)
        case .paused, .stopped, .ended:
1 unmodified line
        case .esAdded:
            selectInitialSubtitleTrackIfNeeded(reason: "esAdded")
#if DEBUG
            logSubtitleTracks(reason: "esAdded")
#endif
            onSubtitleTracksChange?()
        default:
            break
17 unmodified lines
18
19
20
21
22
23
24
80 unmodified lines
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
83 unmodified lines
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
58 unmodified lines
291
292
293
294
295
296
297
298
299
300
301
302
52 unmodified lines
355
356
357
358
359
360
361
1 unmodified line
363
364
365
366
367
368
369
370
371
372
17 unmodified lines
    var onFailure: ((Error) -> Void)?
    var onStateChange: (() -> Void)?
    var onSubtitleTracksChange: (() -> Void)?
    var onAudioTracksChange: (() -> Void)?

+
#if canImport(MobileVLCKit)
    private let mediaPlayer = VLCMediaPlayer()
80 unmodified lines
#endif
    }

+
    func selectAudioTrack(id: Int32) {
#if canImport(MobileVLCKit)
#if DEBUG
        logAudioTracks(reason: "before-select-\(id)")
#endif
        mediaPlayer.currentAudioTrackIndex = id
#if DEBUG
        logAudioTracks(reason: "after-select-\(id)")
#endif
        onAudioTracksChange?()
#endif
    }

+
    func selectSubtitleTrack(id: Int32) {
#if canImport(MobileVLCKit)
        didUserSelectSubtitleTrack = true
83 unmodified lines
#endif
    }

+
    var audioTracks: [AudioTrack] {
#if canImport(MobileVLCKit)
        let names = mediaPlayer.audioTrackNames as? [String] ?? []
        let indexes = mediaPlayer.audioTrackIndexes as? [NSNumber] ?? []
        return zip(indexes, names).map { index, name in
            AudioTrack(id: index.int32Value, name: name)
        }
#else
        []
#endif
    }

+
    var selectedAudioTrackID: Int32 {
#if canImport(MobileVLCKit)
        mediaPlayer.currentAudioTrackIndex
#else
        -1
#endif
    }

+
    var subtitleTracks: [SubtitleTrack] {
#if canImport(MobileVLCKit)
        let names = mediaPlayer.videoSubTitlesNames as? [String] ?? []
58 unmodified lines
    }

+
#if DEBUG
    private func logAudioTracks(reason: String) {
        let names = mediaPlayer.audioTrackNames as? [String] ?? []
        let indexes = mediaPlayer.audioTrackIndexes as? [NSNumber] ?? []
        print("[DreamioVLC] audio tracks reason=\(reason) names=\(names) indexes=\(indexes.map { $0.int32Value }) selected=\(mediaPlayer.currentAudioTrackIndex)")
    }

+
    private func logSubtitleTracks(reason: String) {
        let names = mediaPlayer.videoSubTitlesNames as? [String] ?? []
        let indexes = mediaPlayer.videoSubTitlesIndexes as? [NSNumber] ?? []
52 unmodified lines
            reapplyAutoSelectedSubtitleTrackIfNeeded(reason: stateName(mediaPlayer.state))
            onReady?()
            onStateChange?()
            onAudioTracksChange?()
        case .error:
            onFailure?(NativePlaybackError.playbackFailed)
        case .paused, .stopped, .ended:
1 unmodified line
        case .esAdded:
            selectInitialSubtitleTrackIfNeeded(reason: "esAdded")
#if DEBUG
            logAudioTracks(reason: "esAdded")
            logSubtitleTracks(reason: "esAdded")
#endif
            onAudioTracksChange?()
            onSubtitleTracksChange?()
        default:
            break
+

Dreamio/NativePlayerViewController.swift

Dreamio/NativePlayerViewController.swift
-7+90
8 unmodified lines
9
10
11
12
13
14
19 unmodified lines
34
35
36
37
38
39
40
41
7 unmodified lines
49
50
51
52
53
54
174 unmodified lines
229
230
231
232
233
234
15 unmodified lines
250
251
252
253
254
255
256
257
8 unmodified lines
266
267
268
269
270
271
272
273
274
275
276
277
44 unmodified lines
322
323
324
325
326
327
102 unmodified lines
430
431
432
433
434
435
2 unmodified lines
438
439
440
441
442
443
444
445
446
447
448
3 unmodified lines
452
453
454
455
456
457
18 unmodified lines
476
477
478
479
480
481
482
483
484
485
486
30 unmodified lines
517
518
519
520
521
522
523
524
8 unmodified lines
    private var progressTimer: Timer?
    private var isScrubbing = false
    private var attachedSubtitleURLs: Set<URL>
    private var captionsMenuSignature: String?
    var onDismiss: (() -> Void)?

+
19 unmodified lines
    private let controlsContainer: UIVisualEffectView = {
        let view = UIVisualEffectView(effect: UIBlurEffect(style: .systemUltraThinMaterialDark))
        view.translatesAutoresizingMaskIntoConstraints = false
        view.layer.cornerRadius = 16
        view.clipsToBounds = true
        return view
    }()

+
7 unmodified lines
    private let playPauseButton = NativePlayerViewController.iconButton(systemName: "pause.fill", label: "Play or Pause")
    private let jumpBackButton = NativePlayerViewController.iconButton(systemName: "gobackward.15", label: "Jump Back 15 Seconds")
    private let jumpForwardButton = NativePlayerViewController.iconButton(systemName: "goforward.15", label: "Jump Forward 15 Seconds")
    private let captionsButton = NativePlayerViewController.iconButton(systemName: "captions.bubble", label: "Captions")

+
    private let elapsedLabel: UILabel = {
174 unmodified lines
                self?.refreshControls()
            }
        }
    }

+
    private func startStartupTimer() {
15 unmodified lines
        playPauseButton.addTarget(self, action: #selector(togglePlayPause), for: .touchUpInside)
        jumpBackButton.addTarget(self, action: #selector(jumpBack), for: .touchUpInside)
        jumpForwardButton.addTarget(self, action: #selector(jumpForward), for: .touchUpInside)
        captionsButton.showsMenuAsPrimaryAction = true
        playPauseButton.layer.cornerRadius = 21
        scrubber.addTarget(self, action: #selector(scrubbingStarted), for: .touchDown)
        scrubber.addTarget(self, action: #selector(scrubberChanged), for: .valueChanged)
        scrubber.addTarget(self, action: #selector(scrubbingEnded), for: [.touchUpInside, .touchUpOutside, .touchCancel])
8 unmodified lines
        timeAndScrubRow.alignment = .center
        timeAndScrubRow.spacing = 8

+
        let controlRow = UIStackView(arrangedSubviews: [jumpBackButton, playPauseButton, jumpForwardButton, captionsButton])
        controlRow.translatesAutoresizingMaskIntoConstraints = false
        controlRow.axis = .horizontal
        controlRow.alignment = .center
        controlRow.distribution = .equalSpacing
        controlRow.spacing = 14

+
        let stack = UIStackView(arrangedSubviews: [timeAndScrubRow, controlRow])
        stack.translatesAutoresizingMaskIntoConstraints = false
44 unmodified lines
            playPauseButton.heightAnchor.constraint(equalToConstant: 42),
            jumpForwardButton.widthAnchor.constraint(equalToConstant: 36),
            jumpForwardButton.heightAnchor.constraint(equalToConstant: 36),
            captionsButton.widthAnchor.constraint(equalToConstant: 36),
            captionsButton.heightAnchor.constraint(equalToConstant: 36)
        ])
102 unmodified lines
        return UIMenu(title: "Captions", children: trackActions + [delayActions])
    }

+
    private func startProgressUpdates() {
        progressTimer?.invalidate()
        progressTimer = Timer.scheduledTimer(withTimeInterval: 0.5, repeats: true) { [weak self] _ in
2 unmodified lines
    }

+
    private func refreshControls() {
        let subtitleTracks = backend.subtitleTracks
        playPauseButton.setImage(UIImage(systemName: backend.isPlaying ? "pause.fill" : "play.fill"), for: .normal)
        scrubber.isEnabled = backend.isSeekable
        jumpBackButton.isEnabled = backend.isSeekable
        jumpForwardButton.isEnabled = backend.isSeekable
        updateCaptionsMenuIfNeeded(subtitleTracks: subtitleTracks)
        elapsedLabel.text = PlaybackTimeFormatter.label(for: backend.currentTime)
        remainingLabel.text = "-\(PlaybackTimeFormatter.label(for: backend.remainingTime))"
3 unmodified lines
        [scrubber, jumpBackButton, jumpForwardButton].forEach { $0.alpha = backend.isSeekable ? 1 : 0.45 }
    }

+
    private func updateCaptionsMenuIfNeeded(subtitleTracks: [SubtitleTrack]) {
        let selectedTrackID = backend.selectedSubtitleTrackID
        let signature = captionsMenuSignatureValue(
18 unmodified lines
        tracks: [SubtitleTrack],
        selectedTrackID: Int32,
        delay: TimeInterval
    ) -> String {
        let trackSignature = tracks
            .map { "\($0.id):\($0.name)" }
            .joined(separator: "|")
        return "\(trackSignature)#selected=\(selectedTrackID)#delay=\(String(format: "%.1f", delay))"
    }

+
    private func revealControls() {
30 unmodified lines
        button.translatesAutoresizingMaskIntoConstraints = false
        button.setImage(UIImage(systemName: systemName), for: .normal)
        button.tintColor = .white
        button.backgroundColor = UIColor.black.withAlphaComponent(0.35)
        button.layer.cornerRadius = 18
        button.accessibilityLabel = label
        return button
    }
8 unmodified lines
9
10
11
12
13
14
15
19 unmodified lines
35
36
37
38
39
40
41
42
43
44
45
7 unmodified lines
53
54
55
56
57
58
59
174 unmodified lines
234
235
236
237
238
239
240
241
242
243
244
15 unmodified lines
260
261
262
263
264
265
266
267
268
8 unmodified lines
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
44 unmodified lines
340
341
342
343
344
345
346
347
348
102 unmodified lines
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
2 unmodified lines
489
490
491
492
493
494
495
496
497
498
499
500
501
3 unmodified lines
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
18 unmodified lines
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
30 unmodified lines
598
599
600
601
602
603
604
605
606
607
8 unmodified lines
    private var progressTimer: Timer?
    private var isScrubbing = false
    private var attachedSubtitleURLs: Set<URL>
    private var audioMenuSignature: String?
    private var captionsMenuSignature: String?
    var onDismiss: (() -> Void)?

+
19 unmodified lines
    private let controlsContainer: UIVisualEffectView = {
        let view = UIVisualEffectView(effect: UIBlurEffect(style: .systemUltraThinMaterialDark))
        view.translatesAutoresizingMaskIntoConstraints = false
        view.layer.cornerRadius = 22
        view.clipsToBounds = true
        view.backgroundColor = UIColor.white.withAlphaComponent(0.08)
        view.layer.borderColor = UIColor.white.withAlphaComponent(0.18).cgColor
        view.layer.borderWidth = 1
        return view
    }()

+
7 unmodified lines
    private let playPauseButton = NativePlayerViewController.iconButton(systemName: "pause.fill", label: "Play or Pause")
    private let jumpBackButton = NativePlayerViewController.iconButton(systemName: "gobackward.15", label: "Jump Back 15 Seconds")
    private let jumpForwardButton = NativePlayerViewController.iconButton(systemName: "goforward.15", label: "Jump Forward 15 Seconds")
    private let audioButton = NativePlayerViewController.iconButton(systemName: "waveform.circle", label: "Audio Tracks")
    private let captionsButton = NativePlayerViewController.iconButton(systemName: "captions.bubble", label: "Captions")

+
    private let elapsedLabel: UILabel = {
174 unmodified lines
                self?.refreshControls()
            }
        }
        backend.onAudioTracksChange = { [weak self] in
            DispatchQueue.main.async {
                self?.refreshControls()
            }
        }
    }

+
    private func startStartupTimer() {
15 unmodified lines
        playPauseButton.addTarget(self, action: #selector(togglePlayPause), for: .touchUpInside)
        jumpBackButton.addTarget(self, action: #selector(jumpBack), for: .touchUpInside)
        jumpForwardButton.addTarget(self, action: #selector(jumpForward), for: .touchUpInside)
        audioButton.showsMenuAsPrimaryAction = true
        captionsButton.showsMenuAsPrimaryAction = true
        playPauseButton.layer.cornerRadius = 24
        scrubber.addTarget(self, action: #selector(scrubbingStarted), for: .touchDown)
        scrubber.addTarget(self, action: #selector(scrubberChanged), for: .valueChanged)
        scrubber.addTarget(self, action: #selector(scrubbingEnded), for: [.touchUpInside, .touchUpOutside, .touchCancel])
8 unmodified lines
        timeAndScrubRow.alignment = .center
        timeAndScrubRow.spacing = 8

+
        let playbackCluster = UIStackView(arrangedSubviews: [jumpBackButton, playPauseButton, jumpForwardButton])
        playbackCluster.translatesAutoresizingMaskIntoConstraints = false
        playbackCluster.axis = .horizontal
        playbackCluster.alignment = .center
        playbackCluster.distribution = .equalCentering
        playbackCluster.spacing = 14

+
        let controlRow = UIStackView(arrangedSubviews: [audioButton, playbackCluster, captionsButton])
        controlRow.translatesAutoresizingMaskIntoConstraints = false
        controlRow.axis = .horizontal
        controlRow.alignment = .center
        controlRow.distribution = .equalCentering
        controlRow.spacing = 18

+
        let stack = UIStackView(arrangedSubviews: [timeAndScrubRow, controlRow])
        stack.translatesAutoresizingMaskIntoConstraints = false
44 unmodified lines
            playPauseButton.heightAnchor.constraint(equalToConstant: 42),
            jumpForwardButton.widthAnchor.constraint(equalToConstant: 36),
            jumpForwardButton.heightAnchor.constraint(equalToConstant: 36),
            audioButton.widthAnchor.constraint(equalToConstant: 36),
            audioButton.heightAnchor.constraint(equalToConstant: 36),
            playbackCluster.centerXAnchor.constraint(equalTo: controlRow.centerXAnchor),
            captionsButton.widthAnchor.constraint(equalToConstant: 36),
            captionsButton.heightAnchor.constraint(equalToConstant: 36)
        ])
102 unmodified lines
        return UIMenu(title: "Captions", children: trackActions + [delayActions])
    }

+
    private func audioMenu() -> UIMenu {
        let selectedTrackID = backend.selectedAudioTrackID
        let tracks = backend.audioTracks
        let options = AudioOptionMapper.options(from: tracks)
#if DEBUG
        print("[DreamioAudio] build-menu tracks=\(SubtitleDebugFormatter.trackSummary(tracks)) selected=\(selectedTrackID)")
#endif
        let trackActions = options.map { track in
            UIAction(
                title: track.name,
                state: track.id == selectedTrackID ? .on : .off
            ) { [weak self] _ in
                guard let self else {
                    return
                }
#if DEBUG
                print("[DreamioAudio] select-request id=\(track.id) name=\(track.name) before=\(self.backend.selectedAudioTrackID)")
#endif
                self.backend.selectAudioTrack(id: track.id)
#if DEBUG
                print("[DreamioAudio] select-result id=\(track.id) after=\(self.backend.selectedAudioTrackID) tracks=\(SubtitleDebugFormatter.trackSummary(self.backend.audioTracks))")
#endif
                self.audioMenuSignature = nil
                self.refreshControls()
            }
        }

+
        return UIMenu(title: "Audio", children: trackActions)
    }

+
    private func startProgressUpdates() {
        progressTimer?.invalidate()
        progressTimer = Timer.scheduledTimer(withTimeInterval: 0.5, repeats: true) { [weak self] _ in
2 unmodified lines
    }

+
    private func refreshControls() {
        let audioTracks = backend.audioTracks
        let subtitleTracks = backend.subtitleTracks
        playPauseButton.setImage(UIImage(systemName: backend.isPlaying ? "pause.fill" : "play.fill"), for: .normal)
        scrubber.isEnabled = backend.isSeekable
        jumpBackButton.isEnabled = backend.isSeekable
        jumpForwardButton.isEnabled = backend.isSeekable
        updateAudioMenuIfNeeded(audioTracks: audioTracks)
        updateCaptionsMenuIfNeeded(subtitleTracks: subtitleTracks)
        elapsedLabel.text = PlaybackTimeFormatter.label(for: backend.currentTime)
        remainingLabel.text = "-\(PlaybackTimeFormatter.label(for: backend.remainingTime))"
3 unmodified lines
        [scrubber, jumpBackButton, jumpForwardButton].forEach { $0.alpha = backend.isSeekable ? 1 : 0.45 }
    }

+
    private func updateAudioMenuIfNeeded(audioTracks: [AudioTrack]) {
        let selectedTrackID = backend.selectedAudioTrackID
        let signature = trackMenuSignatureValue(
            tracks: audioTracks,
            selectedTrackID: selectedTrackID
        )
        let hasSelectableTrack = AudioOptionMapper.options(from: audioTracks).count > 1
        audioButton.isEnabled = hasSelectableTrack
        audioButton.alpha = hasSelectableTrack ? 1 : 0.45
        guard signature != audioMenuSignature else {
            return
        }

+
        audioMenuSignature = signature
        audioButton.menu = audioMenu()
#if DEBUG
        print("[DreamioAudio] refresh-menu enabled=\(audioButton.isEnabled) tracks=\(SubtitleDebugFormatter.trackSummary(audioTracks)) selected=\(selectedTrackID)")
#endif
    }

+
    private func updateCaptionsMenuIfNeeded(subtitleTracks: [SubtitleTrack]) {
        let selectedTrackID = backend.selectedSubtitleTrackID
        let signature = captionsMenuSignatureValue(
18 unmodified lines
        tracks: [SubtitleTrack],
        selectedTrackID: Int32,
        delay: TimeInterval
    ) -> String {
        let trackSignature = trackMenuSignatureValue(tracks: tracks, selectedTrackID: selectedTrackID)
        return "\(trackSignature)#delay=\(String(format: "%.1f", delay))"
    }

+
    private func trackMenuSignatureValue(
        tracks: [SubtitleTrack],
        selectedTrackID: Int32
    ) -> String {
        let trackSignature = tracks
            .map { "\($0.id):\($0.name)" }
            .joined(separator: "|")
        return "\(trackSignature)#selected=\(selectedTrackID)"
    }

+
    private func revealControls() {
30 unmodified lines
        button.translatesAutoresizingMaskIntoConstraints = false
        button.setImage(UIImage(systemName: systemName), for: .normal)
        button.tintColor = .white
        button.backgroundColor = UIColor.white.withAlphaComponent(0.12)
        button.layer.cornerRadius = 18
        button.layer.borderColor = UIColor.white.withAlphaComponent(0.16).cgColor
        button.layer.borderWidth = 1
        button.accessibilityLabel = label
        return button
    }
+ +
+

Expected Impact for End-Users

+

When a native-played file has multiple embedded audio tracks, users can open the new audio menu and choose the language or alternate mix they want. The play button remains centered, and the controls should feel more at home on iOS.

+
+ +
+

Validation

+
    +
  • Ran xcodebuild -workspace Dreamio.xcworkspace -scheme Dreamio -destination 'platform=iOS Simulator,name=iPhone 17' build: passed.
    • +
  • Attempted iPhone 16 simulator validation first, but that simulator was not installed on this machine.
    • +
  • No real-device playback stream was available in this turn, so actual multi-audio switching still needs a device check with an MKV that contains multiple audio languages.
    • +
+
+ +
+

Issues, Limitations, and Mitigations

+
    +
  • MobileVLCKit track discovery is event-driven, so the menu appears after VLC reports the stream tracks. The UI refreshes on playback, buffering, and elementary-stream-added events to mitigate delayed discovery.
    • +
  • The visual update uses UIKit blur/material styling rather than directly adopting iOS 26-only UIGlassEffect, keeping the project buildable for its current iOS 16 deployment target while following Liquid Glass principles.
    • +
  • Manual validation is still needed on a device with a known multi-audio stream.
    • +
+
+ +
+

Follow-up Work

+
    +
  • Use a known multi-language MKV on device and confirm the menu lists all audio tracks and switches without playback restart.
    • +
  • Consider parsing VLC track names into cleaner language labels if raw names are noisy.
    • +
  • Promote the track model from a typealias to distinct audio/subtitle structs if audio-specific metadata is added later.
    • +
+
+
+ + \ No newline at end of file diff --git a/docs/turns/2026-05-25-prove-native-subtitle-pipeline.html b/docs/turns/2026-05-25-prove-native-subtitle-pipeline.html new file mode 100644 index 0000000..34e226e --- /dev/null +++ b/docs/turns/2026-05-25-prove-native-subtitle-pipeline.html @@ -0,0 +1,704 @@ + + + + + + Prove Native Subtitle Pipeline + + + +
+
+
Turn document, 2026-05-25
+

Prove Native Subtitle Pipeline

+

Added targeted DEBUG-only logging across subtitle discovery, web-to-native forwarding, native resolution, VLC subtitle attachment, and VLC track exposure. The change is diagnostic only and keeps URL output redacted.

+
+ Issue: dreamio-e9p + Scope: diagnostics only + Validation: build passed +
+
+ +
+

Summary

+

The native subtitle path now reports enough DEBUG data to tell whether subtitles disappear during web discovery, bridge forwarding, native player timing, subtitle resolution, VLC attachment, or VLC track enumeration.

+
+ +
+

Changes Made

+
    +
  • Added web bridge metadata for subtitle discovery, dedupe, and post counts.
    • +
  • Logged subtitle bridge messages after native parsing, including whether a native player is active.
    • +
  • Logged native player forwarding, resolution, duplicate filtering, attachment counts, resulting subtitle tracks, and selected track id.
    • +
  • Logged VLC track state after addPlaybackSlave, after the delayed refresh, and when VLC reports .esAdded.
    • +
  • Added DEBUG-only formatting helpers for subtitle candidates and tracks that include labels, languages, extensions, names, indexes, and selected ids without exposing full subtitle URLs.
    • +
+
+ +
+

Context

+

The current failure mode has already shown native playback starting with subtitles=0. This pass avoids behavior changes and instead makes the next Xcode run produce proof about which stage has zero subtitles or loses them.

+
+ +
+

Important Implementation Details

+
    +
  • Swift logs are wrapped in #if DEBUG, so release behavior is unchanged.
    • +
  • The injected web script now includes debug count metadata in subtitle bridge messages. It still sends the same subtitle payload shape used by native parsing.
    • +
  • Full URLs remain redacted through existing URLRedactor where URLs are printed. Candidate summaries intentionally show only label, language, and file extension.
    • +
  • The VLC logs print videoSubTitlesNames, videoSubTitlesIndexes, and currentVideoSubTitleIndex at the moments most relevant to track exposure.
    • +
+
+ +
+

Relevant Diff Snippets

+

Rendered with @pierre/diffs/ssr using one file diff per render.

+

DreamioWebViewController.swift

Dreamio/DreamioWebViewController.swift
-3+37
126 unmodified lines
127
128
129
130
131
132
2 unmodified lines
135
136
137
138
139
140
141
142
143
144
145
146
333 unmodified lines
480
481
482
483
484
485
3 unmodified lines
489
490
491
492
493
494
495
162 unmodified lines
658
659
660
661
662
663
71 unmodified lines
735
736
737
738
739
740
741
126 unmodified lines
          };

+
          const postSubtitleCandidates = (candidates) => {
            const fresh = candidates.filter((candidate) => {
              if (postedSubtitleURLs.has(candidate.url)) {
                return false;
2 unmodified lines
              return true;
            });
            if (fresh.length === 0) {
              return;
            }
            try {
              window.webkit.messageHandlers.dreamioSubtitleCandidate.postMessage({
                pageUrl: window.location.href,
                subtitles: fresh
              });
            } catch (_) {}
          };
333 unmodified lines
            return
        }

+
        guard let currentNativePlayer else {
#if DEBUG
            print("[DreamioSubtitles] discovered=\(candidates.count) forwarded=0 reason=no-active-native-player")
3 unmodified lines

+
        let forwarded = currentNativePlayer.addSubtitleCandidates(candidates)
#if DEBUG
        print("[DreamioSubtitles] discovered=\(candidates.count) forwarded=\(forwarded)")
#endif
    }

+
162 unmodified lines
    private func redactedURLString(_ value: String) -> String {
        URLRedactor.redactedURLString(value)
    }
#endif
}

+
71 unmodified lines
        }

+
        if message.name == Constants.subtitleCandidateMessageHandler {
            handleSubtitleCandidates(SubtitleCandidateParser.candidates(in: message.body))
            return
        }

+
126 unmodified lines
127
128
129
130
131
132
133
2 unmodified lines
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
333 unmodified lines
497
498
499
500
501
502
503
504
505
3 unmodified lines
509
510
511
512
513
514
515
162 unmodified lines
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
71 unmodified lines
765
766
767
768
769
770
771
772
773
774
775
126 unmodified lines
          };

+
          const postSubtitleCandidates = (candidates) => {
            const discoveredCount = candidates.length;
            const fresh = candidates.filter((candidate) => {
              if (postedSubtitleURLs.has(candidate.url)) {
                return false;
2 unmodified lines
              return true;
            });
            if (fresh.length === 0) {
              try {
                window.webkit.messageHandlers.dreamioSubtitleCandidate.postMessage({
                  pageUrl: window.location.href,
                  subtitles: [],
                  debug: {
                    discovered: discoveredCount,
                    deduped: 0,
                    forwarded: 0
                  }
                });
              } catch (_) {}
              return;
            }
            try {
              window.webkit.messageHandlers.dreamioSubtitleCandidate.postMessage({
                pageUrl: window.location.href,
                subtitles: fresh,
                debug: {
                  discovered: discoveredCount,
                  deduped: fresh.length,
                  forwarded: fresh.length
                }
              });
            } catch (_) {}
          };
333 unmodified lines
            return
        }

+
#if DEBUG
        print("[DreamioSubtitles] native discovered=\(candidates.count) playerActive=\(currentNativePlayer != nil) candidates=\(SubtitleDebugFormatter.candidateSummary(candidates))")
#endif
        guard let currentNativePlayer else {
#if DEBUG
            print("[DreamioSubtitles] discovered=\(candidates.count) forwarded=0 reason=no-active-native-player")
3 unmodified lines

+
        let forwarded = currentNativePlayer.addSubtitleCandidates(candidates)
#if DEBUG
        print("[DreamioSubtitles] discovered=\(candidates.count) forwarded=\(forwarded) reason=active-native-player")
#endif
    }

+
162 unmodified lines
    private func redactedURLString(_ value: String) -> String {
        URLRedactor.redactedURLString(value)
    }

+
    private func logSubtitleBridgeMessage(_ body: Any, parsedCandidates: [SubtitleCandidate]) {
        let dictionary = body as? [String: Any]
        let debug = dictionary?["debug"] as? [String: Any]
        let discovered = debug?["discovered"] as? Int ?? parsedCandidates.count
        let deduped = debug?["deduped"] as? Int ?? parsedCandidates.count
        let posted = debug?["forwarded"] as? Int ?? parsedCandidates.count
        let pageURL = dictionary?["pageUrl"] as? String
        print("[DreamioSubtitles] bridge discovered=\(discovered) deduped=\(deduped) posted=\(posted) parsed=\(parsedCandidates.count) playerActive=\(currentNativePlayer != nil) page=\(pageURL.map(redactedURLString) ?? "unknown") candidates=\(SubtitleDebugFormatter.candidateSummary(parsedCandidates))")
    }
#endif
}

+
71 unmodified lines
        }

+
        if message.name == Constants.subtitleCandidateMessageHandler {
            let candidates = SubtitleCandidateParser.candidates(in: message.body)
#if DEBUG
            logSubtitleBridgeMessage(message.body, parsedCandidates: candidates)
#endif
            handleSubtitleCandidates(candidates)
            return
        }

+
+

NativePlayerViewController.swift

Dreamio/NativePlayerViewController.swift
-3+3
139 unmodified lines
140
141
142
143
144
145
146
8 unmodified lines
155
156
157
158
159
160
161
10 unmodified lines
172
173
174
175
176
177
178
139 unmodified lines
        let pendingCandidates = candidates.filter { !attachedSubtitleURLs.contains($0.url) }
        guard !pendingCandidates.isEmpty else {
#if DEBUG
            print("[DreamioNativePlayer] subtitle candidates=\(candidates.count) pending=0 duplicates=\(candidates.count)")
#endif
            return 0
        }
8 unmodified lines
            await MainActor.run {
                guard !resolvedCandidates.isEmpty else {
#if DEBUG
                    print("[DreamioNativePlayer] subtitle candidates=\(candidates.count) pending=\(pendingCandidates.count) resolved=0 attached=0")
#endif
                    return
                }
10 unmodified lines
                }
#if DEBUG
                let duplicateCount = candidates.count - pendingCandidates.count + resolvedCandidates.count - attachableCandidates.count
                print("[DreamioNativePlayer] subtitle candidates=\(candidates.count) pending=\(pendingCandidates.count) resolved=\(resolvedCandidates.count) attached=\(attachedCount) duplicates=\(duplicateCount)")
#endif
            }
        }
139 unmodified lines
140
141
142
143
144
145
146
8 unmodified lines
155
156
157
158
159
160
161
10 unmodified lines
172
173
174
175
176
177
178
139 unmodified lines
        let pendingCandidates = candidates.filter { !attachedSubtitleURLs.contains($0.url) }
        guard !pendingCandidates.isEmpty else {
#if DEBUG
            print("[DreamioNativePlayer] subtitle candidates=\(candidates.count) pending=0 duplicates=\(candidates.count) resolved=0 attached=0 tracks=\(SubtitleDebugFormatter.trackSummary(backend.subtitleTracks)) selected=\(backend.selectedSubtitleTrackID)")
#endif
            return 0
        }
8 unmodified lines
            await MainActor.run {
                guard !resolvedCandidates.isEmpty else {
#if DEBUG
                    print("[DreamioNativePlayer] subtitle candidates=\(candidates.count) pending=\(pendingCandidates.count) resolved=0 attached=0 tracks=\(SubtitleDebugFormatter.trackSummary(self.backend.subtitleTracks)) selected=\(self.backend.selectedSubtitleTrackID) candidates=\(SubtitleDebugFormatter.candidateSummary(pendingCandidates))")
#endif
                    return
                }
10 unmodified lines
                }
#if DEBUG
                let duplicateCount = candidates.count - pendingCandidates.count + resolvedCandidates.count - attachableCandidates.count
                print("[DreamioNativePlayer] subtitle candidates=\(candidates.count) pending=\(pendingCandidates.count) resolved=\(resolvedCandidates.count) attachable=\(attachableCandidates.count) attached=\(attachedCount) duplicates=\(duplicateCount) tracks=\(SubtitleDebugFormatter.trackSummary(self.backend.subtitleTracks)) selected=\(self.backend.selectedSubtitleTrackID) resolvedCandidates=\(SubtitleDebugFormatter.candidateSummary(resolvedCandidates))")
#endif
            }
        }
+

VLCNativePlaybackBackend.swift

Dreamio/VLCNativePlaybackBackend.swift
-1+16
213 unmodified lines
214
215
216
217
218
219
220
5 unmodified lines
226
227
228
229
230
231
232
233
234
235
12 unmodified lines
248
249
250
251
252
253
213 unmodified lines
            mediaPlayer.addPlaybackSlave(candidate.url, type: .subtitle, enforce: false)
            attachedCount += 1
#if DEBUG
            print("[DreamioVLC] attached subtitle=\(URLRedactor.redactedURLString(candidate.url.absoluteString))")
#endif
        }
#if DEBUG
5 unmodified lines
            return attachedCount
        }
        DispatchQueue.main.asyncAfter(deadline: .now() + 1) { [weak self] in
            self?.onSubtitleTracksChange?()
        }
        return attachedCount
    }
#endif
}

+
12 unmodified lines
        case .paused, .stopped, .ended:
            onStateChange?()
        case .esAdded:
            onSubtitleTracksChange?()
        default:
            break
213 unmodified lines
214
215
216
217
218
219
220
221
5 unmodified lines
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
12 unmodified lines
260
261
262
263
264
265
266
267
268
213 unmodified lines
            mediaPlayer.addPlaybackSlave(candidate.url, type: .subtitle, enforce: false)
            attachedCount += 1
#if DEBUG
            print("[DreamioVLC] addPlaybackSlave subtitle=\(URLRedactor.redactedURLString(candidate.url.absoluteString)) label=\(candidate.label) language=\(candidate.language ?? "unknown") ext=\(candidate.url.pathExtension.lowercased())")
            logSubtitleTracks(reason: "after-addPlaybackSlave")
#endif
        }
#if DEBUG
5 unmodified lines
            return attachedCount
        }
        DispatchQueue.main.asyncAfter(deadline: .now() + 1) { [weak self] in
            #if DEBUG
            self?.logSubtitleTracks(reason: "delayed-refresh")
            #endif
            self?.onSubtitleTracksChange?()
        }
        return attachedCount
    }

+
#if DEBUG
    private func logSubtitleTracks(reason: String) {
        let names = mediaPlayer.videoSubTitlesNames as? [String] ?? []
        let indexes = mediaPlayer.videoSubTitlesIndexes as? [NSNumber] ?? []
        print("[DreamioVLC] subtitle tracks reason=\(reason) names=\(names) indexes=\(indexes.map { $0.int32Value }) selected=\(mediaPlayer.currentVideoSubTitleIndex)")
    }
#endif
#endif
}

+
12 unmodified lines
        case .paused, .stopped, .ended:
            onStateChange?()
        case .esAdded:
#if DEBUG
            logSubtitleTracks(reason: "esAdded")
#endif
            onSubtitleTracksChange?()
        default:
            break
+

StreamCandidate.swift

Dreamio/StreamCandidate.swift
+27
39 unmodified lines
40
41
42
43
44
45
39 unmodified lines
    let name: String
}

+
enum PlaybackTimeFormatter {
    static func label(for seconds: TimeInterval) -> String {
        guard seconds.isFinite, seconds > 0 else {
39 unmodified lines
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
39 unmodified lines
    let name: String
}

+
#if DEBUG
enum SubtitleDebugFormatter {
    static func candidateSummary(_ candidates: [SubtitleCandidate]) -> String {
        guard !candidates.isEmpty else {
            return "[]"
        }

+
        return candidates.map { candidate in
            let extensionLabel = candidate.url.pathExtension.isEmpty ? "none" : candidate.url.pathExtension.lowercased()
            let language = candidate.language?.isEmpty == false ? candidate.language! : "unknown"
            let label = candidate.label.isEmpty ? "External Subtitle" : candidate.label
            return "{label=\(label), language=\(language), ext=\(extensionLabel)}"
        }.joined(separator: ", ")
    }

+
    static func trackSummary(_ tracks: [SubtitleTrack]) -> String {
        guard !tracks.isEmpty else {
            return "[]"
        }

+
        return tracks.map { track in
            "{id=\(track.id), name=\(track.name)}"
        }.joined(separator: ", ")
    }
}
#endif

+
enum PlaybackTimeFormatter {
    static func label(for seconds: TimeInterval) -> String {
        guard seconds.isFinite, seconds > 0 else {
+
+ +
+

Expected Impact for End-Users

+

No user-facing behavior should change. Debug builds should provide clearer Xcode logs for the next playback attempt, making it faster to identify the actual subtitle failure point before changing playback behavior.

+
+ +
+

Validation

+
    +
  • xcodebuild build -workspace Dreamio.xcworkspace -scheme Dreamio -destination 'generic/platform=iOS' CODE_SIGNING_ALLOWED=NO passed.
    • +
  • xcodebuild test -workspace Dreamio.xcworkspace -scheme Dreamio -destination 'platform=iOS Simulator,name=iPhone 16' could not run because the Dreamio scheme is not configured for the test action.
    • +
+
+ +
+

Issues, Limitations, and Mitigations

+
    +
  • This does not fix subtitle playback by design. It proves where the next fix belongs.
    • +
  • The web script can only log subtitle candidates it detects through the existing discovery paths. If the next run shows no bridge discovery logs, Stremio or OpenSubtitles payload discovery is still the likely target.
    • +
  • Duplicate-only subtitle bridge messages now post an empty subtitle list with debug metadata. Native handling ignores empty parsed candidates, so playback behavior remains unchanged.
    • +
+
+ +
+

Follow-up Work

+
    +
  • If logs show no [DreamioSubtitles] bridge discovered entries, inspect the current Stremio/OpenSubtitles payload shape and extend discovery.
    • +
  • If logs show playerActive=false before native playback exists, add a small pending subtitle queue rather than dropping early subtitle messages.
    • +
  • If logs show resolution failures, improve resolver support for the failing OpenSubtitles response format.
    • +
  • If logs show VLC attachment succeeds but track arrays stay empty, test VLC subtitle slave behavior for the resolved file type and timing.
    • +
+
+
+

New Changes as of 2026-05-25 10:19 AM EDT

+

Summary of changes

+

Added DEBUG-only proof logs for the captions menu selection path after the latest Xcode run showed VLC exposing an embedded subtitle track but no new logs during selection.

+

Why this change was made

+

The previous diagnostics proved external subtitles were absent and VLC exposed an embedded English (SDH) track. The missing proof was whether tapping the captions menu fires a native action and whether VLC accepts the selected track id.

+

Code diffs

+

NativePlayerViewController.swift

Dreamio/NativePlayerViewController.swift
-4+23
379 unmodified lines
380
381
382
383
384
385
386
387
388
389
390
391
392
27 unmodified lines
420
421
422
423
424
425
426
427
428
429
430
431
379 unmodified lines

+
    private func captionsMenu() -> UIMenu {
        let selectedTrackID = backend.selectedSubtitleTrackID
        let trackActions = SubtitleOptionMapper.options(from: backend.subtitleTracks).map { track in
            UIAction(
                title: track.name,
                state: track.id == selectedTrackID ? .on : .off
            ) { [weak self] _ in
                self?.backend.selectSubtitleTrack(id: track.id)
                self?.refreshControls()
            }
        }

+
27 unmodified lines
    }

+
    private func refreshControls() {
        playPauseButton.setImage(UIImage(systemName: backend.isPlaying ? "pause.fill" : "play.fill"), for: .normal)
        scrubber.isEnabled = backend.isSeekable
        jumpBackButton.isEnabled = backend.isSeekable
        jumpForwardButton.isEnabled = backend.isSeekable
        captionsButton.isEnabled = !SubtitleOptionMapper.options(from: backend.subtitleTracks).isEmpty
        captionsButton.menu = captionsMenu()
        elapsedLabel.text = PlaybackTimeFormatter.label(for: backend.currentTime)
        remainingLabel.text = "-\(PlaybackTimeFormatter.label(for: backend.remainingTime))"
        if !isScrubbing {
379 unmodified lines
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
27 unmodified lines
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
379 unmodified lines

+
    private func captionsMenu() -> UIMenu {
        let selectedTrackID = backend.selectedSubtitleTrackID
        let tracks = backend.subtitleTracks
        let options = SubtitleOptionMapper.options(from: tracks)
#if DEBUG
        print("[DreamioCaptions] build-menu tracks=\(SubtitleDebugFormatter.trackSummary(tracks)) options=\(SubtitleDebugFormatter.trackSummary(options)) selected=\(selectedTrackID)")
#endif
        let trackActions = options.map { track in
            UIAction(
                title: track.name,
                state: track.id == selectedTrackID ? .on : .off
            ) { [weak self] _ in
                guard let self else {
                    return
                }
#if DEBUG
                print("[DreamioCaptions] select-request id=\(track.id) name=\(track.name) before=\(self.backend.selectedSubtitleTrackID)")
#endif
                self.backend.selectSubtitleTrack(id: track.id)
#if DEBUG
                print("[DreamioCaptions] select-result id=\(track.id) after=\(self.backend.selectedSubtitleTrackID) tracks=\(SubtitleDebugFormatter.trackSummary(self.backend.subtitleTracks))")
#endif
                self.refreshControls()
            }
        }

+
27 unmodified lines
    }

+
    private func refreshControls() {
        let subtitleTracks = backend.subtitleTracks
        let subtitleOptions = SubtitleOptionMapper.options(from: subtitleTracks)
        playPauseButton.setImage(UIImage(systemName: backend.isPlaying ? "pause.fill" : "play.fill"), for: .normal)
        scrubber.isEnabled = backend.isSeekable
        jumpBackButton.isEnabled = backend.isSeekable
        jumpForwardButton.isEnabled = backend.isSeekable
        captionsButton.isEnabled = !subtitleOptions.isEmpty
        captionsButton.menu = captionsMenu()
#if DEBUG
        print("[DreamioCaptions] refresh enabled=\(captionsButton.isEnabled) tracks=\(SubtitleDebugFormatter.trackSummary(subtitleTracks)) selected=\(backend.selectedSubtitleTrackID)")
#endif
        elapsedLabel.text = PlaybackTimeFormatter.label(for: backend.currentTime)
        remainingLabel.text = "-\(PlaybackTimeFormatter.label(for: backend.remainingTime))"
        if !isScrubbing {
+

VLCNativePlaybackBackend.swift

Dreamio/VLCNativePlaybackBackend.swift
+12
99 unmodified lines
100
101
102
103
104
105
106
107
108
109
110
111
112
113
99 unmodified lines

+
    func selectSubtitleTrack(id: Int32) {
#if canImport(MobileVLCKit)
        mediaPlayer.currentVideoSubTitleIndex = id
        onSubtitleTracksChange?()
#endif
    }

+
    func adjustSubtitleDelay(by seconds: TimeInterval) {
#if canImport(MobileVLCKit)
        mediaPlayer.currentVideoSubTitleDelay += Int(seconds * 1_000_000)
        onSubtitleTracksChange?()
#endif
    }
99 unmodified lines
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
99 unmodified lines

+
    func selectSubtitleTrack(id: Int32) {
#if canImport(MobileVLCKit)
#if DEBUG
        logSubtitleTracks(reason: "before-select-\(id)")
#endif
        mediaPlayer.currentVideoSubTitleIndex = id
#if DEBUG
        logSubtitleTracks(reason: "after-select-\(id)")
#endif
        onSubtitleTracksChange?()
#endif
    }

+
    func adjustSubtitleDelay(by seconds: TimeInterval) {
#if canImport(MobileVLCKit)
#if DEBUG
        print("[DreamioVLC] subtitle delay before=\(subtitleDelay) delta=\(seconds)")
#endif
        mediaPlayer.currentVideoSubTitleDelay += Int(seconds * 1_000_000)
#if DEBUG
        print("[DreamioVLC] subtitle delay after=\(subtitleDelay)")
#endif
        onSubtitleTracksChange?()
#endif
    }
+

Related issues or PRs

+

Related Beads issue: dreamio-c1m.

+
+ +
+

New Changes as of 2026-05-25 10:26 AM EDT

+

Summary of changes

+

Stabilized captions menu rebuilding so progress refreshes no longer replace the UIMenu every half second while the user is trying to select a subtitle track.

+

Why this change was made

+

The latest logs showed VLC exposing the embedded subtitle track and the native menu eventually seeing it, but no selection action fired. The repeated build-menu and refresh logs showed the progress timer was constantly recreating the menu.

+

Code diffs

+

NativePlayerViewController.swift

Dreamio/NativePlayerViewController.swift
-6+36
8 unmodified lines
9
10
11
12
13
14
385 unmodified lines
400
401
402
403
404
405
4 unmodified lines
410
411
412
413
414
415
416
417
418
419
15 unmodified lines
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
2 unmodified lines
453
454
455
456
457
458
8 unmodified lines
    private var progressTimer: Timer?
    private var isScrubbing = false
    private var attachedSubtitleURLs: Set<URL>
    var onDismiss: (() -> Void)?

+
    private let loadingView: UIActivityIndicatorView = {
385 unmodified lines
#if DEBUG
                print("[DreamioCaptions] select-result id=\(track.id) after=\(self.backend.selectedSubtitleTrackID) tracks=\(SubtitleDebugFormatter.trackSummary(self.backend.subtitleTracks))")
#endif
                self.refreshControls()
            }
        }
4 unmodified lines
            children: [
                UIAction(title: "Decrease 0.5s") { [weak self] _ in
                    self?.backend.adjustSubtitleDelay(by: -0.5)
                    self?.refreshControls()
                },
                UIAction(title: "Increase 0.5s") { [weak self] _ in
                    self?.backend.adjustSubtitleDelay(by: 0.5)
                    self?.refreshControls()
                },
                UIAction(
15 unmodified lines

+
    private func refreshControls() {
        let subtitleTracks = backend.subtitleTracks
        let subtitleOptions = SubtitleOptionMapper.options(from: subtitleTracks)
        playPauseButton.setImage(UIImage(systemName: backend.isPlaying ? "pause.fill" : "play.fill"), for: .normal)
        scrubber.isEnabled = backend.isSeekable
        jumpBackButton.isEnabled = backend.isSeekable
        jumpForwardButton.isEnabled = backend.isSeekable
        captionsButton.isEnabled = !subtitleOptions.isEmpty
        captionsButton.menu = captionsMenu()
#if DEBUG
        print("[DreamioCaptions] refresh enabled=\(captionsButton.isEnabled) tracks=\(SubtitleDebugFormatter.trackSummary(subtitleTracks)) selected=\(backend.selectedSubtitleTrackID)")
#endif
        elapsedLabel.text = PlaybackTimeFormatter.label(for: backend.currentTime)
        remainingLabel.text = "-\(PlaybackTimeFormatter.label(for: backend.remainingTime))"
        if !isScrubbing {
2 unmodified lines
        [scrubber, jumpBackButton, jumpForwardButton].forEach { $0.alpha = backend.isSeekable ? 1 : 0.45 }
    }

+
    private func revealControls() {
        controlsContainer.isUserInteractionEnabled = true
        closeButton.isUserInteractionEnabled = true
8 unmodified lines
9
10
11
12
13
14
15
385 unmodified lines
401
402
403
404
405
406
407
4 unmodified lines
412
413
414
415
416
417
418
419
420
421
422
423
15 unmodified lines
439
440
441
442
443
444
445
446
447
448
449
2 unmodified lines
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
8 unmodified lines
    private var progressTimer: Timer?
    private var isScrubbing = false
    private var attachedSubtitleURLs: Set<URL>
    private var captionsMenuSignature: String?
    var onDismiss: (() -> Void)?

+
    private let loadingView: UIActivityIndicatorView = {
385 unmodified lines
#if DEBUG
                print("[DreamioCaptions] select-result id=\(track.id) after=\(self.backend.selectedSubtitleTrackID) tracks=\(SubtitleDebugFormatter.trackSummary(self.backend.subtitleTracks))")
#endif
                self.captionsMenuSignature = nil
                self.refreshControls()
            }
        }
4 unmodified lines
            children: [
                UIAction(title: "Decrease 0.5s") { [weak self] _ in
                    self?.backend.adjustSubtitleDelay(by: -0.5)
                    self?.captionsMenuSignature = nil
                    self?.refreshControls()
                },
                UIAction(title: "Increase 0.5s") { [weak self] _ in
                    self?.backend.adjustSubtitleDelay(by: 0.5)
                    self?.captionsMenuSignature = nil
                    self?.refreshControls()
                },
                UIAction(
15 unmodified lines

+
    private func refreshControls() {
        let subtitleTracks = backend.subtitleTracks
        playPauseButton.setImage(UIImage(systemName: backend.isPlaying ? "pause.fill" : "play.fill"), for: .normal)
        scrubber.isEnabled = backend.isSeekable
        jumpBackButton.isEnabled = backend.isSeekable
        jumpForwardButton.isEnabled = backend.isSeekable
        updateCaptionsMenuIfNeeded(subtitleTracks: subtitleTracks)
        elapsedLabel.text = PlaybackTimeFormatter.label(for: backend.currentTime)
        remainingLabel.text = "-\(PlaybackTimeFormatter.label(for: backend.remainingTime))"
        if !isScrubbing {
2 unmodified lines
        [scrubber, jumpBackButton, jumpForwardButton].forEach { $0.alpha = backend.isSeekable ? 1 : 0.45 }
    }

+
    private func updateCaptionsMenuIfNeeded(subtitleTracks: [SubtitleTrack]) {
        let selectedTrackID = backend.selectedSubtitleTrackID
        let signature = captionsMenuSignatureValue(
            tracks: subtitleTracks,
            selectedTrackID: selectedTrackID,
            delay: backend.subtitleDelay
        )
        let hasSelectableTrack = subtitleTracks.contains { $0.id >= 0 }
        captionsButton.isEnabled = hasSelectableTrack
        guard signature != captionsMenuSignature else {
            return
        }

+
        captionsMenuSignature = signature
        captionsButton.menu = captionsMenu()
#if DEBUG
        print("[DreamioCaptions] refresh-menu enabled=\(captionsButton.isEnabled) tracks=\(SubtitleDebugFormatter.trackSummary(subtitleTracks)) selected=\(selectedTrackID)")
#endif
    }

+
    private func captionsMenuSignatureValue(
        tracks: [SubtitleTrack],
        selectedTrackID: Int32,
        delay: TimeInterval
    ) -> String {
        let trackSignature = tracks
            .map { "\($0.id):\($0.name)" }
            .joined(separator: "|")
        return "\(trackSignature)#selected=\(selectedTrackID)#delay=\(String(format: "%.1f", delay))"
    }

+
    private func revealControls() {
        controlsContainer.isUserInteractionEnabled = true
        closeButton.isUserInteractionEnabled = true
+

Related issues or PRs

+

Related Beads issue: dreamio-bd9.

+
+ +
+

New Changes as of 2026-05-25 10:32 AM EDT

+

Summary of changes

+

Extended subtitle discovery diagnostics upstream of the native captions menu so subtitle-shaped fetch and XHR responses now log even when they produce zero parseable external candidates.

+

Why this change was made

+

The latest run confirmed the app only receives VLC embedded captions for this stream. The remaining unknown is whether Stremio is making subtitle addon requests that the injected parser misses, or whether no external subtitle payload is requested at all.

+

Code diffs

+

DreamioWebViewController.swift

Dreamio/DreamioWebViewController.swift
-8+57
80 unmodified lines
81
82
83
84
85
86
39 unmodified lines
126
127
128
129
130
131
132
10 unmodified lines
143
144
145
146
147
148
149
6 unmodified lines
156
157
158
159
160
161
162
163
164
165
166
167
168
169
13 unmodified lines
183
184
185
186
187
188
17 unmodified lines
206
207
208
209
210
211
4 unmodified lines
216
217
218
219
220
221
222
223
224
225
9 unmodified lines
235
236
237
238
239
240
241
443 unmodified lines
685
686
687
688
689
690
691
692
80 unmodified lines
          const subtitleCandidates = [];
          const postedSubtitleURLs = new Set();
          const subtitleURLPattern = /https?:\/\/[^\s"'<>]+(?:\.srt|\.vtt|\.ass|\.ssa|\.sub|opensubtitles|subtitle)[^\s"'<>]*/ig;

+
          const looksNative = (url) => {
            if (!url || typeof url !== "string") {
39 unmodified lines
            } catch (_) {}
          };

+
          const postSubtitleCandidates = (candidates) => {
            const discoveredCount = candidates.length;
            const fresh = candidates.filter((candidate) => {
              if (postedSubtitleURLs.has(candidate.url)) {
10 unmodified lines
                  debug: {
                    discovered: discoveredCount,
                    deduped: 0,
                    forwarded: 0
                  }
                });
              } catch (_) {}
6 unmodified lines
                debug: {
                  discovered: discoveredCount,
                  deduped: fresh.length,
                  forwarded: fresh.length
                }
              });
            } catch (_) {}
          };

+
          const addSubtitleCandidate = (entry) => {
            const rawURL = typeof entry === "string" ? entry : entry && (entry.url || entry.href || entry.src || entry.file || entry.download);
            const url = absoluteURL(rawURL);
            subtitleURLPattern.lastIndex = 0;
            if (!url || !subtitleURLPattern.test(url)) {
13 unmodified lines
            postSubtitleCandidates([candidate]);
          };

+
          const inspectSubtitlePayload = (payload) => {
            if (!payload) {
              return;
17 unmodified lines
            }
          };

+
          const originalFetch = window.fetch;
          if (originalFetch) {
            window.fetch = async (...args) => {
4 unmodified lines
                subtitleURLPattern.lastIndex = 0;
                const shouldInspect = !contentType
                  || /json|text|javascript|xml|subtitle|vtt|srt/i.test(contentType)
                  || subtitleURLPattern.test(url);
                if (shouldInspect) {
                  subtitleURLPattern.lastIndex = 0;
                  response.clone().text().then(inspectSubtitlePayload).catch(() => {});
                }
              } catch (_) {}
              return response;
9 unmodified lines
                  if (responseType && responseType !== "text") {
                    return;
                  }
                  inspectSubtitlePayload(this.responseText);
                } catch (_) {}
              });
            } catch (_) {}
443 unmodified lines
        let discovered = debug?["discovered"] as? Int ?? parsedCandidates.count
        let deduped = debug?["deduped"] as? Int ?? parsedCandidates.count
        let posted = debug?["forwarded"] as? Int ?? parsedCandidates.count
        let pageURL = dictionary?["pageUrl"] as? String
        print("[DreamioSubtitles] bridge discovered=\(discovered) deduped=\(deduped) posted=\(posted) parsed=\(parsedCandidates.count) playerActive=\(currentNativePlayer != nil) page=\(pageURL.map(redactedURLString) ?? "unknown") candidates=\(SubtitleDebugFormatter.candidateSummary(parsedCandidates))")
    }
#endif
}
80 unmodified lines
81
82
83
84
85
86
87
39 unmodified lines
127
128
129
130
131
132
133
10 unmodified lines
144
145
146
147
148
149
150
151
6 unmodified lines
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
13 unmodified lines
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
17 unmodified lines
235
236
237
238
239
240
241
242
243
244
245
246
4 unmodified lines
251
252
253
254
255
256
257
258
259
260
261
262
263
9 unmodified lines
273
274
275
276
277
278
279
280
281
282
283
284
285
443 unmodified lines
729
730
731
732
733
734
735
736
737
738
739
740
741
80 unmodified lines
          const subtitleCandidates = [];
          const postedSubtitleURLs = new Set();
          const subtitleURLPattern = /https?:\/\/[^\s"'<>]+(?:\.srt|\.vtt|\.ass|\.ssa|\.sub|opensubtitles|subtitle)[^\s"'<>]*/ig;
          const subtitleSignalPattern = /subtitle|subtitles|opensubtitles|vtt|srt|ass|ssa/i;

+
          const looksNative = (url) => {
            if (!url || typeof url !== "string") {
39 unmodified lines
            } catch (_) {}
          };

+
          const postSubtitleCandidates = (candidates, debug = {}) => {
            const discoveredCount = candidates.length;
            const fresh = candidates.filter((candidate) => {
              if (postedSubtitleURLs.has(candidate.url)) {
10 unmodified lines
                  debug: {
                    discovered: discoveredCount,
                    deduped: 0,
                    forwarded: 0,
                    ...debug
                  }
                });
              } catch (_) {}
6 unmodified lines
                debug: {
                  discovered: discoveredCount,
                  deduped: fresh.length,
                  forwarded: fresh.length,
                  ...debug
                }
              });
            } catch (_) {}
          };

+
          const addSubtitleCandidate = (entry) => {
            const rawURL = typeof entry === "string"
              ? entry
              : entry && (
                entry.url ||
                entry.href ||
                entry.src ||
                entry.link ||
                entry.file ||
                entry.download ||
                entry.externalUrl ||
                entry.externalURL ||
                entry.fileUrl ||
                entry.fileURL
              );
            const url = absoluteURL(rawURL);
            subtitleURLPattern.lastIndex = 0;
            if (!url || !subtitleURLPattern.test(url)) {
13 unmodified lines
            postSubtitleCandidates([candidate]);
          };

+
          const postSubtitleInspection = (source, url, beforeCount, afterCount, payloadLength) => {
            if (afterCount > beforeCount) {
              return;
            }
            postSubtitleCandidates([], {
              source,
              inspected: true,
              url: url || "",
              payloadLength: payloadLength || 0,
              totalKnown: subtitleCandidates.length
            });
          };

+
          const inspectSubtitlePayload = (payload) => {
            if (!payload) {
              return;
17 unmodified lines
            }
          };

+
          const inspectSubtitleText = (source, url, text) => {
            const beforeCount = subtitleCandidates.length;
            inspectSubtitlePayload(text);
            postSubtitleInspection(source, url, beforeCount, subtitleCandidates.length, text ? text.length : 0);
          };

+
          const originalFetch = window.fetch;
          if (originalFetch) {
            window.fetch = async (...args) => {
4 unmodified lines
                subtitleURLPattern.lastIndex = 0;
                const shouldInspect = !contentType
                  || /json|text|javascript|xml|subtitle|vtt|srt/i.test(contentType)
                  || subtitleURLPattern.test(url)
                  || subtitleSignalPattern.test(url);
                if (shouldInspect) {
                  subtitleURLPattern.lastIndex = 0;
                  response.clone().text().then((text) => {
                    inspectSubtitleText("fetch", url, text);
                  }).catch(() => {});
                }
              } catch (_) {}
              return response;
9 unmodified lines
                  if (responseType && responseType !== "text") {
                    return;
                  }
                  const url = this.responseURL || "";
                  const text = this.responseText || "";
                  if (subtitleSignalPattern.test(url) || subtitleSignalPattern.test(text)) {
                    inspectSubtitleText("xhr", url, text);
                  } else {
                    inspectSubtitlePayload(text);
                  }
                } catch (_) {}
              });
            } catch (_) {}
443 unmodified lines
        let discovered = debug?["discovered"] as? Int ?? parsedCandidates.count
        let deduped = debug?["deduped"] as? Int ?? parsedCandidates.count
        let posted = debug?["forwarded"] as? Int ?? parsedCandidates.count
        let source = debug?["source"] as? String ?? "bridge"
        let inspected = debug?["inspected"] as? Bool ?? false
        let inspectedURL = (debug?["url"] as? String).map(redactedURLString) ?? "none"
        let payloadLength = debug?["payloadLength"] as? Int ?? 0
        let totalKnown = debug?["totalKnown"] as? Int ?? parsedCandidates.count
        let pageURL = dictionary?["pageUrl"] as? String
        print("[DreamioSubtitles] bridge source=\(source) inspected=\(inspected) discovered=\(discovered) deduped=\(deduped) posted=\(posted) parsed=\(parsedCandidates.count) totalKnown=\(totalKnown) payloadLength=\(payloadLength) playerActive=\(currentNativePlayer != nil) inspectedURL=\(inspectedURL) page=\(pageURL.map(redactedURLString) ?? "unknown") candidates=\(SubtitleDebugFormatter.candidateSummary(parsedCandidates))")
    }
#endif
}
+

Related issues or PRs

+

Related Beads issue: dreamio-ese.

+
+ +
+ + diff --git a/docs/turns/2026-05-25-resolve-opensubtitles-subtitle-downloads.html b/docs/turns/2026-05-25-resolve-opensubtitles-subtitle-downloads.html new file mode 100644 index 0000000..2431244 --- /dev/null +++ b/docs/turns/2026-05-25-resolve-opensubtitles-subtitle-downloads.html @@ -0,0 +1,637 @@ + + + + + + Resolve OpenSubtitles Subtitle Downloads + + + +
+
+

Turn document · May 25, 2026

+

Resolve OpenSubtitles subtitle downloads

+

Dreamio now resolves OpenSubtitles V3 API-style subtitle download responses into direct subtitle files before handing them to VLC, reducing the chance that Stremio reports “failed to load subtitles” after native playback opens.

+
+ Issue: dreamio-h5q + Scope: native subtitles + Validation: tests and simulator build +
+
+ +
+

Summary

+

Fixed the OpenSubtitles V3 subtitle handoff path by adding a resolver between Stremio subtitle discovery and VLC subtitle attachment. Direct subtitle files still attach immediately, while OpenSubtitles API/download URLs are fetched and inspected for playable .srt, .vtt, .ass, .ssa, or .sub links.

+
+
BeforeVLC received API-like OpenSubtitles URLs directly.
+
AfterDreamio resolves those endpoints to file URLs first.
+
ResultSubtitle tracks are more likely to attach as usable VLC subtitles.
+
+
+ +
+

Changes Made

+
    +
  • Added a SubtitleResolving protocol and a concrete SubtitleResolver.
    • +
  • Moved initial subtitle attachment out of the VLC backend and through NativePlayerViewController.addSubtitleCandidates, so initial and late subtitle discoveries use one path.
    • +
  • Added support for OpenSubtitles response fields like link and file_name.
    • +
  • Added a regression test for resolving OpenSubtitles V3 download JSON into a direct subtitle file URL.
    • +
+
+ +
+

Context

+

The previous OpenSubtitles pass forwarded late subtitle discoveries to the active native player, but it still assumed every discovered URL was directly consumable by VLC. OpenSubtitles V3 commonly returns API/download endpoints that respond with JSON containing the real subtitle file link. Passing those API URLs straight to VLC can surface as Stremio’s “failed to load subtitles” message.

+
+ +
+

Important Implementation Details

+
    +
  • SubtitleResolver.resolve returns direct subtitle file candidates unchanged.
    • +
  • OpenSubtitles-like URLs are fetched with an Accept header that allows JSON, text, and common subtitle formats.
    • +
  • JSON and text responses are parsed through the existing subtitle candidate parser, then filtered down to direct subtitle file URLs.
    • +
  • The captions menu label preserves the original Stremio/OpenSubtitles label when a download response includes a raw filename.
    • +
  • Duplicate tracking now records the originally discovered URL while resolution is pending, then records the resolved playable URL before VLC attachment.
    • +
+
+ +
+

Relevant Diff Snippets

+

Dreamio/NativePlaybackBackend.swift

Dreamio/NativePlaybackBackend.swift
+4
29 unmodified lines
30
31
32
33
34
35
29 unmodified lines
    func stop()
}

+
enum NativePlaybackError: LocalizedError {
    case backendUnavailable
    case startupTimedOut
29 unmodified lines
30
31
32
33
34
35
36
37
38
39
29 unmodified lines
    func stop()
}

+
protocol SubtitleResolving {
    func resolve(_ candidate: SubtitleCandidate) async -> SubtitleCandidate?
}

+
enum NativePlaybackError: LocalizedError {
    case backendUnavailable
    case startupTimedOut
+

Dreamio/NativePlayerViewController.swift

Dreamio/NativePlayerViewController.swift
-15+57
2 unmodified lines
3
4
5
6
7
8
83 unmodified lines
92
93
94
95
96
97
98
99
100
101
24 unmodified lines
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
5 unmodified lines
157
158
159
160
161
162
2 unmodified lines
final class NativePlayerViewController: UIViewController {
    private let request: NativePlaybackRequest
    private var backend: NativePlaybackBackend
    private var startupTimer: Timer?
    private var controlsTimer: Timer?
    private var progressTimer: Timer?
83 unmodified lines
        return label
    }()

+
    init(request: NativePlaybackRequest, backend: NativePlaybackBackend = VLCNativePlaybackBackend()) {
        self.request = request
        self.backend = backend
        self.attachedSubtitleURLs = Set(request.subtitleCandidates.map(\.url))
        super.init(nibName: nil, bundle: nil)
        modalPresentationStyle = .fullScreen
        modalTransitionStyle = .crossDissolve
24 unmodified lines
        configureLayout()
        startStartupTimer()
        backend.play(request: request)
    }

+
    @discardableResult
    func addSubtitleCandidates(_ candidates: [SubtitleCandidate]) -> Int {
        let newCandidates = candidates.filter { candidate in
            guard !attachedSubtitleURLs.contains(candidate.url) else {
                return false
            }
            attachedSubtitleURLs.insert(candidate.url)
            return true
        }
        let attachedCount = backend.addSubtitleCandidates(newCandidates)
        if attachedCount > 0 {
            refreshControls()
        }
#if DEBUG
        let duplicateCount = candidates.count - newCandidates.count
        print("[DreamioNativePlayer] subtitle candidates=\(candidates.count) forwarded=\(newCandidates.count) attached=\(attachedCount) duplicates=\(duplicateCount)")
#endif
        return attachedCount
    }

+
    override func viewDidDisappear(_ animated: Bool) {
5 unmodified lines
        onDismiss?()
    }

+
    private func configureBackend() {
        backend.prepare(in: self)
        backend.view.translatesAutoresizingMaskIntoConstraints = false
2 unmodified lines
3
4
5
6
7
8
9
83 unmodified lines
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
24 unmodified lines
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
5 unmodified lines
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
2 unmodified lines
final class NativePlayerViewController: UIViewController {
    private let request: NativePlaybackRequest
    private var backend: NativePlaybackBackend
    private let subtitleResolver: SubtitleResolving
    private var startupTimer: Timer?
    private var controlsTimer: Timer?
    private var progressTimer: Timer?
83 unmodified lines
        return label
    }()

+
    init(
        request: NativePlaybackRequest,
        backend: NativePlaybackBackend = VLCNativePlaybackBackend(),
        subtitleResolver: SubtitleResolving = SubtitleResolver()
    ) {
        self.request = request
        self.backend = backend
        self.subtitleResolver = subtitleResolver
        self.attachedSubtitleURLs = []
        super.init(nibName: nil, bundle: nil)
        modalPresentationStyle = .fullScreen
        modalTransitionStyle = .crossDissolve
24 unmodified lines
        configureLayout()
        startStartupTimer()
        backend.play(request: request)
        addSubtitleCandidates(request.subtitleCandidates)
    }

+
    @discardableResult
    func addSubtitleCandidates(_ candidates: [SubtitleCandidate]) -> Int {
        let pendingCandidates = candidates.filter { !attachedSubtitleURLs.contains($0.url) }
        guard !pendingCandidates.isEmpty else {
#if DEBUG
            print("[DreamioNativePlayer] subtitle candidates=\(candidates.count) pending=0 duplicates=\(candidates.count)")
#endif
            return 0
        }

+
        pendingCandidates.forEach { attachedSubtitleURLs.insert($0.url) }

+
        Task { [weak self] in
            guard let self else {
                return
            }
            let resolvedCandidates = await self.resolveSubtitleCandidates(pendingCandidates)
            await MainActor.run {
                guard !resolvedCandidates.isEmpty else {
#if DEBUG
                    print("[DreamioNativePlayer] subtitle candidates=\(candidates.count) pending=\(pendingCandidates.count) resolved=0 attached=0")
#endif
                    return
                }
                let attachableCandidates = resolvedCandidates.filter { candidate in
                    guard !self.attachedSubtitleURLs.contains(candidate.url) || pendingCandidates.contains(where: { $0.url == candidate.url }) else {
                        return false
                    }
                    self.attachedSubtitleURLs.insert(candidate.url)
                    return true
                }
                let attachedCount = self.backend.addSubtitleCandidates(attachableCandidates)
                if attachedCount > 0 {
                    self.refreshControls()
                }
#if DEBUG
                let duplicateCount = candidates.count - pendingCandidates.count + resolvedCandidates.count - attachableCandidates.count
                print("[DreamioNativePlayer] subtitle candidates=\(candidates.count) pending=\(pendingCandidates.count) resolved=\(resolvedCandidates.count) attached=\(attachedCount) duplicates=\(duplicateCount)")
#endif
            }
        }

+
        return pendingCandidates.count
    }

+
    override func viewDidDisappear(_ animated: Bool) {
5 unmodified lines
        onDismiss?()
    }

+
    private func resolveSubtitleCandidates(_ candidates: [SubtitleCandidate]) async -> [SubtitleCandidate] {
        var resolved: [SubtitleCandidate] = []
        for candidate in candidates {
            if let playableCandidate = await subtitleResolver.resolve(candidate) {
                resolved.append(playableCandidate)
            }
        }
        return resolved
    }

+
    private func configureBackend() {
        backend.prepare(in: self)
        backend.view.translatesAutoresizingMaskIntoConstraints = false
+

Dreamio/StreamCandidate.swift

Dreamio/StreamCandidate.swift
-2+2
104 unmodified lines
105
106
107
108
109
110
111
112
104 unmodified lines

+
enum SubtitleCandidateParser {
    private static let supportedExtensions = ["srt", "vtt", "ass", "ssa", "sub"]
    private static let urlFields = ["url", "href", "src", "subtitles", "subtitle", "subtitleUrl", "subtitleURL", "file", "download"]
    private static let labelFields = ["label", "name", "title", "lang", "language", "id"]

+
    static func candidates(in payload: Any?) -> [SubtitleCandidate] {
        var results: [SubtitleCandidate] = []
104 unmodified lines
105
106
107
108
109
110
111
112
104 unmodified lines

+
enum SubtitleCandidateParser {
    private static let supportedExtensions = ["srt", "vtt", "ass", "ssa", "sub"]
    private static let urlFields = ["url", "href", "src", "link", "subtitles", "subtitle", "subtitleUrl", "subtitleURL", "file", "download"]
    private static let labelFields = ["label", "name", "title", "file_name", "lang", "language", "id"]

+
    static func candidates(in payload: Any?) -> [SubtitleCandidate] {
        var results: [SubtitleCandidate] = []
+

Dreamio/StreamResolver.swift

Dreamio/StreamResolver.swift
+100
28 unmodified lines
29
30
31
32
33
34
28 unmodified lines
    }
}

+
protocol StreamResolving {
    func resolve(request: NativePlaybackRequest) async throws -> ResolvedNativeStream
}
28 unmodified lines
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
28 unmodified lines
    }
}

+
final class SubtitleResolver: SubtitleResolving {
    private let session: URLSession

+
    init(session: URLSession = .shared) {
        self.session = session
    }

+
    func resolve(_ candidate: SubtitleCandidate) async -> SubtitleCandidate? {
        if Self.isDirectSubtitleFile(candidate.url) {
            return candidate
        }

+
        guard Self.shouldResolve(candidate.url) else {
            return nil
        }

+
        var request = URLRequest(url: candidate.url)
        request.setValue("application/json, text/plain, text/vtt, application/x-subrip, */*", forHTTPHeaderField: "Accept")

+
        do {
            let (data, response) = try await session.data(for: request)
            if let httpResponse = response as? HTTPURLResponse,
               !(200...299).contains(httpResponse.statusCode) {
#if DEBUG
                print("[DreamioSubtitles] resolve status=\(httpResponse.statusCode) url=\(URLRedactor.redactedURLString(candidate.url.absoluteString))")
#endif
                return nil
            }

+
            if let finalURL = response.url, Self.isDirectSubtitleFile(finalURL) {
                return SubtitleCandidate(url: finalURL, label: candidate.label, language: candidate.language)
            }

+
            return Self.bestPlayableCandidate(
                from: data,
                responseURL: response.url,
                original: candidate
            )
        } catch {
#if DEBUG
            print("[DreamioSubtitles] resolve failure=\(URLRedactor.redactedURLString(error.localizedDescription)) url=\(URLRedactor.redactedURLString(candidate.url.absoluteString))")
#endif
            return nil
        }
    }

+
    static func bestPlayableCandidate(
        from data: Data,
        responseURL: URL?,
        original: SubtitleCandidate
    ) -> SubtitleCandidate? {
        if let responseURL, isDirectSubtitleFile(responseURL) {
            return SubtitleCandidate(url: responseURL, label: original.label, language: original.language)
        }

+
        guard !data.isEmpty else {
            return nil
        }

+
        if let payload = try? JSONSerialization.jsonObject(with: data) {
            return SubtitleCandidateParser.candidates(in: payload)
                .first(where: { isDirectSubtitleFile($0.url) })
                .map { playable in
                    SubtitleCandidate(
                        url: playable.url,
                        label: original.label.isEmpty ? playable.label : original.label,
                        language: playable.language ?? original.language
                    )
                }
        }

+
        if let text = String(data: data, encoding: .utf8) {
            return SubtitleCandidateParser.candidates(in: text)
                .first(where: { isDirectSubtitleFile($0.url) })
                .map { playable in
                    SubtitleCandidate(
                        url: playable.url,
                        label: original.label.isEmpty ? playable.label : original.label,
                        language: playable.language ?? original.language
                    )
                }
        }

+
        return nil
    }

+
    static func isDirectSubtitleFile(_ url: URL) -> Bool {
        let lowercased = url.absoluteString.lowercased()
        return ["srt", "vtt", "ass", "ssa", "sub"].contains(url.pathExtension.lowercased())
            || [".srt?", ".vtt?", ".ass?", ".ssa?", ".sub?", ".srt&", ".vtt&", ".ass&", ".ssa&", ".sub&"].contains(where: lowercased.contains)
    }

+
    private static func shouldResolve(_ url: URL) -> Bool {
        let lowercased = url.absoluteString.lowercased()
        return lowercased.contains("opensubtitles")
            || lowercased.contains("/subtitle")
            || lowercased.contains("subtitle")
    }
}

+
protocol StreamResolving {
    func resolve(request: NativePlaybackRequest) async throws -> ResolvedNativeStream
}
+

Dreamio/VLCNativePlaybackBackend.swift

Dreamio/VLCNativePlaybackBackend.swift
-1
57 unmodified lines
58
59
60
61
62
63
64
57 unmodified lines
        print("[DreamioVLC] opening url=\(URLRedactor.redactedURLString(request.playbackURL.absoluteString))")
#endif
        mediaPlayer.play()
        addSubtitleCandidates(request.subtitleCandidates)
#else
        onFailure?(NativePlaybackError.backendUnavailable)
#endif
57 unmodified lines
58
59
60
61
62
63
57 unmodified lines
        print("[DreamioVLC] opening url=\(URLRedactor.redactedURLString(request.playbackURL.absoluteString))")
#endif
        mediaPlayer.play()
#else
        onFailure?(NativePlaybackError.backendUnavailable)
#endif
+

Tests/StreamResolverTests.swift

Tests/StreamResolverTests.swift
+26
9 unmodified lines
10
11
12
13
14
15
131 unmodified lines
147
148
149
150
151
152
9 unmodified lines
        testPlaybackTimeFormatting()
        testSubtitleCandidateParsing()
        testOpenSubtitlesV3CandidateParsing()
        testSubtitleCandidateDeduplicationPreservesLabels()
        testSubtitleOptionMappingIncludesNone()
        print("StreamResolverTests passed")
131 unmodified lines
        assertEqual(candidates[3].url.absoluteString, "https://cdn.example.test/from-string.ass?source=opensubtitles")
    }

+
    private static func testSubtitleCandidateDeduplicationPreservesLabels() {
        let payload: [String: Any] = [
            "subtitles": [
9 unmodified lines
10
11
12
13
14
15
16
131 unmodified lines
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
9 unmodified lines
        testPlaybackTimeFormatting()
        testSubtitleCandidateParsing()
        testOpenSubtitlesV3CandidateParsing()
        testOpenSubtitlesV3DownloadResponseResolution()
        testSubtitleCandidateDeduplicationPreservesLabels()
        testSubtitleOptionMappingIncludesNone()
        print("StreamResolverTests passed")
131 unmodified lines
        assertEqual(candidates[3].url.absoluteString, "https://cdn.example.test/from-string.ass?source=opensubtitles")
    }

+
    private static func testOpenSubtitlesV3DownloadResponseResolution() {
        let payload = """
        {
          "link": "https://dl.opensubtitles.org/en/download/subtitle.srt?token=secret",
          "file_name": "episode.srt",
          "requests": 1
        }
        """.data(using: .utf8)!
        let original = SubtitleCandidate(
            url: URL(string: "https://api.opensubtitles.com/api/v1/download")!,
            label: "English",
            language: "eng"
        )

+
        let candidate = SubtitleResolver.bestPlayableCandidate(
            from: payload,
            responseURL: original.url,
            original: original
        )

+
        assertEqual(candidate?.url.absoluteString, "https://dl.opensubtitles.org/en/download/subtitle.srt?token=secret")
        assertEqual(candidate?.label, "English")
        assertEqual(candidate?.language, "eng")
    }

+
    private static func testSubtitleCandidateDeduplicationPreservesLabels() {
        let payload: [String: Any] = [
            "subtitles": [
+
+ +
+

Expected Impact for End-Users

+

When a Stremio stream opens in Dreamio’s native player, OpenSubtitles V3 captions should appear more reliably because Dreamio now gives VLC the final subtitle file URL rather than an intermediate API endpoint. The visible label should stay friendly, such as “English,” instead of degrading to a filename.

+
+ +
+

Validation

+
    +
  • Ran swiftc Dreamio/StreamCandidate.swift [temporary SubtitleResolving shim] Dreamio/StreamResolver.swift Tests/StreamResolverTests.swift -o /tmp/StreamResolverTests && /tmp/StreamResolverTests: passed.
    • +
  • Ran xcodebuild -workspace Dreamio.xcworkspace -scheme Dreamio -destination 'generic/platform=iOS Simulator' build CODE_SIGNING_ALLOWED=NO: passed.
    • +
  • The first standalone test attempt failed because NativePlaybackBackend.swift imports UIKit outside the iOS SDK compile context; the follow-up test used a temporary protocol shim for the pure resolver/parser code.
    • +
+
+ +
+

Issues, Limitations, and Mitigations

+
    +
  • Live OpenSubtitles credentials and a real Stremio playback session were not available in this environment, so live subtitle rendering still needs a device check.
    • +
  • If a provider returns raw subtitle text directly from an API URL without a final file URL, this change does not yet write a temporary subtitle file. The resolver is structured so that fallback can be added cleanly.
    • +
  • Network failures while resolving subtitles are logged in debug builds and skipped, so playback should continue even if a subtitle endpoint fails.
    • +
+
+ +
+

Follow-up Work

+
    +
  • Manually test a real OpenSubtitles V3 title on device and confirm selected captions render in VLC.
    • +
  • Add a temporary-file fallback if live testing shows OpenSubtitles sometimes returns subtitle text directly instead of a file link.
    • +
  • Consider surfacing subtitle resolution failures in a debug-only diagnostics panel.
    • +
+
+
+ + \ No newline at end of file diff --git a/skills-lock.json b/skills-lock.json new file mode 100644 index 0000000..e88ea48 --- /dev/null +++ b/skills-lock.json @@ -0,0 +1,11 @@ +{ + "version": 1, + "skills": { + "liquid-glass-design": { + "source": "affaan-m/everything-claude-code", + "sourceType": "github", + "skillPath": "skills/liquid-glass-design/SKILL.md", + "computedHash": "cdd47ce273b84d1e630aa6ea8086baa9b4affd9f881b3f95a7b25334b67a1dc5" + } + } +}