From 419ffae415cb4f6b43cf45bcbb297eb55a17bb89 Mon Sep 17 00:00:00 2001 From: dirtydishes Date: Mon, 25 May 2026 01:05:13 -0400 Subject: [PATCH] add native player controls and captions --- .beads/interactions.jsonl | 1 + .beads/issues.jsonl | 1 + Dreamio/DreamioWebViewController.swift | 135 +++++- Dreamio/NativePlaybackBackend.swift | 18 + Dreamio/NativePlayerViewController.swift | 230 ++++++++++- Dreamio/StreamCandidate.swift | 140 ++++++- Dreamio/VLCNativePlaybackBackend.swift | 145 +++++++ Tests/StreamResolverTests.swift | 45 ++ ...e-player-controls-captions-close-flow.html | 385 ++++++++++++++++++ 9 files changed, 1096 insertions(+), 4 deletions(-) create mode 100644 docs/turns/2026-05-25-native-player-controls-captions-close-flow.html diff --git a/.beads/interactions.jsonl b/.beads/interactions.jsonl index 81948ce..e8fa5cb 100644 --- a/.beads/interactions.jsonl +++ b/.beads/interactions.jsonl @@ -8,3 +8,4 @@ {"id":"int-76aa54ba","kind":"field_change","created_at":"2026-05-25T03:51:39.198446Z","actor":"dirtydishes","issue_id":"dreamio-8vi","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Closed"}} {"id":"int-74805ffd","kind":"field_change","created_at":"2026-05-25T04:21:42.440755Z","actor":"dirtydishes","issue_id":"dreamio-2k5","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Added native backend availability guard, installed CocoaPods, generated workspace metadata, documented setup, and validated available checks."}} {"id":"int-27a61615","kind":"field_change","created_at":"2026-05-25T04:44:35.633997Z","actor":"dirtydishes","issue_id":"dreamio-ija","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Fixed MobileVLCKit linker failures by preparing the XCFramework slice before app linking and preserving the integration through pod install."}} +{"id":"int-fad68cb4","kind":"field_change","created_at":"2026-05-25T05:04:55.103302Z","actor":"dirtydishes","issue_id":"dreamio-mj8","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Implemented native VLC player controls, caption controls, subtitle candidate discovery, and close-flow cleanup."}} diff --git a/.beads/issues.jsonl b/.beads/issues.jsonl index 3b465b4..dfac9eb 100644 --- a/.beads/issues.jsonl +++ b/.beads/issues.jsonl @@ -6,5 +6,6 @@ {"_type":"issue","id":"dreamio-l68","title":"Add native playback for direct debrid streams","description":"Implement a WKWebView JavaScript bridge that detects direct-file debrid media URLs and routes unsupported containers to a native player backend, initially MobileVLCKit, while preserving normal Stremio Web playback for compatible streams.","status":"closed","priority":1,"issue_type":"feature","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T03:13:19Z","created_by":"dirtydishes","updated_at":"2026-05-25T03:20:17Z","started_at":"2026-05-25T03:13:28Z","closed_at":"2026-05-25T03:20:17Z","close_reason":"Implemented native direct-stream bridge, classification, MobileVLCKit backend wiring, CocoaPods workflow docs, and turn documentation. Full iOS build is blocked locally by missing CocoaPods and iPhoneOS SDK.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"dreamio-tnv","title":"Fix iOS bundle identifier install failure","description":"Xcode built Dreamio.app without a valid CFBundleIdentifier, causing device install to fail with CoreDeviceError 3000/3002. Investigate project bundle settings, fix the source configuration, validate the app bundle Info.plist, and document the change.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T01:23:00Z","created_by":"dirtydishes","updated_at":"2026-05-25T01:25:36Z","started_at":"2026-05-25T01:23:07Z","closed_at":"2026-05-25T01:25:36Z","close_reason":"Added bundle metadata to Info.plist and validated processed app bundle identifier.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"dreamio-4yn","title":"Build WKWebView MVP shell","description":"Create the first Dreamio MVP implementation: a minimal iOS WKWebView wrapper around hosted Stremio Web, with configuration, launch behavior, diagnostics, and documentation for real-device viability testing.","acceptance_criteria":"App project exists; WKWebView loads hosted Stremio Web; external/new-window navigation is handled; basic diagnostics and manual test documentation exist; quality gates are run or documented.","status":"closed","priority":1,"issue_type":"feature","owner":"dishes@dpdrm.com","created_at":"2026-05-24T14:55:12Z","created_by":"dirtydishes","updated_at":"2026-05-24T14:59:44Z","closed_at":"2026-05-24T14:59:44Z","close_reason":"Implemented the MVP WKWebView iOS shell, added run and validation documentation, and recorded current validation limits.","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"dreamio-mj8","title":"Add native player controls and captions","description":"Implement a fuller VLC-backed native playback surface with transport controls, caption controls, external subtitle discovery, and a clean close flow back to Stremio episode selection.","status":"closed","priority":2,"issue_type":"feature","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T04:57:53Z","created_by":"dirtydishes","updated_at":"2026-05-25T05:04:55Z","started_at":"2026-05-25T04:57:57Z","closed_at":"2026-05-25T05:04:55Z","close_reason":"Implemented native VLC player controls, caption controls, subtitle candidate discovery, and close-flow cleanup.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"dreamio-evt","title":"Enable WebView inspection and playback diagnostics","description":"Add development-only WKWebView inspection and token-safe playback diagnostics so Dreamio can debug hosted Stremio media failures without changing app navigation, login, or playback behavior.","status":"closed","priority":2,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T02:30:26Z","created_by":"dirtydishes","updated_at":"2026-05-25T02:34:55Z","started_at":"2026-05-25T02:30:32Z","closed_at":"2026-05-25T02:34:55Z","close_reason":"Implemented debug-only WKWebView inspection, token-safe playback diagnostics, navigation logging, validation build, and turn documentation.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"dreamio-a5b","title":"Track HTML diff rendering tooling as dev dependency","description":"Move the HTML diff rendering package into devDependencies and ignore installed Node modules so the repo tracks reproducible tooling without vendoring dependencies.","status":"closed","priority":2,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-25T01:12:07Z","created_by":"dirtydishes","updated_at":"2026-05-25T01:12:44Z","started_at":"2026-05-25T01:12:14Z","closed_at":"2026-05-25T01:12:44Z","close_reason":"Moved @pierre/diffs to devDependencies and ignored node_modules.","dependency_count":0,"dependent_count":0,"comment_count":0} diff --git a/Dreamio/DreamioWebViewController.swift b/Dreamio/DreamioWebViewController.swift index 97da38e..06ecfe8 100644 --- a/Dreamio/DreamioWebViewController.swift +++ b/Dreamio/DreamioWebViewController.swift @@ -72,6 +72,8 @@ final class DreamioWebViewController: UIViewController { /\.m3u8(?:[?#]|$)/i, /\.mp4(?:[?#]|$)/i ]; + const subtitleCandidates = []; + const subtitleURLPattern = /https?:\/\/[^\s"'<>]+(?:\.srt|\.vtt|\.ass|\.ssa|\.sub|opensubtitles|subtitle)[^\s"'<>]*/ig; const looksNative = (url) => { if (!url || typeof url !== "string") { @@ -111,11 +113,72 @@ final class DreamioWebViewController: UIViewController { resolverUrl: findResolverURL(), pageUrl: window.location.href, tagName: element && element.tagName ? element.tagName : "", - currentSrc: element && element.currentSrc ? element.currentSrc : "" + currentSrc: element && element.currentSrc ? element.currentSrc : "", + subtitles: subtitleCandidates.slice(-20) }); } catch (_) {} }; + const addSubtitleCandidate = (entry) => { + const rawURL = typeof entry === "string" ? entry : entry && (entry.url || entry.href || entry.src || entry.file || entry.download); + const url = absoluteURL(rawURL); + if (!url || !subtitleURLPattern.test(url)) { + subtitleURLPattern.lastIndex = 0; + return; + } + subtitleURLPattern.lastIndex = 0; + if (subtitleCandidates.some((candidate) => candidate.url === url)) { + return; + } + subtitleCandidates.push({ + url, + label: entry && (entry.label || entry.name || entry.title || entry.lang || entry.language) || "External Subtitle", + language: entry && (entry.lang || entry.language) || "" + }); + }; + + const inspectSubtitlePayload = (payload) => { + if (!payload) { + return; + } + if (typeof payload === "string") { + const matches = payload.match(subtitleURLPattern) || []; + subtitleURLPattern.lastIndex = 0; + matches.forEach(addSubtitleCandidate); + try { + inspectSubtitlePayload(JSON.parse(payload)); + } catch (_) {} + return; + } + if (Array.isArray(payload)) { + payload.forEach(inspectSubtitlePayload); + return; + } + if (typeof payload === "object") { + addSubtitleCandidate(payload); + Object.values(payload).forEach(inspectSubtitlePayload); + } + }; + + const originalFetch = window.fetch; + if (originalFetch) { + window.fetch = async (...args) => { + const response = await originalFetch(...args); + try { + response.clone().text().then(inspectSubtitlePayload).catch(() => {}); + } catch (_) {} + return response; + }; + } + + const originalXHRSend = XMLHttpRequest.prototype.send; + XMLHttpRequest.prototype.send = function(...args) { + try { + this.addEventListener("load", () => inspectSubtitlePayload(this.responseText)); + } catch (_) {} + return originalXHRSend.apply(this, args); + }; + const stopNativeHandledMedia = (element) => { const media = element instanceof HTMLVideoElement ? element @@ -387,11 +450,13 @@ final class DreamioWebViewController: UIViewController { userAgent: request.userAgent, referer: request.referer, headers: resolved.headers, - classification: request.classification + classification: request.classification, + subtitleCandidates: request.subtitleCandidates ) let player = NativePlayerViewController(request: resolvedRequest) player.onDismiss = { [weak self] in self?.lastNativePlaybackURL = nil + self?.cleanUpStremioPlayerAfterNativeDismiss() } present(player, animated: true) } catch { @@ -423,6 +488,72 @@ final class DreamioWebViewController: UIViewController { present(alert, animated: true) } + private func cleanUpStremioPlayerAfterNativeDismiss() { + let script = #""" + (() => { + const stopMedia = () => { + document.querySelectorAll("video, audio").forEach((media) => { + try { media.pause(); } catch (_) {} + try { media.removeAttribute("src"); } catch (_) {} + try { media.querySelectorAll("source").forEach((source) => source.removeAttribute("src")); } catch (_) {} + try { media.load(); } catch (_) {} + }); + }; + const clickVisible = (selectors) => { + for (const selector of selectors) { + const nodes = Array.from(document.querySelectorAll(selector)); + const match = nodes.find((node) => { + const style = window.getComputedStyle(node); + const rect = node.getBoundingClientRect(); + return style.display !== "none" && style.visibility !== "hidden" && rect.width > 0 && rect.height > 0; + }); + if (match) { + try { match.click(); return true; } catch (_) {} + } + } + return false; + }; + stopMedia(); + const clicked = clickVisible([ + "[aria-label*='Close' i]", + "[aria-label*='Back' i]", + "button[class*='close' i]", + "button[class*='back' i]", + ".player button", + "[role='button']" + ]); + const stillPlayer = /player|stream|buffer|prepar/i.test(document.body.innerText || ""); + return { clicked, stillPlayer, href: window.location.href }; + })(); + """# + + webView.evaluateJavaScript(script) { [weak self] result, error in + guard let self else { + return + } +#if DEBUG + if let error { + print("[DreamioCloseFlow] cleanup error=\(URLRedactor.redactedURLString(error.localizedDescription))") + } else { + print("[DreamioCloseFlow] cleanup result=\(String(describing: result))") + } +#endif + guard error == nil else { + self.loadDreamio() + return + } + if self.webView.canGoBack { + DispatchQueue.main.asyncAfter(deadline: .now() + 0.4) { + self.webView.evaluateJavaScript("(/player|stream|buffer|prepar/i).test(document.body.innerText || '')") { result, _ in + if (result as? Bool) == true { + self.webView.goBack() + } + } + } + } + } + } + #if DEBUG private func logDiagnostic(type: String, payload: Any, pageURL: String?) { let redactedPageURL = pageURL.map(redactedURLString) ?? "unknown" diff --git a/Dreamio/NativePlaybackBackend.swift b/Dreamio/NativePlaybackBackend.swift index 6d44993..57ec708 100644 --- a/Dreamio/NativePlaybackBackend.swift +++ b/Dreamio/NativePlaybackBackend.swift @@ -4,9 +4,27 @@ protocol NativePlaybackBackend: AnyObject { var view: UIView { get } var onReady: (() -> Void)? { get set } var onFailure: ((Error) -> Void)? { get set } + var onStateChange: (() -> Void)? { get set } + var onSubtitleTracksChange: (() -> Void)? { get set } + var isPlaying: Bool { get } + var isSeekable: Bool { get } + var duration: TimeInterval { get } + var currentTime: TimeInterval { get } + var remainingTime: TimeInterval { get } + var position: Float { get } + var subtitleTracks: [SubtitleTrack] { get } + var selectedSubtitleTrackID: Int32 { get } + var subtitleDelay: TimeInterval { get } func prepare(in viewController: UIViewController) func play(request: NativePlaybackRequest) + func play() + func pause() + func togglePlayPause() + func seek(to position: Float) + func jump(by seconds: TimeInterval) + func selectSubtitleTrack(id: Int32) + func adjustSubtitleDelay(by seconds: TimeInterval) func stop() } diff --git a/Dreamio/NativePlayerViewController.swift b/Dreamio/NativePlayerViewController.swift index 4f14c4c..54de22d 100644 --- a/Dreamio/NativePlayerViewController.swift +++ b/Dreamio/NativePlayerViewController.swift @@ -4,6 +4,9 @@ final class NativePlayerViewController: UIViewController { private let request: NativePlaybackRequest private var backend: NativePlaybackBackend private var startupTimer: Timer? + private var controlsTimer: Timer? + private var progressTimer: Timer? + private var isScrubbing = false var onDismiss: (() -> Void)? private let loadingView: UIActivityIndicatorView = { @@ -25,6 +28,49 @@ final class NativePlayerViewController: UIViewController { return button }() + private let controlsContainer: UIVisualEffectView = { + let view = UIVisualEffectView(effect: UIBlurEffect(style: .systemUltraThinMaterialDark)) + view.translatesAutoresizingMaskIntoConstraints = false + view.layer.cornerRadius = 12 + view.clipsToBounds = true + return view + }() + + private let playPauseButton = NativePlayerViewController.iconButton(systemName: "pause.fill", label: "Play or Pause") + private let jumpBackButton = NativePlayerViewController.iconButton(systemName: "gobackward.15", label: "Jump Back 15 Seconds") + private let jumpForwardButton = NativePlayerViewController.iconButton(systemName: "goforward.15", label: "Jump Forward 15 Seconds") + private let captionsButton = NativePlayerViewController.iconButton(systemName: "captions.bubble", label: "Captions") + + private let elapsedLabel: UILabel = { + let label = UILabel() + label.translatesAutoresizingMaskIntoConstraints = false + label.textColor = .white + label.font = .monospacedDigitSystemFont(ofSize: 13, weight: .medium) + label.text = "0:00" + return label + }() + + private let remainingLabel: UILabel = { + let label = UILabel() + label.translatesAutoresizingMaskIntoConstraints = false + label.textColor = .white + label.font = .monospacedDigitSystemFont(ofSize: 13, weight: .medium) + label.textAlignment = .right + label.text = "-0:00" + return label + }() + + private let scrubber: UISlider = { + let slider = UISlider() + slider.translatesAutoresizingMaskIntoConstraints = false + slider.minimumValue = 0 + slider.maximumValue = 1 + slider.minimumTrackTintColor = UIColor(red: 0.64, green: 0.48, blue: 1.0, alpha: 1) + slider.maximumTrackTintColor = UIColor.white.withAlphaComponent(0.3) + slider.thumbTintColor = .white + return slider + }() + private let failureLabel: UILabel = { let label = UILabel() label.translatesAutoresizingMaskIntoConstraints = false @@ -74,6 +120,8 @@ final class NativePlayerViewController: UIViewController { override func viewDidDisappear(_ animated: Bool) { super.viewDidDisappear(animated) startupTimer?.invalidate() + controlsTimer?.invalidate() + progressTimer?.invalidate() backend.stop() onDismiss?() } @@ -86,6 +134,9 @@ final class NativePlayerViewController: UIViewController { self?.startupTimer?.invalidate() self?.loadingView.stopAnimating() self?.loadingView.isHidden = true + self?.startProgressUpdates() + self?.refreshControls() + self?.scheduleControlsHide() } } backend.onFailure = { [weak self] error in @@ -94,6 +145,16 @@ final class NativePlayerViewController: UIViewController { self?.showFailure(error) } } + backend.onStateChange = { [weak self] in + DispatchQueue.main.async { + self?.refreshControls() + } + } + backend.onSubtitleTracksChange = { [weak self] in + DispatchQueue.main.async { + self?.refreshControls() + } + } } private func startStartupTimer() { @@ -108,8 +169,38 @@ final class NativePlayerViewController: UIViewController { view.addSubview(backend.view) view.addSubview(loadingView) view.addSubview(failureLabel) + view.addSubview(controlsContainer) view.addSubview(closeButton) closeButton.addTarget(self, action: #selector(close), for: .touchUpInside) + playPauseButton.addTarget(self, action: #selector(togglePlayPause), for: .touchUpInside) + jumpBackButton.addTarget(self, action: #selector(jumpBack), for: .touchUpInside) + jumpForwardButton.addTarget(self, action: #selector(jumpForward), for: .touchUpInside) + captionsButton.addTarget(self, action: #selector(showCaptions), for: .touchUpInside) + scrubber.addTarget(self, action: #selector(scrubbingStarted), for: .touchDown) + scrubber.addTarget(self, action: #selector(scrubberChanged), for: .valueChanged) + scrubber.addTarget(self, action: #selector(scrubbingEnded), for: [.touchUpInside, .touchUpOutside, .touchCancel]) + + let tap = UITapGestureRecognizer(target: self, action: #selector(toggleControlsVisibility)) + tap.cancelsTouchesInView = false + view.addGestureRecognizer(tap) + + let controlRow = UIStackView(arrangedSubviews: [jumpBackButton, playPauseButton, jumpForwardButton, captionsButton]) + controlRow.translatesAutoresizingMaskIntoConstraints = false + controlRow.axis = .horizontal + controlRow.alignment = .center + controlRow.distribution = .equalCentering + controlRow.spacing = 18 + + let timeRow = UIStackView(arrangedSubviews: [elapsedLabel, remainingLabel]) + timeRow.translatesAutoresizingMaskIntoConstraints = false + timeRow.axis = .horizontal + timeRow.distribution = .fillEqually + + let stack = UIStackView(arrangedSubviews: [scrubber, timeRow, controlRow]) + stack.translatesAutoresizingMaskIntoConstraints = false + stack.axis = .vertical + stack.spacing = 8 + controlsContainer.contentView.addSubview(stack) NSLayoutConstraint.activate([ backend.view.leadingAnchor.constraint(equalTo: view.leadingAnchor), @@ -127,7 +218,25 @@ final class NativePlayerViewController: UIViewController { closeButton.widthAnchor.constraint(equalToConstant: 44), closeButton.heightAnchor.constraint(equalToConstant: 44), closeButton.topAnchor.constraint(equalTo: view.safeAreaLayoutGuide.topAnchor, constant: 12), - closeButton.trailingAnchor.constraint(equalTo: view.safeAreaLayoutGuide.trailingAnchor, constant: -12) + closeButton.trailingAnchor.constraint(equalTo: view.safeAreaLayoutGuide.trailingAnchor, constant: -12), + + controlsContainer.leadingAnchor.constraint(equalTo: view.safeAreaLayoutGuide.leadingAnchor, constant: 18), + controlsContainer.trailingAnchor.constraint(equalTo: view.safeAreaLayoutGuide.trailingAnchor, constant: -18), + controlsContainer.bottomAnchor.constraint(equalTo: view.safeAreaLayoutGuide.bottomAnchor, constant: -18), + + stack.leadingAnchor.constraint(equalTo: controlsContainer.contentView.leadingAnchor, constant: 16), + stack.trailingAnchor.constraint(equalTo: controlsContainer.contentView.trailingAnchor, constant: -16), + stack.topAnchor.constraint(equalTo: controlsContainer.contentView.topAnchor, constant: 14), + stack.bottomAnchor.constraint(equalTo: controlsContainer.contentView.bottomAnchor, constant: -14), + + jumpBackButton.widthAnchor.constraint(equalToConstant: 44), + jumpBackButton.heightAnchor.constraint(equalToConstant: 44), + playPauseButton.widthAnchor.constraint(equalToConstant: 54), + playPauseButton.heightAnchor.constraint(equalToConstant: 54), + jumpForwardButton.widthAnchor.constraint(equalToConstant: 44), + jumpForwardButton.heightAnchor.constraint(equalToConstant: 44), + captionsButton.widthAnchor.constraint(equalToConstant: 44), + captionsButton.heightAnchor.constraint(equalToConstant: 44) ]) } @@ -144,4 +253,123 @@ final class NativePlayerViewController: UIViewController { @objc private func close() { dismiss(animated: true) } + + @objc private func togglePlayPause() { + backend.togglePlayPause() + revealControls() + } + + @objc private func jumpBack() { + backend.jump(by: -15) + revealControls() + } + + @objc private func jumpForward() { + backend.jump(by: 15) + revealControls() + } + + @objc private func scrubbingStarted() { + isScrubbing = true + controlsTimer?.invalidate() + } + + @objc private func scrubberChanged() { + elapsedLabel.text = PlaybackTimeFormatter.label(for: TimeInterval(scrubber.value) * backend.duration) + } + + @objc private func scrubbingEnded() { + backend.seek(to: scrubber.value) + isScrubbing = false + revealControls() + } + + @objc private func toggleControlsVisibility() { + if controlsContainer.alpha < 1 { + revealControls() + } else if backend.isPlaying { + hideControls() + } + } + + @objc private func showCaptions() { + revealControls() + let alert = UIAlertController(title: "Captions", message: nil, preferredStyle: .actionSheet) + SubtitleOptionMapper.options(from: backend.subtitleTracks).forEach { track in + let prefix = track.id == backend.selectedSubtitleTrackID ? "Selected: " : "" + alert.addAction(UIAlertAction(title: "\(prefix)\(track.name)", style: .default) { [weak self] _ in + self?.backend.selectSubtitleTrack(id: track.id) + }) + } + alert.addAction(UIAlertAction(title: "Delay -0.5s", style: .default) { [weak self] _ in + self?.backend.adjustSubtitleDelay(by: -0.5) + }) + alert.addAction(UIAlertAction(title: "Delay +0.5s", style: .default) { [weak self] _ in + self?.backend.adjustSubtitleDelay(by: 0.5) + }) + alert.addAction(UIAlertAction(title: "Current Delay: \(String(format: "%.1fs", backend.subtitleDelay))", style: .default)) + alert.addAction(UIAlertAction(title: "Cancel", style: .cancel)) + if let popover = alert.popoverPresentationController { + popover.sourceView = captionsButton + popover.sourceRect = captionsButton.bounds + } + present(alert, animated: true) + } + + private func startProgressUpdates() { + progressTimer?.invalidate() + progressTimer = Timer.scheduledTimer(withTimeInterval: 0.5, repeats: true) { [weak self] _ in + self?.refreshControls() + } + } + + private func refreshControls() { + playPauseButton.setImage(UIImage(systemName: backend.isPlaying ? "pause.fill" : "play.fill"), for: .normal) + scrubber.isEnabled = backend.isSeekable + jumpBackButton.isEnabled = backend.isSeekable + jumpForwardButton.isEnabled = backend.isSeekable + captionsButton.isEnabled = !SubtitleOptionMapper.options(from: backend.subtitleTracks).isEmpty + elapsedLabel.text = PlaybackTimeFormatter.label(for: backend.currentTime) + remainingLabel.text = "-\(PlaybackTimeFormatter.label(for: backend.remainingTime))" + if !isScrubbing { + scrubber.value = backend.position + } + [scrubber, jumpBackButton, jumpForwardButton].forEach { $0.alpha = backend.isSeekable ? 1 : 0.45 } + } + + private func revealControls() { + UIView.animate(withDuration: 0.18) { + self.controlsContainer.alpha = 1 + self.closeButton.alpha = 1 + } + scheduleControlsHide() + } + + private func hideControls() { + UIView.animate(withDuration: 0.24) { + self.controlsContainer.alpha = 0 + self.closeButton.alpha = 0 + } + } + + private func scheduleControlsHide() { + controlsTimer?.invalidate() + guard backend.isPlaying else { + return + } + controlsTimer = Timer.scheduledTimer(withTimeInterval: 3, repeats: false) { [weak self] _ in + self?.hideControls() + } + } + + private static func iconButton(systemName: String, label: String) -> UIButton { + let button = UIButton(type: .system) + button.translatesAutoresizingMaskIntoConstraints = false + button.setImage(UIImage(systemName: systemName), for: .normal) + button.tintColor = .white + button.backgroundColor = UIColor.black.withAlphaComponent(0.35) + button.layer.cornerRadius = 22 + button.accessibilityLabel = label + return button + } } diff --git a/Dreamio/StreamCandidate.swift b/Dreamio/StreamCandidate.swift index 279cf5f..11ab6b3 100644 --- a/Dreamio/StreamCandidate.swift +++ b/Dreamio/StreamCandidate.swift @@ -26,6 +26,44 @@ struct NativePlaybackRequest { let referer: String let headers: [String: String] let classification: StreamClassification + let subtitleCandidates: [SubtitleCandidate] +} + +struct SubtitleCandidate: Equatable { + let url: URL + let label: String + let language: String? +} + +struct SubtitleTrack: Equatable { + let id: Int32 + let name: String +} + +enum PlaybackTimeFormatter { + static func label(for seconds: TimeInterval) -> String { + guard seconds.isFinite, seconds > 0 else { + return "0:00" + } + + let roundedSeconds = Int(seconds.rounded()) + let hours = roundedSeconds / 3600 + let minutes = (roundedSeconds % 3600) / 60 + let seconds = roundedSeconds % 60 + + if hours > 0 { + return String(format: "%d:%02d:%02d", hours, minutes, seconds) + } + return String(format: "%d:%02d", minutes, seconds) + } +} + +enum SubtitleOptionMapper { + static let offTrack = SubtitleTrack(id: -1, name: "Off") + + static func options(from tracks: [SubtitleTrack]) -> [SubtitleTrack] { + [offTrack] + tracks.filter { $0.id >= 0 } + } } struct StreamClassification { @@ -41,6 +79,7 @@ struct StreamCandidate { let observedURL: URL let resolverURL: URL? let pageURL: URL? + let subtitleCandidates: [SubtitleCandidate] init?(messageBody: Any) { guard let body = messageBody as? [String: Any], @@ -52,6 +91,7 @@ struct StreamCandidate { observedURL = observed resolverURL = Self.url(from: body["resolverUrl"]) pageURL = Self.url(from: body["pageUrl"]) + subtitleCandidates = SubtitleCandidateParser.candidates(in: body["subtitles"]) } private static func url(from value: Any?) -> URL? { @@ -63,6 +103,103 @@ struct StreamCandidate { } } +enum SubtitleCandidateParser { + private static let supportedExtensions = ["srt", "vtt", "ass", "ssa", "sub"] + private static let urlFields = ["url", "href", "src", "subtitles", "subtitle", "subtitleUrl", "subtitleURL", "file", "download"] + private static let labelFields = ["label", "name", "title", "lang", "language", "id"] + + static func candidates(in payload: Any?) -> [SubtitleCandidate] { + var results: [SubtitleCandidate] = [] + collect(from: payload, into: &results) + + var seen = Set() + return results.filter { candidate in + let key = candidate.url.absoluteString + guard !seen.contains(key) else { + return false + } + seen.insert(key) + return true + } + } + + private static func collect(from value: Any?, into results: inout [SubtitleCandidate]) { + switch value { + case let dictionary as [String: Any]: + if let candidate = candidate(from: dictionary) { + results.append(candidate) + } + dictionary.values.forEach { collect(from: $0, into: &results) } + case let array as [Any]: + array.forEach { collect(from: $0, into: &results) } + case let string as String: + if let url = subtitleURL(from: string) { + results.append(SubtitleCandidate(url: url, label: defaultLabel(for: url), language: nil)) + } else { + extractSubtitleURLs(from: string).forEach { url in + results.append(SubtitleCandidate(url: url, label: defaultLabel(for: url), language: nil)) + } + } + default: + break + } + } + + private static func candidate(from dictionary: [String: Any]) -> SubtitleCandidate? { + guard let url = urlFields.lazy.compactMap({ subtitleURL(from: dictionary[$0] as? String) }).first else { + return nil + } + + let label = labelFields.lazy.compactMap { dictionary[$0] as? String }.first + let language = (dictionary["lang"] as? String) ?? (dictionary["language"] as? String) + return SubtitleCandidate( + url: url, + label: label?.isEmpty == false ? label! : defaultLabel(for: url), + language: language + ) + } + + private static func subtitleURL(from string: String?) -> URL? { + guard let string, + let url = URL(string: string), + ["http", "https"].contains(url.scheme?.lowercased()) + else { + return nil + } + + let lowercased = url.absoluteString.lowercased() + guard supportedExtensions.contains(url.pathExtension.lowercased()) + || supportedExtensions.contains(where: { lowercased.contains(".\($0)?") || lowercased.contains(".\($0)&") }) + || lowercased.contains("subtitle") + || lowercased.contains("opensubtitles") + else { + return nil + } + + return url + } + + private static func defaultLabel(for url: URL) -> String { + let lastPathComponent = url.deletingPathExtension().lastPathComponent + return lastPathComponent.isEmpty ? "External Subtitle" : lastPathComponent + } + + private static func extractSubtitleURLs(from string: String) -> [URL] { + let pattern = #"https?://[^\s"'<>]+(?:\.srt|\.vtt|\.ass|\.ssa|\.sub|opensubtitles|subtitle)[^\s"'<>]*"# + let range = NSRange(string.startIndex.. Void)? var onFailure: ((Error) -> Void)? + var onStateChange: (() -> Void)? + var onSubtitleTracksChange: (() -> Void)? #if canImport(MobileVLCKit) private let mediaPlayer = VLCMediaPlayer() #endif + private var attachedSubtitleURLs = Set() override init() { super.init() @@ -54,11 +57,61 @@ final class VLCNativePlaybackBackend: NSObject, NativePlaybackBackend { print("[DreamioVLC] opening url=\(URLRedactor.redactedURLString(request.playbackURL.absoluteString))") #endif mediaPlayer.play() + attachSubtitles(request.subtitleCandidates) #else onFailure?(NativePlaybackError.backendUnavailable) #endif } + func play() { +#if canImport(MobileVLCKit) + mediaPlayer.play() +#endif + } + + func pause() { +#if canImport(MobileVLCKit) + mediaPlayer.pause() +#endif + } + + func togglePlayPause() { + isPlaying ? pause() : play() + } + + func seek(to position: Float) { +#if canImport(MobileVLCKit) + guard isSeekable else { + return + } + mediaPlayer.position = max(0, min(1, position)) +#endif + } + + func jump(by seconds: TimeInterval) { +#if canImport(MobileVLCKit) + guard isSeekable else { + return + } + let nextTime = max(0, min(duration, currentTime + seconds)) + mediaPlayer.time = VLCTime(int: Int32(nextTime * 1000)) +#endif + } + + func selectSubtitleTrack(id: Int32) { +#if canImport(MobileVLCKit) + mediaPlayer.currentVideoSubTitleIndex = id + onSubtitleTracksChange?() +#endif + } + + func adjustSubtitleDelay(by seconds: TimeInterval) { +#if canImport(MobileVLCKit) + mediaPlayer.currentVideoSubTitleDelay += Int(seconds * 1_000_000) + onSubtitleTracksChange?() +#endif + } + func stop() { #if canImport(MobileVLCKit) mediaPlayer.stop() @@ -66,6 +119,93 @@ final class VLCNativePlaybackBackend: NSObject, NativePlaybackBackend { mediaPlayer.media = nil #endif } + + var isPlaying: Bool { +#if canImport(MobileVLCKit) + mediaPlayer.isPlaying +#else + false +#endif + } + + var isSeekable: Bool { +#if canImport(MobileVLCKit) + mediaPlayer.isSeekable +#else + false +#endif + } + + var duration: TimeInterval { +#if canImport(MobileVLCKit) + TimeInterval(max(0, mediaPlayer.media?.length.intValue ?? 0)) / 1000 +#else + 0 +#endif + } + + var currentTime: TimeInterval { +#if canImport(MobileVLCKit) + TimeInterval(max(0, mediaPlayer.time.intValue)) / 1000 +#else + 0 +#endif + } + + var remainingTime: TimeInterval { + max(0, duration - currentTime) + } + + var position: Float { +#if canImport(MobileVLCKit) + mediaPlayer.position +#else + 0 +#endif + } + + var subtitleTracks: [SubtitleTrack] { +#if canImport(MobileVLCKit) + let names = mediaPlayer.videoSubTitlesNames as? [String] ?? [] + let indexes = mediaPlayer.videoSubTitlesIndexes as? [NSNumber] ?? [] + return zip(indexes, names).map { index, name in + SubtitleTrack(id: index.int32Value, name: name) + } +#else + [] +#endif + } + + var selectedSubtitleTrackID: Int32 { +#if canImport(MobileVLCKit) + mediaPlayer.currentVideoSubTitleIndex +#else + -1 +#endif + } + + var subtitleDelay: TimeInterval { +#if canImport(MobileVLCKit) + TimeInterval(mediaPlayer.currentVideoSubTitleDelay) / 1_000_000 +#else + 0 +#endif + } + +#if canImport(MobileVLCKit) + private func attachSubtitles(_ candidates: [SubtitleCandidate]) { + candidates.forEach { candidate in + guard !attachedSubtitleURLs.contains(candidate.url) else { + return + } + attachedSubtitleURLs.insert(candidate.url) + mediaPlayer.addPlaybackSlave(candidate.url, type: .subtitle, enforce: false) +#if DEBUG + print("[DreamioVLC] attached subtitle=\(URLRedactor.redactedURLString(candidate.url.absoluteString))") +#endif + } + } +#endif } #if canImport(MobileVLCKit) @@ -77,8 +217,13 @@ extension VLCNativePlaybackBackend: VLCMediaPlayerDelegate { switch mediaPlayer.state { case .buffering, .playing: onReady?() + onStateChange?() case .error: onFailure?(NativePlaybackError.playbackFailed) + case .paused, .stopped, .ended: + onStateChange?() + case .esAdded: + onSubtitleTracksChange?() default: break } diff --git a/Tests/StreamResolverTests.swift b/Tests/StreamResolverTests.swift index 101bbf4..6cc5573 100644 --- a/Tests/StreamResolverTests.swift +++ b/Tests/StreamResolverTests.swift @@ -7,6 +7,9 @@ struct StreamResolverTests { testResolverSelectsUnsupportedDirectURLAndHeaders() testResolverRejectsHLSOnlyResponse() testRedactorHandlesPercentEncodedPath() + testPlaybackTimeFormatting() + testSubtitleCandidateParsing() + testSubtitleOptionMappingIncludesOff() print("StreamResolverTests passed") } @@ -75,6 +78,48 @@ struct StreamResolverTests { assertEqual(redacted, "https://cdn.example.test/video/%5Bredacted%5D/%E2%9C%93.mp4") } + private static func testPlaybackTimeFormatting() { + assertEqual(PlaybackTimeFormatter.label(for: 0), "0:00") + assertEqual(PlaybackTimeFormatter.label(for: 65), "1:05") + assertEqual(PlaybackTimeFormatter.label(for: 3_725), "1:02:05") + } + + private static func testSubtitleCandidateParsing() { + let payload: [String: Any] = [ + "subtitles": [ + [ + "lang": "eng", + "url": "https://opensubtitles.example.test/download/subtitle.srt?token=secret" + ], + [ + "language": "Spanish", + "file": "https://cdn.example.test/movie.es.vtt" + ], + "https://cdn.example.test/ignored.txt" + ], + "nested": [ + "body": "metadata https://cdn.example.test/movie.fr.ass?download=1" + ] + ] + + let candidates = SubtitleCandidateParser.candidates(in: payload) + + assertEqual(candidates.count, 3) + assertEqual(candidates[0].language, "eng") + assertEqual(candidates[1].label, "Spanish") + assertEqual(candidates[2].url.absoluteString, "https://cdn.example.test/movie.fr.ass?download=1") + } + + private static func testSubtitleOptionMappingIncludesOff() { + let options = SubtitleOptionMapper.options(from: [ + SubtitleTrack(id: 2, name: "English"), + SubtitleTrack(id: 5, name: "Spanish") + ]) + + assertEqual(options.map(\.name), ["Off", "English", "Spanish"]) + assertEqual(options.first?.id, -1) + } + private static func assertEqual(_ actual: T?, _ expected: T, file: StaticString = #file, line: UInt = #line) { assert(actual == expected, "Expected \(String(describing: expected)), got \(String(describing: actual))", file: file, line: line) } diff --git a/docs/turns/2026-05-25-native-player-controls-captions-close-flow.html b/docs/turns/2026-05-25-native-player-controls-captions-close-flow.html new file mode 100644 index 0000000..14256ed --- /dev/null +++ b/docs/turns/2026-05-25-native-player-controls-captions-close-flow.html @@ -0,0 +1,385 @@ + + + + + + Native Player Controls, Captions, and Close Flow + + + +
+
+

Turn document created May 25, 2026 at 01:02 EDT

+

Native Player Controls, Captions, and Close Flow

+

Dreamio now presents a fuller VLC-backed native playback surface: transport controls, scrubbing, caption selection and delay controls, best-effort external subtitle discovery, and cleanup that returns Stremio Web toward episode or stream selection after native playback closes.

+
+ +
+

Summary

+

Implemented native player controls on top of MobileVLCKit and expanded the web bridge so subtitle metadata discovered in Stremio Web can be carried into VLC. Closing the native player now stops the underlying web media and attempts to escape Stremio Web's stuck preparing or buffering player without forcing a full reload unless cleanup fails.

+
+ +
+

Changes Made

+
    +
  • Extended NativePlaybackBackend with player state, transport controls, seeking, subtitle track selection, and subtitle delay APIs.
    • +
  • Added a native overlay in NativePlayerViewController with close, play/pause, 15-second jumps, scrubber, elapsed and remaining labels, captions, tap-to-reveal, and auto-hide while playing.
    • +
  • Implemented MobileVLCKit-backed state reads and controls in VLCNativePlaybackBackend, including subtitle track mapping and remote subtitle attachment.
    • +
  • Added subtitle candidate parsing for Stremio/OpenSubtitles-like payloads and pure helper tests for time labels and caption option mapping.
    • +
  • Observed JavaScript fetch and XMLHttpRequest responses in Stremio Web to collect subtitle-like URLs before native playback opens.
    • +
  • Added a native dismiss cleanup script that pauses/removes in-page media, clicks visible close/back controls, and falls back to web history when the player state appears stuck.
    • +
+
+ +
+

Context

+

The previous native player surface was intentionally minimal: it opened VLC, showed a spinner, and exposed only a close button. That made unsupported containers playable, but it left users without ordinary playback affordances and could leave Stremio Web behind the modal in a preparing or buffering state.

+

This pass keeps the architecture pragmatic: Stremio Web remains the source of stream selection and metadata, while VLC handles native playback for containers WebKit cannot reliably play.

+
+ +
+

Important Implementation Details

+
    +
  • Subtitle discovery is best effort. The injected bridge watches web responses for URLs that look like subtitle assets or OpenSubtitles links, then includes up to the latest 20 candidates in the native stream candidate message.
    • +
  • VLC receives remote subtitles through addPlaybackSlave(_:type:.subtitle,enforce:). Embedded tracks are exposed from videoSubTitlesNames and videoSubTitlesIndexes.
    • +
  • The captions sheet always includes an explicit Off option, plus simple delay controls in half-second increments.
    • +
  • Seek and jump controls disable visually and functionally when VLC reports a non-seekable stream.
    • +
  • The close flow avoids a full reload during normal cleanup so the user can return to the selection context whenever Stremio's UI cooperates.
    • +
+
+ +
+

Relevant Diff Snippets

+

Rendered with @pierre/diffs/ssr using preloadPatchDiff, following the repository turn-document requirement to use diffs.com rendering for diff snippets.

+
+
Dreamio/NativePlaybackBackend.swift
+12
3 unmodified lines
4
5
6
7
8
9
10
11
3 unmodified lines
    var view: UIView { get }
    var onReady: (() -> Void)? { get set }
    var onFailure: ((Error) -> Void)? { get set }

+        
    func prepare(in viewController: UIViewController)
    func play(request: NativePlaybackRequest)
    func stop()
}
3 unmodified lines
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
3 unmodified lines
    var view: UIView { get }
    var onReady: (() -> Void)? { get set }
    var onFailure: ((Error) -> Void)? { get set }
    var onStateChange: (() -> Void)? { get set }
    var onSubtitleTracksChange: (() -> Void)? { get set }
    var isPlaying: Bool { get }
    var isSeekable: Bool { get }
    var duration: TimeInterval { get }
    var currentTime: TimeInterval { get }
    var subtitleTracks: [SubtitleTrack] { get }
    var selectedSubtitleTrackID: Int32 { get }

+        
    func prepare(in viewController: UIViewController)
    func play(request: NativePlaybackRequest)
    func togglePlayPause()
    func seek(to position: Float)
    func selectSubtitleTrack(id: Int32)
    func adjustSubtitleDelay(by seconds: TimeInterval)
    func stop()
}
+
Dreamio/DreamioWebViewController.swift
-1+13
112 unmodified lines
111
112
113
114
115
116
112 unmodified lines
                pageUrl: window.location.href,
                tagName: element && element.tagName ? element.tagName : "",
                currentSrc: element && element.currentSrc ? element.currentSrc : ""
              });
            } catch (_) {}
          };
112 unmodified lines
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
112 unmodified lines
                pageUrl: window.location.href,
                tagName: element && element.tagName ? element.tagName : "",
                currentSrc: element && element.currentSrc ? element.currentSrc : "",
                subtitles: subtitleCandidates.slice(-20)
              });
            } catch (_) {}
          };

+        
          const originalFetch = window.fetch;
          if (originalFetch) {
            window.fetch = async (...args) => {
              const response = await originalFetch(...args);
              try {
                response.clone().text().then(inspectSubtitlePayload).catch(() => {});
              } catch (_) {}
              return response;
            };
          }
+
Dreamio/NativePlayerViewController.swift
+8
269 unmodified lines
269 unmodified lines
269 unmodified lines
270
271
272
273
274
275
276
277
269 unmodified lines
        SubtitleOptionMapper.options(from: backend.subtitleTracks).forEach { track in
            let prefix = track.id == backend.selectedSubtitleTrackID ? "Selected: " : ""
            alert.addAction(UIAlertAction(title: "\(prefix)\(track.name)", style: .default) { [weak self] _ in
                self?.backend.selectSubtitleTrack(id: track.id)
            })
        }
        alert.addAction(UIAlertAction(title: "Delay -0.5s", style: .default))
        alert.addAction(UIAlertAction(title: "Delay +0.5s", style: .default))
+
+
+ +
+

Expected Impact for End-Users

+

Users should be able to control native VLC playback without leaving the app: pause, resume, jump, scrub when possible, switch captions, turn captions off, and make small caption timing corrections. After closing native playback, Stremio Web should more reliably return to episode or stream selection rather than remaining on a stale preparing or buffering player.

+
+ +
+

Validation

+
    +
  • Passed pure Swift tests with swiftc Dreamio/StreamCandidate.swift Dreamio/StreamResolver.swift Tests/StreamResolverTests.swift -o /tmp/dreamio-stream-tests && /tmp/dreamio-stream-tests.
    • +
  • Passed simulator build with xcodebuild -workspace Dreamio.xcworkspace -scheme Dreamio -sdk iphonesimulator -configuration Debug build.
    • +
  • The Xcode build still reports the existing warning that the MobileVLCKit prepare script has no declared outputs.
    • +
  • Ran bd dolt push; Beads reported that no Dolt remote is configured, so issue data remains stored locally and in the committed Beads export.
    • +
  • Manual real-device playback and subtitle validation was not performed in this terminal session.
    • +
+
+ +
+

Issues, Limitations, and Mitigations

+
    +
  • External subtitle support depends on the hosted Stremio Web app exposing subtitle URLs in fetch/XHR responses before native playback starts. If not, VLC will still show embedded tracks.
    • +
  • The close flow uses visible button heuristics because Stremio Web does not provide a native close API to Dreamio. It falls back to web history and only reloads if JavaScript cleanup errors.
    • +
  • The captions sheet is intentionally basic for this pass. It exposes track selection and simple delay adjustments but not full subtitle styling.
    • +
+
+ +
+

Follow-up Work

+
    +
  • Validate embedded and external subtitles on a real device with representative Stremio and OpenSubtitles addons.
    • +
  • Consider a richer caption settings panel if users need style controls or exact delay entry.
    • +
  • Add a UI test harness or injectable mock backend for exercising native player overlay behavior without MobileVLCKit.
    • +
+
+
+ +