Native player audio track selection
Dreamio's VLC-backed native player now exposes embedded audio tracks, adds a far-left audio menu to the control bar, and refreshes the player chrome with a more iOS-native glass treatment.
Summary
Added audio track discovery and selection to native playback so multi-language MKV and similar files can be switched without leaving the player.
Changes Made
- Extended
NativePlaybackBackendwith audio track state, a selection API, and an audio-track-change callback. - Read MobileVLCKit audio track names, indexes, and current selection from
VLCMediaPlayer. - Added an audio menu button on the far left side of the native controls using the
waveform.circlesymbol. - Grouped skip/play/skip into a centered playback cluster so the play button stays visually centered.
- Updated the control surface and buttons with translucent material, softer radius, subtle borders, and lighter glass-like control wells.
Context
The player already exposed embedded subtitle tracks through MobileVLCKit. The same streams often include multiple audio tracks, such as alternate languages or commentary, but the native player had no way to inspect or switch them.
The user-provided diagnostics showed VLC discovering embedded subtitle tracks while the UI still lacked track filtering for audio. This change follows the existing subtitle menu pattern instead of creating a separate player path.
Important Implementation Details
AudioTrackcurrently aliases the existingSubtitleTrackvalue shape because both VLC APIs expose an integer id and display name.- The audio button is enabled only when VLC reports more than one selectable audio track, keeping single-track files quiet.
- The backend fires
onAudioTracksChangewhen VLC enters playback/buffering and when elementary streams are added. - Selection is applied with
mediaPlayer.currentAudioTrackIndex, matching the existing subtitle selection style.
Relevant Diff Snippets
Rendered with @pierre/diffs/ssr from the current git diff.
Dreamio/NativePlaybackBackend.swift
5 unmodified lines678910111213141516175 unmodified lines2324252627285 unmodified linesvar onFailure: ((Error) -> Void)? { get set }var onStateChange: (() -> Void)? { get set }var onSubtitleTracksChange: (() -> Void)? { get set }var isPlaying: Bool { get }var isSeekable: Bool { get }var duration: TimeInterval { get }var currentTime: TimeInterval { get }var remainingTime: TimeInterval { get }var position: Float { get }var subtitleTracks: [SubtitleTrack] { get }var selectedSubtitleTrackID: Int32 { get }var subtitleDelay: TimeInterval { get }5 unmodified linesfunc togglePlayPause()func seek(to position: Float)func jump(by seconds: TimeInterval)func selectSubtitleTrack(id: Int32)func adjustSubtitleDelay(by seconds: TimeInterval)@discardableResult5 unmodified lines678910111213141516171819205 unmodified lines262728293031325 unmodified linesvar onFailure: ((Error) -> Void)? { get set }var onStateChange: (() -> Void)? { get set }var onSubtitleTracksChange: (() -> Void)? { get set }var onAudioTracksChange: (() -> Void)? { get set }var isPlaying: Bool { get }var isSeekable: Bool { get }var duration: TimeInterval { get }var currentTime: TimeInterval { get }var remainingTime: TimeInterval { get }var position: Float { get }var audioTracks: [AudioTrack] { get }var selectedAudioTrackID: Int32 { get }var subtitleTracks: [SubtitleTrack] { get }var selectedSubtitleTrackID: Int32 { get }var subtitleDelay: TimeInterval { get }5 unmodified linesfunc togglePlayPause()func seek(to position: Float)func jump(by seconds: TimeInterval)func selectAudioTrack(id: Int32)func selectSubtitleTrack(id: Int32)func adjustSubtitleDelay(by seconds: TimeInterval)@discardableResult
Dreamio/StreamCandidate.swift
39 unmodified lines40414243444547 unmodified lines93949596979839 unmodified lineslet name: String}#if DEBUGenum SubtitleDebugFormatter {static func candidateSummary(_ candidates: [SubtitleCandidate]) -> String {47 unmodified lines}}struct StreamClassification {let sourceKind: StreamSourceKindlet containerGuess: StreamContainerGuess39 unmodified lines404142434445464747 unmodified lines959697989910010110210310410510639 unmodified lineslet name: String}typealias AudioTrack = SubtitleTrack#if DEBUGenum SubtitleDebugFormatter {static func candidateSummary(_ candidates: [SubtitleCandidate]) -> String {47 unmodified lines}}enum AudioOptionMapper {static func options(from tracks: [AudioTrack]) -> [AudioTrack] {tracks.filter { $0.id >= 0 }}}struct StreamClassification {let sourceKind: StreamSourceKindlet containerGuess: StreamContainerGuess
Dreamio/VLCNativePlaybackBackend.swift
17 unmodified lines18192021222380 unmodified lines10410510610710810983 unmodified lines19319419519619719858 unmodified lines25725825926026126252 unmodified lines3153163173183193201 unmodified line32232332432532632732832917 unmodified linesvar onFailure: ((Error) -> Void)?var onStateChange: (() -> Void)?var onSubtitleTracksChange: (() -> Void)?#if canImport(MobileVLCKit)private let mediaPlayer = VLCMediaPlayer()80 unmodified lines#endif}func selectSubtitleTrack(id: Int32) {#if canImport(MobileVLCKit)didUserSelectSubtitleTrack = true83 unmodified lines#endif}var subtitleTracks: [SubtitleTrack] {#if canImport(MobileVLCKit)let names = mediaPlayer.videoSubTitlesNames as? [String] ?? []58 unmodified lines}#if DEBUGprivate func logSubtitleTracks(reason: String) {let names = mediaPlayer.videoSubTitlesNames as? [String] ?? []let indexes = mediaPlayer.videoSubTitlesIndexes as? [NSNumber] ?? []52 unmodified linesreapplyAutoSelectedSubtitleTrackIfNeeded(reason: stateName(mediaPlayer.state))onReady?()onStateChange?()case .error:onFailure?(NativePlaybackError.playbackFailed)case .paused, .stopped, .ended:1 unmodified linecase .esAdded:selectInitialSubtitleTrackIfNeeded(reason: "esAdded")#if DEBUGlogSubtitleTracks(reason: "esAdded")#endifonSubtitleTracksChange?()default:break17 unmodified lines1819202122232480 unmodified lines10510610710810911011111211311411511611711811912012112212383 unmodified lines20720820921021121221321421521621721821922022122222322422522622722822923023123258 unmodified lines29129229329429529629729829930030130252 unmodified lines3553563573583593603611 unmodified line36336436536636736836937037137217 unmodified linesvar onFailure: ((Error) -> Void)?var onStateChange: (() -> Void)?var onSubtitleTracksChange: (() -> Void)?var onAudioTracksChange: (() -> Void)?#if canImport(MobileVLCKit)private let mediaPlayer = VLCMediaPlayer()80 unmodified lines#endif}func selectAudioTrack(id: Int32) {#if canImport(MobileVLCKit)#if DEBUGlogAudioTracks(reason: "before-select-\(id)")#endifmediaPlayer.currentAudioTrackIndex = id#if DEBUGlogAudioTracks(reason: "after-select-\(id)")#endifonAudioTracksChange?()#endif}func selectSubtitleTrack(id: Int32) {#if canImport(MobileVLCKit)didUserSelectSubtitleTrack = true83 unmodified lines#endif}var audioTracks: [AudioTrack] {#if canImport(MobileVLCKit)let names = mediaPlayer.audioTrackNames as? [String] ?? []let indexes = mediaPlayer.audioTrackIndexes as? [NSNumber] ?? []return zip(indexes, names).map { index, name inAudioTrack(id: index.int32Value, name: name)}#else[]#endif}var selectedAudioTrackID: Int32 {#if canImport(MobileVLCKit)mediaPlayer.currentAudioTrackIndex#else-1#endif}var subtitleTracks: [SubtitleTrack] {#if canImport(MobileVLCKit)let names = mediaPlayer.videoSubTitlesNames as? [String] ?? []58 unmodified lines}#if DEBUGprivate func logAudioTracks(reason: String) {let names = mediaPlayer.audioTrackNames as? [String] ?? []let indexes = mediaPlayer.audioTrackIndexes as? [NSNumber] ?? []print("[DreamioVLC] audio tracks reason=\(reason) names=\(names) indexes=\(indexes.map { $0.int32Value }) selected=\(mediaPlayer.currentAudioTrackIndex)")}private func logSubtitleTracks(reason: String) {let names = mediaPlayer.videoSubTitlesNames as? [String] ?? []let indexes = mediaPlayer.videoSubTitlesIndexes as? [NSNumber] ?? []52 unmodified linesreapplyAutoSelectedSubtitleTrackIfNeeded(reason: stateName(mediaPlayer.state))onReady?()onStateChange?()onAudioTracksChange?()case .error:onFailure?(NativePlaybackError.playbackFailed)case .paused, .stopped, .ended:1 unmodified linecase .esAdded:selectInitialSubtitleTrackIfNeeded(reason: "esAdded")#if DEBUGlogAudioTracks(reason: "esAdded")logSubtitleTracks(reason: "esAdded")#endifonAudioTracksChange?()onSubtitleTracksChange?()default:break
Dreamio/NativePlayerViewController.swift
8 unmodified lines9101112131419 unmodified lines34353637383940417 unmodified lines495051525354174 unmodified lines22923023123223323415 unmodified lines2502512522532542552562578 unmodified lines26626726826927027127227327427527627744 unmodified lines322323324325326327102 unmodified lines4304314324334344352 unmodified lines4384394404414424434444454464474483 unmodified lines45245345445545645718 unmodified lines47647747847948048148248348448548630 unmodified lines5175185195205215225235248 unmodified linesprivate var progressTimer: Timer?private var isScrubbing = falseprivate var attachedSubtitleURLs: Set<URL>private var captionsMenuSignature: String?var onDismiss: (() -> Void)?19 unmodified linesprivate let controlsContainer: UIVisualEffectView = {let view = UIVisualEffectView(effect: UIBlurEffect(style: .systemUltraThinMaterialDark))view.translatesAutoresizingMaskIntoConstraints = falseview.layer.cornerRadius = 16view.clipsToBounds = truereturn view}()7 unmodified linesprivate let playPauseButton = NativePlayerViewController.iconButton(systemName: "pause.fill", label: "Play or Pause")private let jumpBackButton = NativePlayerViewController.iconButton(systemName: "gobackward.15", label: "Jump Back 15 Seconds")private let jumpForwardButton = NativePlayerViewController.iconButton(systemName: "goforward.15", label: "Jump Forward 15 Seconds")private let captionsButton = NativePlayerViewController.iconButton(systemName: "captions.bubble", label: "Captions")private let elapsedLabel: UILabel = {174 unmodified linesself?.refreshControls()}}}private func startStartupTimer() {15 unmodified linesplayPauseButton.addTarget(self, action: #selector(togglePlayPause), for: .touchUpInside)jumpBackButton.addTarget(self, action: #selector(jumpBack), for: .touchUpInside)jumpForwardButton.addTarget(self, action: #selector(jumpForward), for: .touchUpInside)captionsButton.showsMenuAsPrimaryAction = trueplayPauseButton.layer.cornerRadius = 21scrubber.addTarget(self, action: #selector(scrubbingStarted), for: .touchDown)scrubber.addTarget(self, action: #selector(scrubberChanged), for: .valueChanged)scrubber.addTarget(self, action: #selector(scrubbingEnded), for: [.touchUpInside, .touchUpOutside, .touchCancel])8 unmodified linestimeAndScrubRow.alignment = .centertimeAndScrubRow.spacing = 8let controlRow = UIStackView(arrangedSubviews: [jumpBackButton, playPauseButton, jumpForwardButton, captionsButton])controlRow.translatesAutoresizingMaskIntoConstraints = falsecontrolRow.axis = .horizontalcontrolRow.alignment = .centercontrolRow.distribution = .equalSpacingcontrolRow.spacing = 14let stack = UIStackView(arrangedSubviews: [timeAndScrubRow, controlRow])stack.translatesAutoresizingMaskIntoConstraints = false44 unmodified linesplayPauseButton.heightAnchor.constraint(equalToConstant: 42),jumpForwardButton.widthAnchor.constraint(equalToConstant: 36),jumpForwardButton.heightAnchor.constraint(equalToConstant: 36),captionsButton.widthAnchor.constraint(equalToConstant: 36),captionsButton.heightAnchor.constraint(equalToConstant: 36)])102 unmodified linesreturn UIMenu(title: "Captions", children: trackActions + [delayActions])}private func startProgressUpdates() {progressTimer?.invalidate()progressTimer = Timer.scheduledTimer(withTimeInterval: 0.5, repeats: true) { [weak self] _ in2 unmodified lines}private func refreshControls() {let subtitleTracks = backend.subtitleTracksplayPauseButton.setImage(UIImage(systemName: backend.isPlaying ? "pause.fill" : "play.fill"), for: .normal)scrubber.isEnabled = backend.isSeekablejumpBackButton.isEnabled = backend.isSeekablejumpForwardButton.isEnabled = backend.isSeekableupdateCaptionsMenuIfNeeded(subtitleTracks: subtitleTracks)elapsedLabel.text = PlaybackTimeFormatter.label(for: backend.currentTime)remainingLabel.text = "-\(PlaybackTimeFormatter.label(for: backend.remainingTime))"3 unmodified lines[scrubber, jumpBackButton, jumpForwardButton].forEach { $0.alpha = backend.isSeekable ? 1 : 0.45 }}private func updateCaptionsMenuIfNeeded(subtitleTracks: [SubtitleTrack]) {let selectedTrackID = backend.selectedSubtitleTrackIDlet signature = captionsMenuSignatureValue(18 unmodified linestracks: [SubtitleTrack],selectedTrackID: Int32,delay: TimeInterval) -> String {let trackSignature = tracks.map { "\($0.id):\($0.name)" }.joined(separator: "|")return "\(trackSignature)#selected=\(selectedTrackID)#delay=\(String(format: "%.1f", delay))"}private func revealControls() {30 unmodified linesbutton.translatesAutoresizingMaskIntoConstraints = falsebutton.setImage(UIImage(systemName: systemName), for: .normal)button.tintColor = .whitebutton.backgroundColor = UIColor.black.withAlphaComponent(0.35)button.layer.cornerRadius = 18button.accessibilityLabel = labelreturn button}8 unmodified lines910111213141519 unmodified lines35363738394041424344457 unmodified lines53545556575859174 unmodified lines23423523623723823924024124224324415 unmodified lines2602612622632642652662672688 unmodified lines27727827928028128228328428528628728828929029129229329429544 unmodified lines340341342343344345346347348102 unmodified lines4514524534544554564574584594604614624634644654664674684694704714724734744754764774784794804814824834844854862 unmodified lines4894904914924934944954964974984995005013 unmodified lines50550650750850951051151251351451551651751851952052152252352452552652752852953018 unmodified lines54955055155255355455555655755855956056156256356456556656730 unmodified lines5985996006016026036046056066078 unmodified linesprivate var progressTimer: Timer?private var isScrubbing = falseprivate var attachedSubtitleURLs: Set<URL>private var audioMenuSignature: String?private var captionsMenuSignature: String?var onDismiss: (() -> Void)?19 unmodified linesprivate let controlsContainer: UIVisualEffectView = {let view = UIVisualEffectView(effect: UIBlurEffect(style: .systemUltraThinMaterialDark))view.translatesAutoresizingMaskIntoConstraints = falseview.layer.cornerRadius = 22view.clipsToBounds = trueview.backgroundColor = UIColor.white.withAlphaComponent(0.08)view.layer.borderColor = UIColor.white.withAlphaComponent(0.18).cgColorview.layer.borderWidth = 1return view}()7 unmodified linesprivate let playPauseButton = NativePlayerViewController.iconButton(systemName: "pause.fill", label: "Play or Pause")private let jumpBackButton = NativePlayerViewController.iconButton(systemName: "gobackward.15", label: "Jump Back 15 Seconds")private let jumpForwardButton = NativePlayerViewController.iconButton(systemName: "goforward.15", label: "Jump Forward 15 Seconds")private let audioButton = NativePlayerViewController.iconButton(systemName: "waveform.circle", label: "Audio Tracks")private let captionsButton = NativePlayerViewController.iconButton(systemName: "captions.bubble", label: "Captions")private let elapsedLabel: UILabel = {174 unmodified linesself?.refreshControls()}}backend.onAudioTracksChange = { [weak self] inDispatchQueue.main.async {self?.refreshControls()}}}private func startStartupTimer() {15 unmodified linesplayPauseButton.addTarget(self, action: #selector(togglePlayPause), for: .touchUpInside)jumpBackButton.addTarget(self, action: #selector(jumpBack), for: .touchUpInside)jumpForwardButton.addTarget(self, action: #selector(jumpForward), for: .touchUpInside)audioButton.showsMenuAsPrimaryAction = truecaptionsButton.showsMenuAsPrimaryAction = trueplayPauseButton.layer.cornerRadius = 24scrubber.addTarget(self, action: #selector(scrubbingStarted), for: .touchDown)scrubber.addTarget(self, action: #selector(scrubberChanged), for: .valueChanged)scrubber.addTarget(self, action: #selector(scrubbingEnded), for: [.touchUpInside, .touchUpOutside, .touchCancel])8 unmodified linestimeAndScrubRow.alignment = .centertimeAndScrubRow.spacing = 8let playbackCluster = UIStackView(arrangedSubviews: [jumpBackButton, playPauseButton, jumpForwardButton])playbackCluster.translatesAutoresizingMaskIntoConstraints = falseplaybackCluster.axis = .horizontalplaybackCluster.alignment = .centerplaybackCluster.distribution = .equalCenteringplaybackCluster.spacing = 14let controlRow = UIStackView(arrangedSubviews: [audioButton, playbackCluster, captionsButton])controlRow.translatesAutoresizingMaskIntoConstraints = falsecontrolRow.axis = .horizontalcontrolRow.alignment = .centercontrolRow.distribution = .equalCenteringcontrolRow.spacing = 18let stack = UIStackView(arrangedSubviews: [timeAndScrubRow, controlRow])stack.translatesAutoresizingMaskIntoConstraints = false44 unmodified linesplayPauseButton.heightAnchor.constraint(equalToConstant: 42),jumpForwardButton.widthAnchor.constraint(equalToConstant: 36),jumpForwardButton.heightAnchor.constraint(equalToConstant: 36),audioButton.widthAnchor.constraint(equalToConstant: 36),audioButton.heightAnchor.constraint(equalToConstant: 36),playbackCluster.centerXAnchor.constraint(equalTo: controlRow.centerXAnchor),captionsButton.widthAnchor.constraint(equalToConstant: 36),captionsButton.heightAnchor.constraint(equalToConstant: 36)])102 unmodified linesreturn UIMenu(title: "Captions", children: trackActions + [delayActions])}private func audioMenu() -> UIMenu {let selectedTrackID = backend.selectedAudioTrackIDlet tracks = backend.audioTrackslet options = AudioOptionMapper.options(from: tracks)#if DEBUGprint("[DreamioAudio] build-menu tracks=\(SubtitleDebugFormatter.trackSummary(tracks)) selected=\(selectedTrackID)")#endiflet trackActions = options.map { track inUIAction(title: track.name,state: track.id == selectedTrackID ? .on : .off) { [weak self] _ inguard let self else {return}#if DEBUGprint("[DreamioAudio] select-request id=\(track.id) name=\(track.name) before=\(self.backend.selectedAudioTrackID)")#endifself.backend.selectAudioTrack(id: track.id)#if DEBUGprint("[DreamioAudio] select-result id=\(track.id) after=\(self.backend.selectedAudioTrackID) tracks=\(SubtitleDebugFormatter.trackSummary(self.backend.audioTracks))")#endifself.audioMenuSignature = nilself.refreshControls()}}return UIMenu(title: "Audio", children: trackActions)}private func startProgressUpdates() {progressTimer?.invalidate()progressTimer = Timer.scheduledTimer(withTimeInterval: 0.5, repeats: true) { [weak self] _ in2 unmodified lines}private func refreshControls() {let audioTracks = backend.audioTrackslet subtitleTracks = backend.subtitleTracksplayPauseButton.setImage(UIImage(systemName: backend.isPlaying ? "pause.fill" : "play.fill"), for: .normal)scrubber.isEnabled = backend.isSeekablejumpBackButton.isEnabled = backend.isSeekablejumpForwardButton.isEnabled = backend.isSeekableupdateAudioMenuIfNeeded(audioTracks: audioTracks)updateCaptionsMenuIfNeeded(subtitleTracks: subtitleTracks)elapsedLabel.text = PlaybackTimeFormatter.label(for: backend.currentTime)remainingLabel.text = "-\(PlaybackTimeFormatter.label(for: backend.remainingTime))"3 unmodified lines[scrubber, jumpBackButton, jumpForwardButton].forEach { $0.alpha = backend.isSeekable ? 1 : 0.45 }}private func updateAudioMenuIfNeeded(audioTracks: [AudioTrack]) {let selectedTrackID = backend.selectedAudioTrackIDlet signature = trackMenuSignatureValue(tracks: audioTracks,selectedTrackID: selectedTrackID)let hasSelectableTrack = AudioOptionMapper.options(from: audioTracks).count > 1audioButton.isEnabled = hasSelectableTrackaudioButton.alpha = hasSelectableTrack ? 1 : 0.45guard signature != audioMenuSignature else {return}audioMenuSignature = signatureaudioButton.menu = audioMenu()#if DEBUGprint("[DreamioAudio] refresh-menu enabled=\(audioButton.isEnabled) tracks=\(SubtitleDebugFormatter.trackSummary(audioTracks)) selected=\(selectedTrackID)")#endif}private func updateCaptionsMenuIfNeeded(subtitleTracks: [SubtitleTrack]) {let selectedTrackID = backend.selectedSubtitleTrackIDlet signature = captionsMenuSignatureValue(18 unmodified linestracks: [SubtitleTrack],selectedTrackID: Int32,delay: TimeInterval) -> String {let trackSignature = trackMenuSignatureValue(tracks: tracks, selectedTrackID: selectedTrackID)return "\(trackSignature)#delay=\(String(format: "%.1f", delay))"}private func trackMenuSignatureValue(tracks: [SubtitleTrack],selectedTrackID: Int32) -> String {let trackSignature = tracks.map { "\($0.id):\($0.name)" }.joined(separator: "|")return "\(trackSignature)#selected=\(selectedTrackID)"}private func revealControls() {30 unmodified linesbutton.translatesAutoresizingMaskIntoConstraints = falsebutton.setImage(UIImage(systemName: systemName), for: .normal)button.tintColor = .whitebutton.backgroundColor = UIColor.white.withAlphaComponent(0.12)button.layer.cornerRadius = 18button.layer.borderColor = UIColor.white.withAlphaComponent(0.16).cgColorbutton.layer.borderWidth = 1button.accessibilityLabel = labelreturn button}
Expected Impact for End-Users
When a native-played file has multiple embedded audio tracks, users can open the new audio menu and choose the language or alternate mix they want. The play button remains centered, and the controls should feel more at home on iOS.
Validation
- Ran
xcodebuild -workspace Dreamio.xcworkspace -scheme Dreamio -destination 'platform=iOS Simulator,name=iPhone 17' build: passed. - Attempted
iPhone 16simulator validation first, but that simulator was not installed on this machine. - No real-device playback stream was available in this turn, so actual multi-audio switching still needs a device check with an MKV that contains multiple audio languages.
Issues, Limitations, and Mitigations
- MobileVLCKit track discovery is event-driven, so the menu appears after VLC reports the stream tracks. The UI refreshes on playback, buffering, and elementary-stream-added events to mitigate delayed discovery.
- The visual update uses UIKit blur/material styling rather than directly adopting iOS 26-only
UIGlassEffect, keeping the project buildable for its current iOS 16 deployment target while following Liquid Glass principles. - Manual validation is still needed on a device with a known multi-audio stream.
Follow-up Work
- Use a known multi-language MKV on device and confirm the menu lists all audio tracks and switches without playback restart.
- Consider parsing VLC track names into cleaner language labels if raw names are noisy.
- Promote the track model from a typealias to distinct audio/subtitle structs if audio-specific metadata is added later.