Add Chromecast-style remote control API (REST + WebSocket) with CEC home-theater volume support - #5896
Open
akshaynexus wants to merge 38 commits into
Open
Add Chromecast-style remote control API (REST + WebSocket) with CEC home-theater volume support#5896akshaynexus wants to merge 38 commits into
akshaynexus wants to merge 38 commits into
Conversation
- RemoteApiServer: NanoHTTPD HTTP server + UDP discovery listener - RemoteApiBridge: Static adapter between HTTP API and PlayerEngine - RemoteApiAuthProvider: Token/pairing code auth with rate limiting - RemoteApiData: SharedPreferences persistence for API settings - RemoteApiSettingsPresenter: TV settings UI for Remote API config - 40+ REST endpoints for player control, browsing, search, subtitles - API docs at docs/remote-control-api.md
- RemoteApiServer extends NanoWSD instead of NanoHTTPD - WebSocket endpoint at ws://host:8497/ws?token=<token> - Server broadcasts player state at 2Hz (every 500ms) - Client can send commands over WebSocket (play/pause/seek/etc) - Full Chrome extension WebSocket client example in docs - Updated API docs with WebSocket section and architecture diagram
…er/refresh endpoints - HomeTheaterController: standalone class for AudioManager + HDMI CEC control - Volume/mute via AudioManager - Audio output switching via CEC setsystemaudiomode/setarc - Theater levels (subwoofer/rear/immersive AE/sound mode) via CEC vendor commands - Power toggle via KEYCODE_POWER - State refresh from dumpsys hdmi_control with CEC regex parsing - TheaterStateListener callback for WebSocket broadcast on state changes - Auto-refresh after setting changes (900ms for levels, 2200ms for output switch) - RemoteApiServer: - POST /api/theater/power/toggle - toggle TV power - POST /api/theater/refresh - sync refresh from CEC hardware - GET/PUT /api/theater/audio_output - read/set output (tv/theater) - WebSocket commands: theater_power_toggle, theater_refresh - Theater state broadcast via TheaterStateListener on all WS clients - 2Hz player state broadcast continues as before - RemoteApiBridge: search, queue, subtitles, mute toggle support - API docs updated with all new endpoints and WebSocket commands
- GET /api/theater now does synchronous refresh from dumpsys hdmi_control (detects if TV fell back to TV speakers on its own) - theater_get_state WS command also reads fresh from hardware - Added fallback: if dumpsys fails, try cmd hdmi_control dump - Log error when dumpsys permissions are missing
- RemoteApiDebugActivity: scrollable UI with buttons to test CEC commands - Run dumpsys hdmi_control (shows raw output) - Set HT mode (setsystemaudiomode on + setarc on) - Set TV mode (setsystemaudiomode off + setarc off) - Get volume (cmd media_session volume) - CEC vendor command test (subwoofer level) - Theater State (refresh from hardware + display parsed state) - Audio Output (show cached output) - Clear log - Accessible from Remote API settings -> Home Theater Debug button - Monospace green-on-dark theme for readability
- RemoteApiDebugActivity: scrollable UI with buttons to test CEC commands - dumpsys, set HT/TV mode, volume, CEC vendor cmd, theater state - Accessible from Remote API settings -> Home Theater Debug button - AGENTS.md: document Java 17 requirement (AGP 7.4.2 + Java 21 = D8 NPE)
…output only - Removed CEC endpoints (output switch, subwoofer, rear, sound_mode, immersive_ae) - Removed TheaterStateListener and theater state broadcast from WebSocket - HomeTheaterController now only wraps AudioManager (volume/mute/power) - Added getAudioOutput() via AudioManager.getDevices() to detect tv vs theater - Debug activity shows ADB commands reference for CEC features - Docs updated: CEC endpoints removed, ADB commands kept as reference for extensions
- RemoteApiAuthProvider: store the rate-limit window timestamp in a long[] instead of int[]. The old code cast System.currentTimeMillis() to int, truncating the 64-bit value and corrupting the rate-limit window so pairing brute-force protection silently failed. Also switch the per-IP map to ConcurrentHashMap since it is touched by multiple NanoHTTPD request threads. - RemoteApiServer: return 429 Too Many Requests (instead of a generic 401) when pairing is rate-limited, so clients can distinguish a wrong code from a back-off; fix invalid-JSON status from 422 to 400; close the UDP DatagramSocket on stop() so the blocked receive() loop actually exits (interrupt() alone can't wake it), and guard super.stop() so a failure there can't leave the port bound. - RemoteApiBridge: mark the shared static zoom/rotation/flip/volume fields volatile for cross-thread visibility. - docs: document the HTTP status codes the API returns. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- New "Allow all local connections (no pairing)" switch (default on).
When enabled, REST and WebSocket requests need no bearer token —
the LAN is trusted, like Chromecast. /api/system/ping now reports
pairing_required so clients can skip the pairing flow.
- Fix readBody() to read PUT bodies: NanoHTTPD only fills postData
for POST; PUT bodies go to a temp file under "content". Previously
every PUT endpoint (volume, speed, formats, zoom…) saw "{}" and
returned 400 Invalid JSON.
- Document both changes in docs/remote-control-api.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
NanoHTTPD closes sockets after ~5s without a read; state broadcasts are writes and don't reset that timer, so clients were dropped every 5s. Ping all WS clients every 3s from the broadcast loop — the automatic pong from the client resets the read timeout. Required for browsers, whose JS WebSocket API cannot send pings itself. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Commands arrive on NanoHTTPD worker threads, but ExoPlayer and the playback controllers (VideoLoaderController.loadNext/loadPrevious, format selection, openVideo, queue ops, dpad…) must be driven from the main looper. Off-thread calls made next/previous and several other commands silently fail. All mutators in RemoteApiBridge now post to a main-thread Handler; reads keep their existing behavior. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
VideoStateController re-applies PlayerData's saved volume on every video load, so a volume set via the remote API reverted (e.g. back to 10%) as soon as the next video started. setVolume now also writes the value to PlayerData. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- handleNext/handlePrevious drop repeat calls within 700ms: hammering skip before the new video's metadata loads made VideoLoaderController toast "Please wait while data is loading…" repeatedly. - Player state and suggestions now prefer the high-res background image over the small card thumbnail (bestThumbnail). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
AudioManager.setStreamVolume is silently ignored when a CEC soundbar / home theater handles audio — CEC only understands volume-key steps, so the volume slider appeared to do nothing (only ±1 worked). setVolume now tries the absolute set, and when the target isn't reached, ramps the delta with paced adjustStreamVolume steps on a background thread (downs paced slower — devices drop them when sent too fast). A newer setVolume cancels an in-flight ramp. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Measured against a Sony Bravia + HT-A9 over adb: - volume-key steps are dropped unless paced ~200ms apart (150ms lost 3 of 5 presses; 200-250ms landed 100%) - the volume reported back via CEC <Report Audio Status> lags 1-2s, so checking "reached target?" mid-loop reads stale values The ramp now fires the computed number of steps blind at 220ms pacing, waits for the report to settle, then runs up to two correction passes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ed feed - HomeTheaterController: single persistent ramp worker that counts its own steps and retargets on new setVolume calls — cancel/restart let a burst of slider updates strangle the ramp after one step each (CEC reads lag 1-2s) - RemoteApiBridge: report PlayerData volume instead of player.getVolume(); loudness normalization multiplies the effective value per video, so reading it back and persisting ratcheted the volume lower on every video change - getQueue items now include thumbnail_url/duration_ms/is_live - New GET /api/content/recommended: the user's Home recommendations (ContentService.getRecommended, 5-min cache), unlike the related-videos suggestions list; docs updated Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
POST /api/content/suggestions/:idOrIndex now accepts a video ID — the bridge resolves it against the player's suggestion groups (keeping playback context) and falls back to opening the bare ID. Index form kept for compatibility. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Author
|
The player not persisting internal volume still a issue,have to investigate and fix |
…ote API
- Playlist: shuffle() and move(from,to) primitives that keep the current
item current, plus a monotonic mGeneration counter bumped on every
structural mutation (exposed via getGeneration()) so clients can cheaply
detect queue changes.
- RemoteApiBridge:
- addPlaylistToQueue(playlistId, shuffle): pages a whole YouTube playlist
on the worker thread (blocking, like getRecommended) capped at 500 items
/ 20 continuation pages, then enqueues + optional shuffle + auto-starts
if idle on the main thread.
- extractPlaylistIdFromUrl() (list= param); openVideo now resolves a
playlist id from the url when one isn't passed explicitly.
- shuffleQueue() / moveQueueItem(from,to); togglePip() (enters PIP).
- queue_generation added to the player-state JSON.
- RemoteApiServer: POST /api/player/queue/shuffle, /queue/move,
/queue/playlist, /api/player/pip + matching WebSocket commands, with the
existing auth and body-validation patterns.
- docs/remote-control-api.md updated for all new endpoints and commands.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
docs/openapi.yaml describes all 44 REST routes (system, pairing, player transport, playback settings, tracks, video transforms, content, queue, home theater) with request/response schemas, the conditional bearer-auth scheme (paired vs open mode), 202/400/401/429 responses, and reusable component schemas (PlayerState, Video, *Format, QueueItem, etc.). The WebSocket and UDP-discovery channels are documented in the info section since OpenAPI 3.0 can't express them. Also adds the previously-undocumented GET /api/theater/refresh to the markdown reference. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The internal player volume kept getting quieter every video and had to be re-set by hand. Two coupled causes, both gated behind auto-volume (default on): - VideoStateController.restoreVolume(): at a >=100% baseline it REPLACED the user's volume with the per-video loudness-normalization gain (often < 1), so every new video played near-inaudible until the user re-set 100%. Now a >=100% baseline keeps full volume; auto-volume normalization still applies for below-max baselines. - ExoPlayerInitializer.setupVolumeBoost(): armed a x2 LoudnessEnhancer whenever auto-volume was on, whose limiter audibly compresses/"sidechains" the output above ~100%. The booster now only arms for below-max baselines (where it adds normalization headroom), so 100% is a clean literal full volume. Also (from the earlier pass): - RemoteApiBridge.setVolume() clamps to 0..1 so a remote client can't persist a >1.0 gain that re-arms the booster; getVolume() reports the PlayerData baseline. - Utils volume-key paths read/step from the PlayerData baseline instead of the normalized effective volume, which previously ratcheted the baseline down. Net: set the player volume once (e.g. 100% from the macOS controller) and it stays put across video changes, with no limiter artifacts. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
GET /api/content/search/results?query=<text>&limit=<n> returns YouTube search results as JSON without starting playback, unlike POST /api/content/search which blindly plays the first hit. Lets remote clients show a results picker so the user chooses what to play. Items match the /api/content/suggestions shape (video_id, title, author, thumbnail_url, duration_ms, is_live). limit defaults to 20, capped at 50. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
GET /api/player/chapters returns the current video's YouTube chapters (title, start_ms, end_ms, thumbnail_url) so remote clients can render chapter markers and jump menus. Chapters come from the same SuggestionsController list that drives the on-TV seekbar segments; end_ms is derived from the next chapter's start (video duration for the last one). Returns [] when no video is playing or it has none. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Default player volume: 0.10f → 0.50f - getPlayerState() returns idle state when no video loaded (was null) - Push suggestions_updated event to WebSocket clients on load - Add pushEvent() static helper on RemoteApiServer
|
Hi @akshaynexus, it's funny that we both had the same idea, i like it. I think your Code might even offer more functionality than mine. the only thing that might be currently missing is PiP Toggle control. and i am currently not sure about how to use the api. Can you create an openapi spec so we can compare things better? it would be cool if we could define the api in a way, that multiple clients can benefit from it (meaning it is not being tailored exactly to one client, but for overall use) Also i am interested in the recommendations endpoint? does it give recommedations to the current video / current playlist content ? best regards |
Author
There is a openapi specially file in it,alternatively the docs of the remote api has more info on the api endpoints Mostly just vibe coded it as you can see from the commit history and using it locally for a while now with the macos controller app I published and a wip android controller client I'm working on publishing for this api /pr the recommendations yes it give sreceommendations data based on the video playing but th eplaylist fucntioanltiy and how the redommended videos show up isnt tested yet,i halso havent tested how the palylists work with the rest api yet |
Author
|
Yes pip toggle is missing in this pr but can add that easily in next few changes The openapi specially might be outdated so I suggest using either free usage of opencode or other paid coding agents to go through the api docs if you have any issues with the rest /websocket api |
… + caption enable
Tech debt / dedup (net -310 lines in remoteapi/ while adding endpoints):
- RemoteApiServer: route-table dispatch (replaces ~265-line if-chain),
okResponse()/parseBody()/orEmptyArray() helpers, forEachOpenClient()+sendEnvelope()
for the broadcast loops, WS sendResponse(); wire up the dead handleSetString
(drops 4 duplicate handlers).
- RemoteApiBridge: withPlayerOnMain(), videoToJson()/mediaItemToJson(),
forEachSuggestion()/countSuggestions(), setFormatById(), selectedTracksJson();
FQNs -> imports.
- New RemoteApiConstants (dedup DEFAULT_PORT/API_VERSION); hardcoded UI strings -> strings.xml.
New endpoints:
- Real PIP toggle (enter AND exit via ViewManager.movePlayerToForeground()):
GET /api/player/pip (status), POST /api/player/pip/toggle, POST /api/player/pip (alias).
- PUT /api/player/subtitle {enabled} for deterministic caption enable/disable.
Bug/correctness fixes:
- getDeviceId() returned the literal "android_id" key -> persisted UUID.
- getAppVersion() returned the API version -> real PackageManager versionName.
- Constant-time token/code compare; atomic pairing-code consume (double-verify race);
queue-move bounds check; volume NaN/Infinity guard; suggestion id regex {6,16}->{11}.
Docs:
- openapi.yaml: add /api/player/chapters and /api/content/search/results (were in code,
missing from spec), PIP status/toggle, subtitle PUT, Chapter schema (66 ops, 1:1 with code).
- remote-control-api.md: PIP/subtitle rows, captions=subtitle note, toggle_pip semantics,
fix duplicate 4.9/4.10 section numbering + TOC anchor.
Verified: common module compiles; routes match OpenAPI 1:1; no duplicate route keys.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The remote D-pad API (GET /api/system/dpad) used Instrumentation.sendKeySync/ sendKeyDownUpSync to inject keys. On the main thread (where remote commands run) those call validateNotAppThread(), throwing an uncaught RuntimeException that crashed the app. System-wide injection also requires INJECT_EVENTS, which a normal app lacks. Switch to dispatching the key into the app's own current activity: - MotherActivity now tracks the currently-resumed instance (sCurrentActivity, set in onResume / cleared in onPause) and exposes it via getCurrentActivity(), so Utils.sendKey can reach the live view hierarchy. - Utils.sendKey(KeyEvent) dispatches through Activity.dispatchKeyEvent. That alone fixes BACK (handled at the Activity level) but NOT arrow keys or OK: default D-pad focus navigation and item clicks are performed by ViewRootImpl, which we bypass when calling dispatchKeyEvent directly. So when the view hierarchy doesn't consume the key, we now replicate ViewRootImpl.performFocusNavigation() ourselves — focusSearch()+requestFocus() for the arrows, performClick() on the focused view for DPAD_CENTER/ENTER. - RemoteApiBridge.dpad() drops the left/right long-press down/up split (only meaningful with the old auto-repeating injection) for a clean DOWN+UP per press. Verified on a Sony BRAVIA (API 31): up/down/left/right move leanback focus and no longer crash. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A video opened by bare id (search → play, or the remote open endpoint) starts with a null `title` field; only `metadataTitle` gets populated once metadata loads — same as author/channel/duration, which is why those appeared but the title did not. videoToJson() used Video.getTitle() (deArrowTitle ?: title), which ignores metadataTitle, so org.json dropped the null and clients showed "untitled" / "no video loaded". Use Video.getTitleFull() (deArrowTitle ?: metadataTitle ?: title) instead. This shared serializer feeds both /api/player and /api/player/queue, so both now carry the title. Verified on a Sony BRAVIA: open-by-id then GET /api/player returns the proper title. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two related fixes to /api/content/suggestions (the controller's "Up Next"): 1. Populate it for autoplaying videos. getSuggestions() read only the on-screen player's loaded suggestion rows (getSuggestionsByIndex). Those rows are only filled when the player UI renders them — a video autoplaying in the background never loads them, so Up Next came back EMPTY (suggestions_count=0). Now fall back to the current video's related videos straight from metadata (getMetadata(videoId).getSuggestions()), the same robust path getRecommended() uses, so Up Next is populated regardless of the on-TV player UI state. 2. Lead with what Next actually plays. The first Up Next item disagreed with the Next button: Next plays the autoplay continuation (MediaItemMetadata.getNextVideo, cached as Video.nextMediaItem), but the related/suggestion list doesn't start with it (e.g. Up Next showed "ACAI" while Next played "JoyRide"). getSuggestions() now leads with Video.nextMediaItem (no extra network call) and de-dupes the rest, so Up Next[0] == the next-playing video. Verified on a Sony BRAVIA: suggestions[0] is the autoplay-next (JoyRide) and the list is populated (30 items) even right after an autoplay advance. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… section group Reverts the earlier workarounds (per-call metadata network fetch that reshuffled the list, and the re-derived "next" lead) in favour of SmartTube's own loaded suggestion rows (SuggestionsController populates them from cached metadata in a stable order) — so /api/content/suggestions is reliable and consistent across calls with no fresh fetch. Also exclude the current video's OWN group from the result. SuggestionsController .appendSectionPlaylistIfNeeded() adds that group as a suggestion row; when the video was reached via autoplay/Home it IS the Home/recommended feed, which showed up as "Up Next mixed with home recommended". Pre-seeding the seen-set with the group's ids skips them, leaving the video's actual related suggestions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a full HTTP/WebSocket remote control API to SmartTube so anything on your LAN can drive playback, search, and volume. Open/trusted by default, optional 6-digit pairing for auth.
RemoteApiBridgeis a thin static adapter over existingPlayerEnginemethods — no new playback logic. ~4k lines in a newmisc/remoteapipackage, plus settings UI, debug activity, and full spec indocs/remote-control-api.md.Server & transport
RemoteApiServer— NanoWSD on port 8497, HTTP + WebSocket + UDP discovery ({"action":"discover"}→ device info)RemoteApiBridge— static adapter overPlayerEngine, no new playback logicRemoteApiAuthProvider— token + pairing code auth with rate limitingRemoteApiData— SharedPreferences persistenceRemoteApiSettingsPresenter— on-TV settings UI (enable, pairing mode, debug)~45 REST endpoints:
/api/system— ping, dpad, voice/api/player— play/pause/stop/seek/speed/pitch/volume/mute/queue/subtitles/zoom/format selection/api/content— open, search, suggestions, recommended/api/theater— HDMI-CEC volume (abs/up/down), mute, power toggle, audio output switchWebSocket broadcasts player state at 2 Hz, commands accepted over it too. Server-side pings work around NanoHTTPD's ~5s idle timeout.
HDMI-CEC volume
AudioManager.setStreamVolumeis silently ignored when a CEC soundbar owns audio output. Falls back to a pacedadjustStreamVolumeramp on a background thread — ≥200ms between steps, tuned on real Sony Bravia + HT-A9 hardware (CEC drops steps below that; status reports lag 1–2s). Single persistent ramp worker retargets on new calls instead of cancel/restart. TV-speaker vs. theater output detected viaAudioManager.getDevices().RemoteApiDebugActivityavailable on-device for testing CEC commands.Fixes
PlayerData, survives video changes viaVideoStateControllerPlayerDatanotplayer.getVolume()— loudness normalization was ratcheting it downreadBody()handles PUT bodies — NanoHTTPD only populatespostDatafor POST, every PUT was returning400int→long+volatileBuild note
Java 17 required — AGP 7.4.2 + Java 21 hits a D8 NPE. See
AGENTS.md.Testing
Tested full functionality with a custom remote control client utilizing the websockets and REST API
