Skip to content

fix: pass videoCodec=H265 to JMuxer for HEVC streams (closes #23)#24

Open
josha wants to merge 4 commits into
caplaz:mainfrom
josha:fix/h265-muxer-codec
Open

fix: pass videoCodec=H265 to JMuxer for HEVC streams (closes #23)#24
josha wants to merge 4 commits into
caplaz:mainfrom
josha:fix/h265-muxer-codec

Conversation

@josha
Copy link
Copy Markdown

@josha josha commented May 16, 2026

Summary

Fixes #23 — H.265 streams (T86P2 4G LTE Cam S330, T8170 SoloCam S340 in H.265 mode) reach the plugin correctly identified as codec: H265 but downstream consumers (Rebroadcast prebuffer, HomeKit, WebRTC) time out waiting for fMP4 frames they can never decode.

Root cause: JMuxer defaults its videoCodec option to "H264". The current handleMuxedClient never overrides it, so the muxer builds an AVCC sample description over HEVC NAL units and emits un-decodable fMP4. H.264 cameras happen to work because the default matches reality.

Fix: Make handleMuxedClient async, hold the socket in a new pendingMuxerSockets set (so it still counts as a consumer and the upstream livestream starts), await waitForVideoMetadata, then construct JMuxer with videoCodec: "H265" when the stream metadata says HEVC.

Commits

  1. fix(eufy-stream-server): pass videoCodec=H265 to JMuxer for HEVC streams
    • Deferred JMuxer construction until first-frame metadata is known
    • Tightens the H.265 snapshot resolver to require an IRAP slice (types 16–23) rather than accepting parameter-set-only events (defensive for cameras that deliver VPS/SPS/PPS out of band)
    • Fixes the hardcoded Captured H.264 keyframe log to use the actual stream codec
    • 4 new unit tests + updates to existing tests covering the async/deferred path
  2. fix(eufy-stream-server): cap startLivestream retries on wedged upstream
    • Adds a consecutiveNoDataStarts counter; after 3 consecutive startLivestream attempts with no video data, emit streamError and stop the recursive 30s retry timer
    • Counter resets the instant livestreamActualState flips true
    • Prevents the plugin from waking battery cameras every 30s forever when an upstream P2P session is stuck (HomeBase responds acknowledged: true but returns no data — observed in eufy-security-ws logs as Result data for command not received / Stopping the station stream ... no data for 5 seconds). Live diagnostic showed this pattern drained battery-camera batteries during multi-hour test sessions

Diagnostic methodology

I instrumented the live stream pipeline with per-event byte/NAL-type logging (temporary diagnostic branch, not in this PR) on real T86P2 + T8170 cameras connected to a HomeBase S380. The logs definitively showed:

  • Bytes arrive as Annex-B framed (start=Annex-B/4B on every event)
  • H.265 parser correctly identifies VPS_NUT (32), SPS_NUT (33), PPS_NUT (34), and IDR_W_RADL (19) when bundled
  • The snapshot path was succeeding for H.265 (correctly identifies the IRAP, captures the bytes, FFmpeg produces a JPEG with -f hevc)
  • The muxed/fMP4 path was failing solely because of the JMuxer codec mismatch

This rules out the keyframe-detection hypothesis and the codec-misidentification hypothesis from the earlier triage. The actual bug is one line of muxer wiring.

Hypothesis disposition (from the issue)

# Hypothesis Status
1 H.265 keyframe detection missing Ruled out — IDR_W_RADL(19) correctly detected; snapshots succeed
2 Codec misidentification Ruled out — event.metadata.videoCodec is correctly "H265"
3 Missing parameter sets / muxer wiring Confirmed — muxer wiring. JMuxer built without videoCodec: "H265" produces fMP4 with the wrong sample description box; live consumers can't decode it
4 Rapid stream cycling Not primary — but the unbounded retry on a wedged upstream is a real harm vector (the second commit)

Why the previous "NAL type 14 / data partitioning" patch existed

H.265 PREFIX_SEI_NUT (NAL type 39) has first-byte 0x4E. Interpreted as H.264 (which expects byte0 & 0x1F for the type), that's type 14 = Data Partitioning. The earlier ERROR_RESILIENT_CAMERA_TYPES / NAL-type-14 fix was masking the underlying codec confusion for some streams; this PR addresses it at the actual source.

Verification

  • ✅ All 572 unit tests pass; lint zero warnings
  • ✅ Live: T86P2 4G LTE Cam S330 H.265-only (Front Door, Patio) — live view works in Scrypted and HomeKit; snapshot succeeds
  • ✅ Live: T8170 SoloCam S340 in H.265 mode (Upper Garage, Porch) — live view works; snapshot succeeds
  • ✅ Live: T8170 in H.264 mode — no regression (was the working baseline)
  • JMuxer emitting fMP4 (first chunk: ... codec=H265 ...) log line observed for all H.265 cameras after this fix
  • 🟡 Out of scope: occasional T86P2 sessions where the upstream eufy-security-ws delivers audio metadata but never invokes the LIVESTREAM_VIDEO_DATA event. This appears to be an upstream / Eufy P2P bridge reliability issue, not a plugin-layer issue; verified by attaching diagnostic logging and observing zero video events while audio metadata is captured. Left to upstream maintainers (bropat / eufy-security-client).

Notes for reviewers

  • packages/eufy-stream-server/src/jmuxer.d.ts gains a videoCodec field. JMuxer 2.1.0 (current dep) supports "H264" and "H265" natively via separate remuxers; no upstream change needed.
  • handleMuxedClient is now async and waits up to the default waitForVideoMetadata timeout (10s) before either constructing the muxer with the correct codec or closing the socket. Closing on timeout is intentional — it lets the downstream consumer (Rebroadcast) retry rather than building a wrong-codec muxer that wedges the consumer.
  • I'm available for follow-ups, more tests, or to split the retry-cap into a separate PR if you'd prefer two focused changes.

🤖 Generated with Claude Code

Josh Anon added 3 commits May 15, 2026 21:18
JMuxer defaults videoCodec to "H264"; the muxed client handler never
overrode it, so HEVC cameras (T86P2/4G LTE Cam S330, S340 in H.265 mode)
produced fMP4 with an avcC sample description over HEVC NAL units. The
Rebroadcast plugin's downstream ffmpeg then ran into "timeout waiting for
data" and HomeKit/WebRTC live view never started. Snapshot path was
unaffected (writes raw bitstream to ffmpeg with -f hevc) but appeared to
fail because Rebroadcast wedged the upstream session.

Codec isn't known when the muxer client first connects, so handleMuxedClient
is now async: register the socket as pending (counts as a consumer so the
upstream livestream still starts), await waitForVideoMetadata, then build
the JMuxer with videoCodec="H264" or "H265".

Also tightens the H.265 snapshot resolver to require an actual IRAP slice
(types 16-23) rather than accepting parameter-set-only events. T86P2
happens to bundle VPS+SPS+PPS+IDR so this didn't bite, but cameras that
deliver parameter sets out-of-band would resolve snapshots with bytes
ffmpeg couldn't decode.

Plus: log the actual captured-stream codec in "Captured ... keyframe"
instead of always saying "H.264".

H.264 path is unchanged. Tested live on T86P2 (H.265-only) and S340
(both H.264 and H.265 modes).
When the upstream HomeBase / eufy-security-ws is wedged (accepts
CMD_START_REALTIME_MEDIA but returns no P2P data, observed as
'Stopping the station stream ... no data for 5 seconds' / 'Result data
for command not received' in the WS server logs), the previous
ensureLivestreamState recursion would issue another startLivestream
every 30 seconds forever as long as any consumer kept livestreamIntendedState=true.
Each extra command piles backpressure onto an already-stuck HomeBase
and prolongs the wedge.

Add a consecutive-no-data counter. After MAX_NO_DATA_STARTS (=3)
back-to-back startLivestream attempts that never produced a video
event, give up: clear intent, emit streamError, and stop the recursive
30s timer. Counter resets to zero the instant video data actually
arrives (livestreamActualState=true), so this only kicks in for
genuinely-stuck sessions, not slow-warmup ones.

Recovery is consumer-initiated: a fresh muxer or snapshot request
sets intent back to true and the cycle restarts. Combined with a
physical HomeBase reset (the actual fix for a wedged upstream), this
keeps the plugin from making the problem worse while the user
recovers the hardware.
Battery / cellular Eufy cameras (T8170 S340 deep-sleep wake, T86P2 4G
LTE Cam S330 cold P2P) routinely take 30–45s to deliver their first IDR
after startLivestream. Observed empirically: Patio (T86P2) consistently
delivered video on the third 30s retry of ensureLivestreamState (≈45s
after the request), with the camera responding correctly to the
underlying eufy-security-ws commands the whole time.

The previous 15s timeouts caused two failure modes during that warm-up
window:

1. `handleMuxedClient` timed out and built JMuxer with the wrong default
   codec (H.264) just before real H.265 frames arrived. The fMP4 it
   produced was un-decodable; Rebroadcast/WebRTC saw no usable output
   and gave up.
2. `captureSnapshot` timed out and reported "no keyframe received" even
   though one was about to arrive seconds later.

Bumping both defaults to 60s lets the camera's natural startLivestream
retry cycle complete before either consumer abandons the session.
Existing user-supplied timeouts (via RequestPictureOptions.timeout) are
respected unchanged.
@josha
Copy link
Copy Markdown
Author

josha commented May 16, 2026

Adding a third commit based on live-camera evidence from this evening's testing:

fix: extend metadata + snapshot timeouts to 60s for slow-warmup cameras

Battery and 4G LTE Eufy cameras (T8170 SoloCam S340 deep-sleep wake, T86P2 4G LTE Cam S330 cold P2P) routinely take 30–45s to deliver their first IDR after the upstream-issued startLivestream. Observed empirically: a Front Door T86P2 livestream attempt that delivered video on the second 30s retry of ensureLivestreamState (≈45s after request), with Stream started + Captured video metadata: 1280x720 @ 15fps, codec: H265 + Muxed client attached (codec=H265, ...) all firing cleanly when given enough time.

With the previous 15s timeouts:

  1. handleMuxedClient timed out and built JMuxer with the wrong default codec (H.264) just before the real H.265 frames arrived. The resulting fMP4 was un-decodable downstream.
  2. captureSnapshot timed out and reported "no keyframe received" even though one was about to arrive seconds later.

Bumping both waitForVideoMetadata (in handleMuxedClient) and captureSnapshot's default to 60s lets the camera's natural startLivestream retry cycle complete before either consumer abandons. User-supplied RequestPictureOptions.timeout is still respected unchanged. The retry cap from the previous commit (3 × 30s = 90s) still bounds the wedged-upstream case.

Verified end-to-end live in HomeKit on Front Door (T86P2 H.265): live view, snapshot, audio all working after this change.

When bropat's P2P session to a station goes zombie (startLivestream
acked but no LIVESTREAM_VIDEO_DATA events flow), the plugin now
detects the wedge and recycles the per-station P2P session via
station.disconnect/connect — recovering without a HomeBase reboot or
plugin reload.

Two detection paths, both routing through a shared markUpstreamWedged():

- Cold-start: existing 3-consecutive-no-data-starts counter.
- Mid-session: new data-flow watchdog tracking lastVideoDataAt. Fires
  when intent=true, prior data existed, >15s since last frame, and
  consumers are still attached. Catches the common "stream was
  working, then stopped" pattern the counter misses.

Recycle waits for the actual STATION_EVENTS.CONNECTED event (or
CONNECTION_ERROR / 30s timeout) before declaring complete, since
station.connect() returns when bropat accepts the command, not when
P2P is re-established (10–25s for cellular cameras / cold HomeBase).

Stream server defers startLivestream while a recycle is in flight,
and auto-arms once it clears if consumers are still waiting — so a
HomeKit session arriving mid-recovery gets video without the user
retrying.

Battery-safe gating: watchdog requires totalConsumers > 0; no
auto-restart of the livestream after recycle in absence of consumers;
recycle is rate-limited to once per 5 minutes per device.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

T86P2 (4G LTE Cam S330): H.265 stream identified but no keyframes detected — snapshot and prebuffer timeout

1 participant