Skip to content

lamp + monoprior: catalog tools relay the source H.264 video (no per-frame JPEGs); LAMP follow eye, pinhole-space overlays, no 3D joint labels - #229

Open
pablo-rerun wants to merge 4 commits into
mainfrom
lamp/5-viewer-follow
Open

lamp + monoprior: catalog tools relay the source H.264 video (no per-frame JPEGs); LAMP follow eye, pinhole-space overlays, no 3D joint labels#229
pablo-rerun wants to merge 4 commits into
mainfrom
lamp/5-viewer-follow

Conversation

@pablo-rerun

@pablo-rerun pablo-rerun commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Viewer-side fixes for the LAMP robocap catalog tool from watching the saved 60-s recording of s29, plus the switch from per-frame JPEGs to relaying the source H.264 streams in both catalog tools.

Video relay (both tools). The catalog stores H.264 packets; the tools decoded them on NVDEC, re-encoded downsized JPEG previews per frame and logged those. Now simplecv.rerun_dataloader.relay_video_stream re-logs the stored packets for the processed window as a VideoStream at the catalog's own entity path (world/rig_00/<cam>/pinhole/video): one packets query, the codec once statically, then every sample on video_time from the last keyframe at or before the window start. The viewer decodes; the tools stop decoding for display. LAMP's previews and the whole PREVIEW_SCALE machinery are gone, the 2D overlays are logged in native pixel coordinates, and the rig is logged at full resolution again. X-Lens drops its fisheye JPEGs and keeps the derived images (depth colormap, rectified twins, depth PNGs).

recording (60 s, 4 cameras) before after what the viewer shows
LAMP s29 583 MB 363 MB 1920x1080 @ 30 fps (was 960x540 @ 10 fps JPEG)
X-Lens s21 885 MB 860 MB 1920x1080 @ 30 fps (was 1120x630 @ 5 fps JPEG)

LAMP viewer fixes. 2D views rooted at the camera entity so image and detections are both inside; 3D view rooted at the rig with the robocap first-person follow eye (dataforge recipe; forward = +Y, up = -Z in the rig frame, eye 3.5 m behind and 1.8 m above), show_labels=False on the SMPL joints; blueprint sent with make_active=True, make_default=True explicitly.

Verified on 0.37.0 with a headless viewer (pixel evidence in /tmp/fleet-artifacts/rrd/video-relay-validation/ and .../lamp-validation/ on pablo-dl-server): the saved blueprint activates on load; the four 2D panes show the relayed video with overlays; the 3D frustums carry video frames; the follow eye keeps the rig at the same screen position across time; no black panes (the rerun 0.33 H.264 issue does not reproduce with ffmpeg 8.1 on PATH). rerun rrd verify clean for both; served at https://pablo-dl-server.ilish-ruler.ts.net:10499/rrd/{lamp-s29-60s,xlens-s21-60s}.rrd.

Gates: lamp-dev lint/typecheck/deadcode/tests (48 passed); monoprior-dev tests 131 passed, monoprior-catalog-dev lint/typecheck/deadcode clean; simplecv dataloader tests (new relay test asserts keyframe anchoring and sample count). API facts: VideoStream.from_fields(sample=, is_keyframe=); rr.VideoCodec is not an IntEnum (use .value).

Follow-ups noted, not done here: exclude .../pinhole/detections/** from LAMP's 3D view to keep the frustum frames uncluttered; the relay re-queries the segment packets once per camera (about 30 MB each); the CORS static server on :10499 now runs as the transient user unit fleet-artifacts-cors-10499.

@pablo-rerun pablo-rerun changed the title lamp: catalog tool follows the rig, pinhole at preview scale, 2D overlays in the camera views, no 3D joint labels lamp + monoprior: catalog tools relay the source H.264 video (no per-frame JPEGs); LAMP follow eye, pinhole-space overlays, no 3D joint labels Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants