lamp + monoprior: catalog tools relay the source H.264 video (no per-frame JPEGs); LAMP follow eye, pinhole-space overlays, no 3D joint labels - #229
Open
pablo-rerun wants to merge 4 commits into
Conversation
…view scale, 2D overlays in the camera views, no 3D joint labels
…he stored packets
…ad of JPEG frames
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Viewer-side fixes for the LAMP robocap catalog tool from watching the saved 60-s recording of s29, plus the switch from per-frame JPEGs to relaying the source H.264 streams in both catalog tools.
Video relay (both tools). The catalog stores H.264 packets; the tools decoded them on NVDEC, re-encoded downsized JPEG previews per frame and logged those. Now
simplecv.rerun_dataloader.relay_video_streamre-logs the stored packets for the processed window as aVideoStreamat the catalog's own entity path (world/rig_00/<cam>/pinhole/video): one packets query, the codec once statically, then every sample onvideo_timefrom the last keyframe at or before the window start. The viewer decodes; the tools stop decoding for display. LAMP's previews and the wholePREVIEW_SCALEmachinery are gone, the 2D overlays are logged in native pixel coordinates, and the rig is logged at full resolution again. X-Lens drops its fisheye JPEGs and keeps the derived images (depth colormap, rectified twins, depth PNGs).LAMP viewer fixes. 2D views rooted at the camera entity so image and detections are both inside; 3D view rooted at the rig with the robocap first-person follow eye (dataforge recipe; forward = +Y, up = -Z in the rig frame, eye 3.5 m behind and 1.8 m above),
show_labels=Falseon the SMPL joints; blueprint sent withmake_active=True, make_default=Trueexplicitly.Verified on 0.37.0 with a headless viewer (pixel evidence in
/tmp/fleet-artifacts/rrd/video-relay-validation/and.../lamp-validation/on pablo-dl-server): the saved blueprint activates on load; the four 2D panes show the relayed video with overlays; the 3D frustums carry video frames; the follow eye keeps the rig at the same screen position across time; no black panes (the rerun 0.33 H.264 issue does not reproduce with ffmpeg 8.1 on PATH).rerun rrd verifyclean for both; served athttps://pablo-dl-server.ilish-ruler.ts.net:10499/rrd/{lamp-s29-60s,xlens-s21-60s}.rrd.Gates: lamp-dev lint/typecheck/deadcode/tests (48 passed); monoprior-dev tests 131 passed, monoprior-catalog-dev lint/typecheck/deadcode clean; simplecv dataloader tests (new relay test asserts keyframe anchoring and sample count). API facts:
VideoStream.from_fields(sample=, is_keyframe=);rr.VideoCodecis not an IntEnum (use.value).Follow-ups noted, not done here: exclude
.../pinhole/detections/**from LAMP's 3D view to keep the frustum frames uncluttered; the relay re-queries the segment packets once per camera (about 30 MB each); the CORS static server on :10499 now runs as the transient user unitfleet-artifacts-cors-10499.