LangSplatV2: Integrate TensorBoard logging with evaluation images

[swahtz]

Summary

Integrates TensorBoard logging with evaluation images into the LangSplatV2 training pipeline, following the GARfVDB writer pattern (GaussianSplatSegmentationWriter) and using language-feature-specific visualizations (PCA projections, error heatmaps, feature coverage).

• Create LangSplatV2Writer + LangSplatV2WriterConfig in langsplatv2/training/langsplatv2_writer.py — duplicates the GARfVDB writer interface (CSV metrics, disk image saving, checkpoints, TensorBoard) without cross-project dependency
• Add cosine_error_map() visualization utility to langsplatv2/util.py — computes per-pixel 1 - cosine_similarity and maps through the turbo colormap
• Refactor LangSplatV2Trainer to use the writer — replaces internal _log_metric() / _save_checkpoint() / file management with an injected LangSplatV2Writer instance
• Add periodic training image logging — PCA of predicted/GT features, feature coverage mask, cosine error heatmap (controlled by new log_test_images config flag)
• Enhance eval() with full image diagnostics — beauty render, predicted features PCA, GT features PCA, cosine error heatmap, alpha map, and side-by-side comparison composite
• Update train_langsplatv2.py — accepts io: LangSplatV2WriterConfig parameter and instantiates the writer before training

Fixes #54

Test plan

• Run python train_langsplatv2.py with default config (no TensorBoard, no images) and verify metrics CSV is written and checkpoints are saved as before
• Run with --io.use-tensorboard and verify TensorBoard event files are created under tensorboard/
• Run with --io.save-images --config.log-test-images and verify training images (PCA features, coverage, error heatmap) are saved to images/ at log intervals
• Run with --use-every-n-as-val 8 and verify evaluation images (beauty render, PCA features, error heatmap, alpha map, comparison) are saved at eval epochs
• Run with --io.use-tensorboard --io.save-images-to-tensorboard and verify images appear in TensorBoard
• Load a checkpoint with LangSplatV2Trainer.from_state_dict() and verify the new log_test_images config field is restored correctly

[Copilot]

Pull request overview

This pull request successfully integrates TensorBoard logging and evaluation image generation into the LangSplatV2 training pipeline, following the established GARfVDB writer pattern. The implementation adds comprehensive visualization capabilities including PCA feature projections, error heatmaps, and evaluation diagnostics.

Changes:

• Created LangSplatV2Writer and LangSplatV2WriterConfig classes to handle CSV metrics, disk image saving, checkpoints, and optional TensorBoard logging
• Added visualization utilities: pca_projection_fast, calculate_pca_projection, cosine_error_map, and _apply_turbo_colormap
• Refactored LangSplatV2Trainer to replace internal file I/O with the injected writer instance
• Added periodic training image logging (PCA features, coverage masks, error heatmaps) controlled by log_test_images config flag
• Enhanced evaluation to render and save beauty images, feature PCA projections, error heatmaps, alpha maps, and comparison composites
• Updated preprocessing to use multi-crop SAM2 generation with cross-crop NMS
• Added defensive bounds checking in build_feature_map to handle out-of-range seg_map indices
• Modified loss computation to use mask fraction scaling for gradient consistency with the original implementation

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
train_langsplatv2.py	Integrates writer instantiation and passes it to trainer instead of separate save_path/run_name parameters
langsplatv2/util.py	Adds PCA projection and cosine error visualization utilities copied from GARfVDB pattern
langsplatv2/training/trainer.py	Refactors to use writer for all I/O, adds image logging methods for training and evaluation
langsplatv2/training/langsplatv2_writer.py	New writer class following GARfVDB pattern for metrics, images, checkpoints, and TensorBoard
langsplatv2/training/dataset.py	Adds bounds checking for seg_map indices to prevent indexing errors
langsplatv2/training/init.py	Exports new writer classes
langsplatv2/loss.py	Updates loss to use mask fraction scaling and adds cosine_loss_valid metric
langsplatv2/config.py	Adds log_test_images flag and new SAM2 configuration parameters
langsplatv2/init.py	Exports LangSplatV2WriterConfig
langsplatv2/scene_transforms/multi_scale_sam_masks.py	Implements multi-crop SAM2 generation with cross-crop NMS
README.md	Documents new training features, outputs, and preprocessing pipeline

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.