Skip to content

Add topological modeling for action forward dynamics#223

Open
teerthsharma wants to merge 2 commits into
NVIDIA:mainfrom
teerthsharma:codex/topological-modeling-action-fd
Open

Add topological modeling for action forward dynamics#223
teerthsharma wants to merge 2 commits into
NVIDIA:mainfrom
teerthsharma:codex/topological-modeling-action-fd

Conversation

@teerthsharma

Copy link
Copy Markdown

Summary

This PR adds Topological Modeling for Cosmos Action Forward Dynamics as one cohesive optional evaluation layer for Cosmos3 action FD rollouts.

It covers the full modeling scope:

  • FD/FDTD modeling: adds finite-difference topology traces over rollout state so frame/chunk dynamics have deterministic velocity, acceleration, local-change, and convergence diagnostics.
  • Topology metrics: evaluates visual FD rollouts from binary or labeled masks with connected components, hole/loop proxies, Euler-characteristic-style summaries, drift, stability, JSON, and CSV outputs.
  • Topological inference: adds compact topology state vectors, weighted sparse-neighborhood distance, Betti-stability windows, and theorem-style convergence gates.
  • Swarm robotics extension: adds sparse top-k specialist ranking using topology-aware state similarity, task tags, reliability, cost, and utilization signals.

The helper is side-effect-free. It does not alter default Cosmos3 inference and does not require GPU/model weights/segmentation dependencies for CI-style validation.

Files

  • cookbooks/cosmos3/generator/action/topology_helpers.py
    • mask topology metrics
    • optional persistent-homology enrichment when ripser is available
    • finite-difference rollout traces
    • convergence gates
    • topology state vectors
    • sparse specialist ranking
    • JSON/CSV report writers
  • cookbooks/cosmos3/generator/action/topological_modeling.md
    • reviewer-facing modeling note for FD/FDTD, topology metrics, topological inference, and swarm robotics extension
  • cookbooks/cosmos3/generator/action/README.md
    • cookbook entry point and minimal mask-first usage snippet
  • tests/cookbooks/cosmos3/generator/action/test_topology_helpers.py
    • 16 CPU-only synthetic-mask tests

Verification

python -m py_compile cookbooks/cosmos3/generator/action/topology_helpers.py
python -m unittest tests.cookbooks.cosmos3.generator.action.test_topology_helpers -v
git diff --check

All passed locally.

Maintainer context

Opened proposal issue before implementation: #222

Non-goals

  • No change to default Cosmos3 generation behavior.
  • No required segmentation model dependency.
  • No GPU or model-weight requirement for tests.
  • No claim of improved generation quality without a controlled validation protocol.
  • No vendored external research prototype code.

@fwd4

fwd4 commented Jun 24, 2026

Copy link
Copy Markdown
Collaborator

heads up for reviewers: NVIDIA/NeMo-Relay#282

@mli0603

mli0603 commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Interesting feature. @teerthsharma is the idea that if FD retains the same connected components, the prediction is good? And this is used as a proxy for FD eval?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants