TODO list before release

[paul-nechifor]

• Delete dead code (Remove unneeded code files #718 , Remove unnecessary functions #725)
• Clean up the code (whitespace: Fix trailing whitespace #728)
• Documentation (Update README usage instructions #710)
• ??? Switch to uv?
• ??? Add tests for important untested code
• ??? Improve logging (Use structlog and store JSON logs on disk #715)
• ??? Fully working Docker
• ??? Fully working pip install dimos (Make dimos pip-installable #731 )
• ??? Local LLM agents (Support running local agents #739)
• ??? Rename agents to agents_deprecated and agents2 to agents

[linear]

DIM-283

[leshy]

DONE Transport fixes - orders of mangitude gains on LCM #713
obsolete unitree_webrtc/ dir - we can't publish with this connection.py

[leshy]

#719 prune repo garbage large files?

[ym-han]

Here's a summary of what's been done on the docs front and what's left for the rest to do, as of Dec 5, as far as I'm aware.

Substantive content

• Tutorials
  • Ready for review
    • Build your first skill
    • Equip an agent with skills
    • Build yourself a multi-agent RoboButler
  • TODO
    • Tutorials for G1 and Go2 setup and running of the full stack via both python and docker
      • I heard from Kevin that he was thinking of working on this?
    • Tutorial on how to run the drone stack
• Concepts
  • Done
    • Blueprints (thanks to Paul for the review!)
  • Awaiting review
    • Modules (probably ok not to review this again)
    • Skills (feels quite good now. But my main rec here would be to simplify some of the complexity around the software design / architecture of skills x agents)
    • Transport (I have marked with comments the sections that need careful review)
    • Agents
• API documentation
  • Done or very close to done
    • Semi-manually curated API pages for Agents and Skills (see discussion in TODO below to understand what this is about)
    • A page for the CLI tools (I've put this under the API reference, but it might be better to make it the same level as Concepts/Tutorials etc, under a "Utilities" section)
    • Docstrings
      • I think I've made PRs with docstrings for the most important foundational parts of the codebase, but let me know if there are things I've missed.
      • Examples of docstrings I've added:
        • agents2/agent.py
        • AgentSpec
        • Skills: skill.py, type.py, SkillCoordinator
          • Docstrings for the more subtle things here were also informed by additional tests I wrote (but have not included in the branch); i.e., I didn't just naively throw a coding agent at this. But I wouldn't be surprised if there's things I've missed -- there's a lot of complexity here.
        • blueprint infra, ModuleCoordinator, ModuleBase
        • Transport: transport.py, stream.py (inheritance diagram here was also grounded in more classical, programming-language-tech ways of extracting such inheritance info -- not just AI)
        • Navigation: NavigationInterface, ROSNav (not sure about 1-2 small details here), AstarPlanner, map protocols (spec/map.py)
  • TODO
    • Actually getting from docstrings to API reference pages on a docs site
      • Background
        • For any given module, there are two ways to make an API reference page for it. You can either manually put stubs for the things you want docstrings to be populated with or also programmatically generate those stubs from the code. The benefit of the former is that it gives you more control over how to present the API. And of course you could have a mix of the two: more manual curation for the most important APIs, less for the less important ones.
      • TODOs here:
        a. Decide which approach to take
        Note that we do have API pages with manually curated stubs for Agents and Skills (though the stub format is mkdocstrings's; not sure if we want to add other manually curated API pages).
        b. Implement that with Mintlify (I heard from Jimmy that that's the infra you want to use).
        • This tool (mdxify) might also be useful; I saw on GH that some people use it to generate API docs with Mintlify.
    • Adding any docstrings that still need to be added. E.g.:
      • Didn't touch Perception because Ivan said to leave that to him
      • didn't touch Manipulation either because it seemed like it was being actively worked on at the same time
• READMEs and Quickstart
  • Mostly done
    • Docs on how to setup dev environment (devcontainer)
      • There already is a docs/development.md that covers most of this, though there's a line or two that might need to be updated (e.g. "external contributors: we plan to mirror to Docker Hub")
  • Ready for review
    • Quickstart doc, linked to from README
    • The start of an attempt to update and restructure the README (remove outdated sections; restructure README; add TODOs for key remaining things)
      • This does not tackle the installation instructions
  • TODO
    • Figure out what the detailed installation instructions for users should be, and where exactly to put it (e.g. initial python uv pip install and pytest and example runfiles to run simulation to verify proper setup)
      • There was a PR, now closed, by Paul that updates part of the README; also some discord discussion on whether to supply a more convenient install script.
      • See also the README cleanup work in my PR.
    • Docs on how to contribute -- PR template, proper setup of ruff and pre-commit
      • though there's a good deal of overlap with the current docs/development.md

I have checked most of the docs quite a bit by now.

Misc fixes

Some of these might already have been addressed in the meantime by other people.

• accumulate_string reducer bugfix: Fixed leading newline bug
• append_history typing: Fixed typing issues in agent.py and AgentSpec
• _check_if_stuck docstring: Fixed wrong return type in manipulation_module.py

Other misc TODOs

• Migrating to Mintlify or whatever other docs site provider you prefer (important)
• Simple plumbing: when generating the llms.txtes, probably need to export the tutorial notebooks to markdown first and then include or link to the markdown files from the llms.txt
• Make sure the current links work after migration, and probably try to add more links too
• There's some overlap between some of the old docs / readmes (e.g. the old modules doc, the blueprints readme) and the new docs; also probably some parts of the older docs might be out of date. Would be good to minimize overlap (e.g. for the blueprints readme, by removing parts that overlap from the old docs and linking to the new docs instead for those parts; and for the older modules doc, deleting it if it doesn't add anything over and above the new modules+blueprints docs)

Nice-to-haves

• Add xdoctest to CI
• If we want to stick with marimo notebooks, add marimo check etc to CI
• Have some way to keep nix setup in sync with dockerfile, or generate the latter from the former

---

See https://gist.github.com/ym-han/1eefc2057ffd02428ac3405821959768 for a guide to the dependency structure / graph of the docs PRs
All the changes have been combined into one PR (#809 )

[leshy]

DONE lfs download is quiet and makes you think something is wrong. also if git lfs install hasn't been executed for the repo, there needs to be an error message about it

[leshy]

wrap deployed modules to prevent dask from using it's transports for RPC (causes weird issues)

[leshy]

rerun
nav + person follow tune in the large office

[leshy]

• add a message/progress bar for archive extracting post lfs
• how does user get fg config from dimos? can dimos call fg with dash json + url via cli args?
• add a warning if computer clock is out of sync with unitree clock