feat: add managed local AI with llama-server - #1178
Open
joelagnel wants to merge 23 commits into
Open
Conversation
Move physical-memory discovery into one shared implementation. Give setup and inference code a consistent RAM capacity value. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Capture CPU, memory, and GPU facts in a reusable host snapshot. Provide the hardware input used by Local AI eligibility selection. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Roll back both interrupted and completed steps when setup is cancelled. Use an independent cleanup token so cancellation cannot strand partial state. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Detect and configure WSL mirrored networking only with explicit consent. Preserve existing configuration and restore it when setup rolls back. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Centralize runtime, model, manifest, preset, and staging locations. Keep Local AI filesystem decisions immutable and consistent across components. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Download, verify, extract, and install native archives atomically. Clean staging data and restore prior state when acquisition fails. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Persist installed runtime and model metadata with atomic file updates. Give setup, startup, and cleanup one durable source of installation truth. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Host native Local AI processes with bounded startup and shutdown behavior. Capture output and terminate owned process trees reliably on Windows. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Define immutable llama.cpp, CUDA, and GGUF catalog entries. Pin versions, URLs, hashes, and hardware requirements for reproducible installs. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Select the best qualified Local AI recipe from the host snapshot. Return deterministic eligibility decisions and actionable rejection reasons. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Load NVML from trusted locations and collect NVIDIA GPU capabilities. Avoid unsafe library resolution while providing selector-grade hardware data. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Download selected GGUF weights through validated Hugging Face redirects. Verify hashes and publish the model atomically after a complete transfer. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Install the pinned llama-server and CUDA runtime components. Validate the resulting native executable before marking acquisition complete. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Start llama-server on demand behind a managed local HTTP router. Coordinate health checks, request forwarding, idle lifetime, and cleanup. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Add Local AI preflight, selection, and acquisition to the setup pipeline. Keep pipeline ordering explicit and cover only its core integration contract. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Write the selected manifest and router preset during transactional setup. Start the managed runtime and remove persisted state during rollback. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Probe health, execute a real inference request, and verify GPU activity. Fail setup cleanly when the installed Local AI stack is not operational. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Configure the gateway to use the managed llama-server over IPv4 loopback. Use 127.0.0.1 consistently and retain focused configuration assertions. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Start an installed Local AI router with the companion application. Report startup state and stop the owned router during application shutdown. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Surface Local AI eligibility, consent, review, and progress in onboarding. Cache hardware probing and carry detailed acquisition state through setup. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Add Local AI navigation, status, controls, logs, and localized resources. Wire the page through application services and retain focused UI contracts. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
|
🦞👀 Pull request received. I will update this pull request when review starts. |
clawsweeper
Bot
added
merge-risk: 🚨 compatibility
|
Codex review: needs real behavior proof before merge. Reviewed August 19, 2026, 3:23 PM ET / 19:23 UTC. ClawSweeper reviewWhat this changesThe PR adds managed Windows Local AI setup, including pinned llama-server and model downloads, hardware qualification, a loopback gateway provider, and tray controls. Merge readiness⛔ Blocked until stronger real behavior proof is added - 8 items remain Current main does not contain this feature. The branch stays open, but its bundled setup default enables the optional Local AI flow and can stop ordinary setup on unsupported hardware before normal provisioning. Priority: P2 Review scores
Verification
How this fits togetherThe setup engine creates OpenClaw’s app-owned WSL gateway and its provider configuration, while the tray owns companion-runtime lifetime and user controls. This PR inserts a Windows-native llama-server into that path and makes the WSL gateway consume it over IPv4 loopback. flowchart LR
A[Onboarding selection] --> B[Setup pipeline]
B --> C[Hardware and WSL checks]
C --> D[Verified runtime and model]
D --> E[Local llama-server]
E --> F[Gateway provider]
F --> G[Tray and chat]
Before merge
Findings
Agent review detailsSecurityNone. Review metrics
Merge-risk optionsMaintainer options:
Technical reviewBest possible solution: Keep Local AI disabled in the bundled configuration and make the onboarding toggle the sole opt-in; add upgrade and fresh-install coverage, then publish redacted current-head acquisition, inference, WSL-access, and rollback evidence. Do we have a high-confidence way to reproduce the issue? Yes, from source: load the bundled configuration and run setup on a machine outside the three qualified NVIDIA profiles. Local AI is enabled, so the preflight returns a terminal result before ordinary setup can continue. Is this the best way to solve the issue? No. The implementation’s own opt-in claim conflicts with the bundled enabled default; preserving the existing setup path until the user actively enables Local AI is the narrower upgrade-safe solution. Full review comments:
Overall correctness: patch is incorrect AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against b37307cbd51f. LabelsLabel justifications:
EvidenceAcceptance criteria:
What I checked:
Likely related people:
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
HistoryReview history (1 earlier review cycle)
|
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
jacobtomlinson
added
status: 🚢 actively landing
Collaborator
|
This is an impressive, thoughtfully structured feature, especially the immutable artifact pins, SHA-256 checks, path containment, redirect allowlist, process ownership, and rollback compare-and-swap behavior. I validated the exact head on Windows: the full build and Shared, Tray, and SetupEngine unit suites pass. I also verified the llama.cpp and Hugging Face revisions, filenames, sizes, and hashes against upstream. It is not merge-ready yet. Two independent reviews agree on these blockers:
The proof gap is also material: this adds 9,089 changed production lines and 127 runtime types with zero new focused Before merge, please fix the five blockers, add focused regression coverage, and provide redacted current-head proof of a verified download, first inference, WSL reachability, restart, rollback, and interrupted-install recovery. The direction is excellent. The remaining work is about making this large native execution surface supportable and safe to ship. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Problem This Solves
Windows users who want private local inference do not currently have a supported way to qualify their hardware, install a compatible runtime and model, or manage that runtime from the OpenClaw setup and tray experience. This leaves users to assemble an unverified llama.cpp stack manually and gives the companion application no durable installation or lifecycle state.
Why This Change Was Made
This adds an end-to-end managed Local AI path using pinned llama.cpp, CUDA, and GGUF artifacts, deterministic NVIDIA hardware qualification, transactional setup, an on-demand loopback router, and setup/tray integration. The scope is intentionally limited to production code and focused core assertions; broader documentation and media evidence are deferred.
User Impact
Eligible Windows users can select Local AI during onboarding, install a verified model/runtime combination, monitor acquisition progress, and view or control the resulting local service from the tray. Existing users remain unchanged because Local AI is disabled by default and unsupported hardware is rejected with an explicit reason.
Evidence
NotInstalled, opened the tray-hosted setup window, started WinUI 3, and emitted noERRORorFATALlog entries.Change Type
Scope
winnodeValidation
powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\build.ps1win-arm64).dotnet test .\tests\OpenClaw.Shared.Tests\OpenClaw.Shared.Tests.csproj --no-restoredotnet test .\tests\OpenClaw.Tray.Tests\OpenClaw.Tray.Tests.csproj --no-restoredotnet test .\tests\OpenClaw.SetupEngine.Tests\OpenClaw.SetupEngine.Tests.csproj --no-restoredotnet test .\tests\OpenClaw.Tray.UITests\OpenClaw.Tray.UITests.csproj -r win-arm64 --no-restore --filter "FullyQualifiedName~SmokeTests"OPENCLAW_RUN_INTEGRATION=1 dotnet test .\tests\OpenClaw.Tray.IntegrationTests\OpenClaw.Tray.IntegrationTests.csproj -r win-arm64 --no-restore --filter <four MCP smoke cases>git diff --check origin/main...HEADReal Behavior Proof
9eea5322080d8767173a1960deb9bd0871b3922c.win-arm64tray executable with fresh isolatedOPENCLAW_TRAY_DATA_DIR,OPENCLAW_TRAY_APPDATA_DIR, andOPENCLAW_TRAY_LOCALAPPDATA_DIRvalues plusOPENCLAW_SUPPRESS_EXTERNAL_BROWSER=1; observed it for five seconds; inspected its current-run log; then stopped the exact process and removed the temporary directory. Also ran the WinUI and authenticated MCP smoke commands listed under Validation.alive_after_5s=True,service_initialized=True,local_ai_not_installed=True,setup_window_opened=True,application_started=True,error_lines=0, andstopped=True.Yes/No/N/A): No.Security Impact
Yes/No): Yes.Yes/No): No.Yes/No): Yes.Yes/No): Yes.Yes/No): Yes.Yes, explain the risk and mitigation: setup can download and run a native llama-server stack, write installation state under the OpenClaw local-data root, and update mirrored WSL networking only after explicit consent. Risks are bounded through pinned versions and revisions, SHA-256 verification, trusted NVML loading, atomic install/rollback, managed process ownership, loopback-only serving, deterministic hardware eligibility, and restoration of setup-owned configuration during rollback.Compatibility and Migration
Yes/No): Yes.Yes/No): Yes.Yes/No): No.LocalAiconfiguration defaults to disabled, so existing installations retain their current gateway and provider behavior until the user explicitly opts in through setup.Review Conversations
No review conversations existed when this PR was opened.