Skip to content

test(onboard): split helper coverage#3640

Open
cv wants to merge 24 commits into
refactor/delete-unused-codefrom
test/split-onboard-tests
Open

test(onboard): split helper coverage#3640
cv wants to merge 24 commits into
refactor/delete-unused-codefrom
test/split-onboard-tests

Conversation

@cv
Copy link
Copy Markdown
Collaborator

@cv cv commented May 16, 2026

Summary

Splits helper-oriented coverage out of the oversized test/onboard.test.ts file into focused unit test files. The PR keeps source-shape coverage at zero, preserves equivalent behavior coverage, and makes the remaining onboard orchestration tests easier to navigate.

Related Issue

Stacked on #3632 / #3636.

Changes

  • Moved GPU helper coverage into focused module tests:
    • src/lib/onboard/gateway-gpu-passthrough.test.ts
    • src/lib/onboard/sandbox-gpu-create.test.ts
    • src/lib/onboard/sandbox-gpu-mode.test.ts
    • src/lib/onboard/initial-policy.test.ts
  • Moved policy suggestion helper coverage into test/onboard-policy-suggestions.test.ts.
  • Moved dashboard helper coverage into test/onboard-dashboard.test.ts and dashboard port helper coverage into src/lib/onboard/dashboard-port.test.ts.
  • Moved sandbox naming helper coverage into test/onboard-sandbox-name.test.ts.
  • Moved OpenShell version/pinning helper coverage into test/onboard-openshell-version.test.ts.
  • Moved Dockerfile patch helper coverage into src/lib/onboard/dockerfile-patch.test.ts.
  • Moved gateway runtime / Docker-driver / CDI helper coverage into test/onboard-gateway-runtime.test.ts.
  • Moved Model Router orchestration coverage into test/onboard-model-router.test.ts.
  • Moved messaging setup/provider coverage into test/onboard-messaging.test.ts.
  • Moved custom Dockerfile orchestration coverage into test/onboard-custom-dockerfile.test.ts.
  • Moved provider routing/probe/provider-CRUD helper coverage into owning module tests:
    • src/lib/inference/config.test.ts
    • src/lib/inference/onboard-probes.test.ts
    • src/lib/onboard/providers.test.ts
    • src/lib/validation.test.ts
    • src/lib/core/url-utils.test.ts
    • src/lib/adapters/http/probe.test.ts
  • Moved web-search support coverage into src/lib/onboard/web-search-support.test.ts / test/onboard-brave-validation.test.ts.
  • Moved build-context and sandbox-create recovery hint coverage into src/lib/build-context.test.ts.
  • Extracted focused production helpers with direct module coverage:
    • src/lib/onboard/credential-env.ts
    • src/lib/onboard/summary.ts
    • additional exports in src/lib/onboard/sandbox-gpu-mode.ts and src/lib/onboard/dashboard-port.ts
  • Removed the corresponding helper-only cases from test/onboard.test.ts; it is now much smaller while keeping the remaining orchestration/integration-style coverage.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Verification

  • npm run source-shape:check passes
  • npm run build:cli passes
  • npx tsc -p tsconfig.cli.json --noUnusedLocals --noUnusedParameters --pretty false passes
  • npx prek run --all-files passes
  • npm test passes
  • Tests added or updated for new or changed behavior
  • No secrets, API keys, or credentials committed
  • Docs updated for user-facing behavior changes
  • make docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Carlos Villela cvillela@nvidia.com

Summary by CodeRabbit

Release Notes

  • Tests

    • Added comprehensive test coverage for GPU configuration, sandbox naming, credential management, dashboard port handling, custom Dockerfile support, messaging channels, Model Router setup, and OpenShell version management.
    • Added tests for probe error handling, policy validation, and sandbox state detection.
  • Refactor

    • Reorganized onboarding helpers into dedicated modules for improved maintainability.

Review Change Stack

cv added 4 commits May 15, 2026 17:08
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
@cv cv self-assigned this May 16, 2026
@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented May 16, 2026

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 16, 2026

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f36bfc29-184a-4a1e-9dd9-b7f55350ebc3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title 'test(onboard): split helper coverage' accurately summarizes the main change—splitting onboard helper tests from a monolithic file into focused test files for improved organization.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test/split-onboard-tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 16, 2026

E2E Advisor Recommendation

Required E2E: cloud-onboard-e2e, inference-routing-e2e, gpu-e2e, onboard-resume-e2e
Optional E2E: sandbox-operations-e2e, messaging-compatible-endpoint-e2e, gpu-double-onboard-e2e, runtime-overrides-e2e

Dispatch hint: cloud-onboard-e2e,inference-routing-e2e,gpu-e2e,onboard-resume-e2e

Auto-dispatched E2E: cloud-onboard-e2e, inference-routing-e2e, gpu-e2e, onboard-resume-e2e via nightly-e2e.yaml at 4c46670f842d394ebe3353b97a77885b731ad9a1nightly run

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/refactor/delete-unused-code
Head: HEAD
Confidence: high

Required E2E

  • cloud-onboard-e2e (high (~45 min, live NVIDIA endpoint, creates sandbox)): Exercises the primary install/onboard/sandbox-create path with real OpenShell gateway registration, credential staging, policy setup, Dockerfile patching, inference.local HTTPS, and sandbox health. This is the broadest required guard for the onboard.ts refactor and new credential/dashboard/summary modules.
  • inference-routing-e2e (medium-high (~30 min, live NVIDIA endpoint)): Required because the changed tests and onboard path are adjacent to provider selection, probe classification, credential isolation, and inference.local routing. This job validates routed inference behavior and error classification in a live sandbox context.
  • gpu-e2e (high (~30 min, GPU runner)): Required for runtime changes to sandbox GPU mode/config extraction and resume metadata handling. This job validates the real Ollama local-inference GPU user flow, Docker CDI/GPU proof, sandbox GPU status, and sandbox-to-Ollama inference path.
  • onboard-resume-e2e (medium-high): Required because onboarding resume/reuse behavior imports the extracted GPU override/config helpers and credential hydration still needs to work across interrupted/resumed onboarding. This catches regressions not covered by a single fresh onboard.

Optional E2E

  • sandbox-operations-e2e (high (~60 min)): Useful adjacent coverage for sandbox lifecycle, status, destroy, gateway recovery, and forwarding-related operations after changes to dashboard port ownership/allocation. Not strictly merge-blocking if cloud-onboard and onboard-resume pass.
  • messaging-compatible-endpoint-e2e (medium-high (~30 min, hermetic mock endpoint plus sandbox)): Optional confidence for credential-provider placeholder isolation plus OpenAI-compatible endpoint routing, since credential hydration and Dockerfile config patch behavior are adjacent to messaging onboard flows.
  • gpu-double-onboard-e2e (high (~30 min, GPU runner)): Optional deeper GPU lifecycle coverage for re-onboard/token consistency. It is valuable after GPU helper extraction but is mostly redundant with required gpu-e2e plus onboard-resume-e2e for this PR.
  • runtime-overrides-e2e (medium): Optional Dockerfile/config patch confidence for runtime override behavior, useful because Dockerfile patch tests expanded substantially, but no Dockerfile patch source file is in the changed file list.

New E2E recommendations

  • dashboard port conflict handling (medium): Existing E2E coverage exercises normal dashboard forwarding but does not appear to create a competing non-OpenShell listener or stale/foreign OpenShell forward before onboarding. The changed dashboard-port runtime code specifically handles those cases.
    • Suggested test: Add an E2E that pre-binds the preferred dashboard port and verifies onboarding allocates the next available port, records the actual port, and serves the dashboard on that port.
  • sandbox GPU flag/resume matrix (medium): gpu-e2e validates default GPU enablement on a GPU runner, but the extracted sandbox-gpu-mode module includes env/flag/device conflict handling and resume overrides that are only unit-tested.
    • Suggested test: Add a GPU-runner E2E matrix for NEMOCLAW_SANDBOX_GPU=0/1/auto, --sandbox-gpu enable/disable, device override, and resume metadata preservation.

Dispatch hint

  • Workflow: nightly-e2e.yaml
  • jobs input: cloud-onboard-e2e,inference-routing-e2e,gpu-e2e,onboard-resume-e2e

@cv cv changed the base branch from test/source-shape-real-detector to refactor/delete-unused-code May 16, 2026 01:55
cv added 7 commits May 15, 2026 18:55
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
@cv cv changed the title test(onboard): split GPU helper coverage test(onboard): split helper coverage May 16, 2026
cv added 12 commits May 15, 2026 23:42
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
@cv cv marked this pull request as ready for review May 16, 2026 08:22
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/onboard/dashboard-port.ts`:
- Around line 147-154: The current owner lookup re-parses forwardListOutput with
a simple split and may match non-live or ANSI-colored rows; replace this ad-hoc
logic and reuse the same live-forward parsing and ANSI-stripping routine used
for allocation lookup so only live rows are considered. Specifically, instead of
splitting and inspecting lines in the portLine computation that references
forwardListOutput and portToStop, run the existing parser (the function/module
that strips ANSI and determines live status used elsewhere in this file) to
produce row objects, filter for rows where status is live, find the row whose
port equals portToStop, and then return that row's owner/id (or null) instead of
parsing parts[0] from a raw string. Ensure you reference forwardListOutput and
portToStop and preserve null when not found.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 69f135b3-8912-4a3d-a4e5-f6b98ed5d7ad

📥 Commits

Reviewing files that changed from the base of the PR and between e307e79 and 79167d3.

📒 Files selected for processing (27)
  • src/lib/adapters/http/probe.test.ts
  • src/lib/build-context.test.ts
  • src/lib/core/url-utils.test.ts
  • src/lib/inference/config.test.ts
  • src/lib/inference/onboard-probes.test.ts
  • src/lib/onboard.ts
  • src/lib/onboard/credential-env.test.ts
  • src/lib/onboard/credential-env.ts
  • src/lib/onboard/dashboard-port.test.ts
  • src/lib/onboard/dashboard-port.ts
  • src/lib/onboard/dockerfile-patch.test.ts
  • src/lib/onboard/providers.test.ts
  • src/lib/onboard/sandbox-gpu-mode.test.ts
  • src/lib/onboard/sandbox-gpu-mode.ts
  • src/lib/onboard/summary.test.ts
  • src/lib/onboard/summary.ts
  • src/lib/onboard/vm-dns-monkeypatch.test.ts
  • src/lib/onboard/web-search-support.test.ts
  • src/lib/validation.test.ts
  • test/gateway-state.test.ts
  • test/onboard-brave-validation.test.ts
  • test/onboard-custom-dockerfile.test.ts
  • test/onboard-gateway-runtime.test.ts
  • test/onboard-messaging.test.ts
  • test/onboard-model-router.test.ts
  • test/onboard-openshell-version.test.ts
  • test/onboard.test.ts
✅ Files skipped from review due to trivial changes (2)
  • src/lib/build-context.test.ts
  • src/lib/onboard/vm-dns-monkeypatch.test.ts

Comment thread src/lib/onboard/dashboard-port.ts Outdated
Comment on lines +147 to +154
const portLine = forwardListOutput
.split("\n")
.map((line) => line.trim())
.find((line) => {
const parts = line.split(/\s+/);
return parts[2] === portToStop;
});
return portLine ? (portLine.split(/\s+/)[0] ?? null) : null;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Use live-forward parsing for owner lookup to avoid stale matches.

Line 147–154 reparses rows manually and does not apply the same live-status + ANSI-stripping rules used for allocation. This can return an owner from non-live rows and target the wrong sandbox.

Proposed fix
 export function findDashboardForwardOwner(
   forwardListOutput: string | null | undefined,
   portToStop: string,
 ): string | null {
   if (!forwardListOutput) return null;
-  const portLine = forwardListOutput
-    .split("\n")
-    .map((line) => line.trim())
-    .find((line) => {
-      const parts = line.split(/\s+/);
-      return parts[2] === portToStop;
-    });
-  return portLine ? (portLine.split(/\s+/)[0] ?? null) : null;
+  return getOccupiedPorts(forwardListOutput).get(portToStop) ?? null;
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/onboard/dashboard-port.ts` around lines 147 - 154, The current owner
lookup re-parses forwardListOutput with a simple split and may match non-live or
ANSI-colored rows; replace this ad-hoc logic and reuse the same live-forward
parsing and ANSI-stripping routine used for allocation lookup so only live rows
are considered. Specifically, instead of splitting and inspecting lines in the
portLine computation that references forwardListOutput and portToStop, run the
existing parser (the function/module that strips ANSI and determines live status
used elsewhere in this file) to produce row objects, filter for rows where
status is live, find the row whose port equals portToStop, and then return that
row's owner/id (or null) instead of parsing parts[0] from a raw string. Ensure
you reference forwardListOutput and portToStop and preserve null when not found.

Signed-off-by: Carlos Villela <cvillela@nvidia.com>
@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 25957237788
Target ref: 79167d3b3f0ab5f45ab55e2d71e286540a58ef1d
Workflow ref: refactor/delete-unused-code
Requested jobs: cloud-onboard-e2e,inference-routing-e2e,credential-sanitization-e2e,onboard-resume-e2e,sandbox-operations-e2e,gpu-e2e
Summary: 4 passed, 0 failed, 1 skipped

Job Result
cloud-onboard-e2e ✅ success
credential-sanitization-e2e ✅ success
gpu-e2e ⏭️ skipped
inference-routing-e2e ✅ success
onboard-resume-e2e ✅ success
sandbox-operations-e2e ⚠️ cancelled

@github-actions
Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 25957499806
Target ref: 4c46670f842d394ebe3353b97a77885b731ad9a1
Workflow ref: refactor/delete-unused-code
Requested jobs: cloud-onboard-e2e,inference-routing-e2e,gpu-e2e,onboard-resume-e2e
Summary: 3 passed, 0 failed, 1 skipped

Job Result
cloud-onboard-e2e ✅ success
gpu-e2e ⏭️ skipped
inference-routing-e2e ✅ success
onboard-resume-e2e ✅ success

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant