Skip to content

Latest commit

 

History

History
205 lines (165 loc) · 9.91 KB

testing.md

File metadata and controls

205 lines (165 loc) · 9.91 KB

Manual testing: drive the daemon live

← README · see also usage.md, protocol.md

The unit and integration suites (cargo test) cover logic and drive the daemon through a scripted approval. This doc is the complement: how to exercise a new feature end-to-end through the real daemon and the real human gate — the approval prompt appears in your terminal and you approve it yourself — before merging. It is the fastest way to see a change behave exactly as an operator (or the MCP execute tool) will experience it.

The key enabler is sudo-request --plan, which assembles a structured Plan request the same way the MCP tool does (mcp::build_request), so you can drive the full plan surface (exec, run (&&), seq (;), alt (||), pipe, redirect, run-as, setenv, chdir, shell) from the command line — no MCP client and no hand-written socket JSON required.

Setup

Build the branch under test, then run the daemon in one terminal and send requests from another.

Terminal A — the daemon + approval TUI. This is where the prompt appears and where you press the keys. Use a dedicated test socket so you never collide with an installed daemon's default socket:

cargo build --release
./target/release/sudo-proxy -v --socket /tmp/sp-test.sock

Terminal B — send requests with the freshly built client:

# A single command / pipeline (flat form)
./target/release/sudo-request --socket /tmp/sp-test.sock --no-privilege --print \
    echo hi '|' wc -c

# A structured plan — the same assembly the MCP execute tool performs
./target/release/sudo-request --socket /tmp/sp-test.sock --no-privilege --print \
    --reason "try alt" \
    --plan '{"op":"alt","steps":[{"op":"exec","argv":["false"]},{"op":"exec","argv":["echo","fallback-ran"]}]}'

When the request arrives, Terminal A shows the per-stage prompt — each stage with its own resolved real binary and run-as tag — and waits for a key. Press y to approve; the client in Terminal B then prints the result.

For a large plan, keep it in a file and use --plan-file (or --plan-file - to read from stdin):

./target/release/sudo-request --socket /tmp/sp-test.sock --no-privilege --print \
    --plan-file ./scratch/plan.json

Reading the result

With --print, the client shows the structured response:

exit [1, 0]
== stdout == (1 lines)
fallback-ran
  • The exit [...] array is one entry per stage, in prompt order.
  • A stage that was not run (short-circuited) reports the -1 sentinel. For the alt above with a succeeding first step you would instead see exit [0, -1] — the first alternative succeeded, so the second never ran.
  • stdout is the aggregate of every captured stage.

Tips

  • Decline with n or Enter (default is deny) → the client prints Request denied. and exits non-zero. Good for testing the deny path.
  • Mimic the MCP session label with --session sudo-proxy-mcp if a feature keys off the session name.
  • Exercise privileged stages by dropping --no-privilege (you will satisfy a real sudo/pkexec prompt) or by using a run-as node inside the plan.
  • Fidelity caveat. sudo-request reproduces the MCP's request assembly, not its policy layer: the MCP additionally refuses unprivileged local commands (invariant G7 — "use the Bash tool"). That refusal lives in the MCP execute handler, not in the daemon, which is exactly what lets you drive unprivileged local plans here for testing.

Checking the approval prompt

The plan files under tests/manual/ exist to check what a human sees at the prompt: the copy-pasteable Command: line, the deduplicated Resolves: block, the numbered Steps: outline, and the escalation colours. You check these by eye, so this is a manual step. tests/manual_corpus.rs guarantees that every file still parses and renders, and that its Steps: text and Command: line group the same way under bash.

For most of these the prompt is the result, so decline them (n). Approve only the ones you want to see run.

UI checks (tests/manual/ui/)

Send each one and compare the prompt with the expected column:

File Flags Expect
showcase.json --no-privilege Command: reads (export LANG=C && (echo 'a;b' && (echo 'hello world' | wc -w) && sudo id -un)), which pastes into a shell as is. Steps: shows the same groups, one command per line (see How the prompt shows grouping); sudo id -un is bright red.
escalation-a-ambient.json (privileged) Every step light red, && tinted too. Command: has sudo on each leaf, never one sudo sh -c.
escalation-b-boundary.json --no-privilege Only whoami is bright red; echo hi / echo bye uncoloured.
escalation-c-ambient-plus-boundary.json (privileged) Everything light red, whoami bright red, operators tinted.
escalation-d-named-user.json --no-privilege id -un bright yellow (run-as a named non-root user).

Run once more with NO_COLOR=1 set in the daemon's environment: the structure must stay readable with no escape codes, and every escalated step still shows its sudo / sudo -u NAME prefix.

How the prompt shows grouping

The Steps: block and the Command: line are printed from one layout, so they always have the same parentheses. With its line numbers removed, the Steps: block is valid shell that bash parses to the same tree as the Command: line. The layout rules:

  1. One line number per command; each pipe stage counts. A line holding only ( or ) is not numbered.
  2. Every operator is written out, with a space before it: ;, &&, ||, |.
  3. ( goes at the end of the line leading into the group, or on a line of its own when nothing precedes it.
  4. ) goes at the end of the group's last line when an operator or a redirection follows the group; otherwise on its own line, under the line holding its (.
  5. Indentation is the parenthesis depth, nothing else.
  6. A cd / export scope is ( cd DIR && ( body ) ). The parentheses keep it from leaking, including when the whole plan is the scope: the Command: line may be pasted into a stateful shell. The && skips the body when cd fails, as the executor does, instead of running it in the previous directory.
  7. A pipe is always parenthesized.
  8. The Command: line has exactly the Steps: parentheses. Adjacent openers are written ( (, never ((, which bash reads as arithmetic.
  9. Every line of a group ends with the same operator. A nested chain of the same operator is flattened into its parent; one of another operator gets its own group, so an operator never reaches further than its group.
  10. run-as shows as a sudo / sudo -u NAME prefix on each command it covers, in the Steps: block as well as on the Command: line.

For example, a && (b || c):

0  false && (
1    true ||
2    echo RAN
   )

These properties are checked in src/tui.rs: golden layouts, a bash parse comparison of the two over random plans, and a differential run (executor vs sh running each text) over random side-effect-free plans.

Replaying a real approval log

Toy plans miss what real agent traffic looks like. tests/manual/log-corpus/ is the regression set for that: 24 requests from a real session's approval log, which all arrived as opaque bash -lc '…' blobs. They're redacted and translated into 23 structured plans (one was an exact duplicate). Its README shows each original next to its plan, the exact sudo-request line to send, and the translation conventions (;→seq, ||→alt, sudo -u→run-as user, what stays an opaque shell and why).

For each case, check that the prompt makes the original's intent obvious at a glance: which stages run as root and which as a user, which redirects write where, and that anything still opaque is flagged as such. Decline every one. They refer to files and worktrees that don't exist here, and four touch a sysctl as root.

To extend the corpus from a new session, run the daemon with --log-json so each request's plan is logged as JSON, or start from the daemon's approval log as this one did. Redact it (see the corpus README), add the plan as the next NN-*.json file and its original to the README, and run cargo test --test manual_corpus.

Without a human (CI / headless)

The human gate reads /dev/tty, so full execution needs either a person or a pseudo-terminal harness. Three headless-friendly paths:

  • Pre-prompt rejection needs no keypress. A structurally invalid plan (e.g. a redirect wrapping a seq/alt) is rejected before prompting, so a client call returns an invalid plan: … error with no interaction — handy for smoke-testing the client→daemon plumbing against a backgrounded daemon.
  • Scripted approval for real execution: the integration tests drive the daemon through a scripted prompter — see tests/plan_exec.rs (plan_request_end_to_end_through_daemon) and tests/validation.rs. This is the place for request semantics: validation, plan compilation, execution.
  • The live prompt through a pty. tests/pty_e2e.rs runs the real sudo-proxy binary with a pseudo-terminal as its controlling terminal (tests/common/pty.rs), reads the prompt it draws and types the answer. It covers what the scripted prompter cannot: the rendered prompt, single-key reads, termios restoration and the result display. Privileged steps reach tests/shims/sudo, a stand-in that logs its argv and runs the command unescalated, so the literal argv handed to sudo is checked end to end. Send keys with PtyDaemon::answer, which waits for the tty to enter raw mode first; the prompt discards input typed before that.

What stays manual: how colours, the bell and the blink actually look in a terminal, real sudo passwords, the pkexec dialog, and the remote SSH tunnel.