Summary
Create a test harness for validating Pi coding agent behavior — ensuring it responds correctly to prompts, handles tool calls, and integrates with the dev session workflow.
Proposed approach
- A
tests/ directory with scripted test scenarios for Pi
- Harness should verify: agent startup, prompt/response cycles, tool invocations, error handling
- Support headless execution (no tmux required) for CI compatibility
- Output structured results (pass/fail per test, timing, token usage)
Example scenarios
- Pi launches and responds to a basic prompt
- Pi correctly uses Claude as a provider
- Pi handles missing API keys gracefully
- Pi integrates with MCP servers when configured
Acceptance criteria
just test-pi recipe runs the harness
- Tests pass in both interactive and headless modes
- Results are human-readable and CI-parseable
Summary
Create a test harness for validating Pi coding agent behavior — ensuring it responds correctly to prompts, handles tool calls, and integrates with the dev session workflow.
Proposed approach
tests/directory with scripted test scenarios for PiExample scenarios
Acceptance criteria
just test-pirecipe runs the harness