Skip to content

Build test harness for Pi coding agent #8

Description

@bBlazewavE

Summary

Create a test harness for validating Pi coding agent behavior — ensuring it responds correctly to prompts, handles tool calls, and integrates with the dev session workflow.

Proposed approach

  • A tests/ directory with scripted test scenarios for Pi
  • Harness should verify: agent startup, prompt/response cycles, tool invocations, error handling
  • Support headless execution (no tmux required) for CI compatibility
  • Output structured results (pass/fail per test, timing, token usage)

Example scenarios

  • Pi launches and responds to a basic prompt
  • Pi correctly uses Claude as a provider
  • Pi handles missing API keys gracefully
  • Pi integrates with MCP servers when configured

Acceptance criteria

  • just test-pi recipe runs the harness
  • Tests pass in both interactive and headless modes
  • Results are human-readable and CI-parseable

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions