Skip to content

perf(zeroshot-rust): benchmark provider adapters and execution runtime #771

Description

@tomdps

Parent

Execution

AFK. Start only after every issue in Blocked by is merged, required CI is green for the resulting dev SHA, and the worktree is created from a freshly fetched origin/dev.

Outcome

Extend the deterministic profiling harness with provider-adapter and execution-runtime workloads, recording semantic invariants separately from timing observations.

Authoritative contracts/current state

#683 owns the harness/report schema; concrete CLI/ACP/Gateway drivers and coordinator are separate production seams. Performance evidence is manual and must not silently change production behavior.

Owned scope

  • Add fake-provider workloads for CLI framing/process startup, ACP pool/session turns, Gateway HTTP/tool loops, runtime dispatch/inspection, and settlement.
  • Reuse perf(zeroshot-rust): add a deterministic manual scheduler/reducer profiling harness #683 report schema and semantic digest; timing/allocation/RSS observations remain host-specific and excluded from deterministic assertions.
  • Bound seeds, turns, payloads, pool sizes, queue saturation, and artifact references; perform no external provider calls.
  • Record measured bottlenecks as evidence only; any production optimization requires a newly scoped follow-up with before/after results.

Acceptance criteria

  • Equal seed/config produces equal semantic counts/order/digest across runs.
  • Each provider family and runtime recovery disposition has a non-zero workload and invariant assertion.
  • Profiles are manual artifacts under caller-selected output paths and are never committed.
  • No production scheduler/driver/runtime behavior changes in this issue.

Verification

cargo test -p zeroshot-rust --test provider_profile
cargo run -p zeroshot-rust --release --example provider_profile -- --seed 1 --output /tmp/zeroshot-rust-provider-profile.json
npm run rust:check
npm run protocol:check
opcore check --changed

Every named focused target is an issue-owned deliverable and must execute non-zero tests before the common gates.

Non-goals

  • Daemon/CLI control-path profiling.
  • Production optimization or release budgets.
  • Live providers, credentials, or benchmark uploads.

Blocked by

Unblocks

  • No release-critical issue; this is a terminal/manual evidence slice.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions