Skip to content

[GA-M2][testing] Add offline route replay fixtures for cost and quality guardrails #43

Description

@Xunzhuo

Updated from the GA issue triage. This issue is intended to be directly assignable.

Goal

Test route replay behavior when cheaper routing conflicts with verifier or quality guardrails.

Deliverables

  • Fixture where candidate policy reduces cost without quality regression.
  • Fixture where candidate policy is rejected due to verifier weakness or failure.
  • Fixture where self-hosted preference applies only to safe request classes.

Acceptance Criteria

  • Replay tests verify both accepted and rejected policy changes.
  • Quality guardrails can override cost savings.
  • Reports show why a candidate policy was rejected.

Dependencies

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:tokenmaxxing-srTokenmaxxing telemetry, context/cache evidence, SR policy, and routing metadatakind:testingAutomated checks, release gates, regression tests, and validationpriority:p0-blockerBlocks the first credible GA evidence slice or core GA claim

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions