Skip to content

Add expect(transcript).toContainText() for deterministic transcript assertions#172

Merged
gaojude merged 1 commit into
mainfrom
jude/transcript-to-contain-text
Jul 17, 2026
Merged

Add expect(transcript).toContainText() for deterministic transcript assertions#172
gaojude merged 1 commit into
mainfrom
jude/transcript-to-contain-text

Conversation

@gaojude

@gaojude gaojude commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Asserting the agent never said or reached for something currently needs either a judge run (wasteful, and .not.toSatisfyCriterion would invert the judge's fail-closed default into fail-open) or a manual readFileSync(transcriptPath()). This adds a deterministic, judge-free matcher on the transcript sentinel:

expect(transcript).not.toContainText('getServerSideProps');

Misuse (wrong subject, empty needle) and a missing or empty transcript throw instead of returning pass: false, because a returned failure inverts into a silent pass under .not, and an uncaptured transcript is an infra failure rather than evidence of absence. A .not violation cites where the needle appears: [transcript] expected NOT to contain "getServerSideProps", but found it at char 34: …context…. The transcript is the agent's native format (claude-code writes raw session JSONL), so quotes and newlines appear JSON-escaped there; the README says to stick to identifier-like needles.

@gaojude
gaojude marked this pull request as ready for review July 17, 2026 00:48
@gaojude
gaojude merged commit c2c458e into main Jul 17, 2026
4 checks passed
@gaojude
gaojude deleted the jude/transcript-to-contain-text branch July 17, 2026 00:48
@github-actions github-actions Bot mentioned this pull request Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant