Skip to content

Fixture-refresh Action can leak environment-specific data (AEM tools, /Users paths) into latest.ndjson #124

Description

@spopescu

Problem

src/test/resources/cli-fixtures/latest.ndjson is the drift-tripwire fixture, intended to be a generic recorded Claude Code stream-json turn. It's refreshed periodically (the weekly fixture-refresh Action, see #119) by running the CLI and capturing its output.

If the capture runs in an environment with project-specific configuration, that config bleeds into the recorded system/init event and SessionStart hook output — e.g. third-party MCP tool names, custom skill slash-commands, agent/skill lists, and absolute /Users/... memory_paths. This is how AEM tool names, AEM skill commands, and personal home paths ended up committed to this public repo (scrubbed in #123; the same leak also affected the sub-agent and goal fixtures, scrubbed in #121/#122 follow-ups).

It's a recurrence risk: the next scheduled refresh will re-pollute the fixture unless the capture environment is controlled.

Suggested fix

Make the refresh capture environment-neutral, e.g. one or more of:

  • Run the capture with an isolated, empty config home (CLAUDE_CONFIG_DIR/HOME pointing at a temp dir) so no user/project SessionStart hooks, MCP servers, skills, or memory paths are present.
  • After capture, run a sanitizer that whitelists system/init array fields to generic values and strips absolute home paths — the same transformation applied manually in Scrub AEM/personal data from latest.ndjson drift fixture #123.
  • Add a CI guard (a tiny test or grep step) that fails if any cli-fixtures/*.ndjson contains host/org-specific tokens (/Users/, /home/, third-party MCP/skill identifiers), so a polluted fixture can't be committed.

Acceptance

  • Re-running the refresh on a clean checkout produces a latest.ndjson with no host/org-specific data.
  • A guard prevents future polluted fixtures from landing.

Context: PRs #121, #122, #123 (manual scrubs). Related: #119 (the refresh Action).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions