[Schema Consistency] Schema consistency check report - 2026-08-06

[github-actions[bot]]

Caution

agentic threat detected
Threat detection flagged this output in warn mode. Manual review is REQUIRED before any follow-up automation.

Details

Potential security threats were detected in the agent output.

Review the workflow run logs for details.

🔍 Schema Consistency Check - 2026-08-06

Summary

• Inconsistencies Found: 4
• Categories Analyzed: Schema, Parser, Documentation, Workflows
• Strategy Used: Precomputed gap triage + nested field verification
• New Strategy: YES

Critical Issues

1. Precomputed schema diff reports every top-level schema field as absent from the parser, but that result is false.

   • Evidence: /tmp/gh-aw/agent/schema-diff.json shows parser_yaml_fields: [] and therefore all schema fields under field_gaps.in_schema_not_parser.
   • Reality: frontmatter is parsed largely through manual map processing in pkg/workflow/* rather than yaml-tagged structs in pkg/parser/frontmatter.go.
   • Concrete examples implemented in code:
     • max-turn-cache-misses is parsed in pkg/workflow/engine.go and pkg/workflow/engine_config_parser.go
     • status-comment, skip-if-match, and skip-if-no-match are parsed in pkg/workflow/trigger_parser.go
     • ambient-folders is consumed in pkg/workflow/compiler_activation_outputs.go
   • Impact: any automation built from the precomputed diff will generate many false positives unless it understands manual parsing.

2. Glossary documentation says status-comment: is configured in safe-outputs, but implementation and primary docs place it under on:.

   • Incorrect doc: docs/src/content/docs/reference/glossary.md says: “Configured via status-comment: true in safe-outputs.”
   • Correct docs and implementation:
     • docs/src/content/docs/reference/frontmatter.md documents status-comment under on:
     • docs/src/content/docs/reference/triggers.md documents on.status-comment
     • pkg/workflow/trigger_parser.go parses status-comment from the on map
   • Impact: users reading the glossary can put the field in the wrong section and get confusing behavior.

3. Documentation claims gh aw fix migrates deprecated app-id to client-id, but the code shown here only normalizes aliases during parsing.

   • Docs making the migration claim:
     • docs/src/content/docs/reference/glossary.md
     • docs/src/content/docs/reference/checkout.md
     • docs/src/content/docs/reference/frontmatter.md
   • Implementation evidence found:
     • pkg/workflow/safe_outputs_app_config.go accepts both client-id and deprecated app-id
     • pkg/workflow/config_helpers.go normalizes github-app.app-id to client-id in memory while parsing
   • I did not verify any gh aw fix rewrite implementation in this pass.
   • Impact: docs currently make a stronger guarantee than the analyzed implementation proves; this should either be verified end-to-end or softened.

Documentation Gaps

1. ambient-folders is documented as a top-level frontmatter feature and import-safe shared component field, but real workflow usage in this repo appears only in shared workflow content.

   • Workflow evidence: .github/workflows/shared/squad.md
   • Compiler evidence: pkg/workflow/compiler_activation_outputs.go
   • This is not necessarily a bug, but it is a documentation/usage mismatch worth noting because it is easy to misread as a per-workflow mainstream feature when current repo usage is limited to shared imported content.

2. The machine-generated field_gaps.in_used_not_schema list contains many false positives caused by scanning non-frontmatter YAML-like content inside workflow prompts/examples.

   • Examples reported as “used but not in schema”: title, task, repo, noop, try, else
   • These are visibly present in markdown bodies and examples, not necessarily actual frontmatter keys.
   • Impact: schema-vs-workflows reporting built from naive key extraction will over-report invalid workflow usage.

Schema Improvements Needed

1. Analysis tooling should distinguish top-level schema fields from nested/manual parser surfaces.

   • Nested fields such as on.status-comment, on.restore-memory, on.skip-if-match, on.skip-if-no-match, and engine.extensions do exist in the schema and implementation, but they are invisible to the top-level yaml-tag diff strategy.

2. Schema-consistency workflows should exclude markdown-body pseudo-YAML when computing actual workflow key usage.

   • Otherwise field_gaps.in_used_not_schema will continue to produce low-signal findings.

Parser Updates Required

1. No parser bug was confirmed for the verified fields.
   • status-comment, skip-if-match, skip-if-no-match, restore-memory, engine.extensions, ambient-folders, and max-turn-cache-misses all have implementation evidence.
2. However, the repository’s consistency-check method should be updated to account for manual map parsing and alias normalization rather than relying on yaml tags alone.

Workflow Violations

1. No confirmed invalid top-level workflow field usage found in the sampled real workflows.
   • Example workflows like .github/workflows/ab-testing-advisor.md use schema-backed keys such as max-daily-ai-credits, safe-outputs, engine, features, sandbox, and evals.
2. One apparent mismatch is actually valid shared-component usage:
   • .github/workflows/shared/squad.md uses ambient-folders, which is documented and has compiler support.

Recommendations

1. Fix docs/src/content/docs/reference/glossary.md so status-comment is described under on: instead of safe-outputs.
2. Audit the codebase for an actual gh aw fix migration path for app-id → client-id; if none exists, remove or soften that promise in docs.
3. Update the schema-consistency checker to treat /tmp/gh-aw/agent/schema-diff.json as a hint, not ground truth, because parser support is often implemented through manual map[string]any processing.
4. Improve workflow key extraction to read only real frontmatter blocks, excluding YAML-like snippets in markdown bodies.
5. Add a dedicated nested-field consistency pass for on.*, engine.*, and github-app subfields, since top-level diffs miss the most important behavior.

Strategy Performance

• Strategy Used: Precomputed gap triage + nested field verification
• Findings: 4
• Effectiveness: HIGH
• Should Reuse: YES

Next Steps

• Fix glossary references for status-comment
• Verify or correct docs around gh aw fix and app-id migration
• Harden the consistency checker against false positives from manual parsing
• Harden workflow key extraction to ignore markdown-body examples
• Add nested-field validation coverage for on.* and engine.*

Generated by ✅ Schema Consistency Checker · gpt54 · 14.5 AIC · ⌖ 7.06 AIC · ⊞ 8.4K · ◷

• expires on Aug 6, 2026, 10:17 PM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-08-07T06:17:49.617Z.

Closed by Workflow