Skip to content

Improve upstream-sync: replace marketplace.json heuristic with manifest, fix workflow issue churn #23

@circuitfive

Description

@circuitfive

Background

Two pain points surfaced during the sync that closed #20#22:

  1. Script false positives. merge_marketplace.py flags any entry "in our marketplace.json but not in upstream's" as fork-specific. When upstream removes an entry, that entry sits in our pre-merge file and gets re-flagged as ours every sync. This run, 12 of 13 flagged entries were upstream-removed (ai-firstify, elixir-ls-lsp, firetiger, followrabbit, goodmem, helius, migration-to-aws, opsera-devsecops, product-tracking-skills, searchfit-seo, sonarqube-agent-plugins, voila-api). Each had to be passed via --exclude after manual git log verification. Only toolbelt is genuinely ours.

  2. Workflow issue churn. .github/workflows/upstream-sync-check.yml opens a fresh `Upstream Changes Detected` issue every Monday regardless of whether one is already open, and never closes them after a sync. Three issues piled up (Upstream Changes Detected #20, Upstream Changes Detected #21, Upstream Changes Detected #22) all describing the same drift, and had to be closed manually after sync.

Proposed changes

1. Manifest-driven marketplace.json

Add `.claude-plugin/fork-additions.json` listing arizonabay's marketplace entries. `marketplace.json` remains the distributed artifact (consumers still install `toolbelt` from it as today) — it just becomes a generated file rebuilt from `upstream/main:marketplace.json` + `fork-additions.json`, sorted.

Rewrite `merge_marketplace.py` accordingly:

  • Read upstream's marketplace.json from `upstream/main` ref
  • Read `fork-additions.json` from working tree
  • Concatenate, sort by name (case-insensitive), validate no duplicates
  • Write `marketplace.json` and `git add` it

Drop `--exclude` and the diff-against-fork heuristic entirely. The script becomes ~30 lines, fully deterministic, no false positives possible.

Migrate the existing `toolbelt` entry from `marketplace.json` into `fork-additions.json` as part of this change.

Adding a future fork plugin becomes: edit `fork-additions.json` → run script → commit both files.

2. Workflow upsert + auto-close

Modify `.github/workflows/upstream-sync-check.yml`:

  • Search for an existing open issue titled `Upstream Changes Detected` (use `gh issue list` or `octokit.issues.listForRepo` with state=open)
  • If drift exists + open issue exists → update the issue body
  • If drift exists + no open issue → create a new one (today's behavior)
  • If no drift + open issue exists → close it with a brief comment (e.g., "Synced — drift cleared.")
  • Add `push: branches: [main]` to the `on:` triggers so post-sync auto-close fires immediately, not delayed up to a week

Keep the existing failure-path issue creation untouched.

3. Documentation

  • `.claude/skills/upstream-sync/SKILL.md` — replace the "Resolve marketplace.json conflicts" section to reflect the new flow (no conflicts to resolve; just run the regenerator). Remove references to `--exclude`. Note that adding fork plugins = edit `fork-additions.json` + regenerate.
  • `CLAUDE.md` — add a short subsection (under "Repository Overview" or "Development") titled "Adding a fork-specific plugin" pointing at `fork-additions.json` and the regenerator script.

Out of scope

  • CI check that validates `marketplace.json` matches the regenerated output (nice-to-have; not needed today)
  • Changes to issue title/format
  • Replacing issues with a status check
  • Touching the failure-issue path
  • README.md changes (the README is consumer-facing and doesn't document fork mechanics)

Acceptance criteria

  • `.claude-plugin/fork-additions.json` exists and contains `toolbelt`
  • `merge_marketplace.py` rewritten; no `--exclude` flag; deterministic regeneration from upstream + manifest
  • Running the script on a fresh sync produces a clean `marketplace.json` with no manual intervention
  • Workflow upserts a single open drift issue and auto-closes on sync
  • `push: branches: [main]` trigger added
  • SKILL.md and CLAUDE.md updated

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions