Instrument workflows with Kosli attestations (org: kosli-public)

[dangrondahl]

Description

None of our three workflows report anything to Kosli today. We publish @kosli/mcp-server to npm and attach a .mcpb bundle to GitHub Releases, with no binary provenance and no compliance record behind either. Dogfooding matters here too: this is a Kosli-authored repo that's about to become public.

Target organization: kosli-public.

Proposed flows

Naming convention: <repo>-<workflow name> — one flow per pipeline, so the flow name identifies the workflow file that produced the evidence.

Flow	Workflow	Purpose	Trail
mcp-server-ci	ci.yml	PRs and pushes to main	git commit SHA
mcp-server-release	release.yml	Tagged releases	tag (e.g. v0.4.0)
mcp-server-update-catalog	update-catalog.yml	Optional, only if we instrument it	run date or commit SHA

Per-workflow analysis

ci.yml — pull requests and pushes to main

Currently: checkout, setup node, npm ci, npm run build, npm test, npm run pack:mcpb as a smoke test. No Kosli usage.

• kosli begin trail at job start, trail = ${{ github.sha }}.
• kosli attest junit for the vitest suite. Vitest needs to emit JUnit XML first — verified working with vitest --reporter=junit --outputFile=test-results/junit.xml (65 tests). Then attest with --results-dir test-results.
• kosli attest pullrequest github on pushes to main, for code-review evidence. --github-org is defaulted in GitHub Actions; --github-token needs supplying.

Fork PRs need guarding. Once this repo is public, PRs from forks run without access to repository secrets, so KOSLI_API_TOKEN will be empty and any unconditional attestation step will fail every external contributor's PR. Guard the Kosli steps, e.g.:

if: github.event_name == 'push' ||
    github.event.pull_request.head.repo.full_name == github.repository

release.yml — tags matching v*

This is where the shippable artifacts are, and where attestation is most valuable.

• kosli begin trail, trail = the tag.
• kosli attest artifact for the .mcpb bundle: --artifact-type file.
• kosli attest artifact for the npm tarball. npm publish doesn't leave a tarball behind, so add an npm pack step to produce the .tgz to fingerprint.
• kosli attest junit for the release-time test run.

Note issue #4 (re-enable npm --provenance once public) is complementary, not redundant: sigstore provenance proves who built the npm package, Kosli records what the artifact is and what evidence backs it.

update-catalog.yml — scheduled catalog regeneration

Lowest value, and I'd treat it as optional. It produces no shippable artifact — it opens a PR. There is a real provenance story if we want it: src/catalog.json is generated from an external input (app.kosli.com/api/v2/openapi.json) and shipped to users, so a kosli attest generic recording the regeneration, with the source spec's fingerprint, would document where the committed catalog came from. Under the <repo>-<workflow> convention this also means standing up a third flow, so it is best done last, if at all.

Environments

Not applicable. This repo ships a library/CLI to npm and a bundle to Releases; there is no runtime deployment to snapshot, so no Kosli Environment is proposed.

Prerequisites

• Service account + API key in kosli-public; store as KOSLI_API_TOKEN secret
• Set KOSLI_ORG: kosli-public at workflow or job level
• Create the flows (kosli create flow, with a template covering the attestations above)
• Pin kosli-dev/setup-cli-action by full commit SHA with a version comment, matching how every other action in this repo is pinned

Tasks

• Instrument ci.yml (begin trail, junit, pullrequest) with fork-PR guards
• Instrument release.yml (begin trail, artifact x2, junit)
• Decide on update-catalog.yml (optional generic attestation)

References

• CI/CD integration: https://docs.kosli.com/integrations/ci_cd
• kosli attest artifact: https://docs.kosli.com/client_reference/kosli_attest_artifact
• kosli attest junit: https://docs.kosli.com/client_reference/kosli_attest_junit
• kosli attest pullrequest github: https://docs.kosli.com/client_reference/kosli_attest_pullrequest_github
• Flow/trail naming: https://docs.kosli.com/implementation_guide/phase_2/plan_organizational_structure/naming_conventions/flows_and_trails
• Working example: kosli-dev/cli's own .github/workflows/docker.yml