Skip to content

feat: add binary download and caching system with sigstore verification #217

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Cali0707 merged 5 commits into from
Dec 4, 2025
Merged

feat: add binary download and caching system with sigstore verification #217

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
60cfeea
feat: add binary download and caching system with sigstore verification
nader-ziada Nov 19, 2025
3c826ae
fix linter issues
nader-ziada Nov 19, 2025
39f26e8
update DownloadBinaryProvider test
nader-ziada Nov 19, 2025
208a5e6
some updates after rebase
nader-ziada Dec 1, 2025
54442f4
verify files on download and from cache
nader-ziada Dec 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Changed
- **BREAKING**: Migrated from single-file format (schema version 0.1.0) to two-file format (schema version 0.2.0). The `mcpfile.yaml` now contains only tool definitions (`MCPToolDefinitions`), while server runtime configuration has been moved to a separate `mcpserver.yaml` file (`MCPServerConfig`). The `mcpFileVersion` field has been renamed to `schemaVersion`, and both files now require a `kind` field. Users must migrate existing configurations - see `MIGRATION.md` for step-by-step instructions. (#213)

## [Unreleased] – PR #217

### Added
- Binary download and caching system for `genmcp build` command - server binaries are now downloaded from GitHub releases instead of being embedded in the CLI (#151)
- Sigstore-based cryptographic verification of downloaded binaries for security (built into CLI, no external dependencies) (#151)
- Version-platform cache management to store and reuse downloaded binaries across builds (#151)
- `--server-version` flag to specify which server binary version to download (#151)
- Automatic "latest" resolution for development builds - dev CLI versions automatically download the latest stable server binaries (#151)
- Automatic fallback to cached version if download fails - works offline with previously cached binaries (#151)
- Automatic cache cleanup - keeps last 3 versions per platform to prevent unbounded cache growth (#151)

### Changed
- `genmcp build` now downloads server binaries from GitHub releases, significantly reducing CLI binary size (#151)
- Server binaries are cached locally (in user cache directory) and reused across builds (#151)

### Removed
- Embedded server binaries from CLI - binaries are now downloaded on-demand, reducing the CLI size from ~100MB to ~78MB (#151)
- `--use-embedded-binaries` flag (no longer needed as embedded binaries have been removed) (#151)

## [v0.1.1]

### Added
Expand Down
32 changes: 32 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,27 @@ genmcp convert https://petstore.swagger.io/v2/swagger.json

The `build` command packages your MCP server and mcpfile into a container image. By default, it builds multi-architecture images for broader platform support.

**Binary Downloads:** The build process automatically downloads the required server binaries from GitHub releases and caches them locally for reuse. Downloaded binaries are cryptographically verified using Sigstore to ensure authenticity and integrity.

**Version Matching:**
- **Release builds**: Download server binaries matching the CLI version (e.g., `v0.1.0` downloads `v0.1.0` server)
- **Development builds**: Automatically use latest release server binaries
- **Custom version**: Use `--server-version` flag to specify a specific server version

**Requirements:**
- Network access to download binaries from GitHub releases and query GitHub API
- No external dependencies required (verification is built into the CLI)

**Offline Support:**
- If download fails, automatically uses the latest cached version for that platform
- Warns when using cached fallback version
- Works completely offline once binaries are cached

**Cache Management:**
- Automatically keeps the last 3 versions per platform
- Older versions are cleaned up when downloading new binaries
- Manual cleanup: remove `~/.cache/genmcp/binaries/` (Linux), `~/Library/Caches/.genmcp/binaries/` (macOS), or `%LOCALAPPDATA%\.genmcp\binaries\` (Windows)

#### Multi-Architecture Build (Default)

When no `--platform` is specified, builds for both `linux/amd64` and `linux/arm64`:
Expand Down Expand Up @@ -186,14 +207,25 @@ genmcp build --tag myapp:latest \
# Specify mcpfile location
genmcp build --tag myapp:latest -f path/to/mcpfile.yaml

# Use specific server version
genmcp build --tag myapp:latest --server-version v0.1.0

# Full example with all options
genmcp build \
--tag myregistry.io/myapp:v1.0.0 \
--file ./custom-mcpfile.yaml \
--platform linux/amd64 \
--server-version v0.1.1 \
--push
```

**Binary Cache Location:** Downloaded binaries are cached in your user cache directory and reused across builds:
- **Linux**: `~/.cache/genmcp/binaries/`
- **macOS**: `~/Library/Caches/.genmcp/binaries/`
- **Windows**: `%LOCALAPPDATA%\.genmcp\binaries\`

The cache stores version-platform pairs and validates checksums on each use.

**Note:** When building multi-arch locally, Docker daemon doesn't support manifest lists, so each platform is saved with a platform-specific tag (e.g., `myapp:latest-linux-amd64`). Additionally, the original tag (`myapp:latest`) is saved with your host platform's image if available, otherwise the first built platform. When pushing to a registry with `--push`, a proper multi-arch manifest list is created.

### Converting CLI (Experimental)
Expand Down
26 changes: 22 additions & 4 deletions docs/commands.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -341,15 +341,30 @@ genmcp build [flags]
| `--base-image` | | *(auto)* | Base container image to build on |
| `--platform` | | `multi-arch` | Target platform (e.g., `linux/amd64`) |
| `--push` | | `false` | Push to registry instead of saving locally |
| `--server-version`| | *(auto)* | Server binary version to download (default: latest for dev builds, CLI version for releases) |

#### How It Works

The `build` command:

1. **Validates both GenMCP config files** - Ensures both tool definitions and server config are valid
2. **Builds container image** - Creates a containerized MCP server with both files included
3. **Supports multi-arch** - By default builds for `linux/amd64` and `linux/arm64`
4. **Saves or pushes** - Either stores locally or pushes to a container registry
1. **Downloads server binaries** - Fetches required binaries from GitHub releases (with cosign verification)
2. **Validates both GenMCP config files** - Ensures both tool definitions and server config are valid
3. **Builds container image** - Creates a containerized MCP server with both files included
4. **Supports multi-arch** - By default builds for `linux/amd64` and `linux/arm64`
5. **Saves or pushes** - Either stores locally or pushes to a container registry

**Binary Management:**
- Server binaries are downloaded from GitHub releases and cached locally
- Downloaded binaries are cryptographically verified with Sigstore for security (built-in)
- Cache location:
- Linux: `~/.cache/genmcp/binaries/`
- macOS: `~/Library/Caches/.genmcp/binaries/`
- Windows: `%LOCALAPPDATA%\.genmcp\binaries\`
- **Version Matching**:
- Release CLI versions download matching server binaries
- Development CLI versions automatically use latest release
- Use `--server-version` to override
- **Requirements**: Network access to GitHub releases and API

#### Examples

Expand All @@ -363,6 +378,9 @@ genmcp build -f config/tools.yaml -s config/server.yaml --tag myapi:v1.0

# Build for specific platform only
genmcp build --tag myapi:latest --platform linux/amd64

# Build with specific server version
genmcp build --tag myapi:latest --server-version v0.1.0
```

**Multi-architecture build:**
Expand Down
118 changes: 110 additions & 8 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ require (
github.com/onsi/gomega v1.38.2
github.com/openai/openai-go/v2 v2.7.1
github.com/pb33f/libopenapi v0.28.2
github.com/sigstore/sigstore-go v1.1.3
github.com/spf13/cobra v1.10.1
github.com/stretchr/testify v1.11.1
github.com/yosida95/uritemplate/v3 v3.0.2
Expand All @@ -23,7 +24,7 @@ require (
)

require (
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mailru/easyjson v0.9.0 // indirect
github.com/tidwall/gjson v1.18.0 // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.1 // indirect
Expand All @@ -32,32 +33,92 @@ require (
)

require (
cel.dev/expr v0.24.0 // indirect
cloud.google.com/go v0.121.6 // indirect
cloud.google.com/go/auth v0.16.5 // indirect
cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
cloud.google.com/go/compute/metadata v0.8.0 // indirect
cloud.google.com/go/iam v1.5.2 // indirect
cloud.google.com/go/longrunning v0.6.7 // indirect
cloud.google.com/go/monitoring v1.24.2 // indirect
cloud.google.com/go/spanner v1.84.1 // indirect
cloud.google.com/go/storage v1.56.1 // indirect
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect
github.com/GoogleCloudPlatform/grpc-gcp-go/grpcgcp v1.5.3 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.29.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.53.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.53.0 // indirect
github.com/Masterminds/semver/v3 v3.4.0 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect
github.com/bahlo/generic-list-go v0.2.0 // indirect
github.com/blang/semver v3.5.1+incompatible // indirect
github.com/buger/jsonparser v1.1.1 // indirect
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
github.com/cenkalti/backoff/v5 v5.0.3 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/cncf/xds/go v0.0.0-20250501225837-2ac532fd4443 // indirect
github.com/containerd/errdefs v1.0.0 // indirect
github.com/containerd/errdefs/pkg v0.3.0 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.18.1 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/cyberphone/json-canonicalization v0.0.0-20241213102144-19d51d7fe467 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0 // indirect
github.com/digitorus/pkcs7 v0.0.0-20230818184609-3a137a874352 // indirect
github.com/digitorus/timestamp v0.0.0-20231217203849-220c5c2851b7 // indirect
github.com/distribution/reference v0.6.0 // indirect
github.com/docker/cli v29.0.3+incompatible // indirect
github.com/docker/distribution v2.8.3+incompatible // indirect
github.com/docker/docker v28.5.2+incompatible // indirect
github.com/docker/docker-credential-helpers v0.9.3 // indirect
github.com/docker/go-connections v0.5.0 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/envoyproxy/go-control-plane/envoy v1.32.4 // indirect
github.com/envoyproxy/protoc-gen-validate v1.2.1 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/fsnotify/fsnotify v1.8.0 // indirect
github.com/go-chi/chi/v5 v5.2.3 // indirect
github.com/go-jose/go-jose/v4 v4.1.1 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-openapi/analysis v0.23.0 // indirect
github.com/go-openapi/errors v0.22.2 // indirect
github.com/go-openapi/jsonpointer v0.21.0 // indirect
github.com/go-openapi/jsonreference v0.21.0 // indirect
github.com/go-openapi/loads v0.22.0 // indirect
github.com/go-openapi/runtime v0.28.0 // indirect
github.com/go-openapi/spec v0.21.0 // indirect
github.com/go-openapi/strfmt v0.23.0 // indirect
github.com/go-openapi/swag v0.24.1 // indirect
github.com/go-openapi/swag/cmdutils v0.24.0 // indirect
github.com/go-openapi/swag/conv v0.24.0 // indirect
github.com/go-openapi/swag/fileutils v0.24.0 // indirect
github.com/go-openapi/swag/jsonname v0.24.0 // indirect
github.com/go-openapi/swag/jsonutils v0.24.0 // indirect
github.com/go-openapi/swag/loading v0.24.0 // indirect
github.com/go-openapi/swag/mangling v0.24.0 // indirect
github.com/go-openapi/swag/netutils v0.24.0 // indirect
github.com/go-openapi/swag/stringutils v0.24.0 // indirect
github.com/go-openapi/swag/typeutils v0.24.0 // indirect
github.com/go-openapi/swag/yamlutils v0.24.0 // indirect
github.com/go-openapi/validate v0.24.0 // indirect
github.com/go-task/slim-sprig/v3 v3.0.0 // indirect
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
github.com/goccy/go-json v0.10.3 // indirect
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 // indirect
github.com/google/certificate-transparency-go v1.3.2 // indirect
github.com/google/go-cmp v0.7.0 // indirect
github.com/google/pprof v0.0.0-20250403155104-27863c87afa6 // indirect
github.com/google/pprof v0.0.0-20250602020802-c6617b811d0e // indirect
github.com/google/s2a-go v0.1.9 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.6 // indirect
github.com/googleapis/gax-go/v2 v2.15.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.2 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/in-toto/attestation v1.1.2 // indirect
github.com/in-toto/in-toto-golang v0.9.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jedisct1/go-minisign v0.0.0-20211028175153-1c139d1cc84b // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/klauspost/compress v1.18.1 // indirect
github.com/lestrrat-go/blackmagic v1.0.4 // indirect
github.com/lestrrat-go/dsig v1.0.0 // indirect
Expand All @@ -66,22 +127,54 @@ require (
github.com/lestrrat-go/httprc/v3 v3.0.1 // indirect
github.com/lestrrat-go/option v1.0.1 // indirect
github.com/lestrrat-go/option/v2 v2.0.0 // indirect
github.com/letsencrypt/boulder v0.0.0-20240620165639-de9c06129bec // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/sys/sequential v0.6.0 // indirect
github.com/oklog/ulid v1.3.1 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.1 // indirect
github.com/opentracing/opentracing-go v1.2.0 // indirect
github.com/pb33f/jsonpath v0.1.2 // indirect
github.com/pb33f/ordered-map/v2 v2.3.0 // indirect
github.com/pelletier/go-toml/v2 v2.2.3 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/sagikazarmark/locafero v0.7.0 // indirect
github.com/sassoftware/relic v7.2.1+incompatible // indirect
github.com/secure-systems-lab/go-securesystemslib v0.9.1 // indirect
github.com/segmentio/asm v1.2.1 // indirect
github.com/shibumi/go-pathspec v1.3.0 // indirect
github.com/sigstore/protobuf-specs v0.5.0 // indirect
github.com/sigstore/rekor v1.4.2 // indirect
github.com/sigstore/rekor-tiles v0.1.11 // indirect
github.com/sigstore/sigstore v1.9.6-0.20250729224751-181c5d3339b3 // indirect
github.com/sigstore/timestamp-authority v1.2.9 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/spf13/pflag v1.0.9 // indirect
github.com/sourcegraph/conc v0.3.0 // indirect
github.com/spf13/afero v1.12.0 // indirect
github.com/spf13/cast v1.7.1 // indirect
github.com/spf13/pflag v1.0.10 // indirect
github.com/spf13/viper v1.20.1 // indirect
github.com/spiffe/go-spiffe/v2 v2.5.0 // indirect
github.com/stretchr/objx v0.5.2 // indirect
github.com/subosito/gotenv v1.6.0 // indirect
github.com/theupdateframework/go-tuf v0.7.0 // indirect
github.com/theupdateframework/go-tuf/v2 v2.2.0 // indirect
github.com/titanous/rocacheck v0.0.0-20171023193734-afe73141d399 // indirect
github.com/transparency-dev/formats v0.0.0-20250421220931-bb8ad4d07c26 // indirect
github.com/transparency-dev/merkle v0.0.2 // indirect
github.com/transparency-dev/tessera v1.0.0-rc3 // indirect
github.com/valyala/fastjson v1.6.4 // indirect
github.com/vbatts/tar-split v0.12.2 // indirect
github.com/zeebo/errs v1.4.0 // indirect
go.mongodb.org/mongo-driver v1.14.0 // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/contrib/detectors/gcp v1.38.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 // indirect
go.opentelemetry.io/otel v1.38.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.38.0 // indirect
Expand All @@ -90,18 +183,27 @@ require (
go.opentelemetry.io/otel/sdk/metric v1.38.0 // indirect
go.opentelemetry.io/otel/trace v1.38.0 // indirect
go.opentelemetry.io/proto/otlp v1.8.0 // indirect
go.uber.org/multierr v1.10.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.yaml.in/yaml/v2 v2.4.2 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
go.yaml.in/yaml/v4 v4.0.0-rc.2 // indirect
golang.org/x/crypto v0.44.0 // indirect
golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b // indirect
golang.org/x/mod v0.30.0 // indirect
golang.org/x/net v0.47.0 // indirect
golang.org/x/oauth2 v0.33.0 // indirect
golang.org/x/sync v0.18.0 // indirect
golang.org/x/sys v0.38.0 // indirect
golang.org/x/term v0.37.0 // indirect
golang.org/x/text v0.31.0 // indirect
golang.org/x/time v0.12.0 // indirect
golang.org/x/tools v0.39.0 // indirect
google.golang.org/protobuf v1.36.8 // indirect
google.golang.org/api v0.248.0 // indirect
google.golang.org/genproto v0.0.0-20250603155806-513f23925822 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250825161204-c5933d9347a5 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250825161204-c5933d9347a5 // indirect
google.golang.org/grpc v1.75.0 // indirect
google.golang.org/protobuf v1.36.9 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/klog/v2 v2.130.1 // indirect
)
Loading