feat(gateway): add TOML configuration file (RFC 0003)

[TaylorMutch]

Summary

Implements RFC 0003: an opt-in --config / OPENSHELL_GATEWAY_CONFIG flag that loads a TOML file with gateway-wide settings and per-driver tables. Source precedence stays CLI > env > file > built-in default, so existing flags and OPENSHELL_* env vars are not affected when no file is supplied. Includes the matching Helm ConfigMap cutover.

Related Issue

N/A — implements RFC 0003.

Changes

• Loader (crates/openshell-server/src/config_file.rs): TOML parser with deny_unknown_fields everywhere, version-1 enforcement, env-only rejection of database_url / ssh_handshake_secret, and a driver_table() that overlays inheritable [openshell.gateway] defaults onto each [openshell.drivers.<name>] table.
• CLI (crates/openshell-server/src/cli.rs): new --config flag; merge_file_into_args() applies file values only when clap's ValueSource::DefaultValue indicates an arg was not supplied; new build_vm_config() / build_docker_config() consume merged driver tables.
• Driver crates (kubernetes, docker, podman, vm): config structs derive Deserialize/Serialize with struct-level #[serde(default, deny_unknown_fields)]. SupervisorSideloadMethod gains Deserialize with kebab-case rename. Refreshed Default impls where needed.
• run_server / build_compute_runtime thread Option<&ConfigFile> through; the ad-hoc env reads for Kubernetes/Podman supervisor settings now fold into the file-driven pipeline (env still wins).
• Helm chart: new templates/gateway-config.yaml ConfigMap rendering gateway.toml from existing .Values.server.* / .Values.service.* / .Values.supervisor.* keys; StatefulSet mounts it at /etc/openshell/gateway.toml, passes --config, and drops the 18 migrated OPENSHELL_* env entries. Only OPENSHELL_SSH_HANDSHAKE_SECRET (Secret-backed) remains.
• Docs: architecture/gateway.md documents source precedence and the inheritance allowlist; examples/gateway/gateway.example.toml ships a worked example.
• Inheritance scope (Q4 high-overlap set, per-driver allowlist): default_image, supervisor_image, image_pull_policy, client_tls_secret_name, host_gateway_ip, enable_user_namespaces, guest_tls_ca/cert/key.
• Follow-up commit removes ssh_handshake_skew_secs / ssh_handshake_secret references from examples and architecture ahead of their planned removal.

Testing

• mise run pre-commit passes
• Unit tests added/updated — 10 new tests in config_file.rs (parse, secret rejection, version, unknown field, inheritance merge, override) and 6 new tests in cli.rs (file value applies when defaulted, CLI overrides file, env overrides file, OIDC block, driver inheritance, driver override). Full mise run test passes.
• E2E tests — test:e2e label applied; the sandbox-infra path should be exercised against the new --config startup mode.

Manual smoke:

• helm lint . clean, helm template . renders gateway.toml with TLS, with TLS disabled, and with OIDC enabled.
• TOML loader rejects database_url, ssh_handshake_secret, version = 2, and unknown fields with friendly errors.

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)
• Architecture docs updated (if applicable)

Deferred (explicit follow-ups)

• Helm unit tests for the new ConfigMap (separate effort).
• "Raw ConfigMap contents" escape hatch in values.yaml.
• --config-dir directory loader (RFC open question Prototype data access layer that can connect to outlook data #2).
• RFC schema-gap fixes (Kubernetes ssh fields, Podman 6 missing fields, VM 9+ missing fields).

[github-actions]

Label test:e2e applied for e324f7e. Open the existing run and click Re-run all jobs to execute with the label set. The E2E Gate check on this PR will flip green automatically once the run finishes.

[drew]

Ran this through codex, these seem reasonable to consider. "Restart pods when the rendered ConfigMap changes" feels possibly out of scope.

Review Comments

[P1] Move Helm grpc_endpoint into a supported table

/Users/anewberry/dev/openshell.tmutch-gateway-config-impl/deploy/helm/openshell/templates/gateway-config.yaml:36

The generated Helm config always writes grpc_endpoint under [openshell.gateway], but GatewayFileSection uses deny_unknown_fields and has no grpc_endpoint field. Default chart installs will fail during config_file::load before the gateway starts; this needs to move under
[openshell.drivers.kubernetes] or be added to the gateway schema and merge path.

[P1] Keep Kubernetes imagePullPolicy empty by default

/Users/anewberry/dev/openshell.tmutch-gateway-config-impl/crates/openshell-driver-kubernetes/src/config.rs:76

When no Kubernetes image pull policy is configured, this now defaults to missing, which is the Podman vocabulary and is not a valid Kubernetes imagePullPolicy. Because the driver emits the field whenever it is non-empty, default Kubernetes deployments will create pods rejected by the
API; keep this default empty so Kubernetes applies its own default.

[P1] Preserve driver-table gRPC endpoints

/Users/anewberry/dev/openshell.tmutch-gateway-config-impl/crates/openshell-server/src/lib.rs:508

When a TOML file sets [openshell.drivers.kubernetes].grpc_endpoint, kubernetes_config_from_file reads it and this assignment immediately overwrites it with config.grpc_endpoint, which can only come from CLI/env in this patch. File-only Kubernetes configs therefore pass an empty
OPENSHELL_ENDPOINT into sandboxes, so the callback path breaks even though the driver table contains the endpoint.

[P2] Keep disableTls wired after moving config to TOML

/Users/anewberry/dev/openshell.tmutch-gateway-config-impl/deploy/helm/openshell/templates/statefulset.yaml:49

When .Values.server.disableTls is true, the ConfigMap omits the TLS section, but this args/env block no longer passes --disable-tls or OPENSHELL_DISABLE_TLS. RunArgs.disable_tls remains false, so startup still requires TLS cert/key/CA paths and the plaintext Helm option fails.

[P2] Restart pods when the rendered ConfigMap changes

/Users/anewberry/dev/openshell.tmutch-gateway-config-impl/deploy/helm/openshell/templates/statefulset.yaml:65

After moving settings such as log level, sandbox image, TLS paths, and driver options into this ConfigMap, Helm upgrades that change those values no longer change the StatefulSet pod template. The gateway only reads /etc/openshell/gateway.toml at startup, so without a checksum
annotation or similar rollout trigger, pods keep running with stale config.

[P2] Honor health and metrics listener addresses

/Users/anewberry/dev/openshell.tmutch-gateway-config-impl/crates/openshell-server/src/cli.rs:511

For config files that set health or metrics on a different interface than the main listener, these lines copy only the port and later build the listener with args.bind_address. For example, a health listener intended for loopback can be exposed on the public bind address; preserve the
full SocketAddr from the TOML file or validate that separate IPs are unsupported.

[P2] Apply ssh_session_ttl_secs from the TOML file

/Users/anewberry/dev/openshell.tmutch-gateway-config-impl/crates/openshell-server/src/config_file.rs:101

The loader accepts ssh_session_ttl_secs, but merge_file_into_args and run_from_args never apply it to openshell_core::Config with with_ssh_session_ttl_secs. Operators can set the documented key and get a successful parse while session tokens still use the built-in default TTL.

[TaylorMutch]

Sounds good, agree these look reasonable to me.

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-1317.docs.buildwithfern.com/openshell

[TaylorMutch]

Ran an audit with Claude:

---

⏺ Verification complete. No breaking changes to either CLI or env-var surface.

  Gateway binary (crates/openshell-server/src/cli.rs):
  - 0 env = "OPENSHELL_*" declarations removed or renamed
  - 0 --long-flag declarations removed or renamed
  - 1 additive: --config / OPENSHELL_GATEWAY_CONFIG (new TOML loader entry)
  - All 47 prior RunArgs fields preserved; 1 new field (config) added

  Direct env reads (crates/openshell-server/src/lib.rs):
  - All 8 prior std::env::var("OPENSHELL_*") reads preserved
  - 1 additive: OPENSHELL_SUPERVISOR_SIDELOAD_METHOD

  Helm chart StatefulSet env block:
  - 26 envs dropped from the StatefulSet (now in the TOML ConfigMap)
  - 1 retained: OPENSHELL_SSH_HANDSHAKE_SECRET (must stay as a Kubernetes Secret ref)
  - Every dropped env has a verified-rendered TOML equivalent. Spot-checked with helm template (including conditional toggles like disableGatewayAuth, all OIDC fields, server SANs, image-pull policies)

  For operators who set OPENSHELL_* via custom extraEnv or pod-level injection:
  - Every clap env = "OPENSHELL_*" is unchanged, so the gateway binary still reads each one
  - Env still wins over the TOML file — locked in by the merge tests in cli.rs (env_var_overrides_file_value, env_var_overrides_file_loopback_service_http, etc.)

  Net effect

  ┌────────────────────────────┬─────────┬─────────┬────────────────────────────────┐
  │          Surface           │ Removed │ Renamed │             Added              │
  ├────────────────────────────┼─────────┼─────────┼────────────────────────────────┤
  │ Gateway CLI flags          │ 0       │ 0       │ --config                       │
  ├────────────────────────────┼─────────┼─────────┼────────────────────────────────┤
  │ Gateway env vars           │ 0       │ 0       │ OPENSHELL_GATEWAY_CONFIG       │
  ├────────────────────────────┼─────────┼─────────┼────────────────────────────────┤
  │ Helm extraEnv API contract │ 0       │ 0       │ 0 (still respected by gateway) │
  └────────────────────────────┴─────────┴─────────┴────────────────────────────────┘