Skip to content

Commit 5182b36

Browse files
IlyaasKIlyaasK
committed
Define the v1 acceptance release gate
1 parent 91aa9cf commit 5182b36

4 files changed

Lines changed: 98 additions & 13 deletions

File tree

‎README.md‎

Lines changed: 3 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -147,19 +147,14 @@ export KERNEL_API_KEY="..."
147147
export KERNEL_PROJECT_ID="..."
148148
```
149149

150-
Run the durable resource acceptance tests:
151-
152-
```sh
153-
go test -count=1 -timeout=30m -v ./internal/resources/browserpool -run TestAcc
154-
go test -count=1 -timeout=30m -v ./internal/resources/extension -run TestAcc
155-
go test -count=1 -timeout=30m -v ./internal/resources/project -run TestAcc
156-
```
157-
158150
The tests create uniquely named durable resources and register independent
159151
cleanup. Extension acceptance creates a small temporary Manifest V3 archive and
160152
tests checksum-driven replacement. Browser-pool deletion remains `force=false`.
161153
The tests do not acquire browsers or perform runtime recovery.
162154

155+
Use the commands in the [v1 acceptance matrix](docs/acceptance.md), which is the
156+
single source for current coverage, tag blockers, and the release-run record.
157+
163158
## Architecture
164159

165160
See [docs/architecture.md](docs/architecture.md) for package layout, Terraform semantics, testing strategy, and release planning.

‎docs/acceptance.md‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# v1 Acceptance Matrix
2+
3+
This document is the live-API release gate for the first public v1. Unit tests
4+
remain the fast default; acceptance tests run only through explicit local opt-in
5+
or the manual GitHub Actions workflow.
6+
7+
## Gate Rules
8+
9+
- Set both `TF_ACC=1` and `KERNEL_ACC=1`.
10+
- Use unique `kernel-tf-*` names for every created fixture.
11+
- Register cleanup as soon as a canonical ID exists.
12+
- Verify deletion with a coded `not_found` response where the API supports it.
13+
- Never acquire, release, flush, invoke, force-release, or recover runtime state.
14+
- Keep live tests out of pull-request CI.
15+
- Record one complete green matrix run against the release commit before tagging.
16+
- Every registered v1 surface must have a live acceptance test and pass against
17+
the release commit. A fixture blocker is a v1 tag blocker, not a release-note
18+
exception. An unregistered deferred resource is not part of the live matrix.
19+
20+
## Environment
21+
22+
The current workflow uses:
23+
24+
```sh
25+
export TF_ACC=1
26+
export KERNEL_ACC=1
27+
export KERNEL_API_KEY=...
28+
export KERNEL_PROJECT_ID=...
29+
export KERNEL_ALT_PROJECT_ID=... # optional second project
30+
export KERNEL_BASE_URL=... # optional non-production API
31+
```
32+
33+
Future fixture-backed data-source tests may add narrowly named variables only
34+
when the provider cannot create and clean up the fixture through a durable SDK
35+
operation. Secrets must remain GitHub Actions secrets and must not be printed.
36+
37+
## Current Matrix
38+
39+
| Surface | Repository test status | Acceptance scenario | Required follow-up |
40+
| --- | --- | --- | --- |
41+
| `kernel_project` resource | Test present | Create, rename, no-drift plan, import, delete, and 404 verification. | None. |
42+
| `kernel_browser_pool` resource | Test present | Create, durable update with stable ID, no-drift plan, bare and project-qualified import paths, explicit project scope, non-force delete, and 404 verification. | Keep leased-browser conflict behavior in unit tests; acceptance must not create runtime leases. |
43+
| `kernel_extension` resource | Test present | Upload, checksum state, no-drift plan, metadata-only import, content replacement with new ID, old-ID disappearance, delete, and 404 verification. | Add project-qualified import to live coverage before the v1 tag. |
44+
| `kernel_browser_pool` data source | Test present; tag blocker | A uniquely created pool is read by canonical ID and byte-exact name, including normalized durable configuration and a no-drift plan. | Add post-cleanup Get verification that requires coded `not_found`; the current cleanup sends Delete but does not prove absence. |
45+
| `kernel_project` data source | Test missing; tag blocker | Unit and fake-client tests only. | Use a uniquely created `kernel_project` fixture and verify current/ID/name lookup plus no drift. |
46+
| `kernel_extension` data source | Test missing; tag blocker | Unit and fake-client tests only. | Use a uniquely uploaded `kernel_extension` fixture and verify ID/name metadata lookup plus no drift. |
47+
| `kernel_profile` data source | Test missing; tag blocker | Unit and fake-client tests only. | Add durable SDK fixture create/delete helpers, then verify ID/name lookup and cleanup. Do not model runtime-written profile contents. |
48+
| `kernel_proxy` data source | Test missing; tag blocker | Unit and fake-client tests only. | Add a durable, non-secret-leaking proxy fixture strategy and verify ID/name lookup, masked metadata, and cleanup. |
49+
| `kernel_app` data source | Fixture blocked; tag blocker | Unit, SDK transport, pagination, ambiguity, project-scope, and Framework state tests only. | Provide a release-owned running deployment fixture or a deterministic durable deployment setup. Verify exact app/version lookup without invocation and without exposing env values. |
50+
| `kernel_api_key` data source | Deferred; unregistered | No provider surface yet. | Wait for a tagged SDK with exact-name filtering, then add masked ID/name lookup acceptance. |
51+
| Profile, proxy, deployment, and API-key resources | Deferred; unregistered | No provider surfaces yet. | Enter the matrix only after their documented API/SDK/state blockers are resolved and implementation lands. |
52+
53+
"Test present" describes code in the repository; it does not claim a run
54+
against the release commit. The release record below supplies that evidence.
55+
56+
## Current Commands
57+
58+
Run the four existing packages independently for fast failure isolation:
59+
60+
```sh
61+
go test -count=1 -timeout=30m -v ./internal/resources/project -run TestAcc
62+
go test -count=1 -timeout=30m -v ./internal/resources/browserpool -run TestAcc
63+
go test -count=1 -timeout=30m -v ./internal/resources/extension -run TestAcc
64+
go test -count=1 -timeout=30m -v ./internal/datasources/browserpool -run TestAcc
65+
```
66+
67+
The manual `Acceptance` workflow runs the same packages as separate matrix jobs
68+
with `fail-fast: false`. Add a package to that workflow in the same PR that adds
69+
its first live test.
70+
71+
## Release Record
72+
73+
Record this information in the v1 release PR or release issue, not in this
74+
repository with secrets:
75+
76+
```text
77+
Commit:
78+
Workflow run URL:
79+
API environment:
80+
Started at:
81+
Completed at:
82+
Package results:
83+
Interrupted or timed-out jobs:
84+
Leaked-resource audit completed:
85+
Unregistered deferred surfaces:
86+
```
87+
88+
An interrupted process can bypass `t.Cleanup`. Follow the ordered cleanup and
89+
404 verification procedure in [Release And Security Checklist](release.md)
90+
before rerunning or tagging.

‎docs/architecture.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -335,6 +335,9 @@ Acceptance tests must:
335335
- avoid browser/session runtime operations
336336
- exercise import and real delete behavior for each resource
337337

338+
The current live coverage and remaining release gates are tracked in the
339+
[v1 Acceptance Matrix](acceptance.md).
340+
338341
## PR Slicing
339342

340343
PRs must be small, coherent, and shippable. No PR should rely on hidden follow-up work to keep the repo healthy.

‎docs/release.md‎

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -12,11 +12,8 @@ Use this checklist before publishing a Kernel Terraform provider version.
1212
- Run `terraform fmt -check -recursive examples`.
1313
- Run `go test -short -timeout=2m ./...`.
1414
- Run `go vet ./...`.
15-
- Run the complete opt-in acceptance matrix for every v1 resource and data source with real credentials before the first public release.
16-
- Browser pools: `TF_ACC=1 KERNEL_ACC=1 KERNEL_API_KEY=... KERNEL_PROJECT_ID=... go test -count=1 -timeout=30m -v ./internal/resources/browserpool -run TestAcc`.
17-
- Extensions: `TF_ACC=1 KERNEL_ACC=1 KERNEL_API_KEY=... KERNEL_PROJECT_ID=... go test -count=1 -timeout=30m -v ./internal/resources/extension -run TestAcc`.
18-
- Projects: `TF_ACC=1 KERNEL_ACC=1 KERNEL_API_KEY=... go test -count=1 -timeout=30m -v ./internal/resources/project -run TestAcc`.
19-
- The manual `Acceptance` workflow runs all three packages in parallel; expand its matrix as v1 resources land and keep live tests out of normal PR CI.
15+
- Run the complete [v1 acceptance matrix](acceptance.md) for every registered v1 resource and data source with real credentials before the first public release.
16+
- Use the commands and status table in `docs/acceptance.md` as the single source of truth. The manual `Acceptance` workflow runs all current packages in parallel; add each new package in the same PR as its first live test and keep live tests out of normal PR CI.
2017
- Process-level timeouts can bypass Go test cleanup. After an interrupted or hard-timeout run:
2118
1. In the Kernel dashboard or durable API, find projects, browser pools, and extensions named `kernel-tf-*` that were created during the failed workflow run.
2219
2. Delete leaked browser pools first with `force=false`. If deletion conflicts with a lease, wait for the lease to end; do not force-release or recover the browser from Terraform cleanup.

0 commit comments

Comments
 (0)