Add documentation for the case cache (#3178)

Add `npm run node` helper to sit alongside `npm run standalone` and `npm run wpt`.

Update build.md for `out-node`.

Co-authored-by: Kai Ninomiya <[email protected]>

Author: ben-clayton

docs/build.md

@@ -1,16 +1,48 @@
 # Building
 
 Building the project is not usually needed for local development.
-However, for exports to WPT, or deployment (https://gpuweb.github.io/cts/),
+However, for exports to WPT, NodeJS, [or deployment](https://gpuweb.github.io/cts/),
 files can be pre-generated.
 
-The project builds into two directories:
+## Build types
 
-- `out/`: Built framework and test files, needed to run standalone or command line.
-- `out-wpt/`: Build directory for export into WPT. Contains:
-    - An adapter for running WebGPU CTS tests under WPT
-    - A copy of the needed files from `out/`
-    - A copy of any `.html` test cases from `src/`
+The project can be built several different ways, each with a different output directory:
+
+### 0. on-the-fly builds (no output directory)
+
+Use `npm run start` to launch a server that live-compiles everything as needed.
+Use `tools/run_node` and other tools to run under `ts-node` which compiles at runtime.
+
+### 1. `out` directory
+
+**Built with**: `npm run standalone`
+
+**Serve locally with**: `npx grunt serve`
+
+**Used for**: Static deployment of the CTS, primarily for [gpuweb.github.io/cts](https://gpuweb.github.io/cts/).
+
+### 2. `out-wpt` directory
+
+**Built with**: `npm run wpt`
+
+**Used for**: Deploying into [Web Platform Tests](https://web-platform-tests.org/). See [below](#export-to-wpt) for more information.
+
+Contains:
+
+- An adapter for running WebGPU CTS tests under WPT
+- A copy of the needed files from `out/`
+- A copy of any `.html` test cases from `src/`
+
+### 3. `out-node` directory
+
+**Built with**: `npm run node`
+
+**Used for**: Running NodeJS tools, if you want to specifically avoid the live-compilation overhead of the `tools/` versions, or are running on a deployment which no longer has access to `ts-node` (which is a build-time dependency). For example:
+
+- `node out-node/common/runtime/cmdline.js` ([source](../src/common/runtime/cmdline.ts)) - A command line interface test runner
+- `node out-node/common/runtime/server.js` ([source](../src/common/runtime/server.ts)) - An HTTP server for executing CTS tests with a REST interface
+
+## Testing
 
 To build and run all pre-submit checks (including type and lint checks and
 unittests), use:
@@ -25,15 +57,6 @@ For checks only:
 npm run check
 ```
 
-For a quicker iterative build:
-
-```sh
-npm run standalone
-```
-
-## Run
-
-To serve the built files (rather than using the dev server), run `npx grunt serve`.
 
 ## Export to WPT
 

docs/case_cache.md

@@ -0,0 +1,76 @@
+# Case Cache
+
+The WebGPU CTS contains many tests that check that the results of an operation
+fall within limits defined by the WebGPU and WGSL specifications. The
+computation of these allowed limits can be very expensive to calculate, however
+the values do not vary by platform or device, and can be precomputed and reused
+for multiple CTS runs.
+
+## File cache
+
+To speed up execution of the CTS, the CTS git repo holds holds pre-computed
+test cases, serialized in a set of gzip-compressed binary files under
+[`src/resources/cache`](../src/resources/cache).
+
+These files are regenerated by [`src/common/tools/gen_cache.ts`](../src/common/tools/gen_cache.ts)
+which can be run with `npx grunt run:generate-cache`.
+This tool is automatically run by the various Grunt build commands.
+
+As generating the cache is expensive (hence why we build it ahead of time!) the
+cache generation tool will only re-build the cache files it believes may be out
+of date. To determine which files it needs to rebuild, the tool calculates a
+hash of all the transitive source TypeScript files that are used to build the
+output, and compares this hash to the hash stored in
+[`src/resources/cache/hashes.json`](`../src/resources/cache/hashes.json`). Only
+those cache files with differing hashes are rebuilt.
+
+Since source changes will sometimes change the hash without changing the generated cache,
+sometimes the cache will be regenerated unnecessarily. **This is OK, but try to avoid committing
+no-op regenerations - this will happen if your version of Node produces different gzip outputs
+than the original committer's Node did for the same input.**
+
+The cache files are copied from [`src/resources/cache`](../src/resources/cache)
+to the `resources/cache` subdirectory of the
+[`out` and `out-node` build directories](build.md#build-types), so the runner
+can load these cache files.
+
+The GitHub presubmit checks will error if the cache files or
+[`hashes.json`](`../src/resources/cache/hashes.json`) need updating.
+
+## In memory cache
+
+If a cache file cannot be found, then the [`CaseCache`](../src/webgpu/shader/execution/expression/case_cache.ts)
+will build the cases during CTS execution and store the results in an in-memory LRU cache.
+
+## Using the cache
+
+To add test cases to the cache:
+
+1. Import `makeCaseCache` from [`'case_cache.js'`](../src/webgpu/shader/execution/expression/case_cache.ts);
+
+```ts
+import { makeCaseCache } from '../case_cache.js'; // your relative path may vary
+```
+
+2. Declare an exported global variable with the name `d`, assigned with the return value of `makeCaseCache()`:
+
+```ts
+export const d = makeCaseCache('unique/path/of/your/cache/file', {
+  // Declare any number of fields that build the test cases
+  name_of_your_case: () => {
+    return fullI32Range().map(e => { // example case builder
+      return { input: i32(e), expected: i32(-e) };
+    });
+  },
+});
+```
+
+3. To load the cases from the cache, use `d.get();`
+
+```ts
+const cases = await d.get('name_of_your_case');
+// cases will either be loaded from the cache file, loaded from the in-memory
+// LRU, or built on the fly.
+```
+
+4. Run `npx grunt run generate-cache` to generate the new cache file.

package.json

@@ -7,6 +7,7 @@
     "all": "grunt all",
     "standalone": "grunt standalone",
     "wpt": "grunt wpt",
+    "node": "grunt node",
     "checks": "grunt checks",
     "unittest": "grunt unittest",
     "typecheck": "grunt typecheck",