homebrew: port ncurses + bash + nethack + curl Formulae + sidecar data-tree support (kd-1mr.4, kd-1mr.6) by brandonpayton · Pull Request #815 · Automattic/kandelo

brandonpayton · 2026-07-01T06:59:15Z

Ports the four rootfs/shell packages kd-0hns left blocked (no Formula in any convoy branch) and adds sidecar-generator support for runtime data-tree packages (kd-1mr.6). Stacks on the amended #811 (kd-0hns).

Ported (4 of 4)

ncurses — Formula/ncurses.rb (10 terminal-util outputs). Bottle built+tested+bottled; sidecar homebrew-validate: ok, fork not-required, GHCR https url.
bash — Formula/bash.rb. fork required; depends_on ncurses@6.5 closure satisfied. Combined 14-package rootfs/shell tap: homebrew-validate: ok (packages=14).
nethack — Formula/nethack.rb + sidecar-generator data-tree support (kd-1mr.6): ships share/nethack/{nhdat,symbols,license} via [[outputs]].data_files (ProgramOutput.data_files in pkg_manifest.rs + a file link per entry in the generator). 15-package tap: homebrew-validate: ok (packages=15); node smoke nethack --version → NetHack Version 3.6.7.
curl — Formula/curl.rb (8.11.1). fork not-required; depends_on libcurl@8.11.1, openssl@3.3.2, zlib@1.3.1. 4-package curl-closure tap: homebrew-validate: ok (packages=4, bottles=4, link_manifests=4, provenance_reports=4); https GHCR url, sha256 ac5a1522…, ABI 16. Node smoke curl --version under NodeKernelHost → curl 8.11.1 (wasm32-unknown-none) libcurl/8.11.1 OpenSSL/3.3.2 zlib/1.3.1, exit 0.

Scope note (fork_instrumentation fix moved to #811)

An earlier revision of this PR also carried the fork_instrumentation model correction (16 package.toml reverts + set-based generator + docs). Per coordinator decision (kd-3fts, option (b)), that fix was moved to its source, #811 (kd-0hns), so #811 is self-correcting and mergeable in any order. This PR was rebased onto the amended #811 and now keeps only its own scope: the four new Formulae + their build-sandbox fixes + the data-tree generator/manifest support. It no longer touches the 12 non-kd-1mr.4 package.toml, docs/homebrew-publishing.md, or package_fork_instrumentation().

curl dependency composition (merge-ordering)

curl's dependency-root Formulae (libcurl, openssl, zlib) are owned by kd-zq4n / kd-1mr.1 on #792; #792 must merge before or with this PR for curl.rb to resolve on main (bead edge kd-1mr.4 ↔ kd-zq4n). The curl bottle reused here (ac5a1522…, faithful build of this exact curl.rb — built_from.formula_sha256 7c0889a15… matches) came from the kd-xhdd harvest (9d8347b11). Both #792 and this PR add Formula/ncurses.rb — a merge-time reconciliation for the convoy sequence (flagged to mayor).

Key fixes (this PR's scope)

Homebrew sandbox: build scripts (build-ncurses.sh, build-bash.sh, build-nethack.sh) write scratch to WASM_POSIX_DEP_WORK_DIR/_OUT_DIR instead of $SCRIPT_DIR so the sandbox + build-deps resolver can write.
sidecar data-tree support (kd-1mr.6): ProgramOutput.data_files (defaulted, rejects absolute/..) + a file link per entry for packages that ship files outside bin/.

Verification

xtask homebrew-validate green for ncurses (1), the combined bash tap (14), the nethack tap (15), and the curl closure (4). compute-cache-key-sha passes for bash/curl/ncurses/nethack on the amended base. Node smokes pass (infocmp -V, bash -c echo …, nethack --version, curl --version). See test-runs/kd-1mr.4/DISPOSITION.md, test-runs/kd-1mr.4/curl/DISPOSITION.md, and per-run outcome lists.

Publication caveat

Sidecars record canonical GHCR urls; bottle byte upload (needs packages:write GH_TOKEN + outward push) is deferred to the trusted publish flow (kd-yuef), same bar as the accepted pilot sidecars.

🤖 Generated with Claude Code

Repair the two defects that made the VFS rootfs/shell input sidecars nonpublishable: an invalid fork_instrumentation="auto" and local-only file:// bottle urls. - scripts/homebrew-generate-sidecars-from-env.sh: package_fork_instrumentation() now emits only the sidecar schema enum (not-required/required/disabled/ unknown), aggregated from per-output package.toml policies; the legacy "auto" placeholder and any unknown value can no longer reach a sidecar. - packages/registry/{16 scoped}/package.toml: declare fork_instrumentation on each output (bash/dash/git/vim required; the rest not-required), matching whether build-<name>.sh applies wasm-fork-instrument to a forking binary. - docs/homebrew-publishing.md: document the fork_instrumentation disposition values and the publishable https bottle-url requirement. - test-runs/kd-0hns: regenerated sidecars with corrected fork disposition and canonical GHCR https urls. Combined 12-package rootfs/shell tap validates: `xtask homebrew-validate` -> packages=12 bottles=12 ok. Disposition and passed/failed/skipped outcome lists cover all 16 scoped packages. Publishable (12/16): dash, git, vim, less, lsof, modeset, nano, netcat, wget, fbdoom (repaired) + bzip2, xz (adopted pilot). Blocked (4/16, missing Homebrew Formula): bash, ncurses, curl, nethack. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

github-actions · 2026-07-01T10:27:52Z

Phase B-1 matrix build status — `pr-815-staging`

ABI v16. 53 built, 15 failed, 68 total.

Package	Arch	Status	Sha
libcurl	wasm32	failed	—
libcxx	wasm32	built	`309bed52`
libcxx	wasm64	built	`97469b49`
libpng	wasm32	built	`da19660c`
libxml2	wasm32	failed	—
libxml2	wasm64	failed	—
openssl	wasm32	built	`7147c05c`
openssl	wasm64	built	`b6a3f9b1`
sqlite	wasm32	built	`29fd8435`
sqlite	wasm64	built	`5e9ff2df`
zlib	wasm32	built	`90cf6353`
zlib	wasm64	built	`64f7dfc9`
bc	wasm32	built	`de85184a`
bzip2	wasm32	built	`3b938c15`
coreutils	wasm32	built	`27ddf559`
curl	wasm32	failed	—
dash	wasm32	built	`f1cf217a`
diffutils	wasm32	built	`5f6c2432`
dinit	wasm32	built	`a31a9b89`
fbdoom	wasm32	built	`e74be0ea`
file	wasm32	built	`54db045a`
findutils	wasm32	built	`8c37f324`
gawk	wasm32	built	`28cac8a6`
git	wasm32	failed	—
grep	wasm32	failed	—
gzip	wasm32	built	`54a71b67`
hello	wasm32	built	`2c152ee8`
kandelo-sdk	wasm32	built	`27ae1cb3`
kernel	wasm32	built	`b0cf812f`
less	wasm32	built	`3ec5d016`
lsof	wasm32	built	`88e06177`
m4	wasm32	built	`b2e8d996`
make	wasm32	failed	—
mariadb	wasm32	built	`76b7be27`
mariadb	wasm64	built	`8d87af85`
modeset	wasm32	built	`d3fc05bf`
msmtpd	wasm32	built	`5dec8fca`
nano	wasm32	failed	—
ncurses	wasm32	built	`d8f95985`
netcat	wasm32	built	`ecfdca45`
nginx	wasm32	built	`11e38a46`
php	wasm32	failed	—
posix-utils-lite	wasm32	built	`7e291a98`
sed	wasm32	built	`b5c5f32b`
spidermonkey	wasm32	built	`49ef9232`
tar	wasm32	built	`b38257d8`
tcl	wasm32	built	`599fd76a`
unzip	wasm32	built	`01f2d4d4`
userspace	wasm32	built	`355492ee`
vim	wasm32	built	`4a2a54d3`
wget	wasm32	failed	—
xz	wasm32	built	`6e3c6a3b`
zip	wasm32	built	`1e0b11f2`
zstd	wasm32	built	`6a8c2f1f`
bash	wasm32	built	`c67dcfe9`
mariadb-test	wasm32	built	`3f44ecb8`
mariadb-vfs	wasm32	built	`72a4da13`
mariadb-vfs	wasm64	built	`66d31ee7`
nethack	wasm32	built	`cfa9f4e0`
node	wasm32	built	`3210297a`
spidermonkey-node	wasm32	built	`fa666ecc`
vim-browser-bundle	wasm32	built	`2b40c8df`
nethack-browser-bundle	wasm32	built	`b20aeeac`
rootfs	wasm32	failed	—
shell	wasm32	failed	—
lamp	wasm32	failed	—
node-vfs	wasm32	failed	—
wordpress	wasm32	failed	—

_{Auto-generated; replaced on each push. Raw data in the publish-status workflow artifact.}

brandonpayton · 2026-07-01T13:25:43Z

Update: nethack ported + sidecar-generator data-tree support (commit `1c9301b`)

nethack was previously blocked because it ships a runtime data tree
(share/nethack/{nhdat,symbols,license}) that the sidecar generator could not
express as link-manifest entries. This adds that capability and ports nethack
(3/4 of kd-1mr.4 now done; curl remains blocked on the libcurl/zlib/openssl
dependency-root wave).

Generator / tooling (kd-1mr.6):

ProgramOutput.data_files in pkg_manifest.rs (defaulted, relative-path
validated) lets a program declare its runtime data tree in package.toml.
homebrew-generate-sidecars-from-env.sh emits a file link per declared
data_files path (same schema the library path uses for headers/libs),
deduped + sorted, alongside the bin/ symlinks.

nethack port (kd-1mr.4):

package.toml declares the three share/nethack data files.
build-nethack.sh: sandbox-safe scratch dirs (WASM_POSIX_DEP_WORK_DIR);
reuse Homebrew's checksum-verified source (WASM_POSIX_DEP_SOURCE_DIR) rather
than re-downloading in the sandbox; serial util/dat/src make (NetHack's util
Makefile shares flex's lex.yy.c between dgn_lex.c/lev_lex.c, so the
superenv's parallel make races).
Formula/nethack.rb installs bin/nethack + the share/nethack data tree.

Verification (dev-shell, aarch64-apple-darwin): brew --build-bottle +
brew test + brew bottle (wasm32_kandelo sha f15954a0); node smoke
nethack --version → NetHack Version 3.6.7 exit 0; combined 15-package
rootfs/shell tap homebrew-validate: ok (packages=15, bottles=15, link_manifests=15, provenance_reports=15); nethack link manifest carries
bin/nethack + the 3 share/nethack file links; fork_instrumentation=not-required.
xtask: 4 new data_files unit tests pass (7 pre-existing wasm-fixture failures
are unrelated, reproduce on base).

Publication caveat (same as ncurses/bash/kd-0hns): the sidecar records the
canonical GHCR bottle url + real sha256/bytes/cache_key, but the bottle bytes
are uploaded only by the trusted publish flow (packages:write).

…d-0hns/#811) kd-0hns wrote sidecar-disposition values (required/not-required) into 16 package.toml `outputs[].fork_instrumentation`, but that field is the BUILD-policy enum {auto,disabled} (ForkInstrumentationPolicy, deny_unknown_fields). The Rust manifest parser rejects them -> `xtask compute-cache-key-sha` and `cargo xtask build-deps` fail for those packages (e.g. dash: "unknown variant `required`, expected `auto` or `disabled`"). The kd-0hns gauntlet missed it because the delivered sidecars were generated from hand-patched input JSONs, so the broken manifest-parse path was never exercised end-to-end. Fix at source (coordinator option (b), kd-3fts) so #811 is self-correcting and mergeable in any order: - Revert the invalid outputs[].fork_instrumentation lines from all 16 package.toml (bash, bzip2, curl, dash, fbdoom, git, less, lsof, modeset, nano, ncurses, netcat, nethack, vim, wget, xz) -> default `auto`. - Move the corrected set-based generator into #811: package_fork_instrumentation() now derives the sidecar disposition from FORK_INSTRUMENTED_PROGRAMS (bash/coreutils/dash/git/tcl/vim -> required; all-outputs-disabled -> disabled; else not-required), instead of misreading package.toml. - Move the docs/homebrew-publishing.md build-policy-vs-sidecar-disposition fix in. Verified: `xtask compute-cache-key-sha --arch wasm32` now succeeds on all 16 packages (was failing on the affected ones); reverting a single package's invalid value reproduces the parser error. The redundant copies of these fixes are being dropped from #815 (kd-1mr.4) so it keeps only its new Formulae. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…mr.4) Port ncurses (leaf dep of bash/nethack) to Homebrew and correct the fork_instrumentation model that kd-0hns got wrong. - homebrew/kandelo-homebrew/Formula/ncurses.rb: new Formula (10 terminal-util outputs); bottle built + tested + bottled; sidecar generated and validated (xtask homebrew-validate ok, fork=not-required, GHCR https url). - packages/registry/ncurses/build-ncurses.sh, .../bash/build-bash.sh: make the scratch build dirs (src/host-build/terminfo/wasm-build/bin) honor WASM_POSIX_DEP_WORK_DIR so the Homebrew build sandbox (and the build-deps resolver) can write to them instead of the read-only source tree. - scripts/homebrew-generate-sidecars-from-env.sh: fix package_fork_instrumentation() to derive the sidecar disposition from a FORK_INSTRUMENTED_PROGRAMS set (bash/coreutils/dash/git/tcl/vim -> required; all-outputs-disabled -> disabled; else not-required), never emitting "auto". - packages/registry/*/package.toml: revert the kd-0hns edits that wrongly wrote sidecar values into outputs[].fork_instrumentation, which is the build-policy enum {auto, disabled} the Rust manifest parser accepts (kd-0hns broke compute-cache-key-sha / build-deps for these 16 packages). - docs/homebrew-publishing.md: document the build-policy vs sidecar-disposition distinction. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…1mr.4) - homebrew/kandelo-homebrew/Formula/bash.rb: new Formula; bottle built + tested + bottled (sha d853cbef…); sidecar fork=required (via FORK_INSTRUMENTED_PROGRAMS set); GHCR https url; depends_on ncurses@6.5 (resolved through build-deps at build time, recorded in the sidecar for the closure). - test-runs/kd-1mr.4/rootfs-shell-tap: combined 14-package rootfs/shell tap (kd-0hns 12 + ncurses + bash) validates: xtask homebrew-validate ok (packages=14). bash→ncurses@6.5 dependency closure satisfied. - DISPOSITION.md + outcome-lists: 2 ported (ncurses, bash); 2 blocked — curl (deps libcurl/zlib/openssl have no Formula) and nethack (needs sidecar generator support for its runtime data tree). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…(kd-1mr.4, kd-1mr.6) Ports the nethack Homebrew Formula and teaches the sidecar generator to link a program's runtime data tree, so a program package can ship files outside bin/. kd-1mr.6 (generator / tooling): - pkg_manifest.rs: add ProgramOutput.data_files (defaulted, relative-path validated: no absolute, no `..`) so a program declares its runtime data tree in package.toml. The sidecar generator only sees package.toml, not the keg. - homebrew-generate-sidecars-from-env.sh: emit a `file` link-manifest entry for each declared data_files path (same schema the library path uses for headers/libs), deduped + sorted, alongside the bin/ symlinks. kd-1mr.4 (nethack port): - packages/registry/nethack/package.toml: declare share/nethack/{nhdat,symbols,license} as outputs[].data_files. - build-nethack.sh: honor WASM_POSIX_DEP_WORK_DIR for scratch dirs (Homebrew sandbox safe); reuse WASM_POSIX_DEP_SOURCE_DIR (Homebrew's checksum-verified unpack) instead of re-downloading inside the sandbox; build util/dat/src serially (NetHack's util Makefile shares flex's lex.yy.c between dgn_lex.c and lev_lex.c, so the superenv's parallel make races and deletes it mid-rule). - Formula/nethack.rb: build via build-nethack.sh, install bin/nethack plus the share/nethack data tree; test `nethack --version`. Verified in scripts/dev-shell.sh (aarch64-apple-darwin): brew --build-bottle + brew test + brew bottle (wasm32_kandelo sha f15954a0); node smoke `nethack --version` -> "NetHack Version 3.6.7" exit 0; combined 15-package rootfs/shell tap homebrew-validate ok (packages=15, bottles=15, link_manifests=15, provenance_reports=15); nethack link manifest carries bin/nethack + share/nethack/{nhdat,symbols,license} file links; fork_instrumentation=not-required. xtask unit tests: 4 new data_files tests pass (7 pre-existing wasm-fixture failures are unrelated and reproduce on base). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

curl 8.11.1 dependent-program Formula, completing the kd-1mr.4 rootfs/shell port set (ncurses, bash, nethack, curl). Harvested verbatim from the kd-xhdd runtime-advances preservation branch (9d8347b); sha256 7c0889a15faf... equals the built bottle's built_from.formula_sha256, so the reused wasm32 bottle (ac5a1522...fa326, 1106024 B) is a faithful build of this exact Formula. curl depends_on libcurl@8.11.1, openssl@3.3.2, zlib@1.3.1 whose dependency-root Formulae are owned by kd-zq4n / kd-1mr.1 on PR #792; #792 must merge before or with this (bead edge kd-1mr.4 -> kd-zq4n). Validated on this branch: 4-package curl-closure sidecar tap `xtask homebrew-validate: ok (packages=4, bottles=4, link_manifests=4, provenance_reports=4)`; fork_instrumentation=not-required; https GHCR url; kandelo_abi 16. Node smoke: curl --version under NodeKernelHost -> "curl 8.11.1 (wasm32-unknown-none) libcurl/8.11.1 OpenSSL/3.3.2 zlib/1.3.1", exit 0. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

brandonpayton and others added 2 commits June 30, 2026 10:37

Port interactive packages to Homebrew

566243c

brandonpayton changed the title ~~homebrew: port ncurses + bash Formulae; fix fork-instrumentation model (kd-1mr.4)~~ homebrew: port ncurses + bash + nethack Formulae + sidecar data-tree support; fix fork-instrumentation model (kd-1mr.4, kd-1mr.6) Jul 1, 2026

brandonpayton and others added 5 commits July 1, 2026 23:03

brandonpayton force-pushed the gascity/kd-1mr/kd-1mr.4-port-missing-rootfs-shell-formulae-bash-ncurses-curl-net branch from 647674b to 01a5665 Compare July 2, 2026 03:06

brandonpayton changed the title ~~homebrew: port ncurses + bash + nethack + curl Formulae + sidecar data-tree support; fix fork-instrumentation model (kd-1mr.4, kd-1mr.6)~~ homebrew: port ncurses + bash + nethack + curl Formulae + sidecar data-tree support (kd-1mr.4, kd-1mr.6) Jul 2, 2026

brandonpayton mentioned this pull request Jul 2, 2026

homebrew: repair publishable rootfs/shell sidecars (kd-0hns) #811

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

homebrew: port ncurses + bash + nethack + curl Formulae + sidecar data-tree support (kd-1mr.4, kd-1mr.6)#815

homebrew: port ncurses + bash + nethack + curl Formulae + sidecar data-tree support (kd-1mr.4, kd-1mr.6)#815
brandonpayton wants to merge 7 commits into
mainfrom
gascity/kd-1mr/kd-1mr.4-port-missing-rootfs-shell-formulae-bash-ncurses-curl-net

brandonpayton commented Jul 1, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 1, 2026 •

edited

Loading

Uh oh!

brandonpayton commented Jul 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

brandonpayton commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Ported (4 of 4)

Scope note (fork_instrumentation fix moved to #811)

curl dependency composition (merge-ordering)

Key fixes (this PR's scope)

Verification

Publication caveat

Uh oh!

github-actions Bot commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Phase B-1 matrix build status — pr-815-staging

Uh oh!

brandonpayton commented Jul 1, 2026

Update: nethack ported + sidecar-generator data-tree support (commit 1c9301b)

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

brandonpayton commented Jul 1, 2026 •

edited

Loading

github-actions Bot commented Jul 1, 2026 •

edited

Loading

Phase B-1 matrix build status — `pr-815-staging`

Update: nethack ported + sidecar-generator data-tree support (commit `1c9301b`)