Problem
When a Lab benchmark reports discovered ids: <none>, Homeboy has no supported command to run the same scenario-discovery contract on the selected runner.
The obvious diagnostic command is rejected:
homeboy bench list static-site-importer \
--path /path/to/worktree \
--placement lab \
--runner homeboy-lab
error: unexpected argument --placement found
tip: to pass --placement as a value, use -- --placement
homeboy bench list --help exposes --path, --extension, and --rig, but no --runner or --placement. Passing these after -- would send them to the workload rather than inspect discovery remotely, so the generated tip is not a valid remediation.
This forced manual runner-job log inspection while diagnosing browser verification for Automattic/blocks-engine#659.
Expected
bench list should support the same execution environment selection as bench and report what the selected runner can actually discover after materialization/hydration.
Suggested shape:
homeboy bench list static-site-importer \
--runner homeboy-lab \
--path /path/to/worktree \
--rig static-site-importer-fixture-matrix
The output should include controller and runner workload identities, extension/runtime versions, hydration state, discovered IDs, and bounded discovery diagnostics.
Acceptance criteria
bench list accepts --runner and placement policy consistently with bench.
- Local paths and path settings are materialized/remapped exactly as execution would do.
- Discovery runs against the selected runner runtime and extension set.
- The command remains read-only and does not execute benchmark workloads.
- Unsupported-option guidance does not recommend forwarding Homeboy options through workload passthrough argv.
AI assistance
- AI assistance: Yes
- Tool: OpenCode with OpenAI GPT-5.6 Sol
- Used for: Reproduced the CLI gap and drafted the operator contract with Chris.
Problem
When a Lab benchmark reports
discovered ids: <none>, Homeboy has no supported command to run the same scenario-discovery contract on the selected runner.The obvious diagnostic command is rejected:
homeboy bench list --helpexposes--path,--extension, and--rig, but no--runneror--placement. Passing these after--would send them to the workload rather than inspect discovery remotely, so the generated tip is not a valid remediation.This forced manual runner-job log inspection while diagnosing browser verification for Automattic/blocks-engine#659.
Expected
bench listshould support the same execution environment selection asbenchand report what the selected runner can actually discover after materialization/hydration.Suggested shape:
The output should include controller and runner workload identities, extension/runtime versions, hydration state, discovered IDs, and bounded discovery diagnostics.
Acceptance criteria
bench listaccepts--runnerand placement policy consistently withbench.AI assistance