fix(sdk): ship Pi's built-in tools in the default agent template (#5590) - #5597
fix(sdk): ship Pi's built-in tools in the default agent template (#5590)#5597mmabrouk wants to merge 1 commit into
Conversation
dosubot
Bot
added
the
size:XXL
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
📝 WalkthroughSummary by CodeRabbit
WalkthroughThe default Pi agent now grants ChangesDefault Pi built-in tool flow
Estimated code review effort: 4 (Complex) | ~45 minutes Possibly related PRs
Sequence Diagram(s)sequenceDiagram
participant AgentTemplate
participant ToolResolver
participant PiHarness
participant Runner
AgentTemplate->>ToolResolver: Provide read, bash, edit, write
ToolResolver->>PiHarness: Resolve builtin grant names
PiHarness->>Runner: Send serialized tools list
Runner->>Runner: Apply permission gating
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
services/oss/src/agent/config.py (1)
111-116: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winPreserve default tools when
agent.jsonomits the key.
toolsis initialized withDEFAULT_TOOLS, but an existingagent.jsonwithout atoolsfield overwrites that value with[]. This leaves partial templates with no Pi tools, while explicit"tools": []should still retain the documented grant-nothing semantics. Add coverage for this case as well.Proposed fix
meta = json.loads(meta_path.read_text(encoding="utf-8")) model = meta.get("model") or DEFAULT_MODEL - tools = meta.get("tools", []) or [] + if "tools" in meta: + tools = meta["tools"] or []
🧹 Nitpick comments (1)
web/packages/agenta-entities/tests/unit/create-ephemeral-app-from-template.test.ts (1)
58-60: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winRemove
anyfrom the new workspace-package tests.These casts can hide a broken agent-tools payload shape—the primary contract these tests are meant to protect.
web/packages/agenta-entities/tests/unit/create-ephemeral-app-from-template.test.ts#L58-L60: use a minimalunknown-based workflow/config shape and narrow before readingparameters.agent.web/packages/agenta-playground/tests/unit/agentRequest.test.ts#L362-L372: type the request parameters/tools shape and use a narrow type guard for tool names.Source: Coding guidelines
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 729092dc-5e31-4cba-af34-95f5b67f75c9
📒 Files selected for processing (36)
.agents/skills/agent-release-gate/resources/qa_probe.py.agents/skills/agent-release-gate/resources/qa_product.pyapi/oss/tests/pytest/unit/resources/test_workflow_catalog.pydocs/design/agent-workflows/documentation/agent-configuration.mddocs/design/agent-workflows/documentation/tools.mddocs/design/agent-workflows/interfaces/README.mddocs/design/agent-workflows/interfaces/in-service/harness-adapters.mddocs/design/agent-workflows/interfaces/public-edge/agent-config-schema.mddocs/design/agent-workflows/projects/default-agent-builtins/README.mddocs/design/agent-workflows/projects/default-agent-builtins/context.mddocs/design/agent-workflows/projects/default-agent-builtins/design.mddocs/design/agent-workflows/projects/default-agent-builtins/open-questions.mddocs/design/agent-workflows/projects/default-agent-builtins/plan.mddocs/design/agent-workflows/projects/default-agent-builtins/research.mddocs/design/agent-workflows/projects/default-agent-builtins/status.mddocs/design/agent-workflows/projects/default-agent-builtins/testing.mdsdks/python/agenta/sdk/agents/adapters/agenta_builtins.pysdks/python/agenta/sdk/agents/adapters/harnesses.pysdks/python/agenta/sdk/agents/pi_builtins.pysdks/python/agenta/sdk/utils/types.pysdks/python/oss/tests/pytest/unit/agents/golden/pi_default_active_builtins.jsonsdks/python/oss/tests/pytest/unit/agents/test_harness_adapters.pysdks/python/oss/tests/pytest/unit/agents/test_pi_builtins_parity.pysdks/python/oss/tests/pytest/unit/agents/test_wire_contract.pyservices/oss/src/agent/config.pyservices/oss/tests/pytest/unit/agent/test_config_template_fallback.pyservices/oss/tests/pytest/unit/agent/test_default_agent_template.pyservices/runner/config/agent.jsonservices/runner/src/engines/sandbox_agent/run-plan.tsservices/runner/tests/unit/pi-default-builtins-parity.test.tsservices/runner/tests/unit/sandbox-agent-run-plan.test.tsweb/packages/agenta-entities/tests/unit/create-ephemeral-app-from-template.test.tsweb/packages/agenta-entity-ui/src/DrillInView/SchemaControls/PiSettingsControl.tsxweb/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/itemDescriptors.tsxweb/packages/agenta-entity-ui/tests/unit/itemDescriptors.test.tsweb/packages/agenta-playground/tests/unit/agentRequest.test.ts
| | [`/inspect`](public-edge/workflow-inspect.md) | public | `agent/schemas.py`, `agent/app.py` (builtin-URI binding), `models/workflows.py`, `decorators/routing.py` | stable | `unit/agents/test_dtos_agent_config.py`, `unit/agent/test_builtin_uri_binding.py` | | ||
| | [`/messages`](public-edge/agent-messages.md) | public | `adapters/vercel/{routing,messages,stream}.py`, `agentRequest.ts` | evolving (create-or-resume not observable until storage lands) | `utils/test_messages_endpoint.py`, `unit/agents/test_ui_messages.py` | | ||
| | [Agent config schema](public-edge/agent-config-schema.md) | public | `agent/schemas.py`, `sdk/utils/types.py`, `agents/dtos.py` (`HARNESS_IDENTITIES`) | stable | `unit/agents/test_dtos_agent_config.py`, `unit/agents/test_harness_identity.py` | | ||
| | [Agent config schema](public-edge/agent-config-schema.md) | public | `agent/schemas.py`, `sdk/utils/types.py`, `agents/dtos.py` (`HARNESS_IDENTITIES`), `sdk/agents/pi_builtins.py` (`PI_DEFAULT_ACTIVE_BUILTINS`) | stable | `unit/agents/test_dtos_agent_config.py`, `unit/agents/test_harness_identity.py`, `unit/agents/test_pi_builtins_parity.py` + `golden/pi_default_active_builtins.json`, `services/oss unit/agent/test_default_agent_template.py` | |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Fix the malformed test path.
services/oss unit/agent/test_default_agent_template.py is not the repository path; it should be services/oss/tests/pytest/unit/agent/test_default_agent_template.py, otherwise this interface inventory points reviewers to a nonexistent location.
Proposed fix
-| ... | `unit/agents/test_pi_builtins_parity.py` + `golden/pi_default_active_builtins.json`, `services/oss unit/agent/test_default_agent_template.py` |
+| ... | `unit/agents/test_pi_builtins_parity.py` + `golden/pi_default_active_builtins.json`, `services/oss/tests/pytest/unit/agent/test_default_agent_template.py` |📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| | [Agent config schema](public-edge/agent-config-schema.md) | public | `agent/schemas.py`, `sdk/utils/types.py`, `agents/dtos.py` (`HARNESS_IDENTITIES`), `sdk/agents/pi_builtins.py` (`PI_DEFAULT_ACTIVE_BUILTINS`) | stable | `unit/agents/test_dtos_agent_config.py`, `unit/agents/test_harness_identity.py`, `unit/agents/test_pi_builtins_parity.py` + `golden/pi_default_active_builtins.json`, `services/oss unit/agent/test_default_agent_template.py` | | |
| | [Agent config schema](public-edge/agent-config-schema.md) | public | `agent/schemas.py`, `sdk/utils/types.py`, `agents/dtos.py` (`HARNESS_IDENTITIES`), `sdk/agents/pi_builtins.py` (`PI_DEFAULT_ACTIVE_BUILTINS`) | stable | `unit/agents/test_dtos_agent_config.py`, `unit/agents/test_harness_identity.py`, `unit/agents/test_pi_builtins_parity.py` + `golden/pi_default_active_builtins.json`, `services/oss/tests/pytest/unit/agent/test_default_agent_template.py` | |
| - **Agent template**: the saved agent configuration at `parameters.agent` of a workflow revision. | ||
| It holds instructions, model, tools, MCP servers, skills, and the execution selectors | ||
| (`harness`, `runner`, `sandbox`). |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Reconcile the execution-selector field list.
This says the saved template contains harness, runner, and sandbox, but docs/design/agent-workflows/interfaces/public-edge/agent-config-schema.md documents harness, sandbox, and permissions as the run-selection fields. Replace runner with permissions or explicitly define runner if it is intended to be persisted.
| `permission` is dropped — builtins are granted by selection, not gated.) A new agent starts | ||
| with Pi's four defaults (`read`, `bash`, `edit`, `write`), as shown above. Keep them unless the | ||
| user asks you to drop one: an empty `tools` list grants NO built-ins, so an agent that ships | ||
| with `[]` cannot read or write a file at all. |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
Clarify that runner-level permission gating still applies.
The per-tool permission field may be dropped, but built-ins are still gated by the runner permission mode: under allow_reads, read runs without approval while bash, edit, and write require approval. Replace “not gated” with wording that distinguishes selection/granting from execution permission.
A saved agent shipped `tools: []`, which the runner reads as "grant Pi no built-in tools", so the agent had no read, bash, edit or write anywhere except the playground. The playground hid it: its build kit overlay adds read and bash back on every run, and nothing else does. The default template now ships Pi's own four defaults as typed builtin entries. The runner's grant semantics are untouched, and the permission mode still gates bash, edit and write behind an approval. Also: the Claude harness only warns about built-ins that differ from that exact set, the built-in picker no longer claims an empty selection leaves Pi's defaults on, and a built-in row shows its own name. Closes #5590
6144bf9 to
4c6fd34
Compare
Railway Preview Environment
|
There was a problem hiding this comment.
Actionable comments posted: 6
🧹 Nitpick comments (2)
web/packages/agenta-playground/tests/unit/agentRequest.test.ts (1)
362-372: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAvoid introducing
anyin this workspace-package test.Use the existing request type or narrow
unknownvalues before accessingparameters.agent.tools; the newas anyand(tool: any)suppress schema regressions that this test should catch.As per coding guidelines, workspace packages should avoid
any.Source: Coding guidelines
web/packages/agenta-entities/tests/unit/create-ephemeral-app-from-template.test.ts (1)
58-60: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winKeep the test helper type-safe.
Record<string, any>andas anysuppress TypeScript checks, while optional chaining can still returnundefined. Define the small agent-config shape needed by these assertions and fail explicitly when the entity is missing.Proposed fix
-function readAgentConfig(localId: string): Record<string, any> { - const entity = getDefaultStore().get(workflowLocalServerDataAtomFamily(localId)) as any - return entity?.data?.parameters?.agent +type AgentConfig = { + harness?: {kind?: string} + llm?: {model?: string; provider?: string} + tools?: Array<{type: "builtin"; name: string}> +} + +function readAgentConfig(localId: string): AgentConfig { + const entity = getDefaultStore().get(workflowLocalServerDataAtomFamily(localId)) as + | {data?: {parameters?: {agent?: AgentConfig}}} + | undefined + const agent = entity?.data?.parameters?.agent + if (!agent) throw new Error(`Agent workflow ${localId} was not created`) + return agent }As per coding guidelines, workspace package TypeScript code should avoid
any.Source: Coding guidelines
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: a01f2878-bfdb-495e-a417-7f955d1cf5e8
📒 Files selected for processing (36)
.agents/skills/agent-release-gate/resources/qa_probe.py.agents/skills/agent-release-gate/resources/qa_product.pyapi/oss/tests/pytest/unit/resources/test_workflow_catalog.pydocs/design/agent-workflows/documentation/agent-configuration.mddocs/design/agent-workflows/documentation/tools.mddocs/design/agent-workflows/interfaces/README.mddocs/design/agent-workflows/interfaces/in-service/harness-adapters.mddocs/design/agent-workflows/interfaces/public-edge/agent-config-schema.mddocs/design/agent-workflows/projects/default-agent-builtins/README.mddocs/design/agent-workflows/projects/default-agent-builtins/context.mddocs/design/agent-workflows/projects/default-agent-builtins/design.mddocs/design/agent-workflows/projects/default-agent-builtins/open-questions.mddocs/design/agent-workflows/projects/default-agent-builtins/plan.mddocs/design/agent-workflows/projects/default-agent-builtins/research.mddocs/design/agent-workflows/projects/default-agent-builtins/status.mddocs/design/agent-workflows/projects/default-agent-builtins/testing.mdsdks/python/agenta/sdk/agents/adapters/agenta_builtins.pysdks/python/agenta/sdk/agents/adapters/harnesses.pysdks/python/agenta/sdk/agents/pi_builtins.pysdks/python/agenta/sdk/utils/types.pysdks/python/oss/tests/pytest/unit/agents/golden/pi_default_active_builtins.jsonsdks/python/oss/tests/pytest/unit/agents/test_harness_adapters.pysdks/python/oss/tests/pytest/unit/agents/test_pi_builtins_parity.pysdks/python/oss/tests/pytest/unit/agents/test_wire_contract.pyservices/oss/src/agent/config.pyservices/oss/tests/pytest/unit/agent/test_config_template_fallback.pyservices/oss/tests/pytest/unit/agent/test_default_agent_template.pyservices/runner/config/agent.jsonservices/runner/src/engines/sandbox_agent/run-plan.tsservices/runner/tests/unit/pi-default-builtins-parity.test.tsservices/runner/tests/unit/sandbox-agent-run-plan.test.tsweb/packages/agenta-entities/tests/unit/create-ephemeral-app-from-template.test.tsweb/packages/agenta-entity-ui/src/DrillInView/SchemaControls/PiSettingsControl.tsxweb/packages/agenta-entity-ui/src/DrillInView/SchemaControls/agentTemplate/itemDescriptors.tsxweb/packages/agenta-entity-ui/tests/unit/itemDescriptors.test.tsweb/packages/agenta-playground/tests/unit/agentRequest.test.ts
🚧 Files skipped from review as they are similar to previous changes (6)
- docs/design/agent-workflows/interfaces/README.md
- docs/design/agent-workflows/projects/default-agent-builtins/context.md
- docs/design/agent-workflows/documentation/agent-configuration.md
- web/packages/agenta-entity-ui/src/DrillInView/SchemaControls/PiSettingsControl.tsx
- docs/design/agent-workflows/interfaces/public-edge/agent-config-schema.md
- sdks/python/agenta/sdk/agents/adapters/agenta_builtins.py
| # Pi's default active built-ins, matching the shipped default agent template. An empty | ||
| # list means "grant nothing" to the runner, so seeding `[]` would gate the release on a | ||
| # configuration no real agent uses (issue #5590). | ||
| "tools": [ | ||
| {"type": "builtin", "name": "read"}, | ||
| {"type": "builtin", "name": "bash"}, | ||
| {"type": "builtin", "name": "edit"}, | ||
| {"type": "builtin", "name": "write"}, | ||
| ], |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
rg -n -C 4 '\bagent_template\s*\(' .agents/skills/agent-release-gate
rg -n -i -C 4 'claude|harness' .agents/skills/agent-release-gateRepository: Agenta-AI/agenta
Length of output: 42411
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== qa_probe.py relevant ranges =="
sed -n '60,135p' .agents/skills/agent-release-gate/resources/qa_probe.py | cat -n
echo
echo "== qa_product.py relevant ranges =="
sed -n '700,780p' .agents/skills/agent-release-gate/resources/qa_product.py | cat -n
sed -n '230,270p' .agents/skills/agent-release-gate/resources/qa_product.py | cat -n
echo
echo "== definitions/calls focused =="
rg -n -C 3 'def j5_commit|j5_commit|agent_template\(harness|tools=.*Pi|Pi.*tools|read_builtin|read_tool|Terminal|Bash|capabilities' .agents/skills/agent-release-gate/resources/qa_product.py .agents/skills/agent-release-gate/resources/qa_probe.py
echo
echo "== deterministic probe for tool default shape across harnesses =="
python3 - <<'PY'
import re, pathlib
qa_product = pathlib.Path(".agents/skills/agent-release-gate/resources/qa_product.py").read_text()
qa_probe = pathlib.Path(".agents/skills/agent-release-gate/resources/qa_probe.py").read_text()
def find_seed_comment_region(text, needle, start=0, end=None):
idx = text.find(needle, start, end)
if idx == -1:
return None
before = text.rfind("\n", 0, idx)
after = text.find("\n", idx)
# grab from before comment up to next def/class/main/import or end
next_block = re.search(r"\n\ndef |\n\nclass |\nif __name__", text, idx + 1)
boundary = next_block.start() if next_block else len(text)
return text[before:boundary]
for name, txt, needle in [
("qa_probe.py", qa_probe, "Pi's default active built-ins"),
("qa_product.py", qa_product, "Pi's default active built-ins"),
]:
block = find_seed_comment_region(txt, needle)
print(f"--- {name} around seed comment ---")
print(block)
print(f"contains cell[\"harness\"] in seed block? {'cell[\"harness\"]' in block}")
print(f"contains harness == in seed block? {'harness' in block}")
PYRepository: Agenta-AI/agenta
Length of output: 13909
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== j5_commit callers and config construction =="
rg -n -C 8 'template\(|run\.commit|parameters|j5_commit|cell\["harness"\]|harness.*==|startswith' .agents/skills/agent-release-gate/resources/qa_product.py
echo
echo "== call graph for j5_commit and template direct usages =="
python3 - <<'PY'
from pathlib import Path
p = Path(".agents/skills/agent-release-gate/resources/qa_product.py").read_text()
for needle in ["j5_commit(", "'j5_commit':", "template("]:
print(f"-- {needle} --")
for i, line in enumerate(p.splitlines(), 1):
if needle in line:
print(f"{i}: {line}")
PYRepository: Agenta-AI/agenta
Length of output: 13009
Gate Pi tool defaults to Pi harnesses.
The probe seed is unconditional, and j5_commit() builds the same Pi default tool list for any cell["harness"], including Claude cells. Wrap those defaults behind harness.startswith("pi") / cell["harness"].startswith("pi"), or pass non-Pi defaults for Claude.
📍 Affects 2 files
.agents/skills/agent-release-gate/resources/qa_probe.py#L82-L90(this comment).agents/skills/agent-release-gate/resources/qa_product.py#L752-L760
| ## Should `read` be approval-free on the local sandbox? | ||
|
|
||
| **Needs a decision before this ships to a shared deployment.** | ||
|
|
||
| `read` is classified read-only (`permission-plan.ts:40`) and `allow_reads` runs it without asking. | ||
| On the shipped default sandbox `local`, that is an approval-free read of any file the runner | ||
| process can open: the run executes on the runner host (`provider.ts:148`), Pi's `read` accepts | ||
| absolute paths and applies no cwd jail, and `AGENTA_RUNNER_ENABLED_SANDBOX_PROVIDERS` defaults to | ||
| `["local"]` when unset. The full trace is in | ||
| [design.md](design.md#what-read-can-reach-on-the-local-sandbox). | ||
|
|
||
| The capability exists today through the playground overlay. What this change adds is the | ||
| unattended, non-playground version of it. Three responses are available and they are not | ||
| equivalent: stop enabling `local` by default, confine built-in filesystem operations to the run | ||
| cwd in the runner, or reclassify `read` so it also asks. The first is deployment policy, the second | ||
| is runner work, the third would make the default agent ask before every file read and is probably | ||
| too blunt. | ||
|
|
||
| This is not a reason to drop `read` from the default template. It is a reason not to ship the | ||
| default template into a shared deployment that still enables `local`. |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift
Resolve the unconfined local sandbox risk before shipping.
The new default grants read, and allow_reads permits it without approval. With local enabled, an unattended Pi run can read absolute paths on the runner host because no cwd jail is applied. Exclude local from shared deployments or enforce filesystem confinement before releasing this default.
| ## Do the four built-in names now collide with an author's own tool? | ||
|
|
||
| **Known consequence of this change. Not fixed here, and worth watching.** | ||
|
|
||
| `read`, `bash`, `edit` and `write` are now occupied names for every agent created from the default | ||
| template. Built-in names and resolved tool specs share one namespace: `_validate_unique_names` | ||
| (`sdks/python/agenta/sdk/agents/tools/resolver.py:84`) walks `[*builtin_names, *tool_specs]` and | ||
| raises `DuplicateToolNameError` on the first repeat, which fails the whole run. An author-defined | ||
| client tool, gateway tool, or workflow tool called `write` was legal before this change and now | ||
| aborts the run of any agent that still carries the shipped built-ins. | ||
|
|
||
| The blast radius is small (the names are short and generic, but an author who wanted one has to | ||
| have picked exactly it) and the failure is loud rather than silent, which is why the behavior is | ||
| left alone. The options, if it does bite: namespace built-ins on the wire so the two sets cannot | ||
| collide, let an author tool of the same name shadow the built-in rather than fail, or keep failing | ||
| and say so in the picker before the run. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
Prevent built-in/custom-tool name collisions from aborting runs.
The new default occupies read, bash, edit, and write, while the resolver rejects duplicate names. A previously valid custom tool named write can therefore make a newly created agent fail at runtime. Add save-time validation, namespacing, or an explicit precedence rule before shipping.
| The read-only table at `services/runner/src/permission-plan.ts:40` marks `read`, `grep`, `find`, | ||
| and `ls` read-only, and `bash`, `edit`, `write` not read-only. Under the shipped default | ||
| permission mode `allow_reads` (`sdks/python/agenta/sdk/utils/types.py:1072`), the read-only four | ||
| run without asking and the other three raise an approval. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Clarify that allow_reads only permits granted read-only tools.
The default grant contains read, not grep, find, or ls; those tools are classified as read-only but are inactive unless granted. Reword this to avoid implying that all four execute by default.
| **State: implemented, uncommitted. All four pieces are in the working tree; no branch and no pull | ||
| request yet.** | ||
|
|
||
| Last updated 2026-07-30, after implementation. |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Update the implementation status before merging.
This still says the work is uncommitted and has no branch or pull request, which is stale in the current PR. Replace it with the actual implementation/PR state and update the timestamp as needed.
| ```python | ||
| def test_published_default_grants_pi_default_builtins(): | ||
| """A new agent must be able to read, run shell commands, and edit and write files wherever it | ||
| runs, not only in the playground (issue #5590). The runner reads an empty tools list as | ||
| "grant nothing".""" | ||
| expected = [{"type": "builtin", "name": name} for name in PI_DEFAULT_BUILTIN_NAMES] | ||
| assert _inspect_agent_default()["tools"] == expected | ||
| assert _builtin_agent_default()["tools"] == expected | ||
| ``` |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Use the actual constant name in the test example.
PI_DEFAULT_BUILTIN_NAMES does not exist; the shipped symbol is PI_DEFAULT_ACTIVE_BUILTINS. Update the snippet so readers do not copy a failing import or assertion.
1 participant
Context
A saved agent could not run a shell command or write a file anywhere except the playground. Asked to do either, it answered that it had no shell or filesystem tool available.
Pi keeps seven tools inside the harness, and the run request names which of them to switch on. The runner treats a missing
toolsfield as "use Pi's defaults" and an empty array as "grant nothing", and the Pi extension then deletes every non-granted tool from the model's list. The default agent template shipped an empty array, and the Python side always sends the field, so every saved agent said "grant nothing".The playground hid this. Its build kit overlay adds
readandbashback on every run, and nothing else does. So an agent worked while you built it and failed the moment a schedule ran it.Changes
build_agent_v0_default()now ships Pi's own four defaults.Before:
"tools": []After:
The runner's rules are untouched. An empty list still means grant nothing, and the permission mode still gates
bash,editandwritebehind an approval. Having a tool is not the same as being allowed to use it.Three supporting fixes ship with it:
The Claude harness warns only about built-ins outside that exact set. Built-ins are a Pi idea and Claude drops them, so without this every Claude run would log a warning about the four tools the template now always carries.
The built-in tool picker no longer tells you that clearing your selection leaves Pi's defaults on. It does the opposite.
The release gate seeds no longer hand-write the empty list, so the gate stops reporting green on the exact shape this change replaces.
Scope is deliberately new agents only. Agents saved before this keep their empty list and stay broken outside the playground. On one development deployment that is 66 of 73 Pi agents, so it is most of them. Repairing them is its own decision, filed as #5595 with the counting query.
Before merging
Do not deploy this to a shared deployment that still enables the
localsandbox. The default sandbox islocal, which runs on the runner host with no jail, andreadnever asks under the default permission mode. So a new agent could read host files unattended.This adds no capability an author did not already have. Both the tool list and the sandbox are author-writable today, so anyone could already save the same agent and schedule it. What changes is what an unwitting author gets by default. Confirm
AGENTA_RUNNER_ENABLED_SANDBOX_PROVIDERSexcludeslocalbefore deploying, and remember that leaving it unset meanslocalis enabled.Tests
A crossing test now runs the real chain from the shipped default through the harness adapter to the
/runbody and asserts the four names arrive. Neither existing suite crossed that boundary, which is why both stayed green while every agent shipped with no tools.The four names are pinned by a shared golden fixture that the Python and TypeScript suites both read, so the two languages cannot drift apart silently.
Runner 1308 passing, SDK 2330, API 1495, and 1427 across three web packages. Every error in the Python suites is an acceptance test asking for a live stack.
Verified on a running deployment: a fresh account's
/inspectreturns the four built-in entries as the default a new agent pre-fills with.Reviewed by Codex at high effort before implementation, and by two independent reviewers after. Their findings and what was accepted or rejected are recorded in
docs/design/agent-workflows/projects/default-agent-builtins/status.md.What to QA
Closes #5590