Adjust native PHP vs Playground agent expectations#3995
Conversation
📊 Performance Test ResultsComparing ddba485 vs trunk app-size
site-editor
site-startup
Results are median values from multiple test runs. Legend: 🟢 Improvement (faster) | 🔴 Regression (slower) | ⚪ No change (<50ms diff) |
…re-playground-prompt-v2
There was a problem hiding this comment.
Pull request overview
This PR updates Studio’s agent instruction pipeline so runtime-specific guidance (native PHP vs WordPress Playground/PHP WASM sandbox) is delivered correctly for each site, both in installed markdown instruction files and in the CLI agent system prompt.
Changes:
- Added runtime-aware rendering for bundled instruction markdown (
<!-- IF <runtime> -->blocks) when installing/updating site instruction files and skills. - Updated bundled instruction content (
skills/STUDIO.md) to include runtime-conditional guidance and madestudio-cliskill text runtime-neutral. - Adjusted CLI AI system prompt to provide runtime-specific
wp_cli --post_contentguidance, resolving the active site runtime at session creation.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/common/lib/agent-skills.ts | Adds runtime-conditional marker rendering and threads runtime through install/update paths. |
| tools/common/lib/tests/agent-skills.test.ts | Updates tests for read→render→write behavior and adds coverage for conditional marker rendering. |
| skills/STUDIO.md | Introduces <!-- IF playground --> / <!-- IF native-php --> blocks to tailor guidance per runtime. |
| skills/studio-cli/SKILL.md | Removes Playground-only phrasing so the skill applies to both runtimes. |
| apps/studio/src/modules/agent-instructions/lib/skills.ts | Threads runtime through skill-install APIs in the desktop app. |
| apps/studio/src/ipc-handlers.ts | Passes resolved site runtime when installing skills and updating managed instruction files on server start. |
| apps/cli/commands/site/start.ts | Passes site runtime into managed instruction updates during CLI site start. |
| apps/cli/commands/site/create.ts | Passes site runtime into initial AI instruction install during CLI site creation. |
| apps/cli/ai/system-prompt.ts | Adds runtime-specific post content guidance (inline for Playground; scratch file allowed for native PHP). |
| apps/cli/ai/tests/system-prompt.test.ts | Adds tests asserting correct prompt guidance for each runtime and default behavior. |
| apps/cli/ai/runtimes/pi/index.ts | Resolves active local site runtime from CLI config and supplies it to system prompt creation. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| * Loose `.md` files are rendered for the site's runtime (see | ||
| * `renderRuntimeInstructions`) so runtime-specific guidance is dropped when it | ||
| * doesn't apply. |
|
@fredrikekelund I addressed the comments, it is ready for another review |
Checked, but I don't see it inside UPDATE: oh, I see, there is important step missed to do |
Thanks for the review @nightnei ! I will update the testing instructions 👍 Do you think we are good to merge? |
| // content (inline args can hit the OS command-length limit). | ||
| function getPostContentGuidance( runtime?: SiteRuntime ): string { | ||
| const shared = | ||
| 'The `wp_cli` tool takes literal arguments, not shell commands — never use shell substitution or shell syntax such as `$(cat file)`, backticks, pipes, redirection, or environment variables to provide post content.'; |
There was a problem hiding this comment.
Is this needed for the php runtime?
There was a problem hiding this comment.
From my testing, it seems to be needed: The wp_cli tool uses spawn() without a shell on both runtimes, so shell syntax like $(cat file) is passed as literal text. I verified this by creating a post with $(echo INJECTED) as content on a native PHP site and it was stored verbatim, not executed.
Related issues
Fixes STU-1840
How AI was used in this PR
It was used to plan and implement the feature.
Proposed Changes
This PR ensures that when the site is using the native PHP runtime, the agents are using the instructions that are tailored to it. When a site is using a Playground sandbox, the agents are given the correspoding instructions.
Testing Instructions
npm installnpm run cli:buildnode apps/cli/dist/cli/main.mjs site create --name test-native --runtime nativenode apps/cli/dist/cli/main.mjs site create --name test-sandbox --runtime sandboxopen ~/Studio/test-native/STUDIO.mdin your terminal and confirm that you can see references such asCore edits are overwritten on every WordPress update, so extend through hooks insteadopen ~/Studio/test-sandbox/STUDIO.mdand confirm the references are correct for PlaygroundPre-merge Checklist