Skip to content

Adjust native PHP vs Playground agent expectations#3995

Merged
katinthehatsite merged 10 commits into
trunkfrom
runtime-aware-playground-prompt-v2
Jul 3, 2026
Merged

Adjust native PHP vs Playground agent expectations#3995
katinthehatsite merged 10 commits into
trunkfrom
runtime-aware-playground-prompt-v2

Conversation

@katinthehatsite

@katinthehatsite katinthehatsite commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Related issues

Fixes STU-1840

How AI was used in this PR

It was used to plan and implement the feature.

Proposed Changes

This PR ensures that when the site is using the native PHP runtime, the agents are using the instructions that are tailored to it. When a site is using a Playground sandbox, the agents are given the correspoding instructions.

Testing Instructions

  • Pull the changes from this branch
  • Run npm install
  • Rebuild CLI with npm run cli:build
  • Create a site with native PHP by running:

node apps/cli/dist/cli/main.mjs site create --name test-native --runtime native

  • Create a site with Playground sandbox:

node apps/cli/dist/cli/main.mjs site create --name test-sandbox --runtime sandbox

  • Once done, run open ~/Studio/test-native/STUDIO.md in your terminal and confirm that you can see references such as Core edits are overwritten on every WordPress update, so extend through hooks instead
  • Repeat the same step for open ~/Studio/test-sandbox/STUDIO.md and confirm the references are correct for Playground

Pre-merge Checklist

  • Have you checked for TypeScript, React or other console errors?

@katinthehatsite katinthehatsite self-assigned this Jun 29, 2026
@katinthehatsite katinthehatsite marked this pull request as draft June 29, 2026 19:09
@wpmobilebot

wpmobilebot commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator

📊 Performance Test Results

Comparing ddba485 vs trunk

app-size

Metric trunk ddba485 Diff Change
App Size (Mac) 1317.36 MB 1317.37 MB +0.01 MB ⚪ 0.0%

site-editor

Metric trunk ddba485 Diff Change
load 1117 ms 748 ms 369 ms 🟢 -33.0%

site-startup

Metric trunk ddba485 Diff Change
siteCreation 6503 ms 6506 ms +3 ms ⚪ 0.0%
siteStartup 1850 ms 1861 ms +11 ms ⚪ 0.0%

Results are median values from multiple test runs.

Legend: 🟢 Improvement (faster) | 🔴 Regression (slower) | ⚪ No change (<50ms diff)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates Studio’s agent instruction pipeline so runtime-specific guidance (native PHP vs WordPress Playground/PHP WASM sandbox) is delivered correctly for each site, both in installed markdown instruction files and in the CLI agent system prompt.

Changes:

  • Added runtime-aware rendering for bundled instruction markdown (<!-- IF <runtime> --> blocks) when installing/updating site instruction files and skills.
  • Updated bundled instruction content (skills/STUDIO.md) to include runtime-conditional guidance and made studio-cli skill text runtime-neutral.
  • Adjusted CLI AI system prompt to provide runtime-specific wp_cli --post_content guidance, resolving the active site runtime at session creation.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tools/common/lib/agent-skills.ts Adds runtime-conditional marker rendering and threads runtime through install/update paths.
tools/common/lib/tests/agent-skills.test.ts Updates tests for read→render→write behavior and adds coverage for conditional marker rendering.
skills/STUDIO.md Introduces <!-- IF playground --> / <!-- IF native-php --> blocks to tailor guidance per runtime.
skills/studio-cli/SKILL.md Removes Playground-only phrasing so the skill applies to both runtimes.
apps/studio/src/modules/agent-instructions/lib/skills.ts Threads runtime through skill-install APIs in the desktop app.
apps/studio/src/ipc-handlers.ts Passes resolved site runtime when installing skills and updating managed instruction files on server start.
apps/cli/commands/site/start.ts Passes site runtime into managed instruction updates during CLI site start.
apps/cli/commands/site/create.ts Passes site runtime into initial AI instruction install during CLI site creation.
apps/cli/ai/system-prompt.ts Adds runtime-specific post content guidance (inline for Playground; scratch file allowed for native PHP).
apps/cli/ai/tests/system-prompt.test.ts Adds tests asserting correct prompt guidance for each runtime and default behavior.
apps/cli/ai/runtimes/pi/index.ts Resolves active local site runtime from CLI config and supplies it to system prompt creation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +56 to +58
* Loose `.md` files are rendered for the site's runtime (see
* `renderRuntimeInstructions`) so runtime-specific guidance is dropped when it
* doesn't apply.
Comment thread apps/studio/src/ipc-handlers.ts Outdated
@katinthehatsite katinthehatsite requested review from a team and nightnei July 1, 2026 09:29
@katinthehatsite katinthehatsite marked this pull request as ready for review July 1, 2026 09:29
Comment thread apps/cli/ai/runtimes/pi/index.ts Outdated
Comment thread packages/common/lib/agent-skills.ts Outdated
@katinthehatsite

Copy link
Copy Markdown
Contributor Author

@fredrikekelund I addressed the comments, it is ready for another review

@nightnei

nightnei commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Once done, run open ~/Studio/test-native/STUDIO.md in your terminal and confirm that you can see references such as "Core edits are overwritten on every WordPress update, so extend through hooks instead"

Checked, but I don't see it inside test-native/STUDIO.md.

UPDATE: oh, I see, there is important step missed to do npm install, otherwise it bundles the stale STUDIO.md
Works well 👍

@katinthehatsite

Copy link
Copy Markdown
Contributor Author

UPDATE: oh, I see, there is important step missed to do npm install, otherwise it bundles the stale STUDIO.md

Thanks for the review @nightnei ! I will update the testing instructions 👍

Do you think we are good to merge?

@katinthehatsite katinthehatsite merged commit 2520218 into trunk Jul 3, 2026
11 checks passed
@katinthehatsite katinthehatsite deleted the runtime-aware-playground-prompt-v2 branch July 3, 2026 09:39
// content (inline args can hit the OS command-length limit).
function getPostContentGuidance( runtime?: SiteRuntime ): string {
const shared =
'The `wp_cli` tool takes literal arguments, not shell commands — never use shell substitution or shell syntax such as `$(cat file)`, backticks, pipes, redirection, or environment variables to provide post content.';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this needed for the php runtime?

@katinthehatsite katinthehatsite Jul 3, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my testing, it seems to be needed: The wp_cli tool uses spawn() without a shell on both runtimes, so shell syntax like $(cat file) is passed as literal text. I verified this by creating a post with $(echo INJECTED) as content on a native PHP site and it was stored verbatim, not executed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants