Skip to content

AI Agents framework, Phase A: synthetic users + user-meta definitions, abilities runner, chat - #428

Draft
epeicher wants to merge 7 commits into
trunkfrom
add/agents-phase-a
Draft

AI Agents framework, Phase A: synthetic users + user-meta definitions, abilities runner, chat#428
epeicher wants to merge 7 commits into
trunkfrom
add/agents-phase-a

Conversation

@epeicher

@epeicher epeicher commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

What it does

Ships Phase A of the AI Agents framework: durable workers that live on the site as real WordPress users, take orders by chat, and act through the WordPress Abilities API under their own role and capabilities.

Behind the new agents extended option (OS Settings → Features → Extended options, admin-only, default off). With the flag on, a new Agents section appears in My WordPress: create an agent (name, role, description, system prompt), tick the abilities it may use in the Tools pane, configure triggers, and talk to it in the Agent chat window. Every tool call runs as the agent's own user and lands in the standard audit trail (revisions, comments, _edit_lock) attributed to the agent.

Rationale

This is the buildable slice of the direction #240 mocked, with one deliberate storage change: no wp_guideline CPT. An agent is split across exactly two layers:

  • Identity: a synthetic wp_users row. Real role, real capability checks, real attribution. Every login path is blocked (authenticate filter, password resets, application passwords), the email is a never-delivered synthetic address, and the wp-admin Users list labels the row "Agent".
  • Definition: user meta on that row (_desktop_mode_agent_*): description, instructions, ability allowlist, triggers, model override, rate limit.

Dropping the guideline CPT removes the Gutenberg Guidelines-experiment dependency entirely (no soft-gate, no 412 paths). The accepted trade-off is losing free revisions on prompt edits; instead, every mutation fires desktop_mode_agent_{created,updated,deleted} with before/after values per changed field, so logging plugins get a complete audit trail.

Implementation

Runner (includes/agents/runner.php): desktop_mode_agent_invoke() generates through the same Core AI Client adapter the Copilot uses (desktop_mode_ai_client_generate() over wp_ai_client_prompt()), loops function calls to a hard 8-turn cap, and dispatches each call via WP_Ability::execute() so the ability's own permission_callback and schemas gate it. The whole loop runs with wp_set_current_user() switched to the agent, restored in finally:

$previous_user_id = get_current_user_id();
wp_set_current_user( $user->ID );
try {
    $result = desktop_mode_agent_runner_loop( ... );
} finally {
    wp_set_current_user( $previous_user_id );
}

Tool schemas advertised to the model are projected through the Copilot's desktop_mode_ai_normalize_tool_schema() (providers 400 the whole request over one tool with a top-level oneOf/anyOf/allOf). Per-agent hourly rate limits ride a transient counter (default 60/hour, filterable). A desktop_mode_agent_runner_generate pre-filter short-circuits the AI Client, which is how PHPUnit exercises the full loop network-free.

Tools: the picker is a view over wp_get_abilities() with honest read-only vs mutating badges from meta.annotations.readonly. Unlike the Copilot (read-only only), agents may be granted mutating abilities; the compensating controls are the explicit allowlist set by an edit_users human plus the agent's role. Three new abilities ship: desktop-mode/get-post and desktop-mode/get-media (read-only) and desktop-mode/update-post (mutating, reachable only through an agent allowlist).

REST (/desktop-mode/v1/agents): CRUD + /invoke + abilities/trigger-kinds/hooks/roles catalogues. Reads and invokes default to edit_posts, writes to edit_users (all filterable); role assignment is constrained to a whitelist intersected with the acting user's get_editable_roles().

Client: an agent entity kind registered through the existing registerEntityKind() seam in the My WordPress bundle (list, Define/Tools/Triggers panes, create flow, live provider probe against /ai/status), plus a new lazy agent-run-window bundle for the chat window, fed through the cross-bundle desktop-mode/agents-chat shared store.

Triggers: chat is the only wired intake in Phase A. The other kinds (send-to, drag, hook, endpoint, agent) are declared in the trigger-kind catalogue so configuration can be stored now; their intakes land in later phases. All of them collapse to the same desktop_mode_agent_invoke() engine, and desktop_mode_agent_completed is the chaining seam.

First consumer — Remove Background extension (extensions/desktop-mode-remove-background/): a standalone plugin registering media-tools/remove-background, a mutating ability that removes an image's background and sideloads the result as a new PNG attachment authored by the agent. It touches zero desktop-mode internals — its whole integration surface is wp_register_ability() — which is the framework's compatibility claim demonstrated inside the PR. Backends are pluggable; the default is the WordPress AI Client (generative editing riding the site's Connectors credentials — no extension-specific key), with remove.bg and self-hosted rembg as opt-in mask-based alternatives. Configuration is deliberately code-level only (option, constants, or filter — no admin settings UI), so the ability is the extension's entire user-facing surface.

Testing instructions

npm run test:php -- --filter='Tests_DesktopMode_Agents'   # 45 tests
npm run test:js                                            # includes 20 agents tests

Manual, on a WP 7.0+ site with an AI connector configured:

  1. OS Settings → Features → Extended options → tick Enable AI agents, reload.
  2. My WordPress → Agents → create an agent (role author, prompt like "You audit posts"). Tick desktop-mode/get-post + desktop-mode/update-post in Tools.
  3. Press Chat, ask "Fetch post 1 and tighten its title". Expect an answer with an expandable tool-call trace, and the post's revision attributed to the agent user.
  4. Verify the blocks: the agent appears in wp-admin Users with the bot avatar and "Agent" type, logging in as it fails, and no application passwords can be created for it.
  5. Without a connector, the section shows a warning notice pointing at Settings → Connectors and Chat is disabled; definitions remain editable.
Open WordPress Playground Preview

epeicher added 7 commits July 27, 2026 19:43
Agents are durable workers that live on the site as real WordPress
users and act through the Abilities API under their own role. An agent
is a login-blocked wp_users row plus a _desktop_mode_agent_* user-meta
family holding the whole definition: description, instructions
(system prompt), ability allowlist, triggers, model override, rate
limit. No wp_guideline dependency; the audit trail for definition
changes is the desktop_mode_agent_{created,updated,deleted} actions,
each carrying before/after values.

Behind the new 'agents' extended option (default off, games-style
module gating). Phase A ships:

- includes/agents/: store (meta CRUD + orchestrators), identity
  (login/password-reset/app-password blocks, bot avatar, Users list
  column), abilities bridge (desktop-mode/get-post + update-post,
  picker catalogue with readonly badges), runner on the Core AI
  Client (8-turn cap, per-agent hourly rate limit, identity switch in
  try/finally, provider-safe tool-schema normalization shared with
  the Copilot), REST CRUD + /invoke + catalogues, privacy
  exporter/eraser, Agent chat native window, My WordPress entity.
- Client: 'agent' entity-kind renderer in the My WordPress bundle
  (list, Define/Tools/Triggers panes, create flow, live AI-provider
  probe) and a new lazy agent-run-window bundle fed through the
  cross-bundle desktop-mode/agents-chat shared store.
- 45 PHPUnit + 20 vitest tests; docs (hooks-reference,
  javascript-reference, api-index, architecture, rest README,
  examples/agents.md) and the implementation plan under docs/plans/.

Chat is the only wired trigger; send-to/drag, hook, endpoint, and
agent-to-agent intakes are declared in the trigger-kind catalogue and
land in later phases.
Two Gemini-surfaced fixes in the agent invocation path:

- Strip the WordPress-only arg-schema keys (sanitize_callback,
  validate_callback, arg_options) from tool schemas at every depth, in
  the shared desktop_mode_ai_normalize_tool_schema() so the Copilot
  benefits too. Strict providers reject any unknown field and 400 the
  whole request over one property. The walk is structure-aware:
  property NAMES are never stripped, only schema-level keys; covers
  properties, patternProperties, single and tuple items, array-shaped
  additionalProperties, and nested combinators.

- Stop replaying assistant functionCall turns to the provider. Each
  generate turn now sends one user message: the original request plus
  a text transcript of executed tool calls and their results.
  Replaying functionCall parts requires provider-specific signatures
  (Gemini thought_signature, Anthropic thinking signatures) that the
  current provider plugins do not round-trip, and one missing
  signature 400s the request. Text transcripts carry the same
  information with no signature or pairing constraints on any
  provider.
Agents (and the Copilot, via the readonly annotation) can now read a
media library item by id: file URL, mime type, dimensions, alt text,
caption, and the post it is attached to. Closes the gap where no
ability on a stock site could read attachments, so image-referencing
prompts dead-ended.

Permission gates on upload_files (author+), deliberately not on
read_post: for inherit-status attachments that check defers to the
parent (and effectively requires edit rights when unattached), which
wrongly blocks read-only access to media whose file URL is public on
a standard site anyway.
…ity)

New standalone extension plugin under extensions/, riding the agents
PR as the first real consumer of the framework: an ability that
removes the background from a media library image and sideloads the
result as a NEW png attachment (original untouched), authored by the
calling user — for agents, the agent's own account, completing the
attribution story end to end.

Mutating ability (no readonly annotation): invisible to the Copilot,
reachable only through an agent's explicit allowlist. Pluggable
backends behind desktop_mode_remove_background_backends: remove.bg
(default, key required), a self-hosted rembg server, and an
experimental WordPress AI Client generative-editing backend that
reuses the site's configured connectors. Settings live natively on
Settings -> Media. A desktop_mode_remove_background_pre short-circuit
filter keeps PHPUnit network-free; seven tests cover registration,
the execute lifecycle, permissions, and runner dispatch with
agent attribution.
The extension's only user-facing surface is the ability itself — no
Settings -> Media section. Configuration is code/CLI-level, resolved
option -> constants (DESKTOP_MODE_REMOVE_BG_{BACKEND,API_KEY,ENDPOINT})
-> desktop_mode_remove_background_settings filter, with an unknown
backend slug falling back to the default. README documents all three
paths; error messages point at them instead of the removed screen.
The ai backend rides the site's existing Connectors credentials, so a
stock install needs no extension-specific key at all — remove.bg and
rembg become the opt-in paths for mask-based quality. The AI Client
resolves the model from the prompt's modalities (an input image
requires an image-input-capable model), so providers that only do
text-to-image are never silently picked for an edit.
wp_ai_client_prompt() returns WordPress's snake_case wrapper
(WP_AI_Client_Prompt_Builder), not the SDK PromptBuilder: generating
methods are generate_* and failures come back as WP_Error instead of
exceptions. The camelCase generateImageResult() call fell through the
wrapper's fluent path and returned the builder itself, which the
backend then reported as 'no readable image data' regardless of what
the provider said. Now calls generate_image_result(), propagates
WP_Error verbatim (so provider errors like quota exhaustion surface
readably in the agent's tool trace), and keeps the try/catch for
result-shape surprises. Regression test drives the real builder path
with no connector configured.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant