Mockup: Stagehand Chrome extension that links live browser to cli and library#1828
Mockup: Stagehand Chrome extension that links live browser to cli and library#1828
Conversation
Chrome extension that connects the stagehand/understudy library to running browser tabs via chrome.debugger CDP proxy. Inspired by playwriter's CDP extension architecture. - Background service worker with chrome.debugger attachment/detachment - CDP command/event multiplexing with synthetic session IDs - Native browser sidebar panel with AI chat interface - act(), observe(), extract(), and agent execute buttons - Tab activation tracking to keep sidebar in sync with foreground tab - CDPSessionLike adapter bridging chrome.debugger ↔ understudy CDP - Vite-based build producing a loadable Chrome extension in dist/ https://claude.ai/code/session_01TCsbC7eC1Fm7w1VoVPWkpR
|
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
8 issues found across 13 files
Confidence score: 2/5
- There are a few high-confidence, user-impacting risks in
packages/extension/src/sidepanel.ts: selector interpolation inRuntime.evaluatedoes not escape backslashes (can break command execution), andsendCdpCommandlacks a timeout (can leave the UI stuck in a processing state). packages/extension/src/cdp-adapter.tsis missing aport.onDisconnectpath, so MV3 worker restarts can leave in-flight CDP promises hanging indefinitely; this makes connection liveness handling fragile.- Given multiple medium-to-high severity issues (6–8/10) with strong confidence and concrete runtime impact, merge risk is elevated until lifecycle/error-handling fixes are in place.
- Pay close attention to
packages/extension/src/sidepanel.ts,packages/extension/src/cdp-adapter.ts, andpackages/extension/public/sidepanel.html- async hangs, escaping correctness, and input behavior mismatches are the main regression points.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/extension/src/cdp-adapter.ts">
<violation number="1" location="packages/extension/src/cdp-adapter.ts:66">
P1: Missing `port.onDisconnect` handler — if the MV3 service worker goes idle or restarts, all in-flight CDP promises will hang forever and the connection won't know it's dead. Register a disconnect listener in the constructor that sets `_closed = true` and rejects all pending requests.</violation>
<violation number="2" location="packages/extension/src/cdp-adapter.ts:118">
P2: `close()` doesn't clear child sessions or event handlers, leaking the entire session graph. Clear `this.sessions` and `this.eventHandlers` alongside the inflight cleanup.</violation>
</file>
<file name="packages/extension/src/background.ts">
<violation number="1" location="packages/extension/src/background.ts:85">
P2: `isRestrictedUrl` returns `false` when `url` is `undefined`, allowing a debugger attach attempt on tabs whose URL is unknown. Return `true` for undefined/empty URLs to fail fast instead of triggering a debugger error.</violation>
</file>
<file name="packages/extension/src/sidepanel.ts">
<violation number="1" location="packages/extension/src/sidepanel.ts:321">
P1: `sendCdpCommand` has no timeout — if the background worker never responds, the promise hangs indefinitely, permanently locking the UI in the processing state. Add a timeout (e.g., 30s) that rejects the promise and removes the listener.</violation>
<violation number="2" location="packages/extension/src/sidepanel.ts:364">
P1: Custom agent: **Ensure we never check against hardcoded lists of allowed LLM model names**
Hardcoded model name `"claude-sonnet-4-20250514"` should be user-configurable. Make the model selectable (e.g., via an input field or dropdown in the settings bar, stored alongside the API key) so users can specify any model without requiring a code change. The rule requires newly added code to accept any model name and let the provider reject unsupported ones.</violation>
<violation number="3" location="packages/extension/src/sidepanel.ts:425">
P1: Incomplete escaping in `Runtime.evaluate` selector interpolation — backslashes are not escaped before single quotes. A selector containing a trailing backslash (e.g. from LLM output) breaks the string literal, causing syntax errors or arbitrary JS execution in the page context. Escape backslashes first: `.replace(/\\/g, '\\\\').replace(/'/g, "\\'")`.</violation>
<violation number="4" location="packages/extension/src/sidepanel.ts:585">
P2: The click/type/scroll/navigate execution logic in `executeAgent` (~lines 583-615) is nearly identical to `executeAct` (~lines 422-478). Extract a shared helper like `executeDomAction(parsed)` to avoid maintaining duplicate code — bugs (like the selector escaping issue) must currently be fixed in both places.</violation>
</file>
<file name="packages/extension/public/sidepanel.html">
<violation number="1" location="packages/extension/public/sidepanel.html:309">
P2: This `<input type="text">` cannot support the Shift+Enter newline behavior implemented in the JS (`sidepanel.ts:261-264` checks `!e.shiftKey` before sending). Replace with a `<textarea>` so multiline input actually works, or remove the dead Shift+Enter guard from the JS.</violation>
</file>
Architecture diagram
sequenceDiagram
participant User
participant Sidebar as Side Panel UI
participant Storage as chrome.storage
participant CDP as CDP Adapter
participant SW as Background Service Worker
participant Browser as chrome.debugger (Tab)
participant AI as Anthropic API (Claude)
Note over User,AI: NEW: Stagehand Browser Extension Runtime Flow
User->>Sidebar: Select Action (Act/Observe/Extract/Agent)
Sidebar->>Storage: NEW: Get ANTHROPIC_API_KEY
Storage-->>Sidebar: API Key
rect rgb(23, 37, 84)
Note right of Sidebar: Attachment Phase
User->>Sidebar: Click "Attach to Tab"
Sidebar->>SW: NEW: Request attach (tabId)
SW->>Browser: chrome.debugger.attach()
SW->>Browser: NEW: Target.setAutoAttach (OOPIF Support)
Browser-->>SW: Attached
SW-->>Sidebar: NEW: Broadcast tab-state (attached)
end
rect rgb(5, 46, 22)
Note right of Sidebar: Execution Phase (e.g., "Act")
User->>Sidebar: Enter instruction + Submit
Sidebar->>CDP: NEW: send("Page.captureScreenshot")
CDP->>SW: NEW: Port.postMessage(CdpCommandRequest)
SW->>Browser: chrome.debugger.sendCommand()
Browser-->>SW: Base64 Screenshot
SW-->>CDP: Port.onMessage(CdpCommandResponse)
CDP-->>Sidebar: Image Data
Sidebar->>AI: NEW: POST /messages (Image + Instruction)
AI-->>Sidebar: Proposed Action (e.g., Click at X,Y)
Sidebar->>CDP: NEW: send("Input.dispatchMouseEvent")
CDP->>SW: Port.postMessage(CdpCommandRequest)
alt Synthetic Session Multiplexing
SW->>SW: NEW: Map sessionId to Tab/Frame
SW->>Browser: chrome.debugger.sendCommand(target: {tabId, sessionId})
end
Browser-->>SW: Success
SW-->>Sidebar: Action Result
end
rect rgb(69, 26, 3)
Note right of SW: Event Loop
Browser->>SW: chrome.debugger.onEvent (e.g., Page.navigated)
SW->>SW: NEW: Update internal tab-state
SW->>Sidebar: NEW: Broadcast CdpEventMessage
Sidebar->>Sidebar: Update UI logs/state
end
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
| </div> | ||
|
|
||
| <div class="input-area"> | ||
| <input |
There was a problem hiding this comment.
P2: This <input type="text"> cannot support the Shift+Enter newline behavior implemented in the JS (sidepanel.ts:261-264 checks !e.shiftKey before sending). Replace with a <textarea> so multiline input actually works, or remove the dead Shift+Enter guard from the JS.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/extension/public/sidepanel.html, line 309:
<comment>This `<input type="text">` cannot support the Shift+Enter newline behavior implemented in the JS (`sidepanel.ts:261-264` checks `!e.shiftKey` before sending). Replace with a `<textarea>` so multiline input actually works, or remove the dead Shift+Enter guard from the JS.</comment>
<file context>
@@ -0,0 +1,321 @@
+ </div>
+
+ <div class="input-area">
+ <input
+ type="text"
+ id="promptInput"
</file context>
Greptile SummaryThis PR introduces a new Key issues found:
The Confidence Score: 2/5
Important Files Changed
Sequence DiagramsequenceDiagram
participant SP as SidePanel (sidepanel.ts)
participant BG as Background SW (background.ts)
participant RS as Relay Server (/v4/extension)
participant CDP as CDP Client (/v4/cdp)
participant SH as Stagehand Server
SP->>BG: port.postMessage(attach-tab)
BG->>BG: chrome.debugger.attach(tabId)
BG-->>SP: tab-state (attached)
SP->>SH: POST /v4/sessions/start ❌ (route DNE)
Note over SP,SH: Should be POST /v4/browsersession
BG->>RS: WebSocket connect ws://host/v4/extension
RS-->>BG: connected
CDP->>RS: WebSocket connect ws://host/v4/cdp
Note over CDP,RS: No auth check
CDP->>RS: {id, method, params}
RS->>BG: {id, method:"forwardCDPCommand", params}
BG->>BG: chrome.debugger.sendCommand(tabId, method, params)
BG-->>RS: {id, result}
RS-->>CDP: {id, result}
BG->>RS: {method:"forwardCDPEvent", params:{method, sessionId, params}}
RS-->>CDP: {method, sessionId, params}
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| "dependencies": { | ||
| "@anthropic-ai/sdk": "^0.39.0" | ||
| } |
There was a problem hiding this comment.
Unused dependency @anthropic-ai/sdk
The @anthropic-ai/sdk package is listed as a dependency but is never imported anywhere in the extension code. The extension makes direct fetch calls to the Anthropic API instead (in sidepanel.ts). This adds unnecessary bundle size and install weight.
| "dependencies": { | |
| "@anthropic-ai/sdk": "^0.39.0" | |
| } | |
| "dependencies": {} |
Add comprehensive E2E tests verifying the extension's CDP proxy and stagehand library integration with a real Chromium instance: - Extension CDP proxy tests: service worker loading, navigation, screenshots, DOM modification, CDP event reception (6 tests) - Stagehand integration tests: CDP connection, navigation, DOM manipulation, screenshots, locator interaction, element creation (6 tests + 3 conditional LLM tests) - Test helpers: chrome-launcher utilities, CdpClient, local HTTP test server (for Docker environments without external network) https://claude.ai/code/session_01TCsbC7eC1Fm7w1VoVPWkpR
Previously the LLM-dependent tests were skipped without an API key. Now they always run and assert on the real API key error from the LLM layer, proving the full stagehand pipeline (CDP → page control → agent/act → LLM client) executes end-to-end. https://claude.ai/code/session_01TCsbC7eC1Fm7w1VoVPWkpR
- Add server-v4 package from server-v4-understudy-routes branch - Add WebSocket relay route (/v4/extension + /v4/cdp) to server-v4 - Begin rewriting extension background.ts to connect to server relay - Begin rewriting extension sidebar as thin REST client to server API - Work in progress: agents implementing parallel changes https://claude.ai/code/session_01TCsbC7eC1Fm7w1VoVPWkpR
…integration - Extension background.ts now connects to relay via WebSocket and proxies CDP commands through chrome.debugger, with auto-attach to active tab - Handle Target.setAutoAttach, Target.attachToTarget, Target.getTargets etc. by synthesizing Target.attachedToTarget events for stagehand compatibility - Simplify tests to rely on auto-attach instead of manual service worker debugger attachment - Delete cdp-adapter.ts (no longer needed with relay architecture) - All 11 E2E tests pass: 5 CDP proxy + 6 stagehand integration https://claude.ai/code/session_01TCsbC7eC1Fm7w1VoVPWkpR
The extension uses direct fetch calls to the server API rather than the Anthropic SDK. https://claude.ai/code/session_01TCsbC7eC1Fm7w1VoVPWkpR
- sidepanel.ts: reconnect the chrome.runtime port when the MV3 service worker restarts, preventing the UI from getting stuck in a loading state - background.ts: isRestrictedUrl now returns true for undefined URLs, preventing confusing debugger attachment errors on tabs without URLs https://claude.ai/code/session_01TCsbC7eC1Fm7w1VoVPWkpR
| async function startSession(): Promise<string> { | ||
| const cdpUrl = `ws://${serverHost}:${serverPort}/v4/cdp`; | ||
|
|
||
| const result = (await serverFetch("/v4/sessions/start", { | ||
| browser: { | ||
| type: "local", | ||
| cdpUrl, | ||
| }, | ||
| modelApiKey: apiKeyInput.value.trim(), | ||
| })) as { sessionId: string }; | ||
|
|
||
| return result.sessionId; | ||
| } |
There was a problem hiding this comment.
All session management endpoints point to non-existent routes
startSession calls POST /v4/sessions/start, but the server only exposes POST /v4/browsersession (registered in packages/server-v4/src/routes/v4/browsersession/routes.ts). The request body shape is also wrong: the server schema expects { env: "LOCAL", cdpUrl, modelName, ... } (per BrowserSessionCreateRequestSchema) but this code sends { browser: { type: "local", cdpUrl }, modelApiKey }.
endSession (line 177) calls /v4/sessions/${sessionId}/end — the actual endpoint is /v4/browsersession/${sessionId}/end.
executeAction (line 434) calls /v4/sessions/${sessionId}/act|observe|extract|agentExecute — none of these paths exist in the server routes. The act/observe/extract operations are under /v4/page/* and agent operations are not yet exposed as HTTP endpoints.
As written, every server API call from the extension will result in a 404, making the core automation functionality completely non-functional.
| cdpWss.on("connection", (ws: WebSocket) => { | ||
| cdpClients.add(ws); | ||
| log.info( | ||
| { clientCount: cdpClients.size }, | ||
| "CDP client connected to /v4/cdp", | ||
| ); | ||
|
|
||
| ws.on("message", (data: WebSocket.Data) => { | ||
| let msg: { | ||
| id?: number; | ||
| method?: string; | ||
| params?: unknown; | ||
| sessionId?: string; | ||
| }; | ||
| try { | ||
| msg = JSON.parse(data.toString()); | ||
| } catch { | ||
| log.error("Failed to parse CDP client message"); | ||
| return; | ||
| } | ||
|
|
||
| if (msg.id === undefined || !msg.method) { | ||
| log.warn("CDP client message missing id or method"); | ||
| return; | ||
| } | ||
|
|
||
| if (!extensionWs || extensionWs.readyState !== WebSocket.OPEN) { | ||
| // No extension connected — respond with error | ||
| const errorResponse = JSON.stringify({ | ||
| id: msg.id, | ||
| error: { code: -32000, message: "No extension connected" }, | ||
| }); | ||
| ws.send(errorResponse); | ||
| return; | ||
| } |
There was a problem hiding this comment.
/v4/cdp and /v4/extension endpoints are unauthenticated and exposed on all interfaces
The relay WebSocket endpoints attach to the same Fastify HTTP server that listens on 0.0.0.0 (see server.ts:336). Any host on the network — not just localhost — can connect to /v4/cdp and issue arbitrary CDP commands to the user's browser via the extension. Since the extension uses chrome.debugger with full page privileges, this is a significant remote execution vector.
Neither /v4/cdp nor /v4/extension validate any credentials (API key, origin check, etc.) before handling messages.
Consider at minimum validating the Host / Origin header to ensure connections originate from 127.0.0.1 / localhost, or reuse the existing authMiddleware pattern.
| * Used by the background service worker when responding to CDP commands. | ||
| */ | ||
| export interface CdpCommandResponse { | ||
| type: "cdp-response"; | ||
| id: number; | ||
| result?: unknown; | ||
| error?: string; | ||
| } | ||
|
|
||
| /** | ||
| * Background -> Sidebar: forwarded CDP event. | ||
| * Used by the background service worker when forwarding CDP events. | ||
| */ | ||
| export interface CdpEventMessage { | ||
| type: "cdp-event"; | ||
| tabId: number; | ||
| sessionId?: string; | ||
| method: string; | ||
| params?: unknown; | ||
| } | ||
|
|
||
| /** Sidebar -> Background: request to send a CDP command */ | ||
| export interface CdpCommandRequest { | ||
| type: "cdp-command"; | ||
| id: number; | ||
| tabId: number; | ||
| sessionId?: string; | ||
| method: string; | ||
| params?: Record<string, unknown>; | ||
| } |
There was a problem hiding this comment.
CdpCommandRequest and CdpCommandResponse are dead types
These two message types — and the "cdp-command" / "cdp-response" message protocol they describe — are never used. The background service worker's port.onMessage handler (background.ts lines 781-800) only handles "get-state", "attach-tab", and "detach-tab". No code in sidepanel.ts ever emits a "cdp-command" message.
CdpCommandRequest is still included in the SidebarMessage union (line 81), which means TypeScript sees it as a valid inbound message type even though the handler ignores it silently.
These should either be removed or implemented to avoid confusion about the intended architecture.
There was a problem hiding this comment.
15 issues found across 88 files
Confidence score: 1/5
- There are merge-blocking risks with concrete impact:
packages/server-v4/src/routes/v4/extensionRelay.tscurrently upgrades/v4/cdpand/v4/extensionWebSockets without auth/origin validation, which can expose CDP relay access to any reachable client. - Core v4 flows appear broken in their current form:
packages/extension/src/sidepanel.tscalls unregistered/v4/sessions/*routes instead of/v4/browsersessionand/v4/page/*, andpackages/server-v4/src/routes/v4/page/waitForSelector.tsreturns a non-booleanmatchedvalue that is expected to fail response validation. - There is additional high regression risk in request handling/state management (
packages/server-v4/src/routes/v4/page/shared.tsstrict schema parsing on GET params, plus WebSocket lifecycle races inpackages/extension/src/background.ts), so this is not yet safe to merge. - Pay close attention to
packages/server-v4/src/routes/v4/extensionRelay.ts,packages/extension/src/sidepanel.ts,packages/server-v4/src/routes/v4/page/waitForSelector.ts, andpackages/server-v4/src/routes/v4/page/shared.ts- security exposure and API/schema mismatches are likely to break functionality immediately.
Note: This PR contains a large number of files. cubic only reviews up to 75 files per PR, so some files may not have been reviewed.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/server-v4/src/routes/v4/browsersession/index.ts">
<violation number="1" location="packages/server-v4/src/routes/v4/browsersession/index.ts:63">
P1: Custom agent: **Ensure we never check against hardcoded lists of allowed LLM model names**
This check validates `modelName` against the hardcoded `AISDK_PROVIDERS` list and rejects unknown providers with a 400 error. Per the project rule, new code should accept any provider/model-name and let the LLM provider itself throw an error for unsupported models. Hardcoded provider lists require scattered code updates every time a new provider becomes available.
Remove the `AISDK_PROVIDERS` validation block entirely and let the downstream provider call fail naturally if the provider is invalid.</violation>
<violation number="2" location="packages/server-v4/src/routes/v4/browsersession/index.ts:98">
P2: These `error()` calls default to HTTP 400 (BAD_REQUEST), but they represent upstream Browserbase SDK failures (successful call, unexpected response), not malformed client input. Use `StatusCodes.INTERNAL_SERVER_ERROR` so clients can distinguish between "fix your request" and "something went wrong server-side".</violation>
</file>
<file name="packages/server-v4/src/routes/v4/page/waitForSelector.ts">
<violation number="1" location="packages/server-v4/src/routes/v4/page/waitForSelector.ts:33">
P1: `page.waitForSelector()` returns `ElementHandle | null`, but the result schema expects `matched` to be a `boolean`. This will fail Zod validation on every successful call. Coerce the result to a boolean.</violation>
</file>
<file name="packages/extension/public/sidepanel.html">
<violation number="1" location="packages/extension/public/sidepanel.html:290">
P2: The API key `<input>` should have `autocomplete="off"` to prevent browsers from offering to save or autofill the secret. The regular prompt input already sets this attribute, but it's missing on the more sensitive field.</violation>
</file>
<file name="packages/server-v4/src/lib/InMemorySessionStore.ts">
<violation number="1" location="packages/server-v4/src/lib/InMemorySessionStore.ts:317">
P2: Actions stored in `this.actions` and `this.browserSessionActions` are never cleaned up when a session is deleted, only when the entire store is destroyed. In a long-running server, orphaned actions for deleted sessions will accumulate indefinitely, leaking memory. Add cleanup of session-scoped actions inside `deleteSession`.</violation>
</file>
<file name="packages/server-v4/src/routes/v4/extensionRelay.ts">
<violation number="1" location="packages/server-v4/src/routes/v4/extensionRelay.ts:16">
P2: No timeout on `inflightRequests` entries — if the extension stays connected but fails to respond to a command (e.g., it silently drops it), that entry is never cleaned up. Over time this can leak memory and cause stale CDP client references to accumulate. Consider adding a per-request timeout (e.g., 30–60s) that removes the entry and sends an error response to the originating CDP client.</violation>
<violation number="2" location="packages/server-v4/src/routes/v4/extensionRelay.ts:208">
P0: Add authentication and origin/host validation before upgrading `/v4/cdp` and `/v4/extension` WebSocket requests; right now any reachable client can connect to the relay and send CDP traffic.</violation>
</file>
<file name="packages/extension/README.md">
<violation number="1" location="packages/extension/README.md:18">
P2: Node.js version prerequisite is incorrect. The project requires `^20.19.0 || >=22.12.0`, but this says 18+. Users on Node 18 will hit unsupported-engine errors.</violation>
</file>
<file name="packages/server-v4/src/routes/v4/browsersession/_id/index.ts">
<violation number="1" location="packages/server-v4/src/routes/v4/browsersession/_id/index.ts:37">
P2: Session status and availability are hardcoded to `"running"` / `true` instead of being derived from actual session state. If the session has ended or is otherwise unavailable, clients of this "Get browser session status" endpoint will receive stale/incorrect information.</violation>
</file>
<file name="packages/server-v4/src/routes/v4/browsersession/shared.ts">
<violation number="1" location="packages/server-v4/src/routes/v4/browsersession/shared.ts:167">
P2: Empty string is falsy in JS, so `toStringOrRegExp("")` silently returns `undefined` instead of `""`. If a caller passes an empty-string filter (e.g. for cookie `name`), the filter is dropped rather than applied. Use an explicit `undefined`/`null` check instead.</violation>
</file>
<file name="packages/extension/src/background.ts">
<violation number="1" location="packages/extension/src/background.ts:113">
P1: Race condition: old WebSocket's `onclose` fires after new one is created, nulling out the new connection and triggering a spurious reconnect. When `connectWebSocket` is called while a previous socket exists, the old socket's handlers remain active. Since `wsIntentionallyClosed` is reset to `false` before the old `onclose` fires asynchronously, the handler sets `ws = null` (destroying the new socket reference) and calls `scheduleReconnect()`.
Null out the old socket's event handlers before closing to prevent interference.</violation>
<violation number="2" location="packages/extension/src/background.ts:699">
P1: When the debugger is externally detached (e.g., user clicks "Cancel" on the debugger infobar, or DevTools opens), `onDebuggerDetach` cleans up local state but never notifies the relay server. The relay will have stale sessions and keep sending CDP commands that fail.
Consider replacing the body with a call to `detachTab(tabId)`, which already handles both local cleanup and relay notification. The redundant `chrome.debugger.detach()` inside `detachTab` will fail silently due to `.catch(() => {})`.</violation>
</file>
<file name="packages/server-v4/src/routes/v4/page/shared.ts">
<violation number="1" location="packages/server-v4/src/routes/v4/page/shared.ts:168">
P1: GET endpoints will fail at `actionSchema.parse()` because `sessionId` and `id` from the query string leak into `params`. The params schemas are `.strict()` and reject these unknown keys. Destructure out the routing fields before treating the remainder as params.</violation>
</file>
<file name="packages/extension/src/sidepanel.ts">
<violation number="1" location="packages/extension/src/sidepanel.ts:165">
P0: Update the extension to call the registered v4 endpoints (`/v4/browsersession` and `/v4/page/*`) instead of `/v4/sessions/*`; the current paths are not registered, so automation requests fail at the API layer.</violation>
</file>
<file name="packages/extension/src/types.ts">
<violation number="1" location="packages/extension/src/types.ts:81">
P3: Remove `CdpCommandRequest` from the `SidebarMessage` union (or implement a corresponding handler) so the message contract matches actual runtime behavior and unsupported messages are not silently accepted.</violation>
</file>
Architecture diagram
sequenceDiagram
participant UI as Extension Sidebar
participant BG as Background Service Worker
participant Srv as Server V4 (Relay + API)
participant SDK as Stagehand SDK (Local)
participant Tab as Target Browser Tab
Note over UI, Tab: NEW: Stagehand Chrome Extension Flow
rect rgb(30, 41, 59)
Note right of UI: Initialization & Attachment
UI->>BG: NEW: Connect via chrome.runtime.Port
UI->>BG: NEW: Request 'attach-tab' (tabId)
BG->>Tab: NEW: chrome.debugger.attach()
Tab-->>BG: Attached
BG->>Tab: NEW: Target.setAutoAttach (OOPIF support)
BG-->>UI: Broadcast 'tab-state' (attached)
end
rect rgb(23, 37, 84)
Note right of UI: Action Flow (Sidebar UI)
UI->>Srv: NEW: POST /v4/browsersession (start)
Srv-->>UI: sessionId
UI->>Srv: NEW: POST /v4/page/act (instruction)
Note over Srv: LLM Processing (Anthropic)
Srv->>Srv: NEW: Convert instruction to CDP commands
Srv->>BG: NEW: WS (/v4/extension) forwardCDPCommand
BG->>Tab: NEW: chrome.debugger.sendCommand(method, params)
Tab-->>BG: Command Result
BG-->>Srv: NEW: WS forwardCDPEvent / Response
Srv-->>UI: 200 OK + Action Result
end
rect rgb(5, 46, 22)
Note right of SDK: Direct SDK Integration (Remote Control)
SDK->>Srv: NEW: Connect WS to /v4/cdp
SDK->>Srv: Send Raw CDP (e.g., Page.navigate)
Srv->>BG: NEW: Relay message to extension WS
BG->>Tab: NEW: chrome.debugger.sendCommand()
Tab-->>BG: result
BG-->>Srv: Relay back to /v4/cdp
Srv-->>SDK: CDP Response
end
rect rgb(127, 29, 29)
Note right of BG: Event Forwarding
Tab->>BG: NEW: chrome.debugger.onEvent (Runtime/Page/Target)
BG->>Srv: NEW: WS forwardCDPEvent
Srv->>SDK: Broadcast to attached SDK clients
BG->>UI: Broadcast to sidebar (CDP logs/state)
end
alt Connection Error
BG->>BG: CHANGED: Exponential backoff reconnect to /v4/extension
else Restricted URL (chrome://)
BG->>UI: NEW: Error - Cannot attach to restricted tab
end
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
| extensionWss.emit("connection", ws, request); | ||
| }); | ||
| } else if (pathname === "/v4/cdp") { | ||
| cdpWss.handleUpgrade(request, socket, head, (ws) => { |
There was a problem hiding this comment.
P0: Add authentication and origin/host validation before upgrading /v4/cdp and /v4/extension WebSocket requests; right now any reachable client can connect to the relay and send CDP traffic.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/server-v4/src/routes/v4/extensionRelay.ts, line 208:
<comment>Add authentication and origin/host validation before upgrading `/v4/cdp` and `/v4/extension` WebSocket requests; right now any reachable client can connect to the relay and send CDP traffic.</comment>
<file context>
@@ -0,0 +1,217 @@
+ extensionWss.emit("connection", ws, request);
+ });
+ } else if (pathname === "/v4/cdp") {
+ cdpWss.handleUpgrade(request, socket, head, (ws) => {
+ cdpWss.emit("connection", ws, request);
+ });
</file context>
| async function startSession(): Promise<string> { | ||
| const cdpUrl = `ws://${serverHost}:${serverPort}/v4/cdp`; | ||
|
|
||
| const result = (await serverFetch("/v4/sessions/start", { |
There was a problem hiding this comment.
P0: Update the extension to call the registered v4 endpoints (/v4/browsersession and /v4/page/*) instead of /v4/sessions/*; the current paths are not registered, so automation requests fail at the API layer.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/extension/src/sidepanel.ts, line 165:
<comment>Update the extension to call the registered v4 endpoints (`/v4/browsersession` and `/v4/page/*`) instead of `/v4/sessions/*`; the current paths are not registered, so automation requests fail at the API layer.</comment>
<file context>
@@ -0,0 +1,446 @@
+async function startSession(): Promise<string> {
+ const cdpUrl = `ws://${serverHost}:${serverPort}/v4/cdp`;
+
+ const result = (await serverFetch("/v4/sessions/start", {
+ browser: {
+ type: "local",
</file context>
| StatusCodes.BAD_REQUEST, | ||
| ); | ||
| } | ||
| if (!(AISDK_PROVIDERS as readonly string[]).includes(providerName)) { |
There was a problem hiding this comment.
P1: Custom agent: Ensure we never check against hardcoded lists of allowed LLM model names
This check validates modelName against the hardcoded AISDK_PROVIDERS list and rejects unknown providers with a 400 error. Per the project rule, new code should accept any provider/model-name and let the LLM provider itself throw an error for unsupported models. Hardcoded provider lists require scattered code updates every time a new provider becomes available.
Remove the AISDK_PROVIDERS validation block entirely and let the downstream provider call fail naturally if the provider is invalid.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/server-v4/src/routes/v4/browsersession/index.ts, line 63:
<comment>This check validates `modelName` against the hardcoded `AISDK_PROVIDERS` list and rejects unknown providers with a 400 error. Per the project rule, new code should accept any provider/model-name and let the LLM provider itself throw an error for unsupported models. Hardcoded provider lists require scattered code updates every time a new provider becomes available.
Remove the `AISDK_PROVIDERS` validation block entirely and let the downstream provider call fail naturally if the provider is invalid.</comment>
<file context>
@@ -0,0 +1,244 @@
+ StatusCodes.BAD_REQUEST,
+ );
+ }
+ if (!(AISDK_PROVIDERS as readonly string[]).includes(providerName)) {
+ return error(
+ reply,
</file context>
| method: "waitForSelector", | ||
| actionSchema: PageWaitForSelectorActionSchema, | ||
| execute: async ({ page, params }) => { | ||
| const matched = await page.waitForSelector( |
There was a problem hiding this comment.
P1: page.waitForSelector() returns ElementHandle | null, but the result schema expects matched to be a boolean. This will fail Zod validation on every successful call. Coerce the result to a boolean.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/server-v4/src/routes/v4/page/waitForSelector.ts, line 33:
<comment>`page.waitForSelector()` returns `ElementHandle | null`, but the result schema expects `matched` to be a `boolean`. This will fail Zod validation on every successful call. Coerce the result to a boolean.</comment>
<file context>
@@ -0,0 +1,50 @@
+ method: "waitForSelector",
+ actionSchema: PageWaitForSelectorActionSchema,
+ execute: async ({ page, params }) => {
+ const matched = await page.waitForSelector(
+ normalizeXPath(params.selector.xpath),
+ {
</file context>
| if (ws) { | ||
| wsIntentionallyClosed = true; | ||
| ws.close(); | ||
| ws = null; | ||
| } |
There was a problem hiding this comment.
P1: Race condition: old WebSocket's onclose fires after new one is created, nulling out the new connection and triggering a spurious reconnect. When connectWebSocket is called while a previous socket exists, the old socket's handlers remain active. Since wsIntentionallyClosed is reset to false before the old onclose fires asynchronously, the handler sets ws = null (destroying the new socket reference) and calls scheduleReconnect().
Null out the old socket's event handlers before closing to prevent interference.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/extension/src/background.ts, line 113:
<comment>Race condition: old WebSocket's `onclose` fires after new one is created, nulling out the new connection and triggering a spurious reconnect. When `connectWebSocket` is called while a previous socket exists, the old socket's handlers remain active. Since `wsIntentionallyClosed` is reset to `false` before the old `onclose` fires asynchronously, the handler sets `ws = null` (destroying the new socket reference) and calls `scheduleReconnect()`.
Null out the old socket's event handlers before closing to prevent interference.</comment>
<file context>
@@ -0,0 +1,839 @@
+/** Connect (or reconnect) to the relay WebSocket */
+async function connectWebSocket(): Promise<void> {
+ // Clean up any existing connection
+ if (ws) {
+ wsIntentionallyClosed = true;
+ ws.close();
</file context>
| if (ws) { | |
| wsIntentionallyClosed = true; | |
| ws.close(); | |
| ws = null; | |
| } | |
| if (ws) { | |
| ws.onopen = null; | |
| ws.onmessage = null; | |
| ws.onclose = null; | |
| ws.onerror = null; | |
| ws.close(); | |
| ws = null; | |
| } |
|
|
||
| let extensionWs: WebSocket | null = null; | ||
| const cdpClients = new Set<WebSocket>(); | ||
| const inflightRequests = new Map<number, InflightRequest>(); |
There was a problem hiding this comment.
P2: No timeout on inflightRequests entries — if the extension stays connected but fails to respond to a command (e.g., it silently drops it), that entry is never cleaned up. Over time this can leak memory and cause stale CDP client references to accumulate. Consider adding a per-request timeout (e.g., 30–60s) that removes the entry and sends an error response to the originating CDP client.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/server-v4/src/routes/v4/extensionRelay.ts, line 16:
<comment>No timeout on `inflightRequests` entries — if the extension stays connected but fails to respond to a command (e.g., it silently drops it), that entry is never cleaned up. Over time this can leak memory and cause stale CDP client references to accumulate. Consider adding a per-request timeout (e.g., 30–60s) that removes the entry and sends an error response to the originating CDP client.</comment>
<file context>
@@ -0,0 +1,217 @@
+
+ let extensionWs: WebSocket | null = null;
+ const cdpClients = new Set<WebSocket>();
+ const inflightRequests = new Map<number, InflightRequest>();
+ let nextRelayId = 1;
+
</file context>
|
|
||
| ## Prerequisites | ||
|
|
||
| - Node.js 18+ |
There was a problem hiding this comment.
P2: Node.js version prerequisite is incorrect. The project requires ^20.19.0 || >=22.12.0, but this says 18+. Users on Node 18 will hit unsupported-engine errors.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/extension/README.md, line 18:
<comment>Node.js version prerequisite is incorrect. The project requires `^20.19.0 || >=22.12.0`, but this says 18+. Users on Node 18 will hit unsupported-engine errors.</comment>
<file context>
@@ -0,0 +1,132 @@
+
+## Prerequisites
+
+- Node.js 18+
+- pnpm
+- Google Chrome or Chromium
</file context>
| browserSession: buildBrowserSession({ | ||
| id, | ||
| params, | ||
| status: "running", |
There was a problem hiding this comment.
P2: Session status and availability are hardcoded to "running" / true instead of being derived from actual session state. If the session has ended or is otherwise unavailable, clients of this "Get browser session status" endpoint will receive stale/incorrect information.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/server-v4/src/routes/v4/browsersession/_id/index.ts, line 37:
<comment>Session status and availability are hardcoded to `"running"` / `true` instead of being derived from actual session state. If the session has ended or is otherwise unavailable, clients of this "Get browser session status" endpoint will receive stale/incorrect information.</comment>
<file context>
@@ -0,0 +1,62 @@
+ browserSession: buildBrowserSession({
+ id,
+ params,
+ status: "running",
+ available: true,
+ }),
</file context>
| flags?: string; | ||
| }, | ||
| ): string | RegExp | undefined { | ||
| if (!value) { |
There was a problem hiding this comment.
P2: Empty string is falsy in JS, so toStringOrRegExp("") silently returns undefined instead of "". If a caller passes an empty-string filter (e.g. for cookie name), the filter is dropped rather than applied. Use an explicit undefined/null check instead.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/server-v4/src/routes/v4/browsersession/shared.ts, line 167:
<comment>Empty string is falsy in JS, so `toStringOrRegExp("")` silently returns `undefined` instead of `""`. If a caller passes an empty-string filter (e.g. for cookie `name`), the filter is dropped rather than applied. Use an explicit `undefined`/`null` check instead.</comment>
<file context>
@@ -0,0 +1,354 @@
+ flags?: string;
+ },
+): string | RegExp | undefined {
+ if (!value) {
+ return undefined;
+ }
</file context>
|
|
||
| /** Any message from sidebar to background */ | ||
| export type SidebarMessage = | ||
| | CdpCommandRequest |
There was a problem hiding this comment.
P3: Remove CdpCommandRequest from the SidebarMessage union (or implement a corresponding handler) so the message contract matches actual runtime behavior and unsupported messages are not silently accepted.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/extension/src/types.ts, line 81:
<comment>Remove `CdpCommandRequest` from the `SidebarMessage` union (or implement a corresponding handler) so the message contract matches actual runtime behavior and unsupported messages are not silently accepted.</comment>
<file context>
@@ -0,0 +1,101 @@
+
+/** Any message from sidebar to background */
+export type SidebarMessage =
+ | CdpCommandRequest
+ | AttachRequest
+ | DetachRequest
</file context>
pirate
changed the title
2 participants
Summary
This PR introduces a new Chrome extension package that enables browser automation through Stagehand directly from a browser sidebar panel. The extension provides a chat-like UI for executing AI-powered actions (act, observe, extract, agent) on web pages using the Chrome DevTools Protocol (CDP) and Anthropic's Claude API.
Key Changes
New Extension Package (
packages/extension/): Complete Chrome extension implementation with:background.ts) that manages CDP attachment to tabs viachrome.debuggersidepanel.ts) with chat interface for Stagehand operationscdp-adapter.ts) that proxies CDP commands throughchrome.runtime.Porttypes.ts) for inter-process messagingSidebar Panel Features:
claude-sonnet-4-20250514Background Service Worker:
chrome.debuggerattachment lifecycle for multiple tabschrome.debugger.sendCommand()chrome.runtime.PortCDP Adapter:
CDPSessionLikeinterface compatible with Stagehand's understudyBuild & Configuration:
UI/UX:
Notable Implementation Details
sh-tab-{scope}-{id}) to multiplex CDP sessions across extension restartschrome.storage.localWorkspace Changes
packages/extensiontopnpm-workspace.yamlfor monorepo integrationhttps://claude.ai/code/session_01TCsbC7eC1Fm7w1VoVPWkpR
Summary by cubic
Adds a Chrome extension to run Stagehand in a side panel with a server-side CDP relay and v4 REST API. Also fixes MV3 port reconnects in the side panel and prevents debugger attachment on tabs without URLs.
New Features
@browserbasehq/stagehand-extension(MV3 side panel + background worker)./v4/extension↔/v4/cdp) with auto-attach, syntheticTarget.attachedToTarget, and OOPIF; removed in-extension CDP adapter.chrome.storage.local.pageandbrowsersessionroutes with zod schemas and OpenAPI; session store persists and lists actions.Dependencies
@anthropic-ai/sdkfrom@browserbasehq/stagehand-extension; the extension uses REST fetch to the v4 server.pnpm-lock.yaml; addedws/@types/wstoserver-v4and set@browserbasehq/sdkto^2.4.0.Written for commit 2824dee. Summary will update on new commits. Review in cubic