Add fix for page.url() race when navigation happens as a result of act()

[pirate]

why

• Currently if an act() call results in a navigation and we immediately call page.url() or .title() after we may still get the old page, causing the agent to think it needs to navigate again or making it think it's on the wrong page.

what changed

• Add more in-depth logic to detect if navigation is triggered by JS within 400ms of any act() call, and wait until the nav settles before returning so that subsequent steps see the next page instead of the old one.

Important

ALTERNATIVE IDEA (which I prefer) would be add an arg to act({..., expectNavigation: true | false}) and only wait for nav if true is passed.

test plan

• Adds a test for the specific scenarios

---

Summary by cubic

Fixes a race where page.url()/page.title() could return the old page right after an act()-triggered navigation. We now detect main-frame navigation that starts within 400ms of a click/doubleClick and wait for it to settle before returning.

• Bug Fixes
  • In performUnderstudyMethod, arm a main-frame nav watcher before click/doubleClick; if it reports a start within 400ms after the click returns, wait for waitForDomNetworkQuiet.
  • Added watchMainFrameUrlChangeStart(page, initialUrl) to listen for Page.frameNavigated/Page.navigatedWithinDocument only on real URL changes, handle main-frame ID changes, and dispose cleanly.
  • New test perform-understudy-navigation-race.spec.ts verifies wait for <400ms and no wait for >400ms, using live https://example.com URLs.

^{Written for commit a6bb6ef. Summary will update on new commits. Review in cubic}

[greptile-apps]

Greptile Summary

This PR fixes a race condition where calling page.url() or page.title() immediately after an act() click could return stale data because the JS-triggered navigation hadn't committed yet. It introduces a watchMainFrameUrlChangeStart helper that attaches CDP Page.frameNavigated / Page.navigatedWithinDocument listeners before the click executes, then calls wait(400) after the click to detect whether a navigation started within a 400ms window — and if so, waits for the page to fully settle before returning.

Key changes:

• watchMainFrameUrlChangeStart — new function that monitors the main frame for URL changes using raw CDP session events; includes an idempotent dispose(), a matched flag for pre-wait() navigations, and dual frame-ID checking for mid-navigation frame ID changes.
• shouldWaitForPostClickSettle — gates the watcher to click and doubleClick methods only.
• performUnderstudyMethod — creates and disposes the watcher around every click/doubleClick action, then conditionally awaits waitForDomNetworkQuiet.
• New integration test — validates both the "wait" path (250ms nav) and the "no wait" path (650ms nav).

Notable concern: Every click and doubleClick now incurs an unconditional up-to-400ms idle wait even when no navigation occurs, which may add meaningful latency to non-navigating click-heavy workflows. The timing margin in the second test (250ms gap between the 650ms fixture and the 400ms window) may also be too narrow for reliable CI execution on loaded runners.

Confidence Score: 3/5

• Safe to merge as a correctness fix, but the 400ms per-click latency regression warrants review before landing on a high-throughput path.
• The race-condition fix is logically sound and well-tested, but every click/doubleClick now unconditionally waits up to 400ms even when no navigation occurs — a non-trivial performance regression for click-heavy workflows. The test timing margin (250ms) may also be too narrow for reliable CI execution. These issues lower confidence despite the fix being directionally correct.
• Pay close attention to actHandlerUtils.ts — specifically the navWatcher.wait(400) call that adds latency to all click actions regardless of whether navigation occurs.

Important Files Changed

Filename	Overview
packages/core/lib/v3/handlers/handlerUtils/actHandlerUtils.ts	Adds post-click navigation detection via CDP session events; every click/doubleClick now waits up to 400ms for navigation, introducing latency on non-navigating clicks.
packages/core/tests/integration/perform-understudy-navigation-race.spec.ts	New integration test covering the navigation race; timing margin (250ms) between the 650ms fixture delay and 400ms window may cause intermittent CI failures.

Sequence Diagram

sequenceDiagram
    participant Caller
    participant performUnderstudyMethod
    participant navWatcher
    participant CDP Session
    participant waitForDomNetworkQuiet

    Caller->>performUnderstudyMethod: act("click", xpath)
    performUnderstudyMethod->>navWatcher: watchMainFrameUrlChangeStart(page, page.url())
    navWatcher->>CDP Session: session.on("Page.frameNavigated")
    navWatcher->>CDP Session: session.on("Page.navigatedWithinDocument")

    performUnderstudyMethod->>performUnderstudyMethod: clickElement(ctx)

    alt Navigation fires during click (fast)
        CDP Session-->>navWatcher: Page.frameNavigated / NavigatedWithinDocument
        navWatcher->>navWatcher: matched = true
    end

    performUnderstudyMethod->>navWatcher: wait(400ms)

    alt Navigation already matched
        navWatcher-->>performUnderstudyMethod: true (immediate)
    else Navigation fires within 400ms window
        CDP Session-->>navWatcher: Page.frameNavigated / NavigatedWithinDocument
        navWatcher-->>performUnderstudyMethod: true (early resolve)
    else No navigation within 400ms
        navWatcher-->>performUnderstudyMethod: false (timeout)
    end

    alt wait() returned true
        performUnderstudyMethod->>waitForDomNetworkQuiet: waitForDomNetworkQuiet(mainFrame, timeout)
        waitForDomNetworkQuiet-->>performUnderstudyMethod: settled
    end

    performUnderstudyMethod->>navWatcher: dispose()
    navWatcher->>CDP Session: session.off("Page.frameNavigated")
    navWatcher->>CDP Session: session.off("Page.navigatedWithinDocument")

    performUnderstudyMethod-->>Caller: return (page.url() reflects new page)

Loading

_{Last reviewed commit: 14c9ea4}

[greptile-apps]

Unconditional 400ms latency on every click/doubleClick

navWatcher.wait(400) always blocks for up to 400ms after every click or doubleClick action, even when the click does not trigger any navigation. The timer in wait() only resolves early if a Page.frameNavigated or Page.navigatedWithinDocument event fires within the window — if neither fires, the 400ms timeout runs to completion before returning false.

In agentic workflows that perform many clicks (selecting options, toggling checkboxes, closing modals, etc.) this adds up to 400ms of dead wait per non-navigating click. A task with 20 clicks would incur up to 8 seconds of pure idle time.

Consider whether the window should start from when the click handler is dispatched rather than when wait() is called, or whether a shorter window (e.g. 150–200ms) would be enough to catch the typical "JS deferred with setTimeout" pattern while reducing the worst-case overhead on non-navigating clicks.

[pirate]

ALTERNATIVE IDEA (which I prefer) would be add an arg to act({..., expectNavigation: true | false}) and only wait for nav if true is passed.

[greptile-apps]

That's a much better approach. The explicit expectNavigation flag avoids the entire detection problem — no 400ms timeout penalty on non-navigating clicks, and the caller already knows whether they expect navigation (e.g., clicking a submit button vs toggling a checkbox).

The automatic detection forces a trade-off between latency (shorter window = faster but might miss slow navigations) and reliability (longer window = catches more cases but adds overhead everywhere). An explicit flag sidesteps this entirely and makes the behavior predictable.

The only downside is slightly more verbose call sites when navigation is expected, but that's a small price for eliminating 8+ seconds of idle time across a multi-click task.