Skip to content
This repository was archived by the owner on Jul 18, 2026. It is now read-only.

feat(google): add Google My Activity API connector - #68

Merged
Kahtaf merged 9 commits into
feat/instagram-api-playwright-connectorfrom
feat/google-myactivity-api-playwright
Apr 15, 2026
Merged

Kahtaf merged 9 commits into
feat/instagram-api-playwright-connectorfrom
feat/google-myactivity-api-playwright

Conversation

@Kahtaf

@Kahtaf Kahtaf commented Apr 15, 2026

Copy link
Copy Markdown
Member

Summary

Adds a new Google My Activity connector (google-api-playwright) that exports the user's cross-product Google activity feed (Search, Maps, YouTube, Play, Assistant, apps) using an API-first batchexecute replay — no DOM scraping for data, only for login.

Problem

Users have no way to export their full Google My Activity history through Vana DataConnect. The existing youtube-playwright connector only covers YouTube-specific scopes, and there is no cross-product activity scope anywhere in the registry.

Approach

  • Extraction: reuse Google's own footprintsmyactivityui batchexecute RPC endpoint, replaying paginated y3VFHd calls from inside the logged-in page context. SSR initializer globals (SNlM0e auth token, FdrFJe session id, boq_footprintsmyactivityuiserver_* build label, ds:* rpc id map) are scraped from the /myactivity landing HTML so the connector survives Google-side rpc id rotations.
  • Wire format parser: batchexecute responses use a length-prefixed framing format with advisory lengths; the parser uses the numeric-header-between-newlines pattern as a reliable delimiter and falls back to string-scan if the leading length is missing.
  • Entry normalizer: each 29-slot y3VFHd tuple is decoded via fixed positional indices documented at the top of the connector. Output shape: { id, productIds, productName, productIcon, timestampMicros, timestampIso, title, subtitle, action, url, appName, appUrl, device }.
  • Login: drives a URL-classification loop that handles identifier, password, TOTP, SMS, backup-code, and phone-prompt challenges.
    • Text-input challenges use page.requestInput to collect the value, then real CDP keystrokes (page.type + page.press('Enter')) to submit. Google's form validation rejects synthetic setter + button.click() — only trusted Input.dispatchKeyEvent events advance the flow.
    • Prompt-on-device specifically scrapes the verification number Google shows on /challenge/dp (via <samp> tag, leaf-element digit walk, and regex fallback) and surfaces it in a requestInput dialog so users whose host hides the streamed browser canvas still know which number to tap on their phone.
    • Non-text challenges (passkey, captcha, account-chooser, unknown interstitials) fall back to the streamed browser with a promptUser banner.
  • Post-login landing detection: checkLoginStatus accepts /myactivity directly, and for any other authenticated google.com domain (e.g. myaccount.google.com/?utm_source=sign_in_no_continue, which is where Google frequently drops users after an implicit sign-in) it bounces once through MYACTIVITY_URL and re-validates the SSR globals.
  • Pagination budget: MAX_PAGES = 50 so extraction comfortably finishes inside the 5-minute session TTL on the remote-browser-service harness.

Testing

  • node scripts/validate-manifests.mjs — 0 errors, 0 warnings (21 manifests validated)
  • End-to-end against a real Google account with phone-prompt MFA on the remote-browser-service dev sprite pool:
    1. Email submitted via requestInput → real keystrokes → Google advanced /signin/identifier → /challenge/pwd
    2. Password submitted via requestInput → Google advanced /challenge/pwd → /challenge/dp
    3. Prompt-on-device dialog showed Tap 53 on your phone, then press Submit. (scraped from Google's <samp> element)
    4. User tapped 53 on their Galaxy device → Google redirected to myaccount.google.com → checkLoginStatus auto-navigated to /myactivity
    5. Connector scraped SSR globals, started batchexecute pagination, captured 12,336 activities across 177 pages before the 5-minute session TTL expired (this is what drove the 50-page cap in this PR)

Files

  • connectors/google/google-api-playwright.js — connector script (~620 lines)
  • connectors/google/google-api-playwright.json — manifest
  • connectors/google/icons/google.svg — icon
  • connectors/google/schemas/google.myactivity.json — scope schema
  • registry.json — new entry with sha256 checksums + lastUpdated bump

Notes

  • Marked status: "experimental" in the registry — the batchexecute contract is stable at the Google side but this is the first cross-product activity connector so it should burn in before being promoted to beta/stable.
  • Depends on a companion PR in remote-browser-service that exposes fill/press/type/click/url/waitForSelector on ConnectorPageAPI. Those methods are declared in the official types/connector.d.ts PageAPI contract and used by several existing connectors (github, oura, meta/instagram, _conformance), but the sprite-server wrapper wasn't forwarding them until now. This connector cannot sign in without that fix landing on the harness side.

tnunamak and others added 8 commits April 14, 2026 18:14
## Summary
- publish canonical `consumerMetadata` for `heb-playwright`
- publish canonical `consumerMetadata` for `wholefoods-playwright`
- regenerate `connector-index.json` so downstream consumers can resolve
both connectors without local brand/domain fallbacks

## Why
`data-connect` should not need to hard-code `displayName`,
`brandDomain`, or default scope details for these connectors. Those
fields belong upstream in `data-connectors` so generated platform
registries can stay minimal.

## Validation
- `node scripts/generate-connector-index.mjs --check`
## Summary
- run the canonical manifest normalizer on
`connectors/apple/icloud-notes-playwright.json`
- restore the expected canonical/legacy field ordering so contract
guardrails pass

## Validation
- `node scripts/normalize-manifests.mjs --check`
## Summary
- replace the four retained `feat/connector-contract-unification`
connector-index entries with the immutable commit that actually contains
those artifacts
- keep current mainline connector entries unchanged
- refresh the stale `icloud-notes-playwright@0.2.0` manifest checksum in
the index

## Why
`context-gateway` consumes `connector-index.json` as a registry input
when refreshing its pinned snapshot. The latest versions it uses are
already on `main`, but the index still retained four older versions that
referenced a feature branch name:
- `github-playwright@1.1.4`
- `icloud-notes-playwright@0.1.0`
- `instagram-playwright@1.1.0`
- `oura-playwright@1.0.1`

Those artifacts were originally published in commit
`1097ceddc8bff1c60e29ed60cd8af9f65c594646`, so this PR points the
retained historical entries at that immutable ref instead of the mutable
branch name.

## Validation
- `node scripts/generate-connector-index.mjs --check`
- verified `1097ceddc8bff1c60e29ed60cd8af9f65c594646` contains all four
retained artifact paths
## Summary
- republish `icloud-notes-playwright@0.2.0` so the committed artifact
matches the current manifest ordering on `main`
- update `artifactSha256` in `connector-index.json` to match the
republished tarball

## Why
`context-gateway` currently cannot resolve the latest
`data-connectors/main` because `connector-index.json` expects the
current `icloud-notes` manifest ordering, but the committed tarball at
`artifacts/icloud-notes-playwright/icloud-notes-playwright-0.2.0.tgz`
still contains the older manifest bytes. This makes the branch
publication-inconsistent.

## Validation
- `CONNECTOR_ARTIFACT_REF=main node
scripts/generate-connector-index.mjs`
- `node scripts/generate-connector-index.mjs --check`
- verified packaged `./manifest.json` sha256 is
`550ba20690d3d16d809686df73f9f23f18005dd5763ce6b631a17d4d7da370c4`
Exports the cross-product Google My Activity feed (Search, Maps, YouTube,
Play, Assistant, apps) via an API-first batchexecute replay — no DOM
scraping for data.

Login drives the existing challenge-URL classification loop with real CDP
keystrokes (page.type + page.press('Enter')) so Google's form validation
accepts the submission. Prompt-on-device challenges scrape the verification
number from the page and surface it in a requestInput dialog so users on
hosts that hide the streamed canvas still know which number to tap.

MAX_PAGES is capped at 50 to comfortably finish extraction within the
5-minute session TTL on the remote-browser-service harness.
@github-actions

github-actions Bot commented Apr 15, 2026 •

Copy link
Copy Markdown

Schema Health Check — 7 issue(s) found

41/48 scopes consistent | 0 missing schema files | 7 not in Gateway | 0 metadata drift | 0 orphaned

View issues

Not registered in Gateway:

Scope Connector Added in PR?
instagram.following instagram-playwright
google.myactivity google-api-playwright added in this PR
steam.profile steam-playwright
steam.games steam-playwright
steam.friends steam-playwright
icloud_notes.notes icloud-notes-playwright
icloud_notes.folders icloud-notes-playwright

These issues should be resolved before connectors can be used by Personal Servers.

Previously requestEmail/requestPassword cached the first submission and
returned it on every subsequent loop iteration, so when Google rejected
the wrong credential and the URL-classification loop re-entered the
input step, the same wrong value was silently re-submitted and no new
requestInput dialog fired. Users typing a wrong password on their first
attempt had no way to correct it — the connector just spun for the 5
minutes until the session TTL.

Google's /challenge/pwd and /signin/identifier pages stay on the same
URL after a rejection (error rendered inline), so any time the loop
re-enters an input branch it means the previous attempt was rejected.
Dropping the cache entirely makes every revisit prompt the user with a
fresh dialog.

The cache's original purpose was to avoid re-prompting when opensteer's
login loop bounced back through the same step after an account-chooser
interstitial, but in this harness the URL changes during those bounces
so there's no loop-iteration overlap to skip.
@Kahtaf
Kahtaf changed the base branch from main to feat/instagram-api-playwright-connector April 15, 2026 21:34
@Kahtaf
Kahtaf marked this pull request as ready for review April 15, 2026 21:34
@Kahtaf
Kahtaf merged commit 6a09d65 into feat/instagram-api-playwright-connector Apr 15, 2026
0 of 2 checks passed
Kahtaf added a commit that referenced this pull request Apr 15, 2026
Add missing connector-index entry and packaged artifact for
google-api-playwright. The connector source, manifest, and registry
entry landed via #68 but the index + tgz were never updated, so
consumers that enumerate connectors via connector-index.json did not
see Google My Activity.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants