Skip to content

feat(serenity): audience-driven IMS promise pair for Semrush paths - #3033

Open
calvarezg wants to merge 5 commits into
mainfrom
feat/LLMO-6623-semrush-proxy-removal
Open

feat(serenity): audience-driven IMS promise pair for Semrush paths#3033
calvarezg wants to merge 5 commits into
mainfrom
feat/LLMO-6623-semrush-proxy-removal

Conversation

@calvarezg

@calvarezg calvarezg commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

1. Abstract

Lets a request select the dedicated Semrush IMS promise-token pair via an optional x-promise-audience: semrush header, for both the synchronous serenity/elements/brands surfaces and the asynchronous Path-B classify-prompts job.

2. Reasoning

LLMO-6623 gives the Semrush-facing paths a dedicated IMS promise pair (scoped for Semrush UDH) with limited blast radius, instead of the shared pair that serves every promise flow. spacecat-shared-ims-client@1.16.0 (LLMO-6928) added the createFrom(..., { pair }) selector; this change is the api-service consumer of that selector. It is the exchange/mint side of the header the UI will send and auth-service will mint from.

3. High-level overview of the changes

  • New optional request header x-promise-audience. Absent → today's default pair (no behavior change). semrush → the dedicated Semrush pair. Any other value → 400 (fail closed).
  • Sync path: a single helper resolvePromisePair reads the header; resolveSemrushImsToken derives the pair and forwards it to the token exchange. All ~25 serenity/elements/brands call sites are unchanged — they inherit the behavior through that shared helper.
  • Async Path B (classify-prompts job): the pair is resolved at enqueue, persisted on the job's metadata, and read back when the worker exchanges and invalidates the token. A job with no stored pair falls back to the default pair, so jobs already queued at deploy time keep working. The worker self-requeue forwards the job's pair.
  • Bumps @adobe/spacecat-shared-ims-client 1.14.0 → 1.16.0.
  • Inert in production until the UI sends the header. This PR does NOT change SEMRUSH_PROJECTS_BASE_URL — the proxy→native host flip is a separate Vault step after the end-to-end proofs.

4. Required information

  • Jira / issue: LLMO-6930 (parent LLMO-6623)
  • Spec: docs/specs/2026-08-12-llmo-6930-api-service-promise-audience.md (in this PR)

5. Spec deviations

  • None

6. Affected / used mysticat-workspace projects

  • spacecat-shared — consumed: uses ImsPromiseClient.createFrom(..., { pair }) and PROMISE_PAIR from @adobe/spacecat-shared-ims-client@1.16.0 (LLMO-6928 / feat(ims-client): optional pair selector on ImsPromiseClient.createFrom spacecat-shared#1872, merged + published).
  • spacecat-auth-service — contract (sibling, LLMO-6929): mints the promise token from the same Semrush pair; the UI sends x-promise-audience to both. Mint and exchange must use the same pair.
  • project-elmo-ui — contract (downstream, LLMO-6932): the client that will send x-promise-audience on the mint and on data calls.

8. Test plan

  • (a) Local: unit coverage for resolvePromisePair (absent / semrush / case-insensitive / unknown→400), pair pass-through on resolveSemrushImsToken, and Path-B persist/read-back (including the absent→default fallback). The Lambda bundle was built locally and the healthcheck lambda() returns 200 (SITES-45260 gate), since this touches the bundle's dependency set.
  • (b) Per-env: no behavior changes until a client sends x-promise-audience. End-to-end proof of the Semrush pair (mint → exchange → semrush scope on the token) is owned by LLMO-6623's Path A/B runs, already validated at the IMS level for stage and prod. After Vault carries the Semrush pair, exercise one serenity route on dev with the header and confirm the exchange uses the Semrush consumer.

9. Deployment & merge order

🤖 Generated with Claude Code

calvarezg and others added 2 commits August 12, 2026 12:03
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Let a request select the dedicated Semrush IMS promise-token pair via an
optional x-promise-audience: semrush header, for both the synchronous
serenity/elements/brands surfaces and the async Path-B classify-prompts job.
Absent the header, everything uses the default pair (no behavior change);
an unknown audience value returns 400 (fail closed).

- utils.js: resolvePromisePair(context) maps the header to a pair selector or
  throws 400. getIMSPromiseToken/exchangePromiseToken take an optional pair,
  forwarded to ImsPromiseClient.createFrom. resolveSemrushImsToken derives the
  pair from the header, so all ~25 sync Semrush call sites are unchanged.
- async-job-runner.js: persist the pair on job metadata at enqueue; exchange and
  invalidate read it back. Absent metadata -> default pair, so pre-change queued
  jobs keep working. Worker self-requeue forwards the job's pair.
- Bump @adobe/spacecat-shared-ims-client 1.14.0 -> 1.16.0 (provides the pair
  selector); fix the stale 1.12.7 pin comment.

The five stay-behind getIMSPromiseToken callers (edge-routing-auth, fixes,
page-relationships, scrapeJob, suggestions) are untouched.

Part of LLMO-6623. Inert until the UI sends the header; does not flip
SEMRUSH_PROJECTS_BASE_URL.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@calvarezg calvarezg self-assigned this Aug 12, 2026
@calvarezg
calvarezg requested a review from MysticatBot August 12, 2026 10:29
@github-actions

Copy link
Copy Markdown

This PR will trigger a minor release when merged.

@MysticatBot MysticatBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @calvarezg,

Verdict: Approve - clean, well-scoped feature with correct token-flow threading and good test coverage.
Complexity: HIGH - medium diff; auth/IMS signal + dependency bump.
Changes: Adds audience-driven IMS promise-pair selection via an optional x-promise-audience header, threading the pair through sync and async (Path B) token flows (9 files).
Note: CI checks are currently pending - confirm they pass before merge.

Non-blocking (3): minor issues and suggestions
  • suggestion: Consider truncating reflected input in the error message (Unknown promise audience: ${audience}) to bound log/response size from long headers - src/support/utils.js:845
  • nit: JSDoc on getIMSPromiseToken could clarify what pair means for readers unfamiliar with IMS promise pairs (e.g. "determines which client-id/scope pair is used when minting") - src/support/utils.js:817
  • nit: { pair: undefined } is passed to createFrom when no audience is selected, which is semantically different from omitting opts entirely - worth confirming the shared-ims-client 1.16.0 treats both identically - src/support/serenity/async-job-runner.js:167

Skill: pr-review | Model: us.anthropic.claude-opus-4-6-v1[1m] | Duration: 8m 47s | Cost: $8.89 | Commit: 6b7eff7e0a8c743120a1e713fd61fdc918cbaaaa
If this code review was useful, please react with 👍. Otherwise, react with 👎.

@MysticatBot MysticatBot added ai-reviewed Reviewed by AI complexity:high AI-assessed PR complexity: HIGH labels Aug 12, 2026
Address non-blocking MysticatBot review notes on #3033: truncate the reflected
x-promise-audience value in the 'Unknown promise audience' 400 so a long or
hostile header cannot bloat the response body or logs, and expand the
getIMSPromiseToken pair JSDoc. No behavior change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@calvarezg
calvarezg requested a review from MysticatBot August 12, 2026 10:40
@codecov

codecov Bot commented Aug 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@MysticatBot MysticatBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @calvarezg,

Verdict: Approve - clean, well-scoped feature with correct token-flow threading; new commit addresses prior suggestion.
Complexity: HIGH - medium diff; auth/IMS signal + dependency bump + API surface.
Changes: Adds audience-driven IMS promise-pair selection via an optional x-promise-audience header, threading the pair through sync and async (Path B) token flows (9 files).
Note: CI checks are currently pending - confirm they pass before merge.
Note: Recommend a human read before merge - this change introduces a new spec document (docs/specs/), codifying an architectural decision on IMS promise-pair routing. The bot review is a complement to, not a replacement for, a human read here.

Non-blocking (2): minor issues and suggestions
  • nit: No test exercises the 40-char truncation introduced in the latest commit. A string >40 chars would confirm the reflected value is bounded (e.g. pass an 80-char audience, assert error message does not contain the full string) - test/support/utils.test.js
  • suggestion: The classify-prompts-job requeue test asserts promiseToken forwarding but not promisePair forwarding. A one-line expect(enqueueArgs.promisePair).to.equal(expectedPair) closes the coverage gap at that integration seam - test/support/serenity/handlers/classify-prompts-job.test.js

Previously flagged, now resolved

  • Truncate reflected input in error message (src/support/utils.js) - addressed with .slice(0, 40) and explanatory comment.

Skill: pr-review | Model: us.anthropic.claude-opus-4-6-v1[1m] | Duration: 1m 32s | Cost: $5.00 | Commit: 256e1b38a6b4d148a766e20dbaad55c282ee5c70
If this code review was useful, please react with 👍. Otherwise, react with 👎.

@MysticatBot MysticatBot added the needs-human-review AI reviewer recommends a human read before merge label Aug 12, 2026
@calvarezg
calvarezg deployed to dev-branches August 12, 2026 10:53 — with GitHub Actions Active
- CORS: add x-promise-audience to both allow-headers lists (index.js enableCors
  response + OPTIONS preflight). Without it a browser sending the header
  cross-origin fails preflight and the feature silently breaks from the UI.
- utils: sanitize the reflected x-promise-audience value with cleanupHeaderValue
  (repo convention) and trim it before matching, so CR/LF can't inject into the
  400 body/logs and surrounding whitespace doesn't fail closed.
- utils: reorder so resolvePromisePair and getIMSPromiseToken each sit under
  their own JSDoc (the inserted helper had stranded getIMSPromiseToken's docs).
- async-job-runner: when a pre-minted token is supplied, do not re-derive the
  pair from the request header — take the explicit promisePair only, so a
  supplied token and the persisted pair cannot diverge.

Deferred (noted on LLMO-6930): a clearer error when the SEMRUSH env vars are
unprovisioned — the clean fix would couple api-service to the Semrush env-var
names, and the misconfigured window cannot occur under the rollout order (Vault
before UI).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@calvarezg

Copy link
Copy Markdown
Contributor Author

@igrubic thanks for the review. it came in over slack, so capturing it here on the PR for traceability. six findings, five addressed in c47a658, one deferred. rationale below, in your numbering.

1. CORS allowlist missing x-promise-audience (src/index.js) — addressed.
added it to both allow-headers lists (the enableCors response and the OPTIONS preflight). agree this was the load-bearing one: minted UI-side, so without it the browser preflight drops the header and the audience feature silently fails from the UI.

2. stranded JSDoc (src/support/utils.js) — addressed.
reordered so resolvePromisePair and getIMSPromiseToken each sit directly under their own docstring. the inserted helper had left getIMSPromiseToken's contract attached to the wrong function.

3. pair re-derived from header even with a pre-minted token (async-job-runner.js) — addressed.
when a token is pre-supplied the pair now comes from the explicit promisePair only, never re-derived from the request context: promisePair ?? (promiseToken ? undefined : resolvePromisePair(context)). so a supplied token and the persisted pair can't diverge. was latent (the only token-supplying caller, the classify-prompts self-requeue, already forwards promisePair) but the invariant is worth enforcing.

4. unsanitized header in the error message (src/support/utils.js) — addressed.
the reflected value now goes through cleanupHeaderValue() before the length slice, per the repo convention (elements.js, scrapeJob.js, sites.js). truncation alone bounded length but not content, so CR/LF could still inject into the 400 body / logs.

5. opaque 500 when the SEMRUSH env vars are unprovisioned (src/support/utils.js) — deferred.
the clean fix would have api-service check the IMS_PROMISE_SEMRUSH_* vars, which means naming them here. api-service deliberately names zero Semrush env vars today (spacecat-shared owns that contract via the pair selector), and I'd rather not split it across two repos. the misconfigured window also can't occur under the rollout order: Vault (LLMO-6931) is loaded before the UI (LLMO-6932) ever sends the header, so by the time any request carries x-promise-audience: semrush the vars exist. noted on LLMO-6930. happy to add a clearer "audience not configured" signal (without naming the vars, by wrapping the mint/exchange) if you'd rather not leave it, just say so.

6. exact-match without trim (src/support/utils.js) — addressed.
.trim() before the compare, so surrounding whitespace doesn't fail closed with a 400.

tests, coverage, and type-check are green on the final commit. the feature stays dormant in prod until the UI sends the header, and this PR does not flip SEMRUSH_PROJECTS_BASE_URL.

@calvarezg
calvarezg deployed to dev-branches August 12, 2026 14:29 — with GitHub Actions Active
@calvarezg
calvarezg deployed to dev-branches August 12, 2026 14:51 — with GitHub Actions Active
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-reviewed Reviewed by AI complexity:high AI-assessed PR complexity: HIGH needs-human-review AI reviewer recommends a human read before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants