Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions docs/adr/0016-active-session-script-publication.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,20 @@ work, so the caller can retry with plain `close`. Plain `close` tears down ABORT
writing; closing an unpublished ARMED recording retains the existing close-time publication behavior. A
fresh session is the only re-arming boundary.

> **Amendment (2026-08-02, shipped).** A session that was never armed — no recorded
> `open --save-script` at all — has no fourth pre-ARMED state name here, but it previously fell
> through the same close-time write path as an ARMED recording: `close --save-script` on it folded
> the request into the authoring lifecycle at record time and published anyway. Live evidence
> showed this produces a script whose actions carry selector fallback chains but no recording-time
> `target-v1` evidence, with no signal to the caller that evidence capture never ran — degraded
> replay verification is worse than a loud refusal. `close --save-script` on a never-armed session
> is now rejected before any teardown or filesystem work, the same way ABORTED/PUBLISHED are,
> naming `open --save-script` as the recovery; a plain `close` still tears the session down
> without writing. This is distinct from
> [#1533](https://github.com/callstack/agent-device/issues/1533), which is about an
> already-ARMED-then-ABORTED session whose flag ingress re-enables `recordSession` and lets a
> *bare* `close` (no `--save-script` on the close itself) publish; that case is unresolved here.

This lifecycle is distinct from ADR 0012's repair transaction. `session save-script` rejects a session
with `saveScriptBoundary` set and directs the caller to finish or abort the repair through its existing
`replay --from` and teardown commit protocol. Active-session publication never marks a repair COMPLETE,
Expand Down Expand Up @@ -264,6 +278,9 @@ executing that script, not the artifact being saved.
- In ABORTED/PUBLISHED, `close --save-script[=<other>]` is rejected before platform close and plain
`close` tears down without writing; closing an unpublished ARMED recording preserves current
close-time publication behavior.
- On a never-armed session (2026-08-02 amendment), `close --save-script` is likewise rejected before
platform close or filesystem work, naming `open --save-script` as the recovery; plain `close` still
tears down without writing, and the session is not deleted by the rejected request.
- Descriptor completeness tests classify every recordable request's mutation effect, including
request-sensitive read-only/mutating subcommands, and destination-guard ordering consumes only that
trait.
Expand Down
3 changes: 2 additions & 1 deletion src/commands/cli-grammar/flag-definitions-action.ts
Original file line number Diff line number Diff line change
Expand Up @@ -248,7 +248,8 @@ export const ACTION_FLAG_DEFINITIONS: readonly FlagDefinition[] = [
names: ['--save-script'],
type: 'booleanOrString',
usageLabel: '--save-script [path]',
usageDescription: 'Save session script (.ad) on close; optional custom output path',
usageDescription:
'Arm evidence capture on open, publish the armed recording on close; close --save-script alone (without an armed open) is rejected — start with open --save-script, or use session save-script mid-session. Optional custom output path.',
},
{
key: 'networkInclude',
Expand Down
106 changes: 106 additions & 0 deletions src/daemon/handlers/__tests__/session-close-shutdown.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1131,3 +1131,109 @@ test('targeted close skips platform dispatch and preserves the error when the re
expect(mockStopIosRunnerSession.mock.calls.length).toBeGreaterThan(1);
expect(sessionStore.get(sessionName)).toBeUndefined();
});

// Live evidence (2026-08-02): a plain `open` followed by `close --save-script` used to fold the
// never-armed session into the authoring lifecycle and publish anyway, producing a script with
// selector fallback chains but no recording-time `target-v1` evidence. These two tests prove the
// daemon-seam fix: the rejection fires before ANY teardown work (no dispatch mock needed — a
// no-target close on Android never reaches `dispatchCommand`), the session survives so the agent
// can retry, and no script file is written.
test('close --save-script on a never-armed session is rejected before teardown, with no script written', async () => {
const sessionStore = makeSessionStore();
const sessionName = 'android-unarmed-close-save-script-session';
const scriptPath = path.join(os.tmpdir(), `agent-device-unarmed-close-${Date.now()}.ad`);
sessionStore.set(
sessionName,
makeSession(sessionName, {
platform: 'android',
id: 'emulator-5554',
name: 'Pixel_9_API_35',
kind: 'emulator',
booted: true,
}),
);

await expect(
handleSessionCommands({
req: {
token: 't',
session: sessionName,
command: 'close',
positionals: [],
flags: { saveScript: scriptPath },
},
sessionName,
logPath: path.join(os.tmpdir(), 'daemon.log'),
sessionStore,
invoke: noopInvoke,
}),
).rejects.toMatchObject({
code: 'INVALID_ARGS',
message: expect.stringMatching(/not armed/),
details: expect.objectContaining({
hint: expect.stringMatching(/open <app> --save-script/),
}),
});

// The rejection does not tear down the session — it stays retryable/recoverable.
expect(sessionStore.get(sessionName)).toBeDefined();
expect(fs.existsSync(scriptPath)).toBe(false);

// A plain close (no --save-script) still closes the same session cleanly afterward.
const plainClose = await handleSessionCommands({
req: {
token: 't',
session: sessionName,
command: 'close',
positionals: [],
flags: {},
},
sessionName,
logPath: path.join(os.tmpdir(), 'daemon.log'),
sessionStore,
invoke: noopInvoke,
});
expect(plainClose?.ok).toBe(true);
expect(sessionStore.get(sessionName)).toBeUndefined();
});

test('close --save-script on a session with an active .ad repair transaction is unaffected by the unarmed-authoring guard', async () => {
const sessionStore = makeSessionStore();
const sessionName = 'android-repair-close-save-script-session';
const session = {
...makeSession(sessionName, {
platform: 'android',
id: 'emulator-5554',
name: 'Pixel_9_API_35',
kind: 'emulator',
booted: true,
}),
recordSession: true,
scriptPublication: {
kind: 'repair' as const,
status: 'complete' as const,
target: { kind: 'default' as const, force: false },
boundary: 0,
},
};
sessionStore.set(sessionName, session);

const response = await handleSessionCommands({
req: {
token: 't',
session: sessionName,
command: 'close',
positionals: [],
flags: { saveScript: true },
},
sessionName,
logPath: path.join(os.tmpdir(), 'daemon.log'),
sessionStore,
invoke: noopInvoke,
});

// Repair transactions are a disjoint lifecycle (ADR 0012) with their own arming and close-time
// commit protocol; the new unarmed-authoring guard must not intercept them.
expect(response?.ok).toBe(true);
expect(sessionStore.get(sessionName)).toBeUndefined();
});
20 changes: 19 additions & 1 deletion src/daemon/handlers/session-close.ts
Original file line number Diff line number Diff line change
Expand Up @@ -285,11 +285,29 @@ async function stopOrRetainAppleRunnerAfterClose(
scheduleIosRunnerIdleStop(session.device.id);
}

// Live evidence (2026-08-02): a plain `open` followed by `close --save-script` used to fold into
// the authoring lifecycle at close time (`applyRecordedSaveScriptFlags`'s `none -> authoring`
// branch) and publish anyway. That silently produces a script whose actions carry selector
// fallback chains but no `target-v1` recording-time evidence — degraded replay verification with
// no signal to the caller. Recording-time evidence can only be captured from action zero
// (`armAuthoringOnOpen`), so an unarmed session has nothing to retroactively arm; the only
// correct response is refusal, before any teardown or publication work runs. This intentionally
// does not resolve #1533 (aborted-mid-recording close --save-script); that is a distinct,
// already-armed case with its own resolution.
function assertTerminalRecordingCloseAllowed(req: DaemonRequest, session: SessionState): void {
if (!req.flags?.saveScript) return;
if (isAuthoringArmedSession(session)) return;
const state = session.scriptPublication;
if (state?.kind !== 'authoring') return;
if (state?.kind === 'repair') return;
if (state === undefined || state.kind === 'none') {
throw new AppError(
'INVALID_ARGS',
'close --save-script cannot publish this session: recording was not armed before this journey began, so there is no recording-time target evidence to publish.',
{
hint: 'Retry with plain close (it tears down without writing). To capture a publishable recording, start a fresh session with open <app> --save-script[=<path>].',
},
);
}
throw new AppError(
'INVALID_ARGS',
`close --save-script cannot ${state.status === 'published' ? 're-publish' : 'publish'} this terminal recording. Retry with plain close; it will tear down the session without writing.`,
Expand Down
13 changes: 8 additions & 5 deletions src/daemon/session-script-publication-capability.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,11 +61,14 @@ export function abortAuthoringOnSecondOpen(session: SessionState): void {
* other command's raw flag closed at the router). Arms recording and applies target/force to
* whichever lifecycle the session is in:
*
* - `none` -> ordinary authoring armed. This is how a never-armed `close --save-script`
* publishes the whole log: the close request arms at record time and publishes moments later
* in the same request, folding the former "third mode" into the authoring lifecycle. The
* session is deleted by every close path that gets this far, so the transient armed state is
* unobservable to `session save-script` eligibility.
* - `none` -> ordinary authoring armed. This branch is UNREACHABLE for `close`: live evidence
* (2026-08-02) showed it used to let a never-armed `close --save-script` fold into the
* authoring lifecycle and publish moments later in the same request — a script with selector
* fallback chains but no recording-time `target-v1` evidence, and no signal to the caller.
* `session-close.ts`'s `assertTerminalRecordingCloseAllowed` now rejects an unarmed
* `close --save-script` before any action recording runs, so this arm only fires for a
* future non-close caller of the shared ingress; it is kept as that caller's safety net, not
* as a documented close-time behavior.
* - `authoring` -> retarget under the #1258 per-target force rule (`resolveScriptTarget`).
* - `repair` -> retarget the repair target the same way (a replayed step may carry the flag).
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,68 @@ test('a second successful open aborts publication and terminal save flags fail b
});
}, 20_000);

// Live evidence (2026-08-02): a plain `open` (no arming) followed by `close --save-script` used to
// silently publish anyway — the close request armed authoring at record time and published moments
// later in the same request, producing a script with selector fallback chains but NO recording-time
// `target-v1` evidence and no signal to the caller that the evidence was missing. The daemon now
// rejects this before any teardown or filesystem work, and the session stays open so a plain close
// still completes cleanly (it just does not publish).
test('an unarmed session refuses close --save-script and closes cleanly on plain close', async () => {
await withProviderScenarioResource(createAndroidSettingsWorld, async (world) => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-unarmed-close-provider-'));
const scriptPath = path.join(root, 'unarmed.ad');
try {
const opened = await world.daemon.callCommand('open', ['settings'], { ...world.selection });
assertRpcOk(opened);
assert.equal(authoringPublicationStatus(world), undefined);

const flaggedClose = await world.daemon.callCommand('close', [], { saveScript: scriptPath });
assertRpcError(flaggedClose, 'INVALID_ARGS', /not armed/);
assert.ok(world.daemon.session(), 'flagged close must not tear down the unarmed session');
assert.equal(fs.existsSync(scriptPath), false);

const plainClose = await world.daemon.callCommand('close');
assertRpcOk(plainClose);
assert.equal(world.daemon.session(), undefined);
} finally {
fs.rmSync(root, { recursive: true, force: true });
}
});
}, 20_000);

// The counterpart to the unarmed refusal above: an `open --save-script`-armed session must still
// publish through the ordinary close-time route (not just through `session save-script`), and the
// published script must carry the same recording-time `target-v1` evidence.
test('an armed session still publishes target-v1 evidence through close --save-script', async () => {
await withProviderScenarioResource(createAndroidSettingsWorld, async (world) => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-armed-close-provider-'));
const scriptPath = path.join(root, 'armed-close.ad');
const client = world.daemon.client();
try {
await client.apps.open({
app: 'settings',
saveScript: scriptPath,
...world.selection,
});
assert.equal(authoringPublicationStatus(world), 'armed');

const snapshot = await client.capture.snapshot({ interactiveOnly: true, ...world.selection });
const search = snapshot.nodes.find((node) => node.label === 'Search');
assert.ok(search?.ref, JSON.stringify(snapshot.nodes));
await client.interactions.click({ ref: `@${search.ref}`, ...world.selection });

const close = await world.daemon.callCommand('close', [], { saveScript: scriptPath });
assertRpcOk(close);
assert.equal(fs.existsSync(scriptPath), true);
const script = fs.readFileSync(scriptPath, 'utf8');
assert.match(script, /agent-device:target-v1/);
assert.equal(world.daemon.session(), undefined);
} finally {
fs.rmSync(root, { recursive: true, force: true });
}
});
}, 20_000);

test('parameterized fill publishes only ${VAR} and replay resolves it immediately before fill', async () => {
const secret = 'OpaqueProviderValue1348';
let injectedText: string | undefined;
Expand Down
15 changes: 15 additions & 0 deletions test/integration/provider-scenarios/android-lifecycle.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1325,7 +1325,22 @@ async function runAndroidCaptureInteractionAndReplayWorkflow(
assert.equal(beforeCloseOpen.appBundleId, 'com.example.demo');
const logsBeforeClose = await client.observability.logs({ action: 'start', ...selection });
assert.equal(logsBeforeClose.started, true);

// close --save-script now requires the session to have been armed at open (recording-time
// target-v1 evidence cannot be reconstructed retroactively for a session that never recorded
// it). End this long-lived unarmed session plainly, then arm a fresh one before exercising
// close --save-script + shutdown below.
const plainCloseBeforeArm = await daemon.callCommand('close');
assert.equal(plainCloseBeforeArm.statusCode, 200, JSON.stringify(plainCloseBeforeArm.json));
assert.equal(daemon.session(), undefined);

const savedReplayPath = path.join(tempRoot, 'saved-session.ad');
const armedOpen = await client.apps.open({
app: 'com.example.demo',
saveScript: savedReplayPath,
...selection,
});
assert.equal(armedOpen.appBundleId, 'com.example.demo');
const close = await daemon.callCommand('close', [], {
saveScript: savedReplayPath,
shutdown: true,
Expand Down
Loading