Skip to content

ci(mobile): decouple package smokes from e2e so a flaky e2e can't mask them#430

Merged
goosewobbler merged 1 commit into
mainfrom
ci/decouple-mobile-smokes
Jun 21, 2026
Merged

ci(mobile): decouple package smokes from e2e so a flaky e2e can't mask them#430
goosewobbler merged 1 commit into
mainfrom
ci/decouple-mobile-smokes

Conversation

@goosewobbler

Copy link
Copy Markdown
Contributor

What

The mobile package smokes (RN + Flutter, both OSes) run inside each e2e job — right after the e2e suite, reusing the booted device / Metro / WDA (the cheap "make best use of the e2e infra" approach from #387). They ran under set -e in the same step, so an e2e failure aborted before the smoke ran.

The mobile e2e suites are occasionally flaky (iOS CoreSimulator launch wedge — #421; transient Android UiAutomator2 / Dart-VM flakes), and when they flaked the smoke was silently skipped — so we lost the signal for "does the published, packed tarball actually install + compose?", which is the smoke's whole job and is independent of e2e flakiness.

Change

Within each job's run step, decouple the two: capture each exit code with || rc=$? (so the e2e failure no longer aborts the step), run the smoke regardless, and exit 1 at the end if either failed.

e2e_rc=0;   <e2e>   || e2e_rc=$?
smoke_rc=0; <smoke> || smoke_rc=$?
if [ "$e2e_rc" != 0 ] || [ "$smoke_rc" != 0 ]; then exit 1; fi

The job still fails if either fails; the difference is the smoke always runs and emits its signal.

Why in-step (not a separate if: !cancelled() step)

The smoke depends on state that lives in the e2e step — Metro is started (backgrounded) there on the RN legs, and the booted sim + prebuilt WDA are set up per job. Splitting would orphan Metro and duplicate env, so decoupling the exit codes in-place is lower-risk and uniform.

Ordering unchanged (smoke stays second)

Deliberate: the multi-spec e2e suite absorbs the iOS cold-start launch risk, so the smoke runs on an already-warmed device. Running the smoke first would make the single-session smoke the cold-start victim.

Legs covered

  • RN iOS (_ci-e2e-react-native-ios.reusable.yml)
  • RN Android (_ci-e2e-react-native.reusable.yml) — single-line per statement, since android-emulator-runner splits a multi-line if/then/fi
  • Flutter iOS (_ci-e2e-flutter-ios.reusable.yml)
  • Flutter Android (_ci-e2e-flutter.reusable.yml) — same single-line constraint

Workflow-only change; actionlint clean.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Standing release PR: #400 · 13 packages queued · open 107h · ✅ ready to merge

Release Preview — 13 packages

Note: Labels on this PR are advisory in standing-pr mode. Bumps come from conventional commits in the standing PR; override by editing labels on the standing PR itself. Add release:immediate to bypass the standing PR and release this PR directly.

These changes will be added to the release PR (#400) when merged:

Changelog

Project-wide changes

Changed

  • decouple package smokes from e2e so a flaky e2e can't mask them (mobile)
@wdio/dioxus-service N/A → 1.0.1

Changed

  • Update version to 1.0.1
@wdio/electron-cdp-bridge wdio-electron-cdp-bridge@v10.0.0 → 10.0.1

Changed

  • Update version to 10.0.1
@wdio/electron-service wdio-electron-service@v10.0.0 → 10.0.1

Changed

  • Update version to 10.0.1
@wdio/flutter-service v1.0.0-next.0 → 1.0.1

Changed

  • Update version to 1.0.1
@wdio/native-cdp-bridge v0.1.0-next.0 → 0.1.1

Changed

  • Update version to 0.1.1
@wdio/native-mobile-core v1.0.0 → 1.0.1

Changed

  • Update version to 1.0.1
@wdio/native-spy wdio-native-spy@v1.1.0 → 1.1.1

Changed

  • Update version to 1.1.1
@wdio/native-types wdio-native-types@v2.3.1 → 2.3.2

Changed

  • Update version to 2.3.2
@wdio/native-utils wdio-native-utils@v2.4.0 → 2.4.1

Changed

  • Update version to 2.4.1
@wdio/react-native-service v1.0.0-next.0 → 1.0.1

Changed

  • Update version to 1.0.1
@wdio/tauri-service wdio-tauri-service@v1.1.0 → 1.1.1

Changed

  • Update version to 1.1.1
dioxus-package-test-app v0.1.0 → 0.1.1

Changed

  • Update version to 0.1.1
tauri-plugin-wdio-webdriver tauri-plugin-wdio-webdriver@v1.1.0 → 1.1.1

Changed

  • Update version to 1.1.1

After merge — predicted release

No version escalation — this PR's changes will be included in the queued release without affecting the projected versions.

Package Standing PR This PR After merge
@wdio/dioxus-service 1.1.0 1.0.1 1.1.0
@wdio/electron-cdp-bridge 10.1.0 10.0.1 10.1.0
@wdio/electron-service 10.1.0 10.0.1 10.1.0
@wdio/flutter-service 1.1.0 1.0.1 1.1.0
@wdio/native-cdp-bridge 0.2.0 0.1.1 0.2.0
@wdio/native-mobile-core 1.1.0 1.0.1 1.1.0
@wdio/native-spy 1.2.0 1.1.1 1.2.0
@wdio/native-types 2.4.0 2.3.2 2.4.0
@wdio/native-utils 2.5.0 2.4.1 2.5.0
@wdio/react-native-service 1.1.0 1.0.1 1.1.0
@wdio/tauri-service 1.2.0 1.1.1 1.2.0
dioxus-package-test-app 0.2.0 0.1.1 0.2.0
tauri-plugin-wdio-webdriver 1.2.0 1.1.1 1.2.0

Updated automatically by ReleaseKit

@greptile-apps

greptile-apps Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR decouples the package-install smoke tests from the e2e suites across all four mobile CI reusable workflows (RN iOS/Android, Flutter iOS/Android), so that an occasional e2e flake can no longer silently skip the smoke and suppress its signal. The change captures each exit code with || rc=$?, always runs the smoke, and fails the step at the end if either phase failed.

  • iOS legs (bash, set -euo pipefail): e2e_rc and smoke_rc are captured on separate lines; a final if-check exits 1 if either is non-zero. Multi-line continuation with \ and the optional new-arch gate in the RN case are handled correctly.
  • Android legs (android-emulator-runner, POSIX set -eu): the entire pipeline is collapsed onto one line, with a thorough comment explaining why — android-emulator-runner runs each script line in its own sh -c, so variables set on one line would be invisible on the next, making split lines silently always-fail the final check.
  • Ordering preserved: the smoke runs second, on an already-warmed device, which is deliberate.

Confidence Score: 5/5

Safe to merge — workflow-only change with no application code touched; the decoupling pattern is correct and behaves identically to the previous setup when both phases pass.

The || rc=$? pattern correctly prevents set -e from aborting on e2e failure while preserving the exit code, and the final if-check reproduces the original failure semantics. The Android single-line constraint is correctly applied with a clear explanation. The RN iOS new-arch gate leaves smoke_rc=0 when skipped, which is intentional. No logic gaps found across all four legs.

No files require special attention. All four workflow files apply the same pattern consistently and correctly.

Important Files Changed

Filename Overview
.github/workflows/_ci-e2e-flutter-ios.reusable.yml Decouples e2e and smoke exit codes in the bash step using `
.github/workflows/_ci-e2e-flutter.reusable.yml Collapses the decoupled pipeline onto a single line to satisfy android-emulator-runner's per-line sh -c execution model; set -eu (no pipefail) is correct for dash; logic is sound.
.github/workflows/_ci-e2e-react-native-ios.reusable.yml Decouples e2e and conditionally gated smoke (new-arch only) in the bash step; smoke_rc defaults to 0 when the gate is false, so non-new-arch runs still honour e2e failure correctly.
.github/workflows/_ci-e2e-react-native.reusable.yml Single-line decoupled pipeline for android-emulator-runner; the conditional if [ new-arch ] guard is embedded correctly inline; dash-compatible set -eu is unchanged.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Job starts\ndevice/Metro/WDA ready] --> B[Run e2e suite\ncapture exit: e2e_rc]
    B -->|e2e_rc=0| C[Run package smoke\ncapture exit: smoke_rc]
    B -->|e2e_rc≠0| C
    C --> D{e2e_rc≠0\nOR smoke_rc≠0?}
    D -->|Yes| E[exit 1\nJob fails]
    D -->|No| F[exit 0\nJob passes]

    style B fill:#d4e6f1,stroke:#2874a6
    style C fill:#d5f5e3,stroke:#1e8449
    style E fill:#fadbd8,stroke:#922b21
    style F fill:#d5f5e3,stroke:#1e8449
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[Job starts\ndevice/Metro/WDA ready] --> B[Run e2e suite\ncapture exit: e2e_rc]
    B -->|e2e_rc=0| C[Run package smoke\ncapture exit: smoke_rc]
    B -->|e2e_rc≠0| C
    C --> D{e2e_rc≠0\nOR smoke_rc≠0?}
    D -->|Yes| E[exit 1\nJob fails]
    D -->|No| F[exit 0\nJob passes]

    style B fill:#d4e6f1,stroke:#2874a6
    style C fill:#d5f5e3,stroke:#1e8449
    style E fill:#fadbd8,stroke:#922b21
    style F fill:#d5f5e3,stroke:#1e8449
Loading

Reviews (3): Last reviewed commit: "ci(mobile): decouple package smokes from..." | Re-trigger Greptile

@goosewobbler goosewobbler force-pushed the ci/decouple-mobile-smokes branch from 0183f22 to fb4aee7 Compare June 20, 2026 09:55
…k them

The mobile package smokes share each e2e job's booted device/Metro/WDA to reuse
the infra, running right after the e2e suite under `set -e`. So an e2e failure
aborted the step before the smoke ran — and the mobile e2e suites are
occasionally flaky (iOS CoreSimulator launch wedges #421; transient Android
UiAutomator2 / Dart-VM flakes), which hid whether the PACKED tarball actually
installs + composes.

Decouple the two within the step: capture each exit code with `|| rc=$?` (so
the e2e failure no longer aborts before the smoke) and fail only at the end if
either failed. The smoke still runs second, on a device the suite has warmed,
so it doesn't inherit the cold-start risk. Kept in one step (not split) because
Metro/WDA/sim state lives there.

On the iOS legs the step is one bash shell, so the rc vars span lines normally.
On Android the whole e2e+smoke+check pipeline stays on ONE line:
android-emulator-runner runs the script line-by-line, each in its own `sh -c`,
so a var set on one line is gone on the next — split across lines the final
check sees empty rc vars and `[ "" != 0 ]` would always exit 1.

Applies to all four legs: RN iOS, RN Android, Flutter iOS, Flutter Android.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@goosewobbler goosewobbler force-pushed the ci/decouple-mobile-smokes branch from fb4aee7 to 9741a9c Compare June 21, 2026 12:17
@goosewobbler goosewobbler merged commit e04a2ad into main Jun 21, 2026
76 checks passed
@goosewobbler goosewobbler deleted the ci/decouple-mobile-smokes branch June 21, 2026 18:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant