ci(mobile): decouple package smokes from e2e so a flaky e2e can't mask them#430
Conversation
|
Standing release PR: #400 · 13 packages queued · open 107h · ✅ ready to merge Release Preview — 13 packages
These changes will be added to the release PR (#400) when merged: ChangelogProject-wide changesChanged
@wdio/dioxus-service N/A → 1.0.1Changed
@wdio/electron-cdp-bridge wdio-electron-cdp-bridge@v10.0.0 → 10.0.1Changed
@wdio/electron-service wdio-electron-service@v10.0.0 → 10.0.1Changed
@wdio/flutter-service v1.0.0-next.0 → 1.0.1Changed
@wdio/native-cdp-bridge v0.1.0-next.0 → 0.1.1Changed
@wdio/native-mobile-core v1.0.0 → 1.0.1Changed
@wdio/native-spy wdio-native-spy@v1.1.0 → 1.1.1Changed
@wdio/native-types wdio-native-types@v2.3.1 → 2.3.2Changed
@wdio/native-utils wdio-native-utils@v2.4.0 → 2.4.1Changed
@wdio/react-native-service v1.0.0-next.0 → 1.0.1Changed
@wdio/tauri-service wdio-tauri-service@v1.1.0 → 1.1.1Changed
dioxus-package-test-app v0.1.0 → 0.1.1Changed
tauri-plugin-wdio-webdriver tauri-plugin-wdio-webdriver@v1.1.0 → 1.1.1Changed
After merge — predicted release
Updated automatically by ReleaseKit |
|
| Filename | Overview |
|---|---|
| .github/workflows/_ci-e2e-flutter-ios.reusable.yml | Decouples e2e and smoke exit codes in the bash step using ` |
| .github/workflows/_ci-e2e-flutter.reusable.yml | Collapses the decoupled pipeline onto a single line to satisfy android-emulator-runner's per-line sh -c execution model; set -eu (no pipefail) is correct for dash; logic is sound. |
| .github/workflows/_ci-e2e-react-native-ios.reusable.yml | Decouples e2e and conditionally gated smoke (new-arch only) in the bash step; smoke_rc defaults to 0 when the gate is false, so non-new-arch runs still honour e2e failure correctly. |
| .github/workflows/_ci-e2e-react-native.reusable.yml | Single-line decoupled pipeline for android-emulator-runner; the conditional if [ new-arch ] guard is embedded correctly inline; dash-compatible set -eu is unchanged. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Job starts\ndevice/Metro/WDA ready] --> B[Run e2e suite\ncapture exit: e2e_rc]
B -->|e2e_rc=0| C[Run package smoke\ncapture exit: smoke_rc]
B -->|e2e_rc≠0| C
C --> D{e2e_rc≠0\nOR smoke_rc≠0?}
D -->|Yes| E[exit 1\nJob fails]
D -->|No| F[exit 0\nJob passes]
style B fill:#d4e6f1,stroke:#2874a6
style C fill:#d5f5e3,stroke:#1e8449
style E fill:#fadbd8,stroke:#922b21
style F fill:#d5f5e3,stroke:#1e8449
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
A[Job starts\ndevice/Metro/WDA ready] --> B[Run e2e suite\ncapture exit: e2e_rc]
B -->|e2e_rc=0| C[Run package smoke\ncapture exit: smoke_rc]
B -->|e2e_rc≠0| C
C --> D{e2e_rc≠0\nOR smoke_rc≠0?}
D -->|Yes| E[exit 1\nJob fails]
D -->|No| F[exit 0\nJob passes]
style B fill:#d4e6f1,stroke:#2874a6
style C fill:#d5f5e3,stroke:#1e8449
style E fill:#fadbd8,stroke:#922b21
style F fill:#d5f5e3,stroke:#1e8449
Reviews (3): Last reviewed commit: "ci(mobile): decouple package smokes from..." | Re-trigger Greptile
0183f22 to
fb4aee7
Compare
…k them The mobile package smokes share each e2e job's booted device/Metro/WDA to reuse the infra, running right after the e2e suite under `set -e`. So an e2e failure aborted the step before the smoke ran — and the mobile e2e suites are occasionally flaky (iOS CoreSimulator launch wedges #421; transient Android UiAutomator2 / Dart-VM flakes), which hid whether the PACKED tarball actually installs + composes. Decouple the two within the step: capture each exit code with `|| rc=$?` (so the e2e failure no longer aborts before the smoke) and fail only at the end if either failed. The smoke still runs second, on a device the suite has warmed, so it doesn't inherit the cold-start risk. Kept in one step (not split) because Metro/WDA/sim state lives there. On the iOS legs the step is one bash shell, so the rc vars span lines normally. On Android the whole e2e+smoke+check pipeline stays on ONE line: android-emulator-runner runs the script line-by-line, each in its own `sh -c`, so a var set on one line is gone on the next — split across lines the final check sees empty rc vars and `[ "" != 0 ]` would always exit 1. Applies to all four legs: RN iOS, RN Android, Flutter iOS, Flutter Android. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
fb4aee7 to
9741a9c
Compare
What
The mobile package smokes (RN + Flutter, both OSes) run inside each e2e job — right after the e2e suite, reusing the booted device / Metro / WDA (the cheap "make best use of the e2e infra" approach from #387). They ran under
set -ein the same step, so an e2e failure aborted before the smoke ran.The mobile e2e suites are occasionally flaky (iOS CoreSimulator launch wedge — #421; transient Android UiAutomator2 / Dart-VM flakes), and when they flaked the smoke was silently skipped — so we lost the signal for "does the published, packed tarball actually install + compose?", which is the smoke's whole job and is independent of e2e flakiness.
Change
Within each job's run step, decouple the two: capture each exit code with
|| rc=$?(so the e2e failure no longer aborts the step), run the smoke regardless, andexit 1at the end if either failed.The job still fails if either fails; the difference is the smoke always runs and emits its signal.
Why in-step (not a separate
if: !cancelled()step)The smoke depends on state that lives in the e2e step — Metro is started (backgrounded) there on the RN legs, and the booted sim + prebuilt WDA are set up per job. Splitting would orphan Metro and duplicate env, so decoupling the exit codes in-place is lower-risk and uniform.
Ordering unchanged (smoke stays second)
Deliberate: the multi-spec e2e suite absorbs the iOS cold-start launch risk, so the smoke runs on an already-warmed device. Running the smoke first would make the single-session smoke the cold-start victim.
Legs covered
_ci-e2e-react-native-ios.reusable.yml)_ci-e2e-react-native.reusable.yml) — single-line per statement, since android-emulator-runner splits a multi-lineif/then/fi_ci-e2e-flutter-ios.reusable.yml)_ci-e2e-flutter.reusable.yml) — same single-line constraintWorkflow-only change;
actionlintclean.🤖 Generated with Claude Code