fix(rendering): recreate renderer when surface errors persist across frames

[daven613]

Description

After a GPU engine reset (e.g. the Intel xe kernel driver killing a hung GPU job on Linux), surface.get_current_texture() can fail with Validation on every subsequent frame. Today the event loop treats SurfaceError(Validation) as transient — it logs, skips the frame, and hopes — and never escalates. The result is a window that is frozen forever while the process, terminal server, and all pty sessions stay alive, with warp.log filling with:

[WARN] Encountered error while getting the next swap chain texture: Validation error
[WARN] Failed to render frame: Failed to acquire surface texture: Validation error

This PR adds a per-window consecutive_render_failures counter. On any render error the counter increments (reset on the next successful frame). Errors that already trigger recovery (SurfaceConfigureError, SurfaceError(Lost), DeviceLost, Unknown) behave exactly as before. Errors that were previously ignored — notably SurfaceError(Validation) — now escalate to the same existing drop_renderer + recreate_renderer recovery path once they have persisted for MAX_CONSECUTIVE_RENDER_FAILURES (10) consecutive frames. Genuinely transient validation errors (the case the current skip-a-frame behavior was written for) still just skip a frame or two and never reach the threshold.

The counter is reset when the renderer is recreated, so if the first rebuild doesn't take (e.g. the GPU is still resetting), the new renderer gets another full threshold's worth of frames before we try again, rather than rebuilding on every frame.

Linked Issue

Fixes #15303

• The linked issue is labeled ready-to-spec or ready-to-implement — filed together with this PR; flagging for triage per the "PRs opened without a linked issue" flow in CONTRIBUTING.md
• Relevant details included below

Testing

• I have manually tested my changes locally

• Manually reproduced and verified: built warp-oss from this branch on the affected machine, forced GT0 engine resets via xe debugfs until the kernel killed an in-flight warp-oss job (the exact trigger from [Linux] Window frozen forever after Intel xe GPU engine reset: surface Validation error loops instead of triggering renderer recovery #15303), and confirmed recovery: exactly 10 consecutive SurfaceError(Validation) frames, then Recreating the renderer..., then a clean re-init and a fully interactive window — ~1s of visible stall, session preserved. Full logs and method in this comment. On the unpatched stable build, the identical kernel event froze the window permanently (6+ hours of the same log loop until killed).

• Transient path preserved: an organic one-off surface Timeout during the same test session skipped a single frame without triggering a rebuild; idle-window resets and media-engine (GT1) resets cause no errors and no rebuilds.

• cargo check -p warpui passes on the patched tree (Rust 1.92.0).

• Real-world failure evidence and the full root-cause trace are in [Linux] Window frozen forever after Intel xe GPU engine reset: surface Validation error loops instead of triggering renderer recovery #15303.

No new automated tests: the behavior under test is the interaction between the winit event loop and a lost GPU device, which has no existing test harness in warpui; the change is confined to the error-handling arm of redraw_window.

CHANGELOG-BUG-FIX: Recover the renderer when surface errors persist across frames (e.g. after a GPU reset on Linux) instead of leaving the window frozen.

🤖 Generated with Claude Code

[cla-bot]

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have the users @daven613 on file. In order for us to review and merge your code, each contributor must visit https://cla.warp.dev to read and agree to our CLA. Once you have done so, please comment @cla-bot check to trigger another check.

[oz-for-oss]

@daven613

Every PR must be linked to a same-repo issue before Oz can review it.

This PR is linked to #15303, but no linked issue is marked ready-to-implement yet. Only repository maintainers apply that label, so please wait for a maintainer to mark the issue. Once it is marked, push a new commit or comment /oz-review to re-trigger review.

See the contribution guidelines for the full readiness model.

Powered by Oz

[oz-for-oss]

@daven613

Every PR must be linked to a same-repo issue before Oz can review it.

This PR is linked to #15303, but no linked issue is marked ready-to-implement yet. Only repository maintainers apply that label, so please wait for a maintainer to mark the issue. Once it is marked, push a new commit or comment /oz-review to re-trigger review.

See the contribution guidelines for the full readiness model.

Powered by Oz

[daven613]

Manual testing: reproduced the GPU reset and verified recovery ✅

Tested this branch (1ef6569) on the same machine and GPU where the original bug hit twice (#15303): Ubuntu-based Linux, kernel 7.0.0-28-generic, Intel integrated GPU (LNL) on the xe driver, Vulkan backend, Wayland.

Method

1. Built and ran warp-oss from this branch (cargo run --bin warp-oss --features gui equivalent).
2. Generated continuous render load in the window (yes "GPU reset recovery test" scrolling full-speed).
3. Forced GT0 render-engine resets via the driver's debugfs (echo 1 > /sys/kernel/debug/dri/0000:00:02.0/tile0/gt0/force_reset, repeated at 10 Hz) until the kernel reported killing an in-flight job belonging to warp-oss — the exact trigger of the original freezes. Hit on iteration 6:

kernel: xe 0000:00:02.0: [drm] Tile0: GT0: Timedout job: seqno=17858, lrc_seqno=17858, guc_id=8, flags=0x0 in warp-oss [769441]

Result — before/after behavior

Before (unpatched, stable v0.2026.08.05.09.03, from #15303): after the identical kernel event, the window froze permanently; warp.log repeated the two Validation error lines several times per second for 6+ hours until the process was killed.

After (this branch): the window stalled for ~1 second and then resumed scrolling, fully interactive. The log shows the new path firing exactly as designed — device lost → exactly 10 consecutive SurfaceError(Validation) frames (= MAX_CONSECUTIVE_RENDER_FAILURES) → escalation to the existing recreate path → clean re-init:

08:39:42Z [WARN] The current device is lost. Reason: Unknown. Message: Device is lost
08:39:42Z [WARN] Encountered error while getting the next swap chain texture: Validation error
08:39:42Z [INFO] Skipping rendering the current frame...
08:39:42Z [WARN] Failed to render frame: Failed to acquire surface texture: Validation error
        ... (the identical skip/fail pair repeats — 10 consecutive failed frames total) ...
08:39:43Z [WARN] Failed to render frame: Failed to acquire surface texture: Validation error
08:39:43Z [WARN] Recreating the renderer in an attempt to recover...
08:39:43Z [INFO] Verifying adapter "Intel(R) Graphics (LNL)" is valid...
08:39:43Z [INFO] Using Vulkan IntegratedGpu (Intel(R) Graphics (LNL)) for rendering new window.

(then normal frame rendering resumes; the session, scrollback, and running command were all preserved)

Transient-error behavior preserved

During normal use of the same test build, a one-off Timeout while getting next surface texture occurred organically; it skipped a single frame and continued without triggering a renderer rebuild — confirming the threshold keeps the existing skip-a-frame behavior for genuinely transient errors.

Also verified

• 12 GT0 resets while the window was idle (no in-flight job): no render errors, no unnecessary rebuilds.
• A GT1 (media engine) reset: correctly ignored.
• Amusing control group: another GPU-accelerated terminal running on the same desktop had its in-flight job killed during the same test and came out with partially broken rendering — illustrating the failure mode this PR fixes.