Skip to content

Conversation

zojize
Copy link
Contributor

@zojize zojize commented Oct 14, 2025

fixes #708

This PR fixes the flaky test observed in the issue above. Its flakiness was due to comlink's worker function proxy, which turned clearEventListeners into an async function, but was not properly awaited in the test code, and its type definition didn't define it as an async function. By awaiting the clearEventListeners function, then running the assertions, I was able to eliminate flakiness caused by the event listeners. To verify the correctness I used a AI generated bash script that runs the test case a number of times, records the failures, and calculates the #passes/total runs.

#!/usr/bin/env bash

set -euo pipefail

TEST_FILE="example05-event-recording.test.tsx"
CMD=(bun test "$TEST_FILE")
ITERATIONS=500

INDEX_LOG="test-failures.log"
FAIL_PREFIX="test-failure"

passes=0
fails=0

printf '# Run log: %s\n' "$(date '+%Y-%m-%dT%H:%M:%S%z')" > "$INDEX_LOG"

for i in $(seq 1 $ITERATIONS); do
  printf 'Run %d/%d... ' "$i" "$ITERATIONS"
  out=$(mktemp 2>/dev/null) || out=$(mktemp -t run)

  if "${CMD[@]}" >"$out" 2>&1; then
    passes=$((passes+1))
    printf 'PASS\n'
    rm -f "$out"
  else
    fails=$((fails+1))
    per_file="${FAIL_PREFIX}-${i}.log"
    mv "$out" "$per_file" || (cat "$out" > "$per_file" && rm -f "$out")
    printf 'FAIL -> %s\n' "$per_file"
    printf -- '--- Run %d failed at %s -> %s ---\n' "$i" "$(date '+%Y-%m-%dT%H:%M:%S%z')" "$per_file" >> "$INDEX_LOG"
  fi
done

printf '\nResult: %d/%d passes\n' "$passes" "$ITERATIONS"
[ $fails -gt 0 ] && printf 'Failed runs logged: %s\n' "$INDEX_LOG"

exit 0

Running the command above after this patch I get:

Result: 100/100 passes

Before this patch, and removing test.skip I get:

Result: 98/100 passes
Failed runs logged: test-failures.log

Where both failures looked like the screenshot on #708, and sometimes it flakes at a much higher rate. I have locally run more (1000+) iterations comparing both versions and am comfortable saying that this fixes the flakiness.

Additional Notes

  • It might be ideal to use AbortSignals for registering and removing event listeners in entrypoint.ts to simplify implementation.
  • Very rarely, I ran into bun segfaults that crashed the tests, (Worker crashed oven-sh/bun#23604) was an example of one of the crashes I experienced. Since I experienced this before and after the changes I made, I conclude that it was not because of my modifications, but still an interesting find.

More Details on Crash

bun test v1.3.0 (b0a6feca)

tests/examples/example05-event-recording.test.tsx:
============================================================
Bun v1.3.0 (b0a6feca) macOS Silicon
macOS v15.6.1
CPU: fp aes crc32 atomics
Args: "bun" "test" "example05-event-recording.test.tsx"
Features: Bun.stderr(4) bunfig fetch jsc transpiler_cache(40) tsconfig(11) tsconfig_paths(5) workers_spawned napi_module_register process_dlopen 
Builtins: "bun:main" "node:assert" "node:child_process" "node:constants" "node:crypto" "node:events" "node:fs" "node:os" "node:path" "node:stream" "node:tty" "node:util" 
Elapsed: 472ms | User: 358ms | Sys: 50ms
RSS: 0.25GB | Peak: 0.25GB | Commit: 0.87GB | Faults: 72 | Machine: 0.07TB

panic(main thread): Segmentation fault at address 0x440
oh no: Bun has crashed. This indicates a bug in Bun, not your code.

To send a redacted crash report to Bun's team,
please file a GitHub issue using the link below:

 https://bun.report/1.3.0/Mt1b0a6fec41BikkQ_u8zoqBu8zoqB2ivoqBm+qnqB+yl1pB___m7mt+C+xqoiCmph0qCmyj2wCuz5+kB__m7mt+C+xqoiC21+1nCuuirmBA2AgkC

@seveibar seveibar merged commit 20fb9e5 into tscircuit:main Oct 22, 2025
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix flaky test

2 participants