mirror of
https://github.com/garrytan/gbrain.git
synced 2026-08-17 18:32:41 +00:00
Garry TanandClaude Opus 4.7
8ffaa77b55
The E2E suite was flaky. ~3 of every 5 runs had 4-10 failures clustered in Links, Timeline, Versions, Minions resilience, Parallel Import, and Page CRUD tests. Symptoms included "expected 16 pages, got 8" (half), "expected 1 link inserted, got 0", timeline entries missing after round-trip, and similar data-shape mismatches. Root cause: bun test runs test FILES in parallel (each in a worker process). 13 E2E files share one DATABASE_URL, and `setupDB()` in `test/e2e/helpers.ts` does `TRUNCATE ... CASCADE` on all tables before each file's `importFixtures()`. File A's TRUNCATE would race with file B's in-flight INSERT stream, producing the observed half-populated or wrong-count states. An earlier attempt used a Postgres advisory lock held on a dedicated single-connection client for the lifetime of each file's run. It broke because bun's default 5000 ms hook timeout fires on queued beforeAll() calls: with 13 files serializing through the lock, files 2-13 would time out waiting for file 1 to finish. This commit switches to sequential file execution at the harness level via scripts/run-e2e.sh, which loops through test/e2e/*.test.ts one at a time, tracks aggregate pass/fail counts, and exits non-zero on the first failing file. No lock, no timeout issues, no changes to any test file. package.json test:e2e points at the new script. Verified: 5 back-to-back runs against the same Postgres container, each completing in ~5 min. Every run: 13 files, 138 tests, 0 fails. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
67 lines
2.2 KiB
Bash
Executable File
67 lines
2.2 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Run E2E tests ONE FILE AT A TIME.
|
|
#
|
|
# Bun's default is to run test files in parallel (each in its own worker).
|
|
# Our E2E suite shares one Postgres database across all 13 files, and
|
|
# `setupDB()` does TRUNCATE CASCADE + fixture import. When files run in
|
|
# parallel, file A's TRUNCATE can race with file B's fixture import,
|
|
# producing observed fails like "expected 16 pages, got 8", missing
|
|
# links, orphaned timeline entries, etc. The flakiness was visible on
|
|
# ~3 of every 5 runs pre-fix.
|
|
#
|
|
# Running files sequentially eliminates the race entirely. It also costs
|
|
# some startup overhead (each file spins up a fresh bun process) but for
|
|
# a suite this size that is measured in ~1-2s per file, amortized under
|
|
# the natural per-file test time of 5-10s.
|
|
#
|
|
# Exits non-zero on the first failing file so CI fails fast.
|
|
|
|
set -euo pipefail
|
|
|
|
cd "$(dirname "$0")/.."
|
|
|
|
pass_files=0
|
|
fail_files=0
|
|
fail_list=()
|
|
total_pass=0
|
|
total_fail=0
|
|
|
|
for f in test/e2e/*.test.ts; do
|
|
name=$(basename "$f")
|
|
echo ""
|
|
echo "=== $name ==="
|
|
if output=$(bun test "$f" 2>&1); then
|
|
pass_files=$((pass_files + 1))
|
|
# Extract pass/fail counts from bun's summary (e.g., "123 pass")
|
|
p=$(echo "$output" | grep -oE '[0-9]+ pass' | tail -1 | grep -oE '[0-9]+' || echo 0)
|
|
total_pass=$((total_pass + p))
|
|
echo "$output" | tail -8
|
|
else
|
|
fail_files=$((fail_files + 1))
|
|
fail_list+=("$name")
|
|
p=$(echo "$output" | grep -oE '[0-9]+ pass' | tail -1 | grep -oE '[0-9]+' || echo 0)
|
|
fl=$(echo "$output" | grep -oE '[0-9]+ fail' | tail -1 | grep -oE '[0-9]+' || echo 0)
|
|
total_pass=$((total_pass + p))
|
|
total_fail=$((total_fail + fl))
|
|
echo "$output"
|
|
echo ""
|
|
echo "FAILED: $name"
|
|
# Continue so we see all failures; exit nonzero at the end.
|
|
fi
|
|
done
|
|
|
|
echo ""
|
|
echo "========================================"
|
|
echo "E2E SUMMARY (sequential execution)"
|
|
echo "========================================"
|
|
echo "Files: $((pass_files + fail_files)) total, $pass_files passed, $fail_files failed"
|
|
echo "Tests: $total_pass passed, $total_fail failed"
|
|
if [ ${#fail_list[@]} -gt 0 ]; then
|
|
echo ""
|
|
echo "Failing files:"
|
|
for f in "${fail_list[@]}"; do
|
|
echo " - $f"
|
|
done
|
|
exit 1
|
|
fi
|