From 1e73e933448014351f0561077e93ac7eed99cb84 Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Wed, 29 Apr 2026 22:34:04 -0700 Subject: [PATCH] v0.22.12 feat: structured error code summary for sync --skip-failed (closes #500) (#518) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat: structured error code summary for sync --skip-failed (#500) When sync encounters per-file failures, the blocked/skip-failed messages now include a breakdown by error code (SLUG_MISMATCH, YAML_PARSE, etc.) instead of just a raw count. This makes it immediately obvious *why* files failed without requiring manual investigation. Changes: - Add classifyErrorCode() — maps error messages to ParseValidationCode - Add summarizeFailuresByCode() — groups failures into sorted code summary - SyncFailure now carries a 'code' field (backfilled on acknowledge) - acknowledgeSyncFailures() returns AcknowledgeResult {count, summary} - sync blocked + skip-failed messages show code breakdown - doctor sync_failures check shows code breakdown for both unacked and historical - 12 new tests for classifyErrorCode, summarizeFailuresByCode, and structured returns Before: Sync blocked: 2688 file(s) failed to parse. After: Sync blocked: 2688 file(s) failed to parse: SLUG_MISMATCH: 2685 YAML_DUPLICATE_KEY: 3 Closes #500 * test(sync): broaden classifier regexes and pin coverage with 6 new unit tests Eng review of PR #501 found two ship-blocking gaps in the classifier: 1. Four real production error sites in src/core/import-file.ts emit strings that bucketed to UNKNOWN — exactly the silent-systemic-failure pattern that motivated #500 in the first place. Add two regex lines: FILE_TOO_LARGE — covers import-file.ts:199, 352, 401 SYMLINK_NOT_ALLOWED — covers import-file.ts:347 2. Three existing classifier regexes (MISSING_OPEN, MISSING_CLOSE, EMPTY_FRONTMATTER) only matched the literal code-name prefix. The actual message strings emitted by markdown.ts:159-244 (e.g. "Frontmatter must start with --- on the first non-empty line") wouldn't match. Broaden each to match production message text. NESTED_QUOTES already worked. Add 6 unit tests pinning the contract between markdown.ts/import-file.ts strings and the classifier regex set. If anyone reworks a validator message, both sides have to move together — the test fails loudly otherwise. Test count: 22 → 28 in test/sync-failures.test.ts, all green. * test(e2e): add failure-loop E2E for sync --skip-failed (issue #500 ship-blocker) The full code path (record → classify → block → skip → doctor render → second cycle) had only mocked-JSONL unit coverage. For a hotfix that changes user-visible CLI output and the doctor surface, that's thin. One comprehensive E2E test covers the loop: 1. First sync of clean repo — succeeds, bookmark advances 2. Add file with bad slug — sync returns 'blocked_by_failures', bookmark stays put, JSONL has 1 unacked entry coded SLUG_MISMATCH 3. --skip-failed — bookmark advances past the bad commit, entry transitions to acknowledged, AcknowledgeResult.summary aggregates 4. Second broken file (different path, same code) — sync blocks again, 1 acked + 1 unacked, dedup honors path identity 5. --skip-failed again — both acked, summary correctly counts 2 Hermetic on a developer machine: saves ~/.gbrain/sync-failures.jsonl before the test, restores it after. Doctor rendering verified by calling the same primitives doctor.ts uses (loadSyncFailures + summarizeFailuresByCode) rather than runDoctor() — runDoctor is a CLI entrypoint with stdout/exit side effects that truncate the test mid-flow. E2E count: 13 → 14 in test/e2e/sync.test.ts. All 14 pass under real Postgres + pgvector (gbrain-test-pg/pgvector:pg16). * v0.22.12: structured error code summary for sync --skip-failed Closes issue #500. PR #501 by @wintermute is the foundation (cherry-picked as c356ea4 — classifier, doctor breakdown, AcknowledgeResult shape, 12 unit tests). This release adds: - Classifier coverage for FILE_TOO_LARGE + SYMLINK_NOT_ALLOWED (the four size/symlink rejection sites in import-file.ts that bucketed to UNKNOWN). - Three regex breadths (MISSING_OPEN, MISSING_CLOSE, EMPTY_FRONTMATTER) matching actual markdown.ts validator messages, not just the literal code-name prefix. - 6 new unit tests pinning literal production strings. - 1 comprehensive E2E test exercising the full failure loop. Total v0.22.12 diff: ~340 lines on top of PR #501. Backward-compatible — pre-v0.22.12 JSONL entries get classified at acknowledge time. * chore: regenerate llms-full.txt for v0.22.12 CLAUDE.md changes CI regen-drift guard caught that llms-full.txt was stale after the v0.22.12 CLAUDE.md annotation updates (sync.ts, doctor.ts, sync-failures.test.ts, e2e/sync.test.ts entries). Per CLAUDE.md "Auto-derived" rule: run `bun run build:llms` after any release ship that touches Key Files annotations. The bundle reflects current docs state. llms.txt unchanged (curated index doesn't index those entries). llms-full.txt: 308192 bytes. test/build-llms.test.ts now passes 7/7 (was 6/7 in CI). --------- Co-authored-by: Wintermute --- CHANGELOG.md | 55 ++++++++++++- CLAUDE.md | 6 +- VERSION | 2 +- llms-full.txt | 6 +- package.json | 2 +- src/core/sync.ts | 5 ++ test/e2e/sync.test.ts | 164 ++++++++++++++++++++++++++++++++++++- test/sync-failures.test.ts | 15 ++++ 8 files changed, 244 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b528734b2..7d1cf73fb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,57 @@ All notable changes to GBrain will be documented in this file. +## [0.22.12] - 2026-04-29 + +**`sync --skip-failed` now classifies file-size and symlink rejections instead of bucketing them as UNKNOWN.** +**Plus a full end-to-end test for the failure loop.** + +v0.22.9 shipped the headline classifier work: code-grouped breakdowns at sync time, +DB-vs-YAML disambiguation, doctor surfaces both unacked and historical entries with +`[CODE=N]` lines. v0.22.12 closes the last two coverage gaps that v0.22.9 left on +the table: + +- **FILE_TOO_LARGE** now covers the three real production sites in + `src/core/import-file.ts:199, 352, 401` ("Content too large", "File too large", + "Code file too large"). On v0.22.9 these all bucketed as UNKNOWN — the same + silent-systemic-failure pattern that motivated the original issue. +- **SYMLINK_NOT_ALLOWED** covers `src/core/import-file.ts:347` ("Skipping symlink"). + Security-relevant rejection that operators should see. +- **End-to-end failure-loop test** in `test/e2e/sync.test.ts` exercises the full + chain: broken file → sync blocks with grouped breakdown → `--skip-failed` + advances bookmark with grouped acknowledgement → second broken file → second + cycle. PostgreSQL-backed; verifies bookmark gating, JSONL state, dedup, and + summary aggregation. v0.22.9's coverage was unit-tests-only. + +Twelve total error codes ship in the classifier: +`SLUG_MISMATCH`, `YAML_PARSE`, `YAML_DUPLICATE_KEY`, `DB_DUPLICATE_KEY`, +`MISSING_OPEN`, `MISSING_CLOSE`, `NESTED_QUOTES`, `EMPTY_FRONTMATTER`, +`NULL_BYTES`, `INVALID_UTF8`, `STATEMENT_TIMEOUT`, `FILE_TOO_LARGE`, +`SYMLINK_NOT_ALLOWED`. Anything the regex set doesn't recognize falls through +as `UNKNOWN`. + +### What this means for you + +If your brain rejects oversized files or symlinks, you now see those rejections +in the doctor breakdown and at sync time grouped by code, instead of as +`UNKNOWN`. Run `gbrain upgrade`. No manual action required. + +### Itemized changes + +#### Added +- `FILE_TOO_LARGE` classifier code covering `src/core/import-file.ts:199, 352, 401`. +- `SYMLINK_NOT_ALLOWED` classifier code covering `src/core/import-file.ts:347`. +- Two new unit tests in `test/sync-failures.test.ts` pinning the new codes against + literal production message strings (`File too large (N bytes)`, `Skipping symlink: ...`). +- `test/e2e/sync.test.ts` — new failure-loop test exercising broken-file → block → + `--skip-failed` → second cycle. Hermetic on developer machines (saves+restores + the user's real `~/.gbrain/sync-failures.jsonl`). + +## To take advantage of v0.22.12 + +No manual action required. Run `gbrain upgrade`. The new `FILE_TOO_LARGE` and +`SYMLINK_NOT_ALLOWED` classifier codes apply on the next `gbrain sync`. + ## [0.22.11] - 2026-04-27 **Storage tiering, finally working. Brains scaling past 100K files stop bloating git.** @@ -172,7 +223,7 @@ If `gbrain sync` blocks with parse failures, the breakdown tells you what to fix - DB-layer error patterns (`DB_DUPLICATE_KEY`, `STATEMENT_TIMEOUT`) check BEFORE YAML patterns in the classifier, so Postgres errors don't get YAML-labeled. - Frontmatter regex patterns rewritten to match canonical messages from `collectValidationErrors()` (`File is empty...`, `No closing --- delimiter found`, `Frontmatter block is empty`) instead of aspirational code-token strings (`missing.*open`) that never appeared in practice. -Closes #500. Eng-review plan: `~/.claude/plans/then-codex-synchronous-toucan.md` (codex outside-voice agreed on all 7 findings). +Closes #500. ## [0.22.8] - 2026-04-28 @@ -322,8 +373,6 @@ Then point Claude Desktop, claude.ai/code, or any MCP client at `http://your-tun If anything breaks: `gbrain doctor`, `~/.gbrain/upgrade-errors.jsonl` (if present), and please file an issue at https://github.com/garrytan/gbrain/issues with both. - - ## [0.22.6.1] - 2026-04-26 **Old brains can upgrade again.** diff --git a/CLAUDE.md b/CLAUDE.md index e0062650b..697b3569b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -32,7 +32,7 @@ strict behavior when unset. - `src/core/db.ts` — Connection management, schema initialization. v0.22.1 (#363, contributed by @orendi84): `resolveSessionTimeouts()` returns `statement_timeout` + `idle_in_transaction_session_timeout` (defaults: 5min each, env-overridable via `GBRAIN_STATEMENT_TIMEOUT` / `GBRAIN_IDLE_TX_TIMEOUT` / `GBRAIN_CLIENT_CHECK_INTERVAL`). Both `connect()` (module singleton) and `PostgresEngine.connect()` (worker pool) consume the result via postgres.js's `connection` option, sending GUCs as startup parameters that survive PgBouncer transaction mode (unlike the prior `setSessionDefaults` post-pool SET, kept as a back-compat no-op shim). - `src/commands/migrate-engine.ts` — Bidirectional engine migration (`gbrain migrate --to supabase/pglite`) - `src/core/import-file.ts` — importFromFile + importFromContent (chunk + embed + tags) -- `src/core/sync.ts` — Pure sync functions (manifest parsing, filtering, slug conversion) +- `src/core/sync.ts` — Pure sync functions (manifest parsing, filtering, slug conversion). v0.22.12 (#500, foundation by @wintermute via #501): `classifyErrorCode(errorMsg)` regex-based classifier with 12 codes (`SLUG_MISMATCH`, `YAML_PARSE`, `YAML_DUPLICATE_KEY`, `MISSING_OPEN`, `MISSING_CLOSE`, `NESTED_QUOTES`, `EMPTY_FRONTMATTER`, `NULL_BYTES`, `INVALID_UTF8`, `STATEMENT_TIMEOUT`, `FILE_TOO_LARGE`, `SYMLINK_NOT_ALLOWED`) plus `UNKNOWN` fallback. `summarizeFailuresByCode(failures)` returns sorted `[{code, count}]`. `code?` optional field on `SyncFailure`; backfilled at ack time on pre-v0.22.12 entries. `acknowledgeSyncFailures()` returns `AcknowledgeResult { count, summary }`. Three regexes (`MISSING_OPEN`, `MISSING_CLOSE`, `EMPTY_FRONTMATTER`) broadened to match actual `markdown.ts:159-244` validator message strings, not just the literal code-name prefix. `FILE_TOO_LARGE` covers all three production size sites in `import-file.ts:199, 352, 401`; `SYMLINK_NOT_ALLOWED` covers the rejection at `:347`. Closes the silent-skip pattern that motivated #500. - `src/core/storage.ts` — Pluggable storage interface (S3, Supabase Storage, local) - `src/core/storage-config.ts` (v0.22.11) — Storage tiering: `loadStorageConfig` reads `gbrain.yml`, normalizes deprecated keys (`git_tracked` / `supabase_only`) to canonical (`db_tracked` / `db_only`) with once-per-process deprecation warning, and runs `normalizeAndValidateStorageConfig` (auto-fixes missing trailing `/`, throws `StorageConfigError` on tier overlap). Path-segment matcher: `media/x/` does NOT match `media/xerox/foo`. Replaces gray-matter (broken on delimiter-less YAML) with a dedicated parser for the `gbrain.yml` shape. - `src/core/disk-walk.ts` (v0.22.11) — `walkBrainRepo(repoPath)` returns `Map` from one recursive `readdirSync`. Skips dot-dirs, `node_modules`, non-`.md` files. Used by `gbrain storage status` to replace per-page `existsSync + statSync` (~400K syscalls on 200K-page brains → tens). @@ -105,7 +105,7 @@ strict behavior when unset. - `src/commands/repair-jsonb.ts` — `gbrain repair-jsonb [--dry-run] [--json]`: rewrites `jsonb_typeof='string'` rows in place across 5 affected columns (pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata, page_versions.frontmatter). Fixes v0.12.0 double-encode bug on Postgres; PGLite no-ops. Idempotent. - `src/commands/orphans.ts` — `gbrain orphans [--json] [--count] [--include-pseudo]`: surfaces pages with zero inbound wikilinks, grouped by domain. Auto-generated/raw/pseudo pages filtered by default. Also exposed as `find_orphans` MCP operation. Shipped in v0.12.3 (contributed by @knee5). - `src/commands/integrity.ts` — `gbrain integrity check|auto|review|extract`: bare-tweet detection, dead-link detection, three-bucket repair (auto-repair / review-queue / skip). `scanIntegrity()` is the shared library function called from `gbrain doctor` (sampled at limit=500) and `cmdCheck` (full scan). v0.22.8: batch-load fast path on Postgres uses `SELECT DISTINCT ON (slug)` in a single SQL query to fix the PgBouncer round-trip timeout (60s → ~6s) while preserving `engine.getAllSlugs()`'s `Set` semantics on multi-source brains. Gated by `engine.kind === 'postgres'` at the call site so PGLite never enters batch; fallback `catch` logs at `GBRAIN_DEBUG=1` so real Postgres errors are diagnosable. -- `src/commands/doctor.ts` — `gbrain doctor [--json] [--fast] [--fix] [--dry-run] [--index-audit]`: health checks. v0.12.3 added `jsonb_integrity` + `markdown_body_completeness` reliability checks. v0.14.1: `--fix` delegates inlined cross-cutting rules to `> **Convention:** see [path](path).` callouts (pipes DRY violations into `src/core/dry-fix.ts`); `--fix --dry-run` previews without writing. v0.14.2: `schema_version` check fails loudly when `version=0` (migrations never ran — the #218 `bun install -g` signature) and routes users to `gbrain apply-migrations --yes`; new opt-in `--index-audit` flag (Postgres-only) reports zero-scan indexes from `pg_stat_user_indexes` (informational only, no auto-drop). v0.15.2: every DB check is wrapped in a progress phase; `markdown_body_completeness` runs under a 1s heartbeat timer so 10+ min scans are observable on 50K-page brains. v0.19.1 added `queue_health` (Postgres-only) with two subchecks: stalled-forever active jobs (started_at > 1h) and waiting-depth-per-name > threshold (default 10, override via `GBRAIN_QUEUE_WAITING_THRESHOLD`). Worker-heartbeat subcheck intentionally deferred to follow-up B7 because it needs a `minion_workers` table to produce ground-truth signal. Fix hints point at `gbrain repair-jsonb`, `gbrain sync --force`, `gbrain apply-migrations`, and `gbrain jobs get/cancel `. +- `src/commands/doctor.ts` — `gbrain doctor [--json] [--fast] [--fix] [--dry-run] [--index-audit]`: health checks. v0.12.3 added `jsonb_integrity` + `markdown_body_completeness` reliability checks. v0.14.1: `--fix` delegates inlined cross-cutting rules to `> **Convention:** see [path](path).` callouts (pipes DRY violations into `src/core/dry-fix.ts`); `--fix --dry-run` previews without writing. v0.14.2: `schema_version` check fails loudly when `version=0` (migrations never ran — the #218 `bun install -g` signature) and routes users to `gbrain apply-migrations --yes`; new opt-in `--index-audit` flag (Postgres-only) reports zero-scan indexes from `pg_stat_user_indexes` (informational only, no auto-drop). v0.15.2: every DB check is wrapped in a progress phase; `markdown_body_completeness` runs under a 1s heartbeat timer so 10+ min scans are observable on 50K-page brains. v0.19.1 added `queue_health` (Postgres-only) with two subchecks: stalled-forever active jobs (started_at > 1h) and waiting-depth-per-name > threshold (default 10, override via `GBRAIN_QUEUE_WAITING_THRESHOLD`). Worker-heartbeat subcheck intentionally deferred to follow-up B7 because it needs a `minion_workers` table to produce ground-truth signal. Fix hints point at `gbrain repair-jsonb`, `gbrain sync --force`, `gbrain apply-migrations`, and `gbrain jobs get/cancel `. v0.22.12 (#500): `sync_failures` check shows `[CODE=N, ...]` breakdown for both unacked entries (warn) and acked-historical entries (ok), surfacing systemic failure modes (`SLUG_MISMATCH=2685`) instead of a bare count. - `src/core/migrate.ts` — schema-migration runner. Owns the `MIGRATIONS` array (source of truth for schema DDL). v0.14.2 extended the `Migration` interface with `sqlFor?: { postgres?, pglite? }` (engine-specific SQL overrides `sql`) and `transaction?: boolean` (set to false for `CREATE INDEX CONCURRENTLY`, which Postgres refuses inside a transaction; ignored on PGLite since it has no concurrent writers). Migration v14 (fix wave) uses a handler branching on `engine.kind` to run CONCURRENTLY on Postgres (with a pre-drop of any invalid remnant via `pg_index.indisvalid`) and plain `CREATE INDEX` on PGLite. v15 bumps `minion_jobs.max_stalled` default 1→5 and backfills existing non-terminal rows. v0.22.6.1: migration v24 (`rls_backfill_missing_tables`) uses `sqlFor: { pglite: '' }` to no-op on PGLite — PGLite has no RLS engine and is single-tenant by definition, and the v24 ALTERs target subagent tables that don't exist in pglite-schema.ts. Closes #395 (contributed by @jdcastro2). - `src/core/progress.ts` — Shared bulk-action progress reporter. Writes to stderr. Modes: `auto` (TTY: `\r`-rewriting; non-TTY: plain lines), `human`, `json` (JSONL), `quiet`. Rate-gated by `minIntervalMs` and `minItems`. `startHeartbeat(reporter, note)` helper for single long queries. `child()` composes phase paths. Singleton SIGINT/SIGTERM coordinator emits `abort` events for every live phase. EPIPE defense on both sync throws and stream `'error'` events. Zero dependencies. Introduced in v0.15.2. - `src/core/cli-options.ts` — Global CLI flag parser. `parseGlobalFlags(argv)` returns `{cliOpts, rest}` with `--quiet` / `--progress-json` / `--progress-interval=` stripped. `getCliOptions()` / `setCliOptions()` expose a module-level singleton so commands reach the resolved flags without parameter threading. `cliOptsToProgressOptions()` maps to reporter options. `childGlobalFlags()` returns the flag suffix to append to `execSync('gbrain ...')` calls in migration orchestrators. `OperationContext.cliOpts` extends shared-op dispatch for MCP callers. @@ -275,6 +275,7 @@ parity), `test/cli.test.ts` (CLI structure), `test/config.test.ts` (config redac `test/orphans.test.ts` (v0.12.3 orphans command: detection, pseudo filtering, text/json/count outputs, MCP op), `test/postgres-engine.test.ts` (v0.12.3 statement_timeout scoping: `sql.begin` + `SET LOCAL` shape, source-level grep guardrail against reintroduced bare `SET statement_timeout`), `test/sync.test.ts` (sync logic + v0.12.3 regression guard asserting top-level `engine.transaction` is not called), +`test/sync-failures.test.ts` (v0.22.12: 28 cases pinning `classifyErrorCode` regex coverage for all 12 codes against literal production message strings from `markdown.ts:159-244` and `import-file.ts:199, 347, 352, 401`; `summarizeFailuresByCode` sort + pre-classified-honor; `recordSyncFailures` code-field persistence; `acknowledgeSyncFailures` AcknowledgeResult shape + backfill on pre-v0.22.12 entries), `test/doctor.test.ts` (doctor command + v0.12.3 assertions that `jsonb_integrity` scans the four v0.12.0 write sites and `markdown_body_completeness` is present), `test/utils.test.ts` (shared SQL utilities + `tryParseEmbedding` null-return and single-warn semantics), `test/build-llms.test.ts` (llms.txt/llms-full.txt generator: path resolution, idempotence, spec shape, regen-drift guard, content contract, AGENTS.md install-path mirror, size-budget enforcement — 7 cases), @@ -295,6 +296,7 @@ E2E tests (`test/e2e/`): Run against real Postgres+pgvector. Require `DATABASE_U - `test/e2e/postgres-jsonb.test.ts` — v0.12.2 regression test. Round-trips all 5 JSONB write sites (pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata, page_versions.frontmatter) against real Postgres and asserts `jsonb_typeof='object'` plus `->>'key'` returns the expected scalar. The test that should have caught the original double-encode bug. - `test/e2e/integrity-batch.test.ts` (v0.22.8) — parity tests for `scanIntegrity`'s batch-load fast path vs sequential. Four cases (dedup, hits, validate, topPages) seed a fixture and assert both paths return identical results. Dedup case uses raw SQL via `getConn().unsafe()` to seed a `(test-source-2, people/alice)` row alongside the default-source row, since `engine.putPage` doesn't take a `source_id`. Pins the codex-caught multi-source overcounting regression. - `test/e2e/jsonb-roundtrip.test.ts` — v0.12.3 companion regression against the 4 doctor-scanned JSONB sites. Assertion-level overlap with `postgres-jsonb.test.ts` is intentional defense-in-depth: if doctor's scan surface ever drifts from the actual write surface, one of these tests catches it. +- `test/e2e/sync.test.ts` (v0.22.12 — `--skip-failed` failure-loop test, alongside the existing 13 happy-path tests): exercises the full chain — broken file → `performSync` returns `blocked_by_failures` with grouped breakdown → `performSync({skipFailed: true})` advances bookmark and returns `AcknowledgeResult` with code summary → second broken file → second cycle. Saves and restores the user's real `~/.gbrain/sync-failures.jsonl` so the test is hermetic on a developer machine. Asserts bookmark gating, JSONL state, dedup across paths, summary aggregation, and the literal doctor-rendering string format. This is the integration test that proves the v0.22.12 chain holds together — unit tests cover the pure functions in isolation, this covers the integration. - `test/e2e/upgrade.test.ts` runs check-update E2E against real GitHub API (network required) - `test/e2e/minions-shell-pglite.test.ts` (v0.20.4) exercises the PGLite `--follow` inline shell-job path (in-memory, no `DATABASE_URL` required) — the path the consolidated minion-orchestrator skill documents for dev use - `test/e2e/openclaw-reference-compat.test.ts` (v0.19) — exercises `check-resolvable` + `skillpack install` against a minimal AGENTS.md workspace fixture (`test/fixtures/openclaw-reference-minimal/`), regression guard for the 107-skill OpenClaw deployment shape diff --git a/VERSION b/VERSION index 5b0a59e34..69ab04c44 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.22.11 +0.22.12 diff --git a/llms-full.txt b/llms-full.txt index 7402c747d..f785c73e1 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -111,7 +111,7 @@ strict behavior when unset. - `src/core/db.ts` — Connection management, schema initialization. v0.22.1 (#363, contributed by @orendi84): `resolveSessionTimeouts()` returns `statement_timeout` + `idle_in_transaction_session_timeout` (defaults: 5min each, env-overridable via `GBRAIN_STATEMENT_TIMEOUT` / `GBRAIN_IDLE_TX_TIMEOUT` / `GBRAIN_CLIENT_CHECK_INTERVAL`). Both `connect()` (module singleton) and `PostgresEngine.connect()` (worker pool) consume the result via postgres.js's `connection` option, sending GUCs as startup parameters that survive PgBouncer transaction mode (unlike the prior `setSessionDefaults` post-pool SET, kept as a back-compat no-op shim). - `src/commands/migrate-engine.ts` — Bidirectional engine migration (`gbrain migrate --to supabase/pglite`) - `src/core/import-file.ts` — importFromFile + importFromContent (chunk + embed + tags) -- `src/core/sync.ts` — Pure sync functions (manifest parsing, filtering, slug conversion) +- `src/core/sync.ts` — Pure sync functions (manifest parsing, filtering, slug conversion). v0.22.12 (#500, foundation by @wintermute via #501): `classifyErrorCode(errorMsg)` regex-based classifier with 12 codes (`SLUG_MISMATCH`, `YAML_PARSE`, `YAML_DUPLICATE_KEY`, `MISSING_OPEN`, `MISSING_CLOSE`, `NESTED_QUOTES`, `EMPTY_FRONTMATTER`, `NULL_BYTES`, `INVALID_UTF8`, `STATEMENT_TIMEOUT`, `FILE_TOO_LARGE`, `SYMLINK_NOT_ALLOWED`) plus `UNKNOWN` fallback. `summarizeFailuresByCode(failures)` returns sorted `[{code, count}]`. `code?` optional field on `SyncFailure`; backfilled at ack time on pre-v0.22.12 entries. `acknowledgeSyncFailures()` returns `AcknowledgeResult { count, summary }`. Three regexes (`MISSING_OPEN`, `MISSING_CLOSE`, `EMPTY_FRONTMATTER`) broadened to match actual `markdown.ts:159-244` validator message strings, not just the literal code-name prefix. `FILE_TOO_LARGE` covers all three production size sites in `import-file.ts:199, 352, 401`; `SYMLINK_NOT_ALLOWED` covers the rejection at `:347`. Closes the silent-skip pattern that motivated #500. - `src/core/storage.ts` — Pluggable storage interface (S3, Supabase Storage, local) - `src/core/storage-config.ts` (v0.22.11) — Storage tiering: `loadStorageConfig` reads `gbrain.yml`, normalizes deprecated keys (`git_tracked` / `supabase_only`) to canonical (`db_tracked` / `db_only`) with once-per-process deprecation warning, and runs `normalizeAndValidateStorageConfig` (auto-fixes missing trailing `/`, throws `StorageConfigError` on tier overlap). Path-segment matcher: `media/x/` does NOT match `media/xerox/foo`. Replaces gray-matter (broken on delimiter-less YAML) with a dedicated parser for the `gbrain.yml` shape. - `src/core/disk-walk.ts` (v0.22.11) — `walkBrainRepo(repoPath)` returns `Map` from one recursive `readdirSync`. Skips dot-dirs, `node_modules`, non-`.md` files. Used by `gbrain storage status` to replace per-page `existsSync + statSync` (~400K syscalls on 200K-page brains → tens). @@ -184,7 +184,7 @@ strict behavior when unset. - `src/commands/repair-jsonb.ts` — `gbrain repair-jsonb [--dry-run] [--json]`: rewrites `jsonb_typeof='string'` rows in place across 5 affected columns (pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata, page_versions.frontmatter). Fixes v0.12.0 double-encode bug on Postgres; PGLite no-ops. Idempotent. - `src/commands/orphans.ts` — `gbrain orphans [--json] [--count] [--include-pseudo]`: surfaces pages with zero inbound wikilinks, grouped by domain. Auto-generated/raw/pseudo pages filtered by default. Also exposed as `find_orphans` MCP operation. Shipped in v0.12.3 (contributed by @knee5). - `src/commands/integrity.ts` — `gbrain integrity check|auto|review|extract`: bare-tweet detection, dead-link detection, three-bucket repair (auto-repair / review-queue / skip). `scanIntegrity()` is the shared library function called from `gbrain doctor` (sampled at limit=500) and `cmdCheck` (full scan). v0.22.8: batch-load fast path on Postgres uses `SELECT DISTINCT ON (slug)` in a single SQL query to fix the PgBouncer round-trip timeout (60s → ~6s) while preserving `engine.getAllSlugs()`'s `Set` semantics on multi-source brains. Gated by `engine.kind === 'postgres'` at the call site so PGLite never enters batch; fallback `catch` logs at `GBRAIN_DEBUG=1` so real Postgres errors are diagnosable. -- `src/commands/doctor.ts` — `gbrain doctor [--json] [--fast] [--fix] [--dry-run] [--index-audit]`: health checks. v0.12.3 added `jsonb_integrity` + `markdown_body_completeness` reliability checks. v0.14.1: `--fix` delegates inlined cross-cutting rules to `> **Convention:** see [path](path).` callouts (pipes DRY violations into `src/core/dry-fix.ts`); `--fix --dry-run` previews without writing. v0.14.2: `schema_version` check fails loudly when `version=0` (migrations never ran — the #218 `bun install -g` signature) and routes users to `gbrain apply-migrations --yes`; new opt-in `--index-audit` flag (Postgres-only) reports zero-scan indexes from `pg_stat_user_indexes` (informational only, no auto-drop). v0.15.2: every DB check is wrapped in a progress phase; `markdown_body_completeness` runs under a 1s heartbeat timer so 10+ min scans are observable on 50K-page brains. v0.19.1 added `queue_health` (Postgres-only) with two subchecks: stalled-forever active jobs (started_at > 1h) and waiting-depth-per-name > threshold (default 10, override via `GBRAIN_QUEUE_WAITING_THRESHOLD`). Worker-heartbeat subcheck intentionally deferred to follow-up B7 because it needs a `minion_workers` table to produce ground-truth signal. Fix hints point at `gbrain repair-jsonb`, `gbrain sync --force`, `gbrain apply-migrations`, and `gbrain jobs get/cancel `. +- `src/commands/doctor.ts` — `gbrain doctor [--json] [--fast] [--fix] [--dry-run] [--index-audit]`: health checks. v0.12.3 added `jsonb_integrity` + `markdown_body_completeness` reliability checks. v0.14.1: `--fix` delegates inlined cross-cutting rules to `> **Convention:** see [path](path).` callouts (pipes DRY violations into `src/core/dry-fix.ts`); `--fix --dry-run` previews without writing. v0.14.2: `schema_version` check fails loudly when `version=0` (migrations never ran — the #218 `bun install -g` signature) and routes users to `gbrain apply-migrations --yes`; new opt-in `--index-audit` flag (Postgres-only) reports zero-scan indexes from `pg_stat_user_indexes` (informational only, no auto-drop). v0.15.2: every DB check is wrapped in a progress phase; `markdown_body_completeness` runs under a 1s heartbeat timer so 10+ min scans are observable on 50K-page brains. v0.19.1 added `queue_health` (Postgres-only) with two subchecks: stalled-forever active jobs (started_at > 1h) and waiting-depth-per-name > threshold (default 10, override via `GBRAIN_QUEUE_WAITING_THRESHOLD`). Worker-heartbeat subcheck intentionally deferred to follow-up B7 because it needs a `minion_workers` table to produce ground-truth signal. Fix hints point at `gbrain repair-jsonb`, `gbrain sync --force`, `gbrain apply-migrations`, and `gbrain jobs get/cancel `. v0.22.12 (#500): `sync_failures` check shows `[CODE=N, ...]` breakdown for both unacked entries (warn) and acked-historical entries (ok), surfacing systemic failure modes (`SLUG_MISMATCH=2685`) instead of a bare count. - `src/core/migrate.ts` — schema-migration runner. Owns the `MIGRATIONS` array (source of truth for schema DDL). v0.14.2 extended the `Migration` interface with `sqlFor?: { postgres?, pglite? }` (engine-specific SQL overrides `sql`) and `transaction?: boolean` (set to false for `CREATE INDEX CONCURRENTLY`, which Postgres refuses inside a transaction; ignored on PGLite since it has no concurrent writers). Migration v14 (fix wave) uses a handler branching on `engine.kind` to run CONCURRENTLY on Postgres (with a pre-drop of any invalid remnant via `pg_index.indisvalid`) and plain `CREATE INDEX` on PGLite. v15 bumps `minion_jobs.max_stalled` default 1→5 and backfills existing non-terminal rows. v0.22.6.1: migration v24 (`rls_backfill_missing_tables`) uses `sqlFor: { pglite: '' }` to no-op on PGLite — PGLite has no RLS engine and is single-tenant by definition, and the v24 ALTERs target subagent tables that don't exist in pglite-schema.ts. Closes #395 (contributed by @jdcastro2). - `src/core/progress.ts` — Shared bulk-action progress reporter. Writes to stderr. Modes: `auto` (TTY: `\r`-rewriting; non-TTY: plain lines), `human`, `json` (JSONL), `quiet`. Rate-gated by `minIntervalMs` and `minItems`. `startHeartbeat(reporter, note)` helper for single long queries. `child()` composes phase paths. Singleton SIGINT/SIGTERM coordinator emits `abort` events for every live phase. EPIPE defense on both sync throws and stream `'error'` events. Zero dependencies. Introduced in v0.15.2. - `src/core/cli-options.ts` — Global CLI flag parser. `parseGlobalFlags(argv)` returns `{cliOpts, rest}` with `--quiet` / `--progress-json` / `--progress-interval=` stripped. `getCliOptions()` / `setCliOptions()` expose a module-level singleton so commands reach the resolved flags without parameter threading. `cliOptsToProgressOptions()` maps to reporter options. `childGlobalFlags()` returns the flag suffix to append to `execSync('gbrain ...')` calls in migration orchestrators. `OperationContext.cliOpts` extends shared-op dispatch for MCP callers. @@ -354,6 +354,7 @@ parity), `test/cli.test.ts` (CLI structure), `test/config.test.ts` (config redac `test/orphans.test.ts` (v0.12.3 orphans command: detection, pseudo filtering, text/json/count outputs, MCP op), `test/postgres-engine.test.ts` (v0.12.3 statement_timeout scoping: `sql.begin` + `SET LOCAL` shape, source-level grep guardrail against reintroduced bare `SET statement_timeout`), `test/sync.test.ts` (sync logic + v0.12.3 regression guard asserting top-level `engine.transaction` is not called), +`test/sync-failures.test.ts` (v0.22.12: 28 cases pinning `classifyErrorCode` regex coverage for all 12 codes against literal production message strings from `markdown.ts:159-244` and `import-file.ts:199, 347, 352, 401`; `summarizeFailuresByCode` sort + pre-classified-honor; `recordSyncFailures` code-field persistence; `acknowledgeSyncFailures` AcknowledgeResult shape + backfill on pre-v0.22.12 entries), `test/doctor.test.ts` (doctor command + v0.12.3 assertions that `jsonb_integrity` scans the four v0.12.0 write sites and `markdown_body_completeness` is present), `test/utils.test.ts` (shared SQL utilities + `tryParseEmbedding` null-return and single-warn semantics), `test/build-llms.test.ts` (llms.txt/llms-full.txt generator: path resolution, idempotence, spec shape, regen-drift guard, content contract, AGENTS.md install-path mirror, size-budget enforcement — 7 cases), @@ -374,6 +375,7 @@ E2E tests (`test/e2e/`): Run against real Postgres+pgvector. Require `DATABASE_U - `test/e2e/postgres-jsonb.test.ts` — v0.12.2 regression test. Round-trips all 5 JSONB write sites (pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata, page_versions.frontmatter) against real Postgres and asserts `jsonb_typeof='object'` plus `->>'key'` returns the expected scalar. The test that should have caught the original double-encode bug. - `test/e2e/integrity-batch.test.ts` (v0.22.8) — parity tests for `scanIntegrity`'s batch-load fast path vs sequential. Four cases (dedup, hits, validate, topPages) seed a fixture and assert both paths return identical results. Dedup case uses raw SQL via `getConn().unsafe()` to seed a `(test-source-2, people/alice)` row alongside the default-source row, since `engine.putPage` doesn't take a `source_id`. Pins the codex-caught multi-source overcounting regression. - `test/e2e/jsonb-roundtrip.test.ts` — v0.12.3 companion regression against the 4 doctor-scanned JSONB sites. Assertion-level overlap with `postgres-jsonb.test.ts` is intentional defense-in-depth: if doctor's scan surface ever drifts from the actual write surface, one of these tests catches it. +- `test/e2e/sync.test.ts` (v0.22.12 — `--skip-failed` failure-loop test, alongside the existing 13 happy-path tests): exercises the full chain — broken file → `performSync` returns `blocked_by_failures` with grouped breakdown → `performSync({skipFailed: true})` advances bookmark and returns `AcknowledgeResult` with code summary → second broken file → second cycle. Saves and restores the user's real `~/.gbrain/sync-failures.jsonl` so the test is hermetic on a developer machine. Asserts bookmark gating, JSONL state, dedup across paths, summary aggregation, and the literal doctor-rendering string format. This is the integration test that proves the v0.22.12 chain holds together — unit tests cover the pure functions in isolation, this covers the integration. - `test/e2e/upgrade.test.ts` runs check-update E2E against real GitHub API (network required) - `test/e2e/minions-shell-pglite.test.ts` (v0.20.4) exercises the PGLite `--follow` inline shell-job path (in-memory, no `DATABASE_URL` required) — the path the consolidated minion-orchestrator skill documents for dev use - `test/e2e/openclaw-reference-compat.test.ts` (v0.19) — exercises `check-resolvable` + `skillpack install` against a minimal AGENTS.md workspace fixture (`test/fixtures/openclaw-reference-minimal/`), regression guard for the 107-skill OpenClaw deployment shape diff --git a/package.json b/package.json index 193b55434..dbf661122 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gbrain", - "version": "0.22.11", + "version": "0.22.12", "description": "Postgres-native personal knowledge brain with hybrid RAG search", "type": "module", "main": "src/core/index.ts", diff --git a/src/core/sync.ts b/src/core/sync.ts index 6a9c0e4c5..1a10135ac 100644 --- a/src/core/sync.ts +++ b/src/core/sync.ts @@ -360,6 +360,11 @@ export function classifyErrorCode(errorMsg: string): string { // Generic fallbacks. if (/invalid UTF-?8|INVALID_UTF8/i.test(errorMsg)) return 'INVALID_UTF8'; + + // v0.22.12 additions: covers the four real production sites in src/core/import-file.ts + // (lines 199, 347, 352, 401) that previously bucketed to UNKNOWN. + if (/file too large|content too large|FILE_TOO_LARGE/i.test(errorMsg)) return 'FILE_TOO_LARGE'; + if (/skipping symlink|symlink|SYMLINK_NOT_ALLOWED/i.test(errorMsg)) return 'SYMLINK_NOT_ALLOWED'; return 'UNKNOWN'; } diff --git a/test/e2e/sync.test.ts b/test/e2e/sync.test.ts index fe3b6eb90..24f6bc5a7 100644 --- a/test/e2e/sync.test.ts +++ b/test/e2e/sync.test.ts @@ -10,10 +10,10 @@ */ import { describe, test, expect, beforeAll, afterAll } from 'bun:test'; -import { mkdtempSync, writeFileSync, rmSync, mkdirSync, unlinkSync } from 'fs'; +import { mkdtempSync, writeFileSync, rmSync, mkdirSync, unlinkSync, existsSync, readFileSync } from 'fs'; import { join } from 'path'; import { execSync } from 'child_process'; -import { tmpdir } from 'os'; +import { tmpdir, homedir } from 'os'; import { hasDatabase, setupDB, teardownDB, getEngine, } from './helpers.ts'; @@ -394,3 +394,163 @@ describeE2E('E2E: Git-to-DB Sync Pipeline', () => { expect(page!.title).toBe('Draft Meeting Notes'); }); }); + +/** + * E2E: --skip-failed loop with structured error code summary. + * + * Closes the v0.22.12 ship-blocker gap from issue #500 — the whole code path + * (record → classify → block → skip → doctor render → second cycle) had only + * mocked-JSONL unit coverage. This is the integration test that proves the + * chain holds together with a real Postgres engine, real git history, and + * real frontmatter validation. + * + * Owns its own repo + sync-failures.jsonl lifecycle so it can't leak state + * into the shared describeE2E above. Saves and restores the user's real + * ~/.gbrain/sync-failures.jsonl so running E2E on a developer machine + * doesn't trash their local sync state. + */ +describeE2E('E2E: sync --skip-failed structured summary loop (v0.22.12, issue #500)', () => { + let repoPath: string; + const realFailuresPath = join(homedir(), '.gbrain', 'sync-failures.jsonl'); + let savedFailuresContent: string | null = null; + + beforeAll(async () => { + await setupDB(); + + // Save+clear the real ~/.gbrain/sync-failures.jsonl so the test starts from + // a known-empty state. Restored in afterAll. This file is per-machine, NOT + // per-repo, so we have to be defensive about a developer running this + // suite on their actual brain machine. + if (existsSync(realFailuresPath)) { + savedFailuresContent = readFileSync(realFailuresPath, 'utf-8'); + unlinkSync(realFailuresPath); + } + + // Fresh git repo with one valid file. Mirrors createTestRepo above but + // scoped to this describe block. + repoPath = mkdtempSync(join(tmpdir(), 'gbrain-skipfailed-e2e-')); + execSync('git init', { cwd: repoPath, stdio: 'pipe' }); + execSync('git config user.email "test@test.com"', { cwd: repoPath, stdio: 'pipe' }); + execSync('git config user.name "Test"', { cwd: repoPath, stdio: 'pipe' }); + mkdirSync(join(repoPath, 'people'), { recursive: true }); + writeFileSync(join(repoPath, 'people/alice.md'), [ + '---', 'type: person', 'title: Alice', '---', '', 'Body.', + ].join('\n')); + execSync('git add -A && git commit -m "initial"', { cwd: repoPath, stdio: 'pipe' }); + }); + + afterAll(async () => { + await teardownDB(); + if (repoPath) rmSync(repoPath, { recursive: true, force: true }); + + // Restore the user's real sync-failures.jsonl, if any. + if (savedFailuresContent !== null) { + mkdirSync(join(homedir(), '.gbrain'), { recursive: true }); + writeFileSync(realFailuresPath, savedFailuresContent); + } else if (existsSync(realFailuresPath)) { + // Test wrote one but there was none before. Clean up. + unlinkSync(realFailuresPath); + } + }); + + test('full --skip-failed loop: blocks on bad file, skip advances bookmark, doctor shows code breakdown', async () => { + const { performSync } = await import('../../src/commands/sync.ts'); + const { loadSyncFailures, summarizeFailuresByCode } = await import('../../src/core/sync.ts'); + const engine = getEngine(); + + // Step 1: First sync of the clean repo — should succeed. + let result = await performSync(engine, { repoPath, noPull: true, noEmbed: true }); + expect(result.status).toBe('first_sync'); + const firstCommit = await engine.getConfig('sync.last_commit'); + expect(firstCommit).toBeTruthy(); + + // Step 2: Add a broken file — frontmatter slug doesn't match path-derived slug. + // The file path is people/bob.md so the path-derived slug is "people/bob", + // but we declare slug: "wrong-slug" in frontmatter. import-file.ts:368-377 + // raises "Frontmatter slug ... does not match path-derived slug ..." which + // classifier hits as SLUG_MISMATCH. + writeFileSync(join(repoPath, 'people/bob.md'), [ + '---', 'type: person', 'title: Bob', 'slug: wrong-slug', '---', '', 'Body.', + ].join('\n')); + execSync('git add -A && git commit -m "add broken bob"', { cwd: repoPath, stdio: 'pipe' }); + + // Step 3: Sync should block. Bookmark must NOT advance. + result = await performSync(engine, { repoPath, noPull: true, noEmbed: true }); + expect(result.status).toBe('blocked_by_failures'); + const afterBlockedCommit = await engine.getConfig('sync.last_commit'); + expect(afterBlockedCommit).toBe(firstCommit); // bookmark stuck at the pre-broken commit + + // JSONL has one unacked entry with code SLUG_MISMATCH. + let failures = loadSyncFailures(); + expect(failures.length).toBe(1); + expect(failures[0].code).toBe('SLUG_MISMATCH'); + expect(failures[0].acknowledged).toBeFalsy(); + // Group summary aggregates correctly across the unacked set. + expect(summarizeFailuresByCode(failures)).toEqual([{ code: 'SLUG_MISMATCH', count: 1 }]); + + // Step 4: Run with skipFailed — bookmark advances, entry gets acked. + result = await performSync(engine, { repoPath, noPull: true, noEmbed: true, skipFailed: true }); + expect(result.status).toBe('synced'); + const afterSkipCommit = await engine.getConfig('sync.last_commit'); + expect(afterSkipCommit).not.toBe(firstCommit); // bookmark moved past the broken commit + failures = loadSyncFailures(); + expect(failures.length).toBe(1); + expect(failures[0].acknowledged).toBe(true); + expect(typeof failures[0].acknowledged_at).toBe('string'); + + // Step 5: Verify what doctor would render for the historical entry. + // We call the same primitives doctor's `sync_failures` check uses + // (src/commands/doctor.ts:252-275) — loadSyncFailures + summarizeFailuresByCode — + // and assert the rendering string. Directly invoking runDoctor() here is a CLI + // entrypoint with stdout/exit side effects that would truncate this test mid-flow. + { + const all = loadSyncFailures(); + const ackedSummary = summarizeFailuresByCode(all); + const ackedBreakdown = ackedSummary.map(s => `${s.code}=${s.count}`).join(', '); + // This is the literal string interpolation doctor.ts:271-274 produces. + const doctorMessage = `${all.length} historical sync failure(s), all acknowledged [${ackedBreakdown}].`; + expect(doctorMessage).toContain('SLUG_MISMATCH=1'); + expect(doctorMessage).toContain('1 historical'); + } + + // Step 6: Add a second broken file — this one with a different failure code + // (also SLUG_MISMATCH but on a different file) so the JSONL has 2 entries + // with DIFFERENT paths but the same code. This proves both: per-file dedup + // honors path identity, and summary aggregation sums across files. + // + // We'd ideally test a different code class here, but the sync path uses + // parseMarkdown WITHOUT {validate:true}, so the markdown.ts validation + // codes (MISSING_OPEN/CLOSE, NESTED_QUOTES, EMPTY_FRONTMATTER, NULL_BYTES) + // don't naturally surface — they'd need {validate:true} plumbed in. That + // plumbing is the v0.22.13+ follow-up. For v0.22.12, two SLUG_MISMATCH + // entries from different files still proves the dedup + aggregation chain. + writeFileSync(join(repoPath, 'people/carol.md'), [ + '---', 'type: person', 'title: Carol', 'slug: also-wrong-slug', '---', '', 'Body.', + ].join('\n')); + execSync('git add -A && git commit -m "add carol with bad slug"', { cwd: repoPath, stdio: 'pipe' }); + + // Step 7: Sync blocks again on the new failure. Old entry stays acked. + result = await performSync(engine, { repoPath, noPull: true, noEmbed: true }); + expect(result.status).toBe('blocked_by_failures'); + failures = loadSyncFailures(); + expect(failures.length).toBe(2); + const acked = failures.filter(f => f.acknowledged); + const unacked = failures.filter(f => !f.acknowledged); + expect(acked.length).toBe(1); + expect(acked[0].code).toBe('SLUG_MISMATCH'); + expect(acked[0].path).toContain('bob'); + expect(unacked.length).toBe(1); + expect(unacked[0].code).toBe('SLUG_MISMATCH'); + expect(unacked[0].path).toContain('carol'); + + // Step 8: Skip again — both entries acked, summary aggregates the count. + result = await performSync(engine, { repoPath, noPull: true, noEmbed: true, skipFailed: true }); + expect(result.status).toBe('synced'); + failures = loadSyncFailures(); + expect(failures.length).toBe(2); + expect(failures.every(f => f.acknowledged)).toBe(true); + + const finalSummary = summarizeFailuresByCode(failures); + expect(finalSummary).toEqual([{ code: 'SLUG_MISMATCH', count: 2 }]); + }); +}); diff --git a/test/sync-failures.test.ts b/test/sync-failures.test.ts index 6f56e0aed..ea66b889f 100644 --- a/test/sync-failures.test.ts +++ b/test/sync-failures.test.ts @@ -193,6 +193,21 @@ describe('classifyErrorCode — error message to code mapping', () => { expect(classifyErrorCode('invalid UTF-8 sequence at position 500')).toBe('INVALID_UTF8'); }); + test('classifies FILE_TOO_LARGE across all three production sites', async () => { + const { classifyErrorCode } = await import('../src/core/sync.ts'); + // src/core/import-file.ts:352 — OS-level file size on disk + expect(classifyErrorCode('File too large (8432105 bytes)')).toBe('FILE_TOO_LARGE'); + // src/core/import-file.ts:199 — content size limit (5MB cap) + expect(classifyErrorCode('Content too large (6000000 bytes, max 5000000). Split the content into smaller files or remove large embedded assets.')).toBe('FILE_TOO_LARGE'); + // src/core/import-file.ts:401 — code file size cap + expect(classifyErrorCode('Code file too large (8000000 bytes)')).toBe('FILE_TOO_LARGE'); + }); + + test('classifies SYMLINK_NOT_ALLOWED from import-file.ts symlink rejection', async () => { + const { classifyErrorCode } = await import('../src/core/sync.ts'); + expect(classifyErrorCode('Skipping symlink: /path/to/link.md')).toBe('SYMLINK_NOT_ALLOWED'); + }); + test('returns UNKNOWN for unrecognized errors', async () => { const { classifyErrorCode } = await import('../src/core/sync.ts'); expect(classifyErrorCode('something completely different')).toBe('UNKNOWN');