mirror of
https://github.com/garrytan/gbrain.git
synced 2026-08-14 08:53:22 +00:00
* feat: structured error code summary for sync --skip-failed (#500)
When sync encounters per-file failures, the blocked/skip-failed messages
now include a breakdown by error code (SLUG_MISMATCH, YAML_PARSE, etc.)
instead of just a raw count. This makes it immediately obvious *why*
files failed without requiring manual investigation.
Changes:
- Add classifyErrorCode() — maps error messages to ParseValidationCode
- Add summarizeFailuresByCode() — groups failures into sorted code summary
- SyncFailure now carries a 'code' field (backfilled on acknowledge)
- acknowledgeSyncFailures() returns AcknowledgeResult {count, summary}
- sync blocked + skip-failed messages show code breakdown
- doctor sync_failures check shows code breakdown for both unacked and historical
- 12 new tests for classifyErrorCode, summarizeFailuresByCode, and structured returns
Before:
Sync blocked: 2688 file(s) failed to parse.
After:
Sync blocked: 2688 file(s) failed to parse:
SLUG_MISMATCH: 2685
YAML_DUPLICATE_KEY: 3
Closes #500
* test(sync): broaden classifier regexes and pin coverage with 6 new unit tests
Eng review of PR #501 found two ship-blocking gaps in the classifier:
1. Four real production error sites in src/core/import-file.ts emit strings
that bucketed to UNKNOWN — exactly the silent-systemic-failure pattern
that motivated #500 in the first place. Add two regex lines:
FILE_TOO_LARGE — covers import-file.ts:199, 352, 401
SYMLINK_NOT_ALLOWED — covers import-file.ts:347
2. Three existing classifier regexes (MISSING_OPEN, MISSING_CLOSE,
EMPTY_FRONTMATTER) only matched the literal code-name prefix. The actual
message strings emitted by markdown.ts:159-244 (e.g. "Frontmatter must
start with --- on the first non-empty line") wouldn't match. Broaden
each to match production message text. NESTED_QUOTES already worked.
Add 6 unit tests pinning the contract between markdown.ts/import-file.ts
strings and the classifier regex set. If anyone reworks a validator
message, both sides have to move together — the test fails loudly otherwise.
Test count: 22 → 28 in test/sync-failures.test.ts, all green.
* test(e2e): add failure-loop E2E for sync --skip-failed (issue #500 ship-blocker)
The full code path (record → classify → block → skip → doctor render →
second cycle) had only mocked-JSONL unit coverage. For a hotfix that
changes user-visible CLI output and the doctor surface, that's thin.
One comprehensive E2E test covers the loop:
1. First sync of clean repo — succeeds, bookmark advances
2. Add file with bad slug — sync returns 'blocked_by_failures',
bookmark stays put, JSONL has 1 unacked entry coded SLUG_MISMATCH
3. --skip-failed — bookmark advances past the bad commit, entry
transitions to acknowledged, AcknowledgeResult.summary aggregates
4. Second broken file (different path, same code) — sync blocks again,
1 acked + 1 unacked, dedup honors path identity
5. --skip-failed again — both acked, summary correctly counts 2
Hermetic on a developer machine: saves ~/.gbrain/sync-failures.jsonl
before the test, restores it after. Doctor rendering verified by calling
the same primitives doctor.ts uses (loadSyncFailures + summarizeFailuresByCode)
rather than runDoctor() — runDoctor is a CLI entrypoint with stdout/exit
side effects that truncate the test mid-flow.
E2E count: 13 → 14 in test/e2e/sync.test.ts. All 14 pass under real
Postgres + pgvector (gbrain-test-pg/pgvector:pg16).
* v0.22.12: structured error code summary for sync --skip-failed
Closes issue #500. PR #501 by @wintermute is the foundation (cherry-picked
as c356ea4 — classifier, doctor breakdown, AcknowledgeResult shape, 12 unit
tests). This release adds:
- Classifier coverage for FILE_TOO_LARGE + SYMLINK_NOT_ALLOWED (the four
size/symlink rejection sites in import-file.ts that bucketed to UNKNOWN).
- Three regex breadths (MISSING_OPEN, MISSING_CLOSE, EMPTY_FRONTMATTER)
matching actual markdown.ts validator messages, not just the literal
code-name prefix.
- 6 new unit tests pinning literal production strings.
- 1 comprehensive E2E test exercising the full failure loop.
Total v0.22.12 diff: ~340 lines on top of PR #501. Backward-compatible —
pre-v0.22.12 JSONL entries get classified at acknowledge time.
* chore: regenerate llms-full.txt for v0.22.12 CLAUDE.md changes
CI regen-drift guard caught that llms-full.txt was stale after the v0.22.12
CLAUDE.md annotation updates (sync.ts, doctor.ts, sync-failures.test.ts,
e2e/sync.test.ts entries). Per CLAUDE.md "Auto-derived" rule: run
`bun run build:llms` after any release ship that touches Key Files
annotations. The bundle reflects current docs state.
llms.txt unchanged (curated index doesn't index those entries).
llms-full.txt: 308192 bytes.
test/build-llms.test.ts now passes 7/7 (was 6/7 in CI).
---------
Co-authored-by: Wintermute <wintermute@garrytan.com>
This commit is contained in:
+52
-3
@@ -2,6 +2,57 @@
|
||||
|
||||
All notable changes to GBrain will be documented in this file.
|
||||
|
||||
## [0.22.12] - 2026-04-29
|
||||
|
||||
**`sync --skip-failed` now classifies file-size and symlink rejections instead of bucketing them as UNKNOWN.**
|
||||
**Plus a full end-to-end test for the failure loop.**
|
||||
|
||||
v0.22.9 shipped the headline classifier work: code-grouped breakdowns at sync time,
|
||||
DB-vs-YAML disambiguation, doctor surfaces both unacked and historical entries with
|
||||
`[CODE=N]` lines. v0.22.12 closes the last two coverage gaps that v0.22.9 left on
|
||||
the table:
|
||||
|
||||
- **FILE_TOO_LARGE** now covers the three real production sites in
|
||||
`src/core/import-file.ts:199, 352, 401` ("Content too large", "File too large",
|
||||
"Code file too large"). On v0.22.9 these all bucketed as UNKNOWN — the same
|
||||
silent-systemic-failure pattern that motivated the original issue.
|
||||
- **SYMLINK_NOT_ALLOWED** covers `src/core/import-file.ts:347` ("Skipping symlink").
|
||||
Security-relevant rejection that operators should see.
|
||||
- **End-to-end failure-loop test** in `test/e2e/sync.test.ts` exercises the full
|
||||
chain: broken file → sync blocks with grouped breakdown → `--skip-failed`
|
||||
advances bookmark with grouped acknowledgement → second broken file → second
|
||||
cycle. PostgreSQL-backed; verifies bookmark gating, JSONL state, dedup, and
|
||||
summary aggregation. v0.22.9's coverage was unit-tests-only.
|
||||
|
||||
Twelve total error codes ship in the classifier:
|
||||
`SLUG_MISMATCH`, `YAML_PARSE`, `YAML_DUPLICATE_KEY`, `DB_DUPLICATE_KEY`,
|
||||
`MISSING_OPEN`, `MISSING_CLOSE`, `NESTED_QUOTES`, `EMPTY_FRONTMATTER`,
|
||||
`NULL_BYTES`, `INVALID_UTF8`, `STATEMENT_TIMEOUT`, `FILE_TOO_LARGE`,
|
||||
`SYMLINK_NOT_ALLOWED`. Anything the regex set doesn't recognize falls through
|
||||
as `UNKNOWN`.
|
||||
|
||||
### What this means for you
|
||||
|
||||
If your brain rejects oversized files or symlinks, you now see those rejections
|
||||
in the doctor breakdown and at sync time grouped by code, instead of as
|
||||
`UNKNOWN`. Run `gbrain upgrade`. No manual action required.
|
||||
|
||||
### Itemized changes
|
||||
|
||||
#### Added
|
||||
- `FILE_TOO_LARGE` classifier code covering `src/core/import-file.ts:199, 352, 401`.
|
||||
- `SYMLINK_NOT_ALLOWED` classifier code covering `src/core/import-file.ts:347`.
|
||||
- Two new unit tests in `test/sync-failures.test.ts` pinning the new codes against
|
||||
literal production message strings (`File too large (N bytes)`, `Skipping symlink: ...`).
|
||||
- `test/e2e/sync.test.ts` — new failure-loop test exercising broken-file → block →
|
||||
`--skip-failed` → second cycle. Hermetic on developer machines (saves+restores
|
||||
the user's real `~/.gbrain/sync-failures.jsonl`).
|
||||
|
||||
## To take advantage of v0.22.12
|
||||
|
||||
No manual action required. Run `gbrain upgrade`. The new `FILE_TOO_LARGE` and
|
||||
`SYMLINK_NOT_ALLOWED` classifier codes apply on the next `gbrain sync`.
|
||||
|
||||
## [0.22.11] - 2026-04-27
|
||||
|
||||
**Storage tiering, finally working. Brains scaling past 100K files stop bloating git.**
|
||||
@@ -172,7 +223,7 @@ If `gbrain sync` blocks with parse failures, the breakdown tells you what to fix
|
||||
- DB-layer error patterns (`DB_DUPLICATE_KEY`, `STATEMENT_TIMEOUT`) check BEFORE YAML patterns in the classifier, so Postgres errors don't get YAML-labeled.
|
||||
- Frontmatter regex patterns rewritten to match canonical messages from `collectValidationErrors()` (`File is empty...`, `No closing --- delimiter found`, `Frontmatter block is empty`) instead of aspirational code-token strings (`missing.*open`) that never appeared in practice.
|
||||
|
||||
Closes #500. Eng-review plan: `~/.claude/plans/then-codex-synchronous-toucan.md` (codex outside-voice agreed on all 7 findings).
|
||||
Closes #500.
|
||||
|
||||
## [0.22.8] - 2026-04-28
|
||||
|
||||
@@ -322,8 +373,6 @@ Then point Claude Desktop, claude.ai/code, or any MCP client at `http://your-tun
|
||||
|
||||
If anything breaks: `gbrain doctor`, `~/.gbrain/upgrade-errors.jsonl` (if present), and please file an issue at https://github.com/garrytan/gbrain/issues with both.
|
||||
|
||||
|
||||
|
||||
## [0.22.6.1] - 2026-04-26
|
||||
|
||||
**Old brains can upgrade again.**
|
||||
|
||||
@@ -32,7 +32,7 @@ strict behavior when unset.
|
||||
- `src/core/db.ts` — Connection management, schema initialization. v0.22.1 (#363, contributed by @orendi84): `resolveSessionTimeouts()` returns `statement_timeout` + `idle_in_transaction_session_timeout` (defaults: 5min each, env-overridable via `GBRAIN_STATEMENT_TIMEOUT` / `GBRAIN_IDLE_TX_TIMEOUT` / `GBRAIN_CLIENT_CHECK_INTERVAL`). Both `connect()` (module singleton) and `PostgresEngine.connect()` (worker pool) consume the result via postgres.js's `connection` option, sending GUCs as startup parameters that survive PgBouncer transaction mode (unlike the prior `setSessionDefaults` post-pool SET, kept as a back-compat no-op shim).
|
||||
- `src/commands/migrate-engine.ts` — Bidirectional engine migration (`gbrain migrate --to supabase/pglite`)
|
||||
- `src/core/import-file.ts` — importFromFile + importFromContent (chunk + embed + tags)
|
||||
- `src/core/sync.ts` — Pure sync functions (manifest parsing, filtering, slug conversion)
|
||||
- `src/core/sync.ts` — Pure sync functions (manifest parsing, filtering, slug conversion). v0.22.12 (#500, foundation by @wintermute via #501): `classifyErrorCode(errorMsg)` regex-based classifier with 12 codes (`SLUG_MISMATCH`, `YAML_PARSE`, `YAML_DUPLICATE_KEY`, `MISSING_OPEN`, `MISSING_CLOSE`, `NESTED_QUOTES`, `EMPTY_FRONTMATTER`, `NULL_BYTES`, `INVALID_UTF8`, `STATEMENT_TIMEOUT`, `FILE_TOO_LARGE`, `SYMLINK_NOT_ALLOWED`) plus `UNKNOWN` fallback. `summarizeFailuresByCode(failures)` returns sorted `[{code, count}]`. `code?` optional field on `SyncFailure`; backfilled at ack time on pre-v0.22.12 entries. `acknowledgeSyncFailures()` returns `AcknowledgeResult { count, summary }`. Three regexes (`MISSING_OPEN`, `MISSING_CLOSE`, `EMPTY_FRONTMATTER`) broadened to match actual `markdown.ts:159-244` validator message strings, not just the literal code-name prefix. `FILE_TOO_LARGE` covers all three production size sites in `import-file.ts:199, 352, 401`; `SYMLINK_NOT_ALLOWED` covers the rejection at `:347`. Closes the silent-skip pattern that motivated #500.
|
||||
- `src/core/storage.ts` — Pluggable storage interface (S3, Supabase Storage, local)
|
||||
- `src/core/storage-config.ts` (v0.22.11) — Storage tiering: `loadStorageConfig` reads `gbrain.yml`, normalizes deprecated keys (`git_tracked` / `supabase_only`) to canonical (`db_tracked` / `db_only`) with once-per-process deprecation warning, and runs `normalizeAndValidateStorageConfig` (auto-fixes missing trailing `/`, throws `StorageConfigError` on tier overlap). Path-segment matcher: `media/x/` does NOT match `media/xerox/foo`. Replaces gray-matter (broken on delimiter-less YAML) with a dedicated parser for the `gbrain.yml` shape.
|
||||
- `src/core/disk-walk.ts` (v0.22.11) — `walkBrainRepo(repoPath)` returns `Map<slug, {size, mtimeMs}>` from one recursive `readdirSync`. Skips dot-dirs, `node_modules`, non-`.md` files. Used by `gbrain storage status` to replace per-page `existsSync + statSync` (~400K syscalls on 200K-page brains → tens).
|
||||
@@ -105,7 +105,7 @@ strict behavior when unset.
|
||||
- `src/commands/repair-jsonb.ts` — `gbrain repair-jsonb [--dry-run] [--json]`: rewrites `jsonb_typeof='string'` rows in place across 5 affected columns (pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata, page_versions.frontmatter). Fixes v0.12.0 double-encode bug on Postgres; PGLite no-ops. Idempotent.
|
||||
- `src/commands/orphans.ts` — `gbrain orphans [--json] [--count] [--include-pseudo]`: surfaces pages with zero inbound wikilinks, grouped by domain. Auto-generated/raw/pseudo pages filtered by default. Also exposed as `find_orphans` MCP operation. Shipped in v0.12.3 (contributed by @knee5).
|
||||
- `src/commands/integrity.ts` — `gbrain integrity check|auto|review|extract`: bare-tweet detection, dead-link detection, three-bucket repair (auto-repair / review-queue / skip). `scanIntegrity()` is the shared library function called from `gbrain doctor` (sampled at limit=500) and `cmdCheck` (full scan). v0.22.8: batch-load fast path on Postgres uses `SELECT DISTINCT ON (slug)` in a single SQL query to fix the PgBouncer round-trip timeout (60s → ~6s) while preserving `engine.getAllSlugs()`'s `Set<string>` semantics on multi-source brains. Gated by `engine.kind === 'postgres'` at the call site so PGLite never enters batch; fallback `catch` logs at `GBRAIN_DEBUG=1` so real Postgres errors are diagnosable.
|
||||
- `src/commands/doctor.ts` — `gbrain doctor [--json] [--fast] [--fix] [--dry-run] [--index-audit]`: health checks. v0.12.3 added `jsonb_integrity` + `markdown_body_completeness` reliability checks. v0.14.1: `--fix` delegates inlined cross-cutting rules to `> **Convention:** see [path](path).` callouts (pipes DRY violations into `src/core/dry-fix.ts`); `--fix --dry-run` previews without writing. v0.14.2: `schema_version` check fails loudly when `version=0` (migrations never ran — the #218 `bun install -g` signature) and routes users to `gbrain apply-migrations --yes`; new opt-in `--index-audit` flag (Postgres-only) reports zero-scan indexes from `pg_stat_user_indexes` (informational only, no auto-drop). v0.15.2: every DB check is wrapped in a progress phase; `markdown_body_completeness` runs under a 1s heartbeat timer so 10+ min scans are observable on 50K-page brains. v0.19.1 added `queue_health` (Postgres-only) with two subchecks: stalled-forever active jobs (started_at > 1h) and waiting-depth-per-name > threshold (default 10, override via `GBRAIN_QUEUE_WAITING_THRESHOLD`). Worker-heartbeat subcheck intentionally deferred to follow-up B7 because it needs a `minion_workers` table to produce ground-truth signal. Fix hints point at `gbrain repair-jsonb`, `gbrain sync --force`, `gbrain apply-migrations`, and `gbrain jobs get/cancel <id>`.
|
||||
- `src/commands/doctor.ts` — `gbrain doctor [--json] [--fast] [--fix] [--dry-run] [--index-audit]`: health checks. v0.12.3 added `jsonb_integrity` + `markdown_body_completeness` reliability checks. v0.14.1: `--fix` delegates inlined cross-cutting rules to `> **Convention:** see [path](path).` callouts (pipes DRY violations into `src/core/dry-fix.ts`); `--fix --dry-run` previews without writing. v0.14.2: `schema_version` check fails loudly when `version=0` (migrations never ran — the #218 `bun install -g` signature) and routes users to `gbrain apply-migrations --yes`; new opt-in `--index-audit` flag (Postgres-only) reports zero-scan indexes from `pg_stat_user_indexes` (informational only, no auto-drop). v0.15.2: every DB check is wrapped in a progress phase; `markdown_body_completeness` runs under a 1s heartbeat timer so 10+ min scans are observable on 50K-page brains. v0.19.1 added `queue_health` (Postgres-only) with two subchecks: stalled-forever active jobs (started_at > 1h) and waiting-depth-per-name > threshold (default 10, override via `GBRAIN_QUEUE_WAITING_THRESHOLD`). Worker-heartbeat subcheck intentionally deferred to follow-up B7 because it needs a `minion_workers` table to produce ground-truth signal. Fix hints point at `gbrain repair-jsonb`, `gbrain sync --force`, `gbrain apply-migrations`, and `gbrain jobs get/cancel <id>`. v0.22.12 (#500): `sync_failures` check shows `[CODE=N, ...]` breakdown for both unacked entries (warn) and acked-historical entries (ok), surfacing systemic failure modes (`SLUG_MISMATCH=2685`) instead of a bare count.
|
||||
- `src/core/migrate.ts` — schema-migration runner. Owns the `MIGRATIONS` array (source of truth for schema DDL). v0.14.2 extended the `Migration` interface with `sqlFor?: { postgres?, pglite? }` (engine-specific SQL overrides `sql`) and `transaction?: boolean` (set to false for `CREATE INDEX CONCURRENTLY`, which Postgres refuses inside a transaction; ignored on PGLite since it has no concurrent writers). Migration v14 (fix wave) uses a handler branching on `engine.kind` to run CONCURRENTLY on Postgres (with a pre-drop of any invalid remnant via `pg_index.indisvalid`) and plain `CREATE INDEX` on PGLite. v15 bumps `minion_jobs.max_stalled` default 1→5 and backfills existing non-terminal rows. v0.22.6.1: migration v24 (`rls_backfill_missing_tables`) uses `sqlFor: { pglite: '' }` to no-op on PGLite — PGLite has no RLS engine and is single-tenant by definition, and the v24 ALTERs target subagent tables that don't exist in pglite-schema.ts. Closes #395 (contributed by @jdcastro2).
|
||||
- `src/core/progress.ts` — Shared bulk-action progress reporter. Writes to stderr. Modes: `auto` (TTY: `\r`-rewriting; non-TTY: plain lines), `human`, `json` (JSONL), `quiet`. Rate-gated by `minIntervalMs` and `minItems`. `startHeartbeat(reporter, note)` helper for single long queries. `child()` composes phase paths. Singleton SIGINT/SIGTERM coordinator emits `abort` events for every live phase. EPIPE defense on both sync throws and stream `'error'` events. Zero dependencies. Introduced in v0.15.2.
|
||||
- `src/core/cli-options.ts` — Global CLI flag parser. `parseGlobalFlags(argv)` returns `{cliOpts, rest}` with `--quiet` / `--progress-json` / `--progress-interval=<ms>` stripped. `getCliOptions()` / `setCliOptions()` expose a module-level singleton so commands reach the resolved flags without parameter threading. `cliOptsToProgressOptions()` maps to reporter options. `childGlobalFlags()` returns the flag suffix to append to `execSync('gbrain ...')` calls in migration orchestrators. `OperationContext.cliOpts` extends shared-op dispatch for MCP callers.
|
||||
@@ -275,6 +275,7 @@ parity), `test/cli.test.ts` (CLI structure), `test/config.test.ts` (config redac
|
||||
`test/orphans.test.ts` (v0.12.3 orphans command: detection, pseudo filtering, text/json/count outputs, MCP op),
|
||||
`test/postgres-engine.test.ts` (v0.12.3 statement_timeout scoping: `sql.begin` + `SET LOCAL` shape, source-level grep guardrail against reintroduced bare `SET statement_timeout`),
|
||||
`test/sync.test.ts` (sync logic + v0.12.3 regression guard asserting top-level `engine.transaction` is not called),
|
||||
`test/sync-failures.test.ts` (v0.22.12: 28 cases pinning `classifyErrorCode` regex coverage for all 12 codes against literal production message strings from `markdown.ts:159-244` and `import-file.ts:199, 347, 352, 401`; `summarizeFailuresByCode` sort + pre-classified-honor; `recordSyncFailures` code-field persistence; `acknowledgeSyncFailures` AcknowledgeResult shape + backfill on pre-v0.22.12 entries),
|
||||
`test/doctor.test.ts` (doctor command + v0.12.3 assertions that `jsonb_integrity` scans the four v0.12.0 write sites and `markdown_body_completeness` is present),
|
||||
`test/utils.test.ts` (shared SQL utilities + `tryParseEmbedding` null-return and single-warn semantics),
|
||||
`test/build-llms.test.ts` (llms.txt/llms-full.txt generator: path resolution, idempotence, spec shape, regen-drift guard, content contract, AGENTS.md install-path mirror, size-budget enforcement — 7 cases),
|
||||
@@ -295,6 +296,7 @@ E2E tests (`test/e2e/`): Run against real Postgres+pgvector. Require `DATABASE_U
|
||||
- `test/e2e/postgres-jsonb.test.ts` — v0.12.2 regression test. Round-trips all 5 JSONB write sites (pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata, page_versions.frontmatter) against real Postgres and asserts `jsonb_typeof='object'` plus `->>'key'` returns the expected scalar. The test that should have caught the original double-encode bug.
|
||||
- `test/e2e/integrity-batch.test.ts` (v0.22.8) — parity tests for `scanIntegrity`'s batch-load fast path vs sequential. Four cases (dedup, hits, validate, topPages) seed a fixture and assert both paths return identical results. Dedup case uses raw SQL via `getConn().unsafe()` to seed a `(test-source-2, people/alice)` row alongside the default-source row, since `engine.putPage` doesn't take a `source_id`. Pins the codex-caught multi-source overcounting regression.
|
||||
- `test/e2e/jsonb-roundtrip.test.ts` — v0.12.3 companion regression against the 4 doctor-scanned JSONB sites. Assertion-level overlap with `postgres-jsonb.test.ts` is intentional defense-in-depth: if doctor's scan surface ever drifts from the actual write surface, one of these tests catches it.
|
||||
- `test/e2e/sync.test.ts` (v0.22.12 — `--skip-failed` failure-loop test, alongside the existing 13 happy-path tests): exercises the full chain — broken file → `performSync` returns `blocked_by_failures` with grouped breakdown → `performSync({skipFailed: true})` advances bookmark and returns `AcknowledgeResult` with code summary → second broken file → second cycle. Saves and restores the user's real `~/.gbrain/sync-failures.jsonl` so the test is hermetic on a developer machine. Asserts bookmark gating, JSONL state, dedup across paths, summary aggregation, and the literal doctor-rendering string format. This is the integration test that proves the v0.22.12 chain holds together — unit tests cover the pure functions in isolation, this covers the integration.
|
||||
- `test/e2e/upgrade.test.ts` runs check-update E2E against real GitHub API (network required)
|
||||
- `test/e2e/minions-shell-pglite.test.ts` (v0.20.4) exercises the PGLite `--follow` inline shell-job path (in-memory, no `DATABASE_URL` required) — the path the consolidated minion-orchestrator skill documents for dev use
|
||||
- `test/e2e/openclaw-reference-compat.test.ts` (v0.19) — exercises `check-resolvable` + `skillpack install` against a minimal AGENTS.md workspace fixture (`test/fixtures/openclaw-reference-minimal/`), regression guard for the 107-skill OpenClaw deployment shape
|
||||
|
||||
+4
-2
@@ -111,7 +111,7 @@ strict behavior when unset.
|
||||
- `src/core/db.ts` — Connection management, schema initialization. v0.22.1 (#363, contributed by @orendi84): `resolveSessionTimeouts()` returns `statement_timeout` + `idle_in_transaction_session_timeout` (defaults: 5min each, env-overridable via `GBRAIN_STATEMENT_TIMEOUT` / `GBRAIN_IDLE_TX_TIMEOUT` / `GBRAIN_CLIENT_CHECK_INTERVAL`). Both `connect()` (module singleton) and `PostgresEngine.connect()` (worker pool) consume the result via postgres.js's `connection` option, sending GUCs as startup parameters that survive PgBouncer transaction mode (unlike the prior `setSessionDefaults` post-pool SET, kept as a back-compat no-op shim).
|
||||
- `src/commands/migrate-engine.ts` — Bidirectional engine migration (`gbrain migrate --to supabase/pglite`)
|
||||
- `src/core/import-file.ts` — importFromFile + importFromContent (chunk + embed + tags)
|
||||
- `src/core/sync.ts` — Pure sync functions (manifest parsing, filtering, slug conversion)
|
||||
- `src/core/sync.ts` — Pure sync functions (manifest parsing, filtering, slug conversion). v0.22.12 (#500, foundation by @wintermute via #501): `classifyErrorCode(errorMsg)` regex-based classifier with 12 codes (`SLUG_MISMATCH`, `YAML_PARSE`, `YAML_DUPLICATE_KEY`, `MISSING_OPEN`, `MISSING_CLOSE`, `NESTED_QUOTES`, `EMPTY_FRONTMATTER`, `NULL_BYTES`, `INVALID_UTF8`, `STATEMENT_TIMEOUT`, `FILE_TOO_LARGE`, `SYMLINK_NOT_ALLOWED`) plus `UNKNOWN` fallback. `summarizeFailuresByCode(failures)` returns sorted `[{code, count}]`. `code?` optional field on `SyncFailure`; backfilled at ack time on pre-v0.22.12 entries. `acknowledgeSyncFailures()` returns `AcknowledgeResult { count, summary }`. Three regexes (`MISSING_OPEN`, `MISSING_CLOSE`, `EMPTY_FRONTMATTER`) broadened to match actual `markdown.ts:159-244` validator message strings, not just the literal code-name prefix. `FILE_TOO_LARGE` covers all three production size sites in `import-file.ts:199, 352, 401`; `SYMLINK_NOT_ALLOWED` covers the rejection at `:347`. Closes the silent-skip pattern that motivated #500.
|
||||
- `src/core/storage.ts` — Pluggable storage interface (S3, Supabase Storage, local)
|
||||
- `src/core/storage-config.ts` (v0.22.11) — Storage tiering: `loadStorageConfig` reads `gbrain.yml`, normalizes deprecated keys (`git_tracked` / `supabase_only`) to canonical (`db_tracked` / `db_only`) with once-per-process deprecation warning, and runs `normalizeAndValidateStorageConfig` (auto-fixes missing trailing `/`, throws `StorageConfigError` on tier overlap). Path-segment matcher: `media/x/` does NOT match `media/xerox/foo`. Replaces gray-matter (broken on delimiter-less YAML) with a dedicated parser for the `gbrain.yml` shape.
|
||||
- `src/core/disk-walk.ts` (v0.22.11) — `walkBrainRepo(repoPath)` returns `Map<slug, {size, mtimeMs}>` from one recursive `readdirSync`. Skips dot-dirs, `node_modules`, non-`.md` files. Used by `gbrain storage status` to replace per-page `existsSync + statSync` (~400K syscalls on 200K-page brains → tens).
|
||||
@@ -184,7 +184,7 @@ strict behavior when unset.
|
||||
- `src/commands/repair-jsonb.ts` — `gbrain repair-jsonb [--dry-run] [--json]`: rewrites `jsonb_typeof='string'` rows in place across 5 affected columns (pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata, page_versions.frontmatter). Fixes v0.12.0 double-encode bug on Postgres; PGLite no-ops. Idempotent.
|
||||
- `src/commands/orphans.ts` — `gbrain orphans [--json] [--count] [--include-pseudo]`: surfaces pages with zero inbound wikilinks, grouped by domain. Auto-generated/raw/pseudo pages filtered by default. Also exposed as `find_orphans` MCP operation. Shipped in v0.12.3 (contributed by @knee5).
|
||||
- `src/commands/integrity.ts` — `gbrain integrity check|auto|review|extract`: bare-tweet detection, dead-link detection, three-bucket repair (auto-repair / review-queue / skip). `scanIntegrity()` is the shared library function called from `gbrain doctor` (sampled at limit=500) and `cmdCheck` (full scan). v0.22.8: batch-load fast path on Postgres uses `SELECT DISTINCT ON (slug)` in a single SQL query to fix the PgBouncer round-trip timeout (60s → ~6s) while preserving `engine.getAllSlugs()`'s `Set<string>` semantics on multi-source brains. Gated by `engine.kind === 'postgres'` at the call site so PGLite never enters batch; fallback `catch` logs at `GBRAIN_DEBUG=1` so real Postgres errors are diagnosable.
|
||||
- `src/commands/doctor.ts` — `gbrain doctor [--json] [--fast] [--fix] [--dry-run] [--index-audit]`: health checks. v0.12.3 added `jsonb_integrity` + `markdown_body_completeness` reliability checks. v0.14.1: `--fix` delegates inlined cross-cutting rules to `> **Convention:** see [path](path).` callouts (pipes DRY violations into `src/core/dry-fix.ts`); `--fix --dry-run` previews without writing. v0.14.2: `schema_version` check fails loudly when `version=0` (migrations never ran — the #218 `bun install -g` signature) and routes users to `gbrain apply-migrations --yes`; new opt-in `--index-audit` flag (Postgres-only) reports zero-scan indexes from `pg_stat_user_indexes` (informational only, no auto-drop). v0.15.2: every DB check is wrapped in a progress phase; `markdown_body_completeness` runs under a 1s heartbeat timer so 10+ min scans are observable on 50K-page brains. v0.19.1 added `queue_health` (Postgres-only) with two subchecks: stalled-forever active jobs (started_at > 1h) and waiting-depth-per-name > threshold (default 10, override via `GBRAIN_QUEUE_WAITING_THRESHOLD`). Worker-heartbeat subcheck intentionally deferred to follow-up B7 because it needs a `minion_workers` table to produce ground-truth signal. Fix hints point at `gbrain repair-jsonb`, `gbrain sync --force`, `gbrain apply-migrations`, and `gbrain jobs get/cancel <id>`.
|
||||
- `src/commands/doctor.ts` — `gbrain doctor [--json] [--fast] [--fix] [--dry-run] [--index-audit]`: health checks. v0.12.3 added `jsonb_integrity` + `markdown_body_completeness` reliability checks. v0.14.1: `--fix` delegates inlined cross-cutting rules to `> **Convention:** see [path](path).` callouts (pipes DRY violations into `src/core/dry-fix.ts`); `--fix --dry-run` previews without writing. v0.14.2: `schema_version` check fails loudly when `version=0` (migrations never ran — the #218 `bun install -g` signature) and routes users to `gbrain apply-migrations --yes`; new opt-in `--index-audit` flag (Postgres-only) reports zero-scan indexes from `pg_stat_user_indexes` (informational only, no auto-drop). v0.15.2: every DB check is wrapped in a progress phase; `markdown_body_completeness` runs under a 1s heartbeat timer so 10+ min scans are observable on 50K-page brains. v0.19.1 added `queue_health` (Postgres-only) with two subchecks: stalled-forever active jobs (started_at > 1h) and waiting-depth-per-name > threshold (default 10, override via `GBRAIN_QUEUE_WAITING_THRESHOLD`). Worker-heartbeat subcheck intentionally deferred to follow-up B7 because it needs a `minion_workers` table to produce ground-truth signal. Fix hints point at `gbrain repair-jsonb`, `gbrain sync --force`, `gbrain apply-migrations`, and `gbrain jobs get/cancel <id>`. v0.22.12 (#500): `sync_failures` check shows `[CODE=N, ...]` breakdown for both unacked entries (warn) and acked-historical entries (ok), surfacing systemic failure modes (`SLUG_MISMATCH=2685`) instead of a bare count.
|
||||
- `src/core/migrate.ts` — schema-migration runner. Owns the `MIGRATIONS` array (source of truth for schema DDL). v0.14.2 extended the `Migration` interface with `sqlFor?: { postgres?, pglite? }` (engine-specific SQL overrides `sql`) and `transaction?: boolean` (set to false for `CREATE INDEX CONCURRENTLY`, which Postgres refuses inside a transaction; ignored on PGLite since it has no concurrent writers). Migration v14 (fix wave) uses a handler branching on `engine.kind` to run CONCURRENTLY on Postgres (with a pre-drop of any invalid remnant via `pg_index.indisvalid`) and plain `CREATE INDEX` on PGLite. v15 bumps `minion_jobs.max_stalled` default 1→5 and backfills existing non-terminal rows. v0.22.6.1: migration v24 (`rls_backfill_missing_tables`) uses `sqlFor: { pglite: '' }` to no-op on PGLite — PGLite has no RLS engine and is single-tenant by definition, and the v24 ALTERs target subagent tables that don't exist in pglite-schema.ts. Closes #395 (contributed by @jdcastro2).
|
||||
- `src/core/progress.ts` — Shared bulk-action progress reporter. Writes to stderr. Modes: `auto` (TTY: `\r`-rewriting; non-TTY: plain lines), `human`, `json` (JSONL), `quiet`. Rate-gated by `minIntervalMs` and `minItems`. `startHeartbeat(reporter, note)` helper for single long queries. `child()` composes phase paths. Singleton SIGINT/SIGTERM coordinator emits `abort` events for every live phase. EPIPE defense on both sync throws and stream `'error'` events. Zero dependencies. Introduced in v0.15.2.
|
||||
- `src/core/cli-options.ts` — Global CLI flag parser. `parseGlobalFlags(argv)` returns `{cliOpts, rest}` with `--quiet` / `--progress-json` / `--progress-interval=<ms>` stripped. `getCliOptions()` / `setCliOptions()` expose a module-level singleton so commands reach the resolved flags without parameter threading. `cliOptsToProgressOptions()` maps to reporter options. `childGlobalFlags()` returns the flag suffix to append to `execSync('gbrain ...')` calls in migration orchestrators. `OperationContext.cliOpts` extends shared-op dispatch for MCP callers.
|
||||
@@ -354,6 +354,7 @@ parity), `test/cli.test.ts` (CLI structure), `test/config.test.ts` (config redac
|
||||
`test/orphans.test.ts` (v0.12.3 orphans command: detection, pseudo filtering, text/json/count outputs, MCP op),
|
||||
`test/postgres-engine.test.ts` (v0.12.3 statement_timeout scoping: `sql.begin` + `SET LOCAL` shape, source-level grep guardrail against reintroduced bare `SET statement_timeout`),
|
||||
`test/sync.test.ts` (sync logic + v0.12.3 regression guard asserting top-level `engine.transaction` is not called),
|
||||
`test/sync-failures.test.ts` (v0.22.12: 28 cases pinning `classifyErrorCode` regex coverage for all 12 codes against literal production message strings from `markdown.ts:159-244` and `import-file.ts:199, 347, 352, 401`; `summarizeFailuresByCode` sort + pre-classified-honor; `recordSyncFailures` code-field persistence; `acknowledgeSyncFailures` AcknowledgeResult shape + backfill on pre-v0.22.12 entries),
|
||||
`test/doctor.test.ts` (doctor command + v0.12.3 assertions that `jsonb_integrity` scans the four v0.12.0 write sites and `markdown_body_completeness` is present),
|
||||
`test/utils.test.ts` (shared SQL utilities + `tryParseEmbedding` null-return and single-warn semantics),
|
||||
`test/build-llms.test.ts` (llms.txt/llms-full.txt generator: path resolution, idempotence, spec shape, regen-drift guard, content contract, AGENTS.md install-path mirror, size-budget enforcement — 7 cases),
|
||||
@@ -374,6 +375,7 @@ E2E tests (`test/e2e/`): Run against real Postgres+pgvector. Require `DATABASE_U
|
||||
- `test/e2e/postgres-jsonb.test.ts` — v0.12.2 regression test. Round-trips all 5 JSONB write sites (pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata, page_versions.frontmatter) against real Postgres and asserts `jsonb_typeof='object'` plus `->>'key'` returns the expected scalar. The test that should have caught the original double-encode bug.
|
||||
- `test/e2e/integrity-batch.test.ts` (v0.22.8) — parity tests for `scanIntegrity`'s batch-load fast path vs sequential. Four cases (dedup, hits, validate, topPages) seed a fixture and assert both paths return identical results. Dedup case uses raw SQL via `getConn().unsafe()` to seed a `(test-source-2, people/alice)` row alongside the default-source row, since `engine.putPage` doesn't take a `source_id`. Pins the codex-caught multi-source overcounting regression.
|
||||
- `test/e2e/jsonb-roundtrip.test.ts` — v0.12.3 companion regression against the 4 doctor-scanned JSONB sites. Assertion-level overlap with `postgres-jsonb.test.ts` is intentional defense-in-depth: if doctor's scan surface ever drifts from the actual write surface, one of these tests catches it.
|
||||
- `test/e2e/sync.test.ts` (v0.22.12 — `--skip-failed` failure-loop test, alongside the existing 13 happy-path tests): exercises the full chain — broken file → `performSync` returns `blocked_by_failures` with grouped breakdown → `performSync({skipFailed: true})` advances bookmark and returns `AcknowledgeResult` with code summary → second broken file → second cycle. Saves and restores the user's real `~/.gbrain/sync-failures.jsonl` so the test is hermetic on a developer machine. Asserts bookmark gating, JSONL state, dedup across paths, summary aggregation, and the literal doctor-rendering string format. This is the integration test that proves the v0.22.12 chain holds together — unit tests cover the pure functions in isolation, this covers the integration.
|
||||
- `test/e2e/upgrade.test.ts` runs check-update E2E against real GitHub API (network required)
|
||||
- `test/e2e/minions-shell-pglite.test.ts` (v0.20.4) exercises the PGLite `--follow` inline shell-job path (in-memory, no `DATABASE_URL` required) — the path the consolidated minion-orchestrator skill documents for dev use
|
||||
- `test/e2e/openclaw-reference-compat.test.ts` (v0.19) — exercises `check-resolvable` + `skillpack install` against a minimal AGENTS.md workspace fixture (`test/fixtures/openclaw-reference-minimal/`), regression guard for the 107-skill OpenClaw deployment shape
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "gbrain",
|
||||
"version": "0.22.11",
|
||||
"version": "0.22.12",
|
||||
"description": "Postgres-native personal knowledge brain with hybrid RAG search",
|
||||
"type": "module",
|
||||
"main": "src/core/index.ts",
|
||||
|
||||
@@ -360,6 +360,11 @@ export function classifyErrorCode(errorMsg: string): string {
|
||||
|
||||
// Generic fallbacks.
|
||||
if (/invalid UTF-?8|INVALID_UTF8/i.test(errorMsg)) return 'INVALID_UTF8';
|
||||
|
||||
// v0.22.12 additions: covers the four real production sites in src/core/import-file.ts
|
||||
// (lines 199, 347, 352, 401) that previously bucketed to UNKNOWN.
|
||||
if (/file too large|content too large|FILE_TOO_LARGE/i.test(errorMsg)) return 'FILE_TOO_LARGE';
|
||||
if (/skipping symlink|symlink|SYMLINK_NOT_ALLOWED/i.test(errorMsg)) return 'SYMLINK_NOT_ALLOWED';
|
||||
return 'UNKNOWN';
|
||||
}
|
||||
|
||||
|
||||
+162
-2
@@ -10,10 +10,10 @@
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { mkdtempSync, writeFileSync, rmSync, mkdirSync, unlinkSync } from 'fs';
|
||||
import { mkdtempSync, writeFileSync, rmSync, mkdirSync, unlinkSync, existsSync, readFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { execSync } from 'child_process';
|
||||
import { tmpdir } from 'os';
|
||||
import { tmpdir, homedir } from 'os';
|
||||
import {
|
||||
hasDatabase, setupDB, teardownDB, getEngine,
|
||||
} from './helpers.ts';
|
||||
@@ -394,3 +394,163 @@ describeE2E('E2E: Git-to-DB Sync Pipeline', () => {
|
||||
expect(page!.title).toBe('Draft Meeting Notes');
|
||||
});
|
||||
});
|
||||
|
||||
/**
|
||||
* E2E: --skip-failed loop with structured error code summary.
|
||||
*
|
||||
* Closes the v0.22.12 ship-blocker gap from issue #500 — the whole code path
|
||||
* (record → classify → block → skip → doctor render → second cycle) had only
|
||||
* mocked-JSONL unit coverage. This is the integration test that proves the
|
||||
* chain holds together with a real Postgres engine, real git history, and
|
||||
* real frontmatter validation.
|
||||
*
|
||||
* Owns its own repo + sync-failures.jsonl lifecycle so it can't leak state
|
||||
* into the shared describeE2E above. Saves and restores the user's real
|
||||
* ~/.gbrain/sync-failures.jsonl so running E2E on a developer machine
|
||||
* doesn't trash their local sync state.
|
||||
*/
|
||||
describeE2E('E2E: sync --skip-failed structured summary loop (v0.22.12, issue #500)', () => {
|
||||
let repoPath: string;
|
||||
const realFailuresPath = join(homedir(), '.gbrain', 'sync-failures.jsonl');
|
||||
let savedFailuresContent: string | null = null;
|
||||
|
||||
beforeAll(async () => {
|
||||
await setupDB();
|
||||
|
||||
// Save+clear the real ~/.gbrain/sync-failures.jsonl so the test starts from
|
||||
// a known-empty state. Restored in afterAll. This file is per-machine, NOT
|
||||
// per-repo, so we have to be defensive about a developer running this
|
||||
// suite on their actual brain machine.
|
||||
if (existsSync(realFailuresPath)) {
|
||||
savedFailuresContent = readFileSync(realFailuresPath, 'utf-8');
|
||||
unlinkSync(realFailuresPath);
|
||||
}
|
||||
|
||||
// Fresh git repo with one valid file. Mirrors createTestRepo above but
|
||||
// scoped to this describe block.
|
||||
repoPath = mkdtempSync(join(tmpdir(), 'gbrain-skipfailed-e2e-'));
|
||||
execSync('git init', { cwd: repoPath, stdio: 'pipe' });
|
||||
execSync('git config user.email "test@test.com"', { cwd: repoPath, stdio: 'pipe' });
|
||||
execSync('git config user.name "Test"', { cwd: repoPath, stdio: 'pipe' });
|
||||
mkdirSync(join(repoPath, 'people'), { recursive: true });
|
||||
writeFileSync(join(repoPath, 'people/alice.md'), [
|
||||
'---', 'type: person', 'title: Alice', '---', '', 'Body.',
|
||||
].join('\n'));
|
||||
execSync('git add -A && git commit -m "initial"', { cwd: repoPath, stdio: 'pipe' });
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await teardownDB();
|
||||
if (repoPath) rmSync(repoPath, { recursive: true, force: true });
|
||||
|
||||
// Restore the user's real sync-failures.jsonl, if any.
|
||||
if (savedFailuresContent !== null) {
|
||||
mkdirSync(join(homedir(), '.gbrain'), { recursive: true });
|
||||
writeFileSync(realFailuresPath, savedFailuresContent);
|
||||
} else if (existsSync(realFailuresPath)) {
|
||||
// Test wrote one but there was none before. Clean up.
|
||||
unlinkSync(realFailuresPath);
|
||||
}
|
||||
});
|
||||
|
||||
test('full --skip-failed loop: blocks on bad file, skip advances bookmark, doctor shows code breakdown', async () => {
|
||||
const { performSync } = await import('../../src/commands/sync.ts');
|
||||
const { loadSyncFailures, summarizeFailuresByCode } = await import('../../src/core/sync.ts');
|
||||
const engine = getEngine();
|
||||
|
||||
// Step 1: First sync of the clean repo — should succeed.
|
||||
let result = await performSync(engine, { repoPath, noPull: true, noEmbed: true });
|
||||
expect(result.status).toBe('first_sync');
|
||||
const firstCommit = await engine.getConfig('sync.last_commit');
|
||||
expect(firstCommit).toBeTruthy();
|
||||
|
||||
// Step 2: Add a broken file — frontmatter slug doesn't match path-derived slug.
|
||||
// The file path is people/bob.md so the path-derived slug is "people/bob",
|
||||
// but we declare slug: "wrong-slug" in frontmatter. import-file.ts:368-377
|
||||
// raises "Frontmatter slug ... does not match path-derived slug ..." which
|
||||
// classifier hits as SLUG_MISMATCH.
|
||||
writeFileSync(join(repoPath, 'people/bob.md'), [
|
||||
'---', 'type: person', 'title: Bob', 'slug: wrong-slug', '---', '', 'Body.',
|
||||
].join('\n'));
|
||||
execSync('git add -A && git commit -m "add broken bob"', { cwd: repoPath, stdio: 'pipe' });
|
||||
|
||||
// Step 3: Sync should block. Bookmark must NOT advance.
|
||||
result = await performSync(engine, { repoPath, noPull: true, noEmbed: true });
|
||||
expect(result.status).toBe('blocked_by_failures');
|
||||
const afterBlockedCommit = await engine.getConfig('sync.last_commit');
|
||||
expect(afterBlockedCommit).toBe(firstCommit); // bookmark stuck at the pre-broken commit
|
||||
|
||||
// JSONL has one unacked entry with code SLUG_MISMATCH.
|
||||
let failures = loadSyncFailures();
|
||||
expect(failures.length).toBe(1);
|
||||
expect(failures[0].code).toBe('SLUG_MISMATCH');
|
||||
expect(failures[0].acknowledged).toBeFalsy();
|
||||
// Group summary aggregates correctly across the unacked set.
|
||||
expect(summarizeFailuresByCode(failures)).toEqual([{ code: 'SLUG_MISMATCH', count: 1 }]);
|
||||
|
||||
// Step 4: Run with skipFailed — bookmark advances, entry gets acked.
|
||||
result = await performSync(engine, { repoPath, noPull: true, noEmbed: true, skipFailed: true });
|
||||
expect(result.status).toBe('synced');
|
||||
const afterSkipCommit = await engine.getConfig('sync.last_commit');
|
||||
expect(afterSkipCommit).not.toBe(firstCommit); // bookmark moved past the broken commit
|
||||
failures = loadSyncFailures();
|
||||
expect(failures.length).toBe(1);
|
||||
expect(failures[0].acknowledged).toBe(true);
|
||||
expect(typeof failures[0].acknowledged_at).toBe('string');
|
||||
|
||||
// Step 5: Verify what doctor would render for the historical entry.
|
||||
// We call the same primitives doctor's `sync_failures` check uses
|
||||
// (src/commands/doctor.ts:252-275) — loadSyncFailures + summarizeFailuresByCode —
|
||||
// and assert the rendering string. Directly invoking runDoctor() here is a CLI
|
||||
// entrypoint with stdout/exit side effects that would truncate this test mid-flow.
|
||||
{
|
||||
const all = loadSyncFailures();
|
||||
const ackedSummary = summarizeFailuresByCode(all);
|
||||
const ackedBreakdown = ackedSummary.map(s => `${s.code}=${s.count}`).join(', ');
|
||||
// This is the literal string interpolation doctor.ts:271-274 produces.
|
||||
const doctorMessage = `${all.length} historical sync failure(s), all acknowledged [${ackedBreakdown}].`;
|
||||
expect(doctorMessage).toContain('SLUG_MISMATCH=1');
|
||||
expect(doctorMessage).toContain('1 historical');
|
||||
}
|
||||
|
||||
// Step 6: Add a second broken file — this one with a different failure code
|
||||
// (also SLUG_MISMATCH but on a different file) so the JSONL has 2 entries
|
||||
// with DIFFERENT paths but the same code. This proves both: per-file dedup
|
||||
// honors path identity, and summary aggregation sums across files.
|
||||
//
|
||||
// We'd ideally test a different code class here, but the sync path uses
|
||||
// parseMarkdown WITHOUT {validate:true}, so the markdown.ts validation
|
||||
// codes (MISSING_OPEN/CLOSE, NESTED_QUOTES, EMPTY_FRONTMATTER, NULL_BYTES)
|
||||
// don't naturally surface — they'd need {validate:true} plumbed in. That
|
||||
// plumbing is the v0.22.13+ follow-up. For v0.22.12, two SLUG_MISMATCH
|
||||
// entries from different files still proves the dedup + aggregation chain.
|
||||
writeFileSync(join(repoPath, 'people/carol.md'), [
|
||||
'---', 'type: person', 'title: Carol', 'slug: also-wrong-slug', '---', '', 'Body.',
|
||||
].join('\n'));
|
||||
execSync('git add -A && git commit -m "add carol with bad slug"', { cwd: repoPath, stdio: 'pipe' });
|
||||
|
||||
// Step 7: Sync blocks again on the new failure. Old entry stays acked.
|
||||
result = await performSync(engine, { repoPath, noPull: true, noEmbed: true });
|
||||
expect(result.status).toBe('blocked_by_failures');
|
||||
failures = loadSyncFailures();
|
||||
expect(failures.length).toBe(2);
|
||||
const acked = failures.filter(f => f.acknowledged);
|
||||
const unacked = failures.filter(f => !f.acknowledged);
|
||||
expect(acked.length).toBe(1);
|
||||
expect(acked[0].code).toBe('SLUG_MISMATCH');
|
||||
expect(acked[0].path).toContain('bob');
|
||||
expect(unacked.length).toBe(1);
|
||||
expect(unacked[0].code).toBe('SLUG_MISMATCH');
|
||||
expect(unacked[0].path).toContain('carol');
|
||||
|
||||
// Step 8: Skip again — both entries acked, summary aggregates the count.
|
||||
result = await performSync(engine, { repoPath, noPull: true, noEmbed: true, skipFailed: true });
|
||||
expect(result.status).toBe('synced');
|
||||
failures = loadSyncFailures();
|
||||
expect(failures.length).toBe(2);
|
||||
expect(failures.every(f => f.acknowledged)).toBe(true);
|
||||
|
||||
const finalSummary = summarizeFailuresByCode(failures);
|
||||
expect(finalSummary).toEqual([{ code: 'SLUG_MISMATCH', count: 2 }]);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -193,6 +193,21 @@ describe('classifyErrorCode — error message to code mapping', () => {
|
||||
expect(classifyErrorCode('invalid UTF-8 sequence at position 500')).toBe('INVALID_UTF8');
|
||||
});
|
||||
|
||||
test('classifies FILE_TOO_LARGE across all three production sites', async () => {
|
||||
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
||||
// src/core/import-file.ts:352 — OS-level file size on disk
|
||||
expect(classifyErrorCode('File too large (8432105 bytes)')).toBe('FILE_TOO_LARGE');
|
||||
// src/core/import-file.ts:199 — content size limit (5MB cap)
|
||||
expect(classifyErrorCode('Content too large (6000000 bytes, max 5000000). Split the content into smaller files or remove large embedded assets.')).toBe('FILE_TOO_LARGE');
|
||||
// src/core/import-file.ts:401 — code file size cap
|
||||
expect(classifyErrorCode('Code file too large (8000000 bytes)')).toBe('FILE_TOO_LARGE');
|
||||
});
|
||||
|
||||
test('classifies SYMLINK_NOT_ALLOWED from import-file.ts symlink rejection', async () => {
|
||||
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
||||
expect(classifyErrorCode('Skipping symlink: /path/to/link.md')).toBe('SYMLINK_NOT_ALLOWED');
|
||||
});
|
||||
|
||||
test('returns UNKNOWN for unrecognized errors', async () => {
|
||||
const { classifyErrorCode } = await import('../src/core/sync.ts');
|
||||
expect(classifyErrorCode('something completely different')).toBe('UNKNOWN');
|
||||
|
||||
Reference in New Issue
Block a user