Compare commits

...
Author SHA1 Message Date
Garry Tan c4efbcaf70 fix(jobs): chronicle_extract must re-resolve models from the engine (#3387)
`registerBuiltinJob` wraps a handler with `refreshGatewayForJob(engine)` only
when its name is in GATEWAY_REFRESH_JOB_NAMES. That refresh calls
`reconfigureGatewayWithEngine`, which resolves `models.chat` from the DB
config plane via `resolveModel(engine, { configKey: 'models.chat', … })`,
falling back to the file/env plane.

chronicle_extract was not in the set, so it saw only the connect-time file/env
config. A chat model set with `gbrain config set` never reached the gateway,
and extract-events.ts's `if (!isAvailable('chat')) return { events: [] }`
produced a silent `no_events` — the reported symptom.

Why this survived: the bug is invisible when the model comes from an
environment variable, because the connect-time config already carries it. One
reviewer ran the reporter's repro live, it passed, and they concluded
not-a-bug on that basis. A second reviewer caught it statically. Both were
right about different config planes; the DB plane is the one that fails.

Tests pin set membership, since that IS the mechanism, plus a floor listing
every chat-calling job so the next one dropped from this hand-maintained list
fails loudly. 3 pass with the fix, 2 fail on master.

Note the wider hazard, filed separately: this set is hand-maintained, so any
job added without an entry silently ignores DB-plane model config.
2026-07-29 11:59:45 -07:00
Sean Gearin 1057bf4368 feat(import): standalone importer seeding a brain from envelope-v0 chat-history files (#3549)
One Markdown page per conversation from an envelope-v0 file (format spec:
github.com/memvelope/memvelope), written into a directory gbrain sync
ingests. Zero dependencies, deterministic, no network; does not call gbrain.

Filenames are date + conversation id (collision-proof natural key; duplicate
ids overwrite their own file and warn on stderr). Frontmatter carries
type: conversation, source provider, conversation id, and origin. Bodies keep
message-id citations per speaker turn.

Ships as script + test + fixture only; usage and verification steps live in
the script header.
2026-07-29 11:56:20 -07:00
913d2d7f79 fix(test): give slow setup hooks a real timeout budget (#3566)
bun ignores bunfig.toml's timeout key, and beforeAll/beforeEach hooks do
NOT inherit a test's third-arg timeout — a bare `bun test` gives every
hook the 5000ms default even when all tests in the file declare 30s+.
Measured on bun 1.3.14: a 6s hook dies at ~5001ms with the signature
`(unnamed) [5001ms] ... hook timed out` (the #3545 jsonb-parity CI
failure); both `beforeAll(fn, ms)` and the CLI `--timeout` flag are
enforced hook budgets (kills observed at exactly the configured ms).

Fixes:
- e2e.yml (jsonb-parity, tier1, tier2) and release.yml ran bare
  `bun test`; they now pass --timeout=60000 like every scripts/ runner.
- test/e2e/jsonb-roundtrip.test.ts (the #2339 double-encode guard, which
  only real Postgres can surface) additionally carries per-hook 60s
  budgets so a bare local run can't flake either — same pattern as its
  sibling op-checkpoint-jsonb-parity.test.ts.
- scripts/check-bun-test-timeout.sh: CI guard (run from test.yml's
  verify job) failing any future bare `bun test` in workflows/scripts.
- scripts/run-e2e.sh: correct the comment claiming --timeout is
  per-test-only (it covers hooks; the outer gtimeout exists for
  sync-blocking WASM hangs where no timer can fire).

Proof: with Postgres paused for 6s during setupDB's connect, the
unfixed file fails at 5001.81ms with the exact CI signature; the fixed
file passes the identical condition (5 pass, 6.57s). 396 slow
before-hooks across 362 test files lack per-hook budgets; all of them
run through --timeout-passing invocations after this change, enforced
by the new guard.

Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-07-28 20:08:36 -07:00
f9349ba07f fix(doctor,cycle): stop permanent cycle_freshness FAILs on multi-source installs (#2540) (#3562)
Closes the paths #3382 left open (its author said it narrowed the issue
rather than closing it):

1. checkCycleFreshness iterates EVERY local_path source, so an install
   that nightly-dreams one vault via --dir showed a permanent FAIL for
   every other federated source — and for any source added minutes ago.
   'Never completed a full cycle' is now a WARN with the dream/autopilot
   hint; a source that HAS cycled and then went stale still escalates
   through the 6h warn / 24h fail thresholds (the regression signal the
   check exists for). This is the reporter's actual case: the permanent
   red eroded doctor's signal until real staleness hid inside it.

2. resolveSourceForDir's exact-match lookup had no archived filter and
   no ORDER BY, so an archived (or duplicate) alias of the same path
   could shadow the active source; dream's archived guard then refused
   the stamp and the ACTIVE source stayed unstamped forever. The lookup
   now excludes archived rows and orders deterministically, matching
   the canonical-path fallback's posture. The fallback's fail-closed
   ambiguity handling is deliberately unchanged.

3. #3382's own regression test (ii) was environment-sensitive: it
   assumed unsetting OPENAI_API_KEY/ANTHROPIC_API_KEY makes the embed
   phase fail, which is false wherever another embedding provider
   resolves (the cycle then reports 'clean' and the test flips). It now
   fails the sync phase against a vanished checkout — deterministic on
   every machine, same property pinned (a genuinely failing enabled
   phase must prevent the stamp).

New pins fail on unmodified master and pass here: never-cycled→warn
(x2, doctor) and the archived-alias shadow (dream --dir stamp).

Fixes #2540

Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-07-28 19:50:44 -07:00
MasaandTime Attakc e72d93fdb5 fix(sync): reconcile the stale old row when a rename falls back to add (#3056) (#3479)
master's rename loop swallows updateSlug failures with an empty catch
("treat as add"), and updateSlug returns void — so a zero-row UPDATE
(old slug absent) and a thrown collision are both invisible. Either way
the run falls through to importFile at the new path while the old row
stays behind live: slug occupied, 0 chunks after the next embed pass,
page count unchanged. A rename that didn't rename, with no trace.

The fix reconciles the duplicate:

- updateSlug returns the number of rows moved in both engines (a
  zero-row UPDATE does not throw; the count is the only way to see it).
- When the cheap rename didn't move a row AND the destination
  demonstrably materialized — imported, or an errorless skip AT the new
  slug (NOT an identity-dedup skip against the old row, which would mean
  nothing landed and deleting the old row would destroy the only copy) —
  the stale row is located positively by source_path = from and deleted.
  No source_path match → nothing is deleted (code-strategy imports don't
  populate source_path and fall back safely to leaving the row).
- A failed reconcile delete records a <rename:…> sentinel: the failure
  gate hard-blocks the bookmark, the auto-skip valve can never
  chronic-skip it (which would bank the duplicate permanently after a
  multi-run outage), and the rename is not checkpointed — the next run
  retries the same diff and clears the sentinel on convergence.

Co-authored-by: Time Attakc <89218912+time-attack@users.noreply.github.com>
2026-07-28 19:32:55 -07:00
Time AttakcandGarry Tan 85286a556c fix(search): stop boosting compiled_truth at default detail (#3430) (#3514)
* fix(search): stop boosting compiled_truth at default detail (#3430)

COMPILED_TRUTH_BOOST = 2.0 is applied AFTER RRF normalization, and RRF's whole
dynamic range over a 100-deep pool is 1/60 -> 1/160 (a factor of 2.67). So a
2.0x multiplier consumes roughly three quarters of the range: break-even is
`2/(60+r) >= 1/60`, i.e. r <= 60, which means ANY boosted chunk inside the
first 60 ranks outranks an unboosted rank-1 chunk. That is a categorical
filter, not a tilt.

Measured against master's own rrfFusion, with the correct answer in a
fenced_code chunk at vector rank 0:

  compiled_truth chunks in pool | final rank | in top-20
  10                            | 10         | yes
  20                            | 20         | NO
  40                            | 40         | NO
  80                            | 59         | NO

With the boost off the answer stays at rank 0 in every case.

The gate was spelled `detail !== 'high'` -- written as though `high` were the
special case. The documented contract in src/core/operations.ts is
"low (compiled truth only), medium (default, all with dedup), high (all
chunks)", which makes LOW the special one: `low` already restricts to
compiled_truth, so a boost there is a no-op among equals, while `medium` and
`high` are both meant to see everything. So the default detail was silently
compiled-truth-only, contradicting the op's own description.

Three changes:

1. The three fusion call sites now route through a named predicate,
   `shouldBoostCompiledTruth(detail)`, returning true only for 'low'.
   Extracted rather than left inline precisely because an inline expression is
   only reachable through a full hybridSearch round trip -- which is why the
   inversion went unnoticed. The predicate is directly unit-testable.

2. KNOBS_HASH_VERSION 13 -> 14. Results are cached AFTER fusion, so rows
   ranked under the old semantics would otherwise be served under the new ones
   for the whole TTL (3600s default). One-time miss spike on upgrade.

3. test/search-compiled-truth-boost-scope.test.ts pins both the mapping and
   the arithmetic, and documents the displacement it prevents.

Verified the tests discriminate: stubbing the OLD predicate body into master
(so the failure is behavioral rather than a missing export) gives 4 fail /
3 pass; with the fix, 7 pass. typecheck clean, verify 32/32, and 144 pass /
0 fail across the search + fusion + cache suites.

* fix(test): update the three remaining KNOBS_HASH_VERSION pins to 14 (#3430)

Missed in the first pass because I ran a targeted set of test files instead of
the full suite. CI shards 3, 8 and 10 caught them:

  test/search/knobs-hash-reranker.test.ts:67
  test/cross-modal-phase1.test.ts:139,149
  test/search-alias-resolved-boost.test.ts:93

Each carries the running history of why the version moved, so each gets the
13→14 rationale appended rather than just the number swapped. No pins at 13
remain anywhere in test/.

---------

Co-authored-by: Garry Tan <garrytan@gmail.com>
2026-07-28 19:27:46 -07:00
a8a3b6df9f fix(engine): exclude soft-deleted pages from getHealth counts (#1305) (#3556)
getStats() has excluded soft-deleted pages since v0.26.5, but getHealth()
kept counting raw pages rows: page_count, the islanded/orphan scan, the
entity_pages CTE (link/timeline coverage denominators), and most_connected
all included deleted pages, so brain_score never moved when a user
soft-deleted pages. Repro: 50 pages, soft-delete 40 -> getStats 10 vs
getHealth 50, orphan_pages 50, brain_score byte-identical.

Fix: every page-scoped count in getHealth now filters deleted_at IS NULL,
identically in both engines (engine-parity SQL shapes match).

Deliberate boundary: chunk/link storage counts (embed_coverage,
missing_embeddings, link_count, dead_links) stay raw until the purge phase
runs — matching getStats' documented posture — and destructive-removal
counts (#2235) deliberately keep counting all rows. stale_pages already
filtered via buildStalePagesWhere.

Test: test/health-soft-delete.test.ts — 3 of 4 tests fail behaviorally on
unmodified master (page_count 10 vs 4, orphan_pages 8 vs 0, link_coverage
0.5 vs 1), all pass with the fix.

Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-07-28 19:14:39 -07:00
29 changed files with 1211 additions and 48 deletions
+6 -3
View File
@@ -61,7 +61,10 @@ jobs:
- name: Run JSONB double-encode parity tests on real Postgres
env:
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/gbrain_test
run: bun test test/e2e/op-checkpoint-jsonb-parity.test.ts test/e2e/jsonb-roundtrip.test.ts
# --timeout also raises bun's 5s default hook budget (beforeAll/afterAll
# do NOT inherit a test's third-arg timeout; verified on bun 1.3.x).
# Every runner script in scripts/ passes it; bare invocations must too.
run: bun test --timeout=60000 test/e2e/op-checkpoint-jsonb-parity.test.ts test/e2e/jsonb-roundtrip.test.ts
tier1:
name: Tier 1 (Mechanical)
@@ -88,7 +91,7 @@ jobs:
bun-version: 1.3.13
- run: bun install
- name: Run Tier 1 E2E tests
run: bun test test/e2e/mechanical.test.ts test/e2e/mcp.test.ts
run: bun test --timeout=60000 test/e2e/mechanical.test.ts test/e2e/mcp.test.ts
env:
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/gbrain_test
@@ -155,7 +158,7 @@ jobs:
}
EOF
- name: Run Tier 2 skill tests
run: bun test test/e2e/skills.test.ts test/e2e/zeroentropy-live.test.ts
run: bun test --timeout=60000 test/e2e/skills.test.ts test/e2e/zeroentropy-live.test.ts
env:
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/gbrain_test
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
+3 -1
View File
@@ -29,7 +29,9 @@ jobs:
with:
bun-version: 1.3.13
- run: bun install
- run: bun test
# --timeout matches every scripts/ runner and covers hook budgets too
# (bunfig.toml's timeout key is ignored by bun; hooks default to 5s).
- run: bun test --timeout=60000
- run: bun run verify
- run: bun build --compile --target=${{ matrix.target }} --outfile bin/${{ matrix.artifact }} src/cli.ts
- name: Attest build provenance
+5
View File
@@ -113,6 +113,11 @@ jobs:
key: bun-cache-${{ runner.os }}-${{ hashFiles('bun.lock') }}
- run: bun install
- run: bun run verify
# Guard: no bare `bun test` in workflows/scripts — bun ignores
# bunfig.toml's timeout, and hooks (beforeAll/afterAll) get the 5s
# default regardless of per-test third-arg timeouts. Runs directly
# (not via verify's CHECKS array) to avoid a package.json edit.
- run: bash scripts/check-bun-test-timeout.sh
serial-tests:
# *.serial.test.ts at --max-concurrency=1. Lives in its own runner so
+41
View File
@@ -0,0 +1,41 @@
#!/usr/bin/env bash
# CI guard: every `bun test` invocation in workflows and runner scripts must
# pass an explicit --timeout.
#
# Why: bun ignores bunfig.toml's `timeout` key (verified on 1.3.14), so a bare
# `bun test` gets the 5000ms default for BOTH tests and beforeAll/beforeEach/
# afterAll/afterEach hooks. Hooks do NOT inherit a test's third-arg timeout —
# a file whose tests all declare `}, 30_000)` still has a 5s hook budget, and
# slow setup (Postgres connect + migrations, PGLite cold start) flakes on
# loaded CI runners with the signature `(unnamed) [5001ms] ... hook timed out`
# (the #3545 jsonb-parity failure). The CLI --timeout flag is the one measured
# mechanism that raises the hook budget uniformly; per-hook second-arg
# timeouts work too but don't scale to ~400 slow hooks.
#
# Usage: scripts/check-bun-test-timeout.sh
# Exit: 0 when clean, 1 when a bare `bun test` invocation is found.
set -euo pipefail
ROOT="$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
cd "$ROOT"
# Match executable `bun test` invocations. Exclude comment lines (#, //, *)
# and lines that already carry --timeout anywhere.
# Scope: workflows + runner scripts (the surfaces CI executes). package.json
# script bodies route through scripts/ already; editing it is out of scope here.
violations="$(grep -rnE '\bbun test\b' .github/workflows scripts 2>/dev/null \
| grep -v -- '--timeout' \
| grep -vE ':[[:space:]]*(#|//|\*)' \
| grep -v 'check-bun-test-timeout' \
|| true)"
if [ -n "$violations" ]; then
echo "FAIL: bare 'bun test' without --timeout (5s default kills slow setup hooks):" >&2
echo "$violations" >&2
echo "" >&2
echo "Add --timeout=60000 (see scripts/run-unit-shard.sh for the convention)." >&2
exit 1
fi
echo "OK: every bun test invocation passes an explicit --timeout."
+103
View File
@@ -0,0 +1,103 @@
#!/usr/bin/env node
/**
* Import an envelope-v0 file (a JSON serialization of AI chat history; format
* spec: github.com/memvelope/memvelope) into a brain repo as one Markdown page
* per conversation, which `gbrain sync` ingests.
*
* Usage:
* node scripts/envelope-to-gbrain.mjs <envelope.mve.json> [outDir]
*
* Zero dependencies. Deterministic. No network. It does NOT call gbrain — it
* only writes Markdown files.
*
* Output layout:
* - One page per conversation, filename = date + conversation id (shared
* titles cannot collide; the id is the natural key). A duplicate id
* overwrites its own filename and warns on stderr; stdout reports DISTINCT
* files written, not write calls.
* - Frontmatter: `type: conversation` (keeps pages eligible for
* conversation-facts extraction and chronicle behavior after sync), the
* source provider, the conversation id, and `origin: memvelope/envelope-v0`.
* - Page `date` is the first 10 chars of the conversation's ISO-8601
* `created_at`. Body keeps message-id citations beside each speaker turn.
*
* Memory: the whole envelope is held in memory (no streaming); envelopes are
* far smaller than the vendor exports they serialize.
*
* Verify:
* node scripts/envelope-to-gbrain.mjs test/fixtures/memvelope/sample.mve.json /tmp/out
* -> expect "wrote 1 markdown page(s)"
* bun test test/envelope-to-gbrain.test.ts
*
* STATUS: live-verified against gbrain v0.42.56.0 on 2026-07-03: the sample
* fixture -> 1 page; a real 662MB Claude export -> 353 conversations = 353
* distinct pages (no collisions), searchable after sync with provenance and
* message-id citations intact.
*/
import { readFileSync, writeFileSync, mkdirSync } from 'node:fs';
import { join } from 'node:path';
const [, , envelopePath, outDir = './brain/conversations'] = process.argv;
if (!envelopePath) {
console.error('usage: node envelope-to-gbrain.mjs <envelope.mve.json> [outDir]');
process.exit(1);
}
const env = JSON.parse(readFileSync(envelopePath, 'utf8'));
if (env.memvelope !== 'envelope-v0') {
console.error(`not an envelope-v0 file (memvelope field = ${JSON.stringify(env.memvelope)})`);
process.exit(1);
}
const slug = (s, fallback) =>
(String(s || '').toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-+|-+$/g, '') || fallback).slice(0, 60);
mkdirSync(outDir, { recursive: true });
const filesWritten = new Set();
let collisions = 0;
const conversations = env.conversations || [];
for (const [i, c] of conversations.entries()) {
const date = (c.created_at || '').slice(0, 10);
// Name the file by the conversation's own id — the natural unique key — so two
// conversations that share a date and title can never silently overwrite each
// other. The date only leads as a human/chronological sort prefix; the id
// carries uniqueness. Positional fallback keeps names unique and deterministic
// when an envelope omits an id.
const convId = (typeof c.id === 'string' && c.id.trim()) ? c.id.trim() : `conv-${i + 1}`;
const name = `${date || '0000-00-00'}-${slug(convId, `conv-${i + 1}`)}.md`;
// gbrain reads YAML frontmatter + markdown body; keep provenance in frontmatter.
// Emit `type: conversation` so gbrain stores these as conversation pages rather
// than defaulting to the generic `concept`. gbrain is open-typed — it takes an
// explicit frontmatter `type` verbatim — and its conversation-aware features
// (conversation-facts extraction, the conversation_format_coverage check,
// chronicle eligibility) key off `type == 'conversation'`.
const front = [
'---',
'type: conversation',
`title: ${JSON.stringify(c.title || 'Untitled conversation')}`,
`date: ${date || 'null'}`,
`source: ${env.meta?.source_provider || 'unknown'}`,
`memvelope_conversation_id: ${JSON.stringify(c.id)}`,
'origin: memvelope/envelope-v0',
'---',
'',
].join('\n');
const body = (c.messages || [])
.map((m) => `**${m.role === 'user' ? 'Me' : 'Assistant'}** (${m.ts || 'no timestamp'} · ${m.id}):\n\n${m.text}`)
.join('\n\n---\n\n');
// Never lose a page silently: if two conversations still map to the same
// filename (e.g. an envelope carrying duplicate ids), warn loudly instead of
// overwriting in silence, and report the count of DISTINCT files written — not
// the number of write calls, which is what hid the old title-collision bug.
if (filesWritten.has(name)) {
collisions += 1;
console.warn(`warning: filename collision on "${name}" — conversation id ${JSON.stringify(c.id)} is not unique; overwriting the earlier page.`);
}
writeFileSync(join(outDir, name), front + `# ${c.title || 'Conversation'}\n\n` + body + '\n');
filesWritten.add(name);
}
console.log(`wrote ${filesWritten.size} markdown page(s) to ${outDir} — point gbrain's sync at this directory.`);
if (collisions) {
console.warn(`warning: ${collisions} filename collision(s) — ${collisions} page(s) overwritten. Deduplicate conversation ids in the envelope to avoid data loss.`);
}
+3 -2
View File
@@ -162,8 +162,9 @@ for f in "${files[@]}"; do
if [ -n "${DATABASE_URL:-}" ]; then
psql "$DATABASE_URL" -At -c "SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE pid != pg_backend_pid() AND datname = current_database()" >/dev/null 2>&1 || true
fi
# Hard outer timeout (180s per file). bun's --timeout is per-test; if a
# PGLite WASM call hangs in beforeAll/afterAll, --timeout never fires and
# Hard outer timeout (180s per file). bun's --timeout covers tests AND
# hooks (measured on 1.3.14), but it's timer-based: a PGLite WASM call
# that blocks the event loop synchronously never lets the timer fire and
# the file wedges indefinitely. gtimeout/timeout SIGKILLs the file so the
# suite advances. gtimeout (macOS via coreutils) preferred; timeout (Linux)
# fallback; bare bun (no outer cap) if neither is installed.
+12 -2
View File
@@ -4349,8 +4349,18 @@ export async function checkCycleFreshness(
: `'${source.id}'`;
const raw = source.config?.last_full_cycle_at;
if (typeof raw !== 'string') {
// #2540: WARN, not FAIL. This check iterates EVERY local_path source,
// so on a multi-source install where only some vaults are cycled
// (e.g. one nightly `gbrain dream --dir <vault>`), a never-cycled
// sibling source turned doctor permanently red — which erodes the
// check's signal until real staleness hides inside the noise (the
// reporter's install masked genuinely stale sources for weeks this
// way). "Never cycled" also fires on a source added minutes ago.
// A source that HAS cycled and then went stale still escalates
// through the warn/fail age thresholds below — that is the
// regression signal this check exists for.
issues.push(`Source ${display} has never completed a full cycle`);
hasFailures = true;
hasWarnings = true;
continue;
}
const last = new Date(raw).getTime();
@@ -4386,7 +4396,7 @@ export async function checkCycleFreshness(
return {
name: 'cycle_freshness',
status: 'warn',
message: `${issues.join('; ')}.`,
message: `${issues.join('; ')}. Run \`gbrain dream --source <id>\` to cycle a source, or start \`gbrain autopilot\`.`,
};
}
return {
+9
View File
@@ -47,6 +47,15 @@ const GATEWAY_REFRESH_JOB_NAMES = new Set([
'embed-backfill',
'extract-takes-from-pages',
'embed-catch-up',
// #3387: chronicle_extract calls the chat model, so it must re-resolve
// models from the ENGINE (the DB config plane) before running. Without
// this entry the job sees only the connect-time file/env config, so a
// model set via `gbrain config set` is silently ignored and
// extract-events.ts's `if (!isAvailable('chat')) return { events: [] }`
// returns a silent `no_events`. The bug is invisible when the model comes
// from an env var — which is why a live repro can pass while the reported
// (DB-plane) configuration still fails.
'chronicle_extract',
]);
function registerBuiltinJob(
+81 -4
View File
@@ -2874,10 +2874,17 @@ async function performSyncInner(engine: BrainEngine, opts: SyncOpts): Promise<Sy
: await resolveSlugByPathOrSourcePath(engine, from, undefined);
// The new path doesn't yet have a row, so resolve from path only.
const newSlug = resolveSlugForPath(to);
// #3056: the cheap rename is OBSERVED, not assumed. A zero-row UPDATE
// doesn't throw, and a thrown collision used to be swallowed by an
// empty catch — both fell through to importFile, which created/updated
// the row at the new path while the old row stayed behind live. Both
// shapes now fall through to the reconcile below.
let renameApplied = false;
try {
await engine.updateSlug(oldSlug, newSlug, renameOpts);
renameApplied = (await engine.updateSlug(oldSlug, newSlug, renameOpts)) > 0;
} catch {
// Slug doesn't exist or collision, treat as add
// Destination slug occupied or invalid — treat as add; the reconcile
// below removes the stale old row once the destination materialized.
}
// Reimport at new path (picks up content changes). Wrapped to match the
// deletes/adds loops: a malformed renamed file is recorded to failedFiles
@@ -2890,9 +2897,11 @@ async function performSyncInner(engine: BrainEngine, opts: SyncOpts): Promise<Sy
// NAV-1 TOCTOU: refuse a destination that realpath-resolves outside the
// repo (committed symlink pointing out).
const filePath = join(gitContextRoot, to);
let importResult: Awaited<ReturnType<typeof importFile>> | undefined;
if (existsSync(filePath) && isPathSafe(filePath, gitContextRoot)) {
try {
const result = await importFile(engine, filePath, to, { noEmbed, sourceId: opts.sourceId, activePack: syncActivePack });
importResult = result;
if (result.status === 'imported') chunksCreated += result.chunks;
else if (result.status === 'skipped' && (result as { error?: string }).error) {
failedFiles.push({ path: to, error: String((result as { error?: string }).error) });
@@ -2901,9 +2910,68 @@ async function performSyncInner(engine: BrainEngine, opts: SyncOpts): Promise<Sy
failedFiles.push({ path: to, error: e instanceof Error ? e.message : String(e) });
}
}
// #3056 reconcile: the rename fell back to add semantics, so the row
// that still represents the OLD path is the stale half of the rename
// (git reported the old path gone; a plain delete of that path would
// remove this row). Two safety rails, both from the #3252 review:
//
// 1. Delete only after the destination demonstrably materialized —
// `imported`, or an errorless `skipped` AT the new slug. Identity
// dedup can skip against the OLD row (result.slug === oldSlug),
// in which case nothing landed at newSlug and deleting the old
// row would destroy the only copy.
// 2. Locate the stale row POSITIVELY by `source_path = from`, never
// by the oldSlug guess — after a collision, a path-derived
// fallback slug could name an unrelated (e.g. manually curated)
// row. No source_path match → nothing is deleted (this also means
// code-strategy imports, which don't populate source_path, fall
// back safely to leaving the old row rather than guessing).
//
// A failed delete records a `<rename:…>` SENTINEL (not an ordinary
// path failure): the gate hard-blocks the bookmark, and — unlike a
// plain path row — the auto-skip valve can never chronic-skip it after
// N attempts, which would advance the bookmark and make a transient
// delete outage a permanent duplicate. The sentinel clears through the
// ordinary success path once the rename converges on a later run.
let reconcileFailed = false;
if (!renameApplied && importResult !== undefined) {
const destMaterialized = importResult.status === 'imported' ||
(importResult.status === 'skipped' && !importResult.error && importResult.slug === newSlug);
if (destMaterialized) {
try {
const staleMap = await engine.resolveSlugsByPaths([from], { sourceId: opts.sourceId ?? DEFAULT_SOURCE_ID });
const staleSlug = staleMap.get(from);
if (staleSlug !== undefined && staleSlug !== newSlug) {
await engine.deletePage(staleSlug, renameOpts);
deletedSlugs.add(staleSlug); // never hand a deleted slug to auto-embed
serr(` [sync] rename reconciled: removed stale row ${staleSlug} (${from} -> ${to} fell back to add).`);
} else if (staleSlug === undefined) {
serr(` [sync] rename fallback: no row has source_path ${from}; stale row (if any) left in place.`);
}
} catch (e: unknown) {
reconcileFailed = true;
failedFiles.push({
path: `<rename:${to}>`,
error: `rename reconcile failed (stale row for ${from} not removed): ` +
`${e instanceof Error ? e.message : String(e)}`,
});
}
} else {
serr(
` [sync] rename fallback: ${from} -> ${to} did not materialize at ${newSlug} ` +
`(import ${importResult.status}); old row left in place.`,
);
}
}
// Converged (cheap rename, clean reconcile, or nothing to reconcile):
// clear any `<rename:…>` sentinel a previous failing run recorded.
if (!reconcileFailed) succeededPaths.push(`<rename:${to}>`);
pagesAffected.push(newSlug);
deletedSlugs.delete(newSlug); // #1284: rename landed on a previously-deleted slug → embeddable again
await markCompleted(to);
// A failed reconcile must NOT checkpoint: banking `to` would make the
// resume filter skip this rename on the retry run, turning a transient
// delete failure into a permanent duplicate — the exact bug being fixed.
if (!reconcileFailed) await markCompleted(to);
progress.tick(1, newSlug);
}
progress.finish();
@@ -3362,7 +3430,10 @@ async function performSyncInner(engine: BrainEngine, opts: SyncOpts): Promise<Sy
if (!gate.advanced) {
const codeBreakdown = formatCodeBreakdown(failedFiles);
if (gate.sentinelBlocked) {
// Two sentinel classes block here: `<head>` (pin ancestry broken) and
// `<rename:…>` (#3056 — a rename-reconcile delete failed and advancing
// would permanently bank the duplicate). Pick the message by which fired.
if (gate.sentinelBlocked && failedFiles.some(f => f.path === '<head>')) {
serr(
`\nSync blocked: repository history changed during sync (force-push / reset).\n` +
`${codeBreakdown}\n\n` +
@@ -3370,6 +3441,12 @@ async function performSyncInner(engine: BrainEngine, opts: SyncOpts): Promise<Sy
`a commit that doesn't match the indexed tree. Re-run sync to re-pin against ` +
`current HEAD.`,
);
} else if (gate.sentinelBlocked) {
serr(
`\nSync blocked: a rename left a stale duplicate that could not be removed:\n` +
`${codeBreakdown}\n\n` +
`The next 'gbrain sync' retries the reconcile from the same diff.`,
);
} else {
const fileFailCount = failedFiles.filter(f => isSkippablePath(f.path)).length;
serr(
+10 -1
View File
@@ -895,8 +895,17 @@ export async function resolveSourceForDir(
// (the cycleSourceId precedence) or 'default'.
if (brainDir === null) return undefined;
try {
// #2540: exclude archived rows (dream's --source guard refuses to stamp
// them, so an archived alias winning here means the stamp silently never
// lands and doctor's cycle_freshness stays red on a healthy install) and
// order deterministically so a duplicate registration of the same path
// can't shadow the active source on whichever row the engine scans first.
// Ordering matches listAllSources/sources-ops for operator-output parity.
const rows = await engine.executeRaw<{ id: string }>(
`SELECT id FROM sources WHERE local_path = $1 LIMIT 1`,
`SELECT id FROM sources
WHERE local_path = $1 AND archived = false
ORDER BY (id = 'default') DESC, id
LIMIT 1`,
[brainDir],
);
if (rows[0]) return rows[0].id;
+6 -1
View File
@@ -1951,8 +1951,13 @@ export interface BrainEngine {
* preserved via stable page_id). `opts.sourceId` scopes the UPDATE — without
* it, the bare `WHERE slug = old` matches every row across every source and
* would either rename them all OR violate the (source_id, slug) UNIQUE.
*
* Returns the number of rows moved. 0 means the old slug had no row in the
* scoped source — an UPDATE that matches nothing does NOT throw, so callers
* that need to know whether the rename actually happened (the sync rename
* path, #3056) must check the return value rather than rely on the catch.
*/
updateSlug(oldSlug: string, newSlug: string, opts?: { sourceId?: string }): Promise<void>;
updateSlug(oldSlug: string, newSlug: string, opts?: { sourceId?: string }): Promise<number>;
rewriteLinks(oldSlug: string, newSlug: string): Promise<void>;
/**
+13 -5
View File
@@ -5332,12 +5332,16 @@ export class PGLiteEngine implements BrainEngine {
// pages_with_timeline) and v0.10.3 graph layer (link_coverage, timeline_coverage,
// most_connected). Both coexist: master's brain_score is the composite
// dashboard, v0.10.3 metrics give entity-page-level granularity.
// #1305: every page-scoped count here excludes soft-deleted rows — same
// posture as getStats — so brain_score moves when the user deletes pages.
// Chunk/link counts stay raw (storage until the purge phase), matching
// getStats, and destructive-removal counts elsewhere deliberately stay raw.
const { rows: [h] } = await this.db.query(`
WITH entity_pages AS (
SELECT id, slug FROM pages WHERE type IN ('entity', 'person', 'company')
SELECT id, slug FROM pages WHERE type IN ('entity', 'person', 'company') AND deleted_at IS NULL
)
SELECT
(SELECT count(*) FROM pages) as page_count,
(SELECT count(*) FROM pages WHERE deleted_at IS NULL) as page_count,
(SELECT count(*) FROM content_chunks WHERE embedded_at IS NOT NULL)::float /
GREATEST((SELECT count(*) FROM content_chunks), 1)::float as embed_coverage,
0 as stale_pages,
@@ -5362,7 +5366,7 @@ export class PGLiteEngine implements BrainEngine {
SELECT p.slug,
(SELECT count(*) FROM links l WHERE l.from_page_id = p.id OR l.to_page_id = p.id)::int as link_count
FROM pages p
WHERE p.type IN ('entity', 'person', 'company')
WHERE p.type IN ('entity', 'person', 'company') AND p.deleted_at IS NULL
ORDER BY link_count DESC
LIMIT 5
`);
@@ -5381,6 +5385,7 @@ export class PGLiteEngine implements BrainEngine {
AND NOT EXISTS (SELECT 1 FROM links l WHERE l.from_page_id = p.id)) as islanded,
EXISTS (SELECT 1 FROM timeline_entries te WHERE te.page_id = p.id) as has_timeline
FROM pages p
WHERE p.deleted_at IS NULL
`);
const r = h as Record<string, unknown>;
@@ -5475,15 +5480,18 @@ export class PGLiteEngine implements BrainEngine {
}
// Sync
async updateSlug(oldSlug: string, newSlug: string, opts?: { sourceId?: string }): Promise<void> {
async updateSlug(oldSlug: string, newSlug: string, opts?: { sourceId?: string }): Promise<number> {
newSlug = validateSlug(newSlug);
const sourceId = opts?.sourceId ?? 'default';
// Source-qualify so a rename in source A doesn't sweep up same-slug rows
// in sources B/C/D (mirrors postgres-engine.ts).
await this.db.query(
const result = await this.db.query(
`UPDATE pages SET slug = $1, updated_at = now() WHERE slug = $2 AND source_id = $3`,
[newSlug, oldSlug, sourceId]
);
// #3056: rows moved — a zero-row UPDATE does not throw, so the count is
// the only way callers can see the no-op.
return result.affectedRows ?? 0;
}
async rewriteLinks(_oldSlug: string, _newSlug: string): Promise<void> {
+13 -5
View File
@@ -5432,12 +5432,16 @@ export class PostgresEngine implements BrainEngine {
// no outbound links). The raw islanded list is filtered through the same
// policy as `gbrain orphans` so convention pages do not count against
// dashboard health.
// #1305: every page-scoped count here excludes soft-deleted rows — same
// posture as getStats — so brain_score moves when the user deletes pages.
// Chunk/link counts stay raw (storage until the purge phase), matching
// getStats, and destructive-removal counts elsewhere deliberately stay raw.
const [h] = await sql`
WITH entity_pages AS (
SELECT id, slug FROM pages WHERE type IN ('entity', 'person', 'company')
SELECT id, slug FROM pages WHERE type IN ('entity', 'person', 'company') AND deleted_at IS NULL
)
SELECT
(SELECT count(*) FROM pages) as page_count,
(SELECT count(*) FROM pages WHERE deleted_at IS NULL) as page_count,
(SELECT count(*) FROM content_chunks WHERE embedded_at IS NOT NULL)::float /
GREATEST((SELECT count(*) FROM content_chunks), 1)::float as embed_coverage,
0 as stale_pages,
@@ -5459,7 +5463,7 @@ export class PostgresEngine implements BrainEngine {
SELECT p.slug,
(SELECT count(*) FROM links l WHERE l.from_page_id = p.id OR l.to_page_id = p.id)::int as link_count
FROM pages p
WHERE p.type IN ('entity', 'person', 'company')
WHERE p.type IN ('entity', 'person', 'company') AND p.deleted_at IS NULL
ORDER BY link_count DESC
LIMIT 5
`;
@@ -5478,6 +5482,7 @@ export class PostgresEngine implements BrainEngine {
AND NOT EXISTS (SELECT 1 FROM links l WHERE l.from_page_id = p.id)) as islanded,
EXISTS (SELECT 1 FROM timeline_entries te WHERE te.page_id = p.id) as has_timeline
FROM pages p
WHERE p.deleted_at IS NULL
`;
const pageCount = Number(h.page_count);
@@ -5569,14 +5574,17 @@ export class PostgresEngine implements BrainEngine {
}
// Sync
async updateSlug(oldSlug: string, newSlug: string, opts?: { sourceId?: string }): Promise<void> {
async updateSlug(oldSlug: string, newSlug: string, opts?: { sourceId?: string }): Promise<number> {
newSlug = validateSlug(newSlug);
const sql = this.sql;
const sourceId = opts?.sourceId ?? 'default';
// Source-qualify so a rename in source A doesn't sweep up same-slug rows
// in sources B/C/D (which would either rename them all OR fail the
// (source_id, slug) UNIQUE if the new slug already exists in another source).
await sql`UPDATE pages SET slug = ${newSlug}, updated_at = now() WHERE slug = ${oldSlug} AND source_id = ${sourceId}`;
const result = await sql`UPDATE pages SET slug = ${newSlug}, updated_at = now() WHERE slug = ${oldSlug} AND source_id = ${sourceId}`;
// #3056: rows moved — a zero-row UPDATE does not throw, so the count is
// the only way callers can see the no-op.
return result.count ?? 0;
}
async rewriteLinks(_oldSlug: string, _newSlug: string): Promise<void> {
+29 -3
View File
@@ -48,6 +48,32 @@ import {
export const RRF_K = 60;
const COMPILED_TRUTH_BOOST = 2.0;
/**
* Which detail levels get the compiled_truth boost (#3430).
*
* ONLY `low`. The documented contract (`src/core/operations.ts`) is
* "low (compiled truth only), medium (default, all with dedup), high (all
* chunks)" — so `low` is the level that privileges compiled truth, and both
* `medium` and `high` are supposed to see everything on equal footing.
*
* This was previously spelled `detail !== 'high'`, i.e. written as though
* `high` were the special case. Because COMPILED_TRUTH_BOOST is applied AFTER
* RRF normalization, and RRF's whole range over a 100-deep pool is 1/60 → 1/160,
* a 2.0x multiplier is not a tilt — break-even is `2/(60+r) >= 1/60`, so any
* boosted chunk inside the first 60 ranks outranks an unboosted rank-1 chunk.
* At the default detail that made search categorically compiled-truth-only:
* a page whose answer lived in a `fenced_code` chunk returned the prose chunk,
* and the code chunk fell out of the window entirely.
*
* Extracted as a named predicate rather than left inline at three call sites so
* the detail→boost mapping is directly testable. An inline expression can only
* be covered through a full `hybridSearch` round trip, which is why the
* original inversion went unnoticed.
*/
export function shouldBoostCompiledTruth(detail: string | null | undefined): boolean {
return detail === 'low';
}
const pendingCacheWrites = new Set<Promise<unknown>>();
/**
@@ -1169,7 +1195,7 @@ export async function hybridSearch(
const noEmbedLists = [{ list: keywordResults, k: fk }];
if (titleResults.length > 0) noEmbedLists.push({ list: titleResults, k: fk });
if (relationalList.length > 0) noEmbedLists.push({ list: relationalList, k: fk });
noEmbedResults = rrfFusionWeighted(noEmbedLists, detailResolved !== 'high');
noEmbedResults = rrfFusionWeighted(noEmbedLists, shouldBoostCompiledTruth(detailResolved));
}
if (noEmbedResults.length > 0) {
await runPostFusionStages(engine, noEmbedResults, postFusionOpts);
@@ -1413,7 +1439,7 @@ export async function hybridSearch(
const fallbackLists = [{ list: keywordResults, k: fk }];
if (titleResults.length > 0) fallbackLists.push({ list: titleResults, k: fk });
if (relationalList.length > 0) fallbackLists.push({ list: relationalList, k: fk });
fallbackResults = rrfFusionWeighted(fallbackLists, detail !== 'high');
fallbackResults = rrfFusionWeighted(fallbackLists, shouldBoostCompiledTruth(detail));
}
if (fallbackResults.length > 0) {
await runPostFusionStages(engine, fallbackResults, postFusionOpts);
@@ -1500,7 +1526,7 @@ export async function hybridSearch(
// arms BEFORE fusion so the compiled-truth authority boost skips them.
await stampUnverifiedExtractions(engine, allLists.flatMap((l) => l.list));
let fused = rrfFusionWeighted(allLists, detail !== 'high');
let fused = rrfFusionWeighted(allLists, shouldBoostCompiledTruth(detail));
// Cosine re-scoring before dedup so semantically better chunks survive.
// v0.36 (D9): hydrate from the active embedding column so rescore happens
+1 -1
View File
@@ -766,7 +766,7 @@ export function attributeKnob<K extends keyof ModeBundle>(
// written between the #3391 stale-fix (which changes which chunks count as
// current) and the operator's migration run. Same one-time global cold-miss
// pattern as the bumps above.
export const KNOBS_HASH_VERSION = 13;
export const KNOBS_HASH_VERSION = 14;
/**
* v0.36 (D8 / CDX-2) — second-arg context for the cache key. The
@@ -0,0 +1,75 @@
/**
* #3387: `chronicle_extract` must re-resolve models from the ENGINE before it runs.
*
* `registerBuiltinJob` wraps a handler with `refreshGatewayForJob(engine)` only
* when its name is in `GATEWAY_REFRESH_JOB_NAMES`. That refresh calls
* `reconfigureGatewayWithEngine`, which resolves `models.chat` from the DB
* config plane (`resolveModel(engine, { configKey: 'models.chat', … })`),
* falling back to the file/env plane.
*
* Without the entry the job sees only the connect-time file/env config, so a
* chat model set with `gbrain config set` is silently ignored and
* `extract-events.ts` returns a silent `no_events`.
*
* WHY THE TEST IS SHAPED THIS WAY: the bug is invisible when the model comes
* from an environment variable, because the connect-time config already carries
* it. A reviewer ran the reporter's repro live, it passed, and they concluded
* not-a-bug for exactly that reason. Set membership IS the mechanism, so that
* is what this pins.
*/
import { describe, test, expect } from 'bun:test';
import { readFileSync } from 'node:fs';
import { resolve } from 'node:path';
const JOBS_SRC = readFileSync(resolve(import.meta.dir, '../src/commands/jobs.ts'), 'utf-8');
/**
* Members of the GATEWAY_REFRESH_JOB_NAMES set literal.
* Scans only quoted entries on their own line, so prose in comments (which may
* legitimately contain quoted identifiers) cannot be mistaken for a member.
*/
function refreshSetMembers(): string[] {
const start = JOBS_SRC.indexOf('const GATEWAY_REFRESH_JOB_NAMES = new Set([');
if (start === -1) throw new Error('GATEWAY_REFRESH_JOB_NAMES not found — declaration moved?');
const end = JOBS_SRC.indexOf(']);', start);
if (end === -1) throw new Error('GATEWAY_REFRESH_JOB_NAMES has no terminator');
return JOBS_SRC.slice(start, end)
.split('\n')
.map((line) => line.trim())
.filter((line) => !line.startsWith('//'))
.map((line) => /^'([^']+)',$/.exec(line))
.filter((m): m is RegExpExecArray => m !== null)
.map((m) => m[1]);
}
describe('#3387: chronicle_extract re-resolves models from the engine', () => {
test('chronicle_extract is in GATEWAY_REFRESH_JOB_NAMES', () => {
// Fails on master: 13 members, this is not one of them.
expect(refreshSetMembers()).toContain('chronicle_extract');
});
test('the refresh wrapper is what consults the DB config plane', () => {
// Pins the mechanism, so a refactor that drops the engine-aware refresh
// shows up here rather than as a mystery `no_events`.
expect(JOBS_SRC).toContain('GATEWAY_REFRESH_JOB_NAMES.has(name)');
expect(JOBS_SRC).toContain('refreshGatewayForJob(engine)');
});
test('every chat-calling job stays in the set', () => {
// Regression floor: each of these invokes a chat/expansion model and would
// silently ignore DB-plane model config if dropped from the set.
const members = refreshSetMembers();
const required = [
'chronicle_extract',
'extract_facts',
'extract-conversation-facts',
'synthesize',
'patterns',
'consolidate',
'extract-takes-from-pages',
'enrich',
];
const missing = required.filter((n) => !members.includes(n));
expect(missing).toEqual([]);
});
});
+3 -2
View File
@@ -136,7 +136,7 @@ describe('D2 — knobsHash differs across cross-modal knob values', () => {
return resolveSearchMode({ mode: 'balanced' });
}
test('KNOBS_HASH_VERSION is 13 (cross-modal still appended; 12→13 embedding-provider migration #3390)', () => {
test('KNOBS_HASH_VERSION is 14 (cross-modal still appended; 13→14 compiled_truth boost scope #3430)', () => {
// v0.35 ladder: 1→2 reranker, 2→3 floor_ratio. v0.36 piggybacks on v=3
// with 7 cross-modal knobs + column/provider context. v0.40.4 (salem) +
// v0.39 T21 (master) bump to v=4 for graph_signals + schema-pack fields.
@@ -146,7 +146,8 @@ describe('D2 — knobsHash differs across cross-modal knob values', () => {
// v0.43: 9→10 relational recall arm. #1400: 10→11 query-side input_type
// finally reaches asymmetric providers — pre-fix rows were keyed on
// document-side query vectors. #2825: 11→12 hard-exclude fold (hx=).
expect(KNOBS_HASH_VERSION).toBe(13);
// #3430: 13→14 compiled_truth boost no longer applies at detail=medium.
expect(KNOBS_HASH_VERSION).toBe(14);
});
test('flipping unified_multimodal changes the hash', () => {
+14 -8
View File
@@ -38,7 +38,7 @@ import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { resetPgliteState } from './helpers/reset-pglite.ts';
import { withEnv, emptyHome } from './helpers/with-env.ts';
import { runCycle, ALL_PHASES } from '../src/core/cycle.ts';
import { mkdtempSync, writeFileSync } from 'fs';
import { mkdtempSync, writeFileSync, rmSync } from 'fs';
import { execSync } from 'child_process';
import { tmpdir } from 'os';
import { join } from 'path';
@@ -139,19 +139,25 @@ describe('#2540 (i) — pack omitting optional phases, all enabled phases comple
describe('#2540 (ii) — an enabled phase that never completes still prevents the stamp', () => {
test('every selected phase failing reports status=failed and does NOT stamp last_full_cycle_at', async () => {
await withEnv({ GBRAIN_HOME: gbrainHome, OPENAI_API_KEY: undefined, ANTHROPIC_API_KEY: undefined }, async () => {
await withEnv({ GBRAIN_HOME: gbrainHome }, async () => {
await seedSource('always-fails');
expect(await readLastFullCycleAt('always-fails')).toBeNull();
// embed is a real, always-enabled phase (no pack gate, no config
// .enabled toggle). With no embedding provider key configured it
// deterministically fails — this is NOT the fix under test, it's
// the pre-existing "an enabled phase genuinely never completes"
// case the issue says must keep failing doctor's check.
// Deterministic, environment-independent failure: run the sync phase
// against a brain directory that no longer exists. The previous shape
// ('embed' with OPENAI_API_KEY/ANTHROPIC_API_KEY unset) was
// environment-sensitive — on a machine where any OTHER embedding
// provider resolves (Voyage, ZeroEntropy, a local endpoint, …), embed
// with zero stale chunks succeeds and the cycle reports 'clean',
// flipping this test's expectation. A vanished checkout fails the
// sync phase on every machine. This is NOT the fix under test; it's
// the pre-existing "an enabled phase genuinely never completes" case
// the issue says must keep failing doctor's check.
rmSync(brainDir, { recursive: true, force: true });
const report = await runCycle(engine, {
brainDir,
sourceId: 'always-fails',
phases: ['embed'],
phases: ['sync'],
});
expect(report.status).toBe('failed');
+28 -2
View File
@@ -79,12 +79,38 @@ describe('doctor checkCycleFreshness', () => {
expect(result.message).toMatch(/gbrain dream --source/);
});
test('source with NO last_full_cycle_at (never cycled) returns fail', async () => {
test('source with NO last_full_cycle_at (never cycled) returns warn, not fail (#2540)', async () => {
// #2540: never-cycled used to FAIL, which turned doctor permanently red
// on any install that doesn't cycle every local_path source (e.g. one
// nightly `dream --dir <vault>` plus other federated sources) — and on
// any source added minutes ago. It surfaces as a warning; only a source
// that HAS cycled and then went stale escalates to fail.
await engine.executeRaw(`UPDATE sources SET local_path = NULL WHERE id = 'default'`);
await seed('virgin');
const result = await checkCycleFreshness(engine, { nowMs: NOW });
expect(result.status).toBe('fail');
expect(result.status).toBe('warn');
expect(result.message).toMatch(/never completed a full cycle/);
expect(result.message).toMatch(/gbrain dream --source/);
});
test('reporter case (#2540): one cycled vault + never-cycled siblings is warn, not permanent fail', async () => {
await engine.executeRaw(`UPDATE sources SET local_path = NULL WHERE id = 'default'`);
await seed('nightly-vault', agoH(2)); // the one vault dreamt via --dir
await seed('federated-a'); // never cycled
await seed('federated-b'); // never cycled
const result = await checkCycleFreshness(engine, { nowMs: NOW });
expect(result.status).toBe('warn');
expect(result.message).toMatch(/federated-a/);
expect(result.message).toMatch(/federated-b/);
expect(result.message).not.toMatch(/nightly-vault/);
});
test('a previously-cycled source gone stale still fails even next to never-cycled sources', async () => {
await engine.executeRaw(`UPDATE sources SET local_path = NULL WHERE id = 'default'`);
await seed('stale', agoH(72)); // real regression signal
await seed('virgin'); // never cycled — warn-only
const result = await checkCycleFreshness(engine, { nowMs: NOW });
expect(result.status).toBe('fail');
});
test('mixed sources: highest severity wins (fail > warn > ok)', async () => {
+22
View File
@@ -96,6 +96,28 @@ describe('gbrain dream --dir <path> freshness stamp (#1869)', () => {
expect(await readLastFullCycleAt('mothballed')).toBeNull();
});
}, 60_000);
test('an ARCHIVED alias of the same path does not shadow the active source (#2540)', async () => {
await withEnv({ GBRAIN_HOME: gbrainHome }, async () => {
// Ordinary shape: a source was archived and re-added under a new id
// pointing at the same checkout. Seed the archived twin FIRST so a
// filterless `LIMIT 1` scan finds it first.
await seedSource('retired-twin', true);
await seedSource('active-twin', false);
const report = await runDream(engine, ['--dir', brainDir, '--phase', 'lint', '--json']);
expect(report).toBeTruthy();
if (report) expect(['ok', 'clean']).toContain(report.status);
// Pre-fix, resolveSourceForDir's exact match had no `archived = false`
// filter and no ORDER BY, so the archived twin won the lookup; dream's
// archived guard then (correctly) refused to stamp it — and the ACTIVE
// source silently never got its stamp, leaving doctor's cycle_freshness
// permanently stale on a healthy install.
expect(await readLastFullCycleAt('active-twin')).not.toBeNull();
expect(await readLastFullCycleAt('retired-twin')).toBeNull();
});
}, 60_000);
});
/**
+6 -2
View File
@@ -26,8 +26,12 @@ if (skip) {
}
describeE2E('E2E: JSONB roundtrip — v0.12.1 reliability wave', () => {
beforeAll(async () => { await setupDB(); });
afterAll(async () => { await teardownDB(); });
// 60s hook budget: setupDB runs connect + the full migration chain, which
// exceeds bun's default 5s hook timeout on loaded CI runners. Hooks do NOT
// inherit a test's third-arg timeout (verified on bun 1.3.14) — they need
// their own second-arg budget. Same pattern as op-checkpoint-jsonb-parity.
beforeAll(async () => { await setupDB(); }, 60_000);
afterAll(async () => { await teardownDB(); }, 60_000);
test('putPage writes frontmatter as object, not double-encoded string', async () => {
const engine = getEngine();
+159
View File
@@ -0,0 +1,159 @@
/**
* Pins the Memvelope envelope importer contract: deterministic markdown output,
* provenance frontmatter, citation-bearing bodies, and loud collision handling.
*/
import { afterAll, describe, expect, test } from 'bun:test';
import { mkdtempSync, rmSync, readdirSync, readFileSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
const SCRIPT_PATH = join(import.meta.dir, '..', 'scripts', 'envelope-to-gbrain.mjs');
const FIXTURE_PATH = join(import.meta.dir, 'fixtures', 'memvelope', 'sample.mve.json');
const TEMP_DIRS: string[] = [];
afterAll(() => {
for (const dir of TEMP_DIRS) {
rmSync(dir, { recursive: true, force: true });
}
});
function tempDir(): string {
const dir = mkdtempSync(join(tmpdir(), 'envelope-to-gbrain-'));
TEMP_DIRS.push(dir);
return dir;
}
async function runImporter(envelopePath: string, outDir = tempDir()) {
// The script is plain Node-compatible ESM; Bun can execute it directly in CI
// without requiring a separate node toolchain.
const proc = Bun.spawn([process.execPath, SCRIPT_PATH, envelopePath, outDir], {
stdout: 'pipe',
stderr: 'pipe',
});
await proc.exited;
const stdout = await new Response(proc.stdout).text();
const stderr = await new Response(proc.stderr).text();
return { exitCode: proc.exitCode, stdout, stderr, outDir };
}
function markdownFiles(dir: string): string[] {
return readdirSync(dir).filter((name) => name.endsWith('.md')).sort();
}
function readOnlyMarkdown(dir: string): string {
const files = markdownFiles(dir);
expect(files).toHaveLength(1);
return readFileSync(join(dir, files[0]), 'utf8');
}
describe('envelope-to-gbrain importer', () => {
test('sample envelope writes exactly one markdown page and reports count', async () => {
const result = await runImporter(FIXTURE_PATH);
expect(result.exitCode).toBe(0);
expect(markdownFiles(result.outDir)).toHaveLength(1);
expect(result.stdout).toContain('wrote 1 markdown page(s)');
});
test('filename is keyed by conversation id with date prefix', async () => {
const result = await runImporter(FIXTURE_PATH);
expect(result.exitCode).toBe(0);
expect(markdownFiles(result.outDir)).toEqual(['2025-11-02-c-3f9a2b.md']);
});
test('frontmatter carries conversation provenance fields', async () => {
const result = await runImporter(FIXTURE_PATH);
const page = readOnlyMarkdown(result.outDir);
expect(result.exitCode).toBe(0);
expect(page).toContain('type: conversation');
expect(page).toContain('title: "Onboarding Checklist Draft"');
expect(page).toContain('date: 2025-11-02');
expect(page).toContain('source: chatgpt');
expect(page).toContain('memvelope_conversation_id: "c-3f9a2b"');
expect(page).toContain('origin: memvelope/envelope-v0');
});
test('body carries role labels and message-id citations', async () => {
const result = await runImporter(FIXTURE_PATH);
const page = readOnlyMarkdown(result.outDir);
expect(result.exitCode).toBe(0);
expect(page).toContain('· m1');
expect(page).toContain('· m4');
expect(page).toContain('**Me**');
expect(page).toContain('**Assistant**');
});
test('output is deterministic across repeated runs', async () => {
const first = await runImporter(FIXTURE_PATH);
const second = await runImporter(FIXTURE_PATH);
expect(first.exitCode).toBe(0);
expect(second.exitCode).toBe(0);
expect(readOnlyMarkdown(first.outDir)).toBe(readOnlyMarkdown(second.outDir));
});
test('duplicate conversation ids warn and report distinct files written', async () => {
const inputDir = tempDir();
const envelopePath = join(inputDir, 'duplicate.mve.json');
writeFileSync(envelopePath, JSON.stringify({
memvelope: 'envelope-v0',
meta: { source_provider: 'chatgpt' },
conversations: [
{
id: 'c-repeat',
title: 'First repeated id',
created_at: '2025-11-02T14:22:51.000Z',
messages: [{ id: 'm1', role: 'user', ts: '2025-11-02T14:22:51.000Z', text: 'alice-example noted the first checklist draft.' }],
},
{
id: 'c-repeat',
title: 'Second repeated id',
created_at: '2025-11-02T15:22:51.000Z',
messages: [{ id: 'm2', role: 'assistant', ts: '2025-11-02T15:22:51.000Z', text: 'Assistant noted the repeated id collision.' }],
},
],
}));
const result = await runImporter(envelopePath);
expect(result.exitCode).toBe(0);
expect(result.stderr).toContain('warning: filename collision on "2025-11-02-c-repeat.md"');
expect(result.stdout).toContain('wrote 1 markdown page(s)');
expect(markdownFiles(result.outDir)).toHaveLength(1);
});
test('missing or foreign format is rejected', async () => {
const inputDir = tempDir();
const envelopePath = join(inputDir, 'not-envelope.json');
writeFileSync(envelopePath, JSON.stringify({ conversations: [] }));
const result = await runImporter(envelopePath);
expect(result.exitCode).toBe(1);
expect(result.stderr).toContain('envelope-v0');
});
test('missing conversation id uses positional fallback filename', async () => {
const inputDir = tempDir();
const envelopePath = join(inputDir, 'missing-id.mve.json');
writeFileSync(envelopePath, JSON.stringify({
memvelope: 'envelope-v0',
meta: { source_provider: 'chatgpt' },
conversations: [
{
title: 'Missing id example',
created_at: '2025-11-02T14:22:51.000Z',
messages: [{ id: 'm1', role: 'user', ts: '2025-11-02T14:22:51.000Z', text: 'alice-example asked for a fallback filename.' }],
},
],
}));
const result = await runImporter(envelopePath);
expect(result.exitCode).toBe(0);
expect(markdownFiles(result.outDir)).toEqual(['2025-11-02-conv-1.md']);
});
});
+42
View File
@@ -0,0 +1,42 @@
{
"memvelope": "envelope-v0",
"meta": {
"source_provider": "chatgpt",
"conversation_count": 1,
"message_count": 4
},
"conversations": [
{
"id": "c-3f9a2b",
"title": "Onboarding Checklist Draft",
"created_at": "2025-11-02T14:22:51.000Z",
"updated_at": "2025-11-02T14:31:12.000Z",
"messages": [
{
"id": "m1",
"role": "user",
"ts": "2025-11-02T14:22:51.000Z",
"text": "alice-example is drafting acme-example's widget-co onboarding checklist and wants a concise first pass."
},
{
"id": "m2",
"role": "assistant",
"ts": "2025-11-02T14:24:03.000Z",
"text": "Start with account setup, workspace access, sample widget review, and a first-week check-in with the acme-example owner."
},
{
"id": "m3",
"role": "user",
"ts": "2025-11-02T14:28:19.000Z",
"text": "Add a note that bob-example should compare fund-a and fund-b reporting needs before the kickoff."
},
{
"id": "m4",
"role": "assistant",
"ts": "2025-11-02T14:31:12.000Z",
"text": "Include a pre-kickoff step for bob-example to list fund-a and fund-b reporting questions, then confirm owners with charlie-example."
}
]
}
]
}
+123
View File
@@ -0,0 +1,123 @@
/**
* #1305 — getHealth() must exclude soft-deleted pages from every
* page-scoped count, the same posture getStats() has had since v0.26.5.
*
* Pre-fix, getHealth counted raw `pages` rows: page_count and orphan_pages
* included soft-deleted pages, the entity_pages CTE kept deleted entities in
* the link/timeline coverage denominators and in most_connected, and
* brain_score therefore never moved when a user soft-deleted pages.
*
* Boundary (deliberate): chunk- and link-scoped counts (embed_coverage,
* missing_embeddings, link_count, dead_links) stay RAW — they occupy storage
* until the autopilot purge phase, matching getStats. Destructive-removal
* counts (purge paths, #2235) also deliberately count all rows and are
* untouched here.
*
* Runs against PGLite — the fixed SQL shapes are identical in both engines.
*/
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
let engine: PGLiteEngine;
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
}, 60_000);
afterAll(async () => {
await engine.disconnect();
});
beforeEach(async () => {
for (const t of ['links', 'content_chunks', 'timeline_entries', 'tags', 'page_versions', 'pages']) {
await (engine as any).db.exec(`DELETE FROM ${t}`);
}
});
async function seedNote(slug: string): Promise<void> {
await engine.putPage(slug, { type: 'note', title: slug, compiled_truth: `content of ${slug}`, frontmatter: {} });
}
async function pageId(slug: string): Promise<number> {
return (await (engine as any).db.query(`SELECT id FROM pages WHERE slug=$1`, [slug])).rows[0].id;
}
describe('#1305 — getHealth excludes soft-deleted pages', () => {
test('page_count and orphan_pages match getStats after soft-delete (the issue repro)', async () => {
for (let i = 0; i < 10; i++) await seedNote(`wiki/note-${i}`);
for (let i = 0; i < 6; i++) await engine.softDeletePage(`wiki/note-${i}`);
const stats = await engine.getStats();
const health = await engine.getHealth();
expect(stats.page_count).toBe(4);
// Pre-fix: 10 (raw rows). getHealth must agree with getStats.
expect(health.page_count).toBe(4);
// Pre-fix: 10 — deleted pages stayed in the islanded scan.
expect(health.orphan_pages).toBe(4);
});
test('brain_score moves when the user soft-deletes the islanded pages', async () => {
// 2 connected pages + 8 islanded ones.
await seedNote('wiki/hub');
await seedNote('wiki/leaf');
await (engine as any).db.query(
`INSERT INTO links (from_page_id, to_page_id, link_type) VALUES ($1, $2, 'mentions')`,
[await pageId('wiki/hub'), await pageId('wiki/leaf')],
);
for (let i = 0; i < 8; i++) await seedNote(`wiki/clutter-${i}`);
const before = await engine.getHealth();
for (let i = 0; i < 8; i++) await engine.softDeletePage(`wiki/clutter-${i}`);
const after = await engine.getHealth();
// Pre-fix both assertions fail: orphan_pages stayed 8 and brain_score
// was byte-identical before/after the delete.
expect(after.orphan_pages).toBe(0);
expect(after.brain_score).toBeGreaterThan(before.brain_score);
});
test('entity coverage denominators and most_connected exclude deleted entities', async () => {
// Live entity: inbound link + timeline entry → full coverage.
await engine.putPage('people/alice-example', { type: 'person', title: 'Alice', compiled_truth: 'a person', frontmatter: {} });
await engine.putPage('people/bob-example', { type: 'person', title: 'Bob', compiled_truth: 'another person', frontmatter: {} });
await seedNote('wiki/mentions-alice');
const aliceId = await pageId('people/alice-example');
await (engine as any).db.query(
`INSERT INTO links (from_page_id, to_page_id, link_type) VALUES ($1, $2, 'mentions')`,
[await pageId('wiki/mentions-alice'), aliceId],
);
await (engine as any).db.query(
`INSERT INTO timeline_entries (page_id, date, summary) VALUES ($1, '2026-01-01', 'met alice')`,
[aliceId],
);
await engine.softDeletePage('people/bob-example');
const h = await engine.getHealth();
// Pre-fix: bob stayed in the entity_pages CTE → coverage 0.5 each,
// and bob appeared in most_connected.
expect(h.link_coverage).toBe(1);
expect(h.timeline_coverage).toBe(1);
expect(h.most_connected.map((c) => c.slug)).not.toContain('people/bob-example');
});
test('chunk storage counts stay raw (the deliberate boundary)', async () => {
await seedNote('wiki/kept');
await seedNote('wiki/gone');
for (const slug of ['wiki/kept', 'wiki/gone']) {
await (engine as any).db.query(
`INSERT INTO content_chunks (page_id, chunk_index, chunk_text) VALUES ($1, 0, 'chunk')`,
[await pageId(slug)],
);
}
await engine.softDeletePage('wiki/gone');
const h = await engine.getHealth();
// Soft-deleted pages' chunks still occupy storage until purge; the
// missing_embeddings count keeps seeing them, same as getStats.
expect(h.missing_embeddings).toBe(2);
});
});
+2 -2
View File
@@ -89,7 +89,7 @@ describe('alias_resolved boost stage', () => {
});
describe('KNOBS_HASH_VERSION', () => {
it('is 13 (12→13 embedding-provider migration invalidates rows written against the prior embedding space, #3390)', () => {
expect(KNOBS_HASH_VERSION).toBe(13);
it('is 14 (13→14 compiled_truth boost no longer applies at detail=medium, so pre-fix rankings must be unreachable, #3430)', () => {
expect(KNOBS_HASH_VERSION).toBe(14);
});
});
@@ -0,0 +1,112 @@
/**
* #3430: the compiled_truth boost must not apply at `detail=medium`.
*
* `COMPILED_TRUTH_BOOST = 2.0` is applied AFTER RRF score normalization. RRF's
* entire dynamic range over a 100-deep pool is 1/60 → 1/160 (a factor of 2.67),
* so a 2.0x multiplier consumes roughly three quarters of it. Break-even is
* `2/(60+r) >= 1/60`, i.e. r <= 60 — so ANY boosted chunk in the first 60 ranks
* outranks an unboosted rank-1 chunk. That is a categorical filter, not a tilt:
* a page whose actual answer is in a `fenced_code` chunk returns the prose
* chunk instead, and the code chunk leaves the result window entirely.
*
* The gate was written as `detail !== 'high'` — "high is special" — but the
* documented contract in `src/core/operations.ts` is:
*
* low (compiled truth only), medium (default, all with dedup), high (all chunks)
*
* which makes LOW the special one. `low` already restricts to compiled_truth,
* so a boost there is a no-op among equals; `medium` and `high` are both
* supposed to see everything. Hence `detail === 'low'`.
*
* These tests pin the arithmetic, not the constant — they would still fail if
* someone reintroduced a boost at medium with a different multiplier or behind
* a score floor, which is why they assert final RANK rather than score.
*/
import { describe, test, expect } from 'bun:test';
import { rrfFusion, RRF_K, shouldBoostCompiledTruth } from '../src/core/search/hybrid.ts';
import { KNOBS_HASH_VERSION } from '../src/core/search/mode.ts';
import type { SearchResult } from '../src/core/types.ts';
function chunk(slug: string, chunkSource: string): SearchResult {
return { slug, chunk_source: chunkSource, chunk_text: 'x', title: slug, score: 0 } as unknown as SearchResult;
}
/** One vector arm: the correct answer at rank 0, then `n` compiled_truth chunks. */
function poolWithAnswerFirst(n: number): SearchResult[] {
const list = [chunk('code/answer', 'fenced_code')];
for (let i = 0; i < n; i++) list.push(chunk(`prose/p${i}`, 'compiled_truth'));
return list;
}
function rankOfAnswer(results: SearchResult[]): number {
return results.findIndex((r) => r.slug === 'code/answer');
}
describe('#3430: the detail→boost mapping itself', () => {
// These are the assertions that actually FAIL on master. The rrfFusion tests
// below pin the arithmetic but pass either way, because they pass the boost
// flag explicitly — they cannot see how hybridSearch decides it. This is the
// wiring.
test('ONLY detail=low boosts compiled_truth', () => {
expect(shouldBoostCompiledTruth('low')).toBe(true);
expect(shouldBoostCompiledTruth('medium')).toBe(false);
expect(shouldBoostCompiledTruth('high')).toBe(false);
});
test('an absent detail does not boost — medium is the documented default', () => {
// Callers that omit detail get medium semantics, so the unset case must
// match medium, not low. A `!== 'high'` spelling gets this backwards.
expect(shouldBoostCompiledTruth(undefined)).toBe(false);
expect(shouldBoostCompiledTruth(null)).toBe(false);
});
test('an unrecognized detail value does not boost', () => {
// Fail-open toward showing everything rather than silently filtering.
expect(shouldBoostCompiledTruth('')).toBe(false);
expect(shouldBoostCompiledTruth('LOW')).toBe(false);
expect(shouldBoostCompiledTruth('detailed')).toBe(false);
});
test('the cache version was bumped so pre-fix rankings are unreachable', () => {
// Results are cached AFTER fusion, so rows written under the old boost
// semantics would otherwise be served under the new ones for the whole TTL.
// 13 was the pre-fix value.
expect(KNOBS_HASH_VERSION).toBeGreaterThanOrEqual(14);
});
});
describe('#3430: compiled_truth boost scope', () => {
test('boost OFF (detail=medium/high) keeps the vector-ranked answer at rank 0', () => {
// The regression this file exists for. Pre-fix, medium passed applyBoost=true
// and the answer landed at rank n — outside a 20-result window for n >= 20.
for (const n of [10, 20, 40, 80]) {
const fused = rrfFusion([poolWithAnswerFirst(n)], RRF_K, false);
expect(rankOfAnswer(fused), `n=${n}: answer must stay first without the boost`).toBe(0);
}
});
test('boost ON demonstrates the categorical displacement it causes', () => {
// Documents WHY the boost cannot be on at medium. Not an endorsement of
// these numbers — a characterization of the mechanism, so a future reader
// sees the cost rather than re-deriving it.
const observed = [10, 20, 40].map((n) => ({
n,
rank: rankOfAnswer(rrfFusion([poolWithAnswerFirst(n)], RRF_K, true)),
}));
// Displacement scales with pool composition: the answer is pushed back by
// roughly one position per boosted chunk ahead of the break-even rank.
for (const { n, rank } of observed) {
expect(rank, `n=${n}: boosted chunks should displace the answer`).toBeGreaterThan(0);
}
// And past ~20 compiled_truth chunks it leaves a default-size window.
expect(observed.find((o) => o.n === 20)!.rank).toBeGreaterThanOrEqual(20);
});
test('with the boost off, compiled_truth still wins when the vector arm ranks it first', () => {
// Guard against over-correcting: removing the boost must not penalize
// compiled_truth, only stop privileging it.
const list = [chunk('prose/answer', 'compiled_truth'), chunk('code/other', 'fenced_code')];
const fused = rrfFusion([list], RRF_K, false);
expect(fused[0].slug).toBe('prose/answer');
});
});
+6 -3
View File
@@ -413,7 +413,10 @@ describe('knobsHash determinism + cross-mode separation (CDX-4)', () => {
// #3390/#3391: bumped 12→13 for the embedding-provider migration wave —
// legacy callers hash prov=default before AND after a provider swap, so
// pre-migration cache rows must become unreachable on upgrade.
expect(KNOBS_HASH_VERSION).toBe(13);
// v0.42.67.x bumped 13→14: the compiled_truth boost no longer applies at
// detail=medium (#3430). Cached rows were ranked under the old semantics,
// so they must become unreachable rather than be served under the new ones.
expect(KNOBS_HASH_VERSION).toBe(14);
});
test('T1 (codex): floor_ratio set vs unset produces DIFFERENT hashes (cache contamination prevention)', () => {
@@ -578,8 +581,8 @@ describe('v0.40.4 — graph_signals knob', () => {
});
describe('v0.42.3.0 — autocut knobs', () => {
test('KNOBS_HASH_VERSION is 13 (12→13 embedding-migration wave, #3390/#3391)', () => {
expect(KNOBS_HASH_VERSION).toBe(13);
test('KNOBS_HASH_VERSION is 14 (13→14 compiled_truth boost scope fix, #3430)', () => {
expect(KNOBS_HASH_VERSION).toBe(14);
});
test('bundle defaults: conservative off, balanced/tokenmax on @0.20', () => {
+4 -1
View File
@@ -64,7 +64,10 @@ describe('KNOBS_HASH_VERSION + version invariants', () => {
// pre-fix document-side query vectors must not be served.
// #2825: 11→12 to fold the resolved hard-exclude prefix list (hx=) —
// cached rows leaked GBRAIN_SEARCH_EXCLUDE'd slugs across processes.
expect(KNOBS_HASH_VERSION).toBe(13);
// #3430: 13→14 — the compiled_truth boost no longer applies at
// detail=medium. Results are cached after fusion, so rows ranked under
// the old boost semantics must not be served under the new ones.
expect(KNOBS_HASH_VERSION).toBe(14);
});
test('hash is 16 hex chars regardless of reranker config', () => {
+280
View File
@@ -0,0 +1,280 @@
/**
* #3056 — sync rename path: a failed `updateSlug` must not leave a live
* duplicate of the renamed page behind.
*
* Before the fix, the rename loop swallowed `updateSlug` failures with an
* empty catch ("treat as add") and could not see a zero-row UPDATE at all
* (updateSlug returned void). The run then fell through to importFile,
* which created/updated the row at the new path — while the old row stayed
* behind, live, with its slug occupied. Nothing was logged, no counter
* moved, and the duplicate was permanent.
*
* The fix reconciles: when the cheap rename didn't move a row AND the
* destination demonstrably materialized, the stale old row is located
* positively by `source_path = from` and deleted. Two safety rails:
*
* - dedup-skip protection: identity dedup can skip the import against
* the OLD row, in which case nothing landed at the destination and
* deleting the old row would destroy the only copy — no reconcile.
* - no slug-guess deletes: the stale row is found by source_path only;
* an unrelated row that happens to sit at the guessed slug survives.
*
* A failed reconcile delete lands in failedFiles so the existing failure
* gate blocks the bookmark and the next run retries the same rename diff.
*/
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, test } from 'bun:test';
import { execSync } from 'node:child_process';
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { resetPgliteState } from './helpers/reset-pglite.ts';
let engine: PGLiteEngine;
const repos: string[] = [];
// Serial-file requirement: blocked runs write real rows to the sync-failure
// ledger under the gbrain home — isolate it per test so the operator's
// actual ledger is never touched (GBRAIN_HOME is the isolation lever;
// process.env.HOME does not redirect Bun's os.homedir()).
let tmpHome: string;
const originalGbrainHome = process.env.GBRAIN_HOME;
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
});
afterAll(async () => {
await engine.disconnect();
});
beforeEach(async () => {
tmpHome = mkdtempSync(join(tmpdir(), 'gbrain-3056-home-'));
process.env.GBRAIN_HOME = tmpHome;
await resetPgliteState(engine);
});
afterEach(() => {
if (originalGbrainHome !== undefined) process.env.GBRAIN_HOME = originalGbrainHome;
else delete process.env.GBRAIN_HOME;
try { rmSync(tmpHome, { recursive: true, force: true }); } catch { /* ignore */ }
while (repos.length) {
const d = repos.pop();
if (d) rmSync(d, { recursive: true, force: true });
}
});
function personMd(title: string, body: string): string {
return ['---', 'type: person', `title: ${title}`, '---', '', body].join('\n');
}
/** Create a temp git repo seeded with the given files + an initial commit. */
function mkRepo(files: Record<string, string>): string {
const dir = mkdtempSync(join(tmpdir(), 'gbrain-3056-'));
repos.push(dir);
execSync('git init', { cwd: dir, stdio: 'pipe' });
execSync('git config user.email "test@test.com"', { cwd: dir, stdio: 'pipe' });
execSync('git config user.name "Test"', { cwd: dir, stdio: 'pipe' });
for (const [rel, content] of Object.entries(files)) {
mkdirSync(join(dir, rel, '..'), { recursive: true });
writeFileSync(join(dir, rel), content);
}
execSync('git add -A && git commit -m "initial"', { cwd: dir, stdio: 'pipe' });
return dir;
}
const SYNC_OPTS = { noPull: true, noEmbed: true, noExtract: true, sourceId: 'default' } as const;
async function countPages(): Promise<number> {
const rows = await engine.executeRaw<{ n: number | string }>(
`SELECT count(*)::int AS n FROM pages WHERE source_id = 'default'`,
);
return Number(rows[0]?.n ?? 0);
}
describe('updateSlug engine contract (#3056)', () => {
test('returns 1 when the old slug row is moved', async () => {
await engine.putPage('people/old', {
type: 'person', title: 'Old', compiled_truth: 'body',
}, { sourceId: 'default' });
const moved = await engine.updateSlug('people/old', 'people/new', { sourceId: 'default' });
expect(moved).toBe(1);
expect(await engine.getPage('people/new')).not.toBeNull();
});
test('returns 0 when the old slug has no row (the silent no-op case)', async () => {
const moved = await engine.updateSlug('people/ghost', 'people/new', { sourceId: 'default' });
expect(moved).toBe(0);
});
});
describe('#3056: rename fallback reconciles the stale old row', () => {
test('collision: destination slug occupied → stale old row deleted after import lands', async () => {
const { performSync } = await import('../src/commands/sync.ts');
const repo = mkRepo({ 'people/carol.md': personMd('Carol', 'Carol is a person.') });
await performSync(engine, { repoPath: repo, ...SYNC_OPTS });
expect(await engine.getPage('people/carol')).not.toBeNull();
// A pre-existing row already occupies the rename destination, so
// updateSlug throws (source_id, slug) UNIQUE and the loop falls back.
await engine.putPage('people/dana', {
type: 'person', title: 'Dana (stale)', compiled_truth: 'occupies the destination slug',
}, { sourceId: 'default' });
execSync('git mv people/carol.md people/dana.md', { cwd: repo, stdio: 'pipe' });
execSync('git commit -m "rename carol to dana"', { cwd: repo, stdio: 'pipe' });
const result = await performSync(engine, { repoPath: repo, ...SYNC_OPTS });
expect(result.status).toBe('synced');
// The destination carries the renamed file's content...
const dana = await engine.getPage('people/dana');
expect(dana).not.toBeNull();
expect(dana!.compiled_truth).toContain('Carol is a person.');
// ...and the stale old row is gone — no live duplicate.
expect(await engine.getPage('people/carol')).toBeNull();
expect(await countPages()).toBe(1);
});
test('dedup-skip against the old row must NOT reconcile: the only copy survives', async () => {
const { performSync } = await import('../src/commands/sync.ts');
// frontmatter.id gives identity dedup a handle: the import at the new
// path can skip as "identical to <old row>" — in which case NOTHING
// landed at the destination and deleting the old row would destroy the
// only copy of the content.
const md = ['---', 'type: person', 'title: Carol', 'id: ext-3056', '---', '', 'Carol is a person.'].join('\n');
const repo = mkRepo({ 'people/carol.md': md });
await performSync(engine, { repoPath: repo, ...SYNC_OPTS });
expect(await engine.getPage('people/carol')).not.toBeNull();
// Destination occupied → updateSlug throws → fallback path.
await engine.putPage('people/dana', {
type: 'person', title: 'Dana (stale)', compiled_truth: 'occupies the destination slug',
}, { sourceId: 'default' });
execSync('git mv people/carol.md people/dana.md', { cwd: repo, stdio: 'pipe' });
execSync('git commit -m "rename carol to dana"', { cwd: repo, stdio: 'pipe' });
await performSync(engine, { repoPath: repo, ...SYNC_OPTS });
// The import skipped against the OLD row (identity dedup), so the
// destination never materialized with the renamed content — the
// reconcile must not have deleted the old row, which still holds the
// only copy.
const carol = await engine.getPage('people/carol');
expect(carol).not.toBeNull();
expect(carol!.compiled_truth).toContain('Carol is a person.');
});
test('reconcile never deletes by slug guess: unrelated manual row survives', async () => {
const { performSync } = await import('../src/commands/sync.ts');
const repo = mkRepo({ 'people/carol.md': personMd('Carol', 'Carol is a person.') });
await performSync(engine, { repoPath: repo, ...SYNC_OPTS });
// The file's real row drifts to a divergent slug with no source_path
// (unlocatable), and an UNRELATED manually-curated page happens to sit
// at the path-derived slug a naive reconcile would guess.
await engine.executeRaw(
`UPDATE pages SET slug = 'people/carol-divergent', source_path = NULL
WHERE source_id = 'default' AND slug = 'people/carol'`,
);
await engine.putPage('people/carol', {
type: 'person', title: 'Manual Carol', compiled_truth: 'hand-authored, not from the file',
}, { sourceId: 'default' });
// Destination occupied → updateSlug throws UNIQUE → fallback path.
await engine.putPage('people/dana', {
type: 'person', title: 'Dana (stale)', compiled_truth: 'occupies the destination slug',
}, { sourceId: 'default' });
execSync('git mv people/carol.md people/dana.md', { cwd: repo, stdio: 'pipe' });
execSync('git commit -m "rename carol to dana"', { cwd: repo, stdio: 'pipe' });
const result = await performSync(engine, { repoPath: repo, ...SYNC_OPTS });
expect(result.status).toBe('synced');
// The destination materialized with the file's content...
const dana = await engine.getPage('people/dana');
expect(dana).not.toBeNull();
expect(dana!.compiled_truth).toContain('Carol is a person.');
// ...but no row had source_path = from, so the reconcile deleted
// NOTHING: the unrelated manual row at the guessed slug survives.
const manual = await engine.getPage('people/carol');
expect(manual).not.toBeNull();
expect(manual!.compiled_truth).toContain('hand-authored');
});
test('happy path: clean git mv rename keeps page_id and touches nothing else', async () => {
const { performSync } = await import('../src/commands/sync.ts');
const repo = mkRepo({ 'people/carol.md': personMd('Carol', 'Carol is a person.') });
await performSync(engine, { repoPath: repo, ...SYNC_OPTS });
const before = await engine.getPage('people/carol');
expect(before).not.toBeNull();
execSync('git mv people/carol.md people/dana.md', { cwd: repo, stdio: 'pipe' });
execSync('git commit -m "rename carol to dana"', { cwd: repo, stdio: 'pipe' });
const result = await performSync(engine, { repoPath: repo, ...SYNC_OPTS });
expect(result.status).toBe('synced');
const after = await engine.getPage('people/dana');
expect(after).not.toBeNull();
expect(after!.id).toBe(before!.id); // cheap-path rename preserved the row
expect(await engine.getPage('people/carol')).toBeNull();
expect(await countPages()).toBe(1);
});
test('reconcile failure blocks the bookmark and the next run retries to convergence', async () => {
const { performSync } = await import('../src/commands/sync.ts');
const repo = mkRepo({ 'people/carol.md': personMd('Carol', 'Carol is a person.') });
await performSync(engine, { repoPath: repo, ...SYNC_OPTS });
await engine.putPage('people/dana', {
type: 'person', title: 'Dana (stale)', compiled_truth: 'occupies the destination slug',
}, { sourceId: 'default' });
execSync('git mv people/carol.md people/dana.md', { cwd: repo, stdio: 'pipe' });
execSync('git commit -m "rename carol to dana"', { cwd: repo, stdio: 'pipe' });
// Inject a transient failure into the reconcile delete.
const origDelete = engine.deletePage.bind(engine);
engine.deletePage = async () => { throw new Error('injected transient delete failure'); };
let blocked;
try {
blocked = await performSync(engine, { repoPath: repo, ...SYNC_OPTS });
} finally {
engine.deletePage = origDelete;
}
// The failed reconcile is not checkpointed past: the run blocks and the
// stale duplicate is still visible. The failure is recorded as a
// `<rename:…>` SENTINEL, which the auto-skip valve can never
// chronic-skip — an outage lasting longer than the threshold must not
// quietly bank the duplicate.
expect(blocked.status).toBe('blocked_by_failures');
expect(blocked.failedFiles).toBe(1);
expect(await engine.getPage('people/carol')).not.toBeNull();
const { loadSyncFailures } = await import('../src/core/sync-failure-ledger.ts');
const openSentinels = loadSyncFailures().filter(
f => f.path === '<rename:people/dana.md>' && f.state === 'open',
);
expect(openSentinels).toHaveLength(1);
// Next run (failure gone) retries the same rename diff and converges.
const result = await performSync(engine, { repoPath: repo, ...SYNC_OPTS });
expect(result.status).toBe('synced');
expect(await engine.getPage('people/carol')).toBeNull();
const dana = await engine.getPage('people/dana');
expect(dana).not.toBeNull();
expect(dana!.compiled_truth).toContain('Carol is a person.');
expect(await countPages()).toBe(1);
// The convergence also clears the sentinel row — doctor must not keep
// warning about a rename that has since reconciled.
const remaining = loadSyncFailures().filter(
f => f.path === '<rename:people/dana.md>' && f.state === 'open',
);
expect(remaining).toHaveLength(0);
});
});