Compare commits

...
Author SHA1 Message Date
Garry Tan 297318df4f Merge remote-tracking branch 'origin/master' into garrytan/supersede-pr-1623
# Conflicts:
#	CHANGELOG.md
#	CLAUDE.md
#	VERSION
#	llms-full.txt
#	package.json
#	src/core/migrate.ts
2026-05-30 10:36:19 -07:00
Garry Tan d6104a645c Merge remote-tracking branch 'origin/master' into garrytan/supersede-pr-1623
# Conflicts:
#	CHANGELOG.md
#	TODOS.md
#	VERSION
#	package.json
2026-05-30 09:16:03 -07:00
Garry TanandClaude Opus 4.8 d855d906ce chore: bump version and changelog (v0.41.32.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-30 09:04:41 -07:00
Garry Tan d539a5b051 chore(ci): offload tests to on-demand cloud runners from a local CLI
scripts/ship-remote-tests.sh pushes the branch, dispatches the test workflow,
and blocks on `gh run watch --exit-status` — a local caller (human or agent)
awaits the GitHub run exactly like a local `bun run test`, with a real pass/fail
exit code. Frees a load-saturated local machine (many Conductor agents running
their own bun-test suites at once → load avg 120 on 16 cores → PGLite OOM/crawl).

test.yml gains workflow_dispatch so the suite can be triggered from any branch.
2026-05-30 09:04:41 -07:00
Garry Tanandt 46db018f51 fix(staleness): commit-relative sync staleness (HEAD-hash local, durable column remote)
Quiet, fully-caught-up repos no longer false-alarm as SEVERELY STALE in
gbrain doctor / sources status. Staleness now means "is there committed
content the sync hasn't ingested?" not raw wall-clock since the last sync.

- git-head.ts: requireCleanWorkingTree gains 'ignore-untracked' mode (git
  status --porcelain --untracked-files=no). Untracked dirs no longer defeat
  the freshness short-circuit — sync's incremental path keys off the commit
  diff and never imports untracked files, so doctor agrees with sync.
- source-health.ts: newestCommitMs (HEAD committer time) + pure
  lagFromContentMs comparator; computeAllSourceMetrics {probeContent} routes
  local→live commit-hash, remote→stored column. Dead isSourceStale removed.
- migration v108 sources.newest_content_at + fresh-schema blobs.
- sync.ts: writeSyncAnchor stamps newest_content_at atomically with
  last_commit/last_sync_at; buildSyncStatusReport (remote get_status_snapshot)
  reads the column — no git subprocess (v0.41.27.0 trust boundary intact).
- doctor.ts: checkSyncFreshness short-circuit ignores untracked; remote path
  reads the column; clock-skew check stays on raw wall-clock.

Local consumers probe live git (catch HEAD moving to an old-dated commit, which
a timestamp compare would miss); remote consumers read the durable column so a
remote-callable endpoint never shells out to a DB-supplied local_path.

Supersedes #1623 (re-implemented in base repo with the trust boundary preserved).

Co-Authored-By: t <t@t>
2026-05-30 09:04:41 -07:00
21 changed files with 857 additions and 79 deletions
+5
View File
@@ -5,6 +5,11 @@ on:
branches: [master]
pull_request:
branches: [master]
# Manual dispatch lets a local dev/agent offload the suite to GitHub's
# on-demand runners from ANY branch (see scripts/ship-remote-tests.sh).
# Frees a load-saturated local machine (e.g. many Conductor agents running
# their own bun-test suites at once — load avg 120 on 16 cores).
workflow_dispatch:
permissions:
contents: read
+122
View File
@@ -2,6 +2,128 @@
All notable changes to GBrain will be documented in this file.
## [0.41.32.0] - 2026-05-30
**A quiet repo that's fully caught up no longer screams `SEVERELY STALE` in
`gbrain doctor`. Staleness now means "is there committed content the sync hasn't
ingested?" — not "how long has the wall clock been ticking since the last sync
ran."**
If you keep a federated source that doesn't get a commit for days, the old check
kept escalating — 24h "stale", 72h "severely stale" — even though the sync had
everything the repo contained. It was pure wall-clock noise, and it trained you
to ignore the alert. Worse, a repo with a couple of stray untracked folders
(`companies/`, `media/`) tripped it even right after a sync, because the
freshness gate counted untracked files as "uncommitted work."
Now the gate asks the right question. A source is caught up when its current
commit is the one the sync recorded (`HEAD == last_commit`) and there are no
uncommitted edits to *tracked* files. Untracked folders are ignored — they're
not part of the repo, and `gbrain sync` never imports them anyway. If that holds,
lag is `0` and the source reports clean, no matter how long ago the sync ran.
### How to use it
Nothing to turn on. Run `gbrain doctor` (or `gbrain sources status`) and a quiet,
caught-up source now shows fresh instead of stale. After `gbrain upgrade`,
migration v109 adds one column and the next `gbrain sync` starts populating it.
### What you'd see
| Scenario | Old metric | New metric |
|---|---|---|
| Quiet repo, caught up (newest commit predates last sync) | grows forever → SEVERELY STALE | **0 → fresh** |
| Repo with new commits the sync hasn't pulled | wall-clock | wall-clock → stale (correct) |
| HEAD force-pushed to an older-dated commit | could read "caught up" | **stale** (compares the commit hash, not its date) |
| Non-git path / never synced | wall-clock | wall-clock (unchanged) |
| Future `last_sync_at` (clock skew) | warns | warns (unchanged) |
### Local vs remote, and the trust boundary we kept
The local `gbrain doctor` (running on the machine that has your checkouts) reads
the live commit hash, so it always catches new commits the sync hasn't pulled —
your authoritative signal. The remote surfaces (`gbrain remote doctor`,
`federation_health`, and the `get_status_snapshot` MCP op) read a stored
`sources.newest_content_at` column written at sync time instead of running `git`
against a database-supplied path — preserving the v0.41.27.0 rule that a
remote-callable endpoint never shells out to a path an OAuth client could
influence. A `NULL` column falls back to wall-clock, so nothing regresses before
your next sync.
### What we caught before merging
An early version compared content *timestamps* (`newest content <= last sync`).
That's wrong when HEAD moves to a commit with an *older* author date — a rebase
that preserves dates, a branch rewind, an imported old commit — it would call a
genuinely-behind source "caught up." Switching to the commit *hash* fixes it and
also drops a fragile `git status` mtime-parsing path. The remote-path git
subprocess was gated back behind the trust boundary, and two regression tests
pin both: the headline untracked-folders case and the "remote path never shells
out to git" guarantee.
### For contributors
New `scripts/ship-remote-tests.sh` offloads the test suite to GitHub's on-demand
runners and blocks on the result with a real exit code (`gh run watch
--exit-status`). When your local machine is saturated (many agents running their
own `bun test` at once), run the gate in the cloud instead of fighting for CPU.
`test.yml` now also accepts `workflow_dispatch` so it can be triggered from any
branch.
### To take advantage of v0.41.32.0
`gbrain upgrade` should do this automatically. If it didn't, or if `gbrain
doctor` warns about a partial migration:
1. **Apply the migration:**
```bash
gbrain apply-migrations --yes
```
This adds `sources.newest_content_at` (migration v109) — a metadata-only
column add, instant on any brain size.
2. **Run a sync so the column populates:**
```bash
gbrain sync # or `gbrain sync --all` on a federated brain
```
Until a source syncs once post-upgrade, its remote staleness falls back to
the old wall-clock measure (the local doctor is already accurate via live git).
3. **Verify:**
```bash
gbrain doctor --json | grep -A2 sync_freshness
gbrain sources status
```
A quiet, caught-up source should report `ok` / lag 0.
4. **If anything looks wrong,** file an issue with `gbrain doctor` output and the
contents of `~/.gbrain/upgrade-errors.jsonl` if it exists.
### Itemized changes
- **`src/core/git-head.ts`** — `isSourceUnchangedSinceSync`'s `requireCleanWorkingTree`
now accepts `'ignore-untracked'` (alongside `true`/`false`); the clean probe runs
`git status --porcelain --untracked-files=no` in that mode. This is the one-line
headline fix: untracked folders no longer defeat the freshness short-circuit.
- **`src/core/source-health.ts`** — new `newestCommitMs(localPath)` (HEAD committer
time via `git log -1 --format=%ct`, fail-open null) and pure `lagFromContentMs(contentMs,
lastSyncMs, nowMs)` comparator for the remote/column path; `computeAllSourceMetrics`
gains `{ probeContent }` (local opts into the live commit-hash probe, remote reads the
column). Dead `isSourceStale(src, intervalMs)` removed (only `autopilot-fanout.ts`'s
own variant was live).
- **`src/core/migrate.ts` v109** + `src/schema.sql` + `src/core/pglite-schema.ts` (+
regenerated `schema-embedded.ts`) — `sources.newest_content_at TIMESTAMPTZ`.
- **`src/commands/sync.ts`** — `writeSyncAnchor` stamps `newest_content_at` (HEAD committer
time) in the same atomic UPDATE as `last_commit`/`last_sync_at`; `buildSyncStatusReport`
(the remote `get_status_snapshot` op) reads the column via `lagFromContentMs` — no git
subprocess.
- **`src/commands/doctor.ts`** — `checkSyncFreshness` short-circuit uses `'ignore-untracked'`;
the remote (non-`localOnly`) path computes lag from the stored column; the `< 0` clock-skew
check stays on raw wall-clock.
- **`src/commands/sources.ts`** — `gbrain sources status` opts into the live probe
(`probeContent: true`).
- **Tests**`test/source-health.test.ts` (commit-hash caught-up incl. the old-dated-commit
regression, `lagFromContentMs` matrix, `newestCommitMs`, probeContent local/remote),
`test/doctor.test.ts` (T1 untracked-folders headline bug, T2 remote-never-shells-out trust
boundary), `test/sync-all-parallel.test.ts` (column-path staleness).
- **CI tooling**`scripts/ship-remote-tests.sh` + `workflow_dispatch` on `test.yml`.
## [0.41.31.0] - 2026-05-30
**Your nightly `gbrain sync --all` cron stops getting blocked. It used to
+4 -3
View File
File diff suppressed because one or more lines are too long
+33
View File
@@ -1,5 +1,38 @@
# TODOS
## v0.41.32.0 content-relative staleness follow-ups (v0.42+)
Filed from the v0.41.32.0 wave (supersedes #1623 — commit-relative sync
staleness). The wave fixes the LOCAL doctor/sources false-SEVERE and the
REMOTE surfaces via a durable `sources.newest_content_at` column. Two gaps
were deliberately scoped out (CM2 + the remote post-sync-divergence residual).
- [ ] **v0.42+: lightweight local content-probe phase to keep `newest_content_at` fresh between syncs.**
- **What:** an autopilot/cron phase that, for each git-backed source, runs the
cheap `git log -1 --format=%ct` (HEAD committer time) and refreshes
`sources.newest_content_at` even when there's nothing to sync.
- **Why:** the REMOTE staleness path (`doctorReportRemote`'s `checkSyncFreshness`,
`federation_health`, the `get_status_snapshot` MCP op) reads the column and
cannot shell out to git (v0.41.27.0 trust boundary). The column is written at
sync time, so a commit landed AFTER the last sync is invisible to the remote
path until the next sync rewrites it — a narrow false-negative window. The
authoritative LOCAL cron doctor catches those (it probes live git), so this is
a remote-only freshness improvement, not a correctness hole.
- **Pros:** shrinks the remote false-negative window to the probe cadence;
keeps the trust boundary intact (probe runs on the trusted host, not from a
remote caller).
- **Cons:** a new background phase + its own tests + a cadence knob; only
matters for operators who rely on `gbrain remote doctor` instead of the local
cron doctor.
- **Context:** the helper already exists — `newestCommitMs(localPath)` in
`src/core/source-health.ts`. The phase just calls it per source and UPDATEs
the column. See the v0.41.32.0 plan at
`~/.claude/plans/system-instruction-you-are-working-vivid-gizmo.md`.
- **Also note:** `checkCycleFreshness` was deliberately left on wall-clock in
v0.41.32.0 (CM2 — it compares `last_full_cycle_at` via `listAllSources`, a
different axis from sync staleness). Content-relativizing it (a source whose
newest commit predates its last full cycle doesn't need re-cycling) is a
natural companion to this probe phase. Priority: P3.
## brainstorm/lsd --save source-awareness (v0.42+)
Filed from the `--save` dual-sink hardening wave (route through the canonical
+1 -1
View File
@@ -1 +1 @@
0.41.31.0
0.41.32.0
+4 -3
View File
File diff suppressed because one or more lines are too long
+1 -1
View File
@@ -141,5 +141,5 @@
"bun": ">=1.3.10"
},
"license": "MIT",
"version": "0.41.31.0"
"version": "0.41.32.0"
}
+91
View File
@@ -0,0 +1,91 @@
#!/usr/bin/env bash
# ship-remote-tests.sh — run the unit suite on GitHub's on-demand cloud
# runners instead of locally, and block until it finishes with a real
# pass/fail exit code.
#
# WHY: a local machine running many Conductor agents at once gets CPU/memory
# saturated (observed: load avg 120 on 16 cores, ~15 sibling `bun test`
# processes). The PGLite WASM test suite then OOMs (8-shard) or crawls
# (~12min for 1/3 of files vs ~85s normally). The suite already runs on
# GitHub's ephemeral runners on every PR push; this script makes a local
# caller (human or agent, e.g. /ship Step 5) AWAIT that cloud run exactly
# like a local `bun run test` — push, dispatch, `gh run watch --exit-status`.
#
# USAGE:
# scripts/ship-remote-tests.sh [--workflow test.yml] [--branch <name>]
# [--no-push] [--ref <sha>]
#
# EXIT: mirrors the GitHub run — 0 on success, non-zero on failure (so it
# drops into a test gate unchanged). 2 = usage/precondition error.
#
# REQUIRES: `gh` authenticated; the workflow must declare `workflow_dispatch:`
# (test.yml does as of v0.41.32.0).
set -euo pipefail
WORKFLOW="test.yml"
BRANCH=""
DO_PUSH=1
REF=""
while [ $# -gt 0 ]; do
case "$1" in
--workflow) WORKFLOW="$2"; shift 2 ;;
--branch) BRANCH="$2"; shift 2 ;;
--ref) REF="$2"; shift 2 ;;
--no-push) DO_PUSH=0; shift ;;
-h|--help)
sed -n '2,30p' "$0"; exit 0 ;;
*) echo "ship-remote-tests: unknown arg '$1'" >&2; exit 2 ;;
esac
done
command -v gh >/dev/null 2>&1 || { echo "ship-remote-tests: gh CLI not found" >&2; exit 2; }
gh auth status >/dev/null 2>&1 || { echo "ship-remote-tests: gh not authenticated — run 'gh auth login'" >&2; exit 2; }
[ -n "$BRANCH" ] || BRANCH="$(git branch --show-current 2>/dev/null || true)"
[ -n "$BRANCH" ] || { echo "ship-remote-tests: could not determine branch (detached HEAD?) — pass --branch" >&2; exit 2; }
if [ "$DO_PUSH" = "1" ]; then
echo "ship-remote-tests: pushing $BRANCH ..." >&2
git push -u origin "$BRANCH"
fi
# Dispatch against the branch (or an explicit ref). Requires workflow_dispatch
# on the workflow. The HEAD sha lets us disambiguate OUR run from any
# concurrent pull_request run on the same branch.
HEAD_SHA="$(git rev-parse "${REF:-HEAD}")"
echo "ship-remote-tests: dispatching $WORKFLOW on $BRANCH @ ${HEAD_SHA:0:8} ..." >&2
gh workflow run "$WORKFLOW" --ref "${REF:-$BRANCH}" >/dev/null
# Poll for the dispatched run to register (cli/cli#8194: `gh run watch` can
# skip a not-yet-registered run, so we resolve the databaseId ourselves first).
RUN_ID=""
for _ in $(seq 1 30); do
RUN_ID="$(gh run list --workflow "$WORKFLOW" --branch "$BRANCH" \
--event workflow_dispatch --limit 10 \
--json databaseId,headSha,status \
-q "[.[] | select(.headSha==\"$HEAD_SHA\")] | sort_by(.databaseId) | last | .databaseId" 2>/dev/null || true)"
[ -n "$RUN_ID" ] && [ "$RUN_ID" != "null" ] && break
sleep 3
done
if [ -z "$RUN_ID" ] || [ "$RUN_ID" = "null" ]; then
echo "ship-remote-tests: could not find the dispatched run after 90s." >&2
echo " Check manually: gh run list --workflow $WORKFLOW --branch $BRANCH" >&2
exit 2
fi
RUN_URL="$(gh run view "$RUN_ID" --json url -q .url 2>/dev/null || echo "")"
echo "ship-remote-tests: watching run $RUN_ID $RUN_URL" >&2
# Block until the cloud run finishes; mirror its pass/fail as our exit code.
if gh run watch "$RUN_ID" --exit-status; then
echo "ship-remote-tests: PASS $RUN_URL" >&2
exit 0
else
rc=$?
echo "ship-remote-tests: FAIL (exit $rc) $RUN_URL" >&2
echo "--- failed logs ---" >&2
gh run view "$RUN_ID" --log-failed 2>/dev/null | tail -120 >&2 || true
exit "$rc"
fi
+40 -7
View File
@@ -28,6 +28,9 @@ import { dirname, isAbsolute, join, resolve as resolvePath } from 'path';
import { fileURLToPath } from 'url';
import { existsSync, readFileSync, readdirSync, statSync } from 'fs';
import { isSourceUnchangedSinceSync } from '../core/git-head.ts';
// v0.41.32.0: remote staleness reads the stored newest_content_at column via
// this pure comparator (no git subprocess on the HTTP MCP doctor path).
import { lagFromContentMs } from '../core/source-health.ts';
import { CHUNKER_VERSION } from '../core/chunkers/code.ts';
export interface Check {
@@ -2710,8 +2713,11 @@ export async function checkSyncFreshness(
last_sync_at: Date | null;
last_commit: string | null;
chunker_version: string | null;
newest_content_at: Date | null;
}>(
`SELECT id, name, local_path, last_sync_at, last_commit, chunker_version FROM sources WHERE local_path IS NOT NULL`,
// v0.41.32.0: newest_content_at feeds the REMOTE (non-localOnly) lag so
// doctorReportRemote never shells out to git on a DB-supplied local_path.
`SELECT id, name, local_path, last_sync_at, last_commit, chunker_version, newest_content_at FROM sources WHERE local_path IS NOT NULL`,
);
if (sources.length === 0) {
@@ -2791,8 +2797,14 @@ export async function checkSyncFreshness(
// v0.41.27.0: git short-circuit (D4 + D7 combined). Only fires when:
// 1. caller opted in via localOnly=true (trust boundary)
// 2. HEAD === last_commit (no new commits to sync)
// 3. working tree is clean (no uncommitted edits sync would re-walk)
// 4. chunker_version matches CURRENT (no post-upgrade re-chunk pending)
// 3. working tree has no TRACKED changes — untracked files ignored
// (v0.41.32.0: `'ignore-untracked'`. Sync's incremental path keys off
// the commit diff and never imports untracked files, so a quiet repo
// with stray untracked dirs is genuinely caught up. The pre-v0.41.30
// `true` mode counted those as dirty and produced the false-SEVERE
// alarm this wave fixes.)
// 4. chunker_version matches CURRENT (no post-upgrade re-chunk pending —
// still ANDed, so a re-chunk need is never masked)
// All four must hold; otherwise fall through to the time-based check.
// The chunker version match is computed here (not in the helper)
// because it depends on engine state, not git state.
@@ -2800,7 +2812,7 @@ export async function checkSyncFreshness(
const gitUnchanged = isSourceUnchangedSinceSync(
source.local_path,
source.last_commit,
{ requireCleanWorkingTree: true },
{ requireCleanWorkingTree: 'ignore-untracked' },
);
const chunkerMatch = source.chunker_version === currentChunkerVersion;
if (gitUnchanged && chunkerMatch) {
@@ -2809,14 +2821,35 @@ export async function checkSyncFreshness(
}
}
const ageHours = Math.floor(ageMs / (1000 * 60 * 60));
// v0.41.32.0: REMOTE path (doctorReportRemote, !localOnly) computes lag
// from the stored newest_content_at column — NO git subprocess on a
// DB-supplied local_path (preserves the v0.41.27.0 trust boundary). A
// quiet repo whose newest commit predates its last sync reports 0; NULL
// column → wall-clock fallback. LOCAL fall-through keeps wall-clock: the
// short-circuit already failed, so the source genuinely has work and
// "hours since last sync" is the right staleness measure. The `ageMs < 0`
// skew check above still runs on raw wall-clock for both paths (A1).
let thresholdAgeMs = ageMs;
if (!localOnly) {
const contentMs = source.newest_content_at
? new Date(source.newest_content_at).getTime()
: null;
const lagSec = lagFromContentMs(
contentMs !== null && Number.isFinite(contentMs) ? contentMs : null,
lastSync,
now,
);
thresholdAgeMs = lagSec === null ? ageMs : lagSec * 1000;
}
const ageHours = Math.floor(thresholdAgeMs / (1000 * 60 * 60));
const ageDays = Math.floor(ageHours / 24);
if (ageMs > failMs) {
if (thresholdAgeMs > failMs) {
issues.push(`Source ${display} last synced ${ageDays}d ago — brain search is stale!`);
hasFailures = true;
stale_count++;
} else if (ageMs > warnMs) {
} else if (thresholdAgeMs > warnMs) {
issues.push(`Source ${display} last synced ${ageHours}h ago`);
hasWarnings = true;
stale_count++;
+3 -1
View File
@@ -592,7 +592,9 @@ async function runStatus(engine: BrainEngine, args: string[]): Promise<void> {
}
return;
}
const metrics = await computeAllSourceMetrics(engine, sources);
// Local CLI on the trusted host: probe the live commit hash so a quiet,
// caught-up source reports lag 0 instead of growing wall-clock (v0.41.32.0).
const metrics = await computeAllSourceMetrics(engine, sources, { probeContent: true });
if (json) {
console.log(JSON.stringify({ schema_version: 1, sources: metrics }, null, 2));
+48 -11
View File
@@ -54,6 +54,11 @@ import {
} from '../core/console-prefix.ts';
import { loadStorageConfig } from '../core/storage-config.ts';
import { getDefaultSourcePath } from '../core/source-resolver.ts';
// v0.41.32.0: stamp the durable newest-COMMIT timestamp at sync time so the
// remote staleness path reads a column instead of shelling out to git.
// lagFromContentMs is the remote/column comparator (buildSyncStatusReport
// backs the get_status_snapshot MCP op — must NOT shell out to git).
import { newestCommitMs, lagFromContentMs } from '../core/source-health.ts';
import { sortNewestFirst } from '../core/sort-newest-first.ts';
export interface SyncResult {
@@ -460,15 +465,32 @@ async function writeSyncAnchor(
sourceId: string | undefined,
which: 'repo_path' | 'last_commit',
value: string,
// v0.41.32.0 (supersedes #1623): on `last_commit` advances, also stamp the
// durable newest-COMMIT timestamp (HEAD committer time, epoch ms) in the SAME
// atomic UPDATE as last_sync_at — no separate write to leave partial state,
// no clock-domain split (last_sync_at = DB now(); newest_content_at = the
// git-intrinsic committer time of the HEAD we just synced). `undefined` keeps
// the legacy 2-column write; `null` clears the column (git unavailable).
newestContentEpochMs?: number | null,
): Promise<void> {
if (sourceId) {
const col = which === 'repo_path' ? 'local_path' : 'last_commit';
// last_sync_at bookmarked on every last_commit advance.
if (which === 'last_commit') {
await engine.executeRaw(
`UPDATE sources SET last_commit = $1, last_sync_at = now() WHERE id = $2`,
[value, sourceId],
);
if (newestContentEpochMs !== undefined) {
const iso = newestContentEpochMs === null
? null
: new Date(newestContentEpochMs).toISOString();
await engine.executeRaw(
`UPDATE sources SET last_commit = $1, last_sync_at = now(), newest_content_at = $3 WHERE id = $2`,
[value, sourceId, iso],
);
} else {
await engine.executeRaw(
`UPDATE sources SET last_commit = $1, last_sync_at = now() WHERE id = $2`,
[value, sourceId],
);
}
} else {
await engine.executeRaw(
`UPDATE sources SET ${col} = $1 WHERE id = $2`,
@@ -477,6 +499,9 @@ async function writeSyncAnchor(
}
return;
}
// Legacy no-sourceId path (pre-v0.18 global config). Modern sync always
// resolves a sourceId (incl. 'default'), so newest_content_at is written via
// the sourceId branch above; the default source is not stuck on NULL.
await engine.setConfig(`sync.${which}`, value);
}
@@ -1253,7 +1278,7 @@ async function performSyncInner(engine: BrainEngine, opts: SyncOpts): Promise<Sy
if (totalChanges === 0) {
// Update sync state even with no syncable changes (git advanced)
await writeSyncAnchor(engine, opts.sourceId, 'last_commit', headCommit);
await writeSyncAnchor(engine, opts.sourceId, 'last_commit', headCommit, newestCommitMs(repoPath));
await engine.setConfig('sync.last_run', new Date().toISOString());
await writeChunkerVersion(engine, opts.sourceId, String(CHUNKER_VERSION));
return {
@@ -1755,7 +1780,7 @@ async function performSyncInner(engine: BrainEngine, opts: SyncOpts): Promise<Sy
// Update sync state AFTER all changes succeed (source-scoped when
// opts.sourceId is set, global config otherwise).
await writeSyncAnchor(engine, opts.sourceId, 'last_commit', headCommit);
await writeSyncAnchor(engine, opts.sourceId, 'last_commit', headCommit, newestCommitMs(repoPath));
await engine.setConfig('sync.last_run', new Date().toISOString());
await writeSyncAnchor(engine, opts.sourceId, 'repo_path', repoPath);
// v0.20.0 Cathedral II Layer 12: persist the chunker version we just
@@ -1976,7 +2001,7 @@ async function performFullSync(
// Persist sync state so next sync is incremental (C1 fix: was missing).
// v0.18.0 Step 5: routed through writeSyncAnchor so --source pins it
// to the right sources row rather than the global config.
await writeSyncAnchor(engine, opts.sourceId, 'last_commit', headCommit);
await writeSyncAnchor(engine, opts.sourceId, 'last_commit', headCommit, newestCommitMs(repoPath));
await engine.setConfig('sync.last_run', new Date().toISOString());
await writeSyncAnchor(engine, opts.sourceId, 'repo_path', repoPath);
// v0.20.0 Cathedral II Layer 12: persist chunker version for the gate.
@@ -2977,6 +3002,8 @@ export async function buildSyncStatusReport(
id: string;
last_commit: string | null;
last_sync_at: string | Date | null;
// v0.41.32.0: remote staleness reads this column (no git subprocess).
newest_content_at: string | Date | null;
};
type CountRow = {
source_id: string;
@@ -2990,7 +3017,7 @@ export async function buildSyncStatusReport(
const sourceRows = sourceIds.length === 0
? []
: await engine.executeRaw<SourceRow>(
`SELECT id, last_commit, last_sync_at FROM sources WHERE id = ANY($1::text[])`,
`SELECT id, last_commit, last_sync_at, newest_content_at FROM sources WHERE id = ANY($1::text[])`,
[sourceIds],
);
const sourceMap = new Map<string, SourceRow>();
@@ -3085,14 +3112,24 @@ export async function buildSyncStatusReport(
const now = Date.now();
const out: SyncStatusReportSource[] = sources.map((src) => {
const cfgEntry = (src.config || {}) as { syncEnabled?: boolean };
const row = sourceMap.get(src.id) || { id: src.id, last_commit: null, last_sync_at: null };
const row = sourceMap.get(src.id) || { id: src.id, last_commit: null, last_sync_at: null, newest_content_at: null };
const counts = countMap.get(src.id) || { pages: 0, chunks_total: 0, chunks_unembedded: 0 };
const lastSyncMs = row.last_sync_at
? (row.last_sync_at instanceof Date ? row.last_sync_at.getTime() : Date.parse(row.last_sync_at))
: null;
const stalenessHours = lastSyncMs !== null && Number.isFinite(lastSyncMs)
? (now - lastSyncMs) / 3_600_000
// v0.41.32.0: commit-relative staleness from the stored column — NO git
// subprocess (this function backs the remote get_status_snapshot MCP op,
// so it must honor the v0.41.27.0 trust boundary). A quiet repo whose
// newest commit predates its last sync reports 0; null column → wall-clock.
const contentMs = row.newest_content_at
? (row.newest_content_at instanceof Date ? row.newest_content_at.getTime() : Date.parse(row.newest_content_at))
: null;
const lagSeconds = lagFromContentMs(
Number.isFinite(contentMs as number) ? (contentMs as number) : null,
lastSyncMs !== null && Number.isFinite(lastSyncMs) ? lastSyncMs : null,
now,
);
const stalenessHours = lagSeconds === null ? null : lagSeconds / 3600;
let stalenessClass: 'fresh' | 'stale' | 'severe' | 'unknown' = 'unknown';
if (stalenessHours !== null) {
if (stalenessHours < 24) stalenessClass = 'fresh';
+26 -12
View File
@@ -27,7 +27,11 @@ import { execFileSync } from 'node:child_process';
export type GitHeadProbe = (localPath: string) => string | null;
// `null` distinguishes probe error from known-dirty (false). Doctor treats
// both as "do not short-circuit", but tests need to assert which path fired.
export type GitCleanProbe = (localPath: string) => boolean | null;
// `ignoreUntracked` (v0.41.32.0): when true, untracked files (`git status`
// `??` rows) do NOT count as dirty — they are not part of the repo and sync's
// incremental path (commit-diff at sync.ts:1057) never imports them, so a
// quiet repo with stray untracked dirs is still "unchanged".
export type GitCleanProbe = (localPath: string, ignoreUntracked?: boolean) => boolean | null;
const DEFAULT_HEAD_PROBE: GitHeadProbe = (localPath) => {
try {
@@ -42,9 +46,16 @@ const DEFAULT_HEAD_PROBE: GitHeadProbe = (localPath) => {
}
};
const DEFAULT_CLEAN_PROBE: GitCleanProbe = (localPath) => {
const DEFAULT_CLEAN_PROBE: GitCleanProbe = (localPath, ignoreUntracked) => {
try {
const out = execFileSync('git', ['-C', localPath, 'status', '--porcelain'], {
// `--untracked-files=no` makes `git status --porcelain` emit ONLY tracked
// changes. Empty output then means "clean ignoring untracked." This is the
// v0.41.32.0 fix for the false-SEVERE bug: untracked dirs (`?? companies/`,
// `?? media/`) on an otherwise-caught-up repo previously made the tree look
// dirty and defeated the short-circuit.
const args = ['-C', localPath, 'status', '--porcelain'];
if (ignoreUntracked) args.push('--untracked-files=no');
const out = execFileSync('git', args, {
encoding: 'utf8',
timeout: 5000,
stdio: ['ignore', 'pipe', 'ignore'],
@@ -71,15 +82,17 @@ export function _setGitCleanProbeForTests(fn: GitCleanProbe | null): void {
export interface GitFreshnessOpts {
/**
* When true, additionally require working tree to be clean (no
* uncommitted changes). Doctor uses this to mirror `gbrain sync`'s
* working-tree-dirty gate at sync.ts:1075 — otherwise doctor would
* say "unchanged" for a repo with pending local edits that sync would
* actually re-walk on the next run.
*
* Default false (HEAD comparison only). Doctor-callers set true.
* Working-tree cleanliness requirement on top of the HEAD==lastCommit check:
* - `false`/omitted: HEAD comparison only.
* - `true`: require a fully clean tree (tracked AND untracked) — the
* v0.41.27.0 posture mirroring `gbrain sync`'s gate at sync.ts:1075.
* - `'ignore-untracked'` (v0.41.32.0): require no TRACKED changes but allow
* untracked files. This is what doctor/sources should use: sync's
* incremental path keys off the commit diff and never imports untracked
* files, so a quiet repo with stray untracked dirs is genuinely caught up.
* Fixes the false-SEVERE bug without weakening the commit-hash gate.
*/
requireCleanWorkingTree?: boolean;
requireCleanWorkingTree?: boolean | 'ignore-untracked';
}
/**
@@ -102,7 +115,8 @@ export function isSourceUnchangedSinceSync(
const head = _headProbe(localPath);
if (head === null || head !== lastCommit) return false;
if (opts?.requireCleanWorkingTree) {
const isClean = _cleanProbe(localPath);
const ignoreUntracked = opts.requireCleanWorkingTree === 'ignore-untracked';
const isClean = _cleanProbe(localPath, ignoreUntracked);
// null (probe error) AND false (known dirty) both fail the gate.
if (isClean !== true) return false;
}
+16
View File
@@ -4946,6 +4946,22 @@ export const MIGRATIONS: Migration[] = [
ALTER TABLE pages ADD COLUMN IF NOT EXISTS embedding_signature TEXT NULL;
`,
},
{
version: 109,
name: 'sources_newest_content_at',
// v0.41.32.0 (supersedes #1623): durable newest-COMMIT timestamp per source,
// written at sync time (HEAD committer time). The REMOTE staleness path
// (federation_health, get_status_snapshot MCP op) reads this column instead
// of shelling out to git on a DB-supplied local_path — preserving the
// v0.41.27.0 trust boundary while still killing the quiet-repo false-SEVERE
// alarm. ADD COLUMN with a NULL default is metadata-only on both engines
// (instant, no table rewrite). Mirror lives in pglite-schema.ts +
// schema.sql (fresh-install path) and the applyForwardReferenceBootstrap
// probe set in both engines. Renumbered 108→109 on the master merge that
// landed v0.41.31's pages_embedding_signature at v108.
idempotent: true,
sql: `ALTER TABLE sources ADD COLUMN IF NOT EXISTS newest_content_at TIMESTAMPTZ`,
},
];
export const LATEST_VERSION = MIGRATIONS.length > 0
+3
View File
@@ -49,6 +49,9 @@ CREATE TABLE IF NOT EXISTS sources (
-- FALSE for mounts by default; host is always trusted regardless.
contextual_retrieval_mode TEXT,
trust_frontmatter_overrides BOOLEAN NOT NULL DEFAULT false,
-- v0.41.32.0 (supersedes #1623): newest COMMIT timestamp at last sync
-- (mirrors src/schema.sql). REMOTE staleness reads this; NULL wall-clock.
newest_content_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
+5
View File
@@ -53,6 +53,11 @@ CREATE TABLE IF NOT EXISTS sources (
-- (id='default') is always trusted regardless of this column.
contextual_retrieval_mode TEXT,
trust_frontmatter_overrides BOOLEAN NOT NULL DEFAULT false,
-- v0.41.32.0 (supersedes #1623): newest COMMIT timestamp (HEAD committer
-- time) recorded at last sync. The REMOTE staleness path reads this instead
-- of shelling out to git on a DB-supplied local_path, preserving the
-- v0.41.27.0 trust boundary. NULL reader falls back to wall-clock.
newest_content_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
+89 -13
View File
@@ -12,11 +12,19 @@
* D9: resolvePriority(config) accepts 'high'|'normal'|'low', falls back
* to 0 with once-per-source-per-process stderr warn on unknown values.
*
* D17: isSourceStale helper autopilot calls this to decide per-source
* sync dispatch independent of the brain_score gate.
* v0.41.32.0: commit-relative staleness. `lag_seconds` is no longer raw
* wall-clock `now - last_sync_at` (which false-flagged quiet, caught-up
* repos as SEVERE). Local callers pass `probeContent: true` and lag
* becomes 0 when the source is caught up by COMMIT HASH (HEAD ==
* last_commit, untracked ignored, via `isSourceUnchangedSinceSync`).
* Remote callers (federation_health on the HTTP MCP path) read the stored
* `newest_content_at` column instead NO git subprocess on a DB-supplied
* local_path (preserves the v0.41.27.0 trust boundary).
*/
import { execFileSync } from 'child_process';
import type { BrainEngine } from './engine.ts';
import { parseSourceConfig, type SourceRow } from './sources-load.ts';
import { isSourceUnchangedSinceSync } from './git-head.ts';
export interface SourceMetrics {
source_id: string;
@@ -97,15 +105,60 @@ export function resolvePriority(sourceId: string, config: unknown): number {
}
/**
* True iff the source's last_sync_at is older than `intervalMs`, OR it has
* never synced. Sources without a local_path are NOT considered stale (no
* way to sync them). Used by autopilot D17 freshness gate.
* Newest COMMIT timestamp for a source's checkout, in epoch ms, or `null` when
* not determinable cheaply (non-git path, git unavailable, timeout). This is
* the HEAD committer time (`git log -1 --format=%ct`) NOT working-tree mtimes
* (untracked/tracked-uncommitted files are not "committed content," and parsing
* `git status --porcelain` for mtimes is fragile). Used at sync time to populate
* the durable `sources.newest_content_at` column that the REMOTE staleness path
* reads (so the HTTP MCP doctor never shells out to git).
*
* Fail-open: every error path returns `null`; the caller stores NULL and the
* remote reader falls back to wall-clock. Shell-injection-safe (execFileSync
* array args), matching the git-head.ts posture.
*/
export function isSourceStale(src: SourceRow, intervalMs: number): boolean {
if (!src.local_path) return false;
if (!src.last_sync_at) return true;
const lastMs = new Date(src.last_sync_at).getTime();
return Date.now() - lastMs >= intervalMs;
export function newestCommitMs(localPath: string | null): number | null {
if (!localPath) return null;
try {
const out = execFileSync('git', ['-C', localPath, 'log', '-1', '--format=%ct'], {
encoding: 'utf8',
timeout: 10_000,
stdio: ['ignore', 'pipe', 'ignore'],
}).trim();
if (!out) return null;
const ms = Number(out) * 1000;
return Number.isFinite(ms) ? ms : null;
} catch {
return null; // not a git repo / git unavailable / timeout
}
}
/**
* Commit-relative lag in seconds from a STORED content timestamp (the
* `newest_content_at` column), for REMOTE consumers that cannot shell out:
* - `null` when `lastSyncMs` is unknown.
* - Negative wall-clock (future `last_sync_at`) is surfaced as-is so upstream
* clock-skew detection still fires.
* - `0` when the stored content is at or before the last sync (caught up).
* - Wall-clock `now - lastSync` when content is newer, or when `contentMs` is
* null (no column value / pre-migration) detection never regresses.
*
* Pure. The LOCAL path does NOT use this it keys off the live commit hash via
* `isSourceUnchangedSinceSync` (robust against HEAD moving to an old-dated
* commit, which a timestamp comparison would miss).
*/
export function lagFromContentMs(
contentMs: number | null,
lastSyncMs: number | null,
nowMs: number,
): number | null {
if (lastSyncMs === null || !Number.isFinite(lastSyncMs)) return null;
const wallClockSeconds = Math.floor((nowMs - lastSyncMs) / 1000);
if (wallClockSeconds < 0) return wallClockSeconds; // clock skew passthrough
if (contentMs !== null && Number.isFinite(contentMs)) {
return contentMs <= lastSyncMs ? 0 : wallClockSeconds;
}
return wallClockSeconds; // no stored content signal — wall-clock fallback
}
/**
@@ -123,6 +176,7 @@ export function isSourceStale(src: SourceRow, intervalMs: number): boolean {
export async function computeAllSourceMetrics(
engine: BrainEngine,
sources: SourceRow[],
opts?: { probeContent?: boolean },
): Promise<SourceMetrics[]> {
if (sources.length === 0) return [];
@@ -130,6 +184,10 @@ export async function computeAllSourceMetrics(
const chunkCounts = await chunkCountsBySource(engine);
const jobCounts = await jobCountsBySource(engine);
const now = Date.now();
// v0.41.32.0: LOCAL callers (gbrain sources status/audit) opt into a live
// commit-hash probe; the REMOTE federation_health path leaves it off and
// reads the stored column (no subprocess on a DB-supplied local_path).
const probeContent = opts?.probeContent === true;
return sources.map((src) => {
const cfg = parseSourceConfig(src.config);
@@ -142,9 +200,27 @@ export async function computeAllSourceMetrics(
: Math.round((chunkStats.embedded / chunkStats.total) * 1000) / 10;
const lastMs = src.last_sync_at ? new Date(src.last_sync_at).getTime() : null;
const lagSeconds = lastMs === null
? null
: Math.max(0, Math.floor((now - lastMs) / 1000));
// v0.41.32.0: commit-relative lag.
// LOCAL (probeContent): caught up iff HEAD == last_commit AND no tracked
// working-tree changes (untracked ignored) → lag 0; else wall-clock.
// Uses the live commit hash so a HEAD that moved to an old-dated commit
// is correctly NOT caught up. NULL last_commit → not caught up → wall-clock.
// REMOTE (default): read the stored newest_content_at column via
// lagFromContentMs — no git subprocess (v0.41.27.0 trust boundary).
let lagSeconds: number | null;
if (lastMs === null) {
lagSeconds = null;
} else if (probeContent) {
const caughtUp = isSourceUnchangedSinceSync(src.local_path, src.last_commit, {
requireCleanWorkingTree: 'ignore-untracked',
});
lagSeconds = caughtUp ? 0 : Math.max(0, Math.floor((now - lastMs) / 1000));
} else {
const contentMs = src.newest_content_at
? new Date(src.newest_content_at).getTime()
: null;
lagSeconds = lagFromContentMs(contentMs, lastMs, now);
}
return {
source_id: src.id,
+13 -4
View File
@@ -28,6 +28,14 @@ export interface SourceRow {
config: Record<string, unknown> | string;
created_at: Date;
archived?: boolean;
/**
* v0.41.32.0: newest COMMIT timestamp observed at last sync (HEAD committer
* time). The REMOTE staleness path reads this column so it never shells out
* to git on a DB-supplied local_path. Optional because the forward-reference
* fallback SELECT below omits it on pre-v109 brains; null/undefined the
* reader falls back to wall-clock.
*/
newest_content_at?: Date | null;
}
export interface LoadAllSourcesOpts {
@@ -67,13 +75,14 @@ export async function loadAllSources(
let rows: SourceRow[];
try {
rows = await engine.executeRaw<SourceRow>(
`SELECT id, name, local_path, last_commit, last_sync_at, config, created_at, archived
`SELECT id, name, local_path, last_commit, last_sync_at, config, created_at, archived, newest_content_at
FROM sources
ORDER BY (id = 'default') DESC, id`,
);
} catch (err) {
// Forward-reference safety: pre-v0.26.5 brains have no `archived` column.
// Re-issue without it; archived defaults to false.
// Forward-reference safety: pre-v0.26.5 brains lack `archived`; pre-v109
// brains lack `newest_content_at`. Re-issue with the historical minimal
// set; archived defaults false, newest_content_at undefined → wall-clock.
if (isUndefinedColumnError(err)) {
rows = await engine.executeRaw<SourceRow>(
`SELECT id, name, local_path, last_commit, last_sync_at, config, created_at
@@ -102,7 +111,7 @@ export async function fetchSource(
): Promise<SourceRow | null> {
try {
const rows = await engine.executeRaw<SourceRow>(
`SELECT id, name, local_path, last_commit, last_sync_at, config, created_at, archived
`SELECT id, name, local_path, last_commit, last_sync_at, config, created_at, archived, newest_content_at
FROM sources WHERE id = $1`,
[id],
);
+5
View File
@@ -49,6 +49,11 @@ CREATE TABLE IF NOT EXISTS sources (
-- (id='default') is always trusted regardless of this column.
contextual_retrieval_mode TEXT,
trust_frontmatter_overrides BOOLEAN NOT NULL DEFAULT false,
-- v0.41.32.0 (supersedes #1623): newest COMMIT timestamp (HEAD committer
-- time) recorded at last sync. The REMOTE staleness path reads this instead
-- of shelling out to git on a DB-supplied local_path, preserving the
-- v0.41.27.0 trust boundary. NULL → reader falls back to wall-clock.
newest_content_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
+115
View File
@@ -842,6 +842,121 @@ describe('v0.41.27.0 — sync_freshness git short-circuit', () => {
});
});
// ============================================================================
// v0.41.32.0 — commit-relative staleness (supersedes #1623)
// ============================================================================
// Two contracts:
// T1 (headline bug): a quiet repo whose only "dirt" is untracked files
// (`?? companies/`, `?? media/`) is now caught up on the LOCAL path —
// the short-circuit's clean check ignores untracked. Pre-v0.41.30 the
// strict clean check counted those as dirty → fell through to wall-clock
// → false SEVERE.
// T2 (trust boundary): the REMOTE path (no localOnly) computes lag from the
// stored newest_content_at column and NEVER shells out to git on a
// DB-supplied local_path (preserves the v0.41.27.0 boundary).
// ============================================================================
describe('v0.41.32.0 — commit-relative staleness', () => {
function makeStubEngine(rows: any[]): any {
return { executeRaw: async () => rows };
}
function agoMs(ms: number): Date { return new Date(Date.now() - ms); }
let currentChunkerVersion: string;
beforeEach(async () => {
const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } =
await import('../src/core/git-head.ts');
const { CHUNKER_VERSION } = await import('../src/core/chunkers/code.ts');
currentChunkerVersion = String(CHUNKER_VERSION);
_setGitHeadProbeForTests(null);
_setGitCleanProbeForTests(null);
});
afterAll(async () => {
const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } =
await import('../src/core/git-head.ts');
_setGitHeadProbeForTests(null);
_setGitCleanProbeForTests(null);
});
test('T1: stale + HEAD match + DIRTY-by-untracked-only + localOnly → ok (untracked ignored)', async () => {
const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } =
await import('../src/core/git-head.ts');
_setGitHeadProbeForTests(() => 'abc123');
// Clean ONLY when untracked is ignored (the bug scenario: `?? companies/`).
let sawIgnoreUntracked = false;
_setGitCleanProbeForTests((_path, ignoreUntracked) => {
if (ignoreUntracked) { sawIgnoreUntracked = true; return true; }
return false; // strict mode would call it dirty
});
const result = await checkSyncFreshness(makeStubEngine([
{ id: 'media-corpus', name: '', local_path: '/tmp/media',
last_sync_at: agoMs(86 * 60 * 60 * 1000), // 86h — would be SEVERE on wall-clock
last_commit: 'abc123', chunker_version: currentChunkerVersion,
newest_content_at: null },
]), { localOnly: true });
expect(sawIgnoreUntracked).toBe(true); // the short-circuit asked to ignore untracked
expect(result.status).toBe('ok');
expect(result.details).toEqual({ unchanged_count: 1, synced_recently_count: 0, stale_count: 0 });
});
test('T2: REMOTE (no localOnly) reads column, quiet repo → ok, NO git subprocess', async () => {
const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } =
await import('../src/core/git-head.ts');
let headCalls = 0, cleanCalls = 0;
_setGitHeadProbeForTests(() => { headCalls++; return 'x'; });
_setGitCleanProbeForTests(() => { cleanCalls++; return true; });
const result = await checkSyncFreshness(makeStubEngine([
{ id: 'remote', name: '', local_path: '/tmp/remote',
last_sync_at: agoMs(100 * 60 * 60 * 1000),
last_commit: 'x', chunker_version: currentChunkerVersion,
// Content committed BEFORE the last sync → caught up.
newest_content_at: agoMs(200 * 60 * 60 * 1000) },
])); // NOTE: no { localOnly: true } → remote path
expect(headCalls).toBe(0); // trust boundary: no git probe on remote path
expect(cleanCalls).toBe(0);
expect(result.status).toBe('ok');
expect(result.details).toEqual({ unchanged_count: 0, synced_recently_count: 1, stale_count: 0 });
});
test('T2b: REMOTE + NULL column → wall-clock fallback → stale (no git subprocess)', async () => {
const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } =
await import('../src/core/git-head.ts');
let headCalls = 0;
_setGitHeadProbeForTests(() => { headCalls++; return 'x'; });
_setGitCleanProbeForTests(() => true);
const result = await checkSyncFreshness(makeStubEngine([
{ id: 'remote', name: '', local_path: '/tmp/remote',
last_sync_at: agoMs(100 * 60 * 60 * 1000),
last_commit: 'x', chunker_version: currentChunkerVersion,
newest_content_at: null },
]));
expect(headCalls).toBe(0); // still no git probe even on the fallback path
expect(result.status).toBe('fail'); // 100h wall-clock > 72h
expect(result.details?.stale_count).toBe(1);
});
test('T2c: REMOTE + content NEWER than last sync → wall-clock (genuinely behind)', async () => {
const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
const result = await checkSyncFreshness(makeStubEngine([
{ id: 'remote', name: '', local_path: '/tmp/remote',
last_sync_at: agoMs(100 * 60 * 60 * 1000),
last_commit: 'x', chunker_version: currentChunkerVersion,
// committed 10h ago, synced 100h ago → behind.
newest_content_at: agoMs(10 * 60 * 60 * 1000) },
]));
expect(result.status).toBe('fail');
expect(result.details?.stale_count).toBe(1);
});
});
// Supervisor crash classifier wiring. Pre-fix, doctor.ts:1013 counted every
// `worker_exited` event as a crash regardless of `likely_cause`, inflating
// `crashes_24h` to 120+/day from RSS-watchdog drains and SIGTERM stops.
+190 -22
View File
@@ -1,22 +1,59 @@
/**
* Tests for src/core/source-health.ts (v0.40 D12 + D9 + D17).
* Tests for src/core/source-health.ts (v0.40 D12 + D9 + v0.41.32.0).
*
* Validates:
* - computeAllSourceMetrics: batched GROUP BY shape, vacuous truth for zero pages
* - computeAllSourceMetrics: batched GROUP BY shape, vacuous truth for zero
* pages, and v0.41.32.0 commit-relative lag (probeContent local path +
* stored-column remote path).
* - resolvePriorityLabel: high/normal/low, unknown normal + warn-once
* - isSourceStale: never-synced + lag-exceeded + fresh + missing local_path
* - newestCommitMs: HEAD committer time; null for non-git/missing.
* - lagFromContentMs: null/skew/null-contentwall-clock/caught-up0/behind.
* - isSourceUnchangedSinceSync ignore-untracked: the commit-hash caught-up
* contract the local path relies on (incl. the codex old-dated-commit case).
*/
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
import { mkdtempSync, rmSync, writeFileSync } from 'fs';
import { tmpdir } from 'os';
import { join } from 'path';
import { execFileSync } from 'child_process';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import {
computeAllSourceMetrics,
resolvePriorityLabel,
resolvePriority,
isSourceStale,
newestCommitMs,
lagFromContentMs,
_resetPriorityWarningsForTest,
} from '../src/core/source-health.ts';
import { isSourceUnchangedSinceSync } from '../src/core/git-head.ts';
import { loadAllSources } from '../src/core/sources-load.ts';
const HOUR = 3600_000;
/**
* Create a throwaway git repo with one commit dated `commitDate`. Returns the
* dir + its HEAD sha so tests can seed `sources.last_commit` to match.
*/
function makeGitRepo(commitDate: Date, registry: string[]): { dir: string; head: string } {
const dir = mkdtempSync(join(tmpdir(), 'gbrain-srchealth-'));
registry.push(dir);
const iso = commitDate.toISOString();
const env = {
...process.env,
GIT_AUTHOR_DATE: iso, GIT_COMMITTER_DATE: iso,
GIT_AUTHOR_NAME: 't', GIT_AUTHOR_EMAIL: 't@t',
GIT_COMMITTER_NAME: 't', GIT_COMMITTER_EMAIL: 't@t',
};
const run = (args: string[]) =>
execFileSync('git', ['-C', dir, ...args], { stdio: ['ignore', 'pipe', 'ignore'], env });
run(['init', '-q']);
writeFileSync(join(dir, 'a.md'), '# a\n');
run(['add', '-A']);
run(['commit', '-q', '-m', 'seed']);
const head = execFileSync('git', ['-C', dir, 'rev-parse', 'HEAD'], { encoding: 'utf8' }).trim();
return { dir, head };
}
let engine: PGLiteEngine;
beforeAll(async () => {
@@ -49,7 +86,6 @@ describe('resolvePriorityLabel', () => {
expect(resolvePriorityLabel('s', null)).toBe('normal');
});
test('unknown values → normal with warn', () => {
// Reroute stderr to capture
const orig = process.stderr.write.bind(process.stderr);
let captured = '';
process.stderr.write = ((chunk: string | Uint8Array) => {
@@ -75,8 +111,8 @@ describe('resolvePriorityLabel', () => {
}) as never;
try {
resolvePriorityLabel('s1', { priority: 'urgent' });
resolvePriorityLabel('s1', { priority: 'urgent' }); // same source
resolvePriorityLabel('s1', { priority: 42 }); // different bad value, same source
resolvePriorityLabel('s1', { priority: 'urgent' });
resolvePriorityLabel('s1', { priority: 42 });
expect(count).toBe(1);
} finally {
process.stderr.write = orig;
@@ -93,22 +129,95 @@ describe('resolvePriority (numeric)', () => {
});
});
describe('isSourceStale', () => {
test('never-synced (last_sync_at null) → true', () => {
const src = { id: 's', name: 's', local_path: '/path', last_commit: null, last_sync_at: null, config: {}, created_at: new Date() };
expect(isSourceStale(src, 60_000)).toBe(true);
// ── v0.41.32.0 commit-relative staleness ──────────────────────────────
describe('newestCommitMs', () => {
const repos: string[] = [];
afterAll(() => {
for (const d of repos) { try { rmSync(d, { recursive: true, force: true }); } catch { /* best-effort */ } }
});
test('no local_path → false (nothing to sync)', () => {
const src = { id: 's', name: 's', local_path: null, last_commit: null, last_sync_at: null, config: {}, created_at: new Date() };
expect(isSourceStale(src, 60_000)).toBe(false);
test('null for null / non-git / missing paths', () => {
expect(newestCommitMs(null)).toBeNull();
expect(newestCommitMs('/tmp/gbrain-does-not-exist-' + Date.now())).toBeNull();
});
test('synced within interval → false', () => {
const src = { id: 's', name: 's', local_path: '/path', last_commit: null, last_sync_at: new Date(Date.now() - 1000), config: {}, created_at: new Date() };
expect(isSourceStale(src, 60_000)).toBe(false);
test('returns the HEAD committer time in ms', () => {
const when = new Date(Date.now() - 50 * HOUR);
const { dir } = makeGitRepo(when, repos);
const ms = newestCommitMs(dir);
expect(ms).not.toBeNull();
expect(Math.abs((ms as number) - when.getTime())).toBeLessThan(2000);
});
test('synced beyond interval → true', () => {
const src = { id: 's', name: 's', local_path: '/path', last_commit: null, last_sync_at: new Date(Date.now() - 120_000), config: {}, created_at: new Date() };
expect(isSourceStale(src, 60_000)).toBe(true);
});
describe('lagFromContentMs (pure remote/column comparator)', () => {
const now = 1_000_000_000_000;
test('null last sync → null', () => {
expect(lagFromContentMs(now - HOUR, null, now)).toBeNull();
});
test('future last sync (skew) → negative passthrough', () => {
expect(lagFromContentMs(now, now + 10_000, now)).toBe(-10);
});
test('null content → wall-clock fallback', () => {
expect(lagFromContentMs(null, now - 100 * HOUR, now)).toBe(360_000); // 100h in s
});
test('content at/before last sync → caught up (0)', () => {
expect(lagFromContentMs(now - 200 * HOUR, now - 100 * HOUR, now)).toBe(0);
});
test('content after last sync → wall-clock since sync', () => {
expect(lagFromContentMs(now - 10 * HOUR, now - 100 * HOUR, now)).toBe(360_000);
});
});
describe('isSourceUnchangedSinceSync (ignore-untracked) — local caught-up contract', () => {
const repos: string[] = [];
afterAll(() => {
for (const d of repos) { try { rmSync(d, { recursive: true, force: true }); } catch { /* best-effort */ } }
});
test('HEAD == last_commit, clean → caught up', () => {
const { dir, head } = makeGitRepo(new Date(Date.now() - 200 * HOUR), repos);
expect(isSourceUnchangedSinceSync(dir, head, { requireCleanWorkingTree: 'ignore-untracked' })).toBe(true);
});
test('HEAD == last_commit WITH untracked dirs → still caught up (the headline bug)', () => {
const { dir, head } = makeGitRepo(new Date(Date.now() - 200 * HOUR), repos);
// Stray untracked dirs (the `?? companies/`, `?? media/` shape).
writeFileSync(join(dir, 'companies'), 'x'); // file is fine; untracked either way
execFileSync('git', ['-C', dir, 'status', '--porcelain'], { encoding: 'utf8' }); // sanity: dirty by default
expect(isSourceUnchangedSinceSync(dir, head, { requireCleanWorkingTree: 'ignore-untracked' })).toBe(true);
// Strict mode (pre-v0.41.30 behavior) would have called it dirty:
expect(isSourceUnchangedSinceSync(dir, head, { requireCleanWorkingTree: true })).toBe(false);
});
test('tracked uncommitted edit → NOT caught up (sync would re-walk the commit)', () => {
const { dir, head } = makeGitRepo(new Date(Date.now() - 200 * HOUR), repos);
writeFileSync(join(dir, 'a.md'), '# a edited\n'); // a.md is TRACKED
expect(isSourceUnchangedSinceSync(dir, head, { requireCleanWorkingTree: 'ignore-untracked' })).toBe(false);
});
test('HEAD moved to an OLD-dated commit → NOT caught up (codex: hash, not timestamp)', () => {
const { dir, head } = makeGitRepo(new Date(Date.now() - 200 * HOUR), repos);
// Add a SECOND commit with an even OLDER committer date. A timestamp
// comparison (newest content <= last sync) would falsely say "caught up";
// the hash check correctly sees HEAD != last_commit.
const olderIso = new Date(Date.now() - 500 * HOUR).toISOString();
const env = {
...process.env,
GIT_AUTHOR_DATE: olderIso, GIT_COMMITTER_DATE: olderIso,
GIT_AUTHOR_NAME: 't', GIT_AUTHOR_EMAIL: 't@t',
GIT_COMMITTER_NAME: 't', GIT_COMMITTER_EMAIL: 't@t',
};
writeFileSync(join(dir, 'b.md'), '# b\n');
execFileSync('git', ['-C', dir, 'add', '-A'], { stdio: ['ignore', 'pipe', 'ignore'], env });
execFileSync('git', ['-C', dir, 'commit', '-q', '-m', 'old-dated'], { stdio: ['ignore', 'pipe', 'ignore'], env });
// `head` is still the FIRST commit's sha (the recorded last_commit).
expect(isSourceUnchangedSinceSync(dir, head, { requireCleanWorkingTree: 'ignore-untracked' })).toBe(false);
});
test('NULL last_commit → not provably caught up (false)', () => {
const { dir } = makeGitRepo(new Date(Date.now() - 10 * HOUR), repos);
expect(isSourceUnchangedSinceSync(dir, null, { requireCleanWorkingTree: 'ignore-untracked' })).toBe(false);
});
});
@@ -128,7 +237,6 @@ describe('computeAllSourceMetrics', () => {
});
test('aggregates pages + chunks + embedding coverage per source', async () => {
// Two pages with chunks, half embedded
await engine.putPage('a', { type: 'note', title: 'a', compiled_truth: 'a' });
await engine.putPage('b', { type: 'note', title: 'b', compiled_truth: 'b' });
await engine.upsertChunks('a', [
@@ -145,7 +253,6 @@ describe('computeAllSourceMetrics', () => {
expect(dflt.total_pages).toBe(2);
expect(dflt.total_chunks).toBe(3);
expect(dflt.embedded_chunks).toBe(1);
// 1/3 = 33.3%
expect(dflt.embed_coverage_pct).toBeCloseTo(33.3, 1);
});
@@ -196,4 +303,65 @@ describe('computeAllSourceMetrics', () => {
const d = result.find((m) => m.source_id === 'default')!;
expect(d.webhook_configured).toBe(false);
});
// v0.41.32.0: commit-relative lag — local (probeContent) vs remote (column).
describe('commit-relative lag', () => {
const repos: string[] = [];
afterAll(() => {
for (const d of repos) { try { rmSync(d, { recursive: true, force: true }); } catch { /* best-effort */ } }
});
test('LOCAL (probeContent): caught-up repo synced 100h ago → lag 0', async () => {
const { dir, head } = makeGitRepo(new Date(Date.now() - 200 * HOUR), repos);
const syncIso = new Date(Date.now() - 100 * HOUR).toISOString();
await engine.executeRaw(
`INSERT INTO sources (id, name, local_path, last_commit, last_sync_at, config)
VALUES ('quiet', 'quiet', $1, $2, $3, '{"federated":true}'::jsonb)`,
[dir, head, syncIso],
);
const sources = await loadAllSources(engine);
const metrics = await computeAllSourceMetrics(engine, sources, { probeContent: true });
expect(metrics.find((m) => m.source_id === 'quiet')!.lag_seconds).toBe(0);
});
test('LOCAL (probeContent): HEAD moved (behind) → wall-clock lag', async () => {
const { dir } = makeGitRepo(new Date(Date.now() - 100 * HOUR), repos);
const staleCommit = 'b'.repeat(40); // last_commit no longer matches HEAD
const syncIso = new Date(Date.now() - 200 * HOUR).toISOString();
await engine.executeRaw(
`INSERT INTO sources (id, name, local_path, last_commit, last_sync_at, config)
VALUES ('behind', 'behind', $1, $2, $3, '{"federated":true}'::jsonb)`,
[dir, staleCommit, syncIso],
);
const sources = await loadAllSources(engine);
const metrics = await computeAllSourceMetrics(engine, sources, { probeContent: true });
expect(metrics.find((m) => m.source_id === 'behind')!.lag_seconds!).toBeGreaterThan(72 * 3600);
});
test('REMOTE (default): reads newest_content_at column, NO git probe → quiet repo lag 0', async () => {
const contentIso = new Date(Date.now() - 200 * HOUR).toISOString(); // content predates sync
const syncIso = new Date(Date.now() - 100 * HOUR).toISOString();
await engine.executeRaw(
`INSERT INTO sources (id, name, local_path, last_commit, last_sync_at, newest_content_at, config)
VALUES ('remote', 'remote', '/nonexistent/not-a-repo', 'x', $1, $2, '{"federated":true}'::jsonb)`,
[syncIso, contentIso],
);
const sources = await loadAllSources(engine);
// probeContent OFF (remote): even though local_path is bogus, no git runs.
const metrics = await computeAllSourceMetrics(engine, sources);
expect(metrics.find((m) => m.source_id === 'remote')!.lag_seconds).toBe(0);
});
test('REMOTE (default): NULL column → wall-clock fallback', async () => {
const syncIso = new Date(Date.now() - 100 * HOUR).toISOString();
await engine.executeRaw(
`INSERT INTO sources (id, name, local_path, last_commit, last_sync_at, config)
VALUES ('nocol', 'nocol', '/nonexistent', 'x', $1, '{"federated":true}'::jsonb)`,
[syncIso],
);
const sources = await loadAllSources(engine);
const metrics = await computeAllSourceMetrics(engine, sources);
expect(metrics.find((m) => m.source_id === 'nocol')!.lag_seconds!).toBeGreaterThan(99 * 3600);
});
});
});
+43 -1
View File
@@ -135,7 +135,7 @@ describe('buildSyncStatusReport', () => {
// `page_id` shape. The IRON RULE regression case lives in
// test/e2e/sync-status-pglite.test.ts and exercises real SQL.
function makeEngine(scripts: {
sourceRows?: Array<{ id: string; last_commit: string | null; last_sync_at: string | null }>;
sourceRows?: Array<{ id: string; last_commit: string | null; last_sync_at: string | null; newest_content_at?: string | null }>;
countRows?: Array<{ source_id: string; pages: number; chunks_total: number; chunks_unembedded: number }>;
}): BrainEngine {
return {
@@ -192,6 +192,48 @@ describe('buildSyncStatusReport', () => {
expect(byId.get('never')!.staleness_hours).toBeNull();
});
// v0.41.32.0 (supersedes #1623): buildSyncStatusReport backs the REMOTE
// get_status_snapshot MCP op, so staleness reads the stored newest_content_at
// column (NO git subprocess on a DB-supplied local_path). The makeEngine stub
// never runs git — if buildSyncStatusReport shelled out it would hit the real
// filesystem; these cases prove it reads the column instead.
test('content-relative staleness reads newest_content_at column (remote path)', async () => {
const now = Date.now();
const syncIso = new Date(now - 100 * 60 * 60 * 1000).toISOString(); // synced 100h ago
const sources = [
{ id: 'quiet', name: 'quiet', local_path: '/tmp/quiet', config: { syncEnabled: true } },
{ id: 'behind', name: 'behind', local_path: '/tmp/behind', config: { syncEnabled: true } },
{ id: 'nocol', name: 'nocol', local_path: '/tmp/nocol', config: { syncEnabled: true } },
];
const engine = makeEngine({
sourceRows: [
// Newest commit 200h ago, synced 100h ago → caught up → lag 0 → fresh.
{ id: 'quiet', last_commit: 'a'.repeat(40), last_sync_at: syncIso,
newest_content_at: new Date(now - 200 * 60 * 60 * 1000).toISOString() },
// Newest commit 10h ago, synced 100h ago → behind → wall-clock → severe.
{ id: 'behind', last_commit: 'b'.repeat(40), last_sync_at: syncIso,
newest_content_at: new Date(now - 10 * 60 * 60 * 1000).toISOString() },
// NULL column → wall-clock fallback → 100h → severe.
{ id: 'nocol', last_commit: 'c'.repeat(40), last_sync_at: syncIso, newest_content_at: null },
],
countRows: [
{ source_id: 'quiet', pages: 10, chunks_total: 20, chunks_unembedded: 0 },
{ source_id: 'behind', pages: 10, chunks_total: 20, chunks_unembedded: 0 },
{ source_id: 'nocol', pages: 10, chunks_total: 20, chunks_unembedded: 0 },
],
});
const report = await buildSyncStatusReport(engine, sources);
const byId = new Map(report.sources.map((s) => [s.source_id, s]));
// Legacy wall-clock would have called 'quiet' severe (100h). Content-relative
// correctly reports caught-up.
expect(byId.get('quiet')!.staleness_hours).toBe(0);
expect(byId.get('quiet')!.staleness_class).toBe('fresh');
expect(byId.get('behind')!.staleness_class).toBe('severe');
expect(byId.get('nocol')!.staleness_hours).toBeGreaterThan(72);
expect(byId.get('nocol')!.staleness_class).toBe('severe');
});
test('embedding_coverage_pct computed from chunks_total vs chunks_unembedded', async () => {
const sources = [
{ id: 'a', name: 'a', local_path: '/tmp/a', config: {} },