Compare commits

..
Author SHA1 Message Date
Garry TanandClaude Fable 5 bcb9d298f2 fix(pricing): add openai:gpt-5.2 canonical entry for the new default slot A
DEFAULT_SLOTS slot A moved to openai:gpt-5.2, which had no CANONICAL_PRICING
entry — estimateCost silently dropped slot A from the --max-usd pre-flight
and est_cost_usd audit rows (~1/3 under-count on the default panel). Rates
from the OpenAI recipe chat touchpoint (verified 2026-04-20). Also refresh
the --slot-a-model help text default and pin a pricing-presence assertion
in the DEFAULT_SLOTS consistency test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 11:18:09 -07:00
d76bb7fd68 fix(autopilot,eval): nightly quality probe enable path works end-to-end + wire conversation-parser probe
Takeover of #2629 and #2630 (rebased onto master; dropped the
test/engine-find-trajectory.test.ts hunk both PRs carried — master
already ships the equivalent gateway-dims fix).

#2629 — nightly quality probe enable path:
- autopilot + doctor read the probe flag dual-plane (DB config row from
  'gbrain config set' wins, ~/.gbrain/config.json fallback) via new
  resolveProbeEnabled/resolveProbeMaxUsd helpers
- resolveRepoRoot prefers the gbrain package root where the committed
  fixture lives, not the brain repoPath
- rate_limited skips no longer write an audit row every autopilot cycle
- eval-longmemeval strips 'provider:' recipe ids before raw Anthropic SDK
  calls and emits the gold answer for downstream judges
- cross-modal batch folds the gold answer into the judge task; probe
  passes QA-shaped dimensions instead of the agent-response rubric
- DEFAULT_SLOTS slot A moves to openai:gpt-5.2 (gpt-4o left the recipe);
  new consistency test pins every default slot to its recipe

#2630 — conversation-parser nightly probe wire-up:
- autopilot step 4.6 invokes runConversationParserNightlyProbe (dual-plane
  flag + D10 tokenmax mode-gate, package-root fixtures, 24h gate, audit
  trail via new src/core/audit-parser-probe.ts)
- doctor's conversation_parser_probe_health replaces the hardcoded
  'Skipped' stub with a real pure-function check over the audit trail

Co-authored-by: p3ob7o <p3ob7o@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 14:24:41 -07:00
35 changed files with 758 additions and 1515 deletions
-38
View File
@@ -115,7 +115,6 @@ Full subcommand reference:
```
gbrain sources add <id> --path <p> [--name <n>] [--federated|--no-federated] [--force]
[--include <glob>...] [--exclude <glob>...]
Register a source. id: [a-z0-9](?:[a-z0-9-]{0,30}[a-z0-9])?
--path must be a git repo (or a subdirectory of one) — see
"The git requirement for --path sources" below. --force
@@ -132,43 +131,6 @@ gbrain sources federate <id>
gbrain sources unfederate <id>
```
## Filtering what gets synced (--include / --exclude)
`--include` and `--exclude` on `gbrain sources add` accept repeatable glob
patterns and are honored by every subsequent sync AND lint of the source.
Common Obsidian vault setups need to exclude authoring scaffolding so it
doesn't pollute search:
```bash
# Skip Templates/, Drafts/, and the smart-env sidecar; everything else syncs.
gbrain sources add vault \
--path ~/Documents/vault --federated \
--exclude 'Templates/**' \
--exclude 'Drafts/**' \
--exclude '.smart-env/**'
# Or: only sync the people/ and companies/ subtrees of a CRM vault.
gbrain sources add crm \
--path ~/Documents/crm --no-federated \
--include 'people/**' \
--include 'companies/**'
```
Both persist into `sources.config.include_globs` / `exclude_globs` arrays.
The filter runs `include` first, then `exclude`, so a path inside
`people/**` is still rejected if it also matches `exclude_globs`. Globs use
the same matcher as the rest of gbrain's sync classifier (`matchesAnyGlob`
in `src/core/sync.ts`) and are matched against the source-root-relative
path. Exclusion is conservative: it never deletes previously-imported pages.
`gbrain sync --include <glob> --exclude <glob>` and
`gbrain lint <dir> --include <glob> --exclude <glob>` take the same
repeatable flags for one-off scope changes; for lint the persisted source
globs are auto-applied when the lint target matches a source's `local_path`.
Changing the persisted globs on an existing source triggers a full re-walk
on the next sync (the source's config fingerprint invalidates the
"already up to date" gate).
## The git requirement for --path sources
Every `--path` source must be a git repository (or live inside one — a
+82 -4
View File
@@ -527,6 +527,9 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
process.on('SIGINT', () => { void shutdown('SIGINT'); });
let consecutiveErrors = 0;
// Parser-probe fixture warning is once-per-process, not once-per-cycle
// (compiled-binary installs have no source tree; don't spam the log).
let parserProbeFixtureWarned = false;
// v0.37.7.0 #1162 — counter for consecutive reconnect failures.
// Reset on every successful health probe or reconnect. Threshold
// controlled by GBRAIN_AUTOPILOT_MAX_RECONNECT_FAILS env (default 30).
@@ -1073,17 +1076,36 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
// loop. Probe runs even when cycleOk=false (probe may surface signal
// explaining why the cycle is failing).
try {
const probeEnabled = cfg?.autopilot?.nightly_quality_probe?.enabled === true;
const { resolveProbeEnabled, resolveProbeMaxUsd, runNightlyQualityProbe } = await import('../core/cycle/nightly-quality-probe.ts');
// Dual-plane read: `gbrain config set` (what the doctor enable hint
// prints) writes the DB plane; ~/.gbrain/config.json is the fallback.
let dbEnabled: string | null = null;
let dbMaxUsd: string | null = null;
try {
dbEnabled = await engine.getConfig('autopilot.nightly_quality_probe.enabled');
dbMaxUsd = await engine.getConfig('autopilot.nightly_quality_probe.max_usd');
} catch { /* DB unavailable → file plane only */ }
const probeEnabled = resolveProbeEnabled(dbEnabled, cfg?.autopilot?.nightly_quality_probe?.enabled);
if (probeEnabled) {
const { runNightlyQualityProbe } = await import('../core/cycle/nightly-quality-probe.ts');
const { runLongMemEvalForProbe, runCrossModalBatchForProbe } = await import('../core/cycle/nightly-probe-adapters.ts');
const { isAvailable } = await import('../core/ai/gateway.ts');
const maxUsd = Number(cfg?.autopilot?.nightly_quality_probe?.max_usd ?? 5);
const { existsSync } = await import('node:fs');
const { fileURLToPath } = await import('node:url');
const { join } = await import('node:path');
const maxUsd = resolveProbeMaxUsd(dbMaxUsd, cfg?.autopilot?.nightly_quality_probe?.max_usd);
// The committed fixture (test/fixtures/longmemeval-nightly.jsonl)
// lives in the gbrain PACKAGE, not the brain repo — repoPath is
// sync.repo_path (the user's brain), where the fixture never
// exists, so the probe error'd on every real install. Resolve the
// package root from the module location; keep repoPath as the
// fallback for setups that vendor the fixture into the brain repo.
const pkgRoot = fileURLToPath(new URL('../..', import.meta.url));
const fixtureAtPkgRoot = existsSync(join(pkgRoot, 'test', 'fixtures', 'longmemeval-nightly.jsonl'));
await runNightlyQualityProbe({
isEnabled: () => true, // already gated above; phase re-checks for defense-in-depth
hasEmbeddingProvider: () => isAvailable('embedding'),
resolveMaxUsd: () => maxUsd,
resolveRepoRoot: () => repoPath ?? gbrainHomePath('.'),
resolveRepoRoot: () => (fixtureAtPkgRoot ? pkgRoot : repoPath ?? gbrainHomePath('.')),
runLongMemEval: runLongMemEvalForProbe,
runCrossModalBatch: runCrossModalBatchForProbe,
now: () => new Date(),
@@ -1095,6 +1117,62 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
// informational; autopilot loop continues.
}
// 4.6 — Nightly conversation-parser probe (v0.41.16.0 phase module;
// the scheduler wire-up was deferred at ship and is added here). Same
// posture as 4.5: the phase owns its gates (enabled/mode-gate, LLM
// key), the wiring owns invocation + the audit row, and a probe
// failure NEVER crashes the autopilot loop. Per D10 the probe is
// default-ON for search.mode=tokenmax, opt-in otherwise.
try {
const { runConversationParserNightlyProbe } = await import('../core/conversation-parser/nightly-probe.ts');
const { logParserProbeEvent, parserProbeRanWithin } = await import('../core/audit-parser-probe.ts');
const { isAvailable } = await import('../core/ai/gateway.ts');
const { existsSync } = await import('node:fs');
const { fileURLToPath } = await import('node:url');
const { join } = await import('node:path');
// Flag reads dual-plane: the DB row (`gbrain config set …`) wins,
// ~/.gbrain/config.json is the fallback. search.mode lives on the
// DB plane only (mode.ts owns it).
let parserDbEnabled: string | null = null;
let dbSearchMode: string | null = null;
try {
parserDbEnabled = await engine.getConfig('autopilot.conversation_parser_probe.enabled');
dbSearchMode = await engine.getConfig('search.mode');
} catch { /* DB unavailable → file plane only */ }
const parserEnabled = parserDbEnabled != null
? parserDbEnabled === 'true'
: cfg?.autopilot?.conversation_parser_probe?.enabled === true;
const searchMode = dbSearchMode ?? '';
// Fixtures are committed in the gbrain package (test/fixtures/…),
// NOT the brain repo — resolve from the module location. Compiled
// binaries carry no source tree: skip quietly instead of writing
// failure rows that would flip doctor to WARN on every binary install.
const pkgRoot = fileURLToPath(new URL('../..', import.meta.url));
const fixturePath = join(pkgRoot, 'test', 'fixtures', 'conversation-formats', 'all.jsonl');
const adversarialPath = join(pkgRoot, 'test', 'fixtures', 'conversation-formats', 'adversarial.jsonl');
const shouldInvoke = parserEnabled || searchMode === 'tokenmax';
if (shouldInvoke && existsSync(fixturePath) && existsSync(adversarialPath)) {
const result = await runConversationParserNightlyProbe({
isEnabled: () => parserEnabled,
searchMode: () => searchMode,
hasLlmKey: () => isAvailable('chat'),
resolveFixturePath: () => fixturePath,
resolveAdversarialPath: () => adversarialPath,
now: () => new Date(),
shouldSkipForRateLimit: () => parserProbeRanWithin(24 * 60 * 60 * 1000),
});
// rate_limited is a non-run: the loop ticks every few minutes, so
// logging every skip would flood the audit file with no-signal rows.
if (result.outcome !== 'rate_limited') logParserProbeEvent(result);
} else if (shouldInvoke && !parserProbeFixtureWarned) {
parserProbeFixtureWarned = true;
console.error(`[parser-probe] fixtures not found under ${pkgRoot}; skipping (probe needs a source-checkout install)`);
}
} catch (e) {
logError('autopilot.parser_probe', e);
// Informational, like 4.5: do NOT bump consecutiveErrors.
}
// Wait for next cycle
await new Promise(r => setTimeout(r, interval * 1000));
}
+79 -14
View File
@@ -2960,6 +2960,54 @@ function _resolveSyncFreshnessHours(varName: string, fallback: number): number {
* branch (disabled / enabled-no-events / enabled-all-pass / enabled-with-failures)
* without spinning up the audit JSONL or a real config file.
*/
/**
* Pure function form of the conversation_parser_probe_health check.
* Mirrors computeNightlyQualityProbeHealthCheck: skip-with-hint when the
* probe is off and silent, surface the last 7 days of audit events when
* it has run, WARN on any non-pass outcome.
*
* `effectiveEnabled` folds the D10 mode-gate in: explicitly enabled OR
* search.mode=tokenmax (where the probe is default-on).
*/
export function computeConversationParserProbeHealthCheck(
effectiveEnabled: boolean,
events: ReadonlyArray<{ outcome: string; ts: string; reason?: string }>,
): Check {
const name = 'conversation_parser_probe_health';
if (!effectiveEnabled && events.length === 0) {
return {
name,
status: 'ok',
message:
'disabled (opt-in; default-on only for search.mode=tokenmax). Enable with: ' +
'`gbrain config set autopilot.conversation_parser_probe.enabled true`',
};
}
if (events.length === 0) {
return {
name,
status: 'ok',
message: 'enabled but no probe events in the last 7 days (next run by autopilot; fixtures require a source-checkout install).',
};
}
const bad = events.filter(e => e.outcome !== 'pass');
const latest = events[events.length - 1]!;
if (bad.length > 0) {
return {
name,
status: 'warn',
message:
`${bad.length}/${events.length} probe run(s) in the last 7 days did not pass; ` +
`latest: ${latest.outcome}${latest.reason ? ` (${latest.reason})` : ''}`,
};
}
return {
name,
status: 'ok',
message: `${events.length} probe run(s) in the last 7 days, all pass (latest ${latest.ts}).`,
};
}
export function computeNightlyQualityProbeHealthCheck(
probeEnabled: boolean,
events: ReadonlyArray<{ outcome: string; ts: string; detail?: string }>,
@@ -4843,10 +4891,17 @@ export async function buildChecks(
try {
const { readRecentQualityProbeEvents } = await import('../core/audit-quality-probe.ts');
const { loadConfig } = await import('../core/config.ts');
const { resolveProbeEnabled } = await import('../core/cycle/nightly-quality-probe.ts');
let probeEnabled = false;
try {
// Dual-plane read, matching the autopilot gate: the DB row (what the
// enable hint's `gbrain config set` writes) wins; file plane fallback.
let dbVal: string | null = null;
try {
dbVal = engine ? await engine.getConfig('autopilot.nightly_quality_probe.enabled') : null;
} catch { /* DB unavailable → file plane only */ }
const cfg = loadConfig();
probeEnabled = Boolean((cfg as any)?.autopilot?.nightly_quality_probe?.enabled);
probeEnabled = resolveProbeEnabled(dbVal, (cfg as any)?.autopilot?.nightly_quality_probe?.enabled);
} catch { /* config unavailable → treat as disabled */ }
const events = readRecentQualityProbeEvents(7);
const check = computeNightlyQualityProbeHealthCheck(probeEnabled, events);
@@ -5030,19 +5085,29 @@ export async function buildChecks(
// 3d.5 v0.41.13.0 — conversation_parser_probe_health. Mode-gated
// per D10: ON when search.mode=tokenmax, opt-in for other modes.
// Surface the last 7 days of nightly-probe events; warn on FAIL /
// BUDGET_EXCEEDED / adversarial_false_positive.
//
// v0.41.13.0 ships the probe as opt-in (autopilot wiring deferred
// to T7 in the cathedral plan); this check skips with an enable
// hint until the probe has at least one audit event written.
checks.push({
name: 'conversation_parser_probe_health',
status: 'ok',
message:
'Skipped (nightly probe is opt-in; enable with ' +
'`gbrain config set autopilot.conversation_parser_probe.enabled true`)',
});
// Surfaces the last 7 days of nightly-probe audit events; warn on any
// non-pass outcome (fail / budget_exceeded / adversarial_false_positive).
// (Until the autopilot wire-up this was a hardcoded "Skipped" stub.)
try {
const { readRecentParserProbeEvents } = await import('../core/audit-parser-probe.ts');
let parserProbeEnabled = false;
try {
let dbVal: string | null = null;
let dbMode: string | null = null;
try {
dbVal = engine ? await engine.getConfig('autopilot.conversation_parser_probe.enabled') : null;
dbMode = engine ? await engine.getConfig('search.mode') : null;
} catch { /* DB unavailable → file plane only */ }
const { loadConfig } = await import('../core/config.ts');
const fileVal = (loadConfig() as any)?.autopilot?.conversation_parser_probe?.enabled;
const flagOn = dbVal != null ? dbVal === 'true' : fileVal === true;
parserProbeEnabled = flagOn || dbMode === 'tokenmax';
} catch { /* config unavailable → treat as disabled */ }
const parserEvents = readRecentParserProbeEvents(7);
checks.push(computeConversationParserProbeHealthCheck(parserProbeEnabled, parserEvents));
} catch {
// Best-effort; audit-log read failure shouldn't stop doctor.
}
// 3e. home_dir_in_worktree (v0.35.8.0). Walks up from `gbrainPath()`
// looking for a `.git` directory OR file. If found, warns: `~/.gbrain/`
+15 -2
View File
@@ -76,7 +76,7 @@ FLAGS:
dimensions (goal, depth, sourcing, specificity, useful).
--cycles N 1-3. Default: 3 in TTY, 1 in non-TTY (T11). Each
cycle is 3 model calls; verdict aggregates over them.
--slot-a-model <id> Override default 'openai:gpt-4o'.
--slot-a-model <id> Override default 'openai:gpt-5.2'.
--slot-b-model <id> Override default 'anthropic:claude-opus-4-7'.
--slot-c-model <id> Override default 'google:gemini-1.5-pro'.
--receipt-dir <path> Default: gbrainPath('eval-receipts').
@@ -468,6 +468,14 @@ interface BatchRow {
question_id: string;
question: string;
hypothesis: string;
/**
* Gold answer from the benchmark dataset, when the upstream eval emits
* it (eval-longmemeval does). Folded into the judge task so CORRECTNESS
* is verifiable — without it a judge panel that sees only
* {question, hypothesis} cannot validate a terse factual answer against
* a haystack it never saw.
*/
answer?: string;
}
/**
@@ -581,6 +589,7 @@ function readBatchRows(path: string): BatchReadResult {
question_id: typeof obj.question_id === 'string' ? obj.question_id : `line-${lineNo}`,
question: obj.question,
hypothesis: obj.hypothesis,
...(typeof obj.answer === 'string' && obj.answer.length > 0 ? { answer: obj.answer } : {}),
});
}
if (summarySkipped > 0) {
@@ -697,7 +706,11 @@ async function runBatchMode(parsed: ParsedArgs, opts: RunCrossModalOpts): Promis
fn: async (row, idx) => {
process.stderr.write(`[eval cross-modal batch] ${idx + 1}/${rows.length} ${row.question_id} starting...\n`);
return await runEvalFn({
task: row.question,
// With a gold answer the judges can actually verify correctness;
// without one they see only {question, hypothesis} and cannot.
task: row.answer
? `${row.question}\n\nExpected answer (gold label from the benchmark dataset): ${row.answer}`
: row.question,
output: row.hypothesis,
slug: row.question_id,
dimensions,
+17 -3
View File
@@ -33,6 +33,7 @@ import {
type AliasMap,
} from '../eval/longmemeval/extract.ts';
import { extractCandidateEntities } from '../core/think/entity-extract.ts';
import { splitProviderModelId } from '../core/model-id.ts';
import { resolveEntitySlugWithSource, type ResolutionSource } from '../core/entities/resolve.ts';
import { formatTrajectoryBlock } from '../core/trajectory-format.ts';
@@ -469,14 +470,22 @@ export async function runEvalLongMemEval(args: string[], runOpts: RunOpts = {}):
});
// Wrap Anthropic SDK so its `.messages.create` shape matches ThinkLLMClient.
// Same pattern as src/core/think/index.ts:247-249.
// Same pattern as src/core/think/index.ts:247-249 — EXCEPT think's default
// client routes through the gateway, which parses `provider:model` recipe
// ids. This eval's client is a raw SDK by design (hermetic, no gateway
// dependency), and resolveModel returns RECIPE ids (`anthropic:claude-…`);
// passing one through unstripped 404s every answer/extractor call, which
// surfaces downstream as all-upstream_error batches in the nightly probe.
const toSdkModel = (m: string): string => splitProviderModelId(m).model || m;
const realClient = new Anthropic();
const client: ThinkLLMClient = runOpts.client ?? {
create: (params, callOpts) => realClient.messages.create(params, callOpts),
create: (params, callOpts) =>
realClient.messages.create({ ...params, model: toSdkModel(params.model) }, callOpts),
};
// v0.40.2.0 — separate extractor client (defaults to same SDK).
const extractorClient: ThinkLLMClient = runOpts.extractorClient ?? {
create: (params, callOpts) => realClient.messages.create(params, callOpts),
create: (params, callOpts) =>
realClient.messages.create({ ...params, model: toSdkModel(params.model) }, callOpts),
};
const trajectoryEnabled = !opts.noTrajectory;
const extractorModel = trajectoryEnabled
@@ -751,6 +760,11 @@ async function runOneQuestion(
// v0.40.1.0 (Track D / T2) — copy question_type into the row so the
// by_type_summary can be rebuilt from the file on resume runs.
question_type: q.question_type,
// Gold answer for downstream consumers that verify correctness (the
// cross-modal --batch judge folds it into the task; evaluate_qa.py
// ignores unknown fields). Without it a judge can't validate a terse
// factual hypothesis against a haystack it never saw.
...(q.answer !== undefined ? { answer: q.answer } : {}),
hypothesis,
retrieved_session_ids: retrievedSessionIds,
...(recallHit !== undefined ? { recall_hit: recallHit } : {}),
-11
View File
@@ -53,13 +53,6 @@ export async function runImport(
strategy?: SyncStrategy;
sourceId?: string;
managedBookmark?: boolean;
/**
* #2156: allow-list glob patterns — only dir-relative paths matching at
* least one pattern are imported. Applied BEFORE `exclude`. Threaded by
* performFullSync from `gbrain sync --include` / the source row's
* persisted `config.include_globs`.
*/
include?: string[];
/**
* #753/#774: glob patterns to exclude from the import (same semantics as
* `isSyncable`'s `exclude` — matched against the dir-relative path).
@@ -222,10 +215,6 @@ export async function runImport(
);
const fileTypeLabel = strategy === 'code' ? 'code'
: strategy === 'auto' ? 'syncable' : 'markdown';
// #2156: apply --include allow-list globs first (threaded by performFullSync).
if (opts.include && opts.include.length > 0) {
allFiles = allFiles.filter(abs => matchesAnyGlob(relative(dir, abs), opts.include));
}
// #753/#774: apply --exclude glob patterns (threaded by performFullSync).
if (opts.exclude && opts.exclude.length > 0) {
const beforeExclude = allFiles.length;
+12 -172
View File
@@ -17,7 +17,7 @@
*/
import { readFileSync, writeFileSync, readdirSync, statSync, lstatSync, existsSync } from 'fs';
import { join, relative, resolve } from 'path';
import { join, relative } from 'path';
import { isAborted } from '../core/abort-check.ts';
import { parseMarkdown, type ParseValidationCode } from '../core/markdown.ts';
import {
@@ -26,9 +26,7 @@ import {
DEFAULT_BYTES_WARN,
} from '../core/content-sanity.ts';
import { loadOperatorLiterals } from '../core/content-sanity-literals.ts';
import { loadConfig, loadConfigWithEngine, toEngineConfig, gbrainPath } from '../core/config.ts';
import { matchesAnyGlob } from '../core/sync.ts';
import { parseGlobList } from './sync.ts';
import { loadConfig, loadConfigWithEngine, gbrainPath } from '../core/config.ts';
import type { BrainEngine } from '../core/engine.ts';
export interface LintIssue {
@@ -380,89 +378,21 @@ async function resolveLintContentSanity(
};
}
/** Collect markdown files from a directory.
*
* When `opts.include` or `opts.exclude` are set, each candidate `.md` path's
* POSIX-style relative path (relative to `dir`) is matched against the same
* glob semantics sync uses (`matchesAnyGlob`). `include` allow-lists;
* `exclude` deny-lists. Empty or undefined arrays leave the filter
* unengaged. Symmetric with `isSyncable` in `src/core/sync.ts` so a
* source-config `exclude_globs` honored by `gbrain sync` is also honored
* by `gbrain lint` against the same dir.
*/
function collectPages(
dir: string,
opts: { include?: string[]; exclude?: string[] } = {},
): string[] {
const { include, exclude } = opts;
const haveInclude = !!(include && include.length > 0);
const haveExclude = !!(exclude && exclude.length > 0);
/** Collect markdown files from a directory */
function collectPages(dir: string): string[] {
const pages: string[] = [];
function walk(d: string) {
for (const entry of readdirSync(d)) {
if (entry.startsWith('.') || entry.startsWith('_')) continue;
const full = join(d, entry);
if (lstatSync(full).isDirectory()) walk(full);
else if (entry.endsWith('.md')) {
if (haveInclude || haveExclude) {
// Match against the path RELATIVE to `dir` (the source root),
// normalized to POSIX separators by matchesAnyGlob. A
// source-config glob like `Resources/veriff/**` is anchored at
// the source root; matching against the absolute path would
// require the user to anchor on their `$HOME` or repo prefix,
// which is brittle.
const rel = relative(dir, full);
if (haveInclude && !matchesAnyGlob(rel, include)) continue;
if (haveExclude && matchesAnyGlob(rel, exclude)) continue;
}
pages.push(full);
}
else if (entry.endsWith('.md')) pages.push(full);
}
}
walk(dir);
return pages.sort();
}
/** Look up the source row whose `local_path` resolves to the same absolute
* directory as `target`, and return its persisted `include_globs` /
* `exclude_globs` as parsed string arrays. Returns an empty object when no
* matching source exists, when the row has no globs configured, or when the
* lookup throws (best-effort — auto-resolution must never break standalone
* lint on brains without a sources table).
*
* Mirrors how `syncOneSource` lifts the same fields off `src.config` before
* threading them into `SyncOpts.include` / `SyncOpts.exclude`.
*/
async function resolveSourceGlobsForTarget(
engine: BrainEngine,
target: string,
): Promise<{ include?: string[]; exclude?: string[] }> {
try {
const absTarget = resolve(target);
const rows = await engine.executeRaw<{ config: unknown }>(
`SELECT config FROM sources
WHERE archived IS NOT TRUE
AND local_path IS NOT NULL
AND local_path = $1
LIMIT 1`,
[absTarget],
);
if (rows.length === 0) return {};
const cfg = (rows[0].config && typeof rows[0].config === 'object')
? rows[0].config as Record<string, unknown>
: {};
return {
include: parseGlobList(cfg.include_globs),
exclude: parseGlobList(cfg.exclude_globs),
};
} catch {
// Engine not connected, sources table missing on a fresh brain, RLS
// denial in an unusual scope — all best-effort. Lint proceeds without
// filtering rather than fail-closed.
return {};
}
}
export interface LintOpts {
target: string;
fix?: boolean;
@@ -484,22 +414,6 @@ export interface LintOpts {
* yields + checks this every 200 pages.
*/
signal?: AbortSignal;
/**
* Glob filters threaded into the file walker. When set, paths relative to
* `target` are matched against the patterns using the same semantics as
* `gbrain sync` (`matchesAnyGlob` in `src/core/sync.ts`). `include`
* allow-lists; `exclude` deny-lists; both unset == no filter.
*
* When BOTH are unset AND `engine` is provided, `runLintCore` attempts to
* auto-resolve them from the `sources` row whose `local_path` matches
* `target` — symmetric with `syncOneSource`, so a user who has run
* `gbrain sources add --exclude 'Resources/veriff/**'` sees the same
* exclusion applied to `gbrain lint <same-dir>` and to the cycle.lint
* phase without restating it on every invocation. Explicit caller-supplied
* arrays always win over the source-row lift.
*/
include?: string[];
exclude?: string[];
}
export interface LintResult {
@@ -526,21 +440,7 @@ export async function runLintCore(opts: LintOpts): Promise<LintResult> {
}
const isSingleFile = statSync(opts.target).isFile();
// Resolve glob filters. Explicit caller-supplied include/exclude win;
// otherwise lift from `sources.config.{include,exclude}_globs` when an
// engine is available and the target matches a known source's local_path.
// Single-file lints skip the resolve entirely — globs are a directory
// walk concern.
let include = opts.include;
let exclude = opts.exclude;
const haveExplicit = (include && include.length > 0) || (exclude && exclude.length > 0);
if (!isSingleFile && !haveExplicit && opts.engine) {
const resolved = await resolveSourceGlobsForTarget(opts.engine, opts.target);
include = resolved.include;
exclude = resolved.exclude;
}
const pages = isSingleFile ? [opts.target] : collectPages(opts.target, { include, exclude });
const pages = isSingleFile ? [opts.target] : collectPages(opts.target);
// Resolve content-sanity config once for this lint run (D1: lift DB
// config when reachable). Caller can pre-pass via opts.contentSanity
@@ -591,27 +491,14 @@ export async function runLintCore(opts: LintOpts): Promise<LintResult> {
}
export async function runLint(args: string[]) {
const target = args.find(a => !a.startsWith('--') && !args[args.indexOf(a) - 1]?.match(/^--(include|exclude)$/));
const target = args.find(a => !a.startsWith('--'));
const doFix = args.includes('--fix');
const dryRun = args.includes('--dry-run');
// Parse repeatable `--include <glob>` and `--exclude <glob>` flags.
// Symmetric with `gbrain sources add --include / --exclude` from PR #2157;
// explicit flags here override the source-config lift performed below for
// dir-mode lints.
const cliInclude: string[] = [];
const cliExclude: string[] = [];
for (let i = 0; i < args.length; i++) {
if (args[i] === '--include' && i + 1 < args.length) cliInclude.push(args[++i]);
else if (args[i] === '--exclude' && i + 1 < args.length) cliExclude.push(args[++i]);
}
if (!target) {
console.error('Usage: gbrain lint <dir|file.md> [--fix] [--dry-run] [--include <glob>]... [--exclude <glob>]...');
console.error(' --fix Auto-fix fixable issues (LLM preambles, code fences)');
console.error(' --dry-run Preview fixes without writing');
console.error(' --include <glob> Repeatable; only lint paths matching at least one pattern');
console.error(' --exclude <glob> Repeatable; skip paths matching any pattern (applied after --include)');
console.error('Usage: gbrain lint <dir|file.md> [--fix] [--dry-run]');
console.error(' --fix Auto-fix fixable issues (LLM preambles, code fences)');
console.error(' --dry-run Preview fixes without writing');
process.exit(1);
}
@@ -623,44 +510,7 @@ export async function runLint(args: string[]) {
// Single file or directory — print human detail as we go, then rely on
// Core for the aggregate numbers at the end.
const isSingleFile = statSync(target).isFile();
// Resolve glob filters for directory lints. Explicit CLI flags win;
// otherwise lift from `sources.config.{include,exclude}_globs` matching
// `target`. Connect a transient engine for the lookup only when (a) no
// explicit flags were passed AND (b) file/env config suggests an engine is
// available — mirrors the connect-disconnect pattern in
// `resolveLintContentSanity` (issue #1678: standalone CLI never shares the
// db.ts singleton, so create + dispose here is safe).
let runInclude: string[] | undefined = cliInclude.length > 0 ? cliInclude : undefined;
let runExclude: string[] | undefined = cliExclude.length > 0 ? cliExclude : undefined;
if (!isSingleFile && runInclude === undefined && runExclude === undefined) {
const base = loadConfig();
if (base?.database_url || base?.database_path) {
try {
const { createEngine } = await import('../core/engine-factory.ts');
const { connectWithRetry } = await import('../core/db.ts');
const engineCfg = toEngineConfig(base);
const engine = await createEngine(engineCfg);
try {
// Use the same connect path the rest of the CLI uses
// (`connectEngine` in cli.ts). `engine.connect({})` with empty
// opts drops the URL — confirmed by direct probe. `noRetry: true`
// keeps the standalone lint snappy (no retry tax when the brain
// happens to be unreachable; auto-resolve degrades to no-filter).
await connectWithRetry(engine, engineCfg, { noRetry: true });
const lifted = await resolveSourceGlobsForTarget(engine, target);
runInclude = lifted.include;
runExclude = lifted.exclude;
} finally {
await engine.disconnect().catch(() => { /* best-effort */ });
}
} catch {
// best-effort; fall through to no-filter
}
}
}
const pages = isSingleFile ? [target] : collectPages(target, { include: runInclude, exclude: runExclude });
const pages = isSingleFile ? [target] : collectPages(target);
// Progress on stderr. Stdout keeps the per-issue human output it always had.
const { createProgress } = await import('../core/progress.ts');
@@ -707,17 +557,7 @@ export async function runLint(args: string[]) {
// produces canonical numbers for the summary line).
// Pass contentSanity through so runLintCore skips its own resolve
// (we already resolved once for the human-detail loop above).
// Pass include/exclude so the aggregate scope matches the human-detail
// walk above — otherwise the summary line reports the unfiltered count
// even though the per-page details were already filtered.
const result = await runLintCore({
target,
fix: doFix,
dryRun,
contentSanity,
include: runInclude,
exclude: runExclude,
});
const result = await runLintCore({ target, fix: doFix, dryRun, contentSanity });
console.log(`\n${result.pages_scanned} pages scanned. ${result.total_issues} issue(s) in ${result.pages_with_issues} page(s).`);
if (doFix) {
console.log(`${dryRun ? '(dry run) ' : ''}${result.total_fixed} auto-fixed.`);
+1 -34
View File
@@ -122,8 +122,7 @@ async function runAdd(engine: BrainEngine, args: string[]): Promise<void> {
if (!id) {
console.error(
'Usage: gbrain sources add <id> [--path <path> | --url <https-url>] ' +
'[--name <display>] [--federated|--no-federated] [--clone-dir <path>] [--force] ' +
'[--include <glob>...] [--exclude <glob>...]',
'[--name <display>] [--federated|--no-federated] [--clone-dir <path>] [--force]',
);
process.exit(2);
}
@@ -136,12 +135,6 @@ async function runAdd(engine: BrainEngine, args: string[]): Promise<void> {
let patFile: string | undefined;
let noHarden = false;
let force = false;
// Repeatable. `--include 'people/**' --include 'companies/**'` accumulates.
// Persisted into sources.config.include_globs / .exclude_globs and read at
// sync time by commands/sync.ts so `Templates/`, `.smart-env/`, `Drafts/`
// and other vault scaffolding can be skipped without renaming directories.
const includeGlobs: string[] = [];
const excludeGlobs: string[] = [];
for (let i = 1; i < args.length; i++) {
const a = args[i];
@@ -154,24 +147,6 @@ async function runAdd(engine: BrainEngine, args: string[]): Promise<void> {
if (a === '--pat-file') { patFile = args[++i]; continue; }
if (a === '--no-harden') { noHarden = true; continue; }
if (a === '--force') { force = true; continue; }
if (a === '--include') {
const v = args[++i];
if (!v || v.startsWith('--')) {
console.error('Error: --include requires a glob argument (e.g. --include "people/**")');
process.exit(2);
}
includeGlobs.push(v);
continue;
}
if (a === '--exclude') {
const v = args[++i];
if (!v || v.startsWith('--')) {
console.error('Error: --exclude requires a glob argument (e.g. --exclude "Templates/**")');
process.exit(2);
}
excludeGlobs.push(v);
continue;
}
console.error(`Unknown flag: ${a}`);
process.exit(2);
}
@@ -192,8 +167,6 @@ async function runAdd(engine: BrainEngine, args: string[]): Promise<void> {
federated,
cloneDir,
force,
includeGlobs: includeGlobs.length > 0 ? includeGlobs : undefined,
excludeGlobs: excludeGlobs.length > 0 ? excludeGlobs : undefined,
});
// Topology A discovery: if the just-added source carries a brain-resident
@@ -217,12 +190,6 @@ async function runAdd(engine: BrainEngine, args: string[]): Promise<void> {
console.log(
` federated: ${fed}${fed ? ' — appears in cross-source default search' : ' — only searched when explicitly named via --source'}`,
);
if (includeGlobs.length > 0) {
console.log(` include globs: ${includeGlobs.join(', ')}`);
}
if (excludeGlobs.length > 0) {
console.log(` exclude globs: ${excludeGlobs.join(', ')}`);
}
// v0.42.44 — auto-harden managed clones for git durability the moment a brain
// repo is added with a PAT. Best-effort: NEVER fail `add` if hardening fails.
+18 -256
View File
@@ -1,7 +1,6 @@
import { existsSync, readFileSync, writeFileSync, statSync, realpathSync } from 'fs';
import { execFileSync } from 'child_process';
import { join, relative } from 'path';
import { createHash } from 'crypto';
import type { BrainEngine } from '../core/engine.ts';
import { DELETE_BATCH_SIZE } from '../core/engine-constants.ts';
import { importFile } from '../core/import-file.ts';
@@ -757,23 +756,12 @@ export interface SyncOpts {
* are rejected before any git op runs.
*/
srcSubpath?: string;
/**
* #2156 glob patterns files must match to be synced (allow-list).
* Populated from the source row's persisted `config.include_globs`
* (set via `gbrain sources add --include <glob>`) or the repeatable
* `--include` CLI flag. Matched against the scope-relative path, same
* anchoring as `exclude`. `exclude` is applied after `include`: a path
* matching an include pattern is still rejected if it also matches an
* exclude pattern. Empty arrays are the same as undefined (no filter).
*/
include?: string[];
/**
* #753/#774 — glob patterns for files to exclude from sync (repeatable
* `--exclude` on the CLI; #2156: also populated from the source row's
* persisted `config.exclude_globs`). Matched against the scope-relative
* path in both the full-sync and incremental paths. Excluded files are
* never imported; exclusion does NOT delete previously-imported pages
* (conservative, matching the #1433 metafile posture).
* `--exclude` on the CLI). Matched against the scope-relative path in both
* the full-sync and incremental paths. Excluded files are never imported;
* exclusion does NOT delete previously-imported pages (conservative,
* matching the #1433 metafile posture).
*/
exclude?: string[];
/**
@@ -1165,29 +1153,6 @@ function unique<T>(items: T[]): T[] {
// `src/core/sync-delta.ts` (re-imported below) so the inline cost estimator
// prices detached sources through the same code the executor imports them with.
/**
* Defensive parse for the JSONB-loaded `config.include_globs` / `config.exclude_globs`
* arrays read off the sources row. The column is a free-form JSONB and could
* contain anything coerce to a string-only array, drop empties, and return
* undefined when the result has no useful entries so the caller can decide
* not to engage glob-filtering at all.
*/
export function parseGlobList(value: unknown): string[] | undefined {
if (!Array.isArray(value)) return undefined;
const globs = value.filter((v): v is string => typeof v === 'string' && v.length > 0);
return globs.length > 0 ? globs : undefined;
}
/**
* Union of CLI-supplied glob patterns (one-off, this invocation) and the
* source row's persisted config globs (every sync). Deduped; undefined when
* neither side has entries so `SyncOpts` stays unset and no filter engages.
*/
export function mergeGlobs(cli: string[], persisted: string[] | undefined): string[] | undefined {
const merged = [...new Set([...cli, ...(persisted ?? [])])];
return merged.length > 0 ? merged : undefined;
}
// v0.18.0 Step 5: source-scoped sync state helpers. When opts.sourceId
// is set, read/write the per-source row instead of the global config
// keys. These wrappers centralize the branch so every read/write site
@@ -1350,125 +1315,6 @@ async function writeChunkerVersion(
);
}
/**
* #2157 follow-on: detect when sources.config has shifted in a way that
* affects which paths the walker will include this run. The "Already up
* to date" gate at performSync's git-HEAD equality check honored chunker
* version match but ignored config drift a user who changes
* `sources.config.exclude_globs` mid-life got "Already up to date" on
* the next sync because git HEAD was unchanged, with no observable
* effect until `gbrain sync --full`.
*
* Fingerprint covers exactly the walk-affecting fields that flow from
* `sources.config` into `SyncOpts` at the syncOneSource call site:
* `strategy`, `include_globs`, `exclude_globs`. CLI-supplied --include
* / --exclude overrides do NOT participate they are one-off scope
* changes, not source state, and shouldn't invalidate the row's
* checkpoint. (A user running `gbrain sync --exclude X` on a row whose
* stored config has no X is intentionally narrowing this one pass; on
* the next no-flags sync, the row config governs again.)
*
* Array order is normalized (alphabetical, post-defensive-parse) so
* `["a/**", "b/**"]` and `["b/**", "a/**"]` fingerprint identically.
* `parseGlobList` shares the same defensive coercion as the call site
* that builds SyncOpts, so hand-edited or pre-normalization rows
* fingerprint to the same shape the walker actually sees.
*/
export function computeSourceConfigFingerprint(rawConfig: unknown): string {
const cfg = (rawConfig || {}) as {
strategy?: unknown;
include_globs?: unknown;
exclude_globs?: unknown;
};
const canonical = JSON.stringify({
strategy: typeof cfg.strategy === 'string' ? cfg.strategy : null,
include_globs: (parseGlobList(cfg.include_globs) ?? []).slice().sort(),
exclude_globs: (parseGlobList(cfg.exclude_globs) ?? []).slice().sort(),
});
return createHash('sha256').update(canonical).digest('hex');
}
/**
* Read the per-source fingerprint stamp. NULL on pre-migration rows or
* sources that have never been synced the gate treats NULL as
* "fingerprint unknown" and skips the invalidation check so first-time
* post-upgrade syncs don't spuriously force-full.
*/
export async function readConfigFingerprint(
engine: BrainEngine,
sourceId: string | undefined,
): Promise<string | null> {
if (!sourceId) return null;
const rows = await engine.executeRaw<{ config_fingerprint: string | null }>(
`SELECT config_fingerprint FROM sources WHERE id = $1`,
[sourceId],
);
return rows[0]?.config_fingerprint ?? null;
}
export async function writeConfigFingerprint(
engine: BrainEngine,
sourceId: string | undefined,
fingerprint: string,
): Promise<void> {
if (!sourceId) return;
await engine.executeRaw(
`UPDATE sources SET config_fingerprint = $1 WHERE id = $2`,
[fingerprint, sourceId],
);
}
/**
* Read the raw `sources.config` value for the named source. Returns an
* empty object for missing or never-configured rows. The reader is
* defensive about legacy double-encoded JSONB rows (`{"federated":true}`
* stored as a JSON string scalar, the #2339 class) the engine's
* `r.config` may arrive as either a string or an object, and both are
* normalized to an object before the fingerprint computation walks the
* keys.
*/
async function readSourceConfig(
engine: BrainEngine,
sourceId: string | undefined,
): Promise<unknown> {
if (!sourceId) return {};
const rows = await engine.executeRaw<{ config: unknown }>(
`SELECT config FROM sources WHERE id = $1`,
[sourceId],
);
const raw = rows[0]?.config;
if (raw === null || raw === undefined) return {};
if (typeof raw === 'string') {
try { return JSON.parse(raw); } catch { return {}; }
}
return raw;
}
/**
* Read-hash-stamp wrapper for sync-completion sites outside the gate's
* scope (e.g. `performFullSync`'s `advanceFull` closure, which doesn't
* see `performSync`'s cached `currentConfigFp` because it's a separate
* function). Reads the row's current config and stamps a fresh
* fingerprint.
*
* Race note: if `sources.config` was mutated between the gate's read
* and this stamp, the freshly-read value wins. The walker still used
* the gate-time effective globs (already captured into `opts.include` /
* `opts.exclude` upstream), so the stamp can drift from what was
* actually walked. In practice mid-sync mutations are rare and the
* NEXT sync will re-evaluate against the latest config anyway, so the
* minor staleness is acceptable and avoids threading the gate-time
* fingerprint through every helper signature.
*/
async function stampSourceConfigFingerprint(
engine: BrainEngine,
sourceId: string | undefined,
): Promise<void> {
if (!sourceId) return;
const cfg = await readSourceConfig(engine, sourceId);
await writeConfigFingerprint(engine, sourceId, computeSourceConfigFingerprint(cfg));
}
/**
* v0.40 Federated Sync v2: `gbrain sync trigger --source <id> [--priority high|normal|low]`
*
@@ -2317,25 +2163,7 @@ async function performSyncInner(engine: BrainEngine, opts: SyncOpts): Promise<Sy
detachedWorkingTreeManifest.deleted.length > 0 ||
detachedWorkingTreeManifest.renamed.length > 0);
// #2157 follow-on: parallel gate for sources.config drift. Without
// this, changing `sources.config.exclude_globs` (or include_globs /
// strategy) on a synced source had no observable effect on the next
// sync because git HEAD was unchanged — the "Already up to date"
// branch below returned without re-walking. Mismatch path mirrors the
// chunker_version gate exactly so both kinds of drift route through
// the same `performFullSync` recovery.
//
// NULL stored fingerprint is "never stamped" (pre-v125 brain OR fresh
// source whose first sync hasn't completed yet). Treated as
// pass-through in the up-to-date check — first post-upgrade sync
// stamps the column quietly so subsequent passes have a baseline.
const storedConfigFp = await readConfigFingerprint(engine, opts.sourceId);
const currentSourceConfig = await readSourceConfig(engine, opts.sourceId);
const currentConfigFp = computeSourceConfigFingerprint(currentSourceConfig);
const configMismatch = storedConfigFp !== null && storedConfigFp !== currentConfigFp;
const configNeverStamped = storedConfigFp === null && opts.sourceId !== undefined;
if (lastCommit === headCommit && !versionMismatch && !versionNeverSet && !hasDetachedWorkingTreeChanges && !configMismatch) {
if (lastCommit === headCommit && !versionMismatch && !versionNeverSet && !hasDetachedWorkingTreeChanges) {
// v0.42.52.0 (PR #22xx): bump last_sync_at as a heartbeat on every successful
// 0-changes sync. D4 invariant ("never advance last_commit on partial") is
// preserved: last_sync_at is a monitoring signal (doctor sync_freshness
@@ -2348,14 +2176,6 @@ async function performSyncInner(engine: BrainEngine, opts: SyncOpts): Promise<Sy
[opts.sourceId],
);
}
// First post-upgrade sync on a pre-v125 brain lands here with
// configNeverStamped=true; stamp the fingerprint so the gate has a
// baseline for the NEXT pass. A spurious re-walk on the upgrade
// pass would surprise users; quietly establishing the baseline does
// not.
if (configNeverStamped) {
await writeConfigFingerprint(engine, opts.sourceId, currentConfigFp);
}
return {
status: 'up_to_date',
fromCommit: lastCommit,
@@ -2367,21 +2187,13 @@ async function performSyncInner(engine: BrainEngine, opts: SyncOpts): Promise<Sy
};
}
if ((versionMismatch || versionNeverSet || configMismatch) && lastCommit === headCommit) {
const reasons: string[] = [];
if (versionMismatch || versionNeverSet) {
reasons.push(`chunker_version=${storedVersion ?? 'unset'}${currentVersion}`);
}
if (configMismatch) {
reasons.push(`config_fingerprint=${storedConfigFp?.slice(0, 8)}${currentConfigFp.slice(0, 8)}`);
}
if ((versionMismatch || versionNeverSet) && lastCommit === headCommit) {
slog(
`[sync] full re-walk forced (${reasons.join(', ')}): ` +
`git HEAD unchanged but a walk-affecting setting advanced.`,
`[sync] chunker_version gate: stored=${storedVersion ?? 'unset'}, current=${currentVersion}. ` +
`Forcing full re-chunk pass (git HEAD unchanged but pipeline version advanced).`,
);
const result = await performFullSync(engine, fullSyncRoots, headCommit, opts);
await writeChunkerVersion(engine, opts.sourceId, currentVersion);
await writeConfigFingerprint(engine, opts.sourceId, currentConfigFp);
return result;
}
@@ -2425,16 +2237,8 @@ async function performSyncInner(engine: BrainEngine, opts: SyncOpts): Promise<Sy
scoped && p.startsWith(syncScopeRelPath + '/') ? p.slice(syncScopeRelPath.length + 1) : p;
const excluded = (p: string): boolean =>
opts.exclude !== undefined && opts.exclude.length > 0 && matchesAnyGlob(scopeRel(p), opts.exclude);
// #2156: include globs are an allow-list, same scope-relative anchoring as
// exclude. Populated from the source row's persisted config.include_globs
// (or CLI --include). Deliberately NOT threaded into syncOpts/isSyncable:
// the unsyncable-cleanup loop below deletes pages for non-metafile
// classifications, and glob filtering must stay conservative (never delete
// previously-imported pages — the documented #1433 posture for --exclude).
const included = (p: string): boolean =>
opts.include === undefined || opts.include.length === 0 || matchesAnyGlob(scopeRel(p), opts.include);
// Filter to syncable files (strategy-aware + scope-aware + glob-aware)
// Filter to syncable files (strategy-aware + scope-aware + exclude-aware)
const syncOpts = opts.strategy ? { strategy: opts.strategy } : undefined;
// #1970 (F-C): a rename whose DESTINATION is unsyncable drops out of BOTH
// `renamed` (only `r.to` is kept below) AND `deleted` (git emits it as `R`,
@@ -2448,13 +2252,13 @@ async function performSyncInner(engine: BrainEngine, opts: SyncOpts): Promise<Sy
!(inScope(r.to) && isSyncable(r.to, syncOpts)))
.map(r => r.from);
const filtered: SyncManifest = {
added: manifest.added.filter(p => inScope(p) && included(p) && !excluded(p) && isSyncable(p, syncOpts)),
modified: manifest.modified.filter(p => inScope(p) && included(p) && !excluded(p) && isSyncable(p, syncOpts)),
added: manifest.added.filter(p => inScope(p) && !excluded(p) && isSyncable(p, syncOpts)),
modified: manifest.modified.filter(p => inScope(p) && !excluded(p) && isSyncable(p, syncOpts)),
deleted: unique([
...manifest.deleted.filter(p => inScope(p) && isSyncable(p, syncOpts)),
...renamedToUnsyncable,
]),
renamed: manifest.renamed.filter(r => inScope(r.to) && included(r.to) && !excluded(r.to) && isSyncable(r.to, syncOpts)),
renamed: manifest.renamed.filter(r => inScope(r.to) && !excluded(r.to) && isSyncable(r.to, syncOpts)),
};
// NAV-4: warn when --exclude filtered out every candidate change — almost
@@ -2551,7 +2355,6 @@ async function performSyncInner(engine: BrainEngine, opts: SyncOpts): Promise<Sy
await writeSyncAnchor(engine, opts.sourceId, 'last_commit', pin, commitTimeMs(gitContextRoot, pin));
await engine.setConfig('sync.last_run', new Date().toISOString());
await writeChunkerVersion(engine, opts.sourceId, String(CHUNKER_VERSION));
await writeConfigFingerprint(engine, opts.sourceId, currentConfigFp);
await clearOpCheckpoint(engine, ckpt.paths);
await clearOpCheckpoint(engine, ckpt.target);
return {
@@ -3376,7 +3179,6 @@ async function performSyncInner(engine: BrainEngine, opts: SyncOpts): Promise<Sy
await engine.setConfig('sync.last_run', new Date().toISOString());
await writeSyncAnchor(engine, opts.sourceId, 'repo_path', anchorPath);
await writeChunkerVersion(engine, opts.sourceId, String(CHUNKER_VERSION));
await writeConfigFingerprint(engine, opts.sourceId, currentConfigFp);
await clearOpCheckpoint(engine, ckpt.paths);
await clearOpCheckpoint(engine, ckpt.target);
};
@@ -3628,9 +3430,6 @@ async function performFullSync(
// files were waiting.
if (opts.dryRun) {
let allFiles = collectSyncableFiles(syncScopeRoot, { strategy: opts.strategy ?? 'markdown' });
if (opts.include && opts.include.length > 0) {
allFiles = allFiles.filter(abs => matchesAnyGlob(relative(syncScopeRoot, abs), opts.include));
}
if (opts.exclude && opts.exclude.length > 0) {
allFiles = allFiles.filter(abs => !matchesAnyGlob(relative(syncScopeRoot, abs), opts.exclude));
}
@@ -3676,7 +3475,6 @@ async function performFullSync(
commit: headCommit,
strategy: opts.strategy,
sourceId: opts.sourceId,
include: opts.include,
exclude: opts.exclude,
slugRoot,
// issue #1939: performFullSync owns the failure ledger + bookmark via the
@@ -3706,7 +3504,6 @@ async function performFullSync(
await engine.setConfig('sync.last_run', new Date().toISOString());
await writeSyncAnchor(engine, opts.sourceId, 'repo_path', anchorPath);
await writeChunkerVersion(engine, opts.sourceId, String(CHUNKER_VERSION));
await stampSourceConfigFingerprint(engine, opts.sourceId);
};
const fullGate = await applySyncFailureGate({
@@ -4170,12 +3967,8 @@ Options:
run at the repo root; imports are scoped to the subdir
and slugs stay root-relative (wiki/page1). Passing the
subdirectory directly as --repo also works.
--include <glob> Only sync files matching at least one glob (repeatable;
matched against the scope-relative path). Merged with
the source's persisted config.include_globs.
--exclude <glob> Exclude files matching the glob from sync (repeatable;
matched against the scope-relative path; applied after
--include). Merged with config.exclude_globs.
matched against the scope-relative path).
--dry-run Show what would be synced without writing.
--skip-failed Acknowledge previously-recorded sync failures so
the bookmark can advance past unparseable files.
@@ -4336,17 +4129,14 @@ See also:
}
const strategyArg = args.find((a, i) => args[i - 1] === '--strategy') as SyncOpts['strategy'] | undefined;
// #753/#774: monorepo subdir-source flags. --exclude is repeatable.
// #2156: --include is the allow-list counterpart, same repeatable shape.
const srcSubpath = args.find((a, i) => args[i - 1] === '--src-subpath') || undefined;
const excludePatterns: string[] = [];
const includePatterns: string[] = [];
for (let i = 0; i < args.length; i++) {
if (args[i] === '--exclude' && i + 1 < args.length) excludePatterns.push(args[i + 1]);
if (args[i] === '--include' && i + 1 < args.length) includePatterns.push(args[i + 1]);
}
if (syncAll && (srcSubpath || excludePatterns.length > 0 || includePatterns.length > 0)) {
if (syncAll && (srcSubpath || excludePatterns.length > 0)) {
console.error(
`--src-subpath/--include/--exclude scope a single sync invocation; they cannot be combined with --all. ` +
`--src-subpath/--exclude scope a single sync invocation; they cannot be combined with --all. ` +
`For --all runs, register the subdirectory as the source's local_path instead ` +
`(gbrain sources add <id> --path <repo>/<subdir>).`,
);
@@ -4547,11 +4337,7 @@ See also:
const onAllSigint = () => { try { allInterrupt.abort(new Error('SIGINT')); } catch { /* */ } };
const runOne = async (src: typeof sources[number]): Promise<SyncResult> => {
const cfg = (src.config || {}) as {
strategy?: 'markdown' | 'code' | 'auto';
include_globs?: unknown;
exclude_globs?: unknown;
};
const cfg = (src.config || {}) as { strategy?: 'markdown' | 'code' | 'auto' };
// D18: parallel path defers embed; auto-enqueue embed-backfill after.
// v0.42.42.0 (#2139): `autoDeferEmbeds` (the inline gate tripped in a
// non-TTY session) ALSO forces deferral — global by design (the gate's
@@ -4589,8 +4375,6 @@ See also:
skipFailed, retryFailed, noSchemaPack,
sourceId: src.id,
strategy: cfg.strategy,
include: parseGlobList(cfg.include_globs),
exclude: parseGlobList(cfg.exclude_globs),
concurrency,
signal: composeAbortSignals(allInterrupt.signal, controller?.signal),
};
@@ -4802,27 +4586,11 @@ See also:
// lock released by its own finally) instead of a hard cut.
const singleSourceInterrupt = new AbortController();
const onSingleSourceSigint = () => { try { singleSourceInterrupt.abort(new Error('SIGINT')); } catch { /* */ } };
// Read persisted include/exclude globs from the source row, mirroring the
// --all fan-out's `runOne` closure above. Best-effort: a fetch failure
// falls through to "no glob filters", preserving pre-existing behavior.
// sourceId is always set here (resolveSourceWithTier ran above), so this
// path never silently runs without source-config awareness.
let sourceCfg: { include_globs?: unknown; exclude_globs?: unknown } = {};
try {
const { fetchSource } = await import('../core/sources-load.ts');
const src = await fetchSource(engine, sourceId);
if (src?.config && typeof src.config === 'object') {
sourceCfg = src.config as { include_globs?: unknown; exclude_globs?: unknown };
}
} catch { /* fall through to no filters */ }
const opts: SyncOpts = {
repoPath, dryRun, full, noPull, noEmbed, noExtract, skipFailed, retryFailed, noSchemaPack, sourceId,
strategy: strategyArg, concurrency,
srcSubpath,
// #2156: union of the repeatable CLI flags (one-off, this invocation
// only) and the source row's persisted config globs (every sync).
include: mergeGlobs(includePatterns, parseGlobList(sourceCfg.include_globs)),
exclude: mergeGlobs(excludePatterns, parseGlobList(sourceCfg.exclude_globs)),
exclude: excludePatterns.length > 0 ? excludePatterns : undefined,
signal: composeAbortSignals(singleSourceInterrupt.signal, singleSourceController?.signal),
};
@@ -5050,11 +4818,7 @@ export async function syncOneSource(
noExtract?: boolean;
},
): Promise<{ result: SyncResult; log: string }> {
const cfg = (src.config || {}) as {
strategy?: 'markdown' | 'code' | 'auto';
include_globs?: unknown;
exclude_globs?: unknown;
};
const cfg = (src.config || {}) as { strategy?: 'markdown' | 'code' | 'auto' };
const log = `\n--- Syncing source: ${src.name} ---\n`;
const repoOpts: SyncOpts = {
repoPath: src.local_path!,
@@ -5068,8 +4832,6 @@ export async function syncOneSource(
noSchemaPack: shared.noSchemaPack,
sourceId: src.id,
strategy: cfg.strategy,
include: parseGlobList(cfg.include_globs),
exclude: parseGlobList(cfg.exclude_globs),
concurrency: shared.concurrency,
// lockId defaults to `gbrain-sync:${src.id}` via the invariant in
// performSync (no explicit override needed — sourceId triggers it).
+63
View File
@@ -0,0 +1,63 @@
/**
* Nightly conversation-parser probe audit trail.
*
* One event per REAL probe run lands in
* `~/.gbrain/audit/parser-probe-YYYY-Www.jsonl` (ISO-week rotation via the
* shared audit-writer primitive; honors `GBRAIN_AUDIT_DIR`).
* Scheduler-cadence skips (`rate_limited`) are NOT logged — the autopilot
* loop ticks every few minutes, so logging every skip would flood the
* audit file with rows that carry no signal.
*
* Read by `gbrain doctor`'s `conversation_parser_probe_health` check and
* by the autopilot wiring's 24h rate-limit gate (`parserProbeRanWithin`).
*/
import { createAuditWriter } from './audit/audit-writer.ts';
import type { NightlyProbeResult } from './conversation-parser/nightly-probe.ts';
export type ParserProbeAuditEvent = NightlyProbeResult;
const writer = createAuditWriter<ParserProbeAuditEvent>({
featureName: 'parser-probe',
errorLabel: 'gbrain',
errorMessagePrefix: 'parser-probe audit ',
errorTrailer: '; probe continues',
});
/** Append one parser-probe event. Best-effort; never throws. */
export function logParserProbeEvent(event: ParserProbeAuditEvent): void {
writer.log(event);
}
/**
* Read recent parser-probe events (current + previous ISO week, filtered
* to the window). Missing files and corrupt rows are skipped silently.
*/
export function readRecentParserProbeEvents(
days = 7,
now: Date = new Date(),
): ParserProbeAuditEvent[] {
return writer.readRecent(days, now);
}
/** Exposed for tests pinning the rotation edge cases. */
export function computeParserProbeAuditFilename(now: Date = new Date()): string {
return writer.computeFilename(now);
}
/**
* 24h rate-limit gate for the autopilot wiring: true when any audited run
* happened within `windowMs` of `now`. Only REAL outcomes are audited (see
* module header), so a pass/fail today blocks re-runs until tomorrow while
* scheduler-cadence skips never extend the window.
*/
export function parserProbeRanWithin(
windowMs: number,
now: Date = new Date(),
): boolean {
const cutoff = now.getTime() - windowMs;
return readRecentParserProbeEvents(2, now).some((ev) => {
const ts = Date.parse(ev.ts);
return Number.isFinite(ts) && ts >= cutoff;
});
}
+10
View File
@@ -105,6 +105,16 @@ export interface GBrainConfig {
*/
max_usd?: number;
};
/**
* v0.41.16.0 — nightly conversation-parser probe. Per D10: default ON
* for `search.mode=tokenmax` brains, opt-in for conservative/balanced.
* ~$0.05/night with the committed fixtures × Haiku polish. Gated
* INSIDE the autopilot tick body, like nightly_quality_probe.
*/
conversation_parser_probe?: {
/** Enable for non-tokenmax modes. Defaults to false. */
enabled?: boolean;
};
/**
* v0.42.x (#1685 GAP D) — extract_atoms backlog auto-drain. Default ON so a
* pack-gated silent backlog never piles up unseen; daily-spend-capped so the
@@ -17,11 +17,11 @@
* Cost: ~$0.05/night with default fixtures × Haiku polish. Bounded
* by the active BudgetTracker the autopilot loop creates per-tick.
*
* **Wiring into the autopilot loop is deferred to a follow-up**
* (filed in TODOS.md). v0.41.16.0 ships the phase as a callable
* module so doctor + future cron drivers can invoke it; the
* scheduler wire-up follows the same shape as
* `src/core/cycle/nightly-quality-probe.ts` (v0.40.1.0 Track D / T6).
* Wired into the autopilot loop (step 4.6 in autopilot.ts), following
* the same shape as `src/core/cycle/nightly-quality-probe.ts`
* (v0.40.1.0 Track D / T6): the wiring resolves fixtures from the
* gbrain package root, writes real outcomes to the parser-probe audit
* trail (`audit-parser-probe.ts`), and never crashes the loop.
*
* Test seam: all dependencies are injected via NightlyProbeDeps so
* unit tests don't touch real LLMs or real fixtures.
+6 -1
View File
@@ -44,7 +44,12 @@ export const DEFAULT_DIMENSIONS: string[] = [
* `--slot-a-model`, `--slot-b-model`, `--slot-c-model` on the CLI.
*/
export const DEFAULT_SLOTS: SlotConfig[] = [
{ id: 'A', model: 'openai:gpt-4o' },
// Every default MUST be listed in its recipe's chat touchpoint (pinned by
// test/cross-modal-default-slots.test.ts) — `openai:gpt-4o` sat here after
// the OpenAI recipe dropped it, so slot A errored "not listed for OpenAI
// chat" on every install and the 3-slot panel could never reach its
// 2-model quorum without a Google key (verdict: permanently inconclusive).
{ id: 'A', model: 'openai:gpt-5.2' },
{ id: 'B', model: 'anthropic:claude-opus-4-7' },
{ id: 'C', model: 'google:gemini-1.5-pro' },
];
+24
View File
@@ -70,6 +70,28 @@ export async function runLongMemEvalForProbe(args: LongMemEvalProbeArgs): Promis
* the batch input) or unparseable (cross-modal wrote garbage). Both
* cases are paste-ready in the error message.
*/
/**
* QA-shaped judge dimensions for the nightly probe. The batch judge's
* DEFAULT_DIMENSIONS rubric (DEPTH / SOURCING / SPECIFICITY / …) is built
* for rich agent responses; LongMemEval hypotheses are deliberately terse
* factual answers ("in widget-co") that can never score ≥7 on DEPTH or
* SOURCING — so with the default rubric the probe FAILs every night even
* when retrieval + answering are perfectly healthy. The probe owns its
* invocation of the eval tool and passes dimensions matching the
* fixture's QA shape instead.
*
* NOTE: the `--dimensions` CLI flag splits on commas, so these dimension
* descriptions must stay comma-free.
*/
export const PROBE_QA_DIMENSIONS: string[] = [
// No faithfulness/grounding dimension on purpose: the judge never sees
// the haystack, so any accurate detail beyond the terse gold label reads
// as "invented" and correct answers fail (verified empirically — a
// correct "before + dates" answer scored 4/10 on such a dimension).
'CORRECTNESS — Does the hypothesis state the same fact as the expected answer? A terse direct answer is ideal.',
'DIRECTNESS — Does it answer THIS question without hedging or padding or answering something else?',
];
export async function runCrossModalBatchForProbe(
args: CrossModalProbeArgs,
): Promise<{ exitCode: number; summary: CrossModalBatchSummary }> {
@@ -81,6 +103,8 @@ export async function runCrossModalBatchForProbe(
args.summaryPath,
'--max-usd',
String(args.maxUsd),
'--dimensions',
PROBE_QA_DIMENSIONS.join(','),
'--yes',
'--json',
]);
+43 -11
View File
@@ -62,6 +62,42 @@ export interface NightlyProbeDeps {
now: () => Date;
}
/**
* Dual-plane flag resolution (same precedent as `mcp.publish_skills` in
* serve-http.ts): the DB config row — what `gbrain config set` writes —
* wins when present; the file plane (~/.gbrain/config.json) is the
* fallback. Doctor's paste-ready enable hint says `gbrain config set
* autopilot.nightly_quality_probe.enabled true`, so the gate MUST read
* the DB plane — a file-only read turns that hint into a silent no-op.
*/
export function resolveProbeEnabled(
dbVal: string | null | undefined,
fileVal: unknown,
): boolean {
if (dbVal != null) return dbVal === 'true';
return fileVal === true;
}
/**
* Same dual-plane rule for the per-run cost cap. Malformed or negative
* values on either plane fall through to the next plane / the default.
*/
export function resolveProbeMaxUsd(
dbVal: string | null | undefined,
fileVal: unknown,
fallback: number = DEFAULT_MAX_USD,
): number {
if (dbVal != null) {
const n = Number(dbVal);
if (Number.isFinite(n) && n >= 0) return n;
}
if (fileVal != null) {
const n = Number(fileVal);
if (Number.isFinite(n) && n >= 0) return n;
}
return fallback;
}
/**
* Pure function: decide whether the probe should run given the audit
* history. Returns reason when skipping.
@@ -101,21 +137,17 @@ export async function runNightlyQualityProbe(deps: NightlyProbeDeps): Promise<Ni
return { outcome: 'disabled', exit_code: 0, detail: 'feature flag off' };
}
// 24h rate limit — skip + audit "rate_limited".
// 24h rate limit — skip WITHOUT an audit row. The autopilot loop invokes
// the probe every cycle (~5-10 min), so all but one invocation per day
// lands here; logging each skip floods the audit file (~hundreds of
// rows/day) and — because doctor treats any non-pass outcome as bad
// signal — flips nightly_quality_probe_health to a permanent WARN the
// moment the probe is enabled. A skip is a non-event: the real runs are
// the signal, and their rows are what gates the next 24h window.
const now = deps.now();
const recent = readRecentQualityProbeEvents(2, now); // 2-day window is enough for 24h check
const decision = shouldRunNightly(now, recent);
if (!decision.run) {
logQualityProbeEvent({
outcome: 'rate_limited',
exit_code: 0,
pass_count: 0,
fail_count: 0,
inconclusive_count: 0,
error_count: 0,
est_cost_usd: 0,
detail: 'already ran within 24h window',
});
return { outcome: 'rate_limited', exit_code: 0, detail: 'already ran within 24h' };
}
-26
View File
@@ -5671,32 +5671,6 @@ export const MIGRATIONS: Migration[] = [
`);
},
},
{
version: 125,
name: 'sources_config_fingerprint',
// #2157 follow-on: the "Already up to date" gate at sync.ts honors
// git-HEAD equality + chunker-version match but ignored source-config
// drift. A user who runs `gbrain sources add default --exclude
// 'Templates/**'` AFTER an initial sync got "Already up to date" on
// the next pass because git HEAD was unchanged — the new exclusion
// never reached the walk until `gbrain sync --full`.
//
// This column caches a SHA-256 fingerprint of the walk-affecting
// fields in `sources.config` (strategy + include_globs +
// exclude_globs); mismatches trigger a full re-walk via the same code
// path as a chunker_version bump.
//
// NULL on pre-migration rows is treated as "not yet stamped" by
// readConfigFingerprint, so the FIRST sync after upgrade is normal
// (no spurious force-full just because the column was added).
//
// Keep in sync with src/schema.sql and src/core/schema-embedded.ts.
idempotent: true,
sql: `
ALTER TABLE sources
ADD COLUMN IF NOT EXISTS config_fingerprint TEXT;
`,
},
];
export const LATEST_VERSION = MIGRATIONS.length > 0
+5
View File
@@ -75,6 +75,11 @@ export const CANONICAL_PRICING: Record<string, ModelPricing> = {
'openai:gpt-4o': { input: 2.50, output: 10.00 },
'openai:gpt-4o-mini': { input: 0.15, output: 0.60 },
'openai:gpt-5': { input: 5.00, output: 20.00 },
// gpt-5.2: rates from the OpenAI recipe chat touchpoint (verified
// 2026-04-20). Needed here because it's the cross-modal DEFAULT_SLOTS
// slot-A model — without a canonical entry estimateCost silently drops
// slot A from the --max-usd pre-flight and est_cost_usd audit rows.
'openai:gpt-5.2': { input: 1.25, output: 10.00 },
'openai:gpt-5.5': { input: 4.00, output: 16.00 },
// ── Google ─────────────────────────────────────────────────────────────
-7
View File
@@ -39,13 +39,6 @@ CREATE TABLE IF NOT EXISTS sources (
-- bypassing the git-HEAD up_to_date early-return so CHUNKER_VERSION bumps
-- actually trigger re-chunking on upgrade.
chunker_version TEXT,
-- #2157 follow-on: SHA-256 fingerprint of the walk-affecting fields in
-- \`config\` (strategy + include_globs + exclude_globs). Mismatch forces a
-- full re-walk via the same code path as chunker_version, so a user who
-- changes \`sources.config.exclude_globs\` mid-life doesn't get "Already up
-- to date" on the next sync. NULL on pre-migration rows is treated as
-- "not yet stamped" and skips the gate (preserves first-run semantics).
config_fingerprint TEXT,
-- v0.26.5: soft-delete + recovery window. \`archive\` flips archived=true and
-- sets archive_expires_at = now() + 72h. The autopilot purge phase
-- hard-deletes rows where archive_expires_at <= now(). Promoted from a
-25
View File
@@ -155,19 +155,6 @@ export interface AddSourceOpts {
* runs). Does NOT auto-`git init` anything — see `addSource` docstring.
*/
force?: boolean;
/**
* Glob filters persisted into `sources.config.include_globs` /
* `sources.config.exclude_globs`. Read at sync time by
* `commands/sync.ts:syncOneSource` and the single-source path, threaded
* into `isSyncable` / `unsyncableReason` (their `SyncableOptions` shape
* has carried this contract since v0.41.13).
*
* Empty / unspecified arrays are not persisted at all (no `[]` written
* to the JSONB), which keeps the row identical to today for sources
* that don't use filtering.
*/
includeGlobs?: string[];
excludeGlobs?: string[];
}
export interface RemoveSourceOpts {
@@ -442,12 +429,6 @@ export async function addSource(
if (opts.federated !== null && opts.federated !== undefined) {
config.federated = opts.federated;
}
if (opts.includeGlobs && opts.includeGlobs.length > 0) {
config.include_globs = opts.includeGlobs;
}
if (opts.excludeGlobs && opts.excludeGlobs.length > 0) {
config.exclude_globs = opts.excludeGlobs;
}
const displayName = opts.name ?? opts.id;
try {
@@ -527,12 +508,6 @@ export async function addSource(
if (opts.federated !== null && opts.federated !== undefined) {
config.federated = opts.federated;
}
if (opts.includeGlobs && opts.includeGlobs.length > 0) {
config.include_globs = opts.includeGlobs;
}
if (opts.excludeGlobs && opts.excludeGlobs.length > 0) {
config.exclude_globs = opts.excludeGlobs;
}
const displayName = opts.name ?? opts.id;
await engine.executeRaw(
`INSERT INTO sources (id, name, local_path, config)
-7
View File
@@ -219,13 +219,6 @@ function globToRegex(pattern: string): RegExp {
return new RegExp(regex);
}
/**
* Test a normalized POSIX-style path against an array of glob patterns. Returns
* true if any pattern matches. Empty / undefined `patterns` returns false (no
* filter engaged). Exported so non-sync surfaces (lint walker, future ingest
* variants) can apply the same glob semantics as `isSyncable` without
* re-declaring `globToRegex`.
*/
export function matchesAnyGlob(path: string, patterns?: string[]): boolean {
if (!patterns || patterns.length === 0) return false;
const normalized = path.replace(/\\/g, '/');
-7
View File
@@ -35,13 +35,6 @@ CREATE TABLE IF NOT EXISTS sources (
-- bypassing the git-HEAD up_to_date early-return so CHUNKER_VERSION bumps
-- actually trigger re-chunking on upgrade.
chunker_version TEXT,
-- #2157 follow-on: SHA-256 fingerprint of the walk-affecting fields in
-- `config` (strategy + include_globs + exclude_globs). Mismatch forces a
-- full re-walk via the same code path as chunker_version, so a user who
-- changes `sources.config.exclude_globs` mid-life doesn't get "Already up
-- to date" on the next sync. NULL on pre-migration rows is treated as
-- "not yet stamped" and skips the gate (preserves first-run semantics).
config_fingerprint TEXT,
-- v0.26.5: soft-delete + recovery window. `archive` flips archived=true and
-- sets archive_expires_at = now() + 72h. The autopilot purge phase
-- hard-deletes rows where archive_expires_at <= now(). Promoted from a
+102
View File
@@ -0,0 +1,102 @@
/**
* Tests for the parser-probe audit trail + the 24h rate-limit gate.
*
* Uses GBRAIN_AUDIT_DIR override pointed at a tmpdir for hermeticity
* (same pattern as audit-slug-fallback.serial.test.ts). Serial because
* the env override is process-global.
*/
import { afterEach, beforeEach, describe, expect, test } from 'bun:test';
import { mkdtempSync, rmSync, readdirSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import {
computeParserProbeAuditFilename,
logParserProbeEvent,
parserProbeRanWithin,
readRecentParserProbeEvents,
type ParserProbeAuditEvent,
} from '../src/core/audit-parser-probe.ts';
let auditDir: string;
let savedEnv: string | undefined;
beforeEach(() => {
auditDir = mkdtempSync(join(tmpdir(), 'parser-probe-audit-'));
savedEnv = process.env.GBRAIN_AUDIT_DIR;
process.env.GBRAIN_AUDIT_DIR = auditDir;
});
afterEach(() => {
if (savedEnv === undefined) delete process.env.GBRAIN_AUDIT_DIR;
else process.env.GBRAIN_AUDIT_DIR = savedEnv;
rmSync(auditDir, { recursive: true, force: true });
});
function makeEvent(overrides: Partial<ParserProbeAuditEvent> = {}): ParserProbeAuditEvent {
return {
schema_version: 1,
ts: new Date().toISOString(),
outcome: 'pass',
fixtures_total: 12,
fixtures_passed: 12,
recall_mean: 0.98,
participants_recall_mean: 0.97,
adversarial_false_positives: 0,
failed_fixture_ids: [],
...overrides,
};
}
describe('parser-probe audit trail', () => {
test('log + readRecent round-trip', () => {
logParserProbeEvent(makeEvent({ outcome: 'fail', reason: '2 fixture(s) failed' }));
const events = readRecentParserProbeEvents(7);
expect(events.length).toBe(1);
expect(events[0]!.outcome).toBe('fail');
expect(events[0]!.reason).toBe('2 fixture(s) failed');
const files = readdirSync(auditDir);
expect(files.length).toBe(1);
expect(files[0]).toMatch(/^parser-probe-\d{4}-W\d{2}\.jsonl$/);
});
test('filename uses ISO-week rotation with the parser-probe prefix', () => {
// Year-boundary edge pinned by the shared writer's own tests; here we
// pin the prefix wiring.
expect(computeParserProbeAuditFilename(new Date('2026-07-06T12:00:00Z'))).toBe(
'parser-probe-2026-W28.jsonl',
);
});
test('readRecent filters by window', () => {
const old = new Date(Date.now() - 10 * 86400000).toISOString();
logParserProbeEvent(makeEvent({ ts: old }));
expect(readRecentParserProbeEvents(7).length).toBe(0);
});
});
describe('parserProbeRanWithin — 24h rate-limit gate', () => {
const DAY_MS = 24 * 60 * 60 * 1000;
test('false when no runs are audited', () => {
expect(parserProbeRanWithin(DAY_MS)).toBe(false);
});
test('true when a run landed within the window', () => {
logParserProbeEvent(makeEvent({ ts: new Date(Date.now() - 60_000).toISOString() }));
expect(parserProbeRanWithin(DAY_MS)).toBe(true);
});
test('false when the last run is older than the window', () => {
logParserProbeEvent(makeEvent({ ts: new Date(Date.now() - 25 * 3600_000).toISOString() }));
expect(parserProbeRanWithin(DAY_MS)).toBe(false);
});
test('non-pass outcomes also hold the window (mirrors quality-probe semantics)', () => {
logParserProbeEvent(makeEvent({
outcome: 'no_embedding_key',
ts: new Date(Date.now() - 3600_000).toISOString(),
}));
expect(parserProbeRanWithin(DAY_MS)).toBe(true);
});
});
+20 -4
View File
@@ -31,10 +31,15 @@ describe('autopilot wiring: nightly quality probe', () => {
expect(SOURCE).toContain(`runCrossModalBatchForProbe`);
});
test('feature flag gate present: cfg.autopilot.nightly_quality_probe.enabled', () => {
test('feature flag gate present: dual-plane read (DB row wins, file plane fallback)', () => {
// Per D10: the scheduler ONLY checks the feature flag. The 24h rate-limit
// lives inside runNightlyQualityProbe itself (no scheduler-side precheck).
expect(SOURCE).toContain(`nightly_quality_probe?.enabled === true`);
// The flag resolves through resolveProbeEnabled so `gbrain config set
// autopilot.nightly_quality_probe.enabled true` (the doctor hint, DB
// plane) and ~/.gbrain/config.json (file plane) BOTH work — a file-only
// read made the printed hint a silent no-op.
expect(SOURCE).toContain(`getConfig('autopilot.nightly_quality_probe.enabled')`);
expect(SOURCE).toMatch(/resolveProbeEnabled\(dbEnabled,\s*cfg\?\.autopilot\?\.nightly_quality_probe\?\.enabled\)/);
});
test('NO scheduler-side rate-limit check (D10 simplification)', () => {
@@ -64,12 +69,23 @@ describe('autopilot wiring: nightly quality probe', () => {
expect(SOURCE).toContain(`now:`);
});
test('resolveRepoRoot prefers the gbrain package root (committed fixture home), not the brain repoPath', () => {
// The DI harness in nightly-quality-probe.test.ts passes process.cwd()
// (= the gbrain repo in CI), which papered over the wiring passing
// repoPath (= sync.repo_path, the user's BRAIN repo, where the fixture
// never exists). Pin the package-root resolution + existence check.
expect(SOURCE).toMatch(/fileURLToPath\(new URL\('\.\.\/\.\.', import\.meta\.url\)\)/);
expect(SOURCE).toContain(`'longmemeval-nightly.jsonl'`);
expect(SOURCE).toMatch(/fixtureAtPkgRoot \? pkgRoot : repoPath/);
});
test('hasEmbeddingProvider reads from gateway.isAvailable("embedding") (codex round-2 #12 — in-process, not subprocess)', () => {
expect(SOURCE).toContain(`isAvailable('embedding')`);
expect(SOURCE).toContain(`gateway`);
});
test('max_usd default = 5 when config unset (matches plan default per D10)', () => {
expect(SOURCE).toMatch(/max_usd\s*\?\?\s*5/);
test('max_usd resolves dual-plane (default = 5 pinned by resolveProbeMaxUsd unit tests)', () => {
expect(SOURCE).toContain(`getConfig('autopilot.nightly_quality_probe.max_usd')`);
expect(SOURCE).toMatch(/resolveProbeMaxUsd\(dbMaxUsd,\s*cfg\?\.autopilot\?\.nightly_quality_probe\?\.max_usd\)/);
});
});
@@ -0,0 +1,77 @@
/**
* Source-shape regression tests for the autopilot wiring of
* `runConversationParserNightlyProbe` (step 4.6).
*
* Same rationale as autopilot-nightly-probe-wiring.test.ts: the loop is
* hard to drive end-to-end, so these pin the structural protections —
* the dual-plane flag read, the D10 tokenmax mode-gate, the package-root
* fixture resolution, the audit-flood guard, and the try/catch posture.
*
* The probe's own gate/scoring logic is pinned by the module's unit
* tests; the audit trail by audit-parser-probe.serial.test.ts.
*/
import { describe, test, expect } from 'bun:test';
import { readFileSync } from 'node:fs';
import { resolve } from 'node:path';
const AUTOPILOT_SRC = resolve('src/commands/autopilot.ts');
const SOURCE = readFileSync(AUTOPILOT_SRC, 'utf-8');
describe('autopilot wiring: conversation-parser probe', () => {
test('invokes the phase module and the audit trail', () => {
expect(SOURCE).toContain(`runConversationParserNightlyProbe`);
expect(SOURCE).toContain(`conversation-parser/nightly-probe`);
expect(SOURCE).toContain(`logParserProbeEvent`);
expect(SOURCE).toContain(`audit-parser-probe`);
});
test('flag reads dual-plane: DB row (gbrain config set) wins, file plane fallback', () => {
expect(SOURCE).toContain(`getConfig('autopilot.conversation_parser_probe.enabled')`);
expect(SOURCE).toContain(`cfg?.autopilot?.conversation_parser_probe?.enabled === true`);
});
test('D10 mode-gate present: tokenmax brains run the probe by default', () => {
expect(SOURCE).toMatch(/parserEnabled \|\| searchMode === 'tokenmax'/);
});
test('fixtures resolve from the gbrain package root, NOT the brain repoPath', () => {
// The committed fixtures live in the gbrain source tree; resolving
// them against sync.repo_path would point into the user's brain repo.
expect(SOURCE).toMatch(/fileURLToPath\(new URL\('\.\.\/\.\.', import\.meta\.url\)\)/);
expect(SOURCE).toContain(`'conversation-formats', 'all.jsonl'`);
expect(SOURCE).toContain(`'conversation-formats', 'adversarial.jsonl'`);
});
test('missing fixtures skip quietly (no audit row, once-per-process stderr note)', () => {
// Compiled-binary installs carry no source tree; writing failure rows
// would flip doctor to WARN on every binary install.
expect(SOURCE).toContain(`parserProbeFixtureWarned`);
});
test('rate_limited outcomes are NOT audit-logged (flood guard)', () => {
expect(SOURCE).toMatch(/outcome !== 'rate_limited'\) logParserProbeEvent\(result\)/);
});
test('rate-limit gate delegates to the audit module, not inline event reads', () => {
expect(SOURCE).toContain(`parserProbeRanWithin(24 * 60 * 60 * 1000)`);
});
test('LLM-key gate reads gateway.isAvailable("chat") in-process', () => {
expect(SOURCE).toContain(`isAvailable('chat')`);
});
test('probe call wrapped in try/catch that does NOT bump consecutiveErrors', () => {
expect(SOURCE).toMatch(/catch[\s\S]*?autopilot\.parser_probe[\s\S]*?do NOT bump consecutiveErrors/);
});
test('DI shape: the exact 7 fields of the parser probe NightlyProbeDeps', () => {
expect(SOURCE).toContain(`isEnabled:`);
expect(SOURCE).toContain(`searchMode:`);
expect(SOURCE).toContain(`hasLlmKey:`);
expect(SOURCE).toContain(`resolveFixturePath:`);
expect(SOURCE).toContain(`resolveAdversarialPath:`);
expect(SOURCE).toContain(`shouldSkipForRateLimit:`);
expect(SOURCE).toContain(`now:`);
});
});
+47
View File
@@ -0,0 +1,47 @@
/**
* Consistency guard: every cross-modal DEFAULT_SLOTS model must be listed
* in its recipe's chat touchpoint. `openai:gpt-4o` drifted out of the
* OpenAI recipe while remaining the slot-A default — the gateway then
* rejected slot A ("not listed for OpenAI chat") on every install, and the
* 3-slot judge panel could never reach its 2-model quorum without a Google
* key, pinning every batch verdict at inconclusive (which the nightly
* quality probe surfaces as a doctor WARN).
*/
import { describe, expect, test } from 'bun:test';
import { DEFAULT_SLOTS } from '../src/core/cross-modal-eval/runner.ts';
import { getRecipe } from '../src/core/ai/recipes/index.ts';
import { splitProviderModelId } from '../src/core/model-id.ts';
import { canonicalLookup } from '../src/core/model-pricing.ts';
describe('cross-modal DEFAULT_SLOTS ↔ recipe consistency', () => {
test('every default slot model is listed in its recipe chat touchpoint', () => {
for (const slot of DEFAULT_SLOTS) {
const { provider, model } = splitProviderModelId(slot.model);
expect(provider).not.toBeNull();
const recipe = getRecipe(provider!);
expect(recipe, `slot ${slot.id}: unknown recipe "${provider}"`).toBeDefined();
const chatModels = recipe!.touchpoints.chat?.models ?? [];
expect(
chatModels,
`slot ${slot.id}: "${model}" not listed for ${provider} chat — the judge slot can never run`,
).toContain(model);
}
});
test('every default slot model has a canonical pricing entry', () => {
// Without one, estimateCost silently drops the slot from the
// --max-usd pre-flight and est_cost_usd audit rows (~1/3 under-count).
for (const slot of DEFAULT_SLOTS) {
expect(
canonicalLookup(slot.model),
`slot ${slot.id}: "${slot.model}" missing from CANONICAL_PRICING`,
).toBeDefined();
}
});
test('slots span three distinct providers (uncorrelated blind spots)', () => {
const providers = new Set(DEFAULT_SLOTS.map(s => splitProviderModelId(s.model).provider));
expect(providers.size).toBe(3);
});
});
+51
View File
@@ -0,0 +1,51 @@
/**
* Tests for computeConversationParserProbeHealthCheck — the pure function
* behind doctor's conversation_parser_probe_health check, which replaced
* the v0.41.13.0 hardcoded "Skipped" stub when the autopilot wiring
* landed. Mirrors the branch coverage style of the quality-probe check.
*/
import { describe, expect, test } from 'bun:test';
import { computeConversationParserProbeHealthCheck } from '../src/commands/doctor.ts';
const ev = (outcome: string, reason?: string, ts = new Date().toISOString()) => ({
outcome,
ts,
...(reason !== undefined ? { reason } : {}),
});
describe('computeConversationParserProbeHealthCheck', () => {
test('disabled + no events → ok with paste-ready enable hint', () => {
const check = computeConversationParserProbeHealthCheck(false, []);
expect(check.status).toBe('ok');
expect(check.message).toContain('gbrain config set autopilot.conversation_parser_probe.enabled true');
});
test('enabled + no events yet → ok, next run by autopilot', () => {
const check = computeConversationParserProbeHealthCheck(true, []);
expect(check.status).toBe('ok');
expect(check.message).toContain('no probe events');
});
test('disabled flag but events exist (tokenmax mode-gate ran it) → events win over the hint', () => {
const check = computeConversationParserProbeHealthCheck(false, [ev('pass')]);
expect(check.status).toBe('ok');
expect(check.message).toContain('all pass');
});
test('any non-pass outcome in the window → warn, latest surfaced with reason', () => {
const check = computeConversationParserProbeHealthCheck(true, [
ev('pass'),
ev('adversarial_false_positive', '1 adversarial fixture(s) parsed to non-empty'),
]);
expect(check.status).toBe('warn');
expect(check.message).toContain('adversarial_false_positive');
expect(check.message).toContain('parsed to non-empty');
});
test('all pass → ok with run count', () => {
const check = computeConversationParserProbeHealthCheck(true, [ev('pass'), ev('pass')]);
expect(check.status).toBe('ok');
expect(check.message).toContain('2 probe run(s)');
});
});
-153
View File
@@ -1,153 +0,0 @@
/**
* `gbrain lint` source-glob filter — walker integration.
*
* PR #2157 (commit cf9a3b18, `feat/sync-source-glob-filters`) wired
* `sources.config.include_globs` / `exclude_globs` into `gbrain sync` so a
* user could exclude `Resources/veriff/**` and have every subsequent sync
* honor it. The lint command walked the same source dirs blind and emitted
* findings against paths the user had already declared out of scope — a
* half-finished feature.
*
* This patch extends the same persisted glob contract to lint:
* - `gbrain lint` gains `--include / --exclude` flags (parallel to sync).
* - `runLintCore` lifts `sources.config.{include,exclude}_globs` for any
* target whose absolute path matches a source row's `local_path`, so the
* cycle.lint phase + Minion lint handlers honor the same filter without
* restating it.
* - The walker in `collectPages` applies the filter using the SAME
* `matchesAnyGlob` helper sync uses, anchored at the target dir (so a
* persisted `Resources/veriff/**` glob written against the source root
* works without rewriting it as an absolute path).
*
* These tests pin the walker contract. The engine-side lift
* (`resolveSourceGlobsForTarget`) is best-effort by design (returns `{}` on
* any error) and is exercised by the dream-cycle lint phase end-to-end.
*/
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'fs';
import { join } from 'path';
import { tmpdir } from 'os';
// runLintCore is the library entry — the same surface the cycle.lint phase
// and Minion handlers call. Exercising it covers the walker via its real
// callsite; testing `collectPages` directly would skip the wiring.
import { runLintCore } from '../src/commands/lint.ts';
// A self-contained content-sanity stub so the test never touches a real
// engine / config file. Empty operator-literal list keeps the content-sanity
// pass silent so the only findings come from the structural rules
// (no-frontmatter etc.).
const STUB_CS = {
fail_on_throw: false,
warn_on_throw: false,
bytes_warn: 1024 * 1024,
operator_literals: [],
};
describe('runLintCore — source-glob walker filter', () => {
let root: string;
beforeAll(() => {
root = mkdtempSync(join(tmpdir(), 'gbrain-lint-globs-'));
// Three subtrees with mixed structured / archive-style content.
// All pages have `# Title` headers but no frontmatter so each one
// emits at least one `no-frontmatter` issue under the default rule set.
mkdirSync(join(root, 'Notes'), { recursive: true });
mkdirSync(join(root, 'Resources', 'veriff'), { recursive: true });
mkdirSync(join(root, 'Resources', 'prior-art', 'archive-v1'), { recursive: true });
writeFileSync(join(root, 'Notes', 'a.md'), '# A\nbody\n');
writeFileSync(join(root, 'Notes', 'b.md'), '# B\nbody\n');
writeFileSync(join(root, 'Resources', 'veriff', 'spec-1.md'), '# Veriff spec 1\nbody\n');
writeFileSync(join(root, 'Resources', 'veriff', 'spec-2.md'), '# Veriff spec 2\nbody\n');
writeFileSync(join(root, 'Resources', 'prior-art', 'archive-v1', 'old.md'), '# Old\nbody\n');
});
afterAll(() => {
rmSync(root, { recursive: true, force: true });
});
test('no filter — walks every .md (regression guard for default behavior)', async () => {
const result = await runLintCore({
target: root,
contentSanity: STUB_CS,
});
expect(result.pages_scanned).toBe(5);
expect(result.pages_with_issues).toBeGreaterThan(0);
});
test('exclude glob skips matching paths (Resources/veriff/** off-limits)', async () => {
const result = await runLintCore({
target: root,
contentSanity: STUB_CS,
exclude: ['Resources/veriff/**'],
});
// 5 total minus 2 veriff specs = 3 pages walked.
expect(result.pages_scanned).toBe(3);
});
test('exclude with multiple patterns is union (veriff + prior-art both skipped)', async () => {
const result = await runLintCore({
target: root,
contentSanity: STUB_CS,
exclude: ['Resources/veriff/**', 'Resources/prior-art/**'],
});
// 5 total minus 3 (2 veriff + 1 archive-v1) = 2 pages walked.
expect(result.pages_scanned).toBe(2);
});
test('include glob narrows the walk to matching paths only', async () => {
const result = await runLintCore({
target: root,
contentSanity: STUB_CS,
include: ['Notes/**'],
});
expect(result.pages_scanned).toBe(2);
});
test('exclude runs AFTER include (same precedence as `gbrain sync`)', async () => {
const result = await runLintCore({
target: root,
contentSanity: STUB_CS,
include: ['**/*.md'],
exclude: ['Resources/**'],
});
// include lets everything through; exclude drops the 3 Resources/* files.
expect(result.pages_scanned).toBe(2);
});
test('empty include / exclude arrays do NOT engage the filter', async () => {
// Symmetric with `parseGlobList` returning undefined for empty input —
// an empty include would otherwise classify every path as a miss and
// silently zero out the lint scope. Pin the guard at the walker level.
const result = await runLintCore({
target: root,
contentSanity: STUB_CS,
include: [],
exclude: [],
});
expect(result.pages_scanned).toBe(5);
});
test('exclude semantics match sync — `**` matches across path segments', async () => {
const result = await runLintCore({
target: root,
contentSanity: STUB_CS,
exclude: ['**/spec-*.md'],
});
// Both Veriff specs match the deep glob; Notes + archive-v1 survive.
expect(result.pages_scanned).toBe(3);
});
test('single-file target bypasses the filter (file mode is not a walk)', async () => {
// A user lints one .md explicitly: filters are a directory-walk concern,
// so the file is processed even if its name would match an exclude.
const result = await runLintCore({
target: join(root, 'Resources', 'veriff', 'spec-1.md'),
contentSanity: STUB_CS,
exclude: ['Resources/veriff/**'],
});
expect(result.pages_scanned).toBe(1);
});
});
+74
View File
@@ -0,0 +1,74 @@
// Regression test for the nightly-quality-probe config-plane split-brain.
//
// The doctor check prints a paste-ready enable hint — `gbrain config set
// autopilot.nightly_quality_probe.enabled true` — which writes the DB config
// plane. But both the autopilot gate and the doctor check used to read ONLY
// the file plane (~/.gbrain/config.json via loadConfig), so following the
// hint was a silent no-op: the probe never ran and doctor kept reporting
// "disabled (opt-in)".
//
// resolveProbeEnabled / resolveProbeMaxUsd pin the dual-plane rule (same
// precedent as `mcp.publish_skills` in serve-http.ts): DB row wins when
// present, file plane is the fallback.
import { describe, expect, test } from 'bun:test';
import {
resolveProbeEnabled,
resolveProbeMaxUsd,
} from '../src/core/cycle/nightly-quality-probe.ts';
describe('resolveProbeEnabled — dual-plane flag resolution', () => {
test('DB plane "true" enables regardless of file plane (the doctor hint path)', () => {
expect(resolveProbeEnabled('true', undefined)).toBe(true);
expect(resolveProbeEnabled('true', false)).toBe(true);
});
test('explicit DB "false" wins over file-plane true (config set off sticks)', () => {
expect(resolveProbeEnabled('false', true)).toBe(false);
});
test('file plane is the fallback when no DB row exists', () => {
expect(resolveProbeEnabled(null, true)).toBe(true);
expect(resolveProbeEnabled(undefined, true)).toBe(true);
expect(resolveProbeEnabled(null, undefined)).toBe(false);
expect(resolveProbeEnabled(null, false)).toBe(false);
});
test('file plane stays strict boolean — string "true" in config.json does not enable', () => {
// Matches the pre-fix autopilot gate (`=== true`); the doctor check used
// Boolean(...) and could disagree with autopilot on a string value.
// Both call sites now share this helper, so they can no longer diverge.
expect(resolveProbeEnabled(null, 'true')).toBe(false);
expect(resolveProbeEnabled(null, 1)).toBe(false);
});
test('non-"true" DB strings are off (mcp.publish_skills semantics)', () => {
expect(resolveProbeEnabled('1', true)).toBe(false);
expect(resolveProbeEnabled('yes', true)).toBe(false);
expect(resolveProbeEnabled('', true)).toBe(false);
});
});
describe('resolveProbeMaxUsd — dual-plane cost cap resolution', () => {
test('DB plane wins when parseable', () => {
expect(resolveProbeMaxUsd('2.5', 10)).toBe(2.5);
expect(resolveProbeMaxUsd('0', 10)).toBe(0);
});
test('malformed or negative DB value falls through to file plane', () => {
expect(resolveProbeMaxUsd('banana', 3)).toBe(3);
expect(resolveProbeMaxUsd('-1', 3)).toBe(3);
});
test('file plane used when no DB row; default when both absent/invalid', () => {
expect(resolveProbeMaxUsd(null, 7)).toBe(7);
expect(resolveProbeMaxUsd(null, '4')).toBe(4);
expect(resolveProbeMaxUsd(null, undefined)).toBe(5);
expect(resolveProbeMaxUsd(null, 'banana')).toBe(5);
expect(resolveProbeMaxUsd(undefined, -2)).toBe(5);
});
test('explicit fallback override is honored', () => {
expect(resolveProbeMaxUsd(null, undefined, 12)).toBe(12);
});
});
+7 -4
View File
@@ -132,17 +132,20 @@ describe('runNightlyQualityProbe (DI stub harness)', () => {
});
});
test('enabled + recent run within 24h → outcome: rate_limited', async () => {
test('enabled + recent run within 24h → outcome: rate_limited, NO audit row', async () => {
// Pre-seed a recent audit event by running the probe once first.
await withEnv({ GBRAIN_AUDIT_DIR: auditTmp }, async () => {
// First run succeeds.
await runNightlyQualityProbe(makeDeps());
// Second run, same hour → rate_limited.
// Second run, same hour → rate_limited. A skip is a non-event: the
// autopilot loop invokes the probe every cycle (~5-10 min), so
// logging each skip would flood the audit file and flip doctor's
// any-non-pass-is-bad filter to a permanent WARN.
const r2 = await runNightlyQualityProbe(makeDeps());
expect(r2.outcome).toBe('rate_limited');
const events = await readEvents();
expect(events.length).toBe(2);
expect(events[1].outcome).toBe('rate_limited');
expect(events.length).toBe(1);
expect(events[0].outcome).toBe('pass');
});
});
-6
View File
@@ -694,12 +694,6 @@ const COLUMN_EXEMPTIONS = new Set<string>([
'minion_jobs.quiet_hours',
'minion_jobs.stagger_key',
'sources.chunker_version',
// #2157 follow-on (migration v125). TEXT column read by performSync's
// `Already up to date` gate; not referenced by any CREATE INDEX. Same
// upgrade-path coverage as sources.chunker_version above: fresh installs
// get it via the CREATE TABLE in src/schema.sql + schema-embedded.ts;
// pre-existing brains get it via the idempotent ALTER TABLE in v125.
'sources.config_fingerprint',
'access_tokens.permissions',
'takes.resolved_quality',
'pages.emotional_weight_recomputed_at',
-121
View File
@@ -159,127 +159,6 @@ describe('sources add', () => {
await expect(runSources(engine, ['add', 'plans', '--path', '/tmp/gstack/plans']))
.rejects.toThrow(/overlaps with existing source "gstack"/);
});
// Glob filters — TODO #3 from the brettdavies fork recon. Pre-fix, the
// `SyncableOptions` shape in `src/core/sync.ts` had been carrying
// `include` / `exclude` since v0.41.13, but commands/sync.ts:1454 never
// populated them and `sources add` had no flag to persist them — so users
// had no way to tell gbrain to skip `Templates/` in an Obsidian vault.
test('--exclude persists glob into sources.config.exclude_globs', async () => {
const { engine, calls } = makeStub({
'SELECT id, name, local_path, last_commit, last_sync_at, config, created_at': [{
id: 'vault',
name: 'vault',
local_path: '/tmp/vault',
last_commit: null,
last_sync_at: null,
config: '{"exclude_globs":["Templates/**"]}',
created_at: new Date(),
}],
});
await runSources(engine, ['add', 'vault', '--path', '/tmp/vault', '--exclude', 'Templates/**']);
const insert = calls.find(c => c.sql.includes('INSERT INTO sources'));
expect(insert!.params[3]).toBe('{"exclude_globs":["Templates/**"]}');
});
test('--include persists glob into sources.config.include_globs', async () => {
const { engine, calls } = makeStub({
'SELECT id, name, local_path, last_commit, last_sync_at, config, created_at': [{
id: 'wiki',
name: 'wiki',
local_path: '/tmp/wiki',
last_commit: null,
last_sync_at: null,
config: '{"include_globs":["people/**"]}',
created_at: new Date(),
}],
});
await runSources(engine, ['add', 'wiki', '--path', '/tmp/wiki', '--include', 'people/**']);
const insert = calls.find(c => c.sql.includes('INSERT INTO sources'));
expect(insert!.params[3]).toBe('{"include_globs":["people/**"]}');
});
test('--exclude is repeatable; preserves order', async () => {
const { engine, calls } = makeStub({
'SELECT id, name, local_path, last_commit, last_sync_at, config, created_at': [{
id: 'vault',
name: 'vault',
local_path: '/tmp/vault',
last_commit: null,
last_sync_at: null,
config: '{}',
created_at: new Date(),
}],
});
await runSources(engine, [
'add', 'vault', '--path', '/tmp/vault',
'--exclude', 'Templates/**',
'--exclude', '.smart-env/**',
'--exclude', 'Drafts/**',
]);
const insert = calls.find(c => c.sql.includes('INSERT INTO sources'));
expect(insert!.params[3]).toBe('{"exclude_globs":["Templates/**",".smart-env/**","Drafts/**"]}');
});
test('--include and --exclude compose in one command (federated source with both filter axes)', async () => {
const { engine, calls } = makeStub({
'SELECT id, name, local_path, last_commit, last_sync_at, config, created_at': [{
id: 'vault',
name: 'vault',
local_path: '/tmp/vault',
last_commit: null,
last_sync_at: null,
config: '{"federated":true,"include_globs":["people/**"],"exclude_globs":["Templates/**"]}',
created_at: new Date(),
}],
});
await runSources(engine, [
'add', 'vault', '--path', '/tmp/vault', '--federated',
'--include', 'people/**',
'--exclude', 'Templates/**',
]);
const insert = calls.find(c => c.sql.includes('INSERT INTO sources'));
expect(insert!.params[3]).toBe(
'{"federated":true,"include_globs":["people/**"],"exclude_globs":["Templates/**"]}',
);
});
test('omitted glob flags leave config untouched (no [] entries persisted)', async () => {
// Regression guard: empty glob arrays must NOT be written. Otherwise a
// brain that never opts into filtering grows {"include_globs": [],
// "exclude_globs": []} cruft in every source row, and the parseGlobList
// path would return undefined anyway (the cruft is purely noise).
const { engine, calls } = makeStub({
'SELECT id, name, local_path, last_commit, last_sync_at, config, created_at': [{
id: 'gstack',
name: 'gstack',
local_path: '/tmp/gstack',
last_commit: null,
last_sync_at: null,
config: '{}',
created_at: new Date(),
}],
});
await runSources(engine, ['add', 'gstack', '--path', '/tmp/gstack']);
const insert = calls.find(c => c.sql.includes('INSERT INTO sources'));
expect(insert!.params[3]).toBe('{}');
});
test('--exclude requires a glob argument', async () => {
const { engine } = makeStub();
const code = await withExitCapture(() => runSources(engine, [
'add', 'vault', '--path', '/tmp/vault', '--exclude',
]));
expect(code).toBe(2);
});
test('--include rejects a flag-like value (--include --path looks like a typo)', async () => {
const { engine } = makeStub();
const code = await withExitCapture(() => runSources(engine, [
'add', 'vault', '--path', '/tmp/vault', '--include', '--federated',
]));
expect(code).toBe(2);
});
});
// ── add — #2707 git-repo validation (CLI wiring) ───────────────
-95
View File
@@ -1,95 +0,0 @@
/**
* #2157 follow-on (migration v125) end-to-end gate wiring.
*
* `test/sync-config-fingerprint.test.ts` pins the persistence + comparison
* primitives (compute/read/write). This file pins the WIRING inside
* `performSync`: with git HEAD unchanged, a drift in the walk-affecting
* `sources.config` fields must break out of the "Already up to date" early
* return and force a full re-walk and the re-stamped fingerprint must
* settle the gate back to `up_to_date` on the following pass. Deleting the
* `configMismatch` term from the gate condition fails this test; none of the
* primitive tests would catch that.
*/
import { test, expect, beforeAll, afterAll } from 'bun:test';
import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'fs';
import { join } from 'path';
import { tmpdir } from 'os';
import { execFileSync } from 'child_process';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { performSync } from '../src/commands/sync.ts';
let engine: PGLiteEngine;
let repoPath: string;
function git(cwd: string, ...args: string[]) {
execFileSync('git', args, { cwd, stdio: 'pipe' });
}
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
repoPath = mkdtempSync(join(tmpdir(), 'gbrain-fp-gate-'));
mkdirSync(join(repoPath, 'wiki'));
mkdirSync(join(repoPath, 'memory'));
writeFileSync(join(repoPath, 'wiki', 'page1.md'), '# Page 1\n\nbody\n');
writeFileSync(join(repoPath, 'memory', 'note1.md'), '# Note 1\n\nbody\n');
git(repoPath, 'init');
git(repoPath, 'add', '-A');
git(repoPath, '-c', 'user.email=t@example.com', '-c', 'user.name=t', 'commit', '-m', 'init');
await engine.executeRaw(
`INSERT INTO sources (id, name, local_path, config) VALUES ($1, $2, $3, $4::text::jsonb)`,
['vault', 'vault', repoPath, JSON.stringify({ include_globs: ['wiki/**'] })],
);
}, 60_000);
afterAll(async () => {
await engine?.disconnect();
rmSync(repoPath, { recursive: true, force: true });
});
test('config-glob drift with unchanged git HEAD forces a re-walk, then settles', async () => {
// First sync: row config include_globs = ['wiki/**'], caller threads it
// (as syncOneSource / the single-source CLI path do). memory/* skipped.
const first = await performSync(engine, {
repoPath, sourceId: 'vault', include: ['wiki/**'],
noPull: true, noEmbed: true, full: true,
});
expect(first.status).toBe('first_sync');
expect(await engine.getPage('wiki/page1')).not.toBeNull();
expect(await engine.getPage('memory/note1')).toBeNull();
// No drift, HEAD unchanged: gate stays quiet.
const second = await performSync(engine, {
repoPath, sourceId: 'vault', include: ['wiki/**'],
noPull: true, noEmbed: true,
});
expect(second.status).toBe('up_to_date');
// User widens the persisted globs (what `gbrain sources add --include`
// writes). Git HEAD has NOT moved.
await engine.executeRaw(
`UPDATE sources SET config = $1::text::jsonb WHERE id = $2`,
[JSON.stringify({ include_globs: ['wiki/**', 'memory/**'] }), 'vault'],
);
// Pre-fix this returned `up_to_date` (HEAD unchanged) and memory/note1
// stayed missing until a manual `--full`. The fingerprint gate must force
// the full re-walk instead.
const third = await performSync(engine, {
repoPath, sourceId: 'vault', include: ['wiki/**', 'memory/**'],
noPull: true, noEmbed: true,
});
expect(third.status).not.toBe('up_to_date');
expect(await engine.getPage('memory/note1')).not.toBeNull();
// Re-stamped fingerprint matches the current row: gate settles.
const fourth = await performSync(engine, {
repoPath, sourceId: 'vault', include: ['wiki/**', 'memory/**'],
noPull: true, noEmbed: true,
});
expect(fourth.status).toBe('up_to_date');
});
-234
View File
@@ -1,234 +0,0 @@
/**
* #2157 follow-on (migration v125 `sources.config_fingerprint`).
*
* The "Already up to date" gate at performSync's git-HEAD equality check
* honored chunker_version match but ignored `sources.config` drift.
* Changing `sources.config.exclude_globs` (or include_globs / strategy)
* had no observable effect on the next sync because git HEAD was
* unchanged the gate returned early and the new walk scope never
* applied. This file exercises the persistence shape + drift detection
* end-to-end on PGLite, including:
*
* - Migration v125 actually adds the column (regression guard against
* a future re-numbering or accidental deletion).
* - read/write round-trips preserve the value.
* - The fingerprint differs across the three walk-affecting fields
* and is order-insensitive on the array fields.
* - NULL fingerprint on pre-v125 rows treats as "not stamped" so a
* first post-upgrade sync doesn't spuriously force-full.
* - A toggle-and-revert leaves the stored fingerprint matching the
* current row, so the gate stays quiet.
*
* The wired-up gate behavior (force-full triggered on mismatch) is
* exercised by the existing sync end-to-end tests; here we pin the
* persistence + comparison primitives the gate depends on.
*/
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import {
computeSourceConfigFingerprint,
readConfigFingerprint,
writeConfigFingerprint,
} from '../src/commands/sync.ts';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
let engine: PGLiteEngine;
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
}, 60_000);
afterAll(async () => {
await engine?.disconnect();
});
/** Insert a fresh source row with the given config. Returns the id. */
async function makeSource(
id: string,
config: Record<string, unknown> = {},
): Promise<string> {
await engine.executeRaw(
`INSERT INTO sources (id, name, local_path, config) VALUES ($1, $2, $3, $4::text::jsonb)`,
[id, id, `/tmp/${id}`, JSON.stringify(config)],
);
return id;
}
describe('migration v125 — sources.config_fingerprint column', () => {
test('column exists on the sources table', async () => {
const rows = await engine.executeRaw<{ column_name: string }>(
`SELECT column_name FROM information_schema.columns
WHERE table_name = 'sources' AND column_name = 'config_fingerprint'`,
);
expect(rows).toHaveLength(1);
});
test('column is nullable (preserves pre-migration row semantics)', async () => {
const rows = await engine.executeRaw<{ is_nullable: string }>(
`SELECT is_nullable FROM information_schema.columns
WHERE table_name = 'sources' AND column_name = 'config_fingerprint'`,
);
expect(rows[0]?.is_nullable).toBe('YES');
});
});
describe('readConfigFingerprint / writeConfigFingerprint — persistence round-trip', () => {
test('round-trip: write then read returns the same value', async () => {
const id = await makeSource('rt-basic', { exclude_globs: ['Templates/**'] });
const fp = computeSourceConfigFingerprint({ exclude_globs: ['Templates/**'] });
await writeConfigFingerprint(engine, id, fp);
const got = await readConfigFingerprint(engine, id);
expect(got).toBe(fp);
});
test('NULL on never-stamped row (pre-v125 semantics)', async () => {
const id = await makeSource('rt-never');
const got = await readConfigFingerprint(engine, id);
expect(got).toBeNull();
});
test('undefined sourceId returns null (legacy non-source-scoped sync)', async () => {
const got = await readConfigFingerprint(engine, undefined);
expect(got).toBeNull();
});
test('write with undefined sourceId is a no-op (does not throw)', async () => {
// The legacy global-sync code path hits this branch; the guard must
// be silent rather than fail the sync run.
await writeConfigFingerprint(engine, undefined, 'deadbeef'.repeat(8));
// No assertion beyond "did not throw"; the function returns void.
});
test('overwrite: a second write replaces the prior fingerprint', async () => {
const id = await makeSource('rt-overwrite');
await writeConfigFingerprint(engine, id, 'a'.repeat(64));
await writeConfigFingerprint(engine, id, 'b'.repeat(64));
const got = await readConfigFingerprint(engine, id);
expect(got).toBe('b'.repeat(64));
});
});
describe('end-to-end drift simulation — the gate semantics this column enables', () => {
test('first stamp matches computed fingerprint of the row config', async () => {
const cfg = { exclude_globs: ['Templates/**', 'Photos/**'], strategy: 'markdown' };
const id = await makeSource('e2e-first-stamp', cfg);
const computed = computeSourceConfigFingerprint(cfg);
await writeConfigFingerprint(engine, id, computed);
expect(await readConfigFingerprint(engine, id)).toBe(computed);
});
test('exclude_globs mutation makes stored != current (drift detected)', async () => {
const before = { exclude_globs: ['Templates/**'] };
const after = { exclude_globs: ['Templates/**', 'Photos/**'] };
const id = await makeSource('e2e-exclude-drift', before);
const beforeFp = computeSourceConfigFingerprint(before);
await writeConfigFingerprint(engine, id, beforeFp);
// Simulate the user mutating sources.config via `gbrain sources add
// --exclude`. The gate's next read of (stored, computed-from-current)
// detects the drift and forces a re-walk.
await engine.executeRaw(
`UPDATE sources SET config = $1::text::jsonb WHERE id = $2`,
[JSON.stringify(after), id],
);
const afterFp = computeSourceConfigFingerprint(after);
const stored = await readConfigFingerprint(engine, id);
expect(stored).toBe(beforeFp);
expect(stored).not.toBe(afterFp);
});
test('toggle-and-revert: add then remove same pattern leaves stored matching current', async () => {
const original = { exclude_globs: ['Templates/**'] };
const id = await makeSource('e2e-toggle', original);
const originalFp = computeSourceConfigFingerprint(original);
await writeConfigFingerprint(engine, id, originalFp);
// Add a pattern (drift) then remove it (revert).
await engine.executeRaw(
`UPDATE sources SET config = $1::text::jsonb WHERE id = $2`,
[JSON.stringify({ exclude_globs: ['Templates/**', 'Photos/**'] }), id],
);
await engine.executeRaw(
`UPDATE sources SET config = $1::text::jsonb WHERE id = $2`,
[JSON.stringify(original), id],
);
const revertedFp = computeSourceConfigFingerprint(original);
expect(revertedFp).toBe(originalFp);
expect(await readConfigFingerprint(engine, id)).toBe(originalFp);
// ⇒ Gate compares storedFp (==originalFp) to currentFp (==originalFp): no drift, no force-full.
});
test('include_globs drift detected independently', async () => {
const before = { include_globs: ['people/**'] };
const after = { include_globs: ['people/**', 'companies/**'] };
const id = await makeSource('e2e-include-drift', before);
await writeConfigFingerprint(engine, id, computeSourceConfigFingerprint(before));
await engine.executeRaw(
`UPDATE sources SET config = $1::text::jsonb WHERE id = $2`,
[JSON.stringify(after), id],
);
const stored = await readConfigFingerprint(engine, id);
const current = computeSourceConfigFingerprint(after);
expect(stored).not.toBe(current);
});
test('strategy drift detected', async () => {
const before = { strategy: 'markdown' };
const after = { strategy: 'code' };
const id = await makeSource('e2e-strategy-drift', before);
await writeConfigFingerprint(engine, id, computeSourceConfigFingerprint(before));
await engine.executeRaw(
`UPDATE sources SET config = $1::text::jsonb WHERE id = $2`,
[JSON.stringify(after), id],
);
expect(await readConfigFingerprint(engine, id))
.not.toBe(computeSourceConfigFingerprint(after));
});
test('mutating unrelated config field (federated) does NOT drift', async () => {
// The fingerprint hashes ONLY walk-affecting fields. Federation
// changes search visibility, not the walk set — must not invalidate
// the checkpoint.
const id = await makeSource('e2e-federated-toggle', {
federated: true,
exclude_globs: ['Templates/**'],
});
await writeConfigFingerprint(
engine,
id,
computeSourceConfigFingerprint({ exclude_globs: ['Templates/**'] }),
);
await engine.executeRaw(
`UPDATE sources SET config = $1::text::jsonb WHERE id = $2`,
[
JSON.stringify({ federated: false, exclude_globs: ['Templates/**'] }),
id,
],
);
const stored = await readConfigFingerprint(engine, id);
const current = computeSourceConfigFingerprint({
federated: false,
exclude_globs: ['Templates/**'],
});
expect(stored).toBe(current);
});
test('double-encoded JSONB config (the sources-add stringify bug) hashes equivalently to the parsed object', async () => {
// `gbrain sources add` writes `JSON.stringify(config)::jsonb`, which
// double-encodes the value into a JSON-string scalar (`"{\"x\":1}"`)
// rather than a proper JSONB object. The defensive reader in
// postgres-engine.ts:1274 + readSourceConfig parses the string back
// before the fingerprint sees it, so a double-encoded row and a
// properly-shaped row must fingerprint identically.
const cfg = { exclude_globs: ['Templates/**'], strategy: 'markdown' };
const direct = computeSourceConfigFingerprint(cfg);
// The pure compute fn handles a pre-parsed object; the persistence
// layer's job is to deliver a parsed object. We assert that the
// round-trip a real read would produce (parse the string scalar)
// hashes to the same value.
const parsed = JSON.parse(JSON.stringify(cfg));
expect(computeSourceConfigFingerprint(parsed)).toBe(direct);
});
});
-63
View File
@@ -355,69 +355,6 @@ describe('sync monorepo subdir-source support (#753/#774)', () => {
expect(await engine.getPage('wiki/draft-a')).toBeNull();
});
// ─────────────────────────────────────────────────────────────────────────
// --include: allow-list counterpart (#2156). Same scope-relative anchoring
// as --exclude; exclude applies after include.
// ─────────────────────────────────────────────────────────────────────────
test('--include: only matching files import on full sync', async () => {
const { performSync } = await import('../src/commands/sync.ts');
const result = await performSync(engine, {
repoPath,
include: ['wiki/**'],
noPull: true,
noEmbed: true,
full: true,
});
expect(result.status).toBe('first_sync');
expect(result.added).toBe(2); // wiki/page1 + wiki/page2; memory/* miss the allow-list
expect(await engine.getPage('wiki/page1')).not.toBeNull();
expect(await engine.getPage('memory/note1')).toBeNull();
});
test('--include applies to the incremental path too', async () => {
const { performSync } = await import('../src/commands/sync.ts');
const first = await performSync(engine, {
repoPath,
include: ['wiki/**'],
noPull: true,
noEmbed: true,
full: true,
});
expect(first.status).toBe('first_sync');
writeFileSync(join(repoPath, 'wiki', 'page3.md'), mdPage('Wiki Page 3'));
writeFileSync(join(repoPath, 'memory', 'note3.md'), mdPage('Memory Note 3'));
gitCommit(repoPath, 'more pages');
const second = await performSync(engine, {
repoPath,
include: ['wiki/**'],
noPull: true,
noEmbed: true,
});
expect(second.status).toBe('synced');
expect(second.added).toBe(1); // wiki/page3 only; memory/note3 misses the allow-list
expect(await engine.getPage('wiki/page3')).not.toBeNull();
expect(await engine.getPage('memory/note3')).toBeNull();
});
test('--exclude applies after --include (path in both is rejected)', async () => {
const { performSync } = await import('../src/commands/sync.ts');
const result = await performSync(engine, {
repoPath,
include: ['wiki/**'],
exclude: ['wiki/page2.md'],
noPull: true,
noEmbed: true,
full: true,
});
expect(result.status).toBe('first_sync');
expect(result.added).toBe(1); // page1 only: page2 included then excluded
expect(await engine.getPage('wiki/page1')).not.toBeNull();
expect(await engine.getPage('wiki/page2')).toBeNull();
});
// ─────────────────────────────────────────────────────────────────────────
// --exclude '**/*' emits warning (NAV-4)
// ─────────────────────────────────────────────────────────────────────────
-212
View File
@@ -1,212 +0,0 @@
/**
* TODO #3 `parseGlobList` defensive parse.
*
* `sources.config` is a JSONB column with no schema. The runtime can find
* anything in `config.include_globs` / `config.exclude_globs`:
* - A user `gbrain sources add` wrote `["people/**"]` (the happy path).
* - A stray hand-edit wrote `"people/**"` (string, not array).
* - A future migration's null default.
* - A test fixture that left the column at `{}`.
*
* The parse must produce `string[] | undefined` so the downstream
* `SyncOpts.include` / `SyncOpts.exclude` are either undefined (no filter)
* or a non-empty list of usable globs. Returning `[]` would make
* `commands/sync.ts:1454` engage the filter loop with an empty allow-list
* that classifies every path as `include-glob-miss`.
*/
import { describe, test, expect } from 'bun:test';
import { parseGlobList, mergeGlobs, computeSourceConfigFingerprint } from '../src/commands/sync.ts';
describe('parseGlobList — JSONB-safe coercion to string[] | undefined', () => {
test('happy path: array of strings round-trips identically', () => {
expect(parseGlobList(['people/**', 'companies/**'])).toEqual(['people/**', 'companies/**']);
});
test('single-element array returned as-is', () => {
expect(parseGlobList(['Templates/**'])).toEqual(['Templates/**']);
});
test('non-array values return undefined (string, object, number, null)', () => {
expect(parseGlobList('Templates/**')).toBeUndefined();
expect(parseGlobList({ globs: ['Templates/**'] })).toBeUndefined();
expect(parseGlobList(42)).toBeUndefined();
expect(parseGlobList(null)).toBeUndefined();
expect(parseGlobList(undefined)).toBeUndefined();
});
test('empty array returns undefined (no engagement of the filter loop)', () => {
// Critical: a literal `[]` must not slip through. Empty `include` in
// SyncableOptions silently passes everything (good), but empty
// `exclude` is fine too — the real motivation is to keep `SyncOpts`
// unset so callers can ignore the field entirely. Symmetric with the
// `omitted glob flags leave config untouched` regression guard in
// sources.test.ts.
expect(parseGlobList([])).toBeUndefined();
});
test('mixed array drops non-string entries and keeps the rest', () => {
expect(parseGlobList(['people/**', 42, null, 'companies/**'])).toEqual([
'people/**',
'companies/**',
]);
});
test('empty strings dropped (a `""` glob would match every path)', () => {
expect(parseGlobList(['', 'people/**', ''])).toEqual(['people/**']);
});
test('array of only empty strings collapses to undefined', () => {
expect(parseGlobList(['', '', ''])).toBeUndefined();
});
});
describe('mergeGlobs — CLI flags union with persisted source-config globs', () => {
test('both sides present: union, deduped, CLI first', () => {
expect(mergeGlobs(['a/**', 'b/**'], ['b/**', 'c/**'])).toEqual(['a/**', 'b/**', 'c/**']);
});
test('CLI only', () => {
expect(mergeGlobs(['a/**'], undefined)).toEqual(['a/**']);
});
test('persisted only', () => {
expect(mergeGlobs([], ['Templates/**'])).toEqual(['Templates/**']);
});
test('neither side: undefined so SyncOpts stays unset', () => {
expect(mergeGlobs([], undefined)).toBeUndefined();
});
});
/**
* #2157 follow-on (sources_config_fingerprint, migration v125).
*
* computeSourceConfigFingerprint hashes the walk-affecting fields of
* sources.config (strategy + include_globs + exclude_globs) so the
* "Already up to date" gate at performSync's git-HEAD equality check
* can detect drift and force a re-walk. These cases pin the contract
* the gate depends on:
*
* - Deterministic over equivalent inputs (order-insensitive,
* defensively-coerced via parseGlobList).
* - Sensitive to each walk-affecting field separately.
* - Insensitive to fields the walker doesn't read (federated,
* unrelated keys).
* - A toggle-and-revert is a no-op (returns to the original hash).
*
* Without the canonicalization the gate would fire spuriously on
* cosmetic changes (e.g. a user re-ordering their exclude list) and
* miss real drift (e.g. an add-then-remove that nets to a different
* effective set than the stored fingerprint).
*/
describe('computeSourceConfigFingerprint — walk-affecting config drift detector', () => {
test('empty config produces a stable hash', () => {
const a = computeSourceConfigFingerprint({});
const b = computeSourceConfigFingerprint({});
expect(a).toBe(b);
expect(a).toMatch(/^[a-f0-9]{64}$/);
});
test('null / undefined / missing config all hash the same', () => {
const empty = computeSourceConfigFingerprint({});
expect(computeSourceConfigFingerprint(null)).toBe(empty);
expect(computeSourceConfigFingerprint(undefined)).toBe(empty);
});
test('same config → same hash (deterministic)', () => {
const cfg = { strategy: 'markdown', exclude_globs: ['Templates/**', 'Photos/**'] };
expect(computeSourceConfigFingerprint(cfg)).toBe(computeSourceConfigFingerprint(cfg));
});
test('array order does not affect hash (canonical sort)', () => {
const a = computeSourceConfigFingerprint({ exclude_globs: ['a/**', 'b/**', 'c/**'] });
const b = computeSourceConfigFingerprint({ exclude_globs: ['c/**', 'a/**', 'b/**'] });
expect(a).toBe(b);
});
test('exclude_globs change → different hash', () => {
const a = computeSourceConfigFingerprint({ exclude_globs: ['Templates/**'] });
const b = computeSourceConfigFingerprint({ exclude_globs: ['Templates/**', 'Photos/**'] });
expect(a).not.toBe(b);
});
test('include_globs change → different hash', () => {
const a = computeSourceConfigFingerprint({ include_globs: ['people/**'] });
const b = computeSourceConfigFingerprint({ include_globs: ['people/**', 'companies/**'] });
expect(a).not.toBe(b);
});
test('strategy change → different hash', () => {
const a = computeSourceConfigFingerprint({ strategy: 'markdown' });
const b = computeSourceConfigFingerprint({ strategy: 'code' });
expect(a).not.toBe(b);
});
test('strategy unset vs set differ', () => {
const unset = computeSourceConfigFingerprint({});
const set = computeSourceConfigFingerprint({ strategy: 'markdown' });
expect(unset).not.toBe(set);
});
test('add-then-remove returns to original hash (toggle is a no-op)', () => {
const original = computeSourceConfigFingerprint({ exclude_globs: ['Templates/**'] });
const added = computeSourceConfigFingerprint({ exclude_globs: ['Templates/**', 'Photos/**'] });
const reverted = computeSourceConfigFingerprint({ exclude_globs: ['Templates/**'] });
expect(added).not.toBe(original);
expect(reverted).toBe(original);
});
test('non-walk-affecting fields are ignored (federated, unrelated keys)', () => {
const a = computeSourceConfigFingerprint({ exclude_globs: ['Templates/**'], federated: true });
const b = computeSourceConfigFingerprint({ exclude_globs: ['Templates/**'], federated: false });
const c = computeSourceConfigFingerprint({ exclude_globs: ['Templates/**'], some_unrelated_key: 'value' });
expect(a).toBe(b);
expect(a).toBe(c);
});
test('non-string strategy coerced to null (defensive)', () => {
// A hand-edited row could leave `strategy: 42` or `strategy: {}` — both
// collapse to the same shape as `strategy: undefined` so the fingerprint
// doesn't reflect a value the walker can't honor anyway.
const empty = computeSourceConfigFingerprint({});
expect(computeSourceConfigFingerprint({ strategy: 42 })).toBe(empty);
expect(computeSourceConfigFingerprint({ strategy: {} })).toBe(empty);
expect(computeSourceConfigFingerprint({ strategy: null })).toBe(empty);
});
test('defensive parsing: mixed-type glob arrays hash same as cleaned arrays', () => {
// parseGlobList drops non-string + empty entries; the fingerprint must
// reflect what the walker actually uses, not what the raw row says.
const dirty = computeSourceConfigFingerprint({
exclude_globs: ['Templates/**', 42, null, '', 'Photos/**'],
});
const clean = computeSourceConfigFingerprint({
exclude_globs: ['Templates/**', 'Photos/**'],
});
expect(dirty).toBe(clean);
});
test('empty array and missing field hash identically', () => {
const missing = computeSourceConfigFingerprint({});
const emptyArray = computeSourceConfigFingerprint({ exclude_globs: [] });
const emptyAfterClean = computeSourceConfigFingerprint({ exclude_globs: ['', '', ''] });
expect(emptyArray).toBe(missing);
expect(emptyAfterClean).toBe(missing);
});
test('non-array exclude_globs (string, object) hash same as missing', () => {
const missing = computeSourceConfigFingerprint({});
expect(computeSourceConfigFingerprint({ exclude_globs: 'Templates/**' })).toBe(missing);
expect(computeSourceConfigFingerprint({ exclude_globs: { foo: 'bar' } })).toBe(missing);
});
test('SHA-256 output shape: 64 hex characters', () => {
const fp = computeSourceConfigFingerprint({
strategy: 'markdown',
include_globs: ['people/**'],
exclude_globs: ['Templates/**', '.git/**'],
});
expect(fp).toMatch(/^[a-f0-9]{64}$/);
});
});