With unknown op flags now a hard error (this PR), the tutorial's
`gbrain search ... --remote` invocations would fail: no code path ever
read a --remote flag — thin-client installs route shared ops through the
remote MCP server automatically. Update the three examples to the
flagless form and explain the automatic routing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three CLI ergonomics fixes on the shared-op dispatch path:
- parseOpArgs now hard-errors on undeclared flags instead of silently
swallowing them into params (#380). A pass-through allowlist keeps the
undeclared-but-honored flags working: --source (makeContext's source
resolver), --brain (mount axis), --dry-run (ctx.dryRun), --json.
Value-taking flags with no value also error instead of being dropped.
- `gbrain put SLUG --file PATH` reads content from a file (#380, takeover
of #856). Driven by the op's cliHints.stdin declaration rather than
put_page hard-coding, so volunteer-context gets it too. Mutually
exclusive with --content/stdin in either flag order; 5MB cap shared
with stdin; unreadable path is a loud error instead of an empty page.
put --help documents the flag.
- `--json` on shared ops now actually emits raw JSON (docs already
promised it); same seam on local-engine and thin-client routed paths.
- list_pages declares `offset` (both engines already supported it on
PageFilters) and the limit description discloses the 100-row cap
(#2876), so `gbrain list` can paginate past 100 instead of silently
truncating.
Co-authored-by: Kage18 <Kage18@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Synth pages written by dream synthesize currently open straight into
detail (quotes, cross-references) with no framing, so a reader who
lands on the page later — without the source transcript in front of
them — has no way to tell what it's about without reading the whole
thing.
Add OUTPUT POLICY item 5: every new page's body must open with a 2-3
sentence self-contained summary a reader unfamiliar with the source
conversation could understand on its own, before any quotes or detail.
Long-lived minion workers can outlive DB-backed model config changes. Refresh the AI gateway before gateway-backed handlers run so queued cycle/propose_takes work does not fall back to a stale Anthropic default when the operator configured another provider.
Also record the active gateway chat model in propose_takes budget/proposal metadata instead of hardcoding claude-sonnet-4-6, and keep provider:model IDs intact for budget pricing.
Regression coverage verifies queued worker refresh, propose_takes model metadata, nested provider IDs, skipFence threading, and the updated autopilot signal source guard.
Co-authored-by: maxpetrusenkoagent <[REDACTED EMAIL]>
Setting `models.tier.deep anthropic:claude-opus-4-8` silently disabled
think and auto_think: the Anthropic recipe's chat allowlist stopped at
Opus 4.7, the tier-resolved model never joined the extended set that
assertTouchpoint's contract promises for config-chosen models, and the
resulting probe failure was stamped NO_ANTHROPIC_API_KEY — sending the
operator to debug env/keychain when the fix was the model id. Three
fixes, one per layer:
- recipes/anthropic.ts: add claude-fable-5, claude-opus-4-8, and
claude-sonnet-5 to chat models; claude-sonnet-5 to expansion models.
- gateway.ts reconfigureGatewayWithEngine: resolve all four tiers and
register the results as extended models, honoring the documented
contract for models.default / models.tier.* (model-resolver.ts
docstring). A tier-only model now validates like a chat/expansion one.
- think/index.ts: when the gateway client can't be built, re-probe and
label honestly — MODEL_NOT_USABLE:<reason> for unknown_model /
unknown_provider, NO_ANTHROPIC_API_KEY only for the actual missing-key
case; the stub answer carries the probe detail and fix hint.
Tests: recipe-list presence pins; a new gateway-tier-extended-models
suite proving a fictional tier model validates post-reconfigure (and an
unconfigured one still doesn't); think-pipeline coverage for the honest
label (unknown_model beats missing-key even keyless); the existing
non-explicit bogus-provider test updated from the old catch-all label to
the honest one (no-throw contract unchanged).
Verified live: a brain with tier.deep=claude-opus-4-8 had think degrade
to gather-only with the misleading key warning; with this change the
probe passes and synthesis runs.
Co-authored-by: Paolo Belcastro <p3ob7o@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(dream): --once for one-shot phase runs without toggling config gates
Fixes the "toggle enabled true, run, toggle back to false" workaround
that gbrain doctor's extract_atoms_backlog message implicitly
recommends and that #2860's reporter had to script around: with an
external orchestrator running `gbrain dream --phase patterns` on a
cadence outside the autopilot, the only way to run patterns once was
`config set dream.patterns.enabled true` -> run -> `config set ...
false`. A crash between steps left the flag stuck true, and the
autopilot (which polls the same flag) re-enqueued patterns every
cycle -- 119 LLM jobs / ~$400 over 24h before it was caught.
Root cause: `--phase X` only controls which phase FUNCTION cycle.ts
calls; it does not bypass that phase's own `dream.<phase>.enabled` /
`cycle.<phase>.enabled` config read. Each gated phase (patterns,
synthesize, conversation_facts_backfill, enrich_thin, skillopt) reads
its enabled flag internally and skips regardless of how the phase was
selected -- confirmed by reading each phase module, not assumed.
extract_atoms/synthesize_concepts are a DIFFERENT mechanism entirely
(pack-declaration via packDeclaresPhase, not a config .enabled read)
and already have a working one-shot escape hatch: `--drain`. The
existing doctor message for extract_atoms already says `--phase
extract_atoms --drain --window 120`, so no doctor text needed
updating there -- verified by reading src/commands/doctor.ts directly
rather than assuming the paraphrase in the issue was literal.
Design: `gbrain dream --phase <name> --once`. Requires an explicit
--phase (bare --once is a usage error, exit 2) so it can never
force-enable every disabled phase at once in a full/default cycle --
that would recreate the same unbounded-spend risk the flag exists to
prevent. Threaded through CycleOpts as `onceForPhase?: CyclePhase`
(the literal phase name, not a boolean) so the bypass can never leak
to a phase other than the one named, even if a future programmatic
caller passes a wider `phases` array than the CLI does. Never reads
or writes config -- the phase still evaluates its .enabled gate every
call; --once only overrides the boolean OUTCOME for that one
invocation, mirroring the existing --unsafe-bypass-dream-guard /
--input precedents (stderr warning at the bypass point, no new
config-touching code path).
Rejected alternatives (documented per task instructions):
- Making explicit --phase X always bypass .enabled: breaking change
for existing crons that rely on the disabled flag as a cheap no-op;
an upgrade would silently start running LLM/write phases.
- A new subcommand: adds a whole dispatch/help/arg surface that
internally routes through the same override anyway.
- Extending --once to also bypass packDeclaresPhase for
extract_atoms/synthesize_concepts: conflates two different gating
mechanisms (config toggle vs. pack membership) under one flag;
extract_atoms already has --drain, which is purpose-built for its
batched/windowed execution model.
Design was cross-validated by an independent second-model review
(external design consultation) before implementation; its
recommendation to also update the extract_atoms doctor message to
`--once` was NOT adopted because that phase has no .enabled gate to
bypass -- doing so would be a documented no-op, contradicted by
reading src/commands/doctor.ts:3264 directly.
Tests: 9 new (structural CLI-flag wiring in dream-cli-flags.test.ts;
a real PGLite E2E test in dream-patterns-pglite.test.ts proving the
bypass fires AND that dream.patterns.enabled is never written; 4
runCycle-level tests in cycle.serial.test.ts proving onceForPhase
does not leak across phases). Verified 6 of 9 fail against the
pre-fix source (via git stash of source-only changes) to confirm
they're meaningful regressions, not tautologies.
Closes#2860
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(dream): --help short-circuits before --once usage validation
Codex review finding (P2): `gbrain dream --help --once` (no --phase)
called process.exit(2) from the new --once usage-error check inside
parseArgs before runDream's documented IRON RULE ("--help
short-circuits BEFORE any engine-bearing work") ever got a chance to
run -- parseArgs computes ALL its validations unconditionally before
runDream checks opts.help. Repo precedent for this ordering already
exists as a pinned regression test (test/dream.test.ts's "--help
--source whatever prints help and exits 0").
Fix: compute wantsHelp once in parseArgs and exempt the --once
validation when it's set, mirroring that precedent. Added the same
class of pinned tests here: bare `--once` still exits 2 with the
usage hint, `--help --once` prints help and exits 0, and a real
--phase patterns --once run against a PGLite engine proves the
bypass actually fires (falls through to insufficient_evidence
instead of disabled) without writing dream.patterns.enabled. Also
fixed the structural test in dream-cli-flags.test.ts that asserted
the exact pre-fix guard-condition source text.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(dream): --once must require an EXPLICIT --phase, not a derived one
Codex review finding (P3): the --once validation checked the derived
`phase` value, but `phase` gets defaulted implicitly by --input
(implies --phase synthesize) and --drain (implies --phase
extract_atoms) BEFORE that check ran. So `gbrain dream --input <f>
--once` and `gbrain dream --drain --once` both slipped past the
"explicit --phase required" contract silently -- and --once became a
true no-op in both cases: --drain returns from runDream before
onceForPhase is ever read (the drain path doesn't call runCycle at
all), and --input already bypasses the synthesize enabled-gate on its
own via the existing opts.inputFile check, so onceForPhase would
never even be consulted.
Fix: capture `phaseWasExplicit = phaseIdx !== -1` at the very top of
parseArgs, before the --input/--drain defaulting blocks run, and
validate --once against that instead of the derived `phase`. Updated
the usage-error message and --help text to say "an explicit --phase"
so a user hitting this understands why `--input ... --once` doesn't
count.
Tests: 2 new pins in test/dream.test.ts exercising runDream directly
(--input <file> --once exits 2; --drain --once exits 2), plus a
structural test in dream-cli-flags.test.ts pinning that
phaseWasExplicit is captured before both implicit-defaulting blocks.
Updated the two existing structural/behavioral tests whose literal
guard-condition / error-message assertions changed shape.
Verified: dream-cli-flags.test.ts 27/27, dream.test.ts 31/31,
typecheck clean.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: masashiono0611 <masashi.ono.0611@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Garry Tan <garrytan@gmail.com>
* fix(providers): reuse buildGatewayConfig for --model test override (#2863)
`gbrain providers test --model <id>` overrode the gateway with only
embedding_model/chat_model + env, dropping config.provider_base_urls
entirely. A brain configured with a custom endpoint (e.g. a China-region
DashScope base URL) would pass the bare `providers test` (which goes
through configureFromEnv() and does forward base_urls) but fail the
`--model`-scoped probe with a misleading "Incorrect API key" error, even
though the key was valid for the configured endpoint — the probe silently
fell back to the recipe's hardcoded default endpoint instead.
Root cause: two independent, drifted resolvers. The production path
(src/cli.ts#connectEngine, src/core/init-embed-check.ts) builds its
AIGatewayConfig via buildGatewayConfig(), which folds provider_base_urls,
env-sourced local-server base URLs, provider_chat_options, and file-plane
API keys. The --model override branch in runTest() hand-rolled a second,
narrower config object that only carried the overridden model + raw env.
Fix: lift `cfg` out of the existing try/catch (it was already loaded there
for the isolation-warning message) and spread `buildGatewayConfig(cfg)`
into both configureGateway() calls before overriding embedding_model/
chat_model. The isolated --model probe now resolves its endpoint exactly
the way the brain's real import/query path would; only the requested
model is overridden, so the probe still targets exactly the model the
user asked for. Falls back to bare env when no brain is configured yet
(cfg is null), matching prior first-time-install behavior.
Confirmed chat_fallback_chain (also threaded through by buildGatewayConfig)
has no runtime retry effect — it's only consumed to pre-register extended
model ids — so spreading the full production config does not mask an
isolated model's own failures behind a silent fallback.
Other diagnostic surfaces (providers list/env/explain) were checked and
are unaffected: `runProviders()` already calls configureFromEnv() (which
forwards base_urls correctly) before dispatch, and none of them accept
--model, so they never hit the broken override branch.
Adds test/providers-test-model-base-url.test.ts: drives runProviders('test',
...) end-to-end against a mocked fetch + temp GBRAIN_HOME/config.json with
provider_base_urls set for the dashscope recipe (the exact recipe named in
the bug report), asserting the outbound request hits the configured base
URL rather than the recipe default. Verified red on pre-fix code via
git stash, green after.
Closes#2863
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix: drop duplicate buildGatewayConfig import after master merge
Master's f3e78fd2 added the same import the PR carried; the textual
merge was clean but the result failed typecheck (TS2300 duplicate
identifier).
Co-authored-by: masashiono0611 <masashi.ono.0611@gmail.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: masashiono0611 <masashi.ono.0611@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Garry Tan <garrytan@gmail.com>
The file's final test configures the gateway with a remote provider and a
fake key, and its afterEach only clears the mock transport. With no
afterAll, the poisoned global config survives the file boundary; the next
test file in the shard that triggers an embed makes a real HTTP call and
fails. Surfaced on master when #3022's new test file reshuffled shard
composition (shard 6: synthesize-concepts-progress failed twice with a
live Google embed rejection). The legacy-embedding preload can't catch
this: it only re-applies defaults when the gateway slot is empty.
One-line root-cause fix at the leaker. A repo-wide guard for the class
(~70 files call configureGateway without a final reset) is filed as a
follow-up.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Adds NVIDIA NIM / API Catalog as a first-class OpenAI-compatible AI recipe:
- chat via nvidia/nemotron-3-super-120b-a12b (conservative capability
claims: no tools, no subagent loop until proven)
- hosted embedding models incl. nvidia/llama-nemotron-embed-1b-v2 with
Matryoshka-style dimension overrides (1024/1280/1536/2048) and fixed
natural dims for the other catalog models
- asymmetric input_type mapping (document -> passage, query -> query) via a
gateway compat fetch shim, since the generic openai-compatible recipe
cannot infer that provider-specific requirement
- base URL https://integrate.api.nvidia.com/v1 verified live (OpenAI-shaped
/v1/models, all five recipe model ids present in the catalog)
Changed from the original PR: dropped the recipe's custom resolveAuth — it
duplicated defaultResolveAuth's Authorization-Bearer behavior exactly and
violated the IRON RULE that only Azure overrides resolveAuth
(test/ai/recipes-existing-regression.test.ts). NVIDIA_API_KEY now flows
through defaultResolveAuth via auth_env.required, and the recipe test pins
resolveAuth === undefined + the default Bearer resolution + the missing-key
AIConfigError. Also scrubbed a private downstream-agent name from ported
comments per the repo privacy rule.
Takeover of #2965 by @ravehorn.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: SAGE Codex <codex@sage.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
getLastSeen had no upper date bound, so a future-dated chronicle event (a
scheduled calendar-event, a planned milestone) became the entity's "last
seen" date; finalizeLastSeen's Math.max(0, ...) then clamped the negative
delta, reporting days_ago: 0 -- the entity reads as seen-today. Recording
future events is intended (eligibility ELIGIBLE_TYPES includes calendar-event);
the reader just needs to stop counting them as "seen".
Bound the query to te.date <= COALESCE(asof, current_date) in both engines,
mirroring getOnThisDay's existing te.date < target bound. asof now reaches
the WHERE clause (previously it only reached finalizeLastSeen), so as-of
time-travel is honored for the date filter too.
Regression test added: an entity with past events plus a future event -> last
seen returns the most recent PAST event, not the future one; and as-of after
the future date lets it through. Fails before, passes after.
putPage's INSERT used COALESCE(<chunkerVersion>, 1), so callers that don't
supply chunker_version (no MCP/subagent caller does — it's internal metadata)
landed new pages at version 1. Dream subagents write through putPage directly,
so their pages got v1 and doctor's contextual_retrieval_coverage check flagged
them as "older chunker_version" forever, even though they were chunked and
embedded with the current chunker.
Default the INSERT to MARKDOWN_CHUNKER_VERSION on both engines. The ON CONFLICT
UPDATE still COALESCE-preserves an explicitly supplied version. Add an
engine-level regression test.
reconcile-links is advertised in `gbrain --help` and implemented with a
`case 'reconcile-links'` block in handleCliOnly, but it was missing from the
CLI_ONLY Set. Dispatch only enters handleCliOnly when the command is in
CLI_ONLY, so every invocation fell through to the shared-operations lookup and
hit the generic "Unknown command" branch — leaving the documented doc↔impl
edge-rebuild tool silently unreachable via the CLI.
Add 'reconcile-links' to CLI_ONLY, plus a reachability regression test
mirroring the #2035 (`calibration`) guard.
The content-sanity gate's audit logger (logContentSanityAssessment)
defaults, via audit-writer.ts::resolveAuditDir(), to writing
~/.gbrain/audit/content-sanity-YYYY-Www.jsonl on disk. A GBRAIN_AUDIT_DIR
env override exists, but nothing in the shared test bootstrap ever set
it, so any test that exercised an audit-emitting code path without
wrapping the call in its own withEnv() fell through to the operator's
real audit trail. test/import-file.test.ts's oversize-boundary fixture
('borderline-slug', content just under MAX_FILE_SIZE but over
DEFAULT_BYTES_BLOCK) fired a real soft_block event into the developer's
live ~/.gbrain/audit on every run — which doctor's
content_sanity_audit_recent check then reported as production signal.
Fix: add a bootstrap preload (test/helpers/audit-dir-preload.ts, wired
via bunfig.toml) that points GBRAIN_AUDIT_DIR at a fresh per-process
mkdtemp dir before any test file loads. Each run-unit-shard.sh shard is
its own bun process, so each shard gets its own scratch dir with no
cross-shard collision. This closes the leak for every audit-emitting
test, not just this fixture. It respects a developer-exported override
(only sets the var when unset), and files that manage their own
per-test GBRAIN_AUDIT_DIR via withEnv() are unaffected.
Also fix a latent isolation bug this surfaced: gbrain-home-isolation.test.ts
unconditionally deleted GBRAIN_AUDIT_DIR in a finally block instead of
restoring the prior value, which clobbered the bootstrap's scratch dir
for every test file that ran after it in the same shard process.
Adds test/audit/audit-dir-preload.test.ts to pin the behavior: it
reproduces the exact soft_block event shape and asserts it lands in the
scratch dir, never in ~/.gbrain/audit.
Reported by @paul-0320.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
📝 Summary:
• launchd rejects group/world-writable agent plists — when the installer
runs under a umask-0 parent shell, `writeFileSync(plistPath(), plist)`
produces a 0666 plist that makes `launchctl load`/`bootstrap` fail with
the opaque `Bootstrap failed: 5: Input/output error` and the login-time
LaunchAgents scan skip the file silently
• on an affected machine the daemon never registers while everything
looks installed — the plist exists, launchd's disabled-table says
enabled, and no log file is ever created
🔧 Technical Improvements:
• `installLaunchd`: write plist with `{ mode: 0o644 }` AND
`chmodSync(0o644)` — writeFileSync mode applies only on create, so a
reinstall over an existing 0666 plist must normalize explicitly
• `installSystemd`: same hardening on the unit file (systemd warns on
world-writable units); symmetric with the launchd path
• Restart-policy rewrite path (`generateSystemdUnit` rewrite of an
existing unit): chmod is load-bearing here — the file always exists,
so the write mode never applies
• `chmodSync` added to the fs import
📊 Code Changes: 18 insertions, 4 deletions (net +14)
📦 Files Modified:
• src/commands/autopilot.ts (minor updates) — mode + chmod on the three
supervisor-file writers; comments carry the launchd failure signature
so the next EIO hunt greps straight to it
embedStaleForSource rebuilds each page's chunks as a merged ChunkInput[]
carrying only five fields (chunk_index, chunk_text, chunk_source,
embedding, token_count), while upsertChunks writes the metadata columns
as EXCLUDED.<col>. Any page containing at least one stale chunk
therefore has ALL its chunks' metadata reset on the next embed-stale
pass:
- image chunks flip modality 'image' -> 'text' and disappear from the
cross-modal image search arm permanently (it filters
modality='image'), while keeping their embedding_image vector — the
data looks intact but is unreachable;
- code chunks lose language, symbol_name, symbol_type,
symbol_name_qualified.
The read side compounds this: rowToChunk never returned modality, so a
correct merge was impossible without also extending the Chunk shape.
Fix: expose modality on Chunk/rowToChunk and carry modality, language,
and the symbol fields through the merge. embedding_image is deliberately
not carried — upsertChunks already COALESCEs it server-side.
Repair for affected brains: UPDATE content_chunks SET modality='image'
WHERE chunk_source='image_asset' AND embedding_image IS NOT NULL.
The new test seeds a mixed page (settled image chunk + stale text chunk
with symbol metadata) and asserts both survive an embedStaleForSource
pass; it fails on master.
Co-authored-by: Forge (Ron) <forge@zsimovan.dev>
The image_assets check statSyncs files.storage_path directly, but
sync-ingested assets store repo-relative paths. Run doctor from any
directory other than the brain repo and every image is reported
'missing from disk' — a persistent false WARN with a suggested fix
(gbrain sync --skip-failed) that does nothing.
Resolve relative paths against sync.repo_path before statting; absolute
paths are untouched. Falls back to cwd when the config key is unset,
preserving the old behavior for brains without a configured repo.
Co-authored-by: Forge (Ron) <forge@zsimovan.dev>
The submission-time backpressure cap counts all waiting (name, queue)
rows regardless of which source a job targets. On a multi-source brain
this makes per-source submissions with maxWaiting: 1 mutually exclusive:
while one source's sync sits waiting, every other source's freshness
sync coalesces into that row and never runs. The dispatch log shows the
starved source 'dispatched' each interval (queue.add returns the other
source's waiting row), so the starvation is invisible unless you notice
sources.last_sync_at falling behind — we found a secondary source 29
hours stale on a 5-minute freshness interval.
Fix: when the submitted data carries a string sourceId, key the advisory
lock, the waiting count, and the coalesce target on it. Submissions
without sourceId keep the existing single-scope behavior, so
single-source brains and non-sync jobs are unchanged.
The new test asserts same-source submissions still coalesce while a
different source gets its own row and its own cap; it fails on master.
Co-authored-by: Forge (Ron) <forge@zsimovan.dev>
* fix(sources): validate --path is a git repo at registration time (#2707)
`sources add --path <dir>` accepted any existing non-git directory with
zero validation, deferring the failure to the first `gbrain sync` ("Not
inside a git repository: ..."). By the time that surfaces, the source
has been silently stale for however long nobody read the sync logs.
Add a registration-time check (git-remote.ts:isInsideGitRepo, mirroring
sync.ts's discoverGitRoot walk-up so subdir-of-git-repo sources still
pass) that rejects an existing-but-non-git --path directory with an
actionable error pointing at `git init && git add -A && git commit`.
Non-existent paths are unaffected (out of scope — different, pre-existing
failure mode) and `--force` opts out for callers who want to register
before git-init exists.
This is registration-time validation ONLY — it never auto-`git init`s
the directory, preserving the consent boundary #2967 established for
sync-time self-heal (a --path source is the user's own external
directory; gbrain must not mutate it without explicit ask).
Also documents the git requirement (docs/guides/multi-source-brains.md),
including the "files must be committed, not just present" gotcha and
that a stale/unreachable sync anchor already self-heals on plain
`gbrain sync` (verified manually against HEAD — no reset-anchor command
needed).
* fix(sources): require a committed HEAD + shell-quote remediation cmd (codex round 1)
Codex review round 1 on #2707 found two real gaps:
1. isInsideGitRepo alone accepts a `git init`ed-but-never-committed
directory (rev-parse --show-toplevel succeeds with no HEAD), so
registration would still pass a source that fails sync's own
"No commits in repo ... Make at least one commit before syncing."
Add hasGitCommits (git rev-parse HEAD) as a second required check.
2. The remediation command in the error message interpolated the raw
path unquoted — spaces, $(), backticks, etc. would break or, worse,
execute unintended shell syntax if pasted. POSIX single-quote it
(mirrors src/commands/connect.ts:shellQuote; duplicated locally
rather than imported, since commands/ depends on core/ not the
reverse).
* fix(sources): require tracked content in HEAD, not just a resolvable HEAD (codex round 2)
Codex review round 2 P1: hasGitCommits (rev-parse HEAD) accepted a repo
with an empty commit (git commit --allow-empty) followed by untracked
files — HEAD resolves fine (to git's well-known empty-tree object), so
registration passed, but the first sync would "succeed" importing
nothing and then silently never notice the untracked files change. The
exact same gap applied to an untracked subdirectory of an otherwise-
real git repo (monorepo case).
Replace hasGitCommits with hasTrackedContent (`git ls-tree HEAD -- .`,
non-recursive — one entry is enough, no need to walk the whole
subtree). `-C path` + pathspec `.` scopes correctly to both a repo
toplevel and a subdirectory-of-a-repo source, and an empty tree lists
zero entries where a bare `rev-parse HEAD` would still succeed. Also
subsumes the "no commits at all" case hasGitCommits covered (ls-tree
on an unborn repo fails the same way), so this is one check instead
of two.
Updated the error copy and docs/guides/multi-source-brains.md to match
what's actually verified now.
* fix(sources): O(1)-output tree-emptiness probe, avoid maxBuffer overflow (codex round 3)
Codex review round 3 found the round-2 `git ls-tree HEAD -- .` listing
buffers the whole (non-recursive) tree — a real repo with ~17-20K
directly-tracked entries exceeds execFileSync's default 1 MiB
maxBuffer, throws ENOBUFS, and the catch-all incorrectly rejects a
perfectly valid registration.
Replace the listing with `git rev-parse --verify HEAD:./` (resolves
the tree object for `path` specifically, correct for both toplevel and
subdirectory sources same as before) compared against git's canonical
empty-tree SHA-1 (4b825dc6...) — a fixed ~40-byte read regardless of
how many entries the tree has, structurally immune to this class of
bug rather than just raising the threshold. Added a 300-file
regression test locking this in.
Declined a second round-3 finding (P1: reject a tree if ANY untracked
file exists anywhere under the path, not just when the tree is
entirely empty) — untracked files never being synced is standard,
existing git-source behavior throughout this codebase (identical for
--url managed clones), not a bug specific to this validation. Enforcing
zero-untracked-files at registration would reject ordinary repos with
gitignored build output, .DS_Store, editor swapfiles, etc. Out of
scope relative to what #2707 actually asks for (a directory with real,
committed content that will sync) and how every other git source in
this system already behaves.
* fix(sources): derive empty-tree OID per repo instead of hardcoding SHA-1 (codex round 4)
Codex review round 4 P2, confirmed by directly testing against a
`git init --object-format=sha256` repo: the hardcoded SHA-1 empty-tree
constant only matches SHA-1 repositories. An empty SHA-256 repo's real
empty-tree OID is a different (64-char) hash, so the SHA-1 comparison
silently mismatched and let an empty/untracked SHA-256 source through
— exactly the case this validation exists to catch.
Replace the constant with `emptyTreeOid()`: `git hash-object -t tree
--stdin < /dev/null` computed in the target repo's own context, so it
returns the correct empty-tree OID for whichever object format that
repo actually uses, without gbrain needing to know or care which one.
Added gated regression tests (git 2.29+ / --object-format=sha256,
test.skipIf on older git) for both the empty-repo-rejected and
real-content-registers-fine cases.
Converging here (4 review rounds; this is the last outstanding
finding from round 4, and round 4 raised only this one issue).
* fix(test): --force the incidental non-git second source in #1434 routing test (#2707)
CI caught a real regression from this PR's registration-time git
validation: test/sync-sole-non-default-routing.test.ts's "2+ non-default
sources" case registers a bare mkdtempSync temp dir (no git init) as a
second source purely to have 2 sources present — the directory's content
was never meant to be exercised, only its existence as a distinct
local_path. #2707's new validation correctly rejects that dir at
registration time, since nothing else in the test suite told it
otherwise.
--force is the right fix, not adding unnecessary git-init/commit
boilerplate to secondRepo: it documents that this specific registration
intentionally doesn't care about git-validity, matching what a real
caller opting into the legacy lenient behavior would do.
Verified: the specific test (3/3 pass), plus every other test file in
the repo using `sources add --path` (sources.test.ts, sources-ops.test.ts
already covered by the PR's own commits; repos-alias.test.ts,
sync-cost-gate.serial.test.ts — 11/11 pass, no similar fixture gap).
typecheck clean, verify 31/31.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Same class as #3019's v123 fix; the guard test added there flagged this
within one push. Route the notice through process.stderr.write.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
A single markdown page whose compiled_truth exceeds Postgres's hard
1,048,575-byte tsvector cap made update_page_search_vector() throw
"string is too long for tsvector" INSIDE the pages UPSERT transaction.
Not a per-file ledger entry — a transaction abort. The whole source's
sync checkpoint stayed pinned (Sync BLOCKED) until the oversized file was
fixed or manually skipped, even though every other file in the run
imported fine. --retry-failed re-failed the same files every run; the
3-consecutive-failure auto-skip eventually moved past them, but for a
scheduled collector that meant hours of blocked cycles per oversized
file, per source.
Root cause: pages.search_vector indexed compiled_truth — the unbounded
whole-page body — even though it's write-only dead weight for actual
search. searchKeyword() (postgres-engine.ts / pglite-engine.ts) ranks and
queries content_chunks.search_vector exclusively (Cathedral II Layer 3,
chunk-grain, already populated separately from compiled_truth via
chunking at import time, and well under the tsvector cap since
chunkText() targets embedding-sized pieces). Verified directly: no
pages.search_vector / bare search_vector read appears anywhere outside
this trigger's own definition and the reindex/backfill machinery that
maintains it.
Fix: v124 migration recreates update_page_search_vector() without
compiled_truth — title + timeline stay (both naturally small), so the
column keeps carrying some signal rather than going fully inert. Updated
in lockstep (documented contract, see reindex-search-vector.ts's own
comment): migrate.ts's new v124, reindex-search-vector.ts's
recreatePagesFn, and the fresh-install baselines in pglite-schema.ts +
src/schema.sql (regenerates schema-embedded.ts via `bun run
build:schema`). No backfill: existing rows keep whatever search_vector
they already computed until their next UPDATE — harmless, since nothing
reads this column, and the brains that actually hit this bug never
successfully wrote a value for the oversized page in the first place.
Considered (from the issue) and rejected: truncating compiled_truth to
fit under the cap. Silent, position-dependent recall loss, and the byte
cap doesn't line up cleanly with any natural character/token boundary
for UTF-8 content. content_chunks.search_vector already gives full,
untruncated chunk-grain coverage for large pages — truncating a
now-redundant whole-page vector would trade a real bug for a subtler one.
## Test plan
- New test/page-search-vector-overflow.test.ts: a >1MB page (genuinely
diverse tokens — a repetitive lorem-ipsum-style fixture does NOT
reproduce this bug, since to_tsvector's cap is on its DEDUPLICATED
output size, not raw input length) now imports successfully instead of
throwing; remains keyword-searchable via the chunk-grain path; a normal
page's search_vector still carries title signal (not fully inert).
Verified the test is meaningful both directions: fails with the exact
reported error on the pre-fix trigger (git-stashed the fix, reran,
confirmed byte-for-byte match: "string is too long for tsvector
(2684620 bytes, max 1048575 bytes)"), passes with the fix restored.
- Updated fts-language-migration.serial.test.ts: removed an assertion
that configurable_fts_language (v123) is LATEST_VERSION — that was only
ever true until the next migration landed; the codebase's own pattern
elsewhere for this (migrate.test.ts) uses toBeGreaterThanOrEqual, not
exact-match, for exactly this reason.
- bun run typecheck clean, bun run verify 31/31 green.
- test/page-search-vector-overflow.test.ts (3/3),
test/reindex-search-vector.serial.test.ts,
test/fts-language-migration.serial.test.ts, test/migration-v120.test.ts,
test/sync.test.ts, test/bootstrap.test.ts, test/migrate.test.ts — 254
total, 0 fail, no regressions.
## Design consultation
Investigated jointly with masa-codex (async design review) before
implementing — their read of the codebase (content_chunks.search_vector
already covers keyword search; pages.search_vector's compiled_truth feed
is the only overflow-prone, effectively-dead write) matched independent
verification and shaped the "remove from trigger" fix over the issue's
alternative options (chunk-grain rebuild — largely already exists; input
truncation — rejected above; ledger-entry-only — insufficient alone,
since the page upsert failing in the same transaction also loses the
chunk write, so checkpoint advancing without this fix would make the
content permanently unsearchable, not just delayed).
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
resolveGbrainCliPath() (both copies — src/commands/autopilot.ts and the
inlined duplicate in src/core/brain-repo-durability.ts) called `execSync`
without an explicit `env`, relying on default inheritance. Under Bun,
execSync/execFileSync snapshot process.env at BUN'S OWN STARTUP, not at
call time — a runtime PATH mutation (dotenv/config loading, wrapper-script
env sourcing, etc.) happening after Bun boots but before this call is
invisible to `which gbrain` unless the current env is forwarded
explicitly.
This is a known, already-precedented Bun quirk in this exact codebase:
spawn-helpers.ts's detectTini() was already fixed for the identical
symptom with the identical one-line fix (`env: process.env`), with a
comment explaining the mechanism — this call site was simply missed.
Matches the reported symptom precisely: "which gbrain" resolves fine when
run standalone (a fresh Bun process, no prior env mutation to hide), but
throws specifically from inside autopilot's managed-worker spawn path
(src/commands/autopilot.ts:416, guarded by `spawnManagedWorker` — Postgres
engine + minion_mode enabled), which fires after config/dotenv loading has
already run in that process. Impact per the report: this silently
degrades to no worker ever picking up queued jobs (including embed jobs),
with `gbrain doctor` only showing a growing "N stale chunks" warning that
reads like an ordinary backlog rather than a broken worker.
Also improved the throw-path error message to include the actual
PATH/execPath/argv[1] values observed at failure time, so a future report
doesn't require guessing at what the process actually saw.
Not fixed here (documented as a separate, smaller finding): a third call
site with the identical missing-env pattern exists in
src/core/claw-test/runners/openclaw.ts ('which openclaw'). Left out of
scope for this PR, which is specifically about #2747's reported symptom;
worth a small follow-up.
Verification: bun run typecheck clean, bun run verify 31/31 green,
test/autopilot-resolve-cli.test.ts 4/4 pass (existing coverage, no
regressions — a genuinely-simulated Bun-env-snapshot race isn't
reproducible in a same-process unit test, and this codebase's own
convention explicitly avoids mock.module for child_process per
doctor-orphan-ratio.test.ts's stated test-isolation rule, so this PR
relies on the fix's precedent-match + existing coverage rather than a new
mock-based regression test).
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* fix(jobs): retry resets started_at/attempts_made/attempts_started (#2783)
`gbrain jobs retry` re-queued a dead job by resetting status/error_text/
locks/delay/finished_at, but left started_at, attempts_made, and
attempts_started untouched. On re-claim, claim()'s
`started_at = COALESCE(started_at, now())` preserved the ORIGINAL
first-claim timestamp instead of re-stamping it. handleWallClockTimeouts()
anchors on `now() - started_at`: a retry issued more than timeout_ms * 2
after the original claim was immediately dead-lettered again in under a
second, with attempts_made already past max_attempts — making retry
useless for exactly the case it exists for (recovering work after an
outage that outlasted the job's timeout).
An explicit `jobs retry` is an operator asserting "run this fresh", so
retryJob now also clears started_at (NULL, re-stamped on next claim) and
resets attempts_made/attempts_started to 0.
Two new tests: direct assertion that retry resets all three columns, and
a full repro of the reported bug (wall-clock-killed job retried long
after the original claim now survives re-claim instead of being
immediately re-killed).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(jobs): also reset stalled_counter on retry (#2783)
Codex review round 1 found the fix was incomplete: a job dead-lettered by
stall exhaustion (handleStalled() at stalled_counter + 1 >= max_stalled)
retained its exhausted stalled_counter across retry. The retried job's
very first lock expiry after re-claim would immediately re-satisfy the
dead-letter threshold, contradicting the same "run this fresh" intent the
started_at/attempts reset already established.
New test mirrors the existing wall-clock repro: exhaust the stall budget
via two real handleStalled() calls, retry, confirm one more stall now
requeues instead of dead-lettering again.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
resolveTakesSourceId() caught every error from resolveSourceId() and fell
back to undefined, which restores the pre-#2698 unscoped (cross-source)
slug lookup for takes add/update/supersede/resolve. resolveSourceId()
only ever throws when a source was explicitly in play (an invalid or
unregistered GBRAIN_SOURCE, a .gbrain-source dotfile pointing at a
source that doesn't exist, or a genuine DB error) — it never throws for
"nothing configured," which resolves cleanly to the seeded 'default'
source. So swallowing the error had no legitimate case to protect and
only reintroduced the cross-source write bug on any resolution failure.
Let it propagate so the write is blocked instead.
Adds regression coverage for both the unchanged happy path (no source
configured resolves cleanly) and the newly fail-closed path (an
unregistered GBRAIN_SOURCE blocks the write instead of falling back to
an unscoped lookup).
extractTakesFromPages hardcoded anthropic:claude-haiku-4-5 as the classifier
model. On an OAuth/local-only install (no ANTHROPIC_API_KEY; chat routed
through a gateway model) every takes extraction died with llm_unavailable —
the takes layer silently never populated and the takes_count health check
stayed red despite a working configured chat_model.
Resolution is now `opts.model || getChatModel()` — the same file-plane
gateway-config idiom enrich.ts uses — NOT engine.getConfig('chat_model')
(the DB config plane), keeping model routing on the single config plane the
rest of the codebase reads. Explicit opts.model still wins; unconfigured
installs fall through to the gateway's DEFAULT_CHAT_MODEL.
Adds a regression test that pins the file-plane read: a conflicting DB-plane
config.chat_model row is ignored, the gateway-configured chat_model is used
when opts.model is unset, and explicit opts.model wins. Verified the
file-plane test fails against the pre-fix code.
Takeover of #2997 by @Nazim22 with the model read moved from the DB config
plane to the file-plane gateway idiom.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Nazz <nazim.mj@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Bun SQL double-encodes ::jsonb[] array binds on the Postgres engine: every
edge_metadata element landed as a jsonb string scalar instead of an object,
so the resolver's `edge_metadata || jsonb_build_object(...)` UPDATE produced
a jsonb array and resolved_chunk_id was never readable — code_callers /
code_callees / code_blast returned nothing on Postgres-engine brains while
the resolver logged edges_resolved > 0. PGLite was unaffected (per-row
placeholders already).
Rewrites both inserts (code_edges_chunk, code_edges_symbol) to per-row
$n::text::jsonb placeholders via sql.unsafe — the same shape executeRawJsonb
and the PGLite engine use.
Adds the DATABASE_URL-gated Postgres regression test this class requires
(PGLite cannot reproduce it): asserts jsonb_typeof(edge_metadata) = 'object'
for resolved + unresolved inserts and that the resolver-style || UPDATE
keeps object shape. Verified the test fails 3/3 against the pre-fix code
and passes with the fix.
Takeover of #2968 by @zsimovanforgeops with the missing regression test added.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Forge (Ron) <forge@zsimovan.dev>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The top-level `gbrain --help` advertised `sync --install-cron` since the
line was first added, but `src/commands/sync.ts` never parsed or handled
the flag — `gbrain sync --install-cron` silently ran an ordinary one-off
sync instead of installing anything, manufacturing false confidence in
the exact durability layer operators reach for it to secure.
git blame shows the line was introduced once (v0.42.29.0 help-text
scaffold) and never touched again — no design intent to recover.
Implementing it would also compete with autopilot, which already owns
this job: `gbrain autopilot --install` runs a self-maintaining daemon
(sync+extract+embed) on a schedule, including a per-source freshness
check that submits `sync` jobs on its own interval. A second, separate
sync-only cron would be a competing scheduler outside the D10
cycle-lock invariant that already keeps autopilot's own targeted-submit
and full-cycle paths from double-processing.
Removed the misleading line and pointed sync's --watch entry at
`autopilot --install`, mirroring the existing `dream` command's
"See also: autopilot --install (continuous daemon)." pattern one
section below. Added regression coverage to
test/cli-help-discoverability.test.ts asserting the help text no
longer promises install-cron and does point at autopilot.
gateway.chat() requested cacheSystem:true but never got a system-prompt
cache hit on single-turn callers (page-summary, skillopt, enrich): the
call-level providerOptions.anthropic.cacheControl is real (it becomes
Anthropic's documented top-level "auto-cache the last cacheable block"
shorthand via @ai-sdk/anthropic 3.0.47+), but for a stable system prompt
paired with a different user message every call, "the last cacheable
block" is that ever-varying tail -- every call writes a fresh cache
entry there and never reads a prior one.
Fix: pass system as a SystemModelMessage object (ai's documented shape
for attaching provider options to the system block) carrying its own
providerOptions.anthropic.cacheControl when cacheSystem is requested,
and mirror the same marker onto the last tool def (Anthropic caches
everything up to and including the last cache_control block it sees).
The call-level marker is kept, not removed -- it still gives toolLoop()'s
growing multi-turn conversation a rolling cache breakpoint on each
turn's tail. All three markers now derive from one canonical
cacheControlValue computed after provider_chat_options config merging,
so a configured TTL override (e.g. ttl: '1h') applies consistently
instead of only reaching the call-level marker.
Verified red-before-fix by stashing the gateway.ts diff and confirming
the new assertions fail on unfixed code, then restoring.
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* fix(sync): self-heal a never-git-initialized default brain dir (#2964)
The dream cycle's sync phase throws unconditionally on a legacy
sync.repo_path-anchored default brain dir that was never git init-ed
(predates git-backed sync, or was rsync'd without its .git), failing
every nightly run with no recovery. doctor's sync_freshness/
sync_consolidation checks report "ok" for this exact brain, but only
because they query the sources table (0 rows for a legacy default
brain) — a coincidental false-negative, not a real diagnosis.
Self-heal by git-initializing the dir and capturing the current
on-disk state as the sync baseline, scoped to !opts.sourceId only —
gbrain owns this directory outright, unlike a registered local source
(sources add --path, no --url) which is the user's own external
directory and should keep failing loudly.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): dry-run no-write contract, unborn-HEAD recovery, no-gpg-sign (#2964)
Codex review on the initial self-heal patch (b1671ee) found 3 real gaps:
- P1: the self-heal ran even under --dry-run, mutating the filesystem
during what's documented as a preview-only command. Gated the whole
self-heal (both discoverGitRoot and the headCommit read) on
!opts.dryRun, same as the existing !opts.sourceId ownership check.
- P2: if `git init` succeeded but the process died before the baseline
commit landed, the next run's discoverGitRoot would succeed (`.git`
exists) and skip recovery entirely, permanently wedging on "No
commits in repo" forever. Added the same self-heal at the
`git rev-parse HEAD` catch site, sharing a new createSyncBaselineCommit
helper with the discoverGitRoot catch.
- P2: the baseline commit inherited the operator's global
commit.gpgSign, which can block headless cron/launchd runs on an
unavailable signing agent/pinentry. Added --no-gpg-sign.
Two new tests cover dry-run no-mutation and unborn-HEAD recovery.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): restrict git auto-init self-heal to the anchor-resolved path only (#2964)
Second Codex review round (a10eeab) found the ownership check still too
loose:
- P1 (security): !opts.sourceId alone isn't proof gbrain owns repoPath.
jobs.ts's `sync` job handler leaves sourceId undefined whenever
job.data.repoPath doesn't match a registered source's local_path, so
an admin-scope submit_job({name:'sync', data:{repoPath}}) MCP call
could point the self-heal at an arbitrary directory and have it
silently git-init + commit + ingest it. Gated both self-heal sites on
!opts.repoPath too — only the path resolved from gbrain's own
sync.repo_path anchor (never a caller-supplied one) is eligible.
- P2: the unborn-HEAD recovery site calls discoverGitRoot, which walks
UP from repoPath and can resolve to an ANCESTOR repo for a
--src-subpath/subdir-as-repoPath sync with an unborn HEAD. Committing
there would `git add -A` sibling files well outside the sync scope.
Added a check that gitContextRoot === realpathSync(repoPath) before
self-healing; refuses (falls through to the original error) otherwise.
Tests rewritten to exercise the true self-heal-eligible path (anchor
config via engine.setConfig('sync.repo_path', dir), no repoPath/sourceId
passed) instead of an explicit repoPath, plus a new test asserting a
caller-supplied repoPath with no sourceId still throws.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): prove self-heal ownership by anchor VALUE, not field presence (#2964)
Third Codex review round (613ad5b) found the previous round's fix broke
the very call site it was meant to repair, plus a second scope gap:
- P1 (critical): !opts.repoPath rejected self-heal on the REAL production
callers too. runPhaseSync (dream cycle's sync phase, cycle.ts) always
passes `repoPath: brainDir` explicitly after resolving it upstream, and
the CLI's bare `gbrain sync` resolves sourceId='default'. Both made the
ownership gate rethrow, leaving `gbrain dream` and `gbrain sync`
wedged on the exact non-git legacy brain this fix targets — only
synthetic callers that omitted both fields ever healed.
Fixed by proving ownership by VALUE instead of by field absence: a new
isAnchorOwnedSyncPath() re-reads gbrain's own persisted
sync.repo_path config and requires the resolved repoPath to equal it
exactly, regardless of whether the caller passed it explicitly or let
it default. An attacker-supplied arbitrary path (e.g. via
submit_job({name:'sync', data:{repoPath}})) only self-heals if it
happens to already equal gbrain's own anchor — which is the
legitimate case, not an escalation. opts.sourceId and opts.srcSubpath
still disqualify unconditionally (registered/subpath-scoped syncs are
a different ownership context).
- P2: a --src-subpath sync with an unborn parent-repo HEAD would commit
the whole ancestor root, capturing sibling files outside the scope.
isAnchorOwnedSyncPath's opts.srcSubpath check closes this; the
existing gitContextRoot === repoPath check stays as defense in depth.
- P2: manageGitignore's "warn and return" contract (a deliberate side-
effect that must never kill the sync job for its OTHER callers) meant
a broken gbrain.yml or unwritable .gitignore would silently let the
baseline `git add -A` commit db_only content. createSyncBaselineCommit
now recomputes db_only exclusion directly from loadStorageConfig and
passes it to `git add` as pathspecs, independent of the .gitignore
write's success — true fail-closed. (A redundant pathspec exclude for
a path .gitignore ALREADY covers makes git's -A bail with "paths
ignored, use -f" even though the negation is correct, so each dir is
check-ignore'd first and only pathspec-excluded when NOT already
covered.)
Tests rewritten around the anchor-VALUE model: the critical regression
case (explicit repoPath matching the anchor still heals — the exact
scenario Codex proved was broken) plus a true negative (a caller-supplied
path that does NOT match the anchor still throws), --src-subpath refusal,
and db_only fail-closed exclusion.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): allow default-source ownership, realpath compare, generous timeout, --no-verify (#2964)
Fourth Codex review round (9c1d461) found the previous round's ownership
gate still didn't match the REAL installed-brain shape, plus 3 more gaps:
- P1 (critical): rejecting all non-empty opts.sourceId meant self-heal
still never fired on a real brain. Migration sources_table_additive
seeds a 'default' source row whose local_path mirrors sync.repo_path
on every brain that's run it (virtually all of them), so
resolveSourceForDir (dream cycle) and bare `gbrain sync` both resolve
sourceId:'default' in practice, never undefined. isAnchorOwnedSyncPath
now permits sourceId undefined OR exactly 'default' (gbrain's own
bootstrap identity, never something a caller names) and proves
ownership by rereading the LIVE anchor for that same identity
(sources.default.local_path vs config.sync.repo_path).
- P2: compared raw anchor/repoPath strings, so a cosmetic difference
(trailing slash, ..) between the stored anchor and dream.ts's
path.resolve()-normalized brainDir would defeat the match. Now
realpath-compares both sides (fail-closed on ENOENT/dangling).
- P2: the shared git() helper's 30s timeout could abort the baseline
`git add -A` on a large legacy brain mid-way, after `git init` already
created `.git` — leaving an unborn repo every subsequent sync would
retry and time out identically forever. Added an optional timeoutMs
param (default unchanged at 30s); the baseline add call uses 10min.
- P2: the baseline commit could trigger an operator's global
core.hooksPath/init.templateDir hooks (pre-commit/commit-msg),
breaking headless recovery if those hooks need project tooling or
prompt. Added --no-verify.
Tests: rewrote the mis-scoped "registered source" test (it used
sourceId:'default', which is now correctly permitted) into two — a new
regression test proving sourceId='default' + matching local_path heals
(the actual production shape), and a corrected non-default-sourceId
refusal test.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): defer .gitignore write past first import, rebuild index, fail closed on unparseable db_only (#2964)
Fifth Codex review round (c17cd23) found the baseline-commit helper
interacting badly with the pre-existing db_only storage-tiering feature:
- P1 (data loss): createSyncBaselineCommit called manageGitignore BEFORE
performFullSync's collectSyncableFiles ran. collectSyncableFiles
enumerates via `git ls-files --cached --others --exclude-standard`, so
writing db_only entries into .gitignore first would silently exclude
those pages from the DATABASE, not just from git — on a brain's very
first sync. This is the exact bug class runSync's existing "manage
.gitignore ONLY on successful sync" ordering (this file, ~line 4540,
itself a prior Codex P1 fix, comment literally says so) was written to
prevent — my new code reintroduced it in a different spot. Fix: stopped
calling manageGitignore inside the self-heal at all. db_only exclusion
for the COMMIT still happens via the existing pathspec computation
(independent of .gitignore); .gitignore itself gets written by the
already-existing post-sync flow once this sync completes, same as any
other sync.
- P1 (data leak): the unborn-HEAD recovery site can reach
createSyncBaselineCommit with a repo whose INDEX already has entries
staged from some prior operation (manual `git add`, interrupted
workflow) before gbrain's self-heal ever touched it. `git add -A`
only adds/updates — it doesn't drop an already-staged path our
exclusion pathspecs now want excluded. Added `git read-tree --empty`
to reset the index before staging (no-op on a freshly-`git init`-ed
repo, whose index is already empty).
- P2: loadStorageConfig warns-and-returns an EMPTY config (not a throw)
for syntactically-valid-but-unsupported YAML (e.g. flow-style
`db_only: [dir/]` — the narrow custom parser only handles block-style
lists), which would silently resolve zero exclusions from a gbrain.yml
that clearly intended some. Added a sniff-test: if gbrain.yml exists
and mentions db_only but nothing resolved from it, refuse the baseline
commit rather than guess "genuinely empty" vs "syntax silently
ignored" (git init may already have run by this point — same "unborn,
retry on next sync" recovery path handles it, and will hit this same
guard again until the user fixes gbrain.yml).
Tests: a positive regression proving db_only markdown IS imported into
the DB on first sync (the actual data-loss scenario), the sniff-test
refusal, and the stale-staged-content-gets-dropped case for the index
rebuild.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): post-heal .gitignore write, supabase_only alias, honor abort signal (#2964)
Sixth Codex review round (e687913), 3 P2s:
- Dream-cycle callers (cycle.ts:runPhaseSync) invoke performSync directly
and never run runSync's CLI-only post-success manageGitignoreAtGitRoot.
A brain self-healed only via the dream cycle would have db_only content
correctly excluded from the baseline commit (createSyncBaselineCommit's
pathspec exclusion) but no .gitignore ever written, leaving the user's
own future manual git add/commit unprotected. Added
performFullSyncAndMaybeGitignore, a thin wrapper around the 3
post-self-heal performFullSync call sites that writes .gitignore
(same success-status gate runSync already uses) only when didSelfHeal
is true — a no-op for the normal path, which still relies on runSync
exactly as before.
- The fail-closed sniff-test only checked the canonical `db_only` key;
the deprecated-but-still-supported `supabase_only` alias (same
keep-out-of-git semantics) could silently bypass it. Now checks both.
- Self-heal didn't check opts.signal?.aborted before starting the
(now up to 10-minute) git init + baseline commit, so a cancelled sync
could still mutate disk and overrun its budget instead of returning
partial. Added the check at both self-heal sites, before any git
operation runs.
New test proves .gitignore gets written after a bare performSync call
(no runSync wrapper) — the actual dream-cycle shape.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): neutralize a leftover .gitignore during the self-heal first sync (#2964)
Seventh Codex review round (27337b0) ran an actual repro and caught the
primary motivating scenario still broken: a brain rsync'd from another
machine without its .git can retain that machine's old auto-managed
.gitignore. collectSyncableFiles (inside performFullSync) enumerates via
`git ls-files --exclude-standard`, so a leftover db_only ignore rule
would silently omit those pages from THIS first sync's DATABASE import —
the same bug class the round-6 ordering fix prevented for a .gitignore
gbrain would have written itself, just triggered by a pre-existing file
this time.
Fix: performFullSyncAndMaybeGitignore now neutralizes any existing
.gitignore for the duration of the one first-sync call — read, delete,
restore byte-for-byte immediately after (even on error) — before
manageGitignore re-merges the managed db_only block onto the restored
original content. This matches exactly what a truly fresh brain with no
.gitignore at all already does on its first sync (nothing to suppress
collection there either); db_only content stays out of the git COMMIT
independently via createSyncBaselineCommit's pathspec exclusion, which
never depended on .gitignore.
Test proves both halves: db_only markdown IS imported despite a leftover
ignore rule, AND the user's own unrelated .gitignore lines (e.g.
.DS_Store) survive the restore intact.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): simplify — drop db_only-import machinery, isolate hooks fully (#2964)
Eighth Codex review round (54c1e6f) found MORE problems with round 7's
.gitignore-neutralization fix (deleting the whole file loses the user's
own unrelated ignore rules; a multi-sync retry scenario could silently
skip a still-broken db_only file while advancing the bookmark) plus 2
more issues in existing code. Rather than patch those too, stepped back
and checked the actual documented semantics of db_only
(docs/storage-tiering.md): it's for "bulk machine-generated content...
written to disk as a local cache" — DB is the source of truth, disk is a
cache populated FROM the DB (`export --restore-only` restores it), never
the other way. Nothing in the docs says `gbrain sync`'s git-diff-based
file collection is how db_only content is supposed to reach the
database — that's ingest-specific tooling's job. Confirmed directly:
`loadStorageConfig` returns the byte-identical `{db_tracked:[],
db_only:[]}` for a malformed flow-style array AND a literal empty
`db_only: []`, so rounds 6-7's "ensure db_only markdown gets imported on
this first sync" chase was solving a problem outside sync's actual scope
in the first place, on an increasingly complex, adversarially-discovered-
edge-case foundation.
Reverted: performFullSyncAndMaybeGitignore (the wrapper + didSelfHeal
tracking + .gitignore neutralize/restore dance + post-success
manageGitignore call). After self-heal, import and any subsequent
.gitignore management now behave EXACTLY like any other brain,
self-healed or not — runSync's existing post-success
manageGitignoreAtGitRoot covers the CLI path identically either way; the
dream cycle not calling it is a separate, pre-existing characteristic of
the dream cycle in general (applies equally to an already-git-initialized
brain going through the same path), not something this fix introduces.
Kept (still correct, self-contained, don't depend on the reverted
machinery): createSyncBaselineCommit's pathspec-based db_only exclusion
for the COMMIT itself (matches the documented "not committed to git"
requirement), the fail-closed sniff-test guard (now documents its
known, structurally-unavoidable false-positive on a genuinely-empty
`db_only: []` — the trade-off is deliberate: low-cost, self-resolving
false positive vs. high-cost, hard-to-undo false negative), the index
rebuild, and the 600s add timeout.
Improved (round 8, P2): hooks isolation. --no-verify only skips
pre-commit/commit-msg; added `-c core.hooksPath=/dev/null` for the
baseline commit, which disables prepare-commit-msg and post-commit too
(the latter runs synchronously inside the same git invocation and could
otherwise hang past the timeout without even being the slow step).
Tests: removed the 3 that exercised the reverted db_only-import
machinery; the remaining 12 (ownership, dry-run, index rebuild, sniff
test, commit-exclusion, unborn-HEAD recovery) are unaffected by the
simplification.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): unconditional db_only exclusion, literal pathspecs, precise sniff test (#2964)
Ninth Codex review round (a6e07f6):
- P1: the check-ignore pre-filter (skip pathspec-excluding a dir already
covered by .gitignore) could be defeated by a pre-existing .gitignore
that ignores a db_only tree with a wildcard but re-includes a child via
negation (e.g. `private-cache/*` + `!private-cache/index.md`) —
check-ignore on the directory still reports "ignored", so the filter
skipped the pathspec exclusion, and `git add -A` staged the re-included
child anyway. Fixed by making exclusion unconditional: every db_only
dir is always pathspec-excluded now, never pre-filtered against
.gitignore state at all — our own pathspec doesn't consult .gitignore,
so no .gitignore content (negated or not) can defeat it. The advisory
"paths ignored... use -f" error this can now trigger when a dir IS also
already .gitignore'd (verified: git still stages everything else
correctly despite the nonzero exit) is caught and swallowed by matching
its exact stderr text; anything else rethrows.
- P2: `:!dir` pathspec shorthand reinterprets a dir name that itself
starts with a pathspec magic character (e.g. `:private/`) instead of
excluding it literally. Switched to `:(exclude,literal)dir`.
- P2: the fail-closed sniff-test's bare substring search on gbrain.yml's
raw content could trip on a comment or unrelated prose mentioning
"db_only" even when there's no real storage section at all, refusing
self-heal forever on an unrelated false positive. Now requires an
actual YAML key line (`db_only:`/`supabase_only:`, trimmed, ignoring
`#` comments) — the round-8-documented "genuinely empty db_only: []"
false positive is unchanged and remains an accepted trade-off (still
structurally indistinguishable from unsupported syntax at the
loadStorageConfig API boundary), but comment/prose mentions no longer
false-positive.
Not fixed (deliberately, documented trade-off — see PR description):
Codex's other P1 this round (refuse baselining when other git refs/
history exist alongside an unborn HEAD) is a narrow, non-destructive
scenario — self-heal only ever acts on the current branch ref when it's
provably commit-less, never touches or deletes any other ref (remote-
tracking, other branches), so at worst it creates a possibly-unexpected
extra commit on an otherwise-empty branch the user hadn't checked out
yet. Chasing it further trades diminishing real-world risk reduction
against unbounded scope growth in what's fundamentally still the
self-heal fix from round 1.
Two new tests: unconditional exclusion despite a matching pre-existing
.gitignore (proves the advisory-swallow path), and a comment-only
gbrain.yml no longer false-positives the sniff test.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* fix(cycle): budget the patterns subagent from remaining job time, not a fixed constant (#2781)
The autopilot-cycle job gets an interval-derived timeout stamped at submit,
but the patterns phase submits its subagent with a fixed 30-min job timeout
and waits up to 35 min — one phase's worst case exceeds ANY interval-derived
budget <= 35 min, so the parent job dead-letters mid-patterns and the tail
phases (consolidate -> schema-suggest) starve for days (#2781, the deeper
half left open by the #2852 dispatch-floor fix).
- MinionJobContext.deadlineAtMs: absolute deadline from the claim-time
timeout_at stamp (the DB ground truth handleTimeouts() sweeps against;
re-stamped on every claim so retries get a fresh budget). Null when the
job has no per-job timeout.
- worker: the per-job abort timer now derives its delay from timeout_at
when present, so the in-process timer, the DB sweeper, and the
handler-visible deadline agree on ONE absolute instant.
- autopilot-cycle + autopilot-global-maintenance handlers thread
deadlineAtMs into runCycle; CycleOpts carries it to the patterns phase.
- patterns: clampSubagentBudgets() derives BOTH the child job timeout and
the wait timeout from the same child deadline (parent deadline minus a
60s stop-margin reserve — enough for the wait poll + force-evict grace
+ cleanup, deliberately NOT a promise that tail phases complete). Under
a 2-min minimum the phase skips honestly (insufficient_cycle_budget)
instead of submitting a guaranteed-kill LLM call; the next cycle
retries with a fresh budget.
- Direct callers (gbrain dream) pass no deadline and keep the configured
timeouts unchanged.
Follow-up (separate PR): synthesize has the same shape plus sequential
per-child waits that accumulate N x subagent_wait_timeout_ms past any
parent budget; it needs per-wait remaining-time recomputation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RaNDjecPPvhC7LRqkxkhGF
* fix(cycle): address review — cancel timed-out patterns child; thread deadline through phase-wrapper handlers
- P1: the child's timeout_ms clock starts at ITS claim, so a queued child
could outlive the parent deadline the wait was clamped to. On wait
timeout, cancelJob strips it (waiting -> cancelled; active -> lock
stripped, worker abort fires next renew tick).
- P2: makePhaseHandler (standalone patterns/synthesize/... minion jobs)
now threads job.deadlineAtMs into runCycle like the autopilot handlers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RaNDjecPPvhC7LRqkxkhGF
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Adds an EU-hosted provider covering embedding, expansion and chat on one
OpenAI-compatible endpoint (https://api.mistral.ai/v1), so a brain that must
stay inside EU jurisdiction does not need a US hop for any AI touchpoint.
Every field is measured against the live API, not copied from docs:
- mistral-embed is fixed 1024 dims and accepts no dimension parameter.
Both spellings are rejected: {"dimensions": N} returns 400 extra_forbidden,
{"output_dimension": N} returns 400 "does not support output_dimension".
The generic openai-compatible branch of dimsProviderOptions() already falls
through to `return undefined` for these model ids, so nothing is emitted.
Same contract as voyage-4-nano, pinned by a negative assertion in the test.
- max_batch_tokens 65536: a 65,286-token batch is accepted, 66,960 returns
400 code 3210 "Too many tokens overall, split into more batches."
- chars_per_token 2: the value is a DIVISOR in splitByTokenBudget()
(estTokens = text.length / charsPerToken), so lower is the conservative
direction. The module default of 4 assumes English prose; a German-language
corpus measured 3.58 chars/token, which the default overshoots toward
overflow.
codestral-embed is deliberately left out: it returns 1536 dims, and a
touchpoint carries a single default_dims. Listing it under a 1024 declaration
is the mixed-dim case embedding-dim-check.ts exists to catch.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
configureFromEnv() hand-assembled its own AIGatewayConfig instead of calling
buildGatewayConfig(), the single seam that folds file-plane API keys
(openrouter_api_key, zeroentropy_api_key, ...) into the gateway env. That let
`gbrain providers list`/`test` report a provider as missing env even when it
was correctly set in ~/.gbrain/config.json and the real gateway path resolved
it fine. Both configureFromEnv() and runList() now build their env through
buildGatewayConfig() (falling back to a bare process.env passthrough
pre-init), matching what init-provider-picker.ts already does.
The one-shot CLI teardown drains fire-and-forget background sinks with a
hardcoded 2s per-sink budget (DEFAULT_DRAIN_TIMEOUT_MS). That budget
assumes a sub-second cloud chat provider; on a self-hosted provider (e.g.
an ollama model at 10-20s per completion) a facts:absorb extraction can
never finish inside it, so every one-shot CLI exit — sync timers
especially — aborts the in-flight chat with
'pipeline_error: The operation was aborted', and the same touched pages
retry-and-abort on every subsequent sync. Facts from those pages silently
never land, and doctor's facts_extraction_health warns permanently.
Fix: resolveDrainTimeoutMs() — GBRAIN_DRAIN_TIMEOUT_MS env override
(same env-only escape-hatch pattern as GBRAIN_TEARDOWN_DEADLINE_MS and
GBRAIN_FLUSH_GRACE_MS) over the 2000ms default. Explicit drainTimeoutMs
from a call site still wins; computeTeardownDeadlineMs already computes
the backstop from the resolved value, so the deadline scales with it.
Garbage/zero/negative env values fall back to the default.
Tests: default, env override, garbage/zero/negative fallback,
finishCliTeardown drains with the env-resolved budget, explicit opts
still win over env.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
`extract-conversation-facts --dry-run` promises "no DB writes, no
checkpoint advance" (help text) and correctly skips the fact INSERTs,
orphan delete, checkpoint advance, and receipt page. But
writeRunReceiptAndRollup was called unconditionally at both exit paths,
and its upsertExtractRollup always UPSERTs a row into extract_rollup_7d
("ALWAYS fire so doctor's extract_health sees the cycle ran") — so a
dry run mutates the DB.
Gate both writeRunReceiptAndRollup call sites on !dryRun. The writer
returns void and its only non-rollup action (the receipt page) is
already suppressed in dry-run via facts_inserted > 0, so gating at the
call site skips nothing else. Mirrors the existing !dryRun guards on
the fact-insert / checkpoint / audit paths.
Regression test: a dry run leaves extract_rollup_7d empty. Fails before
(row count 1), passes after. Hermetic PGLite + stubbed transports, no
live LLM.
Note: --dry-run still calls the extractor (LLM) by design — facts_extracted
is the reported "would extract N" preview count; left unchanged.
setupDB() TRUNCATEs every data table on whatever DATABASE_URL points at,
and run-e2e.sh deliberately preserves an exported DATABASE_URL — one
stray environment variable away from wiping a production brain, with no
guard of any kind (found during an independent review, 2026-07-18).
assertSafeE2eDatabaseUrl (pure, unit-tested) now runs before any
connection: allowed when the database name carries "test" as a word
segment (the gbrain_test convention used by CI and
.env.testing.example), or when GBRAIN_E2E_ALLOW_DB names the exact
database intentionally. Refusal is loud and actionable. 7 unit tests,
no DB required.
Co-authored-by: Aleksei Razsadin <razsoc.01@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Pages were unreachable by their own exact titles: FTS indexed only chunk
body text while the title-weighted pages.search_vector (GIN-indexed since
its introduction) was never queried by any search path, and
websearch_to_tsquery AND-at-chunk-grain semantics meant one non-matching
token zeroed keyword recall with no fallback — long or acronym-bearing
titles (e.g. "IAWG ... AAR-LL deck") fell through to the vector arm alone
and missed.
- searchTitles (both engines): page-grain candidate arm over
pages.search_vector (title 'A' + compiled_truth 'B' + timeline 'C'),
ts_rank_cd ranked, representative-chunk LATERAL join, full filter
parity with searchKeyword (visibility, soft-delete, source grants,
hard-excludes, dates, types); fused as a weighted RRF list at the
keyword arm's intent-effective k on all three hybrid return paths;
fail-open with warnOncePerProcess. No schema changes — the index
already existed, dark.
- AND->OR one-retry fallback for the keyword arm, gated behind
SearchOpts.orFallback (only hybridSearch opts in; countMentions, link
resolution, eval, and keyword-only MCP callers keep the strict-AND
contract). Refused for queries carrying websearch operators (negation,
quoted phrases). searchTitles carries its own page-grain fallback.
- Lexical arms parallelized (Promise.all) on the main path.
Verified: typecheck clean; 18 hermetic PGLite tests + 2 engine-parity e2e
cases (CI Postgres); consumer regression enrichment 18/0 +
link-extraction 127/0; independent live QA on a 10,664-page brain —
exact-title target miss -> rank 1 (exact_title_match), controls held,
negation/quoted guards proven, strict-consumer contract pinned.
Diagnosed from a 3-lane read-only diagnostic; adversarial review round
closed findings on fallback scope, Postgres test coverage, and operator
handling before this commit.
Co-authored-by: Aleksei Razsadin <razsoc.01@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(search): per-call token-budget meta no longer masks the real cut; restore vacuous search-lite coverage
The search-lite integration tests were structurally vacuous: putPage never
creates chunks and searchKeyword joins content_chunks, so every fixture
query returned zero rows on every machine. The tight-budget cut test's
defensive skip ('keyword search may dedupe by page') silently returned
before its assertions had ever executed anywhere, and the two budget-meta
tests ran against empty result sets.
Restoring the fixture (upsertChunks per page) and hardening the
assertions immediately exposed a real meta bug: with a per-call
tokenBudget, the inner hybridSearch enforces the same resolved budget
(per-call wins in resolveSearchMode) and its meta carries the true
dropped count — but hybridSearchCached re-applies the budget to the
already-cut set and published THAT pass's meta, which always reads
dropped=0. onMeta consumers saw a budget record claiming nothing was
dropped while rows were; telemetry (recorded from the inner meta)
disagreed with the caller-visible meta.
- hybrid.ts finalMeta: prefer innerMeta.token_budget when a per-call
budget is set (outer budgetMeta stays as the fallback and remains the
enforcement for the cache-HIT path, where no inner run exists)
- test fixture: chunk each page (pattern: chunk-grain-fts.test.ts)
- cut test: defensive skip replaced with a hard >=2 precondition;
results non-empty + strictly-fewer-than-unbounded + dropped>0 now
actually execute (revert-checked red on the unfixed meta)
- budget-meta tests: assert non-empty result sets so kept=results.length
can no longer pass vacuously at 0=0
The unbounded 'builder' query returns 2 of 3 fixture pages by design —
dedup Layer 3 caps any single page type at 60% of results and the
fixture is all-person — which the >=2 precondition accommodates.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KFWV7zBZFcmD94Vek6xRDT
* fix(search): cache-HIT budget meta prefers the stored cut record per review; exact-count assertions, mixed-type fixture
- hit path had the symmetric masking (codex P2): the re-application runs
on the already-trimmed stored set and read dropped=0 while the miss
that produced the same result set reported the real cut. Prefer
hit.meta.token_budget unconditionally — tokenBudget is folded into
knobsHash ('tb='), so a hit only ever serves a lookup with the
identical resolved budget as the write and the outer pass can never
cut further (verified against mode.ts; this is why the reviewer's
'outer wins when it drops' branch is unreachable). budgetMeta remains
the fallback for legacy rows stored without a budget record
- new serial test drives a real store-then-hit roundtrip (mocked
embedQuery, real PGLite cache) and pins hit token_budget == miss
token_budget; revert-checked red (dropped=0) on the unfixed path
- lite test: dropped asserted as the exact unbounded-minus-kept count
and used>0 (dropped>0 alone accepts any wrong positive; used<=250
alone accepts a bogus zero), fixture types mixed (person/company/note)
so dedup Layer-3 type-diversity policy no longer shapes the test
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KFWV7zBZFcmD94Vek6xRDT
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(search): classify cache hit/miss in telemetry — hits were invisible, misses unclassified (#2952)
search stats reported 0 hit / 0 miss forever: recordSearchTelemetry fired
only from bare hybridSearch, whose meta never carries a cache field, and a
cache HIT returned from hybridSearchCached before any record at all — so
hit searches also vanished from count/results/tokens/rank-1.
- HybridSearchOpts: internal _telemetryCacheStatus ('miss' | 'disabled')
threaded from hybridSearchCached into the inner hybridSearch (same
pattern as _queryEmbedDeadline), folded into the RECORDED meta only —
onMeta payloads unchanged, count/sum_tokens/budget_dropped/rank-1
behavior byte-identical for the miss/disabled paths
- hit path: record once from hybridSearchCached with the already-built
cachedMeta (cache.status='hit'), post-slice/budget result count, tokens
from the budget pass, and the same rank-1 rule as the inner paths
- bare hybridSearch direct callers (think/gather, brainstorm, enrich,
evals, ...) keep recording exactly as before, with no cache field
- test: serial wiring test drives a real store-then-hit roundtrip through
hybridSearchCached (mocked embedQuery, real PGLite SemanticQueryCache)
and pins the decision matrix (miss / hit / consult-skipped / bare);
revert-checked red on pre-fix source at the miss classification
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KFWV7zBZFcmD94Vek6xRDT
* fix(search): harden cache-hit telemetry per review — mode-gated tokens, embed-failure coverage
- hit-path tokens_estimate now gated on the MODE-resolved budget,
mirroring the inner paths' resolvedMode.tokenBudget > 0 meta condition
(a tokenmax budget-off brain would otherwise record real tokens on hits
but 0 on misses, inflating avg-tokens as the hit rate rises)
- test: exact token-delta parity assertion (hit contributes the same
tokens as the miss that stored the served set) — catches the class of
hit/miss accounting asymmetry the increase-only check accepted
- test: embed-provider-failure flavor of the disabled path (consult
degrades via catch, keyword fallback serves, neither counter bumps)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KFWV7zBZFcmD94Vek6xRDT
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
`jobs list|get` have had remote MCP routing since v0.32, but the CLI
shell still ran connectEngine() before dispatch — on a thin-client
install that fabricates an empty scratch PGLite in the thin-client
GBRAIN_HOME and replays the entire migration chain on every invocation,
before the remote call even runs. Host-only jobs subcommands (work,
supervisor, submit, ...) and `config` did the same instead of refusing.
- cli.ts: dispatch thin-client `jobs list|get` engine-free
(runJobs(null, ...)); refuse the other jobs subcommands with a
pinpoint hint; add `config` to THIN_CLIENT_REFUSED_COMMANDS with a
hint (it reads/writes the host brain's config plane).
- jobs.ts: widen runJobs to accept a null engine, guarded so null can
only reach the MCP-routed list/get branches.
- tests: behavioral (no scratch store created, no migration replay,
refusals carry hints) + source-audit pins in the existing idioms.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
submit_job/get_job return the MinionJob row verbatim; its lifecycle
field is `status` (src/core/minions/types.ts), not `state`. remote ping
typed and read `state`, so every poll saw undefined, the terminal check
never matched, and ping always burned its full --timeout and exited 1
even when the autopilot-cycle had completed — printing
"Job #N is still undefined." on the way out.
Reads fixed to `status`; the ping's own JSON output keys (`state`,
`last_state`) are unchanged for consumers. Source-audit regression test
pins the field reads.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Part of #2946. The hung-COUNT deadline race derived its verdict from a
post-await wall-clock re-check, which races the timer's own drift: on
loaded CI runners setTimeout callbacks fire measurably EARLY relative to
Date.now(), so each padding/boundary adjustment (>=, +1ms) only moved
which wrong status the partial-scan test received ('scanned', then
'partial').
The race's timeout arm now resolves a module-private sentinel; the
sentinel winning IS the deadline verdict (deadlineHit), consulted by the
post-await check without re-reading the clock. A COUNT that resolves
null (failed/absent count) stays distinguishable and does not skip the
scan; a COUNT that resolves slowly without the timer winning is still
caught by the retained wall-clock re-check. The +1ms pad is gone — the
sentinel makes timer drift irrelevant for the hung path.
Verified: partial-scan suite green 8 consecutive runs incl. the new
null-vs-sentinel distinction test.
Claude-Session: https://claude.ai/code/session_01KFWV7zBZFcmD94Vek6xRDT
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The v123 configurable-FTS migration (#2941) logged its completion notice
via console.log. Migrations run lazily inside any command's first DB
connect, so on the nightly heavy run (fresh Postgres service DB) the
line landed as the first line of `gbrain doctor --json` stdout and broke
the fm_wallclock jq parse ("Invalid numeric literal at line 1, column 7",
run 29731426470). runMigrations' contract routes all migration noise to
stderr; move the v123 prints (and the pre-existing v2 slug-rename print)
there.
Also un-vacuous the fm_wallclock harness: its register-source step used
`bun run -e` (bun dumps usage with exit 0 instead of running the code)
and `connect({})` (in-memory), so the source was never registered and
doctor scanned nothing. It now resolves the engine the way the CLI does
and registers the source in the DB doctor actually reads.
Regression test: test/migrate-stdout-clean.test.ts re-runs migrations
from v122 asserting zero stdout writes, plus a source-level guard that
migrate.ts contains no console.log.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(sync): --src-subpath + --exclude for monorepo subdir-source support (#753)
Rebased port of PR #774 onto current master. A single git repo can hold N
logical sources at subdirectories: git operations run at the discovered
repo root (git rev-parse --show-toplevel — worktrees and submodules
resolve natively) while file walking, imports, deletes and renames are
scoped to the subpath. Passing the subdirectory directly as the repo path
(auto-discovery) works through the same code path.
Path-containment guards (the point of the feature):
- NAV-1/NAV-2: the realpath-resolved scope must live inside the
realpath-resolved git root — ../-traversal and symlinked subdirs
pointing outside the repo are rejected before any git op.
- NAV-1 TOCTOU: per-file realpath re-validation during the incremental
import drain and rename reimport; symlink-escape files are recorded as
failures (fail-closed — the bookmark cannot advance past an escape).
- NAV-4: an --exclude set that filters out every candidate warns loudly.
Scoped syncs use git-root-relative slugs + source_path in BOTH the full
and incremental paths (runImport gains slugRoot), fixing the original
PR's full/incremental slug divergence in the auto-discovery flow.
--exclude matches scope-relative paths in both paths; exclusion never
deletes previously-imported pages. The full-sync delete reconcile is
scope-restricted and relativizes against the slug base so a healthy
scoped source can't trip the #2828 mass-delete valve. .gitignore
management resolves to the git root at every call site.
Preserves all master-side sync work since the original branch: the #2828
mass-delete safety valve, #1794 resumable checkpoints + pinned targets,
#1950 stall watchdog, #2335 heartbeat bump, #1970 bookmark reachability,
and the git-ls-files walker (#2315/#2462/#2678, whose symlink/cycle
hardening is untouched).
Co-authored-by: Jeremy Knows <jeremy@veefriends.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: listEverCommittedPaths uses gitContextRoot after #753 root-triple refactor
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Jeremy Knows <jeremy@veefriends.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Squashed superset takeover of the FTS-language trilogy by @rafaelreis-r
(#580 env-var language for query+write side, #581 migration backfill,
#582 gbrain reindex-search-vector), rebased onto current master with the
security review's required fixes applied:
- Migration renumbered v116 -> v123 (master's v116 was already claimed by
code_edges_source_backfill; master is at v122).
- Restored the v120/#1647 search_path hardening: all four CREATE OR
REPLACE trigger-function bodies (migration handler + reindex command)
now carry SET search_path = pg_catalog, public, since CREATE OR REPLACE
resets proconfig and would otherwise strip the hardening on upgrade.
- reindex-search-vector: shared progress reporter (stderr phases
reindex_search_vector.pages/.chunks), id-keyset batched backfill
(5000 rows/UPDATE) instead of single whole-table statements, and
--json no longer bypasses the --yes/TTY confirmation gate.
- Allowlist validation regex + injection tests kept exactly as authored.
- Stale v33/v116 comments swept; docs/guides/multi-language-fts.md
written (README referenced it but no PR added it); llms bundles
regenerated via bun run build:llms.
- Trilogy tests quarantined as *.serial.test.ts (env mutation, per
check-test-isolation).
Verified live on PGLite: fresh init with GBRAIN_FTS_LANGUAGE=portuguese
produces portuguese-stemmed vectors with search_path pinned; the reindex
command retokenizes an english brain to portuguese in place.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Rafael Reis <rafael.reis@contabilizei.com.br>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Takeover of community PR #2387 with the security review's required fixes
applied. Original work by @harrisali0101.
With GBRAIN_RLS_SCOPE_BINDING=1, source-scoped Postgres read methods wrap
their queries in a transaction that binds set_config('app.scopes', $1, true)
(federated sourceIds CSV > scalar sourceId > '*') so operator-managed RLS
policies can filter rows at the SQL layer — defense-in-depth layer 2 under
the mandatory app-layer source filters.
Review fixes on top of the original PR:
- Flag-off is now a TRUE pass-through: no new per-read transaction wrap
(the #1794 PgBouncer pool-exhaustion class). Only the three search
methods keep a transaction when off — exactly the sql.begin() +
SET LOCAL statement_timeout wrap they already had on master — via the
helper's alwaysTransaction option.
- Preserved the PR's latent setseed fix: listCorpusSample pins
setseed() + SELECT to one connection when seeded (alwaysTransaction
gated on opts.seed), so the deterministic path can't split across
pooled connections.
- Updated the two postgres-engine shape tests to pin the new invariant
(search methods route through withScopedReadTransaction with
alwaysTransaction; helper owns the sql.begin(); flag-off path is
callback(this.sql)).
- New behavioral tests (test/postgres-engine-rls-scope.test.ts): flag-off
pass-through, flag-off alwaysTransaction, flag-on set_config emission,
federated > scalar > '*' precedence, and the CSV as a bound parameter
(never interpolated).
- Fixed the helper header comment to match the actual branching behavior.
- Operator docs in docs/ENGINES.md: env var, policy SQL, the
ALTER ROLE ... SET app.scopes='*' default requirement, FORCE ROW LEVEL
SECURITY for owner roles, and the honest caveat about unwrapped paths.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Harris <79081645+harrisali0101@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Takeover of community PR #2387 with the security review's required fixes
applied. Original work by @harrisali0101.
With GBRAIN_RLS_SCOPE_BINDING=1, source-scoped Postgres read methods wrap
their queries in a transaction that binds set_config('app.scopes', $1, true)
(federated sourceIds CSV > scalar sourceId > '*') so operator-managed RLS
policies can filter rows at the SQL layer — defense-in-depth layer 2 under
the mandatory app-layer source filters.
Review fixes on top of the original PR:
- Flag-off is now a TRUE pass-through: no new per-read transaction wrap
(the #1794 PgBouncer pool-exhaustion class). Only the three search
methods keep a transaction when off — exactly the sql.begin() +
SET LOCAL statement_timeout wrap they already had on master — via the
helper's alwaysTransaction option.
- Preserved the PR's latent setseed fix: listCorpusSample pins
setseed() + SELECT to one connection when seeded (alwaysTransaction
gated on opts.seed), so the deterministic path can't split across
pooled connections.
- Updated the two postgres-engine shape tests to pin the new invariant
(search methods route through withScopedReadTransaction with
alwaysTransaction; helper owns the sql.begin(); flag-off path is
callback(this.sql)).
- New behavioral tests (test/postgres-engine-rls-scope.test.ts): flag-off
pass-through, flag-off alwaysTransaction, flag-on set_config emission,
federated > scalar > '*' precedence, and the CSV as a bound parameter
(never interpolated).
- Fixed the helper header comment to match the actual branching behavior.
- Operator docs in docs/ENGINES.md: env var, policy SQL, the
ALTER ROLE ... SET app.scopes='*' default requirement, FORCE ROW LEVEL
SECURITY for owner roles, and the honest caveat about unwrapped paths.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Harris <79081645+harrisali0101@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Five verified-open fixes to the dream/synthesize output family:
- #1586: thread the cycle's resolved sourceId (cycleSourceId) through
runPhaseSynthesize -> SubagentHandlerData.source_id -> subagent tool
OperationContext, and stamp collected refs + summary page with the same
source, so synthesized pages stop landing in 'default'.
- #2415: new config knob dream.synthesize.output_root (default 'wiki',
zero behavior change unless set) drives the synthesize prompt slug
templates, the patterns reflection lookup + prompt, and remaps the
filing-rule allow-list globs. Registered in KNOWN_CONFIG_KEYS.
- #2163: synthesize_concepts writes concept pages through
importFromContent (put_page's parse->chunk->embed pipeline) instead of
bare engine.putPage, so concepts/ pages are chunked + embedded and
reachable by retrieval.
- #2569: stampDreamProvenance persists dream_generated + dream_cycle_date
into pages.frontmatter (JSONB merge via executeRawJsonb) at write time,
so generated pages are DB-queryable and put_page write-through can't
erase the marker.
- #2606: chronicle judge detects stopReason 'length' truncation and
no-JSON-array parse failures as distinct skipped reasons
(judge_truncated / judge_parse_failed) instead of a false terminal
no_events; output cap raised to 4000 and configurable via
chronicle.judge_max_tokens.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Garry Tan <garrytan@gmail.com>
Three verified-open defects, one family: sync silently destroying or
diverging on content it should preserve.
#2404 (P0) — 'ops' was hardcoded in PRUNE_DIR_NAMES (a v0.2.0-era
carve-out), so any path with an ops segment was 'pruned-dir': committed
ops/*.md never imported, and modified ops/* files hit the
unsyncableModified delete loop (whose #1433 guard only spared
'metafile'), silently deleting put-created pages like the bundled
daily-task-manager's canonical ops/tasks on every sync. Fix: remove
'ops' from the prune list (ordinary user content; the vendor/generated
entries stay), and harden the delete loop to also skip 'pruned-dir' —
a page under a pruned dir can only exist via a deliberate put_page.
#2426 (P0) — write-through content stayed DB-only and was deleted by
sync --full. All three compounding bugs fixed:
1. writePageThrough now best-effort commits the artifact (path-limited
git commit) on durability-hardened repos, so the post-commit hook
can push it; result carries committed?: boolean.
2. scripts/brain-commit-push.sh stages+commits BEFORE any pull — the
old fetch+pull-rebase-first order aborted on any dirty tree, so the
helper could never commit a MODIFIED page; brain_push's
rebase-on-reject already handles an advanced remote.
3. The full-sync delete-reconcile partitions stale pages by git
history (listEverCommittedPaths): never-committed source_paths are
DB-only write-through — pages are KEPT and re-exported to the
working tree instead of soft-deleted. Builds on the #2828
mass-delete valve (covers the below-valve cases).
#2607 — the sync --full git ls-files fast path bypassed pruneDir, so a
full pass imported (and resurrected soft-deleted) pages under dot-dirs
and vendored trees that incremental sync excludes. Fix:
isCollectibleForWalker applies the same segment-level pruneDir gate as
classifySync, so full and incremental enumeration agree.
One regression test per defect (all verified failing against master
src): test/sync-ops-pages.serial.test.ts,
test/write-through-commit.serial.test.ts, the #2426 helper-order test
in test/brain-durability-hook.serial.test.ts,
test/sync-reconcile-db-only.serial.test.ts,
test/import-git-fastpath-prune.test.ts.
Fixes#2404Fixes#2426Fixes#2607
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Port of #1837 (mvanhorn) onto current master. The extract_facts cycle
phase unconditionally wipe-and-reinserted every page's fence-owned DB
rows, so re-running a cycle on unchanged content churned rows and — on
the Postgres engine reported in #1781 — accumulated duplicates each run.
The phase now de-dupes extracted facts by the canonical (claim, source)
content key and reconciles the page-scoped DB index: no-op when already
in sync, insert-only for new keys, wipe/reinsert only when stale rows
need cleanup.
Adjustments over the original PR to fit current master:
- preserve #1972's abortSignal threading into the batch embed call
- preserve #1928's excludeSourcePrefixes: ['cli:'] on every wipe, and
exclude cli:-origin conversation facts from the existing-row set so
they neither count as stale (which would force a wipe every cycle)
nor get compared against the fence
Fixes#1781
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Four verified-open issues in the subagent-orchestration family, one PR:
- #1594: dream synthesize subagent job/wait timeouts promoted from
hardcoded 30/35-min constants to config keys
dream.synthesize.subagent_timeout_ms / subagent_wait_timeout_ms.
Approach ported from stale PR #1596 (credit @ai920wisco).
- #2778: add_timeline_entry joins the subagent brain-tool allowlist,
fenced fail-closed by the shared enforceSubagentSlugFence (extracted
from put_page's inline check — same trusted-workspace allow-list /
wiki/agents/<id>/ namespace policy). The per-turn output cap is now
resolveMaxOutputTokens (data.max_tokens → agent.max_output_tokens →
8192, was hardcoded 4096); a max_tokens stop surfaces as
stop_reason 'max_tokens' instead of a silent end_turn, and a
mid-tool-round cap hit injects a truncation note so the model
re-issues the dropped call.
- #2782: patterns phase status now reflects the child outcome —
non-complete outcome with zero writes → fail (PATTERNS_CHILD_<OUTCOME>),
partial writes → warn. Patterns timeouts get the same config-key pair
(dream.patterns.subagent_timeout_ms / subagent_wait_timeout_ms).
- #2113: facts extraction cap is config facts.extraction_max_tokens
(default 4000, was hardcoded 1500); stopReason 'length' is checked,
retried once at 2x the cap, and surfaced on stderr instead of
silently extracting zero facts.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: ai920wisco <ai920wisco@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Absorbs PR #2279's intent (drop the hardcoded ANTHROPIC_API_KEY gate) with
the end-state the gateway world actually wants: the patterns phase now
probes the RESOLVED patterns model through probeChatModel(normalizeModelId)
— the same semantics as think/index.ts and synthesize's makeJudgeClient.
Fixes two misclassifications of the old env gate:
- Non-Anthropic stacks (litellm, deepseek, openrouter, ...) were skipped as
"no upstream" even though the subagent routes them through the gateway
(agent.use_gateway_loop). They now pass the gate; their auth is checked
lazily at dispatch and surfaces in the job outcome.
- Anthropic keys set via `gbrain config set anthropic_api_key` (stdio MCP
launches without shell env) were treated as missing. hasAnthropicKey
inside probeChatModel reads both sources.
Skip reason renames no_api_key → no_provider (carrying the probe's detail).
Both pinning tests updated: the structural test asserts the probe wiring;
the PGLite E2E swaps its env-only helper for the shared hermetic
withoutAnthropicKey (env + config file) so it can't flip to a live LLM call
on a dev machine with a config-file key.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: brettdavies <brettdavies@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Ports the still-unmerged remnant of PR #1891 (#1593 follow-up). Master's
getConfig gained retry-with-reconnect in #1603, but its siblings —
setConfig, unsetConfig, listConfigKeys — still touched `this.sql` bare, so
the first config write/list after a mid-cycle instance-pool teardown threw
the retryable "No database connection" (issue #1678) unhandled instead of
rebuilding the pool.
Adds the connRetry helper from #1891 (same retry+reconnect posture as
batchRetry, but no batch audit JSONL — a config accessor is not a sized
batch), refactors getConfig onto it, and wraps the other three. Writes are
safe to retry: withRetry only retries connection-class failures and both
writes are idempotent (upsert / delete).
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: jalagrange <jalagrange@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Ports the still-unmerged half of PR #2442. OpenAI caches prompt prefixes
automatically, but a stable prompt_cache_key keeps requests that share a
prefix on the same inference engine, lifting the automatic-cache hit rate.
chat() now derives a stable key from the system prompt + sorted tool names
for native-openai models and passes it via providerOptions.openai.
promptCacheKey. Config provider_chat_options still overrides the derived
key; anthropic/google/openai-compatible providers are untouched.
The Anthropic half of #2442 (cache_control "silent no-op") is superseded:
@ai-sdk/anthropic 3.0.74 forwards call-level providerOptions.anthropic.
cacheControl as the Messages API's request-level cache_control (automatic
prefix caching), so master's existing marker is live on current deps.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: CoachRyanNguyen <CoachRyanNguyen@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Port of #1731 (diazMelgarejo) onto current master. gbrain import wrote
~/.gbrain/import-checkpoint.json with the caller's raw dir argument, so
a checkpoint left behind by an interrupted run (e.g. SIGTERM) could carry
"." or a symlinked spelling — an identity that resolves to whatever CWD
the next consumer happens to run from. Downstream tooling that treated
the checkpoint dir as an owned staging boundary could then act on the
wrong directory.
- runImport captures the import target ONCE via resolveImportTargetDir
(resolve + realpathSync) and threads that canonical value through
collection, checkpoint load/save, and resume filtering
- checkpoints are self-describing (schema_version: 1, owner: "gbrain",
kind: "import"); loadCheckpoint tolerates absent metadata (legacy
path-based files) but rejects present-and-wrong metadata and any
relative dir
- checkpoint contract documented in docs/guides/live-sync.md (llms
bundle regenerated)
- test/import-resume.test.ts fixture now realpaths its tmpdir so planted
checkpoints match the canonicalized dir (macOS /var -> /private/var)
Fixes#1728
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Lawrence Melgarejo <Lawrence@cyre.me>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Source-provenance wave, reconnect resilience, SSE proxy fix, rolling
prompt-cache, security CI, three consolidated fix-waves, and twenty more
individually verified community fixes.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(deps): resolve all 34 OSV-flagged vulnerabilities with same-major patch bumps
Direct deps: js-yaml ^3.15.0, marked ^18.0.2 (resolves 18.0.6),
admin vite ^6.4.3. Transitive deps pinned via overrides at their
minimum fixed versions (same major, no promotion to direct):
@hono/node-server, fast-uri, fast-xml-builder, fast-xml-parser,
form-data, hono, ip-address, qs; admin: @babel/core, postcss.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(deps): force js-yaml >=3.15.0 for all resolutions via override
A transitive consumer held a second js-yaml@3.14.2 resolution the
direct-dep range bump did not move.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* test: symlink-walker tests use a non-metafile probe (README now skipped by design, #2315)
The import walker deliberately skips README/metafiles since #2315 (closing
#345); the symlink-hardening tests used README.md as their probe file and
went red on the intersection. Probe with notes.md instead; test intent
(cycle hardening + strategy filter) unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ci: grant security-events write to the OSV caller job — reusable workflow requires it at startup
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
`parseMarkdown` previously walked the lines after the opening `---` and
recorded the first `^#{1,6}\s`-shaped line as a `headingBeforeClose`,
then flagged MISSING_CLOSE when that index came before the actual closing
fence. YAML allows `#` comment lines anywhere inside the document, so a
template that leads with annotation comments inside the fence (e.g. a
`# Research Template` header before the keys) hit a false-positive
MISSING_CLOSE even though the closing `---` was present.
Fix: only walk for the closing `---`. When it is found, content between
the fences is YAML; `#` lines are comments, not headings. When the close
is genuinely missing, surface the first heading-shaped line as a
where-it-went-off-the-rails hint (this path was already correct; we keep
it for the genuine missing-close case).
Two regression tests added to `test/markdown-validation.test.ts`:
- `#` comment lines at the top of a closed frontmatter
- `#` comment lines interleaved with keys
All 68 tests across the four markdown/frontmatter test files stay green.
* fix(extract,ingest,cycle): source-provenance wave — thread source identity through ingest_capture, fs-walk links, and cycle extract (#1522#1747#1503)
Three fixes in the same invariant class (source identity silently dropped
on the write path, collapsing to the 'default' source):
- #1522: the ingest_capture Minion handler validated IngestionEvent
provenance (source_id/source_kind/source_uri) then dropped it on the
importFromContent call. Now threads source_kind/source_uri +
ingested_via='ingest_capture' into the page write, and routes the write
under event.source_id when it names a registered source AND the event
is trusted (fail-closed: untrusted webhook payloads carry a
caller-controlled x-gbrain-source-id header and must not choose their
write source; unregistered emitter ids keep default routing so the
webhook path can't FK-fail).
- #1747: the fs-walk extractors (extractLinksFromDir /
extractTimelineFromDir / extractForSlugs) built batch rows with no
source_id, so addLinksBatch/addTimelineEntriesBatch mapped missing →
'default' and the pages JOIN dropped every row on a non-default source
("Links: created 0 from N pages", no error). ExtractOpts gains
sourceId; the CLI fs path resolves it via resolveSourceId
(--source-id > env > dotfile > registered path > sole-non-default)
and rows are stamped from/to/origin_source_id + timeline source_id.
- #1503: the cycle's extract phase (runPhaseExtract) never passed a
sourceId, so federated-brain dream/autopilot cycles persisted nothing
every night. It now threads cycleSourceId (explicit --source or
resolveSourceForDir(brainDir) — the same seam runPhaseSync uses) into
runExtractCore for both the incremental and full-walk paths.
Regression tests: handler provenance write-through (registered/
unregistered/untrusted routing), fs-walk + CLI resolution + incremental
cycle route landing edges/timeline in the right source (all red on
unfixed master), and a negative control pinning the pre-fix JOIN-drop
shape.
Reuses the ExtractOpts.sourceId threading approach from PR #1719,
rebased onto the current signal-aware signatures and extended to the
cycle + timeline paths.
Fixes#1522Fixes#1747Fixes#1503
Co-authored-by: seungsu <kss530c@gmail.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: update cycle source pin for cycleSourceId threading
The #1972 source-pin test asserts the literal runPhaseExtract call site in
cycle.ts. The #1503 fix appended cycleSourceId after opts.signal; signal
threading is unchanged (still the 5th arg, forwarded to runExtractCore).
Pin updated to the new literal — invariant intact.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: seungsu <kss530c@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Recover the worker-owned Postgres pool when promoteDelayed escapes a retryable connection error, preventing the repeated Promotion error: No database connection loop from issue #1491.\n\nAdds a regression test proving reconnect happens before the worker continues to claim work.
Closes#345.
The bulk-import walker isCollectibleForWalker filtered admitted files by
extension only, while incremental sync excludes README/index/log/schema via
isSyncable -> SYNC_SKIP_FILES. A directory import therefore ingested every
directory README as a folder-titled ghost page that trigram-corrupts fuzzy
entity resolution and inflates orphan count. Apply SYNC_SKIP_FILES (basename
guard) at the top of isCollectibleForWalker so both the FS-walk and the
git-fast-path collection routes agree with sync.
Also add RESOLVER.md to SYNC_SKIP_FILES: a structural routing metafile
(docs-aligned with schema.md/index.md/log.md/README.md), not indexable content.
Co-authored-by: ElliotDrel <ElliotDrel@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
The audit walk in `gbrain sources audit` used `if (pruneDir(entry, dir)) continue;`
but pruneDir() returns true = descend, false = prune (core/sync.ts). The
inversion made the walker skip every legitimate subdirectory (any source with
nested content reports 'Files scanned: 0 markdown files') while descending
into exactly the trees it should skip (node_modules/, .git/, vendor/, .raw/).
The walker's own comment says 'Mirror gbrain sync's descent rules' — this
makes it actually do so.
Repro on a real brain (v0.42.56/57): a comms source with per-contact
subdirectories audits 0 files; a flat source audits correctly.
Co-authored-by: Idrees Kamal <idreeskamal@MacBook-Air.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(minions): restore rolling conversation prompt-cache on the direct SDK path
Regression since v0.42.51 (@ai-sdk bump): the direct/native subagent
path (agent.use_gateway_loop=false) only marks cache_control on the
static system-prompt and last-tool-def blocks (~5.2K tokens). The
`anthroMessages` array — the part of the request that actually grows
every turn — carries no cache marker at all, so Anthropic re-bills the
full conversation as fresh input on every turn instead of reading it
from cache.
Before this regression, cache_read grew with the conversation
(4.6K -> 125K across a session). Since the regression, cache_read is
pinned at the ~5.2K static prefix regardless of conversation length.
Fix: mark the last content block of the last message with
`cache_control: { type: 'ephemeral' }` on every turn, after first
stripping any stale marker left on an earlier message. Anthropic
caches everything up to the last cache_control breakpoint, so this
turns the trailing marker into a rolling window over the growing
conversation while staying within the 4-breakpoint limit (system +
last-tool + 1 rolling = 3 used).
Measured on a real dream synthesize run: cache_read/input ratio
0.000 -> 32-61 across 23 calls, ~78-80% cost reduction for that run
($8.5 no-cache-equivalent -> $1.71), zero dead-letter jobs.
Neither the gateway path (cache_control placed at the top level,
which @ai-sdk 3.x silently ignores — see #2490) nor #2442 (system +
last-tool only) restores this; both leave the conversation body
unrecovered.
* fix(minions): normalize seed message content before caching it
Codex review caught a real gap: a fresh job's seed user message is
initialized as `content: data.prompt` (a plain string), not a content-
block array. The rolling-cache logic added in the previous commit only
attaches cache_control when `Array.isArray(lastMsg.content)`, so it
silently skipped the very first API call — meaning the common
single-tool round-trip (seed prompt -> tool_use -> tool_result -> done)
got no conversation-cache benefit at all, only jobs with 3+ turns did.
Normalize string content to a one-block text array before checking, so
the first call gets the same rolling breakpoint as every later one.
* fix(minions): retain the prior rolling cache breakpoint, not just the newest
Second Codex review pass: with a single rolling marker, deleting every
prior message's cache_control before placing the new one means a turn
that adds more than 20 content blocks since the last marker (e.g. a
large parallel tool_use/tool_result round) can miss Anthropic's cache
lookup entirely -- the API's automatic prefix search only looks back
up to 20 blocks from a breakpoint to find a prior cached prefix.
Keep the immediately-preceding rolling marker in place instead of
stripping down to one; only evict markers older than that. This still
fits the 4-breakpoint budget (system + last-tool + 2 rolling = 4) and
guarantees the previous marker's prefix remains a valid, already-cached
read even when the new marker's own lookback misses.
The instance-pool reconnect() did disconnect() (nulling _sql) BEFORE connect(),
so a connect() failure during a transient Postgres blip left _sql null for the
rest of the process — every subsequent non-retry-wrapped call then fell through
to the never-connected module singleton and threw 'No database connection',
crashing the autopilot worker into a respawn loop. Build-then-swap: snapshot the
live pool, build a fresh one, end the old only once the new validates, restore
on failure. Keeps upstream's reap-detection + pool-recovery audit. Confirmed by
jalagrange on closed PR #1593; this is the Layer-1 fix he left to us.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The litellm recipe shipped only an `embedding` touchpoint. `getProviderCapabilities()` in `src/core/ai/capabilities.ts` throws when `recipe.touchpoints.chat` is missing, `classifyCapabilities()` returns `'unknown'`, and `enforceSubagentCapable()` in `src/core/model-config.ts` silently falls back to `TIER_DEFAULTS.subagent` (anthropic). Any brain that routes paid traffic through a litellm-style proxy AND has no `ANTHROPIC_API_KEY` then sees every subagent loop dispatch throw `AIConfigError: Anthropic ... requires ANTHROPIC_API_KEY`. The user's explicit `models.tier.subagent = litellm:*` choice is overridden without their knowledge.
Declare `chat` and `expansion` touchpoints mirroring the openai recipe's shape: `models: []` (litellm proxies arbitrary backends; allowlist is intentionally empty, since `assertTouchpoint` already skips allowlist checks for `tier: 'openai-compat'`), `supports_tools: true`, `supports_subagent_loop: true`, `supports_prompt_cache: false` (OpenAI-compat backends don't honor Anthropic-style `cache_control`), `max_context_tokens: 200_000` (conservative GPT-5-family default; per-deployment override needed for smaller-context backends), costs `undefined` (varies by proxied provider).
Reproduction (deterministic):
1. Fresh brain with no `ANTHROPIC_API_KEY` in env.
2. `gbrain config set models.tier.subagent litellm:gpt-5.4` (or any `litellm:*` string).
3. `gbrain models` warns and falls back to `anthropic:claude-sonnet-4-6`.
4. Submit any subagent job; throws `AIConfigError: Anthropic ... requires ANTHROPIC_API_KEY`.
After the patch, `classifyCapabilities('litellm:gpt-5.4', recipe)` returns `degraded:no_caching` (chat-capable, no Anthropic-style prompt cache). `enforceSubagentCapable` no longer steals the model choice. The subagent loop emits a one-time `degraded:no_caching` warn about prompt-cache absence; cost scales linearly with conversation length, accepted trade for the proxy path.
D4 invariant ("never advance last_commit on partial", sync.ts comment)
preserved. last_sync_at is a monitoring signal read by doctor
sync_freshness (warn 24h / fail 72h), separate from the import-converged
bookmark. Without this heartbeat write, a cron-driven */15 sync over a
quiet vault pins last_sync_at to the last real commit, so doctor
falsely flags the source as stale for as long as the vault is quiet.
Reproduction (5 lines, no gbrain install required):
1. Setup a fresh obsidian source + commit a single .md file
2. gbrain sync --source obsidian # first_sync, last_sync_at = NOW
3. (do nothing) gbrain sync --source obsidian # up_to_date
4. SELECT last_sync_at FROM sources WHERE id = 'obsidian';
5. Observed: still pinned to step 2. Expected: bumped to step 3.
Fix: in the up_to_date early-return (sync.ts line ~1786), execute a
single `UPDATE sources SET last_sync_at = now() WHERE id = $1` before
returning. The D4-protected writeSyncAnchor path is untouched.
Test: test/sync.test.ts adds a describe block that runs two
consecutive syncs against a quiet vault and asserts last_sync_at
advances while last_commit is unchanged. PGLite + executeRaw pattern
matches the existing #1970 test scaffold.
Workaround: hourly psql touch in WSL crontab documented at
https://github.com/garrytan/gbrain/issues/[link-to-issue]
Co-authored-by: lost9999 <lost9999@users.noreply.github.com>
The renames loop reimports each renamed file via importFile() but, unlike the
deletes and adds/mods loops, does not wrap the call. importFile() still throws
on content sanity-block, duplicate-slug, and missing-link endpoints, so a single
malformed renamed file throws uncaught and crashes the whole sync mid-run —
freezing the checkpoint and defeating --skip-failed. A 'skipped' result carrying
an error was also silently dropped (never recorded to failedFiles).
This wraps the reimport in try/catch and records both the throw and the
skipped-with-error case to failedFiles, matching the existing deletes/adds loop
pattern. Surfaced in the wild by a tree-wide rename (a shared/ -> system/ vault
migration, ~10.8k renames) where one malformed-YAML renamed file crashed the
entire incremental sync at rename ~4900/10857.
Co-authored-by: jaxlewis-swift <jaxlewis@swiftsolutions.ai>
* fix(cli,config,doctor): CLI/config UX wave — config-get file plane, idempotent archive, honest help + doctor text, prefixed model defaults (#2120#2792#1175#1123#2451)
- config get resolves the file/env plane before the DB plane (runtime
precedence) and reports provenance on stderr; stdout stays a bare value.
- sources archive distinguishes already-archived (friendly no-op, exit 0)
from not-found (clear exit-4 error).
- gbrain --help SOURCES block now lists archive/restore/archived/purge/status
plus a pointer at `sources --help` for the long tail.
- multi_source_drift doctor advice references only real CLI surfaces
(drops the never-built 'sources rehome'; pins delete to GBRAIN_SOURCE=default).
- #2451 (bare model ids in calibration defaults) verified already fixed +
tested on master by #2892 — no change needed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: satisfy test-isolation gate for wave-B tests
check-test-isolation R1 forbids direct process.env mutation in non-serial
unit tests (env leaks across files sharing a shard process). Route the
GBRAIN_HOME / GBRAIN_CHAT_MODEL / GBRAIN_PGLITE_SNAPSHOT overrides through
the canonical withEnv() helper instead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Rebase-port of #2452 (spinsirr:fix/calibration-profile-scope-and-cli) onto
current master after tonight's merges made the fork branch conflict.
- cli: add 'calibration' to CLI_ONLY so dispatch reaches its existing
handler instead of falling through to "Unknown command" (#2035); honor
--source / GBRAIN_SOURCE in the calibration CLI.
- takes: route takes_list / takes_search / takes_scorecard /
takes_calibration through sourceScopeOpts(ctx) (federated array > scalar
> nothing) and scope engine reads via the take's page.source_id — JOIN
filter for list/search, EXISTS for scorecard/curve — on both engines
(#2200-class).
- bigint: shared takeHitRowToHit coercion in searchTakes /
searchTakesVector (both engines) + bigintToStringReplacer on the cli.ts
output normalizer and the `gbrain call` exit, so int8/BIGSERIAL columns
no longer crash JSON.stringify (#2450); calibration profile id
BIGSERIAL → string.
- calibration: default model ids route through TIER_DEFAULTS
(provider-prefixed) instead of bare model strings; admin calibration
chart endpoints fixed (takes has no page_slug column; month-precision
since_date; Date generated_at; bigint id in drill-down).
The think/gather source-scope slice of the original PR was dropped: it
already landed on master via #2739.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: spinsirr <ID+spinsirr@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
classifyPgliteInitError() routes bare Emscripten aborts to the 'unknown'
verdict, whose hint unconditionally printed "Most common cause: the macOS
26.3 WASM bug (#223)" — even on Windows and Linux (#2195, #1870).
- buildPgliteInitErrorMessage now takes a platform param (default
process.platform): darwin keeps the #223 link as a *possible* cause;
other platforms get the plausible off-macOS causes (lock contention,
damaged data dir) plus `gbrain doctor` / `gbrain reinit-pglite`.
- New stringifyPgliteInitError(): non-Error rejections (Emscripten
aborts can throw plain objects) no longer print "[object Object]".
- Regression tests for both branches + the stringifier in
test/pglite-init-classifier.test.ts.
Canonical for a 7-report class: #2674, #1870, #1195, #1502, #2195,
#939, #391.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The README/INSTALL updates from #1671 landed without the bundle regen the
freshness gate requires; every branch cut from master since inherits the
failure.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(postinstall): cross-platform node shim instead of POSIX shell
The postinstall script used POSIX shell syntax ('command -v',
'>/dev/null 2>&1', '1>&2') which Bun's built-in script parser rejects
on Windows. 'bun install' aborted with 'expected a command or
assignment but got: "Redirect"' before gbrain could ever be probed.
Replace with a one-liner 'node -e' shim that:
* uses spawnSync to probe 'gbrain --version' (shell:true on win32 so
the Windows shim/.cmd resolution works)
* on success: runs 'gbrain apply-migrations --yes --non-interactive'
and propagates its exit code
* on failure: writes the same skip message to stderr and exits 0, so
fresh clones (where gbrain isn't on PATH yet) still complete install
POSIX hosts retain the original behavior; Windows hosts now succeed
instead of failing the whole install.
Fixes#1486
* fix(postinstall): move logic to scripts/postinstall.ts to survive Bun Windows script-runner
The `node -e` inline shim still failed on Windows under Bun. Embedding a
program inside the package.json postinstall string lets the lifecycle shell
mangle it: Bun's Windows script-runner expands the `\n` in the hint string
into a REAL newline before node sees it, producing `SyntaxError: Invalid or
unexpected token` and aborting the whole install. `node` is also not
guaranteed present under a Bun install (bun is the guaranteed runtime), and
`shell: win32` re-opened a quoting surface.
Move the logic into a checked-in `scripts/postinstall.ts` run via
`bun run scripts/postinstall.ts`, matching the repo's existing convention of
~19 scripts under scripts/*.ts. This sidesteps all three failure modes:
* `which('gbrain')` from bun does Windows-aware PATH resolution (finds
gbrain / gbrain.exe / gbrain.cmd) with no shell.
* `Bun.spawnSync` with an argv array invokes apply-migrations directly —
no shell, nothing to quote, no `\n` expansion.
* No dependency on `node` being present; bun runs the script.
Behavior is preserved exactly: same `apply-migrations --yes
--non-interactive` command, same issue-218 skip hint, and the same
never-fail-the-install guarantee (every path exits 0). Verified on macOS:
`bun run scripts/postinstall.ts` exits 0 on the skip path (gbrain absent),
on a failing migration, and on a successful migration.
Fixes#1486
The stdio parent-death watchdog was hard-wired to spawnSync('ps'), which
does not exist on Windows. The startup probe therefore failed on every
Windows host, the watchdog was permanently disabled ("watchdog disabled:
ps unavailable"), and an orphaned `gbrain serve` held the PGLite write
lock until reboot. Orphans are especially easy to produce on Windows
because MCP hosts launch the server through a cmd.exe wrapper (.bat) and
killing the wrapper does not kill the bun child.
Windows never re-parents orphans, so the cached process.ppid stays
correct for the process lifetime and the watchdog question inverts from
"did the live PPID change?" to "is the original parent still alive?" --
answered in-process with a signal-0 existence probe (process.kill(ppid, 0),
OpenProcess under the hood). No external binary needed. EPERM counts as
alive. Parent dead reports PID 0, which differs from initialParentPid and
fires the existing shutdown path.
- readLiveParentPid / probeWatchdogAvailable: platform split (ps on
POSIX, signal-0 on win32), exported with a platform test seam so CI on
any OS exercises both branches.
- isPidAlive: shared exported helper.
- Watchdog install guard tightened from `!== 1` to `> 1` so a PID-0
"parent already gone" report cannot install a phantom interval that
compares 0 to 0 forever.
- Disabled-mode log generalized (no longer claims ps is the only
mechanism); existing probe-fail test updated to match.
- New unit suite for the platform defaults (6 tests).
Verified live on Windows 11: serve spawned via a .bat wrapper, wrapper
killed without closing stdin -> bun child exits within ~10s and the
PGLite lock dir is released. Before this change the child survived
indefinitely and every subsequent gbrain invocation timed out waiting
for the lock.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Add provider_chat_options alongside provider_base_urls and thread it through
the gateway config path into chat(). The chat request now deep-merges
provider-scoped options and model-scoped overrides into providerOptions keyed
by recipe id, preserving existing gateway-built options such as Anthropic
cacheControl and leaving absent-config behavior unchanged.
This lets operators disable thinking for small-budget hybrid-reasoning utility
calls without hardcoding that behavior for every use of those models.
PGLite's embedded WASM engine crashes on macOS 26.x (Tahoe) on Apple
Silicon during engine initialization. This adds:
- A Troubleshooting section in docs/INSTALL.md with step-by-step
instructions for using native Homebrew PostgreSQL 17 + pgvector
as a workaround
- A callout in README.md's Troubleshooting section pointing users
to the detailed setup guide
Tested on macOS 26.5 (arm64), Bun 1.3.14, gbrain 0.41.29.0,
PostgreSQL 17.10 (Homebrew), pgvector 0.8.0. All 102 schema
migrations pass. gbrain doctor green.
Co-authored-by: Saurav Roy <roysaurav@users.noreply.github.com>
* fix(facts): durable facts-absorb jobs for one-shot CLI processes
Every gbrain capture/put from a short-lived CLI enqueued the facts:absorb
chat into the in-process FactsQueue, then the exit teardown drained for
1-2s and aborted the in-flight call — logging 'pipeline_error: [chat(...)]
The operation was aborted.' on every eligible CLI page write and never
extracting facts.
cli.ts now marks one-shot processes (everything except serve/jobs/
autopilot); runFactsBackstop's queue mode submits a durable facts-absorb
minion job for the long-lived jobs worker instead, with content-hash
idempotency and fallback to the in-process queue if submission fails.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(facts): fence-write resolves source-scoped page path
writeFactsToFence joined local_path + slug directly, writing main-source
fences to the repo ROOT (the default source's tree) and polluting ~/brain
with stray root-level fence files. Route through resolvePageFilePath —
the same helper the put_page write-through and dream-cycle reverse-render
use — so non-default sources fence into .sources/<id>/<slug>.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Ragnar Åström <reghar@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The parser/doctor/extractor read the polished page body (compiled_truth +
timeline) instead of the raw turn-by-turn transcript that meeting pages store
in a `raw_transcript` frontmatter sidecar, and the brain's actual raw format
(`Speaker A: ...` / `Speaker B: ...`) had no built-in pattern. Result: scan
returned no_match / 0 messages and conversation-fact extraction produced 0
segments / 0 facts.
- new readConversationBodyForParsing(): prefer the raw_transcript sidecar when
present, fall back to compiled_truth + timeline (src/core/conversation-parser/body.ts)
- wire it into conversation-parser scan, doctor coverage check, and
extract-conversation-facts (drops the old readPageBody helper)
- add a built-in `speaker-letter-no-time` pattern for plain `Speaker A:` lines
Verified: scan now returns phase=regex_match (speaker-letter-no-time), and the
Ben page extracts 61 facts / 7 segments. Tests added; suite green.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
since/until range filters were applied against updated_at, so edited-but-old
entries leaked into time-bounded queries. Filter on the effective date instead.
Closes#1520
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Cross-referenced the v0.42.59.0 five-fix rollup (#2735-#2739) against the
reference docs and updated every entry that no longer described current
behavior:
- KEY_FILES.md: migrate-engine.ts (source-catalog copy + target-aware resume
manifest), pglite/postgres bootstrap probe set (timeline_entries.event_page_id),
searchTakes/searchTakesVector source scope, think op scope threading through
runGather via thinkSourceScopeOpts, new fence-shared.ts entry (escape-aware
parseRowCells as escapeFenceCell's inverse).
- TESTING.md: one-liners for the three new e2e suites
(think-source-isolation-pglite, facts-fence-reconcile-postgres,
migrate-engine-sources-postgres) + the new multi-source-bug-class case.
- TODOS.md: new v0.42.59.0 follow-ups section (6 items); refreshed the two
existing items the wave partially resolved (think gather scope plumbing,
#2200 takes_search engine-layer scope).
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Markdown pipe tables have no vertical-align control, so every renderer except
GitHub middle-aligns rows — unreadable when the two columns differ in length.
Switch the per-chapter prompt + frontmatter tag to the HTML <table> form with
valign=top on every cell (matches the 20 hand-built mirror pages that already
render correctly).
Co-authored-by: garrytan-agents <agents@garrytan.dev>
* feat(extract): recognize inline [Source: ..., YYYY-MM-DD] citations as timeline entries
gbrain's own quality conventions (skills/conventions/quality.md) require a
dated [Source: ..., YYYY-MM-DD] citation on every brain write, so curated
pages are full of dated evidence — but extractTimelineFromContent only
recognized the timeline-bullet and date-header formats. A page whose dates
all live in citations scored zero timeline coverage in brain_score, and
doctor pointed users at a formatting convention their own citations already
satisfied in spirit.
Format 3 files one entry per citation: date and source from the marker,
summary from the annotated line with citation markers stripped. Lines
already captured by Format 1 are skipped so a timeline bullet carrying its
own citation is not double-filed. Bare citations with no surrounding text
are ignored.
Idempotency is unchanged: persistence already dedupes at the DB layer.
* fix(extract): Format 3 citations also in parseTimelineEntries (db-source + ingest path)
The first commit only taught extractTimelineFromContent (fs-source) the
citation format; the db-source extract and the ingest path parse through
parseTimelineEntries in core/link-extraction.ts, which still could not see
citations. Same rules as the fs parser: bullet-captured lines skipped,
bare citations ignored, invalid calendar dates rejected; the citation
source is preserved in the entry detail.
---------
Co-authored-by: pabloglzg <186649799+pabloglzg@users.noreply.github.com>
extractTakesFromPages selected pages by updated_at DESC + LIMIT with no
exclusion of pages that already hold takes. The CLI clamps --max-pages to
1000, so on a corpus larger than one run every re-run rescanned the same
most-recent 1000: the older tail could never be bootstrapped, and each
rescan re-spent Haiku budget producing upsert-identical rows. Seen live on
a 2,311-eligible-page brain — a second run would have covered 0 new pages.
Covered pages are now skipped by default (NOT EXISTS on takes.page_id), so
repeat runs sweep a large corpus in slices; --include-covered restores the
full rescan for refresh use. Usage text documents both plus the 1000 clamp.
Claude-Session: https://claude.ai/code/session_01FQgByq4aqQq2PP8UHCdnfk
Co-authored-by: Paolo Belcastro <p3ob7o@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
extract_atoms minted duplicate atom pages two ways:
A. Trailing-dash twins. The local slugger truncated the title at 60 chars
with no re-strip, so a cut landing on a hyphen left a trailing dash
(`…would-`). The FS-import normalizer (slugifySegment) strips it
(`…would`), so the same atom persisted under two slugs and the
dedup-by-slug check never collapsed them.
B. Cross-day re-mint. The slug used the run date (todayDate()), while the
idempotency guard keys on the whole-file source_hash. Append-only
sources (chat/transcript exports) grow daily, so the file hash changes,
the guard never matches, the source is re-extracted, and each re-mint
lands under a new date prefix → a new slug → no upsert → a duplicate.
Fix: make the atom slug a deterministic function of stable inputs —
`atoms/<source-date>/<stem>-<title-hash>`:
- source date is parsed from the source ref (transcript filename / page
slug), not the run date, so re-extraction converges on the same slug and
putPage upserts instead of duplicating;
- the 6-char title hash keeps two atoms whose titles share the first 60
chars on distinct slugs (no silent clobber of a different atom);
- the stem routes through the canonical slugifySegment and re-strips a
trailing dash, so the two write paths can no longer disagree.
The whole-file source_hash batch check is retained only as a cost fast-path
(skip re-running the model on an unchanged source); correctness no longer
depends on it.
Adds a hermetic PGLite regression test (no DATABASE_URL) asserting the
source-dated prefix, title-hash suffix, trailing-dash strip, and upsert on
re-extraction of a grown append-only transcript.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A lock file can outlive its autopilot process after a crash or forced termination. The previous mtime-only check treated that file as proof that another instance was running, so supervisor restarts could exit repeatedly until the file aged out.
Read the holder PID and probe it with signal 0. Keep the lock whenever that process is alive; take over only dead, malformed, empty, or self-owned locks. EPERM remains conservative and counts as alive, preventing two autopilot instances from running concurrently.
Add hermetic tests for missing, live, dead, malformed, and empty lock states.
Atom-extraction page discovery hardcoded EXTRACTABLE_PAGE_TYPES and ignored the
active pack's `extractable: true` flags, so a type declared extractable in the
manifest (e.g. `note`) never actually extracted. Closes the D2 TODO the code
already flagged (extract-atoms.ts: 'future pack-aware refactor ... pull from the
active pack manifest').
Resolve the allowlist as: legacy hardcoded floor UNION the pack's extractable
types, MINUS synthesis outputs (atom, concept — extracting from these would loop,
since concepts are synthesized from atoms). Mirrors facts/eligibility.ts, which
excludes concept the same way. Back-compat: gbrain-base brains keep every legacy
target via the union; fail-soft falls back to the legacy floor if the pack can't
load.
- pure unionExtractableTypes() policy, unit-tested
- discoverExtractablePages + countExtractAtomsBacklog resolve from the pack
- page-discovery fixture updated (note now extracts; concept stays excluded)
Co-authored-by: Paolo Belcastro <p3ob7o@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
`schema use` hardcoded `gbrain-base` (and a fixed bundled list), so other
bundled packs could not be selected by name. Use the shared BUNDLED_PACK_NAMES
set and resolve `<name>.yaml` generically so every bundled pack activates.
Closes#1574
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
claude-sonnet-5 and claude-fable-5 are GA Anthropic models, but neither
was in CANONICAL_PRICING. A brain routing a tier to them (e.g.
models.tier.reasoning = anthropic:claude-sonnet-5) ran with cost
telemetry blind on that tier: canonicalLookup missed, the budget meter
logged BUDGET_METER_NO_PRICING and disabled the gate, and cost views
under-reported spend.
- model-pricing.ts: anthropic:claude-sonnet-5 at $3/$15 and
anthropic:claude-fable-5 at $10/$50. Sonnet 5's launch intro discount
($2/$10 through 2026-08-31) is deliberately not modeled — the table
carries standard rates so estimates stay conservative and the entry
needs no time-bombed edit when the promo lapses.
- takes-quality-eval/pricing.ts: claude-sonnet-5 added to the curated
SUPPORTED_MODELS allowlist (a likely judge override). Fable 5 stays
out — priced for warn-only consumers, not a budgeted-eval panel model.
- model-pricing.test.ts: pin tests for both rows, matching the existing
Opus 4.8/4.7 pattern. Drift guards iterate the table; no changes.
All derived views (ANTHROPIC_PRICING bare view -> budget-tracker,
batch-projection, budget-meter) pick the rows up automatically.
Co-authored-by: Paolo Belcastro <p3ob7o@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* v0.42.60.0 chore(release): eleven verified community fixes — changelog + version bump
Windows full-sync mass-delete fix, gateway tool-loop resume consolidation
(fix-wave A), two source-isolation closes, search-cache exclude-policy
keying, and six more verified community fixes. Files the take-writes
fail-open source fallback and the #2112 doctor hunk as follow-up TODOs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: update reference docs for v0.42.60.0
- KEY_FILES.md: unpin stale KNOBS_HASH_VERSION number in the autocut entry
(mode.ts is the single source of truth); document the full-sync reconcile
path-separator normalization + mass-delete safety valve
(planReconcileDeletes, GBRAIN_ALLOW_MASS_RECONCILE); describe the TTY-gated
admin bootstrap token banner (--print-admin-token, env-sourced always hidden)
- docs/mcp/DEPLOY.md + docs/tutorials/company-brain.md: bootstrap token is now
hidden on non-TTY starts; document GBRAIN_ADMIN_BOOTSTRAP_TOKEN and
--print-admin-token for headless deploys
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(docs): regenerate llms bundle after KEY_FILES/deploy-doc sync
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
resolveHardExcludes() only ran at DB-query build time (cache miss), so
query_cache rows written by a process without GBRAIN_SEARCH_EXCLUDE could
be served to a process with it (and vice versa), leaking excluded slugs.
hybridSearchCached now resolves the effective hard-exclude list exactly
as the engines' query-build path does and folds it (sorted, append-only
hx= part) into knobsHash via a new KnobsHashContext.hardExcludes field.
KNOBS_HASH_VERSION 11 -> 12: one-time global cache cold-miss on upgrade,
refills within cache.ttl_seconds.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Collector branch superseding the gateway tool-loop duplicate cluster and
adjacent provider fixes. Re-implemented from the best of each PR (deduped by
content, not file-overlap); every fix carries test coverage.
Gateway tool-loop resume (supersedes #1934#2062#2065#2112#2274#2487#2336#2257#2499#2491, test #2063):
- toolLoop now persists the tool-result user turn per round (onToolResultTurn),
so a resumed subagent job reloads a balanced transcript instead of dangling
assistant tool-calls that non-Anthropic providers reject with
AI_MissingToolResultsError.
- runSubagentViaGateway reconciles an already-corrupted transcript on resume:
it heals every dangling assistant tool-call turn (not just the tail) from the
settled subagent_tool_executions rows, re-dispatching idempotent-pending
tools and throwing on non-idempotent, mirroring the legacy Anthropic path.
Terminal early-return for a transcript that already reached end_turn.
- repairToolPairing() is a last-resort normalization at the chat() boundary
(from #2336): back-fills error stubs for any assistant tool-call still
unanswered (partial turns, provider-duplicated/dropped IDs on local models,
length-truncated batches). No-op on balanced input.
- toModelMessages is Date-safe (Postgres timestamptz -> ISO via a JSON round
trip at the SDK boundary, never a ::jsonb cast; degrades bigint/circular to a
string instead of throwing) and drops non-string text blocks reasoning models
emit that AI SDK v6 rejects (#2488).
Adjacent provider fixes:
- Model-aware default max output tokens: thinking-by-default Claude 5 models get
headroom (gateway 32000, think 16000) while everything else stays 4096/4000,
so DeepSeek/OpenAI subagents don't exceed provider caps (#2614#2806).
- DeepSeek: promote reasoning_content into content when content is empty, via a
fail-open recipe fetch shim (#2617).
- OpenRouter: map openrouter_api_key (config + env) into OPENROUTER_API_KEY
through buildGatewayConfig; register agent.use_gateway_loop, zeroentropy and
openrouter keys in KNOWN_CONFIG_KEYS so `config set` accepts them (#2572,
config key from #2112).
Preserves JSONB (no JSON.stringify into ::jsonb), engine parity, source
isolation, and trust-boundary invariants. Verified with the gbrain-pr-test-env
consumer matrix (clawlancer/Postgres, gstack + hivemindos/PGLite) baseline-FAIL
-> candidate-PASS on the same repro, plus bun run verify (31/31) and 439
targeted unit + e2e tests.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: thomaskong119 <thomaskong119@hotmail.com>
Co-authored-by: maxpetrusenkoagent <max.petrusenko.agent@gmail.com>
Co-authored-by: brettdavies <brettdavies@users.noreply.github.com>
Co-authored-by: ivandebot <187176982+ivandebot@users.noreply.github.com>
Co-authored-by: Rafael Reis <rafael.reis@contabilizei.com.br>
Co-authored-by: fbal23 <fbal.public@gmail.com>
Co-authored-by: javieraldape <javieraldape@users.noreply.github.com>
Co-authored-by: David Carolan <david@joyrestart.com>
Co-authored-by: Masashi-Ono0611 <masashi.ono.0611@gmail.com>
Co-authored-by: spiky02plateau <155588579+spiky02plateau@users.noreply.github.com>
Co-authored-by: psam-717 <mphilannorbah@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(config): register Life Chronicle keys so the documented enable command works
The v0.42.56.0 release notes say `gbrain config set auto_chronicle true`,
but the key was never added to KNOWN_CONFIG_KEYS — the documented command
fails with 'Unknown config key' and the operator has to discover --force by
reading source. Registers 'auto_chronicle' plus the 'chronicle.' prefix
(chronicle.tz and future knobs). Same registration class as the v0.42.42.0
spend-controls fix. Regression test pins both.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FQgByq4aqQq2PP8UHCdnfk
* fix(config): register takes.bootstrap_enabled too — same unregistered-key class
Hit while enabling the takes bootstrap on a live brain: the onboard
remediation's documented enable key fails 'Unknown config key' exactly like
auto_chronicle did. Registered + pinned by the same regression test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FQgByq4aqQq2PP8UHCdnfk
---------
Co-authored-by: Paolo Belcastro <p3ob7o@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Postgres returns BIGINT file sizes as native BigInt values. Returning those values directly from file_list makes JSON serialization fail, and using them in CLI arithmetic can also throw.
Convert size_bytes to Number at the operation boundary and in the CLI display path. File sizes remain exact well beyond any practical attachment size.
Add a unit regression that exercises a native BigInt row and proves the operation result is JSON-serializable, plus a real-Postgres E2E assertion for the file_list response.
doctor-hidden-by-search-policy.test.ts hardcodes Float32Array(1536)
vectors (basisEmbedding) but lets initSchema size its vector columns from
process-global gateway state (getEmbeddingDimensions(), default 1280).
Whether the file passes depends on which test files run before it in the
shard; adding test files to the repo reshuffles the weight-packed shards,
so unrelated PRs trip it (seen on #2800 CI, test (1): every upsertChunks
died with 'expected 1280 dimensions, not 1536').
Same fix + rationale as engine-find-trajectory.test.ts and
cosine-rescore-column.test.ts, which document this exact class:
configureGateway(1536) in beforeAll BEFORE initSchema, resetGateway in
afterAll. The suite is now self-sufficient regardless of predecessor
state. Not reproducible outside CI's exact shard packing; the pin removes
the order-dependence either way.
Co-authored-by: Paolo Belcastro <p3ob7o@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(import): skip marked.lexer on fence-less pages to avoid bulk-import OOM (#2437)
extractFencedChunks() ran marked.lexer() on every page body. The lexer
allocates transient memory proportional to page size even when there is no
code fence to extract — a ~2MB table/doc page spikes ~110MB of heap to
produce zero fenced chunks. During bulk import these per-page spikes stack
on accumulated chunk/embedding memory and can OOM the worker; the existing
try/catch cannot rescue an OOM (process death, not a throw).
On a representative brain ~99% of importable pages have no fence, so the
lexer pass is pure wasted work there. Add a fast-path that returns early
when the body contains no fence marker. Matches both ``` and ~~~ so tilde
fenced code still extracts.
Scope: this removes the fence-less transient-allocation surface (the
observed incident). It does not make marked.lexer safe for pages that DO
contain a fence; an input-size/nesting cap is a sensible follow-up.
Tests: add two regression cases — tilde-fenced code still extracts, and a
large fence-less table page imports with zero fenced chunks.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(import): match marked's \r normalization in the fence fast-path (#2437)
Self-review follow-up. marked normalizes `\r\n|\r → \n` before lexing, but the
no-fence fast-path probed the raw body with `(^|\n)`. A CR-only (classic-Mac)
line-ended page with a real fenced block would be skipped by the guard while
marked would have extracted it — a lost fenced_code chunk. Widen the line-start
class to `(^|[\r\n])` so the probe agrees with marked. CRLF was already covered.
Add a regression test (CR-only fenced page still extracts); it fails on the old
`(^|\n)` regex and passes on the fix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
On a Windows checkout, path.relative yields backslash-separated paths while a
page's stored source_path can hold forward slashes (e.g. git-derived). The
full-sync reconcile compared the two without normalizing separators, so every
file-backed page looked stale and the reconcile deleted the entire source.
- Normalize separators on BOTH sides of the membership test (shared
.replace(/\/g, '/')), so pages written with either separator match on any OS.
- Add a mass-delete safety valve: when a reconcile would sweep > 50% of the
file-backed pages a strategy manages on a source with > 20 of them, skip the
delete and surface a loud warning instead of silently wiping the brain.
GBRAIN_ALLOW_MASS_RECONCILE=1 restores the old behavior.
- Factor the decision into pure, exported helpers (planReconcileDeletes,
massReconcileAllowed) and unit-test the separator matching, the valve
threshold, and the env override without a live engine.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(security): #2624 don't print admin bootstrap token on non-TTY (log-leak)
serve --http printed the generated admin token in the startup banner
unconditionally. In containerized deploys stderr ships to centralized log
storage, turning the token into a standing secret in logs.
Fail-safe default: the generated token now prints only when stderr is an
interactive TTY. Non-TTY starts hide it (--print-admin-token forces it;
$GBRAIN_ADMIN_BOOTSTRAP_TOKEN + --suppress-bootstrap-token already existed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(security): #2624 banner shows 'from env' before non-TTY hidden guard
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Carry the caller's scalar or federated source scope through every think gather stream: hybrid page retrieval, takes keyword/vector retrieval, and graph traversal. Adds source predicates to the takes retrieval methods in both engines. Part of #2200 (the think slice; #2200 stays open as the tracking issue for the remaining by-slug read ops).
parseRowCells now splits on unescaped pipes only and decodes escaped pipes while preserving ordinary backslashes and empty cells, so facts whose text contains literal | survive the fence->DB reconcile instead of being silently deleted. Fixes#2726.
Bare names resolve only when prefix expansion finds exactly one canonical candidate; ambiguous collisions and low-specificity multi-token fuzzy matches fall through to the guarded holding path instead of confident wrong attribution. Fixes#2723.
migrate --to now copies the complete source catalog before pages (fixes the pages_source_id_fkey failure on multi-source brains), and resume manifests carry an opaque target identity so a checkpoint from one target is discarded for a different target. Fixes#2677.
Adds the missing timeline_entries.event_page_id forward-reference bootstrap probe + bare-column repair to both engines so pre-v121 brains can replay the current schema and reach migration v121. Fixes#2724.
* fix(ai): drop empty-string env values before merge so they can't clobber config keys (#1249)
Claude Code injects ANTHROPIC_API_KEY='' to neuter subprocess LLM calls; an
unconditional process.env spread let that empty string override a valid
config.json key, breaking every gateway op with NO_ANTHROPIC_API_KEY. Filter
'' / undefined before the merge; '0' and 'false' are preserved.
* fix(ai): normalize native provider base URLs + replace embedding guard with a dims-presence check (#1250, #1292)
#1250: createAnthropic/createOpenAI were called with no baseURL, so an
env-injected bare host (e.g. ANTHROPIC_BASE_URL without /v1) 404'd. Add a
shared resolveNativeBaseUrl and pass a normalized baseURL at all anthropic +
openai native sites (google deferred until its suffix is verified).
#1292/D6: the user_provided_model_unset guard was structurally unreachable as
a no-model check (parseModelId throws on a bare provider) and only ever
false-positived for litellm:<model>, silently disabling vector search. Replace
it with a real dims-presence check for user-provided/zero-default recipes and
delete the dead branch in both consumers. Also stop configureGateway from
fabricating a default embedding_dimensions, so 'no dims set' stays honest.
* fix(ai): trust user-declared embedding dims for local recipes + litellm /v1 hint (#2271, #2209)
#2271: a new trust_custom_dims flag adds a passthrough tier so ollama /
llama-server / litellm accept a user-supplied --embedding-dimensions instead of
being hard-rejected. Fail-closed for fixed-dim providers (openai/voyage/
zeroentropy) and excludes openrouter (declares dims_options). Register modern
ollama embed model names.
#2209: litellm setup_hint now states the /v1 path convention and the docs
pointer is corrected to docs/integrations/embedding-providers.md.
* docs+test(ai): KEY_FILES current-state for provider-agnostic gateway + embed-preflight dims-unset test (#1249, #1250, #1292)
* fix(ai): point user_provided_dims_unset remediation at 'gbrain init' (config set rejects it) + coverage
Pre-landing adversarial review (P1): the new dims-unset guard told users to run
'gbrain config set embedding_dimensions <N>', which config.ts hard-rejects (it's a
schema-sizing field). Both consumer messages now point at 'gbrain init
--embedding-dimensions'. Adds: pgvector-cap-still-fires regression for the
trust_custom_dims passthrough, and a configureGateway backfill-invariant test.
* chore: bump version and changelog (v0.42.57.0)
Provider-agnostic plumbing wave: #1249 empty-env clobber, #1250 native baseURL
normalization, #1292 embedding dims-presence guard, #2271 trust_custom_dims
passthrough, #2209 litellm /v1 hint.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: sync embedding-providers guide for provider-agnostic gateway wave (v0.42.57.0)
Post-ship doc drift fix for the v0.42.57.0 AI-gateway wave:
- LiteLLM section now names the /v1 base-URL convention (#2209).
- Ollama section lists the newly-registered modern embedders qwen3-embed-8b
+ snowflake-arctic-embed-l-v2, and notes dims-trust for local recipes (#2271).
- llama-server section notes gbrain trusts the user-declared dimension (#2271).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: post-ship doc sweep for v0.42.57.0 provider-agnostic gateway wave
- KEY_FILES.md types.ts entry: document EmbeddingTouchpoint.trust_custom_dims
(#2271 passthrough tier, runs after dims_options + Matryoshka allowlists)
- ENGINES.md: embedding design-choice note now names the provider-agnostic
gateway delegation instead of the stale OpenAI-only parenthetical
- embedding-providers.md: drop an exact-duplicate doctor-8c paragraph
- llms-full.txt regenerated (ENGINES.md is inlined in the bundle)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: apply codex doc-review findings for v0.42.57.0 (base-URL env note, litellm multimodal)
- embedding-providers.md OpenAI section: document OPENAI_BASE_URL /
ANTHROPIC_BASE_URL bare-host /v1 normalization (#1250 user-facing surface)
- TL;DR table: litellm multimodal is backend-permitting (recipe declares
supports_multimodal: true, routed via the openai-compat multimodal path),
not "no"
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: pin engine-find-trajectory schema to 1536 + stop gateway-state leaks across shard files
CI shard 5 failed 7 findTrajectory tests with 'expected 1280 dimensions, not
1536': engine-find-trajectory hardcodes 1536-d vectors but sizes its schema
from AMBIENT gateway state in beforeAll — which runs before the
legacy-embedding-preload's per-test 1536 restore. A preceding file that ends
with a dimensionless configureGateway (facts-extract-silent-no-op) or a bare
resetGateway poisons the next fresh initSchema down to 1280-d columns. The
new test files in this PR reshuffled shard bin-packing and exposed the trap.
- engine-find-trajectory: pin OpenAI/1536 explicitly before initSchema (the
pattern bunfig's preload documents) — deterministic regardless of neighbors
- facts-extract-silent-no-op, diagnose-embedding-dims, embed-preflight:
restore the legacy 1536 pin in afterAll instead of ending reset/dimensionless
Reproduced: synthetic dimensionless-gateway file + old victim = the exact 7
CI failures; with the pin = 0. Verified in-process pair runs both orders.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(pglite): never steal the data-dir lock from a live holder (#2348)
A busy `gbrain dream`/`embed` holder whose 30s heartbeat lapsed (the JS event
loop is blocked during long synchronous WASM imports/CHECKPOINTs) used to get
its lock reaped past the steal-grace window. PGLite/WASM is strictly
single-writer, so a second OS process then opened the same data dir and
corrupted the catalog + pgvector extension state (58P01 / internal_load_library
/ "type vector does not exist"), recoverable only by wipe+restore. Reap ONLY a
dead PID; a live holder is never stolen — a wedged-but-alive or PID-reused
holder makes the acquire time out with a message naming the PID. Removes the
GBRAIN_PGLITE_LOCK_STEAL_GRACE_SECONDS knob (no longer meaningful).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(pglite): point a corrupted store at reinit-pglite recovery (#2348)
classifyPgliteInitError gains a `corrupt` verdict for the 58P01 /
internal_load_library / "vector does not exist" / "content_chunks does not
exist" signature (beating the generic wasm-runtime match), so an already-damaged
store gets actionable recovery (gbrain reinit-pglite / restore a backup) instead
of the wrong macOS-WASM hint. Updates KEY_FILES.md to current lock behavior.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* v0.42.55.0 fix(pglite): incident — never steal a live lock + corrupted-store recovery hint (#2348)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* v0.42.56.0 chore: re-bump past #2399 version collision + refresh ownerToken comment
#2399 (security wave) claimed 0.42.55.0; take the next slot. Also updates the
LockHandle.ownerToken JSDoc to current #2348 behavior (live holders are never
reaped, so reap-then-reacquire is dead-holder-only; token guard stays as
defense-in-depth).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(chronicle): register event + diary page types (#2390)
Life Chronicle Phase A.1. Adds `event` (timeline atom) and `diary`
(first-person interiority) as temporal-primitive page types under
life/events/ and life/diary/, extractable:false — registered in
ALL_PAGE_TYPES, both base schema packs, and the inferType prefix table,
with parity fixtures. Also lands the chronicle read result types
(ChronicleTimelineRow/ChronicleTimelineOpts/LastSeenResult) consumed by
Phase A.2.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(chronicle): event_page_id timeline projection + day/since/last-seen reads (#2390)
Life Chronicle Phase A.2. Adds a nullable event_page_id FK to
timeline_entries (migration v120; mirrored in schema.sql, pglite-schema,
schema-embedded) so a type:event page projects ONE date-index row keyed
to its depth page; a partial UNIQUE(event_page_id, date) makes
re-extraction with a changed summary an update, not a duplicate.
Dual-engine getTimelineForDate / getSince / getLastSeen filter the depth
page on deleted_at, hide soft-deleted event projections at READ time
(not just doctor), order by event effective_date for intra-day sequence,
and honor source scope (sourceIds[] > sourceId). Ops surface as
`gbrain day <date> [--week]`, `gbrain since <date> [--kind]`,
`gbrain last-seen <entity>`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(chronicle): auto-emit extractor — backstop + chronicle_extract job (#2390)
Life Chronicle Phase A.3. A put_page backstop (gated on status==='imported',
the auto-link/timeline trust gate, and the default-OFF auto_chronicle flag;
diary + event pages never eligible) enqueues a chronicle_extract minion job.
The job runs the extractor off the write path: deterministic when/who, an
injectable LLM judge (default = chat gateway; no-op when no gateway), an
all-or-nothing parse barrier (a malformed proposal writes NOTHING), then
content-addressed event pages + a timeline projection via the new dual-engine
upsertEventProjection (idempotent — re-run yields one event + one projection).
New: src/core/chronicle/{eligibility,config,extract-events,backstop}.ts,
engine.upsertEventProjection (both engines), the jobs.ts handler + a 10-min
timeout. 14 unit tests (eligibility, idempotency, parse barrier, backstop
gating + enqueue) green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(chronicle): quick-capture for diary + manual events (#2390)
Life Chronicle Phase A.5. `gbrain capture` now routes the default slug by
type (diary → life/diary/, event → life/events/, else inbox/) and accepts
--who/--what/--where/--kind/--depth to assemble the `event:` frontmatter
block for `--type event`. User-declared event keys win per-key over the
flags. Goes through the existing put_page → write-through → embed path. 6
new unit tests; existing capture tests stay green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(chronicle): bi-temporal per-entity ontology on the facts table (#2390)
Life Chronicle Phase B.10 — the feature's differentiator. Rather than a
parallel store, the open-world per-entity ontology EXTENDS the existing
`facts` table (eng-review G1): migration v121 adds dimension/value/value_hash
/dim_status columns + a deterministic partial-UNIQUE dedup key + an asof read
index. facts already supplies bi-temporal validity, supersession
(superseded_by), visibility (remote redaction), confidence, provenance, and
embedding — all inherited.
Dual-engine methods: mergeOntologyFact (deterministic value_hash dedup →
idempotent retry; same value corroborates; a different value forward-closes
the prior row's valid_until + superseded_by; a BACKDATED conflicting value is
recorded WITHOUT rewriting, surfaced by findOntologyConflicts), getOntology
with `--asof` valid-time travel (expired_at + status + validity-window in the
predicate so quarantined/superseded never leak), discoverOntologyDimensions,
findOntologyConflicts. Novel/LLM-proposed dimensions quarantine; a seed lexicon
canonicalizes name drift (job_role → role). 9 unit tests cover the full
lifecycle; typecheck pins both engines to the interface.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(chronicle): ontology ops — get/add/dimensions/contradictions (#2390)
Life Chronicle Phase B.11. Exposes the bi-temporal ontology over CLI + MCP
(contract-first, auto-generated): `gbrain ontology <entity> [--asof]`,
`gbrain ontology-add <entity> <dim> <value>`, `gbrain ontology-dimensions`
(meta-ontology rollup), `gbrain ontology-contradictions`. All reads route
through sourceScopeOpts. Privacy: ontology_get redacts diary-sourced
observations (source under life/diary/) for untrusted (remote) callers. 3 op
tests (incl. the remote-redaction path); 47 op-registry/tool-def/description
tests stay green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(chronicle): agent-context loader — volunteer_chronicle (#2390)
Life Chronicle Phase B.12. `loadChronicleContext` hands an agent the recent
timeline (last N days) + the validity-resolved current ontology for the
entities in play, in one zero-LLM payload, so it orients before acting — the
exact gap behind fumbled chronology. Pure composition over getSince +
getOntology (no new SQL). Exposed as the `volunteer_chronicle` read op
(`gbrain orient [--days] [--entities a,b]`); diary-sourced ontology is redacted
for remote callers. 2 loader tests + op-registry green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(chronicle): backfill op — sweep existing meetings into events (#2390)
Life Chronicle Phase A.8. `chronicle_backfill` (local-only admin op;
`gbrain chronicle-backfill [--since] [--limit] [--dry-run]`) lists existing
meeting/conversation/calendar pages (source-scoped via listPages), filters
through the chronicle eligibility predicate, and enqueues one chronicle_extract
job per eligible page so existing brains populate the timeline. --dry-run
counts only; per-page enqueue failures are surfaced in `errors`, never
swallowed. 2 op tests (dry-run count + enqueue).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(chronicle): delight — on-this-day + narrative rendering (#2390)
Life Chronicle Phase A.6 (delight). Dual-engine getOnThisDay (events from the
same month-day in prior years; `gbrain on-this-day [--date]`) reusing the
chronicle JOIN shape (deleted-event hiding + source scope). A pure
renderTimelineNarrative turns timeline rows into prose; `gbrain day --narrative`
returns it alongside the events. 5 tests. (Coverage gap-detection ships with
the advisor collectors next.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(chronicle): proactive advisor collector (#2390)
Life Chronicle Phase A.7. A brain-state advisor collector surfaces two
proactive signals in `gbrain advisor`: unresolved ontology conflicts (warn,
→ `gbrain ontology-contradictions`) and recent meetings with no timeline
coverage (info, → `gbrain chronicle-backfill`). Advisory-only (no dispatch_id);
runs over MCP too (not workspace-dependent); tolerant of pre-migration brains.
3 tests + advisor-op-gate green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(chronicle): doctor chronicle_projection_health probe (#2390)
Life Chronicle Phase B.13. An always-run doctor check (keyed off the
event_page_id schema, NOT a migration verify-hook) counts timeline
projections whose event page is soft-deleted — hidden at read time, surfaced
here as a cleanup backlog (`gbrain integrity auto`). Tolerant of pre-migration
brains. 1 detection test. (auto_chronicle / chronicle.tz flags already work
via getConfig defaults; their docs land with document-release at ship.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(chronicle): E1 temporal recall — chronicle-type boost on temporal queries (#2390)
Life Chronicle Phase A.4 (E1, ambient temporality). Rather than a separate RRF
arm (which needs chunk hydration + risks the fusion path), E1 is a bounded
post-fusion boost: applyChronicleTypeBoost lifts `event`/`diary` results on
temporal queries, wired INSIDE runPostFusionStages' `recency !== 'off'` branch
so it fires ONLY on temporal intent — non-temporal search is bit-for-bit
unchanged (proven by 110 passing search-path tests). Bounded ([1.0,1.25]) +
floor-gated like the other metadata stages; attribution via `chronicle_boost`.
3 unit tests. (Empirical precision/negative measurement lands with the eval.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(chronicle): feature eval — gbrain eval chronicle (PRIMARY proof) (#2390)
Life Chronicle Phase A.9, the North-Star proof. A deterministic, CI-safe eval
(brings its own in-memory PGLite; no LLM, no gateway) builds a synthetic month
corpus with a known gold chronology + a planted ontology supersession + a
planted conflict, then scores the chronicle layer on six gold tasks: day
reconstruction (intra-day order), last-seen exact date, ontology supersession,
--asof valid-time travel, contradiction surfacing, and source isolation.
`gbrain eval chronicle [--json]` exits 0 iff all pass — currently 6/6. The OFF
baseline (raw meeting pages) structurally can't order intra-day events or
time-travel ontology; the ON path does. (The live-LLM OFF-vs-ON agent arm +
LongMemEval temporal slice are a follow-up; this deterministic bar gates CI.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(chronicle): pre-landing review — conflict validity, parse-barrier date, remote conflict redaction (#2390)
Three bugs caught by the codex pre-landing review on the diff:
1. findOntologyConflicts ignored valid_until, so a normal forward supersession
(founder→advisor from two sources) falsely reported as a live conflict. Now
restricted to currently-open rows (valid_until IS NULL) in both engines.
2. The extractor parse barrier accepted any when-string >= 4 chars; a non-date
value slipped past, wrote the event page, then threw on the projection's
::date cast (partial write). isValidProposal now requires a real parseable date.
3. The ontology_conflicts op had no remote diary redaction (ontology_get did);
remote callers now get diary-sourced values filtered, and conflicts that lose
their disagreement after redaction are dropped.
Three regression tests added; 29 chronicle tests + eval 6/6 green; typecheck clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* v0.42.56.0 feat(chronicle): Life Chronicle — temporal timeline + diary + bi-temporal ontology (#2390)
Bump VERSION + package.json to 0.42.56.0 and add the CHANGELOG entry for the
Life Chronicle feature (#2390, closes duplicate #2388).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(chronicle): register #2390 surfaces with the five CI guard suites (#2390)
CI caught five guard tests that pin registration invariants my diff tripped:
- schema-bootstrap-coverage: the four v122 facts ontology columns join
COLUMN_EXEMPTIONS (facts is migration-created; the partial indexes live
inside v122; every reader filters dimension IS NOT NULL — same precedent
as facts.claim_metric et al).
- no-valid-until-write (R8): both engines' mergeOntologyFact forward-
supersession is a deliberate, documented valid_until write authority
(engine-layer, dimension IS NOT NULL only; the contradiction probe still
never mutates).
- doctor-categories: chronicle_projection_health registered under
BRAIN_CHECK_NAMES (same class as child_table_orphans).
- checkTypeProliferation: the test is now threshold-relative (computes
declared from the active pack, seeds declared+6) so base-pack growth
can't silently move the fixed threshold again.
- schema-cli: gbrain-base page-type count 25 → 27 (event + diary), with
assertions on both new types.
All 41 guard tests green; typecheck clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: file Life Chronicle follow-up TODOs (v0.42.56.0, #2390)
Eight deferred items from the CEO/eng review decisions (auto-emit default-flip
fast-follow, live eval arm + LongMemEval slice, passive diary consent,
interval-splitting, federation, place-as-entity, meta-ontology dashboard,
materialized daily pages), each with decision provenance.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(architecture): KEY_FILES entry for the Life Chronicle module (#2390)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(security): confine routing dotfiles, skills dir, slugs, and transcription exec
Shared src/core/path-confine.ts consolidates the realpath-containment idiom
(moved from sources-ops.ts) and adds isTrustedDotfile + isWriteTargetContained.
- .gbrain-source (source-resolver) and .gbrain-mount (brain-resolver) walk-up
dotfiles are now lstat trust-gated: a symlink, foreign-owned, or world-writable
file is refused on multi-user hosts (#418), fail-closed on stat error.
- resolveWorkspaceSkillsDir + every skills-dir tier (env, cwd_walk_up, repo_root,
cwd_skills, install_path) route through realpath containment so a symlinked
workspace/skills can't escape the declared workspace (#419).
- resolveSourceId/resolveBrainId realpath both sides of the registered local_path
/ mount prefix match so a symlinked cwd can't misattribute source/brain.
- validateSlug rejects NUL/control, bidi/RTL overrides, backslashes, and
URL-encoded path separators at the shared putPage/updateSlug chokepoint;
write-through confirms the file path stays within the source tree.
- transcribeLargeFile uses execFileSync arg-arrays + fs.rmSync (no shell), so a
path with shell metacharacters is never parsed by a shell (#245).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(security): default dynamic-registration clients to authorization_code
Self-registered DCR clients (the unauthenticated network registration path)
previously defaulted to the client_credentials grant, which bypasses the
/authorize consent screen. They now default to authorization_code; an explicit
client_credentials request is rejected with invalid_client_metadata unless the
operator opts in with the new --enable-dcr-insecure flag. A loud stderr WARNING
prints at startup whenever DCR is enabled (#1353). Manual CLI/admin client
registration is unchanged (operator-trusted).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(security): schema-lint hardening migration (search_path + view security_invoker)
Migration v120 brings existing brains to the same posture as fresh installs:
- ALTER VIEW page_links SET (security_invoker = on) on Postgres so the view
honors the caller's RLS instead of the owner's (the view-through-RLS bypass).
- ALTER FUNCTION ... SET search_path on the gbrain-owned trigger/event functions
(both engines, IF EXISTS so engine-only functions are skipped; body untouched,
so the load-bearing auto_enable_rls event trigger is unchanged). Closes#171.
- Broaden the BYPASSRLS preflight in the historical RLS migration gates to honor
superuser and inherited-role BYPASSRLS, so a superuser-connected fresh install
no longer aborts (#1385).
Fresh-install function definitions in schema.sql / pglite-schema.ts are
born-correct (regenerated schema-embedded.ts). scripts/check-search-path.sh is a
new CI guard (wired into verify) that fails if a trigger function in the schema
base files is added without SET search_path. Postgres-only assertions live in the
bootstrap E2E; the PGLite path is covered by test/migration-v120.test.ts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* v0.42.55.0 fix(security): dotfile/skills/slug confinement, DCR consent default, schema-lint migration
Bump VERSION + package.json to 0.42.55.0 and add the CHANGELOG entry for the
security-hardening wave (#418#419#245#1353#1647#171#1385).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(security): note the DCR consent default in SECURITY.md (#1353)
The "disable client_credentials, only allow authorization_code" guidance is now
the built-in DCR default; document the new --enable-dcr-insecure escape hatch.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(todos): add takes_search + code_def to the federated by-slug P1 (#2200)
The v0.42.55.0 eng-review codex pass flagged takes_search (holder-allowlist only)
and code_def (brain-wide raw SQL over content_chunks) as remaining same-class
surfaces. Noted on the existing #2200 P1 follow-up, with the caveat that the
#2399 close-list deliberately keeps #1371/#2200 open until this lands.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(security): correct plpgsql alias collision in #1385 BYPASSRLS gate (real-PG)
The broadened BYPASSRLS preflight aliased `pg_roles r`, but several RLS DO-blocks
already declare `r record` for their backfill FOR loop, so plpgsql resolved
`r.oid`/`r.rolbypassrls` to the unassigned record variable → "record \"r\" is
not assigned yet" on real Postgres (PGLite tolerated it; the DATABASE_URL-gated
e2e jobs are the backstop). Renamed the subquery alias to `pr` at all 10
migrate.ts sites; also broadened the schema.sql base RLS gate the same way (with
the `pr` alias) for #1385 consistency on superuser fresh installs, and
regenerated schema-embedded.ts.
Also fixes a PRE-EXISTING engine-parity bug (confirmed failing on clean
origin/master): the relationalFanout shape compared `canonical_chunk_id`, a
serial id that diverges between a fresh PGLite engine and a shared Postgres DB
(setupDB TRUNCATEs without RESTART IDENTITY). Compare its presence, not the
exact value.
Validated on real Postgres (pgvector/pg16): migration v120 applies, the v35 RLS
backfill runs, and engine-parity + postgres-bootstrap + jsonb-parity are green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(sync): op_checkpoints pin write double-encodes jsonb — every sync aborts (#2339)
recordCompleted bound JSON.stringify(array) to a $3::jsonb param via postgres.js
.unsafe(), double-encoding it into a jsonb string scalar that violates the v119
op_checkpoints_completed_keys_array CHECK — aborting every multi-source sync on
real Postgres at the first checkpoint write. PGLite parses the string silently,
which is why unit tests stayed green and it shipped. Cast through $3::text::jsonb
so the text->jsonb cast parses a genuine array. Adds a DATABASE_URL-gated parity
test + a dedicated Postgres CI job so the guard can never silently skip.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(db): sweep positional jsonb double-encode sites + AST CI guard (#2324)
Every executeRaw/.unsafe site that bound JSON.stringify(x) to a bare positional
jsonb cast double-encodes on real Postgres (same class as #2339). Sweep them all
to the text::jsonb form across query-cache, sources-ops, llm-base,
calibration-profile, impact-capture, subagent, receipt-write, traversal-cache,
symbol-resolver, and the agent/sources commands. Adds scripts/check-jsonb-params.mjs
(AST-lite scanner for the positional form the legacy template grep misses, incl.
generic-typed calls), wired into check-jsonb-pattern.sh, with a self-test. PGLite's
native db.query is not scanned — it parses text to jsonb natively, so the bug can't
occur there.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(search,eval): alias-hop injected results carry page_id (contradiction-probe crash)
applyAliasHop injected synthetic SearchResults without page_id (the `as
SearchResult` cast hid the missing field), so listActiveTakesForPages bound
undefined/NaN into ANY($1::int[]) and crashed the whole contradiction probe on
real Postgres. Stamp page_id=page.id at the injection site and add a finite-id
filter in generateIntraPagePairs as a defensive backstop (mirrors hybrid.ts:63).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(engines): positional jsonb binding rule (text::jsonb vs the double-encode trap)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* v0.42.53.0 fix(sync,db): #2339 op_checkpoints jsonb double-encode + bug-class sweep + CI guard
Bumps VERSION + package.json to 0.42.53.0, adds the CHANGELOG entry, and
regenerates llms-full.txt. Ships the #2339 sync-abort hotfix, the repo-wide
positional jsonb double-encode sweep, the alias-hop contradiction-probe crash
fix, and the new positional-form CI guard.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: post-ship sync — jsonb invariant now covers the positional form + new guard
CLAUDE.md JSONB invariant + KEY_FILES (sql-query, check-jsonb-pattern, op-checkpoint)
now describe the #2339 positional double-encode class, the $N::text::jsonb fix, and
the new check-jsonb-params.mjs guard. Regenerates llms-full.txt.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(supervisor): pin LOCK_HELD fence-exit is never counted as a crash (#2227)
A duplicate supervisor loses the queue-scoped DB singleton lock (#1849) and
exits LOCK_HELD before spawning a worker or emitting 'started'. summarizeCrashes
counts only worker_exited, so the fence path is structurally uncountable. Pin it
so a future refactor that logs worker_exited on the fence path fails here instead
of silently re-introducing the crash-budget breaker-trip loop.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(autopilot): per-source cycle binds FS phases to source.local_path, not global repo (#2194#2227)
A per-source autopilot-cycle inherited the global sync.repo_path as brainDir while
stamping DB freshness for source_id — mixed scope. FS phases (sync/lint/extract)
ran against the wrong tree, so the failure-cooldown and freshness gates would
attribute work to the wrong source. Resolve the source's local_path in the handler
(reuse the archive-recheck SELECT) and bind brainDir to it; a pure-DB source gets
null (FS phases skip) instead of falling through to the global checkout. Legacy
no-source dispatch keeps the global repoPath. Prerequisite for the cooldown/split
commits (codex outside-voice #8). Resolves TODOS:634.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(supervisor): detect a live supervisor via the DB lock under split $HOME (#2227)
jobs supervisor status + doctor read the HOME-derived pidfile, so a supervisor
started under a different $HOME (keeper=/root vs ops=/data) read as 'not running'
while healthy — the false signal that drives an operator to spawn a duplicate.
Both surfaces now fall back to the queue-scoped DB singleton lock (#1849), the
HOME-independent authority, when the pidfile shows nothing. New isLockHolderLive
keys on lock freshness (ttl + heartbeat steal-grace), never process.kill, so PID
reuse can't false-positive (pid-liveness-alone-pid-reuse). Status surfaces the
holder host/pid + recorded concurrency/max-rss from the latest started event.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(supervisor): degraded retry instead of permanent give-up on crash storm (#1994#2227)
max_crashes_exceeded gave up forever, so a transient DB-pooler blip that tripped
the soft budget wedged the queue until a human restart (#2227's breaker-trips tail).
Crossing the soft budget now enters degraded mode: keep respawning with capped
exponential backoff (60s cap — a paced retry, not a hot loop) and emit a loud
crash_budget_degraded health_warn. The existing stable-run reset clears the count
once a respawn survives >5min, so a recovered DB self-heals. Permanent give-up
fires only at a much-higher hard ceiling (maxCrashes × 10), tunable/disablable via
GBRAIN_SUPERVISOR_HARD_STOP_CRASHES (0 = never). Resolves TODOS:92.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(autopilot): clamp fan-out to worker concurrency + doctor warning (#2194)
Fan-out resolved to 4 (Postgres) regardless of worker --concurrency, so surplus
cycles queued behind the worker and raced the stalled-sweeper. Two fixes for the
same mismatch:
- resolveEffectiveFanoutMax clamps to max(1, concurrency-1) (reserve a slot),
gated on a LIVE DB-lock holder so a stale started-audit row can't shrink
throughput (codex #9/D5); no live holder → unknown → unclamped base. Escape
hatch autopilot.fanout_clamp_to_concurrency.
- doctor's autopilot_fanout_concurrency check warns when fan-out exceeds
effective slots — the misconfig was silent before. Advisory (started-event
concurrency), wired into both doctor surfaces.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(autopilot): per-source failure cooldown — break the dead-job storm (#2194)
Only SUCCESS gated dispatch, so a source whose cycle kept failing/timing-out
re-fanned-out every 5-min tick forever (200+ dead jobs/24h). Now a failed source
backs off with bounded exponential cooldown (10→120min). Read at DISPATCH from
minion_jobs dead/failed rows (timeouts/RSS-kills dead-letter via SQL and never
run handler code, so a write-only hook would miss them) AND re-checked at CLAIM
time in the handler (codex #5: already-queued/retrying jobs). A success clears it
(codex #7); null-source rows excluded (codex #6); engine-parity via executeRaw.
Disable with autopilot.failure_cooldown_min=0. Fail-open if config/history reads
error. Surfaced via fanout_cooldown_skipped + the fanout summary.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(autopilot): split the cycle — per-source phases + one global-maintenance job (#2194#2227)
N per-source cycles each ran the brain-wide global phases (embed-all/orphans/
purge/…) concurrently, thrashing the same rows and taking the worker 4→10GB in
<60s → RSS-kill → orphaned stalls. Split them: per-source jobs now run only
source-scoped (+ mixed) phases and stamp last_source_cycle_at; a new
autopilot-global-maintenance job runs the global phases ONCE per window
(idempotency_key + maxWaiting:1 = structural single-flight) and stamps
autopilot.last_global_at. This is the codex-endorsed design that replaced the
rejected skip-and-stamp-fresh approach (codex #1/#2): no freshness poisoning, no
starvation — global work always runs as its own job, never marked done when it
wasn't. PHASE_SCOPE is now a runtime partition (GLOBAL ∪ NON_GLOBAL == ALL).
last_full_cycle_at still written for doctor/legacy (no longer a global gate).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(doctor): guard nullable engine in supervisor DB-lock fallback (#2227)
Follow-up to the supervisor-visibility commit: doctor's engine binding is
BrainEngine | null, so the inspectLock fallback must guard on a non-null engine
(tsc TS2345). No behavior change — a null engine simply skips the DB-lock probe
and falls back to the pidfile reading, as before.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(doctor): categorize autopilot_fanout_concurrency check as ops (#2194)
Follow-up to the fan-out/concurrency commit: the doctor-categories drift guard
requires every check name in doctor.ts to belong to exactly one category set.
Add the new autopilot_fanout_concurrency check to OPS_CHECK_NAMES (infrastructure
liveness, alongside wedged_queue/supervisor).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: update KEY_FILES for the autopilot cycle split + supervisor degraded-retry (#2194#2227)
Post-ship document-release: refresh the KEY_FILES current-state entries that
drifted — cycle.ts (GLOBAL/NON_GLOBAL phase split + last_source_cycle_at /
autopilot.last_global_at), jobs.ts (per-source local_path brainDir, claim-time
cooldown, autopilot-global-maintenance handler), supervisor.ts + child-worker
(degraded retry instead of permanent give-up; hard ceiling), db-lock.ts
(isLockHolderLive), handler-timeouts (new handler). Regenerated llms bundle.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(minions): handleTimeouts counts the timed-out run as a spent attempt (#1737)
The per-job timeout_at dead-letter (handleTimeouts) set status='dead' without
incrementing attempts_made, unlike the wall-clock and stall dead-letter siblings.
It is the FIRST killer to fire for the long-lane handlers (subagent / embed-backfill
/ autopilot-cycle) because timeout_ms is stamped at submit, so a timed-out long job
reported `attempts: 0/N (started: N)`. Mirror the siblings with attempts_made + 1
(terminal, no retry). Safe against double-count: the worker sweep runs handleStalled
-> handleTimeouts -> handleWallClockTimeouts sequentially and awaited, each guarded on
status='active', so the first to dead-letter excludes the row from the rest.
Regression assertions added (test/minions.test.ts + e2e/minions-resilience.test.ts)
so the increment can't be silently dropped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(agent): recognize trailing switches in `agent run`, keep prompts freeform (#1738)
parseRunFlags() broke flag parsing at the first positional token, so any flag
after the prompt (`gbrain agent run "do X" --detach`) was swallowed into the
prompt string and silently ignored. Now the no-value switches --detach/--follow/
--no-follow are hoisted when they trail the prompt, while everything else stays
verbatim: an unknown --word is treated as prompt text (no "unknown flag" throw),
a --switch mid-prompt is preserved, and `--` suppresses hoisting entirely for a
literal escape. Value-flags now reject a missing or flag-shaped value (and
--max-turns/--timeout-ms a non-number) instead of capturing undefined/NaN.
Contract change: a prompt that starts with or trails an unguarded --word no
longer errors; a literal trailing --detach needs `--`. Help text updated; tests
revised + extended (test/agent-cli.test.ts).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(sync): honest live-sync status + progress-aware stall-abort (#1950)
Finishes the #2255 honest-freshness story for two gaps it left.
(a) `gbrain sources status` printed "idle" while a sync proc held the per-source
lock (the reported bug). New shared liveSyncStatus() helper in db-lock.ts reads
the SAME live-lock signal `gbrain doctor` uses; runStatus now shows "running"
(BACKFILL column + a sync_running field in --json) and suppresses the misleading
"never synced" warning while a sync is live. One helper, so the surfaces can't
drift (doctor/status retrofit tracked as a follow-up).
(b) A sync wedged-but-alive kept refreshing its lock heartbeat (it fires on its
own timer) and hadn't hit the wall-clock deadline, so only a manual pkill freed
it. New in-band stall watchdog keys off FORWARD IMPORT PROGRESS (progress.tick),
not the heartbeat: if no file completes for GBRAIN_SYNC_STALL_ABORT_SECONDS
(default 900s), it aborts via a controller composed into opts.signal, so the
drain returns partial() (last_commit unchanged, next run resumes from the
checkpoint) and withRefreshingLock releases the lock. Limits, documented in
code: a single file slower than the window trips it; a fully starved event loop
won't fire the timer (the wall-clock hard deadline is that backstop).
Tests: liveSyncStatus (live/expired/none/per-source) in db-lock-inspect; the
resolveStallAbortSeconds env matrix in sync-hard-deadline.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(status): version field + per-section --deadline-ms budget (#1984)
`gbrain status` had no version in its JSON envelope and could hang on a slow
connection with no way to get a partial answer. Two additions:
- version: the StatusReport JSON now carries the local gbrain CLI version so a
poller can pin behavior to a build. Thin-client also surfaces remote_version
(the brain server's version), and the get_status_snapshot MCP op reports its
version for that parity.
- --deadline-ms=N / --fast: a shared wall-clock budget. Each section is raced
against the REMAINING budget via Promise.race (NOT process-watchdog, which
SIGKILLs and can't return partial output), so one slow/hung section can't
strand the snapshot — it's marked stale and the rest still return. The
envelope gains partial:true + stale_sections[]; exit code stays 0 (a snapshot
was produced). Invalid --deadline-ms → exit 2.
Tests: parseDeadlineFlag + withSectionDeadline (hermetic), the usage-error exit,
version presence in the PGLite envelope, and the op's version key.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(sync): report stall_timeout distinctly + document in-flight limit (#1950)
Pre-landing review (codex + adversarial): the stall watchdog aborted opts.signal
but the per-iteration abort checks returned partial('timeout'), collapsing a
wedge-reap into a user --timeout/SIGINT so JSON consumers couldn't tell them
apart. Add a 'stall_timeout' reason (set via a stallAborted flag) on the three
import-loop abort sites; deletes/renames-phase and checkpoint sites stay 'timeout'.
Sharpen the watchdog comment: the abort is observed BETWEEN files, so a hang
inside a single importFile is not interrupted until it returns (TODO: thread a
cancellation signal through importFile).
* fix(agent): `--` escape suppresses trailing-switch hoisting anywhere (#1738)
Pre-landing review: the leading-flag loop breaks at the first positional, so the
`escaped` flag only fired for a leading `--`. A `--` placed after a positional
left trailing-switch hoisting active, so `agent run note -- body --detach`
silently detached and dropped the `--` as junk. Suppress hoisting whenever a
literal `--` appears in the prompt. Regression test added.
* fix(status): deadline-ms usage-error + scoped stale_sections + cancel losing remote call (#1984)
Pre-landing review (codex): (1) bare `--deadline-ms` with no value silently fell
through to no-budget/--fast instead of a usage error; (2) thin-client timeout
reported both sync+cycle stale even under `--section sync`, naming a section the
caller excluded (local path was already correct); (3) the section race abandoned
the remote promise locally but didn't cancel the in-flight MCP call — pass the
budget as timeoutMs so the losing side actually cancels. Regression test added.
* v0.42.52.0 fix(reliability): autopilot dead-job storm + supervisor wedge + sync/status/minion reliability (#2194#2227#1994#1737#1738#1950#1984)
Bundles the already-reviewed autopilot/supervisor stabilization (#2194#2227#1994: cycle split, per-source failure cooldown, fan-out clamp, degraded
supervisor retry, DB-lock live-supervisor detection) with four operational
fixes: minion timeout attempt-accounting (#1737), agent-run trailing-flag
parsing (#1738), honest live-sync sources status + progress-aware stall
watchdog (#1950), and status version + --deadline-ms partial result (#1984).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: document GBRAIN_SYNC_STALL_ABORT_SECONDS env knob (#1950)
Post-ship doc sync (/document-release): add the sync stall watchdog env var to
the CLAUDE.md sync-tuning table (Five → Six knobs) + regenerate the llms bundle.
* test: quarantine #2249 fanout tests as *.serial (R1 env-isolation) (#2194)
The cherry-picked autopilot-fanout-clamp + doctor-autopilot-fanout-concurrency
tests mutate process.env.GBRAIN_AUDIT_DIR in beforeEach/afterEach, which the
check:test-isolation R1 lint flags (parallel shards load multiple files per
process). Rename to *.serial.test.ts (sanctioned quarantine — they run under
--max-concurrency=1) instead of restructuring the reviewed test bodies. No logic
change; both files stay green (9 tests). Fixes the failing verify CI check.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(sync): contention-free page-generation clock — sequence swap
The page-generation clock backed the query-cache Layer-1 bookmark via a
FOR EACH STATEMENT trigger running `UPDATE page_generation_clock SET
value=value+1 WHERE id=1`. That took a transaction-length RowExclusiveLock
on one tuple, so every concurrent page writer serialized on the prior
writer's COMMIT — sync ran at ~0.8 cores regardless of worker count.
Swap to a SEQUENCE bumped by nextval() (a microsecond LWLock, never a row
lock). The clock's only contract is monotonic advancement on any page
INSERT/UPDATE/DELETE; last_value is non-transactional, so rolled-back or
concurrent-uncommitted writers only OVER-invalidate the cache (lose a hit),
never serve stale.
- migration v118: CREATE SEQUENCE + load-bearing 2-arg setval (is_called=
true, floor 1, seeded >= old clock and MAX(generation)) + repoint the
trigger function body + DELETE query_cache so no old-clock bookmark
survives the swap. v107 left immutable.
- query-cache-gate.ts: 3 readers -> SELECT last_value FROM page_generation_clock_seq.
- schema.sql + pglite-schema.ts (+ regenerated schema-embedded.ts) ship the
sequence on fresh install; table + trigger names retained.
- tests: clockValue reads last_value; mechanism proof (trigger fn uses
nextval not the row UPDATE); rollback-advances-clock safety pin; real
PGLite sequence round-trip (is_called gotcha); shape test requires _seq.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(sync): op_checkpoints array-shape guard — CHECK + repair + defensive loader
completed_keys is JSONB and the checkpoint loader runs
jsonb_array_elements_text over it. A non-array (scalar) value makes that
throw "cannot extract elements from a scalar", which takes down the whole
UNION load — including the valid op_checkpoint_paths child rows — and loses
all checkpoint progress for that key. No current writer produces a scalar,
but an older binary / external script / future bug could.
Make the corruption class structurally impossible and self-healing:
- migration v119: LOCK TABLE (so an out-of-band scalar can't land between
repair and constrain; no-op on single-connection PGLite), repair any
pre-existing scalar to '[]' (op_checkpoint_paths child rows are the
append-only source of truth, so the reset loses nothing), then add the
named CHECK (jsonb_typeof(completed_keys) = 'array') via a pg_constraint
IF NOT EXISTS guard. A DB-enforced always-on guard — the correct pattern
vs a migration verify-hook, which never runs on already-stamped brains.
- schema.sql + pglite-schema.ts (+ regenerated schema-embedded.ts) ship the
same NAMED inline CHECK so fresh installs match migrated brains and v119
skips the duplicate.
- op-checkpoint.ts loader: gate the legacy arm on jsonb_typeof = 'array' so
a scalar parent is skipped (children still load) instead of throwing the
whole union, and log a specific corruption warning when one is seen.
- tests: CHECK rejects a scalar (exactly one constraint, no blob+migration
dupe); loader survives a scalar parent and returns the children; v119
repair converts a scalar to '[]'.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(doctor): report actively-running sync via live lock, not stale freshness
A slow source that makes partial progress every cycle but never fully
completes used to read as permanently "stale" / "never synced" because
last_sync_at only advances on a full successful sync. The naive fix
(treat recent checkpoint banking as "in progress") is unsafe: a blocked
sync banks the good files then writes no anchor, so banking can't tell
in-progress from wedged.
Use the only honest signal: a LIVE, non-expired per-source sync lock
(inspectLock + syncLockId against gbrain_cycle_locks). Every non-skipLock
sync holds it and refreshes it; a blocked/failed sync's process has exited
(no lock row) and a wedged holder stops refreshing (TTL lapses), so either
correctly falls through to the stale path and is NEVER masked. An
actively-syncing source (including a never-synced source doing its first
sync) counts as synced_recently, preserving the pinned 3-bucket invariant.
The lock lookup reuses doctor's existing dynamic db-lock import and swallows
any throw (stub engine, pre-lock-table brain) to false, so it can only ADD
an in-progress verdict, never suppress a real stale one.
Tests (real PGLiteEngine + real lock rows): stale+no-lock -> fail;
stale+live-lock -> ok; never-synced+live-lock -> ok; never-synced+no-lock
-> fail; expired-TTL lock -> fail (wedged not masked); blocked source with
banked checkpoint rows but no lock -> still fail.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(sync): honest --force-break-lock diagnostic when no lock is held
--force-break-lock used to emit the same terse "Lock ... is not held
(nothing to break)" line and exit 0 even when a sync was genuinely wedged,
sending the operator down a dead end — the wedge was not a held lock. Keep
rc=0 (breaking a non-existent lock is idempotently successful; flipping the
exit code would break automation), but under --force say plainly that
nothing was broken and point at the real next step (gbrain sync / gbrain
doctor) plus a `wedge_hint` field in --json output. The non-force path is
byte-for-byte unchanged.
runBreakLock is exported for the test. Tests: force+no-lock -> wedge_hint
JSON + human hint, rc 0; non-force+no-lock -> unchanged terse line, no hint.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(doctor): surface the in-progress sync holder in the freshness message
Plan-completion follow-up to the BUG 4 live-lock signal: when a source is
actively syncing, name the holder (pid + host) in the check message instead
of silently folding it into synced_recently. The note is appended only when
something is in progress, so steady-state messages stay byte-for-byte
unchanged (the pinned exact-message + 3-bucket-invariant tests still pass).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(sync): pre-landing review fixes — monotonic clock seed, scoped CHECK guard
Adversarial (codex) review of the implementation diff caught three:
- P1 (correctness): the fresh-schema setval was not monotonic. initSchema
replays the schema blob, and the unconditional setval(MAX(generation))
could move page_generation_clock_seq.last_value BACKWARD on an
already-upgraded brain, letting a stored query_cache bookmark serve stale
rows. Seed via GREATEST over the sequence's OWN last_value (+ old table
value + MAX(generation)) in all 3 fresh schemas and migration v118, so a
replay is idempotent — mirrors the old table's ON CONFLICT DO NOTHING.
Pinned by a new monotonic regression test.
- P2: v119's CHECK-exists guard keyed on conname only (not globally unique).
Scope it to conrelid = 'op_checkpoints'::regclass.
- P3: in-progress note ran into the prior sentence in fail/warn doctor
messages; separate it with '. '.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: make Anthropic/ZE no-key tests hermetic against a dev config key
These "no key" tests cleared only ANTHROPIC_API_KEY / ZEROENTROPY_API_KEY
from the env, but hasAnthropicKey() and checkZeEmbeddingHealth() also read
the key from ~/.gbrain/config.json. On a dev machine whose real config holds
a key, the no-key assertions flipped and the tests failed locally (they
passed only in key-less CI). Add a shared with-env emptyHome() helper and
point GBRAIN_HOME at an empty dir in every no-key path so loadConfig finds
nothing — matching the already-hermetic anthropic-key / gateway-probe tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: bump version and changelog (v0.44.1.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(key-files): sync doctor + op-checkpoint entries to v0.44.1.0 truth
checkSyncFreshness now reports an actively-running sync via the live
per-source lock (names holder pid+host, counts as synced_recently) instead
of flagging it stale; loadOpCheckpoint gates the legacy union arm on
jsonb_typeof = 'array' so a scalar parent can't take down the whole load,
and migration v119's CHECK constraint makes the corruption class
structurally impossible. Reference docs describe current behavior only —
both entries updated in place, no release-clause appends. Guard + llms
freshness test green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: re-version to v0.42.51.0 (natural next-off-master)
Maintainer override of the queue allocator's leap to 0.44.1.0 (it jumped past
in-flight sibling PR claims at 0.42.50/0.43.0/0.44.0). Take the natural next
slot in the 0.42.x line above the immediate sibling claim (0.42.50.0); a
merge re-bump resolves any collision if a cathedral PR lands first.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* ci(e2e): bound + retry the OpenClaw install so a transient npm hang can't burn the Tier 2 budget
The Tier 2 (LLM Skills) job failed at 30m16s — the `npm install -g
openclaw@2026.4.9` step hung on a transient npm/registry stall (orphan
`npm install openclaw` was still running at cancel time) and consumed the
entire 30m job budget that v0.42.50.0 (#2254) introduced. The install
normally finishes in under a minute (Tier 2 is ~4m end to end on master),
so this is flaky-install infra, not a test failure.
Wrap the install in `timeout 120` + a 3-attempt retry loop with an 8-minute
step backstop: a hung attempt is killed in 2 min and retried instead of
eating the whole job. Same bound-the-hang philosophy as #2254's job timeouts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* ci: cancel superseded runs + per-job timeouts on test.yml & e2e.yml
Ports the GH-Actions hygiene gbrain already uses in heavy-tests.yml to the two
hot-path workflows. concurrency cancels a superseded run (keyed on PR number for
pull_request events — fork-safe — with github.ref fallback for push/scheduled);
frees runners and stops a stale-SHA run reporting a flaky failure on an obsolete
commit. Per-job timeout-minutes (test matrix 15, verify 12, serial 15, slow-* 12,
e2e tier1 20 / tier2 30, trivial jobs 5-10) convert a wedged job from a 6-hour
zombie (GitHub's default) into a fast legible fail. fail-fast:false already set.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* ci: add actionlint workflow (rhysd/actionlint v1.7.11, SHA-pinned)
Lints workflow YAML on .github/workflows/** changes so a malformed workflow /
bad action ref / missing-permission bug is caught before it ships a broken
pipeline. gbrain edits these workflows often; cheap preventive guard. Mirrors
GStack's actionlint job, SHA-pinned to gbrain's convention.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* ci(e2e): scrub operator/agent env before E2E (hermetic runner)
A dev or Conductor shell exports CONDUCTOR_*/MCP_*/OPENCLAW_*/GBRAIN_* overrides
that silently change test behavior, making hermetic E2E non-hermetic and its
failures unreproducible across machines. run-e2e.sh drops those prefixes before
bun starts (denylist — PATH/HOME/TMPDIR/DATABASE_URL survive; GBRAIN_HOME kept
for the existing HOME isolation). Adapts GStack's buildHermeticEnv to gbrain's
shell runner. Verified: a 78-test e2e file passes with DATABASE_URL surviving +
a planted GBRAIN_BRAIN_ID scrubbed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* v0.42.50.0 ci: reliability hardening — cancel-superseded + per-job timeouts + actionlint + hermetic E2E env
Ports GStack's CI-reliability hygiene to gbrain's hot-path workflows: concurrency
cancel-in-progress (PR-number keyed), per-job timeout-minutes (no more 6-hour
zombie jobs), an actionlint workflow, and an operator-env scrub in run-e2e.sh.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(pace): composable DB-contention pacer primitive
createDbPacer/createNoopPacer (concurrency permit + in-band EWMA + jittered
cooperative sleep, abort-throws, fail-open) + named pace-mode bundles
(env>config>bundle, default off) + shared embed-backfill lock key.
* feat(embed): wire DB-pacing into embed paths + single-flight + bounded keyset re-entry
embedStaleForSource + CLI embedAllStale/embedAll lower worker count to the
resolved cap and observe()/pace() their DB ops; embed job + embed-backfill
handler resolve env>config>bundle; CLI --pace flags; --background carries
overrides into the job payload; single-flight via shared per-source lock;
budget-timer re-arm around paced sleeps; EmbedResult.pacing telemetry.
* feat(sync): shared DB-pacer permit across parallel worker engines
One pacer spans the per-worker PostgresEngines (the multi-pool permit case);
observe() import writes, pace() between files, dispose on all exit paths.
* docs(pace): CLAUDE.md Pace Mode section + regenerated llms bundle
* fix(pace): pre-landing review fixes (Codex P1/P2)
- never unref() the cooperative-sleep timer (could exit mid-sleep)
- pace() excludes the wall-clock budget + re-arm after each sleep
- pacing only lowers concurrency, never raises above an operator cap
- serialized job pace resolves at config tier so GBRAIN_PACE_* still wins
- --pace-max-concurrency consumes its value token
* chore: bump version and changelog (v0.42.48.0)
Native DB-contention pacing for embed/sync backfills.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: bump version to v0.42.49.0
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(skillpack): brain_resident manifest fields + init-brain-pack scaffolder + tools version-skew lint
Add optional brain_resident/schema_pack to the v1 manifest (additive,
forward-compatible). New runInitBrainPack scaffolds a brain-resident pack
(brain_resident:true, exact gbrain_min_version, 5-section machine-parseable
README) beside brain content; applyWritePlan factored out of init-scaffold.
brain-pack-lint validates each skill's declared tools: against the serving op
set (E6 version-skew). Wires gbrain skillpack init-brain-pack.
* feat(skillpack): Topology A brain-pack discovery on sources add + bounded nag
After 'gbrain sources add', if the source ships a brain_resident pack, print an
agent-readable advisory (ask the user before scaffolding). nag-state.ts tracks
declines per (source-repo brain_id, source, pack) with escalate-then-suppress;
declines count only on CLI-interactive displays, never cron/MCP. Fail-open: a
malformed/absent pack never breaks sources add.
* feat(advisor,skillpack): list_brain_skillpack MCP tool + gbrain advisor
Topology B: dedicated source-scoped list_brain_skillpack op + get_skill source_id
disambiguation (brain-resident-locate.ts); git scaffold-spec never a server FS
path; source-aware schema match. LEARN_INSTRUCTION + serve-http banner.
gbrain advisor: read-only ranked actions from brain state (8 resilient collectors,
shared renderer, JSONL history, --json severity exit codes, local-only argv
--apply dispatcher). Exposed over MCP behind mcp.publish_advisor (default off,
read-only on remote; workspace collectors no-op remotely). Generalizes
post-install-advisory to a single current-state recommended set (install→scaffold).
* feat(skills): bundle gbrain-advisor skill + weekly cron recipe + ranking eval
skills/gbrain-advisor teaches a harness to run gbrain advisor on a cadence and
ping the user (read-only; ask before fixing). Registered in manifest.json,
RESOLVER.md, openclaw.plugin.json. E4 ranking-precision eval on seeded-defect
fixtures (100%).
* chore: bump version and changelog (v0.42.47.0)
Brain-resident skillpacks + gbrain advisor (#2180).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: sync CLAUDE.md + KEY_FILES for brain-resident skillpacks + advisor (#2180)
Skill count 29->30, Skills section gains the brain-resident skillpacks + advisor
capability, KEY_FILES gets current-state entries for the new modules. Regenerate
llms bundles.
* test,fix: align stale assertions with generalized advisory + advisor resolver triggers (#2180)
- post-install-advisory.test.ts: install→scaffold wording (the install verb was
removed); restore two-column in book-mirror copy; drop the removed skillpack-list line.
- RESOLVER.md: gbrain-advisor trigger now fuzzy-matches a declared frontmatter
trigger (resolver round-trip D5/C).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: regenerate llms bundle for updated advisor resolver row (#2180)
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(engine): federated sourceIds[] scope on by-slug secondary reads (#2200)
getTags/getLinks/getBacklinks/getTimeline (both engines) + TimelineOpts now
accept a federated `sourceIds[]` read grant, precedence over scalar sourceId,
filtering `source_id = ANY($::text[])` — mirroring getPage from v0.42.37.0.
- getTags: `page_id = (subquery)` -> `IN (subquery)` + DISTINCT so a slug present
in >1 granted source unions tags instead of throwing on a multi-row subquery.
- getLinks/getBacklinks: federated branch scopes ALL THREE page endpoints (from,
to, AND the authoring origin) so a cross-source link can't disclose a foreign
slug. Scalar/unscoped branches unchanged (trusted internal callers keep the
cross-source view).
- getTimeline: Postgres 8-branch cartesian tree collapsed to one fragment-composed
query; PGLite adds the sourceIds branch to its dynamic WHERE.
* fix(ops): route by-slug reads through the federated source scope (#2200)
get_page resolves tags against the concrete page's source; get_tags/get_links/
get_backlinks/get_timeline route through sourceScopeOpts(ctx) (replacing the
copy-pasted scalar `ctx.sourceId ? {sourceId} : {}`). New linkReadScopeOpts
promotes an UNTRUSTED remote scalar scope to sourceIds[] so legacy/pre-federated
tokens also get all-endpoint link scoping; trusted local CLI keeps cross-source.
* test: federated read scope on by-slug reads + engine parity (#2200)
Per-op federated reads, isolation (out-of-grant -> empty), cross-source decoy
guard, far-endpoint + origin leak guards (F1), same-slug union (D3A), empty-array
contract, scalar-remote promotion (D1), getTimeline date-window after the Postgres
fragment refactor (D5A), and engine-parity arms for all four methods.
* chore: bump version and changelog (v0.42.46.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(key-files): document federated by-slug read scope + linkReadScopeOpts (#2200)
The #2200 fix routed get_page tags + get_tags/get_links/get_backlinks/
get_timeline through the federated source scope and added sourceIds[] to the
engine read methods + TimelineOpts. Bring KEY_FILES.md to current state: the
operations.ts entry's sourceScopeOpts read-op list now includes the by-slug
reads and documents the linkReadScopeOpts helper (three-endpoint link scoping +
untrusted-remote scalar promotion); the engine.ts entry notes the by-slug read
methods + TimelineOpts carry the same sourceIds[] federated axis.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(core): shared computeSyncDelta + spend-posture module (#2139)
sync-delta.ts: ONE implementation of "what changed since last_commit",
consumed by both the sync executor and the inline cost estimator so the
gate's dollar figure can't drift from what the sync imports.
spend-posture.ts: spend.posture config + parseUsdLimit/formatUsdLimit
off-switch parsing (off/unlimited/none → Infinity; undefined at the budget
boundary so ledger rows never serialize null).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(sync): delta-aware cost estimator + non-TTY auto-defer + per-source failure acks (#2139)
The inline-embed cost gate was a ~400x phantom: it priced the entire tree
whenever the working tree was dirty (always, on an active brain), then blocked
the daily cron with exit 2. Now:
- performSyncInner + the estimator both route through computeSyncDelta, so the
estimate mirrors execution (fetch-first delta; dirty-but-caught-up tree → $0).
- shouldBlockSync is posture-aware; non-TTY above floor AUTO-DEFERS embeds to
capped backfill jobs (exit 0) instead of wedging — single shared
runInlineCostGate on both --all and single-source paths.
- --full prices delta + stale backlog (full sync sweeps it inline).
- off/unlimited on the cost knobs; tokenmax bypasses the backfill cap (still
ledgered) but never the cooldown.
- --skip-failed/--retry-failed scoped per source; the D15 parallel refusal is
lifted (the #1939 ledger is per-source + lock-serialized).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(config): register spend-control keys + validate spend.posture (#2139)
Adds spend.posture + the five previously --force-only spend knobs to
KNOWN_CONFIG_KEYS so `config set` accepts them directly (removes the
archaeology the issue complained about), and rejects invalid spend.posture
values at set time with a paste-ready hint.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(reindex,enrich,onboard): spend.posture across the remaining cost gates (#2139)
reindex-code: tokenmax makes the cost gate informational; --max-cost accepts
off/unlimited. enrich + onboard --auto: tokenmax lifts the refuse-without-cap
guardrail and runs UNCAPPED (spend still ledgered by BudgetTracker). Explicit
--max-usd always wins over posture.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: cost-gate, delta estimator, spend-posture, off-switch coverage (#2139)
New sync-delta + sync-cost-estimate unit suites; rewritten cost-gate serial
tests (auto-defer instead of exit 2, posture, off-switch, format split,
single-source); parseUsdLimit/posture-aware shouldBlockSync; backfill cap-off
+ tokenmax-bypass + cooldown-still-refuses; config known-key acceptance.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(spend-controls): single spend-control surface + ref-map + follow-up TODOs (#2139)
New docs/operations/spend-controls.md (every gate, key, default, off switch,
posture interaction); CLAUDE.md reference-map row; two P3 follow-up TODOs
(measured chunk-count gating, per-source defer granularity). llms bundles
regenerated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(spend): SSRF-harden estimator fetch + complete off/uncapped across reindex/enrich/onboard (#2139)
Ship-stage codex pre-landing review caught four P1s in the secondary cost gates:
- The delta estimator's fetch-first ran `git fetch` through the plain git()
helper, bypassing the GIT_SSRF_FLAGS + GIT_TERMINAL_PROMPT=0 hardening that
real sync uses. Added `fetchRemote()` to git-remote.ts (same flags as
pullRepo) and route the estimator through it — a cost preview / dry-run can
no longer hit a remote through a less-protected path.
- `reindex --max-cost off`, `enrich --max-usd off`, `onboard --auto --max-usd
off` were parsed but didn't actually proceed/uncap. Now: explicit off (and
spend.posture=tokenmax) proceed past the confirmation/missing-cap refusal AND
run uncapped. enrich threads an Infinity sentinel mapped to "no BudgetTracker
ceiling" (never raw Infinity → no null in audit rows); reindex/onboard use
their native undefined=uncapped path. Spend still ledgered.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: bump version and changelog (v0.42.45.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(KEY_FILES): update sync/embedding/git-remote/reindex entries to post-#2139 truth
document-release pass: the cost-gate entries described the pre-#2139 behavior
(full-tree-ceiling estimator, --skip-failed-rejects-under-parallel, exit-2
confirmation gate). Updated to current truth — delta-aware estimator via the
shared computeSyncDelta, per-source failure acks under parallel, non-TTY
auto-defer (no exit 2), posture-aware shouldBlockSync. Added entries for the
two new core modules (sync-delta.ts, spend-posture.ts) + fetchRemote on
git-remote.ts + reindex --max-cost off.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(tutorial): point AlphaClaw deploy link at the official site (#2165)
Step 4 of the personal-brain tutorial linked to the wrong top-level
domain for AlphaClaw. Corrected so the deploy step works as written.
* chore: bump version and changelog (v0.42.44.0)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(cli): exit deliberately after bounded teardown instead of riding the 10s backstop (#2084)
Root cause: bounded teardown (endPoolBounded, #2015) RESOLVES, but lingering
sockets — embedding-provider fetch keep-alive, PgBouncer txn-mode sockets the
bound raced past — keep Bun's event loop alive, so every `gbrain query` paid
a flat 10s tax exiting via the hard-deadline force-exit banner.
Three changes, one contract:
- flushStdoutThenExit (cli-force-exit.ts): when main() resolves and the
command is not a daemon, exit deliberately — after stdout AND stderr drain
(writableLength===0, 'drain'-event + poll loop, 2s unref'd guard for a
blocked pipe). Incident #1959 (force-exit truncating piped stdout) is the
regression class; pinned by a 256KB real-pipe subprocess test.
- drainThenDisconnect (cli.ts): ONE owner-disconnect helper at all 8 sites
(op-dispatch, CLI_ONLY fall-through, search dashboard, doctor remediation
x3, ze-switch, dream, read-only timeout path). Drains the background-work
registry, then disconnect (best-effort), bounded by the 10s unref'd
hard-deadline — which is now armed around the TEARDOWN window only, not
before the op handler (the old placement would have force-killed any op
slower than 10s). Closes the filed TODOS P3 drain-hoist: six sites
previously skipped the drain entirely and had no hang timer at all.
- Inner process.exit sweep: mid-handler exits in engine-owning/output-bearing
paths (status, friction, claw-test, smoke-test, eval cross-modal /
takes-quality replay / conversation-parser / whoknows-thin, status-thin)
become process.exitCode + return so they flow through the drains and the
flush-exit. Pre-engine usage/parse/refusal exits stay as-is.
BrainRegistry.disconnectAll deliberately unchanged: zero production callers
in src/, per-engine disconnects already bounded, and the kernel reclaims
sockets on exit (src/core/timeout.ts doctrine).
DAEMON_COMMANDS gains 'watch' ahead of the #2095 push transport.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(e2e): PgBouncer transaction-mode pooler in CI + teardown e2e (#2084)
Three consecutive waves (#1972 → #2015 → #2084) fixed pooler-teardown bugs
verified only against one production deployment — CI had no transaction-mode
pooler and could never see the class. Now it can:
- docker-compose.ci.yml: `pgbouncer` service (transaction pooling) fronting
postgres-1, mirroring the production split-pool topology (direct :5432 +
pooled :6543). AUTH_TYPE=plain (pg16 SCRAM verifiers need the plaintext
password in the userlist) + IGNORE_STARTUP_PARAMETERS for the
statement_timeout/idle_in_transaction_session_timeout startup params
gbrain's client sets (the Supabase pooler whitelists the same).
- test/e2e/pgbouncer-teardown.test.ts: schema + fixture via the DIRECT url
into a dedicated `gbrain_pgbouncer` database (never races shard TRUNCATEs),
then spawns the real CLI against the POOLED url and asserts: exit 0,
stdout intact (the #1959 truncation class), and NO
"did not return within 10000ms — force-exiting" banner (pre-#2084 it
printed on 100% of query-shaped ops on this topology). Class bound, not
exact timing. Skips gracefully without GBRAIN_PGBOUNCER_URL.
- scripts/ci-local.sh: threads GBRAIN_PGBOUNCER_URL +
GBRAIN_PGBOUNCER_DIRECT_URL into all three e2e phases.
Verified live: both tests green against pgbouncer 1.25.2 in transaction mode.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(schema): context_volunteer_events table (v116) — push-context feedback log (#2095)
One row per page the brain volunteers (op / reflex / watch channels).
"Used" is DERIVED, never written: pages.last_retrieved_at > volunteered_at
(the existing bumpLastRetrievedAt write-back is the open/cite signal), so
there is no second tracking path. session_id/turn are nullable
caller-supplied attribution; rationale is a deterministic template string,
never raw conversation text.
- Migration v116 (idempotent) + mirrors in src/schema.sql +
src/core/pglite-schema.ts + regenerated schema-embedded.ts (regen also
folds in pre-existing comment-only drift from the v114 links edits).
- src/core/context/volunteer-events.ts: insertVolunteerEvents (ONE
multi-row parameterized INSERT — never per-row awaited round-trips) +
purgeStaleVolunteerEvents (90-day GC, returns 0 on pre-v116 brains).
- Dream cycle purge phase prunes stale events alongside op_checkpoints /
brainstorm checkpoints / batch-retry audit files.
- RLS on Postgres comes from the v35 auto_rls_on_create_table event
trigger (the same mechanism that covered v110 page_aliases and v115
op_checkpoint_paths); the volunteer Postgres e2e pins it.
- No ::jsonb anywhere; no bootstrap probe needed (nothing references the
table pre-creation; writers guard with try/catch).
Tests: v116 shape + columns + indexes + live insert/purge round-trip on
PGLite (test/migrate.test.ts, 161 pass); schema-bootstrap-coverage green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(context): multi-turn window extraction + confidence-scored volunteer core (#2095)
- entity-salience.ts: extractCandidatesFromWindow(turns) — runs the existing
per-turn extractor across the last N turns (oldest→newest), merges by the
normalizeAlias form with occurrence/newest-turn/user-mention metadata, and
orders by salience (recency > frequency > user-role) so the MAX_CANDIDATES
cap drops stale assistant chatter, not the entity the user just named.
Closes the filed assistant-introduced-entities recall TODO; true pronoun
coreference (never-named antecedents) stays out of scope.
- retrieval-reflex.ts: ReflexPointer gains source_id + arm + confidence +
matchedNorm. ARM_CONFIDENCE (alias 0.9 / title 0.8 / slug-suffix 0.6)
lives next to the arm definitions so identity and score can't drift.
Arm-2 provenance is classified in JS (codex D8 — the combined OR can't
report which predicate matched). Federated sourceIds[] scope (alias arm
loops per source; arm 2 uses source_id = ANY — no engine-interface
change). Suppression gains 'slug-only' mode (codex D7, REQUIRED for
windowing): the legacy title-whole-word rule would suppress every entity
merely MENTIONED in a prior window turn, breaking the feature by
construction — slugs only enter context when a pointer/page was actually
surfaced. Default stays 'slug-and-title' for the window=1 legacy path.
- volunteer.ts (new): parseWindow (lenient user:/assistant: prefixes, CRLF,
unprefixed → one user turn), volunteerContext (zero-LLM: extract →
resolve → +0.05 multi-turn/newest-turn boost → min_confidence 0.7 gate →
cap 3/5; deterministic rationale strings, never raw conversation text),
and volunteerUsageStats (per-arm/channel precision from the
last_retrieved_at join, labeled approximate — 5-min throttle false
negatives, unrelated-read false positives; codex D9).
Tests: 35 green across volunteer-context (window parsing, pronoun follow-up
via assistant-introduced entity, confidence gating, slug-only suppression,
takes-fence privacy, multi-source scope, caps, stats join math) +
retrieval-reflex back-compat + resolve-ipc.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(ops): volunteer_context op — CLI (stdin) + MCP, drained event sink (#2095)
New read-scope op on the contract surface (CLI `gbrain volunteer-context`
with stdin → window, MCP tool for free): takes a rolling conversation
window, returns confidence-gated page pointers with rationales + synopses.
`window` is optional-unless-stats (validated in the handler, codex D9);
`stats: true` returns the volunteered-vs-used precision summary, labeled
APPROXIMATE (the 5-min last-retrieved throttle and unrelated reads both
bias the join). Source scope threads through sourceScopeOpts — federated
grants narrow the volunteer to the granted sources.
Event logging is fire-and-forget through a new `volunteer-events`
background-work sink (volunteer-events.ts, mirrors last-retrieved: tracked
dangling promise set + bounded drain + snapshot-drop on timeout so a
long-lived process never accumulates ghosts). ONE batched INSERT per call,
drained on every exit path by the commit-1 drain hoist; failure never
fails the op (pinned by an injected failing-engine test).
cli formatResult renders both shapes (pointer lines with confidence/arm/
rationale; the stats summary with per-arm precision).
Tests: op contract surface, window-required validation, sink round-trip
with session_id/turn attribution, failing-engine fail-open, federated
grant scoping, stats mode (26 green on PGLite) + a real-Postgres e2e
proving the op + sink + stats join AND that context_volunteer_events has
RLS enabled (keeps the auto-RLS event-trigger mechanism honest for v116).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(context): reflex consumes the rolling window + ambient-channel logging (#2095)
The default-on retrieval reflex now extracts entities from the last N turns
(retrieval_reflex_window_turns, default 4; env
GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS; window=1 reproduces the legacy
current-turn-only behavior exactly). assemble() passes the recent
user/assistant turns (hard cap 12); the reflex slices to the configured
window. Assistant-introduced entities and "what did she invest in?"
follow-ups whose antecedent was NAMED in the window now surface pointers —
the issue's "zero agent-initiated queries" success criterion on the
ambient path.
Under windowing, suppression switches to slug-only (codex D7): the legacy
title-whole-word rule would suppress every entity merely MENTIONED in a
prior window turn, breaking the feature by construction. Slugs only enter
prior context when a pointer/page was actually surfaced, so
already-surfaced pages still suppress. The suppression mode flows through
all three resolver rungs (host opts, serve IPC request, direct Postgres).
Ambient-channel feedback (codex D11): the server-side resolver paths
(serve IPC + direct Postgres) log volunteered pointers with
channel: 'reflex' through the drained volunteer-events sink, so
`gbrain volunteer-context --stats` measures the default-on path where most
volunteering happens. Host-injected resolvers (no gbrain engine) can't
log — documented gap. Precision gates, 1.5s ceiling, fail-open, and the
pointer cap are unchanged.
Tests: prev-assistant-turn entity fires; window=1 legacy parity; slug-only
vs already-surfaced suppression; throwing resolver stays fail-open
(16 green).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(cli): gbrain watch — push transport over stdin (#2095)
The issue's headline: the brain volunteers pages as the conversation flows,
instead of waiting to be asked. `some-transcript-feed | gbrain watch` reads
turns line-by-line ('user:'/'assistant:' prefixes set the role; unprefixed
lines are user turns), keeps a rolling window (--window-turns, default 4),
and streams confidence-gated pointers with rationales to stdout (--json for
JSONL). Session dedupe rides the core's slug-only suppression — a slug is
volunteered at most once per session. Events log on channel 'watch' with
session_id + turn through the drained sink.
Lifecycle: watch BLOCKS in the stdin iteration (like `jobs work`) — an
interactive TTY stays alive until Ctrl-C/Ctrl-D, piped input ends at EOF —
so it is deliberately NOT in DAEMON_COMMANDS (reverts the commit-1
placeholder): when main() resolves the work is over, the CLI_ONLY finally
drains volunteer events via drainThenDisconnect, and the entrypoint
flush-exit ends the process. Keeping it in the daemon set would have made
the piped EOF path hang on lingering sockets — the exact #2084 class.
SIGINT closes the stream and flows through the same drain path instead of
killing mid-write. Per-turn resolution failures are fail-open (the stream
never dies on a transient DB error).
Full wiring (eng-review D12): CLI_ONLY + CLI_ONLY_SELF_HELP (WATCH_HELP) +
THIN_CLIENT_REFUSED_COMMANDS (thin clients use the volunteer_context MCP
op) + main --help entry.
Tests: 18 green — help, per-turn volunteering + clean EOF return, rolling
window via assistant-introduced entity, session dedupe, --json shape with
turn attribution, channel-watch event rows, --min-confidence gate, CRLF/
blank tolerance, daemon-gate semantics. Live smoke: piped `gbrain watch`
on a fresh PGLite brain exits 0 at EOF with no force-exit banner.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: KEY_FILES + push-context guide + TODOS for the #2084/#2095 wave
- docs/architecture/KEY_FILES.md (current-state): context entries gain the
window extractor, arm provenance/confidence, suppression modes, volunteer
+ volunteer-events modules; background-work entry now lists FIVE sinks and
the drainThenDisconnect owner-disconnect contract; new entries for
src/core/cli-force-exit.ts (the exit contract) and src/commands/watch.ts.
- docs/guides/push-context.md (new): the three channels (reflex/op/watch),
the confidence model, CLI usage, config keys, and the approximate-stats
caveat. Linked from CLAUDE.md's reference map.
- CLAUDE.md: ops line mentions volunteer_context + the guide link;
bun run build:llms regenerated in the same commit (freshness test green).
- TODOS.md: #2095 deferrals filed (SSE push channel, policy skill + doctor
check, structured messages[] param); the #1981 entity-detection TODO
narrowed (window extraction covered assistant-introduced entities +
named-antecedent follow-ups); the drain-hoist P3 marked DONE by the
#2084 wave.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(e2e): truncate context_volunteer_events in setupDB (#2095)
The new feedback-log table wasn't in ALL_TABLES, so volunteered-event rows
persisted across e2e runs on a reused database and poisoned count/stats
assertions in volunteer-context-postgres on the second run. No FK to pages
(slug join), so position before pages is for hygiene only.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(cli): own the exit verdict — never trust ambient process.exitCode (#2084)
Caught by the full unit suite: `gbrain apply-migrations` on PGLite started
exiting 99. Root cause: PGLite's Emscripten runtime writes the WASM
backend's proc_exit status into process.exitCode (initdb at create-time,
the postmaster at close-time — `exitCode=status` in pglite's dist), and
the writes land ASYNCHRONOUSLY, outside any snapshot/restore window around
create/close (a guarded attempt verified this). The pre-#2084 success path
never read process.exitCode, so the pollution was invisible; the new
deliberate flush-exit propagated it faithfully.
Fix: gbrain records its own verdict. setCliExitCode(n)/getCliExitCode() in
cli-force-exit.ts — every gbrain-owned exit-code assignment routes through
the setter (still mirrored to process.exitCode for outside readers), and
both exit paths (entrypoint flushStdoutThenExit + the drainThenDisconnect
hard-deadline backstop) read the getter. Swept all assignment sites:
cli.ts (op error, friction, claw-test, smoke-test, eval runners, status,
import errors) + reindex/transcripts/brainstorm/frontmatter/autopilot.
Also updates the v0.42.20 structural pins to the drainThenDisconnect shape
(ordering invariant asserted INSIDE the helper + >=8 helper call sites,
superseding the two-inline-pairs assertion).
Verified: apply-migrations spawn test green; `init --migrate-only` exits 0;
an errored op still exits 1.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: re-pin the teardown-arming invariant at its post-#2084 home
Master's v0.42.41.0 triage wave and the #2084 wave fixed the same
pre-armed-timer bug independently; the merge keeps #2084's shape (arming
inside the shared drainThenDisconnect helper, covering all 8 exit paths).
The structural pin now asserts the same invariant — no pre-try arming;
gated, unref'd, before-drain, cleared — at the helper.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: coverage for ambient reflex-channel logging + watch window/cap flags
Ship coverage audit (85%, gate PASS) named five gaps; the two substantive
cheap ones close here: the codex-D11 logChannel='reflex' path now has a
behavioral pin (events land on channel 'reflex' through the drained sink;
no logChannel → no events), and gbrain watch's --window-turns / --max-pages
flags are exercised (turn-1 attribution under window=1; cap to one page).
Remaining flagged-not-blocking: the wallclock-timeout branch (untestable
without >10s real-clock flake — same rationale as the arming pin),
formatResult's volunteer case (module-private), and the cycle purge wiring.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: close the remaining plan-audit gaps — formatResult rendering + watch SIGINT
formatResult exported for tests (same import-safety contract as cliAliases);
test/cli-format-volunteer.test.ts pins the pointer lines, empty-gate message,
and approximate stats summary. test/watch-command.test.ts gains a real
subprocess SIGINT test: piped stdin that never reaches EOF, SIGINT mid-stream,
assert exit 0 with no force-exit banner — the drain-then-exit lifecycle under
the actual signal, not just the shared exit path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: doctor's FAIL verdict was zeroed by the owned exit — sweep stragglers + class pin
The merged-state suite caught it: doctor --fast --json reported FAIL but
exited 0. Master's v0.42.41.0 brought raw `process.exitCode =` writes
(doctor.ts hasFail ternary, extract.ts) that the #2084 verdict-owning exit
silently zeroes — getCliExitCode() deliberately never reads ambient
process.exitCode (the PGLite-Emscripten pollution defense), so any setter
that bypasses setCliExitCode reports success on failure.
Swept both sites and added the structural class pin: a test greps src/ for
raw `process.exitCode =` outside cli-force-exit.ts, so the next merge that
introduces one fails loudly instead of lying about exit codes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: bump version and changelog (v0.42.43.0)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: quarantine the watch SIGINT subprocess test to the serial lane
The parallel unit shards flake on concurrent CLI subprocess spawns (failed
at 7ms in-suite, green solo) — same isolation rationale as
apply-migrations-pglite-spawn.serial.test.ts and #2141's R3 quarantine.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: update project documentation for v0.42.43.0
Post-ship doc verification against the release diff (#2095 push-based
context + #2084 superset hardening), with a cross-model doc review:
- push-context.md: version tag corrected to v0.42.43.0; per-call knobs
now cover prior_context/days and watch's flag surface accurately;
feedback-log writes described as best-effort; synopsis fence-strip
described as unconditional.
- CLAUDE.md: stale operation count (~47 -> ~90); volunteer_context
release reference corrected to v0.42.43.0.
- KEY_FILES.md: ci-local entry rewritten to current topology (4-shard
parallel default, four Postgres services, transaction-mode PgBouncer
+ GBRAIN_PGBOUNCER_URL/_DIRECT_URL exports); stale E2E file counts
dropped from the selector entry.
- TESTING.md: inventory entries for the new #2084 structural pins
(cli-exit-verdict-pin, cli-pipe-truncation), the push-context test
suite (volunteer-context, watch-command, watch-sigint.serial,
cli-format-volunteer), migrate v117 coverage, and the two new E2E
files (pgbouncer-teardown env gating, volunteer-context-postgres RLS
pin); check:all row corrected (not a superset of verify).
- AGENTS.md + RELEASING.md: ci:local descriptions updated to the
sharded + pooler topology.
- CHANGELOG (wording only, entry preserved): "retrieved" instead of
"opened" for the used-signal, pooler scoped to the local CI gate,
feedback log labeled best-effort.
- llms-config.ts: index the new push-context guide; bundles
regenerated (build:llms) and freshness test green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(test): correct the v116 reference — the table shipped as migration v117
* fix: pre-landing review hardening — federated alias parallelism, trust-boundary clamps, shared protocol helpers (#2095)
Five specialist reviewers (testing/maintainability/security/performance/
data-migration) on the reconciled diff; every finding applied:
Performance: the alias arm now resolves all granted sources CONCURRENTLY
(a federated caller paid M sequential RTTs per turn — ~355ms at 5 sources
cross-region, inside the reflex's 1.5s budget); watch's session dedupe is
O(1) Set membership instead of a monotonically growing priorContext string
(O(T²) over a long-lived session); getWindowTurns iterates from the tail
(per-turn cost no longer grows with session length); the resolver's
provenance maps fold into the existing candidate pass.
Security: volunteer_context clamps caller-supplied attribution at the trust
boundary — session_id capped at 256 chars (a read-scoped token could bank
~1MiB TEXT per request, retained 90 days), turn logged only when a safe
integer (a non-integer threw inside the batched INSERT and silently dropped
the whole batch). The privacy comments now state precisely what rationale
may contain (the matched entity's surface form — which by construction
resolved to an existing alias/title/slug — never free conversation text).
Maintainability: TURN_PREFIX_RE + formatVolunteeredPage exported from
volunteer.ts and shared by watch/cli (the two surfaces can no longer
drift); volunteerEventRowsFrom is the single VolunteerEventRow assembly
site for all three channels; watch's window default now honors the same
retrieval_reflex_window_turns config knob the reflex reads; the stale
pre-v116 comments swept to pre-v117.
Testing: the two flake-class CRITICALs fixed (pipe test asserts the
backstop banner instead of a cold-CI-hostile 9s wall bound; the SIGINT test
waits on watch's new machine-readable ready line instead of a fixed 15s
sleep — 2.5s and deterministic now); new coverage for the sink's timeout
branch + ghost-reference drop, watch per-turn fail-open, untrusted knob
clamps (min_confidence/max_pages/days), window-cap ordering (newest user
mention survives), serve-IPC suppression passthrough + channel=reflex
logging, windowTurnCount edge semantics, and structural pins for the sink
registration + cycle purge wiring. The exit-verdict pin's grep is now
operator/whitespace-tolerant.
Deferred with TODOs: resolver index shapes for the per-turn query;
batched first-prune after a long dream-cycle gap.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(context): red-team hardening — pre-cap dedupe, delivery-side reflex logging, window clamp
Four red-team findings on the #2095 push-context surface:
- RT1 starvation: watch's session-dedupe Set filtered AFTER volunteerContext's
cap, so a recurring already-pushed entity burned cap slots every turn and
starved fresh pages behind it. VolunteerOpts.excludeSlugs now skips inside
the pointer loop BEFORE the confidence gate and the cap.
- RT3 honest stats: reflex-channel event logging moved from inside the
resolver to the DELIVERY point — serve's resolve-IPC onDelivered hook fires
only after the response write succeeds, and buildReflexAddition logs only
after the per-turn timeout admits the block. A block the client's 250ms
budget abandoned was never injected and no longer counts as volunteered.
(logChannel resolver opt removed; logDeliveredReflexPointers is the seam.)
- RT5 unbounded window: --window-turns is clamped to [1, 64] so a config typo
can't reintroduce the re-scan-everything-per-turn cost class.
- RT2/RT4 documented + filed: PGLite watch connection monopoly (WATCH_HELP,
push-context guide, TODO to route watch via serve IPC); host-resolver
suppression contract at ResolveEntitiesFn (TODO for a capability gate).
Tests: starvation guard (watch + volunteerContext unit), window clamp floor +
ceiling, delivery-side logging (helper writes channel=reflex through the
drained sink; bare resolver writes nothing; empty list no-op), IPC wiring test
rewired to onDelivered. KEY_FILES.md + push-context.md updated; build:llms run.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(context): env-plane window knob works config-less; harden two gateway-state-leak victims
Three CI-only check failures, two root causes:
1. windowTurnCount ignored GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS when
loadConfig() returned null (no config file AND no DATABASE_URL — a clean
CI shard with no brain). loadConfig drops its env→config mapping in that
case, so the documented escape hatch silently died and the window fell
back to 4 → windowed extraction widened when the test set window=1 →
prior-turn entity leaked. Fixed: read the env var directly in
windowTurnCount, mirroring reflexEnabled's direct process.env read. This
is a real product bug, not just a test artifact — any config-less host
using the env hatch was affected. Regression test pins it.
2. sync-cost-preview + doctor-federation-health failed only IN-SHARD: a
sibling test configured a non-legacy (ZeroEntropy 1280-d / $0.05) gateway
and never reset it. The legacy-embedding preload only restores the
OpenAI/1536 default when the gateway slot is EMPTY, so a non-empty foreign
config survives into the next file — and a file's beforeAll runs BEFORE
the preload's restoring beforeEach, so federation-health built a
vector(1280) column and its 1536-d fixture hit CheckExpectedDim. My new
test files reshuffled the deterministic file→shard assignment, exposing
this latent ordering bug. Hardened both victims to establish the gateway
state they assert (sync-cost-preview resets to the unconfigured fallback;
federation-health pins legacy 1536 before initSchema) so they're
order-independent. Verified against a simulated leaker run before them.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(context): use withEnv() in the window env-hatch test (test-isolation guard)
The regression test added in 82cc7fff mutated process.env directly, which
check:test-isolation (R1) forbids — use the withEnv() helper that restores on
exit, same as the rest of this file. Behavior identical; guard green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(core): finishCliTeardown + flushThenExit — bounded teardown, owned exit verdict (#2084)
cli-force-exit.ts becomes the single owner of one-shot CLI exit + teardown:
- finishCliTeardown: bounded sink drain -> bounded disconnect under a backstop
whose deadline is COMPUTED from the bounds it guards (floor 10s;
GBRAIN_TEARDOWN_DEADLINE_MS env override). Arms at teardown start, never
before the op handler.
- flushThenExit: stdio write-fence (unref'd guard, EPIPE-safe) + REF'D
aliveness grace for non-TTY stdio — Bun only delivers queued pipe writes
while the process is alive (no flush API reaches the native queue).
- setCliExitVerdict/currentExitCode: the exit verdict lives in a gbrain-owned
channel, never read back from process.exitCode (PGLite's Emscripten runtime
scribbles its own status there mid-run).
- background-work.ts exports backgroundWorkSinkCount() for the deadline formula.
Unit tests + a spawned-Bun harness proving byte-complete piped output.
* fix(cli): route all nine disconnect sites through finishCliTeardown; one exit seam (#2084)
Deletes the pre-handler 10s force-exit timer (it measured handler + teardown
combined: PgBouncer txn-mode deployments paid a flat 10s banner tax on every
query, and any >10s op was killed mid-run with exit 0 and truncated output).
Sweeps op-dispatch, CLI_ONLY fall-through, search dashboard, read-only timeout
path, dream, doctor x3 (fixing a pre-existing pool leak when DB checks throw),
and ze-switch. The ONE process exit lives in main().then/catch via
flushThenExit(currentExitCode()), gated by shouldForceExitAfterMain().
Exit-code writers (op-dispatch catch, reindex, transcripts, brainstorm,
autopilot, frontmatter) now set the verdict through setCliExitVerdict.
* fix(pglite): contain Emscripten's process.exitCode writes at PGlite.create (#2084)
PGLite's WASM runtime writes its own status into process.exitCode (99 at
create; in-memory brains run initdb whose status lands on a later tick; the
exit status at close) — on PGLite every error exit was silently clobbered.
preservingProcessExitCode wraps create() to keep the global tidy; db.close()
stays unwrapped (its 0-write is baseline behavior test runners depend on).
The CLI verdict itself is immune: it lives in the owned channel.
* test: e2e + structural pins for the #2084 teardown contract
E2E: failed op exits 1; every swept command spawned (brain-copy isolation for
mutators, no-network); slow-handler regression via the deadline env knob;
piped --json parses complete; teardown banner absent on every happy path;
daemon survival untouched. Structural: no bare awaited engine disconnects in
cli.ts; DISCONNECT_HARD_DEADLINE_MS gone; >=9 helper call sites; verdict
channel + create-wrap pins.
* test: fix R1 env-isolation violations in retrieval-reflex tests
Pre-existing on master: both files mutated GBRAIN_RETRIEVAL_REFLEX directly,
failing scripts/check-test-isolation.sh (bun run verify). Converted to the
canonical withEnv() pattern; the reflex describe's beforeEach also never
restored the flag, leaking it across the shard.
* docs: KEY_FILES entries for the teardown contract; close + file TODOS (#2084)
KEY_FILES.md: current-state entry for cli-force-exit.ts (helper + central exit
seam pair, verdict channel, cli.ts-scoped claim); background-work.ts and
pglite-engine.ts entries updated. TODOS.md: the drain-before-owner-disconnect
P3 (filed from #1972) is done by this wave; files the trigger-gated
GBRAIN_COMMAND_DEADLINE_MS follow-up (eng-review D2/D14).
* fix: pre-landing review fixes (#2084)
Review army (testing/maintainability/security/performance, 0 critical):
- drain defense-in-depth: a throwing drain warns and still disconnects
(cannot escape a caller's finally or skip the engine teardown)
- behavioral tests for preservingProcessExitCode (connect pins 0; create-throw
restores the pre-call verdict)
- D9 widening test (live-registry sink count feeds the deadline formula),
env 0/negative boundary cases, verdict mirror-write assertion
- stale comments: header diagram backstop line, structural-test 'both
lifecycle calls' contradiction, KEY_FILES 10s-force-exit clauses, e2e D11
falsification story corrected
- named the formula's pool-end literals
* fix: adversarial-review hardening — daemon-safe command resolution, flush knob, ref'd backstop (#2084)
Cross-model adversarial review (Claude subagent + Codex, both P1'd it):
- shouldForceExitAfterMain now resolves the command through parseGlobalFlags —
the old first-non-dash heuristic read `gbrain --timeout 30s serve` as
command "30s" and the new exit seam would have killed the daemon ~250ms
after boot with exit 0 (unit-pinned)
- GBRAIN_FLUSH_GRACE_MS env override for the non-TTY aliveness grace (batch
consumers piping large payloads to slow readers can raise it; agent loops
can lower it)
- backstop timer is now REF'D: a hung teardown on an otherwise-empty event
loop previously exited naturally — skipping the flush and surfacing
PGLite's scribbled process.exitCode
- flushThenExit: real process.exit latched once per process
- doctor-site comment corrected; in-command process.exit teardown-bypass
class (pre-existing) filed as a P2 TODO
* chore: bump version and changelog (v0.42.42.0)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: move #2084 exitCode-containment lifecycle tests to the serial quarantine (R3)
* docs: update project documentation for v0.42.42.0
- docs/TESTING.md: replace the stale 4-file serial-quarantine enumeration
with a current-state description (the quarantine is glob-discovered, now
several dozen files incl. the #2084 exitCode-containment suite); add unit
inventory entries for test/cli-finish-teardown.test.ts and
test/flush-then-exit-harness.test.ts.
- docs/architecture/KEY_FILES.md: rephrase the pglite-engine exitCode
containment note to current-state wording (clears the
check-key-files-current-state prose-history warning).
llms bundles regenerated (byte-identical: both docs are link-only).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: apply cross-model doc-review findings for v0.42.42.0
Codex review of docs-vs-shipped-code found 9 gaps; all verified against
the code before fixing:
- CHANGELOG.md (0.42.42.0 entry, precision narrowing only — no entries
touched): "every CLI exit path" -> "every cli.ts disconnect site";
"on every path" -> "on every routed exit path"; dream/doctor/ze-switch
claim scoped to dispatcher teardown (command-internal process.exit
sites are tracked in TODOS as the open P2).
- docs/architecture/KEY_FILES.md: the teardown backstop is REF'D, not
unref'd (matches the F3 adversarial-review decision in the code).
- src/core/cli-force-exit.ts: header diagram comment had the same stale
unref'd claim + `process.exitCode ?? 0`; now matches the implementation
(ref'd timer, `currentExitCode()`). Comment-only change.
- docs/TESTING.md: verify is the 30-check parallel battery via
run-verify-parallel.sh (was described as 4 checks); CI is 10 weighted
LPT shards + dedicated verify/serial/slow jobs (was "4-way FNV on
shard 1"); test:serial runs one bun process per file (not
--max-concurrency=1); dead "cap: 10" line rewritten as debt guidance;
inventory entries added for test/cli-should-force-exit.test.ts and
test/e2e/pglite-cli-exit.serial.test.ts.
bun run verify green (30/30); #2084 test files green; llms bundles
regenerated (byte-identical — reference docs are link-only).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: route v0.42.41.0's raw exitCode writers through the verdict channel; reconcile merged structural pins (#2084)
CI fallout from merging the v0.42.41.0 triage wave into the #2084 exit-seam
design — both waves fixed the same timer-placement bug independently:
- doctor.ts + extract.ts set failure exit codes via raw `process.exitCode =`
writes (v0.42.41.0's process.exit -> exitCode conversion); the #2084 exit
seam reads only the gbrain-owned verdict channel, so doctor FAILs exited 0
(Tier 1 RLS e2e + half-migrated-Minions tests). Converted to
setCliExitVerdict, same as the wallclock-124 site in the merge commit.
- cli-force-exit-teardown-arming.test.ts pinned v0.42.41.0's inline
finally-armed timer, which the merge replaced with finishCliTeardown;
rewritten to pin the merged invariant (no pre-try arming in cli.ts; the
backstop arms inside the helper before the drain).
- eval-capture drain timing bound 1s -> 2s: flaked at 1023ms under CI shard
load after the new test files shifted LPT shard packing (13x budget slack
still proves bounded-not-hung).
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(oauth): default omitted authorize scope to client's full grant
When a client omits `scope` on /authorize, the authorize() grant computed
`(params.scopes || []).filter(...)` → the empty set. That empty grant was
written to oauth_codes and propagated into the access AND refresh tokens, so
every request failed `insufficient_scope` even though the client was
registered with e.g. `read write`. Because refresh inherits the stored grant,
it never self-healed — reconnecting just minted another empty-scoped token.
Some MCP connectors (observed with Claude Desktop) omit `scope` on /authorize,
so they hit this on every connection.
Fix: when no scope is requested, default to the client's full registered scope
(RFC 6749 §3.3 permits a server default). This mirrors exchangeClientCredentials,
which already does `requestedScope ? ... : allowedScopes`. The result is still
clamped to the allowed set, so an explicit over-broad request cannot escalate.
Adds test/oauth-authorize-scope-default.test.ts covering: omitted/empty →
inherits full grant; explicit subset honored; clamp preserved (over-broad and
disallowed-only requests cannot escalate or trigger inheritance).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(sync): skip Python venv/ in the code walker
collectSyncableFiles (first-sync walker) and the incremental PRUNE_DIR_NAMES
set skipped node_modules but not Python venv/. On a Python repo the walker
descended into venv/ (thousands of files); the resulting slug collisions
crashed putPage's INSERT ... ON CONFLICT ... RETURNING with
"undefined is not an object (evaluating 'row.deleted_at')".
Add `venv` alongside node_modules in both the import.ts inline skip and
PRUNE_DIR_NAMES. venv is the Python equivalent of node_modules.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(gateway): carry asymmetric input_type across the AI SDK to the wire body (#1400)
dimsProviderOptions() threads input_type ('query' | 'document') into
providerOptions.openaiCompatible for asymmetric models (ZE zembed-1,
Voyage v3+), but the AI SDK's openai-compatible adapter validates
providerOptions against a fixed schema and silently drops the field
before building the HTTP body. Every embedQuery() was therefore encoded
document-side: the ZE shim's hard default fired ('document'), Voyage and
local openai-compat servers got no input_type at all, and asymmetric
retrieval silently collapsed toward surface-token overlap — while the
providerOptions-level contract test stayed green.
Fix: an AsyncLocalStorage (same pattern as __budgetStore) populated in
embedSubBatch() only when providerOptions actually threads an
input_type, read at body-rewrite time by the fetch shims:
- zeroEntropyCompatFetch: recovers the threaded value; document default
preserved for ingest paths.
- voyageCompatFetch: opt-in like the dims.ts Voyage branch — inject only
when threaded; the field stays off the wire otherwise.
- NEW openAICompatAsymmetricFetch: fallthrough default for every other
openai-compatible recipe (llama-server, litellm, ollama, ...) — the
canonical local/proxy paths for asymmetric models. Strict pass-through
when nothing was threaded, so symmetric deployments see zero wire
change; recipes with their own compat fetch (azure) keep it via the
compat.fetch ?? precedence.
KNOBS_HASH_VERSION bumped 10→11: cached query_cache rows were keyed on
document-side query vectors; pre-fix rows must not be served to post-fix
lookups (same convention as the v=3 embedding-provider bump). One-time
global cold-miss on upgrade; refills within cache.ttl_seconds.
Tests: test/embed-input-type-wire.test.ts runs the REAL SDK transport
with a mocked global fetch and asserts on the outbound body — the only
layer where this regression is observable. Covers ZE hosted, llama-server,
litellm, ollama (query + document sides) and pins the pass-through for
non-asymmetric models and Voyage's opt-in shape. 4 of the original 7
assertions fail on master, proving the pin. One structural pin in
test/ai/zeroentropy-compat-fetch.test.ts updated to the new line shape
(same semantic); KEY_FILES.md gateway.ts entry updated to the new truth.
Supersedes #1400 (closed unmerged) — same ALS mechanism, extended to
Voyage + all openai-compatible recipes. Credit to @billy-armstrong for
the original diagnosis.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(sync): honor .gitignore in code walk; prune vendor/dist/build
collectSyncableFiles (the full-sync / dry-run enumerator) reimplemented its
own directory skip list inline (node_modules || ops), bypassing the canonical
pruneDir gate and ignoring .gitignore entirely. On a Laravel/PHP repo this
descended into vendor/ (~50k Composer files), storage/, and public/build/,
trying to import 52k dependency/build files and flooding the index with
library internals (a 35-min sync that never finished, killed by the watchdog
at 3%).
- collectSyncableFiles now enumerates via `git ls-files --cached --others
--exclude-standard` when dir is a git work tree, so the walk honors
.gitignore (tracked + untracked-not-ignored). Falls back to the FS walk for
non-git dirs. EroLab: 52164 -> 1028 files.
- The FS fallback now prunes through the canonical pruneDir() instead of a
drifted inline list, so the two skip lists can't diverge again.
- PRUNE_DIR_NAMES gains vendor/dist/build (dependency + build-output trees).
Addresses #1483 (.gbrainignore), #1159 (--respect-gitignore), and the
maintainer's #1942 vendor/dist/build prune. Walker regression suites
(sync-walker-symlink, brain-writer-walk-prune, sync, sync-walker-submodule)
green: 90 pass.
* fix(config): ignore DATABASE_URL auto-loaded from cwd .env (#427)
Bun merges .env files from the process cwd into process.env before any
user code runs. loadConfig() prefers env DATABASE_URL over
~/.gbrain/config.json, so any gbrain invocation from inside a web-app
checkout silently retargets the brain at that app's database — reads go
to the wrong DB and apply-migrations can write gbrain's schema into a
production app database (#427).
effectiveEnvDatabaseUrl() re-parses the .env files Bun auto-loads from
cwd and treats a DATABASE_URL whose value matches one of them as
file-origin: ignored, with a one-time stderr notice. GBRAIN_DATABASE_URL
and genuinely exported DATABASE_URLs are honored unchanged, so the
operator escape hatch and the e2e suite's env-provided URL keep working.
Applied at loadConfig, getDbUrlSource (doctor parity), init
--non-interactive, and migrate --to.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(cli): arm the disconnect hard-deadline at teardown entry, not before the op body
The 10s force-exit timer in the shared-op dispatch was armed BEFORE the
try block, so any op whose handler ran past 10s wall-clock was killed
mid-flight with process.exit(0) and zero stdout. On a slow Postgres
pooler (6-10s per fresh connection) a healthy `gbrain search` was
force-exited every time — an empty 'success' indistinguishable from no
results. The v0.42.20.0 exitCode honor can't help: a mid-op kill fires
before any error path sets exitCode.
Move the arming into the finally (teardown entry), matching the
fall-through owner-disconnect site later in main(): the timer still
bounds a hung drain/disconnect (the C13 contract) but can no longer
kill a slow-but-progressing op. Verified on a transaction-pooler
Supabase brain: search went from 0 bytes/exit 0 at 10s to real results
at ~21s.
* fix(import): stamp source_id on extracted call-graph edges
importCodeFile built CodeEdgeInput rows without source_id, so every
edge landed NULL. getCallersOf/getCalleesOf filter
`AND source_id = <scoped>` whenever a worktree pin or --source is in
play — NULL never matches, so scoped call-graph queries silently
returned 0 rows on multi-source brains even though the edges existed
(2,122 edges, 26 targeting the probed symbol, count 0 returned).
One-line fix: carry the sourceId already in scope into the edge input.
Existing NULL rows backfill with:
UPDATE code_edges_symbol e SET source_id = p.source_id
FROM content_chunks c JOIN pages p ON p.id = c.page_id
WHERE c.id = e.from_chunk_id AND e.source_id IS NULL;
(same for code_edges_chunk). Verified: code-callers returns 21 callers
where it returned 0.
* docs(migrations): NULL embeddings BEFORE the column-type alter
The Postgres recipe ordered ALTER COLUMN TYPE vector(N) before the
UPDATE that clears stale embeddings. pgvector refuses to cast existing
vectors across dimensions ('expected 1024 dimensions, not 1536'), so
the recipe as written aborts the transaction on any brain that has
embeddings — which is every brain doing this migration. Swap the steps:
NULLs cast fine.
* fix: honor legacy token source grants in oauth
* fix(cli): bound read-scope op handlers at 180s wallclock (pre-landing review)
With the hard-deadline timer correctly scoped to teardown, a genuinely
wedged read handler (hung pooler connection mid-query) would hang the
CLI forever — the #1633 zombie class the old pre-try timer accidentally
bounded at 10s. Reads now get a generous withTimeout (180s default, far
above any healthy slow-pooler run; --timeout=Ns overrides; exit 124 with
the teardown finally still draining + disconnecting). Writes/admin stay
unbounded: a long import/embed must never be killed by a default.
* fix(import): stamp unscoped edges 'default', matching the pages-table default
Review catch: 'sourceId ?? null' fixed the scoped path but left the
unscoped one (reindex --code without --source, importCodeFile callers
without opts.sourceId) stranding edges at NULL while their pages land
under the schema default (pages.source_id DEFAULT 'default') — so
getCallersOf(sym, { sourceId: 'default' }) missed them. Same bug,
other door. Fallback is now 'default'.
* fix(core): runtime dim-migration recipe NULLs embeddings before the alter
Review catch: the doc fix corrected docs/embedding-migrations.md, but
embeddingMismatchMessage still PRINTED the broken order — ALTER before
UPDATE ... SET embedding = NULL — and linked to the now-contradicting
doc. pgvector refuses to cast existing vectors across dimensions, so
the printed recipe aborted on any brain that has embeddings. Swap the
steps and say why inline.
* feat(migrate): v116 — backfill NULL edge source_id + index from_symbol_qualified
1. Backfill: edges written before the stamping fix sit at source_id=NULL
and stay invisible to scoped call-graph queries until repaired. Derive
each edge's source from its own from_chunk's page (pages.source_id is
NOT NULL DEFAULT 'default'). Same SQL verified live on a 2,122-edge
production brain.
2. Indexes: getCalleesOf filters both edge tables on from_symbol_qualified,
which had no index — every callee lookup was a seq scan, amplified
per-BFS-node by the recursive code walk. With NULL edges repaired,
scoped walks actually expand, so the latent cost becomes real.
Mirrored into src/schema.sql; schema-embedded.ts regenerated.
* docs(migrations): align the rationale list with the corrected recipe order
The 'Why we don't do this automatically' list still said alter-then-wipe;
reorder to wipe-then-alter and replace the fragile 'step 3' numeric
cross-reference with a name-based one.
* test: regression coverage for edge source_id stamping, timer placement, recipe order
- import-code-edges-source-id: scoped import stamps edges + scoped
getCallersOf/getCalleesOf match (verified failing pre-fix), plus the
unscoped-import case asserting 'default' stamping.
- cli-force-exit-teardown-arming: structural pin — the hard-deadline
timer arms inside the finally (teardown entry), never before the op
body; daemon guard, unref, clearTimeout intact.
- embedding-dim-check: recipe order pinned — UPDATE precedes ALTER so
the printed SQL can't drift from docs/embedding-migrations.md again.
* fix(cli): hard-exit after teardown on wallclock timeout; bound makeContext too
Adversarial review, two findings on the new timeout path:
1. On timeout the finally drained, disconnected, then CLEARED the
hard-deadline timer — removing the only backstop while the abandoned
handler (withTimeout races, it does not cancel) can hold ref'd
sockets/SDK timers that keep Bun's loop alive: 'timed out' printed,
process immortal — the zombie class this branch exists to kill,
resurrected through its own fix. The finally now exits explicitly
after teardown completes on the timeout path.
2. makeContext does DB I/O (resolveSourceId) for EVERY op and sat
outside any bound — a pooler wedge at context build hung reads,
writes, and admin alike. It now shares the same wallclock bound.
* fix(import): normalize edge source once — closes the '' door and the unscoped chunk fan-out
Adversarial review: txOpts used truthiness while the edge stamp used
nullish — sourceId:'' put pages under 'default' but stamped edges '',
FK-violating against sources(id) and silently dropping the file's whole
call graph in the best-effort catch. The unscoped getChunks could also
fan out to same-slug chunks from another source. One normalized
edgeSourceId (sourceId || 'default') now drives both the chunk lookup
and the stamp.
* fix(engine): default edge source_id to 'default' at the insert layer (both engines)
Adversarial review: addCodeEdges still wrote e.source_id ?? null, so any
future caller that forgets the field reintroduces invisible NULL edges
the day after the v116 backfill runs. A NULL source_id is invisible to
every scoped call-graph query; default to the schema-default source the
way the pages table does. Applied to both engines (parity).
* fix(core): facts alter recipe NULLs embeddings before cross-dimension alters
Adversarial review: buildFactsAlterRecipe shipped the same defect class
this branch fixes for content_chunks 350 lines up — a cross-dimension
ALTER ... USING cast that pgvector refuses while rows hold old-width
vectors. Dimension changes now wipe first (the facts pipeline re-embeds
on next write); same-dim type swaps (halfvec <-> vector) keep the
lossless cast and PRESERVE data. Both behaviors pinned by tests.
* v0.42.39.0 chore: version bump + CHANGELOG + TODOS
Marks the v0.42.20.0 'decouple the op-dispatch force-exit timer' follow-up
complete — this branch ships exactly that decoupling.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(postgres-engine): atomic JSONB merge in updateSourceConfig — eliminate lost-update race
## Problem
`updateSourceConfig` used a read-then-write pattern: read the current
`config` row, normalize it in JavaScript, then write the merged result
back with `SET config = <normalized> || <patch>`.
Under concurrent callers (two background autopilot/cycle paths patching
different keys simultaneously), both callers can read the same stale
row. The later `SET config = ...` then clobbers the earlier patch,
silently dropping whatever keys the first caller wrote. Reproduced
at 21/25 lost-update events under real Postgres with parallel callers.
## Fix
Fold the normalization and merge into a single atomic `UPDATE … SET
config = CASE … END || patch` statement. Because the `SET` expression
evaluates against the row-locked latest version of `config`, there is
no snapshot window between the read and the write. Concurrent callers
now converge correctly (50/50 clean in reproduction test).
The `CASE` also normalizes historical bad JSONB shapes inline:
- `object` — used as-is
- `string` — double-encoded config; inner text parsed with the SQL
`IS JSON` guard (Postgres 16+) so unparseable strings fall back to
`{}` instead of raising `invalid input syntax for type json`
- `array` — array of patch objects aggregated into a flat object via
`jsonb_object_agg`
- anything else — falls back to `{}`
`pglite-engine.updateSourceConfig` already used an atomic `||` merge;
this change brings postgres-engine to parity.
## Test
Added two assertions to `test/list-all-sources.test.ts`:
1. JSONB string holding non-JSON text normalizes to `{}` (no cast throw)
2. JSONB string holding double-encoded valid JSON is parsed then merged
* fix(doctor): five correctness fixes — stale locks, content sanity, graph coverage, exit code, gateway guard
## 1. Stale lock break hints cover gbrain-cycle: keys
The doctor stale-lock report only recognized `gbrain-sync:` lock prefixes;
everything else fell back to `gbrain sync --break-lock`, which is wrong for
dream/autopilot cycle locks. A `gbrain-cycle:<source>` or `gbrain-cycle`
lock now suggests `gbrain dream --break-lock [--source <name>]`, and
unknown lock shapes fall back to `gbrain doctor` instead of a
misleading sync command.
## 2. content_sanity_audit_recent counts reject and quarantine as hard failures
v0.42 renamed the hard disposition path: rejected pages emit a `reject`
event and quarantined junk pages emit `quarantine`; `hard_block` is now
only the pre-v0.42 legacy alias. The status check only counted `hard_block`,
so fresh `reject` / `quarantine` events from the new path cleared as `ok`
whenever fewer than 10 events existed. The check now sums all three for the
hard count, and `soft_block + flag` for the soft count.
## 3. graph_coverage excludes test fixture entity pages from the denominator
Brains seeded with code sources (e.g. a sync of the gbrain repo itself)
could accumulate test fixture pages typed as `entity` / `person`. Including
these in the entity-count denominator diluted coverage and produced spurious
warnings ("Entity link coverage 0%, timeline 0%") on knowledge-only brains
with no real entity pages. The check now queries a per-entity stats CTE that
excludes `tools/gbrain/test/*` slugs and the `templates/new-person` stub,
with an additional guard for the all-fixture case (`eligibleEntityCount = 0`).
## 4. process.exitCode instead of process.exit at doctor main exit point
`process.exit(hasFail ? 1 : 0)` was a hard kill that prevented cleanup
handlers (Bun unload events, open DB connections) from running. Using
`process.exitCode = hasFail ? 1 : 0` defers the actual termination until
the end of the event loop, allowing cleanup to complete.
## 5. checkSubagentCapability exported for test seams + gateway loop guard
The function was private, making it untestable in isolation. It is now
exported. Additionally, users running gbrain with a non-Anthropic chat model
via `agent.use_gateway_loop=true` no longer receive a spurious warning that
`ANTHROPIC_API_KEY` is missing — subagents route via the gateway loop in
that configuration and do not need the key directly.
## Tests
Doctor test suite: 77 pass, 0 fail (no regressions).
* fix(engine): deleteFactsForPage excludeSourcePrefixes (#1928) + reconnect() parity (#2034)
Engine-layer API for two cycle/availability fixes that share these files:
- deleteFactsForPage gains optional excludeSourcePrefixes so the fence
reconcile can protect non-fence facts (e.g. cli: conversation facts).
- reconnect(ctx?) is now a first-class BrainEngine method on both engines
(PostgresEngine already had it; PGLite gains config capture + reconnect)
so callers stop using disconnect()+bare connect().
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(cycle): stop extract_facts from wiping conversation facts (#1928)
The fence reconcile delete-then-reinsert wiped cli:-origin facts (no fence to
recreate them); a failed-sync full walk turned it brain-wide (1829 rows, 0
reinserted, status ok). Now: exclude cli: rows from the wipe, do NOT inherit
the failed-sync->full-walk fallback for this destructive phase, and warn on
net-negative reconcile.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(autopilot,supervisor): reconnect() instead of disconnect()+bare connect() (#2034)
The autopilot health-probe recovery called connect() with no args after
disconnect(), losing the startup config (database_url undefined -> FATAL
restart-loop on every DB blip) and opening a null-pool window. Both call sites
now use engine.reconnect(), which restores the captured config.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(write-through): mirror to the assigned source's local_path, never the global repo (#2018)
put_page write-through resolved the disk target from the global sync.repo_path,
so a default-source page (local_path NULL) got written into an unrelated
federated source's working tree. Now it uses the assigned source's own
local_path; NULL local_path skips (no leak); the global path is used only as a
sole-source fallback.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(pglite-lock): heartbeat + steal-grace so live holders are never stolen (#2058)
A live holder's lock was force-removed after 5min age alone, letting a second
process share the single-writer data dir -> WAL corruption. The lock now
heartbeats while held; a holder is reaped only when its PID is dead OR its
heartbeat went stale past the steal grace. Pairs PID liveness with heartbeat
age to also defeat PID reuse.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(migrate,doctor): self-heal idx_timeline_dedup drift (#2038)
A migration renumbered during a merge (v102) could be recorded-as-applied
without its DDL running, leaving the 3-column index so every timeline write
failed the 4-column ON CONFLICT. runMigrations now always runs a shape-keyed
drift repair (dedupe-then-rebuild) even when no migration is pending, and
doctor surfaces the drift.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(timeline): un-silence the swallowed batch catch; pin Date-batch round-trip (#2057)
The meetings extractor's bare catch {} hid a brain-wide timeline-write failure
(0 entries, no error). It now counts + surfaces batch errors. Adds a Date-bearing
batch regression test proving the #1861 jsonb_to_recordset refactor already
fixed the original ::text[] cast failure.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: bump version and changelog (v0.42.41.0)
Triage fix wave: 6 authored critical fixes (#1928 facts wipe, #2018
write-through leak, #2034 reconnect loop, #2058 WAL lock, #2038 timeline
migration drift, #2057 timeline silent-empty) + community PRs #2064#2052#2020#2033#2074#2075#2009#2072#2073. TODOS: deferred #1994#1963#2050.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: address adversarial review findings (#1928, #2058, #2038, #2057)
Codex as-built review of the authored fixes surfaced 4 real issues:
- #2058: add a pid+acquired_at ownership token. A stale holder reaped + replaced
past the grace must NOT let its resumed heartbeat refresh, nor releaseLock
remove, the NEW owner's lock (re-opened the concurrent-writer hole). Heartbeat
and release now verify the on-disk lock is still ours. + regression test.
- #1928: the destructive-full-walk guard keyed off phases.includes('sync'),
which wrongly suppressed a legitimate full reconcile when sync was SKIPPED
(no engine / no brainDir). Key off a syncAttempted flag set only when sync
actually ran.
- #2038: dedupe keeps MIN(id) not MIN(ctid) — deterministic and consistent with
the existing v-migration lower-id rule.
- #2057: the extract CLI caller now surfaces batch_errors (stderr + exit 1)
instead of printing a clean success over failed inserts.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(key-files): sync reference to v0.42.41.0 triage-wave behavior
Update KEY_FILES.md to current-state truth for the shipped fixes (no
release-history clauses, per the reference-doc discipline):
- write-through.ts (#2018): resolves the disk target from the assigned
source's own local_path; sole-source falls back to sync.repo_path,
multi-source skips with source_has_no_local_path rather than leak.
- engine.ts (#2034): reconnect() is now a REQUIRED lifecycle method on
both engines; config-restoring, never disconnect()+bare connect().
- migrate.ts (#2073): document v116 edge source_id backfill + callee
index, and the always-run (version-counter-blind) timeline dedup
self-heal.
- new entry for timeline-dedup-repair.ts (#2038) + the
timeline_dedup_index doctor check.
- new entry for pglite-lock.ts (#2058): heartbeat + steal-grace
(GBRAIN_PGLITE_LOCK_STEAL_GRACE_SECONDS) so a live holder is never
stolen.
- extract-facts.ts (#1928): cli:-fact protection, no failed-sync
full-walk inheritance, net_fact_deletion warn floor.
bun run build:llms re-run (KEY_FILES is link-only so bundles unchanged);
freshness + current-state guards green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(write-through): preserve nested multi-source layout; narrow #2018 leak guard
The first #2018 fix skipped any no-local_path source on a multi-source brain,
which broke the legitimate nested layout (a source without its own tree nests
under the host repo at .sources/<id>/ — pinned by put-page-write-through.test).
Narrow the guard: a no-local_path source nests under sync.repo_path as before;
only SKIP when sync.repo_path is literally another source's own local_path
(the actual leak — writing there pollutes that sibling's repo). Caught by the
sharded suite.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: satisfy test-isolation guard for the new lock/reconnect tests
CI `verify` flagged 3 intra-process isolation violations in the tests added
this wave (the parallel runner shares one process per shard):
- pglite-lock.test.ts: the GBRAIN_PGLITE_LOCK_STEAL_GRACE_SECONDS mutation now
goes through withEnv() instead of a raw process.env write (R1).
- pglite-reconnect: renamed to *.serial.test.ts — it creates per-test engines
to exercise the connect/reconnect lifecycle, which doesn't fit the shared
beforeAll-engine model (R3/R4).
verify is now 30/30; both files green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(pglite): reconnect() is a no-op for in-memory engines (#2034)
CI serial-tests + test(5) caught two in-branch regressions from the #2034
PGLite reconnect():
- worker/queue claim-error recovery + their renewLock e2e test assume PGLite
reconnect is absent/no-op (queue.ts documents it). Making it a real
disconnect+reopen wiped an in-memory engine's state mid-job. reconnect() now
no-ops for in-memory (no database_path) — file-backed still re-opens the dir
(state persists on disk). Restores the documented worker assumption.
- connection-resilience 'Supervisor still has the 3-strikes-then-reconnect
path' pinned the removed unsafe-cast text; updated to assert the direct
this.engine.reconnect() call.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: quarantine embed-input-type-wire to serial lane (CI test(5) leak)
#2033's embed-input-type-wire.test.ts configures a 1280-dim embedding gateway;
the active dimension survived into engine-find-trajectory when CI's 10-way
hash-disjoint sharding co-located them (this branch's added files reshuffled the
assignment), failing 7 trajectory tests with 'expected 1280 dimensions, not
1536'. resetGateway() in afterEach clears the gateway but the dimension still
leaked. It mutates global gateway/embedding state, so it belongs in the serial
lane (own bun process, true isolation) by the repo's own definition. Root-caused
by reproducing the exact failing pair locally.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Austin Arnett <austin@sdsconsultinggroup.org>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Dave MacDonald <djmacdonald@ucdavis.edu>
Co-authored-by: pabloglzg <186649799+pabloglzg@users.noreply.github.com>
Co-authored-by: Alex P. <12667893+aphaiboon@users.noreply.github.com>
Co-authored-by: Garry Tan <bo.m.liu@gmail.com>
Co-authored-by: jbarol <barol.j@gmail.com>
Co-authored-by: maxpetrusenkoagent <max.petrusenko.agent@gmail.com>
Co-authored-by: PAI <pai@scaffolde.ai>
* fix(extract,ingest): well-form lone UTF-16 surrogates before jsonb (#2011)
excerpt() in link-extraction.ts sliced the link-context window by raw UTF-16
index, so a boundary landing inside a non-BMP char (emoji, math, CJK) left an
unpaired surrogate half in `context`. Serialized to JSONB for the
jsonb_to_recordset batch insert, Postgres rejects it at the ::jsonb cast and
aborts the whole batch — wedging `extract --stale` because the staleness
bookmark only advances on a clean finish.
- text-safe.ts: new ensureWellFormed() (Bun isWellFormed/toWellFormed) — one
shared surrogate-cleaning primitive.
- link-extraction.ts: excerpt() well-forms the slice (root-cause fix).
- batch-rows.ts: new sanitizeForJsonb() = ensureWellFormed(stripNul(s)) applied
to every free-text body field (link context; timeline summary/detail/source;
take claim/source). Identity/security fields stay un-sanitized and fail closed.
- postgres-engine.ts + pglite-engine.ts: scalar addLink + addTimelineEntry use
sanitizeForJsonb too, matching the batch path on both engines.
- brainstorm/orchestrator.ts: consolidate hand-rolled sanitizeUnicode onto
ensureWellFormed (also fixes consecutive-lone-surrogate mishandling).
Tests: ensureWellFormed unit cases (incl. consecutive lone surrogates), an
excerpt window-split regression, PGLite + Postgres-e2e surrogate cases across
all free-text fields and scalar paths, and fail-closed identity-field tests
proving sanitization was NOT extended to slugs/holders.
* v0.42.39.0 chore: bump version and changelog (#2011)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: update project documentation for v0.42.39.0
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* v0.42.40.0 chore: re-slot release version (was 0.42.39.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: fix env-mutation isolation violations in retrieval-reflex tests
check:test-isolation (rule R1) flags direct process.env mutation in
non-serial test files — bun's parallel runner loads multiple files into
one process, so a leaked GBRAIN_RETRIEVAL_REFLEX flips reflex behavior
in unrelated tests. Both files landed via the #2019 merge; convert the
beforeEach/afterEach env juggling to the canonical withEnv() wrapper,
which restores the prior value via try/finally even on throw.
Fixes the failing `verify` CI check on #2031 (and the `test-status`
aggregate that inherits it). All 30 verify checks green locally.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(integrations): parameterize resolver-row fence by recipe id
The install fence was hardcoded gbrain:agent-voice:resolver-rows, so any
second copy-into-host-repo recipe wrote a mislabeled block (and refresh/
uninstall keyed on recipe id would miss it). Derive it from manifest.recipe.
* feat(context): Retrieval Reflex — teach the agent when/what to retrieve (#1981)
Deterministic per-turn pointer layer in the context engine: a zero-LLM,
precision-biased scan resolves salient entities (names, @handles) to existing
brain pages and injects compact pointers (name → slug → safe synopsis). Detect
+ point, never auto-dump. Fail-open, capped, suppression on prior context only.
Engine-aware resolver ladder (no second DB connection): host ctx.brainQuery →
PGLite serve resolve IPC (unix socket) → Postgres cached direct → disabled.
Synopsis runs through get_page's privacy strip. Plus the retrieval-reflex recipe
+ policy skill, the retrieval_reflex_health doctor check, config gate, and the
init next-step hint.
* chore: bump version and changelog (v0.42.38.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(architecture): KEY_FILES entries for Retrieval Reflex surface (#1981)
Document the new src/core/context/ modules, the context-engine resolver
ladder, the serve resolve IPC, the retrieval_reflex_health doctor check,
and the recipe-id-keyed install fence. Current-state only.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(jobs): reap stale dead-holder cycle/sync locks (#1972)
A crashed sync (OOM, recycle, SIGKILL) stranded its gbrain_cycle_locks row
until something contended for it — reclaim was on-contention only. Add a
host-scoped background reaper: reapDeadHolderLocks deletes locks whose holder
PID is provably dead on this host, scoped to the gbrain-sync:*/gbrain-cycle*
namespaces only (never elections/supervisor/reindex), with a snapshot-matched
delete (date_trunc on acquired_at) that is TOCTOU-safe against PID reuse.
Reuses isHolderDeadLocally (same-host + ESRCH + 60s grace). doctor --fix now
auto-reaps for no-autopilot brains. DRY: selectLockRows + shared mapper now
back inspectLock + listStaleLocks (killed the triplication).
* fix(db): bound pool disconnect so teardown can't eat CLI output (#1972)
pool.end() against PgBouncer transaction-mode never drained, so disconnect
blocked until the CLI's 10s force-exit fired and process.exit()'d mid-write,
truncating stdout (e.g. #1959's relational query returned empty). Add a
gbrain-owned endPoolBounded(pool): Promise.race of pool.end({timeout}) against
a hard timer, so teardown is bounded regardless of what postgres.js does and is
testable. connection-manager ends its direct + read pools concurrently so the
per-pool bounds don't stack. PGLite disconnect is unaffected.
* fix(cycle): complete cooperative-abort coverage + wire lock reaper (#1972)
v0.42.29 made only the embed phase honor the abort signal; a 24h pull still
showed force-evicts from a long non-embed phase ignoring it. Thread the signal
into every cycle-reachable long loop: extract (extractForSlugs + the full-walk
extractLinksFromDir/extractTimelineFromDir), extract_facts (per-page loop +
embed signal + the phantom-redirect 30s lock-retry), and consolidate's bucket
loop. Add a terminal abort check so an aborted cycle never stamps
last_full_cycle_at as a completed run (Codex #9). lint now yields + checks
abort every 200 pages (it's synchronous; the yield is what lets the signal
land). New phase-duration force-evict attribution log names any phase that
crosses the 30s deadline. Wire reapDeadHolderLocks at cycle start.
* chore: bump version and changelog (v0.42.38.0)
#1972 — stale-lock reaper, bounded pool disconnect, and complete
cooperative-abort coverage across cycle phases.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(key-files): current-state for the #1972 reaper, bounded disconnect, abort coverage
document-release: update db-lock.ts (reapDeadHolderLocks + selectLockRows DRY),
db.ts (endPoolBounded), and abort-check.ts (coverage now spans extract/
extract_facts/consolidate/lint + terminal guard) entries to current truth.
* test(isolation): fix shard-order flakes exposed by #1972's new test files
Adding 3 new test files reshuffled the hash-based shards, exposing two
pre-existing test-isolation bugs:
- cycle-consolidate.test.ts assumed the global legacy-embedding preload's
1536-d gateway config still held at initSchema, but a co-sharded test that
calls resetGateway() in teardown nulls it, so initSchema fell back to the
1280-d default and built a halfvec(1280) facts column its 1536-d fixtures
can't fill. Re-pin the legacy OpenAI/1536 config in beforeAll (the pattern
legacy-embedding-preload.ts documents for 1536-d fixture tests).
- db-lock-heartbeat-takeover.test.ts (merged from master's #1794) mutated
process.env.GBRAIN_LOCK_STEAL_GRACE_SECONDS raw, tripping check:test-isolation
rule R1. Convert to withEnv().
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(security): scope cross-source reads to the caller grant; close get_page exact-path leak
One shared resolveRequestedScope() routes every source-scoped read op
(query, code_callers/callees, search_by_image, code_blast/flow, get_page)
through a single fail-closed trust+grant ladder: a remote caller's __all__
collapses to its granted sources (never the whole brain) and an explicit
out-of-grant source_id is rejected. get_page's exact-match path now honors a
federated grant via getPage(sourceIds[]) in both engines. Legacy bearer tokens
carry their stored permissions.source_id grant (bounded, never widened). Also
retries getConfig on transient connection loss.
Closes#1924, #1371, #1393, #1336, #1603.
* fix(ingest): non-string frontmatter no longer aborts lint/sync; embed/hook/catalog papercuts
Parser coerces a non-string title to a string and falls back to inference for
slug/type (never fabricating a "123" slug), with a lint NON_STRING_FIELD finding
surfacing the malformed frontmatter; a defensive guard in content-sanity stops a
non-string title from crashing the whole lint/sync run brain-wide. Plus: embed
--catch-up no longer arms the overflowed 32-bit budget timer (and surfaces
unembeddable chunks); the frontmatter pre-commit hook ships a correct .md/.mdx
regex; and the skill catalog parses YAML block-scalar descriptions.
Closes#1883, #1658, #1556, #1948, #1946, #1840, #1711.
* v0.42.37.0 fix(security,ingest): source-isolation grant enforcement + non-string frontmatter guard + papercuts
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: add NON_STRING_FIELD frontmatter validation class to docs for v0.42.37.0
The v0.42.37.0 non-string-frontmatter fix added an eighth validation
class (NON_STRING_FIELD / lint code frontmatter-non-string-field). Update
the two current-state docs that enumerate the validation classes:
- skills/frontmatter-guard/SKILL.md (seven->eight + table row)
- docs/integrations/pre-commit.md (seven->eight + table row)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* v0.42.35.0 fix(sync): recover from unreachable last_commit instead of full-walking forever (#1970)
When a source's history is rewritten (force-push, master→main consolidation,
squash), the recorded last_commit can fall outside HEAD's history. The old
guard sent both "object missing" and "not an ancestor" to performFullSync — a
full repo re-walk that never advances the bookmark under a cron timeout on a
large cross-region brain, so the source goes silently stale.
Fix: only a truly-absent object forces a full reconcile. A present-but-non-
ancestor bookmark is diffed tree-to-tree directly (git diff A..B needs no
ancestry), importing only the real delta. Adds: oversized-diff fallback to full
reconcile (F-B); performFullSync now purges deleted files, gated to file-backed
pages by source_path so manual/put_page and metafile pages are spared (F-A);
rename-to-unsyncable deletes the stale old page (F-C). 7 new PGLite e2e tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(architecture): record #1970 sync bookmark recovery + full-sync delete reconcile in KEY_FILES
Update the sync.ts entry to current truth: entry-time bookmark-reachability
guard (gc'd anchor → full reconcile; non-ancestor-but-present → direct
tree-to-tree diff), oversized-diff fallback, performFullSync now authoritative
for deletes (file-backed pages by source_path), and rename-to-unsyncable delete.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(search): deterministic relational-query parser
Pure, ReDoS-bounded parser that detects relationship queries ("who invested
in X", "who at X works on Y", "who introduced me to X", "what connects A and
B") and maps them to typed edges. Schema-pack-extensible vocab with subset
validation against the link types ingest produces, so query-side and
ingest-side relation vocabularies can't drift. No-match / pronoun-seed /
adjacency guards keep it precision-first (a candidate only; the arm fires
only when a real seed also resolves).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(engine): relationalFanout typed-edge fan-out (both engines)
Generalizes traversePaths to a SEED ARRAY, aggregating to ranked NODES
(shortest hop, edge-richness count, via-link-types, shortest connecting
path, canonical chunk id) instead of edges. Within-source traversal (never
crosses a source boundary even across a federated scope), link_source=
'mentions' excluded by default, deleted_at filtered at seed + every neighbor
+ every node, bounded depth (<=3) + candidate cap. Adds RelationalFanoutRow
/ RelationalFanoutOpts + the relational SearchResult/SearchOpts fields to
types.
Lands in lockstep in postgres + pglite engines, pinned by a DATABASE_URL-
gated parity block in engine-parity.test.ts; a PGLite unit test exercises
the SQL (typed-edge filter, mentions exclusion, deleted_at, canonical chunk,
multi-seed connects, determinism) in default CI.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(search): relational recall arm + federation key hardening
Wires edge-derived candidates into bare hybridSearch as a FOURTH RRF arm
(relational-recall.ts): parse the original query -> scope-aware,
confidence-gated seed resolution (drops fallback_slugify; never traverses
from a guess) -> relationalFanout -> batch-hydrate, reinforcing each page's
REAL canonical chunk (page-level key for chunkless entity pages) ->
--explain attribution + fail-open audit row. Text-only (no-op in image
mode); pure no-op for non-relational queries; rides every downstream stage
(cosine, post-fusion boosts, dedup, reranker, autocut, token budget).
Mode wiring: relationalRetrieval + relational_retrieval_depth knobs
(conservative off; balanced/tokenmax on; depth 2), per-call thread-through
in both bare + cached paths, KNOBS_HASH_VERSION 9->10 (rel=/reld=), config
keys, modes-dashboard descriptions, and a `relational` param on the query op.
Federation hardening (structural, engine-wide): the RRF/dedup key now
carries source_id via a shared rrfKey() (fixes a latent cross-source
collapse where same-slug pages in different sources merged), and the query
cache scopes by a canonical source-set key (cacheScopeKey) so a federated
read can't be served a single-source row.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(eval): relational benchmark + recall@k harness metrics
NamedThingBench harness gains recall@k / recall@10 (the relational headline
metric) on QuestionResult + FamilyReport, plus typed seed/linkTypes/kind on
NamedThingQuestion so the graph-relationship family is machine-checkable.
Adds the relational benchmark corpus (test/fixtures/retrieval-quality/
relational/): a small entity graph whose answers are LEXICALLY UNRECOVERABLE
— every page body is generic and never names the entity it relates to, so
only the typed edge connects query to answer. corpus.ts is the canonical
source for both the seed loader and the 38-question gold set; relational.jsonl
is generated from it (a drift test pins them equal).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(eval): relational A/B proof + arm fires on all retrieval paths
Fixes the integration bug the eval caught: the relational arm was only
injected on the main RRF path, so it silently did nothing whenever vector
was unavailable — no embedding provider configured (the default in many
deployments) OR embed failure. The arm is now built once and fused via RRF
on ALL THREE hybridSearch return paths (no-embedding-provider, embed-failed
keyword fallback, main path). Without this it would have been dead in
exactly the setups that most need it.
Adds `gbrain eval retrieval-quality --ab-relational`: runs the gold set
twice (arm off vs on) in a fixed mode and reports the graph-relationship
recall@10 lift + Hit@3 + latency add. The CI A/B test pins the headline
result — recall@10 jumps from <25% (lexically unrecoverable) to >75% with
the arm on — and a non-relational query returns identical results arm-on vs
off (the no-op / no-regression gate).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: bump version and changelog (v0.42.34.0)
Relational retrieval feature: typed-edge recall arm + federation key
hardening. Also updates the KNOBS_HASH_VERSION 9→10 assertions across the
remaining search test files (the bump invalidates relational-off cache rows).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: document typed-edge relational retrieval (v0.42.34.0)
CLAUDE.md Search Mode: add relationalRetrieval to the knob table, the
knobs_hash v=9→10 note, and a relational-retrieval summary. RETRIEVAL.md:
add the relational recall arm to the pipeline diagram. Regenerate llms
bundles (build:llms).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(test): size relational fixtures to the actual embedding-column dim
CI shard runs with the ZeroEntropy gateway default (1280-d), but the
relational test fixtures hardcoded 1536-d embeddings, so chunk inserts were
rejected with "expected 1280 dimensions, not 1536" (CheckExpectedDim) — the
`test (6)` shard + `test-status` failures. The column width tracks the
configured gateway default and can shift with shard order, so fixtures now
probe `content_chunks.embedding`'s actual `atttypmod` after initSchema and
size embeddings to it (the pglite-engine.test.ts pattern), via a shared
`probeEmbeddingDim` helper. Verified passing at a forced 1280-d column.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Curated, high-bar diary of the valuable ideas surfaced by the community-PR
wave, grouped into 10 themes with contributor credit and OPEN/CLOSED/HELD
status, so good thinking survives PR closure. Captured during a full triage
+ hygiene pass over the open-PR backlog. Pure docs; no code impact.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(sources): confine sync re-clone to gbrain-owned clones; never delete a user working tree (#1881)
recloneIfMissing deleted local_path whenever a source had a remote_url and a
non-healthy on-disk state, with no check that gbrain actually created the clone.
A source whose local_path was a user's live working tree (remote_url set, no
gbrain-created clone) could have its directory removed and re-cloned over.
- isOwnedClone(): ownership, not path-containment. True only for a config
.managed_clone marker (written by addSource --url) or exact normalized-path
equality with defaultCloneDir(id) (back-compat for pre-marker default clones).
- recloneIfMissing: ownership guard aborts before ANY filesystem op; EXDEV-safe
sibling-temp clone + atomic swap (old aside -> new in -> drop old) with
best-effort restore + a message naming where the original is preserved;
symlink-leaf reject before the destructive rename.
- sync.ts validate_repo_state guards reclone on isOwnedClone (no per-sync warn).
- sources restore degrades to a warning for an unowned source instead of the
misleading "missing clone, try sync" hint.
Tests: #1881 regression (tree survives), isOwnedClone matrix, symlink reject,
EXDEV swap residue-free, --clone-dir owned-via-marker, restore CV3, unownedHint
healthy/degraded, sync-level refusal.
* chore: bump version and changelog (v0.42.33.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: document sources-ops reclone-ownership invariant for v0.42.33.0 (#1881)
Add the missing src/core/sources-ops.ts entry to KEY_FILES.md capturing the
must-never-violate reclone-ownership guarantee: gbrain only deletes/re-clones a
clone it created (isOwnedClone), never a user working tree. Covers managed_clone
marker, defaultCloneDir back-compat, EXDEV-safe swap, TOCTOU + symlink-leaf
guards, unmanaged_path SourceOpError, and the read-only sources restore path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(import): coerce non-string frontmatter title/slug/type (#1939)
YAML `title: 2024-06-01` parses to a Date and `title: 1458` to a number;
the old `(frontmatter.X as string)` cast was a compile-time lie, so
downstream `.toLowerCase()` threw and (via the importer failure gate)
could wedge sync indefinitely. parseMarkdown now coerces via
coerceFrontmatterString (Date -> UTC ISO date, deterministic), and the
pure assessContentSanity self-protects against a non-string title.
* feat(sync): bounded auto-skip failure ledger; poison file can't wedge indexing (#1939)
New src/core/sync-failure-ledger.ts owns the failure store + a crash-safe,
multi-source, concurrent bounded auto-skip valve. A file that fails N
consecutive syncs (GBRAIN_SYNC_AUTOSKIP_AFTER, default 3) auto-skips so it
can't freeze all indexing forever, while fresh failures still fail-closed
and a `<head>` history-rewrite sentinel hard-blocks even with --skip-failed.
- (source_id, path) keying — failures never merge across sources
- success clears a path so attempts are truly consecutive
- advance-before-ack ordering (a crash can't mark a file skipped while wedged)
- shared applySyncFailureGate used by BOTH the incremental and full-sync gates
- legacy-row normalization + duplicate collapse on load
- cross-process lock + atomic temp-rename, age-based stale-lock break
sync.ts re-exports the ledger for existing callers; import.ts records
source-scoped and defers the bookmark to the gate under managedBookmark.
* fix(doctor): sync_failures severity via one shared decision on both surfaces (#1939)
Local buildChecks and remote doctorReportRemote now both route through
decideSyncFailureSeverity, so a stuck bookmark escalates WARN -> FAIL
consistently (oldest-open age > fail cadence, or large unresolved count),
auto-skipped pages stay visible (WARN, not hidden), and the
acknowledged/acknowledged_at field-split that caused drift is gone. The
remote surface stays subprocess-free (file read + Date.parse only).
* chore(test): add trailing newline to e5-lease-cap-ab baseline fixture
* fix(sync): address adversarial review findings on the failure ledger (#1939)
- #1: a parse-failed file that is later deleted/renamed-away no longer leaves
a permanent open ledger row. Removed paths (filtered.deleted, renamed-from,
and the "gone from disk" forward-delete skip branch) are treated as resolved
so the ledger self-heals instead of aging doctor to a stuck FAIL.
- #3: decideSyncFailureSeverity escalates to FAIL on OPEN (blocking) failures
only — auto_skipped rows already advanced the bookmark, so they stay
WARN-visible regardless of count, matching the state-machine contract.
* chore: bump version and changelog (v0.42.30.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: document sync-failure ledger + auto-skip valve for v0.42.30.0
KEY_FILES.md: new src/core/sync-failure-ledger.ts entry (bounded auto-skip
state machine, decideGateAction/decideSyncFailureSeverity/applySyncFailureGate,
GBRAIN_SYNC_AUTOSKIP_AFTER); update sync.ts (failure store moved to ledger,
re-exported), doctor.ts (sync_failures severity via shared rule on both
surfaces), markdown.ts (coerceFrontmatterString), import.ts (managedBookmark).
live-sync.md: poison-file auto-skip tricky-spot. Regenerated llms-full.txt.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: re-bump to v0.42.31.0 (queue collision on 0.42.30.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: re-bump to v0.42.32.0 (queue collision)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(links): relax link_source CHECK to kebab-case provenance + migration v114
Open link_source from a closed allowlist to a kebab-case format gate
(^[a-z][a-z0-9]*(-[a-z0-9]+)*$, char_length<=64) so external derivers
stamp their own provenance (e.g. citation-graph) without a per-deriver
migration. Migration v114: Postgres NOT VALID + VALIDATE (lock-friendly,
transaction:false); PGLite plain DROP+ADD. Updates the schema.sql +
engine provenance contract comments. (#1941)
* feat(links): expose link provenance on link ops + link-add/link-rm/link-sources
add_link/remove_link now accept --link-source/--link-type; add_link guards
the reconciliation-managed built-ins (markdown/frontmatter/mentions/
wikilink-resolved) and defaults omitted provenance to 'manual' (was the
misleading engine default 'markdown'). New cliHints.aliases mechanism with a
startup collision guard registers link-add/link-rm; printOpHelp shows the
invoked alias name. New list_link_sources read op + listLinkSources engine
method (both engines, {sourceId?,sourceIds?}, deterministic order) powers
`gbrain link-sources`, added to the minion read allowlist. (#1941)
* test(links): kebab provenance, op guard, link-sources, aliases + parity
Covers the v114 regex/length boundaries, upgrade-path constraint swap on
existing data, the managed-built-in op guard + manual default, remove_link
type/source filters, list_link_sources scoping (scalar + federated) and
PG/PGLite parity, and alias resolution/collision/help. Fixes the prior
'inferred'-rejection assertion (now valid kebab) in the mentions test. (#1941)
* chore: bump version and changelog (v0.42.31.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: update KEY_FILES for v0.42.31.0 link provenance surface
KEY_FILES.md current-state updates for #1941: link_source now an open
kebab-case provenance (migration v114), the add_link/remove_link guard +
defaults, list_link_sources + listLinkSources, and cliHints.aliases.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(minions): supervisor queue-singleton keying + pidfile cleanup (follow-up #1849)
Two correctness bugs in the v0.42.29.0 supervisor-singleton work, caught by
adversarial review:
- supervisorLockId mixed a config-derived DB identity into the key, but the
lock row already lives inside the target database. Two supervisors on the
same physical DB via different-but-equivalent URLs (pooler vs direct port,
host alias, trailing params) computed different ids and BOTH acquired the
"singleton" lock. Key on the queue alone; the database half of the mutex is
physical. Removes the now-dead currentDbIdentity() from worker-registry.
- The pidfile-cleanup process.on('exit') listener was installed AFTER the
DB-lock acquire, so the LOCK_HELD early-exit stranded the pidfile this
process had just created. Install the listener first.
Regression test pins the listener-before-lock ordering; updates the lockId
test to the queue-only invariant; KEY_FILES updated to current state.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(minions): honest attempt accounting + cooperative abort-honoring + per-handler timeouts (#1737)
- Wall-clock and stall dead-letter paths now increment attempts_made (terminal,
no retry — wall-clock fires at 2x cumulative timeout; retrying non-idempotent
embed/subagent work would duplicate side effects). Surface stalled_counter in
jobs get so 'started 3 / stalled 2 / attempts 0' reads true instead of looking
like broken accounting.
- Thread AbortSignal through embed-backfill/autopilot-cycle -> runPhaseEmbed ->
runEmbedCore -> embedAll(Stale)/embedPage, checking it on BOTH --stale and
--all paths and between embed batches. A timed-out embed phase now bails within
a batch, so the cycle finally releases gbrain_cycle_locks instead of running
the full 10-15 min after the job was killed (the daily cycle-wedge). New shared
src/core/abort-check.ts (isAborted/throwIfAborted/anySignal).
- Per-handler default wall-clock budget (handler-timeouts.ts) stamped at submit
for long handlers without an explicit timeout_ms.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(minions): queue-scoped DB supervisor singleton + canonical pidfile + doctor max-rss check (#1849)
- Acquire a queue-scoped DB lock (tryAcquireDbLock, keyed on the raw DB identity
+ queue) on supervisor.start(): a second supervisor on the same (db, queue)
fails fast with exit 2 regardless of $HOME/--pid-file. Refresh on a dedicated
timer; on refresh failure past the threshold, fail SAFE (exit non-zero) before
the TTL could lapse and let a second supervisor take over. Release on shutdown.
- Canonical default pidfile keyed on brain id (currentBrainId, config-only, no DB
connect) so two brains under one HOME no longer share supervisor.pid.
- doctor: new supervisor_singleton check surfaces the effective --max-rss (from
the started audit event) and warns when the lock holder's (host,pid) differs
from the local pidfile — comparing host+pid, not bare pid. Registered in
doctor-categories.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(agent-voice): topic-aware persona context via server-resolved topicId (#1851)
Summon Mars/Venus into a specific conversation topic so they boot already knowing
the recent thread. A per-topic call link carries only topicId (+ optional display
topicName); the server resolves the recent-conversation context from the brain
(topics/<topicId>.md) — topic CONTENT is never accepted over the wire (that would
be prompt injection + a leak into URLs/referrers/logs). topicId is a strict slug
with a path-traversal guard. New '# Topic Context' prompt slot injected after the
persona body so identity-first ordering still wins; persona identity unchanged.
No topicId -> generic behavior. Contract doc + persona skill docs updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: file #1737 slot-reservation fair-scheduling follow-up TODO (F7)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: bump version and changelog (v0.42.29.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: sync KEY_FILES.md for v0.42.29.0 minions + abort wave (#1737, #1849)
Fold the #1737/#1849 behavior into the existing per-file entries and add the
two new core files, keeping the doc at current-state truth:
- queue.ts: honest attempt accounting on wall-clock + stall dead-letter paths;
defaultTimeoutMsFor stamping at submit.
- supervisor.ts: queue-scoped DB singleton lock (supervisorLockId,
classifySupervisorSingleton, LOCK_LOST, refresh-fail-safe, brain-id pidfile,
max_rss_mb audit).
- worker-registry.ts: currentDbIdentity().
- New entries: src/core/abort-check.ts, src/core/minions/handler-timeouts.ts.
- New doctor.ts extension: supervisor_singleton check.
- cycle.ts / embed.ts extensions: AbortSignal threading note.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(engine): batch inserts use jsonb_to_recordset, not text[] array literals (#1861)
addLinksBatch/addTimelineEntriesBatch/addTakesBatch passed free text through
unnest(${arr}::text[]); postgres.js serialized it to a Postgres text[] literal
that array_in rejected ("malformed array literal") on calendar/Zoom context,
aborting the whole `extract links --stale` sweep. Bind the batch as one JSONB
doc via jsonb_to_recordset(($1::jsonb)->'rows') through the audited
executeRawJsonb contract instead. Shared row builders (src/core/batch-rows.ts)
keep both engines byte-identical; NUL is stripped only from free-text body
fields (context/summary/detail/claim), while identity/security fields
(slugs/source_ids/holder/kind/dates) still reject NUL. addTakesBatch is now
batchRetry-wrapped ('addTakesBatch' audit site) and its BrainEngine signature
takes BatchOpts. Scalar addLink context is NUL-stripped too.
Regression tests on both engines: PGLite always-on poison/NUL/parity suite +
DATABASE_URL-gated Postgres lane (the engine that actually crashed).
* test: make "no Anthropic key" tests hermetic via withoutAnthropicKey
hasAnthropicKey() reads both ANTHROPIC_API_KEY and ~/.gbrain config; tests that
only deleted the env var fired a real LLM call on configured machines (warning
flipped NO_ANTHROPIC_API_KEY -> LLM_OUTPUT_NOT_JSON). New test/helpers/no-anthropic-key.ts
neutralizes both sources (env + GBRAIN_HOME temp dir) for the duration of the call.
Refactors the five no-key tests in think-pipeline + takes-mcp-allowlist to use it,
including two that previously passed only by luck of the live LLM output.
* chore: docs + version bump (v0.42.28.0)
KEY_FILES.md/RETRIEVAL.md describe the jsonb_to_recordset batch path; TODOS.md
files the #1861 follow-ups (element-isolation, remaining ::text[] sites, shared
SQL-string hoist, batch-insert edge-case tests). CHANGELOG + VERSION + package.json
to 0.42.28.0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: sync TESTING.md batch-insert references for v0.42.28.0
The #1861 fix migrated links/timeline/takes batch inserts from
unnest(::text[]) to jsonb_to_recordset. Update the stale "postgres-js
unnest() binding" note and add the two new poison-regression test files
(test/links-timeline-jsonb-poison.test.ts PGLite half,
test/e2e/jsonb-batch-poison-postgres.test.ts Postgres lane) to the inventory.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(sql-query): reject top-level array jsonb params in executeRawJsonb (#1861 P2a)
The "no top-level array" rule was only a comment. A bare JS array bound to a
$N::jsonb position can serialize as a Postgres array literal (not jsonb) through
postgres.js, silently re-entering the "malformed array literal" class #1861 just
escaped. executeRawJsonb now throws a clear error steering callers to the
{ rows: [...] } object wrapper. Verified breaks zero call sites (all pass objects
or null). Codex adversarial P2a; batch-size enforcement (P2b) filed as a TODO.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(supabase): update connection-string setup to new UI + Transaction pooler
Supabase moved the connection string under "Connect" in the top nav and now
shows three options (Direct, Transaction pooler, Session pooler). Update the
tutorial, gbrain init prompts, the setup skill, the verify runbook, and the
live-sync guide to recommend the Transaction pooler (port 6543) — which gbrain
is tuned for (prepared statements disabled, DDL/locks routed to a derived direct
connection).
Document the IPv4 footgun: the derived direct connection is IPv6-only, so on
IPv4-only hosts reads work but sync silently skips pages. Tutorial 7c now leads
with the free fix (GBRAIN_DIRECT_DATABASE_URL -> Session pooler, port 5432) and
keeps the IPv4 add-on as the paid alternative. Removes stale "transaction mode
breaks sync (.begin() is not a function)" warnings and the port-6543 "Session
pooler" mislabels.
Extends PR #1848 by @FilipHarald.
* chore: bump version and changelog (v0.42.26.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(pricing): unify chat-model pricing into one canonical source; add Opus 4.8 (#1819)
Single canonical CANONICAL_PRICING table (src/core/model-pricing.ts) with
canonicalLookup; ANTHROPIC_PRICING and takes-quality MODEL_PRICING become
derived views. cost-tracker, cross-modal runner, skillopt preflight, brainstorm
orchestrator, and brain-score all source from it. Adds Opus 4.8 ($5/$25) so
--max-cost-usd and the dream-cycle budget meter enforce on 4.8 runs; fixes a
stale Opus 4.7 $15/$75 in the takes-quality gate and reconciles Gemini 2.0 Flash
to $0.10/$0.40. Because every table derives from canonical, cross-table price
drift is structurally impossible.
* chore: bump version and changelog (v0.42.25.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: document canonical chat-pricing table for v0.42.25.0
Add KEY_FILES.md entries for src/core/model-pricing.ts (canonical
CANONICAL_PRICING + canonicalLookup) and refresh the now-derived
anthropic-pricing.ts + takes-quality-eval/pricing.ts entries to
current-state. Add the "one canonical chat-pricing table" cross-cutting
invariant to CLAUDE.md. Fix the stale model-price snapshot pointer in
SEARCH_MODE_METHODOLOGY.md (anthropic-pricing.ts -> model-pricing.ts).
Regenerate llms-full.txt.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(minions): route lock claim/renewLock through direct session pool
The Minion lock heartbeat (claim + renewLock) ran every UPDATE through
engine.executeRaw(), which is hardcoded to the read pool. On Supabase that
is the transaction-mode pooler (6543), which recycles connections per
transaction. A lock is held for minutes, so the pooler periodically reaps
the socket mid-heartbeat -> CONNECTION_ENDED -> the lock looks expired ->
the worker force-evicts its own job and the claim loop wedges silently.
Add BrainEngine.executeRawDirect(): same contract as executeRaw, but routes
to the direct session-mode pool (5432, GBRAIN_DIRECT_DATABASE_URL) when
dual-pool is active. No-op delegation on PGLite / non-Supabase / kill-switch.
claim/renewLock now use it. Single-statement UPDATEs only, so the double-claim
guard and the renewLock no-inline-retry contract are preserved. Statements
inside an open transaction keep their tx connection (in-transaction guard keys
on peekReadPool() !== _sql); the lock hot-path never runs inside transaction().
The Postgres impl shares its cancellation plumbing with executeRaw via a
private runUnsafe helper. New test/postgres-execute-raw-direct.test.ts covers
the routing decision (dual-pool on/off x in-tx/not + abort short-circuit)
without a live Postgres; queue-lock-retry.test.ts gains a guard that claim
can never fall back to executeRaw.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* v0.42.24.0 chore: bump version and changelog
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: update project documentation for v0.42.24.0
Document executeRawDirect on the BrainEngine contract and the
claim/renewLock direct-session-pool routing in KEY_FILES.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: make D3 executeRaw-no-retry guard refactor-aware
The DRY refactor in this PR extracted executeRaw/executeRawDirect's shared
cancellation plumbing into a private runUnsafe(conn, ...) helper, so the single
conn.unsafe() call moved out of executeRaw's body. The D3 guard read
executeRaw's source and asserted conn.unsafe( appeared exactly once there,
which now fails (it's zero — executeRaw delegates).
The D3 invariant (no per-call retry wrapper) is unchanged; it just spans the
delegate now. Update the guard to check both public methods delegate to
runUnsafe without reconnect/retry, and assert the exactly-once conn.unsafe +
cancel-only catch in runUnsafe. Also extends coverage to executeRawDirect so
the lock hot-path can't reintroduce a retry wrapper either.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(jobs): niceness core, worker registry, shared supervisor-pid reader
OS scheduling-priority primitives for issue #1815:
- niceness.ts: parseNiceValue (whole-string), applyNiceness (re-reads
effective in success AND failure paths), getEffectiveNiceness, formatNice.
- worker-registry.ts: live workers self-register pid + requested/effective
nice under gbrainPath('workers'); readWorkers prunes ESRCH (keeps EPERM)
with a pid-reuse start-time guard.
- supervisor-pid.ts: readSupervisorPid extracted from the copy-pasted
PID-file + liveness block.
* feat(jobs): --nice flag for jobs work/supervisor + doctor niceness check
Wires the --nice <n> flag (and GBRAIN_NICE env) through the CLI (issue #1815):
- jobs work: applies niceness + registers the worker; cleanup on finally and
process.on('exit').
- jobs supervisor: applies in the foreground-start path only (after the
--detach fork), passes the apply result into MinionSupervisor.
- supervisor.ts: nice opts, extracted testable buildWorkerArgs (appends
--nice), emits niceness on started/worker_spawned audit events.
- jobs stats / supervisor status: surface effective worker + supervisor nice.
- doctor: separate supervisor_niceness check (warns on requested != effective)
so it can't clobber the supervisor crash-check precedence; registered in
doctor-categories.
* test(jobs): cover niceness, worker registry, supervisor-pid, build args
Unit tests for issue #1815: parseNiceValue rejects 3.5/10abc that parseInt
would accept; applyNiceness re-reads effective on EPERM; registry ESRCH/EPERM +
pid-reuse guard + brain-isolated path; readSupervisorPid states; parseNiceFlag
flag>env precedence; buildWorkerArgs --nice propagation.
* chore: bump version and changelog (v0.42.23.0)
--nice flag for jobs work/supervisor (issue #1815).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: document --nice flag for jobs work/supervisor (v0.42.23.0)
- minions-deployment.md: niceness tuning section (full concurrency, low priority).
- KEY_FILES.md: entries for niceness.ts, worker-registry.ts, supervisor-pid.ts;
supervisor.ts entry notes buildWorkerArgs + nice opts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(e2e): add enrich_thin to dream cycle EXPECTED_PHASES
The enrich_thin cycle phase (src/core/cycle.ts ALL_PHASES, between
conversation_facts_backfill and skillopt) shipped without updating the
e2e phase-order expectation, so dream-cycle-phase-order-pglite failed on
master. Sync the expected list to the real ALL_PHASES order.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(e2e): align sync-lock-recovery with the shipped --break-lock --all contract
v0.41.13.0 intentionally dropped the "--break-lock + --all is refused" guard so
cron can self-heal every source in one call (sync.ts runBreakLock iterates
sources under --all). The e2e test still asserted the old exit-1 refusal and
failed on master. Assert the current contract: the combination is accepted and
takes the iterate / no-active-sources path (exit 0, no refusal message).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(e2e): de-flake ingestion-roundtrip chokidar first-drop race
The native fsevents watcher occasionally missed a freshly written file, timing
out the 15s waitFor (~1/3 on master under load). Three fixes:
- inject a polling chokidar watcher via the source's _watchFactory seam
(usePolling, 20ms interval) so detection never depends on fsevents timing;
- drop deterministic fixtures BEFORE start so the initial scan
(ignoreInitial:false) emits them, keeping live-watch coverage only where it's
robust;
- poll for the dedup hit instead of a fixed 600ms sleep.
15/15 green under stress.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(e2e): make hermetic-PGLite serve tests actually hermetic
connect-bearer and serve-stdio-roundtrip init a PGLite brain and spawn serve,
but passed {...process.env} through — leaking an ambient DATABASE_URL /
GBRAIN_DATABASE_URL into the subprocess, which then came up on Postgres and
failed the `engine: pglite` assertion. Strip both DB vars from the spawned env
so the tests are deterministic whether or not the shell/CI has a DB URL set.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(e2e): type the hermetic-PGLite env so tsc passes
The DATABASE_URL/GBRAIN_DATABASE_URL strip used `delete` on a narrowly-typed
env literal (tsc-only failure; bun test doesn't typecheck). Annotate
connect-bearer's env as Record<string,string|undefined> and build serve-stdio's
as a concrete Record<string,string> (StdioClientTransport.env rejects undefined).
Runtime behavior unchanged (7/7 + 3/3 green).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: quarantine worker-registry to *.serial (R1 env-mutation isolation)
worker-registry.test.ts sets process.env.GBRAIN_HOME per-test so gbrainPath
resolves to a temp dir, then lazy-imports the module — a process-global
mutation the parallel isolation lint (rule R1) forbids. Rename to
worker-registry.serial.test.ts: it runs in the serial pass (own bun process,
max-concurrency=1) where env mutation is safe, and the lint skips *.serial
files. No logic change (6/6 green); fixes the failing `verify` CI job.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(minions): supervisor progress watchdog + worker DB self-defense under supervision (#1801)
Alive-but-wedged worker (dead DB pool, process still up) now self-heals in
minutes instead of a silent 15h halt.
- supervisor: progress watchdog restarts a child that makes no forward progress
on claimable work (name+queue-scoped, active_healthy/due-delayed aware,
startup-grace + loop-budget bounded); runtime handler-name derivation.
- child-worker-supervisor: killChild gates on liveness not .killed (also fixes
the existing shutdown SIGKILL no-op); restartCurrentChild kills the captured
child ref; intentional restart doesn't count toward max_crashes.
- worker: DB-liveness probe runs under supervision (db_dead self-exit), stall
detection stays supervised-off.
- doctor: standalone per-queue wedged_queue check + state->status fix in the
remote queue_health check.
- jobs/queue: queue-scoped getStats wedge fields + jobs stats WEDGED line.
* fix(minions): wedge_restart_loop one-shot + supervised-probe comment + jobs-stats threshold (review)
Pre-landing adversarial review findings:
- wedge_restart_loop warn now fires once per exhausted window via a re-arming
flag, not every health tick (was flooding the audit log for the full window).
- Correct the stale GBRAIN_SUPERVISED comment: the DB probe runs under
supervision now; only stall detection is skipped.
- jobs stats WEDGED line reads GBRAIN_WEDGED_QUEUE_WARN_MINUTES so it agrees
with the doctor wedged_queue threshold.
* chore: bump version and changelog (v0.42.22.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: queue-ops runbook + KEY_FILES for the #1801 wedge watchdog (v0.42.22.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(postgres): module-singleton ownership — borrower disconnect no longer nulls the cycle's connection (#1404/#1471/#1619)
gbrain dream on Postgres failed every DB phase with "No database connection:
connect() has not been called": a short-lived borrower probe engine (lint/doctor
config-lift, no poolSize) called db.disconnect() in its own disconnect(), nulling
the shared module singleton the long-lived cycle owner was still using. The module
`sql` is only ever nulled by db.disconnect() (postgres.js auto-reconnects its own
pool), so the failure was always a borrower-disconnect, never an idle-pooler drop.
Fix: db.connect() returns whether THIS call created the singleton (atomic — no
await between the null-check and the sql=postgres() assignment), PostgresEngine
stores it as _ownsModuleSingleton, and disconnect() only calls db.disconnect()
when it owns the connection. Borrowers no-op. Hardening: db.disconnect() snapshots
+nulls sql before awaiting end(); reconnect() shares one in-flight _reconnectPromise.
Tests: new postgres-engine-singleton-ownership.test.ts; expanded DB-gated e2e
matrix (owner/borrower, creation-not-role, symmetric CLI-exit, owner-reconnect-
with-live-borrower); module-style getter asymmetry; #1570 shared-recovery
regression updated to assert the fixed contract.
Co-Authored-By: nullhex-io <noreply@github.com>
Co-Authored-By: joelwp <noreply@github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: bump version and changelog (v0.42.15.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: nullhex-io <noreply@github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(core): drain fire-and-forget sinks before disconnect via a background-work registry (#1762)
New src/core/background-work.ts registry (Map<name,drainer>, ordered drain,
awaited abort). facts-queue (order 0, abort=shutdown), last-retrieved (1), and
eval-capture (3, now self-tracked) register as sinks. Both CLI exit paths
(op-dispatch finally + handleCliOnly finally) drain the registry before
engine.disconnect() so a PGLite db.close() can't race in-flight work into the
re-pump busy-loop that pinned the single-writer lock. Op-dispatch error path
converts process.exit(1) to exitCode+return so the finally still drains.
* fix(ai): bound every outbound AI call so a stalled provider can't hang (#1762/#1775)
withDefaultTimeout composes a per-touchpoint default deadline (chat 300s,
embed/multimodal 60s) with any caller signal via AbortSignal.any. Applied at the
SDK call layer (chat generateText, expand generateObject, OCR, per-sub-batch
embed) — covers native-anthropic + retries — plus per-request multimodal fetch.
embedQuery forwards abortSignal. Env: GBRAIN_AI_{CHAT,EMBED,MULTIMODAL}_TIMEOUT_MS.
* fix(postgres): module-mode reconnect preserves the shared singleton (#1745)
reconnect() branches on connection style. Module-singleton engines re-establish
idempotently via db.connect() (no-op when alive) + refresh the ConnectionManager
read pool, never db.disconnect() — so a transient blip no longer nulls the shared
sql out from under concurrent ops (which threw 'connect() has not been called').
Fail-loud on real connect failure. Instance pools keep teardown+recreate.
* fix(search): bound the query-time embed so a stall falls back to keyword (#1775)
search/query default to cheap-hybrid (embeds the query); a stalled provider made
the embed never settle, so the keyword fallback never engaged and the command
force-exited with no output. One shared QueryEmbedDeadline (6s, floored 2s per
embed) covers both the cache-lookup and inner embeds via embedQueryBounded
(abortSignal + Promise.race) → existing keyword fallback engages. Also registers
the search-cache background-work drainer (now bounded). Env: GBRAIN_QUERY_EMBED_TIMEOUT_MS.
* test+chore: reliability wave tests + v0.42.11.0 (#1762#1745#1775)
New: background-work registry unit, query-embed deadline unit, eval-capture
drain unit, postgres reconnect E2E (#1745), gbrain capture exit-clean case in
the PGLite serial test. Updated fix-wave-structural assertions to the registry
shape. VERSION/package.json/CHANGELOG -> 0.42.11.0; TODOS retrofit marked done.
Incorporates + hardens PR #1763 (drain-before-disconnect + embed fetch timeout);
the residual hung-Haiku hole is closed by the facts shutdown() abort belt.
Co-Authored-By: ElliotDrel <noreply@github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: document background-work registry + v0.42.11.0 reliability wave in CLAUDE.md (regen llms)
* chore: bump release version 0.42.11.0 -> 0.42.20.0
Rename the reliability-wave release version per request. Trio
(VERSION / package.json / CHANGELOG) reconciled; in-code version-tag
comments and test fixtures updated; llms regenerated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: ElliotDrel <noreply@github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ai/gateway): AI SDK v6 tool loop on non-Anthropic providers (#1782, #1764)
v6's asSchema() rejected the bare { jsonSchema } object and called it as a
function ("schema is not a function"), killing every tool-using agent run on
non-Anthropic providers and skillopt on all providers. Wrap tool inputSchema
with the SDK jsonSchema() helper, and add a pure toModelMessages() boundary
adapter in chat() that converts tool results to the v6 ModelMessage shape
(role:'tool' + typed output:{type:'json'|'error-text',value}, isError dropped,
non-JSON-safe output normalized). toolLoop stays provider-neutral and unchanged.
Both skillopt tool builders (rollout.ts, write-capture.ts) switch to the shared
paramDefToSchema mapper so enum/default/items survive. Tests run the produced
shapes through real generateText + MockLanguageModelV3.
Co-Authored-By: michaeladair44 <michaeladair44@users.noreply.github.com>
Co-Authored-By: justemu <justemu@users.noreply.github.com>
Co-Authored-By: JE4NVRG <JE4NVRG@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: bump version and changelog (v0.42.19.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: michaeladair44 <michaeladair44@users.noreply.github.com>
Co-authored-by: justemu <justemu@users.noreply.github.com>
Co-authored-by: JE4NVRG <JE4NVRG@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(extract): links_extraction_lag never clears on Postgres (#1768)
Stamp the full-microsecond updated_at (via to_char ... AT TIME ZONE UTC)
instead of the millisecond-truncated JS Date, so links_extracted_at equals
the DB updated_at exactly and the staleness predicate clears. Stamp SQL
unchanged: version-arm backdating still works, D4 preserved, CDX-1 strengthened.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(core): out-of-band hard-deadline watchdog primitive (#1633)
Bun eval-Worker that SIGTERM->grace->SIGKILLs its own process from a separate
OS thread, so a sync whose main event loop is starved (ReDoS spin) still dies.
Signals SELF (no PID-reuse footgun). Empirically validated on Bun 1.3.13.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(sync): arm hard-deadline watchdog + graceful SIGINT cancel (#1633)
cli.ts installs the watchdog before connectEngine (bounds connect hangs);
resolveSyncHardDeadline + composeAbortSignals in sync.ts; SIGINT graceful
cancel on single-source + --all; withRefreshingLock timer unref'd. Non-TTY
default 3600s makes cron orphan-pileup structurally impossible.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(claude): annotate process-watchdog + #1768/#1633 fixes
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: bump version and changelog (v0.42.13.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: rebump v0.42.13.0 -> v0.42.18.0 (queue collision)
Sibling workspaces claimed v0.42.13-v0.42.17; advance this branch's slot.
VERSION + package.json + CHANGELOG header + CLAUDE.md annotations + llms bundles.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(key-files): current-state phrasing for #1633/#1768 entries (fix check:doc-history)
The doc-history guard bans the bolded **v0.X release-clause marker in reference
docs (history belongs in CHANGELOG + git). Rewrote the extract.ts/sync.ts
additions as current-state prose and de-versioned the process-watchdog entry.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(sync): checkpoint primitives for resumable sync (#1794)
- op-checkpoint.ts: syncFingerprint({sourceId, lastCommit}) keyed on the
anchor (never HEAD) so the checkpoint survives a growing backlog.
- source-health.ts: commitTimeMs(localPath, sha) for stamping
newest_content_at against a pinned (non-HEAD) commit.
- sync-concurrency.ts: resolveMaxConnections + clampWorkersForConnectionBudget
for the opt-in GBRAIN_MAX_CONNECTIONS single-sync footprint clamp.
* feat(sync): resumable incremental sync via pinned-target checkpoint (#1794)
performSyncInner now drains a fixed lastCommit..pin range, banking completed
file paths to op_checkpoints and advancing last_commit (+ last_sync_at) ONLY
at full import completion. A killed/aborted/blocked run leaves the anchor
untouched and resumes from the banked set next run — the convergence fix.
- Pinned target: completion advances to the pin, not live HEAD, so commits
landing after the pin are a clean next-sync diff (kills the staleness window).
History rewrite (pin not an ancestor of HEAD) discards the checkpoint + re-pins.
- Forward-progress head gate: merge-base --is-ancestor pin HEAD replaces the
strict "HEAD == captured" gate that blocked on every concurrent enrich commit.
- Vanished-on-disk added file -> skip + checkpoint, not a failedFiles block.
- Large syncs defer extract/embed to the resumable --stale sweeps (convergence
== import convergence); small syncs keep inline extract/facts/embed.
- GBRAIN_MAX_CONNECTIONS clamp on the worker fan-out (opt-in).
- Typed SyncLockBusyError; the Minion sync handler (jobs.ts) marks the job
SKIPPED (not failed) on a held lock so cron/autopilot defers cleanly.
* feat(doctor): pool_budget check for GBRAIN_MAX_CONNECTIONS (#1794)
computePoolBudgetCheck + checkPoolBudget warn when the parent pool leaves no
room for a parallel sync worker under GBRAIN_MAX_CONNECTIONS, pointing at
GBRAIN_POOL_SIZE=2. Registered in the ops category set.
* test(sync): resumable-sync regression suite + vanished-file contract (#1794)
- sync-resumable-import.serial.test.ts (13 cases): convergence regression,
resume-skips-checkpointed, pinned-target/forward-drift, history-rewrite
re-pin, last_sync_at-not-bumped-on-block + good-file banking, vanished-file
skip, dry-run/empty-diff, + pure fingerprint/clamp/pool-budget helpers.
- sync-parallel.test.ts: vanished-mid-sync added file now asserts the new
skip contract (supersedes the v0.22.13 CODEX-3 failedFiles behavior).
* chore: bump version and changelog (v0.42.17.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(eval): cycle-default — single source of truth for TTY/non-TTY cycle count (#1784)
* fix(jobs): decouple 'jobs watch' format (--json) from loop (--follow); non-TTY prints one human snapshot (#1784)
* fix(reindex): human cost-refusal unless --json; spend guardrail unchanged (#1784)
* fix(eval): annotate non-interactive cycle/budget defaults; runner core TTY-agnostic (#1784)
* chore: bump version and changelog (v0.42.15.0)
Decouple CLI primary output from process.stdout.isTTY (#1784): human by
default, JSON only with --json; jobs watch non-TTY one-shots; eval banners
annotate non-interactive defaults; reindex-code refusal is human unless --json.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: CLAUDE.md Key Files notes for v0.42.15.0 isTTY-output decoupling (#1784)
Annotate jobs.ts (jobs watch format/loop split + resolveWatchMode + the new
cycle-default.ts), reindex-code.ts (human cost-refusal), and eval-cross-modal.ts
(cycle/budget banner annotations). Regenerate llms-full.txt.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(code-intel): readiness signal on code-def/refs/callers/callees (#1780 Gap 1)
New src/core/code-graph-readiness.ts: resolveCodeReadiness() returns a typed
status (not_built | indexing | ready | unknown) + ready boolean so callers can
tell "graph not built / still indexing" apart from "genuinely no match" when
count===0. EXISTS-based (cheap), chunk-grain, resolver-version-matching pending
predicate, fail-open. Wired into the 4 CLI envelopes (+ human hint) and the 4
MCP op handlers. def/refs are 2-state brain-wide; callers/callees 3-state scoped.
* feat(db-lock): automatic same-host dead-pid cycle-lock takeover (#1780 Gap 3)
tryAcquireDbLock now reclaims a held, not-TTL-expired lock when the same-host
holder is provably dead (process.kill ESRCH) past a 60s grace, via guarded
DELETE + one normal-upsert retry returning the normal handle. New shared
injectable classifyHolderLiveness/isHolderDeadLocally (EPERM treated as ALIVE
— never steals a live lock). runBreakLock's safe path consumes the shared
predicate, fixing its prior EPERM-as-dead bug. Cross-host stays TTL-only.
* feat(init): validate the embedding key at gbrain init (#1780 Gap 2)
New src/core/init-embed-check.ts: config-only diagnoseEmbedding (missing key,
all providers) + best-effort 1-token live test-embed (invalid/expired key, 5s
timeout, never blocks). Loud warning to stderr, init still exits 0; skipped by
--no-embedding / --skip-embed-check / GBRAIN_INIT_SKIP_EMBED_CHECK=1. Builds the
effective env (process.env + file-plane keys + --key) via buildGatewayConfig,
extracted to src/core/ai/build-gateway-config.ts (cli.ts re-exports) so the
check sees the same keys + provider base URLs as runtime. embedding_check added
to --json.
* chore: bump version and changelog (v0.42.14.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: document the #1780 zero-config gaps for v0.42.14.0
CLAUDE.md Key Files: add src/core/code-graph-readiness.ts, init-embed-check.ts,
ai/build-gateway-config.ts, and the db-lock auto-takeover + code-* readiness
field behaviors. Regenerate llms-full.txt.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(search): archive/ findable by default — demote not hard-exclude (#1777)
Move archive/ out of DEFAULT_HARD_EXCLUDES into a 0.5 source-boost demote so
archived historical content is findable by default, ranked below curated
content. Add a hidden_by_search_policy doctor check so the surviving exclude
policy (test/, attachments/, .raw/) is auditable. Bump KNOBS_HASH_VERSION 8->9
so the policy change invalidates archive-excluded query_cache rows.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: bump version and changelog (v0.42.13.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: correct search-exclude.test.ts annotation for archive demote (#1777)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(skillopt): wire held-out gate, honest receipts, ENFORCE + ablation opts
Wire the F11 held-out gate into the orchestrator at checkpoint acceptance
(runHeldOutGate was dead code); parse + thread --held-out through CLI, batch,
fleet, background job, and the run_skillopt MCP op. Populate the real
receipt.baseline_sel_score (was hardcoded 0) and add a final-test eval
(test_score + baseline_test_score) via a shared scoreSkillOnTasks primitive.
Fix the --no-mutate proposed.md write (was a stub) and enforce maxRuntimeMin.
D16 ENFORCE in core mutation policy (assertBundledMutationHeldOut): mutating a
bundled skill in place requires a non-empty (>=5), benchmark-disjoint held-out
set or hard-refuses. Add three eval-internal ablation opts (reflectMode,
disableValidationGate, optimizerMode='one-shot-rewrite') recorded in the
receipt + audit; ROLLOUT_SUCCESS_THRESHOLD named constant.
Security: run_skillopt MCP op validates skill_name (kebab-only) and confines
caller-supplied benchmark/held-out paths to the skills dir for remote callers.
* test(skillopt): held-out gate, ENFORCE, one-shot rewrite, runtime + receipt honesty
New test/skillopt/rollout.test.ts (rollout had zero coverage). Held-out ENFORCE
unit cases + one-shot-rewrite fence handling (whole-response unwrap, embedded-fence
preserved, error path). E2E: F11 held-out BLOCKS/ALLOWS, bundled no-mutate write,
reflectMode/disableValidationGate/optimizerMode, maxRuntimeMin abort, receipt
baseline/test-score honesty, held-out/benchmark disjointness, D2 no-DB-pollution.
* chore: bump version and changelog (v0.42.9.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: document skillopt held-out gate + bundled mutation requirement for v0.42.9.0
Wire --held-out into the skill-optimizer SKILL.md, guide flags/safety tables, and
the tutorial's bundled-skill step: mutating a bundled skill in place now requires
--allow-mutate-bundled AND --held-out (>=5 benchmark-disjoint tasks) or it
hard-refuses. Add the --held-out flag row + F11 held-out gate to the guide; update
the receipt contract to the honest baseline/test-score fields.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(gateway): AI SDK v6 toolLoop compat — multi-turn tool calls work again
The ai@6.x bump tightened ModelMessage + tool-schema validation, which
silently broke every multi-turn tool loop. Both `gbrain skillopt` rollouts
and production background `subagent` jobs route through `chat()`/`toolLoop`
and crashed the moment the model called a tool ("messages do not match the
ModelMessage[] schema" / "schema is not a function"). Surfaced end-to-end
by the SkillOpt real-LLM eval.
Three fixes:
- chat(): wrap tool defs with the SDK's `jsonSchema()` helper instead of a
bare `{jsonSchema}` object (v6 asSchema() treated the bare object as a
thunk and threw).
- chat(): new exported pure `toModelMessages()` converts gbrain's
provider-neutral ChatMessage[] into v6 ModelMessage[] — tool results ride
a dedicated `role:'tool'` message with structured `{type,value}` output;
null output preserved as json null. Load-bearing for the production
subagent path, not just skillopt.
- rollout.ts: replace the inline params→schema mapper (dropped `items` on
array params) with the shared `paramDefToSchema` single source of truth.
Pinned by test/gateway-model-messages.test.ts (8 cases). Folds into the
open v0.42.9.0 PR (#1759) — these complete the eval-readiness wave by
making skillopt actually run against a live model.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(skillopt): budget no-pricing for Haiku silently scored every rollout 0
Surfaced by the SkillOpt real-LLM eval (Track B). Two coupled bugs that made
a budget-capped Haiku run report a vacuous "0/N" measurement in ~2ms with
zero LLM calls — indistinguishable from a real deficient-skill score:
1. Claude Haiku 4.5's canonical dateless id (`claude-haiku-4-5`) was missing
from anthropic-pricing.ts (only the dated `-20251001` was present). With
`--max-cost` set, BudgetTracker.reserve() threw no_pricing on the FIRST
chat() of every rollout. Added the dateless entry (sonnet already had its
dateless form).
2. runValidationGate swallowed that BUDGET_EXHAUSTED error — runWithLimit
settled it as {ok:false}, which the gate turned into median:0. A pricing/cap
crash became a fake score. The gate now scans settled results for
isMustAbortError() and re-throws so the caller aborts loudly; ordinary
(non-abort) rollout errors still fail-open to 0 (judge-hiccup posture kept).
Pinned by test/skillopt/validate-gate-abort.test.ts (3 cases). Folds into the
open v0.42.9.0 PR (#1759).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ci): llms-full.txt over size budget — drop what-schemas-unlock from full bundle
The toolLoop + budget bug-fix annotations grew CLAUDE.md, pushing llms-full.txt
to 756KB over the 750KB FULL_SIZE_BUDGET (the `build-llms > size budget` test
failed, failing the `test` CI job). CLAUDE.md stays inlined by design (it's the
point of the one-fetch bundle), so per the budget comment's own guidance ("ship
with includeInFull=false exclusions") this excludes docs/what-schemas-unlock.md
(15.4KB value-explainer, not load-bearing operational reference) from
llms-full.txt; it stays linked in llms.txt. Bundle now 740KB with ~9KB headroom.
No budget bump — 750KB is near the ~190k-token-context fit ceiling.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(ci): re-admit policy docs into ci-cache-hash before doc relocation
docs/**/*.md is deny-listed from the CI cache hash (test-irrelevant). The
CLAUDE.md restructure moves test/release POLICY into docs/TESTING.md +
docs/RELEASING.md, which DO carry contracts the test suite reads. Without
re-admitting them, a policy-only edit would produce the same cache hash and
skip the test shard that runs the build-llms + doc-history guards (false-pass).
Adds an ALLOW_PATTERNS re-admit step after the deny, scoped to the named
policy docs (not a blanket docs un-deny). Lands FIRST, before any doc moves.
Pinned by 3 new cases in test/scripts/ci-cache-hash.test.ts: TESTING.md +
RELEASING.md edits MUST change the hash; docs/guide.md still must not.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(docs): relocate Key files / thin-client / Testing out of CLAUDE.md (verbatim)
CLAUDE.md had grown to 592KB / ~147k tokens auto-loaded every session (~77% of
the llms-full.txt single-fetch bundle). The per-file index was append-only by
mandate. This is the exact thin-dispatcher-vs-fat-blob anti-pattern gbrain exists
to fix, so CLAUDE.md becomes a thin orientation + resolver that points at
on-demand docs.
This commit is the VERBATIM move (content-preserving — the next commit compresses):
- docs/architecture/KEY_FILES.md <- ## Key files + the calibration key-files
cluster + Schema Cathedral v3 impl detail
- docs/architecture/thin-client.md <- ## Thin-client routing
- docs/TESTING.md <- ## Testing
- ## Commands DROPPED (18 'added in vX.Y' history blocks; current surface is
gbrain 0.41.38.0 -- personal knowledge brain
USAGE
gbrain <command> [options]
SETUP
init [--pglite|--supabase|--url] Create brain (PGLite default, no server)
migrate --to <supabase|pglite> Transfer brain between engines
upgrade Self-update
check-update [--json] Check for new versions
doctor [--json] [--fast] Health check (resolver, skills, pgvector, RLS, embeddings)
integrations [subcommand] Manage integration recipes (senses + reflexes)
PAGES
get <slug> Read a page
put <slug> [< file.md] Write/update a page
delete <slug> Delete a page
list [--type T] [--tag T] [-n N] List pages
SEARCH
search <query> Keyword search (tsvector)
query <question> [--no-expand] Hybrid search (RRF + expansion)
ask <question> [--no-expand] Alias for query
IMPORT/EXPORT
import <dir> [--no-embed] Import markdown directory
sync [--repo <path>] [flags] Git-to-brain incremental sync
sync --watch [--interval N] Continuous sync (loops until stopped)
sync --install-cron Install persistent sync daemon
export [--dir ./out/] Export to markdown
export --restore-only [--repo <p>] Restore missing supabase-only files
[--type T] [--slug-prefix S] With optional filters
FILES
files list [slug] List stored files
files upload <file> --page <slug> Upload file to storage
files upload-raw <file> --page <s> Smart upload (size routing + .redirect.yaml)
files signed-url <path> Generate signed URL (1-hour)
files sync <dir> Bulk upload directory
files verify Verify all uploads
EMBEDDINGS
embed [<slug>|--all|--stale] Generate/refresh embeddings
LINKS
link <from> <to> [--type T] Create typed link
unlink <from> <to> Remove link
backlinks <slug> Incoming links
graph <slug> [--depth N] Traverse link graph (returns nodes)
graph-query <slug> [--type T] Edge-based traversal with type/direction filters
[--depth N] [--direction in|out|both]
TAGS
tags <slug> List tags
tag <slug> <tag> Add tag
untag <slug> <tag> Remove tag
TIMELINE
timeline [<slug>] View timeline
timeline-add <slug> <date> <text> Add timeline entry
TOOLS
extract <links|timeline|all> Extract links/timeline (idempotent)
[--source fs|db] fs (default) walks .md files; db iterates engine pages
[--dir <brain>] brain dir for fs source
[--type T] [--since DATE] filters (db source)
[--dry-run] [--json]
publish <page.md> [--password] Shareable HTML (strips private data, optional AES-256)
check-backlinks <check|fix> [dir] Find/fix missing back-links across brain
lint <dir|file> [--fix] Catch LLM artifacts, placeholder dates, bad frontmatter
orphans [--json] [--count] Find pages with no inbound wikilinks
salience [--days N] [--kind P] v0.29: pages ranked by emotional + activity salience
anomalies [--since D] [--sigma N] v0.29: cohort-based statistical anomalies (tag, type)
transcripts recent [--days N] v0.29: recent raw .txt transcripts (local-only)
dream [--dry-run] [--json] Run the overnight maintenance cycle once (cron-friendly).
See also: autopilot --install (continuous daemon).
check-resolvable [--json] [--fix] Validate skill tree (reachability/MECE/DRY)
report --type <name> --content ... Save timestamped report to brain/reports/
BRAIN (capture / ideate / explore — v0.37/v0.38)
capture [content] [--file PATH] Single entrypoint for getting content into the brain
[--stdin] [--slug s] [--type t] Inline content / file / stdin; writes to inbox/ by default
[--source ID] [--quiet|--json] Multi-source brains: route to a non-default source
brainstorm <question> [--json] Bisociation idea generator (hybrid search + far-set + judge)
[--save|--no-save] [--limit N]
lsd <question> [--json] Lateral Synaptic Drift: inverted-judge brainstorm
[--save|--no-save] [--limit N] rewarding far-from-obvious + axiomatic inversions
SOURCES (multi-repo / multi-brain)
sources list Show registered sources
sources add <id> --path <p> Register a source (id = short name, e.g. 'wiki')
sources remove <id> Remove a source + its pages
sync --all Sync all sources with a local_path
sync --source <id> Sync one specific source
repos ... DEPRECATED alias for 'sources' (v0.19.0)
CODE INDEXING (v0.19.0 / v0.20.0 Cathedral II)
code-def <symbol> [--lang l] Find the definition of a symbol across code pages
code-refs <symbol> [--lang l] Find all references to a symbol (JSON-first)
code-callers <symbol> Who calls this symbol? (v0.20.0 A1)
code-callees <symbol> What does this symbol call? (v0.20.0 A1)
query <q> --lang <l> Filter hybrid search to one language (v0.20.0)
query <q> --symbol-kind <k> Filter to symbol type (function|class|method|...) (v0.20.0)
reconcile-links [--dry-run] Batch-recompute doc↔impl edges (v0.20.0)
reindex-code [--source id] [--yes] Explicit code-page reindex (v0.20.0)
sync --strategy code Sync code files into the brain
JOBS (Minions)
jobs submit <name> [--params JSON] Submit background job [--follow] [--dry-run]
jobs list [--status S] [--limit N] List jobs
jobs get <id> Job details + history
jobs cancel <id> Cancel job
jobs retry <id> Re-queue failed/dead job
jobs prune [--older-than 30d] Clean old jobs
jobs stats Job health dashboard
jobs work [--queue Q] Start worker daemon (Postgres only)
ADMIN
stats Brain statistics
health Brain health dashboard
history <slug> Page version history
revert <slug> <version-id> Revert to version
features [--json] [--auto-fix] Scan usage + recommend unused features
autopilot [--repo] [--interval N] Self-maintaining brain daemon
config [show|get|set] <key> [val] Brain config
storage status [--repo <path>] Storage tier status and health
[--json] (git-tracked vs supabase-only)
serve MCP server (stdio)
serve --http [--port N] HTTP MCP server with OAuth 2.1
--token-ttl N Access token TTL in seconds (default: 3600)
--enable-dcr Enable Dynamic Client Registration
--public-url URL Public issuer URL (required behind proxy/tunnel)
call <tool> '<json>' Raw tool invocation
version Version info
--tools-json Tool discovery (JSON)
Run gbrain <command> --help for command-specific help. + the per-command KEY_FILES entries; content stays in git)
CLAUDE.md gains: a Reference map (resolver), a Maintaining section (the
anti-disease rule), and a Cross-cutting invariants subsection under Architecture
so the must-never-violate rules (trust fail-closed, sourceScopeOpts isolation,
JSONB trap, engine parity, contract-first, migrations, multi-source) still
auto-load after the index moved out.
Result: CLAUDE.md 592KB -> 61KB; llms-full.txt 740KB -> 210KB (new docs link-only
until compressed). build-llms drift + budget test green; verify 29/29 green.
The pre-move content is recoverable at git show <this^>:CLAUDE.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(docs): compress relocated docs to current-state + add recurrence guard
Compresses the verbatim-relocated reference docs from append-only release-history
to current-state-only (the disease cure), then makes recurrence structurally
impossible via a CI guard.
Compression (fan-out subagents + adversarial verify, audited mechanically):
- KEY_FILES.md 453KB -> 356KB; TESTING.md 42KB -> 38KB; thin-client.md already clean.
- 393/393 entries preserved; every src/test/scripts path from the verbatim original
survives (mechanical comm-check); zero bolded **v0. markers remain.
- Conservative ratio (~22%) because the content is invariant-dense — correctness
over brevity. Dropped: **vX.Y.Z (#NNN):** clauses, codex/review tags, contributor
credits, PR-numbers-as-ids, pre-fix/then/was-now history deltas. Kept: every
exported symbol, invariant, and Pinned-by reference. Verbatim original recoverable
at git show <relocation-commit>:docs/architecture/KEY_FILES.md.
Recurrence guard (scripts/check-key-files-current-state.sh, wired into verify + check:all):
- HARD: bans the bolded **v0.<digit> marker in the reference docs (scoped — plain
'as of pgvector 0.7' prose is fine, no false positives).
- HARD: CLAUDE.md size cap (90KB; currently 61KB) — the structural backstop.
- Pinned by test/scripts/check-key-files-current-state.test.ts (7 cases).
Content contracts (test/build-llms.test.ts, +5 cases per codex outside-voice):
CLAUDE.md keeps inline ship IRON RULES (version format, document-release,
never-hand-roll); AGENTS.md keeps its boot order; llms indexes the new docs;
KEY_FILES stays link-only (not inlined).
Privacy: scrubbed the relocated 'wintermute/chat/' source-boost examples + the
literal harvest-lint regex to generic placeholders (legitimate in allowlisted
CLAUDE.md; genericized for the new public docs per the privacy rule).
Reverts the 284c50a4 band-aid: re-inlines docs/what-schemas-unlock.md now that the
restructure freed ~530KB of bundle headroom (llms-full.txt 740KB -> 225KB).
verify 30/30 green (incl. new check:doc-history).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(docs): relocate verbose release process to docs/RELEASING.md
The highest-/ship-risk commit (isolated so it can revert alone). Moves the verbose
release + contributor procedure out of CLAUDE.md, keeping every ship-critical IRON
RULE inline so /ship + /document-release (which read CLAUDE.md) cannot regress.
Moved to docs/RELEASING.md: pre-ship test requirements; the CHANGELOG-branch-scoped
+ CHANGELOG voice + release-summary template; the 'To take advantage of vX' block
spec; version migrations + migration-is-canonical; schema state tracking; GitHub
Actions SHA maintenance; PR-descriptions-cover-the-branch; community-PR-wave;
checking-out-PRs-from-garrytan-agents.
Kept INLINE in CLAUDE.md (ship-critical IRON RULES — do NOT move):
- the Version-locations table (5-file sync) + the 3-line consistency audit
- Conductor branch=workspace
- Post-ship /document-release (MANDATORY)
- Privacy + Responsible-disclosure rules (Privacy also anchors the check-privacy
allowlist — the only place allowed to name the fork)
- PR-title-version-first
- never-hand-roll-ship (Skill routing)
Plus a new ## Releasing pointer ('Before any ship, read docs/RELEASING.md in full')
and a resolver row.
CLAUDE.md 61KB -> 39KB (592KB -> 39KB overall, 93% cut; ~9k tokens auto-loaded vs
~147k). CLAUDE.md size-gate tightened 90KB -> 60KB. The content-contract tests pin
that the inline IRON RULES (MAJOR.MINOR.PATCH.MICRO, document-release, hand-roll
ship) did NOT move out. The moved ranges carry no banned fork name, so RELEASING.md
needs no privacy allowlist entry. verify 30/30; bundle 225KB -> 204KB.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(changelog): note CLAUDE.md restructure in v0.42.9.0
The CLAUDE.md thin-resolver restructure (592KB → 39KB) rides in this
release; record it under the existing v0.42.9.0 For-contributors section.
No version bump — v0.42.9.0 is unreleased and already allocated to this PR.
* fix(ci): ci-cache-hash re-admit matched a literal \t, a no-op on GNU grep
The policy-doc re-admit (75992b77) put `\t` inline in the ALLOW patterns
passed to `grep -E`. BSD grep (macOS local) treats `\t` as a tab so it
worked locally; GNU grep (Ubuntu CI) treats it as literal `t`, so nothing
re-admitted and docs/TESTING.md / docs/RELEASING.md stayed deny-listed —
the two policy-doc tests failed on CI shard 6 (1097 pass / 2 fail).
Build ALLOW_RE with `printf '\t(%s)'` so the tab is a real byte, identical
in construction to DENY_RE (line 117), which the CI log shows matches
correctly on GNU grep. End-to-end: editing docs/TESTING.md now flips the
hash; a normal docs/*.md add still does not (deny stays scoped).
* fix(skillopt): feed the scorer's success criteria to the optimizer
Surfaced by the SkillOpt real-LLM eval (Track B). The reflect step was shown
only a pass/fail score and the agent transcript — never WHAT the benchmark
judge rewards. On a skill judged by structure (e.g. "must include a
Confidence: line") the optimizer proposed plausible-but-off edits ("close with
a synthesis") that never satisfied the literal check; every candidate scored 0
on D_sel, the validation gate rejected them all, and the skill text never
changed (optimized === baseline === 0).
Fix: render each benchmark Judge (rule checks / llm rubric / qrels) into
plain-English criteria via new exported describeJudge / describeJudges, and
thread them into the reflect prompt (a SUCCESS CRITERIA block) for both the
loop reflect calls and the one-shot-rewrite path. The orchestrator computes the
distinct criteria across train+sel+test once. The optimizer system prompt now
instructs it to satisfy the criteria through genuine content, never empty
keywords — reward-hacking stays defended by the independent held-out gate
(cat32 confirms the gate catches a keyword-stuffing hack).
End-to-end this took a deficient skill from 0.00 to 1.00 on a held-out set it
never trained on. Pinned by test/skillopt/reflect.test.ts (describeJudge per
kind, describeJudges dedup, criteria present/absent in the prompt). Folds into
the open v0.42.9.0 PR (#1759).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* v0.40.8.2 fix(extract): opt-in global-basename wikilink resolution (#972)
Bare wikilinks like [[struktura]] that point at pages in another folder
were silently dropped from the graph. The issue reporter saw 71 wikilinks
in Obsidian render to 12 in gbrain (~83% lost). Symptoms downstream:
`gbrain graph` returns thin neighborhoods, `gbrain backlinks` undercounts.
This release adds an opt-in mode that resolves bare wikilinks by basename
match, covers all three resolver surfaces (FS-source extract, DB-source
extract, put_page auto-link), and emits one edge per match — no silent
winner on ambiguity. `gbrain doctor` surfaces a paste-ready enable hint
when ≥5 bare wikilinks would resolve under the new mode.
Enable with:
gbrain config set link_resolution.global_basename true
gbrain extract links
Default stays off. Existing brains see zero behavior change on upgrade.
Closes#972. Adapts PR #1233 from @rayers (regex shape + slug-tail index)
into a multi-match, opt-in form with FS-source coverage that the original
PR explicitly skipped.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: document opt-in global-basename wikilink resolution (#972)
The #972 feature shipped with no user-facing docs — only CHANGELOG + CLAUDE.md.
Anyone migrating an Obsidian/Notion vault with bare [[name]] wikilinks couldn't
discover the link_resolution.global_basename flag unless gbrain doctor happened
to surface its hint.
- README "Self-wiring knowledge graph": one sentence on the opt-in mode for
Obsidian-style cross-folder bare wikilinks + the doctor pre-check, linking to
the install step.
- INSTALL_FOR_AGENTS Step 4.5 (Wire the Knowledge Graph): a dedicated agent-
facing subsection — when bare [[name]] links need it, the enable command,
re-running extract, the doctor opportunity hint, and the multi-match behavior.
- Regenerated llms-full.txt.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(#972): resolve aliased wikilinks by target slug, not display text
Codex outside-voice [P1]: `[[struktura|the project]]` resolved the basename
"the project" (the alias) instead of `struktura` (the target), because
extractPageLinks called resolveBasenameMatches(ref.name) and the doctor check
keyed basenameIndex.get(e.name). ref.name is the display alias (match[2]);
ref.slug is the wikilink target (match[1]).
- extractPageLinks resolves ref.slug; context excerpt locates ref.slug.
- doctor link_resolution_opportunity keys e.slug so its estimate matches
what extraction actually resolves.
- Test: aliased wikilink calls resolveBasenameMatches with the target, never
the display text.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(#972): reconcile wikilink-resolved edges in put_page auto-link
Codex outside-voice [P1]: put_page's reconcilableOut filter excluded
link_source='wikilink-resolved', so a basename edge written by auto-link
survived after the bare wikilink was deleted from the page OR the
link_resolution.global_basename flag was turned off (the stale-removal loop
only iterates reconcilableOut). Add 'wikilink-resolved' to the reconcilable
set; manual edges still untouched.
Test: write page with [[struktura]] (flag on) → edge lands; re-put without
the wikilink → edge reconciled away.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(#972): source-scope basename resolution (no cross-source edges)
Codex outside-voice [P1]: makeResolver.resolveBasenameMatches called
engine.getAllSlugs() unscoped, so a bare [[name]] could resolve to a
same-tail page in a DIFFERENT source and create a cross-source edge. The
engine exposes getAllSlugs({sourceId}) precisely to prevent this. #972 is
"global basename across folders," not "cross-source federation" — the
canonical gbrain multi-source bug class.
- makeResolver gains opts.sourceId; ensureBasenameIndex passes it to
getAllSlugs (unscoped only when sourceId omitted — back-compat).
- runAutoLink (put_page) passes opts.sourceId; extractLinksFromDB passes
sourceIdFilter. FS extract is already single-source (walks one dir).
- Tests: scoped index returns only the source's slugs (no cross-source);
unscoped call stays brain-wide.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(#972): FS-source basename edges carry link_source='wikilink-resolved'
The FS extract path is the issue's default repro (gbrain extract links with no
--source db). ExtractedLink had no link_source field, so FS basename edges
landed with the engine default ('markdown') instead of the 'wikilink-resolved'
provenance the DB / put_page paths set and the docs promise. The e2e FS test
only asserted link_type, so it was blind to this.
- ExtractedLink gains link_source?; extractLinksFromFile sets it to
'wikilink-resolved' on basename edges (undefined for ordinary markdown).
- Carries through the addLinksBatch snapshots automatically (LinkBatchInput
already has link_source); single-row addLink fallback now passes it too.
- e2e FS repro asserts link_source === 'wikilink-resolved'.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(#972): one shared basename matcher across resolver/FS/doctor
Codex outside-voice [P2] DRY: three surfaces each hand-rolled a basename
matcher with divergent key sets — the doctor omitted the slugified key, so its
link_resolution_opportunity estimate undercounted what extraction resolves, and
the resolver returned matches in unsorted getAllSlugs bucket order.
New shared exports in link-extraction.ts: buildBasenameIndex(slugs) +
queryBasenameIndex(index, name) (keys raw/lower/slugified tail; stable sort
shorter-first then lexical) + normalizeBasename.
- makeResolver.resolveBasenameMatches → queryBasenameIndex (now stable-sorted).
- extract.ts resolveBasenameMatchesFromSlugs → delegates to the shared pair.
- doctor link_resolution_opportunity → shared builder/query (slugified key
added; estimate now matches extraction).
- Test: doctor counts a slugified-only match ([[Fast Weigh]] → companies/fast-weigh).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(#972): P2 cluster — masking, code-fence, self-link, dedup decision
Codex outside-voice P2 findings:
- P2a markdown-label masking: a wikilink inside a markdown-link label
([see [[acme]]](companies/acme.md)) spawned a stray generic basename ref.
Pass-1 can't match the nested brackets, so a new MARKDOWN_LABEL_WIKILINK_RE
masks those spans out of pass 2c. Inner [[acme]] is now inert.
- P2b FS code-fence: the FS path (extractMarkdownLinks on raw content) didn't
strip code blocks like the DB path. extractLinksFromFile now scans
stripCodeBlocks(content) so [[name]] inside a fence creates no FS edge.
- P2c self-link guard: a basename [[own-tail]] on its own page resolved back
to itself. Dropped in both extractPageLinks and the FS path.
- P2d dedup: documented the decision to KEEP qualified + bare edges to the
same target as separate rows (distinct provenance/audit trail).
- P2e: skipFrontmatter unresolved-contract tests added.
Tests: P2a inert-label, P2c self-link drop, P2b code-fence, P2e unresolved.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* perf(#972): bound the doctor link_resolution_opportunity scan
The check did listAllPageRefs() + a getPage() per page under a 60s budget.
On a large brain (the eng-review concern) it hit the budget every non-fast
doctor run and returned a perpetual partial, adding ~60s.
Now batch-loads the 1000 most-recent pages in ONE query
(ORDER BY id DESC LIMIT SAMPLE_LIMIT) and scans in memory, with the 60s cap
kept as a backstop. Mirrors the v0.40.9 sampling convention. The estimate
message names the bound when the brain exceeds the sample
("scanned the 1000 most-recent of N pages").
Test: source-grep pins the bounded query + the absence of the per-page
getPage walk.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(#972): reconcile stale version/migration references to v112 / 0.42.6.0
Merge churn left intermediate refs: schema.sql + schema-embedded.ts said
"migration v93", CLAUDE.md said "v0.41.32.0 / Migration v109", CHANGELOG said
"Migration v93". Reconciled all to migration v112 / shipping 0.42.6.0. The
CLAUDE.md annotation is also refreshed to describe the final behavior (shared
matcher, source-scoping, alias-by-target, stale-edge reconciliation, bounded
doctor scan) and credit @rayers + @ukd1. Regenerated schema-embedded + llms.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(#972): register doctor check category + bump llms budget to 800KB
Two full-suite gate failures from the re-sync:
- doctor-categories drift guard: the new `link_resolution_opportunity` check
wasn't in any category set. Added to BRAIN_CHECK_NAMES (alongside
graph_coverage / orphan_ratio — it's a graph-quality signal).
- build-llms size budget: the #972 Key Files annotation (landing with master's
#1696/#1699 waves) pushed llms-full.txt past 750KB. Bumped FULL_SIZE_BUDGET
750KB→800KB, the established "budget tracks CLAUDE.md's legitimate per-feature
growth" pattern (600→700→750→800 across releases).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Garry Tan <garrytan@gmail.com>
* feat: content-quality gate on sync — quarantine junk + flag boilerplate (#1699)
Three-tier disposition at the importFromContent narrow waist:
- High-confidence junk (Cloudflare/CAPTCHA interstitial patterns + operator
literals) -> quarantine (hidden from search, zero chunks) or reject.
- Fuzzy markup-heavy (prose-vs-markup ratio, warn-tier window, code-exempt)
-> content_flag marker, stays searchable, agent warned.
- Oversize -> existing embed_skip soft-block + content_flag:oversized warning.
Agent-warning channel: SearchResult.content_flag (stamped in hybridSearch +
the keyword-only search op) and a top-level content_flag on get_page.
New quarantine.ts markers, gbrain quarantine CLI (list/clear/scan), doctor
quarantined_pages + flagged_pages checks (engine.executeRaw, works on PGLite),
sources-audit disposition awareness, markup-heavy lint rule, config keys.
Security: gate-owned markers stripped from untrusted (remote MCP) frontmatter
so a write-scoped client can't hide pages or inject the warning channel.
Markers excluded from content_hash so flagged pages don't re-embed every sync.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: bump version and changelog (v0.42.8.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: document content-quality gate (quarantine + content_flag) for v0.42.8.0
Add CLAUDE.md Key Files + Commands entries for the #1699 content-quality
gate: src/core/quarantine.ts, gbrain quarantine CLI (list/clear/scan),
the agent-warning channel (SearchResult.content_flag + get_page), doctor
quarantined_pages/flagged_pages checks, the markup-heavy lint rule,
sources-audit disposition awareness, and the three new content_sanity
config keys. Regenerate llms-full.txt from CLAUDE.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(extract): link/timeline extraction freshness watermark (#1696)
Closes the "imported != curated" gap: plain `gbrain sync` only extracts
CHANGED pages, so a brain with autopilot off accumulated a links table that
was ~99.7% untyped `mentions` with nothing surfacing it. Adds a per-page
freshness watermark (pages.links_extracted_at, migration v112) and three
things built on it:
- `gbrain extract --stale [--source-id] [--catch-up] [--dry-run] [--json]`:
incremental DB-source link+timeline sweep over pages whose extraction is
stale (never extracted, edited since, or extractor version bumped). Small
byte-bounded batches, non-swallowing flush, stamp-after-flush so a crash
re-extracts idempotently. Stamps with the row's READ updated_at (not now())
so a concurrent edit during the sweep stays stale instead of being lost.
- `links_extraction_lag` doctor check (local + remote): warn-only by default
(>20%), hard-fail only via GBRAIN_EXTRACTION_LAG_FAIL_PCT. Vacuous-skip
<100 pages; pre-v112 brains graceful-skip.
- `gbrain sync --no-extract` flag + end-of-sync nudge (fires on
synced|first_sync|up_to_date so the initial import surfaces its backlog).
Three new BrainEngine methods (countStalePagesForExtraction /
listStalePagesForExtraction / markPagesExtractedBatch) with Postgres<->PGLite
parity + bootstrap probes. Schema parity: schema.sql + regenerated
pglite-schema.ts + schema-embedded.ts + bootstrap-coverage test. Migration
v112 (composite (source_id, links_extracted_at) index, no backfill so the
real backlog surfaces on first doctor run).
* test(audit): hermetic GBRAIN_AUDIT_DIR override for prune ENOENT case
The "no-op when audit dir does not exist (ENOENT)" case called
pruneOldBatchRetryAuditFiles without a GBRAIN_AUDIT_DIR override, so it read
the developer's real ~/.gbrain/audit and flaked (kept>0) on any machine with
prior gbrain audit history. Point it at a guaranteed-nonexistent temp path so
it tests the real missing-dir branch hermetically — matching the file
header's "never touches ~/.gbrain/audit" contract. Pre-existing flake
(introduced by v0.41.19.0 #1537), unrelated to #1696.
* chore: bump version and changelog (v0.42.2.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: CLAUDE.md key-files entry for the #1696 extract-stale wave + regen llms-full
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(minions): self-identifying RSS watchdog + cgroup-aware default + pooler-reap self-heal (#1678)
Problem 1: distinct WORKER_EXIT_RSS_WATCHDOG exit code + cause-keyed supervisor
breaker (bypasses the stable-run reset that hid the 400x/24h loop) + rss_watchdog
audit bucket + 80% soft-warn; cgroup-aware resolveDefaultMaxRssMb replaces the
flat 2048 default at every spawn site.
Problem 2: CONNECTION_ENDED classified retryable; postgres-engine sql getter
throws a retryable error on a reaped instance pool instead of the misleading
module-singleton fallthrough; promoteDelayed reconnect-retry; claim recovers on
the next poll tick (no double-claim); lock-renewal tick reconnect-once dep.
* feat(cycle): surface silent extract_atoms backlog + bounded --drain + fix lint clobbering the shared DB connection (#1678)
Problem 3: extract_atoms_backlog doctor check + pack_gated skip marker +
shared countExtractAtomsBacklog; `gbrain dream --phase extract_atoms --drain
[--window N]` single-hold bounded drain (same cycleLockIdFor, rediscover each
batch, reports remaining, exits non-zero while work remains).
Also fixes a real production bug found via E2E: the cycle lint phase's
resolveLintContentSanity created + disconnected a module-style engine that
nulled the shared db singleton mid-cycle, breaking every later phase with
"connect() has not been called". Lint now reuses the caller's live engine
(cycle + Minion handlers thread it; standalone CLI keeps the create-own path).
* chore: bump version and changelog (v0.41.39.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(#1678): pre-landing review — route transaction/withReservedConnection through the sql getter + drain treats failed count as incomplete
Codex adversarial review findings:
- #2: transaction(), withReservedConnection(), and one other site bypassed the
v0.42.2.0 sql-getter self-heal via `this._sql || db.getConnection()`, so a
reaped instance pool fell through to the module singleton there. Route all
three through `this.sql` so they throw the retryable instance-pool error and
recover consistently (MinionQueue.transaction hits this).
- #4: `gbrain dream --drain` treated a null backlog count (query failure) as
success via `remaining ?? 0`; now null exits EXIT_DRAIN_INCOMPLETE so
automation never believes an unverified backlog drained.
- #1 (claim orphan) + #3 (PGLite drain lock) documented as follow-ups in TODOS.
* docs: document v0.42.2.0 #1678 modules + behavior in CLAUDE.md
Adds Key Files entries for worker-exit-codes.ts, rss-default.ts, and
extract-atoms-drain.ts, plus v0.42.2.0 annotations on worker.ts,
child-worker-supervisor.ts, lock-renewal-tick.ts, and dream.ts. Regenerated
llms-full.txt to match (test/build-llms.test.ts gate).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: re-version v0.42.2.0 → v0.42.5.0 across VERSION/package.json/CHANGELOG/docs/comments
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: think --model fails loud on unresolvable model; never persist empty synthesis (#1698)
Slash-form model ids (anthropic/claude-sonnet-4-6) silently degraded to the
no-LLM stub and wrote empty synthesis pages with exit 0 (reporter saw 200).
Three compounding defects, fixed:
- normalizeModelId (src/core/model-id.ts): one shared provider:model normalizer
replacing 4 colon-only inlines; slash→colon, bare→default, malformed
leading-separator (:foo) returned unchanged so resolveRecipe throws loud.
- validateModelId + probeChatModel (gateway.ts): shared id-validity + key probe;
runThink hard-errors on an explicit --model it can't run (no silent degrade).
- synthesisOk + persist-skip (think/index.ts): empty/malformed/empty-JSON
synthesis is never persisted; --save with no synthesis exits 1.
- auto-think (cycle/auto-think.ts): empty synthesis no longer counts complete or
advances the cooldown (the autonomous third caller of persistSynthesis).
- hasAnthropicKey consolidated into src/core/ai/anthropic-key.ts (3 copies → 1).
MCP think op sets modelExplicit; saved_slug '' maps to null.
* chore: bump version and changelog (v0.42.4.0)
#1698 think --model fail-loud wave. Also files the P3 follow-up TODO for the
provider-symmetric early gate (D1 accept-as-is).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(search): autocut — score-discontinuity result-sizing on the rerank separatrix
Cut the ranked set at the cross-encoder rerank-score cliff instead of a fixed
top-K. Default-ON in reranked modes (balanced/tokenmax), no-op without a
reranker. New pure src/core/search/autocut.ts; mode.ts knobs + reranker_top_n_in
= searchLimit (no unscored tail); query op autocut param; --explain + glossary.
* test(search): autocut pure-fn, agent-surface, behavioral + precision/recall eval gate
Adds autocut.test.ts, query-op-autocut.test.ts, autocut-integration.serial.test.ts
(IRON-RULE behavioral via rerankerFn seam), autocut-eval.test.ts (in-repo
precision-lift-without-recall-regression gate). Updates existing knobsHash/bundle
pins to v=7 + reranker_top_n_in.
* chore: version + changelog + docs for autocut (v0.41.34.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(search): autocut preserves alias-hop exact matches + cache-HIT meta (codex P1/P2)
P1: applyAliasHop injects the canonical page after reranking (no rerank_score);
autocut would drop it when cutting on the scored set. applyAutocut gains an
optional preserve predicate; hybrid passes r => r.alias_hit === true.
P2: cache-HIT cachedMeta now carries autocut/adaptive_return/mode/embedding_column.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: bump version to v0.42.3.0 (autocut wave)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: PR titles lead with the version (IRON RULE in CLAUDE.md)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 20:16:37 -07:00
799 changed files with 83416 additions and 7569 deletions
echo "::error::DATABASE_URL must be set for the jsonb-parity job — the #2339 guard would silently skip (the exact failure PGLite hides). Failing the job." >&2
exit 1
fi
- name:Run JSONB double-encode parity tests on real Postgres
@@ -254,10 +254,28 @@ The whole loop is described in [`docs/architecture/topologies.md`](docs/architec
**Hybrid search.** Vector (HNSW on pgvector) + BM25 keyword + reciprocal-rank fusion + source-tier boost + intent-aware query rewriting. Three named search modes (`conservative`, `balanced`, `tokenmax`) bundle the cost/quality knobs into a single config key. Live cost/recall comparisons in [`docs/eval/SEARCH_MODE_METHODOLOGY.md`](docs/eval/SEARCH_MODE_METHODOLOGY.md). Default: `balanced` with ZeroEntropy reranker on. Per-query graph signals notice when a top result is a hub for THAT query (adjacency boost), is corroborated across team brains (cross-source boost), or is being crowded out by weak chunks from a chatty session (session demote). Run `gbrain search "<query>" --explain` to see per-stage attribution: base score, every boost that fired, what it multiplied. `gbrain doctor` ships a `graph_signals_coverage` check; `gbrain search stats` shows fire counts and failure breakdowns. Vector retrieval pools the best chunk per page, so a page surfaces on its strongest evidence instead of losing to a neighbor on one weak chunk. Queries that match a page's title phrase or a declared free-text alias (`gbrain reindex --aliases` backfills existing pages) get boosted to the page they name. Every result carries an `evidence` tag (why it matched) and a `create_safety` hint (`exists` / `probable` / `unknown`) so an agent decides whether a page already exists instead of guessing from a raw score. `gbrain search diagnose "<query>" --target <slug>` traces which retrieval layer surfaces (or misses) a page.
**Self-wiring knowledge graph.** Every `put_page` extracts entity refs from markdown/wikilinks/typed-link syntax and writes edges with zero LLM calls. Typed edges (`attended`, `works_at`, `invested_in`, `founded`, `advises`, `mentions`, …). Multi-hop traversal via `gbrain graph-query`. The graph is what produces the +31.4 P@5 lift over vector-only RAG.
**Self-wiring knowledge graph.** Every `put_page` extracts entity refs from markdown/wikilinks/typed-link syntax and writes edges with zero LLM calls. Typed edges (`attended`, `works_at`, `invested_in`, `founded`, `advises`, `mentions`, …). Multi-hop traversal via `gbrain graph-query`. The graph is what produces the +31.4 P@5 lift over vector-only RAG.**Obsidian-style vaults:** bare `[[note-name]]` wikilinks that point across folders — you wrote `[[struktura]]` but the page lives at `projects/struktura.md` — resolve by basename once you opt in with `gbrain config set link_resolution.global_basename true`. Off by default; `gbrain doctor` tells you how many edges you'd gain before you flip it. See [migrating an Obsidian vault](INSTALL_FOR_AGENTS.md#step-45-wire-the-knowledge-graph).
**Job queue (Minions).** BullMQ-shaped, Postgres-native job queue. Durable subagents (LLM tool loops that survive crashes via two-phase pending→done persistence), shell jobs with audit, child jobs with cascading timeouts, rate leases for outbound providers, attachments via S3/Supabase storage. Replaces "spawn subagent as fire-and-forget Promise" with something that recovers from anything.
**Non-English brains (FTS language config).** The Postgres full-text search tokenizer is configurable via `GBRAIN_FTS_LANGUAGE`. Defaults to `english`. Set it to any text-search configuration that exists in your Postgres instance:
List available configs: `psql -c "SELECT cfgname FROM pg_ts_config"`. Both the **query side** (`websearch_to_tsquery`) and the **write side** (the trigger functions that populate `pages.search_vector` and `content_chunks.search_vector`) honor `GBRAIN_FTS_LANGUAGE`. On first install (or upgrade), the `configurable_fts_language` schema migration reads the env var and creates trigger functions in the configured language; subsequent inserts/updates tokenize using that setting. To change language on a brain that has already run the migration, use the dedicated CLI command:
The command is idempotent (re-running with the same language is a no-op for vector content) and uses the same recreate-and-backfill primitives as the migration. For accent-insensitive Portuguese (`pt_br`), see [docs/guides/multi-language-fts.md](docs/guides/multi-language-fts.md) for the `unaccent` + portuguese stemmer recipe.
**43 curated skills.** Routing lives in [`skills/RESOLVER.md`](skills/RESOLVER.md). Covers signal capture, ingest (idea / media / meeting), enrichment, querying, brain ops, citation fixing, daily task management, cron scheduling, reports, voice, soul audit, skill creation, eval framework, and migrations. Skills are markdown files (tool-agnostic), packaged as a single skillpack the installer drops into your agent workspace.
**Eval framework.**`gbrain eval longmemeval` runs the public [LongMemEval](https://huggingface.co/datasets/xiaowu0162/longmemeval) benchmark against your hybrid retrieval. `gbrain eval export` + `gbrain eval replay` capture real queries and replay them against code changes (set `GBRAIN_CONTRIBUTOR_MODE=1`). `gbrain eval cross-modal` cross-checks an output against the task using three different-provider frontier models. `gbrain eval retrieval-quality` runs NamedThingBench, which hard-gates the named-thing retrieval families (title-substring, alias-synonym, generic-to-named, multi-chunk-dilution) so a regression in "find the page this query names" fails CI loudly. Full methodology in [`docs/eval/SEARCH_MODE_METHODOLOGY.md`](docs/eval/SEARCH_MODE_METHODOLOGY.md).
@@ -289,6 +307,8 @@ Data flowing into the brain. Each integration is a recipe — markdown + setup h
## Troubleshooting
**`gbrain init --pglite` crashes on macOS 26.x (Tahoe)?** PGLite's embedded WASM engine is incompatible with macOS 26.x on Apple Silicon. The fix is to use native Homebrew PostgreSQL + pgvector instead. Full step-by-step setup in [`docs/INSTALL.md` — Troubleshooting: PGLite crashes on macOS 26.x](docs/INSTALL.md#pglite-crashes-on-macos-26x-tahoe).
**`gbrain import` fails with `expected N dimensions, not M`?** Run `gbrain doctor`. It will print the exact `gbrain config set ...` or `gbrain retrieval-upgrade` command to repair the mismatch. You should not need to delete `~/.gbrain`. Fresh `gbrain init --pglite` auto-detects your embedding provider from API keys in your environment: set `OPENAI_API_KEY` (or `ZEROENTROPY_API_KEY` / `VOYAGE_API_KEY`) before running init, or pass `--embedding-model <provider>:<model>` explicitly. With multiple keys set, init fires an interactive picker. In non-TTY contexts (CI, Docker) with no keys, init exits 1 with a paste-ready setup hint; pass `--no-embedding` to defer setup until runtime. See [`docs/integrations/embedding-providers.md`](docs/integrations/embedding-providers.md) for the full provider matrix and [`docs/operations/headless-install.md`](docs/operations/headless-install.md) for Docker/CI sequencing.
**Hourly cron sync keeps timing out on a federated brain?** v0.41.13.0 ships
**Slug-based API, not ID-based.** Every method takes slugs, not numeric IDs. The engine resolves slugs to IDs internally. This keeps the interface portable... slugs are strings, IDs are database-specific.
**Embedding is NOT in the engine.** The engine stores embeddings and searches by vector, but it doesn't generate embeddings. `src/core/embedding.ts` handles that. This is intentional: embedding is an external API call (OpenAI), not a storage concern. All engines share the same embedding service.
**Embedding is NOT in the engine.** The engine stores embeddings and searches by vector, but it doesn't generate embeddings. `src/core/embedding.ts` handles that (a thin delegation to the provider-agnostic AI gateway in `src/core/ai/gateway.ts`). This is intentional: embedding is an external API call (OpenAI, Voyage, a local Ollama — whichever provider you configured), not a storage concern. All engines share the same embedding service.
**Chunking is NOT in the engine.** Same logic. `src/core/chunkers/` handles chunking. The engine stores and retrieves chunks. All engines share the same chunkers.
@@ -148,6 +148,51 @@ RRF fusion, multi-query expansion, and 4-layer dedup are engine-agnostic. They o
**Why not self-hosted for v0:** The brain should be infrastructure agents use, not something you maintain. Self-hosted Postgres with Docker is a welcome community PR, but v0 optimizes for zero ops.
@@ -111,3 +111,38 @@ gbrain models doctor # 1-token probe per configured model
```
If anything's yellow, `gbrain doctor` names the fix command in the message. Most issues are missing API keys or stale schema (`gbrain upgrade --force-schema`).
## Troubleshooting
### PGLite crashes on macOS 26.x (Tahoe)
PGLite's embedded WASM engine is incompatible with macOS 26.x (Tahoe) on Apple Silicon. If `gbrain init --pglite` crashes during engine initialization, switch to native Homebrew PostgreSQL:
```bash
# Install PostgreSQL + pgvector
brew install postgresql@17
brew services start postgresql@17
createdb gbrain
# Build pgvector from source (required for vector search)
cd /tmp && git clone --branch v0.8.0 https://github.com/pgvector/pgvector.git
cd pgvector && make && make install
psql gbrain -c "CREATE EXTENSION IF NOT EXISTS vector;"
All 102 migrations run on first try. Once `gbrain doctor` shows green, the brain works identically to PGLite — same commands, same skills, same data model. The only difference is the storage backend.
> **Note:** This workaround is temporary. When the upstream WASM runtime fix ships (likely via a Bun update), `--pglite` will work on Tahoe again.
- Good: "Your agent now verifies the entire GBrain installation end-to-end, catching
silent sync failures and stale embeddings before they bite you"
- Bad: "Setup skill Phase H and Phase I added"
- Good: "New installs automatically set up live sync so your brain never falls behind"
- **Always credit community contributions.** When a CHANGELOG entry includes work from
a community PR, name the contributor with `Contributed by @username`. Contributors
did real work. Thank them publicly every time, no exceptions.
### Reference: v0.12.0 entry as canonical example
The v0.12.0 entry in CHANGELOG.md is the canonical example of the format. Match its
structure for every future version: bold headline, lead paragraph, "numbers that
matter" with BrainBench-style before/after table, "what this means" closer, then
`### Itemized changes` with the detailed sections below.
## Version migrations
Create a migration file at `skills/migrations/v[version].md` when a release
includes changes that existing users need to act on. The auto-update agent
reads these files post-upgrade (Section 17, Step 4) and executes them.
**You need a migration file when:**
- New setup step that existing installs don't have (e.g., v0.5.0 added live sync,
existing users need to set it up, not just new installs)
- New SKILLPACK section with a MUST ADD setup requirement
- Schema changes that require `gbrain init` or manual SQL
- Changed defaults that affect existing behavior
- Deprecated commands or flags that need replacement
- New verification steps that should run on existing installs
- New cron jobs or background processes that should be registered
**You do NOT need a migration file when:**
- Bug fixes with no behavior changes
- Documentation-only improvements (the agent re-reads docs automatically)
- New optional features that don't affect existing setups
- Performance improvements that are transparent
**The key test:** if an existing user upgrades and does nothing else, will their
brain work worse than before? If yes, migration file. If no, skip it.
Write migration files as agent instructions, not technical notes. Tell the agent
what to do, step by step, with exact commands. See `skills/migrations/v0.5.0.md`
for the pattern.
## Migration is canonical, not advisory
GBrain's job is to deliver a canonical, working setup to every user on upgrade.
Anything that looks like a "host-repo change" — AGENTS.md, cron manifests,
launchctl units, config files outside `~/.gbrain/` — is a GBrain migration
step, not a nudge we leave for the host-repo maintainer. Migrations edit host
files (with backups) to make the canonical setup real. Exceptions: changes
that require human judgment (content edits, renames that break semantics,
host-specific handler registration where shell-exec would be an RCE surface).
Everything mechanical ships in the migration.
**Test:** if shipping a feature requires a sentence that starts with "in
your AGENTS.md, add…" or "in your cron/jobs.json, rewrite…", the migration
orchestrator should be doing that edit, not the user.
**The exception is host-specific code.** For custom Minion handlers
(host-specific integrations like inbox sweeps or third-party API scanners), shipping them as a
data file the worker would exec is an RCE surface. Those get registered in
the host's own repo via the plugin contract (`docs/guides/plugin-handlers.md`);
the migration orchestrator emits a structured TODO to
`~/.gbrain/migrations/pending-host-work.jsonl` + the host agent walks the
TODOs using `skills/migrations/v0.11.0.md` — stays host-agnostic, still
canonical.
## Schema state tracking
`~/.gbrain/update-state.json` tracks which recommended schema directories the user
adopted, declined, or added custom. The auto-update agent (SKILLPACK Section 17)
reads this during upgrades to suggest new schema additions without re-suggesting
things the user already declined. The setup skill writes the initial state during
Phase C/E. Never modify a user's custom directories or re-suggest declined ones.
## GitHub Actions SHA maintenance
All GitHub Actions in `.github/workflows/` are pinned to commit SHAs. Before shipping
(`/ship`) or reviewing (`/review`), check for stale pins and update them:
```bash
for action in actions/checkout oven-sh/setup-bun actions/upload-artifact actions/download-artifact softprops/action-gh-release gitleaks/gitleaks-action;do
Seven test command tiers, each with a clear scope:
| Command | What it runs | Wallclock | When to use |
|---|---|---|---|
| `bun run test` | Parallel unit-test fast loop. 8-shard fan-out via `scripts/run-unit-parallel.sh`, then a serial pass over `*.serial.test.ts`. Excludes `*.slow.test.ts` and `test/e2e/*`. No pre-checks, no typecheck. | ~85s on a Mac dev box (3650+ tests) | Inner edit loop. Default. |
| `bun run verify` | CI's authoritative pre-test gate set, fanned out in parallel by `scripts/run-verify-parallel.sh`: the full `check:*` battery (~30 checks — privacy, jsonb, progress, source-id, test-isolation, wasm, …) plus `bun run typecheck`. The `CHECKS` array in that script is the single source of truth — CI literally calls `bun run verify` in a dedicated job. | ~16s (parallel; typecheck dominates) | Before pushing; before `/ship`. |
| `bun run test:full` | `verify && bun run test && bun run test:slow && [smart e2e]`. The local equivalent of "everything CI runs." Smart e2e: runs e2e only when `DATABASE_URL` is set; else loud skip notice to stderr. | ~3-5min depending on slow + e2e | Pre-merge sanity, before opening a PR. |
| `bun run test:slow` | Just the `*.slow.test.ts` set (intentional cold-path correctness checks). | seconds-to-minutes | When touching slow-path code. |
| `bun run test:serial` | Just the `*.serial.test.ts` set (cross-file-contention quarantine; one bun process per file for true module-registry isolation). | ~1s per quarantined file | Debugging a specific quarantined file. |
| `bun run test:e2e` | Real Postgres E2E. Requires Docker + `DATABASE_URL`. Sequential. | ~5-10min | Pre-ship; nightly. |
| `bun run check:all` | The historical pre-check scripts (22, chained sequentially in package.json). Overlaps `verify` heavily but is NOT a superset — `verify`'s `CHECKS` array in `scripts/run-verify-parallel.sh` (~30 entries incl. typecheck) is the authoritative gate; `check:all` keeps a few local-only extras (trailing-newline, exports-count, no-legacy-getconnection). | ~10s | Local-only sweep for the extras. |
### CI vs local: intentionally divergent file sets
- **CI matrix** (`.github/workflows/test.yml`) runs `scripts/test-shard.sh` across 10 matrix shards partitioned by weight-aware LPT bin-packing (`scripts/sharding.ts`) and INCLUDES `*.slow.test.ts` (the two outlier slow files run as dedicated jobs alongside the matrix). CI EXCLUDES `*.serial.test.ts` from the shards and runs them in a dedicated job via `bun run test:serial`, one bun process per file — keeping serial files out of the shard processes is what preserves the `mock.module` quarantine (a top-level mock in one file leaks into every other file sharing its process). `bun run verify` gets its own job too. CI is the ground truth for "did everything pass."
- **Local fast loop** (`scripts/run-unit-shard.sh` via the parallel wrapper) uses round-robin-by-index sharding and EXCLUDES `*.slow.test.ts` AND `*.serial.test.ts`. Local trades coverage for inner-loop speed; CI catches what local skips.
This divergence is intentional. Don't try to make them equal — the two scripts deliberately solve different problems. The regression test at `test/scripts/run-unit-shard.test.ts` pins what the local fast loop should and shouldn't include.
### Failure-first logging
When `bun run test` finds any failure, the wrapper:
1. Writes failure blocks (each prefixed with `--- shard N: <test name> ---`) to `.context/test-failures.log` (workspace-local, gitignored). On systems without a writable `.context/`, falls back to `/tmp/gbrain-test-failures.log`.
2. Prints a loud stderr banner with the absolute log path, plus the last 30 lines of the failure log inlined. Banner survives `| head` / `| tail` / agent-side log truncation.
3. Writes a one-line-per-shard summary to `.context/test-summary.txt` (`shard N/M: pass=X fail=Y skip=Z rc=W`).
4. Exits non-zero. Empty failure log + non-zero exit = infrastructure problem (wedged shard, killed child); the banner says so.
If a shard wedges (per-shard `GBRAIN_TEST_SHARD_TIMEOUT` cap, default 600s), the wrapper writes `--- shard N: WEDGED after ${SHARD_TIMEOUT}s ---` to the failure log, includes the last 50 lines of the shard log, and proceeds with other shards' results.
### File taxonomy
- `*.test.ts` → fast loop (parallel 8-shard fan-out).
- `*.slow.test.ts` → run via `bun run test:slow` only (intentional cold-path tests; would dominate the fast loop's wallclock).
- `*.serial.test.ts` → run via `bun run test:serial` after the parallel pass completes; one bun process per file (`--max-concurrency=1` within a shared process is not enough — the module registry still leaks `mock.module`). Quarantine for tests that share file-wide state and race when run alongside other files in the same `bun test` process. Several dozen files, discovered by the `*.serial.test.ts` glob — no list to maintain. Typical residents: `mock.module(...)` users (top-level mocks leak across files in a shard process, e.g. `test/embed.serial.test.ts`), env-coupled files (e.g. `test/brain-registry.serial.test.ts`), and process-lifecycle suites that assert on `process.exitCode` (e.g. `test/pglite-engine-disconnect.serial.test.ts`). **Do not put the parallelism back on a serial file unless you've fixed the contention root cause** (it just re-introduces the flake).
- `test/e2e/*.test.ts` → real-Postgres E2E. Skipped when `DATABASE_URL` is unset.
- `tests/heavy/*.sh` → ops-shape shell scripts. Cost minutes per run; NOT in default `bun test`. Run via `bun run test:heavy` or scheduled nightly via `.github/workflows/heavy-tests.yml`. Examples: pg_upgrade matrix (boot legacy brain → walk to head), RSS budget gate (measure peak worker RSS vs committed baseline), read-latency-under-sync (p50/p95/p99 under concurrent writer load), sync lock regression (N concurrent syncs assert 1 winner + N-1 lock-busy + zero leaked `gbrain_cycle_locks` rows). See `tests/heavy/README.md` for when to add a script here vs `*.slow.test.ts`. Files prefixed with `_` (e.g. `tests/heavy/_build_legacy_fixtures.sh`) are helpers/libs invoked by sibling tests — the runner skips them.
- `test/fuzz/*.test.ts` → property-based fuzz harness. Pure-validator targets in `pure-validators.test.ts` are guarded by `scripts/check-fuzz-purity.sh` (in `bun run verify`), which `bun build --target=bun` bundles each target and greps the resulting bundle for banned transitive imports (`node:fs`, `node:child_process`, engine modules). Anything that fails the guard moves to `mixed-validators.test.ts` (still property-tested, but no purity guarantee) or `filesystem-validators.test.ts` (fs-backed, uses temp dirs). Fuzz tests run in the default `bun test` loop because they're fast (~3s for ~12 properties × 1000 runs each).
### Test-isolation lint and helpers
The cross-file flake class is enforced statically by `scripts/check-test-isolation.sh`, wired into `bun run verify` and `bun run check:all`. Rules (non-serial unit files only; `*.serial.test.ts` and `test/e2e/*` are skipped):
| Rule | What it bans | Fix |
|---|---|---|
| **R1** | `process.env.X = ...`, bracket assignment, `delete process.env.X`, `Object.assign(process.env, ...)`, `Reflect.set(process.env, ...)` | Use `withEnv()` from `test/helpers/with-env.ts`, OR rename file to `*.serial.test.ts` |
| **R2** | `mock.module(...)` anywhere in the file | Rename file to `*.serial.test.ts` (no DI on production code for testability) |
| **R3** | `new PGLiteEngine(` outside ~50 lines after a `beforeAll(` line | Use the canonical block (below) inside `beforeAll(` |
| **R4** | Files creating `new PGLiteEngine(` without `engine.disconnect(` inside an `afterAll(` block | Add `afterAll(() => engine.disconnect())` |
Files that violated these rules at the isolation-lint baseline are listed in `scripts/check-test-isolation.allowlist`. **The allow-list MUST shrink over time** — never add new entries.
#### Canonical PGLite block (R3 + R4 compliant)
Every test file that needs a PGLite engine should use this exact pattern:
```ts
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { resetPgliteState } from './helpers/reset-pglite.ts';
let engine: PGLiteEngine;
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
});
afterAll(async () => {
await engine.disconnect();
});
beforeEach(async () => {
await resetPgliteState(engine);
});
```
Why this exact shape: `beforeAll` creates a single engine per file (PGLite WASM cold-start + initSchema is ~20s); `beforeEach` truncates user data via `resetPgliteState` ("two orders of magnitude faster" than fresh-engine-per-test); `afterAll` disconnects so the engine doesn't leak across file boundaries within a shard process.
`withEnv` saves the prior value of every key it touches and restores via try/finally — including when the callback throws. **It is cross-test safe but NOT intra-file concurrent-safe.**`process.env` is process-global; two `test.concurrent()` calls in the same file both touching the same key will race. Files using `withEnv` stay outside the `test.concurrent()` codemod's eligibility filter.
#### When to quarantine instead of fix
Rename to `*.serial.test.ts` when:
- The file uses `mock.module(...)` (R2 — there's no clean fix without changing production code).
- The file is genuinely env-coupled (e.g. `gbrain-home-isolation.test.ts`, `claw-test-cli.test.ts`) — module-load env readers + ESM caching defeat dynamic-import-after-env tricks.
- The file's tests intentionally share state across `it()` boundaries.
The quarantine has grown to dozens of files — treat it as debt: every addition needs a reason from the list above, and prefer fixing the contention root cause when one exists.
### Unit test inventory
`bun test` runs all tests without a database. E2E tests skip gracefully when `DATABASE_URL` is not set.
- `test/cli-finish-teardown.test.ts` — the #2084 teardown contract: `computeTeardownDeadlineMs` formula/floor/live-registry scaling + `GBRAIN_TEARDOWN_DEADLINE_MS` override (garbage/zero/negative values fall back to the formula); `finishCliTeardown` clean path (drain BEFORE disconnect, no exit, no warn), backstop on hung drain or disconnect (honors an errored op's exit code), throwing drain/disconnect warned + swallowed; the gbrain-owned verdict channel is immune to PGLite WASM `process.exitCode` writes; `flushThenExit` unit coverage with mocked streams (exits once after both stream callbacks, non-TTY aliveness grace, blocked-pipe guard, EPIPE-safe, `GBRAIN_FLUSH_GRACE_MS` override).
- `test/flush-then-exit-harness.test.ts` — real spawned-Bun pipe semantics for `flushThenExit` (fixture: `test/fixtures/flush-then-exit-harness.ts`): a 4MB piped stdout payload arrives byte-complete with the exit code even with a late reader, small output survives exit with a concurrent reader, and the fence resolves promptly (wall time well under the guard + grace ceiling).
- `test/cli-should-force-exit.test.ts` — `shouldForceExitAfterMain` daemon-survival gate: `serve` (stdio and `--http`) never force-exits, including with preceding global flags; op commands / empty / flag-only argv do; the #2084 case that space-separated global-flag VALUES can't fake a command (`--timeout 30s serve` resolves to the `serve` daemon, not a `30s` command).
- `test/cli-exit-verdict-pin.test.ts` — #2084 structural class pin: greps `src/` so the NEXT raw `process.exitCode =` write fails CI (a raw write bypasses the gbrain-owned verdict channel and gets silently zeroed by the deliberate flush-exit — the bug that made doctor's FAIL path exit 0). Runtime variants live in `test/cli-finish-teardown.test.ts`; this is the review-time guard.
- `test/cli-pipe-truncation.test.ts` — real-CLI pipe completeness (the #1959 incident class), implementation-agnostic: the actual CLI run the way agents run it (piped stdout) produces complete, parseable, byte-stable `--tools-json` output and exits deliberately, well under the teardown backstop. Synthetic flush-mechanism coverage stays in `test/flush-then-exit-harness.test.ts`.
- `test/watch-command.test.ts` — `gbrain watch` push transport (#2095): streaming loop, rolling window, session dedupe, `--json` JSONL shape, `channel: 'watch'` event logging, clean EOF return. Hermetic PGLite + injected line/write deps (no subprocess, no real stdin).
- `test/watch-sigint.serial.test.ts` — `gbrain watch` SIGINT lifecycle against a real spawned CLI subprocess with a tmpdir brain. SERIAL: parallel unit shards flake on concurrent subprocess spawns (same rationale as `apply-migrations-pglite-spawn.serial.test.ts`).
- `test/cli-format-volunteer.test.ts` — `formatResult`'s `volunteer_context` human rendering: pointer lines with confidence/arm/rationale, the empty-result message, the approximate stats summary.
- `test/bootstrap.test.ts` — bootstrap contract: no-op on fresh install, idempotent across two `initSchema()` calls, no-op on modern brain that already has every probed column, full bootstrap path on a simulated legacy brain, fresh-install regression guard, legacy `links` shape coverage.
- `test/schema-bootstrap-coverage.test.ts` — CI guard. `REQUIRED_BOOTSTRAP_COVERAGE` lists every forward reference in `PGLITE_SCHEMA_SQL`; the test fails loudly if `applyForwardReferenceBootstrap` skips one (extend both arrays when adding a column-with-index to the embedded schema blob). Also parses `src/core/migrate.ts` source text for every `ALTER TABLE ... ADD COLUMN` (top-level `sql:`, `sqlFor.{postgres,pglite}` overrides, AND handler-body `engine.runMigration(N, \`ALTER TABLE ...\`)`) and asserts each (table, column) pair is covered by the bootstrap OR by the schema blob's CREATE TABLE bodies — catching the column-only forward-reference class (e.g. `sources.archived`, `oauth_clients.source_id`) that a CREATE INDEX parser alone can't see. `parseBaseTableColumns` strips SQL line + block comments before identifying column names so commented-out lines don't hide adjacent columns.
- `test/helpers/schema-diff.ts` + `test/helpers/schema-diff.test.ts` + `test/e2e/schema-drift.test.ts` — cross-engine schema parity gate. Helper exports pure `snapshotSchema(query)` / `diffSnapshots(pg, pglite, opts)` / `formatDiffForFailure(diff)` / `isCleanDiff(diff)` over a four-tuple per column (`data_type`, `udt_name`, `is_nullable`, `column_default`). E2E test spins up fresh PGLite + Postgres, runs `engine.initSchema()` on each, snapshots `information_schema.columns`, then diffs. 2-table allowlist (`files`, `file_migration_ledger`) — every other Postgres table must reach PGLite via `PGLITE_SCHEMA_SQL` or a migration's `sqlFor.pglite` branch. Sentinels for `oauth_clients`, `mcp_request_log`, `access_tokens`, `eval_candidates` give tighter blame messages. Skips without `DATABASE_URL`. Wired into `scripts/e2e-test-map.ts` so changes to `src/schema.sql`, `src/core/pglite-schema.ts`, or `src/core/migrate.ts` trigger it. The failure message names every drift with a paste-ready hint pointing at `src/core/pglite-schema.ts`.
- `test/check-update.test.ts` — version check + update CLI.
- `test/pglite-engine.test.ts` — PGLite engine, all BrainEngine methods including `addLinksBatch` / `addTimelineEntriesBatch` (empty batch, missing optionals, within-batch dedup via ON CONFLICT, missing-slug rows dropped by JOIN, half-existing batch, batch of 100) plus `connect()` error-wrap assertion (original error nested, #223 link in message, lock released).
- `test/links-timeline-jsonb-poison.test.ts` — gbrain#1861 PGLite half (always-on, no `DATABASE_URL`). Locks the `jsonb_to_recordset` batch-insert path for links/timeline/takes against free-text "poison" payloads (commas, quotes, backslashes, braces, em-dashes) and asserts NUL is stripped from free-text body fields but rejected in identity fields. gbrain#2011 adds lone-UTF-16-surrogate cases: every free-text field (link context; timeline summary/detail/source; take claim/source) well-forms to U+FFFD across batch + scalar write paths, while a surrogate in an identity field (slug) still fail-closed rejects the batch. The Postgres lane (`test/e2e/jsonb-batch-poison-postgres.test.ts`) is the one that actually reproduced the original crash.
- `test/resolver.test.ts` — RESOLVER.md coverage, routing validation; round-trip that every quoted RESOLVER.md trigger matches a frontmatter `triggers:` entry in the target skill, and every `name="<word>"` reference in any SKILL.md resolves to a declared op in `src/core/operations.ts` or a Minions handler in `PROTECTED_JOB_NAMES`.
- `test/sync-parallel.test.ts` — PGLite-routed coverage of the bookmark gate under concurrency, head-drift gate, vanished-file failure capture, PGLite-stays-serial, and the `gbrain-sync` writer-lock contract.
- `test/sync-failures.test.ts` — `classifyErrorCode` regex coverage for all 12 codes against literal production message strings from `markdown.ts` and `import-file.ts`; `summarizeFailuresByCode` sort + pre-classified-honor; `recordSyncFailures` code-field persistence; `acknowledgeSyncFailures``AcknowledgeResult` shape + backfill on legacy entries.
- `test/doctor.test.ts` — doctor command; assertions that `jsonb_integrity` scans the four JSONB write sites and `markdown_body_completeness` is present.
- `test/oauth.test.ts` — OAuth 2.1 provider: register, getClient, `client_credentials` grant exchange, `authorization_code` flow with PKCE challenge/verifier, refresh token rotation, `verifyAccessToken` with both OAuth + legacy `access_tokens` fallback, `revokeToken`, `sweepExpiredTokens`; contract test asserting `scope` + `localOnly` annotations on all operations; `coerceTimestamp` unit cases (null/undefined/string/number/throw-on-NaN); NULL-`expires_at`-as-expired contract for both refresh + access token paths; cascade-delete contract asserting `revoke-client` purges `oauth_tokens` + `oauth_codes` via FK CASCADE; cross-client isolation (wrong-client attempt MUST reject AND rightful owner MUST still succeed atomically afterward); empty-string `redirect_uri` bypass guard; PKCE DCR public-client gate (`token_endpoint_auth_method: "none"` returns no `client_secret`, default `client_secret_post` clients get the one-time-reveal secret, `getClient` NULL→undefined normalization, full PKCE `/authorize` → `/token` round-trip against a public client).
- `test/mcp-dispatch-summarize.test.ts` — `summarizeMcpParams` invariants: declared-keys allow-list intersection, attacker-key-name leak guard (unknown keys counted not named), 1KB byte bucketing for size-probe defense, missing op falls through to fully-redacted shape, declared-keys sorted for deterministic output.
- `test/trust-boundary-contract.test.ts` — fail-closed trust semantics under cast bypass: `ctx.remote === undefined` treated as remote/untrusted at every flipped call site; `as any` and `Partial<>` spreads can't downgrade trust by accident.
- `test/repo-root.test.ts` — `findRepoRoot` walk semantics + default-arg parity; the 4-tier `autoDetectSkillsDir` fallback chain (`$OPENCLAW_WORKSPACE` → `~/.openclaw/workspace` → repo-root → `./skills`); RESOLVER.md/AGENTS.md filename precedence; explicit-env-wins-over-repo-root; tier-0 `$GBRAIN_SKILLS_DIR` valid/invalid/precedence-over-`OPENCLAW_WORKSPACE`; the install-path walk in `autoDetectSkillsDirReadOnly`; no-drift on primary success; `AUTO_DETECT_HINT` + `AUTO_DETECT_HINT_READ_ONLY` content; regression guard asserting the shared `autoDetectSkillsDir` MUST NEVER return `'install_path'` source (how the read-path/write-path split stays safe).
- `test/resolver-merge.test.ts` — multi-file resolver merge: `findAllResolverFiles` empty / RESOLVER.md-only / AGENTS.md-only / both-present (RESOLVER.md first); `checkResolvable` merge semantics across `skills/RESOLVER.md` + `../AGENTS.md` for the OpenClaw layout where the skillpack ships a thin RESOLVER.md and the real dispatcher lives at the workspace root; dedup by `skillPath` (first occurrence wins); AGENTS.md-at-workspace-root works alone.
- `test/http-transport.test.ts` — HTTP transport: bearer auth + missing/no-Bearer/unknown/revoked + `/health` bypass; dispatch.ts round-trip; invalid_params; application/json response shape (not SSE); CORS default-deny + allowlist; body cap on Content-Length AND chunked; two-bucket rate limit (refill, exhaust+Retry-After, LRU eviction, TTL prune, pre-auth IP fires before DB); `mcp_request_log` audit on success + auth_failed.
- `test/restart-sweep.test.ts` — `recipes/restart-sweep.md` inlined script: sentinel-anchored fenced-block extraction with salted tmp filenames to bypass ESM cache; constructor-time env reads (proves no module-load snapshot); idempotency layer load/save/atomic-tmp-rename/corrupt-JSON-recovery/30-day-prune; `(sessionKey, lastAlertedAt)` cooldown gate with 6h threshold; AGGRESSIVE-gate two-state tests; execFile argv shape proving shell metachars in `OPENCLAW_TELEGRAM_GROUP` cannot reach `/bin/sh`; real-`\n`-not-literal alert formatting; `GBRAIN_HOME` state path override.
- `test/eval-longmemeval.test.ts` — LongMemEval harness, hermetic with no `DATABASE_URL` and no API keys: PGLite create + reset over runtime-enumerated `pg_tables`, infrastructure-table preservation across resets, JSONL question parsing, retrieval-only and answer-gen modes via stubbed `ThinkLLMClient`, `--limit` cutoff, `--keyword-only` vs hybrid, default `--expansion=off` behavior, perf gate (p50 < 30ms / p99 < 50ms warm reset+import+search on Apple Silicon), `--help` works without a configured brain, fixture round-trip via `test/fixtures/longmemeval-mini.jsonl`.
- `test/longmemeval-sanitize.test.ts` — sanitization parity pinning that `INJECTION_PATTERNS` from `src/core/think/sanitize.ts` is the single source of truth (adding a pattern there must cover both `<take>` framing and `<chat_session>` framing, no per-surface regex drift).
- `test/openai-compat-multimodal.test.ts` — gateway's openai-compatible multimodal path: happy-path single + multi-input embedding, unauthenticated proxy mode, dimension-mismatch guard (throws `AIConfigError` with model id + observed + expected pre-storage), default-dim fallback when recipe declares `default_dims`, HTTP 401 / 400 / malformed-JSON / non-array error paths, regression that the existing Voyage `/multimodalembeddings` recipe still routes through its dedicated path. Hermetic via the `__setEmbedTransportForTests` seam.
- `test/serve-stdio-lifecycle.test.ts` — `MCP_STDIO=1` env guard: stdin EOF does NOT trigger shutdown when the env is set, SIGTERM still does (guard scope is correct), unset env preserves the CLI lifecycle. Exercises the `ServeOptions.mcpStdio?: boolean` test seam directly so tests don't mutate `process.env`.
### E2E test inventory
E2E tests live in `test/e2e/` and run against real Postgres+pgvector (require `DATABASE_URL`), except where noted as PGLite in-memory (no `DATABASE_URL` needed).
- `bun run test:e2e` runs Tier 1 (mechanical, all operations, no API keys). Includes dedicated cases for the postgres-engine `addLinksBatch` / `addTimelineEntriesBatch` bind path — postgres-js's JSONB bind (`jsonb_to_recordset(($1::jsonb)->'rows')`) differs from PGLite's and gets its own coverage.
- `test/e2e/search-quality.test.ts` — search quality against PGLite (no API keys, in-memory).
- `test/e2e/graph-quality.test.ts` — knowledge graph pipeline (auto-link via put_page, reconciliation, traversePaths) against PGLite in-memory.
- `test/e2e/jsonb-batch-poison-postgres.test.ts` — gbrain#1861 regression, the engine that actually crashed. Seeds free-text "poison" context (Zoom URL with `?pwd=`, commas, quotes, Windows backslash path, braces, em-dash) and asserts the links/timeline/takes batch writers no longer error with "malformed array literal"; also asserts NUL is stripped from free-text bodies (`context`/`summary`/`detail`/`claim`) and still rejected in identity fields. gbrain#2011 adds the lone-surrogate crash lock: a lone UTF-16 surrogate in free text (the value that aborted `extract --stale` with `22P02` on Supabase) well-forms to U+FFFD across batch + scalar paths (incl. timeline + take `source`), while a surrogate in an identity field still rejects the batch. `DATABASE_URL`-gated.
- `test/e2e/postgres-jsonb.test.ts` — round-trips all 5 JSONB write sites (`pages.frontmatter`, `raw_data.data`, `ingest_log.pages_updated`, `files.metadata`, `page_versions.frontmatter`) against real Postgres and asserts `jsonb_typeof='object'` plus `->>'key'` returns the expected scalar. Guards against the double-encode bug.
- `test/e2e/integrity-batch.test.ts` — parity for `scanIntegrity`'s batch-load fast path vs sequential. Cases (dedup, hits, validate, topPages) seed a fixture and assert both paths return identical results. Dedup case uses raw SQL via `getConn().unsafe()` to seed a `(test-source-2, people/alice)` row alongside the default-source row, since `engine.putPage` doesn't take a `source_id`. Pins multi-source overcounting; the "multi-source duplicate slugs scan once" case expects both batch + sequential paths to report 2.
- `test/e2e/jsonb-roundtrip.test.ts` — companion regression against the 4 doctor-scanned JSONB sites. Assertion-level overlap with `postgres-jsonb.test.ts` is intentional defense-in-depth: if doctor's scan surface drifts from the actual write surface, one of these tests catches it.
- `test/e2e/sync.test.ts` — `--skip-failed` failure-loop test alongside happy-path tests: broken file → `performSync` returns `blocked_by_failures` with grouped breakdown → `performSync({skipFailed: true})` advances bookmark and returns `AcknowledgeResult` with code summary → second broken file → second cycle. Saves and restores the user's real `~/.gbrain/sync-failures.jsonl` so the test is hermetic. Asserts bookmark gating, JSONL state, dedup across paths, summary aggregation, and the literal doctor-rendering string format.
- `test/e2e/upgrade.test.ts` — check-update against real GitHub API (network required).
- `test/e2e/minions-shell-pglite.test.ts` — PGLite `--follow` inline shell-job path (in-memory, no `DATABASE_URL` required) — the path the minion-orchestrator skill documents for dev use.
- `test/e2e/pglite-cli-exit.serial.test.ts` — real spawned-CLI exit behavior on PGLite (in-memory, no `DATABASE_URL`): read commands (`search`/`get`/`query`) exit 0 promptly; CLI_ONLY `capture` exits clean and frees the single-writer lock; the `#2084` describes pin every swept disconnect site — a failed op exits 1 with the error on stderr, and the dashboard, read-only-timeout, doctor, and `dream --dry-run` paths all exit with no force-exit banner.
- `test/e2e/pgbouncer-teardown.test.ts` — PgBouncer TRANSACTION-mode teardown (#2084 / the #1972→#2015→#2084 class). Pins the bug CLASS, not timings: a CLI op against a txn-mode pooled URL exits 0 with intact stdout and does NOT ride the 10s hard-deadline backstop (the `engine.disconnect() did not return` banner is the smoking gun — pre-#2084 it printed on 100% of query-shaped ops). Gated by `GBRAIN_PGBOUNCER_URL` + `GBRAIN_PGBOUNCER_DIRECT_URL` (NOT `DATABASE_URL`) — set automatically by `bun run ci:local`'s `pgbouncer` compose service; skips gracefully elsewhere. Uses a DEDICATED `gbrain_pgbouncer` database so it never races the `gbrain_test` TRUNCATE fixtures.
- `test/e2e/volunteer-context-postgres.test.ts` — `volunteer_context` on REAL Postgres (#2095; engine parity beyond the hermetic PGLite unit suite): resolution arms through the actual op handler, the fire-and-forget volunteer-event sink landing rows, the stats join, and the RLS pin that `context_volunteer_events` has ROW LEVEL SECURITY enabled (keeps the v35 auto-RLS event trigger honest for migration-created tables). `DATABASE_URL`-gated.
- `test/e2e/openclaw-reference-compat.test.ts` — `check-resolvable` + `skillpack install` against a minimal AGENTS.md workspace fixture (`test/fixtures/openclaw-reference-minimal/`), regression guard for the OpenClaw deployment shape.
- `test/e2e/search-swamp.test.ts` — reproduces the source-swamp case. Seeds a curated `originals/talks/article-outline-fat-code` page against two `<fork>/chat/` pages stuffed with the same multi-word phrase. Asserts the article wins keyword AND vector ranking, that `detail=high` lets the chat swamp re-surface, and that `source_id` passes through the two-stage CTE intact. PGLite in-memory.
- `test/e2e/search-exclude.test.ts` — `test/` + `archive/` pages hidden by default, `include_slug_prefixes` opts back in, caller-supplied `exclude_slug_prefixes` adds to defaults. Both keyword and vector search paths.
- `test/e2e/engine-parity.test.ts` — Postgres ↔ PGLite top-result and result-set parity for `searchKeyword` + `searchVector` (Postgres ranks pages then picks best chunk while PGLite returns chunks directly, so the source-boost behavior needs parity coverage). Skips without `DATABASE_URL`.
- `test/e2e/postgres-bootstrap.test.ts` — exercises `PostgresEngine.initSchema()` directly against a fresh real Postgres database. Asserts the bootstrap path is no-op on fresh installs and that SCHEMA_SQL replays cleanly through the engine path (not via the standalone `db.initSchema` from `src/core/db.ts`).
- `test/e2e/http-transport.test.ts` — `gbrain serve --http` end-to-end against real Postgres: bearer auth round-trip, `last_used_at` SQL-level debounce, `mcp_request_log` row insertion on success and auth_failed paths, `/health` DB-down → 503 (DB-probing health check), and the dispatch round-trip with a real operation. Skips without `DATABASE_URL`.
- `test/e2e/serve-http-oauth.test.ts` — real-Postgres E2E against `gbrain serve --http` with full OAuth 2.1. Spawns a subprocess server, registers a client via the CLI, mints `client_credentials` tokens, exercises the `/mcp` JSON-RPC pipeline. Real DCR `/register` HTTP-level response-shape test (asserts `typeof body.client_id_issued_at === 'number'` over the wire, RFC 7591 §3.2.1); real CLI subprocess test for `revoke-client` (registers → mints token → revokes via `execSync` → asserts token rejected at `/mcp` → asserts re-run exits 1); server fixture flips on `--enable-dcr` so `/register` is reachable. **bun execSync env-inheritance contract:** bun's `execSync` does NOT inherit env mutations done via `process.env.X = ...`, only OS-level env from before bun started. helpers.ts loads `.env.testing` and sets `DATABASE_URL` via `process.env` mutation, which is invisible to subprocesses unless `env: { ...process.env }` is passed explicitly — every subprocess call in this file passes `env: { ...process.env }`. Reference fix for the same failure mode in sibling sync/cycle/dream/claw-test E2Es. `afterAll` cleanup is guarded on `clientId` (won't throw if `beforeAll` failed before registration); cleanup errors surface to stderr without throwing so real test failures aren't masked. Also covers the trust-boundary fix: an HTTP MCP `submit_job` for `name: "shell"` MUST reject with a permission error (request handler sets `remote: true` and `submit_job`'s protected-name guard fires), and the same guard rejects subagent submission. Skips without `DATABASE_URL`.
- `test/e2e/sync-parallel.test.ts` — `DATABASE_URL`-gated. 60-file Postgres sync at concurrency=4 imports all + no connection leak (probes `pg_stat_activity` before/after to confirm worker engines disconnected). 120-file serial-vs-parallel benchmark prints `SYNC_PARALLEL_BENCH N files | serial=Xms | parallel(4)=Yms | speedup=Zx`. Asserts parallel ≤ serial × 1.5 (CI-noise tolerant; not a strict speedup gate).
- `test/e2e/multi-source-bug-class.test.ts` — PGLite in-memory regression suite pinning every multi-source bug site: `listAllPageRefs` ordering by `(source_id, slug)`, `getPage` with sourceId picks the right `(source, slug)` row, `extract-takes` processes both overlapping `people/alice` rows independently, `listPages` filters correctly with `PageFilters.sourceId`, `addLinksBatch` with `from/to_source_id` targets the right rows, `validateSourceId` rejects path traversal, reverse-write disk layout uses `brainDir/.sources/<id>/<slug>.md` for non-default sources, `copyMigrationSources` lands source metadata before overlapping-slug pages. No `DATABASE_URL` needed. Wired into `scripts/e2e-test-map.ts` so changes to extract-takes / patterns / synthesize / embed / extract / migrate-engine auto-trigger it.
- `test/e2e/migrate-engine-sources-postgres.test.ts` — `DATABASE_URL`-gated companion for `gbrain migrate --to`: migrates a PGLite brain carrying two non-default sources with overlapping slugs into real Postgres and asserts `copyMigrationSources` created every `sources` FK parent (config JSONB intact, not double-encoded) before any page write. Unit-level manifest identity (crash manifest resumes only against the SAME target; legacy engine-only manifests start fresh) is `test/migrate-engine-resume.test.ts`.
- `test/e2e/facts-fence-reconcile-postgres.test.ts` — `DATABASE_URL`-gated round-trip for the escape-aware fence parser: renders a `## Facts` fence whose cells carry literal pipes, backslashes (Windows paths), and empty cells via `renderFactsTable`, runs the wipe-and-reinsert reconcile (`runExtractFacts`) on real Postgres, and asserts every cell survives byte-identically with no column shift.
- `test/e2e/source-isolation-pglite.test.ts` — PGLite in-memory regression suite pinning the source-isolation seal at two layers. Engine layer: `searchKeyword` / `searchVector` / `searchKeywordChunks` / `listPages` / `getPage` / `traverseGraph` / `traversePaths` apply `sourceId` (scalar fast path) and `sourceIds` (array path) correctly across both engines. Op-handler layer: routes through `sourceScopeOpts(ctx)` so a `read+write`-scoped OAuth client bound to `--source dept-x` cannot see rows from neighboring sources via `search`, `query`, `list_pages`, `get_page`, or `find_experts`. Covers both `ctx.sourceId` (single-source clients) and `ctx.auth.allowedSources` (federated_read clients) precedence; federated array wins over scalar wins over nothing. No `DATABASE_URL` needed.
- `test/e2e/think-source-isolation-pglite.test.ts` — PGLite in-memory suite pinning the `think` gather stage's source scope: seeds three sources with cross-source links and embedded takes, then asserts `runGather` under a federated `sourceIds` grant (and under a scalar `sourceId`) keeps every stream — hybrid retrieval, takes keyword + vector (`searchTakes`/`searchTakesVector`), and the `traversePaths` graph walk — inside the grant while still reaching authorized neighboring sources. No `DATABASE_URL` needed.
- `test/e2e/skill-brain-first.test.ts` — doctor reports `skill_brain_first` check with structured issues; `--fix --dry-run` previews insertion without writing; `--fix` applies the canonical Convention callout idempotently; `brain_first: exempt` frontmatter resolves the warn; `brain_first_typo` surfaces a paste-ready hint; audit JSONL records `detected` / `resolved` / `fixed` transitions; stable brain emits 0 audit lines/run.
- Tier 2 (`test/e2e/skills.test.ts`) requires OpenClaw + API keys, runs nightly in CI.
- If `.env.testing` doesn't exist in this directory, check sibling worktrees: `find ../ -maxdepth 2 -name .env.testing -print -quit` and copy it here if found.
- **Run E2E tests without asking permission.** When you want to verify behavior, there's a relevant E2E test, or you're shipping anything covered by an E2E suite — spin up the test DB, run the tests, tear down. Don't ask, don't propose it, don't defer. The lifecycle is short (~2-30s startup, sub-minute tests, instant teardown) and the gate value is high. Skipping with "DATABASE_URL unset" is silent regression, not caution.
### API keys and running ALL tests
ALWAYS source the user's shell profile before running tests:
```bash
source ~/.zshrc 2>/dev/null || true
```
This loads `OPENAI_API_KEY` and `ANTHROPIC_API_KEY`. Without these, Tier 2 tests
skip silently. Do NOT skip Tier 2 tests just because they require API keys — load
the keys and run them.
When asked to "run all E2E tests" or "run tests", that means ALL tiers:
- Tier 1: `bun run test:e2e` (mechanical, sync, upgrade — no API keys needed)
- Typed-link blockquotes: `> **Convention:** see [path](path).`
Three regexes, zero LLM tokens, single SQL `addLinksBatch` call with `INSERT ... SELECT FROM unnest(...) JOIN pages ON CONFLICT DO NOTHING RETURNING 1`. The graph grows on every write at near-zero cost. On a 17K-page brain, full graph extract completes in seconds.
Three regexes, zero LLM tokens, single SQL `addLinksBatch` call with `INSERT ... SELECT FROM jsonb_to_recordset(($1::jsonb)->'rows') JOIN pages ON CONFLICT DO NOTHING RETURNING 1` (free-text-safe; the prior `unnest(${arr}::text[])` form crashed on calendar/Zoom context per gbrain#1861). The graph grows on every write at near-zero cost. On a 17K-page brain, full graph extract completes in seconds.
Heuristic link-type inference (`attended`, `works_at`, `invested_in`, `founded`, `advises`) fires from surrounding sentence context — also LLM-free. Power users who want richer types add them via the typed-link blockquote convention.
@@ -54,7 +54,9 @@ The cost: +150ms p50 latency, ~$0.025/M tokens. Disabled with `gbrain config set
## Source-aware ranking
Hybrid search applies a source-factor CASE expression at the SQL layer (lives in `src/core/search/sql-ranking.ts`). Curated content like `originals/`, `concepts/`, `writing/` outranks bulk content like `your-openclaw/chat/`, `daily/`, `media/x/`. Hard-exclude prefixes (`test/`,`archive/`,`attachments/`, `.raw/`) filter at retrieval, not post-rank.
Hybrid search applies a source-factor CASE expression at the SQL layer (lives in `src/core/search/sql-ranking.ts`). Curated content like `originals/`, `concepts/`, `writing/` outranks bulk content like `your-openclaw/chat/`, `daily/`, `media/x/`. Hard-exclude prefixes (`test/`, `attachments/`, `.raw/`) filter at retrieval, not post-rank.
`archive/` is deliberately NOT hard-excluded (issue #1777): it holds high-signal historical content users expect to find, so it is demoted (`0.5x` in `DEFAULT_SOURCE_BOOSTS`), not hidden. The demote is a prior applied in the outer SQL re-rank; the cross-encoder reranker (balanced/tokenmax modes) can still PROMOTE an archive page that survives the demote into the rerank candidate window — it is not an unconditional suppression. `gbrain doctor`'s `hidden_by_search_policy` check reports how many chunked pages remain hidden by the surviving exclude prefixes.
The boost map is configurable via `GBRAIN_SOURCE_BOOST` env var or per-call `SearchOpts.exclude_slug_prefixes`. Temporal queries (`detail: 'high'`) bypass the boost so chat pages re-surface for time-sensitive lookups.
@@ -121,6 +123,7 @@ expansion (if enabled)
hybrid search:
├── vector (HNSW on chunk embeddings)
├── keyword (BM25 via tsvector)
├── relational (v0.42.34.0: typed-edge recall arm — relational queries only)
for most ops, 180s for `think`). `parseTimeout(s)` exported helper.
- `src/core/doctor-remote.ts` — `gbrain remote doctor` adds the
`oauth_client_scopes_probe` check (CDX-5). Probes the read tier via
`get_brain_identity` and admin tier via `get_health`; reports per-tier
status with pinpoint remediation when admin is missing. `buildScopeCheck`
+ `ScopeProbeResult` exported for test access. Skippable via
`GBRAIN_DOCTOR_SKIP_SCOPE_PROBE=1` for fixtures that mock /mcp at JSON-RPC
initialize level only (MCP SDK Client hangs on shape mismatch).
- `src/core/ssrf-validate.ts` (v0.36 Commit 0) — DNS-rebinding-defended URL validation. `validateAndResolveUrl(url)` resolves the hostname via `dns.lookup({all: true, family: 0})`, checks EVERY A AND AAAA record against the internal-IP deny list, returns the resolved IP so callers fetch by IP (defeats DNS rebinding: validation IP === fetch IP). `fetchWithSSRFGuard(url, opts)` does redirect-aware fetching with per-hop re-validation, max 3 hops by default. Reusable across all URL-fetching features. Test seam `__setDnsLookupForTests` for hermetic tests.
- `src/core/search/query-intent.ts` extension (v0.36 cross-modal wave) — new `suggestedModality: 'text' | 'image' | 'both'` axis on `QuerySuggestions`. Module-scope `CROSS_MODAL_PATTERNS` regex array (compiles once at module load). `isAmbiguousModalityQuery(query)` heuristic gate fires when a visual noun + reference marker combination indicates genuinely ambiguous routing — used by the Commit 4 LLM tie-break to bound LLM calls to <1% of queries.
- `src/core/search/mode.ts` extension (v0.36 cross-modal wave) — `ModeBundle` extended with 7 cross-modal knobs: `cross_modal_both_text_weight` / `cross_modal_both_image_weight` (D6 weighted RRF for `'both'` mode, defaults 0.6/0.4), `image_query_text_refinement_weight` / `image_query_image_refinement_weight` (D13 hybrid intersect for `searchByImage` query refinement, defaults 0.4/0.6), `unified_multimodal` + `unified_multimodal_only` (Phase 3 unified column routing flags), `cross_modal_llm_intent` (Commit 4 opt-in escalation). `SEARCH_MODE_CONFIG_KEYS` extended with 7 corresponding config keys. `KNOBS_HASH_VERSION` bumped 2→3 (D2 — closes the silent cache-hit class where a cached text-mode result could leak to an image-mode caller).
- `src/core/search/hybrid.ts` extension (v0.36 cross-modal wave) — cross-modal routing branch at the embed step. Resolves `effectiveModality` from per-call `opts.crossModal` (normalized: literal `'auto'` → undefined per D22-1) → `suggestions.suggestedModality` → `'text'` default. Image route: `embedQueryMultimodal` + `searchVector({embeddingColumn: 'embedding_image'})`, skip expansion + keyword (D9 mode-bundle override). 'both' route: parallel text + image vector searches merged via `rrfFusionWeighted` with `effectiveRrfK(baseRrfK, weight)` from the configured cross-modal weights. Phase 3 unified routing fires when `cfg.search.unified_multimodal === true` — bypasses dual-column branching, runs `embedQueryMultimodal` + `searchVector({embeddingColumn: 'embedding_multimodal'})`, D8 fail-open on zero rows + not strict-mode falls through to dual-column. Commit 4 LLM escalation fires only when (no explicit per-call opt) AND (regex returned 'text') AND (`cfg.search.cross_modal.llm_intent` is true) AND (`isAmbiguousModalityQuery` returns true). Fail-open on every error.
- `src/core/search/image-loader.ts` (v0.36 Phase 2) — `loadImageInput(input, opts)` accepts local path, `data:` URI, or `http(s)://` URL. Magic-byte sniff for PNG/JPEG/WebP. Hard size cap (default 10 MB, configurable via `search.image_query.max_bytes`). For URLs: routes through `fetchWithSSRFGuard` so DNS rebinding + redirect chains are defeated. Pre-flight Content-Length check + post-fetch size guard for lying servers. `ImageLoadError` with discriminated `code` (INVALID_FORMAT / OVERSIZED / INVALID_URL / FETCH_FAILED / TIMEOUT / SSRF_BLOCKED / NOT_FOUND).
- `src/core/search/by-image.ts` (v0.36 Phase 2) — `searchByImage(engine, input, opts)`. Always runs image branch (`embedQueryMultimodalImage` + `searchVector(embedding_image)`). D13 hybrid intersect: when caller provides optional `query`, runs parallel text branch via `embedQueryMultimodal(query)` and merges via `rrfFusionWeighted` with weights from resolved mode. Phase 3 widens to unified column once `search.unified_multimodal=true` (transparently upgrades the retrieval quality post-reindex).
- `src/core/spend-log.ts` (v0.36 Phase 2 D23-#6) — per-OAuth-client paid-API spend tracking against the `mcp_spend_log` table (migration v74). `checkBudget(engine, clientId, capCents)` is the pre-flight gate; throws `BudgetExceededError` when today's spend has hit the cap. `recordSpend(engine, entry)` is best-effort post-call. UTC day-aligned aggregation so caps roll over deterministically regardless of server timezone. Local CLI callers (no clientId) bypass the gate. Pre-v0.36 brains without the table fail open to spend=0. `VOYAGE_MULTIMODAL_3_PER_IMAGE_CENTS` = 0.12 cents per image embed.
- `src/core/search/llm-intent.ts` (v0.36 Commit 4) — opt-in LLM tie-break. `classifyModalityWithLLM(query, fallback)` routes through `gateway.chat()` with a fixed single-word-output system prompt. 1s timeout via AbortController. `parseModality(raw, fallback)` is the pure parser — tolerates trailing punctuation + casing. Fail-open on every error (gateway unavailable, timeout, parse failure, unrecognized output) — returns fallback so a misbehaving LLM can never break search. Cost-bounded by the ambiguity heuristic in `query-intent.ts` (fires <1% of queries when on).
- `src/commands/reindex-multimodal.ts` (v0.36 Phase 3) — `gbrain reindex --multimodal [--limit N] [--dry-run] [--cost-estimate] [--no-embed] [--yes] [--json]`. Walks `content_chunks WHERE embedding_multimodal IS NULL`, batches via `embedMultimodalSafe` (Commit 0 partial-failure-aware), persists. D7 lock acquisition via `tryAcquireDbLock('gbrain-reindex-multimodal', 360min)`. Cost prompt + 10s Ctrl-C grace window in TTY. `GBRAIN_NO_REEMBED=1` bypass. Checkpoint at `~/.gbrain/reindex-multimodal-checkpoint.json` for resume. D23-#2 auto-flip prompt at coverage=100% completion (TTY: interactive; non-TTY: stderr hint with paste-ready command).
- `src/core/backfill-registry.ts` extension (v0.36) — new `modality` backfill kind. SQL filter requires `chunk_source='image_asset'` AND `embedding_image IS NOT NULL` AND `(modality IS NULL OR modality != 'image')`. D22-7 defensive guard: never flag a non-image chunk that happens to have `embedding_image` populated. Idempotent — second run finds zero rows.
- `src/core/migrate.ts` v74 (`mcp_spend_log`) + v75 (`embedding_multimodal_column`) — Phase 2 spend-log table + Phase 3 unified column ALTER. v75 is column-only (no HNSW index — deferred to post-reindex per pgvector best practice). v74 uses BTREE on `(client_id, created_at)` + `(token_name, created_at)` — `date_trunc('day', TIMESTAMPTZ)` is NOT IMMUTABLE so can't appear in index expressions; range scan on created_at covers the per-day rollup query.
- `src/core/operations.ts` — `get_brain_identity` op (read scope, no params,
1. Dropping the HNSW vector index (pgvector won't survive an `ALTER COLUMN TYPE`).
2. Altering the column type (Postgres only — PGLite cannot do this).
3. Wiping every existing embedding (the old vectors are unusable in the new space).
2. Wiping every existing embedding (the old vectors are unusable in the new space — and pgvector refuses to cast them across dimensions, so this must happen before the alter).
3. Altering the column type (Postgres only — PGLite cannot do this).
4. Re-embedding the entire corpus (can take hours on a 50K-page brain and costs $1-100 in API calls depending on model).
5. Conditionally recreating the index (HNSW supports up to 2000 dimensions per pgvector; above that you must use exact scans).
@@ -115,12 +115,17 @@ BEGIN;
-- 1. Drop the HNSW index. It can't survive the column type change.
DROP INDEX IF EXISTS idx_chunks_embedding;
-- 2. Alter the column type.
ALTER TABLE content_chunks ALTER COLUMN embedding TYPE vector(<NEW_DIMS>);
-- 3. Clear stale embeddings so they don't survive into the new space.
-- 2. Clear stale embeddings FIRST. This must happen BEFORE the column
-- alter: pgvector refuses to cast existing vectors across dimensions
-- ("expected <NEW_DIMS> dimensions, not <OLD_DIMS>"), so altering a
-- column that still holds old-width vectors aborts the transaction.
-- NULLs cast fine. (The old vectors are unusable in the new space
-- anyway — this is the wipe step from the rationale above.)
UPDATE content_chunks SET embedding = NULL, embedded_at = NULL;
-- 3. Alter the column type (all rows are NULL now, so the cast succeeds).
ALTER TABLE content_chunks ALTER COLUMN embedding TYPE vector(<NEW_DIMS>);
-- 4. Recreate the HNSW index ONLY IF dims <= 2000. Above that, leave it
-- indexless and rely on exact scans (gbrain searchVector handles this
-- automatically — search just gets slower, not broken).
@@ -150,6 +150,24 @@ Every metric `gbrain eval *` and `gbrain search stats` reports has a plain-Engli
**Range:** 0..unbounded. Warm-cache hits should be <50ms; tokenmax with expansion can exceed 200ms due to the Haiku call.
## Result-Sizing Metrics
### Autocut signal
**Key:** `autocut.signal`
**Plain English:** Which signal autocut used to size the result set. 'rerank' means it found a real score cliff in the cross-encoder rerank scores and cut there; 'none' means no trustworthy cliff (no reranker, <2 scored results, or the gap was too small) so it returned the full list.
**Range:** 'rerank' | 'none'. 'none' is not a failure — it means autocut declined to cut because the signal didn't justify it.
### Autocut gap ratio
**Key:** `autocut.gap_ratio`
**Plain English:** The size of the largest score drop autocut found, as a fraction of the top result's score. A gap of 0.40 means the score fell by 40% of the top score at the steepest point. Autocut cuts there only when this clears the sensitivity threshold (autocut_jump, default 0.20).
**Range:** 0..1, higher = a sharper cliff (more confident cut). Below the autocut_jump threshold → no cut.
@@ -160,7 +160,7 @@ The mode-picker prompt at `gbrain init` and the CLAUDE.md `## Search Mode` table
- Your agent's system prompt + reasoning tokens add input that gbrain doesn't see.
- Compaction reduces input over a long session.
- Most agents make 1-5 searches per turn; cost-per-turn is what bills you, not cost-per-query.
- The model price column drifts as providers reprice; pin the rate via `src/core/anthropic-pricing.ts` for a current snapshot.
- The model price column drifts as providers reprice; pin the rate via `src/core/model-pricing.ts` (the canonical chat-pricing table) for a current snapshot.
The picker copy + CLAUDE.md table are the canonical user-facing source. Update them in lockstep when the underlying chunker size or default `searchLimit` changes.
| `--allow-mutate-bundled` | off | Required to mutate gbrain-bundled skills |
| `--no-mutate` | off | Write proposed.md, don't replace SKILL.md (no held-out needed) |
| `--allow-mutate-bundled` | off | Required to mutate gbrain-bundled skills in place — ALSO requires `--held-out` (>=5 rows) or the run hard-refuses |
| `--held-out <path>` | — | Independent test set (same JSONL shape as the benchmark, task IDs disjoint from it). A candidate that beats the benchmark but regresses on the held-out set is refused. Required for in-place bundled mutation. |
| `--max-cost-usd N` | 5.00 | Hard cap; preflight refuses if exceeded |
| `--max-runtime-min N` | 30 | Wall-clock cap |
| `--force` | off | Bypass dirty-working-tree refusal |
@@ -123,7 +124,8 @@ refuses to start when the estimate exceeds `--max-cost-usd`.
| Validation gate is mandatory | D12 (paper) | Accepting LLM judge noise as improvement |
@@ -16,6 +16,34 @@ benefit-focused bullets, waits for explicit permission, then runs the full
upgrade flow including re-reading skills, running migrations, and syncing
schema. The user gets new capabilities automatically.
## Self-upgrade modes (v0.42)
gbrain now stays current the way gstack does: it rides invocation frequency. A
throttled, cache-read-only check runs at the start of every `gbrain` invocation
(CLI and MCP) and emits an `UPGRADE_AVAILABLE <old> <new>` marker on stderr. No
host cron required — every agent kind (Claude Code, Codex, OpenClaw, Hermes, the
`gbrain serve` host behind a Perplexity thin client) converges to current by
construction. The behavior is governed by one file-plane config key,
`self_upgrade.mode`:
| Mode | Behavior | Who it's for |
|------|----------|--------------|
| `notify` (default) | Emit the marker + a 4-option prompt; never apply without confirmation. | Interactive installs / anyone with a human in the loop. |
| `auto` (opt-in) | Apply silently, but ONLY during quiet hours, ONLY when the brain is idle, doctor-gated, and never re-trying a known-bad version. | Headless / always-on installs (autopilot daemon, the `gbrain serve` host). |
| `together` | `TOGETHER_API_KEY` | 768 | varies | no | no |
| `anthropic` | (no embedding model — chat only) | — | — | — | — |
| `deepseek` | (no embedding model — chat only) | — | — | — | — |
@@ -77,6 +77,8 @@ The doctor distinguishes two repair paths:
Default. Set `OPENAI_API_KEY`. Models: `text-embedding-3-large` (3072 max, 1536 default), `text-embedding-3-small` (1536). Matryoshka via the `dimensions` field — gbrain pins it from `embedding_dimensions` config so existing 1536-dim brains stay aligned across SDK upgrades.
Optional `OPENAI_BASE_URL` — point the native OpenAI provider at an OpenAI-compatible gateway. A bare host is normalized to carry the `/v1` suffix automatically (so `https://gw.example.com` and `https://gw.example.com/v1` both work); when unset, the SDK's default endpoint is untouched. `ANTHROPIC_BASE_URL` gets the same normalization for Anthropic chat/expansion calls.
### Voyage AI
Best-in-class quality on the Voyage 4 family (Jan 2026 release). Set `VOYAGE_API_KEY`. Models: `voyage-4-large`, `voyage-4`, `voyage-4-lite`, `voyage-4-nano`, `voyage-3.5`, `voyage-code-3` (code-tuned), `voyage-finance-2`, `voyage-law-2`, `voyage-multimodal-3` (text + image).
No env required — Ollama runs unauthenticated locally. Optional `OLLAMA_BASE_URL` (default `http://localhost:11434/v1`) and `OLLAMA_API_KEY` (for auth-enabled deployments).
Recipe ships with `nomic-embed-text` (768d, recommended), `mxbai-embed-large` (1024d), `all-minilm` (384d). `gbrain providers test --model ollama:nomic-embed-text` smoke-tests the local install.
Recipe ships with `nomic-embed-text` (768d, recommended), `mxbai-embed-large` (1024d), `all-minilm` (384d), plus the larger modern embedders `qwen3-embed-8b` (4096d) and `snowflake-arctic-embed-l-v2` (1024d). `gbrain providers test --model ollama:nomic-embed-text` smoke-tests the local install.
The recipe default is `nomic-embed-text`'s 768 dims. If you run one of the larger models, declare its native dimension with `--embedding-dimensions <N>` at init — gbrain trusts the value you declare for local recipes instead of rejecting a non-768 width.
### llama-server (local, llama.cpp)
`llama.cpp`'s `llama-server --embeddings` endpoint. No env required. Optional `LLAMA_SERVER_BASE_URL` (default `http://localhost:8080/v1`) and `LLAMA_SERVER_API_KEY`.
User-driven models: launch llama-server with `--model <gguf-path> --embeddings`, then run `gbrain init --embedding-model llama-server:<your-id> --embedding-dimensions <N>`. The recipe refuses the implicit shorthand `--model llama-server` because there's no canonical first model.
User-driven models: launch llama-server with `--model <gguf-path> --embeddings`, then run `gbrain init --embedding-model llama-server:<your-id> --embedding-dimensions <N>`. gbrain trusts the dimension you declare (you know the GGUF you launched); the recipe refuses the implicit shorthand `--model llama-server` because there's no canonical first model.
### LiteLLM proxy (universal escape hatch)
@@ -155,6 +159,8 @@ Run [LiteLLM](https://docs.litellm.ai/docs/proxy/quick_start) in front of any pr
This is the catch-all for "my provider isn't in the list above." Set up LiteLLM, then `gbrain init --embedding-model litellm:<your-model-id> --embedding-dimensions <N>`.
**Include the `/v1` suffix in `LITELLM_BASE_URL` if your proxy serves the OpenAI route there** (e.g. `http://localhost:4000/v1`). Many LiteLLM deployments expose the OpenAI-compatible API only under `/v1`; pointing gbrain at the bare host 404s or fails authentication with no hint. gbrain trusts the dimension you declare for the proxy-backed model — the proxy's backend, not gbrain, decides the true width — so `--embedding-dimensions <N>` is required and accepted as-is.
## Choosing dimensions
Three numbers matter:
@@ -183,5 +189,3 @@ The supported paths:
- **Postgres (Supabase / self-hosted):** follow the SQL recipe in `docs/embedding-migrations.md` (drop the HNSW index, ALTER COLUMN TYPE, clear stale embeddings, recreate the index conditionally, then `gbrain init --supabase --embedding-model X --embedding-dimensions N` to update the file plane and re-embed).
`gbrain doctor` 8c "alternative_providers" surfaces unconfigured providers whose env is already set — useful when you've configured OpenAI but also have e.g. `VOYAGE_API_KEY` exported and want to know you can switch without extra setup.
`gbrain doctor` 8c "alternative_providers" surfaces unconfigured providers whose env is already set — useful when you've configured OpenAI but also have e.g. `VOYAGE_API_KEY` exported and want to know you can switch without extra setup.
GBrain's embedding-spend gates in one place: every gate, its config key, default,
whether it blocks or just informs, how to widen or disable it, and how the
`spend.posture` switch governs all of them.
The orienting idea: **GBrain itself is rounding error; the spend that matters is
downstream embedding.** These gates exist so a routine sync or enrich can't run up
an unexpected embedding bill, while never wedging an unattended cron.
## `spend.posture` — one switch for "cost is not my constraint"
```bash
gbrain config set spend.posture tokenmax # all cost gates become informational
gbrain config set spend.posture gated # default — gates enforce
```
| Value | Effect |
|-------|--------|
| `gated` (default) | Every cost gate enforces its limit as documented below. |
| `tokenmax` | Every cost gate prints its estimate and **proceeds** — informational only. Spend is still recorded to the ledger; posture removes the *ceiling*, not the *accounting*. |
`spend.posture` is deliberately separate from `search.mode=tokenmax` (which governs
retrieval payload size, not embedding spend). When a gate fires and
`search.mode=tokenmax` but `spend.posture` is unset, the gate prints a one-line hint
pointing at this switch.
**Precedence:** an explicit per-call cap (`--max-usd N`, `--max-cost N`) always wins
over posture. `tokenmax` only governs the default/absent case — it never overrides a
number you typed on the command line.
## Off switches (`off` / `unlimited` / `none`)
The USD-limit knobs accept `off`, `unlimited`, or `none` (case-insensitive) to mean
"no limit" — no more setting sentinel values like `100000`.
- `0` is **not** "off". On `sync.cost_gate_min_usd`, `0` means "block on any nonzero
spend" (a real choice). On the backfill caps, `0` falls back to the default.
- Internally "no limit" is the string `unlimited` in any printed/JSON output and "no
cap" inside the budget tracker — never a raw `Infinity` (which would serialize to
Add an `idea-lineage` thinking skill that traces how one idea has evolved through a user's brain: first mention, best articulation, related concepts, reversals, contradictions, abandoned branches, and the current live version. The contribution should start as a read-only skill with routing and conformance coverage, not as a new CLI or MCP operation.
## Problem Frame
GBrain already has two adjacent capabilities that are easy to conflate with this feature:
- `skills/concept-synthesis/SKILL.md` is a mutating, batch-oriented concept map builder. It deduplicates many concept stubs, tiers them, writes concept pages, and creates an intellectual universe.
- `find_trajectory` and `gbrain eval trajectory` are structured entity trajectories over typed facts and events. They work best for questions like metric history, founder consistency, role/status changes, and event timelines.
`idea-lineage` should occupy the narrow space between them: a query-time, single-idea, citation-backed synthesis of conceptual evolution. It should help a user ask "how has my thinking about this idea changed?" without running a global concept-synthesis job or forcing the idea into an entity/metric trajectory model.
## Requirements
**Behavior**
- R1. The skill accepts a single idea, topic, concept phrase, or nearby concept page and produces a focused lineage for that idea only.
- R2. The output identifies first mention, best articulation, related concepts, reversals, contradictions, abandoned branches, and current live version when evidence supports each category.
- R3. Every lineage claim is grounded in existing brain evidence: page links, dates, verbatim snippets, timeline entries, takes, contradiction findings, or trajectory points when applicable.
- R4. The skill distinguishes evidence strength. Missing or weak evidence should be reported as a gap, not filled with plausible narrative.
- R5. The default workflow is read-only and does not write or mutate brain pages.
**Routing**
- R6. Routing should prefer `idea-lineage` for single-idea evolution requests such as "how has my thinking about X changed?".
- R7. Routing should keep broad corpus/map requests on `concept-synthesis`.
- R8. Routing should keep structured entity metric/status questions on `find_trajectory`, `gbrain eval trajectory`, or `gbrain think` trajectory injection.
**Privacy and portability**
- R9. The skill and fixtures must use public, generic examples only.
- R10. The plan and implementation must avoid private fork names, real people, real companies, funds, or host-specific filesystem paths in public artifacts.
## Scope Boundaries
### In Scope
- A new bundled skill under `skills/idea-lineage/`.
- Resolver, manifest, and plugin-bundle wiring.
- Routing fixtures that prove the new intent is reachable and does not swallow `concept-synthesis` or trajectory-shaped prompts.
- Documentation inside the skill body that explains when to use `search`, `query`, `get_page`, `list_pages`, `takes_search`, `find_contradictions`, and optionally `find_trajectory`.
- Focused conformance, resolver, and routing verification.
### Deferred to Follow-Up Work
- A first-class `idea_lineage` MCP operation.
- A `gbrain idea lineage <query>` CLI.
- Persisting lineage reports back into the brain.
- New database tables, schema-pack fields, or concept lineage graph primitives.
- Automated contradiction-probe reruns. The skill should read cached contradiction findings if available, not trigger expensive probes.
- Implementing the broader taxonomy redesign tracked by issue #1668.
## Key Technical Decisions
- **Start as a markdown skill:** GBrain's architecture treats skills as fat markdown workflows. This feature can be useful by orchestrating existing read operations, so a CLI/MCP surface would add contract weight before the behavior is proven.
- **Make the skill non-mutating by default:** The user intent is investigative. Writing lineage pages should remain a later explicit mode after routing and output quality are established.
- **Use evidence buckets rather than a single narrative pass:** The output should force the agent to separately evaluate first mention, articulation, current version, reversals, contradictions, and abandoned branches. That reduces the risk of smoothing over conflict.
- **Keep `find_trajectory` as an optional side-channel:** It is valuable when an idea query resolves to an entity attribute or status history, but `idea-lineage` should not depend on typed facts being present.
- **Avoid the existing "trace idea evolution" trigger phrase:** That phrase already routes to `concept-synthesis`; adding it to the new skill would create avoidable resolver ambiguity.
## High-Level Technical Design
```mermaid
flowchart TB
A["User asks about one idea"] --> B{"Intent shape"}
B -->|"whole corpus / map"| C["concept-synthesis"]
B -->|"entity metric / status over time"| D["trajectory surfaces"]
B -->|"single conceptual idea"| E["idea-lineage skill"]
E --> F["Resolve idea candidates"]
F --> G["Gather evidence via search/query/pages/takes"]
G --> H["Classify lineage moments"]
H --> I["Synthesize cited answer with confidence gaps"]
```
## Implementation Units
### U1. Add the `idea-lineage` Skill
- **Goal:** Create the read-only skill contract and workflow.
- **Requirements:** R1, R2, R3, R4, R5, R9, R10
- **Dependencies:** None
- **Files:**
- `skills/idea-lineage/SKILL.md`
- `test/skills-conformance.test.ts`
- **Approach:** Create a new skill with required frontmatter and conformance sections. The skill should define its workflow in phases: clarify the target idea, resolve likely concept/page anchors, collect evidence, classify lineage moments, produce a cited synthesis, and state gaps. Frontmatter should set `mutating: false` and list read operations only.
- **Patterns to follow:**
- `skills/strategic-reading/SKILL.md` for a read-only thinking-skill shape with related-skill boundaries.
- `skills/query/SKILL.md` for search/query/get-page guidance.
- `skills/concept-synthesis/SKILL.md` for contrast, not for behavior reuse.
- **Test scenarios:**
- A new `SKILL.md` with frontmatter, `## Contract`, `## Output Format`, and `## Anti-Patterns` passes conformance.
- The frontmatter declares a unique `name: idea-lineage`.
- The skill body references only portable, synthetic examples.
- **Verification:**`bun test test/skills-conformance.test.ts` passes.
### U2. Wire Resolver, Manifest, and Bundle Metadata
- **Goal:** Make the skill discoverable by bundled skill users and resolvable by agents.
- **Requirements:** R6, R7, R8, R9, R10
- **Dependencies:** U1
- **Files:**
- `skills/RESOLVER.md`
- `skills/manifest.json`
- `openclaw.plugin.json`
- `test/resolver.test.ts`
- `test/skillpack-reference.test.ts`
- **Approach:** Add `idea-lineage` to the skill manifest and plugin skill list. Add a resolver row in the thinking or uncategorized section with narrow user phrases such as "how has my thinking about", "trace the lineage of this idea", "what is my current version of", and "show reversals in my thinking about". Keep broad concept-map phrases routed to `concept-synthesis`.
- **Patterns to follow:**
- `skills/RESOLVER.md` rows for `strategic-reading`, `concept-synthesis`, and `perplexity-research`.
- Every quoted resolver trigger fuzzy-matches a frontmatter trigger in `skills/idea-lineage/SKILL.md`.
- `idea-lineage` is listed in `skills/manifest.json`.
- `idea-lineage` is listed in `openclaw.plugin.json` if the contribution ships as part of the bundled OpenClaw skillpack.
- Existing skills remain reachable.
- **Verification:**`bun test test/resolver.test.ts` passes.
### U3. Add Routing Eval Fixtures
- **Goal:** Prove the new routing boundary against adjacent skills.
- **Requirements:** R6, R7, R8
- **Dependencies:** U1, U2
- **Files:**
- `skills/idea-lineage/routing-eval.jsonl`
- `skills/concept-synthesis/routing-eval.jsonl`
- `src/core/routing-eval.ts`
- **Approach:** Add positive fixtures for single-idea lineage prompts and negative or ambiguity-declared fixtures around adjacent surfaces. The fixture text should paraphrase triggers rather than copy them exactly, because the routing fixture linter rejects tautological trigger copies.
- **Test scenarios:**
- "Show how my thinking about founder-led sales changed over time" routes to `idea-lineage`.
- "What is my current version of the compounding trust idea?" routes to `idea-lineage`.
- "Synthesize my concepts into a tiered intellectual map" stays on `concept-synthesis`.
- "How has acme-example MRR trended since January?" does not route to `idea-lineage`.
- Negative fixtures avoid false positives for generic "publish this report" or "what is this concept?" prompts.
- **Verification:**`gbrain routing-eval --json` reports no new misses, false positives, or unapproved ambiguity for the added fixtures.
### U4. Add Output Contract and Citation Discipline
- **Goal:** Make the skill's user-facing answer shape predictable and reviewable.
- **Requirements:** R2, R3, R4, R5
- **Dependencies:** U1
- **Files:**
- `skills/idea-lineage/SKILL.md`
- `skills/conventions/quality.md`
- `skills/brain-ops/SKILL.md`
- **Approach:** Define the output format directly in the skill body. The recommended shape should include a compact current answer, evidence timeline, lineage buckets, contradictions/reversals, abandoned branches, related concepts, and confidence gaps. Require page/date/snippet evidence for each non-gap claim. Preserve quote fidelity and avoid hallucinated dates.
- **Patterns to follow:**
- `skills/conventions/quality.md` for citation and quote-fidelity expectations.
- `skills/brain-ops/SKILL.md` for source attribution and source-id formatting.
- `docs/takes-vs-facts.md` for not conflating holder-attributed takes with the brain owner's facts.
- **Test scenarios:**
- Test expectation: none beyond conformance for the markdown-only contract; routing and conformance tests cover the machine-checkable surface.
- **Verification:** Manual review confirms the skill body tells the agent how to cite, label gaps, and separate facts/takes/trajectory evidence.
### U5. Refresh Generated Documentation If Required
- **Goal:** Keep generated LLM-facing docs consistent if the test suite requires it.
- **Requirements:** R9, R10
- **Dependencies:** U1, U2, U3
- **Files:**
- `llms.txt`
- `llms-full.txt`
- `test/build-llms.test.ts`
- **Approach:** Run the build-llms test after adding the skill. If it fails because committed docs are stale, regenerate with the existing generator and include the generated diff. If it passes without regeneration, leave these files unchanged.
- **Patterns to follow:**
- `package.json` script `build:llms`.
- `test/build-llms.test.ts` failure message.
- **Test scenarios:**
- Committed `llms.txt` and `llms-full.txt` match generator output.
- `llms-full.txt` remains within the size budget.
- **Verification:**`bun test test/build-llms.test.ts` passes.
## Acceptance Examples
- AE1. When the user asks "How has my thinking about founder-led sales changed over time?", the agent routes to `idea-lineage`, searches for evidence, and returns a cited lineage rather than running `concept-synthesis`.
- AE2. When the user asks "Run concept synthesis across my notes", the agent routes to `concept-synthesis`, not `idea-lineage`.
- AE3. When the user asks "How did acme-example's MRR trend?", the agent uses trajectory surfaces rather than `idea-lineage`.
- AE4. When the evidence does not support an "abandoned branch" claim, the output includes a gap instead of inventing one.
## Risks & Dependencies
- **Resolver overlap risk:**`concept-synthesis` already uses "trace idea evolution". Mitigate by avoiding that exact trigger and adding routing fixtures around the boundary.
- **Narrative overreach risk:** The feature invites story-making. Mitigate by requiring dates, snippets, links, and explicit gaps for unsupported categories.
- **Privacy risk:** Skill examples can easily drift into real-brain language. Use synthetic examples only and rely on existing privacy checks.
- **Generated-doc churn risk:** Adding a bundled skill may require `llms.txt` and `llms-full.txt` regeneration. Treat generated-doc changes as mechanical and separate from the skill design during review.
- **Future taxonomy dependency:** Issue #1668 may eventually change concept filing and identity. This plan avoids new schema assumptions so the contribution remains compatible with the current repo.
## Sources & Research
- `skills/concept-synthesis/SKILL.md` defines the existing batch, mutating, concept-map surface.
- `skills/RESOLVER.md` and `skills/manifest.json` define current skill reachability and bundle metadata.
- `docs/architecture/lens-packs.md` shows that atoms and concepts are already part of the lens-pack/dream-cycle substrate.
- `docs/proposals/temporal-contradiction-probe.md` and `docs/takes-vs-facts.md` define the temporal and epistemic boundaries this skill must not blur.
- `src/core/operations.ts`, `src/core/trajectory.ts`, `src/commands/eval-trajectory.ts`, and `test/operations-find-trajectory.test.ts` define the current `find_trajectory` contract.
- Pull requests #1131, #1296, and #1364 provide the recent trajectory, think-routing, and lens-pack context.
- Issue #1668 is related future taxonomy work, but not a prerequisite for this contribution.
The `--bind 0.0.0.0` is important. By default the server binds to localhost only, which is correct for a personal install but blocks remote teammates. Setting `0.0.0.0` accepts connections from any interface.
The server prints an admin bootstrap token to stderr on first start. Save it. You'll use it once for the admin dashboard.
The server prints an admin bootstrap token to stderr on first start when run in an interactive terminal. Save it. You'll use it once for the admin dashboard. On a non-TTY start (systemd, Docker, piped logs) the token is hidden from logs — set `GBRAIN_ADMIN_BOOTSTRAP_TOKEN` yourself or pass `--print-admin-token` on a trusted terminal instead.
For development, tunnel the local server out via ngrok:
(On a thin-client install every shared op routes through the remote MCP server automatically — no flag needed. The env vars select whose credentials the call uses.)
Alice should see results only from `customers` and `shared`. The performance-review notes live in `internal`, which she's not scoped to read. She shouldn't see them.
```bash
# Terminal 2, as Bob (export his credentials similarly)
gbrain search "performance review" --remote
gbrain search "performance review"
```
Bob should see the performance-review notes from `internal`, plus anything related from `shared`. He shouldn't see anything that lives only in `customers`.
@@ -514,7 +516,7 @@ The first sync embeds every page, which takes time. Check `gbrain sources status
### "I see a page I shouldn't see"
This shouldn't happen, but if you suspect it, run `gbrain search <query> --remote --json` as the constrained client and inspect the `source_id` field on every returned result. Every row should be in the client's `--federated-read` set. If one isn't, file an issue with the exact slug and source IDs.
This shouldn't happen, but if you suspect it, run `gbrain search <query> --json` as the constrained client (thin-client install, with the client's `GBRAIN_REMOTE_*` env exported) and inspect the `source_id` field on every returned result. Every row should be in the client's `--federated-read` set. If one isn't, file an issue with the exact slug and source IDs.
@@ -86,7 +86,7 @@ Save this token. You'll need it for the AlphaClaw setup.
AlphaClaw is the setup harness that manages OpenClaw deployment.
1. Go to [alphaclaw.com](https://alphaclaw.com)
1. Go to [alphaclaw.md](https://alphaclaw.md)
2. Enter your **workspace repo** (not the brain repo): `your-org/myagent`
3. Select "Use existing" if the repo already exists
4. Enter your GitHub PAT from Step 2
@@ -145,14 +145,15 @@ GBrain uses Supabase for vector embeddings and full-text search at scale. There
Skip this and every embed write fails with "type vector does not exist" the moment GBrain tries to create its schema. pgvector is what stores the embeddings; the schema migrations refuse to run without it. Five seconds in the UI; an hour of debugging if you forget.
### 7b. Get the CONNECTION POOLER connection string, not the direct one
### 7b. Get the TRANSACTION POOLER connection string, not the direct one
In **Project Settings → Database → Connection string**, Supabase shows you two options. They look almost identical. Use the right one.
In the Supabase dashboard, click **Connect** in the top navigation bar, then **Connection String**. Supabase shows three options. They look almost identical. Use the right one.
- **Direct connection** (port 5432). Talks straight to the Postgres instance. IPv6-only. Will fail if your Render host doesn't have IPv6 outbound (most don't by default).
- **Connection pooler** (port 6543, hostname starts with`aws-0-...pooler.supabase.com`). Talks through Supabase's pgbouncer. Works over IPv4. Survives connection storms from parallel workers.
- **Direct connection** (port 5432, host `db.YOUR-PROJECT.supabase.co`). Talks straight to the Postgres instance. IPv6-only. Will fail if your Render host doesn't have IPv6 outbound (most don't by default).
- **Transaction pooler** (port 6543, host `aws-0-...pooler.supabase.com`). Talks through Supabase's pooler (Supavisor) in transaction mode. Works over IPv4. Survives connection storms from parallel workers. GBrain is tuned for this one: it auto-disables prepared statements on port 6543 and routes migrations, DDL, and worker locks to a separate direct connection (see 7c).
- **Session pooler** (port 5432, host `aws-0-...pooler.supabase.com`). Also works over IPv4, with full session features. You don't need it as your main URL, but it's the free way to fix the IPv4 gotcha in 7c.
You want the **connection pooler** string. Format looks like:
You want the **Transaction pooler** string. Format looks like:
gbrain config set database_url "postgresql://postgres.YOUR-PROJECT:YOUR-PASSWORD@aws-0-us-west-1.pooler.supabase.com:6543/postgres"
```
### 7c. Buy the IPv4 add-on if your host is IPv4-only
### 7c. Fix the IPv4 gotcha for migrations, DDL, and worker locks
Even with the pooler, some Supabase regions and some Render plans hit IPv6 resolution snags. If your `gbrain doctor` shows connection failures and the error mentions "network unreachable" or hangs forever on connect, you need Supabase's **IPv4 add-on**.
The transaction pooler (7b) carries your normal reads and writes over IPv4. But GBrain runs schema migrations, DDL, and background-worker locks on a *direct* connection, which it derives from your pooler URL by swapping the host to `db.YOUR-PROJECT.supabase.co:5432`. That direct host is **IPv6-only**. On an IPv4-only host (most Render plans), reads work but migrations hang and worker locks orphan, often silently.
In the Supabase dashboard, **Project Settings → Add-ons → IPv4 address**. About $4 a month. Toggle on, wait a minute, retry the connection. This bit me on multiple installs before I learned to just buy it up front.
Two ways to fix it. The free one first:
**Free: point GBrain's direct connection at the Session pooler.** The session pooler is the same Supavisor host on port 5432, and it's IPv4. Copy the **Session pooler** string from the same **Connect → Connection String** panel and set it as the direct-connection override:
Now both pools — reads on the transaction pooler (6543), DDL and locks on the session pooler (5432) — run over IPv4 at zero extra cost.
**Paid: buy Supabase's IPv4 add-on.** About $4 a month, Pro tier or higher. It makes the direct `db.*.supabase.co` host reachable over IPv4, so the derived direct connection just works with no extra config. In the Supabase dashboard, **Project Settings → Add-ons → IPv4 address**. Toggle on, wait a minute, retry.
Either fixes it. If `gbrain doctor` still shows connection failures that mention "network unreachable" or hangs forever on connect, you haven't done one of these yet.
- [AGENTS.md](https://raw.githubusercontent.com/garrytan/gbrain/master/AGENTS.md): Start here if you are not Claude Code. Install order, trust boundary, skill resolver, config/debug/migration pointers.
- [CLAUDE.md](https://raw.githubusercontent.com/garrytan/gbrain/master/CLAUDE.md): Architecture reference. Key files, trust boundaries, engine factory, test layout.
- [CLAUDE.md](https://raw.githubusercontent.com/garrytan/gbrain/master/CLAUDE.md): Orientation + resolver. North Star, two axes, architecture + cross-cutting invariants, the reference map pointing at on-demand docs, and the inline ship IRON RULES.
- [docs/architecture/KEY_FILES.md](https://raw.githubusercontent.com/garrytan/gbrain/master/docs/architecture/KEY_FILES.md): Per-file index for the gbrain repo: what each src/ file does + its load-bearing invariants. The on-demand detail CLAUDE.md's reference map routes to.
- [docs/guides/minions-deployment.md](https://raw.githubusercontent.com/garrytan/gbrain/master/docs/guides/minions-deployment.md): Deploying the gbrain jobs worker: crontab + watchdog, inline --follow, systemd/Procfile/fly.toml, upgrade checklist.
- [docs/guides/quiet-hours.md](https://raw.githubusercontent.com/garrytan/gbrain/master/docs/guides/quiet-hours.md): Notification hold + timezone-aware delivery.
- [docs/guides/scaling-skills.md](https://raw.githubusercontent.com/garrytan/gbrain/master/docs/guides/scaling-skills.md): Three-tier architecture for agents with 300+ skills: always-loaded, resolver-routed, and dormant. Per-turn token math, the v0.41.7.0 compact list-format resolver, and the `gbrain doctor` safety net. 306 skills, ~21K tokens freed per turn, zero capability loss.
- [docs/guides/push-context.md](https://raw.githubusercontent.com/garrytan/gbrain/master/docs/guides/push-context.md): Push-based context: the brain volunteers confidence-gated pages from the rolling conversation window. Three channels (ambient reflex, volunteer_context op, gbrain watch), config knobs, and the volunteered-vs-used feedback loop.
- [docs/mcp/DEPLOY.md](https://raw.githubusercontent.com/garrytan/gbrain/master/docs/mcp/DEPLOY.md): MCP server deployment.
- [docs/TESTING.md](https://raw.githubusercontent.com/garrytan/gbrain/master/docs/TESTING.md): Test command tiers, the test-isolation lint (R1-R4), the canonical PGLite block, withEnv, the E2E DB lifecycle, and the file taxonomy. Maintainer-facing.
- [docs/RELEASING.md](https://raw.githubusercontent.com/garrytan/gbrain/master/docs/RELEASING.md): Full release + contributor process: pre-ship test requirements, the CHANGELOG voice + release-summary template, the 'To take advantage of vX' block, version migrations, GitHub Actions SHA refresh, PR conventions, community-PR-wave. (Ship IRON RULES stay inline in CLAUDE.md.)
## Philosophy
- [docs/ethos/THIN_HARNESS_FAT_SKILLS.md](https://raw.githubusercontent.com/garrytan/gbrain/master/docs/ethos/THIN_HARNESS_FAT_SKILLS.md): Why skills live in markdown.
"build:pglite-snapshot":"bun run scripts/build-pglite-snapshot.ts",
"test":"bash scripts/run-unit-parallel.sh",
"eval:autocut":"bun test test/search/autocut-eval.test.ts",
"test:full":"bun run verify && bash scripts/run-unit-parallel.sh && bun run test:slow && ([ -n \"$DATABASE_URL\" ] && bash scripts/run-e2e.sh || echo '[test:full] skipped E2E (no DATABASE_URL); run docker-compose -f docker-compose.ci.yml up + bun run test:e2e to include' 1>&2)",
prompt+=`You are ${persona.name}, a voice AI. You are NOT a generic assistant. You are NOT Claude. You are NOT GPT. You are ${persona.name} with the personality below.\n\n`;
// 2. Shared context (date/time + identity if authed).
// 2. Shared context (date/time + identity if authed + topic name if summoned).
@@ -32,6 +32,16 @@ The depth of the conversation is the signal. If it's surface-level scheduling, r
This skill is invoked by the host agent's resolver when the operator's voice or text input matches the triggers above. The voice agent (`services/voice-agent/code/server.mjs`) consumes the persona key (`mars`) at session start via `?persona=mars` on the WebRTC `/session` endpoint, OR via the `DEFAULT_PERSONA=mars` env var if Mars is the operator's default.
### Summoning Mars into a topic (#1851)
To call Mars *from inside* a specific conversation topic, mint a per-topic call link by adding `topicId` (a strict slug, `^[a-z0-9][a-z0-9-]*$`) and an optional `topicName`:
Mars boots already knowing the topic's recent conversation. Only the `topicId` crosses the wire — the server resolves the recent-conversation context from `$BRAIN_ROOT/topics/<topicId>.md`. **Never put topic content in the URL** (prompt injection + a leak into history/referrers/logs). No `topicId` → Mars uses his generic live context (unchanged behavior).
@@ -33,6 +33,16 @@ If a question requires multi-paragraph thinking, Venus tees it up briefly and ro
This skill is invoked by the host agent's resolver when the operator's voice or text input matches the triggers above. The voice agent (`services/voice-agent/code/server.mjs`) reads the persona key (`venus`) at session start via `?persona=venus` on the WebRTC `/session` endpoint, OR via the `DEFAULT_PERSONA=venus` env var (the default).
### Summoning Venus into a topic (#1851)
Mint a per-topic call link by adding `topicId` (a strict slug, `^[a-z0-9][a-z0-9-]*$`) and an optional `topicName`:
Venus boots already knowing the topic's recent conversation. Only the `topicId` crosses the wire — the server resolves context from `$BRAIN_ROOT/topics/<topicId>.md`. **Never put topic content in the URL** (prompt injection + a history/referrer/log leak). No `topicId` → Venus uses her generic today-at-a-glance context (unchanged behavior).
## Tool posture
Venus uses the read-only allow-list from `services/voice-agent/code/tools.mjs`:
description: Teaches the host agent WHEN to look something up and WHAT to pull. Ships a policy skill (trigger + retrieval spec) into the host resolver; pairs with the deterministic pointer layer in the context engine.
category: reflex
install_kind: copy-into-host-repo
requires: []
secrets: []
health_checks:
- type: command
argv: [gbrain, doctor, --json]
label: Retrieval reflex wiring (see retrieval_reflex_health)
"description":"src → target mapping consumed by `gbrain integrations install retrieval-reflex`. Policy-only recipe: ships one SKILL.md into the host resolver and appends a resolver row. The deterministic pointer layer lives in the gbrain context engine and needs no install.",
"retrieval-reflex | a named person/company/project/place becomes the subject; a brain-page pointer appears in context; \"who is\", \"what do we know about\", \"tell me about\"; about to assert a non-trivial detail about a named entity"
console.error(`\n${violations.length} violation(s). Fix: bind through $N::text::jsonb (keeping JSON.stringify), or pass a raw object via executeRawJsonb / sql.json. See docs/ENGINES.md.`);
process.exit(1);
}
console.log('check-jsonb-params: clean (no positional $N::jsonb + JSON.stringify double-encodes)');
"Architecture reference. Key files, trust boundaries, engine factory, test layout.",
"Orientation + resolver. North Star, two axes, architecture + cross-cutting invariants, the reference map pointing at on-demand docs, and the inline ship IRON RULES.",
path:"CLAUDE.md",
},
{
title:"docs/architecture/KEY_FILES.md",
description:
"Per-file index for the gbrain repo: what each src/ file does + its load-bearing invariants. The on-demand detail CLAUDE.md's reference map routes to.",
path:"docs/architecture/KEY_FILES.md",
// Link-only until compressed to current-state (still large pre-compression).
// Flip to inlined once the doc-history compression lands and the bundle
// Re-inlined: the CLAUDE.md resolver restructure (per-file index moved to
// docs/architecture/KEY_FILES.md, link-only) freed ~530KB of bundle
// headroom, so this value-explainer rides the single-fetch bundle again.
title:"docs/what-schemas-unlock.md",
description:
"Why schemas matter: 7 killer use cases (4000 invisible meetings, founder ops brain, research brain, legal brain, team brain, agent-as-co-curator) + the structural argument for typed page kinds. Read this before pitching schema authoring (v0.40.7.0).",
"Three-tier architecture for agents with 300+ skills: always-loaded, resolver-routed, and dormant. Per-turn token math, the v0.41.7.0 compact list-format resolver, and the `gbrain doctor` safety net. 306 skills, ~21K tokens freed per turn, zero capability loss.",
path:"docs/guides/scaling-skills.md",
},
{
title:"docs/guides/push-context.md",
description:
"Push-based context: the brain volunteers confidence-gated pages from the rolling conversation window. Three channels (ambient reflex, volunteer_context op, gbrain watch), config knobs, and the volunteered-vs-used feedback loop.",
"Test command tiers, the test-isolation lint (R1-R4), the canonical PGLite block, withEnv, the E2E DB lifecycle, and the file taxonomy. Maintainer-facing.",
path:"docs/TESTING.md",
includeInFull: false,
},
{
title:"docs/RELEASING.md",
description:
"Full release + contributor process: pre-ship test requirements, the CHANGELOG voice + release-summary template, the 'To take advantage of vX' block, version migrations, GitHub Actions SHA refresh, PR conventions, community-PR-wave. (Ship IRON RULES stay inline in CLAUDE.md.)",
| "get more out of gbrain", "is my brain set up right", "weekly brain checkup", "advise me on my brain", "gbrain advisor" | `skills/gbrain-advisor/SKILL.md` |
| Save or load reports | `skills/reports/SKILL.md` |
| "Create a skill", "improve this skill" | `skills/skill-creator/SKILL.md` |
| "Skillify this", "is this a skill?", "make this proper" | `skills/skillify/SKILL.md` |
@@ -82,6 +83,7 @@ This is the dispatcher. Skills are the implementation. **Read the skill file bef
| "Run dream", "process today's session", "synthesize my conversations", "consolidate yesterday's conversations", "what patterns did you see", "did the dream cycle run" | `skills/maintain/SKILL.md` (dream cycle section) |
| "Brain health", "what features am I missing", "brain score" | Run `gbrain features --json` |
| "Set up autopilot", "run brain maintenance", "keep brain updated" | Run `gbrain autopilot --install --repo ~/brain` |
| "Upgrade gbrain", "update gbrain", "gbrain update available", `UPGRADE_AVAILABLE`, "is gbrain up to date" | `skills/gbrain-upgrade/SKILL.md` |
| "strategic reading", "read this through the lens of", "apply this to my problem", "what can I learn from this about", "extract a playbook from" | `skills/strategic-reading/SKILL.md` |
| "concept synthesis", "synthesize my concepts", "find patterns across my notes", "build my intellectual map", "trace idea evolution" | `skills/concept-synthesis/SKILL.md` |
| "idea lineage", "trace the lineage of this idea", "how my thinking about", "how has my thinking about", "what is my current version of", "show reversals in my thinking about", "where did this idea come from" | `skills/idea-lineage/SKILL.md` |
| "perplexity research", "what's new about", "current state of", "web research", "what changed about" | `skills/perplexity-research/SKILL.md` |
| "crawl my archive", "find gold in my archive", "archive crawler", "scan my dropbox for", "mine my old files for" | `skills/archive-crawler/SKILL.md` |
| "verify this academic claim", "check this study", "academic verify", "validate citation", "is this study real" | `skills/academic-verify/SKILL.md` |
@@ -131,4 +134,3 @@ These apply to ALL brain-writing skills:
| "voice note", "ingest this voice memo", "transcribe and file", "voice note ingest", "save this audio note" | `skills/voice-note-ingest/SKILL.md` |
| "add a page type", "add a type to my schema", "schema author", "schema mutate", "schema pack add", "my brain has untyped pages", "propose new types from my corpus", "backfill page types", "evolve my schema", "researcher type", "make X an expert type" (dispatcher for: gbrain schema active/list/show/validate/graph/lint/stats/explain/use/downgrade/reload/init/fork/edit/diff/add-type/remove-type/update-type/add-alias/remove-alias/add-prefix/remove-prefix/add-link-type/remove-link-type/set-extractable/set-expert-routing/detect/suggest/review-candidates/review-orphans/sync) | `skills/schema-author/SKILL.md` |
| "unify my types", "migrate to gbrain-base-v2", "94 types to 14", "apply canonical taxonomy", "clean up my page types", "pack upgrade", "shrink type proliferation", "consolidate page types", "retype pages to canonical" (dispatcher for: gbrain onboard --check, gbrain onboard --check --explain, gbrain jobs submit unify-types, gbrain pages restore) | `skills/schema-unify/SKILL.md` |
Proactive "make the most of gbrain" coaching. Runs `gbrain advisor` on a
cadence and pings the user with the top high-leverage actions for their brain:
version drift, pending migrations, stalled jobs, low embed coverage, setup
smells, and uninstalled brain skills. Read-only; always asks before fixing.
triggers:
- "what should I do to get more out of gbrain"
- "is my brain set up right"
- "gbrain advisor"
- "advise me on my brain"
- "weekly brain checkup"
tools:
- advisor
mutating: false
---
# gbrain Advisor
> **Convention:** See `skills/conventions/brain-first.md`. This skill is the
> proactive voice of the brain — it tells the owner how to run it better.
## Contract
This skill guarantees:
- **Read-only.**`gbrain advisor` never mutates. It computes a ranked list of
actions from existing brain state.
- **Print, never execute.** You SHOW the user the findings and ASK before running
any fix. The user owns every decision.
- **Bounded nagging.** On a cadence, surface only what changed or what's
critical; don't repeat an ignored low-severity item every run.
## When to run
- On demand when the user asks "how do I get more out of this brain?"
- On a **weekly** cadence via the cron recipe below (even idle brains get a
"here's how to run this better" ping).
## How to run it
```bash
gbrain advisor --json
```
Exit code is the severity gate (E2): `0` clean, `1` warn, `2` critical. The JSON
payload is `{ version, generated_at, worst, findings: [...] }`. Each finding has:
- `severity` — `critical` | `warn` | `info`
- `title` — one-line why-it-matters
- `fix.command_argv` — the exact command to fix it (a structured argv)
- `fix.dispatch_id` — present when the fix is safe to run via `--apply`
## What to do with the findings
1. Read the findings, highest severity first.
2. Summarize the top 1-3 to the user in their own channel/voice. Lead with any
`critical` item (e.g. pending migrations).
3. For each, show the `fix.command_argv` and **ask** whether to run it.
4. If they say yes and the finding has a `fix.dispatch_id`, you may run it
locally with an explicit confirm:
```bash
gbrain advisor --apply <dispatch_id>
```
`--apply` is local-only, runs the fix as a structured argv (no shell), and
confirms first. Findings without a `dispatch_id` are not auto-runnable — run
their `fix.command_argv` yourself after the user agrees.
5. Never run a fix the user didn't approve.
## Cron recipe (weekly checkup)
Install a weekly job via the `cron-scheduler` skill. Keep the prompt THIN — the
job just reads this skill and runs the advisor:
- **Schedule:** weekly, one quiet-hours-respecting slot (e.g. Monday 09:00 local).
- **Job prompt:**`Read skills/gbrain-advisor/SKILL.md and run gbrain advisor --json. If anything is critical or new since last run, ping me with the top items and the exact fix commands. Ask before fixing.`
- **Idempotent:** the advisor is read-only, so a double-fire is harmless.
The advisor records a local run history, so on each fire you can tell the user
what is **new since last run** rather than re-listing everything.
## Output Format
When you surface advisor findings to the user, lead with severity and keep it
scannable:
```
🧠 gbrain checkup — 2 things worth your attention
CRITICAL Schema migrations are pending.
Fix: gbrain apply-migrations --yes (want me to run it?)
WARN gbrain 0.44 is available (you're on 0.43).
Fix: gbrain upgrade
```
- One block per finding, highest severity first.
- Always show the exact `fix` command and ASK before running it.
- If nothing is pressing, say so in one line ("brain looks healthy") — don't
manufacture work.
## Anti-Patterns
- **Running a fix without asking.** The advisor is read-only by contract. Never
run `--apply` (or any `fix` command) without the user's explicit yes.
- **Dumping the raw JSON at the user.** Translate findings into their voice; lead
with what matters.
- **Re-nagging ignored low-severity items every run.** Use the "new since last
run" delta; respect the user's prior non-action.
- **Treating `info` like `critical`.** Only block/insist on `critical` findings
(pending migrations). `info` is a gentle nudge.
- **Calling the MCP `advisor` op for workspace install state.** Over MCP the
advisor returns brain-state signals only; uninstalled-skill findings are a
"description":"Post-restart smoke tests + auto-fix for gbrain and OpenClaw environments"
},
{
"name":"gbrain-upgrade",
"path":"gbrain-upgrade/SKILL.md",
"description":"Keep gbrain current: act on the UPGRADE_AVAILABLE marker per self_upgrade.mode (notify prompt or silent auto)"
},
{
"name":"book-mirror",
"path":"book-mirror/SKILL.md",
@@ -189,6 +194,11 @@
"path":"concept-synthesis/SKILL.md",
"description":"Deduplicate and synthesize raw concept stubs into a tiered intellectual map (T1 Canon to T4 Riff), tracing idea evolution across sources over time."
},
{
"name":"idea-lineage",
"path":"idea-lineage/SKILL.md",
"description":"Trace one idea's evolution through the brain: first mention, best articulation, reversals, contradictions, abandoned branches, related concepts, and current live version."
},
{
"name":"perplexity-research",
"path":"perplexity-research/SKILL.md",
@@ -229,6 +239,11 @@
"path":"functional-area-resolver/SKILL.md",
"description":"Compress an agent's routing file (RESOLVER.md or AGENTS.md) by replacing skill-per-row tables with functional-area dispatcher entries. Two-layer dispatch keeps every sub-skill reachable at ~50% of the file size."
},
{
"name":"gbrain-advisor",
"path":"gbrain-advisor/SKILL.md",
"description":"Proactive 'make the most of gbrain' coaching. Runs gbrain advisor on a cadence and pings the user with the top high-leverage actions for their brain. Read-only; always asks before fixing."
@@ -510,7 +539,7 @@ re-suggesting things the user already declined.
- **Asking for the Supabase anon key.** GBrain connects directly to Postgres over the wire protocol, not through the REST API. Only the database connection string is needed.
- **Skipping live sync setup.** If sync doesn't run automatically, the vector DB falls behind and search returns stale answers. Phase H is not optional.
- **Declaring setup complete without verification.** "The command ran" is not the same as "it worked." Push a test change, wait for sync, search for the corrected text.
- **Using Transaction mode pooler.**Sync uses transactions on every import. Transaction mode pooler causes `.begin() is not a function` errors and silently skips pages. Always use Session mode (port 6543).
- **Leaving the direct connection unreachable on IPv4.**GBrain uses the Transaction pooler (port 6543) for reads and a derived direct connection (`db.<ref>.supabase.co:5432`, IPv6-only) for migrations, DDL, and sync transactions. On an IPv4-only host, reads work but sync silently skips pages. Set `GBRAIN_DIRECT_DATABASE_URL` to the Session pooler string (port 5432, IPv4), or enable the IPv4 add-on.
- **Importing without proving search.** The magical moment is the user seeing search find things grep couldn't. Don't skip it.
- **Bundled skills require explicit opt-in.** Skills shipping with gbrain
cannot be auto-mutated; user passes `--allow-mutate-bundled` or
`--no-mutate` (default for the dream-cycle phase) writes proposed.md
for review.
- **Bundled skills require explicit opt-in AND an independent held-out set.**
Skills shipping with gbrain cannot be auto-mutated. To rewrite one in place
the user passes BOTH `--allow-mutate-bundled` AND `--held-out <path>` with
at least 5 benchmark-disjoint tasks; without the held-out set the run
hard-refuses (exit 2). Drop `--allow-mutate-bundled` (or pass `--no-mutate`,
the default for the dream-cycle phase) to write proposed.md for review
instead — no held-out needed for review-only output.
- **Bootstrap output requires human review.** Both `--bootstrap-from-skill`
and `--bootstrap-from-routing` write a sentinel; you must review + STRENGTHEN
the generated judges, delete the sentinel, and re-run with
@@ -127,8 +130,9 @@ attach >=2 rule checks each, save to `skills/X/skillopt-benchmark.jsonl`, run wi
| Skill has a `routing-eval.jsonl` and you want a head start | `gbrain skillopt foo --bootstrap-from-routing` → review the generated tasks → `--bootstrap-reviewed` (routing tasks test dispatch; tighten them into quality tasks before trusting) |
| Iterating on an existing skill | `gbrain skillopt foo --benchmark skills/foo/skillopt-benchmark.jsonl` |
// in a wrapper, etc.) happening between Bun boot and this call is
// invisible to `which` without explicitly forwarding the current env.
// This is why "which gbrain" succeeds when run standalone (fresh Bun
// process, no prior mutation) but can fail from inside autopilot's own
// process at this exact call site. Same fix already applied to
// detectTini() in spawn-helpers.ts (see its comment) — this call site
// was missed.
constwhich=execSync('which gbrain',{
encoding:'utf-8',
stdio:['ignore','pipe','ignore'],
env: process.env,
}).trim();
if(which)returnwhich;
}catch{/* not on $PATH — fall through */}
@@ -109,13 +137,225 @@ export function resolveGbrainCliPath(): string {
returnarg1;
}
thrownewError('Could not resolve the gbrain CLI path. Install gbrain so it is on $PATH (e.g. /usr/local/bin/gbrain), or run autopilot from the compiled binary directly.');
// #2747: include what we actually saw so an operator (or a future bug
// report) doesn't have to guess whether PATH/execPath/argv[1] looked
// sane at the moment of failure.
thrownewError(
'Could not resolve the gbrain CLI path. Install gbrain so it is on $PATH '+
'(e.g. /usr/local/bin/gbrain), or run autopilot from the compiled binary directly. '+
return`You are analyzing one chapter of "${bookTitle}"${authorLine} for the user.
Youroutputisamarkdowntwo-columntablewheretheLEFTcolumnpreservesthechapter's actual content (stories, frameworks, statistics, named examples) and the RIGHT column maps each idea to the user'sactuallifeusingtheirwords,situations,andpatternsfromthebrain.
Youroutputisatwo-columnHTMLtablewheretheLEFTcolumnpreservesthechapter's actual content (stories, frameworks, statistics, named examples) and the RIGHT column maps each idea to the user'sactuallifeusingtheirwords,situations,andpatternsfromthebrain.
|[Detailedparagraph: asection/argumentfromthechapter,preservingstories,stats,frameworks,namedexamples.Use\`<br><br>\` for paragraph breaks within the cell.] | [Specific personal connection: name dates, people, exact quotes from the user, real situations. Same \`<br><br>\` for breaks.] |
<tr><tdvalign="top">[Detailedparagraph: asection/argumentfromthechapter,preservingstories,stats,frameworks,namedexamples.Use\`<br><br>\` for paragraph breaks within the cell.]</td><td valign="top">[Specific personal connection: name dates, people, exact quotes from the user, real situations. Same \`<br><br>\` for breaks.]</td></tr>
@@ -290,6 +293,7 @@ Return ONLY a single markdown section in this exact shape:
-4-10rowsperchapter.Ifasectionhonestlydoesn't apply, write \`*This section is less directly relevant because [specific reason].*\` Don'tforceconnections.
-Nevergeneric("This might apply if you've ever felt...").Neversycophantic.Neverpreach.
-Use\`<br><br>\` for paragraph breaks inside table cells, not literal newlines.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.