The #2339 parity guard's beforeAll runs setupDB (full migration chain)
under bun's default 5s hook timeout, which flaked on a slow CI runner
(setupDB hit 5001ms). Other e2e suites already pass explicit hook
timeouts; bring this file in line.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
On machines with neither gtimeout nor timeout on PATH, run-verify-parallel.sh
and run-unit-parallel.sh fall back to a bg-pid + sleep-watchdog cap. Both
read $? only after tearing the watchdog down (kill + wait on cap_pid), so the
sentinel .exit files recorded the killed watchdog's status — 143 — instead of
the check/shard's own exit code. Every run reported total failure (verify:
pass=0 fail=31; unit: rc=143 per shard) while every per-check/shard log
showed success.
Capture rc immediately after `wait $pid` in both scripts, and reap the
watchdog's sleep child (pkill -P, children-first — the same orphan quirk the
heartbeat cleanup documents) so the fallback stops leaking one sleep per
check/shard.
Regression tests force the fallback branch hermetically on any host via a
curated PATH with no timeout binaries: the verify dispatcher runs from a
tempdir copy with a stubbed `bun`, pinning exit 0 + all-zero sentinels when
checks pass and the check's own rc (not 143) when one fails; the unit wrapper
runs real two-shard fixture passes, pinning rc=0 sentinels and a real
failure's rc=1.
Co-authored-by: paul-0320 <paul@ymyd.co.kr>
Co-authored-by: YMYD <53603073+OJ-OnJourney@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(embed): stamp gateway-resolved model in content_chunks.model, not compiled default (#2846)
upsertChunks fell back to the compile-time DEFAULT_EMBEDDING_MODEL
('zeroentropyai:zembed-1') when a ChunkInput carried no explicit `model`.
The embed pipeline (src/commands/embed.ts) builds ChunkInputs without a
`model` field, so rows whose vectors were produced by the config-resolved
model (e.g. openai:text-embedding-3-large) were mislabeled with the
hardcoded default — corrupting the provenance that signature-drift
staleness and dimension-migration logic depend on.
Both engines now resolve the gateway's runtime embedding model once per
upsert and use it as the fallback, mirroring the existing resolve-then-
default pattern used for schema sizing. Regression test added (pglite);
verified via negative control that it fails against the old fallback.
This is a write-path change (upsertChunks), not a search-path change, so
retrieval eval replay is not applicable.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* test: Lane A.7 pins gateway-resolved chunk model, not compiled default
#2846 changed upsertChunks' fallback from DEFAULT_EMBEDDING_MODEL to the
gateway-resolved runtime model. Lane A.7 still pinned the old fallback,
and the test preload (test/helpers/legacy-embedding-preload.ts) pins the
gateway to openai:text-embedding-3-large for every test process — so the
original #2846 landing failed this test deterministically and got batch-
reverted. The test now asserts the resolved model (the intended #2846
semantics) while keeping the CDX2-4 bare-literal regression guard.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: SailorJoe6 <SailorJoe6@Gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Garry Tan <garrytan@gmail.com>
DashScope's OpenAI-compatible rerank endpoint lives at
{base}/compatible-api/v1/reranks — PLURAL leaf, different base path from
the embedding surface (compatible-mode). Reusing llama-server-reranker
against DashScope forces users to hand-patch the recipe's '/rerank' leaf
in node_modules, which every upgrade silently reverts (and llama.cpp
genuinely serves singular /rerank, so changing that recipe would break
real llama.cpp users).
New dedicated recipe rides the v0.40.6.1 recipe-pluggable reranker path:
- id dashscope-rerank, base_url_default compatible-api/v1 (intl), ZE wire
- path '/reranks', default_timeout_ms 30s, 5MB payload ceiling
- models: only qwen3-rerank (live-verified 200; gte-rerank-v2 is rejected
by the compat surface with 'Unsupported model for OpenAI compatibility
mode', so it is deliberately not listed)
- separate recipe (not a reranker touchpoint on dashscope) because
provider_base_urls is keyed by recipe id and the two capabilities need
different prefixes — same topology as llama-server vs
llama-server-reranker
Tests: recipe shape smoke mirroring recipe-llama-server-reranker.test.ts
(path/timeout/payload pins, /v1/v1 concat guard, auth resolve, sibling
recipe isolation). bun test test/ai/: 322 pass / 0 fail.
Co-authored-by: Yicon <charlieyiconghuang@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
The four `hybridSearch — reranker enabled (reorder)` cases stub the gateway
at 1536 dims (DIMS). Since v0.36.3.0 hybridSearch resolves the embedding
column via loadConfig(), whose precedence is
cfg.embedding_dimensions > gateway dims > default. On any machine whose
~/.gbrain/config.json sets embedding_dimensions to something other than 1536
(e.g. text-embedding-3-small at 1280), the real config outranks the stub: the
1536-d stub vector fails the gateway dim check, the error is swallowed, search
falls back to keyword-only, and the reranker never runs (rerankerFn gets 0
docs, rerank_score undefined). Green in CI only because a fresh runner has no
config file — deterministic red on a contributor's machine.
Fix (test-only): isolate GBRAIN_HOME to an empty tmpdir in beforeAll so
loadConfig() returns null and the stub's dims win, then restore it and clean
up in afterAll. Same idiom as emptyHome() in
test/ai/gateway-probe-chat-model.test.ts.
Verified with a planted ~/.gbrain/config.json at 1280 dims: 2 pass / 4 fail
before, 6 pass / 0 fail after; still green with no config file. typecheck clean.
Fixes#1527
Co-authored-by: Willisbest <132954469+Willisbest@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Reland of #2639, reverted with its batch in 68e4cebd. getHealth's
entity_pages CTE and the top-linked-pages query only match the legacy
'person' and 'company' types, so brains using the gbrain-base-v2 pack's
'entity' type report 0% entity link/timeline coverage in `gbrain health`.
Add 'entity' to both queries in both engines (PGLite + Postgres, in
lockstep per the engine-parity rule).
Reland fix (the batch-red root cause): the original PR's test expected
the entities/project-x page to appear in orphan_pages, but #3023's shared
orphan-reporting policy (landed before #2639 merged) excludes the
'entities' first segment from orphan reporting, so the test failed on
master. Orphan expectations now account for the policy exclusion.
Co-authored-by: Tyler Robinson <tylr.rob@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Replace the strictly sequential per-chunk synopsis loop with a bounded
sliding worker pool (existing runSlidingPool helper). Results land in
chunk order via index-addressed writes; code chunks still bypass the
wrapper; embedding remains one page-level batch after all synopses.
New knob GBRAIN_CONTEXTUAL_CHUNK_CONCURRENCY, default 4, clamped to
[1,16]; 1 reproduces the prior sequential behavior exactly. Each chunk
task still acquires/releases the global synopsis rate-lease, which
remains the cross-worker governor; the lease id now travels from
acquire to release instead of shared mutable state, and lease waits
are abort-responsive.
At 20-45s per synopsis call, a 120-chunk transcript page previously
needed 60-90+ min wall time and routinely outlived job timeouts.
Co-authored-by: spiky02plateau <155588579+spiky02plateau@users.noreply.github.com>
statusForVersion short-circuited on any 'complete' entry before checking
the trailing 'retry' marker, so --force-retry appended an inert row and a
version marked complete with zero work done could never be re-run without
hand-editing completed.jsonl. Check retry-latest first: an explicit
--force-retry now yields 'pending' even past an earlier 'complete', while
a stray 'partial' after 'complete' still cannot regress the version.
Co-authored-by: spiky02plateau <155588579+spiky02plateau@users.noreply.github.com>
SupabaseStorage.getSignedUrl built the download URL as `${projectUrl}${signedURL}`,
but Supabase's sign API returns `signedURL` relative to the Storage API root
(/object/sign/<bucket>/<path>?token=...), so the generated link dropped /storage/v1
and returned 404. Now prepends `${projectUrl}/storage/v1`, tolerating an
already-absolute URL or a value that already carries the prefix. `gbrain files
signed-url` links resolve again.
Co-authored-by: FloridaStyle <daniel.wiggins@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Reasoning models (MiniMax-M3, DeepSeek-R1, etc.) return <think>...</think>
tags in the content field before the actual JSON output. This caused
parseExtractorOutput to fail in two ways:
1. The fence regex /^\`\`\`(json)?...$/ requires the fence at text start;
<think> preceding it prevents matching, so the raw text (with trailing
fences) hits JSON.parse and throws.
2. When think tags contain [ or { characters, indexOf finds them inside
the reasoning block instead of the actual JSON array.
Changes:
- Strip <think>...</think> tags before any parsing (covers all reasoning models)
- Add JSON.parse fallback: truncate at last ] or } to handle trailing
noise (leftover markdown fences after stripping)
Tests: 28/28 pass (3 new cases for think tags + trailing noise).
Co-authored-by: qaz8545355 <603191978@qq.com>
Co-authored-by: qaz8545355 <junjun@openclaw.local>
The empty-fence guard counted every `row_num IS NULL AND entity_slug IS NOT NULL`
row as a pending v0_32_2 backfill, but the inline facts writer keeps producing
rows of exactly that shape post-migration: when a resolved slug has no fenceable
page (slugify-floor / stub-guard-blocked unprefixed slugs like `wingman`,
`people-jane-doe`), backstop.ts falls through to a DB-only insert with row_num
NULL. Those rows are structurally unfenceable — no page to fence onto, and the
ledger-complete migration won't re-run — so they jammed the phase forever
(~16/day observed) and the warning advised a no-op `apply-migrations --yes`.
Discriminator: a row is a genuine backfill candidate only if its entity_slug
resolves to a LIVE page in the same source (EXISTS in `pages` with deleted_at
NULL) — mirroring the migration's Phase B, which only fences slugs that map to
a writable page. Genuine pre-v0.32.2 rows (their entity page exists) still gate;
inline-writer unfenceable rows no longer do. Warning text updated to name the
"entity page present, not yet fenced" condition.
Regression tests pin both sides: unfenceable rows (no page / soft-deleted page)
do NOT gate and the phase converges; a legacy row WITH a backing page still
gates. Fails pre-fix, passes post-fix.
(#2484)
Reland note: original merge (53c90869) was batch-reverted (4b6cf32c) —
the guard-semantics change broke test/phantom-redirect.test.ts
'round 2 P1: legacy-row guard fires BEFORE phantom-redirect pass',
which seeded a legacy row WITHOUT a backing page and expected the
guard to fire. Under the new (intended) semantics such a row is
structurally unfenceable and must NOT gate. Fixed by seeding a live
backing page for the legacy row, preserving what the test pins
(guard fires before the phantom-redirect pass).
Co-authored-by: Javier Aldape <javieraldape@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Code legitimately contains tiktoken special-token strings (e.g. CLIP/GPT tokenizers embed the literal <|endoftext|>). The default encode() uses disallowed_special='all' and THROWS on those, crashing reindex-code on valid source files. Re-encode treating them as ordinary text (allowed=[], disallowed=[]); heuristic fallback if even that fails. A token COUNT needs no special-token semantics.
Co-authored-by: Jim Tang <jimruitang@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(write-through): guard mkdir against EEXIST on Bun+Windows
* fix(budget): resolve non-Anthropic model pricing via canonical table under --max-cost
* fix(enrich): exclude dream-generated pages from thin candidates
---------
Co-authored-by: nguyenchiviet <40517873+nguyenchiviet@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
OpenRouter (and potentially other proxy providers) expose OpenAI's
text-embedding-3 models with a provider prefix in the model ID, e.g.
`openai/text-embedding-3-large` rather than bare `text-embedding-3-large`.
`dimsProviderOptions()` checks `modelId.startsWith('text-embedding-3')`
which fails for the prefixed form, so the `dimensions` parameter is never
sent. The upstream provider returns its native dimensionality (3072 for
-large) instead of the configured value (e.g. 1536), causing an immediate
"dim mismatch" error on first embed.
The default OpenRouter embedding (`text-embedding-3-small` at 1536d)
masked this because its native size happens to match the default config.
The bug surfaces when using `-large`, or `-small` with a non-1536 dim
(512, 768, 1024 — all listed in the recipe's `dims_options`).
Fix: strip the provider prefix before the `startsWith` check. The full
prefixed ID is preserved in the error message for user clarity.
Co-authored-by: Noetherly <280958447+noetherly@users.noreply.github.com>
* feat(subagent): claude-cli MessagesClient adapter (baseline, no tool use)
Closes#334 (partially — text-only baseline; tool use lands in the next
commit on this branch).
Adds a MessagesClient adapter that shells out to `claude --print
--output-format json --model <model>` instead of the Anthropic SDK. When
`GBRAIN_USE_CLAUDE_CLI=1` is set, the subagent worker registers the adapter
in place of the SDK client; the default path (Anthropic SDK with
ANTHROPIC_API_KEY) is unchanged when the env var is unset or set to
anything else.
The benefit is that Claude Max subscribers can run Minions subagents
against their existing OAuth subscription, no ANTHROPIC_API_KEY needed.
New: src/core/minions/handlers/claude-cli-adapter.ts
- Implements the MessagesClient interface exported from subagent.ts.
- Strips provider prefixes (`anthropic:`, `litellm:`) from the model id
because `claude --print` only accepts CLI-native aliases (`sonnet`,
`opus`, `haiku`, or the bare `claude-*-N-M` form).
- Flattens the Anthropic messages array into a single text prompt for
claude-cli stdin. Tool blocks (tool_use / tool_result) are stringified
as placeholders so multi-turn conversations stay coherent in this
baseline; native tool_use round-tripping is the follow-up commit.
- Spawns claude with stdio piped, captures stdout, parses the
`{type:"result", subtype:"success", result, usage, ...}` JSON envelope,
and returns it as a properly shaped Anthropic.Message with
`stop_reason: 'end_turn'`.
- Token totals propagate from the claude usage block so the subagent
handler's `ctx.updateTokens()` reports usable numbers.
- AbortSignal is wired through to SIGTERM the child so the subagent loop's
cancellation path stays correct.
Modified: src/commands/jobs.ts (worker registration)
- Conditionally constructs a MessagesClient via the new adapter when
GBRAIN_USE_CLAUDE_CLI=1.
- Passes it into makeSubagentHandler({ engine, client: subagentClient }).
- Logs `[minion worker] subagent routing via claude-cli (GBRAIN_USE_CLAUDE_CLI=1)`
on startup so the env var status is operator-visible.
Limitations of this commit (addressed in the follow-up):
- Tool use is not yet supported. Tools in params.tools are ignored; the
adapter returns a single text block with stop_reason='end_turn'.
- Token counts come from claude-cli's reporting and may not match the
Anthropic API's accounting precisely (especially for cache tiers).
Original design from #334; this commit preserves that author's attribution.
The follow-up commits on this branch carry the tool-use implementation.
* feat(subagent): tool use + context isolation + convention rename on top of jarvisdoes baseline
Builds on the previous commit (jarvisdoes's #334 baseline) by adding three
things the upstream issue called out as gaps or that surfaced during review:
1. Tool use support via system-prompt-instructed JSON emission.
2. Context isolation flags so claude-cli does not load operator-level
CLAUDE.md, skills, and local project context into every subagent call.
3. Env var rename from GBRAIN_USE_CLAUDE_CLI=1 to
GBRAIN_SUBAGENT_PROVIDER=claude-cli to match the existing
GBRAIN_<noun>_<role>=<value> convention used by GBRAIN_CHAT_MODEL,
GBRAIN_EMBEDDING_MODEL, GBRAIN_EXPANSION_MODEL.
## Tool use
The MessagesClient interface returns Anthropic.Message objects whose
content array may include tool_use blocks. The subagent handler filters
those blocks and dispatches each tool, so any backend that produces
correctly shaped tool_use blocks gets the same loop behavior as the
Anthropic SDK.
The adapter injects a system-prompt addendum describing the tool registry
plus an emission protocol:
<use_tools>
[{"id": "...", "name": "...", "input": {...}}, ...]
</use_tools>
After the response comes back, extractToolCalls() scans for the block,
parses the JSON (tolerant of optional ```json fencing), and converts each
entry into a tool_use content block. Multiple parallel tool calls in one
turn are supported via the array shape; this is the exact case that
breaks today on the codex-proxy / litellm GPT-5.x bridge where parallel
tool-call response IDs get dropped.
Defensive fallbacks:
- Malformed JSON inside the block: drop to text-only, stop_reason='end_turn'.
- Unterminated <use_tools> (no close tag): drop to text-only.
- Model omits id field: adapter synthesizes a toolu_claude_cli_<rand> id.
- Empty response: still hand the subagent loop a well-formed content
array so the .filter chain does not crash.
## Context isolation
claude-cli auto-discovers CLAUDE.md from cwd upward and injects the
operator's skills + plugins + auto-memory into the default system prompt.
On a real install that is ~42-65k tokens of contamination per subagent
call, with both cost and behavioral consequences (the subagent picks up
the operator's coding conventions, opinions, and preferences).
The maximum suppression that still preserves OAuth / Claude Max
subscription auth is:
- Spawn from a dedicated clean cwd (tmpdir-based) so LOCAL CLAUDE.md
auto-discovery has nothing to find. -13k tokens on a real gbrain
install where CLAUDE.md is substantial.
- --disable-slash-commands so skill resolution does not pull in
/skill-name handlers.
- --system-prompt <gbrain prompt> so the default system prompt is
replaced rather than appended to.
The --bare flag would also strip user-level ~/.claude/CLAUDE.md but it
forces ANTHROPIC_API_KEY auth, defeating the whole point of this adapter.
The remaining ~42k cached tokens from user-level instructions are
accepted as a cost-trivial trade-off because the Max subscription absorbs
the per-call cost. Behavioral contamination is mitigated by gbrain's
strong per-call system prompt overriding any operator-level drift.
## Env var rename
Surveyed all ~140 GBRAIN_* env vars in src/. The codebase uses three
patterns: GBRAIN_NO_<feature> (negative toggles), GBRAIN_<noun>_<role>
=<value> (routing keys), GBRAIN_ALLOW_<feature> (permissive toggles).
GBRAIN_USE_* does not appear anywhere except jarvisdoes's original
commit; it would introduce a fourth pattern.
GBRAIN_SUBAGENT_PROVIDER=claude-cli aligns with the routing-keys family
and is value-extensible — adding codex-cli / meridian-proxy / etc. later
means a new value, not a new env var. The scope ('SUBAGENT_*') is also
unambiguous about which calls the toggle covers; GBRAIN_USE_CLAUDE_CLI
was silent on whether it applied to all gbrain LLM calls or only the
subagent path.
Unknown values are rejected with a fail-fast error message naming the
two valid values rather than silently falling through to the default.
## Tests
New file: test/claude-cli-adapter.test.ts — 12 tests, 33 assertions:
- Text-only round trip (single text block, usage propagation, end_turn).
- Provider prefix stripping ('anthropic:claude-sonnet-4-6' -> 'claude-sonnet-4-6').
- Single tool_use parsing.
- Multiple parallel tool calls in one block (the case that triggered
the codex-proxy regression).
- Fenced JSON inside <use_tools> block.
- Model-omitted id gets synthesized to toolu_claude_cli_<rand>.
- Malformed JSON falls back to text.
- Unterminated block falls back to text.
- AbortSignal SIGTERMs the child.
- Error envelope rejected with informative message.
- Non-JSON output rejected with raw-output excerpt in the error.
- argv + cwd assertion: --disable-slash-commands + --system-prompt are
present and cwd is the dedicated tmpdir.
Tests use a POSIX shell stub at GBRAIN_CLAUDE_CLI_BIN that emits a
scripted --output-format json envelope, so the suite runs without
claude-cli installed and without API credits.
* feat(ai): claude-cli recipe with native gateway integration (supersedes #334 baseline)
Replaces the MessagesClient adapter + GBRAIN_USE_CLAUDE_CLI=1 env-var
gate from the previous commit on this branch with a proper gateway recipe.
The recipe path gives per-call routing as a native capability: a model
string like `claude-cli:claude-sonnet-4-6` lands here while a sibling
`litellm:gpt-5.4` continues through the litellm-proxy / codex-proxy path
in the same worker. No global env-var switch, no agent.use_gateway_loop
bypass, no MessagesClient injection at jobs.ts worker startup.
The previous commit on this branch (jarvisdoes baseline) is preserved
in the history for #334 authorship attribution. Its functional changes
are backed out here because the recipe pattern is gbrain's established
integration seam; introducing a parallel MessagesClient + env-var path
would have created two routing mechanisms competing for the same job.
New: src/core/ai/recipes/claude-cli.ts
- Recipe declaration: id 'claude-cli', tier 'native', implementation
'claude-cli', chat-only (no embedding or expansion touchpoints).
- Models: claude-opus-4-7, claude-sonnet-4-6, claude-haiku-4-5-20251001.
- supports_tools and supports_subagent_loop both true.
- supports_prompt_cache false because the CLI handles caching internally
and does not surface cache_control via the standard control plane.
- auth_env.required is the empty array because the CLI owns auth (OAuth
session managed by `claude login`).
- Friendly aliases mirror the `anthropic` recipe: `sonnet`, `haiku`,
`opus` and the same legacy-id rewrites for back-compat with stale
config strings.
New: src/core/ai/providers/claude-cli-language-model.ts
- ClaudeCliLanguageModel class implementing the ai-sdk LanguageModelV2
interface.
- doGenerate: renders the ai-sdk prompt array into a system text + user
text, injects the use_tools protocol instructions when tools are
present, spawns `claude --print --output-format json --model <X>
--disable-slash-commands --system-prompt <gbrain prompt>` from a
dedicated tmpdir (contamination suppression: no local CLAUDE.md
auto-discovery), parses the JSON envelope, extracts <use_tools>
blocks, and returns ai-sdk-shaped LanguageModelV2Content (text +
tool-call parts with stringified-JSON input matching the V2 contract).
- Tolerates fenced JSON inside use_tools blocks, malformed JSON
(falls back to text), missing close tag (falls back to text),
model-omitted ids (synthesizes toolu_claude_cli_<rand>).
- Parallel tool calls in one block round-trip cleanly: this is the
case that drops IDs on the litellm + codex-proxy bridge today.
- AbortSignal SIGTERMs the child for proper cancellation.
- doStream throws not-supported (gateway.toolLoop is non-streaming).
Modified: src/core/ai/gateway.ts
- Adds case 'claude-cli' to instantiateChat (returns ClaudeCliLanguageModel).
- Adds case 'claude-cli' to instantiateExpansion (same wrapper, reserved
for a future expansion touchpoint declaration).
- Adds case 'claude-cli' to instantiateEmbedding (throws, no embedding
model, mirrors the native-anthropic path).
- Lazy require() at the call site keeps the gateway module load cheap
for users who never use the claude-cli path.
Modified: src/core/ai/recipes/index.ts
- Registers `claudeCli` in the ALL[] array next to `anthropic`.
Modified: src/core/ai/types.ts
- Adds 'claude-cli' to the Implementation union so the gateway switch
is exhaustive at compile time.
Reverted: src/commands/jobs.ts
- Drops the GBRAIN_USE_CLAUDE_CLI=1 env-var gate the prior commit
added. Routing now happens at the gateway based on the model string.
Deleted: src/core/minions/handlers/claude-cli-adapter.ts
- The MessagesClient adapter is superseded by the recipe + LanguageModelV2
path. Two routing mechanisms competing for the same job would have
forced users to reason about which one wins; the recipe is the single
source of truth.
New file: test/claude-cli-recipe.test.ts (16 tests, 46 assertions):
- Recipe registration: getRecipe returns chat-only Recipe; aliases map
short names (sonnet/haiku/opus) to canonical model ids.
- Text round trip: single text content block, usage propagation, stop
finish reason.
- Provider prefix stripping.
- Single tool-call parsing.
- Multiple parallel tool calls in one block.
- Fenced JSON inside the block.
- Model-omitted id synthesizes toolu_claude_cli_<rand>.
- Malformed JSON falls back to text + stop reason.
- Unterminated block falls back to text + stop reason.
- Tools offered but model declines: returns text-only with stop reason
so the gateway-loop treats it as a final answer rather than wedging
for tool calls that never come.
- AbortSignal SIGTERMs the child.
- is_error envelope rejected.
- Non-JSON output rejected.
- doStream throws.
- argv + cwd assertion: --print, --disable-slash-commands,
--system-prompt are present and cwd is the dedicated tmpdir.
Tests use a POSIX shell stub at GBRAIN_CLAUDE_CLI_BIN so the suite runs
without claude-cli installed and without API credits.
End-to-end smoke verified against a real `claude --print --model haiku`
invocation: model emitted `<use_tools>` block with toolu_add_001 +
{"a":12,"b":30}, adapter parsed back into a `tool-call` content block,
finishReason 'tool-calls'.
* feat(ai/claude-cli): harden subagent isolation, env scrub, verbose + stdin robustness
Four defensive fixes to the claude-cli provider so a subagent call behaves
identically regardless of the host's ambient Claude Code config:
- Agent isolation: pass `--tools ''` and `--strict-mcp-config` so the subprocess
runs as a raw LLM with no built-in tools and no inherited user MCP servers.
Without `--strict-mcp-config`, each call boots the user's MCP servers (including
gbrain's own), causing recursion plus PGLite single-writer lock contention.
- Env scrub: drop ANTHROPIC_API_KEY / ANTHROPIC_AUTH_TOKEN / ANTHROPIC_BASE_URL
from the child env so the CLI authenticates via its own OAuth subscription
session. An inherited API key silently flips billing to per-token API usage,
the exact setup this recipe exists to replace.
- Verbose-mode compat: with `"verbose": true` in ~/.claude/settings.json,
`--print --output-format json` emits an event array instead of a bare result
object. Tolerate both shapes and select the result event.
- stdin robustness: handle the child stdin 'error' event and wrap write/end so a
missing binary (ENOENT) or early child death (EPIPE) rejects cleanly instead of
crashing the worker with an unhandled error.
Adds unit coverage for the env scrub, the isolation argv, and the verbose event
array. Verified against claude CLI 2.1.x.
* test(ai/claude-cli): cover verbose-array no-result + missing-binary reject paths
Two error branches in the hardened claude-cli provider had no coverage: the
verbose event-array path when no result event is present, and a missing binary
surfacing as a clean spawn-failed rejection. The missing-binary case is the
deterministic form of the stdin/EPIPE robustness; a synchronous stdin-write
throw is not reliably triggerable in a unit test, so the real ENOENT path the
handlers defend is exercised instead. Both reuse the existing shell-stub harness.
---------
Co-authored-by: Brett <brettdavies@users.noreply.github.com>
Co-authored-by: jarvisdoes <258486803+jarvisdoes@users.noreply.github.com>
Co-authored-by: Marco Maldonado <34176133+loweaxerium@users.noreply.github.com>
queue.add() with an idempotency_key returns any existing row regardless
of status. This means dead jobs (exhausted retries from a transient
provider outage) permanently block re-submission of the same work —
even after the underlying issue is fixed.
Fix: when the existing row is dead or cancelled, NULL its
idempotency_key (preserving the row for audit) and fall through to the
INSERT path so a fresh job can be created.
Affects dream synthesize children that died during provider migrations
(429 rate-limit on old Anthropic proxy, tool-results-missing on old
OpenRouter). 45 dead children were blocking re-synthesis of transcripts
in production.
Includes 4 new tests covering dead, cancelled, completed, and active
status interactions with idempotency dedup.
Co-authored-by: Rafael Reis <57492577+rafaelreis-r@users.noreply.github.com>
Co-authored-by: Rafael Reis <rafael.reis@contabilizei.com.br>
`gbrain doctor --remediation-plan` printed two consecutive lines that
contradicted each other when the brain was below target AND the target
was unreachable with autonomous remediation:
Brain score: 45/100 → target 90
Target unreachable: max with autonomous remediation is 70/100.
No remediations needed. Brain is at target.
The second sentence hid the real next step (configure the prereqs that
would lift `max_reachable_score`) and made the brain look healthy when it
was not.
Fix: gate the "Brain is at target" line on `brain_score_current >=
targetScore`. When the plan is empty AND the brain is below target, the
"Target unreachable" line above is already the user-facing explanation;
the `Blocked checks` block below surfaces the manual gap.
Extracted `renderRemediationPlanLines(plan, targetScore): string[]` as a
pure helper alongside `runRemediationPlan` so the regression coverage
asserts on the rendered output directly rather than mocking
`console.log`. `runRemediationPlan` now joins the lines verbatim through
console.log; behavior is byte-identical for every case other than the
fixed contradiction.
Five regression tests cover: unreachable-and-below-target (the bug
case), reachable-and-at-target, exact-target, below-target-with-plan,
unreachable-with-partial-plan. 38 tests across the adjacent doctor test
files stay green; `bun run typecheck` clean.
Co-authored-by: Brett <brettdavies@users.noreply.github.com>
synthesize-concepts.ts's design comment says extract_atoms stamps a
`concepts:` frontmatter field on each atom and :92 consumes ONLY that
field — but the extractor never wrote it, so the atoms → concepts
pipeline was dead end-to-end: every cycle reported "synthesize_concepts:
skipped — no atoms with concept refs" no matter how many atoms
accumulated (696 page-derived atoms / 0 with concepts on our production
brain before an external backfill).
Fix, all on the extractor side (no synthesize change needed):
- EXTRACT_PROMPT asks for `concepts` (1-3 kebab-case TOPIC labels) with
an explicit reuse-over-coinage instruction — labels must cluster,
since synthesize_concepts only materializes groups of >=2.
- parseAtomsResponse validates labels (kebab regex, max 3, drop
invalid; empty -> undefined).
- The putPage frontmatter write stamps `concepts` alongside lesson /
source_quote.
Tests: 4 parse cases + an end-to-end regression that goes extractor ->
real frontmatter -> synthesize_concepts' OWN DB query path -> concept
page. The existing tests fed synthesize via the `_atoms` seam, which is
exactly how this gap survived.
Validated in production ahead of this PR by stamping the same shape
externally: the next synthesize_concepts run wrote 33 concept pages
(T2=7/T3=26) from 60 stamped atoms, zero failures.
Co-authored-by: 陈源泉 <84364275+ChenyqThu@users.noreply.github.com>
Co-authored-by: 陈源泉 <chenyuanquan@chenyuanquandeMac-mini.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(doctor): raw-source persistence guarantee — warn-only v1 (#1978)
Every synthesized/derived page (dream_generated:true or type:synthesis)
must carry a raw trace or an explicit exemption. v1 is warn-only:
- New doctor check `raw_provenance` (brain category) flags synthesized
pages with none of: raw_trace/raw_source/source_uri/raw_trace_exempt
frontmatter, an attached raw_data row, or synthesis_evidence rows.
- Dream synthesize now stamps `raw_source: <transcript path>` into each
written page's frontmatter via the existing #2569 provenance stamp.
- Dream-cycle summary index pages and extract receipts carry an explicit
`raw_trace_exempt: true` + reason (no source document of their own).
No write path is blocked; fail-closed enforcement is the v2 escalation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(doctor): exclude soft-deleted pages from raw_provenance check
Sibling frontmatter checks (quarantined_pages, flagged_pages) filter
deleted_at IS NULL; without it a deleted synthesized page keeps warning
(and its slug keeps being named) through the 72h recovery window with
no way to clear the warn.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
dream.drift.enabled has gated an unwired scaffold since v0.28: runPhaseDrift
had zero call sites, the resolved model + BudgetMeter were discarded
(void modelId; void meter), and no operator-readable output existed.
- Wire 'drift' as a CyclePhase (default OFF via dream.drift.enabled),
ordered after the calibration trio and before embed so the report page
gets embedded same-cycle. PHASE_SCOPE=global, cycle-lock coordinated,
--once (--phase drift --once) bypasses the gate for one run.
- Implement the LLM judge: soft-band candidates (weight 0.3-0.85, active,
unresolved, fresh timeline evidence) are judged against their page's
recent timeline entries via gateway chat; BudgetMeter-gated
(dream.drift.budget, default $1), capped by dream.drift.max_per_cycle
(default 20). Judge model resolves models.drift -> reasoning tier ->
sonnet fallback (unchanged from scaffold).
- Report-only v1: judged candidates land on a reports/drift-<date> page.
dream.drift.auto_update mutates NOTHING; the flag state is recorded in
the report for operators.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(auth): admin-gated rescope surface for DCR clients stuck on default scope (#1914)
DCR clients self-register with source_id='default' + federated_read=['default']
and the registration comment promised 'rescope via the CLI later' — but no
rescope surface existed. Adds:
- GBrainOAuthProvider.rescopeClient(clientId, { sourceId?, federatedRead? }):
single-statement COALESCE update, canonical source-id validation
(assertValidSourceId), FK-backed existence check on the write source,
friendly errors for pre-v60/v61 schemas and unknown clients. Takes effect
for already-issued tokens because verifyAccessToken re-reads oauth_clients.
- gbrain auth rescope-client <client_id> [--source S] [--federated-read a,b]
(trusted local CLI).
- POST /admin/api/rescope-client (requireAdmin), mirroring the existing
register-client / revoke-client admin endpoints.
Deliberately does NOT let clients self-widen scope (options a/b from the
issue) — fail-closed trust invariant.
Fixes#1914
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(auth): rescope-client admin endpoint returns 400 (not 500) for nonexistent write source
The FK-translated 'Source "x" does not exist' error is a client error;
map it to 400 like the sibling validation failures. ('No OAuth client
found' is matched first, so the 404 path is unaffected.)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Pages ingested into a config.federated=true source were invisible to
normal reads: get_page/list_pages scoped to the scalar resolved source
('default'), while the fully UNSCOPED resolve_slugs leaked every
source's slugs — the reporter's exact observation matrix.
- federatedSearchScope now backs get_page, list_pages and resolve_slugs
(not just search/query), so the unqualified read surface shares one
visibility set: grant > federated set > scalar source. resolve_slugs
gains the missing sourceScopeOpts-family scoping (leak sealed).
- The widening gate is now field-presence instead of ctx.remote:
localFederatedSourceIds is populated only by server-side transports
(never from caller params), so trust stays fail-closed while the
stdio MCP transport (no GBRAIN_SOURCE) and the legacy HTTP token
path (no operator-set permissions.source_id grant) can opt their
unqualified callers into the operator-configured federated set.
Tokens WITH a grant, per-call source_id, and OAuth allowedSources
all still win and never widen.
- gbrain sync now attributes its ingest-log row to the synced source
instead of the shared 'default' bucket (attribution sub-bug).
No engine SQL changes: getPage/listPages/resolveSlugs already accept
sourceIds[] in both engines (#1393/#876).
Fixes#3242
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Both engines' searchTakes used whole-string trigram similarity
(claim % query), which structurally cannot pass the 0.3 threshold for a
short keyword against a 100-200 char claim — keyword search returned
zero results on real brains. Switch the predicate to word similarity
(query <% claim) and rank by word_similarity(query, claim), in both
postgres-engine and pglite-engine per the engine-parity invariant.
Holder allow-list and source-scope filters unchanged.
Regression test: single-word query must match a long claim containing
it (fails under the old predicate).
Fixes#3267
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
OAuth/source scoping only guards the serve --http path; a container
sharing Docker's default bridge with the brain's Postgres can open a
direct DB session without a token. Adds a 'Co-located Docker workloads'
subsection to docs/mcp/DEPLOY.md with the operator checklist, a
trust-boundary paragraph in SECURITY.md, and an ops note + cross-link
in the company-brain tutorial.
Fixes#3270
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Rebase of #3279 onto current master: the whoami oauth shape gains
source_id (AuthInfo.sourceId, null when absent) and federated_read
(AuthInfo.allowedSources, [] when absent) — read-only self-introspection
that widens no grant. Re-applied against the post-#3091 description
string (stdio transport shape preserved) and merged the grant tests
into the current whoami.test.ts alongside the stdio cases.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: boundless-forest <boundless-forest@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Agent tools (Claude Code, Codex, …) support a *.local.md counterpart to the
committed CLAUDE.md/AGENTS.md for personal, per-clone instruction overrides that
load after the committed file and are meant to stay uncommitted.
gbrain already ignores other workspace-local agent artifacts (.context/,
.claude/), so this fills the remaining gap for the two root-level override files.
Explicit filenames rather than a *.local.md glob, matching the existing
commented, specific style.
doctor and the get_health CLI surface printed two different timeline
metrics under one ambiguous 'timeline' label: the entity-scoped
timeline_coverage fraction (eligible entity pages with a timeline entry)
and the whole-brain timeline_coverage_score brain-score component (all
pages with a timeline entry, 0-15). Different numerators AND
denominators, indistinguishable in output.
Label-only fix, scoring unchanged:
- graph_coverage check: 'entity timeline coverage N%'
- brain_score breakdown: 'timeline density (all pages) N/15'
- get_health CLI: 'Timeline coverage (entity pages)' plus a new
'Timeline density (all pages): N/15' line when the score is present
Adds test/doctor-timeline-metric-labels-2298.test.ts pinning the
denominator semantics, the rendered doctor messages, and the CLI
guard matrix (fails on master, passes here).
Takeover of #2761.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: TurgutKural <TurgutKural@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(thin-client): map --source/GBRAIN_SOURCE/.gbrain-source onto source_id for routed ops (#2098)
The thin-client route short-circuits before makeContext, so the 6-tier
source resolution never ran and `gbrain query --source X` against a
remote brain sent the unknown `source` key verbatim — the server op
ignored it and searched unscoped.
applyThinClientSourceScope now runs the engine-free tiers (flag → env →
dotfile; DB-backed tiers need an engine, and the server's grant scoping
covers the rest) and sets the op's source_id wire param. Ops declaring
their own `source` param are untouched; an explicit --source on an op
with no source_id wire param errors loudly instead of silently dropping;
explicit --source-id/--all-sources on the wire win over ambient tiers.
Fixes#2098
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(thin-client): use withEnv() instead of direct process.env mutation (test-isolation R1)
CI verify failed on check:test-isolation — thin-client-source-scope.test.ts
mutated process.env.GBRAIN_SOURCE via beforeEach/afterEach. Wrapped each
test body in withEnv() from test/helpers/with-env.ts instead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(thin-client): keep ambient scope out of get_skill's non-scope source_id param
get_skill's source_id is a mode switch (host catalog vs brain-resident-pack
lookup), not a read-scope filter. Ambient GBRAIN_SOURCE / .gbrain-source
injection would silently reroute 'gbrain skill <name>' on thin clients to
getResidentSkillDetail. Exclude it via NON_SCOPE_SOURCE_ID_OPS; explicit
--source-id still passes through, explicit --source errors with a hint.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Garry Tan <garrytan@gmail.com>
* chore(ci): refresh GitHub Actions SHA pins (checkout v4, action-gh-release v2)
Pre-ship pin staleness check per docs/RELEASING.md: both floating major
tags moved upstream; pins updated to the current tag commits.
* v0.42.65.0 chore(release): 92 verified fixes since v0.42.64.0 — changelog + version bump
Aggregates everything merged to master since the v0.42.64.0 bump commit:
community fixes, credited takeovers, batch re-lands, CI hardening, and
maintainer-approved features. Net commit list excludes revert pairs.
No new schema migrations.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(deps): clear OSV-flagged transitive dependencies via override floors
Raise the existing security-floor overrides so the lockfile resolves
patched versions of three transitive packages flagged by the OSV scan
(@hono/node-server, fast-uri, body-parser). None are on gbrain's own
runtime path (@hono/node-server is only referenced by the MCP SDK's
optional hono transport, which gbrain does not load); the floors keep
the dependency scan green. MCP/OAuth unit tests pass against the
resolved versions.
* chore(release): fold #3110 into the v0.42.65.0 entry (93 net changes)
---------
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(links): resolve [[wikilink]] + slug-path frontmatter values; keep frontmatter links fresh on the incremental cycle
Takeover/rebase of two community PRs:
PR #1983 — frontmatter link fields never resolved Obsidian-style values:
- makeResolver step 1's strict slug regex rejected digit-leading folders
(90-people/nicolai) and nested paths (a/b/c); broadened to any slug-shaped
value with an EXACT getPage match only (no fuzzy, no false positives).
- extractFrontmatterLinks resolved "[[dir/slug]]" verbatim; new anchored
unwrapWikilink() strips wholly-wrapped [[...]] (and |alias/#heading/^block)
before resolution. Bare values pass through unchanged.
- Same broadened slug-shape applied to the fs-path synthetic resolver in
extractLinksFromFile (exact Set membership guards it), so the fs
frontmatter path resolves PARA-numbered slugs too.
PR #2434 — the cycle's incremental extract (extractForSlugs) extracted body
links only, so externally-edited YAML (sources:/related:) edges drifted
stale. Adds an includeFrontmatter opt (threaded as a param after sourceId,
which master added in #1747/#1503 after the PR was cut), gated by the new
config key autopilot.incremental_extract_include_frontmatter (default off,
preserves body-only behavior).
Tests: unwrapWikilink unit coverage, broadened-resolver + end-to-end
frontmatter cases in test/link-extraction.test.ts; fs-resolver digit-leading
case in test/extract.test.ts; incremental gate off/on cases in
test/extract-incremental.test.ts.
Co-authored-by: spiky02plateau <spiky02plateau@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(cycle): honor DB-plane config for incremental_extract_include_frontmatter
The gate read loadConfig() (file/env plane) only, but the documented enable
command — gbrain config set autopilot.incremental_extract_include_frontmatter
true — writes the DB plane via engine.setConfig, so the feature could never be
turned on the documented way (silent no-op, #2120 class). Now the file plane
wins when the key is present there; otherwise the DB plane is consulted,
matching the autopilot.auto_drain.* read pattern.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: spiky02plateau <spiky02plateau@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Garry Tan <garrytan@gmail.com>
Adds a `perplexity` embedding recipe (OpenAI-compatible at
https://api.perplexity.ai/v1, auth via PERPLEXITY_API_KEY only — never an
OPENAI_API_KEY fallback) covering pplx-embed-v1-0.6b and pplx-embed-v1-4b.
Perplexity's /embeddings endpoint diverges from OpenAI's wire shape in two
places that break the AI SDK adapter, handled by a new perplexityCompatFetch
shim (mirrors the Voyage/ZeroEntropy pattern incl. the two-layer OOM caps):
- encoding_format only accepts base64_int8/base64_binary; the SDK's 'float'
default is forced to 'base64_int8' outbound.
- The response embedding is base64-encoded signed int8 components (natively
quantized); decoded to number[] inbound so the SDK's Zod schema validates.
Cosine similarity is scale-invariant, so raw int8 components rank correctly.
Flexible dims (Matryoshka-style 128..native max: 1024 for 0.6b, 2560 for 4b)
validate fail-loud in dims.ts + the init preflight; `dimensions` is
Perplexity's native field so no wire translation is needed. default_dims is
1024 (works on a plain vector column for both models); the 4b model's full
2560 width rides the existing halfvec (>2000 dims) storage/ANN path. Pricing
entries land in embedding-pricing.ts.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(schema): make bundled pack inspection truthful (#2029)
Two live bugs:
- parseYamlMini had no block-scalar support, so a 'description: |' swallowed
every following top-level key — the active gbrain-recommended pack loaded
with 0 page types. Add parseBlockScalar for |/|-/|+ and >/>-/>+ in both
mapping and sequence-sibling positions.
- The bundled-pack list was hand-copied in three places (operations.ts had 2
names, mutate.ts had 3, load-active.ts had 7). New single registry
src/core/schema-pack/bundled.ts carries all 7 shipped packs; every
consumer derives from it.
Takeover of #2029, rebased onto master (schema.ts hunks already landed).
Co-authored-by: JiraiyaETH <JiraiyaETH@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(minions): subagent default client resolves config-stored Anthropic key (#2048)
The legacy subagent path constructed a bare new Anthropic() (env-only), so
launchd/MCP workers whose key lives in gbrain config (anthropic_api_key)
failed auth. anthropic-key.ts now exports resolveAnthropicKey() (env first,
then config; hasAnthropicKey delegates) and makeSubagentHandler passes it as
apiKey.
Partial takeover of #2048 — only the auth patch; the path patches were
superseded by the outputRoot mechanism (#2415).
Co-authored-by: JiraiyaETH <JiraiyaETH@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(schema): point bundled-registry test at bundled.ts source of truth
The bundled pack list moved from load-active.ts to bundled.ts in the
truthful-inspection refactor; the T4 registry test still grepped
load-active.ts source. Assert BUNDLED_PACK_NAMES directly instead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(schema): block scalars keep '#' as literal content
Inside a YAML block scalar '#' is content, not a comment; parseBlockScalar
was routing lines through stripComment/isBlank, truncating descriptions
like 'see issue #2029' and blanking comment-looking lines. Use the raw
line inside the scalar. Adds a pinning test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: JiraiyaETH <JiraiyaETH@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
A serve process that wedges mid-boot (e.g. a boot step blocked on an
unreachable upstream) held the PGLite write lock indefinitely — the
post-#2348 lock discipline never steals from a live holder, so every CLI
consumer timed out until the serve PID was manually killed.
runServe (stdio path) now arms a boot-readiness deadline around
startMcpServer: if the transport hasn't connected within
GBRAIN_SERVE_BOOT_TIMEOUT_SECONDS (default 60, 0 disables), it logs the
condition, awaits engine.disconnect() (raced against the existing
5s cleanup deadline so a wedged WASM close can't trap it either), and
exits non-zero so supervisors restart with backoff. A completed boot
clears the timer; the HTTP path is untouched (own lifecycle).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(cycle): propose_takes skips cleanly when the chat provider is unavailable + narrow page projection
Takeover of PR #1979 by @shawnduggan. The original PR gated on a
hardcoded ANTHROPIC_API_KEY heuristic (modelNeedsAnthropicKey defaulting
to true), which master deliberately removed elsewhere — it misclassified
non-Anthropic stacks and fought the tier-config model resolution. This
lands the intent the master-blessed way: probe the RESOLVED chat model
(opts.model ?? getChatModel()) via probeChatModel — same semantics as
patterns.ts / think/index.ts — and skip the phase cheaply when the
provider can't run. Injected extractors are never gated.
Also keeps the PR's uncontested half: load proposal candidates with a
narrow projection (slug, source_id, compiled_truth) instead of
listPages' SELECT p.*, preserving sourceIds > sourceId scope precedence
and updated_desc ordering.
Co-authored-by: shawnduggan <shawnduggan@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(brainstorm): price cost preview against the configured chat model + models.brainstorm.judge config key
Takeover of PR #1855 by @starm2010, shrunk to the brainstorm-only
portion (the cycle-phase hunks are superseded by the resolveModel-in-
phase approach already on master). The cost preview + hard cost ceiling
previously always priced anthropic:claude-sonnet-4-6 even when the
configured chat_model (which the gateway actually runs) was something
else; modelStr now resolves override → config.chat_model → fallback.
The judge phase honors a new models.brainstorm.judge config key when no
--judge-model flag is passed, resolved in the orchestrator so every
caller (brainstorm, lsd, eval-brainstorm) benefits.
Co-authored-by: starm2010 <starm2010@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(test): use withEnv()/emptyHome() in propose-takes no-key tests
check:test-isolation R1 flagged direct process.env mutation in the two
new no-key tests. Swap the hand-rolled save/mutate/restore for the
canonical withEnv() helper (+ emptyHome() for the hermetic GBRAIN_HOME).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(jobs/autopilot): interval persistence, --lock-duration flag, dead-jobs doctor check, deployment-shape docs
Four backlog items in the jobs/autopilot workers, locks & installers area:
- #2794: `gbrain autopilot --install` silently dropped `--interval`. The
installer now parses + validates it, persists it to config
(autopilot.interval), and threads it into the wrapper's exec line; a
later flag-less --install regenerates the wrapper from the persisted
value, and the daemon run path falls back to the same config key.
- #1014: new `--lock-duration MS` flag (env: GBRAIN_LOCK_DURATION) on
`gbrain jobs work` and `gbrain jobs supervisor` to tune the worker
stall-lock window (and so the lockDuration x max_stalled wall-clock
dead-letter cap). Validated like --health-interval (integer >= 1000ms);
the supervisor propagates it to the spawned worker via buildWorkerArgs;
shown in the worker startup banner.
- Takeover of PR #1185 (@ethanbeard): `gbrain integrations doctor` now
surfaces dead minion jobs as a cross-cutting [queue] check. Reworked
from the original: consumes a new machine-readable `gbrain jobs list
--json` surface instead of screen-scraping the human table (long job
names shift the columns), and scopes to a 24h finished_at window so one
ancient dead job can't flag ISSUES forever (parity with main doctor's
queue checks).
- #631: documented the production deployment shape for autopilot vs jobs
supervisor in docs/guides/minions-deployment.md — recommend the
`autopilot --no-worker` + `jobs supervisor` split, warn against running
both worker lanes, and cross-link the --no-worker liveness probe.
Co-authored-by: ethanbeard <ethanbeard@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(autopilot): make #2794 wrapper-script tests hermetic — fake gbrain on PATH
writeWrapperScript calls resolveGbrainCliPath(), which shells out to
`which gbrain` and throws on CI runners where no gbrain binary is
installed. The two new --interval threading tests failed only in CI
(dev machines have gbrain on PATH). Prepend a fake executable to PATH
for the describe block so resolution is deterministic everywhere.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: ethanbeard <ethanbeard@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(init): explicit --embedding-model overrides the persisted --no-embedding sentinel (#2301)
Pre-fix, once ~/.gbrain/config.json carried embedding_disabled: true (the
--no-embedding deferred-setup sentinel), every re-init silently re-deferred
embedding: resolveAIOptions honored the sentinel BEFORE the explicit
--embedding-model flag and never cleared noEmbedding, and the persistence
merge carried the sentinel forward via ...existingFile. Both recovery paths
were dead ends — `gbrain config set embedding_model` is hard-refused
(schema-sizing field), and re-init hit the sentinel.
Fix:
- resolveAIOptions: an explicit --embedding-model / --model flag clears the
sentinel-derived noEmbedding (explicit --no-embedding on the same
invocation still wins — that branch runs after).
- initPGLite + initPostgres persistence: a resolved (model, dims) tuple
drops the stale embedding_disabled key instead of inheriting it.
- assertEmbeddingEnabled message no longer recommends the refused
`gbrain config set embedding_model` command; the working re-init recipe
leads.
Test: test/e2e/init-reinit-after-deferred.test.ts — deferred init then
re-init with an explicit model recovers (sentinel gone, model persisted);
bare re-init still honors the sentinel.
Fixes#2301
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* review: scrub remaining hard-refused `config set embedding_model` advice from init deferred-setup messages
The PR fixed the recovery recipe in assertEmbeddingEnabled but the
deferred-setup lines in initPGLite/initPostgres and the fail-loud
defer hint still pointed users at the Lane C.2 hard-refused command.
Point all three at the working re-init recipe instead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(dream): stamp path-derived sources so --dir runs land cycle freshness (#1869)
gbrain dream --dir <path> (and the configured sync.repo_path fallback)
never wrote last_source_cycle_at / last_full_cycle_at because runCycle's
stamp gate reads opts.sourceId and dream only set it from --source.
Doctor's cycle_freshness stayed perpetually stale on path-scoped brains.
Fix at the command level: dream derives the source id from the resolved
brain dir via resolveSourceForDir (now exported from cycle.ts) and passes
it as opts.sourceId. runCycle's stamp/lock semantics are untouched, so
legacy global callers (autopilot-global-maintenance runs GLOBAL_PHASES
with a brainDir and no sourceId) cannot falsely stamp per-source
freshness — the flaw that sank the runCycle-wide variant in PR #2549.
A derived match on an archived source is skipped (mirrors the explicit
--source archived guard).
Takeover of #2549.
Co-authored-by: javieraldape <javieraldape@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(autopilot): close the engine on SIGTERM/SIGINT instead of hard-exiting (#1872)
systemctl stop (SIGTERM) previously hard-exited autopilot without ever
closing the engine. On PGLite the cycle steps run INLINE in the autopilot
process, so a mid-write exit kills WASM Postgres with the WAL dirty and
can corrupt the brain.
Now both exit paths close the engine first:
- autopilot's own shutdown() (SIGINT + internal stops like max_crashes /
cycle-failure-cap) aborts the in-flight inline cycle via an
AbortController threaded into runCycle, drains it briefly, and awaits
engine.disconnect() before process.exit(0).
- process-cleanup's SIGTERM handler (installed at cli.ts module load,
exits within its 3s cleanup deadline) reaches the same closeEngine via
a registered 'autopilot-engine-close' cleanup callback.
PGLite's disconnect() drains the pending query and checkpoints before
closing; a second call is a no-op, so both paths firing is safe.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(dream): conform dream-dir-source-stamp to canonical PGLite isolation pattern
check:test-isolation R3/R4 flagged the new test file: engine was created
in beforeEach (outside beforeAll) and never disconnected in afterAll.
Switch to the canonical shared-engine pattern (beforeAll create,
beforeEach resetPgliteState, afterAll disconnect) per
test/helpers/reset-pglite.ts.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: javieraldape <javieraldape@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
orphan_ratio's denominator is swamped on auto_chronicle brains by the
machine-generated chronicle events (life/events/<day>-<hash>, written
per eligible event) — no inbound links by design. The shipped policy
already excludes raw/atoms/skills/dreaming/daily and extracts/, but
life/events/ was still counted; on a 1,657-page auto_chronicle brain it
was ~72% of the orphan mass, enough to pin the ratio red.
Add 'life/events/' to DENY_PREFIXES — a scoped prefix, NOT the whole
`life/` first-segment, so human-authored life/diary/ (gbrain capture
--type diary) stays IN the denominator. Same shipped hardcoded-class
mechanism as the existing entries; not the #2215 user-config route
(closed not_planned). Knowledge classes (concepts/people/notes/projects)
also stay in, so genuine graph decay still trips.
Regression in test/orphans-pure-fn.test.ts: life/events/ now excluded
(fails before, passes after); life/diary/ and concepts//notes//projects/
pinned as still-counted. doctor's orphan_ratio uses the same shouldExclude
path (getOrphansData; local + doctor-remote MCP), covered transitively.
zerank-2 is the default reranker under search_mode: tokenmax, but had no
pricing entry — any --max-cost-capped rerank call TX2 hard-failed in
BudgetTracker.reserve() with "no pricing entry".
Adding the entry to EMBEDDING_PRICING alone (the issue's suggested fix)
does not resolve this: lookupPricing()'s rerank branch in
budget-tracker.ts never consulted that table at all, only
ANTHROPIC_PRICING and the FREE_LOCAL_RERANK_PROVIDERS zero-price set.
Verified by reproducing the hard-fail with only the pricing-table entry
added and confirming it still threw.
Fix: add the $0.025/1M-token entry (docs/ai-providers/zeroentropy.md)
and wire the rerank branch to fall back to lookupEmbeddingPrice, reusing
the existing provider:model-keyed table instead of duplicating a third
pricing surface.
Addresses the report in #3223.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
gbrain migrate's per-page copy loop had no failure handling at all: a page
write that threw (e.g. a NOT-NULL column with no protection against the
JS `undefined` postgres.js's UNDEFINED_VALUE guard rejects) crashed the
whole command outright, with no per-page accounting and no way to tell
which page caused it.
Two changes:
- Normalize `undefined` column values to explicit `null` at the migrate
copy boundary before calling putPage. PGLite can hand back `undefined`
for a column that is legitimately NULL/empty; postgres.js rejects a raw
`undefined` bound parameter but accepts `null` fine. This is the root
cause behind the report: a page whose title/compiled_truth/type came
back `undefined` threw mid-insert.
- Wrap the per-page copy in try/catch: failures are tracked (slug +
reason), excluded from the resume manifest's completed_slugs (so a
retry picks them back up), and the run ends with a non-zero exit
verdict + an honest "N copied, M failed" summary instead of a bare
crash or a false "N/N copied" success.
Fixing this properly also required making the pre-existing resume
manifest actually usable without --force (a matching manifest now
bypasses the non-empty-target guard instead of demanding a wipe that
would orphan already-copied pages), always resetting the manifest on
--force regardless of whether the target looked empty, persisting the
manifest before the copy loop starts (so a run where every page fails
after its row lands still leaves a resumable manifest on disk), only
flipping the active config to the target once the migration is fully
clean, and skipping link-copy for slugs known to have failed above
(avoiding an FK-violation crash on the next phase).
Addresses the report in #3194 (reported by @hbohlen).
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Addresses the report in #2662: buildGatewayConfig folded openai_api_key,
anthropic_api_key, zeroentropy_api_key and openrouter_api_key from
~/.gbrain/config.json into the gateway env, but not voyage_api_key. In
launchd/daemon/MCP contexts (no process-env export), multimodal/image
embeds with Voyage failed silently even though config.json looked complete.
- build-gateway-config.ts: fold voyage_api_key -> VOYAGE_API_KEY, mirroring
the existing zeroentropy/openrouter fold (process.env still wins).
- config.ts: add the voyage_api_key file-plane field to GBrainConfig and
KNOWN_CONFIG_KEYS.
- brain-score-recommendations.ts: HOSTED_EMBED_KEY_CONFIG now maps
VOYAGE_API_KEY -> voyage_api_key so doctor/autopilot judge a config-keyed
Voyage brain as usable instead of dispatching a doomed embed job.
- autopilot.ts: the HOSTED_EMBED_KEY_CONFIG producer closure now resolves
hosted keys via the same file-plane source (loadConfigFileOnly) doctor
already uses, instead of the DB plane (engine.getConfig) - the DB plane
is never threaded into buildGatewayConfig for these fields, so reading it
here would let a DB-only key report "configured" while the gateway still
has no key. This also tightens the pre-existing openai/zeroentropy path,
not just voyage.
- Tests: fold + env-precedence tests in build-gateway-config.test.ts,
HOSTED_EMBED_KEY_CONFIG map test, and a real end-to-end regression in
brain-score-recommendations.test.ts through loadConfigFileOnly() and
buildGatewayConfig() with an actual temp config.json.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(frontmatter): stop treating YAML comments inside the fence as markdown headings
autoFixFrontmatter's MISSING_CLOSE repair walked lines from the opening
`---` and broke out of the scan on the first `#`-prefixed line, treating
it as a markdown heading before it ever reached the real closing fence.
A `#` line inside a closed YAML block is a comment, not a heading — but
the scan never got that far, so it inserted a spurious `---` right
before the comment and split valid frontmatter in two, pushing the real
keys (title, pubDate, ...) into the document body.
This is the same bug PR #2153 fixed in the parseMarkdown validator, but
autoFixFrontmatter in brain-writer.ts is a separate reimplementation of
the same MISSING_CLOSE logic that PR never touched. Because
parseMarkdown's validator now parses this shape cleanly, autoFixFrontmatter
is only reachable when some other fixable error (SLUG_MISMATCH, NULL_BYTES,
etc.) also fires on the same file — a common real-world case (e.g. a
renamed file with a stale slug: field) that still corrupts otherwise-valid
frontmatter today.
Fix: scan the full zone for the closing `---` first; only fall back to
the heading-shaped-line heuristic when no closer is found at all.
Addresses the report in #3225. Thanks to @WilliamCourterWelch for the
clear repro and for catching this via git diff before it reached a live
site.
Tests: 4 new regression cases in test/brain-writer.test.ts covering a
YAML comment before the close, comment-only frontmatter, a `#` inside a
quoted string value, and a comment co-occurring with an unrelated real
fix (SLUG_MISMATCH) — confirmed all 3 corruption-covering cases fail
against the pre-fix code (stash/red/restore) and pass after the fix.
The pre-existing genuinely-missing-closer case is unchanged.
bun test test/brain-writer.test.ts test/markdown-validation.test.ts
test/markdown.test.ts test/lint-frontmatter.test.ts
test/doctor-frontmatter-partial.test.ts test/frontmatter-cli.test.ts
-> 122 pass / 0 fail. bun run typecheck -> clean. Full suite intentionally
not run locally (targeted scope per contribution norms); CI covers it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(frontmatter): swap non-exercising regression case per codex review
The quoted-string test (title: "Chapter #1 recap") never exercised the
fixed branch — the heading regex is line-anchored on the trimmed line,
so a `#` mid-string never matched before or after the fix. Replace it
with an indented `#` line inside a YAML block scalar, which does hit
the same closer-first-scan code path as the other regression cases
with a different real-world shape.
bun test test/brain-writer.test.ts -> 27 pass / 0 fail. bun run
typecheck -> clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
extract-atoms-drain's runBatch discarded runPhaseExtractAtoms's per-item
failures/status, so a batch where EVERY provider call errored collapsed to
{extracted: 0, skipped: 0} — indistinguishable from a legitimate no-op. The
drain loop reported status: 'ok' regardless, the Minion handler returned
normally, and the worker marked the durable job complete while the backlog
sat untouched with no retry ever applied.
- runBatch now derives providerFailure from the same counts the phase
already returns (failures.length > 0 && transcripts_processed +
pages_processed === 0 — every attempted item errored, zero succeeded).
Partial success (>=1 item processed) is unaffected.
- The pure loop surfaces this as status/stopped = 'provider_failure',
breaking immediately (same hot-loop guard as no_progress) instead of
letting a final remaining===0 recount silently overwrite it to 'drained'.
- The extract-atoms-drain Minion handler throws when it sees
status === 'provider_failure', so the worker's ordinary failJob path
(attempt+backoff, dead-letter on exhaustion) takes over. The
LockUnavailableError -> deferred path is unchanged.
- autopilot's auto-drain submission bumps max_attempts from 1 to 3 (queue
default) — with the handler now actually throwing, max_attempts:1 meant
the first provider blip dead-lettered instantly with no backoff attempt.
Tests: pure-loop provider_failure propagation (incl. the remaining===0
precedence case), runPhaseExtractAtoms's all-items-fail counts contract,
and source-shape guards on the handler throw + autopilot max_attempts.
Full suite deferred to CI per repo convention (targeted run: 104 pass / 0
fail across the touched + adjacent extract-atoms/drain/autopilot files;
`bun run typecheck` clean).
Two rounds of codex review (gpt-5.6-sol, high effort): round 1 flagged
autopilot's max_attempts:1 and the stopped-precedence bug (both fixed
above); round 2 confirmed no new issues.
Thanks to @aaronkhawkins for the detailed report. Addresses the report in
#3218.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(sync): report partial (pull_failed) instead of up_to_date when git pull fails with zero imports (#3068)
A warn-and-continue internal git pull failure (e.g. a local-path origin
rejected by protocol.file.allow=never) combined with a zero-import run
previously reported `up_to_date`, exited 0, and bumped the last_sync_at
freshness heartbeat. A permanently-failing pull was therefore invisible
forever: doctor's sync_freshness never fired and every scheduled sync
looked clean while the source silently went stale.
Now, when the pull failed and the run imported nothing, sync returns
`partial` with the new reason `pull_failed`, leaves last_commit AND
last_sync_at untouched (so staleness monitoring fires), and prints a
dedicated non-success message. The fall-through-to-working-tree design
is unchanged: local commits still import when the remote is unreachable,
and the anchor still advances over commits that were actually imported.
Addresses the report in #3068.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(sync): surface pull_failed to CLI exit codes, sync --all JSON, and the cycle phase (#3068 review round)
Codex review round 1 follow-ups:
- Single-source `gbrain sync` sets exit code 1 on partial/pull_failed
(timeout-class partials keep exit 0 — they converge on retry; a failing
pull does not).
- `sync --all` exits 1 when any source reports pull_failed, and the
--json envelope carries the per-source partial `reason`.
- The autopilot cycle's sync phase maps partial/pull_failed to `warn`
with a dedicated summary and a `syncReason` detail, so a scheduled
cycle no longer reports a clean run over a wedged source.
- The regression test now isolates GBRAIN_HOME to a temp dir so the
first full sync cannot touch the real sync-failure ledger.
- Current-state docs: KEY_FILES.md sync.ts entry + TESTING.md inventory
describe the pull_failed contract and the new test.
Addresses the report in #3068.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(sync): route pull_failed exit through the owned verdict channel; make the regression test serial (#3068 review round 2)
Codex review round 2 follow-ups:
- Single-source exit now uses setCliExitVerdict(1) instead of a raw
process.exitCode assignment, which the CLI teardown deliberately
ignores (PGLite's Emscripten runtime clobbers process.exitCode
mid-run; the owned channel in src/core/cli-force-exit.ts is the only
trusted verdict). Pinned by test/cli-exit-verdict-pin.test.ts.
- The regression test is renamed to *.serial.test.ts because it pins
GBRAIN_HOME for the whole file (scripts/check-test-isolation.sh R1);
docs updated to the new name.
Addresses the report in #3068.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Keep each page excerpt within the existing fixed budget while selecting the window with the strongest query-term coverage. Preserve leading truncation for callers without a matching question.
* fix(migrate): drop invalid CONCURRENTLY-build remnants without a DO block
Migration v66 (embed_stale_partial_index) pre-drops an invalid index left
over from a previously interrupted CREATE INDEX CONCURRENTLY using
DO $$ BEGIN ... EXECUTE 'DROP INDEX CONCURRENTLY IF EXISTS ...'; END $$.
Postgres rejects CONCURRENTLY from any function/EXECUTE context, so the
guard's EXISTS check passes but the EXECUTE inside it always throws
"DROP INDEX CONCURRENTLY cannot be executed from a function" -- the
migration only fails on brains carrying an invalid-index leftover.
Add dropInvalidConcurrentIndex(): the validity probe runs as a plain
application-level SELECT, and the DROP runs as its own top-level
runMigration call instead of inside a DO block. Fixes#1178.
* fix(migrate): address codex review — schema-safe index resolution + OID-based no-op assertion
- dropInvalidConcurrentIndex(): resolve indexName via to_regclass() (search_path
resolution, same as the unqualified DROP that follows) instead of matching
pg_class.relname bare, which could hit a same-named index in a different
schema on a non-default search_path.
- e2e test: the no-op re-run case now compares index OID before/after, not
just validity -- validity alone wouldn't catch a spurious drop+recreate.
The conversation-fact extractor renders turns as `${speaker} (${ts}): ${text}`
and its `confidence` field scores confidence-in-the-CLAIM, not confidence-in-
WHO-said-it. So a first-person self-assertion from an anonymous speaker
("Speaker A: I'm joining Acme") could come back with the anonymous label echoed
as `entity` — a confident attribution to a person we cannot identify. That
label is then stored verbatim as the fact's `entity_slug` (the batch insert
path does no canonicalization), polluting entity-scoped queries and the
top_entities aggregation, or misattributing the claim.
Add a deterministic gate (`isUnknownSpeakerLabel`) at the single candidate-loop
choke point that nulls ONLY that self-referential attribution, plus one
EXTRACTOR_SYSTEM rule telling the model not to guess a name for anonymous
first-person turns. Third-person entities from the same turn ("Acme raised $5M"
-> entity=acme) and named-speaker attributions are untouched. The fact itself
is always preserved; only the bad attribution is dropped.
The P1 entry asked for fail-closed semantics in resolveTakesSourceId
(src/commands/takes.ts). That landed in #2973 (merged 2026-07-20):
the function now delegates straight to resolveSourceId with no
catch-and-fallback, so an unresolvable explicit source throws instead
of silently restoring the pre-#2698 unscoped cross-source write path.
Regression tests for the invalid-source path shipped in the same PR.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
discoverGitRoot() probes `git rev-parse --show-toplevel` to locate the repo
root; a miss is expected/routine (a non-git-yet brain dir, a scratch dir) and
is either self-healed via auto git-init or surfaced as a friendlier Error.
Node's execFileSync writes the child's stderr straight to the parent's real
stderr by default unless an explicit `stdio` array is given, so every routine
probe miss dumped git's raw "fatal: not a git repository ..." line into
gbrain's operator logs -- indistinguishable from an actual crash to an
operator grepping logs for "fatal:" as a crash signature.
Add an opt-in `silenceStderr` param to the shared `git()` helper (sets
`stdio: ['ignore', 'pipe', 'pipe']`, which disables the implicit
passthrough-to-parent-stderr behavior) and pass it only from
discoverGitRoot's internal probe call. Every other `git()` call site is
unchanged, so unexpected-failure visibility elsewhere is preserved.
Related to #2964, which added the auto-recovery this probe feeds.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
`getMemoryUsage()` in src/core/backoff.ts computes 1 - freemem()/totalmem(),
where Node's `os.freemem()` returns Linux's `MemFree`. `MemFree` excludes the
page cache, which the kernel grows aggressively in any environment that reads
files (i.e. essentially all containers). On a healthy 4 GB Linux container with
~1.4 GB MemAvailable, MemFree is routinely ~100 MB, so `getMemoryUsage()`
reports 96-97% used and `waitForCapacity()` rejects every job with:
Throttle timeout: system overloaded after 20 attempts (~600s).
Load: ..%, Memory: 97%
even though the host has plenty of usable memory.
Linux exposes `MemAvailable` in `/proc/meminfo` precisely as the kernel's
estimate of memory available for new allocations without swapping (it factors
in reclaimable page cache). This is what `htop` and `free -h` show as
"available". Using it removes the false positive entirely.
Behaviour:
- On Linux (when /proc/meminfo is readable): use 1 - MemAvailable/MemTotal.
- Anywhere else (macOS, Windows, sandboxed envs without /proc): unchanged
fallback to 1 - freemem()/totalmem().
Scope is intentionally minimal — data correctness only. An env override like
GBRAIN_MEMORY_STOP_PCT would also be reasonable but is out of scope here.
Co-authored-by: Kevin Hsu <kevinhsu.ecofirst@gmail.com>
mechanical.test.ts shells out to `gbrain init --non-interactive`,
`gbrain import`, and similar commands via Bun.spawnSync. The four
`cliEnv()` helpers in this file forward `process.env` unchanged, so
`gbrain init` ends up calling saveConfig() against the developer's real
$HOME/.gbrain/config.json, overwriting their production database_url
with the test container's URL on every `bun run test:e2e` invocation.
Sibling test/e2e/migration-flow.test.ts already solved this with a
module-level temp HOME and an afterAll restore. Mirror that pattern in
mechanical.test.ts.
Verified by md5'ing ~/.gbrain/config.json before and after running the
Setup Journey, Init Edge Cases, Schema Idempotency, RLS Verification,
Doctor Command, and Parallel Import describe blocks — config hash is
identical pre and post (26 passing tests, 0 failures, 0 mutations to
the user's real config).
Co-authored-by: Seth Armbrust <setharmbrust@seth.local>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(recipes/x-to-brain): use /users/by/username for app-only bearer health check
Takeover of #2343. /users/me requires user-context OAuth and always fails
under the app-only bearer the recipe collects. Health check + setup curls
now use /users/by/username/$X_HANDLE, with X_HANDLE declared in secrets
so the installer prompts for it. Recipe version 0.8.1 -> 0.8.2.
Co-authored-by: ethanbeard <ethanbeard@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(cycle): bound propose_takes with per-call timeout + phase deadline
Takeover of #2262. The extractor's gateway.chat call had no abortSignal, so
one stalled provider socket could pin the phase for the 300s gateway default
per page; the nightly wrapper then SIGTERMed the whole phase mid-run. Each
extractor call is now bounded at 90s (per-page failure already logs a warning
and continues), and the page loop carries a 30-min wall-clock deadline that
breaks cleanly into a partial result with deadline_hit:true + warn status.
Unlike the original PR, the default pageLimit stays at 100 — shrinking it to
30 was an unrelated product-knob change that would permanently cut nightly
take coverage.
Co-authored-by: tschew72 <tschew72@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(capture): make fallback title truncation explicit and astral-safe
Takeover of #2310. deriveTitle's silent .slice(0, 80) could split an astral
surrogate pair mid-character and gave no signal the title was cut. Truncation
is now codepoint-aware and appends an ellipsis (still capped at 80 codepoints).
Unlike the original PR, this stays a three-line change: no whitespace
normalization of every derived title, no word-boundary heuristics.
Co-authored-by: xd-Neji <xd-Neji@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(doctor): clear extract_atoms raw source-holder backlog + normalize pooler direct-URL overrides
Takeover of #2242, split to the two concerns that survive review:
- extract_atoms: exclude source pages whose frontmatter declares a raw
payload pointer from discovery AND the doctor backlog count (shared SQL
fragment so they can't drift). Extraction on these yields zero atoms, so
no atom row is ever written and they re-enter the backlog every cycle —
a permanent no-progress doctor blocker.
- connection-manager: a direct-URL override (opts/env) that still points at
the Supavisor TRANSACTION pooler (port 6543, usually a copy-paste of the
primary URL) is normalized to the real direct host via deriveDirectUrl.
Session-mode pooler overrides (port 5432) pass through — they are a
legitimate direct-ish target, which the original PR would have nulled out.
Dropped from the original PR: orphan-reporting atom exclusions (master
already excludes atoms/ and raw/ first segments plus /raw/ segments in
src/commands/orphans.ts) and the drain dry-run status tweak.
Co-authored-by: benjonp <benjonp@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(cycle): record propose_takes deadline break as a halt in the extract rollup
A deadline-hit run breaks the page loop mid-list — same posture as budget
exhaustion — but the rollup still counted it as a completed round with no
halt, hiding chronic never-finishing nightly runs from extract-status/
doctor. Treat deadline_hit like budget_exhausted in the rollup deltas;
deadline test now pins halt=1 / completed=0.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: ethanbeard <ethanbeard@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: tschew72 <tschew72@users.noreply.github.com>
Co-authored-by: xd-Neji <xd-Neji@users.noreply.github.com>
Co-authored-by: benjonp <benjonp@users.noreply.github.com>
* fix(cycle): drain PGLite synth subagents inline (takeover of #2699)
PGLite holds an exclusive file lock on its embedded data-dir, so no
separate Minions worker can serve the subagent children the synthesize
phase enqueues — they sat in 'waiting' until waitForCompletion timed
out. Drain a private per-run child queue inline (claim → run →
complete/fail, plus the promote/stall/timeout housekeeping a worker
would perform). No-op on Postgres, where children stay on the shared
'default' queue.
Rebased onto the reworked synthesize (config.subagentTimeoutMs, #1586
source scoping): the inline job context now carries deadlineAtMs from
the claim-time timeout_at stamp, and opts.yieldDuringPhase is ticked on
a 60s keepalive while each child runs so the 5-min cycle lock TTL
refreshes across long (up to 30-min) children.
Co-authored-by: TheRealMrSystem <TheRealMrSystem@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(lint): --exclude flag for mixed-content repos (takeover of #2649)
Adds --exclude=a,b (and LintOpts.exclude) so mixed-content repos can
skip software trees and repo metafiles by basename when collecting
pages. The only built-in default is node_modules — vendored dependency
trees are never knowledge pages; dot/underscore entries were already
skipped by the walk.
Diverges from #2649 deliberately: the original hardcoded an opinionated
default list (README.md, CHANGELOG.md, CLAUDE.md, test/ dirs at any
depth, plus fork-specific filenames), which silently changed lint
counts for every existing repo. Those are repo policy — pass --exclude.
Co-authored-by: ryangu00 <ryangu00@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(cycle): enforce per-job timeout_ms in the PGLite inline subagent drain
The inline drain claimed children with deadlineAtMs derived from timeout_at
but never armed the worker's timeout timer — and the handleTimeouts sweep
only runs between jobs, so nothing could stop a child that blew past its
30-min timeout_ms. A hung LLM call wedged the drain loop (and the whole
cycle) indefinitely, with the 60s keepalive refreshing the cycle lock
forever. Worker.ts parity: arm a timer from the claim-time timeout_at
stamp, abort ctx.signal on fire, and dead-letter (never delayed-retry)
timed-out children, mirroring handleTimeouts' stall→retry / timeout→dead
split.
Regression test: a child with timeout_ms=100 whose handler only ends on
ctx.signal abort is dead-lettered with 'timeout exceeded'; pre-fix the
test hangs to its 30s timeout.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: TheRealMrSystem <TheRealMrSystem@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: ryangu00 <ryangu00@users.noreply.github.com>
Takeover of #2525, rebased onto current master. getHealth() in both engines
now computes orphan_pages and the timeline component over a linkable_pages
CTE driven by the same constants the orphans audit uses
(src/core/linkable-scope.ts), so one doctor report can no longer show a 19%
orphan_ratio next to a no-orphans score implying ~70%. Master's newer
first-segment exclusions (raw, atoms, skills) are folded into the shared
scope so the orphans audit loses nothing in the move.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: pabloglzg <pabloglzg@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Takeover of #1988 (OpenRouter prompt caching), reimplemented on current
master: supports_prompt_cache may now be a per-model-id predicate; the
OpenRouter recipe marks openai/* chat and anthropic/claude-* routes
cacheable. Claude routes get an explicit cache_control on the system
content block via the recipe compat fetch shim (OpenRouter's documented
per-block format, not a top-level body field), signaled through a private
in-process marker header instead of the promptCacheKey sentinel that now
collides with the real OpenAI prompt_cache_key derivation. Cache reads on
OpenAI-compatible routes surface via the SDK's cachedInputTokens.
Root fix for #1135: deepseek, groq, and together now declare expansion
touchpoints (their expansion path is the same plain OpenAI-compatible
languageModel call as chat), so an explicit expansion_model pointed at
them no longer silently yields zero expansion.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: tmchow <tmchow@users.noreply.github.com>
Co-authored-by: warkcod <warkcod@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
putPage lowercases slugs via validateSlug, but upsertChunks queried
pages by the caller's raw slug — so a mixed-case slug through
importFromContent created the page row, then failed the chunk upsert
with 'Page not found' and rolled back the whole import.
Normalize via validateSlug at importFromContent entry and inside
_upsertChunksOnce on BOTH engines (postgres + pglite parity).
Takeover of #855, rebased onto current master shapes (batchRetry
wrapper / _upsertChunksOnce, rewritten importFromContent opts block).
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Kage18 <Kage18@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
A source registered with `gbrain sources add --federated` was invisible to
an unqualified `gbrain search`/`gbrain query`: the local CLI always emitted
a scalar {sourceId} scope, and nothing on the read path ever consulted
sources.config.federated — contradicting docs/guides/multi-source-brains.md
('Source participates in unqualified gbrain search results').
Fix, at the trusted-local boundary only:
- src/cli.ts makeContext resolves the source WITH its tier and, when the
tier is non-explicit (local_path / brain_default / sole_non_default /
seed_default), computes ctx.localFederatedSourceIds = [resolved source,
...other config.federated=true sources] (archived excluded).
- New federatedSearchScope (operations.ts) delegates to
resolveRequestedScope, then widens an unqualified trusted-local scalar
scope to that set. Used by the search + query handlers only.
- Expansion NEVER applies when ctx.remote !== false (fail-closed source
isolation), when a per-call source_id/__all__ is passed, when an OAuth
grant (allowedSources) is present, or when --source/GBRAIN_SOURCE/dotfile
named the source explicitly.
Deliberately NOT inside sourceScopeOpts: code-intel ops reject multi-source
scopes (resolveCodeIntelScope) and non-search reads keep their scalar
behavior. Cache contamination is already handled — cacheScopeKey folds
sourceIds sets into the query-cache key.
Fixes#2561
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
- #1035: importFromContent preserves an existing page's type when incoming
frontmatter omits an explicit type: field. Explicit type stays an override;
absence means preserve; new pages still path-infer. The existing-page fetch
moved above the content-hash compute so a no-op re-put stays a hash-match
skip. Root-cause fix covers put_page, sync, capture — every caller.
- #1284: sync's end-of-run auto-embed no longer receives slugs deleted in the
same run (embedPage threw 'Page not found' per deleted slug and serr-logged
noise on every rename/delete sync). pagesAffected stays the full manifest
for extract/report paths; a slug deleted then re-imported in the same run
stays embeddable.
- #1196: gbrain serve --http now runs doctor's embedding_width_consistency
check at startup and prints a loud stderr banner (with the paste-ready
recipe + GBRAIN_EMBEDDING_MODEL/DIMENSIONS hint) when the resolved width
diverges from the brain's vector(N) column — the stateless-container
fallthrough that broke every write. Fail-open; reads unaffected.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Missing ZEROENTROPY_API_KEY threw AIConfigError from auth resolution, which
rerank.ts recorded as reason 'unknown' — and doctor's reranker_health had no
unknown bucket, so it reported ok while every rerank silently failed open.
- gateway.rerank wraps AIConfigError from applyResolveAuth as
RerankError(reason: 'auth') before any HTTP call.
- checkRerankerHealth warns on >=3 'unknown' failures in the 7-day window
(covers historical pre-fix audit rows), with a ZEROENTROPY_API_KEY setup
hint when the error summary points at a missing key.
- Tests: RerankError(auth) classification, applyReranker fail-open + audit
reason, doctor warn on repeated unknowns.
Takeover of #2070.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: maxpetrusenkoagent <maxpetrusenkoagent@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
fetchCountRows and detectDeadPrefixes in src/core/schema-pack/stats.ts
swallowed EVERY engine error into empty results, so any real failure
printed 'Total pages: 0' + a vacuous 100% coverage on a populated brain.
Both catches now swallow only isUndefinedTableError (pre-init brain,
missing pages table) and rethrow everything else. Four regression tests:
real non-zero count on a populated PGLite brain, rethrow on non-missing-
table errors in both catch sites, and the missing-table degrade path.
Takeover of #2493.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: javieraldape <javieraldape@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
resolveTodayTasks only matched a bare '## Today' heading and bold-prefixed
'- [ ] **task**' lines, while the daily-task-manager skill's documented
Output Format writes '## P1 — Today' with plain '- [ ] task' lines — so
documented writes surfaced zero tasks in live context.
Reader now accepts both heading forms and both line forms, two-step: the
legacy bold prefix extracts just the task name (dropping trailing metadata),
falling back to the plain full-line form.
Salvaged from PR #2188 (reader-side half). The skill-doc rewrites in that PR
are dropped: master #2938 kept ops/ synced and made put_page write-through
durable, so the 'gbrain get/put ops/tasks' docs are correct as-is. The PR's
single-regex line matcher is replaced with the two-step match because its
alternation captured '**name** — metadata' verbatim for bold lines.
Takeover of #2188. Fixes#2186.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: caioribeiroclw-pixel <caioribeiroclw-pixel@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
On macOS/Windows (case-folding filesystems), the write-through rename
silently clobbered a differently-cased file already occupying the target
path (uncontrolled repo files like README.md vs slug readme, or unicode
normalization variants between slugs). Refuse with
skipped: 'case_insensitive_collision' when the path exists on disk but no
exactly-named directory entry does; exact-case updates fall through and
case-sensitive filesystems are unaffected.
Fixes#2831
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The Dream cycle disables sync's inline extraction and routes changed
slugs through extractForSlugs, which flushed link/timeline batches but
never stamped links_extracted_at — so incrementally extracted pages
stayed permanently visible to `extract --stale` / doctor.
Collect processedRefs per successfully processed page and stamp them
via stampExtracted (best-effort) after both batch flushes, non-dry-run
mode 'all' only. Source-id threading from the original PR #2637 already
landed on master via #1503/#1747, so this rebase carries only the
missing watermark stamp plus regression tests.
Takeover of #2637.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: JavanC <JavanC@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Adds GBRAIN_OAUTH_TOKEN_RATE_LIMIT_MAX and
GBRAIN_OAUTH_TOKEN_RATE_LIMIT_WINDOW_MS to tune the /token
client_credentials limiter (default unchanged: 50 req / 15 min).
Invalid, zero, or negative values fall back to the default.
Takeover of #2501 (mechanical rebase onto master after #2625 shifted
the surrounding context in serve-http.ts). Fixes#2463.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: techtony2018 <techtony2018@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The cycle's embed phase called runEmbedCore with no output suppression, so
the '[dry-run] Would embed ...' / 'Embedded N chunks ...' slog summaries
landed on stdout ahead of the JSON CycleReport, breaking the documented
stdout-clean-for-JSON contract (docs/progress-events.md).
Adds EmbedOpts.quiet gating the human stdout summary slog sites in
embed.ts (embedPage, embedAll, embedAllStale); the cycle's runPhaseEmbed
sets quiet: true since it reports counts via its own PhaseResult. Errors
and warnings still go to stderr regardless.
Takeover of #854 (same approach, reimplemented on current master — the
original patch predates the slog migration and the widened
embedAll/embedAllStale signatures). Regression test ported from #854.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Kage18 <Kage18@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Takeover of #2161. runRemediation ignored onboard-check extras in three
places: the pre-flight plan, the initial recommendation build, and the D7
mid-run recheck that rebuilds recs after every completed step. The --check
path threaded extras correctly, so `gbrain onboard --apply --auto` reported
"Nothing to do" when the only remediable work came from onboard checks —
and even with the first two sites fixed (the original PR diff), any plan
with 2+ steps dropped all remaining extras after step 1 via the recheck.
- Add RemediationOpts.extraRemediations; thread it through the pre-flight
plan, initial recs, and the mid-run recheck.
- Recheck filters extras to ids not already processed this run: extras
carry static status:'remediable', so unfiltered threading would resubmit
completed extras forever.
- Wire the CLI --auto path (onboard.ts) AND the MCP run_onboard auto path
(operations.ts), which already computed the scope-filtered allowedExtras
and then dropped it.
- Regression test: extras-only plan on an empty brain runs BOTH extras
exactly once and terminates (serial file: mock.module queue stub +
GBRAIN_HOME tmpdir).
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: brettdavies <brettdavies@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(autopilot,eval): nightly quality probe enable path works end-to-end + wire conversation-parser probe
Takeover of #2629 and #2630 (rebased onto master; dropped the
test/engine-find-trajectory.test.ts hunk both PRs carried — master
already ships the equivalent gateway-dims fix).
#2629 — nightly quality probe enable path:
- autopilot + doctor read the probe flag dual-plane (DB config row from
'gbrain config set' wins, ~/.gbrain/config.json fallback) via new
resolveProbeEnabled/resolveProbeMaxUsd helpers
- resolveRepoRoot prefers the gbrain package root where the committed
fixture lives, not the brain repoPath
- rate_limited skips no longer write an audit row every autopilot cycle
- eval-longmemeval strips 'provider:' recipe ids before raw Anthropic SDK
calls and emits the gold answer for downstream judges
- cross-modal batch folds the gold answer into the judge task; probe
passes QA-shaped dimensions instead of the agent-response rubric
- DEFAULT_SLOTS slot A moves to openai:gpt-5.2 (gpt-4o left the recipe);
new consistency test pins every default slot to its recipe
#2630 — conversation-parser nightly probe wire-up:
- autopilot step 4.6 invokes runConversationParserNightlyProbe (dual-plane
flag + D10 tokenmax mode-gate, package-root fixtures, 24h gate, audit
trail via new src/core/audit-parser-probe.ts)
- doctor's conversation_parser_probe_health replaces the hardcoded
'Skipped' stub with a real pure-function check over the audit trail
Co-authored-by: p3ob7o <p3ob7o@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(pricing): add openai:gpt-5.2 canonical entry for the new default slot A
DEFAULT_SLOTS slot A moved to openai:gpt-5.2, which had no CANONICAL_PRICING
entry — estimateCost silently dropped slot A from the --max-usd pre-flight
and est_cost_usd audit rows (~1/3 under-count on the default panel). Rates
from the OpenAI recipe chat touchpoint (verified 2026-04-20). Also refresh
the --slot-a-model help text default and pin a pricing-presence assertion
in the DEFAULT_SLOTS consistency test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: p3ob7o <p3ob7o@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Garry Tan <garrytan@gmail.com>
- phaseBFenceFacts now queries legacy rows FIRST and dirty-checks only
the source_ids it will actually write into. Zero fenceable rows (or
rows scoped to clean sources) no longer fail on an unrelated dirty
source. Targeted-dirty-source refusal unchanged. Fixes#927.
- apply-migrations now prints each failed phase's name + detail to
stderr alongside 'reported status=failed', instead of burying the
actionable message in the ledger. Fixes#921.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Two backlog fixes:
- init (#1058): loadConfig() returns null on a cold install (no config.json
AND no DATABASE_URL), short-circuiting before its env merge — so
GBRAIN_EMBEDDING_MODEL / GBRAIN_EMBEDDING_DIMENSIONS /
GBRAIN_EXPANSION_MODEL / GBRAIN_CHAT_MODEL were silently ignored and
Tier-3 detection auto-picked by API key instead. resolveAIOptions' config
seed now falls back to those env vars directly when loadConfig() is null
(new exported helper seedAIOptionsFromConfig, env-injectable for tests).
- whoami (#1061): the stdio MCP dispatch is remote/untrusted by design but
has no per-token auth (local pipe), so whoami threw unknown_transport on
the primary stdio surface. The stdio dispatch now marks
ctx.transport = 'stdio' and whoami returns {transport: 'stdio', scopes: []}
for it. Trust posture unchanged: remote stays true, the marker is never
used for trust decisions, and an unmarked auth-less remote context still
throws (fail-closed preserved).
Co-authored-by: Garry Tan <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
MiniMax's /v1/embeddings endpoint is not OpenAI-compatible: it requires
texts (not input) plus a type field and returns {vectors} instead of
{data:[{embedding}]}. The recipe shipped no transport shim, so every
embed call failed with an invalid-params error, and it declared no chat
touchpoint, so assertTouchpoint blocked gbrain think even though
MiniMax chat is genuinely OpenAI-compatible.
Fix (takeover of #2882, corrected):
- minimaxCompatFetch via the DeepSeek-style compat.fetch seam (keeps
cfg.base_urls overrides working; no new env var), gated on the
/embeddings path so chat requests/responses pass through untouched.
- Response rewrite parses via resp.clone() and rebuilds with fresh
headers — never returns a body-consumed Response (the flaw in #2882's
wrapper, which broke every non-streaming chat completion).
- chat touchpoint with the /v1/models list from #1977.
Fixes#1977
Co-authored-by: Garry Tan <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: ArthurHeung <ArthurHeung@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
deriveDirectUrl() swaps the Supabase pooler host to db.<ref>.supabase.co:5432,
which is IPv6-only without the paid IPv4 add-on. On IPv4-only networks the
direct pool could never connect, and initDirectPool()'s throw killed
'gbrain init --url' and migrations with ENOTFOUND/ECONNREFUSED.
getDirectPool() now classifies network-unreachable errors (ENOTFOUND,
ECONNREFUSED, ENETUNREACH, EHOSTUNREACH, ETIMEDOUT, CONNECT_TIMEOUT) via the
new isNetworkUnreachableError(), self-activates the kill-switch, logs one
stderr line pointing at GBRAIN_DIRECT_DATABASE_URL / GBRAIN_DISABLE_DIRECT_POOL,
and returns the read pool. Auth/SQL errors still throw (misconfig, not
unreachability). The failed pool is ended via endPoolBounded so it can't
leak sockets into the now-continuing process.
Also surfaces the kill-switch + override envs in the init.ts IPv6 warnings
and docs/guides/live-sync.md (they were previously undocumented outside
connection-manager.ts).
Fixes#1641
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The zhipu recipe was embedding-only, so models.tier.subagent=zhipu:glm-5.1
threw "does not offer a chat touchpoint" — while the error hint falsely
listed zhipu (and dashscope/minimax, also embedding-only) among providers
with chat.
- zhipu recipe: add a chat touchpoint (glm-5.1 family, supports_tools +
supports_subagent_loop; no Anthropic-style prompt cache on the
OpenAI-compat path, so the loop runs with the degraded:no_caching warn).
openai-compat tier means newer GLM ids pass without a recipe edit.
- capabilities.ts: compute the "Known providers with chat" hint from the
recipe registry instead of a hardcoded list, so it can never drift into
naming chat-less providers again.
- Declines the originally requested models.anthropic_compatible_prefixes
config: v0.38's recipe-driven capability gate already replaced the
Anthropic-only enforcement, so a recipe chat touchpoint is the whole fix.
Fixes#1157
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Master already widened slugifySegment (sync.ts) and validatePageSlug
(operations.ts) to CJK in v0.32.7, but the other two validators #782
targeted stayed ASCII-only: SlugRegistry's SLUG_RE rejected any CJK
desiredSlug from BrainWriter, and synthesize.ts's SUMMARY_SLUG_RE (whose
comment claimed it was kept in sync with validatePageSlug) rejected CJK
output roots.
Hoist the segment grammar into cjk.ts as PAGE_SLUG_SEG and compose all
three regex sites from it, so the four slug validators share one grammar.
Each site keeps its own shape (SlugRegistry's >=2-segment dir/name form,
validatePageSlug's case-insensitive flag).
Scope stays CJK (matching v0.32.7), not full \p{L} Unicode as #782
proposed — all-scripts slugs (lookalike/RTL spoofing) is a maintainer
policy call.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: tamagodo-fu <tamagodo-fu@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Takeover of #2510: migrations v40 (facts) and v55 (query_cache)
unconditionally created HNSW indexes with the configured embedding
dimension, so `gbrain init` with embedding_dimensions above pgvector's
per-type HNSW caps (vector 2000 / halfvec 4000) failed with
"column cannot have more than 4000 dimensions for hnsw index".
- vector-index.ts: add PGVECTOR_HNSW_HALFVEC_MAX_DIMS + hnswMaxDimsForType
- migrate.ts v40/v55: emit the HNSW index only when dims fit the cap,
otherwise a comment noting exact scans remain available
- embedding-dim-check.ts: buildFactsAlterRecipe skips the reindex step
above the cap for the same reason
- tests: 4096d init round-trip on PGLite (columns exist, indexes
skipped) + recipe-skip unit test
Drops the unrelated context-engine.ts interface change and the
tsconfig.json strictFunctionTypes=false hunk from #2510; typecheck is
clean without them.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: javieraldape <javieraldape@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(calibration): resolve owner holder via config (default 'self'), fixes#2464
Takeover of #2467 (rebased onto master). consolidate writes owner takes
with holder='self' while calibration-profile, calibration CLI/op, think's
calibration block, emotional-weight, and doctor's calibration_freshness
all defaulted to a hardcoded 'garry' — so getScorecard returned 0
resolved and the calibration profile never built on non-upstream brains.
New src/core/owner-holder.ts is the single source of truth:
resolveOwnerHolder({override, configValue}) = override >
emotional_weight.user_holder config > 'self'. All six call sites route
through it; doctor's freshness SQL is parameterized ().
Upgrade note: upstream-owner brains with historical holder='garry'
profiles should `gbrain config set emotional_weight.user_holder garry`
to keep reading them.
Co-authored-by: devty <devty@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(calibration): replace real-name holder fixture with charlie-example placeholder
Privacy iron rule: no real people's names in checked-in code. The sanctioned
placeholder mapping uses people/charlie-example.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: devty <devty@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Garry Tan <garrytan@gmail.com>
The help text (gbrain check-backlinks <check|fix> [dir]) promised a
positional directory argument, but runBacklinks only parsed --dir and
defaulted to cwd, so the walker ran from the wrong root and could hit
EPERM on unreadable sibling dirs.
Extract parseBacklinksArgs: positional [dir] is now honored, --dir still
overrides it, --dry-run preserved, and a --dir flag missing its value
falls back to the positional dir instead of picking up undefined.
Takeover of #852 (rebased onto master past the findBacklinkGaps dedupe
test block). Fixes#485.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Kage18 <Kage18@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
doctor.ts pushes runAllOnboardChecks results into the checks list, but the
7 onboard check names (embed_staleness, entity_link_coverage,
timeline_coverage, takes_count, dangling_aliases, pack_upgrade_available,
type_proliferation) were never added to doctor-categories.ts, so every
doctor run emitted an 'unknown check name' stderr warn per onboard check.
Registers the 5 data-quality names under BRAIN and the 2 schema-pack names
under META (alphabetical order preserved), and widens the drift-guard test
to scan src/core/onboard/checks.ts alongside src/commands/doctor.ts so
future onboard checks can't drift uncategorized.
Takeover of #1839, rebased onto master (keeps master's timeline_dedup_index).
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: mvanhorn <mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Title precedence is now frontmatter title: > body's first ATX H1 > the
slug/filename-humanized fallback. Slug-based imports (contacts, calendar)
carry a correct # Heading but no frontmatter title; without the H1 fallback
they got junk titles humanized from the slug. The H1 scan skips h2+ and
lines inside fenced code blocks, and strips closed-ATX trailing hashes.
Takeover of #2495.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: javieraldape <javieraldape@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Three fixes in the same leak class (a remote caller reading or writing
outside its granted sources), for multi-source / multi-tenant brains:
1. dispatchToolCall now refuses remote calls that arrive without a
resolved sourceId (missing_source_scope error envelope) instead of
silently falling back to the shared 'default' source. Every shipped
transport already passes sourceId explicitly (serve-http from the
OAuth client row, http-transport from the legacy token grant, stdio
from GBRAIN_SOURCE); reaching the fallback remotely always meant a
programmatic caller skipped scope resolution — the bug class behind
#1924 / #1371. Trusted local callers (remote === false) keep the
historical fallback. Direct-dispatch tests updated to carry an
explicit sourceId, matching the real transport contract.
2. log_ingest threads ctx.sourceId (same pattern as get_chunks /
get_page), so ingest events are attributed to the caller's source
instead of piling into 'default'. Engines already accept
entry.source_id (v0.31.2).
3. get_ingest_log is source-scoped for remote callers via the
linkReadScopeOpts collapse rule (scalar grant → [scalar]; federated
grant → granted array); it previously returned the whole brain's
ingest log to any read-scoped remote client, and ingest summaries can
carry another source's private context. Trusted local callers keep
the whole-brain view.
Tests: dispatch guard (refuse remote-without-source, keep local
fallback, guard ordering after op lookup) and end-to-end ingest-log
attribution + scoping over the real dispatch path, on PGLite.
gbrain init --no-embedding writes embedding_disabled: true as a
deferred-setup sentinel, and init/import/embed honor it via
assertEmbeddingEnabled. sync's embed credential preflight (v0.41.6.0 D1)
only checked the --no-embed CLI flag, so every gbrain sync on a keyless
deferred-setup brain exited 1 demanding <PROVIDER>_API_KEY — including
orchestrated callers (gstack /sync-gbrain) that never pass --no-embed.
embed-preflight.ts's own skip protocol documents that the sentinel is
owned upstream of the credential check; this wires that contract into
sync by deriving noEmbed from CLI args + config in one exported pure
helper (resolveNoEmbed), covered by test/sync-no-embed-sentinel.test.ts.
Co-authored-by: Gawie van Blerk <gawie.vanblerk@emeraldlife.co.za>
A page write is not 'done' until it is readable back. After the import
transaction commits, verify the page resolves via getPage and its
content_hash matches what was just written. On mismatch or miss, fail
LOUDLY instead of reporting success, and record the failure in
ingest_log (best-effort) so it is durable and agent-inspectable rather
than a transient stderr message.
This catches the silent-desync class: the page file exists on disk (or
the git commit landed) but the DB index never picked the write up —
the operation previously reported success while the page stayed
invisible to every read path (get_page, search, query) until someone
noticed the gap manually.
Guard applies to both importFromContent (markdown) and importCodeFile.
Tests: new write-verify-guard suite (hermetic PGLite) covering the
happy path, index-miss, stale-hash, ingest_log record, and the put_page
operation surface; import-file.test.ts mock upgraded to simulate a
readable DB (writes are read-backable), matching the new guard.
PRJ-2026-032
Co-authored-by: merlin-drizzyenterprises[bot] <144527811+merlin-drizzyenterprises[bot]@users.noreply.github.com>
list_pages clamps limit to max 100 (default 50) — deliberate server
protection, pinned in test/search-limit.test.ts. But the clamp was
SILENT: a caller whose limit was defaulted or clamped got a
full-looking array with no signal that rows were dropped, and with the
default updated_desc sort the dropped rows are always the OLDEST —
precisely what exhaustive consumers (audits, scans, backfills) exist
to find. Observed in the field: a source with 212 pages enumerated as
80 visible rows, hiding 26 pages from a compliance scan for days.
Fix, with no response-shape change (MCP consumers still get an array)
and no engine surface change (handler probes limit+1):
- handler probes one row past the effective limit; when the caller's
limit was NOT honored (unset -> default, or clamped to cap) and rows
were dropped, it warns on stderr for local (CLI) callers — same
operator-facing channel as the put_page unknown-type hint, but
without the isTTY gate: scripted callers are exactly the consumers
that cannot detect truncation any other way, and stderr keeps stdout
parseable. An explicit honored limit stays silent (ordinary
pagination), as does a clamped-but-complete result. Remote (MCP)
ctx never writes to stderr.
- LIST_PAGES_DESCRIPTION documents the cap and the exhaustive-listing
recipe (sort=updated_asc + updated_after cursor) — the description
is the signal channel MCP clients actually read.
- regression suite: default-limit truncation warns, honored limit
silent, clamped-but-complete silent, remote silent, and the
documented cursor recipe enumerates a corpus to completion.
Co-authored-by: YMYD <53603073+OJ-OnJourney@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
On machines with neither gtimeout nor timeout on PATH, run-verify-parallel.sh
and run-unit-parallel.sh fall back to a bg-pid + sleep-watchdog cap. Both
read $? only after tearing the watchdog down (kill + wait on cap_pid), so the
sentinel .exit files recorded the killed watchdog's status — 143 — instead of
the check/shard's own exit code. Every run reported total failure (verify:
pass=0 fail=31; unit: rc=143 per shard) while every per-check/shard log
showed success.
Capture rc immediately after `wait $pid` in both scripts, and reap the
watchdog's sleep child (pkill -P, children-first — the same orphan quirk the
heartbeat cleanup documents) so the fallback stops leaking one sleep per
check/shard.
Regression tests force the fallback branch hermetically on any host via a
curated PATH with no timeout binaries: the verify dispatcher runs from a
tempdir copy with a stubbed `bun`, pinning exit 0 + all-zero sentinels when
checks pass and the check's own rc (not 143) when one fails; the unit wrapper
runs real two-shard fixture passes, pinning rc=0 sentinels and a real
failure's rc=1.
Co-authored-by: YMYD <53603073+OJ-OnJourney@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
When the recommendation list is refreshed between remediation steps, a
remediation that doesn't clear its own health signal is reintroduced
under its stable id and attempted again, indefinitely on long runs.
Track attempted recommendation ids for the run and skip re-attempts.
Includes a behavioral regression test: a persistently-stuck signal is
attempted once, the loop terminates, and other remediations still run.
Dispatch timeout was derived as interval*2 with a 5-minute floor, tuned
for light per-interval work. A full autopilot cycle routinely needs more
than 10 minutes at common intervals, so healthy full cycles were killed
mid-run. Full-cycle dispatch now gets a 30-minute floor; lighter
dispatches keep the interval-derived budget.
Adds a regression test for the full-cycle floor.
upsertChunks fell back to the compile-time DEFAULT_EMBEDDING_MODEL
('zeroentropyai:zembed-1') when a ChunkInput carried no explicit `model`.
The embed pipeline (src/commands/embed.ts) builds ChunkInputs without a
`model` field, so rows whose vectors were produced by the config-resolved
model (e.g. openai:text-embedding-3-large) were mislabeled with the
hardcoded default — corrupting the provenance that signature-drift
staleness and dimension-migration logic depend on.
Both engines now resolve the gateway's runtime embedding model once per
upsert and use it as the fallback, mirroring the existing resolve-then-
default pattern used for schema sizing. Regression test added (pglite);
verified via negative control that it fails against the old fallback.
This is a write-path change (upsertChunks), not a search-path change, so
retrieval eval replay is not applicable.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
`sources.config` is a jsonb OBJECT column, but a read→write cycle that
JSON.stringify'd an already-stringified value re-wrapped it into a JSON string
scalar ("{}", "\"{}\"", ...) that grew one layer per write. parseSourceConfig
only unwrapped one layer, so the corruption never healed and federation/ACL
reads saw a string instead of the settings object.
- Add normalizeSourceConfig: a bounded (10-iteration) loop that JSON.parses
while the value is a string and returns {} (with a console.warn) when the
result is not a plain object. All six `UPDATE sources SET config` writers run
their config through it before stringify, converging the stored value back to
a jsonb object on the next write.
- parseSourceConfig now does the same bounded unwrap and warns once when more
than one layer was found (one layer is the normal PGLite path).
- Add a `source_config_shape` doctor check that flags any sources row where
jsonb_typeof(config) <> 'object', with the repair path.
- Unit-test the helper (object passthrough, 1-layer, 5-layer nested, garbage
and over-bound inputs) and the doctor check (mock engine).
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Issue #2298: 'gbrain doctor' surfaced two distinct timeline
metrics under the same user-facing 'timeline' label:
1. Entity timeline coverage (graph_coverage metric)
- numerator: eligible entity pages WITH a timeline entry
- denominator: eligible entity pages
- 0-1 fraction, surfaced by graph_coverage check
2. Whole-brain timeline density (brain-score 0-15 component)
- numerator: all pages WITH a timeline entry
- denominator: all pages
- 0-15 scale, surfaced by brain_score breakdown
These have DIFFERENT numerators/denominators. The old single
'timeline X%' label let a reader mistake the entity-scoped
percentage for whole-brain density.
Presentation/contract clarity only — scoring formula,
health weights, takes, source routing, extraction UNCHANGED.
- doctor.ts graph_coverage: 'timeline X%' -> 'entity timeline coverage X%'
- doctor.ts brain_score: 'timeline X/15' -> 'timeline density (all pages) X/15'
- cli.ts get_health: 'Timeline coverage (entity pages)' ->
'Timeline density (all pages): X/15 (whole-brain brain-score component)'
Test: test/doctor-timeline-metric-labels-2298.test.ts uses a
synthetic in-memory PGLite fixture (NO private EriadorMu data):
4 total pages, 2 eligible entity pages, 1 entity page with a
timeline entry, 1 total page with a timeline entry.
Expected: entity coverage 1/2 = 50%; whole-brain density
1/4 -> round(25% * 15) = 4/15. Asserts the two metrics
render with distinct scoped labels and the brain-score component
is explicitly whole-brain (no 'entity' in its label). 5/5 pass.
Addresses #2298
extractStaleFromDB still used the pre-#972 `includeFrontmatter ? resolver :
nullResolver` ternary. The synthetic resolver has no resolveBasenameMatches,
so the gate in extractPageLinks skipped the issue-#972 bare-wikilink pass
regardless of link_resolution.global_basename — the sweep stamped every page
as extracted while silently dropping its [[bare-name]] links. Same brain,
same pages: `extract --stale` created 0 links where `extract links --source
db` created 218.
- Always pass the real batch resolver; gate passes via extractPageLinks opts
({ skipFrontmatter: !includeFrontmatter, globalBasename }), mirroring
extractLinksFromDB — including the codex-[P1] sourceId scoping.
- Bump LINK_EXTRACTOR_VERSION_TS (documented protocol) so pages stamped by
the broken sweep re-flag stale and re-extract under the fixed logic.
- Regression tests: bare wikilink resolves on --stale with the flag ON;
still drops with the flag OFF (back-compat). The #1768 fixture now derives
its updated_at from LINK_EXTRACTOR_VERSION_TS instead of a hardcoded date,
so future version bumps can't silently flip its version arm.
Fixes bug 1 + bug 3 of #2576. Bug 2 (DIR_PATTERN gaps) is a separate
whitelist design call, intentionally not addressed here.
Co-authored-by: YMYD <53603073+OJ-OnJourney@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
An empty --content (most commonly a non-interactive caller that meant
file input — put has no --file flag — so the missing --content fell
back to reading empty stdin) silently blanked existing pages. put_page
now rejects an empty/whitespace-only body over an existing non-empty
page with invalid_params, pointing at `gbrain capture --file PATH
--slug SLUG` for file input; allow_empty: true (CLI: --allow-empty)
opts into an intentional blank. The guard read is scoped to the exact
(source_id, slug) row the write targets; new-slug creates and
soft-deleted-page overwrites stay allowed.
Co-authored-by: Matthew Thompson <matthew@symmetric-consulting.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
DashScope's OpenAI-compatible rerank endpoint lives at
{base}/compatible-api/v1/reranks — PLURAL leaf, different base path from
the embedding surface (compatible-mode). Reusing llama-server-reranker
against DashScope forces users to hand-patch the recipe's '/rerank' leaf
in node_modules, which every upgrade silently reverts (and llama.cpp
genuinely serves singular /rerank, so changing that recipe would break
real llama.cpp users).
New dedicated recipe rides the v0.40.6.1 recipe-pluggable reranker path:
- id dashscope-rerank, base_url_default compatible-api/v1 (intl), ZE wire
- path '/reranks', default_timeout_ms 30s, 5MB payload ceiling
- models: only qwen3-rerank (live-verified 200; gte-rerank-v2 is rejected
by the compat surface with 'Unsupported model for OpenAI compatibility
mode', so it is deliberately not listed)
- separate recipe (not a reranker touchpoint on dashscope) because
provider_base_urls is keyed by recipe id and the two capabilities need
different prefixes — same topology as llama-server vs
llama-server-reranker
Tests: recipe shape smoke mirroring recipe-llama-server-reranker.test.ts
(path/timeout/payload pins, /v1/v1 concat guard, auth resolve, sibling
recipe isolation). bun test test/ai/: 322 pass / 0 fail.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
The four `hybridSearch — reranker enabled (reorder)` cases stub the gateway
at 1536 dims (DIMS). Since v0.36.3.0 hybridSearch resolves the embedding
column via loadConfig(), whose precedence is
cfg.embedding_dimensions > gateway dims > default. On any machine whose
~/.gbrain/config.json sets embedding_dimensions to something other than 1536
(e.g. text-embedding-3-small at 1280), the real config outranks the stub: the
1536-d stub vector fails the gateway dim check, the error is swallowed, search
falls back to keyword-only, and the reranker never runs (rerankerFn gets 0
docs, rerank_score undefined). Green in CI only because a fresh runner has no
config file — deterministic red on a contributor's machine.
Fix (test-only): isolate GBRAIN_HOME to an empty tmpdir in beforeAll so
loadConfig() returns null and the stub's dims win, then restore it and clean
up in afterAll. Same idiom as emptyHome() in
test/ai/gateway-probe-chat-model.test.ts.
Verified with a planted ~/.gbrain/config.json at 1280 dims: 2 pass / 4 fail
before, 6 pass / 0 fail after; still green with no config file. typecheck clean.
Fixes#1527
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
getHealth's entity_pages CTE and the top-linked-pages query only match the
legacy 'person' and 'company' types, so brains using the gbrain-base-v2
pack's 'entity' type report 0% entity link/timeline coverage in `gbrain
health` even when doctor's graph_coverage shows real coverage. Add 'entity'
to both queries in both engines (PGLite + Postgres, in lockstep per the
engine-parity rule) and extend the getHealth graph-metrics test with an
entity-typed page.
Validation: bun test test/pglite-engine.test.ts --test-name-pattern 'getHealth graph metrics' (5 pass).
Replace the strictly sequential per-chunk synopsis loop with a bounded
sliding worker pool (existing runSlidingPool helper). Results land in
chunk order via index-addressed writes; code chunks still bypass the
wrapper; embedding remains one page-level batch after all synopses.
New knob GBRAIN_CONTEXTUAL_CHUNK_CONCURRENCY, default 4, clamped to
[1,16]; 1 reproduces the prior sequential behavior exactly. Each chunk
task still acquires/releases the global synopsis rate-lease, which
remains the cross-worker governor; the lease id now travels from
acquire to release instead of shared mutable state, and lease waits
are abort-responsive.
At 20-45s per synopsis call, a 120-chunk transcript page previously
needed 60-90+ min wall time and routinely outlived job timeouts.
statusForVersion short-circuited on any 'complete' entry before checking
the trailing 'retry' marker, so --force-retry appended an inert row and a
version marked complete with zero work done could never be re-run without
hand-editing completed.jsonl. Check retry-latest first: an explicit
--force-retry now yields 'pending' even past an earlier 'complete', while
a stray 'partial' after 'complete' still cannot regress the version.
gbrain list --limit 100000 silently returned 100 rows (default 50) with
no warning, and --offset was accepted but dropped at the op layer even
though PageFilters has supported it all along.
- Local CLI callers (ctx.remote === false, the same trust boundary that
already bypasses scope enforcement) get an explicit limit above 100
honored — full enumeration is a legitimate local operation.
- Remote MCP/OAuth callers keep the 100-row DoS cap, now loud: one
logger.warn (stderr, stdout stays script-clean) with both numbers,
parity with the three search-path clamp warnings.
- offset is declared as a param (so the CLI coerces it to number) and
threaded to engine.listPages for real pagination.
Claude-Session: https://claude.ai/code/session_01Vswwe1y5fQbJWfbaSK3enT
Co-authored-by: deacon-botdoctor <291411030+deacon-botdoctor@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
SupabaseStorage.getSignedUrl built the download URL as `${projectUrl}${signedURL}`,
but Supabase's sign API returns `signedURL` relative to the Storage API root
(/object/sign/<bucket>/<path>?token=...), so the generated link dropped /storage/v1
and returned 404. Now prepends `${projectUrl}/storage/v1`, tolerating an
already-absolute URL or a value that already carries the prefix. `gbrain files
signed-url` links resolve again.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Reasoning models (MiniMax-M3, DeepSeek-R1, etc.) return <think>...</think>
tags in the content field before the actual JSON output. This caused
parseExtractorOutput to fail in two ways:
1. The fence regex /^\`\`\`(json)?...$/ requires the fence at text start;
<think> preceding it prevents matching, so the raw text (with trailing
fences) hits JSON.parse and throws.
2. When think tags contain [ or { characters, indexOf finds them inside
the reasoning block instead of the actual JSON array.
Changes:
- Strip <think>...</think> tags before any parsing (covers all reasoning models)
- Add JSON.parse fallback: truncate at last ] or } to handle trailing
noise (leftover markdown fences after stripping)
Tests: 28/28 pass (3 new cases for think tags + trailing noise).
Co-authored-by: qaz8545355 <junjun@openclaw.local>
The idempotency row is only written inside `for (const p of proposals)`, so a
page that extracts ZERO gradeable claims never records an idempotency tuple
and is re-sent to the LLM on every cycle forever. The docstring's "unchanged
page never re-spends tokens" contract only holds for pages that produce >=1
claim; a page that legitimately has no gradeable claims (or any machine-
generated page) is a perpetual cache miss and re-spends tokens indefinitely.
Fix: when `proposals.length === 0`, write one tombstone row keyed by the same
(source_id, page_slug, content_hash, prompt_version) tuple, with
status='rejected' so it never surfaces in a pending-review query (the pending
index filters status='pending'). Content changes (new content_hash) or a
PROPOSE_TAKES_PROMPT_VERSION bump still miss the tombstone and re-extract. The
extractor-throw path `continue`s before the tombstone, so failed pages are
retried rather than cached.
Guard against a subtle regression: `parseExtractorOutput` returns [] for BOTH
a genuine empty extraction AND malformed/prose/truncated model output, so
naively tombstoning every [] would permanently suppress a page that has claims
but hit a transient parse failure. `defaultExtractor` now throws when the
output is empty-but-not-a-clean-`[]` (new `isWellFormedEmptyExtraction`
predicate), routing transient failures into the existing retry path; only a
cleanly-parsed empty array is memoized.
Adds a `tombstones_written` counter for observability.
Tests: tombstone written on genuine empty extraction; two-cycle idempotency
(no repeat LLM call on an unchanged zero-claim page); extractor error writes
no tombstone; isWellFormedEmptyExtraction discriminates clean-[] from
malformed/prose/non-empty output. propose-takes suite: 36 pass / 0 fail.
Co-authored-by: ivandebot <187176982+ivandebot@users.noreply.github.com>
The empty-fence guard counted every `row_num IS NULL AND entity_slug IS NOT NULL`
row as a pending v0_32_2 backfill, but the inline facts writer keeps producing
rows of exactly that shape post-migration: when a resolved slug has no fenceable
page (slugify-floor / stub-guard-blocked unprefixed slugs like `wingman`,
`people-jane-doe`), backstop.ts falls through to a DB-only insert with row_num
NULL. Those rows are structurally unfenceable — no page to fence onto, and the
ledger-complete migration won't re-run — so they jammed the phase forever
(~16/day observed) and the warning advised a no-op `apply-migrations --yes`.
Discriminator: a row is a genuine backfill candidate only if its entity_slug
resolves to a LIVE page in the same source (EXISTS in `pages` with deleted_at
NULL) — mirroring the migration's Phase B, which only fences slugs that map to
a writable page. Genuine pre-v0.32.2 rows (their entity page exists) still gate;
inline-writer unfenceable rows no longer do. Warning text updated to name the
"entity page present, not yet fenced" condition.
Regression tests pin both sides: unfenceable rows (no page / soft-deleted page)
do NOT gate and the phase converges; a legacy row WITH a backing page still
gates. Fails pre-fix, passes post-fix.
(#2484)
Co-authored-by: Javier Aldape <javieraldape@Javiers-Laptop.local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`gbrain schema stats` reported "Total pages: 0" on populated brains because
fetchCountRows wrapped its count query in a bare `catch { return []; }` that
converted EVERY error into zero rows — false 0 pages, false "100% coverage"
(0/0 → vacuous 1.0), and a starved `schema suggest`. A sibling bare catch in
detectDeadPrefixes had the same defect.
Root cause is the masked error, NOT a PGLite query incompatibility: reproduced
the exact COUNT query (COALESCE/NULLIF/GROUP BY/ORDER BY ... NULLS LAST) against
the pinned PGLite 0.4.3 (PG17.5) through the real engine + full schema, plus
PG18 and NULL/empty edge-case data — it returns correct counts every time and
never throws. The issue's "the query is failing on PGLite" premise doesn't
reproduce; the actual failure on the reporter's brain was hidden by the catch
(they could not capture it, consistent with an engine/init-level throw). The
honest fix is to stop hiding it.
Both catches now swallow ONLY the genuine missing-table case via the existing
isUndefinedTableError helper (SQLSTATE 42P01 + PGLite "relation ... does not
exist") and rethrow everything else, so the next occurrence shows the real
error instead of a fake zero. Pre-init/empty-brain behavior is preserved.
Regression: 4 new cases in test/schema-pack-stats.test.ts pin (1) real non-zero
count on a populated PGLite brain, (2) fetchCountRows rethrows a non-missing-
table error, (3) fetchCountRows still degrades to empty on 42P01, (4)
detectDeadPrefixes rethrows via the sibling catch. Each error-surfacing test
verified to fail when its catch is re-broadened.
Co-authored-by: Javier Aldape <javieraldape@Javiers-Laptop.local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Code legitimately contains tiktoken special-token strings (e.g. CLIP/GPT tokenizers embed the literal <|endoftext|>). The default encode() uses disallowed_special='all' and THROWS on those, crashing reindex-code on valid source files. Re-encode treating them as ordinary text (allowed=[], disallowed=[]); heuristic fallback if even that fails. A token COUNT needs no special-token semantics.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
putPage lowercases the slug via validateSlug, but the tag/link/timeline
reconcilers (tx.addTag, addLink, addTimelineEntry) query the slug as passed.
A remote put_page with a capitalized slug (e.g. 'Projects/Team-Wiki/Quarterly-Roadmap')
stored the page under 'projects/team-wiki/quarterly-roadmap', then threw
'addTag failed: page "…" not found' on the existence check, rolling back the
entire write — so the page never persisted under either casing. Any agent
driving the HTTP MCP server (where slugs arrive verbatim) lost every page whose
slug carried a capital letter plus a frontmatter tag.
Normalize the slug once at the top of importFromContent (the shared chokepoint
for MCP put_page and CLI capture) so putPage and every reconciler agree on the
canonical lowercased slug. No-op for disk imports (already slugifyPath output),
idempotent with putPage's own validateSlug call. Engine-agnostic, so PGLite and
Postgres move together.
Adds test/put-page-mixed-case-slug-tags.test.ts pinning the regression on PGLite.
* fix(write-through): guard mkdir against EEXIST on Bun+Windows
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(budget): resolve non-Anthropic model pricing via canonical table under --max-cost
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(enrich): exclude dream-generated pages from thin candidates
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
When link_resolution.global_basename is enabled, extend basename-index
resolution to frontmatter link fields (FRONTMATTER_LINK_MAP), mirroring the
body bare-wikilink path added in #972.
Problem: a bare-title wikilink in a frontmatter list -- e.g.
sources:
- "[[2025-12-25_mentor-extraction]]"
never resolves. SlugResolver.resolve() has no '/' to hit the slug-direct
getPage, and the field's dirHint (sources -> ['source','media']) may name
folders absent from the brain, so the dir-scoped exact + fuzzy steps also
miss. The frontmatter path never consulted resolveBasenameMatches -- that was
wired only for body bare-wikilinks. On a PARA/Obsidian vault this silently
drops the bulk of sources:/related: provenance edges.
Fix: extractFrontmatterLinks takes a globalBasename flag (threaded from
extractPageLinks). On a resolve() miss, unwrap [[ ]] and fall back to
resolver.resolveBasenameMatches -- UNIQUE-MATCH-ONLY, so ambiguous basenames
(archive dupes, generic hubs like _index) stay unresolved rather than create
a wrong edge. Purely additive; resolved frontmatter edges are unchanged.
Scope: covers the db-source extract and live put_page paths (real
makeResolver). The --source fs extract uses an inline resolver without a
basename index, so it gracefully no-ops there (typeof guard).
Tested: 3 new cases (resolves-when-on, ambiguous-stays-unresolved,
gated-off-by-flag); full link-extraction suite green (130 pass).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The hybrid recency stage in runPostFusionStages imported
DEFAULT_RECENCY_DECAY directly, so operator overrides via the
GBRAIN_RECENCY_DECAY env var and the gbrain.yml `recency:` section were
honored only on the get_recent_salience SQL path and silently ignored on
the hot hybridSearch path. Non-default vault layouts therefore stayed on
the baked-in defaults / DEFAULT_FALLBACK (90d / 0.5) regardless of
tuning.
Call resolveRecencyDecayMap() (already used by the SQL path) so the
configured decay map reaches the boost stage. Behavior is unchanged when
no override is set — resolveRecencyDecayMap() returns DEFAULT_RECENCY_DECAY.
Adds test/hybrid-recency-config.test.ts asserting the env override
reaches the applied recency factor (fails against the prior wiring).
Single-file `frontmatter validate` derived the expected slug from the
absolute path: relative(resolve(target), file) is empty when target IS the
file, so it fell back to `|| file` (the full path), yielding "root/<abs>"
slugs and a false SLUG_MISMATCH. The pre-commit hook from install-hook
validates staged files one-by-one, so this rejected every commit in a
markdown brain (only bypassable with --no-verify).
Walk up to the brain root (nearest .git) and use relative(brainRoot, file)
|| basename(file), matching runAudit/runGenerate and sync/extract. Files
above the root fall back to basename instead of a ../-prefixed slug.
Reopens#565. Present since v0.32.0; reproduced on v0.42.51.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
OpenRouter (and potentially other proxy providers) expose OpenAI's
text-embedding-3 models with a provider prefix in the model ID, e.g.
`openai/text-embedding-3-large` rather than bare `text-embedding-3-large`.
`dimsProviderOptions()` checks `modelId.startsWith('text-embedding-3')`
which fails for the prefixed form, so the `dimensions` parameter is never
sent. The upstream provider returns its native dimensionality (3072 for
-large) instead of the configured value (e.g. 1536), causing an immediate
"dim mismatch" error on first embed.
The default OpenRouter embedding (`text-embedding-3-small` at 1536d)
masked this because its native size happens to match the default config.
The bug surfaces when using `-large`, or `-small` with a non-1536 dim
(512, 768, 1024 — all listed in the recipe's `dims_options`).
Fix: strip the provider prefix before the `startsWith` check. The full
prefixed ID is preserved in the error message for user clarity.
* feat(subagent): claude-cli MessagesClient adapter (baseline, no tool use)
Closes#334 (partially — text-only baseline; tool use lands in the next
commit on this branch).
Adds a MessagesClient adapter that shells out to `claude --print
--output-format json --model <model>` instead of the Anthropic SDK. When
`GBRAIN_USE_CLAUDE_CLI=1` is set, the subagent worker registers the adapter
in place of the SDK client; the default path (Anthropic SDK with
ANTHROPIC_API_KEY) is unchanged when the env var is unset or set to
anything else.
The benefit is that Claude Max subscribers can run Minions subagents
against their existing OAuth subscription, no ANTHROPIC_API_KEY needed.
New: src/core/minions/handlers/claude-cli-adapter.ts
- Implements the MessagesClient interface exported from subagent.ts.
- Strips provider prefixes (`anthropic:`, `litellm:`) from the model id
because `claude --print` only accepts CLI-native aliases (`sonnet`,
`opus`, `haiku`, or the bare `claude-*-N-M` form).
- Flattens the Anthropic messages array into a single text prompt for
claude-cli stdin. Tool blocks (tool_use / tool_result) are stringified
as placeholders so multi-turn conversations stay coherent in this
baseline; native tool_use round-tripping is the follow-up commit.
- Spawns claude with stdio piped, captures stdout, parses the
`{type:"result", subtype:"success", result, usage, ...}` JSON envelope,
and returns it as a properly shaped Anthropic.Message with
`stop_reason: 'end_turn'`.
- Token totals propagate from the claude usage block so the subagent
handler's `ctx.updateTokens()` reports usable numbers.
- AbortSignal is wired through to SIGTERM the child so the subagent loop's
cancellation path stays correct.
Modified: src/commands/jobs.ts (worker registration)
- Conditionally constructs a MessagesClient via the new adapter when
GBRAIN_USE_CLAUDE_CLI=1.
- Passes it into makeSubagentHandler({ engine, client: subagentClient }).
- Logs `[minion worker] subagent routing via claude-cli (GBRAIN_USE_CLAUDE_CLI=1)`
on startup so the env var status is operator-visible.
Limitations of this commit (addressed in the follow-up):
- Tool use is not yet supported. Tools in params.tools are ignored; the
adapter returns a single text block with stop_reason='end_turn'.
- Token counts come from claude-cli's reporting and may not match the
Anthropic API's accounting precisely (especially for cache tiers).
Original design from #334; this commit preserves that author's attribution.
The follow-up commits on this branch carry the tool-use implementation.
* feat(subagent): tool use + context isolation + convention rename on top of jarvisdoes baseline
Builds on the previous commit (jarvisdoes's #334 baseline) by adding three
things the upstream issue called out as gaps or that surfaced during review:
1. Tool use support via system-prompt-instructed JSON emission.
2. Context isolation flags so claude-cli does not load operator-level
CLAUDE.md, skills, and local project context into every subagent call.
3. Env var rename from GBRAIN_USE_CLAUDE_CLI=1 to
GBRAIN_SUBAGENT_PROVIDER=claude-cli to match the existing
GBRAIN_<noun>_<role>=<value> convention used by GBRAIN_CHAT_MODEL,
GBRAIN_EMBEDDING_MODEL, GBRAIN_EXPANSION_MODEL.
## Tool use
The MessagesClient interface returns Anthropic.Message objects whose
content array may include tool_use blocks. The subagent handler filters
those blocks and dispatches each tool, so any backend that produces
correctly shaped tool_use blocks gets the same loop behavior as the
Anthropic SDK.
The adapter injects a system-prompt addendum describing the tool registry
plus an emission protocol:
<use_tools>
[{"id": "...", "name": "...", "input": {...}}, ...]
</use_tools>
After the response comes back, extractToolCalls() scans for the block,
parses the JSON (tolerant of optional ```json fencing), and converts each
entry into a tool_use content block. Multiple parallel tool calls in one
turn are supported via the array shape; this is the exact case that
breaks today on the codex-proxy / litellm GPT-5.x bridge where parallel
tool-call response IDs get dropped.
Defensive fallbacks:
- Malformed JSON inside the block: drop to text-only, stop_reason='end_turn'.
- Unterminated <use_tools> (no close tag): drop to text-only.
- Model omits id field: adapter synthesizes a toolu_claude_cli_<rand> id.
- Empty response: still hand the subagent loop a well-formed content
array so the .filter chain does not crash.
## Context isolation
claude-cli auto-discovers CLAUDE.md from cwd upward and injects the
operator's skills + plugins + auto-memory into the default system prompt.
On a real install that is ~42-65k tokens of contamination per subagent
call, with both cost and behavioral consequences (the subagent picks up
the operator's coding conventions, opinions, and preferences).
The maximum suppression that still preserves OAuth / Claude Max
subscription auth is:
- Spawn from a dedicated clean cwd (tmpdir-based) so LOCAL CLAUDE.md
auto-discovery has nothing to find. -13k tokens on a real gbrain
install where CLAUDE.md is substantial.
- --disable-slash-commands so skill resolution does not pull in
/skill-name handlers.
- --system-prompt <gbrain prompt> so the default system prompt is
replaced rather than appended to.
The --bare flag would also strip user-level ~/.claude/CLAUDE.md but it
forces ANTHROPIC_API_KEY auth, defeating the whole point of this adapter.
The remaining ~42k cached tokens from user-level instructions are
accepted as a cost-trivial trade-off because the Max subscription absorbs
the per-call cost. Behavioral contamination is mitigated by gbrain's
strong per-call system prompt overriding any operator-level drift.
## Env var rename
Surveyed all ~140 GBRAIN_* env vars in src/. The codebase uses three
patterns: GBRAIN_NO_<feature> (negative toggles), GBRAIN_<noun>_<role>
=<value> (routing keys), GBRAIN_ALLOW_<feature> (permissive toggles).
GBRAIN_USE_* does not appear anywhere except jarvisdoes's original
commit; it would introduce a fourth pattern.
GBRAIN_SUBAGENT_PROVIDER=claude-cli aligns with the routing-keys family
and is value-extensible — adding codex-cli / meridian-proxy / etc. later
means a new value, not a new env var. The scope ('SUBAGENT_*') is also
unambiguous about which calls the toggle covers; GBRAIN_USE_CLAUDE_CLI
was silent on whether it applied to all gbrain LLM calls or only the
subagent path.
Unknown values are rejected with a fail-fast error message naming the
two valid values rather than silently falling through to the default.
## Tests
New file: test/claude-cli-adapter.test.ts — 12 tests, 33 assertions:
- Text-only round trip (single text block, usage propagation, end_turn).
- Provider prefix stripping ('anthropic:claude-sonnet-4-6' -> 'claude-sonnet-4-6').
- Single tool_use parsing.
- Multiple parallel tool calls in one block (the case that triggered
the codex-proxy regression).
- Fenced JSON inside <use_tools> block.
- Model-omitted id gets synthesized to toolu_claude_cli_<rand>.
- Malformed JSON falls back to text.
- Unterminated block falls back to text.
- AbortSignal SIGTERMs the child.
- Error envelope rejected with informative message.
- Non-JSON output rejected with raw-output excerpt in the error.
- argv + cwd assertion: --disable-slash-commands + --system-prompt are
present and cwd is the dedicated tmpdir.
Tests use a POSIX shell stub at GBRAIN_CLAUDE_CLI_BIN that emits a
scripted --output-format json envelope, so the suite runs without
claude-cli installed and without API credits.
* feat(ai): claude-cli recipe with native gateway integration (supersedes #334 baseline)
Replaces the MessagesClient adapter + GBRAIN_USE_CLAUDE_CLI=1 env-var
gate from the previous commit on this branch with a proper gateway recipe.
The recipe path gives per-call routing as a native capability: a model
string like `claude-cli:claude-sonnet-4-6` lands here while a sibling
`litellm:gpt-5.4` continues through the litellm-proxy / codex-proxy path
in the same worker. No global env-var switch, no agent.use_gateway_loop
bypass, no MessagesClient injection at jobs.ts worker startup.
The previous commit on this branch (jarvisdoes baseline) is preserved
in the history for #334 authorship attribution. Its functional changes
are backed out here because the recipe pattern is gbrain's established
integration seam; introducing a parallel MessagesClient + env-var path
would have created two routing mechanisms competing for the same job.
New: src/core/ai/recipes/claude-cli.ts
- Recipe declaration: id 'claude-cli', tier 'native', implementation
'claude-cli', chat-only (no embedding or expansion touchpoints).
- Models: claude-opus-4-7, claude-sonnet-4-6, claude-haiku-4-5-20251001.
- supports_tools and supports_subagent_loop both true.
- supports_prompt_cache false because the CLI handles caching internally
and does not surface cache_control via the standard control plane.
- auth_env.required is the empty array because the CLI owns auth (OAuth
session managed by `claude login`).
- Friendly aliases mirror the `anthropic` recipe: `sonnet`, `haiku`,
`opus` and the same legacy-id rewrites for back-compat with stale
config strings.
New: src/core/ai/providers/claude-cli-language-model.ts
- ClaudeCliLanguageModel class implementing the ai-sdk LanguageModelV2
interface.
- doGenerate: renders the ai-sdk prompt array into a system text + user
text, injects the use_tools protocol instructions when tools are
present, spawns `claude --print --output-format json --model <X>
--disable-slash-commands --system-prompt <gbrain prompt>` from a
dedicated tmpdir (contamination suppression: no local CLAUDE.md
auto-discovery), parses the JSON envelope, extracts <use_tools>
blocks, and returns ai-sdk-shaped LanguageModelV2Content (text +
tool-call parts with stringified-JSON input matching the V2 contract).
- Tolerates fenced JSON inside use_tools blocks, malformed JSON
(falls back to text), missing close tag (falls back to text),
model-omitted ids (synthesizes toolu_claude_cli_<rand>).
- Parallel tool calls in one block round-trip cleanly: this is the
case that drops IDs on the litellm + codex-proxy bridge today.
- AbortSignal SIGTERMs the child for proper cancellation.
- doStream throws not-supported (gateway.toolLoop is non-streaming).
Modified: src/core/ai/gateway.ts
- Adds case 'claude-cli' to instantiateChat (returns ClaudeCliLanguageModel).
- Adds case 'claude-cli' to instantiateExpansion (same wrapper, reserved
for a future expansion touchpoint declaration).
- Adds case 'claude-cli' to instantiateEmbedding (throws, no embedding
model, mirrors the native-anthropic path).
- Lazy require() at the call site keeps the gateway module load cheap
for users who never use the claude-cli path.
Modified: src/core/ai/recipes/index.ts
- Registers `claudeCli` in the ALL[] array next to `anthropic`.
Modified: src/core/ai/types.ts
- Adds 'claude-cli' to the Implementation union so the gateway switch
is exhaustive at compile time.
Reverted: src/commands/jobs.ts
- Drops the GBRAIN_USE_CLAUDE_CLI=1 env-var gate the prior commit
added. Routing now happens at the gateway based on the model string.
Deleted: src/core/minions/handlers/claude-cli-adapter.ts
- The MessagesClient adapter is superseded by the recipe + LanguageModelV2
path. Two routing mechanisms competing for the same job would have
forced users to reason about which one wins; the recipe is the single
source of truth.
New file: test/claude-cli-recipe.test.ts (16 tests, 46 assertions):
- Recipe registration: getRecipe returns chat-only Recipe; aliases map
short names (sonnet/haiku/opus) to canonical model ids.
- Text round trip: single text content block, usage propagation, stop
finish reason.
- Provider prefix stripping.
- Single tool-call parsing.
- Multiple parallel tool calls in one block.
- Fenced JSON inside the block.
- Model-omitted id synthesizes toolu_claude_cli_<rand>.
- Malformed JSON falls back to text + stop reason.
- Unterminated block falls back to text + stop reason.
- Tools offered but model declines: returns text-only with stop reason
so the gateway-loop treats it as a final answer rather than wedging
for tool calls that never come.
- AbortSignal SIGTERMs the child.
- is_error envelope rejected.
- Non-JSON output rejected.
- doStream throws.
- argv + cwd assertion: --print, --disable-slash-commands,
--system-prompt are present and cwd is the dedicated tmpdir.
Tests use a POSIX shell stub at GBRAIN_CLAUDE_CLI_BIN so the suite runs
without claude-cli installed and without API credits.
End-to-end smoke verified against a real `claude --print --model haiku`
invocation: model emitted `<use_tools>` block with toolu_add_001 +
{"a":12,"b":30}, adapter parsed back into a `tool-call` content block,
finishReason 'tool-calls'.
* feat(ai/claude-cli): harden subagent isolation, env scrub, verbose + stdin robustness
Four defensive fixes to the claude-cli provider so a subagent call behaves
identically regardless of the host's ambient Claude Code config:
- Agent isolation: pass `--tools ''` and `--strict-mcp-config` so the subprocess
runs as a raw LLM with no built-in tools and no inherited user MCP servers.
Without `--strict-mcp-config`, each call boots the user's MCP servers (including
gbrain's own), causing recursion plus PGLite single-writer lock contention.
- Env scrub: drop ANTHROPIC_API_KEY / ANTHROPIC_AUTH_TOKEN / ANTHROPIC_BASE_URL
from the child env so the CLI authenticates via its own OAuth subscription
session. An inherited API key silently flips billing to per-token API usage,
the exact setup this recipe exists to replace.
- Verbose-mode compat: with `"verbose": true` in ~/.claude/settings.json,
`--print --output-format json` emits an event array instead of a bare result
object. Tolerate both shapes and select the result event.
- stdin robustness: handle the child stdin 'error' event and wrap write/end so a
missing binary (ENOENT) or early child death (EPIPE) rejects cleanly instead of
crashing the worker with an unhandled error.
Adds unit coverage for the env scrub, the isolation argv, and the verbose event
array. Verified against claude CLI 2.1.x.
* test(ai/claude-cli): cover verbose-array no-result + missing-binary reject paths
Two error branches in the hardened claude-cli provider had no coverage: the
verbose event-array path when no result event is present, and a missing binary
surfacing as a clean spawn-failed rejection. The missing-binary case is the
deterministic form of the stdin/EPIPE robustness; a synchronous stdin-write
throw is not reliably triggerable in a unit test, so the real ENOENT path the
handlers defend is exercised instead. Both reuse the existing shell-stub harness.
---------
Co-authored-by: jarvisdoes <258486803+jarvisdoes@users.noreply.github.com>
Co-authored-by: Marco Maldonado <34176133+loweaxerium@users.noreply.github.com>
queue.add() with an idempotency_key returns any existing row regardless
of status. This means dead jobs (exhausted retries from a transient
provider outage) permanently block re-submission of the same work —
even after the underlying issue is fixed.
Fix: when the existing row is dead or cancelled, NULL its
idempotency_key (preserving the row for audit) and fall through to the
INSERT path so a fresh job can be created.
Affects dream synthesize children that died during provider migrations
(429 rate-limit on old Anthropic proxy, tool-results-missing on old
OpenRouter). 45 dead children were blocking re-synthesis of transcripts
in production.
Includes 4 new tests covering dead, cancelled, completed, and active
status interactions with idempotency dedup.
Co-authored-by: Rafael Reis <rafael.reis@contabilizei.com.br>
OpenRouter's POST /api/v1/rerank is wire-compatible with gateway.rerank()
({query, documents, model} → {results: [{index, relevance_score}]}). This
adds a recipe-only reranker touchpoint declaring four models:
- cohere/rerank-v3.5 (default; $0.001/search)
- cohere/rerank-4-fast ($0.002/search, 32K context)
- cohere/rerank-4-pro ($0.0025/search, SOTA quality)
- nvidia/llama-nemotron-rerank-vl-1b-v2:free (multimodal)
Unlike embedding/chat, the reranker path strictly enforces the models
allowlist — the openai-compat extended-model bypass does not apply. New
rerank models must be added to this recipe before they can be called.
The cost_per_1m_tokens_usd value is a pseudo-rate for the budget tracker's
chars/4 heuristic — Cohere bills per-search, not per-token. At ~4K chars
the estimated cost is in the right ballpark.
Recipe-only change; no gateway or search-layer modifications. gateway
auto-concatenates path → .../api/v1/rerank.
Adds hermetic unit test (test/openrouter-reranker-recipe.test.ts) covering
shape, models, default_model, path, max_payload_bytes, default_timeout_ms,
and cost field. No DB, no env mutation — survives the parallel 8-shard
fan-out.
Verified: bun run verify (30/30 green); 285 targeted recipe+rerank+budget
tests pass.
Co-authored-by: Hippityy <Hippityy@users.noreply.github.com>
`gbrain doctor --remediation-plan` printed two consecutive lines that
contradicted each other when the brain was below target AND the target
was unreachable with autonomous remediation:
Brain score: 45/100 → target 90
Target unreachable: max with autonomous remediation is 70/100.
No remediations needed. Brain is at target.
The second sentence hid the real next step (configure the prereqs that
would lift `max_reachable_score`) and made the brain look healthy when it
was not.
Fix: gate the "Brain is at target" line on `brain_score_current >=
targetScore`. When the plan is empty AND the brain is below target, the
"Target unreachable" line above is already the user-facing explanation;
the `Blocked checks` block below surfaces the manual gap.
Extracted `renderRemediationPlanLines(plan, targetScore): string[]` as a
pure helper alongside `runRemediationPlan` so the regression coverage
asserts on the rendered output directly rather than mocking
`console.log`. `runRemediationPlan` now joins the lines verbatim through
console.log; behavior is byte-identical for every case other than the
fixed contradiction.
Five regression tests cover: unreachable-and-below-target (the bug
case), reachable-and-at-target, exact-target, below-target-with-plan,
unreachable-with-partial-plan. 38 tests across the adjacent doctor test
files stay green; `bun run typecheck` clean.
Idempotency was keyed on atom rows alone — a page the LLM judges
un-atomizable leaves no row, so it re-entered the discovery window every
run. Two production consequences: --drain false-stopped with
no_progress once the window head was mostly zero-yield pages (remaining
frozen while batches report +0), and every nightly re-spent extraction
budget on the same pages.
Fix:
- After a SUCCESSFUL chat call that parses to zero atoms, stamp the
source page with frontmatter.atoms_scan_hash = contentHash16. LLM
failures take the catch path and stay retryable.
- discoverExtractablePages + countExtractAtomsBacklog (both variants)
exclude pages whose stamp matches the CURRENT content hash prefix —
content edits re-eligibilize, mirroring atom-row staleness semantics.
- Drain no_progress now recounts the backlog on a zero-atom batch and
only stops when it genuinely didn't shrink — tombstoning IS progress.
Tests: +2 pure-loop drain cases (shrinking backlog continues / flat
backlog stops) and +3 PGLite integration cases (stamp + exclusion /
content-change re-eligibility / failed chat does not stamp).
29 pass / 0 fail across the two files; tsc clean.
Co-authored-by: 陈源泉 <chenyuanquan@chenyuanquandeMac-mini.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
synthesize-concepts.ts's design comment says extract_atoms stamps a
`concepts:` frontmatter field on each atom and :92 consumes ONLY that
field — but the extractor never wrote it, so the atoms → concepts
pipeline was dead end-to-end: every cycle reported "synthesize_concepts:
skipped — no atoms with concept refs" no matter how many atoms
accumulated (696 page-derived atoms / 0 with concepts on our production
brain before an external backfill).
Fix, all on the extractor side (no synthesize change needed):
- EXTRACT_PROMPT asks for `concepts` (1-3 kebab-case TOPIC labels) with
an explicit reuse-over-coinage instruction — labels must cluster,
since synthesize_concepts only materializes groups of >=2.
- parseAtomsResponse validates labels (kebab regex, max 3, drop
invalid; empty -> undefined).
- The putPage frontmatter write stamps `concepts` alongside lesson /
source_quote.
Tests: 4 parse cases + an end-to-end regression that goes extractor ->
real frontmatter -> synthesize_concepts' OWN DB query path -> concept
page. The existing tests fed synthesize via the `_atoms` seam, which is
exactly how this gap survived.
Validated in production ahead of this PR by stamping the same shape
externally: the next synthesize_concepts run wrote 33 concept pages
(T2=7/T3=26) from 60 stamped atoms, zero failures.
Co-authored-by: 陈源泉 <chenyuanquan@chenyuanquandeMac-mini.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The wrapper script that 'gbrain autopilot --install' writes to
~/.gbrain/autopilot-run.sh sources ~/.bashrc to inherit PATH for the
exec'd gbrain binary (which has a '#!/usr/bin/env bun' shebang). The
standard Debian/Ubuntu ~/.bashrc ships a non-interactive guard that
returns early when bash is launched non-interactively (cron, launchd,
systemd) — so PATH exports operators add to ~/.bashrc never reach the
wrapper subprocess.
The result: the wrapper dies silently with 'env: bun: No such file or
directory', leaves a stale lockfile, and every subsequent cron tick
hits the lockfile and bails. The nightly dream cycle hangs waiting on
a worker that never comes back, and the wrapper's own 10-min
stale-lock window is the only thing that can recover it.
This bites every operator whose bashrc is the standard distro default
(which is the default), and there is no warning at install time.
Fix: prepend ~/.bun/bin to PATH directly in the wrapper, so it is
self-contained regardless of which init file the OS loaded. Add a
regression test alongside the existing zshenv/zshrc source-order test
(v0.36.1.x #966) so this class of bug stays caught.
The conversation_format_coverage check recommended `gbrain config set
conversation_parser.llm_fallback_enabled true`, but that config key is dead
(never read) — see #1890. The recommendation is a no-op and misleads users into
thinking a fallback will kick in. Drop it; keep the actionable `scan` hint.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Backlinks Minion jobs submitted with an empty payload (the
sync→embed→backlinks chains enqueued after every ingestion) defaulted to
action='fix', rewriting tracked brain pages with generated "Referenced in"
timeline bullets on every routine run — 129 vault files polluted in one
day on our production brain before we traced it.
This contradicts the documented intent in src/core/cycle.ts
(runPhaseBacklinks): "Maintenance cycles must not rewrite tracked brain
pages with generated 'Referenced in' timeline bullets. [...] the legacy
filesystem fixer remains available explicitly via `gbrain check-backlinks
fix`." — the jobs-worker handler simply inverted that default.
Fix: default to 'check'; 'fix' requires explicit opt-in via
'{"action":"fix"}' (the documented submit shape) or
`gbrain check-backlinks fix`. Both explicit paths are unchanged.
Adds a structural regression test (fix-wave-structural.test.ts precedent)
pinning the default, since the handler dynamically imports
runBacklinksCore and walks a real repo dir — a behavioral test would
require mocking that hides the regression behind a test seam.
Co-authored-by: Valentin Ferriere <valentin@v-labs.fr>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`extractStaleFromDB` stamped `links_extracted_at` with each page's read
`updated_at` (the D4 race-fix). But the stale predicate also flags
`links_extracted_at < LINK_EXTRACTOR_VERSION_TS`. Any page last edited
BEFORE the version timestamp got stamped below the threshold, so the
version arm re-flagged it stale on every run — an infinite re-extract
loop that never cleared the lag.
Since the v112 watermark column ships with no backfill, every
pre-existing page starts stale, and most pre-date the version bump. In
practice this left ~97% of pages permanently stale: `extract --stale`
reported "done" each run but `links_extraction_lag` never dropped.
Fix: stamp `GREATEST(read updated_at, versionTs)`. Old pages lift to the
threshold so the version arm clears; a real future edit still advances
`updated_at` past the stamp, so the CDX-1 edited-after-stamp race
protection is preserved.
Adds a regression test: a page with `updated_at` before
LINK_EXTRACTOR_VERSION_TS must clear after extract AND stay clear on a
second run (the existing tests only used now()-dated pages, so the
old-page case was uncovered).
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(drift): skip node_modules/dist/build in multi-source drift walk
The drift walker recursed into node_modules (50k+ files in RN/Astro repos),
exhausting the time budget before completing, so multi_source_drift always
reported 'walk hit limit/timeout' on real projects. Skip heavy non-content
dirs + add a deadline check on directory descent.
* fix(integrity): skip inline-code spans + [Source:] citations in bare-tweet detection
Recipe/doc pages that show the CORRECT citation format inline (e.g.
`Tweeted about {topic} [Source: X, @handle, date]`) were false-flagged.
The fenced-code skip didn't cover inline backticks; add inline-code
stripping + an explicit-citation exemption.
---------
Co-authored-by: Son Le <tuanson1200@gmail.com>
* test(e2e): drop flaky wall-clock bounds in minions-resilience
The runaway-dead-letter and cascade-kill tests asserted tight real-clock
upper bounds (<2000ms, <3000ms) on top of already-complete terminal-state
checks. Those bounds carry no correctness signal — the dead/cancelled status
and abortedChildren==10 assertions fully prove behavior — and flake on loaded
CI runners where the stall/timeout sweep cadence varies. Removed both bounds,
kept the diagnostic values, de-promised the test titles.
* test(e2e): kill order-dependence + no-op assertions in mechanical
- traverse_graph: self-contained (re-adds its own idempotent link) and asserts
the linked company is reachable, instead of depending on a prior it() and
only checking array shape.
- file_list-without-slug: seeds its own >100 rows instead of relying on the
previous test's 150 surviving in the DB; asserts the cap is exercised.
- precision@5: add a loose floor (every known-item query surfaces >=1 truth doc
in top-5) so a 0% retrieval regression no longer passes silently.
- get_health: assert value bounds (page_count==16, embed_coverage 0..1) not just
typeof; get_chunks: assert non-empty text, numeric non-decreasing chunk_index,
and that the page name appears, instead of toBeTruthy on chunks[0].
* test(e2e): strengthen graph/search quality assertions + close coverage gaps
graph-quality: truncate+reseed 'config' in truncateAll (kills config leak where a
setConfig test throwing before its finally bleeds into later tests); replace
toBeGreaterThan(0) link/timeline floors with fixture-derived minimums; assert exact
attendee slugs are 'attended' instead of a vacuous .every; pin autoLinks.created to
the provable 2 (Alice+Acme); add direction out/both + depth:2 multi-hop and a
cycle-safety (A->B->A terminates) test.
search-quality: fix the vacuous detail=low vector test; assert pedro returns >=2
chunks; assert detail=high includes the timeline chunk; add empty-query and
zero-vector no-throw edge tests.
* test(e2e): self-contain multi-source sync test + assert ledger cascade
Break the sequential dependency where 'performSync no sourceId' relied on a prior
test writing sync.repo_path — it now sets its own config. Add the missing
file_migration_ledger COUNT(*)==0 cascade assertion. Tighten the source_id default
check from toContain('default') to exact "'default'::text".
* test(e2e): make migration-flow HOME/PATH swap throw-safe
The suite repoints process.env.HOME/PATH to a temp dir and only restored them in
afterAll, so a mid-test throw left HOME dead for the rest of the bun process and
silently broke sibling suites. Wrap each test body in try/finally restore + a
defensive restore at the top of beforeEach.
* test(e2e): loud-skip jsonb-roundtrip + doctor-progress
Both skipped silently with no DATABASE_URL, giving zero signal the regression guard
never ran. Add the console.log skip line matching the sibling e2e files.
* test(e2e): robust check-update contract + find_orphans tool coverage
upgrade: the 'no-releases' test hard-asserted update_available===false, which flips
to failing the moment the repo has a real release. Assert the JSON contract shape
(boolean update_available, current_version===VERSION, typed optional fields) instead.
mcp: add find_orphans to the asserted generated tool names.
splitLargeNode can only break up a node that exposes a `body` with >= 2
named children. A node without one -- a giant object/array literal, a single
huge assignment, a massive template literal -- is emitted whole. On real
source that yields a chunk far larger than the embedder's context window; the
embedder then rejects it ("input exceeds context length") and it is never
embedded. Example: a 372 KB service file produced 113 chunks, one a single
281 KB (~70k-token) node -> permanently unembedded.
Add a final safety-net pass (capOversizedChunks) that recursively re-splits
any chunk over a token budget (default 2000, configurable via maxChunkTokens),
with a hard character split as a last resort for no-whitespace content
(minified one-liners). Normal files are untouched.
Verified: that 372 KB file now yields 188 chunks, max ~1.5k tokens, zero
oversized; a small file is unchanged.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
gbrain think printed "## Gaps" twice: the synthesis prompt asked the model for a Gaps section inside the answer body AND a separate structured gaps array, then both render paths printed both — the CLI human output (src/commands/think.ts) and the --save page (persistSynthesis in src/core/think/index.ts).
Make the structured gaps array the single source. The prompt now routes gaps into the array, not an answer-body section. New exported stripGapsSection(answer) defensively removes any "## Gaps" section a model still emits (any heading level, case-insensitive, bounded by the next same/higher heading); both render sites call it, so the dedup is structural rather than dependent on the model obeying the prompt.
Adds test/think-gaps.test.ts (hermetic): strip helper across heading levels / case / no-section / mid-document / false-match, the one-render-only repro, and the prompt contract.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Time Attakc <89218912+time-attack@users.noreply.github.com>
PGLite can return zero rows from INSERT ... ON CONFLICT DO UPDATE ...
RETURNING in no-op/trigger edge cases. The previous code called
rowToPage(rows[0]) unconditionally, so rows[0] was undefined and
rowToPage threw "undefined is not an object (evaluating 'row.deleted_at')",
which aborted the import and silently skipped the file during sync.
getPage() already has the empty-rows guard; putPage() was missing the
parallel one. The row was in fact written by the upsert, so re-read it
via getPage() instead of crashing. On a real monorepo index this
recovered ~19% of files (985/5148) that were failing to embed.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Time Attakc <89218912+time-attack@users.noreply.github.com>
DEF_TYPES listed only canonical symbol-type names (function, class, interface, ...). But normalizeSymbolType in the code chunker canonicalizes only some tree-sitter node types and lets the rest fall through type.replace(/_/g, ' '). So method_declaration is stored as 'method declaration', struct_specifier as 'struct specifier', protocol_declaration as 'protocol declaration'. None were in DEF_TYPES, so code-def returned 0 hits for every method, constructor, field, C struct, and Swift protocol. The plain 'struct' entry never matched either. Add the fallthrough definition forms. Read-path only; no reindex needed (0 -> N on existing indexes).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Time Attakc <89218912+time-attack@users.noreply.github.com>
The code-fence-wrap detector in lintContent used the /m multiline flag, so
^/$ matched start/end of any line. The rule fired on any page that simply
contained a ```markdown code block, not only pages wrapped end-to-end.
The matching fixer in fixContent has no /m flag, so it can only strip
whole-file wrappers. Result: detected issues were marked fixable: true,
yet fixContent could never strip them. `gbrain dream` reported
"0 fix(es) applied, N remaining" perpetually for the rule.
Drops the /m flag from the detector so detector and fixer stay in sync.
Whole-file wrapper detection is preserved; inner code blocks no longer
trigger the rule.
Real-world impact: a brain with 5 docs pages containing markdown examples
(skill READMEs, decision registry, journal templates) reports 5 phantom
"fixable: true" issues every dream cycle, never converging. After this
fix, the dream-cycle lint phase reports only real-and-unfixable issues
(missing frontmatter, missing title/type) which is the intended behavior.
Two regression tests added in test/lint.test.ts:
- Page contains a single inner ```markdown block
- Page contains multiple inner ```markdown blocks
Both assert no code-fence-wrap issue is reported. The existing
"detects wrapping code fences" test (true-positive case) continues to
pass; total tests in the file are 18 -> 20.
Co-authored-by: Thomas Chung <thomaschung@macbookair.lan>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Time Attakc <89218912+time-attack@users.noreply.github.com>
`findByTitleFuzzy` on both `postgres-engine.ts` and `pglite-engine.ts`
has no `source_id` filter and no `deleted_at IS NULL` filter. `tryFuzzyMatch`
in `src/core/entities/resolve.ts` got both of those filters via #1436
(v0.41.13.0) for exactly the reasons that apply to its sibling here:
fuzzy resolution can suggest cross-source slug candidates that the
caller then silently drops at the FK filter (or worse, picks a
soft-deleted page).
This is the missing twin of #1436. In multi-source brains, the live-mode
auto-link resolver invoked from `put_page` (`operations.ts:937`) calls
`engine.findByTitleFuzzy` with no scope. When two sources contain pages
with similar titles (`people/alice-example` on `source-a`,
`people/alice-other` on `source-b`), the fuzzy lookup can return the
wrong-source slug, which then fails the downstream `allSlugs` /
`addLink` FK filter — the link silently doesn't get created, and from
the caller's view the resolver "failed" even though the page existed
under the right source.
Reproducible with a 2-source PGLite setup + identical-title pages on
both sides; the fuzzy call returns a slug whose `source_id` doesn't
match the put_page caller's source.
- 2-source brains: auto-links between same-title-different-source pages
now resolve under the caller's source instead of the wrong neighbor.
- Soft-deleted pages can no longer be returned as fuzzy candidates
(mirroring the resolve.ts fix from #1436).
- 1-source brains: no behavior change. `sourceId` is optional; when
omitted the SQL takes the pre-existing unscoped path.
- `engine.ts`: add optional 4th `sourceId` param to the
`findByTitleFuzzy` interface + JSDoc explaining the scope semantics.
- `postgres-engine.ts` / `pglite-engine.ts`: implement the param via a
conditional SQL branch that adds `AND source_id = $N AND
deleted_at IS NULL` when `sourceId` is set; existing query path
unchanged when omitted.
- `link-extraction.ts`: add optional `sourceId` to `makeResolver` opts,
forward to `findByTitleFuzzy` in step 3 of the resolve chain.
- `operations.ts`: pass `opts?.sourceId` to `makeResolver` from the
live-mode put_page resolver (the place that already knows the
caller's source).
- New unit tests in `test/link-extraction.test.ts` (2 cases):
- `opts.sourceId` is forwarded to `findByTitleFuzzy` when set.
- `opts.sourceId` omitted → `findByTitleFuzzy` receives `undefined`
(back-compat).
- `bun run typecheck` clean.
- `bun test test/link-extraction.test.ts test/entity-resolve.test.ts
test/operations.test.ts test/extract.test.ts` — 145/145 pass.
Co-authored-by: Time Attakc <89218912+time-attack@users.noreply.github.com>
* fix(models): dispatch subcommand reads args[0] not args[1]
`gbrain models doctor` silently fell through to the read view
instead of running the reachability probe.
`runModels` checks `args[1] === 'doctor'`, but the caller —
`handleCliOnly(command, subArgs)` in `src/cli.ts:113` — passes
`subArgs` (the leading command token already stripped). So inside
`runModels`, args[0] is the subcommand. args[1] is undefined.
The doctor probe path has been unreachable from the CLI since the
handleCliOnly refactor. `gbrain models help` happened to work by
falling through to the `--help` flag detection.
Two-char fix: `args[1]` → `args[0]` on both branches of the
ternary.
Verified by manual probe — `gbrain models doctor` now prints
"Model reachability probe:" with per-model results (real production
brain, 4 touchpoints probed):
```
Model reachability probe:
embedding_config ollama:bge-m3 ok (0ms)
reranker_config (none) ok (0ms)
chat lmstudio:mistralai/magistral-small-2509 unknown (5012ms)
[chat(lmstudio:mistralai/magistral-small-2509)] probe timed out after 5s
expansion lmstudio:google/gemma-4-e2b ok (535ms)
Summary: 3/4 reachable.
```
RECOVERY REBUILD 2026-05-26 of original 20ed0eee.
* fix: honor --help before doctor dispatch to avoid running probes on `models doctor --help`
Codex review of #1428 flagged that the args[1]→args[0] rewrite
regressed `gbrain models doctor --help` into running network
probes instead of printing usage. The original args[1]-shaped
ternary happened to dodge this by always falling through to the
args.includes('--help') branch when args[1] === 'doctor' was
false; the new args[0] code checks doctor first, so --help no
longer wins.
Reorder ternary: `hasHelp` is computed FIRST from
(--help / -h / args[0] === 'help'), then the sub is hasHelp ?
'help' : args[0] === 'doctor' ? 'doctor' : 'read'.
Addresses codex review P2 on PR #1428.
---------
Co-authored-by: Time Attakc <89218912+time-attack@users.noreply.github.com>
The HTTP MCP server's 401 responses missed the `resource_metadata`
parameter in the WWW-Authenticate header. MCP authorization spec
(2025-06-18 draft §5.1) and RFC 9728 require:
WWW-Authenticate: Bearer resource_metadata="<url>"
MCP-aware OAuth clients (claude.ai, Cursor, etc.) use that URL to find
the authorization-server discovery doc without the user manually
configuring the issuer. Pre-fix the header shipped only `Bearer
error="invalid_token", error_description="..."` and MCP clients silently
failed to begin the OAuth flow — symptom on claude.ai's UI was "Couldn't
reach the MCP server" even when discovery + /token + /register all
responded 200 individually.
The `requireBearerAuth` middleware in @modelcontextprotocol/sdk's
BearerAuthMiddlewareOptions already supports a `resourceMetadataUrl`
parameter. Two call sites (`/mcp` and `/ingest`) now pass it.
Verified against a real claude.ai connector attempt: pre-fix the
connector showed "Couldn't reach the MCP server" with no OAuth redirect.
Post-fix the connector successfully begins the authorization flow.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Time Attakc <89218912+time-attack@users.noreply.github.com>
llama.cpp's llama-server rejects /v1/embeddings requests with more inputs
than its launch --batch-size (default 32): "batch size 100 > maximum allowed
batch size 32". gbrain sends batches of 100, so any page with >32 chunks fails
to embed, and embed --stale then trips the Postgres statement_timeout retrying
the doomed batches. The existing token-based protection (max_batch_tokens)
can't bound item count — N tiny chunks fit under any token budget.
Add an optional max_batch_items count cap to EmbeddingTouchpoint, enforced as a
hard re-split after the token split in embed(), and set it to 32 on the
llama-server recipe (replacing no_batch_cap: true, which wrongly assumed
llama.cpp has no per-request item cap). A declared item cap also suppresses the
missing-max_batch_tokens startup warning.
Co-authored-by: Time Attakc <89218912+time-attack@users.noreply.github.com>
The cron daily briefing writes 90_Briefings/<date>.md, which gets
re-ingested on the next sync and then dominates tomorrow's
getRecentSalience output as pure self-reference (observed: top
result score 0.9956, everyone else clustered at 0.587).
Filter `p.slug LIKE 'briefings/%'` out of getRecentSalience in both
the PG and PGLite engines. Suppressed by default; callers can still
opt in by passing `slugPrefix: 'briefings/'` (or `--kind briefings/`
from the CLI). search and list_pages are unaffected.
Co-authored-by: CTO <cto@timelycare.local>
Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Time Attakc <89218912+time-attack@users.noreply.github.com>
On Windows, `core.autocrlf=true` is the default and SKILL.md files are
checked out with CRLF line endings. `extractTriggers` used regexes
anchored to `\n` (`/^---\n.../` and `/^triggers:\s*\n.../`), which
never matched `\r\n`, so the parser returned `[]` for every skill.
Result: `gbrain doctor --fast --json` on Windows reported every skill
not in `OVERLAP_WHITELIST` (39 of 42) as a false `mece_gap` warning —
even though `skill_conformance` in the same run reported "42/42 skills
pass". CI runs Ubuntu-only so the divergence never surfaced.
Fix: normalize CRLF → LF at the top of `extractTriggers`. Single-line
change preserves existing LF behavior. Function is now exported so the
test can target it directly.
Tests: added `describe("extractTriggers")` block covering LF input,
CRLF input (regression case), missing frontmatter, missing triggers
field, and quote-stripping. All 30 tests in `check-resolvable.test.ts`
pass.
Verified locally on Windows: `gbrain doctor --fast --json` now reports
`resolver_health: ok, 42 skills, all reachable` (health_score 90 → 95).
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Time Attakc <89218912+time-attack@users.noreply.github.com>
The signal-handler test asserted an absolute liveReporters===0 on a
module-global set, so any other test file in the shard holding a live
reporter flaked it — it red-flagged ~12 unrelated PR runs and one master
push in two days, purely as a function of shard composition. The delta
form pins the same claim (50 reporter lifecycles leak nothing).
The 15-minute shard timeout cancelled 13 fully-passing runs under
parallel PR load (PGLite WASM cold-starts stretch shards); the
test-status gate then reported the cancellations as failures.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Qwen3-Embedding family on Ollama supports Matryoshka truncation via the
'dimensions' field on /v1/embeddings. Without this passthrough, gbrain
ignores user-selected reduced dims and the provider returns its native
size, causing dim-mismatch errors against brains configured for narrower
widths (e.g. existing 1536-dim brains).
Matches by bare name 'qwen3-embedding' or any tag variant
'qwen3-embedding:0.6b' / ':4b' / ':8b'.
Native dims: 0.6B=1024, 4B=2560, 8B=4096. All MRL-truncatable.
5 new tests; full AI suite 137/137 green.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Time Attakc <89218912+time-attack@users.noreply.github.com>
Add the OpenClaw-required top-level id to openclaw.plugin.json, export a
direct register(api) entrypoint from src/openclaw-context-engine.ts, add a
manifest regression test, and document that skillpack harvest must preserve
OpenClaw-native manifest fields (id, configSchema, contracts).
llms bundles regenerated (bun run build:llms) — no content drift.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Filip <FilipHarald@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Takeover of #2719 (fork head; rebased onto origin/master).
- writeProposed now writes both best.md (current-best pointer) and
proposed.md (stable human-review artifact); returns the proposal path.
- Orchestrator reports the real proposed.md path for accepted --no-mutate runs.
- Tutorial updated; llms bundles regenerated (no content drift — tutorial
is not inlined in the bundle).
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Ziyang Guo <121015044+RerankerGuo@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Point the OpenClaw and Hermes anchors in the "Have your agent install
it" section at their real upstream repos; the previous openclawagents
org URLs 404. Regenerated llms-full.txt to match.
Takeover of #1961 (fork branch) rebased onto current master.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: jessems <jessems@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Ports #3015 by @jdewoski-cmd onto current master:
- src/core/orphan-policy.ts centralizes the orphan-reporting exclusion
convention so `gbrain orphans`, doctor's orphan_ratio, and both engines'
getHealth orphan_pages can no longer drift.
- getHealth stale_pages now uses the link-extractor stale watermark
(countStalePagesForExtraction) so health agrees with what `gbrain extract
--stale` will actually process.
- New `gbrain maintain` command: dry-run by default, `--safe` applies only
the conservative runbook actions (DB-backed stale extraction + source-scoped
dream cycles for doctor cycle_freshness findings), `--json` for structured
before/action/after reports. Frontmatter mutations, schema-pack upgrades,
and semantic hub links stay review-only by design.
Changed from the original PR: the shared defaults carried slugs specific to
the contributor's own brain ('josa-secrets/', '*-ga4-property-id.md',
'*-josa-test', literal 'welcome'/'untitled' fixtures). Global defaults now
carry only GBrain-wide conventions; brain-specific exclusions move to a new
per-brain config plane the policy reads through loadOrphanPolicyOverrides:
gbrain config set orphans.exclude_prefixes "my-private-folder/,archive/"
gbrain config set orphans.exclude_slugs "some-one-off-page"
Both engines' getHealth and the orphans command thread the overrides;
tests cover the neutral defaults, the override plane, and health parity.
Also registered `maintain` in CLI_ONLY_SELF_HELP so `gbrain maintain --help`
reaches the command's own usage block.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: jdewoski-cmd <jdewoski@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The generic wikilink pass (issue #972) forwarded the raw literal to
resolveBasenameMatches, whose index is keyed by final path segments —
so [[notes/struktura]] (any dir outside DIR_PATTERN) silently produced
zero edges from `extract links --source db` and put_page auto-link,
while the FS extractor resolves the identical content (resolveSlugAll
strips the dirname before its basename lookup).
Query by the literal's final segment, then keep only matches whose slug
ends with the written path — [[notes/struktura]] can resolve to
vault/notes/struktura but never attach to wiki/struktura. Bare literals
are untouched. Flag-gated by link_resolution.global_basename as before.
Co-authored-by: YMYD <53603073+OJ-OnJourney@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Time Attakc <89218912+time-attack@users.noreply.github.com>
Closes#769. Every re-embed pass clobbered code-chunk metadata
(language, symbol_name, symbol_type, start_line, end_line,
parent_symbol_path, doc_comment, symbol_name_qualified) to NULL,
disabling code-def queries across thousands of indexed chunks.
Two complementary fixes:
embed.ts — three re-upsert call sites (embedPage, embedAll
non-stale, embedAllStale autopilot path) build ChunkInputs from
loaded chunks; they were stripping the 8 metadata fields. New
preserveCodeMetadata helper threads those fields through
consistently. Integrated cleanly with v0.34.4.0's cursor-paginated
--stale hardening — the wrap sits inside the worker function
between embedBatchWithBackoff and engine.upsertChunks.
postgres-engine.ts + pglite-engine.ts — upsertChunks ON CONFLICT
clause OVERWROTE metadata columns from EXCLUDED. Asymmetric vs the
embedding/embedded_at columns which already used a chunk_text-gated
CASE pattern (re-chunk → trust EXCLUDED, re-embed → COALESCE
preserve). Applied the same pattern to all 8 metadata columns.
Three regression tests in test/embed.serial.test.ts cover --stale
(autopilot), --all, and --slugs paths. Each loads a chunk with
full metadata, runs runEmbed, and asserts engine.upsertChunks
receives the metadata round-tripped. Coexists with master's D5
embedBatchWithBackoff test block.
Backfill required after deploy: \`gbrain sync --strategy code
--force --source <id>\` per code source to re-populate metadata via
the chunker. Without backfill, existing NULL columns stay NULL —
re-embed alone never produces metadata, only the chunker does.
Originally landed as part of PR #768 (the wave that bundled #767 +
fix; this PR carries the #769 fix alone with no scope overlap.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Time Attakc <89218912+time-attack@users.noreply.github.com>
* feat(synopsis): tail-truncate documentText for small-model chat handlers
Small local chat models (Gemma 4 E2B, Qwen3 4B) get dramatically
slower on long contexts even at 131K declared windows. A 73K-char
page synopsis on Gemma 4 E2B takes 60-120s, exceeding the worker's
default 30s `lockDuration` and tripping `lock-lost` errors.
Add `SYNOPSIS_DOC_MAX_CHARS` env-overridable cap (default 32768
chars, ~8K tokens) applied in `buildUserPrompt`. Truncate the TAIL
so the head (title, frontmatter, intro) preserves the document-level
anchor the synopsis needs.
Anthropic Haiku is unaffected at this cap; bump via
`GBRAIN_SYNOPSIS_DOC_MAX_CHARS` for frontier models that want
richer document anchoring.
Belt-and-suspenders companion to commit 0aaff691 (--lock-duration
flag on the worker). Combined: bumping lock TTL gives the handler
more time, AND truncating doc cap makes the handler complete faster.
Either alone helps; both together get the synopsis backfill running
reliably on small local LLMs.
Verified: real 1383-chunk personal brain backfill at
GBRAIN_SYNOPSIS_MODEL=lmstudio:google/gemma-4-e2b +
GBRAIN_SYNOPSIS_DOC_MAX_CHARS=16384 +
`gbrain jobs work --concurrency 4 --lock-duration 300000`
transitions from "lock-lost on every transcript page" to "no
deaths, no stalls, steady throughput."
RECOVERY REBUILD 2026-05-26 of original ac213aa6.
* fix: fold SYNOPSIS_DOC_MAX_CHARS into corpus_generation hash
Codex review of #1427 flagged that changing GBRAIN_SYNOPSIS_DOC_MAX_CHARS
shifts the synopsis prompt + downstream embeddings for long documents
but was NOT folded into the computeCorpusGeneration hash. Pages
re-embedded with a different cap would retain the same
corpus_generation, defeating the v0.40.3.0 D27 P1-5 cache invalidation
contract.
Three changes:
1. Export SYNOPSIS_DOC_MAX_CHARS from src/core/page-summary.ts
2. computeCorpusGeneration accepts optional synopsisDocMaxChars param.
When set, folded into hash via '|doc_cap=<N>'. Omitted for
non-synopsis modes (title / none don't consult the cap) so existing
pre-PR caches stay valid for those.
3. Service-layer call sites (2 in contextual-retrieval-service.ts)
pass SYNOPSIS_DOC_MAX_CHARS when attemptMode/resolution.mode is
per_chunk_synopsis, undefined otherwise.
4. import-file.ts inline path passes undefined (per_chunk_synopsis
refused upstream there).
One-time effect: per_chunk_synopsis pages re-embedded post-PR get a
NEW corpus_generation including the cap. v0.40.3.0 query_cache.page_generations
contract auto-invalidates cached query results on first re-embed.
Future cap changes track correctly.
Addresses codex review P2 on PR #1427.
---------
Co-authored-by: Time Attakc <89218912+time-attack@users.noreply.github.com>
Synth pages written by dream synthesize currently open straight into
detail (quotes, cross-references) with no framing, so a reader who
lands on the page later — without the source transcript in front of
them — has no way to tell what it's about without reading the whole
thing.
Add OUTPUT POLICY item 5: every new page's body must open with a 2-3
sentence self-contained summary a reader unfamiliar with the source
conversation could understand on its own, before any quotes or detail.
Long-lived minion workers can outlive DB-backed model config changes. Refresh the AI gateway before gateway-backed handlers run so queued cycle/propose_takes work does not fall back to a stale Anthropic default when the operator configured another provider.
Also record the active gateway chat model in propose_takes budget/proposal metadata instead of hardcoding claude-sonnet-4-6, and keep provider:model IDs intact for budget pricing.
Regression coverage verifies queued worker refresh, propose_takes model metadata, nested provider IDs, skipFence threading, and the updated autopilot signal source guard.
Co-authored-by: maxpetrusenkoagent <[REDACTED EMAIL]>
Setting `models.tier.deep anthropic:claude-opus-4-8` silently disabled
think and auto_think: the Anthropic recipe's chat allowlist stopped at
Opus 4.7, the tier-resolved model never joined the extended set that
assertTouchpoint's contract promises for config-chosen models, and the
resulting probe failure was stamped NO_ANTHROPIC_API_KEY — sending the
operator to debug env/keychain when the fix was the model id. Three
fixes, one per layer:
- recipes/anthropic.ts: add claude-fable-5, claude-opus-4-8, and
claude-sonnet-5 to chat models; claude-sonnet-5 to expansion models.
- gateway.ts reconfigureGatewayWithEngine: resolve all four tiers and
register the results as extended models, honoring the documented
contract for models.default / models.tier.* (model-resolver.ts
docstring). A tier-only model now validates like a chat/expansion one.
- think/index.ts: when the gateway client can't be built, re-probe and
label honestly — MODEL_NOT_USABLE:<reason> for unknown_model /
unknown_provider, NO_ANTHROPIC_API_KEY only for the actual missing-key
case; the stub answer carries the probe detail and fix hint.
Tests: recipe-list presence pins; a new gateway-tier-extended-models
suite proving a fictional tier model validates post-reconfigure (and an
unconfigured one still doesn't); think-pipeline coverage for the honest
label (unknown_model beats missing-key even keyless); the existing
non-explicit bogus-provider test updated from the old catch-all label to
the honest one (no-throw contract unchanged).
Verified live: a brain with tier.deep=claude-opus-4-8 had think degrade
to gather-only with the misleading key warning; with this change the
probe passes and synthesis runs.
Co-authored-by: Paolo Belcastro <p3ob7o@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(dream): --once for one-shot phase runs without toggling config gates
Fixes the "toggle enabled true, run, toggle back to false" workaround
that gbrain doctor's extract_atoms_backlog message implicitly
recommends and that #2860's reporter had to script around: with an
external orchestrator running `gbrain dream --phase patterns` on a
cadence outside the autopilot, the only way to run patterns once was
`config set dream.patterns.enabled true` -> run -> `config set ...
false`. A crash between steps left the flag stuck true, and the
autopilot (which polls the same flag) re-enqueued patterns every
cycle -- 119 LLM jobs / ~$400 over 24h before it was caught.
Root cause: `--phase X` only controls which phase FUNCTION cycle.ts
calls; it does not bypass that phase's own `dream.<phase>.enabled` /
`cycle.<phase>.enabled` config read. Each gated phase (patterns,
synthesize, conversation_facts_backfill, enrich_thin, skillopt) reads
its enabled flag internally and skips regardless of how the phase was
selected -- confirmed by reading each phase module, not assumed.
extract_atoms/synthesize_concepts are a DIFFERENT mechanism entirely
(pack-declaration via packDeclaresPhase, not a config .enabled read)
and already have a working one-shot escape hatch: `--drain`. The
existing doctor message for extract_atoms already says `--phase
extract_atoms --drain --window 120`, so no doctor text needed
updating there -- verified by reading src/commands/doctor.ts directly
rather than assuming the paraphrase in the issue was literal.
Design: `gbrain dream --phase <name> --once`. Requires an explicit
--phase (bare --once is a usage error, exit 2) so it can never
force-enable every disabled phase at once in a full/default cycle --
that would recreate the same unbounded-spend risk the flag exists to
prevent. Threaded through CycleOpts as `onceForPhase?: CyclePhase`
(the literal phase name, not a boolean) so the bypass can never leak
to a phase other than the one named, even if a future programmatic
caller passes a wider `phases` array than the CLI does. Never reads
or writes config -- the phase still evaluates its .enabled gate every
call; --once only overrides the boolean OUTCOME for that one
invocation, mirroring the existing --unsafe-bypass-dream-guard /
--input precedents (stderr warning at the bypass point, no new
config-touching code path).
Rejected alternatives (documented per task instructions):
- Making explicit --phase X always bypass .enabled: breaking change
for existing crons that rely on the disabled flag as a cheap no-op;
an upgrade would silently start running LLM/write phases.
- A new subcommand: adds a whole dispatch/help/arg surface that
internally routes through the same override anyway.
- Extending --once to also bypass packDeclaresPhase for
extract_atoms/synthesize_concepts: conflates two different gating
mechanisms (config toggle vs. pack membership) under one flag;
extract_atoms already has --drain, which is purpose-built for its
batched/windowed execution model.
Design was cross-validated by an independent second-model review
(external design consultation) before implementation; its
recommendation to also update the extract_atoms doctor message to
`--once` was NOT adopted because that phase has no .enabled gate to
bypass -- doing so would be a documented no-op, contradicted by
reading src/commands/doctor.ts:3264 directly.
Tests: 9 new (structural CLI-flag wiring in dream-cli-flags.test.ts;
a real PGLite E2E test in dream-patterns-pglite.test.ts proving the
bypass fires AND that dream.patterns.enabled is never written; 4
runCycle-level tests in cycle.serial.test.ts proving onceForPhase
does not leak across phases). Verified 6 of 9 fail against the
pre-fix source (via git stash of source-only changes) to confirm
they're meaningful regressions, not tautologies.
Closes#2860
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(dream): --help short-circuits before --once usage validation
Codex review finding (P2): `gbrain dream --help --once` (no --phase)
called process.exit(2) from the new --once usage-error check inside
parseArgs before runDream's documented IRON RULE ("--help
short-circuits BEFORE any engine-bearing work") ever got a chance to
run -- parseArgs computes ALL its validations unconditionally before
runDream checks opts.help. Repo precedent for this ordering already
exists as a pinned regression test (test/dream.test.ts's "--help
--source whatever prints help and exits 0").
Fix: compute wantsHelp once in parseArgs and exempt the --once
validation when it's set, mirroring that precedent. Added the same
class of pinned tests here: bare `--once` still exits 2 with the
usage hint, `--help --once` prints help and exits 0, and a real
--phase patterns --once run against a PGLite engine proves the
bypass actually fires (falls through to insufficient_evidence
instead of disabled) without writing dream.patterns.enabled. Also
fixed the structural test in dream-cli-flags.test.ts that asserted
the exact pre-fix guard-condition source text.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(dream): --once must require an EXPLICIT --phase, not a derived one
Codex review finding (P3): the --once validation checked the derived
`phase` value, but `phase` gets defaulted implicitly by --input
(implies --phase synthesize) and --drain (implies --phase
extract_atoms) BEFORE that check ran. So `gbrain dream --input <f>
--once` and `gbrain dream --drain --once` both slipped past the
"explicit --phase required" contract silently -- and --once became a
true no-op in both cases: --drain returns from runDream before
onceForPhase is ever read (the drain path doesn't call runCycle at
all), and --input already bypasses the synthesize enabled-gate on its
own via the existing opts.inputFile check, so onceForPhase would
never even be consulted.
Fix: capture `phaseWasExplicit = phaseIdx !== -1` at the very top of
parseArgs, before the --input/--drain defaulting blocks run, and
validate --once against that instead of the derived `phase`. Updated
the usage-error message and --help text to say "an explicit --phase"
so a user hitting this understands why `--input ... --once` doesn't
count.
Tests: 2 new pins in test/dream.test.ts exercising runDream directly
(--input <file> --once exits 2; --drain --once exits 2), plus a
structural test in dream-cli-flags.test.ts pinning that
phaseWasExplicit is captured before both implicit-defaulting blocks.
Updated the two existing structural/behavioral tests whose literal
guard-condition / error-message assertions changed shape.
Verified: dream-cli-flags.test.ts 27/27, dream.test.ts 31/31,
typecheck clean.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: masashiono0611 <masashi.ono.0611@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Garry Tan <garrytan@gmail.com>
* fix(providers): reuse buildGatewayConfig for --model test override (#2863)
`gbrain providers test --model <id>` overrode the gateway with only
embedding_model/chat_model + env, dropping config.provider_base_urls
entirely. A brain configured with a custom endpoint (e.g. a China-region
DashScope base URL) would pass the bare `providers test` (which goes
through configureFromEnv() and does forward base_urls) but fail the
`--model`-scoped probe with a misleading "Incorrect API key" error, even
though the key was valid for the configured endpoint — the probe silently
fell back to the recipe's hardcoded default endpoint instead.
Root cause: two independent, drifted resolvers. The production path
(src/cli.ts#connectEngine, src/core/init-embed-check.ts) builds its
AIGatewayConfig via buildGatewayConfig(), which folds provider_base_urls,
env-sourced local-server base URLs, provider_chat_options, and file-plane
API keys. The --model override branch in runTest() hand-rolled a second,
narrower config object that only carried the overridden model + raw env.
Fix: lift `cfg` out of the existing try/catch (it was already loaded there
for the isolation-warning message) and spread `buildGatewayConfig(cfg)`
into both configureGateway() calls before overriding embedding_model/
chat_model. The isolated --model probe now resolves its endpoint exactly
the way the brain's real import/query path would; only the requested
model is overridden, so the probe still targets exactly the model the
user asked for. Falls back to bare env when no brain is configured yet
(cfg is null), matching prior first-time-install behavior.
Confirmed chat_fallback_chain (also threaded through by buildGatewayConfig)
has no runtime retry effect — it's only consumed to pre-register extended
model ids — so spreading the full production config does not mask an
isolated model's own failures behind a silent fallback.
Other diagnostic surfaces (providers list/env/explain) were checked and
are unaffected: `runProviders()` already calls configureFromEnv() (which
forwards base_urls correctly) before dispatch, and none of them accept
--model, so they never hit the broken override branch.
Adds test/providers-test-model-base-url.test.ts: drives runProviders('test',
...) end-to-end against a mocked fetch + temp GBRAIN_HOME/config.json with
provider_base_urls set for the dashscope recipe (the exact recipe named in
the bug report), asserting the outbound request hits the configured base
URL rather than the recipe default. Verified red on pre-fix code via
git stash, green after.
Closes#2863
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix: drop duplicate buildGatewayConfig import after master merge
Master's f3e78fd2 added the same import the PR carried; the textual
merge was clean but the result failed typecheck (TS2300 duplicate
identifier).
Co-authored-by: masashiono0611 <masashi.ono.0611@gmail.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: masashiono0611 <masashi.ono.0611@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Garry Tan <garrytan@gmail.com>
The file's final test configures the gateway with a remote provider and a
fake key, and its afterEach only clears the mock transport. With no
afterAll, the poisoned global config survives the file boundary; the next
test file in the shard that triggers an embed makes a real HTTP call and
fails. Surfaced on master when #3022's new test file reshuffled shard
composition (shard 6: synthesize-concepts-progress failed twice with a
live Google embed rejection). The legacy-embedding preload can't catch
this: it only re-applies defaults when the gateway slot is empty.
One-line root-cause fix at the leaker. A repo-wide guard for the class
(~70 files call configureGateway without a final reset) is filed as a
follow-up.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Adds NVIDIA NIM / API Catalog as a first-class OpenAI-compatible AI recipe:
- chat via nvidia/nemotron-3-super-120b-a12b (conservative capability
claims: no tools, no subagent loop until proven)
- hosted embedding models incl. nvidia/llama-nemotron-embed-1b-v2 with
Matryoshka-style dimension overrides (1024/1280/1536/2048) and fixed
natural dims for the other catalog models
- asymmetric input_type mapping (document -> passage, query -> query) via a
gateway compat fetch shim, since the generic openai-compatible recipe
cannot infer that provider-specific requirement
- base URL https://integrate.api.nvidia.com/v1 verified live (OpenAI-shaped
/v1/models, all five recipe model ids present in the catalog)
Changed from the original PR: dropped the recipe's custom resolveAuth — it
duplicated defaultResolveAuth's Authorization-Bearer behavior exactly and
violated the IRON RULE that only Azure overrides resolveAuth
(test/ai/recipes-existing-regression.test.ts). NVIDIA_API_KEY now flows
through defaultResolveAuth via auth_env.required, and the recipe test pins
resolveAuth === undefined + the default Bearer resolution + the missing-key
AIConfigError. Also scrubbed a private downstream-agent name from ported
comments per the repo privacy rule.
Takeover of #2965 by @ravehorn.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: SAGE Codex <codex@sage.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
getLastSeen had no upper date bound, so a future-dated chronicle event (a
scheduled calendar-event, a planned milestone) became the entity's "last
seen" date; finalizeLastSeen's Math.max(0, ...) then clamped the negative
delta, reporting days_ago: 0 -- the entity reads as seen-today. Recording
future events is intended (eligibility ELIGIBLE_TYPES includes calendar-event);
the reader just needs to stop counting them as "seen".
Bound the query to te.date <= COALESCE(asof, current_date) in both engines,
mirroring getOnThisDay's existing te.date < target bound. asof now reaches
the WHERE clause (previously it only reached finalizeLastSeen), so as-of
time-travel is honored for the date filter too.
Regression test added: an entity with past events plus a future event -> last
seen returns the most recent PAST event, not the future one; and as-of after
the future date lets it through. Fails before, passes after.
putPage's INSERT used COALESCE(<chunkerVersion>, 1), so callers that don't
supply chunker_version (no MCP/subagent caller does — it's internal metadata)
landed new pages at version 1. Dream subagents write through putPage directly,
so their pages got v1 and doctor's contextual_retrieval_coverage check flagged
them as "older chunker_version" forever, even though they were chunked and
embedded with the current chunker.
Default the INSERT to MARKDOWN_CHUNKER_VERSION on both engines. The ON CONFLICT
UPDATE still COALESCE-preserves an explicitly supplied version. Add an
engine-level regression test.
reconcile-links is advertised in `gbrain --help` and implemented with a
`case 'reconcile-links'` block in handleCliOnly, but it was missing from the
CLI_ONLY Set. Dispatch only enters handleCliOnly when the command is in
CLI_ONLY, so every invocation fell through to the shared-operations lookup and
hit the generic "Unknown command" branch — leaving the documented doc↔impl
edge-rebuild tool silently unreachable via the CLI.
Add 'reconcile-links' to CLI_ONLY, plus a reachability regression test
mirroring the #2035 (`calibration`) guard.
The content-sanity gate's audit logger (logContentSanityAssessment)
defaults, via audit-writer.ts::resolveAuditDir(), to writing
~/.gbrain/audit/content-sanity-YYYY-Www.jsonl on disk. A GBRAIN_AUDIT_DIR
env override exists, but nothing in the shared test bootstrap ever set
it, so any test that exercised an audit-emitting code path without
wrapping the call in its own withEnv() fell through to the operator's
real audit trail. test/import-file.test.ts's oversize-boundary fixture
('borderline-slug', content just under MAX_FILE_SIZE but over
DEFAULT_BYTES_BLOCK) fired a real soft_block event into the developer's
live ~/.gbrain/audit on every run — which doctor's
content_sanity_audit_recent check then reported as production signal.
Fix: add a bootstrap preload (test/helpers/audit-dir-preload.ts, wired
via bunfig.toml) that points GBRAIN_AUDIT_DIR at a fresh per-process
mkdtemp dir before any test file loads. Each run-unit-shard.sh shard is
its own bun process, so each shard gets its own scratch dir with no
cross-shard collision. This closes the leak for every audit-emitting
test, not just this fixture. It respects a developer-exported override
(only sets the var when unset), and files that manage their own
per-test GBRAIN_AUDIT_DIR via withEnv() are unaffected.
Also fix a latent isolation bug this surfaced: gbrain-home-isolation.test.ts
unconditionally deleted GBRAIN_AUDIT_DIR in a finally block instead of
restoring the prior value, which clobbered the bootstrap's scratch dir
for every test file that ran after it in the same shard process.
Adds test/audit/audit-dir-preload.test.ts to pin the behavior: it
reproduces the exact soft_block event shape and asserts it lands in the
scratch dir, never in ~/.gbrain/audit.
Reported by @paul-0320.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
📝 Summary:
• launchd rejects group/world-writable agent plists — when the installer
runs under a umask-0 parent shell, `writeFileSync(plistPath(), plist)`
produces a 0666 plist that makes `launchctl load`/`bootstrap` fail with
the opaque `Bootstrap failed: 5: Input/output error` and the login-time
LaunchAgents scan skip the file silently
• on an affected machine the daemon never registers while everything
looks installed — the plist exists, launchd's disabled-table says
enabled, and no log file is ever created
🔧 Technical Improvements:
• `installLaunchd`: write plist with `{ mode: 0o644 }` AND
`chmodSync(0o644)` — writeFileSync mode applies only on create, so a
reinstall over an existing 0666 plist must normalize explicitly
• `installSystemd`: same hardening on the unit file (systemd warns on
world-writable units); symmetric with the launchd path
• Restart-policy rewrite path (`generateSystemdUnit` rewrite of an
existing unit): chmod is load-bearing here — the file always exists,
so the write mode never applies
• `chmodSync` added to the fs import
📊 Code Changes: 18 insertions, 4 deletions (net +14)
📦 Files Modified:
• src/commands/autopilot.ts (minor updates) — mode + chmod on the three
supervisor-file writers; comments carry the launchd failure signature
so the next EIO hunt greps straight to it
embedStaleForSource rebuilds each page's chunks as a merged ChunkInput[]
carrying only five fields (chunk_index, chunk_text, chunk_source,
embedding, token_count), while upsertChunks writes the metadata columns
as EXCLUDED.<col>. Any page containing at least one stale chunk
therefore has ALL its chunks' metadata reset on the next embed-stale
pass:
- image chunks flip modality 'image' -> 'text' and disappear from the
cross-modal image search arm permanently (it filters
modality='image'), while keeping their embedding_image vector — the
data looks intact but is unreachable;
- code chunks lose language, symbol_name, symbol_type,
symbol_name_qualified.
The read side compounds this: rowToChunk never returned modality, so a
correct merge was impossible without also extending the Chunk shape.
Fix: expose modality on Chunk/rowToChunk and carry modality, language,
and the symbol fields through the merge. embedding_image is deliberately
not carried — upsertChunks already COALESCEs it server-side.
Repair for affected brains: UPDATE content_chunks SET modality='image'
WHERE chunk_source='image_asset' AND embedding_image IS NOT NULL.
The new test seeds a mixed page (settled image chunk + stale text chunk
with symbol metadata) and asserts both survive an embedStaleForSource
pass; it fails on master.
Co-authored-by: Forge (Ron) <forge@zsimovan.dev>
The image_assets check statSyncs files.storage_path directly, but
sync-ingested assets store repo-relative paths. Run doctor from any
directory other than the brain repo and every image is reported
'missing from disk' — a persistent false WARN with a suggested fix
(gbrain sync --skip-failed) that does nothing.
Resolve relative paths against sync.repo_path before statting; absolute
paths are untouched. Falls back to cwd when the config key is unset,
preserving the old behavior for brains without a configured repo.
Co-authored-by: Forge (Ron) <forge@zsimovan.dev>
The submission-time backpressure cap counts all waiting (name, queue)
rows regardless of which source a job targets. On a multi-source brain
this makes per-source submissions with maxWaiting: 1 mutually exclusive:
while one source's sync sits waiting, every other source's freshness
sync coalesces into that row and never runs. The dispatch log shows the
starved source 'dispatched' each interval (queue.add returns the other
source's waiting row), so the starvation is invisible unless you notice
sources.last_sync_at falling behind — we found a secondary source 29
hours stale on a 5-minute freshness interval.
Fix: when the submitted data carries a string sourceId, key the advisory
lock, the waiting count, and the coalesce target on it. Submissions
without sourceId keep the existing single-scope behavior, so
single-source brains and non-sync jobs are unchanged.
The new test asserts same-source submissions still coalesce while a
different source gets its own row and its own cap; it fails on master.
Co-authored-by: Forge (Ron) <forge@zsimovan.dev>
* fix(sources): validate --path is a git repo at registration time (#2707)
`sources add --path <dir>` accepted any existing non-git directory with
zero validation, deferring the failure to the first `gbrain sync` ("Not
inside a git repository: ..."). By the time that surfaces, the source
has been silently stale for however long nobody read the sync logs.
Add a registration-time check (git-remote.ts:isInsideGitRepo, mirroring
sync.ts's discoverGitRoot walk-up so subdir-of-git-repo sources still
pass) that rejects an existing-but-non-git --path directory with an
actionable error pointing at `git init && git add -A && git commit`.
Non-existent paths are unaffected (out of scope — different, pre-existing
failure mode) and `--force` opts out for callers who want to register
before git-init exists.
This is registration-time validation ONLY — it never auto-`git init`s
the directory, preserving the consent boundary #2967 established for
sync-time self-heal (a --path source is the user's own external
directory; gbrain must not mutate it without explicit ask).
Also documents the git requirement (docs/guides/multi-source-brains.md),
including the "files must be committed, not just present" gotcha and
that a stale/unreachable sync anchor already self-heals on plain
`gbrain sync` (verified manually against HEAD — no reset-anchor command
needed).
* fix(sources): require a committed HEAD + shell-quote remediation cmd (codex round 1)
Codex review round 1 on #2707 found two real gaps:
1. isInsideGitRepo alone accepts a `git init`ed-but-never-committed
directory (rev-parse --show-toplevel succeeds with no HEAD), so
registration would still pass a source that fails sync's own
"No commits in repo ... Make at least one commit before syncing."
Add hasGitCommits (git rev-parse HEAD) as a second required check.
2. The remediation command in the error message interpolated the raw
path unquoted — spaces, $(), backticks, etc. would break or, worse,
execute unintended shell syntax if pasted. POSIX single-quote it
(mirrors src/commands/connect.ts:shellQuote; duplicated locally
rather than imported, since commands/ depends on core/ not the
reverse).
* fix(sources): require tracked content in HEAD, not just a resolvable HEAD (codex round 2)
Codex review round 2 P1: hasGitCommits (rev-parse HEAD) accepted a repo
with an empty commit (git commit --allow-empty) followed by untracked
files — HEAD resolves fine (to git's well-known empty-tree object), so
registration passed, but the first sync would "succeed" importing
nothing and then silently never notice the untracked files change. The
exact same gap applied to an untracked subdirectory of an otherwise-
real git repo (monorepo case).
Replace hasGitCommits with hasTrackedContent (`git ls-tree HEAD -- .`,
non-recursive — one entry is enough, no need to walk the whole
subtree). `-C path` + pathspec `.` scopes correctly to both a repo
toplevel and a subdirectory-of-a-repo source, and an empty tree lists
zero entries where a bare `rev-parse HEAD` would still succeed. Also
subsumes the "no commits at all" case hasGitCommits covered (ls-tree
on an unborn repo fails the same way), so this is one check instead
of two.
Updated the error copy and docs/guides/multi-source-brains.md to match
what's actually verified now.
* fix(sources): O(1)-output tree-emptiness probe, avoid maxBuffer overflow (codex round 3)
Codex review round 3 found the round-2 `git ls-tree HEAD -- .` listing
buffers the whole (non-recursive) tree — a real repo with ~17-20K
directly-tracked entries exceeds execFileSync's default 1 MiB
maxBuffer, throws ENOBUFS, and the catch-all incorrectly rejects a
perfectly valid registration.
Replace the listing with `git rev-parse --verify HEAD:./` (resolves
the tree object for `path` specifically, correct for both toplevel and
subdirectory sources same as before) compared against git's canonical
empty-tree SHA-1 (4b825dc6...) — a fixed ~40-byte read regardless of
how many entries the tree has, structurally immune to this class of
bug rather than just raising the threshold. Added a 300-file
regression test locking this in.
Declined a second round-3 finding (P1: reject a tree if ANY untracked
file exists anywhere under the path, not just when the tree is
entirely empty) — untracked files never being synced is standard,
existing git-source behavior throughout this codebase (identical for
--url managed clones), not a bug specific to this validation. Enforcing
zero-untracked-files at registration would reject ordinary repos with
gitignored build output, .DS_Store, editor swapfiles, etc. Out of
scope relative to what #2707 actually asks for (a directory with real,
committed content that will sync) and how every other git source in
this system already behaves.
* fix(sources): derive empty-tree OID per repo instead of hardcoding SHA-1 (codex round 4)
Codex review round 4 P2, confirmed by directly testing against a
`git init --object-format=sha256` repo: the hardcoded SHA-1 empty-tree
constant only matches SHA-1 repositories. An empty SHA-256 repo's real
empty-tree OID is a different (64-char) hash, so the SHA-1 comparison
silently mismatched and let an empty/untracked SHA-256 source through
— exactly the case this validation exists to catch.
Replace the constant with `emptyTreeOid()`: `git hash-object -t tree
--stdin < /dev/null` computed in the target repo's own context, so it
returns the correct empty-tree OID for whichever object format that
repo actually uses, without gbrain needing to know or care which one.
Added gated regression tests (git 2.29+ / --object-format=sha256,
test.skipIf on older git) for both the empty-repo-rejected and
real-content-registers-fine cases.
Converging here (4 review rounds; this is the last outstanding
finding from round 4, and round 4 raised only this one issue).
* fix(test): --force the incidental non-git second source in #1434 routing test (#2707)
CI caught a real regression from this PR's registration-time git
validation: test/sync-sole-non-default-routing.test.ts's "2+ non-default
sources" case registers a bare mkdtempSync temp dir (no git init) as a
second source purely to have 2 sources present — the directory's content
was never meant to be exercised, only its existence as a distinct
local_path. #2707's new validation correctly rejects that dir at
registration time, since nothing else in the test suite told it
otherwise.
--force is the right fix, not adding unnecessary git-init/commit
boilerplate to secondRepo: it documents that this specific registration
intentionally doesn't care about git-validity, matching what a real
caller opting into the legacy lenient behavior would do.
Verified: the specific test (3/3 pass), plus every other test file in
the repo using `sources add --path` (sources.test.ts, sources-ops.test.ts
already covered by the PR's own commits; repos-alias.test.ts,
sync-cost-gate.serial.test.ts — 11/11 pass, no similar fixture gap).
typecheck clean, verify 31/31.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Same class as #3019's v123 fix; the guard test added there flagged this
within one push. Route the notice through process.stderr.write.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
A single markdown page whose compiled_truth exceeds Postgres's hard
1,048,575-byte tsvector cap made update_page_search_vector() throw
"string is too long for tsvector" INSIDE the pages UPSERT transaction.
Not a per-file ledger entry — a transaction abort. The whole source's
sync checkpoint stayed pinned (Sync BLOCKED) until the oversized file was
fixed or manually skipped, even though every other file in the run
imported fine. --retry-failed re-failed the same files every run; the
3-consecutive-failure auto-skip eventually moved past them, but for a
scheduled collector that meant hours of blocked cycles per oversized
file, per source.
Root cause: pages.search_vector indexed compiled_truth — the unbounded
whole-page body — even though it's write-only dead weight for actual
search. searchKeyword() (postgres-engine.ts / pglite-engine.ts) ranks and
queries content_chunks.search_vector exclusively (Cathedral II Layer 3,
chunk-grain, already populated separately from compiled_truth via
chunking at import time, and well under the tsvector cap since
chunkText() targets embedding-sized pieces). Verified directly: no
pages.search_vector / bare search_vector read appears anywhere outside
this trigger's own definition and the reindex/backfill machinery that
maintains it.
Fix: v124 migration recreates update_page_search_vector() without
compiled_truth — title + timeline stay (both naturally small), so the
column keeps carrying some signal rather than going fully inert. Updated
in lockstep (documented contract, see reindex-search-vector.ts's own
comment): migrate.ts's new v124, reindex-search-vector.ts's
recreatePagesFn, and the fresh-install baselines in pglite-schema.ts +
src/schema.sql (regenerates schema-embedded.ts via `bun run
build:schema`). No backfill: existing rows keep whatever search_vector
they already computed until their next UPDATE — harmless, since nothing
reads this column, and the brains that actually hit this bug never
successfully wrote a value for the oversized page in the first place.
Considered (from the issue) and rejected: truncating compiled_truth to
fit under the cap. Silent, position-dependent recall loss, and the byte
cap doesn't line up cleanly with any natural character/token boundary
for UTF-8 content. content_chunks.search_vector already gives full,
untruncated chunk-grain coverage for large pages — truncating a
now-redundant whole-page vector would trade a real bug for a subtler one.
## Test plan
- New test/page-search-vector-overflow.test.ts: a >1MB page (genuinely
diverse tokens — a repetitive lorem-ipsum-style fixture does NOT
reproduce this bug, since to_tsvector's cap is on its DEDUPLICATED
output size, not raw input length) now imports successfully instead of
throwing; remains keyword-searchable via the chunk-grain path; a normal
page's search_vector still carries title signal (not fully inert).
Verified the test is meaningful both directions: fails with the exact
reported error on the pre-fix trigger (git-stashed the fix, reran,
confirmed byte-for-byte match: "string is too long for tsvector
(2684620 bytes, max 1048575 bytes)"), passes with the fix restored.
- Updated fts-language-migration.serial.test.ts: removed an assertion
that configurable_fts_language (v123) is LATEST_VERSION — that was only
ever true until the next migration landed; the codebase's own pattern
elsewhere for this (migrate.test.ts) uses toBeGreaterThanOrEqual, not
exact-match, for exactly this reason.
- bun run typecheck clean, bun run verify 31/31 green.
- test/page-search-vector-overflow.test.ts (3/3),
test/reindex-search-vector.serial.test.ts,
test/fts-language-migration.serial.test.ts, test/migration-v120.test.ts,
test/sync.test.ts, test/bootstrap.test.ts, test/migrate.test.ts — 254
total, 0 fail, no regressions.
## Design consultation
Investigated jointly with masa-codex (async design review) before
implementing — their read of the codebase (content_chunks.search_vector
already covers keyword search; pages.search_vector's compiled_truth feed
is the only overflow-prone, effectively-dead write) matched independent
verification and shaped the "remove from trigger" fix over the issue's
alternative options (chunk-grain rebuild — largely already exists; input
truncation — rejected above; ledger-entry-only — insufficient alone,
since the page upsert failing in the same transaction also loses the
chunk write, so checkpoint advancing without this fix would make the
content permanently unsearchable, not just delayed).
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
resolveGbrainCliPath() (both copies — src/commands/autopilot.ts and the
inlined duplicate in src/core/brain-repo-durability.ts) called `execSync`
without an explicit `env`, relying on default inheritance. Under Bun,
execSync/execFileSync snapshot process.env at BUN'S OWN STARTUP, not at
call time — a runtime PATH mutation (dotenv/config loading, wrapper-script
env sourcing, etc.) happening after Bun boots but before this call is
invisible to `which gbrain` unless the current env is forwarded
explicitly.
This is a known, already-precedented Bun quirk in this exact codebase:
spawn-helpers.ts's detectTini() was already fixed for the identical
symptom with the identical one-line fix (`env: process.env`), with a
comment explaining the mechanism — this call site was simply missed.
Matches the reported symptom precisely: "which gbrain" resolves fine when
run standalone (a fresh Bun process, no prior env mutation to hide), but
throws specifically from inside autopilot's managed-worker spawn path
(src/commands/autopilot.ts:416, guarded by `spawnManagedWorker` — Postgres
engine + minion_mode enabled), which fires after config/dotenv loading has
already run in that process. Impact per the report: this silently
degrades to no worker ever picking up queued jobs (including embed jobs),
with `gbrain doctor` only showing a growing "N stale chunks" warning that
reads like an ordinary backlog rather than a broken worker.
Also improved the throw-path error message to include the actual
PATH/execPath/argv[1] values observed at failure time, so a future report
doesn't require guessing at what the process actually saw.
Not fixed here (documented as a separate, smaller finding): a third call
site with the identical missing-env pattern exists in
src/core/claw-test/runners/openclaw.ts ('which openclaw'). Left out of
scope for this PR, which is specifically about #2747's reported symptom;
worth a small follow-up.
Verification: bun run typecheck clean, bun run verify 31/31 green,
test/autopilot-resolve-cli.test.ts 4/4 pass (existing coverage, no
regressions — a genuinely-simulated Bun-env-snapshot race isn't
reproducible in a same-process unit test, and this codebase's own
convention explicitly avoids mock.module for child_process per
doctor-orphan-ratio.test.ts's stated test-isolation rule, so this PR
relies on the fix's precedent-match + existing coverage rather than a new
mock-based regression test).
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* fix(jobs): retry resets started_at/attempts_made/attempts_started (#2783)
`gbrain jobs retry` re-queued a dead job by resetting status/error_text/
locks/delay/finished_at, but left started_at, attempts_made, and
attempts_started untouched. On re-claim, claim()'s
`started_at = COALESCE(started_at, now())` preserved the ORIGINAL
first-claim timestamp instead of re-stamping it. handleWallClockTimeouts()
anchors on `now() - started_at`: a retry issued more than timeout_ms * 2
after the original claim was immediately dead-lettered again in under a
second, with attempts_made already past max_attempts — making retry
useless for exactly the case it exists for (recovering work after an
outage that outlasted the job's timeout).
An explicit `jobs retry` is an operator asserting "run this fresh", so
retryJob now also clears started_at (NULL, re-stamped on next claim) and
resets attempts_made/attempts_started to 0.
Two new tests: direct assertion that retry resets all three columns, and
a full repro of the reported bug (wall-clock-killed job retried long
after the original claim now survives re-claim instead of being
immediately re-killed).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(jobs): also reset stalled_counter on retry (#2783)
Codex review round 1 found the fix was incomplete: a job dead-lettered by
stall exhaustion (handleStalled() at stalled_counter + 1 >= max_stalled)
retained its exhausted stalled_counter across retry. The retried job's
very first lock expiry after re-claim would immediately re-satisfy the
dead-letter threshold, contradicting the same "run this fresh" intent the
started_at/attempts reset already established.
New test mirrors the existing wall-clock repro: exhaust the stall budget
via two real handleStalled() calls, retry, confirm one more stall now
requeues instead of dead-lettering again.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
resolveTakesSourceId() caught every error from resolveSourceId() and fell
back to undefined, which restores the pre-#2698 unscoped (cross-source)
slug lookup for takes add/update/supersede/resolve. resolveSourceId()
only ever throws when a source was explicitly in play (an invalid or
unregistered GBRAIN_SOURCE, a .gbrain-source dotfile pointing at a
source that doesn't exist, or a genuine DB error) — it never throws for
"nothing configured," which resolves cleanly to the seeded 'default'
source. So swallowing the error had no legitimate case to protect and
only reintroduced the cross-source write bug on any resolution failure.
Let it propagate so the write is blocked instead.
Adds regression coverage for both the unchanged happy path (no source
configured resolves cleanly) and the newly fail-closed path (an
unregistered GBRAIN_SOURCE blocks the write instead of falling back to
an unscoped lookup).
extractTakesFromPages hardcoded anthropic:claude-haiku-4-5 as the classifier
model. On an OAuth/local-only install (no ANTHROPIC_API_KEY; chat routed
through a gateway model) every takes extraction died with llm_unavailable —
the takes layer silently never populated and the takes_count health check
stayed red despite a working configured chat_model.
Resolution is now `opts.model || getChatModel()` — the same file-plane
gateway-config idiom enrich.ts uses — NOT engine.getConfig('chat_model')
(the DB config plane), keeping model routing on the single config plane the
rest of the codebase reads. Explicit opts.model still wins; unconfigured
installs fall through to the gateway's DEFAULT_CHAT_MODEL.
Adds a regression test that pins the file-plane read: a conflicting DB-plane
config.chat_model row is ignored, the gateway-configured chat_model is used
when opts.model is unset, and explicit opts.model wins. Verified the
file-plane test fails against the pre-fix code.
Takeover of #2997 by @Nazim22 with the model read moved from the DB config
plane to the file-plane gateway idiom.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Nazz <nazim.mj@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Bun SQL double-encodes ::jsonb[] array binds on the Postgres engine: every
edge_metadata element landed as a jsonb string scalar instead of an object,
so the resolver's `edge_metadata || jsonb_build_object(...)` UPDATE produced
a jsonb array and resolved_chunk_id was never readable — code_callers /
code_callees / code_blast returned nothing on Postgres-engine brains while
the resolver logged edges_resolved > 0. PGLite was unaffected (per-row
placeholders already).
Rewrites both inserts (code_edges_chunk, code_edges_symbol) to per-row
$n::text::jsonb placeholders via sql.unsafe — the same shape executeRawJsonb
and the PGLite engine use.
Adds the DATABASE_URL-gated Postgres regression test this class requires
(PGLite cannot reproduce it): asserts jsonb_typeof(edge_metadata) = 'object'
for resolved + unresolved inserts and that the resolver-style || UPDATE
keeps object shape. Verified the test fails 3/3 against the pre-fix code
and passes with the fix.
Takeover of #2968 by @zsimovanforgeops with the missing regression test added.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Forge (Ron) <forge@zsimovan.dev>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The top-level `gbrain --help` advertised `sync --install-cron` since the
line was first added, but `src/commands/sync.ts` never parsed or handled
the flag — `gbrain sync --install-cron` silently ran an ordinary one-off
sync instead of installing anything, manufacturing false confidence in
the exact durability layer operators reach for it to secure.
git blame shows the line was introduced once (v0.42.29.0 help-text
scaffold) and never touched again — no design intent to recover.
Implementing it would also compete with autopilot, which already owns
this job: `gbrain autopilot --install` runs a self-maintaining daemon
(sync+extract+embed) on a schedule, including a per-source freshness
check that submits `sync` jobs on its own interval. A second, separate
sync-only cron would be a competing scheduler outside the D10
cycle-lock invariant that already keeps autopilot's own targeted-submit
and full-cycle paths from double-processing.
Removed the misleading line and pointed sync's --watch entry at
`autopilot --install`, mirroring the existing `dream` command's
"See also: autopilot --install (continuous daemon)." pattern one
section below. Added regression coverage to
test/cli-help-discoverability.test.ts asserting the help text no
longer promises install-cron and does point at autopilot.
gateway.chat() requested cacheSystem:true but never got a system-prompt
cache hit on single-turn callers (page-summary, skillopt, enrich): the
call-level providerOptions.anthropic.cacheControl is real (it becomes
Anthropic's documented top-level "auto-cache the last cacheable block"
shorthand via @ai-sdk/anthropic 3.0.47+), but for a stable system prompt
paired with a different user message every call, "the last cacheable
block" is that ever-varying tail -- every call writes a fresh cache
entry there and never reads a prior one.
Fix: pass system as a SystemModelMessage object (ai's documented shape
for attaching provider options to the system block) carrying its own
providerOptions.anthropic.cacheControl when cacheSystem is requested,
and mirror the same marker onto the last tool def (Anthropic caches
everything up to and including the last cache_control block it sees).
The call-level marker is kept, not removed -- it still gives toolLoop()'s
growing multi-turn conversation a rolling cache breakpoint on each
turn's tail. All three markers now derive from one canonical
cacheControlValue computed after provider_chat_options config merging,
so a configured TTL override (e.g. ttl: '1h') applies consistently
instead of only reaching the call-level marker.
Verified red-before-fix by stashing the gateway.ts diff and confirming
the new assertions fail on unfixed code, then restoring.
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* fix(sync): self-heal a never-git-initialized default brain dir (#2964)
The dream cycle's sync phase throws unconditionally on a legacy
sync.repo_path-anchored default brain dir that was never git init-ed
(predates git-backed sync, or was rsync'd without its .git), failing
every nightly run with no recovery. doctor's sync_freshness/
sync_consolidation checks report "ok" for this exact brain, but only
because they query the sources table (0 rows for a legacy default
brain) — a coincidental false-negative, not a real diagnosis.
Self-heal by git-initializing the dir and capturing the current
on-disk state as the sync baseline, scoped to !opts.sourceId only —
gbrain owns this directory outright, unlike a registered local source
(sources add --path, no --url) which is the user's own external
directory and should keep failing loudly.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): dry-run no-write contract, unborn-HEAD recovery, no-gpg-sign (#2964)
Codex review on the initial self-heal patch (b1671ee) found 3 real gaps:
- P1: the self-heal ran even under --dry-run, mutating the filesystem
during what's documented as a preview-only command. Gated the whole
self-heal (both discoverGitRoot and the headCommit read) on
!opts.dryRun, same as the existing !opts.sourceId ownership check.
- P2: if `git init` succeeded but the process died before the baseline
commit landed, the next run's discoverGitRoot would succeed (`.git`
exists) and skip recovery entirely, permanently wedging on "No
commits in repo" forever. Added the same self-heal at the
`git rev-parse HEAD` catch site, sharing a new createSyncBaselineCommit
helper with the discoverGitRoot catch.
- P2: the baseline commit inherited the operator's global
commit.gpgSign, which can block headless cron/launchd runs on an
unavailable signing agent/pinentry. Added --no-gpg-sign.
Two new tests cover dry-run no-mutation and unborn-HEAD recovery.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): restrict git auto-init self-heal to the anchor-resolved path only (#2964)
Second Codex review round (a10eeab) found the ownership check still too
loose:
- P1 (security): !opts.sourceId alone isn't proof gbrain owns repoPath.
jobs.ts's `sync` job handler leaves sourceId undefined whenever
job.data.repoPath doesn't match a registered source's local_path, so
an admin-scope submit_job({name:'sync', data:{repoPath}}) MCP call
could point the self-heal at an arbitrary directory and have it
silently git-init + commit + ingest it. Gated both self-heal sites on
!opts.repoPath too — only the path resolved from gbrain's own
sync.repo_path anchor (never a caller-supplied one) is eligible.
- P2: the unborn-HEAD recovery site calls discoverGitRoot, which walks
UP from repoPath and can resolve to an ANCESTOR repo for a
--src-subpath/subdir-as-repoPath sync with an unborn HEAD. Committing
there would `git add -A` sibling files well outside the sync scope.
Added a check that gitContextRoot === realpathSync(repoPath) before
self-healing; refuses (falls through to the original error) otherwise.
Tests rewritten to exercise the true self-heal-eligible path (anchor
config via engine.setConfig('sync.repo_path', dir), no repoPath/sourceId
passed) instead of an explicit repoPath, plus a new test asserting a
caller-supplied repoPath with no sourceId still throws.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): prove self-heal ownership by anchor VALUE, not field presence (#2964)
Third Codex review round (613ad5b) found the previous round's fix broke
the very call site it was meant to repair, plus a second scope gap:
- P1 (critical): !opts.repoPath rejected self-heal on the REAL production
callers too. runPhaseSync (dream cycle's sync phase, cycle.ts) always
passes `repoPath: brainDir` explicitly after resolving it upstream, and
the CLI's bare `gbrain sync` resolves sourceId='default'. Both made the
ownership gate rethrow, leaving `gbrain dream` and `gbrain sync`
wedged on the exact non-git legacy brain this fix targets — only
synthetic callers that omitted both fields ever healed.
Fixed by proving ownership by VALUE instead of by field absence: a new
isAnchorOwnedSyncPath() re-reads gbrain's own persisted
sync.repo_path config and requires the resolved repoPath to equal it
exactly, regardless of whether the caller passed it explicitly or let
it default. An attacker-supplied arbitrary path (e.g. via
submit_job({name:'sync', data:{repoPath}})) only self-heals if it
happens to already equal gbrain's own anchor — which is the
legitimate case, not an escalation. opts.sourceId and opts.srcSubpath
still disqualify unconditionally (registered/subpath-scoped syncs are
a different ownership context).
- P2: a --src-subpath sync with an unborn parent-repo HEAD would commit
the whole ancestor root, capturing sibling files outside the scope.
isAnchorOwnedSyncPath's opts.srcSubpath check closes this; the
existing gitContextRoot === repoPath check stays as defense in depth.
- P2: manageGitignore's "warn and return" contract (a deliberate side-
effect that must never kill the sync job for its OTHER callers) meant
a broken gbrain.yml or unwritable .gitignore would silently let the
baseline `git add -A` commit db_only content. createSyncBaselineCommit
now recomputes db_only exclusion directly from loadStorageConfig and
passes it to `git add` as pathspecs, independent of the .gitignore
write's success — true fail-closed. (A redundant pathspec exclude for
a path .gitignore ALREADY covers makes git's -A bail with "paths
ignored, use -f" even though the negation is correct, so each dir is
check-ignore'd first and only pathspec-excluded when NOT already
covered.)
Tests rewritten around the anchor-VALUE model: the critical regression
case (explicit repoPath matching the anchor still heals — the exact
scenario Codex proved was broken) plus a true negative (a caller-supplied
path that does NOT match the anchor still throws), --src-subpath refusal,
and db_only fail-closed exclusion.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): allow default-source ownership, realpath compare, generous timeout, --no-verify (#2964)
Fourth Codex review round (9c1d461) found the previous round's ownership
gate still didn't match the REAL installed-brain shape, plus 3 more gaps:
- P1 (critical): rejecting all non-empty opts.sourceId meant self-heal
still never fired on a real brain. Migration sources_table_additive
seeds a 'default' source row whose local_path mirrors sync.repo_path
on every brain that's run it (virtually all of them), so
resolveSourceForDir (dream cycle) and bare `gbrain sync` both resolve
sourceId:'default' in practice, never undefined. isAnchorOwnedSyncPath
now permits sourceId undefined OR exactly 'default' (gbrain's own
bootstrap identity, never something a caller names) and proves
ownership by rereading the LIVE anchor for that same identity
(sources.default.local_path vs config.sync.repo_path).
- P2: compared raw anchor/repoPath strings, so a cosmetic difference
(trailing slash, ..) between the stored anchor and dream.ts's
path.resolve()-normalized brainDir would defeat the match. Now
realpath-compares both sides (fail-closed on ENOENT/dangling).
- P2: the shared git() helper's 30s timeout could abort the baseline
`git add -A` on a large legacy brain mid-way, after `git init` already
created `.git` — leaving an unborn repo every subsequent sync would
retry and time out identically forever. Added an optional timeoutMs
param (default unchanged at 30s); the baseline add call uses 10min.
- P2: the baseline commit could trigger an operator's global
core.hooksPath/init.templateDir hooks (pre-commit/commit-msg),
breaking headless recovery if those hooks need project tooling or
prompt. Added --no-verify.
Tests: rewrote the mis-scoped "registered source" test (it used
sourceId:'default', which is now correctly permitted) into two — a new
regression test proving sourceId='default' + matching local_path heals
(the actual production shape), and a corrected non-default-sourceId
refusal test.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): defer .gitignore write past first import, rebuild index, fail closed on unparseable db_only (#2964)
Fifth Codex review round (c17cd23) found the baseline-commit helper
interacting badly with the pre-existing db_only storage-tiering feature:
- P1 (data loss): createSyncBaselineCommit called manageGitignore BEFORE
performFullSync's collectSyncableFiles ran. collectSyncableFiles
enumerates via `git ls-files --cached --others --exclude-standard`, so
writing db_only entries into .gitignore first would silently exclude
those pages from the DATABASE, not just from git — on a brain's very
first sync. This is the exact bug class runSync's existing "manage
.gitignore ONLY on successful sync" ordering (this file, ~line 4540,
itself a prior Codex P1 fix, comment literally says so) was written to
prevent — my new code reintroduced it in a different spot. Fix: stopped
calling manageGitignore inside the self-heal at all. db_only exclusion
for the COMMIT still happens via the existing pathspec computation
(independent of .gitignore); .gitignore itself gets written by the
already-existing post-sync flow once this sync completes, same as any
other sync.
- P1 (data leak): the unborn-HEAD recovery site can reach
createSyncBaselineCommit with a repo whose INDEX already has entries
staged from some prior operation (manual `git add`, interrupted
workflow) before gbrain's self-heal ever touched it. `git add -A`
only adds/updates — it doesn't drop an already-staged path our
exclusion pathspecs now want excluded. Added `git read-tree --empty`
to reset the index before staging (no-op on a freshly-`git init`-ed
repo, whose index is already empty).
- P2: loadStorageConfig warns-and-returns an EMPTY config (not a throw)
for syntactically-valid-but-unsupported YAML (e.g. flow-style
`db_only: [dir/]` — the narrow custom parser only handles block-style
lists), which would silently resolve zero exclusions from a gbrain.yml
that clearly intended some. Added a sniff-test: if gbrain.yml exists
and mentions db_only but nothing resolved from it, refuse the baseline
commit rather than guess "genuinely empty" vs "syntax silently
ignored" (git init may already have run by this point — same "unborn,
retry on next sync" recovery path handles it, and will hit this same
guard again until the user fixes gbrain.yml).
Tests: a positive regression proving db_only markdown IS imported into
the DB on first sync (the actual data-loss scenario), the sniff-test
refusal, and the stale-staged-content-gets-dropped case for the index
rebuild.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): post-heal .gitignore write, supabase_only alias, honor abort signal (#2964)
Sixth Codex review round (e687913), 3 P2s:
- Dream-cycle callers (cycle.ts:runPhaseSync) invoke performSync directly
and never run runSync's CLI-only post-success manageGitignoreAtGitRoot.
A brain self-healed only via the dream cycle would have db_only content
correctly excluded from the baseline commit (createSyncBaselineCommit's
pathspec exclusion) but no .gitignore ever written, leaving the user's
own future manual git add/commit unprotected. Added
performFullSyncAndMaybeGitignore, a thin wrapper around the 3
post-self-heal performFullSync call sites that writes .gitignore
(same success-status gate runSync already uses) only when didSelfHeal
is true — a no-op for the normal path, which still relies on runSync
exactly as before.
- The fail-closed sniff-test only checked the canonical `db_only` key;
the deprecated-but-still-supported `supabase_only` alias (same
keep-out-of-git semantics) could silently bypass it. Now checks both.
- Self-heal didn't check opts.signal?.aborted before starting the
(now up to 10-minute) git init + baseline commit, so a cancelled sync
could still mutate disk and overrun its budget instead of returning
partial. Added the check at both self-heal sites, before any git
operation runs.
New test proves .gitignore gets written after a bare performSync call
(no runSync wrapper) — the actual dream-cycle shape.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): neutralize a leftover .gitignore during the self-heal first sync (#2964)
Seventh Codex review round (27337b0) ran an actual repro and caught the
primary motivating scenario still broken: a brain rsync'd from another
machine without its .git can retain that machine's old auto-managed
.gitignore. collectSyncableFiles (inside performFullSync) enumerates via
`git ls-files --exclude-standard`, so a leftover db_only ignore rule
would silently omit those pages from THIS first sync's DATABASE import —
the same bug class the round-6 ordering fix prevented for a .gitignore
gbrain would have written itself, just triggered by a pre-existing file
this time.
Fix: performFullSyncAndMaybeGitignore now neutralizes any existing
.gitignore for the duration of the one first-sync call — read, delete,
restore byte-for-byte immediately after (even on error) — before
manageGitignore re-merges the managed db_only block onto the restored
original content. This matches exactly what a truly fresh brain with no
.gitignore at all already does on its first sync (nothing to suppress
collection there either); db_only content stays out of the git COMMIT
independently via createSyncBaselineCommit's pathspec exclusion, which
never depended on .gitignore.
Test proves both halves: db_only markdown IS imported despite a leftover
ignore rule, AND the user's own unrelated .gitignore lines (e.g.
.DS_Store) survive the restore intact.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): simplify — drop db_only-import machinery, isolate hooks fully (#2964)
Eighth Codex review round (54c1e6f) found MORE problems with round 7's
.gitignore-neutralization fix (deleting the whole file loses the user's
own unrelated ignore rules; a multi-sync retry scenario could silently
skip a still-broken db_only file while advancing the bookmark) plus 2
more issues in existing code. Rather than patch those too, stepped back
and checked the actual documented semantics of db_only
(docs/storage-tiering.md): it's for "bulk machine-generated content...
written to disk as a local cache" — DB is the source of truth, disk is a
cache populated FROM the DB (`export --restore-only` restores it), never
the other way. Nothing in the docs says `gbrain sync`'s git-diff-based
file collection is how db_only content is supposed to reach the
database — that's ingest-specific tooling's job. Confirmed directly:
`loadStorageConfig` returns the byte-identical `{db_tracked:[],
db_only:[]}` for a malformed flow-style array AND a literal empty
`db_only: []`, so rounds 6-7's "ensure db_only markdown gets imported on
this first sync" chase was solving a problem outside sync's actual scope
in the first place, on an increasingly complex, adversarially-discovered-
edge-case foundation.
Reverted: performFullSyncAndMaybeGitignore (the wrapper + didSelfHeal
tracking + .gitignore neutralize/restore dance + post-success
manageGitignore call). After self-heal, import and any subsequent
.gitignore management now behave EXACTLY like any other brain,
self-healed or not — runSync's existing post-success
manageGitignoreAtGitRoot covers the CLI path identically either way; the
dream cycle not calling it is a separate, pre-existing characteristic of
the dream cycle in general (applies equally to an already-git-initialized
brain going through the same path), not something this fix introduces.
Kept (still correct, self-contained, don't depend on the reverted
machinery): createSyncBaselineCommit's pathspec-based db_only exclusion
for the COMMIT itself (matches the documented "not committed to git"
requirement), the fail-closed sniff-test guard (now documents its
known, structurally-unavoidable false-positive on a genuinely-empty
`db_only: []` — the trade-off is deliberate: low-cost, self-resolving
false positive vs. high-cost, hard-to-undo false negative), the index
rebuild, and the 600s add timeout.
Improved (round 8, P2): hooks isolation. --no-verify only skips
pre-commit/commit-msg; added `-c core.hooksPath=/dev/null` for the
baseline commit, which disables prepare-commit-msg and post-commit too
(the latter runs synchronously inside the same git invocation and could
otherwise hang past the timeout without even being the slow step).
Tests: removed the 3 that exercised the reverted db_only-import
machinery; the remaining 12 (ownership, dry-run, index rebuild, sniff
test, commit-exclusion, unborn-HEAD recovery) are unaffected by the
simplification.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): unconditional db_only exclusion, literal pathspecs, precise sniff test (#2964)
Ninth Codex review round (a6e07f6):
- P1: the check-ignore pre-filter (skip pathspec-excluding a dir already
covered by .gitignore) could be defeated by a pre-existing .gitignore
that ignores a db_only tree with a wildcard but re-includes a child via
negation (e.g. `private-cache/*` + `!private-cache/index.md`) —
check-ignore on the directory still reports "ignored", so the filter
skipped the pathspec exclusion, and `git add -A` staged the re-included
child anyway. Fixed by making exclusion unconditional: every db_only
dir is always pathspec-excluded now, never pre-filtered against
.gitignore state at all — our own pathspec doesn't consult .gitignore,
so no .gitignore content (negated or not) can defeat it. The advisory
"paths ignored... use -f" error this can now trigger when a dir IS also
already .gitignore'd (verified: git still stages everything else
correctly despite the nonzero exit) is caught and swallowed by matching
its exact stderr text; anything else rethrows.
- P2: `:!dir` pathspec shorthand reinterprets a dir name that itself
starts with a pathspec magic character (e.g. `:private/`) instead of
excluding it literally. Switched to `:(exclude,literal)dir`.
- P2: the fail-closed sniff-test's bare substring search on gbrain.yml's
raw content could trip on a comment or unrelated prose mentioning
"db_only" even when there's no real storage section at all, refusing
self-heal forever on an unrelated false positive. Now requires an
actual YAML key line (`db_only:`/`supabase_only:`, trimmed, ignoring
`#` comments) — the round-8-documented "genuinely empty db_only: []"
false positive is unchanged and remains an accepted trade-off (still
structurally indistinguishable from unsupported syntax at the
loadStorageConfig API boundary), but comment/prose mentions no longer
false-positive.
Not fixed (deliberately, documented trade-off — see PR description):
Codex's other P1 this round (refuse baselining when other git refs/
history exist alongside an unborn HEAD) is a narrow, non-destructive
scenario — self-heal only ever acts on the current branch ref when it's
provably commit-less, never touches or deletes any other ref (remote-
tracking, other branches), so at worst it creates a possibly-unexpected
extra commit on an otherwise-empty branch the user hadn't checked out
yet. Chasing it further trades diminishing real-world risk reduction
against unbounded scope growth in what's fundamentally still the
self-heal fix from round 1.
Two new tests: unconditional exclusion despite a matching pre-existing
.gitignore (proves the advisory-swallow path), and a comment-only
gbrain.yml no longer false-positives the sniff test.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* fix(cycle): budget the patterns subagent from remaining job time, not a fixed constant (#2781)
The autopilot-cycle job gets an interval-derived timeout stamped at submit,
but the patterns phase submits its subagent with a fixed 30-min job timeout
and waits up to 35 min — one phase's worst case exceeds ANY interval-derived
budget <= 35 min, so the parent job dead-letters mid-patterns and the tail
phases (consolidate -> schema-suggest) starve for days (#2781, the deeper
half left open by the #2852 dispatch-floor fix).
- MinionJobContext.deadlineAtMs: absolute deadline from the claim-time
timeout_at stamp (the DB ground truth handleTimeouts() sweeps against;
re-stamped on every claim so retries get a fresh budget). Null when the
job has no per-job timeout.
- worker: the per-job abort timer now derives its delay from timeout_at
when present, so the in-process timer, the DB sweeper, and the
handler-visible deadline agree on ONE absolute instant.
- autopilot-cycle + autopilot-global-maintenance handlers thread
deadlineAtMs into runCycle; CycleOpts carries it to the patterns phase.
- patterns: clampSubagentBudgets() derives BOTH the child job timeout and
the wait timeout from the same child deadline (parent deadline minus a
60s stop-margin reserve — enough for the wait poll + force-evict grace
+ cleanup, deliberately NOT a promise that tail phases complete). Under
a 2-min minimum the phase skips honestly (insufficient_cycle_budget)
instead of submitting a guaranteed-kill LLM call; the next cycle
retries with a fresh budget.
- Direct callers (gbrain dream) pass no deadline and keep the configured
timeouts unchanged.
Follow-up (separate PR): synthesize has the same shape plus sequential
per-child waits that accumulate N x subagent_wait_timeout_ms past any
parent budget; it needs per-wait remaining-time recomputation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RaNDjecPPvhC7LRqkxkhGF
* fix(cycle): address review — cancel timed-out patterns child; thread deadline through phase-wrapper handlers
- P1: the child's timeout_ms clock starts at ITS claim, so a queued child
could outlive the parent deadline the wait was clamped to. On wait
timeout, cancelJob strips it (waiting -> cancelled; active -> lock
stripped, worker abort fires next renew tick).
- P2: makePhaseHandler (standalone patterns/synthesize/... minion jobs)
now threads job.deadlineAtMs into runCycle like the autopilot handlers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RaNDjecPPvhC7LRqkxkhGF
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Adds an EU-hosted provider covering embedding, expansion and chat on one
OpenAI-compatible endpoint (https://api.mistral.ai/v1), so a brain that must
stay inside EU jurisdiction does not need a US hop for any AI touchpoint.
Every field is measured against the live API, not copied from docs:
- mistral-embed is fixed 1024 dims and accepts no dimension parameter.
Both spellings are rejected: {"dimensions": N} returns 400 extra_forbidden,
{"output_dimension": N} returns 400 "does not support output_dimension".
The generic openai-compatible branch of dimsProviderOptions() already falls
through to `return undefined` for these model ids, so nothing is emitted.
Same contract as voyage-4-nano, pinned by a negative assertion in the test.
- max_batch_tokens 65536: a 65,286-token batch is accepted, 66,960 returns
400 code 3210 "Too many tokens overall, split into more batches."
- chars_per_token 2: the value is a DIVISOR in splitByTokenBudget()
(estTokens = text.length / charsPerToken), so lower is the conservative
direction. The module default of 4 assumes English prose; a German-language
corpus measured 3.58 chars/token, which the default overshoots toward
overflow.
codestral-embed is deliberately left out: it returns 1536 dims, and a
touchpoint carries a single default_dims. Listing it under a 1024 declaration
is the mixed-dim case embedding-dim-check.ts exists to catch.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
configureFromEnv() hand-assembled its own AIGatewayConfig instead of calling
buildGatewayConfig(), the single seam that folds file-plane API keys
(openrouter_api_key, zeroentropy_api_key, ...) into the gateway env. That let
`gbrain providers list`/`test` report a provider as missing env even when it
was correctly set in ~/.gbrain/config.json and the real gateway path resolved
it fine. Both configureFromEnv() and runList() now build their env through
buildGatewayConfig() (falling back to a bare process.env passthrough
pre-init), matching what init-provider-picker.ts already does.
The one-shot CLI teardown drains fire-and-forget background sinks with a
hardcoded 2s per-sink budget (DEFAULT_DRAIN_TIMEOUT_MS). That budget
assumes a sub-second cloud chat provider; on a self-hosted provider (e.g.
an ollama model at 10-20s per completion) a facts:absorb extraction can
never finish inside it, so every one-shot CLI exit — sync timers
especially — aborts the in-flight chat with
'pipeline_error: The operation was aborted', and the same touched pages
retry-and-abort on every subsequent sync. Facts from those pages silently
never land, and doctor's facts_extraction_health warns permanently.
Fix: resolveDrainTimeoutMs() — GBRAIN_DRAIN_TIMEOUT_MS env override
(same env-only escape-hatch pattern as GBRAIN_TEARDOWN_DEADLINE_MS and
GBRAIN_FLUSH_GRACE_MS) over the 2000ms default. Explicit drainTimeoutMs
from a call site still wins; computeTeardownDeadlineMs already computes
the backstop from the resolved value, so the deadline scales with it.
Garbage/zero/negative env values fall back to the default.
Tests: default, env override, garbage/zero/negative fallback,
finishCliTeardown drains with the env-resolved budget, explicit opts
still win over env.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
`extract-conversation-facts --dry-run` promises "no DB writes, no
checkpoint advance" (help text) and correctly skips the fact INSERTs,
orphan delete, checkpoint advance, and receipt page. But
writeRunReceiptAndRollup was called unconditionally at both exit paths,
and its upsertExtractRollup always UPSERTs a row into extract_rollup_7d
("ALWAYS fire so doctor's extract_health sees the cycle ran") — so a
dry run mutates the DB.
Gate both writeRunReceiptAndRollup call sites on !dryRun. The writer
returns void and its only non-rollup action (the receipt page) is
already suppressed in dry-run via facts_inserted > 0, so gating at the
call site skips nothing else. Mirrors the existing !dryRun guards on
the fact-insert / checkpoint / audit paths.
Regression test: a dry run leaves extract_rollup_7d empty. Fails before
(row count 1), passes after. Hermetic PGLite + stubbed transports, no
live LLM.
Note: --dry-run still calls the extractor (LLM) by design — facts_extracted
is the reported "would extract N" preview count; left unchanged.
setupDB() TRUNCATEs every data table on whatever DATABASE_URL points at,
and run-e2e.sh deliberately preserves an exported DATABASE_URL — one
stray environment variable away from wiping a production brain, with no
guard of any kind (found during an independent review, 2026-07-18).
assertSafeE2eDatabaseUrl (pure, unit-tested) now runs before any
connection: allowed when the database name carries "test" as a word
segment (the gbrain_test convention used by CI and
.env.testing.example), or when GBRAIN_E2E_ALLOW_DB names the exact
database intentionally. Refusal is loud and actionable. 7 unit tests,
no DB required.
Co-authored-by: Aleksei Razsadin <razsoc.01@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Pages were unreachable by their own exact titles: FTS indexed only chunk
body text while the title-weighted pages.search_vector (GIN-indexed since
its introduction) was never queried by any search path, and
websearch_to_tsquery AND-at-chunk-grain semantics meant one non-matching
token zeroed keyword recall with no fallback — long or acronym-bearing
titles (e.g. "IAWG ... AAR-LL deck") fell through to the vector arm alone
and missed.
- searchTitles (both engines): page-grain candidate arm over
pages.search_vector (title 'A' + compiled_truth 'B' + timeline 'C'),
ts_rank_cd ranked, representative-chunk LATERAL join, full filter
parity with searchKeyword (visibility, soft-delete, source grants,
hard-excludes, dates, types); fused as a weighted RRF list at the
keyword arm's intent-effective k on all three hybrid return paths;
fail-open with warnOncePerProcess. No schema changes — the index
already existed, dark.
- AND->OR one-retry fallback for the keyword arm, gated behind
SearchOpts.orFallback (only hybridSearch opts in; countMentions, link
resolution, eval, and keyword-only MCP callers keep the strict-AND
contract). Refused for queries carrying websearch operators (negation,
quoted phrases). searchTitles carries its own page-grain fallback.
- Lexical arms parallelized (Promise.all) on the main path.
Verified: typecheck clean; 18 hermetic PGLite tests + 2 engine-parity e2e
cases (CI Postgres); consumer regression enrichment 18/0 +
link-extraction 127/0; independent live QA on a 10,664-page brain —
exact-title target miss -> rank 1 (exact_title_match), controls held,
negation/quoted guards proven, strict-consumer contract pinned.
Diagnosed from a 3-lane read-only diagnostic; adversarial review round
closed findings on fallback scope, Postgres test coverage, and operator
handling before this commit.
Co-authored-by: Aleksei Razsadin <razsoc.01@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(search): per-call token-budget meta no longer masks the real cut; restore vacuous search-lite coverage
The search-lite integration tests were structurally vacuous: putPage never
creates chunks and searchKeyword joins content_chunks, so every fixture
query returned zero rows on every machine. The tight-budget cut test's
defensive skip ('keyword search may dedupe by page') silently returned
before its assertions had ever executed anywhere, and the two budget-meta
tests ran against empty result sets.
Restoring the fixture (upsertChunks per page) and hardening the
assertions immediately exposed a real meta bug: with a per-call
tokenBudget, the inner hybridSearch enforces the same resolved budget
(per-call wins in resolveSearchMode) and its meta carries the true
dropped count — but hybridSearchCached re-applies the budget to the
already-cut set and published THAT pass's meta, which always reads
dropped=0. onMeta consumers saw a budget record claiming nothing was
dropped while rows were; telemetry (recorded from the inner meta)
disagreed with the caller-visible meta.
- hybrid.ts finalMeta: prefer innerMeta.token_budget when a per-call
budget is set (outer budgetMeta stays as the fallback and remains the
enforcement for the cache-HIT path, where no inner run exists)
- test fixture: chunk each page (pattern: chunk-grain-fts.test.ts)
- cut test: defensive skip replaced with a hard >=2 precondition;
results non-empty + strictly-fewer-than-unbounded + dropped>0 now
actually execute (revert-checked red on the unfixed meta)
- budget-meta tests: assert non-empty result sets so kept=results.length
can no longer pass vacuously at 0=0
The unbounded 'builder' query returns 2 of 3 fixture pages by design —
dedup Layer 3 caps any single page type at 60% of results and the
fixture is all-person — which the >=2 precondition accommodates.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KFWV7zBZFcmD94Vek6xRDT
* fix(search): cache-HIT budget meta prefers the stored cut record per review; exact-count assertions, mixed-type fixture
- hit path had the symmetric masking (codex P2): the re-application runs
on the already-trimmed stored set and read dropped=0 while the miss
that produced the same result set reported the real cut. Prefer
hit.meta.token_budget unconditionally — tokenBudget is folded into
knobsHash ('tb='), so a hit only ever serves a lookup with the
identical resolved budget as the write and the outer pass can never
cut further (verified against mode.ts; this is why the reviewer's
'outer wins when it drops' branch is unreachable). budgetMeta remains
the fallback for legacy rows stored without a budget record
- new serial test drives a real store-then-hit roundtrip (mocked
embedQuery, real PGLite cache) and pins hit token_budget == miss
token_budget; revert-checked red (dropped=0) on the unfixed path
- lite test: dropped asserted as the exact unbounded-minus-kept count
and used>0 (dropped>0 alone accepts any wrong positive; used<=250
alone accepts a bogus zero), fixture types mixed (person/company/note)
so dedup Layer-3 type-diversity policy no longer shapes the test
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KFWV7zBZFcmD94Vek6xRDT
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(search): classify cache hit/miss in telemetry — hits were invisible, misses unclassified (#2952)
search stats reported 0 hit / 0 miss forever: recordSearchTelemetry fired
only from bare hybridSearch, whose meta never carries a cache field, and a
cache HIT returned from hybridSearchCached before any record at all — so
hit searches also vanished from count/results/tokens/rank-1.
- HybridSearchOpts: internal _telemetryCacheStatus ('miss' | 'disabled')
threaded from hybridSearchCached into the inner hybridSearch (same
pattern as _queryEmbedDeadline), folded into the RECORDED meta only —
onMeta payloads unchanged, count/sum_tokens/budget_dropped/rank-1
behavior byte-identical for the miss/disabled paths
- hit path: record once from hybridSearchCached with the already-built
cachedMeta (cache.status='hit'), post-slice/budget result count, tokens
from the budget pass, and the same rank-1 rule as the inner paths
- bare hybridSearch direct callers (think/gather, brainstorm, enrich,
evals, ...) keep recording exactly as before, with no cache field
- test: serial wiring test drives a real store-then-hit roundtrip through
hybridSearchCached (mocked embedQuery, real PGLite SemanticQueryCache)
and pins the decision matrix (miss / hit / consult-skipped / bare);
revert-checked red on pre-fix source at the miss classification
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KFWV7zBZFcmD94Vek6xRDT
* fix(search): harden cache-hit telemetry per review — mode-gated tokens, embed-failure coverage
- hit-path tokens_estimate now gated on the MODE-resolved budget,
mirroring the inner paths' resolvedMode.tokenBudget > 0 meta condition
(a tokenmax budget-off brain would otherwise record real tokens on hits
but 0 on misses, inflating avg-tokens as the hit rate rises)
- test: exact token-delta parity assertion (hit contributes the same
tokens as the miss that stored the served set) — catches the class of
hit/miss accounting asymmetry the increase-only check accepted
- test: embed-provider-failure flavor of the disabled path (consult
degrades via catch, keyword fallback serves, neither counter bumps)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KFWV7zBZFcmD94Vek6xRDT
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
`jobs list|get` have had remote MCP routing since v0.32, but the CLI
shell still ran connectEngine() before dispatch — on a thin-client
install that fabricates an empty scratch PGLite in the thin-client
GBRAIN_HOME and replays the entire migration chain on every invocation,
before the remote call even runs. Host-only jobs subcommands (work,
supervisor, submit, ...) and `config` did the same instead of refusing.
- cli.ts: dispatch thin-client `jobs list|get` engine-free
(runJobs(null, ...)); refuse the other jobs subcommands with a
pinpoint hint; add `config` to THIN_CLIENT_REFUSED_COMMANDS with a
hint (it reads/writes the host brain's config plane).
- jobs.ts: widen runJobs to accept a null engine, guarded so null can
only reach the MCP-routed list/get branches.
- tests: behavioral (no scratch store created, no migration replay,
refusals carry hints) + source-audit pins in the existing idioms.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
submit_job/get_job return the MinionJob row verbatim; its lifecycle
field is `status` (src/core/minions/types.ts), not `state`. remote ping
typed and read `state`, so every poll saw undefined, the terminal check
never matched, and ping always burned its full --timeout and exited 1
even when the autopilot-cycle had completed — printing
"Job #N is still undefined." on the way out.
Reads fixed to `status`; the ping's own JSON output keys (`state`,
`last_state`) are unchanged for consumers. Source-audit regression test
pins the field reads.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Part of #2946. The hung-COUNT deadline race derived its verdict from a
post-await wall-clock re-check, which races the timer's own drift: on
loaded CI runners setTimeout callbacks fire measurably EARLY relative to
Date.now(), so each padding/boundary adjustment (>=, +1ms) only moved
which wrong status the partial-scan test received ('scanned', then
'partial').
The race's timeout arm now resolves a module-private sentinel; the
sentinel winning IS the deadline verdict (deadlineHit), consulted by the
post-await check without re-reading the clock. A COUNT that resolves
null (failed/absent count) stays distinguishable and does not skip the
scan; a COUNT that resolves slowly without the timer winning is still
caught by the retained wall-clock re-check. The +1ms pad is gone — the
sentinel makes timer drift irrelevant for the hung path.
Verified: partial-scan suite green 8 consecutive runs incl. the new
null-vs-sentinel distinction test.
Claude-Session: https://claude.ai/code/session_01KFWV7zBZFcmD94Vek6xRDT
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The v123 configurable-FTS migration (#2941) logged its completion notice
via console.log. Migrations run lazily inside any command's first DB
connect, so on the nightly heavy run (fresh Postgres service DB) the
line landed as the first line of `gbrain doctor --json` stdout and broke
the fm_wallclock jq parse ("Invalid numeric literal at line 1, column 7",
run 29731426470). runMigrations' contract routes all migration noise to
stderr; move the v123 prints (and the pre-existing v2 slug-rename print)
there.
Also un-vacuous the fm_wallclock harness: its register-source step used
`bun run -e` (bun dumps usage with exit 0 instead of running the code)
and `connect({})` (in-memory), so the source was never registered and
doctor scanned nothing. It now resolves the engine the way the CLI does
and registers the source in the DB doctor actually reads.
Regression test: test/migrate-stdout-clean.test.ts re-runs migrations
from v122 asserting zero stdout writes, plus a source-level guard that
migrate.ts contains no console.log.
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(sync): --src-subpath + --exclude for monorepo subdir-source support (#753)
Rebased port of PR #774 onto current master. A single git repo can hold N
logical sources at subdirectories: git operations run at the discovered
repo root (git rev-parse --show-toplevel — worktrees and submodules
resolve natively) while file walking, imports, deletes and renames are
scoped to the subpath. Passing the subdirectory directly as the repo path
(auto-discovery) works through the same code path.
Path-containment guards (the point of the feature):
- NAV-1/NAV-2: the realpath-resolved scope must live inside the
realpath-resolved git root — ../-traversal and symlinked subdirs
pointing outside the repo are rejected before any git op.
- NAV-1 TOCTOU: per-file realpath re-validation during the incremental
import drain and rename reimport; symlink-escape files are recorded as
failures (fail-closed — the bookmark cannot advance past an escape).
- NAV-4: an --exclude set that filters out every candidate warns loudly.
Scoped syncs use git-root-relative slugs + source_path in BOTH the full
and incremental paths (runImport gains slugRoot), fixing the original
PR's full/incremental slug divergence in the auto-discovery flow.
--exclude matches scope-relative paths in both paths; exclusion never
deletes previously-imported pages. The full-sync delete reconcile is
scope-restricted and relativizes against the slug base so a healthy
scoped source can't trip the #2828 mass-delete valve. .gitignore
management resolves to the git root at every call site.
Preserves all master-side sync work since the original branch: the #2828
mass-delete safety valve, #1794 resumable checkpoints + pinned targets,
#1950 stall watchdog, #2335 heartbeat bump, #1970 bookmark reachability,
and the git-ls-files walker (#2315/#2462/#2678, whose symlink/cycle
hardening is untouched).
Co-authored-by: Jeremy Knows <jeremy@veefriends.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: listEverCommittedPaths uses gitContextRoot after #753 root-triple refactor
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Jeremy Knows <jeremy@veefriends.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Squashed superset takeover of the FTS-language trilogy by @rafaelreis-r
(#580 env-var language for query+write side, #581 migration backfill,
#582 gbrain reindex-search-vector), rebased onto current master with the
security review's required fixes applied:
- Migration renumbered v116 -> v123 (master's v116 was already claimed by
code_edges_source_backfill; master is at v122).
- Restored the v120/#1647 search_path hardening: all four CREATE OR
REPLACE trigger-function bodies (migration handler + reindex command)
now carry SET search_path = pg_catalog, public, since CREATE OR REPLACE
resets proconfig and would otherwise strip the hardening on upgrade.
- reindex-search-vector: shared progress reporter (stderr phases
reindex_search_vector.pages/.chunks), id-keyset batched backfill
(5000 rows/UPDATE) instead of single whole-table statements, and
--json no longer bypasses the --yes/TTY confirmation gate.
- Allowlist validation regex + injection tests kept exactly as authored.
- Stale v33/v116 comments swept; docs/guides/multi-language-fts.md
written (README referenced it but no PR added it); llms bundles
regenerated via bun run build:llms.
- Trilogy tests quarantined as *.serial.test.ts (env mutation, per
check-test-isolation).
Verified live on PGLite: fresh init with GBRAIN_FTS_LANGUAGE=portuguese
produces portuguese-stemmed vectors with search_path pinned; the reindex
command retokenizes an english brain to portuguese in place.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Rafael Reis <rafael.reis@contabilizei.com.br>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Takeover of community PR #2387 with the security review's required fixes
applied. Original work by @harrisali0101.
With GBRAIN_RLS_SCOPE_BINDING=1, source-scoped Postgres read methods wrap
their queries in a transaction that binds set_config('app.scopes', $1, true)
(federated sourceIds CSV > scalar sourceId > '*') so operator-managed RLS
policies can filter rows at the SQL layer — defense-in-depth layer 2 under
the mandatory app-layer source filters.
Review fixes on top of the original PR:
- Flag-off is now a TRUE pass-through: no new per-read transaction wrap
(the #1794 PgBouncer pool-exhaustion class). Only the three search
methods keep a transaction when off — exactly the sql.begin() +
SET LOCAL statement_timeout wrap they already had on master — via the
helper's alwaysTransaction option.
- Preserved the PR's latent setseed fix: listCorpusSample pins
setseed() + SELECT to one connection when seeded (alwaysTransaction
gated on opts.seed), so the deterministic path can't split across
pooled connections.
- Updated the two postgres-engine shape tests to pin the new invariant
(search methods route through withScopedReadTransaction with
alwaysTransaction; helper owns the sql.begin(); flag-off path is
callback(this.sql)).
- New behavioral tests (test/postgres-engine-rls-scope.test.ts): flag-off
pass-through, flag-off alwaysTransaction, flag-on set_config emission,
federated > scalar > '*' precedence, and the CSV as a bound parameter
(never interpolated).
- Fixed the helper header comment to match the actual branching behavior.
- Operator docs in docs/ENGINES.md: env var, policy SQL, the
ALTER ROLE ... SET app.scopes='*' default requirement, FORCE ROW LEVEL
SECURITY for owner roles, and the honest caveat about unwrapped paths.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Harris <79081645+harrisali0101@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Takeover of community PR #2387 with the security review's required fixes
applied. Original work by @harrisali0101.
With GBRAIN_RLS_SCOPE_BINDING=1, source-scoped Postgres read methods wrap
their queries in a transaction that binds set_config('app.scopes', $1, true)
(federated sourceIds CSV > scalar sourceId > '*') so operator-managed RLS
policies can filter rows at the SQL layer — defense-in-depth layer 2 under
the mandatory app-layer source filters.
Review fixes on top of the original PR:
- Flag-off is now a TRUE pass-through: no new per-read transaction wrap
(the #1794 PgBouncer pool-exhaustion class). Only the three search
methods keep a transaction when off — exactly the sql.begin() +
SET LOCAL statement_timeout wrap they already had on master — via the
helper's alwaysTransaction option.
- Preserved the PR's latent setseed fix: listCorpusSample pins
setseed() + SELECT to one connection when seeded (alwaysTransaction
gated on opts.seed), so the deterministic path can't split across
pooled connections.
- Updated the two postgres-engine shape tests to pin the new invariant
(search methods route through withScopedReadTransaction with
alwaysTransaction; helper owns the sql.begin(); flag-off path is
callback(this.sql)).
- New behavioral tests (test/postgres-engine-rls-scope.test.ts): flag-off
pass-through, flag-off alwaysTransaction, flag-on set_config emission,
federated > scalar > '*' precedence, and the CSV as a bound parameter
(never interpolated).
- Fixed the helper header comment to match the actual branching behavior.
- Operator docs in docs/ENGINES.md: env var, policy SQL, the
ALTER ROLE ... SET app.scopes='*' default requirement, FORCE ROW LEVEL
SECURITY for owner roles, and the honest caveat about unwrapped paths.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Harris <79081645+harrisali0101@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Five verified-open fixes to the dream/synthesize output family:
- #1586: thread the cycle's resolved sourceId (cycleSourceId) through
runPhaseSynthesize -> SubagentHandlerData.source_id -> subagent tool
OperationContext, and stamp collected refs + summary page with the same
source, so synthesized pages stop landing in 'default'.
- #2415: new config knob dream.synthesize.output_root (default 'wiki',
zero behavior change unless set) drives the synthesize prompt slug
templates, the patterns reflection lookup + prompt, and remaps the
filing-rule allow-list globs. Registered in KNOWN_CONFIG_KEYS.
- #2163: synthesize_concepts writes concept pages through
importFromContent (put_page's parse->chunk->embed pipeline) instead of
bare engine.putPage, so concepts/ pages are chunked + embedded and
reachable by retrieval.
- #2569: stampDreamProvenance persists dream_generated + dream_cycle_date
into pages.frontmatter (JSONB merge via executeRawJsonb) at write time,
so generated pages are DB-queryable and put_page write-through can't
erase the marker.
- #2606: chronicle judge detects stopReason 'length' truncation and
no-JSON-array parse failures as distinct skipped reasons
(judge_truncated / judge_parse_failed) instead of a false terminal
no_events; output cap raised to 4000 and configurable via
chronicle.judge_max_tokens.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Garry Tan <garrytan@gmail.com>
Three verified-open defects, one family: sync silently destroying or
diverging on content it should preserve.
#2404 (P0) — 'ops' was hardcoded in PRUNE_DIR_NAMES (a v0.2.0-era
carve-out), so any path with an ops segment was 'pruned-dir': committed
ops/*.md never imported, and modified ops/* files hit the
unsyncableModified delete loop (whose #1433 guard only spared
'metafile'), silently deleting put-created pages like the bundled
daily-task-manager's canonical ops/tasks on every sync. Fix: remove
'ops' from the prune list (ordinary user content; the vendor/generated
entries stay), and harden the delete loop to also skip 'pruned-dir' —
a page under a pruned dir can only exist via a deliberate put_page.
#2426 (P0) — write-through content stayed DB-only and was deleted by
sync --full. All three compounding bugs fixed:
1. writePageThrough now best-effort commits the artifact (path-limited
git commit) on durability-hardened repos, so the post-commit hook
can push it; result carries committed?: boolean.
2. scripts/brain-commit-push.sh stages+commits BEFORE any pull — the
old fetch+pull-rebase-first order aborted on any dirty tree, so the
helper could never commit a MODIFIED page; brain_push's
rebase-on-reject already handles an advanced remote.
3. The full-sync delete-reconcile partitions stale pages by git
history (listEverCommittedPaths): never-committed source_paths are
DB-only write-through — pages are KEPT and re-exported to the
working tree instead of soft-deleted. Builds on the #2828
mass-delete valve (covers the below-valve cases).
#2607 — the sync --full git ls-files fast path bypassed pruneDir, so a
full pass imported (and resurrected soft-deleted) pages under dot-dirs
and vendored trees that incremental sync excludes. Fix:
isCollectibleForWalker applies the same segment-level pruneDir gate as
classifySync, so full and incremental enumeration agree.
One regression test per defect (all verified failing against master
src): test/sync-ops-pages.serial.test.ts,
test/write-through-commit.serial.test.ts, the #2426 helper-order test
in test/brain-durability-hook.serial.test.ts,
test/sync-reconcile-db-only.serial.test.ts,
test/import-git-fastpath-prune.test.ts.
Fixes#2404Fixes#2426Fixes#2607
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Port of #1837 (mvanhorn) onto current master. The extract_facts cycle
phase unconditionally wipe-and-reinserted every page's fence-owned DB
rows, so re-running a cycle on unchanged content churned rows and — on
the Postgres engine reported in #1781 — accumulated duplicates each run.
The phase now de-dupes extracted facts by the canonical (claim, source)
content key and reconciles the page-scoped DB index: no-op when already
in sync, insert-only for new keys, wipe/reinsert only when stale rows
need cleanup.
Adjustments over the original PR to fit current master:
- preserve #1972's abortSignal threading into the batch embed call
- preserve #1928's excludeSourcePrefixes: ['cli:'] on every wipe, and
exclude cli:-origin conversation facts from the existing-row set so
they neither count as stale (which would force a wipe every cycle)
nor get compared against the fence
Fixes#1781
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Four verified-open issues in the subagent-orchestration family, one PR:
- #1594: dream synthesize subagent job/wait timeouts promoted from
hardcoded 30/35-min constants to config keys
dream.synthesize.subagent_timeout_ms / subagent_wait_timeout_ms.
Approach ported from stale PR #1596 (credit @ai920wisco).
- #2778: add_timeline_entry joins the subagent brain-tool allowlist,
fenced fail-closed by the shared enforceSubagentSlugFence (extracted
from put_page's inline check — same trusted-workspace allow-list /
wiki/agents/<id>/ namespace policy). The per-turn output cap is now
resolveMaxOutputTokens (data.max_tokens → agent.max_output_tokens →
8192, was hardcoded 4096); a max_tokens stop surfaces as
stop_reason 'max_tokens' instead of a silent end_turn, and a
mid-tool-round cap hit injects a truncation note so the model
re-issues the dropped call.
- #2782: patterns phase status now reflects the child outcome —
non-complete outcome with zero writes → fail (PATTERNS_CHILD_<OUTCOME>),
partial writes → warn. Patterns timeouts get the same config-key pair
(dream.patterns.subagent_timeout_ms / subagent_wait_timeout_ms).
- #2113: facts extraction cap is config facts.extraction_max_tokens
(default 4000, was hardcoded 1500); stopReason 'length' is checked,
retried once at 2x the cap, and surfaced on stderr instead of
silently extracting zero facts.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: ai920wisco <ai920wisco@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Absorbs PR #2279's intent (drop the hardcoded ANTHROPIC_API_KEY gate) with
the end-state the gateway world actually wants: the patterns phase now
probes the RESOLVED patterns model through probeChatModel(normalizeModelId)
— the same semantics as think/index.ts and synthesize's makeJudgeClient.
Fixes two misclassifications of the old env gate:
- Non-Anthropic stacks (litellm, deepseek, openrouter, ...) were skipped as
"no upstream" even though the subagent routes them through the gateway
(agent.use_gateway_loop). They now pass the gate; their auth is checked
lazily at dispatch and surfaces in the job outcome.
- Anthropic keys set via `gbrain config set anthropic_api_key` (stdio MCP
launches without shell env) were treated as missing. hasAnthropicKey
inside probeChatModel reads both sources.
Skip reason renames no_api_key → no_provider (carrying the probe's detail).
Both pinning tests updated: the structural test asserts the probe wiring;
the PGLite E2E swaps its env-only helper for the shared hermetic
withoutAnthropicKey (env + config file) so it can't flip to a live LLM call
on a dev machine with a config-file key.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: brettdavies <brettdavies@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Ports the still-unmerged remnant of PR #1891 (#1593 follow-up). Master's
getConfig gained retry-with-reconnect in #1603, but its siblings —
setConfig, unsetConfig, listConfigKeys — still touched `this.sql` bare, so
the first config write/list after a mid-cycle instance-pool teardown threw
the retryable "No database connection" (issue #1678) unhandled instead of
rebuilding the pool.
Adds the connRetry helper from #1891 (same retry+reconnect posture as
batchRetry, but no batch audit JSONL — a config accessor is not a sized
batch), refactors getConfig onto it, and wraps the other three. Writes are
safe to retry: withRetry only retries connection-class failures and both
writes are idempotent (upsert / delete).
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: jalagrange <jalagrange@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Ports the still-unmerged half of PR #2442. OpenAI caches prompt prefixes
automatically, but a stable prompt_cache_key keeps requests that share a
prefix on the same inference engine, lifting the automatic-cache hit rate.
chat() now derives a stable key from the system prompt + sorted tool names
for native-openai models and passes it via providerOptions.openai.
promptCacheKey. Config provider_chat_options still overrides the derived
key; anthropic/google/openai-compatible providers are untouched.
The Anthropic half of #2442 (cache_control "silent no-op") is superseded:
@ai-sdk/anthropic 3.0.74 forwards call-level providerOptions.anthropic.
cacheControl as the Messages API's request-level cache_control (automatic
prefix caching), so master's existing marker is live on current deps.
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: CoachRyanNguyen <CoachRyanNguyen@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Port of #1731 (diazMelgarejo) onto current master. gbrain import wrote
~/.gbrain/import-checkpoint.json with the caller's raw dir argument, so
a checkpoint left behind by an interrupted run (e.g. SIGTERM) could carry
"." or a symlinked spelling — an identity that resolves to whatever CWD
the next consumer happens to run from. Downstream tooling that treated
the checkpoint dir as an owned staging boundary could then act on the
wrong directory.
- runImport captures the import target ONCE via resolveImportTargetDir
(resolve + realpathSync) and threads that canonical value through
collection, checkpoint load/save, and resume filtering
- checkpoints are self-describing (schema_version: 1, owner: "gbrain",
kind: "import"); loadCheckpoint tolerates absent metadata (legacy
path-based files) but rejects present-and-wrong metadata and any
relative dir
- checkpoint contract documented in docs/guides/live-sync.md (llms
bundle regenerated)
- test/import-resume.test.ts fixture now realpaths its tmpdir so planted
checkpoints match the canonicalized dir (macOS /var -> /private/var)
Fixes#1728
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Lawrence Melgarejo <Lawrence@cyre.me>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 14:17:37 -07:00
456 changed files with 30131 additions and 2544 deletions
All notable changes to GBrain will be documented in this file.
## [0.42.65.0] - 2026-07-23
**A large maintenance release: 93 verified fixes and small features merged since v0.42.64.0, most of them community contributions.**
If you use gbrain day to day, this release makes the boring parts trustworthy. Importing and syncing notes is safer: a failed pull no longer pretends everything is up to date, imported pages are read back after writing to confirm they landed, and a page with real content can no longer be silently overwritten by an empty one. Search answers get better inputs: the think command now picks excerpts that actually match your question, and results respect your federated source settings. Background enrichment (the "dream" cycle) wastes less money and retries properly when an AI provider is down. Spending caps now fail closed, so a billing hiccup can never turn into an uncapped spend. And `gbrain doctor` is quieter, with several false alarms removed and real problems (like an embedding backlog with no worker running) now flagged.
More AI providers work out of the box, including OpenRouter prompt caching, MiniMax and Zhipu GLM recipes, Ollama Matryoshka embedding dimensions, and llama-server batch limits.
## To take advantage of v0.42.65.0
`gbrain upgrade` should do this automatically. No new schema migrations ship in this release.
1. **Upgrade and verify:**
```bash
gbrain upgrade
gbrain doctor
gbrain stats
```
2. **If `gbrain doctor` reports new findings after upgrading,** that is the quieter, more accurate check set working as intended. Each finding names its fix.
3. **If any step fails,** please file an issue at https://github.com/garrytan/gbrain/issues with the output of `gbrain doctor` and `~/.gbrain/upgrade-errors.jsonl` if it exists.
### Itemized changes
#### Security
- MCP source scoping for remote callers got a hardening pass, so agent-facing connections stay confined to the sources they were granted. (#2881, contributed by @spinsirr)
- Paid MCP spend accounting is now atomic and fails closed, and resolver spend is recorded before a cap error is raised, so caps cannot be raced past or undercounted. (#3203, #3204, contributed by @caterpillarC15)
- The OAuth token endpoint rate limit on the HTTP server is now configurable via env for deployments behind shared IPs. (#3114, contributed by @time-attack)
- `WWW-Authenticate` responses now carry `resource_metadata` per the MCP spec and RFC 9728, so conforming clients can discover the auth server. (#1410, contributed by @rayers)
#### Search, retrieval, and think
- `think` selects query-relevant excerpts instead of generic ones. (#3197, contributed by @Y0lan)
- Unqualified local CLI `search`/`query` now honors `sources.config.federated` read visibility. (#2561, #3141, contributed by @time-attack)
- Email citation metadata is projected into search results. (#2873, contributed by @amtagrwl)
- The `think` Gaps section renders once instead of twice. (#1662, contributed by @howwohmm)
- Fuzzy entity lookup threads the caller's source scope and skips soft-deleted entities. (#1508, contributed by @tim404x)
- `code-def` surfaces method, constructor, field, and struct definitions, not just top-level symbols. (#1628, contributed by @rayers)
- Briefing pages are excluded from their own Brain Pulse salience. (#1202, contributed by @rwbaker)
- Reranker calls with missing auth are classified as configuration errors before falling back. (#2059, #3139, contributed by @time-attack)
#### Import, sync, and ingestion
- A failed git pull with zero imports reports `partial (pull_failed)` instead of `up_to_date`. (#3068, #3253, contributed by @Masashi-Ono0611)
- Imports run a post-write read-back verification with a durable ingest-log record. (#2869, contributed by @Andredsouza1984)
- `put` refuses to overwrite a non-empty page with empty content. (#2708, contributed by @symmetric-matthew)
- `putPage` restores soft-deleted rows instead of colliding with them. (#2779, contributed by @RerankerGuo)
- Mixed-case slugs are normalized before chunk upsert, ending duplicate-chunk churn. (#430, #3143, contributed by @time-attack)
- Imports fall back to the body H1 for the title when frontmatter lacks `title:`. (#2446, #3072, contributed by @time-attack)
- YAML comments inside the frontmatter fence are no longer treated as markdown headings. (#3225, #3247, contributed by @Masashi-Ono0611)
- Write-through guards case-insensitive filesystem collisions before the atomic write. (#2831, #3119, contributed by @time-attack)
- Path-qualified wikilinks outside the known directory pattern resolve on the DB/put_page path. (#2866, contributed by @paul-0320)
- CJK slugs are supported in the slug registry and dream-cycle summary slugs. (#782, #738, #3083, contributed by @time-attack)
- Three ingest/sync/serve singleton fixes: page-type round-trip, deleted-slug embed noise, and a stateless width guard. (#3140, contributed by @time-attack)
- Sync honors the `embedding_disabled` sentinel as an implicit `--no-embed`. (#2879, contributed by @gawievanblerk)
- Verified sync head sentinels are cleared correctly. (#2734, contributed by @symmetric-matthew)
- Resumed syncs report the pinned commit they actually landed on. (#3202, contributed by @caterpillarC15)
- The expected `discover_git_root` probe failure stays off stderr. (#3232, contributed by @Masashi-Ono0611)
- `extract --stale` runs the real resolver so basename resolution reaches stale pages, and clears pre-version-bump pages. (#2576, #2717, contributed by @paul-0320; #1791, contributed by @Nazim22)
- Oversized code chunks are capped so they stay embeddable, and code-chunk metadata survives re-embeds. (#1675, contributed by @lubosxyz; #769, #1232, contributed by @rayers)
#### Background cycle, dream, and facts
- Path-derived dream sources are stamped, and the engine closes cleanly on autopilot shutdown. (#3178, contributed by @time-attack)
- All-provider-failed atom drains propagate so durable jobs retry instead of silently dropping work. (#3218, #3248, contributed by @Masashi-Ono0611)
- Atom extraction raises `maxTokens` and case-normalizes `atom_type` for Gemini models. (#3211, contributed by @alexey-metaengage)
- The conversation extractor gates anonymous-speaker self-attribution instead of guessing. (#3228, contributed by @asenkovskiy)
- Incremental dream extraction stamps its watermark so re-runs stop reprocessing. (#2636, #3115, contributed by @time-attack)
- `dream --dry-run --json` keeps stdout clean of embed summaries. (#394, #3109, contributed by @time-attack)
- Synthesized dream pages require a self-contained opening summary. (#2770, contributed by @Masashi-Ono0611)
- PGLite inline synth subagent drains complete, and `lint` gains `--exclude`. (#2699, #2649, #3162, contributed by @time-attack)
- Live context reads the documented "P1 Today" heading form with plain checkbox tasks, matching the daily-task-manager skill's output format. (#2186, #3124, contributed by @time-attack)
- Queued AI jobs refresh gateway config at execution time instead of using a stale snapshot. (#2125, contributed by @maxpetrusenkoagent)
- `brainstorm`/`propose_takes` honor configured models: cost preview uses the configured model, the judge reads its config key, provider probes are skipped when unneeded, and page projection is narrowed. (#3120, contributed by @time-attack)
- Backlog hardening wave: x-to-brain health check, propose_takes deadlines, capture title truncation, extract_atoms backlog handling, and pooler direct-URL routing. (#3165, contributed by @time-attack)
- `skillopt` emits `proposed.md` in no-mutate mode. (#2635, #3182, contributed by @time-attack)
- Nightly quality probe enable path and conversation-parser probe are wired up. (#2629, #2630, #3094, contributed by @time-attack)
#### Doctor, health, and maintenance
- New safe maintenance automation with a shared orphan-exclusion policy, so routine cleanup runs without risking linked content. (#3015, #3023, contributed by @time-attack)
- `orphan_ratio` excludes the chronicle volume under `life/events/`. (#2264, #3214, contributed by @asenkovskiy)
- `brain_score` orphan/timeline components use the orphans-audit linkable scope. (#3155, contributed by @time-attack)
- Entity timeline coverage is measured separately from whole-brain density. (#2761, contributed by @TurgutKural)
- Doctor flags embed backfills queued with no worker running. (#2696, contributed by @javieraldape)
- Two doctor false-positive/timeout fixes: the drift walk skips `node_modules`, and the bare-tweet check skips inline code and cited lines. (#1772, contributed by @sonlndv)
- A dead `llm_fallback_enabled` recommendation is dropped from conversation format coverage. (#1903, contributed by @ElliotDrel)
- Skill triggers with CRLF line endings parse on Windows. (#1149, contributed by @samporter-31)
- Onboard check names are registered in doctor categories, ending unknown-check warnings, and onboard-check remediations survive the `--apply --auto` path. (#3075, #3097, contributed by @time-attack)
- Dead slug prefixes are counted by slug. (#2697, contributed by @RerankerGuo)
- The backlinks worker defaults to check, not fix, and `check-backlinks` honors its positional directory argument. (#1853, contributed by @choomz; #3076, contributed by @time-attack)
- Calibration resolves the owner holder via config, defaulting to `self`. (#3077, contributed by @time-attack)
- Memory throttling on Linux reads `/proc/meminfo` MemAvailable. (#556, contributed by @chengzehsu)
#### AI providers and gateway
- OpenRouter gets family-scoped prompt caching, and query expansion works on chat-capable openai-compat recipes. (#3152, contributed by @time-attack)
- MiniMax recipe: embedding wire-shape compat fetch plus a chat touchpoint. (#1977, #3089, contributed by @time-attack)
- The Zhipu recipe gains a chat touchpoint so GLM subagents work. (#1157, #3084, contributed by @time-attack)
- Tier-configured models reach the recipe allowlist, Anthropic model lists are refreshed, tier resolutions are registered, and probe labels are honest. (#2800, contributed by @p3ob7o)
- Provider base URL config merges from the DB. (#1676, contributed by @TheLordArgus)
- The gateway falls back to the pooler when the derived direct host is unreachable. (#1641, #3088, contributed by @time-attack)
- Config-plane `voyage_api_key` folds into `VOYAGE_API_KEY` like the other hosted keys. (#3236, contributed by @Masashi-Ono0611)
- The `zeroentropyai:zerank-2` reranker has a pricing entry so the budget tracker can meter it. (#3223, #3233, contributed by @Masashi-Ono0611)
- llama-server embedding batches are capped at its 32-input request limit. (#1281, contributed by @mmekkaoui)
- Matryoshka dimensions thread through for Qwen3-Embedding on Ollama. (#1072, contributed by @mgandal)
- `init` seeds AI options from env on cold install, and `whoami` reports the stdio transport. (#3091, contributed by @time-attack)
- The `models` dispatch subcommand reads its first argument correctly. (#1428, contributed by @BenjaminDSmithy)
- Synopsis generation tail-truncates document text for small-model chat handlers. (#1427, contributed by @BenjaminDSmithy)
- The contradiction judge token cap is raised for thinking models. (#3210, contributed by @alexey-metaengage)
#### Schema, migrations, and storage engines
- Engine migration counts and surfaces per-page copy failures instead of silently advancing. (#3241, contributed by @Masashi-Ono0611)
- Invalid `CONCURRENTLY`-build index remnants are dropped without a DO block. (#3191, contributed by @Masashi-Ono0611)
- Unsupported large-dimension HNSW indexes are skipped instead of failing schema setup. (#1734, #3080, contributed by @time-attack)
- The v0.32.2 migration dirty-check scopes to targeted sources and surfaces failed phase detail. (#3093, contributed by @time-attack)
- Schema packs merge the full `extends` chain and `borrow_from` into the resolved manifest. (#1749, #3181, contributed by @time-attack)
- The schema-pack stats catch-all is narrowed so masked errors surface instead of fake zero-page counts. (#2466, #3133, contributed by @time-attack)
- Bundled schema-pack inspection reports the pack actually shipped in the binary, and minion subagent auth resolves through config. (#3110, contributed by @time-attack)
- PGLite `putPage` guards against zero-row RETURNING. (#1649, contributed by @alexhawkins)
#### MCP server and CLI surface
- `list_pages` rows include `source_id`. (#3209, contributed by @alexey-metaengage)
- Running CLI commands while `gbrain serve` (MCP) holds the brain now notifies about the conflict instead of failing confusingly. (#3243, contributed by @fdefitte)
- The OpenClaw plugin manifest entry is declared so the plugin loads. (#2551, #3185, contributed by @time-attack)
#### For contributors
- CI scanner roots are normalized on macOS. (#3198, contributed by @caterpillarC15)
- CI shard timeout raised to 22 minutes plus a delta-assert reporter leak test. (#3231, contributed by @time-attack)
- E2E suite hardening: flaky tests, no-op assertions, and cross-test coupling removed. (#1704, contributed by @auroracapital)
- `mechanical.test.ts` isolates `$HOME` so the E2E suite stops clobbering user config. (#434, contributed by @lloydarmbrust)
- The lint code-fence-wrap detector and fixer regex now agree. (#1597, contributed by @chungty)
- README project links for OpenClaw and Hermes are corrected. (#1961, #3179, contributed by @time-attack)
- A completed TODOS entry is dropped. (#3229, contributed by @Masashi-Ono0611)
## [0.42.64.0] - 2026-07-20
### Fixed
- Confidential OAuth clients can now revoke access tokens through the standard revocation endpoint when client secrets are stored as hashes. Invalid credentials fail closed, malformed or mixed authentication is rejected, backend failures remain retryable, and discovery metadata accurately advertises supported authentication methods.
No schema migrations.
## [0.42.63.0] - 2026-07-20
**Schema commands now open the local brain you actually configured.**
If your PGLite brain lives at a custom path, commands such as `gbrain schema stats` previously ignored that path and could inspect the default brain instead. That made a healthy configured brain look empty or report the wrong schema counts. Schema commands now use the same complete database configuration as the rest of GBrain. PostgreSQL behavior is unchanged, and no migration is required.
### How to use it
Upgrade, then run the schema command normally:
```bash
gbrain upgrade
gbrain schema stats --json
```
The reported page and type counts now come from the `database_path` in `~/.gbrain/config.json` when the engine is PGLite.
### Itemized changes
#### Fixed
- **Schema CLI commands preserve configured PGLite paths.** Engine construction and connection now receive the canonical complete engine configuration, including both `database_path` and `database_url` where applicable.
- **CLI tests are isolated from ambient database URLs.** Schema subprocess tests explicitly clear inherited PostgreSQL URL variables, and a persistent-PGLite regression test proves `schema stats` reads the configured database rather than the default brain.
## [0.42.62.0] - 2026-07-17
**If your brain holds more than one source, everything now lands in the right one. Link extraction, timeline extraction, background cycles, and webhook captures used to quietly file some of their output under the default source; all of those paths now carry the correct source identity. Background agent jobs got tougher too: a failed database reconnect can no longer wedge the engine, and workers recover from dropped connections instead of crash-looping. If you run the admin dashboard behind a reverse proxy, the live activity panel finally connects. Long agent conversations cost less because repeated context is reused between turns on Anthropic calls. Local LiteLLM proxies work out of the box. Nested sources scan correctly again instead of reporting zero files. And the project's automated checks now include dependency vulnerability scanning, static code-security analysis, and signed provenance for release builds. Thirty merged changes in all, the largest batch to date, each one reviewed and verified against the live codebase before landing.**
- **[Hermes](https://github.com/NousResearch/hermes-agent)** — deploy on [Railway](https://github.com/praveen-ks-2001/hermes-agent-template) (one click)
Then paste this into your agent:
@@ -258,6 +258,24 @@ The whole loop is described in [`docs/architecture/topologies.md`](docs/architec
**Job queue (Minions).** BullMQ-shaped, Postgres-native job queue. Durable subagents (LLM tool loops that survive crashes via two-phase pending→done persistence), shell jobs with audit, child jobs with cascading timeouts, rate leases for outbound providers, attachments via S3/Supabase storage. Replaces "spawn subagent as fire-and-forget Promise" with something that recovers from anything.
**Non-English brains (FTS language config).** The Postgres full-text search tokenizer is configurable via `GBRAIN_FTS_LANGUAGE`. Defaults to `english`. Set it to any text-search configuration that exists in your Postgres instance:
List available configs: `psql -c "SELECT cfgname FROM pg_ts_config"`. Both the **query side** (`websearch_to_tsquery`) and the **write side** (the trigger functions that populate `pages.search_vector` and `content_chunks.search_vector`) honor `GBRAIN_FTS_LANGUAGE`. On first install (or upgrade), the `configurable_fts_language` schema migration reads the env var and creates trigger functions in the configured language; subsequent inserts/updates tokenize using that setting. To change language on a brain that has already run the migration, use the dedicated CLI command:
The command is idempotent (re-running with the same language is a no-op for vector content) and uses the same recreate-and-backfill primitives as the migration. For accent-insensitive Portuguese (`pt_br`), see [docs/guides/multi-language-fts.md](docs/guides/multi-language-fts.md) for the `unaccent` + portuguese stemmer recipe.
**43 curated skills.** Routing lives in [`skills/RESOLVER.md`](skills/RESOLVER.md). Covers signal capture, ingest (idea / media / meeting), enrichment, querying, brain ops, citation fixing, daily task management, cron scheduling, reports, voice, soul audit, skill creation, eval framework, and migrations. Skills are markdown files (tool-agnostic), packaged as a single skillpack the installer drops into your agent workspace.
**Eval framework.**`gbrain eval longmemeval` runs the public [LongMemEval](https://huggingface.co/datasets/xiaowu0162/longmemeval) benchmark against your hybrid retrieval. `gbrain eval export` + `gbrain eval replay` capture real queries and replay them against code changes (set `GBRAIN_CONTRIBUTOR_MODE=1`). `gbrain eval cross-modal` cross-checks an output against the task using three different-provider frontier models. `gbrain eval retrieval-quality` runs NamedThingBench, which hard-gates the named-thing retrieval families (title-substring, alias-synonym, generic-to-named, multi-chunk-dilution) so a regression in "find the page this query names" fails CI loudly. Full methodology in [`docs/eval/SEARCH_MODE_METHODOLOGY.md`](docs/eval/SEARCH_MODE_METHODOLOGY.md).
@@ -148,6 +148,51 @@ RRF fusion, multi-query expansion, and 4-layer dedup are engine-agnostic. They o
**Why not self-hosted for v0:** The brain should be infrastructure agents use, not something you maintain. Self-hosted Postgres with Docker is a welcome community PR, but v0 optimizes for zero ops.
-`test/postgres-engine.test.ts` — `statement_timeout` scoping: `sql.begin` + `SET LOCAL` shape, source-level grep guardrail against a reintroduced bare `SET statement_timeout`.
-`test/sync.test.ts` — sync logic + regression guard asserting top-level `engine.transaction` is not called.
-`test/sync-pull-failed-anchor.serial.test.ts` — #3068 regression: a failed internal `git pull` (local-path origin vs `protocol.file.allow=never`) with zero imports returns `partial`/`pull_failed` (not `up_to_date`), freezes `last_commit` + `last_sync_at`, recovers after a manual pull; fall-through import of local commits preserved. Serial: pins `GBRAIN_HOME` to a temp dir for the whole file.
-`test/sync-parallel.test.ts` — PGLite-routed coverage of the bookmark gate under concurrency, head-drift gate, vanished-file failure capture, PGLite-stays-serial, and the `gbrain-sync` writer-lock contract.
-`test/sync-failures.test.ts` — `classifyErrorCode` regex coverage for all 12 codes against literal production message strings from `markdown.ts` and `import-file.ts`; `summarizeFailuresByCode` sort + pre-classified-honor; `recordSyncFailures` code-field persistence; `acknowledgeSyncFailures``AcknowledgeResult` shape + backfill on legacy entries.
@@ -484,6 +484,10 @@ Returns a per-source dashboard: when each source last synced, how many pages, ho
The admin dashboard at `https://brain.acme-co.com/admin` shows live request volume, registered OAuth clients, recent activity, and brain stats. Use the admin bootstrap token from Part 4 to log in the first time, then register additional admin users from inside the dashboard.
### If agents run as containers on the same Docker host
OAuth source scoping only guards the HTTP MCP path. If the brain's Postgres and your teammates' agent runtimes are containers on the same Docker host, make sure the agents can't reach Postgres directly over Docker's default bridge network — a direct DB session skips OAuth entirely. Put Postgres on its own user-defined network, publish it loopback-only if at all, and never hand agent containers a `DATABASE_URL`. The copy-paste operator checklist lives in [docs/mcp/DEPLOY.md — Co-located Docker workloads](../mcp/DEPLOY.md#co-located-docker-workloads-self-hosted-postgres).
- **[Hermes](https://github.com/NousResearch/hermes-agent)** — deploy on [Railway](https://github.com/praveen-ks-2001/hermes-agent-template) (one click)
Then paste this into your agent:
@@ -1752,6 +1752,24 @@ The whole loop is described in [`docs/architecture/topologies.md`](docs/architec
**Job queue (Minions).** BullMQ-shaped, Postgres-native job queue. Durable subagents (LLM tool loops that survive crashes via two-phase pending→done persistence), shell jobs with audit, child jobs with cascading timeouts, rate leases for outbound providers, attachments via S3/Supabase storage. Replaces "spawn subagent as fire-and-forget Promise" with something that recovers from anything.
**Non-English brains (FTS language config).** The Postgres full-text search tokenizer is configurable via `GBRAIN_FTS_LANGUAGE`. Defaults to `english`. Set it to any text-search configuration that exists in your Postgres instance:
List available configs: `psql -c "SELECT cfgname FROM pg_ts_config"`. Both the **query side** (`websearch_to_tsquery`) and the **write side** (the trigger functions that populate `pages.search_vector` and `content_chunks.search_vector`) honor `GBRAIN_FTS_LANGUAGE`. On first install (or upgrade), the `configurable_fts_language` schema migration reads the env var and creates trigger functions in the configured language; subsequent inserts/updates tokenize using that setting. To change language on a brain that has already run the migration, use the dedicated CLI command:
The command is idempotent (re-running with the same language is a no-op for vector content) and uses the same recreate-and-backfill primitives as the migration. For accent-insensitive Portuguese (`pt_br`), see [docs/guides/multi-language-fts.md](docs/guides/multi-language-fts.md) for the `unaccent` + portuguese stemmer recipe.
**43 curated skills.** Routing lives in [`skills/RESOLVER.md`](skills/RESOLVER.md). Covers signal capture, ingest (idea / media / meeting), enrichment, querying, brain ops, citation fixing, daily task management, cron scheduling, reports, voice, soul audit, skill creation, eval framework, and migrations. Skills are markdown files (tool-agnostic), packaged as a single skillpack the installer drops into your agent workspace.
**Eval framework.** `gbrain eval longmemeval` runs the public [LongMemEval](https://huggingface.co/datasets/xiaowu0162/longmemeval) benchmark against your hybrid retrieval. `gbrain eval export` + `gbrain eval replay` capture real queries and replay them against code changes (set `GBRAIN_CONTRIBUTOR_MODE=1`). `gbrain eval cross-modal` cross-checks an output against the task using three different-provider frontier models. `gbrain eval retrieval-quality` runs NamedThingBench, which hard-gates the named-thing retrieval families (title-substring, alias-synonym, generic-to-named, multi-chunk-dilution) so a regression in "find the page this query names" fails CI loudly. Full methodology in [`docs/eval/SEARCH_MODE_METHODOLOGY.md`](docs/eval/SEARCH_MODE_METHODOLOGY.md).
@@ -2095,6 +2113,51 @@ RRF fusion, multi-query expansion, and 4-layer dedup are engine-agnostic. They o
**Why not self-hosted for v0:** The brain should be infrastructure agents use, not something you maintain. Self-hosted Postgres with Docker is a welcome community PR, but v0 optimizes for zero ops.
description: Twitter timeline, mentions, and keyword monitoring flow into brain pages. Tracks deletions, engagement velocity, OCR on images, and real-time alerts.
category: sense
requires: []
@@ -9,9 +9,12 @@ secrets:
- name: X_BEARER_TOKEN
description: X API v2 Bearer token (Basic tier minimum, $200/mo for full archive search)
where: https://developer.x.com/en/portal/dashboard — create a project + app, copy the Bearer Token from "Keys and tokens"
- name: X_HANDLE
description: Your X username without the @ (used for the app-only health check — /users/me requires user-context OAuth, which app-only bearer tokens don't have)
where: Your X profile — the handle in your profile URL, e.g. x.com/yourhandle → yourhandle
'code-refs':'`code-refs` has no MCP op yet. Run on the host.',
'code-callers':'`code-callers` has no MCP op yet. Run on the host.',
'code-callees':'`code-callees` has no MCP op yet. Run on the host.',
// scratch-DB audit additions
config:"config reads/writes the host brain's config plane. Edit the host's .gbrain/config.json (file-plane keys) or run on the host with GBRAIN_HOME set.",
jobs:'`jobs list` and `jobs get <id>` are thin-client routable; this subcommand runs against the host queue. Use the submit_job / list_jobs / get_job MCP tools from your agent, or run on the host with GBRAIN_HOME set.',
// in a wrapper, etc.) happening between Bun boot and this call is
// invisible to `which` without explicitly forwarding the current env.
// This is why "which gbrain" succeeds when run standalone (fresh Bun
// process, no prior mutation) but can fail from inside autopilot's own
// process at this exact call site. Same fix already applied to
// detectTini() in spawn-helpers.ts (see its comment) — this call site
// was missed.
constwhich=execSync('which gbrain',{
encoding:'utf-8',
stdio:['ignore','pipe','ignore'],
env: process.env,
}).trim();
if(which)returnwhich;
}catch{/* not on $PATH — fall through */}
@@ -123,7 +138,14 @@ export function resolveGbrainCliPath(): string {
returnarg1;
}
thrownewError('Could not resolve the gbrain CLI path. Install gbrain so it is on $PATH (e.g. /usr/local/bin/gbrain), or run autopilot from the compiled binary directly.');
// #2747: include what we actually saw so an operator (or a future bug
// report) doesn't have to guess whether PATH/execPath/argv[1] looked
// sane at the moment of failure.
thrownewError(
'Could not resolve the gbrain CLI path. Install gbrain so it is on $PATH '+
'(e.g. /usr/local/bin/gbrain), or run autopilot from the compiled binary directly. '+
console.error('Chat gateway unavailable. Configure a chat model (e.g. `gbrain config set chat_model anthropic:claude-haiku-4-5`), or pass --dry-run to preview candidates.');
console.log(`\nMigration completed with errors. ${migrated} of ${pagesToMigrate.length} pages copied, ${failures.length} failed (${completedSet.size} already done from a prior run). See failure list above.`);
console.log(`Config NOT switched — still using engine: ${config.engine}. Re-run \`gbrain migrate --to ${opts.targetEngine}\` to retry; already-copied pages resume via the manifest.`);
`GBrain MCP server: boot did not complete within ${bootTimeoutMs}ms — releasing DB lock and exiting so other consumers unblock (check configured provider endpoints; tune via GBRAIN_SERVE_BOOT_TIMEOUT_SECONDS, 0 disables)`,
// Same call as the Moonshot recipe: ordinary tool calls are fine, but
// gbrain's subagent loop stays Anthropic-pinned for stable tool_use_id
// behavior across crashes/replays.
supports_subagent_loop: false,
supports_prompt_cache: false,
max_context_tokens: 262144,
price_last_verified:'2026-07-19',
},
},
setup_hint:'Get an API key at https://console.mistral.ai/api-keys, then `export MISTRAL_API_KEY=...` and use `mistral:mistral-embed` (1024 dims) for embeddings.',
'Get an API key at https://openrouter.ai/settings/keys, then `export OPENROUTER_API_KEY=...` and use `openrouter:<provider>/<model>`. Optional overrides: OPENROUTER_BASE_URL (proxy), OPENROUTER_REFERER (attribution URL), OPENROUTER_TITLE (attribution name).',
cost_per_1m_tokens_usd: 0.03,// pplx-embed-v1-4b; 0.6b is $0.004/M
price_last_verified:'2026-07-21',
// Perplexity enforces 120K combined tokens (and 512 texts) per
// request. Same pre-split posture as Voyage: assume a dense
// tokenizer (1 char ≈ 1 token) at 0.5 utilization; the gateway's
// recursive halving is the runtime safety net.
max_batch_tokens: 120_000,
chars_per_token: 1,
safety_factor: 0.5,
},
},
setup_hint:'Get an API key at https://www.perplexity.ai/settings/api, then `export PERPLEXITY_API_KEY=...` (re-export PPLX_API_KEY if that is where your key lives).',
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
