admin/gbrain
1
0
Fork 0
mirror of https://github.com/garrytan/gbrain.git synced 2026-08-14 00:48:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/master' into garrytan/codex-as-agent-default-install

Browse Source 
# Conflicts:
#	CLAUDE.md
#	CONTRIBUTING.md
#	README.md
#	docs/INSTALL.md
#	docs/TESTING.md
#	docs/architecture/KEY_FILES.md
#	docs/architecture/thin-client.md
#	docs/guides/search-modes.md
#	docs/mcp/CLAUDE_CODE.md
#	docs/mcp/DEPLOY.md
#	llms-full.txt
#	scripts/run-unit-parallel.sh
#	src/cli.ts
This commit is contained in:
Garry Tan
2026-08-09 17:50:53 -07:00
parent 989794701e c5952b8714
commit c1d7a61a6d
98 changed files with 7648 additions and 263 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/test.yml
+13
View File
@@ -187,6 +187,19 @@ jobs:
key: bun-cache-${{ runner.os }}-${{ hashFiles('bun.lock') }}
- run: bun install
- run: bun test test/entity-resolve-perf.slow.test.ts --timeout=300000
# MEMORY_VERBS v1 (Cathedral 1): the entity() p99 < 100ms contract gate
# (20K-page corpus + ratio guard) shares this runner — same perf-job
# shape, runs in parallel with the matrix.
- run: bun test test/entity-card-perf.slow.test.ts --timeout=300000
# Protocol self-certification: init a scratch brain and run the
# conformance kit against gbrain's own stdio server. --synthesize is
# safe here: no LLM key in CI, so it asserts the clean `unavailable`
# protocol error instead of spending tokens.
- name: MEMORY_VERBS conformance (self-certify, stdio)
run: |
export GBRAIN_HOME="$RUNNER_TEMP/gbrain-conformance"
bun run src/cli.ts init --pglite --no-embedding --non-interactive
bun run src/cli.ts protocol conformance --synthesize
test:
# Pure matrix shard — no verify, no serial. Each shard runs its slice
CHANGELOG.md
+100
View File
@@ -2,6 +2,99 @@
All notable changes to GBrain will be documented in this file.
## [0.43.0.0] - 2026-08-08
**Your agent now has five memory verbs it can actually reach.** Cathedral 1 freezes
a stable, versioned memory protocol — `recall`, `remember`, `entity`, `synthesize`,
`forget` — over the brain's operation catalog, the way Postgres speaks one wire
protocol to every client. Point any MCP harness at it (`claude mcp add gbrain --
gbrain serve --surface verbs`, or the Codex/OpenClaw equivalents) and the agent sees
exactly five self-describing tools instead of a wall of internal ops. Every response
carries what it is, why it matched, where it came from, and what it spent (the token
budget on `recall`, latency on `entity`, the full cost block on `synthesize`) — and the
contract never breaks: v1 field names and meanings are frozen, changes are
additive-forever.
What you can do now that you couldn't before:
- **Remember a fact once, recall it in a fresh session — in any harness.** `remember`
takes mandatory provenance (where the fact came from) and an optional expiry, dedupes
against what's already known, and supersedes the old fact when it changes
("X joined acme-example" → "X left acme-example" — the outdated fact expires, the
history stays). `recall` retrieves saved facts and,
with a query, budget-packed page snippets — the server enforces the token budget and
tells you what it dropped instead of trusting the client to trim.
- **Look up one person/company/project as a compact card in well under 100ms, zero LLM
calls.** `entity` resolves a name to a privacy-safe card (who/what, aliases,
last-touched, open threads, top typed edges) — or, on a miss, near-miss suggestions
instead of a dead end.
- **Reason across pages when you actually need it.** `synthesize` is the explicitly
expensive verb (it says so in its own description), with a best-effort cost block so
agents choose it deliberately. No API key configured? It says `unavailable` with a
fix, never a fake answer.
- **Certify any memory server against the contract.** `gbrain protocol --json` publishes
the machine-readable spec; `gbrain protocol conformance [--target <endpoint>]` runs the
frozen-contract test suite against gbrain's own server or any MCP endpoint — and
provably fails servers that don't comply. `gbrain protocol stats` shows per-verb
adoption and your real time-to-first-use, all from a local file that never leaves
your machine.
`gbrain serve` keeps every operation by default (`--surface full`) — existing installs
are unchanged. The new `--surface verbs` is the quickstart surface for agents. No schema
migration; the verbs ride the existing facts, pages, and typed-graph tables. The full
contract, per-harness install blocks, and the additive-forever versioning policy live in
[docs/protocol/MEMORY_VERBS_v1.md](docs/protocol/MEMORY_VERBS_v1.md).
**This release also fixes which search verb your agent reaches for (#2416).** The
`search` and `query` tool descriptions, the mandatory lookup chain, and the search
guides had drifted from reality: they described `search` as keyword-only (it has been
cheap hybrid — vector + keyword, no LLM expansion — for many releases) and told agents
to try `search` first for everything, falling back to `query` only when results looked
"thin". For concept questions ("all the X that do Y", "the ecosystem around Z") that
fallback never fired, and synonym-phrased matches dropped silently.
- **Concept questions now route to `query` by default.** The tool descriptions and the
lookup-chain convention are intent-driven: exact known tokens → `search` (cheaper, no
expansion call); concept / synonym / exhaustive-set questions → `query` first. Verified
with a live LLM routing eval: all concept phrasings route to `query`, and personal
questions still route to the salience ops.
- **"Got results" is no longer treated as "got everything."** The descriptions, docs, and
conventions now say it plainly: a populated `search` result set is not proof of
coverage, and `query` is still top-K — literal exhaustive enumeration belongs to
`list_pages` pagination.
- **The CLI nudges you when it can help.** A concept-shaped `gbrain search "..."` prints a
one-line hint on stderr suggesting the equivalent `gbrain query` call (results stay
clean on stdout; `--quiet` silences it; never auto-reroutes).
- **`think` cost accounting reads consistently.** When no LLM call ran (stubbed or no
API key), `usage` is now uniformly `null` in `--json` output rather than sometimes
missing.
To take advantage of v0.43.0.0: re-run `gbrain serve` with `--surface verbs` to give
your coding agent the five-verb memory protocol (or keep `--surface full` for the
complete operation catalog — both speak the verbs). Run `gbrain protocol conformance`
to self-certify, and `gbrain protocol stats` to watch adoption. Memories your agent
saves are readable by every agent connected to the brain by default; pass
`visibility: "private"` for local-only facts. If your agent instructions or skill
files copy the old "search first, query if thin" rule, refresh them from
`skills/conventions/brain-first.md` — the shipped skillpack carries the corrected
routing.
## [0.42.76.0] - 2026-08-08
**Mistyped or unsupported flags now fail loudly instead of being silently ignored — including the ones that were supposed to make a command safe.**
**Strict flag validation, CLI-wide.** Every gbrain command now rejects a flag it does not understand, with a clear error naming the flag and the command, before any work runs. Before, commands read their flags ad hoc and ignored the rest — so `gbrain post-upgrade --dry-run` accepted the flag, ignored it, and applied migrations for real. That class is gone: the legal flags for every command are derived from each command's own source into a generated registry, checked before dispatch, and a command may only advertise a safety flag like `--dry-run` if its code actually reads it. On commands routed through the operations contract, a trailing `--dry-run` is now a real rehearsal switch rather than a no-op. `--json` invocations get the same error as a structured payload, so scripts fail cleanly too.
**A word of warning (intentional breaking change):** cron jobs or scripts that pass stray, misspelled, or long-removed flags have been running on luck — the flag did nothing. Those invocations now exit with an error naming the flag. That is the point: fix the invocation once and it means what it says forever. Everything after `--` is passthrough and remains untouched.
**Upgrades can't wedge on forward-referenced columns anymore — as a class.** The v0.42.56.0-era startup wedge (a schema blob referencing a column that pre-existing brains didn't have yet) had two more latent instances waiting in the jobs table. Both are now probed and healed at startup, and the schema coverage guard was rewritten to cross-reference every column referenced by the embedded schema against the set of columns any migration has ever added — so a new forward reference cannot ship without its startup probe. A recovery test walks the exact journey an affected brain takes: failed upgrade, retry on the fixed binary, converge with no leftover state blocking the way.
**Remote agents get more, within the same fences.** The `think` operation is now available to remote MCP callers as a read-only synthesis — the local CLI can still persist results, while remote callers are forced read-only. Chunk reads now resolve through the same source-scope rules as page reads, so a federated grant that can open a page can also read that page's chunks, and a caller without the grant cannot reach chunks outside its own floor. Chunk payloads also stop carrying raw embedding vectors over the wire — noticeably smaller responses with no behavior change, since no consumer ever read them. Two internal call sites that forward caller identity now treat anything ambiguous as untrusted, matching the fail-closed rule the rest of the codebase already follows.
**Source-bound clients can be minted over HTTP.** The `/admin/api/register-client` endpoint now accepts `source` and `federatedRead` bindings, mirroring the CLI's `--source` / `--federated-read` flags — so an admin UI or provisioning proxy can create a client confined to a specific brain source without shelling out to the CLI. Omitting both preserves the historical default, and invalid source ids get a structured 400.
**`gbrain doctor` and `repair-jsonb` see further and misfire less.** The double-encoded-JSON scan now covers the subagent execution columns, and the damage test requires the stored text to actually parse as JSON before flagging it — a legitimate string value that merely starts with `[` or `{` (a log line, a code snippet) is no longer misclassified, and a repair pass can no longer corrupt it. One damaged table no longer aborts the scan of the rest.
### To take advantage of v0.42.76.0
## [0.42.75.0] - 2026-08-08
**The "PGLite crashes on macOS 26" era is over: gbrain now repairs a torn brain in place, automatically, with your data preserved.**
@@ -31,6 +124,13 @@ Credit where due: @yang1996202-cpu (#2575), @AndreLYL (#223), and @roysaurav (#1
gbrain upgrade
```
Nothing to configure. If a cron job or script starts failing with `unknown flag`, that invocation was passing a flag that did nothing — remove or fix the flag and it will not regress silently again.
### For contributors
Community fixes absorbed with credit: @colinagent (#2598 think read-scope; the upgrade-rewind e2e pattern from #2623), @guim4dev (#2016 register-client source bindings), @vinsew (#597 repair-jsonb coverage extension), @javieraldape (#2494/#2531 output-correctness class — BigInt-safe local rendering and the search `--json` regression pin land here; parts of both PRs shipped earlier from master). Thank you — superseded PRs are being closed with notes.
The unit-test runner is now memory-safe on machines running multiple workspaces: shard concurrency adapts to actually-available memory, and a serial rescue lane re-runs files that died to OOM or external kills before calling them failures — a red suite now means real failures, not memory pressure. The flag registry regenerates via `bun run build:flag-registry` and is pinned by freshness, drift, and consumption-evidence guards.
If your brain currently won't open, that's it — the next command repairs it. If you'd rather look first: `gbrain pglite-repair --dry-run`.
## [0.42.74.0] - 2026-08-07
CLAUDE.md
+2 -1
View File
@@ -38,7 +38,7 @@ mount, CEO-class with multiple team brains) and
## Architecture
Contract-first: `src/core/operations.ts` defines 100+ shared operations (including `volunteer_context` — push-based context, see `docs/guides/push-context.md`). CLI and MCP
Contract-first: `src/core/operations.ts` defines 100+ shared operations (including `volunteer_context` — push-based context, see `docs/guides/push-context.md` — and the five frozen MEMORY_VERBS `recall`/`remember`/`entity`/`synthesize`/`forget`, servable alone via `gbrain serve --surface verbs`, see `docs/protocol/MEMORY_VERBS_v1.md`). CLI and MCP
server are both generated from this single source. Engine factory (`src/core/engine-factory.ts`)
dynamically imports the configured engine (`'pglite'` or `'postgres'`). Skills are fat
markdown files (tool-agnostic, work with both CLI and plugin contexts).
@@ -119,6 +119,7 @@ detail on demand.)
| push-based context (volunteer/watch/reflex window) | `docs/guides/push-context.md` |
| schema packs / page types / extraction | `docs/architecture/schema-packs.md`, `type-taxonomy.md`, `lens-packs.md` |
| thin-client / remote MCP / cross-modal | `docs/architecture/thin-client.md` |
| memory verbs / MCP tool surface (`--surface`) / conformance | `docs/protocol/MEMORY_VERBS_v1.md` + the `verbs*`/`surface.ts`/`protocol.ts` entries in `KEY_FILES.md` |
| the CLI surface (commands + flags) | `gbrain --help` / `gbrain --tools-json`, plus the relevant `KEY_FILES.md` entry |
| running or writing tests | `docs/TESTING.md` |
| bulk-command progress wiring | `docs/progress-events.md` |
CONTRIBUTING.md
+17 -5
View File
@@ -93,7 +93,7 @@ lifecycle is [`docs/TESTING.md`](docs/TESTING.md). The short version:
```bash
# Inner edit loop (~85s on a Mac dev box)
bun run test # parallel 8-shard fan-out + serial post-pass
bun run test # parallel 4-shard fan-out (memory-adaptive) + serial post-pass
bun test test/markdown.test.ts # specific unit test
# Pre-push gate (19+ parallel checks + typecheck)
@@ -129,10 +129,13 @@ historical sweep including the trailing-newline and exports-count checks.
### Writing tests that survive the parallel loop
`bun run test` shards the unit-test files (1000+) across 8 worker processes.
Files in the same shard share a process, so process-global state leaks between
them. Four lint rules (`scripts/check-test-isolation.sh`, R1–R4) enforce
isolation: no direct `process.env` mutation (use `withEnv()` from
`bun run test` shards 1000+ unit-test files across up to 4 worker processes,
capping total concurrency (shards × intra-shard files) to available memory and
re-running OOM-killed or externally-killed files serially before calling them
failures (see `docs/TESTING.md` for the rescue-pass details and knobs). Files
in the same shard share a process, so process-global state leaks between them.
Four lint rules (`scripts/check-test-isolation.sh`, R1–R4) enforce isolation:
no direct `process.env` mutation (use `withEnv()` from
`test/helpers/with-env.ts`), no `mock.module(...)` outside `*.serial.test.ts`,
and every `new PGLiteEngine(` goes inside the canonical `beforeAll` block with
a paired `afterAll(disconnect)`.
@@ -188,6 +191,15 @@ automatically appears in the CLI, MCP server, and tools-json:
For CLI-only commands (init, upgrade, import, export, files, embed, doctor, sync):
1. Create `src/commands/mycommand.ts`
2. Add the case to `src/cli.ts`
3. Regenerate the flag registry: `bun run build:flag-registry`. The CLI rejects
unknown flags before dispatch; each CLI-only command's legal flag set is
derived from its source into `src/core/cli-flag-registry.generated.ts`.
`test/cli-flag-validation.test.ts` pins registry freshness, drift, and
consumption evidence (a safety flag like `--dry-run` may only be advertised
if the command's code actually reads it), so a stale registry fails the
build. At runtime a missing registry entry fails open — a forgotten regen
never bricks a command. Rerun the regen whenever you add or remove a flag
on an existing command, too.
Parity tests (`test/parity.test.ts`) verify CLI/MCP/tools-json stay in sync.
README.md
+6 -4
View File
@@ -127,13 +127,15 @@ The agent installs GBrain, creates the brain, asks for your API keys, loads the
### Lighter ways in
**Just want a memory for your coding agent — no identity, no repo.** Spin up a local brain and connect it in two commands — zero server, zero token, zero tunnel:
**Just want a memory for your coding agent — no identity, no repo.** Spin up a local brain and connect it in two commands — zero server, zero token, zero tunnel. `--surface verbs` gives your agent the five-verb memory protocol (`recall`, `remember`, `entity`, `synthesize`, `forget` — [MEMORY_VERBS v1](docs/protocol/MEMORY_VERBS_v1.md), frozen + additive-forever) instead of the full tool wall; drop the flag for every operation:
```bash
gbrain init --pglite # 2-second local brain (no Docker)
claude mcp add gbrain -- gbrain serve # or: codex mcp add gbrain -- gbrain serve
gbrain init --pglite # 2-second local brain (no Docker)
claude mcp add gbrain -- gbrain serve --surface verbs # or: codex mcp add gbrain -- gbrain serve --surface verbs
```
If `claude` is not found, install Claude Code first — or use the per-harness blocks in the [protocol doc](docs/protocol/MEMORY_VERBS_v1.md). Heads-up: memories agents save default to brain-wide visibility (every connected agent can recall them); pass `visibility: "private"` for local-only facts.
**Already have a brain on a remote host** (OpenClaw, Hermes, or any `gbrain serve --http`)? Point your laptop agents at it with one command each — `--install` wires it up and smoke-tests the token before handoff:
```bash
@@ -159,7 +161,7 @@ Postgres-at-scale, Supabase, and thin-client setup paths live in [`docs/INSTALL.
### Connect GBrain to your AI client (MCP)
GBrain exposes nearly all of its 100+ operations as MCP tools (stdio and HTTP; a handful of local-only ops stay CLI-side). The specific snippet depends on which client you use:
GBrain exposes nearly all of its 100+ operations as MCP tools (stdio and HTTP; a handful of local-only ops stay CLI-side) — or exactly the five memory verbs with `--surface verbs`. The specific snippet depends on which client you use:
- **[Claude Code](docs/mcp/CLAUDE_CODE.md)** — local: one command, `claude mcp add gbrain -- gbrain serve` (zero server, zero tunnel). Remote with just a bearer token: `gbrain connect https://your-host/mcp --token gbrain_xxx` prints a paste-ready block (or `--install` wires it up and smoke-tests the token).
- **[Codex](docs/mcp/CODEX.md)** — `gbrain connect https://your-host/mcp --token gbrain_xxx --agent codex` (or `--install`). Codex reads the bearer from `$GBRAIN_REMOTE_TOKEN` at runtime, so the token never lands in Codex config.
TODOS.md
+112
View File
@@ -1,5 +1,116 @@
# TODOS
## #2416 follow-ups (query-steering wave)
- [ ] **P2 — MCP-envelope `hint` field for concept-shaped `search` calls.**
**What:** surface the concept→query nudge to remote/MCP agent callers, not
just the CLI. **Why:** MCP agents are the primary misrouting class the
#2416 issue describes; the shipped CLI stderr nudge covers the caller class
*least* at risk. **Context:** the `search` op returns a bare
`SearchResult[]` (`src/core/operations.ts` — both return sites), so a hint
needs an envelope change that ripples into `formatResult`, MCP
serialization, and array-shape tests — deliberately kept out of the atomic
#2416 commit. The pure classifier already exists
(`looksConceptShaped`/`conceptNudge` in `src/core/search/query-intent.ts`);
only the transport is missing. Consider a sibling metadata channel (like
`_meta.metric_glossary`) rather than changing the array shape.
**Depends on:** agreeing an envelope pattern that doesn't break existing
MCP consumers.
## MEMORY_VERBS v1 follow-ups (filed v0.43.0.0 — Cathedral 1)
Deferred from the Cathedral 1 ship (CEO review, EXPANSION mode). Both are
additive to the frozen v1 contract — neither breaks it. See plan + GSTACK
REVIEW REPORT at `~/.claude/plans/system-instruction-you-are-working-agile-iverson.md`
and the scope record at `~/.gstack/projects/garrytan-gbrain/ceo-plans/2026-06-12-memory-verbs-protocol.md`.
- [ ] **P3 — external-implementation certification PROGRAM.** The conformance
TOOLING shipped (`gbrain protocol conformance --target <endpoint>`); the
PROGRAM around it (badges, a registry of conformant implementations, listed
third-party servers) waits for a second implementation to exist. **Why:** the
protocol-not-product thesis only pays off once someone else implements
MEMORY_VERBS; until then a certification program certifies an empty set.
**Where:** new — would build on `src/commands/protocol.ts` conformance output.
- [ ] **P3 — persistent open-threads model for the entity card.** v1 derives
`entity.open_threads` from active commitment-kind facts + recent timeline
entries (best-effort, possibly empty). A richer model (a real threads table:
conversation id, opened/closed state, last activity) would make open-threads
authoritative. **Why:** the card's open-threads field is the weakest signal
in v1; a first-class threads store would make it load-bearing. **Where:**
`src/core/verbs/entity-card.ts` open-threads assembly + a new schema table
(additive — the card field already exists, so this is a quality upgrade, not
a contract change).
- [ ] **P2 — `recall` filter composition vs the spec (found by the v0.43.0.0
cross-model doc review).** The handler dispatch is first-match
(`supersessions` > `entity` > `session_id` > `since`), so `since` is
silently ignored when `entity`/`session_id` is supplied, and `limit` has no
server-side cap. Either compose the filters (additive — the spec's "filters
the FACTS arm" wording already reads that way) or spell the precedence out
in `docs/protocol/MEMORY_VERBS_v1.md`. **Where:** the `recall` handler in
`src/core/operations.ts`.
- [ ] **P3 — widen `synthesize`'s `unavailable` mapping.** Only the
missing-key gateway warning maps to the `unavailable` error today; other
no-usable-model failures can surface as `internal` (contract-legal but less
actionable) or, worst case, a stubbed success. Audit the gateway failure
modes and map every model-unusable path to `unavailable` with a fix
suggestion. **Where:** the `synthesize` handler in `src/core/verbs.ts`.
## Fix-wave 1 follow-ups (upgrade-wedge + trust-seam wave, 2026-08)
Deferred from the un-wedge-v121 hotfix wave (eng review + codex outside voice
CLEARED; every item an explicit review decision). Waves 2–6 of the sequence are
planned separately (provider-compat rescue is next; its original 2026-07-24
DeepSeek-deprecation deadline has now PASSED — re-verify each cluster against
master before starting, several fixes landed independently).
- [ ] **P2 — Shared strict `parseFlags` helper as the #2185 end-state (eng
review 2B).** This wave ships the generated known-flags registry +
pre-dispatch validator (parser and registry can drift only until the
freshness guard fires). The structural end-state migrates commands onto one
shared strict parser so parser == registry by construction; mechanical but
touches 60+ command files — its own PR. Where: `src/commands/*.ts`,
`src/cli.ts`, `scripts/generate-flag-registry.ts` (retires).
- [ ] **P2 — `whoknows` CLI routing (surfaced by the #2035-class sweep).**
`handleCliOnly`'s `whoknows` case (the dedicated CLI renderer with
thin-client routing) is dead code — the command resolves via the
`find_experts` op alias, and adding it to CLI_ONLY trips the alias-collision
guard. Decide the intended surface alongside PR #2509 (whoknows --explain
per-result factor breakdown) and delete whichever lane loses. Where:
`src/cli.ts`, `src/commands/whoknows.ts`, PR #2509.
- [ ] **P3 — #2544 second half: per-put_page `getAllSlugs` full scan.** The
getChunks egress half shipped in this wave (explicit non-vector column
list). The remaining Postgres-egress cost is put_page's per-call
`getAllSlugs` table scan — needs a targeted existence probe or cached slug
set. Where: `src/core/operations.ts` put_page path, both engines.
- [ ] **P3 — #1558 admin-UI register form.** The `/admin/api/register-client`
API now accepts `source` + `federatedRead` (this wave, PR #2016 absorbed);
the admin SPA form fields + `/admin/api/sources` picker are the UI layer.
Where: `src/commands/serve-http.ts` admin SPA blob.
- [ ] **P3 — jsonb-integrity surfaces: batch + share (ship-review follow-up).**
doctor's jsonbIntegrityCheck runs 2 queries per target (16 round-trips) and
duplicates the TARGETS table with repair-jsonb (already drifted once on the
jsonPayloadOnly predicate before being mirrored by hand). Batch the counts
into one UNION ALL query and extract a shared targets constant
(src/core/jsonb-integrity-targets.ts) consumed by both. Where:
`src/commands/doctor.ts` jsonbIntegrityCheck, `src/commands/repair-jsonb.ts`.
- [ ] **P3 — register-client HTTP-level e2e (ship-review follow-up).** The
source/federatedRead lane is covered by unit normalizers + a structural
route pin; a DATABASE_URL-gated serve-http e2e (register with bindings →
assert stored client via /admin/api/agents; invalid source → 400
invalid_source) closes the wire-level gap. Where:
`test/e2e/serve-http-oauth.test.ts`.
- [ ] **P3 — get_chunks `__all__` sentinel narrows to 'default' (red-team,
Wave 3 territory).** `sourceScopeOpts` returns `{}` for a trusted local
`--source __all__` caller (documented "spans the brain"), but both engines'
getChunks map empty scope to the 'default' floor — the one read op where
`{}` is reinterpreted. Fold into the Wave 3 source-federation cluster's
`__all__` work (an explicit unscoped signal in the engine signature, or
handler-side expansion for trusted callers). Where: `src/core/operations.ts`
get_chunks, both engines' getChunks.
- [ ] **P3 — #2536 wedged-migration diagnostics.** The v121 wedge aborted
initSchema BEFORE runMigrations, so the wedged-migration diagnostics row was
never written — operators got a bare SQL error with no remediation hint.
Write the diagnostics row (or a stderr remediation block) from the blob-replay
catch path too. Where: `src/core/migrate.ts`, `src/commands/apply-migrations.ts`.
## WAL-repair wave follow-ups (#223/#1670/#2575)
- [ ] **P2 — gate auto-repair on an unclean-shutdown marker (adversarial F7).** The classifier
@@ -78,6 +189,7 @@ voice CLEARED). None block the wave.
clause (`src/mcp/http-transport.ts` validateToken). Apply the same pattern —
one fewer write per request on the `serve --http` hot path.
Where: `src/core/oauth-provider.ts`.
## v0.42.67.0 follow-ups (Windows build tooling)
Filed as follow-ups from v0.42.67.0 (`.gitattributes` LF pin for `*.sh` +
VERSION
+1 -1
View File
@@ -1 +1 @@
0.42.75.0
0.43.0.0
docs/INSTALL.md
+4 -3
View File
@@ -67,16 +67,17 @@ gbrain autopilot --install # background daemon for nightly enrichment
**Wire this same local brain into your coding agent** — zero server, zero token:
```bash
claude mcp add gbrain -- gbrain serve # Claude Code
codex mcp add gbrain -- gbrain serve # Codex
claude mcp add gbrain -- gbrain serve --surface verbs # Claude Code
codex mcp add gbrain -- gbrain serve --surface verbs # Codex
```
The agent spawns `gbrain serve` as a stdio subprocess against your local brain. Full walkthrough (both this local path and connecting to a remote brain), plus the brain-first protocol to paste into `CLAUDE.md` / `AGENTS.md`: **[Give your coding agent a memory](tutorials/connect-coding-agent.md)**.
The agent spawns `gbrain serve` as a stdio subprocess against your local brain. `--surface verbs` gives the agent the five-verb memory protocol (`recall`, `remember`, `entity`, `synthesize`, `forget` — [MEMORY_VERBS v1](protocol/MEMORY_VERBS_v1.md)) instead of the full tool catalog; drop the flag (default `full`) for every operation. Full walkthrough (both this local path and connecting to a remote brain), plus the brain-first protocol to paste into `CLAUDE.md` / `AGENTS.md`: **[Give your coding agent a memory](tutorials/connect-coding-agent.md)**.
## 3. MCP server (any MCP client)
```bash
gbrain serve # stdio MCP (Claude Desktop / Code / Cursor)
gbrain serve --surface verbs # stdio MCP, just the 5 memory verbs (quickstart)
gbrain serve --http # HTTP MCP with OAuth 2.1 + admin dashboard
```
docs/TESTING.md
+4 -4
View File
@@ -11,7 +11,7 @@ Seven test command tiers, each with a clear scope:
| Command | What it runs | Wallclock | When to use |
|---|---|---|---|
| `bun run test` | Parallel unit-test fast loop. Sharded fan-out via `scripts/run-unit-parallel.sh` (default 4 shards — CPU-detected, clamped to a max of 8, and defaulted down to 4 when there's no `--shards`/`SHARDS` override; 4 matches CI's fan-out and avoids PGLite WASM-init contention), then a serial pass over `*.serial.test.ts`. Excludes `*.slow.test.ts` and `test/e2e/*`. No pre-checks, no typecheck. | a few minutes on a Mac dev box | Inner edit loop. Default. |
| `bun run test` | Parallel unit-test fast loop. Sharded fan-out via `scripts/run-unit-parallel.sh` (default 4 shards — CPU-detected, clamped to a max of 8; 4 matches CI's fan-out and avoids PGLite WASM-init contention), then a serial pass over `*.serial.test.ts`. Excludes `*.slow.test.ts` and `test/e2e/*`. No pre-checks, no typecheck. Memory-safe by default: total concurrency (shards × intra-shard files) is capped to available memory at `GBRAIN_TEST_MEM_PER_FILE_MB` (default 1536 — a PGLite WASM instance) per concurrent file, and two phantom-failure classes are automatically re-run serially (the rescue pass): failures carrying the WASM out-of-memory signature, and shards killed externally (SIGTERM/SIGKILL well before the shard timeout — sibling workspaces' process cleanup, memory jetsam). Phantoms pass serially and the run goes green with an `oom_rescued` note; real failures fail again serially and stay red. Knobs: `GBRAIN_TEST_NO_MEM_ADAPT=1`, `GBRAIN_TEST_NO_OOM_FALLBACK=1`, `GBRAIN_TEST_MAX_CONCURRENCY` (intra-shard, default 4), `GBRAIN_TEST_SHARD_TIMEOUT` / `GBRAIN_TEST_SHARD_KILL_AFTER`, plus `--shards N` / `--max-concurrency N` / `--dry-run` script args. | a few minutes on a Mac dev box | Inner edit loop. Default. |
| `bun run verify` | CI's authoritative pre-test gate set, fanned out in parallel by `scripts/run-verify-parallel.sh`: the full `check:*` battery (privacy, jsonb, progress, source-id, test-isolation, wasm, …) plus `bun run typecheck`. The `CHECKS` array in that script is the single source of truth — CI literally calls `bun run verify` in a dedicated job. | ~16s (parallel; typecheck dominates) | Before pushing; before `/ship`. |
| `bun run test:full` | `verify && bun run test && bun run test:slow && [smart e2e]`. The local equivalent of "everything CI runs." Smart e2e: runs e2e only when `DATABASE_URL` is set; else loud skip notice to stderr. | ~3-5min depending on slow + e2e | Pre-merge sanity, before opening a PR. |
| `bun run test:slow` | Just the `*.slow.test.ts` set (intentional cold-path correctness checks). | seconds-to-minutes | When touching slow-path code. |
@@ -50,7 +50,7 @@ there even though they pass on Linux and macOS.
- **CI matrix** (`.github/workflows/test.yml`) runs `scripts/test-shard.sh` across 10 matrix shards partitioned by weight-aware LPT bin-packing (`scripts/sharding.ts`) and INCLUDES `*.slow.test.ts` (the two outlier slow files run as dedicated jobs alongside the matrix). CI EXCLUDES `*.serial.test.ts` from the shards and runs them in a dedicated job via `bun run test:serial`, one bun process per file — keeping serial files out of the shard processes is what preserves the `mock.module` quarantine (a top-level mock in one file leaks into every other file sharing its process). `bun run verify` gets its own job too. CI is the ground truth for "did everything pass."
- **Local fast loop** (`scripts/run-unit-shard.sh` via the parallel wrapper) uses round-robin-by-index sharding and EXCLUDES `*.slow.test.ts` AND `*.serial.test.ts`. Local trades coverage for inner-loop speed; CI catches what local skips.
This divergence is intentional. Don't try to make them equal — the two scripts deliberately solve different problems. The regression test at `test/scripts/run-unit-shard.test.ts` pins what the local fast loop should and shouldn't include.
This divergence is intentional. Don't try to make them equal — the two scripts deliberately solve different problems. The regression test at `test/scripts/run-unit-shard.test.ts` pins what the local fast loop should and shouldn't include; `test/scripts/run-unit-parallel.test.ts` pins the wrapper's memory-adaptive concurrency and the OOM/external-kill serial rescue pass.
### Failure-first logging
@@ -61,7 +61,7 @@ When `bun run test` finds any failure, the wrapper:
3. Writes a one-line-per-shard summary to `.context/test-summary.txt` (`shard N/M: pass=X fail=Y skip=Z rc=W`).
4. Exits non-zero. Empty failure log + non-zero exit = infrastructure problem (wedged shard, killed child); the banner says so.
If a shard hits the per-shard `GBRAIN_TEST_SHARD_TIMEOUT` cap (default 2400s — sized so the heaviest count-balanced shard finishes under 4-way contention), the wrapper classifies the kill one of two ways:
If a shard hits the per-shard `GBRAIN_TEST_SHARD_TIMEOUT` cap (default 3000s — sized so the heaviest count-balanced shard finishes under 4-way contention; `GBRAIN_TEST_SHARD_KILL_AFTER` sets the grace after TERM before KILL, default 30s), the wrapper classifies the kill one of two ways:
- **EXIT-HANG → warn-pass.** If the shard's log had been silent for ≥300s at kill time AND shows zero `(fail)` markers, the shard finished all its work, leaked a handle, and never exited (a pre-existing, master-reproducible PGLite-adjacent leak — see TODOS.md "unit-shard exit hang"). The wrapper prints a `⚠️ shard N/M: EXIT-HANG ... Treating as pass-with-warning` banner, writes `EXIT-HANG (idle Ns, 0 fails) ... warn-pass` to the summary, and does NOT fail the run. Its pass counts are undercounted (bun never printed its final summary). Bun's per-test `--timeout` turns a genuinely hung TEST into a printed `(fail)` — new output — so this classification cannot mask a hung test; the residual maskable case is a file-level import hang in the very last file, which the banner keeps visible.
- **WEDGED → hard failure.** Anything else (failures present, or the log was still growing) writes `--- shard N: WEDGED after ${SHARD_TIMEOUT}s ---` to the failure log with the last 50 lines of the shard log, marks the run failed, and proceeds with other shards' results.
@@ -70,7 +70,7 @@ Triage rule: a `warn-pass` EXIT-HANG line in `.context/test-summary.txt` is NOT
### File taxonomy
- `*.test.ts` → fast loop (parallel sharded fan-out, default 4 shards).
- `*.test.ts` → fast loop (parallel up-to-4-shard fan-out, memory-adaptive).
- `*.slow.test.ts` → run via `bun run test:slow` only (intentional cold-path tests; would dominate the fast loop's wallclock).
- `*.serial.test.ts` → run via `bun run test:serial` after the parallel pass completes; one bun process per file (`--max-concurrency=1` within a shared process is not enough — the module registry still leaks `mock.module`). Quarantine for tests that share file-wide state and race when run alongside other files in the same `bun test` process. Several dozen files, discovered by the `*.serial.test.ts` glob — no list to maintain. Typical residents: `mock.module(...)` users (top-level mocks leak across files in a shard process, e.g. `test/embed.serial.test.ts`), env-coupled files (e.g. `test/brain-registry.serial.test.ts`), and process-lifecycle suites that assert on `process.exitCode` (e.g. `test/pglite-engine-disconnect.serial.test.ts`). **Do not put the parallelism back on a serial file unless you've fixed the contention root cause** (it just re-introduces the flake).
- `test/e2e/*.test.ts` → real-Postgres E2E. Skipped when `DATABASE_URL` is unset.
docs/architecture/KEY_FILES.md
+15 -6
View File
File diff suppressed because one or more lines are too long
docs/architecture/RETRIEVAL.md
+6 -2
View File
@@ -98,7 +98,11 @@ Promote or reject them via `gbrain extraction-pending` / `gbrain
extraction-review`.
The `search` MCP/CLI op is **cheap-hybrid** (vector + keyword + RRF + pool +
title + alias, expansion off); `query` is the full-control variant. NamedThingBench
title + alias, expansion off); `query` is the full-control variant. Route
concept / landscape / "all-of-X" questions to `query` — expansion recovers
synonym-phrased matches `search` can miss, and a populated `search` result set
is not proof of coverage (both are top-K; exhaustive enumeration belongs to
`list_pages`). NamedThingBench
(`gbrain eval retrieval-quality`) gates these families on every PR. Diagnose a
specific miss with `gbrain search diagnose "<q>" --target <slug>`.
@@ -117,7 +121,7 @@ The classifier is deterministic (no LLM call). Wrong classification degrades gra
For `detail: 'high'` searches, `src/core/search/expansion.ts` runs a Haiku-class LLM call to produce 2-3 query variants. Each variant runs through the full hybrid stack; results merge via RRF. Catches synonym misses without recall loss.
Expansion is opt-in per mode bundle (`tokenmax` on by default; `balanced` + `conservative` off). Default off in the cheap tiers because the LLM call adds ~$0.001/query and ~200ms — real money at scale.
Expansion is opt-in per mode bundle (`tokenmax` on by default; `balanced` + `conservative` off). Default off in the cheap tiers because the LLM call adds ~$0.001/query and ~200ms — real money at scale. The `query` op is the exception: it defaults `expand: true` per call (pass `expand: false` to opt out) — expansion-by-default is what makes it the concept/landscape verb.
## Putting it together
docs/architecture/thin-client.md
+5 -4
View File
@@ -52,10 +52,11 @@ carries the routing-seam picture):
thin-client routing branches. These commands bypass the operation-layer
dispatch in cli.ts (call `engine.foo()` directly), so each gets its own
`if (isThinClient(cfg)) { callRemoteTool(...) }` branch that maps CLI flags
to op params. `think` is a special case: the server's `think` op
intentionally disables `--save`/`--take` for remote callers (the
`safeSave`/`safeTake` trust-boundary gate in the `think` handler in
`operations.ts`); thin-client `think` warns loudly when those flags are set.
to op params. `think` is a special case: the server's `think` op is
read-scoped for OAuth/MCP and intentionally disables `--save`/`--take` for
remote callers (the `safeSave`/`safeTake` trust-boundary gate in the `think`
handler in `operations.ts`); thin-client `think` warns loudly when those
flags are set.
Cross-modal search files (image query, SSRF-guarded image loading, spend
tracking, multimodal reindex) are indexed per-file in
docs/guides/search-modes.md
+25 -16
View File
@@ -109,36 +109,44 @@ on user_asks_about(topic):
# Returns the FULL page -- compiled truth + timeline
elif topic.is_exact_name or topic.is_keyword:
# Keyword search -- fast, no embeddings needed, day-one ready
# MODE 1: Cheap-hybrid search -- vector + keyword + RRF, NO LLM
# expansion. Embeds the query when embeddings are configured; the
# keyword arm still works day-one without them (keyword-only is
# also available via the search.mcp_keyword_only opt-out).
results = gbrain search "{name_or_keyword}"
# e.g., "Find anything about Series A" -> gbrain search "Series A"
# Returns CHUNKS, not full pages
# IMPORTANT: keyword search returns chunks
# IMPORTANT: search returns chunks
# If the chunk confirms relevance, THEN load the full page:
if chunk.confirms_relevance:
full_page = gbrain get <slug_from_chunk>
elif topic.is_semantic_question:
# Hybrid search -- semantic + keyword, needs embeddings
elif topic.is_semantic_question or topic.is_concept_or_landscape:
# MODE 2: Full hybrid -- adds multi-query LLM expansion on top of
# vector + keyword + RRF. Owns concept / landscape / "all-of-X"
# questions: expansion recovers synonym- and outcome-phrased
# matches a single embedding misses. Costs one LLM expansion call
# per query -- worth it for these question shapes.
results = gbrain query "{natural language question}"
# e.g., "Who do I know at fintech companies?" -> gbrain query "fintech contacts"
# Returns ranked chunks via vector + keyword + RRF
# e.g., "all the companies doing offshore wind" -> gbrain query "..."
# Returns ranked chunks via vector + keyword + expansion + RRF
# Same rule: chunks first, then get full page if needed
if chunk.confirms_relevance:
full_page = gbrain get <slug_from_chunk>
# Quick reference:
# | Verb | Command | Needs Embeddings | Speed | Best For |
# |---------|----------------------|------------------|---------|---------------------------------|
# | Keyword | gbrain search "term" | No | Fastest | Known names, exact matches |
# | Hybrid | gbrain query "..." | Yes | Fast | Semantic questions, fuzzy match |
# | Direct | gbrain get <slug> | No | Instant | When you know the slug |
# | Mode | Command | Needs Embeddings | Speed | Best For |
# |-------------|----------------------|------------------|---------|-------------------------------------------|
# | Cheap-hybrid| gbrain search "term" | Uses if present | Fastest | Known names, exact tokens |
# | Full hybrid | gbrain query "..." | Yes | Fast | Concept / landscape / "all-of-X", synonyms |
# | Direct | gbrain get <slug> | No | Instant | When you know the slug |
# Progression over time:
# Day 1: keyword search (works without embeddings)
# After first embed: hybrid search unlocked
# Day 1: search (keyword arm works without embeddings)
# After first embed: vector arm + full hybrid (query) unlocked
# Once you know slugs: direct get for speed
# Precedence for conflicting information within a page:
@@ -151,17 +159,18 @@ on user_asks_about(topic):
### Tricky Spots
1. **Search returns chunks, not full pages.** After `gbrain search` or `gbrain query`, you get excerpts. Always run `gbrain get <slug>` to load the full page when the chunk confirms relevance. Don't answer questions from chunks alone when the full context matters.
2. **Keyword search works without embeddings.** On day one before any embedding run, `gbrain search` still works. Don't tell the user "search isn't available yet" -- keyword search is always available.
3. **Don't use hybrid search for known names.** `gbrain query "Alice Example"` wastes embedding compute. Use `gbrain search "Alice Example"` or better yet `gbrain get alice-example` if you know the slug.
2. **Search works without embeddings.** On day one before any embedding run, `gbrain search` still works (the keyword arm carries it; the vector arm joins once embeddings exist). Don't tell the user "search isn't available yet" -- search is always available.
3. **Don't use full hybrid for known names.** `gbrain query "Alice Example"` wastes an LLM expansion call. Use `gbrain search "Alice Example"` or better yet `gbrain get alice-example` if you know the slug.
4. **Token budget awareness.** A full page via `gbrain get` can be large. Read the search chunks first to confirm relevance before pulling the full page. "Did anyone mention the Series A?" -- search results (chunks) are probably enough. "Tell me everything about Alice" -- get the full page.
5. **Hybrid search needs embeddings to have been run.** If `gbrain query` returns nothing but `gbrain search` finds results, the embeddings haven't been generated yet. Run the embedding pipeline first.
5. **Full hybrid needs embeddings to have been run.** If `gbrain query` returns nothing but `gbrain search` finds results, the embeddings haven't been generated yet. Run the embedding pipeline first.
6. **A populated `gbrain search` result set is not proof you found everything.** Search runs without query expansion, so synonym- and outcome-phrased matches can be missed even when it returns plenty of hits. For "find every / all / the landscape of" questions, use `gbrain query`; for literal exhaustive enumeration ("list every page of type X"), use `list_pages` pagination. A nonzero count is not a completeness signal.
### How to Verify
1. Run `gbrain search "Alice"` -- confirm it returns chunks with matching text and slug references.
2. Run `gbrain query "who works at fintech companies"` -- confirm it returns semantically relevant results (not just keyword matches on "fintech").
3. Run `gbrain get alice-example` -- confirm it returns the full page with compiled truth and timeline.
4. Compare: search for the same entity using all three verbs. Keyword should be fastest, hybrid should surface conceptual matches, direct should return the complete page.
4. Compare: search for the same entity using all three modes. Keyword should be fastest, hybrid should surface conceptual matches, direct should return the complete page.
5. After a search returns a chunk, run `gbrain get` on the slug from that chunk. Confirm the full page contains more context than the chunk alone.
6. Run `gbrain search modes` -- confirm the active mode bundle and any per-key overrides are what you expect.
docs/mcp/CLAUDE_CODE.md
+15 -5
View File
@@ -12,12 +12,18 @@
## Option 1: Local (recommended, zero server needed)
```bash
claude mcp add gbrain -- gbrain serve
claude mcp add gbrain -- gbrain serve --surface verbs
```
That's it. Claude Code spawns `gbrain serve` as a stdio subprocess. No server, no
tunnel, no token needed. Works with both PGLite and Supabase engines.
`--surface verbs` exposes the five-verb memory protocol (`recall`, `remember`,
`entity`, `synthesize`, `forget` — [MEMORY_VERBS v1](../protocol/MEMORY_VERBS_v1.md)),
the surface built for agents and quickstarts. Drop the flag for the full
operation catalog (`get_page`, `put_page`, `search`, graph ops, …) — `full` is
the default and what existing installs already run.
## Option 2: Remote, one command (fastest from a bearer token)
If GBrain is running somewhere as an HTTP server (`gbrain serve --http`, see the
@@ -81,10 +87,14 @@ search for [any topic in your brain]
You should see results from your GBrain knowledge base.
> **`list_skills` returns nothing?** Skill discovery is gated by `mcp.publish_skills`
> on the host — enable it with `gbrain config set mcp.publish_skills true`. The core
> tools (search, query, get_page, put_page, think, find_experts) work regardless;
> `capture` is CLI-only, so agents write over MCP with `put_page`. Why brains differ
> on the default: [tutorial A1](../tutorials/connect-coding-agent.md#a1-on-the-host-serve-over-http).
> on the host. New brains from `gbrain init` default it ON; brains upgraded from an
> older release stay OFF until you opt in. Enable it on the host with
> `gbrain config set mcp.publish_skills true`. Skill discovery and the core tools
> named here (search, query, get_page, put_page, think, find_experts) are
> full-surface — on `--surface verbs` the agent sees only the five memory verbs,
> and `list_skills` isn't on the surface at all. Note: `capture` is a
> CLI-only command, not an MCP tool — the agent writes over MCP with `put_page`.
> Why brains differ on the default: [tutorial A1](../tutorials/connect-coding-agent.md#a1-on-the-host-serve-over-http).
## Remove
docs/mcp/CODEX.md
+3 -1
View File
@@ -72,4 +72,6 @@ codex mcp remove gbrain
- The token is a long-lived, full-access secret. Keep `GBRAIN_REMOTE_TOKEN` out of
version control and prefer a scoped token if your host supports one.
- Local stdio also works if you run the brain on the same machine:
`codex mcp add gbrain -- gbrain serve`.
`codex mcp add gbrain -- gbrain serve --surface verbs` — the five-verb memory
protocol ([MEMORY_VERBS v1](../protocol/MEMORY_VERBS_v1.md)); drop the flag
for the full operation catalog.
docs/mcp/DEPLOY.md
+10 -4
View File
@@ -4,9 +4,10 @@
> PKCE, refresh rotation, optional DCR), an embedded React admin dashboard at
> `/admin`, scoped operations, and a live SSE activity feed. Legacy bearer
> tokens still work — `verifyAccessToken` falls back to the `access_tokens`
> table and grandfathers tokens to `read+write+admin`. Both the OAuth surface
> and the bearer fallback work on both engines (PGLite and Postgres). See
> [SECURITY.md](../../SECURITY.md) for env vars and tunable defaults.
> table and grandfathers tokens to `read+write+admin`. Both the legacy fallback
> and the OAuth tables work on PGLite and Postgres (both engine schemas carry
> `access_tokens`). See [SECURITY.md](../../SECURITY.md) for env vars and
> tunable defaults.
Access your brain from any device, any AI client. GBrain ships two transports:
`gbrain serve` (stdio) for local agents, and `gbrain serve --http` for remote
@@ -17,11 +18,16 @@ clients over OAuth 2.1.
### Local stdio (zero setup)
```bash
gbrain serve
gbrain serve # full operation catalog (default)
gbrain serve --surface verbs # just the 5 memory verbs (quickstart surface)
```
Works with Claude Code, Cursor, Windsurf, and any MCP client that supports stdio.
No server, no tunnel, no token needed. Works on both PGLite and Postgres engines.
`--surface verbs` exposes exactly the five-verb memory protocol (`recall`,
`remember`, `entity`, `synthesize`, `forget` —
[MEMORY_VERBS v1](../protocol/MEMORY_VERBS_v1.md)) instead of the full catalog;
omit the flag (default `full`) for every operation.
### Remote over OAuth 2.1 (recommended)
docs/protocol/MEMORY_VERBS_v1.md
+260
View File
@@ -0,0 +1,260 @@
# MEMORY_VERBS v1 — the memory wire protocol
GBrain's frozen five-verb memory interface over MCP: `recall`, `remember`,
`entity`, `synthesize`, `forget`. The contract every harness can rely on the
way every Postgres client relies on the wire protocol — and the contract any
OTHER memory server can implement and certify against
(`gbrain protocol conformance --target <endpoint>`).
```
agent (any MCP harness)
│ remember("picked Stripe over Adyen", provenance: "chat 2026-06-11")
▼
five verbs ── recall ── remember ── entity ── synthesize ── forget
│ self-describing envelopes: protocol_version, evidence, provenance,
│ budget meta, cost block, enumerated error codes + a populated fix
▼
your brain (reference implementation: gbrain; any conformant server)
```
**Machine-readable spec:** `gbrain protocol --json` emits the input schemas
from the live operation definitions plus the response-shape registry — doc and
code structurally cannot drift; conformance validates live responses against
the same registry.
## Versioning policy (the point of the freeze)
- Every field NAME and its SEMANTICS in v1 are frozen forever — never removed,
renamed, or re-typed; meanings never change.
- New OPTIONAL params and new OPTIONAL response fields may be added at any
time (additive-forever). A conformant CLIENT must ignore unknown fields; a
conformant SERVER must never reject unknown-to-v1 additions it itself ships.
- `protocol_version` (integer, starts at `1`) rides every verb response and
every verb error. It increments ONLY on a breaking change, which by policy
requires a new `MEMORY_VERBS_v2` document — expected never.
- Conformance pins a minimum version; certification asserts shape, enum
validity, contract behavior, and round-trips — never ranking quality (that
is BrainBench's job).
- Enum values are part of the contract. Where an enum's DERIVATION is
implementation-defined (noted per field), implementations may improve the
derivation without a version bump; the values and their meanings stay fixed.
## Install (the 4-command quickstart)
```bash
gbrain init --pglite # 2-second local brain
claude mcp add gbrain -- gbrain serve --surface verbs # the five-verb surface
gbrain remember "I prefer dark mode in every editor" --provenance demo --entity people/me
gbrain recall --entity people/me # …now ask your agent in a NEW session
```
> Memories agents save are readable by every agent connected to this brain;
> pass `visibility: "private"` for local-CLI-only facts.
If `claude` is not found: install Claude Code first, or use a block below.
**Codex**
```bash
codex mcp add gbrain -- gbrain serve --surface verbs
```
**OpenClaw / any stdio MCP host** — register the server command
`gbrain serve --surface verbs`. Remote brains: `gbrain serve --http` on the
host, then `gbrain connect https://host/mcp --token gbrain_xxx --install` on
each client.
**Surface modes:** `--surface verbs` exposes EXACTLY the five verbs —
advertised list AND dispatch are filtered fail-closed (a hidden op returns
`unknown_tool` even when called by name). `--surface full` (the default)
exposes every operation, verbs included. Why default full: verbs is for
agents and quickstarts; full preserves existing advanced tooling. Persist a
default with `gbrain config set mcp_surface verbs`.
## The verbs
### recall(query?, entity?, budget_tokens?, since?, session_id?, limit?, …) — read
Retrieve saved facts and (with `query`) budget-packed page snippets.
- `entity` scopes the FACTS arm; `query` runs the hybrid-search arm over
pages; both present ⇒ both arms run.
- `since`: ISO 8601 date/datetime — filters the FACTS arm only in v1. (The
reference implementation also accepts relative phrases like `"8 hours ago"`
as a convenience; only ISO 8601 is part of the frozen contract.)
- `limit` is a PER-ARM cap (facts and search results each).
- `budget_tokens`: SERVER-side packing — facts pack first (limit-capped
one-liners, so search-arm starvation is bounded), search results take the
remainder. The estimator is char/4 (±10–15%); `budget_used` reports packed
tokens, `dropped_count` what didn't fit. Never advisory, never client-side.
- No embedding provider configured? The search arm degrades to keyword-only
and the response notes `search_degraded` — never an error.
Response — an additive SUPERSET of the pre-v1 facts envelope on EVERY call
(all legacy fields unchanged; JSON consumers ignore additions):
| field | type | semantics |
|---|---|---|
| `protocol_version` | int | always present (every verb, every call) |
| `facts[]` | array | legacy fact fields unchanged, PLUS per fact: `fact_id` (opaque STRING — the value `forget` accepts; the legacy numeric `id` stays for pre-v1 consumers) and `provenance` (the stored source attribution) |
| `total` | int | count of facts returned |
| `results[]` | array | search arm only: `slug`, `title`, `chunk`, `evidence`, `create_safety`, `provenance` (origin page slug) |
| `search_degraded` | string? | present when keyword-only fallback fired |
| `budget_tokens` / `budget_used` / `dropped_count` | int? | present when `budget_tokens` was passed |
**evidence** (enum, zero-LLM heuristic): `alias_hit` \| `exact_title_match` \|
`high_vector_match` \| `keyword_exact` \| `weak_semantic` — why each result
matched. **create_safety** (enum): `exists` (a page for this already exists)
\| `probable` (likely exists; check before creating) \| `unknown` (no
signal). The derivation of both is implementation-defined and may improve;
the values are frozen.
### remember(fact, provenance, ttl?, entity?, kind?, visibility?) — write
Save ONE fact with mandatory attribution.
- `provenance` (REQUIRED, free text ≤500 chars, stored verbatim): e.g.
`"conversation 2026-06-12"`, `"user said in chat"`, `"import: notes.md"`.
Empty ⇒ `provenance_required` error with a fix.
- `entity`: set whenever the fact is about a specific person/company/project —
entity-scoped recall will not find unattributed facts.
- `ttl`: duration shorthand (`"30d"`, `"12h"`, `"45m"`) or an absolute ISO 8601
timestamp. ISO-8601 DURATIONS (`P30D`) are rejected with a self-correcting
suggestion. Omitted ⇒ never expires.
- `kind`: `event` \| `preference` \| `commitment` \| `belief` \| `fact`
(default).
- `visibility`: `world` (DEFAULT — readable by every agent connected to this
brain; required for the remote remember→recall round-trip) \| `private`
(local CLI reads only). The init quickstart carries the consent line.
Response: `{ id, status, status_text, entity_slug, valid_until,
protocol_version }` (+ `degraded_dedup: true` when no embedding provider —
near-duplicates may insert; dedup and supersession ride embedding similarity).
- `id` — opaque STRING (gbrain serializes integers; another implementation may
use UUIDs). On `status: "duplicate"` it is the EXISTING fact's id.
- `status` — `inserted` \| `duplicate` \| `superseded`. **Branch on `status`,
never on `status_text`** (the human rendering). Supersession is
implementation-defined; the reference rule: same entity + same kind +
similarity above the dedup threshold + different text = the new fact
supersedes the old ("X at acme-example" → "X left acme-example").
- Omitted optional inputs echo as `null`, never absent.
### entity(name) — read, zero LLM, p99 < 100ms
One known person/company/project card. NEVER errors on a miss.
Resolution (frozen precedence): alias > exact title > slug/slug-suffix; ties
break on most-recently-touched. Multi-hit ⇒ best match's card + runners-up in
`suggestions`. Miss ⇒ `found: false` + keyword near-misses with
`create_safety` hints.
Response: `{ protocol_version, found, latency_ms, card?, suggestions? }`.
Card: `{ entity{slug,title,type}, aka[], summary, last_touched{updated_at,
last_retrieved_at, last_timeline_date}, open_threads[], edges[],
backlink_count, active_fact_count }`.
- `summary` passes the same privacy fences as `get_page` (takes + private
facts stripped); remote callers never see private facts in the card.
- `open_threads` (best-effort in v1): active commitment-kind facts + timeline
entries from the last 90 days, capped at 3.
- `edges`: top ~10 typed edges, mentions excluded, out-edges first.
- The p99 < 100ms promise is op-layer latency (transport excluded), CI-gated
on a 20K-page corpus. 200K validation recipe below.
### synthesize(question, since?, until?) — read, EXPENSIVE
`[EXPENSIVE / SLOW — makes LLM calls, seconds-to-minutes latency, costs
money]` — the deliberately-priced slow verb. Prefer `recall`/`entity` for
lookups; use synthesize only when the answer requires combining evidence
across pages.
Response: `{ answer, sources[], gaps[], cost{model, input_tokens,
output_tokens, usd_estimate}, protocol_version }`.
- The `cost` block is a BEST-EFFORT AGGREGATE (retries/multi-call flows sum;
cache hits may undercount; token fields are `null` when a provider returns
no accounting). Honest signal, not an invoice.
- No LLM configured ⇒ the protocol error `unavailable` with a fix — never a
fake answer.
### forget(id, reason?) — write
Expire a fact by its opaque string id (from `remember` or
`recall.facts[].fact_id` — never a page slug). Idempotent: re-forgetting an
already-expired fact returns `expired: false` (success); unknown id ⇒
`not_found`. Facts are expired with an audit trail, never deleted.
Response: `{ id, expired, reason, protocol_version }`.
## Error contract (uniform across all five verbs)
```json
{ "error": "<code>", "message": "...", "suggestion": "problem + cause + fix",
"detail": "freeform specifics", "protocol_version": 1 }
```
Codes (coarse on purpose — codes are for branching; `detail` carries the
story): `invalid_params`, `provenance_required`, `not_found`, `scope_denied`,
`unavailable` (a required dependency cannot serve: no API key, gateway down,
model refusal — configure/retry, not a server bug), `budget_unsatisfiable`
(RESERVED — schema-listed, never returned in v1), `internal`.
Every verb error carries a POPULATED `suggestion`. Specific cases: `recall` on
an empty brain returns empty arrays (success, not an error); auth/scope
failures fail closed via the standard dispatch.
## Trust boundary
Verbs are ordinary operations: they inherit fail-closed `remote` semantics,
OAuth scope enforcement (`remember`/`forget` are write-scope), and per-source
isolation on every read. Remote callers see `visibility = world` facts only.
## Conformance + certification
```bash
gbrain protocol conformance # self-certify (stdio)
gbrain protocol conformance --target http://localhost:3131/mcp --token gbrain_xxx
gbrain protocol conformance --target "bun run src/cli.ts serve"
gbrain protocol conformance --synthesize # also live-call synthesize
```
Pass criteria: response SHAPE (required fields, enum validity), CONTRACT
BEHAVIOR (provenance rejected when empty; budget arithmetic consistent;
entity miss ⇒ `found:false`, not an error; private facts absent from remote
cards; idempotent forget), and ROUND-TRIP (remember → recall by entity — a
plain indexed read, deterministic). It does NOT judge ranking quality.
Entity-card cases need a seedable page (`put_page`); against verbs-only
targets they skip honestly. `--synthesize` is cost-gated: with no LLM key it
asserts the clean `unavailable` error (what CI does); with a key it spends
real tokens.
Conformance is a LIVE test that WRITES: it seeds a marker-suffixed synthetic
entity page (`people/conformance-<marker>`, when the target exposes
`put_page`) and writes/expires facts through `remember`/`forget`. Point it at
write-capable credentials and a brain you're comfortable leaving those
synthetic artifacts in — they're marker-named for easy cleanup, not
auto-deleted. The fixture set ships as data
(`test/fixtures/memory-verbs/cases.json`) and seeds BrainBench's
protocol-compliance arm. gbrain's CI certifies its own stdio + HTTP
transports; external certification is best-effort tooling until a second
implementation exists.
## Observability (local only)
Every verb call appends one line to
`~/.gbrain/integrations/memory-verbs/usage.jsonl` — **local JSONL only, never
uploaded**, stats-only (lock-free rotation may drop lines; POSIX O_APPEND
line-atomic, best-effort on Windows). `gbrain protocol stats [--days N]`
aggregates per-verb calls, error rate, latency, budget drops, entity hit rate,
and the measured TTHW (install → first verb call, from the
`protocol_installed_at` stamp). `gbrain doctor` carries a
`memory_verbs_usage` health line.
## 200K-page latency validation (manual recipe)
CI gates entity() p99 < 100ms on a 20K-page corpus
(`test/entity-card-perf.slow.test.ts`). To validate at 200K, edit the
constants at the top of that file (`PAGES = 200_000`, `LINKS = 1_000_000`,
`ALIASES = 300_000`, `FACTS = 400_000`) and run
`bun test test/entity-card-perf.slow.test.ts --timeout=1800000` — seeding
dominates (~minutes); the measured calls report p50/p99 + the ratio guard.
docs/tutorials/connect-coding-agent.md
+28 -14
View File
@@ -147,15 +147,23 @@ generate while working, and is genuinely useful by day two.
```bash
# Claude Code
claude mcp add gbrain -- gbrain serve
claude mcp add gbrain -- gbrain serve --surface verbs
# Codex
codex mcp add gbrain -- gbrain serve
codex mcp add gbrain -- gbrain serve --surface verbs
```
That's the whole wire-up. No token, no URL, no tunnel. The agent spawns
`gbrain serve` as a stdio subprocess and talks to your local brain directly.
`--surface verbs` exposes exactly the five-verb memory protocol
(`recall`, `remember`, `entity`, `synthesize`, `forget` —
[MEMORY_VERBS v1](../protocol/MEMORY_VERBS_v1.md), frozen + additive-forever)
instead of the full operation catalog, so the agent sees a tight, stable surface
instead of a 110-tool wall. Drop the flag (or pass `--surface full`) for every
operation. The default when the flag is omitted is `full`, so existing wire-ups
are unchanged.
### B4. Verify
In the agent: *"search my brain for PGLite"* (or whatever you just captured). You
@@ -180,14 +188,19 @@ on the patterns.
You have a knowledge brain connected over MCP. Before answering any question
about people, companies, decisions, projects, or past context:
1. **Search first.** Call `search` (or `query` for a synthesized answer) against
the brain BEFORE answering from memory or asking me. If the brain has the
answer, use it. Never ask "who is X?" or "what did we decide about Y?" before
searching — the brain probably already knows.
1. **Brain first — route by the shape of the question.** Exact names or known
tokens → `search` (cheap hybrid, no expansion). Concept, landscape, or
"all the X that do Y" questions → `query` FIRST — it recovers synonym
phrasings `search` misses, and a populated `search` result set is not proof
of coverage. On the five-verb surface the same split is `recall` (retrieve)
vs `synthesize` (reasoned answer). Check the brain BEFORE answering from
memory or asking me. Never ask "who is X?" or "what did we decide about Y?"
before checking — the brain probably already knows.
2. **Write back.** When I make a decision, mention a new person/company, or land
on an idea worth keeping, write it to the brain with `put_page` (entity pages
under people/, companies/; decisions under decisions/ or notes/). One insight,
one page, linked.
on an idea worth keeping, write it to the brain: `remember` on the five-verb
surface (one fact, with provenance), or `put_page` on the full surface
(entity pages under people/, companies/; decisions under decisions/ or
notes/). One insight, one page, linked.
3. **Cite.** When you answer from the brain, name the page you used.
```
@@ -209,13 +222,14 @@ hundreds of linked pages and patterns you didn't know were there.
**3. Briefing from your brain (not from the internet).** *"What do I need to know
before my 2pm with the Acme team?"* pulls your meeting history, the people,
what's still open, what the brain doesn't know yet. The agent does your prep
because it read your context. (`query` gives you the synthesized answer with
citations; this is the example on the [README](../../README.md).)
because it read your context. (`query` — `synthesize` on the five-verb surface —
gives you the synthesized answer with citations; this is the example on the
[README](../../README.md).)
**4. whoknows (expertise routing).** *"Who do I know who's shipped a rate
limiter in Postgres?"* The `find_experts` tool ranks people in your brain by
relevance + recency. Useful the moment your brain has more than a handful of
people in it.
limiter in Postgres?"* The `find_experts` tool (full surface) ranks people in
your brain by relevance + recency. Useful the moment your brain has more than a
handful of people in it.
That's the spine of it. Two commands to connect, one protocol to paste, four
habits to build. Your agent stops being amnesiac.
llms-full.txt
+285 -9
View File
@@ -193,7 +193,7 @@ mount, CEO-class with multiple team brains) and
## Architecture
Contract-first: `src/core/operations.ts` defines 100+ shared operations (including `volunteer_context` — push-based context, see `docs/guides/push-context.md`). CLI and MCP
Contract-first: `src/core/operations.ts` defines 100+ shared operations (including `volunteer_context` — push-based context, see `docs/guides/push-context.md` — and the five frozen MEMORY_VERBS `recall`/`remember`/`entity`/`synthesize`/`forget`, servable alone via `gbrain serve --surface verbs`, see `docs/protocol/MEMORY_VERBS_v1.md`). CLI and MCP
server are both generated from this single source. Engine factory (`src/core/engine-factory.ts`)
dynamically imports the configured engine (`'pglite'` or `'postgres'`). Skills are fat
markdown files (tool-agnostic, work with both CLI and plugin contexts).
@@ -274,6 +274,7 @@ detail on demand.)
| push-based context (volunteer/watch/reflex window) | `docs/guides/push-context.md` |
| schema packs / page types / extraction | `docs/architecture/schema-packs.md`, `type-taxonomy.md`, `lens-packs.md` |
| thin-client / remote MCP / cross-modal | `docs/architecture/thin-client.md` |
| memory verbs / MCP tool surface (`--surface`) / conformance | `docs/protocol/MEMORY_VERBS_v1.md` + the `verbs*`/`surface.ts`/`protocol.ts` entries in `KEY_FILES.md` |
| the CLI surface (commands + flags) | `gbrain --help` / `gbrain --tools-json`, plus the relevant `KEY_FILES.md` entry |
| running or writing tests | `docs/TESTING.md` |
| bulk-command progress wiring | `docs/progress-events.md` |
@@ -1677,13 +1678,15 @@ The agent installs GBrain, creates the brain, asks for your API keys, loads the
### Lighter ways in
**Just want a memory for your coding agent — no identity, no repo.** Spin up a local brain and connect it in two commands — zero server, zero token, zero tunnel:
**Just want a memory for your coding agent — no identity, no repo.** Spin up a local brain and connect it in two commands — zero server, zero token, zero tunnel. `--surface verbs` gives your agent the five-verb memory protocol (`recall`, `remember`, `entity`, `synthesize`, `forget` — [MEMORY_VERBS v1](docs/protocol/MEMORY_VERBS_v1.md), frozen + additive-forever) instead of the full tool wall; drop the flag for every operation:
```bash
gbrain init --pglite # 2-second local brain (no Docker)
claude mcp add gbrain -- gbrain serve # or: codex mcp add gbrain -- gbrain serve
gbrain init --pglite # 2-second local brain (no Docker)
claude mcp add gbrain -- gbrain serve --surface verbs # or: codex mcp add gbrain -- gbrain serve --surface verbs
```
If `claude` is not found, install Claude Code first — or use the per-harness blocks in the [protocol doc](docs/protocol/MEMORY_VERBS_v1.md). Heads-up: memories agents save default to brain-wide visibility (every connected agent can recall them); pass `visibility: "private"` for local-only facts.
**Already have a brain on a remote host** (OpenClaw, Hermes, or any `gbrain serve --http`)? Point your laptop agents at it with one command each — `--install` wires it up and smoke-tests the token before handoff:
```bash
@@ -1709,7 +1712,7 @@ Postgres-at-scale, Supabase, and thin-client setup paths live in [`docs/INSTALL.
### Connect GBrain to your AI client (MCP)
GBrain exposes nearly all of its 100+ operations as MCP tools (stdio and HTTP; a handful of local-only ops stay CLI-side). The specific snippet depends on which client you use:
GBrain exposes nearly all of its 100+ operations as MCP tools (stdio and HTTP; a handful of local-only ops stay CLI-side) — or exactly the five memory verbs with `--surface verbs`. The specific snippet depends on which client you use:
- **[Claude Code](docs/mcp/CLAUDE_CODE.md)** — local: one command, `claude mcp add gbrain -- gbrain serve` (zero server, zero tunnel). Remote with just a bearer token: `gbrain connect https://your-host/mcp --token gbrain_xxx` prints a paste-ready block (or `--install` wires it up and smoke-tests the token).
- **[Codex](docs/mcp/CODEX.md)** — `gbrain connect https://your-host/mcp --token gbrain_xxx --agent codex` (or `--install`). Codex reads the bearer from `$GBRAIN_REMOTE_TOKEN` at runtime, so the token never lands in Codex config.
@@ -3810,9 +3813,10 @@ Source: https://raw.githubusercontent.com/garrytan/gbrain/master/docs/mcp/DEPLOY
> PKCE, refresh rotation, optional DCR), an embedded React admin dashboard at
> `/admin`, scoped operations, and a live SSE activity feed. Legacy bearer
> tokens still work — `verifyAccessToken` falls back to the `access_tokens`
> table and grandfathers tokens to `read+write+admin`. Both the OAuth surface
> and the bearer fallback work on both engines (PGLite and Postgres). See
> [SECURITY.md](../../SECURITY.md) for env vars and tunable defaults.
> table and grandfathers tokens to `read+write+admin`. Both the legacy fallback
> and the OAuth tables work on PGLite and Postgres (both engine schemas carry
> `access_tokens`). See [SECURITY.md](../../SECURITY.md) for env vars and
> tunable defaults.
Access your brain from any device, any AI client. GBrain ships two transports:
`gbrain serve` (stdio) for local agents, and `gbrain serve --http` for remote
@@ -3823,11 +3827,16 @@ clients over OAuth 2.1.
### Local stdio (zero setup)
```bash
gbrain serve
gbrain serve # full operation catalog (default)
gbrain serve --surface verbs # just the 5 memory verbs (quickstart surface)
```
Works with Claude Code, Cursor, Windsurf, and any MCP client that supports stdio.
No server, no tunnel, no token needed. Works on both PGLite and Postgres engines.
`--surface verbs` exposes exactly the five-verb memory protocol (`recall`,
`remember`, `entity`, `synthesize`, `forget` —
[MEMORY_VERBS v1](../protocol/MEMORY_VERBS_v1.md)) instead of the full catalog;
omit the flag (default `full`) for every operation.
### Remote over OAuth 2.1 (recommended)
@@ -4144,6 +4153,273 @@ built-in server is the recommended path.
---
## docs/protocol/MEMORY_VERBS_v1.md
Source: https://raw.githubusercontent.com/garrytan/gbrain/master/docs/protocol/MEMORY_VERBS_v1.md
# MEMORY_VERBS v1 — the memory wire protocol
GBrain's frozen five-verb memory interface over MCP: `recall`, `remember`,
`entity`, `synthesize`, `forget`. The contract every harness can rely on the
way every Postgres client relies on the wire protocol — and the contract any
OTHER memory server can implement and certify against
(`gbrain protocol conformance --target <endpoint>`).
```
agent (any MCP harness)
│ remember("picked Stripe over Adyen", provenance: "chat 2026-06-11")
▼
five verbs ── recall ── remember ── entity ── synthesize ── forget
│ self-describing envelopes: protocol_version, evidence, provenance,
│ budget meta, cost block, enumerated error codes + a populated fix
▼
your brain (reference implementation: gbrain; any conformant server)
```
**Machine-readable spec:** `gbrain protocol --json` emits the input schemas
from the live operation definitions plus the response-shape registry — doc and
code structurally cannot drift; conformance validates live responses against
the same registry.
## Versioning policy (the point of the freeze)
- Every field NAME and its SEMANTICS in v1 are frozen forever — never removed,
renamed, or re-typed; meanings never change.
- New OPTIONAL params and new OPTIONAL response fields may be added at any
time (additive-forever). A conformant CLIENT must ignore unknown fields; a
conformant SERVER must never reject unknown-to-v1 additions it itself ships.
- `protocol_version` (integer, starts at `1`) rides every verb response and
every verb error. It increments ONLY on a breaking change, which by policy
requires a new `MEMORY_VERBS_v2` document — expected never.
- Conformance pins a minimum version; certification asserts shape, enum
validity, contract behavior, and round-trips — never ranking quality (that
is BrainBench's job).
- Enum values are part of the contract. Where an enum's DERIVATION is
implementation-defined (noted per field), implementations may improve the
derivation without a version bump; the values and their meanings stay fixed.
## Install (the 4-command quickstart)
```bash
gbrain init --pglite # 2-second local brain
claude mcp add gbrain -- gbrain serve --surface verbs # the five-verb surface
gbrain remember "I prefer dark mode in every editor" --provenance demo --entity people/me
gbrain recall --entity people/me # …now ask your agent in a NEW session
```
> Memories agents save are readable by every agent connected to this brain;
> pass `visibility: "private"` for local-CLI-only facts.
If `claude` is not found: install Claude Code first, or use a block below.
**Codex**
```bash
codex mcp add gbrain -- gbrain serve --surface verbs
```
**OpenClaw / any stdio MCP host** — register the server command
`gbrain serve --surface verbs`. Remote brains: `gbrain serve --http` on the
host, then `gbrain connect https://host/mcp --token gbrain_xxx --install` on
each client.
**Surface modes:** `--surface verbs` exposes EXACTLY the five verbs —
advertised list AND dispatch are filtered fail-closed (a hidden op returns
`unknown_tool` even when called by name). `--surface full` (the default)
exposes every operation, verbs included. Why default full: verbs is for
agents and quickstarts; full preserves existing advanced tooling. Persist a
default with `gbrain config set mcp_surface verbs`.
## The verbs
### recall(query?, entity?, budget_tokens?, since?, session_id?, limit?, …) — read
Retrieve saved facts and (with `query`) budget-packed page snippets.
- `entity` scopes the FACTS arm; `query` runs the hybrid-search arm over
pages; both present ⇒ both arms run.
- `since`: ISO 8601 date/datetime — filters the FACTS arm only in v1. (The
reference implementation also accepts relative phrases like `"8 hours ago"`
as a convenience; only ISO 8601 is part of the frozen contract.)
- `limit` is a PER-ARM cap (facts and search results each).
- `budget_tokens`: SERVER-side packing — facts pack first (limit-capped
one-liners, so search-arm starvation is bounded), search results take the
remainder. The estimator is char/4 (±10–15%); `budget_used` reports packed
tokens, `dropped_count` what didn't fit. Never advisory, never client-side.
- No embedding provider configured? The search arm degrades to keyword-only
and the response notes `search_degraded` — never an error.
Response — an additive SUPERSET of the pre-v1 facts envelope on EVERY call
(all legacy fields unchanged; JSON consumers ignore additions):
| field | type | semantics |
|---|---|---|
| `protocol_version` | int | always present (every verb, every call) |
| `facts[]` | array | legacy fact fields unchanged, PLUS per fact: `fact_id` (opaque STRING — the value `forget` accepts; the legacy numeric `id` stays for pre-v1 consumers) and `provenance` (the stored source attribution) |
| `total` | int | count of facts returned |
| `results[]` | array | search arm only: `slug`, `title`, `chunk`, `evidence`, `create_safety`, `provenance` (origin page slug) |
| `search_degraded` | string? | present when keyword-only fallback fired |
| `budget_tokens` / `budget_used` / `dropped_count` | int? | present when `budget_tokens` was passed |
**evidence** (enum, zero-LLM heuristic): `alias_hit` \| `exact_title_match` \|
`high_vector_match` \| `keyword_exact` \| `weak_semantic` — why each result
matched. **create_safety** (enum): `exists` (a page for this already exists)
\| `probable` (likely exists; check before creating) \| `unknown` (no
signal). The derivation of both is implementation-defined and may improve;
the values are frozen.
### remember(fact, provenance, ttl?, entity?, kind?, visibility?) — write
Save ONE fact with mandatory attribution.
- `provenance` (REQUIRED, free text ≤500 chars, stored verbatim): e.g.
`"conversation 2026-06-12"`, `"user said in chat"`, `"import: notes.md"`.
Empty ⇒ `provenance_required` error with a fix.
- `entity`: set whenever the fact is about a specific person/company/project —
entity-scoped recall will not find unattributed facts.
- `ttl`: duration shorthand (`"30d"`, `"12h"`, `"45m"`) or an absolute ISO 8601
timestamp. ISO-8601 DURATIONS (`P30D`) are rejected with a self-correcting
suggestion. Omitted ⇒ never expires.
- `kind`: `event` \| `preference` \| `commitment` \| `belief` \| `fact`
(default).
- `visibility`: `world` (DEFAULT — readable by every agent connected to this
brain; required for the remote remember→recall round-trip) \| `private`
(local CLI reads only). The init quickstart carries the consent line.
Response: `{ id, status, status_text, entity_slug, valid_until,
protocol_version }` (+ `degraded_dedup: true` when no embedding provider —
near-duplicates may insert; dedup and supersession ride embedding similarity).
- `id` — opaque STRING (gbrain serializes integers; another implementation may
use UUIDs). On `status: "duplicate"` it is the EXISTING fact's id.
- `status` — `inserted` \| `duplicate` \| `superseded`. **Branch on `status`,
never on `status_text`** (the human rendering). Supersession is
implementation-defined; the reference rule: same entity + same kind +
similarity above the dedup threshold + different text = the new fact
supersedes the old ("X at acme-example" → "X left acme-example").
- Omitted optional inputs echo as `null`, never absent.
### entity(name) — read, zero LLM, p99 < 100ms
One known person/company/project card. NEVER errors on a miss.
Resolution (frozen precedence): alias > exact title > slug/slug-suffix; ties
break on most-recently-touched. Multi-hit ⇒ best match's card + runners-up in
`suggestions`. Miss ⇒ `found: false` + keyword near-misses with
`create_safety` hints.
Response: `{ protocol_version, found, latency_ms, card?, suggestions? }`.
Card: `{ entity{slug,title,type}, aka[], summary, last_touched{updated_at,
last_retrieved_at, last_timeline_date}, open_threads[], edges[],
backlink_count, active_fact_count }`.
- `summary` passes the same privacy fences as `get_page` (takes + private
facts stripped); remote callers never see private facts in the card.
- `open_threads` (best-effort in v1): active commitment-kind facts + timeline
entries from the last 90 days, capped at 3.
- `edges`: top ~10 typed edges, mentions excluded, out-edges first.
- The p99 < 100ms promise is op-layer latency (transport excluded), CI-gated
on a 20K-page corpus. 200K validation recipe below.
### synthesize(question, since?, until?) — read, EXPENSIVE
`[EXPENSIVE / SLOW — makes LLM calls, seconds-to-minutes latency, costs
money]` — the deliberately-priced slow verb. Prefer `recall`/`entity` for
lookups; use synthesize only when the answer requires combining evidence
across pages.
Response: `{ answer, sources[], gaps[], cost{model, input_tokens,
output_tokens, usd_estimate}, protocol_version }`.
- The `cost` block is a BEST-EFFORT AGGREGATE (retries/multi-call flows sum;
cache hits may undercount; token fields are `null` when a provider returns
no accounting). Honest signal, not an invoice.
- No LLM configured ⇒ the protocol error `unavailable` with a fix — never a
fake answer.
### forget(id, reason?) — write
Expire a fact by its opaque string id (from `remember` or
`recall.facts[].fact_id` — never a page slug). Idempotent: re-forgetting an
already-expired fact returns `expired: false` (success); unknown id ⇒
`not_found`. Facts are expired with an audit trail, never deleted.
Response: `{ id, expired, reason, protocol_version }`.
## Error contract (uniform across all five verbs)
```json
{ "error": "<code>", "message": "...", "suggestion": "problem + cause + fix",
"detail": "freeform specifics", "protocol_version": 1 }
```
Codes (coarse on purpose — codes are for branching; `detail` carries the
story): `invalid_params`, `provenance_required`, `not_found`, `scope_denied`,
`unavailable` (a required dependency cannot serve: no API key, gateway down,
model refusal — configure/retry, not a server bug), `budget_unsatisfiable`
(RESERVED — schema-listed, never returned in v1), `internal`.
Every verb error carries a POPULATED `suggestion`. Specific cases: `recall` on
an empty brain returns empty arrays (success, not an error); auth/scope
failures fail closed via the standard dispatch.
## Trust boundary
Verbs are ordinary operations: they inherit fail-closed `remote` semantics,
OAuth scope enforcement (`remember`/`forget` are write-scope), and per-source
isolation on every read. Remote callers see `visibility = world` facts only.
## Conformance + certification
```bash
gbrain protocol conformance # self-certify (stdio)
gbrain protocol conformance --target http://localhost:3131/mcp --token gbrain_xxx
gbrain protocol conformance --target "bun run src/cli.ts serve"
gbrain protocol conformance --synthesize # also live-call synthesize
```
Pass criteria: response SHAPE (required fields, enum validity), CONTRACT
BEHAVIOR (provenance rejected when empty; budget arithmetic consistent;
entity miss ⇒ `found:false`, not an error; private facts absent from remote
cards; idempotent forget), and ROUND-TRIP (remember → recall by entity — a
plain indexed read, deterministic). It does NOT judge ranking quality.
Entity-card cases need a seedable page (`put_page`); against verbs-only
targets they skip honestly. `--synthesize` is cost-gated: with no LLM key it
asserts the clean `unavailable` error (what CI does); with a key it spends
real tokens.
Conformance is a LIVE test that WRITES: it seeds a marker-suffixed synthetic
entity page (`people/conformance-<marker>`, when the target exposes
`put_page`) and writes/expires facts through `remember`/`forget`. Point it at
write-capable credentials and a brain you're comfortable leaving those
synthetic artifacts in — they're marker-named for easy cleanup, not
auto-deleted. The fixture set ships as data
(`test/fixtures/memory-verbs/cases.json`) and seeds BrainBench's
protocol-compliance arm. gbrain's CI certifies its own stdio + HTTP
transports; external certification is best-effort tooling until a second
implementation exists.
## Observability (local only)
Every verb call appends one line to
`~/.gbrain/integrations/memory-verbs/usage.jsonl` — **local JSONL only, never
uploaded**, stats-only (lock-free rotation may drop lines; POSIX O_APPEND
line-atomic, best-effort on Windows). `gbrain protocol stats [--days N]`
aggregates per-verb calls, error rate, latency, budget drops, entity hit rate,
and the measured TTHW (install → first verb call, from the
`protocol_installed_at` stamp). `gbrain doctor` carries a
`memory_verbs_usage` health line.
## 200K-page latency validation (manual recipe)
CI gates entity() p99 < 100ms on a 20K-page corpus
(`test/entity-card-perf.slow.test.ts`). To validate at 200K, edit the
constants at the top of that file (`PAGES = 200_000`, `LINKS = 1_000_000`,
`ALIASES = 300_000`, `FACTS = 400_000`) and run
`bun test test/entity-card-perf.slow.test.ts --timeout=1800000` — seeding
dominates (~minutes); the measured calls report p50/p99 + the ratio guard.
---
# AI providers
# Debugging
llms.txt
+1
View File
@@ -27,6 +27,7 @@ Repo: https://github.com/garrytan/gbrain
- [docs/guides/scaling-skills.md](https://raw.githubusercontent.com/garrytan/gbrain/master/docs/guides/scaling-skills.md): Three-tier architecture for agents with 300+ skills: always-loaded, resolver-routed, and dormant. Per-turn token math, the v0.41.7.0 compact list-format resolver, and the `gbrain doctor` safety net. 306 skills, ~21K tokens freed per turn, zero capability loss.
- [docs/guides/push-context.md](https://raw.githubusercontent.com/garrytan/gbrain/master/docs/guides/push-context.md): Push-based context: the brain volunteers confidence-gated pages from the rolling conversation window. Three channels (ambient reflex, volunteer_context op, gbrain watch), config knobs, and the volunteered-vs-used feedback loop.
- [docs/mcp/DEPLOY.md](https://raw.githubusercontent.com/garrytan/gbrain/master/docs/mcp/DEPLOY.md): MCP server deployment.
- [docs/protocol/MEMORY_VERBS_v1.md](https://raw.githubusercontent.com/garrytan/gbrain/master/docs/protocol/MEMORY_VERBS_v1.md): The frozen five-verb memory protocol (recall/remember/entity/synthesize/forget): response envelopes, error contract, additive-forever versioning, surface modes, conformance certification, per-harness installs.
## AI providers
package.json
+2 -1
View File
@@ -37,6 +37,7 @@
"build:admin-embedded": "bun run scripts/build-admin-embedded.ts",
"build:schema": "bash scripts/build-schema.sh",
"build:llms": "bun run scripts/build-llms.ts",
"build:flag-registry": "bun run scripts/generate-flag-registry.ts",
"build:pglite-snapshot": "bun run scripts/build-pglite-snapshot.ts",
"test": "bash scripts/run-unit-parallel.sh",
"eval:autocut": "bun test test/search/autocut-eval.test.ts",
@@ -150,7 +151,7 @@
"bun": ">=1.3.10"
},
"license": "MIT",
"version": "0.42.75.0",
"version": "0.43.0.0",
"overrides": {
"@hono/node-server": "^2.0.5",
"fast-uri": "^3.1.5",
scripts/generate-flag-registry.ts
+177
View File
@@ -0,0 +1,177 @@
/**
* #2185 — known-flags registry generator for CLI_ONLY commands.
*
* gbrain's CLI_ONLY commands read flags ad hoc (`args.includes('--force')`,
* per-command parseFlags helpers), so there is no parser to make strict. The
* pre-dispatch validator in src/cli.ts needs to know each command's legal
* flags; this script derives them from the source instead of a hand-typed
* list that would rot.
*
* How: parse handleCliOnly's top-level `case 'X': {` blocks out of src/cli.ts,
* collect every `import('./commands/Y.ts')` inside each block, then scan the
* case-block text plus each imported module (plus one level of that module's
* ./relative same-directory imports) for `--flag` string literals — including
* help text, which deliberately over-includes: accepting a flag the handler
* ignores is the pre-#2185 status quo for that flag, while missing a real
* flag would break working invocations on upgrade.
*
* Output: src/core/cli-flag-registry.generated.ts (committed; freshness is
* pinned by test/cli-flag-validation.test.ts the same way build:llms pins the
* llms bundles). Regenerate: bun run build:flag-registry
*
* Hand-tuning lane: EXTRA_FLAGS below, for flags that live deeper than the
* one-level scan (add with a comment naming the deep module).
*/
import { readFileSync, writeFileSync, existsSync } from 'fs';
import { dirname, resolve as resolvePath, join } from 'path';
import { fileURLToPath } from 'url';
const ROOT = resolvePath(dirname(fileURLToPath(import.meta.url)), '..');
/** Flags that live deeper than the one-level module scan. Keep commented. */
const EXTRA_FLAGS: Record<string, string[]> = {
// embed's pace knobs resolve inside src/core/pace-mode.ts (two levels deep).
embed: ['--pace', '--pace-max-concurrency'],
// sync shares the same pace surface via env/config plus CLI passthrough.
sync: ['--pace', '--pace-max-concurrency'],
};
/** Universal helper flags every command may see (parsed or short-circuited upstream). */
const UNIVERSAL_FLAGS = ['--help', '--json', '--brain', '--source'];
const FLAG_RE = /--[a-z0-9][a-z0-9-]*/g;
function flagsInText(text: string): Set<string> {
const out = new Set<string>();
for (const m of text.matchAll(FLAG_RE)) {
// Template-literal prefixes (`--bound-${key}` scans as `--bound-`) are
// not real flags — a trailing hyphen would make the validator accept
// every typo sharing the prefix.
if (!m[0].endsWith('-')) out.add(m[0]);
}
return out;
}
/** One level of ./relative imports (static or dynamic) from a module's source. */
function relativeImports(src: string, fromDir: string): string[] {
const paths = new Set<string>();
for (const m of src.matchAll(/from\s+'(\.\.?\/[^']+\.ts)'/g)) paths.add(m[1]);
for (const m of src.matchAll(/import\('(\.\.?\/[^']+\.ts)'\)/g)) paths.add(m[1]);
return [...paths]
.map(p => resolvePath(fromDir, p))
.filter(p => existsSync(p));
}
export function buildFlagRegistry(): Record<string, string[]> {
const cliSource = readFileSync(join(ROOT, 'src/cli.ts'), 'utf-8');
// CLI_ONLY membership (the single source of truth in src/cli.ts). Strip
// line comments first — the set literal carries commentary whose quoted
// words ('Unknown command', 'pages') must not parse as members.
const onlyMatch = cliSource.match(/const CLI_ONLY = new Set(?:<string>)?\(\[([\s\S]*?)\]\)/);
if (!onlyMatch) throw new Error('CLI_ONLY set not found in src/cli.ts');
const onlyBody = onlyMatch[1].replace(/\/\/[^\n]*/g, '');
const commands = [...onlyBody.matchAll(/'([^']+)'/g)].map(m => m[1]);
// handleCliOnly body — bounded at the function's closing brace (column 0).
// Unbounded, the LAST case block absorbed every --flag literal in the rest
// of cli.ts (printHelp's full flag surface included), handing whichever
// command sits last in the switch a ~100-flag junk allowlist that made
// strict validation a no-op for it.
const fnStart = cliSource.indexOf('async function handleCliOnly');
if (fnStart < 0) throw new Error('handleCliOnly not found in src/cli.ts');
const fnTail = cliSource.slice(fnStart);
const fnEndRel = fnTail.search(/\n\}\n/);
const fnSrc = fnEndRel > 0 ? fnTail.slice(0, fnEndRel) : fnTail;
// handleCliOnly dispatches through TWO styles: an `if (command === 'X')`
// chain (DB-free commands like init/auth/schema) and a switch with
// `case 'X':` labels. Segment on BOTH marker kinds; the text between a
// marker and the next marker belongs to that label.
const markRe = /(?:^\s*if \(command === '([a-z0-9-]+)'\)|^ case '([a-z0-9-]+)':)/gm;
const marks: Array<{ label: string; start: number }> = [];
let m: RegExpExecArray | null;
while ((m = markRe.exec(fnSrc)) !== null) {
marks.push({ label: (m[1] ?? m[2])!, start: m.index });
}
const blocks = new Map<string, string>();
for (let i = 0; i < marks.length; i++) {
const end = i + 1 < marks.length ? marks[i + 1].start : fnSrc.length;
const body = fnSrc.slice(marks[i].start, end);
// Fall-through labels share the following block.
blocks.set(marks[i].label, (blocks.get(marks[i].label) ?? '') + body);
}
// Safety flags carry destructive-bypass semantics: allowlisting one that
// the handler never reads recreates the #2185 repro (`post-upgrade
// --dry-run` accepted, ignored, migrations run for real). Presence isn't
// enough — upgrade.ts prints a HINT naming another command's --dry-run,
// which is depth-0 text for post-upgrade. These flags are only legal with
// CONSUMPTION evidence in the command's own code: the flag as a TIGHT-QUOTED
// standalone literal (`includes('--dry-run')`, `has('--dry-run')`,
// `=== '--dry-run'`). Prose bleed embeds the flag inside a longer string, so
// it never has quotes on both sides of the bare flag.
const SAFETY_FLAGS = new Set(['--dry-run']);
const consumes = (text: string, flag: string): boolean =>
new RegExp(`['"\`]${flag}['"\`]`).test(text);
const registry: Record<string, string[]> = {};
for (const command of commands) {
const block = blocks.get(command) ?? '';
const flags = new Set<string>(UNIVERSAL_FLAGS);
const depthZero = new Set<string>();
let depthZeroText = block;
for (const f of flagsInText(block)) { flags.add(f); depthZero.add(f); }
// Modules imported inside the case block, plus one level of each module's
// own ./relative imports.
const commandModules = [...block.matchAll(/import\('(\.\/[^']+\.ts)'\)/g)]
.map(mm => resolvePath(join(ROOT, 'src'), mm[1]))
.filter(p => existsSync(p));
for (const modPath of commandModules) {
const modSrc = readFileSync(modPath, 'utf-8');
depthZeroText += modSrc;
for (const f of flagsInText(modSrc)) { flags.add(f); depthZero.add(f); }
for (const dep of relativeImports(modSrc, dirname(modPath))) {
for (const f of flagsInText(readFileSync(dep, 'utf-8'))) flags.add(f);
}
}
for (const f of EXTRA_FLAGS[command] ?? []) { flags.add(f); depthZero.add(f); }
for (const f of SAFETY_FLAGS) {
if (flags.has(f) && !consumes(depthZeroText, f)) flags.delete(f);
}
registry[command] = [...flags].sort();
}
return registry;
}
export function renderRegistryModule(registry: Record<string, string[]>): string {
const entries = Object.keys(registry)
.sort()
.map(cmd => ` '${cmd}': [${registry[cmd].map(f => `'${f}'`).join(', ')}],`)
.join('\n');
return `// AUTO-GENERATED by scripts/generate-flag-registry.ts — do not edit by hand.
// Regenerate: bun run build:flag-registry
// Freshness + drift pinned by test/cli-flag-validation.test.ts (#2185).
//
// Per-command legal flags for CLI_ONLY commands, derived from each command's
// source (case block + imported modules + one level of relative imports +
// scripts/generate-flag-registry.ts EXTRA_FLAGS). Deliberately over-inclusive
// (help-text mentions count): accepting an ignored flag is the pre-#2185
// status quo; missing a real one breaks working invocations.
export const CLI_FLAG_REGISTRY: Record<string, readonly string[]> = {
${entries}
};
`;
}
if (import.meta.main) {
const registry = buildFlagRegistry();
const outPath = join(ROOT, 'src/core/cli-flag-registry.generated.ts');
writeFileSync(outPath, renderRegistryModule(registry));
const n = Object.keys(registry).length;
const total = Object.values(registry).reduce((a, v) => a + v.length, 0);
console.log(`wrote ${outPath} (${n} commands, ${total} flag entries)`);
}
scripts/llms-config.ts
+6
View File
@@ -162,6 +162,12 @@ export const SECTIONS: DocSection[] = [
description: "MCP server deployment.",
path: "docs/mcp/DEPLOY.md",
},
{
title: "docs/protocol/MEMORY_VERBS_v1.md",
description:
"The frozen five-verb memory protocol (recall/remember/entity/synthesize/forget): response envelopes, error contract, additive-forever versioning, surface modes, conformance certification, per-harness installs.",
path: "docs/protocol/MEMORY_VERBS_v1.md",
},
],
},
{
scripts/run-unit-parallel.sh
+299 -16
View File
@@ -13,9 +13,29 @@
#
# Env overrides:
# SHARDS=N same as --shards
# GBRAIN_TEST_SHARD_TIMEOUT per-shard wallclock cap, seconds (default 2400)
# GBRAIN_TEST_SHARD_TIMEOUT per-shard wallclock cap, seconds (default 3000)
# GBRAIN_TEST_SHARD_KILL_AFTER grace after TERM before KILL (default 30)
# GBRAIN_TEST_MAX_CONCURRENCY passed through to bun test (default 4)
# GBRAIN_TEST_MEM_PER_FILE_MB memory budget per concurrent test file used by
# the adaptive sizing below (default 1536 — a
# PGLite WASM instance reserves ~1-1.5GB)
# GBRAIN_TEST_NO_MEM_ADAPT=1 disable memory-aware concurrency reduction
# GBRAIN_TEST_NO_OOM_FALLBACK=1 disable the serial OOM-rescue pass
#
# Memory safety (two layers; both default-on):
# 1. ADAPTIVE SIZING — before spawning, total concurrency (shards ×
# intra-shard --max-concurrency) is capped to what available memory can
# hold at GBRAIN_TEST_MEM_PER_FILE_MB per concurrent file. Concurrent
# Conductor workspaces running their own suites shrink the budget
# automatically instead of OOMing each other.
# 2. SERIAL PHANTOM RESCUE — two phantom classes are re-run serially
# (--max-concurrency 1) after the parallel pass: (a) failures whose
# shard log carries the PGLite WASM out-of-memory signature, and
# (b) shards killed EXTERNALLY (SIGTERM/SIGKILL well before the shard
# timeout — sibling Conductor workspaces' process cleanup, macOS memory
# jetsam). Phantoms pass serially and the run goes green with an
# oom_rescued note; real failures fail again and stay red. Plain
# assertion failures never match either signature.
#
# Output files (workspace-local; falls back to /tmp if .context/ unwritable):
# .context/test-failures.log failure blocks (cleared at start)
@@ -38,6 +58,35 @@ detect_cpus() {
echo 4
}
# ──────────────────────────────────────────────────────────────────────────
# Available-memory detection (MB). macOS: vm_stat free + inactive +
# speculative + purgeable pages (inactive/purgeable are reclaimable on
# pressure, which is exactly the scenario we size for). Linux: MemAvailable.
# Unknown platform → 0, and the caller skips adaptation entirely.
# ──────────────────────────────────────────────────────────────────────────
detect_available_mem_mb() {
if command -v vm_stat >/dev/null 2>&1; then
vm_stat 2>/dev/null | awk '
/page size of/ { psize = $8 }
/Pages free/ { free = $NF }
/Pages inactive/ { inactive = $NF }
/Pages speculative/ { spec = $NF }
/Pages purgeable/ { purge = $NF }
END {
gsub(/\./, "", free); gsub(/\./, "", inactive)
gsub(/\./, "", spec); gsub(/\./, "", purge)
if (psize == 0) psize = 16384
printf "%d\n", (free + inactive + spec + purge) * psize / 1048576
}'
return
fi
if [ -r /proc/meminfo ]; then
awk '/MemAvailable/ { printf "%d\n", $2 / 1024; found = 1 } END { if (!found) print 0 }' /proc/meminfo
return
fi
echo 0
}
# ──────────────────────────────────────────────────────────────────────────
# Argument parsing. --shards N override wins over $SHARDS; both are clamped.
# ──────────────────────────────────────────────────────────────────────────
@@ -76,21 +125,65 @@ INTRA_CONC="${MAX_CONCURRENCY_OVERRIDE:-${GBRAIN_TEST_MAX_CONCURRENCY:-4}}"
# 4-shard default each shard runs 159 files / ~2420 tests with internal
# wallclock 960-1020s. The 900s value (sized for 8-shard's ~80 files /
# 1100 tests at 620-770s) false-killed shard 1 at 900s even though it
# had completed in 968s. The cap must track suite growth: at ~13000 tests
# (agent-bootstrap wave) the heaviest count-balanced shard is still making
# progress at 1800s under 4-way contention (observed: log growth 22s
# before an 1800s kill) while its siblings finish at 1150-1550s — the
# split balances file COUNT, not weight. 2400s covers the heavy shard
# with headroom; genuinely hung TESTS still die at bun's per-test
# timeout, mid-run stalls still hit this cap, and post-completion
# exit-hangs are classified separately (see the EXIT-HANG block below).
# Override via GBRAIN_TEST_SHARD_TIMEOUT=N.
SHARD_TIMEOUT="${GBRAIN_TEST_SHARD_TIMEOUT:-2400}"
# had completed in 968s. The cap must track suite growth: the suite roughly
# tripled since the 1500s cap was set (June: ~3900 tests, 92-migration PGLite
# replay; now: 13k+ tests with the agent-bootstrap wave, 120-migration replay
# per PGLite init). The split balances file COUNT, not weight — the heaviest
# count-balanced shard is still making steady per-test progress at 1800s under
# 4-way contention while its siblings finish at 1150-1550s. 3000s keeps the
# ~55%-headroom doctrine over observed wallclock; genuinely hung TESTS still
# die at bun's per-test timeout, mid-run stalls still hit this cap, and
# post-completion exit-hangs are classified separately (see the EXIT-HANG
# block below). Override via GBRAIN_TEST_SHARD_TIMEOUT=N.
SHARD_TIMEOUT="${GBRAIN_TEST_SHARD_TIMEOUT:-3000}"
SHARD_KILL_AFTER="${GBRAIN_TEST_SHARD_KILL_AFTER:-30}"
if ! printf '%s' "$SHARD_KILL_AFTER" | grep -qE '^[0-9]+$' || [ "$SHARD_KILL_AFTER" -lt 1 ]; then
echo "ERROR: invalid shard kill-after: $SHARD_KILL_AFTER" >&2; exit 2
fi
# ──────────────────────────────────────────────────────────────────────────
# Memory-aware concurrency (layer 1). Total concurrent test files =
# N shards × INTRA_CONC; each concurrent file can hold a PGLite WASM
# instance (~1-1.5GB reserved). 4×4 = 16 concurrent instances OOM'd on a
# 128GB machine when other Conductor workspaces ran their suites at the
# same time — every PGLite connect across every shard failed at once
# ("Out of memory" at PGlite.create). Cap total concurrency to what's
# actually available, keeping a 4GB reserve for the OS + bun itself.
# Applies to explicit --shards overrides too (an operator who wants an
# over-committed run sets GBRAIN_TEST_NO_MEM_ADAPT=1).
# ──────────────────────────────────────────────────────────────────────────
MEM_PER_FILE_MB="${GBRAIN_TEST_MEM_PER_FILE_MB:-1536}"
MEM_NOTE=""
if [ "${GBRAIN_TEST_NO_MEM_ADAPT:-0}" != "1" ]; then
AVAIL_MB=$(detect_available_mem_mb)
if [ "${AVAIL_MB:-0}" -gt 0 ] 2>/dev/null; then
BUDGET_MB=$((AVAIL_MB - 4096))
[ "$BUDGET_MB" -lt "$MEM_PER_FILE_MB" ] && BUDGET_MB="$MEM_PER_FILE_MB"
MAX_TOTAL=$((BUDGET_MB / MEM_PER_FILE_MB))
[ "$MAX_TOTAL" -lt 1 ] && MAX_TOTAL=1
ORIG_N="$N"; ORIG_INTRA="$INTRA_CONC"
# Shed shards before intra-shard concurrency: fewer bun processes frees
# more than narrower ones (each process carries its own heap + WASM).
while [ $((N * INTRA_CONC)) -gt "$MAX_TOTAL" ]; do
if [ "$N" -gt 1 ]; then N=$((N - 1))
elif [ "$INTRA_CONC" -gt 1 ]; then INTRA_CONC=$((INTRA_CONC - 1))
else break
fi
done
if [ "$N" != "$ORIG_N" ] || [ "$INTRA_CONC" != "$ORIG_INTRA" ]; then
# Fewer shards → more files per shard → each shard legitimately runs
# longer. Scale the per-shard cap by the shed ratio so adaptation
# doesn't convert memory safety into false WEDGED verdicts.
if [ "$N" -lt "$ORIG_N" ]; then
SHARD_TIMEOUT=$((SHARD_TIMEOUT * ORIG_N / N))
fi
MEM_NOTE=" | mem-adapted ${ORIG_N}x${ORIG_INTRA}→${N}x${INTRA_CONC} (avail=${AVAIL_MB}MB, ${MEM_PER_FILE_MB}MB/file, timeout→${SHARD_TIMEOUT}s)"
else
MEM_NOTE=" | mem-ok (avail=${AVAIL_MB}MB)"
fi
fi
fi
# ──────────────────────────────────────────────────────────────────────────
# Output directories. Prefer workspace-local .context/, fall back to /tmp.
# ──────────────────────────────────────────────────────────────────────────
@@ -106,7 +199,7 @@ else
mkdir -p "$LOG_DIR" || { echo "ERROR: cannot create log dir" >&2; exit 2; }
fi
# Clear from prior run.
rm -f "$LOG_DIR"/shard-*.log "$LOG_DIR"/shard-*.exit "$LOG_DIR"/shard-*.wedged "$LOG_DIR"/shard-*.lastkb "$LOG_DIR"/shard-*.lastprogress 2>/dev/null
rm -f "$LOG_DIR"/shard-*.log "$LOG_DIR"/shard-*.exit "$LOG_DIR"/shard-*.wedged "$LOG_DIR"/shard-*.lastkb "$LOG_DIR"/shard-*.lastprogress "$LOG_DIR"/shard-*.start "$LOG_DIR"/shard-*.end 2>/dev/null
: > "$FAILURES_LOG"
: > "$SUMMARY_FILE"
@@ -120,7 +213,7 @@ elif command -v timeout >/dev/null 2>&1; then TIMEOUT_BIN="timeout"
fi
START_TS=$(date +%s)
echo "[unit-parallel] N=$N shards | --max-concurrency=$INTRA_CONC | timeout=${SHARD_TIMEOUT}s | kill-after=${SHARD_KILL_AFTER}s | logs=$LOG_DIR" >&2
echo "[unit-parallel] N=$N shards | --max-concurrency=$INTRA_CONC | timeout=${SHARD_TIMEOUT}s | kill-after=${SHARD_KILL_AFTER}s | logs=$LOG_DIR${MEM_NOTE}" >&2
if [ "$DRY_RUN" = "1" ]; then
echo "[unit-parallel] dry-run: would spawn $N shards with the above settings."
@@ -139,6 +232,7 @@ SHARD_PIDS=()
for i in $(seq 1 "$N"); do
(
SHARD_LOG="$LOG_DIR/shard-$i.log"
date +%s > "$LOG_DIR/shard-$i.start"
if [ -n "$TIMEOUT_BIN" ]; then
"$TIMEOUT_BIN" --signal=TERM --kill-after="${SHARD_KILL_AFTER}s" "${SHARD_TIMEOUT}s" \
env SHARD="$i/$N" \
@@ -168,6 +262,7 @@ for i in $(seq 1 "$N"); do
kill "$cap_pid" 2>/dev/null
wait "$cap_pid" 2>/dev/null
fi
date +%s > "$LOG_DIR/shard-$i.end"
echo "$rc" > "$LOG_DIR/shard-$i.exit"
{ [ "$rc" = "124" ] || [ "$rc" = "137" ]; } && echo "WEDGED" > "$LOG_DIR/shard-$i.wedged"
) &
@@ -331,6 +426,40 @@ TOTAL_FAILURES=0
TOTAL_PASS=0
TOTAL_SKIP=0
TOTAL_RC=0
# Layer 2 state (serial OOM rescue). A shard whose log carries the WASM
# out-of-memory signature gets its failing files queued for a serial re-run;
# NON_OOM_FAIL records that at least one failure exists that the rescue lane
# must NOT absolve (plain assertion failures, wedges without the signature).
OOM_RE='Out of memory|WebAssembly\.Memory|RuntimeError: [Aa]borted|Aborted\(\)'
OOM_RESCUE_LIST="$LOG_DIR/oom-rescue-files.txt"
: > "$OOM_RESCUE_LIST"
NON_OOM_FAIL=0
# Set when any shard was killed externally — killed-midrun shards leave lock/
# state residue that can poison the LATER serial pass, so serial failures are
# only rescue-eligible under this flag (or their own OOM signature). A flaky
# serial test in an otherwise-clean run must stay red.
EXTERNAL_KILL_ANY=0
# failing_files_in_log: attribute each `(fail)` block to the test file whose
# `path.test.ts:` header most recently preceded it in bun's output. Under
# GITHUB_ACTIONS the shard wraps each file section as `::group::path.test.ts:`
# — strip that prefix or the rescue pass feeds bun literal `::group::...`
# non-paths that match zero test files (CI-only; local runs have no groups).
failing_files_in_log() {
local file="$1"
[ -f "$file" ] || return 0
awk '
/^(::group::)?[^ ].*\.test\.ts:$/ {
current = $0
sub(/^::group::/, "", current)
current = substr(current, 1, length(current) - 1)
next
}
/^\(fail\) / && current != "" { print current }
' "$file" | sort -u
}
for i in $(seq 1 "$N"); do
SHARD_LOG="$LOG_DIR/shard-$i.log"
EXIT_FILE="$LOG_DIR/shard-$i.exit"
@@ -345,6 +474,33 @@ for i in $(seq 1 "$N"); do
TOTAL_FAILURES=$((TOTAL_FAILURES + fail_count))
TOTAL_SKIP=$((TOTAL_SKIP + skip_count))
shard_oom=0
if [ "$rc" != "0" ] && [ "${GBRAIN_TEST_NO_OOM_FALLBACK:-0}" != "1" ] \
&& [ -f "$SHARD_LOG" ] && grep -qE "$OOM_RE" "$SHARD_LOG"; then
shard_oom=1
fi
# External-kill detection: rc 143 (SIGTERM) / 137 (SIGKILL) with the shard
# dying before 80% of the shard timeout means something OUTSIDE the runner
# killed it — sibling Conductor workspaces' process cleanup and macOS
# memory jetsam both present exactly this way (observed: 3 shards TERM'd +
# 1 KILL'd at ~700s under a 3000s cap, all mid-progress). A REAL wedge is
# killed BY the runner at ~SHARD_TIMEOUT and stays red. Externally-killed
# shards are phantoms: queue for the serial rescue lane like OOM.
shard_external_kill=0
if [ "$shard_oom" = "0" ] && [ "${GBRAIN_TEST_NO_OOM_FALLBACK:-0}" != "1" ] \
&& { [ "$rc" = "143" ] || [ "$rc" = "137" ]; }; then
s_start=$(cat "$LOG_DIR/shard-$i.start" 2>/dev/null) || s_start=""
s_end=$(cat "$LOG_DIR/shard-$i.end" 2>/dev/null) || s_end=""
if [ -n "$s_start" ] && [ -n "$s_end" ]; then
s_elapsed=$((s_end - s_start))
if [ "$s_elapsed" -lt $((SHARD_TIMEOUT * 80 / 100)) ]; then
shard_external_kill=1
EXTERNAL_KILL_ANY=1
fi
fi
fi
if [ -f "$WEDGED_FILE" ]; then
# EXIT-HANG classifier (pre-existing PGLite-adjacent leak, TODOS.md
# "unit-shard exit hang"): a shard killed by the watchdog whose log shows
@@ -376,15 +532,45 @@ for i in $(seq 1 "$N"); do
continue
fi
TOTAL_RC=1
if [ "$shard_external_kill" = "1" ]; then
SHARD="$i/$N" bash scripts/run-unit-shard.sh --dry-run-list 2>/dev/null >> "$OOM_RESCUE_LIST"
echo "shard $i/$N: KILLED externally after ${s_elapsed}s (rc=$rc, well before ${SHARD_TIMEOUT}s cap — queued for serial rescue)" >> "$SUMMARY_FILE"
elif [ "$shard_oom" = "1" ]; then
# Wedged UNDER memory pressure: we can't attribute failures, so queue
# the shard's entire file list for the serial rescue pass.
SHARD="$i/$N" bash scripts/run-unit-shard.sh --dry-run-list 2>/dev/null >> "$OOM_RESCUE_LIST"
echo "shard $i/$N: WEDGED after ${SHARD_TIMEOUT}s (rc=$rc, OOM signature — queued for serial rescue)" >> "$SUMMARY_FILE"
else
NON_OOM_FAIL=1
echo "shard $i/$N: WEDGED after ${SHARD_TIMEOUT}s (rc=$rc)" >> "$SUMMARY_FILE"
fi
{
echo "--- shard $i: WEDGED after ${SHARD_TIMEOUT}s ---"
[ -f "$SHARD_LOG" ] && tail -50 "$SHARD_LOG"
echo ""
} >> "$FAILURES_LOG"
echo "shard $i/$N: WEDGED after ${SHARD_TIMEOUT}s (rc=$rc)" >> "$SUMMARY_FILE"
continue
fi
if [ "$rc" != "0" ]; then
if [ "$shard_oom" = "1" ]; then
# One scan, reused for both the queue append and the emptiness check.
shard_failing_files=$(failing_files_in_log "$SHARD_LOG")
if [ -n "$shard_failing_files" ]; then
printf '%s\n' "$shard_failing_files" >> "$OOM_RESCUE_LIST"
else
# OOM signature but no attributable files (e.g. bun died before any
# file header) → rescue the whole shard.
SHARD="$i/$N" bash scripts/run-unit-shard.sh --dry-run-list 2>/dev/null >> "$OOM_RESCUE_LIST"
fi
elif [ "$shard_external_kill" = "1" ]; then
SHARD="$i/$N" bash scripts/run-unit-shard.sh --dry-run-list 2>/dev/null >> "$OOM_RESCUE_LIST"
echo "shard $i/$N: KILLED externally after ${s_elapsed}s (rc=$rc — queued for serial rescue)" >> "$SUMMARY_FILE"
else
NON_OOM_FAIL=1
fi
fi
echo "shard $i/$N: pass=$pass_count fail=$fail_count skip=$skip_count rc=$rc" >> "$SUMMARY_FILE"
if [ "$rc" != "0" ]; then
@@ -439,6 +625,17 @@ if [ "$SERIAL_FILES_COUNT" -gt 0 ]; then
cat "$LOG_DIR/serial.log"
if [ "$SERIAL_RC" != "0" ]; then
TOTAL_RC=1
if [ "${GBRAIN_TEST_NO_OOM_FALLBACK:-0}" != "1" ] \
&& { grep -qE "$OOM_RE" "$LOG_DIR/serial.log" || [ "$EXTERNAL_KILL_ANY" = "1" ]; }; then
# Serial failures are rescue-eligible ONLY with their own OOM signature
# or when an externally-killed shard ran earlier in this invocation
# (killed-midrun shards leave lock/state residue that poisons the serial
# pass). A merely-OOM'd sibling shard is NOT grounds — a flaky serial
# test must stay red rather than get silently absolved.
failing_files_in_log "$LOG_DIR/serial.log" >> "$OOM_RESCUE_LIST"
else
NON_OOM_FAIL=1
fi
s_fail=$(bun_summary_count "fail" "$LOG_DIR/serial.log")
TOTAL_FAILURES=$((TOTAL_FAILURES + s_fail))
if [ "$s_fail" -gt 0 ]; then
@@ -464,6 +661,92 @@ if [ "$SERIAL_FILES_COUNT" -gt 0 ]; then
fi
fi
# ──────────────────────────────────────────────────────────────────────────
# Layer 2: serial OOM rescue. Re-run every file that failed inside an
# OOM-signature shard, one at a time (1 shard, --max-concurrency 1), after
# the parallel fan-out has fully drained. Phantom failures (the WASM ran out
# of memory because 16 instances were up at once) pass here and the run goes
# green with an oom_rescued note; real failures fail again and stay red.
# ──────────────────────────────────────────────────────────────────────────
OOM_RESCUED=0
OOM_RESCUE_NOTE=""
sort -u "$OOM_RESCUE_LIST" -o "$OOM_RESCUE_LIST" 2>/dev/null
# grep -c exits 1 on zero matches — assign in two steps so an empty rescue
# list yields a single "0" (the grep_count double-output bug, same class).
RESCUE_COUNT=$(grep -c . "$OOM_RESCUE_LIST" 2>/dev/null) || RESCUE_COUNT=0
if [ "$TOTAL_RC" != "0" ] && [ "${RESCUE_COUNT:-0}" -gt 0 ]; then
echo "════════════ OOM rescue pass ($RESCUE_COUNT files, serial) ════════════"
echo "[unit-parallel] OOM signature detected — re-running $RESCUE_COUNT failing file(s) at --max-concurrency 1" >&2
RESCUE_LOG="$LOG_DIR/oom-rescue.log"
# 60s-per-file floor with the shard cap as a minimum, and 2x the shard cap
# as a CEILING: a wedged shard queueing its whole file list must not turn
# `bun run test` into an unbounded multi-hour serial re-run — hitting the
# ceiling reads as a red rescue, not silence.
RESCUE_TIMEOUT=$((RESCUE_COUNT * 60))
[ "$RESCUE_TIMEOUT" -lt "$SHARD_TIMEOUT" ] && RESCUE_TIMEOUT="$SHARD_TIMEOUT"
[ "$RESCUE_TIMEOUT" -gt $((SHARD_TIMEOUT * 2)) ] && RESCUE_TIMEOUT=$((SHARD_TIMEOUT * 2))
# Split the queue: *.serial.test.ts files require one bun PROCESS per file
# (run-serial-tests.sh's isolation contract — top-level mock.module leaks
# across files in a shared registry); the remainder batches in one process.
# Both lanes mirror the shard invocation's --timeout=60000 — bun's default
# 5s per-test timeout would re-fail PGLite phantoms (120-migration replay)
# and mislabel them 'confirmed real'.
grep -v '\.serial\.test\.ts$' "$OOM_RESCUE_LIST" > "$LOG_DIR/oom-rescue-batch.txt" || true
grep '\.serial\.test\.ts$' "$OOM_RESCUE_LIST" > "$LOG_DIR/oom-rescue-serial.txt" || true
RESCUE_RC=0
: > "$RESCUE_LOG"
run_rescue() { # $1 = per-invocation timeout seconds; rest = test-file args
local t="$1"; shift
if [ -n "$TIMEOUT_BIN" ]; then
"$TIMEOUT_BIN" --signal=TERM --kill-after="${SHARD_KILL_AFTER}s" "${t}s" \
bun test --max-concurrency 1 --timeout=60000 "$@" >> "$RESCUE_LOG" 2>&1
else
bun test --max-concurrency 1 --timeout=60000 "$@" >> "$RESCUE_LOG" 2>&1
fi
}
if [ -s "$LOG_DIR/oom-rescue-batch.txt" ]; then
# shellcheck disable=SC2046
run_rescue "$RESCUE_TIMEOUT" $(cat "$LOG_DIR/oom-rescue-batch.txt") || RESCUE_RC=1
fi
if [ -s "$LOG_DIR/oom-rescue-serial.txt" ]; then
while IFS= read -r serial_file; do
[ -n "$serial_file" ] || continue
run_rescue 300 "$serial_file" || RESCUE_RC=1
done < "$LOG_DIR/oom-rescue-serial.txt"
fi
cat "$RESCUE_LOG"
r_pass=$(bun_summary_count "pass" "$RESCUE_LOG")
r_fail=$(bun_summary_count "fail" "$RESCUE_LOG")
if [ "$RESCUE_RC" = "0" ] && [ "$NON_OOM_FAIL" = "0" ]; then
# Every failure in the run was OOM-phantom and every rescued file passed
# serially: the run is green. Adjust the headline numbers so they reflect
# the rescue verdict, and mark the earlier failure blocks superseded.
TOTAL_RC=0
OOM_RESCUED=1
# Do NOT fold r_pass into TOTAL_PASS — the failing shard's own summary
# already counted the rescued files' passing tests, so folding would
# double-count. Rescue results ride in the note instead.
TOTAL_FAILURES=0
OOM_RESCUE_NOTE=" | oom_rescued=${RESCUE_COUNT}files(${r_pass}p serial)"
{
echo "--- OOM rescue: all $RESCUE_COUNT file(s) passed serially (${r_pass} tests) ---"
echo "--- failure blocks above were WASM out-of-memory phantoms, superseded ---"
} >> "$FAILURES_LOG"
echo "oom-rescue: $RESCUE_COUNT files pass=$r_pass rc=0 (phantom OOM failures superseded)" >> "$SUMMARY_FILE"
else
# Real failures confirmed serially (or a non-OOM failure exists anyway).
OOM_RESCUE_NOTE=" | oom_rescue_failed=${r_fail}real"
awk '
/^\(fail\) / { in_block=1; print "--- oom-rescue (serial, confirmed real): " $0; next }
in_block {
if (/^\(pass\)/ || /^\(skip\)/ || /^[[:space:]]*$/ || /__bun_test_summary__/) { in_block=0; print ""; next }
print $0
}
' "$RESCUE_LOG" >> "$FAILURES_LOG"
echo "oom-rescue: $RESCUE_COUNT files pass=$r_pass fail=$r_fail rc=$RESCUE_RC (real failures confirmed)" >> "$SUMMARY_FILE"
fi
fi
END_TS=$(date +%s)
ELAPSED=$((END_TS - START_TS))
@@ -480,10 +763,10 @@ if [ "$TOTAL_RC" != "0" ]; then
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
tail -30 "$FAILURES_LOG"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "[unit-parallel] elapsed=${ELAPSED}s | pass=$TOTAL_PASS fail=$TOTAL_FAILURES skip=$TOTAL_SKIP"
echo "[unit-parallel] elapsed=${ELAPSED}s | pass=$TOTAL_PASS fail=$TOTAL_FAILURES skip=$TOTAL_SKIP${OOM_RESCUE_NOTE}"
} >&2
exit 1
fi
echo "[unit-parallel] elapsed=${ELAPSED}s | pass=$TOTAL_PASS fail=$TOTAL_FAILURES skip=$TOTAL_SKIP" >&2
echo "[unit-parallel] elapsed=${ELAPSED}s | pass=$TOTAL_PASS fail=$TOTAL_FAILURES skip=$TOTAL_SKIP${OOM_RESCUE_NOTE}" >&2
exit 0
skills/brain-ops/SKILL.md
+15 -4
View File
@@ -34,6 +34,17 @@ flows through in both directions.
> **Convention:** See `skills/conventions/brain-first.md` for the 5-step lookup protocol.
> **Convention:** See `skills/conventions/quality.md` for citation and back-link rules.
> **Memory verbs (MEMORY_VERBS v1, gbrain ≥ 0.43).** Over MCP, prefer the five
> frozen memory verbs for the read/write cycle: **`remember(fact, provenance,
> ttl?)`** to save a single durable fact (mandatory provenance; dedupes +
> supersedes), **`recall(query | entity, budget_tokens)`** to read it back
> budget-packed, **`entity(name)`** for a zero-LLM card, **`synthesize(question)`**
> for the expensive cross-page answer, **`forget(id)`** to expire a fact. Use
> `remember` instead of `extract_facts` when you already have ONE formed fact;
> `put_page` / `add_link` / `add_timeline_entry` stay the page/graph write path.
> Fall back to the classic ops when the verbs aren't on the surface. Contract:
> `docs/protocol/MEMORY_VERBS_v1.md`.
## Contract
This skill guarantees:
@@ -56,8 +67,8 @@ broken brain. See `skills/conventions/quality.md` for format.
Before using ANY external API to research a person, company, or topic:
1. `gbrain search "name"` — keyword search for existing pages
2. `gbrain query "natural question about name"` — hybrid search for context
1. `gbrain search "name"` — exact-token lookup for existing pages (cheap hybrid, no expansion)
2. `gbrain query "natural question about name"` — concept/landscape questions go here FIRST (expansion recovers synonym phrasings; a nonzero `search` count is not proof of completeness)
3. `gbrain get <slug>` — if you know the slug, read the full page
4. Check backlinks: who references this entity?
5. Check timeline: recent events involving this entity
@@ -153,8 +164,8 @@ the citation is `[gstack:plans/foo]`. That's the whole rule.
## Tools Used
- `search` — keyword search
- `query` — hybrid vector+keyword search
- `search` — cheap hybrid search (vector + keyword, no expansion)
- `query` — hybrid search + LLM multi-query expansion (concept/landscape questions)
- `get_page` — read a brain page
- `put_page` — create/update brain pages
- `add_link` — cross-reference entities
skills/conventions/brain-first.md
+18 -6
View File
@@ -11,8 +11,8 @@ Your tool inventory includes these (prefixed `gbrain__` in OpenClaw):
| Tool | Use for |
|------|---------|
| `gbrain__search` / `search` | Keyword search — fast, always works |
| `gbrain__query` / `query` | Hybrid search (keyword + semantic) — best quality |
| `gbrain__search` / `search` | Exact tokens / known names — cheap hybrid, no expansion |
| `gbrain__query` / `query` | Concept / landscape questions — hybrid + LLM expansion |
| `gbrain__get_page` / `get_page` | Direct page read when you know the slug |
| `gbrain__get_links` / `get_links` | Outgoing links from a page |
| `gbrain__get_backlinks` / `get_backlinks` | Who references this entity |
@@ -28,10 +28,22 @@ Tool names vary by transport (MCP uses short names, OpenClaw plugin uses
## The Lookup Chain (MANDATORY ORDER)
1. **`search`** first — keyword search, fast, zero API cost
2. **`query`** if search is thin — hybrid semantic search, uses embedding API
3. **`get_page`** if you found a slug — read the full compiled truth
4. **External APIs only after steps 1-2 return nothing useful**
Route by the SHAPE of the question, then escalate:
1. **Exact known token / name / structured field** → **`search`** — cheap
hybrid (vector + keyword, no expansion; embedding-only cost).
2. **Concept / landscape / synonym-phrased question** ("all the X that do Y",
"the landscape of Z") → **`query`** FIRST — multi-query expansion recovers
phrasings `search` misses. Costs one extra LLM expansion call; worth it
for these.
3. **`get_page`** if you found a slug — read the full compiled truth.
4. **External APIs only after steps 1-2 return nothing useful.**
**A nonzero `search` count is NOT a completeness signal.** For "did I capture
everything about X?" run `query` even if `search` already returned hits —
synonym- and outcome-phrased matches drop silently otherwise. And `query` is
still top-K: for literal "list every page that…" enumeration, use `list_pages`
with pagination.
Never skip to external APIs without completing steps 1-2. The brain has
thousands of pages. The answer is almost always there.
skills/query/SKILL.md
+17 -2
View File
@@ -33,6 +33,21 @@ mutating: false
Answer questions using the brain's knowledge with 3-layer search and synthesis.
> **Memory verbs (MEMORY_VERBS v1, gbrain ≥ 0.43).** When connected to a brain
> over MCP, prefer the five frozen memory verbs for memory work — they carry
> provenance, evidence, and a server-enforced token budget:
> - **`recall(query | entity, budget_tokens)`** — the budget-packed memory read.
> Use it instead of bare `search` for "what do we know that we SAVED about X".
> - **`entity(name)`** — a zero-LLM person/company/project card (aliases,
> last-touched, open threads, top edges). Use it instead of `get_page` +
> `get_backlinks` when you just need the card.
> - **`synthesize(question)`** — the explicitly-expensive cross-page answer; the
> heavy version of `query`. Reach for it only when the answer must combine
> evidence across pages.
> Fall back to `search`/`query`/`get_page` when the verbs aren't on the surface
> (pre-0.43 servers; `--surface full` includes the verbs alongside every other
> op). See `docs/protocol/MEMORY_VERBS_v1.md`.
## Contract
This skill guarantees:
@@ -49,8 +64,8 @@ This skill guarantees:
- Semantic query for conceptual questions
- Structured queries (list by type, backlinks) for relational questions
2. **Execute searches:**
- Keyword search gbrain for FTS matches (search)
- Hybrid search gbrain for semantic+keyword with expansion (query)
- Cheap-hybrid search gbrain for exact tokens / known names (search)
- Full-hybrid search gbrain with multi-query expansion for concept questions (query)
- List pages in gbrain by type or check backlinks for structural queries
3. **Read top results.** Read the top 3-5 pages from gbrain to get full context.
4. **Synthesize answer** with citations. Every claim traces back to a specific page slug.
skills/skills.lock.json
+3 -3
View File
@@ -14,7 +14,7 @@
"ask-user/SKILL.md": "a40f484721e548a3a14d4b33a4636111d92f99619ecc4e4ea54c3da3a15f8331",
"book-mirror/SKILL.md": "e8b8cc7a6eba4ecd302a840446b48c0e1aecc245e8f240e233c738daa8dff78a",
"book-mirror/routing-eval.jsonl": "79fd23642cfa37b1255a799907e71bb2904585cf79dd6a596e3a2f019787e54c",
"brain-ops/SKILL.md": "40553ad3bf0f27fc8363b69bec3ef89ae0ea0d9290de8d1efbaa0c532c2a9590",
"brain-ops/SKILL.md": "5f221e3de45845b050b90fac935ac70c55ed5148649fb47c4a30989c1d42c40a",
"brain-pdf/SKILL.md": "13c3e3162763a4503685db0a10663475d3687c4874b5f04d539af83a990f643e",
"brain-pdf/routing-eval.jsonl": "119e4fa113ea45783cee4499e63a729fdeecb4d9a45d47497754b4f5b21d0734",
"brain-taxonomist/SKILL.md": "dea4557b540868ec2c56bf43ee7f63c5d03a22d4047cd0dfbeaf19adef334f60",
@@ -26,7 +26,7 @@
"cold-start/SKILL.md": "33dfc46ad186322823f84efa58589daf0aaea8fffe03da77984e6f0c8494e334",
"concept-synthesis/SKILL.md": "2bc060ae6d706c4e8e7d784cbe3e577b85e211a21c68cba094a1754d3f34436b",
"concept-synthesis/routing-eval.jsonl": "51d1da894158503ce18b892a34edd203f40732e79ac1c0e85141fd37e0b9922f",
"conventions/brain-first.md": "29d020470d0168f8f0b29dde0350a485a9b0472f7ac9962e34948f4897455590",
"conventions/brain-first.md": "14370d89209c7d7e2673c6a4d4e7545fd3330f0744ab598170865a41ae20210b",
"conventions/brain-routing.md": "a8035f7dbadff0ea68b8babb8314b3d044cafbed8242dce5b931fa08b028fc45",
"conventions/calibration.md": "eda7ca76f80c8a17ae546110484389f805c5b21fc0a57f951bbe8b6abba26e03",
"conventions/cron-via-minions.md": "badb1cd6cd825d6f1ac0b6b28cc47e5d80facc783a3e59a14146ae901ee0f933",
@@ -103,7 +103,7 @@
"perplexity-research/SKILL.md": "c25f5c471cbe3c6e0f975d8397e8382b00a85f8aa75302231d53c52855369e97",
"perplexity-research/routing-eval.jsonl": "f1a40d87e710d5d2acd602a372d83f46c95da022b6e635228fffeaacb3bb2b27",
"publish/SKILL.md": "e06b609db780a3cc93a1755a87b30ff08ffdc0fdbc834c1422b2ad2489b57497",
"query/SKILL.md": "e155a08049984c524b838988ba456d16ccedf162442160f6ba66bfd97cd5208e",
"query/SKILL.md": "8672fb9c9315f01274b1a7d3ad35f903df9705b2e806e2fa4c6add027ecef96f",
"query/routing-eval.jsonl": "74f5a91e52fabc54e0e9403fa17db87ee26bb7ebb8ae8005148c51142abc62fe",
"repo-architecture/SKILL.md": "4ec2b8f45d168aaa55f17ecd1ed404ab04217a75c2317f0710c71705846f5394",
"reports/SKILL.md": "5dc190a0c3a2ee518254e8b596418dbe19ff389ea5ec8c8d30fcb0dfef4d0ed5",
src/cli.ts
+277 -6
View File
@@ -30,9 +30,11 @@ import type { Operation, OperationContext } from './core/operations.ts';
import { shouldForceExitAfterMain, finishCliTeardown, flushThenExit, currentExitCode, setCliExitVerdict } from './core/cli-force-exit.ts';
import { serializeMarkdown } from './core/markdown.ts';
import { parseGlobalFlags, setCliOptions, getCliOptions } from './core/cli-options.ts';
import { conceptNudge } from './core/search/query-intent.ts';
import type { CliOptions } from './core/cli-options.ts';
import { callRemoteTool, RemoteMcpError, unpackToolResult } from './core/mcp-client.ts';
import { maybePromptForUpgrade } from './core/thin-client-upgrade-prompt.ts';
import { CLI_FLAG_REGISTRY } from './core/cli-flag-registry.generated.ts';
import { VERSION } from './version.ts';
// Build CLI name -> operation lookup
@@ -54,8 +56,19 @@ export function bigintToStringReplacer(_key: string, value: unknown): unknown {
return typeof value === 'bigint' ? value.toString() : value;
}
// ENG-2 renderer parity: round-trip a local-engine op's return value so
// renderers see the same shape the routed path produces. Bigint-safe via
// bigintToStringReplacer. Exported for tests (same import-safety contract as
// cliAliases/formatResult). (#2450)
export function normalizeLocalResult(rawResult: unknown): unknown {
return JSON.parse(JSON.stringify(rawResult, bigintToStringReplacer));
}
// CLI-only commands that bypass the operation layer
export const CLI_ONLY = new Set(['init', 'reinit-pglite', 'pglite-repair', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'extract-conversation-facts', 'enrich', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'maintain', 'sources', 'mounts', 'dream', 'check-resolvable', 'routing-eval', 'skillify', 'smoke-test', 'providers', 'storage', 'repos', 'code-def', 'code-refs', 'reindex', 'reindex-code', 'reindex-frontmatter', 'code-callers', 'code-callees', 'reconcile-links', 'frontmatter', 'auth', 'friction', 'claw-test', 'book-mirror', 'takes', 'think', 'salience', 'anomalies', 'calibration', 'transcripts', 'models', 'remote', 'recall', 'forget', 'edges-backfill', 'cache', 'ze-switch', 'retrieval-upgrade', 'founder', 'brainstorm', 'lsd', 'schema', 'capture', 'onboard', 'conversation-parser', 'status', 'connect', 'skillopt', 'quarantine', 'self-upgrade', 'advisor', 'watch', 'reindex-search-vector', 'pages', 'bench', 'backfill',
export const CLI_ONLY = new Set(['init', 'reinit-pglite', 'pglite-repair', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'extract-conversation-facts', 'enrich', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'maintain', 'sources', 'mounts', 'dream', 'check-resolvable', 'routing-eval', 'skillify', 'smoke-test', 'providers', 'storage', 'repos', 'code-def', 'code-refs', 'reindex', 'reindex-code', 'reindex-frontmatter', 'code-callers', 'code-callees', 'reconcile-links', 'frontmatter', 'auth', 'friction', 'claw-test', 'book-mirror', 'takes', 'think', 'salience', 'anomalies', 'calibration', 'transcripts', 'models', 'remote', 'recall', 'forget', 'edges-backfill', 'cache', 'ze-switch', 'retrieval-upgrade', 'founder', 'brainstorm', 'lsd', 'schema', 'capture', 'onboard', 'conversation-parser', 'status', 'connect', 'skillopt', 'quarantine', 'self-upgrade', 'protocol', 'advisor', 'watch', 'reindex-search-vector', 'pages', 'bench', 'backfill',
// v0.42.58 (#2035 class, caught by the handleCliOnly reachability sweep):
// full handler at `case 'notability-eval'` but never dispatchable.
'notability-eval',
// Agent-bootstrap family (ENG-2 three-touchpoint rule): `bootstrap` + `hook`
// are ENGINE-FREE (dispatched in handleCliOnly before the connectEngine
// terminator) and must NEVER enter THIN_CLIENT_REFUSED_COMMANDS. `sweep` is
@@ -123,6 +136,9 @@ const CLI_ONLY_SELF_HELP = new Set([
// `gbrain connect --help` prints its own usage (flags + examples) from
// runConnect; route around the generic one-line short-circuit.
'connect',
// MEMORY_VERBS v1 (Cathedral 1): protocol ships its own detailed HELP
// (subcommands, conformance targets, the cost-gated --synthesize flag).
'protocol',
// `gbrain init --help` prints its own usage from runInit; route around the
// generic one-line short-circuit (matches `connect`). Without this, `init`
// is in CLI_ONLY but not CLI_ONLY_SELF_HELP, so the dispatcher's generic
@@ -338,6 +354,30 @@ async function main() {
}
}
// #2185: strict unknown-flag validation — pre-dispatch, pre-engine. A flag
// no handler consults (the repro: `init --migrate-only --dry-run` applying
// REAL migrations while the user asked for a rehearsal) fails loud here
// instead of silently doing the destructive thing. Runs after the --help
// short-circuit so `gbrain x --help` never errors; runs before any dispatch
// or engine connect so the error is instant and side-effect-free.
{
const unknown = validateCommandFlags(command, subArgs);
if (unknown) {
// Message contract shared with init.ts's in-handler check (which this
// pre-dispatch validator now reaches first): lowercase 'unknown flag'
// on stderr; --json callers get the structured error on stdout with
// reason 'invalid_flag' (pinned by test/init-migrate-only.test.ts).
const message = `unknown flag ${unknown} for 'gbrain ${command}'`;
// Both --json spellings get the structured envelope (--json=false opts out).
if (subArgs.some(a => a === '--json' || (a.startsWith('--json=') && a !== '--json=false'))) {
process.stdout.write(JSON.stringify({ status: 'error', reason: 'invalid_flag', message }) + '\n');
}
console.error(`gbrain ${command}: ${message}`);
console.error(`Run: gbrain ${command} --help`);
process.exit(1);
}
}
// DB-free durability pull (v0.42.44 D2): the harden cron calls
// `gbrain sources pull --path <dir>` every ~30 min. It must NOT open PGLite
// (a live long-lived session holds the single-writer lock), so handle it
@@ -522,9 +562,10 @@ async function main() {
// path's return value so renderers see the same shape they'd see on the
// routed path. Date → ISO string; bigint → string (postgres.js shape);
// Buffer → object. Microsecond-cost; eliminates a whole drift bug class.
const result = JSON.parse(JSON.stringify(rawResult, bigintToStringReplacer));
const output = formatResult(op.name, result);
const result = normalizeLocalResult(rawResult);
const output = formatResult(op.name, result, params);
if (output) process.stdout.write(output);
maybePrintConceptNudge(op.name, params);
} catch (e: unknown) {
// v0.42.20.0 (codex D4): on error, set exitCode + return so the `finally`
// STILL runs (drains every background-work sink + disconnects). A bare
@@ -606,8 +647,9 @@ async function runThinClientRouted(
signal: sigintController.signal,
});
const result = unpackToolResult(raw);
const output = formatResult(op.name, result);
const output = formatResult(op.name, result, params);
if (output) process.stdout.write(output);
maybePrintConceptNudge(op.name, params);
} catch (e: unknown) {
if (e instanceof RemoteMcpError) {
const url = cfg.remote_mcp!.mcp_url;
@@ -824,6 +866,29 @@ export function parseOpArgs(op: Operation, args: string[]): Record<string, unkno
for (let i = 0; i < args.length; i++) {
const arg = args[i];
if (arg.startsWith('--')) {
// #2185: `--key=value` inline form. Pre-fix this parsed as junk key
// 'key=value' and consumed the NEXT token as its value, corrupting
// positional parsing. Recognized here so the strict-flag validator and
// the parser agree on the idiom.
const eq = arg.indexOf('=');
if (eq > 2) {
const key = arg.slice(2, eq).replace(/-/g, '_');
// CLI-local booleans: `--json=<v>` / `--dry-run=<v>` must parse as
// booleans, not fall through to the junk-key path (which would
// consume the NEXT token as a value and corrupt positional parsing).
if (key === 'json' || key === 'dry_run') {
params[key] = arg.slice(eq + 1) !== 'false';
continue;
}
const def = op.params[key];
if (def) {
const raw = arg.slice(eq + 1);
params[key] = def.type === 'boolean' ? raw !== 'false'
: def.type === 'number' ? Number(raw)
: raw;
continue;
}
}
if (arg.startsWith('--no-')) {
const positiveKey = arg.slice(5).replace(/-/g, '_');
const positiveDef = op.params[positiveKey];
@@ -836,6 +901,14 @@ export function parseOpArgs(op: Operation, args: string[]): Record<string, unkno
const paramDef = op.params[key];
if (paramDef?.type === 'boolean') {
params[key] = true;
} else if (key === 'json' || key === 'dry_run') {
// CLI-local booleans, intentionally NOT on the operation contract
// exposed over MCP/tools: json is the formatter flag; dry_run feeds
// makeContext's ctx.dryRun. Both must never consume a value token —
// pre-fix, `gbrain delete x --dry-run` (trailing) set NOTHING, so
// ctx.dryRun stayed false and the REAL delete ran despite the
// rehearsal request (the resurrected #2185 class the red team caught).
params[key] = true;
} else if (i + 1 < args.length) {
params[key] = args[++i];
if (paramDef?.type === 'number') params[key] = Number(params[key]);
@@ -997,6 +1070,112 @@ export function applyThinClientSourceScope(
}
// Exported for tests (same import-safety contract as applyThinClientSourceScope).
// ─────────────────────────────────────────────────────────────────
// #2185 — strict unknown-flag validation (pre-dispatch, pre-engine).
// A flag no handler consults must fail loud instead of silently doing the
// destructive thing (`init --migrate-only --dry-run` applied REAL migrations
// while the user asked for a rehearsal). Two lanes:
// - op commands: legal flags derive from the operation contract
// (op.params) + the CLI-local formatter flags, mirroring parseOpArgs's
// traversal so values that begin with '--' are never misread.
// - CLI_ONLY commands: legal flags come from the generated
// CLI_FLAG_REGISTRY (scripts/generate-flag-registry.ts scans each
// command's source; freshness + coverage pinned by
// test/cli-flag-validation.test.ts).
// Everything after a literal `--` is passthrough and never validated.
// ─────────────────────────────────────────────────────────────────
// Exempt by contract, not oversight:
// - call: the generic op invoker — arbitrary --param names are its interface.
// - config: `config set <key> <value>` values are arbitrary strings.
// - jobs submit: job payloads carry handler-defined params (shell lane incl.).
function flagValidationExempt(command: string, subArgs: string[]): boolean {
return command === 'call' || command === 'config'
|| (command === 'jobs' && subArgs[0] === 'submit');
}
/** Returns the first unknown flag (e.g. '--dry-run') or null when clean. */
export function validateCommandFlags(command: string, subArgs: string[]): string | null {
if (flagValidationExempt(command, subArgs)) return null;
// Lane order MUST mirror dispatch order (CLI_ONLY first): commands that are
// BOTH an op and a CLI_ONLY member (think, salience, anomalies) dispatch to
// handleCliOnly, whose handlers parse flags the op contract doesn't declare
// (`salience --kind`, `think --with-calibration`) — validating those
// against op.params rejected documented invocations.
if (CLI_ONLY.has(command)) {
const legal = CLI_FLAG_REGISTRY[command];
// Registry drift fails OPEN at runtime (never brick a command); the
// drift-guard test fails the build instead.
if (!legal) return null;
return findUnknownFlag(subArgs, new Set(legal));
}
const op = cliOps.get(command) ?? cliAliases.get(command);
if (op) return findUnknownOpFlag(op, subArgs);
return null; // unknown command — the dispatcher's own error handles it
}
/** CLI_ONLY lane: token scan against the generated legal set. */
export function findUnknownFlag(args: string[], legal: ReadonlySet<string>): string | null {
for (const a of args) {
if (a === '--') break;
const m = /^--([a-z0-9][a-z0-9-]*)(?:=.*)?$/i.exec(a);
if (!m) continue;
// Casing typo = unknown flag: every handler in the repo is
// case-sensitive-lowercase, so `--MIGRATE-ONLY` passing validation would
// just be silently ignored downstream — the exact class this validator
// exists to kill.
if (/[A-Z]/.test(m[1])) return `--${m[1]}`;
const name = `--${m[1]}`;
if (legal.has(name)) continue;
// --no-<flag> negation of a known flag is legal.
if (name.startsWith('--no-') && legal.has(`--${name.slice(5)}`)) continue;
return name;
}
return null;
}
/** Op lane: mirrors parseOpArgs so flag VALUES starting with '--' are skipped. */
export function findUnknownOpFlag(op: Operation, args: string[]): string | null {
for (let i = 0; i < args.length; i++) {
const a = args[i];
if (a === '--') break;
const m = /^--([a-z0-9][a-z0-9-]*)(?:=(.*))?$/i.exec(a);
if (!m) continue;
// Casing typo = unknown flag (see findUnknownFlag).
if (/[A-Z]/.test(m[1])) return `--${m[1]}`;
const rawKey = m[1];
// CLI-local flags consumed OUTSIDE the op contract (never wire params):
// json/explain — formatter flags; help — short-circuits pre-dispatch;
// source — makeContext's 6-tier source resolution (deleted before wire);
// dry-run — makeContext's ctx.dryRun projection.
// Pre-fix, rejecting these broke documented invocations
// (`gbrain search "x" --source y`, `gbrain put x --dry-run`).
if (rawKey === 'json') continue;
if ((rawKey === 'explain' || rawKey === 'help') && m[2] === undefined) continue;
if (rawKey === 'source' || rawKey === 'dry-run') {
// Non-boolean-style CLI-locals consume the next token as their value
// in parseOpArgs (source does; dry-run is boolean-read) — mirror the
// parser: source consumes a value when not inline-`=`.
if (rawKey === 'source' && m[2] === undefined) i++;
continue;
}
if (rawKey.startsWith('no-')) {
const positive = rawKey.slice(3).replace(/-/g, '_');
if (op.params[positive]?.type === 'boolean') continue;
}
const key = rawKey.replace(/-/g, '_');
const paramDef = op.params[key];
if (paramDef) {
// Non-boolean flags consume the next token as their value unless
// provided inline via `=` — exactly like parseOpArgs.
if (paramDef.type !== 'boolean' && m[2] === undefined) i++;
continue;
}
return `--${rawKey}`;
}
return null;
}
export async function makeContext(engine: BrainEngine, params: Record<string, unknown>): Promise<OperationContext> {
// v0.31.8 (D11): resolve sourceId via the canonical 6-tier chain. Honors
// --source / GBRAIN_SOURCE / .gbrain-source / path-match / brain default /
@@ -1051,7 +1230,25 @@ export async function makeContext(engine: BrainEngine, params: Record<string, un
}
// Exported for tests (same import-safety contract as cliAliases/printOpHelp).
export function formatResult(opName: string, result: unknown): string {
/**
* #2416: hint-only steering — a concept-shaped `search` gets a one-line
* stderr nudge toward `query`. Never fires for other ops, never reroutes
* (search stays the cheap hot path), and honors --quiet — the same silence
* discipline as the identity banner. Called from BOTH result paths (local
* engine + thin-client routed); formatResult can't host this because it
* never sees the query text.
*/
export function maybePrintConceptNudge(opName: string, params: Record<string, unknown>): void {
if (opName !== 'search' || getCliOptions().quiet) return;
const nudge = conceptNudge(String(params.query ?? ''));
if (nudge) process.stderr.write(nudge + '\n');
}
export function formatResult(
opName: string,
result: unknown,
params: Record<string, unknown> = {},
): string {
switch (opName) {
case 'volunteer_context': {
const r = result as any;
@@ -1090,6 +1287,7 @@ export function formatResult(opName: string, result: unknown): string {
case 'search':
case 'query': {
const results = result as any[];
if (params.json === true) return JSON.stringify(results, null, 2) + '\n';
if (results.length === 0) return 'No results.\n';
// v0.40.4 — --explain switches to per-stage attribution formatter.
// Reads CliOptions.explain via the module-level singleton.
@@ -1175,9 +1373,70 @@ export function formatResult(opName: string, result: unknown): string {
`#${v.id} ${v.snapshot_at?.toString().slice(0, 19) || '?'} ${v.compiled_truth?.slice(0, 60) || ''}...`,
).join('\n') + '\n';
}
// MEMORY_VERBS v1 [F-E]: human-readable by default; trailing `--json`
// escapes to the raw envelope (parseOpArgs ignores an unmatched trailing
// flag, so the argv probe is safe).
case 'remember': {
if (process.argv.includes('--json')) break;
const r = result as any;
if (r.dry_run) return `[dry-run] would remember: ${r.fact}\n`;
const lines = [r.status_text || `${r.status} (fact #${r.id})`];
if (r.entity_slug) lines.push(` entity: ${r.entity_slug}`);
if (r.valid_until) lines.push(` expires: ${r.valid_until}`);
if (r.degraded_dedup) lines.push(' note: no embedding provider — duplicate detection degraded');
return lines.join('\n') + '\n';
}
case 'entity': {
if (process.argv.includes('--json')) break;
const r = result as any;
if (!r.found) {
const lines = [`No entity found. (${r.latency_ms}ms)`];
if (Array.isArray(r.suggestions) && r.suggestions.length) {
lines.push('Did you mean:');
for (const s of r.suggestions) lines.push(` ${s.slug} — ${s.title} [${s.create_safety}]`);
}
return lines.join('\n') + '\n';
}
const c = r.card;
const lines = [`${c.entity.title} (${c.entity.slug})${c.entity.type ? ` [${c.entity.type}]` : ''} (${r.latency_ms}ms)`];
if (c.summary) lines.push(` ${c.summary}`);
if (c.aka?.length) lines.push(` aka: ${c.aka.join(', ')}`);
const lt = c.last_touched || {};
const touched = lt.updated_at || lt.last_retrieved_at || lt.last_timeline_date;
if (touched) lines.push(` last touched: ${String(touched).slice(0, 10)}`);
if (c.open_threads?.length) {
lines.push(' open threads:');
for (const t of c.open_threads) lines.push(` [${t.kind}] ${t.text}${t.date ? ` (${String(t.date).slice(0, 10)})` : ''}`);
}
if (c.edges?.length) {
lines.push(' edges:');
for (const e of c.edges) lines.push(` ${e.direction === 'out' ? '→' : '←'} ${e.type} ${e.slug}`);
}
lines.push(` backlinks: ${c.backlink_count} | active facts: ${c.active_fact_count}`);
if (Array.isArray(r.suggestions) && r.suggestions.length) {
lines.push(' other matches:');
for (const s of r.suggestions) lines.push(` ${s.slug} — ${s.title}`);
}
return lines.join('\n') + '\n';
}
case 'synthesize': {
if (process.argv.includes('--json')) break;
const r = result as any;
const lines = [r.answer || '(no answer)'];
if (Array.isArray(r.sources) && r.sources.length) lines.push('', `sources: ${r.sources.join(', ')}`);
if (Array.isArray(r.gaps) && r.gaps.length) lines.push(`gaps: ${r.gaps.join('; ')}`);
const cost = r.cost || {};
const tok = cost.input_tokens != null ? `${cost.input_tokens} in / ${cost.output_tokens} out` : 'tokens n/a';
const usd = cost.usd_estimate != null ? ` (~$${Number(cost.usd_estimate).toFixed(4)})` : '';
lines.push(`cost: ${cost.model} — ${tok}${usd}`);
return lines.join('\n') + '\n';
}
default:
return JSON.stringify(result, null, 2) + '\n';
// bigintToStringReplacer keeps this fallback renderer crash-proof even
// if a future caller hands it a not-yet-normalized result. (#2450)
return JSON.stringify(result, bigintToStringReplacer, 2) + '\n';
}
return JSON.stringify(result, null, 2) + '\n';
}
/**
@@ -1311,6 +1570,14 @@ async function handleCliOnly(command: string, args: string[]) {
await runSchema(args);
return;
}
// MEMORY_VERBS v1 (Cathedral 1): protocol introspection + conformance +
// local usage stats. No pre-bound engine — conformance spawns its own
// server; stats reads the local JSONL sidecar.
if (command === 'protocol') {
const { runProtocol } = await import('./commands/protocol.ts');
await runProtocol(args);
return;
}
if (command === 'init') {
const { runInit } = await import('./commands/init.ts');
await runInit(args);
@@ -2801,9 +3068,13 @@ ADMIN
features [--json] [--auto-fix] Scan usage + recommend unused features
autopilot [--repo] [--interval N] Self-maintaining brain daemon
config [show|get|set] <key> [val] Brain config
protocol [conformance|stats] MEMORY_VERBS v1: schemas, conformance
certification, local usage stats + TTHW
storage status [--repo <path>] Storage tier status and health
[--json] (git-tracked vs supabase-only)
serve MCP server (stdio)
--surface verbs|full Tool surface: the 5 memory verbs only, or
every op (default full; verbs = quickstart)
serve --http [--port N] HTTP MCP server with OAuth 2.1
--token-ttl N Access token TTL in seconds (default: 3600)
--enable-dcr Enable Dynamic Client Registration (DCR clients default to authorization_code)
src/commands/doctor.ts
+74 -8
View File
@@ -383,19 +383,38 @@ export async function jsonbIntegrityCheck(
progress?: Pick<ProgressReporter, 'heartbeat'>,
): Promise<Check> {
try {
const targets: Array<{ table: string; col: string; expected: 'object' | 'array' }> = [
{ table: 'pages', col: 'frontmatter', expected: 'object' },
{ table: 'raw_data', col: 'data', expected: 'object' },
{ table: 'ingest_log', col: 'pages_updated', expected: 'array' },
{ table: 'files', col: 'metadata', expected: 'object' },
{ table: 'page_versions', col: 'frontmatter', expected: 'object' },
const targets: Array<{ table: string; col: string; expected: 'object' | 'array'; jsonPayloadOnly?: boolean }> = [
{ table: 'pages', col: 'frontmatter', expected: 'object' },
{ table: 'raw_data', col: 'data', expected: 'object' },
{ table: 'ingest_log', col: 'pages_updated', expected: 'array' },
{ table: 'files', col: 'metadata', expected: 'object' },
{ table: 'page_versions', col: 'frontmatter', expected: 'object' },
// Subagent persistence — second double-encode site (historical damage
// rows from the pre-v0.42.53.0 positional bind; write paths fixed in
// #2375). Mirrors repair-jsonb's targets incl. jsonPayloadOnly: these
// columns can legitimately hold jsonb STRING scalars (persistToolExec
// binds pre-serialized string payloads as-is), so only JSON-container
// content counts as damage.
{ table: 'subagent_messages', col: 'content_blocks', expected: 'array', jsonPayloadOnly: true },
{ table: 'subagent_tool_executions', col: 'input', expected: 'object', jsonPayloadOnly: true },
{ table: 'subagent_tool_executions', col: 'output', expected: 'object', jsonPayloadOnly: true },
];
let totalBad = 0;
const breakdown: string[] = [];
for (const { table, col } of targets) {
for (const { table, col, jsonPayloadOnly } of targets) {
progress?.heartbeat(`jsonb_integrity.${table}.${col}`);
// Skip targets whose table doesn't exist on this brain (subagent_*
// tables are v0.15+; pre-v0.15 brains naturally lack them).
const existsRows = await engine.executeRaw<{ exists: boolean }>(
`SELECT to_regclass($1) IS NOT NULL AS exists`,
[table],
);
if (!existsRows[0]?.exists) continue;
const damage = jsonPayloadOnly
? `jsonb_typeof(${col}) = 'string' AND (${col} #>> '{}') ~ '^[[:space:]]*[\\[{]' AND pg_input_is_valid(${col} #>> '{}', 'jsonb')`
: `jsonb_typeof(${col}) = 'string'`;
const rows = await engine.executeRaw<{ n: number }>(
`SELECT count(*)::int AS n FROM ${table} WHERE jsonb_typeof(${col}) = 'string'`,
`SELECT count(*)::int AS n FROM ${table} WHERE ${damage}`,
);
const n = Number(rows[0]?.n ?? 0);
if (n > 0) { totalBad += n; breakdown.push(`${table}.${col}=${n}`); }
@@ -4932,6 +4951,46 @@ export async function computePoolReapHealthCheck(
* Policy-skill install state is reported in details (it ships into the HOST
* repo, so absence in gbrain's own skills dir is expected, not a failure).
*/
/**
* MEMORY_VERBS v1 (Cathedral 1, E4) — usage-sidecar health. Read-only,
* fail-open. Stats only (local JSONL, never uploaded; never source of truth):
* - no sidecar file → ok, "no verb calls recorded yet" (fresh install)
* - recent events parse → ok, names the last verb + timestamp
* - file exists, unreadable→ warn (observability degraded, verbs unaffected)
*/
export async function buildMemoryVerbsCheck(): Promise<Check> {
const name = 'memory_verbs_usage';
try {
const { readVerbUsage, usageLogPath } = await import('../core/verbs/usage-log.ts');
if (!existsSync(usageLogPath())) {
return {
name,
status: 'ok',
message: 'no verb calls recorded yet (sidecar appears on first remember/recall/entity/synthesize/forget)',
};
}
const events = await readVerbUsage({ days: 30 });
if (events.length === 0) {
return { name, status: 'ok', message: 'sidecar present; no verb calls in the last 30 days' };
}
const last = events[events.length - 1];
const byVerb = new Map<string, number>();
for (const e of events) byVerb.set(e.verb, (byVerb.get(e.verb) ?? 0) + 1);
const mix = [...byVerb.entries()].map(([v, n]) => `${v}:${n}`).join(' ');
return {
name,
status: 'ok',
message: `${events.length} verb calls in 30d (${mix}); last ${last.verb} at ${last.ts} — local JSONL only, never uploaded`,
};
} catch (e) {
return {
name,
status: 'warn',
message: `verb usage sidecar unreadable (${e instanceof Error ? e.message : String(e)}) — observability degraded; verbs unaffected`,
};
}
}
export function buildRetrievalReflexCheck(skillsDir: string | null): Check {
const name = 'retrieval_reflex_health';
try {
@@ -5125,6 +5184,13 @@ export async function buildChecks(
checks.push(buildRetrievalReflexCheck(skillsDir));
}
// 1c. MEMORY_VERBS v1 usage sidecar health (Cathedral 1, E4). Read-only,
// fail-open: reports whether the local JSONL sidecar is present + parseable
// and when a verb last fired. Local file only — never uploaded.
if (scope === 'all') {
checks.push(await buildMemoryVerbsCheck());
}
// 2. Skill conformance (SKILL group — gated)
if (scope === 'all' && skillsDir) {
const conformanceResult = skillConformanceCheck(skillsDir);
src/commands/init.ts
+27
View File
@@ -1049,6 +1049,9 @@ async function initPGLite(opts: {
// gbrain invocation). mode_prompted=true so the upgrade-time banner doesn't
// also fire on a fresh install. Hands-off: gbrain config set self_upgrade.mode auto
config.self_upgrade = { mode: 'notify', mode_prompted: true, ...(config.self_upgrade ?? {}) };
// MEMORY_VERBS v1 [D6C]: TTHW stamp — `gbrain protocol stats` derives
// install→first-verb-call from this. Idempotent on re-init.
config.protocol_installed_at = config.protocol_installed_at ?? new Date().toISOString();
saveConfig(config);
if (opts.schemaPack) {
process.stderr.write(
@@ -1086,6 +1089,7 @@ async function initPGLite(opts: {
} else {
console.log('Next: gbrain import <dir>');
}
printMemoryVerbsQuickstart();
console.log('');
console.log('When you outgrow local: gbrain migrate --to supabase');
reportModStatus();
@@ -1103,6 +1107,26 @@ async function initPGLite(opts: {
}
}
/**
* MEMORY_VERBS v1 quickstart funnel (E3 + D4B + T1 consent). Printed at the
* end of both init epilogues. The copy-next block is EXACTLY three commands
* (codex DX 9): wire the harness, write a memory, prove the resurrection.
* The demo uses the facts arm only, so it works with NO embedding key [F-B].
*/
function printMemoryVerbsQuickstart(): void {
console.log('');
console.log('Give your agent memory (copy these three commands):');
console.log(' claude mcp add gbrain -- gbrain serve --surface verbs');
console.log(' gbrain remember "I prefer dark mode in every editor" --provenance demo --entity people/me');
console.log(' gbrain recall --entity people/me');
console.log('Now ask your agent in a NEW session — it remembers.');
console.log('');
console.log('Note: memories agents save are readable by every agent connected to');
console.log('this brain; use visibility:"private" for local-only facts.');
console.log('Other harnesses (Codex, OpenClaw): docs/protocol/MEMORY_VERBS_v1.md');
console.log('If `claude` is not found: install Claude Code first, or use the per-harness blocks in that doc.');
}
async function initPostgres(opts: {
databaseUrl: string;
jsonOutput: boolean;
@@ -1297,6 +1321,8 @@ async function initPostgres(opts: {
// gbrain invocation). mode_prompted=true so the upgrade-time banner doesn't
// also fire on a fresh install. Hands-off: gbrain config set self_upgrade.mode auto
config.self_upgrade = { mode: 'notify', mode_prompted: true, ...(config.self_upgrade ?? {}) };
// MEMORY_VERBS v1 [D6C]: TTHW stamp (see the PGLite path).
config.protocol_installed_at = config.protocol_installed_at ?? new Date().toISOString();
saveConfig(config);
console.log('Config saved to ~/.gbrain/config.json');
if (opts.schemaPack) {
@@ -1331,6 +1357,7 @@ async function initPostgres(opts: {
} else {
console.log('Next: gbrain import <dir>');
}
printMemoryVerbsQuickstart();
reportModStatus();
const { printAdvisoryIfRecommended } = await import('../core/skillpack/post-install-advisory.ts');
const { VERSION } = await import('../version.ts');
src/commands/migrations/index.ts
+2
View File
@@ -26,6 +26,7 @@ import { v0_28_0 } from './v0_28_0.ts';
import { v0_29_1 } from './v0_29_1.ts';
import { v0_31_0 } from './v0_31_0.ts';
import { v0_32_2 } from './v0_32_2.ts';
import { v0_43_0 } from './v0_43_0.ts';
export const migrations: Migration[] = [
v0_11_0,
@@ -43,6 +44,7 @@ export const migrations: Migration[] = [
v0_29_1,
v0_31_0,
v0_32_2,
v0_43_0,
];
/** Look up a migration by exact version string. */
src/commands/migrations/v0_43_0.ts
+33
View File
@@ -0,0 +1,33 @@
/**
* v0.43.0.0 migration — MEMORY_VERBS v1 (Cathedral 1).
*
* PITCH-ONLY. There is NO schema or data migration: the five frozen memory
* verbs (recall/remember/entity/synthesize/forget) ride the existing facts,
* pages, and typed-graph tables. This entry exists solely so `gbrain
* post-upgrade` / the self-upgrade NOTIFY channel actively tells an existing
* install that the verbs landed and how to switch a harness onto them — the
* propagation path the verbs otherwise lacked (the default surface stays
* `full`, so an upgrade alone does NOT steer agents to the verbs).
*
* The orchestrator is a no-op that reports `complete` immediately —
* idempotent by construction (it does nothing), so apply-migrations records
* the ledger row and moves on.
*/
import type { Migration, OrchestratorOpts, OrchestratorResult } from './types.ts';
async function orchestrator(_opts: OrchestratorOpts): Promise<OrchestratorResult> {
// No schema/data work — MEMORY_VERBS v1 is a façade over existing tables.
return { version: '0.43.0', status: 'complete', phases: [] };
}
export const v0_43_0: Migration = {
version: '0.43.0',
featurePitch: {
headline:
'Five memory verbs — recall, remember, entity, synthesize, forget — are now the agent-facing memory protocol (MEMORY_VERBS v1).',
description:
'Point any MCP harness at `gbrain serve --surface verbs` to expose exactly these five self-describing tools instead of the full op wall: remember(fact, provenance) writes durable facts; recall(query|entity, budget_tokens) returns budget-packed memory; entity(name) is a zero-LLM card; synthesize(question) is the explicitly-expensive cross-page answer; forget(id) expires a fact. Existing `gbrain serve` (full surface) keeps working and now also lists the verbs, but `--surface verbs` is the clean agent surface. Set a default with `gbrain config set mcp_surface verbs`. Verify any endpoint with `gbrain protocol conformance`; see usage with `gbrain protocol stats`. Full contract: docs/protocol/MEMORY_VERBS_v1.md.',
},
orchestrator,
};
src/commands/protocol.ts
+304
View File
@@ -0,0 +1,304 @@
/**
* MEMORY_VERBS v1 — `gbrain protocol` (Cathedral 1, E2 + E4 + D6C).
*
* gbrain protocol [--json] machine-readable verb schemas + version
* gbrain protocol conformance [...] certify an MCP endpoint against the contract
* gbrain protocol stats [--days N] local verb usage + TTHW (never uploaded)
*
* Conformance targets:
* (default) spawn gbrain's own stdio server (self-certify)
* --target http://host/mcp Streamable HTTP endpoint [--token gbrain_xxx]
* --target "cmd arg arg" spawn any stdio MCP server (space-split; for
* commands with complex quoting, certify via a
* small wrapper script)
*
* Input schemas emit from the LIVE Operation defs (doc/code can't drift);
* response shapes come from the hand-authored RESPONSE_SCHEMAS registry,
* which conformance validates LIVE responses against — registry-vs-code
* drift is caught by the same fixtures that certify servers [c8].
*/
import { operationsByName } from '../core/operations.ts';
import {
RESPONSE_SCHEMAS,
ERROR_SCHEMA,
MEMORY_VERBS_VERSION,
VERB_NAMES,
type VerbName,
} from '../core/verbs.ts';
import { buildToolDefs } from '../mcp/tool-defs.ts';
import { runConformance, type ConformanceClient } from '../core/verbs/conformance.ts';
import { readVerbUsage, earliestVerbUsageTs, usageLogPath } from '../core/verbs/usage-log.ts';
import { loadConfig } from '../core/config.ts';
import { setCliExitVerdict } from '../core/cli-force-exit.ts';
const HELP = `gbrain protocol — the MEMORY_VERBS v1 wire contract (frozen, additive-forever)
Usage:
gbrain protocol [--json] Print the protocol: verb input schemas
(from live defs), response schemas, error
contract, version. --json for machines.
gbrain protocol conformance Certify an MCP endpoint against the
contract. Default target: gbrain's own
stdio server (self-certification).
--target http://host:3131/mcp HTTP MCP endpoint (add --token gbrain_xxx)
--target "bun run src/cli.ts serve" Any stdio MCP server command
--synthesize Also live-call synthesize (costs money
when an LLM key is configured; without
one it asserts the clean 'unavailable'
error — what CI does)
--json Machine-readable report
gbrain protocol stats [--days N] Per-verb usage, error rate, budget drops,
entity hit rate, TTHW (install -> first
verb call). Local JSONL only — this data
never leaves the machine. Default 30 days.
Docs: docs/protocol/MEMORY_VERBS_v1.md
Why default surface is 'full': verbs is for agents and quickstarts
(gbrain serve --surface verbs); full preserves existing advanced tooling.`;
export async function runProtocol(args: string[]): Promise<void> {
if (args.includes('--help') || args.includes('-h')) {
console.log(HELP);
return;
}
const sub = args[0] && !args[0].startsWith('--') ? args[0] : null;
if (sub === 'conformance') {
await runConformanceCommand(args.slice(1));
return;
}
if (sub === 'stats') {
await runStatsCommand(args.slice(1));
return;
}
if (sub === null) {
printProtocol(args.includes('--json'));
return;
}
console.error(`Unknown protocol subcommand: ${sub}`);
console.log(HELP);
// [ship P1.3] PGLite/WASM clobbers process.exitCode; the CLI exit seam reads
// the gbrain-owned verdict (setCliExitVerdict), not process.exitCode.
setCliExitVerdict(1);
}
// ─── protocol [--json] ───────────────────────────────────────────────────────
function buildProtocolDocument() {
const verbOps = VERB_NAMES.map(n => operationsByName[n]).filter(Boolean);
const toolDefs = buildToolDefs(verbOps);
const verbs: Record<string, unknown> = {};
for (const def of toolDefs) {
verbs[def.name] = {
description: def.description,
input_schema: def.inputSchema,
...(def.annotations ? { annotations: def.annotations } : {}),
response_schema: RESPONSE_SCHEMAS[def.name as VerbName],
};
}
return {
protocol: 'MEMORY_VERBS',
protocol_version: MEMORY_VERBS_VERSION,
versioning_policy:
'additive-forever: v1 field names and semantics never change; new optional fields/params may be added; breaking changes require MEMORY_VERBS_v2 (expected never)',
verbs,
error_schema: ERROR_SCHEMA,
};
}
function printProtocol(json: boolean): void {
const doc = buildProtocolDocument();
if (json) {
console.log(JSON.stringify(doc, null, 2));
return;
}
console.log(`MEMORY_VERBS v${MEMORY_VERBS_VERSION} — the frozen memory protocol (additive-forever)\n`);
for (const name of VERB_NAMES) {
const op = operationsByName[name];
if (!op) continue;
const params = Object.entries(op.params)
.map(([k, v]) => `${k}${v.required ? '' : '?'}`)
.join(', ');
console.log(` ${name}(${params})`);
console.log(` ${op.description.split('. ')[0]}.`);
}
console.log(`\nFull schemas: gbrain protocol --json`);
console.log(`Doc: docs/protocol/MEMORY_VERBS_v1.md`);
}
// ─── protocol conformance ────────────────────────────────────────────────────
async function runConformanceCommand(args: string[]): Promise<void> {
const json = args.includes('--json');
const synthesize = args.includes('--synthesize');
const targetIdx = args.indexOf('--target');
const target = targetIdx >= 0 ? args[targetIdx + 1] : null;
const tokenIdx = args.indexOf('--token');
const token = tokenIdx >= 0 ? args[tokenIdx + 1] : null;
const { Client } = await import('@modelcontextprotocol/sdk/client/index.js');
const client = new Client({ name: 'gbrain-conformance', version: '1.0.0' }, { capabilities: {} });
let transport: { close(): Promise<void> };
if (target && /^https?:\/\//.test(target)) {
const { StreamableHTTPClientTransport } = await import('@modelcontextprotocol/sdk/client/streamableHttp.js');
const t = new StreamableHTTPClientTransport(new URL(target), {
...(token ? { requestInit: { headers: { Authorization: `Bearer ${token}` } } } : {}),
});
await client.connect(t);
transport = t;
} else {
const { StdioClientTransport } = await import('@modelcontextprotocol/sdk/client/stdio.js');
let command: string;
let cmdArgs: string[];
if (target) {
const parts = target.split(/\s+/).filter(Boolean);
command = parts[0];
cmdArgs = parts.slice(1);
} else {
// Self-certification: spawn our own server. Dev (bun run src/cli.ts)
// vs compiled binary both resolve to "this gbrain, serve".
const entry = process.argv[1] ?? '';
if (entry.endsWith('.ts')) {
command = process.execPath;
cmdArgs = ['run', entry, 'serve'];
} else {
command = process.execPath;
cmdArgs = ['serve'];
}
}
const env: Record<string, string> = {};
for (const [k, v] of Object.entries(process.env)) if (v !== undefined) env[k] = v;
const t = new StdioClientTransport({ command, args: cmdArgs, env });
await client.connect(t);
transport = t;
}
const adapter: ConformanceClient = {
listTools: async () => {
const { tools } = await client.listTools();
return tools.map(t => ({
name: t.name,
description: t.description,
annotations: (t as { annotations?: unknown }).annotations,
}));
},
callTool: async (name, callArgs) => {
const res = (await client.callTool({ name, arguments: callArgs })) as {
isError?: boolean;
content?: Array<{ type?: string; text?: string }>;
};
const text = (res.content ?? []).map(c => (typeof c.text === 'string' ? c.text : '')).join('\n');
return { isError: res.isError, text };
},
};
try {
const report = await runConformance(adapter, { synthesize });
if (json) {
console.log(JSON.stringify(report, null, 2));
} else {
console.log(`MEMORY_VERBS v${report.protocol_version} conformance — target: ${target ?? 'self (stdio)'}\n`);
for (const r of report.results) {
const mark = r.status === 'pass' ? '✓' : r.status === 'skip' ? '−' : '✗';
console.log(` ${mark} ${r.name}${r.detail ? ` — ${r.detail}` : ''}`);
}
console.log(`\n${report.passed} passed, ${report.failed} failed, ${report.skipped} skipped`);
console.log(report.ok ? 'CONFORMANT' : 'NOT CONFORMANT');
}
// [ship P1.3] non-conformant target MUST exit non-zero so CI fails. The
// CLI exit seam reads setCliExitVerdict (process.exitCode is unreliable on
// PGLite/WASM and ignored by the force-exit path).
if (!report.ok) setCliExitVerdict(1);
} finally {
try { await client.close(); } catch { /* best-effort */ }
try { await transport.close(); } catch { /* best-effort */ }
}
}
// ─── protocol stats ──────────────────────────────────────────────────────────
async function runStatsCommand(args: string[]): Promise<void> {
const json = args.includes('--json');
const daysIdx = args.indexOf('--days');
const days = daysIdx >= 0 ? parseInt(args[daysIdx + 1] ?? '30', 10) || 30 : 30;
const events = await readVerbUsage({ days });
const byVerb = new Map<string, { calls: number; errors: number; latency: number; budgetDropped: number; entityFound: number; entityMiss: number }>();
for (const e of events) {
const v = byVerb.get(e.verb) ?? { calls: 0, errors: 0, latency: 0, budgetDropped: 0, entityFound: 0, entityMiss: 0 };
v.calls += 1;
if (!e.ok) v.errors += 1;
v.latency += e.latency_ms;
if (typeof e.budget_dropped === 'number') v.budgetDropped += e.budget_dropped;
if (e.entity_found === true) v.entityFound += 1;
if (e.entity_found === false) v.entityMiss += 1;
byVerb.set(e.verb, v);
}
// TTHW [D6C]: install stamp → first verb call, the real measured number the
// post-ship boomerang review compares against the 2–5 min target.
const cfg = loadConfig();
const installedAt = cfg?.protocol_installed_at ?? null;
const firstCall = await earliestVerbUsageTs();
let tthw: string | null = null;
if (installedAt && firstCall) {
const deltaMs = Date.parse(firstCall) - Date.parse(installedAt);
if (Number.isFinite(deltaMs) && deltaMs >= 0) tthw = formatDuration(deltaMs);
}
if (json) {
console.log(JSON.stringify({
days,
total_calls: events.length,
by_verb: Object.fromEntries(
[...byVerb.entries()].map(([k, v]) => [k, {
calls: v.calls,
errors: v.errors,
avg_latency_ms: v.calls ? Math.round(v.latency / v.calls) : 0,
budget_dropped_total: v.budgetDropped,
...(k === 'entity' ? { found: v.entityFound, miss: v.entityMiss } : {}),
}]),
),
tthw_install_to_first_verb: tthw,
installed_at: installedAt,
first_verb_call: firstCall,
sidecar: usageLogPath(),
privacy: 'local JSONL only — never uploaded',
}, null, 2));
return;
}
console.log(`MEMORY_VERBS usage — last ${days} days (local JSONL only, never uploaded)\n`);
if (events.length === 0) {
console.log(' no verb calls recorded yet');
} else {
for (const [verb, v] of [...byVerb.entries()].sort((a, b) => b[1].calls - a[1].calls)) {
const extras = [
v.errors ? `${v.errors} errors` : null,
v.budgetDropped ? `${v.budgetDropped} budget-dropped` : null,
verb === 'entity' && (v.entityFound || v.entityMiss)
? `hit ${v.entityFound}/${v.entityFound + v.entityMiss}`
: null,
].filter(Boolean).join(', ');
console.log(` ${verb.padEnd(11)} ${String(v.calls).padStart(5)} calls avg ${Math.round(v.latency / v.calls)}ms${extras ? ` (${extras})` : ''}`);
}
}
if (tthw) console.log(`\n TTHW: first verb call ${tthw} after install`);
else if (installedAt) console.log(`\n TTHW: no verb calls since install (${installedAt})`);
console.log(` sidecar: ${usageLogPath()}`);
}
function formatDuration(ms: number): string {
const s = Math.round(ms / 1000);
if (s < 60) return `${s}s`;
const m = Math.floor(s / 60);
if (m < 60) return `${m}m${s % 60 ? `${s % 60}s` : ''}`;
const h = Math.floor(m / 60);
if (h < 48) return `${h}h${m % 60 ? `${m % 60}m` : ''}`;
return `${Math.floor(h / 24)}d`;
}
/** Test seam: the protocol document, without printing. */
export { buildProtocolDocument };
src/commands/recall.ts
+78
View File
@@ -66,6 +66,10 @@ interface ParsedFlags {
json: boolean;
source: string;
limit: number;
// MEMORY_VERBS v1 [c4]: recall's verb params, routed through the recall OP
// (this hand-rolled CLI otherwise ignores unknown flags silently).
query: string | null;
budgetTokens: number | null;
// v0.32
sinceLastRun: boolean;
pending: boolean;
@@ -94,6 +98,8 @@ function parseFlags(args: string[]): ParsedFlags {
json: false,
source: 'default',
limit: 50,
query: null,
budgetTokens: null,
sinceLastRun: false,
pending: false,
rollup: false,
@@ -112,6 +118,8 @@ function parseFlags(args: string[]): ParsedFlags {
if (a === '--json') { out.json = true; continue; }
if (a === '--source') { out.source = args[++i] ?? 'default'; continue; }
if (a === '--limit') { out.limit = parseInt(args[++i] ?? '50', 10) || 50; continue; }
if (a === '--query') { out.query = args[++i] ?? null; continue; }
if (a === '--budget-tokens') { out.budgetTokens = parseInt(args[++i] ?? '', 10) || null; continue; }
if (a === '--since-last-run') { out.sinceLastRun = true; continue; }
if (a === '--pending') { out.pending = true; continue; }
if (a === '--rollup') { out.rollup = true; continue; }
@@ -223,6 +231,13 @@ export async function runRecall(engine: BrainEngine, args: string[]): Promise<vo
const sourceId = await resolveSourceForRecall(engine, flags.source, thinClient);
// MEMORY_VERBS v1 [c4]: the verb params route through the recall OP so the
// CLI and MCP exercise the same arm (query/budget packing/superset envelope).
if (flags.query !== null || flags.budgetTokens !== null) {
await runRecallVerb(engine, flags, sourceId);
return;
}
if (flags.watchSeconds !== null) {
await runWatchLoop(engine, flags, sourceId, thinClient, flags.watchSeconds);
return;
@@ -230,6 +245,69 @@ export async function runRecall(engine: BrainEngine, args: string[]): Promise<vo
await runRecallOnce(engine, flags, sourceId, thinClient, 'briefing');
}
/**
* MEMORY_VERBS v1 — `gbrain recall --query ... [--budget-tokens N]` routes
* through the recall OP (same code path MCP exercises) and renders facts +
* search results with the budget footer. `--json` prints the raw envelope.
*/
async function runRecallVerb(engine: BrainEngine, flags: ParsedFlags, sourceId: string): Promise<void> {
const { operationsByName } = await import('../core/operations.ts');
const op = operationsByName['recall'];
const ctx = {
engine,
config: loadConfig() || { engine: 'pglite' as const },
logger: {
info: (m: string) => process.stderr.write(`[info] ${m}\n`),
warn: (m: string) => process.stderr.write(`[warn] ${m}\n`),
error: (m: string) => process.stderr.write(`[error] ${m}\n`),
},
dryRun: false,
remote: false as const,
sourceId,
};
const result = (await op.handler(ctx, {
...(flags.entity ? { entity: flags.entity } : {}),
...(flags.query ? { query: flags.query } : {}),
...(flags.budgetTokens ? { budget_tokens: flags.budgetTokens } : {}),
...(flags.since ? { since: flags.since.toISOString() } : {}),
...(flags.grep ? { grep: flags.grep } : {}),
include_expired: flags.includeExpired,
limit: flags.limit,
})) as {
facts: Array<{ fact_id: string; fact: string; kind: string; entity_slug: string | null; provenance: string }>;
results?: Array<{ slug: string; title: string | null; evidence: string; chunk: string | null }>;
search_degraded?: string;
budget_tokens?: number;
budget_used?: number;
dropped_count?: number;
};
if (flags.json) {
process.stdout.write(JSON.stringify(result, null, 2) + '\n');
return;
}
const lines: string[] = [];
if (result.facts.length) {
lines.push('Facts:');
for (const f of result.facts) {
lines.push(` #${f.fact_id} [${f.kind}]${f.entity_slug ? ` (${f.entity_slug})` : ''} ${f.fact} — ${f.provenance}`);
}
}
if (result.results?.length) {
lines.push('Pages:');
for (const r of result.results) {
lines.push(` ${r.slug} [${r.evidence}] ${r.title ?? ''}`);
if (r.chunk) lines.push(` ${r.chunk.replace(/\s+/g, ' ').slice(0, 160)}`);
}
}
if (!lines.length) lines.push('Nothing recalled.');
if (result.search_degraded) lines.push(`note: search degraded (${result.search_degraded})`);
if (result.budget_tokens !== undefined) {
lines.push(`budget: ${result.budget_used}/${result.budget_tokens} tokens used, ${result.dropped_count} dropped`);
}
process.stdout.write(lines.join('\n') + '\n');
}
async function runRecallOnce(
engine: BrainEngine,
flags: ParsedFlags,
src/commands/repair-jsonb.ts
+71 -16
View File
@@ -16,16 +16,27 @@
* (it never wrote string-typed JSONB).
*
* Affected columns (audit of src/schema.sql):
* - pages.frontmatter (postgres-engine.ts:107 putPage)
* - raw_data.data (postgres-engine.ts:668 putRawData)
* - ingest_log.pages_updated (postgres-engine.ts:846 logIngest)
* - files.metadata (commands/files.ts:254 file upload)
* - page_versions.frontmatter (downstream of pages.frontmatter via
* INSERT...SELECT FROM pages)
* - pages.frontmatter (postgres-engine.ts:107 putPage)
* - raw_data.data (postgres-engine.ts:668 putRawData)
* - ingest_log.pages_updated (postgres-engine.ts:846 logIngest)
* - files.metadata (commands/files.ts:254 file upload)
* - page_versions.frontmatter (downstream of pages.frontmatter via
* INSERT...SELECT FROM pages)
* - subagent_messages.content_blocks (subagent.ts:599 persistMessage —
* v0.16.0+, write path fixed in
* v0.42.53.0 #2375)
* - subagent_tool_executions.input (subagent.ts:625/660 persistToolExec
* Pending/Failed — same wave)
* - subagent_tool_executions.output (subagent.ts:639 persistToolExecComplete
* — same wave)
*
* Other JSONB columns (minion_jobs.{data,result,progress,stacktrace},
* minion_inbox.payload) were always written via parameterized form ($N::jsonb
* with a string parameter, not interpolation) so they were never affected.
* The subagent_* writes were broken via a slightly different shape than the
* v0.12.0 wave: they used `engine.executeRaw` (postgres.js `unsafe`) with
* `JSON.stringify(value)` + `$N::jsonb` cast. postgres.js's unsafe path
* binds the resulting string as text, then the `::jsonb` cast wraps it as
* a jsonb string scalar instead of parsing it. queue.ts and other
* `executeRaw` callers were not affected because they pass raw objects
* (postgres.js v3 auto-encodes objects to jsonb).
*/
import { loadConfig, toEngineConfig } from '../core/config.ts';
@@ -39,16 +50,40 @@ interface RepairTarget {
column: string;
/** Optional secondary key column for logging. */
keyCol?: string;
/**
* Only unwrap string scalars whose CONTENT is a JSON container ({...} or
* [...]). The subagent columns can legitimately hold jsonb string scalars
* (persistToolExec binds `typeof input === 'string' ? input : stringify`,
* so a tool's pre-serialized plain-text payload lands as a JSON string) —
* an unconditional unwrap would cast non-JSON text and abort the entire
* repair run, or corrupt a legitimate value on a second pass.
*/
jsonPayloadOnly?: boolean;
}
const TARGETS: RepairTarget[] = [
{ table: 'pages', column: 'frontmatter', keyCol: 'slug' },
{ table: 'raw_data', column: 'data', keyCol: 'source' },
{ table: 'ingest_log', column: 'pages_updated', keyCol: 'source_ref' },
{ table: 'files', column: 'metadata', keyCol: 'storage_path' },
{ table: 'page_versions', column: 'frontmatter', keyCol: 'snapshot_at' },
{ table: 'pages', column: 'frontmatter', keyCol: 'slug' },
{ table: 'raw_data', column: 'data', keyCol: 'source' },
{ table: 'ingest_log', column: 'pages_updated', keyCol: 'source_ref' },
{ table: 'files', column: 'metadata', keyCol: 'storage_path' },
{ table: 'page_versions', column: 'frontmatter', keyCol: 'snapshot_at' },
{ table: 'subagent_messages', column: 'content_blocks', keyCol: 'job_id', jsonPayloadOnly: true },
{ table: 'subagent_tool_executions', column: 'input', keyCol: 'tool_use_id', jsonPayloadOnly: true },
{ table: 'subagent_tool_executions', column: 'output', keyCol: 'tool_use_id', jsonPayloadOnly: true },
];
/** The double-encode predicate for a target (see jsonPayloadOnly). */
function damagePredicate(t: RepairTarget): string {
const base = `jsonb_typeof(${t.column}) = 'string'`;
// Container-looking is not enough: '[INFO] fetch complete' matches the
// shape probe but is NOT valid JSON — the repair cast would throw and
// abort the run. pg_input_is_valid (PG16+, same floor as the IS JSON
// predicate updateSourceConfig already relies on) gates on parseability.
return t.jsonPayloadOnly
? `${base} AND (${t.column} #>> '{}') ~ '^[[:space:]]*[\\[{]' AND pg_input_is_valid(${t.column} #>> '{}', 'jsonb')`
: base;
}
export interface RepairResult {
engine: string;
per_target: Array<{
@@ -113,20 +148,40 @@ export async function repairJsonb(opts: RepairOpts = { dryRun: false }): Promise
let repaired = 0;
try {
// Skip targets whose table doesn't exist yet — relevant for the
// v0_12_2 migration on pre-v0.15 brains (subagent_* tables hadn't
// been added yet) and any future schema additions to TARGETS.
const existsRows = await sql.unsafe(
`SELECT to_regclass($1) IS NOT NULL AS exists`,
[t.table],
) as Array<{ exists: boolean }>;
if (!existsRows[0]?.exists) {
progress.tick(1, `${t.table}.${t.column}=skipped(no-table)`);
result.per_target.push({ table: t.table, column: t.column, rows_repaired: 0 });
continue;
}
if (opts.dryRun) {
const rows = await sql.unsafe(
`SELECT count(*)::int AS n FROM ${t.table} WHERE jsonb_typeof(${t.column}) = 'string'`,
`SELECT count(*)::int AS n FROM ${t.table} WHERE ${damagePredicate(t)}`,
);
repaired = (rows[0] as unknown as { n: number }).n;
} else {
const rows = await sql.unsafe(
`UPDATE ${t.table}
SET ${t.column} = (${t.column} #>> '{}')::jsonb
WHERE jsonb_typeof(${t.column}) = 'string'
WHERE ${damagePredicate(t)}
RETURNING 1`,
);
repaired = rows.length;
}
} catch (e) {
// One target's failure (unexpected content shape, permission, etc.)
// must not abort the remaining targets — earlier repairs are already
// committed and the v0_12_2 migration orchestrator JSON-parses our
// stdout. Record, report, continue.
console.error(`[repair-jsonb] ${t.table}.${t.column} failed: ${(e as Error).message} — continuing with remaining targets`);
repaired = 0;
} finally {
stopHb();
}
src/commands/serve-http.ts
+42 -3
View File
@@ -30,8 +30,10 @@ import type { OperationContext, AuthInfo } from '../core/operations.ts';
import { GBrainOAuthProvider, validateTokenEndpointAuthMethod } from '../core/oauth-provider.ts';
import type { SqlQuery } from '../core/oauth-provider.ts';
import { hasScope, ALLOWED_SCOPES_LIST, normalizeScopesInput } from '../core/scope.ts';
import { normalizeSourceInput, normalizeFederatedReadInput } from '../core/source-id.ts';
import { summarizeMcpParams, dispatchToolCall } from '../mcp/dispatch.ts';
import { paramDefToSchema } from '../mcp/tool-defs.ts';
import { filterOpsForSurface } from '../mcp/surface.ts';
import { getBrainHotMemoryMeta } from '../core/facts/meta-hook.ts';
import { loadConfig } from '../core/config.ts';
import { buildError, serializeError } from '../core/errors.ts';
@@ -474,6 +476,12 @@ interface ServeHttpOptions {
* tracking the regenerated value through other means.
*/
suppressBootstrapToken?: boolean;
/**
* MEMORY_VERBS v1: tool-surface mode. 'verbs' = exactly the five protocol
* verbs; 'full' (default) = every non-localOnly operation. Enforced on the
* tool list AND in dispatch (fail-closed).
*/
surface?: 'verbs' | 'full';
/**
* #2624: force-print the generated admin bootstrap token even on a
* non-TTY (containerized) start. By default the raw token is only printed
@@ -1652,7 +1660,7 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption
// and other malformed inputs
// normalizeScopesInput handles all four valid shapes (string, string[],
// missing, empty) and rejects the rest with a structured 400.
const { name, tokenTtl, grantTypes, redirectUris, tokenEndpointAuthMethod } = req.body;
const { name, source, federatedRead, tokenTtl, grantTypes, redirectUris, tokenEndpointAuthMethod } = req.body;
const rawScopes = (req.body as Record<string, unknown>).scopes ?? (req.body as Record<string, unknown>).scope;
if (!name) { res.status(400).json({ error: 'Name required' }); return; }
let scopeString: string;
@@ -1684,8 +1692,27 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption
});
return;
}
// v0.41.x: honor optional `source` (write source_id) and `federatedRead`
// (read source set) from the request body, mirroring the CLI's
// `--source` / `--federated-read` flags. Omitting both preserves the
// historical behavior (source_id='default', federated_read=[source_id]).
// Pre-fix this endpoint hardcoded 'default'/undefined, so an admin SPA or
// a proxy could never mint a client bound to a non-default brain source
// over HTTP — only the CLI could. Validated here for a structured 400.
let sourceId: string;
let federatedReadIds: string[] | undefined;
try {
sourceId = normalizeSourceInput(source);
federatedReadIds = normalizeFederatedReadInput(federatedRead);
} catch (e) {
res.status(400).json({
error: 'invalid_source',
message: e instanceof Error ? e.message : String(e),
});
return;
}
const result = await oauthProvider.registerClientManual(
name, grants, scopeString, uris, 'default', undefined, validatedAuthMethod,
name, grants, scopeString, uris, sourceId, federatedReadIds, validatedAuthMethod,
);
// Set per-client TTL if specified
if (tokenTtl && Number(tokenTtl) > 0) {
@@ -1839,7 +1866,13 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption
// ---------------------------------------------------------------------------
// MCP tool calls (bearer auth + scope enforcement)
// ---------------------------------------------------------------------------
const mcpOperations = operations.filter(op => !op.localOnly);
// MEMORY_VERBS v1: surface filter applies AFTER the localOnly filter; the
// same set feeds dispatch as allowedOps so hidden ops are uncallable, not
// just unlisted [c2].
const surface = options.surface ?? 'full';
const mcpOperations = filterOpsForSurface(operations.filter(op => !op.localOnly), surface);
const surfaceAllowedOps: ReadonlySet<string> | undefined =
surface === 'full' ? undefined : new Set(mcpOperations.map(o => o.name));
// v0.36.x #1076: MCP Streamable HTTP spec — GET /mcp opens an optional SSE
// backchannel for server-initiated messages. gbrain's transport is stateless
@@ -1902,6 +1935,9 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption
),
required: Object.entries(op.params).filter(([, v]) => v.required).map(([k]) => k),
},
// MEMORY_VERBS v1: ToolAnnotations emitted only when the op defines
// them — existing tools stay byte-identical (mirrors buildToolDefs).
...(op.annotations ? { annotations: op.annotations } : {}),
})),
};
});
@@ -2027,6 +2063,9 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption
takesHoldersAllowList: tokenAllowList,
sourceId: tokenSourceId,
metaHook: getBrainHotMemoryMeta,
// MEMORY_VERBS v1: fail-closed surface enforcement + usage attribution.
...(surfaceAllowedOps ? { allowedOps: surfaceAllowedOps } : {}),
surface,
// v0.31 follow-up fix: thread auth so the whoami op (and any
// future scope-aware handlers) can introspect the caller. The
// original D12/eE1 refactor moved dispatch into dispatchToolCall
src/commands/serve.ts
+15 -4
View File
@@ -56,7 +56,7 @@ export interface ServeOptions {
// (which unconditionally attaches a 'data' listener to real
// process.stdin and would pollute the test runner's stdin handle).
// Defaults to the real implementation when omitted.
startMcpServer?: (engine: BrainEngine) => Promise<void>;
startMcpServer?: (engine: BrainEngine, opts?: { surface?: 'verbs' | 'full' }) => Promise<void>;
// Test seam for the parent-process watchdog. The default
// (`readLiveParentPid`) reads the live kernel PPID via `ps` on POSIX
// because `process.ppid` is captured at process creation and does not
@@ -170,6 +170,13 @@ export async function runServe(
// that used `gbrain auth create` keep working unchanged).
const isHttp = args.includes('--http');
// MEMORY_VERBS v1: tool-surface mode. Flag > config `mcp_surface` > 'full'.
// 'verbs' exposes exactly the five protocol verbs (the quickstart surface);
// 'full' (default) keeps every operation — existing installs see no change.
const { parseSurfaceFlag, resolveSurface } = await import('../mcp/surface.ts');
const { loadConfig } = await import('../core/config.ts');
const surface = resolveSurface(parseSurfaceFlag(args), loadConfig());
if (isHttp) {
const portIdx = args.indexOf('--port');
const port = portIdx >= 0 ? parseInt(args[portIdx + 1]) || 3131 : 3131;
@@ -216,7 +223,7 @@ export async function runServe(
const printAdminToken = args.includes('--print-admin-token');
const { runServeHttp } = await import('./serve-http.ts');
await runServeHttp(engine, { port, tokenTtl, enableDcr, enableDcrInsecure, publicUrl, logFullParams, bind, suppressBootstrapToken, printAdminToken });
await runServeHttp(engine, { port, tokenTtl, enableDcr, enableDcrInsecure, publicUrl, logFullParams, bind, suppressBootstrapToken, printAdminToken, surface });
await finishHttpServe(engine, opts);
return;
@@ -227,7 +234,11 @@ export async function runServe(
// trigger graceful release of the PGLite write lock held by `engine`.
// The HTTP / OAuth path above has its own lifecycle in serve-http.ts
// and is intentionally NOT wired into this stdio plumbing.
console.error('Starting GBrain MCP server (stdio)...');
console.error(
surface === 'verbs'
? 'Starting GBrain MCP server (stdio) — serving 5 memory verbs (MEMORY_VERBS v1)...'
: 'Starting GBrain MCP server (stdio)...',
);
installStdioLifecycle(engine, args, opts);
@@ -265,7 +276,7 @@ export async function runServe(
}
try {
await start(engine);
await start(engine, { surface });
} finally {
if (bootDeadline) clearTimeout(bootDeadline);
}
src/core/brainstorm/orchestrator.ts
+5 -3
View File
@@ -252,9 +252,11 @@ export interface BrainstormResult {
// ---------------------------------------------------------------------------
/**
* Per-profile cost estimate. brainstorm: ~$0.05-0.15. lsd: ~$0.20-0.40.
* Real numbers depend on configured model; we anchor on Sonnet pricing.
* The estimate is informational — operators see actuals printed at run-end.
* Per-profile cost estimate. At Sonnet pricing ($3/M in, $15/M out — the
* gateway fallback), this formula yields brainstorm ~$0.8 and lsd ~$1.0;
* it scales linearly with the configured chat model's pricing (a Haiku 4.5
* chat_model at $1/$5 lands exactly 3x lower). The estimate is
* informational — operators see actuals printed at run-end.
*/
export function estimateCost(profile: BrainstormProfile, model: string): number {
const crosses = profile.k_close * profile.m_far;
src/core/cli-flag-registry.generated.ts
+106
View File
@@ -0,0 +1,106 @@
// AUTO-GENERATED by scripts/generate-flag-registry.ts — do not edit by hand.
// Regenerate: bun run build:flag-registry
// Freshness + drift pinned by test/cli-flag-validation.test.ts (#2185).
//
// Per-command legal flags for CLI_ONLY commands, derived from each command's
// source (case block + imported modules + one level of relative imports +
// scripts/generate-flag-registry.ts EXTRA_FLAGS). Deliberately over-inclusive
// (help-text mentions count): accepting an ignored flag is the pre-#2185
// status quo; missing a real one breaks working invocations.
export const CLI_FLAG_REGISTRY: Record<string, readonly string[]> = {
'advisor': ['--aliases', '--all', '--apply', '--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--skills-dir', '--source', '--stale', '--supersessions', '--surface', '--thin', '--verbose', '--workspace', '--yes'],
'agent': ['--aliases', '--all', '--brain', '--detach', '--fanout-manifest', '--flag', '--flags', '--follow', '--help', '--include-null-signature', '--json', '--max-turns', '--model', '--no-extract', '--no-follow', '--note', '--pattern', '--pending', '--reset', '--resolve', '--since', '--source', '--stale', '--subagent-def', '--supersessions', '--thin', '--timeout-ms', '--tools', '--word'],
'anomalies': ['--aliases', '--all', '--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--lookback-days', '--mcp-only', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--sigma', '--since', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout'],
'apply-migrations': ['--ab', '--all', '--auto-update', '--brain', '--break-lock', '--build-index', '--by-mention', '--compile', '--days', '--dry-run', '--exclusive', '--fast', '--force', '--force-all', '--force-orchestrator', '--force-retry', '--force-schema', '--from-meetings', '--from-pages', '--help', '--history', '--host-dir', '--http', '--json', '--lang', '--list', '--locks', '--markdown', '--max-age', '--migrate-only', '--migration', '--mode', '--multimodal', '--no-autopilot-install', '--no-embedding', '--no-extract', '--non-interactive', '--phase', '--priority', '--refresh-unqualified', '--remediate', '--rollback', '--skip-verify', '--source', '--stale', '--surface', '--undo-wave', '--use-captured-snapshot', '--with-calibration', '--yes'],
'auth': ['--aliases', '--all', '--bound-brain', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--enable-dcr', '--enable-dcr-insecure', '--fast', '--federated-read', '--force', '--from-pages', '--grant-types', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--pattern', '--pending', '--redirect-uri', '--reset', '--resolve', '--scopes', '--source', '--stale', '--supersessions', '--surface', '--takes-holders', '--thin', '--token', '--token-endpoint-auth-method', '--yes'],
'autopilot': ['--aliases', '--all', '--auto-fix', '--batch', '--brain', '--break-lock', '--by-type', '--check', '--dimensions', '--dir', '--drain', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--ff-only', '--fix', '--force', '--force-break-lock', '--force-retry', '--from-pages', '--full', '--help', '--http', '--include-null-signature', '--inject-bootstrap', '--inline', '--input', '--install', '--interval', '--json', '--markdown', '--max-age', '--max-rss', '--max-usd', '--migrate-only', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--no-extract', '--no-inject', '--no-mutate', '--no-worker', '--non-interactive', '--now', '--once', '--output', '--path', '--pattern', '--pending', '--phase', '--refresh-cache', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--source', '--stale', '--status', '--supersessions', '--surface', '--swap-only', '--target', '--target-score', '--thin', '--timeout', '--to', '--uninstall', '--unsafe-bypass-dream-guard', '--user', '--version', '--yes'],
'backfill': ['--aliases', '--all', '--batch-size', '--brain', '--concurrency', '--dry-run', '--fresh', '--help', '--include-null-signature', '--json', '--keep-index', '--list', '--max-errors', '--max-rows', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--resume', '--source', '--stale', '--supersessions', '--thin'],
'bench': ['--baseline', '--brain', '--explain', '--force', '--from', '--help', '--json', '--label', '--lang', '--limit', '--markdown', '--multimodal', '--near-symbol', '--restore-only', '--source', '--stale', '--symbol-kind', '--thin', '--threshold-jaccard', '--threshold-latency-multiplier', '--threshold-top1', '--to', '--tool'],
'book-mirror': ['--aliases', '--all', '--allow-empty', '--apply', '--asof', '--author', '--auto', '--background', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--brain-wide-max-cost-usd', '--budget-usd-per-day', '--by-mention', '--chapters-dir', '--content', '--context-file', '--date', '--days', '--dry-run', '--entities', '--explain', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--help', '--http', '--image', '--include-null-signature', '--json', '--kind', '--limit', '--max-turns', '--max-usd', '--mode', '--model', '--multimodal', '--no-confirm', '--no-embedding', '--no-extract', '--no-follow', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--save', '--session', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--timeout', '--timeout-ms', '--title', '--trusted-extraction', '--url', '--with-db', '--yes'],
'brainstorm': ['--aliases', '--all', '--brain', '--chunker-debug', '--code', '--compile', '--fast', '--fix', '--force', '--force-rechunk', '--force-resume', '--from-pages', '--full', '--help', '--http', '--include-null-signature', '--json', '--judge-model', '--lang', '--limit', '--list-runs', '--markdown', '--max-cost', '--max-far-set', '--max-ideas-per-judge-call', '--model', '--no-embed', '--no-embedding', '--no-extract', '--no-save', '--pattern', '--pending', '--reset', '--resolve', '--resume', '--retry-failed', '--retry-judge', '--save', '--source', '--stale', '--strict-budget', '--supersessions', '--surface', '--thin', '--timeout', '--yes'],
'cache': ['--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--no-embedding', '--source', '--surface', '--yes'],
'calibration': ['--ab', '--aliases', '--all', '--allow-empty', '--apply', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--dry-run', '--entities', '--explain', '--fast', '--federated', '--file', '--force', '--from-pages', '--help', '--holder', '--http', '--image', '--include-null-signature', '--json', '--key-prefix', '--kind', '--lang', '--limit', '--markdown', '--max-usd', '--mode', '--multimodal', '--near-symbol', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--phase', '--progress-interval', '--progress-json', '--quiet', '--regenerate', '--repo', '--reset', '--resolve', '--restore-only', '--save', '--scrub-gstack', '--session', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--symbol-kind', '--thin', '--trusted-extraction', '--undo-wave', '--url', '--with-calibration', '--with-db', '--yes'],
'call': ['--aliases', '--all', '--all-sources', '--as-context', '--auto-fix', '--background', '--brain', '--by-mention', '--catch-up', '--concurrency', '--confirm-destructive', '--content', '--cost-estimate', '--count', '--days', '--depth', '--dim', '--dir', '--direction', '--enable-dcr', '--enable-dcr-insecure', '--explain', '--fast', '--federated', '--file', '--fix', '--follow', '--force', '--from', '--from-meetings', '--grant-types', '--grep', '--hard-deadline', '--help', '--http', '--image', '--include-expired', '--include-frontmatter', '--include-null-signature', '--infer-dates', '--install', '--interval', '--json', '--key', '--kind', '--lang', '--limit', '--link-source', '--link-type', '--migrate-only', '--missing-path', '--multimodal', '--ner', '--no-embed', '--no-expand', '--no-extract', '--no-federated', '--no-hard-deadline', '--no-retry-connect', '--no-save', '--older-than', '--page', '--param', '--params', '--password', '--path', '--pattern', '--pending', '--pglite', '--port', '--progress-interval', '--progress-json', '--public-url', '--queue', '--quiet', '--reenrich-after', '--refresh-cache', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--restore-only', '--save', '--scopes', '--session', '--sigma', '--since', '--slug', '--slug-prefix', '--source', '--source-id', '--stale', '--status', '--stdin', '--strategy', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--synthesize', '--tag', '--thin', '--timeout', '--to', '--today', '--token', '--token-ttl', '--tools-json', '--type', '--url', '--version', '--watch', '--with-calibration', '--workers', '--yes'],
'capture': ['--aliases', '--all', '--allow-empty', '--apply', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--depth', '--entities', '--fast', '--federated', '--file', '--force', '--from-pages', '--help', '--http', '--image', '--include-null-signature', '--json', '--kind', '--limit', '--max-usd', '--mcp-only', '--mode', '--multimodal', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--repo', '--reset', '--resolve', '--restore-only', '--save', '--scopes', '--session', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--stdin', '--supersessions', '--surface', '--thin', '--timeout', '--trusted-extraction', '--type', '--url', '--what', '--where', '--who', '--with-db', '--yes'],
'check-backlinks': ['--background', '--brain', '--brain-wide-max-cost-usd', '--dir', '--dry-run', '--explain', '--follow', '--help', '--include-frontmatter', '--json', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--source', '--stale', '--timeout', '--type'],
'check-resolvable': ['--brain', '--dry-run', '--fix', '--help', '--json', '--skills-dir', '--source', '--strict', '--verbose'],
'check-update': ['--all', '--brain', '--check', '--ff-only', '--help', '--json', '--markdown', '--migrate-only', '--non-interactive', '--refresh-cache', '--source', '--swap-only', '--to', '--version', '--yes'],
'claw-test': ['--agent', '--brain', '--dir', '--help', '--json', '--keep-tempdir', '--list-agents', '--live', '--local', '--message', '--no-embed', '--no-embedding', '--path', '--pglite', '--progress-json', '--prompt-file', '--run-id', '--scenario', '--source', '--transcripts'],
'code-callees': ['--aliases', '--all', '--all-sources', '--brain', '--chunker-debug', '--clone-dir', '--confirm-destructive', '--federated', '--force', '--help', '--include-null-signature', '--json', '--limit', '--no-extract', '--no-federated', '--no-json', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--thin', '--url', '--url-managed', '--yes'],
'code-callers': ['--aliases', '--all', '--all-sources', '--brain', '--chunker-debug', '--clone-dir', '--confirm-destructive', '--federated', '--force', '--help', '--include-null-signature', '--json', '--limit', '--no-extract', '--no-federated', '--no-json', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--thin', '--url', '--url-managed', '--yes'],
'code-def': ['--aliases', '--all', '--brain', '--chunker-debug', '--help', '--include-null-signature', '--json', '--lang', '--limit', '--no-extract', '--no-json', '--pattern', '--pending', '--pretty', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--yes'],
'code-refs': ['--aliases', '--all', '--brain', '--chunker-debug', '--help', '--include-null-signature', '--json', '--lang', '--limit', '--no-extract', '--no-json', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--yes'],
'config': ['--aliases', '--all', '--brain', '--column', '--coverage-override', '--embedding-dimensions', '--embedding-model', '--fast', '--follow', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--markdown', '--model', '--multimodal', '--no-embedding', '--no-extract', '--pattern', '--pending', '--pglite', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--surface', '--thin', '--yes'],
'connect': ['--agent', '--bearer-token-env-var', '--bind', '--brain', '--client-id', '--client-secret', '--force', '--grant-types', '--help', '--http', '--install', '--json', '--name', '--oauth', '--public-url', '--register', '--scopes', '--show-token', '--source', '--timeout-ms', '--token', '--token-endpoint-auth-method', '--url', '--version', '--yes'],
'conversation-parser': ['--aliases', '--all', '--brain', '--help', '--include-null-signature', '--json', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin'],
'doctor': ['--ab', '--abi', '--aliases', '--all', '--allow-shell-jobs', '--auto', '--auto-fix', '--auto-update', '--background', '--batch', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--build-index', '--by-mention', '--by-type', '--check', '--column', '--compile', '--concurrency', '--confidence', '--content-audit', '--count', '--days', '--detach', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--exclusive', '--explain', '--fast', '--fix', '--follow', '--force', '--force-break-lock', '--force-retry', '--force-schema', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--grant-types', '--health-interval', '--help', '--history', '--http', '--include-flagged', '--include-frontmatter', '--include-null-signature', '--include-pseudo', '--index-audit', '--input', '--json', '--lang', '--limit', '--locks', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-crashes', '--max-jobs', '--max-rss', '--max-usd', '--mcp-only', '--migrate-only', '--model', '--multimodal', '--name-only', '--name-status', '--near-symbol', '--nice', '--no', '--no-embed', '--no-embedding', '--no-extract', '--no-mutate', '--oauth-client-secret', '--older-than', '--once', '--overwrite', '--parallel', '--params', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--priority', '--progress-interval', '--progress-json', '--query', '--queue', '--quiet', '--rebuild-rollup', '--refresh', '--refresh-unqualified', '--regenerate', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--restore-only', '--resume', '--review-lower', '--rollback', '--scope', '--scopes', '--since', '--skills-dir', '--skip-bare-tweet', '--skip-failed', '--skip-urls', '--skip-verify', '--slugs', '--source', '--source-id', '--stale', '--status', '--strategy', '--strict', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--target', '--target-score', '--thin', '--timeout', '--to', '--top-k', '--type', '--undo-wave', '--unsafe-bypass-dream-guard', '--untracked-files', '--url', '--use-captured-snapshot', '--verbose', '--version', '--window', '--with-calibration', '--workers', '--yes'],
'dream': ['--against', '--aliases', '--all', '--anchor', '--asof', '--background', '--batch', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--budget-usd', '--by-type', '--by-type-floor', '--code', '--compile', '--concurrent', '--ctx-size', '--cycles', '--date', '--dimensions', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--embeddings', '--expansion', '--explain', '--fast', '--federated', '--fix', '--fixtures', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--force-retry', '--from', '--from-db', '--from-pages', '--help', '--http', '--include-null-signature', '--input', '--install', '--json', '--judge-model', '--keyword-only', '--lang', '--limit', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--max-tokens', '--max-usd', '--mcp-only', '--min-recall', '--mode', '--model', '--models', '--multimodal', '--name', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-llm', '--no-mutate', '--no-trajectory', '--once', '--output', '--path', '--pattern', '--pending', '--pglite', '--phase', '--priority', '--progress-interval', '--progress-json', '--pull', '--quiet', '--receipt-dir', '--remediate', '--repo', '--reranking', '--reset', '--resolve', '--restore-only', '--resume-from', '--retrieval-only', '--rounds', '--rubric-version', '--save', '--seed', '--skip-replay', '--slot-a-model', '--slot-b-model', '--slot-c-model', '--slug', '--slug-prefix', '--source', '--source-id', '--stale', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--take', '--task', '--thin', '--threshold', '--timeout', '--to', '--top-k', '--undo', '--unsafe-bypass-dream-guard', '--version', '--window', '--yes'],
'edges-backfill': ['--aliases', '--all', '--all-sources', '--brain', '--concurrency', '--federated', '--help', '--include-null-signature', '--json', '--max-age', '--max-chunks', '--max-cost-usd', '--no-extract', '--no-federated', '--older-than', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--thin', '--timeout', '--workers'],
'embed': ['--aliases', '--all', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--catch-up', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--fix', '--follow', '--force', '--force-break-lock', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--lang', '--markdown', '--max-age', '--max-cost-usd', '--model', '--multimodal', '--name', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--pace', '--pace-max-concurrency', '--parallel', '--path', '--pattern', '--pending', '--pglite', '--prefix', '--priority', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--restore-only', '--serial', '--slugs', '--source', '--stale', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--thin', '--timeout', '--version'],
'enrich': ['--aliases', '--all', '--all-sources', '--allow-empty', '--apply', '--asof', '--auto', '--background', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--budget-usd-per-day', '--by-mention', '--clone-dir', '--code', '--concurrency', '--confirm-destructive', '--content', '--date', '--days', '--dry-run', '--embedding-dimensions', '--embedding-model', '--entities', '--explain', '--fast', '--federated', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--from-pages', '--help', '--http', '--image', '--include-null-signature', '--json', '--judge-model', '--kind', '--lang', '--limit', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--max-usd', '--min-context', '--mode', '--model', '--multimodal', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--offset', '--older-than', '--order', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--reenrich-after', '--remediate', '--reset', '--resolve', '--restore-only', '--resume', '--save', '--session', '--since', '--slug', '--slugs', '--source', '--source-id', '--stale', '--stats', '--supersessions', '--surface', '--symbol-kind', '--thin', '--thin-threshold', '--timeout', '--trusted-extraction', '--types', '--url', '--url-managed', '--version', '--with-db', '--workers', '--yes'],
'eval': ['--ab-relational', '--against', '--aliases', '--all', '--background', '--baseline', '--batch', '--brain', '--brain-wide-max-cost-usd', '--budget-usd', '--budget-usd-answer', '--budget-usd-retrieval', '--compare', '--compare-limit', '--concurrent', '--config-a', '--config-b', '--corpus', '--cycles', '--days', '--dedup-cosine', '--dedup-max-per-page', '--dedup-type-ratio', '--dimensions', '--distance-min', '--embedding-dimensions', '--embedding-model', '--expand', '--explain', '--fast', '--fixtures', '--follow', '--force', '--from-capture', '--from-db', '--from-pages', '--grounding-min', '--help', '--http', '--include-null-signature', '--input', '--json', '--judge', '--k', '--limit', '--max-pair-chars', '--max-tokens', '--max-usd', '--md', '--metric', '--min-recall', '--mode', '--model', '--models', '--modes', '--multimodal', '--name', '--no', '--no-cache', '--no-embed', '--no-embedding', '--no-expand', '--no-extract', '--no-llm', '--older-than', '--output', '--output-dir', '--parallel', '--pattern', '--pending', '--progress-interval', '--progress-json', '--qrels', '--queries-file', '--query', '--questions', '--quiet', '--receipt-dir', '--refresh-cache', '--remediate', '--reset', '--resolve', '--rrf-k', '--rubric-version', '--runs', '--sampling', '--save', '--seed', '--severity', '--short', '--show-toplevel', '--since', '--skip-replay', '--slot-a-model', '--slot-b-model', '--slot-c-model', '--slug', '--slug-prefix', '--source', '--stale', '--strategy', '--strict', '--suite', '--suites', '--supersessions', '--surface', '--task', '--thin', '--threshold', '--threshold-expected-top1', '--threshold-first-relevant-hit', '--threshold-jaccard', '--threshold-latency-multiplier', '--threshold-recall-at-k', '--threshold-top1', '--timeout', '--tool', '--top-k', '--top-regressions', '--until', '--usefulness-min', '--verbose', '--version', '--with-code-intel', '--yes'],
'export': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dir', '--explain', '--federated', '--fix', '--follow', '--help', '--include-null-signature', '--json', '--lang', '--markdown', '--multimodal', '--near-symbol', '--no-extract', '--no-federated', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--repo', '--reset', '--resolve', '--restore-only', '--slug-prefix', '--source', '--stale', '--supersessions', '--symbol-kind', '--thin', '--timeout', '--type'],
'extract': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--by-mention', '--catch-up', '--code', '--concurrency', '--dir', '--dry-run', '--explain', '--federated', '--follow', '--from-meetings', '--help', '--include-frontmatter', '--include-null-signature', '--infer-dates', '--json', '--kind', '--lang', '--markdown', '--max-age', '--max-cost-usd', '--multimodal', '--name-status', '--near-symbol', '--ner', '--no-extract', '--no-federated', '--older-than', '--pack', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--repo', '--reset', '--resolve', '--restore-only', '--run-id', '--since', '--slug', '--source', '--source-id', '--stale', '--strategy', '--supersessions', '--symbol-kind', '--thin', '--timeout', '--type', '--verbose', '--workers', '--yes'],
'extract-conversation-facts': ['--aliases', '--all', '--all-sources', '--background', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--by-mention', '--clone-dir', '--code', '--concurrency', '--confirm-destructive', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fix', '--follow', '--force', '--force-break-lock', '--help', '--include-null-signature', '--json', '--judge-model', '--lang', '--limit', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--model', '--multimodal', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--older-than', '--override-disabled', '--path', '--pattern', '--pending', '--pglite', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--restore-only', '--segment-limit', '--session', '--since', '--sleep', '--slug', '--source', '--source-id', '--stale', '--supabase', '--supersessions', '--symbol-kind', '--thin', '--timeout', '--types', '--url', '--url-managed', '--version', '--workers', '--yes'],
'features': ['--aliases', '--all', '--auto-fix', '--background', '--batch-size', '--brain', '--by-mention', '--catch-up', '--concurrency', '--dir', '--explain', '--from-meetings', '--help', '--include-frontmatter', '--include-null-signature', '--infer-dates', '--json', '--kind', '--ner', '--no-extract', '--pace', '--pace-max-concurrency', '--pack', '--path', '--pattern', '--pending', '--priority', '--progress-json', '--quiet', '--repo', '--reset', '--resolve', '--run-id', '--since', '--slugs', '--source', '--source-id', '--stale', '--supersessions', '--thin', '--type', '--verbose', '--workers'],
'files': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--explain', '--fast', '--follow', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--no-pointer', '--page', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--retry-failed', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout', '--type', '--yes'],
'forget': ['--aliases', '--all', '--allow-empty', '--apply', '--as-context', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-tokens', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--entities', '--fast', '--federated', '--file', '--force', '--from-pages', '--grep', '--help', '--http', '--image', '--include-expired', '--include-null-signature', '--json', '--kind', '--limit', '--max-usd', '--mcp-only', '--mode', '--multimodal', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--query', '--quiet', '--reason', '--repo', '--reset', '--resolve', '--restore-only', '--rollup', '--save', '--session', '--session-id', '--since', '--since-last-run', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--timeout', '--today', '--trusted-extraction', '--url', '--watch', '--with-db', '--yes'],
'founder': ['--aliases', '--all', '--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--mcp-only', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--since', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout', '--until'],
'friction': ['--agent', '--brain', '--help', '--hint', '--json', '--kind', '--message', '--no-redact', '--phase', '--redact', '--run-id', '--severity', '--source', '--transcript-path', '--transcripts'],
'frontmatter': ['--aliases', '--all', '--allow-catch-all', '--brain', '--cached', '--diff-filter', '--dry-run', '--exclude-standard', '--fast', '--fix', '--force', '--from-pages', '--get', '--help', '--http', '--include-catch-all', '--include-null-signature', '--json', '--name-only', '--name-status', '--no-embedding', '--no-extract', '--no-verify', '--others', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--strategy', '--supersessions', '--surface', '--thin', '--timeout', '--uninstall', '--write-back'],
'graph-query': ['--aliases', '--all', '--brain', '--depth', '--direction', '--explain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-foreign', '--include-null-signature', '--json', '--lang', '--markdown', '--mcp-only', '--multimodal', '--near-symbol', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--surface', '--symbol-kind', '--thin', '--timeout', '--type'],
'import': ['--aliases', '--all', '--asof', '--background', '--brain', '--brain-wide-max-cost-usd', '--by-mention', '--cached', '--code', '--compile', '--concurrency', '--embedding-dimensions', '--embedding-model', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--fix', '--follow', '--force', '--force-rechunk', '--fresh', '--from-pages', '--full', '--help', '--http', '--include-gitignored', '--include-null-signature', '--json', '--lang', '--markdown', '--max-age', '--multimodal', '--name-status', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--older-than', '--others', '--path', '--pattern', '--pending', '--pglite', '--priority', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--since', '--skip-failed', '--source', '--source-id', '--stale', '--strategy', '--supabase', '--supersessions', '--surface', '--thin', '--timeout', '--url', '--workers'],
'init': ['--all', '--brain', '--chat-model', '--check', '--ctx-size', '--embedding-dimensions', '--embedding-model', '--entity', '--expansion-model', '--fast', '--force', '--from-pages', '--grant-types', '--help', '--http', '--issuer-url', '--json', '--judge-model', '--key', '--mcp-only', '--mcp-url', '--migrate-only', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--non-interactive', '--oauth-client-id', '--oauth-client-secret', '--path', '--pglite', '--provenance', '--schema-pack', '--scopes', '--skip-embed-check', '--source', '--stale', '--supabase', '--surface', '--target', '--to', '--touchpoint', '--url', '--version'],
'integrations': ['--auto', '--brain', '--dry-run', '--embeddings', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--no-embedding', '--overwrite', '--refresh', '--reranking', '--source', '--surface', '--target'],
'integrity': ['--aliases', '--all', '--auto', '--backend', '--background', '--brain', '--brain-wide-max-cost-usd', '--check', '--confidence', '--cost', '--dry-run', '--explain', '--fast', '--follow', '--force', '--fresh', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--limit', '--no-embedding', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--review-lower', '--skip-bare-tweet', '--skip-urls', '--source', '--stale', '--supabase', '--supersessions', '--surface', '--thin', '--timeout', '--type', '--url'],
'jobs': ['--abbrev-ref', '--aliases', '--all', '--allow-empty', '--allow-protected', '--allow-shell-jobs', '--apply', '--asof', '--auto', '--auto-with-prompt', '--background', '--backoff-delay', '--backoff-jitter', '--backoff-type', '--batch-size', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--break-lock', '--budget-usd', '--budget-usd-per-day', '--by-mention', '--cached', '--catch-up', '--check', '--cli-path', '--cluster', '--cluster-errors', '--code', '--concurrency', '--confidence', '--confirm-destructive', '--content', '--date', '--days', '--delay', '--detach', '--diff-filter', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--empty', '--entities', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--federated-read', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--force-retry', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--hard-deadline', '--health-interval', '--held-out', '--help', '--http', '--idempotency-key', '--image', '--include-frontmatter', '--include-gitignored', '--include-null-signature', '--infer-dates', '--inject-bootstrap', '--inline', '--input', '--install', '--interval', '--is-ancestor', '--json', '--kind', '--lang', '--limit', '--lock', '--markdown', '--max-age', '--max-attempts', '--max-cost-usd', '--max-crashes', '--max-rss', '--max-runtime-min', '--max-sources', '--max-stalled', '--max-usd', '--max-waiting', '--mcp-only', '--min-context', '--missing-path', '--mode', '--model', '--multimodal', '--name-only', '--name-status', '--near-symbol', '--ner', '--nice', '--no', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-federate', '--no-gpg-sign', '--no-hard-deadline', '--no-inject', '--no-mutate', '--no-pull', '--no-renames', '--no-schema-pack', '--no-verify', '--no-worker', '--now', '--offset', '--older-than', '--once', '--order', '--others', '--override-disabled', '--pace', '--pace-max-concurrency', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--phase', '--pid-file', '--priority', '--progress-interval', '--progress-json', '--queue', '--quiet', '--redact-secrets', '--reenrich-after', '--refresh-ms', '--remediate', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--resume', '--retry-failed', '--review-lower', '--run-id', '--save', '--segment-limit', '--serial', '--session', '--short', '--show-toplevel', '--sigkill-rescue', '--since', '--skip-bare-tweet', '--skip-failed', '--skip-urls', '--sleep', '--slug', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--stats', '--status', '--strategy', '--supersessions', '--surface', '--swap-only', '--symbol-kind', '--target', '--thin', '--thin-threshold', '--timeout', '--timeout-ms', '--trusted-extraction', '--type', '--types', '--uninstall', '--unsafe-bypass-dream-guard', '--url', '--user', '--verbose', '--version', '--watch', '--wedge-rescue', '--with-db', '--workers', '--yes'],
'lint': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--exclude', '--explain', '--fast', '--fix', '--follow', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout'],
'lsd': ['--brain', '--force-resume', '--help', '--json', '--judge-model', '--limit', '--list-runs', '--max-cost', '--max-far-set', '--max-ideas-per-judge-call', '--no-save', '--resume', '--retry-judge', '--save', '--source', '--strict-budget', '--yes'],
'maintain': ['--aliases', '--all', '--background', '--brain', '--break-lock', '--by-mention', '--catch-up', '--column', '--concurrency', '--content-audit', '--count', '--detach', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--fix', '--force', '--force-retry', '--force-schema', '--from-meetings', '--full', '--help', '--include-flagged', '--include-frontmatter', '--include-null-signature', '--index-audit', '--infer-dates', '--input', '--json', '--kind', '--lang', '--locks', '--markdown', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-usd', '--migrate-only', '--multimodal', '--near-symbol', '--ner', '--nice', '--no-extract', '--no-mutate', '--older-than', '--once', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--progress-json', '--query', '--queue', '--quiet', '--rebuild-rollup', '--regenerate', '--remediate', '--remediation-plan', '--reset', '--resolve', '--restore-only', '--resume', '--run-id', '--safe', '--scope', '--since', '--skills-dir', '--skip-failed', '--slugs', '--source', '--source-id', '--stale', '--status', '--supabase', '--supersessions', '--symbol-kind', '--target', '--target-score', '--thin', '--to', '--top-k', '--type', '--unsafe-bypass-dream-guard', '--url', '--verbose', '--window', '--workers', '--yes'],
'migrate': ['--ab', '--aliases', '--all', '--auto-update', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--build-index', '--by-mention', '--catch-up', '--compile', '--days', '--dim', '--dry-run', '--embedding-dimensions', '--embedding-model', '--embeddings', '--exclusive', '--explain', '--fast', '--follow', '--force', '--force-retry', '--force-schema', '--from-meetings', '--from-pages', '--help', '--history', '--http', '--ignore-env-override', '--ignore-missing-key', '--include-null-signature', '--json', '--lang', '--locks', '--markdown', '--max-age', '--model', '--multimodal', '--name', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--non-interactive', '--pace', '--pace-max-concurrency', '--parallel', '--path', '--pattern', '--pending', '--phase', '--prefix', '--priority', '--progress-interval', '--progress-json', '--quiet', '--refresh-unqualified', '--remediate', '--reranking', '--reset', '--resolve', '--restore-only', '--resume', '--rollback', '--skip-verify', '--slugs', '--source', '--stale', '--supersessions', '--surface', '--symbol-kind', '--thin', '--timeout', '--to', '--undo', '--undo-wave', '--url', '--use-captured-snapshot', '--version', '--with-calibration', '--yes'],
'models': ['--aliases', '--all', '--brain', '--embedding-dimensions', '--embedding-model', '--embeddings', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--judge-model', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reranking', '--reset', '--resolve', '--skip', '--source', '--stale', '--supersessions', '--surface', '--thin', '--undo', '--version'],
'mounts': ['--alias', '--brain', '--cache', '--database-path', '--database-url', '--db-path', '--db-url', '--engine', '--explain', '--help', '--id', '--json', '--lang', '--lock', '--markdown', '--mcp-url', '--multimodal', '--near-symbol', '--path', '--restore-only', '--skills-dir', '--source', '--stale', '--symbol-kind', '--thin', '--verbose'],
'notability-eval': ['--aliases', '--all', '--brain', '--embedding-dimensions', '--embedding-model', '--help', '--in', '--include-null-signature', '--json', '--model', '--multimodal', '--no', '--no-embed', '--no-extract', '--out', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--skip-llm', '--source', '--stale', '--supersessions', '--target-high', '--target-low', '--target-medium', '--thin', '--version'],
'onboard': ['--aliases', '--all', '--allow-empty', '--allow-protected', '--apply', '--asof', '--auto', '--auto-with-prompt', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--check', '--content', '--date', '--days', '--entities', '--explain', '--federated', '--file', '--from-pages', '--help', '--history', '--http', '--image', '--include-null-signature', '--json', '--kind', '--limit', '--max-usd', '--mode', '--multimodal', '--no-extract', '--offset', '--params', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--remediation-plan', '--reset', '--resolve', '--resume', '--save', '--session', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--target-score', '--thin', '--trusted-extraction', '--url', '--with-db', '--yes'],
'orphans': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--count', '--explain', '--follow', '--help', '--include-null-signature', '--include-pseudo', '--json', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--timeout'],
'pages': ['--aliases', '--all', '--brain', '--dry-run', '--help', '--include-null-signature', '--json', '--no-extract', '--older-than', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin'],
'pglite-repair': ['--brain', '--break-lock', '--dry-rnu', '--dry-run', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--no-embedding', '--path', '--quiet', '--source', '--surface', '--yes'],
'post-upgrade': ['--all', '--apply-clean-hunks', '--brain', '--check', '--code', '--compile', '--concurrency', '--embedding-dimensions', '--embedding-model', '--fast', '--ff-only', '--force', '--force-all', '--force-orchestrator', '--force-retry', '--force-schema', '--format', '--from-pages', '--help', '--host-dir', '--http', '--inject-bootstrap', '--inline', '--install', '--interval', '--json', '--limit', '--list', '--markdown', '--max-rss', '--migrate-only', '--migration', '--mode', '--model', '--multimodal', '--name-only', '--no', '--no-autopilot-install', '--no-embed', '--no-embedding', '--no-inject', '--no-worker', '--non-interactive', '--now', '--quiet', '--repo', '--since', '--skills-dir', '--skip-verify', '--source', '--stale', '--status', '--surface', '--swap-only', '--target', '--to', '--uninstall', '--user', '--verbose', '--verify', '--version', '--workers', '--yes'],
'protocol': ['--all', '--allow-empty', '--apply', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--entities', '--fast', '--federated', '--file', '--force', '--from-pages', '--help', '--http', '--image', '--json', '--kind', '--limit', '--max-usd', '--mode', '--multimodal', '--no-embedding', '--offset', '--path', '--progress-interval', '--progress-json', '--quiet', '--save', '--session', '--since', '--slug', '--slugs', '--source', '--stats', '--surface', '--synthesize', '--target', '--timeout', '--token', '--trusted-extraction', '--url', '--with-db', '--yes'],
'providers': ['--brain', '--ctx-size', '--embedding-dimensions', '--embedding-model', '--embeddings', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--reranking', '--source', '--surface', '--touchpoint', '--version'],
'publish': ['--accent', '--bg', '--border', '--brain', '--card-bg', '--code-bg', '--error', '--fg', '--help', '--json', '--link', '--muted', '--out', '--password', '--source', '--title'],
'quarantine': ['--aliases', '--all', '--apply', '--brain', '--code', '--compile', '--explain', '--fast', '--fix', '--force', '--force-rechunk', '--from-pages', '--help', '--http', '--include-flagged', '--include-null-signature', '--json', '--lang', '--limit', '--markdown', '--multimodal', '--near-symbol', '--no-embed', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--surface', '--symbol-kind', '--thin'],
'recall': ['--aliases', '--all', '--allow-empty', '--apply', '--as-context', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-tokens', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--entities', '--fast', '--federated', '--file', '--force', '--from-pages', '--grep', '--help', '--http', '--image', '--include-expired', '--include-null-signature', '--json', '--kind', '--limit', '--max-usd', '--mcp-only', '--mode', '--multimodal', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--query', '--quiet', '--reason', '--repo', '--reset', '--resolve', '--restore-only', '--rollup', '--save', '--session', '--session-id', '--since', '--since-last-run', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--timeout', '--today', '--trusted-extraction', '--url', '--watch', '--with-db', '--yes'],
'reconcile-links': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--explain', '--follow', '--help', '--include-frontmatter', '--include-null-signature', '--json', '--name-status', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--source', '--stale', '--strategy', '--supersessions', '--thin', '--timeout', '--type'],
'reindex': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--code', '--compile', '--concurrency', '--cost-estimate', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--fix', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--lang', '--limit', '--markdown', '--max-age', '--max-cost-usd', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--no-extract', '--older-than', '--path', '--pattern', '--pending', '--pglite', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--repo', '--reset', '--resolve', '--source', '--stale', '--supabase', '--supersessions', '--surface', '--thin', '--timeout', '--version', '--workers', '--yes'],
'reindex-code': ['--abi', '--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--chunker-debug', '--code', '--compile', '--concurrency', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fix', '--follow', '--force', '--force-rechunk', '--help', '--include-null-signature', '--json', '--judge-model', '--lang', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--model', '--multimodal', '--no', '--no-embed', '--no-extract', '--older-than', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--serial', '--source', '--stale', '--supersessions', '--thin', '--timeout', '--version', '--workers', '--yes'],
'reindex-frontmatter': ['--aliases', '--all', '--brain', '--concurrency', '--dry-run', '--force', '--help', '--include-null-signature', '--json', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--slug-prefix', '--source', '--stale', '--supersessions', '--thin', '--workers', '--yes'],
'reindex-search-vector': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--explain', '--follow', '--help', '--include-null-signature', '--json', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--timeout', '--yes'],
'reinit-pglite': ['--abbrev-ref', '--all', '--allow-empty', '--brain', '--break-lock', '--chat-model', '--concurrency', '--confirm-destructive', '--diff-filter', '--embedding-dimensions', '--embedding-model', '--empty', '--entity', '--exclude', '--exclude-standard', '--expansion-model', '--fast', '--force', '--force-break-lock', '--format', '--from-pages', '--full', '--grant-types', '--hard-deadline', '--help', '--http', '--include-gitignored', '--interval', '--is-ancestor', '--issuer-url', '--json', '--key', '--lock', '--max-age', '--max-sources', '--mcp-only', '--mcp-url', '--migrate-only', '--missing-path', '--model', '--name-only', '--name-status', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-gpg-sign', '--no-hard-deadline', '--no-pull', '--no-renames', '--no-schema-pack', '--no-sync', '--no-verify', '--non-interactive', '--oauth-client-id', '--oauth-client-secret', '--parallel', '--path', '--pglite', '--priority', '--provenance', '--quiet', '--repo', '--retry-failed', '--schema-pack', '--scopes', '--serial', '--short', '--show-toplevel', '--skip-embed-check', '--skip-failed', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--strategy', '--supabase', '--surface', '--target', '--timeout', '--to', '--url', '--version', '--watch', '--workers', '--yes'],
'remote': ['--all', '--background', '--brain', '--break-lock', '--by-mention', '--column', '--concurrency', '--content-audit', '--count', '--detach', '--dim', '--dir', '--drain', '--embedding-dimensions', '--embedding-model', '--fast', '--fix', '--force', '--force-retry', '--force-schema', '--from-pages', '--full', '--help', '--http', '--include-flagged', '--index-audit', '--json', '--locks', '--markdown', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-usd', '--mcp-only', '--migrate-only', '--multimodal', '--nice', '--no-embedding', '--older-than', '--parallel', '--params', '--pglite', '--phase', '--pid-file', '--progress-json', '--query', '--queue', '--rebuild-rollup', '--regenerate', '--remediate', '--remediation-plan', '--reset', '--resume', '--scope', '--scopes', '--since', '--skills-dir', '--skip-failed', '--slugs', '--source', '--source-id', '--stale', '--status', '--supabase', '--surface', '--target', '--target-score', '--timeout', '--to', '--top-k', '--url', '--window', '--workers', '--yes'],
'repair-jsonb': ['--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--explain', '--fast', '--follow', '--force', '--from-pages', '--help', '--http', '--json', '--lang', '--markdown', '--multimodal', '--near-symbol', '--no-embedding', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--restore-only', '--source', '--stale', '--supabase', '--surface', '--symbol-kind', '--thin', '--timeout', '--url'],
'report': ['--brain', '--content', '--dir', '--help', '--json', '--source', '--title', '--type'],
'repos': ['--abbrev-ref', '--abort', '--aliases', '--all', '--all-sources', '--brain', '--branch', '--break-lock', '--cached', '--clone-dir', '--compile', '--confirm-destructive', '--detect', '--dry-run', '--explain', '--fast', '--federated', '--file', '--fix', '--force', '--force-break-lock', '--format', '--from-pages', '--full', '--get', '--git-dir', '--github-repo', '--help', '--http', '--include-null-signature', '--include-warns', '--json', '--keep-storage', '--lang', '--local', '--markdown', '--max-age', '--max-cost-usd', '--multimodal', '--name', '--name-only', '--name-status', '--near-symbol', '--no-cron', '--no-embedding', '--no-extract', '--no-federate', '--no-federated', '--no-harden', '--no-verify', '--params', '--pat-file', '--path', '--pattern', '--pending', '--porcelain', '--push-only', '--quiet', '--rebase', '--repo', '--reset', '--resolve', '--restore-only', '--secret', '--set', '--short', '--source', '--source-id', '--stale', '--status', '--strategy', '--supersessions', '--surface', '--symbol-kind', '--thin', '--unset-all', '--url', '--url-managed', '--yes'],
'resolvers': ['--auto', '--backend', '--brain', '--cost', '--help', '--json', '--source'],
'retrieval-upgrade': ['--aliases', '--all', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--catch-up', '--dim', '--dry-run', '--embedding-dimensions', '--embedding-model', '--embeddings', '--explain', '--fast', '--follow', '--force', '--from-pages', '--help', '--http', '--ignore-env-override', '--ignore-missing-key', '--include-null-signature', '--json', '--markdown', '--model', '--multimodal', '--name', '--no', '--no-embed', '--no-embedding', '--no-extract', '--non-interactive', '--pace', '--pace-max-concurrency', '--parallel', '--pattern', '--pending', '--prefix', '--priority', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reranking', '--reset', '--resolve', '--resume', '--slugs', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout', '--to', '--undo', '--version', '--yes'],
'routing-eval': ['--brain', '--fix', '--help', '--json', '--llm', '--skills-dir', '--source', '--strict', '--verbose'],
'salience': ['--aliases', '--all', '--brain', '--days', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--kind', '--limit', '--mcp-only', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--slug-prefix', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout'],
'schema': ['--alias', '--aliases', '--all', '--apply', '--as-filing-rules', '--brain', '--dims', '--expert', '--expert-routing', '--extractable', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--inverse', '--json', '--kind', '--no-embedding', '--no-extract', '--pack', '--page-type', '--pattern', '--pending', '--prefix', '--primitive', '--reset', '--resolve', '--schema-pack', '--since', '--source', '--source-id', '--stale', '--supersessions', '--surface', '--target-type', '--thin', '--to', '--with-db'],
'self-upgrade': ['--all', '--brain', '--check', '--check-only', '--ff-only', '--force', '--help', '--json', '--markdown', '--migrate-only', '--non-interactive', '--refresh-cache', '--source', '--swap-only', '--to', '--version', '--yes'],
'serve': ['--aliases', '--all', '--bind', '--bound-slug-prefixes', '--brain', '--enable-dcr', '--enable-dcr-insecure', '--fast', '--federated-read', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--log-full-params', '--no-embedding', '--no-extract', '--pattern', '--pending', '--port', '--print-admin-token', '--public-url', '--reset', '--resolve', '--source', '--stale', '--stdio-idle-timeout', '--supersessions', '--suppress', '--suppress-bootstrap-token', '--surface', '--thin', '--token-ttl', '--yes'],
'skillify': ['--brain', '--description', '--dry-run', '--force', '--help', '--json', '--mutating', '--recent', '--skills-dir', '--source', '--strict', '--triggers', '--verbose', '--writes-pages', '--writes-to'],
'skillopt': ['--aliases', '--all', '--allow-mutate-bundled', '--background', '--batch-size', '--benchmark', '--bootstrap-from-routing', '--bootstrap-from-skill', '--bootstrap-reviewed', '--bootstrap-tasks', '--brain', '--brain-wide-max-cost-usd', '--chunker-debug', '--dry-run', '--epochs', '--follow', '--force', '--held-out', '--help', '--include-null-signature', '--json', '--judge-model', '--lr', '--lr-schedule', '--max-cost-usd', '--max-runtime-min', '--model', '--no-extract', '--no-mutate', '--optimizer-model', '--patch', '--pattern', '--pending', '--reset', '--resolve', '--resume', '--rewrite', '--skills-dir', '--source', '--split', '--stale', '--supersessions', '--target-model', '--target-models', '--thin', '--verbose', '--yes'],
'skillpack': ['--all', '--apply-clean-hunks', '--author', '--brain', '--dry-run', '--exit-code', '--fast', '--fix', '--force', '--force-unlock', '--format', '--from', '--from-pages', '--frontmatter', '--full', '--help', '--homepage', '--http', '--json', '--license', '--list', '--minimal', '--name-only', '--no-cache', '--no-embedding', '--no-lint', '--note', '--out', '--overwrite-local', '--push', '--quick', '--quiet', '--refresh', '--repo', '--schema-pack', '--short', '--since', '--skills-dir', '--skip-doctor', '--source', '--strict', '--surface', '--target', '--tier', '--trust', '--url', '--verbose', '--verify', '--workspace', '--yes'],
'skillpack-check': ['--background', '--brain', '--brain-wide-max-cost-usd', '--explain', '--fast', '--follow', '--help', '--json', '--list', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--source', '--stale', '--strict', '--timeout', '--yes'],
'smoke-test': ['--brain', '--help', '--json', '--source'],
'sources': ['--abbrev-ref', '--abort', '--aliases', '--all', '--all-sources', '--brain', '--branch', '--break-lock', '--cached', '--clone-dir', '--compile', '--confirm-destructive', '--detect', '--dry-run', '--explain', '--fast', '--federated', '--file', '--fix', '--force', '--force-break-lock', '--format', '--from-pages', '--full', '--get', '--git-dir', '--github-repo', '--help', '--http', '--include-null-signature', '--include-warns', '--json', '--keep-storage', '--lang', '--local', '--markdown', '--max-age', '--max-cost-usd', '--multimodal', '--name', '--name-only', '--name-status', '--near-symbol', '--no-cron', '--no-embedding', '--no-extract', '--no-federate', '--no-federated', '--no-harden', '--no-verify', '--params', '--pat-file', '--path', '--pattern', '--pending', '--porcelain', '--push-only', '--quiet', '--rebase', '--repo', '--reset', '--resolve', '--restore-only', '--secret', '--set', '--short', '--source', '--source-id', '--stale', '--status', '--strategy', '--supersessions', '--surface', '--symbol-kind', '--thin', '--unset-all', '--url', '--url-managed', '--yes'],
'status': ['--abbrev-ref', '--abi', '--abort', '--aliases', '--all', '--all-sources', '--allow-empty', '--apply', '--asof', '--auto', '--background', '--batch-size', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--brain-wide-max-cost-usd', '--branch', '--break-lock', '--budget-usd-per-day', '--by-mention', '--cached', '--catch-up', '--clone-dir', '--code', '--column', '--compile', '--concurrency', '--confirm-destructive', '--content', '--content-audit', '--count', '--date', '--days', '--deadline-ms', '--depth', '--detach', '--diff-filter', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--empty', '--entities', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--ff-only', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--force-retry', '--force-schema', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--git-path', '--hard-deadline', '--help', '--http', '--image', '--include-flagged', '--include-frontmatter', '--include-gitignored', '--include-null-signature', '--index-audit', '--infer-dates', '--install', '--interval', '--is-ancestor', '--json', '--judge-model', '--kind', '--lang', '--limit', '--lock', '--locks', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-runtime', '--max-sources', '--max-usd', '--mcp-only', '--migrate-only', '--min-context', '--missing-path', '--mode', '--model', '--multimodal', '--name', '--name-only', '--name-status', '--near-symbol', '--ner', '--nice', '--no', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-gpg-sign', '--no-hard-deadline', '--no-pull', '--no-recurse-submodules', '--no-renames', '--no-schema-pack', '--no-verify', '--object-format', '--offset', '--older-than', '--order', '--others', '--overwrite', '--pace', '--pace-max-concurrency', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--prefix', '--priority', '--progress-interval', '--progress-json', '--query', '--queue', '--quiet', '--rebase', '--rebuild-rollup', '--reenrich-after', '--refresh', '--regenerate', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--resume', '--retry-failed', '--run-id', '--save', '--scope', '--scopes', '--section', '--serial', '--session', '--short', '--show-toplevel', '--since', '--skills-dir', '--skip-failed', '--slug', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--stats', '--status', '--stdin', '--strategy', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--target', '--target-score', '--thin', '--thin-threshold', '--timeout', '--to', '--top-k', '--trusted-extraction', '--type', '--types', '--url', '--url-managed', '--verbose', '--verify', '--version', '--watch', '--what', '--where', '--who', '--window', '--with-db', '--workers', '--yes'],
'storage': ['--aliases', '--all', '--brain', '--federated', '--fix', '--help', '--include-null-signature', '--json', '--no-extract', '--no-federated', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--thin', '--to'],
'sync': ['--abbrev-ref', '--abi', '--abort', '--aliases', '--all', '--all-sources', '--allow-empty', '--asof', '--auto', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--branch', '--break-lock', '--by-mention', '--cached', '--catch-up', '--clone-dir', '--code', '--column', '--compile', '--concurrency', '--confirm-destructive', '--content-audit', '--count', '--depth', '--detach', '--diff-filter', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--empty', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--ff-only', '--fix', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--force-retry', '--force-schema', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--git-path', '--hard-deadline', '--help', '--http', '--include-flagged', '--include-frontmatter', '--include-gitignored', '--include-null-signature', '--index-audit', '--infer-dates', '--interval', '--is-ancestor', '--json', '--kind', '--lang', '--lock', '--locks', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-sources', '--max-usd', '--migrate-only', '--missing-path', '--model', '--multimodal', '--name', '--name-only', '--name-status', '--ner', '--nice', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-gpg-sign', '--no-hard-deadline', '--no-pull', '--no-recurse-submodules', '--no-renames', '--no-schema-pack', '--no-verify', '--object-format', '--older-than', '--others', '--overwrite', '--pace', '--pace-max-concurrency', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--prefix', '--priority', '--progress-interval', '--progress-json', '--query', '--queue', '--quiet', '--rebase', '--rebuild-rollup', '--refresh', '--regenerate', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--resume', '--retry-failed', '--run-id', '--save', '--scope', '--serial', '--short', '--show-toplevel', '--since', '--skills-dir', '--skip-failed', '--slug', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--status', '--stdin', '--strategy', '--supabase', '--supersessions', '--surface', '--target', '--target-score', '--thin', '--timeout', '--to', '--top-k', '--type', '--url', '--url-managed', '--verbose', '--verify', '--watch', '--window', '--workers', '--yes'],
'takes': ['--aliases', '--all', '--brain', '--bucket-size', '--by', '--claim', '--dir', '--domain', '--dry-run', '--evidence', '--expired', '--fast', '--federated', '--force', '--from-pages', '--help', '--holder', '--http', '--include-covered', '--include-null-signature', '--json', '--kind', '--limit', '--max-pages', '--no-embedding', '--no-extract', '--no-federated', '--outcome', '--path', '--pattern', '--pending', '--quality', '--refresh', '--repo', '--reset', '--resolve', '--restore-only', '--row', '--since', '--slugs', '--sort', '--source', '--source-id', '--stale', '--supersessions', '--surface', '--thin', '--unit', '--until', '--value', '--weight', '--who', '--yes'],
'think': ['--aliases', '--all', '--anchor', '--brain', '--calibration-holder', '--explain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--max-usd', '--mcp-only', '--model', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--rounds', '--save', '--since', '--source', '--stale', '--supersessions', '--surface', '--take', '--thin', '--timeout', '--until', '--with-calibration'],
'transcripts': ['--aliases', '--all', '--brain', '--days', '--full', '--help', '--include-null-signature', '--json', '--limit', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--timeout'],
'upgrade': ['--all', '--apply-clean-hunks', '--brain', '--check', '--code', '--compile', '--concurrency', '--embedding-dimensions', '--embedding-model', '--fast', '--ff-only', '--force', '--force-all', '--force-orchestrator', '--force-retry', '--force-schema', '--format', '--from-pages', '--help', '--host-dir', '--http', '--inject-bootstrap', '--inline', '--install', '--interval', '--json', '--limit', '--list', '--markdown', '--max-rss', '--migrate-only', '--migration', '--mode', '--model', '--multimodal', '--name-only', '--no', '--no-autopilot-install', '--no-embed', '--no-embedding', '--no-inject', '--no-worker', '--non-interactive', '--now', '--quiet', '--repo', '--since', '--skills-dir', '--skip-verify', '--source', '--stale', '--status', '--surface', '--swap-only', '--target', '--to', '--uninstall', '--user', '--verbose', '--verify', '--version', '--workers', '--yes'],
'watch': ['--aliases', '--all', '--brain', '--fast', '--federated', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--max-pages', '--min-confidence', '--no-embedding', '--no-extract', '--no-federated', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--surface', '--thin', '--window-turns'],
'ze-switch': ['--aliases', '--all', '--brain', '--confirm-reembed', '--dry-run', '--force', '--help', '--ignore-env-override', '--ignore-missing-key', '--include-null-signature', '--json', '--markdown', '--no-extract', '--non-interactive', '--pattern', '--pending', '--reset', '--resolve', '--resume', '--source', '--stale', '--supersessions', '--thin', '--undo', '--yes'],
};
src/core/config.ts
+14
View File
@@ -114,6 +114,17 @@ export interface GBrainConfig {
provider_base_urls?: Record<string, string>;
/** Optional chat request providerOptions overrides keyed by recipe id or "recipe:modelId". */
provider_chat_options?: Record<string, Record<string, unknown>>;
/**
* MEMORY_VERBS v1 (Cathedral 1): default MCP tool surface for `gbrain serve`.
* 'verbs' = exactly the 5 protocol verbs (the quickstart surface);
* 'full' (default) = every operation. The `--surface` flag overrides per-run.
*/
mcp_surface?: 'verbs' | 'full';
/**
* MEMORY_VERBS v1 [D6C]: ISO timestamp stamped by `gbrain init` so
* `gbrain protocol stats` can derive real TTHW (install → first verb call).
*/
protocol_installed_at?: string;
/**
* Optional storage backend config (S3/Supabase/local). Shape matches
* `StorageConfig` in `./storage.ts`. Typed as `unknown` here to avoid
@@ -948,6 +959,9 @@ export const KNOWN_CONFIG_KEYS: readonly string[] = [
'chat_model',
'chat_fallback_chain',
'provider_base_urls',
// MEMORY_VERBS v1 (Cathedral 1)
'mcp_surface',
'protocol_installed_at',
'provider_chat_options',
'storage',
'eval',
src/core/context/retrieval-reflex.ts
+5 -2
View File
@@ -106,7 +106,7 @@ export interface ResolvePointersOpts {
sourceIds?: string[];
}
interface PageRow {
export interface PageRow {
slug: string;
source_id: string;
title: string;
@@ -290,8 +290,11 @@ function displayForRow(row: PageRow, displayByNorm: Map<string, string>): string
* otherwise strip takes/private-fact fences from the body (the same boundary
* get_page applies to untrusted readers) and take the first sentence. Never
* returns raw compiled_truth.
*
* Exported for the MEMORY_VERBS v1 entity card (verbs/entity-card.ts) — the
* card's `summary` field runs through THIS boundary, not a parallel one.
*/
function safeSynopsis(row: PageRow): string {
export function safeSynopsis(row: PageRow): string {
const fmSummary = row.frontmatter?.summary;
if (typeof fmSummary === 'string' && fmSummary.trim()) {
return clip(collapse(fmSummary), SYNOPSIS_MAX);
src/core/cycle/synthesize-concepts.ts
+19 -2
View File
@@ -30,8 +30,20 @@ import { chat as gatewayChat, isAvailable } from '../ai/gateway.ts';
// source-boost's 1.3× 'concepts/' weighting can actually reach them.
import { importFromContent } from '../import-file.ts';
import { serializeMarkdown } from '../markdown.ts';
import { canonicalLookup, type ModelPricing } from '../model-pricing.ts';
const DEFAULT_BUDGET_USD = 1.5;
// Canonical-miss policy — mirrors skillopt/preflight.ts's lookupPrice:
// assume Sonnet-tier pricing for models absent from CANONICAL_PRICING.
// Conservative and non-throwing; keeps the budget gate effective (and
// matches this file's pre-canonical behavior) instead of letting an
// unpriced model run unmetered. The rates are DERIVED from the canonical
// table (never hand-copied — CLAUDE.md invariant); the literal pair only
// fires if the Sonnet key itself ever leaves the table.
const FALLBACK_PRICING: ModelPricing = canonicalLookup('anthropic:claude-sonnet-4-6') ?? {
input: 3.0,
output: 15.0,
};
const TIER_T1_MIN = 10;
const TIER_T2_MIN = 5;
const TIER_T3_MIN = 2;
@@ -204,9 +216,14 @@ export async function runPhaseSynthesizeConcepts(
// codex flagged. Throttle inside maybeYield bounds the actual
// refresh rate.
await maybeYield();
// Sonnet at ~$3/M input + $15/M output
// Price from the model that actually answered, through the one
// canonical chat-pricing table (CLAUDE.md invariant). Canonical
// miss → Sonnet-tier FALLBACK_PRICING (see constant above).
const pricing = canonicalLookup(result.model) ?? FALLBACK_PRICING;
estimatedSpendUsd +=
(result.usage.input_tokens * 3.0 + result.usage.output_tokens * 15.0) / 1_000_000;
(result.usage.input_tokens * pricing.input +
result.usage.output_tokens * pricing.output) /
1_000_000;
narrative = result.text.trim() || deterministicNarrative(group);
} catch (err) {
failures.push({
src/core/doctor-categories.ts
+1
View File
@@ -127,6 +127,7 @@ export const BRAIN_CHECK_NAMES: ReadonlySet<string> = new Set([
* skill-flavored name) live under 'brain'.
*/
export const SKILL_CHECK_NAMES: ReadonlySet<string> = new Set([
'memory_verbs_usage',
'resolver_health',
'retrieval_reflex_health',
'skill_brain_first',
src/core/engine.ts
+10 -4
View File
@@ -990,10 +990,13 @@ export interface BrainEngine {
*/
upsertChunks(slug: string, chunks: ChunkInput[], opts?: { sourceId?: string } & BatchOpts): Promise<void>;
/**
* Read every chunk for a page. `opts.sourceId` source-scopes the page
* lookup; without it, multi-source brains return chunks from every
* same-slug source (importCodeFile uses this for incremental embedding
* reuse, which would then attach the wrong source's embeddings).
* Read every chunk for a page. Scope precedence mirrors getPage (#2555):
* a federated grant (`sourceIds[]`) wins over scalar `sourceId`; with
* neither set, the lookup falls back to the `'default'` source (the
* local-untyped-call default that importCodeFile's incremental embedding
* reuse relies on). Embedding vectors are never selected — rowToChunk
* discards them at these call sites, so pulling them was pure egress
* (#2544).
*/
getChunks(slug: string, opts?: { sourceId?: string; sourceIds?: string[] }): Promise<Chunk[]>;
/**
@@ -2104,6 +2107,9 @@ export interface BrainEngine {
// Migration support
runMigration(version: number, sql: string): Promise<void>;
// Deliberately scalar-only (no sourceIds[] widening): engine-internal with
// zero remote-reachable callers (verified #2555 review), so the federated
// read-scope contract doesn't apply. Widen only if an op ever exposes it.
getChunksWithEmbeddings(slug: string, opts?: { sourceId?: string }): Promise<Chunk[]>;
// Raw SQL (for Minions job queue and other internal modules)
src/core/facts/extract.ts
+8 -1
View File
@@ -308,9 +308,16 @@ export async function extractFactsFromTurnWithOutcome(
}
const parsedShape = parseExtractorJsonDetailed(result.text);
if (!parsedShape || parsedShape.invalidCandidates > 0) {
if (!parsedShape ||
(parsedShape.invalidCandidates > 0 && parsedShape.facts.length === 0)) {
return { ok: false, reason: 'malformed_output' };
}
if (parsedShape.invalidCandidates > 0) {
process.stderr.write(
`[facts-extract] WARN: dropped ${parsedShape.invalidCandidates} malformed candidate(s); ` +
`kept ${parsedShape.facts.length}\n`,
);
}
const parsedRaw = parsedShape.facts;
const facts: ExtractedFact[] = [];
src/core/facts/fence-write.ts
+10 -1
View File
@@ -65,6 +65,12 @@ export interface FenceInputFact {
/** Defaults to 1.0 when undefined (matches engine.insertFact behavior). */
confidence?: number;
validFrom?: Date;
/**
* MEMORY_VERBS v1 (c5): remember's ttl → valid_until. Date-only in the
* fence cell; the DB column derives from it on the stamp step.
* Undefined/null = never expires (pre-v1 behavior unchanged).
*/
validUntil?: Date | null;
embedding: Float32Array | null;
sessionId: string | null;
}
@@ -270,7 +276,10 @@ export async function writeFactsToFence(
visibility: f.visibility,
notability: f.notability ?? 'medium',
validFrom: validFromStr,
validUntil: undefined,
// MEMORY_VERBS v1 (c5): remember's ttl threads through to the fence
// cell — was hard-coded undefined, which silently dropped expiry on
// this path. extractFactsFromFenceText derives the DB column from it.
validUntil: f.validUntil ? f.validUntil.toISOString().slice(0, 10) : undefined,
source: f.source,
context: f.context ?? undefined,
});
src/core/facts/forget.ts
+31 -2
View File
@@ -55,6 +55,7 @@ interface FactDbRow {
row_num: number | null;
source_markdown_slug: string | null;
expired_at: Date | null;
visibility: string;
}
interface SourceRow {
@@ -83,15 +84,43 @@ function todayUtc(): string {
export async function forgetFactInFence(
engine: BrainEngine,
factId: number,
opts: { reason?: string } = {},
opts: {
reason?: string;
/**
* MEMORY_VERBS v1 trust boundary [ship P1.1]: when set, the fact must
* belong to this source or the call returns `not_found` (indistinguishable
* from a truly-missing id — no cross-source existence leak). The `forget`
* verb passes ctx.sourceId so a remote caller scoped to source A cannot
* expire facts in source B by guessing global ids.
*/
sourceId?: string;
/**
* When true (remote callers), the fact must be visibility='world' or the
* call returns `not_found` — a remote caller can't expire private facts it
* could never read (mirrors recall's remote posture).
*/
worldOnly?: boolean;
} = {},
): Promise<ForgetFactResult> {
const reason = opts.reason ?? 'forgotten';
const rows = await engine.executeRaw<FactDbRow>(
`SELECT id, source_id, entity_slug, row_num, source_markdown_slug, expired_at
`SELECT id, source_id, entity_slug, row_num, source_markdown_slug, expired_at, visibility
FROM facts WHERE id = $1`,
[factId],
);
// Trust-boundary scope check BEFORE any state inspection: a row outside the
// caller's source (or private, for remote callers) is reported as not_found,
// never distinguished from a missing id.
if (rows.length === 1) {
const r = rows[0];
const outOfScope =
(opts.sourceId !== undefined && r.source_id !== opts.sourceId) ||
(opts.worldOnly === true && r.visibility !== 'world');
if (outOfScope) {
return { ok: false, path: 'not_found', reason };
}
}
if (rows.length === 0) {
return { ok: false, path: 'not_found', reason };
}
src/core/facts/write-single.ts
+227
View File
@@ -0,0 +1,227 @@
/**
* MEMORY_VERBS v1 — `writeSingleFact`: the zero-LLM single-fact write seam
* behind the `remember` verb [E1].
*
* `runFactsPipeline` is extraction-first (LLM-gated in extract.ts) and cannot
* back a verb whose fact arrives pre-formed. This module reuses the pipeline's
* post-extraction stages directly: resolve → dedup (embedding cosine, same
* 0.95 threshold) → fence-first write (markdown durability) with the same
* legacy DB-only fallbacks (thin-client, unparented, stub-guard).
*
* Supersession [X1, frozen as implementation-defined]: minimal deterministic
* rule, zero LLM — when the top dedup candidate scores >= threshold with the
* SAME kind but DIFFERENT text, the new fact supersedes it (a near-duplicate
* with changed content is an update: "X at Acme" → "X left Acme"). Same text
* → plain duplicate (existing id returned, nothing written).
*
* Degradation (documented in the protocol doc): with no embedding provider,
* dedup/supersession are skipped on the fence path and near-duplicates may
* insert — `degraded_dedup: true` tells the caller.
*
* Provenance (c6): callers pass free-text provenance which lands on
* `NewFact.source` verbatim — this seam deliberately does NOT take a
* FactsBackstopCtx (whose `source` union is pipeline-internal).
*/
import type { BrainEngine, FactInsertStatus, NewFact } from '../engine.ts';
const DEDUP_THRESHOLD = 0.95;
const DEDUP_CANDIDATE_LIMIT = 5;
export interface SingleFactInput {
fact: string;
/** Free-text attribution, stored verbatim as the fact's `source`. */
provenance: string;
kind?: NewFact['kind'];
/** Free-form entity ref; canonicalized via resolveEntitySlug. */
entity?: string | null;
/** Facts-layer default 'private'; the remember VERB passes 'world' [F2]. */
visibility?: 'private' | 'world';
validUntil?: Date | null;
sessionId?: string | null;
confidence?: number;
}
export interface SingleFactResult {
id: number;
status: FactInsertStatus;
entity_slug: string | null;
valid_until: Date | null;
/** True when no embedding provider — dedup/supersession skipped. */
degraded_dedup: boolean;
}
export async function writeSingleFact(
engine: BrainEngine,
sourceId: string,
input: SingleFactInput,
): Promise<SingleFactResult> {
const { resolveEntitySlug } = await import('../entities/resolve.ts');
const { cosineSimilarity } = await import('./classify.ts');
const { writeFactsToFence, lookupSourceLocalPath } = await import('./fence-write.ts');
const { isAvailable, embedOne } = await import('../ai/gateway.ts');
const factText = input.fact.trim();
const kind = input.kind ?? 'fact';
const visibility = input.visibility ?? 'private';
const validUntil = input.validUntil ?? null;
const resolvedSlug = input.entity
? ((await resolveEntitySlug(engine, sourceId, input.entity)) ?? input.entity)
: null;
// Embedding (NOT an LLM call): powers dedup + downstream recall. Fail-soft —
// a missing/failing provider degrades dedup, never the write.
let embedding: Float32Array | null = null;
let degradedDedup = false;
if (isAvailable('embedding')) {
try {
embedding = await embedOne(factText);
} catch {
degradedDedup = true;
}
} else {
degradedDedup = true;
}
// Dedup + supersession decision (same candidates + threshold as the pipeline).
let supersedeId: number | null = null;
if (resolvedSlug && embedding) {
const candidates = await engine.findCandidateDuplicates(sourceId, resolvedSlug, factText, {
embedding,
k: DEDUP_CANDIDATE_LIMIT,
});
let top: (typeof candidates)[number] | null = null;
let topScore = -1;
for (const c of candidates) {
if (!c.embedding) continue;
const s = cosineSimilarity(embedding, c.embedding);
if (s > topScore) {
topScore = s;
top = c;
}
}
if (top && topScore >= DEDUP_THRESHOLD) {
const textDiffers = collapse(top.fact) !== collapse(factText);
if (top.kind === kind && textDiffers) {
supersedeId = top.id; // X1: near-duplicate with changed content = update
} else {
return {
id: top.id,
status: 'duplicate',
entity_slug: resolvedSlug,
valid_until: top.valid_until ?? null,
degraded_dedup: false,
};
}
}
}
const newFact: NewFact = {
fact: factText,
kind,
entity_slug: resolvedSlug,
visibility,
source: input.provenance,
source_session: input.sessionId ?? null,
confidence: input.confidence ?? 1.0,
valid_until: validUntil,
embedding,
};
// Fence-first write (markdown durability — same policy as the pipeline):
// requires a resolved, prefixed entity slug and a local_path. Everything
// else takes the legacy DB-only insertFact path, which also handles the
// supersedeId bookkeeping engine-side.
const localPath = resolvedSlug ? await lookupSourceLocalPath(engine, sourceId) : null;
const fenceable = resolvedSlug !== null && localPath !== null;
if (fenceable) {
const result = await writeFactsToFence(
engine,
{ sourceId, localPath, slug: resolvedSlug },
[
{
fact: factText,
kind,
notability: 'medium',
source: input.provenance,
context: null,
visibility,
confidence: input.confidence ?? 1.0,
validFrom: new Date(),
validUntil,
embedding,
sessionId: input.sessionId ?? null,
},
],
);
if (result.fenceWriteFailed) {
// Parse-validate rejected the .tmp (quarantined). Hard failure — do NOT
// fall through to a DB row whose fence is broken (pipeline policy).
throw new Error(
`facts fence write failed for ${resolvedSlug} — .tmp quarantined; see the facts write-failure JSONL log`,
);
}
if (!result.stubGuardBlocked && !result.legacyFallback) {
const newId = result.ids[0];
if (supersedeId !== null && newId !== undefined) {
await expireSuperseded(engine, supersedeId, newId);
return {
id: newId,
status: 'superseded',
entity_slug: resolvedSlug,
valid_until: validUntil,
degraded_dedup: degradedDedup,
};
}
return {
id: newId,
status: 'inserted',
entity_slug: resolvedSlug,
valid_until: validUntil,
degraded_dedup: degradedDedup,
};
}
// stubGuardBlocked / defensive legacyFallback → DB-only path below.
}
const inserted = await engine.insertFact(newFact, { // gbrain-allow-direct-insert: writeSingleFact legacy path for unparented / thin-client / stub-guarded facts (mirrors the pipeline's fallback buckets)
source_id: sourceId,
...(supersedeId !== null ? { supersedeId } : {}),
});
return {
id: inserted.id,
status: supersedeId !== null ? 'superseded' : inserted.status,
entity_slug: resolvedSlug,
valid_until: validUntil,
degraded_dedup: degradedDedup,
};
}
/**
* Fence-path supersession bookkeeping: expire the old row through the fence
* (strikethrough + valid_until, the same surface `forget` uses) and link
* `superseded_by` for the audit trail. Both steps best-effort — the new fact
* is already durably written; a partial supersede is an audit gap, not data
* loss.
*/
async function expireSuperseded(engine: BrainEngine, oldId: number, newId: number): Promise<void> {
try {
const { forgetFactInFence } = await import('./forget.ts');
await forgetFactInFence(engine, oldId, { reason: `superseded by fact #${newId}` });
} catch {
/* best-effort */
}
try {
await engine.executeRaw(`UPDATE facts SET superseded_by = $1 WHERE id = $2`, [newId, oldId]);
} catch {
/* best-effort */
}
}
function collapse(s: string): string {
return s.replace(/\s+/g, ' ').trim().toLowerCase();
}
src/core/operations-descriptions.ts
+14 -2
View File
@@ -66,6 +66,11 @@ export const LIST_PAGES_DESCRIPTION =
export const QUERY_DESCRIPTION =
"Hybrid search with vector + keyword + multi-query expansion. " +
"Prefer `query` for concept / synonym / landscape questions ('all the X that " +
"do Y', 'the landscape of Z') — expansion recovers synonym- and " +
"outcome-phrased matches a single embedding misses. Still top-K: for " +
"exhaustive enumeration use list_pages; for exact known tokens `search` is " +
"cheaper (no expansion LLM call). " +
"For personal/emotional questions ('what's going on with me', 'anything notable', " +
"'how am I feeling'), prefer get_recent_salience, find_anomalies, or " +
"get_recent_transcripts. Semantic search returns polished pages and misses " +
@@ -73,12 +78,19 @@ export const QUERY_DESCRIPTION =
"mean impressive — they often mean difficult or emotionally charged.";
export const SEARCH_DESCRIPTION =
"Keyword search using full-text search. For personal/emotional questions, " +
"Cheap hybrid search (vector + keyword + RRF) with no LLM expansion. " +
"Best for exact known tokens, names, and structured-field lookups. A populated " +
"result set is NOT proof of coverage — for concept / synonym / landscape " +
"questions use `query` (adds multi-query expansion); for exhaustive " +
"enumeration use list_pages pagination. " +
"For personal/emotional questions, " +
"prefer get_recent_salience or find_anomalies — they surface activity bursts " +
"without needing a search term. " +
"For code-symbol questions (callers, callees, definitions, blast radius), use " +
"code_callers / code_callees / code_def / code_refs instead — those return " +
"structural graph data, not text chunks.";
"structural graph data, not text chunks. " +
"For agent memory reads (saved facts + budget-packed retrieval), prefer the " +
"`recall` verb.";
// ──────────────────────────────────────────────────────────────────────────────
// v0.32.6 — contradiction probe MCP surface (M3)
src/core/operations.ts
+213 -10
View File
@@ -26,6 +26,10 @@ import { buildVisibilityClause } from './search/sql-ranking.ts';
import { bumpLastRetrievedAt } from './last-retrieved.ts';
import { isSearchMode } from './search/mode.ts';
import { stampEvidence } from './search/evidence.ts';
import { packToBudget, estimateTokens, resultTokens } from './search/token-budget.ts';
import { isAvailable } from './ai/gateway.ts';
import { verbOperations, MEMORY_VERBS_VERSION } from './verbs.ts';
export { MEMORY_VERBS_VERSION };
import type { SearchResult } from './types.ts';
import { CJK_SLUG_CHARS, PAGE_SLUG_SEG } from './cjk.ts';
import { ALL_SOURCES } from './source-id.ts';
@@ -72,10 +76,27 @@ export type ErrorCode =
| 'rate_limited' // v0.31: gateway rate-limit upstream
| 'extraction_failed' // v0.31: facts extractor failed (refusal, parse, abort)
| 'fact_not_found' // v0.31: forget_fact / recall on unknown id
// MEMORY_VERBS v1 protocol codes (frozen — docs/protocol/MEMORY_VERBS_v1.md).
// Coarse on purpose: codes are for branching (configure/retry vs caller bug
// vs server bug); the freeform `detail` field carries specifics.
| 'not_found' // verb-level: unknown fact id (forget)
| 'scope_denied' // verb-level: OAuth scope / trust-boundary refusal
| 'provenance_required' // remember: provenance missing or empty
| 'unavailable' // a required dependency cannot serve (no API key, gateway down, model refusal)
| 'budget_unsatisfiable' // RESERVED in v1 — schema-listed, never returned
// eslint-disable-next-line @typescript-eslint/ban-types
| (string & {}); // OPEN union for forward-compat (eE7 / D13)
export class OperationError extends Error {
/**
* MEMORY_VERBS v1: verb handlers set `protocolVersion` (=1) and may set
* `detail` (freeform specifics, e.g. which dependency failed). Both are
* additive — non-verb ops never set them and their envelopes are unchanged
* (undefined keys drop out of JSON.stringify).
*/
public detail?: string;
public protocolVersion?: number;
constructor(
public code: ErrorCode,
message: string,
@@ -92,10 +113,29 @@ export class OperationError extends Error {
message: this.message,
suggestion: this.suggestion,
docs: this.docs,
detail: this.detail,
protocol_version: this.protocolVersion,
};
}
}
/**
* MEMORY_VERBS v1 error constructor. Every verb error carries a populated
* `suggestion` (problem + cause + fix — agents read it and self-correct;
* conformance asserts non-empty) and `protocol_version: 1`.
*/
export function verbError(
code: ErrorCode,
message: string,
suggestion: string,
detail?: string,
): OperationError {
const e = new OperationError(code, message, suggestion);
e.protocolVersion = MEMORY_VERBS_VERSION;
if (detail !== undefined) e.detail = detail;
return e;
}
// --- Upload validators (Fix 1 / B5 / H5 / M4) ---
/**
@@ -342,7 +382,8 @@ export function normalizeSlugPrefix(prefix: string): string {
/**
* Write ops a slug-bound client may call: every op that routes through
* `enforceClientSlugFence`, plus `think` (scope `write`, but remote callers
* `enforceClientSlugFence`, plus `think` (scope `read` for remote callers;
* it stays on this list because it is `mutating` locally, but remote callers
* cannot persist — `save`/`take` are forced false for `remote !== false`).
*
* This list is an ALLOW-list on purpose. The fence used to be enforced op
@@ -951,6 +992,22 @@ export interface Operation {
*/
scope?: 'read' | 'write' | 'admin' | 'sources_admin' | 'users_admin';
localOnly?: boolean;
/**
* MEMORY_VERBS v1: marks the five frozen protocol verbs (recall, remember,
* entity, synthesize, forget). `gbrain serve --surface verbs` exposes
* EXACTLY the ops with `verb: true`; `full` (default) exposes everything.
*/
verb?: boolean;
/**
* MCP ToolAnnotations passthrough (SDK 1.29+). Emitted by buildToolDefs
* ONLY when set — existing tools keep byte-identical definitions.
*/
annotations?: {
title?: string;
readOnlyHint?: boolean;
destructiveHint?: boolean;
idempotentHint?: boolean;
};
cliHints?: {
name?: string;
/**
@@ -2282,7 +2339,7 @@ const takes_calibration: Operation = {
const think: Operation = {
name: 'think',
description: 'Multi-hop synthesis across pages + takes + graph. Pulls relevant evidence and produces a cited answer with conflict + gap analysis.',
scope: 'write',
scope: 'read',
params: {
question: { type: 'string', required: true, description: 'The question to think about' },
anchor: { type: 'string', description: 'Pull the entity subgraph around this slug' },
@@ -2293,6 +2350,8 @@ const think: Operation = {
since: { type: 'string', description: 'Start of temporal window (YYYY-MM-DD or YYYY-MM)' },
until: { type: 'string', description: 'End of temporal window' },
},
// Local CLI can persist with save/take; remote/MCP callers are forced
// read-only below before runThink/persistSynthesis sees those flags.
mutating: true,
handler: async (ctx, p) => {
const remote = ctx.remote ?? true;
@@ -2322,7 +2381,7 @@ const think: Operation = {
until: p.until ? String(p.until) : undefined,
takesHoldersAllowList: ctx.takesHoldersAllowList,
...thinkScope,
remote: ctx.remote === true,
remote: ctx.remote !== false, // fail-closed: anything not strictly false is untrusted (CLAUDE.md invariant)
});
// Persist if --save was passed locally
@@ -3107,6 +3166,9 @@ const get_chunks: Operation = {
slug: { type: 'string', required: true },
},
handler: async (ctx, p) => {
// #2555: route through the canonical scope ladder (federated array >
// scalar floor > nothing) instead of the pre-#2200 scalar-only pattern —
// a federated grant could read the page via get_page but got [] here.
return ctx.engine.getChunks(p.slug as string, sourceScopeOpts(ctx));
},
scope: 'read',
@@ -4159,7 +4221,7 @@ const find_trajectory: Operation = {
const points = await ctx.engine.findTrajectory({
entitySlug: p.entity_slug,
...scope,
remote: ctx.remote === true,
remote: ctx.remote !== false, // fail-closed: anything not strictly false is untrusted (CLAUDE.md invariant)
metric,
kind,
since,
@@ -4457,7 +4519,7 @@ const sources_status: Operation = {
const extract_facts: Operation = {
name: 'extract_facts',
description:
'v0.31: extract personal-knowledge facts (events, preferences, commitments, beliefs) from a conversation turn into the per-source hot memory. Sanitizes turn_text via INJECTION_PATTERNS, calls Haiku to extract structured claims, runs the cosine fast-path + classifier dedup pipeline, INSERTs into facts. Returns counts by status. Skips extraction when the turn is dream-generated content (anti-loop).',
'v0.31: extract personal-knowledge facts (events, preferences, commitments, beliefs) from a conversation turn into the per-source hot memory. Sanitizes turn_text via INJECTION_PATTERNS, calls Haiku to extract structured claims, runs the cosine fast-path + classifier dedup pipeline, INSERTs into facts. Returns counts by status. Skips extraction when the turn is dream-generated content (anti-loop). For agent memory writes of a SINGLE already-formed fact, prefer the `remember` verb (zero LLM, mandatory provenance).',
params: {
turn_text: { type: 'string', required: true, description: 'The user message or page body to extract facts from. Sanitized via INJECTION_PATTERNS before the LLM call.' },
session_id: { type: 'string', description: 'Opaque session id (e.g. topic-id from MCP _meta.session_id, or CLI --session). Stored on each fact for the recall --session filter. Not an auth surface.' },
@@ -4517,18 +4579,22 @@ const extract_facts: Operation = {
const recall: Operation = {
name: 'recall',
description:
'v0.31: query per-source hot memory (facts table). Filters by entity / since / session. Remote callers see only visibility=world facts. Returns most-recent first. v0.32 adds optional include_pending to return pending_consolidation_count alongside facts in one round trip.',
'MEMORY VERB (v1): retrieve saved facts/snippets — the protocol read verb. Filters hot-memory facts by entity / since / session_id; pass `query` to ALSO run hybrid search over pages (results[] arm); pass `budget_tokens` for server-side packing (response reports budget_used + dropped_count — never trims client-side). Remote callers see visibility=world facts only. Routing: for ONE known person/company/project card use `entity` (zero LLM); for broad questions needing reasoning use `synthesize` (expensive). Branch on structured fields (status/kind/evidence), never on prose. Every response carries protocol_version.',
params: {
entity: { type: 'string', description: 'Entity slug (canonical). Returns facts about this entity newest first.' },
since: { type: 'string', description: 'ISO datetime or duration shorthand (e.g. "8 hours ago"). Returns facts created since.' },
query: { type: 'string', description: 'MEMORY_VERBS v1: free-text retrieval over pages (hybrid search arm). Response adds results[] (slug, title, chunk, evidence, create_safety, provenance). Combinable with entity (both arms run). Degrades to keyword-only search when no embedding provider is configured (search_degraded notes it; never an error).' },
budget_tokens: { type: 'number', description: 'MEMORY_VERBS v1: server-side token budget (char/4 estimate). Facts pack first, then results. Response adds budget_tokens, budget_used, dropped_count.' },
since: { type: 'string', description: 'ISO 8601 datetime or duration shorthand (e.g. "8 hours ago"). Filters the FACTS arm only.' },
session_id: { type: 'string', description: 'Source session id (e.g. topic-A). Returns facts captured in that session.' },
include_expired: { type: 'boolean', description: 'When true, include expired_at IS NOT NULL rows. Default false.' },
supersessions: { type: 'boolean', description: 'When true, return only the supersession audit log (expired_at + superseded_by both set).' },
limit: { type: 'number', description: 'Max rows to return. Default 50, cap 100.' },
limit: { type: 'number', description: 'Per-arm cap: max fact rows AND max search results. Default 50, cap 100.' },
grep: { type: 'string', description: 'Substring filter on fact text (case-insensitive). Applied client-side after recall.' },
include_pending: { type: 'boolean', description: 'v0.32: when true, response includes pending_consolidation_count (facts not yet promoted to takes by the dream-cycle consolidate phase). One round trip; backward-compatible (field omitted when false).' },
},
scope: 'read',
verb: true,
annotations: { title: 'recall (memory read)', readOnlyHint: true },
handler: async (ctx, p) => {
const sourceId = ctx.sourceId ?? 'default';
const limit = typeof p.limit === 'number' ? p.limit : 50;
@@ -4599,8 +4665,58 @@ const recall: Operation = {
}
}
// ── MEMORY_VERBS v1 — query arm (G1B superset). Hybrid search over pages
// when `query` is present; degrades to keyword-only with a note (never an
// error) when no embedding provider is configured [F-B].
const queryText = typeof p.query === 'string' && p.query.trim().length > 0 ? p.query.trim() : null;
const budgetTokens =
typeof p.budget_tokens === 'number' && Number.isFinite(p.budget_tokens) && p.budget_tokens > 0
? Math.floor(p.budget_tokens)
: null;
let searchResults: SearchResult[] = [];
let searchDegraded: string | undefined;
if (queryText) {
const searchScope = sourceScopeOpts(ctx);
if (!isAvailable('embedding')) {
const raw = await ctx.engine.searchKeyword(queryText, { limit, ...searchScope });
searchResults = dedupResults(raw);
stampEvidenceSafe(searchResults);
await stampContentFlags(ctx.engine, searchResults);
searchDegraded = 'keyword_only_no_embedding_provider';
} else {
searchResults = await hybridSearchCached(ctx.engine, queryText, {
limit,
expansion: false,
...searchScope,
});
}
bumpLastRetrievedAt(ctx.engine, searchResults.map(r => r.page_id));
}
// ── MEMORY_VERBS v1 — server-side budget packing. Facts pack first (cheap,
// high-precision one-liners, per-arm limit-capped so starvation is bounded),
// then search results take the remainder. packToBudget treats budget<=0 as
// a no-op, so an exhausted remainder must drop explicitly.
let packedFacts = rows;
let packedResults = searchResults;
let budgetUsed: number | undefined;
let droppedCount: number | undefined;
if (budgetTokens !== null) {
const factsPack = packToBudget(rows, r => estimateTokens(r.fact), budgetTokens);
packedFacts = factsPack.items;
const remaining = budgetTokens - factsPack.meta.used;
const resultsPack =
remaining > 0
? packToBudget(searchResults, resultTokens, remaining)
: { items: [] as SearchResult[], meta: { budget: 0, used: 0, dropped: searchResults.length, kept: 0 } };
packedResults = resultsPack.items;
budgetUsed = factsPack.meta.used + resultsPack.meta.used;
droppedCount = factsPack.meta.dropped + resultsPack.meta.dropped;
}
return {
facts: rows.map(r => ({
facts: packedFacts.map(r => ({
id: r.id,
fact: r.fact,
kind: r.kind,
@@ -4620,9 +4736,33 @@ const recall: Operation = {
source_session: r.source_session,
confidence: r.confidence,
created_at: r.created_at.toISOString(),
// MEMORY_VERBS v1 additive fields (G1B). `fact_id` is the opaque
// STRING id the `forget` verb accepts (legacy numeric `id` stays for
// pre-v1 consumers — legacy fields are frozen byte-equal). `provenance`
// is the protocol name for the stored source attribution.
fact_id: String(r.id),
provenance: r.source,
})),
total: rows.length,
total: packedFacts.length,
...(pending_consolidation_count !== undefined ? { pending_consolidation_count } : {}),
// MEMORY_VERBS v1 envelope (G1B superset — additive on every response).
protocol_version: MEMORY_VERBS_VERSION,
...(queryText
? {
results: packedResults.map(r => ({
slug: r.slug,
title: r.title,
chunk: r.chunk_text,
evidence: r.evidence,
create_safety: r.create_safety,
provenance: r.slug,
})),
...(searchDegraded ? { search_degraded: searchDegraded } : {}),
}
: {}),
...(budgetTokens !== null
? { budget_tokens: budgetTokens, budget_used: budgetUsed, dropped_count: droppedCount }
: {}),
};
},
};
@@ -4683,6 +4823,65 @@ function parseSinceParam(raw: unknown): Date | null {
return null;
}
/**
* MEMORY_VERBS v1 — parse the `remember` verb's `ttl` param into a
* `valid_until` Date. Sibling of parseSinceParam, pointed FORWARD.
*
* Accepted forms (frozen in docs/protocol/MEMORY_VERBS_v1.md):
* - relative duration shorthand: '30d', '12h', '45m', '90s' (also
* spelled-out: '30 days', '12 hours') → now + duration
* - absolute ISO 8601 date or datetime: '2026-07-12', '2026-07-12T00:00:00Z'
*
* Explicitly REJECTED with a self-correcting suggestion: ISO-8601 duration
* syntax ('P30D', 'PT12H') — agents that read "ISO 8601" as durations get a
* fix, not a mystery. Returns null for null/undefined/empty (= never expires).
* Throws verbError('invalid_params') on anything unparseable.
*/
export function parseTtlParam(raw: unknown): Date | null {
if (raw == null) return null;
if (typeof raw !== 'string') {
throw verbError(
'invalid_params',
`ttl must be a string, got ${typeof raw}.`,
'Pass a duration like "30d" or "12h", or an absolute ISO 8601 timestamp like "2026-07-12T00:00:00Z".',
);
}
const s = raw.trim();
if (!s) return null;
// ISO-8601 DURATION syntax is a documented trap — reject with the fix.
if (/^P(T|\d)/i.test(s) && /^P(?:\d+[YMWD])*(?:T(?:\d+[HMS])+)?$/i.test(s)) {
throw verbError(
'invalid_params',
`ttl "${s}" looks like an ISO-8601 duration, which is not accepted.`,
`Use the shorthand form instead (e.g. "${s.replace(/^PT?/i, '').toLowerCase()}" style: "30d", "12h"), or an absolute ISO 8601 expiry timestamp.`,
);
}
// Relative duration shorthand → now + duration.
const dur = s.match(/^(\d+)\s*(s|sec|seconds?|m|min|minutes?|h|hr|hours?|d|days?)$/i);
if (dur) {
const n = parseInt(dur[1], 10);
const unit = dur[2].toLowerCase();
const ms =
unit.startsWith('s') ? n * 1000 :
unit.startsWith('m') ? n * 60 * 1000 :
unit.startsWith('h') ? n * 60 * 60 * 1000 :
n * 24 * 60 * 60 * 1000;
return new Date(Date.now() + ms);
}
// Absolute ISO 8601 date or datetime.
const iso = Date.parse(s);
if (Number.isFinite(iso)) return new Date(iso);
throw verbError(
'invalid_params',
`Cannot parse ttl "${s}".`,
'Pass a duration like "30d" or "12h", or an absolute ISO 8601 timestamp like "2026-07-12T00:00:00Z". Omit ttl for a fact that never expires.',
);
}
// ──────────────────────────────────────────────────────────────────────────────
// v0.34 Cathedral III — code-intelligence ops (MCP-exposed).
//
@@ -6166,6 +6365,10 @@ const extraction_review: Operation = {
};
export const operations: Operation[] = [
// MEMORY_VERBS v1 (Cathedral 1) — remember/entity/synthesize/forget live in
// verbs.ts; the fifth verb is the extended `recall` op below. Spread first
// so `--surface verbs` agents see them at the top of the tool list.
...verbOperations,
// Page CRUD
get_page, put_page, delete_page, list_pages,
// v0.26.5 destructive-guard ops (page-level soft-delete + recovery + admin purge)
src/core/pglite-engine.ts
+39 -3
View File
@@ -751,7 +751,13 @@ export class PGLiteEngine implements BrainEngine {
EXISTS (SELECT 1 FROM information_schema.tables
WHERE table_schema='public' AND table_name='timeline_entries') AS timeline_entries_exists,
EXISTS (SELECT 1 FROM information_schema.columns
WHERE table_schema='public' AND table_name='timeline_entries' AND column_name='event_page_id') AS timeline_event_page_id_exists
WHERE table_schema='public' AND table_name='timeline_entries' AND column_name='event_page_id') AS timeline_event_page_id_exists,
EXISTS (SELECT 1 FROM information_schema.tables
WHERE table_schema='public' AND table_name='minion_jobs') AS minion_jobs_exists,
EXISTS (SELECT 1 FROM information_schema.columns
WHERE table_schema='public' AND table_name='minion_jobs' AND column_name='timeout_at') AS minion_jobs_timeout_at_exists,
EXISTS (SELECT 1 FROM information_schema.columns
WHERE table_schema='public' AND table_name='minion_jobs' AND column_name='idempotency_key') AS minion_jobs_idempotency_key_exists
`);
const probe = rows[0] as {
pages_exists: boolean;
@@ -796,6 +802,9 @@ export class PGLiteEngine implements BrainEngine {
pages_links_extracted_at_exists: boolean;
timeline_entries_exists: boolean;
timeline_event_page_id_exists: boolean;
minion_jobs_exists: boolean;
minion_jobs_timeout_at_exists: boolean;
minion_jobs_idempotency_key_exists: boolean;
};
const needsPagesBootstrap = probe.pages_exists && !probe.source_id_exists;
@@ -874,6 +883,12 @@ export class PGLiteEngine implements BrainEngine {
const needsPagesLinksExtractedAt = probe.pages_exists && !probe.pages_links_extracted_at_exists;
// v121: schema-blob indexes reference event_page_id before migrations run.
const needsTimelineEventPageId = probe.timeline_entries_exists && !probe.timeline_event_page_id_exists;
// v7-era (#2626 class sweep): minion_jobs.timeout_at + idempotency_key are
// migration-added AND referenced by blob indexes (idx_minion_jobs_timeout,
// uniq_minion_jobs_idempotency) — a pre-v7 minion_jobs wedges blob replay
// exactly like the v121 incident.
const needsMinionJobsTimeoutAt = probe.minion_jobs_exists && !probe.minion_jobs_timeout_at_exists;
const needsMinionJobsIdempotencyKey = probe.minion_jobs_exists && !probe.minion_jobs_idempotency_key_exists;
// Fresh installs (no tables yet) and modern brains both no-op.
if (!needsPagesBootstrap && !needsLinksBootstrap && !needsChunksBootstrap
@@ -886,7 +901,8 @@ export class PGLiteEngine implements BrainEngine {
&& !needsContextualRetrievalColumns && !needsPagesGeneration
&& !needsPagesEmbeddingSignature
&& !needsPagesLinksExtractedAt
&& !needsTimelineEventPageId) return;
&& !needsTimelineEventPageId
&& !needsMinionJobsTimeoutAt && !needsMinionJobsIdempotencyKey) return;
process.stderr.write(' Pre-v0.21 brain detected, applying forward-reference bootstrap\n');
@@ -1141,6 +1157,20 @@ export class PGLiteEngine implements BrainEngine {
ALTER TABLE timeline_entries ADD COLUMN IF NOT EXISTS event_page_id INTEGER;
`);
}
if (needsMinionJobsTimeoutAt) {
// v7: blob index idx_minion_jobs_timeout references timeout_at; a
// pre-v7 minion_jobs wedges blob replay without it (same class as v121).
await this.db.exec(`
ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS timeout_at TIMESTAMPTZ;
`);
}
if (needsMinionJobsIdempotencyKey) {
// v7: blob index uniq_minion_jobs_idempotency references idempotency_key.
await this.db.exec(`
ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS idempotency_key TEXT;
`);
}
}
async withReservedConnection<T>(fn: (conn: ReservedConnection) => Promise<T>): Promise<T> {
@@ -2664,8 +2694,14 @@ export class PGLiteEngine implements BrainEngine {
async getChunks(slug: string, opts?: { sourceId?: string; sourceIds?: string[] }): Promise<Chunk[]> {
const sourceIds = opts?.sourceIds && opts.sourceIds.length > 0 ? opts.sourceIds : undefined;
const source = sourceIds ?? opts?.sourceId ?? 'default';
// #2544: explicit non-vector column list — rowToChunk discards embeddings
// at this call site, so `cc.*` shipped every vector only to be thrown away.
const { rows } = await this.db.query(
`SELECT cc.* FROM content_chunks cc
`SELECT cc.id, cc.page_id, cc.chunk_index, cc.chunk_text, cc.chunk_source,
cc.model, cc.token_count, cc.embedded_at, cc.language,
cc.symbol_name, cc.symbol_type, cc.start_line, cc.end_line,
cc.parent_symbol_path, cc.doc_comment, cc.symbol_name_qualified, cc.modality
FROM content_chunks cc
JOIN pages p ON p.id = cc.page_id
WHERE p.slug = $1 AND ${sourceIds ? 'p.source_id = ANY($2::text[])' : 'p.source_id = $2'}
ORDER BY cc.chunk_index`,
src/core/postgres-engine.ts
+42 -3
View File
@@ -624,7 +624,13 @@ export class PostgresEngine implements BrainEngine {
EXISTS (SELECT 1 FROM information_schema.tables
WHERE table_schema = current_schema() AND table_name = 'timeline_entries') AS timeline_entries_exists,
EXISTS (SELECT 1 FROM information_schema.columns
WHERE table_schema = current_schema() AND table_name = 'timeline_entries' AND column_name = 'event_page_id') AS timeline_event_page_id_exists
WHERE table_schema = current_schema() AND table_name = 'timeline_entries' AND column_name = 'event_page_id') AS timeline_event_page_id_exists,
EXISTS (SELECT 1 FROM information_schema.tables
WHERE table_schema = current_schema() AND table_name = 'minion_jobs') AS minion_jobs_exists,
EXISTS (SELECT 1 FROM information_schema.columns
WHERE table_schema = current_schema() AND table_name = 'minion_jobs' AND column_name = 'timeout_at') AS minion_jobs_timeout_at_exists,
EXISTS (SELECT 1 FROM information_schema.columns
WHERE table_schema = current_schema() AND table_name = 'minion_jobs' AND column_name = 'idempotency_key') AS minion_jobs_idempotency_key_exists
`;
const probe = probeRows[0]!;
@@ -703,6 +709,9 @@ export class PostgresEngine implements BrainEngine {
pages_links_extracted_at_exists?: boolean;
timeline_entries_exists?: boolean;
timeline_event_page_id_exists?: boolean;
minion_jobs_exists?: boolean;
minion_jobs_timeout_at_exists?: boolean;
minion_jobs_idempotency_key_exists?: boolean;
};
const needsContextualRetrievalColumns = (probe.pages_exists
&& (!probeCr.pages_cr_mode_exists || !probeCr.pages_corpus_generation_exists))
@@ -725,6 +734,14 @@ export class PostgresEngine implements BrainEngine {
// v121: schema-blob indexes reference event_page_id before migrations run.
const needsTimelineEventPageId = probeCr.timeline_entries_exists === true
&& !probeCr.timeline_event_page_id_exists;
// v7-era (#2626 class sweep): minion_jobs.timeout_at + idempotency_key are
// migration-added AND referenced by blob indexes (idx_minion_jobs_timeout,
// uniq_minion_jobs_idempotency) — a pre-v7 minion_jobs wedges blob replay
// exactly like the v121 incident.
const needsMinionJobsTimeoutAt = probeCr.minion_jobs_exists === true
&& !probeCr.minion_jobs_timeout_at_exists;
const needsMinionJobsIdempotencyKey = probeCr.minion_jobs_exists === true
&& !probeCr.minion_jobs_idempotency_key_exists;
if (!needsPagesBootstrap && !needsLinksBootstrap && !needsChunksBootstrap
&& !needsPagesDeletedAt && !needsMcpLogBootstrap && !needsSubagentProviderId
@@ -736,7 +753,8 @@ export class PostgresEngine implements BrainEngine {
&& !needsContextualRetrievalColumns && !needsPagesGeneration
&& !needsPagesEmbeddingSignature
&& !needsPagesLinksExtractedAt
&& !needsTimelineEventPageId) return;
&& !needsTimelineEventPageId
&& !needsMinionJobsTimeoutAt && !needsMinionJobsIdempotencyKey) return;
process.stderr.write(' Pre-v0.21 brain detected, applying forward-reference bootstrap\n');
@@ -991,6 +1009,20 @@ export class PostgresEngine implements BrainEngine {
ALTER TABLE timeline_entries ADD COLUMN IF NOT EXISTS event_page_id INTEGER;
`);
}
if (needsMinionJobsTimeoutAt) {
// v7: blob index idx_minion_jobs_timeout references timeout_at; a
// pre-v7 minion_jobs wedges blob replay without it (same class as v121).
await conn.unsafe(`
ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS timeout_at TIMESTAMPTZ;
`);
}
if (needsMinionJobsIdempotencyKey) {
// v7: blob index uniq_minion_jobs_idempotency references idempotency_key.
await conn.unsafe(`
ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS idempotency_key TEXT;
`);
}
}
async transaction<T>(fn: (engine: BrainEngine) => Promise<T>): Promise<T> {
@@ -2613,8 +2645,15 @@ export class PostgresEngine implements BrainEngine {
const scope = sourceIds
? tx`p.source_id = ANY(${sourceIds}::text[])`
: tx`p.source_id = ${scalarSourceId}`;
// #2544: explicit non-vector column list — rowToChunk discards
// embeddings at this call site (includeEmbedding defaults false), so
// `cc.*` shipped every vector over the wire only to be thrown away.
const rows = await tx`
SELECT cc.* FROM content_chunks cc
SELECT cc.id, cc.page_id, cc.chunk_index, cc.chunk_text, cc.chunk_source,
cc.model, cc.token_count, cc.embedded_at, cc.language,
cc.symbol_name, cc.symbol_type, cc.start_line, cc.end_line,
cc.parent_symbol_path, cc.doc_comment, cc.symbol_name_qualified, cc.modality
FROM content_chunks cc
JOIN pages p ON p.id = cc.page_id
WHERE p.slug = ${slug} AND ${scope}
ORDER BY cc.chunk_index
src/core/search/query-intent.ts
+61
View File
@@ -340,3 +340,64 @@ export function intentToDetail(intent: QueryIntent): 'low' | 'medium' | 'high' |
export function autoDetectDetail(query: string): 'low' | 'medium' | 'high' | undefined {
return classifyQuery(query).suggestedDetail;
}
// ─────────────────────────────────────────────────────────
// #2416 — concept-shaped query detection (CLI nudge)
// ─────────────────────────────────────────────────────────
// Fuzzy-quantifier / landscape cues. A concept-shaped question asks for a
// SET defined by meaning ("all the X that do Y", "the landscape of Z") —
// exactly where `query`'s multi-query expansion recovers synonym- and
// outcome-phrased matches that the expansion-off `search` op can miss.
//
// Deliberately EXCLUDED cues (owned by other routers — a nudge toward
// `query` on these would fight their descriptions):
// - "who are the …" → find_experts
// - bare "anything …" → get_recent_salience / find_anomalies
const CONCEPT_CUE_PATTERNS: RegExp[] = [
/\b(all|every)\b.+\b(that|who|which|doing|with|about|related to)\b/i,
/\b(find|list|show)\s+(all|every|everything)\b/i,
/\beverything\s+(about|on|matching|related)\b/i,
/\bthe\s+(landscape|ecosystem|space|universe)\s+of\b/i,
/\b(landscape|ecosystem)\s+(of|around)\b/i,
/\bwhich\s+\w+[\w\s]*\b(do|does|are|have|use|work)\b/i,
];
// Exact-identifier anti-signals: the query names a specific thing, so the
// cheap `search` op is the right tool and a nudge would be noise.
const CONCEPT_ANTI_PATTERNS: RegExp[] = [
/["'“”][^"'“”]+["'“”]/, // quoted phrase — exact-match intent
/\b[a-z0-9]+(?:-[a-z0-9]+){1,}\b/, // slug-like token (kebab-case)
];
/**
* True when a query is concept-shaped: it carries a fuzzy-quantifier or
* landscape cue AND no exact-identifier anti-signal. Tuned to favor
* false-negatives (silence) over false-positives (noise): short queries,
* quoted phrases, slugs, and entity lookups (per classifyQueryIntent)
* never trigger. Pure function; no LLM, no DB.
*/
export function looksConceptShaped(query: string): boolean {
const q = query.trim();
if (q.split(/\s+/).length < 3) return false; // bare token / proper-noun lookup
if (!matches(CONCEPT_CUE_PATTERNS, q)) return false;
if (matches(CONCEPT_ANTI_PATTERNS, q)) return false;
if (classifyQueryIntent(q) === 'entity') return false;
return true;
}
/**
* One-line CLI hint steering a concept-shaped `search` toward `query`.
* Returns null when the query is not concept-shaped. Message generation
* lives here (not in cli.ts) so the full string is unit-testable; the CLI
* wiring is a two-liner per dispatch path, stderr only, `--quiet`-gated.
*/
export function conceptNudge(query: string): string | null {
if (!looksConceptShaped(query)) return null;
const preview = query.length > 60 ? `${query.slice(0, 57)}...` : query;
return (
`hint: concept-shaped question — try \`gbrain query "${preview}"\` ` +
`(adds multi-query expansion; recovers synonym-phrased matches search can miss). ` +
`A nonzero search count is not proof of completeness.`
);
}
src/core/search/token-budget.ts
+52 -36
View File
@@ -63,50 +63,66 @@ export interface TokenBudgetMeta {
}
/**
* Greedy top-down budget enforcement. Walks the input in order, accumulates
* token costs, and stops as soon as adding the next result would exceed
* the budget. Results are NOT re-ranked — caller's order is preserved.
* Generic greedy top-down budget packer (v1 memory-verbs protocol). Walks
* the input in order, accumulates per-item costs via the caller-supplied
* cost function, and stops as soon as adding the next item would exceed
* the budget. Items are NOT re-ranked — caller's order is preserved.
*
* Edge cases (all preserve the pre-v0.32 contract):
* - budget undefined / <= 0: returns input unchanged; dropped=0, kept=N.
* - First result alone exceeds budget: returns []; dropped=N, kept=0.
* - First item alone exceeds budget: returns []; dropped=N, kept=0.
* (Intentionally strict: the caller asked for a hard cap.)
* - Input empty: returns []; budget unused.
*/
export function packToBudget<T>(
items: T[],
cost: (item: T) => number,
budget: number | undefined,
): { items: T[]; meta: TokenBudgetMeta } {
const safeBudget = typeof budget === 'number' && budget > 0 ? budget : 0;
if (safeBudget === 0 || items.length === 0) {
return {
items,
meta: {
budget: safeBudget,
used: items.reduce((acc, it) => acc + cost(it), 0),
dropped: 0,
kept: items.length,
},
};
}
const kept: T[] = [];
let used = 0;
for (const it of items) {
const c = cost(it);
if (used + c > safeBudget) break;
kept.push(it);
used += c;
}
return {
items: kept,
meta: {
budget: safeBudget,
used,
dropped: items.length - kept.length,
kept: kept.length,
},
};
}
/**
* Search-pipeline budget enforcement — a thin wrapper over packToBudget
* with the SearchResult cost model (title + chunk_text). Behavior is
* byte-identical to the pre-refactor implementation; pinned by
* test/token-budget.test.ts.
*/
export function enforceTokenBudget(
results: SearchResult[],
budget: number | undefined,
): { results: SearchResult[]; meta: TokenBudgetMeta } {
const safeBudget = typeof budget === 'number' && budget > 0 ? budget : 0;
if (safeBudget === 0 || results.length === 0) {
return {
results,
meta: {
budget: safeBudget,
used: results.reduce((acc, r) => acc + resultTokens(r), 0),
dropped: 0,
kept: results.length,
},
};
}
const kept: SearchResult[] = [];
let used = 0;
for (const r of results) {
const cost = resultTokens(r);
if (used + cost > safeBudget) break;
kept.push(r);
used += cost;
}
return {
results: kept,
meta: {
budget: safeBudget,
used,
dropped: results.length - kept.length,
kept: kept.length,
},
};
const { items, meta } = packToBudget(results, resultTokens, budget);
return { results: items, meta };
}
src/core/source-id.ts
+40
View File
@@ -63,3 +63,43 @@ export function assertValidSourceId(s: unknown): asserts s is string {
);
}
}
/**
* Normalize the optional `source` field of an `/admin/api/register-client`
* request body into a write source_id.
*
* Mirrors the CLI's `--source` flag. Returns the literal `'default'` when the
* field is omitted (`undefined`/`null`) so every caller that doesn't send
* `source` keeps landing on source_id='default' (the pre-source HTTP
* register-client behavior). A present-but-invalid value throws (via
* `assertValidSourceId`) so the route can surface a structured 400 instead of
* failing at INSERT time.
*/
export function normalizeSourceInput(raw: unknown): string {
if (raw === undefined || raw === null) return 'default';
assertValidSourceId(raw);
return raw;
}
/**
* Normalize the optional `federatedRead` field of an
* `/admin/api/register-client` request body into a source_id array, or
* `undefined` when omitted.
*
* Mirrors the CLI's `--federated-read` flag. `undefined`/`null` → `undefined`
* so `registerClientManual` applies its own default (`[sourceId]`, a
* non-federated client whose read scope equals its write scope). A present
* value must be a non-empty array whose every element is a valid source_id;
* anything else throws for a structured 400.
*/
export function normalizeFederatedReadInput(raw: unknown): string[] | undefined {
if (raw === undefined || raw === null) return undefined;
if (!Array.isArray(raw) || raw.length === 0) {
throw new Error(
`Invalid federatedRead: ${JSON.stringify(raw)}. ` +
`Must be a non-empty array of source_ids.`,
);
}
for (const s of raw) assertValidSourceId(s);
return raw as string[];
}
src/core/think/index.ts
+35 -13
View File
@@ -140,6 +140,13 @@ export interface ThinkResult {
* pre-existing/test `ThinkResult` literals → treated as persistable (back-compat).
*/
synthesisOk?: boolean;
/**
* MEMORY_VERBS v1 [E2] — gateway token usage for the synthesis call(s),
* summed across rounds. Best-effort: null when no LLM ran (graceful stub),
* when a test client returns no usage, or when a provider omits accounting.
* The synthesize verb maps this to its frozen `cost` block.
*/
usage?: { input_tokens: number; output_tokens: number } | null;
/** Only set when --save was true and the caller persisted a synthesis page. */
savedSlug?: string;
/** Diagnostics for `--explain` callers (CLI surface for v0.29). */
@@ -149,15 +156,6 @@ export interface ThinkResult {
takesFromVector: number;
graphHits: number;
};
/**
* Token usage from the real LLM call, when one happened. Undefined on the
* no-client/stub paths (no Anthropic key, model not usable) — same
* distinction `synthesisOk` already makes. `think`'s cost was previously
* unsurfaced anywhere: not in this CLI's own output, not in
* `budget_ledger`, and invisible to a wrapping caller's own token
* accounting (the LLM call `think` makes is its own separate API call).
*/
usage?: { input_tokens: number; output_tokens: number };
/** USD cost computed from `usage` + `canonicalLookup(modelUsed)`, when both are available. */
cost_usd?: number;
}
@@ -171,8 +169,19 @@ const DEFAULT_MAX_OUTPUT_TOKENS = 4000;
// keeps 4000.
const THINKING_DEFAULT_MAX_OUTPUT_TOKENS = 16000;
const THINKING_BY_DEFAULT_MODEL_RE = /^anthropic[:/]claude-[a-z0-9]+-5(?:[.-]|$)/i;
// OpenAI reasoning models spend output budget on internal reasoning tokens
// the same way — reasoning tokens are billed as output and count against
// `max_tokens` — so they get the same headroom. Deliberately scoped to the
// gpt-5 family and the numbered o-series only; anything else (gpt-4o, the
// non-reasoning `*-chat` snapshots like gpt-5-chat-latest, other providers'
// reasoning models routed through their own recipes) keeps the conservative
// 4000 default.
const OPENAI_REASONING_MODEL_RE = /^openai[:/](?:gpt-5|o[0-9]+)(?:[.-]|$)/i;
const OPENAI_CHAT_SNAPSHOT_RE = /-chat(?:-|$)/i; // gpt-5-chat-latest, gpt-5.2-chat-latest
export function maxOutputTokensFor(modelStr: string): number {
return THINKING_BY_DEFAULT_MODEL_RE.test(modelStr)
const openaiReasoning =
OPENAI_REASONING_MODEL_RE.test(modelStr) && !OPENAI_CHAT_SNAPSHOT_RE.test(modelStr);
return THINKING_BY_DEFAULT_MODEL_RE.test(modelStr) || openaiReasoning
? THINKING_DEFAULT_MAX_OUTPUT_TOKENS
: DEFAULT_MAX_OUTPUT_TOKENS;
}
@@ -451,8 +460,10 @@ export async function runThink(
// sentinel, which is non-JSON) and on the no-client early return below; the final
// return ANDs it with a non-empty-answer check (catches valid-but-empty JSON).
let synthesisOk = true;
// [E2] best-effort usage aggregation across synthesis calls (single-pass in
// v0.28+, but summed so the round loop inherits it when gap-fill lands).
let usage: { input_tokens: number; output_tokens: number } | null = null;
let response: ThinkResponse;
let usage: { input_tokens: number; output_tokens: number } | undefined;
if (opts.stubResponse) {
response = opts.stubResponse;
} else {
@@ -502,6 +513,7 @@ export async function runThink(
rounds: 0,
warnings,
synthesisOk: false, // #1698: no LLM ran — never persist this
usage: null, // [E2] no LLM ran — no accounting
diagnostics: {
pagesFromHybrid: gather.diagnostics.pagesFromHybrid,
takesFromKeyword: gather.diagnostics.takesFromKeyword,
@@ -516,7 +528,17 @@ export async function runThink(
system: systemPrompt,
messages: [{ role: 'user', content: userMessage }],
});
usage = { input_tokens: result.usage.input_tokens, output_tokens: result.usage.output_tokens };
// [E2] capture usage when the message carries it (test-injected clients
// and providers without accounting leave it null).
const u = (result as { usage?: { input_tokens?: number; output_tokens?: number } }).usage;
if (u && typeof u.input_tokens === 'number' && typeof u.output_tokens === 'number') {
// Single synthesis call in v0.28+; when gap-driven rounds land, sum here.
const prev = usage as { input_tokens: number; output_tokens: number } | null;
usage = {
input_tokens: (prev?.input_tokens ?? 0) + u.input_tokens,
output_tokens: (prev?.output_tokens ?? 0) + u.output_tokens,
};
}
const block = result.content.find(b => b.type === 'text');
const text = block && 'text' in block ? block.text : '';
const parsed = tryParseJSON(text);
@@ -561,13 +583,13 @@ export async function runThink(
// #1698: persistable only when a real synthesis produced a non-empty answer.
// ANDs the not-JSON/sentinel flag with a content check (catches valid-but-empty JSON).
synthesisOk: synthesisOk && response.answer.trim().length > 0,
usage,
diagnostics: {
pagesFromHybrid: gather.diagnostics.pagesFromHybrid,
takesFromKeyword: gather.diagnostics.takesFromKeyword,
takesFromVector: gather.diagnostics.takesFromVector,
graphHits: gather.diagnostics.graphHits,
},
usage,
};
}
src/core/verbs.ts
+559
View File
@@ -0,0 +1,559 @@
/**
* MEMORY_VERBS v1 — the frozen memory protocol verbs (Cathedral 1).
*
* Four first-class Operations (`remember`, `entity`, `synthesize`, `forget`)
* that join the extended `recall` op (operations.ts) as the five-verb façade
* over the operation catalog. Frozen contract: docs/protocol/MEMORY_VERBS_v1.md
* — field names and semantics in v1 never change; additions are forever-
* additive; `protocol_version` rides every response; errors carry enumerated
* codes + populated `suggestion` (agents read it and self-correct).
*
* These are ordinary Operations: they inherit trust-boundary fail-closed
* semantics (ctx.remote), scope enforcement, and source isolation like every
* other op. `gbrain serve --surface verbs` exposes exactly the ops marked
* `verb: true`.
*
* Import-cycle note: operations.ts spreads these into its `operations` array
* at MODULE-EVAL time, so this file must be a RUNTIME LEAF — it may import
* operations.ts types (erased) but never its values statically. Handlers load
* verbError/parseTtlParam/sourceScopeOpts via dynamic import (the file's
* existing style), which resolves after both modules finish evaluating.
* MEMORY_VERBS_VERSION lives HERE (operations.ts imports it from us) for the
* same reason. Violating this reintroduces the TDZ crash on whichever module
* evaluates second.
*/
import type { Operation } from './operations.ts';
/** Frozen protocol version for the MEMORY_VERBS v1 verb set. Single source of truth. */
export const MEMORY_VERBS_VERSION = 1;
export const VERB_NAMES = ['recall', 'remember', 'entity', 'synthesize', 'forget'] as const;
export type VerbName = (typeof VERB_NAMES)[number];
const FACT_KINDS = ['event', 'preference', 'commitment', 'belief', 'fact'] as const;
const PROVENANCE_MAX = 500;
// ─── remember ────────────────────────────────────────────────────────────────
const remember: Operation = {
name: 'remember',
description:
'MEMORY VERB (v1): save one fact to durable agent memory — the protocol write verb. ' +
'provenance is REQUIRED (free text, e.g. "conversation 2026-06-12", "user said in chat", "import: notes.md"). ' +
'Set `entity` whenever the fact is about a specific person/company/project — entity-scoped recall will not find it otherwise. ' +
'ttl accepts duration shorthand ("30d", "12h") or an absolute ISO 8601 timestamp; ISO-8601 durations like "P30D" are rejected with a fix. ' +
'visibility defaults to "world" (readable by every agent connected to this brain; pass "private" for local-CLI-only facts). ' +
'Response: branch on `status` (inserted|duplicate|superseded), never on `status_text` (human rendering only). ' +
'On duplicate, `id` is the EXISTING fact\'s id. For bulk extraction from a raw transcript use extract_facts instead.',
params: {
fact: { type: 'string', required: true, description: 'The fact to remember, one claim per call.' },
provenance: {
type: 'string',
required: true,
description:
'Where this fact came from (REQUIRED, free text, max 500 chars). Examples: "conversation 2026-06-12", "user said in chat", "import: meeting-notes.md".',
},
ttl: {
type: 'string',
description:
'Optional expiry: duration shorthand ("30d", "12h", "45m") or absolute ISO 8601 timestamp ("2026-07-12T00:00:00Z"). NOT ISO-8601 durations ("P30D" is rejected). Omit = never expires.',
},
entity: {
type: 'string',
description:
'Person/company/project this fact is about (name or slug; canonicalized server-side). Set it whenever the fact has a subject — entity-scoped recall misses unattributed facts.',
},
kind: {
type: 'string',
enum: [...FACT_KINDS],
description: 'Fact kind: event | preference | commitment | belief | fact (default).',
},
visibility: {
type: 'string',
enum: ['world', 'private'],
description:
'world (default): readable by every agent connected to this brain — required for the remote remember→recall round-trip. private: local CLI reads only.',
},
},
mutating: true,
scope: 'write',
verb: true,
annotations: { title: 'remember (memory write)', idempotentHint: true },
handler: async (ctx, p) => {
const { verbError, parseTtlParam } = await import('./operations.ts');
const fact = typeof p.fact === 'string' ? p.fact.trim() : '';
if (!fact) {
throw verbError(
'invalid_params',
'fact must be a non-empty string.',
'Pass the claim to remember, e.g. fact: "picked Stripe over Adyen — onboarding speed".',
);
}
const provenance = typeof p.provenance === 'string' ? p.provenance.trim() : '';
if (!provenance) {
throw verbError(
'provenance_required',
'provenance is required and must be non-empty.',
'Pass where the fact came from, e.g. provenance: "user told me, 2026-06-12" or "import: notes.md".',
);
}
if (provenance.length > PROVENANCE_MAX) {
throw verbError(
'invalid_params',
`provenance exceeds ${PROVENANCE_MAX} chars (got ${provenance.length}).`,
'Shorten the attribution — provenance is a pointer, not a transcript.',
);
}
const kind = typeof p.kind === 'string' ? p.kind : 'fact';
if (!FACT_KINDS.includes(kind as (typeof FACT_KINDS)[number])) {
throw verbError(
'invalid_params',
`kind "${kind}" is not a fact kind.`,
`Use one of: ${FACT_KINDS.join(' | ')}.`,
);
}
const visibility = typeof p.visibility === 'string' ? p.visibility : 'world';
if (visibility !== 'world' && visibility !== 'private') {
throw verbError(
'invalid_params',
`visibility "${visibility}" is not valid.`,
'Use "world" (default — agents can recall it) or "private" (local CLI reads only).',
);
}
const validUntil = parseTtlParam(p.ttl); // throws verbError(invalid_params) on bad input
if (ctx.dryRun) {
return {
dry_run: true,
action: 'remember',
fact,
protocol_version: MEMORY_VERBS_VERSION,
};
}
const { writeSingleFact } = await import('./facts/write-single.ts');
const result = await writeSingleFact(ctx.engine, ctx.sourceId ?? 'default', {
fact,
provenance,
kind: kind as (typeof FACT_KINDS)[number],
entity: typeof p.entity === 'string' && p.entity.trim() ? p.entity.trim() : null,
visibility,
validUntil,
});
const statusText =
result.status === 'inserted'
? `remembered as fact #${result.id}`
: result.status === 'duplicate'
? `already knew this — kept fact #${result.id}`
: `updated — fact #${result.id} supersedes the previous version`;
return {
// Opaque STRING at the protocol level [T4]; gbrain serializes its ints.
id: String(result.id),
status: result.status,
status_text: statusText,
entity_slug: result.entity_slug ?? null,
valid_until: result.valid_until ? result.valid_until.toISOString() : null,
...(result.degraded_dedup ? { degraded_dedup: true } : {}),
protocol_version: MEMORY_VERBS_VERSION,
};
},
cliHints: { name: 'remember', positional: ['fact'] },
};
// ─── entity ──────────────────────────────────────────────────────────────────
const entity: Operation = {
name: 'entity',
description:
'MEMORY VERB (v1): inspect ONE known person/company/project card — zero LLM calls, sub-100ms. ' +
'Resolution: alias > exact title > slug-suffix; ties break on most-recently-touched. ' +
'NEVER errors on a miss: returns found:false plus near-miss suggestions with create_safety hints ' +
'(exists | probable | unknown — whether writing a new page would duplicate). ' +
'Routing: for facts/snippets retrieval use recall; for broad questions needing reasoning use synthesize (expensive).',
params: {
name: { type: 'string', required: true, description: 'Free-text name, alias, or slug (e.g. "Alice Example", "people/alice-example").' },
},
scope: 'read',
verb: true,
annotations: { title: 'entity (card lookup, zero LLM)', readOnlyHint: true },
handler: async (ctx, p) => {
const { verbError } = await import('./operations.ts');
const name = typeof p.name === 'string' ? p.name.trim() : '';
if (!name) {
throw verbError(
'invalid_params',
'name must be a non-empty string.',
'Pass the entity to look up, e.g. name: "Alice Example" or name: "people/alice-example".',
);
}
const t0 = Date.now();
const { buildEntityCard } = await import('./verbs/entity-card.ts');
const result = await buildEntityCard(ctx.engine, ctx.sourceId ?? 'default', name, {
remote: ctx.remote !== false,
});
return {
protocol_version: MEMORY_VERBS_VERSION,
found: result.found,
latency_ms: Date.now() - t0,
...(result.card ? { card: result.card } : {}),
...(result.suggestions !== undefined ? { suggestions: result.suggestions } : {}),
};
},
cliHints: { name: 'entity', positional: ['name'] },
};
// ─── synthesize ──────────────────────────────────────────────────────────────
const synthesize: Operation = {
name: 'synthesize',
description:
'[EXPENSIVE / SLOW — makes LLM calls, seconds-to-minutes latency, costs money] ' +
'MEMORY VERB (v1): answer a broad question using cross-page LLM reasoning with citations and gap analysis. ' +
'Prefer recall (facts/snippets) or entity (one known card, zero LLM) for lookups — use synthesize only when the answer ' +
'requires combining evidence across pages. Response carries a best-effort cost block (model, tokens, usd_estimate).',
params: {
question: { type: 'string', required: true, description: 'The question to answer.' },
since: { type: 'string', description: 'Optional temporal window start (ISO 8601 date or datetime).' },
until: { type: 'string', description: 'Optional temporal window end (ISO 8601 date or datetime).' },
},
scope: 'read',
verb: true,
annotations: { title: 'synthesize (slow, costly — LLM-backed)', readOnlyHint: true },
handler: async (ctx, p) => {
const { verbError, sourceScopeOpts } = await import('./operations.ts');
const question = typeof p.question === 'string' ? p.question.trim() : '';
if (!question) {
throw verbError(
'invalid_params',
'question must be a non-empty string.',
'Pass the question to synthesize an answer for, e.g. question: "what is our payments strategy?".',
);
}
const scope = sourceScopeOpts(ctx);
const { runThink } = await import('./think/index.ts');
// Remote-safe delegation: save/take are NEVER offered through this verb,
// for any caller — the verb is a pure read.
const result = await runThink(ctx.engine, {
question,
since: p.since ? String(p.since) : undefined,
until: p.until ? String(p.until) : undefined,
takesHoldersAllowList: ctx.takesHoldersAllowList,
...(scope.sourceId !== undefined ? { sourceId: scope.sourceId } : {}),
...(scope.sourceIds !== undefined ? { allowedSources: scope.sourceIds } : {}),
remote: ctx.remote === true,
});
// [c10] runThink degrades gracefully to a no-LLM stub RESULT; the protocol
// contract converts that state into an explicit `unavailable` error so
// agents branch on configure/retry instead of relaying a fake answer.
if (result.warnings.includes('NO_ANTHROPIC_API_KEY')) {
throw verbError(
'unavailable',
'synthesize needs an LLM and none is configured.',
'Set an API key (e.g. `gbrain config set anthropic_api_key sk-...` or ANTHROPIC_API_KEY) and retry. recall and entity work without one.',
'chat gateway unconfigured (NO_ANTHROPIC_API_KEY)',
);
}
// Best-effort cost block [E5/m3]: actual tokens when the gateway reported
// usage, priced via the canonical table; nulls when accounting is absent.
const { canonicalLookup } = await import('./model-pricing.ts');
const usage = result.usage ?? null;
const pricing = canonicalLookup(result.modelUsed);
const usdEstimate =
usage && pricing
? (usage.input_tokens * pricing.input + usage.output_tokens * pricing.output) / 1_000_000
: null;
return {
answer: result.answer,
sources: result.citations.map(c => c.page_slug),
gaps: result.gaps,
cost: {
model: result.modelUsed,
input_tokens: usage?.input_tokens ?? null,
output_tokens: usage?.output_tokens ?? null,
usd_estimate: usdEstimate,
},
protocol_version: MEMORY_VERBS_VERSION,
};
},
cliHints: { name: 'synthesize', positional: ['question'] },
};
// ─── forget ──────────────────────────────────────────────────────────────────
const forget: Operation = {
name: 'forget',
description:
'MEMORY VERB (v1): expire a remembered fact by id — the protocol delete verb. ' +
'`id` is the opaque string id returned by remember and recall (facts[].fact_id) — never a page slug. ' +
'Idempotent: forgetting an already-expired fact returns expired:false (success), unknown id returns a not_found error. ' +
'The fact is expired (audit trail kept), not deleted.',
params: {
id: { type: 'string', required: true, description: 'Opaque fact id from remember/recall (facts[].fact_id). Never a page slug.' },
reason: { type: 'string', description: 'Optional reason, written to the fact\'s audit trail. Default: "forgotten".' },
},
mutating: true,
scope: 'write',
verb: true,
annotations: { title: 'forget (expire a fact)', destructiveHint: true, idempotentHint: true },
handler: async (ctx, p) => {
const { verbError } = await import('./operations.ts');
const rawId = typeof p.id === 'string' ? p.id.trim() : typeof p.id === 'number' ? String(p.id) : '';
const numericId = Number(rawId);
if (!rawId || !Number.isInteger(numericId) || numericId <= 0) {
throw verbError(
'not_found',
`No fact with id "${String(p.id)}".`,
'Pass the opaque string id returned by remember or recall (facts[].fact_id) — page slugs are not fact ids.',
);
}
const reason = typeof p.reason === 'string' && p.reason.trim() ? p.reason.trim() : null;
if (ctx.dryRun) {
return { dry_run: true, action: 'forget', id: rawId, protocol_version: MEMORY_VERBS_VERSION };
}
const { forgetFactInFence } = await import('./facts/forget.ts');
// [ship P1.1] trust boundary: scope the forget to the caller's source, and
// for remote callers to world-visible facts only — a guessed global id
// can't expire facts outside the caller's source or reach private facts.
const result = await forgetFactInFence(ctx.engine, numericId, {
...(reason ? { reason } : {}),
sourceId: ctx.sourceId ?? 'default',
worldOnly: ctx.remote !== false,
});
if (!result.ok && result.path === 'not_found') {
throw verbError(
'not_found',
`No fact with id "${rawId}".`,
'Ids come from remember/recall (facts[].fact_id). recall the entity first to find the right fact.',
);
}
if (!result.ok && result.path === 'already_expired') {
// Idempotent re-forget: success, nothing changed.
return {
id: rawId,
expired: false,
reason,
protocol_version: MEMORY_VERBS_VERSION,
};
}
return {
id: rawId,
expired: true,
reason,
protocol_version: MEMORY_VERBS_VERSION,
};
},
// NO cliHints: `gbrain forget` is a CLI_ONLY command (recall.ts runForget)
// that dispatches BEFORE cliOps — a cliHint here would be silently
// shadowed. The CLI surface for forgetting stays the existing command;
// this verb is the MCP/protocol surface.
};
export const verbOperations: Operation[] = [remember, entity, synthesize, forget];
// ─── RESPONSE_SCHEMAS — the protocol's response-shape registry [c8] ─────────
//
// `Operation` carries input params only; response envelopes live HERE, hand-
// authored, and conformance validates LIVE responses against this registry so
// registry-vs-code drift is caught by the same fixtures that certify servers.
// Field names and semantics are FROZEN (additive-forever); enum values are
// part of the contract.
const EVIDENCE_ENUM = ['alias_hit', 'exact_title_match', 'high_vector_match', 'keyword_exact', 'weak_semantic'];
const CREATE_SAFETY_ENUM = ['exists', 'probable', 'unknown'];
const STATUS_ENUM = ['inserted', 'duplicate', 'superseded'];
export const RESPONSE_SCHEMAS: Record<VerbName, Record<string, unknown>> = {
recall: {
type: 'object',
required: ['facts', 'total', 'protocol_version'],
properties: {
protocol_version: { type: 'integer', const: MEMORY_VERBS_VERSION },
total: { type: 'integer' },
facts: {
type: 'array',
items: {
type: 'object',
required: ['id', 'fact', 'kind', 'fact_id', 'provenance'],
properties: {
id: { type: 'integer', description: 'LEGACY numeric id (pre-v1 consumers). Use fact_id.' },
fact_id: { type: 'string', description: 'Opaque protocol id — the value forget accepts.' },
fact: { type: 'string' },
kind: { type: 'string', enum: FACT_KINDS as unknown as string[] },
entity_slug: { type: ['string', 'null'] },
provenance: { type: 'string' },
valid_until: { type: ['string', 'null'] },
visibility: { type: 'string', enum: ['private', 'world'] },
},
},
},
results: {
type: 'array',
description: 'Search arm — present only when `query` was passed.',
items: {
type: 'object',
required: ['slug', 'title', 'evidence', 'create_safety', 'provenance'],
properties: {
slug: { type: 'string' },
title: { type: ['string', 'null'] },
chunk: { type: ['string', 'null'] },
evidence: { type: 'string', enum: EVIDENCE_ENUM },
create_safety: { type: 'string', enum: CREATE_SAFETY_ENUM },
provenance: { type: 'string', description: 'Origin page slug.' },
},
},
},
search_degraded: { type: 'string', description: 'Present when the search arm fell back to keyword-only (no embedding provider).' },
budget_tokens: { type: 'integer', description: 'Present when budget_tokens was passed.' },
budget_used: { type: 'integer' },
dropped_count: { type: 'integer' },
},
},
remember: {
type: 'object',
required: ['id', 'status', 'status_text', 'entity_slug', 'valid_until', 'protocol_version'],
properties: {
protocol_version: { type: 'integer', const: MEMORY_VERBS_VERSION },
id: { type: 'string', description: 'Opaque fact id. On status=duplicate this is the EXISTING fact\'s id.' },
status: { type: 'string', enum: STATUS_ENUM, description: 'Branch on THIS, never on status_text.' },
status_text: { type: 'string', description: 'Human rendering of status. Display only — never branch on it.' },
entity_slug: { type: ['string', 'null'] },
valid_until: { type: ['string', 'null'], description: 'ISO 8601 or null (never expires).' },
degraded_dedup: { type: 'boolean', description: 'Present (true) when no embedding provider — near-duplicates may insert.' },
},
},
entity: {
type: 'object',
required: ['protocol_version', 'found', 'latency_ms'],
properties: {
protocol_version: { type: 'integer', const: MEMORY_VERBS_VERSION },
found: { type: 'boolean' },
latency_ms: { type: 'integer' },
card: {
type: 'object',
required: ['entity', 'aka', 'summary', 'last_touched', 'open_threads', 'edges', 'backlink_count', 'active_fact_count'],
properties: {
entity: {
type: 'object',
required: ['slug', 'title', 'type'],
properties: { slug: { type: 'string' }, title: { type: 'string' }, type: { type: ['string', 'null'] } },
},
aka: { type: 'array', items: { type: 'string' } },
summary: { type: 'string' },
last_touched: {
type: 'object',
required: ['updated_at', 'last_retrieved_at', 'last_timeline_date'],
properties: {
updated_at: { type: ['string', 'null'] },
last_retrieved_at: { type: ['string', 'null'] },
last_timeline_date: { type: ['string', 'null'] },
},
},
open_threads: {
type: 'array',
items: {
type: 'object',
required: ['kind', 'text', 'date'],
properties: {
kind: { type: 'string', enum: ['commitment', 'recent_event'] },
text: { type: 'string' },
date: { type: ['string', 'null'] },
},
},
},
edges: {
type: 'array',
items: {
type: 'object',
required: ['type', 'direction', 'slug'],
properties: {
type: { type: 'string' },
direction: { type: 'string', enum: ['out', 'in'] },
slug: { type: 'string' },
context: { type: ['string', 'null'] },
},
},
},
backlink_count: { type: 'integer' },
active_fact_count: { type: 'integer' },
},
},
suggestions: {
type: 'array',
items: {
type: 'object',
required: ['slug', 'title', 'create_safety'],
properties: {
slug: { type: 'string' },
title: { type: 'string' },
create_safety: { type: 'string', enum: CREATE_SAFETY_ENUM },
},
},
},
},
},
synthesize: {
type: 'object',
required: ['answer', 'sources', 'cost', 'protocol_version'],
properties: {
protocol_version: { type: 'integer', const: MEMORY_VERBS_VERSION },
answer: { type: 'string' },
sources: { type: 'array', items: { type: 'string' } },
gaps: { type: 'array', items: { type: 'string' } },
cost: {
type: 'object',
required: ['model', 'input_tokens', 'output_tokens', 'usd_estimate'],
description: 'Best-effort aggregate (retries/multi-call flows sum; cache hits may undercount). Honest signal, not an invoice.',
properties: {
model: { type: 'string' },
input_tokens: { type: ['integer', 'null'] },
output_tokens: { type: ['integer', 'null'] },
usd_estimate: { type: ['number', 'null'] },
},
},
},
},
forget: {
type: 'object',
required: ['id', 'expired', 'reason', 'protocol_version'],
properties: {
protocol_version: { type: 'integer', const: MEMORY_VERBS_VERSION },
id: { type: 'string' },
expired: { type: 'boolean', description: 'true = this call expired the fact; false = it was ALREADY expired (idempotent re-forget).' },
reason: { type: ['string', 'null'] },
},
},
};
/** Error envelope schema (uniform across all five verbs). */
export const ERROR_SCHEMA: Record<string, unknown> = {
type: 'object',
required: ['error', 'message'],
properties: {
error: {
type: 'string',
enum: [
'invalid_params',
'provenance_required',
'not_found',
'scope_denied',
'unavailable',
'budget_unsatisfiable', // RESERVED — schema-listed, never returned in v1
'internal',
],
},
message: { type: 'string' },
suggestion: { type: 'string', description: 'Populated on every verb error: problem + cause + fix.' },
detail: { type: 'string', description: 'Freeform specifics (e.g. which dependency failed).' },
protocol_version: { type: 'integer', const: MEMORY_VERBS_VERSION },
},
};
src/core/verbs/conformance-fixtures.ts
+226
View File
@@ -0,0 +1,226 @@
/**
* MEMORY_VERBS v1 — conformance fixtures (E2).
*
* The canonical, embedded fixture set the conformance runner executes against
* ANY MCP endpoint (default: gbrain's own server). Deterministic by
* construction: every check is SHAPE (required fields, enum validity, const),
* CONTRACT BEHAVIOR (provenance rejected, idempotent forget, miss→found:false),
* or ROUND-TRIP (remember → recall by entity, a plain indexed read — no
* embeddings, no ranking judgment). Ranking quality is BrainBench's job.
*
* These double as BrainBench seeds: `test/fixtures/memory-verbs/cases.json` is
* the generated data-file mirror (drift-guarded by the conformance test) so
* the eval suite imports the same cases without importing gbrain source.
*
* Stateful round-trips share a per-run marker (substituted for `{{marker}}`)
* and thread ids via `saveAs`/`{{id:<key>}}` substitution.
*/
export interface ConformanceCase {
name: string;
verb: 'recall' | 'remember' | 'entity' | 'synthesize' | 'forget';
/** `{{marker}}` and `{{id:<key>}}` substitute at run time. */
params: Record<string, unknown>;
/** Validate the (parsed) response body against RESPONSE_SCHEMAS[verb]. */
validateSchema?: boolean;
/** Expect an isError response with this protocol code. */
expectErrorCode?: string;
/** On error responses: suggestion must be non-empty (F-D mandate). */
expectSuggestion?: boolean;
/** Field equality/predicate checks on the parsed response body. */
expect?: Array<
| { path: string; equals: unknown }
| { path: string; oneOf: unknown[] }
| { path: string; type: 'string' | 'number' | 'boolean' | 'array' | 'object' }
| { path: string; gte: number }
| { path: string; lte: number }
| { path: string; nonEmptyString: true }
| { path: string; absentOrNotContains: string }
>;
/** Save a response field for later cases ({{id:<key>}}). */
saveAs?: { key: string; path: string };
/** Only run when the runner was invoked with --synthesize (cost gate). */
requiresSynthesizeFlag?: boolean;
/**
* Only run when the runner could seed the conformance entity PAGE (via
* put_page when the target exposes it — full surface). entity() resolves
* pages; on a verbs-only target with no seeding path these cases skip
* honestly instead of failing on a structurally absent page.
*/
requiresSeededEntity?: boolean;
}
export const CONFORMANCE_CASES: ConformanceCase[] = [
// ── remember: contract behavior ─────────────────────────────────────────
{
name: 'remember rejects missing provenance (provenance_required + suggestion)',
verb: 'remember',
params: { fact: 'conformance {{marker}} fact without provenance' },
expectErrorCode: 'invalid_params', // MCP-level required-param rejection
expectSuggestion: false, // transport-level validation may not carry one
},
{
name: 'remember rejects empty provenance (provenance_required + suggestion)',
verb: 'remember',
params: { fact: 'conformance {{marker}} fact empty provenance', provenance: ' ' },
expectErrorCode: 'provenance_required',
expectSuggestion: true,
},
{
name: 'remember rejects ISO-8601 duration ttl with a fix (P30D trap)',
verb: 'remember',
params: { fact: 'conformance {{marker}} ttl trap', provenance: 'conformance run', ttl: 'P30D' },
expectErrorCode: 'invalid_params',
expectSuggestion: true,
},
{
name: 'remember writes a fact (string id, enum status, echoed nulls)',
verb: 'remember',
params: {
fact: 'conformance {{marker}}: the protocol round-trip fact',
provenance: 'conformance run {{marker}}',
entity: 'people/conformance-{{marker}}',
kind: 'fact',
},
validateSchema: true,
expect: [
{ path: 'id', type: 'string' },
{ path: 'status', oneOf: ['inserted', 'duplicate', 'superseded'] },
{ path: 'protocol_version', equals: 1 },
],
saveAs: { key: 'fact1', path: 'id' },
},
{
name: 'remember with ttl returns ISO valid_until',
verb: 'remember',
params: {
fact: 'conformance {{marker}}: expiring fact',
provenance: 'conformance run {{marker}}',
entity: 'people/conformance-{{marker}}',
ttl: '30d',
},
validateSchema: true,
expect: [{ path: 'valid_until', type: 'string' }],
saveAs: { key: 'fact2', path: 'id' },
},
{
name: 'remember private fact (fence test setup)',
verb: 'remember',
params: {
fact: 'conformance {{marker}} PRIVATE-SENTINEL commitment',
provenance: 'conformance run {{marker}}',
entity: 'people/conformance-{{marker}}',
kind: 'commitment',
visibility: 'private',
},
validateSchema: true,
},
// ── recall: round-trip + superset + budget ──────────────────────────────
{
name: 'recall by entity round-trips the remembered fact (superset envelope)',
verb: 'recall',
params: { entity: 'people/conformance-{{marker}}' },
validateSchema: true,
expect: [
{ path: 'protocol_version', equals: 1 },
{ path: 'total', gte: 1 },
{ path: 'facts.0.fact_id', type: 'string' },
{ path: 'facts.0.provenance', type: 'string' },
],
},
{
name: 'recall with budget reports consistent budget meta',
verb: 'recall',
params: { entity: 'people/conformance-{{marker}}', budget_tokens: 10000 },
validateSchema: true,
expect: [
{ path: 'budget_tokens', equals: 10000 },
{ path: 'budget_used', gte: 0 },
{ path: 'budget_used', lte: 10000 },
{ path: 'dropped_count', gte: 0 },
],
},
{
name: 'recall with budget smaller than the first item drops everything',
verb: 'recall',
params: { entity: 'people/conformance-{{marker}}', budget_tokens: 1 },
validateSchema: true,
expect: [
{ path: 'total', equals: 0 },
{ path: 'dropped_count', gte: 1 },
],
},
{
name: 'recall with query returns the search arm (degradation allowed, never an error)',
verb: 'recall',
params: { query: 'conformance {{marker}} protocol round-trip', budget_tokens: 8000 },
validateSchema: true,
expect: [{ path: 'results', type: 'array' }],
},
// ── entity: hit, miss, privacy fence ────────────────────────────────────
{
name: 'entity resolves the round-trip entity to a card',
verb: 'entity',
params: { name: 'people/conformance-{{marker}}' },
validateSchema: true,
requiresSeededEntity: true,
expect: [
{ path: 'found', equals: true },
{ path: 'protocol_version', equals: 1 },
{ path: 'card.entity.slug', type: 'string' },
{ path: 'card.backlink_count', gte: 0 },
],
},
{
name: 'entity miss returns found:false + suggestions (never an error)',
verb: 'entity',
params: { name: 'zzz-no-such-entity-{{marker}}' },
validateSchema: true,
expect: [
{ path: 'found', equals: false },
{ path: 'suggestions', type: 'array' },
],
},
{
name: 'entity card never leaks private facts to remote callers (fence test)',
verb: 'entity',
params: { name: 'people/conformance-{{marker}}' },
requiresSeededEntity: true,
expect: [{ path: 'card.open_threads', absentOrNotContains: 'PRIVATE-SENTINEL' }],
},
// ── forget: idempotency + not_found ─────────────────────────────────────
{
name: 'forget expires the remembered fact',
verb: 'forget',
params: { id: '{{id:fact2}}', reason: 'conformance cleanup' },
validateSchema: true,
expect: [{ path: 'expired', equals: true }],
},
{
name: 'forget again is idempotent (expired:false, success)',
verb: 'forget',
params: { id: '{{id:fact2}}' },
validateSchema: true,
expect: [{ path: 'expired', equals: false }],
},
{
name: 'forget unknown id returns not_found with a suggestion',
verb: 'forget',
params: { id: '999999999' },
expectErrorCode: 'not_found',
expectSuggestion: true,
},
// ── synthesize: cost-gated live call ────────────────────────────────────
{
name: 'synthesize answers (or reports unavailable) — cost-gated, pass --synthesize',
verb: 'synthesize',
params: { question: 'What do we know about conformance {{marker}}?' },
requiresSynthesizeFlag: true,
// Either a schema-valid answer (key configured) or a clean unavailable
// error (no key). The runner accepts both; anything else fails.
},
];
src/core/verbs/conformance.ts
+269
View File
@@ -0,0 +1,269 @@
/**
* MEMORY_VERBS v1 — conformance runner core (E2).
*
* Executes the embedded fixture set (conformance-fixtures.ts) against any MCP
* endpoint via a minimal client surface (list_tools + call_tool only) and
* returns a pass/fail table. Transport-agnostic: the CLI wraps it with stdio
* spawn / HTTP transports; the negative self-test [F3] feeds it a mutated
* in-process double.
*
* Validation is deliberately NON-STRICT on extra fields — the contract is
* additive-forever, so unknown fields are always legal. A certifier that
* rejected additions would break the versioning policy it certifies.
*/
import { RESPONSE_SCHEMAS, ERROR_SCHEMA, MEMORY_VERBS_VERSION, type VerbName } from '../verbs.ts';
import { CONFORMANCE_CASES, type ConformanceCase } from './conformance-fixtures.ts';
export interface ConformanceClient {
listTools(): Promise<Array<{ name: string; description?: string; annotations?: unknown }>>;
callTool(name: string, args: Record<string, unknown>): Promise<{ isError?: boolean; text: string }>;
}
export interface CaseResult {
name: string;
verb: string;
status: 'pass' | 'fail' | 'skip';
detail: string;
}
export interface ConformanceReport {
protocol_version: number;
results: CaseResult[];
passed: number;
failed: number;
skipped: number;
ok: boolean;
}
/**
* Minimal JSON-Schema-subset validator: type (incl. union arrays + integer),
* required, properties (NON-strict), enum, const, items. Returns violations
* as "path: problem" strings.
*/
export function validateAgainstSchema(
value: unknown,
schema: Record<string, unknown>,
path = '$',
): string[] {
const out: string[] = [];
const t = schema.type as string | string[] | undefined;
if (t !== undefined) {
const types = Array.isArray(t) ? t : [t];
if (!types.some(ty => matchesType(value, ty))) {
out.push(`${path}: expected ${types.join('|')}, got ${describe(value)}`);
return out; // structural mismatch — deeper checks are noise
}
}
if (schema.const !== undefined && value !== schema.const) {
out.push(`${path}: expected const ${JSON.stringify(schema.const)}, got ${JSON.stringify(value)}`);
}
if (Array.isArray(schema.enum) && value !== null && !schema.enum.includes(value)) {
out.push(`${path}: ${JSON.stringify(value)} not in enum [${(schema.enum as unknown[]).join(', ')}]`);
}
if (value !== null && typeof value === 'object' && !Array.isArray(value)) {
const obj = value as Record<string, unknown>;
for (const req of (schema.required as string[] | undefined) ?? []) {
if (!(req in obj)) out.push(`${path}.${req}: required field missing`);
}
const props = (schema.properties as Record<string, Record<string, unknown>> | undefined) ?? {};
for (const [k, sub] of Object.entries(props)) {
if (k in obj && obj[k] !== undefined) out.push(...validateAgainstSchema(obj[k], sub, `${path}.${k}`));
}
}
if (Array.isArray(value) && schema.items && typeof schema.items === 'object') {
value.forEach((item, i) =>
out.push(...validateAgainstSchema(item, schema.items as Record<string, unknown>, `${path}[${i}]`)),
);
}
return out;
}
function matchesType(v: unknown, t: string): boolean {
switch (t) {
case 'string': return typeof v === 'string';
case 'integer': return typeof v === 'number' && Number.isInteger(v);
case 'number': return typeof v === 'number';
case 'boolean': return typeof v === 'boolean';
case 'array': return Array.isArray(v);
case 'object': return v !== null && typeof v === 'object' && !Array.isArray(v);
case 'null': return v === null;
default: return false;
}
}
function describe(v: unknown): string {
if (v === null) return 'null';
if (Array.isArray(v)) return 'array';
return typeof v;
}
function getPath(obj: unknown, path: string): unknown {
let cur: unknown = obj;
for (const seg of path.split('.')) {
if (cur === null || cur === undefined) return undefined;
cur = (cur as Record<string, unknown>)[seg];
}
return cur;
}
function substitute(value: unknown, marker: string, ids: Map<string, string>): unknown {
if (typeof value === 'string') {
let s = value.replaceAll('{{marker}}', marker);
s = s.replace(/\{\{id:([a-z0-9_-]+)\}\}/gi, (_, key) => ids.get(key) ?? `MISSING-ID-${key}`);
return s;
}
if (Array.isArray(value)) return value.map(v => substitute(v, marker, ids));
if (value !== null && typeof value === 'object') {
return Object.fromEntries(Object.entries(value as Record<string, unknown>).map(([k, v]) => [k, substitute(v, marker, ids)]));
}
return value;
}
function runChecks(body: unknown, checks: NonNullable<ConformanceCase['expect']>): string[] {
const problems: string[] = [];
for (const c of checks) {
const v = getPath(body, c.path);
if ('equals' in c) {
if (v !== c.equals) problems.push(`${c.path}: expected ${JSON.stringify(c.equals)}, got ${JSON.stringify(v)}`);
} else if ('oneOf' in c) {
if (!c.oneOf.includes(v)) problems.push(`${c.path}: ${JSON.stringify(v)} not in ${JSON.stringify(c.oneOf)}`);
} else if ('type' in c) {
if (!matchesType(v, c.type)) problems.push(`${c.path}: expected ${c.type}, got ${describe(v)}`);
} else if ('gte' in c) {
if (typeof v !== 'number' || v < c.gte) problems.push(`${c.path}: expected >= ${c.gte}, got ${JSON.stringify(v)}`);
} else if ('lte' in c) {
if (typeof v !== 'number' || v > c.lte) problems.push(`${c.path}: expected <= ${c.lte}, got ${JSON.stringify(v)}`);
} else if ('nonEmptyString' in c) {
if (typeof v !== 'string' || !v.trim()) problems.push(`${c.path}: expected non-empty string`);
} else if ('absentOrNotContains' in c) {
const json = v === undefined ? '' : JSON.stringify(v);
if (json.includes(c.absentOrNotContains)) problems.push(`${c.path}: must not contain "${c.absentOrNotContains}"`);
}
}
return problems;
}
export async function runConformance(
client: ConformanceClient,
opts: { marker?: string; synthesize?: boolean } = {},
): Promise<ConformanceReport> {
const marker = opts.marker ?? `run-${Date.now().toString(36)}`;
const ids = new Map<string, string>();
const results: CaseResult[] = [];
// Seed the conformance entity PAGE when the target exposes put_page (full
// surface). entity() resolves pages; verbs-only targets with no seeding
// path skip the entity-hit cases honestly (requiresSeededEntity).
let seededEntity = false;
try {
const seed = await client.callTool('put_page', {
slug: `people/conformance-${marker}`,
content: `---\ntitle: Conformance ${marker}\ntype: person\n---\n\n# Conformance ${marker}\n\nSynthetic entity for a MEMORY_VERBS conformance run.\n`,
});
seededEntity = !seed.isError;
} catch {
seededEntity = false;
}
// List-level checks first: the five verbs are advertised, synthesize is
// marked expensive (description prefix is the load-bearing channel).
try {
const tools = await client.listTools();
const byName = new Map(tools.map(t => [t.name, t]));
for (const verb of Object.keys(RESPONSE_SCHEMAS) as VerbName[]) {
results.push(
byName.has(verb)
? { name: `tools/list advertises ${verb}`, verb, status: 'pass', detail: '' }
: { name: `tools/list advertises ${verb}`, verb, status: 'fail', detail: 'not advertised' },
);
}
const synth = byName.get('synthesize');
const marked = !!synth?.description?.startsWith('[EXPENSIVE');
results.push({
name: 'synthesize is marked expensive ([EXPENSIVE prefix)',
verb: 'synthesize',
status: marked ? 'pass' : 'fail',
detail: marked ? '' : `description starts: "${synth?.description?.slice(0, 40) ?? '(missing)'}..."`,
});
} catch (e) {
results.push({
name: 'tools/list',
verb: '-',
status: 'fail',
detail: e instanceof Error ? e.message : String(e),
});
}
for (const c of CONFORMANCE_CASES) {
if (c.requiresSynthesizeFlag && !opts.synthesize) {
results.push({ name: c.name, verb: c.verb, status: 'skip', detail: 'costs money — pass --synthesize' });
continue;
}
if (c.requiresSeededEntity && !seededEntity) {
results.push({ name: c.name, verb: c.verb, status: 'skip', detail: 'target has no put_page to seed the entity page (verbs-only surface)' });
continue;
}
const params = substitute(c.params, marker, ids) as Record<string, unknown>;
let res: { isError?: boolean; text: string };
try {
res = await client.callTool(c.verb, params);
} catch (e) {
results.push({ name: c.name, verb: c.verb, status: 'fail', detail: `transport: ${e instanceof Error ? e.message : String(e)}` });
continue;
}
let body: unknown;
try {
body = JSON.parse(res.text);
} catch {
results.push({ name: c.name, verb: c.verb, status: 'fail', detail: `response not JSON: ${res.text.slice(0, 120)}` });
continue;
}
const problems: string[] = [];
if (c.verb === 'synthesize' && c.requiresSynthesizeFlag) {
// Accept either a schema-valid answer (key configured) or a clean
// `unavailable` protocol error (no key). Anything else fails.
if (res.isError) {
const err = body as { error?: string; suggestion?: string };
if (err.error !== 'unavailable') problems.push(`expected unavailable, got error=${err.error}`);
if (!err.suggestion?.trim()) problems.push('error.suggestion empty');
problems.push(...validateAgainstSchema(body, ERROR_SCHEMA, '$err'));
} else {
problems.push(...validateAgainstSchema(body, RESPONSE_SCHEMAS.synthesize));
}
} else if (c.expectErrorCode) {
if (!res.isError) {
problems.push(`expected error ${c.expectErrorCode}, got success`);
} else {
const err = body as { error?: string; suggestion?: string; protocol_version?: number };
if (err.error !== c.expectErrorCode) problems.push(`expected error=${c.expectErrorCode}, got ${err.error}`);
if (c.expectSuggestion && !err.suggestion?.trim()) problems.push('error.suggestion empty (F-D mandate)');
}
} else {
if (res.isError) {
problems.push(`unexpected error: ${res.text.slice(0, 160)}`);
} else {
if (c.validateSchema) problems.push(...validateAgainstSchema(body, RESPONSE_SCHEMAS[c.verb]));
if (c.expect) problems.push(...runChecks(body, c.expect));
if (c.saveAs) {
const v = getPath(body, c.saveAs.path);
if (typeof v === 'string') ids.set(c.saveAs.key, v);
else problems.push(`saveAs ${c.saveAs.path}: expected string id, got ${describe(v)}`);
}
}
}
results.push(
problems.length === 0
? { name: c.name, verb: c.verb, status: 'pass', detail: '' }
: { name: c.name, verb: c.verb, status: 'fail', detail: problems.join('; ') },
);
}
const passed = results.filter(r => r.status === 'pass').length;
const failed = results.filter(r => r.status === 'fail').length;
const skipped = results.filter(r => r.status === 'skip').length;
return { protocol_version: MEMORY_VERBS_VERSION, results, passed, failed, skipped, ok: failed === 0 };
}
src/core/verbs/entity-card.ts
+348
View File
@@ -0,0 +1,348 @@
/**
* MEMORY_VERBS v1 — `entity(name)` card builder (zero LLM, p99 < 100ms).
*
* Resolves a free-text name to ONE brain page via the Retrieval Reflex's
* precision-biased arms (alias-first, then exact-title / exact-slug /
* slug-suffix), then assembles a compact self-describing card from parallel
* depth-1 indexed reads. Deliberately NOT the recursive-CTE traversal
* (traversePaths) — the card is a latency contract, not a graph walk.
*
* Resolution precedence (frozen): alias > exact title > slug-suffix; ties
* break on GREATEST(updated_at, last_retrieved_at) — "last_touched" is the
* card's OUTPUT name, not a column. Multi-hit → best match wins, runners-up
* land in `suggestions`. Miss → `found: false` + keyword near-misses with
* create_safety hints. NEVER throws for data reasons; each arm is guarded so
* a pre-page_aliases brain still resolves via arm 2 (same posture as the
* shipped reflex).
*
* Privacy: `summary` runs through safeSynopsis (the get_page fence boundary);
* facts respect visibility for remote callers (world-only).
*/
import type { BrainEngine, FactRow } from '../engine.ts';
import { normalizeAlias } from '../search/alias-normalize.ts';
import { slugify } from '../entities/resolve.ts';
import { safeSynopsis } from '../context/retrieval-reflex.ts';
import { stampEvidence } from '../search/evidence.ts';
import type { SearchResult } from '../types.ts';
const EDGE_CAP = 10;
const OPEN_THREADS_CAP = 3;
const OPEN_THREAD_TIMELINE_WINDOW_DAYS = 90;
const SUGGESTION_CAP = 3;
const FACT_FETCH_CAP = 100;
export interface EntityCardEdge {
type: string;
direction: 'out' | 'in';
slug: string;
context: string | null;
}
export interface EntityOpenThread {
kind: 'commitment' | 'recent_event';
text: string;
date: string | null;
}
export interface EntityCard {
entity: { slug: string; title: string; type: string | null };
/** page_aliases reverse lookup (normalized forms). Empty on pre-migration brains. */
aka: string[];
/** Privacy-safe synopsis — same fence boundary as get_page. */
summary: string;
last_touched: {
updated_at: string | null;
last_retrieved_at: string | null;
last_timeline_date: string | null;
};
/** Best-effort in v1: active commitment-kind facts + recent timeline entries. */
open_threads: EntityOpenThread[];
/** Top typed edges, mentions excluded, out-edges first. */
edges: EntityCardEdge[];
backlink_count: number;
/** Active facts about this entity (capped count; visibility-filtered for remote). */
active_fact_count: number;
}
export interface EntitySuggestion {
slug: string;
title: string;
create_safety: string;
}
export interface EntityCardResult {
found: boolean;
card?: EntityCard;
suggestions?: EntitySuggestion[];
}
interface CardPageRow {
slug: string;
// v0.43 merge: retrieval-reflex's exported PageRow (safeSynopsis's param)
// now requires source_id (federated push-context wave #2095). The card row
// carries it too so it remains assignable.
source_id: string;
title: string;
type: string | null;
frontmatter: Record<string, unknown> | null;
compiled_truth: string | null;
updated_at: Date | string | null;
last_retrieved_at: Date | string | null;
}
/** Resolution arm rank: lower = higher confidence (frozen precedence ladder). */
const ARM_ALIAS = 0;
const ARM_EXACT = 1;
const ARM_SUFFIX = 2;
export async function buildEntityCard(
engine: BrainEngine,
sourceId: string,
name: string,
opts: { remote: boolean },
): Promise<EntityCardResult> {
const trimmed = (name ?? '').trim();
if (!trimmed) return { found: false, suggestions: [] };
const norm = normalizeAlias(trimmed);
const titleLc = trimmed.toLowerCase();
// Two exact-slug candidates: the slugified form for free-text names AND the
// raw input — a caller passing an already-namespaced slug
// ("people/alice-example") must hit exactly (slugify flattens the slash).
const slug = slugify(trimmed);
const exactSlugs = [...new Set([slug, trimmed].filter(Boolean))];
// Candidate slugs with their best arm rank.
const rankBySlug = new Map<string, number>();
const consider = (s: string, rank: number) => {
if (!s) return;
const prev = rankBySlug.get(s);
if (prev === undefined || rank < prev) rankBySlug.set(s, rank);
};
// Arm 1 — alias-first. Guarded: pre-migration brains lack page_aliases.
if (norm) {
try {
const aliasMap = await engine.resolveAliases([norm], { sourceId });
for (const hit of aliasMap.get(norm) ?? []) consider(hit.slug, ARM_ALIAS);
} catch {
/* no page_aliases table — degrade to arm 2 [E3] */
}
}
// Arm 2 — exact title / exact slug / slug-suffix, with the columns the
// card's tie-break needs. Guarded like the reflex.
let rows: CardPageRow[] = [];
try {
rows = await engine.executeRaw<CardPageRow>(
`SELECT slug, source_id, title, type, frontmatter, compiled_truth, updated_at, last_retrieved_at
FROM pages
WHERE deleted_at IS NULL
AND source_id = $1
AND ( lower(title) = $2
OR slug = ANY($3::text[])
OR slug LIKE $4 )`,
[sourceId, titleLc, exactSlugs, `%/${slug || trimmed}`],
);
} catch {
rows = [];
}
const rowBySlug = new Map<string, CardPageRow>();
for (const r of rows) {
rowBySlug.set(r.slug, r);
const isExact = (r.title ?? '').toLowerCase() === titleLc || exactSlugs.includes(r.slug);
consider(r.slug, isExact ? ARM_EXACT : ARM_SUFFIX);
}
// Hydrate alias-resolved slugs that arm 2 didn't fetch.
const missing = [...rankBySlug.keys()].filter(s => !rowBySlug.has(s));
if (missing.length) {
try {
const extra = await engine.executeRaw<CardPageRow>(
`SELECT slug, source_id, title, type, frontmatter, compiled_truth, updated_at, last_retrieved_at
FROM pages
WHERE deleted_at IS NULL AND source_id = $1 AND slug = ANY($2::text[])`,
[sourceId, missing],
);
for (const r of extra) rowBySlug.set(r.slug, r);
} catch {
/* stale alias rows — drop */
}
}
// Rank candidates: arm rank asc, then GREATEST(updated_at, last_retrieved_at) desc.
const candidates = [...rankBySlug.entries()]
.map(([s, rank]) => ({ slug: s, rank, row: rowBySlug.get(s) }))
.filter((c): c is { slug: string; rank: number; row: CardPageRow } => c.row !== undefined)
.sort((a, b) => a.rank - b.rank || lastTouchedMs(b.row) - lastTouchedMs(a.row));
if (candidates.length === 0) {
return { found: false, suggestions: await nearMissSuggestions(engine, sourceId, trimmed) };
}
const best = candidates[0];
const runnersUp: EntitySuggestion[] = candidates.slice(1, 1 + SUGGESTION_CAP).map(c => ({
slug: c.slug,
title: c.row.title ?? c.slug,
// A page that resolved through the precision arms exists by definition.
create_safety: 'exists',
}));
const card = await assembleCard(engine, sourceId, best.row, opts.remote);
return {
found: true,
card,
...(runnersUp.length ? { suggestions: runnersUp } : {}),
};
}
async function assembleCard(
engine: BrainEngine,
sourceId: string,
row: CardPageRow,
remote: boolean,
): Promise<EntityCard> {
const pageSlug = row.slug;
const visibility = remote ? (['world'] as ('private' | 'world')[]) : undefined;
// Parallel depth-1 reads — every arm individually fail-soft so a partial
// brain (no aliases, no timeline) still returns a card.
//
// [ship P1.2] Incoming edges + backlink_count are SOURCE-SAFE on BOTH sides.
// engine.getBacklinks(slug,{sourceId}) only scopes the TARGET page's source,
// so a foreign-source page linking to a same-named entity would leak its
// slug; engine.getBacklinkCounts has no source param at all. We instead run
// a both-sides-scoped query here (f.source_id = t.source_id = this source),
// mentions excluded (matching the backlink-count convention). Outgoing edges
// (getLinks) are the entity's OWN declared links — from-side scoped — so they
// stay as-is.
const [aka, outLinks, inEdges, backlinkCount, timeline, facts] = await Promise.all([
engine
.executeRaw<{ alias_norm: string }>(
`SELECT alias_norm FROM page_aliases WHERE source_id = $1 AND slug = $2 ORDER BY alias_norm`,
[sourceId, pageSlug],
)
.then(rs => rs.map(r => r.alias_norm))
.catch(() => [] as string[]),
engine.getLinks(pageSlug, { sourceId }).catch(() => []),
engine
.executeRaw<{ from_slug: string; link_type: string; context: string | null }>(
`SELECT f.slug AS from_slug, l.link_type, l.context
FROM links l
JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id
WHERE t.slug = $1 AND t.source_id = $2 AND f.source_id = $2
AND COALESCE(l.link_source, '') <> 'mentions'`,
[pageSlug, sourceId],
)
.catch(() => [] as Array<{ from_slug: string; link_type: string; context: string | null }>),
engine
.executeRaw<{ n: string | number }>(
`SELECT COUNT(*) AS n
FROM links l
JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id
WHERE t.slug = $1 AND t.source_id = $2 AND f.source_id = $2
AND COALESCE(l.link_source, '') <> 'mentions'`,
[pageSlug, sourceId],
)
.then(rs => Number(rs[0]?.n ?? 0))
.catch(() => 0),
engine.getTimeline(pageSlug, { limit: 5, sourceId }).catch(() => []),
engine
.listFactsByEntity(sourceId, pageSlug, {
activeOnly: true,
limit: FACT_FETCH_CAP,
...(visibility ? { visibility } : {}),
})
.catch(() => [] as FactRow[]),
]);
const edges: EntityCardEdge[] = [];
for (const l of outLinks) {
if (l.link_source === 'mentions') continue;
edges.push({ type: l.link_type, direction: 'out', slug: l.to_slug, context: l.context || null });
if (edges.length >= EDGE_CAP) break;
}
if (edges.length < EDGE_CAP) {
for (const l of inEdges) {
edges.push({ type: l.link_type, direction: 'in', slug: l.from_slug, context: l.context || null });
if (edges.length >= EDGE_CAP) break;
}
}
// Open threads (best-effort v1): active commitments first, then recent
// timeline entries inside the window, capped together.
const openThreads: EntityOpenThread[] = [];
for (const f of facts) {
if (f.kind !== 'commitment') continue;
openThreads.push({ kind: 'commitment', text: f.fact, date: f.valid_from?.toISOString() ?? null });
if (openThreads.length >= OPEN_THREADS_CAP) break;
}
if (openThreads.length < OPEN_THREADS_CAP) {
const cutoff = Date.now() - OPEN_THREAD_TIMELINE_WINDOW_DAYS * 24 * 60 * 60 * 1000;
for (const t of timeline) {
const ts = Date.parse(t.date);
if (!Number.isFinite(ts) || ts < cutoff) continue;
openThreads.push({ kind: 'recent_event', text: t.summary, date: t.date });
if (openThreads.length >= OPEN_THREADS_CAP) break;
}
}
return {
entity: { slug: pageSlug, title: row.title ?? pageSlug, type: row.type ?? null },
aka,
summary: safeSynopsis(row),
last_touched: {
updated_at: toIso(row.updated_at),
last_retrieved_at: toIso(row.last_retrieved_at),
last_timeline_date: timeline.length ? timeline[0].date : null,
},
open_threads: openThreads,
edges,
backlink_count: backlinkCount,
active_fact_count: facts.length,
};
}
/**
* Near-miss suggestions on a total miss (E5 delight): keyword search top-N
* with evidence-derived create_safety so a typo'd name becomes a next move
* instead of a dead end. Zero LLM; fail-soft to [].
*/
async function nearMissSuggestions(
engine: BrainEngine,
sourceId: string,
name: string,
): Promise<EntitySuggestion[]> {
try {
const raw = await engine.searchKeyword(name, { limit: SUGGESTION_CAP, sourceId });
const results = raw as SearchResult[];
stampEvidence(results);
return results.map(r => ({
slug: r.slug,
title: r.title ?? r.slug,
create_safety: r.create_safety ?? 'unknown',
}));
} catch {
return [];
}
}
function lastTouchedMs(row: CardPageRow): number {
const u = toMs(row.updated_at);
const l = toMs(row.last_retrieved_at);
return Math.max(u, l);
}
function toMs(v: Date | string | null): number {
if (v == null) return 0;
const ms = v instanceof Date ? v.getTime() : Date.parse(v);
return Number.isFinite(ms) ? ms : 0;
}
function toIso(v: Date | string | null): string | null {
const ms = toMs(v);
return ms > 0 ? new Date(ms).toISOString() : null;
}
src/core/verbs/usage-log.ts
+124
View File
@@ -0,0 +1,124 @@
/**
* MEMORY_VERBS v1 — per-verb usage sidecar (E4 observability).
*
* One JSONL line per verb call, written from the DISPATCH layer (so
* param-validation failures are counted too — c11), fire-and-forget.
* LOCAL ONLY: this file never leaves the machine and is never uploaded —
* it is observability, never source of truth. Stats tolerate loss.
*
* Concurrency: append-only, one line-buffered write() per event (<4KB ⇒
* atomic under POSIX O_APPEND; serve + jobs worker interleave safely at line
* granularity; best-effort on Windows, documented). Rotation at 10MB is
* lock-free best-effort — a concurrent double-rotate can drop lines, which
* is acceptable for stats.
*
* Path: ~/.gbrain/integrations/memory-verbs/usage.jsonl via gbrainPath, so
* GBRAIN_HOME is honored and brain_id (the resolved gbrain home) is a true
* per-brain disambiguator [c11/m1].
*/
import { appendFile, mkdir, rename, stat, readFile } from 'node:fs/promises';
import { dirname, join } from 'node:path';
import { gbrainPath } from '../config.ts';
const ROTATE_BYTES = 10 * 1024 * 1024;
export interface VerbUsageEvent {
ts: string;
verb: string;
surface: 'verbs' | 'full';
remote: boolean;
ok: boolean;
latency_ms: number;
brain_id: string;
source_id: string;
budget_dropped?: number;
entity_found?: boolean;
}
let _pathOverride: string | null = null;
/**
* Test-only seam: redirect the sidecar to a temp file without mutating
* process.env.GBRAIN_HOME (the test-isolation lint forbids global env
* mutation). Pass null to restore. @internal exported for tests.
*/
export function __setUsageLogPathForTests(path: string | null): void {
_pathOverride = path;
}
export function usageLogPath(): string {
return _pathOverride ?? gbrainPath('integrations', 'memory-verbs', 'usage.jsonl');
}
/** The resolved gbrain home — the per-brain disambiguator for multi-brain stats. */
export function brainId(): string {
return gbrainPath();
}
/**
* Fire-and-forget append. NEVER throws, NEVER blocks the verb call — callers
* do not await this (dispatch invokes it without await).
*/
export function logVerbUsage(event: Omit<VerbUsageEvent, 'ts' | 'brain_id'>): void {
void (async () => {
try {
const path = usageLogPath();
await mkdir(dirname(path), { recursive: true });
// Best-effort lock-free rotation (stats-only; dropped lines tolerated).
try {
const s = await stat(path);
if (s.size > ROTATE_BYTES) {
await rename(path, join(dirname(path), 'usage.jsonl.1'));
}
} catch {
/* no file yet, or a concurrent rotate won — either is fine */
}
const line =
JSON.stringify({ ts: new Date().toISOString(), brain_id: brainId(), ...event }) + '\n';
await appendFile(path, line, 'utf-8');
} catch {
/* observability never breaks the verb call */
}
})();
}
/**
* Read events for `gbrain protocol stats` + the doctor check. Tolerates
* malformed lines (torn writes on non-POSIX appends) by skipping them.
*/
export async function readVerbUsage(opts: { days?: number } = {}): Promise<VerbUsageEvent[]> {
let raw: string;
try {
raw = await readFile(usageLogPath(), 'utf-8');
} catch {
return [];
}
const cutoff = opts.days ? Date.now() - opts.days * 24 * 60 * 60 * 1000 : null;
const events: VerbUsageEvent[] = [];
for (const line of raw.split('\n')) {
if (!line.trim()) continue;
try {
const e = JSON.parse(line) as VerbUsageEvent;
if (!e || typeof e.verb !== 'string' || typeof e.ts !== 'string') continue;
if (cutoff !== null) {
const ms = Date.parse(e.ts);
if (!Number.isFinite(ms) || ms < cutoff) continue;
}
events.push(e);
} catch {
/* torn line — skip */
}
}
return events;
}
/** Earliest event timestamp — the TTHW numerator for `protocol stats` [D6C]. */
export async function earliestVerbUsageTs(): Promise<string | null> {
const events = await readVerbUsage();
let earliest: string | null = null;
for (const e of events) {
if (earliest === null || e.ts < earliest) earliest = e.ts;
}
return earliest;
}
src/mcp/dispatch.ts
+76 -2
View File
@@ -10,6 +10,10 @@ import type { BrainEngine } from '../core/engine.ts';
import { operations, OperationError, enforceBoundClientOpAllowList } from '../core/operations.ts';
import type { Operation, OperationContext, AuthInfo } from '../core/operations.ts';
import { loadConfig } from '../core/config.ts';
import { VERB_NAMES, MEMORY_VERBS_VERSION } from '../core/verbs.ts';
import { logVerbUsage } from '../core/verbs/usage-log.ts';
const VERB_NAME_SET: ReadonlySet<string> = new Set(VERB_NAMES);
export interface ToolResult {
content: { type: 'text'; text: string }[];
@@ -91,6 +95,20 @@ export interface DispatchOpts {
* was replaced by dispatchToolCall.
*/
auth?: AuthInfo;
/**
* MEMORY_VERBS v1 surface enforcement [c2]. When set, a tool name outside
* the set returns the unknown_tool envelope BEFORE resolution — fail-closed
* at the SHARED layer, so a hidden op stays uncallable on every transport
* even when only the tool LIST was filtered. Unset = full catalog
* (pre-existing behavior, all current callers).
*/
allowedOps?: ReadonlySet<string>;
/**
* Which surface this transport is serving — recorded on the verb usage
* sidecar so adoption stats can split quickstart installs from full
* surfaces. Defaults to 'full'.
*/
surface?: 'verbs' | 'full';
}
/**
@@ -273,6 +291,33 @@ export async function dispatchToolCall(
params: Record<string, unknown> | undefined,
opts: DispatchOpts = {},
): Promise<ToolResult> {
const startedMs = Date.now();
const isVerb = VERB_NAME_SET.has(name);
// [c11] dispatch-layer usage sidecar for the five verbs — counts validation
// failures too. Fire-and-forget; never awaited, never throws.
const logVerb = (ok: boolean, extra?: { budget_dropped?: number; entity_found?: boolean }) => {
if (!isVerb) return;
logVerbUsage({
verb: name,
surface: opts.surface ?? 'full',
remote: opts.remote ?? true,
ok,
latency_ms: Date.now() - startedMs,
source_id: opts.sourceId ?? 'default',
...(extra ?? {}),
});
};
// [c2] surface enforcement at the SHARED layer: a hidden op is uncallable
// on every transport, not just unlisted. Same envelope as unknown ops so
// the surface doesn't leak which names exist.
if (opts.allowedOps && !opts.allowedOps.has(name)) {
return {
content: [{ type: 'text', text: JSON.stringify({ error: 'unknown_tool', message: `Unknown tool: ${name}` }, null, 2) }],
isError: true,
};
}
const op = operations.find(o => o.name === name);
if (!op) {
// Always return JSON-shaped error content. v0.31 e2e tests
@@ -289,8 +334,19 @@ export async function dispatchToolCall(
const safeParams = params || {};
const validationError = validateParams(op, safeParams);
if (validationError) {
logVerb(false);
// [c7] verb validation errors speak the protocol envelope (suggestion +
// protocol_version); non-verb ops keep the pre-existing shape untouched.
const envelope = isVerb
? {
error: 'invalid_params',
message: validationError,
suggestion: 'Check the tool schema — required params and types are declared there.',
protocol_version: MEMORY_VERBS_VERSION,
}
: { error: 'invalid_params', message: validationError };
return {
content: [{ type: 'text', text: JSON.stringify({ error: 'invalid_params', message: validationError }, null, 2) }],
content: [{ type: 'text', text: JSON.stringify(envelope, null, 2) }],
isError: true,
};
}
@@ -319,6 +375,14 @@ export async function dispatchToolCall(
// a silent hole. See CLIENT_FENCED_WRITE_OPS in operations.ts.
enforceBoundClientOpAllowList(ctx.auth, op);
const result = await op.handler(ctx, safeParams);
// [E4] verb success metrics: budget drops + entity hit/miss when present.
{
const r = result as { dropped_count?: number; found?: boolean } | null;
logVerb(true, {
...(typeof r?.dropped_count === 'number' ? { budget_dropped: r.dropped_count } : {}),
...(name === 'entity' && typeof r?.found === 'boolean' ? { entity_found: r.found } : {}),
});
}
const out: ToolResult = { content: [{ type: 'text', text: JSON.stringify(result, null, 2) }] };
// v0.31 (eD3 + eE4): best-effort _meta.brain_hot_memory injection.
// The hook is wrapped in its own try/catch — any DB blip / cache miss /
@@ -335,6 +399,7 @@ export async function dispatchToolCall(
}
return out;
} catch (e: unknown) {
logVerb(false);
if (e instanceof OperationError) {
return { content: [{ type: 'text', text: JSON.stringify(e.toJSON(), null, 2) }], isError: true };
}
@@ -343,8 +408,17 @@ export async function dispatchToolCall(
// plain `Error: ${msg}` strings here, which broke any caller that
// tried JSON.parse(content).
const msg = e instanceof Error ? e.message : String(e);
// [c7] verbs speak the protocol envelope even for uncaught throws.
const envelope = isVerb
? {
error: 'internal',
message: msg,
suggestion: 'This is a server-side failure, not a caller mistake. Retry once; if it persists, run `gbrain doctor`.',
protocol_version: MEMORY_VERBS_VERSION,
}
: { error: 'internal_error', message: msg };
return {
content: [{ type: 'text', text: JSON.stringify({ error: 'internal_error', message: msg }, null, 2) }],
content: [{ type: 'text', text: JSON.stringify(envelope, null, 2) }],
isError: true,
};
}
src/mcp/http-transport.ts
+17 -1
View File
@@ -32,6 +32,7 @@ import { operations } from '../core/operations.ts';
import type { AuthInfo } from '../core/operations.ts';
import { VERSION } from '../version.ts';
import { dispatchToolCall } from './dispatch.ts';
import { filterOpsForSurface } from './surface.ts';
import { buildDefaultLimiters, type RateLimiter } from './rate-limit.ts';
import { sqlQueryForEngine } from '../core/sql-query.ts';
import { parseLegacyTokenScope, parseTakesHoldersAllowList, coerceLegacyPermissions } from '../core/legacy-token-scope.ts';
@@ -61,6 +62,12 @@ interface HttpTransportOptions {
engine: BrainEngine;
/** Override limiters (for tests). Defaults to env-driven buildDefaultLimiters. */
limiters?: { ip: RateLimiter; token: RateLimiter };
/**
* MEMORY_VERBS v1 [c1]: tool-surface mode for this transport (the SECOND
* HTTP path — the OAuth path in serve-http.ts carries its own). 'verbs' =
* exactly the five protocol verbs; 'full' (default) = everything.
*/
surface?: 'verbs' | 'full';
}
interface AuthResult {
@@ -156,7 +163,13 @@ export async function startHttpTransport(opts: HttpTransportOptions) {
const limiters = opts.limiters || buildDefaultLimiters();
const bodyCap = envInt('GBRAIN_HTTP_MAX_BODY_BYTES', DEFAULT_BODY_CAP);
const corsAllowlist = parseCorsAllowlist();
const tools = buildToolDefs(operations);
// MEMORY_VERBS v1 [c1]: surface filter applies to THIS transport too —
// the advertised list AND dispatch (allowedOps), fail-closed.
const surface = opts.surface ?? 'full';
const surfacedOps = filterOpsForSurface(operations, surface);
const surfaceAllowedOps: ReadonlySet<string> | undefined =
surface === 'full' ? undefined : new Set(surfacedOps.map(o => o.name));
const tools = buildToolDefs(surfacedOps);
/**
* v0.41.3 (T6): single consolidated CORS header builder. Pre-fix there were
@@ -411,6 +424,9 @@ export async function startHttpTransport(opts: HttpTransportOptions) {
// #1336: thread the token's federated_read grant so read ops scope
// to the operator-granted sources via sourceScopeOpts.
auth: auth.auth,
// MEMORY_VERBS v1 [c1/c2]: fail-closed surface enforcement here too.
...(surfaceAllowedOps ? { allowedOps: surfaceAllowedOps } : {}),
surface,
});
const status = result.isError ? 'error' : 'success';
logRequest(auth.tokenName!, `tools/call:${toolName}`, status, Date.now() - startedMs);
src/mcp/server.ts
+13 -2
View File
@@ -6,6 +6,7 @@ import { operations } from '../core/operations.ts';
import { VERSION } from '../version.ts';
import { buildToolDefs } from './tool-defs.ts';
import { dispatchToolCall, validateParams, buildOperationContext } from './dispatch.ts';
import { filterOpsForSurface, allowedOpNames, type McpSurface } from './surface.ts';
import { getBrainHotMemoryMeta } from '../core/facts/meta-hook.ts';
import { loadConfig } from '../core/config.ts';
import {
@@ -17,17 +18,24 @@ import {
import { resolveEntitiesToPointers, logDeliveredReflexPointers } from '../core/context/retrieval-reflex.ts';
import { assembleTurnContext } from '../core/context/turn-context.ts';
export async function startMcpServer(engine: BrainEngine) {
export async function startMcpServer(engine: BrainEngine, opts: { surface?: McpSurface } = {}) {
const server = new Server(
{ name: 'gbrain', version: VERSION },
{ capabilities: { tools: {} } },
);
// MEMORY_VERBS v1 surface mode: 'full' (default — every op, byte-identical
// to pre-surface behavior) or 'verbs' (exactly the 5 protocol verbs).
// Enforced BOTH on the advertised list and in dispatch (fail-closed [c2]).
const surface: McpSurface = opts.surface ?? 'full';
const surfacedOps = filterOpsForSurface(operations, surface);
const allowedOps = surface === 'full' ? undefined : allowedOpNames(operations, surface);
// Generate tool definitions from operations. Extracted to buildToolDefs so
// the subagent tool registry (v0.15+) can call the same mapper against a
// filtered OPERATIONS subset instead of duplicating this shape.
server.setRequestHandler(ListToolsRequestSchema, async () => ({
tools: buildToolDefs(operations),
tools: buildToolDefs(surfacedOps),
}));
// Dispatch tool calls via shared dispatch.ts (parity with HTTP transport).
@@ -81,6 +89,9 @@ export async function startMcpServer(engine: BrainEngine) {
// Code see the brain's relevant hot memory automatically alongside
// every tool-call response. Best-effort; absorbs errors.
metaHook: getBrainHotMemoryMeta,
// MEMORY_VERBS v1: fail-closed surface enforcement + usage attribution.
...(allowedOps ? { allowedOps } : {}),
surface,
});
});
src/mcp/surface.ts
+62
View File
@@ -0,0 +1,62 @@
/**
* MEMORY_VERBS v1 — MCP tool-surface modes (Cathedral 1).
*
* 'full' (default) — every operation, verbs included. Existing installs
* see no change; e2e tool-count assertions hold.
* 'verbs' — EXACTLY the five frozen protocol verbs (ops marked
* `verb: true`). The quickstart surface: agents see
* recall/remember/entity/synthesize/forget and nothing
* else.
*
* Enforcement is two-layer and fail-closed: ListTools advertises the filtered
* set, AND dispatchToolCall receives the same set as `allowedOps` so a hidden
* op stays uncallable even if a client guesses its name (tool-list filtering
* alone leaves dispatch resolving the global catalog — codex c2).
*
* Resolution: --surface flag > config `mcp_surface` > 'full'. Why default
* full: verbs is for agents and quickstarts; full preserves existing advanced
* tooling.
*/
import type { Operation } from '../core/operations.ts';
import type { GBrainConfig } from '../core/config.ts';
export type McpSurface = 'verbs' | 'full';
export function isMcpSurface(v: unknown): v is McpSurface {
return v === 'verbs' || v === 'full';
}
/** Strict flag parser — unknown values reject loudly (parseStdioIdleTimeout pattern). */
export function parseSurfaceFlag(args: string[]): McpSurface | null {
const idx = args.indexOf('--surface');
if (idx < 0) return null;
const raw = args[idx + 1];
if (raw === undefined || raw.startsWith('--')) {
throw new Error(`--surface requires a value: verbs | full`);
}
if (!isMcpSurface(raw)) {
throw new Error(`Unknown --surface "${raw}". Use: verbs (the 5 memory verbs) | full (all operations, default)`);
}
return raw;
}
/** Flag > config `mcp_surface` > 'full'. */
export function resolveSurface(
flag: McpSurface | null,
config: Pick<GBrainConfig, 'mcp_surface'> | null | undefined,
): McpSurface {
if (flag) return flag;
if (config && isMcpSurface(config.mcp_surface)) return config.mcp_surface;
return 'full';
}
export function filterOpsForSurface(ops: Operation[], surface: McpSurface): Operation[] {
if (surface === 'full') return ops;
return ops.filter(op => op.verb === true);
}
/** The fail-closed allow-set handed to dispatchToolCall. */
export function allowedOpNames(ops: Operation[], surface: McpSurface): ReadonlySet<string> {
return new Set(filterOpsForSurface(ops, surface).map(o => o.name));
}
src/mcp/tool-defs.ts
+12
View File
@@ -8,6 +8,17 @@ export interface McpToolDef {
properties: Record<string, unknown>;
required: string[];
};
/**
* MCP ToolAnnotations (SDK 1.29+), emitted ONLY when the op defines them —
* existing tools keep byte-identical definitions (the byte-equality
* regression test depends on absent keys staying absent).
*/
annotations?: {
title?: string;
readOnlyHint?: boolean;
destructiveHint?: boolean;
idempotentHint?: boolean;
};
}
/**
@@ -50,5 +61,6 @@ export function buildToolDefs(ops: Operation[]): McpToolDef[] {
.filter(([, v]) => v.required)
.map(([k]) => k),
},
...(op.annotations ? { annotations: op.annotations } : {}),
}));
}
test/apply-migrations.test.ts
+2 -2
View File
@@ -108,7 +108,7 @@ describe('buildPlan — diff against completed + installed VERSION', () => {
// autopilot cooperative, v0.16.0 = subagent runtime, v0.18.0 = multi-
// source brains, v0.18.1 = RLS hardening, v0.21.0 = Cathedral II
// (renumbered from v0.20.0 after master shipped v0.20.x in parallel).
expect(plan.skippedFuture.map(m => m.version)).toEqual(['0.12.0', '0.12.2', '0.13.0', '0.13.1', '0.14.0', '0.16.0', '0.18.0', '0.18.1', '0.21.0', '0.22.4', '0.28.0', '0.29.1', '0.31.0', '0.32.2']);
expect(plan.skippedFuture.map(m => m.version)).toEqual(['0.12.0', '0.12.2', '0.13.0', '0.13.1', '0.14.0', '0.16.0', '0.18.0', '0.18.1', '0.21.0', '0.22.4', '0.28.0', '0.29.1', '0.31.0', '0.32.2', '0.43.0']);
});
test('already applied → v0.11.0 lands in `applied` bucket, not pending', () => {
@@ -148,7 +148,7 @@ describe('buildPlan — diff against completed + installed VERSION', () => {
// v0.22.4, v0.28.0, v0.29.1, v0.31.0 were added later; installed=0.12.0
// means they belong in skippedFuture, not pending. v0.11.0 and v0.12.0
// stay pending despite being ≤ installed — that is the H9 invariant.
expect(plan.skippedFuture.map(m => m.version)).toEqual(['0.12.2', '0.13.0', '0.13.1', '0.14.0', '0.16.0', '0.18.0', '0.18.1', '0.21.0', '0.22.4', '0.28.0', '0.29.1', '0.31.0', '0.32.2']);
expect(plan.skippedFuture.map(m => m.version)).toEqual(['0.12.2', '0.13.0', '0.13.1', '0.14.0', '0.16.0', '0.18.0', '0.18.1', '0.21.0', '0.22.4', '0.28.0', '0.29.1', '0.31.0', '0.32.2', '0.43.0']);
});
test('--migration filter narrows to one version', () => {
test/bootstrap.test.ts
+63
View File
@@ -253,4 +253,67 @@ describe('PGLiteEngine#applyForwardReferenceBootstrap', () => {
await engine.disconnect();
}
}, 30000);
test('wedged-brain recovery: a brain that already FAILED the v0.42.56 upgrade converges on retry', async () => {
// The loudest #2626-class cohort: operators who upgraded, wedged, and are
// retrying with a fixed binary. Simulates the failed attempt (the blob's
// CREATE INDEX crashing on the missing column) and asserts the retry
// converges to the FULL final shape (column + FK + both partial indexes)
// with no residue — the failed attempt must not advance the version ledger.
const engine = new PGLiteEngine();
await engine.connect({});
try {
await engine.initSchema();
const db = (engine as any).db;
// Rewind to the pre-v121 shape: schema AND the version counter.
await db.exec(`
DROP INDEX IF EXISTS idx_timeline_event_page;
DROP INDEX IF EXISTS idx_timeline_event_dedup;
ALTER TABLE timeline_entries DROP CONSTRAINT IF EXISTS timeline_entries_event_page_id_fkey;
ALTER TABLE timeline_entries DROP COLUMN IF EXISTS event_page_id;
`);
await engine.setConfig('version', '120');
// The failed old-binary attempt: without the bootstrap probe, the blob's
// CREATE INDEX was the first statement to touch the missing column.
let wedgeError: Error | null = null;
try {
await db.exec(
`CREATE INDEX IF NOT EXISTS idx_timeline_event_page
ON timeline_entries(event_page_id) WHERE event_page_id IS NOT NULL`,
);
} catch (e) {
wedgeError = e as Error;
}
expect(wedgeError?.message ?? '').toContain('event_page_id');
// The failed attempt must not have advanced the ledger.
expect(parseInt((await engine.getConfig('version')) || '1', 10)).toBe(120);
// Retry with the fixed binary: full initSchema converges to LATEST with
// the complete final shape.
await engine.initSchema();
expect(await engine.getConfig('version')).toBe(String(LATEST_VERSION));
const { rows: col } = await db.query(`
SELECT column_name FROM information_schema.columns
WHERE table_name = 'timeline_entries' AND column_name = 'event_page_id'
`);
expect(col).toHaveLength(1);
const { rows: fk } = await db.query(`
SELECT conname FROM pg_constraint
WHERE conname = 'timeline_entries_event_page_id_fkey'
`);
expect(fk).toHaveLength(1);
const { rows: idx } = await db.query(`
SELECT indexname FROM pg_indexes
WHERE tablename = 'timeline_entries'
AND indexname IN ('idx_timeline_event_page', 'idx_timeline_event_dedup')
`);
expect(idx).toHaveLength(2);
} finally {
await engine.disconnect();
}
}, 30000);
});
test/cli-concept-nudge.test.ts
+68
View File
@@ -0,0 +1,68 @@
// #2416: CLI wiring for the concept-shaped search nudge. Same in-process
// pattern as cli-query-image.test.ts (import the helper from src/cli.ts —
// safe, the import.meta.main seam guards top-level execution). The helper is
// called from BOTH result paths (local engine + thin-client routed), so
// pinning its behavior here covers the shared wiring; the per-path call
// sites are two-liners.
import { describe, expect, test, beforeEach, afterEach } from 'bun:test';
import { maybePrintConceptNudge } from '../src/cli.ts';
import {
DEFAULT_CLI_OPTIONS,
setCliOptions,
_resetCliOptionsForTest,
} from '../src/core/cli-options.ts';
let stderrChunks: string[];
let originalWrite: typeof process.stderr.write;
beforeEach(() => {
stderrChunks = [];
originalWrite = process.stderr.write.bind(process.stderr);
process.stderr.write = ((chunk: string | Uint8Array) => {
stderrChunks.push(String(chunk));
return true;
}) as typeof process.stderr.write;
});
afterEach(() => {
process.stderr.write = originalWrite;
_resetCliOptionsForTest();
});
const CONCEPT_QUERY = 'all the companies that do offshore wind';
describe('#2416 — maybePrintConceptNudge wiring', () => {
test('concept-shaped search emits the one-line query nudge to stderr', () => {
setCliOptions({ ...DEFAULT_CLI_OPTIONS });
maybePrintConceptNudge('search', { query: CONCEPT_QUERY });
expect(stderrChunks.length).toBe(1);
expect(stderrChunks[0]).toContain('gbrain query');
expect(stderrChunks[0]).toContain('not proof of completeness');
expect(stderrChunks[0].endsWith('\n')).toBe(true);
});
test('exact-token search stays silent', () => {
setCliOptions({ ...DEFAULT_CLI_OPTIONS });
maybePrintConceptNudge('search', { query: 'stripe' });
expect(stderrChunks.length).toBe(0);
});
test('--quiet silences the nudge even for concept-shaped queries', () => {
setCliOptions({ ...DEFAULT_CLI_OPTIONS, quiet: true });
maybePrintConceptNudge('search', { query: CONCEPT_QUERY });
expect(stderrChunks.length).toBe(0);
});
test('the query op never nudges (only search does)', () => {
setCliOptions({ ...DEFAULT_CLI_OPTIONS });
maybePrintConceptNudge('query', { query: CONCEPT_QUERY });
expect(stderrChunks.length).toBe(0);
});
test('missing query param is a silent no-op, not a crash', () => {
setCliOptions({ ...DEFAULT_CLI_OPTIONS });
maybePrintConceptNudge('search', {});
expect(stderrChunks.length).toBe(0);
});
});
test/cli-flag-validation.test.ts
+273
View File
@@ -0,0 +1,273 @@
/**
* #2185 — strict unknown-flag validation.
*
* The repro that filed the issue: `gbrain init --migrate-only --dry-run`
* applied REAL migrations — no handler consults --dry-run, and the ad-hoc
* `args.includes()` flag style silently ignores anything it doesn't look for.
* The pre-dispatch validator in src/cli.ts fails loud instead.
*
* Four guard classes:
* 1. sweep — every CLI_ONLY command (minus documented exemptions) and every
* op command rejects a nonsense flag via the pure validator.
* 2. acceptance — real flags and passthrough forms stay accepted.
* 3. drift — every CLI_ONLY member has a registry entry.
* 4. freshness — the committed generated registry matches a fresh
* generator run (same doctrine as the llms-bundle freshness test).
* Plus subprocess smokes for the end-to-end error surface.
*/
import { describe, test, expect } from 'bun:test';
import { spawnSync } from 'child_process';
import {
validateCommandFlags,
findUnknownFlag,
findUnknownOpFlag,
CLI_ONLY,
} from '../src/cli.ts';
import { CLI_FLAG_REGISTRY } from '../src/core/cli-flag-registry.generated.ts';
import { operations, operationsByName } from '../src/core/operations.ts';
import { buildFlagRegistry } from '../scripts/generate-flag-registry.ts';
const BOGUS = '--definitely-not-a-real-flag-xyz';
const EXEMPT = new Set(['call', 'config']); // jobs is exempt only for `submit`
describe('#2185 sweep — every command rejects a nonsense flag', () => {
test('every CLI_ONLY command rejects the bogus flag (validator lane)', () => {
const accepted: string[] = [];
for (const command of CLI_ONLY) {
if (EXEMPT.has(command)) continue;
const verdict = validateCommandFlags(command, [BOGUS]);
if (verdict !== BOGUS) accepted.push(command);
}
expect(accepted).toEqual([]);
});
test('every op command rejects the bogus flag (op lane)', () => {
const accepted: string[] = [];
for (const op of operations) {
if (!op.cliHints) continue;
const verdict = findUnknownOpFlag(op, [BOGUS]);
if (verdict !== BOGUS) accepted.push(op.name);
}
expect(accepted).toEqual([]);
});
test('the literal #2185 repro is rejected: init --migrate-only --dry-run', () => {
expect(validateCommandFlags('init', ['--migrate-only', '--dry-run'])).toBe('--dry-run');
// And --migrate-only alone stays legal.
expect(validateCommandFlags('init', ['--migrate-only'])).toBeNull();
});
});
describe('#2185 acceptance — real usage stays legal', () => {
test('op flags from the contract are accepted, including values starting with --', () => {
const search = operationsByName.search;
expect(findUnknownOpFlag(search, ['needle', '--limit', '5'])).toBeNull();
// A VALUE that begins with -- is consumed as the value, not validated.
expect(findUnknownOpFlag(search, ['--query', '--weird-looking-value'])).toBeNull();
// Inline = form.
expect(findUnknownOpFlag(search, ['needle', '--limit=5'])).toBeNull();
// CLI-local formatter flags.
expect(findUnknownOpFlag(search, ['needle', '--json', '--explain'])).toBeNull();
});
test('--no-<flag> negation of a known boolean op param is legal', () => {
const withBool = operations.find(o =>
o.cliHints && Object.values(o.params).some(p => p.type === 'boolean'));
expect(withBool).toBeDefined();
const boolKey = Object.entries(withBool!.params).find(([, p]) => p.type === 'boolean')![0];
const flag = `--no-${boolKey.replace(/_/g, '-')}`;
expect(findUnknownOpFlag(withBool!, [flag])).toBeNull();
});
test('everything after a literal -- is passthrough, never validated', () => {
expect(findUnknownFlag(['--', BOGUS], new Set(['--help']))).toBeNull();
expect(validateCommandFlags('agent', ['run', '--', BOGUS])).toBeNull();
expect(findUnknownOpFlag(operationsByName.search, ['--', BOGUS])).toBeNull();
});
test('exempt commands accept arbitrary flags by contract', () => {
expect(validateCommandFlags('call', ['some_op', BOGUS])).toBeNull();
expect(validateCommandFlags('config', ['set', 'k', BOGUS])).toBeNull();
expect(validateCommandFlags('jobs', ['submit', 'shell', BOGUS])).toBeNull();
// ...but non-submit jobs subcommands are validated.
expect(validateCommandFlags('jobs', ['list', BOGUS])).toBe(BOGUS);
});
test('registry-listed CLI_ONLY flags are accepted', () => {
expect(validateCommandFlags('serve', ['--http', '--port', '4444'])).toBeNull();
expect(validateCommandFlags('serve', ['--print-admin-token'])).toBeNull();
expect(validateCommandFlags('embed', ['--stale', '--pace'])).toBeNull();
expect(validateCommandFlags('sync', ['--full'])).toBeNull();
});
// Pre-landing review regression: the validator rejected the CLI-local
// flags makeContext consumes OUTSIDE the op contract — `gbrain search
// "x" --source y` and `--dry-run` invocations exited 1 as unknown flags.
test('op commands accept --source/--dry-run (makeContext CLI-locals, not wire params)', () => {
const search = operationsByName.search;
expect(findUnknownOpFlag(search, ['needle', '--source', 'yc-media'])).toBeNull();
expect(findUnknownOpFlag(search, ['needle', '--source=yc-media'])).toBeNull();
// --source's VALUE is consumed, never validated as a flag itself.
expect(findUnknownOpFlag(search, ['--source', '--weird-value', 'needle'])).toBeNull();
expect(findUnknownOpFlag(search, ['needle', '--dry-run'])).toBeNull();
// Still strict right next to them.
expect(findUnknownOpFlag(search, ['needle', '--source', 'y', BOGUS])).toBe(BOGUS);
});
test('--json=<v> is coherent between validator and parser (no positional corruption)', async () => {
const { parseOpArgs } = await import('../src/cli.ts');
const search = operationsByName.search;
// Validator accepts any --json form.
expect(findUnknownOpFlag(search, ['--json=true', 'needle'])).toBeNull();
// Parser sets the boolean and PRESERVES the positional (pre-fix the
// junk-key path consumed 'needle' as the value of key 'json=true').
const p = parseOpArgs(search, ['--json=true', 'needle']);
expect(p.json).toBe(true);
expect(p.query).toBe('needle');
expect((p as Record<string, unknown>)['json=true']).toBeUndefined();
expect(parseOpArgs(search, ['needle', '--json=false']).json).toBe(false);
// = forms of bare-only CLI-locals reject loud instead of silently eating tokens.
expect(findUnknownOpFlag(search, ['--explain=verbose'])).toBe('--explain');
});
});
describe('#2185 parseOpArgs inline = form (regression rule: changed token consumption)', () => {
test('string and number params via =, positional untouched', async () => {
const { parseOpArgs } = await import('../src/cli.ts');
const search = operationsByName.search;
const p = parseOpArgs(search, ['needle', '--limit=5']);
expect(p.query).toBe('needle');
expect(p.limit).toBe(5);
});
test('boolean =false negates; =0 keeps the raw!==false rule (pinned semantics)', async () => {
const { parseOpArgs } = await import('../src/cli.ts');
const withBool = operations.find(o =>
o.cliHints && Object.values(o.params).some(pp => pp.type === 'boolean'))!;
const key = Object.entries(withBool.params).find(([, pp]) => pp.type === 'boolean')![0];
const flag = `--${key.replace(/_/g, '-')}`;
expect(parseOpArgs(withBool, [`${flag}=false`])[key]).toBe(false);
expect(parseOpArgs(withBool, [`${flag}=0`])[key]).toBe(true);
});
test('undeclared =-form key keeps the historical junk-fallthrough (validator rejects it first)', async () => {
const { parseOpArgs } = await import('../src/cli.ts');
const search = operationsByName.search;
// The validator is the strict gate; the parser's legacy behavior for
// undeclared keys is unchanged — pinned so a refactor can't silently
// change what unvalidated callers (tests, internal) see.
expect(findUnknownOpFlag(search, ['--not-a-param=x', 'needle'])).toBe('--not-a-param');
const p = parseOpArgs(search, ['--not-a-param=x', 'needle']);
expect(p.query).toBeUndefined(); // consumed by the junk key — validator prevents reaching here
});
});
describe('#2185 red-team regressions', () => {
test('dual-lane commands (op AND CLI_ONLY) validate against the DISPATCHED lane', () => {
// think/salience/anomalies are both ops and CLI_ONLY members; dispatch
// runs handleCliOnly, whose handlers parse flags the op contract never
// declares. Pre-fix the validator checked the op lane first and rejected
// documented invocations.
expect(validateCommandFlags('salience', ['--kind', 'entity'])).toBeNull();
expect(validateCommandFlags('think', ['what changed?', '--with-calibration'])).toBeNull();
expect(validateCommandFlags('salience', [BOGUS])).toBe(BOGUS);
});
test('--dry-run is a real CLI-local boolean on op commands (trailing position sets it)', async () => {
const { parseOpArgs } = await import('../src/cli.ts');
// An op WITHOUT a declared dry_run param — pre-fix, trailing --dry-run
// set NOTHING (ctx.dryRun stayed false → the REAL destructive action
// ran despite the rehearsal request), and leading --dry-run consumed
// the next token as its value.
const search = operationsByName.search;
expect(parseOpArgs(search, ['needle', '--dry-run']).dry_run).toBe(true);
const leading = parseOpArgs(search, ['--dry-run', 'needle']);
expect(leading.dry_run).toBe(true);
expect(leading.query).toBe('needle');
expect(parseOpArgs(search, ['needle', '--dry-run=false']).dry_run).toBe(false);
});
test('uppercase flag typos reject loudly in both lanes (handlers are case-sensitive)', () => {
expect(findUnknownFlag(['--MIGRATE-ONLY'], new Set(['--migrate-only']))).toBe('--MIGRATE-ONLY');
expect(findUnknownOpFlag(operationsByName.search, ['--LIMIT', '5'])).toBe('--LIMIT');
});
test('safety flags need consumption evidence, not prose bleed (codex P1-A)', () => {
// upgrade.ts prints a help HINT naming another command's --dry-run; that
// literal sits at depth 0 for post-upgrade. Pre-fix the generator
// allowlisted it, recreating the exact #2185 repro the wave exists to
// kill: `post-upgrade --dry-run` accepted, ignored, migrations run for
// real. The generator now requires a tight-quoted standalone literal
// (an args read like has('--dry-run')) before granting a safety flag.
expect(CLI_FLAG_REGISTRY['post-upgrade']).not.toContain('--dry-run');
// backfill genuinely consumes it (has('--dry-run') in backfill.ts) —
// the gate must not strip real consumers.
expect(CLI_FLAG_REGISTRY['backfill']).toContain('--dry-run');
});
});
describe('#2185 drift + freshness guards', () => {
test('every CLI_ONLY member has a registry entry (drift guard)', () => {
const missing = [...CLI_ONLY].filter(c => !CLI_FLAG_REGISTRY[c]);
expect(missing).toEqual([]);
});
test('committed registry matches a fresh generator run (freshness guard)', () => {
const fresh = buildFlagRegistry();
const freshKeys = Object.keys(fresh).sort();
const committedKeys = Object.keys(CLI_FLAG_REGISTRY).sort();
expect(committedKeys).toEqual(freshKeys);
const stale = freshKeys.filter(
key => JSON.stringify([...CLI_FLAG_REGISTRY[key]]) !== JSON.stringify(fresh[key]),
);
// Any listed command means: run `bun run build:flag-registry` and commit.
expect(stale).toEqual([]);
});
});
describe('#2185 subprocess smokes — end-to-end error surface', () => {
const run = (args: string[]) =>
spawnSync('bun', ['src/cli.ts', ...args], {
encoding: 'utf-8',
timeout: 30_000,
env: { ...process.env, GBRAIN_SKIP_STARTUP_HOOKS: '1' },
});
test('init --migrate-only --dry-run fails loud BEFORE any engine work', () => {
const r = run(['init', '--migrate-only', '--dry-run']);
expect(r.status).toBe(1);
expect(r.stderr).toContain("unknown flag --dry-run for 'gbrain init'");
// Pre-engine: no migration output may appear.
expect(r.stderr).not.toContain('migration');
});
test('typo on an op command fails loud with the command named', () => {
const r = run(['search', 'needle', '--jsno']);
expect(r.status).toBe(1);
expect(r.stderr).toContain("unknown flag --jsno for 'gbrain search'");
});
test('--help still short-circuits before validation', () => {
const r = run(['init', '--help']);
expect(r.status).toBe(0);
expect(r.stderr).not.toContain('unknown flag');
});
test('global flags are accepted on every command (stripped pre-dispatch)', () => {
// --quiet is a parseGlobalFlags global: it never reaches the validator.
// The bogus flag proves validation still ran on what remained.
const r = run(['init', '--migrate-only', '--quiet', '--definitely-bogus-xyz']);
expect(r.status).toBe(1);
expect(r.stderr).toContain("unknown flag --definitely-bogus-xyz for 'gbrain init'");
expect(r.stderr).not.toContain('--quiet');
});
test('op command accepts --source end-to-end (the makeContext CLI-local regression)', () => {
// Fast path: --help short-circuits AFTER global parse, so a bogus flag
// alongside --source proves ordering: --source accepted, bogus rejected.
const r = run(['search', 'needle', '--source', 'nope-source', '--definitely-bogus-xyz']);
expect(r.status).toBe(1);
expect(r.stderr).toContain("unknown flag --definitely-bogus-xyz for 'gbrain search'");
expect(r.stderr).not.toContain('unknown flag --source');
});
});
test/cli-format-search-json.test.ts
+28
View File
@@ -0,0 +1,28 @@
import { describe, expect, test } from 'bun:test';
import { formatResult } from '../src/cli.ts';
describe('formatResult - search/query --json', () => {
test('search --json renders the raw result array as parseable JSON', () => {
const out = formatResult('search', [
{
slug: 'docs/example',
score: 0.42,
chunk_text: 'Example result text',
},
], { json: true });
expect(JSON.parse(out)).toEqual([
{
slug: 'docs/example',
score: 0.42,
chunk_text: 'Example result text',
},
]);
});
test('query --json keeps empty results machine-readable', () => {
const out = formatResult('query', [], { json: true });
expect(JSON.parse(out)).toEqual([]);
});
});
test/cli.test.ts
+85
View File
@@ -49,6 +49,91 @@ describe('CLI structure', () => {
test('has formatResult function for CLI output', () => {
expect(cliSource).toContain('function formatResult');
});
// #2035-class dispatch-gap guard: every `case '...'` label inside
// handleCliOnly's top-level dispatch must be a member of CLI_ONLY, else the
// command is registered but unreachable — 'calibration' shipped exactly this
// way. Structural, self-updating: a new case without a CLI_ONLY entry fails
// here at PR time.
test('every handleCliOnly top-level case label is reachable via CLI_ONLY', () => {
const onlyMatch = cliSource.match(/const CLI_ONLY = new Set(?:<string>)?\(\[([\s\S]*?)\]\)/);
expect(onlyMatch).not.toBeNull();
// Strip line comments before member extraction — the set literal carries
// commentary whose quoted words must not count as members.
const onlyBody = onlyMatch![1].replace(/\/\/[^\n]*/g, '');
const members = new Set([...onlyBody.matchAll(/'([^']+)'/g)].map(m => m[1]));
const fnStart = cliSource.indexOf('async function handleCliOnly');
expect(fnStart).toBeGreaterThan(0);
const fnSrc = cliSource.slice(fnStart);
// Top-level dispatch labels sit at a fixed indent (6 spaces); nested
// sub-switches are indented deeper and stay out of this scan.
const caseLabels = [...fnSrc.matchAll(/^ case '([a-z0-9-]+)':/gm)].map(m => m[1]);
expect(caseLabels.length).toBeGreaterThan(20);
// Reachable outside CLI_ONLY, each with a documented route:
// - 'search': pre-dispatch subcommand gate (modes|stats|tune) in main();
// the bare command must keep routing to the `search` op for queries.
// - 'whoknows': currently routes via the find_experts op alias; its
// handleCliOnly case is dead (adding it to CLI_ONLY would trip the
// alias-collision guard and silently change output). Tracked follow-up
// alongside PR #2509 (whoknows --explain).
const REACHABLE_VIA_OTHER_ROUTE = new Set(['search', 'whoknows']);
const missing = caseLabels.filter(
label => !members.has(label) && !REACHABLE_VIA_OTHER_ROUTE.has(label),
);
expect(missing).toEqual([]);
// The search gate itself must exist — losing it re-deadens the dashboards.
// (master's gate is a superset: modes|stats|tune|diagnose.)
expect(cliSource).toMatch(/\['modes', 'stats', 'tune'(?:, 'diagnose')?\]\.includes\(subArgs\[0\] \?\? ''\)/);
});
});
// #2450 — the local-engine output normalizer used a bare JSON.stringify with
// no replacer. A bigint anywhere in an op's return value (e.g. a BIGSERIAL
// primary key read back by the Postgres engine) made JSON.stringify THROW
// "Do not know how to serialize a BigInt", crashing the command before any
// renderer ran. normalizeLocalResult stringifies via bigintToStringReplacer
// (bigint → string, postgres.js wire shape).
describe('BigInt-safe output normalization (#2450)', () => {
test('bare JSON.stringify throws on a bigint (the pre-fix crash)', () => {
expect(() => JSON.stringify({ id: 9999999999999999999n })).toThrow(
/serialize BigInt|serialize a BigInt/i,
);
});
test('normalizeLocalResult serializes bigint → string without throwing', async () => {
const { normalizeLocalResult } = await import('../src/cli.ts');
const out = normalizeLocalResult({
id: 42n,
nested: { count: 7n },
arr: [1n, 2n],
str: 'unchanged',
num: 3,
}) as Record<string, unknown>;
expect(out.id).toBe('42');
expect((out.nested as Record<string, unknown>).count).toBe('7');
expect(out.arr).toEqual(['1', '2']);
expect(out.str).toBe('unchanged');
expect(out.num).toBe(3);
});
test('bigint past Number.MAX_SAFE_INTEGER keeps full precision as a string', async () => {
const { normalizeLocalResult } = await import('../src/cli.ts');
const big = 9007199254740993n; // MAX_SAFE_INTEGER + 2
const out = normalizeLocalResult({ id: big }) as Record<string, unknown>;
expect(out.id).toBe('9007199254740993');
});
test("formatResult's default renderer is bigint-safe", async () => {
const { formatResult } = await import('../src/cli.ts');
expect(() => formatResult('__no_such_op__', { id: 5n })).not.toThrow();
expect(formatResult('__no_such_op__', { id: 5n })).toContain('"5"');
});
test('cli.ts no longer uses a replacer-less stringify on the normalize path', () => {
expect(cliSource).toContain('normalizeLocalResult(rawResult)');
expect(cliSource).not.toContain('JSON.parse(JSON.stringify(rawResult))');
});
});
describe('CLI version', () => {
test/cycle/extract-atoms-synthesize-concepts.test.ts
+88
View File
@@ -17,6 +17,7 @@ import { runPhaseExtractAtoms, parseAtomsResponse } from '../../src/core/cycle/e
import { runPhaseSynthesizeConcepts } from '../../src/core/cycle/synthesize-concepts.ts';
import { resetPgliteState } from '../helpers/reset-pglite.ts';
import type { ChatResult, ChatOpts } from '../../src/core/ai/gateway.ts';
import { canonicalLookup } from '../../src/core/model-pricing.ts';
let engine: PGLiteEngine;
@@ -371,6 +372,93 @@ describe('v0.41 T6: runPhaseSynthesizeConcepts via stubbed chat', () => {
});
});
// Canonical pricing: synthesize_concepts' estimated_spend_usd must derive
// from the model that actually answered (ChatResult.model) through
// canonicalLookup — not hardcoded Sonnet rates. A wrong per-model rate both
// trips the $1.50 budget gate early (deterministic-template fallback for
// work that had budget left) and persists an inflated cost into
// receipts/rollups. Canonical-miss models keep Sonnet-tier pricing (the
// same conservative fallback as skillopt/preflight's lookupPrice).
describe('synthesize_concepts: cost estimate uses canonical per-model pricing', () => {
// 6 atoms on one concept → single T2 group → exactly one LLM call.
const t2Atoms = Array.from({ length: 6 }, (_, i) => ({
slug: `priced-${i}`,
title: `Priced ${i}`,
body: `Priced body ${i}.`,
concept_refs: ['priced-concept'],
}));
// 1M input + 1M output tokens → estimated_spend_usd equals
// (input_rate + output_rate) in dollars, read straight off the table.
function pricedChat(model: string): (o: ChatOpts) => Promise<ChatResult> {
return async (_o: ChatOpts) => ({
text: 'Priced narrative.',
blocks: [{ type: 'text', text: 'Priced narrative.' }],
stopReason: 'end',
usage: {
input_tokens: 1_000_000,
output_tokens: 1_000_000,
cache_read_tokens: 0,
cache_creation_tokens: 0,
},
model,
providerId: model.split(':')[0] ?? 'anthropic',
});
}
// Expected values derive from the canonical table (never hand-copied
// dollar literals — the same invariant the fix enforces), so these tests
// survive future price refreshes in model-pricing.ts.
const sonnetRate = canonicalLookup('anthropic:claude-sonnet-4-6')!;
const gpt52Rate = canonicalLookup('openai:gpt-5.2')!;
test('non-Sonnet canonical model is priced at its own rates (openai:gpt-5.2)', async () => {
const result = await runPhaseSynthesizeConcepts(engine, {
_atoms: t2Atoms,
_chat: pricedChat('openai:gpt-5.2') as typeof import('../../src/core/ai/gateway.ts').chat,
dryRun: true,
});
// gpt-5.2's own canonical rates — NOT Sonnet's (the pre-fix hardcode).
// Guard the guard: the two rate cards must actually differ, or this
// test can't discriminate.
expect(gpt52Rate.input + gpt52Rate.output).not.toBeCloseTo(
sonnetRate.input + sonnetRate.output,
6,
);
expect(result.details?.estimated_spend_usd).toBeCloseTo(
gpt52Rate.input + gpt52Rate.output,
6,
);
});
test('canonical-miss model falls back to Sonnet-tier pricing', async () => {
const result = await runPhaseSynthesizeConcepts(engine, {
_atoms: t2Atoms,
_chat: pricedChat('acme:unpriced-model-x') as typeof import('../../src/core/ai/gateway.ts').chat,
dryRun: true,
});
// Not in CANONICAL_PRICING → conservative Sonnet-tier fallback.
expect(result.details?.estimated_spend_usd).toBeCloseTo(
sonnetRate.input + sonnetRate.output,
6,
);
});
test('control: Sonnet model estimate is unchanged by canonical routing', async () => {
const result = await runPhaseSynthesizeConcepts(engine, {
_atoms: t2Atoms,
_chat: pricedChat('anthropic:claude-sonnet-4-6') as typeof import('../../src/core/ai/gateway.ts').chat,
dryRun: true,
});
// Same Sonnet-tier rates the pre-canonical hardcode used ($3/$15 at
// the time of writing) → identical estimate before and after the fix.
expect(result.details?.estimated_spend_usd).toBeCloseTo(
sonnetRate.input + sonnetRate.output,
6,
);
});
});
// #2123 — extract_atoms must stamp `concepts` so synthesize_concepts has
// material. The pre-fix pipeline was broken end-to-end: the extractor
// never wrote the field, and every synthesize_concepts cycle skipped with
test/doctor.test.ts
+52
View File
@@ -222,6 +222,58 @@ describe('doctor command', () => {
}
});
test('jsonb_integrity: flags double-encoded subagent payloads, ignores legitimate string scalars, skips absent tables', async () => {
const { PGLiteEngine } = await import('../src/core/pglite-engine.ts');
const { jsonbIntegrityCheck } = await import('../src/commands/doctor.ts');
const engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
try {
// Seed a minion job to satisfy the FK, then two subagent rows:
// one DOUBLE-ENCODED (string scalar whose content is a JSON array —
// the pre-#2375 damage class) and one LEGITIMATE string scalar
// (persistToolExec binds pre-serialized string payloads as-is).
await engine.executeRaw(
`INSERT INTO minion_jobs (id, name, data, status) VALUES (990001, 'doctor-jsonb-test', '{}'::jsonb, 'completed')`,
);
await engine.executeRaw(
`INSERT INTO subagent_messages (job_id, message_idx, role, content_blocks)
VALUES (990001, 0, 'assistant', to_jsonb('[{"type":"text"}]'::text))`,
);
await engine.executeRaw(
`INSERT INTO subagent_messages (job_id, message_idx, role, content_blocks)
VALUES (990001, 1, 'assistant', to_jsonb('plain text payload, not JSON'::text))`,
);
// Container-LOOKING but invalid JSON — matches the shape probe but
// pg_input_is_valid must exclude it (repairing it would throw).
await engine.executeRaw(
`INSERT INTO subagent_messages (job_id, message_idx, role, content_blocks)
VALUES (990001, 2, 'assistant', to_jsonb('[INFO] fetch complete'::text))`,
);
const damaged = await jsonbIntegrityCheck(engine);
expect(damaged.status).toBe('warn');
// Exactly the double-encoded row counts — the legit string scalar doesn't.
expect(damaged.message).toContain('subagent_messages.content_blocks=1');
// Cleanup, then prove the ok path again.
await engine.executeRaw(`DELETE FROM subagent_messages WHERE job_id = 990001`);
await engine.executeRaw(`DELETE FROM minion_jobs WHERE id = 990001`);
expect((await jsonbIntegrityCheck(engine)).status).toBe('ok');
// Absent-table skip lane: rename a target table; the check must skip
// it without throwing (pre-v0.15 brains lack subagent_* entirely).
await engine.executeRaw(`ALTER TABLE subagent_tool_executions RENAME TO subagent_tool_executions_bak`);
try {
expect((await jsonbIntegrityCheck(engine)).status).toBe('ok');
} finally {
await engine.executeRaw(`ALTER TABLE subagent_tool_executions_bak RENAME TO subagent_tool_executions`);
}
} finally {
await engine.disconnect();
}
});
test('skill conformance derives a valid host manifest when manifest.json is absent', async () => {
const { skillConformanceCheck } = await import('../src/commands/doctor.ts');
const skillsDir = join(tmpdir(), `gbrain-doctor-skills-${crypto.randomUUID()}`);
test/e2e/engine-parity.test.ts
+32
View File
@@ -528,6 +528,38 @@ describeBoth('Engine parity — Postgres vs PGLite', () => {
}
});
test('#2555 getChunks sourceIds[] parity: federated grant + scalar floor + unset default identical on both engines', async () => {
for (const eng of [pgEngine, pgliteEngine]) {
await eng.executeRaw(`INSERT INTO sources (id, name, local_path) VALUES ('gcp-beta', 'gcp-beta', '/tmp/gcp-beta') ON CONFLICT (id) DO NOTHING`);
await eng.putPage('wiki/gcp-doc', {
type: 'note', title: 'beta doc', compiled_truth: 'beta body', timeline: '',
}, { sourceId: 'gcp-beta' });
await eng.upsertChunks('wiki/gcp-doc', [
{ chunk_index: 0, chunk_text: 'gcp beta chunk', chunk_source: 'compiled_truth' },
], { sourceId: 'gcp-beta' });
await eng.putPage('wiki/gcp-doc', {
type: 'note', title: 'default decoy', compiled_truth: 'decoy body', timeline: '',
}, { sourceId: 'default' });
await eng.upsertChunks('wiki/gcp-doc', [
{ chunk_index: 0, chunk_text: 'gcp default decoy', chunk_source: 'compiled_truth' },
], { sourceId: 'default' });
}
for (const eng of [pgEngine, pgliteEngine]) {
// Federated array wins over scalar and reaches the non-default source.
const federated = await eng.getChunks('wiki/gcp-doc', { sourceId: 'default', sourceIds: ['gcp-beta'] });
expect(federated.map(c => c.chunk_text)).toEqual(['gcp beta chunk']);
// Out-of-grant array → empty, never a fall-through to 'default'.
const outOfGrant = await eng.getChunks('wiki/gcp-doc', { sourceIds: ['gcp-nonexistent'] });
expect(outOfGrant).toEqual([]);
// Unset opts keep the historical 'default' floor.
const unset = await eng.getChunks('wiki/gcp-doc');
expect(unset.map(c => c.chunk_text)).toEqual(['gcp default decoy']);
// #2544 trim keeps the Chunk shape (embedding deliberately unselected → null).
expect(federated[0].embedding).toBeNull();
}
});
test('v114 (#1941) listLinkSources parity: same ordered provenance counts on both engines', async () => {
const mk = async (eng: BrainEngine) => {
for (const s of ['lsp-a', 'lsp-b', 'lsp-c']) {
test/e2e/list-all-sources-postgres.test.ts
+30
View File
@@ -160,4 +160,34 @@ describeIfDB('Postgres parity — updateSourceConfig', () => {
expect(rows[0]?.typeof).toBe('object');
expect(rows[0]?.value).toBe('2026-05-22T12:00:00.000Z');
});
test('#2251: mixed-array config (non-object elements) merges instead of throwing, and self-heals to a flat object', async () => {
await seedSource('mixed');
// The historical bad shape that permanently blocked last_full_cycle_at
// writes: a JSONB array holding a non-object element. The bare
// jsonb_each(elem) threw 'cannot call jsonb_each on a non-object'
// DURING row production, failing every subsequent updateSourceConfig.
await engine.executeRaw(
`UPDATE sources
SET config = '["stray-string", {"remote_url": "https://kept"}, 42]'::jsonb
WHERE id = 'mixed'`,
);
const ok = await engine.updateSourceConfig('mixed', {
last_full_cycle_at: '2026-07-09T00:00:00.000Z',
});
expect(ok).toBe(true);
const rows = await engine.executeRaw<{ typeof: string; cycle: string | null; kept: string | null }>(
`SELECT jsonb_typeof(config) AS typeof,
config->>'last_full_cycle_at' AS cycle,
config->>'remote_url' AS kept
FROM sources WHERE id = 'mixed'`,
);
// Self-healed: flat object, patch applied, object elements' keys recovered,
// non-object stragglers dropped.
expect(rows[0]?.typeof).toBe('object');
expect(rows[0]?.cycle).toBe('2026-07-09T00:00:00.000Z');
expect(rows[0]?.kept).toBe('https://kept');
});
});
test/e2e/postgres-bootstrap.test.ts
+64
View File
@@ -91,6 +91,70 @@ describe.skipIf(skip)('PostgresEngine forward-reference bootstrap (E2E)', () =>
expect(await engine.getConfig('version')).toBe(String(LATEST_VERSION));
});
test('pre-v121 timeline shape converges to full final shape on REAL Postgres (#2626 wedge class)', async () => {
// The v121 wedge was Postgres-visible in production (blob CREATE INDEX
// on a column migration v121 hadn't added yet); the PGLite twins live in
// test/bootstrap.test.ts. Rewind schema AND the version counter to the
// wedged cohort's true state, then assert full initSchema convergence:
// column + FK + BOTH partial indexes, ledger at LATEST.
await engine.initSchema();
const conn = (engine as any).sql;
await conn.unsafe(`
DROP INDEX IF EXISTS idx_timeline_event_dedup;
DROP INDEX IF EXISTS idx_timeline_event_page;
ALTER TABLE timeline_entries DROP CONSTRAINT IF EXISTS timeline_entries_event_page_id_fkey;
ALTER TABLE timeline_entries DROP COLUMN IF EXISTS event_page_id;
`);
await engine.setConfig('version', '120');
await engine.initSchema();
expect(await engine.getConfig('version')).toBe(String(LATEST_VERSION));
const col = await conn`
SELECT column_name FROM information_schema.columns
WHERE table_schema = current_schema()
AND table_name = 'timeline_entries' AND column_name = 'event_page_id'
`;
expect(col).toHaveLength(1);
const fk = await conn`
SELECT conname FROM pg_constraint WHERE conname = 'timeline_entries_event_page_id_fkey'
`;
expect(fk).toHaveLength(1);
const idx = await conn`
SELECT indexname FROM pg_indexes
WHERE tablename = 'timeline_entries'
AND indexname IN ('idx_timeline_event_page', 'idx_timeline_event_dedup')
`;
expect(idx).toHaveLength(2);
}, 60_000);
test('pre-v7 minion_jobs shape (scanner-sweep wedge class) converges on REAL Postgres', async () => {
await engine.initSchema();
const conn = (engine as any).sql;
await conn.unsafe(`
DROP INDEX IF EXISTS idx_minion_jobs_timeout;
DROP INDEX IF EXISTS uniq_minion_jobs_idempotency;
ALTER TABLE minion_jobs DROP COLUMN IF EXISTS timeout_at;
ALTER TABLE minion_jobs DROP COLUMN IF EXISTS idempotency_key;
`);
await engine.initSchema();
const cols = await conn`
SELECT column_name FROM information_schema.columns
WHERE table_schema = current_schema()
AND table_name = 'minion_jobs'
AND column_name IN ('timeout_at', 'idempotency_key')
`;
expect(cols).toHaveLength(2);
const idx = await conn`
SELECT indexname FROM pg_indexes
WHERE tablename = 'minion_jobs'
AND indexname IN ('idx_minion_jobs_timeout', 'uniq_minion_jobs_idempotency')
`;
expect(idx).toHaveLength(2);
}, 60_000);
// Migration v120 — schema-lint hardening (#1647 / #171). Postgres-only
// assertions (security_invoker has no surface on embedded PGLite).
test('v120: page_links view runs with security_invoker=on (#1647b)', async () => {
test/e2e/salience-llm-routing.test.ts
+26
View File
@@ -158,3 +158,29 @@ describeIfKey('v0.29 — LLM routes personal queries to v0.29 ops, not query() /
}, 30_000);
}
});
// #2416 — concept/landscape questions must route to `query` (hybrid +
// expansion), not `search` (cheap-hybrid, expansion off). This block is the
// feature assertion for the #2416 description edits; the block above is the
// regression guard (the new "prefer query for concept questions" copy must
// NOT pull personal queries away from the salience ops).
const CONCEPT_QUERY_PHRASINGS = [
'find all the companies doing offshore wind',
'the landscape of agent memory startups',
'every investor that focuses on climate tech',
'all the projects that use vector databases',
'everything about my fundraising strategy discussions',
'which portfolio companies have shipped AI features',
'the ecosystem of MCP server implementations',
'all the people that work on developer tools',
];
describeIfKey('#2416 — LLM routes concept/landscape questions to query, not search', () => {
for (const prompt of CONCEPT_QUERY_PHRASINGS) {
test(`routes "${prompt}" to query`, async () => {
const { tool } = await callClaudeWithTools(prompt);
expect(tool).not.toBeNull();
expect(tool).toBe('query');
}, 30_000);
}
});
test/entity-card-perf.slow.test.ts
+180
View File
@@ -0,0 +1,180 @@
/**
* MEMORY_VERBS v1 — entity() latency gate (Cathedral 1, frozen contract:
* p99 < 100ms on a large corpus, zero LLM).
*
* Corpus: 20K pages / 100K links / 30K aliases / 40K facts seeded via
* generate_series (pattern: entity-resolve-perf.slow.test.ts). 20 warmup +
* 200 measured buildEntityCard calls over a mixed name set exercising all
* three resolution arms (alias hit / exact title / slug-suffix) + misses.
*
* Two gates:
* 1. HARD ABSOLUTE — p99 < 100ms × GBRAIN_PERF_BUDGET_MULTIPLIER (default 1;
* loosen in CI only with evidence of runner noise). The protocol DOC
* promises this number; the bound is op-layer latency (transport
* excluded, as documented).
* 2. RATIO GUARD (machine-independent) — entity p99 ≤ 50× max(getPage p50,
* 1ms) on the same corpus. Calibration: the card is ~7 indexed reads +
* a keyword search on the miss path, measured ~21× a 1ms-floored
* getPage at 20K pages — an O(N) scan regression lands at 200ms+
* (≥200×), far past the ceiling even on a slow runner, while the
* 2.4× headroom absorbs planner noise.
*
* The 200K-page validation is a documented MANUAL recipe in
* docs/protocol/MEMORY_VERBS_v1.md — not CI-gated (seed time would dominate).
*
* .slow.test.ts suffix keeps it out of the fast loop (`bun run test:slow`).
*/
import { describe, it, expect, beforeAll, afterAll } from 'bun:test';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { buildEntityCard } from '../src/core/verbs/entity-card.ts';
let engine: PGLiteEngine;
const PAGES = 20_000;
const LINKS = 100_000;
const ALIASES = 30_000;
const FACTS = 40_000;
const WARMUP = 20;
const MEASURED = 200;
const TARGET_ENTITIES = 50; // pages the measured calls rotate over
const P99_BUDGET_MS = 100 * (Number(process.env.GBRAIN_PERF_BUDGET_MULTIPLIER) || 1);
// entity p99 ≤ 50× max(getPage p50, 1ms) — see the calibration note above.
const RATIO_CEILING = 50;
function percentile(sorted: number[], p: number): number {
const idx = Math.min(sorted.length - 1, Math.ceil((p / 100) * sorted.length) - 1);
return sorted[Math.max(0, idx)];
}
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const db = (engine as any).db;
// Target entities (real putPage so frontmatter/title behave like prod pages).
for (let i = 0; i < TARGET_ENTITIES; i++) {
const slug = `people/target-person-${i}`;
await engine.putPage(slug, {
type: 'person',
title: `Target Person ${i}`,
compiled_truth: `# Target Person ${i}\n\nRuns area ${i} at a-company. Synthetic perf-corpus entity.`,
frontmatter: { type: 'person', title: `Target Person ${i}`, slug, summary: `Synthetic target ${i} for the entity-card latency gate.` },
}, { sourceId: 'default' });
}
// Filler pages in one generate_series insert.
await db.query(
`INSERT INTO pages (slug, type, title, compiled_truth, frontmatter, source_id, created_at, updated_at)
SELECT 'filler/page-' || gs::text, 'note', 'Filler ' || gs::text, '# Filler', '{}', 'default', NOW(), NOW()
FROM generate_series(1, ${PAGES}) gs`,
);
// Links: filler→filler hub noise plus a fan-in/out around every target
// (the card reads getLinks/getBacklinks — targets must have real edges).
await db.query(
`INSERT INTO links (from_page_id, to_page_id, link_type, link_source)
SELECT p1.id, p2.id, 'mentions', 'mentions'
FROM (SELECT id, row_number() OVER (ORDER BY id) rn FROM pages WHERE slug LIKE 'filler/%') p1
JOIN (SELECT id, row_number() OVER (ORDER BY id) rn FROM pages WHERE slug LIKE 'filler/%') p2
ON p2.rn = ((p1.rn * 7919) % ${PAGES}) + 1 AND p1.id <> p2.id
CROSS JOIN generate_series(1, ${Math.ceil(LINKS / PAGES)}) g
ON CONFLICT DO NOTHING`,
);
await db.query(
`INSERT INTO links (from_page_id, to_page_id, link_type, link_source)
SELECT t.id, f.id, 'works_at', 'markdown'
FROM (SELECT id, row_number() OVER (ORDER BY id) rn FROM pages WHERE slug LIKE 'people/target-%') t
JOIN (SELECT id, row_number() OVER (ORDER BY id) rn FROM pages WHERE slug LIKE 'filler/%' LIMIT 2000) f
ON (f.rn % ${TARGET_ENTITIES}) + 1 = t.rn
ON CONFLICT DO NOTHING`,
);
// Aliases: bulk noise + 2 aliases per target.
await db.query(
`INSERT INTO page_aliases (source_id, alias_norm, slug)
SELECT 'default', 'alias noise ' || gs::text, 'filler/page-' || ((gs % ${PAGES}) + 1)::text
FROM generate_series(1, ${ALIASES}) gs
ON CONFLICT DO NOTHING`,
);
for (let i = 0; i < TARGET_ENTITIES; i++) {
await db.query(
`INSERT INTO page_aliases (source_id, alias_norm, slug) VALUES
('default', $1, $2), ('default', $3, $2)
ON CONFLICT DO NOTHING`,
[`tp${i}`, `people/target-person-${i}`, `target alias ${i}`],
);
}
// Facts: bulk noise across fillers + 20 active facts per target entity.
await db.query(
`INSERT INTO facts (source_id, entity_slug, fact, kind, visibility, notability, valid_from, source, confidence, created_at)
SELECT 'default', 'filler/page-' || ((gs % ${PAGES}) + 1)::text,
'noise fact ' || gs::text, 'fact', 'world', 'medium', NOW(), 'perf-seed', 1.0, NOW()
FROM generate_series(1, ${FACTS - TARGET_ENTITIES * 20}) gs`,
);
await db.query(
`INSERT INTO facts (source_id, entity_slug, fact, kind, visibility, notability, valid_from, source, confidence, created_at)
SELECT 'default', 'people/target-person-' || t::text,
'target fact ' || g::text || ' about person ' || t::text,
CASE WHEN g % 5 = 0 THEN 'commitment' ELSE 'fact' END,
'world', 'medium', NOW(), 'perf-seed', 1.0, NOW()
FROM generate_series(0, ${TARGET_ENTITIES - 1}) t, generate_series(1, 20) g`,
);
}, 300_000);
afterAll(async () => {
await engine.disconnect();
});
describe('entity card p99 latency gate', () => {
it(`p99 < ${P99_BUDGET_MS}ms on ${PAGES} pages AND ≤ ${RATIO_CEILING}× getPage p50`, async () => {
// Mixed name set: alias hits, exact titles, namespaced slugs, suffixes, misses.
const names: string[] = [];
for (let i = 0; i < TARGET_ENTITIES; i++) {
names.push(`tp${i}`); // alias arm
names.push(`Target Person ${i}`); // exact-title arm
names.push(`people/target-person-${i}`); // exact-slug arm
names.push(`target-person-${i}`); // slug-suffix arm
names.push(`zzz-absent-${i}`); // miss (suggestions path)
}
for (let i = 0; i < WARMUP; i++) {
await buildEntityCard(engine, 'default', names[i % names.length], { remote: true });
}
const samples: number[] = [];
for (let i = 0; i < MEASURED; i++) {
const name = names[(i * 13) % names.length];
const t0 = performance.now();
await buildEntityCard(engine, 'default', name, { remote: true });
samples.push(performance.now() - t0);
}
samples.sort((a, b) => a - b);
const p50 = percentile(samples, 50);
const p99 = percentile(samples, 99);
// Ratio baseline: getPage p50 on the same corpus.
const pageSamples: number[] = [];
for (let i = 0; i < 50; i++) {
const t0 = performance.now();
await engine.getPage(`people/target-person-${i % TARGET_ENTITIES}`, { sourceId: 'default' });
pageSamples.push(performance.now() - t0);
}
pageSamples.sort((a, b) => a - b);
const pageP50 = Math.max(percentile(pageSamples, 50), 1.0); // 1ms floor vs sub-ms division noise
// eslint-disable-next-line no-console
console.log(
`[entity-card-perf] corpus=${PAGES}p+${LINKS}l+${ALIASES}a+${FACTS}f ` +
`entity p50=${p50.toFixed(2)}ms p99=${p99.toFixed(2)}ms | getPage p50=${pageP50.toFixed(2)}ms ` +
`| ratio=${(p99 / pageP50).toFixed(1)}x (ceiling ${RATIO_CEILING}x) | budget=${P99_BUDGET_MS}ms`,
);
expect(p99).toBeLessThan(P99_BUDGET_MS);
expect(p99 / pageP50).toBeLessThanOrEqual(RATIO_CEILING);
}, 300_000);
});
test/facts-extract-salvage.test.ts
+75
View File
@@ -0,0 +1,75 @@
import { afterEach, describe, expect, test } from 'bun:test';
import {
__setChatTransportForTests,
resetGateway,
type ChatResult,
} from '../src/core/ai/gateway.ts';
import { extractFactsFromTurnWithOutcome } from '../src/core/facts/extract.ts';
afterEach(() => {
__setChatTransportForTests(null);
resetGateway();
});
function stubFacts(facts: unknown[]): void {
__setChatTransportForTests(async (): Promise<ChatResult> => ({
text: JSON.stringify({ facts }),
blocks: [],
stopReason: 'end',
usage: {
input_tokens: 0,
output_tokens: 0,
cache_read_tokens: 0,
cache_creation_tokens: 0,
},
model: 'test:stub',
providerId: 'test',
}));
}
async function extract() {
return extractFactsFromTurnWithOutcome({
turnText: 'Conversation segment under test.',
source: 'test:salvage',
});
}
describe('facts extractor candidate salvage (#3866)', () => {
test('keeps valid facts when another candidate is malformed', async () => {
stubFacts([
{
fact: 'The migration completed',
kind: 'event',
entity: null,
confidence: 1.0,
notability: 'high',
},
{
fact: 'This candidate has no valid kind',
kind: null,
},
]);
const outcome = await extract();
expect(outcome.ok).toBe(true);
if (!outcome.ok) throw new Error(outcome.reason);
expect(outcome.facts).toHaveLength(1);
expect(outcome.facts[0]!.fact).toBe('The migration completed');
});
test('keeps malformed_output when every candidate is invalid', async () => {
stubFacts([
{ fact: 'Missing kind', kind: null },
{ fact: null, kind: 'fact' },
]);
expect(await extract()).toEqual({ ok: false, reason: 'malformed_output' });
});
test('keeps an explicitly empty array as a successful empty result', async () => {
stubFacts([]);
expect(await extract()).toEqual({ ok: true, facts: [] });
});
});
test/fixtures/memory-verbs/cases.json
Vendored
+290
View File
@@ -0,0 +1,290 @@
[
{
"name": "remember rejects missing provenance (provenance_required + suggestion)",
"verb": "remember",
"params": {
"fact": "conformance {{marker}} fact without provenance"
},
"expectErrorCode": "invalid_params",
"expectSuggestion": false
},
{
"name": "remember rejects empty provenance (provenance_required + suggestion)",
"verb": "remember",
"params": {
"fact": "conformance {{marker}} fact empty provenance",
"provenance": " "
},
"expectErrorCode": "provenance_required",
"expectSuggestion": true
},
{
"name": "remember rejects ISO-8601 duration ttl with a fix (P30D trap)",
"verb": "remember",
"params": {
"fact": "conformance {{marker}} ttl trap",
"provenance": "conformance run",
"ttl": "P30D"
},
"expectErrorCode": "invalid_params",
"expectSuggestion": true
},
{
"name": "remember writes a fact (string id, enum status, echoed nulls)",
"verb": "remember",
"params": {
"fact": "conformance {{marker}}: the protocol round-trip fact",
"provenance": "conformance run {{marker}}",
"entity": "people/conformance-{{marker}}",
"kind": "fact"
},
"validateSchema": true,
"expect": [
{
"path": "id",
"type": "string"
},
{
"path": "status",
"oneOf": [
"inserted",
"duplicate",
"superseded"
]
},
{
"path": "protocol_version",
"equals": 1
}
],
"saveAs": {
"key": "fact1",
"path": "id"
}
},
{
"name": "remember with ttl returns ISO valid_until",
"verb": "remember",
"params": {
"fact": "conformance {{marker}}: expiring fact",
"provenance": "conformance run {{marker}}",
"entity": "people/conformance-{{marker}}",
"ttl": "30d"
},
"validateSchema": true,
"expect": [
{
"path": "valid_until",
"type": "string"
}
],
"saveAs": {
"key": "fact2",
"path": "id"
}
},
{
"name": "remember private fact (fence test setup)",
"verb": "remember",
"params": {
"fact": "conformance {{marker}} PRIVATE-SENTINEL commitment",
"provenance": "conformance run {{marker}}",
"entity": "people/conformance-{{marker}}",
"kind": "commitment",
"visibility": "private"
},
"validateSchema": true
},
{
"name": "recall by entity round-trips the remembered fact (superset envelope)",
"verb": "recall",
"params": {
"entity": "people/conformance-{{marker}}"
},
"validateSchema": true,
"expect": [
{
"path": "protocol_version",
"equals": 1
},
{
"path": "total",
"gte": 1
},
{
"path": "facts.0.fact_id",
"type": "string"
},
{
"path": "facts.0.provenance",
"type": "string"
}
]
},
{
"name": "recall with budget reports consistent budget meta",
"verb": "recall",
"params": {
"entity": "people/conformance-{{marker}}",
"budget_tokens": 10000
},
"validateSchema": true,
"expect": [
{
"path": "budget_tokens",
"equals": 10000
},
{
"path": "budget_used",
"gte": 0
},
{
"path": "budget_used",
"lte": 10000
},
{
"path": "dropped_count",
"gte": 0
}
]
},
{
"name": "recall with budget smaller than the first item drops everything",
"verb": "recall",
"params": {
"entity": "people/conformance-{{marker}}",
"budget_tokens": 1
},
"validateSchema": true,
"expect": [
{
"path": "total",
"equals": 0
},
{
"path": "dropped_count",
"gte": 1
}
]
},
{
"name": "recall with query returns the search arm (degradation allowed, never an error)",
"verb": "recall",
"params": {
"query": "conformance {{marker}} protocol round-trip",
"budget_tokens": 8000
},
"validateSchema": true,
"expect": [
{
"path": "results",
"type": "array"
}
]
},
{
"name": "entity resolves the round-trip entity to a card",
"verb": "entity",
"params": {
"name": "people/conformance-{{marker}}"
},
"validateSchema": true,
"requiresSeededEntity": true,
"expect": [
{
"path": "found",
"equals": true
},
{
"path": "protocol_version",
"equals": 1
},
{
"path": "card.entity.slug",
"type": "string"
},
{
"path": "card.backlink_count",
"gte": 0
}
]
},
{
"name": "entity miss returns found:false + suggestions (never an error)",
"verb": "entity",
"params": {
"name": "zzz-no-such-entity-{{marker}}"
},
"validateSchema": true,
"expect": [
{
"path": "found",
"equals": false
},
{
"path": "suggestions",
"type": "array"
}
]
},
{
"name": "entity card never leaks private facts to remote callers (fence test)",
"verb": "entity",
"params": {
"name": "people/conformance-{{marker}}"
},
"requiresSeededEntity": true,
"expect": [
{
"path": "card.open_threads",
"absentOrNotContains": "PRIVATE-SENTINEL"
}
]
},
{
"name": "forget expires the remembered fact",
"verb": "forget",
"params": {
"id": "{{id:fact2}}",
"reason": "conformance cleanup"
},
"validateSchema": true,
"expect": [
{
"path": "expired",
"equals": true
}
]
},
{
"name": "forget again is idempotent (expired:false, success)",
"verb": "forget",
"params": {
"id": "{{id:fact2}}"
},
"validateSchema": true,
"expect": [
{
"path": "expired",
"equals": false
}
]
},
{
"name": "forget unknown id returns not_found with a suggestion",
"verb": "forget",
"params": {
"id": "999999999"
},
"expectErrorCode": "not_found",
"expectSuggestion": true
},
{
"name": "synthesize answers (or reports unavailable) — cost-gated, pass --synthesize",
"verb": "synthesize",
"params": {
"question": "What do we know about conformance {{marker}}?"
},
"requiresSynthesizeFlag": true
}
]
test/get-page-federated-scope.test.ts
+85
View File
@@ -373,3 +373,88 @@ describe('#2200 engine secondary-fetch methods honor sourceIds[]', () => {
expect(windowed.map(e => e.summary)).toEqual(['june event']);
});
});
// ---------------------------------------------------------------------------
// #2555 — get_chunks honors the federated source grant (same class as #1393/
// #2200, chunk read path). Pre-fix the op used the pre-#2200 scalar pattern
// and engine.getChunks had no sourceIds[] support: a federated client that
// could read a page via get_page got [] from get_chunks.
// ---------------------------------------------------------------------------
describe('#2555 get_chunks federated scope', () => {
const get_chunks = operations.find(o => o.name === 'get_chunks')!;
beforeEach(async () => {
await engine.upsertChunks('secret/beta-doc', [
{ chunk_index: 0, chunk_text: 'beta chunk zero', chunk_source: 'compiled_truth' },
{ chunk_index: 1, chunk_text: 'beta chunk one', chunk_source: 'compiled_truth' },
], { sourceId: 'beta' });
// Same-slug decoy chunks in 'default' — the cross-source bleed guard.
await engine.upsertChunks('secret/beta-doc', [
{ chunk_index: 0, chunk_text: 'default decoy chunk', chunk_source: 'compiled_truth' },
], { sourceId: 'default' });
});
test('op: federated grant including the page source returns its chunks (the #2555 repro)', async () => {
const ctx = ctxOf({ remote: true, sourceId: undefined, auth: { token: 't', clientId: 'c', scopes: [], allowedSources: ['alpha', 'beta'] } as any });
const chunks = await get_chunks.handler(ctx, { slug: 'secret/beta-doc' }) as Array<{ chunk_text: string }>;
expect(chunks.map(c => c.chunk_text)).toEqual(['beta chunk zero', 'beta chunk one']);
});
test('op: grant excluding the page source stays empty — never falls through to default', async () => {
const ctx = ctxOf({ remote: true, sourceId: undefined, auth: { token: 't', clientId: 'c', scopes: [], allowedSources: ['alpha'] } as any });
const chunks = await get_chunks.handler(ctx, { slug: 'secret/beta-doc' }) as Array<{ chunk_text: string }>;
expect(chunks).toEqual([]);
});
test('op: no grant + default floor sees only the default decoy, never beta chunks', async () => {
const ctx = ctxOf({ remote: true, sourceId: 'default', auth: undefined });
const chunks = await get_chunks.handler(ctx, { slug: 'secret/beta-doc' }) as Array<{ chunk_text: string }>;
expect(chunks.map(c => c.chunk_text)).toEqual(['default decoy chunk']);
});
test('engine: sourceIds[] precedence over scalar; trimmed SELECT keeps the Chunk shape', async () => {
// array beats scalar: scalar 'default' would return the decoy; array ['beta'] must win.
const prec = await engine.getChunks('secret/beta-doc', { sourceId: 'default', sourceIds: ['beta'] });
expect(prec.map(c => c.chunk_text)).toEqual(['beta chunk zero', 'beta chunk one']);
// #2544 trim: embedding is deliberately not selected (rowToChunk discards
// it here anyway) and the rest of the Chunk shape survives.
expect(prec[0].embedding).toBeNull();
expect(prec[0].chunk_index).toBe(0);
expect(prec[0].chunk_source).toBe('compiled_truth');
// Unset opts keep the historical 'default' floor (importCodeFile contract).
const def = await engine.getChunks('secret/beta-doc');
expect(def.map(c => c.chunk_text)).toEqual(['default decoy chunk']);
});
test('#2544 structural pin: neither engine SELECTs cc.* in getChunks (the trim survives merges)', async () => {
// The behavioral assertion above is vacuous for the trim itself —
// rowToChunk hard-nulls embedding regardless of the SELECT. This pin
// exists because a master merge once silently restored `SELECT cc.*`
// while the doc comment kept claiming the trim: assert the SELECT shape
// at the source level for BOTH engines.
const { readFileSync } = await import('fs');
for (const enginePath of ['src/core/postgres-engine.ts', 'src/core/pglite-engine.ts']) {
const src = readFileSync(new URL(`../${enginePath}`, import.meta.url), 'utf-8');
const start = src.indexOf('async getChunks(slug');
expect(start).toBeGreaterThan(0);
// The method's own close (`\n }` at 2-space indent) — an inline
// `async (tx) =>` callback must not truncate the body, and the NEXT
// method (e.g. buildStaleChunkWhere's `cc.embedding IS NULL` WHERE
// predicate) must not leak in. Strip line comments: the pin targets
// the SQL, not prose that may cite the anti-pattern.
const end = src.indexOf('\n }\n', start + 10);
const body = src.slice(start, end).replace(/\/\/[^\n]*/g, '');
expect(body, `${enginePath} getChunks must not SELECT cc.*`).not.toContain('cc.*');
// Every non-vector field rowToChunk reads MUST be selected — omitting
// one silently degrades round-trips (embed.ts getChunks→upsertChunks
// rewrote image chunks as text when cc.modality was dropped).
for (const col of ['chunk_text', 'chunk_source', 'model', 'token_count', 'embedded_at',
'language', 'symbol_name', 'symbol_type', 'start_line', 'end_line',
'parent_symbol_path', 'doc_comment', 'symbol_name_qualified', 'modality']) {
expect(body, `${enginePath} getChunks must select cc.${col}`).toContain(`cc.${col}`);
}
// The vector columns stay unselected.
expect(body).not.toMatch(/cc\.embedding\b/);
}
});
});
test/mcp-surface.test.ts
+110
View File
@@ -0,0 +1,110 @@
/**
* MEMORY_VERBS v1 — surface-mode tests (Cathedral 1).
*
* - 'verbs' filters to EXACTLY the five protocol verbs
* - 'full' is the identity (existing installs unchanged)
* - dispatch-layer allowedOps is FAIL-CLOSED: a hidden op is uncallable
* (unknown_tool), not merely unlisted [c2]
* - flag parsing is strict (unknown value rejects loudly)
* - resolution: flag > config mcp_surface > 'full'
*/
import { afterAll, beforeAll, describe, expect, it } from 'bun:test';
import { mkdtempSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { operations } from '../src/core/operations.ts';
import { VERB_NAMES } from '../src/core/verbs.ts';
import {
filterOpsForSurface,
allowedOpNames,
parseSurfaceFlag,
resolveSurface,
} from '../src/mcp/surface.ts';
import { dispatchToolCall } from '../src/mcp/dispatch.ts';
import { __setUsageLogPathForTests } from '../src/core/verbs/usage-log.ts';
let engine: PGLiteEngine;
let home: string;
beforeAll(async () => {
// Sidecar writes go to a temp file via the test seam — no global env mutation.
home = mkdtempSync(join(tmpdir(), 'gbrain-surface-test-'));
__setUsageLogPathForTests(join(home, 'usage.jsonl'));
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
});
afterAll(async () => {
await engine.disconnect();
__setUsageLogPathForTests(null);
try { rmSync(home, { recursive: true, force: true }); } catch { /* best-effort */ }
});
describe('filterOpsForSurface', () => {
it("'verbs' returns exactly the five protocol verbs", () => {
const names = filterOpsForSurface(operations, 'verbs').map(o => o.name).sort();
expect(names).toEqual([...VERB_NAMES].sort());
});
it("'full' is the identity — existing installs see every op", () => {
expect(filterOpsForSurface(operations, 'full')).toEqual(operations);
});
});
describe('dispatch allowedOps — fail-closed [c2]', () => {
it('a hidden op returns unknown_tool even when called by name', async () => {
const allowed = allowedOpNames(operations, 'verbs');
const res = await dispatchToolCall(engine, 'get_page', { slug: 'x' }, {
remote: true,
sourceId: 'default',
allowedOps: allowed,
surface: 'verbs',
});
expect(res.isError).toBe(true);
const body = JSON.parse(res.content[0].text);
expect(body.error).toBe('unknown_tool');
});
it('a surfaced verb still dispatches under the same allowedOps set', async () => {
const allowed = allowedOpNames(operations, 'verbs');
const res = await dispatchToolCall(engine, 'entity', { name: 'zzz-nobody' }, {
remote: true,
sourceId: 'default',
allowedOps: allowed,
surface: 'verbs',
});
expect(res.isError ?? false).toBe(false);
const body = JSON.parse(res.content[0].text);
expect(body.found).toBe(false);
});
it('without allowedOps (full surface) every op stays callable — pre-existing behavior', async () => {
const res = await dispatchToolCall(engine, 'get_stats', {}, {
remote: true,
sourceId: 'default',
});
expect(res.isError ?? false).toBe(false);
});
});
describe('parseSurfaceFlag + resolveSurface', () => {
it('parses verbs/full, rejects unknown values loudly, requires a value', () => {
expect(parseSurfaceFlag(['--surface', 'verbs'])).toBe('verbs');
expect(parseSurfaceFlag(['--surface', 'full'])).toBe('full');
expect(parseSurfaceFlag(['serve'])).toBe(null);
expect(() => parseSurfaceFlag(['--surface', 'all'])).toThrow(/Unknown --surface/);
expect(() => parseSurfaceFlag(['--surface'])).toThrow(/requires a value/);
expect(() => parseSurfaceFlag(['--surface', '--http'])).toThrow(/requires a value/);
});
it('resolution: flag > config mcp_surface > full', () => {
expect(resolveSurface('verbs', { mcp_surface: 'full' })).toBe('verbs');
expect(resolveSurface(null, { mcp_surface: 'verbs' })).toBe('verbs');
expect(resolveSurface(null, {})).toBe('full');
expect(resolveSurface(null, null)).toBe('full');
expect(resolveSurface(null, { mcp_surface: 'bogus' as never })).toBe('full');
});
});
test/mcp-tool-defs.test.ts
+15
View File
@@ -56,6 +56,10 @@ function legacyInlineMap(ops: typeof operations) {
.filter(([, v]) => v.required)
.map(([k]) => k),
},
// MEMORY_VERBS v1: ToolAnnotations passthrough, emitted ONLY when the op
// defines them. The byte-stability contract is per-op: ops WITHOUT
// annotations keep the exact pre-v1 shape (pinned explicitly below).
...(op.annotations ? { annotations: op.annotations } : {}),
}));
}
@@ -66,6 +70,17 @@ describe('buildToolDefs', () => {
expect(JSON.stringify(extracted)).toBe(JSON.stringify(inline));
});
test('ops without annotations keep the pre-annotations shape exactly (no annotations key)', () => {
const extracted = buildToolDefs(operations);
for (const def of extracted) {
const op = operations.find(o => o.name === def.name)!;
if (!op.annotations) {
expect('annotations' in def).toBe(false);
expect(Object.keys(def)).toEqual(['name', 'description', 'inputSchema']);
}
}
});
test('preserves operation count', () => {
expect(buildToolDefs(operations).length).toBe(operations.length);
});
test/memory-verbs-conformance.test.ts
+533
View File
@@ -0,0 +1,533 @@
/**
* MEMORY_VERBS v1 conformance — the frozen-contract pin (Cathedral 1).
*
* In-process through dispatchToolCall (the exact layer both MCP transports
* share), against in-memory PGLite. Covers:
* - recall legacy SUPERSET regression (G1B: legacy fields byte-equal,
* additions allowed — protocol_version everywhere, no carve-out)
* - server-side budget packing math (incl. budget < first item)
* - query-arm keyword degradation (never an error without embeddings)
* - remember contract: provenance_required, ttl forms (P30D trap), enum
* kinds, world default + the remote remember→recall round-trip [F2],
* private facts hidden from remote readers
* - entity: card shape vs RESPONSE_SCHEMAS, three resolution arms,
* miss→suggestions, ZERO-LLM guard (chat transport rigged to throw)
* - synthesize: [EXPENSIVE prefix, annotations, clean `unavailable` with
* suggestion when no LLM is configured [c10]
* - forget: idempotency (expired:false), not_found with suggestion
* - writeSingleFact supersession rule [X1] + degraded dedup (embed seam)
* - negative conformance self-test [F3]: the runner FAILS a lying server
* - fixture mirror drift guard (cases.json === embedded module)
*/
import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'bun:test';
import { mkdtempSync, readFileSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { resetPgliteState } from './helpers/reset-pglite.ts';
import { operationsByName } from '../src/core/operations.ts';
import type { OperationContext } from '../src/core/operations.ts';
import { dispatchToolCall } from '../src/mcp/dispatch.ts';
import { buildToolDefs } from '../src/mcp/tool-defs.ts';
import { RESPONSE_SCHEMAS, ERROR_SCHEMA, VERB_NAMES } from '../src/core/verbs.ts';
import {
runConformance,
validateAgainstSchema,
type ConformanceClient,
} from '../src/core/verbs/conformance.ts';
import { CONFORMANCE_CASES } from '../src/core/verbs/conformance-fixtures.ts';
import { writeSingleFact } from '../src/core/facts/write-single.ts';
import {
configureGateway,
__setChatTransportForTests,
__setEmbedTransportForTests,
} from '../src/core/ai/gateway.ts';
import { __setUsageLogPathForTests } from '../src/core/verbs/usage-log.ts';
import { withEnv } from './helpers/with-env.ts';
let engine: PGLiteEngine;
let home: string;
beforeAll(async () => {
// Sidecar writes go to a temp file via the test seam — no global env mutation.
home = mkdtempSync(join(tmpdir(), 'gbrain-verbs-test-'));
__setUsageLogPathForTests(join(home, 'usage.jsonl'));
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
});
afterAll(async () => {
await engine.disconnect();
__setUsageLogPathForTests(null);
try { rmSync(home, { recursive: true, force: true }); } catch { /* best-effort */ }
});
beforeEach(async () => {
await resetPgliteState(engine);
__setChatTransportForTests(null);
__setEmbedTransportForTests(null);
});
function localCtx(sourceId = 'default'): OperationContext {
return {
engine,
config: {},
logger: { info: () => {}, warn: () => {}, error: () => {} },
dryRun: false,
remote: false,
sourceId,
} as unknown as OperationContext;
}
/** Remote-shaped call through the shared dispatcher (what both transports do). */
async function callRemote(name: string, params: Record<string, unknown>) {
const res = await dispatchToolCall(engine, name, params, {
remote: true,
takesHoldersAllowList: ['world'],
sourceId: 'default',
});
return { isError: res.isError === true, body: JSON.parse(res.content[0].text) };
}
async function seedEntityPage(slug: string, title: string, body = 'A synthetic test entity.') {
const put = operationsByName['put_page'];
await put.handler(localCtx(), {
slug,
content: `---\ntitle: ${title}\ntype: person\n---\n\n# ${title}\n\n${body}\n`,
});
}
describe('recall — G1B superset + budget packing', () => {
it('legacy-param recall keeps every legacy field shape and adds only the v1 fields', async () => {
const r1 = await callRemote('remember', {
fact: 'superset regression fact',
provenance: 'conformance test',
entity: 'people/superset-test',
});
expect(r1.isError).toBe(false);
const { isError, body } = await callRemote('recall', { entity: 'people/superset-test' });
expect(isError).toBe(false);
// Legacy envelope fields, unchanged shapes.
expect(typeof body.total).toBe('number');
expect(Array.isArray(body.facts)).toBe(true);
const f = body.facts[0];
const LEGACY_FACT_KEYS = [
'id', 'fact', 'kind', 'entity_slug', 'visibility', 'notability', 'valid_from',
'valid_until', 'expired_at', 'superseded_by', 'consolidated_at',
'consolidated_into', 'source', 'source_session', 'confidence', 'created_at',
];
for (const k of LEGACY_FACT_KEYS) expect(k in f).toBe(true);
expect(typeof f.id).toBe('number'); // legacy numeric id is FROZEN
// v1 additions (G1B superset — on EVERY response, no carve-out).
expect(body.protocol_version).toBe(1);
expect(f.fact_id).toBe(String(f.id));
expect(f.provenance).toBe(f.source);
// No query/budget params → no search/budget fields.
expect('results' in body).toBe(false);
expect('budget_tokens' in body).toBe(false);
});
it('budget packing reports consistent meta and drops everything under a 1-token budget', async () => {
for (let i = 0; i < 3; i++) {
await callRemote('remember', {
fact: `budget fact number ${i} with some padding text to cost tokens`,
provenance: 'conformance test',
entity: 'people/budget-test',
});
}
const big = await callRemote('recall', { entity: 'people/budget-test', budget_tokens: 10000 });
expect(big.body.budget_tokens).toBe(10000);
expect(big.body.budget_used).toBeGreaterThan(0);
expect(big.body.budget_used).toBeLessThanOrEqual(10000);
expect(big.body.dropped_count).toBe(0);
expect(big.body.total).toBe(3);
const tiny = await callRemote('recall', { entity: 'people/budget-test', budget_tokens: 1 });
expect(tiny.body.total).toBe(0);
expect(tiny.body.dropped_count).toBeGreaterThanOrEqual(3);
});
it('query arm degrades to keyword-only without an embedding provider — never an error', async () => {
await seedEntityPage('people/query-arm-test', 'Query Arm Marker Qzx');
const { isError, body } = await callRemote('recall', { query: 'Query Arm Marker Qzx' });
expect(isError).toBe(false);
expect(Array.isArray(body.results)).toBe(true);
expect(body.search_degraded).toBe('keyword_only_no_embedding_provider');
const violations = validateAgainstSchema(body, RESPONSE_SCHEMAS.recall);
expect(violations).toEqual([]);
});
});
describe('remember — contract behavior', () => {
it('rejects empty provenance with provenance_required + a populated suggestion', async () => {
const { isError, body } = await callRemote('remember', { fact: 'x', provenance: ' ' });
expect(isError).toBe(true);
expect(body.error).toBe('provenance_required');
expect(typeof body.suggestion).toBe('string');
expect(body.suggestion.length).toBeGreaterThan(0);
expect(body.protocol_version).toBe(1);
});
it('rejects ISO-8601 duration ttl (P30D) with a self-correcting suggestion', async () => {
const { isError, body } = await callRemote('remember', {
fact: 'ttl trap', provenance: 'test', ttl: 'P30D',
});
expect(isError).toBe(true);
expect(body.error).toBe('invalid_params');
expect(body.suggestion).toContain('30d');
});
it('accepts duration ttl and returns a future ISO valid_until; echoes null entity_slug', async () => {
const { isError, body } = await callRemote('remember', {
fact: 'expiring fact with ttl', provenance: 'test', ttl: '30d',
});
expect(isError).toBe(false);
expect(typeof body.id).toBe('string');
expect(body.status).toBe('inserted');
expect(body.entity_slug).toBe(null); // omitted optional inputs echo as null
expect(Date.parse(body.valid_until)).toBeGreaterThan(Date.now());
const violations = validateAgainstSchema(body, RESPONSE_SCHEMAS.remember);
expect(violations).toEqual([]);
});
it('remote remember→recall round-trip holds (world default [F2]); private facts stay hidden', async () => {
await callRemote('remember', {
fact: 'world-visible round-trip fact', provenance: 'test', entity: 'people/roundtrip-test',
});
await callRemote('remember', {
fact: 'PRIVATE-SENTINEL fact', provenance: 'test', entity: 'people/roundtrip-test',
visibility: 'private',
});
const { body } = await callRemote('recall', { entity: 'people/roundtrip-test' });
const texts = body.facts.map((f: { fact: string }) => f.fact).join('|');
expect(texts).toContain('world-visible round-trip fact');
expect(texts).not.toContain('PRIVATE-SENTINEL');
});
});
describe('entity — card, arms, zero LLM', () => {
it('resolves an exact namespaced slug to a schema-valid card with the chat gateway rigged to throw', async () => {
__setChatTransportForTests(() => {
throw new Error('entity must NEVER call the chat LLM');
});
await seedEntityPage('people/card-test', 'Card Test Person', 'Runs engineering at a-company.');
const { isError, body } = await callRemote('entity', { name: 'people/card-test' });
expect(isError).toBe(false);
expect(body.found).toBe(true);
expect(typeof body.latency_ms).toBe('number');
expect(body.card.entity.slug).toBe('people/card-test');
expect(body.card.summary.length).toBeGreaterThan(0);
const violations = validateAgainstSchema(body, RESPONSE_SCHEMAS.entity);
expect(violations).toEqual([]);
});
it('resolves by exact title and by slug suffix (arms 2)', async () => {
await seedEntityPage('people/arm-test-alice', 'Arm Test Alice');
const byTitle = await callRemote('entity', { name: 'Arm Test Alice' });
expect(byTitle.body.found).toBe(true);
const bySuffix = await callRemote('entity', { name: 'arm-test-alice' });
expect(bySuffix.body.found).toBe(true);
});
it('miss returns found:false + suggestions, never an error', async () => {
await seedEntityPage('people/suggestion-source', 'Suggestion Source Person');
const { isError, body } = await callRemote('entity', { name: 'zzz-definitely-absent-entity' });
expect(isError).toBe(false);
expect(body.found).toBe(false);
expect(Array.isArray(body.suggestions)).toBe(true);
const violations = validateAgainstSchema(body, RESPONSE_SCHEMAS.entity);
expect(violations).toEqual([]);
});
it('[ship P1.2] entity card backlinks are source-isolated on BOTH sides (no foreign from_slug leak)', async () => {
// Same slug exists in two sources; a foreign-source page links to the
// default-source entity. The card must NOT surface the foreign edge/count.
await seedEntityPage('people/iso-target', 'Iso Target');
// Register the foreign tenant source (FK target) + a foreign page that
// links INTO the default-source entity.
await engine.executeRaw(
`INSERT INTO sources (id, name, config) VALUES ('other', 'other-tenant', '{}'::jsonb)
ON CONFLICT (id) DO NOTHING`,
[],
);
await engine.executeRaw(
`INSERT INTO pages (slug, type, title, compiled_truth, frontmatter, source_id, created_at, updated_at)
VALUES ('people/foreign-linker', 'person', 'Foreign Linker', '# Foreign', '{}', 'other', NOW(), NOW())`,
[],
);
await engine.executeRaw(
`INSERT INTO links (from_page_id, to_page_id, link_type, link_source)
SELECT f.id, t.id, 'works_at', 'markdown'
FROM pages f, pages t
WHERE f.slug = 'people/foreign-linker' AND f.source_id = 'other'
AND t.slug = 'people/iso-target' AND t.source_id = 'default'`,
[],
);
const { body } = await callRemote('entity', { name: 'people/iso-target' });
expect(body.found).toBe(true);
// The foreign cross-source backlink must not appear in edges OR the count.
const edgeSlugs = (body.card.edges as Array<{ slug: string }>).map(e => e.slug);
expect(edgeSlugs).not.toContain('people/foreign-linker');
expect(body.card.backlink_count).toBe(0);
});
it('remote card never carries private commitment facts (fence test)', async () => {
await seedEntityPage('people/fence-test', 'Fence Test Person');
await callRemote('remember', {
fact: 'PRIVATE-SENTINEL commitment text', provenance: 'test',
entity: 'people/fence-test', kind: 'commitment', visibility: 'private',
});
const { body } = await callRemote('entity', { name: 'people/fence-test' });
expect(body.found).toBe(true);
expect(JSON.stringify(body.card.open_threads)).not.toContain('PRIVATE-SENTINEL');
});
});
describe('synthesize — marked expensive + unavailable conversion [c10]', () => {
it('description starts with [EXPENSIVE and the tool def carries annotations', () => {
const op = operationsByName['synthesize'];
expect(op.description.startsWith('[EXPENSIVE')).toBe(true);
const def = buildToolDefs([op])[0];
expect(def.annotations?.readOnlyHint).toBe(true);
expect(def.annotations?.title).toContain('costly');
});
it('delegates to runThink and returns the frozen envelope with a priced cost block (chat seam — no real LLM)', async () => {
// Fully hermetic regardless of the dev/CI machine's ambient credentials:
// runThink builds its client via a real-key check (hasAnthropicKey reads
// process.env), NOT the gateway chat seam — so we must BOTH provide a fake
// key via withEnv (so the client builds) AND install the chat seam (so no
// real API call fires). Without the env key, CI (credential-free) takes the
// NO_ANTHROPIC_API_KEY path → the verb's `unavailable` conversion → isError.
__setChatTransportForTests(async () => ({
text: JSON.stringify({ answer: 'Synthesized test answer.', citations: [], gaps: ['none'] }),
blocks: [],
stopReason: 'end' as const,
usage: { input_tokens: 1200, output_tokens: 80, cache_read_tokens: 0, cache_creation_tokens: 0 },
model: 'anthropic:claude-haiku-4-5-20251001',
providerId: 'anthropic',
}));
const { isError, body } = await withEnv({ ANTHROPIC_API_KEY: 'sk-test-hermetic' }, async () =>
callRemote('synthesize', { question: 'what do we know?' }),
);
expect(isError).toBe(false);
expect(body.answer).toBe('Synthesized test answer.');
expect(body.protocol_version).toBe(1);
expect(body.cost.input_tokens).toBe(1200);
expect(body.cost.output_tokens).toBe(80);
expect(Array.isArray(body.sources)).toBe(true);
const violations = validateAgainstSchema(body, RESPONSE_SCHEMAS.synthesize);
expect(violations).toEqual([]);
});
});
describe('forget — idempotency + not_found', () => {
it('expires once, reports expired:false on re-forget, not_found on unknown id', async () => {
const r = await callRemote('remember', {
fact: 'fact to forget', provenance: 'test', entity: 'people/forget-test',
});
const id = r.body.id as string;
const first = await callRemote('forget', { id, reason: 'test cleanup' });
expect(first.isError).toBe(false);
expect(first.body.expired).toBe(true);
expect(first.body.reason).toBe('test cleanup');
expect(validateAgainstSchema(first.body, RESPONSE_SCHEMAS.forget)).toEqual([]);
const second = await callRemote('forget', { id });
expect(second.isError).toBe(false);
expect(second.body.expired).toBe(false);
expect(second.body.reason).toBe(null); // omitted optional → null
const missing = await callRemote('forget', { id: '999999999' });
expect(missing.isError).toBe(true);
expect(missing.body.error).toBe('not_found');
expect(missing.body.suggestion.length).toBeGreaterThan(0);
});
it('[ship P1.1] a remote caller in source A cannot forget a fact in source B (returns not_found, not expired)', async () => {
// Register the 'other' tenant source (FK target) then seed a fact in it.
await engine.executeRaw(
`INSERT INTO sources (id, name, config) VALUES ('other', 'other-tenant', '{}'::jsonb)
ON CONFLICT (id) DO NOTHING`,
[],
);
await engine.insertFact(
{ fact: 'cross-source secret fact', kind: 'fact', entity_slug: null, visibility: 'world', source: 'seed' },
{ source_id: 'other' },
);
const otherRows = await engine.executeRaw<{ id: number }>(
`SELECT id FROM facts WHERE source_id = 'other' AND fact = 'cross-source secret fact' LIMIT 1`,
[],
);
const foreignId = String(otherRows[0].id);
// Remote caller scoped to 'default' tries to forget the 'other'-source id.
const res = await dispatchToolCall(engine, 'forget', { id: foreignId }, {
remote: true, takesHoldersAllowList: ['world'], sourceId: 'default',
});
const body = JSON.parse(res.content[0].text);
expect(res.isError).toBe(true);
expect(body.error).toBe('not_found'); // no cross-source existence leak
// And the foreign fact is STILL active (not expired by the cross-source call).
const stillActive = await engine.executeRaw<{ expired_at: Date | null }>(
`SELECT expired_at FROM facts WHERE id = $1`,
[otherRows[0].id],
);
expect(stillActive[0].expired_at).toBe(null);
});
it('[ship P1.1] a remote caller cannot forget a private fact (world-only)', async () => {
const r = await callRemote('remember', {
fact: 'private fact remote cannot forget', provenance: 'test',
entity: 'people/private-forget-test', visibility: 'private',
});
// remote remember defaults world; force a private one locally instead.
const localRes = await dispatchToolCall(engine, 'remember', {
fact: 'truly private fact', provenance: 'test',
entity: 'people/private-forget-test', visibility: 'private',
}, { remote: false, sourceId: 'default' });
const localId = JSON.parse(localRes.content[0].text).id as string;
const res = await dispatchToolCall(engine, 'forget', { id: localId }, {
remote: true, takesHoldersAllowList: ['world'], sourceId: 'default',
});
const body = JSON.parse(res.content[0].text);
expect(res.isError).toBe(true);
expect(body.error).toBe('not_found'); // remote can't reach a private fact
void r;
});
});
describe('writeSingleFact — supersession rule [X1] + degraded dedup', () => {
function installDeterministicEmbedder() {
// The schema's facts.embedding column is the init-time default dim (1536);
// test vectors must match or pgvector's CheckExpectedDim rejects the row.
const DIM = 1536;
configureGateway({
embedding_model: 'openai:text-embedding-3-small',
embedding_dimensions: DIM,
env: { OPENAI_API_KEY: 'sk-test-deterministic' },
});
// The seam replaces the AI SDK's embedMany({ model, values }) call.
__setEmbedTransportForTests((async (opts: { values: string[] }) => ({
embeddings: opts.values.map(t => {
// Same vector for the SUPERSEDE-PAIR family (cosine 1.0 — above the
// dedup threshold); a distinct deterministic vector otherwise.
const v = new Array(DIM).fill(0);
if (t.includes('SUPERSEDE-PAIR')) v[0] = 1;
else for (let i = 0; i < 16; i++) v[i] = ((t.charCodeAt(i % t.length) % 13) + 1) / 13;
return v;
}),
})) as never);
}
it('near-duplicate with changed text and same kind SUPERSEDES; identical text is a duplicate', async () => {
installDeterministicEmbedder();
const a = await writeSingleFact(engine, 'default', {
fact: 'SUPERSEDE-PAIR alice works at acme-example',
provenance: 'test', entity: 'people/supersede-test', kind: 'fact',
});
expect(a.status).toBe('inserted');
expect(a.degraded_dedup).toBe(false);
const dup = await writeSingleFact(engine, 'default', {
fact: 'SUPERSEDE-PAIR alice works at acme-example',
provenance: 'test', entity: 'people/supersede-test', kind: 'fact',
});
expect(dup.status).toBe('duplicate');
expect(dup.id).toBe(a.id);
const updated = await writeSingleFact(engine, 'default', {
fact: 'SUPERSEDE-PAIR alice LEFT acme-example, now at widget-co',
provenance: 'test', entity: 'people/supersede-test', kind: 'fact',
});
expect(updated.status).toBe('superseded');
expect(updated.id).not.toBe(a.id);
const rows = await engine.executeRaw<{ id: number; superseded_by: number | null }>(
`SELECT id, superseded_by FROM facts WHERE id = $1`, [a.id],
);
expect(rows[0].superseded_by).toBe(updated.id);
});
it('reports degraded_dedup when no embedding provider is configured', async () => {
const r = await writeSingleFact(engine, 'default', {
fact: 'a fact written with no embedding provider',
provenance: 'test', entity: 'people/degraded-test',
});
expect(r.status).toBe('inserted');
expect(r.degraded_dedup).toBe(true);
});
});
describe('conformance runner — negative self-test [F3]', () => {
function lyingClient(corrupt: (verb: string, body: Record<string, unknown>) => Record<string, unknown>): ConformanceClient {
return {
listTools: async () =>
VERB_NAMES.map(name => ({
name,
description: name === 'synthesize' ? '[EXPENSIVE / SLOW] x' : `MEMORY VERB (v1): ${name}`,
})),
callTool: async (name, params) => {
const res = await dispatchToolCall(engine, name, params, {
remote: true, takesHoldersAllowList: ['world'], sourceId: 'default',
});
const body = JSON.parse(res.content[0].text);
const mutated = res.isError ? body : corrupt(name, body);
return { isError: res.isError, text: JSON.stringify(mutated) };
},
};
}
it('a certifier that cannot fail certifies nothing: missing fields, bad enums, wrong id types all flag', async () => {
// Mutation 1: remember drops the required `status` field.
const r1 = await runConformance(
lyingClient((verb, body) => (verb === 'remember' ? (({ status: _s, ...rest }) => rest)(body as { status?: unknown } & Record<string, unknown>) : body)),
{ marker: 'neg1' },
);
expect(r1.ok).toBe(false);
// Mutation 2: remember returns an out-of-enum status.
const r2 = await runConformance(
lyingClient((verb, body) => (verb === 'remember' ? { ...body, status: 'absorbed' } : body)),
{ marker: 'neg2' },
);
expect(r2.ok).toBe(false);
// Mutation 3: recall re-types fact_id to a number (the opaque-string mandate [T4]).
const r3 = await runConformance(
lyingClient((verb, body) => {
if (verb !== 'recall' || !Array.isArray((body as { facts?: unknown[] }).facts)) return body;
return {
...body,
facts: (body.facts as Array<Record<string, unknown>>).map(f => ({ ...f, fact_id: Number(f.fact_id) })),
};
}),
{ marker: 'neg3' },
);
expect(r3.ok).toBe(false);
// Honest server passes (sanity: the failures above are the mutations' doing).
const honest = await runConformance(lyingClient((_v, b) => b), { marker: 'pos1' });
const failures = honest.results.filter(r => r.status === 'fail');
expect(failures).toEqual([]);
});
});
describe('fixture mirror + surface invariants', () => {
it('test/fixtures/memory-verbs/cases.json matches the embedded fixture module (BrainBench seed drift guard)', () => {
const onDisk = JSON.parse(readFileSync(join(import.meta.dir, 'fixtures/memory-verbs/cases.json'), 'utf-8'));
expect(onDisk).toEqual(JSON.parse(JSON.stringify(CONFORMANCE_CASES)));
});
it('exactly five ops carry verb: true and they match VERB_NAMES', async () => {
const { operations } = await import('../src/core/operations.ts');
const verbs = operations.filter(o => o.verb === true).map(o => o.name).sort();
expect(verbs).toEqual([...VERB_NAMES].sort());
});
});
test/oauth.test.ts
+12 -2
View File
@@ -894,13 +894,23 @@ describe('operation scope annotations', () => {
}
});
test('mutating operations are write/admin/sources_admin/users_admin/agent scoped', () => {
test('mutating operations are write/admin/sources_admin/users_admin/agent scoped unless remote-gated', () => {
const { operations } = require('../src/core/operations.ts');
// #2598, same allowlist as test/operations-trust-boundary.test.ts: think
// is read-scoped for OAuth/MCP because its handler forces save/take off
// for remote callers before persistence (pinned by
// test/takes-mcp-allowlist.serial.test.ts); local CLI can still persist.
const remoteReadOnlyMutatingOps = new Set(['think']);
for (const op of operations) {
if (op.mutating) {
if (remoteReadOnlyMutatingOps.has(op.name)) {
expect(op.scope, `${op.name} remote-gated mutating op should be read-scoped`).toBe('read');
continue;
}
// v0.28: sources_admin permits sources_add / sources_remove (mutating
// sources, not pages); read scope is the only thing too narrow for
// any mutating op. v0.38: 'agent' is a mutating-axis scope for
// a mutating op unless its remote path forces persistence off before
// the handler writes. v0.38: 'agent' is a mutating-axis scope for
// submit_agent (creates jobs, spends money, but contained by bindings).
expect(
['write', 'admin', 'sources_admin', 'users_admin', 'agent'],
test/operations-descriptions.test.ts
+22
View File
@@ -109,6 +109,28 @@ describe('v0.29 — redirect hints on existing ops', () => {
});
});
describe('#2416 — concept/landscape routing between search and query', () => {
test('search describes the cheap-hybrid default, not the keyword-era model', () => {
expect(SEARCH_DESCRIPTION).toContain("Cheap hybrid search");
expect(SEARCH_DESCRIPTION).toContain("no LLM expansion");
expect(SEARCH_DESCRIPTION).not.toContain("Keyword search using full-text search");
});
test('search declares the completeness boundary and both escape routes', () => {
expect(SEARCH_DESCRIPTION).toContain("NOT proof of coverage");
expect(SEARCH_DESCRIPTION).toContain("landscape");
expect(SEARCH_DESCRIPTION).toContain("list_pages");
});
test('query owns concept/landscape questions but does not oversell coverage', () => {
expect(QUERY_DESCRIPTION).toContain("landscape");
expect(QUERY_DESCRIPTION).toContain("expansion recovers synonym");
expect(QUERY_DESCRIPTION).toContain("Still top-K");
expect(QUERY_DESCRIPTION).toContain("list_pages");
expect(QUERY_DESCRIPTION).toContain("cheaper");
});
});
describe('v0.29 — subagent allow-list', () => {
test('includes get_recent_salience and find_anomalies', () => {
expect(BRAIN_TOOL_ALLOWLIST.has('get_recent_salience')).toBe(true);
test/operations-trust-boundary.test.ts
+9
View File
@@ -86,8 +86,17 @@ describe('operations contract — every op has scope + correct mutability shape'
'users_admin',
'agent',
]);
// Remote-gated exception (#2598, same allowlist as test/oauth.test.ts):
// `think` is read-scoped for OAuth/MCP because its handler forces
// save/take OFF for remote callers before persistence — pinned by
// test/takes-mcp-allowlist.serial.test.ts. Local CLI can still persist.
const REMOTE_READ_ONLY_MUTATING_OPS = new Set(['think']);
for (const op of operations) {
if (op.mutating === true) {
if (REMOTE_READ_ONLY_MUTATING_OPS.has(op.name)) {
expect(op.scope, `remote-gated mutating op "${op.name}" should be read-scoped`).toBe('read');
continue;
}
expect(
WRITE_CLASS_SCOPES.has(op.scope ?? 'read'),
`mutating op "${op.name}" has read-tier scope "${op.scope}"; expected one of ${[...WRITE_CLASS_SCOPES].join('/')}`,
test/query-intent-concept.test.ts
+88
View File
@@ -0,0 +1,88 @@
/**
* #2416 — concept-shaped query detection (CLI nudge toward `query`).
*
* The classifier is tuned to favor false-negatives (silence) over
* false-positives (noise): the cost of a missed nudge is zero; the cost of
* nagging on an exact-token lookup is trust erosion. The non-collision block
* pins the deliberate cue exclusions — "who are the" (find_experts) and bare
* "anything…" (salience ops) must NEVER trigger the query nudge.
*/
import { describe, test, expect } from 'bun:test';
import { looksConceptShaped, conceptNudge } from '../src/core/search/query-intent.ts';
describe('#2416 — looksConceptShaped: concept/landscape queries → true', () => {
const CONCEPT_SHAPED = [
'all the companies that do offshore wind',
'every project that uses pgvector',
'find all startups doing agent memory',
'list everything about vector databases',
'everything related to embedding pricing',
'the landscape of agent memory startups',
'the ecosystem of MCP servers',
'which funds have invested in climate tech',
'show all notes that mention fundraising strategy',
];
for (const q of CONCEPT_SHAPED) {
test(`true: "${q}"`, () => {
expect(looksConceptShaped(q)).toBe(true);
});
}
});
describe('#2416 — looksConceptShaped: exact-token / entity / quoted → false', () => {
const NOT_CONCEPT_SHAPED = [
// bare tokens / proper-noun lookups (short-query guard)
'stripe',
'Series A',
'acme-example',
// quoted phrase — exact-match intent
'find all notes with "offshore wind"',
// slug-like token
'all the pages that link to widget-co-seed',
// entity lookups (classifyQueryIntent === 'entity')
'who is alice from acme',
'tell me about widget co',
// plain questions without a fuzzy-quantifier cue
'how does the embed backfill work',
'meeting notes from tuesday',
];
for (const q of NOT_CONCEPT_SHAPED) {
test(`false: "${q}"`, () => {
expect(looksConceptShaped(q)).toBe(false);
});
}
});
describe('#2416 — cue non-collision with other routers', () => {
test('"who are the …" stays silent (owned by find_experts)', () => {
expect(looksConceptShaped('who are the ML people in my network')).toBe(false);
});
test('bare "anything …" stays silent (owned by salience ops)', () => {
expect(looksConceptShaped('anything notable lately')).toBe(false);
expect(looksConceptShaped('anything crazy happening in my brain lately?')).toBe(false);
});
});
describe('#2416 — conceptNudge message', () => {
test('returns null for non-concept queries', () => {
expect(conceptNudge('stripe')).toBeNull();
expect(conceptNudge('who is alice from acme')).toBeNull();
});
test('returns a single-line hint naming `gbrain query` and the completeness caveat', () => {
const msg = conceptNudge('all the companies that do offshore wind');
expect(msg).not.toBeNull();
expect(msg!).toContain('gbrain query');
expect(msg!).toContain('all the companies that do offshore wind');
expect(msg!).toContain('not proof of completeness');
expect(msg!.includes('\n')).toBe(false);
});
test('truncates long queries in the copy-paste suggestion', () => {
const long = 'all the companies that are doing something with autonomous underwater drone inspection services';
const msg = conceptNudge(long)!;
expect(msg).toContain('...');
expect(msg.length).toBeLessThan(320);
});
});
test/register-client-source-normalize.test.ts
+96
View File
@@ -0,0 +1,96 @@
/**
* normalizeSourceInput / normalizeFederatedReadInput contract.
*
* The `/admin/api/register-client` HTTP endpoint historically hardcoded
* source_id='default' (and federated_read=[source_id]) — only the CLI
* (`--source` / `--federated-read`) could bind a client to a non-default
* brain source. These two normalizers let the HTTP endpoint accept the same
* inputs from the request body while preserving the historical default when
* the fields are omitted.
*
* Hermetic — pure-function unit tests, no engine, no HTTP.
*/
import { describe, test, expect } from 'bun:test';
import { normalizeSourceInput, normalizeFederatedReadInput } from '../src/core/source-id.ts';
describe('normalizeSourceInput', () => {
test('undefined → "default" (backward compat)', () => {
expect(normalizeSourceInput(undefined)).toBe('default');
});
test('null → "default" (backward compat)', () => {
expect(normalizeSourceInput(null)).toBe('default');
});
test('valid source_id passes through', () => {
expect(normalizeSourceInput('mind-agent-brain')).toBe('mind-agent-brain');
});
test('single-char source_id is valid', () => {
expect(normalizeSourceInput('a')).toBe('a');
});
test('invalid source_id (underscore) throws', () => {
expect(() => normalizeSourceInput('mind_agent_brain')).toThrow(/Invalid source_id/);
});
test('invalid source_id (uppercase) throws', () => {
expect(() => normalizeSourceInput('Mind')).toThrow(/Invalid source_id/);
});
test('invalid source_id (edge hyphen) throws', () => {
expect(() => normalizeSourceInput('-mind')).toThrow(/Invalid source_id/);
});
test('non-string (number) throws', () => {
expect(() => normalizeSourceInput(42)).toThrow(/Invalid source_id/);
});
});
describe('register-client route wiring (structural)', () => {
test('normalized source + federatedRead reach registerClientManual in the right positions', () => {
// The unit tests above prove the normalizers; this pins the ROUTE —
// a transposition of the two new positional args (or a regression to
// the hardcoded 'default') would pass every unit test and still ship
// clients bound to the wrong source.
const { readFileSync } = require('fs');
const src = readFileSync(new URL('../src/commands/serve-http.ts', import.meta.url), 'utf-8');
expect(src).toContain('sourceId = normalizeSourceInput(source)');
expect(src).toContain('federatedReadIds = normalizeFederatedReadInput(federatedRead)');
expect(src).toMatch(/registerClientManual\(\s*name,\s*grants,\s*scopeString,\s*uris,\s*sourceId,\s*federatedReadIds,\s*validatedAuthMethod/);
});
});
describe('normalizeFederatedReadInput', () => {
test('undefined → undefined (let registerClientManual default to [sourceId])', () => {
expect(normalizeFederatedReadInput(undefined)).toBeUndefined();
});
test('null → undefined', () => {
expect(normalizeFederatedReadInput(null)).toBeUndefined();
});
test('valid single-element array passes through', () => {
expect(normalizeFederatedReadInput(['mind-agent-brain'])).toEqual(['mind-agent-brain']);
});
test('valid multi-element array passes through (read across both sources)', () => {
expect(normalizeFederatedReadInput(['default', 'mind-agent-brain'])).toEqual([
'default',
'mind-agent-brain',
]);
});
test('empty array throws (ambiguous — omit the field instead)', () => {
expect(() => normalizeFederatedReadInput([])).toThrow(/Invalid federatedRead/);
});
test('non-array (string) throws', () => {
expect(() => normalizeFederatedReadInput('default')).toThrow(/Invalid federatedRead/);
});
test('array with an invalid source_id element throws', () => {
expect(() => normalizeFederatedReadInput(['default', 'bad_id'])).toThrow(/Invalid source_id/);
});
});
test/repair-jsonb.test.ts
+7 -3
View File
@@ -19,9 +19,10 @@ describe('repairJsonb — PGLite short-circuit', () => {
});
expect(result.engine).toBe('pglite');
expect(result.total_repaired).toBe(0);
// All 5 columns reported: pages.frontmatter, raw_data.data,
// ingest_log.pages_updated, files.metadata, page_versions.frontmatter.
expect(result.per_target.length).toBe(5);
// All 8 columns reported: 5 from the v0.12.0 wave + 3 from the v0.16.0
// subagent_* wave (added when persistMessage / persistToolExec* was
// identified as a second double-encode site).
expect(result.per_target.length).toBe(8);
for (const t of result.per_target) {
expect(t.rows_repaired).toBe(0);
}
@@ -32,6 +33,9 @@ describe('repairJsonb — PGLite short-circuit', () => {
'page_versions.frontmatter',
'pages.frontmatter',
'raw_data.data',
'subagent_messages.content_blocks',
'subagent_tool_executions.input',
'subagent_tool_executions.output',
]);
});
});
test/schema-bootstrap-coverage.test.ts
+97 -13
View File
@@ -171,6 +171,13 @@ const REQUIRED_BOOTSTRAP_COVERAGE: ForwardReference[] = [
// v121 — referenced by the timeline event lookup and dedup indexes before
// the numbered migration can add the column on an existing brain.
{ kind: 'column', table: 'timeline_entries', column: 'event_page_id' },
// v7-era — surfaced by the #2626-class scanner sweep: both columns are
// migration-added (v7) AND referenced by blob indexes
// (`idx_minion_jobs_timeout` partial on timeout_at, the partial UNIQUE
// `uniq_minion_jobs_idempotency` on idempotency_key). A pre-v7 minion_jobs
// wedges the blob replay exactly like the v121 incident.
{ kind: 'column', table: 'minion_jobs', column: 'timeout_at' },
{ kind: 'column', table: 'minion_jobs', column: 'idempotency_key' },
];
test('applyForwardReferenceBootstrap covers every forward reference declared in REQUIRED_BOOTSTRAP_COVERAGE', async () => {
@@ -261,6 +268,13 @@ test('applyForwardReferenceBootstrap covers every forward reference declared in
DROP INDEX IF EXISTS idx_timeline_event_page;
ALTER TABLE timeline_entries DROP CONSTRAINT IF EXISTS timeline_entries_event_page_id_fkey;
ALTER TABLE timeline_entries DROP COLUMN IF EXISTS event_page_id;
-- v7 minion_jobs strip (#2626 class sweep): timeout_at + idempotency_key
-- are migration-added and blob-indexed; strip so bootstrap must re-add.
DROP INDEX IF EXISTS idx_minion_jobs_timeout;
DROP INDEX IF EXISTS uniq_minion_jobs_idempotency;
ALTER TABLE minion_jobs DROP COLUMN IF EXISTS timeout_at;
ALTER TABLE minion_jobs DROP COLUMN IF EXISTS idempotency_key;
`);
// Note: we don't strip sources.archived* here because they're inline in the
@@ -346,6 +360,14 @@ test('after bootstrap, PGLITE_SCHEMA_SQL replays without crashing on missing for
DROP INDEX IF EXISTS idx_timeline_event_page;
ALTER TABLE timeline_entries DROP CONSTRAINT IF EXISTS timeline_entries_event_page_id_fkey;
ALTER TABLE timeline_entries DROP COLUMN IF EXISTS event_page_id;
-- v7 minion_jobs strip (#2626 class sweep): the SCHEMA_SQL replay would
-- crash on idx_minion_jobs_timeout / uniq_minion_jobs_idempotency
-- without the bootstrap re-adding these migration-added columns.
DROP INDEX IF EXISTS idx_minion_jobs_timeout;
DROP INDEX IF EXISTS uniq_minion_jobs_idempotency;
ALTER TABLE minion_jobs DROP COLUMN IF EXISTS timeout_at;
ALTER TABLE minion_jobs DROP COLUMN IF EXISTS idempotency_key;
`);
// Bootstrap, then schema replay. Either step crashing fails the test.
@@ -599,16 +621,72 @@ function parseAlterAddColumns(sql: string): Array<{ table: string; column: strin
return result;
}
/**
* The coverage predicate for blob index-column references, extracted so the
* v121-regression unit test below can exercise it with synthetic inputs.
*
* v0.42.58 (#2626 class): CREATE TABLE presence must NOT count as coverage
* for a column that ANY migration also adds via ALTER TABLE ADD COLUMN. The
* migration's existence proves pre-existing tables can lack the column, and
* on those brains `CREATE TABLE IF NOT EXISTS` no-ops — so the blob's
* CREATE INDEX crashes initSchema before runMigrations can help. For such
* columns, only an applyForwardReferenceBootstrap ALTER counts. This is
* exactly how `timeline_entries.event_page_id` (v121, Life Chronicle) shipped
* a P0 upgrade wedge past the old predicate: it was in the current CREATE
* TABLE body, so the check passed while every pre-v121 brain wedged.
*/
function buildIndexRefCoveragePredicate(
tableColumns: Map<string, Set<string>>,
bootstrapAdds: Array<{ table: string; column: string }>,
migrationAddedKeys: Set<string>,
): (table: string, column: string) => boolean {
return (table: string, column: string): boolean => {
const inBootstrap = bootstrapAdds.some(a => a.table === table && a.column === column);
if (inBootstrap) return true;
// Migration-added columns are forward references by definition —
// CREATE TABLE presence is exactly the mask that hid the v121 wedge.
if (migrationAddedKeys.has(`${table}.${column}`)) return false;
const cols = tableColumns.get(table);
return Boolean(cols && cols.has(column));
};
}
test('buildIndexRefCoveragePredicate: CREATE TABLE presence does not mask migration-added columns (v121 regression shape)', () => {
const tableColumns = new Map([['timeline_entries', new Set(['id', 'event_page_id'])]]);
const migrationAdded = new Set(['timeline_entries.event_page_id']);
// The exact pre-fix v121 shape: column in CREATE TABLE, added by migration,
// NO bootstrap probe → must be UNCOVERED (old predicate said covered).
const withoutProbe = buildIndexRefCoveragePredicate(tableColumns, [], migrationAdded);
expect(withoutProbe('timeline_entries', 'event_page_id')).toBe(false);
// Plain blob-native column (not migration-added) stays covered by CREATE TABLE.
expect(withoutProbe('timeline_entries', 'id')).toBe(true);
// With the bootstrap probe present, the same column is covered.
const withProbe = buildIndexRefCoveragePredicate(
tableColumns,
[{ table: 'timeline_entries', column: 'event_page_id' }],
migrationAdded,
);
expect(withProbe('timeline_entries', 'event_page_id')).toBe(true);
});
test('every CREATE INDEX column in PGLITE_SCHEMA_SQL is covered by CREATE TABLE or bootstrap (A2 static check)', async () => {
// The structural test that closes the 11-incident wedge class. Static
// contract: every column referenced by a CREATE INDEX in PGLITE_SCHEMA_SQL
// must be either (a) declared in the current CREATE TABLE body, or
// (b) added by `applyForwardReferenceBootstrap` in pglite-engine.ts.
// must be either (a) declared in the current CREATE TABLE body AND not
// added by any migration (see buildIndexRefCoveragePredicate — migration-
// added columns are forward references even when the CREATE TABLE body has
// them), or (b) added by `applyForwardReferenceBootstrap` in
// pglite-engine.ts.
//
// Codex outside-voice review caught the 11th wedge: composite-index second
// columns (`provider_id` in `(job_id, provider_id)`) are forward references
// that earlier extractors missed. This parser walks the full column list
// of every index — composite or not — and asserts each one is covered.
// The 12th wedge (v121 `timeline_entries.event_page_id`, #2626 #2594 #2579
// #2537 #2536) slipped through because CREATE TABLE presence masked the
// forward reference; the predicate now cross-references MIGRATIONS.
//
// Self-updating: when a future migration adds a CREATE INDEX in
// PGLITE_SCHEMA_SQL on a column that bootstrap doesn't yet provide, this
@@ -616,6 +694,7 @@ test('every CREATE INDEX column in PGLITE_SCHEMA_SQL is covered by CREATE TABLE
const { readFileSync } = await import('fs');
const { resolve: resolvePath } = await import('path');
const { PGLITE_SCHEMA_SQL } = await import('../src/core/pglite-schema.ts');
const { extractAddedColumnsFromMigrations } = await import('./helpers/extract-added-columns.ts');
const enginePath = resolvePath(process.cwd(), 'src/core/pglite-engine.ts');
const engineSrc = readFileSync(enginePath, 'utf-8');
@@ -623,21 +702,23 @@ test('every CREATE INDEX column in PGLITE_SCHEMA_SQL is covered by CREATE TABLE
const tableColumns = parseBaseTableColumns(PGLITE_SCHEMA_SQL);
const indexRefs = parseIndexColumnReferences(PGLITE_SCHEMA_SQL);
const bootstrapAdds = parseAlterAddColumns(engineSrc);
const migrationAddedKeys = new Set(
extractAddedColumnsFromMigrations().map(a => `${a.table}.${a.column}`),
);
// Build the "covered" set: for each (table, column) pair, true iff it's in
// the table's CREATE TABLE columns OR added by an ALTER TABLE in the
// bootstrap function.
const covered = (table: string, column: string): boolean => {
const cols = tableColumns.get(table);
if (cols && cols.has(column)) return true;
return bootstrapAdds.some(a => a.table === table && a.column === column);
};
const covered = buildIndexRefCoveragePredicate(tableColumns, bootstrapAdds, migrationAddedKeys);
// Sanity checks: parser caught the codex case AND bootstrap provides it.
expect(indexRefs).toContainEqual({ table: 'subagent_messages', column: 'provider_id' });
expect(bootstrapAdds).toContainEqual({ table: 'subagent_messages', column: 'provider_id' });
expect(covered('subagent_messages', 'provider_id')).toBe(true);
// Direct pin of the v121 incident: the column is migration-added, blob-
// indexed, and MUST be bootstrap-covered.
expect(migrationAddedKeys.has('timeline_entries.event_page_id')).toBe(true);
expect(indexRefs.some(r => r.table === 'timeline_entries' && r.column === 'event_page_id')).toBe(true);
expect(bootstrapAdds).toContainEqual({ table: 'timeline_entries', column: 'event_page_id' });
// The actual contract: every index column reference must be covered.
const uncovered: Array<{ table: string; column: string }> = [];
for (const ref of indexRefs) {
@@ -650,10 +731,13 @@ test('every CREATE INDEX column in PGLITE_SCHEMA_SQL is covered by CREATE TABLE
const list = uncovered.map(u => ` ${u.table}.${u.column}`).join('\n');
throw new Error(
`PGLITE_SCHEMA_SQL has ${uncovered.length} CREATE INDEX column reference(s) ` +
`that are neither in the table's CREATE TABLE body nor added by ` +
`applyForwardReferenceBootstrap:\n${list}\n\n` +
`that are not safely covered (in the CREATE TABLE body AND not migration-added, ` +
`or added by applyForwardReferenceBootstrap):\n${list}\n\n` +
`Fix: extend applyForwardReferenceBootstrap in src/core/pglite-engine.ts ` +
`(and the matching Postgres engine) with the missing ALTER TABLE ADD COLUMN.`,
`(and the matching Postgres engine) with the missing ALTER TABLE ADD COLUMN. ` +
`A column that is BOTH in the blob's CREATE TABLE AND added by a migration ` +
`is a forward reference for pre-existing tables — CREATE TABLE presence ` +
`does not cover it (that mask shipped the v121 upgrade wedge).`,
);
}
}, 30000);
test/schema-pack-unify-types-handler.test.ts
+18
View File
@@ -156,3 +156,21 @@ describe('runUnifyTypes', () => {
});
});
});
// #1575 — the jobs worker registration must honor the handler's documented
// dry-run default. `apply: data.apply ?? true` made the canonical operator
// invocation (`gbrain jobs submit unify-types --allow-protected --params
// '{"target_pack":...}'`) destructively retype pages by default while
// UnifyTypesOpts.apply documents 'Default false (dry-run)'. Structural pin —
// the worker source must default apply to false.
import { readFileSync } from 'fs';
describe('#1575 unify-types worker dry-run default', () => {
it('jobs.ts worker registration defaults apply to false, matching the handler contract', () => {
const jobsSource = readFileSync(new URL('../src/commands/jobs.ts', import.meta.url), 'utf-8');
const workerBlock = jobsSource.slice(jobsSource.indexOf("worker.register('unify-types'"));
const registration = workerBlock.slice(0, workerBlock.indexOf('});'));
expect(registration).toContain('apply: data.apply ?? false');
expect(registration).not.toContain('apply: data.apply ?? true');
});
});
test/scripts/run-unit-parallel.test.ts
+144 -1
View File
@@ -72,7 +72,10 @@ function runWrapper(extraArgs: string[] = []): { code: number; stdout: string; s
const result = spawnSync(
'bash',
[join(TMPROOT, 'scripts', 'run-unit-parallel.sh'), '--shards', '2', ...extraArgs],
{ cwd: TMPROOT, encoding: 'utf-8', env: { ...process.env } },
// Shard-mechanics tests pin explicit --shards behavior with tiny
// synthetic files; disable mem-adaptation so a RAM-limited runner (CI's
// ~7GB) can't collapse 2 shards -> 1 and break the shard 1/2 expectations.
{ cwd: TMPROOT, encoding: 'utf-8', env: { ...process.env, GBRAIN_TEST_NO_MEM_ADAPT: '1' } },
);
return {
code: result.status ?? -1,
@@ -209,6 +212,8 @@ describe('passing', () => {
HOME: process.env.HOME ?? FROOT,
TMPDIR: process.env.TMPDIR ?? '/tmp',
GBRAIN_TEST_SHARD_TIMEOUT: '300',
// Same rationale as runWrapper: explicit-shard mechanics under test.
GBRAIN_TEST_NO_MEM_ADAPT: '1',
};
});
@@ -257,3 +262,141 @@ describe('failing-on-purpose', () => {
}
});
});
describe('run-unit-parallel.sh OOM rescue lane', () => {
// A fixture that fails WITH the WASM out-of-memory signature on its first
// run (no sentinel file yet) and passes once the sentinel exists — exactly
// the phantom-failure shape: dies under parallel memory pressure, passes
// serially. The runner must (1) detect the signature, (2) re-run the file
// at --max-concurrency 1, (3) exit 0 with an oom_rescued note.
let OROOT: string;
beforeAll(() => {
OROOT = mkdtempSync(join(tmpdir(), 'gbrain-parallel-oom-'));
mkdirSync(join(OROOT, 'scripts'), { recursive: true });
mkdirSync(join(OROOT, 'test'), { recursive: true });
for (const s of ['run-unit-parallel.sh', 'run-unit-shard.sh', 'run-serial-tests.sh']) {
copyFileSync(resolve(REPO_ROOT, 'scripts', s), join(OROOT, 'scripts', s));
chmodSync(join(OROOT, 'scripts', s), 0o755);
}
const passing = `import { describe, it, expect } from 'bun:test';
describe('passing', () => {
it('arithmetic works', () => { expect(1 + 1).toBe(2); });
});`;
const oomOnce = `import { describe, it, expect } from 'bun:test';
import { existsSync, writeFileSync } from 'fs';
describe('oom-once', () => {
it('fails with the WASM OOM signature on first run, passes on retry', () => {
const sentinel = new URL('./oom-sentinel.txt', import.meta.url).pathname;
if (!existsSync(sentinel)) {
writeFileSync(sentinel, 'ran-once');
console.error('Original error: Out of memory');
throw new Error('Out of memory (simulated PGLite WASM connect failure)');
}
expect(1).toBe(1);
});
});`;
writeFileSync(join(OROOT, 'test', 'a-pass.test.ts'), passing);
writeFileSync(join(OROOT, 'test', 'b-oom-once.test.ts'), oomOnce);
});
afterAll(() => {
if (OROOT) rmSync(OROOT, { recursive: true, force: true });
});
function runOom(env: Record<string, string> = {}): { code: number; stdout: string; stderr: string } {
rmSync(join(OROOT, 'test', 'oom-sentinel.txt'), { force: true });
const result = spawnSync(
'bash',
[join(OROOT, 'scripts', 'run-unit-parallel.sh'), '--shards', '2'],
{ cwd: OROOT, encoding: 'utf-8', env: { ...process.env, GBRAIN_TEST_NO_MEM_ADAPT: '1', ...env } },
);
return { code: result.status ?? -1, stdout: result.stdout || '', stderr: result.stderr || '' };
}
it('rescues an OOM-signature failure serially and exits 0 with an oom_rescued note', () => {
const r = runOom();
expect(r.stdout + r.stderr).toContain('OOM rescue pass');
expect(r.stderr).toContain('oom_rescued=');
expect(r.code).toBe(0);
}, 120_000);
it('GBRAIN_TEST_NO_OOM_FALLBACK=1 disables the rescue lane (stays red)', () => {
const r = runOom({ GBRAIN_TEST_NO_OOM_FALLBACK: '1' });
expect(r.code).not.toBe(0);
expect(r.stdout + r.stderr).not.toContain('OOM rescue pass');
}, 120_000);
it('memory-aware sizing is advertised in the banner (mem-ok or mem-adapted)', () => {
// The one test that needs adaptation ON — override the harness-wide
// NO_MEM_ADAPT base (which keeps the shard-mechanics tests deterministic
// on RAM-limited CI runners).
const r = runOom({ GBRAIN_TEST_NO_MEM_ADAPT: '0' });
expect(r.stderr).toMatch(/mem-(ok|adapted)/);
}, 120_000);
it('mixed run: a plain assertion failure stays red even when the OOM phantom rescues green', () => {
// The NON_OOM_FAIL gate — the branch that stops the rescue lane from
// absolving real failures that happened to share a run with phantoms.
const realFail = `import { describe, it, expect } from 'bun:test';
describe('real-failure', () => {
it('expects 1 to equal 2', () => { expect(1).toBe(2); });
});`;
writeFileSync(join(OROOT, 'test', 'c-real-fail.test.ts'), realFail);
try {
const r = runOom();
expect(r.code).not.toBe(0);
} finally {
rmSync(join(OROOT, 'test', 'c-real-fail.test.ts'), { force: true });
}
}, 120_000);
it('a deterministic failure carrying the OOM signature re-fails serially and stays red', () => {
// The oom_rescue_failed lane: signature match queues the file, but the
// serial re-run confirms the failure is real — run must stay red.
const alwaysOom = `import { describe, it } from 'bun:test';
describe('oom-always', () => {
it('always fails with the signature', () => {
console.error('Original error: Out of memory');
throw new Error('Out of memory (deterministic)');
});
});`;
writeFileSync(join(OROOT, 'test', 'd-oom-always.test.ts'), alwaysOom);
try {
const r = runOom();
expect(r.code).not.toBe(0);
expect(r.stderr).toContain('oom_rescue_failed=');
expect(r.stdout + r.stderr).toContain('oom-rescue (serial, confirmed real)');
} finally {
rmSync(join(OROOT, 'test', 'd-oom-always.test.ts'), { force: true });
}
}, 120_000);
});
describe('run-unit-parallel.sh external-kill rescue contract', () => {
// An externally-killed shard (sibling workspace pkill, memory jetsam)
// presents as rc 143/137 well before the shard timeout. Simulating a
// mid-run external kill deterministically in a fixture is flaky, so this
// pins the load-bearing structure instead: the early-death detector, the
// 80%-of-timeout threshold that separates external kills from real wedges,
// and the rescue-queue routing for both the wedged and non-wedged branches.
it('detects early SIGTERM/SIGKILL deaths against the 80% timeout threshold', () => {
const source = readFileSync(PARALLEL_SH_SRC, 'utf-8');
expect(source).toContain('[ "$rc" = "143" ] || [ "$rc" = "137" ]');
expect(source).toContain('$((SHARD_TIMEOUT * 80 / 100))');
expect(source).toContain('shard_external_kill=1');
});
it('routes externally-killed shards into the serial rescue queue, not the red path', () => {
const source = readFileSync(PARALLEL_SH_SRC, 'utf-8');
const killBranches = source.split('shard_external_kill" = "1"').length - 1;
expect(killBranches).toBeGreaterThanOrEqual(2); // wedged + non-wedged branch
expect(source).toContain('KILLED externally after ${s_elapsed}s');
});
it('stamps per-shard start/end epochs so early death is measurable', () => {
const source = readFileSync(PARALLEL_SH_SRC, 'utf-8');
expect(source).toContain('date +%s > "$LOG_DIR/shard-$i.start"');
expect(source).toContain('date +%s > "$LOG_DIR/shard-$i.end"');
});
});
test/takes-mcp-allowlist.serial.test.ts
+6
View File
@@ -17,6 +17,7 @@ import { withoutAnthropicKey } from './helpers/no-anthropic-key.ts';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { dispatchToolCall } from '../src/mcp/dispatch.ts';
import { TAKES_FENCE_BEGIN, TAKES_FENCE_END } from '../src/core/takes-fence.ts';
import { operationsByName } from '../src/core/operations.ts';
let engine: PGLiteEngine;
let alicePageId: number;
@@ -198,6 +199,11 @@ describe('per-token takes-holder allow-list — get_versions body channel', () =
});
describe('think op — read-only on remote callers (Lane D landed)', () => {
test('think is read-scoped for MCP while local persistence remains possible', () => {
expect(operationsByName.think.scope).toBe('read');
expect(operationsByName.think.mutating).toBe(true);
});
test('remote save/take is forced read-only via remote_persisted_blocked flag', async () => {
// Hermetic no-key: neutralize BOTH env var AND ~/.gbrain config key, else a
// configured machine fires a real LLM call and the warning flips to
test/think-max-output-tokens.test.ts
+35 -4
View File
@@ -1,9 +1,10 @@
/**
* Pins `maxOutputTokensFor` — the per-model output-token budget `runThink`
* passes to `client.create`. Thinking-by-default Claude 5 models
* (`anthropic:claude-*-5`) spend a large share of the budget on internal
* reasoning before emitting an answer, so the 4000 default left `think` with
* empty/truncated text. They now get 16000; everything else stays 4000.
* (`anthropic:claude-*-5`) and OpenAI reasoning models (gpt-5 family,
* o-series) spend a large share of the budget on internal reasoning before
* emitting an answer, so the 4000 default left `think` with empty/truncated
* text. They get 16000; everything else stays 4000.
*/
import { describe, test, expect } from 'bun:test';
import { maxOutputTokensFor } from '../src/core/think/index.ts';
@@ -17,12 +18,42 @@ describe('maxOutputTokensFor — thinking-default headroom', () => {
expect(maxOutputTokensFor('anthropic/claude-sonnet-5')).toBe(16000); // slash form
});
test('non-Claude-5 and non-Anthropic keep 4000', () => {
test('OpenAI reasoning models (gpt-5 family, o-series) get 16000', () => {
expect(maxOutputTokensFor('openai:gpt-5')).toBe(16000);
expect(maxOutputTokensFor('openai:gpt-5.2')).toBe(16000);
expect(maxOutputTokensFor('openai:gpt-5.5')).toBe(16000);
expect(maxOutputTokensFor('openai:gpt-5-mini')).toBe(16000);
expect(maxOutputTokensFor('openai:o1')).toBe(16000);
expect(maxOutputTokensFor('openai:o3')).toBe(16000);
expect(maxOutputTokensFor('openai:o4-mini')).toBe(16000);
expect(maxOutputTokensFor('openai/gpt-5.2')).toBe(16000); // slash form
});
test('non-Claude-5 and non-reasoning models keep 4000', () => {
expect(maxOutputTokensFor('anthropic:claude-opus-4-8')).toBe(4000);
expect(maxOutputTokensFor('anthropic:claude-haiku-4-5')).toBe(4000);
expect(maxOutputTokensFor('anthropic:claude-sonnet-4-6')).toBe(4000);
expect(maxOutputTokensFor('anthropic:claude-3-haiku')).toBe(4000);
expect(maxOutputTokensFor('openai:gpt-4o')).toBe(4000);
expect(maxOutputTokensFor('openai:gpt-4o-mini')).toBe(4000);
expect(maxOutputTokensFor('openai:gpt-4.1')).toBe(4000);
// Non-reasoning ChatGPT snapshots of the gpt-5 family stay at 4000.
expect(maxOutputTokensFor('openai:gpt-5-chat-latest')).toBe(4000);
expect(maxOutputTokensFor('openai:gpt-5.2-chat-latest')).toBe(4000);
// Scope is the gpt-5 family + numbered o-series only — other OpenAI
// reasoning-capable ids (e.g. codex-mini-latest) keep the conservative
// default until deliberately added.
expect(maxOutputTokensFor('openai:codex-mini-latest')).toBe(4000);
// Version/name boundaries: `gpt-50` and `o3foo` are not gpt-5 / o3.
expect(maxOutputTokensFor('openai:gpt-50')).toBe(4000);
expect(maxOutputTokensFor('openai:o3foo')).toBe(4000);
// Other providers' reasoning models are out of scope here — the routed
// provider recipe, not this budget, is what changes for them.
expect(maxOutputTokensFor('deepseek:deepseek-reasoner')).toBe(4000);
// Prefix must be the openai provider — a bare model name or another
// provider's gpt-5 spelling doesn't match.
expect(maxOutputTokensFor('o3')).toBe(4000);
expect(maxOutputTokensFor('gpt-5.2')).toBe(4000);
expect(maxOutputTokensFor('openrouter:openai/gpt-5.2')).toBe(4000);
});
});
test/think-pipeline.serial.test.ts
+5 -3
View File
@@ -423,15 +423,17 @@ describe('runThink + persistSynthesis — #1698 never persist empty', () => {
expect(full.synthesisOk).toBe(true);
});
test('opts.stubResponse path never made a real LLM call — usage stays undefined', async () => {
test('opts.stubResponse path never made a real LLM call — usage stays null', async () => {
// Same distinction synthesisOk already makes: opts.stubResponse bypasses
// client.create() entirely, so there is no real usage to report. cost_usd
// must not be computed (and should render as null in --json) when this
// happens, since there is nothing to compute it from.
// happens, since there is nothing to compute it from. Since the [E2]
// MEMORY_VERBS usage-accounting change, "no LLM ran" is spelled `null`
// (the frozen cost-block contract), not `undefined`.
const result = await runThink(engine, {
question: 'stub no usage', stubResponse: { answer: 'has content', citations: [], gaps: [] },
});
expect(result.usage).toBeUndefined();
expect(result.usage).toBeNull();
});
test('pre-existing ThinkResult literal without synthesisOk still persists (back-compat)', async () => {