admin/gbrain
1
0
Fork 0
mirror of https://github.com/garrytan/gbrain.git synced 2026-08-14 08:53:22 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: repo-wide accuracy + MECE reform from the 9-bucket markdown audit

Browse Source 
A code-grounded audit of every markdown file (root, architecture, guides,
mcp, tutorials, docs-root, operations/eval/designs, skills, recipes) followed
by a fix wave with per-bucket ownership. Four classes of change:

Accuracy — every documented command/flag verified against src/ before writing:
dead commands replaced with working ones (pages purge-deleted, jobs watch
--follow, gbrain restore, import-based Obsidian flow, space-separated --scopes,
real thin-client recipes, working isolation verification, real supervisor
restart procedure, curl-based ngrok health check, real minion_mode setter);
count drift fixed with rot-proof phrasing (100+ ops, 50+ bundled skills via
skills/manifest.json, 140+ engine methods, KNOBS_HASH_VERSION pointer instead
of hardcoded versions); stale claims corrected (search-mode defaults, RETRIEVAL
pipeline order incl. autocut, sentinel rules, refusal-list mechanism, engine
snapshot, shard cap 2400s + EXIT-HANG classifier in TESTING.md, latest-stable +
publish-template documented in RELEASING.md as release.yml promises).

MECE — one home per concept, pointers elsewhere: test isolation → TESTING.md;
OAuth registration + --bind/--public-url lore → DEPLOY.md; mode bundles →
guides/search-modes.md (the home the CLAUDE.md dispatcher always promised);
merge contract → schema-packs.md; WAL ladder → ENGINES.md; quiet-hours →
quiet-hours.md; capture taxonomy → entity-detection.md; person-page taxonomy →
compiled-truth.md; brain-first protocol → brain-first-lookup.md; refresh
semantics → refresh-algorithm.md; KEY_FILES.md deduplicated (58 extension
entries merged, one entry per file); infra-layer.md rewritten as a pointer page.

Privacy — placeholder sweep across guides, docs, skills, and recipes per the
iron rule; per-release narration stripped from reference docs (current-state
prose only).

Bootstrap coverage — AGENTS.md pointer, RESOLVER routing row, INSTALL.md path,
tutorial cross-links, keyless-mode sections in spend-controls/headless-install.

skills.lock.json regenerated; llms.txt/llms-full.txt rebuilt. Gates: verify
36/36, typecheck clean, doctor 96/96, skills-integrity + resolver + build-llms
+ config-set + migrations all green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-08-09 17:22:42 -07:00
co-authored by Claude Fable 5
parent 64e70c53a0
commit f94b9ef316
121 changed files with 2327 additions and 1885 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+19 -13
View File
@@ -4,8 +4,16 @@ This is your install + operating protocol. Claude Code reads `./CLAUDE.md` autom
Everyone else (Codex, Cursor, OpenClaw, Aider, Continue, or an LLM fetching via URL):
start here.
> **Becoming someone's persistent personal agent** (identity + memory + private repo)?
> Follow [`BOOTSTRAP_FOR_AGENTS.md`](./BOOTSTRAP_FOR_AGENTS.md) — the `gbrain bootstrap`
> flow — instead of the plain install below, then come back here for the operating
> protocol. Connecting to an EXISTING remote brain from a laptop agent?
> `gbrain connect https://your-host/mcp --token gbrain_xxx --install` (see the MCP
> table in [`README.md`](./README.md)).
## Install (5 min)
<!-- npm-trap + #218 recovery: canonical copy lives in README.md ("Install" warning) — sync edits. -->
1. Install gbrain via Bun (the canonical path):
```bash
curl -fsSL https://bun.sh/install | bash
@@ -26,8 +34,8 @@ start here.
[`./INSTALL_FOR_AGENTS.md`](./INSTALL_FOR_AGENTS.md) Step 3.5 for the
exact ask-the-user protocol. Same banner fires on `gbrain post-upgrade`
for existing users (search modes were added in v0.32.3).
4. Read [`./INSTALL_FOR_AGENTS.md`](./INSTALL_FOR_AGENTS.md) for the full 9-step flow
(API keys, identity, cron, verification).
4. Read [`./INSTALL_FOR_AGENTS.md`](./INSTALL_FOR_AGENTS.md) for the full step-by-step
flow (API keys, identity, cron, verification).
## Read this order
@@ -69,10 +77,10 @@ writing or reviewing an operation, consult `src/core/operations.ts` for the cont
`GBRAIN_CONTRIBUTOR_MODE=1`, then `gbrain eval export --since 7d > base.ndjson`
and `gbrain eval replay --against base.ndjson`. For public benchmark
coverage (LongMemEval, ground-truth scoring), `gbrain eval longmemeval
<dataset.jsonl>` (v0.28.8) runs against an isolated in-memory PGLite
<dataset.jsonl>` runs against an isolated in-memory PGLite
per question — your `~/.gbrain` is never opened. Full guide:
[`docs/eval-bench.md`](./docs/eval-bench.md).
- **Drive the brain to a target health score (v0.36.4.0):** the one-command
- **Drive the brain to a target health score:** the one-command
loop. `gbrain doctor --remediation-plan --json` previews what would be
fixed; `gbrain doctor --remediate --yes --target-score 90 --max-usd 5`
walks a dependency-ordered plan (sync before extract, embed after
@@ -81,22 +89,20 @@ writing or reviewing an operation, consult `src/core/operations.ts` for the cont
keys hit a `max_reachable_score` ceiling and bail with what's missing.
Three phase handlers (synthesize / patterns / consolidate) are
PROTECTED — only trusted local callers can submit them; MCP cannot.
Reference: [`docs/architecture/topologies.md`](./docs/architecture/topologies.md)
and the CHANGELOG entry for v0.36.4.0.
- **Track a founder/company over time (v0.35.7):** when an entity has
Reference: [`docs/architecture/topologies.md`](./docs/architecture/topologies.md).
- **Track a founder/company over time:** when an entity has
typed metric claims in its `## Facts` fence (`metric: mrr`, `value: 50000`,
`unit: USD`, `period: monthly` columns), run
`gbrain eval trajectory <entity-slug>` for the chronological history
with regressions auto-flagged, or `gbrain founder scorecard <entity-slug>`
for a four-signal JSON rollup (claim_accuracy / consistency /
growth_trajectory / red_flags). MCP op `find_trajectory` exposes the
same data — read scope, visibility-filtered for remote callers. **v0.40.2.0:**
`gbrain think` now uses this substrate automatically on temporal /
same data — read scope, visibility-filtered for remote callers.
`gbrain think` uses this substrate automatically on temporal /
knowledge_update intent (default ON; flip `think.trajectory_enabled=false`
to opt out). Migration v82 added `facts.event_type` so non-metric event
rows (`meeting`, `job_change`, `location_change`) ride through the same
pipeline; pass `kind: 'event'` or `'all'` to `find_trajectory` to query
them.
to opt out). Non-metric event rows (`meeting`, `job_change`,
`location_change`) ride through the same pipeline via `facts.event_type`;
pass `kind: 'event'` or `'all'` to `find_trajectory` to query them.
- **Everything else:** [`./llms.txt`](./llms.txt) is the full documentation map.
[`./llms-full.txt`](./llms-full.txt) is the same map with core docs inlined for
single-fetch ingestion.
CLAUDE.md
+8 -6
View File
@@ -38,7 +38,7 @@ mount, CEO-class with multiple team brains) and
## Architecture
Contract-first: `src/core/operations.ts` defines ~90 shared operations (v0.29 adds `get_recent_salience`, `find_anomalies`, `get_recent_transcripts`; v0.42.43.0 adds `volunteer_context` — push-based context, see `docs/guides/push-context.md`). CLI and MCP
Contract-first: `src/core/operations.ts` defines 100+ shared operations (including `volunteer_context` — push-based context, see `docs/guides/push-context.md`). CLI and MCP
server are both generated from this single source. Engine factory (`src/core/engine-factory.ts`)
dynamically imports the configured engine (`'pglite'` or `'postgres'`). Skills are fat
markdown files (tool-agnostic, work with both CLI and plugin contexts).
@@ -191,9 +191,10 @@ Mismatches (tokenmax+Haiku, conservative+Opus) waste capacity differently
expensive one.
tokenmax adds ~\$1.50 per 1K queries in Haiku expansion calls on top of
the matrix (\$15/mo @ 10K). Cache hits cut all numbers ~50%. **The cost
picker copy in `gbrain init` carries the same matrix verbatim** — update
both when refreshing.
the matrix (\$15/mo @ 10K). Cache hits cut all numbers ~50%. **The matrix
has three verbatim homes: this section, the `gbrain init` picker copy
(`src/commands/init-mode-picker.ts`), and `INSTALL_FOR_AGENTS.md` Step
3.5** — update all three when refreshing.
**Per-query math vs real-world spend.** The matrix above is what an
isolated benchmark would measure. Real agent loops with disciplined
@@ -273,8 +274,9 @@ audit trail lives in the source repo's git history.
## Skills
Read the skill files in `skills/` before doing brain operations. GBrain ships 30 skills
organized by `skills/RESOLVER.md` (`AGENTS.md` is also accepted as of v0.19):
Read the skill files in `skills/` before doing brain operations. GBrain ships 50+ skills
(the current list lives in `skills/manifest.json`) organized by `skills/RESOLVER.md`
(`AGENTS.md` is also accepted as of v0.19):
**Original 8 (conformance-migrated):** ingest (thin router), query, maintain, enrich,
briefing, migrate, setup, publish.
CONTRIBUTING.md
+57 -77
View File
@@ -48,7 +48,9 @@ src/
core/
operations.ts Contract-first operation definitions (the foundation)
engine.ts BrainEngine interface
postgres-engine.ts Postgres implementation
engine-factory.ts Engine factory (dynamic import of the configured engine)
postgres-engine.ts Postgres + pgvector implementation
pglite-engine.ts PGLite (embedded Postgres via WASM) implementation
db.ts Connection management + schema loader
import-file.ts Import pipeline (chunk + embed + tags)
types.ts TypeScript types
@@ -59,12 +61,16 @@ src/
supabase-admin.ts Supabase admin API
file-resolver.ts MIME detection + content hashing
migrate.ts Migration helpers
bootstrap/ Agent-bootstrap flow (interview, hooks, repo, verify)
yaml-lite.ts Lightweight YAML parser
chunkers/ 3-tier chunking (recursive, semantic, llm)
search/ Hybrid search (vector, keyword, hybrid, expansion, dedup)
embedding.ts OpenAI embedding service
embedding.ts Embedding service (provider-routed; ZeroEntropy default)
mcp/
server.ts MCP stdio server (generated from operations)
http-transport.ts HTTP MCP transport (OAuth, body caps)
dispatch.ts Op dispatch + scope enforcement + param redaction
rate-limit.ts Rate limiting
schema.sql Postgres DDL
skills/ Fat markdown skills for AI agents
test/ Unit tests (bun test, no DB required)
@@ -77,15 +83,21 @@ test/e2e/ E2E tests (requires DATABASE_URL, real Postgres+pgvect
docs/ Architecture docs
```
Per-file invariants live in `docs/architecture/KEY_FILES.md` — read a file's entry
before editing it.
## Running tests
The canonical reference for test tiers, isolation rules, timing, and the E2E
lifecycle is [`docs/TESTING.md`](docs/TESTING.md). The short version:
```bash
# Inner edit loop (~85s on a Mac dev box, 3700+ unit tests)
# Inner edit loop (~85s on a Mac dev box)
bun run test # parallel 8-shard fan-out + serial post-pass
bun test test/markdown.test.ts # specific unit test
# Pre-push gate (matches what CI runs on shard 1 + typecheck)
bun run verify # privacy + jsonb + progress + test-isolation + wasm + admin-build + resolver + typecheck
# Pre-push gate (19+ parallel checks + typecheck)
bun run verify
# Pre-merge sanity (everything CI runs)
bun run test:full # verify + parallel unit + slow + smart e2e
@@ -103,92 +115,51 @@ DATABASE_URL=postgresql://postgres:postgres@localhost:5434/gbrain_test bun run t
DATABASE_URL=postgresql://... bun run test:e2e
```
Use `bun run verify` before pushing. The guard chain catches: banned fork-name
leaks (`scripts/check-privacy.sh`), `JSON.stringify(x)::jsonb` interpolation
Use `bun run verify` before pushing. It runs 19+ guard checks in parallel
(`scripts/run-verify-parallel.sh`), including: banned fork-name leaks
(`scripts/check-privacy.sh`), `JSON.stringify(x)::jsonb` interpolation
patterns (`scripts/check-jsonb-pattern.sh`), `\r` progress bleed to stdout
(`scripts/check-progress-to-stdout.sh`), test-isolation rule violations
(`scripts/check-test-isolation.sh` — see "Writing tests that survive the parallel
loop" below), silent fallback to recursive chunking in the compiled binary
(`scripts/check-wasm-embedded.sh`), stale admin-dashboard build artifacts
(`scripts/check-admin-build.sh`), and resolver drift on bundled skills
(`bun run check:resolver` — strict-mode `check-resolvable` that exit-1s on any
warning, added in v0.41.14.0 to catch SKILL.md frontmatter ↔ RESOLVER.md drift
before merge). `bun run check:all` runs the full historical sweep including the
trailing-newline and exports-count checks.
(`scripts/check-admin-build.sh`), resolver drift on bundled skills
(`bun run check:resolver`), and typecheck. `bun run check:all` runs the full
historical sweep including the trailing-newline and exports-count checks.
### Writing tests that survive the parallel loop
`bun run test` shards 92+ unit-test files across 8 worker processes. Files in the
same shard share a process, so process-global state leaks between them. Four
lint rules (`scripts/check-test-isolation.sh`, R1-R4) enforce isolation:
`bun run test` shards the unit-test files (1000+) across 8 worker processes.
Files in the same shard share a process, so process-global state leaks between
them. Four lint rules (`scripts/check-test-isolation.sh`, R1–R4) enforce
isolation: no direct `process.env` mutation (use `withEnv()` from
`test/helpers/with-env.ts`), no `mock.module(...)` outside `*.serial.test.ts`,
and every `new PGLiteEngine(` goes inside the canonical `beforeAll` block with
a paired `afterAll(disconnect)`.
| Rule | What it bans | Fix |
|---|---|---|
| **R1** | Direct `process.env.X = ...` mutation | Use `withEnv()` from `test/helpers/with-env.ts`, or rename to `*.serial.test.ts` |
| **R2** | `mock.module(...)` anywhere in the file | Rename to `*.serial.test.ts` |
| **R3** | `new PGLiteEngine(` outside ~50 lines after `beforeAll(` | Use the canonical PGLite block (see below) |
| **R4** | `new PGLiteEngine(` without paired `afterAll(disconnect)` | Add the `afterAll(() => engine.disconnect())` |
**The full rules, the canonical PGLite block, the `withEnv` pattern, and the
`*.serial.test.ts` quarantine policy live in
[`docs/TESTING.md`](docs/TESTING.md#test-isolation-lint-and-helpers)
— read that before writing a new test file.** Files that predate the rules are
listed in `scripts/check-test-isolation.allowlist`; the allow-list MUST shrink
over time — never add new entries.
Canonical PGLite block (R3 + R4 compliant — paste this verbatim):
```ts
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { resetPgliteState } from './helpers/reset-pglite.ts';
let engine: PGLiteEngine;
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
});
afterAll(async () => { await engine.disconnect(); });
beforeEach(async () => { await resetPgliteState(engine); });
```
Env-touching tests:
```ts
import { withEnv } from './helpers/with-env.ts';
test('reads OPENAI_API_KEY', async () => {
await withEnv({ OPENAI_API_KEY: 'sk-test' }, async () => {
expect(loadConfig().openai_key).toBe('sk-test');
});
});
```
`withEnv` saves and restores keys via try/finally including when the callback
throws. Cross-test safe; **NOT** intra-file concurrent-safe (`process.env` is
process-global). Files using `withEnv` stay outside the future
`test.concurrent()` codemod's eligibility filter.
When to quarantine instead of fix: rename to `*.serial.test.ts` if the file
uses `mock.module(...)`, is genuinely env-coupled (module-load env readers +
ESM caching defeat dynamic-import-after-env tricks), or intentionally shares
state across `it()` boundaries. Quarantine count cap: 10 (informational).
Files that violated these rules at the v0.26.7 baseline are listed in
`scripts/check-test-isolation.allowlist`. **The allow-list MUST shrink over
time** ... never add new entries. v0.26.8 (env sweep) and v0.26.9 (PGLite sweep
+ codemod) remove entries as files get fixed.
### Local CI gate (recommended before pushing, v0.23.1+)
### Local CI gate (recommended before pushing)
```bash
bun run ci:local # full gate: gitleaks + unit + ALL 29 E2E files (sequential)
bun run ci:local # full gate: gitleaks + guards/typecheck + 4-shard parallel unit + E2E
bun run ci:local:diff # gate with diff-aware E2E selector
bun run ci:select-e2e # print which E2E files the selector would run
```
`ci:local` spins up `pgvector/pgvector:pg16` + `oven/bun:1` via
`docker-compose.ci.yml`, runs everything PR CI runs plus the full E2E suite, then
tears down. Named volumes keep the install warm across runs (~16-20 min sequential
E2E after the first cold pull). Requires Docker (Docker Desktop, OrbStack, or
Colima) and `gitleaks` on host (`brew install gitleaks`). Override the postgres
host port with `GBRAIN_CI_PG_PORT=5435 bun run ci:local` if 5434 collides.
`ci:local` spins up four pgvector services plus a transaction-mode PgBouncer via
`docker-compose.ci.yml`, runs everything PR CI runs plus the full E2E suite
sharded 4 ways in parallel, then tears down. Named volumes keep the install warm
across runs. Requires Docker (Docker Desktop, OrbStack, or Colima) and `gitleaks`
on host (`brew install gitleaks`). Override the postgres host port with
`GBRAIN_CI_PG_PORT=5435 bun run ci:local` if 5434 collides.
Fail-closed selector: an unmapped `src/` change runs all 29 E2E files. Hand-tune
Fail-closed selector: an unmapped `src/` change runs ALL E2E files. Hand-tune
narrower mappings via `scripts/e2e-test-map.ts`.
### PR-side security checks
@@ -225,7 +196,7 @@ Parity tests (`test/parity.test.ts`) verify CLI/MCP/tools-json stay in sync.
See `docs/ENGINES.md` for the full guide. In short:
1. Create `src/core/myengine-engine.ts` implementing `BrainEngine`
2. Add to engine factory in `src/core/engine.ts`
2. Add to the engine factory in `src/core/engine-factory.ts`
3. Run the test suite against your engine
4. Document in `docs/`
@@ -310,7 +281,7 @@ NDJSON wire format is documented in
[`docs/eval-capture.md`](./docs/eval-capture.md).
For public benchmark coverage on top of replay, `gbrain eval longmemeval
<dataset.jsonl>` (v0.28.1) runs LongMemEval against gbrain's hybrid
<dataset.jsonl>` runs LongMemEval against gbrain's hybrid
retrieval. One in-memory PGLite per question, runtime-enumerated
`TRUNCATE` between questions, ground-truth scoring via LongMemEval's
published `evaluate_qa.py`. Use it alongside replay when changes affect
@@ -319,9 +290,18 @@ regressions on YOUR queries, LongMemEval catches them on a public set the
benchmark community already cites. See the "Public benchmarks: LongMemEval"
section in [`docs/eval-bench.md`](./docs/eval-bench.md).
## Shipping
Releases go through the `/ship` skill, never hand-rolled. The full release +
contributor process (CHANGELOG voice, version-locations sync, PR conventions,
community-PR-wave workflow) lives in [`docs/RELEASING.md`](docs/RELEASING.md).
Community PRs are batched into release waves rather than merged one-by-one;
contributor attribution stays attached via `Co-Authored-By:` trailers and every
accepted contribution is credited in `CHANGELOG.md`.
## Welcome PRs
- SQLite engine implementation
- Additional engine implementations (see [`docs/ENGINES.md`](docs/ENGINES.md))
- Docker Compose for self-hosted Postgres
- Additional migration sources
- New enrichment API integrations
INSTALL_FOR_AGENTS.md
+19 -6
View File
@@ -21,10 +21,13 @@ If you fetched this file by URL without cloning yet, the companion files live at
## Step 1: Install GBrain
<!-- npm-trap + #218 recovery: canonical copy lives in README.md ("Install" warning) — sync edits. -->
> **NEVER install from the npm registry.** GBrain is not distributed on npm; the npm
> package named `gbrain` is an unrelated package. Do NOT run `npm install -g gbrain` or
> `bun add -g gbrain` (note the missing `github:` prefix — that's the trap). The only
> supported sources are `github:garrytan/gbrain` and a git clone, exactly as shown below.
> supported sources are `github:garrytan/gbrain` (optionally pinned as
> `github:garrytan/gbrain#latest-stable`, the form the bootstrap flow mandates) and a
> git clone, exactly as shown below.
> If an unrelated npm install is already present, remove it first
> (`npm uninstall -g gbrain` / `bun remove -g gbrain`); `gbrain doctor` also detects this.
@@ -93,6 +96,7 @@ default. Stop and ask the operator.
**Present this matrix verbatim:**
<!-- Cost matrix: three verbatim homes — CLAUDE.md "Search Mode", src/commands/init-mode-picker.ts, and this block. Sync all three when refreshing. -->
```
Per-query cost @ 10K queries/mo (typical single-user volume):
@@ -203,13 +207,13 @@ scaffold the bundled skills into it:
```bash
cd /path/to/agent/workspace
gbrain skillpack scaffold --all # copy 43 curated skills + RESOLVER.md
gbrain skillpack scaffold --all # copy the 50+ bundled skills + RESOLVER.md
```
Scaffolded skills are first-class files in your repo. Edit freely; re-running scaffold
refuses to overwrite anything that exists. Use `gbrain skillpack reference <name>` to
diff against gbrain's bundle when you want upstream improvements. (The legacy
`gbrain skillpack install` managed-block model was retired in v0.36.0.0 — run
`gbrain skillpack install` managed-block model was removed in v0.33 — run
`gbrain skillpack migrate-fence` once if upgrading from an older release.)
Whether you scaffolded or not, read `skills/RESOLVER.md` (in your workspace, or the
@@ -266,8 +270,17 @@ Verify: `gbrain integrations doctor` (after at least one is configured)
## Step 9: Verify
Read `docs/GBRAIN_VERIFY.md` and run all 7 verification checks. Check #4 (live sync
actually works) is the most important.
Read `docs/GBRAIN_VERIFY.md` and run every verification check in it. Check #4
(live sync actually works) is the most important.
Once verification passes and the brain has content, run the activation probe:
```bash
gbrain onboard --check --json
```
See "The onboard surface" below for what the recommendations mean and the
consent gates around unattended remediation.
## Upgrade
@@ -310,7 +323,7 @@ columns. PGLite brains no-op. If wiki-style imports were truncated by the old
`splitBody` bug, run `gbrain sync --full` after upgrading to rebuild
`compiled_truth` from source markdown.
## v0.42.0+ onboard surface (NEW)
## The onboard surface
`gbrain onboard` is the activation surface gbrain did not have before.
Once your brain has any content, run `gbrain onboard --check --json` to
README.md
+48 -52
View File
@@ -121,7 +121,7 @@ Retrieve and follow the instructions at:
https://raw.githubusercontent.com/garrytan/gbrain/master/INSTALL_FOR_AGENTS.md
```
The agent installs GBrain, creates the brain, asks for your API keys, loads 43 skills, configures the dream cycle, and verifies the install end-to-end. ~30 minutes. You answer questions, it does the work.
The agent installs GBrain, creates the brain, asks for your API keys, loads the 50+ bundled skills, configures the dream cycle, and verifies the install end-to-end. ~30 minutes. You answer questions, it does the work.
> **Never set up an AI agent platform before?** The [personal-brain tutorial](docs/tutorials/personal-brain.md) walks the whole path end-to-end — picking OpenClaw vs Hermes, deploying it, pointing it at INSTALL_FOR_AGENTS.md, getting the API keys, and verifying the first query. Start there if any of the above is new.
@@ -159,7 +159,7 @@ Postgres-at-scale, Supabase, and thin-client setup paths live in [`docs/INSTALL.
### Connect GBrain to your AI client (MCP)
GBrain exposes 30+ tools over MCP (stdio and HTTP). The specific snippet depends on which client you use:
GBrain exposes nearly all of its 100+ operations as MCP tools (stdio and HTTP; a handful of local-only ops stay CLI-side). The specific snippet depends on which client you use:
- **[Claude Code](docs/mcp/CLAUDE_CODE.md)** — local: one command, `claude mcp add gbrain -- gbrain serve` (zero server, zero tunnel). Remote with just a bearer token: `gbrain connect https://your-host/mcp --token gbrain_xxx` prints a paste-ready block (or `--install` wires it up and smoke-tests the token).
- **[Codex](docs/mcp/CODEX.md)** — `gbrain connect https://your-host/mcp --token gbrain_xxx --agent codex` (or `--install`). Codex reads the bearer from `$GBRAIN_REMOTE_TOKEN` at runtime, so the token never lands in Codex config.
@@ -281,7 +281,7 @@ The whole loop is described in [`docs/architecture/topologies.md`](docs/architec
## Capabilities
**Hybrid search.** Vector (HNSW on pgvector) + BM25 keyword + reciprocal-rank fusion + source-tier boost + intent-aware query rewriting. Three named search modes (`conservative`, `balanced`, `tokenmax`) bundle the cost/quality knobs into a single config key. Live cost/recall comparisons in [`docs/eval/SEARCH_MODE_METHODOLOGY.md`](docs/eval/SEARCH_MODE_METHODOLOGY.md). Default: `balanced` with ZeroEntropy reranker on. Per-query graph signals notice when a top result is a hub for THAT query (adjacency boost), is corroborated across team brains (cross-source boost), or is being crowded out by weak chunks from a chatty session (session demote). Run `gbrain search "<query>" --explain` to see per-stage attribution: base score, every boost that fired, what it multiplied. `gbrain doctor` ships a `graph_signals_coverage` check; `gbrain search stats` shows fire counts and failure breakdowns. Vector retrieval pools the best chunk per page, so a page surfaces on its strongest evidence instead of losing to a neighbor on one weak chunk. Queries that match a page's title phrase or a declared free-text alias (`gbrain reindex --aliases` backfills existing pages) get boosted to the page they name. Every result carries an `evidence` tag (why it matched) and a `create_safety` hint (`exists` / `probable` / `unknown`) so an agent decides whether a page already exists instead of guessing from a raw score. `gbrain search diagnose "<query>" --target <slug>` traces which retrieval layer surfaces (or misses) a page.
**Hybrid search.** Vector (HNSW on pgvector) + BM25 keyword + reciprocal-rank fusion + source-tier boost + intent-aware query rewriting. Three named search modes (`conservative`, `balanced`, `tokenmax`) bundle the cost/quality knobs into a single config key. Live cost/recall comparisons in [`docs/eval/SEARCH_MODE_METHODOLOGY.md`](docs/eval/SEARCH_MODE_METHODOLOGY.md). The install picker default-applies `tokenmax` (it recommends `conservative` for Haiku-class subagent tiers or keyless setups); a brain with `search.mode` unset resolves to `balanced` at query time. The ZeroEntropy reranker is on in `balanced` and `tokenmax`, off in `conservative`. Per-query graph signals notice when a top result is a hub for THAT query (adjacency boost), is corroborated across team brains (cross-source boost), or is being crowded out by weak chunks from a chatty session (session demote). Run `gbrain search "<query>" --explain` to see per-stage attribution: base score, every boost that fired, what it multiplied. `gbrain doctor` ships a `graph_signals_coverage` check; `gbrain search stats` shows fire counts and failure breakdowns. Vector retrieval pools the best chunk per page, so a page surfaces on its strongest evidence instead of losing to a neighbor on one weak chunk. Queries that match a page's title phrase or a declared free-text alias (`gbrain reindex --aliases` backfills existing pages) get boosted to the page they name. Every result carries an `evidence` tag (why it matched) and a `create_safety` hint (`exists` / `probable` / `unknown`) so an agent decides whether a page already exists instead of guessing from a raw score. `gbrain search diagnose "<query>" --target <slug>` traces which retrieval layer surfaces (or misses) a page.
**Self-wiring knowledge graph.** Every `put_page` extracts entity refs from markdown/wikilinks/typed-link syntax and writes edges with zero LLM calls. Typed edges (`attended`, `works_at`, `invested_in`, `founded`, `advises`, `mentions`, …). Multi-hop traversal via `gbrain graph-query`. The graph is what produces the +31.4 P@5 lift over vector-only RAG. **Obsidian-style vaults:** bare `[[note-name]]` wikilinks that point across folders — you wrote `[[struktura]]` but the page lives at `projects/struktura.md` — resolve by basename once you opt in with `gbrain config set link_resolution.global_basename true`. Off by default; `gbrain doctor` tells you how many edges you'd gain before you flip it. See [migrating an Obsidian vault](INSTALL_FOR_AGENTS.md#step-45-wire-the-knowledge-graph).
@@ -305,7 +305,7 @@ gbrain reindex-search-vector --yes # recreate triggers + backfill
The command is idempotent (re-running with the same language is a no-op for vector content) and uses the same recreate-and-backfill primitives as the migration. For accent-insensitive Portuguese (`pt_br`), see [docs/guides/multi-language-fts.md](docs/guides/multi-language-fts.md) for the `unaccent` + portuguese stemmer recipe.
**43 curated skills.** Routing lives in [`skills/RESOLVER.md`](skills/RESOLVER.md). Covers signal capture, ingest (idea / media / meeting), enrichment, querying, brain ops, citation fixing, daily task management, cron scheduling, reports, voice, soul audit, skill creation, eval framework, and migrations. Skills are markdown files (tool-agnostic), packaged as a single skillpack the installer drops into your agent workspace.
**50+ curated skills** (the current list lives in [`skills/manifest.json`](skills/manifest.json)). Routing lives in [`skills/RESOLVER.md`](skills/RESOLVER.md). Covers signal capture, ingest (idea / media / meeting), enrichment, querying, brain ops, citation fixing, daily task management, cron scheduling, reports, voice, soul audit, skill creation, eval framework, and migrations. Skills are markdown files (tool-agnostic), packaged as a single skillpack the installer drops into your agent workspace.
**Eval framework.** `gbrain eval longmemeval` runs the public [LongMemEval](https://huggingface.co/datasets/xiaowu0162/longmemeval) benchmark against your hybrid retrieval. `gbrain eval export` + `gbrain eval replay` capture real queries and replay them against code changes (set `GBRAIN_CONTRIBUTOR_MODE=1`). `gbrain eval cross-modal` cross-checks an output against the task using three different-provider frontier models. `gbrain eval retrieval-quality` runs NamedThingBench, which hard-gates the named-thing retrieval families (title-substring, alias-synonym, generic-to-named, multi-chunk-dilution) so a regression in "find the page this query names" fails CI loudly. Full methodology in [`docs/eval/SEARCH_MODE_METHODOLOGY.md`](docs/eval/SEARCH_MODE_METHODOLOGY.md).
@@ -319,14 +319,14 @@ Data flowing into the brain. Each integration is a recipe — markdown + setup h
- **Voice**: Phone calls create brain pages via Twilio + OpenAI Realtime (or DIY STT+LLM+TTS). Setup recipe: [`recipes/twilio-voice-brain.md`](recipes/twilio-voice-brain.md).
- **Email + calendar**: webhook handlers that route to brain signals. [`docs/integrations/meeting-webhooks.md`](docs/integrations/meeting-webhooks.md).
- **Embedding providers**: 16 recipes covering OpenAI (default fallback), OpenRouter, Voyage, ZeroEntropy (default), Google Gemini, Azure OpenAI, MiniMax, Alibaba DashScope, Zhipu, Ollama (local), llama.cpp llama-server (local), LiteLLM proxy. Pricing matrix + decision tree in [`docs/integrations/embedding-providers.md`](docs/integrations/embedding-providers.md).
- **Rerankers**: ZeroEntropy `zerank-2` hosted (default in `tokenmax` mode) plus the v0.40.6.1 `llama-server-reranker` recipe for fully-local cross-encoder rerank via llama.cpp — runs Qwen3-Reranker or self-hosted ZeroEntropy weights against the same `gateway.rerank()` seam. Setup walkthrough in [`docs/ai-providers/llama-server-reranker.md`](docs/ai-providers/llama-server-reranker.md).
- **Embedding providers**: a dozen providers covered — OpenAI (default fallback), OpenRouter, Voyage, ZeroEntropy (default), Google Gemini, Azure OpenAI, MiniMax, Alibaba DashScope, Zhipu, Ollama (local), llama.cpp llama-server (local), LiteLLM proxy. Pricing matrix + decision tree in [`docs/integrations/embedding-providers.md`](docs/integrations/embedding-providers.md).
- **Rerankers**: ZeroEntropy `zerank-2` hosted (the default; on in `balanced` and `tokenmax` modes) plus the `llama-server-reranker` recipe for fully-local cross-encoder rerank via llama.cpp — runs Qwen3-Reranker or self-hosted ZeroEntropy weights against the same `gateway.rerank()` seam. Setup walkthrough in [`docs/ai-providers/llama-server-reranker.md`](docs/ai-providers/llama-server-reranker.md).
- **Credential gateway**: vault-aware secret distribution. [`docs/integrations/credential-gateway.md`](docs/integrations/credential-gateway.md).
- **MCP clients**: every major MCP client is supported. [`docs/mcp/`](docs/mcp/) per-client setup.
## Architecture
**Two engines, one contract.** PGLite (Postgres 17 via WASM, zero-config, default) for personal brains up to ~50K pages. Postgres + pgvector (Supabase or self-hosted) for shared / large / multi-machine deployments. The contract-first `BrainEngine` interface in [`src/core/engine.ts`](src/core/engine.ts) defines ~47 operations both engines implement; CLI and MCP server are generated from one source.
**Two engines, one contract.** PGLite (Postgres 17 via WASM, zero-config, default) for personal brains up to ~50K pages. Postgres + pgvector (Supabase or self-hosted) for shared / large / multi-machine deployments. The contract-first `BrainEngine` interface in [`src/core/engine.ts`](src/core/engine.ts) defines the 140+ methods both engines implement; CLI and MCP server are generated from one source.
**Brain repo is the system of record.** Your knowledge lives in a regular git repo (your "brain repo") as markdown files. GBrain syncs the repo into Postgres for retrieval; deletes in git become soft-deletes in DB. You can publish public subsets, share team mounts, run thin-client setups pointing at a colleague's brain server. Topologies in [`docs/architecture/topologies.md`](docs/architecture/topologies.md).
@@ -340,10 +340,9 @@ Data flowing into the brain. Each integration is a recipe — markdown + setup h
**`gbrain import` fails with `expected N dimensions, not M`?** Run `gbrain doctor`. It will print the exact `gbrain config set ...` or `gbrain retrieval-upgrade` command to repair the mismatch. You should not need to delete `~/.gbrain`. Fresh `gbrain init --pglite` auto-detects your embedding provider from API keys in your environment: set `OPENAI_API_KEY` (or `ZEROENTROPY_API_KEY` / `VOYAGE_API_KEY`) before running init, or pass `--embedding-model <provider>:<model>` explicitly. With multiple keys set, init fires an interactive picker. In non-TTY contexts (CI, Docker) with no keys, init exits 1 with a paste-ready setup hint; pass `--no-embedding` to defer setup until runtime. See [`docs/integrations/embedding-providers.md`](docs/integrations/embedding-providers.md) for the full provider matrix and [`docs/operations/headless-install.md`](docs/operations/headless-install.md) for Docker/CI sequencing.
**Hourly cron sync keeps timing out on a federated brain?** v0.41.13.0 ships
two flags + a recommended pattern. Switch your cron to a per-source loop
with shell `timeout(1)` doing the OS-level kill and gbrain self-terminating
gracefully half-a-minute earlier:
**Hourly cron sync keeps timing out on a federated brain?** Switch your
cron to a per-source loop with shell `timeout(1)` doing the OS-level kill
and gbrain self-terminating gracefully half-a-minute earlier:
```bash
gbrain sync --break-lock --all --max-age 1800
@@ -356,19 +355,17 @@ When `--timeout` fires mid-import, `gbrain sync` exits 0 with status
`partial` and `last_commit` UNCHANGED — the next run re-walks the same
diff and `content_hash` short-circuits already-imported files. The
`--max-age 1800` first command self-heals any wedged-but-alive locks
left by a hung previous run, using the v98 `last_refreshed_at` semantic
(NOT `acquired_at`) so healthy long-running holders are safe by
construction. See the v0.41.13.0 entry in [`CHANGELOG.md`](CHANGELOG.md)
for the honest scope notes (extract + embed phases run to completion;
30-min rollout window for `--max-age` post-migration v98; full-sync
triggers deferred to v0.42+).
left by a hung previous run, keyed on the lock's last refresh time
(NOT when it was acquired) so healthy long-running holders are safe by
construction. Scope note: the extract + embed phases still run to
completion once started; `--timeout` interrupts the import walk only.
**Dream cycle silently losing wiki links on Supabase?** v0.41.19.0 fixes
the bug class structurally. The engine now self-retries every bulk batch
write (`addLinksBatch` / `addTimelineEntriesBatch` / `upsertChunks`) on
Supavisor pooler blips, with a 12s worst-case wait that covers the full
5-10s circuit-breaker recovery window. `gbrain doctor` surfaces incidents
via the new `batch_retry_health` check (reads the last 24h of
**Dream cycle silently losing wiki links on Supabase?** The engine
self-retries every bulk batch write (`addLinksBatch` /
`addTimelineEntriesBatch` / `upsertChunks`) on Supavisor pooler blips,
with a 12s worst-case wait that covers the full 5-10s circuit-breaker
recovery window. `gbrain doctor` surfaces incidents via the
`batch_retry_health` check (reads the last 24h of
`~/.gbrain/audit/batch-retry-YYYY-Www.jsonl`). To tune for an unusually
slow pooler:
@@ -386,34 +383,33 @@ retry wrap is engine-level, but PGLite has no pooler so retries never
fire in practice.
**Dream cycle losing ~150 link rows per run with `'No database
connection: connect() has not been called'` errors in the log?** v0.41.27.0
makes the retry layer self-heal on a nulled-out database singleton. A
new `reconnect` callback on `withRetry` rebuilds the connection between
attempts; `PostgresEngine.batchRetry` injects `() => this.reconnect()`
connection: connect() has not been called'` errors in the log?** The
retry layer self-heals on a nulled-out database singleton: a
`reconnect` callback on `withRetry` rebuilds the connection between
attempts, and `PostgresEngine.batchRetry` injects `() => this.reconnect()`
so engine-level batch writes survive a mid-cycle disconnect by something
else in the same process. Same release: `gbrain capture` no longer trails
a `'No database connection'` stderr line from a background facts:absorb
worker firing after CLI exit — the op-dispatch finally block awaits
else in the same process. `gbrain capture` also no longer trails a
`'No database connection'` stderr line from a background facts:absorb
worker firing after CLI exit — op dispatch awaits
`getFactsQueue().drainPending({timeout: 1000})` before
`engine.disconnect()`. To find which code path is still calling
disconnect mid-process, run `gbrain doctor --json | jq '.checks[] |
select(.id=="batch_retry_health")'`; the extended check now surfaces
24h disconnect-call count and the most-recent caller frame from a new
`~/.gbrain/audit/db-disconnect-YYYY-Www.jsonl` audit. (Closes #1570.)
select(.id=="batch_retry_health")'`; the check surfaces the
24h disconnect-call count and the most-recent caller frame from the
`~/.gbrain/audit/db-disconnect-YYYY-Www.jsonl` audit.
**`gbrain brainstorm` returning `judge_failed: true` with 0 scored
ideas?** v0.41.21.0 closes the two bugs that caused it. The judge
hard-coded a 4K-token output cap; for any run past ~40 ideas the call
truncated mid-JSON and the parser threw. Same release closes a slash-
form pricing miss: `gbrain brainstorm --judge-model
anthropic/claude-sonnet-4-6 --max-cost 5` failed with
`BudgetExhausted reason=no_pricing` because every pricing site only
matched the colon form. Both shapes work now. No config change, no
schema migration — `gbrain upgrade` is the whole fix.
ideas?** Two historical bugs caused it, both fixed: the judge
hard-coded a 4K-token output cap (any run past ~40 ideas truncated
mid-JSON and the parser threw), and slash-form model ids
(`gbrain brainstorm --judge-model anthropic/claude-sonnet-4-6
--max-cost 5`) failed with `BudgetExhausted reason=no_pricing` because
pricing lookups only matched the colon form. Both shapes work now. No
config change, no schema migration — `gbrain upgrade` is the whole fix.
**`gbrain reindex --markdown` wiped your auto/dream/signal-detector
tags?** v0.41.37.0 makes tag reconciliation add-only. Re-import and
`reindex --markdown` now ADD current frontmatter tags and never delete,
tags?** Upgrade — tag reconciliation is add-only now. Re-import and
`reindex --markdown` ADD current frontmatter tags and never delete,
so enrichment tags written to the DB (auto-tag, dream synthesize,
signal-detector) survive a re-chunk. The reindex DB-only fallback also
reconstructs the full markdown (frontmatter + body + timeline) before
@@ -421,10 +417,10 @@ re-chunking, so a page with no on-disk source keeps its frontmatter,
title, and timeline instead of getting overwritten with empty
frontmatter. Trade-off: removing a tag from a page's frontmatter no
longer removes it from the DB on the next sync (frontmatter-tag removal
needs a provenance column, deferred). (Closes #1621.)
needs a provenance column, deferred).
**`gbrain sync` wedges on a large brain (no progress, high CPU)?**
v0.41.37.0 ships three things. First, name the stalling file:
Three tools. First, name the stalling file:
```bash
GBRAIN_SYNC_TRACE=1 gbrain sync --no-pull --no-embed --yes
@@ -439,28 +435,28 @@ the sync with the pack disabled and re-run extraction later:
gbrain sync --no-schema-pack --no-pull --no-embed --yes
```
`gbrain schema lint` now warns on the classic nested-quantifier ReDoS
`gbrain schema lint` warns on the classic nested-quantifier ReDoS
shapes (`(a+)+`, `(a*)*`, …) in pack regexes, and the runtime caps
inference-regex input length (override via `GBRAIN_MAX_REGEX_INPUT_CHARS`).
Third, on a PGLite brain, stop `gbrain serve` before a large sync —
PGLite is single-writer and a live MCP server contends for the write
lock. See [`docs/architecture/serve-sync-concurrency.md`](docs/architecture/serve-sync-concurrency.md)
for the full triage. (Closes #1569.)
for the full triage.
**`gbrain init --migrate-only` / a schema migration fails on Windows
with `getaddrinfo ENOTFOUND`?** v0.41.37.0 runs the 9 schema-bring-up
with `getaddrinfo ENOTFOUND`?** Upgrade — schema bring-up now runs its
phases in-process instead of spawning a child `gbrain init
--migrate-only` per phase. The spawned child died on
Windows + bun + Supabase pooler with a DNS-resolution failure even
though the parent connected fine; running in-process removes the spawn
entirely. The v0.13.1 grandfather migration that hung 70+ minutes on an
82K-page PGLite brain is also fixed — it now runs as a chunked bulk SQL
pass (keyed on the page PK, soft-delete-filtered, source-safe) that
completes in ~1-2 seconds. (Closes #1605, #1581.)
entirely. The grandfather migration that used to hang 70+ minutes on an
80K-page PGLite brain also runs as a chunked bulk SQL pass now (keyed on
the page PK, soft-delete-filtered, source-safe) and completes in seconds.
## Docs
- [`docs/INSTALL.md`](docs/INSTALL.md) — every install path, end to end
- [`docs/guides/bootstrap.md`](docs/guides/bootstrap.md) — the persistent-personal-agent bootstrap contract (interview, identity files, hooks, private repo, security posture, uninstall)
- [`docs/what-schemas-unlock.md`](docs/what-schemas-unlock.md) — why schemas matter: 7 killer use cases, the structural argument for typed page kinds, the agent-co-curates pattern (v0.40.7.0)
- [`docs/schema-author-tutorial.md`](docs/schema-author-tutorial.md) — 5-minute walkthrough: fork the bundled pack, add a custom type, backfill existing pages, prove the wiring via `gbrain whoknows`
- [`docs/architecture/`](docs/architecture/) — system design, topologies, retrieval theory
docs/ENGINES.md
+44 -82
View File
@@ -4,7 +4,7 @@
Every GBrain operation goes through `BrainEngine`. The engine is the contract between "what the brain can do" and "how it's stored." Swap the engine, keep everything else.
v0 shipped `PostgresEngine` backed by Supabase. v0.7 adds `PGLiteEngine` -- embedded Postgres 17.5 via WASM (@electric-sql/pglite), zero-config default. The interface is designed so a `DuckDBEngine`, `TursoEngine`, or any custom backend could slot in without touching the CLI, MCP server, skills, or any consumer code.
Two engines ship today: `PGLiteEngine` — embedded Postgres via WASM (@electric-sql/pglite), the zero-config default — and `PostgresEngine`, backed by Supabase or any Postgres + pgvector. The interface is designed so a `DuckDBEngine`, `TursoEngine`, or any custom backend could slot in without touching the CLI, MCP server, skills, or any consumer code.
## Why this matters
@@ -12,7 +12,7 @@ Different users have different constraints:
| User | Needs | Best engine |
|------|-------|-------------|
| Getting started | Zero-config, no accounts, no server | PGLiteEngine (default since v0.7) |
| Getting started | Zero-config, no accounts, no server | PGLiteEngine (the default) |
| Power user (you) | World-class search, 7K+ pages, zero-ops | PostgresEngine + Supabase |
| Open source hacker | Single file, no server, git-friendly | PGLiteEngine |
| Team/enterprise | Multi-user, RLS, audit trail | PostgresEngine + self-hosted |
@@ -23,72 +23,30 @@ The engine interface means we don't have to choose. PGLite is the zero-friction
## The interface
```typescript
// src/core/engine.ts
**The single source of truth is `export interface BrainEngine` in
`src/core/engine.ts`.** It is large (100+ methods) and grows with every
feature wave — do NOT work from any snapshot of it, including an old copy of
this doc. Read the interface itself, and let
`test/e2e/engine-parity.test.ts` + `test/pglite-engine.test.ts` tell you
whether both engines agree.
export interface BrainEngine {
// Lifecycle
connect(config: EngineConfig): Promise<void>;
disconnect(): Promise<void>;
initSchema(): Promise<void>;
transaction<T>(fn: (engine: BrainEngine) => Promise<T>): Promise<T>;
The method families, to orient you before opening the file:
// Pages CRUD
getPage(slug: string): Promise<Page | null>;
putPage(slug: string, page: PageInput): Promise<Page>;
deletePage(slug: string): Promise<void>;
listPages(filters: PageFilters): Promise<Page[]>;
// Search
searchKeyword(query: string, opts?: SearchOpts): Promise<SearchResult[]>;
searchVector(embedding: Float32Array, opts?: SearchOpts): Promise<SearchResult[]>;
// Chunks
upsertChunks(slug: string, chunks: ChunkInput[]): Promise<void>;
getChunks(slug: string): Promise<Chunk[]>;
// Links
addLink(from: string, to: string, context?: string, linkType?: string): Promise<void>;
removeLink(from: string, to: string): Promise<void>;
getLinks(slug: string): Promise<Link[]>;
getBacklinks(slug: string): Promise<Link[]>;
traverseGraph(slug: string, depth?: number): Promise<GraphNode[]>;
// Tags
addTag(slug: string, tag: string): Promise<void>;
removeTag(slug: string, tag: string): Promise<void>;
getTags(slug: string): Promise<string[]>;
// Timeline
addTimelineEntry(slug: string, entry: TimelineInput): Promise<void>;
getTimeline(slug: string, opts?: TimelineOpts): Promise<TimelineEntry[]>;
// Raw data
putRawData(slug: string, source: string, data: object): Promise<void>;
getRawData(slug: string, source?: string): Promise<RawData[]>;
// Versions
createVersion(slug: string): Promise<PageVersion>;
getVersions(slug: string): Promise<PageVersion[]>;
revertToVersion(slug: string, versionId: number): Promise<void>;
// Stats + health
getStats(): Promise<BrainStats>;
getHealth(): Promise<BrainHealth>;
// Ingest log
logIngest(entry: IngestLogInput): Promise<void>;
getIngestLog(opts?: IngestLogOpts): Promise<IngestLogEntry[]>;
// Config
getConfig(key: string): Promise<string | null>;
setConfig(key: string, value: string): Promise<void>;
// Migration + advanced (added v0.7)
runMigration(sql: string): Promise<void>;
getChunksWithEmbeddings(slug: string): Promise<ChunkWithEmbedding[]>;
}
```
- **Lifecycle + identity** — `connect` / `disconnect` / `reconnect`,
`initSchema`, `transaction`, `withReservedConnection`, and the `kind`
discriminator (`'pglite' | 'postgres'`) for the rare engine-specific branch.
- **Pages CRUD** — `getPage`, `putPage`, `deletePage`, `listPages`, slug
resolution.
- **Search** — `searchKeyword`, `searchVector`, chunk-level variants, takes
search (keyword + vector), and `relationalFanout` (the typed-edge recall
arm).
- **Chunks + embeddings** — upsert/get, embedding-bearing variants.
- **Graph** — links (single + batch writers), backlinks, `traverseGraph`,
`traversePaths`.
- **Tags, timeline (single + batch), raw data, versions.**
- **Takes / facts / eval / salience** — the epistemological layer and the
instruments over it.
- **Stats, health, ingest log, config, migrations.**
### Key design choices
@@ -131,7 +89,7 @@ export interface BrainEngine {
RRF fusion, multi-query expansion, and 4-layer dedup are engine-agnostic. They operate on `SearchResult[]` arrays. Only the raw keyword and vector searches are engine-specific.
## PostgresEngine (v0, ships)
## PostgresEngine
**Dependencies:** `postgres` (porsager/postgres), `pgvector`
@@ -144,9 +102,7 @@ RRF fusion, multi-query expansion, and 4-layer dedup are engine-agnostic. They o
- JSONB for frontmatter with GIN index
- Connection pooling via Supabase Supavisor (port 6543)
**Hosting:** Supabase Pro ($25/mo). Zero-ops. Managed Postgres with pgvector built in.
**Why not self-hosted for v0:** The brain should be infrastructure agents use, not something you maintain. Self-hosted Postgres with Docker is a welcome community PR, but v0 optimizes for zero ops.
**Hosting:** Supabase Pro ($25/mo, zero-ops, pgvector built in) is the managed path; self-hosted Postgres + pgvector (Docker or Homebrew — recipe in the troubleshooting section below) works the same.
### Opt-in RLS source-scope binding (`GBRAIN_RLS_SCOPE_BINDING`)
@@ -193,17 +149,17 @@ run under the role default and are not backstopped per caller. This is layer 2;
the app-layer source filters remain layer 1 and stay mandatory. Behavioral pins
live in `test/postgres-engine-rls-scope.test.ts`.
## PGLiteEngine (v0.7, ships)
## PGLiteEngine
**Dependencies:** `@electric-sql/pglite` (v0.4.4+)
**Dependencies:** `@electric-sql/pglite`
**What it is:** Embedded Postgres 17.5 compiled to WASM via ElectricSQL's PGLite. Runs in-process, no server, no Docker, no accounts. Same SQL as PostgresEngine -- not a separate dialect. All 37 BrainEngine methods implemented.
**What it is:** Embedded Postgres compiled to WASM via ElectricSQL's PGLite. Runs in-process, no server, no Docker, no accounts. Same SQL as PostgresEngine -- not a separate dialect. Implements the full `BrainEngine` interface; `test/e2e/engine-parity.test.ts` pins that the two engines move in lockstep.
**PGLite-specific details:**
- Uses `pglite-schema.ts` for DDL (pgvector extension, pg_trgm, triggers, indexes)
- Parameterized queries throughout (shared utilities in `src/core/utils.ts`)
- `hybridSearch` keyword-only fallback when `OPENAI_API_KEY` is not set
- Data stored at `~/.gbrain/brain.db` (configurable)
- Data stored at `~/.gbrain/brain.pglite` (configurable)
- pgvector HNSW index for cosine similarity vector search (same as Postgres)
- tsvector + ts_rank for full-text search (same as Postgres)
- pg_trgm for fuzzy slug resolution (same as Postgres)
@@ -319,16 +275,22 @@ and assert `jsonb_typeof` — the assertion PGLite cannot make.
1. Create `src/core/<name>-engine.ts` implementing `BrainEngine`
2. Add to engine factory in `src/core/engine-factory.ts`:
```typescript
export function createEngine(type: string): BrainEngine {
switch (type) {
case 'pglite': return new PGLiteEngine();
case 'postgres': return new PostgresEngine();
case 'myengine': return new MyEngine();
default: throw new Error(`Unknown engine: ${type}`);
export async function createEngine(config: EngineConfig): Promise<BrainEngine> {
switch (config.engine || 'postgres') {
case 'pglite': {
const { PGLiteEngine } = await import('./pglite-engine.ts');
return new PGLiteEngine();
}
case 'myengine': {
const { MyEngine } = await import('./my-engine.ts');
return new MyEngine();
}
// ...
}
}
```
The factory uses dynamic imports so engines are only loaded when selected.
The factory uses dynamic imports so an engine's dependencies (e.g. the
PGLite WASM blob) are only loaded when that engine is selected.
3. Store engine type in `~/.gbrain/config.json`: `{ "engine": "myengine", ... }`
4. Add tests. The test suite should be engine-agnostic where possible... same test cases, different engine constructor.
5. Document in this file + add a design doc in `docs/`
@@ -359,7 +321,7 @@ Every method in `BrainEngine`. The full interface. No optional methods, no featu
| JSONB queries | GIN index | GIN index | Identical |
| Concurrent access | Connection pooling | Single process | PGLite limitation |
| Hosting | Supabase, self-hosted, Docker | Local file | |
| Migration methods | runMigration, getChunksWithEmbeddings | Same | Added v0.7 |
| Migration methods | runMigration, getChunksWithEmbeddings | Same | Identical |
## Future engine ideas
docs/GBRAIN_RECOMMENDED_SCHEMA.md
+8
View File
@@ -6,6 +6,14 @@ A system prompt for any AI agent that wants to build and maintain a personal kno
Drop this into your agent's workspace as a skill or system prompt. Your agent will build the rest.
> **Relationship to schema packs:** this document is the prose, paste-in
> version of the schema pattern. gbrain also ships a machine-enforced
> counterpart — schema packs (`gbrain schema`, typed pages, extraction,
> aliases, lint) — documented in `docs/architecture/schema-packs.md` and
> `docs/schema-author-tutorial.md`. The prose schema here and the active
> schema pack should describe the same brain; when you evolve one, evolve
> the other.
---
## What this is
docs/GBRAIN_SKILLPACK.md
+4 -2
View File
@@ -1,10 +1,10 @@
<!-- skillpack-version: 0.7.0 -->
<!-- source: https://raw.githubusercontent.com/garrytan/gbrain/master/docs/GBRAIN_SKILLPACK.md -->
# GBrain Skillpack: Reference Architecture for AI Agents
This is a reference architecture for how a production AI agent uses gbrain as its
knowledge backbone. Based on patterns from a real deployment with 14,700+ brain
files, 40+ skills, and 20+ cron jobs running continuously.
files, the 50+ bundled skills (`skills/manifest.json`), and 20+ cron jobs running
continuously.
**The memex vision, realized.** Vannevar Bush imagined a device where an individual
stores everything, mechanized so it may be consulted with exceeding speed. GBrain is
@@ -25,6 +25,7 @@ The foundational read-write loop and data model.
| [Entity Detection](guides/entity-detection.md) | Run it on every message. Capture original thinking + entity mentions |
| [The Originals Folder](guides/originals-folder.md) | Capturing WHAT YOU THINK, not just what you found |
| [Brain-First Lookup](guides/brain-first-lookup.md) | Check the brain before calling any external API |
| [Push-Based Context](guides/push-context.md) | volunteer_context: the brain volunteers relevant pages instead of waiting to be asked |
| [Compiled Truth + Timeline](guides/compiled-truth.md) | Above the line: current synthesis. Below: append-only evidence |
| [Source Attribution](guides/source-attribution.md) | Every fact needs a citation. Format and hierarchy |
@@ -99,6 +100,7 @@ Keeping it running and up to date.
| Guide | What It Covers |
|-------|---------------|
| [Agent Bootstrap](guides/bootstrap.md) | The paste-in install: `gbrain bootstrap`, hooks, `bootstrap verify`, keyless mode |
| [Upgrades & Auto-Update](guides/upgrades-auto-update.md) | check-update, agent notifications, migration files |
| [Live Sync](guides/live-sync.md) | Keep the index current: cron, --watch, webhook approaches |
docs/GBRAIN_VERIFY.md
+28 -14
View File
@@ -1,5 +1,13 @@
# GBrain Installation Verification Runbook
> **One-command equivalent:** `gbrain bootstrap verify` runs the whole install
> contract (round-trip, graph floor, and more) automatically and exits non-zero
> on failure — it is the modern first thing to run after any install. See
> [docs/guides/bootstrap.md](guides/bootstrap.md). This runbook is the
> **manual, deep-verification** companion: use it when `bootstrap verify` fails
> and you need to isolate which layer broke, or when you want to understand
> what "healthy" looks like check by check.
Run these checks after install to confirm every part of GBrain is working.
Each check includes the command, expected output, and what to do if it fails.
@@ -20,7 +28,8 @@ gbrain doctor --json
**Expected:** All checks return `"ok"`:
- `connection`: connected, N pages
- `pgvector`: extension installed
- `rls`: enabled on all tables
- `rls`: enabled on all tables (Postgres/Supabase brains only — PGLite brains
skip this check; the embedded engine has no remote surface)
- `schema_version`: current
- `embeddings`: coverage percentage
@@ -33,12 +42,12 @@ check. See `skills/setup/SKILL.md` Error Recovery table.
**Check:** Ask the agent: "What is the brain-agent loop?"
**Expected:** The agent references GBRAIN_SKILLPACK.md Section 2 and describes
the read-write cycle: detect entities, read brain, respond with context, write
brain, sync.
**Expected:** The agent describes the read-write cycle documented in
[docs/guides/brain-agent-loop.md](guides/brain-agent-loop.md): detect entities,
read brain, respond with context, write brain, sync.
**If it fails:** The agent hasn't loaded the skillpack. Run step 6 from the
install paste (read `docs/GBRAIN_SKILLPACK.md`).
**If it fails:** The agent hasn't loaded the skillpack. Have it read
`docs/GBRAIN_SKILLPACK.md` (the index) and follow the Core Patterns links.
---
@@ -53,8 +62,8 @@ gbrain check-update --json
**Expected:** Returns JSON with `current_version`, `latest_version`,
`update_available` (boolean). The cron `gbrain-update-check` is registered.
**If it fails:** Run step 7 from the install paste. See GBRAIN_SKILLPACK.md
Section 17.
**If it fails:** See [docs/guides/upgrades-auto-update.md](guides/upgrades-auto-update.md)
for how to register the update-check cron.
---
@@ -88,8 +97,9 @@ find /data/brain -name '*.md' \
Some difference is normal (files added since last sync), but if page count is
less than half the file count, sync is silently skipping pages.
**If page count is way too low:** The #1 cause is an unreachable direct
connection on an IPv4-only host. GBrain uses the Transaction pooler (port 6543)
**If page count is way too low (Supabase/Postgres brains):** The #1 cause is an
unreachable direct connection on an IPv4-only host. (PGLite brains have no
network layer — for them, check that the sync cron/watch is actually running.) GBrain uses the Transaction pooler (port 6543)
for reads, but routes migrations, DDL, and sync transactions to a derived direct
connection (`db.<ref>.supabase.co:5432`), which is IPv6-only.
- On an IPv4-only host, reads work but sync transactions fail and silently skip
@@ -122,7 +132,7 @@ This is the real test. Edit a brain page, push, wait, search.
1. Edit a page in the brain repo (e.g., correct a fact on a person's page):
```bash
# Example: fix a line in Gustaf's page
# Example: fix a line in alice-example's page
cd /data/brain
# Make a small edit to any .md file
git add -A && git commit -m "test: verify live sync" && git push
@@ -253,19 +263,23 @@ gbrain repair-jsonb
Idempotent. PGLite brains always report 0 (unaffected by the original bug).
**Bonus check** — frontmatter-keyed queries actually resolve:
**Bonus check** — the doctor's dedicated JSONB scan agrees:
```bash
gbrain call list_pages '{"frontmatterKey": "type", "frontmatterValue": "person"}'
gbrain doctor --json | grep -o '"name":"jsonb_integrity"[^}]*'
```
If this returns rows on a brain with person pages, the JSONB path is healthy.
**Expected:** the fragment contains `"status":"ok"` ("All JSONB columns store
objects/arrays"). If it reports double-encoded rows, run `gbrain repair-jsonb`.
---
## Quick Verification (all checks in one pass)
```bash
# 0. The one-command contract check (exits non-zero on failure)
gbrain bootstrap verify
# 1. Schema
gbrain doctor --json
docs/INSTALL.md
+31 -46
View File
@@ -1,15 +1,24 @@
# Install
Three install paths. Pick one. Mix later if needed.
**Recommended door: the agent bootstrap.** Open your agent (Codex, Claude Code,
or any harness) in the folder that will become its home and paste the block
from the [README's install section](../README.md) — the agent fetches
`BOOTSTRAP_FOR_AGENTS.md` from the `latest-stable` tag, installs the CLI,
initializes a local PGLite brain, wires MCP, and isn't done until
`gbrain bootstrap verify` exits 0. Full contract, security posture, and
uninstall: [docs/guides/bootstrap.md](guides/bootstrap.md).
## 1. Run with an agent platform (recommended)
The paths below are the manual equivalents and deep-dive detail. Pick one.
Mix later if needed.
## 1. Run with an agent platform
Already running [OpenClaw](https://github.com/garrytan/openclaw) or [Hermes](https://github.com/garrytan/hermes)?
```bash
bun install -g github:garrytan/gbrain
bun install -g github:garrytan/gbrain#latest-stable
gbrain init --pglite # 2 seconds; no server
gbrain skillpack scaffold --all # 43 skills scaffolded into your agent workspace
gbrain skillpack scaffold --all # scaffolds every bundled skill (skills/manifest.json) into your agent workspace
gbrain doctor # green checks all the way down
```
@@ -24,7 +33,7 @@ To upgrade later: `gbrain upgrade` runs schema migrations + post-upgrade prompts
No agent platform, just shell + MCP-aware editor.
```bash
bun install -g github:garrytan/gbrain
bun install -g github:garrytan/gbrain#latest-stable
gbrain init --pglite
```
@@ -106,55 +115,31 @@ Useful for: team mounts, brain-as-a-service deployments, dev machines without di
## Verifying the install
```bash
gbrain bootstrap verify # the whole install contract; exits non-zero on failure
gbrain doctor --json # full health check
gbrain models # which AI models are configured for what
gbrain models doctor # 1-token probe per configured model
```
If anything's yellow, `gbrain doctor` names the fix command in the message. Most issues are missing API keys or stale schema (`gbrain upgrade --force-schema`).
If anything's yellow, `gbrain doctor` names the fix command in the message. Most issues are missing API keys or stale schema (`gbrain upgrade --force-schema`). For the manual check-by-check runbook, see [docs/GBRAIN_VERIFY.md](GBRAIN_VERIFY.md).
## Troubleshooting
### PGLite crashes on macOS 26.x (Tahoe)
### PGLite crashes at startup (`RuntimeError: Aborted()`)
This crash (`RuntimeError: Aborted()` at engine startup, typically first seen
after a macOS upgrade) is **not** a macOS/WASM incompatibility. The upgrade
reboot kills gbrain mid-write and tears the data dir's write-ahead log; every
subsequent open then fails WAL replay. Recovery ladder:
This crash (typically first seen after a macOS upgrade) is **not** a
macOS/WASM incompatibility — an unclean shutdown tore the data dir's
write-ahead log, and every subsequent open fails WAL replay. The short
version of the recovery ladder:
1. **Auto-repair (default):** just run any gbrain command — gbrain detects the
abort, resets the WAL in place (data preserved; a backup of the pre-repair
state is kept next to the data dir), and continues. Then run `gbrain doctor`.
2. **Manual repair:** `gbrain pglite-repair --dry-run` to diagnose,
`gbrain pglite-repair --yes` to repair in place.
3. **Rebuild:** `gbrain reinit-pglite` (wipes and re-creates the brain from
your brain repo; embedding settings default from your config).
4. **Switch engines** — if you prefer a server database anyway, native
Homebrew PostgreSQL works great and supports multiple concurrent agents:
1. **Auto-repair (default):** run any gbrain command — gbrain detects the
abort, resets the WAL in place (data preserved, backup kept), and
continues. Then run `gbrain doctor`.
2. **Manual repair:** `gbrain pglite-repair --dry-run`, then
`gbrain pglite-repair --yes`.
3. **Rebuild:** `gbrain reinit-pglite`.
4. **Switch engines:** Supabase or native Homebrew Postgres + pgvector.
```bash
# Install PostgreSQL + pgvector
brew install postgresql@17
brew services start postgresql@17
createdb gbrain
# Build pgvector from source (required for vector search)
cd /tmp && git clone --branch v0.8.0 https://github.com/pgvector/pgvector.git
cd pgvector && make && make install
psql gbrain -c "CREATE EXTENSION IF NOT EXISTS vector;"
# Point gbrain at your local Postgres
cat > ~/.gbrain/config.json << 'EOF'
{
"engine": "postgres",
"database_url": "postgresql://localhost:5432/gbrain",
"schema_pack": "gbrain-base-v2"
}
EOF
# Run migrations and verify
gbrain apply-migrations --yes
gbrain doctor
```
Once `gbrain doctor` shows green, the brain works identically to PGLite — same commands, same skills, same data model. The only difference is the storage backend (plus multi-connection support: several agents can share one Postgres brain, which PGLite's single-process lock doesn't allow).
The full ladder — safety bounds, kill-switches, when WAL repair can't help,
and the Homebrew Postgres recipe — lives in
[docs/ENGINES.md](ENGINES.md#troubleshooting-startup-abort-runtimeerror-aborted).
docs/RELEASING.md
+57 -11
View File
@@ -26,19 +26,21 @@ Two equivalent paths:
**Path B — manual lifecycle (still supported):**
- `bun test` — unit tests (no database required)
- Follow the "E2E test DB lifecycle" steps above to spin up the test DB,
run `bun run test:e2e`, then tear it down.
- Follow the "E2E test DB lifecycle" steps in
[docs/TESTING.md](TESTING.md) to spin up the test DB, run
`bun run test:e2e`, then tear it down.
Both must pass. Do not ship with failing E2E tests. Do not skip E2E tests.
**Always run typecheck before pushing.** `bun test` (the bun runner)
skips TypeScript type checking — it only enforces runtime behavior.
**Always run typecheck before pushing.** Neither `bun test` (the bun runner)
nor `bun run test` gates on types — `bun run test` is just
`bash scripts/run-unit-parallel.sh` (the sharded unit runner; no typecheck,
no shell pre-checks — see the test-tier table in [docs/TESTING.md](TESTING.md)).
Three ways to actually gate on types:
1. `bun run test` (npm script in `package.json`) — includes `bun run typecheck`
plus the four shell pre-checks (`check-jsonb-pattern.sh`,
`check-progress-to-stdout.sh`, `check-trailing-newline.sh`,
`check-wasm-embedded.sh`) before the runner. Use this mid-branch.
1. `bun run verify` — runs the shell guard checks (privacy, jsonb, source-id,
progress-to-stdout, …) plus `bun run typecheck` in parallel
(`scripts/run-verify-parallel.sh`). Use this mid-branch.
2. `bun run typecheck` — `tsc --noEmit` standalone. Fast (~5s on this repo).
3. `bun run ci:local` — the full local CI gate from Path A.
@@ -290,7 +292,8 @@ matter" with BrainBench-style before/after table, "what this means" closer, then
Create a migration file at `skills/migrations/v[version].md` when a release
includes changes that existing users need to act on. The auto-update agent
reads these files post-upgrade (Section 17, Step 4) and executes them.
reads these files post-upgrade (see `docs/guides/upgrades-auto-update.md`)
and executes them.
**You need a migration file when:**
- New setup step that existing installs don't have (e.g., v0.5.0 added live sync,
@@ -343,8 +346,8 @@ canonical.
## Schema state tracking
`~/.gbrain/update-state.json` tracks which recommended schema directories the user
adopted, declined, or added custom. The auto-update agent (SKILLPACK Section 17)
reads this during upgrades to suggest new schema additions without re-suggesting
adopted, declined, or added custom. The auto-update agent
(`docs/guides/upgrades-auto-update.md`) reads this during upgrades to suggest new schema additions without re-suggesting
things the user already declined. The setup skill writes the initial state during
Phase C/E. Never modify a user's custom directories or re-suggest declined ones.
@@ -373,6 +376,46 @@ the release the same, uses that version's `CHANGELOG.md` entry as the notes
(`scripts/changelog-entry.sh`; falls back to a CHANGELOG link if the entry is
missing), and attaches the compiled binaries.
### The `latest-stable` tag
The **final step of the release job** force-advances the `latest-stable` tag to
the release commit (`git push origin "+${GITHUB_SHA}:refs/tags/latest-stable"`).
`latest-stable` is the single sanctioned distribution ref: the README paste
block, the `BOOTSTRAP_FOR_AGENTS.md` fetch URL, and
`bun install -g github:garrytan/gbrain#latest-stable` all reference it
permanently, so paste blocks copied into the wild never rot and there is no 404
window between VERSION landing and assets publishing.
`scripts/check-bootstrap-tag.sh` keeps the entry docs pinned to this ref.
Because it moves ONLY after binaries + provenance attestation have fully
published, a half-built release never advances it. If the tag-advance step
alone fails, re-advance by hand (a full workflow re-run would skip — the
release already exists with all assets):
```bash
git push origin "+refs/tags/v<VERSION>^{commit}:refs/tags/latest-stable"
```
### The `publish-template` job
After the release job, a `publish-template` job force-pushes the rendered
agent-workspace template repo (the GitHub "Use this template" door,
`vars.TEMPLATE_REPO`, default `garrytan/gbrain-agent-template`) from CI only —
no human pushes it by hand, so what adopters clone is exactly what this repo
reviewed. It is guarded three ways: the release above fully published; the
vendored tree `templates/bootstrap/template-repo/` exists (skip, never fail,
if not); and the `TEMPLATE_REPO_PAT` secret is configured (skip if not).
Before pushing, it regenerates the template tree
(`bun run scripts/generate-template-repo.ts`) and byte-diffs it against the
vendored copy — a mismatch fails the job; regenerate + commit the vendored
tree (`scripts/check-bootstrap-templates.sh` runs the same diff offline in
`bun run verify`).
**`TEMPLATE_REPO_PAT` scope:** a fine-grained PAT with `contents: write` on
the template repository ONLY — no other repositories, no other permissions.
Configure it as a repo secret; when absent, template publishing is disabled
and the job skips cleanly.
Why every bump, not selective: `gbrain check-update` resolves the latest
version from `VERSION` on master, while binary self-update
(`src/core/binary-self-update.ts`) downloads assets from `releases/latest`.
@@ -395,6 +438,9 @@ Invariants:
history; every new 4-segment `VERSION` mints a fresh tag.
- **Permissions stay scoped.** `contents: write` lives on the release job
only; everything else runs read-only.
- **Never advance `latest-stable` on a partial release.** The tag moves only
as the final release-job step, after every asset has published. Manual
re-advances must point at a fully published `v<VERSION>` release.
## PR descriptions cover the whole branch
docs/TESTING.md
+12 -5
View File
@@ -11,13 +11,13 @@ Seven test command tiers, each with a clear scope:
| Command | What it runs | Wallclock | When to use |
|---|---|---|---|
| `bun run test` | Parallel unit-test fast loop. 8-shard fan-out via `scripts/run-unit-parallel.sh`, then a serial pass over `*.serial.test.ts`. Excludes `*.slow.test.ts` and `test/e2e/*`. No pre-checks, no typecheck. | ~85s on a Mac dev box (3650+ tests) | Inner edit loop. Default. |
| `bun run verify` | CI's authoritative pre-test gate set, fanned out in parallel by `scripts/run-verify-parallel.sh`: the full `check:*` battery (~30 checks — privacy, jsonb, progress, source-id, test-isolation, wasm, …) plus `bun run typecheck`. The `CHECKS` array in that script is the single source of truth — CI literally calls `bun run verify` in a dedicated job. | ~16s (parallel; typecheck dominates) | Before pushing; before `/ship`. |
| `bun run test` | Parallel unit-test fast loop. Sharded fan-out via `scripts/run-unit-parallel.sh` (default 4 shards — CPU-detected, clamped to a max of 8, and defaulted down to 4 when there's no `--shards`/`SHARDS` override; 4 matches CI's fan-out and avoids PGLite WASM-init contention), then a serial pass over `*.serial.test.ts`. Excludes `*.slow.test.ts` and `test/e2e/*`. No pre-checks, no typecheck. | a few minutes on a Mac dev box | Inner edit loop. Default. |
| `bun run verify` | CI's authoritative pre-test gate set, fanned out in parallel by `scripts/run-verify-parallel.sh`: the full `check:*` battery (privacy, jsonb, progress, source-id, test-isolation, wasm, …) plus `bun run typecheck`. The `CHECKS` array in that script is the single source of truth — CI literally calls `bun run verify` in a dedicated job. | ~16s (parallel; typecheck dominates) | Before pushing; before `/ship`. |
| `bun run test:full` | `verify && bun run test && bun run test:slow && [smart e2e]`. The local equivalent of "everything CI runs." Smart e2e: runs e2e only when `DATABASE_URL` is set; else loud skip notice to stderr. | ~3-5min depending on slow + e2e | Pre-merge sanity, before opening a PR. |
| `bun run test:slow` | Just the `*.slow.test.ts` set (intentional cold-path correctness checks). | seconds-to-minutes | When touching slow-path code. |
| `bun run test:serial` | Just the `*.serial.test.ts` set (cross-file-contention quarantine; one bun process per file for true module-registry isolation). | ~1s per quarantined file | Debugging a specific quarantined file. |
| `bun run test:e2e` | Real Postgres E2E. Requires Docker + `DATABASE_URL`. Sequential. | ~5-10min | Pre-ship; nightly. |
| `bun run check:all` | The historical pre-check scripts (22, chained sequentially in package.json). Overlaps `verify` heavily but is NOT a superset — `verify`'s `CHECKS` array in `scripts/run-verify-parallel.sh` (~30 entries incl. typecheck) is the authoritative gate; `check:all` keeps a few local-only extras (trailing-newline, exports-count, no-legacy-getconnection). | ~10s | Local-only sweep for the extras. |
| `bun run check:all` | The historical pre-check scripts (chained sequentially in package.json). Overlaps `verify` heavily but is NOT a superset — `verify`'s `CHECKS` array in `scripts/run-verify-parallel.sh` is the authoritative gate; `check:all` keeps a few local-only extras (trailing-newline, exports-count, no-legacy-getconnection). | ~10s | Local-only sweep for the extras. |
### Shell dispatch and Windows
@@ -61,11 +61,16 @@ When `bun run test` finds any failure, the wrapper:
3. Writes a one-line-per-shard summary to `.context/test-summary.txt` (`shard N/M: pass=X fail=Y skip=Z rc=W`).
4. Exits non-zero. Empty failure log + non-zero exit = infrastructure problem (wedged shard, killed child); the banner says so.
If a shard wedges (per-shard `GBRAIN_TEST_SHARD_TIMEOUT` cap, default 600s), the wrapper writes `--- shard N: WEDGED after ${SHARD_TIMEOUT}s ---` to the failure log, includes the last 50 lines of the shard log, and proceeds with other shards' results.
If a shard hits the per-shard `GBRAIN_TEST_SHARD_TIMEOUT` cap (default 2400s — sized so the heaviest count-balanced shard finishes under 4-way contention), the wrapper classifies the kill one of two ways:
- **EXIT-HANG → warn-pass.** If the shard's log had been silent for ≥300s at kill time AND shows zero `(fail)` markers, the shard finished all its work, leaked a handle, and never exited (a pre-existing, master-reproducible PGLite-adjacent leak — see TODOS.md "unit-shard exit hang"). The wrapper prints a `⚠️ shard N/M: EXIT-HANG ... Treating as pass-with-warning` banner, writes `EXIT-HANG (idle Ns, 0 fails) ... warn-pass` to the summary, and does NOT fail the run. Its pass counts are undercounted (bun never printed its final summary). Bun's per-test `--timeout` turns a genuinely hung TEST into a printed `(fail)` — new output — so this classification cannot mask a hung test; the residual maskable case is a file-level import hang in the very last file, which the banner keeps visible.
- **WEDGED → hard failure.** Anything else (failures present, or the log was still growing) writes `--- shard N: WEDGED after ${SHARD_TIMEOUT}s ---` to the failure log with the last 50 lines of the shard log, marks the run failed, and proceeds with other shards' results.
Triage rule: a `warn-pass` EXIT-HANG line in `.context/test-summary.txt` is NOT a test failure — don't burn time bisecting it; a `WEDGED` line is.
### File taxonomy
- `*.test.ts` → fast loop (parallel 8-shard fan-out).
- `*.test.ts` → fast loop (parallel sharded fan-out, default 4 shards).
- `*.slow.test.ts` → run via `bun run test:slow` only (intentional cold-path tests; would dominate the fast loop's wallclock).
- `*.serial.test.ts` → run via `bun run test:serial` after the parallel pass completes; one bun process per file (`--max-concurrency=1` within a shared process is not enough — the module registry still leaks `mock.module`). Quarantine for tests that share file-wide state and race when run alongside other files in the same `bun test` process. Several dozen files, discovered by the `*.serial.test.ts` glob — no list to maintain. Typical residents: `mock.module(...)` users (top-level mocks leak across files in a shard process, e.g. `test/embed.serial.test.ts`), env-coupled files (e.g. `test/brain-registry.serial.test.ts`), and process-lifecycle suites that assert on `process.exitCode` (e.g. `test/pglite-engine-disconnect.serial.test.ts`). **Do not put the parallelism back on a serial file unless you've fixed the contention root cause** (it just re-introduces the flake).
- `test/e2e/*.test.ts` → real-Postgres E2E. Skipped when `DATABASE_URL` is unset.
@@ -83,6 +88,8 @@ Any change under `skills/` must regenerate it: `bun run scripts/generate-skills-
### Test-isolation lint and helpers
**This section is the canonical home of the test-isolation discipline** — CONTRIBUTING.md and other docs link here rather than restating the rules.
The cross-file flake class is enforced statically by `scripts/check-test-isolation.sh`, wired into `bun run verify` and `bun run check:all`. Rules (non-serial unit files only; `*.serial.test.ts` and `test/e2e/*` are skipped):
| Rule | What it bans | Fix |
docs/UPGRADING_DOWNSTREAM_AGENTS.md
+8
View File
@@ -1,5 +1,13 @@
# Upgrading Downstream Agents
> **Currency note:** this file is an append-only historical log and stopped
> receiving new sections after v0.36.5.0. **The canonical, maintained upgrade
> channel is `skills/migrations/v*.md`** (the agent-executed migration files
> that `gbrain upgrade` / `gbrain post-upgrade` route through), plus
> `CHANGELOG.md` for what each release changed. Use this file only to catch a
> long-diverged fork up through the versions it covers; for anything after
> v0.36.5.0, walk the migration files and CHANGELOG instead.
GBrain ships skills in `skills/`. Downstream agents (custom OpenClaw deployments,
agent forks of any kind) often **copy** these skill files into their own workspace and
diverge over time — adding agent-specific phases, removing irrelevant ones, tightening
docs/architecture/KEY_FILES.md
+61 -110
View File
File diff suppressed because one or more lines are too long
docs/architecture/RETRIEVAL.md
+46 -19
View File
@@ -4,14 +4,14 @@ Vector search alone underdelivers on real personal-knowledge queries. This doc e
## The four strategies in concert
1. **Vector (HNSW on pgvector)** — semantic similarity. Catches "who works on retrieval quality at YC?" → pages mentioning "Garry Tan + retrieval" even when the user never typed "YC".
1. **Vector (HNSW on pgvector)** — semantic similarity. Catches "who works on retrieval quality at acme-example?" → pages mentioning "alice-example + retrieval" even when the user never typed "acme".
2. **BM25 keyword** — lexical match. Catches names, exact phrases, code identifiers, anything where the user remembers the literal token. Survives the cases where vector search drifts into thematic neighbors.
3. **Reciprocal-rank fusion (RRF)** — merges vector + keyword rankings without weighting one over the other globally. Each strategy gets to vote.
4. **Knowledge graph traversal** — follows typed edges. Catches "what did Bob invest in this quarter?" by walking `bob ── invested_in ──> company ── dated ──> Q1`. Vector search can't see causal chains; the graph can.
## Why each one alone fails
**Vector only.** Returns chunks semantically close to the query. Misses any factual relationship not directly encoded in the embedding. "Companies in Garry's portfolio" returns essays about portfolios, not company pages.
**Vector only.** Returns chunks semantically close to the query. Misses any factual relationship not directly encoded in the embedding. "Companies in alice-example's portfolio" returns essays about portfolios, not company pages.
**Keyword only (ripgrep-style).** Brittle to phrasing. "Who works on retrieval?" misses pages that say "search ranking" instead of "retrieval." Garbage on synonyms, near-misses, or paraphrases.
@@ -36,8 +36,8 @@ BrainBench (corpus + harness in the sibling [gbrain-evals](https://github.com/ga
Every `put_page` runs `extractEntityRefs` on the markdown body. It matches:
- Standard markdown links: `[Garry Tan](wiki/people/garry-tan)`
- Obsidian wikilinks: `[[wiki/people/garry-tan|Garry Tan]]`
- Standard markdown links: `[Alice Example](wiki/people/alice-example)`
- Obsidian wikilinks: `[[wiki/people/alice-example|Alice Example]]`
- Typed-link blockquotes: `> **Convention:** see [path](path).`
Three regexes, zero LLM tokens, single SQL `addLinksBatch` call with `INSERT ... SELECT FROM jsonb_to_recordset(($1::jsonb)->'rows') JOIN pages ON CONFLICT DO NOTHING RETURNING 1` (free-text-safe; the prior `unnest(${arr}::text[])` form crashed on calendar/Zoom context per gbrain#1861). The graph grows on every write at near-zero cost. On a 17K-page brain, full graph extract completes in seconds.
@@ -46,7 +46,7 @@ Heuristic link-type inference (`attended`, `works_at`, `invested_in`, `founded`,
## ZeroEntropy as reranker: 60% top-1 reshuffle
v0.36.0.0 ships ZeroEntropy's `zerank-2` as the default reranker (on for the `balanced` mode bundle). On a real-corpus benchmark across 20 queries, zerank-2 reshuffles **60% of top-1 results** after the hybrid + RRF + graph stack. That's the headline number.
ZeroEntropy's `zerank-2` is the default reranker (on for the `balanced` and `tokenmax` mode bundles, off for `conservative`). On a real-corpus benchmark across 20 queries, zerank-2 reshuffles **60% of top-1 results** after the hybrid + RRF + graph stack. That's the headline number.
The mechanical reason: hybrid ranking is locally optimal per strategy but globally suboptimal. A cross-encoder reranker reads the query + each candidate document jointly, with full attention. It catches the cases where the vector + keyword + graph signals all agreed on a document that's semantically related but topically wrong.
@@ -62,8 +62,9 @@ The boost map is configurable via `GBRAIN_SOURCE_BOOST` env var or per-call `Sea
## Named-thing retrieval (per-page pool + title + alias + evidence)
A brain organized around *chosen names* (Mingtang, Hall of Light) needs more than
embedding proximity. Four layers, added after the incident in
A brain organized around *chosen names* (project codenames, place nicknames —
say a project named "Helios" whose page is also known as "the Sun Room") needs
more than embedding proximity. Four layers, added after the incident in
[`RETRIEVAL_MAXPOOL_INCIDENT.md`](./RETRIEVAL_MAXPOOL_INCIDENT.md):
- **Per-page max-pool** — `searchVector` (both engines) collapses chunk-grain
@@ -79,7 +80,7 @@ embedding proximity. Four layers, added after the incident in
`page_aliases` table (separate from the `slug_aliases` wikilink redirect) and
consulted at query time: a full normalized-query match injects/boosts the
canonical page (`applyAliasHop`). The only layer that bridges true synonyms
with zero surface overlap ("Hall of Light" → the Mingtang page). Backfill
with zero surface overlap ("the Sun Room" → the Helios page). Backfill
existing pages with `gbrain reindex --aliases`.
- **Evidence contract** — every result carries `evidence`
(`alias_hit | exact_title_match | high_vector_match | keyword_exact |
@@ -103,7 +104,7 @@ specific miss with `gbrain search diagnose "<q>" --target <slug>`.
## Intent-aware query rewriting
`src/core/search/intent.ts` classifies queries into `entity`, `temporal`, `event`, or `general`. Each routes through different ranking knobs:
`src/core/search/query-intent.ts` classifies queries into `entity`, `temporal`, `event`, or `general`. Each routes through different ranking knobs:
- **Entity** queries ("who works at X?") apply a higher graph-traversal weight.
- **Temporal** queries ("what happened last week?") bypass source-boost so chat/daily pages surface.
@@ -123,35 +124,61 @@ Expansion is opt-in per mode bundle (`tokenmax` on by default; `balanced` + `con
The full pipeline for a `query` op:
```
intent classify
intent classify (query-intent.ts — deterministic, no LLM)
│
▼
expansion (if enabled)
expansion (if enabled — tokenmax only by default)
│
▼
hybrid search:
├── vector (HNSW on chunk embeddings)
hybrid recall + fusion:
├── vector (HNSW on chunk embeddings, per-page max-pool)
├── keyword (BM25 via tsvector)
├── relational (v0.42.34.0: typed-edge recall arm — relational queries only)
├── title-phrase arm
├── relational (typed-edge recall arm — relational queries only)
├── source-aware re-rank (CASE in SQL)
└── RRF fusion → top 30
└── RRF fusion → cosine re-score → post-fusion boosts
(backlink / salience / recency / graph signals / exact-match)
│
▼
graph augment (typed-edge traversal from any seed)
graph augment (optional two-pass structural expansion — walkDepth > 0)
│
▼
reranker (zerank-2 cross-encoder, top 30 → reordered)
deduplication (4-layer: per-page cap, Jaccard, type diversity)
│
▼
token-budget enforcement (per mode bundle)
reranker (zerank-2 cross-encoder — balanced/tokenmax; fail-open)
│
▼
deduplication (same slug, different chunks → keep best)
alias hop (exact alias match injects/boosts the canonical page)
│
▼
evidence stamp → adaptive return (opt-in) → autocut (reranked modes)
│
▼
limit slice → token-budget enforcement (per mode bundle)
│
▼
results
```
The stage order is pinned by `hybridSearch` in `src/core/search/hybrid.ts`:
dedup runs BEFORE the reranker (so the reranker sees a diverse candidate pool,
capped by its own `topNIn`), the alias hop runs AFTER the reranker (so a query
that is a page's declared name reliably surfaces that page regardless of how
the reranker scored body chunks), and the token budget is enforced last, on
the final slice.
### Autocut: score-discontinuity result-sizing
Default-on for `balanced` and `tokenmax` (off for `conservative`, which has no
reranker and therefore no trustworthy cliff signal). `applyAutocut`
(`src/core/search/autocut.ts`) cuts the ranked set at the largest
cross-encoder rerank-score cliff, before the limit slice, first page only.
Never-empty failsafe (`minKeep`), no-op when fewer than 2 results carry a
finite rerank score (covers the fail-open reranker path), and alias-hop exact
matches are preserved through the cut. Knobs: per-call `SearchOpts.autocut` →
`search.autocut` / `search.autocut_jump` config → mode bundle.
Each stage is testable in isolation. Each stage is replaceable. The whole pipeline is < 1ms of orchestration cost; the latency budget goes to the upstream HTTP calls (embedding, rerank) and the index scans.
## How to verify on your own brain
docs/architecture/brains-and-sources.md
+16 -9
View File
@@ -19,18 +19,18 @@ need to understand both of them, or queries misroute silently.
A **brain** is one database — PGLite file, self-hosted Postgres, or Supabase.
Each brain has:
- Its own `pages` table, `chunks` table, `embeddings`, etc.
- Its own OAuth surface if served over HTTP MCP (v0.19+, PR 2).
- Its own OAuth surface if served over HTTP MCP.
- Its own separate lifecycle, backup, access control.
Brains are enumerated by:
- **host** — your default brain, configured in `~/.gbrain/config.json`.
- **mounts** — additional brains registered in `~/.gbrain/mounts.json` via
`gbrain mounts add <id>` (v0.19+).
`gbrain mounts add <id>`.
Routing: `--brain <id>`, `GBRAIN_BRAIN_ID`, `.gbrain-mount` dotfile, or
longest-path match against registered mount paths. Falls back to `host`.
### Sources (the repo axis, v0.18.0+)
### Sources (the repo axis)
A **source** is a named content repo *inside* one brain. Every `pages` row
carries a `source_id`. Slugs are unique per source, not globally.
@@ -142,7 +142,7 @@ Use this topology when:
You're senior enough to sit across multiple teams. You maintain your personal
brain (with N sources inside) AND mount several work team brains. Each team
brain is itself a multi-source brain in the v0.18.0 sense — organized
brain is itself a multi-source brain — organized
internally however the team owner chose.
```
@@ -181,7 +181,7 @@ Use this topology when:
- You need latent-space federation (agent decides when to query across
brains), not SQL federation.
Cross-brain queries are **not deterministic** in v0.19. The agent sees the
Cross-brain queries are **not deterministic**. The agent sees the
brain list and re-queries as needed. That's the feature — it keeps debugging
sane and access control clean.
@@ -202,6 +202,13 @@ WHICH BRAIN (DB)? WHICH SOURCE (repo in DB)?
Both axes follow the same layered pattern on purpose. If you know one, you
know the other.
One addition on the source axis for remote (MCP/OAuth) callers: a client
registered with federated reads carries `ctx.auth.allowedSources` — an
ARRAY of readable sources that takes precedence over the scalar
`ctx.sourceId` on every read path (`sourceScopeOpts(ctx)` in the
operations layer). Local CLI callers never set it; the scalar chain above
is the whole story for them.
---
## For agents reading this
@@ -236,7 +243,7 @@ know the other.
## Further reading
- v0.18.0 CHANGELOG — introduced `sources` primitive.
- v0.19.0 CHANGELOG (TBD after PR 0+1+2 ship) — introduces `mounts`.
- `docs/mounts/publishing-a-team-brain.md` (PR 2) — how to be the brain
publisher, not just the subscriber.
- [`topologies.md`](./topologies.md) — where the DB lives (operator recipes
for each deployment shape).
- `skills/conventions/brain-routing.md` — the agent-facing decision table.
- `CHANGELOG.md` — release history for the `sources` and `mounts` primitives.
docs/architecture/calibration-quality-gate-spec.md
+8 -6
View File
@@ -10,14 +10,16 @@
> v0.36.1.0 historical comparison semantics). Migration renumbered v74→v79→v80
> during successive master merges — v0.37.0.0's autonomous-remediation wave
> claimed v68-v78, then v0.37.1.0 (brainstorm/lsd) claimed v79.
> - **Follow-up minor** (forthcoming): falsifiability + category extraction at
> `propose_takes`, SQL-side grade gate, per-category calibration scorecards,
> pg_trgm-based proposal dedup. Wave-blocking on cat15 F1 re-validation
> against the v0.36.1.0 fixtures.
> - **Follow-up minor — NEVER IMPLEMENTED.** The falsifiability + category
> extraction at `propose_takes`, SQL-side grade gate, per-category
> calibration scorecards, and pg_trgm-based proposal dedup described in the
> sections below remain UNSHIPPED design. Do not read §§1–4 as current
> behavior; only the `unresolvable` hotfix above landed.
>
> Preserved here per the hotfix plan's PR #1191 close protocol so the
> production context (96K-page brain, 6.8% falsifiability rate, category
> breakdown) doesn't get lost in the CHANGELOG → release-notes condensation.
> production context (falsifiability rate + category breakdown observed on a
> large real brain) doesn't get lost in the CHANGELOG → release-notes
> condensation.
## Problem
docs/architecture/frontmatter-scan-incremental.md
+6 -4
View File
@@ -68,11 +68,13 @@ existing.
## Migration shape
```ts
// src/core/migrate.ts — append after the v80 entry
// src/core/migrate.ts — append after the CURRENT last entry in the
// MIGRATIONS array (take the next unused version number at implementation
// time; the numbers below are placeholders, not a reserved slot)
const migrations = [
// ...existing v1-v80...
// ...existing entries...
{
version: 81,
version: NEXT_VERSION, // next unused number in the MIGRATIONS array
name: 'frontmatter_scan_state',
sql: `
CREATE TABLE IF NOT EXISTS frontmatter_scan_state (...);
@@ -194,7 +196,7 @@ stale data as authoritative.
```
- [ ] Implement Phase 2: DB-backed frontmatter scan state.
Design lives at docs/architecture/frontmatter-scan-incremental.md.
Schema migration v81 + sync-side UPSERT + incremental scan command
New schema migration + sync-side UPSERT + incremental scan command
+ autopilot cycle phase + doctor reader. Two-phase rollout: ship
table + writes first; flip the reader one release later.
```
docs/architecture/infra-layer.md
+18 -90
View File
@@ -1,105 +1,33 @@
# GBrain Infrastructure Layer
# GBrain Infrastructure Layer (orientation pointer)
The shared foundation that all skills, recipes, and integrations build on.
This page is a router — the detailed, current-state references live in the
docs below (this file once carried its own copies of the pipeline and schema;
those rotted, so each concept now has exactly one home).
## Data Pipeline
## Where things live
```
INPUT (markdown files, git repo)
↓
FILE RESOLUTION (local → .redirect → .supabase → error)
↓
MARKDOWN PARSER (gray-matter frontmatter + body)
→ compiled_truth + timeline separation
↓
CONTENT HASH (SHA-256 idempotency check — skip if unchanged)
↓
CHUNKING (3 strategies, configurable)
├── Recursive: 300-word chunks, 50-word overlap, 5-level delimiter hierarchy
├── Semantic: embed sentences, cosine similarity, Savitzky-Golay smoothing
└── LLM-guided: Claude Haiku identifies topic shifts in 128-word candidates
↓
EMBEDDING (OpenAI text-embedding-3-large, 1536 dimensions)
→ batch 100, exponential backoff, non-fatal if fails
↓
DATABASE TRANSACTION (atomic: page + chunks + tags + version)
↓
SEARCH (hybrid, available immediately)
```
## Search Architecture
GBrain uses Reciprocal Rank Fusion (RRF) to merge vector and keyword search:
```
User Query
↓
EXPANSION (optional: Claude Haiku generates 2 alternative phrasings)
↓
├── VECTOR SEARCH (pgvector HNSW, cosine distance)
│ → 2x limit results per query variant
│
└── KEYWORD SEARCH (PostgreSQL tsvector, ts_rank)
→ 2x limit results
↓
RRF MERGE (score = Σ(1/(60 + rank)), balances both fairly)
↓
4-LAYER DEDUP
├── Best 3 chunks per page (source dedup)
├── Jaccard similarity > 0.85 (text dedup)
├── No type exceeds 60% (diversity)
└── Max 2 chunks per page (page cap)
↓
TOP N RESULTS (default 20)
```
## Key Components
| File | Purpose |
|------|---------|
| `src/core/engine.ts` | Pluggable engine interface (BrainEngine) |
| `src/core/postgres-engine.ts` | Postgres + pgvector implementation |
| `src/core/import-file.ts` | importFromFile + importFromContent pipeline |
| `src/core/sync.ts` | Git-based incremental change detection |
| `src/core/markdown.ts` | YAML frontmatter + compiled_truth/timeline parsing |
| `src/core/embedding.ts` | OpenAI embedding with batch, retry, backoff |
| `src/core/chunkers/recursive.ts` | Base chunker (300w, 5-level delimiters) |
| `src/core/chunkers/semantic.ts` | Embedding-based topic boundary detection |
| `src/core/chunkers/llm.ts` | Claude Haiku guided chunking |
| `src/core/search/hybrid.ts` | RRF merge of vector + keyword |
| `src/core/search/dedup.ts` | 4-layer result deduplication |
| `src/core/search/expansion.ts` | Multi-query expansion via Claude Haiku |
| `src/core/storage.ts` | Pluggable storage (S3, Supabase, local) |
| `src/core/operations.ts` | Contract-first operation definitions (31 ops) |
| `src/schema.sql` | Full DDL (10 tables, RLS, tsvector, HNSW) |
## Schema Overview
10 tables in Postgres:
- **pages** — slug (unique), type, title, compiled_truth, timeline, frontmatter (JSONB)
- **content_chunks** — pgvector 1536-dim embedding, chunk_source (compiled_truth|timeline)
- **links** — typed edges (knows, works_at, invested_in, founded, etc.)
- **tags** — many-to-many page tagging
- **timeline_entries** — structured events (date, source, summary, detail)
- **page_versions** — snapshot history for diff/revert
- **raw_data** — sidecar JSON from external APIs (preserves provenance)
- **files** — binary attachments in storage backend
- **ingest_log** — audit trail of import operations
- **config** — brain-level settings (version, embedding model, chunk strategy)
Full-text search uses weighted tsvector: title (A), compiled_truth (B), timeline (C).
Vector search uses HNSW index with cosine distance on content_chunks.embedding.
| Topic | Home |
|---|---|
| Ingest pipeline (file resolution → frontmatter parse → content-hash idempotency → chunking → embedding → atomic write) | per-file entries in [`KEY_FILES.md`](./KEY_FILES.md): `src/core/import-file.ts`, `src/core/sync.ts`, `src/core/markdown.ts`, `src/core/embedding.ts`, `src/core/chunkers/*` |
| Chunking strategies (recursive / semantic / LLM-guided) | `src/core/chunkers/{recursive,semantic,llm}.ts` entries in [`KEY_FILES.md`](./KEY_FILES.md) |
| Search pipeline (hybrid RRF, graph, reranker, autocut, dedup, budgets) | [`RETRIEVAL.md`](./RETRIEVAL.md) |
| Search modes + cost knobs | `docs/guides/search-modes.md` + the CLAUDE.md Search Mode table |
| Per-file index of `src/` (what each file does + its invariants) | [`KEY_FILES.md`](./KEY_FILES.md) |
| Schema DDL | the `MIGRATIONS` array in `src/core/migrate.ts` (source of truth) + `src/schema.sql`; per-table classification in [`system-of-record.md`](./system-of-record.md) |
| Engines (PGLite vs Postgres, parity rules) | `docs/ENGINES.md` + the engine entries in [`KEY_FILES.md`](./KEY_FILES.md) |
| Operations contract (CLI + MCP generated from one source) | `src/core/operations.ts` (100+ operations; run `gbrain --tools-json` for the live list) |
| Brains vs sources (which database vs which repo inside it) | [`brains-and-sources.md`](./brains-and-sources.md) |
## The Thin Harness Principle
GBrain is the deterministic layer. Skills and recipes are the latent space layer.
GBrain is the deterministic layer. Skills and recipes are the latent-space layer.
See [Thin Harness, Fat Skills](../ethos/THIN_HARNESS_FAT_SKILLS.md) for the full
architecture philosophy.
- **GBrain CLI** = thin harness (same input → same output)
- **Skills** (ingest, query, maintain, enrich, briefing, migrate, setup) = fat skills
- **Skills** (the bundled set routed by `skills/RESOLVER.md`) = fat skills
- **Recipes** (voice-to-brain, email-to-brain) = fat skills that install infrastructure
The agent reads the skill/recipe and uses GBrain's deterministic tools to do the work.
docs/architecture/lens-packs.md
+25 -29
View File
@@ -1,4 +1,4 @@
# Lens packs (v0.41.2.0)
# Lens packs
Four bundled schema packs that turn the gbrain dream cycle into a multi-lens
brain. Activate one with `gbrain config set schema_pack <name>` and the cycle
@@ -7,7 +7,7 @@ picks up the pack's declared phases on the next `gbrain dream` run.
## The four packs
```
gbrain-base (shipped v0.38)
gbrain-base
▲
│ extends
┌──────────────┼──────────────────────┐
@@ -60,37 +60,33 @@ conviction so high-stakes misses cost more).
### gbrain-engineer
Bridge-only pack. Declares `learning` page type + reuses base `code`.
No new cycle phases — the daemon-side `gstack-learnings` IngestionSource
(T8) watches `~/.gstack/projects/{repo}/learnings.jsonl` and emits
No new cycle phases — the daemon-side `gstack-learnings` IngestionSource watches `~/.gstack/projects/{repo}/learnings.jsonl` and emits
each JSONL line as a `learning` page when this pack is active. Three
calibration domains: `architecture_calls` (scalar_brier),
`effort_estimates` (weighted_brier), `risk_assessment` (scalar_brier).
Speculative ADR/postmortem/refactor_thesis/tech_debt types deferred
to v0.42+ — they'll ship when a real user authors the first one (D8).
Speculative ADR/postmortem/refactor_thesis/tech_debt types are
deferred — they'll ship when a real user authors the first one.
### gbrain-everything
Meta-pack stacking creator + investor + engineer via the v0.38
Meta-pack stacking creator + investor + engineer via the
`extends` + `borrow_from` chain. Single-active-pack constraint
preserved — this IS the active pack; the registry walks extends +
borrow to materialize the merged view.
**Merge contract (T20 / #1749).** `resolvePack` merges parent → child
(child-wins) for the six ingest/query-shaping fields: `page_types`,
`link_types`, `frontmatter_links`, `enrichable_types`, `filing_rules`,
and `takes_kinds` (unioned — a child cannot narrow it). `phases` and
`calibration_domains` are **NOT** inherited: they gate cycle execution,
so each pack must declare its own participation explicitly. That is why
`gbrain-everything` re-declares all its phases and all 7
`calibration_domains` — inheritance does not carry them.
**Merge contract.** The full `extends` + `borrow_from` merge rules live in
[`schema-packs.md` § Merge contract](./schema-packs.md#merge-contract-extends--borrow_from).
The one rule that matters here: `phases` and `calibration_domains` are
**NOT** inherited (they gate cycle execution, so each pack must declare
its own participation explicitly) — which is why `gbrain-everything`
re-declares all its phases and all 7 `calibration_domains`.
Activate via `gbrain config set schema_pack gbrain-everything` and
calibration_profile produces all 7 domain scorecards in one JSONB.
## Calibration profile widening (T10)
## Calibration profile domains
Before v0.41.2.0, `calibration_profiles.domain_scorecards` was a
`JSON.stringify({})` placeholder. v0.41.2.0 widens it: each declared
Each declared
domain produces a `{n, brier, accuracy, aggregator, page_types,
extras}` entry. Four aggregator algorithms (closed enum):
@@ -109,20 +105,20 @@ Domain names are OPEN (third-party packs can declare new domain labels
without a gbrain release). Aggregator algorithms are CLOSED (safe SQL
stays in code, validated at pack-load).
## take_domain_assignments table (T1)
## take_domain_assignments table
New JOIN table (migration v94):
JOIN table (migration v94):
`take_domain_assignments(take_id BIGINT FK, domain TEXT, pack TEXT,
source TEXT, confidence REAL, assigned_at TIMESTAMPTZ, PK(take_id,
domain))`. Multi-domain assignment honest — a take about "Sequoia's
investment in Anthropic" can land in BOTH `deal_success` AND
domain))`. Multi-domain assignment honest — a take about "fund-a's
investment in acme-example" can land in BOTH `deal_success` AND
`market_call` rather than being force-bucketed.
## What this enables for the user
- **Atoms + concepts ship in the binary.** Your OpenClaw's parallel
atom-pipeline-coordinator + atom-backfill-coordinator + concept-
synthesis crons can retire (T12 follow-up). One `gbrain dream` cron
synthesis crons can retire. One `gbrain dream` cron
covers everything.
- **gstack learnings reach gbrain.** Engineer-pack-active brains
surface every gstack-logged learning as a queryable page within
@@ -131,22 +127,22 @@ investment in Anthropic" can land in BOTH `deal_success` AND
often you're wrong on deals AND market calls AND architecture
AND effort estimates in one `gbrain calibration --json` call.
- **Lossless OpenClaw migration.** The `markdown-greenfield`
importer (T7, mode='migration') re-ingests existing OpenClaw
importer (mode='migration') re-ingests existing OpenClaw
pages with permanent slug-keyed idempotency + per-row JSONL audit
+ the `imported_from` marker so extract_atoms + synthesize_concepts
don't re-extract already-atomized material.
## v0.41.2.1 follow-ups (filed in plan)
## Known gaps / deferred follow-ups
- Per-page-type `frontmatter_validators` on PageTypeSchema so the
atom_type enum (currently hardcoded in extract_atoms.ts) reads from
the active pack manifest at runtime per D11.
the active pack manifest at runtime.
- 3-check quality gate (truism / punchline / entity-page reject) as
a multi-pass extract_atoms refinement.
- Embedding-similarity dedup in synthesize_concepts (currently
exact-string concept ref match only).
- Voice gate integration for T1 Canon narratives.
- Voice gate integration for concept narratives.
- op_checkpoint resumability for cross-cycle continuation in both
phases.
- Parity-baseline eval gates against your OpenClaw's existing 13K atoms
+ 11K concepts on a 500-page sample subset.
- Parity-baseline eval gates against a pre-existing downstream
atom/concept corpus on a sample subset.
docs/architecture/pack-upgrade-mechanism.md
+11 -11
View File
@@ -127,10 +127,10 @@ candidate ≠ the active pack name, loads the manifest via
migration_from.version)`. Returns matching packs sorted by version
descending.
v0.41.22 covers bundled packs only. v0.43+ TODO: enumerate user-installed
packs at `~/.gbrain/schema-packs/*/pack.yaml` (defer to v0.43 since the
filesystem-scan cost needs the cache invalidation strategy from
`registry.ts`).
Successor detection covers bundled packs only. Future work: enumerate
user-installed packs at `~/.gbrain/schema-packs/*/pack.yaml` (deferred
because the filesystem-scan cost needs the cache invalidation strategy
from `registry.ts`).
## The manual_only apply policy
@@ -173,8 +173,8 @@ migration_from:
version: "1.x"
page_types:
# Inherit gbrain-base-v2's 15 types here (or use extends to merge
# automatically once v0.43+ extends-chain composition lands)
# Inherit gbrain-base-v2's 15 types here (or declare `extends:
# gbrain-base-v2` and let the merge contract in schema-packs.md merge them)
- { name: person, primitive: entity, path_prefixes: [people/], expert_routing: true }
- { name: company, primitive: entity, path_prefixes: [companies/], expert_routing: true }
# ... all 13 other v2 canonicals ...
@@ -222,17 +222,17 @@ Every unify run writes to `~/.gbrain/audit/schema-unify-YYYY-Www.jsonl`
identities (before + after), per-phase counts (would_apply + applied),
warnings, completion timestamp. Privacy: page slugs are NOT logged in
bulk (only the per-rule sample_slugs[≤10]); for forensic debugging
add `GBRAIN_AUDIT_FULL=1` (v0.43+ TODO; not yet wired).
a `GBRAIN_AUDIT_FULL=1` escape hatch has been proposed but is not yet wired.
## What's NOT yet supported
- Subprocess sandbox for the publish-gate (v0.43+ TODO)
- Subprocess sandbox for the publish-gate
- Per-source pack-upgrade (the handler accepts `sourceId` but
`findPackSuccessors` doesn't yet pass it through)
- Cross-brain federated mounts that disagree on canonical packs
- Automatic rollback (today: manual SQL or `gbrain restore`)
- LLM-assisted mapping_rules codegen from production data (`gbrain
schema detect-mappings`; deferred to v0.43+)
- LLM-assisted mapping_rules codegen from production data (a proposed
`gbrain schema detect-mappings`)
## Reference
@@ -242,6 +242,6 @@ add `GBRAIN_AUDIT_FULL=1` (v0.43+ TODO; not yet wired).
- Onboard check: `src/core/onboard/checks.ts:checkPackUpgradeAvailable`
- Render allowlist: `src/core/onboard/render.ts:MANUAL_ONLY_PROTECTED_JOBS`
- Handler: `src/core/schema-pack/unify-types-handler.ts`
- Migration: `src/core/migrate.ts:105` (slug_aliases table)
- Migration: the `slug_aliases` entry in `src/core/migrate.ts`'s `MIGRATIONS` array
- Type taxonomy doc: `docs/architecture/type-taxonomy.md`
- Skill: `skills/schema-unify/SKILL.md`
docs/architecture/schema-packs.md
+54 -49
View File
@@ -7,20 +7,25 @@ paths, and which link verbs connect what to what. The schema pack is the
querying, or routing experts. It is the single source of truth for
"what's in your brain."
The v0.39.0.0 wave shipped a full schema-pack cathedral. This doc is the
user-facing reference; for implementation details see
`docs/designs/V038_SCHEMA_PACKS.md` (CEO plan) and the engine layer in
`src/core/schema-pack/`.
This doc is the user-facing reference; for implementation details see
`docs/designs/V038_SCHEMA_PACKS.md` (the original design) and the engine
layer in `src/core/schema-pack/`.
## What ships in the box
Two bundled packs:
Seven bundled packs (`src/core/schema-pack/base/`):
- **`gbrain-base`** (default) — reproduces pre-v0.38 hardcoded behavior
byte-for-byte. Existing brains see zero behavior change after upgrade.
Covers: person, company, deal, meeting, project, place, concept, writing,
analysis, guide, hardware, architecture, etc. (the original
`ALL_PAGE_TYPES` list).
- **`gbrain-base-v2`** — the 15-type canonical taxonomy. Fresh installs
(`gbrain init`) activate this by default. See
[`type-taxonomy.md`](./type-taxonomy.md) for the full type list and the
upgrade path from `gbrain-base`.
- **`gbrain-base`** — the original hardcoded behavior, byte-for-byte
(person, company, deal, meeting, project, place, concept, writing,
analysis, guide, hardware, architecture, etc. — the original
`ALL_PAGE_TYPES` list). Still the resolution-chain fallback (tier 7)
for brains with no pack configured anywhere, so pre-existing brains see
zero behavior change until they opt in to something newer.
- **`gbrain-recommended`** — extends `gbrain-base` with the 13 additional
directories described in `docs/GBRAIN_RECOMMENDED_SCHEMA.md`: deal,
@@ -32,12 +37,17 @@ Two bundled packs:
gbrain schema use gbrain-recommended
```
- **`gbrain-creator`**, **`gbrain-investor`**, **`gbrain-engineer`**,
**`gbrain-everything`** — the lens packs, which add cycle phases and
calibration domains on top of the base taxonomy. See
[`lens-packs.md`](./lens-packs.md).
Plus user-installed packs at `~/.gbrain/schema-packs/<name>/pack.yaml`
that you author with `gbrain schema init` or `gbrain schema fork`.
## CLI surface
Five inspection verbs (shipped in v0.38):
Inspection verbs:
```bash
gbrain schema active # show resolved pack + which tier set it
@@ -47,7 +57,7 @@ gbrain schema validate # validate a manifest's shape
gbrain schema use <pack> # activate a pack (writes ~/.gbrain/config.json)
```
Eight authoring + discovery verbs (shipped in v0.39):
Authoring + discovery verbs:
```bash
gbrain schema detect # propose types matching brain shape
@@ -62,12 +72,12 @@ gbrain schema graph # ASCII type listing (experimental)
gbrain schema lint # flag duplicates + missing prefixes
gbrain schema explain <type> # plain-English type description (experimental)
gbrain schema downgrade --to <p> # restore previous pack (recovery)
gbrain schema usage --since 30d # per-verb invocation counts (D14 telemetry)
gbrain schema usage --since 30d # per-verb invocation counts (telemetry)
```
The verbs marked `experimental` are demand-gated per D14: their usage is
tracked via T15's schema-events audit, and v0.40+ retro decides whether
to deprecate any that stay <5% usage.
The verbs marked `experimental` are demand-gated: usage is tracked via the
schema-events audit (`gbrain schema usage`), which informs whether
rarely-used verbs get deprecated.
## Resolution chain (7 tiers)
@@ -78,10 +88,10 @@ this chain top-down. First match wins.
|------|--------|-------|
| 1 | Per-call `schema_pack` opt | CLI only (`ctx.remote === false`); MCP rejected. |
| 2 | `GBRAIN_SCHEMA_PACK` env | Process-scope override. |
| 3 | Per-source DB config key `schema_pack:source:<id>` | New in v0.38. |
| 3 | Per-source DB config key `schema_pack:source:<id>` | |
| 4 | Brain-wide DB config key `schema_pack` | |
| 5 | `gbrain.yml schema:` section | Repo-checked. |
| 6 | `~/.gbrain/config.json` `schema_pack` field | What `gbrain schema use` writes. |
| 6 | `~/.gbrain/config.json` `schema_pack` field | What `gbrain schema use` (and `gbrain init`, which sets `gbrain-base-v2`) writes. |
| 7 | Default: `gbrain-base` | Always present. |
## How the agent uses the active pack
@@ -97,18 +107,18 @@ Every read + write path consults the active pack at runtime:
- **`extract_facts`** runs only on `extractable: true` types.
- **`enrichment-service`** routes person/company enrichment based on the
pack's primitive declarations.
- **Search hybrid cache** (`knobsHash`) folds in pack name + version
(v0.39 T21). A cache row written under pack A is unreachable when pack
- **Search hybrid cache** (`knobsHash`) folds in pack name + version.
A cache row written under pack A is unreachable when pack
B is active. Cross-pack contamination is structurally impossible.
## The magical moment (T2-T4 + T10)
## The magical moment
Persona A (Notion refugee) installs gbrain, imports her exports, and the
brain looks unfamiliar — the default `gbrain-base` pack expects
`people/`, `companies/`, etc., but her files live under `Projects/`,
`Reading/`, `Daily Notes/`. The friction signal fires in two places:
1. **Import warn (T7):** the end of `gbrain import` prints
1. **Import warn:** the end of `gbrain import` prints
`[schema] X of Y pages (Z%) have no type matching the active schema
pack. Run gbrain schema detect to propose a pack matching your
content shape.`
@@ -124,7 +134,7 @@ gbrain schema review-candidates # human gate on promotion
gbrain schema review-candidates --apply Projects/ # accept
```
The agent (via the new EIIRP skill) automates phases 1-3 of this for any
The agent (via the EIIRP skill, `skills/eiirp/SKILL.md`) automates phases 1-3 of this for any
significant work session. The brain's schema becomes a living artifact
the agent maintains, not a hardcoded ceremony the user authors.
@@ -172,9 +182,10 @@ filing_rules: []
## Merge contract (`extends` + `borrow_from`)
This section is the single home for the merge rules (other docs link here).
`resolvePack` composes a pack against its `extends` chain (and any
`borrow_from` targets) into the `resolved.manifest` every consumer reads
(T20 / #1749). The rules:
`borrow_from` targets) into the `resolved.manifest` every consumer reads.
The rules:
- **Six fields inherit, child-wins:** `page_types`, `link_types`,
`frontmatter_links`, `enrichable_types`, `filing_rules`, and `takes_kinds`.
@@ -200,9 +211,8 @@ filing_rules: []
## Recovery + revert
The single-PR cathedral is hard to revert atomically. Per codex finding
#4 from plan-eng-review, T20 ships `gbrain schema downgrade` to restore
the active-pack config field:
A pack activation is config, not code, so reverting code alone doesn't
undo it. `gbrain schema downgrade` restores the active-pack config field:
```bash
gbrain schema downgrade --to gbrain-base
@@ -214,19 +224,19 @@ gbrain schema downgrade
1. `git revert <merge-commit>` — restores the code.
2. `gbrain schema downgrade --to gbrain-base` — restores config.
3. (Optional) `gbrain purge-deleted --older-than 0h` — drops
v0.39-typed pages that no longer have a matching type in the active
3. (Optional) `gbrain pages purge-deleted --older-than 0h` — hard-deletes
soft-deleted pages that no longer have a matching type in the active
pack.
The cache + eval rows that pack-aware code wrote are isolated by the
`knobsHash` pack-folding (T21) — they become unreachable under the
`knobsHash` pack-folding — they become unreachable under the
restored pack so no eviction is needed.
## Distribution
`.gbrain-schema` tarballs ride the same v0.37 skillpack pipeline as
`.gbrain-skillpack` tarballs (T14 artifact abstraction). The
discriminator is `api_version` in the manifest:
`.gbrain-schema` tarballs ride the same distribution pipeline as
`.gbrain-skillpack` tarballs. The discriminator is `api_version` in the
manifest:
- `gbrain-schema-pack-v1` → schemapack
- `gbrain-skillpack-v1` → skillpack
@@ -237,22 +247,17 @@ respectively.
Publication to the public registries (`garrytan/gbrain-schema-registry`,
`garrytan/gbrain-skillpack-registry`) follows the same publish-as-PR
workflow as v0.37 skillpack publishing.
workflow as skillpack publishing.
## What's deferred to v0.40+
## Known limits / deferred work
- **Per-source pack federation across mounts.** A query crossing multiple
sources currently rejects with `permission_denied` when those sources
have divergent active packs (T19 + codex finding #2). The v0.40+ work
computes a true per-source closure via the existing
`buildSourceClosureCte` engine surface.
- **`extends` chain semver compatibility checks** between pack versions.
- **`skillpack ↔ schemapack` cross-reference declarations** — a skillpack
can declare "I work best with these primitives present in your pack."
- **Live schema migration helpers** — when you add a type, auto-suggest
backfill of existing pages.
- **Authoring vs derivation thesis reframe (D14).** v0.39.0.0 ships the
full 11-verb cathedral with 6 verbs marked experimental-tier. v0.40+
retro reads T23 usage telemetry to decide which to deprecate.
sources rejects with `permission_denied` when those sources have
divergent active packs (`src/core/schema-pack/op-trust-gate.ts`). A true
per-source closure via the existing `buildSourceClosureCte` engine
surface remains future work.
- **Pack version upgrades** (e.g. `gbrain-base` → `gbrain-base-v2`) are
handled by the successor-detection + unify-types mechanism — see
[`pack-upgrade-mechanism.md`](./pack-upgrade-mechanism.md).
See `TODOS.md` v0.40+ section for the full deferred list.
The live deferred list is in `TODOS.md`.
docs/architecture/serve-sync-concurrency.md
+10
View File
@@ -52,3 +52,13 @@ gbrain sync --no-schema-pack --no-pull --no-embed --yes
`gbrain schema lint` flags the classic nested-quantifier ReDoS shapes
(`(a+)+`, `(a*)*`, …) in pack regexes as warnings.
The manual diagnosis above has an automated cousin: the progress-aware stall
watchdog. If the import drain makes no forward progress for
`GBRAIN_SYNC_STALL_ABORT_SECONDS` (default 900; keyed on file-import
progress, not the lock heartbeat), the run aborts with
`reason: 'stall_timeout'` and releases the per-source lock so the next
`gbrain sync` resumes from the checkpoint. It fires BETWEEN files — a hang
inside one file's import runs until the wall-clock hard deadline. `0`
disables it. The full sync-resumability knob table lives in CLAUDE.md
("Sync resumability + lock tuning").
docs/architecture/system-of-record.md
+1 -3
View File
@@ -88,6 +88,7 @@ the repo. The architectural rule still holds — these aren't
| `eval_candidates` / `eval_capture_failures` | Contributor-mode dev loop; opt-in capture. |
| `dream_verdicts` | Cheap verdict cache. Rebuildable by re-running Haiku. |
| `gbrain_cycle_locks` / migration ledger | Infrastructure. |
| `op_checkpoint_paths` | Sync-resume checkpoint. Append-only progress banking; a completed sync makes it irrelevant. |
| `config` (some keys) | Site-local routing config (e.g. `sync.repo_path`). |
A new derived table that holds user-knowledge MUST land FS-first.
@@ -189,9 +190,6 @@ reconciler / migration layer without the explicit allow-list comment.
## Related
- `~/.claude/plans/system-instruction-you-are-working-expressive-pony.md`
— the v0.32.2 design plan (decisions D1-D22 + Q1-Q8, Codex round 1
and round 2 finds)
- `skills/migrations/v0.32.2.md` — the agent-facing migration guide
- `CHANGELOG.md` v0.32.2 entry — the release manifesto
- `scripts/check-system-of-record.sh` — the CI gate that enforces
docs/architecture/thin-client.md
+38 -44
View File
@@ -3,68 +3,62 @@
On-demand reference (see CLAUDE.md Reference map). Current behavior + invariants
only; release history lives in `CHANGELOG.md` + git.
`gbrain init --mcp-only` (v0.29.2) sets up a thin-client install: no local
brain content, just an OAuth client pointing at a remote `gbrain serve --http`.
v0.29.2/v0.30.0 only refused 9 obvious local-only commands; the other ~25
silently fell through to `connectEngine()` and opened the empty local PGLite,
returning "No results." against a populated remote brain. v0.31.1 fixes the
silent-empty-results bug class for every operation surface.
`gbrain init --mcp-only` sets up a thin-client install: no local brain content,
just an OAuth client pointing at a remote `gbrain serve --http`. Every operation
surface routes through the remote brain — a thin-client install never opens the
empty local PGLite, so a populated remote brain can't silently return
"No results." Local-only commands refuse with a pinpoint hint instead of
falling through.
Key files:
Key files (per-file detail lives in each file's `KEY_FILES.md` entry; this doc
carries the routing-seam picture):
- `src/cli.ts` — Routing seam INSIDE the existing op-dispatch path (CDX-1: no
- `src/cli.ts` — Routing seam INSIDE the existing op-dispatch path (no
parallel `src/core/thin-client/` module; routing is a ~80-line conditional
in `runThinClientRouted`). Detects `isThinClient(cfg)` BEFORE `connectEngine`
so thin-client installs never open the empty PGLite. localOnly ops on
thin-client refuse via `refuseThinClient` (with pinpoint hint table
`THIN_CLIENT_REFUSE_HINTS`). Banner via `printIdentityBannerBestEffort`
`THIN_CLIENT_REFUSE_HINTS`, which covers the full DB-bound command surface —
sync, embed, extract, migrate, enrich, dream, jobs, sources, pages, files,
eval, code-*, and more). Banner via `printIdentityBannerBestEffort`
before each routed call (suppressed by `--quiet`, `GBRAIN_NO_BANNER=1`,
non-TTY default). Exhaustive TS `never` switch on `RemoteMcpError.reason`
for canned, actionable error messages. ENG-2 renderer parity: local-engine
for canned, actionable error messages. Renderer parity: the local-engine
path runs `JSON.parse(JSON.stringify(result))` so renderers see the same
shape on both paths (kills Date/bigint/Buffer drift class).
- `src/core/mcp-client.ts` — `callRemoteTool(config, toolName, args, opts)`.
Hardened in v0.31.1 (CDX-4): all transport errors normalized to
`RemoteMcpError` via the `toRemoteMcpError` funnel. New `CallRemoteToolOptions
{timeoutMs, signal}`; `buildAbortController` composes external signal with
timeout. New `RemoteMcpErrorReason` stable union, `RemoteMcpErrorDetail.kind`
('timeout' | 'aborted' | 'unreachable') sub-tag, `RemoteMcpErrorDetail.code`
field carrying server-supplied error codes (e.g. `missing_scope`).
`extractToolErrorCode` parses JSON envelopes first, falls back to substring
detection for legacy server messages. `unpackToolResult<T>(res)` unchanged
(parses tool-call JSON content). `_clearMcpClientTokenCache()` test escape.
- `src/core/cli-options.ts` — `parseGlobalFlags` adds `--timeout=Ns` (accepts
`30s`, `2m`, `500ms`, plain ms). Default `null` = per-command default (30s
for most ops, 180s for `think`). `parseTimeout(s)` exported helper.
- `src/core/doctor-remote.ts` — `gbrain remote doctor` adds the
`oauth_client_scopes_probe` check (CDX-5). Probes the read tier via
`get_brain_identity` and admin tier via `get_health`; reports per-tier
shape on both paths (kills the Date/bigint/Buffer drift class).
- `src/core/mcp-client.ts` — `callRemoteTool(config, toolName, args, opts)`,
the transport under the routing seam. All transport errors normalize to
`RemoteMcpError` via the `toRemoteMcpError` funnel, with a stable
`RemoteMcpErrorReason` union the dispatcher's `never` switch keys off.
Full symbol-level detail: the `src/core/mcp-client.ts` entry in
[`KEY_FILES.md`](./KEY_FILES.md).
- `src/core/cli-options.ts` — `parseGlobalFlags` supports `--timeout=Ns`
(accepts `30s`, `2m`, `500ms`, plain ms). Default `null` = per-command
default (30s for most ops, 180s for `think`). `parseTimeout(s)` exported
helper.
- `src/core/doctor-remote.ts` — `gbrain remote doctor` includes the
`oauth_client_scopes_probe` check. Probes the read tier via
`get_brain_identity` and the admin tier via `get_health`; reports per-tier
status with pinpoint remediation when admin is missing. `buildScopeCheck`
+ `ScopeProbeResult` exported for test access. Skippable via
`GBRAIN_DOCTOR_SKIP_SCOPE_PROBE=1` for fixtures that mock /mcp at JSON-RPC
initialize level only (MCP SDK Client hangs on shape mismatch).
- `src/core/ssrf-validate.ts` (v0.36 Commit 0) — DNS-rebinding-defended URL validation. `validateAndResolveUrl(url)` resolves the hostname via `dns.lookup({all: true, family: 0})`, checks EVERY A AND AAAA record against the internal-IP deny list, returns the resolved IP so callers fetch by IP (defeats DNS rebinding: validation IP === fetch IP). `fetchWithSSRFGuard(url, opts)` does redirect-aware fetching with per-hop re-validation, max 3 hops by default. Reusable across all URL-fetching features. Test seam `__setDnsLookupForTests` for hermetic tests.
- `src/core/search/query-intent.ts` extension (v0.36 cross-modal wave) — new `suggestedModality: 'text' | 'image' | 'both'` axis on `QuerySuggestions`. Module-scope `CROSS_MODAL_PATTERNS` regex array (compiles once at module load). `isAmbiguousModalityQuery(query)` heuristic gate fires when a visual noun + reference marker combination indicates genuinely ambiguous routing — used by the Commit 4 LLM tie-break to bound LLM calls to <1% of queries.
- `src/core/search/mode.ts` extension (v0.36 cross-modal wave) — `ModeBundle` extended with 7 cross-modal knobs: `cross_modal_both_text_weight` / `cross_modal_both_image_weight` (D6 weighted RRF for `'both'` mode, defaults 0.6/0.4), `image_query_text_refinement_weight` / `image_query_image_refinement_weight` (D13 hybrid intersect for `searchByImage` query refinement, defaults 0.4/0.6), `unified_multimodal` + `unified_multimodal_only` (Phase 3 unified column routing flags), `cross_modal_llm_intent` (Commit 4 opt-in escalation). `SEARCH_MODE_CONFIG_KEYS` extended with 7 corresponding config keys. `KNOBS_HASH_VERSION` bumped 2→3 (D2 — closes the silent cache-hit class where a cached text-mode result could leak to an image-mode caller).
- `src/core/search/hybrid.ts` extension (v0.36 cross-modal wave) — cross-modal routing branch at the embed step. Resolves `effectiveModality` from per-call `opts.crossModal` (normalized: literal `'auto'` → undefined per D22-1) → `suggestions.suggestedModality` → `'text'` default. Image route: `embedQueryMultimodal` + `searchVector({embeddingColumn: 'embedding_image'})`, skip expansion + keyword (D9 mode-bundle override). 'both' route: parallel text + image vector searches merged via `rrfFusionWeighted` with `effectiveRrfK(baseRrfK, weight)` from the configured cross-modal weights. Phase 3 unified routing fires when `cfg.search.unified_multimodal === true` — bypasses dual-column branching, runs `embedQueryMultimodal` + `searchVector({embeddingColumn: 'embedding_multimodal'})`, D8 fail-open on zero rows + not strict-mode falls through to dual-column. Commit 4 LLM escalation fires only when (no explicit per-call opt) AND (regex returned 'text') AND (`cfg.search.cross_modal.llm_intent` is true) AND (`isAmbiguousModalityQuery` returns true). Fail-open on every error.
- `src/core/search/image-loader.ts` (v0.36 Phase 2) — `loadImageInput(input, opts)` accepts local path, `data:` URI, or `http(s)://` URL. Magic-byte sniff for PNG/JPEG/WebP. Hard size cap (default 10 MB, configurable via `search.image_query.max_bytes`). For URLs: routes through `fetchWithSSRFGuard` so DNS rebinding + redirect chains are defeated. Pre-flight Content-Length check + post-fetch size guard for lying servers. `ImageLoadError` with discriminated `code` (INVALID_FORMAT / OVERSIZED / INVALID_URL / FETCH_FAILED / TIMEOUT / SSRF_BLOCKED / NOT_FOUND).
- `src/core/search/by-image.ts` (v0.36 Phase 2) — `searchByImage(engine, input, opts)`. Always runs image branch (`embedQueryMultimodalImage` + `searchVector(embedding_image)`). D13 hybrid intersect: when caller provides optional `query`, runs parallel text branch via `embedQueryMultimodal(query)` and merges via `rrfFusionWeighted` with weights from resolved mode. Phase 3 widens to unified column once `search.unified_multimodal=true` (transparently upgrades the retrieval quality post-reindex).
- `src/core/spend-log.ts` (v0.36 Phase 2 D23-#6) — per-OAuth-client paid-API spend tracking against the `mcp_spend_log` table (migration v74). `checkBudget(engine, clientId, capCents)` is the pre-flight gate; throws `BudgetExceededError` when today's spend has hit the cap. `recordSpend(engine, entry)` is best-effort post-call. UTC day-aligned aggregation so caps roll over deterministically regardless of server timezone. Local CLI callers (no clientId) bypass the gate. Pre-v0.36 brains without the table fail open to spend=0. `VOYAGE_MULTIMODAL_3_PER_IMAGE_CENTS` = 0.12 cents per image embed.
- `src/core/search/llm-intent.ts` (v0.36 Commit 4) — opt-in LLM tie-break. `classifyModalityWithLLM(query, fallback)` routes through `gateway.chat()` with a fixed single-word-output system prompt. 1s timeout via AbortController. `parseModality(raw, fallback)` is the pure parser — tolerates trailing punctuation + casing. Fail-open on every error (gateway unavailable, timeout, parse failure, unrecognized output) — returns fallback so a misbehaving LLM can never break search. Cost-bounded by the ambiguity heuristic in `query-intent.ts` (fires <1% of queries when on).
- `src/commands/reindex-multimodal.ts` (v0.36 Phase 3) — `gbrain reindex --multimodal [--limit N] [--dry-run] [--cost-estimate] [--no-embed] [--yes] [--json]`. Walks `content_chunks WHERE embedding_multimodal IS NULL`, batches via `embedMultimodalSafe` (Commit 0 partial-failure-aware), persists. D7 lock acquisition via `tryAcquireDbLock('gbrain-reindex-multimodal', 360min)`. Cost prompt + 10s Ctrl-C grace window in TTY. `GBRAIN_NO_REEMBED=1` bypass. Checkpoint at `~/.gbrain/reindex-multimodal-checkpoint.json` for resume. D23-#2 auto-flip prompt at coverage=100% completion (TTY: interactive; non-TTY: stderr hint with paste-ready command).
- `src/core/backfill-registry.ts` extension (v0.36) — new `modality` backfill kind. SQL filter requires `chunk_source='image_asset'` AND `embedding_image IS NOT NULL` AND `(modality IS NULL OR modality != 'image')`. D22-7 defensive guard: never flag a non-image chunk that happens to have `embedding_image` populated. Idempotent — second run finds zero rows.
- `src/core/migrate.ts` v74 (`mcp_spend_log`) + v75 (`embedding_multimodal_column`) — Phase 2 spend-log table + Phase 3 unified column ALTER. v75 is column-only (no HNSW index — deferred to post-reindex per pgvector best practice). v74 uses BTREE on `(client_id, created_at)` + `(token_name, created_at)` — `date_trunc('day', TIMESTAMPTZ)` is NOT IMMUTABLE so can't appear in index expressions; range scan on created_at covers the per-day rollup query.
- `src/core/operations.ts` — `get_brain_identity` op (read scope, no params,
banner-only): cheap counter packet `{version, engine, page_count,
chunk_count, last_sync_iso}` for the thin-client identity banner. Reuses
`engine.getStats()`; banner's 60s client-side TTL bounds frequency to
≤1/60s per CLI process (well below the Fly.io health-check cadence that
motivated the original `getStats` cost warning).
`engine.getStats()`; the banner's 60s client-side TTL bounds frequency to
≤1/60s per CLI process.
- `src/commands/{salience,anomalies,graph-query,think}.ts` — Per-command
thin-client routing branches. These commands bypass the operation-layer
dispatch in cli.ts (call `engine.foo()` directly), so each gets its own
`if (isThinClient(cfg)) { callRemoteTool(...) }` branch that maps CLI flags
to op params. `think` is a special case: the server's `think` op
intentionally disables `--save`/`--take` for remote callers
(operations.ts:1103-1135 trust-boundary gate); thin-client `think` warns
loudly when those flags are set.
intentionally disables `--save`/`--take` for remote callers (the
`safeSave`/`safeTake` trust-boundary gate in the `think` handler in
`operations.ts`); thin-client `think` warns loudly when those flags are set.
Cross-modal search files (image query, SSRF-guarded image loading, spend
tracking, multimodal reindex) are indexed per-file in
[`KEY_FILES.md`](./KEY_FILES.md) and described behaviorally in
[`RETRIEVAL.md`](./RETRIEVAL.md) — they are not part of the thin-client
routing seam.
docs/architecture/topologies.md
+10 -5
View File
@@ -108,11 +108,13 @@ instead of a local DB connection:
}
```
The CLI dispatch guard refuses any DB-bound command (`sync`, `embed`,
`extract`, `migrate`, `apply-migrations`, `repair-jsonb`, `orphans`,
`integrity`, `serve`) on a thin-client install with a clear error pointing
at the remote host. `gbrain doctor` runs a dedicated thin-client check set
(OAuth discovery, token round-trip, MCP smoke).
The CLI dispatch guard refuses every DB-bound command (`sync`, `embed`,
`extract`, `migrate`, `serve`, `enrich`, `jobs`, `sources`, `pages`,
`files`, `eval`, and the rest of the local-only surface — the full hint
table is `THIN_CLIENT_REFUSE_HINTS` in `src/cli.ts`) on a thin-client
install with a clear error pointing at the remote host. `gbrain doctor`
runs a dedicated thin-client check set (OAuth discovery, token round-trip,
MCP smoke). See [`thin-client.md`](./thin-client.md) for the routing seam.
### Setup
@@ -394,6 +396,9 @@ simultaneously — that's by design.
## See also
- `docs/guides/bootstrap.md` — `gbrain bootstrap`, the paved-road paste-in
install for Topology 1 with a desktop coding agent (interview, hooks,
MCP registration, verify).
- `docs/architecture/brains-and-sources.md` — in-brain organization (brains
vs sources axes).
- `docs/mcp/CLAUDE_DESKTOP.md` and siblings — per-client MCP setup.
docs/architecture/type-taxonomy.md
+8 -7
View File
@@ -1,7 +1,7 @@
# Type Taxonomy (v0.41.22: gbrain-base-v2)
# Type Taxonomy (gbrain-base-v2)
> The 14-canonical-type DRY/MECE taxonomy shipped in v0.41.22. Predecessor
> `gbrain-base` (24 types) stays bundled for back-compat; v0.42+ installs
> The 14-canonical-type DRY/MECE taxonomy. Predecessor
> `gbrain-base` (24 types) stays bundled for back-compat; fresh installs
> default to `gbrain-base-v2`.
## Why
@@ -79,7 +79,7 @@ gbrain jobs submit unify-types \ # PROTECTED + manual_only
--params '{"target_pack":"gbrain-base-v2","apply":true}'
# omit "apply":true → dry-run (default)
↓
Handler runs 4 phases:
Handler runs 8 phases:
┌─────────────────────────────────────┐
│ Phase 1: Preflight + lock │ → gbrain-unify db-lock (60min TTL)
├─────────────────────────────────────┤
@@ -165,8 +165,10 @@ explicitly disambiguated this as canonical, so it should outrank fuzzy
matches that hit aliases by accident."
`SearchResult.alias_resolved_boost` is stamped on touched results for
`--explain` formatter visibility. KNOBS_HASH_VERSION bumped 5→6 to
invalidate pre-v0.42 cache rows that don't reflect the new stage.
`--explain` formatter visibility. The stage participates in the search
cache key (`KNOBS_HASH_VERSION` in `src/core/search/mode.ts` is the
single source of truth for the current cache-key version), so cache rows
written before the stage existed are unreachable.
## Reference
@@ -176,4 +178,3 @@ invalidate pre-v0.42 cache rows that don't reflect the new stage.
- Migration handler: `src/core/schema-pack/unify-types-handler.ts`
- Onboard checks: `src/core/onboard/checks.ts`
- Skill: `skills/schema-unify/SKILL.md`
- Plan + decisions: `~/.claude/plans/system-instruction-you-are-working-transient-elephant.md`
docs/contradictions.md
+21 -19
View File
@@ -1,4 +1,4 @@
# gbrain eval suspected-contradictions (v0.32.6)
# gbrain eval suspected-contradictions
The contradiction probe samples retrieval results, asks an LLM judge whether
any pair contradicts on a factual claim relevant to the user's query, and
@@ -142,25 +142,27 @@ pay near-zero on re-runs (until you bump PROMPT_VERSION).
gate makes accidental private-data commits hard, but the operator MUST
inspect every redaction before commit.
## Temporal axis
The judge distinguishes real contradictions from legitimate change-over-time.
The verdict enum has six members (`no_contradiction | contradiction |
temporal_supersession | temporal_regression | temporal_evolution |
negation_artifact`), and `pages.effective_date` is threaded into the judge
prompt so the probe doesn't cry wolf on facts that simply changed.
The trajectory substrate builds on the same signal:
`gbrain eval trajectory <entity>` shows the chronological typed-claim
history with regressions flagged inline; `gbrain founder scorecard
<entity>` rolls up four signals (accuracy, consistency, growth
direction, red flags) into a stable JSON contract. MCP op
`find_trajectory` (read scope, visibility-filtered for remote callers)
exposes the same data to agents. The probe's `temporal_supersession`
verdict and the consolidate phase's `valid_until` writeback both
preserve the `auto-supersession.ts` "NEVER auto-applies" invariant
— the probe only emits paste-ready commands; only `consolidate`
writes `valid_until` (a grep guard pins this).
## See also
- Plan: `~/.claude/plans/system-instruction-you-are-working-hashed-dewdrop.md`
- CHANGELOG: `## [0.32.6]` entry covers the whole release.
- Cost discipline: `docs/eval-bench.md` for the recommended nightly cadence
+ trend-tracking workflow.
- **Temporal axis follow-on (v0.35.3.1 + v0.35.7):** v0.35.3.1 added a
six-member verdict enum (`no_contradiction | contradiction |
temporal_supersession | temporal_regression | temporal_evolution |
negation_artifact`) and threaded `pages.effective_date` into the judge
prompt so the probe stops crying wolf on legitimate change-over-time.
v0.35.7 lands the trajectory substrate the probe pointed at:
`gbrain eval trajectory <entity>` shows the chronological typed-claim
history with regressions flagged inline; `gbrain founder scorecard
<entity>` rolls up four signals (accuracy, consistency, growth
direction, red flags) into a stable JSON contract. MCP op
`find_trajectory` (read scope, visibility-filtered for remote callers)
exposes the same data to agents. The probe's `temporal_supersession`
verdict and the consolidate phase's `valid_until` writeback both
preserve the `auto-supersession.ts:4` "NEVER auto-applies" invariant
— the probe still emits paste-ready commands, only `consolidate`
writes `valid_until` (R1+R8 grep guard pins this).
docs/designs/AGENT_BOOTSTRAP_DESIGN.md
+7 -3
View File
@@ -68,7 +68,8 @@ open >10 days from first code commit)
0. **Spike + quota gate** (manual, gates door-1 ship; per-harness quota measured; a
p90 day must fit ≤10% of weekly subscription quota or schedule scope is cut).
1. **Shared body + engine machinery:** `gbrain bootstrap` family, templates, format
spec, secret-scan-gated persistence, verify.
spec, secret-scan-gated persistence, verify, uninstall (v1 via CEO-review
expansion; receipt-keyed scope per the PLAN's CX2-12).
2. **Codex door ships first** (runbook variant + approvals preflight + capability
probe; CLI path not spike-gated).
3. **Claude Code door:** hooks, IPC turn_context, transcript ingestion, greeting
@@ -111,8 +112,11 @@ Paste block + tag-pinned runbook (`BOOTSTRAP_FOR_AGENTS.md`, fetched at the
`latest-stable` ref — advanced by the release job only after assets publish, so
published copies never rot); optional GitHub template repo (generated at release from
the same renderer); binary via `bun install -g github:garrytan/gbrain#latest-stable`
(never npm). The paste block lives in the README's "Quick start: Claude Code or
Codex" section; `INSTALL_FOR_AGENTS.md` remains the paste path for agent platforms.
(never npm). The paste block lives in the README's `## Install` section, as
per-harness subsections ordered "For Codex — the recommended first step" → "For
Claude Code" → "For OpenClaw or Hermes" (the 2026-08-09 ordering decision, recorded
in the PLAN's artifact table). `INSTALL_FOR_AGENTS.md` remains the paste path for
agent platforms and lives inside the OpenClaw/Hermes subsection.
## Threat model (v1 summary)
docs/designs/AGENT_BOOTSTRAP_PLAN.md
+37 -2
View File
@@ -14,6 +14,36 @@ below with their finding IDs. 0 unresolved decisions.
---
## As-shipped deltas (read this first — where the code moved after the plan froze)
This plan is layered: later absorption sections (the post-design-review deltas, the
CX2 series) override earlier prose, and THIS section overrides everything below it.
The shipped implementation matches the plan except for these deltas:
1. **Verify runs LAST, not before host registration.** [CX2-5]'s determinism goal
survived, but the shipped phase order (single TS source:
`src/core/bootstrap/status.ts` `PHASES`) is
preflight → engine → interview → render → skills → wire → repo → **verify**,
and verify runs in-process on the caller-held engine, calling
`runMaintenanceSweep` directly — no transient serve. It works pre-registration
AND as the weekly re-run (`src/core/bootstrap/verify.ts`).
2. **Uninstall scope: [CX2-12] wins over the CEO-expansion bullet.** `~/.gbrain` is
NEVER deleted wholesale — only receipt-enumerated bootstrap-created state
(`src/core/bootstrap/uninstall.ts`).
3. **Module naming/layout:** `private-repo.ts` shipped as `repo.ts`; additional
shipped modules the artifact table doesn't list: `attach.ts, assets.ts,
format.ts, host-specs.ts, hooks.ts, lock.ts, status.ts, template-repo.ts,
uninstall.ts`.
4. **Templates layout:** all bootstrap templates live under `templates/bootstrap/`
(not at `templates/` root).
5. **Test filenames:** `test/hook-command.serial.test.ts` and
`test/e2e/bootstrap-*.serial.test.ts` — the `.serial` variants the plan's own
[A7] mandated; the artifact table predates that.
6. **README ordering:** the D5 placement was superseded by the 2026-08-09 user
decision — per-harness `## Install` sections ordered Codex → Claude Code →
OpenClaw/Hermes, with `INSTALL_FOR_AGENTS.md` living inside the OpenClaw/Hermes
section (annotated in the artifact table; the D5 prose at the bottom is stale).
## Post-design-review deltas (2026-08-07, /office-hours APPROVED — these override below)
Product: **"GBrain for Codex" + "GBrain for Claude Code"** (names contingent on
@@ -90,7 +120,9 @@ ChatGPT-app user). CLIs come along via shared machinery.
kept in sync with templates/ by extending scripts/check-bootstrap-templates.sh to
diff the template repo content. Build order 2.
- **`gbrain bootstrap uninstall`** in v1 (was fast-follow): removes MCP registration +
hooks + `~/.gbrain` (confirm-gated), leaves the repo ("the body remains yours").
hooks + bootstrap-created state (confirm-gated), leaves the repo ("the body remains
yours"). [Scope superseded by CX2-12 + as-shipped delta 2: `~/.gbrain` is never
deleted wholesale — only receipt-enumerated bootstrap-created state.]
- **Docker cold-machine e2e (offline parts) in CI** in v1: networkless read-only
container running interview → render → verify with fake gh (codex-as-agent
tests/docker shape). The full networked paste flow stays a fast-follow (flake).
@@ -269,6 +301,8 @@ ChatGPT-app user). CLIs come along via shared machinery.
local-only CLI entry (`gbrain sweep --once`, CLI_ONLY, never over MCP), and
`bootstrap verify` runs BEFORE host registration on its own transient serve/engine:
write via op → `sweep --once` → edge query. No timing nondeterminism.
[Sequencing superseded by as-shipped delta 1: verify shipped as the LAST phase,
in-process on the caller-held engine; the determinism goal is unchanged.]
- [CX2-6 P1] **Cross-platform lock replaces flock dependence:** flock(1) absent ⇒
locking silently disabled (brain-repo-durability.ts:137) — macOS is the v1 target.
One cross-platform lock (atomic mkdir/lockfile with PID+age+token semantics) spans
@@ -767,7 +801,8 @@ settings.local.json + config.toml writers (single module owns each host format).
your data). The routing-table-size concern is mitigated by frontmatter-trigger routing
(authoritative since v0.36) and noted for a future curated-profile fast-follow if dispatch
accuracy suffers in practice.
- **D5 = Codex/Claude-Code-scoped placement.** This is NOT the new headline install —
- **D5 = Codex/Claude-Code-scoped placement.** [Superseded by the 2026-08-09 user
decision — see as-shipped delta 6 and the artifact table's README row.] This is NOT the new headline install —
most users still use GBrain with OpenClaw/Hermes, so `INSTALL_FOR_AGENTS.md` remains the
primary paste path at the top of the README. The bootstrap paste block becomes the
flagship "For Codex" / "For Claude Code" README sections, ahead of the OpenClaw/Hermes path at equal weight (and
docs/designs/AGENT_BOOTSTRAP_SPIKE.md
+4
View File
@@ -57,3 +57,7 @@ A filled copy of this doc committed as `AGENT_BOOTSTRAP_SPIKE_RESULTS.md`
(scrubbed: no real names beyond the maintainer, no account identifiers), plus the
gate decision recorded in the design doc: door-1 ships full / ships as documented
beta / schedule scope cut per quota.
**Gate status:** not yet run — no `AGENT_BOOTSTRAP_SPIKE_RESULTS.md` is committed,
so no gate decision is recorded and door 1 has not been promoted past the
documented-beta bar by this instrument. Update this line when the results land.
docs/eval-bench.md
+22 -29
View File
@@ -8,11 +8,10 @@ For the **NDJSON wire format** consumed by gbrain-evals, see
[`eval-capture.md`](./eval-capture.md). This doc is the human dev loop
that lives on top of that format.
## v0.41 update — the LOOP is now real
## The eval gate loop
Before v0.41, you could capture eval rows and replay them but nothing
stitched them into a gate. `gbrain bench publish` + `gbrain eval gate`
close the loop. Two gates:
`gbrain bench publish` + `gbrain eval gate` stitch captured eval rows into
a pass/fail gate. Two gates:
- **Regression gate** (`--baseline X.baseline.ndjson`): replays a baseline
you captured against your current brain. Catches: "did my refactor break
@@ -39,7 +38,7 @@ gbrain bench publish --from /tmp/captured.ndjson --to ~/.gbrain/baselines/person
gbrain eval gate --baseline ~/.gbrain/baselines/personal.baseline.ndjson
```
### Privacy posture (D9)
### Privacy posture
**Public baselines in `gbrain-evals` are hermetic-synthetic ONLY.** Real
user captures stay local in `~/.gbrain/baselines/`. The boundary is
@@ -131,14 +130,9 @@ gbrain query "anything" >/dev/null
psql $DATABASE_URL -c 'SELECT count(*) FROM eval_candidates' # should be > 0
```
To override (force on/off regardless of env var), edit `~/.gbrain/config.json`:
```json
{"eval": {"capture": true}} // force on
{"eval": {"capture": false}} // force off
```
Explicit config beats the env var both directions.
The full on/off resolution order (config beats env var, both directions) is
documented once in [`eval-capture.md`](./eval-capture.md) — that file is the
capture contract.
## The 4-command loop
@@ -205,7 +199,7 @@ retrieval, and which queries did it move most?"
For a third evaluation axis — public benchmark, ground-truth labels, full
question-answer pipeline (not just retrieval) — `gbrain eval longmemeval
<dataset.jsonl>` (v0.28.8) runs the LongMemEval benchmark against gbrain's
<dataset.jsonl>` runs the LongMemEval benchmark against gbrain's
hybrid retrieval. Each question gets a clean in-memory PGLite, its haystack
imported, the question asked, the hypothesis emitted as JSONL — exactly the
shape LongMemEval's `evaluate_qa.py` consumes. Your `~/.gbrain` brain is
@@ -337,7 +331,7 @@ Existing `eval_candidates` rows stay until you `gbrain eval prune
| `rows_errored > 0` | One or more queries threw. Inspect first 3 in human output, or `--json` to see all `error_message` fields |
| Many `skipped: empty query` | Capture ran on rows where someone passed empty `query` — check why those were captured |
## Public benchmarks: LongMemEval (v0.28.8)
## Public benchmarks: LongMemEval
`gbrain eval longmemeval` runs the public [LongMemEval](https://huggingface.co/datasets/xiaowu0162/longmemeval)
benchmark directly against gbrain's hybrid retrieval. Different evaluation
@@ -398,7 +392,7 @@ p50 25.9ms / p99 30.3ms warm reset+import+search on Apple Silicon (per the
500ms speed gate. 500 questions = ~13s of overhead plus your retrieval and
LLM latency.
## Measuring brain consistency over time (v0.32.6)
## Measuring brain consistency over time
`gbrain eval suspected-contradictions` is a complementary measurement
instrument: it samples retrieval results for unmarked semantic
@@ -435,20 +429,19 @@ commands per high-severity finding.
- CHANGELOG `## [0.32.6]` — full release notes including the bigger-swing
decision criteria gated on Wilson CI lower-bound.
## v0.40.1.0 Track D — Eval infrastructure
## Eval infrastructure: by-type breakdowns, the hermetic gate, batch scoring
Three eval surfaces grew non-trivial capabilities in v0.40.1.0. This section
covers the dev loop that uses them and the gates they enforce.
Three further eval surfaces, and the dev loop that uses them.
### `gbrain eval longmemeval --by-type` — per-question-type R@k breakdown
LongMemEval has always computed per-question-type recall internally; v0.40.1.0
surfaces it in machine-readable form. Two additive changes:
LongMemEval computes per-question-type recall internally, and surfaces it in
machine-readable form:
1. Every per-question JSONL row now includes a `question: string` field so the
1. Every per-question JSONL row includes a `question: string` field so the
`gbrain eval cross-modal --batch` consumer (below) can read it without
joining back against the source dataset.
2. New `--by-type` flag emits a final aggregate line keyed by `question_type`:
2. The `--by-type` flag emits a final aggregate line keyed by `question_type`:
```json
{"schema_version": 1, "kind": "by_type_summary",
@@ -480,11 +473,11 @@ echo "exit=$?" # 1 if any type fell below 0.80
### Hermetic retrieval gate — `test/eval-replay-gate.test.ts`
The v0.40.1.0 Track D structural fix for "PRs touching `src/core/search/`
silently regress retrieval." Replaces the original "replay against captured
eval_candidates" design (which Codex caught as non-functional in CI — see
the `v0.41+: contributor-mode CI capture` TODO in `TODOS.md` for the deferred
real-query version).
The structural fix for "PRs touching `src/core/search/` silently regress
retrieval." A "replay against captured eval_candidates" design can't work in
CI (CI has no captured production queries), so the gate is hermetic; see the
`contributor-mode CI capture` TODO in `TODOS.md` for the deferred
real-query version.
How it works:
- Hand-curated qrels fixture at `test/fixtures/eval-baselines/qrels-search.json`
@@ -499,7 +492,7 @@ How it works:
- Lives in the unit-shard test matrix (`.github/workflows/test.yml`) so it
runs on every PR via `bun test`, NOT in the E2E fixed-file workflow.
#### Refreshing the qrels fixture (the `Why:` discipline, D4)
#### Refreshing the qrels fixture (the `Why:` discipline)
When CI fails because a legitimate ranking change moved expected slugs, the
fix is to edit `qrels-search.json` directly. **Always include a `Why:` line
docs/eval-takes-quality.md
+7 -7
View File
@@ -28,7 +28,7 @@ receipt file from disk and re-renders it. The other modes need the brain.
|---|---|---|
| `--limit N` | 100 | Random sample of N takes from the brain. |
| `--cycles N` | 3 (TTY) / 1 (non-TTY) | Up to N panel calls before giving up; early-stop on PASS or INCONCLUSIVE. |
| `--budget-usd N` | unset | Abort before next call's projected cost would exceed cap. Models without a `pricing.ts` entry fail loud (codex #4). |
| `--budget-usd N` | unset | Abort before next call's projected cost would exceed cap. Models without a `pricing.ts` entry fail loud rather than silently blowing the budget. |
| `--source db|fs` | `db` | `fs` is reserved for v0.33+. |
| `--slug-prefix P` | unset | Filter takes to pages whose slug starts with P. |
| `--models a,b,c` | `openai:gpt-5.2,anthropic:claude-opus-4-7,google:gemini-2.0-flash` | Comma-separated panel. |
@@ -73,8 +73,8 @@ receipt file from disk and re-renders it. The other modes need the brain.
- `schema_version` — locks the contract. Adding optional fields is additive
and compatible. Renaming, removing, or changing semantics bumps the version.
- `rubric_version` + `rubric_sha8` — segregate trend rows by rubric epoch
(codex review #3). When the rubric definition changes, both fields update,
- `rubric_version` + `rubric_sha8` — segregate trend rows by rubric epoch.
When the rubric definition changes, both fields update,
and trend mode groups runs accordingly so a stricter rubric doesn't
silently look like a quality drop.
- `corpus.corpus_sha8` — fingerprint over the joined takes-text the judge
@@ -83,7 +83,7 @@ receipt file from disk and re-renders it. The other modes need the brain.
models in `--models` doesn't change the sha (sort is stable).
- `successes_per_cycle` — count of contributing models per cycle. A model
contributes when (a) its JSON parsed AND (b) every declared rubric dim
has a finite score (codex review #5 — missing-dim drops the contribution).
has a finite score (a missing dim drops the whole contribution).
- `verdict` — `pass` if every dim mean >= 7 AND every dim min across
contributing models >= 5; `fail` otherwise; `inconclusive` if fewer than
2/3 models contributed complete scores.
@@ -93,11 +93,11 @@ receipt file from disk and re-renders it. The other modes need the brain.
## Receipt persistence
Receipts persist to **`eval_takes_quality_runs`** (DB-authoritative per
codex review #6) AND to disk at `~/.gbrain/eval-receipts/takes-quality-<corpus>-<prompt>-<models>-<rubric>.json`
Receipts persist to **`eval_takes_quality_runs`** (the DB is authoritative)
AND to disk at `~/.gbrain/eval-receipts/takes-quality-<corpus>-<prompt>-<models>-<rubric>.json`
as a best-effort artifact. The DB row carries the full receipt JSON in the
`receipt_json` JSONB column, so when the disk artifact is gone, `replay`
can still reconstruct via `loadReceiptFromDb` (v0.33+ flag wiring).
can still reconstruct via `loadReceiptFromDb`.
The 4-sha primary key is unique (`UNIQUE` constraint) so re-running an
identical eval is `INSERT ... ON CONFLICT DO NOTHING` — idempotent.
docs/eval/SEARCH_MODE_METHODOLOGY.md
+10 -9
View File
@@ -1,6 +1,6 @@
# Search Mode Evaluation Methodology
_How v0.32.3 measures the difference between `conservative`, `balanced`, and `tokenmax`. Written haters-immune: every claim is reproducible from the committed dataset + raw outputs._
_How gbrain measures the difference between `conservative`, `balanced`, and `tokenmax`. Written haters-immune: every claim is reproducible — pinned datasets, recorded seeds, and the exact run commands below._
## 1. What this measures and what it doesn't
@@ -21,14 +21,15 @@ If you want to know how a mode behaves on YOUR brain, run `gbrain search stats -
- **Replay captures** — NDJSON from the sibling `gbrain-evals` repo, `n=200` queries. Each query carries a `retrieved_slugs` baseline + a `latency_ms` measurement from the original production run.
- **BrainBench v1** — `n=1240` documents / `n=350` qrels (binary relevance judgments). Lives in the sibling [`gbrain-evals`](https://github.com/garrytan/gbrain-evals) repo, SHA-pinned at every run.
No private brain content is used in any reported result. The committed NDJSON dumps under `<repo>/.gbrain-evals/` contain only the LongMemEval question IDs + the rank-ordered retrieved session IDs.
No private brain content is used in any reported result. The NDJSON run records under `<repo>/.gbrain-evals/` contain only the LongMemEval question IDs + the rank-ordered retrieved session IDs.
## 3. Sample selection
- **Random seed:** `42` throughout. Set via `--seed N` on `gbrain eval run-all`; recorded in every per-run record.
- **No per-question curation.** Splits are taken whole; no question is filtered for reporting.
- **No mode-specific tuning.** The same dataset + same seed feeds every mode. The mode is the only independent variable.
- **Stability across re-runs:** with `--seed 42` and the same dataset SHA, two runs of the same (mode, suite) produce identical retrieval orderings (modulo the optional Haiku expansion call, which is non-deterministic). Persisted in `eval_results` so anyone can re-score from the committed dumps.
- **No mode-specific tuning.** The same dataset + same seed feeds every mode. The mode bundle is the only independent variable. A mode Δ therefore measures the joint effect of every knob the bundles differ on — today that's `tokenBudget`, `expansion`, `relationalRetrieval` (the typed-edge fourth recall arm, ON for balanced/tokenmax, OFF for conservative), and `searchLimit`; the canonical diff is `MODE_BUNDLES` in `src/core/search/mode.ts`.
- **Cache comparability across upgrades.** The query cache keys on a versioned knobs hash (`KNOBS_HASH_VERSION` in `mode.ts`) that folds in the active knob set + embedding column/provider, so one mode's cached results can't be served to another mode's queries — and a version bump makes prior rows unreachable (one-time miss spike). Cross-run comparisons that straddle a knobs-hash bump see a cold cache on the first re-run.
- **Stability across re-runs:** with `--seed 42` and the same dataset SHA, two runs of the same (mode, suite) produce identical retrieval orderings (modulo the optional Haiku expansion call, which is non-deterministic). Persisted in `eval_results` so anyone can re-score from a run's `--output` dumps.
## 4. Run procedure
@@ -46,14 +47,14 @@ gbrain eval run-all \
--limit 500 \
--budget-usd-retrieval 5 \
--budget-usd-answer 20 \
--output docs/eval/results/v0.32.3/
--output docs/eval/results/<version>/
# Render the comparison.
gbrain eval compare --md > docs/eval/results/v0.32.3/README.md
gbrain eval compare --json > docs/eval/results/v0.32.3/comparison.json
gbrain eval compare --md > docs/eval/results/<version>/README.md
gbrain eval compare --json > docs/eval/results/<version>/comparison.json
```
The orchestrator writes per-run records to `<repo>/.gbrain-evals/eval-results.jsonl`. Every record carries: `run_id`, `ran_at`, `suite`, `mode`, `commit`, `seed`, `limit`, `params`, `status`, `duration_ms`. The dumps under `docs/eval/results/v0.32.3/` carry the raw question-level outputs so a reviewer can re-score with their own metric implementation.
The orchestrator writes per-run records to `<repo>/.gbrain-evals/eval-results.jsonl`. Every record carries: `run_id`, `ran_at`, `suite`, `mode`, `commit`, `seed`, `limit`, `params`, `status`, `duration_ms`. When a release publishes eval numbers, the `--output` dumps under `docs/eval/results/<version>/` carry the raw question-level outputs so a reviewer can re-score with their own metric implementation. **No dumps are committed in the repo right now** — reproduce by running the commands above; determinism (§3) means your re-run matches the reported orderings.
## 5. Threats to validity
@@ -68,7 +69,7 @@ Honest list. We name what would let a critic dismiss the numbers.
## 6. Per-question raw outputs
Every reported metric is reproducible from the NDJSON dumps committed at `docs/eval/results/v0.32.3/`. The commit SHA in the methodology footer pins the code version.
Every reported metric is reproducible from the NDJSON dumps a run writes to its `--output` directory (`docs/eval/results/<version>/` when a release publishes numbers; none are committed right now — see §4). The commit SHA in the methodology footer pins the code version.
**Examples per mode:** the auto-generated `README.md` next to the dumps includes both winning and losing examples per mode, chosen by the deterministic rule:
docs/guides/agent-to-gbrain.md
+43 -36
View File
@@ -12,13 +12,13 @@ surfaces**, and which one you pick depends on the operation.
│ gbrain process │
│ │
Agent (hermes, │ ┌──────────────────┐ ┌────────────────┐ │
openclaw, fork) ────┼──▶ MCP ops surface │ │ localOnly │ │
│ │ (HTTP + OAuth) │ │ admin ops │ │
openclaw, fork) ────┼──▶ MCP ops surface │ │ local-only │ │
│ │ (HTTP + OAuth) │ │ commands │ │
│ │ │ │ │ │
│ │ search, query, │ │ sync, embed, │ │
│ │ put_page, │ │ dream, doctor,│ │
│ │ get_page, │ │ autopilot, │ │
│ │ find_experts, │ │ init, secrets │ │
│ │ put_page, │ │ extract, │ │
│ │ get_page, │ │ dream, │ │
│ │ find_experts, │ │ enrich, ... │ │
│ │ ... │ │ │ │
│ └──────────────────┘ └────────────────┘ │
│ ▲ ▲ │
@@ -26,7 +26,7 @@ surfaces**, and which one you pick depends on the operation.
│ │ │ │
│ thin-client OAuth shell-job `inherit:`│
│ (preferred for (only path for │
│ MCP-equivalent ops) localOnly ops) │
│ MCP-equivalent ops) local-only work) │
└─────────────────────────────────────────────┘
```
@@ -45,7 +45,7 @@ the set of ops in `src/core/operations.ts` whose `localOnly` flag is unset
The host runs gbrain as a long-lived HTTP server:
```bash
GBRAIN_ALLOW_SHELL_JOBS=1 gbrain serve --http --port 3131
gbrain serve --http --port 3131
```
The agent registers as an OAuth client (one-time):
@@ -75,22 +75,27 @@ commands through the configured remote MCP. The agent can call
agent to a specific source within a federated brain.
- One audit surface (`mcp_request_log`) covers every op call uniformly.
## Surface 2 — localOnly admin ops via shell-job `inherit:`
## Surface 2 — local-only work via shell-job `inherit:`
Some operations are flagged `localOnly: true` in `src/core/operations.ts` and
are **refused** in thin-client mode at `src/cli.ts:isThinClient`. The full
list (as of v0.36.5.0) includes:
Two mechanisms keep local-only work off the remote surface, and they operate
at different layers:
- `sync` (filesystem walks need local FS access)
- `embed` (orchestrates the embed pipeline)
- `extract` (walks markdown files)
- `dream` (synthesis cycle)
- `doctor` (filesystem hygiene checks)
- `autopilot` (background daemon orchestration)
- `init` (creates `~/.gbrain/`)
- `secrets` (config management)
- **Op layer:** operations flagged `localOnly: true` in
`src/core/operations.ts` are filtered out of the HTTP MCP surface entirely
— a remote caller never sees them.
- **CLI layer:** on a thin-client install (remote MCP configured, no local
engine), commands that require a local engine or the local filesystem are
refused at dispatch with a pinpoint hint naming the closest alternative.
The authoritative set is `THIN_CLIENT_REFUSED_COMMANDS` in `src/cli.ts` —
read it there rather than trusting any list copied into a doc; it covers
`sync`, `embed`, `extract`, `dream`, `enrich`, `serve`, `config`, and a
couple dozen more.
For these, the agent cannot route through HTTP MCP. The only path is to run
Notable non-members: `doctor` is NOT refused on a thin client — it reroutes
to an outbound-HTTP probe set (`src/core/doctor-remote.ts`); `bootstrap` and
`hook` are engine-free and work on any install shape.
For refused commands, the agent cannot route through HTTP MCP. The path is to run
`gbrain` as a CLI subprocess. The recommended pattern is to submit the
subprocess as a shell job to the gbrain Minions worker so retry / backoff /
DLQ / audit trail all come for free.
@@ -114,12 +119,11 @@ full validation rules and error catalog.
### Why this is preferred over writing secrets into `env:` per-job
- Pre-v0.36.5.0 callers passed `env: { GBRAIN_DATABASE_URL: "postgresql://..." }`
per job. The URL landed plaintext in `minion_jobs.data` and the shell-audit
JSONL. Anyone with brain-DB read access (or a brain dump, or a shared brain
via mounts) saw the URL. As of v0.36.5.0, this is rejected at pre-enqueue
validation. The error message names `inherit: ["database_url"]` as the
replacement.
- Passing `env: { GBRAIN_DATABASE_URL: "postgresql://..." }` per job would
land the URL plaintext in `minion_jobs.data` and the shell-audit JSONL —
visible to anyone with brain-DB read access (or a brain dump, or a shared
brain via mounts). Pre-enqueue validation rejects it; the error message
names `inherit: ["database_url"]` as the replacement.
### Worker setup (one-time, per host)
@@ -146,11 +150,11 @@ proxy for worker env.
| `get_page` / `list_pages` | HTTP MCP | Same. |
| `put_page` | HTTP MCP | Same; respects subagent allow-list when applicable. |
| `find_experts` / `find_orphans` | HTTP MCP | Same. |
| `sync` / `embed` / `extract` | Shell job + `inherit:` | `localOnly: true`. |
| `dream` | Shell job + `inherit:` | `localOnly: true`. |
| `doctor` | Shell job + `inherit:` (or no inherit if no DB) | `localOnly: true`. |
| `sync` / `embed` / `extract` | Shell job + `inherit:` | Thin-client refused; needs local engine + FS. |
| `dream` | Shell job + `inherit:` | Thin-client refused; synthesis runs on the host. |
| `doctor` | Run directly (any install) | Not refused: thin clients get the remote probe set. |
| `autopilot` | Run as a daemon directly on the host | Long-lived, not job-shaped. |
| `init` / `secrets` | One-time host setup | Operator action, not agent action. |
| `init` / `config` | One-time host setup | Operator action, not agent action. |
## Recommended patterns
@@ -166,16 +170,19 @@ proxy for worker env.
- **`env:` still works** for non-secret values, or for cases where you
WANT the value in the row (e.g. an opaque correlation token your audit
flow needs to read back later). The validator doesn't second-guess you.
- **Never try to route a `localOnly` op through thin-client MCP.** It will
fail with `localOnly op refused in thin-client mode`. Use shell-job +
`inherit:` (for secrets) or `env:` (for non-secrets).
- **Never try to route a refused command through a thin client.** The CLI
refuses it at dispatch with a hint. Use shell-job + `inherit:` (for
secrets) or `env:` (for non-secrets) on the host instead.
- **Push-based context.** Beyond request/response ops, MCP clients can
receive volunteered context via the `volunteer_context` op — see
[push-context.md](./push-context.md).
## Migration: from pre-v0.36.5.0
## Migration: from `env:`-passed secrets
If your agent submits shell jobs that pass secrets via `env:`:
```jsonc
// Pre-v0.36.5.0: works but URL persists in minion_jobs.data plaintext.
// Rejected at submit: the URL would persist in minion_jobs.data plaintext.
{
"cmd": "gbrain sync --skip-failed",
"cwd": "/data/gbrain",
@@ -186,7 +193,7 @@ If your agent submits shell jobs that pass secrets via `env:`:
Switch to (recommended):
```jsonc
// v0.36.5.0+: name in row, value resolved at child-spawn from worker config.
// Name in row, value resolved at child-spawn from worker config.
{
"cmd": "gbrain sync --skip-failed",
"cwd": "/data/gbrain",
docs/guides/bootstrap.md
+2 -1
View File
@@ -75,7 +75,8 @@ zero in keyless mode; with a key, the standard spend gates apply
the cause.
- **Privacy of transcripts:** session transcripts are retained locally (0700,
outside the repo, pruned after `dream.synthesize.corpus_retention_days`, default
30) and secret-redacted at write time. They never enter the repo. The extraction
30 — set it in the config file, `~/.gbrain/config.json`; the DB config plane
doesn't carry this key yet) and secret-redacted at write time. They never enter the repo. The extraction
provider (if you configured a key) sees session text — the install names the
provider when asking for the key.
docs/guides/brain-agent-loop.md
+8 -5
View File
@@ -27,6 +27,7 @@ READ: check brain FIRST (before responding)
│ → gbrain search "{entity name}"
│ → gbrain get {slug} (if you know it)
│ → gbrain query "what do we know about {topic}"
│ → full protocol: brain-first-lookup.md
│
▼
RESPOND with brain context (every answer is better with context)
@@ -100,12 +101,14 @@ on_message(text):
Write immediately after the conversation, while the context is fresh.
3. **Sync after every write batch.** Without sync, the brain search index is
stale. The next query won't find what you just wrote.
stale. The next query won't find what you just wrote. On installs set up
via `gbrain bootstrap`, per-turn context injection and session-end
persistence hooks automate parts of this loop — see
[bootstrap.md](bootstrap.md) and [push-context.md](push-context.md).
4. **External APIs are fallback, not primary.** `gbrain search` before
Brave Search. `gbrain get` before Crustdata. The brain has relationship
history, your own assessments, meeting transcripts, cross-references.
No external API can provide that.
4. **External APIs are fallback, not primary.** `gbrain search` before any
web or enrichment API. The full brain-before-external protocol (and why)
lives in [brain-first-lookup.md](brain-first-lookup.md).
## How to Verify It Works
docs/guides/brain-first-lookup.md
+12 -9
View File
@@ -51,22 +51,25 @@ The brain has context no external API can provide:
- Timeline (what changed recently, what's trending)
A LinkedIn scrape gives you their job title. The brain gives you: "co-founded
Brex, you had coffee with him 3 times, last discussed the payments infrastructure
thesis, he's interested in your take on AI agents."
widget-co, you had coffee with her 3 times, last discussed the payments
infrastructure thesis, she's interested in your take on AI agents."
## Tricky Spots
1. **Try keyword first, then hybrid.** Keyword search works without embeddings
(day one). Hybrid search needs embeddings but finds semantic matches. Try
both in sequence.
(day one — and it's ALL you get in keyless mode, see
[bootstrap.md](bootstrap.md)). Hybrid search needs embeddings but finds
semantic matches. Try both in sequence.
2. **Fuzzy slug matching.** `gbrain get` supports fuzzy matching. If the exact
slug doesn't exist, it suggests alternatives. Use this for name variants
("Pedro" → "pedro-franceschi").
2. **Fuzzy slug matching is opt-in.** Pass `--fuzzy` (the `fuzzy: true` param
on `get_page`) and a near-miss slug resolves to the unique candidate, or
returns an `ambiguous_slug` error listing the candidates. WITHOUT the flag
a miss just throws `page_not_found` (with a hint to retry with
`fuzzy: true`). Use it for name variants ("Alice" → "alice-example").
3. **Don't skip for "simple" questions.** Even "what's Acme Corp's address?"
should check the brain first. The brain might have it, and the lookup adds
no latency (< 100ms for keyword search).
should check the brain first. The brain might have it, and a keyword
lookup is fast enough to be effectively free.
4. **Load compiled truth + recent timeline.** The compiled truth gives you the
state of play in 30 seconds. The timeline gives you what changed recently.
docs/guides/brain-vs-memory.md
+5 -4
View File
@@ -17,8 +17,8 @@ on new_information(info):
# This is world knowledge -- facts about entities external to the agent
gbrain put <slug> --content "..."
# Examples:
# "Pedro is CEO of Brex" -> gbrain (person page)
# "Brex raised Series D at $12B" -> gbrain (company page)
# "alice-example is CEO of widget-co" -> gbrain (person page)
# "widget-co raised Series D at $12B" -> gbrain (company page)
# "Tuesday's meeting covered Q2" -> gbrain (meeting page)
# "The meatsuit maintenance tax" -> gbrain (originals page)
@@ -57,15 +57,16 @@ on user_asks(question):
## Tricky Spots
1. **Don't store people in agent memory.** "Pedro prefers email over Slack" feels like a preference, but it's a fact about Pedro -- it goes in GBrain on Pedro's page. Agent memory is for the agent's own operational state, not facts about people in the world.
1. **Don't store people in agent memory.** "alice-example prefers email over Slack" feels like a preference, but it's a fact about Alice -- it goes in GBrain on her page. Agent memory is for the agent's own operational state, not facts about people in the world.
2. **Don't store user preferences in GBrain.** "User likes bullet points over paragraphs" is about how the agent should behave, not about the world. It goes in agent memory. GBrain pages are for entities, not for agent configuration.
3. **Synthesis of external ideas goes in GBrain.** "User's take on Peter Thiel's zero-to-one framework" is the user's original thinking -- it goes in GBrain under originals/, not in agent memory.
4. **Agent memory doesn't survive agent resets on some platforms.** Critical world knowledge MUST be in GBrain, which is durable. If the agent loses memory, the brain still has everything.
On installs set up via `gbrain bootstrap`, "agent memory" has a concrete file form: MEMORY.md and the other identity files in the agent repo (see [bootstrap.md](bootstrap.md)). The routing rule is unchanged -- those files hold operational state and identity, not world knowledge.
5. **When in doubt, ask: is this about the world or about how to operate?** World -> GBrain. Operations -> agent memory. Current conversation -> session.
## How to Verify
1. Ask the agent "Who is Pedro?" -- confirm it runs `gbrain search` or `gbrain get`, not `memory_search`. Person lookup should hit GBrain.
1. Ask the agent "Who is alice-example?" -- confirm it runs `gbrain search` or `gbrain get`, not `memory_search`. Person lookup should hit GBrain.
2. Ask the agent "How should I format responses?" -- confirm it checks agent memory, not GBrain. Preferences are operational state.
3. Check that no person or company pages exist in agent memory storage. Run `memory_search "person"` -- it should return preferences, not dossiers.
4. Check that GBrain doesn't contain pages about agent behavior. Run `gbrain search "user prefers"` -- it should return nothing (preferences belong in agent memory).
docs/guides/compiled-truth.md
+16 -6
View File
@@ -44,12 +44,12 @@ Sharp technical leader. Under-appreciated internally. Watch for signs of burnout
Ascending. Likely CTO track if the migration succeeds.
## Relationship
Met through Pedro. Had coffee 3x. Last: discussed API architecture thesis.
Met through alice-example. Had coffee 3x. Last: discussed API architecture thesis.
## Contact
sarah@acmecorp.com | @sarahchen | linkedin.com/in/sarahchen
---
<!-- timeline -->
## Timeline
@@ -58,7 +58,7 @@ sarah@acmecorp.com | @sarahchen | linkedin.com/in/sarahchen
[Source: Meeting notes, 2026-04-07 2:00 PM PT]
- **2026-04-03** | Mentioned in email re Q2 planning. Taking lead on ops.
[Source: Gmail, sarah@acmecorp.com, 2026-04-03 10:30 AM PT]
- **2026-03-15** | First meeting. Intro from Pedro. Strong technical background.
- **2026-03-15** | First meeting. Intro from alice-example. Strong technical background.
[Source: User, direct conversation, 2026-03-15 3:00 PM PT]
```
@@ -113,9 +113,19 @@ support that claim.
truth chunks with higher relevance than timeline chunks. This means the freshest
synthesis surfaces first in search results.
4. **The --- separator matters.** GBrain uses the first standalone `---` after
frontmatter to split compiled_truth from timeline. Everything above is compiled
truth, everything below is timeline.
4. **The timeline sentinel matters — and a bare `---` is NOT one.** GBrain
splits compiled_truth from timeline at the first recognized sentinel, in
order of precedence:
1. `<!-- timeline -->` — preferred; unambiguous, and what GBrain itself
emits when it writes a page.
2. `--- timeline ---` — decorated separator.
3. `---` ONLY when the next non-empty line is `## Timeline` or
`## History` (backward-compat for older gbrain-written files).
A plain `---` line anywhere else is a markdown horizontal rule, not a
separator. Author new pages with `<!-- timeline -->` (as in the example
above); everything above it is compiled truth, everything below is
timeline.
5. **Don't skip the Assessment section.** The assessment is the value. "Strong
technical leader" is something no API can provide. It's YOUR read on this
docs/guides/content-media.md
+7 -1
View File
@@ -8,6 +8,12 @@ Without this: media links are bookmarks that decay -- you remember watching a vi
## Implementation
gbrain's own media surfaces complement this pattern: the bundled
`media-ingest` skill (`skills/media-ingest/`) ships the ingestion workflow,
and `gbrain files` handles binary/file upload for attachments that should
live alongside pages. For meeting recordings specifically, see
[meeting-ingestion.md](meeting-ingestion.md).
```
on user_shares_media(url_or_file):
@@ -119,7 +125,7 @@ on user_shares_media(url_or_file):
## Tricky Spots
1. **Always FULL transcript, never AI summary.** YouTube's auto-summary and AI-generated summaries lose the texture: who said what, exact phrasing, tone, what was left unsaid. The full diarized transcript is the evidence base. The agent's analysis goes above it.
2. **The agent's OWN analysis is the value, not regurgitation.** "The video discussed AI safety" is worthless. "Dario made a specific claim about compute scaling that contradicts what Ilya said in the NeurIPS talk -- see media/youtube/ilya-neurips-2025" is useful. The analysis connects the new media to the existing brain.
2. **The agent's OWN analysis is the value, not regurgitation.** "The video discussed AI safety" is worthless. "The speaker made a specific claim about compute scaling that contradicts what another researcher said in their NeurIPS talk -- see media/youtube/a-researcher-neurips-2025" is useful. The analysis connects the new media to the existing brain.
3. **Social media is a bundle, not a single tweet.** A tweet without its thread, quoted tweets, linked articles, and engagement context is a fragment. Reconstruct the full context before creating the brain page.
4. **Cross-references make media pages alive.** A YouTube page without back-links to the people and companies mentioned is a dead archive. Every mentioned entity gets a link and a timeline entry.
5. **Over time, `media/` becomes a searchable archive.** Every video, podcast, talk, interview, article, and tweet the user has consumed, with the agent's commentary layered on top. This is the memex at full power.
docs/guides/cron-schedule.md
+37 -16
View File
@@ -26,6 +26,27 @@ fixed. You wake up and the brain is smarter than when you went to sleep.
| Weekly | Brain maintenance | `gbrain doctor`, embed stale, orphan detection | [maintain skill](../../skills/maintain/SKILL.md) |
| Nightly | Dream cycle | Entity sweep, enrich thin spots, fix citations | See below |
### Prefer gbrain's native schedulers where they fit
System cron is the lowest common denominator, but gbrain ships its own
scheduling surfaces — reach for these first:
- **`gbrain dream`** — the shipped nightly maintenance cycle (lint,
backlinks, extract, sync, embed, synthesize). Schedule THIS instead of
hand-rolling the dream cycle below.
- **`gbrain jobs` / minions** — queue shell jobs or LLM subagents with retry,
backoff, and an audit trail. See the `minion-orchestrator` skill.
- **`gbrain autopilot`** — the long-lived background daemon that runs cycles
on its own cadence.
- **`cron-scheduler` skill** (`skills/cron-scheduler/`) — teaches an agent to
manage its harness's scheduler.
- **Bootstrap session-triggered schedules** — `gbrain bootstrap` installs
HEARTBEAT.md-driven schedules that fire on session activity; see
[bootstrap.md](bootstrap.md).
For scheduling `sync` + `embed --stale` specifically, the home doc is
[live-sync.md](live-sync.md).
## Implementation: Setting Up Cron Jobs
```bash
@@ -50,18 +71,12 @@ fixed. You wake up and the brain is smarter than when you went to sleep.
### Quiet Hours Gate (MANDATORY)
Every cron job that sends notifications MUST check quiet hours first.
See [Quiet Hours](quiet-hours.md) for the full pattern.
```bash
# In every cron script:
if ! bash scripts/quiet-hours-gate.sh; then
mkdir -p /tmp/cron-held
echo "$OUTPUT" > /tmp/cron-held/$(basename "$0" .sh).md
exit 0
fi
# Not quiet hours — send normally
```
Every cron job that sends notifications MUST check quiet hours first. The
gate is a small script YOU create (it doesn't ship with gbrain) and call at
the top of every notification-sending cron script; held output goes to a
holding directory that the morning briefing drains. See
[Quiet Hours](quiet-hours.md) for the gate script and the full pattern —
don't copy a snippet from here, that page is the single home.
### Travel-Aware Timezone Handling
@@ -88,6 +103,12 @@ morning briefing. Zero config change needed.
The most important cron job. Runs while you sleep.
**gbrain ships this**: `gbrain dream` runs the maintenance half of the cycle
(lint, backlinks, extract, sync, embed, synthesize) as one command — schedule
it nightly and Phase 4 below (plus most of Phase 2's hygiene checks) is
covered. The pseudocode that follows is the harness-side variant for agents
that also do LLM-driven entity sweeps and memory consolidation on top.
### What It Does
```
@@ -150,11 +171,11 @@ echo "Dream cycle starting at $(date)"
# Phase 1: Entity sweep (spawn sub-agent)
# Read today's conversation logs, extract entities, update brain
# Phase 2: Citation hygiene
gbrain doctor --json | jq '.checks[] | select(.status=="warn")'
# Phase 2: Shipped maintenance cycle (lint, backlinks, extract, sync, embed, synthesize)
gbrain dream
# Phase 3: Embed any stale content
gbrain embed --stale
# Phase 3: Surface anything the cycle flagged
gbrain doctor --json | jq '.checks[] | select(.status=="warn")'
echo "Dream cycle complete at $(date)"
```
docs/guides/deterministic-collectors.md
+4 -1
View File
@@ -143,4 +143,7 @@ the same pass.
---
*Part of the [GBrain Skillpack](../GBRAIN_SKILLPACK.md).*
*Part of the [GBrain Skillpack](../GBRAIN_SKILLPACK.md). The
[email-to-brain recipe](../../recipes/email-to-brain.md) implements this
collector pattern end-to-end; [cron-schedule.md](cron-schedule.md) covers
scheduling it.*
docs/guides/diligence-ingestion.md
+4 -1
View File
@@ -53,6 +53,8 @@ Write extracted content to `brain/diligence/{company}/{doc-name}.md`:
**Step 5: Save Raw Files.**
Copy original PDFs/files to `brain/diligence/{company}/.raw/`
Preserve originals for reference. The diarized version is for search.
This is safe by design: `.raw/` directories are excluded from sync, so the
originals never enter the search index — only your extracted markdown does.
**Step 6: Create or Update index.md.**
Every diligence directory needs an `index.md`:
@@ -103,7 +105,8 @@ cd brain/ && git add -A && git commit -m "diligence: {Company} — {doc type} in
**Step 9: Publish (if asked).**
When the user wants a shareable brief, create a password-protected
published version. Strip internal notes and raw assessment language.
published version via the `publish` skill (`skills/publish/`). Strip
internal notes and raw assessment language.
### Quality Bar
docs/guides/embedding-migration.md
+2 -2
View File
@@ -7,8 +7,8 @@ sunsetting provider (for example ZeroEntropy's hosted API, which shuts down
but it is provider-agnostic: any configured `provider:model` works as a
target.
Also reachable as `gbrain retrieval-upgrade` (the name `doctor` and the
README reference).
Also reachable as `gbrain retrieval-upgrade` — the alias that `gbrain doctor`
repair hints and the README point at.
## Quick start
docs/guides/enrichment-pipeline.md
+10 -4
View File
@@ -8,6 +8,11 @@ Without this: brain pages are thin shells with only what the user manually typed
## Implementation
gbrain ships both halves of this: `gbrain enrich` is the batch enrichment
primitive (finds thin pages and enriches at scale), and the `enrich` skill
(`skills/enrich/`) is the agent-driven page-at-a-time workflow. The pipeline
below is the pattern they implement — use it to customize or extend.
```
on enrich(entity, trigger):
# trigger: meeting mention, email thread, social interaction, user request
@@ -68,9 +73,10 @@ on enrich(entity, trigger):
gbrain link <person_slug> <deal_slug> # person -> deal
# Every entity page links to every other entity page that references it
# People page sections (not a LinkedIn profile -- a living portrait):
# Executive Summary, State, What They Believe, What They're Building,
# What Motivates Them, Assessment, Trajectory, Relationship, Contact, Timeline
# People page sections: use the person-page structure from compiled-truth.md
# (Executive Summary, State, What They Believe, ... Timeline) -- that doc is
# the single home for the section taxonomy. Enrichment can add texture
# sections on top (What Motivates Them, Hobby Horses, Open Threads).
# Facts are table stakes. TEXTURE is the value.
# Extract texture, not just facts:
@@ -100,4 +106,4 @@ on enrich(entity, trigger):
5. Try to re-enrich the same person. Confirm the system checks the `fetched_at` timestamp and skips if less than a week old.
---
*Part of the [GBrain Skillpack](../GBRAIN_SKILLPACK.md).*
*Part of the [GBrain Skillpack](../GBRAIN_SKILLPACK.md). See also: [Compiled Truth](compiled-truth.md) for the person-page section taxonomy, [Spend Controls](../operations/spend-controls.md) for gbrain's own embedding/LLM spend gates.*
docs/guides/entity-detection.md
+21 -9
View File
@@ -7,10 +7,10 @@ Every inbound message gets scanned for original thinking AND entity mentions so
## What the User Gets
Without this: the agent answers questions but forgets everything. You mention
Pedro in a meeting, and next week the agent doesn't know who Pedro is.
Alice in a meeting, and next week the agent doesn't know who Alice is.
With this: every person, company, and idea mentioned in conversation gets a
brain page. Next time Pedro comes up, the agent already has context. The
brain page. Next time Alice comes up, the agent already has context. The
brain compounds.
## Implementation
@@ -18,6 +18,11 @@ brain compounds.
Spawn a lightweight sub-agent on EVERY inbound message. Do NOT wait for it
to finish before responding. It runs in parallel.
This pattern is harness-side by design, but gbrain ships help on both ends:
the `signal-detector` skill (`skills/signal-detector/`) is the bundled
version of this detection loop, and `gbrain extract` runs gbrain's own
extraction machinery (entities, facts) over already-synced content.
```
on_every_message(message_text, source_context):
@@ -109,6 +114,7 @@ is_notable(entity):
| Pattern recognition ("I keep seeing X in every Y") | Acknowledgments and reactions |
| Hot takes with reasoning | Routine operational messages |
| Metaphors that reveal new angles | Requests without embedded insight |
| Emotional/psychological insights about self or others | |
### Filing Rules
@@ -116,11 +122,17 @@ is_notable(entity):
|--------|-------------|
| User generated the idea | `brain/originals/{slug}.md` |
| User's synthesis of others' ideas | `brain/originals/` (the synthesis is original) |
| User's ghostwritten book/essay | `brain/originals/` (note ghostwriter in metadata) |
| World concept someone else coined | `brain/concepts/{slug}.md` |
| Product or business idea | `brain/ideas/{slug}.md` |
| Person mentioned | `brain/people/{slug}.md` |
| Company mentioned | `brain/companies/{slug}.md` |
| Media referenced | `brain/media/{type}/{slug}.md` |
| Article ABOUT the user | `brain/media/writings/{slug}.md` |
This table is the single home for the capture/filing taxonomy. Other guides
([idea-capture](idea-capture.md) especially) link here rather than carrying
their own copy.
### The Iron Law of Back-Linking
@@ -128,21 +140,21 @@ Every entity mention MUST create a back-link FROM the entity page TO the
source. This is not optional.
```
// When message mentions "Pedro" and creates a meeting page:
// When message mentions "Alice" and creates a meeting page:
// 1. Update the meeting page (normal)
brain/meetings/2026-04-10-board-sync.md:
- Pedro presented Q1 numbers
- Alice presented Q1 numbers
// 2. ALSO update Pedro's page (back-link)
brain/people/pedro-franceschi.md:
// 2. ALSO update Alice's page (back-link)
brain/people/alice-example.md:
## Timeline
- **2026-04-10** | Presented Q1 numbers at board sync
[Source: User, board meeting, 2026-04-10]
```
Without back-links, you can't traverse the graph. "Show me everything related
to Pedro" only works if Pedro's page links back to every mention.
to Alice" only works if Alice's page links back to every mention.
## Tricky Spots
@@ -163,7 +175,7 @@ to Pedro" only works if Pedro's page links back to every mention.
5. **Dedup before creating.** Always `gbrain search` before creating a page.
Variant spellings, nicknames, and company abbreviations cause duplicates.
"Pedro Franceschi" and "Pedro" might be the same person.
"Alice Example" and "Alice" might be the same person.
## How to Verify
@@ -182,7 +194,7 @@ to Pedro" only works if Pedro's page links back to every mention.
4. **Send a boring message.** Say "ok sounds good." Verify: nothing was
created. The detector should report "No signals detected."
5. **Check for duplicates.** Mention "Pedro" then later "Pedro Franceschi."
5. **Check for duplicates.** Mention "Alice" then later "Alice Example."
Verify: one page, not two.
---
docs/guides/executive-assistant.md
+9 -4
View File
@@ -8,6 +8,11 @@ Without this: the agent triages email mechanically ("you have 12 unread"), preps
## Implementation
Before hand-rolling these: gbrain bundles the morning-briefing half of this
pattern as the `briefing` skill (`skills/briefing/`) and the task-prep half
as `daily-task-prep` (`skills/daily-task-prep/`). Use the workflows below to
extend or customize what those skills already ship.
```
# WORKFLOW 1: Email Triage
on email_batch(emails):
@@ -59,8 +64,8 @@ on upcoming_meeting(meeting):
briefing[attendee] = "No brain page -- consider enriching"
# Surface: shared history, what to follow up on, what to watch for
# "Last time you discussed the Series B timeline. Pedro was concerned
# about burn rate. Here's the latest from his company page."
# "Last time you discussed the Series B timeline. alice-example was
# concerned about burn rate. Here's the latest from her company page."
# WORKFLOW 3: Post-Inbox Brain Updates
on inbox_cleared():
@@ -93,9 +98,9 @@ on schedule_request(meeting):
1. **Search sender BEFORE reading the email.** This is counterintuitive but critical. Loading brain context first means you know who they are, what you're working on together, and what they care about -- before you even see the subject line. The triage is informed, not mechanical.
2. **Unknown senders with no brain page are almost always noise.** If `gbrain search` returns nothing for a sender, they're probably not important. Classify as low priority unless the email content signals otherwise.
3. **Meeting prep is the highest-leverage EA workflow.** The user walks into every meeting already briefed on each attendee: last interaction, open threads, relationship history. This is the difference between "you have a meeting at 3" and "you have a meeting at 3 with Pedro -- last time you discussed the Series B, he was concerned about burn rate."
3. **Meeting prep is the highest-leverage EA workflow.** The user walks into every meeting already briefed on each attendee: last interaction, open threads, relationship history. This is the difference between "you have a meeting at 3" and "you have a meeting at 3 with alice-example -- last time you discussed the Series B, she was concerned about burn rate."
4. **Post-inbox brain updates are where the brain compounds.** Every email is signal. If you clear the inbox without updating brain pages, the information is lost. This is the step most agents skip.
5. **Scheduling nudges require timeline data.** "You haven't met with Diana in 6 weeks" only works if meeting pages have been ingested with proper entity propagation (see meeting-ingestion guide).
5. **Scheduling nudges require timeline data.** "You haven't met with charlie-example in 6 weeks" only works if meeting pages have been ingested with proper entity propagation (see meeting-ingestion guide).
## How to Verify
docs/guides/idea-capture.md
+13 -39
View File
@@ -61,14 +61,13 @@ capture_idea(message_text, source_context):
### The Authorship Test
| Signal | Destination |
|--------|-------------|
| User generated the idea | `brain/originals/{slug}.md` |
| User's unique synthesis of others' ideas | `brain/originals/` (the synthesis is original) |
| World concept someone else coined | `brain/concepts/{slug}.md` |
| Product or business idea | `brain/ideas/{slug}.md` |
| User's ghostwritten book/essay | `brain/originals/` (note ghostwriter in metadata) |
| Article ABOUT user | `brain/media/writings/` |
Who authored the idea determines where it files: user-generated ideas,
syntheses, and ghostwritten work go to `brain/originals/`; borrowed world
concepts to `brain/concepts/`; product ideas to `brain/ideas/`; articles
ABOUT the user to `brain/media/writings/`. The full filing-rules table (and
the what-counts-as-original-thinking criteria) lives in
[entity-detection.md](entity-detection.md) — the single home for the capture
taxonomy.
### Capture Standards
@@ -78,21 +77,6 @@ capture_idea(message_text, source_context):
"tension between ambition and mortality" doesn't. Don't clean it up. Don't paraphrase.
The vivid version is the real version.
**What counts as worth capturing:**
- Original observations about how the world works
- Novel connections between disparate things
- Frameworks and mental models
- Pattern recognition moments ("I keep seeing X in every Y")
- Hot takes with reasoning behind them
- Metaphors that reveal new angles
- Emotional/psychological insights about self or others
**What does NOT count:**
- Routine operational messages ("ok", "do it")
- Pure questions without embedded observations
- Echoing back something the agent said
- Acknowledgments and reactions
### The Depth Test
**Could someone unfamiliar with the user read this page and understand not
@@ -137,21 +121,9 @@ Every original MUST link to:
### Notability Filtering
Before creating any entity page, check notability:
**Create a page for:**
- People you know or discuss with specificity
- Companies you're evaluating, working with, or investing in
- Media you mention with personal reaction
- Anyone you've explicitly engaged with
**Don't create pages for:**
- Generic references or passing examples
- Low-engagement accounts who mentioned you once
- Pure metaphors ("like the Roman Empire...")
- One-off encounters with no follow-up
**Decision:** If notable AND no page exists, create a full page with web
Before creating any entity page, check notability — the full create/skip
criteria live in [entity-detection.md](entity-detection.md#notability-filtering).
The decision rule: if notable AND no page exists, create a FULL page with web
search enrichment. No stubs. If you make a page, make it good.
## Tricky Spots
@@ -187,4 +159,6 @@ search enrichment. No stubs. If you make a page, make it good.
---
*Part of the [GBrain Skillpack](../GBRAIN_SKILLPACK.md).*
*Part of the [GBrain Skillpack](../GBRAIN_SKILLPACK.md). The bundled
`idea-ingest` skill (`skills/idea-ingest/`) ships this workflow. See also:
[Entity Detection](entity-detection.md).*
docs/guides/live-sync.md
+22 -5
View File
@@ -43,7 +43,12 @@ gbrain sync --repo /path/to/brain && gbrain embed --stale
- `gbrain sync --repo <path>` -- one-shot incremental sync. Detects changes via
`git diff`, imports only what changed. For small changesets (<= 100 files),
embeddings are generated inline during import.
embeddings are generated inline during import — unless the inline cost gate
intervenes: when the estimated embedding spend crosses the configured floor
in a non-interactive session (cron, `--json`), sync auto-defers embeds to a
capped `embed-backfill` job instead of spending silently. Either way the
chunks get embedded; a deferred run just finishes asynchronously. See
[spend controls](../operations/spend-controls.md).
- `gbrain embed --stale` -- backfill embeddings for any chunks that don't have
them. Safety net for large syncs (>100 files) or prior `--no-embed` runs.
- `gbrain sync --watch --repo <path>` -- foreground polling loop, every 60s
@@ -97,15 +102,27 @@ Triggers sync on push events for instant sync (<5s).
### What Gets Synced
Sync only indexes "syncable" markdown files. These are excluded by design:
- Hidden paths (`.git/`, `.raw/`, etc.)
- The `ops/` directory
- Meta files: `README.md`, `index.md`, `schema.md`, `log.md`
- Hidden paths (`.git/`, `.raw/`, etc.) and vendored/generated trees
(`node_modules/`, `dist/`, `build/`, `venv/`)
- Meta files: `README.md`, `index.md`, `schema.md`, `log.md`, `RESOLVER.md`
### Sync is Idempotent
Everything else is ordinary synced content — including `ops/` (the bundled
daily-task-manager skill files its canonical page under `ops/tasks`).
### Sync is Idempotent — and Resumable
Concurrent runs are safe. Two syncs on the same commit no-op because content
hashes match. If both a cron and `--watch` fire simultaneously, no conflict.
Long syncs also survive being killed: progress checkpoints into the database
as files drain, so a killed or aborted run resumes from where it stopped, and
the sync bookmark only advances on true completion. A progress-aware stall
watchdog (`GBRAIN_SYNC_STALL_ABORT_SECONDS`, default 900, `0` disables) aborts
a run that stops making forward progress and releases the per-source lock so
the next `gbrain sync` picks up from the checkpoint. The checkpoint cadence
and lock-steal grace are tunable via `GBRAIN_SYNC_*` / `GBRAIN_LOCK_*` env
vars — incident-time escape hatches, not everyday knobs.
## Tricky Spots
1. **Always chain sync + embed.** Running `gbrain sync` without
docs/guides/meeting-ingestion.md
+13 -2
View File
@@ -58,15 +58,20 @@ on new_meeting_transcript(meeting):
# Schedule: cron 3x/day (10 AM, 4 PM, 9 PM) to catch new meetings
# Source: Circleback (https://circleback.ai) or any service with
# speaker diarization + API/webhook access
# Automation: the built-in `extract-timeline-from-meetings` Minion job
# automates step 3 (entity timeline propagation) for already-ingested
# meeting pages: gbrain jobs submit extract-timeline-from-meetings --follow
```
## Tricky Spots
1. **Always pull the COMPLETE transcript, never the AI summary.** AI summaries hallucinate framing -- they editorialize what was "agreed" or "decided" when no such agreement happened. The diarized transcript is ground truth.
2. **Entity propagation is the step most agents skip.** A meeting is NOT fully ingested until every attendee's page, every mentioned person's page, and every company's page has a new timeline entry. The meeting page alone is useless without propagation.
3. **Mentioned people are not just attendees.** If the meeting discussed "Sarah's team at Brex," then Sarah's page AND Brex's page need updates -- even though Sarah wasn't in the room.
4. **The agent's analysis is the value, not a summary.** "They discussed Q2 targets" is worthless. "Pedro pushed back on the burn rate, Diana didn't commit to the timeline, and nobody addressed the pricing gap" is useful.
3. **Mentioned people are not just attendees.** If the meeting discussed "Alice's team at widget-co," then Alice's page AND widget-co's page need updates -- even though Alice wasn't in the room.
4. **The agent's analysis is the value, not a summary.** "They discussed Q2 targets" is worthless. "Alice pushed back on the burn rate, Charlie didn't commit to the timeline, and nobody addressed the pricing gap" is useful.
5. **Back-links must be bidirectional.** The meeting page links to attendee pages AND attendee pages link back to the meeting. The graph is bidirectional. Always.
6. **`--source` on `timeline-add` is the citation text, not source routing.** Because the op declares its own `source` parameter, the CLI binds `--source` to it. To write into a different registered source, use the `.gbrain-source` dotfile or `GBRAIN_SOURCE` env for routing instead.
## How to Verify
@@ -76,5 +81,11 @@ on new_meeting_transcript(meeting):
4. Run `gbrain call get_links '{"slug": "meetings/{date}-{slug}"}'`. Verify back-links exist to all attendee and entity pages.
5. Run `gbrain search "{meeting_topic}"`. Confirm the meeting page appears in search results (verifies sync ran).
## Related
- `skills/meeting-ingestion/SKILL.md` — the bundled, agent-executable skill
for this workflow (the canonical step-by-step home; this guide is the
pattern overview).
---
*Part of the [GBrain Skillpack](../GBRAIN_SKILLPACK.md).*
docs/guides/minions-deployment.md
+29 -22
View File
@@ -40,6 +40,10 @@ gbrain jobs supervisor status --json
# Graceful stop (SIGTERM + drain wait + SIGKILL fallback).
gbrain jobs supervisor stop
# Optional: cap worker memory in MB (--max-rss). Without the flag the RSS
# watchdog is still on, at a RAM-relative auto-sized cap.
gbrain jobs supervisor --concurrency 4 --max-rss 4096
```
**Exit codes:**
@@ -50,9 +54,11 @@ gbrain jobs supervisor stop
| 1 | Max crashes exceeded (worker kept dying) |
| 2 | Another supervisor holds the PID lock |
| 3 | PID file unwritable (permission / path error) |
| 4 | Queue-scoped DB lock lost mid-run (`LOCK_LOST` — exited rather than risk a split-brain) |
An agent seeing exit=2 can safely treat it as "one is already running";
exit=1 should page a human.
exit=4 as "restart me — the DB lock refresh failed"; exit=1 should page
a human.
### Lowering scheduling priority (`--nice`)
@@ -233,7 +239,7 @@ use a dedicated queue name like `nightly-enrich` above.
## Upgrading from an older deployment
### From `minion-watchdog.sh` (pre-v0.20)
### From `minion-watchdog.sh`
Earlier versions of this guide shipped a 68-line bash watchdog
(`minion-watchdog.sh`). It's been replaced by `gbrain jobs supervisor`
@@ -270,10 +276,10 @@ Regardless of which deployment path you're upgrading from:
in-flight job landing partial schema.
2. **Run `gbrain upgrade`**. Then `gbrain apply-migrations --yes` if
`gbrain doctor` reports any migration as `partial` or `pending`.
3. **If you run shell jobs:** from v0.14 onward, pass
`--allow-shell-jobs` to the supervisor (or keep
`GBRAIN_ALLOW_SHELL_JOBS=1` in `/etc/gbrain.env`). Submitters don't
need the flag; only the worker does.
3. **If you run shell jobs:** pass `--allow-shell-jobs` to the
supervisor (or keep `GBRAIN_ALLOW_SHELL_JOBS=1` in
`/etc/gbrain.env`). Submitters don't need the flag; only the worker
does.
4. **Verify.** `gbrain doctor` should report zero `pending` or `partial`
migrations plus a healthy `supervisor` check. `gbrain jobs stats`
should show no unexplained growth in `dead` between pre- and
@@ -283,29 +289,30 @@ Regardless of which deployment path you're upgrading from:
### Supabase connection drops
The worker uses a single Postgres connection. If Supabase drops it
(maintenance, connection limits, network blip), lock renewal fails
silently. The stall detector then dead-letters the job after
`max_stalled` misses.
If Supabase drops the worker's Postgres connection (maintenance,
connection limits, network blip), this now self-heals under the
supervisor: the worker's DB-liveness probe self-exits (`db_dead`) on a
dead pool and the supervisor respawns it with a fresh pool, and the
supervisor also restarts a worker that stops making progress while
claimable work waits. The escalation commands and thresholds live in the
[queue operations runbook](queue-operations-runbook.md) — that's the
canonical home for wedge recovery.
**Current defaults that make this worse:**
What can still bite: a *brief* blip during a long-running job can make
lock renewal miss, and the stall detector dead-letters the job after
`max_stalled` misses (schema column default 5; lock duration and stall
check interval are both 30 s).
- `lockDuration: 30000` (30 s) — too short for long jobs during
connection blips.
- `max_stalled: 5` (schema column default — see `src/schema.sql` and
`src/core/pglite-schema.ts`). Five missed heartbeats before dead-letter.
- `stalledInterval: 30000` (30 s) — checks too aggressively.
**Tune per-job today.** `gbrain jobs submit` accepts `--max-stalled N`,
**Tune per-job.** `gbrain jobs submit` accepts `--max-stalled N`,
`--backoff-type fixed|exponential`, `--backoff-delay <ms>`,
`--backoff-jitter 0..1`, and `--timeout-ms N` as first-class flags
(since v0.13.1). These write onto the job row at submit time — which is
what `handleStalled()` reads — so per-job tuning is the real knob today.
`--backoff-jitter 0..1`, and `--timeout-ms N` as first-class flags.
These write onto the job row at submit time — which is what
`handleStalled()` reads — so per-job tuning is the real knob.
### DO NOT pass `maxStalledCount` to `MinionWorker`
It's a no-op. The stall detector reads the row's `max_stalled` column
(set at submit time), not the worker opt in `src/core/minions/worker.ts:74`.
(set at submit time), not the worker opt in `src/core/minions/worker.ts`.
Use `gbrain jobs submit --max-stalled N` per-job instead.
### Zombie shell children
docs/guides/minions-fix.md
+9 -6
View File
@@ -1,5 +1,9 @@
# Minions fix — repairing a half-migrated install
> **Historical repair guide** for the v0.11.0 → v0.11.1 migration. If you're
> on any recent release, the canonical fix below (`gbrain apply-migrations
> --yes`) is all you need; the stopgap sections exist for archaeology.
**tl;dr:** on v0.11.1+ everything should self-heal. If Minions is partially
set up (no `~/.gbrain/preferences.json`, autopilot still inline, cron jobs
still on `agentTurn`), run:
@@ -34,17 +38,16 @@ stopgap for pre-v0.11.1 binaries that don't have `apply-migrations`.
gbrain doctor
```
If the install is half-migrated, you'll see:
If the install is half-migrated, you'll see the `minions_migration` check
fail:
```
[FAIL] minions_migration: MINIONS HALF-INSTALLED (partial migration: 0.11.0). Run: gbrain apply-migrations --yes
```
or
```
[FAIL] minions_config: MINIONS HALF-INSTALLED (schema v7+ but no ~/.gbrain/preferences.json). Run: gbrain apply-migrations --yes
```
(Missing `~/.gbrain/preferences.json` on a fresh install is a valid
pre-`apply-migrations` state — doctor deliberately does NOT fail on that
alone; the partial-migration record is the canonical half-migration signal.)
For a machine-readable report (cron-friendly):
docs/guides/minions-shell-jobs.md
+6 -7
View File
@@ -129,11 +129,10 @@ JSONL records the same. Pre-enqueue validation rejects the submission if the
worker can't resolve the requested key, with a paste-ready
`gbrain config set database_url <value>` hint.
**Why not just write the URL into `env:` directly?** Pre-v0.36.5.0 callers
wrote things like:
**Why not just write the URL into `env:` directly?** You *can*:
```jsonc
// ❌ Deprecated as of v0.36.5.0 — REJECTED at submit time.
// ❌ Works, but plants the secret in the job row. Prefer inherit:.
{
"cmd": "gbrain stats",
"cwd": "/data/gbrain",
@@ -141,14 +140,14 @@ wrote things like:
}
```
This planted plaintext secrets in `minion_jobs.data` (DB row) and in the
This plants plaintext secrets in `minion_jobs.data` (DB row) and in the
shell-audit JSONL. Anyone with read access to the brain DB (or a brain dump,
or a shared brain via the mounts feature) saw the URL. v0.36.5.0 doesn't
forbid that pattern — the validator trusts the agent — but **prefer
or a shared brain via the mounts feature) sees the URL. The validator
doesn't forbid the pattern — it trusts the agent — but **prefer
`inherit:`** for any secret you want kept out of the row. Names land in the
row; values resolve at child-spawn from the worker's config.
**Scope:** v0.36.5.0 `inherit:` is **free-form**. Pass any snake_case
**Scope:** `inherit:` is **free-form**. Pass any snake_case
config-key name and the worker resolves the value from `loadConfig()` at
child-spawn time:
docs/guides/multi-source-brains.md
+34 -19
View File
@@ -5,6 +5,11 @@ is a `source`: a logical brain-within-the-brain with its own slug
namespace, its own sync state, and its own federation policy. The rest
of this guide walks the three canonical scenarios.
(Sources are the *within-one-database* axis. If you want to connect a
whole separate database — a team-published brain with its own access
policy — that's the *brain* axis: `gbrain mounts add`. See
`docs/architecture/brains-and-sources.md` for the two-axis topology.)
## The three scenarios
### 1. Unified knowledge recall (wiki + gstack)
@@ -66,14 +71,14 @@ gbrain search "tech layoffs" --source yc-media,garrys-list
### 3. Mixed (wiki federated + sessions isolated)
Your main wiki is federated with a few trusted sources. Your session
transcripts (coming in v0.18) land in a separate isolated source so
they don't dominate every search result.
transcripts (`gbrain transcripts` ingests them) land in a separate
isolated source so they don't dominate every search result.
```bash
# Federated sources
gbrain sources add gstack --path ~/.gstack --federated
# Isolated source (future v0.18 — sessions use this shape today for ingest)
# Isolated source for session transcripts
gbrain sources add sessions --path ~/.claude/sessions --no-federated
```
@@ -104,14 +109,16 @@ Every source row stores `config.federated: boolean` in its JSONB config.
| `true` | Source participates in unqualified `gbrain search "X"` results. |
| `false` (default for new sources) | Source only searched when explicitly named via `--source <id>` or qualified citation. |
The seeded `default` source is `federated=true` so pre-v0.17 brains
behave exactly as before — every page appears in search.
The seeded `default` source is `federated=true` so single-source brains
behave as you'd expect — every page appears in search.
Flip later with `gbrain sources federate <id>` / `unfederate <id>`.
## Commands
Full subcommand reference:
The most-used subcommands (run `gbrain sources --help` for the full,
always-current reference — it also covers `status`, `current`,
`set-cr-mode`, and the `push`/`pull` durability surface):
```
gbrain sources add <id> --path <p> [--name <n>] [--federated|--no-federated] [--force]
@@ -119,9 +126,17 @@ gbrain sources add <id> --path <p> [--name <n>] [--federated|--no-federated] [--
--path must be a git repo (or a subdirectory of one) — see
"The git requirement for --path sources" below. --force
skips that check to register before git-init exists.
gbrain sources add <id> --url <git-url> [--pat-file <p>] [--clone-dir <path>] [--no-harden]
Clone + register a remote repo in one step; auto-hardens
for durability when a PAT is provided (see "Durability" below).
gbrain sources list [--json] List all sources with page counts + federation state.
gbrain sources remove <id> [--yes] [--dry-run] [--keep-storage]
Cascade-delete a source (pages, chunks, timeline).
gbrain sources archive <id> Soft-delete: hide from search, keep data for a TTL
grace window. Prefer this over `remove`.
gbrain sources restore <id> Un-archive. `gbrain sources archived` lists expiries;
`gbrain sources purge` permanently deletes expired archives.
gbrain sources remove <id> [--confirm-destructive] [--dry-run]
Permanently cascade-delete a source (pages, chunks,
timeline). Shows an impact preview first.
gbrain sources rename <id> <new-name>
Change display name only; id is immutable.
gbrain sources default <id> Set the brain-level default.
@@ -253,8 +268,8 @@ reachable only over a filesystem path, set `GBRAIN_GIT_ALLOW_FILE_TRANSPORT=1`
## Upgrading an existing brain
`gbrain upgrade` runs the v16 + v17 migrations automatically. Your
existing pages all move under `source_id='default'`. Behavior is
`gbrain upgrade` runs the needed schema migrations automatically. Your
existing pages all live under `source_id='default'`. Behavior is
unchanged until you add a second source.
To add one:
@@ -266,13 +281,13 @@ cd ~/.gstack && gbrain sources attach gstack && gbrain sync
Two commands. The existing default source is untouched.
## Not in v0.18.0
## Related features that build on sources
- Session transcript ingest (`.jsonl`, raised size cap, session
PageType) — v0.18.
- Per-source retention/TTL (`gbrain sources prune`) — v0.18.
- ACL enforcement via caller-identity — v0.17.1.
- `gbrain sources import-from-github <url>` one-shot bootstrap — patch
release after the core plumbing stabilizes.
All of these build on the `sources` primitive shipped here.
- **Session transcript ingest** — `gbrain transcripts` (server-private:
raw chat exports stay on the host machine).
- **Per-source retention** — `gbrain sources archive` / `archived` /
`purge` (soft-delete with a TTL grace window).
- **One-shot remote bootstrap** — `gbrain sources add <id> --url <git-url>`
(clone + register + auto-harden).
- **Access control across brains** — the *brain* axis (`gbrain mounts`);
see `docs/architecture/brains-and-sources.md`.
docs/guides/operational-disciplines.md
+13 -10
View File
@@ -43,13 +43,14 @@ on information_needed(topic):
# An agent that reaches for the web before checking its own brain
# is wasting money and giving worse answers.
# DISCIPLINE 3: Sync After Every Write (MANDATORY)
on brain_write_complete():
# DISCIPLINE 3: Sync After Every Repo Write (MANDATORY)
on brain_repo_files_changed():
gbrain sync
# Without this, search results are stale.
# The page you just wrote won't appear in gbrain search or gbrain query
# until sync runs. Skipping this means the next lookup misses the
# most recent data.
# `gbrain put` indexes immediately -- pages written through the CLI/MCP
# are searchable the moment the command returns. No sync needed there.
# But files written DIRECTLY to the brain repo (an editor, a script,
# another agent committing markdown) are invisible to search until
# `gbrain sync` imports them. If anything touched repo files, sync.
# DISCIPLINE 4: Daily Heartbeat Check
on daily_schedule("09:00"):
@@ -62,6 +63,8 @@ on daily_schedule("09:00"):
on nightly_schedule("02:00"):
# The dream cycle is the most important discipline.
# The brain COMPOUNDS overnight.
# gbrain ships this as a first-class command: `gbrain dream`.
# The pseudocode below is the shape of the work it does.
# 5a: Entity sweep -- find unlinked mentions
pages = gbrain list
@@ -103,16 +106,16 @@ on nightly_schedule("02:00"):
## Tricky Spots
1. **The dream cycle is the most important discipline.** Brains compound overnight. Entity sweeps fix broken graphs, citation audits catch sourceless facts, and memory consolidation keeps compiled truth current. Skip the dream cycle and the brain slowly rots.
2. **Skipping Discipline 3 (sync after write) means stale search results.** You write a page, then immediately search for it -- and get nothing back. The page exists but isn't indexed. Always sync after writes.
3. **Signal detection must fire on EVERY message.** Not just messages that look important. The user says "I talked to Pedro yesterday about the board seat" in passing -- that's a timeline entry on Pedro's page, a potential update to his State section, and a signal about the board. If the agent doesn't catch it, the system is broken.
4. **Brain-first saves money AND gives better answers.** The brain has context that external APIs don't: relationship history, meeting notes, the user's own assessment. An API lookup for "Pedro Franceschi" returns a LinkedIn profile. The brain returns the full picture including private context.
2. **Skipping Discipline 3 (sync after repo writes) means stale search results.** A file lands in the brain repo, then you search for it -- and get nothing back. The file exists but isn't imported. Always sync after repo-file writes. (`gbrain put` is exempt: it indexes on write.)
3. **Signal detection must fire on EVERY message.** Not just messages that look important. The user says "I talked to Alice yesterday about the board seat" in passing -- that's a timeline entry on Alice's page, a potential update to her State section, and a signal about the board. If the agent doesn't catch it, the system is broken.
4. **Brain-first saves money AND gives better answers.** The brain has context that external APIs don't: relationship history, meeting notes, the user's own assessment. An API lookup for "Alice Example" returns a LinkedIn profile. The brain returns the full picture including private context.
5. **`gbrain doctor` catches silent failures.** Embedding pipelines can stall, sync can fail silently, database connections can drop. The daily heartbeat catches these before they compound into data loss.
## How to Verify
1. Send a message mentioning a person with a brain page. Confirm the agent detects the entity and adds a timeline entry to their page (`gbrain timeline <slug>`).
2. Ask the agent about someone in the brain. Confirm it runs `gbrain search` or `gbrain get` BEFORE reaching for external APIs (check the tool call order).
3. Write a new page with `gbrain put`, then immediately run `gbrain search` for it. Confirm it appears in results (verifies sync ran).
3. Write a markdown file directly into the brain repo (not via `gbrain put`), run `gbrain sync`, then `gbrain search` for it. Confirm it appears in results (verifies the sync discipline). A `gbrain put` page should appear in search immediately, with no sync.
4. Run `gbrain doctor`. Confirm it returns a health report with database status, page count, and any flagged issues.
5. After a dream cycle runs, check a page that had unlinked entity mentions. Confirm new links were added (`gbrain call get_links '{"slug": "<slug>"}'`).
docs/guides/plugin-authors.md
+14 -10
View File
@@ -1,4 +1,4 @@
# Plugin authors guide (v0.15)
# Plugin authors guide — subagent definitions
`gbrain` discovers subagent definitions from outside this repo via
`GBRAIN_PLUGIN_PATH`. If you maintain a downstream agent (your OpenClaw
@@ -7,6 +7,11 @@ subagents alongside it, drop a plugin directory on that env path.
This guide is for plugin authors. The CLI user doesn't need to read it.
> **Two plugin systems.** This doc covers *subagent definitions*
> (markdown prompts the `subagent` job handler runs). Custom *job
> handlers* (code the Minion worker executes) are a separate system —
> see [plugin-handlers.md](plugin-handlers.md).
## Minimum viable plugin
```
@@ -65,7 +70,7 @@ You control where your plugin lives on disk; `gbrain` doesn't guess.
the one listed FIRST in `GBRAIN_PLUGIN_PATH` wins. The other is dropped
with a warning naming both sources.
**Trust policy.** Plugins ship subagent definitions ONLY in v0.15:
**Trust policy.** Plugins ship subagent definitions ONLY:
- You **cannot** declare new tools.
- You **cannot** extend the brain tool allow-list.
@@ -76,8 +81,8 @@ with a warning naming both sources.
your plugin gives you a loud startup error, not a silent "tool never
fires" at 3am.
v0.16+ may open up plugin-declared tools with a separate contract. Don't
expect it.
Plugin-declared tools would require a new `plugin_version` contract;
nothing under `gbrain-plugin-v1` opens that up.
## `gbrain.plugin.json`
@@ -85,9 +90,9 @@ expect it.
|------------------|--------|----------|--------------------------------------------------------------------|
| `name` | string | yes | Human-readable plugin id. Shows up in warnings and collision logs. |
| `version` | string | yes | Your plugin's semver. Informational. |
| `plugin_version` | string | yes | Contract lock. Must equal `"gbrain-plugin-v1"` for v0.15. |
| `plugin_version` | string | yes | Contract lock. Must equal `"gbrain-plugin-v1"`. |
| `subagents` | string | no | Subdir name (default `subagents`). Escape-attempts are rejected. |
| `description` | string | no | Shown in a future plugin-listing command. |
| `description` | string | no | Informational; appears in load/collision warnings. |
## Subagent definition files
@@ -103,8 +108,7 @@ Recognized frontmatter fields:
| `max_turns` | number | no | Cap on assistant turns. Defaults to 20. |
| `allowed_tools` | string[] | no | Whitelist of tool names. Must subset the derived brain registry. Rejected on mismatch. |
Unknown frontmatter fields are preserved but ignored by the handler. v0.16
may consume more of them.
Unknown frontmatter fields are preserved but ignored by the handler.
## Caveats that will bite you
@@ -115,8 +119,8 @@ may consume more of them.
2. **`~/.gbrain/audit/subagent-jobs-*.jsonl` is local only.** If your
worker runs on a different host than the `gbrain agent logs` caller,
the CLI won't see heartbeats from that worker. v0.16 will unify this;
for now assume worker + CLI share a filesystem.
the CLI won't see heartbeats from that worker. Assume worker + CLI
share a filesystem.
3. **Tool calls always run with `ctx.remote = true`.** Even on local CLI
invocation. Tools that gate on `remote=true` (file_upload's strict
docs/guides/plugin-handlers.md
+26 -7
View File
@@ -1,7 +1,12 @@
# Plugin handlers — registering host-specific Minion handlers
GBrain's Minion worker ships with seven built-in handlers: `sync`,
`embed`, `lint`, `import`, `extract`, `backlinks`, `autopilot-cycle`.
GBrain's Minion worker ships with a full set of built-in handlers,
registered by `registerBuiltinHandlers` in `src/commands/jobs.ts` —
that registry is the source of truth. Examples: `sync`, `embed`,
`lint`, `import`, `extract`, `backlinks`, `autopilot-cycle`, `shell`,
`subagent`, `orphans`, `integrity`, plus dream-cycle phases and other
maintenance jobs. Submitting an unknown job name with
`gbrain jobs submit <name> --follow` prints the full registered list.
These cover every background operation the gbrain CLI itself performs.
Host platforms (OpenClaw deployments, future hosts) register their own
@@ -10,6 +15,11 @@ handlers via a plugin bootstrap that imports
code, loaded by the worker, with the same trust model as any other
code in the host's repo.
> **Two plugin systems.** This doc covers *job handlers* (code the Minion
> worker runs). Custom *subagent definitions* (markdown prompts loaded via
> `GBRAIN_PLUGIN_PATH`) are a separate system — see
> [plugin-authors.md](plugin-authors.md).
## Why code, not data
An earlier design draft shipped `~/.claude/gbrain-handlers.json` where
@@ -64,14 +74,23 @@ auto-loads on startup (configurable via a host-provided entry point).
## Handler contract
Every handler receives a `MinionJobContext`:
Every handler receives a `MinionJobContext` (canonical definition:
`src/core/minions/types.ts`). The load-bearing fields:
```ts
interface MinionJobContext {
data: Record<string, unknown>; // job params (whatever the cron submit passed)
job: MinionJob; // full job row (id, queue, attempts, etc.)
signal: AbortSignal; // set to aborted when the worker is shutting down
inbox: MinionInbox; // read messages sent to this job while it runs
id: number; // job id
name: string; // job type
data: Record<string, unknown>; // job params (whatever the cron submit passed)
attempts_made: number;
signal: AbortSignal; // fires on timeout, cancel, pause, or lock loss
shutdownSignal: AbortSignal; // fires only on worker SIGTERM/SIGINT
deadlineAtMs: number | null; // wall-clock deadline from timeout_at, if set
updateProgress(progress: unknown): Promise<void>;
updateTokens(tokens: TokenUpdate): Promise<void>;
log(message: string | TranscriptEntry): Promise<void>;
isActive(): Promise<boolean>; // is the job lock still held?
readInbox(): Promise<InboxMessage[]>; // unread messages sent to this job
}
```
docs/guides/queue-operations-runbook.md
+19 -13
View File
@@ -1,8 +1,8 @@
# Queue operations runbook
"My queue looks wedged — what do I run?" The commands below are in the order
you probably want them. Shipped with v0.19.1 after a production incident
where the queue held for 90+ minutes before the operator noticed.
you probably want them. Born from a production incident where the queue held
for 90+ minutes before the operator noticed.
## First signal: jobs aren't running
@@ -23,16 +23,18 @@ container health), but its DB connection died (common behind a transaction
pooler) and never came back, so it claims no jobs and finishes nothing. Jobs
pile up with **0 active**. Liveness checks all pass; nothing crashes.
As of v0.42.22.0 this self-heals — you usually won't have to do anything:
This self-heals — you usually won't have to do anything:
- **The worker exits on its own dead pool.** Under a supervisor, the worker's
DB-liveness probe runs and self-exits (`db_dead`) after ~3 minutes; the
supervisor respawns it with a fresh pool.
- **The supervisor restarts a worker that stops making progress.** If a queue
has claimable work, **0 live-lock active jobs**, and no completions for 15
minutes while the child is alive, the supervisor restarts it (covers stuck
handlers too, not just dead pools). Tune with `--wedge-restart-minutes` /
`--wedge-restart-checks` on `gbrain jobs supervisor` (0 disables).
minutes across 3 consecutive health checks while the child is alive, the
supervisor restarts it (covers stuck handlers too, not just dead pools).
These thresholds are built in — there are no CLI flags to tune them. A
restart-loop breaker caps wedge restarts at 3 per 30-minute window, then
switches to a one-shot `wedge_restart_loop` alert in the audit log.
The signal is loud now — check either:
@@ -45,8 +47,12 @@ gbrain doctor --json | jq '.checks[] | select(.name == "wedged_queue")'
stale completions). Manual fix if you ever need it:
```bash
gbrain jobs supervisor stop && gbrain jobs supervisor start # fresh pool
gbrain jobs retry <id> # dead-lettered jobs
# Restart the supervisor with a fresh pool. `start` alone runs in the
# FOREGROUND (blocks); use --detach to get your shell back.
gbrain jobs supervisor stop && gbrain jobs supervisor start --detach --json
# Re-queue any jobs that were dead-lettered during the wedge.
gbrain jobs retry <id>
```
## Triage commands
@@ -101,9 +107,9 @@ claiming. Start one:
GBRAIN_ALLOW_SHELL_JOBS=1 gbrain jobs work --concurrency 4
```
## Follow-ups tracked for v0.20+
## Related
- B7 — `minion_workers` heartbeat table for ground-truth liveness (the
`--no-worker` probe and the dropped `queue_health` worker-heartbeat
subcheck both need this).
- B3 — `gbrain doctor --fix` learns to rescue queue wedges.
- [Minions worker deployment](minions-deployment.md) — supervisor lifecycle,
exit codes, and per-platform deployment (systemd / Fly / Render).
- [Minions shell jobs](minions-shell-jobs.md) — the `shell` job type's
security model and error table.
docs/guides/quiet-hours.md
+21 -1
View File
@@ -113,6 +113,22 @@ fi
send_notification "$OUTPUT"
```
### GBrain-native hooks
Two places gbrain already understands quiet hours natively — use these
before rolling your own gate for the same job:
- **Self-upgrade** — `auto` mode only applies upgrades during quiet hours,
configured via `gbrain config set self_upgrade.quiet_hours
'{"start":23,"end":8,"tz":"US/Pacific"}'`. See
[upgrades-auto-update.md](upgrades-auto-update.md).
- **Cron prompts** — schedule-driven notification jobs should carry the
gate described in this doc; [cron-schedule.md](cron-schedule.md) covers
the scheduling side.
The shell pattern below is for everything else: your own cron jobs,
collectors, and notification paths that gbrain doesn't gate for you.
### Configurable Hours
Some users want different quiet hours. Store the config:
@@ -140,7 +156,11 @@ Set `enabled: false` to disable quiet hours entirely (e.g., for 24/7 monitoring)
skill reads and clears the held directory. Orphaned held files mean the
pickup integration is broken.
3. **Timezone auto-detection is fragile.** Calendar-based timezone detection
3. **`/tmp` doesn't survive reboots (or, on macOS, periodic cleanup).** If a
held message must not be lost across a restart, use a durable held
directory (e.g. `~/.local/state/cron-held/`) instead of `/tmp/cron-held/`.
4. **Timezone auto-detection is fragile.** Calendar-based timezone detection
relies on the user having airline/hotel events with location data. If the
user books travel without calendar entries, the system won't detect the
move. Fall back to activity-hour analysis (responding at 3 AM PT = probably
docs/guides/repo-architecture.md
+18 -6
View File
@@ -9,8 +9,9 @@ Separate agent behavior (replaceable) from world knowledge (permanent) into two
Without this: agent config and world knowledge are mixed together. Switch agents
and you lose your knowledge. Switch knowledge tools and you lose your agent setup.
With this: your brain (14,700+ files of people, companies, meetings, ideas)
survives any agent swap. Your agent config survives any knowledge tool swap.
With this: your brain (thousands of files of people, companies, meetings,
ideas) survives any agent swap. Your agent config survives any knowledge
tool swap.
## Implementation
@@ -119,7 +120,15 @@ without losing your agent setup.
notes). The agent repo contains operational config. Different access controls.
**GBrain indexes the brain repo.** Run `gbrain sync --repo ~/brain/` to keep
the search index current. The agent repo is never indexed by GBrain.
the search index current. The agent repo is not indexed by default.
**Multi-source nuance.** With multi-source brains
([multi-source-brains.md](multi-source-brains.md)), "the brain repo" means
"each registered source." You CAN deliberately register a non-brain repo
(e.g. `~/.gstack`) as its own isolated or federated source — that's a
conscious registration with its own slug namespace, not a violation of the
boundary. The rule below is about *unregistered, accidental* indexing of
agent config.
## Tricky Spots
@@ -133,9 +142,12 @@ the search index current. The agent repo is never indexed by GBrain.
belongs in the brain. Agent configs, skills, cron jobs, and operational
state are replaceable. People, companies, ideas, and meetings are not.
3. **Don't index the agent repo.** GBrain indexes the brain repo only.
Running `gbrain sync` against the agent repo pollutes search results
with operational config instead of world knowledge.
3. **Don't casually index the agent repo.** Running `gbrain sync` against
the agent repo pollutes search results with operational config instead
of world knowledge. (Registering it deliberately as an isolated source
is different — see the multi-source nuance above. Pin the working
directory to the right source with a `.gbrain-source` dotfile via
`gbrain sources attach <id>`.)
## How to Verify
docs/guides/rls-and-you.md
+43 -20
View File
@@ -34,35 +34,35 @@ docs/guides/rls-and-you.md for the GBRAIN:RLS_EXEMPT comment escape hatch.
99% of the time, you want the fix. Run the SQL. Re-run `gbrain doctor`. Done.
## v0.26.7 — auto-RLS event trigger and one-time backfill
## Auto-RLS: the event trigger and the one-time backfill
Starting in v0.26.7 (migration v35), gbrain ships two changes that close the
gap where a table could exist in your `public` schema without RLS for any
amount of time at all.
gbrain ships two mechanisms (schema migration v35) that close the gap where a
table could exist in your `public` schema without RLS for any amount of time
at all.
**1. The event trigger.** A Postgres DDL event trigger named
`auto_rls_on_create_table` runs `ALTER TABLE … ENABLE ROW LEVEL SECURITY`
on every newly created `public.*` table. It covers `CREATE TABLE`,
`CREATE TABLE AS … SELECT`, and `SELECT … INTO` — every syntax Postgres
reports as a table-creation command. Tables created by gbrain itself, by
your other apps sharing the same Supabase project (Baku, Hermes, anything),
or by a human running raw SQL all get RLS enabled the moment they exist.
any other app sharing the same Supabase project, or by a human running raw
SQL all get RLS enabled the moment they exist.
Non-`public` schemas (`auth`, `storage`, `realtime`, etc.) are explicitly
ignored — Supabase manages those, and we should not touch them.
**2. The one-time backfill.** When you upgrade to v0.26.7, the migration
**2. The one-time backfill.** The first upgrade that applies migration v35
walks every existing `public.*` base table whose RLS is off and whose comment
doesn't carry the `GBRAIN:RLS_EXEMPT` exemption (see below) and enables RLS
on each. After the upgrade, `gbrain doctor`'s `rls` check should be a no-op
on every brain.
### Breaking change: read this before upgrading
### Read this before upgrading a pre-auto-RLS brain
If you have public tables that are intentionally RLS-off and you want them
to stay that way, you MUST add the `GBRAIN:RLS_EXEMPT` comment **before**
running `gbrain upgrade` to v0.26.7. The backfill flips RLS on for any public
table that doesn't carry the exact comment contract documented below. There
is no `--dry-run` flag on the migration.
the upgrade that applies migration v35. The backfill flips RLS on for any
public table that doesn't carry the exact comment contract documented below.
There is no `--dry-run` flag on the migration.
The minimum cost of getting this wrong is one round-trip: the operator runs
the SQL to enable RLS on a table that should have been exempt, then
@@ -71,7 +71,7 @@ prevent a re-flip on a later doctor run. No data is lost.
### Cross-app implications
If a non-gbrain app (Baku, Hermes, a script you wrote, anything) creates
If a non-gbrain app (a side project, a script you wrote, anything) creates
tables in the same Supabase project, the trigger will enable RLS on those
tables too. Two ways to handle that:
@@ -90,17 +90,40 @@ ship a policy.
### What if the trigger gets dropped?
`gbrain doctor` includes a new `rls_event_trigger` check that verifies the
`gbrain doctor` includes an `rls_event_trigger` check that verifies the
trigger is installed and enabled. If you drop it manually for any reason
(debugging, migration testing, anything), doctor warns and gives you the
recovery command:
(debugging, migration testing, anything), doctor warns and points you here.
```
gbrain apply-migrations --force-retry 35
Recreate it by re-running the trigger DDL from migration v35 — idempotent
(`CREATE OR REPLACE` + `DROP EVENT TRIGGER IF EXISTS`), safe to paste into
psql as a BYPASSRLS role (e.g. `postgres`):
```sql
CREATE OR REPLACE FUNCTION auto_enable_rls()
RETURNS event_trigger AS $$
DECLARE
obj record;
BEGIN
FOR obj IN SELECT * FROM pg_event_trigger_ddl_commands()
WHERE object_type = 'table'
AND schema_name = 'public'
LOOP
EXECUTE format('ALTER TABLE %s ENABLE ROW LEVEL SECURITY', obj.object_identity);
END LOOP;
END;
$$ LANGUAGE plpgsql;
DROP EVENT TRIGGER IF EXISTS auto_rls_on_create_table;
CREATE EVENT TRIGGER auto_rls_on_create_table
ON ddl_command_end
WHEN TAG IN ('CREATE TABLE', 'CREATE TABLE AS', 'SELECT INTO')
EXECUTE FUNCTION auto_enable_rls();
```
Re-running migration v35 is idempotent — it `DROP EVENT TRIGGER IF EXISTS`
and recreates cleanly.
(This is the same DDL migration v35 runs — the canonical copy lives in the
`MIGRATIONS` array in `src/core/migrate.ts`. There's no CLI shortcut:
`gbrain apply-migrations --force-retry` targets the vX.Y.Z orchestrator
registry, not numeric schema migrations like v35.)
### Why no FORCE ROW LEVEL SECURITY?
@@ -147,7 +170,7 @@ Rules:
```sql
ALTER TABLE public.expenses_ramp DISABLE ROW LEVEL SECURITY;
COMMENT ON TABLE public.expenses_ramp IS
'GBRAIN:RLS_EXEMPT reason=analytics-only, anon-readable ok, owner=garry, 2026-04-22';
'GBRAIN:RLS_EXEMPT reason=analytics-only, anon-readable ok, owner=you, 2026-04-22';
```
After that, `gbrain doctor` reports:
docs/guides/scaling-skills.md
+10 -11
View File
@@ -127,9 +127,9 @@ you can use as a reference shape. The skillpack story for distributing
your own resolvers across machines is covered in
[skillpacks as scaffolding](skillpacks-as-scaffolding.md).
## The compact list format (v0.41.7.0)
## The compact list format
GBrain's resolver parser used to require markdown tables:
GBrain's resolver parser originally required markdown tables:
```markdown
| Trigger | Skill |
@@ -146,14 +146,14 @@ format that scales better:
- **flight-tracker**: track my flight | flight status | when does my flight land
```
Before v0.41.7.0, `gbrain doctor` only spoke the table dialect. On a
306-skill compact-format resolver, the doctor reported every skill as
unreachable: **238 FAIL errors on every doctor run**. The parser was
silently treating the compact dialect as zero skills.
When `gbrain doctor` only spoke the table dialect, a 306-skill
compact-format resolver reported every skill as unreachable: **238 FAIL
errors on every doctor run**. The parser was silently treating the compact
dialect as zero skills.
v0.41.7.0 ships dual-format support. The same `parseResolverEntries`
function reads both table rows and list rows in the same file, with the
v0.31.7 multi-resolver merge (skillpack `skills/RESOLVER.md` + workspace
Today the parser supports both. The same `parseResolverEntries`
function reads table rows and list rows in the same file, with the
multi-resolver merge (skillpack `skills/RESOLVER.md` + workspace
`../AGENTS.md`) folding everything into one unified view. Run `gbrain doctor`
and the 238 FAILs collapse to 0.
@@ -267,8 +267,7 @@ I initially converted my resolver from a clean list format to a table
format because the validator only spoke tables. That was wrong. When a
tool fails against valid data, the right move is to fix the tool, not
reshape the data. The list format was correct, compact, readable, easy
to maintain. The parser needed to support both shapes. v0.41.7.0 is
that fix.
to maintain. The parser needed to support both shapes — and now it does.
The same principle applies everywhere in agent systems. Your SKILL.md is
the source of truth. Your AGENTS.md is the source of truth. Your resolver
docs/guides/search-modes.md
+109 -18
View File
@@ -1,25 +1,115 @@
# Search Modes
## Goal
Know which search command to use and when -- keyword, hybrid, or direct -- so every lookup is fast and returns the right result.
Two decisions shape every gbrain lookup, and this guide covers both:
## What the User Gets
Without this: the agent fumbles between search commands, returns chunks when full pages are needed, runs expensive semantic searches when a direct get would do, or misses results entirely. With this: every lookup uses the optimal mode, token budgets are respected, and the user gets the right information in the fewest calls.
1. **Which mode bundle** your brain runs — `conservative` / `balanced` /
`tokenmax`, the named cost-knob presets that control cache, token budget,
query expansion, and result count. This is the config-level decision you
make once (at `gbrain init` or via `gbrain config set search.mode`).
2. **Which lookup verb** to use per call — `gbrain search` (keyword),
`gbrain query` (hybrid), or `gbrain get` (direct). This is the
per-lookup decision an agent makes on every question.
## Implementation
## The three mode bundles
A search mode is a named preset that sets every search-cost knob at once.
The bundles are frozen in `src/core/search/mode.ts` (`MODE_BUNDLES`):
| Knob | `conservative` | `balanced` | `tokenmax` |
|-------------------------------|----------------|------------|----------------|
| `cache.enabled` | true | true | true |
| `cache.similarity_threshold` | 0.92 | 0.92 | 0.92 |
| `cache.ttl_seconds` | 3600 | 3600 | 3600 |
| `intentWeighting` | true | true | true |
| `tokenBudget` | **4000** | **12000** | **off** |
| `expansion` (LLM multi-query) | false | false | **true** |
| `relationalRetrieval` | false | **true** | **true** |
| `searchLimit` default | 10 | 25 | 50 |
- **`conservative`** — smallest payloads. Pairs naturally with a cheap
downstream model (Haiku-class) or a high query volume.
- **`balanced`** — the default and the fallback when no mode is set.
- **`tokenmax`** — no token budget, LLM query expansion on, 50 results.
Pairs with an expensive downstream model you want fully fed.
Two of the knobs deserve a sentence:
- **`expansion`** rewrites your query into multiple variants via a cheap
LLM call per search (adds roughly $1.50 per 1K queries) — better recall,
small extra cost.
- **`relationalRetrieval`** adds a graph-walk recall arm for relational
questions ("who invested in X", "what connects A and B"); it's a pure
no-op for non-relational queries. The `query` op's `relational` flag
forces it on/off per call.
### Setting and resolving the mode
```bash
gbrain config set search.mode tokenmax
```
Per-knob resolution (highest first):
per-call SearchOpts → per-key config override (search.cache.enabled, …) →
MODE_BUNDLES[search.mode] → MODE_BUNDLES.balanced (fallback)
Mode resolution lives in bare `hybridSearch`, not just the cached wrapper,
so eval replays test the same mode-affected behavior as the production
`query` op. The query cache folds the active knobs into its key
(`knobs_hash`), so switching modes never serves you a stale result set
from a different configuration.
### Cost intuition
gbrain's own cost is rounding error; what the mode really controls is how
many tokens your *downstream agent* pays to read per query. The
corner-to-corner spread is ~25x once you pair mode with downstream model.
Rough anchors at 10K queries/month, full payload, no cache savings:
| Mode \ Downstream | Haiku-class (\$1/M in) | Sonnet-class (\$3/M in) | Opus-class (\$5/M in) |
|---|---|---|---|
| conservative (~4K tok) | **\$40/mo** | \$120/mo | \$200/mo |
| balanced (~10K tok) | \$100/mo | \$300/mo | \$500/mo |
| tokenmax (~20K tok) | \$200/mo | \$600/mo | **\$1,000/mo** |
Scales linearly with volume. Cache hits cut all numbers ~50%; disciplined
prompt caching in the agent loop cuts further. Mismatched pairings waste
capacity in both directions — a tokenmax payload overwhelms a cheap model,
a conservative payload starves an expensive one. The full methodology and
realistic-scale walkthrough live in
[`docs/eval/SEARCH_MODE_METHODOLOGY.md`](../eval/SEARCH_MODE_METHODOLOGY.md).
### CLI surfaces
```bash
gbrain search modes # what is running, with per-knob attribution
gbrain search modes --reset # clear search.* overrides (mode bundle wins)
gbrain search stats [--days N] # cache hit rate, intent mix, budget drops
gbrain search tune [--apply] # data-driven recommendations
gbrain search diagnose "<query>" --target <slug>
# trace where a page surfaces (or fails to)
# across the keyword/vector/alias/hybrid layers
```
The mode picker runs inside `gbrain init` (non-TTY auto-selects `balanced`).
## Choosing a lookup verb (search vs query vs get)
Independent of which bundle is active, every individual lookup should use
the cheapest verb that answers the question.
```
on user_asks_about(topic):
# Decision tree: pick the right search mode
# Decision tree: pick the right lookup verb
if know_exact_slug(topic):
# MODE 3: Direct get -- instant, no search overhead
# Direct get -- instant, no search overhead
result = gbrain get <slug>
# e.g., "Tell me about Pedro" -> gbrain get pedro-franceschi
# e.g., "Tell me about Alice" -> gbrain get alice-example
# Returns the FULL page -- compiled truth + timeline
elif topic.is_exact_name or topic.is_keyword:
# MODE 1: Keyword search -- fast, no embeddings needed, day-one ready
# Keyword search -- fast, no embeddings needed, day-one ready
results = gbrain search "{name_or_keyword}"
# e.g., "Find anything about Series A" -> gbrain search "Series A"
# Returns CHUNKS, not full pages
@@ -30,7 +120,7 @@ on user_asks_about(topic):
full_page = gbrain get <slug_from_chunk>
elif topic.is_semantic_question:
# MODE 2: Hybrid search -- semantic + keyword, needs embeddings
# Hybrid search -- semantic + keyword, needs embeddings
results = gbrain query "{natural language question}"
# e.g., "Who do I know at fintech companies?" -> gbrain query "fintech contacts"
# Returns ranked chunks via vector + keyword + RRF
@@ -40,7 +130,7 @@ on user_asks_about(topic):
full_page = gbrain get <slug_from_chunk>
# Quick reference:
# | Mode | Command | Needs Embeddings | Speed | Best For |
# | Verb | Command | Needs Embeddings | Speed | Best For |
# |---------|----------------------|------------------|---------|---------------------------------|
# | Keyword | gbrain search "term" | No | Fastest | Known names, exact matches |
# | Hybrid | gbrain query "..." | Yes | Fast | Semantic questions, fuzzy match |
@@ -58,21 +148,22 @@ on user_asks_about(topic):
# 4. External sources (web search, APIs)
```
## Tricky Spots
### Tricky Spots
1. **Search returns chunks, not full pages.** After `gbrain search` or `gbrain query`, you get excerpts. Always run `gbrain get <slug>` to load the full page when the chunk confirms relevance. Don't answer questions from chunks alone when the full context matters.
2. **Keyword search works without embeddings.** On day one before any embedding run, `gbrain search` still works. Don't tell the user "search isn't available yet" -- keyword search is always available.
3. **Don't use hybrid search for known names.** `gbrain query "Pedro Franceschi"` wastes embedding compute. Use `gbrain search "Pedro Franceschi"` or better yet `gbrain get pedro-franceschi` if you know the slug.
4. **Token budget awareness.** A full page via `gbrain get` can be large. Read the search chunks first to confirm relevance before pulling the full page. "Did anyone mention the Series A?" -- search results (chunks) are probably enough. "Tell me everything about Pedro" -- get the full page.
3. **Don't use hybrid search for known names.** `gbrain query "Alice Example"` wastes embedding compute. Use `gbrain search "Alice Example"` or better yet `gbrain get alice-example` if you know the slug.
4. **Token budget awareness.** A full page via `gbrain get` can be large. Read the search chunks first to confirm relevance before pulling the full page. "Did anyone mention the Series A?" -- search results (chunks) are probably enough. "Tell me everything about Alice" -- get the full page.
5. **Hybrid search needs embeddings to have been run.** If `gbrain query` returns nothing but `gbrain search` finds results, the embeddings haven't been generated yet. Run the embedding pipeline first.
## How to Verify
### How to Verify
1. Run `gbrain search "Pedro"` -- confirm it returns chunks with matching text and slug references.
1. Run `gbrain search "Alice"` -- confirm it returns chunks with matching text and slug references.
2. Run `gbrain query "who works at fintech companies"` -- confirm it returns semantically relevant results (not just keyword matches on "fintech").
3. Run `gbrain get pedro-franceschi` -- confirm it returns the full page with compiled truth and timeline.
4. Compare: search for the same entity using all three modes. Keyword should be fastest, hybrid should surface conceptual matches, direct should return the complete page.
3. Run `gbrain get alice-example` -- confirm it returns the full page with compiled truth and timeline.
4. Compare: search for the same entity using all three verbs. Keyword should be fastest, hybrid should surface conceptual matches, direct should return the complete page.
5. After a search returns a chunk, run `gbrain get` on the slug from that chunk. Confirm the full page contains more context than the chunk alone.
6. Run `gbrain search modes` -- confirm the active mode bundle and any per-key overrides are what you expect.
---
*Part of the [GBrain Skillpack](../GBRAIN_SKILLPACK.md).*
docs/guides/skill-development.md
+5 -2
View File
@@ -40,7 +40,9 @@ Show the user the results. Get feedback.
- Revise the process based on what you learned.
**Step 4: Codify into a Skill.**
Write the SKILL.md. Either:
Write the SKILL.md (`gbrain skillify scaffold <name>` scaffolds the tree for
you; `gbrain skillopt` optimizes an existing skill against a benchmark).
Either:
- **New skill** -- genuinely new capability
- **Add to existing skill** -- variation of something that exists (parameterize it)
@@ -63,7 +65,8 @@ Skills should be **Mutually Exclusive, Collectively Exhaustive**:
- Each signal source has exactly ONE owner skill
- Two skills creating the same brain page = MECE violation
**Example ownership (no overlap):**
**Example ownership (no overlap — illustrative; your skill roster will
differ):**
| Signal Source | Owner Skill | Creates |
|--------------|-------------|---------|
docs/guides/skillopt.md
+4 -4
View File
@@ -65,14 +65,14 @@ For each step:
more than 0.05.
6. **Commit.** D8 history-intent-first 5-step atomic write — crash-safe.
After each epoch with no improvement: D6 slow-update fires one meta-edit
proposal (this lives in v0.42 follow-up; v1 emits the audit event).
After each epoch with no improvement: D6 slow-update fires. Today it emits
the audit event only; the full meta-edit proposal is a tracked follow-up.
## Flags
| Flag | Default | Purpose |
|---|---|---|
| `--benchmark <path>` | `skills/<n>/skillopt-benchmark.jsonl` | Path to benchmark JSONL |
| `--benchmark <path>` | `skills/<name>/skillopt-benchmark.jsonl` | Path to benchmark JSONL |
| `--bootstrap-from-skill` | off | Generate a starter benchmark from SKILL.md (recommended; no routing-eval needed) |
| `--bootstrap-tasks N` | 15 | How many starter tasks `--bootstrap-from-skill` generates (max 50) |
| `--bootstrap-from-routing` | off | Auto-build benchmark from routing-eval.jsonl |
@@ -136,7 +136,7 @@ refuses to start when the estimate exceeds `--max-cost-usd`.
- **No benchmark.** Optimizing against guesses is worse than not optimizing.
- **Write-flavored skills.** Skills whose job is to `put_page` heavily can't
use the v1 read-only sandbox; mocked-write capture is a v0.42 follow-up.
use the read-only sandbox; mocked-write capture is a tracked follow-up.
- **Tiny benchmarks (<10 tasks).** D_sel < 5 refuses by default; meaningful
validation needs ≥20 tasks total per the paper.
docs/guides/skillpacks-as-scaffolding.md
+23 -14
View File
@@ -1,12 +1,11 @@
# Skillpacks as scaffolding, not amber
GBrain v0.33 reshapes `gbrain skillpack` from a package manager into a
scaffold + reference library. This guide explains the model and the
workflow.
`gbrain skillpack` is a scaffold + reference library, not a package
manager. This guide explains the model and the workflow.
## Why we changed it
## Why it works this way
Pre-v0.33 (the "amber" model):
An earlier design (the "amber" model):
- `gbrain skillpack install <name>` copied bundled skills into your
workspace AND wrote a managed-block fence into your `RESOLVER.md` /
@@ -26,7 +25,16 @@ repo. You scaffold once, you own them, you fork and edit freely. When
gbrain ships a new version, you ask "what changed?" — the agent reads
the diff and decides what (if anything) to integrate.
## The five commands
## The core workflow commands
The five commands below are the scaffold-and-own workflow. The full
`gbrain skillpack` surface is larger (`list`, `diff`, `check`, `search`,
`info`, `registry`, `doctor`, `init`, `pack`, `endorse`, …) — run
`gbrain skillpack --help` for the always-current list. One worth calling
out here: **`gbrain skillpack init-brain-pack <name>`** scaffolds a
*brain-resident* pack inside a brain/source repo (`brain_resident: true`
plus a machine-parseable README) that connecting harnesses discover on
`gbrain sources add`.
### `gbrain skillpack scaffold <name> [--workspace PATH]`
@@ -77,15 +85,15 @@ gbrain skillpack reference book-mirror
`reference <name> --apply-clean-hunks` is the auto-apply path. It
parses the diff between gbrain's bundle and your local copy, applies
every hunk whose pre-change context matches uniquely. **Two-way merge
limitation**: without scaffold-time base tracking (intentionally
out-of-scope for v0.33), this cannot distinguish "gbrain changed X"
limitation**: without scaffold-time base tracking (intentionally out of
scope), this cannot distinguish "gbrain changed X"
from "you changed X." Applied hunks align everything to gbrain. Use
`--dry-run` first to preview, or run plain `reference` to inspect the
diff before letting auto-apply touch anything.
### `gbrain skillpack migrate-fence [--workspace PATH] [--dry-run]`
One-shot conversion for workspaces on the pre-v0.33 managed-block
One-shot conversion for workspaces still on the legacy managed-block
model. Strips the `<!-- gbrain:skillpack:begin -->` / `end -->`
markers and the manifest receipt comment from your resolver file.
@@ -157,7 +165,7 @@ Your agent's job at runtime is to walk `skills/*/SKILL.md`, parse the
frontmatter, and match the user's intent against every skill's
`triggers:` array. When a match scores high enough, invoke that skill.
This replaces the v0.32 model where `gbrain skillpack install` wrote
This replaces the legacy model where `gbrain skillpack install` wrote
table rows into your `RESOLVER.md`. Rows are gone (or, for users
migrating from the old model, preserved transitionally by
`migrate-fence` until they run `scrub-legacy-fence-rows`).
@@ -173,7 +181,7 @@ If you're a downstream agent author updating to this model:
## Removing a scaffolded skill
There's no `gbrain skillpack uninstall` command in v0.33. The files
There's no `gbrain skillpack uninstall` command. The files
in your `skills/<slug>/` are first-class members of your repo —
delete them like any other code:
@@ -195,16 +203,17 @@ You own the files. There's no manifest to update, no fence to rebuild.
## When to use which command (quick decision tree)
- **New host repo, want a gbrain skill** → `scaffold`
- **Shipping a pack from inside a brain/source repo** → `init-brain-pack`
- **gbrain shipped a new version, want to see what's changed**
→ `reference` (read-only) or `reference --apply-clean-hunks` (auto)
- **Upgrading from v0.32 or earlier** → `migrate-fence` (one-shot)
- **Upgrading from the legacy managed-block model** → `migrate-fence` (one-shot)
- **Cleanup after `migrate-fence`** → `scrub-legacy-fence-rows`
- **Lift your fork's skill back into gbrain** → `harvest` + the
`skillpack-harvest` editorial skill
## What about `install` and `uninstall`?
Both are removed in v0.33. Running either prints an error pointing at
the replacement command. No deprecated alias — this is a clean break.
Both are removed. Running either prints an error pointing at the
replacement command. No deprecated alias — this is a clean break.
If you have existing scripts referencing the old names, update them
once and move on.
docs/guides/source-attribution.md
+13 -5
View File
@@ -4,7 +4,7 @@
Every fact in the brain traces to where it came from -- who said it, in what context, and when.
## What the User Gets
Without this: six months from now, someone reads a brain page and has no idea if "Pedro co-founded Brex" came from Pedro himself, a LinkedIn scrape, or a hallucination. With this: every claim is auditable, conflicts are surfaced, and the brain is a court-admissible record of reality.
Without this: six months from now, someone reads a brain page and has no idea if "Alice co-founded widget-co" came from Alice herself, a LinkedIn scrape, or a hallucination. With this: every claim is auditable, conflicts are surfaced, and the brain is a court-admissible record of reality.
## Implementation
@@ -23,11 +23,11 @@ on brain_write(page, fact):
# [Source: Crustdata LinkedIn enrichment, 2026-04-07 12:35 PM PT]
elif source.type == "social_media":
# MUST include full URL -- not just @handle
# [Source: X/@pedroh96 tweet, product launch, 2026-04-07](https://x.com/pedroh96/status/...)
# [Source: X/@alice_example tweet, product launch, 2026-04-07](https://x.com/alice_example/status/...)
elif source.type == "email":
# [Source: email from Sarah Chen re Q2 board deck, 2026-04-05 2:30 PM PT]
# [Source: email from Alice Example re Q2 board deck, 2026-04-05 2:30 PM PT]
elif source.type == "workspace":
# [Source: Slack #engineering, Keith re deploy schedule, 2026-04-06 11:45 AM PT]
# [Source: Slack #engineering, Charlie re deploy schedule, 2026-04-06 11:45 AM PT]
elif source.type == "web":
# [Source: Happenstance research, 2026-04-07 12:35 PM PT]
elif source.type == "published":
@@ -57,7 +57,7 @@ SOURCE_PRIORITY = [
## Tricky Spots
1. **Compiled truth is NOT exempt from citations.** "Pedro co-founded Brex" in the synthesis section needs `[Source: ...]` just as much as a timeline entry does. Most agents skip citations above the bar.
1. **Compiled truth is NOT exempt from citations.** "Alice co-founded widget-co" in the synthesis section needs `[Source: ...]` just as much as a timeline entry does. Most agents skip citations above the bar.
2. **Tweet URLs are mandatory.** `[Source: X/@handle tweet, topic, date]` without a URL is a broken citation. Hundreds of brain pages end up with unreachable tweet references when the URL is omitted. Always: `[Source: X/@handle tweet, topic, date](https://x.com/handle/status/ID)`.
3. **"User said it" isn't enough.** WHERE, ABOUT WHAT, WHEN. `[Source: User, direct message, 2026-04-07 12:33 PM PT]` -- not just `[Source: User]`.
4. **Don't silently resolve conflicts.** When the user says one thing and an API says another, note the contradiction in compiled truth with both citations. Let the reader decide.
@@ -71,5 +71,13 @@ SOURCE_PRIORITY = [
4. Check timeline entries on 3 random pages. Each entry should have a source citation with date and context.
5. Look for a page where the user stated something that contradicts an API result. Confirm the contradiction is noted, not silently resolved.
## Related
- `skills/_brain-filing-rules.md` — the canonical citation-format rules every
brain write follows (this guide is the narrative walkthrough of the same
contract).
- `skills/citation-fixer/SKILL.md` — audits and repairs existing pages
against that format.
---
*Part of the [GBrain Skillpack](../GBRAIN_SKILLPACK.md).*
docs/guides/sub-agent-routing.md
+50 -23
View File
@@ -2,19 +2,42 @@
## Goal
Route sub-agents to the cheapest model that can do the job, saving 10-40x on costs without sacrificing quality.
Route sub-agents to the cheapest model that can do the job, saving large
multiples on cost without sacrificing quality.
## What the User Gets
Without this: every sub-agent runs on Opus ($15/MTok). Entity detection on
every message costs $3-5/day. Research tasks cost $10+ each.
Without this: every sub-agent runs on your most expensive model. Entity
detection fires on every message at top-tier rates; research tasks cost
several dollars each.
With this: entity detection runs on Sonnet ($3/MTok, 5x cheaper). Research
runs on DeepSeek ($0.50/MTok, 30x cheaper). Main session stays on Opus for
quality. Total cost drops 70-80%.
With this: entity detection runs on a cheap fast model, research execution
runs on a budget model, and only planning/synthesis touch the expensive
model. Total cost drops 70-80%.
(Illustrative input-token anchors from gbrain's canonical pricing table,
`src/core/model-pricing.ts`: Opus-class $5/MTok, Sonnet-class $3/MTok,
Haiku-class $1/MTok. Budget providers run well under $1/MTok. Prices
drift — the pricing table is the source of truth, not this doc.)
## Implementation
### GBrain's native mechanism: model tiers
Before hardcoding vendors, use gbrain's tier routing. Every gbrain
subagent/LLM call resolves through a named tier
(`utility` / `reasoning` / `deep` / `subagent`), and you point each tier
at whatever model you want once:
```bash
gbrain config set models.tier.subagent anthropic:claude-haiku-4-5
gbrain config set models.tier.deep anthropic:claude-opus-4-7
```
Per-call override: `gbrain agent run --model <provider:model>`. The
conventions file `skills/conventions/model-routing.md` is the canonical
routing policy; this guide is the cost rationale behind it.
### Routing Table
| Task Type | Recommended Model | Why |
@@ -44,14 +67,16 @@ on_every_message(text):
3. FACTS: New info about existing entities -> update timeline
4. CITATIONS: Every fact needs [Source: ...] attribution
5. Sync changes to brain repo`,
model: "sonnet-class", // fast + cheap
model: "sonnet-class", // fast + cheap; haiku-class is cheaper still
timeout: 120s
})
```
**Why Sonnet-class for detection:** Entity detection is pattern matching, not
deep reasoning. Sonnet is 5-10x cheaper than Opus and fast enough for async
detection. The main session continues on Opus while detection runs in parallel.
**Why a cheaper class for detection:** Entity detection is pattern matching,
not deep reasoning. Sonnet-class runs at a fraction of Opus-class cost, and
Haiku-class at a fraction of that — both fast enough for async detection.
The main session continues on your best model while detection runs in
parallel.
### Research Pipeline Pattern
@@ -64,9 +89,9 @@ For research-heavy tasks, use a multi-model pipeline:
```
**Why this works:** The planning and synthesis steps need taste and judgment
(Opus). The execution step is mechanical data gathering (DeepSeek at 25-40x
lower cost). You get Opus-quality output at DeepSeek-level cost for 80% of
the work.
(Opus-class). The execution step is mechanical data gathering (a budget
model at a small fraction of the cost). You get top-tier output at
budget-model cost for 80% of the work.
### When to Spawn Sub-Agents
@@ -82,15 +107,16 @@ the work.
The main session runs on your best model. Everything else runs on the
cheapest model that can do the job. In practice, 60-70% of sub-agent
work is entity detection (Sonnet) and research execution (DeepSeek),
which are 10-40x cheaper than the main session model.
work is entity detection and research execution, which run at a small
fraction of the main session model's cost.
## Tricky Spots
1. **Sonnet, not Opus, for detection.** The most common mistake is running
entity detection on Opus. Detection is pattern matching, not deep reasoning.
Sonnet is 5-10x cheaper and fast enough. Reserve Opus for the main session
where reasoning quality matters.
1. **A cheap class, not Opus, for detection.** The most common mistake is
running entity detection on Opus-class. Detection is pattern matching, not
deep reasoning. Sonnet- or Haiku-class is several times cheaper and fast
enough. Reserve Opus-class for the main session where reasoning quality
matters.
2. **Don't block the main thread.** Sub-agents must run asynchronously. If the
signal detector runs synchronously, the user waits 30-120 seconds for every
@@ -98,10 +124,11 @@ which are 10-40x cheaper than the main session model.
a response immediately.
3. **Cost optimization is multiplicative.** Entity detection runs on every
single message. If you use Opus at $15/MTok for detection across 50
messages/day, that's $3-5/day just for detection. Sonnet at $3/MTok brings
that to $0.60-1.00/day. Over a month, the wrong model choice costs $100+
more than necessary.
single message, so the per-call price difference compounds across 50+
messages/day. Routing detection from Opus-class ($5/MTok in) to
Haiku-class ($1/MTok in) is a flat 5x cut on your highest-frequency LLM
call — over a month, the wrong model choice for detection alone costs
real money. (Current per-model rates: `src/core/model-pricing.ts`.)
## How to Verify
docs/guides/upgrades-auto-update.md
+18 -7
View File
@@ -16,9 +16,9 @@ benefit-focused bullets, waits for explicit permission, then runs the full
upgrade flow including re-reading skills, running migrations, and syncing
schema. The user gets new capabilities automatically.
## Self-upgrade modes (v0.42)
## Self-upgrade modes
gbrain now stays current the way gstack does: it rides invocation frequency. A
gbrain stays current the way gstack does: it rides invocation frequency. A
throttled, cache-read-only check runs at the start of every `gbrain` invocation
(CLI and MCP) and emits an `UPGRADE_AVAILABLE <old> <new>` marker on stderr. No
host cron required — every agent kind (Claude Code, Codex, OpenClaw, Hermes, the
@@ -44,6 +44,14 @@ execution. The trust model is TLS + GitHub (same as `gbrain upgrade`);
signature verification is a tracked follow-up. Apply manually any time with
`gbrain self-upgrade`.
The `auto` quiet-hours window is configured via the
`self_upgrade.quiet_hours` config key
(`gbrain config set self_upgrade.quiet_hours '{"start":23,"end":8,"tz":"US/Pacific"}'`).
The quiet-hours *pattern* itself — gating any notification or background
action on the user's local sleep window — is owned by
[quiet-hours.md](quiet-hours.md); this doc only covers the self-upgrade
hook into it.
## Implementation
### The Check (cron-initiated)
@@ -163,15 +171,18 @@ Also persist in `~/.gbrain/update-state.json` so it survives agent context reset
If you loaded this SKILLPACK directly (copied or read from GitHub) without
installing gbrain, you can still stay current. Both GBRAIN_SKILLPACK.md and
GBRAIN_RECOMMENDED_SCHEMA.md have version markers:
GBRAIN_RECOMMENDED_SCHEMA.md carry a `<!-- source: ... -->` header pointing
at their canonical copies, and GBRAIN_RECOMMENDED_SCHEMA.md also carries a
version marker:
```bash
curl -s https://raw.githubusercontent.com/garrytan/gbrain/master/docs/GBRAIN_SKILLPACK.md | head -1
# Returns: <!-- skillpack-version: X.Y.Z -->
curl -s https://raw.githubusercontent.com/garrytan/gbrain/master/docs/GBRAIN_RECOMMENDED_SCHEMA.md | head -1
# Returns: <!-- schema-version: X.Y.Z -->
```
If the remote version is newer, fetch the full file and replace your local
copy. Set up a weekly cron to check automatically.
If the remote version is newer (or the remote SKILLPACK content differs from
your local copy), fetch the full file and replace your local copy. Set up a
weekly cron to check automatically.
## Tricky Spots
docs/mcp/ALTERNATIVES.md
+9 -7
View File
@@ -2,8 +2,9 @@
GBrain's MCP server runs via `gbrain serve` (stdio transport). To make it
accessible from other devices and AI clients, run `gbrain serve --http`
(built-in HTTP transport with bearer auth, Postgres-only ... see
[DEPLOY.md](DEPLOY.md)) behind a public tunnel. Here are your tunnel options.
(built-in HTTP transport with OAuth 2.1 + bearer auth, works on both PGLite
and Postgres brains — see [DEPLOY.md](DEPLOY.md)) behind a public tunnel.
Here are your tunnel options.
## ngrok (recommended)
@@ -58,10 +59,11 @@ Both run Bun natively. No bundling, no Deno, no cold start, no timeout limits.
| Works when laptop is off | No | No | Yes |
| Cold start | None | None | None |
| Timeout limits | None | None | None |
| All 30 operations | Yes | Yes | Yes |
| Full remote operation surface (100+ ops, minus `localOnly`) | Yes | Yes | Yes |
| Setup time | 5 min | 10 min | 15 min |
**Note:** `gbrain serve --http` is the built-in HTTP transport (v0.22.7+). Bearer auth
against the `access_tokens` table, default-deny CORS, two-bucket rate limit, body cap,
per-request audit log. Postgres-only by design (PGLite is local-only). See
[DEPLOY.md](DEPLOY.md) and [SECURITY.md](../../SECURITY.md) for env vars and tunables.
**Note:** `gbrain serve --http` is the built-in HTTP transport. OAuth 2.1 plus
bearer auth against the `access_tokens` table, default-deny CORS, two-bucket rate
limit, body cap, per-request audit log. Works on both PGLite and Postgres brains.
See [DEPLOY.md](DEPLOY.md) and [SECURITY.md](../../SECURITY.md) for env vars and
tunables.
docs/mcp/CHATGPT.md
+28 -34
View File
@@ -1,35 +1,39 @@
# Connect GBrain to ChatGPT
**Status (v0.26.0):** Unblocked. GBrain's `gbrain serve --http` ships OAuth 2.1
with PKCE, which is the ChatGPT MCP connector's hard requirement. Before v1.0,
this was a P0 TODO — the only major AI client that could not connect.
ChatGPT's MCP connector requires OAuth 2.1 with PKCE — it does not support
bearer-token MCP servers. GBrain's `gbrain serve --http` speaks exactly that,
so ChatGPT connects natively.
ChatGPT does not support bearer-token MCP servers. You must use the OAuth 2.1
HTTP server.
This page covers only the ChatGPT-specific parts. The full server setup —
starting `gbrain serve --http`, the admin bootstrap token, the `/admin`
dashboard, tunnels, and `--bind` / `--public-url` — lives in
[DEPLOY.md](DEPLOY.md). Do steps 1 (start the server) and 3 (expose it)
from there, then come back for the ChatGPT client.
## Setup
### 1. Start the HTTP server
### 1. Start and expose the server (DEPLOY.md steps 1 + 3)
```bash
gbrain serve --http --port 3131
```
Save the admin bootstrap token printed on stderr. Open
`http://localhost:3131/admin` and paste it to access the dashboard.
Follow [DEPLOY.md — OAuth 2.1 Setup](DEPLOY.md#oauth-21-setup) to start
`gbrain serve --http`, save the admin bootstrap token, and expose the server
at a public HTTPS URL (e.g. `https://your-brain.ngrok.app`). ChatGPT's
connector auto-discovers the spec-compliant endpoint at
`/.well-known/oauth-authorization-server`.
### 2. Register a ChatGPT client
ChatGPT uses the authorization code flow with PKCE (browser-based OAuth).
Register from the `/admin` dashboard:
The ChatGPT-specific delta: ChatGPT uses the **authorization code flow with
PKCE** (browser-based OAuth), so the client needs the `authorization_code`
grant type and a redirect URI. Register from the `/admin` dashboard:
1. Click **Register client**.
2. Name: `chatgpt`.
3. Grant type: `authorization_code`.
4. Scopes: `read`, `write` (leave `admin` unchecked for ChatGPT).
5. Redirect URI: ChatGPT's OAuth redirect (copy it from the ChatGPT
connector setup screen — something like
`https://chat.openai.com/connector_platform_oauth_redirect`).
5. Redirect URI: ChatGPT's OAuth redirect — **always copy the exact value
from the ChatGPT connector setup screen** (it looks like
`https://chatgpt.com/connector_platform_oauth_redirect`, but the domain
has changed before; trust the setup screen, not this doc).
6. Hit **Register**. The credential-reveal modal shows the `client_id` once
with Copy and Download JSON buttons. There is no client secret for
PKCE-based public clients.
@@ -41,22 +45,11 @@ await oauthProvider.registerClientManual(
'chatgpt',
['authorization_code'],
'read write',
['https://chat.openai.com/connector_platform_oauth_redirect'],
['<ChatGPT redirect URI from the connector setup screen>'],
);
```
### 3. Expose the server publicly
```bash
brew install ngrok
ngrok http 3131 --url your-brain.ngrok.app
```
Your OAuth issuer URL becomes `https://your-brain.ngrok.app`. ChatGPT's
connector auto-discovers the spec-compliant endpoint at
`/.well-known/oauth-authorization-server`.
### 4. Add the connector in ChatGPT
### 3. Add the connector in ChatGPT
1. Open ChatGPT > Settings > Connectors.
2. Click **Add connector**.
@@ -71,10 +64,11 @@ calls show up in the admin dashboard's live SSE feed in real time.
## Scopes
ChatGPT clients can request any combination of `read`, `write`, `admin`. The
scopes granted at consent time are enforced on every tool call. Four
operations are `localOnly` and rejected over HTTP regardless of scope:
`sync_brain`, `file_upload`, `file_list`, `file_url`. The HTTP server fails
closed for any attempt to reach local filesystem surface area.
scopes granted at consent time are enforced on every tool call. Operations
flagged `localOnly: true` in `src/core/operations.ts` (10 today — `sync_brain`
and the `file_*` ops among them) are rejected over HTTP regardless of scope.
The HTTP server fails closed for any attempt to reach local filesystem
surface area.
Recommended ChatGPT scope: `read write`. Leave `admin` for your local CLI
and the admin dashboard.
docs/mcp/CLAUDE_CODE.md
+4 -5
View File
@@ -81,11 +81,10 @@ search for [any topic in your brain]
You should see results from your GBrain knowledge base.
> **`list_skills` returns nothing?** Skill discovery is gated by `mcp.publish_skills`
> on the host. New brains from `gbrain init` default it ON; brains upgraded from an
> older release stay OFF until you opt in. Enable it on the host with
> `gbrain config set mcp.publish_skills true`. The core tools (search, query,
> get_page, put_page, think, find_experts) work regardless. Note: `capture` is a
> CLI-only command, not an MCP tool — the agent writes over MCP with `put_page`.
> on the host — enable it with `gbrain config set mcp.publish_skills true`. The core
> tools (search, query, get_page, put_page, think, find_experts) work regardless;
> `capture` is CLI-only, so agents write over MCP with `put_page`. Why brains differ
> on the default: [tutorial A1](../tutorials/connect-coding-agent.md#a1-on-the-host-serve-over-http).
## Remove
docs/mcp/CLAUDE_COWORK.md
+13
View File
@@ -27,6 +27,19 @@ Desktop bridges local MCP servers into Cowork via its SDK layer.
This means: if `gbrain serve` is running and configured in Claude Desktop,
you don't need a separate server for Cowork.
## Verify
In a Cowork session, try:
```
Call get_brain_identity, then search my brain for [any topic]
```
You should get pages from your brain back. If `list_skills` returns nothing,
skill publishing is off on the host — enable it with
`gbrain config set mcp.publish_skills true` (see
[CLAUDE_CODE.md](CLAUDE_CODE.md) for the full gotcha).
## Which to use?
- **Remote server:** works even when your laptop is closed, available to all org members
docs/mcp/CLAUDE_DESKTOP.md
+5
View File
@@ -1,5 +1,10 @@
# Connect GBrain to Claude Desktop
This page covers connecting Claude Desktop to a **remote** brain. For a brain
on the same machine as Claude Desktop, a local stdio entry in
`claude_desktop_config.json` with `"command": "gbrain", "args": ["serve"]`
works too — but only against a full local install, never a thin-client one.
**Important:** Claude Desktop does NOT connect to remote MCP servers via
`claude_desktop_config.json`. That file only works for local stdio servers.
Remote HTTP servers must be added through the GUI.
docs/mcp/CODEX.md
+8 -8
View File
@@ -9,9 +9,9 @@
> durable body — not just a connection? That's `gbrain bootstrap`: see the paste
> block in the README and [docs/guides/bootstrap.md](../guides/bootstrap.md).
Codex CLI (`@openai/codex`, v0.130+) supports remote streamable-HTTP MCP servers
with a bearer token read from an environment variable. The token lives in your
shell env, not in Codex's config file.
Recent versions of the Codex CLI (`@openai/codex`) support remote
streamable-HTTP MCP servers with a bearer token read from an environment
variable. The token lives in your shell env, not in Codex's config file.
## Fastest path: `gbrain connect`
@@ -55,11 +55,11 @@ Call get_brain_identity, then search my brain for [topic].
`get_brain_identity` confirms whose brain you're connected to; `list_skills` shows
everything it can do.
> **`list_skills` empty?** It's gated by `mcp.publish_skills` on the host (default
> ON for `gbrain init` brains, OFF for brains upgraded from older releases). Enable
> it on the host: `gbrain config set mcp.publish_skills true`. The core tools
> (search, query, get_page, put_page, think, find_experts) work regardless.
> `capture` is CLI-only, not an MCP tool — write over MCP with `put_page`.
> **`list_skills` empty?** It's gated by `mcp.publish_skills` on the host — enable
> it with `gbrain config set mcp.publish_skills true`. The core tools (search,
> query, get_page, put_page, think, find_experts) work regardless; `capture` is
> CLI-only, so write over MCP with `put_page`. Why brains differ on the default:
> [tutorial A1](../tutorials/connect-coding-agent.md#a1-on-the-host-serve-over-http).
## Remove
docs/mcp/DEPLOY.md
+56 -47
View File
@@ -1,17 +1,16 @@
# Deploy GBrain Remote MCP Server
> **v0.26.0+:** `gbrain serve --http` ships full OAuth 2.1 (client credentials,
> auth code + PKCE, refresh rotation, optional DCR), an embedded React admin
> dashboard at `/admin`, scoped operations, and a live SSE activity feed.
> Pre-v0.26 legacy bearer tokens still work — `verifyAccessToken` falls back
> to the `access_tokens` table and grandfathers tokens to `read+write+admin`.
> Postgres-only for the legacy fallback (the `access_tokens` table is Postgres-only);
> OAuth tables work on both PGLite and Postgres. See [SECURITY.md](../../SECURITY.md)
> for env vars and tunable defaults.
> `gbrain serve --http` ships full OAuth 2.1 (client credentials, auth code +
> PKCE, refresh rotation, optional DCR), an embedded React admin dashboard at
> `/admin`, scoped operations, and a live SSE activity feed. Legacy bearer
> tokens still work — `verifyAccessToken` falls back to the `access_tokens`
> table and grandfathers tokens to `read+write+admin`. Both the OAuth surface
> and the bearer fallback work on both engines (PGLite and Postgres). See
> [SECURITY.md](../../SECURITY.md) for env vars and tunable defaults.
Access your brain from any device, any AI client. GBrain ships two transports:
`gbrain serve` (stdio) for local agents, and `gbrain serve --http` (v0.26.0+)
for remote clients over OAuth 2.1.
`gbrain serve` (stdio) for local agents, and `gbrain serve --http` for remote
clients over OAuth 2.1.
## Three Paths
@@ -24,7 +23,7 @@ gbrain serve
Works with Claude Code, Cursor, Windsurf, and any MCP client that supports stdio.
No server, no tunnel, no token needed. Works on both PGLite and Postgres engines.
### Remote over OAuth 2.1 (recommended, v0.26.0+)
### Remote over OAuth 2.1 (recommended)
```bash
gbrain serve --http --port 3131
@@ -45,28 +44,27 @@ Supported clients:
- **Perplexity** — OAuth 2.1 client credentials grant.
- **Claude Code, Cursor, Windsurf** — can use OAuth or legacy bearer.
See the [OAuth 2.1 setup](#oauth-21-setup-v100) section below.
See the [OAuth 2.1 setup](#oauth-21-setup) section below.
### Remote with legacy bearer tokens (pre-v0.26 deployments) — Postgres only
### Remote with legacy bearer tokens (simplest)
```
Your AI client (Claude Desktop, Perplexity, etc.)
→ ngrok tunnel (https://YOUR-DOMAIN.ngrok.app)
→ gbrain serve --http (built-in transport with bearer auth)
→ Postgres (pooler connection or self-hosted)
→ Postgres or PGLite
```
This requires:
1. A Postgres-backed brain (the `access_tokens` table only exists on Postgres;
running `gbrain serve --http` against a PGLite install fails fast at startup)
2. A machine running `gbrain serve --http`
3. A public tunnel (ngrok, Tailscale, or cloud host)
4. A bearer token created via `gbrain auth create <name>`
1. A machine running `gbrain serve --http` (works on both PGLite and Postgres
brains)
2. A public tunnel (ngrok, Tailscale, or cloud host)
3. A bearer token created via `gbrain auth create <name>`
Pre-v1.0 tokens are grandfathered as `read+write+admin` scopes when you upgrade
to the HTTP server, so no migration is required.
Existing bearer tokens are grandfathered as `read+write+admin` scopes on the
OAuth-capable HTTP server, so no migration is required.
## OAuth 2.1 Setup (v0.26.0+)
## OAuth 2.1 Setup
### 1. Start the HTTP server
@@ -92,8 +90,8 @@ Save this token. Open `http://localhost:3131/admin` and paste it to access the
dashboard. The dashboard shows live activity, registered clients, request logs,
and per-client config export.
> **v0.26.9+:** `mcp_request_log.params` and the live SSE activity feed default
> to a redacted summary `{redacted, kind, declared_keys, unknown_key_count, approx_bytes}`.
> `mcp_request_log.params` and the live SSE activity feed default to a redacted
> summary `{redacted, kind, declared_keys, unknown_key_count, approx_bytes}`.
> Declared param keys are kept (intersected against the operation's spec); unknown
> keys are counted but never named, and byte sizes round up to 1KB so size-probe
> attacks can't binary-search secret content. Operators on a personal laptop who
@@ -124,9 +122,9 @@ gbrain auth register-client perplexity \
--scopes "read write"
```
**v0.34 — source-scoped clients.** Multi-source brains can scope a client's
write authority to one source and its read scope to a curated set with the
new `--source` and `--federated-read` flags:
**Source-scoped clients.** Multi-source brains can scope a client's write
authority to one source and its read scope to a curated set with the
`--source` and `--federated-read` flags:
```bash
gbrain auth register-client dept-x-agent \
@@ -138,9 +136,12 @@ gbrain auth register-client dept-x-agent \
`--source` controls the write authority — `put_page` / `add_link` / etc only
land in `dept-x`. `--federated-read` controls the read axis independently;
queries return rows from any of the listed sources. Omit both flags for the
v0.33-compatible super-client shape. Pre-v0.34 clients are backfilled to
`source_id='default'` on `gbrain upgrade`.
queries return rows from any of the listed sources. Omit both flags for an
unscoped super-client. Clients registered before source scoping existed are
backfilled to `source_id='default'` on `gbrain upgrade`. Within a source,
slug-level write fencing is also available: `--bound-slug-prefixes p1/,p2/`
rejects slug-mutating writes outside the listed prefixes (update later with
`gbrain auth rescope-client <id> --bound-slug-prefixes <p1,p2|none>`).
Host-repo wrappers can register programmatically:
@@ -158,7 +159,7 @@ start the server with `--enable-dcr`. DCR is off by default.
### 3. Expose the server
**v0.34 — bind explicitly.** `gbrain serve --http` defaults to `127.0.0.1`.
**Bind explicitly.** `gbrain serve --http` defaults to `127.0.0.1`.
To accept connections from the ngrok tunnel (or any non-loopback source),
restart with `--bind`:
@@ -182,10 +183,10 @@ router exposes the spec-compliant discovery endpoint at
### 4. Scopes and localOnly
Every operation is tagged `read | write | admin`. Four operations are
`localOnly` and rejected over HTTP regardless of scope: `sync_brain`,
`file_upload`, `file_list`, `file_url`. Remote agents cannot reach local
filesystem surface area.
Every operation is tagged `read | write | admin`. Operations flagged
`localOnly: true` in `src/core/operations.ts` (10 today — `sync_brain` and
the `file_*` ops among them) are rejected over HTTP regardless of scope.
Remote agents cannot reach local filesystem surface area.
| Scope | What it allows |
|-------|---------------|
@@ -193,10 +194,13 @@ filesystem surface area.
| `write` | `put_page`, `delete_page`, `add_link`, `add_timeline_entry` |
| `admin` | Client management, token revocation, sweep, local-only ops |
Write ops can additionally be fenced per client with `--bound-slug-prefixes`
(see [Register OAuth clients](#2-register-oauth-clients) above).
## Legacy Bearer Token Setup
Keep using pre-v0.26 bearer tokens if you aren't ready to migrate. They
grandfather to `read+write+admin` scopes on the HTTP server.
Bearer tokens are the simple path when you don't need per-client scoping.
They grandfather to `read+write+admin` scopes on the HTTP server.
### 1. Set up the tunnel
@@ -243,15 +247,20 @@ gbrain auth test \
## Operations
All 30 GBrain operations are available remotely, including `sync_brain` and
`file_upload` (no timeout limits with self-hosted server).
GBrain's full operation catalog (100+ operations in `src/core/operations.ts`)
is available remotely, with no timeout limits on a self-hosted server. The
only exceptions are the operations flagged `localOnly: true` — `sync_brain`
and the `file_*` ops among them — which are rejected over HTTP regardless of
scope (see [Scopes and localOnly](#4-scopes-and-localonly) above).
**Security note on `file_upload`:** remote MCP callers are confined to the working
directory where `gbrain serve` was launched. Symlinks, `..` traversal, and absolute
paths outside cwd are rejected. Page slugs and filenames are allowlist-validated
(alphanumeric + hyphens; no control chars, RTL overrides, or backslashes). Local
CLI callers (`gbrain files upload ...`) keep unrestricted filesystem access since
the user owns the machine.
**Security note on file access:** the `file_*` operations being localOnly is
the first line of defense; as defense-in-depth, `file_upload` also confines
any caller that isn't verifiably the trusted local CLI to the working
directory where `gbrain serve` was launched. Symlinks, `..` traversal, and
absolute paths outside cwd are rejected, and page slugs and filenames are
allowlist-validated (alphanumeric + hyphens; no control chars, RTL overrides,
or backslashes). Local CLI callers (`gbrain files upload ...`) keep
unrestricted filesystem access since the user owns the machine.
## Deployment Options
@@ -321,8 +330,8 @@ Remote servers must be added via Settings > Integrations, NOT
| put_page | 100-500ms | Write + trigger search_vector update |
| get_stats | < 100ms | Aggregate query |
**Note:** `gbrain serve --http` shipped in v0.26.0 with OAuth 2.1 + admin
dashboard baked into the binary. The custom HTTP wrapper pattern (see
**Note:** `gbrain serve --http` has OAuth 2.1 + the admin dashboard baked
into the binary. The custom HTTP wrapper pattern (see
[voice recipe](../../recipes/twilio-voice-brain.md)) is still supported for
teams that need bespoke middleware, but for most remote deployments the
built-in server is the recommended path.
docs/mcp/PERPLEXITY.md
+11 -5
View File
@@ -18,13 +18,16 @@ gbrain serve --http --port 3131 --bind 0.0.0.0 \
--public-url https://YOUR-DOMAIN.ngrok.app
```
- **`--bind 0.0.0.0` is required.** Since v0.34, `--http` defaults to
`127.0.0.1`, so without it the tunnel reaches the server but the connection is
refused (`ECONNREFUSED`).
- **`--bind 0.0.0.0` is required.** `--http` defaults to `127.0.0.1`, so
without it the tunnel reaches the server but the connection is refused
(`ECONNREFUSED`).
- **`--public-url` must match the tunnel.** The OAuth issuer in the discovery
metadata has to line up with the URL Perplexity actually hits (RFC 8414 §3.3),
or OAuth client-credentials auth fails.
Full detail on both flags (and the rest of the server setup) lives in
[DEPLOY.md — Expose the server](DEPLOY.md#3-expose-the-server).
## 2. Expose it with a tunnel
```bash
@@ -36,9 +39,12 @@ tunnel.
## 3. Create credentials
Two supported auth paths.
Two supported auth paths. (Full client-registration mechanics — the `/admin`
dashboard flow, grant types, scope format — live in
[DEPLOY.md — Register OAuth clients](DEPLOY.md#2-register-oauth-clients);
below is the Perplexity-specific shape.)
**OAuth 2.1 client credentials (recommended, v0.26.0+).** Perplexity is a cloud
**OAuth 2.1 client credentials (recommended).** Perplexity is a cloud
service, so it holds whatever credential you give it. OAuth is the correct choice:
least-privilege scopes + short-lived rotating access tokens instead of a
long-lived full-access secret. Mint a client and print the connector fields in
docs/operations/headless-install.md
+20 -6
View File
@@ -1,8 +1,8 @@
# Headless install: Docker, CI, postinstall
As of v0.37, `gbrain init --pglite` in a non-TTY context (Docker `RUN`, CI step, postinstall hook) exits 1 when no embedding-provider API key is present in the environment. This is a deliberate fail-loud — the alternative was the v0.36 silent-broken-state class where init succeeded with a default that didn't match any real key.
`gbrain init --pglite` in a non-TTY context (Docker `RUN`, CI step, postinstall hook) exits 1 when no embedding-provider API key is present in the environment. This is a deliberate fail-loud — the alternative is a silent-broken state where init succeeds with a default that doesn't match any real key.
Two patterns work for headless installs. Pick whichever fits your image lifecycle.
Three patterns work for headless installs. Pick whichever fits your image lifecycle.
## Pattern 1: Provider key available at image build time
@@ -16,7 +16,7 @@ FROM oven/bun:1 AS builder
ARG OPENAI_API_KEY
ENV OPENAI_API_KEY=$OPENAI_API_KEY
RUN bun install -g github:garrytan/gbrain
RUN bun install -g github:garrytan/gbrain#latest-stable
RUN gbrain init --pglite # auto-picks OpenAI, persists config
```
@@ -26,7 +26,7 @@ RUN gbrain init --pglite # auto-picks OpenAI, persists config
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: |
bun install -g github:garrytan/gbrain
bun install -g github:garrytan/gbrain#latest-stable
gbrain init --pglite
```
@@ -38,7 +38,7 @@ If the API key is a runtime secret (Kubernetes secret, runtime env injection, en
```dockerfile
FROM oven/bun:1
RUN bun install -g github:garrytan/gbrain
RUN bun install -g github:garrytan/gbrain#latest-stable
# Build the brain shape without a provider — schema lands at the default
# width, but no embed callsite will actually run until runtime config.
@@ -59,6 +59,20 @@ The runtime `gbrain init --force` re-runs the init flow against the now-populate
- Resolves the provider via env detection.
- Re-templates the PGLite schema if dim differs from the build-time default.
## Pattern 3: No key, ever (keyless mode)
`--no-embedding` isn't only a deferral — it's also the install shape for **keyless mode**, a first-class supported end state (not a broken one). With zero provider keys, gbrain runs keyword-only (BM25) search and takes memory from agent-authored `## Facts` fences and write ops; embedding and extraction paths refuse cleanly instead of failing silently.
```dockerfile
FROM oven/bun:1
RUN bun install -g github:garrytan/gbrain#latest-stable
RUN gbrain init --pglite --no-embedding # keyless install — done; no runtime re-init needed
```
`gbrain bootstrap verify` (and the agent-bootstrap flow generally) prints an honest capability report for this posture — `gbrain capabilities: keyless mode`, per-touchpoint lines, and the one-key upsell (`src/core/capability.ts`). Keyless installs for the agent-bootstrap path are covered in `docs/guides/bootstrap.md`; this doc covers the Docker/CI shape. Adding a single provider key later upgrades in place via Pattern 2's runtime `gbrain init --force`.
Since every embedding cost gate is structurally moot with no key, none of `docs/operations/spend-controls.md` applies until you add one.
## What WON'T work
```dockerfile
@@ -67,7 +81,7 @@ The runtime `gbrain init --force` re-runs the init flow against the now-populate
RUN gbrain init --pglite
```
If you upgrade from a pre-v0.37 image that used this pattern, `gbrain doctor` will surface the mismatch on first run after upgrade and print a paste-ready repair command (`gbrain init --force --embedding-model …` for empty brains, `gbrain retrieval-upgrade --reindex` for non-empty).
If an older image used this pattern, `gbrain doctor` will surface the mismatch on first run after upgrade and print a paste-ready repair command — `gbrain init --force --pglite --embedding-model <model> --embedding-dimensions <dims>` for brains with no embeddings yet, `gbrain migrate embeddings --to <model> --dim <dims>` for non-empty brains.
## Verifying a headless install
docs/operations/spend-controls.md
+6 -1
View File
@@ -8,6 +8,11 @@ The orienting idea: **GBrain itself is rounding error; the spend that matters is
downstream embedding.** These gates exist so a routine sync or enrich can't run up
an unexpected embedding bill, while never wedging an unattended cron.
**Keyless mode:** if you run with zero provider keys (`gbrain init --no-embedding`,
the keyless bootstrap posture — see `docs/guides/bootstrap.md` and
`docs/operations/headless-install.md`), nothing here can spend and none of these
gates ever fire. This doc applies once you add a key.
## `spend.posture` — one switch for "cost is not my constraint"
```bash
@@ -18,7 +23,7 @@ gbrain config set spend.posture gated # default — gates enforce
| Value | Effect |
|-------|--------|
| `gated` (default) | Every cost gate enforces its limit as documented below. |
| `tokenmax` | Every cost gate prints its estimate and **proceeds** — informational only. Spend is still recorded to the ledger; posture removes the *ceiling*, not the *accounting*. |
| `tokenmax` | Every embedding-spend gate in the table below prints its estimate and **proceeds** — informational only. Spend is still recorded to the ledger; posture removes the *ceiling*, not the *accounting*. (Commands with their own LLM cost caps outside this doc's embedding scope — e.g. `extract-conversation-facts --max-cost-usd` — don't resolve posture; their per-call flags govern.) |
`spend.posture` is deliberately separate from `search.mode=tokenmax` (which governs
retrieval payload size, not embedding spend). When a gate fires and
docs/schema-author-tutorial.md
+2 -3
View File
@@ -212,13 +212,12 @@ gbrain schema lint --with-db
**Commit your pack to source control.** If `~/.gbrain/schema-packs/mine/` is a git repo, commit `pack.json` and push. Your pack survives across machines, and the `mutation_count_anomaly` lint rule will nudge you when you hit >50 mutations in a week (the "you should be committing this" signal).
**For agents (MCP):** the same operations are reachable over HTTPS MCP via 9 new ops. Register an admin-scope OAuth client and `schema_apply_mutations` lets a remote agent compose multi-step refactors as one atomic batch. The batched MCP op + per-pack lock + audit log are the load-bearing primitives that make remote schema authoring safe. See [`skills/schema-author/SKILL.md`](../skills/schema-author/SKILL.md) for the agent dispatcher.
**For agents (MCP):** the same operations are reachable over HTTPS MCP as schema ops. Register an admin-scope OAuth client and `schema_apply_mutations` lets a remote agent compose multi-step refactors as one atomic batch. The batched MCP op + per-pack lock + audit log are the load-bearing primitives that make remote schema authoring safe. See [`skills/schema-author/SKILL.md`](../skills/schema-author/SKILL.md) for the agent dispatcher.
**Undo a mistake.** Every mutation primitive has an inverse (`remove-type`, `remove-alias`, `remove-prefix`, `remove-link-type`, `set-extractable false`, etc.). If you fork twice and want to revert, `gbrain schema downgrade` restores the previous active pack from `~/.gbrain/schema-pack-history.jsonl`.
## Related docs
- **Reference:** `gbrain schema --help` for the full 22-verb CLI surface; CLAUDE.md's "Schema Cathedral v3 (v0.40.7.0)" section for the module-by-module architecture.
- **Reference:** `gbrain schema --help` for the full CLI surface (30+ subcommands); the "Schema Cathedral v3" section of `docs/architecture/KEY_FILES.md` for the module-by-module architecture.
- **How-to:** [`skills/schema-author/SKILL.md`](../skills/schema-author/SKILL.md) — the agent dispatcher with the 7-phase workflow (brain → assess → propose → apply → sync → verify → commit).
- **Explanation:** [`skills/conventions/schema-evolution.md`](../skills/conventions/schema-evolution.md) — when to add a type vs alias vs prefix.
- **Plan + decisions:** the original design captured 21 decisions including the bundled-pack guard rationale (D6), the empty-filter fallback contract (D4), and the MCP non-localOnly trust posture (D2). Lives in `~/.claude/plans/system-instruction-you-are-working-recursive-thacker.md` (private).
docs/skillpack-anatomy.md
+15
View File
@@ -105,6 +105,21 @@ gbrain skillpack scaffold <source> # owner/repo, https, ./dir, ./*.tgz
gbrain skillpack registry --url X # point at a custom registry
```
## Brain-resident packs
A brain/source repo can carry its own publishable skillpack (`brain_resident: true`
in `skillpack.json`, plus a `schema_pack` declaration). Scaffold one with:
```bash
gbrain skillpack init-brain-pack <name> # inside the brain repo; --dry-run to preview
```
Connecting harnesses discover the pack on `gbrain sources add`, and remote
agents reach it over MCP via the source-scoped `list_brain_skillpack` op +
`get_skill --source_id` (gated by the `mcp.publish_skills` config key). The
anatomy above applies unchanged — a brain-resident pack is a normal pack that
happens to live inside a brain repo.
## See also
- `examples/skillpack-reference/` — the live 10/10 reference pack
docs/storage-tiering.md
+1 -1
View File
@@ -4,7 +4,7 @@
GBrain supports storage tiering to separate version-controlled content from bulk machine-generated data. This prevents git repositories from becoming bloated with large amounts of automatically generated content while still preserving it in the database.
> Note on naming: prior to v0.22.11 the keys were `git_tracked` / `supabase_only`. The canonical names are now `db_tracked` / `db_only` (engine-agnostic — works on both PGLite and Postgres). The deprecated keys still load with a once-per-process warning. Run `gbrain doctor --fix` for an automated rename when that path lands.
> Note on naming: prior to v0.22.11 the keys were `git_tracked` / `supabase_only`. The canonical names are now `db_tracked` / `db_only` (engine-agnostic — works on both PGLite and Postgres). The deprecated keys still load with a once-per-process warning; rename them in your config by hand to silence it.
## Configuration
docs/takes-vs-facts.md
+9 -9
View File
@@ -14,10 +14,10 @@ The epistemological layer. WHO believes WHAT, with confidence weight and time.
- **Scale:** 100K+ rows across thousands of holders in a mature brain
**Example takes:**
- `holder=people/garry-tan kind=bet` "AI will replace 50% of coding by 2030" (w=0.75)
- `holder=people/jared-friedman kind=take` "Momo has strong retention" (w=0.80)
- `holder=world kind=fact` "Clipboard raised $100M Series C" (w=1.0)
- `holder=brain kind=hunch` "Garry has a hero/rescuer pattern" (w=0.70)
- `holder=people/alice-example kind=bet` "AI will replace 50% of coding by 2030" (w=0.75)
- `holder=people/bob-example kind=take` "widget-co has strong retention" (w=0.80)
- `holder=world kind=fact` "acme-example raised a Series C" (w=1.0)
- `holder=brain kind=hunch` "alice-example has a hero/rescuer pattern" (w=0.70)
**Query surface:** `gbrain takes list`, `gbrain takes search`, `gbrain think`
@@ -32,9 +32,9 @@ Personal knowledge from the brain owner's conversations. Real-time capture.
- **Bridge:** Dream cycle `consolidate` phase promotes hot facts → cold takes nightly
**Example facts:**
- `kind=event` "I have a meeting with Brian tomorrow"
- `kind=event` "I have a meeting with alice-example tomorrow"
- `kind=preference` "I don't drink coffee"
- `kind=commitment` "We decided on nesting custody"
- `kind=commitment` "We decided to move the offsite to March"
- `kind=belief` "I think the market is overheated"
**Query surface:** `gbrain recall`, MCP `_meta.brain_hot_memory`
@@ -42,8 +42,8 @@ Personal knowledge from the brain owner's conversations. Real-time capture.
## The Category Error
**Never dump takes into the facts table.** Takes include other people's attributed
beliefs (Jared's assessment of a company, PG's view on schools, a founder's
revenue claims). These are NOT the brain owner's personal facts.
beliefs (a partner's assessment of a company, an investor's view on markets, a
founder's revenue claims). These are NOT the brain owner's personal facts.
**Never dump facts into the takes table without transformation.** Facts are
scoped to what the owner said in conversation. They become takes only through
@@ -85,7 +85,7 @@ First full takes extraction run on a ~100K-page brain:
### Key Learnings for Extraction Prompts
1. **Holder ≠ subject.** "Garry has a hero/rescuer pattern" → holder=brain, NOT people/garry-tan
1. **Holder ≠ subject.** "alice-example has a hero/rescuer pattern" → holder=brain, NOT people/alice-example
2. **Atomic claims.** Split compound claims into separate rows
3. **Amplification ≠ endorsement.** Retweet-only → max weight 0.55
4. **Self-reported ≠ verified.** "Reports 7 figures" → holder=person, weight=0.75, NOT world/1.0
docs/tutorials/README.md
+2 -1
View File
@@ -32,5 +32,6 @@ Tutorials follow the [Diataxis](https://diataxis.fr/) tutorial pattern: learning
- **Reference:** [`docs/architecture/`](../architecture/) — system design, topologies, retrieval theory
- **How-to:** [`docs/guides/`](../guides/) — task-oriented runbooks (sub-agent routing, minion deployment, skill development, brain-first lookup, idea capture, diligence ingestion). Highlight: [scaling skills past 300](../guides/scaling-skills.md) — the three-tier architecture for agents that have outgrown the always-loaded skill manifest.
- **Integrations:** [`docs/integrations/`](../integrations/) — connecting external data sources (voice, email, calendar, embedding providers)
- **MCP setup:** [`docs/mcp/`](../mcp/) — per-client setup (Claude Desktop, Code, Cursor, ChatGPT, Perplexity, Cowork)
- **MCP setup:** [`docs/mcp/`](../mcp/) — per-client setup (Claude Desktop, Claude Code, Codex, ChatGPT, Perplexity, Cowork)
- **Install paths:** [`docs/INSTALL.md`](../INSTALL.md) — every install path, end to end
- **Agent bootstrap:** [`docs/guides/bootstrap.md`](../guides/bootstrap.md) — the paste-in path that turns a coding agent into a full agent (identity, per-turn context, schedules, a private repo as its durable body)
docs/tutorials/company-brain.md
+40 -39
View File
@@ -85,7 +85,7 @@ git clone git@github.com:your-org/customers.git customers
git clone git@github.com:your-org/internal-docs.git internal
```
You can also keep the existing personal-brain repo as one of the sources. Just pick the role it plays (probably `shared` if it's already org-wide content).
You can also keep the existing personal-brain repo as one of the sources. Just pick the role it plays (probably `shared` if it's already org-wide content). When agents on the host write pages into a source, `gbrain sources push <id>` (run on the host) commits and pushes those changes back to the source's git repo, so the repo stays the durable system of record.
### Two scoping models (pick the one that matches your shape)
@@ -156,7 +156,7 @@ The personal brain talks to you through the AlphaClaw harness over Telegram. For
gbrain serve --http --port 3131 --bind 0.0.0.0
```
The `--bind 0.0.0.0` is important. By default the server binds to localhost only, which is correct for a personal install but blocks remote teammates. Setting `0.0.0.0` accepts connections from any interface.
The `--bind 0.0.0.0` is important. By default the server binds to localhost only, which is correct for a personal install but blocks remote teammates. Setting `0.0.0.0` accepts connections from any interface. (Full detail on `--bind` / `--public-url`, including the ECONNREFUSED failure mode they prevent, lives in [DEPLOY.md — Expose the server](../mcp/DEPLOY.md#3-expose-the-server).)
The server prints an admin bootstrap token to stderr on first start when run in an interactive terminal. Save it. You'll use it once for the admin dashboard. On a non-TTY start (systemd, Docker, piped logs) the token is hidden from logs — set `GBRAIN_ADMIN_BOOTSTRAP_TOKEN` yourself or pass `--print-admin-token` on a trusted terminal instead.
@@ -186,56 +186,67 @@ Each teammate (or each AI agent for a teammate) gets their own OAuth client. The
# Alice (sales): writes customers/alice-example, reads customers + shared
gbrain auth register-client alice-example \
--grant-types client_credentials \
--scopes read,write \
--scopes "read write" \
--source customers \
--federated-read customers,shared
# Bob (ops): writes internal/bob-example, reads internal + shared
gbrain auth register-client bob-example \
--grant-types client_credentials \
--scopes read,write \
--scopes "read write" \
--source internal \
--federated-read internal,shared
# Carol (legal): writes shared/legal, reads all three
gbrain auth register-client carol-example \
--grant-types client_credentials \
--scopes read,write \
--scopes "read write" \
--source shared \
--federated-read shared,customers,internal
```
Each `register-client` command prints a `client_id` and a `client_secret`. Save both for each teammate. They go into the teammate's local agent config.
Each `register-client` command prints a `client_id` and a `client_secret`. Save both for each teammate. They go into the teammate's local agent config. (The full registration reference — grant types, the `/admin` dashboard flow, DCR — lives in [DEPLOY.md — Register OAuth clients](../mcp/DEPLOY.md#2-register-oauth-clients). What follows is the multi-user delta.)
A note on the flags:
- `--scopes read,write` lets the client query the brain and write new pages. You can omit `write` for read-only clients (executive summaries, dashboards). The `admin` scope is needed for operational commands like `gbrain remote doctor` and is usually reserved for your own admin client.
- `--scopes "read write"` (space-separated, quoted — the OAuth wire format; a comma-separated list is rejected at registration) lets the client query the brain and write new pages. You can omit `write` for read-only clients (executive summaries, dashboards). The `admin` scope is needed for operational commands like `gbrain remote doctor` and is usually reserved for your own admin client.
- `--source` controls write authority. A client can only write to one source. Within that source, your folder convention from Part 3 keeps each person's writes in their own subfolder — and you can make that server-enforced with `--bound-slug-prefixes alice-example/` (v0.42.72.0+): every slug-mutating write op (put_page, delete_page, tags, links, timeline, revert, raw data) outside the bound prefixes is rejected with `permission_denied`. Update the binding later with `gbrain auth rescope-client <id> --bound-slug-prefixes <p1,p2|none>`. **Adding a binding to an existing client narrows it in ways you should expect:** ops that write by something other than a slug (`extract_entities`, `extract_facts`, `forget_fact`, `ontology_propose`, `sources_add`/`sources_remove`) and `POST /ingest` become unavailable to that client, and `put_page`'s automatic fact extraction is skipped — all because none of them can be confined to a prefix. Reads are unaffected. See [the qm-harness guide](../integrations/qm-harness.md) for the full model.
- `--federated-read` controls read scope. A client can read from one or more sources.
### Verify the scoping actually scopes
Before you hand the brain to teammates, verify isolation. Two terminal windows on your local machine using each client's credentials:
Before you hand the brain to teammates, verify isolation. The clean way is a **thin-client install** on a second machine (or a scratch shell): `gbrain init --mcp-only` writes a config that routes every CLI command through your remote server as one specific OAuth client, so a plain `gbrain search` exercises exactly the path teammates will use.
```bash
# Terminal 1, as Alice
export GBRAIN_REMOTE_CLIENT_ID=<Alice's client_id>
export GBRAIN_REMOTE_CLIENT_SECRET=<Alice's client_secret>
export GBRAIN_REMOTE_MCP_URL=https://brain.acme-co.com/mcp
# As Alice (on a machine that is NOT the brain host)
gbrain init --mcp-only \
--issuer-url https://brain.acme-co.com \
--mcp-url https://brain.acme-co.com/mcp \
--oauth-client-id <Alice's client_id> \
--oauth-client-secret <Alice's client_secret>
gbrain search "performance review" --remote
gbrain whoami # confirms which client you're acting as
gbrain search "performance review"
```
Alice should see results only from `customers` and `shared`. The performance-review notes live in `internal`, which she's not scoped to read. She shouldn't see them.
Now re-run the same check as Bob. On the same test machine, swap the credentials with `--force` (it overwrites the thin-client config):
```bash
# Terminal 2, as Bob (export his credentials similarly)
gbrain search "performance review" --remote
gbrain init --mcp-only --force \
--issuer-url https://brain.acme-co.com \
--mcp-url https://brain.acme-co.com/mcp \
--oauth-client-id <Bob's client_id> \
--oauth-client-secret <Bob's client_secret>
gbrain whoami
gbrain search "performance review"
```
Bob should see the performance-review notes from `internal`, plus anything related from `shared`. He shouldn't see anything that lives only in `customers`.
If both queries return correctly scoped results, isolation is working.
If both queries return correctly scoped results, isolation is working. (There is no per-query "act as client X" flag — the thin-client config decides which credential the CLI uses; only the client secret can be overridden at call time via `GBRAIN_REMOTE_CLIENT_SECRET`.)
---
@@ -274,13 +285,13 @@ copy to customers/alice-example/digests/YYYY-MM-DD-pipeline.md.
The `client:` field tells the cron runner which OAuth client to use, which enforces the scoping. Alice's cron can only read Alice's sources and write to Alice's folder. It cannot accidentally touch Bob's customer notes.
To install the cron schedule, commit the file to the workspace repo and let AlphaClaw pick it up on next deploy. The cron-scheduler skill (one of the 60 that GBrain installed) handles the dispatch.
To install the cron schedule, commit the file to the workspace repo and let AlphaClaw pick it up on next deploy. The cron-scheduler skill (one of the bundled skills GBrain installed) handles the dispatch. (The `client:` frontmatter field is a workspace/harness convention — your cron runner reads it and picks the matching OAuth credential; GBrain enforces the scoping once the credential is used.)
---
## Part 7: Add per-person skills
The 60+ skills GBrain installs are generic. Your team probably wants a few that are specific to them. Examples:
The bundled skills GBrain installs are generic. Your team probably wants a few that are specific to them. Examples:
- `onboarding-new-hire`. Only Carol (HR) runs this. Walks through generating a welcome packet, scheduling intro meetings, provisioning accounts.
- `customer-success-followup`. Only Alice (sales) runs this. Pulls latest customer page, drafts a follow-up email, posts to her review queue.
@@ -307,7 +318,7 @@ gbrain skillify scaffold onboarding-new-hire
That creates the directory + SKILL.md + routing entry. Edit the SKILL.md to describe the procedure, commit, deploy. The agent picks up the new skill on next request.
Per-person scoping for skills is handled at the routing layer: a skill can declare `allowed_clients: [carol-example]` in its frontmatter. If Alice asks her agent to run that skill, the agent refuses with "this skill is scoped to carol-example."
Per-person scoping for skills is a routing-layer **convention, enforced by your agent harness, not by GBrain**: declare something like `allowed_clients: [carol-example]` in the skill's frontmatter and instruct your agent (in its routing rules) to refuse the skill for anyone else. The hard guarantee stays at the data layer — even if the agent runs the skill anyway, Alice's OAuth credential still can't read or write outside her scoped sources.
### Shared rule files at the skills root
@@ -382,9 +393,9 @@ Repeat this flow for every new teammate. About 45 minutes per person, total. Com
## Part 10: Connect each teammate's AI client
Each teammate runs their AI client (Claude Code, Cursor, Claude Desktop, OpenClaw, Hermes, whatever) configured to point at your brain server through their OAuth credentials.
Each teammate runs their AI client (Claude Code, Codex, Claude Desktop, OpenClaw, Hermes, whatever) configured to point at your brain server. Two pieces, both direct-to-server — there is no local relay in between:
Recommended path for each teammate: the thin-client install. On their machine:
**1. The GBrain CLI, as a thin client (recommended for everyone).** On their machine:
```bash
curl -fsSL https://bun.sh/install | bash
@@ -397,24 +408,14 @@ gbrain init --mcp-only \
--oauth-client-secret <their client_secret>
```
The thin-client install creates a local config that knows how to talk to your brain but never opens its own database. Most CLI commands route through the remote server transparently.
The thin-client install creates a local config that knows how to talk to your brain but never opens its own database. From then on, plain CLI commands (`gbrain search`, `gbrain query`, `gbrain think`, `gbrain whoami`, ...) route through your remote server transparently, as that teammate's OAuth client. Local-only commands (`gbrain sync`, `gbrain serve`, `gbrain embed`, ...) are refused with a hint — those run on the brain host, not on teammate laptops.
Now they configure their AI client. For Claude Desktop, the teammate adds an MCP server entry in `~/Library/Application Support/Claude/claude_desktop_config.json`:
**2. Their AI client, connected directly to `https://brain.acme-co.com/mcp`.** Each client has its own connection shape; the per-client pages in [`docs/mcp/`](../mcp/) are the reference:
```jsonc
{
"mcpServers": {
"company-brain": {
"command": "gbrain",
"args": ["serve"]
}
}
}
```
When Claude Desktop launches, it talks to the local `gbrain serve` stdio bridge, which forwards every request to your remote brain over HTTPS with their OAuth token attached. From Claude Desktop's perspective it's just one MCP server.
For Claude Code, Cursor, OpenClaw, Hermes, and other clients, per-client setup steps live in [`docs/mcp/`](../mcp/). They all follow the same shape: point the agent at the local `gbrain serve` bridge, which knows about the remote.
- **Claude Code / Codex** — one command from anywhere `gbrain` is installed: `gbrain connect https://brain.acme-co.com/mcp --token <token> --install` (see [CLAUDE_CODE.md](../mcp/CLAUDE_CODE.md) / [CODEX.md](../mcp/CODEX.md)). Note the credential type: `gbrain connect` for these two agents uses **bearer tokens** (`gbrain auth create <name>`), which are full-access. That's fine for you as the admin; for source-scoped teammates, the scoped credential is their OAuth client — use it via the thin-client CLI above and the OAuth-capable clients below.
- **Claude Desktop** — remote servers are added through the GUI: **Settings > Integrations**, URL `https://brain.acme-co.com/mcp`. Do **not** put a remote server in `claude_desktop_config.json`; that file only works for local stdio servers and fails silently for remote ones. See [CLAUDE_DESKTOP.md](../mcp/CLAUDE_DESKTOP.md).
- **ChatGPT** ([CHATGPT.md](../mcp/CHATGPT.md)) and **Perplexity** ([PERPLEXITY.md](../mcp/PERPLEXITY.md)) — both speak OAuth to the server directly, so per-teammate scoping carries into those tools. Perplexity uses the same `client_credentials` clients you registered in Part 5. ChatGPT needs an `authorization_code` (PKCE) client — register one per teammate with the same `--source` / `--federated-read` flags.
- **OpenClaw / Hermes forks** — if the teammate's own agent runs on a machine with a full local gbrain install, it can use local stdio (`gbrain serve`) against its own brain and reach yours over HTTP MCP like any other remote client.
---
@@ -518,7 +519,7 @@ The first sync embeds every page, which takes time. Check `gbrain sources status
### "I see a page I shouldn't see"
This shouldn't happen, but if you suspect it, run `gbrain search <query> --remote --json` as the constrained client and inspect the `source_id` field on every returned result. Every row should be in the client's `--federated-read` set. If one isn't, file an issue with the exact slug and source IDs.
This shouldn't happen, but if you suspect it, run `gbrain search "<query>" --json` from a thin-client install configured with the constrained client's credentials (the Part 5 verification setup) and inspect the `source_id` field on every returned result. Every row should be in the client's `--federated-read` set. If one isn't, file an issue with the exact slug and source IDs.
### "The synthesized answer is wrong"
@@ -537,7 +538,7 @@ Each parallel sync worker opens its own pool. With three sources and the default
```bash
gbrain auth register-client diana-example \
--grant-types client_credentials \
--scopes read,write \
--scopes "read write" \
--source shared \
--federated-read shared,customers,internal
```
docs/tutorials/connect-coding-agent.md
+5 -5
View File
@@ -66,9 +66,9 @@ special). Turn it on:
gbrain config set mcp.publish_skills true
```
(New brains from `gbrain init` default this ON. Brains upgraded from before
v0.41.36 stay OFF until you opt in, so this is the common gotcha for existing
OpenClaw users.)
(New brains from `gbrain init` default this ON. Brains upgraded from a release
before skill publishing existed stay OFF until you opt in, so this is the
common gotcha for existing OpenClaw users.)
### A2. On the host: mint a token
@@ -96,7 +96,7 @@ token fails right now, not silently on the agent's first request. You'll see:
```
Added MCP server 'gbrain' -> https://your-host.example.com/mcp.
Verified: {"version":"0.42.x","engine":"postgres","page_count":146646,...}
Verified: {"version":"0.42.x","engine":"postgres","page_count":1204,...}
```
Drop `--install` to print a paste-ready block instead (useful when the host and
@@ -234,7 +234,7 @@ habits to build. Your agent stops being amnesiac.
## Next steps
- Go full autonomous: the overnight enrichment daemon ([dream cycle](../../CHANGELOG.md)) fixes citations, dedupes people, builds scorecards while you sleep. See `gbrain autopilot --install`.
- Go full autonomous: the overnight enrichment daemon ([dream cycle](../guides/operational-disciplines.md)) fixes citations, dedupes people, builds scorecards while you sleep. See `gbrain autopilot --install`.
- Run a real agent platform on top: [personal-brain tutorial](personal-brain.md).
- Scale to a team: [company-brain tutorial](company-brain.md).
- Every MCP client's exact setup: [`docs/mcp/`](../mcp/).
docs/tutorials/personal-brain.md
+4 -4
View File
@@ -6,7 +6,7 @@ This is the install I'd run if I were setting up the whole stack from scratch to
> "This is the Apple I, we're just soldering breadboards over here."
If you only want the **brain layer** (no agent, no Telegram, just gbrain as memory for an MCP client you already use), skip to the [CLI standalone install](../INSTALL.md#2-cli-standalone) in INSTALL.md. If you want the whole agent **shared with a team**, read the [company brain tutorial](company-brain.md) instead. This tutorial is the solo, full-stack, talk-to-it-on-Telegram path.
If you only want the **brain layer** (no agent, no Telegram, just gbrain as memory for an MCP client you already use), skip to the [CLI standalone install](../INSTALL.md#2-cli-standalone) in INSTALL.md. If your daily driver is a **coding agent** (Claude Code / Codex) and you want it to bootstrap its own full agent — identity, per-turn context, schedules, a private repo as its durable body — that's `gbrain bootstrap`: see the paste blocks in the README and [docs/guides/bootstrap.md](../guides/bootstrap.md). If you want the whole agent **shared with a team**, read the [company brain tutorial](company-brain.md) instead. This tutorial is the solo, full-stack, talk-to-it-on-Telegram path.
---
@@ -17,7 +17,7 @@ A personal AI agent with four pieces:
- **A brain** (git repo). Your knowledge base, constantly ingesting and growing.
- **A harness** (OpenClaw via AlphaClaw). The runtime that gives the LLM tools, memory, and integrations.
- **A chat interface** (Telegram). How you talk to it.
- **Skills** (60+ installed via GBrain). Reusable capabilities the agent can invoke.
- **Skills** (50+ installed via GBrain). Reusable capabilities the agent can invoke.
Architecture:
@@ -127,7 +127,7 @@ gbrain skillpack scaffold --all
`gbrain init --supabase` walks a short wizard that asks for your Supabase connection string and creates the schema. You'll get that connection string in Step 7 — read 7a and 7b first so you paste the right one (the transaction pooler, not the direct connection). If you'd rather try things locally before paying for a database, `gbrain init --pglite` gives you a zero-config embedded engine instead; you can migrate to Supabase later with `gbrain migrate --to supabase`.
`gbrain skillpack scaffold --all` copies the ~43 bundled skills into your agent workspace as first-class files you can edit freely. (The old managed-install model was retired in v0.36.0.0; see `docs/INSTALL.md` if you're upgrading from an older release.)
`gbrain skillpack scaffold --all` copies the 50+ bundled skills into your agent workspace as first-class files you can edit freely. (The old managed-install model was retired; see `docs/INSTALL.md` if you're upgrading from an older release.)
From this point, the agent has working memory and access to every skill.
@@ -252,7 +252,7 @@ My production setup is about $10,000 a month, but that's 10 instances, 200 crons
2. **GitHub PAT can't see the repos.** Reload the page after creating repos. Make sure the fine-grained token has the correct repo selection.
3. **Telegram bot doesn't respond.** Check the bot token in AlphaClaw. Make sure the Render instance is actually running.
4. **Supabase bottleneck on heavy ingestion.** Upgrade the DB instance size before the small one chokes.
5. **GBrain.io provisioning fails.** The hosted instance may need Pro tier. Check the machine allocation in the AlphaClaw UI.
5. **Hosted-instance provisioning fails.** If you're using a hosted brain instance instead of self-hosting, it may need the Pro tier. Check the machine allocation in the AlphaClaw UI.
---
docs/what-schemas-unlock.md
+4 -4
View File
@@ -107,7 +107,7 @@ The schema is the team's tribal knowledge made explicit. Two engineers on differ
This is what v0.40.7.0 actually enabled, and what the closed PR #1321 was reaching for.
Your OpenClaw (or any agent connected to your brain over HTTPS MCP with admin scope) watches your ingestion stream. After a week of you dumping notes under `garrytan/companies/yc-w24/`, the agent runs `gbrain schema detect` periodically, sees that prefix accumulating, and proposes:
Your OpenClaw (or any agent connected to your brain over HTTPS MCP with admin scope) watches your ingestion stream. After a week of you dumping notes under `companies/yc-w24/`, the agent runs `gbrain schema detect` periodically, sees that prefix accumulating, and proposes:
> You have 47 pages under `companies/yc-w24/` typed as `company` (generic). They share a structural pattern (founder names, raise amounts, batch tag). Should I add a `yc-w24-company` type with `extractable: true` and the existing aliases pointing back to `company`? I'd backfill the 47 pages and add `cohort=W24` as a typed fact extracted from each page.
@@ -147,7 +147,7 @@ Three things gbrain does that generic note systems can't:
**2. Untyped content is invisible content.** If your meetings are typed as `note`, expert routing skips them, facts extraction ignores them, link inference doesn't fire. They exist on disk and they're indexed for text search, but the structural surfaces (whoknows, find_experts, recall, think) treat them as second-class. Adding a type isn't cosmetic; it's structural promotion.
**3. The schema is queryable AND mutable AND auditable.** You can ask the brain what its schema looks like (`gbrain schema graph`), evolve it through 14 atomic CLI verbs + 9 MCP ops with full lock + audit semantics, and recover from any mistake (every primitive has an inverse, plus `gbrain schema downgrade` restores the previous active pack). This isn't "vibes-based knowledge management." It's a production system with structural integrity guarantees.
**3. The schema is queryable AND mutable AND auditable.** You can ask the brain what its schema looks like (`gbrain schema graph`), evolve it through atomic CLI verbs + MCP ops (`gbrain schema --help` for the full surface) with full lock + audit semantics, and recover from any mistake (every primitive has an inverse, plus `gbrain schema downgrade` restores the previous active pack). This isn't "vibes-based knowledge management." It's a production system with structural integrity guarantees.
## What changed in v0.40.7.0 specifically
@@ -156,7 +156,7 @@ v0.39.1.0 shipped the schema-pack engine. You could ALREADY fork the bundled pac
v0.40.7.0 closed those gaps:
- **`withMutation` skeleton** wraps every primitive in 8 ordered safety steps (bundled-guard → lock → read → mutate → validate → atomic write → audit → invalidate). The pack file on disk is never partial. Two concurrent agents can't race.
- **Per-pack `O_CREAT|O_EXCL` atomic lock** (not the TOCTOU `existsSync+writeFileSync` pattern from page-lock.ts — codex caught that during plan review). TTL refresh every 10s while a mutation runs; `--force` means "steal stale lock" not "skip locking."
- **Per-pack `O_CREAT|O_EXCL` atomic lock** (deliberately NOT the TOCTOU-prone `existsSync+writeFileSync` pattern). TTL refresh every 10s while a mutation runs; `--force` means "steal stale lock" not "skip locking."
- **Privacy-redacted audit log** at `~/.gbrain/audit/schema-mutations-YYYY-Www.jsonl`. Type names sha8-hashed, prefixes truncated to first segment only. A leaked screenshot of the audit can't reveal sensitive taxonomy like `personal/oncology/` or `legal/depositions/`.
- **9 new MCP ops** including the batched `schema_apply_mutations` (admin scope, NOT localOnly — your OpenClaw and any remote agent author packs over normal HTTPS MCP, with `client_id` captured as `actor: mcp:<clientId8>`).
- **T1.5 wiring** finally completes for `whoknows` and `find_experts`: a custom `researcher` type marked `--expert` now actually surfaces in query results. Pre-v0.40.7 it silently never matched because the query path read hardcoded `['person', 'company']`.
@@ -169,7 +169,7 @@ The cumulative effect: an agent can safely co-curate your ontology with a comple
- **Want to see it work in 5 minutes?** Run the [tutorial](schema-author-tutorial.md). Forks the bundled pack, adds a researcher type, proves the wiring end-to-end.
- **Want the agent recipe?** Read [`skills/schema-author/SKILL.md`](../skills/schema-author/SKILL.md). 7-phase workflow agents follow when they detect a schema-evolution opportunity.
- **Want the rules of thumb?** Read [`skills/conventions/schema-evolution.md`](../skills/conventions/schema-evolution.md). Decision tree for when to add a type vs alias vs prefix. <20 pages don't pack-codify. 100+ pages need first-class types.
- **Want the architecture?** The "Schema Cathedral v3 (v0.40.7.0)" section in `CLAUDE.md` has the 14-bullet module-by-module breakdown, each citing the design decision and codex finding that motivated it.
- **Want the architecture?** The "Schema Cathedral v3" section in `docs/architecture/KEY_FILES.md` has the module-by-module breakdown.
- **Want to set up an agent that co-curates your brain?** Run `gbrain auth register-client my-agent --scopes admin` to mint an OAuth client your remote agent can use to call `schema_apply_mutations` over MCP. The agent then runs detect → suggest → apply on its own cadence and asks you to approve substantive changes.
The killer feature isn't "schemas." Personal knowledge systems have had schemas forever. The killer feature is that your AGENT can shape them safely on your behalf, with structural integrity guarantees that match what you'd expect from a database, not a notes app.
llms-full.txt
+338 -268
View File
File diff suppressed because it is too large Load Diff
recipes/agent-voice.md
+15 -19
View File
@@ -10,15 +10,9 @@ secrets:
- name: OPENAI_API_KEY
description: OpenAI API key with Realtime API access enabled
where: https://platform.openai.com/api-keys — click "+ Create new secret key", copy immediately
- name: TWILIO_ACCOUNT_SID
description: (optional) Twilio Account SID — only if wiring inbound Twilio calls
where: https://www.twilio.com/console
- name: TWILIO_AUTH_TOKEN
description: (optional) Twilio auth token — only if wiring inbound Twilio calls
where: https://www.twilio.com/console
health_checks:
- type: env_exists
var: OPENAI_API_KEY
name: OPENAI_API_KEY
label: OPENAI_API_KEY present
setup_time: 10 min
cost_estimate: "$0.06-0.24/min OpenAI Realtime, optional $1-2/mo Twilio number"
@@ -34,20 +28,22 @@ A reference voice agent (WebRTC-first; OpenAI Realtime) shipped as **copy-into-y
- **WebRTC browser client** at `/call?test=1` for the production-grade voice loop. Production load installs zero test instrumentation; `?test=1` enables Web Audio API tee → MediaRecorder capture for the E2E.
- **Tool router** with a read-only allow-list by default (search, query, get_page, list_pages, find_experts, get_recent_salience, get_recent_transcripts, read_article). Write ops are denylisted; operators opt in to a bounded set via local override.
- **Persona-aware prompt builder** with identity-first composition + Unicode sanitization for Realtime API safety.
- **Optional Twilio adapter** (`/voice` TwiML, WSS bridge) for phone inbound. Skip if you only want browser voice.
- **Optional Twilio adapter** (`/voice` TwiML, WSS bridge) for phone inbound. Skip if you only want browser voice. If you wire it, set `TWILIO_ACCOUNT_SID` and `TWILIO_AUTH_TOKEN` in `$TARGET_REPO/.env` (from https://www.twilio.com/console). They're deliberately NOT in this recipe's `secrets:` frontmatter — every listed secret must be set before the integration reports `configured`, and Twilio is genuinely optional.
- **Three skills** for resolver routing: `voice-persona-mars`, `voice-persona-venus`, `voice-post-call`.
- **Unit + E2E tests** that ride with the copy. PII-shape regex guards every prompt, classifier triages upstream vs plumbing failures.
- **Unit tests** that ride with the copy (PII-shape regex guards every prompt; a classifier triages upstream vs plumbing failures). The E2E and eval suites stay gbrain-side under `recipes/agent-voice/tests/` — see Tests below.
## The skillpack-as-reference paradigm
Earlier gbrain skillpacks installed to `~/.gbrain/skills/<name>/` as managed-block-canonical first-class skills. The user's local edits drifted from the canonical and updates were either "overwrite local" or "skip update" — neither is what an operator wants on code they've extended.
This recipe ships a different shape: gbrain holds the up-to-date REFERENCE, and `gbrain integrations install agent-voice --target <host-repo>` COPIES it into the operator's repo. The code now lives in the host repo, on the operator's release cadence, with the operator's edits. Subsequent `--refresh` invocations diff host-side files against gbrain's reference and propose changes; the operator picks per-file (keep mine / take theirs / merge).
This recipe ships a different shape: gbrain holds the up-to-date REFERENCE, and `gbrain integrations install agent-voice --target <host-repo>` COPIES it into the operator's repo. The code now lives in the host repo, on the operator's release cadence, with the operator's edits. Subsequent `--refresh` invocations diff host-side files against gbrain's reference and apply updates while preserving local edits by default (`--auto keep-mine|take-theirs` for CI lanes).
The shipped reference does NOT contain personal names, hardcoded private paths, or upstream-agent codenames. A CI guard (`scripts/check-no-pii-in-agent-voice.sh`) blocks any drift back; a deterministic import script (`scripts/import-from-upstream.sh`) refreshes the gbrain reference from an upstream voice-agent source.
## Install
> Note: this recipe's category is `voice`, which the `gbrain integrations list` dashboard does not render (it shows infra / sense / reflex sections only). Install it directly by id, as below.
```bash
# 1. Detect target repo
export TARGET_REPO=$OPENCLAW_WORKSPACE # or your agent repo path
@@ -85,15 +81,9 @@ git -C $(which gbrain | xargs -I{} dirname {})/.. pull # or your gbrain update
gbrain integrations install agent-voice --target $TARGET_REPO --refresh
```
`--refresh` reads the `.gbrain-source.json` manifest written by the original install, re-computes per-file SHA-256 against gbrain's current reference, and classifies each file:
`--refresh` reads the `.gbrain-source.json` manifest written by the original install, re-computes per-file SHA-256 against gbrain's current reference, and classifies each file into one of six states (identical / stale / locally-modified / host-deleted / source-deleted / new-in-manifest). Stale and new files are updated automatically; **locally-modified files are preserved by default** (`--auto take-theirs` to overwrite; `--dry-run` to preview). An append-only audit journal is written to `<target>/services/voice-agent/.gbrain-source.refresh.log`.
- **unchanged-identical** — host file matches gbrain reference; skip.
- **unchanged-stale** — host file matches the recorded SHA but reference moved; offer to update.
- **locally-modified** — host file diverges from the recorded SHA; show diff, offer three options (keep mine / take theirs / merge).
- **source-deleted** — gbrain reference removed a file; offer cleanup.
- **source-renamed** — detected via path-mapping; offer to follow.
A transaction journal at `<target>/services/voice-agent/.gbrain-source.refresh.log` allows partial-apply recovery if the refresh is interrupted.
The full state machine and per-state decisions live in [`recipes/agent-voice/install/refresh-algorithm.md`](agent-voice/install/refresh-algorithm.md) — the single home for refresh semantics.
## Architecture
@@ -142,9 +132,15 @@ Reference code ships intentionally minimal. Before public deployment:
## Tests
The install copies the **unit suites only**; the E2E and eval suites stay gbrain-side under `recipes/agent-voice/tests/` (they carry puppeteer fixtures and live-API costs the host repo shouldn't inherit).
```bash
# Host-side (rides with the copy)
cd $TARGET_REPO/services/voice-agent
bun run test # host-side unit tests (5 suites, ~100 cases)
bun run test # unit tests
# gbrain-side (from your gbrain checkout)
cd <gbrain-checkout>/recipes/agent-voice && bun install
AGENT_VOICE_E2E=1 bun run test:e2e # WebRTC roundtrip (~$0.10/run)
AGENT_VOICE_FULL_E2E=1 bun run test:full-flow # openclaw-driven install + roundtrip (~$1-2/run)
```
recipes/agent-voice/README.md
+5 -5
View File
@@ -25,10 +25,10 @@ recipes/<name>/
├── README.md # paradigm doc; gbrain-side only (not copied)
├── package.json # top-of-bundle; copied to <host>/services/<name>/package.json
├── code/ # copied to <host>/services/<name>/code/
├── tests/ # copied to <host>/services/<name>/tests/
│ ├── unit/
│ ├── e2e/
│ └── evals/
├── tests/
│ ├── unit/ # copied to <host>/services/<name>/tests/unit/
│ ├── e2e/ # gbrain-side only (puppeteer + live-API costs)
│ └── evals/ # gbrain-side only (LLM-judge suites)
├── skills/ # copied to <host>/skills/<skill-name>/
├── install/ # gbrain-side only; install metadata
│ ├── manifest.json # src → target map + per-file SHA-256
@@ -54,7 +54,7 @@ Three rules for new recipes following this shape:
- `code/lib/personas/private-name-blocklist.json` — privacy guard source of truth (read by the shipped guard script and by host-side prompt-shape tests).
- `code/lib/personas/context-builder.contract.md` — API the operator implements for live brain context.
## Files (in `bundle = code/ + tests/ + skills/ + package.json`) — copied to host repo
## Files (in `bundle = code/ + tests/unit/ + skills/ + package.json`) — copied to host repo
The install subcommand reads `install/manifest.json` and copies each listed file to its target path under the host repo. SHA-256s computed at copy time get persisted into `<host>/services/<name>/.gbrain-source.json` so `--refresh` can do three-way classification (unchanged-identical / unchanged-stale / locally-modified) without re-walking the entire bundle.
recipes/agent-voice/install/post-install-hint.md
+10 -5
View File
@@ -67,9 +67,14 @@ bun run start # or `npm start`
Open `http://localhost:8765/call` in a browser, click Connect, grant mic permission. You should be talking to Venus (or Mars if you set `DEFAULT_PERSONA=mars`).
### 6. (Optional) Run the WebRTC roundtrip E2E
### 6. (Optional) Run the WebRTC roundtrip E2E — from the gbrain checkout
The install copies **unit tests only**. The E2E and eval suites stay gbrain-side
(under `recipes/agent-voice/tests/`), so run them from your gbrain checkout,
not from `<target-repo>`:
```bash
cd <gbrain-checkout>/recipes/agent-voice && bun install
export AGENT_VOICE_E2E=1 OPENAI_API_KEY=sk-...
bun run test:e2e
# → ~$0.10/run; spawns server, drives puppeteer with a fake-audio WAV
@@ -83,15 +88,15 @@ gbrain claw-test --scenario voice-agent-install --live --agent openclaw
# → ~$1-2/run; friction-discovery test, NOT a ship gate
```
### 7. (Optional) Run the LLM-judge persona evals
### 7. (Optional) Run the LLM-judge persona evals — from the gbrain checkout
```bash
cd <target-repo>/services/voice-agent
cd <gbrain-checkout>/recipes/agent-voice
node tests/evals/mars-eval.mjs # ~$1-3 for the full 3-model judge sweep
node tests/evals/venus-eval.mjs
```
Synthetic canonical baselines are committed under `tests/evals/baseline-runs/canonical/`. Live receipts you generate go to `tests/evals/baseline-runs/` (gitignored — they may contain residual brain content from your live personas).
Live receipts you generate go to `tests/evals/baseline-runs/` (gitignored — they may contain residual brain content from your live personas). See `tests/evals/README.md` for the pass criteria and failure triage.
### 8. Update later
@@ -101,4 +106,4 @@ When gbrain ships a new agent-voice reference, refresh your local copy:
gbrain integrations install agent-voice --target <target-repo> --refresh
```
The refresh classifies each file (identical / stale / locally-modified / source-deleted / host-deleted) and lets you decide per-file. See `<target>/services/voice-agent/code/install/refresh-algorithm.md` (copied from gbrain) for the contract.
Refresh classifies each file (six states — identical / stale / locally-modified / host-deleted / source-deleted / new-in-manifest) and applies a deterministic decision per state: local edits are preserved by default; pass `--auto take-theirs` to take upstream everywhere, or `--dry-run` to preview. The full contract lives gbrain-side at `recipes/agent-voice/install/refresh-algorithm.md` (not copied to the host repo).
recipes/agent-voice/install/refresh-algorithm.md
+40 -40
View File
@@ -1,74 +1,74 @@
# Refresh algorithm (diff-and-propose)
`gbrain integrations install agent-voice --refresh` re-walks the manifest, classifies every file into one of five states, and lets the operator decide per-file. The reference implementation is in `src/commands/integrations.ts` under the `install_kind: copy-into-host-repo` branch.
`gbrain integrations install agent-voice --refresh` re-walks the manifest, classifies every file into one of six states, and applies a deterministic decision per state. The implementation is in `src/commands/integrations.ts` under the `install_kind: copy-into-host-repo` branch (`refreshRecipeIntoHostRepo` / `classifyForRefresh`).
This file is the single home for refresh semantics. `recipes/agent-voice.md` and `install/post-install-hint.md` summarize and link here.
## State machine
For each file declared in `install/manifest.json`:
For each file declared in `install/manifest.json` (plus each file in the prior install record):
```
Let src_hash = SHA-256 of gbrain-side file at manifest.src
Let host_path = <target-repo>/<manifest.target>
Let recorded = .gbrain-source.json.files[].sha256 for this entry (or absent if first refresh)
Let host_hash = SHA-256 of host_path (or absent if file deleted on host side)
Let src_hash = SHA-256 of gbrain-side file at manifest.src
Let host_path = <target-repo>/<manifest.target>
Let recorded = .gbrain-source.json.files[].sha256 for this entry (absent if new)
Let host_hash = SHA-256 of host_path (absent if file missing on host side)
State:
State (and what refresh does about it):
- "unchanged-identical" iff host_hash == src_hash
→ no-op
- "unchanged-stale" iff host_hash == recorded AND host_hash != src_hash
→ operator unmodified, source moved → offer update
- "locally-modified" iff host_hash != recorded AND host_hash != src_hash AND host_hash is defined
→ operator edited locally; offer three options (see below)
- "host-deleted" iff host_hash is absent AND src exists
→ operator removed the file; offer to restore or to remove from manifest
- "source-deleted" iff src is absent AND host_hash is defined
→ gbrain reference removed the file; offer cleanup (remove from host)
→ operator unmodified, source moved → auto-updated (copied over)
- "locally-modified" iff host_hash != recorded AND host_hash != src_hash AND host exists
→ operator edited locally → default keep-mine; see below
- "host-deleted" iff host file absent AND src exists
→ left deleted, UNLESS --auto take-theirs (restores the file)
- "source-deleted" iff entry in the prior record but not in the current manifest
→ left in place ("orphan"), UNLESS --auto take-theirs (removes it)
- "new-in-manifest" iff entry in the manifest but not in the prior record
→ auto-installed (copied in)
```
A path-mapping renames table in the manifest (`renames: [{from, to}]`, not yet shipped) allows the refresh algorithm to detect a source-renamed file as a logical update rather than a delete+add.
There is no interactive per-file prompt: every run is non-interactive, and the only lever is `--auto keep-mine|take-theirs`. Without `--auto`, the defaults above apply (they match `--auto keep-mine`). Run `--dry-run` first to see the per-file classification before anything is written.
## "Locally-modified" decision
A path-mapping renames table in the manifest (`renames: [{from, to}]`, not yet shipped) would let refresh detect a source-renamed file as a logical update rather than a delete+add.
When a file shows `locally-modified`, the operator picks one of three options:
## The "locally-modified" decision
- **keep-mine** — leave host file untouched. The manifest entry's `sha256` is updated to the current host hash (the operator's edit becomes the new "recorded" baseline; future refreshes won't re-flag it until they edit it again OR the source changes).
- **take-theirs** — copy the gbrain reference over the host file. The recorded SHA becomes the new src_hash.
- **merge** — print a unified diff. Operator hand-merges in their editor; the refresh command exits without writing. Re-run `--refresh` after the merge to confirm.
- **keep-mine** (the default) — leave the host file untouched. The recorded `sha256` in `.gbrain-source.json` is re-baselined to the current host hash, so future refreshes won't re-flag this file until either side changes again.
- **take-theirs** (`--auto take-theirs`) — copy the gbrain reference over the host file. The recorded SHA becomes the new src_hash.
## Transaction journal
There is no `merge` option and no diff output. To hand-merge: run `--dry-run` to find locally-modified files, diff them yourself against the gbrain-side reference (the `src` path printed per file), merge in your editor, then re-run `--refresh`.
`<target-repo>/services/voice-agent/.gbrain-source.refresh.log` is a JSONL append-only file. Each line records:
## Transaction journal (audit log)
`<target-repo>/services/voice-agent/.gbrain-source.refresh.log` is a JSONL append-only file. Each line records one refresh event:
```json
{"ts": "2026-05-17T12:34:56Z", "src": "code/server.mjs", "state": "locally-modified", "decision": "keep-mine"}
{"ts": "2026-05-17T12:34:56Z", "event": "preserved_local", "src": "code/server.mjs", "target": "services/voice-agent/code/server.mjs", "decision": "keep-mine"}
```
The journal exists for two reasons:
1. **Partial-apply recovery.** If the refresh is interrupted mid-loop (Ctrl-C, crash, machine reboot), re-running `--refresh` reads the journal and resumes where it stopped.
2. **Audit.** Operators can grep the journal to see which files were touched and why.
The journal is rotated by file size (>1MB triggers rename to `.gbrain-source.refresh.log.1`) and ignored by `--refresh`'s own scan (the journal is host-only metadata, not a managed file).
## Concurrent refresh guard
`--refresh` acquires an advisory file lock at `<target-repo>/services/voice-agent/.gbrain-source.refresh.lock` for the duration of the run. Concurrent `--refresh` invocations on the same host repo fail-fast with "refresh already in progress."
The journal is an **audit log only** — grep it to see which files were touched by which refresh and why. It is never read back by `--refresh` (every run re-classifies from scratch), it is not rotated, and it is ignored by the scan itself (host-only metadata, not a managed file). Delete or truncate it whenever you like.
## CLI surface
```bash
gbrain integrations install agent-voice --target <repo> --refresh
gbrain integrations install agent-voice --target <repo> --refresh --dry-run # report-only
gbrain integrations install agent-voice --target <repo> --refresh --auto take-theirs # non-interactive
gbrain integrations install agent-voice --target <repo> --refresh --auto keep-mine # bias toward operator's edits
gbrain integrations install agent-voice --target <repo> --refresh --dry-run # report-only, per-file detail
gbrain integrations install agent-voice --target <repo> --refresh --auto take-theirs # always take upstream
gbrain integrations install agent-voice --target <repo> --refresh --auto keep-mine # explicit form of the default
```
`--auto <decision>` applies the named decision to ALL `locally-modified` files without prompting. Useful for CI lanes that want either "always take upstream" or "always preserve local" without operator interaction.
`--auto <decision>` applies the named decision to ALL `locally-modified` files (and, for `take-theirs`, also restores host-deleted files and cleans up source-deleted orphans). Useful for CI lanes.
## What this v0 deliberately skips
- Conflict resolution for files that exist in both manifests but at different paths (treated as add+delete).
- Concurrent edits on the SAME file mid-refresh (the advisory lock + per-file atomic write covers this).
- Semantic merges (we offer file-level diff only; no per-hunk picking).
- Manifest schema migration (v0.1.0 → v0.2.0 changes are handled by the install command refusing to refresh old manifests and asking the operator to re-install).
- **Interactive per-file prompting and a merge option** — every run is batch; hand-merges happen in your editor between a `--dry-run` and a re-run.
- **Journal replay / partial-apply resume** — an interrupted refresh is simply re-run; classification is recomputed from scratch, and completed copies classify as `unchanged-identical` on the second pass.
- **Journal rotation** — the log grows unbounded (slowly); truncate it yourself if it bothers you.
- **A concurrent-refresh lock** — don't run two refreshes against the same host repo at once.
- Renamed-path detection (the `renames` table above).
- Semantic merges (file-level only; no per-hunk picking).
- Manifest schema migration (breaking manifest changes are handled by the install command refusing to refresh and asking the operator to re-install).
Each of those is a follow-up TODO.
recipes/agent-voice/skills/voice-persona-mars/SKILL.md
+5 -5
View File
@@ -52,9 +52,9 @@ The persona prompt (`services/voice-agent/code/lib/personas/mars.mjs`) carries t
## Solo-mode tool posture
Mars uses tools SPARINGLY in solo mode. The right tools are:
- `search_brain` (find related concepts/people/meetings to deepen the reflection)
- `read_brain_page` (read a specific page aloud when the operator says "tell me about X")
Mars uses tools SPARINGLY in solo mode. The right tools (op names from the shipped `tools.mjs` allow-list) are:
- `search` (find related concepts/people/meetings to deepen the reflection)
- `get_page` (read a specific page aloud when the operator says "tell me about X")
- `read_article` (summarize a link the operator shared)
Calendar, tasks, email tools are DELIBERATELY ABSENT from Mars's solo-mode usage even though they're in the read-only allow-list. Mars redirects logistical questions to Venus.
@@ -62,8 +62,8 @@ Calendar, tasks, email tools are DELIBERATELY ABSENT from Mars's solo-mode usage
## Demo-mode tool posture
Mars uses tools AGGRESSIVELY in demo mode:
- Search the brain for people/companies the operator introduces
- Pull current events via `web_search` (when wired)
- Search the brain (`search` / `query`) for people/companies the operator introduces
- Summarize links via `read_article` (a general web-search tool is NOT in the shipped allow-list — wire one host-side if you want it)
- Cross-reference what the operator is saying against the brain in near-real-time
The goal: make the demo audience think "oh, this is what a personal AI can actually do."
recipes/agent-voice/skills/voice-persona-venus/SKILL.md
+5 -4
View File
@@ -45,16 +45,17 @@ Venus boots already knowing the topic's recent conversation. Only the `topicId`
## Tool posture
Venus uses the read-only allow-list from `services/voice-agent/code/tools.mjs`:
Venus uses the read-only allow-list from `services/voice-agent/code/tools.mjs` (op names as advertised to the model):
- `search_brain` (semantic + keyword search)
- `read_brain_page` (full page read aloud)
- `search` / `query` (semantic + keyword search)
- `get_page` (full page read aloud) / `list_pages`
- `read_article` (URL fetch + summarize)
- `web_search` (when wired)
- `get_recent_salience` (what's been emotionally active lately)
- `get_recent_transcripts` (recent voice notes / meeting transcripts)
- `find_experts` (who knows about a topic)
There is no general web-search tool in the shipped allow-list; wire one host-side if you want it.
Write tools (`put_page`, `submit_job`, `set_reminder` unless opted in, etc.) are NOT in Venus's tool surface. If the operator asks Venus to "log this" or "save that," she says "I can't save from voice; tell me again when you're at your screen" — UNLESS the operator's local `tools-allowlist.local.json` opts into the bounded write set.
## Language
recipes/agent-voice/skills/voice-post-call/SKILL.md
+15 -12
View File
@@ -1,7 +1,7 @@
---
name: voice-post-call
version: 0.1.0
description: Post-call handling for a voice session — turn the transcript into a brain page, post the summary to the operator's messaging surface, archive the audio. Belt-and-suspenders: fires both from a tool the voice persona can call mid-call AND from the automatic call-end handler in server.mjs.
description: Post-call handling for a voice session — turn the transcript into a brain page, post the summary to the operator's messaging surface, archive the audio. The pipeline is the contract; the firing paths are operator-wired (see "Two firing paths" below for what ships today).
triggers:
- "after the call"
- "call ended"
@@ -18,13 +18,11 @@ writes_to:
# voice-post-call — Post-session transcript + summary handling
> **Convention:** see [conventions/quality.md](../conventions/quality.md) for citation rules + back-link enforcement.
>
> **Convention:** see [_brain-filing-rules.md](../_brain-filing-rules.md) for filing decision protocol.
> **Convention:** see gbrain's `skills/conventions/quality.md` for citation rules + back-link enforcement, and `skills/_brain-filing-rules.md` for the filing decision protocol. (These are not copied by the install; the relative paths resolve only if your host repo mirrors gbrain's skills layout.)
## Iron Law
**Every call gets processed, even on tool-call failure.** The voice persona MAY call a `log_call_summary` tool mid-session, OR the call may end without that tool firing (model forgot, WebRTC dropped, browser crashed). The automatic call-end handler in `services/voice-agent/code/server.mjs` posts a structured signal regardless so the brain still gets the transcript + audio reference.
**Every call gets processed, even on tool-call failure.** The voice persona MAY log mid-session via an opted-in write tool, OR the call may end without that tool firing (model forgot, WebRTC dropped, browser crashed). A call-end handler should post a structured signal regardless so the brain still gets the transcript + audio reference — see "Two firing paths" below for which of these ships today and which the operator implements.
If both paths fire (the tool call AND the call-end handler), the second one is idempotent — it sees the brain page already exists and updates instead of duplicating.
@@ -52,13 +50,13 @@ If both paths fire (the tool call AND the call-end handler), the second one is i
Slack, Discord — whichever is wired in $TARGET_REPO/.env).
```
## Two firing paths (belt + suspenders)
## Two firing paths (both operator-wired today)
**Path A — Persona-initiated mid-call:**
The voice persona calls `log_call_summary` via the WebRTC data channel. The host-repo `/tool` endpoint dispatches to `tools.mjs`. Note: `log_call_summary` is in `OPTIONAL_OPS`, not `READ_ONLY_OPS`, so this only works if the operator's `tools-allowlist.local.json` opts in.
**Path A — Persona-initiated mid-call (opt-in):**
The voice persona calls `log_to_brain` via the WebRTC data channel; the host-repo `/tool` endpoint dispatches through `tools.mjs`. `log_to_brain` is in `OPTIONAL_OPS`, not `READ_ONLY_OPS`, so this only works if the operator's `tools-allowlist.local.json` opts in (there is no `log_call_summary` tool — the override can only enable ops listed in `OPTIONAL_OPS`).
**Path B — Automatic call-end (default):**
When the WebSocket / WebRTC connection closes, `server.mjs` fires a `call_end` event. The host repo's post-call handler (operator-implemented; the recipe ships a stub) reads the captured audio + transcript, runs the pipeline above. This path requires NO operator opt-in to work — the call-end handler is part of the shipped server.
**Path B — Call-end handler (not yet shipped):**
The shipped `server.mjs` has **no automatic call-end handler** — nothing fires when the WebSocket / WebRTC connection closes. To get the safety-net behavior, implement a post-call handler in your host repo that reads the captured audio + transcript on connection close and runs the pipeline above. Until you do, Path A (opt-in) is the only firing path, and calls where the persona never logs are NOT processed.
## Brain page format
@@ -116,10 +114,15 @@ created: 2026-05-17
## Related skills
Ships with this bundle (sibling directories after install):
- [voice-persona-mars](../voice-persona-mars/SKILL.md) — the persona that may invoke this
- [voice-persona-venus](../voice-persona-venus/SKILL.md) — the other persona that may invoke this
- [meeting-ingestion](../meeting-ingestion/SKILL.md) — analogous flow for multi-party meeting transcripts (different in that voice-call is typically 1:1)
- [voice-note-ingest](../voice-note-ingest/SKILL.md) — for recorded one-way voice memos (different from live voice calls)
Lives in gbrain's `skills/` (present on the host only if your repo mirrors gbrain's skills layout):
- `meeting-ingestion` — analogous flow for multi-party meeting transcripts (different in that voice-call is typically 1:1)
- `media-ingest` — for recorded one-way voice memos (different from live voice calls)
## Contract
recipes/agent-voice/tests/evals/README.md
+5 -5
View File
@@ -13,10 +13,10 @@ Pass criterion: every axis mean ≥ 7/10 AND no model scored any axis < 5 AND
## Running
```bash
# All four eval suites at the default judge tier (~$1-3/full run)
bun run gen:baselines # mars-eval + venus-eval + persona-routing + mars-multilingual
# Baseline receipts for the two persona evals (~$1-3/full run)
bun run gen:baselines # mars-eval --baseline + venus-eval --baseline
# Individually
# Individually (all four suites)
node tests/evals/mars-eval.mjs
node tests/evals/venus-eval.mjs
node tests/evals/persona-routing-eval.mjs
@@ -45,9 +45,9 @@ Capped well below the $1-3 budget. Cost stays low because the judge runs are sho
## Receipts
`baseline-runs/canonical/*.json` carries **agent-authored synthetic exemplars** — what a passing eval verdict looks like, with no real model output. Used for code-review and onboarding ("what does the harness produce?") without ever shipping residual private context.
`baseline-runs/canonical/` is **reserved for agent-authored synthetic exemplars** — what a passing eval verdict looks like, with no real model output — for code-review and onboarding ("what does the harness produce?") without ever shipping residual private context. No exemplars are committed yet; see `canonical/README.md` for the contract they must follow. (Note for whoever lands them: the sibling `.gitignore`'s `!canonical/` pattern does not unignore files inside the directory — it needs `!canonical/*.json`.)
`baseline-runs/*.json` (non-`canonical/`) is **gitignored**. Live receipts you generate against your own scrubbed personas live there; never commit them — they may carry response text that leaks operator-specific configuration.
`baseline-runs/*.json` is **gitignored**. Live receipts you generate against your own scrubbed personas live there; never commit them — they may carry response text that leaks operator-specific configuration.
## When evals fail
recipes/agent-voice/tests/evals/baseline-runs/canonical/README.md
+3 -3
View File
@@ -1,12 +1,12 @@
# Canonical baselines (synthetic exemplars)
These JSON files are **agent-authored synthetic exemplars** — what a passing eval verdict looks like. They contain NO real model output, NO real persona responses, NO operator-specific brain content. PII-impossible by construction.
This directory is reserved for **agent-authored synthetic exemplars** — what a passing eval verdict looks like. Exemplars must contain NO real model output, NO real persona responses, NO operator-specific brain content: PII-impossible by construction. **None are committed yet**; whoever lands the first ones must also fix the parent `.gitignore` (its `!canonical/` pattern doesn't unignore files inside the directory — it needs `!canonical/*.json`).
Use them as:
Once landed, use them as:
1. **Code-review reference** — when reviewing changes to `judge.mjs` or the persona prompts, eyeball these to see what the receipt schema looks like.
2. **Onboarding** — new contributors can read these to understand what the eval suite produces without spending API tokens.
3. **Schema documentation** — the field shape is the contract that live receipts must match.
**Never commit live receipts here.** Live receipts go in `../` (gitignored). The canonical/ subdirectory is the ONLY committed eval output in the entire bundle.
If the eval harness changes its receipt schema, regenerate these by hand-editing the JSON to match — do NOT generate them by running the harness against the real personas.
If the eval harness changes its receipt schema, regenerate exemplars by authoring the JSON to match the new schema — do NOT generate them by running the harness against the real personas.
recipes/calendar-to-brain.md
+20 -73
View File
@@ -90,7 +90,7 @@ Agent reads daily files
- 09:00-09:30 **Team standup** (Work) — with Alice, Bob, Carol
- 10:00-11:00 **Board meeting** (Work) 📍 Office — with Diana, Eduardo, Fiona
- 12:00-13:00 **Lunch with Pedro** (Personal) 📍 Chez Panisse — with Pedro Franceschi
- 12:00-13:00 **Lunch with Charlie** (Personal) 📍 A Restaurant — with charlie-example
- 14:00-14:30 **1:1 with Jordan** (Work) — with Jordan Lee
```
@@ -113,79 +113,26 @@ This builds the full relationship graph from day one.
## Setup Flow
### Step 1: Choose and Configure Calendar Access
### Step 1: Configure Calendar Access (via credential-gateway)
Ask the user: "How do you want to connect to Google Calendar?
Credential setup (ClawVisor vs direct Google OAuth, consent screen, validation
commands) lives in ONE place: run the **[credential-gateway](credential-gateway.md)**
recipe first — this recipe declares `requires: [credential-gateway]` for exactly
that reason. Then apply the two calendar-specific details:
**Option A: ClawVisor (recommended)**
ClawVisor handles OAuth, token refresh, and encryption. You never touch Google
credentials directly. If you already use ClawVisor for email, this uses the same setup.
- **Option A (ClawVisor):** activate the **Google Calendar** service and use a
task purpose like: "Full calendar access for historical backfill and ongoing
sync. List events, read event details, search across all calendars."
(Be EXPANSIVE — narrow purposes block requests; see credential-gateway's
Tricky Spots.)
- **Option B (direct OAuth):** the scope is
`https://www.googleapis.com/auth/calendar.readonly`, and the sync script's
OAuth flow stores tokens in `~/.gbrain/google-tokens.json` (auto-refreshes
on expiry). Also enable the Calendar API at
https://console.cloud.google.com/apis/library/calendar-json.googleapis.com
**Option B: Google OAuth2 directly**
Connect to Google Calendar API directly. No extra service needed, but you manage
OAuth tokens yourself. Good if you don't want another dependency."
#### Option A: ClawVisor Setup
Tell the user:
"I need your ClawVisor URL and agent token.
1. Go to https://clawvisor.com
2. Create an agent (or use existing)
3. Activate the **Google Calendar** service
4. Create a standing task with purpose: 'Full calendar access for historical
backfill and ongoing sync. List events, read event details, search across
all calendars.'
IMPORTANT: Be EXPANSIVE in the task purpose. Narrow purposes block requests.
5. Copy the gateway URL and agent token"
Validate:
```bash
curl -sf "$CLAWVISOR_URL/health" && echo "PASS: ClawVisor reachable" || echo "FAIL"
```
**STOP until ClawVisor validates.**
#### Option B: Google OAuth2 Setup
Tell the user:
"I need Google OAuth2 credentials. Here's exactly how to set them up:
1. Go to https://console.cloud.google.com/apis/credentials
(create a Google Cloud project if you don't have one)
2. Click **'+ CREATE CREDENTIALS'** at the top, select **'OAuth client ID'**
3. If prompted, configure the OAuth consent screen first:
- User type: **External** (or Internal if you have Google Workspace)
- App name: anything (e.g., 'GBrain Calendar')
- Scopes: add **'Google Calendar API .../auth/calendar.readonly'**
- Test users: add your own email
4. Back on Credentials, create the OAuth client ID:
- Application type: **Desktop app**
- Name: anything (e.g., 'GBrain')
5. Click **'Create'**. You'll see the Client ID and Client Secret.
6. Copy both and paste them to me.
Also enable the Calendar API:
7. Go to https://console.cloud.google.com/apis/library/calendar-json.googleapis.com
8. Click **'Enable'**"
Validate the credentials are set:
```bash
[ -n "$GOOGLE_CLIENT_ID" ] && [ -n "$GOOGLE_CLIENT_SECRET" ] \
&& echo "PASS: Google OAuth credentials set" \
|| echo "FAIL: Missing GOOGLE_CLIENT_ID or GOOGLE_CLIENT_SECRET"
```
Then run the OAuth flow to get an access token:
```bash
# The sync script should handle the OAuth flow:
# 1. Open browser to Google auth URL with calendar.readonly scope
# 2. User grants access
# 3. Script receives auth code, exchanges for access + refresh token
# 4. Stores tokens in ~/.gbrain/google-tokens.json
# 5. Auto-refreshes on expiry
```
**STOP until OAuth flow completes and tokens are stored.**
**STOP until credential-gateway's validation passes** (ClawVisor `/health` OK,
or OAuth tokens stored).
### Step 2: Identify Calendar Accounts
@@ -281,7 +228,7 @@ gbrain sync --no-pull --no-embed && gbrain embed --stale
```bash
mkdir -p ~/.gbrain/integrations/calendar-to-brain
echo '{"ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","event":"setup_complete","source_version":"0.7.0","status":"ok","details":{"accounts":"ACCOUNT_COUNT","start_year":"YYYY"}}' >> ~/.gbrain/integrations/calendar-to-brain/heartbeat.jsonl
echo '{"ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","event":"setup_complete","source_version":"0.8.0","status":"ok","details":{"accounts":"ACCOUNT_COUNT","start_year":"YYYY"}}' >> ~/.gbrain/integrations/calendar-to-brain/heartbeat.jsonl
```
Tell the user: "Calendar-to-brain is set up. You have [N] days of calendar history
@@ -322,7 +269,7 @@ filter_attendees(attendees):
return attendees.filter(a =>
!a.email?.includes('@resource.calendar.google.com') AND // conference rooms
!a.email?.includes('@group.calendar.google.com') AND // mailing lists
!a.name?.startsWith('YC-SF-') // internal distros
!a.name?.startsWith('ORG-') // internal distros (use your org's prefix)
)
```

Some files were not shown because too many files have changed in this diff Show More