Compare commits

..
Author SHA1 Message Date
root 820e138db0 fix: post-migration schema verification with self-healing
PgBouncer transaction-mode poolers can silently swallow ALTER TABLE
statements: the SQL doesn't error, but the column never gets created.
The migration system increments the schema version counter anyway, so
gbrain thinks it's on the latest version but the actual table is missing
columns. This caused production embed failures when the embed handler
tried to INSERT into columns that didn't exist.

Add verifySchema() that runs after all migrations complete:
1. Parses CREATE TABLE + ALTER TABLE ADD COLUMN from schema-embedded.ts
2. Queries information_schema.columns for actual DB state
3. Diffs expected vs actual columns
4. Self-heals missing columns via ALTER TABLE ADD COLUMN IF NOT EXISTS
5. Throws with actionable diagnostics if self-heal fails

Called from PostgresEngine.initSchema() after runMigrations().
PGLite skipped (in-process, no PgBouncer).
2026-04-28 04:46:38 +00:00
2510 changed files with 11237 additions and 531880 deletions
-16
View File
@@ -1,16 +0,0 @@
# Line-ending policy.
#
# Shell scripts MUST be checked out with LF endings on every platform.
# Git for Windows installs with `core.autocrlf=true` by default, which
# rewrites LF -> CRLF on checkout. A strict bash (WSL, Linux CI, macOS)
# then chokes on the trailing CR:
#
# scripts/run-unit-parallel.sh: line 23: $'\r': command not found
# scripts/run-unit-parallel.sh: line 24: set: pipefail : invalid option name
# scripts/run-unit-parallel.sh: line 32: syntax error near unexpected token `$'{\r''
#
# That silently disabled `bun run test`, `bun run verify`, `bun run ci:local`
# and `bun run test:e2e` for Windows contributors, since all four dispatch
# through bash. `eol=lf` pins the checkout regardless of the user's
# core.autocrlf setting.
*.sh text eol=lf
-32
View File
@@ -1,32 +0,0 @@
name: Actionlint
# Lints the GitHub Actions workflow YAML on every change so a malformed
# workflow / bad action ref / missing-permission bug is caught before it ships
# a broken pipeline. gbrain edits .github/workflows/* often (sharding, cache,
# timeouts); this is the cheap guard that keeps those edits honest.
on:
push:
branches: [master]
paths:
- '.github/workflows/**'
pull_request:
branches: [master]
paths:
- '.github/workflows/**'
workflow_dispatch:
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
actionlint:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: rhysd/actionlint@393031adb9afb225ee52ae2ccd7a5af5525e03e8 # v1.7.11
+7 -82
View File
@@ -12,61 +12,10 @@ on:
permissions:
contents: read
# Cancel a superseded run when a newer commit lands on the same PR/branch.
# PR number for pull_request events (fork-safe), github.ref fallback for
# push/scheduled runs.
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
jsonb-parity:
# Dedicated required guard for the JSONB double-encode bug-class (#2339).
# PGLite parses a double-encoded jsonb string silently, so this assertion can
# ONLY be made on real Postgres — a normal gated e2e file would skip without
# DATABASE_URL and let the bug ship green (as #2339 did). This job provisions
# Postgres and HARD-FAILS if DATABASE_URL is missing, so the guard can never
# silently skip.
name: JSONB parity (#2339 regression guard)
runs-on: ubuntu-latest
timeout-minutes: 15
services:
postgres:
image: pgvector/pgvector:pg16
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: gbrain_test
ports:
- 5432:5432
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.3.13
- run: bun install
- name: Require DATABASE_URL (no silent skip)
env:
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/gbrain_test
run: |
if [ -z "$DATABASE_URL" ]; then
echo "::error::DATABASE_URL must be set for the jsonb-parity job — the #2339 guard would silently skip (the exact failure PGLite hides). Failing the job." >&2
exit 1
fi
- name: Run JSONB double-encode parity tests on real Postgres
env:
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/gbrain_test
run: bun test test/e2e/op-checkpoint-jsonb-parity.test.ts test/e2e/jsonb-roundtrip.test.ts
tier1:
name: Tier 1 (Mechanical)
runs-on: ubuntu-latest
timeout-minutes: 20
services:
postgres:
image: pgvector/pgvector:pg16
@@ -82,10 +31,10 @@ jobs:
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.3.13
bun-version: latest
- run: bun install
- name: Run Tier 1 E2E tests
run: bun test test/e2e/mechanical.test.ts test/e2e/mcp.test.ts
@@ -95,12 +44,8 @@ jobs:
tier2:
name: Tier 2 (LLM Skills)
runs-on: ubuntu-latest
# Runs on every push/PR now (promoted from schedule-only in v0.19.0).
# Tier 1 must pass first; Tier 2 uses OPENAI_API_KEY + ANTHROPIC_API_KEY
# from repo/org secrets. Nightly + manual triggers still supported via
# the workflow-level `on:` list.
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
needs: tier1
timeout-minutes: 30
services:
postgres:
image: pgvector/pgvector:pg16
@@ -116,28 +61,13 @@ jobs:
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.3.13
bun-version: latest
- run: bun install
- name: Install OpenClaw
# Bound + retry the install: a transient npm/registry stall here used to
# hang unbounded and (since the v0.42.50.0 job timeout) burn the entire
# 30m Tier 2 budget before failing — even though the install normally
# finishes in well under a minute. `timeout` kills a hung attempt fast;
# up to 3 attempts ride out a flaky registry. Step cap is a backstop.
timeout-minutes: 8
run: |
for attempt in 1 2 3; do
if timeout 120 npm install -g openclaw@2026.4.9; then
exit 0
fi
echo "::warning::openclaw install attempt $attempt failed or timed out; retrying in 10s" >&2
sleep 10
done
echo "::error::openclaw install failed after 3 attempts" >&2
exit 1
run: npm install -g openclaw@2026.4.9
- name: Configure OpenClaw MCP
run: |
mkdir -p ~/.openclaw
@@ -155,13 +85,8 @@ jobs:
}
EOF
- name: Run Tier 2 skill tests
run: bun test test/e2e/skills.test.ts test/e2e/zeroentropy-live.test.ts
run: bun test test/e2e/skills.test.ts
env:
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/gbrain_test
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
# v0.33.3.0: ZE live API tests skip gracefully when this is unset,
# so forks without the secret stay green. The test exercises the
# zeroEntropyCompatFetch response-rewriter + URL rewrite + flexible
# dim handling + gateway.rerank against the real provider.
ZEROENTROPY_API_KEY: ${{ secrets.ZEROENTROPY_API_KEY }}
-89
View File
@@ -1,89 +0,0 @@
name: Heavy Tests
# Heavy ops-shape tests under tests/heavy/. Cost minutes per run; NOT part
# of default PR CI. Two triggers:
# - Nightly schedule (catches regressions within 24h of merge to master).
# - On-demand opt-in via PR label `heavy-tests` (slow loop kept off by default).
# - Manual workflow_dispatch for triage.
#
# See CLAUDE.md "tests/heavy/*.sh" entry and tests/heavy/README.md.
on:
schedule:
- cron: '17 8 * * *' # 08:17 UTC daily — staggered to avoid noisy slots
pull_request:
# `synchronize` + `reopened` fire on subsequent pushes / reopens — without
# them, a PR labeled `heavy-tests` would NEVER re-run heavy on later
# commits. The job-level `if:` below filters to PRs that still carry the
# label so we don't fan out on unrelated label changes.
types: [labeled, synchronize, reopened]
workflow_dispatch:
permissions:
contents: read
# When a PR gets the heavy-tests label, cancel any in-flight heavy-tests run on
# the same ref so we only ever measure the latest commit.
concurrency:
group: heavy-tests-${{ github.ref }}
cancel-in-progress: true
jobs:
heavy:
name: Heavy tests
# On pull_request: only run when the PR currently carries the `heavy-tests`
# label. Works for all three trigger types (labeled, synchronize, reopened)
# because `contains(labels.*.name, ...)` reads the live label set, not the
# event payload's `label.name` (which is only populated for `labeled`).
if: |
github.event_name != 'pull_request' ||
contains(github.event.pull_request.labels.*.name, 'heavy-tests')
runs-on: ubuntu-latest
timeout-minutes: 30
services:
postgres:
image: pgvector/pgvector:pg16
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: gbrain_test
ports:
- 5432:5432
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.3.13
- run: bun install
- name: Run heavy tests
env:
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/gbrain_test
run: bun run test:heavy
# The heavy runner writes per-script logs to ~/.gbrain/audit/ on every
# run. Upload those + the rss workload JSON on failure for triage
# without re-running locally.
#
# actions/upload-artifact runs as a node action — `~` is NOT expanded by
# the shell here. Stage logs into the workspace first, then upload from
# the stable workspace-relative path.
- name: Stage heavy-test logs into workspace
if: always()
run: |
mkdir -p heavy-artifacts
cp -r "$HOME/.gbrain/audit"/heavy-* heavy-artifacts/ 2>/dev/null || true
cp tests/heavy/rss-baseline.json heavy-artifacts/ 2>/dev/null || true
- name: Upload heavy-test artifacts
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: heavy-tests-${{ github.run_id }}-${{ github.run_attempt }}
path: heavy-artifacts/
retention-days: 14
if-no-files-found: ignore
-33
View File
@@ -1,33 +0,0 @@
name: OSV-Scanner
# Dependency vulnerability scan (#2182) via Google's official reusable
# workflow. Runs weekly and on any PR that touches the dependency manifests.
# Tokenless: needs zero secrets. Findings are reported in the job log and as
# a SARIF artifact on the run; code-scanning upload is deliberately disabled
# so the workflow stays read-only (no security-events: write).
on:
pull_request:
branches: [master]
paths:
- 'bun.lock'
- 'package.json'
schedule:
- cron: '30 6 * * 1' # weekly, Monday 06:30 UTC
workflow_dispatch:
permissions:
contents: read
jobs:
osv-scan:
permissions:
actions: read
contents: read
# Required by the reusable workflow's own top-level permissions block —
# GitHub validates the caller grants a superset AT STARTUP, even with
# upload-sarif: false (nothing is actually uploaded; see #2117 upstream).
security-events: write
uses: google/osv-scanner-action/.github/workflows/osv-scanner-reusable.yml@9a498708959aeaef5ef730655706c5a1df1edbc2 # v2.3.8
with:
upload-sarif: false
+3 -12
View File
@@ -19,23 +19,14 @@ jobs:
target: bun-linux-x64
artifact: gbrain-linux-x64
runs-on: ${{ matrix.os }}
permissions:
contents: read
id-token: write # for attest-build-provenance (Sigstore OIDC)
attestations: write # for attest-build-provenance
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.3.13
bun-version: latest
- run: bun install
- run: bun test
- run: bun run verify
- run: bun build --compile --target=${{ matrix.target }} --outfile bin/${{ matrix.artifact }} src/cli.ts
- name: Attest build provenance
uses: actions/attest-build-provenance@0f67c3f4856b2e3261c31976d6725780e5e4c373 # v4.1.1
with:
subject-path: bin/${{ matrix.artifact }}
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: ${{ matrix.artifact }}
@@ -49,7 +40,7 @@ jobs:
with:
path: artifacts
- name: Create release
uses: softprops/action-gh-release@3bb12739c298aeb8a4eeaf626c5b8d85266b0e65 # v2
uses: softprops/action-gh-release@153bb8e04406b158c6c84fc1615b65b24149a1fe # v2
with:
files: |
artifacts/gbrain-darwin-arm64/gbrain-darwin-arm64
-36
View File
@@ -1,36 +0,0 @@
name: Semgrep
# Static analysis (SAST) with Semgrep Community Edition (#2272). Tokenless:
# uses the public registry rulesets, needs zero secrets. Findings print in
# the job log; no code-scanning/SARIF upload by design (keeps permissions
# read-only, no security-events: write).
on:
pull_request:
branches: [master]
schedule:
- cron: '30 7 * * 1' # weekly, Monday 07:30 UTC
workflow_dispatch:
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
semgrep:
runs-on: ubuntu-latest
timeout-minutes: 20
container:
image: semgrep/semgrep:1.170.0@sha256:c98f8829eea377274ee4b10656458b078b88232469b2ff913f091c2317347c9d
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
# Non-blocking initially (continue-on-error): the first runs establish a
# baseline without failing unrelated PRs. Graduation path: once the
# baseline findings are triaged (fixed or `# nosemgrep`'d), remove
# continue-on-error so new findings block PRs.
- name: Semgrep scan (report-only)
run: semgrep scan --config p/default --config p/typescript --error
continue-on-error: true
+4 -263
View File
@@ -5,286 +5,27 @@ on:
branches: [master]
pull_request:
branches: [master]
# Manual dispatch lets a local dev/agent offload the suite to GitHub's
# on-demand runners from ANY branch (see scripts/ship-remote-tests.sh).
# Frees a load-saturated local machine (e.g. many Conductor agents running
# their own bun-test suites at once — load avg 120 on 16 cores).
workflow_dispatch:
permissions:
contents: read
# Cancel a superseded run when a newer commit lands on the same PR/branch.
# Keyed on the PR number for pull_request events (unique per PR, so two PRs
# from forks sharing a branch name don't cancel each other) and falls back to
# github.ref for push/scheduled runs. Mirrors heavy-tests.yml; frees runners
# and stops a stale-SHA run from reporting a flaky failure on an obsolete commit.
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
# ──────────────────────────────────────────────────────────────────────
# cache-check: runs first, computes the content hash of every tracked
# file EXCEPT the deny-list (CHANGELOG.md, README.md, docs/**/*.md, etc.
# — see scripts/ci-cache-hash.sh for the full list). Looks up
# `ci-pass-<hash>` in actions/cache; if hit, the test matrix + verify
# + serial jobs all skip and test-status reports green immediately.
# If miss, the full suite runs and cache-write seals it on success.
#
# Hit rate covers re-pushes (same SHA twice), branch rebases that
# don't touch tracked code, and any branch update that touches only
# the deny-listed doc files.
# ──────────────────────────────────────────────────────────────────────
cache-check:
runs-on: ubuntu-latest
timeout-minutes: 10
outputs:
hit: ${{ steps.lookup.outputs.cache-hit }}
hash: ${{ steps.compute.outputs.hash }}
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- name: Compute content hash
id: compute
run: |
# --verbose writes the "X/Y files in hash" diagnostic to stderr;
# stdout carries the 16-char hash. Capture both.
HASH=$(bash scripts/ci-cache-hash.sh --verbose 2>/tmp/cache-diag)
cat /tmp/cache-diag
echo "Computed cache hash: $HASH"
echo "hash=$HASH" >> "$GITHUB_OUTPUT"
- name: Lookup actions/cache for ci-pass-<hash>
id: lookup
uses: actions/cache/restore@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3
with:
key: ci-pass-${{ steps.compute.outputs.hash }}
path: .ci-cache-marker
# `lookup-only: true` means we only probe whether the cache
# entry exists — we don't download it (the marker contents
# don't matter, only the key match does). `cache-hit` returns
# true only on EXACT key match (per actions/cache docs); a
# restore-keys prefix fallback would set cache-hit=false, so
# it's deliberately omitted here. Cross-branch scoping works
# naturally: PR branches can read default-branch (master)
# cache entries via exact key match when the content hash
# matches, which happens whenever the tree is doc-only
# different from a green master run.
lookup-only: true
- name: Cache status
run: |
if [ "${{ steps.lookup.outputs.cache-hit }}" = "true" ]; then
echo "✓ cache HIT for hash ${{ steps.compute.outputs.hash }} — test jobs will skip"
else
echo "✗ cache MISS for hash ${{ steps.compute.outputs.hash }} — full suite will run"
fi
gitleaks:
needs: cache-check
if: needs.cache-check.outputs.hit != 'true'
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
fetch-depth: 0
- uses: gitleaks/gitleaks-action@dcedce43c6f43de0b836d1fe38946645c9c638dc # v2
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
verify:
# Pre-test gates: privacy/jsonb/source-id/etc + typecheck + admin-build.
# Lives in its own runner so the matrix shards aren't carrying ~2-3min
# of verify work in addition to their test files (the old shape stuffed
# this into `test (1)` via `if: matrix.shard == 1`, which made shard 1
# the slowest matrix worker). scripts/run-verify-parallel.sh fans out
# the 20 checks via & + wait (~5s vs ~15-25s sequential).
needs: cache-check
if: needs.cache-check.outputs.hit != 'true'
runs-on: ubuntu-latest
timeout-minutes: 12
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.3.13
- uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3
with:
path: ~/.bun/install/cache
key: bun-cache-${{ runner.os }}-${{ hashFiles('bun.lock') }}
- run: bun install
- run: bun run verify
serial-tests:
# *.serial.test.ts at --max-concurrency=1. Lives in its own runner so
# the matrix shards aren't carrying the serial-pass tail (the old shape
# stuffed this into `test (1)` after the matrix work, which compounded
# shard 1's overload).
needs: cache-check
if: needs.cache-check.outputs.hit != 'true'
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.3.13
- uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3
with:
path: ~/.bun/install/cache
key: bun-cache-${{ runner.os }}-${{ hashFiles('bun.lock') }}
- run: bun install
- run: bun run test:serial
slow-eval-longmemeval:
# Dedicated runner for the LongMemEval end-to-end test file. The file
# was originally 359s. TODO #1 (engine-sharing in runEvalLongMemEval
# via RunOpts.engine) cut it to ~200s by amortizing PGLite cold-create
# across all 13 runEvalLongMemEval calls in one beforeAll-shared brain.
# Pulled out of the matrix (see scripts/test-shard.sh) so a single 200s
# atom doesn't dominate a shard's wallclock. Companion file
# test/eval-longmemeval.slow.test.ts (the pure-bucket half) stays in
# the matrix because it's light (~42s).
needs: cache-check
if: needs.cache-check.outputs.hit != 'true'
runs-on: ubuntu-latest
timeout-minutes: 12
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.3.13
- uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3
with:
path: ~/.bun/install/cache
key: bun-cache-${{ runner.os }}-${{ hashFiles('bun.lock') }}
- run: bun install
- run: bun test test/eval-longmemeval-e2e.slow.test.ts --timeout=60000
slow-entity-resolve-perf:
# Dedicated runner for the entity-resolve perf test (~159s, single perf
# describe with one test that builds 5000+ pages and asserts the NEW
# tryPrefixExpansion shape is 5x faster than the OLD shape — not
# subdivisible without weakening the perf guarantee). Pulled out of the
# matrix (see scripts/test-shard.sh) so a single 159s atom doesn't
# dominate a shard's wallclock. Runs in parallel with the matrix.
needs: cache-check
if: needs.cache-check.outputs.hit != 'true'
runs-on: ubuntu-latest
timeout-minutes: 12
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.3.13
- uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3
with:
path: ~/.bun/install/cache
key: bun-cache-${{ runner.os }}-${{ hashFiles('bun.lock') }}
- run: bun install
- run: bun test test/entity-resolve-perf.slow.test.ts --timeout=300000
test:
# Pure matrix shard — no verify, no serial. Each shard runs its slice
# of the unit test set under one `bun test` invocation.
#
# 10 shards (was 6) drops per-shard total from 532s → 287s. With the two
# dedicated jobs (slow-eval-longmemeval, slow-entity-resolve-perf) also
# pulled out, the matrix is bounded by ~287s ≈ 4.8 min. Total CI ≈ max
# of matrix + slow-eval (~3.3 min after engine-sharing in TODO #1) +
# slow-entity-resolve-perf (~2.6 min) ≈ 4.8 min.
#
# Concurrency budget: 10 shards + verify + serial + slow-eval +
# slow-entity-resolve-perf + gitleaks + cache-check + cache-write +
# test-status = ~18 jobs × 2 concurrent PRs = 36. GitHub free-tier
# caps at ~20 concurrent jobs, so multi-PR days will see some queue
# pressure. Single-PR runs are unaffected.
#
# Partition policy is weight-aware LPT bin-packing via scripts/sharding.ts
# (replaces FNV-1a path hash). Weights live in scripts/test-weights.json,
# mined from real CI logs via scripts/mine-shard-weights.ts. Missing
# weights fall back to corpus median — new test files work immediately.
needs: cache-check
if: needs.cache-check.outputs.hit != 'true'
runs-on: ubuntu-latest
# 22, not 15: under parallel PR load the PGLite WASM cold-starts stretch a
# shard past 15 min while every test is still passing — the timeout then
# cancels the job and the test-status gate reads it as a failure. 13 runs
# died this way on 2026-07-21/22 alone.
timeout-minutes: 22
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.3.13
- uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3
with:
path: ~/.bun/install/cache
key: bun-cache-${{ runner.os }}-${{ hashFiles('bun.lock') }}
bun-version: latest
- run: bun install
- name: Run test shard ${{ matrix.shard }}/10
run: scripts/test-shard.sh ${{ matrix.shard }} 10
# ──────────────────────────────────────────────────────────────────────
# cache-write: ONLY runs when every gated job succeeded. Writes the
# cache entry under `ci-pass-<hash>` so future runs at the same hash
# hit cache. Codex's load-bearing correctness point: writing the
# cache before the matrix completes would permanently bless bad states
# (a future run at the same hash would skip tests because of a cache
# entry written when tests hadn't actually passed).
# ──────────────────────────────────────────────────────────────────────
cache-write:
needs: [cache-check, gitleaks, verify, serial-tests, slow-eval-longmemeval, slow-entity-resolve-perf, test]
if: success() && needs.cache-check.outputs.hit != 'true'
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Create cache marker
run: |
mkdir -p .ci-cache-marker
echo "${{ needs.cache-check.outputs.hash }}" > .ci-cache-marker/hash
echo "$GITHUB_SHA" > .ci-cache-marker/sha
echo "$GITHUB_REF" > .ci-cache-marker/ref
- uses: actions/cache/save@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3
with:
key: ci-pass-${{ needs.cache-check.outputs.hash }}
path: .ci-cache-marker
# ──────────────────────────────────────────────────────────────────────
# test-status: the single user-visible "did CI pass?" check.
# Runs always (if: always()), succeeds when EITHER cache-check.hit==true
# OR all gated jobs (gitleaks, verify, serial-tests, test) succeeded.
# Branch protection (when configured) gates on this single job name.
# ──────────────────────────────────────────────────────────────────────
test-status:
needs: [cache-check, gitleaks, verify, serial-tests, slow-eval-longmemeval, slow-entity-resolve-perf, test]
if: always()
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Aggregate result
run: |
HIT="${{ needs.cache-check.outputs.hit }}"
GITLEAKS="${{ needs.gitleaks.result }}"
VERIFY="${{ needs.verify.result }}"
SERIAL="${{ needs.serial-tests.result }}"
SLOW_EVAL="${{ needs.slow-eval-longmemeval.result }}"
SLOW_PERF="${{ needs.slow-entity-resolve-perf.result }}"
TEST="${{ needs.test.result }}"
echo "cache-check.hit=$HIT"
echo "gitleaks=$GITLEAKS verify=$VERIFY serial-tests=$SERIAL slow-eval-longmemeval=$SLOW_EVAL slow-entity-resolve-perf=$SLOW_PERF test=$TEST"
if [ "$HIT" = "true" ]; then
echo "✓ cache HIT for hash ${{ needs.cache-check.outputs.hash }} — CI green"
exit 0
fi
# Cache miss: every gated job must have succeeded.
for r in "$GITLEAKS" "$VERIFY" "$SERIAL" "$SLOW_EVAL" "$SLOW_PERF" "$TEST"; do
if [ "$r" != "success" ]; then
echo "✗ gated job did not succeed (got $r) — CI fail"
exit 1
fi
done
echo "✓ all gated jobs succeeded — CI green"
- run: bun run test
+1 -33
View File
@@ -1,7 +1,4 @@
# No trailing slash: a bare `node_modules/` pattern matches directories only,
# so a *symlink* named node_modules slips past it and can be committed
# (that's how the /tmp-pointing symlink in faf5cdba got in). Match any type.
node_modules
node_modules/
bin/
.DS_Store
*.log
@@ -14,38 +11,9 @@ bin/
.gstack/
supabase/.temp/
.claude/skills/
# admin/dist/ is the React SPA bundle. CLAUDE.md says it's committed for
# self-contained binaries (the bun --compile path embeds it via
# `import path from 'admin/dist/index.html' with { type: 'file' }`).
# Build via: cd admin && bun install && bun run build.
admin/node_modules
.idea
eval/reports/
eval/data/world-v1/world.html
# BrainBench amara-life-v1 Opus cache (regenerate via eval:generate-amara-life)
eval/data/amara-life-v1/_cache/
# claw-test E2E build cache (shim + scratch outputs)
test/.cache/
.claude/
export/
# Conductor workspace-local agent artifacts: plans, todos, run-unit-parallel
# failure logs and per-shard test output. v0.26.4 (run-unit-parallel.sh)
# writes .context/test-failures.log + .context/test-summary.txt +
# .context/test-shards/. Workspace-local by design — never committed.
.context/
# Local agent instruction overrides (CLAUDE.local.md / AGENTS.local.md) — personal,
# per-clone, loaded after the committed CLAUDE.md/AGENTS.md. Never committed.
CLAUDE.local.md
AGENTS.local.md
# Tier 3 PGLite snapshot fixture (built on demand by build:pglite-snapshot)
test/fixtures/pglite-snapshot.tar
test/fixtures/pglite-snapshot.version
# Private brain reports — never check these in (per CLAUDE.md privacy rule)
reports/network-intelligence/
+11 -81
View File
@@ -6,45 +6,19 @@ start here.
## Install (5 min)
1. Install gbrain via Bun (the canonical path):
```bash
curl -fsSL https://bun.sh/install | bash
export PATH="$HOME/.bun/bin:$PATH"
bun install -g github:garrytan/gbrain
```
If `bun install -g` aborts or `gbrain doctor` reports `schema_version: 0`,
the CLI prints a recovery hint pointing at [#218](https://github.com/garrytan/gbrain/issues/218).
Run `gbrain apply-migrations --yes` to recover, or fall back to the
deterministic install: `git clone https://github.com/garrytan/gbrain.git ~/gbrain && cd ~/gbrain && bun install && bun link`.
2. Init the brain: `gbrain init` (defaults to PGLite, zero-config). For 1000+ files or
1. Clone: `git clone https://github.com/garrytan/gbrain ~/gbrain && cd ~/gbrain`
2. Install: `bun install`
3. Init the brain: `gbrain init` (defaults to PGLite, zero-config). For 1000+ files or
multi-machine sync, init suggests Postgres + pgvector via Supabase.
3. **STOP — ask the user about search mode.** `gbrain init` auto-applied a
default but printed a 9-cell cost matrix (mode × downstream model)
preceded by `[AGENT]` markers. You MUST relay the matrix to the operator
and confirm their choice before continuing. Cost spread between corners
is 25x — silent acceptance is the wrong default. See
[`./INSTALL_FOR_AGENTS.md`](./INSTALL_FOR_AGENTS.md) Step 3.5 for the
exact ask-the-user protocol. Same banner fires on `gbrain post-upgrade`
for existing users (search modes were added in v0.32.3).
4. Read [`./INSTALL_FOR_AGENTS.md`](./INSTALL_FOR_AGENTS.md) for the full 9-step flow
(API keys, identity, cron, verification).
## Read this order
1. `./AGENTS.md` (this file) — install + operating protocol.
2. [`./CLAUDE.md`](./CLAUDE.md) — orientation + resolver: architecture, cross-cutting
invariants, the reference map, inline ship rules. It routes to on-demand detail docs:
[`./docs/architecture/KEY_FILES.md`](./docs/architecture/KEY_FILES.md) (per-file index —
read a file's entry before editing it), [`./docs/TESTING.md`](./docs/TESTING.md) (test
tiers + isolation lint + E2E lifecycle), and
[`./docs/architecture/thin-client.md`](./docs/architecture/thin-client.md) (remote-MCP seam).
3. [`./docs/architecture/brains-and-sources.md`](./docs/architecture/brains-and-sources.md)
— the two-axis mental model (brain = which DB, source = which repo in the DB). Every
query routes on both axes. Read before writing anything that touches brain ops.
4. [`./skills/conventions/brain-routing.md`](./skills/conventions/brain-routing.md) —
agent-facing decision table: when to switch brain, when to switch source, how
cross-brain federation works (latent-space only; the agent decides).
5. [`./skills/RESOLVER.md`](./skills/RESOLVER.md) — skill dispatcher. Read before any task.
2. [`./CLAUDE.md`](./CLAUDE.md) — architecture reference, key files, trust boundaries,
test layout.
3. [`./skills/RESOLVER.md`](./skills/RESOLVER.md) — skill dispatcher. Read before any task.
## Trust boundary (critical)
@@ -61,61 +35,17 @@ writing or reviewing an operation, consult `src/core/operations.ts` for the cont
[`docs/mcp/DEPLOY.md`](./docs/mcp/DEPLOY.md).
- **Debug:** [`docs/GBRAIN_VERIFY.md`](./docs/GBRAIN_VERIFY.md),
[`docs/guides/minions-fix.md`](./docs/guides/minions-fix.md), `gbrain doctor --fix`.
- **Migrate / upgrade:** `gbrain upgrade` (binary self-update + schema migrations + post-upgrade prompts),
[`docs/UPGRADING_DOWNSTREAM_AGENTS.md`](./docs/UPGRADING_DOWNSTREAM_AGENTS.md),
[`skills/migrations/`](./skills/migrations/), `gbrain apply-migrations --yes` (manual schema-only).
- **Eval retrieval changes:** capture is off by default. To benchmark a
retrieval change against real captured queries, set
`GBRAIN_CONTRIBUTOR_MODE=1`, then `gbrain eval export --since 7d > base.ndjson`
and `gbrain eval replay --against base.ndjson`. For public benchmark
coverage (LongMemEval, ground-truth scoring), `gbrain eval longmemeval
<dataset.jsonl>` (v0.28.8) runs against an isolated in-memory PGLite
per question — your `~/.gbrain` is never opened. Full guide:
[`docs/eval-bench.md`](./docs/eval-bench.md).
- **Drive the brain to a target health score (v0.36.4.0):** the one-command
loop. `gbrain doctor --remediation-plan --json` previews what would be
fixed; `gbrain doctor --remediate --yes --target-score 90 --max-usd 5`
walks a dependency-ordered plan (sync before extract, embed after
consolidate), re-checking score between every step, refusing to spend
past the cost cap. Empty brains (no entity pages) or unconfigured embedding
keys hit a `max_reachable_score` ceiling and bail with what's missing.
Three phase handlers (synthesize / patterns / consolidate) are
PROTECTED — only trusted local callers can submit them; MCP cannot.
Reference: [`docs/architecture/topologies.md`](./docs/architecture/topologies.md)
and the CHANGELOG entry for v0.36.4.0.
- **Track a founder/company over time (v0.35.7):** when an entity has
typed metric claims in its `## Facts` fence (`metric: mrr`, `value: 50000`,
`unit: USD`, `period: monthly` columns), run
`gbrain eval trajectory <entity-slug>` for the chronological history
with regressions auto-flagged, or `gbrain founder scorecard <entity-slug>`
for a four-signal JSON rollup (claim_accuracy / consistency /
growth_trajectory / red_flags). MCP op `find_trajectory` exposes the
same data — read scope, visibility-filtered for remote callers. **v0.40.2.0:**
`gbrain think` now uses this substrate automatically on temporal /
knowledge_update intent (default ON; flip `think.trajectory_enabled=false`
to opt out). Migration v82 added `facts.event_type` so non-metric event
rows (`meeting`, `job_change`, `location_change`) ride through the same
pipeline; pass `kind: 'event'` or `'all'` to `find_trajectory` to query
them.
- **Migrate:** [`docs/UPGRADING_DOWNSTREAM_AGENTS.md`](./docs/UPGRADING_DOWNSTREAM_AGENTS.md),
[`skills/migrations/`](./skills/migrations/), `gbrain apply-migrations`.
- **Everything else:** [`./llms.txt`](./llms.txt) is the full documentation map.
[`./llms-full.txt`](./llms-full.txt) is the same map with core docs inlined for
single-fetch ingestion.
## Before shipping
Easiest path: `bun run ci:local` runs the full CI gate inside Docker (gitleaks,
guards + typecheck, then 4-shard parallel unit + E2E against four pgvector
containers plus a transaction-mode PgBouncer; unit phase keeps `DATABASE_URL`
unset) and tears down. Use `bun run ci:local:diff` for the
diff-aware subset during fast iteration on a focused branch. Requires Docker
(Docker Desktop / OrbStack / Colima) and `gitleaks` (`brew install gitleaks`).
Manual path: `bun test` plus the E2E lifecycle described in `./CLAUDE.md` (spin
up the test Postgres container, run `bun run test:e2e`, tear it down).
Ship via the `/ship` skill, not by hand. The full release + contributor process
(CHANGELOG voice, version-locations sync, PR conventions, community-PR-wave) lives in
[`./docs/RELEASING.md`](./docs/RELEASING.md); read it before shipping.
Run `bun test` plus the E2E lifecycle described in `./CLAUDE.md` (spin up the test
Postgres container, run `bun run test:e2e`, tear it down). Ship via the `/ship` skill,
not by hand.
## Privacy
+19 -19398
View File
File diff suppressed because it is too large Load Diff
+580 -520
View File
File diff suppressed because it is too large Load Diff
+3 -221
View File
@@ -11,28 +11,6 @@ bun test
Requires Bun 1.0+.
### Windows
`bun run test`, `verify`, `ci:local` and `test:e2e` all dispatch through bash, so
the shell scripts under `scripts/` must be checked out with Unix line endings.
The root `.gitattributes` pins `*.sh text eol=lf`, which overrides the
`core.autocrlf=true` that Git for Windows installs by default. A fresh clone is
correct with no extra steps.
If you cloned before that pin existed, your working copy still has the old
Windows line endings and bash will fail with `$'\r': command not found`. Refresh
it once, from the repository root:
```bash
git rm --cached -r . -q
git reset --hard
bash -n scripts/run-unit-parallel.sh # silence means bash can read the scripts
```
Every `check:*` entry in `package.json` invokes its script as `bash scripts/<name>.sh`
rather than relying on the shebang, because bun on Windows cannot exec a `.sh`
directly. Keep that prefix when you add a new shell-script check.
## Project structure
```
@@ -74,22 +52,10 @@ docs/ Architecture docs
## Running tests
```bash
# Inner edit loop (~85s on a Mac dev box, 3700+ unit tests)
bun run test # parallel 8-shard fan-out + serial post-pass
bun test # all tests (unit + E2E skipped without DB)
bun test test/markdown.test.ts # specific unit test
# Pre-push gate (matches what CI runs on shard 1 + typecheck)
bun run verify # privacy + jsonb + progress + test-isolation + wasm + admin-build + resolver + typecheck
# Pre-merge sanity (everything CI runs)
bun run test:full # verify + parallel unit + slow + smart e2e
# Slow / serial / e2e in isolation
bun run test:slow # *.slow.test.ts only (cold-path correctness)
bun run test:serial # *.serial.test.ts only (--max-concurrency=1)
bun run test:e2e # real-Postgres E2E (requires DATABASE_URL)
# E2E setup (Postgres with pgvector)
# E2E tests (requires Postgres with pgvector)
docker compose -f docker-compose.test.yml up -d
DATABASE_URL=postgresql://postgres:postgres@localhost:5434/gbrain_test bun run test:e2e
@@ -97,102 +63,6 @@ DATABASE_URL=postgresql://postgres:postgres@localhost:5434/gbrain_test bun run t
DATABASE_URL=postgresql://... bun run test:e2e
```
Use `bun run verify` before pushing. The guard chain catches: banned fork-name
leaks (`scripts/check-privacy.sh`), `JSON.stringify(x)::jsonb` interpolation
patterns (`scripts/check-jsonb-pattern.sh`), `\r` progress bleed to stdout
(`scripts/check-progress-to-stdout.sh`), test-isolation rule violations
(`scripts/check-test-isolation.sh` — see "Writing tests that survive the parallel
loop" below), silent fallback to recursive chunking in the compiled binary
(`scripts/check-wasm-embedded.sh`), stale admin-dashboard build artifacts
(`scripts/check-admin-build.sh`), and resolver drift on bundled skills
(`bun run check:resolver` — strict-mode `check-resolvable` that exit-1s on any
warning, added in v0.41.14.0 to catch SKILL.md frontmatter ↔ RESOLVER.md drift
before merge). `bun run check:all` runs the full historical sweep including the
trailing-newline and exports-count checks.
### Writing tests that survive the parallel loop
`bun run test` shards 92+ unit-test files across 8 worker processes. Files in the
same shard share a process, so process-global state leaks between them. Four
lint rules (`scripts/check-test-isolation.sh`, R1-R4) enforce isolation:
| Rule | What it bans | Fix |
|---|---|---|
| **R1** | Direct `process.env.X = ...` mutation | Use `withEnv()` from `test/helpers/with-env.ts`, or rename to `*.serial.test.ts` |
| **R2** | `mock.module(...)` anywhere in the file | Rename to `*.serial.test.ts` |
| **R3** | `new PGLiteEngine(` outside ~50 lines after `beforeAll(` | Use the canonical PGLite block (see below) |
| **R4** | `new PGLiteEngine(` without paired `afterAll(disconnect)` | Add the `afterAll(() => engine.disconnect())` |
Canonical PGLite block (R3 + R4 compliant — paste this verbatim):
```ts
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { resetPgliteState } from './helpers/reset-pglite.ts';
let engine: PGLiteEngine;
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
});
afterAll(async () => { await engine.disconnect(); });
beforeEach(async () => { await resetPgliteState(engine); });
```
Env-touching tests:
```ts
import { withEnv } from './helpers/with-env.ts';
test('reads OPENAI_API_KEY', async () => {
await withEnv({ OPENAI_API_KEY: 'sk-test' }, async () => {
expect(loadConfig().openai_key).toBe('sk-test');
});
});
```
`withEnv` saves and restores keys via try/finally including when the callback
throws. Cross-test safe; **NOT** intra-file concurrent-safe (`process.env` is
process-global). Files using `withEnv` stay outside the future
`test.concurrent()` codemod's eligibility filter.
When to quarantine instead of fix: rename to `*.serial.test.ts` if the file
uses `mock.module(...)`, is genuinely env-coupled (module-load env readers +
ESM caching defeat dynamic-import-after-env tricks), or intentionally shares
state across `it()` boundaries. Quarantine count cap: 10 (informational).
Files that violated these rules at the v0.26.7 baseline are listed in
`scripts/check-test-isolation.allowlist`. **The allow-list MUST shrink over
time** ... never add new entries. v0.26.8 (env sweep) and v0.26.9 (PGLite sweep
+ codemod) remove entries as files get fixed.
### Local CI gate (recommended before pushing, v0.23.1+)
```bash
bun run ci:local # full gate: gitleaks + unit + ALL 29 E2E files (sequential)
bun run ci:local:diff # gate with diff-aware E2E selector
bun run ci:select-e2e # print which E2E files the selector would run
```
`ci:local` spins up `pgvector/pgvector:pg16` + `oven/bun:1` via
`docker-compose.ci.yml`, runs everything PR CI runs plus the full E2E suite, then
tears down. Named volumes keep the install warm across runs (~16-20 min sequential
E2E after the first cold pull). Requires Docker (Docker Desktop, OrbStack, or
Colima) and `gitleaks` on host (`brew install gitleaks`). Override the postgres
host port with `GBRAIN_CI_PG_PORT=5435 bun run ci:local` if 5434 collides.
Fail-closed selector: an unmapped `src/` change runs all 29 E2E files. Hand-tune
narrower mappings via `scripts/e2e-test-map.ts`.
### PR-side security checks
Besides the test gate, PRs may trigger three security workflows: Semgrep CE
SAST (every PR — **advisory/non-blocking** while the baseline is tuned, so a
Semgrep finding won't fail your PR), OSV-Scanner (only when `package.json` or
`bun.lock` change), and actionlint (only when `.github/workflows/**` change).
See `SECURITY.md` → "Automated security scanning" for details.
## Building
```bash
@@ -223,95 +93,7 @@ See `docs/ENGINES.md` for the full guide. In short:
3. Run the test suite against your engine
4. Document in `docs/`
The original SQLite engine plan was superseded by PGLite (embedded Postgres 17 via WASM), which uses the same SQL dialect as Postgres and eliminates the need for a separate FTS5/sqlite-vss translation layer. See [`docs/ENGINES.md`](docs/ENGINES.md) for the engine architecture and the rationale.
## CONTRIBUTOR_MODE — turn on the dev loop
gbrain captures retrieval traffic so you can replay real queries against
your code changes before merging. **This is off by default** (production
users get a quiet brain, no surprise data accumulation). Contributors turn
it on with one shell rc line:
```bash
# In ~/.zshrc or ~/.bashrc:
export GBRAIN_CONTRIBUTOR_MODE=1
```
That's it. Every `query` / `search` you (or agents pointed at your dev
brain) run from that shell now writes a row to `eval_candidates`, and the
[replay tool](#running-real-world-eval-benchmarks-touching-retrieval-code)
has data to work against.
What CONTRIBUTOR_MODE actually does:
- Turns on `query`/`search` capture into the local `eval_candidates` table.
Without it the gate is closed and capture is a no-op.
- That's all. PII scrubbing, retention, and replay are independent.
Resolution order (most explicit wins):
1. `eval.capture: true` in `~/.gbrain/config.json` → on
2. `eval.capture: false` in `~/.gbrain/config.json` → off
3. `GBRAIN_CONTRIBUTOR_MODE=1` → on
4. otherwise → off
Quick check that capture is actually running:
```bash
gbrain query "anything" >/dev/null
psql $DATABASE_URL -c 'SELECT count(*) FROM eval_candidates'
# (or `gbrain doctor` — surfaces silent capture failures cross-process)
```
To disable capture even with the env var set, write
`{"eval": {"capture": false}}` to `~/.gbrain/config.json` — explicit config
beats the env var both directions.
## Running real-world eval benchmarks (touching retrieval code)
If your PR touches retrieval — search ranking, RRF fusion, embeddings,
intent classification, query expansion, source boost, or the `query` /
`search` op handlers — run `gbrain eval replay` against a snapshot of
real traffic before merging. Requires `CONTRIBUTOR_MODE` (above) so you
have captured rows to replay against.
Quick loop:
```bash
gbrain eval export --since 7d > baseline.ndjson # snapshot before your change
# ... make your change ...
gbrain eval replay --against baseline.ndjson # diff retrieval, get Jaccard@k
```
Three numbers come back: mean Jaccard@k between captured and current slug
sets, top-1 stability, and mean latency Δ. The replay tool flags the worst
regressions so you can eyeball whether the change is hurting real queries.
Trigger paths (rerun if your diff touches any of these):
- `src/core/search/hybrid.ts`
- `src/core/search/source-boost.ts`, `sql-ranking.ts`
- `src/core/search/intent.ts`, `expansion.ts`, `dedup.ts`
- `src/core/embedding.ts`
- `src/core/operations.ts` (query / search handlers)
- `src/core/postgres-engine.ts` / `pglite-engine.ts` (searchKeyword /
searchVector SQL)
See [`docs/eval-bench.md`](./docs/eval-bench.md) for the full guide
including CI integration, hand-crafted NDJSON corpora (so a fresh checkout
without captured data can still replay), and cost considerations. The
NDJSON wire format is documented in
[`docs/eval-capture.md`](./docs/eval-capture.md).
For public benchmark coverage on top of replay, `gbrain eval longmemeval
<dataset.jsonl>` (v0.28.1) runs LongMemEval against gbrain's hybrid
retrieval. One in-memory PGLite per question, runtime-enumerated
`TRUNCATE` between questions, ground-truth scoring via LongMemEval's
published `evaluate_qa.py`. Use it alongside replay when changes affect
retrieval quality on long-context conversational data — replay catches
regressions on YOUR queries, LongMemEval catches them on a public set the
benchmark community already cites. See the "Public benchmarks: LongMemEval"
section in [`docs/eval-bench.md`](./docs/eval-bench.md).
The SQLite engine is designed and ready for implementation. See `docs/SQLITE_ENGINE.md`.
## Welcome PRs
-148
View File
@@ -1,148 +0,0 @@
# DESIGN.md
The design system source of truth for gbrain. Born from the de facto tokens
that landed in `admin/src/index.css` during the v0.26.0 admin SPA work and
formalized during the v0.36.1.0 Hindsight calibration wave's design review.
This doc is the calibration target for `/plan-design-review` and `/design-review`.
When a question is "does this UI fit the system?", the answer is here.
## Voice
GBrain talks like a smart friend who knows your past, not a clinical scoring
system. Every user-facing string passes through this filter:
- Second person, contractions allowed.
- Grounded in concrete data the user can verify ("2 of 3 missed" beats
"Brier 0.31").
- Never preachy. Never "we recommend." Never "according to your data."
- Short. Under 25 words for narrative; under one line for status.
- Numbers grounded in real outcomes, never abstract metrics without
translation.
Five surfaces use this voice (v0.36.1.0+):
`pattern_statement`, `nudge`, `forecast_blurb`, `dashboard_caption`,
`morning_pulse`. All five pass through `gateVoice()` in
`src/core/calibration/voice-gate.ts` with mode-specific rubrics. A Haiku
judge rejects academic-sounding candidates; up to 2 regens; then fall
back to a hand-written template from `src/core/calibration/templates.ts`.
## Color tokens
CSS variables in `admin/src/index.css`. SVG renderer inlines literals
matching these tokens (`src/core/calibration/svg-renderer.ts`).
| Token | Value | Use |
|--------------------|-----------|-------------------------------------------|
| `--bg-primary` | `#0a0a0f` | Page background |
| `--bg-secondary` | `#14141f` | Sidebar, cards |
| `--bg-tertiary` | `#1e1e2e` | Subtle surfaces, borders |
| `--text-primary` | `#e0e0e0` | Body text |
| `--text-secondary` | `#888` | Headings, labels |
| `--text-muted` | `#777` | Tertiary text — TD2 bumped from #555 for WCAG AA contrast (~5.5:1) |
| `--accent` | `#3b82f6` | Active states, links, primary CTAs |
| `--success` | `#22c55e` | Healthy / ok status |
| `--warning` | `#f59e0b` | Doctor warnings |
| `--error` | `#ef4444` | Failures, destructive confirmations |
Dark theme is the only theme. No light mode toggle planned — admin is an
operator tool, not a marketing surface. Users live in the terminal with a
dark theme already.
WCAG contrast:
- Body text (#e0e0e0 on #0a0a0f) → ~14:1, AAA
- Muted text (#777 on #0a0a0f) → ~5.5:1, AA (was 4.0 / fail before TD2)
- Accent links (#3b82f6 on #0a0a0f) → ~5.7:1, AA
## Typography
| Variable | Value | Use |
|--------------------|-----------------------------|---------------------------------|
| `--font-sans` | `Inter, system-ui, sans-serif` | UI text, headings, body |
| `--font-mono` | `JetBrains Mono, monospace` | Numbers, slugs, code, terminal-ish data |
Type scale (de facto, not formalized yet):
- 18px: sidebar logo / page title
- 14px: body
- 13px: nav items
- 12px: chart captions, secondary labels
- 11px: tertiary labels in dense charts
Numbers in tables and metrics use JetBrains Mono so column alignment is
mechanical. Avoid mixing Inter and JetBrains Mono in the same line.
## Spacing scale
4 / 8 / 16 / 24 / 32px. Linear-app-style density: 24-32px between major
sections, 16px between row groups, 8px within a row. The Calibration tab
(approved variant-B mockup) is the canonical example.
## Layout
- Sidebar 200px on the left. Active item gets a 3px left-border in `--accent`.
- Main content area uses the remaining width.
- Max content width: 720px for text-heavy pages (Calibration), 960px for
data tables (Request Log).
- No 3-column feature grids. No icons in colored circles. No decorative blobs.
- Cards earn their existence — heading + content works without a card frame
in most cases.
## Charts
Server-rendered SVG via `src/core/calibration/svg-renderer.ts`. Pure
functions: data → SVG string. No DOM, no React component, no chart library.
XSS posture: server-side `escapeXml()` on every caller-controlled string.
Numeric inputs `.toFixed()`-coerced. Admin SPA renders via
`<TrustedSVG>` wrapper with `dangerouslySetInnerHTML`. Endpoint gated by
`requireAdmin` middleware.
Why server-rendered SVG (per D23):
- Chart logic stays close to the data math.
- Zero new client-side chart-library dep.
- SVG is accessible (text labels), scalable, copy-paste-friendly to PR
descriptions and docs.
- Sets the precedent for future admin charts (contradictions trend, takes
scorecard, etc.).
Four chart renderers in v0.36.1.0:
- `renderBrierTrend({ series })` — sparkline + baseline reference at 0.25
- `renderDomainBars({ bars })` — horizontal accuracy bars
- `renderAbandonedThreadsCard(threads)` — text rows + "revisit now" links
- `renderPatternStatementsCard(statements)` — clickable drill-down anchors
## Interaction patterns
- Keyboard navigation is REQUIRED for all CLI interaction surfaces. The
propose-queue review uses J/K/space/u/q shortcuts (gmail-style).
- Loading states: "Loading...". Don't show spinners on sub-200ms operations.
- Empty states ARE features: warmth + primary action + context. Cold-brain
Calibration page tells the user EXACTLY how to build a profile, not
"no data available."
- Error states: name what failed + name the next step. Never "an error
occurred — please try again."
## What's NOT here yet (v0.37+ roadmap)
- Type scale formalization (current values are de facto, not enforced)
- Animation tokens (admin SPA has zero animations on purpose; v0.37 may
add subtle progress / loading transitions)
- Print stylesheet
- Light mode (NOT planned — see "Dark theme is the only theme" above)
- Component library extraction (the React components live inline in admin/src/pages/;
no `<Button>` / `<Card>` abstraction layer yet)
## How to use this document
When adding a new UI surface to gbrain:
1. Pick existing tokens before introducing new ones. New tokens go through
`/plan-design-review`.
2. Match the voice rules. Run candidates through `gateVoice()` before
shipping any user-facing string in the calibration surfaces.
3. Match the spacing scale and density. Linear-calm-clarity over
dashboard-card-mosaic.
4. Match the typography: Inter for UI, JetBrains Mono for numbers.
When updating this document: it's a living target, not a frozen spec.
Major changes go through `/plan-design-review` to keep the system coherent.
+19 -203
View File
@@ -16,43 +16,32 @@ If you fetched this file by URL without cloning yet, the companion files live at
## Step 1: Install GBrain
Default path (Bun is required — gbrain is a Bun + TypeScript runtime):
```bash
git clone https://github.com/garrytan/gbrain.git ~/gbrain && cd ~/gbrain
curl -fsSL https://bun.sh/install | bash
export PATH="$HOME/.bun/bin:$PATH"
bun install -g github:garrytan/gbrain
bun install && bun link
```
Verify: `gbrain --version` should print a version number. If `gbrain` is not found,
restart the shell or add the PATH export to the shell profile.
> **If `bun install -g` aborts or `gbrain doctor` reports `schema_version: 0`** (Bun
> occasionally blocks the top-level postinstall hook on global installs, so schema
> migrations don't run automatically), the CLI prints a recovery hint pointing at
> [#218](https://github.com/garrytan/gbrain/issues/218). Run `gbrain apply-migrations --yes`
> to recover. If that doesn't work, fall back to the deterministic install path:
>
> ```bash
> git clone https://github.com/garrytan/gbrain.git ~/gbrain && cd ~/gbrain
> bun install && bun link
> ```
> **Do NOT use `bun install -g github:garrytan/gbrain`.** Bun blocks the top-level
> postinstall hook on global installs, so schema migrations never run and the CLI
> aborts with `Aborted()` when it opens PGLite. Use the `git clone + bun link` path
> above. Tracking issue: [#218](https://github.com/garrytan/gbrain/issues/218).
## Step 2: API Keys
Ask the user for these. gbrain defaults to the ZeroEntropy embedding + reranker stack
(as of v0.36.2.0); OpenAI/Voyage are still supported as fallbacks via `gbrain config
set embedding_model <provider:model>`.
Ask the user for these:
```bash
export ZEROENTROPY_API_KEY=ze-... # default embedding + reranker (v0.36.2.0+)
export OPENAI_API_KEY=sk-... # fallback for vector search; also used for chat models
export ANTHROPIC_API_KEY=sk-ant-... # optional, improves search quality via query expansion
export OPENAI_API_KEY=sk-... # required for vector search
export ANTHROPIC_API_KEY=sk-ant-... # optional, improves search quality
```
Save to shell profile or `.env`. Keys are picked up by `gbrain config set` automatically
or can be stored in `~/.gbrain/config.json` (file plane). Without any embedding provider,
keyword search still works. Without Anthropic, search works but skips query expansion.
Save to shell profile or `.env`. Without OpenAI, keyword search still works.
Without Anthropic, search works but skips query expansion.
## Step 3: Create the Brain
@@ -72,63 +61,6 @@ Read `~/gbrain/docs/GBRAIN_RECOMMENDED_SCHEMA.md` and set up the MECE directory
structure (people/, companies/, concepts/, etc.) inside the user's brain repo,
NOT inside ~/gbrain.
## Step 3.5: Confirm search mode with the user (DO NOT SKIP)
`gbrain init` auto-applied a default search mode (`tokenmax` unless your subagent
tier is Haiku-class or no OpenAI key is configured). The init output included the
cost matrix below preceded by `[AGENT]` markers. You must NOT silently accept the
default. Stop and ask the operator.
**Present this matrix verbatim:**
```
Per-query cost @ 10K queries/mo (typical single-user volume):
Haiku 4.5 Sonnet 4.6 Opus 4.7
($1/M) ($3/M) ($5/M)
conservative $40/mo $120/mo $200/mo
balanced $100/mo $300/mo $500/mo
tokenmax $200/mo $600/mo $1,000/mo
(scales linearly: ×10 for 100K/mo, ÷10 for 1K. 25x corner-to-corner spread.
Natural diagonal pairings — cheap/cheap → frontier/frontier — span ~4x.)
```
**Ask the operator (paraphrase if needed):**
> Your gbrain just installed with search mode `<auto-applied default>`. This is
> a one-time setup decision that controls retrieval payload size. Which mode
> do you want?
>
> 1) conservative — tight 4K budget, no LLM expansion, 10 chunks max.
> Best for Haiku subagents, cost-sensitive setups, high-volume loops.
>
> 2) balanced — 12K budget, no expansion, 25 chunks. Sonnet-tier sweet spot.
>
> 3) tokenmax (recommended default — preserves v0.31.x retrieval shape) —
> no budget, LLM expansion ON, 50 chunks. Best for Opus/frontier models.
>
> Cost depends on BOTH the mode AND the downstream model you run. See the
> matrix above for the 9-cell breakdown.
If the operator picks a non-default mode, run:
```bash
gbrain config set search.mode <mode>
```
If they pick tokenmax AND want to preserve the literal v0.31.x default
(limit=20 instead of tokenmax's 50), also run:
```bash
gbrain config set search.searchLimit 20
```
Verify the choice with `gbrain search modes` before continuing.
**Why this matters:** the cost spread between corners of the matrix is 25x.
An agent that silently accepts the default and starts running queries against
a user who didn't expect tokenmax-class context loads can rack up surprise
spend. Confirm before continuing.
## Step 4: Import and Index
```bash
@@ -161,49 +93,10 @@ After this step:
If a user has a very large brain (>10K pages), `extract --source db` is idempotent
and supports `--since YYYY-MM-DD` for incremental runs.
### Obsidian-style bare wikilinks (opt-in)
If the user imported an Obsidian or Notion vault that uses **bare** `[[note-name]]`
wikilinks — where `[[struktura]]` written in one folder means the page that lives
at `projects/struktura.md` in another — GBrain does NOT connect those by default.
Out of the box it only resolves path-qualified refs like `[[projects/struktura]]`,
so a vault full of bare links shows up as a thin, broken graph. Turn on basename
resolution so the cross-folder links connect:
```bash
gbrain config set link_resolution.global_basename true
gbrain extract links --source db # re-run so the new edges land
```
`gbrain doctor` surfaces a `link_resolution_opportunity` hint with the exact count
("47 of 60 bare wikilinks would resolve") so you know whether it's worth enabling
before you flip it. When a bare name matches more than one page (`[[struktura]]`
both `projects/struktura` and `archive/struktura`), GBrain emits one edge to each
rather than guessing a winner — review and prune the duplicates with
`gbrain graph-query <slug>`. The mode is also honored on the filesystem-walk path
(`gbrain extract links` with no `--source db`) and by auto-link on every future
`put_page`.
## Step 5: Load Skills
If you're running an agent platform (OpenClaw, Hermes, or any repo with a workspace),
scaffold the bundled skills into it:
```bash
cd /path/to/agent/workspace
gbrain skillpack scaffold --all # copy 43 curated skills + RESOLVER.md
```
Scaffolded skills are first-class files in your repo. Edit freely; re-running scaffold
refuses to overwrite anything that exists. Use `gbrain skillpack reference <name>` to
diff against gbrain's bundle when you want upstream improvements. (The legacy
`gbrain skillpack install` managed-block model was retired in v0.36.0.0 — run
`gbrain skillpack migrate-fence` once if upgrading from an older release.)
Whether you scaffolded or not, read `skills/RESOLVER.md` (in your workspace, or the
bundled copy at `~/gbrain/skills/RESOLVER.md` when running from the cloned repo). It's
the skill dispatcher — tells you which skill to read for any task. Save this to your
memory permanently.
Read `~/gbrain/skills/RESOLVER.md`. This is the skill dispatcher. It tells you which
skill to read for any task. Save this to your memory permanently.
The three most important skills to adopt immediately:
@@ -231,17 +124,13 @@ If skipped, minimal defaults are installed automatically.
## Step 7: Recurring Jobs
Set up using your platform's scheduler (OpenClaw cron, Railway cron, crontab), or skip the
platform glue entirely with `gbrain autopilot --install` (built-in self-maintaining daemon):
Set up using your platform's scheduler (OpenClaw cron, Railway cron, crontab):
- **Live sync** (every 15 min): `gbrain sync --repo ~/brain && gbrain embed --stale`
— or `gbrain sync --watch` for a continuous loop.
- **Auto-update** (daily): `gbrain check-update --json` (tell user, never auto-install).
- **Dream cycle** (nightly): `gbrain dream` runs the 8-phase overnight maintenance cycle.
Entity sweep, citation fixes, memory consolidation, plus (v0.23+) overnight conversation
synthesis and cross-session pattern detection. One cron-friendly command. This is what
makes the brain compound. Do not skip it. See `docs/guides/cron-schedule.md` for the
full protocol.
- **Auto-update** (daily): `gbrain check-update --json` (tell user, never auto-install)
- **Dream cycle** (nightly): read `docs/guides/cron-schedule.md` for the full protocol.
Entity sweep, citation fixes, memory consolidation. This is what makes the brain
compound. Do not skip it.
- **Weekly**: `gbrain doctor --json && gbrain embed --stale`
## Step 8: Integrations
@@ -259,18 +148,9 @@ actually works) is the most important.
## Upgrade
If you installed via `bun install -g`:
```bash
gbrain upgrade # self-updates the binary, runs schema migrations,
# and prints post-upgrade notes for the version range
```
If you installed via `git clone + bun link`:
```bash
cd ~/gbrain && git pull origin master && bun install
gbrain apply-migrations --yes # apply schema migrations (idempotent)
gbrain init # apply schema migrations (idempotent)
gbrain post-upgrade # show migration notes for the version range
```
@@ -278,15 +158,6 @@ Then read `~/gbrain/skills/migrations/v<NEW_VERSION>.md` (and any intermediate
versions you skipped) and run any backfill or verification steps it lists. Skipping
this is how features ship in the binary but stay dormant in the user's brain.
**v0.32.3 search modes (one-time upgrade prompt):** if the user's brain was
created before v0.32.3, `gbrain post-upgrade` prints a banner including the
9-cell cost matrix (mode × downstream model) preceded by `[AGENT]` markers.
**Do NOT silently move past the banner.** Present the matrix to the operator
verbatim, ask which mode they want (recommended default: `tokenmax` to preserve
v0.31.x retrieval shape), then run `gbrain config set search.mode <mode>`. See
Step 3.5 above for the full ask-the-user protocol — the upgrade path uses the
same matrix and same default.
For v0.12.0+ specifically: if your brain was created before v0.12.0, run
`gbrain extract links --source db && gbrain extract timeline --source db` to
backfill the new graph layer (see Step 4.5 above).
@@ -297,58 +168,3 @@ automatically during `gbrain post-upgrade` to fix the double-encoded JSONB
columns. PGLite brains no-op. If wiki-style imports were truncated by the old
`splitBody` bug, run `gbrain sync --full` after upgrading to rebuild
`compiled_truth` from source markdown.
## v0.42.0+ onboard surface (NEW)
`gbrain onboard` is the activation surface gbrain did not have before.
Once your brain has any content, run `gbrain onboard --check --json` to
see structured recommendations across 5 brain-health axes (orphans,
stale embeddings, entity link coverage, timeline coverage, takes count).
**On first connect (after `gbrain init`):**
```bash
gbrain onboard --check --json
```
The JSON envelope (`schema_version: 1`) carries `recommendations[]` with
`apply_policy` per item: `auto_apply` (safe to run unattended),
`prompt_required` (needs explicit user consent), or `manual_only`
(LLM-bearing, user must run themselves).
**After every `gbrain upgrade`:**
```bash
gbrain onboard --check --json
```
New versions may surface new opportunities. The post-upgrade banner
nudges the user when it runs, but agents should re-probe as a hygiene
step regardless.
**Unattended remediation (cron / autopilot):**
```bash
gbrain onboard --auto --max-usd 5
```
Refuses without `--max-usd N`. Runs auto-eligible items only. The
autopilot daemon also consults onboard recommendations on its tick — no
explicit agent action needed for the autonomous path.
**Remote / federated brain installs (MCP):**
The `run_onboard` MCP op (admin scope) lets thin-client agents probe
brain health + drive remediation over OAuth-authenticated MCP. Protected
LLM-bearing handlers (synthesize, patterns, consolidate, takes-bootstrap,
contextual_reindex_per_chunk) require the additional `run_protected_onboard`
scope — admin alone is insufficient. The MCP op returns
`skipped_missing_scope[]` listing what would have run with the right
grants.
**Privacy + consent gates:**
- `gbrain takes extract --from-pages` sends concept/atom/lore/briefing/
writing/originals page content to your configured chat model (default
Anthropic Haiku). Refuses to run unless `takes.bootstrap_enabled=true`
is set in config AND `--yes` is passed. Two-gate opt-in by design.
- Autopilot's auto-apply tier for takes-bootstrap stays `manual_only`
until v0.42.1's eval gate (do not bypass).
**Suppress nudges in CI / scripted environments:**
```bash
export GBRAIN_NO_ONBOARD_NUDGE=1
```
Init + upgrade banners auto-skip in non-TTY too.
+602 -353
View File
File diff suppressed because it is too large Load Diff
-308
View File
@@ -1,308 +0,0 @@
# Security
## Reporting Vulnerabilities
If you discover a security issue in GBrain, please report it privately by opening
a [private security advisory](https://github.com/garrytan/gbrain/security/advisories/new)
on GitHub.
Do not open a public issue for security vulnerabilities.
## Automated security scanning
CI runs three automated security checks alongside secret scanning (Gitleaks):
- **Dependency vulnerabilities** — OSV-Scanner
(`.github/workflows/osv-scanner.yml`) runs weekly and on any PR that touches
`package.json` or `bun.lock`.
- **Static analysis (SAST)** — Semgrep CE (`.github/workflows/semgrep.yml`)
runs on every PR and weekly. It is currently **advisory (non-blocking)**
while the finding baseline is tuned; the graduation path to a blocking check
is documented in the workflow file.
- **Release binary provenance** — release builds
(`.github/workflows/release.yml`) attest each compiled binary with
[GitHub artifact attestations](https://docs.github.com/en/actions/security-for-github-actions/using-artifact-attestations).
Verify a downloaded release binary with:
```bash
gh attestation verify ./gbrain-darwin-arm64 -R garrytan/gbrain
gh attestation verify ./gbrain-linux-x64 -R garrytan/gbrain
```
All security workflows use SHA-pinned actions and least-privilege permissions,
enforced structurally by actionlint on every workflow change.
## Remote MCP Security
### ⚠️ Do NOT use open OAuth client registration for remote MCP
If you deploy GBrain's MCP server behind an HTTP wrapper with OAuth 2.1
support, **never allow unauthenticated client registration**. An attacker
who discovers your server URL can:
1. Register a new OAuth client via `POST /register`
2. Use `client_credentials` grant to obtain a bearer token
3. Access all brain data via the MCP tools
### Recommended: `gbrain serve --http`
As of v0.22.7, GBrain ships a built-in HTTP transport that uses the
existing `access_tokens` table for authentication:
```bash
# Create a token
gbrain auth create "my-client"
# Start the HTTP server
gbrain serve --http --port 8787
# Connect via ngrok, Tailscale, or any tunnel
ngrok http 8787 --url your-brain.ngrok.app
```
This is the recommended way to expose GBrain remotely. No OAuth, no
registration endpoint, no self-service tokens. Tokens are managed
exclusively via `gbrain auth create/list/revoke`.
### If you must use a custom HTTP wrapper
1. **Require a secret for client registration** — check a header or body
parameter before creating new OAuth clients
2. **Disable `client_credentials` grant** — only allow `authorization_code`
with browser-based approval
3. **Restrict scopes** — never issue tokens with unlimited scope
4. **Log all token issuance** — alert on unexpected registrations
5. **Rate-limit registration and token endpoints**
### Pre-registering claude.ai / ChatGPT clients without DCR (v0.41.3+)
The recommended hardening posture above is: ship `gbrain serve --http`
**without** `--enable-dcr` and pre-register every client manually. As of
v0.41.3, `gbrain auth register-client` accepts the OAuth fields
browser-based clients need:
```bash
# Pre-register claude.ai (confidential client; two redirect URIs)
gbrain auth register-client claude-ai \
--scopes "read write" \
--redirect-uri https://claude.ai/api/mcp/auth_callback \
--redirect-uri https://claude.com/api/mcp/auth_callback
# --grant-types is auto-set to authorization_code,refresh_token when
# --redirect-uri is passed; pass --grant-types explicitly to override.
# Pre-register ChatGPT (public PKCE client; no client_secret minted)
gbrain auth register-client chatgpt \
--scopes "read write" \
--redirect-uri https://chatgpt.com/connector/oauth/<HASH> \
--token-endpoint-auth-method none
```
Auth methods (`--token-endpoint-auth-method`):
- `client_secret_post` (default) — confidential client, secret in body
- `client_secret_basic` — confidential client, secret in `Authorization` header
- `none` — public PKCE-only client (no secret minted; ChatGPT custom
connector, Claude Code, Cursor)
The validator rejects unknown methods at the registration boundary, and
the same gate applies to the admin endpoint `POST /admin/api/register-client`
and the DCR `POST /register` path. Pre-v0.41.3 the CLI hard-coded
`redirect_uris = []` and `token_endpoint_auth_method = NULL`, forcing
operators to UPDATE `oauth_clients` rows by hand to make claude.ai work
without `--enable-dcr`. That footgun is gone.
### DCR consent default (v0.42.55+)
The "disable `client_credentials`, only allow `authorization_code`" guidance
above is now the built-in default for the DCR path, not just advice for custom
wrappers. With `--enable-dcr` on, a self-registered client defaults to the
`authorization_code` (browser-approval) grant, and an explicit
`client_credentials` request is rejected with `invalid_client_metadata`.
Operators who genuinely need the machine-to-machine grant on the registration
endpoint opt in with `--enable-dcr-insecure` (which implies `--enable-dcr`); a
startup WARNING prints whenever DCR is enabled, and a second when the insecure
grant is allowed. Pre-registering clients via the CLI / admin API is unchanged.
### Token Management
```bash
gbrain auth create "claude-desktop" # Create a new token
gbrain auth list # List all tokens
gbrain auth revoke "claude-desktop" # Revoke a token
gbrain auth test <url> --token <tok> # Smoke-test a remote server
```
Tokens are stored as SHA-256 hashes in the `access_tokens` table. The
plaintext token is shown once at creation and never stored.
## `gbrain serve --http` hardening (v0.22.7+)
The built-in HTTP transport ships with several layers of hardening on by
default. All env vars below are optional; the defaults are intentionally
conservative.
### Bind address (v0.34: loopback by default)
`gbrain serve --http` listens on `127.0.0.1` by default. Personal-laptop
installs cannot accidentally publish the brain to the LAN. Self-hosted
deployments that need remote access pass `--bind 0.0.0.0` (all
interfaces) or `--bind <interface-ip>` (specific NIC). A stderr WARN
fires when `--public-url` is set without `--bind` so the operator sees
the binding before the first request — common cause of "ngrok forwards
to me but the agent can't reach the upstream" misconfigurations.
### Postgres-only
`gbrain serve --http` requires a Postgres engine. PGLite is local-only by
design and the `access_tokens` / `mcp_request_log` tables don't exist in
the PGLite schema. Local agents continue to use stdio (`gbrain serve`).
Running `--http` against a PGLite-backed install fails fast with a clear
error message at startup.
### Docker network isolation (self-hosted Postgres)
OAuth and source scoping enforce isolation on the `serve --http` path only.
Raw Postgres reachability bypasses both: a container that shares Docker's
default `bridge` network with the brain's Postgres can open a direct DB
session without any token and read every source. Put the brain's Postgres on
a user-defined Docker network with nothing untrusted on it, publish its port
loopback-only (if at all), and never put `DATABASE_URL` or a Postgres
password in untrusted agent containers — those should reach the brain
exclusively via OAuth against `serve --http`. Full operator checklist:
[docs/mcp/DEPLOY.md — Co-located Docker workloads](docs/mcp/DEPLOY.md#co-located-docker-workloads-self-hosted-postgres).
### CORS
Default-deny: no `Access-Control-Allow-Origin` header is sent unless an
allowlist is configured. To allow browser-based MCP clients:
```bash
GBRAIN_HTTP_CORS_ORIGIN=https://claude.ai gbrain serve --http --port 8787
# Multiple origins: comma-separated
GBRAIN_HTTP_CORS_ORIGIN=https://claude.ai,https://your.app gbrain serve --http
```
When the request `Origin` matches the allowlist, the server echoes it
back in `Access-Control-Allow-Origin` (with `Vary: Origin`). Otherwise no
CORS header is sent and the browser blocks the request.
**v0.41.3:** the same allowlist now gates every OAuth endpoint (`/mcp`,
`/token`, `/authorize`, `/register`, `/revoke`). Pre-v0.41.3 these used
default-wide-open `cors()` middleware, leaking
`Access-Control-Allow-Origin: *` on every response — any web origin could
complete a token exchange from a logged-in operator's browser. The CORS
preflight handler in the legacy bearer transport was also asymmetric
(actual-request path correctly default-deny, but OPTIONS preflight leaked
`Access-Control-Allow-Methods` + `Access-Control-Allow-Headers` to every
Origin); both are now consolidated through a single allowlist-gated path.
A startup stderr WARN fires when `--bind 0.0.0.0` is set without
`GBRAIN_HTTP_CORS_ORIGIN`, surfacing the default-deny posture before the
first request.
### Rate limiting
Two buckets, both stored in a bounded LRU map (default 10K keys, evicts
least-recently-used on overflow, prunes entries older than 2× the
window):
| Bucket | When it fires | Default | Env var |
|---|---|---|---|
| Pre-auth IP | Before the DB lookup, on every `/mcp` request | 30 req / 60s | `GBRAIN_HTTP_RATE_LIMIT_IP` |
| Post-auth token | After a valid token is resolved | 60 req / 60s | `GBRAIN_HTTP_RATE_LIMIT_TOKEN` |
| LRU cap | Maximum distinct keys across both buckets | 10000 | `GBRAIN_HTTP_RATE_LIMIT_LRU` |
On exhaustion the server returns `429 Too Many Requests` with a
`Retry-After` header.
**Caveat for tunneled deployments (ngrok, Tailscale Funnel, Cloudflare
Tunnel):** all requests share one egress IP, so the pre-auth IP bucket
becomes effectively shared by all clients on that tunnel. The
post-auth token-id bucket is the load-bearing limiter for tunnel-fronted
deployments.
### Reverse-proxy trust
**Loopback-only by default** (v0.41.3+ Express server agrees with the
legacy transport; pre-v0.41.3 the Express server hardcoded `'loopback'`
while docs claimed "disabled by default" — that disagreement is gone).
The default trusts only same-host proxies (127.0.0.1, ::1, fc00::/7);
external forwarded-for headers are ignored regardless. To widen or
narrow trust:
```bash
# Trust exactly one hop — Fly.io, Render, Vercel, single-layer nginx
GBRAIN_HTTP_TRUST_PROXY=1 gbrain serve --http --port 8787
# Trust N hops — Cloudflare → nginx → gbrain
GBRAIN_HTTP_TRUST_PROXY=2 gbrain serve --http --port 8787
# Disable entirely — direct-exposure deployment with no proxy
GBRAIN_HTTP_TRUST_PROXY=0 gbrain serve --http --port 8787
# Named Express modes (uniquelocal, linklocal) or CIDR lists pass through
GBRAIN_HTTP_TRUST_PROXY=uniquelocal gbrain serve --http --port 8787
GBRAIN_HTTP_TRUST_PROXY="10.0.0.0/8,192.168.1.0/24" gbrain serve --http --port 8787
```
Both transports (Express OAuth server in `src/commands/serve-http.ts` and
the legacy bearer transport in `src/mcp/http-transport.ts`) read the same
env var, so single source of truth.
**Critical safety contract:** only widen past `'loopback'` when **both**
of these are true:
1. gbrain is reachable only via a trusted reverse proxy (not directly
exposed to the internet on the configured port). As of v0.34
`gbrain serve --http` binds `127.0.0.1` by default, so the
reverse-proxy-only posture is the out-of-the-box shape; only
override with `--bind 0.0.0.0` (or a specific interface IP) when
gbrain itself needs to accept remote connections directly.
2. The proxy strips any client-supplied `X-Forwarded-For` and `X-Real-IP`
headers, then sets them itself. (nginx with `proxy_set_header
X-Forwarded-For $remote_addr` does this; Cloudflare and most cloud
load balancers handle it automatically.)
If gbrain is reachable directly AND `GBRAIN_HTTP_TRUST_PROXY=1` (or any
non-loopback value) is set, clients can spoof their IP by sending
arbitrary `X-Forwarded-For` headers, defeating the pre-auth IP rate
limit. The `'loopback'` default protects against this by ignoring all
forwarded-for headers and using the socket peer address.
### Body size cap
Default 1 MiB, stream-counted (chunked transfers without
`Content-Length` are still capped). Override:
```bash
GBRAIN_HTTP_MAX_BODY_BYTES=2097152 gbrain serve --http # 2 MiB
```
Over-cap requests get `413 Payload Too Large` immediately, before any
body is materialized in memory.
### Audit log
Every `/mcp` request writes one row to `mcp_request_log`:
```bash
psql "$DATABASE_URL" -c \
"SELECT created_at, token_name, operation, status, latency_ms
FROM mcp_request_log
ORDER BY created_at DESC LIMIT 100"
```
`status` is one of: `success`, `error`, `auth_failed`, `rate_limited`,
`body_too_large`, `parse_error`, `unknown_method`. Failed-auth rows have
`token_name = NULL`. Inserts are fire-and-forget so audit failures
never block requests.
**v0.26.9 redaction default.** The `params` column now stores
`{redacted, kind, declared_keys, unknown_key_count, approx_bytes}` instead
of raw JSON-RPC payloads. Declared keys (intersected against the operation's
spec) preserve for debug visibility; unknown keys are counted but never
named so attackers can't probe key existence; byte sizes bucket to 1KB so
content sizes can't be binary-searched. The same shape is broadcast on the
admin SSE feed at `/admin/events`. Operators on a personal laptop who want
raw payloads back can pass `gbrain serve --http --log-full-params` (loud
stderr warning at startup). Multi-tenant deployments should leave it
on the redacted default.
+14 -4258
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -1 +1 @@
0.42.67.0
0.22.5
-158
View File
@@ -1,158 +0,0 @@
# Design System — GBrain Admin Dashboard
## Product Context
- **What this is:** Admin dashboard for GBrain MCP server — manage OAuth agents, API keys, monitor requests
- **Who it's for:** GBrain operators managing multi-agent access to their brain
- **Space/industry:** Developer infrastructure (peers: Supabase dashboard, Vercel, Railway)
- **Project type:** Dense utilitarian admin panel — Steve Krug "Don't Make Me Think"
## Aesthetic Direction
- **Direction:** Industrial/Utilitarian — function-first, data-dense, zero decoration
- **Decoration level:** None — every pixel earns its place with information
- **Mood:** Ops dashboard for someone who builds. Not a marketing site. Not a consumer app. A cockpit.
- **Reference:** Supabase dashboard (dark + dense), Linear (restrained), Grafana (data-forward)
## Alignment
- **Text alignment:** Left-align everything. No centered text in tables, cards, forms, or labels.
- **Headings:** Left-aligned
- **Table data:** Left-aligned (including numbers — contextual readability over columnar alignment)
- **Form labels:** Left-aligned above inputs
- **Buttons in forms:** Right-aligned (action flows left-to-right: Cancel → Submit)
- **Modal titles:** Left-aligned
- **Page titles:** Left-aligned
- **Only exception:** Empty states and the login page lock icon can center for visual weight
## Typography
- **Display/Headings:** Inter (Semibold 600) — clean, neutral, disappears into the content
- **Body/UI:** Inter (Regular 400 / Medium 500)
- **Data/Tables/Code:** JetBrains Mono (Regular 400 / Medium 500) — monospace for anything the user might copy, any ID, any token, any technical value
- **Loading:** Google Fonts. `display=swap`.
- **Scale:**
- Page title: 24px / Inter Semibold
- Section title: 14px / Inter Semibold, uppercase, letter-spacing 0.5px
- Table header: 12px / Inter Medium, uppercase, letter-spacing 1px, muted color
- Body: 14px / Inter Regular
- Small/Caption: 13px
- Micro: 12px (badges, timestamps)
- Code/Data: 13px / JetBrains Mono
## Color
- **Approach:** Monochrome base + semantic color only. No primary brand color. Color means something.
- **Background:**
- Base: #0a0a0f (near-black with blue undertone)
- Surface/cards: #12121a
- Hover: #1a1a2a
- Input/code blocks: #0f0f1a
- **Borders:** #1e1e2e (default), #3a3a5a (hover/active)
- **Text:**
- Primary: #e0e0e0
- Secondary: #888888
- Muted: #555555
- Link: #88aaff
- **Semantic (badges only):**
- Success/active: #34a853
- Error/danger: #ff6b6b
- Warning: #f5a623
- Read scope: #3b82f6
- Write scope: #f59e0b
- Admin scope: #ef4444
- **No accent color.** The data IS the interface. Badges carry all the color.
## Spacing
- **Base unit:** 4px
- **Density:** Dense — this is an ops tool, not a landing page
- **Scale:** 4px, 8px, 12px, 16px, 20px, 24px, 32px, 48px
- **Table row padding:** 10px 16px
- **Card padding:** 24px
- **Modal padding:** 24px
- **Section gaps:** 24px between sections, 12px between related elements
## Layout
- **Sidebar:** Fixed left, 200px wide, dark (#0a0a0f)
- **Main content:** Fluid, max-width none (fills available space)
- **Grid:** Single column for tables (full width), 2-column for stats cards
- **Border radius:**
- Cards/panels: 16px
- Buttons/inputs: 8px
- Badges: 9999px (pill)
- Tables: 0 (sharp edges — data is rectangular)
## Components
### Tables
- Full-width, no outer border
- Header row: uppercase, letter-spaced, muted color, no background
- Data rows: subtle hover (#1a1a2a), pointer cursor when clickable
- All text left-aligned
- Monospace for IDs, tokens, latency values
### Badges
- Pill shape (border-radius: 9999px)
- Padding: 2px 8px
- Font: 12px
- Scoped to semantic meaning: `success`, `danger`, `read`, `write`, `admin`
### Buttons
- Primary: white text on #3a3a5a, hover brightens
- Secondary: muted text on transparent, border #1e1e2e
- Danger: white text on #ff6b6b background
- Size: 13px font, 6px 14px padding
### Modals
- Overlay: rgba(0,0,0,0.7)
- Card: #12121a, border #1e1e2e, border-radius 16px, max-width 480px
- Title: 18px Semibold, left-aligned
- Close: top-right ✕ button
### Drawers
- Right-side panel, 400px wide
- Slide in from right
- Dark overlay behind
- Close button top-right
- Sections separated by section titles (uppercase, muted)
### Tabs
- Inline horizontal, wrapping allowed
- Active: white text, bottom border
- Inactive: muted text, no border
- No background color on tabs
### Code blocks
- Background: rgba(0,0,0,0.3)
- Border-radius: 8px
- Padding: 10px 14px
- Font: JetBrains Mono 12px
- Copy button: right-aligned, subtle
### Empty states
- Centered text (only exception to left-align rule)
- Muted color
- Suggest next action
## Motion
- **Approach:** Minimal — transitions for hover states only
- **Duration:** 150ms for hovers, 200ms for drawer slide
- **No loading spinners** — show stale data until fresh arrives
- **SSE live feed:** Real-time, no animation on new entries (just prepend)
## Anti-Patterns (do NOT do these)
- ❌ Center-aligned table data
- ❌ Center-aligned headings or labels (except empty states)
- ❌ Gradient backgrounds
- ❌ Shadows (the dark theme IS the depth model)
- ❌ Rounded table corners
- ❌ Icons as navigation (use text labels)
- ❌ Loading skeletons (show real data or nothing)
- ❌ Confirmation toasts (action → result is immediate and visible)
- ❌ Color for decoration (every color means something)
## Decisions Log
| Date | Decision | Rationale |
|------|----------|-----------|
| 2026-05-01 | Dark theme only | Ops dashboard. No light mode needed. |
| 2026-05-01 | Steve Krug lens | Zero happy talk, mindless choices, scannable tables, billboard-speed comprehension. |
| 2026-05-01 | JetBrains Mono for data | Anything copyable or technical should be monospace. |
| 2026-05-03 | Left-align everything | Garry preference. Centered text is a design crutch. Left-align forces hierarchy through typography weight and spacing, not position. |
| 2026-05-03 | Incorporate GStack design DNA | Same family: Inter + JetBrains Mono, dark base, semantic-only color. Diverges on accent (GStack: amber; GBrain: none — data is the color). |
| 2026-05-03 | Per-client config export tabs | Claude Code, ChatGPT, Claude.ai, Cursor, Perplexity, JSON. Every agent has a copy-paste setup path. |
| 2026-05-03 | Magic link auth | Login page tells you to ask your agent. No pasting hex strings into forms. |
-289
View File
@@ -1,289 +0,0 @@
{
"lockfileVersion": 1,
"configVersion": 1,
"workspaces": {
"": {
"name": "gbrain-admin",
"dependencies": {
"react": "^19.1.0",
"react-dom": "^19.1.0",
},
"devDependencies": {
"@types/react": "^19.1.2",
"@types/react-dom": "^19.1.2",
"@vitejs/plugin-react": "^4.4.1",
"typescript": "^5.8.3",
"vite": "^6.4.3",
},
},
},
"overrides": {
"@babel/core": "^7.29.6",
"postcss": "^8.5.10",
},
"packages": {
"@babel/code-frame": ["@babel/code-frame@7.29.7", "", { "dependencies": { "@babel/helper-validator-identifier": "^7.29.7", "js-tokens": "^4.0.0", "picocolors": "^1.1.1" } }, "sha512-Aup7aUOfpbAUg2ROOJN6Iw5f9DMBlzu0mIkm/malLQFN/YQgO48wCj0Kxa3sEHJvPVFg7siR+qRInwXd2qhQKw=="],
"@babel/compat-data": ["@babel/compat-data@7.29.7", "", {}, "sha512-locTkQyKvwIEgBzVrn8693ebc97F2U8ZHjbXwDXJ5Fn2TCpNwTlKcaKLkdHop5c/icOFE7qt7Q9JC5hnKNa6Gg=="],
"@babel/core": ["@babel/core@7.29.7", "", { "dependencies": { "@babel/code-frame": "^7.29.7", "@babel/generator": "^7.29.7", "@babel/helper-compilation-targets": "^7.29.7", "@babel/helper-module-transforms": "^7.29.7", "@babel/helpers": "^7.29.7", "@babel/parser": "^7.29.7", "@babel/template": "^7.29.7", "@babel/traverse": "^7.29.7", "@babel/types": "^7.29.7", "@jridgewell/remapping": "^2.3.5", "convert-source-map": "^2.0.0", "debug": "^4.1.0", "gensync": "^1.0.0-beta.2", "json5": "^2.2.3", "semver": "^6.3.1" } }, "sha512-RgHBCvtjbOK2gXSNBNIkNoEc9qoVEtau3hj8gEqKQuL3HZAibKarWFEI3Lfm6EYKkLalOh8eSrj9b+ch9H/VBA=="],
"@babel/generator": ["@babel/generator@7.29.7", "", { "dependencies": { "@babel/parser": "^7.29.7", "@babel/types": "^7.29.7", "@jridgewell/gen-mapping": "^0.3.12", "@jridgewell/trace-mapping": "^0.3.28", "jsesc": "^3.0.2" } }, "sha512-DkXD5OJQaAQIdZ1bt3UZdEnHAn9Imd3IVBdX03UFe+ony9Ojw5pzr9YVKGDY1jt+Gcn/FnGkNf8r+Vj5NOJWtQ=="],
"@babel/helper-compilation-targets": ["@babel/helper-compilation-targets@7.29.7", "", { "dependencies": { "@babel/compat-data": "^7.29.7", "@babel/helper-validator-option": "^7.29.7", "browserslist": "^4.24.0", "lru-cache": "^5.1.1", "semver": "^6.3.1" } }, "sha512-wem6WaBj4NaVYVdNhLPPVacES6ZJ+KBBfSkTMD3YZxbP3rm3Di85tJU5ljaUNhaOynt+Aj0xruhYuzQBt8n71g=="],
"@babel/helper-globals": ["@babel/helper-globals@7.29.7", "", {}, "sha512-3nQVUAtvkKH9zahfWgw96Jc/uFOmjACE1kQz82E2lqWmHBgjzbNlsC22nuQTfahmWeQtTq5nQ/4Nnd2A1wj4zA=="],
"@babel/helper-module-imports": ["@babel/helper-module-imports@7.29.7", "", { "dependencies": { "@babel/traverse": "^7.29.7", "@babel/types": "^7.29.7" } }, "sha512-ejHwrQQYcm9xnTivShn2IDOlIzInN34AXskvq9QicvCtEzq1Vzclu/tKF8Jq1Cg8JG2GL6/EmjgsCT7lXepE3g=="],
"@babel/helper-module-transforms": ["@babel/helper-module-transforms@7.29.7", "", { "dependencies": { "@babel/helper-module-imports": "^7.29.7", "@babel/helper-validator-identifier": "^7.29.7", "@babel/traverse": "^7.29.7" }, "peerDependencies": { "@babel/core": "^7.0.0" } }, "sha512-UPUVSyXbOh627KiCIGQSgwWzGeBKLkaJ9PJEdrngIwMSzxLR4jS4+f1f1jb7VzBbg8nFLaYotvVPFCTqdrmTAg=="],
"@babel/helper-plugin-utils": ["@babel/helper-plugin-utils@7.28.6", "", {}, "sha512-S9gzZ/bz83GRysI7gAD4wPT/AI3uCnY+9xn+Mx/KPs2JwHJIz1W8PZkg2cqyt3RNOBM8ejcXhV6y8Og7ly/Dug=="],
"@babel/helper-string-parser": ["@babel/helper-string-parser@7.29.7", "", {}, "sha512-Pb5ijPrZ89GDH8223L4UP8i6QApWxs04RbPQJTeWDV0/keR2E36MeKnyr6LYmUUvqRRI+Iv87SuF1W6ErINzYw=="],
"@babel/helper-validator-identifier": ["@babel/helper-validator-identifier@7.29.7", "", {}, "sha512-qehxGkRj55h/ff8EMaJ+cYhyaKlHIxqYDn682wQD7RNp9UujOQsHog2uS0r2vzr4pW+sXf90NeeayjcNaX3fFg=="],
"@babel/helper-validator-option": ["@babel/helper-validator-option@7.29.7", "", {}, "sha512-N9ZErrD+yW5geCDtBqnOoxmR8+tNKiGuxKlDpuJxfsqpa2dFcexaziGAE/qoHLiDDreVNMupxGmSoNlyvsA3gw=="],
"@babel/helpers": ["@babel/helpers@7.29.7", "", { "dependencies": { "@babel/template": "^7.29.7", "@babel/types": "^7.29.7" } }, "sha512-1k2lAGRMfHTcwuNYcCNUmaUffmQv8KWMfh2iJUUeRlwlwH4FdNG7mfPI10NPfLHJFThE4Tyr4mv7kTNZOiPuBg=="],
"@babel/parser": ["@babel/parser@7.29.7", "", { "dependencies": { "@babel/types": "^7.29.7" }, "bin": "./bin/babel-parser.js" }, "sha512-hnORnjP/1P/zFEndoeX+n+t1RwWRJiJpM/jO7FW32Kn9r5+sJB2JWOdYo4L6k78j15eCwY3Gm/7364B1EMwtNg=="],
"@babel/plugin-transform-react-jsx-self": ["@babel/plugin-transform-react-jsx-self@7.27.1", "", { "dependencies": { "@babel/helper-plugin-utils": "^7.27.1" }, "peerDependencies": { "@babel/core": "^7.0.0-0" } }, "sha512-6UzkCs+ejGdZ5mFFC/OCUrv028ab2fp1znZmCZjAOBKiBK2jXD1O+BPSfX8X2qjJ75fZBMSnQn3Rq2mrBJK2mw=="],
"@babel/plugin-transform-react-jsx-source": ["@babel/plugin-transform-react-jsx-source@7.27.1", "", { "dependencies": { "@babel/helper-plugin-utils": "^7.27.1" }, "peerDependencies": { "@babel/core": "^7.0.0-0" } }, "sha512-zbwoTsBruTeKB9hSq73ha66iFeJHuaFkUbwvqElnygoNbj/jHRsSeokowZFN3CZ64IvEqcmmkVe89OPXc7ldAw=="],
"@babel/template": ["@babel/template@7.29.7", "", { "dependencies": { "@babel/code-frame": "^7.29.7", "@babel/parser": "^7.29.7", "@babel/types": "^7.29.7" } }, "sha512-puq+Gf35oI24FeN11LkoUQFqv9uwNeWpxXZi/Ji3rRIoKAzKnxRaZ+Gkj0vKS9ZCiTESfng1N9LyOyXvo+m+Gg=="],
"@babel/traverse": ["@babel/traverse@7.29.7", "", { "dependencies": { "@babel/code-frame": "^7.29.7", "@babel/generator": "^7.29.7", "@babel/helper-globals": "^7.29.7", "@babel/parser": "^7.29.7", "@babel/template": "^7.29.7", "@babel/types": "^7.29.7", "debug": "^4.3.1" } }, "sha512-EhlfNQtZ+NK22w5BM61ciuiq1m58ed33Wr1Xan//ZRTy6hgjnwyCffRYwzsGXdASJSUJ1guZILsErh1eQcl+zw=="],
"@babel/types": ["@babel/types@7.29.7", "", { "dependencies": { "@babel/helper-string-parser": "^7.29.7", "@babel/helper-validator-identifier": "^7.29.7" } }, "sha512-4zBIxpPzowiZpusoFkyGVwakdRJUyuH5PxQ/PrqghfdFWWasvnCdPfQXHrenDai+gyLARulZjZowCOj6fjT4pA=="],
"@esbuild/aix-ppc64": ["@esbuild/aix-ppc64@0.25.12", "", { "os": "aix", "cpu": "ppc64" }, "sha512-Hhmwd6CInZ3dwpuGTF8fJG6yoWmsToE+vYgD4nytZVxcu1ulHpUQRAB1UJ8+N1Am3Mz4+xOByoQoSZf4D+CpkA=="],
"@esbuild/android-arm": ["@esbuild/android-arm@0.25.12", "", { "os": "android", "cpu": "arm" }, "sha512-VJ+sKvNA/GE7Ccacc9Cha7bpS8nyzVv0jdVgwNDaR4gDMC/2TTRc33Ip8qrNYUcpkOHUT5OZ0bUcNNVZQ9RLlg=="],
"@esbuild/android-arm64": ["@esbuild/android-arm64@0.25.12", "", { "os": "android", "cpu": "arm64" }, "sha512-6AAmLG7zwD1Z159jCKPvAxZd4y/VTO0VkprYy+3N2FtJ8+BQWFXU+OxARIwA46c5tdD9SsKGZ/1ocqBS/gAKHg=="],
"@esbuild/android-x64": ["@esbuild/android-x64@0.25.12", "", { "os": "android", "cpu": "x64" }, "sha512-5jbb+2hhDHx5phYR2By8GTWEzn6I9UqR11Kwf22iKbNpYrsmRB18aX/9ivc5cabcUiAT/wM+YIZ6SG9QO6a8kg=="],
"@esbuild/darwin-arm64": ["@esbuild/darwin-arm64@0.25.12", "", { "os": "darwin", "cpu": "arm64" }, "sha512-N3zl+lxHCifgIlcMUP5016ESkeQjLj/959RxxNYIthIg+CQHInujFuXeWbWMgnTo4cp5XVHqFPmpyu9J65C1Yg=="],
"@esbuild/darwin-x64": ["@esbuild/darwin-x64@0.25.12", "", { "os": "darwin", "cpu": "x64" }, "sha512-HQ9ka4Kx21qHXwtlTUVbKJOAnmG1ipXhdWTmNXiPzPfWKpXqASVcWdnf2bnL73wgjNrFXAa3yYvBSd9pzfEIpA=="],
"@esbuild/freebsd-arm64": ["@esbuild/freebsd-arm64@0.25.12", "", { "os": "freebsd", "cpu": "arm64" }, "sha512-gA0Bx759+7Jve03K1S0vkOu5Lg/85dou3EseOGUes8flVOGxbhDDh/iZaoek11Y8mtyKPGF3vP8XhnkDEAmzeg=="],
"@esbuild/freebsd-x64": ["@esbuild/freebsd-x64@0.25.12", "", { "os": "freebsd", "cpu": "x64" }, "sha512-TGbO26Yw2xsHzxtbVFGEXBFH0FRAP7gtcPE7P5yP7wGy7cXK2oO7RyOhL5NLiqTlBh47XhmIUXuGciXEqYFfBQ=="],
"@esbuild/linux-arm": ["@esbuild/linux-arm@0.25.12", "", { "os": "linux", "cpu": "arm" }, "sha512-lPDGyC1JPDou8kGcywY0YILzWlhhnRjdof3UlcoqYmS9El818LLfJJc3PXXgZHrHCAKs/Z2SeZtDJr5MrkxtOw=="],
"@esbuild/linux-arm64": ["@esbuild/linux-arm64@0.25.12", "", { "os": "linux", "cpu": "arm64" }, "sha512-8bwX7a8FghIgrupcxb4aUmYDLp8pX06rGh5HqDT7bB+8Rdells6mHvrFHHW2JAOPZUbnjUpKTLg6ECyzvas2AQ=="],
"@esbuild/linux-ia32": ["@esbuild/linux-ia32@0.25.12", "", { "os": "linux", "cpu": "ia32" }, "sha512-0y9KrdVnbMM2/vG8KfU0byhUN+EFCny9+8g202gYqSSVMonbsCfLjUO+rCci7pM0WBEtz+oK/PIwHkzxkyharA=="],
"@esbuild/linux-loong64": ["@esbuild/linux-loong64@0.25.12", "", { "os": "linux", "cpu": "none" }, "sha512-h///Lr5a9rib/v1GGqXVGzjL4TMvVTv+s1DPoxQdz7l/AYv6LDSxdIwzxkrPW438oUXiDtwM10o9PmwS/6Z0Ng=="],
"@esbuild/linux-mips64el": ["@esbuild/linux-mips64el@0.25.12", "", { "os": "linux", "cpu": "none" }, "sha512-iyRrM1Pzy9GFMDLsXn1iHUm18nhKnNMWscjmp4+hpafcZjrr2WbT//d20xaGljXDBYHqRcl8HnxbX6uaA/eGVw=="],
"@esbuild/linux-ppc64": ["@esbuild/linux-ppc64@0.25.12", "", { "os": "linux", "cpu": "ppc64" }, "sha512-9meM/lRXxMi5PSUqEXRCtVjEZBGwB7P/D4yT8UG/mwIdze2aV4Vo6U5gD3+RsoHXKkHCfSxZKzmDssVlRj1QQA=="],
"@esbuild/linux-riscv64": ["@esbuild/linux-riscv64@0.25.12", "", { "os": "linux", "cpu": "none" }, "sha512-Zr7KR4hgKUpWAwb1f3o5ygT04MzqVrGEGXGLnj15YQDJErYu/BGg+wmFlIDOdJp0PmB0lLvxFIOXZgFRrdjR0w=="],
"@esbuild/linux-s390x": ["@esbuild/linux-s390x@0.25.12", "", { "os": "linux", "cpu": "s390x" }, "sha512-MsKncOcgTNvdtiISc/jZs/Zf8d0cl/t3gYWX8J9ubBnVOwlk65UIEEvgBORTiljloIWnBzLs4qhzPkJcitIzIg=="],
"@esbuild/linux-x64": ["@esbuild/linux-x64@0.25.12", "", { "os": "linux", "cpu": "x64" }, "sha512-uqZMTLr/zR/ed4jIGnwSLkaHmPjOjJvnm6TVVitAa08SLS9Z0VM8wIRx7gWbJB5/J54YuIMInDquWyYvQLZkgw=="],
"@esbuild/netbsd-arm64": ["@esbuild/netbsd-arm64@0.25.12", "", { "os": "none", "cpu": "arm64" }, "sha512-xXwcTq4GhRM7J9A8Gv5boanHhRa/Q9KLVmcyXHCTaM4wKfIpWkdXiMog/KsnxzJ0A1+nD+zoecuzqPmCRyBGjg=="],
"@esbuild/netbsd-x64": ["@esbuild/netbsd-x64@0.25.12", "", { "os": "none", "cpu": "x64" }, "sha512-Ld5pTlzPy3YwGec4OuHh1aCVCRvOXdH8DgRjfDy/oumVovmuSzWfnSJg+VtakB9Cm0gxNO9BzWkj6mtO1FMXkQ=="],
"@esbuild/openbsd-arm64": ["@esbuild/openbsd-arm64@0.25.12", "", { "os": "openbsd", "cpu": "arm64" }, "sha512-fF96T6KsBo/pkQI950FARU9apGNTSlZGsv1jZBAlcLL1MLjLNIWPBkj5NlSz8aAzYKg+eNqknrUJ24QBybeR5A=="],
"@esbuild/openbsd-x64": ["@esbuild/openbsd-x64@0.25.12", "", { "os": "openbsd", "cpu": "x64" }, "sha512-MZyXUkZHjQxUvzK7rN8DJ3SRmrVrke8ZyRusHlP+kuwqTcfWLyqMOE3sScPPyeIXN/mDJIfGXvcMqCgYKekoQw=="],
"@esbuild/openharmony-arm64": ["@esbuild/openharmony-arm64@0.25.12", "", { "os": "none", "cpu": "arm64" }, "sha512-rm0YWsqUSRrjncSXGA7Zv78Nbnw4XL6/dzr20cyrQf7ZmRcsovpcRBdhD43Nuk3y7XIoW2OxMVvwuRvk9XdASg=="],
"@esbuild/sunos-x64": ["@esbuild/sunos-x64@0.25.12", "", { "os": "sunos", "cpu": "x64" }, "sha512-3wGSCDyuTHQUzt0nV7bocDy72r2lI33QL3gkDNGkod22EsYl04sMf0qLb8luNKTOmgF/eDEDP5BFNwoBKH441w=="],
"@esbuild/win32-arm64": ["@esbuild/win32-arm64@0.25.12", "", { "os": "win32", "cpu": "arm64" }, "sha512-rMmLrur64A7+DKlnSuwqUdRKyd3UE7oPJZmnljqEptesKM8wx9J8gx5u0+9Pq0fQQW8vqeKebwNXdfOyP+8Bsg=="],
"@esbuild/win32-ia32": ["@esbuild/win32-ia32@0.25.12", "", { "os": "win32", "cpu": "ia32" }, "sha512-HkqnmmBoCbCwxUKKNPBixiWDGCpQGVsrQfJoVGYLPT41XWF8lHuE5N6WhVia2n4o5QK5M4tYr21827fNhi4byQ=="],
"@esbuild/win32-x64": ["@esbuild/win32-x64@0.25.12", "", { "os": "win32", "cpu": "x64" }, "sha512-alJC0uCZpTFrSL0CCDjcgleBXPnCrEAhTBILpeAp7M/OFgoqtAetfBzX0xM00MUsVVPpVjlPuMbREqnZCXaTnA=="],
"@jridgewell/gen-mapping": ["@jridgewell/gen-mapping@0.3.13", "", { "dependencies": { "@jridgewell/sourcemap-codec": "^1.5.0", "@jridgewell/trace-mapping": "^0.3.24" } }, "sha512-2kkt/7niJ6MgEPxF0bYdQ6etZaA+fQvDcLKckhy1yIQOzaoKjBBjSj63/aLVjYE3qhRt5dvM+uUyfCg6UKCBbA=="],
"@jridgewell/remapping": ["@jridgewell/remapping@2.3.5", "", { "dependencies": { "@jridgewell/gen-mapping": "^0.3.5", "@jridgewell/trace-mapping": "^0.3.24" } }, "sha512-LI9u/+laYG4Ds1TDKSJW2YPrIlcVYOwi2fUC6xB43lueCjgxV4lffOCZCtYFiH6TNOX+tQKXx97T4IKHbhyHEQ=="],
"@jridgewell/resolve-uri": ["@jridgewell/resolve-uri@3.1.2", "", {}, "sha512-bRISgCIjP20/tbWSPWMEi54QVPRZExkuD9lJL+UIxUKtwVJA8wW1Trb1jMs1RFXo1CBTNZ/5hpC9QvmKWdopKw=="],
"@jridgewell/sourcemap-codec": ["@jridgewell/sourcemap-codec@1.5.5", "", {}, "sha512-cYQ9310grqxueWbl+WuIUIaiUaDcj7WOq5fVhEljNVgRfOUhY9fy2zTvfoqWsnebh8Sl70VScFbICvJnLKB0Og=="],
"@jridgewell/trace-mapping": ["@jridgewell/trace-mapping@0.3.31", "", { "dependencies": { "@jridgewell/resolve-uri": "^3.1.0", "@jridgewell/sourcemap-codec": "^1.4.14" } }, "sha512-zzNR+SdQSDJzc8joaeP8QQoCQr8NuYx2dIIytl1QeBEZHJ9uW6hebsrYgbz8hJwUQao3TWCMtmfV8Nu1twOLAw=="],
"@rolldown/pluginutils": ["@rolldown/pluginutils@1.0.0-beta.27", "", {}, "sha512-+d0F4MKMCbeVUJwG96uQ4SgAznZNSq93I3V+9NHA4OpvqG8mRCpGdKmK8l/dl02h2CCDHwW2FqilnTyDcAnqjA=="],
"@rollup/rollup-android-arm-eabi": ["@rollup/rollup-android-arm-eabi@4.60.1", "", { "os": "android", "cpu": "arm" }, "sha512-d6FinEBLdIiK+1uACUttJKfgZREXrF0Qc2SmLII7W2AD8FfiZ9Wjd+rD/iRuf5s5dWrr1GgwXCvPqOuDquOowA=="],
"@rollup/rollup-android-arm64": ["@rollup/rollup-android-arm64@4.60.1", "", { "os": "android", "cpu": "arm64" }, "sha512-YjG/EwIDvvYI1YvYbHvDz/BYHtkY4ygUIXHnTdLhG+hKIQFBiosfWiACWortsKPKU/+dUwQQCKQM3qrDe8c9BA=="],
"@rollup/rollup-darwin-arm64": ["@rollup/rollup-darwin-arm64@4.60.1", "", { "os": "darwin", "cpu": "arm64" }, "sha512-mjCpF7GmkRtSJwon+Rq1N8+pI+8l7w5g9Z3vWj4T7abguC4Czwi3Yu/pFaLvA3TTeMVjnu3ctigusqWUfjZzvw=="],
"@rollup/rollup-darwin-x64": ["@rollup/rollup-darwin-x64@4.60.1", "", { "os": "darwin", "cpu": "x64" }, "sha512-haZ7hJ1JT4e9hqkoT9R/19XW2QKqjfJVv+i5AGg57S+nLk9lQnJ1F/eZloRO3o9Scy9CM3wQ9l+dkXtcBgN5Ew=="],
"@rollup/rollup-freebsd-arm64": ["@rollup/rollup-freebsd-arm64@4.60.1", "", { "os": "freebsd", "cpu": "arm64" }, "sha512-czw90wpQq3ZsAVBlinZjAYTKduOjTywlG7fEeWKUA7oCmpA8xdTkxZZlwNJKWqILlq0wehoZcJYfBvOyhPTQ6w=="],
"@rollup/rollup-freebsd-x64": ["@rollup/rollup-freebsd-x64@4.60.1", "", { "os": "freebsd", "cpu": "x64" }, "sha512-KVB2rqsxTHuBtfOeySEyzEOB7ltlB/ux38iu2rBQzkjbwRVlkhAGIEDiiYnO2kFOkJp+Z7pUXKyrRRFuFUKt+g=="],
"@rollup/rollup-linux-arm-gnueabihf": ["@rollup/rollup-linux-arm-gnueabihf@4.60.1", "", { "os": "linux", "cpu": "arm" }, "sha512-L+34Qqil+v5uC0zEubW7uByo78WOCIrBvci69E7sFASRl0X7b/MB6Cqd1lky/CtcSVTydWa2WZwFuWexjS5o6g=="],
"@rollup/rollup-linux-arm-musleabihf": ["@rollup/rollup-linux-arm-musleabihf@4.60.1", "", { "os": "linux", "cpu": "arm" }, "sha512-n83O8rt4v34hgFzlkb1ycniJh7IR5RCIqt6mz1VRJD6pmhRi0CXdmfnLu9dIUS6buzh60IvACM842Ffb3xd6Gg=="],
"@rollup/rollup-linux-arm64-gnu": ["@rollup/rollup-linux-arm64-gnu@4.60.1", "", { "os": "linux", "cpu": "arm64" }, "sha512-Nql7sTeAzhTAja3QXeAI48+/+GjBJ+QmAH13snn0AJSNL50JsDqotyudHyMbO2RbJkskbMbFJfIJKWA6R1LCJQ=="],
"@rollup/rollup-linux-arm64-musl": ["@rollup/rollup-linux-arm64-musl@4.60.1", "", { "os": "linux", "cpu": "arm64" }, "sha512-+pUymDhd0ys9GcKZPPWlFiZ67sTWV5UU6zOJat02M1+PiuSGDziyRuI/pPue3hoUwm2uGfxdL+trT6Z9rxnlMA=="],
"@rollup/rollup-linux-loong64-gnu": ["@rollup/rollup-linux-loong64-gnu@4.60.1", "", { "os": "linux", "cpu": "none" }, "sha512-VSvgvQeIcsEvY4bKDHEDWcpW4Yw7BtlKG1GUT4FzBUlEKQK0rWHYBqQt6Fm2taXS+1bXvJT6kICu5ZwqKCnvlQ=="],
"@rollup/rollup-linux-loong64-musl": ["@rollup/rollup-linux-loong64-musl@4.60.1", "", { "os": "linux", "cpu": "none" }, "sha512-4LqhUomJqwe641gsPp6xLfhqWMbQV04KtPp7/dIp0nzPxAkNY1AbwL5W0MQpcalLYk07vaW9Kp1PBhdpZYYcEw=="],
"@rollup/rollup-linux-ppc64-gnu": ["@rollup/rollup-linux-ppc64-gnu@4.60.1", "", { "os": "linux", "cpu": "ppc64" }, "sha512-tLQQ9aPvkBxOc/EUT6j3pyeMD6Hb8QF2BTBnCQWP/uu1lhc9AIrIjKnLYMEroIz/JvtGYgI9dF3AxHZNaEH0rw=="],
"@rollup/rollup-linux-ppc64-musl": ["@rollup/rollup-linux-ppc64-musl@4.60.1", "", { "os": "linux", "cpu": "ppc64" }, "sha512-RMxFhJwc9fSXP6PqmAz4cbv3kAyvD1etJFjTx4ONqFP9DkTkXsAMU4v3Vyc5BgzC+anz7nS/9tp4obsKfqkDHg=="],
"@rollup/rollup-linux-riscv64-gnu": ["@rollup/rollup-linux-riscv64-gnu@4.60.1", "", { "os": "linux", "cpu": "none" }, "sha512-QKgFl+Yc1eEk6MmOBfRHYF6lTxiiiV3/z/BRrbSiW2I7AFTXoBFvdMEyglohPj//2mZS4hDOqeB0H1ACh3sBbg=="],
"@rollup/rollup-linux-riscv64-musl": ["@rollup/rollup-linux-riscv64-musl@4.60.1", "", { "os": "linux", "cpu": "none" }, "sha512-RAjXjP/8c6ZtzatZcA1RaQr6O1TRhzC+adn8YZDnChliZHviqIjmvFwHcxi4JKPSDAt6Uhf/7vqcBzQJy0PDJg=="],
"@rollup/rollup-linux-s390x-gnu": ["@rollup/rollup-linux-s390x-gnu@4.60.1", "", { "os": "linux", "cpu": "s390x" }, "sha512-wcuocpaOlaL1COBYiA89O6yfjlp3RwKDeTIA0hM7OpmhR1Bjo9j31G1uQVpDlTvwxGn2nQs65fBFL5UFd76FcQ=="],
"@rollup/rollup-linux-x64-gnu": ["@rollup/rollup-linux-x64-gnu@4.60.1", "", { "os": "linux", "cpu": "x64" }, "sha512-77PpsFQUCOiZR9+LQEFg9GClyfkNXj1MP6wRnzYs0EeWbPcHs02AXu4xuUbM1zhwn3wqaizle3AEYg5aeoohhg=="],
"@rollup/rollup-linux-x64-musl": ["@rollup/rollup-linux-x64-musl@4.60.1", "", { "os": "linux", "cpu": "x64" }, "sha512-5cIATbk5vynAjqqmyBjlciMJl1+R/CwX9oLk/EyiFXDWd95KpHdrOJT//rnUl4cUcskrd0jCCw3wpZnhIHdD9w=="],
"@rollup/rollup-openbsd-x64": ["@rollup/rollup-openbsd-x64@4.60.1", "", { "os": "openbsd", "cpu": "x64" }, "sha512-cl0w09WsCi17mcmWqqglez9Gk8isgeWvoUZ3WiJFYSR3zjBQc2J5/ihSjpl+VLjPqjQ/1hJRcqBfLjssREQILw=="],
"@rollup/rollup-openharmony-arm64": ["@rollup/rollup-openharmony-arm64@4.60.1", "", { "os": "none", "cpu": "arm64" }, "sha512-4Cv23ZrONRbNtbZa37mLSueXUCtN7MXccChtKpUnQNgF010rjrjfHx3QxkS2PI7LqGT5xXyYs1a7LbzAwT0iCA=="],
"@rollup/rollup-win32-arm64-msvc": ["@rollup/rollup-win32-arm64-msvc@4.60.1", "", { "os": "win32", "cpu": "arm64" }, "sha512-i1okWYkA4FJICtr7KpYzFpRTHgy5jdDbZiWfvny21iIKky5YExiDXP+zbXzm3dUcFpkEeYNHgQ5fuG236JPq0g=="],
"@rollup/rollup-win32-ia32-msvc": ["@rollup/rollup-win32-ia32-msvc@4.60.1", "", { "os": "win32", "cpu": "ia32" }, "sha512-u09m3CuwLzShA0EYKMNiFgcjjzwqtUMLmuCJLeZWjjOYA3IT2Di09KaxGBTP9xVztWyIWjVdsB2E9goMjZvTQg=="],
"@rollup/rollup-win32-x64-gnu": ["@rollup/rollup-win32-x64-gnu@4.60.1", "", { "os": "win32", "cpu": "x64" }, "sha512-k+600V9Zl1CM7eZxJgMyTUzmrmhB/0XZnF4pRypKAlAgxmedUA+1v9R+XOFv56W4SlHEzfeMtzujLJD22Uz5zg=="],
"@rollup/rollup-win32-x64-msvc": ["@rollup/rollup-win32-x64-msvc@4.60.1", "", { "os": "win32", "cpu": "x64" }, "sha512-lWMnixq/QzxyhTV6NjQJ4SFo1J6PvOX8vUx5Wb4bBPsEb+8xZ89Bz6kOXpfXj9ak9AHTQVQzlgzBEc1SyM27xQ=="],
"@types/babel__core": ["@types/babel__core@7.20.5", "", { "dependencies": { "@babel/parser": "^7.20.7", "@babel/types": "^7.20.7", "@types/babel__generator": "*", "@types/babel__template": "*", "@types/babel__traverse": "*" } }, "sha512-qoQprZvz5wQFJwMDqeseRXWv3rqMvhgpbXFfVyWhbx9X47POIA6i/+dXefEmZKoAgOaTdaIgNSMqMIU61yRyzA=="],
"@types/babel__generator": ["@types/babel__generator@7.27.0", "", { "dependencies": { "@babel/types": "^7.0.0" } }, "sha512-ufFd2Xi92OAVPYsy+P4n7/U7e68fex0+Ee8gSG9KX7eo084CWiQ4sdxktvdl0bOPupXtVJPY19zk6EwWqUQ8lg=="],
"@types/babel__template": ["@types/babel__template@7.4.4", "", { "dependencies": { "@babel/parser": "^7.1.0", "@babel/types": "^7.0.0" } }, "sha512-h/NUaSyG5EyxBIp8YRxo4RMe2/qQgvyowRwVMzhYhBCONbW8PUsg4lkFMrhgZhUe5z3L3MiLDuvyJ/CaPa2A8A=="],
"@types/babel__traverse": ["@types/babel__traverse@7.28.0", "", { "dependencies": { "@babel/types": "^7.28.2" } }, "sha512-8PvcXf70gTDZBgt9ptxJ8elBeBjcLOAcOtoO/mPJjtji1+CdGbHgm77om1GrsPxsiE+uXIpNSK64UYaIwQXd4Q=="],
"@types/estree": ["@types/estree@1.0.8", "", {}, "sha512-dWHzHa2WqEXI/O1E9OjrocMTKJl2mSrEolh1Iomrv6U+JuNwaHXsXx9bLu5gG7BUWFIN0skIQJQ/L1rIex4X6w=="],
"@types/react": ["@types/react@19.2.14", "", { "dependencies": { "csstype": "^3.2.2" } }, "sha512-ilcTH/UniCkMdtexkoCN0bI7pMcJDvmQFPvuPvmEaYA/NSfFTAgdUSLAoVjaRJm7+6PvcM+q1zYOwS4wTYMF9w=="],
"@types/react-dom": ["@types/react-dom@19.2.3", "", { "peerDependencies": { "@types/react": "^19.2.0" } }, "sha512-jp2L/eY6fn+KgVVQAOqYItbF0VY/YApe5Mz2F0aykSO8gx31bYCZyvSeYxCHKvzHG5eZjc+zyaS5BrBWya2+kQ=="],
"@vitejs/plugin-react": ["@vitejs/plugin-react@4.7.0", "", { "dependencies": { "@babel/core": "^7.28.0", "@babel/plugin-transform-react-jsx-self": "^7.27.1", "@babel/plugin-transform-react-jsx-source": "^7.27.1", "@rolldown/pluginutils": "1.0.0-beta.27", "@types/babel__core": "^7.20.5", "react-refresh": "^0.17.0" }, "peerDependencies": { "vite": "^4.2.0 || ^5.0.0 || ^6.0.0 || ^7.0.0" } }, "sha512-gUu9hwfWvvEDBBmgtAowQCojwZmJ5mcLn3aufeCsitijs3+f2NsrPtlAWIR6OPiqljl96GVCUbLe0HyqIpVaoA=="],
"baseline-browser-mapping": ["baseline-browser-mapping@2.10.18", "", { "bin": { "baseline-browser-mapping": "dist/cli.cjs" } }, "sha512-VSnGQAOLtP5mib/DPyg2/t+Tlv65NTBz83BJBJvmLVHHuKJVaDOBvJJykiT5TR++em5nfAySPccDZDa4oSrn8A=="],
"browserslist": ["browserslist@4.28.2", "", { "dependencies": { "baseline-browser-mapping": "^2.10.12", "caniuse-lite": "^1.0.30001782", "electron-to-chromium": "^1.5.328", "node-releases": "^2.0.36", "update-browserslist-db": "^1.2.3" }, "bin": { "browserslist": "cli.js" } }, "sha512-48xSriZYYg+8qXna9kwqjIVzuQxi+KYWp2+5nCYnYKPTr0LvD89Jqk2Or5ogxz0NUMfIjhh2lIUX/LyX9B4oIg=="],
"caniuse-lite": ["caniuse-lite@1.0.30001788", "", {}, "sha512-6q8HFp+lOQtcf7wBK+uEenxymVWkGKkjFpCvw5W25cmMwEDU45p1xQFBQv8JDlMMry7eNxyBaR+qxgmTUZkIRQ=="],
"convert-source-map": ["convert-source-map@2.0.0", "", {}, "sha512-Kvp459HrV2FEJ1CAsi1Ku+MY3kasH19TFykTz2xWmMeq6bk2NU3XXvfJ+Q61m0xktWwt+1HSYf3JZsTms3aRJg=="],
"csstype": ["csstype@3.2.3", "", {}, "sha512-z1HGKcYy2xA8AGQfwrn0PAy+PB7X/GSj3UVJW9qKyn43xWa+gl5nXmU4qqLMRzWVLFC8KusUX8T/0kCiOYpAIQ=="],
"debug": ["debug@4.4.3", "", { "dependencies": { "ms": "^2.1.3" } }, "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA=="],
"electron-to-chromium": ["electron-to-chromium@1.5.336", "", {}, "sha512-AbH9q9J455r/nLmdNZes0G0ZKcRX73FicwowalLs6ijwOmCJSRRrLX63lcAlzy9ux3dWK1w1+1nsBJEWN11hcQ=="],
"esbuild": ["esbuild@0.25.12", "", { "optionalDependencies": { "@esbuild/aix-ppc64": "0.25.12", "@esbuild/android-arm": "0.25.12", "@esbuild/android-arm64": "0.25.12", "@esbuild/android-x64": "0.25.12", "@esbuild/darwin-arm64": "0.25.12", "@esbuild/darwin-x64": "0.25.12", "@esbuild/freebsd-arm64": "0.25.12", "@esbuild/freebsd-x64": "0.25.12", "@esbuild/linux-arm": "0.25.12", "@esbuild/linux-arm64": "0.25.12", "@esbuild/linux-ia32": "0.25.12", "@esbuild/linux-loong64": "0.25.12", "@esbuild/linux-mips64el": "0.25.12", "@esbuild/linux-ppc64": "0.25.12", "@esbuild/linux-riscv64": "0.25.12", "@esbuild/linux-s390x": "0.25.12", "@esbuild/linux-x64": "0.25.12", "@esbuild/netbsd-arm64": "0.25.12", "@esbuild/netbsd-x64": "0.25.12", "@esbuild/openbsd-arm64": "0.25.12", "@esbuild/openbsd-x64": "0.25.12", "@esbuild/openharmony-arm64": "0.25.12", "@esbuild/sunos-x64": "0.25.12", "@esbuild/win32-arm64": "0.25.12", "@esbuild/win32-ia32": "0.25.12", "@esbuild/win32-x64": "0.25.12" }, "bin": { "esbuild": "bin/esbuild" } }, "sha512-bbPBYYrtZbkt6Os6FiTLCTFxvq4tt3JKall1vRwshA3fdVztsLAatFaZobhkBC8/BrPetoa0oksYoKXoG4ryJg=="],
"escalade": ["escalade@3.2.0", "", {}, "sha512-WUj2qlxaQtO4g6Pq5c29GTcWGDyd8itL8zTlipgECz3JesAiiOKotd8JU6otB3PACgG6xkJUyVhboMS+bje/jA=="],
"fdir": ["fdir@6.5.0", "", { "peerDependencies": { "picomatch": "^3 || ^4" }, "optionalPeers": ["picomatch"] }, "sha512-tIbYtZbucOs0BRGqPJkshJUYdL+SDH7dVM8gjy+ERp3WAUjLEFJE+02kanyHtwjWOnwrKYBiwAmM0p4kLJAnXg=="],
"fsevents": ["fsevents@2.3.3", "", { "os": "darwin" }, "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw=="],
"gensync": ["gensync@1.0.0-beta.2", "", {}, "sha512-3hN7NaskYvMDLQY55gnW3NQ+mesEAepTqlg+VEbj7zzqEMBVNhzcGYYeqFo/TlYz6eQiFcp1HcsCZO+nGgS8zg=="],
"js-tokens": ["js-tokens@4.0.0", "", {}, "sha512-RdJUflcE3cUzKiMqQgsCu06FPu9UdIJO0beYbPhHN4k6apgJtifcoCtT9bcxOpYBtpD2kCM6Sbzg4CausW/PKQ=="],
"jsesc": ["jsesc@3.1.0", "", { "bin": { "jsesc": "bin/jsesc" } }, "sha512-/sM3dO2FOzXjKQhJuo0Q173wf2KOo8t4I8vHy6lF9poUp7bKT0/NHE8fPX23PwfhnykfqnC2xRxOnVw5XuGIaA=="],
"json5": ["json5@2.2.3", "", { "bin": { "json5": "lib/cli.js" } }, "sha512-XmOWe7eyHYH14cLdVPoyg+GOH3rYX++KpzrylJwSW98t3Nk+U8XOl8FWKOgwtzdb8lXGf6zYwDUzeHMWfxasyg=="],
"lru-cache": ["lru-cache@5.1.1", "", { "dependencies": { "yallist": "^3.0.2" } }, "sha512-KpNARQA3Iwv+jTA0utUVVbrh+Jlrr1Fv0e56GGzAFOXN7dk/FviaDW8LHmK52DlcH4WP2n6gI8vN1aesBFgo9w=="],
"ms": ["ms@2.1.3", "", {}, "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA=="],
"nanoid": ["nanoid@3.3.16", "", { "bin": { "nanoid": "bin/nanoid.cjs" } }, "sha512-bzlKTyNJ7+LdGIIwy8ijFpIqEQIvafahV7eYykJ8Cvh42EdJeODoJ6gUJXpQJvej1BddH8OqTXZNE/KfbWAu8Q=="],
"node-releases": ["node-releases@2.0.37", "", {}, "sha512-1h5gKZCF+pO/o3Iqt5Jp7wc9rH3eJJ0+nh/CIoiRwjRxde/hAHyLPXYN4V3CqKAbiZPSeJFSWHmJsbkicta0Eg=="],
"picocolors": ["picocolors@1.1.1", "", {}, "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA=="],
"picomatch": ["picomatch@4.0.4", "", {}, "sha512-QP88BAKvMam/3NxH6vj2o21R6MjxZUAd6nlwAS/pnGvN9IVLocLHxGYIzFhg6fUQ+5th6P4dv4eW9jX3DSIj7A=="],
"postcss": ["postcss@8.5.19", "", { "dependencies": { "nanoid": "^3.3.12", "picocolors": "^1.1.1", "source-map-js": "^1.2.1" } }, "sha512-Mz8SaolMd8nB+G13WkORcxQKHZ/NE4xXevtkJHVuG+guo9/wYKlIMTKAqGdEmYOXR2ijPjTYNHssizdaVSUNdQ=="],
"react": ["react@19.2.5", "", {}, "sha512-llUJLzz1zTUBrskt2pwZgLq59AemifIftw4aB7JxOqf1HY2FDaGDxgwpAPVzHU1kdWabH7FauP4i1oEeer2WCA=="],
"react-dom": ["react-dom@19.2.5", "", { "dependencies": { "scheduler": "^0.27.0" }, "peerDependencies": { "react": "^19.2.5" } }, "sha512-J5bAZz+DXMMwW/wV3xzKke59Af6CHY7G4uYLN1OvBcKEsWOs4pQExj86BBKamxl/Ik5bx9whOrvBlSDfWzgSag=="],
"react-refresh": ["react-refresh@0.17.0", "", {}, "sha512-z6F7K9bV85EfseRCp2bzrpyQ0Gkw1uLoCel9XBVWPg/TjRj94SkJzUTGfOa4bs7iJvBWtQG0Wq7wnI0syw3EBQ=="],
"rollup": ["rollup@4.60.1", "", { "dependencies": { "@types/estree": "1.0.8" }, "optionalDependencies": { "@rollup/rollup-android-arm-eabi": "4.60.1", "@rollup/rollup-android-arm64": "4.60.1", "@rollup/rollup-darwin-arm64": "4.60.1", "@rollup/rollup-darwin-x64": "4.60.1", "@rollup/rollup-freebsd-arm64": "4.60.1", "@rollup/rollup-freebsd-x64": "4.60.1", "@rollup/rollup-linux-arm-gnueabihf": "4.60.1", "@rollup/rollup-linux-arm-musleabihf": "4.60.1", "@rollup/rollup-linux-arm64-gnu": "4.60.1", "@rollup/rollup-linux-arm64-musl": "4.60.1", "@rollup/rollup-linux-loong64-gnu": "4.60.1", "@rollup/rollup-linux-loong64-musl": "4.60.1", "@rollup/rollup-linux-ppc64-gnu": "4.60.1", "@rollup/rollup-linux-ppc64-musl": "4.60.1", "@rollup/rollup-linux-riscv64-gnu": "4.60.1", "@rollup/rollup-linux-riscv64-musl": "4.60.1", "@rollup/rollup-linux-s390x-gnu": "4.60.1", "@rollup/rollup-linux-x64-gnu": "4.60.1", "@rollup/rollup-linux-x64-musl": "4.60.1", "@rollup/rollup-openbsd-x64": "4.60.1", "@rollup/rollup-openharmony-arm64": "4.60.1", "@rollup/rollup-win32-arm64-msvc": "4.60.1", "@rollup/rollup-win32-ia32-msvc": "4.60.1", "@rollup/rollup-win32-x64-gnu": "4.60.1", "@rollup/rollup-win32-x64-msvc": "4.60.1", "fsevents": "~2.3.2" }, "bin": { "rollup": "dist/bin/rollup" } }, "sha512-VmtB2rFU/GroZ4oL8+ZqXgSA38O6GR8KSIvWmEFv63pQ0G6KaBH9s07PO8XTXP4vI+3UJUEypOfjkGfmSBBR0w=="],
"scheduler": ["scheduler@0.27.0", "", {}, "sha512-eNv+WrVbKu1f3vbYJT/xtiF5syA5HPIMtf9IgY/nKg0sWqzAUEvqY/xm7OcZc/qafLx/iO9FgOmeSAp4v5ti/Q=="],
"semver": ["semver@6.3.1", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-BR7VvDCVHO+q2xBEWskxS6DJE1qRnb7DxzUrogb71CWoSficBxYsiAGd+Kl0mmq/MprG9yArRkyrQxTO6XjMzA=="],
"source-map-js": ["source-map-js@1.2.1", "", {}, "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA=="],
"tinyglobby": ["tinyglobby@0.2.16", "", { "dependencies": { "fdir": "^6.5.0", "picomatch": "^4.0.4" } }, "sha512-pn99VhoACYR8nFHhxqix+uvsbXineAasWm5ojXoN8xEwK5Kd3/TrhNn1wByuD52UxWRLy8pu+kRMniEi6Eq9Zg=="],
"typescript": ["typescript@5.9.3", "", { "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" } }, "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw=="],
"update-browserslist-db": ["update-browserslist-db@1.2.3", "", { "dependencies": { "escalade": "^3.2.0", "picocolors": "^1.1.1" }, "peerDependencies": { "browserslist": ">= 4.21.0" }, "bin": { "update-browserslist-db": "cli.js" } }, "sha512-Js0m9cx+qOgDxo0eMiFGEueWztz+d4+M3rGlmKPT+T4IS/jP4ylw3Nwpu6cpTTP8R1MAC1kF4VbdLt3ARf209w=="],
"vite": ["vite@6.4.3", "", { "dependencies": { "esbuild": "^0.25.0", "fdir": "^6.4.4", "picomatch": "^4.0.2", "postcss": "^8.5.3", "rollup": "^4.34.9", "tinyglobby": "^0.2.13" }, "optionalDependencies": { "fsevents": "~2.3.3" }, "peerDependencies": { "@types/node": "^18.0.0 || ^20.0.0 || >=22.0.0", "jiti": ">=1.21.0", "less": "*", "lightningcss": "^1.21.0", "sass": "*", "sass-embedded": "*", "stylus": "*", "sugarss": "*", "terser": "^5.16.0", "tsx": "^4.8.1", "yaml": "^2.4.2" }, "optionalPeers": ["@types/node", "jiti", "less", "lightningcss", "sass", "sass-embedded", "stylus", "sugarss", "terser", "tsx", "yaml"], "bin": { "vite": "bin/vite.js" } }, "sha512-NTKlcQjlAK7MlQoyb6LgaqHc8sso/pVyUJYWMws3jg21uTJw/LddqIFPcPqP6PzpgbIcZyKI85sFE4HBrQDA8A=="],
"yallist": ["yallist@3.1.1", "", {}, "sha512-a4UGQaWPH59mOXUYnAG2ewncQS4i4F43Tv3JoAM+s2VDAmS9NsK8GpDMLrCHPksFT7h3K6TOoUNn2pb7RoXx4g=="],
"@types/babel__core/@babel/parser": ["@babel/parser@7.29.2", "", { "dependencies": { "@babel/types": "^7.29.0" }, "bin": "./bin/babel-parser.js" }, "sha512-4GgRzy/+fsBa72/RZVJmGKPmZu9Byn8o4MoLpmNe1m8ZfYnz5emHLQz3U4gLud6Zwl0RZIcgiLD7Uq7ySFuDLA=="],
"@types/babel__core/@babel/types": ["@babel/types@7.29.0", "", { "dependencies": { "@babel/helper-string-parser": "^7.27.1", "@babel/helper-validator-identifier": "^7.28.5" } }, "sha512-LwdZHpScM4Qz8Xw2iKSzS+cfglZzJGvofQICy7W7v4caru4EaAmyUuO6BGrbyQ2mYV11W0U8j5mBhd14dd3B0A=="],
"@types/babel__generator/@babel/types": ["@babel/types@7.29.0", "", { "dependencies": { "@babel/helper-string-parser": "^7.27.1", "@babel/helper-validator-identifier": "^7.28.5" } }, "sha512-LwdZHpScM4Qz8Xw2iKSzS+cfglZzJGvofQICy7W7v4caru4EaAmyUuO6BGrbyQ2mYV11W0U8j5mBhd14dd3B0A=="],
"@types/babel__template/@babel/parser": ["@babel/parser@7.29.2", "", { "dependencies": { "@babel/types": "^7.29.0" }, "bin": "./bin/babel-parser.js" }, "sha512-4GgRzy/+fsBa72/RZVJmGKPmZu9Byn8o4MoLpmNe1m8ZfYnz5emHLQz3U4gLud6Zwl0RZIcgiLD7Uq7ySFuDLA=="],
"@types/babel__template/@babel/types": ["@babel/types@7.29.0", "", { "dependencies": { "@babel/helper-string-parser": "^7.27.1", "@babel/helper-validator-identifier": "^7.28.5" } }, "sha512-LwdZHpScM4Qz8Xw2iKSzS+cfglZzJGvofQICy7W7v4caru4EaAmyUuO6BGrbyQ2mYV11W0U8j5mBhd14dd3B0A=="],
"@types/babel__traverse/@babel/types": ["@babel/types@7.29.0", "", { "dependencies": { "@babel/helper-string-parser": "^7.27.1", "@babel/helper-validator-identifier": "^7.28.5" } }, "sha512-LwdZHpScM4Qz8Xw2iKSzS+cfglZzJGvofQICy7W7v4caru4EaAmyUuO6BGrbyQ2mYV11W0U8j5mBhd14dd3B0A=="],
"@types/babel__core/@babel/types/@babel/helper-string-parser": ["@babel/helper-string-parser@7.27.1", "", {}, "sha512-qMlSxKbpRlAridDExk92nSobyDdpPijUq2DW6oDnUqd0iOGxmQjyqhMIihI9+zv4LPyZdRje2cavWPbCbWm3eA=="],
"@types/babel__core/@babel/types/@babel/helper-validator-identifier": ["@babel/helper-validator-identifier@7.28.5", "", {}, "sha512-qSs4ifwzKJSV39ucNjsvc6WVHs6b7S03sOh2OcHF9UHfVPqWWALUsNUVzhSBiItjRZoLHx7nIarVjqKVusUZ1Q=="],
"@types/babel__generator/@babel/types/@babel/helper-string-parser": ["@babel/helper-string-parser@7.27.1", "", {}, "sha512-qMlSxKbpRlAridDExk92nSobyDdpPijUq2DW6oDnUqd0iOGxmQjyqhMIihI9+zv4LPyZdRje2cavWPbCbWm3eA=="],
"@types/babel__generator/@babel/types/@babel/helper-validator-identifier": ["@babel/helper-validator-identifier@7.28.5", "", {}, "sha512-qSs4ifwzKJSV39ucNjsvc6WVHs6b7S03sOh2OcHF9UHfVPqWWALUsNUVzhSBiItjRZoLHx7nIarVjqKVusUZ1Q=="],
"@types/babel__template/@babel/types/@babel/helper-string-parser": ["@babel/helper-string-parser@7.27.1", "", {}, "sha512-qMlSxKbpRlAridDExk92nSobyDdpPijUq2DW6oDnUqd0iOGxmQjyqhMIihI9+zv4LPyZdRje2cavWPbCbWm3eA=="],
"@types/babel__template/@babel/types/@babel/helper-validator-identifier": ["@babel/helper-validator-identifier@7.28.5", "", {}, "sha512-qSs4ifwzKJSV39ucNjsvc6WVHs6b7S03sOh2OcHF9UHfVPqWWALUsNUVzhSBiItjRZoLHx7nIarVjqKVusUZ1Q=="],
"@types/babel__traverse/@babel/types/@babel/helper-string-parser": ["@babel/helper-string-parser@7.27.1", "", {}, "sha512-qMlSxKbpRlAridDExk92nSobyDdpPijUq2DW6oDnUqd0iOGxmQjyqhMIihI9+zv4LPyZdRje2cavWPbCbWm3eA=="],
"@types/babel__traverse/@babel/types/@babel/helper-validator-identifier": ["@babel/helper-validator-identifier@7.28.5", "", {}, "sha512-qSs4ifwzKJSV39ucNjsvc6WVHs6b7S03sOh2OcHF9UHfVPqWWALUsNUVzhSBiItjRZoLHx7nIarVjqKVusUZ1Q=="],
}
}
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
-16
View File
@@ -1,16 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>GBrain Admin</title>
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600&family=JetBrains+Mono:wght@400;500&display=swap" rel="stylesheet" />
<script type="module" crossorigin src="/admin/assets/index-CviJXT-1.js"></script>
<link rel="stylesheet" crossorigin href="/admin/assets/index-GxkWX7v3.css">
</head>
<body>
<div id="root"></div>
</body>
</html>
-15
View File
@@ -1,15 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>GBrain Admin</title>
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600&family=JetBrains+Mono:wght@400;500&display=swap" rel="stylesheet" />
</head>
<body>
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
</html>
-25
View File
@@ -1,25 +0,0 @@
{
"name": "gbrain-admin",
"private": true,
"type": "module",
"scripts": {
"dev": "vite",
"build": "vite build",
"preview": "vite preview"
},
"dependencies": {
"react": "^19.1.0",
"react-dom": "^19.1.0"
},
"devDependencies": {
"@types/react": "^19.1.2",
"@types/react-dom": "^19.1.2",
"@vitejs/plugin-react": "^4.4.1",
"vite": "^6.4.3",
"typescript": "^5.8.3"
},
"overrides": {
"@babel/core": "^7.29.6",
"postcss": "^8.5.10"
}
}
-92
View File
@@ -1,92 +0,0 @@
import React, { useState, useEffect } from 'react';
import { LoginPage } from './pages/Login';
import { DashboardPage } from './pages/Dashboard';
import { AgentsPage } from './pages/Agents';
import { RequestLogPage } from './pages/RequestLog';
import { CalibrationPage } from './pages/Calibration';
import { JobsWatchPage } from './pages/JobsWatch';
import { api } from './api';
type Page = 'login' | 'dashboard' | 'agents' | 'log' | 'calibration' | 'jobs';
function getPage(): Page {
const hash = window.location.hash.replace('#', '') || 'dashboard';
if (['login', 'dashboard', 'agents', 'log', 'calibration', 'jobs'].includes(hash)) return hash as Page;
return 'dashboard';
}
export function App() {
const [page, setPage] = useState<Page>(getPage);
useEffect(() => {
const onHash = () => setPage(getPage());
window.addEventListener('hashchange', onHash);
return () => window.removeEventListener('hashchange', onHash);
}, []);
const navigate = (p: Page) => {
window.location.hash = p;
setPage(p);
};
if (page === 'login') {
return <LoginPage onLogin={() => navigate('dashboard')} />;
}
const handleSignOutEverywhere = async () => {
if (!confirm('Sign out every active admin session, including other browsers and tabs? Each one will need to re-authenticate via a fresh magic link.')) {
return;
}
try {
await api.signOutEverywhere();
} catch {
// Even if the call fails, push to login — cookie is likely already invalid.
}
navigate('login');
};
return (
<div className="app">
<nav className="sidebar">
<div className="sidebar-logo">GBrain</div>
<div className="sidebar-nav">
<a className={`nav-item ${page === 'dashboard' ? 'active' : ''}`}
onClick={() => navigate('dashboard')}>Dashboard</a>
<a className={`nav-item ${page === 'agents' ? 'active' : ''}`}
onClick={() => navigate('agents')}>Agents</a>
<a className={`nav-item ${page === 'log' ? 'active' : ''}`}
onClick={() => navigate('log')}>Request Log</a>
<a className={`nav-item ${page === 'calibration' ? 'active' : ''}`}
onClick={() => navigate('calibration')}>Calibration</a>
<a className={`nav-item ${page === 'jobs' ? 'active' : ''}`}
onClick={() => navigate('jobs')}>Jobs Watch</a>
</div>
<div style={{ marginTop: 'auto', padding: '16px 12px', borderTop: '1px solid var(--border)' }}>
<button
onClick={handleSignOutEverywhere}
style={{
background: 'transparent',
border: '1px solid var(--border)',
color: 'var(--text-secondary)',
padding: '6px 10px',
borderRadius: 6,
fontSize: 12,
cursor: 'pointer',
width: '100%',
}}
title="Revoke every active admin session — every browser, every tab"
>
Sign out everywhere
</button>
</div>
</nav>
<main className="main">
{page === 'dashboard' && <DashboardPage />}
{page === 'agents' && <AgentsPage />}
{page === 'log' && <RequestLogPage />}
{page === 'calibration' && <CalibrationPage />}
{page === 'jobs' && <JobsWatchPage />}
</main>
</div>
);
}
-65
View File
@@ -1,65 +0,0 @@
const BASE = '';
// v0.26.3 trust model (D11 + D12): the admin UI does NOT cache the
// bootstrap token in browser JS state. On 401, redirect to login —
// no auto-reauth via saved token, no localStorage/sessionStorage read.
// The HttpOnly cookie set by /admin/login is the only session credential.
async function apiFetch(path: string, options?: RequestInit) {
const res = await fetch(`${BASE}${path}`, {
...options,
credentials: 'same-origin',
headers: { 'Content-Type': 'application/json', ...options?.headers },
});
if (res.status === 401) {
// No token cache to retry from. Redirect to login.
window.location.hash = '#login';
throw new Error('Unauthorized');
}
if (!res.ok) {
const body = await res.json().catch(() => ({}));
throw new Error(body.error || `HTTP ${res.status}`);
}
return res.json();
}
// v0.36.1.0 (T15 / E6) — SVG fetch (text/plain payload, NOT JSON).
async function apiFetchText(path: string) {
const res = await fetch(`${BASE}${path}`, { credentials: 'same-origin' });
if (res.status === 401) {
window.location.hash = '#login';
throw new Error('Unauthorized');
}
if (!res.ok) throw new Error(`HTTP ${res.status}`);
return res.text();
}
export const api = {
login: (token: string) => apiFetch('/admin/login', { method: 'POST', body: JSON.stringify({ token }) }),
signOutEverywhere: () => apiFetch('/admin/api/sign-out-everywhere', { method: 'POST' }),
stats: () => apiFetch('/admin/api/stats'),
health: () => apiFetch('/admin/api/health-indicators'),
agents: () => apiFetch('/admin/api/agents'),
sources: () => apiFetch('/admin/api/sources'),
requests: (page = 1, qs = '') => apiFetch(`/admin/api/requests?page=${page}${qs}`),
apiKeys: () => apiFetch('/admin/api/api-keys'),
createApiKey(keyName: string) {
return apiFetch('/admin/api/api-keys', { method: 'POST', body: JSON.stringify({ name: keyName }) });
},
revokeApiKey(keyName: string) {
return apiFetch('/admin/api/api-keys/revoke', { method: 'POST', body: JSON.stringify({ name: keyName }) });
},
updateClientTtl: (clientId: string, tokenTtl: number | null) => apiFetch('/admin/api/update-client-ttl', { method: 'POST', body: JSON.stringify({ clientId, tokenTtl }) }),
rescopeClient: (clientId: string, sourceId: string, federatedRead: string[]) =>
apiFetch('/admin/api/rescope-client', {
method: 'POST',
body: JSON.stringify({ clientId, sourceId, federatedRead }),
}),
revokeClient: (clientId: string) => apiFetch('/admin/api/revoke-client', { method: 'POST', body: JSON.stringify({ clientId }) }),
// v0.36.1.0 (T15 / E6) — calibration endpoints.
calibrationProfile: (holder?: string) =>
apiFetch(`/admin/api/calibration/profile${holder ? `?holder=${encodeURIComponent(holder)}` : ''}`),
calibrationChart: (type: string, holder?: string) =>
apiFetchText(`/admin/api/calibration/charts/${encodeURIComponent(type)}${holder ? `?holder=${encodeURIComponent(holder)}` : ''}`),
// v0.41 D2 — live minion-jobs dashboard snapshot.
jobsWatch: () => apiFetch('/admin/api/jobs/watch'),
};
-359
View File
@@ -1,359 +0,0 @@
:root {
--bg-primary: #0a0a0f;
--bg-secondary: #14141f;
--bg-tertiary: #1e1e2e;
--text-primary: #e0e0e0;
--text-secondary: #888;
/* v0.36.1.0 TD2 bumped from #555 (contrast 4.0 on #0a0a0f bg, below WCAG AA
4.5 for body text) to #777 (contrast ~5.5, passes AA). Applies globally
to Dashboard, Agents, RequestLog, and the new Calibration tab. */
--text-muted: #777;
--accent: #3b82f6;
--success: #22c55e;
--warning: #f59e0b;
--error: #ef4444;
--font-mono: 'JetBrains Mono', monospace;
--font-sans: 'Inter', system-ui, sans-serif;
}
* { margin: 0; padding: 0; box-sizing: border-box; }
body {
font-family: var(--font-sans);
background: var(--bg-primary);
color: var(--text-primary);
font-size: 14px;
line-height: 1.5;
}
/* Layout */
.app { display: flex; min-height: 100vh; }
.sidebar {
width: 200px;
background: var(--bg-secondary);
border-right: 1px solid #1e1e2e;
padding: 16px 0;
flex-shrink: 0;
display: flex;
flex-direction: column;
}
.sidebar-logo {
font-size: 18px;
font-weight: 600;
padding: 0 16px 24px;
color: var(--text-primary);
}
.sidebar-nav { display: flex; flex-direction: column; gap: 2px; }
.nav-item {
display: flex;
align-items: center;
gap: 8px;
padding: 8px 16px;
color: var(--text-secondary);
text-decoration: none;
font-size: 13px;
cursor: pointer;
border-left: 3px solid transparent;
transition: all 0.15s;
}
.nav-item:hover { background: var(--bg-tertiary); color: var(--text-primary); }
.nav-item.active {
border-left-color: var(--accent);
background: var(--bg-tertiary);
color: var(--text-primary);
}
.main { flex: 1; padding: 24px 32px; overflow-y: auto; }
.page-title {
font-size: 24px;
font-weight: 600;
margin-bottom: 24px;
}
/* Metrics bar */
.metrics { display: flex; gap: 16px; margin-bottom: 24px; }
.metric {
background: var(--bg-secondary);
padding: 16px 20px;
border-radius: 6px;
min-width: 140px;
}
.metric-value {
font-family: var(--font-mono);
font-size: 28px;
font-weight: 500;
}
.metric-label { font-size: 12px; color: var(--text-secondary); margin-top: 4px; }
/* Tables */
table { width: 100%; border-collapse: collapse; }
th {
text-align: left;
font-size: 11px;
text-transform: uppercase;
color: var(--text-muted);
padding: 8px 12px;
font-weight: 500;
letter-spacing: 0.5px;
}
td {
padding: 10px 12px;
font-size: 13px;
border-top: 1px solid #1a1a2a;
}
tr:hover td { background: var(--bg-tertiary); }
/* Badges */
.badge {
display: inline-block;
padding: 2px 8px;
border-radius: 10px;
font-size: 11px;
font-weight: 500;
}
.badge-read { background: rgba(59,130,246,0.15); color: var(--accent); }
.badge-write { background: rgba(245,158,11,0.15); color: var(--warning); }
.badge-admin { background: rgba(239,68,68,0.15); color: var(--error); }
.badge-success { background: rgba(34,197,94,0.15); color: var(--success); }
.badge-error { background: rgba(239,68,68,0.15); color: var(--error); }
/* Status dots */
.status-dot {
display: inline-block;
width: 8px;
height: 8px;
border-radius: 50%;
margin-right: 6px;
}
.status-active { background: var(--success); }
.status-warning { background: var(--warning); }
.status-inactive { background: var(--text-muted); }
/* Buttons */
.btn {
padding: 8px 16px;
border-radius: 6px;
font-size: 13px;
font-weight: 500;
cursor: pointer;
border: none;
transition: all 0.15s;
}
.btn-primary { background: var(--accent); color: white; }
.btn-primary:hover { background: #2563eb; }
.btn-secondary { background: transparent; color: var(--text-secondary); border: 1px solid #333; }
.btn-secondary:hover { border-color: var(--text-secondary); color: var(--text-primary); }
.btn-danger { background: transparent; color: var(--error); border: 1px solid var(--error); }
.btn-danger:hover { background: rgba(239,68,68,0.1); }
/* Forms */
input, select {
background: var(--bg-primary);
border: 1px solid #333;
color: var(--text-primary);
padding: 8px 12px;
border-radius: 6px;
font-size: 13px;
font-family: var(--font-sans);
width: 100%;
}
input:focus, select:focus {
outline: none;
border-color: var(--accent);
box-shadow: 0 0 0 2px rgba(59,130,246,0.2);
}
input::placeholder { color: var(--text-muted); }
label { display: block; font-size: 13px; font-weight: 500; margin-bottom: 6px; }
/* Modal */
.modal-overlay {
position: fixed;
inset: 0;
background: rgba(0,0,0,0.7);
display: flex;
align-items: center;
justify-content: center;
z-index: 100;
}
.modal {
background: var(--bg-secondary);
border-radius: 8px;
padding: 24px;
min-width: 420px;
max-width: 520px;
}
.modal-title { font-size: 18px; font-weight: 600; margin-bottom: 20px; }
/* Drawer */
.drawer-overlay {
position: fixed;
inset: 0;
background: rgba(0,0,0,0.5);
z-index: 90;
}
.drawer {
position: fixed;
right: 0;
top: 0;
bottom: 0;
width: 420px;
background: var(--bg-secondary);
border-left: 1px solid var(--accent);
padding: 24px;
z-index: 91;
overflow-y: auto;
}
.drawer-close {
position: absolute;
top: 16px;
right: 16px;
background: none;
border: none;
color: var(--text-muted);
font-size: 18px;
cursor: pointer;
}
/* Section headers */
.section-title {
font-size: 11px;
text-transform: uppercase;
color: var(--text-muted);
letter-spacing: 0.5px;
margin: 20px 0 12px;
font-weight: 500;
}
/* Health panel */
.health-panel {
background: var(--bg-secondary);
border-radius: 6px;
padding: 16px;
}
.health-row {
display: flex;
justify-content: space-between;
padding: 6px 0;
font-size: 13px;
}
/* Code block */
.code-block {
background: var(--bg-primary);
border-radius: 6px;
padding: 12px;
font-family: var(--font-mono);
font-size: 12px;
overflow-x: auto;
position: relative;
}
.code-block .copy-btn {
position: absolute;
top: 8px;
right: 8px;
background: var(--accent);
color: white;
border: none;
padding: 4px 10px;
border-radius: 4px;
font-size: 11px;
cursor: pointer;
}
/* Activity feed */
.feed { max-height: 400px; overflow-y: auto; }
.feed-empty {
color: var(--text-muted);
text-align: center;
padding: 32px;
font-size: 13px;
}
/* Sparkline */
.sparkline { display: inline-block; vertical-align: middle; }
/* Filter bar */
.filter-bar { display: flex; gap: 12px; margin-bottom: 16px; align-items: center; }
.filter-bar select { width: auto; min-width: 140px; }
/* Pagination */
.pagination {
display: flex;
justify-content: space-between;
align-items: center;
padding: 12px 0;
font-size: 13px;
color: var(--text-secondary);
}
.pagination button {
background: var(--bg-secondary);
border: 1px solid #333;
color: var(--text-primary);
padding: 6px 12px;
border-radius: 4px;
cursor: pointer;
font-size: 12px;
}
.pagination button:disabled { opacity: 0.3; cursor: default; }
/* Warning bar */
.warning-bar {
background: rgba(245,158,11,0.15);
border: 1px solid var(--warning);
color: var(--warning);
padding: 10px 16px;
border-radius: 6px;
font-size: 13px;
margin: 12px 0;
}
/* Checkbox */
.checkbox-group { display: flex; gap: 16px; flex-wrap: wrap; }
.checkbox-label {
display: flex;
align-items: center;
gap: 6px;
font-size: 13px;
cursor: pointer;
}
/* Tabs */
.tabs { display: flex; gap: 0; margin-bottom: 12px; }
.tab {
padding: 6px 12px;
font-size: 13px;
color: var(--text-secondary);
cursor: pointer;
border-bottom: 2px solid transparent;
}
.tab.active { color: var(--accent); border-bottom-color: var(--accent); }
/* Login page */
.login-page {
display: flex;
align-items: center;
justify-content: center;
min-height: 100vh;
background: var(--bg-primary);
}
.login-box { text-align: left; width: 340px; }
.login-logo { font-size: 32px; font-weight: 600; margin-bottom: 32px; }
.login-hint { color: var(--text-muted); font-size: 12px; margin-top: 12px; }
.login-error { color: var(--error); font-size: 13px; margin-top: 8px; }
/* Monospace data */
.mono { font-family: var(--font-mono); font-size: 12px; }
/* Responsive */
@media (max-width: 768px) {
.sidebar { display: none; }
.main { padding: 16px; }
.metrics { flex-wrap: wrap; }
.drawer { width: 100%; }
}
-25
View File
@@ -1,25 +0,0 @@
/**
* Admin SPA scope constants HAND-MAINTAINED MIRROR of src/core/scope.ts.
*
* The admin tsconfig.json scopes `include: ['src']` to admin/src/, so we
* cannot directly import from ../../src/core/scope.ts without breaking the
* SPA's compile boundary. Instead, this file is a hand-maintained duplicate;
* scripts/check-admin-scope-drift.sh fails the build if the two lists drift.
*
* If you change ALLOWED_SCOPES in src/core/scope.ts, update this file too,
* or `bun run verify` will reject the change.
*/
export type Scope = 'read' | 'write' | 'admin' | 'sources_admin' | 'users_admin' | 'agent';
// MIRROR OF src/core/scope.ts ALLOWED_SCOPES_LIST — keep alphabetically sorted.
// v0.38: 'agent' added for the submit_agent remote-MCP op (sibling to admin,
// NOT implied — existing admin clients must re-register to opt in).
export const ALLOWED_SCOPES_LIST: ReadonlyArray<Scope> = [
'admin',
'agent',
'read',
'sources_admin',
'users_admin',
'write',
];
-10
View File
@@ -1,10 +0,0 @@
import React from 'react';
import ReactDOM from 'react-dom/client';
import { App } from './App';
import './index.css';
ReactDOM.createRoot(document.getElementById('root')!).render(
<React.StrictMode>
<App />
</React.StrictMode>,
);
-798
View File
@@ -1,798 +0,0 @@
import React, { useState, useEffect } from 'react';
import { api } from '../api';
import { ALLOWED_SCOPES_LIST, type Scope } from '../lib/scope-constants';
function timeAgo(date: Date): string {
const s = Math.floor((Date.now() - date.getTime()) / 1000);
if (s < 60) return 'just now';
if (s < 3600) return `${Math.floor(s / 60)}m ago`;
if (s < 86400) return `${Math.floor(s / 3600)}h ago`;
return `${Math.floor(s / 86400)}d ago`;
}
interface Agent {
id: string;
name: string;
auth_type: 'oauth' | 'api_key';
client_id?: string; // compat
client_name?: string; // compat
grant_types: string[];
scope: string;
source_id: string | null;
federated_read: string[];
created_at: string;
last_used_at: string | null;
total_requests: number;
requests_today: number;
token_ttl: number | null;
status: 'active' | 'revoked';
}
interface Source {
id: string;
name: string;
federated: boolean;
}
interface ApiKey {
id: string;
name: string;
created_at: string;
last_used_at: string | null;
status: 'active' | 'revoked';
}
export function AgentsPage() {
const [agents, setAgents] = useState<Agent[]>([]);
const [sources, setSources] = useState<Source[]>([]);
const [hideRevoked, setHideRevoked] = useState(true);
const [showRegister, setShowRegister] = useState(false);
const [showCredentials, setShowCredentials] = useState<{ clientId: string; clientSecret: string; name: string } | null>(null);
const [showApiKeyCreate, setShowApiKeyCreate] = useState(false);
const [showApiKeyToken, setShowApiKeyToken] = useState<{ name: string; token: string } | null>(null);
const [selectedAgent, setSelectedAgent] = useState<Agent | null>(null);
useEffect(() => {
loadAgents();
api.sources().then(setSources).catch(() => {});
}, []);
const loadAgents = () => { api.agents().then(setAgents).catch(() => {}); };
return (
<>
<div style={{ display: 'flex', justifyContent: 'space-between', alignItems: 'center', marginBottom: 24 }}>
<h1 className="page-title" style={{ marginBottom: 0 }}>Agents</h1>
<div style={{ display: 'flex', gap: 8, alignItems: 'center' }}>
<label style={{ fontSize: 13, color: 'var(--text-secondary)', display: 'flex', alignItems: 'center', gap: 6, cursor: 'pointer' }}>
<input type="checkbox" checked={hideRevoked} onChange={e => setHideRevoked(e.target.checked)} /> Hide revoked
</label>
<button className="btn btn-secondary" onClick={() => setShowApiKeyCreate(true)}>+ API Key</button>
<button className="btn btn-primary" onClick={() => setShowRegister(true)}>+ OAuth Client</button>
</div>
</div>
{(() => {
// Filter once and reuse, so the empty-state guard sees the same
// rows the table renders. Pre-fix: agents.length === 0 used the
// unfiltered array, so an all-revoked dataset with hideRevoked=on
// showed a header-only table with no placeholder.
const visibleAgents = agents.filter(a => !hideRevoked || a.status !== 'revoked');
if (agents.length === 0) {
return (
<div style={{ textAlign: 'center', padding: 48, color: 'var(--text-muted)' }}>
No agents registered. Register your first agent to get started.
</div>
);
}
if (visibleAgents.length === 0) {
return (
<div style={{ textAlign: 'center', padding: 48, color: 'var(--text-muted)' }}>
All agents are revoked. Uncheck "Hide revoked" to view them.
</div>
);
}
return (
<>
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Scopes</th>
<th>Sources</th>
<th>Status</th>
<th>Requests</th>
<th>Last Used</th>
</tr>
</thead>
<tbody>
{visibleAgents.map(a => (
<tr key={a.id} onClick={() => setSelectedAgent(a)}
style={{ cursor: 'pointer' }}>
<td style={{ fontWeight: 500 }}>{a.name || a.client_name}</td>
<td>
<span className={`badge ${a.auth_type === 'oauth' ? 'badge-read' : 'badge-write'}`} style={{ fontSize: 11 }}>
{a.auth_type === 'oauth' ? 'OAuth' : 'API Key'}
</span>
</td>
<td>
{(a.scope || '').split(' ').filter(Boolean).map(s => (
<span key={s} className={`badge badge-${s}`} style={{ marginRight: 4 }}>{s}</span>
))}
</td>
<td style={{ color: 'var(--text-secondary)', fontSize: 12 }}>
{a.auth_type === 'oauth'
? `${a.source_id || 'none'} · ${(a.federated_read || []).length} readable`
: 'Unscoped'}
</td>
<td>
<span className={`badge ${a.status === 'active' ? 'badge-success' : 'badge-danger'}`}>{a.status}</span>
</td>
<td>
<span style={{ fontWeight: 500 }}>{a.requests_today || 0}</span>
<span style={{ color: 'var(--text-muted)', fontSize: 12 }}> / {a.total_requests || 0}</span>
</td>
<td style={{ color: 'var(--text-secondary)' }}>
{a.last_used_at ? timeAgo(new Date(a.last_used_at)) : 'Never'}
</td>
</tr>
))}
</tbody>
</table>
<div style={{ color: 'var(--text-muted)', fontSize: 13, marginTop: 12 }}>
{agents.filter(a => a.status === 'active').length} active / {agents.length} total
</div>
</>
);
})()}
{showRegister && (
<RegisterModal
onClose={() => setShowRegister(false)}
onRegistered={(creds) => { setShowRegister(false); setShowCredentials(creds); loadAgents(); }}
/>
)}
{showCredentials && (
<CredentialsModal
credentials={showCredentials}
onClose={() => setShowCredentials(null)}
/>
)}
{selectedAgent && (
<AgentDrawer
key={selectedAgent.id}
agent={selectedAgent}
sources={sources}
onClose={() => setSelectedAgent(null)}
onRevoked={loadAgents}
onRescoped={({ sourceId, federatedRead }) => {
setSelectedAgent(current => current ? {
...current,
source_id: sourceId,
federated_read: federatedRead,
} : current);
loadAgents();
}}
/>
)}
{showApiKeyCreate && (
<ApiKeyCreateModal
onClose={() => setShowApiKeyCreate(false)}
onCreated={(result) => { setShowApiKeyCreate(false); setShowApiKeyToken(result); loadAgents(); }}
/>
)}
{showApiKeyToken && (
<ApiKeyTokenModal token={showApiKeyToken} onClose={() => setShowApiKeyToken(null)} />
)}
</>
);
}
function ApiKeyCreateModal({ onClose, onCreated }: {
onClose: () => void;
onCreated: (result: { name: string; token: string }) => void;
}) {
const [name, setName] = useState('');
const [loading, setLoading] = useState(false);
const [error, setError] = useState('');
const handleSubmit = async (e: React.FormEvent) => {
e.preventDefault();
if (!name.trim()) { setError('Name required'); return; }
setLoading(true);
try {
const data = await api.createApiKey(name.trim());
onCreated({ name: data.name, token: data.token });
} catch (err) {
setError(err instanceof Error ? err.message : 'Failed');
} finally { setLoading(false); }
};
return (
<div className="modal-overlay" onClick={onClose}>
<form className="modal" onClick={e => e.stopPropagation()} onSubmit={handleSubmit}>
<div className="modal-title">Create API Key</div>
<p style={{ color: 'var(--text-secondary)', fontSize: 13, marginBottom: 16 }}>
API keys use simple bearer token auth. They grant full read+write+admin access.
For scoped access, use OAuth clients instead.
</p>
<div style={{ marginBottom: 16 }}>
<label>Key Name</label>
<input placeholder="e.g. claude-code-local" value={name} onChange={e => setName(e.target.value)} autoFocus />
</div>
{error && <div style={{ color: 'var(--error)', fontSize: 13, marginBottom: 12 }}>{error}</div>}
<div style={{ display: 'flex', gap: 12, justifyContent: 'flex-end' }}>
<button type="button" className="btn btn-secondary" onClick={onClose}>Cancel</button>
<button type="submit" className="btn btn-primary" disabled={loading}>
{loading ? 'Creating...' : 'Create Key'}
</button>
</div>
</form>
</div>
);
}
function ApiKeyTokenModal({ token, onClose }: {
token: { name: string; token: string };
onClose: () => void;
}) {
const copy = (text: string) => navigator.clipboard.writeText(text);
return (
<div className="modal-overlay">
<div className="modal" style={{ maxWidth: 560 }}>
<div style={{ textAlign: 'center', marginBottom: 16 }}>
<div style={{ fontSize: 36, color: 'var(--success)', marginBottom: 8 }}>&#10003;</div>
<div style={{ fontSize: 20, fontWeight: 600 }}>API Key Created</div>
</div>
<div style={{ marginBottom: 12 }}>
<label style={{ fontSize: 12 }}>Name</label>
<div className="code-block"><span>{token.name}</span></div>
</div>
<div style={{ marginBottom: 12 }}>
<label style={{ fontSize: 12 }}>Bearer Token</label>
<div className="code-block">
<span>{token.token}</span>
<button className="copy-btn" onClick={() => copy(token.token)}>Copy</button>
</div>
</div>
<div style={{ marginBottom: 12 }}>
<label style={{ fontSize: 12 }}>Usage</label>
<div className="code-block">
<pre style={{ whiteSpace: 'pre-wrap', margin: 0, fontSize: 12 }}>{`Authorization: Bearer ${token.token}`}</pre>
<button className="copy-btn" onClick={() => copy(`Authorization: Bearer ${token.token}`)}>Copy</button>
</div>
</div>
<div className="warning-bar">Save this token now. It will not be shown again.</div>
<div style={{ display: 'flex', gap: 12, justifyContent: 'flex-end', marginTop: 20 }}>
<button className="btn btn-primary" onClick={onClose}>Done</button>
</div>
</div>
</div>
);
}
function RegisterModal({ onClose, onRegistered }: {
onClose: () => void;
onRegistered: (creds: { clientId: string; clientSecret: string; name: string }) => void;
}) {
const [name, setName] = useState('');
// v0.28: scope set sourced from admin/src/lib/scope-constants.ts (mirror
// of src/core/scope.ts). CI drift check at scripts/check-admin-scope-drift.sh
// fails the build if these diverge.
const [scopes, setScopes] = useState<Record<Scope, boolean>>(() =>
Object.fromEntries(ALLOWED_SCOPES_LIST.map(s => [s, s === 'read'])) as Record<Scope, boolean>,
);
const [ttl, setTtl] = useState('86400'); // 24h default
const [loading, setLoading] = useState(false);
const [error, setError] = useState('');
const ttlOptions = [
{ label: '1 hour', value: '3600' },
{ label: '24 hours', value: '86400' },
{ label: '7 days', value: '604800' },
{ label: '30 days', value: '2592000' },
{ label: '1 year', value: '31536000' },
{ label: 'No expiry', value: '0' },
];
const handleSubmit = async (e: React.FormEvent) => {
e.preventDefault();
if (!name.trim()) { setError('Name required'); return; }
setLoading(true);
setError('');
try {
// Use the CLI registration endpoint (POST to admin API)
const selectedScopes = Object.entries(scopes).filter(([, v]) => v).map(([k]) => k).join(' ');
const res = await fetch('/admin/api/register-client', {
method: 'POST',
credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ name: name.trim(), scopes: selectedScopes, tokenTtl: ttl === '0' ? 315360000 : Number(ttl) }),
});
if (!res.ok) throw new Error('Registration failed');
const data = await res.json();
onRegistered({ clientId: data.clientId, clientSecret: data.clientSecret, name: name.trim() });
} catch (err) {
setError(err instanceof Error ? err.message : 'Registration failed');
} finally {
setLoading(false);
}
};
return (
<div className="modal-overlay" onClick={onClose}>
<form className="modal" onClick={e => e.stopPropagation()} onSubmit={handleSubmit}>
<div className="modal-title">Register Agent</div>
<div style={{ marginBottom: 16 }}>
<label>Agent Name</label>
<input placeholder="e.g. perplexity-production" value={name} onChange={e => setName(e.target.value)} autoFocus />
</div>
<div style={{ marginBottom: 16 }}>
<label>Scopes</label>
<div className="checkbox-group">
{ALLOWED_SCOPES_LIST.map(s => (
<label key={s} className="checkbox-label">
<input type="checkbox" checked={scopes[s]} onChange={e => setScopes(p => ({ ...p, [s]: e.target.checked }))} />
{s}
</label>
))}
</div>
</div>
<div style={{ marginBottom: 20 }}>
<label>Token Lifetime</label>
<select value={ttl} onChange={e => setTtl(e.target.value)}
style={{ width: '100%', background: 'var(--bg-secondary)', color: 'var(--text-primary)', border: '1px solid var(--border)', borderRadius: 6, padding: '6px 10px', fontSize: 14 }}>
{ttlOptions.map(o => <option key={o.value} value={o.value}>{o.label}</option>)}
</select>
</div>
{error && <div style={{ color: 'var(--error)', fontSize: 13, marginBottom: 12 }}>{error}</div>}
<div style={{ display: 'flex', gap: 12, justifyContent: 'flex-end' }}>
<button type="button" className="btn btn-secondary" onClick={onClose}>Cancel</button>
<button type="submit" className="btn btn-primary" disabled={loading}>
{loading ? 'Registering...' : 'Register'}
</button>
</div>
</form>
</div>
);
}
function CredentialsModal({ credentials, onClose }: {
credentials: { clientId: string; clientSecret: string; name: string };
onClose: () => void;
}) {
const copy = (text: string) => navigator.clipboard.writeText(text);
const downloadJson = () => {
const blob = new Blob([JSON.stringify(credentials, null, 2)], { type: 'application/json' });
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url; a.download = `${credentials.name}-credentials.json`; a.click();
URL.revokeObjectURL(url);
};
return (
<div className="modal-overlay">
<div className="modal" style={{ maxWidth: 560 }}>
<div style={{ textAlign: 'center', marginBottom: 16 }}>
<div style={{ fontSize: 36, color: 'var(--success)', marginBottom: 8 }}>&#10003;</div>
<div style={{ fontSize: 20, fontWeight: 600 }}>Agent Registered</div>
</div>
<div style={{ marginBottom: 12 }}>
<label style={{ fontSize: 12 }}>Client ID</label>
<div className="code-block">
<span>{credentials.clientId}</span>
<button className="copy-btn" onClick={() => copy(credentials.clientId)}>Copy</button>
</div>
</div>
<div style={{ marginBottom: 12 }}>
<label style={{ fontSize: 12 }}>Client Secret</label>
<div className="code-block">
<span>{credentials.clientSecret}</span>
<button className="copy-btn" onClick={() => copy(credentials.clientSecret)}>Copy</button>
</div>
</div>
<div className="warning-bar">
Save this secret now. It will not be shown again.
</div>
<div style={{ display: 'flex', gap: 12, justifyContent: 'flex-end', marginTop: 20 }}>
<button className="btn btn-secondary" onClick={downloadJson}>Download as JSON</button>
<button className="btn btn-primary" onClick={onClose}>Done</button>
</div>
</div>
</div>
);
}
function SourceAccessEditor({ clientId, agent, sources, onRescoped }: {
clientId: string;
agent: Agent;
sources: Source[];
onRescoped: (scope: { sourceId: string; federatedRead: string[] }) => void;
}) {
const [writeSource, setWriteSource] = useState(agent.source_id || 'default');
const [readSources, setReadSources] = useState<string[]>(agent.federated_read || []);
const [saving, setSaving] = useState(false);
const [error, setError] = useState('');
const [saved, setSaved] = useState(false);
const readableSet = new Set(readSources);
const activeSourceIds = new Set(sources.map(source => source.id));
const unavailableReadSources = readSources.filter(sourceId => !activeSourceIds.has(sourceId));
const primaryUnavailable = !activeSourceIds.has(writeSource);
const save = async () => {
if (readSources.length === 0) {
setError('Select at least one readable source.');
return;
}
setSaving(true);
setError('');
setSaved(false);
try {
const result = await api.rescopeClient(clientId, writeSource, readSources) as {
sourceId: string;
federatedRead: string[];
};
setWriteSource(result.sourceId);
setReadSources(result.federatedRead);
setSaved(true);
onRescoped(result);
} catch (e) {
setError(e instanceof Error ? e.message : 'Failed to save source access');
} finally {
setSaving(false);
}
};
return (
<>
<div className="section-title">Source Access</div>
<div style={{ color: 'var(--text-secondary)', fontSize: 12, lineHeight: 1.5, marginBottom: 12 }}>
The primary source is the write destination. Read access is an explicit allowlist and does not widen automatically.
</div>
<div style={{ marginBottom: 14 }}>
<label htmlFor="agent-write-source">Primary / write source</label>
<select
id="agent-write-source"
value={writeSource}
onChange={e => { setWriteSource(e.target.value); setSaved(false); }}
style={{ width: '100%', background: 'var(--bg-secondary)', color: 'var(--text-primary)', border: '1px solid var(--border)', borderRadius: 6, padding: '6px 10px', fontSize: 14 }}
>
{primaryUnavailable && (
<option value={writeSource} disabled>{writeSource} · unavailable</option>
)}
{sources.map(source => (
<option key={source.id} value={source.id}>{source.name} ({source.id})</option>
))}
</select>
</div>
<fieldset style={{ border: 0, padding: 0, margin: '0 0 14px' }}>
<legend>Readable sources</legend>
<div className="checkbox-group" style={{ marginTop: 6 }}>
{sources.map(source => (
<label key={source.id} className="checkbox-label">
<input
type="checkbox"
checked={readableSet.has(source.id)}
onChange={e => {
setSaved(false);
setReadSources(current => e.target.checked
? [...current, source.id]
: current.filter(id => id !== source.id));
}}
/>
{source.name} ({source.id}){source.federated ? ' · federated' : ' · private'}
</label>
))}
{unavailableReadSources.map(sourceId => (
<label key={sourceId} className="checkbox-label" style={{ color: 'var(--warning)' }}>
<input
type="checkbox"
checked
onChange={() => {
setSaved(false);
setReadSources(current => current.filter(id => id !== sourceId));
}}
/>
{sourceId} · unavailable (clear to remove grant)
</label>
))}
</div>
</fieldset>
{(primaryUnavailable || unavailableReadSources.length > 0) && (
<div style={{ color: 'var(--warning)', fontSize: 13, marginBottom: 10 }}>
This client references unavailable or archived sources. Choose an active primary source and clear unavailable read grants before saving.
</div>
)}
{error && <div style={{ color: 'var(--error)', fontSize: 13, marginBottom: 10 }}>{error}</div>}
{saved && <div style={{ color: 'var(--success)', fontSize: 13, marginBottom: 10 }}>Source access saved.</div>}
<button
type="button"
className="btn btn-primary"
disabled={saving || readSources.length === 0 || sources.length === 0 || primaryUnavailable || unavailableReadSources.length > 0}
onClick={save}
>
{saving ? 'Saving...' : 'Save Source Access'}
</button>
</>
);
}
function AgentDrawer({ agent, sources, onClose, onRevoked, onRescoped }: {
agent: Agent;
sources: Source[];
onClose: () => void;
onRevoked: () => void;
onRescoped: (scope: { sourceId: string; federatedRead: string[] }) => void;
}) {
const [tab, setTab] = useState<'claude-code' | 'chatgpt' | 'claude-cowork' | 'perplexity' | 'cursor' | 'json'>('claude-code');
const copy = (text: string) => navigator.clipboard.writeText(text);
const serverUrl = window.location.origin;
const cid = agent.id || agent.client_id || '';
const isOAuth = agent.auth_type === 'oauth';
const agentName = agent.name || agent.client_name || 'unknown';
// For API keys, we can't show the actual token (it was shown once at creation).
// For OAuth, we show the client_id and tell them to use their secret.
const configSnippets: Record<string, string> = {
'claude-code': isOAuth
? [
`The user wants to connect GBrain MCP to your context. Here's how.`,
``,
`# Step 1 — read the client secret from the operator without putting it in shell history.`,
`# read -s suppresses the echo; export keeps it scoped to this shell session.`,
`read -rs -p "Client Secret: " GBRAIN_CS && echo`,
``,
`# Step 2 — mint a token and register GBrain in one go.`,
`claude mcp add --transport http gbrain ${serverUrl}/mcp \\`,
` --header "Authorization: Bearer $(curl -s -X POST ${serverUrl}/token \\`,
` -d 'grant_type=client_credentials' \\`,
` -d 'client_id=${cid}' \\`,
` --data-urlencode "client_secret=$GBRAIN_CS" \\`,
` -d 'scope=${agent.scope || 'read write'}' | jq -r .access_token)"`,
``,
`# Step 3 — clear the secret from this shell.`,
`unset GBRAIN_CS`,
``,
`The token will last ${agent.token_ttl ? (agent.token_ttl >= 86400 ? Math.floor(agent.token_ttl / 86400) + ' days' : Math.floor(agent.token_ttl / 3600) + ' hours') : '1 hour (default)'}.`,
``,
`─── Fallback: 2-step curl + paste ───`,
``,
`If your shell doesn't support read -s, mint the token first, then paste:`,
``,
`curl -s -X POST ${serverUrl}/token \\`,
` -d 'grant_type=client_credentials' \\`,
` -d 'client_id=${cid}' \\`,
` -d 'client_secret=PASTE_YOUR_CLIENT_SECRET_HERE' \\`,
` -d 'scope=${agent.scope || 'read write'}' | jq -r .access_token`,
``,
`claude mcp add --transport http gbrain ${serverUrl}/mcp \\`,
` --header "Authorization: Bearer PASTE_TOKEN_FROM_ABOVE"`,
``,
`# Then run: history -d $((HISTCMD-1)) # zsh: print -s -- '' && fc -p`,
].join('\n')
: [
`The user wants to connect GBrain MCP to your context. Here's how.`,
``,
`Run this command in terminal to register GBrain:`,
``,
`claude mcp add --transport http gbrain ${serverUrl}/mcp \\`,
` --header "Authorization: Bearer PASTE_YOUR_API_KEY_HERE"`,
``,
`Replace PASTE_YOUR_API_KEY_HERE with the API key shown when "${agentName}" was created.`,
`API keys never expire.`,
].join('\n'),
'chatgpt': [
`The user wants to connect GBrain MCP to ChatGPT.`,
``,
`1. Open ChatGPT > Settings > Tools & Integrations > Add MCP Server`,
`2. Enter this URL — ChatGPT will auto-discover the OAuth configuration:`,
``,
` ${serverUrl}/.well-known/oauth-authorization-server`,
``,
`3. When prompted for credentials:`,
` Client ID: ${cid}`,
` Client Secret: (the secret from agent registration)`,
` Grant Type: client_credentials`,
` Scope: ${agent.scope || 'read write'}`,
].join('\n'),
'claude-cowork': [
`The user wants to connect GBrain MCP to Claude.ai.`,
``,
`1. Open claude.ai > Settings > Connected Apps > Add MCP Server`,
`2. Server URL: ${serverUrl}/mcp`,
`3. When prompted for auth:`,
` Token endpoint: ${serverUrl}/token`,
` Client ID: ${cid}`,
` Client Secret: (the secret from agent registration)`,
` Scope: ${agent.scope || 'read write'}`,
``,
`Discovery URL: ${serverUrl}/.well-known/oauth-authorization-server`,
].join('\n'),
cursor: isOAuth
? [
`The user wants to connect GBrain MCP to Cursor.`,
``,
`Cursor supports OAuth for remote MCP. Add to .cursor/mcp.json:`,
``,
`{`,
` "mcpServers": {`,
` "gbrain": {`,
` "url": "${serverUrl}/mcp",`,
` "transport": "sse"`,
` }`,
` }`,
`}`,
``,
`Cursor will auto-discover OAuth via:`,
`${serverUrl}/.well-known/oauth-authorization-server`,
``,
`When prompted: Client ID ${cid}, use the secret from registration.`,
].join('\n')
: [
`The user wants to connect GBrain MCP to Cursor.`,
``,
`Add to .cursor/mcp.json:`,
``,
`{`,
` "mcpServers": {`,
` "gbrain": {`,
` "url": "${serverUrl}/mcp",`,
` "transport": "sse",`,
` "headers": {`,
` "Authorization": "Bearer PASTE_YOUR_API_KEY_HERE"`,
` }`,
` }`,
` }`,
`}`,
``,
`Replace PASTE_YOUR_API_KEY_HERE with the API key shown when "${agentName}" was created.`,
].join('\n'),
perplexity: [
`The user wants to connect GBrain MCP to Perplexity.`,
``,
`1. Go to Settings > Connectors > Add MCP`,
`2. Server URL: ${serverUrl}/mcp`,
`3. Client ID: ${cid}`,
`4. Client Secret: (the secret from agent registration)`,
].join('\n'),
json: JSON.stringify({
server_url: serverUrl + '/mcp',
token_url: serverUrl + '/token',
discovery_url: serverUrl + '/.well-known/oauth-authorization-server',
client_id: cid,
client_name: agentName,
auth_type: agent.auth_type,
scope: agent.scope,
}, null, 2),
};
return (
<>
<div className="drawer-overlay" onClick={onClose} />
<div className="drawer">
<button className="drawer-close" onClick={onClose}>&#10005;</button>
<div style={{ fontSize: 18, fontWeight: 600, marginBottom: 4 }}>{agent.name || agent.client_name}</div>
<span className={`badge ${agent.status === 'active' ? 'badge-success' : 'badge-danger'}`}>{agent.status}</span>
<div className="section-title">Details</div>
<div style={{ display: 'grid', gridTemplateColumns: '100px 1fr', gap: '6px 12px', fontSize: 13 }}>
<span style={{ color: 'var(--text-secondary)' }}>Client ID</span>
<span className="mono">{(agent.id || agent.id || agent.client_id || '').substring(0, 24)}...</span>
<span style={{ color: 'var(--text-secondary)' }}>Scopes</span>
<span>{(agent.scope || '').split(' ').filter(Boolean).map(s => (
<span key={s} className={`badge badge-${s}`} style={{ marginRight: 4 }}>{s}</span>
))}</span>
<span style={{ color: 'var(--text-secondary)' }}>Registered</span>
<span>{new Date(agent.created_at).toLocaleDateString()}</span>
<span style={{ color: 'var(--text-secondary)' }}>Token TTL</span>
<span>{agent.token_ttl ? (agent.token_ttl >= 31536000 ? 'No expiry' : agent.token_ttl >= 86400 ? `${Math.floor(agent.token_ttl / 86400)}d` : agent.token_ttl >= 3600 ? `${Math.floor(agent.token_ttl / 3600)}h` : `${agent.token_ttl}s`) : '1h (default)'}</span>
</div>
{isOAuth && (
<SourceAccessEditor
clientId={cid}
agent={agent}
sources={sources}
onRescoped={onRescoped}
/>
)}
{/*
Config Export visible for both auth_type=oauth AND auth_type=api_key.
Claude Code + Cursor + JSON tabs render real snippets regardless
(commit 15's snippets are auth-type-aware for those two clients;
JSON is just structured metadata). ChatGPT, Claude.ai, and
Perplexity tabs render an "OAuth client required" message on
api_key agents those MCP clients only speak OAuth 2.0
client_credentials, not raw bearer tokens.
Pre-fix (Wintermute commit 16): the entire Config Export
section was hidden for api_key agents, dropping the working
Claude Code + Cursor snippets along with the broken ones.
(D5=C in the eng review.)
*/}
<div className="section-title">Config Export</div>
<div className="tabs" style={{ flexWrap: 'wrap' }}>
<div className={`tab ${tab === 'claude-code' ? 'active' : ''}`} onClick={() => setTab('claude-code')}>Claude Code</div>
<div className={`tab ${tab === 'chatgpt' ? 'active' : ''}`} onClick={() => setTab('chatgpt')}>ChatGPT</div>
<div className={`tab ${tab === 'claude-cowork' ? 'active' : ''}`} onClick={() => setTab('claude-cowork')}>Claude.ai</div>
<div className={`tab ${tab === 'cursor' ? 'active' : ''}`} onClick={() => setTab('cursor')}>Cursor</div>
<div className={`tab ${tab === 'perplexity' ? 'active' : ''}`} onClick={() => setTab('perplexity')}>Perplexity</div>
<div className={`tab ${tab === 'json' ? 'active' : ''}`} onClick={() => setTab('json')}>JSON</div>
</div>
{(() => {
const oauthOnlyTabs = new Set(['chatgpt', 'claude-cowork', 'perplexity']);
if (!isOAuth && oauthOnlyTabs.has(tab)) {
const clientName = tab === 'chatgpt'
? 'ChatGPT'
: tab === 'claude-cowork'
? 'Claude.ai'
: 'Perplexity';
return (
<div style={{
background: 'rgba(255, 200, 100, 0.08)',
border: '1px solid rgba(255, 200, 100, 0.2)',
borderRadius: 8,
padding: '14px 16px',
marginTop: 12,
fontSize: 13,
lineHeight: 1.6,
color: 'var(--text-secondary)',
}}>
<div style={{ fontWeight: 600, color: 'var(--text-primary)', marginBottom: 6 }}>
{clientName} requires an OAuth client
</div>
{clientName} only supports OAuth 2.0 (client_credentials). API keys use raw bearer tokens, which {clientName} does not accept. Register a separate OAuth client and use that to connect this AI.
</div>
);
}
return (
<div className="code-block">
<pre style={{ whiteSpace: 'pre-wrap', margin: 0 }}>{configSnippets[tab]}</pre>
<button className="copy-btn" onClick={() => copy(configSnippets[tab])}>Copy</button>
</div>
);
})()}
<div style={{ marginTop: 32 }}>
{agent.status === 'active' && (
<button className="btn btn-danger" onClick={async () => {
if (!confirm(`Revoke ${agent.name || agent.client_name}? All active tokens will be invalidated.`)) return;
try {
if (agent.auth_type === 'oauth') {
await api.revokeClient(agent.id || agent.client_id || '');
} else {
await api.revokeApiKey(agent.name || '');
}
onRevoked();
onClose();
} catch (e) {
alert('Revoke failed: ' + (e instanceof Error ? e.message : 'unknown error'));
}
}}>Revoke Agent</button>
)}
{agent.status === 'revoked' && (
<span style={{ color: 'var(--text-muted)', fontSize: 13 }}>This agent has been revoked.</span>
)}
</div>
</div>
</>
);
}
-174
View File
@@ -1,174 +0,0 @@
/**
* v0.36.1.0 (T15 / E6) Calibration tab.
*
* Fetches the active calibration profile + 4 server-rendered SVG charts.
* Layout: Linear calm clarity (per D23 mockup variant-B) single column,
* generous whitespace, ONE big sparkline as hero, then patterns, then
* domain bars, then abandoned threads.
*
* Per D23 SVG markup comes from the server (image/svg+xml endpoint).
* Admin SPA renders inside a TrustedSVG wrapper that uses
* dangerouslySetInnerHTML. XSS posture: server-side escapeXml() on all
* caller-controlled strings + requireAdmin middleware on the endpoint.
*/
import React, { useEffect, useState } from 'react';
import { api } from '../api';
interface CalibrationProfileSummary {
holder: string;
source_id: string;
generated_at: string;
published: boolean;
total_resolved: number;
brier: number | null;
accuracy: number | null;
partial_rate: number | null;
grade_completion: number;
pattern_statements: string[];
active_bias_tags: string[];
voice_gate_passed: boolean;
voice_gate_attempts: number;
}
interface ChartSvgProps {
type: string;
ariaLabel: string;
}
function TrustedSVG({ markup }: { markup: string }) {
return (
<div
style={{ width: '100%', overflow: 'auto' }}
// Server-rendered SVG (image/svg+xml) gated by requireAdmin middleware.
// All caller-controlled strings pass through escapeXml() server-side.
dangerouslySetInnerHTML={{ __html: markup }}
/>
);
}
function ChartSvg({ type, ariaLabel }: ChartSvgProps) {
const [markup, setMarkup] = useState<string>('');
const [error, setError] = useState<string>('');
useEffect(() => {
let cancelled = false;
api
.calibrationChart(type)
.then(svg => {
if (!cancelled) setMarkup(svg);
})
.catch(err => {
if (!cancelled) setError(err.message ?? 'fetch failed');
});
return () => {
cancelled = true;
};
}, [type]);
if (error) {
return (
<div style={{ padding: 16, color: 'var(--error)' }} role="alert">
{ariaLabel}: {error}
</div>
);
}
if (!markup) {
return <div style={{ padding: 16, color: 'var(--text-muted)' }}>{ariaLabel} loading...</div>;
}
return <TrustedSVG markup={markup} />;
}
export function CalibrationPage() {
const [profile, setProfile] = useState<CalibrationProfileSummary | null>(null);
const [loading, setLoading] = useState(true);
const [error, setError] = useState<string>('');
useEffect(() => {
api
.calibrationProfile()
.then(p => {
setProfile(p);
setLoading(false);
})
.catch(err => {
setError(err.message ?? 'fetch failed');
setLoading(false);
});
}, []);
if (loading) {
return <div style={{ padding: 24, color: 'var(--text-secondary)' }}>Loading calibration profile</div>;
}
if (error) {
return (
<div style={{ padding: 24, color: 'var(--error)' }} role="alert">
Could not load calibration profile: {error}
</div>
);
}
if (!profile) {
return (
<div style={{ padding: 24, maxWidth: 700 }}>
<h1 style={{ marginBottom: 16 }}>Calibration</h1>
<p style={{ color: 'var(--text-secondary)' }}>
No calibration profile yet. Builds after 5+ resolved takes.
</p>
<pre
style={{
background: 'var(--bg-secondary)',
padding: 12,
borderRadius: 4,
color: 'var(--text-primary)',
marginTop: 12,
fontFamily: 'var(--font-mono)',
}}
>
gbrain dream --phase calibration_profile
</pre>
</div>
);
}
const generated = new Date(profile.generated_at);
const generatedAgo = Math.floor((Date.now() - generated.getTime()) / (1000 * 60 * 60 * 24));
return (
<div style={{ padding: 32, maxWidth: 720 }}>
<h1 style={{ marginBottom: 8 }}>Calibration</h1>
<div style={{ color: 'var(--text-muted)', fontSize: 13, marginBottom: 24 }}>
Holder: {profile.holder}
{' · '}
Updated {generatedAgo === 0 ? 'today' : `${generatedAgo}d ago`}
{profile.published && ' · published'}
{profile.grade_completion < 0.9 && ` · ~${Math.round(profile.grade_completion * 100)}% graded`}
{!profile.voice_gate_passed && ' · voice gate fell back to template'}
</div>
<section style={{ marginBottom: 32 }}>
<ChartSvg type="brier-trend" ariaLabel="Brier trend" />
</section>
<section style={{ marginBottom: 32 }}>
<h2 style={{ fontSize: 14, color: 'var(--text-secondary)', marginBottom: 12, fontWeight: 400 }}>
Pattern statements
</h2>
<ChartSvg type="pattern-statements" ariaLabel="Pattern statements" />
</section>
<section style={{ marginBottom: 32 }}>
<ChartSvg type="domain-bars" ariaLabel="Per-domain accuracy" />
</section>
<section style={{ marginBottom: 32 }}>
<ChartSvg type="abandoned-threads" ariaLabel="Abandoned threads" />
</section>
{profile.active_bias_tags.length > 0 && (
<section style={{ marginBottom: 32, color: 'var(--text-muted)', fontSize: 13 }}>
Active bias tags: {profile.active_bias_tags.join(', ')}
</section>
)}
</div>
);
}
-137
View File
@@ -1,137 +0,0 @@
import React, { useState, useEffect, useRef } from 'react';
import { api } from '../api';
interface FeedEvent {
agent: string;
operation: string;
scopes: string;
latency_ms: number;
status: string;
timestamp: string;
}
export function DashboardPage() {
const [stats, setStats] = useState({ connected_agents: 0, requests_today: 0, active_tokens: 0 });
const [health, setHealth] = useState({ expiring_soon: 0, error_rate: '0%' });
const [events, setEvents] = useState<FeedEvent[]>([]);
const [sseStatus, setSseStatus] = useState<'connecting' | 'connected' | 'disconnected'>('connecting');
const eventSourceRef = useRef<EventSource | null>(null);
useEffect(() => {
api.stats().then(setStats).catch(() => {});
api.health().then(setHealth).catch(() => {});
const es = new EventSource('/admin/events', { withCredentials: true });
eventSourceRef.current = es;
es.onopen = () => setSseStatus('connected');
es.onmessage = (e) => {
try {
const event = JSON.parse(e.data) as FeedEvent;
setEvents(prev => [event, ...prev].slice(0, 50));
} catch {}
};
es.onerror = () => {
setSseStatus('disconnected');
setTimeout(() => {
setSseStatus('connecting');
es.close();
// Reconnect handled by browser EventSource auto-retry
}, 3000);
};
const interval = setInterval(() => {
api.stats().then(setStats).catch(() => {});
api.health().then(setHealth).catch(() => {});
}, 30000);
return () => { es.close(); clearInterval(interval); };
}, []);
const timeAgo = (ts: string) => {
const diff = Date.now() - new Date(ts).getTime();
if (diff < 60000) return `${Math.floor(diff / 1000)}s ago`;
if (diff < 3600000) return `${Math.floor(diff / 60000)} min ago`;
return `${Math.floor(diff / 3600000)}h ago`;
};
return (
<>
<h1 className="page-title">Dashboard</h1>
<div style={{ display: 'flex', gap: 24 }}>
<div style={{ flex: 1 }}>
<div className="metrics">
<div className="metric">
<div className="metric-value">{stats.connected_agents}</div>
<div className="metric-label">Connected Agents</div>
</div>
<div className="metric">
<div className="metric-value">{stats.requests_today}</div>
<div className="metric-label">Requests Today</div>
</div>
<div className="metric">
<div className="metric-value">{stats.active_tokens}</div>
<div className="metric-label">Active Tokens</div>
</div>
</div>
<h2 className="section-title">
Live Activity
<span style={{ marginLeft: 8, fontSize: 10, color: sseStatus === 'connected' ? 'var(--success)' : sseStatus === 'connecting' ? 'var(--warning)' : 'var(--error)' }}>
{sseStatus === 'connected' ? '● connected' : sseStatus === 'connecting' ? '● connecting...' : '● disconnected'}
</span>
</h2>
<div className="feed">
{events.length === 0 ? (
<div className="feed-empty">
{sseStatus === 'connected' ? 'No requests yet. Agents will appear when they connect.' : 'Connecting...'}
</div>
) : (
<table>
<thead>
<tr>
<th>Agent</th>
<th>Operation</th>
<th>Scopes</th>
<th>Latency</th>
<th>Status</th>
<th>Time</th>
</tr>
</thead>
<tbody>
{events.map((e, i) => (
<tr key={i}>
<td className="mono">{e.agent}</td>
<td className="mono">{e.operation}</td>
<td>{e.scopes.split(',').map(s => (
<span key={s} className={`badge badge-${s.trim()}`} style={{ marginRight: 4 }}>{s.trim()}</span>
))}</td>
<td className="mono">{e.latency_ms} ms</td>
<td><span className={`badge badge-${e.status}`}>{e.status}</span></td>
<td style={{ color: 'var(--text-secondary)' }}>{timeAgo(e.timestamp)}</td>
</tr>
))}
</tbody>
</table>
)}
</div>
</div>
<div style={{ width: 220 }}>
<h2 className="section-title">Token Health</h2>
<div className="health-panel">
<div className="health-row">
<span style={{ color: 'var(--warning)' }}>Expiring Soon</span>
<span className="mono">{health.expiring_soon}</span>
</div>
<div className="health-row">
<span style={{ color: 'var(--error)' }}>Error Rate</span>
<span className="mono">{health.error_rate}</span>
</div>
</div>
</div>
</div>
</>
);
}
-174
View File
@@ -1,174 +0,0 @@
import React, { useEffect, useState } from 'react';
import { api } from '../api';
/**
* v0.41 D2 live jobs dashboard. Browser counterpart to the TTY
* `gbrain jobs watch` command. Polls `/admin/api/jobs/watch` every
* 1s (matches TTY refresh cadence; SSE upgrade is a v0.42 follow-up
* once the same wiring lands in serve-http for the TTY command).
*
* Layout intentionally matches the TTY 1:1 so an operator looking at
* both surfaces sees the same panels in the same order.
*/
interface WatchSnapshot {
ts_ms: number;
by_type: Array<{ name: string; total: number; completed: number; failed: number; dead: number }>;
queue_health: { waiting: number; active: number; stalled: number };
lease_pressure_1h: number;
top_errors: Array<{ cluster: string; count: number }>;
budget_owners: Array<{ owner_id: number; remaining_cents: number; total_spent_cents: number }>;
}
function leasePressureColor(n: number): string {
if (n === 0) return 'var(--accent-success, #2ea043)';
if (n >= 100) return 'var(--accent-danger, #f85149)';
return 'var(--accent-warn, #d29922)';
}
function dollars(cents: number): string {
return `$${(cents / 100).toFixed(2)}`;
}
export function JobsWatchPage() {
const [snap, setSnap] = useState<WatchSnapshot | null>(null);
const [err, setErr] = useState<string | null>(null);
useEffect(() => {
let alive = true;
let timer: ReturnType<typeof setTimeout> | null = null;
const tick = async () => {
try {
const data = await api.jobsWatch();
if (alive) {
setSnap(data);
setErr(null);
}
} catch (e) {
if (alive) setErr(e instanceof Error ? e.message : String(e));
}
if (alive) timer = setTimeout(tick, 1000);
};
tick();
return () => {
alive = false;
if (timer) clearTimeout(timer);
};
}, []);
if (err) {
return (
<div style={{ padding: 24, color: 'var(--accent-danger, #f85149)' }}>
<h2>Jobs Watch error</h2>
<pre style={{ whiteSpace: 'pre-wrap' }}>{err}</pre>
</div>
);
}
if (!snap) {
return <div style={{ padding: 24, color: 'var(--text-muted, #777)' }}>Loading jobs watch</div>;
}
const ts = new Date(snap.ts_ms).toLocaleTimeString();
return (
<div style={{ padding: 24, fontFamily: 'var(--font-mono, "JetBrains Mono", monospace)' }}>
<h1 style={{ fontSize: 18, marginBottom: 4 }}>
Jobs Watch
<span style={{ marginLeft: 12, color: 'var(--text-muted, #777)', fontSize: 12, fontWeight: 'normal' }}>
updated {ts}
</span>
</h1>
<section style={{ marginTop: 24 }}>
<h2 style={{ fontSize: 14, marginBottom: 8 }}>Queue</h2>
<div>
waiting=<b>{snap.queue_health.waiting}</b>{' '}
active=<b>{snap.queue_health.active}</b>{' '}
stalled=<b style={{ color: snap.queue_health.stalled > 0 ? 'var(--accent-warn, #d29922)' : undefined }}>
{snap.queue_health.stalled}
</b>
</div>
</section>
{snap.by_type.length > 0 && (
<section style={{ marginTop: 24 }}>
<h2 style={{ fontSize: 14, marginBottom: 8 }}>By type (24h)</h2>
<table style={{ borderCollapse: 'collapse' }}>
<thead>
<tr style={{ color: 'var(--text-muted, #777)', fontSize: 12 }}>
<th style={{ textAlign: 'left', padding: '4px 12px 4px 0' }}>name</th>
<th style={{ textAlign: 'right', padding: '4px 12px' }}>total</th>
<th style={{ textAlign: 'right', padding: '4px 12px' }}>done</th>
<th style={{ textAlign: 'right', padding: '4px 12px' }}>fail</th>
<th style={{ textAlign: 'right', padding: '4px 12px' }}>dead</th>
</tr>
</thead>
<tbody>
{snap.by_type.slice(0, 6).map(t => (
<tr key={t.name}>
<td style={{ padding: '4px 12px 4px 0' }}>{t.name}</td>
<td style={{ textAlign: 'right', padding: '4px 12px' }}>{t.total}</td>
<td style={{ textAlign: 'right', padding: '4px 12px' }}>{t.completed}</td>
<td style={{ textAlign: 'right', padding: '4px 12px' }}>{t.failed}</td>
<td style={{ textAlign: 'right', padding: '4px 12px' }}>{t.dead}</td>
</tr>
))}
</tbody>
</table>
</section>
)}
<section style={{ marginTop: 24 }}>
<h2 style={{ fontSize: 14, marginBottom: 8 }}>Lease pressure (1h)</h2>
<div style={{ color: leasePressureColor(snap.lease_pressure_1h) }}>
{snap.lease_pressure_1h} bounce{snap.lease_pressure_1h === 1 ? '' : 's'}
</div>
</section>
{snap.top_errors.length > 0 && (
<section style={{ marginTop: 24 }}>
<h2 style={{ fontSize: 14, marginBottom: 8 }}>Top errors (24h)</h2>
<table style={{ borderCollapse: 'collapse' }}>
<tbody>
{snap.top_errors.slice(0, 5).map(e => (
<tr key={e.cluster}>
<td style={{ textAlign: 'right', padding: '4px 12px 4px 0', color: 'var(--text-muted, #777)' }}>
{e.count}×
</td>
<td style={{ padding: '4px 12px 4px 0' }}>{e.cluster}</td>
</tr>
))}
</tbody>
</table>
</section>
)}
{snap.budget_owners.length > 0 && (
<section style={{ marginTop: 24 }}>
<h2 style={{ fontSize: 14, marginBottom: 8 }}>Budget owners</h2>
<table style={{ borderCollapse: 'collapse' }}>
<thead>
<tr style={{ color: 'var(--text-muted, #777)', fontSize: 12 }}>
<th style={{ textAlign: 'left', padding: '4px 12px 4px 0' }}>owner</th>
<th style={{ textAlign: 'right', padding: '4px 12px' }}>spent</th>
<th style={{ textAlign: 'right', padding: '4px 12px' }}>remaining</th>
</tr>
</thead>
<tbody>
{snap.budget_owners.slice(0, 5).map(b => (
<tr key={b.owner_id}>
<td style={{ padding: '4px 12px 4px 0' }}>{b.owner_id}</td>
<td style={{ textAlign: 'right', padding: '4px 12px' }}>{dollars(b.total_spent_cents)}</td>
<td style={{ textAlign: 'right', padding: '4px 12px' }}>{dollars(b.remaining_cents)}</td>
</tr>
))}
</tbody>
</table>
</section>
)}
</div>
);
}
-96
View File
@@ -1,96 +0,0 @@
import React, { useState } from 'react';
import { api } from '../api';
// v0.26.3 trust model (D11 + D12):
// - The bootstrap token is NEVER stored in browser JS state. No
// localStorage, no sessionStorage, no React state beyond the form
// submit cycle. After successful POST /admin/login the operator's
// token only lives in the HttpOnly cookie that the server set.
// - Magic-link URLs use single-use server-issued nonces, not the
// bootstrap token itself (see /admin/api/issue-magic-link). The
// bootstrap token never appears in a URL.
// - Closing the tab ends the session client-side. Reopening the
// dashboard 401s and shows this page again. Operator asks the agent
// for a fresh magic link or pastes the bootstrap token from the
// server's terminal scrollback.
export function LoginPage({ onLogin }: { onLogin: () => void }) {
const [token, setToken] = useState('');
const [error, setError] = useState('');
const [loading, setLoading] = useState(false);
const handleSubmit = async (e: React.FormEvent) => {
e.preventDefault();
setError('');
setLoading(true);
try {
await api.login(token);
// Don't persist the token. The HttpOnly cookie is the only
// session credential after this point.
setToken('');
onLogin();
} catch (err) {
setError('Invalid token.');
} finally {
setLoading(false);
}
};
return (
<div className="login-page">
<div className="login-box">
<div className="login-logo">GBrain</div>
<div style={{
background: 'rgba(136, 170, 255, 0.08)',
border: '1px solid rgba(136, 170, 255, 0.2)',
borderRadius: 8,
padding: '14px 16px',
marginBottom: 20,
fontSize: 13,
lineHeight: 1.5,
color: 'var(--text-secondary)',
}}>
<div style={{ fontWeight: 600, color: 'var(--text-primary)', marginBottom: 6 }}>
🔒 This is a protected dashboard
</div>
Ask your AI agent for the admin login link:
<div style={{
background: 'rgba(0,0,0,0.3)',
borderRadius: 6,
padding: '8px 12px',
marginTop: 8,
fontFamily: 'var(--font-mono)',
fontSize: 12,
color: '#88aaff',
wordBreak: 'break-all',
}}>
"Give me the GBrain admin login link"
</div>
<div style={{ marginTop: 8, fontSize: 12, color: 'var(--text-muted)' }}>
Each link is single-use. Your agent generates a fresh one each time.
</div>
</div>
<details style={{ marginBottom: 16 }}>
<summary style={{ cursor: 'pointer', fontSize: 13, color: 'var(--text-muted)' }}>
Or paste bootstrap token manually
</summary>
<form onSubmit={handleSubmit} style={{ marginTop: 12 }}>
<div style={{ marginBottom: 12 }}>
<input
type="password"
placeholder="Admin Token"
value={token}
onChange={e => setToken(e.target.value)}
/>
</div>
<button className="btn btn-primary" style={{ width: '100%' }} disabled={loading}>
{loading ? 'Authenticating...' : 'Submit'}
</button>
{error && <div className="login-error">{error}</div>}
</form>
</details>
</div>
</div>
);
}
-150
View File
@@ -1,150 +0,0 @@
import React, { useState, useEffect } from 'react';
import { api } from '../api';
interface LogEntry {
id: number;
token_name: string;
agent_name: string;
operation: string;
latency_ms: number;
status: string;
params: Record<string, unknown> | null;
error_message: string | null;
created_at: string;
}
export function RequestLogPage() {
const [data, setData] = useState<{ rows: LogEntry[]; total: number; page: number; pages: number }>({
rows: [], total: 0, page: 1, pages: 1,
});
const [page, setPage] = useState(1);
const [agentFilter, setAgentFilter] = useState('all');
const [expandedRow, setExpandedRow] = useState<number | null>(null);
useEffect(() => { loadPage(page); }, [page, agentFilter]);
const loadPage = (p: number) => {
const qs = agentFilter !== 'all' ? `&agent=${encodeURIComponent(agentFilter)}` : '';
api.requests(p, qs).then(setData).catch(() => {});
};
const timeAgo = (ts: string) => {
const diff = Date.now() - new Date(ts).getTime();
if (diff < 60000) return `${Math.floor(diff / 1000)}s ago`;
if (diff < 3600000) return `${Math.floor(diff / 60000)} min ago`;
if (diff < 86400000) return `${Math.floor(diff / 3600000)}h ago`;
return new Date(ts).toLocaleDateString();
};
const formatParams = (params: Record<string, unknown> | null) => {
if (!params) return null;
const { query, slug, partial, limit, ...rest } = params as any;
const parts: string[] = [];
if (query) parts.push(`"${query}"`);
if (slug) parts.push(slug);
if (partial) parts.push(`~${partial}`);
if (limit) parts.push(`limit=${limit}`);
if (Object.keys(rest).length > 0) parts.push(`+${Object.keys(rest).length} params`);
return parts.join(' ');
};
// Collect unique agents for filter (use name for display, token_name for value)
const agentMap = new Map<string, string>();
data.rows.forEach(r => { if (r.token_name) agentMap.set(r.token_name, r.agent_name || r.token_name); });
return (
<>
<div style={{ display: 'flex', justifyContent: 'space-between', alignItems: 'center', marginBottom: 24 }}>
<h1 className="page-title" style={{ marginBottom: 0 }}>Request Log</h1>
<select value={agentFilter} onChange={e => { setAgentFilter(e.target.value); setPage(1); }}
style={{ background: 'var(--bg-secondary)', color: 'var(--text-primary)', border: '1px solid var(--border)', borderRadius: 6, padding: '4px 8px', fontSize: 13 }}>
<option value="all">All agents</option>
{[...agentMap.entries()].map(([id, name]) => <option key={id} value={id}>{name}</option>)}
</select>
</div>
{data.rows.length === 0 ? (
<div style={{ textAlign: 'center', padding: 48, color: 'var(--text-muted)' }}>
No requests yet.
</div>
) : (
<>
<table>
<thead>
<tr>
<th>Time</th>
<th>Agent</th>
<th>Operation</th>
<th>Params</th>
<th>Latency</th>
<th>Status</th>
</tr>
</thead>
<tbody>
{data.rows.map(r => (
<React.Fragment key={r.id}>
<tr onClick={() => setExpandedRow(expandedRow === r.id ? null : r.id)}
style={{ cursor: 'pointer' }}>
<td style={{ color: 'var(--text-secondary)', whiteSpace: 'nowrap' }}>{timeAgo(r.created_at)}</td>
<td>
<a style={{ color: 'var(--text-link, #88aaff)', cursor: 'pointer', textDecoration: 'none', fontWeight: 500 }}
onClick={(e) => { e.stopPropagation(); setAgentFilter(r.token_name); setPage(1); }}>
{r.agent_name || r.token_name}
</a>
</td>
<td className="mono">{r.operation}</td>
<td style={{ color: 'var(--text-secondary)', fontSize: 12, maxWidth: 200, overflow: 'hidden', textOverflow: 'ellipsis', whiteSpace: 'nowrap' }}>
{formatParams(r.params)}
</td>
<td className="mono">{r.latency_ms}ms</td>
<td><span className={`badge badge-${r.status}`}>{r.status}</span></td>
</tr>
{expandedRow === r.id && (
<tr>
<td colSpan={6} style={{ background: 'var(--bg-secondary, #0f0f1a)', padding: 16 }}>
<div style={{ display: 'grid', gridTemplateColumns: '100px 1fr', gap: '6px 12px', fontSize: 13 }}>
<span style={{ color: 'var(--text-muted)' }}>Time</span>
<span>{new Date(r.created_at).toLocaleString()}</span>
<span style={{ color: 'var(--text-muted)' }}>Agent</span>
<span className="mono">{r.token_name}</span>
<span style={{ color: 'var(--text-muted)' }}>Operation</span>
<span className="mono">{r.operation}</span>
<span style={{ color: 'var(--text-muted)' }}>Latency</span>
<span>{r.latency_ms}ms</span>
{r.params && (
<>
<span style={{ color: 'var(--text-muted)' }}>Params</span>
<pre className="mono" style={{ margin: 0, whiteSpace: 'pre-wrap', fontSize: 12 }}>
{JSON.stringify(r.params, null, 2)}
</pre>
</>
)}
{r.error_message && (
<>
<span style={{ color: 'var(--error, #ff6b6b)' }}>Error</span>
<span style={{ color: 'var(--error, #ff6b6b)' }}>{r.error_message}</span>
</>
)}
</div>
</td>
</tr>
)}
</React.Fragment>
))}
</tbody>
</table>
<div className="pagination">
<span>Page {data.page} of {data.pages} ({data.total} total)</span>
<div style={{ display: 'flex', gap: 8 }}>
<button disabled={data.page <= 1} onClick={() => setPage(p => p - 1)}>Previous</button>
<button disabled={data.page >= data.pages} onClick={() => setPage(p => p + 1)}>Next</button>
</div>
</div>
</>
)}
</>
);
}
-1
View File
@@ -1 +0,0 @@
/// <reference types="vite/client" />
-17
View File
@@ -1,17 +0,0 @@
{
"compilerOptions": {
"target": "ES2020",
"useDefineForClassFields": true,
"lib": ["ES2020", "DOM", "DOM.Iterable"],
"module": "ESNext",
"skipLibCheck": true,
"moduleResolution": "bundler",
"allowImportingTsExtensions": true,
"isolatedModules": true,
"moduleDetection": "force",
"noEmit": true,
"jsx": "react-jsx",
"strict": true
},
"include": ["src"]
}
-11
View File
@@ -1,11 +0,0 @@
import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react';
export default defineConfig({
plugins: [react()],
base: '/admin/',
build: {
outDir: 'dist',
emptyOutDir: true,
},
});
+30 -161
View File
@@ -5,44 +5,21 @@
"": {
"name": "gbrain",
"dependencies": {
"@ai-sdk/anthropic": "^3.0.71",
"@ai-sdk/google": "^3.0.64",
"@ai-sdk/openai": "^3.0.53",
"@ai-sdk/openai-compatible": "^2.0.41",
"@anthropic-ai/sdk": "^0.30.0",
"@aws-sdk/client-s3": "^3.1028.0",
"@dqbd/tiktoken": "^1.0.22",
"@electric-sql/pglite": "0.4.3",
"@jsquash/avif": "^2.1.1",
"@jsquash/png": "^3.1.1",
"@modelcontextprotocol/sdk": "1.29.0",
"ai": "^6.0.168",
"chokidar": "^4.0.3",
"cookie-parser": "^1.4.7",
"cors": "^2.8.5",
"eventsource-parser": "^3.0.8",
"exifr": "^7.1.3",
"express": "^5.1.0",
"express-rate-limit": "^7.5.0",
"@modelcontextprotocol/sdk": "^1.0.0",
"gray-matter": "^4.0.3",
"heic-decode": "^2.1.0",
"js-yaml": "^3.15.0",
"marked": "^18.0.2",
"marked": "^18.0.0",
"openai": "^4.0.0",
"pgvector": "^0.2.0",
"postgres": "^3.4.0",
"tree-sitter-wasms": "0.1.13",
"web-tree-sitter": "0.22.6",
"zod": "^4.3.6",
},
"devDependencies": {
"@types/bun": "latest",
"@types/cookie-parser": "^1.4.7",
"@types/cors": "^2.8.19",
"@types/express": "^5.0.6",
"@types/js-yaml": "^3.12.10",
"bun-types": "^1.3.13",
"fast-check": "^4.8.0",
"typescript": "^5.6.0",
},
},
@@ -50,33 +27,7 @@
"trustedDependencies": [
"@electric-sql/pglite",
],
"overrides": {
"@hono/node-server": "^2.0.5",
"body-parser": "^2.3.0",
"fast-uri": "^3.1.4",
"fast-xml-builder": "^1.1.7",
"fast-xml-parser": "^5.7.0",
"form-data": "^4.0.6",
"hono": "^4.12.25",
"ip-address": "^10.1.1",
"js-yaml": "^3.15.0",
"qs": "^6.15.2",
},
"packages": {
"@ai-sdk/anthropic": ["@ai-sdk/anthropic@3.0.74", "", { "dependencies": { "@ai-sdk/provider": "3.0.10", "@ai-sdk/provider-utils": "4.0.26" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-Xew9rfz9WWhDSyF8rNhjT/XWOWelNfJrMlmG0Ahw210hStisRpQZ1s+7VeI9JTJOZ5y5tXqBi5kfPwYnCfyRTA=="],
"@ai-sdk/gateway": ["@ai-sdk/gateway@3.0.109", "", { "dependencies": { "@ai-sdk/provider": "3.0.10", "@ai-sdk/provider-utils": "4.0.26", "@vercel/oidc": "3.2.0" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-r6dOqThjODp1vOhGRJg2OCmyB/ZOQtGx1esZ2SDvwDX5XoX8dBqYaYjLg8MPXTzMGJSgOkJyCxWgUcZtAl16pw=="],
"@ai-sdk/google": ["@ai-sdk/google@3.0.67", "", { "dependencies": { "@ai-sdk/provider": "3.0.10", "@ai-sdk/provider-utils": "4.0.26" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-Qeq+SidYtzMrcf0fdw3L0QLmtXK+ErwdBzbxS4+0Q/2UP85Ges8RJJcbAj7SO8e2JbeJoM35BLqkeNy1o3wJvQ=="],
"@ai-sdk/openai": ["@ai-sdk/openai@3.0.58", "", { "dependencies": { "@ai-sdk/provider": "3.0.10", "@ai-sdk/provider-utils": "4.0.26" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-2+5xGMROmrBboJuoOwqLL3b/o3i56+NRdxXDNVAiTyYjLiBj6KzembeuyuBT217be1X+zkEfAqD1H0irJlGIyw=="],
"@ai-sdk/openai-compatible": ["@ai-sdk/openai-compatible@2.0.45", "", { "dependencies": { "@ai-sdk/provider": "3.0.10", "@ai-sdk/provider-utils": "4.0.26" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-5YBvurNL7Oj7mT3srws4Rh4cQidoorfEGObAOb5jV40eld8IC7EkXWARZjnWYqgYzabUs6Sn6muiXfQVkgOyOQ=="],
"@ai-sdk/provider": ["@ai-sdk/provider@3.0.10", "", { "dependencies": { "json-schema": "^0.4.0" } }, "sha512-Q3BZ27qfpYqnCYGvE3vt+Qi6LGOF9R5Nmzn+9JoM1lCRsD9mYaIhfJLkSunN48nfGXJ6n+XNV0J/XVpqGQl7Dw=="],
"@ai-sdk/provider-utils": ["@ai-sdk/provider-utils@4.0.26", "", { "dependencies": { "@ai-sdk/provider": "3.0.10", "@standard-schema/spec": "^1.1.0", "eventsource-parser": "^3.0.8" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-CsKNLKsOpvPujRlIYvoz+Ybw+kGn7J4/fIZa/58+R7iWLLfwn6ifE2G6Yq8K9XvH/I/3bzaDAJ3NhRwEMsLBKQ=="],
"@anthropic-ai/sdk": ["@anthropic-ai/sdk@0.30.1", "", { "dependencies": { "@types/node": "^18.11.18", "@types/node-fetch": "^2.6.4", "abort-controller": "^3.0.0", "agentkeepalive": "^4.2.1", "form-data-encoder": "1.7.2", "formdata-node": "^4.3.2", "node-fetch": "^2.6.7" } }, "sha512-nuKvp7wOIz6BFei8WrTdhmSsx5mwnArYyJgh4+vYu3V4J0Ltb8Xm3odPm51n1aSI0XxNCrDl7O88cxCtUdAkaw=="],
"@aws-crypto/crc32": ["@aws-crypto/crc32@5.2.0", "", { "dependencies": { "@aws-crypto/util": "^5.2.0", "@aws-sdk/types": "^3.222.0", "tslib": "^2.6.2" } }, "sha512-nLbCWqQNgUiwwtFsen1AdzAtvuLRsQS8rYgMuxCrdKf9kOssamGLuPwyTY9wyYblNr9+1XM8v6zoDTPPSIeANg=="],
@@ -163,18 +114,10 @@
"@electric-sql/pglite": ["@electric-sql/pglite@0.4.3", "", {}, "sha512-ichuWTgtd4mOM1G4SpyGJa5trT03lWbMypDV0fUXUCXg5hiHqVAz/bZyV68NqmkLB7WcYmj1RMJVSp8HV/v/ZQ=="],
"@hono/node-server": ["@hono/node-server@2.0.11", "", { "peerDependencies": { "hono": "^4" } }, "sha512-bjD221KPLoJTWUwso1J6fGKiTXEUFedG/s0visavY4zakFPkeGURMRNly+FhBHs7T8Dz4qHaZIMX9ZoJHSJtKA=="],
"@jsquash/avif": ["@jsquash/avif@2.1.1", "", { "dependencies": { "wasm-feature-detect": "^1.2.11" } }, "sha512-LMRxd0fMgfCLtobDh0/sFYJMMiRJTNYSEEWvRDKXlAeZ08t3gI5V+1thIT0XjXJ+SVG7Zug9B0XPyx0Ti5VRNA=="],
"@jsquash/png": ["@jsquash/png@3.1.1", "", {}, "sha512-C10pc+0H6j0h8fENOfnGOvkXCmvpSQTDGlfGd0sHphZhPSGTyLjIrHba0FaZZdsKqA/wlmhYicUHb92vfZphaw=="],
"@hono/node-server": ["@hono/node-server@1.19.12", "", { "peerDependencies": { "hono": "^4" } }, "sha512-txsUW4SQ1iilgE0l9/e9VQWmELXifEFvmdA1j6WFh/aFPj99hIntrSsq/if0UWyGVkmrRPKA1wCeP+UCr1B9Uw=="],
"@modelcontextprotocol/sdk": ["@modelcontextprotocol/sdk@1.29.0", "", { "dependencies": { "@hono/node-server": "^1.19.9", "ajv": "^8.17.1", "ajv-formats": "^3.0.1", "content-type": "^1.0.5", "cors": "^2.8.5", "cross-spawn": "^7.0.5", "eventsource": "^3.0.2", "eventsource-parser": "^3.0.0", "express": "^5.2.1", "express-rate-limit": "^8.2.1", "hono": "^4.11.4", "jose": "^6.1.3", "json-schema-typed": "^8.0.2", "pkce-challenge": "^5.0.0", "raw-body": "^3.0.0", "zod": "^3.25 || ^4.0", "zod-to-json-schema": "^3.25.1" }, "peerDependencies": { "@cfworker/json-schema": "^4.1.1" }, "optionalPeers": ["@cfworker/json-schema"] }, "sha512-zo37mZA9hJWpULgkRpowewez1y6ML5GsXJPY8FI0tBBCd77HEvza4jDqRKOXgHNn867PVGCyTdzqpz0izu5ZjQ=="],
"@nodable/entities": ["@nodable/entities@3.0.0", "", {}, "sha512-8L9xFeTYKhm49xfIypoe2W5wV1m/3Z58kT+7kR9A8OyFxcPduI4VmxaUMQyKYrRjUoLLSXv6EKKID5Tvj9cUVw=="],
"@opentelemetry/api": ["@opentelemetry/api@1.9.0", "", {}, "sha512-3giAOQvZiH5F9bMlMiv8+GSPMeqg0dbaeo58/0SlA9sxSqZhnUtxzX9/2FzyhS9sWQf5S0GJE0AKBrFqjpeYcg=="],
"@smithy/chunked-blob-reader": ["@smithy/chunked-blob-reader@5.2.2", "", { "dependencies": { "tslib": "^2.6.2" } }, "sha512-St+kVicSyayWQca+I1rGitaOEH6uKgE8IUWoYnnEX26SWdWQcL6LvMSD19Lg+vYHKdT9B2Zuu7rd3i6Wnyb/iw=="],
"@smithy/chunked-blob-reader-native": ["@smithy/chunked-blob-reader-native@4.2.3", "", { "dependencies": { "@smithy/util-base64": "^4.3.2", "tslib": "^2.6.2" } }, "sha512-jA5k5Udn7Y5717L86h4EIv06wIr3xn8GM1qHRi/Nf31annXcXHJjBKvgztnbn2TxH3xWrPBfgwHsOwZf0UmQWw=="],
@@ -275,63 +218,31 @@
"@smithy/uuid": ["@smithy/uuid@1.1.2", "", { "dependencies": { "tslib": "^2.6.2" } }, "sha512-O/IEdcCUKkubz60tFbGA7ceITTAJsty+lBjNoorP4Z6XRqaFb/OjQjZODophEcuq68nKm6/0r+6/lLQ+XVpk8g=="],
"@standard-schema/spec": ["@standard-schema/spec@1.1.0", "", {}, "sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w=="],
"@types/body-parser": ["@types/body-parser@1.19.6", "", { "dependencies": { "@types/connect": "*", "@types/node": "*" } }, "sha512-HLFeCYgz89uk22N5Qg3dvGvsv46B8GLvKKo1zKG4NybA8U2DiEO3w9lqGg29t/tfLRJpJ6iQxnVw4OnB7MoM9g=="],
"@types/bun": ["@types/bun@1.3.11", "", { "dependencies": { "bun-types": "1.3.11" } }, "sha512-5vPne5QvtpjGpsGYXiFyycfpDF2ECyPcTSsFBMa0fraoxiQyMJ3SmuQIGhzPg2WJuWxVBoxWJ2kClYTcw/4fAg=="],
"@types/connect": ["@types/connect@3.4.38", "", { "dependencies": { "@types/node": "*" } }, "sha512-K6uROf1LD88uDQqJCktA4yzL1YYAK6NgfsI0v/mTgyPKWsX1CnJ0XPSDhViejru1GcRkLWb8RlzFYJRqGUbaug=="],
"@types/cookie-parser": ["@types/cookie-parser@1.4.10", "", { "peerDependencies": { "@types/express": "*" } }, "sha512-B4xqkqfZ8Wek+rCOeRxsjMS9OgvzebEzzLYw7NHYuvzb7IdxOkI0ZHGgeEBX4PUM7QGVvNSK60T3OvWj3YfBRg=="],
"@types/cors": ["@types/cors@2.8.19", "", { "dependencies": { "@types/node": "*" } }, "sha512-mFNylyeyqN93lfe/9CSxOGREz8cpzAhH+E93xJ4xWQf62V8sQ/24reV2nyzUWM6H6Xji+GGHpkbLe7pVoUEskg=="],
"@types/express": ["@types/express@5.0.6", "", { "dependencies": { "@types/body-parser": "*", "@types/express-serve-static-core": "^5.0.0", "@types/serve-static": "^2" } }, "sha512-sKYVuV7Sv9fbPIt/442koC7+IIwK5olP1KWeD88e/idgoJqDm3JV/YUiPwkoKK92ylff2MGxSz1CSjsXelx0YA=="],
"@types/express-serve-static-core": ["@types/express-serve-static-core@5.1.1", "", { "dependencies": { "@types/node": "*", "@types/qs": "*", "@types/range-parser": "*", "@types/send": "*" } }, "sha512-v4zIMr/cX7/d2BpAEX3KNKL/JrT1s43s96lLvvdTmza1oEvDudCqK9aF/djc/SWgy8Yh0h30TZx5VpzqFCxk5A=="],
"@types/http-errors": ["@types/http-errors@2.0.5", "", {}, "sha512-r8Tayk8HJnX0FztbZN7oVqGccWgw98T/0neJphO91KkmOzug1KkofZURD4UaD5uH8AqcFLfdPErnBod0u71/qg=="],
"@types/js-yaml": ["@types/js-yaml@3.12.10", "", {}, "sha512-/Mtaq/wf+HxXpvhzFYzrzCqNRcA958sW++7JOFC8nPrZcvfi/TrzOaaGbvt27ltJB2NQbHVAg5a1wUCsyMH7NA=="],
"@types/node": ["@types/node@25.5.2", "", { "dependencies": { "undici-types": "~7.18.0" } }, "sha512-tO4ZIRKNC+MDWV4qKVZe3Ql/woTnmHDr5JD8UI5hn2pwBrHEwOEMZK7WlNb5RKB6EoJ02gwmQS9OrjuFnZYdpg=="],
"@types/node": ["@types/node@18.19.130", "", { "dependencies": { "undici-types": "~5.26.4" } }, "sha512-GRaXQx6jGfL8sKfaIDD6OupbIHBr9jv7Jnaml9tB7l4v068PAOXqfcujMMo5PhbIs6ggR1XODELqahT2R8v0fg=="],
"@types/node-fetch": ["@types/node-fetch@2.6.13", "", { "dependencies": { "@types/node": "*", "form-data": "^4.0.4" } }, "sha512-QGpRVpzSaUs30JBSGPjOg4Uveu384erbHBoT1zeONvyCfwQxIkUshLAOqN/k9EjGviPRmWTTe6aH2qySWKTVSw=="],
"@types/qs": ["@types/qs@6.15.0", "", {}, "sha512-JawvT8iBVWpzTrz3EGw9BTQFg3BQNmwERdKE22vlTxawwtbyUSlMppvZYKLZzB5zgACXdXxbD3m1bXaMqP/9ow=="],
"@types/range-parser": ["@types/range-parser@1.2.7", "", {}, "sha512-hKormJbkJqzQGhziax5PItDUTMAM9uE2XXQmM37dyd4hVM+5aVl7oVxMVUiVQn2oCQFN/LKCZdvSM0pFRqbSmQ=="],
"@types/send": ["@types/send@1.2.1", "", { "dependencies": { "@types/node": "*" } }, "sha512-arsCikDvlU99zl1g69TcAB3mzZPpxgw0UQnaHeC1Nwb015xp8bknZv5rIfri9xTOcMuaVgvabfIRA7PSZVuZIQ=="],
"@types/serve-static": ["@types/serve-static@2.2.0", "", { "dependencies": { "@types/http-errors": "*", "@types/node": "*" } }, "sha512-8mam4H1NHLtu7nmtalF7eyBH14QyOASmcxHhSfEoRyr0nP/YdoesEtU+uSRvMe96TW/HPTtkoKqQLl53N7UXMQ=="],
"@vercel/oidc": ["@vercel/oidc@3.2.0", "", {}, "sha512-UycprH3T6n3jH0k44NHMa7pnFHGu/N05MjojYr+Mc6I7obkoLIJujSWwin1pCvdy/eOxrI/l3uDLQsmcrOb4ug=="],
"abort-controller": ["abort-controller@3.0.0", "", { "dependencies": { "event-target-shim": "^5.0.0" } }, "sha512-h8lQ8tacZYnR3vNQTgibj+tODHI5/+l06Au2Pcriv/Gmet0eaj4TwWH41sO9wnHDiQsEj19q0drzdWdeAHtweg=="],
"accepts": ["accepts@2.0.0", "", { "dependencies": { "mime-types": "^3.0.0", "negotiator": "^1.0.0" } }, "sha512-5cvg6CtKwfgdmVqY1WIiXKc3Q1bkRqGLi+2W/6ao+6Y7gu/RCwRuAhGEzh5B4KlszSuTLgZYuqFqo5bImjNKng=="],
"agentkeepalive": ["agentkeepalive@4.6.0", "", { "dependencies": { "humanize-ms": "^1.2.1" } }, "sha512-kja8j7PjmncONqaTsB8fQ+wE2mSU2DJ9D4XKoJ5PFWIdRMa6SLSN1ff4mOr4jCbfRSsxR4keIiySJU0N9T5hIQ=="],
"ai": ["ai@6.0.174", "", { "dependencies": { "@ai-sdk/gateway": "3.0.109", "@ai-sdk/provider": "3.0.10", "@ai-sdk/provider-utils": "4.0.26", "@opentelemetry/api": "1.9.0" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-bTrfLUWHWtkjzWyCY4bmyuk4Qvmj4S4NSNsXyNSVVqkmftQNtxRj7dzUoMeQDBBwlJO6fC7m2Q/lNOPqQQfAGA=="],
"ajv": ["ajv@8.18.0", "", { "dependencies": { "fast-deep-equal": "^3.1.3", "fast-uri": "^3.0.1", "json-schema-traverse": "^1.0.0", "require-from-string": "^2.0.2" } }, "sha512-PlXPeEWMXMZ7sPYOHqmDyCJzcfNrUr3fGNKtezX14ykXOEIvyK81d+qydx89KY5O71FKMPaQ2vBfBFI5NHR63A=="],
"ajv-formats": ["ajv-formats@3.0.1", "", { "dependencies": { "ajv": "^8.0.0" } }, "sha512-8iUql50EUR+uUcdRQ3HDqa6EVyo3docL8g5WJ3FNcWmu62IbkGUue/pEyLBW8VGKKucTPgqeks4fIU1DA4yowQ=="],
"anynum": ["anynum@1.0.1", "", {}, "sha512-N6//FLET/tXYNM/F6ABca1oH6fWB+KlTt909Le28WMDBk8oaT4vY17DCrwg2MvmuqUKt3Ni4N5dGJ/EoBgcO6A=="],
"argparse": ["argparse@1.0.10", "", { "dependencies": { "sprintf-js": "~1.0.2" } }, "sha512-o5Roy6tNG4SL/FOkCAN6RzjiakZS25RLYFrcMttJqbdd8BWrnA+fGz57iN5Pb06pvBGvl5gQ0B48dJlslXvoTg=="],
"asynckit": ["asynckit@0.4.0", "", {}, "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q=="],
"body-parser": ["body-parser@2.3.0", "", { "dependencies": { "bytes": "^3.1.2", "content-type": "^2.0.0", "debug": "^4.4.3", "http-errors": "^2.0.1", "iconv-lite": "^0.7.2", "on-finished": "^2.4.1", "qs": "^6.15.2", "raw-body": "^3.0.2", "type-is": "^2.1.0" } }, "sha512-2cGmJupaNgg+QUwVLAucDuWuoMZ6EX9iHDRswZ5lsNYEmwPaRknMPCLZz07yTzVq/83p4o/wzbDZbBrTvGGTIw=="],
"body-parser": ["body-parser@2.2.2", "", { "dependencies": { "bytes": "^3.1.2", "content-type": "^1.0.5", "debug": "^4.4.3", "http-errors": "^2.0.0", "iconv-lite": "^0.7.0", "on-finished": "^2.4.1", "qs": "^6.14.1", "raw-body": "^3.0.1", "type-is": "^2.0.1" } }, "sha512-oP5VkATKlNwcgvxi0vM0p/D3n2C3EReYVX+DNYs5TjZFn/oQt2j+4sVJtSMr18pdRr8wjTcBl6LoV+FUwzPmNA=="],
"bowser": ["bowser@2.14.1", "", {}, "sha512-tzPjzCxygAKWFOJP011oxFHs57HzIhOEracIgAePE4pqB3LikALKnSzUyU4MGs9/iCEUuHlAJTjTc5M+u7YEGg=="],
"bun-types": ["bun-types@1.3.13", "", { "dependencies": { "@types/node": "*" } }, "sha512-QXKeHLlOLqQX9LgYaHJfzdBaV21T63HhFJnvuRCcjZiaUDpbs5ED1MgxbMra71CsryN/1dAoXuJJJwIv/2drVA=="],
"bun-types": ["bun-types@1.3.11", "", { "dependencies": { "@types/node": "*" } }, "sha512-1KGPpoxQWl9f6wcZh57LvrPIInQMn2TQ7jsgxqpRzg+l0QPOFvJVH7HmvHo/AiPgwXy+/Thf6Ov3EdVn1vOabg=="],
"bytes": ["bytes@3.1.2", "", {}, "sha512-/Nf7TyzTx6S3yRJObOAV7956r8cr2+Oj8AC5dt8wSP3BQAoeX58NoHyCU8P8zGkNXStjTSi6fzO6F0pBdcYbEg=="],
@@ -339,8 +250,6 @@
"call-bound": ["call-bound@1.0.4", "", { "dependencies": { "call-bind-apply-helpers": "^1.0.2", "get-intrinsic": "^1.3.0" } }, "sha512-+ys997U96po4Kx/ABpBCqhA9EuxJaQWDQg7295H4hBphv3IZg0boBKuwYpt4YXp6MZ5AmZQnU/tyMTlRpaSejg=="],
"chokidar": ["chokidar@4.0.3", "", { "dependencies": { "readdirp": "^4.0.1" } }, "sha512-Qgzu8kfBvo+cA4962jnP1KkS6Dop5NS6g7R5LFYJr4b8Ub94PPQXUksCw9PvXoeXPRRddRNC5C1JQUR2SMGtnA=="],
"combined-stream": ["combined-stream@1.0.8", "", { "dependencies": { "delayed-stream": "~1.0.0" } }, "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg=="],
"content-disposition": ["content-disposition@1.0.1", "", {}, "sha512-oIXISMynqSqm241k6kcQ5UwttDILMK4BiurCfGEREw6+X9jkkpEe5T9FZaApyLGGOnFuyMWZpdolTXMtvEJ08Q=="],
@@ -349,9 +258,7 @@
"cookie": ["cookie@0.7.2", "", {}, "sha512-yki5XnKuf750l50uGTllt6kKILY4nQ1eNIQatoXEByZ5dWgnKqbnqmTrBE5B4N7lrMJKQ2ytWMiTO2o0v6Ew/w=="],
"cookie-parser": ["cookie-parser@1.4.7", "", { "dependencies": { "cookie": "0.7.2", "cookie-signature": "1.0.6" } }, "sha512-nGUvgXnotP3BsjiLX2ypbQnWoGUPIIfHQNZkkC668ntrzGWEZVW70HDEB1qnNGMicPje6EttlIgzo51YSwNQGw=="],
"cookie-signature": ["cookie-signature@1.0.6", "", {}, "sha512-QADzlaHc8icV8I7vbaJXJwod9HWYp8uCqf1xa4OfNu1T7JVxQIrUgOWtHdNDtPiywmFbiS12VjotIXLrKM3orQ=="],
"cookie-signature": ["cookie-signature@1.2.2", "", {}, "sha512-D76uU73ulSXrD1UXF4KE2TMxVVwhsnCgfAyTg9k8P6KGZjlXKrOLe4dJQKI3Bxi5wjesZoFXJWElNWBjPZMbhg=="],
"cors": ["cors@2.8.6", "", { "dependencies": { "object-assign": "^4", "vary": "^1" } }, "sha512-tJtZBBHA6vjIAaF6EnIaq6laBBP9aq/Y3ouVJjEfoHbRBcHBAHYcMh/w8LDrk2PvIMMq8gmopa5D4V8RmbrxGw=="],
@@ -387,29 +294,25 @@
"eventsource": ["eventsource@3.0.7", "", { "dependencies": { "eventsource-parser": "^3.0.1" } }, "sha512-CRT1WTyuQoD771GW56XEZFQ/ZoSfWid1alKGDYMmkt2yl8UXrVR4pspqWNEcqKvVIzg6PAltWjxcSSPrboA4iA=="],
"eventsource-parser": ["eventsource-parser@3.0.8", "", {}, "sha512-70QWGkr4snxr0OXLRWsFLeRBIRPuQOvt4s8QYjmUlmlkyTZkRqS7EDVRZtzU3TiyDbXSzaOeF0XUKy8PchzukQ=="],
"exifr": ["exifr@7.1.3", "", {}, "sha512-g/aje2noHivrRSLbAUtBPWFbxKdKhgj/xr1vATDdUXPOFYJlQ62Ft0oy+72V6XLIpDJfHs6gXLbBLAolqOXYRw=="],
"eventsource-parser": ["eventsource-parser@3.0.6", "", {}, "sha512-Vo1ab+QXPzZ4tCa8SwIHJFaSzy4R6SHf7BY79rFBDf0idraZWAkYrDjDj8uWaSm3S2TK+hJ7/t1CEmZ7jXw+pg=="],
"express": ["express@5.2.1", "", { "dependencies": { "accepts": "^2.0.0", "body-parser": "^2.2.1", "content-disposition": "^1.0.0", "content-type": "^1.0.5", "cookie": "^0.7.1", "cookie-signature": "^1.2.1", "debug": "^4.4.0", "depd": "^2.0.0", "encodeurl": "^2.0.0", "escape-html": "^1.0.3", "etag": "^1.8.1", "finalhandler": "^2.1.0", "fresh": "^2.0.0", "http-errors": "^2.0.0", "merge-descriptors": "^2.0.0", "mime-types": "^3.0.0", "on-finished": "^2.4.1", "once": "^1.4.0", "parseurl": "^1.3.3", "proxy-addr": "^2.0.7", "qs": "^6.14.0", "range-parser": "^1.2.1", "router": "^2.2.0", "send": "^1.1.0", "serve-static": "^2.2.0", "statuses": "^2.0.1", "type-is": "^2.0.1", "vary": "^1.1.2" } }, "sha512-hIS4idWWai69NezIdRt2xFVofaF4j+6INOpJlVOLDO8zXGpUVEVzIYk12UUi2JzjEzWL3IOAxcTubgz9Po0yXw=="],
"express-rate-limit": ["express-rate-limit@7.5.1", "", { "peerDependencies": { "express": ">= 4.11" } }, "sha512-7iN8iPMDzOMHPUYllBEsQdWVB6fPDMPqwjBaFrgr4Jgr/+okjvzAy+UHlYYL/Vs0OsOrMkwS6PJDkFlJwoxUnw=="],
"express-rate-limit": ["express-rate-limit@8.3.2", "", { "dependencies": { "ip-address": "10.1.0" }, "peerDependencies": { "express": ">= 4.11" } }, "sha512-77VmFeJkO0/rvimEDuUC5H30oqUC4EyOhyGccfqoLebB0oiEYfM7nwPrsDsBL1gsTpwfzX8SFy2MT3TDyRq+bg=="],
"extend-shallow": ["extend-shallow@2.0.1", "", { "dependencies": { "is-extendable": "^0.1.0" } }, "sha512-zCnTtlxNoAiDc3gqY2aYAWFx7XWWiasuF2K8Me5WbN8otHKTUKBwjPtNpRs/rbUZm7KxWAaNj7P1a/p52GbVug=="],
"fast-check": ["fast-check@4.8.0", "", { "dependencies": { "pure-rand": "^8.0.0" } }, "sha512-GOJ158CUMnN6cSahsv4+ExARvIDuzzinFjkp0E9WtiBa5zcVeLozVkWaE4IzFcc+Y48Wp1EDlUZsXRyAztQcSg=="],
"fast-deep-equal": ["fast-deep-equal@3.1.3", "", {}, "sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q=="],
"fast-uri": ["fast-uri@3.1.4", "", {}, "sha512-8JnbkQ4juDyvYs4mgFGQqg4yCYtFDtUtmp2QIQq11ZZe5CFQ5wcqm1rqDgAh/QdMySuBnPzMUiJUNZG5N/AiQw=="],
"fast-uri": ["fast-uri@3.1.0", "", {}, "sha512-iPeeDKJSWf4IEOasVVrknXpaBV0IApz/gp7S2bb7Z4Lljbl2MGJRqInZiUrQwV16cpzw/D3S5j5Julj/gT52AA=="],
"fast-xml-builder": ["fast-xml-builder@1.3.0", "", { "dependencies": { "path-expression-matcher": "^1.6.2", "xml-naming": "^0.3.0" } }, "sha512-F74cZEdCvuw9P41GAC3rod4X04jjWGM1JPEv/GWSqFTWLsdyMSBMBMlm9Hk3GLBgLBbdBNY8yee0pQh2RBVESQ=="],
"fast-xml-builder": ["fast-xml-builder@1.1.4", "", { "dependencies": { "path-expression-matcher": "^1.1.3" } }, "sha512-f2jhpN4Eccy0/Uz9csxh3Nu6q4ErKxf0XIsasomfOihuSUa3/xw6w8dnOtCDgEItQFJG8KyXPzQXzcODDrrbOg=="],
"fast-xml-parser": ["fast-xml-parser@5.10.1", "", { "dependencies": { "@nodable/entities": "^3.0.0", "fast-xml-builder": "^1.2.0", "is-unsafe": "^2.0.0", "path-expression-matcher": "^1.6.2", "strnum": "^2.4.1", "xml-naming": "^0.3.0" }, "bin": { "fxparser": "src/cli/cli.js" } }, "sha512-IEMIf7298kXuZSRFoGfMYrl7is8LpavODgbNz1cwIudv7KwVFnuU+UsMporfq6PD6aXSlawZlARiA3UywCTfMw=="],
"fast-xml-parser": ["fast-xml-parser@5.5.8", "", { "dependencies": { "fast-xml-builder": "^1.1.4", "path-expression-matcher": "^1.2.0", "strnum": "^2.2.0" }, "bin": { "fxparser": "src/cli/cli.js" } }, "sha512-Z7Fh2nVQSb2d+poDViM063ix2ZGt9jmY1nWhPfHBOK2Hgnb/OW3P4Et3P/81SEej0J7QbWtJqxO05h8QYfK7LQ=="],
"finalhandler": ["finalhandler@2.1.1", "", { "dependencies": { "debug": "^4.4.0", "encodeurl": "^2.0.0", "escape-html": "^1.0.3", "on-finished": "^2.4.1", "parseurl": "^1.3.3", "statuses": "^2.0.1" } }, "sha512-S8KoZgRZN+a5rNwqTxlZZePjT/4cnm0ROV70LedRHZ0p8u9fRID0hJUZQpkKLzro8LfmC8sx23bY6tVNxv8pQA=="],
"form-data": ["form-data@4.0.6", "", { "dependencies": { "asynckit": "^0.4.0", "combined-stream": "^1.0.8", "es-set-tostringtag": "^2.1.0", "hasown": "^2.0.4", "mime-types": "^2.1.35" } }, "sha512-vKatAh4SlVfgbv+YtmhiRjhEMJsYpsG1Y2rMQtR+SVSbytsSD1YGzDIcrAJmdFec88u/+VoGmxnl+80gL1tRCQ=="],
"form-data": ["form-data@4.0.5", "", { "dependencies": { "asynckit": "^0.4.0", "combined-stream": "^1.0.8", "es-set-tostringtag": "^2.1.0", "hasown": "^2.0.2", "mime-types": "^2.1.12" } }, "sha512-8RipRLol37bNs2bhoV67fiTEvdTrbMUYcFTiy3+wuuOnUog2QBHCZWXDRijWQfAkhBj2Uf5UnVaiWwA5vdd82w=="],
"form-data-encoder": ["form-data-encoder@1.7.2", "", {}, "sha512-qfqtYan3rxrnCk1VYaA4H+Ms9xdpPqvLZa6xmMgFvhO32x7/3J/ExcTd6qpxM0vH2GdMI+poehyBZvqfMTto8A=="],
@@ -433,11 +336,9 @@
"has-tostringtag": ["has-tostringtag@1.0.2", "", { "dependencies": { "has-symbols": "^1.0.3" } }, "sha512-NqADB8VjPFLM2V0VvHUewwwsw0ZWBaIdgo+ieHtK3hasLz4qeCRjYcqfB6AQrBggRKppKF8L52/VqdVsO47Dlw=="],
"hasown": ["hasown@2.0.4", "", { "dependencies": { "function-bind": "^1.1.2" } }, "sha512-T2UbfbBEF32wiepXIsMlTW9+dDYC6wMh/t/vYA4tuOMKqWz/n3vr1NFSxQiyP+zk2mXsoMA/i/7qV6LKut1t1A=="],
"hasown": ["hasown@2.0.2", "", { "dependencies": { "function-bind": "^1.1.2" } }, "sha512-0hJU9SCPvmMzIBdZFqNPXWa6dqh7WdH0cII9y+CyS8rG3nL48Bclra9HmKhVVUHyPWNH5Y7xDwAB7bfgSjkUMQ=="],
"heic-decode": ["heic-decode@2.1.0", "", { "dependencies": { "libheif-js": "^1.19.8" } }, "sha512-0fB3O3WMk38+PScbHLVp66jcNhsZ/ErtQ6u2lMYu/YxXgbBtl+oKOhGQHa4RpvE68k8IzbWkABzHnyAIjR758A=="],
"hono": ["hono@4.12.30", "", {}, "sha512-emn+JoJjrN9YTpRDS5it/UI2SO9BAE37T6I3d963RxcZ81G9A4pr2SZTEiiaiKbzx+NKRg5BZ89fCL7gCJCUog=="],
"hono": ["hono@4.12.10", "", {}, "sha512-mx/p18PLy5og9ufies2GOSUqep98Td9q4i/EF6X7yJgAiIopxqdfIO3jbqsi3jRgTgw88jMDEzVKi+V2EF+27w=="],
"http-errors": ["http-errors@2.0.1", "", { "dependencies": { "depd": "~2.0.0", "inherits": "~2.0.4", "setprototypeof": "~1.2.0", "statuses": "~2.0.2", "toidentifier": "~1.0.1" } }, "sha512-4FbRdAX+bSdmo4AUFuS0WNiPz8NgFt+r8ThgNWmlrjQjt1Q7ZR9+zTlce2859x4KSXrwIsaeTqDoKQmtP8pLmQ=="],
@@ -447,7 +348,7 @@
"inherits": ["inherits@2.0.4", "", {}, "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ=="],
"ip-address": ["ip-address@10.2.0", "", {}, "sha512-/+S6j4E9AHvW9SWMSEY9Xfy66O5PWvVEJ08O0y5JGyEKQpojb0K0GKpz/v5HJ/G0vi3D2sjGK78119oXZeE0qA=="],
"ip-address": ["ip-address@10.1.0", "", {}, "sha512-XXADHxXmvT9+CRxhXg56LJovE+bmWnEWB78LB83VZTprKTmaC5QfruXocxzTZ2Kl0DNwKuBdlIhjL8LeY8Sf8Q=="],
"ipaddr.js": ["ipaddr.js@1.9.1", "", {}, "sha512-0KI/607xoxSToH7GjN1FfSbLoU0+btTicjsQSWQlh/hZykN8KpmMf7uYwPW3R+akZ6R/w18ZlXSHBYXiYUPO3g=="],
@@ -455,15 +356,11 @@
"is-promise": ["is-promise@4.0.0", "", {}, "sha512-hvpoI6korhJMnej285dSg6nu1+e6uxs7zG3BYAm5byqDsgJNWwxzM6z6iZiAgQR4TJ30JmBTOwqZUw3WlyH3AQ=="],
"is-unsafe": ["is-unsafe@2.0.0", "", {}, "sha512-2LdV822R+wmI86unXA93WCFpL6g+av8ynWk0nrHyJqGop5VoocYsSLFgN8jrfalT6iGeLNM4KXuVSsULP53kEA=="],
"isexe": ["isexe@2.0.0", "", {}, "sha512-RHxMLp9lnKHGHRng9QFhRCMbYAcVpn69smSGcq3f36xjgVVWThj4qqLbTLlq7Ssj8B+fIQ1EuCEGI2lKsyQeIw=="],
"jose": ["jose@6.2.2", "", {}, "sha512-d7kPDd34KO/YnzaDOlikGpOurfF0ByC2sEV4cANCtdqLlTfBlw2p14O/5d/zv40gJPbIQxfES3nSx1/oYNyuZQ=="],
"js-yaml": ["js-yaml@3.15.0", "", { "dependencies": { "argparse": "^1.0.7", "esprima": "^4.0.0" }, "bin": { "js-yaml": "bin/js-yaml.js" } }, "sha512-ttBQIIQPDeLjpPOohtUdXuXUVoA2uIB6fEH9HyJ7234s5mBJ5wTx20njxplLZQgLaOfpmPQA7X2t5AX6tIPbog=="],
"json-schema": ["json-schema@0.4.0", "", {}, "sha512-es94M3nTIfsEPisRafak+HDLfHXnKBhV3vU5eqPcS3flIWqcxJWgXHXiey3YrpaNsanY5ei1VoYEbOzijuq9BA=="],
"js-yaml": ["js-yaml@3.14.2", "", { "dependencies": { "argparse": "^1.0.7", "esprima": "^4.0.0" }, "bin": { "js-yaml": "bin/js-yaml.js" } }, "sha512-PMSmkqxr106Xa156c2M265Z+FTrPl+oxd/rgOQy2tijQeK5TxQ43psO1ZCwhVOSdnn+RzkzlRz/eY4BgJBYVpg=="],
"json-schema-traverse": ["json-schema-traverse@1.0.0", "", {}, "sha512-NM8/P9n3XjXhIZn1lLhkFaACTOURQXjWhV4BA/RnOv8xvgqtqpAX9IO4mRQxSx1Rlo4tqzeqb0sOlruaOy3dug=="],
@@ -471,9 +368,7 @@
"kind-of": ["kind-of@6.0.3", "", {}, "sha512-dcS1ul+9tmeD95T+x28/ehLgd9mENa3LsvDTtzm3vyBEO7RPptvAD+t44WVXaUjTBRcrpFeFlC8WCruUR456hw=="],
"libheif-js": ["libheif-js@1.19.8", "", {}, "sha512-vQJWusIxO7wavpON1dusciL8Go9jsIQ+EUrckauFYAiSTjcmLAsuJh3SszLpvkwPci3JcL41ek2n+LUZGFpPIQ=="],
"marked": ["marked@18.0.6", "", { "bin": { "marked": "bin/marked.js" } }, "sha512-MrV5puXBfuiy6wl6DLaq3BtIJQAJToAd5zt/ZKhRfGRAuFPALE7/4Y7jnxRQoEgK/pBgurGqLyAuRgZ2xOjr6w=="],
"marked": ["marked@18.0.0", "", { "bin": { "marked": "bin/marked.js" } }, "sha512-2e7Qiv/HJSXj8rDEpgTvGKsP8yYtI9xXHKDnrftrmnrJPaFNM7VRb2YCzWaX4BP1iCJ/XPduzDJZMFoqTCcIMA=="],
"math-intrinsics": ["math-intrinsics@1.1.0", "", {}, "sha512-/IXtbwEk5HTPyEwyKX6hGkYXxM9nbj64B+ilVJnC/R6B0pH5G4V3b0pVbL7DBj4tkhBAppbQUlf6F6Xl9LHu1g=="],
@@ -505,7 +400,7 @@
"parseurl": ["parseurl@1.3.3", "", {}, "sha512-CiyeOxFT/JZyN5m0z9PfXw4SCBJ6Sygz1Dpl0wqjlhDEGGBP1GnsUVEL0p63hoG1fcj3fHynXi9NYO4nWOL+qQ=="],
"path-expression-matcher": ["path-expression-matcher@1.6.2", "", {}, "sha512-enSlaiat05iasnzmgNxRj8reFdj3puY2QpNgP1aPIaVfT6nn9ICuPoFlKHk8EN22HcwewshO+mN2DGbkCEOtqQ=="],
"path-expression-matcher": ["path-expression-matcher@1.5.0", "", {}, "sha512-cbrerZV+6rvdQrrD+iGMcZFEiiSrbv9Tfdkvnusy6y0x0GKBXREFg/Y65GhIfm0tnLntThhzCnfKwp1WRjeCyQ=="],
"path-key": ["path-key@3.1.1", "", {}, "sha512-ojmeN0qd+y0jszEtoY48r0Peq5dwMEkIlCOu6Q5f41lfkswXuKtYrhgoTpLnyIcHm24Uhqx+5Tqm2InSwLhE6Q=="],
@@ -519,16 +414,12 @@
"proxy-addr": ["proxy-addr@2.0.7", "", { "dependencies": { "forwarded": "0.2.0", "ipaddr.js": "1.9.1" } }, "sha512-llQsMLSUDUPT44jdrU/O37qlnifitDP+ZwrmmZcoSKyLKvtZxpyV0n2/bD/N4tBAAZ/gJEdZU7KMraoK1+XYAg=="],
"pure-rand": ["pure-rand@8.4.0", "", {}, "sha512-IoM8YF/jY0hiugFo/wOWqfmarlE6J0wc6fDK1PhftMk7MGhVZl88sZimmqBBFomLOCSmcCCpsfj7wXASCpvK9A=="],
"qs": ["qs@6.15.3", "", { "dependencies": { "es-define-property": "^1.0.1", "side-channel": "^1.1.1" } }, "sha512-O9gl3zCl5h5blw1KGUzQKhA5oUXSl8rwUIM5o0S3nCXMliSvy5Dzx7/DJcI+SwgICv+IneSZwhBh1oSyEHA71A=="],
"qs": ["qs@6.15.0", "", { "dependencies": { "side-channel": "^1.1.0" } }, "sha512-mAZTtNCeetKMH+pSjrb76NAM8V9a05I9aBZOHztWy/UqcJdQYNsf59vrRKWnojAT9Y+GbIvoTBC++CPHqpDBhQ=="],
"range-parser": ["range-parser@1.2.1", "", {}, "sha512-Hrgsx+orqoygnmhFbKaHE6c296J+HTAQXoxEF6gNupROmmGJRoyzfG3ccAveqCBrwr/2yxQ5BVd/GTl5agOwSg=="],
"raw-body": ["raw-body@3.0.2", "", { "dependencies": { "bytes": "~3.1.2", "http-errors": "~2.0.1", "iconv-lite": "~0.7.0", "unpipe": "~1.0.0" } }, "sha512-K5zQjDllxWkf7Z5xJdV0/B0WTNqx6vxG70zJE4N0kBs4LovmEYWJzQGxC9bS9RAKu3bgM40lrd5zoLJ12MQ5BA=="],
"readdirp": ["readdirp@4.1.2", "", {}, "sha512-GDhwkLfywWL2s6vEjyhri+eXmfH6j1L7JE27WhqLeYzoh/A3DBaYGEj2H/HFZCn/kMfim73FXxEJTw06WtxQwg=="],
"require-from-string": ["require-from-string@2.0.2", "", {}, "sha512-Xf0nWe6RseziFMu+Ap9biiUbmplq6S9/p+7w7YXP/JBHhrUDDUhwa+vANyubuqfZWTveU//DYVGsDG7RKL/vEw=="],
"router": ["router@2.2.0", "", { "dependencies": { "debug": "^4.4.0", "depd": "^2.0.0", "is-promise": "^4.0.0", "parseurl": "^1.3.3", "path-to-regexp": "^8.0.0" } }, "sha512-nLTrUKm2UyiL7rlhapu/Zl45FwNgkZGaCpZbIHajDYgwlJCOzLSk+cIPAnsEqV955GjILJnKbdQC1nVPz+gAYQ=="],
@@ -547,9 +438,9 @@
"shebang-regex": ["shebang-regex@3.0.0", "", {}, "sha512-7++dFhtcx3353uBaq8DDR4NuxBetBzC7ZQOhmTQInHEd6bSrXdiEyzCvG07Z44UYdLShWUyXt5M/yhz8ekcb1A=="],
"side-channel": ["side-channel@1.1.1", "", { "dependencies": { "es-errors": "^1.3.0", "object-inspect": "^1.13.4", "side-channel-list": "^1.0.1", "side-channel-map": "^1.0.1", "side-channel-weakmap": "^1.0.2" } }, "sha512-6x6dK6zJdpTzF4sQeNYxwtvBzf6Eg4GtlesS94HOvTudUeyK2WXAaIfmDgsyslYrRBeFIlsi54AYsFGUuhmvrQ=="],
"side-channel": ["side-channel@1.1.0", "", { "dependencies": { "es-errors": "^1.3.0", "object-inspect": "^1.13.3", "side-channel-list": "^1.0.0", "side-channel-map": "^1.0.1", "side-channel-weakmap": "^1.0.2" } }, "sha512-ZX99e6tRweoUXqR+VBrslhda51Nh5MTQwou5tnUDgbtyM0dBgmhEDtWGP/xbKn6hqfPRHujUNwz5fy/wbbhnpw=="],
"side-channel-list": ["side-channel-list@1.0.1", "", { "dependencies": { "es-errors": "^1.3.0", "object-inspect": "^1.13.4" } }, "sha512-mjn/0bi/oUURjc5Xl7IaWi/OJJJumuoJFQJfDDyO46+hBWsfaVM65TBHq2eoZBhzl9EchxOijpkbRC8SVBQU0w=="],
"side-channel-list": ["side-channel-list@1.0.0", "", { "dependencies": { "es-errors": "^1.3.0", "object-inspect": "^1.13.3" } }, "sha512-FCLHtRD/gnpCiCHEiJLOwdmFP+wzCmDEkc9y7NsYxeF4u7Btsn1ZuwgwJGxImImHicJArLP4R0yX4c2KCrMrTA=="],
"side-channel-map": ["side-channel-map@1.0.1", "", { "dependencies": { "call-bound": "^1.0.2", "es-errors": "^1.3.0", "get-intrinsic": "^1.2.5", "object-inspect": "^1.13.3" } }, "sha512-VCjCNfgMsby3tTdo02nbjtM/ewra6jPHmpThenkTYh8pG9ucZ/1P8So4u4FGBek/BjpOVsDCMoLA/iuBKIFXRA=="],
@@ -561,7 +452,7 @@
"strip-bom-string": ["strip-bom-string@1.0.0", "", {}, "sha512-uCC2VHvQRYu+lMh4My/sFNmF2klFymLX1wHJeXnbEJERpV/ZsVuonzerjfrGpIGF7LBVa1O7i9kjiWvJiFck8g=="],
"strnum": ["strnum@2.4.1", "", { "dependencies": { "anynum": "^1.0.1" } }, "sha512-M9eUSMT2dCB2cTNPG7UYj6KuK7RJR2SN2+yCV/fTW3xzTCS6EaGZ5pSMgDIjB7r8zSfTGk+dvvn9rTjpVS9Mwg=="],
"strnum": ["strnum@2.2.3", "", {}, "sha512-oKx6RUCuHfT3oyVjtnrmn19H1SiCqgJSg+54XqURKp5aCMbrXrhLjRN9TjuwMjiYstZ0MzDrHqkGZ5dFTKd+zg=="],
"toidentifier": ["toidentifier@1.0.1", "", {}, "sha512-o5sSPKEkg/DIQNmH43V0/uerLrpzVedkUh8tGNvaeXpfpuwjKenlSox/2O/BTlZUtEe+JG7s5YhEz608PlAHRA=="],
@@ -575,14 +466,12 @@
"typescript": ["typescript@5.9.3", "", { "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" } }, "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw=="],
"undici-types": ["undici-types@7.18.2", "", {}, "sha512-AsuCzffGHJybSaRrmr5eHr81mwJU3kjw6M+uprWvCXiNeN9SOGwQ3Jn8jb8m3Z6izVgknn1R0FTCEAP2QrLY/w=="],
"undici-types": ["undici-types@5.26.5", "", {}, "sha512-JlCMO+ehdEIKqlFxk6IfVoAUVmgz7cU7zD/h9XZ0qzeosSHmUJVOzSQvvYSYWXkFXC+IfLKSIffhv0sVZup6pA=="],
"unpipe": ["unpipe@1.0.0", "", {}, "sha512-pjy2bYhSsufwWlKwPc+l3cN7+wuJlK6uz0YdJEOlQDbl6jo/YlPi4mb8agUkVC8BF7V8NuzeyPNqRksA3hztKQ=="],
"vary": ["vary@1.1.2", "", {}, "sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg=="],
"wasm-feature-detect": ["wasm-feature-detect@1.8.0", "", {}, "sha512-zksaLKM2fVlnB5jQQDqKXXwYHLQUVH9es+5TOOHwGOVJOCeRBCiPjwSg+3tN2AdTCzjgli4jijCH290kXb/zWQ=="],
"web-streams-polyfill": ["web-streams-polyfill@4.0.0-beta.3", "", {}, "sha512-QW95TCTaHmsYfHDybGMwO5IJIM93I/6vTRk+daHTWFPhwh+C8Cg7j7XyKrwrj8Ib6vYXe0ocYNrmzY4xAAN6ug=="],
"web-tree-sitter": ["web-tree-sitter@0.22.6", "", {}, "sha512-hS87TH71Zd6mGAmYCvlgxeGDjqd9GTeqXNqTT+u0Gs51uIozNIaaq/kUAbV/Zf56jb2ZOyG8BxZs2GG9wbLi6Q=="],
@@ -595,53 +484,33 @@
"wrappy": ["wrappy@1.0.2", "", {}, "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ=="],
"xml-naming": ["xml-naming@0.3.0", "", {}, "sha512-ghig2TBE/H11aOVgmahA3MhimvkBr6JIYknH/Dhdk10nXwdbIqBJsbfMxpvFPG8bAw77gN29aQWvKpmVoPlvPQ=="],
"zod": ["zod@4.3.6", "", {}, "sha512-rftlrkhHZOcjDwkGlnUtZZkvaPHCsDATp4pGpuOOMDaTdDDXF91wuVDJoWoPsKX/3YPQ5fHuF3STjcYyKr+Qhg=="],
"zod-to-json-schema": ["zod-to-json-schema@3.25.2", "", { "peerDependencies": { "zod": "^3.25.28 || ^4" } }, "sha512-O/PgfnpT1xKSDeQYSCfRI5Gy3hPf91mKVDuYLUHZJMiDFptvP41MSnWofm8dnCm0256ZNfZIM7DSzuSMAFnjHA=="],
"@anthropic-ai/sdk/@types/node": ["@types/node@18.19.130", "", { "dependencies": { "undici-types": "~5.26.4" } }, "sha512-GRaXQx6jGfL8sKfaIDD6OupbIHBr9jv7Jnaml9tB7l4v068PAOXqfcujMMo5PhbIs6ggR1XODELqahT2R8v0fg=="],
"@aws-crypto/sha1-browser/@smithy/util-utf8": ["@smithy/util-utf8@2.3.0", "", { "dependencies": { "@smithy/util-buffer-from": "^2.2.0", "tslib": "^2.6.2" } }, "sha512-R8Rdn8Hy72KKcebgLiv8jQcQkXoLMOGGv5uI1/k0l+snqkOzQ1R0ChUBCxWMlBsFMekWjq0wRudIweFs7sKT5A=="],
"@aws-crypto/sha256-browser/@smithy/util-utf8": ["@smithy/util-utf8@2.3.0", "", { "dependencies": { "@smithy/util-buffer-from": "^2.2.0", "tslib": "^2.6.2" } }, "sha512-R8Rdn8Hy72KKcebgLiv8jQcQkXoLMOGGv5uI1/k0l+snqkOzQ1R0ChUBCxWMlBsFMekWjq0wRudIweFs7sKT5A=="],
"@aws-crypto/util/@smithy/util-utf8": ["@smithy/util-utf8@2.3.0", "", { "dependencies": { "@smithy/util-buffer-from": "^2.2.0", "tslib": "^2.6.2" } }, "sha512-R8Rdn8Hy72KKcebgLiv8jQcQkXoLMOGGv5uI1/k0l+snqkOzQ1R0ChUBCxWMlBsFMekWjq0wRudIweFs7sKT5A=="],
"@modelcontextprotocol/sdk/eventsource-parser": ["eventsource-parser@3.0.6", "", {}, "sha512-Vo1ab+QXPzZ4tCa8SwIHJFaSzy4R6SHf7BY79rFBDf0idraZWAkYrDjDj8uWaSm3S2TK+hJ7/t1CEmZ7jXw+pg=="],
"@types/node-fetch/@types/node": ["@types/node@25.5.2", "", { "dependencies": { "undici-types": "~7.18.0" } }, "sha512-tO4ZIRKNC+MDWV4qKVZe3Ql/woTnmHDr5JD8UI5hn2pwBrHEwOEMZK7WlNb5RKB6EoJ02gwmQS9OrjuFnZYdpg=="],
"@modelcontextprotocol/sdk/express-rate-limit": ["express-rate-limit@8.3.2", "", { "dependencies": { "ip-address": "10.1.0" }, "peerDependencies": { "express": ">= 4.11" } }, "sha512-77VmFeJkO0/rvimEDuUC5H30oqUC4EyOhyGccfqoLebB0oiEYfM7nwPrsDsBL1gsTpwfzX8SFy2MT3TDyRq+bg=="],
"@types/bun/bun-types": ["bun-types@1.3.11", "", { "dependencies": { "@types/node": "*" } }, "sha512-1KGPpoxQWl9f6wcZh57LvrPIInQMn2TQ7jsgxqpRzg+l0QPOFvJVH7HmvHo/AiPgwXy+/Thf6Ov3EdVn1vOabg=="],
"body-parser/content-type": ["content-type@2.0.0", "", {}, "sha512-j/O/d7GcZCyNl7/hwZAb606rzqkyvaDctLmckbxLzHvFBzTJHuGEdodATcP3yIRoDrLHkIATJuvzbFlp/ki2cQ=="],
"body-parser/type-is": ["type-is@2.1.0", "", { "dependencies": { "content-type": "^2.0.0", "media-typer": "^1.1.0", "mime-types": "^3.0.0" } }, "sha512-faYHw0anBbc/kWF3zFTEnxSFOAGUX9GFbOBthvDdLsIlEoWOFOtS0zgCiQYwIskL9iGXZL3kAXD8OoZ4GmMATA=="],
"es-set-tostringtag/hasown": ["hasown@2.0.2", "", { "dependencies": { "function-bind": "^1.1.2" } }, "sha512-0hJU9SCPvmMzIBdZFqNPXWa6dqh7WdH0cII9y+CyS8rG3nL48Bclra9HmKhVVUHyPWNH5Y7xDwAB7bfgSjkUMQ=="],
"eventsource/eventsource-parser": ["eventsource-parser@3.0.6", "", {}, "sha512-Vo1ab+QXPzZ4tCa8SwIHJFaSzy4R6SHf7BY79rFBDf0idraZWAkYrDjDj8uWaSm3S2TK+hJ7/t1CEmZ7jXw+pg=="],
"express/cookie-signature": ["cookie-signature@1.2.2", "", {}, "sha512-D76uU73ulSXrD1UXF4KE2TMxVVwhsnCgfAyTg9k8P6KGZjlXKrOLe4dJQKI3Bxi5wjesZoFXJWElNWBjPZMbhg=="],
"bun-types/@types/node": ["@types/node@25.5.2", "", { "dependencies": { "undici-types": "~7.18.0" } }, "sha512-tO4ZIRKNC+MDWV4qKVZe3Ql/woTnmHDr5JD8UI5hn2pwBrHEwOEMZK7WlNb5RKB6EoJ02gwmQS9OrjuFnZYdpg=="],
"form-data/mime-types": ["mime-types@2.1.35", "", { "dependencies": { "mime-db": "1.52.0" } }, "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw=="],
"get-intrinsic/hasown": ["hasown@2.0.2", "", { "dependencies": { "function-bind": "^1.1.2" } }, "sha512-0hJU9SCPvmMzIBdZFqNPXWa6dqh7WdH0cII9y+CyS8rG3nL48Bclra9HmKhVVUHyPWNH5Y7xDwAB7bfgSjkUMQ=="],
"openai/@types/node": ["@types/node@18.19.130", "", { "dependencies": { "undici-types": "~5.26.4" } }, "sha512-GRaXQx6jGfL8sKfaIDD6OupbIHBr9jv7Jnaml9tB7l4v068PAOXqfcujMMo5PhbIs6ggR1XODELqahT2R8v0fg=="],
"@anthropic-ai/sdk/@types/node/undici-types": ["undici-types@5.26.5", "", {}, "sha512-JlCMO+ehdEIKqlFxk6IfVoAUVmgz7cU7zD/h9XZ0qzeosSHmUJVOzSQvvYSYWXkFXC+IfLKSIffhv0sVZup6pA=="],
"@aws-crypto/sha1-browser/@smithy/util-utf8/@smithy/util-buffer-from": ["@smithy/util-buffer-from@2.2.0", "", { "dependencies": { "@smithy/is-array-buffer": "^2.2.0", "tslib": "^2.6.2" } }, "sha512-IJdWBbTcMQ6DA0gdNhh/BwrLkDR+ADW5Kr1aZmd4k3DIF6ezMV4R2NIAmT08wQJ3yUK82thHWmC/TnK/wpMMIA=="],
"@aws-crypto/sha256-browser/@smithy/util-utf8/@smithy/util-buffer-from": ["@smithy/util-buffer-from@2.2.0", "", { "dependencies": { "@smithy/is-array-buffer": "^2.2.0", "tslib": "^2.6.2" } }, "sha512-IJdWBbTcMQ6DA0gdNhh/BwrLkDR+ADW5Kr1aZmd4k3DIF6ezMV4R2NIAmT08wQJ3yUK82thHWmC/TnK/wpMMIA=="],
"@aws-crypto/util/@smithy/util-utf8/@smithy/util-buffer-from": ["@smithy/util-buffer-from@2.2.0", "", { "dependencies": { "@smithy/is-array-buffer": "^2.2.0", "tslib": "^2.6.2" } }, "sha512-IJdWBbTcMQ6DA0gdNhh/BwrLkDR+ADW5Kr1aZmd4k3DIF6ezMV4R2NIAmT08wQJ3yUK82thHWmC/TnK/wpMMIA=="],
"form-data/mime-types/mime-db": ["mime-db@1.52.0", "", {}, "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg=="],
"@types/node-fetch/@types/node/undici-types": ["undici-types@7.18.2", "", {}, "sha512-AsuCzffGHJybSaRrmr5eHr81mwJU3kjw6M+uprWvCXiNeN9SOGwQ3Jn8jb8m3Z6izVgknn1R0FTCEAP2QrLY/w=="],
"openai/@types/node/undici-types": ["undici-types@5.26.5", "", {}, "sha512-JlCMO+ehdEIKqlFxk6IfVoAUVmgz7cU7zD/h9XZ0qzeosSHmUJVOzSQvvYSYWXkFXC+IfLKSIffhv0sVZup6pA=="],
"bun-types/@types/node/undici-types": ["undici-types@7.18.2", "", {}, "sha512-AsuCzffGHJybSaRrmr5eHr81mwJU3kjw6M+uprWvCXiNeN9SOGwQ3Jn8jb8m3Z6izVgknn1R0FTCEAP2QrLY/w=="],
"form-data/mime-types/mime-db": ["mime-db@1.52.0", "", {}, "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg=="],
"@aws-crypto/sha1-browser/@smithy/util-utf8/@smithy/util-buffer-from/@smithy/is-array-buffer": ["@smithy/is-array-buffer@2.2.0", "", { "dependencies": { "tslib": "^2.6.2" } }, "sha512-GGP3O9QFD24uGeAXYUjwSTXARoqpZykHadOmA8G5vfJPK0/DC67qa//0qvqrJzL1xc8WQWX7/yc7fwudjPHPhA=="],
+9 -18
View File
@@ -1,21 +1,12 @@
[test]
# PGLite WASM cold start + initSchema() runs ~520s on loaded machines.
# Default 5s is too short for those tests' beforeAll hooks. 60s is the
# empirical ceiling we observed for the slowest cold-init paths.
# PGLite initialization can be slow under parallel test execution.
# Default 5s is too short when many test files boot PGLite instances at once.
# 60s is the empirical ceiling we observed before the first file's beforeAll
# completed on a loaded machine.
#
# v0.26.4: scripts/run-unit-parallel.sh and scripts/run-unit-shard.sh
# also pass `--timeout=60000` explicitly so the ceiling is consistent
# whether tests are invoked through the wrapper or directly via bun test.
# NOTE: this bunfig.toml `timeout` key is read by `bun test` but empirically
# does NOT apply to beforeEach/afterEach hook timeouts under `bun run test`
# chained behind `bun run typecheck`. The test script in package.json passes
# `--timeout=60000` explicitly to cover both per-test and per-hook timeouts.
# Leaving both in place as belt-and-suspenders.
timeout = 60_000
# v0.37 fix wave: pin gateway defaults to legacy OpenAI/1536 BEFORE any
# test runs, so the 20+ test files with hardcoded 1536-d Float32Array
# fixtures still match the schema. v0.37's production default is ZE/1280;
# tests that want the new default call configureGateway() explicitly in
# their own beforeAll.
#
# #2823: redirect GBRAIN_AUDIT_DIR to a per-run scratch dir BEFORE any test
# runs, so audit-emitting code paths (content-sanity, shell-audit, etc.)
# can't leak fixture events into the operator's real ~/.gbrain/audit/. See
# test/helpers/audit-dir-preload.ts for the full rationale.
preload = ["./test/helpers/legacy-embedding-preload.ts", "./test/helpers/audit-dir-preload.ts"]
-153
View File
@@ -1,153 +0,0 @@
# docker-compose.ci.yml
#
# Local CI gate with 4-way E2E sharding. Spins up 4 pgvector services + a bun
# runner that bind-mounts the repo. Used by `bun run ci:local` and
# `bun run ci:local:diff` (see scripts/ci-local.sh).
#
# All services are pulled as `image:` (no build) so `docker compose pull`
# refreshes everything. The bun version floats with `oven/bun:1` to track CI's
# `bun-version: latest`. Named volumes isolate the Linux container's deps from
# the host's darwin-arm64 deps and keep bun + postgres data warm across runs.
#
# Why 4 postgres services: bun's E2E suite shares one DB across 36 files and
# uses TRUNCATE CASCADE in setupDB(). Running files in parallel against ONE DB
# races (file A's TRUNCATE clobbers file B's fixture import). 4 separate DBs
# remove the race; we shard the file list 1/4..4/4 and run shards in parallel.
# Within a shard, files still run sequentially. Total wall-time on a 16-core
# host: ~6 min sequential -> ~1.5-2 min sharded.
#
# Postgres host ports default to 5434-5437 (avoid 5432 manual `gbrain-test-pg`
# and 5433 sibling-project conflicts). Override BASE port with GBRAIN_CI_PG_PORT;
# shards take BASE..BASE+3.
services:
postgres-1:
image: pgvector/pgvector:pg16
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: gbrain_test
ports:
- "${GBRAIN_CI_PG_PORT:-5434}:5432"
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres -d gbrain_test"]
interval: 10s
timeout: 5s
retries: 5
volumes:
- gbrain-ci-pg-data-1:/var/lib/postgresql/data
postgres-2:
image: pgvector/pgvector:pg16
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: gbrain_test
ports:
- "${GBRAIN_CI_PG_PORT_2:-5435}:5432"
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres -d gbrain_test"]
interval: 10s
timeout: 5s
retries: 5
volumes:
- gbrain-ci-pg-data-2:/var/lib/postgresql/data
postgres-3:
image: pgvector/pgvector:pg16
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: gbrain_test
ports:
- "${GBRAIN_CI_PG_PORT_3:-5436}:5432"
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres -d gbrain_test"]
interval: 10s
timeout: 5s
retries: 5
volumes:
- gbrain-ci-pg-data-3:/var/lib/postgresql/data
postgres-4:
image: pgvector/pgvector:pg16
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: gbrain_test
ports:
- "${GBRAIN_CI_PG_PORT_4:-5437}:5432"
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres -d gbrain_test"]
interval: 10s
timeout: 5s
retries: 5
volumes:
- gbrain-ci-pg-data-4:/var/lib/postgresql/data
# v0.43 (#2084 / eng-review TD1): PgBouncer in TRANSACTION pooling mode
# fronting postgres-1 — the production topology (Supabase direct :5432 +
# pooled :6543) behind three consecutive pooler-teardown waves
# (#1972 → #2015 → #2084) that CI could never reproduce.
# test/e2e/pgbouncer-teardown.test.ts uses a DEDICATED database
# (gbrain_pgbouncer) on postgres-1 so it never races shard 1's
# TRUNCATE-based fixtures; pgbouncer's wildcard [databases] section
# forwards any dbname to DB_HOST.
pgbouncer:
image: edoburu/pgbouncer:latest
environment:
DB_HOST: postgres-1
DB_PORT: "5432"
DB_USER: postgres
DB_PASSWORD: postgres
POOL_MODE: transaction
# plain (CI-only): pg16 stores SCRAM verifiers, and pgbouncer can only
# answer the server's SCRAM challenge when its userlist holds the
# PLAINTEXT password — an md5-hashed userlist fails with
# "server login failed: wrong password type".
AUTH_TYPE: plain
MAX_CLIENT_CONN: "200"
DEFAULT_POOL_SIZE: "10"
# gbrain's client sets statement_timeout + idle_in_transaction_session_timeout
# as startup parameters (db.ts buildConnectionParams); the Supabase pooler
# whitelists them, so this pooler must too or every connection is refused
# before the teardown path is even reached.
IGNORE_STARTUP_PARAMETERS: extra_float_digits,statement_timeout,idle_in_transaction_session_timeout,search_path
ports:
- "${GBRAIN_CI_PGBOUNCER_PORT:-6543}:5432"
depends_on:
postgres-1:
condition: service_healthy
runner:
image: oven/bun:1
working_dir: /app
depends_on:
postgres-1:
condition: service_healthy
postgres-2:
condition: service_healthy
postgres-3:
condition: service_healthy
postgres-4:
condition: service_healthy
pgbouncer:
condition: service_started
# No global DATABASE_URL — scripts/ci-local.sh sets per-shard URL via -e.
# Unit phase explicitly unsets DATABASE_URL so test/e2e/* gracefully skip.
volumes:
- .:/app
# Linux container's node_modules MUST be isolated from host darwin-arm64.
# Without this, container `bun install` stomps host node_modules and
# subsequent `bun test` on host fails with binary-incompat errors.
- gbrain-ci-node-modules:/app/node_modules
# Warm install cache across runs.
- gbrain-ci-bun-cache:/root/.bun/install/cache
volumes:
gbrain-ci-pg-data-1:
gbrain-ci-pg-data-2:
gbrain-ci-pg-data-3:
gbrain-ci-pg-data-4:
gbrain-ci-node-modules:
gbrain-ci-bun-cache:
+1 -79
View File
@@ -94,7 +94,7 @@ export interface BrainEngine {
**Slug-based API, not ID-based.** Every method takes slugs, not numeric IDs. The engine resolves slugs to IDs internally. This keeps the interface portable... slugs are strings, IDs are database-specific.
**Embedding is NOT in the engine.** The engine stores embeddings and searches by vector, but it doesn't generate embeddings. `src/core/embedding.ts` handles that (a thin delegation to the provider-agnostic AI gateway in `src/core/ai/gateway.ts`). This is intentional: embedding is an external API call (OpenAI, Voyage, a local Ollama — whichever provider you configured), not a storage concern. All engines share the same embedding service.
**Embedding is NOT in the engine.** The engine stores embeddings and searches by vector, but it doesn't generate embeddings. `src/core/embedding.ts` handles that. This is intentional: embedding is an external API call (OpenAI), not a storage concern. All engines share the same embedding service.
**Chunking is NOT in the engine.** Same logic. `src/core/chunkers/` handles chunking. The engine stores and retrieves chunks. All engines share the same chunkers.
@@ -148,51 +148,6 @@ RRF fusion, multi-query expansion, and 4-layer dedup are engine-agnostic. They o
**Why not self-hosted for v0:** The brain should be infrastructure agents use, not something you maintain. Self-hosted Postgres with Docker is a welcome community PR, but v0 optimizes for zero ops.
### Opt-in RLS source-scope binding (`GBRAIN_RLS_SCOPE_BINDING`)
Defense-in-depth layer for Postgres deployments that want the database itself
to enforce source isolation, in addition to the mandatory app-layer filters
(`sourceScopeOpts` — layer 1, always on).
**Mechanism.** With `GBRAIN_RLS_SCOPE_BINDING=1` (or `true`), the engine's
source-scoped read methods wrap their queries in a transaction that first runs
`SELECT set_config('app.scopes', $1, true)` — the value is a bound parameter
(federated `sourceIds` CSV > scalar `sourceId` > `'*'` for unscoped internal
reads), transaction-local (equivalent to `SET LOCAL`, which itself can't take
bound params). An RLS policy can then filter rows by
`current_setting('app.scopes', true)`.
**Default off.** With the env var unset, reads call through on the shared pool
exactly as before — no per-read transaction, no pool-slot hold (the search
methods keep the transaction they always had for their `SET LOCAL
statement_timeout`). Existing operators see zero behavior change.
**Enabling it** (operator-managed SQL; gbrain ships no DDL for this):
```sql
ALTER TABLE pages ENABLE ROW LEVEL SECURITY;
CREATE POLICY pages_scope_filter ON pages
USING (current_setting('app.scopes', true) = '*'
OR source_id = ANY(string_to_array(current_setting('app.scopes', true), ',')));
-- Required: connections that don't run through the scoped read helper
-- (admin, autopilot, cycle, writes) must default to unscoped, or they
-- see zero rows once the policy exists:
ALTER ROLE <runtime-role> SET app.scopes = '*';
-- If the runtime role OWNS the table, RLS is skipped for it unless forced:
ALTER TABLE pages FORCE ROW LEVEL SECURITY;
```
Safe to enable in either order: the env var without a policy is a no-op
setting; a policy without the env var is enforced only via the role default.
**Honest caveat:** only read paths routed through the scoped helper carry a
per-request scope binding — unwrapped paths (writes, admin/maintenance reads)
run under the role default and are not backstopped per caller. This is layer 2;
the app-layer source filters remain layer 1 and stay mandatory. Behavioral pins
live in `test/postgres-engine-rls-scope.test.ts`.
## PGLiteEngine (v0.7, ships)
**Dependencies:** `@electric-sql/pglite` (v0.4.4+)
@@ -221,39 +176,6 @@ live in `test/postgres-engine-rls-scope.test.ts`.
**Migration:** `gbrain migrate --to supabase` exports everything (pages, chunks, embeddings, links, tags, timeline) and imports into Supabase. `gbrain migrate --to pglite` goes the other direction. Bidirectional, lossless.
## JSONB writes: never double-encode (the #2339 trap)
Writing a JS value into a `jsonb` column has exactly two correct forms. Get this
wrong and the write succeeds on PGLite but stores a **jsonb string scalar** on
real Postgres — `col ->> 'k'` returns NULL, `jsonb_array_elements` throws, and a
`jsonb_typeof = 'array'` CHECK rejects the row (this aborted every sync in #2339).
| Form | Verdict |
|---|---|
| Template tag: `` sql`... ${sql.json(obj)}` `` (postgres-engine only) | ✅ native jsonb serialization |
| Positional raw call, raw object: `executeRawJsonb(engine, sql, scalars, [obj])` | ✅ object reaches the wire as jsonb |
| Positional raw call, stringified: `executeRaw(\`... $N::text::jsonb\`, [JSON.stringify(x)])` | ✅ binds as text, the cast parses it |
| Positional raw call, BARE cast: `executeRaw(\`... $N::jsonb\`, [JSON.stringify(x)])` | ❌ **double-encodes** under postgres.js `.unsafe()` |
| Template literal interpolation: `` `... ${JSON.stringify(x)}::jsonb` `` | ❌ double-encodes |
**Why:** postgres.js `.unsafe(sql, params)` (the path behind `executeRaw` /
`executeRawDirect`) binds a JS **string** as a text param. A bare `$N::jsonb`
cast then wraps that already-JSON string into a jsonb scalar string instead of
parsing it. Casting through `$N::text::jsonb` forces a text→jsonb parse.
**PGLite's `db.query` parses text→jsonb natively, so it hides the bug** — which is
why a regression only shows up on Postgres (and why the parity test must run there).
**Two CI guards enforce this, both wired into `scripts/check-jsonb-pattern.sh`:**
- the template-tag grep (`${JSON.stringify(x)}::jsonb`), and
- `scripts/check-jsonb-params.mjs`, an AST-lite scanner for the positional
`$N::jsonb` + `JSON.stringify` form the grep misses. Sanctioned escapes:
`$N::text::jsonb`, `$N::text[]`, `executeRawJsonb`, `sql.json`, or an inline
`jsonb-guard-ok` comment.
The real backstop is `test/e2e/op-checkpoint-jsonb-parity.test.ts` +
`test/e2e/jsonb-roundtrip.test.ts`, which round-trip writes through real Postgres
and assert `jsonb_typeof` — the assertion PGLite cannot make.
## Adding a new engine
1. Create `src/core/<name>-engine.ts` implementing `BrainEngine`
-10
View File
@@ -102,16 +102,6 @@ Keeping it running and up to date.
| [Upgrades & Auto-Update](guides/upgrades-auto-update.md) | check-update, agent notifications, migration files |
| [Live Sync](guides/live-sync.md) | Keep the index current: cron, --watch, webhook approaches |
## Getting Started
After setup, the brain is empty. The cold-start skill sequences the highest-leverage
data sources to populate it:
| Guide | What It Covers |
|-------|---------------|
| [Cold Start](../skills/cold-start/SKILL.md) | Day-one bootstrapping: contacts, calendar, email, conversations, social, archives. Uses ClawVisor for safe credential handling — agents never hold raw API keys. |
| [Ask User](../skills/ask-user/SKILL.md) | Choice-gate pattern for human input at decision points. Used by cold-start and other skills. |
---
## Appendix: GBrain CLI Quick Reference
+2 -9
View File
@@ -1,12 +1,5 @@
# GBrain v0: Postgres-Native Personal Knowledge Brain
> **Historical design doc.** This is the original v0 spec from before PGLite landed. Several
> forward-looking sections — most notably the SQLite engine plan — were superseded by
> PGLite (embedded Postgres via WASM), which uses the same SQL dialect as Postgres and
> eliminates the need for a separate FTS5/sqlite-vss translation layer. Kept here for
> historical context; see [`ENGINES.md`](ENGINES.md) for the current engine architecture and
> the [`CHANGELOG.md`](../CHANGELOG.md) for the actual implementation history.
## What this is
GBrain is a compiled intelligence system. Not a note-taking app. Not "chat with your notes."
@@ -524,7 +517,7 @@ See `docs/ENGINES.md` for the pluggable engine architecture and future backend p
- **Intelligence compiler.** Treat every fact as a first-class claim with source span, entity links, validity window, confidence, and contradiction status. "What changed, why, and what evidence would flip it again?" From Codex review. Builds on compiled truth model.
- **Active skills via Trigger.dev.** Application-specific briefings, meeting prep. Belongs in OpenClaw, not generic brain infra.
- **Multi-user access.** Supabase RLS + per-user API keys. v0 is single-user.
- **SQLite engine.** Superseded by PGLite (embedded Postgres 17 via WASM) before v1. See [`ENGINES.md`](ENGINES.md) for the current engine architecture.
- **SQLite engine.** Community PRs welcome. See `docs/SQLITE_ENGINE.md`.
- **Docker Compose for self-hosted Postgres.** Community PRs welcome.
- **Web UI.** Optional Vercel-hosted dashboard for browsing brain pages.
@@ -538,7 +531,7 @@ This means:
- A future DuckDB engine could implement analytics-heavy workloads
- The CLI, MCP server, and library consumers never know which engine runs underneath
See [`ENGINES.md`](ENGINES.md) for the full interface spec. (The original SQLite engine plan was superseded by PGLite; the contract-first `BrainEngine` interface made that swap clean.)
See `docs/ENGINES.md` for the full interface spec and `docs/SQLITE_ENGINE.md` for the SQLite implementation plan.
## Review history
+9 -11
View File
@@ -88,15 +88,14 @@ find /data/brain -name '*.md' \
Some difference is normal (files added since last sync), but if page count is
less than half the file count, sync is silently skipping pages.
**If page count is way too low:** The #1 cause is an unreachable direct
connection on an IPv4-only host. GBrain uses the Transaction pooler (port 6543)
for reads, but routes migrations, DDL, and sync transactions to a derived direct
connection (`db.<ref>.supabase.co:5432`), which is IPv6-only.
- On an IPv4-only host, reads work but sync transactions fail and silently skip
pages.
- Fix: set `GBRAIN_DIRECT_DATABASE_URL` to the **Session pooler** string (port
5432 on the `pooler.supabase.com` host, IPv4), or enable Supabase's IPv4
add-on. Then run `gbrain sync --full` to reimport everything.
**If page count is way too low:** The #1 cause is the connection pooler bug.
Check your `DATABASE_URL`:
- If it contains `pooler.supabase.com:6543`, verify it's using **Session mode**,
not Transaction mode.
- Transaction mode breaks `engine.transaction()` and causes `.begin() is not a
function` errors.
- Fix: switch to Session mode pooler string, then run `gbrain sync --full`
to reimport everything.
### 4b. Embed Check
@@ -143,8 +142,7 @@ gbrain search "<text from the correction>"
- Is `gbrain sync --watch` still alive (if using watch mode)?
- Run `gbrain config get sync.last_run` to see when sync last ran.
- Run `gbrain sync --repo /data/brain` manually and check for errors.
- If sync errors mention an unreachable host or connection timeout, the direct
connection isn't reachable on IPv4 (see 4a above).
- If you see `.begin() is not a function`, fix the pooler (see 4a above).
---
-149
View File
@@ -1,149 +0,0 @@
# Install
Three install paths. Pick one. Mix later if needed.
## 1. Run with an agent platform (recommended)
Already running [OpenClaw](https://github.com/garrytan/openclaw) or [Hermes](https://github.com/garrytan/hermes)?
```bash
bun install -g github:garrytan/gbrain
gbrain init --pglite # 2 seconds; no server
gbrain skillpack scaffold --all # 43 skills scaffolded into your agent workspace
gbrain doctor # green checks all the way down
```
Your agent now reads `skills/RESOLVER.md` once per request, routes intent to the right skill, executes. New entity mentions create new pages. Daily cron runs enrichment overnight.
Scaffolded skills are first-class files in your agent repo — edit freely. To pull upstream gbrain improvements later, `gbrain skillpack reference <name>` diffs your local copy vs the bundle. The legacy `skillpack install` managed-block model was retired in v0.36.0.0; if you're upgrading from an older release, run `gbrain skillpack migrate-fence` once to strip the legacy fence and keep your existing skill rows.
To upgrade later: `gbrain upgrade` runs schema migrations + post-upgrade prompts (chunker bumps, the v0.36.2.0 ZeroEntropy switch). Always TTY-only; non-TTY upgrades skip prompts with informational stderr lines.
## 2. CLI standalone
No agent platform, just shell + MCP-aware editor.
```bash
bun install -g github:garrytan/gbrain
gbrain init --pglite
```
> **If `bun install -g` hits a postinstall error** (Bun blocks postinstall hooks in some environments), the CLI prints a recovery hint pointing at [#218](https://github.com/garrytan/gbrain/issues/218). Run `gbrain doctor` to diagnose, then `gbrain apply-migrations --yes` manually. The deterministic fallback is `git clone https://github.com/garrytan/gbrain.git ~/gbrain && cd ~/gbrain && bun install && bun link`.
The init flow detects your repo size and suggests Supabase for brains > 1000 markdown files. To switch later:
```bash
gbrain migrate --to supabase # PGLite → Postgres
gbrain migrate --to pglite # Postgres → PGLite (rare)
```
For shared / large / multi-machine deployments (a team or company brain with multiple users hitting one server over HTTP MCP with OAuth scoping per user), follow the dedicated walkthrough: **[Tutorial: set up GBrain as your company brain](tutorials/company-brain.md)**.
API keys live in `~/.gbrain/config.json` (file plane) or env vars (`OPENAI_API_KEY`, `OPENROUTER_API_KEY`, `ZEROENTROPY_API_KEY`, `VOYAGE_API_KEY`, `ANTHROPIC_API_KEY`). Set via CLI:
```bash
gbrain config set zeroentropy_api_key sk-...
gbrain config set openrouter_api_key sk-or-...
gbrain config set anthropic_api_key sk-ant-...
```
Common follow-ups:
```bash
gbrain import ~/my-knowledge # bulk-import a markdown folder
gbrain sync --watch # live-sync a git repo (autopilot mode)
gbrain autopilot --install # background daemon for nightly enrichment
```
**Wire this same local brain into your coding agent** — zero server, zero token:
```bash
claude mcp add gbrain -- gbrain serve # Claude Code
codex mcp add gbrain -- gbrain serve # Codex
```
The agent spawns `gbrain serve` as a stdio subprocess against your local brain. Full walkthrough (both this local path and connecting to a remote brain), plus the brain-first protocol to paste into `CLAUDE.md` / `AGENTS.md`: **[Give your coding agent a memory](tutorials/connect-coding-agent.md)**.
## 3. MCP server (any MCP client)
```bash
gbrain serve # stdio MCP (Claude Desktop / Code / Cursor)
gbrain serve --http # HTTP MCP with OAuth 2.1 + admin dashboard
```
**Wire a coding agent to a remote brain in one command** (when you have an HTTP
server + a bearer token): `gbrain connect` prints a paste-ready setup block, or
`--install` runs it and smoke-tests the token.
```bash
gbrain auth create "claude-code"
gbrain connect https://your-host/mcp --token gbrain_xxx # Claude Code (default)
gbrain connect https://your-host/mcp --token gbrain_xxx --agent codex # Codex (env-var bearer)
gbrain connect https://your-host/mcp --agent perplexity --oauth --register # Perplexity (OAuth)
```
Per-client setup guides live in [`docs/mcp/`](mcp/):
- [`docs/mcp/CLAUDE_CODE.md`](mcp/CLAUDE_CODE.md)
- [`docs/mcp/CODEX.md`](mcp/CODEX.md)
- [`docs/mcp/CLAUDE_DESKTOP.md`](mcp/CLAUDE_DESKTOP.md)
- [`docs/mcp/CHATGPT.md`](mcp/CHATGPT.md)
- [`docs/mcp/PERPLEXITY.md`](mcp/PERPLEXITY.md)
- [`docs/mcp/DEPLOY.md`](mcp/DEPLOY.md) — production deploy patterns
The HTTP server ships with an admin SPA at `/admin`, an SSE activity feed at `/admin/events`, DCR-style client registration, scope-gated `read`/`write`/`admin` access, and rate limiting.
## Thin-client mode
Connect to someone else's brain without running a local engine:
```bash
gbrain init --mcp-only # configures remote MCP, skips local DB
```
Useful for: team mounts, brain-as-a-service deployments, dev machines without disk space. Most local commands refuse with a paste-ready hint. See [`docs/architecture/topologies.md`](architecture/topologies.md).
## Verifying the install
```bash
gbrain doctor --json # full health check
gbrain models # which AI models are configured for what
gbrain models doctor # 1-token probe per configured model
```
If anything's yellow, `gbrain doctor` names the fix command in the message. Most issues are missing API keys or stale schema (`gbrain upgrade --force-schema`).
## Troubleshooting
### PGLite crashes on macOS 26.x (Tahoe)
PGLite's embedded WASM engine is incompatible with macOS 26.x (Tahoe) on Apple Silicon. If `gbrain init --pglite` crashes during engine initialization, switch to native Homebrew PostgreSQL:
```bash
# Install PostgreSQL + pgvector
brew install postgresql@17
brew services start postgresql@17
createdb gbrain
# Build pgvector from source (required for vector search)
cd /tmp && git clone --branch v0.8.0 https://github.com/pgvector/pgvector.git
cd pgvector && make && make install
psql gbrain -c "CREATE EXTENSION IF NOT EXISTS vector;"
# Point gbrain at your local Postgres
cat > ~/.gbrain/config.json << 'EOF'
{
"engine": "postgres",
"database_url": "postgresql://localhost:5432/gbrain",
"schema_pack": "gbrain-base-v2"
}
EOF
# Run migrations and verify
gbrain apply-migrations --yes
gbrain doctor
```
All 102 migrations run on first try. Once `gbrain doctor` shows green, the brain works identically to PGLite — same commands, same skills, same data model. The only difference is the storage backend.
> **Note:** This workaround is temporary. When the upstream WASM runtime fix ships (likely via a Bun update), `--pglite` will work on Tahoe again.
-435
View File
@@ -1,435 +0,0 @@
# Releasing & contributing (gbrain)
The full release + contributor process. CLAUDE.md keeps the ship-critical IRON RULES
inline (the Version-locations table, branch=workspace, post-ship `/document-release`,
the Privacy + Responsible-disclosure rules, PR-title-version-first, never-hand-roll-ship)
and points here for everything else. **Before any ship, read this in full. Use `/ship`
never hand-roll a release.**
## Pre-ship requirements
Before shipping (/ship) or reviewing (/review), always run the full test suite.
Two equivalent paths:
**Path A — local CI gate (recommended, v0.23.1+):**
- `bun run ci:local` runs the entire stack inside Docker: gitleaks (host),
guards + typecheck, then 4-shard parallel unit + E2E against four pgvector
containers plus a transaction-mode PgBouncer service (unit phase keeps
`DATABASE_URL` unset; `--no-shard` for the legacy sequential flow). Stronger
than PR CI's 2-file Tier 1 set; closer to what nightly Tier 1 catches. Spins
up + tears down postgres automatically via `docker-compose.ci.yml`. Override
the host port with `GBRAIN_CI_PG_PORT=5435 bun run ci:local` if 5434 collides.
- `bun run ci:local:diff` runs only the E2E files matched by the diff selector
(`scripts/select-e2e.ts`), falling back to ALL E2E files on unmapped src/
paths or schema/skills/package.json changes. Fast iteration during a focused
branch.
**Path B — manual lifecycle (still supported):**
- `bun test` — unit tests (no database required)
- Follow the "E2E test DB lifecycle" steps above to spin up the test DB,
run `bun run test:e2e`, then tear it down.
Both must pass. Do not ship with failing E2E tests. Do not skip E2E tests.
**Always run typecheck before pushing.** `bun test` (the bun runner)
skips TypeScript type checking — it only enforces runtime behavior.
Three ways to actually gate on types:
1. `bun run test` (npm script in `package.json`) — includes `bun run typecheck`
plus the four shell pre-checks (`check-jsonb-pattern.sh`,
`check-progress-to-stdout.sh`, `check-trailing-newline.sh`,
`check-wasm-embedded.sh`) before the runner. Use this mid-branch.
2. `bun run typecheck``tsc --noEmit` standalone. Fast (~5s on this repo).
3. `bun run ci:local` — the full local CI gate from Path A.
The trap is: writing a new test, running `bun test test/foo.test.ts`,
seeing it pass, pushing — and CI's separate typecheck stage rejects an
invalid type literal that the runner accepted. Caught one of these
shipping the v0.23.2 round-trip E2E (`type: 'reflection'` is not a
member of `PageType`). Run `bun run typecheck` once before push, even
when only test files changed.
## CHANGELOG + VERSION are branch-scoped
**VERSION and CHANGELOG describe what THIS branch adds vs master, not how we got
here.** Every feature branch that ships gets its own version bump and CHANGELOG
entry. The entry is product release notes for users; it is not a log of internal
decisions, review rounds, or codex findings.
**Write the CHANGELOG entry at /ship time, not during development.** Mid-branch
iterations, review rounds (CEO/Eng/Codex/DX), and implementation detours belong
in the plan file at `~/.claude/plans/`, not in the CHANGELOG. One unified entry
per branch, covering what the branch added vs the base branch.
**Never edit a CHANGELOG entry that already landed on master.** If master has
v0.18.2 and your branch adds features, bump to the next version (v0.19.0, not
editing master's v0.18.2). When merging master into your branch, master may
bring new CHANGELOG entries above yours — push your entry above master's
latest and verify:
- Does CHANGELOG have your branch's own entry separate from master's entries?
- Is VERSION higher than master's VERSION?
- Is your entry the topmost `## [X.Y.Z]` entry?
- `grep "^## \[" CHANGELOG.md` shows a contiguous version sequence?
If any answer is no, fix it before continuing.
**CHANGELOG is for users, not contributors.** Write like product release notes:
- Lead with what the user can now **do** that they couldn't before. Sell the capability.
- Plain language, not implementation details. "You can now..." not "Refactored the..."
- **Never mention internal artifacts**: plan file IDs, decision tags (D-CX-#, F-ENG-#),
review rounds, codex findings, subcontractor credits. These are invisible to users.
- Put contributor-facing changes in a separate `### For contributors` section at the bottom.
- Every entry should make someone think "oh nice, I want to try that."
**What to omit:**
- "Codex caught X that the CEO review missed" — private process detail.
- "D-CX-3 split errors/warnings" — tag is meaningless to users; name the feature instead.
- "Fix-wave PR #N supersedes #M" — supersede chains belong in PR bodies, not release notes.
- "215 new cases, 3 decisions applied, 7 reviews cleared" — these are planning-mode metrics.
**What to keep:**
- The user-facing change: what commands exist now, what flag was added, what behavior fixed.
- Numbers that mean something to the user: TTHW, commands that timed out before, detection counts.
- Upgrade instructions: `gbrain upgrade` + any manual step if needed.
- Credit to external contributors when a community PR was incorporated.
## CHANGELOG voice + release-summary format
**IRON RULE: the CHANGELOG describes what the user gets, not how the work
happened.** Nobody reading release notes cares that codex caught a bug, that
the plan went through CEO + eng review, that the migration was originally
numbered v68 and renumbered to v79 during master merge, or that two
review rounds caught architectural mistakes. The reader cares what
`gbrain brainstorm` does and how to use it. If a fact only exists because
of the development process, it does NOT belong in the CHANGELOG.
**Specifically forbidden in CHANGELOG entries:**
- Any mention of review processes (CEO review, eng review, codex review,
plan-eng-review, outside voice, adversarial review, autoplan, /review).
- "What we caught and fixed before merging" sections. Bugs found pre-merge
are not changes — they're things that didn't ship.
- Plan file references, plan IDs, plan decision tags (D1, D14, D-CDX-3).
- Migration version drama ("originally v68", "renumbered to v77", "claimed
by parallel waves") — just say "Migration v79 adds X." If the user
cares about migration ordering, they read the diff.
- Round counts, finding counts, decision counts ("25 findings across 2
rounds", "8 architectural decisions", "5/6 expansions accepted").
- Names of internal collaborators ("codex caught", "the reviewer flagged",
"Claude noticed").
- "Plan + reviews" summary bullets. The plan lives in `~/.claude/plans/`;
if a future reader wants the backstory they can grep there.
- Any wording that frames a shipped feature as a *recovery* from a planning
mistake ("the first plan was wrong", "we corrected the approach", "the
shipped version supersedes the original design").
**Smell test:** read the entry as a stranger who has never touched gbrain.
If any sentence makes them think "why are you telling me this?", cut it.
Every sentence in the release-summary AND in the itemized changes must
answer one of three questions: *What can I now do? How do I use it? What
should I watch for after I upgrade?*
Every version entry in `CHANGELOG.md` MUST start with a release-summary section in
the GStack/Garry voice — one viewport's worth of prose + tables that lands like a
verdict, not marketing. The itemized changelog (subsections, bullets, files) goes
BELOW that summary, separated by a `### Itemized changes` header.
The release-summary section gets read by humans, by the auto-update agent, and by
anyone deciding whether to upgrade. The itemized list is for agents that need to
know exactly what changed.
### Release-summary template
**Iron rule: lead ELI10, get precise after.** The first ~150 words of every entry
must be readable by someone who does NOT know gbrain's internals. No file paths,
no function names, no internal constants, no acronyms (no "RRF", no "knobsHash",
no "MODE_BUNDLES", no "CDX-4"), no jargon that requires reading the codebase to
parse. Lead with the user-visible behavior change, in everyday English, like
you're explaining it to a smart engineer who has never opened the repo.
THEN, once the reader knows what shipped and why they'd care, drill into the
precise details: real file paths, real function names, real config keys, real
numbers. The precision part is required (the entry is also the technical record
of what changed), but it lives AFTER the plain-English lead, never before it.
The shape:
1. **One-line bold headline.** What changed for the user, in human English. No
jargon. No internal terms. Example good: "Your search stops boosting weak
pages just because they have a lot of links pointing at them." Example bad:
"PostFusionOpts gains floorRatio; KNOBS_HASH_VERSION bumped 2→3."
2. **Plain-English opener** (~3-5 sentences). Describe the problem this fixes in
everyday terms. Pretend the reader has a brain full of meeting notes and
people pages and wants to know if this release helps them. Concrete example
beats abstract description.
3. **A "How to turn it on" or "How to use it" section** with paste-ready
commands. Real flags, real config keys. This is where precision starts.
4. **A "What you'd see in a concrete example" or "The X numbers that matter"
section** with a table. Use everyday-language column headers ("Page",
"Match quality", "Has many backlinks?") even when the underlying mechanism
is technical. The table teaches what the feature does without requiring the
reader to understand how.
5. **A "What's safe to know about" or "Things to watch" section** for caveats,
side effects, cache invalidation, mid-deploy notes. Still in plain language.
6. **A "What we caught and fixed before merging" section** if the work went
through review (CEO/eng/codex/outside-voice). Translate review findings into
plain English. "We caught a stale-cache bug" beats "knobsHash() did not
include floorRatio in the v=2 hash input."
7. **`### Itemized changes`** (precision lives here). File paths, function
names, types, constants, line numbers. This section is for engineers who
need to know exactly what moved.
Voice rules (apply throughout):
- No em dashes (use commas, periods, "...").
- No AI vocabulary (delve, robust, comprehensive, nuanced, fundamental, etc.) or
banned phrases ("here's the kicker", "the bottom line", etc.).
- Real numbers, real file names, real commands AFTER the ELI10 lead. Not "fast"
but "~30s on 30K pages." In the ELI10 lead, "fast enough that you won't
notice" or "~30 seconds even on a big brain."
- Short paragraphs, mix one-sentence punches with 2-3 sentence runs.
- Connect to user outcomes: "the agent does ~3x less reading" beats "improved
precision."
- Be direct about quality. "Well-designed" or "this is a mess." No dancing.
**The smell test:** if someone who has never opened gbrain reads the first 150
words and walks away knowing what shipped and whether they care, the entry
passes. If they need to grep the codebase to follow along, rewrite the lead.
**Canonical examples in this CHANGELOG:** v0.35.6.0 (floor-ratio gate, written
ELI10-lead-first), v0.34.4.0 (embed stale fix wave). Use those shapes when in
doubt. Avoid the shape of entries that lead with internal constants or release
mechanics; those exist in older history but should not be the model for new
work.
Source material to pull from:
- CHANGELOG.md previous entry for prior context
- Latest `gbrain-evals/docs/benchmarks/[latest].md` for headline numbers (sibling repo)
- Recent commits (`git log <prev-version>..HEAD --oneline`) for what shipped
- Don't make up numbers. If a metric isn't in a benchmark or production data, don't
include it. Say "no measurement yet" if asked.
Target length: ~250-350 words for the summary. Should render as one viewport.
### "To take advantage of v[version]" block (required, v0.13+)
After the release-summary and BEFORE `### Itemized changes`, every `## [X.Y.Z]`
entry MUST include a human-readable self-repair block under the heading
`## To take advantage of v[version]`.
Why: `gbrain upgrade` runs `gbrain post-upgrade` which runs `gbrain apply-migrations`.
This chain has a known weak link — `upgrade.ts` catches post-upgrade failures as
best-effort (so the binary still works). When that chain silently fails, users end
up with half-upgraded brains. The self-repair block gives them a paste-ready
recovery path; the v0.13+ `~/.gbrain/upgrade-errors.jsonl` trail + `gbrain doctor`
integration close the loop.
Template (adapt the verify commands per release):
```markdown
## To take advantage of v[version]
`gbrain upgrade` should do this automatically. If it didn't, or if `gbrain doctor`
warns about a partial migration:
1. **Run the orchestrator manually:**
```bash
gbrain apply-migrations --yes
```
2. **Your agent reads `skills/migrations/v[version].md` the next time you interact with it.**
[One sentence on whether headless agents need manual action, or whether the
orchestrator already handled the mechanical side.]
3. **Verify the outcome:**
```bash
[release-specific verify commands, e.g. `gbrain graph ... --depth 2`]
gbrain stats
```
4. **If any step fails or the numbers look wrong,** please file an issue:
https://github.com/garrytan/gbrain/issues with:
- output of `gbrain doctor`
- contents of `~/.gbrain/upgrade-errors.jsonl` if it exists
- which step broke
This feedback loop is how the gbrain maintainers find fragile upgrade paths. Thank you.
```
**Skip this block** for patches that are pure bug fixes with zero user-facing action
(rare). If the release has a schema migration, data backfill, or new feature the
user needs to verify, the block is required.
The v0.13.0 entry in CHANGELOG.md is the canonical example.
### Itemized changes (the existing rules)
Below the release summary, write `### Itemized changes` and continue with the
detailed subsections (Knowledge Graph Layer, Schema migrations, Security hardening,
Tests, etc.). Same rules as before:
- Lead with what the user can now DO that they couldn't before
- Frame as benefits and capabilities, not files changed or code written
- Make the user think "hell yeah, I want that"
- Bad: "Added GBRAIN_VERIFY.md installation verification runbook"
- Good: "Your agent now verifies the entire GBrain installation end-to-end, catching
silent sync failures and stale embeddings before they bite you"
- Bad: "Setup skill Phase H and Phase I added"
- Good: "New installs automatically set up live sync so your brain never falls behind"
- **Always credit community contributions.** When a CHANGELOG entry includes work from
a community PR, name the contributor with `Contributed by @username`. Contributors
did real work. Thank them publicly every time, no exceptions.
### Reference: v0.12.0 entry as canonical example
The v0.12.0 entry in CHANGELOG.md is the canonical example of the format. Match its
structure for every future version: bold headline, lead paragraph, "numbers that
matter" with BrainBench-style before/after table, "what this means" closer, then
`### Itemized changes` with the detailed sections below.
## Version migrations
Create a migration file at `skills/migrations/v[version].md` when a release
includes changes that existing users need to act on. The auto-update agent
reads these files post-upgrade (Section 17, Step 4) and executes them.
**You need a migration file when:**
- New setup step that existing installs don't have (e.g., v0.5.0 added live sync,
existing users need to set it up, not just new installs)
- New SKILLPACK section with a MUST ADD setup requirement
- Schema changes that require `gbrain init` or manual SQL
- Changed defaults that affect existing behavior
- Deprecated commands or flags that need replacement
- New verification steps that should run on existing installs
- New cron jobs or background processes that should be registered
**You do NOT need a migration file when:**
- Bug fixes with no behavior changes
- Documentation-only improvements (the agent re-reads docs automatically)
- New optional features that don't affect existing setups
- Performance improvements that are transparent
**The key test:** if an existing user upgrades and does nothing else, will their
brain work worse than before? If yes, migration file. If no, skip it.
Write migration files as agent instructions, not technical notes. Tell the agent
what to do, step by step, with exact commands. See `skills/migrations/v0.5.0.md`
for the pattern.
## Migration is canonical, not advisory
GBrain's job is to deliver a canonical, working setup to every user on upgrade.
Anything that looks like a "host-repo change" — AGENTS.md, cron manifests,
launchctl units, config files outside `~/.gbrain/` — is a GBrain migration
step, not a nudge we leave for the host-repo maintainer. Migrations edit host
files (with backups) to make the canonical setup real. Exceptions: changes
that require human judgment (content edits, renames that break semantics,
host-specific handler registration where shell-exec would be an RCE surface).
Everything mechanical ships in the migration.
**Test:** if shipping a feature requires a sentence that starts with "in
your AGENTS.md, add…" or "in your cron/jobs.json, rewrite…", the migration
orchestrator should be doing that edit, not the user.
**The exception is host-specific code.** For custom Minion handlers
(host-specific integrations like inbox sweeps or third-party API scanners), shipping them as a
data file the worker would exec is an RCE surface. Those get registered in
the host's own repo via the plugin contract (`docs/guides/plugin-handlers.md`);
the migration orchestrator emits a structured TODO to
`~/.gbrain/migrations/pending-host-work.jsonl` + the host agent walks the
TODOs using `skills/migrations/v0.11.0.md` — stays host-agnostic, still
canonical.
## Schema state tracking
`~/.gbrain/update-state.json` tracks which recommended schema directories the user
adopted, declined, or added custom. The auto-update agent (SKILLPACK Section 17)
reads this during upgrades to suggest new schema additions without re-suggesting
things the user already declined. The setup skill writes the initial state during
Phase C/E. Never modify a user's custom directories or re-suggest declined ones.
## GitHub Actions SHA maintenance
All GitHub Actions in `.github/workflows/` are pinned to commit SHAs. Before shipping
(`/ship`) or reviewing (`/review`), check for stale pins and update them:
```bash
for action in actions/checkout oven-sh/setup-bun actions/upload-artifact actions/download-artifact softprops/action-gh-release gitleaks/gitleaks-action; do
tag=$(grep -r "$action@" .github/workflows/ | head -1 | grep -o '#.*' | tr -d '# ')
[ -n "$tag" ] && echo "$action@$tag: $(gh api repos/$action/git/ref/tags/$tag --jq .object.sha 2>/dev/null)"
done
```
If any SHA differs from what's in the workflow files, update the pin and version comment.
## PR descriptions cover the whole branch
Pull request titles and bodies must describe **everything in the PR diff against the
base branch**, not just the most recent commit you made. When you open or update a
PR, walk the full commit range with `git log --oneline <base>..<head>` and write the
body to cover all of it. Group by feature area (schema, code, tests, docs) — not
chronologically by commit.
This matters because reviewers read the PR body to understand what's shipping. If
the body only covers your last commit, they miss everything else and can't review
properly. A 7-commit PR with a body that describes commit 7 is worse than no body
at all — it actively misleads.
When in doubt, run `gh pr view <N> --json commits --jq '[.commits[].messageHeadline]'`
to see what's actually in the PR before writing the body.
## Community PR wave process
Never merge external PRs directly into master. Instead, use the "fix wave" workflow:
1. **Categorize** — group PRs by theme (bug fixes, features, infra, docs)
2. **Deduplicate** — if two PRs fix the same thing, pick the one that changes fewer
lines. Close the other with a note pointing to the winner.
3. **Collector branch** — create a feature branch (e.g. `garrytan/fix-wave-N`), cherry-pick
or manually re-implement the best fixes from each PR. Do NOT merge PR branches directly —
read the diff, understand the fix, and write it yourself if needed.
4. **Test the wave** — verify with `bun test && bun run test:e2e` (full E2E lifecycle).
Every fix in the wave must have test coverage.
5. **Close with context** — every closed PR gets a comment explaining why and what (if
anything) supersedes it. Contributors did real work; respect that with clear communication
and thank them.
6. **Ship as one PR** — single PR to master with all attributions preserved via
`Co-Authored-By:` trailers. Include a summary of what merged and what closed.
**Community PR guardrails:**
- Always AskUserQuestion before accepting commits that touch voice, tone, or
promotional material (README intro, CHANGELOG voice, skill templates).
- Never auto-merge PRs that remove YC references or "neutralize" the founder perspective.
- Preserve contributor attribution in commit messages.
## Checking out PRs from garrytan-agents
`garrytan-agents` is the AI-authored PR account and is NOT a collaborator on
this repo. Its PRs live in a fork, so GitHub Actions triggered by
`pull_request` events on those PRs do not receive base-repo secrets. Any CI
job that needs `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, or similar will fail
with empty-env auth errors, regardless of what's set on the base repo. This
is a GitHub security default, not a config bug.
When the user says "check out <PR link>" and the PR is from `garrytan-agents`
(or any other non-collaborator fork), move the branch into the base repo
before running CI:
1. `gh pr checkout <N>` — pull down the fork's branch. Note the PR number and
head branch name (`gh pr view <N> --json headRefName --jq .headRefName`).
2. `git push origin HEAD:<branch-name>` — push the same branch to the base
repo (origin points at `garrytan/gbrain`, not the fork). This is the move
that gives CI access to secrets.
3. `gh pr close <N> --comment "moving to base-repo branch for secret access"`
— close the fork PR so the queue stays clean.
4. `gh pr create --base master --head <branch-name>` — open the replacement
PR from the base-repo branch. **Preserve the original PR's title and body
verbatim** (`gh pr view <N> --json title,body`); contributor attribution
moves to a `Co-Authored-By:` trailer if needed.
Why this over alternatives: adding `garrytan-agents` as a collaborator, or
flipping the repo-wide "send secrets to fork PRs" toggle, both broaden
secret distribution to every fork PR from that account or any fork. Moving
the branch keeps secret scope tight to just the one PR being shipped.
-333
View File
@@ -1,333 +0,0 @@
# Testing (gbrain repo)
On-demand reference (see CLAUDE.md Reference map). Current behavior + invariants
only.
`test/e2e/serve-http-oauth.test.ts` additionally pins confidential POST/Basic revocation, public-client SDK fallthrough, malformed/mixed authentication rejection, cross-client isolation, unknown-token opacity, metadata auth methods, no-store responses, strict post-revoke `401`, and retryable backend `503` semantics.
### Test command tiers
Seven test command tiers, each with a clear scope:
| Command | What it runs | Wallclock | When to use |
|---|---|---|---|
| `bun run test` | Parallel unit-test fast loop. 8-shard fan-out via `scripts/run-unit-parallel.sh`, then a serial pass over `*.serial.test.ts`. Excludes `*.slow.test.ts` and `test/e2e/*`. No pre-checks, no typecheck. | ~85s on a Mac dev box (3650+ tests) | Inner edit loop. Default. |
| `bun run verify` | CI's authoritative pre-test gate set, fanned out in parallel by `scripts/run-verify-parallel.sh`: the full `check:*` battery (~30 checks — privacy, jsonb, progress, source-id, test-isolation, wasm, …) plus `bun run typecheck`. The `CHECKS` array in that script is the single source of truth — CI literally calls `bun run verify` in a dedicated job. | ~16s (parallel; typecheck dominates) | Before pushing; before `/ship`. |
| `bun run test:full` | `verify && bun run test && bun run test:slow && [smart e2e]`. The local equivalent of "everything CI runs." Smart e2e: runs e2e only when `DATABASE_URL` is set; else loud skip notice to stderr. | ~3-5min depending on slow + e2e | Pre-merge sanity, before opening a PR. |
| `bun run test:slow` | Just the `*.slow.test.ts` set (intentional cold-path correctness checks). | seconds-to-minutes | When touching slow-path code. |
| `bun run test:serial` | Just the `*.serial.test.ts` set (cross-file-contention quarantine; one bun process per file for true module-registry isolation). | ~1s per quarantined file | Debugging a specific quarantined file. |
| `bun run test:e2e` | Real Postgres E2E. Requires Docker + `DATABASE_URL`. Sequential. | ~5-10min | Pre-ship; nightly. |
| `bun run check:all` | The historical pre-check scripts (22, chained sequentially in package.json). Overlaps `verify` heavily but is NOT a superset — `verify`'s `CHECKS` array in `scripts/run-verify-parallel.sh` (~30 entries incl. typecheck) is the authoritative gate; `check:all` keeps a few local-only extras (trailing-newline, exports-count, no-legacy-getconnection). | ~10s | Local-only sweep for the extras. |
### Shell dispatch and Windows
All four of `test`, `verify`, `ci:local` and `test:e2e` hand off to shell scripts
under `scripts/`, so every `check:*` entry in `package.json` invokes its script as
`bash scripts/<name>.sh` instead of relying on the shebang — bun on Windows cannot
exec a `.sh` directly. Add a new shell-script check with that same prefix. The
`scripts/*.ts` entries run under bun and take no prefix.
The scripts must also be on disk with Unix line endings. A strict bash (WSL, Linux
CI, macOS) rejects CRLF and dies on the script's first meaningful line; the Cygwin
bash that ships with Git for Windows tolerates it, so a green local run is not by
itself evidence that a script is CRLF-clean.
The root `.gitattributes` pins `*.sh text eol=lf`, which overrides the
`core.autocrlf=true` default that Git for Windows installs. Working copies cloned
before that pin need a one-time `git rm --cached -r . -q && git reset --hard` to
pick it up; see the Windows section of `CONTRIBUTING.md`.
Wallclock figures in the table above are from a Mac dev box. Windows is
substantially slower because each check pays full process-creation cost, and three
tree-walking checks (`check:privacy`, `check:test-names`, `check:test-isolation`)
plus `typecheck` can exceed the 120s per-check cap in `run-verify-parallel.sh`
there even though they pass on Linux and macOS.
### CI vs local: intentionally divergent file sets
- **CI matrix** (`.github/workflows/test.yml`) runs `scripts/test-shard.sh` across 10 matrix shards partitioned by weight-aware LPT bin-packing (`scripts/sharding.ts`) and INCLUDES `*.slow.test.ts` (the two outlier slow files run as dedicated jobs alongside the matrix). CI EXCLUDES `*.serial.test.ts` from the shards and runs them in a dedicated job via `bun run test:serial`, one bun process per file — keeping serial files out of the shard processes is what preserves the `mock.module` quarantine (a top-level mock in one file leaks into every other file sharing its process). `bun run verify` gets its own job too. CI is the ground truth for "did everything pass."
- **Local fast loop** (`scripts/run-unit-shard.sh` via the parallel wrapper) uses round-robin-by-index sharding and EXCLUDES `*.slow.test.ts` AND `*.serial.test.ts`. Local trades coverage for inner-loop speed; CI catches what local skips.
This divergence is intentional. Don't try to make them equal — the two scripts deliberately solve different problems. The regression test at `test/scripts/run-unit-shard.test.ts` pins what the local fast loop should and shouldn't include.
### Failure-first logging
When `bun run test` finds any failure, the wrapper:
1. Writes failure blocks (each prefixed with `--- shard N: <test name> ---`) to `.context/test-failures.log` (workspace-local, gitignored). On systems without a writable `.context/`, falls back to `/tmp/gbrain-test-failures.log`.
2. Prints a loud stderr banner with the absolute log path, plus the last 30 lines of the failure log inlined. Banner survives `| head` / `| tail` / agent-side log truncation.
3. Writes a one-line-per-shard summary to `.context/test-summary.txt` (`shard N/M: pass=X fail=Y skip=Z rc=W`).
4. Exits non-zero. Empty failure log + non-zero exit = infrastructure problem (wedged shard, killed child); the banner says so.
If a shard wedges (per-shard `GBRAIN_TEST_SHARD_TIMEOUT` cap, default 600s), the wrapper writes `--- shard N: WEDGED after ${SHARD_TIMEOUT}s ---` to the failure log, includes the last 50 lines of the shard log, and proceeds with other shards' results.
### File taxonomy
- `*.test.ts` → fast loop (parallel 8-shard fan-out).
- `*.slow.test.ts` → run via `bun run test:slow` only (intentional cold-path tests; would dominate the fast loop's wallclock).
- `*.serial.test.ts` → run via `bun run test:serial` after the parallel pass completes; one bun process per file (`--max-concurrency=1` within a shared process is not enough — the module registry still leaks `mock.module`). Quarantine for tests that share file-wide state and race when run alongside other files in the same `bun test` process. Several dozen files, discovered by the `*.serial.test.ts` glob — no list to maintain. Typical residents: `mock.module(...)` users (top-level mocks leak across files in a shard process, e.g. `test/embed.serial.test.ts`), env-coupled files (e.g. `test/brain-registry.serial.test.ts`), and process-lifecycle suites that assert on `process.exitCode` (e.g. `test/pglite-engine-disconnect.serial.test.ts`). **Do not put the parallelism back on a serial file unless you've fixed the contention root cause** (it just re-introduces the flake).
- `test/e2e/*.test.ts` → real-Postgres E2E. Skipped when `DATABASE_URL` is unset.
- `tests/heavy/*.sh` → ops-shape shell scripts. Cost minutes per run; NOT in default `bun test`. Run via `bun run test:heavy` or scheduled nightly via `.github/workflows/heavy-tests.yml`. Examples: pg_upgrade matrix (boot legacy brain → walk to head), RSS budget gate (measure peak worker RSS vs committed baseline), read-latency-under-sync (p50/p95/p99 under concurrent writer load), sync lock regression (N concurrent syncs assert 1 winner + N-1 lock-busy + zero leaked `gbrain_cycle_locks` rows). See `tests/heavy/README.md` for when to add a script here vs `*.slow.test.ts`. Files prefixed with `_` (e.g. `tests/heavy/_build_legacy_fixtures.sh`) are helpers/libs invoked by sibling tests — the runner skips them.
- `test/fuzz/*.test.ts` → property-based fuzz harness. Pure-validator targets in `pure-validators.test.ts` are guarded by `scripts/check-fuzz-purity.sh` (in `bun run verify`), which `bun build --target=bun` bundles each target and greps the resulting bundle for banned transitive imports (`node:fs`, `node:child_process`, engine modules). Anything that fails the guard moves to `mixed-validators.test.ts` (still property-tested, but no purity guarantee) or `filesystem-validators.test.ts` (fs-backed, uses temp dirs). Fuzz tests run in the default `bun test` loop because they're fast (~3s for ~12 properties × 1000 runs each).
### Test-isolation lint and helpers
The cross-file flake class is enforced statically by `scripts/check-test-isolation.sh`, wired into `bun run verify` and `bun run check:all`. Rules (non-serial unit files only; `*.serial.test.ts` and `test/e2e/*` are skipped):
| Rule | What it bans | Fix |
|---|---|---|
| **R1** | `process.env.X = ...`, bracket assignment, `delete process.env.X`, `Object.assign(process.env, ...)`, `Reflect.set(process.env, ...)` | Use `withEnv()` from `test/helpers/with-env.ts`, OR rename file to `*.serial.test.ts` |
| **R2** | `mock.module(...)` anywhere in the file | Rename file to `*.serial.test.ts` (no DI on production code for testability) |
| **R3** | `new PGLiteEngine(` outside ~50 lines after a `beforeAll(` line | Use the canonical block (below) inside `beforeAll(` |
| **R4** | Files creating `new PGLiteEngine(` without `engine.disconnect(` inside an `afterAll(` block | Add `afterAll(() => engine.disconnect())` |
Files that violated these rules at the isolation-lint baseline are listed in `scripts/check-test-isolation.allowlist`. **The allow-list MUST shrink over time** — never add new entries.
#### Canonical PGLite block (R3 + R4 compliant)
Every test file that needs a PGLite engine should use this exact pattern:
```ts
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { resetPgliteState } from './helpers/reset-pglite.ts';
let engine: PGLiteEngine;
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
});
afterAll(async () => {
await engine.disconnect();
});
beforeEach(async () => {
await resetPgliteState(engine);
});
```
Why this exact shape: `beforeAll` creates a single engine per file (PGLite WASM cold-start + initSchema is ~20s); `beforeEach` truncates user data via `resetPgliteState` ("two orders of magnitude faster" than fresh-engine-per-test); `afterAll` disconnects so the engine doesn't leak across file boundaries within a shard process.
#### `withEnv` pattern (R1 fix)
```ts
import { withEnv } from './helpers/with-env.ts';
test('reads OPENAI_API_KEY', async () => {
await withEnv({ OPENAI_API_KEY: 'sk-test' }, async () => {
expect(loadConfig().openai_key).toBe('sk-test');
});
});
// Delete a var (override is undefined):
await withEnv({ GBRAIN_HOME: undefined }, fn);
// Multiple keys:
await withEnv({ A: '1', B: '2', C: undefined }, fn);
```
`withEnv` saves the prior value of every key it touches and restores via try/finally — including when the callback throws. **It is cross-test safe but NOT intra-file concurrent-safe.** `process.env` is process-global; two `test.concurrent()` calls in the same file both touching the same key will race. Files using `withEnv` stay outside the `test.concurrent()` codemod's eligibility filter.
#### When to quarantine instead of fix
Rename to `*.serial.test.ts` when:
- The file uses `mock.module(...)` (R2 — there's no clean fix without changing production code).
- The file is genuinely env-coupled (e.g. `gbrain-home-isolation.test.ts`, `claw-test-cli.test.ts`) — module-load env readers + ESM caching defeat dynamic-import-after-env tricks.
- The file's tests intentionally share state across `it()` boundaries.
The quarantine has grown to dozens of files — treat it as debt: every addition needs a reason from the list above, and prefer fixing the contention root cause when one exists.
### Unit test inventory
`bun test` runs all tests without a database. E2E tests skip gracefully when `DATABASE_URL` is not set.
Unit tests and what they cover:
- `test/markdown.test.ts` — frontmatter parsing; `splitBody` sentinel precedence, horizontal-rule preservation, `inferType` wiki subtypes.
- `test/chunkers/recursive.test.ts` — chunking.
- `test/parity.test.ts` — operations contract parity.
- `test/cli.test.ts` — CLI structure.
- `test/cli-finish-teardown.test.ts` — the #2084 teardown contract: `computeTeardownDeadlineMs` formula/floor/live-registry scaling + `GBRAIN_TEARDOWN_DEADLINE_MS` override (garbage/zero/negative values fall back to the formula); `finishCliTeardown` clean path (drain BEFORE disconnect, no exit, no warn), backstop on hung drain or disconnect (honors an errored op's exit code), throwing drain/disconnect warned + swallowed; the gbrain-owned verdict channel is immune to PGLite WASM `process.exitCode` writes; `flushThenExit` unit coverage with mocked streams (exits once after both stream callbacks, non-TTY aliveness grace, blocked-pipe guard, EPIPE-safe, `GBRAIN_FLUSH_GRACE_MS` override).
- `test/flush-then-exit-harness.test.ts` — real spawned-Bun pipe semantics for `flushThenExit` (fixture: `test/fixtures/flush-then-exit-harness.ts`): a 4MB piped stdout payload arrives byte-complete with the exit code even with a late reader, small output survives exit with a concurrent reader, and the fence resolves promptly (wall time well under the guard + grace ceiling).
- `test/cli-should-force-exit.test.ts``shouldForceExitAfterMain` daemon-survival gate: `serve` (stdio and `--http`) never force-exits, including with preceding global flags; op commands / empty / flag-only argv do; the #2084 case that space-separated global-flag VALUES can't fake a command (`--timeout 30s serve` resolves to the `serve` daemon, not a `30s` command).
- `test/cli-exit-verdict-pin.test.ts`#2084 structural class pin: greps `src/` so the NEXT raw `process.exitCode =` write fails CI (a raw write bypasses the gbrain-owned verdict channel and gets silently zeroed by the deliberate flush-exit — the bug that made doctor's FAIL path exit 0). Runtime variants live in `test/cli-finish-teardown.test.ts`; this is the review-time guard.
- `test/cli-pipe-truncation.test.ts` — real-CLI pipe completeness (the #1959 incident class), implementation-agnostic: the actual CLI run the way agents run it (piped stdout) produces complete, parseable, byte-stable `--tools-json` output and exits deliberately, well under the teardown backstop. Synthetic flush-mechanism coverage stays in `test/flush-then-exit-harness.test.ts`.
- `test/volunteer-context.test.ts` — push-based context core (#2095), hermetic in-memory PGLite: `parseWindow` lenient `user:`/`assistant:` parsing, multi-turn window extraction, confidence-gated volunteering (arm confidences, multi-turn/newest-turn boosts, `min_confidence` gate, max-pages cap), slug-only suppression, privacy (rationales are deterministic templates; synopses pass the takes/facts fence), and the approximate usage-stats join.
- `test/watch-command.test.ts``gbrain watch` push transport (#2095): streaming loop, rolling window, session dedupe, `--json` JSONL shape, `channel: 'watch'` event logging, clean EOF return. Hermetic PGLite + injected line/write deps (no subprocess, no real stdin).
- `test/watch-sigint.serial.test.ts``gbrain watch` SIGINT lifecycle against a real spawned CLI subprocess with a tmpdir brain. SERIAL: parallel unit shards flake on concurrent subprocess spawns (same rationale as `apply-migrations-pglite-spawn.serial.test.ts`).
- `test/cli-format-volunteer.test.ts``formatResult`'s `volunteer_context` human rendering: pointer lines with confidence/arm/rationale, the empty-result message, the approximate stats summary.
- `test/config.test.ts` — config redaction.
- `test/files.test.ts` — MIME/hash.
- `test/import-file.test.ts` — import pipeline.
- `test/upgrade.test.ts` — schema migrations.
- `test/file-migration.test.ts` — file migration.
- `test/file-resolver.test.ts` — file resolution.
- `test/import-resume.test.ts` — import checkpoints.
- `test/migrate.test.ts` — migration: v8/v9 helper-btree-index SQL structural assertions; 1000-row wall-clock fixtures guarding the O(n²)→O(n log n) fix; v12/v13 SQL shape; `sqlFor` + `transaction:false` runner semantics; the `max_stalled DEFAULT 1` regression guard; v24 `sqlFor.pglite: ''` no-op assertion; v117 `context_volunteer_events` (named + idempotent entry, documented columns + both source-scoped indexes after `initSchema`, insert + 90-day `purgeStaleVolunteerEvents` round-trip).
- `test/bootstrap.test.ts` — bootstrap contract: no-op on fresh install, idempotent across two `initSchema()` calls, no-op on modern brain that already has every probed column, full bootstrap path on a simulated legacy brain, fresh-install regression guard, legacy `links` shape coverage.
- `test/schema-bootstrap-coverage.test.ts` — CI guard. `REQUIRED_BOOTSTRAP_COVERAGE` lists every forward reference in `PGLITE_SCHEMA_SQL`; the test fails loudly if `applyForwardReferenceBootstrap` skips one (extend both arrays when adding a column-with-index to the embedded schema blob). Also parses `src/core/migrate.ts` source text for every `ALTER TABLE ... ADD COLUMN` (top-level `sql:`, `sqlFor.{postgres,pglite}` overrides, AND handler-body `engine.runMigration(N, \`ALTER TABLE ...\`)`) and asserts each (table, column) pair is covered by the bootstrap OR by the schema blob's CREATE TABLE bodies — catching the column-only forward-reference class (e.g. `sources.archived`, `oauth_clients.source_id`) that a CREATE INDEX parser alone can't see. `parseBaseTableColumns` strips SQL line + block comments before identifying column names so commented-out lines don't hide adjacent columns.
- `test/helpers/schema-diff.ts` + `test/helpers/schema-diff.test.ts` + `test/e2e/schema-drift.test.ts` — cross-engine schema parity gate. Helper exports pure `snapshotSchema(query)` / `diffSnapshots(pg, pglite, opts)` / `formatDiffForFailure(diff)` / `isCleanDiff(diff)` over a four-tuple per column (`data_type`, `udt_name`, `is_nullable`, `column_default`). E2E test spins up fresh PGLite + Postgres, runs `engine.initSchema()` on each, snapshots `information_schema.columns`, then diffs. 2-table allowlist (`files`, `file_migration_ledger`) — every other Postgres table must reach PGLite via `PGLITE_SCHEMA_SQL` or a migration's `sqlFor.pglite` branch. Sentinels for `oauth_clients`, `mcp_request_log`, `access_tokens`, `eval_candidates` give tighter blame messages. Skips without `DATABASE_URL`. Wired into `scripts/e2e-test-map.ts` so changes to `src/schema.sql`, `src/core/pglite-schema.ts`, or `src/core/migrate.ts` trigger it. The failure message names every drift with a paste-ready hint pointing at `src/core/pglite-schema.ts`.
- `test/setup-branching.test.ts` — setup flow.
- `test/slug-validation.test.ts` — slug validation.
- `test/storage.test.ts` — storage backends.
- `test/supabase-admin.test.ts` — Supabase admin.
- `test/yaml-lite.test.ts` — YAML parsing.
- `test/check-update.test.ts` — version check + update CLI.
- `test/pglite-engine.test.ts` — PGLite engine, all BrainEngine methods including `addLinksBatch` / `addTimelineEntriesBatch` (empty batch, missing optionals, within-batch dedup via ON CONFLICT, missing-slug rows dropped by JOIN, half-existing batch, batch of 100) plus `connect()` error-wrap assertion (original error nested, #223 link in message, lock released).
- `test/links-timeline-jsonb-poison.test.ts` — gbrain#1861 PGLite half (always-on, no `DATABASE_URL`). Locks the `jsonb_to_recordset` batch-insert path for links/timeline/takes against free-text "poison" payloads (commas, quotes, backslashes, braces, em-dashes) and asserts NUL is stripped from free-text body fields but rejected in identity fields. gbrain#2011 adds lone-UTF-16-surrogate cases: every free-text field (link context; timeline summary/detail/source; take claim/source) well-forms to U+FFFD across batch + scalar write paths, while a surrogate in an identity field (slug) still fail-closed rejects the batch. The Postgres lane (`test/e2e/jsonb-batch-poison-postgres.test.ts`) is the one that actually reproduced the original crash.
- `test/engine-factory.test.ts` — engine factory + dynamic imports.
- `test/integrations.test.ts` — recipe parsing, CLI routing, recipe validation.
- `test/publish.test.ts` — content stripping, encryption, password generation, HTML output.
- `test/backlinks.test.ts` — entity extraction, back-link detection, timeline entry generation.
- `test/lint.test.ts` — LLM artifact detection, code fence stripping, frontmatter validation.
- `test/report.test.ts` — report format, directory structure.
- `test/skills-conformance.test.ts` — skill frontmatter + required sections validation.
- `test/resolver.test.ts` — RESOLVER.md coverage, routing validation; round-trip that every quoted RESOLVER.md trigger matches a frontmatter `triggers:` entry in the target skill, and every `name="<word>"` reference in any SKILL.md resolves to a declared op in `src/core/operations.ts` or a Minions handler in `PROTECTED_JOB_NAMES`.
- `test/search.test.ts` — RRF normalization, compiled truth boost, cosine similarity, dedup key.
- `test/sql-ranking.test.ts` — source-boost helpers: longest-prefix-match in SQL CASE, `detail=high` temporal-bypass, three-meta-char LIKE escape (`%`, `_`, `\`), single-quote SQL-literal doubling, env override parsing for `GBRAIN_SOURCE_BOOST` + `GBRAIN_SEARCH_EXCLUDE`, `resolveBoostMap` / `resolveHardExcludes` merge semantics.
- `test/dedup.test.ts` — source-aware dedup, compiled truth guarantee, layer interactions.
- `test/intent.test.ts` — query intent classification: entity/temporal/event/general.
- `test/eval.test.ts` — retrieval metrics: `precisionAtK`, `recallAtK`, `mrr`, `ndcgAtK`, `parseQrels`.
- `test/check-resolvable.test.ts` — resolver reachability, MECE overlap, gap detection, proximity-based DRY detection, `extractDelegationTargets` coverage.
- `test/dry-fix.test.ts` — auto-fix: three shape-aware expander pure-function tests; five guards (working-tree-dirty, no-git-backup, inside-code-fence, already-delegated within 40 lines, ambiguous-multi-match, block-is-callout).
- `test/doctor-fix.test.ts``gbrain doctor --fix` CLI integration: dry-run preview, apply path, JSON output shape.
- `test/backoff.test.ts` — load-aware throttling, concurrency limits, active hours.
- `test/fail-improve.test.ts` — deterministic/LLM cascade, JSONL logging, test generation, rotation.
- `test/transcription.test.ts` — provider detection, format validation, API key errors.
- `test/enrichment-service.test.ts` — entity slugification, extraction, tier escalation.
- `test/data-research.test.ts` — recipe validation, MRR/ARR extraction, dedup, tracker parsing, HTML stripping.
- `test/minions.test.ts` — Minions job queue: CRUD, state machine, backoff, stall detection, dependencies, worker lifecycle, lock management, claim mechanics, depth/child-cap, timeouts, cascade kill, idempotency, `child_done` inbox, attachments, removeOnComplete/Fail, `max_stalled` clamp/default/plumbing coverage.
- `test/extract.test.ts` — link extraction, timeline extraction, frontmatter parsing, directory type inference.
- `test/extract-db.test.ts``gbrain extract --source db`: typed link inference, idempotency, `--type` filter, `--dry-run` JSON output.
- `test/extract-fs.test.ts``gbrain extract --source fs`: first-run inserts + second-run reports zero, dry-run dedups candidates across files, second-run perf regression guard for the N+1 dedup bug.
- `test/link-extraction.test.ts` — canonical `extractEntityRefs` both formats, `extractPageLinks` dedup, `inferLinkType` heuristics, `parseTimelineEntries` date variants, `isAutoLinkEnabled` config.
- `test/graph-query.test.ts` — direction in/out/both, type filter, indented tree output.
- `test/features.test.ts` — feature scanning, brain_score calculation, CLI routing, persistence.
- `test/file-upload-security.test.ts` — symlink traversal, cwd confinement, slug + filename allowlists, remote vs local trust.
- `test/query-sanitization.test.ts` — prompt-injection stripping, output sanitization, structural boundary.
- `test/search-limit.test.ts``clampSearchLimit` default/cap behavior across `list_pages` and `get_ingest_log`.
- `test/repair-jsonb.test.ts` — JSONB repair: TARGETS list, idempotency, engine-awareness.
- `test/migrations-v0_12_2.test.ts` — JSONB-repair orchestrator phases: schema → repair → verify → record.
- `test/orphans.test.ts` — orphans command: detection, pseudo filtering, text/json/count outputs, MCP op.
- `test/postgres-engine.test.ts``statement_timeout` scoping: `sql.begin` + `SET LOCAL` shape, source-level grep guardrail against a reintroduced bare `SET statement_timeout`.
- `test/sync.test.ts` — sync logic + regression guard asserting top-level `engine.transaction` is not called.
- `test/sync-pull-failed-anchor.serial.test.ts`#3068 regression: a failed internal `git pull` (local-path origin vs `protocol.file.allow=never`) with zero imports returns `partial`/`pull_failed` (not `up_to_date`), freezes `last_commit` + `last_sync_at`, recovers after a manual pull; fall-through import of local commits preserved. Serial: pins `GBRAIN_HOME` to a temp dir for the whole file.
- `test/sync-concurrency.test.ts``autoConcurrency()` thresholds + PGLite-forces-serial + explicit-override clamping; `shouldRunParallel()` explicit-bypasses-floor contract; `parseWorkers()` validation rejecting `'0'`/`'-3'`/`'foo'`/`'1.5'`/trailing chars.
- `test/sync-parallel.test.ts` — PGLite-routed coverage of the bookmark gate under concurrency, head-drift gate, vanished-file failure capture, PGLite-stays-serial, and the `gbrain-sync` writer-lock contract.
- `test/sync-all-missing-path.test.ts``sync --all --missing-path <fail|skip>` pure helpers: `parseMissingPathMode` (default fail, explicit values, loud rejection of bad/dangling values, never swallows a following flag) and `partitionMissingPathSources` (classification driven only by the injected pathExists predicate — no fs; null `local_path` passes through runnable; order preserved).
- `test/sync-failures.test.ts``classifyErrorCode` regex coverage for all 12 codes against literal production message strings from `markdown.ts` and `import-file.ts`; `summarizeFailuresByCode` sort + pre-classified-honor; `recordSyncFailures` code-field persistence; `acknowledgeSyncFailures` `AcknowledgeResult` shape + backfill on legacy entries.
- `test/doctor.test.ts` — doctor command; assertions that `jsonb_integrity` scans the four JSONB write sites and `markdown_body_completeness` is present.
- `test/utils.test.ts` — shared SQL utilities + `tryParseEmbedding` null-return and single-warn semantics.
- `test/build-llms.test.ts``llms.txt`/`llms-full.txt` generator: path resolution, idempotence, spec shape, regen-drift guard, content contract, AGENTS.md install-path mirror, size-budget enforcement.
- `test/oauth.test.ts` — OAuth 2.1 provider: register, getClient, `client_credentials` grant exchange, `authorization_code` flow with PKCE challenge/verifier, refresh token rotation, `verifyAccessToken` with both OAuth + legacy `access_tokens` fallback, `revokeToken`, `sweepExpiredTokens`; contract test asserting `scope` + `localOnly` annotations on all operations; `coerceTimestamp` unit cases (null/undefined/string/number/throw-on-NaN); NULL-`expires_at`-as-expired contract for both refresh + access token paths; cascade-delete contract asserting `revoke-client` purges `oauth_tokens` + `oauth_codes` via FK CASCADE; cross-client isolation (wrong-client attempt MUST reject AND rightful owner MUST still succeed atomically afterward); empty-string `redirect_uri` bypass guard; PKCE DCR public-client gate (`token_endpoint_auth_method: "none"` returns no `client_secret`, default `client_secret_post` clients get the one-time-reveal secret, `getClient` NULL→undefined normalization, full PKCE `/authorize``/token` round-trip against a public client).
- `test/mcp-dispatch-summarize.test.ts``summarizeMcpParams` invariants: declared-keys allow-list intersection, attacker-key-name leak guard (unknown keys counted not named), 1KB byte bucketing for size-probe defense, missing op falls through to fully-redacted shape, declared-keys sorted for deterministic output.
- `test/trust-boundary-contract.test.ts` — fail-closed trust semantics under cast bypass: `ctx.remote === undefined` treated as remote/untrusted at every flipped call site; `as any` and `Partial<>` spreads can't downgrade trust by accident.
- `test/check-resolvable-cli.test.ts` — CLI wrapper: exit codes, JSON envelope shape, AGENTS.md fallback chain.
- `test/regression-v0_16_4.test.ts``findRepoRoot` regression guard, hermetic startDir parameterization.
- `test/repo-root.test.ts``findRepoRoot` walk semantics + default-arg parity; the 4-tier `autoDetectSkillsDir` fallback chain (`$OPENCLAW_WORKSPACE``~/.openclaw/workspace` → repo-root → `./skills`); RESOLVER.md/AGENTS.md filename precedence; explicit-env-wins-over-repo-root; tier-0 `$GBRAIN_SKILLS_DIR` valid/invalid/precedence-over-`OPENCLAW_WORKSPACE`; the install-path walk in `autoDetectSkillsDirReadOnly`; no-drift on primary success; `AUTO_DETECT_HINT` + `AUTO_DETECT_HINT_READ_ONLY` content; regression guard asserting the shared `autoDetectSkillsDir` MUST NEVER return `'install_path'` source (how the read-path/write-path split stays safe).
- `test/resolver-merge.test.ts` — multi-file resolver merge: `findAllResolverFiles` empty / RESOLVER.md-only / AGENTS.md-only / both-present (RESOLVER.md first); `checkResolvable` merge semantics across `skills/RESOLVER.md` + `../AGENTS.md` for the OpenClaw layout where the skillpack ships a thin RESOLVER.md and the real dispatcher lives at the workspace root; dedup by `skillPath` (first occurrence wins); AGENTS.md-at-workspace-root works alone.
- `test/filing-audit.test.ts` — filing audit: `writes_pages` / `writes_to` frontmatter, filing-rules JSON validation.
- `test/skill-brain-first.test.ts` — shared frontmatter parser; `analyzeSkillBrainFirst` compliance ladder across 9 fixtures under `test/fixtures/brain-first-skills/` (compliant-callout, compliant-phase, compliant-position, exempt-frontmatter, missing-brain-first, multi-pattern, negation-prose, no-external, typo-frontmatter); offset helpers; external-lookup regex shape; audit snapshot+diff transition logic; `FORMERLY_HARDCODED_EXEMPT` regression absorption.
- `test/routing-eval.test.ts` — fixture parsing, structural routing, `ambiguous_with`, Haiku tie-break layer.
- `test/skill-manifest.test.ts` — skill manifest parser: drift detection, managed-block markers.
- `test/skillify-scaffold.test.ts``gbrain skillify scaffold` stubs: SKILL.md, script, tests, routing-eval fixtures.
- `test/skillpack-install.test.ts``gbrain skillpack install` managed-block install / update / no-clobber semantics.
- `test/skillpack-sync-guard.test.ts` — sync-guard: bundled skills stay byte-identical to `skills/` source.
- `test/http-transport.test.ts` — HTTP transport: bearer auth + missing/no-Bearer/unknown/revoked + `/health` bypass; dispatch.ts round-trip; invalid_params; application/json response shape (not SSE); CORS default-deny + allowlist; body cap on Content-Length AND chunked; two-bucket rate limit (refill, exhaust+Retry-After, LRU eviction, TTL prune, pre-auth IP fires before DB); `mcp_request_log` audit on success + auth_failed.
- `test/restart-sweep.test.ts``recipes/restart-sweep.md` inlined script: sentinel-anchored fenced-block extraction with salted tmp filenames to bypass ESM cache; constructor-time env reads (proves no module-load snapshot); idempotency layer load/save/atomic-tmp-rename/corrupt-JSON-recovery/30-day-prune; `(sessionKey, lastAlertedAt)` cooldown gate with 6h threshold; AGGRESSIVE-gate two-state tests; execFile argv shape proving shell metachars in `OPENCLAW_TELEGRAM_GROUP` cannot reach `/bin/sh`; real-`\n`-not-literal alert formatting; `GBRAIN_HOME` state path override.
- `test/eval-longmemeval.test.ts` — LongMemEval harness, hermetic with no `DATABASE_URL` and no API keys: PGLite create + reset over runtime-enumerated `pg_tables`, infrastructure-table preservation across resets, JSONL question parsing, retrieval-only and answer-gen modes via stubbed `ThinkLLMClient`, `--limit` cutoff, `--keyword-only` vs hybrid, default `--expansion=off` behavior, perf gate (p50 < 30ms / p99 < 50ms warm reset+import+search on Apple Silicon), `--help` works without a configured brain, fixture round-trip via `test/fixtures/longmemeval-mini.jsonl`.
- `test/longmemeval-sanitize.test.ts` — sanitization parity pinning that `INJECTION_PATTERNS` from `src/core/think/sanitize.ts` is the single source of truth (adding a pattern there must cover both `<take>` framing and `<chat_session>` framing, no per-surface regex drift).
- `test/openai-compat-multimodal.test.ts` — gateway's openai-compatible multimodal path: happy-path single + multi-input embedding, unauthenticated proxy mode, dimension-mismatch guard (throws `AIConfigError` with model id + observed + expected pre-storage), default-dim fallback when recipe declares `default_dims`, HTTP 401 / 400 / malformed-JSON / non-array error paths, regression that the existing Voyage `/multimodalembeddings` recipe still routes through its dedicated path. Hermetic via the `__setEmbedTransportForTests` seam.
- `test/serve-stdio-lifecycle.test.ts``MCP_STDIO=1` env guard: stdin EOF does NOT trigger shutdown when the env is set, SIGTERM still does (guard scope is correct), unset env preserves the CLI lifecycle. Exercises the `ServeOptions.mcpStdio?: boolean` test seam directly so tests don't mutate `process.env`.
### E2E test inventory
E2E tests live in `test/e2e/` and run against real Postgres+pgvector (require `DATABASE_URL`), except where noted as PGLite in-memory (no `DATABASE_URL` needed).
- `bun run test:e2e` runs Tier 1 (mechanical, all operations, no API keys). Includes dedicated cases for the postgres-engine `addLinksBatch` / `addTimelineEntriesBatch` bind path — postgres-js's JSONB bind (`jsonb_to_recordset(($1::jsonb)->'rows')`) differs from PGLite's and gets its own coverage.
- `test/e2e/search-quality.test.ts` — search quality against PGLite (no API keys, in-memory).
- `test/e2e/graph-quality.test.ts` — knowledge graph pipeline (auto-link via put_page, reconciliation, traversePaths) against PGLite in-memory.
- `test/e2e/jsonb-batch-poison-postgres.test.ts` — gbrain#1861 regression, the engine that actually crashed. Seeds free-text "poison" context (Zoom URL with `?pwd=`, commas, quotes, Windows backslash path, braces, em-dash) and asserts the links/timeline/takes batch writers no longer error with "malformed array literal"; also asserts NUL is stripped from free-text bodies (`context`/`summary`/`detail`/`claim`) and still rejected in identity fields. gbrain#2011 adds the lone-surrogate crash lock: a lone UTF-16 surrogate in free text (the value that aborted `extract --stale` with `22P02` on Supabase) well-forms to U+FFFD across batch + scalar paths (incl. timeline + take `source`), while a surrogate in an identity field still rejects the batch. `DATABASE_URL`-gated.
- `test/e2e/postgres-jsonb.test.ts` — round-trips all 5 JSONB write sites (`pages.frontmatter`, `raw_data.data`, `ingest_log.pages_updated`, `files.metadata`, `page_versions.frontmatter`) against real Postgres and asserts `jsonb_typeof='object'` plus `->>'key'` returns the expected scalar. Guards against the double-encode bug.
- `test/e2e/integrity-batch.test.ts` — parity for `scanIntegrity`'s batch-load fast path vs sequential. Cases (dedup, hits, validate, topPages) seed a fixture and assert both paths return identical results. Dedup case uses raw SQL via `getConn().unsafe()` to seed a `(test-source-2, people/alice)` row alongside the default-source row, since `engine.putPage` doesn't take a `source_id`. Pins multi-source overcounting; the "multi-source duplicate slugs scan once" case expects both batch + sequential paths to report 2.
- `test/e2e/jsonb-roundtrip.test.ts` — companion regression against the 4 doctor-scanned JSONB sites. Assertion-level overlap with `postgres-jsonb.test.ts` is intentional defense-in-depth: if doctor's scan surface drifts from the actual write surface, one of these tests catches it.
- `test/e2e/sync.test.ts``--skip-failed` failure-loop test alongside happy-path tests: broken file → `performSync` returns `blocked_by_failures` with grouped breakdown → `performSync({skipFailed: true})` advances bookmark and returns `AcknowledgeResult` with code summary → second broken file → second cycle. Saves and restores the user's real `~/.gbrain/sync-failures.jsonl` so the test is hermetic. Asserts bookmark gating, JSONL state, dedup across paths, summary aggregation, and the literal doctor-rendering string format.
- `test/e2e/upgrade.test.ts` — check-update against real GitHub API (network required).
- `test/e2e/minions-shell-pglite.test.ts` — PGLite `--follow` inline shell-job path (in-memory, no `DATABASE_URL` required) — the path the minion-orchestrator skill documents for dev use.
- `test/e2e/pglite-cli-exit.serial.test.ts` — real spawned-CLI exit behavior on PGLite (in-memory, no `DATABASE_URL`): read commands (`search`/`get`/`query`) exit 0 promptly; CLI_ONLY `capture` exits clean and frees the single-writer lock; the `#2084` describes pin every swept disconnect site — a failed op exits 1 with the error on stderr, and the dashboard, read-only-timeout, doctor, and `dream --dry-run` paths all exit with no force-exit banner.
- `test/e2e/pgbouncer-teardown.test.ts` — PgBouncer TRANSACTION-mode teardown (#2084 / the #1972#2015#2084 class). Pins the bug CLASS, not timings: a CLI op against a txn-mode pooled URL exits 0 with intact stdout and does NOT ride the 10s hard-deadline backstop (the `engine.disconnect() did not return` banner is the smoking gun — pre-#2084 it printed on 100% of query-shaped ops). Gated by `GBRAIN_PGBOUNCER_URL` + `GBRAIN_PGBOUNCER_DIRECT_URL` (NOT `DATABASE_URL`) — set automatically by `bun run ci:local`'s `pgbouncer` compose service; skips gracefully elsewhere. Uses a DEDICATED `gbrain_pgbouncer` database so it never races the `gbrain_test` TRUNCATE fixtures.
- `test/e2e/volunteer-context-postgres.test.ts``volunteer_context` on REAL Postgres (#2095; engine parity beyond the hermetic PGLite unit suite): resolution arms through the actual op handler, the fire-and-forget volunteer-event sink landing rows, the stats join, and the RLS pin that `context_volunteer_events` has ROW LEVEL SECURITY enabled (keeps the v35 auto-RLS event trigger honest for migration-created tables). `DATABASE_URL`-gated.
- `test/e2e/openclaw-reference-compat.test.ts``check-resolvable` + `skillpack install` against a minimal AGENTS.md workspace fixture (`test/fixtures/openclaw-reference-minimal/`), regression guard for the OpenClaw deployment shape.
- `test/e2e/search-swamp.test.ts` — reproduces the source-swamp case. Seeds a curated `originals/talks/article-outline-fat-code` page against two `<fork>/chat/` pages stuffed with the same multi-word phrase. Asserts the article wins keyword AND vector ranking, that `detail=high` lets the chat swamp re-surface, and that `source_id` passes through the two-stage CTE intact. PGLite in-memory.
- `test/e2e/search-exclude.test.ts``test/` + `archive/` pages hidden by default, `include_slug_prefixes` opts back in, caller-supplied `exclude_slug_prefixes` adds to defaults. Both keyword and vector search paths.
- `test/e2e/engine-parity.test.ts` — Postgres ↔ PGLite top-result and result-set parity for `searchKeyword` + `searchVector` (Postgres ranks pages then picks best chunk while PGLite returns chunks directly, so the source-boost behavior needs parity coverage). Skips without `DATABASE_URL`.
- `test/e2e/postgres-bootstrap.test.ts` — exercises `PostgresEngine.initSchema()` directly against a fresh real Postgres database. Asserts the bootstrap path is no-op on fresh installs and that SCHEMA_SQL replays cleanly through the engine path (not via the standalone `db.initSchema` from `src/core/db.ts`).
- `test/e2e/http-transport.test.ts``gbrain serve --http` end-to-end against real Postgres: bearer auth round-trip, `last_used_at` SQL-level debounce, `mcp_request_log` row insertion on success and auth_failed paths, `/health` DB-down → 503 (DB-probing health check), and the dispatch round-trip with a real operation. Skips without `DATABASE_URL`.
- `test/e2e/serve-http-oauth.test.ts` — real-Postgres E2E against `gbrain serve --http` with full OAuth 2.1. Spawns a subprocess server, registers a client via the CLI, mints `client_credentials` tokens, exercises the `/mcp` JSON-RPC pipeline. Real DCR `/register` HTTP-level response-shape test (asserts `typeof body.client_id_issued_at === 'number'` over the wire, RFC 7591 §3.2.1); real CLI subprocess test for `revoke-client` (registers → mints token → revokes via `execSync` → asserts token rejected at `/mcp` → asserts re-run exits 1); server fixture flips on `--enable-dcr` so `/register` is reachable. **bun execSync env-inheritance contract:** bun's `execSync` does NOT inherit env mutations done via `process.env.X = ...`, only OS-level env from before bun started. helpers.ts loads `.env.testing` and sets `DATABASE_URL` via `process.env` mutation, which is invisible to subprocesses unless `env: { ...process.env }` is passed explicitly — every subprocess call in this file passes `env: { ...process.env }`. Reference fix for the same failure mode in sibling sync/cycle/dream/claw-test E2Es. `afterAll` cleanup is guarded on `clientId` (won't throw if `beforeAll` failed before registration); cleanup errors surface to stderr without throwing so real test failures aren't masked. Also covers the trust-boundary fix: an HTTP MCP `submit_job` for `name: "shell"` MUST reject with a permission error (request handler sets `remote: true` and `submit_job`'s protected-name guard fires), and the same guard rejects subagent submission. Skips without `DATABASE_URL`.
- `test/e2e/sync-parallel.test.ts``DATABASE_URL`-gated. 60-file Postgres sync at concurrency=4 imports all + no connection leak (probes `pg_stat_activity` before/after to confirm worker engines disconnected). 120-file serial-vs-parallel benchmark prints `SYNC_PARALLEL_BENCH N files | serial=Xms | parallel(4)=Yms | speedup=Zx`. Asserts parallel ≤ serial × 1.5 (CI-noise tolerant; not a strict speedup gate).
- `test/e2e/multi-source-bug-class.test.ts` — PGLite in-memory regression suite pinning every multi-source bug site: `listAllPageRefs` ordering by `(source_id, slug)`, `getPage` with sourceId picks the right `(source, slug)` row, `extract-takes` processes both overlapping `people/alice` rows independently, `listPages` filters correctly with `PageFilters.sourceId`, `addLinksBatch` with `from/to_source_id` targets the right rows, `validateSourceId` rejects path traversal, reverse-write disk layout uses `brainDir/.sources/<id>/<slug>.md` for non-default sources, `copyMigrationSources` lands source metadata before overlapping-slug pages. No `DATABASE_URL` needed. Wired into `scripts/e2e-test-map.ts` so changes to extract-takes / patterns / synthesize / embed / extract / migrate-engine auto-trigger it.
- `test/e2e/migrate-engine-sources-postgres.test.ts``DATABASE_URL`-gated companion for `gbrain migrate --to`: migrates a PGLite brain carrying two non-default sources with overlapping slugs into real Postgres and asserts `copyMigrationSources` created every `sources` FK parent (config JSONB intact, not double-encoded) before any page write. Unit-level manifest identity (crash manifest resumes only against the SAME target; legacy engine-only manifests start fresh) is `test/migrate-engine-resume.test.ts`.
- `test/e2e/facts-fence-reconcile-postgres.test.ts``DATABASE_URL`-gated round-trip for the escape-aware fence parser: renders a `## Facts` fence whose cells carry literal pipes, backslashes (Windows paths), and empty cells via `renderFactsTable`, runs the wipe-and-reinsert reconcile (`runExtractFacts`) on real Postgres, and asserts every cell survives byte-identically with no column shift.
- `test/e2e/source-isolation-pglite.test.ts` — PGLite in-memory regression suite pinning the source-isolation seal at two layers. Engine layer: `searchKeyword` / `searchVector` / `searchKeywordChunks` / `listPages` / `getPage` / `traverseGraph` / `traversePaths` apply `sourceId` (scalar fast path) and `sourceIds` (array path) correctly across both engines. Op-handler layer: routes through `sourceScopeOpts(ctx)` so a `read+write`-scoped OAuth client bound to `--source dept-x` cannot see rows from neighboring sources via `search`, `query`, `list_pages`, `get_page`, or `find_experts`. Covers both `ctx.sourceId` (single-source clients) and `ctx.auth.allowedSources` (federated_read clients) precedence; federated array wins over scalar wins over nothing. No `DATABASE_URL` needed.
- `test/e2e/think-source-isolation-pglite.test.ts` — PGLite in-memory suite pinning the `think` gather stage's source scope: seeds three sources with cross-source links and embedded takes, then asserts `runGather` under a federated `sourceIds` grant (and under a scalar `sourceId`) keeps every stream — hybrid retrieval, takes keyword + vector (`searchTakes`/`searchTakesVector`), and the `traversePaths` graph walk — inside the grant while still reaching authorized neighboring sources. No `DATABASE_URL` needed.
- `test/e2e/skill-brain-first.test.ts` — doctor reports `skill_brain_first` check with structured issues; `--fix --dry-run` previews insertion without writing; `--fix` applies the canonical Convention callout idempotently; `brain_first: exempt` frontmatter resolves the warn; `brain_first_typo` surfaces a paste-ready hint; audit JSONL records `detected` / `resolved` / `fixed` transitions; stable brain emits 0 audit lines/run.
- Tier 2 (`test/e2e/skills.test.ts`) requires OpenClaw + API keys, runs nightly in CI.
- If `.env.testing` doesn't exist in this directory, check sibling worktrees: `find ../ -maxdepth 2 -name .env.testing -print -quit` and copy it here if found.
- **Run E2E tests without asking permission.** When you want to verify behavior, there's a relevant E2E test, or you're shipping anything covered by an E2E suite — spin up the test DB, run the tests, tear down. Don't ask, don't propose it, don't defer. The lifecycle is short (~2-30s startup, sub-minute tests, instant teardown) and the gate value is high. Skipping with "DATABASE_URL unset" is silent regression, not caution.
### API keys and running ALL tests
ALWAYS source the user's shell profile before running tests:
```bash
source ~/.zshrc 2>/dev/null || true
```
This loads `OPENAI_API_KEY` and `ANTHROPIC_API_KEY`. Without these, Tier 2 tests
skip silently. Do NOT skip Tier 2 tests just because they require API keys — load
the keys and run them.
When asked to "run all E2E tests" or "run tests", that means ALL tiers:
- Tier 1: `bun run test:e2e` (mechanical, sync, upgrade — no API keys needed)
- Tier 2: `test/e2e/skills.test.ts` (requires OpenAI + Anthropic + openclaw CLI)
- Always spin up the test DB, source zshrc, run everything, tear down.
### E2E test DB lifecycle (ALWAYS follow this)
You are responsible for spinning up and tearing down the test Postgres container.
Do not leave containers running after tests. Do not skip E2E tests, do not ask
permission to run them — see the "run without asking" rule above.
1. **Check for `.env.testing`** — if missing, copy from sibling worktree.
Read it to get the DATABASE_URL (it has the port number).
2. **Check if the port is free:**
`docker ps --filter "publish=PORT"` — if another container is on that port,
pick a different port (try 5435, 5436, 5437) and start on that one instead.
3. **Start the test DB:**
```bash
docker run -d --name gbrain-test-pg \
-e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=postgres \
-e POSTGRES_DB=gbrain_test \
-p PORT:5432 pgvector/pgvector:pg16
```
Wait for ready: `docker exec gbrain-test-pg pg_isready -U postgres`
4. **Bootstrap the schema** (required — fresh containers have no `oauth_clients`,
`mcp_request_log`, `pages` etc.; tests like `serve-http-oauth.test.ts` will fail
with `relation "oauth_clients" does not exist` if you skip this):
```bash
DATABASE_URL=postgresql://postgres:postgres@localhost:PORT/gbrain_test \
bun run src/cli.ts doctor --json > /dev/null 2>&1
```
`gbrain doctor` triggers `initSchema()` on first connect, which is the canonical
way to bring a fresh DB to head. `apply-migrations --yes` alone does NOT seed
the base schema — it runs ALTER-style migrations on top of `initSchema`. Tests
that bypass the engine (raw `execSync`-spawned `auth register-client`) hit the
schema directly and need this step to have run first.
5. **Run E2E tests:**
`DATABASE_URL=postgresql://postgres:postgres@localhost:PORT/gbrain_test bun run test:e2e`
6. **Tear down immediately after tests finish (pass or fail):**
`docker stop gbrain-test-pg && docker rm gbrain-test-pg`
Never leave `gbrain-test-pg` running. If you find a stale one from a previous run,
stop and remove it before starting a new one.
-72
View File
@@ -538,75 +538,3 @@ To check what your fork is missing:
diff <(grep -A3 "Based on gbrain" ~/<your-fork>/skills/brain-ops/SKILL.md) \
<(grep "v[0-9]" ~/gbrain/skills/migrations/ | tail -3)
```
## v0.36.5.0 — Free-form secret inheritance for shell jobs calling `gbrain` CLI
**The change.** Shell-job params get a new `inherit:` field. Pass any
snake_case config-key name on it; the worker resolves the value from its
`loadConfig()` at child-spawn time and injects it into the child env. Names
land in the row; values never persist from `inherit:`. Validation runs
**pre-enqueue** in both submit paths (CLI + `submit_job` op), so a malformed
payload never lands in `minion_jobs.data`.
**Why.** Pre-v0.36.5.0, agents that wanted to call `gbrain` from shell jobs
had to either write `database_url` to `~/.gbrain/config.json` plaintext or
pass `env: { GBRAIN_DATABASE_URL: "..." }` per-job. Both left plaintext
secrets somewhere — disk or DB row. `inherit:` keeps names in the row and
resolves values at spawn time.
**What your agent can do.** `inherit:` is free-form. Pass any config-key:
```jsonc
{
"cmd": "gbrain sync --skip-failed && gbrain embed --stale",
"cwd": "/data/gbrain",
"inherit": ["database_url", "anthropic_api_key", "voyage_api_key"]
}
```
The env-key name in the child is derived by uppercasing the config-key:
`database_url``GBRAIN_DATABASE_URL`, `anthropic_api_key`
`ANTHROPIC_API_KEY`, `voyage_api_key``VOYAGE_API_KEY`, etc. The validator
does NOT police which config keys you inherit — the agent is in the same
uid as the worker, so it's the agent's call.
**You can still use `env:`.** v0.36.5.0 does not forbid `env:{ ANYTHING }`.
If you have a reason to put a value in the row plaintext (a non-secret
correlation token, or a secret you know is OK to persist), pass it via
`env:`. Prefer `inherit:` when you want the value out of the row.
**Worker setup** (one-time, per host):
- `gbrain config set database_url postgresql://...` (or any other key you
want available for inherit)
- OR put the key in `~/.gbrain/config.json` directly
- OR set `GBRAIN_DATABASE_URL` / `DATABASE_URL` / per-provider env on the
worker process
If the worker can't resolve a requested name, the validator fail-fasts at
submit time with `gbrain config set <X>` hint. No more silent "No database
URL" failures in child stderr minutes after submission.
**Also new.** A `gbrain doctor` check `home_dir_in_worktree` warns if
`~/.gbrain/` lives inside a git worktree. A retroactive `~/.gbrain/.gitignore`
(single line `*`) is now laid down by every `saveConfig()` call AND by
`gbrain post-upgrade`, so existing users get coverage without re-running
`gbrain init`. Honest scope: the `.gitignore` covers casual `git add` but does
NOT cover already-tracked files, screenshots, backups, or `git add -f`.
**Strategy framing.** For agent-to-gbrain calls, the new canonical guide is
`docs/guides/agent-to-gbrain.md`. Two distinct surfaces: HTTP MCP via OAuth
for ops with MCP equivalents (`search`, `query`, `put_page`, etc.), and shell
job + `inherit:` for `localOnly` admin ops (`sync`, `embed`, `dream`,
`doctor`, etc.). Not a fallback hierarchy — pick by op.
**Errors to handle** (your agent submits shell jobs; surface these clearly):
| Error | What it means | Agent action |
|---|---|---|
| `shell: inherit must be an array of config-key names` | `inherit` wasn't an array. | Pass `"inherit": ["database_url", ...]`. |
| `shell: inherit entries must be non-empty strings` | Element was empty, non-string, or null. | Use snake_case config-key names. |
| `shell: inherit name "<X>" must match [a-z][a-z0-9_]*` | Name failed snake_case regex (uppercase, leading underscore, etc.). | Use the config-key verbatim — `database_url`, not `DATABASE_URL`. |
| `shell: inherit requested "<X>" but worker has no <X> configured` | Worker can't resolve the name from its `loadConfig()`. | Run `gbrain config set <X> <value>` on the worker host. |
-161
View File
@@ -1,161 +0,0 @@
# llama-server reranker (local) — Qwen3-Reranker, self-hosted ZE, any ZE-wire-shape provider
[`llama-server`](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md)
is the HTTP wrapper that ships with llama.cpp. With `--reranking`, it
exposes an OpenAI-style `POST /v1/rerank` endpoint that returns
`{results: [{index, relevance_score}]}` — exactly the wire shape gbrain
already drives for ZeroEntropy's hosted reranker. The
`llama-server-reranker` recipe (added in v0.40.6.1) routes
`gateway.rerank()` at your local llama.cpp instance instead of ZE.
Two flavors of "local" this recipe covers:
- **Qwen3-Reranker** (0.6B / 4B / 8B) — open-weight cross-encoder; pull
the GGUF from HuggingFace and serve.
- **Self-hosted ZeroEntropy** (`zerank-2`, `zerank-1-small`) — the
weights are on HuggingFace too. GGUF-convert them and serve them the
same way. **Quality is not guaranteed to match ZE-hosted:** GGUF
conversion + quantization + pooling/rank metadata + tokenizer special
tokens all affect scores. If you self-host ZE for production
retrieval, pin your own brain-relevant eval (
[docs/eval-bench.md](../eval-bench.md)) as a regression guard.
This recipe is the path override + recipe shape. Any provider whose
request/response wire matches ZE/llama.cpp can use it by just pointing
at a different base URL. Providers whose wire shape differs (Voyage uses
`top_k` not `top_n`, returns `data[]` not `results[]`) need a separate
recipe with adapter hooks — that lands in a follow-up plan.
## Setup
### 1. Build llama.cpp (or download a release)
```bash
# Clone and build (CPU only; add `-DGGML_CUDA=ON` for GPU)
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build --config Release -j
```
Pin a specific commit when you ship — `llama-server`'s path aliases
(`/rerank`, `/v1/rerank`, `/reranking`, `/v1/reranking`) have shifted
across releases. The recipe sends to `/v1/rerank`.
### 2. Pull a reranker GGUF
For Qwen3-Reranker-4B (quantized Q4_K_M is the sweet spot for CPU):
```bash
# Pick a quant level — Q4_K_M is the usual CPU sweet spot.
huggingface-cli download \
Qwen/Qwen3-Reranker-4B-GGUF qwen3-reranker-4b-q4_k_m.gguf \
--local-dir ./models
```
For self-hosted ZeroEntropy weights, find a community GGUF conversion
or convert from the HuggingFace weights yourself (out of scope of this
doc — see llama.cpp's `convert_hf_to_gguf.py`).
### 3. Launch llama-server with --reranking AND --alias
```bash
./build/bin/llama-server \
--model ./models/qwen3-reranker-4b-q4_k_m.gguf \
--alias qwen3-reranker-4b \
--reranking \
--port 8081
```
The `--alias` matters: without it, llama-server's `/v1/models` (and the
`model` field rerank requests echo) defaults to the full gguf file
path, which makes the gbrain config string ugly and brittle. With
`--alias qwen3-reranker-4b`, your config string is short and stable.
`--reranking` and `--embeddings` are mutually exclusive at server
launch. If you also run a local embedder via the
[`llama-server`](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md)
recipe, run two separate llama-server processes on two different ports
(typically 8080 for embeddings, 8081 for reranking — gbrain's defaults
match that convention).
### 4. Wire gbrain at your server
```bash
# Point gbrain at the llama.cpp host (skip if running locally on default port)
gbrain config set provider_base_urls.llama-server-reranker http://your-host:8081/v1
# Tell search to use this reranker
gbrain config set search.reranker.model llama-server-reranker:qwen3-reranker-4b
gbrain config set search.reranker.enabled true
```
The `qwen3-reranker-4b` after the colon is your `--alias` value from
step 3. Any string works as long as it matches your server's alias.
Env vars work too as an alternative to the config set above:
```bash
export LLAMA_SERVER_RERANKER_BASE_URL=http://your-host:8081/v1
# Optional: if you front llama-server with nginx + bearer auth
export LLAMA_SERVER_RERANKER_API_KEY=your-bearer-token
```
### 5. Verify
```bash
gbrain models doctor
# Expect: ✔ reranker_config llama-server-reranker:qwen3-reranker-4b ok
# ✔ reranker_config llama-server-reranker:qwen3-reranker-4b ok (reachability)
gbrain search "some query" --json | jq '.[].rerank_score'
# Expect: rerank_score on every row
```
If `gbrain models doctor` reports the reachability probe as `network`
status, two common causes:
1. The server is reachable but in embedding mode, not reranking mode.
`--reranking` and `--embeddings` are mutually exclusive at launch
— relaunch the right one.
2. The recipe path doesn't match what your llama.cpp version serves.
This recipe sends `/v1/rerank`; older llama.cpp installs may only
serve `/rerank`. Pin to a recent llama.cpp commit.
## Cold-start headroom
CPU-only first-call warmup on a 4B reranker can take 8-15 seconds. The
recipe declares `default_timeout_ms: 30000` so the first call after a
server restart doesn't fail-open silently. That value flows through
search-mode resolution unless you override it:
```bash
# Tighten or loosen per-search timeout (overrides recipe default):
gbrain config set search.reranker.timeout_ms 60000
```
Per-call overrides in `SearchOpts.reranker_timeout_ms` still win for
any single call.
## Budget caps + local rerank
The recipe declares `cost_per_1m_tokens_usd: 0` and registers under
`FREE_LOCAL_RERANK_PROVIDERS` in the budget tracker, so
`--max-cost`-bounded callers (autopilot loops, batch jobs) do NOT
hard-fail when configured for local rerank. Local rerank costs
electricity, not API tokens.
```bash
GBRAIN_MAX_USD=0.01 gbrain search "..." --reranker llama-server-reranker:qwen3-reranker-4b
# Works: rerank fires, recorded at $0, cumulative cap untouched.
```
## Fail-open contract preserved
`applyReranker` in `src/core/search/rerank.ts` still has the
fail-open posture: any error class (network, timeout, malformed
response) logs to `~/.gbrain/audit/rerank-failures-*.jsonl` and
returns the original RRF order unchanged. Search reliability beats
reranker quality. If your llama.cpp host goes down, your searches keep
working — they just stop ranking against the cross-encoder until you
restart the server.
-173
View File
@@ -1,173 +0,0 @@
# ZeroEntropy — zembed-1 + zerank-2
[ZeroEntropy](https://zeroentropy.dev) ships two specialized small models
for retrieval pipelines:
- **`zembed-1`** — multilingual embedding distilled from zerank-2.
Flexible Matryoshka dims (2560/1280/640/320/160/80/40), 32K context,
asymmetric `input_type: query|document` encoding. $0.025/1M tokens
(sale) / $0.05 regular.
- **`zerank-2`** — SOTA multilingual cross-encoder reranker.
$0.025/1M tokens (~50% cheaper than Cohere/Voyage rerankers).
Plus `zerank-1` and `zerank-1-small` for legacy / open-source needs.
Both land in gbrain v0.35.0.0 behind the openai-compatible recipe path,
alongside OpenAI and Voyage.
## Setup
1. Get an API key at
[dashboard.zeroentropy.dev](https://dashboard.zeroentropy.dev).
2. Export it:
```bash
export ZEROENTROPY_API_KEY=<your-key>
```
## Embedding switch — zembed-1
**Important:** `gbrain config set embedding_model …` is NOT a live
gateway switch. `embedding_model` and `embedding_dimensions` size the
schema and must be stable across engine connects, so they only resolve
from the **file plane** (`~/.gbrain/config.json`) and the **env plane**
(`GBRAIN_EMBEDDING_MODEL` / `GBRAIN_EMBEDDING_DIMENSIONS`). The DB plane
is intentionally ignored for these two keys (same posture as today's
Voyage setup).
### Option A — file plane (recommended for stable installs)
Edit `~/.gbrain/config.json`:
```json
{
"embedding_model": "zeroentropyai:zembed-1",
"embedding_dimensions": 2560
}
```
Valid dims: `2560` (default), `1280`, `640`, `320`, `160`, `80`, `40`.
Matryoshka-style — smaller trades quality for storage monotonically.
Pick the largest that fits your column width.
### Option B — env plane (CI / Docker)
```bash
export GBRAIN_EMBEDDING_MODEL=zeroentropyai:zembed-1
export GBRAIN_EMBEDDING_DIMENSIONS=2560
```
### Re-embed
Switching embedding models invalidates the vector index. Re-embed:
```bash
gbrain embed --stale --limit 50 # smoke a small batch
gbrain embed --stale # full re-embed
```
### Verify
```bash
gbrain models doctor --json | jq '.probes[] | select(.touchpoint=="embedding_config")'
```
Expected: `status: "ok"`. Invalid dims (e.g. `1024`, `1536`, `3072`)
surface as `status: "config"` with a paste-ready
`gbrain config set embedding_dimensions <one of 2560|1280|640|320|160|80|40>` fix hint.
## Reranker switch — zerank-2
The reranker is the bigger story: gbrain had no cross-encoder reranker
stage before v0.35.0.0. It slots between RRF dedup and token-budget
enforcement in hybrid search.
### Default-on with `tokenmax` mode
`tokenmax` mode now defaults `search.reranker.enabled = true` with
`zerank-2`. If you already use `tokenmax` AND have `ZEROENTROPY_API_KEY`
set, reranker fires automatically. Without the key, every rerank call
fails-open (audit-logged) and search returns RRF order — same UX as
before, just with an observable failure surfaced via `gbrain doctor`.
### Opt-in on `conservative` or `balanced` mode
```bash
gbrain config set search.reranker.enabled true
```
The override sits above the mode-bundle default; opt-out is one flip.
### Cost anchor
At 30 candidates × ~400 tokens/chunk × $0.025/1M = **~$0.0003/query**.
Rounding error against the `tokenmax + Opus` pairing's ~$700/mo at
single-user volume per the CLAUDE.md cost matrix.
### Verify
```bash
gbrain models doctor --json | jq '.probes[] | select(.touchpoint=="reranker_config")'
```
Two probes run for reranker:
- `reranker_config` (zero-network) — validates the model resolves
through the recipe registry and is in the touchpoint's allowlist.
- A reachability probe sends a minimal `{query: "probe", documents:
["probe"]}` rerank to verify auth + URL.
## Knobs reference
| Config key | Default | Notes |
|---|---|---|
| `search.reranker.enabled` | `true` for tokenmax, `false` for others | One-flip opt-in/out |
| `search.reranker.model` | `zeroentropyai:zerank-2` | Try `zerank-1` (older SOTA) or `zerank-1-small` (Apache-2.0 open) |
| `search.reranker.top_n_in` | `30` | Candidates sent to reranker (caps API spend) |
| `search.reranker.top_n_out` | `null` (no truncate) | Truncate reranked output to this many; `null` preserves full length |
| `search.reranker.timeout_ms` | `5000` | HTTP timeout; long stalls degrade UX worse than RRF fallback |
## Failure observability
Reranker is fail-open by construction: every error class (auth, rate-limit,
network, timeout, payload-too-large, unknown) returns the original RRF
order unchanged. Failures log to
`~/.gbrain/audit/rerank-failures-YYYY-Www.jsonl` (ISO-week rotation).
`gbrain doctor` reads the audit and surfaces:
- **auth failures** — any single one warns (config-time problem doctor's
own probe should have caught)
- **payload-too-large** — any single one warns (workload-mismatch signal)
- **transient (network/timeout/rate_limit)** — warns at >=5 in 7 days
Query text is SHA-256 hashed in the audit; never logged raw.
## Asymmetric input_type
ZE zembed-1 (and Voyage v3+) use asymmetric query/document encoding for
better retrieval. The gateway's `embedQuery(text)` companion threads
`input_type: 'query'`; standard `embed(texts)` defaults to
`'document'`. Hybrid search's two query-side embed sites use
`embedQuery()` automatically; all ingest paths use `embed()`.
Symmetric providers (OpenAI text-embedding-3, fixed-dim Voyage models)
ignore the field — no behavior change.
## Cache key versioning
v0.35.0.0 bumped `KNOBS_HASH_VERSION` 1 → 2 to fold reranker config into
the `query_cache.knobs_hash` column. During a rolling deploy:
- Expect a temporary cache hit-rate dip (~1 hour at default
`cache.ttl_seconds = 3600s`)
- Hot queries may briefly double their cache row count (one row per
version)
Both clear naturally; no operator action required.
## Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| `embedding_config` probe says invalid dim | Defaulting to 1536 (OpenAI default) | Set `embedding_dimensions` to one of 2560/1280/640/320/160/80/40 |
| `reranker_config` probe says model not in allowlist | Typo in `search.reranker.model` | Use one of `zerank-2` / `zerank-1` / `zerank-1-small` |
| `reranker_health` doctor warns about auth | `ZEROENTROPY_API_KEY` not set or invalid | Re-export the env var; `gbrain models doctor` to verify |
| `reranker_health` doctor warns about transient failures | Upstream flake or rate limit | Reranker fails open to RRF; check ZE status page if persistent |
| Cache hit rate dipped after upgrade | Expected during rolling deploy | Clears within `cache.ttl_seconds` (default 3600s) |
File diff suppressed because one or more lines are too long
-174
View File
@@ -1,174 +0,0 @@
# Why the hybrid + graph stack works
Vector search alone underdelivers on real personal-knowledge queries. This doc explains why gbrain layers four strategies together and how they compound.
## The four strategies in concert
1. **Vector (HNSW on pgvector)** — semantic similarity. Catches "who works on retrieval quality at YC?" → pages mentioning "Garry Tan + retrieval" even when the user never typed "YC".
2. **BM25 keyword** — lexical match. Catches names, exact phrases, code identifiers, anything where the user remembers the literal token. Survives the cases where vector search drifts into thematic neighbors.
3. **Reciprocal-rank fusion (RRF)** — merges vector + keyword rankings without weighting one over the other globally. Each strategy gets to vote.
4. **Knowledge graph traversal** — follows typed edges. Catches "what did Bob invest in this quarter?" by walking `bob ── invested_in ──> company ── dated ──> Q1`. Vector search can't see causal chains; the graph can.
## Why each one alone fails
**Vector only.** Returns chunks semantically close to the query. Misses any factual relationship not directly encoded in the embedding. "Companies in Garry's portfolio" returns essays about portfolios, not company pages.
**Keyword only (ripgrep-style).** Brittle to phrasing. "Who works on retrieval?" misses pages that say "search ranking" instead of "retrieval." Garbage on synonyms, near-misses, or paraphrases.
**Graph only.** Excellent at "neighbors of Alice" but blind to anything not yet linked. Sparse on fresh pages until backlinks accumulate.
**Hybrid (vector + keyword + RRF), no graph.** Decent at "what is X?" type queries. Fails on "what is Y's relationship to X?" — those are graph queries and no amount of embedding tuning recovers them.
## The benchmark
BrainBench (corpus + harness in the sibling [gbrain-evals](https://github.com/garrytan/gbrain-evals) repo) measures retrieval P@5, R@5, MRR, nDCG@5 on a 240-page Opus-generated rich-prose corpus.
| Strategy | P@5 | R@5 | Notes |
|---|---|---|---|
| ripgrep BM25 only | ~18 | ~75 | Lexical-only baseline |
| vector-only RAG | ~18 | ~80 | Standard RAG implementation |
| gbrain graph-disabled (hybrid + RRF, no graph traversal) | ~18 | ~85 | Hybrid alone |
| **gbrain default (full stack)** | **49.1** | **97.9** | Graph + extract-quality lift |
**+31 P@5 points** from the graph + extract quality work. The graph isn't a marginal feature; it's the load-bearing wall.
## Auto-link: why zero-LLM-call edge extraction works
Every `put_page` runs `extractEntityRefs` on the markdown body. It matches:
- Standard markdown links: `[Garry Tan](wiki/people/garry-tan)`
- Obsidian wikilinks: `[[wiki/people/garry-tan|Garry Tan]]`
- Typed-link blockquotes: `> **Convention:** see [path](path).`
Three regexes, zero LLM tokens, single SQL `addLinksBatch` call with `INSERT ... SELECT FROM jsonb_to_recordset(($1::jsonb)->'rows') JOIN pages ON CONFLICT DO NOTHING RETURNING 1` (free-text-safe; the prior `unnest(${arr}::text[])` form crashed on calendar/Zoom context per gbrain#1861). The graph grows on every write at near-zero cost. On a 17K-page brain, full graph extract completes in seconds.
Heuristic link-type inference (`attended`, `works_at`, `invested_in`, `founded`, `advises`) fires from surrounding sentence context — also LLM-free. Power users who want richer types add them via the typed-link blockquote convention.
## ZeroEntropy as reranker: 60% top-1 reshuffle
v0.36.0.0 ships ZeroEntropy's `zerank-2` as the default reranker (on for the `balanced` mode bundle). On a real-corpus benchmark across 20 queries, zerank-2 reshuffles **60% of top-1 results** after the hybrid + RRF + graph stack. That's the headline number.
The mechanical reason: hybrid ranking is locally optimal per strategy but globally suboptimal. A cross-encoder reranker reads the query + each candidate document jointly, with full attention. It catches the cases where the vector + keyword + graph signals all agreed on a document that's semantically related but topically wrong.
The cost: +150ms p50 latency, ~$0.025/M tokens. Disabled with `gbrain config set search.reranker.enabled false`. For agent loops that do downstream LLM work after retrieval, the latency is invisible.
## Source-aware ranking
Hybrid search applies a source-factor CASE expression at the SQL layer (lives in `src/core/search/sql-ranking.ts`). Curated content like `originals/`, `concepts/`, `writing/` outranks bulk content like `your-openclaw/chat/`, `daily/`, `media/x/`. Hard-exclude prefixes (`test/`, `attachments/`, `.raw/`) filter at retrieval, not post-rank.
`archive/` is deliberately NOT hard-excluded (issue #1777): it holds high-signal historical content users expect to find, so it is demoted (`0.5x` in `DEFAULT_SOURCE_BOOSTS`), not hidden. The demote is a prior applied in the outer SQL re-rank; the cross-encoder reranker (balanced/tokenmax modes) can still PROMOTE an archive page that survives the demote into the rerank candidate window — it is not an unconditional suppression. `gbrain doctor`'s `hidden_by_search_policy` check reports how many chunked pages remain hidden by the surviving exclude prefixes.
The boost map is configurable via `GBRAIN_SOURCE_BOOST` env var or per-call `SearchOpts.exclude_slug_prefixes`. Temporal queries (`detail: 'high'`) bypass the boost so chat pages re-surface for time-sensitive lookups.
## Named-thing retrieval (per-page pool + title + alias + evidence)
A brain organized around *chosen names* (Mingtang, Hall of Light) needs more than
embedding proximity. Four layers, added after the incident in
[`RETRIEVAL_MAXPOOL_INCIDENT.md`](./RETRIEVAL_MAXPOOL_INCIDENT.md):
- **Per-page max-pool**`searchVector` (both engines) collapses chunk-grain
candidates to the best chunk per page (`DISTINCT ON (slug)`) over the full
candidate set before the user `LIMIT`, via the shared `buildBestPerPagePoolCte`
in `sql-ranking.ts`. The vector side returns N distinct pages by best chunk,
not N chunks that collapse to fewer pages downstream.
- **Title-phrase boost** — when the normalized query is a contiguous token-run
inside `page.title` (or an exact full-title match), a floor-ratio-gated,
bounded multiplier fires (`applyTitleBoost`, `search.title_boost` knob). A
query that is a phrase from the title can't lose to a body chunk by luck.
- **Alias hop** — free-text `aliases:` frontmatter is projected into a
`page_aliases` table (separate from the `slug_aliases` wikilink redirect) and
consulted at query time: a full normalized-query match injects/boosts the
canonical page (`applyAliasHop`). The only layer that bridges true synonyms
with zero surface overlap ("Hall of Light" → the Mingtang page). Backfill
existing pages with `gbrain reindex --aliases`.
- **Evidence contract** — every result carries `evidence`
(`alias_hit | exact_title_match | high_vector_match | keyword_exact |
weak_semantic`) and `create_safety` (`exists | probable | unknown`). An agent
deciding "is this page already here, safe to NOT write a duplicate?" keys off
`create_safety`, not a raw blended score.
**Extraction quarantine lane (issue #160):** pages carrying the unverified
auto-extracted markers (frontmatter `provenance: auto-extracted` +
`status: unverified`, see `src/core/extraction-review.ts`) rank as ordinary
content — they are skipped by the compiled-truth fusion boost and by the
`people/`/`companies/` namespace source-boost, and every search result from
such a page carries `unverified: true` so agents can label the provenance.
Promote or reject them via `gbrain extraction-pending` / `gbrain
extraction-review`.
The `search` MCP/CLI op is **cheap-hybrid** (vector + keyword + RRF + pool +
title + alias, expansion off); `query` is the full-control variant. NamedThingBench
(`gbrain eval retrieval-quality`) gates these families on every PR. Diagnose a
specific miss with `gbrain search diagnose "<q>" --target <slug>`.
## Intent-aware query rewriting
`src/core/search/intent.ts` classifies queries into `entity`, `temporal`, `event`, or `general`. Each routes through different ranking knobs:
- **Entity** queries ("who works at X?") apply a higher graph-traversal weight.
- **Temporal** queries ("what happened last week?") bypass source-boost so chat/daily pages surface.
- **Event** queries ("Acme AI Series A") engage the timeline index.
- **General** queries hit the standard hybrid stack.
The classifier is deterministic (no LLM call). Wrong classification degrades gracefully — the hybrid stack still works without it.
## Multi-query expansion
For `detail: 'high'` searches, `src/core/search/expansion.ts` runs a Haiku-class LLM call to produce 2-3 query variants. Each variant runs through the full hybrid stack; results merge via RRF. Catches synonym misses without recall loss.
Expansion is opt-in per mode bundle (`tokenmax` on by default; `balanced` + `conservative` off). Default off in the cheap tiers because the LLM call adds ~$0.001/query and ~200ms — real money at scale.
## Putting it together
The full pipeline for a `query` op:
```
intent classify
expansion (if enabled)
hybrid search:
├── vector (HNSW on chunk embeddings)
├── keyword (BM25 via tsvector)
├── relational (v0.42.34.0: typed-edge recall arm — relational queries only)
├── source-aware re-rank (CASE in SQL)
└── RRF fusion → top 30
graph augment (typed-edge traversal from any seed)
reranker (zerank-2 cross-encoder, top 30 → reordered)
token-budget enforcement (per mode bundle)
deduplication (same slug, different chunks → keep best)
results
```
Each stage is testable in isolation. Each stage is replaceable. The whole pipeline is < 1ms of orchestration cost; the latency budget goes to the upstream HTTP calls (embedding, rerank) and the index scans.
## How to verify on your own brain
```bash
# Run the public LongMemEval benchmark
gbrain eval longmemeval datasets/longmemeval_s.jsonl
# Capture your own queries and replay against retrieval changes
export GBRAIN_CONTRIBUTOR_MODE=1
# ... use gbrain normally ...
gbrain eval export > before.ndjson
# ... change something ...
gbrain eval replay --against before.ndjson
# A/B retrieval strategies on a labeled fixture
gbrain eval --qrels labels.tsv --config balanced.json
```
Methodology + metric glossary in [`docs/eval/SEARCH_MODE_METHODOLOGY.md`](../eval/SEARCH_MODE_METHODOLOGY.md).
@@ -1,97 +0,0 @@
# Retrieval Incident: a chosen-name page was missed, and the fix
**Status:** Resolved (retrieval-cathedral wave). Supersedes the docs-only RFC in
closed PR #1616 — the diagnosis there was directionally right about the disease
but wrong on several mechanics; this is the corrected record + what shipped.
**Original author:** Garry Tan's OpenClaw. **Severity at the time:** High.
**Related:** [`RETRIEVAL.md`](./RETRIEVAL.md), [`../eval/METRIC_GLOSSARY.md`](../eval/METRIC_GLOSSARY.md).
---
## 1. What happened
The agent was asked to log that Garry "wants to build a Greek amphitheater." It
ran a retrieval for the concept, the canonical concept page (titled "...Indoor
Greek Amphitheater...") did **not** surface with enough confidence to be
recognized as the existing page, and the agent wrote a **duplicate stub** on top
of a fully-developed concept doc. Garry caught it: "It's in the brain. It's the
Hall of Light. Why did you forget?"
The page is *about* a Greek amphitheater — the phrase is in its title and first
sentence. A healthy index returns it at the top. It didn't.
## 2. The disease (the RFC got this right)
The brain is stored by **meaning and chosen name** (Mingtang, Hall of Light) but
was retrieved by **literal embedding proximity to a body chunk**, and the agent's
"is this already here?" decision keyed off a single fuzzy blended score. Three
retrieval gaps plus one contract gap produced the miss.
## 3. Verified ground truth (corrections to the RFC)
These were checked in code during the fix; several change the remedy:
1. **`gbrain search` was keyword-only**, not hybrid — so the RFC's cosine scores
(0.64/0.98) came from the hybrid `query`/MCP path the agent actually hit, not
`gbrain search`. The repro command in the RFC was mislabeled.
2. **`--mode` was never a CLI param** — mode resolves server-side from the
`search.mode` config key, which is why all three "modes" returned identical
results (the flag was silently dropped; `thorough` isn't a real mode).
3. **`hybridSearch` already max-pooled per page at the dedup layer.** So the
per-page max-pool fix's real win is *candidate-set page recall* (the vector
side returned N chunks that could collapse to fewer pages), and it is
necessary-but-not-sufficient: if a page's title chunk scores below a body
chunk on a 2-word query, or falls outside the candidate pool, pooling alone
doesn't rescue it.
4. **Frontmatter `aliases:` was dead to search** — stored in `pages.frontmatter`
JSONB, never consulted. `slug_aliases` is a *slug→slug* wikilink redirect, a
different concept.
## 4. The fix that shipped (four layers + a contract)
| Layer | Fixes | Where |
|---|---|---|
| **Per-page max-pool** (T1) | a page scored by its weakest chunk; vector page-recall | `searchVector` both engines, shared `buildBestPerPagePoolCte` |
| **Title-phrase boost** (T2) | query is a phrase in the title but matched a body chunk | `applyTitleBoost` (reads `page.title`), `title_boost` mode knob |
| **Alias hop** (T3) | true synonyms with zero surface overlap ("Hall of Light" → Mingtang) | `page_aliases` table, `applyAliasHop`, ingest projection + `reindex --aliases` backfill |
| **Evidence contract** (T4) | the agent keyed "don't duplicate" off a fuzzy score | `evidence` + `create_safety` on every result; the agent keys off `create_safety='exists'`, not a threshold |
Plus: `gbrain search "<text>"` is now cheap-hybrid (the obvious verb gives the
good path); `modes/stats/tune` stay subcommands; `--mode` works per-call for
local callers; rank-1 score drift telemetry; and **NamedThingBench**, a CI gate
that hard-gates the families that ARE this incident.
## 5. How to confirm / triage a recurrence
```
# Which layer surfaces (or misses) the target page?
gbrain search diagnose "Greek amphitheater" --target projects/new-greek-theater/concept_v0
# Backfill aliases for existing pages whose frontmatter predates the alias layer:
gbrain reindex --aliases
# Watch retrieval quality over time (a downward avg rank-1 score = regressing):
gbrain search stats --days 30
# The gate that prevents silent reintroduction:
gbrain eval retrieval-quality test/fixtures/retrieval-quality/namedthing.jsonl
```
For a page to be reliably found by its chosen name, give it `aliases:` frontmatter:
```yaml
---
title: The Mingtang — Indoor Greek Amphitheater
aliases:
- Hall of Light
- 明堂
---
```
## 6. The discipline this teaches
A benchmark that scores 97.9 R@5 while production returns a flagship page at 0.64
means the benchmark and the shipped path diverged. NamedThingBench runs the same
families through the real pipeline on every PR, and the evidence contract means
the agent's duplicate-or-not decision is grounded in *why* a page matched, not a
number that was never a calibrated probability.
-242
View File
@@ -1,242 +0,0 @@
# Brains and Sources — the mental model
GBrain has two orthogonal axes for organizing knowledge. Users and agents both
need to understand both of them, or queries misroute silently.
**TL;DR:**
- A **brain** is a database. You can have many.
- A **source** is a named repo of content *inside* a brain. One brain can hold many.
- `--brain <id>` picks WHICH DATABASE.
- `--source <id>` picks WHICH REPO WITHIN that database.
- They're independent. You can target any combination.
---
## The two axes
### Brains (the DB axis)
A **brain** is one database — PGLite file, self-hosted Postgres, or Supabase.
Each brain has:
- Its own `pages` table, `chunks` table, `embeddings`, etc.
- Its own OAuth surface if served over HTTP MCP (v0.19+, PR 2).
- Its own separate lifecycle, backup, access control.
Brains are enumerated by:
- **host** — your default brain, configured in `~/.gbrain/config.json`.
- **mounts** — additional brains registered in `~/.gbrain/mounts.json` via
`gbrain mounts add <id>` (v0.19+).
Routing: `--brain <id>`, `GBRAIN_BRAIN_ID`, `.gbrain-mount` dotfile, or
longest-path match against registered mount paths. Falls back to `host`.
### Sources (the repo axis, v0.18.0+)
A **source** is a named content repo *inside* one brain. Every `pages` row
carries a `source_id`. Slugs are unique per source, not globally.
Example: in one brain, the slug `topics/ai` can exist under `source=wiki`
AND under `source=gstack` — they're different pages.
Routing: `--source <id>`, `GBRAIN_SOURCE`, `.gbrain-source` dotfile, or
registered `local_path` match in the `sources` table.
### When does each axis move?
| You want to | Adjust |
|---|---|
| Work in a different repo within the same brain (wiki → gstack notes) | `--source` |
| Query a team-published brain that isn't yours | `--brain` |
| Isolate a topic so it never leaks into personal search | `--source` with `federated=false` |
| Share a brain with teammates | `--brain` (mount the team brain) |
| Add a new repo to your personal brain | `--source` via `gbrain sources add` |
| Add a team brain | `--brain` via `gbrain mounts add` |
**Rule of thumb:** if the data owner changes, it's a brain boundary. If the
data owner stays the same but the topic/repo changes, it's a source boundary.
---
## Topology: a single-person developer
Simplest case. One brain, one source.
```
┌─────────────────────────────────────────┐
│ host brain (~/.gbrain) │
│ ├── source: default (federated=true) │
│ │ └── all pages │
└─────────────────────────────────────────┘
```
`gbrain query "retry budgets"` finds everything. No `--brain`, no `--source`
needed.
---
## Topology: a personal brain with multiple repos
You maintain several codebases or writing streams. Each is its own source
inside one brain. Cross-source search is on by default so a query about
"caching" returns hits from every repo.
```
┌──────────────────────────────────────────────┐
│ host brain (~/.gbrain) │
│ ├── source: wiki (federated=true) │
│ │ └── personal notes, people, companies │
│ ├── source: gstack (federated=true) │
│ │ └── gstack plans, learnings │
│ ├── source: openclaw (federated=true) │
│ │ └── openclaw docs, memos │
│ └── source: essays (federated=false) │
│ └── draft essays, isolated on purpose │
└──────────────────────────────────────────────┘
```
Inside `~/openclaw/` the `.gbrain-source` dotfile pins every command to
`source=openclaw`. Inside `~/gstack/` the dotfile pins to `source=gstack`.
Everything still targets one DB.
Use this topology when:
- You own all the content.
- You want cross-repo search to just work.
- You don't need to share any of it with someone who isn't you.
---
## Topology: personal brain + one team brain
You're on a team that publishes a shared brain. Your personal brain stays
as-is; you mount the team brain alongside it.
```
┌──────────────────────────────────────────────┐
│ host brain (~/.gbrain) — YOUR personal DB │
│ ├── source: wiki │
│ ├── source: gstack │
│ └── ... │
└──────────────────────────────────────────────┘
┌──────────────────────────────────────────────┐
│ mount: media-team │
│ path: ~/team-brains/media │
│ engine: postgres (team's Supabase) │
│ └── sources: wiki, raw, enriched │
└──────────────────────────────────────────────┘
```
`gbrain query "X"` (no flags) → runs against host (your personal brain).
`gbrain query "X" --brain media-team` → runs against the team's DB.
Inside `~/team-brains/media/` a `.gbrain-mount` dotfile pins brain to
`media-team` automatically.
Use this topology when:
- You're on a team and someone publishes a brain the team subscribes to.
- You need data isolation between work and personal.
- Different teams/orgs own different brains.
---
## Topology: a CEO-class user with multiple team memberships
You're senior enough to sit across multiple teams. You maintain your personal
brain (with N sources inside) AND mount several work team brains. Each team
brain is itself a multi-source brain in the v0.18.0 sense — organized
internally however the team owner chose.
```
┌──────────────────────────────────────────────┐
│ host brain — YOUR personal DB │
│ ├── source: wiki │
│ ├── source: essays │
│ ├── source: gstack │
│ └── source: openclaw │
└──────────────────────────────────────────────┘
┌──────────────────────────────────────────────┐
│ mount: media-team (your media team's brain) │
│ └── sources: wiki, pipeline, enriched │
└──────────────────────────────────────────────┘
┌──────────────────────────────────────────────┐
│ mount: policy-team (your policy team's) │
│ └── sources: wiki, research, letters │
└──────────────────────────────────────────────┘
┌──────────────────────────────────────────────┐
│ mount: portfolio (another team's) │
│ └── sources: companies, deals, diligence │
└──────────────────────────────────────────────┘
```
Inside each team's checkout, a `.gbrain-mount` dotfile pins the brain. Inside
a specific subdirectory, a `.gbrain-source` dotfile pins the source. So `cd
~/team-brains/policy/research && gbrain query "X"` targets
`brain=policy-team, source=research` with zero flags.
Use this topology when:
- You cross-cut multiple teams.
- Each team owns its own brain with its own access policy.
- You need latent-space federation (agent decides when to query across
brains), not SQL federation.
Cross-brain queries are **not deterministic** in v0.19. The agent sees the
brain list and re-queries as needed. That's the feature — it keeps debugging
sane and access control clean.
---
## Resolution precedence (one page to remember)
```
WHICH BRAIN (DB)? WHICH SOURCE (repo in DB)?
1. --brain <id> 1. --source <id>
2. GBRAIN_BRAIN_ID env 2. GBRAIN_SOURCE env
3. .gbrain-mount dotfile 3. .gbrain-source dotfile
4. longest-prefix mount path match 4. longest-prefix source path match
5. (reserved: brains.default v2) 5. sources.default config
6. fallback: 'host' 6. fallback: 'default'
```
Both axes follow the same layered pattern on purpose. If you know one, you
know the other.
---
## For agents reading this
- Default assumption when the user asks a question: start in the current
brain (resolved via the precedence above). Don't jump brains without a
reason.
- If the user asks a question that crosses topic areas a team might own
(e.g. "what did Team X decide last week?"), the right move is to *query
the team's brain explicitly* rather than searching host with "team x".
- Cross-brain federation is YOUR JOB, not the DB's. You have the brain list
(`gbrain mounts list`). You decide when to fan out. You synthesize
findings. You cite `brain:source:slug`.
- When writing a page, respect the brain boundary. A fact about a team's
work belongs in the team's brain, not in the user's personal brain. Ask
before writing cross-brain.
- See `skills/conventions/brain-routing.md` for the full decision table.
## For users reading this
- **Default path:** set up your personal brain (`gbrain init`), add a source
per repo you care about (`gbrain sources add gstack --path ~/gstack`).
You'll almost never need `--brain`.
- **When a team publishes a brain:** `gbrain mounts add <team-id> --path
<clone> --db-url <url>` and the `.gbrain-mount` dotfile in that checkout
routes queries there automatically.
- **When you are the CEO-class user with multiple team memberships:** mount
each team brain. Trust the resolver — inside a team's directory the
dotfile picks the brain, inside a subdirectory the dotfile picks the
source. The flags are for when you want to query across the boundary
deliberately.
## Further reading
- v0.18.0 CHANGELOG — introduced `sources` primitive.
- v0.19.0 CHANGELOG (TBD after PR 0+1+2 ship) — introduces `mounts`.
- `docs/mounts/publishing-a-team-brain.md` (PR 2) — how to be the brain
publisher, not just the subscriber.
@@ -1,215 +0,0 @@
# Calibration Quality Gate — Falsifiability Filter + Category Classification
> **Historical context.** This is the source spec absorbed from PR #1191 into
> two waves of implementation:
>
> - **v0.37.2.0 hotfix** (this release): widens the `takes_resolution_consistency`
> CHECK constraint to accept `quality='unresolvable'` as a 4th valid state.
> Unblocks the production grading script. Adds `unresolvable_count` +
> `unresolvable_rate` to `TakesScorecard` as sibling fields (preserves
> v0.36.1.0 historical comparison semantics). Migration renumbered v74→v79→v80
> during successive master merges — v0.37.0.0's autonomous-remediation wave
> claimed v68-v78, then v0.37.1.0 (brainstorm/lsd) claimed v79.
> - **Follow-up minor** (forthcoming): falsifiability + category extraction at
> `propose_takes`, SQL-side grade gate, per-category calibration scorecards,
> pg_trgm-based proposal dedup. Wave-blocking on cat15 F1 re-validation
> against the v0.36.1.0 fixtures.
>
> Preserved here per the hotfix plan's PR #1191 close protocol so the
> production context (96K-page brain, 6.8% falsifiability rate, category
> breakdown) doesn't get lost in the CHANGELOG → release-notes condensation.
## Problem
v0.36.1.0 ships `propose_takes`, `grade_takes`, and `calibration_profile` as a
connected pipeline: extract claims → grade them against outcomes → build a
calibration profile showing systematic biases.
In production on a 96K-page brain with 36K takes across 6,239 holders, the
grade_takes phase produces noisy results:
- **6.8% falsifiability rate**: Of 500 candidate takes (weight ≥ 0.7), only 34
passed an LLM falsifiability filter. The other 93% were philosophical beliefs,
present-state observations, advice, logistics, or vague vibes.
- **50% unresolvable**: Even after filtering, 17/34 predictions couldn't be
graded because evidence was insufficient or the claim was too ambiguous.
- **Duplicates**: Same claim from the same page extracted multiple times with
slightly different wording.
The root cause: `propose_takes` extracts everything that looks like a belief or
assertion. That's correct for the *takes* table (epistemological layer), but
`grade_takes` needs a much narrower subset: **falsifiable predictions about
future outcomes** where we can check what actually happened.
### Example classifications from production testing
**Genuine predictions (grade-worthy):**
- "X will reach $1M ARR very soon" → company_outcome
- "X is going to leave Y" → people_move
- "AI will make authentic authorship more important" → technology
- "X was convinced Y would win the Z market" → market_call
**Not predictions (should skip grading):**
- "Desire is mimetic" → philosophical belief
- "X should charge 10x more" → advice
- "Return from Toronto on Monday" → logistics
- "Something is going to happen there" → vague/unfalsifiable
- "X is growing very quickly" → present-state observation
## Solution
### 1. Falsifiability score at extraction time
Add a `falsifiability` column to the `takes` table (real, 0.01.0, nullable,
default null). `propose_takes` sets this during extraction using the same LLM
call that already produces the take — one additional field in the JSON schema.
```sql
ALTER TABLE takes ADD COLUMN IF NOT EXISTS falsifiability real;
ALTER TABLE takes ADD COLUMN IF NOT EXISTS falsifiability_category text;
```
The LLM prompt addition (appended to the existing propose_takes extraction prompt):
```
For each claim, also assess:
- falsifiability (0.0-1.0): Can this claim be checked against future reality?
1.0 = specific, measurable, time-bounded prediction about an outcome
0.5 = directional claim that's partially checkable
0.0 = philosophical belief, advice, observation, or unfalsifiable assertion
- falsifiability_category: one of
company_outcome | fundraising | technology | people_move | market_call | other_prediction | not_prediction
```
Cost: ~0 incremental tokens (the claim is already being extracted; this adds
two fields to the JSON output schema).
### 2. Grade gate in `grade_takes`
Before attempting grading, filter:
```typescript
const gradeable = candidates.filter(t =>
t.falsifiability !== null && t.falsifiability >= 0.7
&& t.falsifiability_category !== 'not_prediction'
);
```
This reduces grading volume by ~93% in production, which means:
- LLM cost for grading drops proportionally
- Evidence retrieval load drops (each grade attempt triggers hybrid search)
- Calibration profiles are built on real predictions, not noise
### 3. Deduplication at extraction
`propose_takes` should check for near-duplicate claims before inserting:
```typescript
// Before inserting a new take, check if a similar claim exists
// for the same holder from the same page
const existing = await engine.sql`
SELECT id, claim FROM takes
WHERE holder = ${holder}
AND page_id = ${pageId}
AND similarity(claim, ${newClaim}) > 0.8
LIMIT 1
`;
if (existing.length > 0) {
// Skip — near-duplicate
continue;
}
```
Requires `pg_trgm` extension (already available on most Postgres installations).
Falls back gracefully: if `similarity()` isn't available, skip the dedup check.
### 4. Category-aware calibration profiles
The `calibration_profile` phase can now group resolved takes by
`falsifiability_category` to produce per-domain scorecards:
```
"Your company_outcome calls are 73% accurate.
Your people_move calls are 90% accurate.
Your technology calls are 60% accurate — you tend to be ~18 months early."
```
This is the tweetable output: a calibration profile that says "here's how you're
systematically right and wrong by category."
## Schema Changes
```sql
-- Migration: add falsifiability columns to takes
ALTER TABLE takes ADD COLUMN IF NOT EXISTS falsifiability real;
ALTER TABLE takes ADD COLUMN IF NOT EXISTS falsifiability_category text;
-- Index for grade_takes filter
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_takes_falsifiability
ON takes (falsifiability)
WHERE falsifiability IS NOT NULL AND falsifiability >= 0.7;
-- Optional: pg_trgm for dedup (CREATE EXTENSION IF NOT EXISTS pg_trgm;)
```
## Evidence Retrieval (v0.36.1.0 → v0.37 enhancement)
The current `grade_takes` evidence retriever returns a stub placeholder. In
production testing, we wired real evidence retrieval via `gbrain query` (hybrid
search). The pattern that works:
1. Extract the core claim from the take (first 150 chars)
2. Run `engine.query(claim)` to get relevant pages
3. Filter to pages updated AFTER the take's `since_date` (evidence must be newer)
4. Pass top-5 chunks as the evidence block to the judge
This should replace the stub in the `evidenceRetriever` injection point.
## Production Results
After implementing the falsifiability filter (as a pre-processing step outside
the cycle):
| Metric | Before (v2, no filter) | After (v3, with filter) |
|--------|----------------------|----------------------|
| Candidates evaluated | 50 | 34 (from 500 screened) |
| Falsifiable predictions | ~19 (38%) | 34 (100%) |
| Correct | 10 (52.6% of resolvable) | 10 (58.8% of resolvable) |
| Incorrect | 5 (26.3%) | 2 (11.8%) |
| Partial | 4 (21.1%) | 5 (29.4%) |
| Unresolvable | 31 (62%) | 17 (50%) |
| Category breakdown | N/A | people_move:13, company_outcome:11, technology:4, market_call:2 |
Key improvement: **the false positive rate dropped from 62% noise to 0% noise**
in the gradeable set. The remaining 50% unresolvable rate is genuine — those
predictions are about outcomes that haven't happened yet or where the brain
lacks evidence. That's correct behavior, not noise.
## Files to Change
1. **`src/core/cycle/propose-takes.ts`** — Add falsifiability + category to
extraction prompt and output schema
2. **`src/core/cycle/grade-takes.ts`** — Add falsifiability gate before grading;
wire real evidence retrieval
3. **`src/core/cycle/calibration-profile.ts`** — Group scorecards by category
4. **`src/core/engine.ts`** — Add `similarity()` helper for dedup (graceful
fallback)
5. **New migration** — Add columns + index
## Testing
- Unit test: falsifiability classifier on 20 known-good and 20 known-noise takes
- Unit test: dedup correctly merges near-identical claims
- Unit test: grade gate filters below threshold
- Integration test: full cycle with falsifiability → grade → profile pipeline
- Regression test: existing takes without falsifiability score are not broken
(null falsifiability = ungated, backward compatible)
## Backward Compatibility
- `falsifiability` defaults to null. Existing takes are unaffected.
- `grade_takes` with null falsifiability: configurable behavior. Default:
grade all (backward compat). Operator can set
`cycle.grade_takes.require_falsifiability: true` to gate.
- Category column is purely additive.
- Dedup is opt-in: `cycle.propose_takes.dedup.enabled: true`.
@@ -1,146 +0,0 @@
# Conversation parser patterns
The conversation parser turns exported chat and meeting transcripts into a
common message stream without requiring an LLM call for known formats. This
document describes the built-in pattern contract and the checks required when
adding or changing a format.
## Data flow
`parseConversation` uses this sequence:
1. Resolve the page date and timezone context.
2. Score every enabled built-in and user pattern against the first ten
non-blank lines.
3. Re-score the full body when the head score is inconclusive, or when a broad
pattern explicitly requires full-body scoring.
4. Reject the winner when its acceptance score is below the false-positive
floor.
5. Apply the winning pattern to every line and attach continuation lines to the
preceding message.
6. Optionally run LLM polish or fallback when those features are enabled.
Pattern order is only a tie-breaker. A new regex must be structurally distinct
from neighboring formats; moving it earlier in the registry is not a valid
non-shadowing strategy.
## Built-in pattern contract
Every `PatternEntry` in `builtins.ts` declares:
- A stable, kebab-case `id`.
- A hand-vetted line regex and explicit capture-group indexes.
- Where the date comes from and how the time is represented.
- A timezone policy.
- Whether the format supports multi-line message bodies.
- Positive and negative samples that run during module initialization.
- A documentation pointer describing the source format.
The registry refuses to load when a positive sample stops matching, a negative
sample starts matching, or a capture map becomes invalid. This catches local
regex mistakes before extraction can silently produce empty conversations.
### Date and timezone rules
Formats with an inline date should capture it from each message. Time-only
formats use an explicit caller fallback first, then the page frontmatter date,
then the page effective date. If none is available, the parser uses
`1970-01-01` so the missing date remains visible instead of inventing a current
date.
Time-only formats normally use `utc_assumed_with_warn`. The parser constructs a
UTC timestamp and returns a timezone warning when the page does not provide a
timezone. A new pattern should not imply local-time precision that the source
format does not contain.
### Multi-line messages
An anchor regex identifies the first line of a message. Subsequent non-anchor
lines are appended to that message until another anchor appears. Set
`multi_line: true` when continuation content is part of the documented format,
such as Markdown bullets, blockquotes, or an exported message body on the next
line.
Tests for a multi-line format should assert the complete message text, including
newlines. A message-count assertion alone will not detect lost bullets or a
continuation attached to the wrong speaker.
### Scoring and false positives
The score compares matched anchors with the pattern's relevant candidate lines.
The first pass uses the head of the page for speed. Low-confidence pages are
re-scored across the full body before the parser accepts a winner.
Multi-line formats may opt into `score_continuations_as_body` when their anchor
grammar is distinctive. Candidate-only scoring activates only after two anchors
match, or when the first non-blank line is an anchor. This evidence threshold
lets a single long message keep its continuation body without turning one stray
anchor in a prose page into a conversation. Candidate anchor lines that fail the
full regex still lower the score. Other patterns continue to use all non-blank
lines in their density score.
Use `score_full_body: true` for a broad grammar that also occurs in ordinary
prose. For example, `**Label:** text` can be either a transcript line or a bold
label in meeting notes. Narrow formats with a timestamp and a distinctive
separator generally do not need this override.
`quick_reject` is a performance hint, not an acceptance rule. It should cheaply
exclude obviously unrelated lines while admitting every string accepted by the
main regex.
## Normalized Slack Markdown
The `bold-time-dash` pattern parses message anchors shaped like:
```text
**Alice Example** 09:15 — first message
- supporting detail
**Bob Example** 09:18 — second message
```
Its grammar is:
```text
**speaker** H:MM <dash> text
```
where:
- `H:MM` is a valid 24-hour time from `0:00` through `23:59`.
- `<dash>` may be an em dash (`—`), en dash (``), or ASCII hyphen (`-`).
- The date comes from the resolved page date context.
- Continuation lines belong to the preceding message.
- The captured clock value is emitted with `Z`. Timezone metadata suppresses
the missing-timezone warning but is not currently used for IANA conversion.
The required time and dash distinguish it from all existing bold-speaker
formats:
- `**Speaker** (09:15): text` uses `bold-paren-time`.
- `**Speaker** (9:15 AM): text` uses `bold-paren-time-12h`.
- `**Speaker:** text` uses `bold-name-no-time`.
- `**Speaker** (2026-04-09 9:15 AM): text` uses `imessage-slack`.
Keeping these examples in both `test_negative` and parser regression tests makes
the non-shadowing contract executable.
## Adding a built-in format
1. Collect multiple anonymized examples, including separator and timestamp
variants that occur in the same export family.
2. Choose the narrowest grammar that represents the format. Constrain numeric
fields such as hours and minutes when possible.
3. Add at least two positive module-load samples and negative samples for every
neighboring pattern that could plausibly overlap.
4. Add parser tests that verify speakers, timestamps, text, continuation
handling, and non-shadowing behavior.
5. Add a dedicated JSONL fixture and include the same cases in
`test/fixtures/conversation-formats/all.jsonl`.
6. Run the focused parser tests and the fixture evaluator.
7. Run the repository verification and full test suites before submission.
8. Update `docs/architecture/KEY_FILES.md` when the registry count or supported
format inventory changes.
Use generic fixture identities such as `Alice Example`, `Bob Example`, and
`Summary Bot`. Never copy real transcript names or private content into source,
tests, documentation, commits, or pull-request descriptions.
@@ -1,200 +0,0 @@
# Frontmatter scan: DB-backed incremental state (Phase 2 design sketch)
**Status:** Designed, not built. Captured here as the starting point for the
follow-up PR after v0.38.2.0.
## Why this exists
v0.38.2.0 fixed the load-bearing bug class that caused `gbrain doctor` to
hang on large brains: the disk walker descended into `node_modules/`, `.git/`,
and other vendor trees on every tick. After that fix doctor completes in
seconds on most brains, and bounded wall-clock (default 30s, with honest
partial-state surfacing) on any brain.
But the steady-state cost of `frontmatter_integrity` is still O(N) in real
syncable pages: every doctor tick re-walks the filesystem and re-parses
every `.md` file. For users with 200K+ pages the steady-state cost is in
the seconds even after Fix 1. For sub-second steady-state doctor (the
right shape for cron-monitored health checks), the scan needs to become
incremental.
This document captures the Phase 2 design before the follow-up PR starts,
so the implementer doesn't have to re-derive it.
## Goal
Doctor's `frontmatter_integrity` check completes in O(1) SQL queries
regardless of brain size, with the same per-source breakdown and partial-
state semantics as v0.38.2.0's bounded-walk approach. Incremental refresh
runs as a sync-side write + an autopilot cycle phase, so the steady-state
work is amortized across the workflow that already touches each file.
## Schema
New table:
```sql
CREATE TABLE frontmatter_scan_state (
source_id TEXT NOT NULL REFERENCES sources(id) ON DELETE CASCADE,
path TEXT NOT NULL, -- relative to source.local_path
mtime_ms BIGINT NOT NULL,
content_hash TEXT NOT NULL, -- sha256 of file content at scan time
codes JSONB NOT NULL DEFAULT '[]'::jsonb, -- ParseValidationCode[]
last_scanned_at TIMESTAMPTZ NOT NULL DEFAULT now(),
PRIMARY KEY (source_id, path)
);
CREATE INDEX frontmatter_scan_state_has_issues_idx
ON frontmatter_scan_state (source_id)
WHERE codes != '[]'::jsonb;
```
Why these columns:
- `mtime_ms` + `content_hash`: incremental check picks one. mtime is faster
(no read); content_hash is the truth (defeats touch-without-change cases).
The incremental walker uses mtime as a fast gate and content_hash as the
fallback when mtime suggests change.
- `codes` JSONB: per-row error code list, NULL/`[]` means clean. Doctor
aggregates with `jsonb_array_length(codes) > 0`.
- Partial index on `WHERE codes != '[]'::jsonb`: doctor's aggregate query
only walks rows with issues, which is a small fraction of pages.
This follows the canonical `applyForwardReferenceBootstrap` pattern in
`src/core/pglite-engine.ts` (and `postgres-engine.ts`) — the new column /
table additions go into the bootstrap probe set per CLAUDE.md so old brains
walking forward through the schema chain don't wedge on the table not
existing.
## Migration shape
```ts
// src/core/migrate.ts — append after the v80 entry
const migrations = [
// ...existing v1-v80...
{
version: 81,
name: 'frontmatter_scan_state',
sql: `
CREATE TABLE IF NOT EXISTS frontmatter_scan_state (...);
CREATE INDEX IF NOT EXISTS frontmatter_scan_state_has_issues_idx ...;
`,
},
];
```
Plus the forward-reference probe entries in both engine bootstraps. Plus
the `REQUIRED_BOOTSTRAP_COVERAGE` extension in
`test/schema-bootstrap-coverage.test.ts`.
## Writers
Two paths write rows:
1. **Sync-side write** (canonical). `src/core/sync.ts:performSync` already
parses every file it touches. After the existing `parseMarkdown` call,
`UPSERT` into `frontmatter_scan_state` with the file's path / mtime /
content_hash / codes. Cost: one row per file synced. Zero extra parse
work — the parse already happened.
2. **Incremental scan** (`gbrain frontmatter scan --incremental`). Walks
the disk via `walkBrainTree`, for each file checks `mtime > last_scanned_at`
OR `content_hash != stored`, only re-parses changed files. Most ticks:
zero work after the first full backfill. Also exposed as an autopilot
cycle phase (`frontmatter_scan`) so it runs alongside the other periodic
maintenance phases.
The incremental walker handles two cases sync misses:
- Files edited outside sync (user opens an editor, saves, never `git
commit`s).
- Sources whose `local_path` isn't a git repo (sync only sees git-touched
files).
## Doctor reader
```ts
// src/commands/doctor.ts:frontmatter_integrity (Phase 2 shape)
const rows = await engine.executeRaw<{ source_id: string; issues: number }>(
`SELECT source_id, count(*) FILTER (WHERE jsonb_array_length(codes) > 0)::int AS issues
FROM frontmatter_scan_state
GROUP BY source_id`,
);
```
One SQL query, constant time regardless of brain size. The partial-state
surfacing from v0.38.2.0 stays — when `frontmatter_scan_state` is stale
(no rows for a registered source, or `last_scanned_at` >24h old for any
source), doctor warns about freshness rather than reporting potentially-
stale data as authoritative.
## Sequencing concerns
1. **First-ever scan.** A fresh upgrade has no rows in
`frontmatter_scan_state`. Two options:
- Lazy: doctor reports "no scan state yet; run `gbrain frontmatter scan
--incremental` once" (operator-driven).
- Eager: the migration that creates the table also enqueues an autopilot
cycle job to do the first full scan.
Recommendation: lazy, with a clear hint. The autopilot path is heavier
surface (must add the new `frontmatter_scan` phase to the existing
cycle.ts machinery + the doctor-routed background job system).
2. **Source archival / deletion.** `frontmatter_scan_state` has `ON DELETE
CASCADE` on `sources(id)`, so soft-delete + 72h TTL + purge already
clean it up. No additional logic needed.
3. **Path renames inside a source.** Sync would `DELETE` the old row by
path (via a periodic reconcile step) and `INSERT` the new row. Without
that step, the table accumulates stale path rows. Either:
- A reconcile step in the incremental scanner: any path-row not seen
during the walk gets deleted.
- Or: doctor reports "N stale rows in frontmatter_scan_state" as a
freshness signal, with `gbrain frontmatter scan --reconcile` as the
remediation.
## Cost estimate
- One UPSERT per file synced. Negligible vs the parse + DB write that sync
already does.
- Incremental refresh runtime: dominated by mtime stats. ~ms per 1000 files
on SSD.
- Doctor read: one indexed SQL query. Sub-100ms on any brain size.
## What this design deliberately does NOT do
- **Replace v0.38.2.0's bounded-walk safety net.** Phase 2 makes the
steady-state cheap, but the disk walker (with its deadline check) stays
as the source-of-truth fallback for sources whose scan state is missing
or stale. Belt-and-suspenders.
- **Introduce a separate frontmatter validation rule set.** Reuses
`parseMarkdown(..., {validate: true})` and the existing
`ParseValidationCode` enum. Single source of truth.
- **Add a new background daemon.** Wires into the existing
`autopilot-cycle` Minion handler as a new phase, alongside sync /
extract / embed / etc.
## Open questions for the implementer
1. **Path normalization.** `pages.source_path` and the disk walker's
relative path computation are similar but not identical (slashes,
leading `./`, etc.). The incremental scanner needs to match what sync
stores so UPSERTs key correctly. Audit before writing.
2. **Soft-delete interaction.** A page that gets soft-deleted in the DB
(v0.26.5) still has a file on disk. Should the incremental scan
continue to track its frontmatter state? Probably yes (so a future
`restore_page` doesn't surprise with stale frontmatter), but worth
confirming with the soft-delete owner.
3. **Two-phase rollout.** Land the table + writes first, let it backfill
for a release cycle, then switch the doctor reader. Avoids the
"Phase 2 ships but the table is empty" case where doctor regresses to
reporting "no scan state."
## TODO file entry
```
- [ ] Implement Phase 2: DB-backed frontmatter scan state.
Design lives at docs/architecture/frontmatter-scan-incremental.md.
Schema migration v81 + sync-side UPSERT + incremental scan command
+ autopilot cycle phase + doctor reader. Two-phase rollout: ship
table + writes first; flip the reader one release later.
```
-152
View File
@@ -1,152 +0,0 @@
# Lens packs (v0.41.2.0)
Four bundled schema packs that turn the gbrain dream cycle into a multi-lens
brain. Activate one with `gbrain config set schema_pack <name>` and the cycle
picks up the pack's declared phases on the next `gbrain dream` run.
## The four packs
```
gbrain-base (shipped v0.38)
│ extends
┌──────────────┼──────────────────────┐
│ │ │
gbrain-creator gbrain-investor gbrain-engineer
(atom + concept (deal/thesis/ (learning bridge
lifecycle) bet_resolution) for gstack)
│ │ │
└──────────────┼───────────────────────┘
│ extends + borrow chain
gbrain-everything (meta-pack)
one brain, three lenses active
```
### gbrain-creator
Atom + concept content-creator lifecycle. Drives two cycle phases:
- `extract_atoms` — per source, Haiku extracts 1-3 atoms from each
transcript with the closed 11-value `atom_type` enum (insight,
anecdote, quote, framework, statistic, story_angle, strategy_angle,
strategy, endorsement, critique, collection). Writes
`atoms/{YYYY-MM-DD}/{slug}` pages. Budget cap $0.30/source/run.
- `synthesize_concepts` — globally aggregates atoms by frontmatter
`concepts:` ref. Tier by count: T1 ≥10, T2 ≥5, T3 ≥2. T1/T2 get
Sonnet narratives; T3 falls back to a deterministic stub. Writes
`concepts/{slug}` pages. Budget cap $1.50/run.
One calibration domain: `concept_themes` / cluster_summary / [concept]
— tier histogram + page count, not Brier (concepts don't have binary
outcomes to score against).
### gbrain-investor
YC / investor lens. Declares 2 net-new page types on top of
gbrain-base's deal/person/company/yc seed:
- `thesis` (NEW) — investment thesis with thesis_text + key_bets[] +
market_view + vintage. Files at `investing/theses/{slug}`. Extractable
(the LLM mines claims into facts).
- `bet_resolution_log` (NEW) — outcome record for a thesis's bet. FK
to a take row via take_id; carries resolved_outcome + resolved_at +
learned_pattern. Files at `investing/bets/{YYYY-MM}/{slug}`.
No new cycle phases — consumes the existing
extract_facts/propose_takes/grade_takes/calibration_profile loop. Three
calibration domains: `deal_success` (scalar_brier over deal-attached
takes), `founder_evaluation` (scalar_brier over person-attached takes),
`market_call` (weighted_brier over thesis-attached takes; weighted by
conviction so high-stakes misses cost more).
### gbrain-engineer
Bridge-only pack. Declares `learning` page type + reuses base `code`.
No new cycle phases — the daemon-side `gstack-learnings` IngestionSource
(T8) watches `~/.gstack/projects/{repo}/learnings.jsonl` and emits
each JSONL line as a `learning` page when this pack is active. Three
calibration domains: `architecture_calls` (scalar_brier),
`effort_estimates` (weighted_brier), `risk_assessment` (scalar_brier).
Speculative ADR/postmortem/refactor_thesis/tech_debt types deferred
to v0.42+ — they'll ship when a real user authors the first one (D8).
### gbrain-everything
Meta-pack stacking creator + investor + engineer via the v0.38
`extends` + `borrow_from` chain. Single-active-pack constraint
preserved — this IS the active pack; the registry walks extends +
borrow to materialize the merged view.
**Merge contract (T20 / #1749).** `resolvePack` merges parent → child
(child-wins) for the six ingest/query-shaping fields: `page_types`,
`link_types`, `frontmatter_links`, `enrichable_types`, `filing_rules`,
and `takes_kinds` (unioned — a child cannot narrow it). `phases` and
`calibration_domains` are **NOT** inherited: they gate cycle execution,
so each pack must declare its own participation explicitly. That is why
`gbrain-everything` re-declares all its phases and all 7
`calibration_domains` — inheritance does not carry them.
Activate via `gbrain config set schema_pack gbrain-everything` and
calibration_profile produces all 7 domain scorecards in one JSONB.
## Calibration profile widening (T10)
Before v0.41.2.0, `calibration_profiles.domain_scorecards` was a
`JSON.stringify({})` placeholder. v0.41.2.0 widens it: each declared
domain produces a `{n, brier, accuracy, aggregator, page_types,
extras}` entry. Four aggregator algorithms (closed enum):
- **scalar_brier**`AVG(POWER(weight - outcome::int, 2))`. Default for
probabilistic predictions.
- **weighted_brier** — Brier weighted by `ABS(weight - 0.5) * 2`
(conviction proxy). High-conviction misses cost more.
- **count_based** — simple `SUM(hit) / COUNT(*)` accuracy without
Brier. Use when probability isn't natural.
- **cluster_summary** — descriptive rollup (page count + tier
histogram). For domains like `concept_themes` where there's no
binary outcome.
Pack manifests declare domains with `{name, aggregator, page_types}`.
Domain names are OPEN (third-party packs can declare new domain labels
without a gbrain release). Aggregator algorithms are CLOSED (safe SQL
stays in code, validated at pack-load).
## take_domain_assignments table (T1)
New JOIN table (migration v94):
`take_domain_assignments(take_id BIGINT FK, domain TEXT, pack TEXT,
source TEXT, confidence REAL, assigned_at TIMESTAMPTZ, PK(take_id,
domain))`. Multi-domain assignment honest — a take about "Sequoia's
investment in Anthropic" can land in BOTH `deal_success` AND
`market_call` rather than being force-bucketed.
## What this enables for the user
- **Atoms + concepts ship in the binary.** Your OpenClaw's parallel
atom-pipeline-coordinator + atom-backfill-coordinator + concept-
synthesis crons can retire (T12 follow-up). One `gbrain dream` cron
covers everything.
- **gstack learnings reach gbrain.** Engineer-pack-active brains
surface every gstack-logged learning as a queryable page within
seconds of being written.
- **Multi-lens calibration.** Activate gbrain-everything and see how
often you're wrong on deals AND market calls AND architecture
AND effort estimates in one `gbrain calibration --json` call.
- **Lossless OpenClaw migration.** The `markdown-greenfield`
importer (T7, mode='migration') re-ingests existing OpenClaw
pages with permanent slug-keyed idempotency + per-row JSONL audit
+ the `imported_from` marker so extract_atoms + synthesize_concepts
don't re-extract already-atomized material.
## v0.41.2.1 follow-ups (filed in plan)
- Per-page-type `frontmatter_validators` on PageTypeSchema so the
atom_type enum (currently hardcoded in extract_atoms.ts) reads from
the active pack manifest at runtime per D11.
- 3-check quality gate (truism / punchline / entity-page reject) as
a multi-pass extract_atoms refinement.
- Embedding-similarity dedup in synthesize_concepts (currently
exact-string concept ref match only).
- Voice gate integration for T1 Canon narratives.
- op_checkpoint resumability for cross-cycle continuation in both
phases.
- Parity-baseline eval gates against your OpenClaw's existing 13K atoms
+ 11K concepts on a 500-page sample subset.
-246
View File
@@ -1,246 +0,0 @@
# Pack-Upgrade Mechanism (v0.41.22)
> How `gbrain-base@1.x → gbrain-base-v2@1.0.0` (and any future pack
> succession) wires through the onboard cathedral.
## The contract
A schema pack manifest can declare a `migration_from` field:
```yaml
api_version: gbrain-schema-pack-v1
name: gbrain-base-v2
version: 1.0.0
migration_from:
pack: gbrain-base
version: "1.x"
```
When this declaration is present + a `mapping_rules:` block is
populated, the pack registers itself as the successor to
`(parent_pack, version_range)`. Any brain whose active pack matches
that tuple lights up the `pack_upgrade_available` onboard check.
## End-to-end flow
```
┌────────────────────────────────────────────────────────────────┐
│ PACK AUTHORING │
│ │
│ Author declares: migration_from: {pack: P, version: R} │
│ + mapping_rules: [retype/page_to_link/page_to_alias] │
│ Pack ships bundled OR via ~/.gbrain/schema-packs/<name>/ │
└──────────────────────────┬─────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────┐
│ ONBOARD CHECK DISCOVERY │
│ │
│ checkPackUpgradeAvailable(engine) at src/core/onboard/ │
│ checks.ts: │
│ 1. Read engine.getConfig('schema_pack') for dbConfig tier │
│ 2. loadActivePack({cfg: null, remote: false, dbConfig}) │
│ 3. findPackSuccessors(active.name, active.version) │
│ → walks BUNDLED_PACK_NAMES + ~/.gbrain/schema-packs/ │
│ → matches via _versionRangeMatches(version, range) │
│ → returns ResolvedPack[] sorted by successor version │
│ 4. If successors.length > 0, emit OnboardCheckResult │
│ with RemediationStep targeting `unify-types` handler │
│ + protected: true (D17 → manual_only via render │
│ allowlist) │
└──────────────────────────┬─────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────┐
│ USER DECIDES │
│ │
│ gbrain onboard --check shows finding │
│ gbrain onboard --check --explain shows per-cluster narrative │
│ User reviews; if OK, runs: │
│ gbrain jobs submit unify-types --allow-protected \ │
│ --params '{"target_pack":"gbrain-base-v2"}' │
│ (Autopilot never auto-fires this; manual_only) │
└──────────────────────────┬─────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────┐
│ HANDLER EXECUTION (src/core/schema-pack/unify-types-handler.ts) │
│ │
│ 1. Preflight: load target pack; assert mapping_rules present │
│ 2. Stats snapshot (pre-state for celebration) │
│ 3. Acquire gbrain-unify db-lock (60min TTL) │
│ 4. Apply phases (4): │
│ a. Explicit retype rules (chunked UPDATE 1000/batch) │
│ - frontmatter.legacy_type ALWAYS preserved (D8) │
│ - frontmatter.subtype stamped when subtype set │
│ b. Catch-all retype: synthesize per-unknown-type rule │
│ excluding declared types + explicit targets + page_to_ │
│ link/alias sources (D12 + critical bug fix) │
│ c. Page-to-link: parse body+frontmatter, insert link row, │
│ soft-delete source page (per-page atomicity per F7) │
│ d. Page-to-alias: insert slug_aliases row, soft-delete │
│ source page (NO rewriteLinks per D15) │
│ 5. Final sync: path-prefix typing for residual UNTYPED rows │
│ 6. ACTIVE-PACK FLIP (D13): │
│ - engine.setConfig('schema_pack', target_pack) │
│ - saveConfig({...existing, schema_pack: target_pack}) │
│ 7. Verify: re-run stats; warn if ≤ declared + 5 violated │
│ 8. Celebration summary to stderr + audit JSONL │
│ 9. Release db-lock │
└──────────────────────────┬─────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────┐
│ POST-UPGRADE STATE │
│ │
│ • pages.type updated with canonical types │
│ • frontmatter.legacy_type preserved for rollback │
│ • slug_aliases populated for old-slug → canonical lookup │
│ • links table has new partner_of / relates_to rows │
│ • Source pages soft-deleted (72h TTL for restore) │
│ • Active pack flipped to target_pack │
│ • Next gbrain onboard --check shows ok │
└────────────────────────────────────────────────────────────────┘
```
## Version-range semantics
`migration_from.version` accepts three shapes:
| Form | Matches |
|------|---------|
| `1.0.0` (exact literal) | `1.0.0` only |
| `1.x` (major wildcard) | `1.0.0`, `1.5.2`, `1.99.99` |
| `1.0.x` (minor wildcard) | `1.0.0`, `1.0.5`, `1.0.99` |
`*` is accepted as an alias for `x`.
Implementation: `_versionRangeMatches(version, range)` in
`src/core/schema-pack/load-active.ts`. Pinned by
`test/schema-pack-find-pack-successors.test.ts`.
## findPackSuccessors discovery
Walks `BUNDLED_PACK_NAMES` (currently `gbrain-base`,
`gbrain-recommended`, `gbrain-creator`, `gbrain-investor`,
`gbrain-engineer`, `gbrain-everything`, `gbrain-base-v2`). For each
candidate ≠ the active pack name, loads the manifest via
`loadActivePack({ perCall: candidate })`, checks
`migration_from.pack === activeName && _versionRangeMatches(activeVer,
migration_from.version)`. Returns matching packs sorted by version
descending.
v0.41.22 covers bundled packs only. v0.43+ TODO: enumerate user-installed
packs at `~/.gbrain/schema-packs/*/pack.yaml` (defer to v0.43 since the
filesystem-scan cost needs the cache invalidation strategy from
`registry.ts`).
## The manual_only apply policy
The shipped onboard contract has 3 apply_policy values:
| Policy | Meaning |
|--------|---------|
| `auto_apply` | Autopilot runs unattended |
| `prompt_required` | Autopilot in `--auto-with-prompt` mode prompts user |
| `manual_only` | Autopilot NEVER auto-fires; user must explicitly submit |
`pack_upgrade_available` emits a `RemediationStep` with `protected:
true` + `job: 'unify-types'`. `toOnboardRecommendation` in
`src/core/onboard/render.ts` maps this to `manual_only` via the
`MANUAL_ONLY_PROTECTED_JOBS` allowlist (which also contains
`extract-takes-from-pages` per v0.41.18 A12+A24).
Rationale: pack upgrades change the brain's taxonomy. Taxonomy is a
user judgment call — not autopilot's call. Even with `--auto-with-
prompt`, prompting the user to confirm a pack upgrade mid-tick is the
wrong UX (the user came to fix orphans, not to be interrupted with
"hey want to migrate your taxonomy?"). Explicit submission is the
right boundary.
## Authoring a successor pack
Minimal example for an academic-research brain that adds a
`researcher` canonical:
```yaml
api_version: gbrain-schema-pack-v1
name: gbrain-academic-v1
version: 1.0.0
description: Academic research brain — adds researcher canonical
gbrain_min_version: 0.42.0
extends: null
migration_from:
pack: gbrain-base-v2
version: "1.x"
page_types:
# Inherit gbrain-base-v2's 15 types here (or use extends to merge
# automatically once v0.43+ extends-chain composition lands)
- { name: person, primitive: entity, path_prefixes: [people/], expert_routing: true }
- { name: company, primitive: entity, path_prefixes: [companies/], expert_routing: true }
# ... all 13 other v2 canonicals ...
- { name: note, primitive: concept, path_prefixes: [notes/], extractable: true }
# Academic addition:
- name: researcher
primitive: entity
path_prefixes: [researchers/]
aliases: [academic, professor, scholar]
extractable: false
expert_routing: true
mapping_rules:
# All v2 mapping rules (copy from v2 yaml)
# ... ~40 rules ...
# Custom: relocate v2-tagged academics to researcher
- { kind: retype, from_type: person, to_type: researcher, path_filter: 'researchers/%' }
# Catch-all
- kind: retype
from_type: "*unknown*"
to_type: note
subtype_field: legacy_type
subtype: "*original_type*"
```
Drop at `~/.gbrain/schema-packs/gbrain-academic-v1/pack.yaml`.
Discoverable via `gbrain schema list`. Activatable via
`gbrain schema use gbrain-academic-v1`. Once active, the
`pack_upgrade_available` check fires for any brain on
`gbrain-base-v2@1.x` and surfaces a `unify-types` RemediationStep
targeting your pack.
## Lock + concurrency
`gbrain-unify` is a dedicated `gbrain_cycle_locks` row name (60min
TTL). The handler acquires it before any apply phase + releases in
`finally`. Two simultaneous `gbrain jobs submit unify-types`
invocations: second one fails fast at lock acquisition with a clear
error. Same pattern as `gbrain-sync` (v0.22.13 PR #490).
## Audit trail
Every unify run writes to `~/.gbrain/audit/schema-unify-YYYY-Www.jsonl`
(ISO-week rotation, mirrors existing audit channels). Records: pack
identities (before + after), per-phase counts (would_apply + applied),
warnings, completion timestamp. Privacy: page slugs are NOT logged in
bulk (only the per-rule sample_slugs[≤10]); for forensic debugging
add `GBRAIN_AUDIT_FULL=1` (v0.43+ TODO; not yet wired).
## What's NOT yet supported
- Subprocess sandbox for the publish-gate (v0.43+ TODO)
- Per-source pack-upgrade (the handler accepts `sourceId` but
`findPackSuccessors` doesn't yet pass it through)
- Cross-brain federated mounts that disagree on canonical packs
- Automatic rollback (today: manual SQL or `gbrain restore`)
- LLM-assisted mapping_rules codegen from production data (`gbrain
schema detect-mappings`; deferred to v0.43+)
## Reference
- Pack file: `src/core/schema-pack/base/gbrain-base-v2.yaml`
- Manifest extension: `src/core/schema-pack/manifest-v1.ts`
- Successor walker: `src/core/schema-pack/load-active.ts:findPackSuccessors`
- Onboard check: `src/core/onboard/checks.ts:checkPackUpgradeAvailable`
- Render allowlist: `src/core/onboard/render.ts:MANUAL_ONLY_PROTECTED_JOBS`
- Handler: `src/core/schema-pack/unify-types-handler.ts`
- Migration: `src/core/migrate.ts:105` (slug_aliases table)
- Type taxonomy doc: `docs/architecture/type-taxonomy.md`
- Skill: `skills/schema-unify/SKILL.md`
-258
View File
@@ -1,258 +0,0 @@
# Schema Packs
A schema pack tells gbrain what shape your brain takes — which directories
exist, what types live in them, how the agent should infer types from
paths, and which link verbs connect what to what. The schema pack is the
**dynamic, always-consulted artifact** every skill reads when filing,
querying, or routing experts. It is the single source of truth for
"what's in your brain."
The v0.39.0.0 wave shipped a full schema-pack cathedral. This doc is the
user-facing reference; for implementation details see
`docs/designs/V038_SCHEMA_PACKS.md` (CEO plan) and the engine layer in
`src/core/schema-pack/`.
## What ships in the box
Two bundled packs:
- **`gbrain-base`** (default) — reproduces pre-v0.38 hardcoded behavior
byte-for-byte. Existing brains see zero behavior change after upgrade.
Covers: person, company, deal, meeting, project, place, concept, writing,
analysis, guide, hardware, architecture, etc. (the original
`ALL_PAGE_TYPES` list).
- **`gbrain-recommended`** — extends `gbrain-base` with the 13 additional
directories described in `docs/GBRAIN_RECOMMENDED_SCHEMA.md`: deal,
meeting, concept, project, source, daily, personal, civic, original,
place, trip, conversation, writing. If you like the documented
operational-brain pattern, activate this with:
```bash
gbrain schema use gbrain-recommended
```
Plus user-installed packs at `~/.gbrain/schema-packs/<name>/pack.yaml`
that you author with `gbrain schema init` or `gbrain schema fork`.
## CLI surface
Five inspection verbs (shipped in v0.38):
```bash
gbrain schema active # show resolved pack + which tier set it
gbrain schema list # list bundled + installed packs
gbrain schema show # pretty-print the active pack
gbrain schema validate # validate a manifest's shape
gbrain schema use <pack> # activate a pack (writes ~/.gbrain/config.json)
```
Eight authoring + discovery verbs (shipped in v0.39):
```bash
gbrain schema detect # propose types matching brain shape
gbrain schema suggest # LLM-refined proposals on top of detect
gbrain schema review-candidates # promote / rename / ignore candidates
gbrain schema review-orphans # surface pages with no matching type
gbrain schema init <name> # scaffold a stub pack (experimental)
gbrain schema fork <a> <b> # copy + rename a pack (experimental)
gbrain schema edit <name> # surface the pack path (experimental)
gbrain schema diff <a> <b> # set-diff two packs (experimental)
gbrain schema graph # ASCII type listing (experimental)
gbrain schema lint # flag duplicates + missing prefixes
gbrain schema explain <type> # plain-English type description (experimental)
gbrain schema downgrade --to <p> # restore previous pack (recovery)
gbrain schema usage --since 30d # per-verb invocation counts (D14 telemetry)
```
The verbs marked `experimental` are demand-gated per D14: their usage is
tracked via T15's schema-events audit, and v0.40+ retro decides whether
to deprecate any that stay <5% usage.
## Resolution chain (7 tiers)
When the engine decides "which pack is active for this query?", it walks
this chain top-down. First match wins.
| Tier | Source | Notes |
|------|--------|-------|
| 1 | Per-call `schema_pack` opt | CLI only (`ctx.remote === false`); MCP rejected. |
| 2 | `GBRAIN_SCHEMA_PACK` env | Process-scope override. |
| 3 | Per-source DB config key `schema_pack:source:<id>` | New in v0.38. |
| 4 | Brain-wide DB config key `schema_pack` | |
| 5 | `gbrain.yml schema:` section | Repo-checked. |
| 6 | `~/.gbrain/config.json` `schema_pack` field | What `gbrain schema use` writes. |
| 7 | Default: `gbrain-base` | Always present. |
## How the agent uses the active pack
Every read + write path consults the active pack at runtime:
- **`parseMarkdown`** infers page `type` from path prefixes declared in
the active pack (`page_types[].path_prefixes`). Without an active pack
threaded, falls back to the legacy hardcoded `inferType()` so the
byte-for-byte parity gate stays green.
- **`whoknows` / `find_experts`** scopes candidates to `expert_routing:
true` types in the active pack.
- **`extract_facts`** runs only on `extractable: true` types.
- **`enrichment-service`** routes person/company enrichment based on the
pack's primitive declarations.
- **Search hybrid cache** (`knobsHash`) folds in pack name + version
(v0.39 T21). A cache row written under pack A is unreachable when pack
B is active. Cross-pack contamination is structurally impossible.
## The magical moment (T2-T4 + T10)
Persona A (Notion refugee) installs gbrain, imports her exports, and the
brain looks unfamiliar — the default `gbrain-base` pack expects
`people/`, `companies/`, etc., but her files live under `Projects/`,
`Reading/`, `Daily Notes/`. The friction signal fires in two places:
1. **Import warn (T7):** the end of `gbrain import` prints
`[schema] X of Y pages (Z%) have no type matching the active schema
pack. Run gbrain schema detect to propose a pack matching your
content shape.`
2. **`gbrain doctor` schema_pack_consistency check** keeps surfacing
the warning persistently after the import session ends.
She runs the magical moment:
```bash
gbrain schema detect # heuristic clustering on her actual shape
gbrain schema suggest # LLM-refined proposals
gbrain schema review-candidates # human gate on promotion
gbrain schema review-candidates --apply Projects/ # accept
```
The agent (via the new EIIRP skill) automates phases 1-3 of this for any
significant work session. The brain's schema becomes a living artifact
the agent maintains, not a hardcoded ceremony the user authors.
## Authoring your own pack
```bash
gbrain schema init my-pack # scaffolds ~/.gbrain/schema-packs/my-pack/pack.yaml
$EDITOR ~/.gbrain/schema-packs/my-pack/pack.yaml
gbrain schema validate my-pack # check shape
gbrain schema use my-pack # activate
gbrain schema active # confirm
```
A minimal pack:
```yaml
api_version: gbrain-schema-pack-v1
name: my-pack
version: 0.0.1
gbrain_min_version: 0.39.0
extends: gbrain-base # inherits base's TYPES (see Merge contract below); add overrides
description: |
My personal pack.
page_types:
- name: project-x
primitive: entity
path_prefixes:
- Projects/
aliases: []
extractable: false
expert_routing: false
# Add more types here. Each maps a path prefix to a primitive +
# opt-in flags. See src/core/schema-pack/base/gbrain-recommended.yaml
# for a worked example.
link_types: []
takes_kinds: [fact, take, bet, hunch]
borrow_from: []
frontmatter_links: []
enrichable_types: []
filing_rules: []
```
## Merge contract (`extends` + `borrow_from`)
`resolvePack` composes a pack against its `extends` chain (and any
`borrow_from` targets) into the `resolved.manifest` every consumer reads
(T20 / #1749). The rules:
- **Six fields inherit, child-wins:** `page_types`, `link_types`,
`frontmatter_links`, `enrichable_types`, `filing_rules`, and `takes_kinds`.
A child value with the same key (type name, link name, etc.) overrides the
parent's; keys the child doesn't declare come through from the parent.
- **`page_types` ordering:** overrides of a base type keep the base's declared
position (base's `inferType` prefix priority is authoritative); a genuinely
new type — from the child, a `borrow_from`, or a middle pack in the chain —
is prepended nearest-first, so a more-derived type's `path_prefix` wins
regardless of how deep the chain is.
- **`takes_kinds` is UNION, not replace** — it carries a Zod default, so an
omitted field is indistinguishable from an explicit one. A child can ADD
kinds but **cannot narrow** `takes_kinds` below base parent. If you need a
smaller set, don't `extends` a pack that declares the larger one.
- **`phases` and `calibration_domains` are NOT inherited** (child-only). They
gate real cycle execution, so each pack must declare its own participation
explicitly — inheriting them would silently make a child run phases it never
requested. This is why `gbrain-everything` re-declares all its phases and
calibration domains by hand. See `lens-packs.md` for the worked example.
- **`borrow_from` is selective + non-transitive + fail-closed:** it pulls only
the named `types`/`link_types` from the target's OWN declarations (omitting a
category borrows none of it); a missing target throws `UnknownPackError`.
## Recovery + revert
The single-PR cathedral is hard to revert atomically. Per codex finding
#4 from plan-eng-review, T20 ships `gbrain schema downgrade` to restore
the active-pack config field:
```bash
gbrain schema downgrade --to gbrain-base
# OR auto-detect previous from ~/.gbrain/schema-pack-history.jsonl:
gbrain schema downgrade
```
**Code revert alone is NOT sufficient.** The full revert procedure:
1. `git revert <merge-commit>` — restores the code.
2. `gbrain schema downgrade --to gbrain-base` — restores config.
3. (Optional) `gbrain purge-deleted --older-than 0h` — drops
v0.39-typed pages that no longer have a matching type in the active
pack.
The cache + eval rows that pack-aware code wrote are isolated by the
`knobsHash` pack-folding (T21) — they become unreachable under the
restored pack so no eviction is needed.
## Distribution
`.gbrain-schema` tarballs ride the same v0.37 skillpack pipeline as
`.gbrain-skillpack` tarballs (T14 artifact abstraction). The
discriminator is `api_version` in the manifest:
- `gbrain-schema-pack-v1` → schemapack
- `gbrain-skillpack-v1` → skillpack
Both install via the same scaffold + copy path; install targets are
`~/.gbrain/schema-packs/<name>/` and `~/.gbrain/skillpacks/<name>/`
respectively.
Publication to the public registries (`garrytan/gbrain-schema-registry`,
`garrytan/gbrain-skillpack-registry`) follows the same publish-as-PR
workflow as v0.37 skillpack publishing.
## What's deferred to v0.40+
- **Per-source pack federation across mounts.** A query crossing multiple
sources currently rejects with `permission_denied` when those sources
have divergent active packs (T19 + codex finding #2). The v0.40+ work
computes a true per-source closure via the existing
`buildSourceClosureCte` engine surface.
- **`extends` chain semver compatibility checks** between pack versions.
- **`skillpack ↔ schemapack` cross-reference declarations** — a skillpack
can declare "I work best with these primitives present in your pack."
- **Live schema migration helpers** — when you add a type, auto-suggest
backfill of existing pages.
- **Authoring vs derivation thesis reframe (D14).** v0.39.0.0 ships the
full 11-verb cathedral with 6 verbs marked experimental-tier. v0.40+
retro reads T23 usage telemetry to decide which to deprecate.
See `TODOS.md` v0.40+ section for the full deferred list.
@@ -1,54 +0,0 @@
# `gbrain serve``gbrain sync` concurrency (PGLite)
**Short version: on a PGLite brain, stop `gbrain serve` before a large sync.**
## Why
PGLite is a single-writer embedded Postgres (WASM). A running `gbrain serve`
(stdio or HTTP MCP) holds an open PGLite connection on the brain's data
directory. `gbrain sync` needs to write to that same data directory. The two
contend for PGLite's single-writer connection / write-lock — **this is NOT the
`gbrain-sync` advisory lock** (that's a separate, DB-row coordination lock for
two concurrent *syncs*). Confusing the two sends you debugging the wrong surface.
Symptoms of serve↔sync contention on PGLite:
- `gbrain sync` blocks acquiring the PGLite write lock, or makes very slow
progress, while a `gbrain serve` process is alive on the same brain.
- Killing stale `gbrain serve` MCP processes frees the lock and sync proceeds.
## What to do
1. Stop any `gbrain serve` process for this brain before a large sync:
```bash
pkill -f 'gbrain serve' # or stop your MCP client / Claude Desktop / Cursor
gbrain sync --no-pull --no-embed --yes
```
2. Restart `gbrain serve` after the sync completes.
This contention does **not** apply to the Postgres engine — Postgres tolerates
concurrent connections, so `serve` and `sync` can run simultaneously there.
## Diagnosing a sync hang
If a sync wedges (no progress, high CPU), re-run with the per-file begin trace
so the stalling file is named:
```bash
GBRAIN_SYNC_TRACE=1 gbrain sync --no-pull --no-embed --yes
```
The last `[sync] begin import: <path>` line with no following completion is the
file being processed when the hang occurred. Under `--workers >1` / `--all`,
the stuck file is in the set of begin-lines without a matching completion.
If you suspect a schema-pack regex is the cause (a pack with a
catastrophic-backtracking `inference.regex`), complete the sync with the pack
disabled and re-run extraction afterward:
```bash
gbrain sync --no-schema-pack --no-pull --no-embed --yes
```
`gbrain schema lint` flags the classic nested-quantifier ReDoS shapes
(`(a+)+`, `(a*)*`, …) in pack regexes as warnings.
-198
View File
@@ -1,198 +0,0 @@
# System of record
**The GitHub repo (markdown + frontmatter) is the system of record.
The Postgres/PGLite database is a derived cache. We do not back up
the database — we rebuild it from the repo.**
This document is the canonical reference for that contract. Every code
path that writes user-knowledge state should match the pattern
described here. The CI gate at `scripts/check-system-of-record.sh`
enforces it programmatically.
## Why this matters
The DB is a derived index over the markdown content. It exists to make
search fast, to dedup embedding-similar claims, to materialize the
cross-page graph. None of that data is irreplaceable — as long as the
markdown is intact, `gbrain sync && gbrain extract all` rebuilds the
entire DB from scratch.
This means:
- **Disaster recovery is a short, boring sequence.** If your DB volume
corrupts, if Postgres eats itself, if PGLite's WASM lock wedges — you
don't need a backup. You wipe the derived tables (on PGLite,
`gbrain reinit-pglite` wipes the whole embedded DB), re-import from
your brain repo with `gbrain sync`, and `gbrain extract all`
regenerates the derived state. See "Disaster recovery" below for the
exact commands.
- **Multi-machine sync is git.** Your brain is a repo. Push from one
machine, pull from another, and the second machine's DB rebuilds on
its next sync. No "back up the database" step.
- **Privacy is in your hands.** Sensitive entity pages can be
gitignored (via `gbrain.yml` `db_only` paths or per-page) and they
stay on disk but not in git. The fence respects whatever git
tracking choice you make at the page level.
- **Cross-agent collaboration is possible.** Multiple agents can write
to the same brain because the fence is the merge point, not the DB.
Git handles concurrent edits the way git handles concurrent edits.
## The three categories
Every table in the gbrain schema belongs to exactly one of three
categories. The category determines how it gets rebuilt during
disaster recovery.
### FS-canonical (markdown is the source of truth)
These are user-authored knowledge. The DB row is a derived index over
the markdown — wipe the table and `gbrain extract` rebuilds it
identically. The CI gate keeps direct DB writes from drifting away
from the markdown contract.
| Category | How it's stored in markdown | Derived DB table | Reconciler |
|---|---|---|---|
| **Takes** (incl. hunches, bets) | `## Takes` fenced table between `<!--- gbrain:takes:begin -->` / `:end -->` markers | `takes` | `extract takes` |
| **Facts** | `## Facts` fenced table between `<!--- gbrain:facts:begin -->` / `:end -->` markers | `facts` | `extract_facts` cycle phase |
| **Links** | Inline `[text](slug)` / `[[slug]]` in markdown body + frontmatter `direction: incoming` | `links` | `extract links` |
| **Timeline** | `## Timeline` section after `<!-- timeline -->` sentinel | `timeline_entries` | `extract timeline` |
| **Tags** | Frontmatter `tags:` YAML array | `tags` | `importFromFile` (reconciles per-page on import) |
| **emotional_weight** | Recomputed from takes + tags | `pages.emotional_weight` (signal column) | `recompute_emotional_weight` cycle phase |
| **synthesis_evidence** | FK into `takes` rows (`slug#N`) inside synthesis pages | `synthesis_evidence` | `extract takes` (transitively) |
### Derived from FS but not user-authored
These hold derived state that's automatically reconstructible from the
markdown but not directly authored as markdown by the user. The
chunker + embedder rebuild these on import.
| Table | Source | Notes |
|---|---|---|
| `pages` | The markdown file as a whole | One row per file; `compiled_truth` + `frontmatter` come from parse |
| `content_chunks` | `pages.compiled_truth` after chunker strip | Re-chunked on content_hash change; embedded via configured model |
| `page_versions` | Each `pages` UPDATE | Audit history; rebuildable in principle but not in practice |
### DB-only by design (named exceptions)
These hold runtime / infrastructure state that's intentionally not in
the repo. The architectural rule still holds — these aren't
"user knowledge" — but they're DB-only by design.
| Category | Why it's OK to be DB-only |
|---|---|
| `raw_data` | Webhook/transcript sidecars; not user-authored knowledge. |
| `subagent_messages` / `subagent_tool_executions` / `subagent_rate_leases` | Runtime job state. Replay-only, not persistent knowledge. |
| `oauth_clients` / `oauth_tokens` / `access_tokens` | Credentials. Not in source control by definition. |
| `mcp_request_log` | Audit trail. Volatile by design. |
| `minion_jobs` / `minion_inbox` / `minion_attachments` | Job queue. Restarts re-enqueue or drop. |
| `eval_candidates` / `eval_capture_failures` | Contributor-mode dev loop; opt-in capture. |
| `dream_verdicts` | Cheap verdict cache. Rebuildable by re-running Haiku. |
| `gbrain_cycle_locks` / migration ledger | Infrastructure. |
| `config` (some keys) | Site-local routing config (e.g. `sync.repo_path`). |
A new derived table that holds user-knowledge MUST land FS-first.
If you're tempted to add one as "DB-only for now," the structural
question is: does it belong in this DB-only-by-design list? If not,
it's FS-canonical and needs a fence (or frontmatter field) plus a
reconciler.
## The privacy boundary
Private knowledge in a fence still lives in the markdown file. If the
user commits the page to git, the private data lands in git too. This
is the existing operational model — we don't infer git policy.
For untrusted readers (remote MCP, subagent), the v0.32.2 release ships
a 3-layer strip:
1. **Layer A (chunker):** `src/core/chunkers/recursive.ts` calls
`stripFactsFence({keepVisibility: ['world']})` + `stripTakesFence`
before chunking. Private fact text never reaches
`content_chunks.chunk_text`, embeddings, or search results.
2. **Layer B (get_page):** when `ctx.remote === true`, the response
body has both fences stripped (private rows from facts; entire
takes fence). Local CLI (`ctx.remote === false`) sees the full
fence.
3. **Layer C (git tracking):** the user decides whether to commit the
entity page. `gbrain.yml` `db_only` paths are gitignored
automatically; per-page choices via the user's normal git workflow.
For universally-private entities (a friend's name, an investor's
internal notes), mark the entity page's directory as `db_only` in
`gbrain.yml`. The file stays on disk but never lands in git.
## The forget contract
`gbrain forget <id>` and the MCP `forget_fact` op rewrite the fence
row with strikethrough + `valid_until = today` + `context: "forgotten:
<reason>"`. The DB's `expired_at = valid_until + now()` derivation
reconstructs the forget state on every rebuild because the fence is
canonical.
Strikethrough has two semantics distinguished by context:
- `~~claim~~` + `context: "superseded by #N"` → row was replaced by
a newer row in the same fence
- `~~claim~~` + `context: "forgotten: <reason>"` → row was retracted
via the forget op
Both encodings keep the row in the markdown for audit history. To
permanently delete a fact, edit the fence directly in markdown and
remove the row. The next `extract_facts` cycle wipes the DB row.
## Disaster recovery
The promise the rule makes:
```bash
# Snapshot what's there
gbrain stats > /tmp/before.txt
# Wipe and rebuild — delete the derived tables (pages + content_chunks
# survive the CASCADE-safe design), then re-derive from the repo.
# On PGLite, `gbrain reinit-pglite` wipes the whole embedded DB instead.
psql -c 'DELETE FROM facts; DELETE FROM takes; DELETE FROM links; DELETE FROM timeline_entries;'
gbrain sync
gbrain extract all
# Counts match
gbrain stats > /tmp/after.txt
diff /tmp/before.txt /tmp/after.txt
```
The invariant E2E test at `test/e2e/system-of-record-invariant.test.ts`
exercises this exact flow on every CI run.
## Rule for new code
When you add a new user-knowledge category:
1. **Define the markdown shape.** Fence (`<!--- gbrain:NAME:begin
--> ... :end -->` table) or frontmatter field.
2. **Build a parser** that produces structured data from markdown.
See `src/core/fence-shared.ts` for the shared primitives.
3. **Build a writer** that round-trips: parse + edit + render produces
byte-identical markdown for identical input.
4. **Add the engine method** that takes parsed data and stamps a
derived table. The method gets an entry in the CI gate's
banned-direct-call list.
5. **Add a reconciler:** a cycle phase that walks pages, parses the
fence, and rebuilds the derived table from scratch. The reconciler
is the only legitimate call site for the engine method;
`// gbrain-allow-direct-insert: <reason>` annotates it explicitly.
6. **Add a round-trip test** in `test/e2e/system-of-record-invariant.test.ts`
that proves DELETE + reconcile rebuilds the table byte-identically.
The CI gate at `scripts/check-system-of-record.sh` fails any PR that
adds a new direct call to a derived-table writer outside the
reconciler / migration layer without the explicit allow-list comment.
## Related
- `~/.claude/plans/system-instruction-you-are-working-expressive-pony.md`
— the v0.32.2 design plan (decisions D1-D22 + Q1-Q8, Codex round 1
and round 2 finds)
- `skills/migrations/v0.32.2.md` — the agent-facing migration guide
- `CHANGELOG.md` v0.32.2 entry — the release manifesto
- `scripts/check-system-of-record.sh` — the CI gate that enforces
the rule
-70
View File
@@ -1,70 +0,0 @@
# Thin-client routing (remote MCP)
On-demand reference (see CLAUDE.md Reference map). Current behavior + invariants
only; release history lives in `CHANGELOG.md` + git.
`gbrain init --mcp-only` (v0.29.2) sets up a thin-client install: no local
brain content, just an OAuth client pointing at a remote `gbrain serve --http`.
v0.29.2/v0.30.0 only refused 9 obvious local-only commands; the other ~25
silently fell through to `connectEngine()` and opened the empty local PGLite,
returning "No results." against a populated remote brain. v0.31.1 fixes the
silent-empty-results bug class for every operation surface.
Key files:
- `src/cli.ts` — Routing seam INSIDE the existing op-dispatch path (CDX-1: no
parallel `src/core/thin-client/` module; routing is a ~80-line conditional
in `runThinClientRouted`). Detects `isThinClient(cfg)` BEFORE `connectEngine`
so thin-client installs never open the empty PGLite. localOnly ops on
thin-client refuse via `refuseThinClient` (with pinpoint hint table
`THIN_CLIENT_REFUSE_HINTS`). Banner via `printIdentityBannerBestEffort`
before each routed call (suppressed by `--quiet`, `GBRAIN_NO_BANNER=1`,
non-TTY default). Exhaustive TS `never` switch on `RemoteMcpError.reason`
for canned, actionable error messages. ENG-2 renderer parity: local-engine
path runs `JSON.parse(JSON.stringify(result))` so renderers see the same
shape on both paths (kills Date/bigint/Buffer drift class).
- `src/core/mcp-client.ts``callRemoteTool(config, toolName, args, opts)`.
Hardened in v0.31.1 (CDX-4): all transport errors normalized to
`RemoteMcpError` via the `toRemoteMcpError` funnel. New `CallRemoteToolOptions
{timeoutMs, signal}`; `buildAbortController` composes external signal with
timeout. New `RemoteMcpErrorReason` stable union, `RemoteMcpErrorDetail.kind`
('timeout' | 'aborted' | 'unreachable') sub-tag, `RemoteMcpErrorDetail.code`
field carrying server-supplied error codes (e.g. `missing_scope`).
`extractToolErrorCode` parses JSON envelopes first, falls back to substring
detection for legacy server messages. `unpackToolResult<T>(res)` unchanged
(parses tool-call JSON content). `_clearMcpClientTokenCache()` test escape.
- `src/core/cli-options.ts``parseGlobalFlags` adds `--timeout=Ns` (accepts
`30s`, `2m`, `500ms`, plain ms). Default `null` = per-command default (30s
for most ops, 180s for `think`). `parseTimeout(s)` exported helper.
- `src/core/doctor-remote.ts``gbrain remote doctor` adds the
`oauth_client_scopes_probe` check (CDX-5). Probes the read tier via
`get_brain_identity` and admin tier via `get_health`; reports per-tier
status with pinpoint remediation when admin is missing. `buildScopeCheck`
+ `ScopeProbeResult` exported for test access. Skippable via
`GBRAIN_DOCTOR_SKIP_SCOPE_PROBE=1` for fixtures that mock /mcp at JSON-RPC
initialize level only (MCP SDK Client hangs on shape mismatch).
- `src/core/ssrf-validate.ts` (v0.36 Commit 0) — DNS-rebinding-defended URL validation. `validateAndResolveUrl(url)` resolves the hostname via `dns.lookup({all: true, family: 0})`, checks EVERY A AND AAAA record against the internal-IP deny list, returns the resolved IP so callers fetch by IP (defeats DNS rebinding: validation IP === fetch IP). `fetchWithSSRFGuard(url, opts)` does redirect-aware fetching with per-hop re-validation, max 3 hops by default. Reusable across all URL-fetching features. Test seam `__setDnsLookupForTests` for hermetic tests.
- `src/core/search/query-intent.ts` extension (v0.36 cross-modal wave) — new `suggestedModality: 'text' | 'image' | 'both'` axis on `QuerySuggestions`. Module-scope `CROSS_MODAL_PATTERNS` regex array (compiles once at module load). `isAmbiguousModalityQuery(query)` heuristic gate fires when a visual noun + reference marker combination indicates genuinely ambiguous routing — used by the Commit 4 LLM tie-break to bound LLM calls to <1% of queries.
- `src/core/search/mode.ts` extension (v0.36 cross-modal wave) — `ModeBundle` extended with 7 cross-modal knobs: `cross_modal_both_text_weight` / `cross_modal_both_image_weight` (D6 weighted RRF for `'both'` mode, defaults 0.6/0.4), `image_query_text_refinement_weight` / `image_query_image_refinement_weight` (D13 hybrid intersect for `searchByImage` query refinement, defaults 0.4/0.6), `unified_multimodal` + `unified_multimodal_only` (Phase 3 unified column routing flags), `cross_modal_llm_intent` (Commit 4 opt-in escalation). `SEARCH_MODE_CONFIG_KEYS` extended with 7 corresponding config keys. `KNOBS_HASH_VERSION` bumped 2→3 (D2 — closes the silent cache-hit class where a cached text-mode result could leak to an image-mode caller).
- `src/core/search/hybrid.ts` extension (v0.36 cross-modal wave) — cross-modal routing branch at the embed step. Resolves `effectiveModality` from per-call `opts.crossModal` (normalized: literal `'auto'` → undefined per D22-1) → `suggestions.suggestedModality``'text'` default. Image route: `embedQueryMultimodal` + `searchVector({embeddingColumn: 'embedding_image'})`, skip expansion + keyword (D9 mode-bundle override). 'both' route: parallel text + image vector searches merged via `rrfFusionWeighted` with `effectiveRrfK(baseRrfK, weight)` from the configured cross-modal weights. Phase 3 unified routing fires when `cfg.search.unified_multimodal === true` — bypasses dual-column branching, runs `embedQueryMultimodal` + `searchVector({embeddingColumn: 'embedding_multimodal'})`, D8 fail-open on zero rows + not strict-mode falls through to dual-column. Commit 4 LLM escalation fires only when (no explicit per-call opt) AND (regex returned 'text') AND (`cfg.search.cross_modal.llm_intent` is true) AND (`isAmbiguousModalityQuery` returns true). Fail-open on every error.
- `src/core/search/image-loader.ts` (v0.36 Phase 2) — `loadImageInput(input, opts)` accepts local path, `data:` URI, or `http(s)://` URL. Magic-byte sniff for PNG/JPEG/WebP. Hard size cap (default 10 MB, configurable via `search.image_query.max_bytes`). For URLs: routes through `fetchWithSSRFGuard` so DNS rebinding + redirect chains are defeated. Pre-flight Content-Length check + post-fetch size guard for lying servers. `ImageLoadError` with discriminated `code` (INVALID_FORMAT / OVERSIZED / INVALID_URL / FETCH_FAILED / TIMEOUT / SSRF_BLOCKED / NOT_FOUND).
- `src/core/search/by-image.ts` (v0.36 Phase 2) — `searchByImage(engine, input, opts)`. Always runs image branch (`embedQueryMultimodalImage` + `searchVector(embedding_image)`). D13 hybrid intersect: when caller provides optional `query`, runs parallel text branch via `embedQueryMultimodal(query)` and merges via `rrfFusionWeighted` with weights from resolved mode. Phase 3 widens to unified column once `search.unified_multimodal=true` (transparently upgrades the retrieval quality post-reindex).
- `src/core/spend-log.ts` (v0.36 Phase 2 D23-#6) — per-OAuth-client paid-API spend tracking against the `mcp_spend_log` table (migration v74). `checkBudget(engine, clientId, capCents)` is the pre-flight gate; throws `BudgetExceededError` when today's spend has hit the cap. `recordSpend(engine, entry)` is best-effort post-call. UTC day-aligned aggregation so caps roll over deterministically regardless of server timezone. Local CLI callers (no clientId) bypass the gate. Pre-v0.36 brains without the table fail open to spend=0. `VOYAGE_MULTIMODAL_3_PER_IMAGE_CENTS` = 0.12 cents per image embed.
- `src/core/search/llm-intent.ts` (v0.36 Commit 4) — opt-in LLM tie-break. `classifyModalityWithLLM(query, fallback)` routes through `gateway.chat()` with a fixed single-word-output system prompt. 1s timeout via AbortController. `parseModality(raw, fallback)` is the pure parser — tolerates trailing punctuation + casing. Fail-open on every error (gateway unavailable, timeout, parse failure, unrecognized output) — returns fallback so a misbehaving LLM can never break search. Cost-bounded by the ambiguity heuristic in `query-intent.ts` (fires <1% of queries when on).
- `src/commands/reindex-multimodal.ts` (v0.36 Phase 3) — `gbrain reindex --multimodal [--limit N] [--dry-run] [--cost-estimate] [--no-embed] [--yes] [--json]`. Walks `content_chunks WHERE embedding_multimodal IS NULL`, batches via `embedMultimodalSafe` (Commit 0 partial-failure-aware), persists. D7 lock acquisition via `tryAcquireDbLock('gbrain-reindex-multimodal', 360min)`. Cost prompt + 10s Ctrl-C grace window in TTY. `GBRAIN_NO_REEMBED=1` bypass. Checkpoint at `~/.gbrain/reindex-multimodal-checkpoint.json` for resume. D23-#2 auto-flip prompt at coverage=100% completion (TTY: interactive; non-TTY: stderr hint with paste-ready command).
- `src/core/backfill-registry.ts` extension (v0.36) — new `modality` backfill kind. SQL filter requires `chunk_source='image_asset'` AND `embedding_image IS NOT NULL` AND `(modality IS NULL OR modality != 'image')`. D22-7 defensive guard: never flag a non-image chunk that happens to have `embedding_image` populated. Idempotent — second run finds zero rows.
- `src/core/migrate.ts` v74 (`mcp_spend_log`) + v75 (`embedding_multimodal_column`) — Phase 2 spend-log table + Phase 3 unified column ALTER. v75 is column-only (no HNSW index — deferred to post-reindex per pgvector best practice). v74 uses BTREE on `(client_id, created_at)` + `(token_name, created_at)``date_trunc('day', TIMESTAMPTZ)` is NOT IMMUTABLE so can't appear in index expressions; range scan on created_at covers the per-day rollup query.
- `src/core/operations.ts``get_brain_identity` op (read scope, no params,
banner-only): cheap counter packet `{version, engine, page_count,
chunk_count, last_sync_iso}` for the thin-client identity banner. Reuses
`engine.getStats()`; banner's 60s client-side TTL bounds frequency to
≤1/60s per CLI process (well below the Fly.io health-check cadence that
motivated the original `getStats` cost warning).
- `src/commands/{salience,anomalies,graph-query,think}.ts` — Per-command
thin-client routing branches. These commands bypass the operation-layer
dispatch in cli.ts (call `engine.foo()` directly), so each gets its own
`if (isThinClient(cfg)) { callRemoteTool(...) }` branch that maps CLI flags
to op params. `think` is a special case: the server's `think` op
intentionally disables `--save`/`--take` for remote callers
(operations.ts:1103-1135 trust-boundary gate); thin-client `think` warns
loudly when those flags are set.
-403
View File
@@ -1,403 +0,0 @@
# GBrain Deployment Topologies
GBrain supports three deployment shapes. They compose: a single user can mix
all three on the same machine without conflict, because every shape resolves
to "which `~/.gbrain/config.json` is active right now?" and `GBRAIN_HOME`
controls that selection.
This page covers the three topologies, when each fits, and concrete setup
recipes. Pair this doc with `docs/architecture/brains-and-sources.md` (which
covers the in-brain organization axes) — that doc is about WHICH database;
this doc is about WHERE that database lives.
## Quick decision tree
```
"I'm setting up gbrain..."
Just for me, on one machine? ─── yes ───▶ Topology 1 (single brain)
no
Will a remote machine host the brain
while my agent runs locally? ──── yes ───▶ Topology 2 (cross-machine thin client)
no
Multiple Conductor worktrees that
shouldn't share a code index? ─── yes ───▶ Topology 3 (split-engine)
```
Topologies 2 and 3 stack: a thin-client install can also host per-worktree
code engines, and a per-worktree code engine can also point its artifact
brain at a remote server.
## Topology 1 — Single brain (today's default)
```
┌────────────────┐
│ one machine │
│ ┌──────────┐ │
│ │ gbrain │──┼──→ ~/.gbrain/ → PGLite or Supabase
│ │ CLI │ │
│ └──────────┘ │
└────────────────┘
```
What you get: one local DB (PGLite for small brains, Supabase for ~1000+
files). All commands work directly against it. `gbrain serve` exposes it
to a single agent over MCP.
When it fits: solo use, single machine, one agent, no Conductor parallelism.
This is the default; `gbrain init` (no flags) gives you this.
Setup:
```
gbrain init # interactive — defaults to PGLite
gbrain init --pglite # explicit local
gbrain init --supabase # remote Supabase (recommended for 1000+ files)
```
Nothing else here is special. The other two topologies are variations on
"who owns the DB" and "how does the agent talk to it."
## Topology 2 — Cross-machine thin client
```
┌────────────┐ ┌──────────────────┐
│ neuromancer│ │ brain-host │
│ ┌────────┐ │ HTTP MCP / OAuth │ ┌────────────┐ │
│ │ Hermes │─┼───────────────────→│ │ gbrain │──┼──→ Supabase
│ │ agent │ │ │ │ serve --http│ │
│ └────────┘ │ │ └────────────┘ │
│ │ │ (with autopilot)│
│ no local │ │ │
│ gbrain DB │ │ │
└────────────┘ └──────────────────┘
```
What you get: the agent on one machine ("neuromancer") consumes a brain
hosted on another machine ("brain-host") over HTTP MCP with OAuth. The
agent's machine has NO local engine. All queries, searches, embeddings,
and indexing happen on the host.
When it fits:
- Heavy brain (Supabase + autopilot) lives on a beefy machine; agents
elsewhere just consume it.
- You want one source of truth across many machines.
- Spinning up a parallel local install would create source-ID contention or
duplicate work.
The thin client's `~/.gbrain/config.json` carries a `remote_mcp` field
instead of a local DB connection:
```jsonc
{
"engine": "postgres", // ignored — never used
"remote_mcp": {
"issuer_url": "https://brain-host.local:3001",
"mcp_url": "https://brain-host.local:3001/mcp",
"oauth_client_id": "neuromancer-...",
"oauth_client_secret": "..." // or set GBRAIN_REMOTE_CLIENT_SECRET
}
}
```
The CLI dispatch guard refuses any DB-bound command (`sync`, `embed`,
`extract`, `migrate`, `apply-migrations`, `repair-jsonb`, `orphans`,
`integrity`, `serve`) on a thin-client install with a clear error pointing
at the remote host. `gbrain doctor` runs a dedicated thin-client check set
(OAuth discovery, token round-trip, MCP smoke).
### Setup
**Step 1 — On the host (brain-host):**
```bash
gbrain init --supabase # or --pglite, doesn't matter
gbrain serve --http --port 3001 --bind 0.0.0.0 # v0.34: bind explicitly for remote access
# (defaults to 127.0.0.1 since v0.34)
gbrain auth register-client neuromancer \
--grant-types client_credentials \
--scopes read,write,admin # admin needed for ping/doctor
# v0.34: source-scoped client (write to one source, federate reads across
# multiple sources). Omit both flags for a v0.33-compatible super-client.
gbrain auth register-client neuromancer-dept \
--grant-types client_credentials \
--scopes read,write \
--source dept-x \
--federated-read dept-x,shared,parent-canon
```
The `register-client` command prints a `client_id` and `client_secret`.
Note both. **Scope must include `admin`**`submit_job` (used by
`gbrain remote ping`) and `run_doctor` (used by `gbrain remote doctor`)
both require it.
**Step 2 — On the thin client (neuromancer):**
```bash
gbrain init --mcp-only \
--issuer-url https://brain-host.local:3001 \
--mcp-url https://brain-host.local:3001/mcp \
--oauth-client-id <id> \
--oauth-client-secret <secret>
```
Pre-flight smoke runs three probes (OAuth discovery, token round-trip,
MCP initialize). If any fails, init exits with an actionable error. On
success, `~/.gbrain/config.json` gets `remote_mcp` set and NO local DB
is created.
**Step 3 — Configure your agent's MCP client.**
For Claude Desktop / Hermes / openclaw, add a single MCP server entry
pointing at the host's `mcp_url` with the bearer token from `register-client`.
Example for Claude Desktop's `~/.config/claude/claude_desktop_config.json`:
```jsonc
{
"mcpServers": {
"gbrain": {
"type": "url",
"url": "https://brain-host.local:3001/mcp",
"headers": { "Authorization": "Bearer <client_secret>" }
}
}
}
```
**Step 4 — Verify.**
```bash
gbrain doctor # runs thin-client checks (no local DB needed)
gbrain remote ping # triggers an autopilot cycle on the host (Tier B)
gbrain remote doctor # asks the host to run its own doctor (Tier B)
```
`gbrain sync` and friends will refuse with a clear thin-client error
naming the `mcp_url`. That's the correct behavior — those commands need
a local engine that doesn't exist here.
### Re-run guard
Running `gbrain init` (no flags) on a machine that already has thin-client
config set refuses without `--force`. This catches the scripted-setup-loop
friction where an orchestrator keeps trying to create a local DB. Use
`gbrain init --mcp-only --force` to refresh thin-client config.
### Storing the OAuth secret
Three storage paths in priority order:
1. **`GBRAIN_REMOTE_CLIENT_SECRET` env var** (preferred for headless agents).
When set, overrides whatever's in the config file. The init flow doesn't
persist a config-file copy when the env var was the source.
2. **`~/.gbrain/config.json` with 0600 perms** (default for interactive
setup; mirrors how Supabase keys are stored today).
3. macOS Keychain integration is on the roadmap; not in v1.
## Topology 3 — Split-engine, per-worktree code + remote artifacts
```
┌──────────────────────────────────────────────────────┐
│ one machine │
│ │
│ ┌─ worktree A ──────────────┐ │
│ │ GBRAIN_HOME=A/.conductor │ │
│ │ gbrain serve --port 3001 │── PGLite (code A) │
│ └───────────────────────────┘ │
│ │
│ ┌─ worktree B ──────────────┐ │
│ │ GBRAIN_HOME=B/.conductor │ │
│ │ gbrain serve --port 3002 │── PGLite (code B) │
│ └───────────────────────────┘ │
│ │
│ ┌─ default ~/.gbrain ───────┐ HTTP MCP / OAuth │
│ │ gbrain serve --port 3000 │──────────────────────→ remote artifacts
│ └───────────────────────────┘ (Supabase / brain-host)
│ │
│ Agent's MCP config (Hermes / Claude Desktop): │
│ mcp__gbrain_code__* → http://localhost:3001 │
│ mcp__gbrain_artifacts__* → http://brain-host/mcp │
└──────────────────────────────────────────────────────┘
```
What you get: each Conductor worktree has its own per-worktree code index
(local PGLite, disposable when the worktree dies). Artifacts (plans,
learnings, transcripts) still live in a shared brain that all worktrees
can see and write to.
When it fits:
- Multiple Conductor worktrees on one machine, all touching the same code
repo.
- You don't want each worktree's code-import to clobber the others'
`last_commit`, source IDs, or symbol tables.
- You DO want artifacts (plans, learnings, retros, transcripts) to be
visible across worktrees.
### How it works
`GBRAIN_HOME` selects which `~/.gbrain` directory is active. Set per worktree:
```bash
export GBRAIN_HOME=/path/to/worktree-A/.conductor/gbrain
gbrain init --pglite
gbrain serve --http --port 3001
```
Each worktree's `gbrain serve` instance binds its own port and indexes its
own DB. Multiple `gbrain serve` processes coexist fine — they're separate
OS processes with separate config and separate connection pools.
The artifact brain runs as a separate `gbrain serve` instance with the
default `~/.gbrain` (no GBRAIN_HOME override) — or remote, in which case
it's a Topology 2 setup.
The agent's MCP client config lists multiple servers, each with a unique
alias. Tool names are namespaced as `mcp__<alias>__<tool>`, so the agent
calls `mcp__gbrain_code__search` for code lookups and `mcp__gbrain_artifacts__search`
for artifact lookups.
### Recommended embedding model
Per-worktree code brains index source files only — no meeting notes,
no people pages, no transcripts. Configure each code brain to use
Voyage's code-tuned model at init time so the config can't be lost to a
later `init` overwrite:
```bash
export GBRAIN_HOME=/path/to/worktree-A/.conductor/gbrain
gbrain init --pglite \
--embedding-model voyage:voyage-code-3 \
--embedding-dimensions 1024
```
`voyage-code-3` is Voyage's code-specialized embedding model with
head-to-head numbers above their general flagships on code retrieval
([voyageai.com/blog](https://voyageai.com/blog)). For already-initialized
brains, switch with the one-command wipe-and-reinit (preserves every
other config field):
```bash
gbrain reinit-pglite --embedding-model voyage:voyage-code-3 --embedding-dimensions 1024
gbrain reindex --code --yes
```
(`gbrain config set embedding_model` is refused as of v0.37.11.0 because
the schema column has to resize alongside the config.)
`gbrain reindex --code` prints a recommendation when the configured
embedding model isn't code-tuned. Suppress with
`GBRAIN_NO_CODE_MODEL_NUDGE=1` if you've intentionally chosen another
provider (single-vendor procurement, compliance, no Voyage key).
### CRITICAL: alias-level routing is manual
Topology 3 has no smart per-tool routing inside gbrain. The agent picks
which brain to query when it picks the alias. **A wrong alias writes (or
queries) the wrong brain silently.** This is intentional (explicit beats
magic) but real:
- If the agent calls `mcp__gbrain_artifacts__put_page` with code-shaped
content, that page lands in the artifact brain forever.
- If the agent calls `mcp__gbrain_code__search` for a question that
actually wants artifact context, the search comes back empty.
Mitigations:
- Name aliases clearly. `gbrain_code` vs `gbrain_artifacts` is unambiguous;
`gbrain` vs `gbrain_local` is not.
- Document in your agent's system prompt or rules which alias goes where.
Be explicit about "code questions → `gbrain_code`; everything else →
`gbrain_artifacts`."
- Pair Topology 3 with `gstack`'s per-worktree wiring (which sets the
alias names + agent rules consistently across worktrees).
### Setup (manual; gstack automates this side)
The gbrain side requires zero new code — `GBRAIN_HOME` and `--port` already
exist. Setup looks like:
```bash
# Start the artifact brain (default ~/.gbrain) on port 3000
gbrain serve --http --port 3000 &
# Start a per-worktree code brain on port 3001
export GBRAIN_HOME=/path/to/worktree-A/.conductor/gbrain
gbrain init --pglite
gbrain serve --http --port 3001 &
unset GBRAIN_HOME
```
Then configure the agent's MCP config with two entries (different aliases,
different ports). For Claude Desktop:
```jsonc
{
"mcpServers": {
"gbrain_artifacts": {
"type": "url",
"url": "http://localhost:3000/mcp",
"headers": { "Authorization": "Bearer <token-A>" }
},
"gbrain_code": {
"type": "url",
"url": "http://localhost:3001/mcp",
"headers": { "Authorization": "Bearer <token-B>" }
}
}
}
```
The gstack-side wiring (per-worktree home setup, port allocation, automatic
MCP config generation, gitignore for the per-worktree DB) is in the gstack
repo's setup-gbrain skill — it composes these primitives, gbrain doesn't
have to know about Conductor.
## Combining topologies
The three shapes compose. A single machine can run:
- A thin-client default config pointing at a remote artifact brain
(Topology 2).
- Plus per-worktree code brains under their own `GBRAIN_HOME` (Topology 3).
- Each worktree's `gbrain serve` instance is local; the agent's MCP config
lists them alongside the remote artifact brain.
`GBRAIN_HOME` controls which config file is active for any one CLI
invocation. `gbrain serve --port` controls which port a server listens on.
The agent's MCP client picks the alias and thus the destination per tool
call. There's no global gbrain orchestrator that knows about all of them
simultaneously — that's by design.
## When NOT to use these topologies
- **Don't use Topology 2 if your agent only ever runs on the same machine
as the brain.** A local `gbrain` install + `gbrain serve` (stdio) is
simpler and faster.
- **Don't use Topology 3 if you only have one Conductor worktree at a
time.** Per-worktree engines exist to prevent contention; one-at-a-time
use has no contention.
- **Don't use a `remote_mcp` thin client AND a local engine on the same
machine in the same `GBRAIN_HOME`.** The dispatch guard refuses DB-bound
commands when `remote_mcp` is set. If you genuinely want both modes on
one machine, use `GBRAIN_HOME` to separate them (one home for the thin
client, another for the local engine).
## See also
- `docs/architecture/brains-and-sources.md` — in-brain organization (brains
vs sources axes).
- `docs/mcp/CLAUDE_DESKTOP.md` and siblings — per-client MCP setup.
- `gbrain init --help` and `gbrain auth --help` for command-level details.
- [`docs/tutorials/`](../tutorials/) — end-to-end walkthroughs that combine
these topologies into working setups (company brain, personal brain,
agent integration, etc.).
-177
View File
@@ -1,177 +0,0 @@
# Type Taxonomy (v0.41.22: gbrain-base-v2)
> The 14-canonical-type DRY/MECE taxonomy shipped in v0.41.22. Predecessor
> `gbrain-base` (24 types) stays bundled for back-compat; v0.42+ installs
> default to `gbrain-base-v2`.
## Why
A production gbrain brain (186K pages) had accreted **94 distinct
`pages.type` values** in 9 clusters of redundancy. The type system is
the foundation for schema packs, search filtering, extract behavior,
enrichment routing, and expert routing. When types are noisy, every
downstream feature degrades:
- **Search filtering is ambiguous**`--type article` misses 2.2K
articles typed as `media/article`, `sources/article`, etc.
- **Enrichment routing is incomplete**`enrichable_types` could only
list a few canonical types; 80+ legacy types meant most pages never
got enriched.
- **Agent confusion** — when ingesting a new article, should it be
`article`, `media/article`, `sources/article`, or `source/article`?
Four reasonable choices, none of them right.
- **Orphan inflation** — 5,521 concept-redirect pages inflated orphan
counts without adding knowledge value.
Issue #1479 catalogues the 9 clusters with exact counts. This doc is
the response: a coherent 14-type taxonomy with subtypes/format/origin
pushed to frontmatter, alias-table rows for redirects, real link-table
rows for edge-shaped pages.
## The 14 canonical types (+ `note` catch-all)
| Type | Primitive | What it holds | Examples |
|------|-----------|---------------|----------|
| `person` | entity | People | Founders, partners, individuals |
| `company` | entity | Companies, products, orgs (subtype-distinguished) | Companies, YC-companies, products |
| `media` | media | Articles, videos, essays, books, podcasts (subtype-distinguished) | Substack posts, YouTube videos, books |
| `tweet` | media | Twitter posts (single/bundle/stub subtype) | Single tweets, threads, bundles |
| `social-digest` | temporal | Period-grouped social summaries (daily/monthly) | X account daily digests |
| `analysis` | media | Research + competitive intel | Market analysis, pricing analysis |
| `atom` | annotation | Knowledge units (extraction/manual/lore subtype) | Extracted facts, manual notes, lore |
| `concept` | concept | Ideas + reference pages | Wiki concepts |
| `source` | media | Transcripts, references | Interview transcripts |
| `deal` | temporal | Investment deals | Term sheets, investments |
| `email` | temporal | Email threads | Email correspondence |
| `slack` | temporal | Slack messages + threads | Slack conversations |
| `writing` | media | Original writing | Drafts, essays in progress |
| `project` | concept | Initiatives, workstreams | Internal projects |
| `note` | concept | **Catch-all** for one-offs (legacy_type preserved) | Memos, anecdotes, insights, etc. |
15 types total (14 canonical + `note`). The catch-all retype rule
binds any uncovered legacy type to `note` with
`frontmatter.legacy_type = <original>` preserved for rollback.
## Subtypes (declared in frontmatter post-unify)
| Canonical | Subtype field | Values |
|-----------|---------------|--------|
| `company` | `subtype` | `company` / `product` / `org` |
| `media` | `subtype` | `video` / `article` / `essay` / `book` / `podcast` / `blog` |
| `tweet` | `subtype` | `single` / `bundle` / `stub` |
| `social-digest` | `subtype` | `daily` / `monthly` |
| `atom` | `subtype` | `extraction` / `manual` / `lore` |
`subtype_field` for retype rules is restricted to an allowlist:
`{subtype, legacy_type, origin, format, kind, period, domain}`. This
prevents third-party packs from injecting `title`, `slug`, or `type`
via mapping_rules (codex D9 security hardening).
## Migration flow
```
gbrain onboard --check # surfaces pack_upgrade_available
gbrain onboard --check --explain # per-cluster narrative dry-run
gbrain jobs submit unify-types \ # PROTECTED + manual_only
--allow-protected \
--params '{"target_pack":"gbrain-base-v2"}'
Handler runs 4 phases:
┌─────────────────────────────────────┐
│ Phase 1: Preflight + lock │ → gbrain-unify db-lock (60min TTL)
├─────────────────────────────────────┤
│ Phase 2: Retype explicit rules │ → chunked UPDATE 1000/batch
├─────────────────────────────────────┤
│ Phase 3: Retype catch-all sentinel │ → 'note' with legacy_type
├─────────────────────────────────────┤
│ Phase 4: Page-to-link conversions │ → insert links + soft-delete
├─────────────────────────────────────┤
│ Phase 5: Page-to-alias conversions │ → insert slug_aliases + soft-delete
├─────────────────────────────────────┤
│ Phase 6: Final sync (residual) │ → path-prefix typing
├─────────────────────────────────────┤
│ Phase 7: Flip active pack (D13) │ → engine.setConfig + saveConfig
├─────────────────────────────────────┤
│ Phase 8: Verify + celebrate │ → assert ≤16 types; stderr summary
└─────────────────────────────────────┘
gbrain onboard --check # pack_upgrade_available cleared
# type_proliferation cleared
```
## Rollback paths
Every primitive ships with a documented rollback:
| Operation | Rollback |
|-----------|----------|
| Retype | `frontmatter.legacy_type = <original>` preserved on every page (D8). One SQL UPDATE restores types: `UPDATE pages SET type = frontmatter->>'legacy_type' WHERE frontmatter ? 'legacy_type'`. |
| Page-to-link | Source page soft-deleted with 72h TTL. `gbrain restore <slug>` within 72h. Link row stays harmless if source restored. |
| Page-to-alias | Source page soft-deleted with 72h TTL. `gbrain restore <slug>` within 72h. Alias row stays harmless (or `DELETE FROM slug_aliases WHERE alias_slug = <slug>` to clean up). |
| Active-pack flip | `gbrain schema use gbrain-base` reverses the flip. |
## What if my brain doesn't fit?
The catch-all retype rule (`from_type: '*unknown*'`) handles long-tail
types automatically — any page whose type isn't covered by an explicit
rule AND isn't a page_to_link / page_to_alias source gets retyped to
`note` with `legacy_type` preserved. Guarantees ≤16 distinct types
post-unify on ANY brain.
For brains with substantial custom types that deserve their own canonical
(e.g. `researcher` for an academic brain), the right move is:
1. Fork gbrain-base-v2: `gbrain schema fork gbrain-base-v2 my-pack`
2. Edit your fork to add page_types + mapping_rules covering your
custom domain.
3. Target your fork: `gbrain jobs submit unify-types --allow-protected
--params '{"target_pack":"my-pack"}'`
Your fork can also declare `migration_from: {pack: gbrain-base-v2,
version: "1.x"}` to register itself as a successor — future agents
discovering your pack via `pack_upgrade_available` will offer the
migration.
## Wikilink resolution post-unify
The slug_aliases table IS the resolver (D15: codex outside voice —
don't rewrite body-text wikilinks; the alias table is the right
primitive). Wikilinks like `[[old-redirect-slug]]` keep working post-
unify because:
1. The wikilink resolver short-circuits through
`engine.resolveSlugWithAlias(slug, sourceId)` BEFORE the existing
fuzzy/prefix cascade.
2. The lookup queries `slug_aliases` for any matching alias_slug in
the provided source(s).
3. If found, returns the canonical_slug. The renderer then resolves
the wikilink to the canonical page.
Multi-source ambiguity (same alias_slug in two registered sources)
emits a once-per-process `multi_match` stderr warning and returns the
first match by source array order. Federated reads pass the full
allowed-source array.
## Search ranking signal: alias_resolved_boost
Post-unify, search results whose slug is a canonical_slug in
slug_aliases get a 1.05x score multiplier via the
`applyAliasResolvedBoost` post-fusion stage. Semantic intent: "user
explicitly disambiguated this as canonical, so it should outrank fuzzy
matches that hit aliases by accident."
`SearchResult.alias_resolved_boost` is stamped on touched results for
`--explain` formatter visibility. KNOBS_HASH_VERSION bumped 5→6 to
invalidate pre-v0.42 cache rows that don't reflect the new stage.
## Reference
- Issue: https://github.com/garrytan/gbrain/issues/1479
- Pack file: `src/core/schema-pack/base/gbrain-base-v2.yaml`
- Pack-upgrade mechanism: `docs/architecture/pack-upgrade-mechanism.md`
- Migration handler: `src/core/schema-pack/unify-types-handler.ts`
- Onboard checks: `src/core/onboard/checks.ts`
- Skill: `skills/schema-unify/SKILL.md`
- Plan + decisions: `~/.claude/plans/system-instruction-you-are-working-transient-elephant.md`
-166
View File
@@ -1,166 +0,0 @@
# gbrain eval suspected-contradictions (v0.32.6)
The contradiction probe samples retrieval results, asks an LLM judge whether
any pair contradicts on a factual claim relevant to the user's query, and
aggregates into a calibrated report. The output is data — the operator
decides what to act on. This doc covers the architecture, severity rubric,
how to interpret the headline number, and when to act.
## Why this exists
gbrain handles contradictions for *curated* pages via compiled-truth-plus-
timeline and source-boost: when `companies/acme.md` says MRR is $2M and a
chat transcript from 2024 says MRR was $50K, the curated page outranks the
chat. `takes.active` filtering hides explicitly-superseded takes. Recency
decay biases ranking toward fresher content per source-tier.
What none of those mechanisms measure: how often do unmarked semantic
contradictions actually surface in retrieval? Without a probe, every
"should we build the bigger swing (chunk-level `revises` field + ranking
change)" decision is vibes. The probe produces evidence.
## Architecture
```
┌──────────────────────────────────────┐
│ gbrain eval suspected-contradictions │
└──────────────────┬───────────────────┘
┌──────────────────▼───────────────────┐
│ For each query: hybridSearch top-K │
│ → cross_slug_chunks + intra_page │
│ chunk-vs-take pairs │
└──────────────────┬───────────────────┘
┌──────────────────▼───────────────────┐
│ Date pre-filter: skip pairs whose │
│ dates are >30d apart (Codex fix: │
│ same-paragraph-dual-date overrides) │
└──────────────────┬───────────────────┘
┌──────────────────▼───────────────────┐
│ Persistent cache lookup │
│ (chunk_a_hash, chunk_b_hash, model, │
│ prompt_version, truncation_policy) │
└────────┬─────────┬────────────────────┘
hit│ │miss
│ ▼
│ ┌─────────────────────────┐
│ │ LLM judge call │
│ │ → JudgeVerdict │
│ │ confidence floor ≥ 0.7 │
│ └─────────┬───────────────┘
│ │
▼ ▼
┌──────────────────────────────────────┐
│ Aggregate per-query + global stats │
│ Wilson 95% CI on headline % │
│ source-tier breakdown │
│ hot pages + resolution proposals │
└──────────────────┬───────────────────┘
ProbeReport JSON
┌──────────────────┼──────────────────────┬───────────────┐
▼ ▼ ▼ ▼
doctor (M1) MCP (M3) synthesize (M2) trend (M5)
surfaces find_contradictions informational persistent
findings op for agents block in prompt tracking
```
## Severity rubric
The judge assigns severity per finding:
| Level | Rubric | Example |
|---|---|---|
| `low` | naming/format differences | "Alice Smith" vs "A. Smith" |
| `medium` | factual values that may be stale | revenue figure, headcount, valuation |
| `high` | identity / structural claims | founder/CEO/CFO role, company status |
Doctor sorts findings by severity DESC. The MCP op accepts a severity filter
so agents can fetch just the high-priority items.
## How to interpret the headline number
The probe outputs `queries_with_contradiction / queries_evaluated` with a
Wilson 95% confidence interval:
```
Queries with >=1 contradiction: 12 / 50 (24%) Wilson CI 95%: 1437%
```
What this says: with 95% confidence, the true rate is between 14% and 37%.
The 24% point estimate is the most-likely-value but bounded by sampling
noise. **`small_sample_note` fires when n < 30** — at that scale the CI is
too wide to act on.
Decision criteria for the bigger swing (chunk-level `revises` field):
| Wilson CI lower bound | What it says | Action |
|---|---|---|
| < 5% | Source-boost + recency-decay + curated pages handle the load | Stop here; this is the right scope |
| 515% | Real but bounded | Operator decides whether the cost justifies the swing |
| > 15% | Real and substantial | Plan the bigger swing in v0.34+ |
## When to act on findings
Each finding ships with a `resolution_command` field — paste-ready:
- `gbrain takes supersede <slug> --row N` — newer take should replace
the older chunk text on the same page (intra_page kind).
- `gbrain dream --phase synthesize --slug <slug>` — compiled_truth for
the curated entity needs an update (cross_slug curated-vs-bulk).
- `gbrain takes mark-debate <slug> --row N` — intentional disagreement
(e.g., two opinions you want to keep both of).
- `# manual review: <a> vs <b>` — judge wasn't sure; operator decides.
Run `gbrain eval suspected-contradictions review --severity high` to
inspect findings without re-running the probe.
## Cost model
Default judge is `claude-haiku-4-5` at ~$1/Mtok in, $5/Mtok out. With
the v0.32.6 truncation at 1500 chars per pair, ~500 input + 80 output
tokens per judge call. Budget cap defaults to $5 in TTY / $1 non-TTY.
- ~$0.0006 per judge call
- ~$0.005 per query (after date pre-filter + cache hits)
- ~$0.50 per 100 queries
The persistent cache means nightly runs against the same query set
pay near-zero on re-runs (until you bump PROMPT_VERSION).
## Trust posture
- Probe never mutates the brain. Runs only read pages/takes/chunks.
Writes go only to `eval_contradictions_runs` and `eval_contradictions_cache`.
- MCP `find_contradictions` is read-scope. NOT in the subagent allowlist —
user-initiated only, not autonomous-action surface.
- Build-fixture script is local-only. The redactor + `isCleanForCommit`
gate makes accidental private-data commits hard, but the operator MUST
inspect every redaction before commit.
## See also
- Plan: `~/.claude/plans/system-instruction-you-are-working-hashed-dewdrop.md`
- CHANGELOG: `## [0.32.6]` entry covers the whole release.
- Cost discipline: `docs/eval-bench.md` for the recommended nightly cadence
+ trend-tracking workflow.
- **Temporal axis follow-on (v0.35.3.1 + v0.35.7):** v0.35.3.1 added a
six-member verdict enum (`no_contradiction | contradiction |
temporal_supersession | temporal_regression | temporal_evolution |
negation_artifact`) and threaded `pages.effective_date` into the judge
prompt so the probe stops crying wolf on legitimate change-over-time.
v0.35.7 lands the trajectory substrate the probe pointed at:
`gbrain eval trajectory <entity>` shows the chronological typed-claim
history with regressions flagged inline; `gbrain founder scorecard
<entity>` rolls up four signals (accuracy, consistency, growth
direction, red flags) into a stable JSON contract. MCP op
`find_trajectory` (read scope, visibility-filtered for remote callers)
exposes the same data to agents. The probe's `temporal_supersession`
verdict and the consolidate phase's `valid_until` writeback both
preserve the `auto-supersession.ts:4` "NEVER auto-applies" invariant
— the probe still emits paste-ready commands, only `consolidate`
writes `valid_until` (R1+R8 grep guard pins this).
-580
View File
@@ -1,580 +0,0 @@
# Embedder Shootout — May 2026 Eval Plan
**Status:** approved, ready to execute
**Owner:** Garry
**Plan source:** `~/.claude/plans/system-instruction-you-are-working-linear-origami.md` (review log)
**Target wallclock:** ~2 weeks
**Target API spend:** ~$525 (hard cap $700)
## What this is
A head-to-head A/B/C comparison of three embedding providers under v0.35.0.0's new
multi-vendor gateway routing:
- **OpenAI** `text-embedding-3-large` @ 1536 dims
- **Voyage** `voyage-4-large` @ 2048 dims
- **ZeroEntropy** `zembed-1` @ 2560 dims (also 1280 in a Matryoshka ablation)
Each tested with and without the `zerank-2` reranker. Two corpora: public LongMemEval
(500q) and BrainBench in-house (145 relational queries + 50 newly-curated Cat 13
embedder-sensitive queries).
The goal: produce a publishable comparison report that answers "which embedder wins,
and does zerank-2 carry the win for ZeroEntropy" with bootstrap p-values, suitable
for a v0.35.2.0 release-note headline.
## Why this design
Locked decisions from the planning review (see plan file + `GSTACK REVIEW REPORT` at
the bottom of the linked plan):
- **Synthetic-only** — LongMemEval (public) + BrainBench (in-house). No `~/.gbrain` data.
- **Answer-gen mode**`gbrain eval longmemeval` runs the default answer-gen path
(Anthropic Sonnet), then feeds the resulting hypothesis JSONL to LongMemEval's
published `evaluate_qa.py` (OpenAI gpt-4o judge) for real correctness numbers.
`--retrieval-only` is NOT used (would produce an attackable headline; the judge
expects answer text, not retrieval text).
- **`tokenmax` search mode** pinned across all cells (expansion + reranker slot active).
- **Serial execution** in one workspace. Clean rate-limit profile; first-contact run on
ZE wants debuggable signal.
- **7-cell matrix** (no matched-dim cross-vendor row — no shared dim exists across
all three vendors; honest framing is "each vendor at marketed sweet spot").
## Architectural facts that constrain the plan
- `content_chunks.embedding vector(N)` dim is fixed per brain. Per-question PGLite in
LongMemEval makes this free; BrainBench needs separate brain per cell.
- pgvector HNSW caps at **2000 dims** (`PGVECTOR_HNSW_VECTOR_MAX_DIMS` in
`src/core/vector-index.ts:19`). Voyage 2048 and ZE 2560 fall back to exact vector
scan. Helps quality (no HNSW approximation) but adds latency. Footnoted in writeup.
- Reranker disable key is **`search.reranker.enabled false`**, NOT `reranker_model none`.
`tokenmax` mode defaults reranker=true.
- `gbrain/ai/gateway` is NOT exported in v0.35.0.0. PR α exposes it.
## Matrix
| Cell | Embedder | Dim | HNSW | Reranker | Notes |
|---|---|---|---|---|---|
| A0 | `openai:text-embedding-3-large` | 1536 | yes | none | OpenAI baseline |
| A1 | `openai:text-embedding-3-large` | 1536 | yes | `zerank-2` | mixed-vendor |
| B0 | `voyage:voyage-4-large` | 2048 | no (exact) | none | Voyage solo |
| B1 | `voyage:voyage-4-large` | 2048 | no (exact) | `zerank-2` | mixed-vendor |
| C0 | `zeroentropyai:zembed-1` | 2560 | no (exact) | none | ZE embedder solo |
| C1 | `zeroentropyai:zembed-1` | 2560 | no (exact) | `zerank-2` | **ZE full stack** |
| C2 | `zeroentropyai:zembed-1` | 1280 | yes | `zerank-2` | ZE-Matryoshka ablation |
## PR structure — as few as possible
**PR α — gbrain repo: v0.35.1.0 infra.** All gbrain changes bundled. Lands first.
Bisect-friendly commits inside, ship at the very end.
**PR β — gbrain-evals repo: adapter + smoke + curation + eval receipts + writeup.** The
big one. Includes the full eval-run output committed alongside the code that produced
it, plus the comparison writeup. Lands when everything is done.
**PR γ (optional) — gbrain repo: v0.35.2.0 release** that cross-links the gbrain-evals
benchmark in CHANGELOG. Small commit; no code changes.
Total: 2 substantive PRs + 1 optional release commit. **No mid-stream ships.**
## Conductor sessions
Each section below is a self-contained brief. Copy-paste into a fresh Conductor session
to hand off. Each session ends with a clean deliverable.
---
## Session 1 — PR α: gbrain infra (v0.35.1.0)
**Repo:** `/Users/garrytan/conductor/workspaces/gbrain/<NEW-WORKSPACE>` (fresh from `master`)
**Branch:** `garrytan/v0.35.1.0-infra`
**Wallclock:** ~2h
**API spend:** $0
### What this session ships
Three changes in one PR, bundled so the embedder shootout in gbrain-evals (PR β) has a
clean prereq baseline:
1. Add `voyage:voyage-4-large` ($0.18/M) and `zeroentropyai:zembed-1` ($0.05/M) to the
embedding pricing table. Patch the `gbrain models doctor` cost estimator + test.
2. Expose `gbrain/ai/gateway` in `package.json` exports map so the gbrain-evals
adapters can call `configureGateway({embedding_model, embedding_dimensions, reranker_model})`
from outside the gbrain process.
3. Add `--resume-from <jsonl>` to `gbrain eval longmemeval` so a mid-run abort
(rate-limit, cost-cap, OS interrupt) doesn't lose the cells we already paid for.
Ships at the end as v0.35.1.0.
### Prereqs (verify before starting)
- On gbrain master at v0.35.0.0 baseline. `cat VERSION` shows `0.35.0.0`.
- `bun test` and `bun run verify` both pass on master.
### Commits (bisect-friendly, one feature per commit)
```
1. feat(pricing): add voyage-4-large + zembed-1 to EMBEDDING_PRICING
- src/core/embedding-pricing.ts: add both entries
- test/embedding-pricing.test.ts: pin both with $0.18 and $0.05
- Verify: bun test test/embedding-pricing.test.ts
2. feat(exports): expose gbrain/ai/gateway with canary test
- package.json: add "./ai/gateway" to exports map
- test/public-exports.test.ts: add canary for configureGateway + embed
- scripts/check-exports-count.sh: 17 -> 18
- Verify: bun run verify
3. feat(eval): add --resume-from <jsonl> to longmemeval
- src/commands/eval-longmemeval.ts: parse flag, skip questions already in input JSONL
- test/eval-longmemeval.test.ts: simulated mid-run abort + resume regression
- Verify: bun test test/eval-longmemeval.test.ts
4. chore: v0.35.1.0
- VERSION: 0.35.1.0
- package.json: 0.35.1.0
- CHANGELOG.md: new entry
- bun install (refresh lockfile)
```
### Verify before /ship
```bash
bun run typecheck
bun run verify
bun test test/embedding-pricing.test.ts test/public-exports.test.ts test/eval-longmemeval.test.ts
```
### Ship
```bash
/ship
```
### Deliverable
- `master` of gbrain at v0.35.1.0
- `gbrain/ai/gateway` reachable from external consumers (verified by canary test)
- `git tag eval-run-v0.35.1.0-baseline` (annotated, names this exact commit)
- `gbrain --version` prints `0.35.1.0`
### Hand-off to Session 2
- gbrain-evals can now `bun update gbrain` to v0.35.1.0
- The tag preserves the exact commit for any future reproducibility need
---
## Session 2 — PR β setup: gbrain-evals adapter + smoke + subset flag
**Repo:** `/Users/garrytan/git/gbrain-evals` (or a fresh Conductor workspace cloned from it)
**Branch:** `garrytan/embedder-shootout`
**Wallclock:** ~3-4h
**API spend:** ~$0.10 (smoke verification calls only)
### What this session ships into PR β (does NOT merge yet)
Wire the harness to drive 3 embedding providers via the newly-exposed gbrain gateway:
1. New typed `EvalAdapterConfig {embedder, dim, reranker?}` passed into each adapter.
2. Rewrite `vector.ts` + `hybrid-rrf.ts` to call `configureGateway()` from
`gbrain/ai/gateway` instead of the hardcoded `gbrain/embedding` import.
3. Critical: hybrid adapter must also route `search.reranker.enabled` (true/false) and
`search.mode` (tokenmax) — codex flagged that the existing hybrid never sets these.
4. New 3-phase smoke harness: wiring (5 queries × embed roundtrip + dim check) +
long-haystack (1 query × 50K-token synthetic haystack) + rerank-payload (1 query
× `topNIn=30`). Exit code is the gate.
5. New `--include-subset <name>` flag on the BrainBench runner (Cat 13 wiring; subset
itself comes in Session 3).
### Prereqs
- Session 1 done. gbrain master at v0.35.1.0.
- API keys present: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `VOYAGE_API_KEY`,
`ZEROENTROPY_API_KEY`. Smoke fails-loud on missing key.
### Commits
```
1. chore(deps): bump gbrain pin to v0.35.1.0
- package.json + bun.lock
- Verify: bun install && bun run typecheck
2. feat(adapter): typed EvalAdapterConfig + gateway swap
- NEW: eval/runner/eval-adapter-config.ts (the type)
- eval/runner/adapters/vector.ts: constructor takes EvalAdapterConfig,
calls configureGateway({embedding_model, embedding_dimensions})
- Drop hardcoded gbrain/embedding import
- Verify: existing vector adapter unit tests still pass
3. feat(adapter): hybrid-rrf wires reranker_enabled + search.mode
- eval/runner/adapters/hybrid-rrf.ts: constructor takes EvalAdapterConfig,
plumbs search.reranker.enabled + search.mode = tokenmax through
- Verify: bun test eval/
4. feat(smoke): 3-phase smoke harness
- NEW: eval/runner/smoke.ts (CLI entry: bun run eval:smoke -- --embedder X --dim Y [--reranker Z])
- Phase 1: 5 queries × embed roundtrip, assert vector dim matches config
- Phase 2: 1 query × synthetic 50K-token haystack, assert no token-limit error
- Phase 3: 1 query × topNIn=30 documents, assert no 5MB payload cap hit
- Non-zero exit on any failure
- Verify: bun run eval:smoke -- --embedder openai:text-embedding-3-large --dim 1536
5. feat(runner): --include-subset flag for BrainBench
- eval/runner/multi-adapter.ts: parse flag, filter queries by subset tag
- Subset itself comes in next commit (Session 3)
- Verify: bun run eval:run -- --include-subset cat13-embedder (errors politely because subset file doesn't exist yet)
```
### Smoke verification (run manually before opening PR)
```bash
bun run eval:smoke -- --embedder openai:text-embedding-3-large --dim 1536
bun run eval:smoke -- --embedder voyage:voyage-4-large --dim 2048
bun run eval:smoke -- --embedder zeroentropyai:zembed-1 --dim 2560
bun run eval:smoke -- --embedder zeroentropyai:zembed-1 --dim 2560 --reranker zeroentropyai:zerank-2
```
All four MUST exit 0. Reports should print the observed vector dim, matching the
configured dim.
### Open PR β
```bash
gh pr create --base main --title "feat: embedder shootout (adapter + smoke + Cat 13 + eval receipts)" --body "$(cat <<'EOF'
## Summary
v0.35.0.0 shipped ZeroEntropy zembed-1 + zerank-2 reranker support. This PR runs a head-to-head A/B/C comparison across OpenAI, Voyage, and ZeroEntropy under the new gateway routing.
This first commit batch lands the harness. Cat 13 curation, Phase 1+2 evals, and the
writeup follow in subsequent commits to this same PR.
## Test plan
- [x] Adapter unit tests pass
- [x] Smoke harness exits 0 against all 3 providers
- [ ] Cat 13 subset committed (Session 3)
- [ ] LongMemEval x 7 cells run (Session 4)
- [ ] BrainBench x 7 cells run (Session 5)
- [ ] Writeup committed (Session 5)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
EOF
)"
```
### Deliverable
- PR β open against gbrain-evals `main`, green CI
- Smoke verified against all 3 providers (paste the smoke output in the PR body)
- Branch ready for Session 3 (Cat 13 curation)
### Hand-off to Session 3
- Branch `garrytan/embedder-shootout` exists on origin
- The `--include-subset cat13-embedder` flag is wired but the subset file doesn't exist
yet — that's Session 3
---
## Session 3 — PR β: Cat 13 conceptual-recall curation
**Repo:** `/Users/garrytan/git/gbrain-evals`, branch `garrytan/embedder-shootout` (same as Session 2)
**Wallclock:** ~3-4h (heavily user-interactive; AI proposes, you review each)
**API spend:** $0
### What this session ships into PR β
Hand-curated 50 embedder-sensitive queries from BrainBench's Cat 13 (conceptual recall)
corpus. These are the queries where a graph/keyword adapter would likely miss but a
semantic adapter would find.
Codex flagged the existing 145-query relational corpus as graph/keyword-dominated and
weak for embedder claims. Cat 13 is closer to the embedder-sensitive workload but
needs hand-selection.
### Prereqs
- Session 2 done. PR β open with adapter + smoke + subset flag.
### Workflow
Interactive: Claude proposes queries in batches of 10, you accept/reject/edit each.
1. Claude reads the existing Cat 13 raw query pool:
```bash
ls eval/data/raw/ | grep -i cat13
cat eval/data/raw/cat13-*.json | jq '.'
```
2. Claude proposes 10 candidate queries per batch, each tagged with the inclusion
reasoning ("would a graph adapter miss this?")
3. User accepts/rejects/edits inline. Target: 50 queries × ~5 batches.
4. Claude commits to `eval/data/gold/brainbench-cat13-embedder-subset.json`:
```json
{
"schema_version": 1,
"subset": "cat13-embedder",
"queries": [
{
"id": "cat13-emb-001",
"query": "...",
"relevant_chunk_ids": ["..."],
"inclusion_reason": "paraphrase relationship; graph adapter wouldn't catch the synonym"
}
// ... 49 more
]
}
```
### Commit
```
feat(eval): curate Cat 13 conceptual-recall subset (50 embedder-sensitive queries)
- NEW: eval/data/gold/brainbench-cat13-embedder-subset.json
- Each query tagged with inclusion_reason for future audit
```
### Spot-check before commit
- Pick 5 random queries, run them against a hypothetical graph adapter (e.g. grep on
the relevant terms) and verify they would NOT surface the right chunk.
- Run the same 5 against the existing hybrid adapter and verify they DO.
### Deliverable
- `eval/data/gold/brainbench-cat13-embedder-subset.json` committed to PR β
- Exactly 50 queries
- Spot-check evidence in the commit message
### Hand-off to Session 4
- PR β now has: adapter + smoke + Cat 13 subset
- Ready for the actual eval runs
---
## Session 4 — PR β Phase 1: LongMemEval × 7 cells (overnight)
**Repo:** Same gbrain-evals branch
**Wallclock:** ~10.5h (mostly hands-off, kick off and walk away)
**API spend:** ~$476 (LongMemEval-heavy; 7 × $68/cell)
### What this session ships into PR β
7 LongMemEval scored receipts (one per matrix cell). Each is a JSONL of 500
hypotheses + a JSON file of correctness scores from `evaluate_qa.py`.
### Prereqs
- Sessions 1+2+3 done. PR β has adapter + smoke + Cat 13.
- LongMemEval dataset downloaded (gated HuggingFace; one-time setup).
- `evaluate_qa.py` checked out somewhere (from
https://github.com/xiaowu0162/LongMemEval) with its own venv set up.
- API keys: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `VOYAGE_API_KEY`,
`ZEROENTROPY_API_KEY`.
### Wrapper script
Claude writes `scripts/run-shootout-phase1.sh` in the gbrain-evals branch. Single
entry point that loops the 7 cells serially with smoke gating + cost-cap aborts.
```
NEW: scripts/run-shootout-phase1.sh
- Per cell: gbrain config set (embedder, dim, reranker, search.reranker.enabled, search.mode=tokenmax)
- Per cell: bun run eval:smoke (abort cell on non-zero)
- Per cell: gbrain eval longmemeval ... --output results/longmemeval-{cell}.jsonl
- Per cell: cost-cap check ($90/cell hard stop)
- Per cell: --resume-from existing results/longmemeval-{cell}.jsonl if present
- Logs to results/phase1-run-log.txt
```
### Run
```bash
# Kick off in background; check back in 10-12h
bash scripts/run-shootout-phase1.sh 2>&1 | tee results/phase1-run-log.txt &
```
Use `run_in_background: true` if running through Claude. Check back periodically.
### Scoring (after all 7 cells done)
```bash
for cell in A0 A1 B0 B1 C0 C1 C2; do
python evaluate_qa.py \
--input results/longmemeval-${cell}.jsonl \
--output results/longmemeval-${cell}-scored.json
done
```
Each scored file has correctness %.
### Commits
```
1. feat(scripts): Phase 1 LongMemEval wrapper with smoke gating + cost cap
- NEW: scripts/run-shootout-phase1.sh
2. data(phase1): 7 LongMemEval cells (raw hypothesis JSONL)
- results/longmemeval-{A0,A1,B0,B1,C0,C1,C2}.jsonl
- results/phase1-run-log.txt (run timing + cost ledger)
3. data(phase1): evaluate_qa.py scoring results
- results/longmemeval-{cell}-scored.json × 7
```
### Verify
- Each `longmemeval-{cell}.jsonl` has exactly 500 lines
- Each `hypothesis` field is non-empty AND is actual answer text (NOT retrieval text)
- Each `scored.json` has a `correctness_score` field
### Deliverable
- 7 scored LongMemEval receipts committed to PR β
- Real cost ledger committed alongside (compare against estimate)
### Hand-off to Session 5
- Phase 1 done. Phase 2 (BrainBench, ~3.5h) and writeup remaining.
---
## Session 5 — PR β Phase 2 + writeup + ship
**Repo:** Same gbrain-evals branch
**Wallclock:** ~7h (3.5h BrainBench + 3h writeup + /ship)
**API spend:** ~$56 (BrainBench is cheap)
### What this session ships into PR β
- 7 BrainBench cells (relational corpus + Cat 13 subset)
- Final comparison writeup
- PR β merged
### Prereqs
- Session 4 done. PR β has Phase 1 receipts.
### Phase 2 wrapper script
```
NEW: scripts/run-shootout-phase2.sh
- Per cell: configure provider (same as Phase 1)
- Per cell: bun run eval:run -- --N 10 --include-subset cat13-embedder
--output docs/benchmarks/2026-05-22-{cell}.md
- Cost-cap check
```
### Run
```bash
bash scripts/run-shootout-phase2.sh 2>&1 | tee results/phase2-run-log.txt
```
### Writeup
`docs/benchmarks/2026-05-22-embedder-shootout.md`. Structure:
1. **Headline table** — 7 cells × {LongMemEval correctness %, BrainBench relational MRR + P@5, Cat 13 correctness %, total cost}
2. **Two questions answered:**
- Which embedder wins solo? (A0 vs B0 vs C0)
- Does zerank-2 carry ZE's win? (C0 vs C1 vs A1 vs B1)
- Bonus: does dim matter for ZE? (C1 vs C2)
3. **Paired-bootstrap p-values** per headline pair (methodology in
`gbrain/docs/eval/SEARCH_MODE_METHODOLOGY.md`)
4. **HNSW footnote** — Voyage 2048 and ZE 2560 used exact vector scan; OpenAI 1536
and ZE 1280 used HNSW. Quality is primary, latency is secondary
5. **What this does NOT prove** — synthetic-only, tokenmax-only, no real-brain replay
6. **Recommendation:** explicit NON-recommendation to change `gbrain init` default;
defer to a v0.36.x evidence pass with real-brain replay data
### Commits
```
1. feat(scripts): Phase 2 BrainBench wrapper
- NEW: scripts/run-shootout-phase2.sh
2. data(phase2): 7 BrainBench cells
- docs/benchmarks/2026-05-22-{cell}.md × 7
3. docs(benchmark): embedder shootout comparison writeup
- NEW: docs/benchmarks/2026-05-22-embedder-shootout.md
- Bootstrap p-values, HNSW footnote, NOT-in-scope section
```
### Ship
```bash
# Merge PR β to gbrain-evals main
gh pr merge --squash --auto
# Or non-auto if reviewing one more time:
gh pr merge --squash
```
### Deliverable
- PR β merged to gbrain-evals `main`
- Comparison report public at
`gbrain-evals/docs/benchmarks/2026-05-22-embedder-shootout.md`
### Hand-off to Session 6 (optional)
- gbrain-evals master has the full data + writeup
- Ready for a v0.35.2.0 gbrain release that cross-links it
---
## Session 6 (optional) — PR γ: gbrain v0.35.2.0 release
**Repo:** `/Users/garrytan/conductor/workspaces/gbrain/<NEW-WORKSPACE>` (fresh from master)
**Branch:** `garrytan/v0.35.2.0-benchmark-release`
**Wallclock:** ~30min
**API spend:** $0
### What this session ships
A release-notes-only PR that bumps gbrain to v0.35.2.0 with a CHANGELOG entry
cross-linking the embedder shootout benchmark. Optional — could be folded into the
next routine release if no rush.
### Prereqs
- Session 5 done. gbrain-evals merged with the comparison writeup.
### Commits
```
1. docs(benchmark): mirror embedder shootout summary
- NEW: docs/benchmarks/2026-05-22-embedder-shootout.md (slim mirror)
- Cross-link to gbrain-evals canonical version
2. chore: v0.35.2.0
- VERSION: 0.35.2.0
- package.json: 0.35.2.0
- CHANGELOG.md: new entry with the GStack-voice release summary
+ "numbers that matter" table from the benchmark
```
### Ship
```bash
/ship
```
### Deliverable
- gbrain v0.35.2.0 on master
- CHANGELOG entry that drives the release-note headline
---
## Cost ledger (revised, post-review)
| Component | Per cell | × 7 cells |
|---|---|---|
| LongMemEval embed | <$0.05 | <$0.35 |
| LongMemEval Sonnet answer-gen (500q × 2K tokens × $3/M) | $18 | $126 |
| LongMemEval gpt-4o judge (500q × $0.10/q) | $50 | $350 |
| BrainBench relational embed | $0.05-0.18 | <$1 |
| BrainBench Cat 13 answer-gen + judge (50q × $0.14) | $7 | $49 |
| Smoke harness (30 calls/cell) | <$0.10 | <$1 |
| **Total** | **~$75/cell** | **~$525** |
**Hard cap: $700.** Per-cell hard cap: $90 (wrapper aborts cell if exceeded; partial
JSONL preserved for resume).
## Failure modes and recovery
| Failure | Recovery |
|---|---|
| Voyage/ZE 429 rate-limit mid-cell | `gateway._shrinkState` halves safety_factor and retries. Cell continues. |
| ZE 5MB rerank payload cap hit | `applyReranker` fail-opens, returns un-reranked results. Stderr warn. |
| Mid-cell OS interrupt / cost-cap abort | Re-run with `gbrain eval longmemeval --resume-from results/longmemeval-{cell}.jsonl`. Picks up where it left off. |
| `evaluate_qa.py` auth fail | OPENAI_API_KEY check in wrapper aborts before any spend. |
| Adapter typo (bad dim) | `EvalAdapterConfig` runtime assertion at constructor throws AIConfigError. Cell aborts before API call. |
## NOT in scope (deliberate)
- **Real `~/.gbrain` replay** — adds 6-12h wallclock + $40-80 embed. Filed as v0.36.x.
- **All 3 search modes** — pinned to tokenmax. `conservative` + `balanced` are v0.35.3.0
follow-ups if reviewers push back.
- **Matched-dim cross-vendor row** — no shared dim exists across all 3 vendors.
Permanently out.
- **`gbrain eval whoknows` / `cross-modal` / `takes-quality`** — embedding-invariant;
rerunning across embedders produces noise.
- **`gbrain eval code-retrieval`** — code corpus, separate concern.
- **`gbrain eval suspected-contradictions`** — wants a real brain.
- **`gbrain init --recommended` default change** — codex correctly flagged the evidence
base as insufficient. Defer to v0.36.x with real-brain replay data.
## What already exists (reused, not rebuilt)
- `gbrain eval longmemeval` CLI (in-tree, answer-gen mode default)
- gbrain-evals BrainBench runner (`eval:run`) — needs adapter parameterization but
per-cell test plumbing is reused
- Gateway routing for Voyage + ZE (shipped v0.35.0.0)
- Reranker pipeline (`src/core/search/rerank.ts`, fail-open)
- Pricing table (extended, not rebuilt)
- Paired-bootstrap methodology (`docs/eval/SEARCH_MODE_METHODOLOGY.md`)
- LongMemEval published `evaluate_qa.py` (invoked externally, not bundled)
-367
View File
@@ -1,367 +0,0 @@
# Community Ideas Ledger
> A diary of the **valuable ideas** surfaced by the community-PR wave, kept so that
> good thinking survives even when the PR that carried it is closed. gbrain moves
> fast and the maintainer's "cathedral" rewrites supersede most individual PRs —
> but the *idea* behind a closed PR is often still worth something.
>
> **Bar for this file:** an idea only earns a line if it is (a) still live on
> master and (b) genuinely valuable to gbrain users. **Graduating an idea to
> `TODOS.md` is a higher bar still** — it must serve the North Star (next-Postgres-
> for-memory: widest coverage, best-for-the-most-at-the-least) and be worth a
> maintainer-owned implementation. Most lines here will never graduate. That's fine.
>
> Status legend: **OPEN** = PR still open as a real merge candidate · **CLOSED** =
> PR closed, idea captured here · **HELD** = strategic, awaiting maintainer call.
> Provenance is credited to the contributor; scrub real private-network names per
> the repo privacy rule when anything here graduates to a public artifact.
_Generated from a full triage of the open-PR backlog (436 community PRs), 2026-06-07._
---
## 1. Internationalization — non-English brains are second-class
The single biggest coverage gap for "serve a billion people." Several independent
contributors hit the same walls.
- **Configurable FTS language** (#580/#581/#582, @rafaelreis-r) — **OPEN, high.**
Every `to_tsvector`/`tsquery` is hardcoded `'english'` (query side, trigger side,
and no reindex path), so non-English brains run every search through the English
stemmer. A coherent 3-PR set: `GBRAIN_FTS_LANGUAGE` config → migration recreating
triggers with the chosen language → `gbrain reindex-search-vector` to change it
post-install. **Strongest i18n candidate to graduate.**
- **Full-Unicode slugs** (#782, @tamagodo-fu; #514 zh, @JimmyJiang67) — **HELD, high.**
CJK slugs already work (`CJK_SLUG_CHARS`); generalize to all scripts (Cyrillic,
Devanagari, Hangul, …) and widen the remaining ASCII-only validators so non-ASCII
slugs flow end-to-end instead of being generated then rejected. #514 also carries a
corpus-driven `relationships-zh.json` verb dictionary for `inferLinkType` — a
reusable artifact for Chinese relationship typing.
- **CJK entity extraction** (#1637, @alkalide) — **OPEN, high.** Mention extraction is
ASCII-only (`TOKEN_RE`, `MIN_NAME_LENGTH=4`), so 23 char Chinese/Japanese/Korean
names are invisible to the gazetteer (there's an in-code TODO acknowledging it).
CJK detection + lower min-length + single-token pure-CJK titles + substring pass.
## 2. Reliability — the daily-driver failure modes
Recurring, production-observed failures. Many are tiny fixes with outsized impact;
these are the densest source of real bugs in the whole backlog.
- **Embedding egress waste** (#347/#460, @notjbg) — **OPEN, high.** `getChunks` does
`SELECT cc.*`, shipping the ~6KB pgvector embedding that `rowToChunk` immediately
discards — ~1922 GB/day egress on a busy Supabase brain. Enumerate the columns;
add a CI guard. (#460 dup of #347.)
- **Body-keyed embedding reuse** (#1424, @defenestrate2) — **OPEN, high.** Markdown
import re-embeds byte-identical chunks that merely shifted position, turning a
cosmetic edit into ~99K wasted re-embeds. Reuse by chunk-text hash like the code
path already does; add `--force` + a no-hash sentinel.
- **`embed --stale` full re-pull** (#775, @kyledeanjackson) — **CLOSED (partial on
master), high.** Re-pulled all chunks every cycle (~3TB/mo egress); steady-state
brains should do near-zero work. Master added a `countStaleChunks` early-exit;
verify it fully closes this.
- **Config round-trip storm** (#1694, @Omerbahari) — **OPEN, high.** A single query
fires ~85 serial single-key config `SELECT`s — invisible on PGLite, ~85 network
RTTs on a remote pooler. Batch + cache `getConfig` (`getConfigMany`).
- **cgroup-aware worker sizing** (#1244, @tyler3k1) — **OPEN, high.** `defaultWorkers()`
sizes from `os.totalmem()` (host RAM), so containerized installs (Railway/Fly/Render/
Cloud Run/ECS) oversize the pool and get OOM-killed mid-import. Use
`process.constrainedMemory()`.
- **Linux memory-pressure throttle** (#556, @chengzehsu) — **OPEN, high.** `os.freemem()`
is `MemFree` (excludes reclaimable cache), so healthy containers reject every batch
job. Read `MemAvailable` from `/proc/meminfo`.
- **propose_takes never caches empties** (#1218 @AdityaRajeshGadgil / #1760 @notjbg) —
**OPEN, high.** A valid `[]` extractor result writes no cache row, so unchanged pages
re-spend extractor tokens every ~5min cycle (57,885 calls/11 days observed). Sentinel
row keyed on `(source_id, page_slug, content_hash, prompt_version)`.
- **Prompt-cache opt-in on hot paths** (#1761, @notjbg) — **OPEN, high.** Only ~4.9% of
input tokens hit the Anthropic prompt cache because the highest-volume cycle/extraction
call sites don't set `cacheSystem:true` despite gateway support. One-line opt-ins.
- **Autopilot reliability cluster** (#232 @ianderse, #464/#465 @notjbg, #289 @RyanAlberts,
#477 @vinsew, #1935/#1936 @mdcruz88, #1906/#1891 @rayers/@jalagrange) — **OPEN, high.**
A family of distinct live bugs: argless `engine.connect()` wipes saved config and
crash-loops under launchd; `cwd=/` wrappers miss `brain/.env`; mtime-only lock probing
blocks respawn for 10min after OOM; no backoff on the 5-failure suicide cap;
disconnect-before-connect `reconnect()` bricks the engine on a transient blip; config
accessors lack the retry wrapper. **Pick the best fix per layer and land as a wave.**
- **lint `--fix` corrupts mid-doc fences** (#1417 @trinh-macbook, #1597 @chungty) —
**OPEN, high.** Detector/fixer regex disagree, so `lint --fix` strips the closing fence
of mid-document ```` ```markdown ```` blocks and autopilot re-corrupts the page every
cycle. Only unwrap whole-page fences.
- **backlinks worker defaults to `fix`** (#1853 @choomz; #1027 @sliday; #495 @23salus) —
**OPEN, high.** Empty-payload backlinks jobs default to `action='fix'`, silently
rewriting tracked markdown ("Referenced in" bullets) on every sync→embed→backlinks
chain (129 files/day in the wild). Default to `check`; require explicit opt-in. Also
fixes a duplicate-line accumulation bug.
- **`DATABASE_URL` hijack** (#1884, @awilkinson) — **OPEN, high.** A co-located app's
generic `DATABASE_URL` silently overrides the configured brain (wrong DB, or
auto-migrates it). Fix precedence: `GBRAIN_DATABASE_URL` > config.json > `DATABASE_URL`.
- **Engine-switch strips config** (#1088, @samchaudhary) — **OPEN, high.** `migrate --to`
rewrites config to just `{engine,url}`, dropping `embedding_model`/`dimensions`/keys;
migration "succeeds" but new embeds break.
- **Re-init silently corrupts the brain** (#1060, @vincedk-alt) — **OPEN, high.** Flag-less
re-init ignores persisted `embedding_model`/`dimensions` and writes a wrong-shape
OpenAI-1536 brain before the dim-check catches it.
- **IPv6-only direct URL** (#1006, @diazMelgarejo) — **OPEN, high.** `deriveDirectUrl`
turns a Session-Pooler URL into an IPv6-only host, ECONNREFUSED on IPv4-only networks
(the majority). Return null for pooler URLs.
- **HOME-isolation in tests** (#205/#517/#534 @orendi84, #434 @lloydarmbrust) — **OPEN,
high.** The E2E suite spawns `gbrain init/import` against the developer's real
`~/.gbrain/config.json`, clobbering their live DB URL+keys. Isolate HOME to a tmpdir.
*(A footgun that bites contributors of this very repo.)*
- **dim-aware embed write target** (#1263, @DmitryBMsk) — **OPEN, high.** `upsertChunks`
always writes the legacy `embedding vector(1536)` column, so brains on an alternate
column (`embedding_ze halfvec(2560)`) fail with dim-mismatch on every write.
- **Oversized chunks silently unembedded** (#1675, @lubos-buracinsky) — **OPEN, high.**
The code chunker emits giant literals/template strings whole; the embedder rejects
them and they vanish from semantic search. Cap chunk size so they stay embeddable.
- **Token-vs-char truncation** (#557 @chengzehsu, #990 @mgunnin, #1180 @kkroo,
#1281 @mmekkaoui, #1947 @100menotu001) — **OPEN, high.** The embed path truncates by
chars (`MAX_CHARS`) not tokens, so dense pages still exceed the 8192/300K-token ceiling
and loop forever on HTTP 400 with `embedded_at` never cleared; `isTokenLimitError`
misses OpenAI's real error string; llama-server's 32-input limit isn't capped; and
`--catch-up`'s unbounded budget overflows the 32-bit `setTimeout` and aborts after one
batch. A "make embedding backfills never silently wedge" cluster.
## 3. Search & retrieval quality
- **Keyword search ignores page titles** (#1646, @jeades) — **OPEN, high.** `searchKeyword`
ranks only chunk `search_vector`, never `pages.search_vector` (weight-A titles), so an
exact-title `gbrain search` returns nothing while `query` finds it. High-impact, tiny.
- **`code-def` misses most OO symbols** (#1628, @rayers) — **OPEN, high.** `DEF_TYPES`
omits method/constructor/field/struct/protocol, so `code-def` returns 0 for most
object-oriented code. Root-cause fix in `normalizeSymbolType` + `DEF_TYPES`.
(Prefer over #1701's fallback-only approach.)
- **doc-comment column is wired but dead** (#520, @Evode-Manirahari) — **OPEN, high.** FTS
weights `content_chunks.doc_comment` above chunk text but the column is never populated.
Extract JSDoc/docstrings per symbol via AST and thread through import.
- **autocut weak-top collapse** (#1863, @rayers) — **OPEN, high.** The fresh autocut
feature (#1682) normalizes the rerank gap by the top score, so a weak top (0.317→1.0)
looks like a confident cliff and rare cross-source queries collapse to 1 result. Add a
`minTopScore` floor.
- **Graph-hop wikilink rerank** (#717, @gwanghoon91) — **HELD, high.** Zero-token
score-shapers (graph-hop wikilink rerank + query-token disambiguation) claimed
+2.6/+2.8pt P@5/R@5 on BrainBench. Worth re-evaluating against the new retrieval
cathedral's ranker rather than merging the old diff.
- **Effective-date time filters** (#1706, @mvanhorn) — **OPEN, med.** `since`/`until`
filter on `updated_at`, so content dated to the past but edited recently is mis-filtered;
filter on `COALESCE(effective_date, updated_at, created_at)`.
## 4. Extraction & the knowledge graph
- **Obsidian wikilink → typed graph edges** (#87 @franmaranchello; alias/title/basename
fallback #1188 @rwbaker) — **OPEN/HELD, high.** `[[wikilinks]]`/`![[embeds]]` are
invisible to the graph. Materialize them as typed edges with alias (frontmatter
`aliases:`), first-H1-title, and basename fallback resolution (path-equality-only gives
~5.5% edge recall on real vaults). Master shipped global-basename (#1388); the alias/
title fallbacks are the still-novel part.
- **Schema-pack-aware link extraction** (#1547, @billy-armstrong) — **OPEN, high.** The
link extractor's `DIR_PATTERN` is a frozen 16-prefix const that ignores pack-declared
`path_prefixes`, so default-pack installs silently lose wikilinks to `person/`,
`writing/`, `wiki/*`. Resolve prefixes from the active pack.
- **DB-source extraction** (#1539, @afshaker) — **OPEN, high.** The cycle's extract phase
only walks the filesystem, so DB-resident pages (imported transcripts, remote-DB brains)
never get links/timeline and `brain_score` is capped. Thread `source:'db'`.
- **source_id threaded through fs-walk extract** (#1719, @seungsu-kr) — **OPEN, high.**
fs-walk extractors omit `source_id`, defaulting to `'default'`, so the `pages` INNER JOIN
drops every row on non-default-source brains — silent 0 inserted.
- **extract `--stale` permanent-lag loop** (#1791, @Nazim22) — **OPEN, high.** Pages last
edited before the link-extractor version bump get stamped below the version threshold and
re-flag every run (~97% pages permanently "stale"). Stamp `GREATEST(updated_at, versionTs)`.
- **Plain-text NER for auto-link** (#1565, @donogeme) — **HELD, med.** Plain mentions of
people (no `[[wikilink]]`) never become edges. The opt-in idea is right; the shipped
implementation (capitalized-bigram regex, Western-names-only) is too crude — needs a
real NER pass to clear the graph-integrity bar.
## 5. Providers & the gateway
The AI-gateway + recipes + `user_provided_models` system already absorbed ~40
per-vendor embedding PRs (Ollama, Gemini, Azure, DashScope, DeepSeek, Zhipu, E5,
bge-m3, Copilot, Composio, Kimi, LM Studio, Mistral, Hunyuan, MiniMax…). The
*residue* worth keeping:
- **litellm proxy unusable for chat** (#1953 @miroslavb, #1938 @BKF-Gitty) — **OPEN, high.**
The `litellm-proxy` recipe declares only an embedding touchpoint (no chat), so
`chat_model=litellm:*` fails validation and `think` degrades to a misleading "set
ANTHROPIC_API_KEY"; and `build-gateway-config` never folds `litellm/openrouter/together`
keys, so configured proxy auth goes out unauthenticated. Plus user-provided custom-dim
embeddings are double-false-rejected in preflight. **The general-OpenAI-compat-proxy
story.**
- **Matryoshka dims threading** (#1072 @mgandal, #1240 @mike7seven) — **OPEN, high.**
Qwen3-Embedding returns its native dim (2560/4096) not the requested one because
`dimensions:N` isn't threaded for the openai-compat path, hard-failing a 1536-dim brain.
- **"Freeze provider at init, clear vectors on dim change"** (#100/#172, @niallobrien/
@nbzy1995) — **CLOSED, med.** A safety insight worth keeping even though the provider
PRs are superseded: persist+freeze the brain's provider/dim at init so a later env change
can't silently corrupt the vector space; clear stale embeddings on an intentional change.
- **China-region provider coverage** (#59 @Magicray1217, #1071 @AzeWZ) — **CLOSED, med.**
Make DashScope/DeepSeek/Zhipu first-class recipes that honor `provider_base_urls` (the
China-region endpoints) and provider batch limits — on-mission for global coverage.
- **Amazon Bedrock native** (#1826, @naterchrdsn) / **Jina asymmetric retrieval**
(#1930, @Whamp) — **HELD, high/med.** The maintainer pattern prefers the universal
litellm-proxy over per-vendor native recipes, but Bedrock (AWS IAM credential chain) and
Jina's asymmetric `input_type=document|query` are distinct enough to warrant a call.
- **Local-first chat parity** (#1854/#1855/#1858 @starm2010, #1423 @pabloglzg,
#1618 @punksterlabs) — **OPEN, high.** `FREE_LOCAL_CHAT_PROVIDERS` doesn't exist (only
embed), brainstorm/cycle/takes hardcode `anthropic:claude-sonnet-4-6`, and the
openai-compat `generateObject` path silently fails on providers that reject
`json_schema`. The "run gbrain fully local" cluster.
- **OpenRouter config key** (#1714 @tmchow), **OAuth bearer for AI providers**
(#1312 @pabloglzg), **API-key files** (#570 @shawnduggan) — **OPEN, med.** Credential
ergonomics: config-file key (not just env), externally-minted bearer tokens, and
`OPENAI_API_KEY_FILE` so OAuth harnesses don't inherit a raw key in `process.env`.
## 6. Auth, federation & access control (security-adjacent)
These cluster into a real theme: **runtime access control for remote/multi-tenant MCP
beyond prompt discipline.** Several are live security gaps (see the security list in the
triage report) and should be treated as a coordinated design, not piecemeal merges.
- **Clamp remote source overrides** (#1372, @jlfetter1) — **OPEN, high, SECURITY.** A
remote MCP caller can pass `source_id` (or `__all__`) to `query`/`get_page` to read
sources outside their OAuth `allowedSources` — the param bypasses `sourceScopeOpts`
(CWE-285). Clamp to token claims, fail-closed. **#1394 (get_page source_id) must land
*with* this clamp, not before it.**
- **Read-side prefix/federation enforcement** (#1860 @choomz, #1790 @colin-atlas,
#470 @AdityaRajeshGadgil, #1508 @tim404x) — **OPEN, high.** `bound_slug_prefixes` is
enforced on write but not read; exact `get_page` uses scalar `ctx.sourceId` while fuzzy
uses the federation ladder; unqualified search can scan isolated `--no-federated` sources.
Unify on one fail-closed visibility predicate across every read surface.
- **Per-OIDC-user access tiers** (#789, @0x471) — **HELD, high, SECURITY.** Map verified
OIDC end-users to `oauth_clients.access_tier` dispatch gates + shape filters — real
runtime access control. Pairs with multi-agent MCP hardening (#1316, @chipoto69, HELD).
- **Federated-read management CLI + admin UI** (#1592/#1601 @bitak1, #1558 @flamerged) —
**OPEN, high.** No CLI/UI to inspect or change a client's `federated_read` scope (raw
SQL only today). Atomic `array_append`/`array_remove` SQL to avoid read-modify-write
races, plus an admin Sources tab.
- **Pre-registration flow flags** (#894, @panda850819) — **OPEN, high, SECURITY.**
`register-client` hardcodes `redirect_uris=[]`, making the SECURITY.md-recommended
pre-registration (DCR-off) flow unusable for Claude.ai/ChatGPT connectors.
- **RFC 9728 `resource_metadata`** (#1410, @rayers) — **OPEN, high.** HTTP MCP 401s omit
the `resource_metadata` param the MCP auth spec + RFC 9728 require, so claude.ai/Cursor
can't discover the auth server and never start OAuth.
- **Server-enforced memory groups** (#1497, @oldmate99) — **HELD, med.** Audience-based
read/write via `memory_groups` + client-to-group assignment — strategic for hosted
multi-tenant, but overlaps the existing source-isolation model; a design call.
## 7. Security hardening (must not be lost)
- **Command injection in transcription** (#245, @aliceagent) — **OPEN, high, SECURITY.**
`transcription.ts` shell-interpolates an agent-controlled `audioPath` into `execSync`
ffprobe/ffmpeg/`rm -rf`. **Confirmed still present on master.** Switch to
`execFileSync` arg arrays + `fs.rmSync`.
- **Dotfile / skills-dir confinement** (#418/#419, @garagon) — **OPEN, high, SECURITY.**
`.gbrain-source` walk-up trusts any ancestor dotfile (source hijack on shared hosts);
`resolveWorkspaceSkillsDir` never canonicalizes (symlink escape). `lstat` ownership/
symlink/world-writable checks + realpath containment.
- **Destructive reclone gate** (#1705, @mvanhorn) — **OPEN, high, SECURITY.**
`recloneIfMissing` does `rm`+rename over `src.local_path` without verifying it's
gbrain-managed, so a re-pointed source can wipe a user's working tree. Gate behind
`isManagedRecloneTarget()` + reject `..`. *(The maintainer's own #1960 is the canonical
landing for this class — cross-check.)*
- **CORS preflight asymmetry** (#983, @yashkot007) — **OPEN, high, SECURITY.** Preflight
returns the full method/header surface unconditionally while the actual-request path
gates on the allowlist — leaks allowed surface to non-allowlisted origins.
- **jsonb double-encode corruption** (#1584 @warkcod, #597 @vinsew) — **OPEN, high,
SECURITY/integrity.** Source-config and subagent writers `JSON.stringify` into a
`::jsonb` cast — the exact postgres.js trap CLAUDE.md forbids; corrupts source config
(freshness/autopilot) and breaks dream synthesize slug-collection on real Postgres.
## 8. Developer experience & platform reach
- **Windows / CRLF portability** (#1294 @xwang4-svg, #1149 @samporter-31, #1554 @Sanjays2402,
#1396 @xuezhaolan) — **OPEN, high.** CRLF breaks frontmatter + skill-trigger parsing
(CI is Ubuntu-only so it never surfaces), `/dev/stdin` doesn't exist, a POSIX postinstall
one-liner hard-fails `bun install`, backslash bundle keys. A coordinated "first-class
Windows" pass. *(A working Windows binary + CI target #180/#181 is the prerequisite for
the full story.)*
- **`.gbrainignore` / per-repo exclusion** (#1483 @eepaul; repo-local code filters
#1011 @AndrewLauder; `--respect-gitignore` #1159 @jetsetterfl) — **OPEN, high.** Sync
indexes every file with no ignore mechanism (`data/`, `*.parquet`, fixtures, vendored
trees), bloating DB + embedding cost. gitignore-parity `.gbrainignore` + per-source
`excludePatterns`. *(See also the maintainer's walker-prune work; #1942 prunes
vendor/dist/build.)*
- **Monorepo sub-path sources** (#774, @jeremyknows) — **HELD, high.** `--src-subpath`
(split repo into git-root + logical-source axes) + `--exclude` so one repo can hold N
sources at subdirs.
- **MCP tool filtering** (#747, @joelwp) — **OPEN, high.** MCP advertises all ~51 ops to
every consumer (~10K tokens of schemas, tool confusion); `GBRAIN_EXPOSED_TOOLS` filters
the advertised surface.
- **Install-method detection for upgrade** (#538, @brucek) — **OPEN, high.** The README's
own recommended git-clone+bun-link install detects as `unknown`, so `gbrain upgrade`
offers three dead ends including a wrong npm package.
- **Runtime subagent defs** (#1282, @dcarolan1) — **OPEN, high.** The plugin loader
validates `SubagentDefinition[]` at startup but the handler never reads
`data.subagent_def`, so the persisted field is dead at runtime — callers must re-embed
the full system body in every job.
- **macOS Tahoe PGLite workaround** (#1671, @roysaurav) — **HELD, med.** PGLite's WASM
engine crashes on macOS 26 (Apple Silicon); document the native Homebrew Postgres+pgvector
fallback. Reader-valuable until the WASM crash is fixed upstream.
## 9. Capabilities & integrations (strategic — maintainer call)
These are net-new surfaces held for a product decision, not auto-closed.
- **Alternative engines** — SQLite/`bun:sqlite`+FTS5 single-file backend (#291, @mvanhorn)
and Neo4j GraphBrain REST backend (#594, @pkyanam). Both conflict with the two-engine
lockstep invariant and the Postgres-for-memory North Star, but the *zero-WASM single-file*
install story (SQLite) is strategically interesting. **HELD.**
- **Page versioning / soft-delete / read audit** (#573, @cropsgg) — **HELD, high.** Snapshots
with provenance, soft-delete tombstones + hard purge, read-path audit treating edits as
derivative works. Ambitious cathedral-scope; maintainer-owned territory.
- **Configurable embedding dimension** (#1051, @vincedk-alt) — **HELD, high.** `schema.sql`
hardcodes `vector(1536)`; read `embedding_dimensions` from config (default 1536). The
canonical fix that dozens of local-provider PRs hack around. *(Pairs with #1263.)*
- **Transcribe skill** (#1449, @RyanAlberts) — **OPEN, high.** Implements the empty
video/audio branch of `media-ingest` (YouTube captions fast path + yt-dlp/whisper
fallback), $0 by default. A genuine capability gap.
- **iPhone backup importer** (#1733, @H4RR1SON) — **HELD, med.** Local-CLI-only importer
for decrypted iPhone backups (contacts→person pages, iMessage→conversation pages); zero
network, thin-client refused.
- **Compounding dream phase** (#509, @durang) — **HELD, high.** An LLM "7th phase" that
*creates* structure (orphan-mention people, knowledge gaps, concept-dup at cosine>0.92,
decay, incomplete pages) vs the deterministic phases. Overlaps `enrich --thin`.
- **Codex-OAuth for dream** (#977, @barronlroth) / **dream gateway + `migrate-embedding-dim`**
(#1013, @cxbitz) — **HELD, high.** OAuth-backed chat for synthesis; a command to resize
the vector schema + clear incompatible embeddings.
- **Voice-extraction skill** (#300, @harjclaw) — **CLOSED, med.** Mine the user's outbound-
email corpus already in the brain to build a queryable writing-voice profile so agents
draft in the user's voice. Overlaps soul-audit.
- **MCP put_page parity + DB→markdown reconciliation** (#438, @rayzhux) — **HELD, high.**
A frontmatter-only safe auto-link mode for remote callers + `GBRAIN_BRAIN_ROOT` to render
remote writes back to markdown so MCP writes reach the git source-of-truth. Touches the
remote trust boundary — a design proposal, not a merge.
- **Recipe discovery convention** (#1279, @ialmeida-jera) — **OPEN, med.** `~/.gbrain/recipes/`
auto-discovery + `--external-dir`, loaded untrusted to keep the command-spawn boundary.
- **Destructive-op audit trail + audit-factory** (#1069/#1070, @vincedk-alt) — **HELD, med.**
Rotating JSONL forensic trail for hard-deletes + a shared `createAuditLogger` factory.
## 10. Doctor & brain-health observability
- **Queue dead-job visibility** (#1185, @ethanbeard) — **OPEN, high.** A collector can
heartbeat green while all its jobs die in the worker (3561 dead in the wild) and doctor
has zero view into the minions queue. Add a cross-cutting `[queue]` dead-jobs check.
- **Orphan-metric alignment** (#1107 @colin477, #915 @xaviroblessarries, #1202 @rwbaker) —
**OPEN, high.** `get_health` counts ingestion-by-design (`daily/`, briefings), soft-deleted,
and hub pages as orphans, distorting `brain_score`; CLI `find_orphans` uses a *different*
predicate than `getHealth`. Unify on one islanded predicate with sensible exclusions.
- **doctor check-name registry drift** (#1839, @mvanhorn) — **OPEN, med.** Several emitted
checks aren't registered in `doctor-categories`, printing `unknown check name` every run;
the drift guard only scanned `doctor.ts`, missing `onboard/checks.ts` emitters.
- **Honest stale-lock hint** (#1553, @Sanjays2402) — **OPEN, med.** doctor always says
`gbrain sync --break-lock`, which silently no-ops on `gbrain-cycle` locks.
---
## Cross-cutting observations for the maintainer
- **The same bug was filed many times.** `extract_facts.entity_hints` missing an `items`
schema came in ≥5 times (#812/#832/#847/#863/…, already fixed); the Postgres-singleton
disconnect class a dozen+ times; sync no-op freshness, slug-casing, and the embedding-
preflight false-reject each 515 times. A short "already fixed / known" note in the
release notes or a CONTRIBUTING "before you file" list would cut the re-file rate.
- **The recipe system is working as a pressure valve** — it correctly absorbed ~40 vendor
PRs into config rather than code. The remaining provider asks are about *capabilities*
the recipe schema doesn't yet express (asymmetric `input_type`, Matryoshka dims, per-item
RPM caps, alternative credential groups), not new vendors.
- **i18n (§1) and local-first chat (§5) are the two biggest "serve a billion" coverage
gaps** the community is repeatedly hitting and the best candidates to graduate to TODOs.
File diff suppressed because it is too large Load Diff
-221
View File
@@ -1,221 +0,0 @@
---
status: ACTIVE
---
# CEO Plan: v0.38 Schema Packs — Bring Your Own Shape
Generated by /plan-ceo-review on 2026-05-19
Branch: garrytan/houston-v1 | Mode: EXPANSION
Repo: garrytan/gbrain
## Definitions (terms used throughout)
- **Primitive** — a named bundle of (default link verbs, default
frontmatter fields, expert-routing flag, enrichment rubric slot).
Five built-in: `entity`, `media`, `temporal`, `annotation`,
`concept`. A pack type extends one primitive by name, inheriting
its defaults, then optionally overriding specific fields. Not a
table shape, not a schema in the SQL sense — a behavioral
template the engine consults at inference and search time.
- **Alias closure** — for read paths, when a pack declares type
`researcher` aliases base type `person`, queries for `researcher`
expand the WHERE clause to `type IN ('researcher','person', + any
other type aliasing person)`. The closure is computed once at
pack load, cached on the pack object, and inlined into search
SQL. Aliasing is one-directional (researcher → person; querying
`person` does NOT surface `researcher` rows unless the inverse is
declared).
- **Pack resolution chain (7 tiers)** — extends model-config's
6-tier pattern. Order: (1) per-call `schema_pack` opt, (2)
`GBRAIN_SCHEMA_PACK` env, (3) per-source `--source <id>` override
via DB config key `schema_pack:source:<id>`, (4) brain-wide DB
config key `schema_pack`, (5) `gbrain.yml schema:` section,
(6) `~/.gbrain/config.json schema_pack`, (7) default `gbrain-base`.
Tier 3 is the new tier introduced in v0.38; tiers 1, 2, 4-7
mirror existing patterns.
## Vision
### 10x Check
The plan as accepted ships a self-EXPANDING engine, not just a
self-describing one. The differences from the baseline plan:
- The brain watches what you create and proposes schema refinements
you didn't think to ask for (`schema suggest`)
- Schema is per-source (ISOLATED reads), so ~/git/brain and
~/git/zion-brain hold different mental models in the same engine
without renames. Cross-source federated reads still see per-source
packs in isolation — a query joining results across mounts does
NOT compute a closure across both packs. Federation (closure
across mounts) is explicitly deferred to v0.39.
- The pack is inspectable: ASCII graph, plain-English explanation,
consistency lint against actual content
- First unknown-type write asks "Add to pack?" with a primitive
inference, instead of silently logging
- Schema packs distribute as `.gbrain-schema` tarballs through the
v0.37 skillpack pipeline; skillpacks rename to `.gbrain-skillpack`
for symmetry. Community schema packs propagate the same way
community skillpacks do.
### Platonic Ideal
A new user clones gbrain and types `gbrain init`. Within 30 seconds
gbrain has read their existing markdown anywhere on disk, proposed a
schema matching their organic shape, asked 3-5 yes/no questions to
refine, and the brain is live. They never author YAML unless they
want to. They can publish their pack as a `.gbrain-schema` tarball
for anyone to install and fork.
The 12-month state: `gbrain init` runs `schema detect` automatically,
proposes a primitive structure, and 90% of users never see the
manifest format. The 10% who want to customize see a clean YAML they
can edit. Community packs cover the long tail of domains.
## Scope Decisions
| # | Proposal | Effort | Decision | Reasoning |
|---|----------|--------|----------|-----------|
| 0C-bis | Approach C (Full Cathedral) | ~4 weeks | ACCEPTED | User explicitly chose the most ambitious of three approaches; ecosystem + engine in one ship |
| D2 | Per-source schema packs | ~1 week | ACCEPTED | User owns two brains today; v0.34.1.0 source-isolation makes the seam architecturally clean |
| D3 | `gbrain schema suggest` (LLM-powered) | ~3-5 days | ACCEPTED | Closes the gap from "what exists" to "what your brain implies"; bounded cost via sampling |
| D4 | `schema graph` + `lint` + `explain` | ~2 days | ACCEPTED | Schema becomes legible and self-documenting; tiny effort, large UX delta |
| D5 | Auto-prompt on first unknown type | ~1-2 days | ACCEPTED | TTY-gated + per-type silenceable; turns lenient-mode from fallback to feature |
| D6-orig | `fork-from <brain-path>` (live-brain) | ~3-5 days | REJECTED | Privacy hazard (read access to whole repo); unclear value vs published tarballs |
| D6-reframed | Skillpack tarball reuse + extension expansion | ~3-5 days | ACCEPTED | Schema packs ship as `.gbrain-schema`; skillpacks gain `.gbrain-skillpack` extension alongside existing `.tgz`; both ride v0.37 pipeline parameterized on manifest discriminator. Extension is the install-time type detector — lets validation route to the right manifest validator before extraction. |
Total budget: **revised 9-11 weeks** (vs ~6.5-7 initial estimate;
spec review surfaced LLM prompt-tuning loops for `schema suggest`,
primitive-inference heuristics for auto-prompt, 7-tier × federated-
read interaction edges, full rename-migration surface, and 400-600
test cases at v0.36/v0.37 scope precedent). If budget pressure
emerges, the safest cuts in order are: D5 auto-prompt (~2 days),
D4 inspect triad (~2 days), reduce examples 7→3 (~3 days), defer
suggest LLM polish to v0.38.1 (~1 week).
## Accepted Scope (added to this plan)
- **Engine layer:** gbrain-base universal starter pack; 5 composable
primitives (entity, media, temporal, annotation, concept); alias
closure for read paths; lenient-by-default with audit for write
paths; strict mode opt-in.
- **Detect layer:** `gbrain schema detect` SQL-driven heuristic
clustering proposing a pack manifest matching brain shape.
- **Suggest layer:** `gbrain schema suggest` LLM-powered refinement
via gateway.chat() over a bounded sample.
- **Inspect layer:** `gbrain schema graph` (ASCII viz),
`gbrain schema lint` (consistency check), `gbrain schema explain
<type>` (plain English).
- **Author layer:** `gbrain schema init/use/fork/edit/validate/
diff/review-candidates` CLI.
- **Source layer:** per-source schema-pack resolution; pack
resolution gets a 7th tier (per-source override before per-brain);
`--source <id>` flag on every relevant command.
- **Auto-prompt layer:** TTY-gated interrupt on first unknown-type
`put_page` with primitive inference; per-type "always silent"
escape hatch.
- **Distribution layer:** `.gbrain-schema` tarball format; rename
skillpacks to `.gbrain-skillpack`; v0.37 skillpack pipeline
parameterized on artifact type (manifest discriminator drives
type-specific validation); both extensions accepted on install
for back-compat.
- **Examples:** 7 example packs in-tree (minimal, person-first,
media-archive, temporal-archive, research-notebook, founder-ops,
personal-archive) explicitly framed as sketches not products.
- **gbrain-base:** byte-for-byte reproduces today's hardcoded
behavior so existing brains see zero change after upgrade.
- **Migrations:** v76 drops `takes.kind` CHECK constraint;
validation moves to runtime against active pack's declared kinds.
- **Doctor checks:** schema_pack_active, schema_pack_consistency,
per-source pack drift.
- **Engine refactor coverage:** the v0.38 plan parameterizes EVERY
hardcoded type-coupling site listed in the original exploration,
not just `takes.kind`. Concretely: `inferType` path-prefix table,
`inferLinkType` regex bank, `FRONTMATTER_FIELD_OVERRIDES` table,
`find_experts` SQL (`type IN (…)`), `whoknows` `DEFAULT_TYPES`,
`enrichment-service` person/company restriction,
`completeness.ts` rubric map, dream-cycle entity-type prompts.
gbrain-base reproduces today's values for each.
- **Cache + rollback story:**
- `query_cache.knobs_hash` (v0.32.3 column) folds `schema_pack`
name + version into the hash so a cache row written under
`vc` is unreachable when `research-state` is active. Cross-
pack contamination structurally impossible.
- `eval_candidates` rows (v0.25.0) gain a `schema_pack` column
so `gbrain eval replay` reproduces the same retrieval space.
Migration v77 adds the column NULL-tolerant; pre-v0.38 rows
fall back to active pack during replay.
- HNSW indexes are pack-agnostic (vector columns don't change
shape across packs); no reindex needed on pack switch.
- Rollback: every `gbrain schema use` operation writes the
previous pack name to `~/.gbrain/schema-pack-history.jsonl`
so `gbrain schema use --previous` is one keystroke. Strict-
mode failures on switch surface the offending pages with
paste-ready "rename type to X" hints before any data
mutation runs. Soft-deletes from autopilot purge are NOT
triggered by pack changes.
- **Test budget:** ~400-600 cases across unit + e2e per the
v0.36/v0.37 precedent. Specifically: ~150 cases for engine layer
+ alias closure, ~50 for detect heuristic accuracy, ~50 for
suggest LLM prompts (hermetic via stubbed gateway), ~30 for per-
source resolution × 7-tier matrix, ~40 for auto-prompt UX
states, ~30 for inspect triad output stability, ~30 for tarball
type-detection + parameterized install, ~50 for migration v76 +
v77 + bootstrap parity, ~50 for examples × byte-for-byte
gbrain-base equivalence regression. **gbrain-base byte-for-byte
parity is a CI gate**, not a hope — pinned by
`test/regressions/gbrain-base-equivalence.test.ts` asserting the
pre-v0.38 hardcoded behavior reproduces from the pack-driven
paths on a fixture brain.
## Deferred to TODOS.md (v0.39+)
- Live-brain `fork-from <brain-path>` (rejected for privacy; revisit
if a sandboxed schema-only extraction path is designed)
- Per-source pack FEDERATION across mounts (a query crossing
multiple sources can use closure over each source's schema; right
now per-source is isolated reads only)
- Schema versioning + semver compatibility checks between pack
versions
- Skillpack ↔ schema-pack cross-reference (a skillpack can declare
"I work best with these primitives present in your pack")
- Live schema migration helpers (when you add a type, auto-suggest
backfill of existing pages)
- Schema diff in PR review (rendering pack changes as human-readable
diffs for community pack PRs)
## Reviewer Concerns (from spec review loop, partially addressed)
- Quality score on first pass: 6.5/10. Issues addressed in this
revision: definitions block (primitive, alias closure, 7-tier
resolution chain), per-source isolation vs federation contradiction
clarified, skillpack extension framing changed from rename to
expansion, full hardcoded-site coverage enumerated, cache +
rollback story added, test budget enumerated, budget revised to
9-11 weeks honestly.
- Issues NOT fully addressed, surfaced for the 11-section review:
- `schema suggest` LLM prompt-tuning iteration budget remains a
range estimate, not a measured number. The 11-section review
should pin a specific eval fixture set (size + diversity) and
a target accuracy threshold before code lands.
- 7-tier resolution × v0.34.1 federated_read OAuth scoping has
edge cases at the intersection that the 11-section review must
enumerate (specifically: an OAuth client with read scope across
federated sources but no source-specific pack override — which
pack drives the alias closure for cross-source queries?).
- The 7→3 example pack reduction is a real cut consideration. The
11-section review should decide whether 7 examples is the right
number or whether 3 + community-derived is more honest.
## Cathedral risks worth surfacing in 11-section review
1. The 7-week budget vs 4-week original ask. If pressure emerges,
D5 (auto-prompt) and D4 (inspect triad) are the safest cuts.
2. v0.37 skillpack registry currently has zero published packs.
The `.gbrain-schema` rename and tarball reuse doubles down on a
distribution layer with no usage signal.
3. Per-source pack resolution adds a 7th tier to the resolution
chain. The model-config 6-tier pattern is already cognitively
dense; tier 7 is an inflection point.
4. `schema suggest` introduces ongoing LLM cost per invocation.
Bounded by sampling, but sets a precedent for "gbrain commands
that cost money."
5. Auto-prompt UX is novel. TTY gate + per-type silencing helps,
but bulk-import flows could hit unexpected interruption patterns.
-184
View File
@@ -1,184 +0,0 @@
# Switching embedding models or dimensions on an existing brain
GBrain stores embeddings in a fixed-dimension `vector(N)` column on
`content_chunks`. If you switch to a model with a different dimension
(e.g. `openai:text-embedding-3-large` 1536 → `zeroentropyai:zembed-1`
1280, or `voyage:voyage-4-large` 2048), the on-disk column type doesn't
change automatically.
`gbrain init`, `gbrain doctor`, and `gbrain embed --stale` all detect
this mismatch and refuse to silently proceed. This doc is the recipe
they point at.
## Same-dimension model swaps (v0.41.31.0 — automatic)
If you switch to a different model at the **same** dimension count
(e.g. one 1536-dim provider to another, or a re-tuned model that keeps
its width), the column type doesn't change, so no `ALTER`/wipe recipe
is needed. As of v0.41.31.0, gbrain stamps an embedding-provenance
signature (`<provider:model>:<dims>`) onto each page when its chunks are
embedded. After you point the config at the new model, the stored
signatures differ from the current one, and `gbrain embed --stale`
re-embeds exactly those pages:
```bash
# After switching to the new same-dim model in your config:
gbrain embed --stale # re-embeds signature-drifted pages
gbrain embed --stale --dry-run # preview the count without re-embedding
```
Under federated_v2, the same drift is picked up by the per-source
`embed-backfill` jobs that `gbrain sync --all` enqueues (capped
`$X/source/24h`). **Grandfather:** pages embedded before v0.41.31.0
carry a NULL signature and are NEVER flagged stale, so upgrading to
v0.41.31.0 does NOT trigger a whole-corpus re-embed. Signatures only
get stamped going forward.
A **dimension** change still requires the wipe-and-reinit (PGLite) or
column-alter (Postgres) recipe below — the on-disk `vector(N)` width
genuinely has to change.
## Why we don't do this automatically
Switching dimensions requires:
1. Dropping the HNSW vector index (pgvector won't survive an `ALTER COLUMN TYPE`).
2. Wiping every existing embedding (the old vectors are unusable in the new space — and pgvector refuses to cast them across dimensions, so this must happen before the alter).
3. Altering the column type (Postgres only — PGLite cannot do this).
4. Re-embedding the entire corpus (can take hours on a 50K-page brain and costs $1-100 in API calls depending on model).
5. Conditionally recreating the index (HNSW supports up to 2000 dimensions per pgvector; above that you must use exact scans).
That's not an upgrade-time auto-run. It's a deliberate, expensive
operation. Run it when you've decided you actually want the new model.
## PGLite (default install)
**PGLite cannot `ALTER COLUMN TYPE vector(N)`.** pgvector ships as
embedded WASM, not a native extension, and the WASM build rejects the
column-type alter with `could not access file "$libdir/vector"`. The
SQL recipe below works against Postgres only.
The path that works on PGLite is **wipe-and-reinit**. v0.37 ships a
single-command wrapper:
```bash
gbrain reinit-pglite \
--embedding-model zeroentropyai:zembed-1 \
--embedding-dimensions 1280
```
This backs up the existing brain to `<path>.bak`, runs `gbrain init`
with the new flags (preserving every other field in
`~/.gbrain/config.json`), and re-syncs the brain repo. Add `--no-sync`
to skip the resync, `--yes` to skip the TTY confirmation, `--json` for
structured output.
Equivalent by hand:
```bash
# 1. Back up the existing brain (in case you want to roll back).
mv ~/.gbrain/brain.pglite ~/.gbrain/brain.pglite.bak
# 2. Re-init with the new model + dimensions. `gbrain init` writes
# the schema sized to the new dim, and (as of v0.37) preserves
# every other field in ~/.gbrain/config.json (chat model,
# expansion model, API keys).
gbrain init --pglite \
--embedding-model zeroentropyai:zembed-1 \
--embedding-dimensions 1280
# 3. Re-import your brain repo. `gbrain sync` reads the brain repo
# from disk and re-creates the page rows.
gbrain sync
# 4. Re-embed. The embed pipeline now uses the new model and the
# column accepts the new dim.
gbrain embed --stale
```
If your brain repo is large enough that re-syncing from disk is
expensive (>50K pages), see the Postgres section below — migrating to
Postgres temporarily lets you run the SQL recipe, then migrate back to
PGLite.
`GBRAIN_HOME` users: substitute the active database path (or use
`gbrain config get database_path` to find it).
## Postgres (Supabase / self-hosted)
Postgres supports the in-place column alter. Replace `<NEW_DIMS>` with
your target dimension count.
```sql
BEGIN;
-- 1. Drop the HNSW index. It can't survive the column type change.
DROP INDEX IF EXISTS idx_chunks_embedding;
-- 2. Clear stale embeddings FIRST. This must happen BEFORE the column
-- alter: pgvector refuses to cast existing vectors across dimensions
-- ("expected <NEW_DIMS> dimensions, not <OLD_DIMS>"), so altering a
-- column that still holds old-width vectors aborts the transaction.
-- NULLs cast fine. (The old vectors are unusable in the new space
-- anyway — this is the wipe step from the rationale above.)
UPDATE content_chunks SET embedding = NULL, embedded_at = NULL;
-- 3. Alter the column type (all rows are NULL now, so the cast succeeds).
ALTER TABLE content_chunks ALTER COLUMN embedding TYPE vector(<NEW_DIMS>);
-- 4. Recreate the HNSW index ONLY IF dims <= 2000. Above that, leave it
-- indexless and rely on exact scans (gbrain searchVector handles this
-- automatically — search just gets slower, not broken).
-- For dims <= 2000 (e.g. 1024, 1280, 1536, 768):
CREATE INDEX IF NOT EXISTS idx_chunks_embedding
ON content_chunks USING hnsw (embedding vector_cosine_ops);
-- For dims > 2000 (e.g. 2048 Voyage 4 Large): skip step 4.
COMMIT;
```
Then re-init config with the new model:
```bash
gbrain init --supabase \
--embedding-model <provider:model> \
--embedding-dimensions <NEW_DIMS>
```
And re-embed:
```bash
gbrain embed --stale
```
## A note on `gbrain config set`
Pre-v0.37 docs recommended `gbrain config set embedding_model X` to
switch models. **This is a no-op for the embed pipeline.** `config set`
writes the DB plane; the embed gateway reads the file plane
(`~/.gbrain/config.json`). The pre-v0.37 recipe shipped the lie because
the contract wasn't surfaced.
As of v0.37, `gbrain config set embedding_model` and `gbrain config set
embedding_dimensions` REFUSE and print the wipe-and-reinit recipe.
To change schema-sizing fields, use `gbrain init` (PGLite) or the SQL
recipe (Postgres). Both update the file plane AND the schema together.
## Verify
After the recipe lands, `gbrain doctor --fast` should report green and
`gbrain doctor` should pass the `embedding_width_consistency` check:
```
✓ embedding_width_consistency dim parity: config 1280 / column vector(1280)
```
If it doesn't, file an issue with the doctor output and the steps you
ran.
## v0.37+ followups
- Auto-fallback to alternative embedding providers when the primary
fails quota/auth. Tracked; requires explicit `--try-fallback`
consent because mixing provider vectors silently corrupts retrieval.
+1 -1
View File
@@ -183,6 +183,6 @@ This also means the best AI agent setups will be open source by default. Closed,
Software distribution reimagined: the package is a markdown file, the runtime is a sufficiently smart model, the package manager is your AI agent, and the app store is a git repo.
`gbrain skillpack scaffold voice-agent`
`gbrain install voice-agent`
That's it.
-29
View File
@@ -1,29 +0,0 @@
# Origin story
GBrain came out of building OpenClaw — Garry's personal AI agent fork. The first version had skills and a brain, but the brain was a flat directory of markdown files. Search was ripgrep. Memory was vibes.
Two problems surfaced almost immediately.
First, the agent forgot things between conversations. Every new session re-asked basic questions. Names of people Garry had introduced last week were gone. Decisions made on Tuesday didn't survive to Thursday. The brain existed but the agent couldn't actually use it.
Second, the agent kept duplicating work. Two different signals about the same company became two different people pages. Three meetings with the same person became three uncorrelated timeline entries. The signal-to-noise ratio decayed in real time.
GBrain is what you build when you decide both of those are unacceptable.
The fix wasn't one big idea. It was many small ones layered together:
- Brain-first lookup before any external API call.
- Auto-linking on every page write so the graph grows for free.
- Typed edges so "who works at Acme AI?" actually returns something.
- Hybrid search because vector alone underdelivers.
- Reranker on top because hybrid alone is locally optimal but globally suboptimal.
- Nightly cron to dedup, enrich, fix citations, surface contradictions.
- An agent that reads `skills/RESOLVER.md` once and knows what to do.
None of those are novel ideas. The contribution is shipping all of them together, on Postgres + pgvector that runs in WASM (no server), with skills that are markdown (not code), routed by a small text file (not a router LLM).
The production brain has been running for months now. 17,888 pages. 4,383 people. 723 companies. 21 cron jobs running autonomously. It wakes Garry up smarter than the day before.
GBrain is what happens when you write the brain you actually wanted to have.
The reason the brain is worth building is `gbrain think`. Without it, the brain is just a place that holds your notes. With it, the brain is a thing you can query about itself: what does it know, what does it not know yet, where does it contradict itself, where are the holes. The 24/7 cron cycle keeps the brain sharp. `think` is what makes a sharp brain useful.
-598
View File
@@ -1,598 +0,0 @@
# Running real-world eval benchmarks against your gbrain changes
Audience: gbrain maintainers and contributors. If you're touching retrieval
(search, ranking, embeddings, intent classification, query expansion, source
boost, hybrid fusion), this is the doc.
For the **NDJSON wire format** consumed by gbrain-evals, see
[`eval-capture.md`](./eval-capture.md). This doc is the human dev loop
that lives on top of that format.
## v0.41 update — the LOOP is now real
Before v0.41, you could capture eval rows and replay them but nothing
stitched them into a gate. `gbrain bench publish` + `gbrain eval gate`
close the loop. Two gates:
- **Regression gate** (`--baseline X.baseline.ndjson`): replays a baseline
you captured against your current brain. Catches: "did my refactor break
search?" Compares jaccard / top-1 stability / latency multiplier.
- **Correctness gate** (`--qrels Y.qrels.json`): runs known-right queries
against your current brain via bare `hybridSearch`. Catches: "is my
retrieval actually any good?" Computes recall@K, first-relevant-hit-rate,
expected_top1-hit-rate.
Both can be passed together; both must pass for verdict `pass`. At least
one is required.
### The full LOOP for your own brain
```bash
# 1. Capture (one-time; uses queries already in eval_candidates)
gbrain eval export --limit 200 --tool query > /tmp/captured.ndjson
# 2. Publish a baseline
mkdir -p ~/.gbrain/baselines
gbrain bench publish --from /tmp/captured.ndjson --to ~/.gbrain/baselines/personal.baseline.ndjson --label "personal-$(date +%Y%m%d)"
# 3. Gate against it
gbrain eval gate --baseline ~/.gbrain/baselines/personal.baseline.ndjson
```
### Privacy posture (D9)
**Public baselines in `gbrain-evals` are hermetic-synthetic ONLY.** Real
user captures stay local in `~/.gbrain/baselines/`. The boundary is
enforced at the file source, not by post-hoc scrubbing. If you publish a
baseline to `gbrain-evals`, generate it from a fixture-seeded test brain
(placeholder names like `alice-example`, `widget-co-example`) — never
from a real user's `eval_candidates` table.
### Deterministic-pipeline disclosure
`gbrain eval gate --qrels` uses bare `hybridSearch` (not the production
`query` op handler). This is deliberate: gates need to be deterministic in
CI. Production retrieval differs via the query cache, salience freshness,
expansion, etc. The gate measures retrieval quality with a fixed pipeline;
your users may see different results when the cache is warm.
### `.qrels.json` shape
Two equivalent representations per entry:
```json
{
"schema_version": 1,
"queries": [
{
"query_id": "q1",
"query": "fintech founder",
"relevant_slugs": ["people/alice-example"],
"first_relevant_slug": "people/alice-example"
}
]
}
```
For federated / multi-source brains, use the explicit shape (no defaults
to `source_id='default'`):
```json
{
"query_id": "q2",
"query": "anything",
"relevant": [
{"source_id": "host", "slug": "people/alice"},
{"source_id": "team-a", "slug": "people/alice"}
],
"expected_top1": {"source_id": "host", "slug": "people/alice"}
}
```
Without `source_id`, a hit from the wrong source could false-pass the
gate. The compare everywhere is `${source_id}::${slug}` strings.
### Example GitHub Actions workflow
```yaml
name: gbrain-eval-gate
on: [pull_request]
jobs:
gate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: oven-sh/setup-bun@v2
- run: bun install
- run: |
# Run both gates; CI fails on any breach.
gbrain eval gate \
--baseline gbrain-evals/baselines/v0.41-launch.baseline.ndjson \
--qrels gbrain-evals/qrels/v0.41-launch.qrels.json \
--json | tee /tmp/gate.json
```
---
## Prerequisite: turn on contributor mode
Capture is **off by default** for production users (privacy-positive — no
surprise data accumulation). Contributors flip it on with one line:
```bash
# In ~/.zshrc or ~/.bashrc:
export GBRAIN_CONTRIBUTOR_MODE=1
```
Verify:
```bash
gbrain query "anything" >/dev/null
psql $DATABASE_URL -c 'SELECT count(*) FROM eval_candidates' # should be > 0
```
To override (force on/off regardless of env var), edit `~/.gbrain/config.json`:
```json
{"eval": {"capture": true}} // force on
{"eval": {"capture": false}} // force off
```
Explicit config beats the env var both directions.
## The 4-command loop
```bash
# ① Capture: writes to eval_candidates whenever CONTRIBUTOR_MODE is set.
# Inspect what's been collected:
gbrain doctor # surfaces capture failures
psql $DATABASE_URL -c 'SELECT count(*) FROM eval_candidates'
# ② Snapshot: freeze a baseline before your code change.
gbrain eval export --since 7d > baseline.ndjson
# ③ Code change: do whatever you want — tune RRF_K, swap embed model, edit
# hybrid.ts, add a new boost source, change the intent classifier.
# ④ Replay: re-run every captured query against the current build.
gbrain eval replay --against baseline.ndjson
```
Output:
```
Replaying 247 captured queries…
...25/247
...50/247
...
Replayed 247 of 247 captured queries (0 skipped, 0 errored)
Mean Jaccard@k: 0.927
Top-1 stability: 91.5%
Mean latency Δ: +14ms (current vs captured)
Top 5 regression(s):
jaccard=0.20 captured=12 current=3 "find every reference to widget-co"
jaccard=0.43 captured=14 current=8 "show me everything tagged for review"
jaccard=0.50 captured=8 current=4 "what did alice say about the spec"
...
```
Three numbers tell you whether the change is safe to land:
| Metric | What it means | Healthy range |
|---|---|---|
| **Mean Jaccard@k** | Average overlap between captured retrieved slugs and current run's slugs. 1.0 = identical sets. | ≥0.85 for "neutral" changes. <0.7 means major retrieval shift. |
| **Top-1 stability** | Fraction of queries whose #1 result didn't change. | ≥85% for tuning passes. <70% means top-of-funnel broke. |
| **Mean latency Δ** | Current minus captured. Positive = slower now. | Within ±50ms of captured. >2× anywhere = regression alarm. |
## What it actually does
`gbrain eval replay` reads your NDJSON snapshot and, for each row:
1. Re-executes the same op (`searchKeyword` for `tool_name='search'`,
`hybridSearch` for `tool_name='query'`) with the captured `detail` and
`expand_enabled` values threaded back in.
2. Captures the current `retrieved_slugs` (deduped, in result order).
3. Computes set-Jaccard between captured and current slug sets.
4. Records top-1 match (was the #1 result the same slug?).
5. Records latency delta vs captured `latency_ms`.
It does NOT compute MRR or nDCG — those need ground-truth relevance labels,
not a baseline comparison. For metric-against-truth eval, use
`gbrain eval --qrels <path>` (the legacy IR-eval path, still supported). The
replay tool answers a different question: "did my code change move
retrieval, and which queries did it move most?"
For a third evaluation axis — public benchmark, ground-truth labels, full
question-answer pipeline (not just retrieval) — `gbrain eval longmemeval
<dataset.jsonl>` (v0.28.8) runs the LongMemEval benchmark against gbrain's
hybrid retrieval. Each question gets a clean in-memory PGLite, its haystack
imported, the question asked, the hypothesis emitted as JSONL — exactly the
shape LongMemEval's `evaluate_qa.py` consumes. Your `~/.gbrain` brain is
never opened. See `## Public benchmarks: LongMemEval` below.
## Best-effort by design
Replay is not pure. Three things can drift between capture and replay:
1. **Brain state** — your brain probably has more pages now than when the
snapshot was taken. Unless you explicitly seed a fixed corpus, mean
Jaccard will drop simply because new pages are eligible.
2. **Embedding source** — if you changed `OPENAI_API_KEY` between capture
and replay (or the embedding model rotated), vector-path results drift
even with identical code.
3. **Capture cap** — captured `retrieved_slugs` is a deduped set; it doesn't
preserve internal ranking metadata. Two tools can return the same slug
set with different scores — Jaccard will say 1.0, but a downstream
consumer that orders by score may behave differently.
The metrics are **regression alarms on real queries**, not a hash check.
Pair them with manual inspection of the top regressions.
## Cost
Every `query` row in the snapshot embeds the query string via OpenAI to run
the vector half of `hybridSearch`. Cost is identical to a normal `gbrain
query` invocation — text-embedding-3-large at OpenAI list price, batched
inside a single replay row.
If you're iterating locally and don't want to pay per change, use
`--limit 50` to cap rows replayed. The 50 most recent rows are usually
enough to catch direction; expand for the final pre-merge run.
```bash
# Iteration mode — 50 most recent queries
gbrain eval replay --against baseline.ndjson --limit 50
# Pre-merge — full snapshot
gbrain eval replay --against baseline.ndjson --top-regressions 20
```
## CI integration
```bash
gbrain eval replay --against baseline.ndjson --json > replay.json
jq -e '.summary.mean_jaccard >= 0.85' replay.json || exit 1
jq -e '.summary.top1_stability_rate >= 0.85' replay.json || exit 1
```
Stable JSON shape (schema_version: 1):
```json
{
"schema_version": 1,
"summary": {
"rows_total": 247,
"rows_replayed": 247,
"rows_skipped": 0,
"rows_errored": 0,
"mean_jaccard": 0.927,
"top1_stability_rate": 0.915,
"mean_latency_delta_ms": 14,
"rows_over_2x_latency": 0
}
}
```
`--verbose` adds a `results: [...]` array with one entry per replayed row
(useful for piping into jq or a notebook for deeper analysis).
## When to run this
Before merging anything that touches:
- `src/core/search/hybrid.ts` (RRF, fusion, dedup, two-pass retrieval)
- `src/core/search/source-boost.ts` / `sql-ranking.ts` (per-source ranking)
- `src/core/search/intent.ts` (auto-detail classification)
- `src/core/search/expansion.ts` (Haiku query expansion)
- `src/core/search/dedup.ts` (cross-page result collapse)
- `src/core/embedding.ts` or any embedding model swap
- `src/core/operations.ts` `query` or `search` op handlers (capture surface)
- `src/core/postgres-engine.ts` / `pglite-engine.ts` `searchKeyword` /
`searchVector` SQL
Skip for: schema-only migrations, doc changes, tests-only PRs, CLI ergonomics
that don't touch retrieval.
## Building your own corpus
If you don't have captured traffic yet (fresh install, can't dogfood for a
week before merging), you can hand-author an NDJSON file:
```jsonl
{"schema_version":1,"id":1,"tool_name":"query","query":"who is alice","retrieved_slugs":["people/alice","people/alice-bio"],"expand_enabled":false,"detail":null,"latency_ms":0,"remote":false}
{"schema_version":1,"id":2,"tool_name":"search","query":"acme deal","retrieved_slugs":["deals/acme-seed","companies/acme"],"latency_ms":0,"remote":false}
```
Then run `gbrain eval replay --against handcrafted.ndjson` to confirm the
authoritative slugs come back. This is the seam between the BrainBench-Real
pipeline (replay against live captures) and the BrainBench fixed-fixture
pipeline (`gbrain eval --qrels` with the sibling
[gbrain-evals](https://github.com/garrytan/gbrain-evals) corpus).
## Off-switch
Two ways to disable capture:
```bash
unset GBRAIN_CONTRIBUTOR_MODE # easy: just unset the env var
```
Or force off regardless of the env var via `~/.gbrain/config.json`:
```json
{"eval": {"capture": false}}
```
Existing `eval_candidates` rows stay until you `gbrain eval prune
--older-than 0d` (or just drop the table).
## Failure modes
| What you see | What it means |
|---|---|
| `Mean Jaccard@k: 0.4`, top regressions all in one source dir | Source boost or hard-exclude regression on that prefix |
| `Top-1 stability: 30%`, mean Jaccard still high | RRF tuning shifted the rank order without changing the set — re-tune `rrfK` |
| `Mean latency Δ: +500ms`, jaccard high | Vector path got slower; check embedding API or HNSW probes |
| `rows_errored > 0` | One or more queries threw. Inspect first 3 in human output, or `--json` to see all `error_message` fields |
| Many `skipped: empty query` | Capture ran on rows where someone passed empty `query` — check why those were captured |
## Public benchmarks: LongMemEval (v0.28.8)
`gbrain eval longmemeval` runs the public [LongMemEval](https://huggingface.co/datasets/xiaowu0162/longmemeval)
benchmark directly against gbrain's hybrid retrieval. Different evaluation
axis from `eval replay`: public dataset with ground-truth labels, end-to-end
question-answer pipeline, hermetic per-question brains.
```bash
# Download the dataset (visit the HF page in a browser; gated/manual download).
# Place longmemeval_oracle.json (or _s.json) somewhere local.
# Retrieval-only (no LLM answer-gen, fastest path, no Anthropic key needed):
gbrain eval longmemeval ./longmemeval_oracle.json --limit 50 --retrieval-only \
> /tmp/hypothesis.jsonl
# Full pipeline (Anthropic key required for answer-gen):
gbrain eval longmemeval ./longmemeval_oracle.json --limit 50 \
> /tmp/hypothesis.jsonl
# Score with LongMemEval's published evaluate_qa.py (not bundled — needs
# OpenAI gpt-4o per their spec):
python evaluate_qa.py /tmp/hypothesis.jsonl
```
### Architecture (read this if you're touching the harness)
- One in-memory PGLite per benchmark run via `createBenchmarkBrain` +
`withBenchmarkBrain`. Your `~/.gbrain` is never opened.
- Between questions: `TRUNCATE` over runtime-enumerated `pg_tables`, NOT a
hardcoded list — schema migrations don't silently leak data across
questions. Infrastructure tables (`sources`, `config`,
`gbrain_cycle_locks`, `subagent_rate_leases`) are preserved across resets.
- Sanitization parity: re-uses `INJECTION_PATTERNS` from
`src/core/think/sanitize.ts` so adding a new injection pattern
automatically covers takes AND benchmarks. One source of truth.
- Retrieved chat content is wrapped in `<chat_session id="..." date="...">`
framing; the answer-gen system prompt declares the content UNTRUSTED.
Same posture as `<take>` framing.
- LLM injection seam: `runEvalLongMemEval(args, {client?: ThinkLLMClient})`.
Tests stub the client so the full pipeline runs hermetically without any
API key.
### Flags
| Flag | Default | Purpose |
|---|---|---|
| `--limit N` | run all | Cap question count (iterate fast) |
| `--retrieval-only` | off | Emit retrieved chunks; no LLM answer-gen |
| `--keyword-only` | off | Disable vector path (debug retrieval issues) |
| `--expansion` | **off** | Multi-query expansion. Off by default for determinism (no per-query Haiku call). Pass to opt in. |
| `--top-k K` | 10 | Retrieval depth |
| `--model M` | resolved | Default resolves through `resolveModel()` 6-tier chain (`models.eval.longmemeval` config key) |
| `--output FILE` | stdout | Write hypothesis JSONL to file instead of stdout |
### Numbers
p50 25.9ms / p99 30.3ms warm reset+import+search on Apple Silicon (per the
`test/eval-longmemeval.test.ts` perf gate). Per-question cost well under the
500ms speed gate. 500 questions = ~13s of overhead plus your retrieval and
LLM latency.
## Measuring brain consistency over time (v0.32.6)
`gbrain eval suspected-contradictions` is a complementary measurement
instrument: it samples retrieval results for unmarked semantic
contradictions (e.g., compiled_truth vs chat content, intra-page chunk
vs active take). Where LongMemEval measures retrieval correctness on a
fixed labeled set, the contradiction probe measures how often a real
brain surfaces conflicting answers.
### Recommended nightly cadence
```bash
# Once a day, against your top 50 most-frequent queries:
gbrain eval suspected-contradictions \
--queries-file ~/.gbrain/queries.jsonl \
--top-k 5 \
--budget-usd 5 \
--output ~/.gbrain/probe-runs/$(date +%Y-%m-%d).json
```
Persistent cache (`eval_contradictions_cache`) makes re-runs near-zero
cost until you bump `PROMPT_VERSION`. Trend-track via:
```bash
gbrain eval suspected-contradictions trend --days 30
```
The ASCII bar chart shows total flagged per day. Headline % surfaces in
`gbrain doctor`'s `contradictions` check with paste-ready resolution
commands per high-severity finding.
### See also
- `docs/contradictions.md` — architecture, severity rubric, action criteria.
- CHANGELOG `## [0.32.6]` — full release notes including the bigger-swing
decision criteria gated on Wilson CI lower-bound.
## v0.40.1.0 Track D — Eval infrastructure
Three eval surfaces grew non-trivial capabilities in v0.40.1.0. This section
covers the dev loop that uses them and the gates they enforce.
### `gbrain eval longmemeval --by-type` — per-question-type R@k breakdown
LongMemEval has always computed per-question-type recall internally; v0.40.1.0
surfaces it in machine-readable form. Two additive changes:
1. Every per-question JSONL row now includes a `question: string` field so the
`gbrain eval cross-modal --batch` consumer (below) can read it without
joining back against the source dataset.
2. New `--by-type` flag emits a final aggregate line keyed by `question_type`:
```json
{"schema_version": 1, "kind": "by_type_summary",
"recall_by_type": {"single-session-user": {"hit": 18, "total": 19, "rate": 0.947}},
"aggregate": {"hit": 110, "total": 120, "rate": 0.917}}
```
**Resume-safe.** When `--resume-from` is the same path as `--output`, the
summary is rebuilt from the file (each per-row includes `question_type` and
`recall_hit`) so the final aggregate covers all resumed questions, not just
this run's slice. The prior summary at the file tail is replaced, not
appended — a brain that resumes 5 times across a 500-question run ends with
exactly ONE summary at the tail.
**Optional gate.** `--by-type-floor 0.85` exits non-zero when any
`question_type`'s rate falls below 0.85. Default: informational only.
```bash
# Diagnose per-type ranking quality after a search-touching change.
gbrain eval longmemeval ~/datasets/longmemeval_s.jsonl \
--by-type --output /tmp/run.jsonl
tail -1 /tmp/run.jsonl | jq . # summary line
# Strict gate in a CI script.
gbrain eval longmemeval test/fixtures/longmemeval-mini.jsonl \
--by-type --by-type-floor 0.80 --output /tmp/run.jsonl
echo "exit=$?" # 1 if any type fell below 0.80
```
### Hermetic retrieval gate — `test/eval-replay-gate.test.ts`
The v0.40.1.0 Track D structural fix for "PRs touching `src/core/search/`
silently regress retrieval." Replaces the original "replay against captured
eval_candidates" design (which Codex caught as non-functional in CI — see
the `v0.41+: contributor-mode CI capture` TODO in `TODOS.md` for the deferred
real-query version).
How it works:
- Hand-curated qrels fixture at `test/fixtures/eval-baselines/qrels-search.json`
with PLACEHOLDER names only (no real people / companies per CLAUDE.md privacy
rule).
- The test seeds a PGLite engine with synthetic pages whose embeddings are
basis vectors (the same `basisEmbedding(idx)` pattern as
`test/e2e/search-quality.test.ts`). No API keys, no DATABASE_URL.
- For each qrels query, calls `engine.searchVector(basisEmbedding(dim))` and
computes `top1_match_rate` and `recall@10`. Asserts both meet floors
(`>= 0.80` and `>= 0.85` by default).
- Lives in the unit-shard test matrix (`.github/workflows/test.yml`) so it
runs on every PR via `bun test`, NOT in the E2E fixed-file workflow.
#### Refreshing the qrels fixture (the `Why:` discipline, D4)
When CI fails because a legitimate ranking change moved expected slugs, the
fix is to edit `qrels-search.json` directly. **Always include a `Why:` line
in the commit body** so future maintainers can read the audit trail. Without
the `Why:`, the gate degrades to a rubber stamp within months. The convention
is informational (not a commit-hook block), but enforce it in PR review.
Example commit body:
```
chore(eval): refresh qrels for new source-boost ordering
Why: v0.40.x source-boost now weights originals/ over concepts/, so
q12 (founder-mode) now correctly surfaces originals/founder-mode-example
top-1. Manual verification: ran the production query; new ranking is
clearly better-aligned with the query intent.
```
#### Env-overrides for floors
```bash
GBRAIN_REPLAY_GATE_TOP1_FLOOR=0.85 \
GBRAIN_REPLAY_GATE_RECALL_FLOOR=0.90 \
bun test test/eval-replay-gate.test.ts
```
Use to tighten or loosen the gate as the qrels fixture matures.
### `gbrain eval cross-modal --batch` — batch quality scoring
Single-task cross-modal eval scores one (task, output) pair. Batch mode runs
the same scoring over an entire LongMemEval JSONL output, with cost guardrails.
```bash
# Step 1: produce LongMemEval hypotheses (real cost: depends on model + N).
gbrain eval longmemeval ~/datasets/longmemeval_s.jsonl \
--limit 10 --output /tmp/run.jsonl
# Step 2: batch-score those hypotheses (real cost: ~$0.70 for 10 questions,
# 1 cycle, 3 model slots at default --max-usd 5 budget cap).
gbrain eval cross-modal --batch /tmp/run.jsonl \
--limit 10 --cycles 1 --concurrent 3 --max-usd 5 --json
echo "exit=$?" # 0=all-pass, 1=any-fail, 2=any-error-or-inconclusive
```
**Key behaviors:**
- Default `--cycles 1` in batch mode (single-task default is 3 in TTY) to bound
cost. Pass `--cycles 3` to match single-task strictness.
- `--concurrent 3` runs up to 3 questions in parallel x 3 model slots each =
9 simultaneous API calls. Below tier-1 rate limits for all three providers.
- `--max-usd FLOAT` refuses to start if the pre-flight cost estimate exceeds
the cap, unless `--yes` bypasses (required for non-interactive cron / CI).
- Filters `kind: "by_type_summary"` rows automatically (the LongMemEval
`--by-type` summary line is metadata, not a question).
- `--batch` is mutually exclusive with `--task`; fail-fast usage error if both
are set.
- Exit precedence (fail-loud): ERROR > FAIL > INCONCLUSIVE > PASS.
- Per-question receipts land in a tempdir and are deleted at end of batch; the
summary inlines per-question verdicts so the audit trail is self-contained.
### Nightly cross-modal quality probe (opt-in, autopilot)
`src/core/cycle/nightly-quality-probe.ts` ships a phase that runs the longmemeval
+ cross-modal pipeline once per 24h. **Disabled by default** to avoid surprise
API spend. Enable per-host:
```bash
gbrain config set autopilot.nightly_quality_probe.enabled true
gbrain config set autopilot.nightly_quality_probe.max_usd 5.00 # optional override
```
Note: `--phase nightly_quality_probe` wiring into the autopilot scheduler is
deferred to a v0.41+ follow-up (see TODOS.md). For now the phase is callable
in isolation; the test harness exercises it via DI stubs.
```bash
# Manual smoke (exercises the path via DI stubs, no real API spend).
bun test test/nightly-quality-probe.test.ts
```
Observability:
- `~/.gbrain/audit/quality-probe-YYYY-Www.jsonl` — one event per run with
outcome (pass / fail / inconclusive / error / budget_exceeded /
rate_limited / no_embedding_key), pass/fail/inconclusive/error counts,
est_cost_usd, fixture_sha8. ISO-week rotation (mirrors slug-fallback
audit).
- `gbrain doctor` surfaces `nightly_quality_probe_health`:
- SKIPPED (disabled) — with paste-ready enable command.
- OK (enabled, no events yet) — autopilot hasn't fired its first run.
- OK (last 7d all PASS) — with timestamp of latest run.
- WARN — any FAIL / ERROR / BUDGET_EXCEEDED in the window, with outcome
counts and the latest run's reason.
Real expected cost: ~$0.35 per nightly run (5 questions x 3 slots x 1 cycle
x ~$0.02/call) ≈ $10.50/month. Worst-case under the default budget cap:
$150/month. Opt-in default prevents discovering this in your card statement.
-160
View File
@@ -1,160 +0,0 @@
# Eval capture — NDJSON schema reference
**Status:** stable from v0.21.0. Schema versioning via `schema_version`
on every row; additive changes increment the minor version; removals
are breaking-schema-v2.
**Audience:** downstream consumers (primarily the sibling
[gbrain-evals](https://github.com/garrytan/gbrain-evals) repo) that
replay captured real-world queries as a BrainBench-Real fixture.
## The pipeline
```
MCP / CLI / subagent tool-bridge caller
src/core/operations.ts — query + search op handlers
│ (hybridSearch or searchKeyword)
{results, meta: HybridSearchMeta} ┌── captureEvalCandidate
│ │ (fire-and-forget)
▼ │
return to caller ▼
scrubPii(query) ←── src/core/eval-capture-scrub.ts
buildEvalCandidateInput
engine.logEvalCandidate
┌──────────────┴──────────────┐
│ success │ fail
▼ ▼
INSERT into eval_candidates engine.logEvalCaptureFailure
(reason: db_down | rls_reject |
check_violation |
scrubber_exception | other)
```
## `gbrain eval export` — the consumer contract
```sh
gbrain eval export [--since DUR] [--limit N] [--tool query|search]
```
Emits NDJSON to **stdout**. One JSON object per `\n`-terminated line.
stderr receives progress heartbeats. Every line starts with
`"schema_version": 1` so a forward-compat parser can fail loudly on
schema v2 instead of silently misparsing.
Typical usage from gbrain-evals:
```sh
# Snapshot the last week of real traffic for replay
gbrain eval export --since 7d > brainbench-real.ndjson
```
```sh
# Stream through jq for ad-hoc analysis
gbrain eval export --tool query | jq -c 'select(.latency_ms > 500)'
```
## Row schema (v1)
Every exported row has this shape. Field order in JSON output is not
guaranteed; consumers MUST key by name, not position.
| Field | Type | Notes |
|---|---|---|
| `schema_version` | number | Always `1` on v1 rows. Forward-compat gate. |
| `id` | number | Autoincrement primary key. Stable across exports. |
| `tool_name` | `"query"` \| `"search"` | Which MCP operation captured this row. |
| `query` | string | **Already PII-scrubbed** by `scrubPii` unless `eval.scrub_pii: false`. Emails / phones / SSN / Luhn-verified credit cards / JWTs / bearer tokens replaced with `[REDACTED]`. Max length 50KB (CHECK-enforced). |
| `retrieved_slugs` | string[] | Deduplicated slugs that came back in `SearchResult[]`. |
| `retrieved_chunk_ids` | number[] | Every chunk id in result order (duplicates preserved — one per hit). |
| `source_ids` | string[] | Distinct `sources.id` values across the result set (v0.18 multi-source). Empty for pre-v0.18 rows that lacked the column. |
| `expand_enabled` | boolean \| null | Whether the caller **requested** Haiku expansion. `null` for `search` (no expansion concept). |
| `detail` | `"low"` \| `"medium"` \| `"high"` \| null | Detail level the caller **requested**. `null` when omitted. |
| `detail_resolved` | `"low"` \| `"medium"` \| `"high"` \| null | What `hybridSearch` **actually used** after auto-detect. `null` when neither caller nor heuristic classified. |
| `vector_enabled` | boolean | True iff vector search actually ran. `false` when `OPENAI_API_KEY` was missing or the embed call failed. **Replay MUST respect this** — rows with `false` only exercised the keyword path. |
| `expansion_applied` | boolean | True iff Haiku expansion actually produced variants (not just "was requested"). |
| `latency_ms` | number | Wall-clock duration of the op handler (includes capture itself — negligible since it's fire-and-forget). |
| `remote` | boolean | `true` for MCP callers (untrusted), `false` for local CLI. Partitions "real agent traffic" from "operator probing." |
| `job_id` | number \| null | `OperationContext.jobId` when the caller was a subagent tool-bridge. Null for MCP + CLI. |
| `subagent_id` | number \| null | `OperationContext.subagentId` for subagent-owned runs. |
| `created_at` | string (ISO 8601) | UTC timestamp of insert. |
## Ordering + determinism
`listEvalCandidates` orders by `created_at DESC, id DESC`. Same-
millisecond inserts tie on `created_at`; `id DESC` is the stable
tiebreaker. Replay tools can consume rows in order and assume:
- no duplicate rows across calls with non-overlapping `--since` windows
- no missed rows across calls that chain `--since` windows (window end
of run 1 is the strict upper bound, not a soft cursor)
## Schema versioning promise
- **v1 (shipped v0.21.0)** — this document. All fields listed above.
- **Additive changes** increment gbrain minor version (v0.25.0, v0.23.0
…) and ship with new optional fields. Consumers keyed on known fields
ignore unknown keys and keep working.
- **Breaking changes** (rename, type change, removal) increment
`schema_version` to 2. Consumers MUST branch on `schema_version` to
stay compatible.
## `eval_capture_failures` — companion audit table
Not exported by `gbrain eval export`. Surfaced via `gbrain doctor`:
```sh
gbrain doctor # warns when failures in last 24h > 0
```
Reason enum (stable): `db_down` | `rls_reject` | `check_violation` |
`scrubber_exception` | `other`. Cross-process visibility is the whole
point — `gbrain doctor` runs in its own process and reads the table
directly, so in-process counters wouldn't work.
## Config + CONTRIBUTOR_MODE
Capture is **off by default** as of v0.25.0 (was on for everyone in
earlier drafts). Two paths to turn it on:
**Path A — env var (contributor opt-in, the common case):**
```bash
export GBRAIN_CONTRIBUTOR_MODE=1 # in ~/.zshrc or ~/.bashrc
```
**Path B — explicit config (`~/.gbrain/config.json`, file-plane only):**
```json
{
"engine": "postgres",
"database_url": "...",
"eval": {
"capture": true,
"scrub_pii": true
}
}
```
Resolution order (most explicit wins):
1. `eval.capture: true` in config → on
2. `eval.capture: false` in config → off (overrides CONTRIBUTOR_MODE=1)
3. `GBRAIN_CONTRIBUTOR_MODE === '1'` → on
4. otherwise → off
`scrub_pii` defaults to `true` independent of capture. Set
`eval.scrub_pii: false` to preserve raw query text (only if you control
the brain's distribution).
`gbrain config set eval.capture false` does **not** work — that
command writes the DB-plane config, and the MCP server reads the
file-plane. Edit the JSON directly or use the env var.
-159
View File
@@ -1,159 +0,0 @@
# `gbrain eval takes-quality` — reproducible cross-modal quality eval
v0.32+ ships a CI-able quality gate for the takes layer. Three frontier models
score a sample of takes against a 5-dimension rubric, the runner aggregates to
PASS / FAIL / INCONCLUSIVE, and the receipt persists to `eval_takes_quality_runs`
so a follow-up `trend` or `regress` can compare against history.
This doc is the consumer contract. The sibling [gbrain-evals](https://github.com/garrytan/gbrain-evals)
repo and any future CI gate read receipts shaped exactly like the JSON below.
Fields are additive-stable at `schema_version: 1`. A breaking shape change
bumps the version.
## Subcommands
| Command | Brain required? | Exit codes |
|---|---|---|
| `gbrain eval takes-quality run [flags]` | yes (samples takes) | 0 PASS, 1 FAIL, 2 INCONCLUSIVE |
| `gbrain eval takes-quality replay <receipt>` | **no** (disk-only) | 0 PASS, 1 FAIL, 2 INCONCLUSIVE |
| `gbrain eval takes-quality trend [flags]` | yes (reads runs table) | 0 |
| `gbrain eval takes-quality regress --against <receipt>` | yes | 0 OK, 1 regression |
`replay` is the only mode that runs without `DATABASE_URL` — it reads the
receipt file from disk and re-renders it. The other modes need the brain.
## `run` flags
| Flag | Default | Notes |
|---|---|---|
| `--limit N` | 100 | Random sample of N takes from the brain. |
| `--cycles N` | 3 (TTY) / 1 (non-TTY) | Up to N panel calls before giving up; early-stop on PASS or INCONCLUSIVE. |
| `--budget-usd N` | unset | Abort before next call's projected cost would exceed cap. Models without a `pricing.ts` entry fail loud (codex #4). |
| `--source db|fs` | `db` | `fs` is reserved for v0.33+. |
| `--slug-prefix P` | unset | Filter takes to pages whose slug starts with P. |
| `--models a,b,c` | `openai:gpt-4o,anthropic:claude-opus-4-7,google:gemini-1.5-pro` | Comma-separated panel. |
| `--json` | off | Emit the full receipt to stdout. |
## Receipt JSON shape (`schema_version: 1`)
```json
{
"schema_version": 1,
"ts": "2026-05-09T22:00:00.000Z",
"rubric_version": "v1.0",
"rubric_sha8": "abcd1234",
"corpus": {
"source": "db",
"n_takes": 100,
"slug_prefix": null,
"corpus_sha8": "abcd1234"
},
"prompt_sha8": "abcd1234",
"models_sha8": "abcd1234",
"models": ["openai:gpt-4o", "anthropic:claude-opus-4-7", "google:gemini-1.5-pro"],
"cycles_run": 3,
"successes_per_cycle": [3, 3, 2],
"verdict": "pass",
"scores": {
"accuracy": { "mean": 7.8, "min": 7, "max": 9, "scores": [9,7,7], "per_model": {...} },
"attribution": { "mean": 7.0, "min": 7, "max": 7, "scores": [7,7,7], "per_model": {...} },
"weight_calibration": { "mean": 7.5, "min": 7, "max": 8, "scores": [8,7,7], "per_model": {...} },
"kind_classification": { "mean": 7.2, "min": 7, "max": 8, "scores": [7,8,7], "per_model": {...} },
"signal_density": { "mean": 7.0, "min": 6, "max": 8, "scores": [8,7,6], "per_model": {...} }
},
"overall_score": 7.3,
"cost_usd": 1.85,
"improvements": ["..."],
"errors": [],
"verdictMessage": "PASS: every dim mean >=7 and min >=5 ..."
}
```
### Field reference
- `schema_version` — locks the contract. Adding optional fields is additive
and compatible. Renaming, removing, or changing semantics bumps the version.
- `rubric_version` + `rubric_sha8` — segregate trend rows by rubric epoch
(codex review #3). When the rubric definition changes, both fields update,
and trend mode groups runs accordingly so a stricter rubric doesn't
silently look like a quality drop.
- `corpus.corpus_sha8` — fingerprint over the joined takes-text the judge
saw. Determines whether two runs are over the "same" sample.
- `models_sha8` — fingerprint over the sorted model id list. Re-ordering
models in `--models` doesn't change the sha (sort is stable).
- `successes_per_cycle` — count of contributing models per cycle. A model
contributes when (a) its JSON parsed AND (b) every declared rubric dim
has a finite score (codex review #5 — missing-dim drops the contribution).
- `verdict``pass` if every dim mean >= 7 AND every dim min across
contributing models >= 5; `fail` otherwise; `inconclusive` if fewer than
2/3 models contributed complete scores.
- `cost_usd` — sum of per-call cost via `pricing.ts`. Unknown models when
`--budget-usd` is set produce a `PricingNotFoundError` before any call
fires.
## Receipt persistence
Receipts persist to **`eval_takes_quality_runs`** (DB-authoritative per
codex review #6) AND to disk at `~/.gbrain/eval-receipts/takes-quality-<corpus>-<prompt>-<models>-<rubric>.json`
as a best-effort artifact. The DB row carries the full receipt JSON in the
`receipt_json` JSONB column, so when the disk artifact is gone, `replay`
can still reconstruct via `loadReceiptFromDb` (v0.33+ flag wiring).
The 4-sha primary key is unique (`UNIQUE` constraint) so re-running an
identical eval is `INSERT ... ON CONFLICT DO NOTHING` — idempotent.
## Trend output
Plain text (default):
```
ts rubric verdict overall cost corpus
─────────────────────────────────────────────────────────────────────────────
2026-05-09T22:00:00 v1.0 pass 7.3 $1.85 abcd1234
2026-05-08T18:30:00 v1.0 fail 6.8 $1.92 ef567890
```
JSON shape (`--json`):
```json
{
"schema_version": 1,
"rows": [
{ "id": 42, "ts": "...", "rubric_version": "v1.0", "verdict": "pass",
"overall_score": 7.3, "cost_usd": 1.85, "corpus_sha8": "abcd1234" }
]
}
```
## Regress: gating CI on quality
```bash
# Capture a baseline.
gbrain eval takes-quality run --limit 100 --json \
> .ci/takes-quality-baseline.json
# Later, after changing the extraction prompt:
gbrain eval takes-quality regress --against .ci/takes-quality-baseline.json \
--threshold 0.5
# exit 0 → no regression past threshold
# exit 1 → some dim dropped > 0.5; CI fails
```
The threshold is the per-dim-mean drop counting as regression. Default 0.5.
Regress reuses the **same** model panel + slug prefix + source as the prior
receipt for an apples-to-apples compare. Diffs in `corpus_sha8` /
`prompt_sha8` / `rubric_sha8` are surfaced as informational warnings (the
runner doesn't refuse — that's the caller's call).
## Contract stability
The shape above is the read contract for downstream consumers. Anything
not listed (e.g. internal aggregator state, gateway providerMetadata) is
**not** in the receipt and may change without notice.
When you need to evolve the schema:
1. Additive optional field → no version bump; old consumers ignore the
new key, new consumers read it.
2. Renamed or removed field, or changed semantics → bump
`schema_version` to `2`; runner emits both shapes for one release as
a deprecation runway.
-176
View File
@@ -1,176 +0,0 @@
# Evaluation Metric Glossary
**Auto-generated from `src/core/eval/metric-glossary.ts`. Do not edit by hand.** Run `bun run scripts/generate-metric-glossary.ts` to regenerate.
Every metric `gbrain eval *` and `gbrain search stats` reports has a plain-English explanation here. Industry terms are preserved verbatim so users searching the literature find what we report.
## Retrieval Metrics
### Precision at k (P@k)
**Key:** `precision@k`
**Plain English:** Of the top k results the engine returned, what fraction were actually relevant? High precision means few junk results in the top of the list.
**Range:** 0..1, higher is better. P@10 = 0.7 means 7 of the top 10 results were on-topic.
### Recall at k (R@k)
**Key:** `recall@k`
**Plain English:** Of all the relevant results that exist in the brain, what fraction did the engine find in its top k? High recall means few missed answers.
**Range:** 0..1, higher is better. R@10 = 0.81 means out of every 100 questions, the right answer was in the top 10 for 81 of them.
### Mean Reciprocal Rank (MRR)
**Key:** `mrr`
**Plain English:** On average, how far down the list is the FIRST relevant result? An MRR of 1.0 means the first hit is always right; an MRR of 0.5 means it's typically at rank 2.
**Range:** 0..1, higher is better. Computed as the average of 1/rank-of-first-relevant-result across all test queries.
### Normalized Discounted Cumulative Gain at k (nDCG@k)
**Key:** `ndcg@k`
**Plain English:** Like precision@k, but the engine gets MORE credit for putting good results near the top than near rank k. A perfect ordering scores 1.0; a totally random ordering scores near 0.
**Range:** 0..1, higher is better. nDCG@10 above 0.65 is the common "ship it" threshold for hybrid retrieval on technical corpora.
## Retrieval-Quality / Evidence Metrics (NamedThingBench)
### Hit rate at 1 (Hit@1)
**Key:** `hit@1`
**Plain English:** Fraction of queries where the right page is the very first result. NamedThingBench hard-gates title-substring Hit@1 >= 0.95 and alias Hit@1 >= 0.98 — a query that is a page's name or title phrase should land it at rank 1, not "somewhere in the top 10".
**Range:** 0..1, higher is better.
### Hit rate at 3 (Hit@3)
**Key:** `hit@3`
**Plain English:** Fraction of queries where the right page is in the top 3 results. NamedThingBench requires the multi-chunk-dilution family to hit 1.0 — a page with one strong chunk among many weak ones must never be buried.
**Range:** 0..1, higher is better.
### Average rank-1 match score
**Key:** `avg_rank1_score`
**Plain English:** The mean base (pre-boost) retrieval score of the TOP result across recent searches, from `gbrain search stats`. It is NOT a labeled accuracy number — it is a drift signal: if this trends DOWN over time, retrieval quality is regressing (the early warning that would have caught the duplicate-page incident before a human did).
**Range:** 0..1. Watch the trend, not the absolute value; pair with the <0.6 / 0.6-0.85 / >=0.85 bucket counts for shape.
### Create-safety hint (evidence contract)
**Key:** `create_safety`
**Plain English:** A result's answer to "is this page already in the brain — safe to NOT write a new one?" Derived from the strongest evidence, NOT a raw score: exists (alias_hit / exact_title_match / high_vector_match — do not duplicate), probable (solid keyword match — prefer updating), unknown (weak match — look closer). An agent keys its don't-duplicate decision off this, which is what prevents the incident's duplicate-stub class.
**Range:** enum: exists | probable | unknown
## Set-Similarity / Stability Metrics
### Jaccard similarity at k (set Jaccard @k)
**Key:** `jaccard@k`
**Plain English:** How much do two result lists overlap? Compare the top k slugs from the captured baseline against the current run; Jaccard@10 = 1.0 means perfect agreement, 0.0 means zero overlap.
**Range:** 0..1, higher = more stable. Below 0.5 on a stable corpus means retrieval changed significantly.
### Top-1 stability rate
**Key:** `top1_stability`
**Plain English:** Fraction of queries where the #1 result is the same between two runs. The most aggressive stability check — small ranking shifts that don't change the top answer don't hurt it.
**Range:** 0..1, higher = more stable. Above 0.85 typically means safe-to-merge for retrieval changes.
## Statistical-Significance Metrics
### p-value (paired bootstrap)
**Key:** `p_value`
**Plain English:** How likely the observed difference between two modes is just noise. Lower = stronger evidence the difference is real. We compute paired bootstrap with 10,000 resamples and Bonferroni correction across the 12 comparisons (3 modes × 4 metrics).
**Range:** 0..1, lower = stronger signal. Below 0.05 is the common "statistically significant" threshold; below 0.01 is strong evidence.
### 95% Confidence Interval (CI)
**Key:** `confidence_interval`
**Plain English:** The range we're 95% sure the true value falls inside, given the sample we measured. Narrower CI = more reliable estimate. Computed via bootstrap resampling.
**Range:** Two-tuple [low, high]. If 0 is inside the CI for a Δ, the difference isn't statistically significant.
## Operational / Cost Metrics
### Cache hit rate
**Key:** `cache_hit_rate`
**Plain English:** Fraction of searches that reused a recent cached answer instead of running fresh. Higher hit rate = lower latency + lower LLM spend, but stale results may slip through if the threshold is too loose.
**Range:** 0..1, higher generally better. 0.7-0.9 is the sweet spot for a busy brain; above 0.9 may indicate the similarity threshold is too loose.
### Average results returned
**Key:** `avg_results`
**Plain English:** Mean number of search-result rows the engine returned per call. Should be near the active mode's searchLimit unless the brain is small or the budget is dropping results.
**Range:** 0..searchLimit. Far below searchLimit suggests budget pressure or sparse retrieval.
### Average tokens delivered
**Key:** `avg_tokens`
**Plain English:** Estimated tokens (chars / 4) in the chunk text returned per search call. The direct measure of how much context an agent loop is paying for each search.
**Range:** 0..tokenBudget. Approximates OpenAI tiktoken count for English; off by ~5-10% for Anthropic and worse for non-English.
### Cost per query (USD)
**Key:** `cost_per_query_usd`
**Plain English:** Sum of LLM + embedding API charges for one search call. Includes Haiku expansion call (tokenmax mode only) + embedding cost + downstream answer-model cost if measured.
**Range:** 0..unbounded. Conservative mode is typically <\$0.001 per call; tokenmax with answer-gen can exceed \$0.01.
### p99 latency (ms)
**Key:** `p99_latency_ms`
**Plain English:** 99th percentile wall-clock time per search call. The latency that 1% of users see — long-tail experience, not the average.
**Range:** 0..unbounded. Warm-cache hits should be <50ms; tokenmax with expansion can exceed 200ms due to the Haiku call.
## Result-Sizing Metrics
### Autocut signal
**Key:** `autocut.signal`
**Plain English:** Which signal autocut used to size the result set. 'rerank' means it found a real score cliff in the cross-encoder rerank scores and cut there; 'none' means no trustworthy cliff (no reranker, <2 scored results, or the gap was too small) so it returned the full list.
**Range:** 'rerank' | 'none'. 'none' is not a failure — it means autocut declined to cut because the signal didn't justify it.
### Autocut gap ratio
**Key:** `autocut.gap_ratio`
**Plain English:** The size of the largest score drop autocut found, as a fraction of the top result's score. A gap of 0.40 means the score fell by 40% of the top score at the steepest point. Autocut cuts there only when this clears the sensitivity threshold (autocut_jump, default 0.20).
**Range:** 0..1, higher = a sharper cliff (more confident cut). Below the autocut_jump threshold → no cut.
---
## Coverage
Every metric printed by any `gbrain eval *` or `gbrain search stats` command resolves through `getMetricGloss()` in `src/core/eval/metric-glossary.ts`. Adding a new metric to the glossary REQUIRES updating this doc; the CI guard catches drift.
-285
View File
@@ -1,285 +0,0 @@
# Search Mode Evaluation Methodology
_How v0.32.3 measures the difference between `conservative`, `balanced`, and `tokenmax`. Written haters-immune: every claim is reproducible from the committed dataset + raw outputs._
## 1. What this measures and what it doesn't
**Measures:** retrieval quality and operational cost on fixed public datasets, under each named search mode, against the same brain content.
**Does NOT measure:**
- Your specific brain content (this is a benchmark, not your bill).
- Your specific query distribution.
- End-user satisfaction or downstream task success.
- Latency under concurrent load.
- Production cost (the cost numbers are model-pricing estimates × dataset size, not your actual API spend).
If you want to know how a mode behaves on YOUR brain, run `gbrain search stats --days 30` after a real usage window, then run `gbrain search tune` for actionable recommendations.
## 2. Datasets and sizes
- **LongMemEval** — public split, `n=500` questions. Downloaded from [Hugging Face](https://huggingface.co/datasets/xiaowu0162/longmemeval). The corpus + answer keys are pinned to a specific commit; recorded in every per-run record.
- **Replay captures** — NDJSON from the sibling `gbrain-evals` repo, `n=200` queries. Each query carries a `retrieved_slugs` baseline + a `latency_ms` measurement from the original production run.
- **BrainBench v1**`n=1240` documents / `n=350` qrels (binary relevance judgments). Lives in the sibling [`gbrain-evals`](https://github.com/garrytan/gbrain-evals) repo, SHA-pinned at every run.
No private brain content is used in any reported result. The committed NDJSON dumps under `<repo>/.gbrain-evals/` contain only the LongMemEval question IDs + the rank-ordered retrieved session IDs.
## 3. Sample selection
- **Random seed:** `42` throughout. Set via `--seed N` on `gbrain eval run-all`; recorded in every per-run record.
- **No per-question curation.** Splits are taken whole; no question is filtered for reporting.
- **No mode-specific tuning.** The same dataset + same seed feeds every mode. The mode is the only independent variable.
- **Stability across re-runs:** with `--seed 42` and the same dataset SHA, two runs of the same (mode, suite) produce identical retrieval orderings (modulo the optional Haiku expansion call, which is non-deterministic). Persisted in `eval_results` so anyone can re-score from the committed dumps.
## 4. Run procedure
The command is the doc. Anyone can reproduce.
```bash
# Setup: in your gbrain working tree, with OPENAI_API_KEY + ANTHROPIC_API_KEY exported.
git rev-parse HEAD # record the commit for the methodology footer
# Sweep all 3 modes × 2 retrieval-focused suites with seed 42.
gbrain eval run-all \
--modes conservative,balanced,tokenmax \
--suites longmemeval,replay \
--seed 42 \
--limit 500 \
--budget-usd-retrieval 5 \
--budget-usd-answer 20 \
--output docs/eval/results/v0.32.3/
# Render the comparison.
gbrain eval compare --md > docs/eval/results/v0.32.3/README.md
gbrain eval compare --json > docs/eval/results/v0.32.3/comparison.json
```
The orchestrator writes per-run records to `<repo>/.gbrain-evals/eval-results.jsonl`. Every record carries: `run_id`, `ran_at`, `suite`, `mode`, `commit`, `seed`, `limit`, `params`, `status`, `duration_ms`. The dumps under `docs/eval/results/v0.32.3/` carry the raw question-level outputs so a reviewer can re-score with their own metric implementation.
## 5. Threats to validity
Honest list. We name what would let a critic dismiss the numbers.
- **LongMemEval skews English + technical.** The questions are software-engineering and consumer-product flavored. Performance on a brain rich in non-English / non-technical content (writing, art history, etc.) may differ.
- **BrainBench is small** (1240 docs) relative to a production brain (10K-100K pages). Absolute scores aren't predictive of your hit rate; the _delta_ between modes is.
- **char/4 token heuristic.** Token-budget enforcement and cost estimates use a character-count / 4 heuristic. Accurate within ~5-10% for English with the OpenAI tiktoken family; off worse for Voyage (we don't use Voyage in chat retrieval, so it doesn't bias the reported numbers, but if you do, your budget caps will be approximate).
- **Expansion's quality lift varies by query distribution.** The eval data shows ~97.6% relative quality with LLM expansion vs without (i.e., barely measurable lift) on the LongMemEval corpus. On rarer-entity / longer-tail queries, the lift can be larger. We report the corpus we measured; YMMV.
- **Paired bootstrap assumes question-level independence.** Multi-hop questions within the same conversation thread aren't independent; the bootstrap CI is slightly tighter than reality.
- **Single brain instance per benchmark.** The benchmark spins up an in-memory PGLite per question. Cache hit rate measured here doesn't reflect a long-running production brain's cache state.
## 6. Per-question raw outputs
Every reported metric is reproducible from the NDJSON dumps committed at `docs/eval/results/v0.32.3/`. The commit SHA in the methodology footer pins the code version.
**Examples per mode:** the auto-generated `README.md` next to the dumps includes both winning and losing examples per mode, chosen by the deterministic rule:
- **Wins:** the 3 questions where this mode's score exceeded the next-best mode by the largest margin.
- **Losses:** the 3 questions where this mode's score fell short of the next-best mode by the largest margin.
Picked by the score delta, NOT cherry-picked by hand. The README documents the rule so a critic can verify.
## 7. Pre-registered expectations
Before running, we expect:
1. **tokenmax wins Recall@10** by 5-15 percentage points over conservative. LLM expansion + 50-result ceiling helps rare-entity surface forms.
2. **conservative wins cost-per-query** by 5-15× over tokenmax. No Haiku expansion + tight 4K budget cap = single-digit-cent queries.
3. **balanced lands within 3pp of tokenmax** on Recall@10. Intent weighting (zero-LLM cost) closes most of the expansion gap on common queries.
4. **No mode breaks nDCG@10 ≥ 0.65** — the published "ship it" threshold for hybrid retrieval on technical corpora.
Then we publish whether the data agrees. **If a hypothesis fails, that's documented honestly** in the release README, not buried. Pre-registration is what makes the comparison defensible — without it, a "we expected X and got X" outcome is observation, not prediction.
## 8. Re-run cadence
This document + the eval results are regenerated on every release that touches retrieval-affecting code. The `gbrain doctor eval_drift` check surfaces changes to the curated watch-list in `src/core/eval/drift-watch.ts`:
- `src/core/search/**`
- `src/core/embedding.ts`
- `src/core/chunkers/**`
- `src/core/ai/recipes/anthropic.ts`
- `src/core/ai/recipes/openai.ts`
- `src/core/operations.ts`
Additions to the watch-list require a CHANGELOG line.
## Statistical-significance discipline
When `gbrain eval compare --md` reports a Δ between two modes, it computes:
- **Paired bootstrap** with 10,000 resamples per metric. Each resample draws _question-level_ pairs (same question, mode A vs mode B), so question-level variance is differenced out.
- **Bonferroni correction** across the 12 comparisons (3 modes × 4 metrics). The reported p-value is the comparison's raw p-value × 12 (clamped at 1.0).
- **95% confidence intervals** computed from the bootstrap distribution.
If the CI for a Δ includes 0 OR the Bonferroni-adjusted p-value exceeds 0.05, the difference is **not** statistically significant. The MD report says "not significant" verbatim.
## Glossary
Every metric the report prints has a plain-English entry in `docs/eval/METRIC_GLOSSARY.md`, auto-generated from `src/core/eval/metric-glossary.ts`. The CI guard at `scripts/check-eval-glossary-fresh.sh` regenerates and diffs against the committed file on every test run; a stale doc fails the build.
## Cost anchors
The mode-picker prompt at `gbrain init` and the CLAUDE.md `## Search Mode` table both surface these rough cost anchors. Working through the math so they're auditable:
**Variables:**
- `T` = avg tokens per search-result chunk. The recursive chunker targets 300 words / chunk → ~400 tokens (English, OpenAI tiktoken approx).
- `N` = chunks delivered per query (capped by the mode's `searchLimit`).
- `R` = downstream model input rate. Sonnet 4.6 = \$3/M. Opus 4.7 = \$5/M. Haiku 4.5 = \$1/M.
- `Q` = queries per month.
**Per-query input cost** (downstream agent reads the chunks):
cost_per_query = T × N × R
| Mode | T (tokens) | N (chunks) | Sonnet (\$3/M) | Opus (\$5/M) | Haiku (\$1/M) |
|---|---|---|---|---|---|
| conservative (4K cap, 10 max) | ~400 | 10 (or fewer if budget hits) | \$0.012 | \$0.020 | \$0.004 |
| balanced (12K cap, 25 max) | ~400 | ~25 | \$0.030 | \$0.050 | \$0.010 |
| tokenmax (no cap, 50 max) | ~400 | ~50 | \$0.060 | \$0.100 | \$0.020 |
**Monthly cost** (Q × per-query):
| Mode @ Sonnet | 1K Q/mo | 10K Q/mo | 100K Q/mo |
|---|---|---|---|
| conservative | \$12 | \$120 | \$1,200 |
| balanced | \$30 | \$300 | \$3,000 |
| tokenmax | \$60 | \$600 | \$6,000 |
| Mode @ Opus | 1K Q/mo | 10K Q/mo | 100K Q/mo |
|---|---|---|---|
| conservative | \$20 | \$200 | \$2,000 |
| balanced | \$50 | \$500 | \$5,000 |
| tokenmax | \$100 | \$1,000 | \$10,000 |
**gbrain's own cost** on top:
- Query embedding (text-embedding-3-large @ \$0.13/M tokens): ~\$0.00001 per query. Negligible at every scale.
- Tokenmax Haiku expansion call (\$1/M input, \$5/M output, ~500 input + 200 output per call): ~\$0.0015 per query, or \$150/mo at 100K queries. Cache hits cut this in half.
- Per-page indexing (one-time): bounded by your import volume, not query volume. Not modeled here.
**Cache hit adjustment.** A warmed brain typically sees 30-50% cache hits on repeat-query traffic. Cache hits skip the downstream input cost entirely (the cached result was already in the agent's context once). So real-world costs run ~50-70% of the table above on a busy brain.
**Why these numbers DRIFT from your actual bill:**
- Your agent's system prompt + reasoning tokens add input that gbrain doesn't see.
- Compaction reduces input over a long session.
- Most agents make 1-5 searches per turn; cost-per-turn is what bills you, not cost-per-query.
- The model price column drifts as providers reprice; pin the rate via `src/core/model-pricing.ts` (the canonical chat-pricing table) for a current snapshot.
The picker copy + CLAUDE.md table are the canonical user-facing source. Update them in lockstep when the underlying chunker size or default `searchLimit` changes.
## Mode × Model matrix (the 25x spread)
The per-query math above assumes Sonnet 4.6 downstream. In reality, the
downstream model tier is the BIGGER cost lever. Per-query cost at 10K
queries/month (typical single-user volume), search payload only (no cache
savings):
| Mode (search tokens) | Haiku 4.5 (\$1/M) | Sonnet 4.6 (\$3/M) | Opus 4.7 (\$5/M) |
|---|---|---|---|
| conservative (~4K) | **\$40/mo** | \$120/mo | \$200/mo |
| balanced (~10K) | \$100/mo | \$300/mo | \$500/mo |
| tokenmax (~20K) | \$200/mo | \$600/mo | **\$1,000/mo** |
Scales linearly: multiply by 10 for 100K/mo (heavy power user / multi-user
fleet); divide by 10 for 1K/mo (light usage).
**Natural pairings span ~4x** (cheap model + tight mode → frontier model + loose
mode). **Mismatches waste capacity:**
- `tokenmax + Haiku`: Haiku gets 20K of search results stuffed into its
context per query. Haiku's reasoning is weaker; more chunks = more noise,
not more signal. You pay Haiku rates but get sub-Haiku quality. Wrong
direction.
- `conservative + Opus`: Opus has 200K context window and can synthesize
across many chunks. Capping at 10 chunks / 4K tokens leaves Opus
reasoning underfed. You pay Opus rates but get conservative-shape
retrieval. Wasted spend.
**Right-sizing rule:** match the mode's `searchLimit` to the downstream
model's "useful context depth":
- Haiku struggles past ~5-10 chunks of cross-referenced content → conservative
- Sonnet handles ~25-40 chunks well → balanced
- Opus benefits from 50+ chunks for multi-hop reasoning → tokenmax
## Realistic-scale anchor (single power-user agent loop)
The per-query math above is honest but theoretical: it treats each search as an isolated billable event. Real agent loops amortize a lot of context across turns via Anthropic prompt caching. Here's what one heavy power-user loop actually looks like in production, anonymized + scaled so the numbers represent a representative power user rather than any specific deployment.
**Reference shape — tokenmax in production at a single-user scale:**
| Quantity | Approximate value |
|---|---|
| 30-day total agent spend | ~\$700/mo |
| 30-day total tokens billed | ~800M |
| Turns per month | ~860 (~29/day; one active agent loop) |
| Average tokens per turn | ~900K |
| Average cost per turn | ~\$0.85 |
| Anthropic prompt-cache hit rate | ~88% |
A "turn" here is one agent loop iteration: read user message, plan, execute tool calls (including gbrain searches), generate response. Each turn typically includes 2-4 gbrain searches.
**Per-mode scaling from the tokenmax anchor:**
The cost difference between modes is concentrated in the search-attributable fraction of per-turn cost. System prompt, tool definitions, conversation history, and reasoning tokens don't change with mode — only the chunks gbrain delivers do. Assume 3 searches per turn at the mode's `searchLimit`:
| Mode | Search tokens/turn | Search cost/turn (at \$3/M effective) | Search-attributable @ 860 turns | Δ vs tokenmax |
|---|---|---|---|---|
| tokenmax | ~60K (3 × 20K) | ~\$0.18 | ~\$155/mo | — |
| balanced | ~30K (3 × 10K) | ~\$0.09 | ~\$77/mo | -\$78 |
| conservative | ~12K (3 × 4K) | ~\$0.036 | ~\$31/mo | -\$124 |
**Implied total agent spend by NATURAL PAIRING** (mode + matched
downstream model). Per-turn cost scales with the downstream model's
per-token rate, since the cached prefix + uncached portion + reasoning
tokens all bill at that rate:
| Pairing | Per-turn cost | Total @ 860 turns/mo |
|---|---|---|
| tokenmax + Opus (frontier, max quality) | ~\$0.85 | ~\$700/mo |
| balanced + Sonnet (the sweet spot) | ~\$0.50 | ~\$430/mo |
| conservative + Haiku (cost-sensitive) | ~\$0.20 | ~\$170/mo |
**4x spread across natural pairings.** The model tier dominates because
the per-token rate applies to the WHOLE per-turn payload (system + tools
+ history + reasoning + search), not just gbrain's chunks. Mode choice
contributes ~10-20% on top of that base.
**Mismatched pairings push you off the curve:**
| Pairing | Per-turn estimate | Total @ 860 turns/mo | Compared to natural |
|---|---|---|---|
| tokenmax + Haiku | ~\$0.20 | ~\$170/mo | Same cost as conservative+Haiku, worse quality |
| conservative + Opus | ~\$0.75 | ~\$640/mo | 92% of tokenmax+Opus spend, conservative-shape retrieval |
The mismatch math says: a tokenmax+Haiku user pays the same as
conservative+Haiku but gets a noisier context (Haiku can't filter signal
from 50 chunks). A conservative+Opus user pays nearly the same as
tokenmax+Opus but starves Opus on retrieval depth. Both burn budget for
no improvement.
**What this anchor tells us that the per-query math doesn't:**
1. **At realistic agent-loop scale with disciplined prompt caching, mode choice saves 10-20% of total agent spend** — meaningful, but smaller than the per-query 5x ratio implies. Disciplined prompt-cache layouts blunt the mode delta because most of the per-turn cost is the cached prefix, not the search payload.
2. **Without that prompt-cache discipline, the per-query framing reasserts itself.** Setups that churn the prompt prefix on every turn (frequent system-prompt edits, untemplated tool defs, no prompt-cache structuring) see search payload contribute a much larger fraction of total cost. Those setups should care about mode choice more, not less.
3. **The cache hit rate quoted here (~88%) is achievable but not automatic.** It requires structuring the prompt so the cached prefix stays stable across turns: system prompt + tool defs first, history compacted but cache-aware, retrieved chunks appended LAST (where their volatility doesn't invalidate the prefix). Agents that interleave search results inside the cached region pay the prefix-rebuild tax on every turn.
**Caveats stacked here:**
- The anchor represents ONE power-user loop. Multi-user fleets aggregate proportionally; the per-user shape doesn't change.
- The "3 searches per turn" assumption varies wildly. A code-review agent might issue 10+ searches per turn; a chat-only loop might do 0.
- The 88% cache hit rate is the high end of what's achievable. Half that is closer to a default agent without cache-aware prompt layout.
- The "Δ vs tokenmax" math assumes the OTHER cost components (system, tools, history, reasoning) stay constant. In practice, conservative's smaller per-turn payload also leaves more room in the context window for history → which can change agent behavior in either direction.
This anchor + the per-query math both live in this doc on purpose. The per-query framing is what an isolated benchmark would measure (and what `gbrain eval run-all` will produce). The realistic-scale anchor is what an operator actually pays. Both are honest; neither is the whole truth.
## Reproducibility footer
Every release that publishes eval numbers includes a footer with:
- Code commit SHA
- Dataset SHA (LongMemEval, BrainBench, Replay)
- `--seed N`
- Run commands verbatim
- API model identifiers used (Anthropic + OpenAI + judge model)
Without these, the numbers are unfalsifiable. With them, anyone with API keys can re-score.
-102
View File
@@ -1,102 +0,0 @@
# Content Guardrail Seams
GBrain exposes **vendor-neutral guardrail seams** at the boundaries where
external content enters the retrieval layer and where queries/tool-inputs enter
the LLM gateway. A guardrail is any external classifier — a content firewall, a
prompt-injection detector, a PII scrubber — that wants to *observe* content at
those boundaries.
The OSS distribution ships **inert**: zero guardrails are registered by default,
and every seam is a no-op until an operator registers a provider.
## Design contract (hard invariants)
These hold for every seam and are enforced by `test/guardrails.test.ts`:
- **Observe-only.** `runGuardrails()` returns `void`. Callers never branch on a
provider verdict. A guardrail registered through this interface *cannot*
block, rewrite, drop, retry, or reorder GBrain behavior. Enforcement, if ever
added, will get its own explicitly-named seam and its own RFC — it will not
silently reuse this one.
- **Fail open.** Missing config, provider throw/reject, timeout, and network
error are all swallowed. A broken guardrail never breaks an ingest, a query,
or a tool call.
- **Inline await.** Hooks await the provider before proceeding, so the
classifier sees content at the exact pre-persist / pre-inference moment.
- **No verdict persistence.** GBrain writes no guardrail rows. Providers own
their own audit trail.
- **Content boundaries.** Hooks pass only the ingest/user-facing payload — the
markdown/code body, the last user message, the expansion query, the tool
input. They never pass system prompts, full chat history, tool *output*, LLM
output, embeddings, or multimodal/OCR/rerank payloads.
## The five seams
All seams call `runGuardrails({ hook, content, metadata })` from
`src/core/guardrails.ts`.
| `hook` | Location | Fires |
| --- | --- | --- |
| `file_storage.markdown` | `import-file.ts``importFromContent` | After `parseMarkdown` + size guard, **before** content-sanity, hashing, chunking, embedding, DB write |
| `file_storage.code` | `import-file.ts``importCodeFile` | After code size guard, **before** hashing, code-chunking, embedding, DB write |
| `ai_gateway.chat` | `ai/gateway.ts``chat` | On the **latest user message only**, before provider inference |
| `ai_gateway.expand` | `ai/gateway.ts``expand` | On the query, before the expansion model call |
| `ai_gateway.tool_input` | `ai/gateway.ts``toolLoop` | On `{toolName, input}`, before pending-persist and before tool execution |
The two `file_storage.*` hooks cover every natural ingest caller that routes
through `importFromContent` / `importCodeFile`: `gbrain import`, sync, capture,
`put_page`, subagent `brain_put_page`, trusted-workspace writes,
`ingest_capture`, inbox daemon dispatch, reindex, code reindex, and the public
import APIs.
## Writing a guardrail provider
```ts
import { registerGuardrailProvider, type GuardrailInput } from 'gbrain/core/guardrails';
registerGuardrailProvider({
id: 'my-firewall',
async classify(input: GuardrailInput) {
// input.hook — which boundary ('file_storage.markdown', etc.)
// input.content — the raw text to classify
// input.metadata — provider-opaque context (slug, source_kind, tool_name, model, ...)
//
// Do your own timeout/retry/logging here. The return value is IGNORED by
// GBrain — return a typed verdict only if your own audit code consumes it.
await fetch(MY_API, { method: 'POST', body: JSON.stringify({ text: input.content }) });
},
});
```
Register once at process init (e.g. from a plugin entry or an operator boot
hook). Registration is idempotent by `id`, so a re-init won't double-fire.
### Provider responsibilities
GBrain deliberately keeps the seam minimal. The provider owns:
- **Timeout discipline.** GBrain does not impose a timeout in `runGuardrails`
so you can tune per-deployment latency. Use an `AbortController`.
- **Secret handling.** Read API keys from env at call time. Never log the key.
- **Redacted logging.** Don't log raw classified content (it may itself be the
payload you're trying to protect). Log a hash + verdict, not the body.
- **Async fan-out.** If you don't want to block ingest on your classifier,
enqueue inside `classify` and return immediately. The seam awaits *your*
function; what it does is up to you.
## Example: shadow-mode firewall provider
A typical "shadow mode" provider (classify, log a redacted verdict, change
nothing) is ~80 lines and lives entirely in the provider's own package. See
the reference provider doc shipped to integration partners for a complete
`classify` implementation that:
1. resolves `<base>/classify` from an env URL,
2. posts `{ text, hook, metadata }` with an `x-api-key` header,
3. parses a `{ prediction, blocked, score, threshold }` response,
4. emits one redacted stderr line (`status=… prediction=… content_sha256=…`),
5. fails open on every error path.
Because the verdict is ignored by GBrain, "shadow mode" requires *no* special
GBrain flag — it is the only mode this interface supports. Enforcement would be
a separate, future, RFC-gated seam.
-200
View File
@@ -1,200 +0,0 @@
# How a downstream agent should talk to gbrain
This guide is for authors of downstream agents (hermes, openclaw, future
forks) that need to call gbrain operations from their own runtime. Reading
this first will save you a debugging cycle: gbrain has **two distinct
surfaces**, and which one you pick depends on the operation.
## The two surfaces
```
┌─────────────────────────────────────────────┐
│ gbrain process │
│ │
Agent (hermes, │ ┌──────────────────┐ ┌────────────────┐ │
openclaw, fork) ────┼──▶ MCP ops surface │ │ localOnly │ │
│ │ (HTTP + OAuth) │ │ admin ops │ │
│ │ │ │ │ │
│ │ search, query, │ │ sync, embed, │ │
│ │ put_page, │ │ dream, doctor,│ │
│ │ get_page, │ │ autopilot, │ │
│ │ find_experts, │ │ init, secrets │ │
│ │ ... │ │ │ │
│ └──────────────────┘ └────────────────┘ │
│ ▲ ▲ │
│ │ │ │
│ │ │ │
│ thin-client OAuth shell-job `inherit:`
│ (preferred for (only path for │
│ MCP-equivalent ops) localOnly ops) │
└─────────────────────────────────────────────┘
```
The two surfaces are **not interchangeable**. Pick by op, not by preference.
## Surface 1 — MCP ops over HTTP (thin-client + OAuth)
Use for any operation that has an MCP equivalent: `search`, `query`,
`put_page`, `get_page`, `find_experts`, `find_orphans`, `find_anomalies`,
`get_recent_salience`, `find_trajectory`, and so on. The canonical list is
the set of ops in `src/core/operations.ts` whose `localOnly` flag is unset
(or `false`).
### Setup
The host runs gbrain as a long-lived HTTP server:
```bash
GBRAIN_ALLOW_SHELL_JOBS=1 gbrain serve --http --port 3131
```
The agent registers as an OAuth client (one-time):
```bash
gbrain auth register-client hermes \
--grant-types client_credentials \
--scopes read,write
# Prints client_id + client_secret one-time. Store securely.
```
The agent's runtime calls `/mcp` with a bearer token from `client_credentials`
grant. Secrets stay in the gbrain serve process; the agent never sees
DATABASE_URL or API keys.
Thin-client mode (`gbrain init --mcp-only`) gives the agent the same
client-credentials wiring, plus the `gbrain` CLI itself routes MCP-eligible
commands through the configured remote MCP. The agent can call
`gbrain search` / `gbrain query` directly and the CLI does the OAuth dance.
### Why this is preferred for MCP ops
- Secrets never leave the server process.
- OAuth scopes give you `read`, `write`, `admin` separation — agent only gets
what it needs.
- Source-scoped tokens (`--source dept-x` on `register-client`) confine the
agent to a specific source within a federated brain.
- One audit surface (`mcp_request_log`) covers every op call uniformly.
## Surface 2 — localOnly admin ops via shell-job `inherit:`
Some operations are flagged `localOnly: true` in `src/core/operations.ts` and
are **refused** in thin-client mode at `src/cli.ts:isThinClient`. The full
list (as of v0.36.5.0) includes:
- `sync` (filesystem walks need local FS access)
- `embed` (orchestrates the embed pipeline)
- `extract` (walks markdown files)
- `dream` (synthesis cycle)
- `doctor` (filesystem hygiene checks)
- `autopilot` (background daemon orchestration)
- `init` (creates `~/.gbrain/`)
- `secrets` (config management)
For these, the agent cannot route through HTTP MCP. The only path is to run
`gbrain` as a CLI subprocess. The recommended pattern is to submit the
subprocess as a shell job to the gbrain Minions worker so retry / backoff /
DLQ / audit trail all come for free.
### Setup
```bash
gbrain jobs submit shell --params '{
"cmd": "gbrain sync --skip-failed && gbrain embed --stale",
"cwd": "/data/gbrain",
"inherit": ["database_url"]
}'
```
The `inherit: ["database_url"]` field tells the worker to look up
`database_url` from its `loadConfig()` and inject the value into the child
env as `GBRAIN_DATABASE_URL`. The DB row in `minion_jobs.data` carries the
names only — `inherit: ["database_url"]` — never the value. See
[minions-shell-jobs.md#secrets](./minions-shell-jobs.md#secrets) for the
full validation rules and error catalog.
### Why this is preferred over writing secrets into `env:` per-job
- Pre-v0.36.5.0 callers passed `env: { GBRAIN_DATABASE_URL: "postgresql://..." }`
per job. The URL landed plaintext in `minion_jobs.data` and the shell-audit
JSONL. Anyone with brain-DB read access (or a brain dump, or a shared brain
via mounts) saw the URL. As of v0.36.5.0, this is rejected at pre-enqueue
validation. The error message names `inherit: ["database_url"]` as the
replacement.
### Worker setup (one-time, per host)
The agent's host needs a worker that processes shell jobs:
```bash
# One-shot inline execution (PGLite or Postgres):
gbrain jobs submit shell --params '{...}' --follow
# Persistent worker (Postgres only — PGLite uses --follow inline):
GBRAIN_ALLOW_SHELL_JOBS=1 gbrain jobs work
```
`GBRAIN_ALLOW_SHELL_JOBS=1` is the worker-side opt-in. Without it, shell jobs
sit in `waiting` indefinitely. Set it on the worker process env (or in your
deploy unit / launchd plist), not per-submission — submitter env is a weak
proxy for worker env.
## Decision table
| Operation | Surface | Why |
|---|---|---|
| `search` / `query` | HTTP MCP via thin-client | Has MCP op; OAuth-scoped. |
| `get_page` / `list_pages` | HTTP MCP | Same. |
| `put_page` | HTTP MCP | Same; respects subagent allow-list when applicable. |
| `find_experts` / `find_orphans` | HTTP MCP | Same. |
| `sync` / `embed` / `extract` | Shell job + `inherit:` | `localOnly: true`. |
| `dream` | Shell job + `inherit:` | `localOnly: true`. |
| `doctor` | Shell job + `inherit:` (or no inherit if no DB) | `localOnly: true`. |
| `autopilot` | Run as a daemon directly on the host | Long-lived, not job-shaped. |
| `init` / `secrets` | One-time host setup | Operator action, not agent action. |
## Recommended patterns
- **Prefer `inherit:` for secrets you don't want in the row.** Names land in
`minion_jobs.data`; values resolve at child-spawn from the worker's config.
If a brain DB ever traverses a trust boundary, secrets stay out.
- **Free-form names.** `inherit:` accepts any snake_case config-key on your
worker — `database_url`, `anthropic_api_key`, `openai_api_key`,
`openrouter_api_key`, `voyage_api_key`, `groq_api_key`,
`zeroentropy_api_key`, or any custom
field you stuff into `~/.gbrain/config.json`. The agent picks what it
needs.
- **`env:` still works** for non-secret values, or for cases where you
WANT the value in the row (e.g. an opaque correlation token your audit
flow needs to read back later). The validator doesn't second-guess you.
- **Never try to route a `localOnly` op through thin-client MCP.** It will
fail with `localOnly op refused in thin-client mode`. Use shell-job +
`inherit:` (for secrets) or `env:` (for non-secrets).
## Migration: from pre-v0.36.5.0
If your agent submits shell jobs that pass secrets via `env:`:
```jsonc
// Pre-v0.36.5.0: works but URL persists in minion_jobs.data plaintext.
{
"cmd": "gbrain sync --skip-failed",
"cwd": "/data/gbrain",
"env": { "GBRAIN_DATABASE_URL": "postgresql://..." }
}
```
Switch to (recommended):
```jsonc
// v0.36.5.0+: name in row, value resolved at child-spawn from worker config.
{
"cmd": "gbrain sync --skip-failed",
"cwd": "/data/gbrain",
"inherit": ["database_url"]
}
```
Make sure the worker host has `database_url` configured (either via
`gbrain config set database_url <value>` or via `GBRAIN_DATABASE_URL` /
`DATABASE_URL` env on the worker process). If the worker can't resolve the
key, the validator rejects the job at submit time with a paste-ready hint.
+1 -1
View File
@@ -69,7 +69,7 @@ update_brain_page(slug, new_info, source):
page = gbrain get {slug}
// TIMELINE: always APPEND (never edit existing entries)
gbrain timeline-add {slug} {
gbrain add_timeline_entry {slug} {
date: today,
summary: new_info.summary,
detail: new_info.detail,
+9 -9
View File
@@ -46,10 +46,10 @@ on user_shares_media(url_or_file):
# Step 4: Extract and cross-reference entities
for person in transcript.mentioned_people:
gbrain link <slug> <person_slug>
gbrain link <person_slug> <slug>
gbrain timeline-add <person_slug> {date} \
"Discussed in {video_title}: {what_was_said}" \
gbrain add_link <slug> <person_slug>
gbrain add_link <person_slug> <slug>
gbrain add_timeline_entry <person_slug> \
--entry "Discussed in {video_title}: {what_was_said}" \
--source "YouTube: {url}"
# PATTERN 2: Social Media Bundles
@@ -80,8 +80,8 @@ on user_shares_media(url_or_file):
# Extract entities and cross-reference
for entity in bundle.mentioned_entities:
gbrain link <slug> <entity_slug>
gbrain link <entity_slug> <slug>
gbrain add_link <slug> <entity_slug>
gbrain add_link <entity_slug> <slug>
# PATTERN 3: PDFs and Documents
elif media.type == "pdf" or media.type == "document":
@@ -109,8 +109,8 @@ on user_shares_media(url_or_file):
"""
for entity in document.mentioned_entities:
gbrain link <slug> <entity_slug>
gbrain link <entity_slug> <slug>
gbrain add_link <slug> <entity_slug>
gbrain add_link <entity_slug> <slug>
# Always sync after ingestion
gbrain sync
@@ -127,7 +127,7 @@ on user_shares_media(url_or_file):
## How to Verify
1. Ingest a YouTube video. Run `gbrain get media/youtube/{slug}`. Confirm the page has: the agent's analysis (not just a summary), key quotes with speaker attribution, and the full diarized transcript.
2. Run `gbrain call get_links '{"slug": "media/youtube/{slug}"}'`. Confirm back-links exist to brain pages for every person and company mentioned in the video.
2. Run `gbrain get_links media/youtube/{slug}`. Confirm back-links exist to brain pages for every person and company mentioned in the video.
3. Pick a person mentioned in the video. Run `gbrain get <person_slug>`. Confirm their timeline has a new entry referencing the video with specific context.
4. Ingest a tweet. Confirm the brain page includes the thread context, linked article summaries, and entity cross-references -- not just the tweet text.
5. Run `gbrain search "{topic_from_video}"`. Confirm the media page appears in search results (verifies the content is indexed and searchable).
-138
View File
@@ -1,138 +0,0 @@
# Embedding migration — moving a brain to another embedding provider
`gbrain migrate embeddings` re-embeds an entire brain onto a different
embedding provider/model, safely and resumably. It is the forward path off a
sunsetting provider (for example ZeroEntropy's hosted API, which shuts down
2026-09-04 and is the shipped default for brains that never picked a model) —
but it is provider-agnostic: any configured `provider:model` works as a
target.
Also reachable as `gbrain retrieval-upgrade` (the name `doctor` and the
README reference).
## Quick start
```bash
# Preview the work + cost. Changes nothing.
gbrain migrate embeddings --to openai:text-embedding-3-small --dry-run
# Run it (interactive confirm shows chunk count + $ estimate first).
gbrain migrate embeddings --to openai:text-embedding-3-small
# Non-interactive (cron / scripts): --yes is required, else exit 2.
gbrain migrate embeddings --to voyage:voyage-3-large --yes
```
`--dim <N>` overrides the target width; it defaults to the provider recipe's
declared width and is required for recipes that don't declare one (litellm,
llama-server, and other bring-your-own-model providers).
## What it does, in order
1. **Plan.** Counts every chunk not already in the target embedding space —
including chunks on pages with **no recorded embedding signature**
(pages embedded before the v108 provenance stamp). Prices the re-embed
from the pricing table; unknown providers print "estimate unavailable"
instead of a fabricated number.
2. **Consent gate.** Prints the plan; requires an interactive `y` or `--yes`.
Non-TTY without `--yes` refuses with exit 2 (mirrors the `reindex-code`
gate in [spend-controls](../operations/spend-controls.md)). Unlike the pure
cost gates there, `spend.posture=tokenmax` does **not** bypass this one:
posture waives the spend *ceiling*, and this gate also guards a
destructive schema rebuild. Under `tokenmax` the dollar figure is marked
informational and the confirmation is still asked. `--yes` is the single
scripted bypass.
3. **Live probe.** One tiny embed against the TARGET provider before any
mutation — validates the API key, model id, and dimension support in a
single call. A bad key fails here, with nothing changed.
4. **Env-override gate.** Refuses when `GBRAIN_EMBEDDING_MODEL` /
`GBRAIN_EMBEDDING_DIMENSIONS` would silently defeat the switch at
runtime (the same guard `ze-switch` uses). `--ignore-env-override` for
people running deliberate experiments.
5. **Apply.** When the target width differs from the actual column width,
runs the same atomic schema transition `ze-switch` uses, in one
transaction. It rebuilds **all three dim-pinned text-embedding-space
columns** — `content_chunks.embedding`, `query_cache.embedding`, and
`facts.embedding` — at the new width, preserving each column's type
(`vector` vs `halfvec`) and recreating its HNSW index. Missing any of the
three leaves it silently broken: a narrow `query_cache.embedding` makes
every cache write and read fail *by design* (the cache swallows errors so
it can never break search) for a permanent 0% hit rate, and a narrow
`facts.embedding` fails every per-fact embed write. The image/multimodal
columns ARE deliberately untouched — they use separate models whose
dimensions are independent of the text embedding model.
Writes `embedding_model` + `embedding_dimensions` to BOTH config planes
(file plane for the runtime gateway, DB plane for doctor), invalidates
every chunk still in the old space — **including NULL-signature pages**
and purges the semantic query cache so stale cached results can't be
served across the swap.
6. **Re-embed.** The standard embed pipeline (`embed --stale --catch-up`)
with per-source single-flight locks, rate-limit backoff, stderr progress,
and optional DB-contention pacing (`--pace[=mode]`).
## What the rebuild deletes
The dimension change **deletes every stored embedding vector** in the brain —
they are in the old model's space and unusable. They are not recoverable:
going back to the previous provider means paying for a second full re-embed.
`content_chunks` vectors are rebuilt by the re-embed pass, the query cache
refills on the next query, and fact embeddings are rewritten on their next
write (or a `gbrain extract` pass).
## Resume after a kill
The NULL-embedding column is the checkpoint. If the run is killed (or some
pages fail to embed), re-run the **same command**: chunks already embedded on
the target are never re-embedded, the schema/config steps no-op, and the run
continues where it stopped. An in-flight marker (`embedding_migration.state`
in DB config) records the target; it is cleared only when the backlog drains
to zero.
A page whose chunks straddle two stale batches is embedded correctly but not
stamped by the embed loop (which only stamps all-or-nothing per batch), so the
migration runs one reconcile pass after the drain that stamps every
fully-embedded page. Without it a large brain would report "incomplete" and the
re-run would pay again for those pages. `--batch-size N` tunes the batch
(default 2000).
`--no-embed` applies schema + config + invalidation and stops, so you can run
the (potentially long) re-embed later or in the background:
```bash
gbrain migrate embeddings --to openai:text-embedding-3-small --yes --no-embed
gbrain embed --stale --catch-up --include-null-signature --background
```
## During the migration
While the re-embed runs, semantic search returns degraded (lexical-arm-only)
results for not-yet-re-embedded content. Pick a quiet window for large
brains, or use `--pace` to keep the DB responsive.
## Pages without an embedding signature (#3391)
Pages embedded before provenance stamping have `embedding_signature IS NULL`
and are grandfathered by the routine stale sweep (so an upgrade never
surprise-re-embeds a whole corpus). After a provider swap that grandfather
clause would silently leave those pages in the OLD embedding space — mixed
vector spaces in one index, degrading retrieval with nothing in the logs.
- `gbrain migrate embeddings` always includes them.
- Plain `gbrain embed --stale` warns when a model swap leaves NULL-signature
pages behind, and `gbrain embed --stale --include-null-signature` re-embeds
them.
## Reranker
Migrating embeddings does not touch the reranker. If
`search.reranker.model` points at the outgoing provider, the plan prints a
warning; disable it (`gbrain config set search.reranker.enabled false`) or
point it at another provider.
## Self-hosting instead of migrating
If the outgoing model's weights are available (zembed-1's are Apache-2.0),
serving them locally via `llama-server` / `ollama` / a LiteLLM proxy
preserves your existing vectors — no re-embed at all. Point
`embedding_model` at the local recipe and keep the same dimensions. The
migration command is for when you'd rather move to a hosted provider.
+9 -9
View File
@@ -49,23 +49,23 @@ on enrich(entity, trigger):
data["contacts"] = google_contacts(entity.email) # Contact data
# Step 5: Store raw data (auditable, re-processable)
gbrain call put_raw_data \
'{"slug": "<entity_slug>", "data": {"sources": {"crustdata": {"fetched_at": "...", "data": {...}}, ...}}}'
gbrain put_raw_data <entity_slug> \
--data '{"sources": {"crustdata": {"fetched_at": "...", "data": {...}}, ...}}'
# Overwrite on re-enrichment, don't append
# Step 6: Write to brain page
if path == "CREATE":
gbrain put <entity_slug> --content "<compiled_truth_from_all_sources>"
gbrain timeline-add <entity_slug> {date} "Page created via enrichment"
gbrain add_timeline_entry <entity_slug> --entry "Page created via enrichment"
elif path == "UPDATE":
# Append timeline, update compiled truth ONLY if materially new
gbrain timeline-add <entity_slug> {date} "Enriched: {new_signal}"
gbrain add_timeline_entry <entity_slug> --entry "Enriched: {new_signal}"
# Flag contradictions -- don't silently resolve them
# Step 7: Cross-reference the graph
gbrain link <person_slug> <company_slug> # person -> company
gbrain link <company_slug> <person_slug> # company -> person
gbrain link <person_slug> <deal_slug> # person -> deal
gbrain add_link <person_slug> <company_slug> # person -> company
gbrain add_link <company_slug> <person_slug> # company -> person
gbrain add_link <person_slug> <deal_slug> # person -> deal
# Every entity page links to every other entity page that references it
# People page sections (not a LinkedIn profile -- a living portrait):
@@ -94,8 +94,8 @@ on enrich(entity, trigger):
## How to Verify
1. Enrich a Tier 1 person. Run `gbrain get <slug>` and confirm the page has Executive Summary, State, What They Believe, Contact, and Timeline sections populated from multiple sources.
2. Run `gbrain call get_raw_data '{"slug": "<slug>"}'`. Confirm raw API responses are stored with `sources.{provider}.fetched_at` timestamps.
3. Run `gbrain call get_links '{"slug": "<slug>"}'`. Confirm cross-reference links exist to the person's company page, deal pages, and related entities.
2. Run `gbrain get_raw_data <slug>`. Confirm raw API responses are stored with `sources.{provider}.fetched_at` timestamps.
3. Run `gbrain get_links <slug>`. Confirm cross-reference links exist to the person's company page, deal pages, and related entities.
4. Check a page that was enriched AND has a user-written Assessment. Confirm the Assessment section was preserved, not overwritten by API data.
5. Try to re-enrich the same person. Confirm the system checks the `fetched_at` timestamp and skips if less than a week old.
+5 -5
View File
@@ -53,7 +53,7 @@ on upcoming_meeting(meeting):
"last_interaction": page.timeline[0], # most recent
"open_threads": page.open_threads,
"relationship_temperature": page.relationship,
"relevant_deals": gbrain call get_links '{"slug": "<attendee_slug>"}',
"relevant_deals": gbrain get_links <attendee_slug>,
}
else:
briefing[attendee] = "No brain page -- consider enriching"
@@ -67,14 +67,14 @@ on inbox_cleared():
for email in processed_emails:
if email.contained_new_information:
# Update the sender's brain page with new signal
gbrain timeline-add <sender_slug> {date} \
"Email re: {subject}. Key info: {extracted_signal}" \
gbrain add_timeline_entry <sender_slug> \
--entry "Email re: {subject}. Key info: {extracted_signal}" \
--source "email from {sender} re {subject}, {date}"
# Update any mentioned entity pages too
for entity in email.mentioned_entities:
gbrain timeline-add <entity_slug> {date} \
"{what_was_said_about_them}" \
gbrain add_timeline_entry <entity_slug> \
--entry "{what_was_said_about_them}" \
--source "email from {sender}, {date}"
# WORKFLOW 4: Scheduling Nudges
+13 -41
View File
@@ -15,23 +15,17 @@ with the brain repo automatically. You never have to remember to run sync.
## Implementation
### Prerequisite: a reachable direct connection
### Prerequisite: Session Mode Pooler
GBrain is tuned for the Supabase **Transaction pooler** (port 6543): it
auto-disables prepared statements there and routes `engine.transaction()`
(migrations, DDL, sync imports) to a derived **direct** connection
(`db.<ref>.supabase.co:5432`). That direct host is IPv6-only, so on an
IPv4-only host it is unreachable. When that happens gbrain now falls back to
the pooler automatically (one stderr warning, then single-pool mode for the
rest of the process) — but the pooler's ~2-min statement timeout can truncate
very long migrations or bulk imports.
Sync uses `engine.transaction()` on every import. If `DATABASE_URL` points to
Supabase's **Transaction mode** pooler, sync will throw `.begin() is not a
function` and **silently skip most pages**. This is the number one cause of
"sync ran but nothing happened."
Fix: make the direct connection reachable over IPv4. Either set
`GBRAIN_DIRECT_DATABASE_URL` to the **Session pooler** string (port 5432 on the
`pooler.supabase.com` host, IPv4), or enable Supabase's IPv4 add-on.
`GBRAIN_DISABLE_DIRECT_POOL=1` skips the direct pool (and the fallback warning)
entirely. Verify by running `gbrain sync` and checking that the page count in
`gbrain stats` matches the syncable file count in the repo.
Fix: use the **Session mode** pooler string (port 6543, Session mode) or the
direct connection (port 5432, IPv6-only). Verify by running `gbrain sync` and
checking that the page count in `gbrain stats` matches the syncable file count
in the repo.
### The Primitives
@@ -64,9 +58,8 @@ gbrain sync --repo /data/brain && gbrain embed --stale
Name: gbrain-auto-sync
Schedule: */15 * * * *
Prompt: "Run: gbrain sync --repo /data/brain && gbrain embed --stale
Log the result. If sync errors mention an unreachable host or timeout,
the direct connection isn't reachable over IPv4 (set
GBRAIN_DIRECT_DATABASE_URL to the Session pooler, or enable the IPv4 add-on)."
Log the result. If sync fails with .begin() is not a function,
the DATABASE_URL is using Transaction mode pooler."
```
**Hermes:**
@@ -123,27 +116,6 @@ hashes match. If both a cron and `--watch` fire simultaneously, no conflict.
server is down when a push happens, that sync is missed. Pair webhooks
with a cron fallback that catches anything the webhook missed.
4. **A single un-parseable file can't wedge all indexing.** When a file fails
to import (malformed YAML frontmatter, an unquoted colon, etc.), sync holds
the bookmark and tells you exactly which file broke — a *fresh* failure
fails closed so nothing is silently dropped. But a file that fails the same
way `GBRAIN_SYNC_AUTOSKIP_AFTER` consecutive syncs (default 3, set `0` to
disable) is auto-skipped so the rest of the brain keeps indexing past it.
Skipped files don't disappear: `gbrain doctor` keeps warning until you fix
or delete them, and fixing the file clears it on the next sync. A repository
history rewrite still hard-blocks even with `--skip-failed`. Run
`gbrain sync --skip-failed` to acknowledge a known-bad set yourself.
5. **Import checkpoints name the import target, not the caller's CWD.**
Interrupted `gbrain import <dir>` runs may leave
`~/.gbrain/import-checkpoint.json` so the next import can resume. The
checkpoint `dir` is the absolute, resolved import target captured when
import starts. It is not a cleanup instruction and it must not be
re-derived from the process working directory. Checkpoints written by
gbrain include `schema_version: 1`, `owner: "gbrain"`, and
`kind: "import"` so downstream tools can validate the contract before
deciding whether to resume.
## How to Verify
1. **Edit a file and search for the change.** Edit a brain markdown file,
@@ -153,8 +125,8 @@ hashes match. If both a cron and `--watch` fire simultaneously, no conflict.
2. **Compare page count to file count.** Run `gbrain stats` and count the
syncable markdown files in the brain repo. The page count in the database
should match. If they diverge, files are being silently skipped (likely an
unreachable direct connection on IPv4 — see the prerequisite above).
should match. If they diverge, files are being silently skipped (likely
a Transaction mode pooler issue).
3. **Check embedded chunk count.** In `gbrain stats`, the embedded chunk
count should be close to the total chunk count. A large gap means
+7 -7
View File
@@ -32,15 +32,15 @@ on new_meeting_transcript(meeting):
# Step 3: Propagate to ALL entity pages (MANDATORY -- most agents skip this)
for person in meeting.attendees + meeting.mentioned_people:
gbrain timeline-add <person_slug> {date} \
"Met in '{meeting.title}' on {date}. Key points: ..." \
gbrain add_timeline_entry <person_slug> \
--entry "Met in '{meeting.title}' on {date}. Key points: ..." \
--source "Meeting notes '{meeting.title}', {date}"
# Update their State section if new information surfaced
# Update company pages for each person's company if relevant
for company in meeting.mentioned_companies:
gbrain timeline-add <company_slug> {date} \
"Discussed in '{meeting.title}': {what_was_said}" \
gbrain add_timeline_entry <company_slug> \
--entry "Discussed in '{meeting.title}': {what_was_said}" \
--source "Meeting notes '{meeting.title}', {date}"
# Step 4: Extract action items
@@ -49,8 +49,8 @@ on new_meeting_transcript(meeting):
# Step 5: Back-link everything (bidirectional graph)
for entity in all_entities_mentioned:
gbrain link <slug> <entity_slug> # meeting -> entity
gbrain link <entity_slug> <slug> # entity -> meeting
gbrain add_link <slug> <entity_slug> # meeting -> entity
gbrain add_link <entity_slug> <slug> # entity -> meeting
# Step 6: Sync so new pages are immediately searchable
gbrain sync
@@ -73,7 +73,7 @@ on new_meeting_transcript(meeting):
1. After ingesting a meeting, run `gbrain get meetings/{date}-{slug}`. Confirm the page has the agent's analysis above the bar and the full diarized transcript below it.
2. For each attendee, run `gbrain get <attendee_slug>`. Check that their timeline has a new entry referencing the meeting with specific insights (not just "attended meeting").
3. Pick a company mentioned in the meeting. Run `gbrain get <company_slug>`. Confirm a timeline entry exists referencing what was discussed about the company.
4. Run `gbrain call get_links '{"slug": "meetings/{date}-{slug}"}'`. Verify back-links exist to all attendee and entity pages.
4. Run `gbrain get_links meetings/{date}-{slug}`. Verify back-links exist to all attendee and entity pages.
5. Run `gbrain search "{meeting_topic}"`. Confirm the meeting page appears in search results (verifies sync ran).
---
-27
View File
@@ -54,33 +54,6 @@ gbrain jobs supervisor stop
An agent seeing exit=2 can safely treat it as "one is already running";
exit=1 should page a human.
### Lowering scheduling priority (`--nice`)
When the worker pool runs at full concurrency on a machine you also use
interactively, it can drive the load average high enough to starve your
shell. Cutting `--concurrency` throws away throughput. Reach for `--nice`
instead — it lowers the job tree's CPU scheduling priority without touching
width, so the work runs full-speed when the box is idle and yields when it
isn't:
```bash
# Full concurrency, low priority. Propagates to the spawned worker and its
# children (shell jobs, subagents) via OS niceness inheritance.
gbrain jobs supervisor --concurrency 4 --nice 10
# Equivalent for a bare worker, or set it durably in the environment.
GBRAIN_NICE=10 gbrain jobs work --concurrency 4
```
`--nice` takes a POSIX value from `-20` (highest priority) to `19`
(nicest/lowest); positive values need no privilege, negative values need
root. `GBRAIN_NICE` is the env equivalent (the flag wins). Confirm the
effective value with `gbrain jobs stats`, `gbrain jobs supervisor status
--json`, or the `supervisor_niceness` check in `gbrain doctor` — the doctor
check warns if what you asked for isn't what's actually running (e.g. a
negative value denied without privilege, or an OS `RLIMIT_NICE` clamp). This
is distinct from the concurrency / inflight cap and composes with it.
### Which supervisor when?
The supervisor solves in-process crash recovery. Platform-level
+4 -122
View File
@@ -46,13 +46,10 @@ pass:
**What the env allowlist does AND does not do.** Shell jobs run with a minimal
env: `PATH, HOME, USER, LANG, TZ, NODE_ENV`. Your secrets like `OPENAI_API_KEY`
and `DATABASE_URL` are NOT passed to the child. You opt-in additional keys per
job via `env: { ... }` (non-secret values only — see "Secrets" below) or via
`inherit: ["database_url"]` (recommended for secrets — names only in the row,
values resolved at child-spawn from `gbrain config set`). This stops accidental
`$OPENAI_API_KEY` interpolation in a user-authored script. It does **not**
sandbox filesystem reads: a shell script can `cat ~/.env` or any file the
worker process can read. The operator picks a safe `cwd`. That is the trust
boundary.
job via `env: { ... }`. This stops accidental `$OPENAI_API_KEY` interpolation in
a user-authored script. It does **not** sandbox filesystem reads: a shell
script can `cat ~/.env` or any file the worker process can read. The operator
picks a safe `cwd`. That is the trust boundary.
**Audit trail, not forensic insurance.** Every submission writes a JSONL line
to `~/.gbrain/audit/shell-jobs-YYYY-Www.jsonl` (ISO-week rotation; override
@@ -109,116 +106,6 @@ Note: `--follow` blocks the crontab slot until the job finishes. If 14 shell
crons land at the same minute and each takes 30s, they serialize through
crontab's spawning limits. Postgres + persistent worker scales better.
### Calling `gbrain` itself from a shell job — use `inherit:` for DATABASE_URL {#secrets}
A common pattern is submitting shell jobs that run `gbrain` CLI commands:
```bash
gbrain jobs submit shell --params '{
"cmd": "gbrain sync --skip-failed && gbrain embed --stale",
"cwd": "/data/gbrain",
"inherit": ["database_url"]
}'
```
`inherit: ["database_url"]` tells the worker to look up `database_url` from its
own `loadConfig()` (file + env merged) and inject the value into the child's
env as `GBRAIN_DATABASE_URL`. The job row in `minion_jobs.data` stores
`inherit: ["database_url"]`**names only, never values**. The shell-audit
JSONL records the same. Pre-enqueue validation rejects the submission if the
worker can't resolve the requested key, with a paste-ready
`gbrain config set database_url <value>` hint.
**Why not just write the URL into `env:` directly?** Pre-v0.36.5.0 callers
wrote things like:
```jsonc
// ❌ Deprecated as of v0.36.5.0 — REJECTED at submit time.
{
"cmd": "gbrain stats",
"cwd": "/data/gbrain",
"env": { "GBRAIN_DATABASE_URL": "postgresql://..." }
}
```
This planted plaintext secrets in `minion_jobs.data` (DB row) and in the
shell-audit JSONL. Anyone with read access to the brain DB (or a brain dump,
or a shared brain via the mounts feature) saw the URL. v0.36.5.0 doesn't
forbid that pattern — the validator trusts the agent — but **prefer
`inherit:`** for any secret you want kept out of the row. Names land in the
row; values resolve at child-spawn from the worker's config.
**Scope:** v0.36.5.0 `inherit:` is **free-form**. Pass any snake_case
config-key name and the worker resolves the value from `loadConfig()` at
child-spawn time:
- `inherit: ["database_url"]` → child env `GBRAIN_DATABASE_URL`
- `inherit: ["anthropic_api_key"]` → child env `ANTHROPIC_API_KEY`
- `inherit: ["openai_api_key"]` → child env `OPENAI_API_KEY`
- `inherit: ["openrouter_api_key"]` → child env `OPENROUTER_API_KEY`
- `inherit: ["voyage_api_key"]` → child env `VOYAGE_API_KEY`
- `inherit: ["groq_api_key", "zeroentropy_api_key"]` → both injected
- Or any arbitrary config-key your worker has (`my_custom_field`
`MY_CUSTOM_FIELD`)
The env-key name is derived by uppercasing the config-key name. The one
override is `database_url``GBRAIN_DATABASE_URL` (plain `DATABASE_URL` is
ambiguous in most Postgres-app contexts).
Pre-enqueue validation fail-fasts if the worker can't resolve a requested
name. The validator does NOT police which secrets you choose to inherit —
the agent submitting the minion is in the same uid as the worker, so it's
your call.
**Output-side leakage (read this).** The `inherit:` allowlist prevents
secrets from landing in the JOB ROW INPUT fields (`data.cmd`, `data.argv`,
`data.env`). By default it does NOT scrub the OUTPUT fields — if your
script prints the secret to stdout or stderr (`echo "$GBRAIN_DATABASE_URL"`,
`psql "$GBRAIN_DATABASE_URL"` echoing the URL on error), the value lands
plaintext in `result.stdout_tail` / `result.stderr_tail` / `error_text`,
and from there into the brain DB row.
**`redact_secrets: true` opts into output-side scrubbing.** Set it per-job
(or pass `--redact-secrets` on the CLI):
```bash
gbrain jobs submit shell --params '{
"cmd": "gbrain sync --skip-failed",
"cwd": "/data/gbrain",
"inherit": ["database_url"],
"redact_secrets": true
}'
# Or, equivalently:
gbrain jobs submit shell \
--params '{"cmd":"gbrain sync --skip-failed","cwd":"/data/gbrain","inherit":["database_url"]}' \
--redact-secrets
```
When `redact_secrets: true`, the worker resolves each name in `inherit:` to
a value, runs the child, then string-replaces every occurrence of those
values in `stdout_tail` / `stderr_tail` (and in the `error_text` derived from
`stderr_tail` on non-zero exit) with `<REDACTED:name>` before persistence.
Only `inherit:`-resolved values are scrubbed; caller-supplied `env:` values
are not (those are the "I'm fine with this in the row" channel by design).
**Heuristic, not perfect.** The redactor uses literal string-replace. A
script that base64-encodes the secret before printing, or that emits it
one character at a time, will bypass the scrub. Those are adversarial
shapes — the agent + the script are in the same trust domain, so this
layer defends against accidental echo (the common case), not deliberate
exfiltration.
**Three rules for shell-job authors who deal with secrets:**
- **Prefer not to echo secrets at all.** Even with `redact_secrets`, less
output means less risk if the redactor ever has an edge-case miss.
- **Wrap noisy CLI tools to suppress URLs on error.** `psql --quiet`,
`pg_dump --quiet`, or pipe through
`2>&1 | sed 's|postgresql://[^@]*@|postgresql://REDACTED@|g'`.
- **Inspect with `gbrain jobs get <id>` after a failure** to verify what
actually persisted.
### Submitting with `argv` (no shell interpolation)
For programmatic callers assembling commands from JSON, use `argv` instead of
@@ -274,11 +161,6 @@ gbrain jobs list --status waiting --name shell
| `shell: cwd is required and must be an absolute path` | `cwd` must be a string starting with `/`. | Set `cwd` in `--params` to an absolute path. |
| `shell: argv must be an array of strings` | `argv` has a non-string entry or isn't an array. | Pass `argv: ["bin","arg1","arg2"]`. |
| `shell: env values must all be strings` | `env` has a number/bool/object value. | Stringify: `"env":{"COUNT":"3"}` not `"env":{"COUNT":3}`. |
| `shell: inherit must be an array of config-key names` | `inherit` wasn't an array. | Pass `"inherit": ["database_url", ...]`. |
| `shell: inherit entries must be non-empty strings` | An element of `inherit` was empty, non-string, or null. | Use snake_case config-key names like `database_url`, `anthropic_api_key`. |
| `shell: inherit name "<X>" must match [a-z][a-z0-9_]*` | Name failed snake_case regex (uppercase, leading digit/underscore, special char). | Use the config-key name verbatim — `database_url`, not `DATABASE_URL`. |
| `shell: inherit requested "<X>" but worker has no <X> configured` | Worker can't resolve the requested name from `loadConfig()`. | Run `gbrain config set <X> <value>` on the worker host, OR check the config file at `~/.gbrain/config.json`. |
| `shell: redact_secrets must be a boolean if set` | Caller passed a non-boolean for `redact_secrets`. | Pass `true` or `false` (or omit). The CLI `--redact-secrets` flag sets it automatically. |
| `permission_denied: shell jobs cannot be submitted over MCP` | An MCP client tried to submit a shell job. By design CLI-only. | Submit from CLI or via a trusted operation handler (`ctx.remote === false`). |
| `protected job name 'shell' requires CLI or operation-local submitter` | A caller invoked `MinionQueue.add('shell', ...)` without the `trusted` opt-in. | Pass `{ allowProtectedSubmit: true }` as the 4th arg. CLI and `submit_job` do this automatically. |
| `aborted: timeout` / `aborted: cancel` / `aborted: shutdown` / `aborted: lock-lost` | The worker's abort signal fired mid-execution. Child got SIGTERM, 5s grace, then SIGKILL. | Expected: timeout / user cancel / deploy restart / stall. Inspect `gbrain jobs get` to see which. |
-97
View File
@@ -1,97 +0,0 @@
# Multi-language full-text search
GBrain's keyword search arm uses Postgres full-text search (tsvector/tsquery).
The tokenizer language is configurable via the `GBRAIN_FTS_LANGUAGE`
environment variable. Default: `english`.
## How it works
Postgres text-search configurations control stemming and stop-word removal.
`GBRAIN_FTS_LANGUAGE` is read by `src/core/fts-language.ts` and applied on
both sides of the search:
- **Query side**`websearch_to_tsquery('<lang>', $query)` in both engines
(Postgres and PGLite).
- **Write side** — the `update_page_search_vector` and
`update_chunk_search_vector` trigger functions that populate
`pages.search_vector` and `content_chunks.search_vector`.
The value is validated against `/^[a-z][a-z0-9_]*$/` before it is ever
interpolated into SQL (tsvector functions don't accept parameterized config
names). Invalid values fall back to `english` with a warning.
## Built-in languages
Set the env var to any configuration your Postgres instance ships:
```bash
export GBRAIN_FTS_LANGUAGE=portuguese
export GBRAIN_FTS_LANGUAGE=spanish
export GBRAIN_FTS_LANGUAGE=german
```
List what's available:
```sql
SELECT cfgname FROM pg_ts_config;
```
PGLite (the embedded default engine) ships the same built-in snowball
configurations as stock Postgres.
## First install vs. changing language later
On first install (or upgrade), the `configurable_fts_language` schema
migration reads `GBRAIN_FTS_LANGUAGE` and stamps the trigger functions with
that language. After the migration has run, changing the env var alone does
NOT retokenize existing rows — the migration shows as applied and is skipped.
Use the explicit command:
```bash
export GBRAIN_FTS_LANGUAGE=portuguese
gbrain reindex-search-vector --dry-run # preview: language + row counts
gbrain reindex-search-vector --yes # recreate triggers + backfill
```
The command recreates both trigger functions under the new language and
backfills every existing `pages` and `content_chunks` row in batches,
streaming progress to stderr. It is idempotent: re-running with the same
language produces identical vectors. `--json` prints a machine-readable
result envelope but still requires `--yes` (or an interactive confirm).
## Recipe: accent-insensitive Portuguese (`pt_br`)
Brazilian Portuguese content often mixes accented and unaccented spellings
("São Paulo" vs "Sao Paulo"). Build a custom config that folds accents via
the `unaccent` extension, then stems with the portuguese snowball dictionary:
```sql
CREATE EXTENSION IF NOT EXISTS unaccent;
CREATE TEXT SEARCH CONFIGURATION pt_br (COPY = portuguese);
ALTER TEXT SEARCH CONFIGURATION pt_br
ALTER MAPPING FOR hword, hword_part, word
WITH unaccent, portuguese_stem;
```
Then point GBrain at it:
```bash
export GBRAIN_FTS_LANGUAGE=pt_br
gbrain reindex-search-vector --yes
```
Note: custom configurations require a real Postgres instance (e.g. the
Supabase engine). The config must exist BEFORE the migration or the reindex
command runs, or Postgres will reject the trigger recreation with
`text search configuration "pt_br" does not exist`.
## Caveats
- One language per brain: the setting is global to the database, not
per-source. Mixed-language brains should pick the dominant language (the
vector-search arm is language-agnostic and covers the rest).
- Keep `GBRAIN_FTS_LANGUAGE` set consistently in every environment that
writes to the brain (CLI shells, MCP server, cron jobs) — a writer without
the env var tokenizes new rows in `english` until the next reindex.
+4 -100
View File
@@ -91,7 +91,7 @@ first):
6. The seeded `default` source.
So inside `~/.gstack/plans/` on a brain that pinned `gstack` to
`~/.gstack` via `.gbrain-source`, `gbrain put` implicitly writes to
`~/.gstack` via `.gbrain-source`, `gbrain put-page` implicitly writes to
the `gstack` source. Outside any registered directory with no env/dotfile
set, it writes to the default.
@@ -114,11 +114,8 @@ Flip later with `gbrain sources federate <id>` / `unfederate <id>`.
Full subcommand reference:
```
gbrain sources add <id> --path <p> [--name <n>] [--federated|--no-federated] [--force]
gbrain sources add <id> --path <p> [--name <n>] [--federated|--no-federated]
Register a source. id: [a-z0-9](?:[a-z0-9-]{0,30}[a-z0-9])?
--path must be a git repo (or a subdirectory of one) — see
"The git requirement for --path sources" below. --force
skips that check to register before git-init exists.
gbrain sources list [--json] List all sources with page counts + federation state.
gbrain sources remove <id> [--yes] [--dry-run] [--keep-storage]
Cascade-delete a source (pages, chunks, timeline).
@@ -131,47 +128,6 @@ gbrain sources federate <id>
gbrain sources unfederate <id>
```
## The git requirement for --path sources
Every `--path` source must be a git repository (or live inside one — a
subdirectory of a git repo works too) with at least one committed, tracked
file under that path. `gbrain sources add` validates this at registration
time and refuses a directory that doesn't qualify — no `.git` at all, a
`git init` with no commit yet, or a commit made before `git add` — with an
actionable error instead of silently registering a source that will fail
(or worse, "succeed" while importing nothing) on its first `gbrain sync`.
Fix it with:
```bash
git -C <path> init
git -C <path> add -A
git -C <path> commit -m "initial import"
gbrain sources add <id> --path <path>
```
Two details that are easy to miss:
- **Files must actually be committed, not just present.** The sync walker
reads files through git objects, so `git init` alone — even followed by an
empty commit (`git commit --allow-empty`) — isn't enough. Registration
checks for real tracked content (`git ls-tree HEAD` scoped to the path),
not just a resolvable `HEAD`, so this footgun is caught immediately
instead of surfacing later as a sync that imports nothing.
- **`--force` registers the source anyway**, skipping the check. Use this if
you're registering a path before an automated pipeline gets around to
`git init`-ing it. GBrain never auto-`git init`s a `--path` source for
you — it's your directory, not a gbrain-managed clone (same consent
boundary as sync-time self-heal, which also never mutates a `--path`
source without an explicit ask).
**If sync ever reports a problem with the sync anchor** (`last_commit`) —
after a force-push, a history rewrite, or a from-scratch `git init` on a
directory that was synced before — you do not need to reset anything by
hand. `gbrain sync` detects an unreachable or non-ancestor anchor
automatically and recovers: either a full reimport (anchor object missing)
or a direct tree-to-tree diff against the orphaned bookmark (anchor present
but rewritten), advancing the anchor to the new HEAD when it completes.
## Citation format for agents
When agents receive multi-source results they MUST cite pages in
@@ -188,10 +144,10 @@ citations keep working.
```bash
# Pass --source explicitly
gbrain put topics/ai ... --source wiki
gbrain put-page topics/ai ... --source wiki
# Or rely on the dotfile / env / CWD match
cd ~/.gstack && gbrain put plans/multi-repo ...
cd ~/.gstack && gbrain put-page plans/multi-repo ...
# → source auto-resolves to gstack
```
@@ -199,58 +155,6 @@ Reads span federated sources by default. Writes require a resolved
source (explicit, inferred, or default). The resolver never picks a
source silently when ambiguous — it errors with a clear fix.
## Durability: keep a brain repo in sync (auto-harden)
A long-lived agent that writes to a knowledge-wiki git repo needs three
things to never lose work: pull before it edits, push every write, and not
go stale while it sits idle. `gbrain sources harden` installs all of that,
idempotently. The moment you add a brain repo with a token, it runs
automatically:
```bash
# Clone + register a GitHub repo, then auto-harden it for durability.
# Use a fine-grained PAT scoped to just this repo.
gbrain sources add wiki --url https://github.com/you/brain-wiki.git --pat-file ~/.secrets/wiki-pat
# → clones, then installs: local auto-push hook, scripts/brain-commit-push.sh,
# always-on durability rules in AGENTS.md/RESOLVER.md, a 30-min pull cron,
# and a repo-scoped credential. Verifies push works before declaring done.
# Run the same audit on an existing source any time (idempotent):
gbrain sources harden wiki --pat-file ~/.secrets/wiki-pat
# Pull on demand (the cron calls the --path form, which never opens the DB):
gbrain sources pull wiki
# Remove the durability scaffolding (also runs automatically on `sources remove`):
gbrain sources unharden wiki
```
What hardening guarantees:
- **Pull-first, conflict-safe.** Every pull is a divergence-safe rebase. A
dirty working tree is skipped (your in-progress edits are never touched); a
rebase conflict is aborted cleanly and flagged for attention, never left
half-applied.
- **Push is never deferred.** `scripts/brain-commit-push.sh "<msg>" <path>`
commits and pushes atomically and refuses to report success without a
confirmed push. The post-commit hook is a best-effort background fallback;
the helper is the guarantee.
- **No silent staleness.** A 30-minute background pull keeps an idle session
current. It runs DB-free, so it never contends with a live brain for the
PGLite single-writer lock.
Flags: `--no-cron` skips the scheduled pull, `--no-verify` skips the push
probe, `--dry-run` reports what would change, `--json` emits a machine
report, `--all` hardens every source with a remote (same-account only).
`--no-harden` on `sources add` opts out of auto-harden.
Security: the push automation is installed locally per machine (never
committed into the repo), the token is wired per-repo (an existing
credential helper is reused when present), and it never appears in the repo,
the remote URL, logs, or the JSON report. For a self-hosted git server
reachable only over a filesystem path, set `GBRAIN_GIT_ALLOW_FILE_TRANSPORT=1`
(default is HTTPS-only).
## Upgrading an existing brain
`gbrain upgrade` runs the v16 + v17 migrations automatically. Your
+8 -8
View File
@@ -20,8 +20,8 @@ on every_inbound_message(message):
for entity in entities:
existing = gbrain search "{entity.name}"
if existing:
gbrain timeline-add <entity_slug> {date} \
"{what_was_said}" \
gbrain add_timeline_entry <entity_slug> \
--entry "{what_was_said}" \
--source "User, direct message, {timestamp}"
# else: flag for enrichment if important enough
@@ -64,13 +64,13 @@ on nightly_schedule("02:00"):
# The brain COMPOUNDS overnight.
# 5a: Entity sweep -- find unlinked mentions
pages = gbrain list
pages = gbrain list_pages
for page in pages:
mentions = extract_entity_mentions(page.content)
existing_links = gbrain call get_links '{"slug": "<page.slug>"}'
existing_links = gbrain get_links <page.slug>
for mention in mentions:
if mention not in existing_links:
gbrain link <page.slug> <mention_slug> # fix broken graph
gbrain add_link <page.slug> <mention_slug> # fix broken graph
# 5b: Citation audit -- find facts without sources
for page in pages:
@@ -80,7 +80,7 @@ on nightly_schedule("02:00"):
# 5c: Memory consolidation -- update compiled truth from timeline
for page in stale_pages(older_than="7d"):
timeline = gbrain timeline <page.slug>
timeline = gbrain get_timeline <page.slug>
if timeline.has_new_entries_since_last_consolidation:
# Re-synthesize compiled truth from accumulated timeline
updated_truth = consolidate(page.compiled_truth, timeline.new_entries)
@@ -110,11 +110,11 @@ on nightly_schedule("02:00"):
## How to Verify
1. Send a message mentioning a person with a brain page. Confirm the agent detects the entity and adds a timeline entry to their page (`gbrain timeline <slug>`).
1. Send a message mentioning a person with a brain page. Confirm the agent detects the entity and adds a timeline entry to their page (`gbrain get_timeline <slug>`).
2. Ask the agent about someone in the brain. Confirm it runs `gbrain search` or `gbrain get` BEFORE reaching for external APIs (check the tool call order).
3. Write a new page with `gbrain put`, then immediately run `gbrain search` for it. Confirm it appears in results (verifies sync ran).
4. Run `gbrain doctor`. Confirm it returns a health report with database status, page count, and any flagged issues.
5. After a dream cycle runs, check a page that had unlinked entity mentions. Confirm new links were added (`gbrain call get_links '{"slug": "<slug>"}'`).
5. After a dream cycle runs, check a page that had unlinked entity mentions. Confirm new links were added (`gbrain get_links <slug>`).
---
*Part of the [GBrain Skillpack](../GBRAIN_SKILLPACK.md).*
+3 -3
View File
@@ -47,8 +47,8 @@ on user_message(message):
# Step 3: Cross-link to everything that shaped the thinking
for entity in idea.influences:
gbrain link originals/{slug} <entity_slug>
gbrain link <entity_slug> originals/{slug}
gbrain add_link originals/{slug} <entity_slug>
gbrain add_link <entity_slug> originals/{slug}
# Step 4: Sync
gbrain sync
@@ -79,7 +79,7 @@ on user_message(message):
1. Generate an original idea in conversation (e.g., "I call this the 'ambition debt' problem -- every year you delay going big, the compound interest works against you"). Confirm a new page appears at `brain/originals/ambition-debt` with `gbrain get originals/ambition-debt`.
2. Check that the page uses the user's exact phrasing for the title and slug -- not a sanitized version.
3. Run `gbrain call get_links '{"slug": "originals/ambition-debt"}'`. Confirm cross-links exist to related people, meetings, or other originals.
3. Run `gbrain get_links originals/ambition-debt`. Confirm cross-links exist to related people, meetings, or other originals.
4. Express a take on someone else's idea (e.g., "I think Thiel's contrarian question is wrong because..."). Confirm it goes to `originals/` (synthesis is original), not `concepts/`.
5. Run `gbrain search "ambition debt"`. Confirm the originals page appears in search results and is discoverable.
+1 -1
View File
@@ -87,7 +87,7 @@ expect it.
| `version` | string | yes | Your plugin's semver. Informational. |
| `plugin_version` | string | yes | Contract lock. Must equal `"gbrain-plugin-v1"` for v0.15. |
| `subagents` | string | no | Subdir name (default `subagents`). Escape-attempts are rejected. |
| `description` | string | no | Shown in a future plugin-listing command. |
| `description` | string | no | Shown in future `gbrain plugin list`. |
## Subagent definition files
-79
View File
@@ -1,79 +0,0 @@
# Push-based context (#2095, v0.42.43.0)
Retrieval used to be pull-only: the agent had to *know to ask* before the brain
contributed anything. Push-based context inverts that — the brain volunteers
relevant pages from the recent conversation, confidence-gated so push noise
never becomes worse than pull silence.
Three channels share one zero-LLM core (`src/core/context/volunteer.ts`):
| Channel | Surface | When to use |
|---|---|---|
| `reflex` | automatic, inside the context engine | default-on for plugin hosts; nothing to call |
| `op` | `gbrain volunteer-context` / MCP `volunteer_context` | agents without the plugin; one call per turn |
| `watch` | `gbrain watch` | stream a transcript in, volunteered pages stream out |
## How it decides
1. **Extract** entities across the last N turns (capitalized runs, `@handles`),
merged with recency / frequency / user-role salience. Assistant-introduced
entities and "what did she invest in?" follow-ups whose antecedent was named
in the window now resolve.
2. **Resolve** through the alias table, exact titles, and slug suffixes — each
arm carries an honest confidence: alias 0.9, exact title 0.8, slug-suffix 0.6,
+0.05 when mentioned in ≥2 turns or the newest turn.
3. **Gate** at `min_confidence` (default 0.7 — slug-suffix matches need an
explicit lower gate), suppress pages already surfaced (slug-presence only),
cap at 3 pages (hard cap 5).
## CLI
```bash
# one-shot: pipe recent turns (oldest → newest)
printf 'user: ask alice-example about the deal\nassistant: noted\nuser: what did she say?\n' \
| gbrain volunteer-context
# streaming: volunteered pages print as the transcript flows
some-transcript-feed | gbrain watch --json
# the feedback loop: how often were volunteered pages actually opened?
gbrain volunteer-context --stats
```
Stats are **approximate** by design: "used" means `pages.last_retrieved_at >
volunteered_at` — the 5-minute last-retrieved throttle causes false negatives
and unrelated reads of the same page cause false positives. Use the per-arm
precision to tune `min_confidence`, not as an exact metric.
**PGLite + `gbrain watch`:** PGLite is single-connection, and watch holds its
connection for the whole session — a concurrent `gbrain serve` or any write
path blocks until watch exits. On a PGLite brain, run watch in bursts (piped
input exits at EOF) or use the ambient reflex channel instead, which routes
through a running serve's resolve socket rather than taking the lock. Routing
watch through that same socket is a filed follow-up (TODOS.md). Postgres
brains are unaffected.
## Config
| Key | Default | What it does |
|---|---|---|
| `retrieval_reflex_window_turns` | 4 | turns the ambient reflex extracts from; 1 = legacy current-turn-only (file/env plane: `GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS`) |
| `retrieval_reflex` | true | the ambient channel's master switch |
| `retrieval_reflex_max_pointers` | 3 | pointer cap per turn |
Per-call knobs: `max_pages` + `min_confidence` on both the op and `gbrain watch`
(`--max-pages` / `--min-confidence`, plus `--window-turns` / `--source` on watch);
on the op only: `prior_context` (text whose already-surfaced slugs are suppressed),
`session_id` / `turn` attribution params (watch stamps its own per-session id and
turn numbers in the feedback log), and `days` to size the `--stats` window.
## Storage + privacy
Volunteered pages log to `context_volunteer_events` (migration v117): slug,
arm, confidence, channel, optional session/turn — the rationale is a
deterministic template string, never raw conversation text. Event writes are
best-effort (fire-and-forget, drained at CLI exit) — the log is a tuning signal,
not an audit trail. Rows are pruned after 90 days by the dream cycle's purge
phase. Synopses always strip the takes/facts fences — the same strip `get_page`
applies to untrusted callers, applied unconditionally here so private fence rows
never reach a prompt regardless of caller trust.
-33
View File
@@ -16,39 +16,6 @@ gbrain doctor --json | jq '.checks[] | select(.name == "queue_health")'
- **waiting-depth**: any per-name queue deeper than 10 (override via
`GBRAIN_QUEUE_WAITING_THRESHOLD`). Signals a missing `maxWaiting`.
## The worker is alive but wedged (dead pool)
The nastiest stall: the worker process is *running* (passes `ps` / `kill -0` /
container health), but its DB connection died (common behind a transaction
pooler) and never came back, so it claims no jobs and finishes nothing. Jobs
pile up with **0 active**. Liveness checks all pass; nothing crashes.
As of v0.42.22.0 this self-heals — you usually won't have to do anything:
- **The worker exits on its own dead pool.** Under a supervisor, the worker's
DB-liveness probe runs and self-exits (`db_dead`) after ~3 minutes; the
supervisor respawns it with a fresh pool.
- **The supervisor restarts a worker that stops making progress.** If a queue
has claimable work, **0 live-lock active jobs**, and no completions for 15
minutes while the child is alive, the supervisor restarts it (covers stuck
handlers too, not just dead pools). Tune with `--wedge-restart-minutes` /
`--wedge-restart-checks` on `gbrain jobs supervisor` (0 disables).
The signal is loud now — check either:
```bash
gbrain jobs stats --queue default # prints a WEDGED QUEUE line
gbrain doctor --json | jq '.checks[] | select(.name == "wedged_queue")'
```
`wedged_queue` is a per-queue health **error** (0 active_healthy + waiting > 0 +
stale completions). Manual fix if you ever need it:
```bash
gbrain jobs supervisor stop && gbrain jobs supervisor start # fresh pool
gbrain jobs retry <id> # dead-lettered jobs
```
## Triage commands
```bash
-79
View File
@@ -34,85 +34,6 @@ docs/guides/rls-and-you.md for the GBRAIN:RLS_EXEMPT comment escape hatch.
99% of the time, you want the fix. Run the SQL. Re-run `gbrain doctor`. Done.
## v0.26.7 — auto-RLS event trigger and one-time backfill
Starting in v0.26.7 (migration v35), gbrain ships two changes that close the
gap where a table could exist in your `public` schema without RLS for any
amount of time at all.
**1. The event trigger.** A Postgres DDL event trigger named
`auto_rls_on_create_table` runs `ALTER TABLE … ENABLE ROW LEVEL SECURITY`
on every newly created `public.*` table. It covers `CREATE TABLE`,
`CREATE TABLE AS … SELECT`, and `SELECT … INTO` — every syntax Postgres
reports as a table-creation command. Tables created by gbrain itself, by
your other apps sharing the same Supabase project (Baku, Hermes, anything),
or by a human running raw SQL all get RLS enabled the moment they exist.
Non-`public` schemas (`auth`, `storage`, `realtime`, etc.) are explicitly
ignored — Supabase manages those, and we should not touch them.
**2. The one-time backfill.** When you upgrade to v0.26.7, the migration
walks every existing `public.*` base table whose RLS is off and whose comment
doesn't carry the `GBRAIN:RLS_EXEMPT` exemption (see below) and enables RLS
on each. After the upgrade, `gbrain doctor`'s `rls` check should be a no-op
on every brain.
### Breaking change: read this before upgrading
If you have public tables that are intentionally RLS-off and you want them
to stay that way, you MUST add the `GBRAIN:RLS_EXEMPT` comment **before**
running `gbrain upgrade` to v0.26.7. The backfill flips RLS on for any public
table that doesn't carry the exact comment contract documented below. There
is no `--dry-run` flag on the migration.
The minimum cost of getting this wrong is one round-trip: the operator runs
the SQL to enable RLS on a table that should have been exempt, then
`ALTER TABLE … DISABLE ROW LEVEL SECURITY` and adds the exempt comment to
prevent a re-flip on a later doctor run. No data is lost.
### Cross-app implications
If a non-gbrain app (Baku, Hermes, a script you wrote, anything) creates
tables in the same Supabase project, the trigger will enable RLS on those
tables too. Two ways to handle that:
1. **The app's connection role has BYPASSRLS** (e.g. it's also using the
`postgres` role). Newly created tables get RLS on but the app reads/writes
freely because BYPASSRLS bypasses policies entirely.
2. **The app's role does NOT have BYPASSRLS.** Then the app needs to add a
`CREATE POLICY` immediately after creating the table, granting itself
the read/write access it needs. The trigger does NOT add policies — it
only enables RLS, leaving the deny-by-default posture in place until the
app's policy lands.
If neither condition holds, the app will fail to read its own freshly-created
tables. The fix is at the app side, not gbrain's: either grant BYPASSRLS or
ship a policy.
### What if the trigger gets dropped?
`gbrain doctor` includes a new `rls_event_trigger` check that verifies the
trigger is installed and enabled. If you drop it manually for any reason
(debugging, migration testing, anything), doctor warns and gives you the
recovery command:
```
gbrain apply-migrations --force-retry 35
```
Re-running migration v35 is idempotent — it `DROP EVENT TRIGGER IF EXISTS`
and recreates cleanly.
### Why no FORCE ROW LEVEL SECURITY?
Postgres has two RLS dials. `ENABLE` blocks anon/authenticated; `FORCE` also
blocks the table OWNER unless they hold BYPASSRLS. We use `ENABLE` only,
matching the posture in `src/schema.sql`, migrations v24, and v29. `FORCE`
would lock non-BYPASSRLS apps out of their own freshly-created tables (the
trigger function inherits the caller's role, not the gbrain role) — which
defeats the cross-app coexistence story above. If you want defense-in-depth
`FORCE` on a specific gbrain-owned table, add it explicitly in your own
migration; gbrain's auto-RLS does not opt you in by default.
## The 1% case: deliberate exemption
Sometimes a public table is supposed to be readable by the anon key. An

Some files were not shown because too many files have changed in this diff Show More