diff --git a/CHANGELOG.md b/CHANGELOG.md index a40e76340..3aaaecf18 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,35 @@ All notable changes to GBrain will be documented in this file. +## [0.42.72.0] - 2026-08-01 + +**Per-person write isolation inside a shared source, and a guide for putting gbrain behind a multi-user agent harness.** + +Until now, `--source` was the only write boundary: a client could write anywhere inside the source it was scoped to, and keeping each person in their own folder was a convention the agent had to honor by itself. Registering a client with `--bound-slug-prefixes` now makes that boundary real. Writes outside the bound prefixes are refused by the server, on every op that can name a page. + +**Adding a binding to an existing client narrows it on purpose.** Ops that write by something other than a page slug can't be confined to a prefix, so a bound client is refused them outright rather than left with an unfenced path: `extract_entities`, `extract_facts`, `forget_fact`, `ontology_propose`, `sources_add`/`sources_remove`, and `POST /ingest`. `put_page`'s automatic fact extraction is skipped for the same reason — it writes to entity pages the caller never named. Reads are unaffected, and unbound clients behave exactly as before. The gate keys on "anything that is not a plain read", so an op added in a future release is refused to bound clients until it is explicitly fenced. + +Both prefix spellings work: the `wiki/agents/alice/*` glob that `submit_agent` bindings already use, and the plainer `emp-alice/` form. Change a binding in place with `gbrain auth rescope-client --bound-slug-prefixes ` — existing tokens pick it up on their next request, so no secret rotation is needed when someone joins or leaves a team. + +**New guide: [gbrain as the company brain for a qm deployment](docs/integrations/qm-harness.md).** qm is a multiplayer agent harness where each employee and each channel gets an isolated agent scope. The guide covers the whole path — one central `gbrain serve --http`, the thin-client binary baked into the sandbox image, one OAuth client per scope, and a roster-driven provisioning script that converges the brain to a list of people and channels. It also states plainly what the model does *not* give you: within a shared source, reads stay source-granular, so prefix isolation is a write boundary, not a privacy boundary. +gbrain upgrade # or: bun install -g gbrain@0.42.72.0 +gbrain apply-migrations --yes # required: the fence refuses writes it cannot evaluate +``` + +To fence an existing client to a folder: + +```bash +gbrain auth rescope-client --bound-slug-prefixes partners/alice-example/ +gbrain auth rescope-client --bound-slug-prefixes none # undo +``` + +Verify it took, from a client holding that credential — the first write should succeed and the second should be refused: + +```bash +gbrain put partners/alice-example/notes/test --content "mine" +gbrain put partners/bob-example/notes/test --content "not mine" +``` + ## [0.42.71.0] - 2026-08-01 **GBrain now publishes real releases. Every version bump from here on lands on the [Releases page](https://github.com/garrytan/gbrain/releases) with organized notes and downloadable binaries — and binary self-update finally works.** @@ -46,10 +75,6 @@ Contributed by @time-attack (#3573, closing #3521). **Windows and self-hosters.** Markdown files keep LF endings so frontmatter parsers stop mis-reading on Windows checkouts; the archive-crawler path gate no longer denies every real Windows path (and no longer fail-opens on NTFS case-insensitivity); a chat-synopsis tier that was hardcoded to one provider now follows your configured models; vector search asks the index for as many candidates as it was told to consider. **Quieter, more honest infrastructure.** `serve --http` no longer leaves an orphan holding the database lock after Ctrl-C; a minion child that fails to launch settles immediately instead of hanging its slot; doctor gains checks for content-hash duplicates, undeclared database-only pages, stale heartbeats, and a tamper-evident manifest for the skills directory; federated reads respect per-source isolation settings in two more paths; and the security docs were rewritten to describe fixes without cataloguing attack surface. - -### To take advantage of v0.42.70.0 - -```bash gbrain upgrade gbrain extract --stale # re-extracts links under the fixed resolver gbrain doctor # includes the new silent-failure checks diff --git a/VERSION b/VERSION index dc851c36a..0cee48d46 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.42.71.0 \ No newline at end of file +0.42.72.0 \ No newline at end of file diff --git a/docs/architecture/KEY_FILES.md b/docs/architecture/KEY_FILES.md index 4afd268cb..0afb03990 100644 --- a/docs/architecture/KEY_FILES.md +++ b/docs/architecture/KEY_FILES.md @@ -12,7 +12,7 @@ per-release `**vX.Y.Z:**` narration — CI enforces this - `src/commands/serve-http.ts` confidential revoke extension — a pre-router `/revoke` handler validates the RFC 7009 body, verifies hash-only secrets for both `client_secret_post` and `client_secret_basic`, rejects mixed authentication, preserves the SDK path for public clients, and separates opaque client-auth failures from retryable/backend failures. OAuth metadata advertises both confidential methods. Pinned by `test/e2e/serve-http-oauth.test.ts`. -- `src/core/operations.ts` — Contract-first operation definitions (the foundation). Exports upload validators `validateUploadPath`, `validatePageSlug`, `validateFilename`, plus `matchesSlugAllowList(slug, prefixes)` (glob matcher: `/*` matches recursive children; bare `` matches exact only). `OperationContext.remote` is a REQUIRED field flagging untrusted callers; `OperationContext.allowedSlugPrefixes` is the trusted-workspace allow-list set by the dream cycle; `OperationContext.auth?: AuthInfo` is threaded through HTTP dispatch for scope enforcement in `serve-http.ts` before the op runs. OAuth `whoami` exposes the authenticated `AuthInfo.sourceId` and `AuthInfo.allowedSources` grants as `source_id` and `federated_read`; absent grants serialize fail-closed as `null` and `[]`, while local, legacy, and stdio response shapes stay unchanged. `enforceSubagentSlugFence(ctx, slug, opName)` is the shared fail-closed subagent write fence: when `viaSubagent` and `allowedSlugPrefixes` is set, the slug must match the allow-list; else the legacy `wiki/agents//...` namespace check applies. Both `put_page` and `add_timeline_entry` (subagent-allowlisted) route through it. Auto-link skipped only when `remote=true && !trustedWorkspace`. Every `Operation` carries `scope?: 'read' | 'write' | 'admin'` + `localOnly?: boolean`; `sync_brain`, `file_upload`, `file_list`, `file_url` are `admin + localOnly` (rejected over HTTP). Four trust-boundary call sites (`put_page` allowlist, file_upload trust-narrowing, submit_job protected-name guard, auto-link skip) use FAIL-CLOSED semantics: `ctx.remote === false` for trusted-only sites, `ctx.remote !== false` for "untrust unless explicit-false" — anything not strictly `false` is treated as remote (closes the HTTP MCP shell-job RCE where a read+write OAuth token could submit `shell` jobs). `sourceScopeOpts(ctx)` encodes the source-scoped read precedence ladder — federated array (`ctx.auth.allowedSources`) wins over scalar (`ctx.sourceId`/`ctx.auth.sourceId`) over nothing; every read-side op handler routes through it so a source-bound OAuth client can't see neighboring sources via `search`/`query`/`list_pages`/`get_page`/`find_experts`/query's image path, plus the by-slug reads `get_tags`/`get_links`/`get_backlinks`/`get_timeline` (and `get_page`'s tag fetch, which resolves against the concrete page's own `source_id`). `linkReadScopeOpts(ctx)` is the link-read sibling for `get_links`/`get_backlinks`: a link row references three pages (from/to/origin), and the engine's federated (`sourceIds[]`) branch scopes ALL THREE while its scalar (`sourceId`) branch scopes only the near endpoint (by design — trusted internal callers like `reconcileLinks` and back-link validators call the engine with a scalar scope and need the cross-source view). For an UNTRUSTED remote caller (`ctx.remote !== false`) carrying only a scalar scope it promotes that scope to a single-element `sourceIds:[id]`, routing them through the all-endpoint branch so a far/origin slug from outside the grant isn't disclosed; a federated array passes through unchanged and trusted local CLI (`ctx.remote === false`) keeps the scalar cross-source view. `thinkSourceScopeOpts(ctx)` maps the same precedence ladder onto `runThink`'s public options (`allowedSources`/`sourceId`) so the `think` op's gather and trajectory stages inherit the caller's source grant. `put_page`'s inline disk write-through is the shared `writePageThrough` helper (`src/core/write-through.ts`), ATOMIC via temp-sibling + rename so a crash or concurrent `gbrain sync` can't read a half-written `.md`; same helper backs `gbrain brainstorm/lsd --save`. Link provenance surface (#1941): `add_link` (`gbrain link`/`link-add`) + `remove_link` (`gbrain unlink`/`link-rm`) expose `link_source`/`link_type`; `add_link` rejects the reconciliation-managed built-ins via `MANAGED_LINK_SOURCES` (`markdown`/`frontmatter`/`mentions`/`wikilink-resolved`) and defaults omitted provenance to `'manual'` (the engine's own default stays `'markdown'` for internal callers); `list_link_sources` (`gbrain link-sources`, read) lists provenances via `sourceScopeOpts`. CLI aliases register through `cliHints.aliases` (collision-guarded in `src/cli.ts`). +- `src/core/operations.ts` — Contract-first operation definitions (the foundation). Exports upload validators `validateUploadPath`, `validatePageSlug`, `validateFilename`, plus `matchesSlugAllowList(slug, prefixes)` (glob matcher: `/*` matches recursive children; bare `` matches exact only). `OperationContext.remote` is a REQUIRED field flagging untrusted callers; `OperationContext.allowedSlugPrefixes` is the trusted-workspace allow-list set by the dream cycle; `OperationContext.auth?: AuthInfo` is threaded through HTTP dispatch for scope enforcement in `serve-http.ts` before the op runs. OAuth `whoami` exposes the authenticated `AuthInfo.sourceId` and `AuthInfo.allowedSources` grants as `source_id` and `federated_read`; absent grants serialize fail-closed as `null` and `[]`, while local, legacy, and stdio response shapes stay unchanged. `enforceSubagentSlugFence(ctx, slug, opName)` is the shared fail-closed subagent write fence: when `viaSubagent` and `allowedSlugPrefixes` is set, the slug must match the allow-list; else the legacy `wiki/agents//...` namespace check applies. Both `put_page` and `add_timeline_entry` (subagent-allowlisted) route through it. Auto-link skipped only when `remote=true && !trustedWorkspace`. `enforceClientSlugFence(ctx, slug, opName)` is the OAuth-client write fence: when `ctx.auth.boundSlugPrefixes` is present (threaded from `oauth_clients.bound_slug_prefixes` at token-verification time), every direct slug-mutating write op — `put_page`, `delete_page`, `restore_page`, `add_tag`, `remove_tag`, `add_link`/`remove_link` (`from` endpoint only; linking TO a readable page is a reference), `add_timeline_entry`, `revert_version`, `put_raw_data` — rejects out-of-prefix slugs with `permission_denied`, BEFORE each op's dry-run short-circuit. Plain-startsWith semantics matching `submit_agent`'s check for the same column (NOT the glob grammar of the subagent allow-list); empty-array binding is deny-all (fail-closed); no auth / no binding = no fence. The match rule itself lives in the exported `slugUnderBoundPrefixes(prefixes, slug)` so non-op write surfaces reuse it verbatim. It is BOUNDARY-AWARE (a prefix matches whole segments, so `emp-alice` does not admit `emp-alice-2/…`), lowercases both sides (stored slugs are lowercased by `validateSlug`, so comparing the caller's raw string let a mixed-case slug commit and only then trip the resolved-slug re-check), accepts BOTH the trailing-slash and the v85 `/*` glob spelling via `normalizeSlugPrefix` (the column predates this fence as submit_agent's binding, so one stored value must mean one span of slugs on both paths), and ignores empty-string prefixes. `assertValidSlugPrefixes` (`oauth-provider.ts`) rejects empty, whitespace-bearing, non-lowercase, and boundary-less entries at registration and rescope. `submit_agent` applies the same boundary-aware rule when validating a requested prefix against the binding, normalizes trailing-slash prefixes to the glob form `matchesSlugAllowList` expects before handing them to the child job, and collapses an EXPLICIT empty `allowed_tools`/`allowed_slug_prefixes` to the binding (the worker reads empty as "full registry" / "legacy `wiki/agents//` namespace", so `??` — which only substitutes null/undefined — left a vacuous-subset bypass). `put_page` additionally fences the RESOLVED slug when importFromContent's dedup pre-check redirects the write to a different page (same content_hash / `frontmatter.id`), since the disk write-through runs against that slug. `CLIENT_FENCED_WRITE_OPS` + `enforceBoundClientOpAllowList(auth, op)` are the fail-closed companion, applied once in `src/mcp/dispatch.ts` (the choke point both MCP transports share): a slug-bound client calling ANY `write`/`admin` op not on the allow-list gets `permission_denied`. This covers the ops that write by a key other than a slug and therefore cannot be fenced — `extract_entities`/`extract_facts` (mutate `people/*`, `companies/*`), `forget_fact` (numeric fact id, crosses sources), `ontology_propose` — and makes a write op added later denied-by-default instead of silently unfenced. `think` is on the allow-list because remote callers cannot persist from it. Pinned by `test/client-slug-fence.test.ts` and over-the-wire by `test/e2e/qm-provisioning.test.ts`. Every `Operation` carries `scope?: 'read' | 'write' | 'admin'` + `localOnly?: boolean`; `sync_brain`, `file_upload`, `file_list`, `file_url` are `admin + localOnly` (rejected over HTTP). Four trust-boundary call sites (`put_page` allowlist, file_upload trust-narrowing, submit_job protected-name guard, auto-link skip) use FAIL-CLOSED semantics: `ctx.remote === false` for trusted-only sites, `ctx.remote !== false` for "untrust unless explicit-false" — anything not strictly `false` is treated as remote (closes the HTTP MCP shell-job RCE where a read+write OAuth token could submit `shell` jobs). `sourceScopeOpts(ctx)` encodes the source-scoped read precedence ladder — federated array (`ctx.auth.allowedSources`) wins over scalar (`ctx.sourceId`/`ctx.auth.sourceId`) over nothing; every read-side op handler routes through it so a source-bound OAuth client can't see neighboring sources via `search`/`query`/`list_pages`/`get_page`/`find_experts`/query's image path, plus the by-slug reads `get_tags`/`get_links`/`get_backlinks`/`get_timeline` (and `get_page`'s tag fetch, which resolves against the concrete page's own `source_id`). `linkReadScopeOpts(ctx)` is the link-read sibling for `get_links`/`get_backlinks`: a link row references three pages (from/to/origin), and the engine's federated (`sourceIds[]`) branch scopes ALL THREE while its scalar (`sourceId`) branch scopes only the near endpoint (by design — trusted internal callers like `reconcileLinks` and back-link validators call the engine with a scalar scope and need the cross-source view). For an UNTRUSTED remote caller (`ctx.remote !== false`) carrying only a scalar scope it promotes that scope to a single-element `sourceIds:[id]`, routing them through the all-endpoint branch so a far/origin slug from outside the grant isn't disclosed; a federated array passes through unchanged and trusted local CLI (`ctx.remote === false`) keeps the scalar cross-source view. `thinkSourceScopeOpts(ctx)` maps the same precedence ladder onto `runThink`'s public options (`allowedSources`/`sourceId`) so the `think` op's gather and trajectory stages inherit the caller's source grant. `put_page`'s inline disk write-through is the shared `writePageThrough` helper (`src/core/write-through.ts`), ATOMIC via temp-sibling + rename so a crash or concurrent `gbrain sync` can't read a half-written `.md`; same helper backs `gbrain brainstorm/lsd --save`. Link provenance surface (#1941): `add_link` (`gbrain link`/`link-add`) + `remove_link` (`gbrain unlink`/`link-rm`) expose `link_source`/`link_type`; `add_link` rejects the reconciliation-managed built-ins via `MANAGED_LINK_SOURCES` (`markdown`/`frontmatter`/`mentions`/`wikilink-resolved`) and defaults omitted provenance to `'manual'` (the engine's own default stays `'markdown'` for internal callers); `list_link_sources` (`gbrain link-sources`, read) lists provenances via `sourceScopeOpts`. CLI aliases register through `cliHints.aliases` (collision-guarded in `src/cli.ts`). - `src/core/engine.ts` — Pluggable engine interface (BrainEngine). `clampSearchLimit(limit, default, cap)` takes an explicit cap so per-operation caps can be tighter than `MAX_SEARCH_LIMIT`. Exports `LinkBatchInput`/`TimelineBatchInput` for the bulk-insert API (`addLinksBatch`/`addTimelineEntriesBatch`). `readonly kind: 'postgres' | 'pglite'` discriminator lets `src/core/migrate.ts` and others branch without `instanceof` + dynamic imports. Methods: `batchLoadEmotionalInputs(slugs?)` (CTE-shaped read with per-table aggregates so page × N tags × M takes never produces N×M rows), `setEmotionalWeightBatch(rows)` (`UPDATE FROM unnest($1::text[],$2::text[],$3::real[])` composite-keyed on `(slug, source_id)`), `getRecentSalience(opts)`, `findAnomalies(opts)`. `PageFilters` has `sort?: 'updated_desc'|'updated_asc'|'created_desc'|'slug'` + `PAGE_SORT_SQL` whitelist consumed by both engines. `listAllPageRefs(): Promise>` ordered by `(source_id, slug)` — cheap cross-source enumeration replacing the `getAllSlugs()→getPage(slug)` N+1 (which silently defaulted to `source_id='default'`); parity across postgres-engine.ts + pglite-engine.ts; Pinned by `test/e2e/multi-source-bug-class.test.ts`. `SearchOpts`+`PageFilters` add `sourceIds?: string[]` (federated read axis; both engines apply `WHERE source_id = ANY($N::text[])` when set, preserve scalar `sourceId` fast path when unset); `traverseGraph(slug, depth, opts?)` and `traversePaths(slug, opts?)` accept `opts.sourceId`/`opts.sourceIds`. The by-slug read methods carry the same federated axis: `getTags`/`getLinks`/`getBacklinks` opts and `TimelineOpts` (consumed by `getTimeline`) accept `sourceIds?: string[]` taking precedence over the scalar `sourceId` (`source_id = ANY($::text[])` scoping the slug→page-id lookup); the link reads (`getLinks`/`getBacklinks`) scope ALL THREE endpoints (from/to/origin) on the federated branch while the scalar branch scopes only the near endpoint for trusted internal cross-source callers. `traverseGraph` opts has `frontierCap?: number` (per-iteration recursive-CTE cap, approx per-BFS-layer); return type `Promise` for MCP wire stability; export `TraverseGraphOpts`; Postgres uses parenthesized `LIMIT N ORDER BY (slug, id)` inside the recursive term, PGLite mirrors with positional params; Pinned by `test/regressions/v0_36_frontier_cap.test.ts`. Phantom-redirect methods: `refreshPageBody(slug, sourceId, compiled_truth, timeline, content_hash)` narrow-UPDATEs three columns + updated_at, skipping soft-deleted rows (content_hash refresh required so `gbrain sync` sees the canonical as unchanged after fence merge); `migrateFactsToCanonical(phantomSlug, canonicalSlug, sourceId)` UPDATEs `entity_slug`+`source_markdown_slug` on every active fact row keyed on the phantom, preserving embedding/validUntil/kind/status/source_session/confidence; parity at `test/phantom-redirect-engine-parity.test.ts`. `getAdjacencyBoosts(pageIds): Promise>` powers the per-query graph-signals stage — one SQL query returning inbound-link counts among top-K plus a cross-source count (links from differing `source_id`); `COALESCE(p.source_id,'default')` null safety, `HAVING >= 1`, cross-source CASE-WHEN excludes the target's own source; parity SQL across both engines; `SearchResult` gains optional `base_score`, `backlink_boost`, `salience_boost`, `recency_boost`, `exact_match_boost`, `graph_adjacency_boost`, `graph_cross_source_boost`, `session_demote_factor`, `reranker_delta` + internal staging fields; Pinned by `test/e2e/graph-signals-engine.test.ts`. Two REQUIRED methods: `deletePages(slugs, {sourceId}): Promise` (single-batch primitive returning slugs actually deleted) and `resolveSlugsByPaths(paths, {sourceId}): Promise>` (batch path→slug lookup); `sourceId` REQUIRED on both at the type level (asymmetric with single-row `deletePage` which keeps optional/'default'); both short-circuit on empty input and throw when `> DELETE_BATCH_SIZE`. Embedding-signature stale-detection quartet: `countStaleChunks(opts?)` gains optional `signature?: string` widening the stale predicate from `embedding IS NULL` to ALSO include chunks whose JOINed page `embedding_signature IS NOT NULL AND <> $signature` (NULL signature is GRANDFATHERED, never counted; omit `signature` for the legacy NULL-only count); `sumStaleChunkChars(opts?: {sourceId?, signature?}): Promise` = `SUM(LENGTH(chunk_text))` over stale chunks (same predicate + embed_skip filter + optional sourceId scope), used by `gbrain sync --all` cost preview via `estimateCostFromChars`; `setPageEmbeddingSignature(slug, {sourceId?, signature})` stamps `pages.embedding_signature` after a page's chunks (re)embed, idempotent no-op when page absent; `invalidateStaleSignatureEmbeddings({signature, sourceId?}): Promise` NULLs `embedding`+`embedded_at` on every chunk whose page signature is set AND differs, returning the count, called BEFORE `listStaleChunks` so signature-drift pages flow through the NULL-embedding keyset cursor unchanged (NULL never invalidated). Widens `findOrphanPages(opts?: {sourceId?, sourceIds?})` (candidate-side scoping only; inbound links counted from any source). Pinned by `test/sum-stale-chunk-chars.test.ts`, `test/embedding-signature-stale.test.ts`, `test/e2e/engine-parity.test.ts`. Free-text alias layer: `resolveAliases(aliasNorms, opts?): Promise>>` (READ; maps each normalized alias to declaring `(slug, source_id)` pairs, source-scoped) and `setPageAliases(slug, sourceId, aliasNorms)` (WRITE; replaces the full alias set, delete-then-insert, empty clears, idempotent on the unique triple), called by the `importFromContent` ingest projection and the `reindex --aliases` backfill; parity across both engines, Pinned by `test/search/page-aliases-engine.test.ts`. `searchVector` in both engines injects the shared `buildBestPerPagePoolCte` per-page max-pool so a page surfaces on its strongest chunk. `executeRawDirect(sql, params?, opts?)` is the lock-hot-path sibling of `executeRaw`: same single-statement contract, but routes to the direct session-mode pool when dual-pool is active (Postgres/Supabase port 5432) so a long-held lock heartbeat survives the transaction pooler's per-transaction connection recycling; PGLite delegates straight to `executeRaw` (no pooler). Both engines implement it; the Minion lock path (`claim`/`renewLock`) is the consumer. `reconnect(ctx?: {error?})` is a REQUIRED lifecycle method on BOTH engines: it recovers a dropped connection using the config captured at the last `connect()`, so callers (autopilot health probe, `batchRetry`) never `disconnect()` + bare `connect()` (which loses the config and throws `database_url undefined` forever, and opens a null-connection window). PostgresEngine rebuilds its pool with a `_reconnecting` reentrancy guard and emits pool-recovery audit; PGLite is single-writer in-process so it just restores the saved data dir for parity. - `src/core/engine-constants.ts` — single source of truth for engine batch-sizing constants. Exports `DELETE_BATCH_SIZE = 500` consumed by both engines' `deletePages` + `resolveSlugsByPaths` and by the sync delete + rename loops. Lives outside `engine.ts` (the interface module) to avoid circular-import worry — bounded per-statement work for predictable lock hold time + write amplification. - `src/core/background-work.ts` (#1762/#1745/#1775) — process background-work registry: the single owner of "drain every fire-and-forget DB-write sink before the CLI disconnects." `registerBackgroundWorkDrainer({name, order, drain(timeoutMs), abort?})` + `drainAllBackgroundWorkForCliExit({timeoutMs})` over a `Map` (idempotent registration by name; `__registerDrainerForTest` returns an unregister handle). Drains in explicit `(order, name)` order — facts FIRST (order 0) so its abort-path DB `logIngest` runs against the freshest live engine — and AWAITS `abort()` only when `drain()` reports `unfinished>0`. Best-effort per drainer: one sink's failure never blocks the others or the disconnect. FIVE sinks register at module import: `facts/queue.ts` (order 0; `abort`=`shutdown()` cancels a hung facts:absorb Haiku via internalAbort), `last-retrieved.ts` (order 1), `search/hybrid.ts` (order 2; `awaitPendingSearchCacheWrites` bounded via `Promise.race`), `eval-capture.ts` (order 3; `captureEvalCandidate` self-tracks its promise via `awaitPendingEvalCaptures`), `context/volunteer-events.ts` (order 4, #2095 — batched volunteer-event INSERTs, drained like the rest). Every cli.ts teardown site reaches it through `finishCliTeardown` (`src/core/cli-force-exit.ts`), which drains the registry before `engine.disconnect()` — closing the PGLite busy-loop where `db.close()` raced an in-flight job and pinned the single-writer lock (#1762). Exports `backgroundWorkSinkCount()` so the teardown helper computes its backstop deadline from the registered sink count. CLI-EXIT-ONLY: the facts `shutdown()` abort is permanent process state, never call in a long-lived `gbrain serve`. Companion changes: `src/core/ai/gateway.ts` `withDefaultTimeout(caller, ms)` bounds every outbound AI call (chat 300s, embed+multimodal 60s; env `GBRAIN_AI_{CHAT,EMBED,MULTIMODAL}_TIMEOUT_MS`; composed with caller signals via `AbortSignal.any`) and the teardown backstop honors an errored op's exit code so a hung disconnect can't mask failure as success (see `cli-force-exit.ts`); `src/core/postgres-engine.ts` `reconnect()` module-mode branch re-establishes via idempotent `db.connect()` + `connectionManager.setReadPool` refresh instead of `db.disconnect()` (no null window for concurrent ops; fail-loud on real connect failure — #1745); `src/core/search/hybrid.ts` `embedQueryBounded` + a shared `QueryEmbedDeadline` (6s, floored 2s per embed via `MIN_QUERY_EMBED_BUDGET_MS`; env `GBRAIN_QUERY_EMBED_TIMEOUT_MS`) bounds the cache-lookup AND inner query embeds so a stalled provider falls back to keyword instead of stalling the whole op (#1775). Incorporates + hardens PR #1763 (@ElliotDrel). Pinned by `test/core/background-work.test.ts`, `test/search/query-embed-deadline.test.ts`, `test/eval-capture-drain.test.ts`, `test/e2e/postgres-reconnect-singleton.test.ts`, `test/e2e/pglite-cli-exit.serial.test.ts`, `test/fix-wave-structural.test.ts`. @@ -279,10 +279,10 @@ per-release `**vX.Y.Z:**` narration — CI enforces this - `src/mcp/server.ts` — MCP stdio server (generated from operations). Tool-call handler delegates to `dispatchToolCall` from `src/mcp/dispatch.ts` so stdio + HTTP transports share one validation, context-build, and error-format path. Stdin `'end'` / `'close'` shutdown hooks are skipped when `process.env.MCP_STDIO === '1'` — gateway-piped stdio MCP wrappers (OpenClaw's `bundle-mcp`) pipe the handshake then close their stdin half, which would otherwise kill the server before the first tool call; signal handlers (SIGTERM/SIGINT/SIGHUP) + the parent-process watchdog still cover legitimate disconnects. `src/commands/serve.ts` exposes `ServeOptions.mcpStdio?: boolean` as a test seam so the guard is exercisable without process.env mutation. Pinned by `test/serve-stdio-lifecycle.test.ts`. - `src/mcp/dispatch.ts` — shared tool-call dispatch consumed by both stdio (`server.ts`) and HTTP transports. Exports `dispatchToolCall(engine, name, params, opts)`, `buildOperationContext(engine, params, opts)`, `validateParams(op, params)`. Single source of truth for `(ctx, params)` handler arg order and the 5-field `OperationContext` shape (engine + config + logger + dryRun + remote). Defaults `remote: true` (untrusted); local CLI callers pass `remote: false`. Also exports `summarizeMcpParams(opName, params)` — privacy-preserving redactor for `mcp_request_log` and the admin SSE feed, returns `{redacted, kind, declared_keys, unknown_key_count, approx_bytes}`. Intersects submitted top-level keys against the operation's declared `params` allow-list (declared keys preserved sorted; unknown keys counted but never named, closing the attacker-controlled-key-name leak). Byte counts bucketed up to nearest 1KB so an attacker can't binary-search secret-content sizes by probing. Raw payload visibility is opt-in via `gbrain serve --http --log-full-params` (loud stderr warning). New logging paths route through this helper, not `JSON.stringify(params)`. - `src/mcp/rate-limit.ts` — Bounded-LRU token-bucket limiter. `buildDefaultLimiters()` returns the two-bucket pipeline: pre-auth IP (30/60s, fires BEFORE the DB lookup so brute-force load against `access_tokens` is capped) + post-auth token-id (60/60s). Tracks `lastTouchedMs` separately from `lastRefillMs` so an exhausted key can't be reset by hammering past the TTL. LRU cap bounds memory under attacker-controlled key growth. -- `src/commands/serve-http.ts` — Express 5 HTTP MCP server with OAuth 2.1, admin dashboard, and SSE live activity feed. Started via `gbrain serve --http [--port N] [--token-ttl N] [--enable-dcr] [--public-url URL] [--bind HOST] [--log-full-params]`. Combines MCP SDK's `mcpAuthRouter` (authorize/token/register/revoke), a custom `client_credentials` handler running BEFORE the router (SDK's token endpoint throws `UnsupportedGrantTypeError` for CC; custom handler falls through for `auth_code` / `refresh_token`), `requireBearerAuth` middleware for `/mcp` with scope enforcement + `localOnly` rejection before op dispatch, and `express-rate-limit` at 50 req / 15 min on `/token`. Serves the built admin SPA from `admin/dist/` with SPA fallback. `/admin/events` SSE broadcasts every MCP request. `cookie-parser` wired (Express 5 has no built-in). Startup logging prints port, engine, issuer URL (honors `--public-url`), client count, DCR status, and the admin bootstrap token line — but the generated token's raw value only prints when stderr is an interactive TTY (`shouldSuppressBootstrapPrint`): a non-TTY (containerized/piped) start hides it so the secret never lands in centralized log storage, env-sourced tokens (`$GBRAIN_ADMIN_BOOTSTRAP_TOKEN`) are always hidden, `--print-admin-token` forces the raw value on a trusted terminal, and `--suppress-bootstrap-token` hides everything. The `/mcp` request handler's OperationContext literal sets `remote: true` explicitly (without it `submit_job`'s protected-name guard at `operations.ts:1391` saw a falsy undefined and a `read+write`-scoped OAuth token could submit `shell` jobs — RCE). `summarizeMcpParams` from `src/mcp/dispatch.ts` feeds both `mcp_request_log` writes and the SSE feed by default (raw via `--log-full-params`). Cookie `Secure` flag set behind HTTPS or a public-URL proxy; magic-link nonce store LRU-bounded; DCR disable routes through the `GBrainOAuthProvider` `dcrDisabled` constructor option (not a router monkey-patch); `transport.handleRequest` wrapped in try/catch to return a JSON-RPC 500 envelope; OperationError + unexpected exceptions unified through `buildError` / `serializeError` so `/mcp` always returns the same envelope. `/health` is liveness-only via `probeLiveness(sql, engineName, version, timeoutMs)` racing `sql\`SELECT 1\`` against the exported `HEALTH_TIMEOUT_MS = 3000` (returns the same `ProbeHealthResult` tagged-union as `probeHealth`, single timer-cleanup site, single 503 envelope); body shape `{status, version, engine}` only. Full stats moved to admin-only `/admin/api/full-stats` (gated by `requireAdmin`, calls `probeHealth(engine, ...)`) — keeps `getStats()`'s 6× count(*) off the public route so a saturated pool doesn't trigger orchestrator restart cascades. Every OAuth/admin/audit SQL call routes through `sqlQueryForEngine(engine)` from `src/core/sql-query.ts` so it works against PGLite; the four `mcp_request_log.params` INSERT sites (success / auth_failed / scope_denied / server-error) go through `executeRawJsonb(engine, ...)` so the column stores real objects (`params->>'op'` returns `search`, not the quoted string). `--bind HOST` defaults `127.0.0.1` (self-hosters pass `--bind 0.0.0.0`); a stderr WARN fires when `--public-url` is set without `--bind`; the banner prints a `Bind:` line. `AuthInfo.sourceId` + `AuthInfo.allowedSources` are the typed source of truth, populated by `oauth-provider.ts:verifyAccessToken` from the `oauth_clients` row. The HTTP MCP `tools/list` handler at `:837-849` uses `paramDefToSchema(v)` from `src/mcp/tool-defs.ts` so array params keep `items` (strict-mode OAuth clients otherwise reject the whole tool list). +- `src/commands/serve-http.ts` — Express 5 HTTP MCP server with OAuth 2.1, admin dashboard, and SSE live activity feed. Started via `gbrain serve --http [--port N] [--token-ttl N] [--enable-dcr] [--public-url URL] [--bind HOST] [--log-full-params]`. Combines MCP SDK's `mcpAuthRouter` (authorize/token/register/revoke), a custom `client_credentials` handler running BEFORE the router (SDK's token endpoint throws `UnsupportedGrantTypeError` for CC; custom handler falls through for `auth_code` / `refresh_token`), `requireBearerAuth` middleware for `/mcp` with scope enforcement + `localOnly` rejection before op dispatch, and `express-rate-limit` at 50 req / 15 min on `/token`. Serves the built admin SPA from `admin/dist/` with SPA fallback. `/admin/events` SSE broadcasts every MCP request. `cookie-parser` wired (Express 5 has no built-in). Startup logging prints port, engine, issuer URL (honors `--public-url`), client count, DCR status, and the admin bootstrap token line — but the generated token's raw value only prints when stderr is an interactive TTY (`shouldSuppressBootstrapPrint`): a non-TTY (containerized/piped) start hides it so the secret never lands in centralized log storage, env-sourced tokens (`$GBRAIN_ADMIN_BOOTSTRAP_TOKEN`) are always hidden, `--print-admin-token` forces the raw value on a trusted terminal, and `--suppress-bootstrap-token` hides everything. The `/mcp` request handler's OperationContext literal sets `remote: true` explicitly (without it `submit_job`'s protected-name guard at `operations.ts:1391` saw a falsy undefined and a `read+write`-scoped OAuth token could submit `shell` jobs — RCE). `summarizeMcpParams` from `src/mcp/dispatch.ts` feeds both `mcp_request_log` writes and the SSE feed by default (raw via `--log-full-params`). Cookie `Secure` flag set behind HTTPS or a public-URL proxy; magic-link nonce store LRU-bounded; DCR disable routes through the `GBrainOAuthProvider` `dcrDisabled` constructor option (not a router monkey-patch); `transport.handleRequest` wrapped in try/catch to return a JSON-RPC 500 envelope; OperationError + unexpected exceptions unified through `buildError` / `serializeError` so `/mcp` always returns the same envelope. `/health` is liveness-only via `probeLiveness(sql, engineName, version, timeoutMs)` racing `sql\`SELECT 1\`` against the exported `HEALTH_TIMEOUT_MS = 3000` (returns the same `ProbeHealthResult` tagged-union as `probeHealth`, single timer-cleanup site, single 503 envelope); body shape `{status, version, engine}` only. Full stats moved to admin-only `/admin/api/full-stats` (gated by `requireAdmin`, calls `probeHealth(engine, ...)`) — keeps `getStats()`'s 6× count(*) off the public route so a saturated pool doesn't trigger orchestrator restart cascades. Every OAuth/admin/audit SQL call routes through `sqlQueryForEngine(engine)` from `src/core/sql-query.ts` so it works against PGLite; the four `mcp_request_log.params` INSERT sites (success / auth_failed / scope_denied / server-error) go through `executeRawJsonb(engine, ...)` so the column stores real objects (`params->>'op'` returns `search`, not the quoted string). `--bind HOST` defaults `127.0.0.1` (self-hosters pass `--bind 0.0.0.0`); a stderr WARN fires when `--public-url` is set without `--bind`; the banner prints a `Bind:` line. `AuthInfo.sourceId` + `AuthInfo.allowedSources` are the typed source of truth, populated by `oauth-provider.ts:verifyAccessToken` from the `oauth_clients` row. The HTTP MCP `tools/list` handler at `:837-849` uses `paramDefToSchema(v)` from `src/mcp/tool-defs.ts` so array params keep `items` (strict-mode OAuth clients otherwise reject the whole tool list). `POST /ingest` enforces the slug-prefix write fence at the ROUTE, not the op layer: the route hands its payload to the `ingest_capture` minion handler, which deliberately bypasses `put_page`, so no `OperationContext` exists and `enforceClientSlugFence` never runs — a slug-bound client must therefore supply `X-Gbrain-Slug` and it must satisfy `slugUnderBoundPrefixes`, else 403 (without the check a bound client could overwrite any page, in the `default` source, since untrusted payloads carry no source grant). - `src/core/sql-query.ts` — engine-aware tagged-template SQL adapter for OAuth/admin/auth infrastructure. `sqlQueryForEngine(engine)` returns a `SqlQuery` (`(strings, ...values) => Promise`) that walks the template, builds `$N` positional SQL, asserts every value is a `SqlValue` (string | number | bigint | boolean | Date | null), and routes through `engine.executeRaw(sql, params)` (Postgres via postgres.js `unsafe(sql, params)`, PGLite via `db.query(sql, params)`). Deliberately narrower than postgres.js's `sql` tag: no nested fragments, `sql.json()`, `sql.unsafe()`, `sql.begin()`, or array binding — the narrow scalar-only surface is the feature (keeps it from drifting into a partial postgres.js clone). JSONB writes go through `executeRawJsonb(engine, sql, scalarParams, jsonbParams)` which composes positional `$N::jsonb` casts and passes JS **objects** through; an object reaches the wire with the correct type oid, so executeRawJsonb is safe (verified by `test/sql-query.test.ts` on PGLite, `test/e2e/auth-permissions.test.ts:67` on Postgres). Positional binding is NOT universally immune, though: binding a `JSON.stringify(x)` **string** to a bare `$N::jsonb` via `unsafe()` double-encodes it into a jsonb string scalar on real Postgres (the #2339 class; PGLite hides it). Fixes: pass a raw object (executeRawJsonb / `sql.json`), or cast through `$N::text::jsonb`. `scripts/check-jsonb-pattern.sh` (template grep) doesn't fire on `executeRawJsonb(...)` because it passes objects; the positional `$N::jsonb` + `JSON.stringify` form is caught by `scripts/check-jsonb-params.mjs`. Consumed by `src/commands/auth.ts`, `src/commands/serve-http.ts`, `src/core/oauth-provider.ts`, `src/commands/files.ts`, `src/mcp/http-transport.ts` so all five work uniformly against PGLite and Postgres. - `src/commands/serve.ts` — `gbrain serve` stdio MCP entrypoint with idempotent shutdown across every parent-disconnect signal. Stdio EOF, SIGTERM, SIGINT, SIGHUP, and parent-process death (every reparent case — PID 1, launchd subreaper, systemd, tmux, or a parent shell with `PR_SET_CHILD_SUBREAPER`) all funnel into one `cleanup(reason)` that releases the engine and the PGLite write-lock dir within 5 seconds (otherwise the lock is held indefinitely after Claude Desktop / Cursor / launchd-managed gateways disconnect, forcing a 5-minute stale-lock wait on next start). Watchdog reparent check is `getParentPid() !== initialParentPid` (the `=== 1` check missed the subreaper case under launchd/systemd). Bun's `process.ppid` cache is stale across reparenting ([oven-sh/bun#30305](https://github.com/oven-sh/bun/issues/30305)) so `getParentPid()` runs `spawnSync('ps', ['-o', 'ppid=', '-p', PID])` per tick. Startup probe verifies `ps` is on PATH; if not (stripped containers, busybox), the watchdog skips installing AND emits a loud `[gbrain serve] watchdog disabled: ps unavailable ...` stderr line so operators see the degraded mode. Pinned by `test/serve-stdio-lifecycle.test.ts` (22 cases). Credit @Aragorn2046 + @seungsu-kr. -- `src/core/oauth-provider.ts` — `GBrainOAuthProvider` implementing the MCP SDK's `OAuthServerProvider` + `OAuthRegisteredClientsStore`. Backed by raw SQL (works on both PGLite and Postgres — OAuth is infrastructure, not a BrainEngine concern). Full OAuth 2.1: `authorize` + `exchangeAuthorizationCode` with PKCE, `client_credentials`, `refresh_token` with rotation, `revokeToken`, `registerClient` (DCR validates redirect_uri is `https://` or loopback per RFC 6749 §3.1.2.1). All tokens + client secrets SHA-256 hashed before storage. Auth codes single-use with 10-minute TTL via atomic `DELETE...RETURNING` (closes RFC 6749 §10.5 TOCTOU); refresh rotation also `DELETE...RETURNING` (§10.4 stolen-token detection). `pgArray()` escapes commas/quotes/braces so a comma-bearing redirect_uri can't smuggle a second array element. Legacy `access_tokens` fallback in `verifyAccessToken` grandfathers pre-v0.26 bearer tokens as `read+write+admin`. `sweepExpiredTokens()` runs on startup in try/catch and returns the count via `RETURNING 1` + array length. RFC hardening: `client_id` folded atomically into the `DELETE WHERE` for both auth-code exchange and refresh rotation (wrong-client paths don't burn the row); refresh-scope-subset enforced against the original grant on the row (RFC 6749 §6, so revoking a scope shrinks existing refresh tokens); `client_id` bound on `revokeToken` (RFC 7009 §2.1); `/token` `redirect_uri` validated against the `/authorize` value (RFC 6749 §4.1.3, empty-string treated as missing not wildcard); bare `catch {}` in `verifyAccessToken`/`getClient` replaced by `isUndefinedColumnError` from `src/core/utils.ts` (only SQLSTATE 42703 falls through to legacy; lock timeouts/network blips throw); `dcrDisabled` constructor option lets `serve-http.ts` disable `/register` without monkey-patching the router. Module-private `coerceTimestamp()` normalizes postgres-driver-as-string BIGINT columns to JS numbers at 5 read sites (`getClient` for RFC 7591 §3.2.1 numeric timestamps, `exchangeRefreshToken` + `verifyAccessToken` for the SDK's `typeof === 'number'` check); throws on NaN/Infinity (fail loud at boundary), returns undefined for SQL NULL (callers treat NULL as expired). Not promoted to `utils.ts` — generic BIGINT precision-loss risk. `registerClient` honors `token_endpoint_auth_method: "none"` (RFC 7591 §3.2.1): public PKCE clients store `client_secret_hash = NULL` and the response omits `client_secret`; confidential clients (`client_secret_post` / `client_secret_basic`) keep their one-time-reveal shape; `getClient` normalizes NULL `client_secret_hash` to JS `undefined` so the SDK's clientAuth path accepts public clients. `verifyAccessToken` JOINs `oauth_clients.source_id` (write scope, scalar) + `oauth_clients.federated_read` (read scope, TEXT[]) onto the returned `AuthInfo`; legacy brains degrade via `isUndefinedColumnError` fallback. +- `src/core/oauth-provider.ts` — `GBrainOAuthProvider` implementing the MCP SDK's `OAuthServerProvider` + `OAuthRegisteredClientsStore`. Backed by raw SQL (works on both PGLite and Postgres — OAuth is infrastructure, not a BrainEngine concern). Full OAuth 2.1: `authorize` + `exchangeAuthorizationCode` with PKCE, `client_credentials`, `refresh_token` with rotation, `revokeToken`, `registerClient` (DCR validates redirect_uri is `https://` or loopback per RFC 6749 §3.1.2.1). All tokens + client secrets SHA-256 hashed before storage. Auth codes single-use with 10-minute TTL via atomic `DELETE...RETURNING` (closes RFC 6749 §10.5 TOCTOU); refresh rotation also `DELETE...RETURNING` (§10.4 stolen-token detection). `pgArray()` escapes commas/quotes/braces so a comma-bearing redirect_uri can't smuggle a second array element. Legacy `access_tokens` fallback in `verifyAccessToken` grandfathers pre-v0.26 bearer tokens as `read+write+admin`. `sweepExpiredTokens()` runs on startup in try/catch and returns the count via `RETURNING 1` + array length. RFC hardening: `client_id` folded atomically into the `DELETE WHERE` for both auth-code exchange and refresh rotation (wrong-client paths don't burn the row); refresh-scope-subset enforced against the original grant on the row (RFC 6749 §6, so revoking a scope shrinks existing refresh tokens); `client_id` bound on `revokeToken` (RFC 7009 §2.1); `/token` `redirect_uri` validated against the `/authorize` value (RFC 6749 §4.1.3, empty-string treated as missing not wildcard); bare `catch {}` in `verifyAccessToken`/`getClient` replaced by `isUndefinedColumnError` from `src/core/utils.ts` (only SQLSTATE 42703 falls through to legacy; lock timeouts/network blips throw); `dcrDisabled` constructor option lets `serve-http.ts` disable `/register` without monkey-patching the router. Module-private `coerceTimestamp()` normalizes postgres-driver-as-string BIGINT columns to JS numbers at 5 read sites (`getClient` for RFC 7591 §3.2.1 numeric timestamps, `exchangeRefreshToken` + `verifyAccessToken` for the SDK's `typeof === 'number'` check); throws on NaN/Infinity (fail loud at boundary), returns undefined for SQL NULL (callers treat NULL as expired). Not promoted to `utils.ts` — generic BIGINT precision-loss risk. `registerClient` honors `token_endpoint_auth_method: "none"` (RFC 7591 §3.2.1): public PKCE clients store `client_secret_hash = NULL` and the response omits `client_secret`; confidential clients (`client_secret_post` / `client_secret_basic`) keep their one-time-reveal shape; `getClient` normalizes NULL `client_secret_hash` to JS `undefined` so the SDK's clientAuth path accepts public clients. `verifyAccessToken` JOINs `oauth_clients.source_id` (write scope, scalar) + `oauth_clients.federated_read` (read scope, TEXT[]) + `oauth_clients.bound_slug_prefixes` (write fence, TEXT[] — consumed by `enforceClientSlugFence` in `operations.ts`) onto the returned `AuthInfo`; legacy brains degrade via `isUndefinedColumnError` fallback, dropping the newest projection first. `rescopeClient(clientId, {sourceId?, federatedRead?, boundSlugPrefixes?})` is the trusted-operator rescope (CLI `gbrain auth rescope-client`, admin `POST /admin/api/rescope-client`); `boundSlugPrefixes` is tri-state — undefined leaves the binding untouched, `null` clears it, a non-empty array replaces it (explicit empty array rejected as ambiguous deny-all) — so roster churn updates the write fence in place without rotating secrets. - `admin/` — React 19 + Vite + TypeScript admin SPA embedded in the binary via `admin/dist/` served by `serve-http.ts`. 7 screens: Login (bootstrap token → session cookie), Dashboard (metrics + SSE feed + token health), Agents (sortable table + sparklines + Register), Register (modal with scope checkboxes + grant type selector), Credentials reveal (Copy + Download JSON + one-time-only warning), Request Log (filterable paginated), Agent Detail drawer (Details / Activity / Config Export tabs + Revoke). Design tokens: `#0a0a0f` bg, Inter for UI, JetBrains Mono for data, 4-32px spacing scale, rounded pill badges. HTTP-only SameSite=Strict cookie auth. 65KB gzip. Build: `cd admin && bun install && bun run build`; output at `admin/dist/` is committed for self-contained binaries. - `src/commands/auth.ts` — token management. `gbrain auth create/list/revoke/test` for legacy bearer tokens, plus `gbrain auth register-client` and `gbrain auth revoke-client ` for OAuth 2.1 client lifecycle. `revoke-client` runs an atomic `DELETE...RETURNING` on `oauth_clients`; FK `ON DELETE CASCADE` on `oauth_tokens.client_id` and `oauth_codes.client_id` purges every active token + auth code in one transaction; `process.exit(1)` on no-such-client (idempotent). Legacy tokens stored as SHA-256 hashes in `access_tokens`; OAuth clients in `oauth_clients`; legacy tokens grandfather to `read+write+admin` scopes on the OAuth HTTP server (no migration). Every SQL site routes through `sqlQueryForEngine(engine)` from `src/core/sql-query.ts` (and `executeRawJsonb` for the takes-holders `permissions` JSONB column) so `gbrain auth` works against PGLite; the takes-holders write goes through `executeRawJsonb(engine, sql, [name, hash], [{takes_holders:[...]}])` which round-trips with `jsonb_typeof = 'object'`. `register-client` accepts `--source ` (write authority, scalar) and `--federated-read ` (read scope, array) and prints the resolved `Write source` + `Federated reads`; pre-v0.34 clients backfill to `source_id='default'` via migration v60. The bare `gbrain auth create ` form (no `--takes-holders`) mints a token via the exported pure `parseAuthCreateArgs(rest)` (the inline version used `rest[takesIdx + 1]` resolving to `rest[0]` when `takesIdx === -1`, excluding the name from the positional search). Pinned by `test/auth-create-args.test.ts`. - `src/commands/connect.ts` + `src/core/connect-probe.ts` — `gbrain connect [--token ]` one-command coding-agent onboarding from a bearer token. Turns an MCP URL + token into a paste-ready `claude mcp add ... -H "Authorization: Bearer ..."` block (default) or, with `--install`, runs it directly and smoke-tests the token. Direct HTTP MCP — Claude Code talks straight to a remote `gbrain serve --http`, no local install needed. Token resolution: `--token` > `$GBRAIN_REMOTE_TOKEN` > placeholder (print) / error (install). The generated block tells the agent to call `get_brain_identity` + `list_skills` (the `LEARN_INSTRUCTION` export, which names `put_page` not `capture` since `capture` is CLI-only, not an MCP tool) with a core-tools fallback for hosts without skill publishing. URL normalization appends `/mcp` to a bare host but REJECTS a scheme-less host; pure helpers (`isLinkLocalOrMetadata`, URL parse, render) are unit-tested. Flags: `--token`, `--name ` (default `gbrain`, validated against `NAME_RE`), `--agent claude-code|codex|perplexity|generic`, `--install`, `--yes` (required for `--install` in non-TTY), `--force`, `--json` (token redacted unless `--show-token`), `--timeout-ms`. `connect` is in `CLI_ONLY` + `CLI_ONLY_SELF_HELP`; dispatched in `cli.ts:handleCliOnly` with no local DB connect. `AGENT_SPECS` drives per-agent rendering + `--install`: `claude-code` → `buildClaudeMcpAddArgv` (literal `-H "Authorization: Bearer "`); `codex` → `buildCodexMcpAddArgv` = `codex mcp add --url --bearer-token-env-var GBRAIN_REMOTE_TOKEN` (Codex reads the token from the env var at runtime, never written to config; `--install` runs it and prints an `export GBRAIN_REMOTE_TOKEN` hint when missing); `perplexity` + `generic` are `installable:false` and reject `--install`. `--oauth` (`supportsOAuth:true` = perplexity/generic only) emits an OAuth 2.1 client-credentials connector block (Issuer URL via `issuerFromMcpUrl` = mcp-url minus `/mcp`, Client ID, Client Secret) — least-privilege scopes + short-lived rotating tokens vs a long-lived full-access secret. Creds from `--client-id`/`--client-secret` (BYO) or `--register` (`deps.registerOAuthClient` shells `gbrain auth register-client --grant-types client_credentials --scopes --token-endpoint-auth-method client_secret_post` and parses `Client ID:`/`Client Secret:`); `--oauth` rejected for claude-code/codex and incompatible with `--install`. `buildJson` is a generic shape (`agent`, `command`/`command_argv` null for perplexity/generic, `header`, `env_var`, oauth fields with redaction); the codex `command` carries only the env-var name, never the token. `cmdString(binary, argv)` POSIX-single-quotes args. `ConnectDeps` = `{isTTY, promptYesNo, hasBinary(bin), runBinary(bin, argv), probe, env(name)}` — binary-generic so `claude` and `codex` share the path; `env` injectable for tests. Security: rendered command single-quotes the token so shell metacharacters can't run code when pasted; token validated before it lands in an HTTP header; link-local / cloud-metadata addresses (incl. IPv4-mapped IPv6 `::ffff:169.254.x.x` and AWS IMDSv2-over-IPv6 `fd00:ec2::254`) refused as a token-exfil guard while localhost/RFC1918/LAN stay allowed; token redacted from all error output. `src/core/connect-probe.ts` is the raw-bearer MCP smoke probe backing `--install`: connects the official MCP SDK `Client` over `StreamableHTTPClientTransport` with a STATIC `Authorization` header (no OAuth/discovery — distinct from `mcp-client.ts:callRemoteTool` which is OAuth-only and `remote-mcp-probe.ts:smokeTestMcp` which only sends `initialize`), runs the full `initialize` handshake via `client.connect()`, then calls `get_brain_identity` (read-scope, non-localOnly) to prove a tool call round-trips. Never throws — every failure maps to `{ ok: false, reason: 'auth' | 'unreachable' | 'timeout' | 'tool_error' | 'unknown', message }` so a wrong/expired token fails at setup, not on the agent's first request. `DEFAULT_PROBE_TIMEOUT_MS = 15_000` shared with `connect.ts`. `serve-http.ts` adds exported pure `skillPublishStatus(publishSkills)` for the startup banner `Skills: published / not published` line + a one-line `gbrain config set mcp.publish_skills true` stderr nudge when publishing is OFF. Docs: `docs/mcp/CODEX.md`, `docs/mcp/PERPLEXITY.md`, `docs/mcp/CLAUDE_CODE.md`, `docs/tutorials/connect-coding-agent.md`. Pinned by `test/connect.test.ts` (pure-helper + render, all four agents) + `test/e2e/connect-bearer.test.ts` (raw-bearer probe + full OAuth chain register→connect→discovery→`/token` mint→`get_brain_identity`, client registered in `beforeAll` before serve takes the PGLite single-writer lock; drives real `claude` + `codex` binaries through `connect --install` with sandboxed `HOME`/`CODEX_HOME`, asserts registration + token never in Codex config, skips when a binary is absent) + `test/e2e/serve-stdio-roundtrip.test.ts` (spawns real `gbrain serve` stdio against a fresh `init --pglite` brain, drives the SDK client through `initialize`→`tools/list`→`tools/call`, asserts the advertised core-tool set and that `capture` is NOT advertised) + `test/serve-skills-publish-nudge.test.ts` (the `test/audit/batch-retry-audit.test.ts` ENOENT case was made hermetic — it had read the real `~/.gbrain/audit`). diff --git a/docs/integrations/README.md b/docs/integrations/README.md index 31c85dcd5..32033f1dc 100644 --- a/docs/integrations/README.md +++ b/docs/integrations/README.md @@ -44,6 +44,7 @@ These require manual setup (no self-installing recipe yet): |-------|-------------| | [Credential Gateway](credential-gateway.md) | Set up ClawVisor or Hermes for Gmail, Calendar, Contacts access | | [Meeting & Call Webhooks](meeting-webhooks.md) | Circleback meeting transcripts + Quo/OpenPhone SMS/calls | +| [qm Harness](qm-harness.md) | gbrain as the company brain for a qm (multi-user agent harness) deployment — central HTTP MCP, per-scope clients, roster provisioning, write fencing | ## How to Read a Recipe diff --git a/docs/integrations/qm-harness-snippets/SKILL.md b/docs/integrations/qm-harness-snippets/SKILL.md new file mode 100644 index 000000000..79c4db152 --- /dev/null +++ b/docs/integrations/qm-harness-snippets/SKILL.md @@ -0,0 +1,79 @@ +--- +name: gbrain +description: Search and write the company knowledge brain. Use for any question about the org, people, projects, decisions, or history, and to persist durable knowledge beyond this scope's notebook. +--- + +# gbrain — the company brain + +This sandbox has the `gbrain` CLI connected (thin-client) to the org's central +brain. It is the deep, indexed, cross-source memory: org docs, shared channel +knowledge, and every agent's durable notes. Your scope's own notebook stays the +fast per-turn memory; the brain is where knowledge outlives a scope and becomes +searchable by everyone entitled to it. + +## First-run setup (once per sandbox — skip if `gbrain remote doctor` passes) + +Your scope's brain credentials arrive via the deployment's secret handoff +(keychain entry or one-time secret drop named `gbrain`). Then: + +```bash +gbrain init --mcp-only \ + --issuer-url "https://brain..com" \ + --mcp-url "https://brain..com/mcp" \ + --oauth-client-id "" \ + --oauth-client-secret "" +gbrain whoami # must succeed before using any other command +``` + +Pass the secret with `--oauth-client-secret`, not via `GBRAIN_REMOTE_CLIENT_SECRET`: +an env-sourced secret is deliberately NOT written to `~/.gbrain/config.json`, so +every later command would fail with "No client_secret available" once the +variable is out of scope. The flag persists it to the config file on this +sandbox's durable disk, which is what the tool's credential capture expects. + +Do not run `gbrain remote doctor` — it needs `admin` scope, which your client +does not have (by design). `gbrain whoami` is the read-scope health check. + +## Reading (do this liberally) + +```bash +gbrain search "who decided X and why" # hybrid semantic + keyword search +gbrain get # read one page +gbrain query "question" --json # search tuned for agent consumption +``` + +You can read: the shared agent-memory source, org read-only sources (wiki, +handbook), and everything under them. Reads are isolation-enforced server-side; +you only ever see sources your client is entitled to. + +## Writing (durable knowledge only, under YOUR prefixes) + +Your client is write-fenced to slug prefixes — your own namespace plus the +channels you belong to. Writes outside them are rejected server-side. + +```bash +# personal durable memory (your namespace): +gbrain put emp-/people/jane-example --content "..." + +# shared channel knowledge (channels you are in): +gbrain put chan-eng/decisions/2026-08-database-choice --content "..." +``` + +Conventions: +- Write conclusions and durable facts, not chat transcripts. One page per + entity/decision/topic; update the page rather than appending near-duplicates. +- Markdown with YAML frontmatter; the brain chunks, embeds, and links it. +- Cross-reference liberally: `gbrain link ` (from must be in your + namespace; linking TO any readable page is fine). +- When you learn something channel-relevant in personal work, mirror the + conclusion into the channel prefix with a `(said in )` provenance + note. + +## When to reach for the brain + +- Any question about the org, a person, a project, a decision, or history → + `gbrain search` FIRST, then answer. +- You produced knowledge with value beyond this conversation → `gbrain put`. +- Something looks wrong (auth errors, empty results you don't expect) → + `gbrain whoami` to confirm which client and scopes you're using, and report + its output. diff --git a/docs/integrations/qm-harness-snippets/provision-scopes.sh b/docs/integrations/qm-harness-snippets/provision-scopes.sh new file mode 100755 index 000000000..ffa8c666e --- /dev/null +++ b/docs/integrations/qm-harness-snippets/provision-scopes.sh @@ -0,0 +1,258 @@ +#!/usr/bin/env bash +# provision-scopes.sh — roster-driven gbrain provisioning for a qm deployment +# (or any multi-user agent harness with per-person + per-channel scopes). +# +# Reads a roster of channels + employees and converges the brain to it: +# - ensures the shared agent-memory source exists (path-less: agents write +# pages into it over MCP; `gbrain sync` skips it; if the brain host has +# sync.repo_path configured, pages also write through to .sources// +# on disk for git-backed durability) +# - registers one OAuth client per employee, write-fenced via +# bound_slug_prefixes to emp-/ plus chan-/ for each channel +# they are in, with federated reads over the memory source + any +# read-only sources you pass +# - re-running after roster edits rescopes existing clients IN PLACE +# (client ids are remembered in the state file; secrets never rotate +# unless you revoke + delete the state row) +# +# Usage: +# provision-scopes.sh roster.tsv \ +# [--memory-source agents] [--read-sources org-wiki,handbook] \ +# [--budget-usd-per-day 5] [--state-file roster.state.tsv] \ +# [--secrets-out new-credentials.tsv] [--gbrain gbrain] [--dry-run] +# +# Roster format (one entry per line; '#' comments and blank lines ignored): +# channel +# employee [comma-separated channel slugs] +# +# SECURITY: --secrets-out receives client secrets for NEW registrations, +# written exactly once (gbrain never re-shows them). Deliver each row to its +# scope's sandbox (e.g. via the harness keychain or a one-time secret drop), +# then delete the file. +# +# ponytail: sequential CLI loop, one gbrain invocation per roster row — fine +# to hundreds of employees; batch via the admin API if that ever hurts. + +# -f (noglob) is load-bearing, not stylistic: roster lines are word-split +# unquoted below, so without it a line like `employee * eng` would expand +# against the working directory and silently provision a filename as a +# person — i.e. the wrong write fence. Nothing here needs globbing. +set -euf -o pipefail + +# Client secrets and the id state file are written by this script; 077 makes +# them 0600 instead of the default 0644. Set before the first file is created. +umask 077 + +die() { echo "ERROR: $*" >&2; exit 1; } + +# Slugs become source ids, client names, AND slug-prefix write fences. The +# fence list is comma-separated, so an unvalidated slug containing a comma +# would inject an EXTRA prefix and hand the client write access to someone +# else's namespace. Fail closed on anything that isn't plain kebab-case. +valid_slug() { + case "$1" in + '') return 1 ;; + -*|*-) return 1 ;; + *[!a-z0-9-]*) return 1 ;; + *) return 0 ;; + esac +} +require_slug() { + valid_slug "$2" || die "roster: invalid $1 slug '$2' (allowed: lowercase a-z, 0-9, interior hyphens)" +} + +ROSTER="${1:-}" +[ -n "$ROSTER" ] && [ -f "$ROSTER" ] || die "usage: provision-scopes.sh [flags] (roster not found: '$ROSTER')" +shift + +GBRAIN="${GBRAIN:-gbrain}" +MEMORY_SOURCE="agents" +READ_SOURCES="" +BUDGET="5" +STATE_FILE="" +SECRETS_OUT="" +DRY_RUN=0 + +while [ $# -gt 0 ]; do + case "$1" in + --memory-source) MEMORY_SOURCE="$2"; shift 2 ;; + --read-sources) READ_SOURCES="$2"; shift 2 ;; + --budget-usd-per-day) BUDGET="$2"; shift 2 ;; + --state-file) STATE_FILE="$2"; shift 2 ;; + --secrets-out) SECRETS_OUT="$2"; shift 2 ;; + --gbrain) GBRAIN="$2"; shift 2 ;; + --dry-run) DRY_RUN=1; shift ;; + *) die "unknown flag: $1" ;; + esac +done + +STATE_FILE="${STATE_FILE:-${ROSTER}.state.tsv}" +SECRETS_OUT="${SECRETS_OUT:-${ROSTER}.new-credentials.tsv}" + +# The roster usually lives in the deployment repo, so the default secrets and +# state paths land there too — one `git add -A` from committing live +# credentials. The STATE file matters as much as the secrets file: it maps +# employee -> client_id, and this script feeds that id straight to +# `rescope-client`, so whoever can write it decides which client receives a +# given employee's write authority. Treat both as privileged infrastructure, +# at the same trust level as the roster itself. +for f in "$SECRETS_OUT" "$STATE_FILE"; do + if git -C "$(dirname "$f")" rev-parse --is-inside-work-tree >/dev/null 2>&1; then + echo "WARN: $f is inside a git work tree. Never commit it;" >&2 + echo " gitignore it, or pass --secrets-out/--state-file outside the repo." >&2 + fi +done + +# A group/world-writable parent directory defeats the symlink and ownership +# checks below: anyone with write access there can swap the file between our +# check and our append. Refuse rather than pretend the checks hold. +for d in "$(dirname "$SECRETS_OUT")" "$(dirname "$STATE_FILE")"; do + perms=$(ls -ld "$d" | awk '{print $1}') + case "$perms" in + ?????w*|????????w*) die "refusing to write credentials into a group/world-writable directory: $d ($perms)" ;; + esac +done + +# Secure the credential sinks BEFORE anything is appended. umask only governs +# files this script creates; a pre-existing world-readable file would receive +# secrets first and be chmod'ed only afterwards, and a symlink planted at +# either path would redirect them entirely. +for f in "$SECRETS_OUT" "$STATE_FILE"; do + [ -L "$f" ] && die "refusing to write credentials through a symlink: $f" + if [ -e "$f" ]; then + [ -f "$f" ] || die "refusing to write credentials to a non-regular file: $f" + [ -O "$f" ] || die "refusing to write credentials to a file owned by another user: $f" + else + : > "$f" + fi + chmod 600 "$f" +done + +run() { + if [ "$DRY_RUN" = 1 ]; then echo "DRY-RUN: $GBRAIN $*" >&2; return 0; fi + # shellcheck disable=SC2086 — $GBRAIN may carry args ("bun run src/cli.ts") + $GBRAIN "$@" +} + +state_lookup() { # state_lookup -> client_id or empty + [ -f "$STATE_FILE" ] || return 0 + awk -F'\t' -v s="$1" '$1 == s { print $2; exit }' "$STATE_FILE" +} + +# ── Pass 1: parse roster, collect declared channels ───────────────────────── +CHANNELS="" +EMPLOYEES="" +lineno=0 +while IFS= read -r line || [ -n "$line" ]; do + lineno=$((lineno + 1)) + line="${line%%#*}" + line="${line%$'\r'}" # a CRLF roster would otherwise yield 'emp-alice\r/' prefixes that fence everything out + [ -z "${line//[[:space:]]/}" ] && continue + # shellcheck disable=SC2086 — deliberate word split; globbing is off (set -f above) + set -- $line + [ "$#" -le 3 ] || die "roster line $lineno: too many fields ('$line'). Channels are ONE comma-separated field with no spaces: 'employee alice eng,product'" + case "$1" in + channel) + require_slug channel "${2:-}" + CHANNELS="$CHANNELS $2" + ;; + employee) + require_slug employee "${2:-}" + case " $EMPLOYEES " in *" $2:"*) die "roster line $lineno: employee '$2' listed twice" ;; esac + if [ -n "${3:-}" ]; then + for c in ${3//,/ }; do require_slug "channel-reference" "$c"; done + fi + EMPLOYEES="$EMPLOYEES $2:${3:-}" + ;; + *) die "roster line $lineno: unknown entry type '$1' (expected 'channel' or 'employee')" ;; + esac +done < "$ROSTER" + +# ── Pass 2: ensure the shared memory source exists (path-less) ────────────── +if out=$(run sources add "$MEMORY_SOURCE" --name "agent memory ($MEMORY_SOURCE)" 2>&1); then + echo "source '$MEMORY_SOURCE': created" +else + echo "$out" | grep -q "already registered" || die "sources add failed: $out" + echo "source '$MEMORY_SOURCE': already exists" +fi + +# ── Pass 3: converge one client per employee ──────────────────────────────── +FED_READ="$MEMORY_SOURCE${READ_SOURCES:+,$READ_SOURCES}" +new_secrets=0 + +for entry in $EMPLOYEES; do + slug="${entry%%:*}" + chans="${entry#*:}" + + prefixes="emp-$slug/" + if [ -n "$chans" ]; then + for c in ${chans//,/ }; do + echo " $CHANNELS " | grep -q " $c " || echo "WARN: employee '$slug' references undeclared channel '$c'" >&2 + prefixes="$prefixes,chan-$c/" + done + fi + + client_id="$(state_lookup "$slug")" + if [ -n "$client_id" ]; then + # The state file usually sits in the deployment repo, so anyone who can + # edit it could otherwise retarget this privileged rescope at an arbitrary + # client id (e.g. point alice's row at an admin client). Shape-check it. + case "$client_id" in + gbrain_cl_) die "state file: empty client id for '$slug'" ;; + gbrain_cl_*[!a-zA-Z0-9_]*) die "state file: malformed client id for '$slug': $client_id" ;; + gbrain_cl_*) ;; + *) die "state file: client id for '$slug' does not look like a gbrain client: $client_id" ;; + esac + # --source too, so a re-run actually CONVERGES the client to the roster: + # without it, changing --memory-source (or inheriting a state row written + # against an older one) silently leaves the old write source in place + # while the script reports success. + run auth rescope-client "$client_id" --source "$MEMORY_SOURCE" \ + --federated-read "$FED_READ" --bound-slug-prefixes "$prefixes" >/dev/null + echo "employee '$slug': rescoped $client_id [write: $prefixes]" + elif [ "$DRY_RUN" = 1 ]; then + echo "employee '$slug': WOULD register qm-emp-$slug [write: $prefixes] [read: $FED_READ]" + continue + else + out=$(run auth register-client "qm-emp-$slug" \ + --grant-types client_credentials --scopes "read write" \ + --source "$MEMORY_SOURCE" --federated-read "$FED_READ" \ + --bound-slug-prefixes "$prefixes" --budget-usd-per-day "$BUDGET" 2>&1) \ + || die "register-client failed for '$slug' (output withheld: it can contain a secret). Re-run the command by hand to see why." + client_id=$(echo "$out" | sed -n 's/.*Client ID:[[:space:]]*\(gbrain_cl_[^[:space:]]*\).*/\1/p' | head -1) + secret=$(echo "$out" | sed -n 's/.*Client Secret:[[:space:]]*\(gbrain_cs_[^[:space:]]*\).*/\1/p' | head -1) + if [ -z "$client_id" ] || [ -z "$secret" ]; then + # The client may well have been created — dying silently would strand a + # live credential nobody can find. Say so WITHOUT echoing the captured + # output: it contains the freshly minted secret, and this path ends up + # in CI logs. + die "could not parse client id/secret for '$slug' from register-client output (output withheld: it contains a secret). A client MAY have been created; check \`gbrain auth list\` and revoke any stray 'qm-emp-$slug'." + fi + printf '%s\t%s\n' "$slug" "$client_id" >> "$STATE_FILE" + printf '%s\t%s\t%s\n' "$slug" "$client_id" "$secret" >> "$SECRETS_OUT" + chmod 600 "$STATE_FILE" "$SECRETS_OUT" 2>/dev/null || true # umask covers new files; this covers pre-existing ones + new_secrets=$((new_secrets + 1)) + echo "employee '$slug': registered $client_id [write: $prefixes]" + fi +done + +# ── Pass 4: flag offboarded employees ─────────────────────────────────────── +# Removing someone from the roster is the highest-stakes edit there is, and +# this script cannot safely revoke on its own (a typo'd roster would nuke live +# credentials). Report instead, with the exact command. +if [ -f "$STATE_FILE" ]; then + while IFS=$'\t' read -r st_slug st_client _rest; do + [ -n "${st_slug:-}" ] || continue + case " $EMPLOYEES " in + *" $st_slug:"*) ;; + *) echo "STALE: '$st_slug' ($st_client) is no longer in the roster but its credentials still work." >&2 + echo " Revoke with: $GBRAIN auth revoke-client $st_client" >&2 ;; + esac + done < "$STATE_FILE" +fi + +echo +echo "Done. State: $STATE_FILE" +if [ "$new_secrets" -gt 0 ]; then + echo "$new_secrets NEW client secret(s) written to $SECRETS_OUT — deliver to each scope's sandbox, then DELETE the file." +fi diff --git a/docs/integrations/qm-harness-snippets/roster.example.tsv b/docs/integrations/qm-harness-snippets/roster.example.tsv new file mode 100644 index 000000000..c00bfec1c --- /dev/null +++ b/docs/integrations/qm-harness-snippets/roster.example.tsv @@ -0,0 +1,10 @@ +# Roster for provision-scopes.sh — one line per channel / employee. +# channel +# employee [comma-separated channels they belong to] + +channel eng +channel product + +employee alice-example eng,product +employee bob-example eng +employee carol-example diff --git a/docs/integrations/qm-harness-snippets/tool.json b/docs/integrations/qm-harness-snippets/tool.json new file mode 100644 index 000000000..2beb50f7c --- /dev/null +++ b/docs/integrations/qm-harness-snippets/tool.json @@ -0,0 +1,19 @@ +{ + "id": "gbrain", + "label": "gbrain company brain", + "advertise": "gbrain", + "hints": [ + "Company knowledge brain: searchable, cross-source, persistent.", + "Search it BEFORE answering questions about the org, people, projects, decisions, or history: `gbrain search \"\"`.", + "Write durable knowledge with `gbrain put --content ...`, only under your own slug prefixes.", + "See the gbrain skill for slug conventions and first-run setup." + ], + "auth": { + "check": "gbrain whoami", + "reauth": "gbrain init --mcp-only --force --issuer-url \"$GBRAIN_ISSUER_URL\" --mcp-url \"$GBRAIN_MCP_URL\" --oauth-client-id \"$GBRAIN_CLIENT_ID\" --oauth-client-secret \"$GBRAIN_CLIENT_SECRET\"", + "credentialPaths": [ + { "path": ".gbrain/config.json", "kind": "file" } + ] + }, + "install": { "binary": "gbrain" } +} diff --git a/docs/integrations/qm-harness.md b/docs/integrations/qm-harness.md new file mode 100644 index 000000000..cdf785b0d --- /dev/null +++ b/docs/integrations/qm-harness.md @@ -0,0 +1,224 @@ +# qm (multi-user agent harness) — gbrain as the company brain + +Connect gbrain to [qm](https://github.com/yc-software/qm) — the multiplayer +agent harness where each employee and each channel gets an isolated agent +scope — so every scope's agent can search and write one shared, indexed, +isolation-enforced company brain. The same recipe fits any harness with +per-person sandboxes that can run a CLI. + +**Shape:** one central `gbrain serve --http` (OAuth 2.1) next to qm's core; +the `gbrain` binary baked into qm's sandbox image as a thin client; one OAuth +client per employee, read-fenced by source federation and write-fenced by +`bound_slug_prefixes`. Zero qm code changes — everything lives in the qm +*deployment directory*. + +qm's native memory (per-scope notebook) stays as-is for fast per-turn recall. +gbrain adds what qm doesn't have: semantic + hybrid search, cross-scope +knowledge, entity graphs, and durable memory that outlives a scope. + +## Topology + +| gbrain concept | qm concept | +|---|---| +| one brain (one Postgres/Supabase DB) | the org | +| source `agents` (path-less, shared) | all agent-written memory | +| slug prefix `emp-/` in `agents` | an employee's personal scope | +| slug prefix `chan-/` in `agents` | a channel/room scope | +| source `org-wiki` (git-backed, read-only) | company docs | +| OAuth client `qm-emp-` | one employee's agent identity | + +Isolation model: + +- **Reads** are source-granular, SQL-enforced (`federated_read`): every + employee client reads `agents` + the read-only sources you grant. +- **Writes** are slug-prefix-granular, server-enforced (`bound_slug_prefixes`, + v0.42.72.0+): a client can only mutate pages under its own `emp-/` + and its channels' `chan-/` prefixes — on `put_page`, `delete_page`, + `restore_page`, `add_tag`, `remove_tag`, `add_link`/`remove_link`, + `add_timeline_entry`, `revert_version` and `put_raw_data`, plus the + `POST /ingest` webhook route. Not by convention. +- **Every op that is not a plain read is denied unless allow-listed.** Ops + that write by a key other than a slug — `extract_entities` and + `extract_facts` (which mutate `people/*` and `companies/*`), `forget_fact` + (targets a fact by numeric id, across sources), `ontology_propose`, and the + `sources_admin` pair `sources_add`/`sources_remove` — cannot be fenced by + slug, so a bound client gets `permission_denied` at dispatch. The gate keys + on "not a pure read", not on a list of scope strings, so a write op added + later (or one carrying a bespoke scope) is denied until it is explicitly + fenced and added to `CLIENT_FENCED_WRITE_OPS` (`src/core/operations.ts`). + `think` is allow-listed because remote callers cannot persist from it; + `submit_agent` because it enforces this same column itself. +- **Indirect write paths are gated too, not just the ops.** `put_page`'s + facts backstop would otherwise extract entities from the page body and + write fact rows (and a `## Facts` fence on git-backed sources) onto + `people/*` pages the caller never named — the same capability + `extract_facts` is denied for, reached through an in-prefix write. It is + skipped for bound clients. `POST /ingest` is refused outright: its handler + bypasses the op layer *and* discards the source grant for untrusted + payloads, so it would write into the `default` source. +### Known limitations — read these before you rely on the fence + +The write fence is a **write** boundary within a source. It is not a privacy +boundary, and it does not make every side effect prefix-clean. As of +v0.42.72.0: + +- **`add_link`/`remove_link` fence the `from` endpoint only.** A bound client + can create an edge pointing AT a page it cannot write; the edge's `context` + text surfaces in that page's backlinks and contributes to its search + ranking. Fencing `to` would break legitimate cross-referencing into + `org-wiki`, so this is deliberate — treat inbound-edge context as untrusted + content, the same way you treat page bodies. +- **Reads are source-granular, never prefix-granular.** Everyone entitled to + a source can read every prefix in it. If a scope needs genuine read + privacy, give it its own source. +- **`put_page` can create one reverse graph edge outside the fence.** If a + page body cites a code location (`src/x.ts:42`) and a code page for it + exists *in the same source*, doc↔impl reconciliation adds an edge + originating from that code page. It affects graph/backlink ranking, not + page content. Unreachable in the layout above (the `agents` source is + path-less and holds no code pages); it applies only if you point employee + writes at a code-synced source. +- **A few read ops are still brain-wide** and ignore the federated grant: + `get_recent_salience`, `find_anomalies`, `find_contradictions`, and + `sources_list`/`sources_status` (which expose source ids, paths and URLs). + A read-scoped client can learn facts derived from sources it was not + granted. Pre-existing, not introduced by the fence; if that matters for + your deployment, withhold those tools at the harness layer for now. +- **Reads touch `last_retrieved_at`** on the pages they return, including + pages in read-only sources. Freshness/usage signals are therefore + writable-by-reading; nothing else about the page is. +- **`POST /ingest` writes land in the `default` source** regardless of the + calling client's `source_id`, because the handler discards the source for + untrusted payloads. Bound clients are refused the route outright for this + reason; if you point a webhook integration at it, scope that brain's + `default` source deliberately. +- **Tradeoff to state out loud:** read isolation is per-source, so within the + shared `agents` source every employee can *read* every prefix (including + other employees' `emp-*/`). That matches qm's transparent-by-default, + everything-audited posture. If you need hard read privacy for personal + memory, give those employees their own write source instead of a prefix + (one `sources add emp-` + `--source emp-` per client) and keep + channel prefixes in `agents` via a second, channels-only client — at the + cost of two credentials in that sandbox. + +## Host setup (the machine running qm's core, or any box its sandboxes can reach) + +```bash +# 1. Engine: Postgres/Supabase. PGLite is single-process and cannot serve +# many concurrent sandboxes. +gbrain init --supabase --embedding-model voyage:voyage-4-large + +# 2. Modes + gates (publish_* default OFF and fail as silent 403s): +gbrain config set search.mode balanced +gbrain config set mcp.publish_skills true +gbrain config set mcp.publish_advisor true + +# 3. Read-only org sources + first sync: +gbrain sources add org-wiki --path ~/brains/org-wiki +gbrain sync --all # cron this + +# 4. Serve over HTTP MCP (OAuth 2.1): +gbrain serve --http --bind 0.0.0.0 --port 3131 \ + --public-url https://brain.acme-example.com +``` + +Never hand sandboxes `DATABASE_URL` — direct DB access bypasses OAuth, source +federation, and the write fence entirely. + +## Provision scopes from a roster + +[`qm-harness-snippets/provision-scopes.sh`](qm-harness-snippets/provision-scopes.sh) +converges the brain to a roster file +([`roster.example.tsv`](qm-harness-snippets/roster.example.tsv)): + +```bash +bash provision-scopes.sh roster.tsv --read-sources org-wiki +``` + +- Creates the path-less `agents` source (agent-written memory needs no git + clone; if the host has `sync.repo_path` configured, pages also write + through to `.sources/agents/` for git-backed durability). +- Registers `qm-emp-` clients: `--scopes "read write"`, + `--source agents`, `--federated-read agents,org-wiki`, + `--bound-slug-prefixes emp-/,chan-/,...`, per-day budget. +- **Idempotent:** re-run after every roster edit; existing clients are + `rescope-client`ed in place (channel joins/leaves update the write fence + without rotating secrets). +- New client secrets land once in `.new-credentials.tsv` — deliver + each row to its scope (qm keychain / one-time secret drop), then delete + the file. + +## qm deployment directory + +In the org's qm deployment repo (the directory `qm init` produced): + +1. **Tool:** copy [`qm-harness-snippets/tool.json`](qm-harness-snippets/tool.json) + to `sandbox/tools/gbrain/tool.json` and drop the compiled `gbrain` binary + beside it (`bun build --compile --outfile gbrain src/cli.ts`, built for + the sandbox image's OS/arch). `auth.credentialPaths` marks + `~/.gbrain/config.json` as the scope's resident credential file; + `auth.check` wires `gbrain whoami` into qm's connector status (read-scope; + see the note below on why `remote doctor` cannot be used here). +2. **Skill:** copy [`qm-harness-snippets/SKILL.md`](qm-harness-snippets/SKILL.md) + to `sandbox/skills/gbrain/SKILL.md` (edit slug conventions to taste). +3. Ship it: `qm sandbox build && qm sandbox publish && qm up`. + +Per scope, one-time (agent- or operator-run, credentials from the handoff): + +```bash +gbrain init --mcp-only \ + --issuer-url https://brain.acme-example.com \ + --mcp-url https://brain.acme-example.com/mcp \ + --oauth-client-id gbrain_cl_... --oauth-client-secret gbrain_cs_... +gbrain whoami # must succeed +``` + +Use `--oauth-client-secret`, not `GBRAIN_REMOTE_CLIENT_SECRET`: an env-sourced +secret is deliberately not written to `~/.gbrain/config.json` +(`src/commands/init.ts`), so with the env var alone every later command fails +once it leaves scope — and qm's `sandbox.secretEnv` is org-wide, so there is no +per-scope env to keep it in. With the flag, the credential lands in the config +file on the scope's durable disk and this runs once per scope, ever. + +`gbrain remote doctor` is **not** the health check here: `run_doctor` is an +`admin`-scope op and these clients are `read write` on purpose. `gbrain whoami` +is read-scope and reports the client's identity, source, and grants. + +## Verify isolation before rollout + +From two differently-scoped sandboxes (or two thin-client configs): + +```bash +# alice-example (bound to emp-alice-example/, chan-eng/): +gbrain put emp-alice-example/notes/test --content "mine" # OK +gbrain put chan-eng/notes/test --content "shared" # OK +gbrain put emp-bob-example/notes/test --content "not mine" # permission_denied +gbrain put chan-product/notes/test --content "not my channel" # permission_denied +gbrain search "test" # sees agents + org-wiki only +``` + +## Cost + operations + +- `search.mode balanced` (12K token budget, relational retrieval on) is the + right default for a startup fleet; see `docs/guides/search-modes.md` for + the cost matrix before changing it. +- Budgets: `--budget-usd-per-day` is recorded on the client but only enforced + on the `submit_agent` path (`src/core/minions/budget-meter.ts`), which these + `read write` clients cannot reach — so it does **not** cap spend from + ordinary `search`/`put_page` traffic. Treat runaway-agent containment as an + open item: watch the admin SPA (`/admin`) and `gbrain search stats`, and cap + at the model/harness layer. +- Backfills on a live brain: `gbrain embed --stale --pace` (see Pace Mode in + CLAUDE.md / `docs/operations/spend-controls.md`). + +## Deliberately deferred + +- **qm `MemoryService` decorator** (mirror notebook captures into gbrain, + fan `recall` out and merge, `volunteer_context` push): needs a qm code + change; today's integration is agent-initiated via the CLI + skill. +- **MCP-native attach:** qm pins `strictMcpConfig` with only its in-process + server, so gbrain's MCP-discovered brain-resident skillpacks don't reach + qm agents; the sandbox skill above covers it. +- **Read-side prefix fencing** (hard privacy for `emp-*/` inside a shared + source) — tracked upstream; the roster layout is forward-compatible with + it. diff --git a/docs/tutorials/company-brain.md b/docs/tutorials/company-brain.md index 6f4dfcd76..537a3ed3f 100644 --- a/docs/tutorials/company-brain.md +++ b/docs/tutorials/company-brain.md @@ -93,7 +93,7 @@ There are two ways to scope teammates' access. They suit different deployment sh **Model A: separate sources with OAuth scoping (recommended for true multi-user with different AI clients).** What this tutorial walks you through. Each teammate gets their own OAuth client, which carries `--source` + `--federated-read` flags. The brain refuses cross-source reads at the SQL layer; isolation is database-enforced. Each teammate can run their own MCP-aware client (Claude Code, Cursor, their own OpenClaw, etc.) and the scoping holds. -**Model B: one source, directory-based per-person scoping (simpler for one-agent-serves-everyone setups).** The shape I actually run in production: a single source called `default`, with a `partners//` convention inside it (e.g. `partners/alice-example/`, `partners/bob-example/`). Each partner gets their own subdirectory holding their personal pages: `partners/alice-example/USER.md`, `partners/alice-example/concepts/`, `partners/alice-example/sources/`, etc. There's no OAuth-enforced isolation; the agent itself enforces "Alice's writes go to her partners/ subdir." This is the right model when ONE agent (yours) serves everyone over Telegram or a single shared interface. It's simpler ops, no per-user OAuth, but the scoping is convention-only. +**Model B: one source, directory-based per-person scoping (simpler for one-agent-serves-everyone setups).** The shape I actually run in production: a single source called `default`, with a `partners//` convention inside it (e.g. `partners/alice-example/`, `partners/bob-example/`). Each partner gets their own subdirectory holding their personal pages: `partners/alice-example/USER.md`, `partners/alice-example/concepts/`, `partners/alice-example/sources/`, etc. This is the right model when ONE agent (yours) serves everyone over Telegram or a single shared interface. It's simpler ops, no per-user OAuth. **Write scoping within the shared source can be server-enforced:** register each per-person client with `--bound-slug-prefixes partners/alice-example/` and every slug-mutating write outside that prefix is rejected with `permission_denied` (v0.42.72.0+). Without the binding, the scoping is convention-only (the agent polices itself). Read scoping stays source-granular in both models — within a shared source, everyone entitled to the source can read every folder. For most company-brain installs (10+ teammates each with their own AI client), Model A is the right starting point. If you're running the fat-agent-serves-everyone pattern from the personal-brain tutorial, Model B is genuinely simpler. You can also mix: separate sources for the obviously-different ones (customer notes vs internal-only) AND a `partners//` convention inside the shared source for per-person workspace. @@ -210,7 +210,7 @@ Each `register-client` command prints a `client_id` and a `client_secret`. Save A note on the flags: - `--scopes read,write` lets the client query the brain and write new pages. You can omit `write` for read-only clients (executive summaries, dashboards). The `admin` scope is needed for operational commands like `gbrain remote doctor` and is usually reserved for your own admin client. -- `--source` controls write authority. A client can only write to one source. Within that source, your folder convention from Part 3 keeps each person's writes in their own subfolder. +- `--source` controls write authority. A client can only write to one source. Within that source, your folder convention from Part 3 keeps each person's writes in their own subfolder — and you can make that server-enforced with `--bound-slug-prefixes alice-example/` (v0.42.72.0+): every slug-mutating write op (put_page, delete_page, tags, links, timeline, revert, raw data) outside the bound prefixes is rejected with `permission_denied`. Update the binding later with `gbrain auth rescope-client --bound-slug-prefixes `. **Adding a binding to an existing client narrows it in ways you should expect:** ops that write by something other than a slug (`extract_entities`, `extract_facts`, `forget_fact`, `ontology_propose`, `sources_add`/`sources_remove`) and `POST /ingest` become unavailable to that client, and `put_page`'s automatic fact extraction is skipped — all because none of them can be confined to a prefix. Reads are unaffected. See [the qm-harness guide](../integrations/qm-harness.md) for the full model. - `--federated-read` controls read scope. A client can read from one or more sources. ### Verify the scoping actually scopes diff --git a/package.json b/package.json index 296396893..98d3a560c 100644 --- a/package.json +++ b/package.json @@ -148,7 +148,7 @@ "bun": ">=1.3.10" }, "license": "MIT", - "version": "0.42.71.0", + "version": "0.42.72.0", "overrides": { "@hono/node-server": "^2.0.5", "fast-uri": "^3.1.4", diff --git a/src/commands/auth.ts b/src/commands/auth.ts index af759b576..ce972bc41 100644 --- a/src/commands/auth.ts +++ b/src/commands/auth.ts @@ -524,13 +524,17 @@ async function registerClient(name: string, args: string[]) { * /admin/api/rescope-client endpoint. */ async function rescopeClient(clientId: string, args: string[]) { - const usage = 'Usage: auth rescope-client [--source SOURCE] [--federated-read SRC1,SRC2,...]'; + const usage = 'Usage: auth rescope-client [--source SOURCE] [--federated-read SRC1,SRC2,...] [--bound-slug-prefixes P1,P2|none]'; if (!clientId) { console.error(usage); process.exit(1); } let sourceId: string | undefined; let federatedRead: string[] | undefined; + // v0.42.72.0: tri-state — undefined = untouched, null = clear ('none'), + // array = replace. Lets roster churn (channel joins/leaves) update the + // write fence in place instead of register+rotate. + let boundSlugPrefixes: string[] | null | undefined; for (let i = 0; i < args.length; i += 2) { const flag = args[i]; const value = args[i + 1]; @@ -542,14 +546,18 @@ async function rescopeClient(clientId: string, args: string[]) { if (flag === '--source') sourceId = value; else if (flag === '--federated-read') { federatedRead = value.split(',').map(s => s.trim()).filter(Boolean); + } else if (flag === '--bound-slug-prefixes') { + boundSlugPrefixes = value === 'none' + ? null + : value.split(',').map(s => s.trim()).filter(Boolean); } else { console.error(`Error: Unknown flag: ${flag}`); console.error(usage); process.exit(1); } } - if (sourceId === undefined && federatedRead === undefined) { - console.error('Error: pass --source and/or --federated-read'); + if (sourceId === undefined && federatedRead === undefined && boundSlugPrefixes === undefined) { + console.error('Error: pass --source, --federated-read, and/or --bound-slug-prefixes'); console.error(usage); process.exit(1); } @@ -557,10 +565,13 @@ async function rescopeClient(clientId: string, args: string[]) { await withConfiguredSql(async (sql) => { const { GBrainOAuthProvider } = await import('../core/oauth-provider.ts'); const provider = new GBrainOAuthProvider({ sql }); - const result = await provider.rescopeClient(clientId, { sourceId, federatedRead }); + const result = await provider.rescopeClient(clientId, { sourceId, federatedRead, boundSlugPrefixes }); console.log(`OAuth client rescoped: "${result.clientName}" (${result.clientId})\n`); console.log(` Write source: ${result.sourceId}`); console.log(` Federated reads: ${result.federatedRead.join(', ') || ''}`); + if (result.boundSlugPrefixes !== undefined) { + console.log(` Bound slug prefixes: ${result.boundSlugPrefixes?.join(', ') ?? ''}`); + } console.log('\nTakes effect on the client\'s next request (existing tokens included).'); }); } catch (e: any) { @@ -645,14 +656,22 @@ Usage: --bound-tools Bind submit_agent to an allow-list of tools --bound-source Bind submit_agent jobs to a source id --bound-brain Bind submit_agent jobs to a brain id - --bound-slug-prefixes Bind submit_agent writes to slug prefixes + --bound-slug-prefixes Fence ALL direct slug writes (put_page, delete_page, + tags, links, timeline, revert, raw data) AND + submit_agent to these prefixes. Each MUST end with + '/' or '/*' — a boundary-less 'emp-alice' would also + name 'emp-alice-2/...'. Ops that write by something + other than a slug (extract_*, forget_fact, + ontology_propose, sources_*) and POST /ingest become + unavailable to a bound client. Omit = full-source writes. --bound-max-concurrent Bound submit_agent concurrency (default: 1) --budget-usd-per-day Bound submit_agent daily spend cap gbrain auth rescope-client [options] Change an existing client's source scope (e.g. a DCR client stuck on the 'default' source). Only the flags - you pass change; the other axis is left as-is. + you pass change; the other axes are left as-is. --source New write source --federated-read New read-scope source list + --bound-slug-prefixes Replace the slug-prefix write fence ('none' clears it) gbrain auth revoke-client Hard-delete an OAuth 2.1 client (cascades to tokens + codes) gbrain auth test --token Smoke-test a remote MCP server `); diff --git a/src/commands/serve-http.ts b/src/commands/serve-http.ts index a0e6ed314..4940fbb88 100644 --- a/src/commands/serve-http.ts +++ b/src/commands/serve-http.ts @@ -1717,7 +1717,7 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption // validator inside rescopeClient. app.post('/admin/api/rescope-client', requireAdmin, express.json(), async (req: Request, res: Response) => { try { - const { clientId, sourceId, federatedRead } = req.body ?? {}; + const { clientId, sourceId, federatedRead, boundSlugPrefixes } = req.body ?? {}; if (!clientId || typeof clientId !== 'string') { res.status(400).json({ error: 'clientId required' }); return; @@ -1731,12 +1731,20 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption res.status(400).json({ error: 'sourceId must be a string' }); return; } - const result = await oauthProvider.rescopeClient(clientId, { sourceId, federatedRead }); + // v0.42.72.0: tri-state write-fence rescope — omitted = untouched, + // null = clear, array of strings = replace (mirrors the CLI's + // --bound-slug-prefixes p1,p2|none). + if (boundSlugPrefixes !== undefined && boundSlugPrefixes !== null && + !(Array.isArray(boundSlugPrefixes) && boundSlugPrefixes.every((s: unknown) => typeof s === 'string'))) { + res.status(400).json({ error: 'boundSlugPrefixes must be null or an array of slug-prefix strings' }); + return; + } + const result = await oauthProvider.rescopeClient(clientId, { sourceId, federatedRead, boundSlugPrefixes }); res.json(result); } catch (e) { const message = e instanceof Error ? e.message : 'Rescope failed'; const status = /No OAuth client found/.test(message) ? 404 - : /Invalid source_id|requires --source|cannot be empty|does not exist/.test(message) ? 400 + : /Invalid source_id|requires --source|cannot be empty|does not exist|cannot be an empty list|bound_slug_prefixes entr/.test(message) ? 400 : 500; res.status(status).json({ error: message }); } @@ -2278,6 +2286,31 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption const sourceId = (req.header('x-gbrain-source-id') || `webhook-${authInfo.clientId}`).slice(0, 256); const callerSlug = req.header('x-gbrain-slug'); + // Slug-bound clients cannot use /ingest at all. The route hands its + // payload to the ingest_capture minion handler, which deliberately + // bypasses the put_page op layer — so no OperationContext exists and + // enforceClientSlugFence never runs, and because the payload is marked + // untrusted the handler also refuses to honor any source id, landing + // every write in the DEFAULT source. Fencing just the slug here would + // still write the right slug into the WRONG source, outside the + // client's grant. These clients have put_page over MCP, which enforces + // both the prefix fence and the source scope; webhook integrations use + // unbound clients. + const boundPrefixes = authInfo.boundSlugPrefixes; + if (boundPrefixes || authInfo.fenceProjectionDegraded) { + res.status(403).json({ + error: 'permission_denied', + message: authInfo.fenceProjectionDegraded + ? 'POST /ingest is unavailable: this brain\'s oauth_clients projection is missing ' + + 'bound_slug_prefixes, so client write bindings cannot be evaluated. ' + + 'Run `gbrain apply-migrations --yes` on the brain host.' + : 'POST /ingest is not available to clients restricted to slug prefixes ' + + `(bound_slug_prefixes: ${boundPrefixes!.join(', ')}). Write through the MCP put_page op, ` + + 'which enforces the prefix fence and your source scope.', + }); + return; + } + const event: IngestionEvent = { source_id: sourceId, source_kind: 'webhook', diff --git a/src/core/oauth-provider.ts b/src/core/oauth-provider.ts index 17f383a7b..1979ff185 100644 --- a/src/core/oauth-provider.ts +++ b/src/core/oauth-provider.ts @@ -28,6 +28,38 @@ import { assertValidSourceId } from './source-id.ts'; import { hasScope, assertAllowedScopes, parseScopeString, InvalidScopeError } from './scope.ts'; import type { AuthInfo as CoreAuthInfo } from './operations.ts'; import { parseLegacyTokenScope } from './legacy-token-scope.ts'; + +/** + * A slug-prefix write binding is only meaningful if every entry actually + * constrains something. `''` (or whitespace) matches every slug under + * `startsWith`, so one unset variable in a provisioning template would turn + * a binding into a silent wildcard while still displaying as "fenced". + * Reject at every write surface: registration, rescope, admin API. + */ +export function assertValidSlugPrefixes(prefixes: readonly string[]): void { + for (const p of prefixes) { + if (typeof p !== 'string' || p.trim() === '') { + throw new Error('bound_slug_prefixes entries must be non-empty, non-whitespace slug prefixes (e.g. "emp-alice/")'); + } + if (p !== p.trim()) { + throw new Error(`bound_slug_prefixes entry "${p}" has leading/trailing whitespace; slugs never do, so it would fence nothing`); + } + // Slugs are lowercased by validateSlug before storage, so a prefix with + // uppercase in it cannot correspond to anything actually written. + if (p !== p.toLowerCase()) { + throw new Error(`bound_slug_prefixes entry "${p}" must be lowercase; stored slugs are lowercased, so a mixed-case prefix fences unpredictably`); + } + // Require an explicit segment boundary. Slug namespaces collide on their + // own naming scheme — `emp-alice` and `emp-alice-2` are different people — + // and a boundary-less entry reads as "everything starting with these + // characters". The matcher is boundary-aware regardless, but saying it at + // registration is what stops an operator writing a binding whose meaning + // isn't what it looks like. + if (!p.endsWith('/') && !p.endsWith('/*')) { + throw new Error(`bound_slug_prefixes entry "${p}" must end with "/" (or "/*"); a boundary-less prefix reads as a character prefix, so "${p}" would look like it covers only "${p}/..." while naming sibling namespaces like "${p}-2/..."`); + } + } +} import type { SqlQuery, SqlValue } from './sql-query.ts'; export type { SqlQuery, SqlValue }; @@ -606,41 +638,61 @@ export class GBrainOAuthProvider implements OAuthServerProvider { try { oauthRows = await this.sql` SELECT t.client_id, t.scopes, t.expires_at, t.resource, c.client_name, - c.source_id, c.federated_read + c.source_id, c.federated_read, c.bound_slug_prefixes FROM oauth_tokens t LEFT JOIN oauth_clients c ON c.client_id = t.client_id WHERE t.token_hash = ${tokenHash} AND t.token_type = 'access' `; } catch (err) { - // v0.34.1: pre-v60 brain → source_id column missing. Pre-v61 brain → - // federated_read column missing. Both classes degrade to legacy - // projection so auth keeps working until the operator runs - // apply-migrations. Probe both column names so partial-upgrade brains - // (v60 applied but v61 didn't yet) also fall through cleanly. - if (isUndefinedColumnError(err, 'source_id') || isUndefinedColumnError(err, 'federated_read')) { - // Try the v60-only projection first (source_id but no federated_read). + // Degrade ladder for brains that haven't run apply-migrations yet: + // bound_slug_prefixes (v85) → federated_read (v61) → source_id (v60) → + // pre-v0.34 base projection. Auth must keep working the whole way down. + // + // `isUndefinedColumnError(err, name)` canNOT actually tell us WHICH + // column was missing — with SQLSTATE 42703 present it returns true for + // any undefined column, and the name is only consulted in the message + // fallback. So the ladder must not branch on the reported name; it + // walks every narrower projection in turn, each guarded, and only + // rethrows once the narrowest one still fails. (Branching on the name + // is what made the first cut of this hard-fail every token + // verification on a pre-v61 brain.) + // Any of the three optional columns may be the missing one, and on the + // message-fallback path (drivers that don't surface SQLSTATE) the name + // is what identifies it — so probe all three at every rung. + const missingOAuthColumn = (e: unknown): boolean => + isUndefinedColumnError(e, 'bound_slug_prefixes') || + isUndefinedColumnError(e, 'federated_read') || + isUndefinedColumnError(e, 'source_id'); + if (!missingOAuthColumn(err)) throw err; + try { + // v85 missing: keep source_id + federated_read, drop the fence column. + oauthRows = await this.sql` + SELECT t.client_id, t.scopes, t.expires_at, t.resource, c.client_name, + c.source_id, c.federated_read + FROM oauth_tokens t + LEFT JOIN oauth_clients c ON c.client_id = t.client_id + WHERE t.token_hash = ${tokenHash} AND t.token_type = 'access' + `; + } catch (err2) { + if (!missingOAuthColumn(err2)) throw err2; try { + // v61 missing: source_id only. oauthRows = await this.sql` SELECT t.client_id, t.scopes, t.expires_at, t.resource, c.client_name, c.source_id FROM oauth_tokens t LEFT JOIN oauth_clients c ON c.client_id = t.client_id WHERE t.token_hash = ${tokenHash} AND t.token_type = 'access' `; - } catch (err2) { - if (isUndefinedColumnError(err2, 'source_id')) { - // Truly pre-v60: no source_id either. Pre-v0.34 projection. - oauthRows = await this.sql` - SELECT t.client_id, t.scopes, t.expires_at, t.resource, c.client_name - FROM oauth_tokens t - LEFT JOIN oauth_clients c ON c.client_id = t.client_id - WHERE t.token_hash = ${tokenHash} AND t.token_type = 'access' - `; - } else { - throw err2; - } + } catch (err3) { + if (!missingOAuthColumn(err3)) throw err3; + // Truly pre-v60: pre-v0.34 projection. + oauthRows = await this.sql` + SELECT t.client_id, t.scopes, t.expires_at, t.resource, c.client_name + FROM oauth_tokens t + LEFT JOIN oauth_clients c ON c.client_id = t.client_id + WHERE t.token_hash = ${tokenHash} AND t.token_type = 'access' + `; } - } else { - throw err; } } @@ -659,9 +711,39 @@ export class GBrainOAuthProvider implements OAuthServerProvider { // array vs undefined matters: empty array = explicit no-federated- // read; undefined = column missing on this brain. const federatedRaw = row.federated_read; - const allowedSources = Array.isArray(federatedRaw) + const rowSourceId = (row.source_id as string | null) ?? undefined; + let allowedSources = Array.isArray(federatedRaw) ? (federatedRaw as string[]) : undefined; + // Degraded-projection safety: `resolveRequestedScope` only authorizes an + // explicitly requested `source_id` when `allowedSources` is a NON-EMPTY + // array — with it undefined, a remote caller naming any source is + // accepted. On a brain missing `federated_read` the ladder above returns + // exactly that undefined, so a client scoped to one source could read + // every other source by passing `source_id`. Synthesize the client's own + // source as its grant so the authorization check stays armed. (Legacy + // `access_tokens` keep their historical scope handling below — this only + // covers the OAuth rows whose column we just dropped.) + if (allowedSources === undefined && rowSourceId !== undefined) { + allowedSources = [rowSourceId]; + } + // v0.42.72.0: slug-prefix write binding. Array (even empty — the + // fence treats [] as deny-all, matching submit_agent's fail-closed + // posture) when the client carries a binding; undefined when the + // column is NULL, the projection degraded, or the brain predates + // the column. + const boundRaw = row.bound_slug_prefixes; + const boundSlugPrefixes = Array.isArray(boundRaw) + ? (boundRaw as string[]) + : undefined; + // Fail CLOSED on the fence axis. If the projection degraded, we do not + // know whether this client carries a binding, and "column absent" is + // indistinguishable from "no binding" downstream. On a genuinely + // pre-v85 brain no binding can exist and this is harmless; the case + // that matters is a partially broken schema (interrupted migration, + // restored dump missing one column) where bindings DO exist and every + // bound client would otherwise be silently unfenced. + const fenceProjectionDegraded = !('bound_slug_prefixes' in row); return { token, clientId: row.client_id as string, @@ -672,11 +754,15 @@ export class GBrainOAuthProvider implements OAuthServerProvider { // v0.34.1 (#861, D2): source-isolation scope from oauth_clients. // Undefined when the row predates v60 or when the brain itself // predates v60 (fell through to the legacy projection above). - sourceId: (row.source_id as string | null) ?? undefined, + sourceId: rowSourceId, // v0.34.1 (#876): federated read scope. sourceScopeOpts in // operations.ts prefers this array over scalar sourceId when set // and non-empty. allowedSources, + // v0.42.72.0: write fence — consumed by enforceClientSlugFence in + // operations.ts on every direct slug-mutating write op. + boundSlugPrefixes, + ...(fenceProjectionDegraded ? { fenceProjectionDegraded: true } : {}), } as CoreAuthInfo as SdkAuthInfo; } @@ -903,6 +989,20 @@ export class GBrainOAuthProvider implements OAuthServerProvider { // existing rows aren't re-validated). assertAllowedScopes(parseScopeString(scopes)); + // A bound_slug_prefixes entry that is empty or whitespace-only makes + // `startsWith` true for every slug — a binding that looks set in + // `auth list` and the admin UI while fencing nothing. Reject at + // registration, the same way source ids are validated. + if (agentBindings?.boundSlugPrefixes) { + // Same rule as rescopeClient: an empty list is ambiguous. It registers + // as deny-all for every direct write while printing an empty binding + // line, so an operator cannot tell it from an unbound client. + if (agentBindings.boundSlugPrefixes.length === 0) { + throw new Error('--bound-slug-prefixes cannot be an empty list (pass prefixes, or omit the flag for full-source write authority)'); + } + assertValidSlugPrefixes(agentBindings.boundSlugPrefixes); + } + // v0.41.3 (T1+T2): validate token_endpoint_auth_method at the registration // boundary. Throws InvalidTokenEndpointAuthMethodError on bad input. // Default is `client_secret_post` (RFC 7591 §2). @@ -1022,11 +1122,11 @@ export class GBrainOAuthProvider implements OAuthServerProvider { */ async rescopeClient( clientId: string, - opts: { sourceId?: string; federatedRead?: string[] }, - ): Promise<{ clientId: string; clientName: string; sourceId: string; federatedRead: string[] }> { - const { sourceId, federatedRead } = opts; - if (sourceId === undefined && federatedRead === undefined) { - throw new Error('rescope-client requires --source and/or --federated-read'); + opts: { sourceId?: string; federatedRead?: string[]; boundSlugPrefixes?: string[] | null }, + ): Promise<{ clientId: string; clientName: string; sourceId: string; federatedRead: string[]; boundSlugPrefixes?: string[] | null }> { + const { sourceId, federatedRead, boundSlugPrefixes } = opts; + if (sourceId === undefined && federatedRead === undefined && boundSlugPrefixes === undefined) { + throw new Error('rescope-client requires --source, --federated-read, and/or --bound-slug-prefixes'); } if (sourceId !== undefined) assertValidSourceId(sourceId); if (federatedRead !== undefined) { @@ -1035,17 +1135,48 @@ export class GBrainOAuthProvider implements OAuthServerProvider { } for (const s of federatedRead) assertValidSourceId(s); } + // v0.42.72.0: bound_slug_prefixes rescope, so channel-membership churn + // (the qm-harness roster case) updates the write fence in place instead + // of forcing a register+rotate cycle. Tri-state: undefined = untouched, + // null = clear the binding (client returns to unbound full-source write + // authority), non-empty array = replace. Empty array is rejected here — + // it means deny-all at the fence, which an operator should express by + // revoking write scope, not by an ambiguous empty list. + if (Array.isArray(boundSlugPrefixes)) { + if (boundSlugPrefixes.length === 0) { + throw new Error('--bound-slug-prefixes cannot be an empty list (pass prefixes, or "none" to clear the binding)'); + } + assertValidSlugPrefixes(boundSlugPrefixes); + } let rows: Record[]; try { - rows = await this.sql` - UPDATE oauth_clients - SET source_id = COALESCE(${sourceId ?? null}::text, source_id), - federated_read = COALESCE(${federatedRead ? pgArray(federatedRead) : null}::text[], federated_read) - WHERE client_id = ${clientId} - RETURNING client_id, client_name, source_id, federated_read - `; + // Only touch bound_slug_prefixes when the caller actually passed it. + // Naming the column unconditionally would make a plain + // `rescope-client --source wiki` fail on a brain that has the v60/v61 + // OAuth columns but not v85's bound_* set — a regression on an axis + // the caller never asked about. + rows = boundSlugPrefixes === undefined + ? await this.sql` + UPDATE oauth_clients + SET source_id = COALESCE(${sourceId ?? null}::text, source_id), + federated_read = COALESCE(${federatedRead ? pgArray(federatedRead) : null}::text[], federated_read) + WHERE client_id = ${clientId} + RETURNING client_id, client_name, source_id, federated_read + ` + : await this.sql` + UPDATE oauth_clients + SET source_id = COALESCE(${sourceId ?? null}::text, source_id), + federated_read = COALESCE(${federatedRead ? pgArray(federatedRead) : null}::text[], federated_read), + bound_slug_prefixes = ${boundSlugPrefixes ? pgArray(boundSlugPrefixes) : null}::text[] + WHERE client_id = ${clientId} + RETURNING client_id, client_name, source_id, federated_read, bound_slug_prefixes + `; } catch (err) { - if (isUndefinedColumnError(err, 'source_id') || isUndefinedColumnError(err, 'federated_read')) { + if ( + isUndefinedColumnError(err, 'source_id') || + isUndefinedColumnError(err, 'federated_read') || + isUndefinedColumnError(err, 'bound_slug_prefixes') + ) { throw new Error('rescope-client requires an up-to-date OAuth schema; run `gbrain apply-migrations --yes` and retry.'); } // FK oauth_clients.source_id → sources(id): translate the raw 23503 @@ -1064,6 +1195,11 @@ export class GBrainOAuthProvider implements OAuthServerProvider { clientName: (row.client_name as string | null) ?? '', sourceId: (row.source_id as string | null) ?? 'default', federatedRead: Array.isArray(row.federated_read) ? (row.federated_read as string[]) : [], + // undefined = the column wasn't read this call (caller left the + // binding untouched), which is distinct from null = no binding set. + boundSlugPrefixes: 'bound_slug_prefixes' in row + ? (Array.isArray(row.bound_slug_prefixes) ? (row.bound_slug_prefixes as string[]) : null) + : undefined, }; } diff --git a/src/core/operations.ts b/src/core/operations.ts index 38d139884..46fdfb3ad 100644 --- a/src/core/operations.ts +++ b/src/core/operations.ts @@ -229,6 +229,148 @@ function enforceSubagentSlugFence(ctx: OperationContext, slug: string, opName: s } } +/** + * OAuth-client slug-fence enforcement (v0.42.72.0 — write-side isolation + * symmetry). When the authenticated client was registered with + * --bound-slug-prefixes, every direct slug-mutating write must target a + * slug under one of those prefixes. Shared by put_page, delete_page, + * restore_page, add_tag, remove_tag, add_link/remove_link (`from` + * endpoint), add_timeline_entry, revert_version, and put_raw_data; runs + * BEFORE each op's dry-run short-circuit so preview calls surface the + * same rejection. + * + * Semantics deliberately match submit_agent's bound_slug_prefixes check + * (plain startsWith, NOT the `/*` glob grammar of the subagent allow-list + * above): a non-null binding fences fail-closed (empty array = deny all + * writes), no binding / no auth = no fence (local CLI and unbound clients + * keep full-source write authority). Register prefixes with a trailing + * slash ('wiki/agents/alice/') — a bare 'notes' also admits + * 'notes-archive/...' by startsWith construction. + */ +function enforceClientSlugFence(ctx: OperationContext, slug: string, opName: string): void { + if (ctx.auth?.fenceProjectionDegraded) { + throw new OperationError( + 'permission_denied', + `${opName}: this brain's oauth_clients projection is missing bound_slug_prefixes, so the write fence cannot be evaluated. Refusing the write rather than running unfenced.`, + 'Run `gbrain apply-migrations --yes` on the brain host.', + ); + } + const prefixes = ctx.auth?.boundSlugPrefixes; + if (!prefixes) return; + if (!slugUnderBoundPrefixes(prefixes, slug)) { + throw new OperationError( + 'permission_denied', + `${opName}: slug '${slug}' is not under any of client ${ctx.auth?.clientId ?? '(unknown)'}'s bound_slug_prefixes (${prefixes.join(', ')})`, + ); + } +} + +/** + * The one place the fence's match rule lives. Exported so non-op write + * surfaces that never build an OperationContext (the `/ingest` route in + * serve-http.ts) enforce byte-identical semantics instead of re-deriving + * them. + * + * An empty-string prefix is IGNORED rather than honored: `startsWith('')` + * is true for every slug, so a stray `''` (an unset shell variable in a + * provisioning template) would silently turn a binding into a wildcard + * while still rendering as "fenced" to the operator. Registration now + * rejects empty prefixes outright; this is the second line of defence for + * rows already in the database. + */ +export function slugUnderBoundPrefixes(prefixes: readonly string[], slug: string): boolean { + // Compare against the CANONICAL slug. `validateSlug` lowercases before the + // row is written, so checking the caller's raw string let `EMP-ALICE/x` + // satisfy an `EMP-ALICE/` binding, commit as `emp-alice/x`, and only then + // trip the resolved-slug re-check — an error returned after the write had + // already landed. Registration rejects non-lowercase prefixes going + // forward; lowercasing both sides keeps pre-existing rows meaning what + // their operator intended. + const canonical = slug.toLowerCase(); + return prefixes.some((bp) => { + const base = normalizeSlugPrefix(bp); + if (base === '') return false; + // Boundary-aware: a prefix must match whole SEGMENTS. Plain `startsWith` + // let a boundary-less `emp-alice` admit `emp-alice-2/onboarding` — and + // with the `emp-` naming this guide recommends, sibling collisions + // (`alice` vs `alice-2`) are the common case, not a corner case. + return base.endsWith('/') + ? canonical.startsWith(base) + : canonical === base || canonical.startsWith(`${base}/`); + }); +} + +/** + * Canonical form of one stored prefix, lowercased. `oauth_clients.bound_slug_prefixes` + * predates this fence — migration v85 introduced it as submit_agent's binding, + * whose grammar is the `/*` glob of `matchesSlugAllowList` — so both + * spellings have to mean the same span of slugs or upgrading silently changes + * what an existing client may write. + */ +export function normalizeSlugPrefix(prefix: string): string { + return (prefix.endsWith('/*') ? prefix.slice(0, -1) : prefix).toLowerCase(); +} + +/** + * Write ops a slug-bound client may call: every op that routes through + * `enforceClientSlugFence`, plus `think` (scope `write`, but remote callers + * cannot persist — `save`/`take` are forced false for `remote !== false`). + * + * This list is an ALLOW-list on purpose. The fence used to be enforced op + * by op, which made every unfenced write op a silent hole — `extract_entities` + * mutating `people/*` timelines, `forget_fact` rewriting another source's + * page by numeric id, `extract_facts` appending to any entity's fact fence. + * Enumerating what is SAFE fails closed instead: a write op added later is + * denied to bound clients until someone fences it and adds it here. + */ +export const CLIENT_FENCED_WRITE_OPS: ReadonlySet = new Set([ + 'put_page', 'delete_page', 'restore_page', 'add_tag', 'remove_tag', + 'add_link', 'remove_link', 'add_timeline_entry', 'revert_version', + 'put_raw_data', 'think', + // submit_agent enforces bound_slug_prefixes itself (it is the op the column + // was introduced for — see its bound_* binding check), so denying it here + // would break the original feature for clients that legitimately hold both + // a binding and `agent` scope. + 'submit_agent', +]); + +/** + * Fail-closed gate for slug-bound clients, applied at dispatch (the single + * choke point both MCP transports share) so it cannot be forgotten per op. + * Read ops are untouched — read scope is enforced by source federation. + */ +export function enforceBoundClientOpAllowList( + auth: AuthInfo | undefined, + op: Pick, +): void { + // A degraded projection means we could not read the binding, not that + // there isn't one. Deny every non-read op outright — otherwise the + // unfenceable ops stay reachable precisely when the fence is unreadable. + const degraded = auth?.fenceProjectionDegraded === true; + if (!degraded && !auth?.boundSlugPrefixes) return; + // Gate on "mutates, or carries any non-read scope" rather than on the two + // literal scope strings 'write'/'admin': `sources_add` / `sources_remove` + // carry the bespoke `sources_admin` scope and are `mutating: true`, so a + // scope-string check let a bound client DROP AN ENTIRE SOURCE — every page + // in it, far outside any prefix. Anything that isn't a plain read must be + // explicitly allow-listed. + const isRead = op.scope === 'read' && op.mutating !== true; + if (isRead) return; + if (degraded) { + throw new OperationError( + 'permission_denied', + `${op.name}: this brain's oauth_clients projection is missing bound_slug_prefixes, so client write bindings cannot be evaluated. Refusing every non-read operation rather than running unfenced.`, + 'Run `gbrain apply-migrations --yes` on the brain host.', + ); + } + if (CLIENT_FENCED_WRITE_OPS.has(op.name)) return; + throw new OperationError( + 'permission_denied', + `${op.name} is not available to slug-bound clients: it can write outside client ${auth?.clientId ?? '(unknown)'}'s bound_slug_prefixes (${(auth?.boundSlugPrefixes ?? []).join(', ')}).`, + 'Use put_page / add_timeline_entry / add_link under your own prefixes, or ask an operator to clear the binding with `gbrain auth rescope-client --bound-slug-prefixes none`.', + ); +} + /** * Allowlist validator for uploaded file basenames. Rejects control chars, backslashes, * RTL overrides (\u202E), leading dot (hidden files) and leading dash (CLI flag confusion). @@ -308,6 +450,31 @@ export interface AuthInfo { * case (back-compat). */ allowedSources?: string[]; + /** + * v0.42.72.0: slug-prefix WRITE binding from + * `oauth_clients.bound_slug_prefixes`, threaded at token-verification + * time (same JOIN as sourceId/allowedSources — no per-op roundtrip). + * When present, every direct slug-mutating write op is fenced to slugs + * under one of these prefixes via `enforceClientSlugFence` — the same + * plain-startsWith semantics (and the same fail-closed empty-array + * posture) as submit_agent's bound_slug_prefixes check, so one column + * means one thing everywhere it's read. Closes the write-side half of + * shared-source isolation: reads were SQL-fenced via `allowedSources`, + * but same-source writes were folder-convention-only. + * + * Undefined = client has no binding, or the brain predates the + * bound_slug_prefixes column → no fence (unbound clients keep + * full-source write authority, back-compat). + */ + boundSlugPrefixes?: string[]; + /** + * Set when token verification could not read `bound_slug_prefixes` (the + * projection degraded on a brain missing an OAuth column). The fence can't + * distinguish "no binding" from "binding unknown" otherwise, so writes are + * refused rather than silently unfenced. Read/auth degradation is + * unaffected — this axis alone fails closed. + */ + fenceProjectionDegraded?: boolean; } export interface OperationContext { @@ -904,6 +1071,7 @@ const put_page: Operation = { // short-circuit so preview calls surface the same rejection. See // enforceSubagentSlugFence for the fail-closed policy. enforceSubagentSlugFence(ctx, slug, 'put_page'); + enforceClientSlugFence(ctx, slug, 'put_page'); if (ctx.dryRun) return { dry_run: true, action: 'put_page', slug: p.slug }; @@ -978,6 +1146,29 @@ const put_page: Operation = { ingested_via: provenanceVia, }); + // The dedup pre-check in importFromContent can resolve the write to a + // DIFFERENT page than the one requested (same content_hash, or the same + // `frontmatter.id`), and the disk write-through below runs against that + // RESOLVED slug. Fence it too: a bound client can read a victim page's + // frontmatter id over its federated grant, echo it back in an in-prefix + // put_page, and otherwise have write-through rewrite the victim's file + // with falsified provenance. Dedup returns status 'skipped' without + // touching the DB, so throwing here leaves nothing to roll back. + if (result.slug && result.slug !== slug) { + // Deliberately does NOT name the resolved slug: it belongs to a page + // outside the binding, and echoing it would turn frontmatter-id guessing + // into a slug-enumeration oracle. + if (!slugUnderBoundPrefixes(ctx.auth?.boundSlugPrefixes ?? [], result.slug) + && ctx.auth?.boundSlugPrefixes) { + ctx.logger.warn(`[put_page] dedup resolved '${slug}' to an out-of-fence page; refusing (client ${ctx.auth.clientId ?? 'unknown'})`); + throw new OperationError( + 'permission_denied', + `put_page: this content already exists on a page outside your bound_slug_prefixes, so the write would have modified that page instead.`, + 'Remove the `id:` frontmatter field (or change the content) to write a new page under your own prefix.', + ); + } + } + // v0.39 T13 — auto-prompt on first unknown-type write. // // Contract (codex finding #8 honored — 7 cases covered): @@ -1127,6 +1318,22 @@ const put_page: Operation = { // (MEDIUM facts wait for the dream cycle but DO land via put_page, // matching the pre-fix behavior on this surface). let factsQueued: { queued: boolean } | { skipped: string } | undefined; + // Slug-bound clients do not get the facts backstop. It extracts entities + // from the (attacker-controllable) page body and writes fact rows — and, + // on a source with a local_path, a `## Facts` fence in the entity's own + // .md — keyed to `people/…` / `companies/…` slugs the caller never named. + // That is exactly the capability `extract_facts` is denied at dispatch + // for, reachable indirectly through a perfectly in-prefix put_page. The + // sibling post-hooks above already skip for untrusted callers (auto-link + // at `remote !== false && !trustedWorkspace`, chronicle at + // `remote !== false`); this one had no gate at all. + // Keyed on "the caller is slug-confined at all", not on ctx.auth alone: + // the delegated (submit_agent → subagent) context carries + // `allowedSlugPrefixes` but NOT `auth`, so an auth-only test would let a + // bound client re-open this path simply by delegating the write. + if (ctx.auth?.boundSlugPrefixes || ctx.viaSubagent === true) { + factsQueued = { skipped: 'slug_bound_client' }; + } else { try { const { runFactsBackstop } = await import('./facts/backstop.ts'); const r = await runFactsBackstop( @@ -1159,6 +1366,7 @@ const put_page: Operation = { } catch { factsQueued = { skipped: 'backstop_error' }; } + } // v0.42.x (#2390): Life Chronicle backstop. ONLY on a real import // (status==='imported' — a skipped/unchanged rewrite still carries @@ -1421,6 +1629,7 @@ const delete_page: Operation = { scope: 'write', handler: async (ctx, p) => { const slug = p.slug as string; + enforceClientSlugFence(ctx, slug, 'delete_page'); if (ctx.dryRun) return { dry_run: true, action: 'soft_delete_page', slug }; // v0.31.8 (D7): thread ctx.sourceId so multi-source brains soft-delete the // intended row instead of always targeting (default, slug). @@ -1454,6 +1663,7 @@ const restore_page: Operation = { scope: 'write', handler: async (ctx, p) => { const slug = p.slug as string; + enforceClientSlugFence(ctx, slug, 'restore_page'); if (ctx.dryRun) return { dry_run: true, action: 'restore_page', slug }; // v0.31.8 (D7): thread ctx.sourceId. const sourceOpts = ctx.sourceId ? { sourceId: ctx.sourceId } : {}; @@ -2097,6 +2307,7 @@ const add_tag: Operation = { mutating: true, scope: 'write', handler: async (ctx, p) => { + enforceClientSlugFence(ctx, p.slug as string, 'add_tag'); if (ctx.dryRun) return { dry_run: true, action: 'add_tag', slug: p.slug, tag: p.tag }; // v0.31.8 (D7): thread ctx.sourceId. const sourceOpts = ctx.sourceId ? { sourceId: ctx.sourceId } : {}; @@ -2116,6 +2327,7 @@ const remove_tag: Operation = { mutating: true, scope: 'write', handler: async (ctx, p) => { + enforceClientSlugFence(ctx, p.slug as string, 'remove_tag'); if (ctx.dryRun) return { dry_run: true, action: 'remove_tag', slug: p.slug, tag: p.tag }; const sourceOpts = ctx.sourceId ? { sourceId: ctx.sourceId } : {}; await ctx.engine.removeTag(p.slug as string, p.tag as string, sourceOpts); @@ -2169,6 +2381,10 @@ const add_link: Operation = { mutating: true, scope: 'write', handler: async (ctx, p) => { + // Client fence on the `from` endpoint only: the edge originates from + // (and renders on) the from page; linking TO a page outside the + // binding is a reference, not a mutation of the target. + enforceClientSlugFence(ctx, p.from as string, 'add_link'); if (ctx.dryRun) return { dry_run: true, action: 'add_link', from: p.from, to: p.to }; // v114 (#1941): default omitted provenance to 'manual' (NOT the engine's // 'markdown' default) so hand/tool-created CLI edges are honestly manual, @@ -2209,6 +2425,7 @@ const remove_link: Operation = { mutating: true, scope: 'write', handler: async (ctx, p) => { + enforceClientSlugFence(ctx, p.from as string, 'remove_link'); if (ctx.dryRun) return { dry_run: true, action: 'remove_link', from: p.from, to: p.to }; const linkOpts = ctx.sourceId ? { fromSourceId: ctx.sourceId, toSourceId: ctx.sourceId } @@ -2336,6 +2553,7 @@ const add_timeline_entry: Operation = { // confined to the same namespace/allow-list as page writes. Runs before // the dry-run short-circuit so preview calls surface the same rejection. enforceSubagentSlugFence(ctx, p.slug as string, 'add_timeline_entry'); + enforceClientSlugFence(ctx, p.slug as string, 'add_timeline_entry'); if (ctx.dryRun) return { dry_run: true, action: 'add_timeline_entry', slug: p.slug }; const date = p.date as string; // Reject anything that isn't a strict YYYY-MM-DD with year 1900-2199 and @@ -2730,6 +2948,7 @@ const revert_version: Operation = { mutating: true, scope: 'write', handler: async (ctx, p) => { + enforceClientSlugFence(ctx, p.slug as string, 'revert_version'); if (ctx.dryRun) return { dry_run: true, action: 'revert_version', slug: p.slug, version_id: p.version_id }; // v0.31.8 (D7): thread ctx.sourceId so multi-source brains revert the // intended page row instead of whichever same-slug row Postgres returns @@ -2789,6 +3008,7 @@ const put_raw_data: Operation = { mutating: true, scope: 'write', handler: async (ctx, p) => { + enforceClientSlugFence(ctx, p.slug as string, 'put_raw_data'); if (ctx.dryRun) return { dry_run: true, action: 'put_raw_data', slug: p.slug, source: p.source }; // v0.31.8 (D7 + D21): thread ctx.sourceId. const sourceOpts = ctx.sourceId ? { sourceId: ctx.sourceId } : {}; @@ -3189,7 +3409,20 @@ const submit_agent: Operation = { } // Validate each param against the binding. - const requestedTools = (p.allowed_tools as string[] | undefined) ?? boundTools; + // + // An EXPLICIT empty array is not "no restriction" here — downstream the + // subagent worker reads empty `allowed_tools` as "the full tool registry" + // and empty `allowed_slug_prefixes` as "fall back to the legacy + // wiki/agents// namespace". Both subset loops below pass + // vacuously over an empty list, so `{allowed_tools: [], allowed_slug_prefixes: []}` + // from a client bound to `['search']` + `['emp-alice/']` would hand its + // subagent the whole registry (including put_page) writing outside the + // binding. `??` only substitutes null/undefined, so collapse the empty + // case to the binding explicitly. + const requestedToolsRaw = p.allowed_tools as string[] | undefined; + const requestedTools = requestedToolsRaw === undefined || requestedToolsRaw.length === 0 + ? boundTools + : requestedToolsRaw; for (const t of requestedTools) { if (!boundTools.includes(t)) { throw new OperationError( @@ -3198,10 +3431,35 @@ const submit_agent: Operation = { ); } } - const requestedSlugPrefixes = (p.allowed_slug_prefixes as string[] | undefined) ?? boundSlugPrefixes ?? []; + const requestedSlugPrefixesRaw = p.allowed_slug_prefixes as string[] | undefined; + const requestedSlugPrefixes = + requestedSlugPrefixesRaw === undefined || requestedSlugPrefixesRaw.length === 0 + ? (boundSlugPrefixes ?? []) + : requestedSlugPrefixesRaw; + // A bound client must end up with a non-empty delegated fence: an empty + // list reaches the subagent as "use the legacy wiki/agents// namespace", + // which is outside every bound prefix. + if (boundSlugPrefixes !== null && requestedSlugPrefixes.length === 0) { + throw new OperationError( + 'permission_denied', + `submit_agent: client ${clientId} is slug-bound but its binding resolved to an empty prefix list, which the subagent would read as the unfenced legacy namespace.`, + 'Re-scope the client with a non-empty --bound-slug-prefixes.', + ); + } if (boundSlugPrefixes !== null) { for (const sp of requestedSlugPrefixes) { - if (!boundSlugPrefixes.some(bp => sp.startsWith(bp) || bp === sp)) { + // Boundary-aware, same rule as the direct fence: a raw `startsWith` + // let a boundary-less binding (`emp-alice`) authorize a requested + // prefix in a SIBLING namespace (`emp-alice-2/`), which is then handed + // to the child as a full glob grant over another employee's pages. + if (!boundSlugPrefixes.some(bp => { + const base = normalizeSlugPrefix(bp); + const req = normalizeSlugPrefix(sp); + if (base === '') return false; + return base.endsWith('/') + ? req.startsWith(base) + : req === base || req.startsWith(`${base}/`); + })) { throw new OperationError( 'permission_denied', `submit_agent: slug_prefix "${sp}" is not under any of client ${clientId}'s bound_slug_prefixes.`, @@ -3227,6 +3485,14 @@ const submit_agent: Operation = { } // Dry-run echo. + // The subagent fence uses `matchesSlugAllowList`, whose grammar makes a + // BARE entry match that one slug exactly — so a plain `emp-alice/` binding + // would let the delegated agent write nothing. Normalize the + // trailing-slash form into the glob the delegated matcher expects, so one + // stored column means the same span of slugs on both paths. + const delegatedSlugPrefixes = requestedSlugPrefixes.map(sp => + sp.endsWith('/') ? `${sp}*` : sp); + if (ctx.dryRun) { return { dry_run: true, @@ -3235,6 +3501,10 @@ const submit_agent: Operation = { bound_tools: boundTools, bound_source: boundSource, bound_max_concurrent: boundMaxConcurrent, + // What the delegated job would ACTUALLY be granted, after the binding + // is applied — a preview that hides this can't show a widening bug. + resolved_tools: requestedTools, + resolved_slug_prefixes: delegatedSlugPrefixes, }; } @@ -3248,11 +3518,24 @@ const submit_agent: Operation = { prompt: p.prompt as string, max_turns: Math.min((p.max_turns as number) ?? 20, 100), allowed_tools: requestedTools, - allowed_slug_prefixes: requestedSlugPrefixes, + allowed_slug_prefixes: delegatedSlugPrefixes, __owner_client_id: clientId, }; if (typeof p.model === 'string') jobData.model = p.model; - if (boundSource) jobData.source_id = boundSource; + // Write source for the delegated job comes from the AUTHENTICATED client + // whenever we have it. `bound_source_id` is an optional, separately-set + // column: unset it defaulted the child to 'default', and if it disagreed + // with the token's own source the child followed the column — either way + // a correctly slug-fenced client could act on the wrong source. + const delegatedSource = ctx.auth?.sourceId ?? boundSource; + if (boundSource && ctx.auth?.sourceId && boundSource !== ctx.auth.sourceId) { + throw new OperationError( + 'permission_denied', + `submit_agent: client ${clientId}'s bound_source_id (${boundSource}) disagrees with its authenticated source (${ctx.auth.sourceId}); refusing to guess which one governs the delegated write.`, + 'Re-scope the client so the two agree: `gbrain auth rescope-client --source `.', + ); + } + if (delegatedSource) jobData.source_id = delegatedSource; const job = await queue.add( 'subagent', jobData, diff --git a/src/mcp/dispatch.ts b/src/mcp/dispatch.ts index 18552fc80..abf1c27c8 100644 --- a/src/mcp/dispatch.ts +++ b/src/mcp/dispatch.ts @@ -7,7 +7,7 @@ */ import type { BrainEngine } from '../core/engine.ts'; -import { operations, OperationError } from '../core/operations.ts'; +import { operations, OperationError, enforceBoundClientOpAllowList } from '../core/operations.ts'; import type { Operation, OperationContext, AuthInfo } from '../core/operations.ts'; import { loadConfig } from '../core/config.ts'; @@ -280,6 +280,11 @@ export async function dispatchToolCall( const ctx = buildOperationContext(engine, safeParams, opts); try { + // Fail-closed gate for slug-bound OAuth clients, applied here because + // this is the one path both MCP transports share. Per-op fences still + // run inside the handlers; this stops an unfenced write op from being + // a silent hole. See CLIENT_FENCED_WRITE_OPS in operations.ts. + enforceBoundClientOpAllowList(ctx.auth, op); const result = await op.handler(ctx, safeParams); const out: ToolResult = { content: [{ type: 'text', text: JSON.stringify(result, null, 2) }] }; // v0.31 (eD3 + eE4): best-effort _meta.brain_hot_memory injection. diff --git a/test/client-slug-fence.test.ts b/test/client-slug-fence.test.ts new file mode 100644 index 000000000..9ce970de4 --- /dev/null +++ b/test/client-slug-fence.test.ts @@ -0,0 +1,264 @@ +/** + * OAuth-client slug-fence tests (v0.42.70.0 — write-side isolation symmetry). + * + * enforceClientSlugFence confines a bound client's direct writes to slugs + * under its `oauth_clients.bound_slug_prefixes`. This pins: + * - regression: no auth / unbound client → every op accepts any slug + * (local CLI and unbound-remote behavior unchanged); + * - fence: each slug-mutating write op rejects out-of-binding slugs with + * permission_denied, BEFORE the dry-run short-circuit (all denials here + * run with dryRun=true and an empty engine stub); + * - fail-closed: an empty-array binding denies all writes (matches + * submit_agent's posture for the same column); + * - add_link/remove_link fence the `from` endpoint only — linking TO a + * page outside the binding is a reference, not a mutation of it. + */ + +import { describe, test, expect } from 'bun:test'; +import { + operations, OperationError, slugUnderBoundPrefixes, + enforceBoundClientOpAllowList, CLIENT_FENCED_WRITE_OPS, +} from '../src/core/operations.ts'; +import type { OperationContext, Operation, AuthInfo } from '../src/core/operations.ts'; +import type { BrainEngine } from '../src/core/engine.ts'; + +function op(name: string): Operation { + const found = operations.find(o => o.name === name); + if (!found) throw new Error(`${name} op missing`); + return found; +} + +function makeCtx(overrides: Partial = {}): OperationContext { + const engine = {} as BrainEngine; // dry_run short-circuits before touching the engine + return { + engine, + config: { engine: 'postgres' } as any, + logger: { info: () => {}, warn: () => {}, error: () => {} }, + dryRun: true, + remote: true, + sourceId: 'shared', + ...overrides, + }; +} + +function boundAuth(prefixes: string[] | undefined): AuthInfo { + return { + token: 'test-token', + clientId: 'gbrain_cl_fence_test', + scopes: ['read', 'write'], + sourceId: 'shared', + ...(prefixes !== undefined ? { boundSlugPrefixes: prefixes } : {}), + }; +} + +// Every fenced op with a params factory for an arbitrary slug. +const FENCED_OPS: Array<{ name: string; params: (slug: string) => Record }> = [ + { name: 'put_page', params: (slug) => ({ slug, content: 'stub' }) }, + { name: 'delete_page', params: (slug) => ({ slug }) }, + { name: 'restore_page', params: (slug) => ({ slug }) }, + { name: 'add_tag', params: (slug) => ({ slug, tag: 't' }) }, + { name: 'remove_tag', params: (slug) => ({ slug, tag: 't' }) }, + { name: 'add_link', params: (slug) => ({ from: slug, to: 'org-wiki/roadmap' }) }, + { name: 'remove_link', params: (slug) => ({ from: slug, to: 'org-wiki/roadmap' }) }, + { name: 'add_timeline_entry', params: (slug) => ({ slug, date: '2026-08-01', summary: 's' }) }, + { name: 'revert_version', params: (slug) => ({ slug, version_id: 1 }) }, + { name: 'put_raw_data', params: (slug) => ({ slug, source: 'src', data: {} }) }, +]; + +describe('client slug fence (bound_slug_prefixes on direct writes)', () => { + describe('regression: unbound callers unchanged', () => { + for (const { name, params } of FENCED_OPS) { + test(`${name}: no ctx.auth accepts arbitrary slug`, async () => { + const result = await op(name).handler(makeCtx(), params('anywhere/at-all')); + expect(result).toMatchObject({ dry_run: true }); + }); + + test(`${name}: authed client WITHOUT binding accepts arbitrary slug`, async () => { + const ctx = makeCtx({ auth: boundAuth(undefined) }); + const result = await op(name).handler(ctx, params('anywhere/at-all')); + expect(result).toMatchObject({ dry_run: true }); + }); + } + }); + + describe('fence: bound client confined to its prefixes', () => { + const auth = boundAuth(['chan-eng/', 'emp-alice/']); + + for (const { name, params } of FENCED_OPS) { + test(`${name}: in-binding slug accepted`, async () => { + const ctx = makeCtx({ auth }); + const result = await op(name).handler(ctx, params('chan-eng/standup-notes')); + expect(result).toMatchObject({ dry_run: true }); + }); + + test(`${name}: out-of-binding slug rejected with permission_denied`, async () => { + const ctx = makeCtx({ auth }); + try { + await op(name).handler(ctx, params('chan-product/roadmap')); + throw new Error('should have thrown'); + } catch (e) { + expect(e).toBeInstanceOf(OperationError); + expect((e as OperationError).code).toBe('permission_denied'); + expect((e as Error).message).toContain('bound_slug_prefixes'); + } + }); + } + + test('second prefix also admits writes', async () => { + const ctx = makeCtx({ auth }); + const result = await op('put_page').handler(ctx, { slug: 'emp-alice/journal', content: 'stub' }); + expect(result).toMatchObject({ dry_run: true }); + }); + + test('prefix match is plain startsWith — bare slug equal to a prefix-less-slash is rejected', async () => { + const ctx = makeCtx({ auth }); + const p = op('put_page').handler(ctx, { slug: 'chan-eng', content: 'stub' }); + await expect(p).rejects.toBeInstanceOf(OperationError); + }); + + test('add_link: `to` outside the binding is allowed (reference, not mutation)', async () => { + const ctx = makeCtx({ auth }); + const result = await op('add_link').handler(ctx, { from: 'chan-eng/decision', to: 'org-wiki/anything' }); + expect(result).toMatchObject({ dry_run: true }); + }); + + test('local CLI (no auth, remote=false) is never fenced', async () => { + const ctx = makeCtx({ remote: false }); + const result = await op('put_page').handler(ctx, { slug: 'people/alice', content: 'stub' }); + expect(result).toMatchObject({ dry_run: true }); + }); + }); + + describe('fail-closed: empty-array binding denies all writes', () => { + test('put_page with boundSlugPrefixes=[] rejects every slug', async () => { + const ctx = makeCtx({ auth: boundAuth([]) }); + const p = op('put_page').handler(ctx, { slug: 'anywhere/at-all', content: 'stub' }); + await expect(p).rejects.toBeInstanceOf(OperationError); + }); + }); + + describe('empty-string prefix cannot silently disable the fence', () => { + // startsWith('') is true for every slug, so a stray '' (an unset variable + // in a provisioning template) would render as "bound" while fencing + // nothing. Registration rejects it; the matcher ignores it anyway. + test("[''] denies every slug rather than allowing every slug", async () => { + const ctx = makeCtx({ auth: boundAuth(['']) }); + const p = op('put_page').handler(ctx, { slug: 'anywhere/at-all', content: 'stub' }); + await expect(p).rejects.toBeInstanceOf(OperationError); + }); + + test("a real prefix alongside '' still fences to the real one", async () => { + const ctx = makeCtx({ auth: boundAuth(['chan-eng/', '']) }); + const ok = await op('put_page').handler(ctx, { slug: 'chan-eng/x', content: 'stub' }); + expect(ok).toMatchObject({ dry_run: true }); + await expect(op('put_page').handler(ctx, { slug: 'other/x', content: 'stub' })) + .rejects.toBeInstanceOf(OperationError); + }); + + test('slugUnderBoundPrefixes ignores empty prefixes', () => { + expect(slugUnderBoundPrefixes([''], 'anything')).toBe(false); + expect(slugUnderBoundPrefixes(['a/'], 'a/b')).toBe(true); + expect(slugUnderBoundPrefixes(['a/'], 'b/a')).toBe(false); + }); + }); + + describe('dispatch allow-list: unfenceable write ops are denied outright', () => { + const bound = boundAuth(['emp-alice/']); + const unbound = boundAuth(undefined); + + // These write by a key other than a slug (derived entity names, numeric + // fact ids), so no per-op fence can confine them. + for (const name of ['extract_entities', 'extract_facts', 'forget_fact', 'ontology_propose']) { + test(`${name} is denied for a bound client`, () => { + const o = operations.find(x => x.name === name); + if (!o) throw new Error(`${name} missing`); + expect(() => enforceBoundClientOpAllowList(bound, o)).toThrow(/not available to slug-bound clients/); + expect(() => enforceBoundClientOpAllowList(unbound, o)).not.toThrow(); + }); + } + + test('every fenced write op is allowed', () => { + for (const name of CLIENT_FENCED_WRITE_OPS) { + const o = operations.find(x => x.name === name); + if (!o) throw new Error(`${name} missing from operations`); + expect(() => enforceBoundClientOpAllowList(bound, o)).not.toThrow(); + } + }); + + test('read ops are never gated', () => { + for (const o of operations.filter(x => x.scope === 'read')) { + expect(() => enforceBoundClientOpAllowList(bound, o)).not.toThrow(); + } + }); + + // The regression this exists to prevent: a write op added later must be + // denied by default, not silently unfenced. + test('a hypothetical new write op is denied by default', () => { + expect(() => enforceBoundClientOpAllowList(bound, { name: 'brand_new_write_op', scope: 'write' })) + .toThrow(/not available to slug-bound clients/); + }); + }); + + describe('both prefix grammars are accepted (the column predates this fence)', () => { + // v85 introduced bound_slug_prefixes for submit_agent, whose grammar is + // matchesSlugAllowList's `/*` glob. Rejecting it here would deny + // every direct write to already-configured clients on upgrade. + test('a glob-style binding still matches', () => { + expect(slugUnderBoundPrefixes(['wiki/agents/alice/*'], 'wiki/agents/alice/notes')).toBe(true); + expect(slugUnderBoundPrefixes(['wiki/agents/alice/*'], 'wiki/agents/bob/notes')).toBe(false); + }); + + test('a trailing-slash binding still matches', () => { + expect(slugUnderBoundPrefixes(['emp-alice/'], 'emp-alice/notes')).toBe(true); + expect(slugUnderBoundPrefixes(['emp-alice/'], 'emp-alice-evil/notes')).toBe(false); + }); + + // The `emp-` scheme makes sibling collisions the common case: + // `alice` and `alice-2` are different people. A plain startsWith let a + // boundary-less binding reach the neighbour's namespace. + test('a boundary-less prefix does NOT reach a sibling namespace', () => { + expect(slugUnderBoundPrefixes(['emp-alice'], 'emp-alice/notes')).toBe(true); + expect(slugUnderBoundPrefixes(['emp-alice'], 'emp-alice')).toBe(true); + expect(slugUnderBoundPrefixes(['emp-alice'], 'emp-alice-2/onboarding')).toBe(false); + expect(slugUnderBoundPrefixes(['emp-alice'], 'emp-alicexyz/secret')).toBe(false); + }); + + test('trailing-slash and glob forms are equally boundary-safe', () => { + for (const p of ['emp-alice/', 'emp-alice/*']) { + expect(slugUnderBoundPrefixes([p], 'emp-alice/notes')).toBe(true); + expect(slugUnderBoundPrefixes([p], 'emp-alice-2/notes')).toBe(false); + } + }); + + test('the canonical (lowercased) slug is what is matched', () => { + // validateSlug lowercases before storage, so the fence must compare the + // form that actually gets written — not the caller's raw string. + expect(slugUnderBoundPrefixes(['emp-alice/'], 'EMP-ALICE/Notes')).toBe(true); + expect(slugUnderBoundPrefixes(['emp-alice/'], 'EMP-BOB/Notes')).toBe(false); + }); + }); + + describe('degraded fence projection fails closed', () => { + test('writes are refused when bound_slug_prefixes could not be read', async () => { + const ctx = makeCtx({ + auth: { ...boundAuth(undefined), fenceProjectionDegraded: true }, + }); + const p = op('put_page').handler(ctx, { slug: 'anything/at-all', content: 'stub' }); + await expect(p).rejects.toBeInstanceOf(OperationError); + await expect(p).rejects.toThrow(/cannot be evaluated/); + }); + }); + + describe('composition with the subagent fence', () => { + test('both fences apply: subagent namespace passes but client binding rejects', async () => { + const ctx = makeCtx({ + viaSubagent: true, + subagentId: 42, + auth: boundAuth(['chan-eng/']), + }); + const p = op('put_page').handler(ctx, { slug: 'wiki/agents/42/notes', content: 'stub' }); + await expect(p).rejects.toBeInstanceOf(OperationError); + await expect(p).rejects.toThrow(/bound_slug_prefixes/); + }); + }); +}); diff --git a/test/e2e/qm-provisioning.test.ts b/test/e2e/qm-provisioning.test.ts new file mode 100644 index 000000000..31d7e2922 --- /dev/null +++ b/test/e2e/qm-provisioning.test.ts @@ -0,0 +1,350 @@ +/** + * E2E for the qm-harness integration recipe (docs/integrations/qm-harness.md): + * roster-driven provisioning + over-the-wire write fencing. + * + * PGLite-based and ungated (no DATABASE_URL needed) — PGLite is + * single-process, so every provisioning step runs BEFORE `serve --http` + * starts; after that all access goes over HTTP MCP. + * + * Pins, end to end: + * - provision-scopes.sh creates a path-less shared source + one bound + * client per employee, is idempotent on re-run, and RESCOPES in place + * (no secret rotation) when the roster changes; + * - thin clients (`init --mcp-only`) can write inside their + * bound_slug_prefixes and are rejected with the fence error outside + * them (v0.42.70.0 enforceClientSlugFence, over the real transport); + * - reads stay source-granular (a bob-example client CAN read + * chan-eng/ — the documented shared-source tradeoff). + */ + +import { describe, test as testRaw, expect, beforeAll, afterAll } from 'bun:test'; +import { mkdtempSync, rmSync, writeFileSync, readFileSync, existsSync } from 'fs'; +import { join } from 'path'; +import { tmpdir } from 'os'; + +function test(name: string, fn: () => void | Promise): void { + testRaw(name, fn, 120000); +} + +const CLI = join(__dirname, '..', '..', 'src', 'cli.ts'); +const SCRIPT = join(__dirname, '..', '..', 'docs', 'integrations', 'qm-harness-snippets', 'provision-scopes.sh'); + +interface RunResult { exitCode: number; stdout: string; stderr: string; } + +async function spawn(cmd: string[], env: Record, cwd?: string): Promise { + const fullEnv: Record = {}; + for (const [k, v] of Object.entries(process.env)) { + if (v !== undefined) fullEnv[k] = v; + } + delete fullEnv.GBRAIN_REMOTE_CLIENT_SECRET; + delete fullEnv.DATABASE_URL; + for (const [k, v] of Object.entries(env)) { + if (v === undefined) delete fullEnv[k]; + else fullEnv[k] = v; + } + const proc = Bun.spawn({ cmd, env: fullEnv, cwd, stdin: 'ignore', stdout: 'pipe', stderr: 'pipe' }); + const [stdout, stderr, exitCode] = await Promise.all([ + new Response(proc.stdout).text(), + new Response(proc.stderr).text(), + proc.exited, + ]); + return { exitCode, stdout, stderr }; +} + +const gbrain = (args: string[], home: string, extraEnv: Record = {}) => + spawn(['bun', 'run', CLI, ...args], { GBRAIN_HOME: home, ...extraEnv }); + +describe('qm-harness provisioning + write fence (e2e, PGLite)', () => { + let hostHome: string; + let workDir: string; + let aliceHome: string; + let bobHome: string; + let serverProc: ReturnType | null = null; + let serverPort: number; + const creds: Record = {}; + let rerunCredsGrew = true; // set false when idempotency holds + + const rosterPath = () => join(workDir, 'roster.tsv'); + const statePath = () => join(workDir, 'roster.tsv.state.tsv'); + const secretsPath = () => join(workDir, 'roster.tsv.new-credentials.tsv'); + + async function provision(): Promise { + return spawn( + ['bash', SCRIPT, rosterPath(), '--gbrain', `bun run ${CLI}`, '--budget-usd-per-day', '5'], + { GBRAIN_HOME: hostHome }, + workDir, + ); + } + + beforeAll(async () => { + hostHome = mkdtempSync(join(tmpdir(), 'gbrain-qm-host-')); + workDir = mkdtempSync(join(tmpdir(), 'gbrain-qm-work-')); + aliceHome = mkdtempSync(join(tmpdir(), 'gbrain-qm-alice-')); + bobHome = mkdtempSync(join(tmpdir(), 'gbrain-qm-bob-')); + + // 1. Host brain on PGLite, embedding deferred (FTS is enough here). + const init = await gbrain(['init', '--pglite', '--no-embedding'], hostHome); + if (init.exitCode !== 0) throw new Error(`host init failed: ${init.stderr || init.stdout}`); + + // 2. Roster v1: alice in eng, bob in product. + writeFileSync(rosterPath(), [ + 'channel eng', + 'channel product', + 'employee alice-example eng', + 'employee bob-example product', + '', + ].join('\n')); + const p1 = await provision(); + if (p1.exitCode !== 0) throw new Error(`provision v1 failed: ${p1.stderr || p1.stdout}`); + + for (const line of readFileSync(secretsPath(), 'utf8').trim().split('\n')) { + const [slug, clientId, secret] = line.split('\t'); + creds[slug] = { clientId, secret }; + } + + // 3. Idempotency: re-run with the same roster mints no new secrets. + const before = readFileSync(secretsPath(), 'utf8'); + const p2 = await provision(); + if (p2.exitCode !== 0) throw new Error(`provision re-run failed: ${p2.stderr || p2.stdout}`); + rerunCredsGrew = readFileSync(secretsPath(), 'utf8') !== before; + + // 4. Roster churn: alice joins product → rescope in place. + writeFileSync(rosterPath(), [ + 'channel eng', + 'channel product', + 'employee alice-example eng,product', + 'employee bob-example product', + '', + ].join('\n')); + const p3 = await provision(); + if (p3.exitCode !== 0) throw new Error(`provision rescope failed: ${p3.stderr || p3.stdout}`); + + // 4b. An UNBOUND client, standing in for a webhook integration. Registered + // here because PGLite is single-process: once serve --http holds the + // lock, no host-side CLI command can run. + const wh = await gbrain([ + 'auth', 'register-client', 'webhook-integration', + '--grant-types', 'client_credentials', '--scopes', 'read write', + ], hostHome); + if (wh.exitCode !== 0) throw new Error(`webhook client registration failed: ${wh.stderr || wh.stdout}`); + creds['webhook-integration'] = { + clientId: wh.stdout.match(/Client ID:\s+(gbrain_cl_\S+)/)?.[1] ?? '', + secret: wh.stdout.match(/Client Secret:\s+(gbrain_cs_\S+)/)?.[1] ?? '', + }; + + // 5. Serve over HTTP MCP (holds the PGLite lock from here on). + serverPort = 30000 + Math.floor(Math.random() * 30000); + const env: Record = {}; + for (const [k, v] of Object.entries(process.env)) { + if (v !== undefined) env[k] = v; + } + env.GBRAIN_HOME = hostHome; + delete env.DATABASE_URL; + serverProc = Bun.spawn({ + cmd: ['bun', 'run', CLI, 'serve', '--http', '--port', String(serverPort)], + env, stdin: 'ignore', stdout: 'pipe', stderr: 'pipe', + }); + const deadline = Date.now() + 30_000; + let up = false; + while (Date.now() < deadline) { + try { + const res = await fetch(`http://127.0.0.1:${serverPort}/.well-known/oauth-authorization-server`, { + signal: AbortSignal.timeout(500), + }); + if (res.ok) { up = true; break; } + } catch { /* retry */ } + await new Promise(r => setTimeout(r, 250)); + } + if (!up) throw new Error('serve --http did not come up'); + + // 6. Thin-client bootstrap for both scopes (the once-per-sandbox step). + // --oauth-client-secret (NOT the env var) on purpose: an env-sourced + // secret is deliberately not persisted to config.json, and qm has no + // per-scope env to keep it in. Every later call below runs WITHOUT the + // env var, so the suite proves the documented setup actually survives + // the init session instead of masking it. + for (const [slug, home] of [['alice-example', aliceHome], ['bob-example', bobHome]] as const) { + const tc = await gbrain([ + 'init', '--mcp-only', + '--issuer-url', `http://127.0.0.1:${serverPort}`, + '--mcp-url', `http://127.0.0.1:${serverPort}/mcp`, + '--oauth-client-id', creds[slug].clientId, + '--oauth-client-secret', creds[slug].secret, + ], home); + if (tc.exitCode !== 0) throw new Error(`thin-client init (${slug}) failed: ${tc.stderr || tc.stdout}`); + } + }, 300_000); + + afterAll(async () => { + if (serverProc) { + serverProc.kill(); + await serverProc.exited.catch(() => {}); + } + for (const dir of [hostHome, workDir, aliceHome, bobHome]) { + if (dir) rmSync(dir, { recursive: true, force: true }); + } + }, 30_000); + + // No GBRAIN_REMOTE_CLIENT_SECRET: auth must come from the persisted config. + const asAlice = (args: string[]) => gbrain(args, aliceHome); + const asBob = (args: string[]) => gbrain(args, bobHome); + + async function mintToken(slug: string): Promise { + const { clientId, secret } = creds[slug]; + const res = await fetch(`http://127.0.0.1:${serverPort}/token`, { + method: 'POST', + headers: { 'Content-Type': 'application/x-www-form-urlencoded' }, + body: `grant_type=client_credentials&client_id=${encodeURIComponent(clientId)}` + + `&client_secret=${encodeURIComponent(secret)}&scope=${encodeURIComponent('read write')}`, + }); + if (!res.ok) throw new Error(`token mint failed: ${res.status} ${await res.text()}`); + return ((await res.json()) as { access_token: string }).access_token; + } + + async function mcpCall(token: string, toolName: string, args: Record): Promise { + const res = await fetch(`http://127.0.0.1:${serverPort}/mcp`, { + method: 'POST', + headers: { + 'Authorization': `Bearer ${token}`, + 'Content-Type': 'application/json', + 'Accept': 'application/json, text/event-stream', + }, + body: JSON.stringify({ + jsonrpc: '2.0', id: 1, method: 'tools/call', + params: { name: toolName, arguments: args }, + }), + }); + return res.text(); + } + + test('provisioning minted one bound client per employee, exactly once', () => { + // Only the roster-provisioned clients; 'webhook-integration' is registered + // separately by the suite to prove the /ingest deny is scoped to bound clients. + expect(Object.keys(creds).filter(k => k !== 'webhook-integration').sort()) + .toEqual(['alice-example', 'bob-example']); + expect(creds['alice-example'].clientId).toStartWith('gbrain_cl_'); + expect(creds['alice-example'].secret).toStartWith('gbrain_cs_'); + expect(rerunCredsGrew).toBe(false); + expect(existsSync(statePath())).toBe(true); + }); + + test('alice writes inside her prefixes (personal + channel)', async () => { + const own = await asAlice(['put', 'emp-alice-example/notes/hello', '--content', '# hello\nmine']); + expect(own.exitCode).toBe(0); + const chan = await asAlice(['put', 'chan-eng/notes/standup', '--content', '# standup\nshared']); + expect(chan.exitCode).toBe(0); + }); + + test('roster churn took effect: alice can write chan-product/ after rescope', async () => { + const joined = await asAlice(['put', 'chan-product/notes/joined', '--content', '# joined']); + expect(joined.exitCode).toBe(0); + }); + + test("alice cannot write bob's namespace or an unbound prefix", async () => { + const bobNs = await asAlice(['put', 'emp-bob-example/notes/nope', '--content', 'x']); + expect(bobNs.exitCode).not.toBe(0); + expect(bobNs.stdout + bobNs.stderr).toMatch(/bound_slug_prefixes/); + + const stray = await asAlice(['put', 'org-notes/anything', '--content', 'x']); + expect(stray.exitCode).not.toBe(0); + expect(stray.stdout + stray.stderr).toMatch(/bound_slug_prefixes/); + }); + + test('bob is fenced to HIS prefixes (not in eng)', async () => { + const own = await asBob(['put', 'emp-bob-example/notes/hello', '--content', '# hi']); + expect(own.exitCode).toBe(0); + const eng = await asBob(['put', 'chan-eng/notes/nope', '--content', 'x']); + expect(eng.exitCode).not.toBe(0); + expect(eng.stdout + eng.stderr).toMatch(/bound_slug_prefixes/); + }); + + test('reads stay source-granular: bob CAN read chan-eng pages (documented tradeoff)', async () => { + const read = await asBob(['get', 'chan-eng/notes/standup']); + expect(read.exitCode).toBe(0); + expect(read.stdout).toContain('standup'); + }); + + test('the documented health check works on a read+write client (no admin scope)', async () => { + const who = await asAlice(['whoami']); + expect(who.exitCode).toBe(0); + expect(who.stdout).toContain(creds['alice-example'].clientId); + }); + + test('POST /ingest is closed to bound clients — it bypasses the op layer entirely', async () => { + const post = async (token: string, slug: string | null) => { + const headers: Record = { + 'Authorization': `Bearer ${token}`, + 'Content-Type': 'text/markdown', + }; + if (slug) headers['X-Gbrain-Slug'] = slug; + const res = await fetch(`http://127.0.0.1:${serverPort}/ingest`, { + method: 'POST', headers, + body: '---\ntype: note\ntitle: x\n---\n# injected', + }); + return { status: res.status, body: await res.text() }; + }; + const bound = await mintToken('alice-example'); + + // The bypass this closes: /ingest queues a job for a handler that skips + // the put_page op layer AND refuses to honor a source id for untrusted + // payloads, so the write lands in the `default` source. Fencing only the + // slug would still have written the right slug into the wrong source. + const outside = await post(bound, 'wiki/ceo-comp'); + expect(outside.status).toBe(403); + expect(outside.body).toContain('not available to clients restricted to slug prefixes'); + + // Even an IN-prefix slug is refused — the source, not just the slug, is + // outside the client's grant. + expect((await post(bound, 'emp-alice-example/inbox/note')).status).toBe(403); + expect((await post(bound, null)).status).toBe(403); + }); + + test('/ingest still works for an unbound webhook client (deny is scoped to bound clients)', async () => { + expect(creds['webhook-integration'].clientId).toStartWith('gbrain_cl_'); + const token = await mintToken('webhook-integration'); + + const res = await fetch(`http://127.0.0.1:${serverPort}/ingest`, { + method: 'POST', + headers: { + 'Authorization': `Bearer ${token}`, + 'Content-Type': 'text/markdown', + 'X-Gbrain-Slug': 'inbox/webhook-note', + }, + body: '---\ntype: note\ntitle: x\n---\n# from a webhook', + }); + expect([200, 202]).toContain(res.status); + }); + + test('write ops that cannot be slug-fenced are denied to a bound client', async () => { + // extract_entities mutates people/* and companies/* timelines; extract_facts + // appends to any entity's fact fence; forget_fact targets a fact by numeric + // id across sources; ontology_propose writes claims keyed to any entity. + // None takes a fenceable slug, so all are denied at dispatch rather than + // left silently unfenced. + const token = await mintToken('alice-example'); + const cases: Array<[string, Record]> = [ + ['extract_entities', { text: 'Bob Victim did a bad thing.', source_slug: 'emp-alice-example/notes/hello' }], + ['extract_facts', { turn_text: 'Bob Victim admitted it.', entity_hints: ['people/bob-victim'] }], + ['forget_fact', { id: 1, reason: 'retracted' }], + ['ontology_propose', { entity: 'emp-bob-example/profile', dimension: 'role', value: 'terminated' }], + ]; + for (const [tool, args] of cases) { + const body = await mcpCall(token, tool, args); + expect(body).toMatch(/not available to slug-bound clients/); + } + }); + + test('a fenced write op still works over the same transport (allow-list is not a blanket deny)', async () => { + const token = await mintToken('alice-example'); + const ok = await mcpCall(token, 'put_page', { + slug: 'emp-alice-example/notes/via-mcp', content: '# via mcp', + }); + expect(ok).not.toMatch(/not available to slug-bound clients/); + expect(ok).not.toMatch(/permission_denied/); + + const denied = await mcpCall(token, 'put_page', { + slug: 'emp-bob-example/notes/nope', content: '# nope', + }); + expect(denied).toMatch(/bound_slug_prefixes/); + }); +}); diff --git a/test/oauth.test.ts b/test/oauth.test.ts index 66eb7d3ee..cd223ae87 100644 --- a/test/oauth.test.ts +++ b/test/oauth.test.ts @@ -231,7 +231,22 @@ describe('rescopeClient', () => { await expect(provider.rescopeClient(clientId, { sourceId: '../etc' })).rejects.toThrow('Invalid source_id'); await expect(provider.rescopeClient(clientId, { federatedRead: ['ok', 'Not Valid!'] })).rejects.toThrow('Invalid source_id'); await expect(provider.rescopeClient(clientId, { federatedRead: [] })).rejects.toThrow('cannot be empty'); - await expect(provider.rescopeClient(clientId, {})).rejects.toThrow('requires --source and/or --federated-read'); + await expect(provider.rescopeClient(clientId, {})).rejects.toThrow('requires --source, --federated-read, and/or --bound-slug-prefixes'); + // v0.42.70.0: an explicit empty prefix list is ambiguous (deny-all) — rejected. + await expect(provider.rescopeClient(clientId, { boundSlugPrefixes: [] })).rejects.toThrow('cannot be an empty list'); + // An empty/whitespace ENTRY matches every slug under startsWith — it would + // look like a binding while fencing nothing. Rejected at every write surface. + await expect(provider.rescopeClient(clientId, { boundSlugPrefixes: [''] })).rejects.toThrow('non-empty'); + await expect(provider.rescopeClient(clientId, { boundSlugPrefixes: ['ok/', ' '] })).rejects.toThrow('non-empty'); + await expect(provider.rescopeClient(clientId, { boundSlugPrefixes: [' ok/'] })).rejects.toThrow('whitespace'); + // A boundary-less entry reads as a character prefix, so it would silently + // cover sibling namespaces (emp-alice -> emp-alice-2/...). + await expect(provider.rescopeClient(clientId, { boundSlugPrefixes: ['emp-alice'] })).rejects.toThrow('must end with'); + await expect(provider.rescopeClient(clientId, { boundSlugPrefixes: ['emp-alice/', 'chan-eng'] })).rejects.toThrow('must end with'); + await expect(provider.registerClientManual( + 'empty-prefix-reject', ['client_credentials'], 'read write', [], 'default', undefined, undefined, + { boundSlugPrefixes: [''] }, + )).rejects.toThrow('non-empty'); await expect(provider.rescopeClient('gbrain_cl_nonexistent', { sourceId: 'wiki' })).rejects.toThrow('No OAuth client found'); // FK: write source must exist in sources(id). await expect(provider.rescopeClient(clientId, { sourceId: 'no-such-source' })).rejects.toThrow('does not exist'); @@ -240,6 +255,40 @@ describe('rescopeClient', () => { const [row] = await sql`SELECT source_id FROM oauth_clients WHERE client_id = ${clientId}`; expect(row.source_id).toBe('default'); }); + + // v0.42.70.0: bound_slug_prefixes rescope — roster churn (channel + // joins/leaves) updates the write fence in place; 'none' (null) clears it. + test('bound_slug_prefixes: replace, leave-untouched, and clear; live tokens pick it up', async () => { + const { clientId, clientSecret } = await provider.registerClientManual( + 'rescope-fence', ['client_credentials'], 'read write', [], 'default', undefined, undefined, { + boundSlugPrefixes: ['emp-carol/'], + }, + ); + const tokens = await provider.exchangeClientCredentials(clientId, clientSecret!, 'read write'); + + // Replace the binding (carol joins chan-eng). + const replaced = await provider.rescopeClient(clientId, { boundSlugPrefixes: ['emp-carol/', 'chan-eng/'] }); + expect(replaced.boundSlugPrefixes).toEqual(['emp-carol/', 'chan-eng/']); + expect(replaced.sourceId).toBe('default'); // untouched + + // The already-issued token sees the new binding on next verification. + const live = await provider.verifyAccessToken(tokens.access_token) as unknown as CoreAuthInfo; + expect(live.boundSlugPrefixes).toEqual(['emp-carol/', 'chan-eng/']); + + // Rescoping another axis leaves the binding untouched — and doesn't even + // name the column, so brains predating it can still rescope --source. + // `undefined` here means "not read this call", distinct from null = unset. + const other = await provider.rescopeClient(clientId, { federatedRead: ['alpha'] }); + expect(other.boundSlugPrefixes).toBeUndefined(); + const stillBound = await provider.verifyAccessToken(tokens.access_token) as unknown as CoreAuthInfo; + expect(stillBound.boundSlugPrefixes).toEqual(['emp-carol/', 'chan-eng/']); + + // null clears it — client returns to unbound full-source write authority. + const cleared = await provider.rescopeClient(clientId, { boundSlugPrefixes: null }); + expect(cleared.boundSlugPrefixes).toBeNull(); + const unfenced = await provider.verifyAccessToken(tokens.access_token) as unknown as CoreAuthInfo; + expect(unfenced.boundSlugPrefixes).toBeUndefined(); + }); }); // --------------------------------------------------------------------------- @@ -311,6 +360,27 @@ describe('verifyAccessToken', () => { expect(authInfo.token).toBe(tokens.access_token); }); + // v0.42.70.0: bound_slug_prefixes threads through token verification on + // the same JOIN as source_id/federated_read, so enforceClientSlugFence + // can fence direct writes without a per-op DB lookup. + test('bound_slug_prefixes threads into AuthInfo; absent binding stays undefined', async () => { + const bound = await provider.registerClientManual( + 'fence-thread-test', ['client_credentials'], 'read write', [], 'default', undefined, undefined, { + boundSlugPrefixes: ['chan-eng/', 'wiki/agents/fence-thread-test/'], + }, + ); + const boundTokens = await provider.exchangeClientCredentials(bound.clientId, bound.clientSecret!, 'read write'); + const boundInfo = await provider.verifyAccessToken(boundTokens.access_token) as unknown as CoreAuthInfo; + expect(boundInfo.boundSlugPrefixes).toEqual(['chan-eng/', 'wiki/agents/fence-thread-test/']); + + const unbound = await provider.registerClientManual( + 'fence-unbound-test', ['client_credentials'], 'read write', + ); + const unboundTokens = await provider.exchangeClientCredentials(unbound.clientId, unbound.clientSecret!, 'read write'); + const unboundInfo = await provider.verifyAccessToken(unboundTokens.access_token) as unknown as CoreAuthInfo; + expect(unboundInfo.boundSlugPrefixes).toBeUndefined(); + }); + test('expired token is rejected', async () => { // Insert a token that's already expired const expiredToken = generateToken('gbrain_at_'); diff --git a/test/submit-agent.test.ts b/test/submit-agent.test.ts index 1ea2a7c3c..ba817ec33 100644 --- a/test/submit-agent.test.ts +++ b/test/submit-agent.test.ts @@ -197,6 +197,37 @@ describe('submit_agent op (v0.38 Slice 3 — remote-callable agent dispatch with const result = await callSubmitAgent(ctx, { prompt: 'go' }); expect(result.dry_run).toBe(true); }); + + // An EXPLICIT [] used to pass both subset loops vacuously and reach the + // worker, which reads empty allowed_tools as "the whole registry" — so a + // client bound to ['search'] got put_page. `??` doesn't substitute for an + // empty array, only for null/undefined. + it('collapses an explicit empty allowed_tools to the binding, not the full registry', async () => { + await seedClient('cursor', { + bound_tools: ['search'], + bound_source_id: 'default', + bound_slug_prefixes: ['wiki/'], + }); + const ctx = makeCtx({ clientId: 'cursor', dryRun: true }); + const result = await callSubmitAgent(ctx, { prompt: 'go', allowed_tools: [] }); + expect(result.dry_run).toBe(true); + expect(result.resolved_tools).toEqual(['search']); + }); + + // Empty prefixes reached the subagent as "use the legacy + // wiki/agents// namespace" — outside every bound prefix. + it('collapses an explicit empty allowed_slug_prefixes to the binding', async () => { + await seedClient('cursor', { + bound_tools: ['put_page'], + bound_source_id: 'default', + bound_slug_prefixes: ['emp-alice/'], + }); + const ctx = makeCtx({ clientId: 'cursor', dryRun: true }); + const result = await callSubmitAgent(ctx, { prompt: 'go', allowed_slug_prefixes: [] }); + // Normalized into the glob the delegated matcher understands, so the + // subagent can write descendants rather than one exact slug. + expect(result.resolved_slug_prefixes).toEqual(['emp-alice/*']); + }); }); describe('allowed_slug_prefixes enforcement', () => {