admin/gbrain
1
0
Fork 0
mirror of https://github.com/garrytan/gbrain.git synced 2026-08-14 00:48:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge origin/master (v0.45.10.0 community fix-wave 2) — re-bump to v0.45.11.0

Browse Source 
Master claimed 0.45.10.0 while this branch was in flight; per the
version-gate (strictly greater than master) the wave re-bumps to
0.45.11.0 across all version locations (VERSION, package.json,
CHANGELOG entry header, openclaw.plugin.json, bootstrap runbook stamp,
regenerated template stamp, CLAUDE.md example cell). CHANGELOG keeps
both entries. README verb-protocol hunk resolved to master's phrasing
(adds since-version provenance); llms bundle, CLI flag registry, and
skills manifest regenerated against the merged tree.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-08-13 15:28:15 -07:00
co-authored by Claude Fable 5
parent 70f7e74c2f ac402f55f8
commit 6d7d9c4632
158 changed files with 10327 additions and 729 deletions
Download Patch File Download Diff File Expand all files Collapse all files
BOOTSTRAP_FOR_AGENTS.md
+63 -7
View File
@@ -1,4 +1,4 @@
<!-- gbrain-runbook-stamp: 0.45.10.0 -->
<!-- gbrain-runbook-stamp: 0.45.11.0 -->
<!-- This stamp must equal the VERSION file at every release; CI enforces it
(scripts/check-bootstrap-tag.sh). `gbrain bootstrap status` compares it to
the installed binary and warns on skew. -->
@@ -50,7 +50,17 @@ platform package manager first (`brew install gh`, `apt install gh`, `dnf instal
`winget install GitHub.cli` per the official instructions); never a piped
curl-to-shell one-liner. Install gbrain ONLY as
`bun install -g github:garrytan/gbrain#latest-stable` — the npm package named
"gbrain" is an unrelated project.
"gbrain" is an unrelated project. (Cloud-sandbox exception: bun's package fetching
is proxy-incompatible there — use the `gbrain bootstrap cloud-setup-script` recipe,
which installs from the same pinned GitHub source through npm.)
⛔ **NEVER FABRICATE TOOLING.** If gh or any preflight binary is missing, blocked
by a sandbox egress proxy, or answering 403s, report that through
`status`/`doctor` output and follow the cloud-sandbox guidance below. Never
hand-roll a gh shim, stub a fake binary into /usr/local/bin, or fake a passing
check — a fabricated tool poisons every later verification, and the one time it
was tried it masked a real silent-persistence failure. The CLI degrades honestly
on its own; your job is to relay, not to bridge.
## Codex preflight (ChatGPT desktop / Codex CLI only)
@@ -99,9 +109,15 @@ you needed; report the count at the end (it feeds the install-time measurement).
4. **Render.** `gbrain bootstrap render` — identity files appear. Show the human
SOUL.md. Existing files are never overwritten (re-runs are safe; `--force`
backs up first).
5. **Skills + brain wiring.** The CLI scaffolds the skill set and registers
`brain/` as the workspace source. Nothing to judge here; relay the output.
6. **Wire the harness.** `gbrain bootstrap hooks --harness <detected>`:
5. **Skills.** `gbrain skillpack scaffold --all` — the CLI scaffolds the skill
set. Nothing to judge here; relay the output.
6. **Wire the harness + register the brain source.** `gbrain bootstrap hooks
--harness <detected>` creates `<workspace>/brain` and prints the exact
`gbrain sources add <source_id> --path <brain> --force` command for THIS
workspace — run it verbatim (don't guess a different id; a guessed id
only surfaces as an FK error at `verify` time, by which point a wrong
guess also blocks the correct id with an `overlapping_path` error). It
also:
- Claude Code: installs per-turn hooks ON by default — do NOT ask; loading the
brain every turn is the whole point of installing gbrain for your agent. Tell
the human it is on and how to turn it off (`GBRAIN_HOOKS=0`, or re-run with
@@ -121,8 +137,9 @@ you needed; report the count at the end (it feeds the install-time measurement).
instead of creating one — verifies it is private and pushes the workspace. A
non-empty repo, or one owned by an org, is refused with a clear message (make an
empty personal repo, or run `gbrain bootstrap attach` for an existing agent
clone). Asks the background-persistence consent (15-minute scan-gated push job;
declining still persists at session end). If the human has no GitHub or declines:
clone). Asks the background-persistence consent (a git post-commit auto-push
plus a 30-minute pull job for multi-machine freshness; declining still persists
via the per-turn and session-end pushes). If the human has no GitHub or declines:
local-only mode with an honest warning; `bootstrap repo` can run any time later.
Note: the per-turn/session push stays deferred until this phase records the
verified repo, so nothing is ever pushed to an unverified-privacy origin.
@@ -139,6 +156,42 @@ initialized), run `gbrain bootstrap attach` instead of the interview/render/repo
phases — it wires this machine (source, hooks, MCP) and verifies. If agent.json
says it is an uninitialized template, proceed with the normal flow from phase 1.
## Cloud sandboxes (claude.ai/code and similar proxied environments)
**How you know:** `gbrain bootstrap status --json` reports
`execution_environment: "cloud-sandbox"` (the CLI detects the documented
signals — the CLAUDE_CODE_REMOTE env var, the proxy-injected token
placeholder). Trust the CLI's detection over your own guesses.
**Expected degradations — these are facts to relay, not bugs to bridge:**
- **No crontab, no surviving background processes.** The VM is reclaimed after
inactivity. The scheduled pull is skipped honestly; the per-turn (Stop hook)
and session-end pushes carry persistence. Decline nothing, fabricate nothing.
- **GitHub GraphQL is always blocked** by the egress proxy, and **REST reaches
only repos attached to the session** — a repo created mid-session is NOT
attached, so `gbrain bootstrap repo` refuses fast in cloud with the flow
that works. Privacy verification falls back to pure git protocol on its own.
- **`git push` works only against the session's working branch.** A user PAT
does not bypass any of this.
- **Only repo-committed files carry into the next session.** `~/.gbrain`,
`~/.claude`, and the gitignored `.claude/settings.local.json` evaporate.
Hooks therefore live in the COMMITTED `.claude/settings.json` (the CLI
writes PATH-resolved, fail-open commands there in cloud); hook config is
snapshotted at session start, so hooks written mid-session activate on the
NEXT session — say so instead of debugging it.
**The correct cloud flow:**
1. The human creates the private repo from a normal machine (or github.com)
and opens the cloud session ON that repo.
2. The environment's setup script installs the gbrain binary — print it with
`gbrain bootstrap cloud-setup-script` and have the human paste it into the
environment config (npm-based; bun's fetching is proxy-incompatible there).
3. Inside the session: `gbrain bootstrap attach`, then
`gbrain bootstrap hooks --harness claude-code` (writes the committed
carrier), commit + push, and tell the human the hooks go live next session.
## Failure modes, and what they actually mean
| Symptom | Real cause | Fix |
@@ -150,6 +203,9 @@ says it is an uninitialized template, proceed with the normal flow from phase 1.
| "bootstrap already running (pid N)" | A concurrent bootstrap holds the lock | Wait or investigate that pid; the lock self-clears when stale. |
| Brain tools fail with a lock error | Another live session's serve owns the database | Close the other session; sequential use is the v1 contract. |
| Hook reports "brain context unavailable" | serve not running or degraded | `gbrain doctor` names it; hooks fail open by design. |
| gh answers 403 "not enabled for this session" | Cloud proxy scoping — the repo is not attached to the session | Expected in cloud; the visibility ladder falls back to git protocol. NEVER shim gh. |
| "crontab: command not found" / cron skipped | Containers and cloud sandboxes ship without a scheduler | Expected; event-driven pushes cover it — the skip message says exactly this. |
| A turn shows "workspace push is FAILING" | The background push is refusing (visibility, secret-scan, or network reasons) | Run `gbrain doctor`; the banner repeats every 30 min until fixed. |
## Hand off
CHANGELOG.md
+99 -8
View File
@@ -2,7 +2,7 @@
All notable changes to GBrain will be documented in this file.
## [0.45.10.0] - 2026-08-12
## [0.45.11.0] - 2026-08-13
**Hermes joins the tested-install club: a real-binary harness now proves gbrain works inside Hermes, and `gbrain friction diff` tells you whether an install problem is the agent's or ours.**
@@ -22,7 +22,7 @@ the registry, the new `gbrain friction diff --base openclaw --compare hermes` tu
friction reports into a comparison instrument: pain unique to one agent is that agent's
contract problem; pain common to both is ours.
## To take advantage of v0.45.10.0
## To take advantage of v0.45.11.0
`gbrain upgrade` is enough — no schema migration.
@@ -64,6 +64,103 @@ contract problem; pain common to both is ours.
- `claw-test --list-agents` no longer races CLI teardown; output is complete and ordered.
- Live runs keep the agent's gbrain children pointed at the run's own hermetic brain even when the surrounding shell exports a database-pointing environment variable — the harness's verification and the agent's work can no longer land in two different places.
- The test real-name guard now correctly distinguishes the public Hermes platform (documented and tested) from private deployment names (still banned).
## [0.45.10.0] - 2026-08-13
**21 more community and maintainer bug fixes. Search answers get more complete, sync gets safer, and doctor learns to warn you before a provider dies.**
This wave continues the v0.45.8.0 cleanup: no new product surface, just fixes. The
standouts: pages created by the idea-extraction cycle were invisible to search (they
were written without search chunks) and now show up like everything else, with a repair
path for existing brains. Query caching now keys on your detail setting, so a compact
answer is never served to a full-detail request. And doctor now warns you loudly if your
brain is pinned to an embedding provider that has announced a shutdown, weeks before it
happens instead of after.
Also riding: the rerank budget fix that landed directly this week. Contributed by @javieraldape.
## To take advantage of v0.45.10.0
`gbrain upgrade` is enough. No schema migration.
1. **Upgrade and check:**
```bash
gbrain upgrade
gbrain doctor
```
2. **If doctor now warns about your embedding provider,** that is the new sunset check
doing its job. It names the provider, the date, and the migration command.
3. **Heal previously-invisible atom pages:**
```bash
gbrain embed --stale
```
4. **Things to watch:** the query cache key version moved, so the first re-ask of a
cached question is a one-time cache miss. If anything else looks wrong, file an issue
with `gbrain doctor` output: https://github.com/garrytan/gbrain/issues
### Itemized changes
**Search and recall**
- Atom pages produced by the extraction cycle are chunked and embedded like every other page, so they appear in search results. Contributed by @awilhite.
- `embed --stale` detects and heals pages that have content but no chunks. Contributed by @Masashi-Ono0611.
- The query cache folds the detail knob into its key, so compact and full-detail answers never cross. Contributed by @time-attack.
- Rerank budget failures are bucketed under their real cause instead of "unknown". Contributed by @javieraldape.
**Sync, import, and write-through**
- Deferred link extraction above the size gate is consumed instead of dropped. Contributed by @time-attack.
- Import error summaries name the failing table and constraint. Contributed by @bo-developing.
- Write-through honors the page's recorded source path instead of recomputing it. Contributed by @JonMcCutchen.
- The managed filing-rules block renders each repo's own taxonomy, not the bundled default. Contributed by @dovstern.
- Timeline extraction no longer splits on bare hyphens inside link labels. Contributed by @time-attack.
- Export scopes tag and raw-data sidecar reads to the page's source. Contributed by @alexey-metaengage.
- Cross-source link targets survive an engine migration. Contributed by @RerankerGuo.
**Doctor and diagnostics**
- A damaged PGLite store is reported as store damage, with runtime problems kept separate, and the verdict requires positive evidence. Contributed by @time-attack.
- New check: brains pinned to an embedding provider with an announced shutdown get a loud warning with the migration path. Contributed by @time-attack.
- Source listing distinguishes unset federation from explicit false. Contributed by @dovstern.
- `put_page` reports push state honestly instead of implying success. Contributed by @dovstern.
- Flow-style skill triggers parse correctly in skill health checks. Contributed by @RerankerGuo.
- Sync-failure records auto-skipped as chronic stay visible to doctor until a human resolves them. Contributed by @RerankerGuo.
**Autopilot and agents**
- The drain worker no longer self-deadlocks at concurrency=1, and its DB reconnect logic is shared with queue operations. Contributed by @time-attack.
- Stale-lock reaping ignores foreign PIDs it did not create. Contributed by @javieraldape.
- Agent jobs resolve their brain source at submit time, not execution time. Contributed by @Masashi-Ono0611.
**OAuth**
- Dynamic client registration accepts `token_ttl_seconds`, clamped to admin policy, and an unset TTL cap now derives from `--token-ttl` instead of a permissive default. Contributed by @time-attack.
**Models**
- The claude-cli recipe lists the Claude 5 family ids the CLI already serves, with pins. Contributed by @clement0909472.
**For contributors**
- The CLI flag registry, one wave rider test, and the bootstrap version stamps were refreshed as part of assembly.
## [0.45.9.0] - 2026-08-12
**Your agent's memory keeps saving itself — even in a cloud sandbox, even on `/exit`, and it tells you the moment it can't.** The paste-in personal-agent install now works first-class in Claude Code's cloud environment, not just on a laptop. The persistence lane got three fixes that matter whether you're local or in the cloud: the workspace push now verifies repo privacy through a portable ladder that keeps working when the sandbox blocks the GitHub API, it runs after every turn (not only at session end, which the harness never fires on `/exit`), and a failed push surfaces on your next turn instead of failing in silence. Setup adapts to where it runs — no more scheduled-job errors on hosts without a scheduler, and no half-created repos in an environment that can't push them.
To take advantage of v0.45.9.0: upgrade and re-run `gbrain bootstrap verify` on each machine — it re-attests the install and now reports the execution environment and any push-health or hygiene issue with the exact one-line fix. Existing installs pick up the per-turn push and the new verification automatically on the binary update; no re-render needed. If you run in a cloud sandbox, `gbrain bootstrap cloud-setup-script` prints the environment setup recipe, and `gbrain bootstrap status --json` now tells you which environment you're in.
### Added
- **Execution-environment detection** — `local`, `cloud-sandbox`, or `ephemeral-container`. Bootstrap, the doctor, and the runbook branch on it so each environment gets honest behavior and honest messages. `gbrain bootstrap status --json` and `gbrain bootstrap verify` both report it.
- **Per-turn workspace persistence.** A debounced, detached push runs after each assistant turn (default every 5 minutes locally, every turn in a reclaimed-VM cloud sandbox), closing the gap where a session that ends on `/exit` — which never fires the session-end hook — could strand committed work. Off-ramp: `GBRAIN_STOP_PUSH=0`; cadence: `GBRAIN_STOP_PUSH_DEBOUNCE_MIN` or `gbrain config set hooks.stop_push_debounce_min <n>`.
- **Same-session push-failure notice.** When a background push is refused or fails, the next turn surfaces it both to the agent and to you directly (not buried where only the model sees it), re-announced at most every 30 minutes until it clears. `gbrain doctor` and `gbrain bootstrap status` name the failing workspace and the fix.
- **`gbrain bootstrap cloud-setup-script`** — prints the ready-to-paste cloud environment setup script that installs the gbrain binary into the environment's cached filesystem so it survives across sessions.
- **`bootstrap_durability_job` doctor check** — presence *and* liveness of the optional background-persistence job, so a job that exists on disk but no longer runs is reported instead of certified healthy.
### Changed
- **Repo-privacy verification is now a portable ladder** (`src/core/repo-visibility.ts`), replacing three separate probes with one: it checks via the GitHub REST API first, then falls back to pure git protocol so verification keeps working where a sandbox proxy blocks the API. It fails closed in both directions — an origin that can't be proven private is refused, and a proven-public origin is always refused. Fresh private verdicts are cached briefly to keep the per-turn push cheap. Escape hatch for self-hosted git you trust (each use warns): `--allow-unverified-remote`, `GBRAIN_ALLOW_UNVERIFIED_REMOTE=1`, or `gbrain config set push.allow_unverified_remote true`; the escape hatch only relaxes an *unverifiable* verdict, never a proven-public one.
- **Cloud sandboxes get a committed hook carrier.** Because a cloud session starts from a fresh clone and never sees the machine-local settings file, cloud installs write hooks into the repo-committed `.claude/settings.json` with a PATH-resolved, fail-open command; local installs keep the gitignored settings file, and the writers guarantee one event never fires from both.
- **Background-persistence copy tells the truth.** The optional job is a git post-commit auto-push plus a 30-minute freshness pull; the interview, docs, and templates now describe exactly that. On a host without a scheduler the pull is skipped with an honest note rather than a failed-install warning.
- The installing-agent runbook gains a hard rule against fabricating tooling (no hand-rolled `gh` shims), a cloud-sandbox section, and the honest degradation matrix for a proxied environment.
### Fixed
- `gbrain bootstrap uninstall` now tears down the background-persistence wiring it installed (scheduled job, the untracked auto-push hook, credential wiring) instead of leaving it behind; the committed helper and agent-rules stay, since those are your repo's content.
- Machine-specific harness wiring (`.mcp.json`, hook-settings backups) is gitignored so it can't be committed into the private brain repo; `gbrain bootstrap verify` warns and gives the one-line fix for installs that already committed it.
- Repo creation is refused inside a cloud sandbox with the flow that actually works (create the repo elsewhere, open the session on it, `gbrain bootstrap attach`) instead of leaving a half-created, unpushable repo.
- Push-status is tracked per workspace, so with more than one brain workspace on a machine, one workspace's success can no longer mask another's failed pushes.
- Hardening pass (both an in-house and a cross-model adversarial review): the privacy ladder never treats an ambiguous authentication challenge as proof a repo is private, the per-turn retry can't turn into an every-turn network storm, remote-supplied text is sanitized before it reaches any agent- or user-visible surface, and stale state from a deleted workspace no longer re-fires notices forever.
## [0.45.8.0] - 2026-08-12
@@ -16936,8 +17033,6 @@ If anything looks off, file at https://github.com/garrytan/gbrain/issues
with `gbrain doctor` output.
## [0.28.11] - 2026-05-07
**Mix providers: OpenAI for text, Voyage for images. One brain, two embedding pipelines.**
@@ -18903,9 +18998,6 @@ React admin dashboard baked into the binary. Seven screens designed through Stev
- `test/oauth.test.ts` ... 34 test cases covering provider: register, getClient, client_credentials exchange, auth_code flow with PKCE, refresh rotation, verifyAccessToken (OAuth + legacy fallback), revokeToken, sweepExpiredTokens, scope annotations on all 30 operations. Plus the post-/cso security-fix regressions: 10-concurrent auth code exchange (only 1 wins), 10-concurrent refresh rotation (only 1 wins), redirect_uri HTTPS-or-loopback gate, and pgArray comma-element round-trip (1 element in → 1 element out).
## [0.25.1] - 2026-05-01
## **Your brain can now read books with you. Nine new skills land at once.**
@@ -20155,7 +20247,6 @@ Then point Claude Desktop, claude.ai/code, or any MCP client at `http://your-tun
If anything breaks: `gbrain doctor`, `~/.gbrain/upgrade-errors.jsonl` (if present), and please file an issue at https://github.com/garrytan/gbrain/issues with both.
## [0.22.6.1] - 2026-04-26
**Old brains can upgrade again.**
CLAUDE.md
+1 -1
View File
@@ -506,7 +506,7 @@ four numeric segments are required first. Historical 3-segment versions
| `CHANGELOG.md` | Top entry header `## [0.31.4.1] - YYYY-MM-DD` plus the "To take advantage of v0.31.4.1" block. | Standard Keep-a-Changelog header. |
| `TODOS.md` | Any TODO entries that mention "follow-up from vX.Y.Z.W" use the version of the release that filed them. Update only when filing NEW follow-up TODOs. | Inline `vX.Y.Z.W` references in TODO bodies. |
| `CLAUDE.md` | The Key Files section's per-file annotations carry `vX.Y.Z.W (#NNN)` tags noting which release introduced a behavior. Update whenever a wave's annotations get folded in. | Inline `vX.Y.Z.W (#NNN, contributed by @user)` references. |
| `openclaw.plugin.json` | OpenClaw plugin manifest (v0.45.6.0, #4033). Hand-maintained; `test/openclaw-plugin-manifest.test.ts` fails the suite if it drifts from `package.json`. Merges from master auto-resolve it to master's version — re-bump it with the trio. | `"version": "0.45.10.0"` |
| `openclaw.plugin.json` | OpenClaw plugin manifest (v0.45.6.0, #4033). Hand-maintained; `test/openclaw-plugin-manifest.test.ts` fails the suite if it drifts from `package.json`. Merges from master auto-resolve it to master's version — re-bump it with the trio. | `"version": "0.45.11.0"` |
| `BOOTSTRAP_FOR_AGENTS.md` | Runbook stamp on line 1. `scripts/check-bootstrap-tag.sh` (in `bun run verify` + CI) fails when it drifts from `VERSION`; refresh it in the same commit as the bump. | `<!-- gbrain-runbook-stamp: X.Y.Z.W -->` |
| `templates/bootstrap/template-repo/` | Vendored template tree with an embedded version stamp. Auto-derived, but NOT by `bun install`: run `bun run scripts/generate-template-repo.ts --out templates/bootstrap/template-repo` after the bump; `scripts/check-bootstrap-templates.sh` fails CI on drift. | `<!-- gbrain-template-stamp: X.Y.Z.W -->` in generated files. |
README.md
+2 -2
View File
@@ -107,7 +107,7 @@ answers. Ask before anything destructive. You are not done until
`gbrain bootstrap verify` exits 0.
```
Everything from the Codex path applies — interview, identity from your own answers, local brain, private repo, keyless mode — plus Claude Code gets **per-turn context hooks** (consent-gated): your brain loads automatically into every prompt, and each session persists itself to your private repo at exit. Restart the session after install and ask "what did I tell you my top jobs were?" — that's the moment it clicks. Full contract, security posture, and uninstall: [docs/guides/bootstrap.md](docs/guides/bootstrap.md).
Everything from the Codex path applies — interview, identity from your own answers, local brain, private repo, keyless mode — plus Claude Code gets **per-turn context hooks**: your brain loads automatically into every prompt, and your work persists to your private repo on a per-turn cadence (debounced ~5 min locally, every turn in a cloud sandbox — this covers the `/exit` case the harness never fires a session-end hook on), with a notice on your next turn if a push ever fails. This works in a **Claude Code cloud session** too, not just on your laptop: verification falls back to pure git protocol when the sandbox blocks the GitHub API, and `gbrain bootstrap cloud-setup-script` prints the environment setup recipe. Restart the session after install and ask "what did I tell you my top jobs were?" — that's the moment it clicks. Full contract, security posture, cloud sandboxes, and uninstall: [docs/guides/bootstrap.md](docs/guides/bootstrap.md).
> **Prefer to make the repo yourself?** Create a new **empty** private repo **under your own GitHub account** (no README/.gitignore/license), clone it, open the clone in Claude Code (CLI or the desktop app's open-a-repo flow), and paste the same block — bootstrap adopts your empty repo instead of creating one. The repo must be empty and personal-account-owned; org-owned repos are refused.
@@ -131,7 +131,7 @@ The agent installs GBrain, creates the brain, asks for your API keys, loads the
### Lighter ways in
**Just want a memory for your coding agent — no identity, no repo.** Spin up a local brain and connect it in two commands — zero server, zero token, zero tunnel. `--surface verbs` gives your agent the seven-verb memory protocol (`recall`, `remember`, `entity`, `synthesize`, `forget`, `context_pack`, `delta` — [MEMORY_VERBS v1](docs/protocol/MEMORY_VERBS_v1.md), frozen + additive-forever) instead of the full tool wall; drop the flag for every operation:
**Just want a memory for your coding agent — no identity, no repo.** Spin up a local brain and connect it in two commands — zero server, zero token, zero tunnel. `--surface verbs` gives your agent the seven-verb memory protocol (`recall`, `remember`, `entity`, `synthesize`, `forget`, plus `context_pack` + `delta` since v0.45.7 — [MEMORY_VERBS v1](docs/protocol/MEMORY_VERBS_v1.md), frozen + additive-forever) instead of the full tool wall; drop the flag for every operation:
```bash
gbrain init --pglite # 2-second local brain (no Docker)
TODOS.md
+42
View File
@@ -5197,6 +5197,48 @@ respective shapes. Small, mechanical; pinned by `test/init-embed-check.test.ts`
## Agent-bootstrap wave follow-ups (filed at build time)
- [ ] **P2 — repoPhaseComplete is single-workspace (one global receipt).** The
no-daemon push gate binds to the one `receipt.repo_url`, so with two bootstrap
workspaces sharing a gbrain home, workspace B's `bootstrap repo` overwrites the
receipt and permanently leaves A's per-turn/session-end pushes at
`push_deferred_repo_pending`. Fails CLOSED (defers, never mis-pushes) and
matches the v1 single-workspace contract, but the per-turn push made it more
visible. Fix = per-root repo binding (a receipt map or a per-root marker).
Surfaced by both v0.45.9.0 adversarial reviewers.
- [ ] **P2 — visibility ladder subprocess/body bounds.** `runWithTimeout`
(`src/core/repo-visibility.ts`) races the `gh`/`git` probe against a timer but
doesn't kill the raced child, and the anon-probe `res.text()` buffers the whole
(operator-configured-origin) body before slicing. Bounded in practice by the
detached push child's lifetime, but a proper fix kills the raced process and
caps the body read. Filed from the v0.45.9.0 Codex adversarial pass.
- [ ] **P3 — `config set` for the file-plane hook-lane keys is engine-bound.**
`runConfig` dispatches through the engine path, so `gbrain config set
push.allow_unverified_remote true` can fail while a live PGLite serve holds the
writer lock — the documented recovery command, unavailable exactly when needed.
The env-var form (`GBRAIN_ALLOW_UNVERIFIED_REMOTE=1`) is the cloud path and needs
no engine, so this is convenience-only; fix = route these two keys through the
no-engine CLI dispatch. Filed from the v0.45.9.0 Codex adversarial pass.
- [ ] **P3 — plugin-based hook distribution for Claude Code.** Ship gbrain's
hooks as a Claude Code plugin (`hooks/hooks.json` + `.claude-plugin/plugin.json`
manifest, installed via the plugin marketplace flow) instead of two settings
files. Plugins merge hooks first-class across scopes and update centrally —
it would REPLACE both current carriers (repo-committed `.claude/settings.json`
for cloud installs + gitignored `settings.local.json` for local), so it must
migrate, not join; a third simultaneous carrier would double-fire events.
Cons: needs marketplace repo hosting; enterprise `allowManagedHooksOnly`
policies can block plugin hooks entirely. Start at
`src/core/bootstrap/hooks.ts` (both writers + the dedupe rule live there).
Filed from the cloud-DX eng review (v0.46.x wave).
- [ ] **P3 — watch Claude Code Channels as the push path for
volunteer_context/signals.** Channels (research preview) push external events
into a LIVE session — the native version of gbrain's push-context lane
(`docs/guides/push-context.md`). Not actionable today: delivery requires an
always-on session plus an Anthropic-allowlisted channel plugin. Revisit when
channel-plugin distribution opens; the win is replacing per-turn pull with
event push for signals/reflex windows. Filed from the cloud-DX eng review.
- [ ] **P1 — enforce op scope/localOnly on the stdio MCP dispatch when no auth
context is present, and consider a narrower default surface for pull-mode
harness registrations.** HTTP dispatch enforces `scope`/`localOnly` before
VERSION
+1 -1
View File
@@ -1 +1 @@
0.45.10.0
0.45.11.0
docs/TESTING.md
+1
View File
@@ -176,6 +176,7 @@ Unit tests and what they cover:
- `test/watch-command.test.ts` — `gbrain watch` push transport (#2095): streaming loop, rolling window, session dedupe, `--json` JSONL shape, `channel: 'watch'` event logging, clean EOF return. Hermetic PGLite + injected line/write deps (no subprocess, no real stdin).
- `test/watch-sigint.serial.test.ts` — `gbrain watch` SIGINT lifecycle against a real spawned CLI subprocess with a tmpdir brain. SERIAL: parallel unit shards flake on concurrent subprocess spawns (same rationale as `apply-migrations-pglite-spawn.serial.test.ts`).
- `test/autopilot-launchd-lifecycle.serial.test.ts` — autopilot lifecycle behavior, not generated-string assertions: the full install → self-disable → status → reinstall → uninstall arc with `launchctl` replaced by an argv recorder and the generated wrapper executed by a REAL bash against a genuinely deleted repo (every platform), plus a darwin-only fail-SKIP describe against the real launchd under a per-run unique label (`GBRAIN_AUTOPILOT_LABEL`) so it can never collide with — or tear down — a real install on the host. Serial: spawns subprocesses and pins HOME/GBRAIN_HOME for the whole file.
- `test/autopilot-fanout.test.ts` — Autopilot fan-out and #4046 policy regression: targeted idempotency keys reopen per dispatch interval while stable doctor/remediate keys remain unchanged; the 60-minute full-cycle floor wins with a remaining small plan, and an all-fresh restart check advances the process-local clock without masking failed stale-source submissions.
- `test/agent-scheduler-contract.serial.test.ts` — the documented external agent-scheduler shell chain (`gbrain sync --repo X && gbrain embed --stale`, live-sync.md / INSTALL_FOR_AGENTS.md Step 7) driven end-to-end through a real `/bin/sh` against a keyless PGLite brain: the `&&` short-circuit IS the contract (argv arrays can't exercise it), the keyless bare stale embed exits 0, and the pull-failure case that must break the chain does. Anti-vacuity: the fixture commits a real page and every read-back asserts pages >= 1. Serial: real spawned CLI + tmpdir HOME.
- `test/cli-format-volunteer.test.ts` — `formatResult`'s `volunteer_context` human rendering: pointer lines with confidence/arm/rationale, the empty-result message, the approximate stats summary.
- `test/config.test.ts` — config redaction.
docs/ai-providers/zeroentropy.md
+18
View File
@@ -1,5 +1,23 @@
# ZeroEntropy — zembed-1 + zerank-2
> **Hosted API shutdown: 2026-09-04.** ZeroEntropy announced (2026-07-24)
> that its hosted endpoints — `/models/embed` and `/models/rerank` — shut
> down on that date. A brain still embedding through the hosted API loses
> semantic retrieval entirely on that date: query embedding uses the same
> endpoint, so **existing vectors become unqueryable**, not just new
> content. Two fixes, either works:
>
> 1. **Self-host the same model** — zembed-1 weights are Apache-2.0. Serve
> them via `llama-server` or Ollama and point the config at the local
> endpoint. Keeps every existing vector; no re-embed at all.
> 2. **Migrate to another provider** — `gbrain migrate embeddings --to
> <provider:model> --dim <N> --dry-run` (resumable; see
> [the migration guide](../guides/embedding-migration.md)). `gbrain
> doctor` (check `provider_sunset`) prints this command with your
> brain's actual `--dim` filled in.
>
> The hosted setup below remains accurate until the shutdown date.
[ZeroEntropy](https://zeroentropy.dev) ships two specialized small models
for retrieval pipelines:
docs/architecture/KEY_FILES.md
+14 -16
View File
File diff suppressed because one or more lines are too long
docs/guides/bootstrap.md
+39 -5
View File
@@ -20,9 +20,11 @@ follows is `BOOTSTRAP_FOR_AGENTS.md` at the repo root, fetched at the
| `agent.json` manifest + `brain/`, `memory/`, `skills/`, `state/` | workspace | — |
| Local brain (PGLite) | `~/.gbrain/` (never in the repo) | while a session's MCP serve is open |
| MCP registration (`gbrain serve`) | Claude Code: project scope by default; Codex: user-global (no scope flag) | spawned by your harness per session |
| Hooks (Claude Code, ON by default) | `.claude/settings.local.json` (gitignored) | each prompt; fail-open; `--no-hooks` opts out at install, `GBRAIN_HOOKS=0` disables at runtime |
| Session persistence | SessionEnd hook → scan-gated commit+push | at session end |
| Optional 15-min push job | launchd/cron (consent-gated) | while logged in |
| Hooks (Claude Code, ON by default) | local installs: `.claude/settings.local.json` (gitignored); cloud sandboxes: the COMMITTED `.claude/settings.json` (PATH-resolved, fail-open commands) | each prompt; fail-open; `--no-hooks` opts out at install, `GBRAIN_HOOKS=0` disables at runtime |
| Per-turn persistence | Stop hook → debounced, detached scan-gated push (per workspace; 5 min default, every turn in cloud sandboxes) | after each assistant turn; `GBRAIN_STOP_PUSH=0` disables; `GBRAIN_STOP_PUSH_DEBOUNCE_MIN` / config `hooks.stop_push_debounce_min` tune it |
| Session persistence | SessionEnd hook → scan-gated commit+push | at session end (note: the harness never fires SessionEnd on `/exit` — the per-turn push is what covers that) |
| Push-failure visibility | next turn's context + a user-visible notice; re-announces every 30 min while failing | whenever a background push fails |
| Optional background job (consent-gated) | git post-commit auto-push + launchd/cron 30-min pull (pull job skipped honestly on hosts without a scheduler) | while logged in |
| Private GitHub repo | your account, created by `bootstrap repo` (or an empty repo you made yourself, adopted) | privacy verified via API |
| Machine receipt | `~/.gbrain/bootstrap/receipt.json` | uninstall is keyed to it |
@@ -30,6 +32,36 @@ follows is `BOOTSTRAP_FOR_AGENTS.md` at the repo root, fetched at the
schedules fire at turn/session boundaries only. True 24/7 operation is what a
hosted brain provides — this is the honest desktop contract.
## Cloud sandboxes (claude.ai/code and similar)
Cloud sessions run in a reclaimed-after-inactivity VM behind a
credential-injecting egress proxy. `gbrain bootstrap status --json` reports
`execution_environment: "cloud-sandbox"` there, and the install adapts:
- **Hooks live in the committed `.claude/settings.json`** with PATH-resolved,
fail-open commands (no machine paths). The gitignored local settings file
never survives into the next session's fresh clone, and hook config is
snapshotted at session start — so hooks written mid-session go live on the
NEXT session. Commit and push the file.
- **The per-turn push runs every turn** (debounce 0) — a reclaimed VM's tail
loss is permanent, so each turn banks to the private repo.
- **Repo-privacy verification falls back to pure git protocol** when the proxy
blocks the GitHub API (GraphQL is always pinned there; REST reaches only
session-attached repos). Confirmed-public origins still always refuse.
- **Repo creation is refused in cloud** with the flow that works: create the
private repo from a normal machine or github.com, open the cloud session ON
that repo, run `gbrain bootstrap attach`.
- **The gbrain binary installs via the environment setup script** — print it
with `gbrain bootstrap cloud-setup-script` and paste it into the environment
config (npm-based; bun's package fetching is proxy-incompatible there).
- **No scheduler exists** — the consent-gated pull job is skipped with an
honest message; event-driven pushes cover persistence.
Escape hatch for self-hosted git you trust (every use warns loudly):
the CLI flag on `sources push`, `GBRAIN_ALLOW_UNVERIFIED_REMOTE=1`, or
`gbrain config set push.allow_unverified_remote true` (file-plane — the only
form that reaches detached hook children inside a sandbox).
## Bring your own repo (create-repo-first)
By default bootstrap creates the private GitHub repo for you. If you prefer to own
@@ -91,8 +123,10 @@ zero in keyless mode; with a key, the standard spend gates apply
contract. Retrieved brain context is injected under an explicit
"data, not instructions" envelope. Facts visible to the harness respect the
brain's visibility tiers.
- **Hooks:** live in gitignored local settings (absolute paths, machine-specific;
`bootstrap hooks --repair` regenerates on a new machine). Every hook fails open
- **Hooks:** on a local install, gitignored local settings (absolute paths,
machine-specific; `bootstrap hooks --repair` regenerates on a new machine); in a
cloud sandbox, the committed `.claude/settings.json` (PATH-resolved, fail-open —
see the Cloud sandboxes section). Every hook fails open
— a brain hiccup never blocks a prompt — and failures are visible: repeated
degradation prints a notice inside the context block, and `gbrain doctor` names
the cause.
docs/guides/embedding-migration.md
+37
View File
@@ -27,6 +27,37 @@ gbrain migrate embeddings --to voyage:voyage-3-large --yes
declared width and is required for recipes that don't declare one (litellm,
llama-server, and other bring-your-own-model providers).
**Pick `--dim` = your brain's current column width when the target supports
it.** A different width triggers the destructive schema transition (column +
index rebuild across all three dim-pinned tables); the same width skips it
entirely. `gbrain doctor` (check `provider_sunset`, for providers with an
announced shutdown) prints the paste-ready command with your actual width
already filled in — it reads the real `vector(N)` column, not the config
value, which can drift.
## How affected brains find out (provider sunsets)
Two surfaces flag a brain whose embedding model (or reranker) is on a
provider with an announced hosted-API shutdown, such as ZeroEntropy
(2026-09-04):
- **`gbrain doctor`** — the `provider_sunset` check warns on every run until
the brain is off the provider. After the shutdown date it escalates to
`fail` only when embedded vectors actually exist on the dead provider
(retrieval is genuinely down); a zero-vector brain whose config merely
resolves to the dead default stays `warn`, so doctor-as-CI-gate setups
don't start exiting 1 on the date. The reranker side resolves through the
same plane search actually reranks with (the mode bundle +
`search.reranker.*` overrides). The message carries the paste-ready
migration command with the brain's actual `--dim`. Accepted the risk?
`gbrain config set doctor.suppress_provider_sunset true` silences it.
- **`gbrain upgrade`** — a one-shot banner (gated by
`ze_sunset_notice_shown`) with the same two fixes.
Both state the full consequence: after the shutdown, **existing vectors
become unqueryable** — query embedding uses the same endpoint as ingestion —
not just new content.
## What it does, in order
1. **Plan.** Counts every chunk not already in the target embedding space —
@@ -88,6 +119,12 @@ continues where it stopped. An in-flight marker (`embedding_migration.state`
in DB config) records the target; it is cleared only when the backlog drains
to zero.
One caveat after a HARD kill (SIGKILL, crash, power loss — not Ctrl-C): the
run's per-source single-flight embed lock is left behind, and an immediate
re-run skips the re-embed and reports the migration as paused. The command
says so explicitly (`lock_skipped` in `--json`); the lock expires on its own
after at most 60 minutes, then the same re-run resumes normally.
A page whose chunks straddle two stale batches is embedded correctly but not
stamped by the embed loop (which only stamps all-or-nothing per batch), so the
migration runs one reconcile pass after the drain that stamps every
docs/integrations/embedding-providers.md
+3 -1
View File
@@ -23,7 +23,7 @@ The resolved provider + dimensions get persisted to `~/.gbrain/config.json` atom
| Provider | env vars | default dims | cost ($/1M tokens) | local? | multimodal? |
|---|---|---|---|---|---|
| `zeroentropyai` | `ZEROENTROPY_API_KEY` | 2560 (Matryoshka to 1280/640/320/...) | 0.05 | no | no |
| `zeroentropyai` (hosted API **shuts down 2026-09-04** — see note below) | `ZEROENTROPY_API_KEY` | 2560 (Matryoshka to 1280/640/320/...) | 0.05 | no | no |
| `openai` | `OPENAI_API_KEY` | 1536 | 0.13 | no | no |
| `openrouter` | `OPENROUTER_API_KEY` | 1536 | 0.02 | no | model-dependent |
| `voyage` | `VOYAGE_API_KEY` | 1024 | 0.18 | no | yes (`voyage-multimodal-3`) |
@@ -42,6 +42,8 @@ The resolved provider + dimensions get persisted to `~/.gbrain/config.json` atom
**Note on local providers.** Ollama and llama-server have no required API key, so they don't show up in env-detection auto-pick. Pick them explicitly with `--embedding-model ollama:<model>` to avoid silently routing to a daemon that may not be running.
**Note on the ZeroEntropy hosted API.** ZeroEntropy announced (2026-07-24) that its hosted endpoints shut down on **2026-09-04**. A brain still embedding through the hosted API loses semantic retrieval entirely on that date — query embedding uses the same endpoint, so existing vectors become unqueryable, not just new content. Either self-host the Apache-2.0 zembed-1 weights via llama-server/Ollama (keeps every existing vector, no re-embed), or migrate with `gbrain migrate embeddings` — see [the migration guide](../guides/embedding-migration.md). `gbrain doctor` (check `provider_sunset`) flags affected brains and prints the paste-ready command with the brain's actual `--dim` filled in.
## If first import fails
If `gbrain import` fails with `expected N dimensions, not M`, run `gbrain doctor`. The output will print the exact `gbrain config set ...` or `gbrain retrieval-upgrade` command to repair the mismatch. **You should not need to delete `~/.gbrain`.** The bug-class that historically forced `rm -rf` recoveries is closed as of v0.37.
docs/mcp/DEPLOY.md
+16
View File
@@ -163,6 +163,22 @@ await oauthProvider.registerClientManual(
For self-service client registration (Dynamic Client Registration, RFC 7591),
start the server with `--enable-dcr`. DCR is off by default.
DCR requests may include an optional `token_ttl_seconds` field (integer,
seconds) to request a per-client access-token lifetime. The server clamps the
request into an admin-configured window — never rejects over it — persists the
effective value as the client's TTL override, and echoes it back as
`token_ttl_seconds` in the registration response. Subsequent `/token` responses
for that client carry the matching `expires_in`. Clients that omit the field
keep the server default (`--token-ttl`). The window defaults fail-closed: min
300 seconds, max bounded by your `--token-ttl` — a self-registering client
cannot request a longer-lived token than the server default unless you
explicitly widen the window:
```bash
gbrain config set oauth.dcr_ttl_min_seconds 600
gbrain config set oauth.dcr_ttl_max_seconds 86400
```
### 3. Expose the server
**Bind explicitly.** `gbrain serve --http` defaults to `127.0.0.1`.
docs/progress-events.md
+4
View File
@@ -135,6 +135,10 @@ sync that calls import emits `sync.import.<file>`, not `import.<file>`.
Stable phase names shipped in v0.15.2:
- `doctor.db_checks` (umbrella for all DB-side doctor checks)
- `doctor.pglite_probe` (the #2674 scratch-store probe; only when PGLite init
failed with an unexplained/damage-class disk state or `--probe-pglite` was
passed — a cold start can take 5–20s, so the heartbeat is the only sign of
life)
- `orphans.scan`
- `embed.pages`
- `extract.links_fs`, `extract.timeline_fs`, `extract.links_db`, `extract.timeline_db`
llms-full.txt
+19 -3
View File
@@ -661,7 +661,7 @@ four numeric segments are required first. Historical 3-segment versions
| `CHANGELOG.md` | Top entry header `## [0.31.4.1] - YYYY-MM-DD` plus the "To take advantage of v0.31.4.1" block. | Standard Keep-a-Changelog header. |
| `TODOS.md` | Any TODO entries that mention "follow-up from vX.Y.Z.W" use the version of the release that filed them. Update only when filing NEW follow-up TODOs. | Inline `vX.Y.Z.W` references in TODO bodies. |
| `CLAUDE.md` | The Key Files section's per-file annotations carry `vX.Y.Z.W (#NNN)` tags noting which release introduced a behavior. Update whenever a wave's annotations get folded in. | Inline `vX.Y.Z.W (#NNN, contributed by @user)` references. |
| `openclaw.plugin.json` | OpenClaw plugin manifest (v0.45.6.0, #4033). Hand-maintained; `test/openclaw-plugin-manifest.test.ts` fails the suite if it drifts from `package.json`. Merges from master auto-resolve it to master's version — re-bump it with the trio. | `"version": "0.45.10.0"` |
| `openclaw.plugin.json` | OpenClaw plugin manifest (v0.45.6.0, #4033). Hand-maintained; `test/openclaw-plugin-manifest.test.ts` fails the suite if it drifts from `package.json`. Merges from master auto-resolve it to master's version — re-bump it with the trio. | `"version": "0.45.11.0"` |
| `BOOTSTRAP_FOR_AGENTS.md` | Runbook stamp on line 1. `scripts/check-bootstrap-tag.sh` (in `bun run verify` + CI) fails when it drifts from `VERSION`; refresh it in the same commit as the bump. | `<!-- gbrain-runbook-stamp: X.Y.Z.W -->` |
| `templates/bootstrap/template-repo/` | Vendored template tree with an embedded version stamp. Auto-derived, but NOT by `bun install`: run `bun run scripts/generate-template-repo.ts --out templates/bootstrap/template-repo` after the bump; `scripts/check-bootstrap-templates.sh` fails CI on drift. | `<!-- gbrain-template-stamp: X.Y.Z.W -->` in generated files. |
@@ -1698,7 +1698,7 @@ answers. Ask before anything destructive. You are not done until
`gbrain bootstrap verify` exits 0.
```
Everything from the Codex path applies — interview, identity from your own answers, local brain, private repo, keyless mode — plus Claude Code gets **per-turn context hooks** (consent-gated): your brain loads automatically into every prompt, and each session persists itself to your private repo at exit. Restart the session after install and ask "what did I tell you my top jobs were?" — that's the moment it clicks. Full contract, security posture, and uninstall: [docs/guides/bootstrap.md](docs/guides/bootstrap.md).
Everything from the Codex path applies — interview, identity from your own answers, local brain, private repo, keyless mode — plus Claude Code gets **per-turn context hooks**: your brain loads automatically into every prompt, and your work persists to your private repo on a per-turn cadence (debounced ~5 min locally, every turn in a cloud sandbox — this covers the `/exit` case the harness never fires a session-end hook on), with a notice on your next turn if a push ever fails. This works in a **Claude Code cloud session** too, not just on your laptop: verification falls back to pure git protocol when the sandbox blocks the GitHub API, and `gbrain bootstrap cloud-setup-script` prints the environment setup recipe. Restart the session after install and ask "what did I tell you my top jobs were?" — that's the moment it clicks. Full contract, security posture, cloud sandboxes, and uninstall: [docs/guides/bootstrap.md](docs/guides/bootstrap.md).
> **Prefer to make the repo yourself?** Create a new **empty** private repo **under your own GitHub account** (no README/.gitignore/license), clone it, open the clone in Claude Code (CLI or the desktop app's open-a-repo flow), and paste the same block — bootstrap adopts your empty repo instead of creating one. The repo must be empty and personal-account-owned; org-owned repos are refused.
@@ -1722,7 +1722,7 @@ The agent installs GBrain, creates the brain, asks for your API keys, loads the
### Lighter ways in
**Just want a memory for your coding agent — no identity, no repo.** Spin up a local brain and connect it in two commands — zero server, zero token, zero tunnel. `--surface verbs` gives your agent the seven-verb memory protocol (`recall`, `remember`, `entity`, `synthesize`, `forget`, `context_pack`, `delta` — [MEMORY_VERBS v1](docs/protocol/MEMORY_VERBS_v1.md), frozen + additive-forever) instead of the full tool wall; drop the flag for every operation:
**Just want a memory for your coding agent — no identity, no repo.** Spin up a local brain and connect it in two commands — zero server, zero token, zero tunnel. `--surface verbs` gives your agent the seven-verb memory protocol (`recall`, `remember`, `entity`, `synthesize`, `forget`, plus `context_pack` + `delta` since v0.45.7 — [MEMORY_VERBS v1](docs/protocol/MEMORY_VERBS_v1.md), frozen + additive-forever) instead of the full tool wall; drop the flag for every operation:
```bash
gbrain init --pglite # 2-second local brain (no Docker)
@@ -4137,6 +4137,22 @@ await oauthProvider.registerClientManual(
For self-service client registration (Dynamic Client Registration, RFC 7591),
start the server with `--enable-dcr`. DCR is off by default.
DCR requests may include an optional `token_ttl_seconds` field (integer,
seconds) to request a per-client access-token lifetime. The server clamps the
request into an admin-configured window — never rejects over it — persists the
effective value as the client's TTL override, and echoes it back as
`token_ttl_seconds` in the registration response. Subsequent `/token` responses
for that client carry the matching `expires_in`. Clients that omit the field
keep the server default (`--token-ttl`). The window defaults fail-closed: min
300 seconds, max bounded by your `--token-ttl` — a self-registering client
cannot request a longer-lived token than the server default unless you
explicitly widen the window:
```bash
gbrain config set oauth.dcr_ttl_min_seconds 600
gbrain config set oauth.dcr_ttl_max_seconds 86400
```
### 3. Expose the server
**Bind explicitly.** `gbrain serve --http` defaults to `127.0.0.1`.
openclaw.plugin.json
+1 -1
View File
@@ -1,7 +1,7 @@
{
"id": "gbrain-context-engine",
"name": "gbrain",
"version": "0.45.10.0",
"version": "0.45.11.0",
"description": "Personal knowledge brain with Postgres + pgvector hybrid search",
"family": "bundle-plugin",
"configSchema": {
package.json
+1 -1
View File
@@ -154,7 +154,7 @@
"bun": ">=1.3.10"
},
"license": "MIT",
"version": "0.45.10.0",
"version": "0.45.11.0",
"overrides": {
"@hono/node-server": "^2.0.5",
"fast-uri": "^3.1.5",
skills/citation-graph-ingest/SKILL.md
+1 -1
View File
@@ -159,7 +159,7 @@ mismatch, typo'd `--type`) before reporting anything.
```bash
gbrain link-sources # citation-graph should appear with the expected count
gbrain check-backlinks # confirm no orphaned references
gbrain check-backlinks check # confirm no orphaned references
```
## Run it (worked example, synthetic fixture)
skills/company-brainify/SKILL.md
+28 -13
View File
@@ -139,13 +139,17 @@ cd "$BRAIN"
gbrain recall --grep "salary"
```
Collect every returned slug into the scope list.
Resolve every returned slug to its repo-relative file path and write the
paths into `/tmp/brainify-scope.txt` (one per line). This file is the
scope list; the structural pass below APPENDS to it — nothing later in
the procedure may truncate it, or the retrieval-discovered pages
silently drop out of scope.
2. Structural discovery — people files that belong to the company, plus
keyword hits across the wider scan scope:
```bash
grep -rli 'company: *"acme-example"' people/ --include="*.md" | sort > /tmp/brainify-scope.txt
grep -rli 'company: *"acme-example"' people/ --include="*.md" | sort >> /tmp/brainify-scope.txt
grep -rli -E 'salary|equity|carry|retention|underperform|performance review|hard conversation' \
meetings/ daily/ companies/ projects/ analysis/ --include="*.md" 2>/dev/null >> /tmp/brainify-scope.txt
sort -u -o /tmp/brainify-scope.txt /tmp/brainify-scope.txt
@@ -254,16 +258,22 @@ For sanitization, sensitive fact rows must be ACTUALLY REMOVED: find them
(`gbrain recall --grep`), then delete the row from the page's Facts fence
(step 5), exactly like a sensitive take. On an in-place shared brain, the
page edit must then be re-synced (`gbrain sync` re-imports the edited page)
so the shared database no longer serves the row — an edited page over an
un-synced DB still leaks through retrieval. `forget` alone can never certify
a brain clean.
AND the facts index reconciled — sync's convergence contract covers page
import only; downstream fact extraction is explicitly decoupled
(`src/commands/sync.ts`, "CONVERGENCE CONTRACT"), so the DB keeps serving
the deleted row until the extract-facts reconcile runs. Trigger it
(`gbrain sweep`, or wait for the serve-resident sweep), then confirm with
`gbrain recall --grep` that the row is actually gone. An edited page over
an un-reconciled facts index still leaks through retrieval. `forget` alone
can never certify a brain clean.
After edits: on the **staging-copy** path the fact rows are removed by editing
the copied markdown directly (there is no live DB to re-sync yet — the team DB
is built fresh when Phase 5 Step 0 turns the export into a source). On the
**in-place shared-brain** path, `gbrain sync` re-imports the changed pages so
the DB matches the markdown. Either way, run `gbrain check-backlinks check` to
catch pages still pointing at removed content.
**in-place shared-brain** path, run `gbrain sync` so the page content matches
the markdown, then reconcile and verify the facts index as above. Either way,
run `gbrain check-backlinks check` to catch pages still pointing at removed
content.
### Phase 4: Verify
@@ -502,7 +512,10 @@ recovery line.
mirror-clone backup in `~/.gbrain/backups/` for a retention window
(~30 days is a sane default), then delete it — it contains the
pre-sanitization history and should not accumulate indefinitely:
`rm -rf ~/.gbrain/backups/brain-history-backup-<date>.git`
`rm -rf ~/.gbrain/backups/shared-brain-history-backup-<date>.git`
(the glob must match the `shared-brain-history-backup-*` name the backup
step created — a mismatched pattern deletes nothing and silently retains
the pre-sanitization history forever)
- If the repo carries push hooks or auto-hardening wiring, re-verify remotes
and hooks survived the rewrite before handing the repo to the team
@@ -592,9 +605,11 @@ This skill guarantees:
covered by the sanitization scan; everything else is excluded by default,
and the Phase 4 verification greps run against the exported tree before
the first push.
- Sensitive fact rows are deleted from the page's Facts fence and re-synced,
never merely expired — `gbrain forget` retains the row (struck through,
served via `--include-expired`) and can never certify clean.
- Sensitive fact rows are deleted from the page's Facts fence, re-synced,
and the facts index reconciled (extract-facts sweep) with the removal
verified via `gbrain recall --grep`, never merely expired — `gbrain
forget` retains the row (struck through, served via `--include-expired`)
and can never certify clean.
- The history-purge filter list and its restore manifest both derive from
the COMPLETE set of sanitized paths, never a subset.
- Every strip decision is a per-file model judgment grounded in a full read;
@@ -623,7 +638,7 @@ Three artifacts:
- Scope: [N files scanned across people/, meetings/, daily/, ...]
- Flagged: [M files with hits] (triage list attached)
- Edited: [K files sanitized; T takes removed; F fact rows removed + re-synced]
- Edited: [K files sanitized; T takes removed; F fact rows removed + re-synced + facts index reconciled]
- Verification: [grep residuals: 0 confirmed-sensitive; retrieval checks: clean]
- History: [not purged | fresh-export | purged after confirmed gate — backup at <path>]
- Next re-audit: [date / cron slot]
skills/skills.lock.json
+3 -3
View File
@@ -3,7 +3,7 @@
"_AGENT_README.md": "62613f7f1e061576b6c1b18844f59bd35f2df96ca5c45c8c41fae0772b9ce4d3",
"_brain-filing-rules.json": "cf850df6a7425464c6d63b3ace71991cc93497fa0cc8cd21acd31883e17939c6",
"_brain-filing-rules.md": "2d2d75b7c76081c56f41b2c0a5a978c355ce957300f9b0a5575dc4079ef1f877",
"_friction-protocol.md": "1b6e7cfa58725a6a5dc2dc787242141bc33f5fde524540d85b14ec22266140f7",
"_friction-protocol.md": "51353207240142024ff1facc25f225712275ecdb4a034ffffdd83740c8d328e3",
"_output-rules.md": "0722ec2ecea7f9fa2f065cf12dfe1347956a9709d29898bf9fe95e875c64b800",
"academic-verify/SKILL.md": "1c19e27e75249d869da428ce8d060075feef8fbbfe146af58b305d11a260ebbc",
"academic-verify/routing-eval.jsonl": "90d894a9829d9936e6ac7a6507e4de67ad26e46a1fe13b7a34e7dec1c0d887dd",
@@ -33,10 +33,10 @@
"capture/SKILL.md": "98568ac96331f57397ea072749641d9748b1ce31e8b09d512b8db25c8fcda65f",
"citation-fixer/SKILL.md": "abdadbf0740a529b9c4f86f05bba416417624503fdcbc6054402d5546afd08b4",
"citation-fixer/routing-eval.jsonl": "52b23b71e66fdc18aee67d0576099b0c83997d648cf4ecf8fe7753b91b6c9c53",
"citation-graph-ingest/SKILL.md": "6510856cc14a653dcade510702890343bc0527de14bc2f1ed0d2524f248c798c",
"citation-graph-ingest/SKILL.md": "849b0cdc64b7ff14d0e6771bde15f0edc3c2fc29af08be015753a5f88a03205f",
"citation-graph-ingest/routing-eval.jsonl": "a1ba605d35e736b741b9e8aac1e7d50b61a7cbcada893d67099b55bf5a0d2635",
"cold-start/SKILL.md": "20be3d1b637621fd9fbd268072f6647533a23f596e30cb593523b051708aaddd",
"company-brainify/SKILL.md": "2c058b39f5364b8ceb5c53b4525cce8645734f16cc3c229a490b005d58a78311",
"company-brainify/SKILL.md": "ae48372512645f532820e43faaf18a8fa768a691b2144973dfc89465f84d84c6",
"company-brainify/routing-eval.jsonl": "6f27f835eda9ae77a2b694534c78a043a871349820e8c638c3d8bbba6d3aa17b",
"concept-synthesis/SKILL.md": "ed02d2e385143b16a1e69ee5934288fb4d0b755f68c4312faff663e6b2d7c4ed",
"concept-synthesis/routing-eval.jsonl": "96dbd7d9c1b606e9e06262d0c06282399741e2bccb8eeb7b9ca88c20f44cda0e",
src/cli.ts
+1 -1
View File
@@ -3028,7 +3028,7 @@ SETUP
migrate embeddings --to <p:model> Re-embed onto another embedding provider
upgrade Self-update
check-update [--json] Check for new versions
doctor [--json] [--fast] Health check (resolver, skills, pgvector, RLS, embeddings)
doctor [--json] [--fast] [--probe-pglite] Health check (resolver, skills, pgvector, RLS, embeddings; --probe-pglite runs the scratch-store probe)
integrations [subcommand] Manage integration recipes (senses + reflexes)
PAGES
src/commands/agent.ts
+82 -3
View File
@@ -18,6 +18,8 @@ import type { BrainEngine } from '../core/engine.ts';
import { MinionQueue } from '../core/minions/queue.ts';
import { waitForCompletion, TimeoutError } from '../core/minions/wait-for-completion.ts';
import type { MinionJobInput, SubagentHandlerData, AggregatorHandlerData } from '../core/minions/types.ts';
import { resolveSourceId, ALL_SOURCES } from '../core/source-resolver.ts';
import { fetchSource } from '../core/sources-load.ts';
import { runAgentLogs } from './agent-logs.ts';
// ── arg parsing helpers ────────────────────────────────────
@@ -72,6 +74,10 @@ SUBMITTING
--max-turns <n> Max assistant turns (default 20)
--tools a,b,c Subset of registered tool names (comma list)
--timeout-ms <n> Per-job wall-clock timeout
--source <id> Brain source the subagent's writes are scoped to.
Default: the standard resolution chain (GBRAIN_SOURCE,
.gbrain-source, sources.default, ...) — see
\`gbrain sources current\`
--fanout-manifest <path> JSON array of {prompt, input_vars?} — one child each
--follow Tail status until terminal (default on TTY)
--detach Submit + print job id, exit immediately
@@ -116,6 +122,7 @@ interface RunFlags {
maxTurns?: number;
tools?: string[];
timeoutMs?: number;
source?: string;
fanoutManifest?: string;
follow: boolean;
detach: boolean;
@@ -181,6 +188,7 @@ function parseRunFlags(args: string[]): { flags: RunFlags; rest: string[] } {
case '--max-turns': flags.maxTurns = parseIntFlagValue(requireFlagValue(args, ++i, a), a); break;
case '--tools': flags.tools = requireFlagValue(args, ++i, a).split(',').map(s => s.trim()).filter(Boolean); break;
case '--timeout-ms': flags.timeoutMs = parseIntFlagValue(requireFlagValue(args, ++i, a), a); break;
case '--source': flags.source = requireFlagValue(args, ++i, a); break;
case '--fanout-manifest': flags.fanoutManifest = requireFlagValue(args, ++i, a); break;
case '--follow': flags.follow = true; break;
case '--no-follow': flags.follow = false; break;
@@ -203,17 +211,86 @@ function parseRunFlags(args: string[]): { flags: RunFlags; rest: string[] } {
return { flags, rest };
}
/**
* Predicate: is this error one of the source resolver's user-facing throws
* we want to surface as a clean stderr line + exit 1? Mirrors
* dream.ts:isResolverUserError — anything else (connection failures,
* genuine bugs) propagates with a stack trace.
*/
function isResolverUserError(e: unknown): boolean {
if (!(e instanceof Error)) return false;
const m = e.message;
return (m.startsWith('Source "') && m.includes(' not found.'))
|| m.startsWith('Invalid --source value')
|| m.startsWith('Invalid GBRAIN_SOURCE value');
}
/**
* #2922: resolve the brain source for a subagent submission via the
* canonical chain (explicit --source → GBRAIN_SOURCE → .gbrain-source →
* local_path match → sources.default → sole non-default → 'default').
* Pre-fix, `gbrain agent run` never resolved a source, so every page an
* agent job wrote landed in the seed 'default' source even on brains with
* `gbrain sources default <id>` configured.
*
* The `__all__` sentinel is rejected here: subagent writes must target
* exactly one source (and `validateSourceId` at tool-registry build time
* would reject it anyway — better to fail at submit than at claim).
*/
async function resolveAgentSource(engine: BrainEngine, explicit: string | undefined): Promise<string> {
// An empty `--source ""` must fail loudly, not silently degrade to the
// env/dotfile/default tiers (resolveSourceId's `if (explicit)` treats a
// falsy value as omitted — explicit-but-empty would slip through).
if (explicit !== undefined && explicit.trim() === '') {
console.error('gbrain agent run: --source requires a non-empty value. Run `gbrain agent run --help`.');
process.exit(2);
}
let resolved: string;
try {
resolved = await resolveSourceId(engine, explicit ?? null);
} catch (e) {
if (isResolverUserError(e)) {
console.error(`gbrain agent run: ${(e as Error).message}`);
process.exit(1);
}
throw e;
}
if (resolved === ALL_SOURCES) {
console.error(
`gbrain agent run: --source ${ALL_SOURCES} is not supported — ` +
`subagent writes must target exactly one source. Pass a concrete --source <id>.`,
);
process.exit(2);
}
// Archived-source guard, mirroring dream.ts: writing subagent pages into
// an archived (normally invisible) source would mask them until restore.
const src = await fetchSource(engine, resolved);
if (src?.archived === true) {
console.error(
`gbrain agent run: source ${resolved} is archived; restore with ` +
`\`gbrain sources restore ${resolved}\` before submitting agent jobs`,
);
process.exit(1);
}
return resolved;
}
export async function runAgentRun(engine: BrainEngine, args: string[]): Promise<void> {
const { flags, rest } = parseRunFlags(args);
const queue = new MinionQueue(engine);
// #2922: resolve once at submit time; both the single-job and fan-out
// paths stamp it on SubagentHandlerData.source_id so buildOpContext
// scopes every tool call to it instead of the legacy 'default'.
const sourceId = await resolveAgentSource(engine, flags.source);
// Fan-out path: --fanout-manifest supplies explicit child inputs. The
// aggregator submits first (so its id is available as parent for each
// child); children submit with on_child_fail='continue' so mixed
// outcomes don't cascade; aggregator waits in waiting-children until
// Lane 1B's terminal-set check unblocks it.
if (flags.fanoutManifest) {
await runFanout(engine, queue, flags, rest.join(' '));
await runFanout(engine, queue, flags, rest.join(' '), sourceId);
return;
}
@@ -223,7 +300,7 @@ export async function runAgentRun(engine: BrainEngine, args: string[]): Promise<
process.exit(2);
}
const data: SubagentHandlerData = { prompt };
const data: SubagentHandlerData = { prompt, source_id: sourceId };
if (flags.subagentDef) data.subagent_def = flags.subagentDef;
if (flags.model) data.model = flags.model;
if (flags.maxTurns) data.max_turns = flags.maxTurns;
@@ -248,7 +325,7 @@ export async function runAgentRun(engine: BrainEngine, args: string[]): Promise<
// ── fan-out ───────────────────────────────────────────────
async function runFanout(engine: BrainEngine, queue: MinionQueue, flags: RunFlags, promptTemplate: string): Promise<void> {
async function runFanout(engine: BrainEngine, queue: MinionQueue, flags: RunFlags, promptTemplate: string, sourceId: string): Promise<void> {
const manifestPath = flags.fanoutManifest!;
let manifest: Array<{ prompt?: string; input_vars?: Record<string, unknown> }>;
try {
@@ -272,6 +349,7 @@ async function runFanout(engine: BrainEngine, queue: MinionQueue, flags: RunFlag
const entry = manifest[0]!;
const data: SubagentHandlerData = {
prompt: entry.prompt ?? promptTemplate,
source_id: sourceId,
...(entry.input_vars ? { input_vars: entry.input_vars } : {}),
...(flags.subagentDef ? { subagent_def: flags.subagentDef } : {}),
...(flags.model ? { model: flags.model } : {}),
@@ -303,6 +381,7 @@ async function runFanout(engine: BrainEngine, queue: MinionQueue, flags: RunFlag
for (const entry of manifest) {
const data: SubagentHandlerData = {
prompt: entry.prompt ?? promptTemplate,
source_id: sourceId,
...(entry.input_vars ? { input_vars: entry.input_vars } : {}),
...(flags.subagentDef ? { subagent_def: flags.subagentDef } : {}),
...(flags.model ? { model: flags.model } : {}),
src/commands/autopilot-fanout.ts
+26 -6
View File
@@ -34,8 +34,7 @@ import type { BrainEngine, SourceRow } from '../core/engine.ts';
import type { MinionQueue } from '../core/minions/queue.ts';
import { NON_GLOBAL_PHASES, GLOBAL_PHASES, LAST_GLOBAL_AT_KEY } from '../core/cycle.ts';
import { sourceConfigHasRemoteUrl } from '../core/sources-load.ts';
const FULL_CYCLE_FLOOR_MIN = 60;
import { AUTOPILOT_FULL_CYCLE_FLOOR_MINUTES } from './autopilot-remediation-policy.ts';
// #2194 fix #2: failure cooldown. A source whose autopilot-cycle keeps
// failing/timing-out re-dispatches every tick today (only SUCCESS gates
@@ -81,6 +80,8 @@ export interface FanoutResult {
/** True when this tick fell back to the legacy single-job path
* (no sources rows / engine empty). */
legacy_fallback: boolean;
/** True when every enumerated source is inside the freshness window. */
all_sources_fresh: boolean;
}
/**
@@ -180,7 +181,11 @@ export function readLastFullCycleAt(src: SourceRow): Date | null {
* a brain may have fresh sync but stale extract/embed. The 60-min floor on
* full-cycle is the canonical freshness signal for autopilot dispatch.
*/
export function isSourceStale(src: SourceRow, now = Date.now(), floorMin = FULL_CYCLE_FLOOR_MIN): boolean {
export function isSourceStale(
src: SourceRow,
now = Date.now(),
floorMin = AUTOPILOT_FULL_CYCLE_FLOOR_MINUTES,
): boolean {
const last = readLastFullCycleAt(src);
if (last === null) return true;
const ageMin = (now - last.getTime()) / 60_000;
@@ -328,7 +333,7 @@ export function selectSourcesForDispatch(
sources: SourceRow[],
fanoutMax: number,
now = Date.now(),
floorMin = FULL_CYCLE_FLOOR_MIN,
floorMin = AUTOPILOT_FULL_CYCLE_FLOOR_MINUTES,
recentFailures: Map<string, SourceFailure> = new Map(),
cooldownOpts: CooldownOpts = { baseMin: FAILURE_COOLDOWN_BASE_MIN, capMin: FAILURE_COOLDOWN_CAP_MIN },
): { dispatch: SourceRow[]; skippedFresh: SourceRow[]; skippedCap: SourceRow[]; skippedCooldown: SourceRow[] } {
@@ -406,7 +411,14 @@ export async function dispatchPerSource(
} else {
log(`[dispatch] job #${job.id} autopilot-cycle (legacy single-source)`);
}
return { dispatched: [], skipped_fresh: [], skipped_cap: [], skipped_cooldown: [], legacy_fallback: true };
return {
dispatched: [],
skipped_fresh: [],
skipped_cap: [],
skipped_cooldown: [],
legacy_fallback: true,
all_sources_fresh: false,
};
}
// #2194 fix #2: load recent per-source failures + cooldown knobs so a
@@ -426,7 +438,14 @@ export async function dispatchPerSource(
}
const { dispatch, skippedFresh, skippedCap, skippedCooldown } =
selectSourcesForDispatch(sources, opts.fanoutMax, Date.now(), FULL_CYCLE_FLOOR_MIN, recentFailures, cooldownOpts);
selectSourcesForDispatch(
sources,
opts.fanoutMax,
Date.now(),
AUTOPILOT_FULL_CYCLE_FLOOR_MINUTES,
recentFailures,
cooldownOpts,
);
const dispatched: string[] = [];
for (const src of dispatch) {
@@ -509,6 +528,7 @@ export async function dispatchPerSource(
skipped_cap: skippedCap.map(s => s.id),
skipped_cooldown: skippedCooldown.map(s => s.id),
legacy_fallback: false,
all_sources_fresh: skippedFresh.length === sources.length,
};
}
src/commands/autopilot-remediation-policy.ts
+46
View File
@@ -0,0 +1,46 @@
export const AUTOPILOT_FULL_CYCLE_FLOOR_MINUTES = 60;
export interface AutopilotRemediationPlanShape {
score: number;
planLength: number;
estimatedSeconds: number;
minutesSinceLastFull: number;
}
/**
* Keep recommendation keys stable for doctor/remediate checkpoints while
* giving Autopilot a fresh single-flight slot on every dispatch interval.
*/
export function autopilotRemediationIdempotencyKey(
recommendationKey: string,
dispatchSlot: string,
): string {
return `${recommendationKey}:autopilot:${dispatchSlot}`;
}
/**
* A full cycle is a freshness invariant, independent of the current score or
* targeted plan. Large/slow/severely degraded plans retain the existing
* hammer behavior before the freshness floor is reached.
*/
export function shouldRunAutopilotFullCycle({
score,
planLength,
estimatedSeconds,
minutesSinceLastFull,
}: AutopilotRemediationPlanShape): boolean {
return minutesSinceLastFull >= AUTOPILOT_FULL_CYCLE_FLOOR_MINUTES
|| planLength > 3
|| estimatedSeconds >= 300
|| score < 70;
}
export function shouldSleepHealthyAutopilot(
score: number,
planLength: number,
minutesSinceLastFull: number,
): boolean {
return score >= 95
&& planLength === 0
&& minutesSinceLastFull < AUTOPILOT_FULL_CYCLE_FLOOR_MINUTES;
}
src/commands/autopilot.ts
+58 -30
View File
@@ -19,11 +19,17 @@
import { existsSync, readFileSync, writeFileSync, mkdirSync, appendFileSync, utimesSync, unlinkSync, chmodSync, statSync } from 'fs';
import { setCliExitVerdict } from '../core/cli-force-exit.ts';
import { detectExecutionEnvironment } from '../core/execution-env.ts';
import { join, dirname, isAbsolute } from 'path';
import { execSync } from 'child_process';
import type { BrainEngine } from '../core/engine.ts';
import { loadPreferences } from '../core/preferences.ts';
import { loadConfig, loadConfigFileOnly, saveConfig, gbrainPath as gbrainHomePath } from '../core/config.ts';
import {
classifyAutopilotLockHolder,
type AutopilotLockProbeDeps,
isPidAlive,
} from '../core/autopilot-lock.ts';
import { ChildWorkerSupervisor } from '../core/minions/child-worker-supervisor.ts';
import { VERSION } from '../version.ts';
import {
@@ -40,6 +46,11 @@ import { evaluateQuietHours } from '../core/minions/quiet-hours.ts';
import { inspectLock } from '../core/db-lock.ts';
import { registerCleanup } from '../core/process-cleanup.ts';
import { resolveAutopilotDispatchTimeoutMs } from './autopilot-timeout.ts';
import {
autopilotRemediationIdempotencyKey,
shouldRunAutopilotFullCycle,
shouldSleepHealthyAutopilot,
} from './autopilot-remediation-policy.ts';
// Path helpers live in a LEAF core module so other commands (gbrain migrate)
// can read the daemon's state files without importing this one — a dynamic
// import of a command module drags its whole flag surface into the importer's
@@ -243,19 +254,22 @@ export function shouldSpawnAutopilotWorker(args: string[]): boolean {
return !args.includes('--no-worker');
}
export function isPidAlive(pid: number): boolean {
if (!Number.isFinite(pid) || pid <= 0) return false;
export { isPidAlive };
export const AUTOPILOT_FOREIGN_PID_TAKEOVER_GRACE_MS = 10 * 60 * 1000;
function autopilotLockAgeMs(lockPath: string): number | null {
try {
process.kill(pid, 0);
return true;
} catch (error: unknown) {
return (error as NodeJS.ErrnoException).code === 'EPERM';
return Date.now() - statSync(lockPath).mtimeMs;
} catch {
return null;
}
}
export function decideLockAcquisition(
lockPath: string,
currentPid: number,
deps: AutopilotLockProbeDeps = {},
): { action: 'acquire' } | { action: 'exit'; holderPid: number } | { action: 'takeover'; reason: string } {
if (!existsSync(lockPath)) return { action: 'acquire' };
@@ -267,10 +281,21 @@ export function decideLockAcquisition(
}
const holderPid = Number.parseInt(raw, 10);
const sameProcess = Number.isFinite(holderPid) && holderPid === currentPid;
const alive = !sameProcess && isPidAlive(holderPid);
const holder = classifyAutopilotLockHolder(holderPid, currentPid, deps);
if (alive) return { action: 'exit', holderPid };
if (holder.state === 'alive-autopilot' || holder.state === 'alive-unknown') {
return { action: 'exit', holderPid };
}
if (holder.state === 'alive-foreign') {
const lockAgeMs = autopilotLockAgeMs(lockPath);
if (lockAgeMs !== null && lockAgeMs >= AUTOPILOT_FOREIGN_PID_TAKEOVER_GRACE_MS) {
return { action: 'takeover', reason: `foreign pid ${raw || '<empty>'} with stale lock` };
}
return { action: 'exit', holderPid };
}
if (holder.state === 'self') {
return { action: 'takeover', reason: `own pid ${raw || '<empty>'}` };
}
return { action: 'takeover', reason: `dead pid ${raw || '<empty>'}` };
}
@@ -874,8 +899,8 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
//
// New logic: compute the remediation plan (cheap; no full doctor
// walk), then route to the right level of intervention:
// - Score >= 95 + empty plan: full cycle every 60min (phase-
// coupling exercise), otherwise sleep.
// - Full cycle every 60min regardless of score/plan (phase-
// coupling + freshness invariant); healthy brains sleep before it.
// - Small plan (<=3 steps, <5min): submit individual handlers.
// - Large plan or low score: full autopilot-cycle (the hammer).
//
@@ -1120,16 +1145,16 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
const estTotal = plan.reduce((s, r) => s + r.est_seconds, 0);
// Track time since last full cycle for the 60-min floor.
const FULL_CYCLE_FLOOR_MIN = 60;
const minutesSinceLastFull = (Date.now() - lastFullCycleAt) / 60000;
const shouldFullCycle =
(score >= 95 && plan.length === 0 && minutesSinceLastFull >= FULL_CYCLE_FLOOR_MIN) ||
plan.length > 3 ||
estTotal >= 300 ||
score < 70;
const shouldFullCycle = shouldRunAutopilotFullCycle({
score,
planLength: plan.length,
estimatedSeconds: estTotal,
minutesSinceLastFull,
});
const shouldSleep = score >= 95 && plan.length === 0 && minutesSinceLastFull < FULL_CYCLE_FLOOR_MIN;
const shouldSleep = shouldSleepHealthyAutopilot(score, plan.length, minutesSinceLastFull);
if (shouldSleep) {
if (jsonMode) {
@@ -1180,7 +1205,11 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
if (jsonMode) process.stderr.write(JSON.stringify({ event: 'global_maintenance_dispatch_failed', error: e instanceof Error ? e.message : String(e) }) + '\n');
}
}
if (result.dispatched.length > 0 || result.legacy_fallback) {
// On restart the process-local clock starts overdue. If persisted
// source timestamps say every source is fresh, advance the local
// clock too; otherwise a non-empty targeted plan would be skipped
// on every tick until the persisted 60-minute window elapsed.
if (result.dispatched.length > 0 || result.legacy_fallback || result.all_sources_fresh) {
lastFullCycleAt = Date.now();
}
if (jsonMode) {
@@ -1204,15 +1233,17 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
}
} else {
// Small targeted plan — submit individual handlers per step.
// D9 content-hash idempotency keys (from computeRecommendations).
// maxWaiting:1 per submit per codex #17 (closes the backpressure
// gap the prior implementation had for targeted submits).
// Recommendation keys stay stable for doctor/remediate checkpoints;
// Autopilot adds the dispatch interval so completed rows cannot hold
// the remediation slot forever (#4046).
// maxWaiting:1 per submit per codex #17 bounds the cross-window
// backlog if a targeted handler runs longer than one interval.
for (const step of plan) {
try {
const isProtected = !!step.protected;
const submitOpts = {
queue: 'default',
idempotency_key: step.idempotency_key,
idempotency_key: autopilotRemediationIdempotencyKey(step.idempotency_key, slot),
max_attempts: 2,
timeout_ms: timeoutMs,
maxWaiting: 1,
@@ -1444,13 +1475,10 @@ export type InstallTarget = 'macos' | 'linux-systemd' | 'ephemeral-container' |
export function detectInstallTarget(): InstallTarget {
if (process.platform === 'darwin') return 'macos';
const ephemeral = !!(
process.env.RENDER
|| process.env.RAILWAY_ENVIRONMENT
|| process.env.FLY_APP_NAME
|| existsSync('/.dockerenv')
);
if (ephemeral) return 'ephemeral-container';
// Shared detector (execution-env.ts): covers the original Render/Railway/
// Fly//.dockerenv signals AND the cloud-sandbox signature — both get the
// start-script treatment here (no reliable scheduler in either).
if (detectExecutionEnvironment() !== 'local') return 'ephemeral-container';
if (existsSync('/run/systemd/system')) {
try {
src/commands/bootstrap.ts
+219 -19
View File
@@ -34,6 +34,7 @@ import { VERSION } from '../version.ts';
import { loadConfig, loadConfigFileOnly, toEngineConfig } from '../core/config.ts';
import { createEngine } from '../core/engine-factory.ts';
import { resolveGbrainHome } from '../core/gbrain-home.ts';
import { detectExecutionEnvironment } from '../core/execution-env.ts';
import { realpathOrResolve } from '../core/path-confine.ts';
import { loadQuestionBank } from '../core/bootstrap/assets.ts';
import {
@@ -56,6 +57,7 @@ import {
registerClaudeMcp,
registerCodexMcp,
writeClaudeHooks,
writeCommittedClaudeHooks,
removeClaudeHooks,
} from '../core/bootstrap/hooks.ts';
import {
@@ -72,7 +74,7 @@ import {
statusReport,
type StatusReport,
} from '../core/bootstrap/status.ts';
import { verifyWorkspace } from '../core/bootstrap/verify.ts';
import { verifyWorkspace, deriveWorkspaceSourceId } from '../core/bootstrap/verify.ts';
export const BOOTSTRAP_HELP = `gbrain bootstrap — paste-in agent install (Claude Code / Codex)
@@ -100,6 +102,10 @@ Subcommands (run \`gbrain bootstrap status\` first — it is the resume entrypoi
verify [--json] The whole install contract (round-trip, graph floor,
magic moment, scans, hooks smoke). Exit 0 or not done.
attach [--harness H] Machine two: adopt a cloned agent workspace.
cloud-setup-script Print the paste-ready cloud environment setup
script (installs the gbrain binary into the
environment snapshot; npm-based — bun fetching
is proxy-incompatible in cloud sandboxes).
uninstall [--delete-brain] [--home <dir>] [--yes]
Receipt-keyed removal. The repo stays yours.
@@ -110,6 +116,60 @@ Env: GBRAIN_BOOTSTRAP_ABORT_AFTER=<phase> (test seam — abort after that phase'
const SUPPORT_HINT =
'If you are stuck: run `gbrain bootstrap status --json` and relay the "support" block verbatim.';
/**
* Per-subcommand `--help`/`-h`/`help` usage text for the subcommands that
* MUTATE state (create a repo, register MCP/hooks, run the verify contract,
* adopt a workspace, remove receipt-tracked paths, record an interview
* answer). `runBootstrap`'s dispatch checks `args[0]` for top-level help
* (`--help`/`-h`/`help`/no args), but a help token AFTER the subcommand name
* (e.g. `gbrain bootstrap repo --help`, `gbrain bootstrap uninstall help`)
* previously fell straight into the subcommand's own arg parsing, which had
* no help handling of its own — so it ran the real mutation instead of
* printing help. `status`/`cloud-setup-script` are pure reads, so they don't
* need a guard.
*/
const SUBCOMMAND_HELP: Record<string, string> = {
render:
'gbrain bootstrap render [--force] [--only F] [--minimal]\n' +
' Render identity files from the confirmed interview answers. Never clobbers; --force backs up first.',
repo:
'gbrain bootstrap repo\n' +
' Create the dedicated PRIVATE GitHub repo (or adopt an EMPTY private repo you created\n' +
' under your own account), verify the privacy bit via the API, push.',
hooks:
'gbrain bootstrap hooks [--harness claude-code|codex] [--repair] [--no-hooks] [--gbrain-bin <path>]\n' +
' Register MCP (+ per-turn hooks on Claude Code, ON by default; --no-hooks opts out).',
verify:
'gbrain bootstrap verify [--json]\n' +
' The whole install contract (round-trip, graph floor, magic moment, scans, hooks smoke). Exit 0 or not done.',
attach:
'gbrain bootstrap attach [--harness H]\n' +
' Machine two: adopt a cloned agent workspace.',
uninstall:
'gbrain bootstrap uninstall [--delete-brain] [--home <dir>] [--yes]\n' +
' Receipt-keyed removal. The repo stays yours.',
interview:
'gbrain bootstrap interview --init | --set KEY "value" | --skip KEY | --status | --show | --confirm <hash>\n' +
' Create/record/read interview state. See `gbrain bootstrap --help` for the per-flag description.',
};
/**
* `--help`/`-h` are always recognized. The bare word `help` (no dashes) is
* ALSO recognized for every subcommand above EXCEPT `interview` — mirroring
* the top-level `sub === 'help'` handling for a user who tries the same
* spelling after a subcommand name. `interview` is excluded from the
* bare-word form because its `--set KEY "value"` free-text answers could
* legitimately BE the literal word "help" (e.g. a one-word answer); none of
* the other subcommands' flags take arbitrary prose, only booleans, enums,
* or paths, so the bare-word collision risk there is negligible (matches the
* already-accepted low-impact risk of `-h` colliding with a literal path
* value like `--home -h`).
*/
function hasHelpToken(args: string[], allowBareWord: boolean): boolean {
if (args.includes('--help') || args.includes('-h')) return true;
return allowBareWord && args.includes('help');
}
/** Thrown by the A7 abort seam; mapped to exit 130 (simulated kill). */
export class BootstrapAbortInjected extends Error {
constructor(phase: string) {
@@ -148,6 +208,19 @@ function resolveWorkspace(args: string[]): string {
return ws ? resolve(ws) : process.cwd();
}
/**
* POSIX single-quote anything not already shell-safe, for commands printed
* as copy/paste guidance (mirror of the private `shellQuote` in
* core/bootstrap/hooks.ts, core/sources-ops.ts, and commands/connect.ts —
* same contract: `$()`/backticks in a value are inert literals once quoted).
* A workspace path containing a space or shell metacharacter must not turn
* "the exact command to run" into a broken (or, pasted blind, dangerous) one.
*/
function shellQuoteForDisplay(arg: string): string {
if (/^[A-Za-z0-9_.:/@=-]+$/.test(arg)) return arg;
return `'${arg.replace(/'/g, "'\\''")}'`;
}
// ── Shared plumbing ─────────────────────────────────────────────────────────
type Harness = 'claude-code' | 'codex';
@@ -607,37 +680,57 @@ async function runRepo(ws: string, rest: string[], home: string, runner: ExecRun
abortIfInjected('repo');
// PERSIST_CRON consent [D3 resolved]: opt-in 15-min scan-gated push job
// via the existing sources-harden machinery. Best-effort: a harden
// failure never fails the repo phase — the SessionEnd push backstop is
// always on.
// PERSIST_CRON consent [D3 resolved]: opt-in background persistence via
// the existing sources-harden machinery — a git post-commit auto-push plus
// a 30-minute scheduled pull that keeps multi-machine checkouts fresh
// (honest copy [D9]: the event-driven pushes do the durability work; the
// timer is the freshener). Best-effort: a harden failure never fails the
// repo phase — the per-turn and session-end pushes are always on.
const persist = (consentAnswer(ws, 'PERSIST_CRON') ?? 'no').toLowerCase();
const envKind = detectExecutionEnvironment();
if (persist === 'yes') {
try {
const { hardenBrainRepo } = await import('../core/brain-repo-durability.ts');
const state = readManifest(ws);
const sourceId = state.state === 'initialized' ? state.manifest.source_id : 'workspace';
// Containers/cloud sandboxes have no reliable scheduler — install the
// container-friendly half (post-commit hook + helper) and say so,
// instead of failing a crontab write that could never survive anyway.
const installCron = envKind === 'local';
const report = await hardenBrainRepo({
repoPath: ws,
sourceId,
installCron: true,
installCron,
verify: false,
logger: (l: string) => process.stderr.write(`[harden] ${l}\n`),
});
const attention = report.steps.filter((s) => s.status === 'needs_attention');
console.log(
attention.length === 0
? 'background persistence enabled (15-min scan-gated push job installed).'
: `background persistence partially enabled — needs attention: ${attention.map((s) => `${s.step}: ${s.detail}`).join('; ')}`,
);
if (attention.length > 0) {
console.log(
`background persistence partially enabled — needs attention: ${attention.map((s) => `${s.step}: ${s.detail}`).join('; ')}`,
);
} else if (installCron) {
console.log(
'background persistence enabled (post-commit auto-push + 30-min scheduled pull installed).',
);
} else {
console.log(
`background persistence enabled for this ${envKind === 'cloud-sandbox' ? 'cloud sandbox' : 'container'}: ` +
'post-commit auto-push installed; per-turn and session-end pushes are already on. ' +
'No scheduler exists in this environment, so the 30-min pull is skipped — run ' +
'`gbrain sources harden` on a persistent machine to add it.',
);
}
} catch (e) {
console.error(
`note: background-persistence install failed (${(e as Error).message}). ` +
'Session-end pushes still persist your work; re-try later with `gbrain sources harden`.',
'Per-turn and session-end pushes still persist your work; re-try later with `gbrain sources harden`.',
);
}
} else {
console.log('background persistence declined — the session-end push remains the persistence backstop.');
console.log(
'background persistence declined — the per-turn and session-end pushes remain the persistence backstop.',
);
}
return 0;
});
@@ -709,6 +802,41 @@ async function runHooks(ws: string, rest: string[], home: string, runner: ExecRu
const gbrainHome = process.env.GBRAIN_HOME?.trim() || undefined;
return withLock(ws, async () => {
// 0. source_id visibility seam: `hooks` is the last ENGINE-FREE phase
// before `verify` (which alone can detect a source_id collision — the
// sources registry lives only in the DB). Without this, a human who
// hand-registers a source before verify has no way to know the exact id
// the workspace expects, guesses an "intuitive" name instead, and only
// discovers the mismatch via a `verify` roundtrip FK error — then, after
// switching to the manifest's id, an `overlapping_path` error from the
// earlier guess still claiming the same brain/ dir. Printing the current
// id (and the collision-fallback id verify would derive, a pure path
// hash that needs no engine) up front — plus creating brain/ so
// registration can happen immediately — collapses that multi-round-trip
// loop to one command.
const brainDir = join(ws, 'brain');
mkdirSync(brainDir, { recursive: true });
// --force: brain/ was just created empty — `sources add` fail-fasts on a
// --path that exists but isn't a git repo with committed, tracked
// content (#2707), and gbrain deliberately never auto-git-inits a --path
// source itself (a --path source is the user's own directory — the
// consent boundary #2967 established for sync-time self-heal applies
// here too). --force is the sanctioned opt-in for exactly this "register
// before git-init exists" case (see sources-ops.ts's own not_a_git_repo
// message), and it is safe here because brainDir is not an arbitrary
// user path — it is the fixed `<workspace>/brain` subdir this phase just
// created. Without --force, the printed command below would itself throw
// not_a_git_repo the instant it's pasted.
const quoted = shellQuoteForDisplay(brainDir);
console.log(
`brain source: register this workspace's brain/ now if you haven't — ` +
`\`gbrain sources add ${sourceId} --path ${quoted} --force\` (brain/ is freshly created and empty; ` +
`--force is the documented opt-in for registering before git-init exists). If '${sourceId}' is ` +
`already claimed by a different checkout on this brain, \`gbrain bootstrap verify\` will detect the ` +
`collision and switch this workspace to '${deriveWorkspaceSourceId(ws)}' — re-run the same command ` +
`with that id instead.`,
);
// 1. MCP registration — argv built by the host-format module, executed
// through the runner seam, recorded on the receipt.
const argvs =
@@ -789,12 +917,28 @@ async function runHooks(ws: string, rest: string[], home: string, runner: ExecRu
let hooksWritten = false;
if (harness === 'claude-code') {
if (hooksConsent) {
const r = writeClaudeHooks(ws, {
gbrainBin,
env: { GBRAIN_SOURCE: sourceId, ...(gbrainHome ? { GBRAIN_HOME: gbrainHome } : {}) },
});
// Carrier choice [D12]: cloud sandboxes clone fresh and snapshot hook
// config at session start — only the repo-COMMITTED settings file
// exists there, so cloud installs write the committed carrier
// (PATH-resolved, fail-open commands; no machine paths). Local
// installs keep the gitignored settings.local.json with the absolute
// binary path. The writers enforce that one event never fires from
// both files.
const hookEnv = { GBRAIN_SOURCE: sourceId, ...(gbrainHome ? { GBRAIN_HOME: gbrainHome } : {}) };
const cloudCarrier = detectExecutionEnvironment() === 'cloud-sandbox';
const r = cloudCarrier
? writeCommittedClaudeHooks(ws, { env: hookEnv })
: writeClaudeHooks(ws, { gbrainBin, env: hookEnv });
hooksWritten = true;
console.log(`hooks installed (${r.installed.length} event(s)) in ${r.settingsPath}${repair ? ' [repair]' : ''} — your brain now loads every turn. Turn off any time with GBRAIN_HOOKS=0, or re-run with --no-hooks.`);
console.log(
`hooks installed (${r.installed.length} event(s)) in ${r.settingsPath}${repair ? ' [repair]' : ''} — your brain now loads every turn. Turn off any time with GBRAIN_HOOKS=0, or re-run with --no-hooks.`,
);
if (cloudCarrier) {
console.log(
'cloud sandbox: hooks written to the COMMITTED .claude/settings.json (fail-open, PATH-resolved) — ' +
'commit + push it so the next session starts with hooks live; hooks written mid-session activate on the NEXT session (startup snapshot).',
);
}
for (const note of r.notes) console.error(note);
} else {
console.log(
@@ -913,6 +1057,10 @@ async function runUninstall(ws: string, rest: string[], home: string, runner: Ex
'offered: export facts before deletion (`gbrain facts export`) — the brain DB is about to be removed and facts are not derived state',
);
}
// Read the source id BEFORE uninstallWorkspace removes rendered files —
// the durability teardown below needs it and the manifest may not survive.
const preState = readManifest(ws);
const durabilitySourceId = preState.state === 'initialized' ? preState.manifest.source_id : 'workspace';
const result = await uninstallWorkspace(ws, {
deleteBrain,
...(yes ? { confirm: async () => true } : {}),
@@ -935,6 +1083,41 @@ async function runUninstall(ws: string, rest: string[], home: string, runner: Ex
}
}
// Per-root hook-lane state [D13]: remove this workspace's push-status,
// debounce, and announce files — a dead root's failing record would
// otherwise re-fire the failure banner forever (it can never be cleared
// by a re-push once the workspace is gone).
try {
const { pushStatusPathForRoot, workspaceRootHash } = await import('../core/workspace-push.ts');
const { execFileSync } = await import('node:child_process');
let root = ws;
try {
root = execFileSync('git', ['-C', ws, 'rev-parse', '--show-toplevel'], {
stdio: ['ignore', 'pipe', 'ignore'], timeout: 5_000, env: process.env,
}).toString().trim() || ws;
} catch { /* not a repo — use ws as-is */ }
const { rmSync } = await import('node:fs');
const statusFile = pushStatusPathForRoot(root);
for (const f of [statusFile, `${statusFile}.announced`, join(resolveGbrainHome(), 'bootstrap', `stop-push-${workspaceRootHash(root)}.json`)]) {
rmSync(f, { force: true });
}
} catch { /* best-effort */ }
// Durability teardown [B6]: uninstall previously left the launchd/cron
// job, the untracked post-commit hook, and the credential wiring behind.
// Best-effort — a teardown hiccup never fails the uninstall. The COMMITTED
// helper script and AGENTS.md rules stay (repo content is the user's).
try {
const { unhardenBrainRepo } = await import('../core/brain-repo-durability.ts');
const steps = await unhardenBrainRepo({ repoPath: ws, sourceId: durabilitySourceId });
const acted = steps.filter((s) => s.status === 'fixed');
if (acted.length > 0) {
console.log(`durability wiring removed: ${acted.map((s) => s.step).join(', ')} (committed helper + AGENTS rules stay — repo content is yours)`);
}
} catch (e) {
console.error(`note: durability teardown incomplete (${(e as Error).message}) — run \`gbrain sources unharden ${durabilitySourceId}\` by hand if a scheduled job lingers.`);
}
// The facts-export offer already printed BEFORE deletion (above); don't
// repeat it after the brain is gone.
for (const step of result.steps) {
@@ -972,13 +1155,23 @@ export async function runBootstrap(args: string[], opts: RunBootstrapOpts = {}):
const logCtx: LogCtx = { home, ws, ...(harnessForLog ? { harness: harnessForLog } : {}) };
const t0 = Date.now();
const KNOWN = new Set(['status', 'interview', 'render', 'repo', 'hooks', 'verify', 'attach', 'uninstall']);
const KNOWN = new Set(['status', 'interview', 'render', 'repo', 'hooks', 'verify', 'attach', 'uninstall', 'cloud-setup-script']);
if (!KNOWN.has(sub)) {
console.error(`unknown subcommand: ${sub}`);
console.error(BOOTSTRAP_HELP);
return 2;
}
// Subcommand-level help: BEFORE any subcommand body runs, so a help token
// after a mutating subcommand (repo/hooks/verify/attach/uninstall/render/
// interview) never falls through into the real operation, regardless of
// what other flags/values precede it in `rest`. No install-log entry
// either — this isn't a phase run.
if (SUBCOMMAND_HELP[sub] && hasHelpToken(rest, sub !== 'interview')) {
console.log(SUBCOMMAND_HELP[sub]);
return 0;
}
// The install log records the PHASE name, and the hooks subcommand is the
// 'wire' phase (status.ts phase list) — one mapping, used at every log site.
const logPhaseName = sub === 'hooks' ? 'wire' : sub;
@@ -989,6 +1182,13 @@ export async function runBootstrap(args: string[], opts: RunBootstrapOpts = {}):
case 'status':
// status is the read surface — it does not log itself into install.jsonl.
return await runStatus(ws, rest, home);
case 'cloud-setup-script': {
// Pure print [D16]: the paste-ready cloud environment setup script.
// Read surface like status — no install log entry.
const { loadCloudSetupScript } = await import('../core/bootstrap/assets.ts');
console.log(loadCloudSetupScript().trimEnd());
return 0;
}
case 'interview':
code = await runInterview(ws, rest);
break;
src/commands/config.ts
+55 -1
View File
@@ -91,6 +91,21 @@ export async function runConfig(engine: BrainEngine, args: string[]) {
console.error('Usage: gbrain config unset <key> | --pattern <prefix>');
process.exit(1);
}
if (key === 'push.allow_unverified_remote' || key === 'hooks.stop_push_debounce_min') {
const { loadConfigFileOnly, saveConfig } = await import('../core/config.ts');
const cfg = loadConfigFileOnly();
const [top, leaf] = key.split('.') as ['push' | 'hooks', string];
const branch = cfg?.[top] as Record<string, unknown> | undefined;
if (cfg && branch && leaf in branch) {
delete branch[leaf];
saveConfig(cfg);
console.log(`Unset ${key} (file plane)`);
} else {
console.error(`Config key not found: ${key}`);
process.exit(1);
}
return;
}
const n = await engine.unsetConfig(key);
if (n > 0) {
console.log(`Unset ${key}`);
@@ -111,7 +126,14 @@ export async function runConfig(engine: BrainEngine, args: string[]) {
// overlays env onto the file) — and report which plane answered on
// stderr, keeping stdout a bare value for scripts.
const filePlane = loadConfig() as Record<string, unknown> | null;
const fileVal = filePlane?.[key];
// Dotted keys (push.allow_unverified_remote, hooks.stop_push_debounce_min)
// are stored NESTED by `set`; resolve the path so `get`/`unset` see them.
const resolveDotted = (obj: Record<string, unknown> | null, k: string): unknown => {
if (!obj) return undefined;
if (k in obj) return obj[k];
return k.split('.').reduce<unknown>((acc, seg) => (acc && typeof acc === 'object' ? (acc as Record<string, unknown>)[seg] : undefined), obj);
};
const fileVal = resolveDotted(filePlane, key);
const dbVal = await engine.getConfig(key);
const val = fileVal !== undefined && fileVal !== null ? fileVal : dbVal;
if (val !== null && val !== undefined) {
@@ -129,6 +151,38 @@ export async function runConfig(engine: BrainEngine, args: string[]) {
process.exit(1);
}
} else if (action === 'set' && key && value) {
// Bootstrap hook-lane keys are FILE-plane canonical: they are read by
// engine-free processes (the harness hook children and the detached
// `sources push` child) via loadConfigFileOnly, which never sees the DB
// plane — and the DB plane is unreadable anyway while a `gbrain serve`
// holds the single-writer lock. Route them to ~/.gbrain/config.json.
if (key === 'push.allow_unverified_remote' || key === 'hooks.stop_push_debounce_min') {
const { loadConfigFileOnly, saveConfig, isConfigTruthy } = await import('../core/config.ts');
const cfg = (loadConfigFileOnly() ?? { engine: 'pglite' }) as Parameters<typeof saveConfig>[0];
if (key === 'push.allow_unverified_remote') {
const on = isConfigTruthy(value);
cfg.push = { ...(cfg.push ?? {}), allow_unverified_remote: on };
saveConfig(cfg);
console.log(`Set ${key} = ${on} (file plane: ~/.gbrain/config.json)`);
if (on) {
console.log(
'WARNING: workspace pushes now SKIP repo-visibility verification. ' +
'This trusts the remote on your word — unset it once verification works: ' +
'gbrain config set push.allow_unverified_remote false',
);
}
} else {
const n = Number.parseInt(value, 10);
if (!Number.isFinite(n) || n < 0) {
console.error(`[config] ${key} must be an integer >= 0 (minutes; 0 = push every turn)`);
process.exit(1);
}
cfg.hooks = { ...(cfg.hooks ?? {}), stop_push_debounce_min: n };
saveConfig(cfg);
console.log(`Set ${key} = ${n} (file plane: ~/.gbrain/config.json)`);
}
return;
}
// v0.37.11.0 fix wave (Lane C.2 + CDX2-13): refuse writes to schema-sizing
// fields unconditionally. These fields size the `content_chunks.embedding`
// column at init time and are file-plane canonical. `gbrain config set
src/commands/doctor.ts
+447 -41
View File
@@ -782,6 +782,124 @@ export async function checkSourceConfigShape(engine: BrainEngine): Promise<Check
}
}
/**
* #2674 — pglite_scratch_probe: distinguish a damaged PGLite store from a
* broken WASM runtime.
*
* PGLite reports only `Aborted()` to JS (the PANIC goes to its own stderr),
* so when init fails, the error string cannot say WHICH of the two it is.
* The probe initializes a throwaway store in a temp dir, round-trips a row,
* and reads the outcome:
*
* - scratch works, real init failed → the runtime is fine; the failure is
* specific to YOUR store. The store-damage verdict is only ASSERTED when
* the caller supplies positive evidence (`storeDamageEvidence`: a
* damage-class disk diagnosis from `inspectPgliteDataDir`, or a
* wasm-abort/corrupt classification of the real init error). engine=null
* alone also covers locks and config refusals — blaming the store for
* those was the original false-positive defect; without evidence the
* message hedges and points at the `pglite_data_dir` diagnosis instead.
* - scratch fails too → the runtime cannot start on this machine; report
* OS + Bun versions on #223.
*
* COST GATE: a PGLite cold start is 5–20s on loaded machines, so this never
* runs on a routine `gbrain doctor`. It runs only when (a) the real PGLite
* engine actually failed to open (engine=null, not --fast, configured engine
* is pglite) AND the disk diagnosis didn't already fully explain the failure
* (a live lock / missing dir needs no runtime probe), or (b) the operator
* asks with `--probe-pglite`.
*
* `probeFn` is a test seam so message routing can be pinned without paying
* real cold starts.
*/
export async function checkPgliteScratchProbe(opts: {
realInitFailed: boolean;
/**
* Positive evidence the REAL store is damaged: `inspectPgliteDataDir`
* verdict wal-corruption-likely/unsupported-layout (buildChecks path) or a
* wasm-abort/corrupt classification of the actual connect error (remote
* path). Without it the scratch-ok arm hedges instead of asserting damage.
*/
storeDamageEvidence?: boolean;
realStorePath?: string;
probeFn?: () => Promise<import('../core/pglite-engine.ts').PgliteScratchProbeResult>;
}): Promise<Check> {
const name = 'pglite_scratch_probe';
try {
const probe =
opts.probeFn ??
(async () => {
const { probePgliteScratchStore } = await import('../core/pglite-engine.ts');
return probePgliteScratchStore(opts.realStorePath);
});
const r = await probe();
const secs = (r.duration_ms / 1000).toFixed(1);
if (r.ok) {
if (opts.realInitFailed && opts.storeDamageEvidence) {
return {
name,
status: 'fail',
message:
`A scratch PGLite store initialized, wrote and read back fine on this machine (${secs}s), ` +
`so the runtime is healthy and YOUR STORE is damaged — not the WASM runtime. ` +
`Your markdown is unaffected: the DB holds derived data (chunks, embeddings, links, facts) that a re-sync rebuilds. ` +
`Recover: \`gbrain pglite-repair --dry-run\` to diagnose, \`gbrain pglite-repair --yes\` for in-place WAL repair (data preserved); ` +
`if that can't fix it, restore a backup of the store directory or run \`gbrain reinit-pglite\` (wipes + re-inits + re-syncs; ` +
`defaults embedding flags from your config file).`,
details: { scratch_ok: true, duration_ms: r.duration_ms },
};
}
if (opts.realInitFailed) {
// Runtime proven healthy, but no independent evidence of store DAMAGE
// — engine=null also covers locks, config refusals, and transient
// failures. Hedge rather than convict the store (#2674 review).
return {
name,
status: 'warn',
message:
`A scratch PGLite store initialized, wrote and read back fine on this machine (${secs}s), ` +
`so the WASM runtime is healthy — the failure opening your brain is specific to your store, ` +
`its lock, or its configuration. See the \`pglite_data_dir\` check for the on-disk diagnosis; ` +
`\`gbrain pglite-repair --dry-run\` diagnoses without mutating anything.`,
details: { scratch_ok: true, duration_ms: r.duration_ms },
};
}
return {
name,
status: 'ok',
message: `PGLite runtime healthy: scratch store round-trip in ${secs}s.`,
details: { scratch_ok: true, duration_ms: r.duration_ms },
};
}
const errLine = (r.error ?? 'unknown error').split('\n')[0];
if (opts.realInitFailed) {
return {
name,
status: 'fail',
message:
`A fresh scratch PGLite store ALSO failed to start (${secs}s), so the WASM runtime cannot run ` +
`on this machine — your store is not necessarily damaged. Report your OS and Bun versions on ` +
`https://github.com/garrytan/gbrain/issues/223. Scratch error: ${errLine}`,
details: { scratch_ok: false, duration_ms: r.duration_ms, error: r.error, verdict: r.verdict },
};
}
return {
name,
status: 'warn',
message:
`Your real store opened, but a fresh scratch PGLite store failed to initialize (${secs}s) — ` +
`new stores can't be created on this machine. Report your OS and Bun versions on ` +
`https://github.com/garrytan/gbrain/issues/223. Scratch error: ${errLine}`,
details: { scratch_ok: false, duration_ms: r.duration_ms, error: r.error, verdict: r.verdict },
};
} catch (e) {
// Includes the never-touch-the-real-store guard refusal. The probe not
// running is a diagnostic gap, not a diagnosis — warn, don't fail.
const msg = e instanceof Error ? e.message : String(e);
return { name, status: 'warn', message: `scratch probe could not run: ${msg}` };
}
}
export async function doctorReportRemote(
engine: BrainEngine,
opts: { sourceIds?: string[] } = {},
@@ -804,6 +922,23 @@ export async function doctorReportRemote(
status: 'fail',
message: e instanceof Error ? e.message : String(e),
});
// #2674: on PGLite, a dead connection is exactly the ambiguous case the
// scratch probe exists for — pay its cold start only on this failure path.
// Unlike buildChecks (where the connect error was swallowed upstream), the
// real error IS in hand here: classify it, and only let the probe assert
// store damage on a damage-class verdict (wasm-abort/corrupt) — a lock or
// config refusal classifies 'unknown' and gets the hedged message.
if (engine.kind === 'pglite') {
let realStorePath: string | undefined;
try { realStorePath = loadConfig()?.database_path; } catch { /* no config */ }
let storeDamageEvidence = false;
try {
const { classifyPgliteInitError, stringifyPgliteInitError } = await import('../core/pglite-engine.ts');
const verdict = classifyPgliteInitError(stringifyPgliteInitError(e));
storeDamageEvidence = verdict === 'wasm-abort' || verdict === 'corrupt';
} catch { /* classifier unavailable — stay hedged (fail-closed) */ }
checks.push(await checkPgliteScratchProbe({ realInitFailed: true, storeDamageEvidence, realStorePath }));
}
// Without a connection, every other check is meaningless — short-circuit.
return computeDoctorReport(checks);
}
@@ -1746,6 +1881,8 @@ export async function checkVoiceGateHealth(engine: BrainEngine): Promise<Check>
* Below that they're noise; reranker fails open anyway.
* 5) Payload-too-large failures: warn at >=1 (indicates a workload
* mismatch that the operator should know about).
* 6) Budget/pricing failures: warn at >=1 with the rerank pricing surface
* and --max-cost escape hatch.
*
* Engine-agnostic (file-based + one config-key read).
*/
@@ -1784,6 +1921,15 @@ export async function checkRerankerHealth(engine: BrainEngine): Promise<Check> {
};
}
const budgetFails = failures.filter((f) => f.reason === 'budget');
if (budgetFails.length > 0) {
return {
name: 'reranker_health',
status: 'warn',
message: `${budgetFails.length} reranker budget/pricing failure(s) in last 7 days. Fix: add rerank pricing to src/core/embedding-pricing.ts or drop --max-cost.`,
};
}
const transientFails = failures.filter(
(f) => f.reason === 'network' || f.reason === 'timeout' || f.reason === 'rate_limit',
);
@@ -2501,6 +2647,131 @@ export async function checkZeEmbeddingHealth(engine: BrainEngine): Promise<Check
}
}
/**
* provider_sunset doctor check (#3390 follow-up).
*
* Detects a brain whose EFFECTIVE embedding model (gateway-resolved, which is
* how default-config brains land on the shipped default) is on a provider
* with an announced hosted-API shutdown, and prints a paste-ready migration
* command with the brain's ACTUAL `content_chunks.embedding` column width
* filled in — not the config value, which can drift. Keeping the current
* width avoids a needless dimension transition + index rebuild when the
* target supports it.
*
* Unlike the one-shot upgrade banner (`ze_sunset_notice_shown`), this fires
* on every `gbrain doctor` run until the brain is off the provider —
* warn before the shutdown date; fail after it ONLY when the brain is
* actually exposed (embedded vectors exist in the affected column, so
* retrieval is genuinely down). A zero-vector brain whose config merely
* RESOLVES to the dead default stays warn — otherwise every stock fresh
* install (and every doctor-as-CI-gate) starts exiting 1 on the date with
* no code change. Suppress entirely (accepted-risk installs) via
* `gbrain config set doctor.suppress_provider_sunset true`.
* No network call; one catalog query for the column width.
*
* `now` is injectable so tests can pin BOTH sides of the date without
* waiting for the calendar (the date itself is a compile-time constant).
*/
export async function checkProviderSunset(engine: BrainEngine, now: number = Date.now()): Promise<Check> {
const name = 'provider_sunset';
try {
const suppressed = await engine.getConfig('doctor.suppress_provider_sunset').catch(() => null);
if (suppressed === 'true' || suppressed === '1') {
return {
name,
status: 'ok',
message: 'Check suppressed via doctor.suppress_provider_sunset (unset it to re-enable).',
};
}
const { DEFAULT_EMBEDDING_MODEL, ZEROENTROPY_SUNSET_DATE } = await import('../core/ai/defaults.ts');
// Effective model: gateway when configured (file/env plane, the runtime
// truth); the shipped default otherwise — an unset-config brain resolves
// to the default at runtime, so it is just as affected.
let model = DEFAULT_EMBEDDING_MODEL;
try {
const { getEmbeddingModel } = await import('../core/ai/gateway.ts');
model = getEmbeddingModel();
} catch {
// Gateway unconfigured — runtime resolves the shipped default.
}
// Effective reranker: resolve through the SAME plane search actually
// reranks with — resolveSearchMode (mode bundle + search.reranker.*
// config overrides; hybrid.ts passes `resolvedMode.reranker_model`).
// The gateway plane is unset by default while balanced/tokenmax rerank
// with the bundle's zeroentropyai model — reading the gateway here
// would false-ok the exact brains this check exists to protect.
let reranker: string | undefined;
try {
const { loadSearchModeConfig, resolveSearchMode } = await import('../core/search/mode.ts');
const knobs = resolveSearchMode(await loadSearchModeConfig(engine));
if (knobs.reranker_enabled) reranker = knobs.reranker_model;
} catch {
// Mode resolution failed — make no reranker-exposure claim.
}
const onSunsetEmbedding = model.startsWith('zeroentropyai:');
const onSunsetReranker = !!reranker?.startsWith('zeroentropyai:');
if (!onSunsetEmbedding && !onSunsetReranker) {
return {
name,
status: 'ok',
message: `No configured provider has an announced shutdown (embedding: ${model}).`,
};
}
const past = now >= Date.parse(`${ZEROENTROPY_SUNSET_DATE}T00:00:00Z`);
const parts: string[] = [];
let hasVectors = false;
if (onSunsetEmbedding) {
let dims: number | null = null;
try {
const { readContentChunksEmbeddingDim } = await import('../core/embedding-dim-check.ts');
dims = (await readContentChunksEmbeddingDim(engine)).dims;
} catch {
// Column probe failed (fresh/odd brain) — omit --dim from the hint.
}
try {
const rows = await engine.executeRaw(
`SELECT 1 AS one FROM content_chunks WHERE embedding IS NOT NULL LIMIT 1`,
);
hasVectors = rows.length > 0;
} catch {
// Probe failed (fresh/odd brain) — no exposure claim, warn-only.
}
const dimFlag = dims ? ` --dim ${dims}` : '';
parts.push(
past
? hasVectors
? `embedding_model="${model}": the hosted API shut down on ${ZEROENTROPY_SUNSET_DATE} — semantic retrieval is offline (queries can no longer be embedded against your existing vectors).`
: `embedding_model="${model}": the hosted API shut down on ${ZEROENTROPY_SUNSET_DATE}. No embedded vectors exist yet, so retrieval is not impacted — but embedding will fail until the config points elsewhere.`
: `embedding_model="${model}": the hosted API shuts down on ${ZEROENTROPY_SUNSET_DATE}. On that date semantic retrieval stops entirely — existing vectors become unqueryable (query embedding uses the same endpoint), not just new content.`,
);
parts.push(
`Two fixes, either works: ` +
`[1] self-host the same model — zembed-1 weights are Apache-2.0; serve them via llama-server or Ollama and point the config at the local endpoint. Keeps every existing vector, no re-embed (docs/guides/embedding-migration.md, "Self-hosting instead of migrating"). ` +
`[2] migrate to another provider (resumable; preview cost first): ` +
`gbrain migrate embeddings --to <provider:model>${dimFlag} --dry-run` +
(dims ? ` — keep --dim ${dims} (this brain's actual index width) to avoid a needless schema rebuild when the target supports it.` : ''),
);
}
if (onSunsetReranker) {
parts.push(
`The reranker (${reranker}) is on the same provider; after the shutdown search falls back to unreranked ordering. ` +
`Fix: gbrain config set search.reranker.enabled false, or point search.reranker.model at another provider.`,
);
}
if (onSunsetEmbedding || onSunsetReranker) {
parts.push('Accepted the risk? Silence this check: gbrain config set doctor.suppress_provider_sunset true');
}
// fail = retrieval is ACTUALLY down (past the date AND embedded vectors
// exist on the dead provider). Reranker-only exposure stays warn — search
// fails open to unreranked ordering (degraded, not down).
const failNow = past && onSunsetEmbedding && hasVectors;
return { name, status: failNow ? 'fail' : 'warn', message: parts.join(' ') };
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
return { name, status: 'warn', message: `Could not check provider sunset status: ${msg}` };
}
}
/**
* v0.36.0.0 (A5): embedding_width_consistency doctor check.
*
@@ -5605,7 +5876,15 @@ export async function buildChecks(
if (lastStarted && engine) {
const queue = typeof lastStarted.queue === 'string' ? lastStarted.queue : 'default';
const effectiveMaxRss = typeof lastStarted.max_rss_mb === 'number' ? lastStarted.max_rss_mb : null;
const localPid = readSupervisorPid(DEFAULT_PID_FILE).pid;
// The 'started' event already records the pid-file path actually in use
// (this.opts.pidFile, which reflects a custom --pid-file). Prefer that
// over re-deriving DEFAULT_PID_FILE locally so a custom --pid-file
// deployment doesn't false-positive a singleton mismatch against itself.
// Falls back to DEFAULT_PID_FILE when the event carries no usable value.
const pidFilePath = typeof lastStarted.pid_file === 'string' && lastStarted.pid_file.length > 0
? lastStarted.pid_file
: DEFAULT_PID_FILE;
const localPid = readSupervisorPid(pidFilePath).pid;
const localHost = hostname();
// Read the DB singleton lock holder for this queue.
@@ -6313,24 +6592,62 @@ export async function buildChecks(
// Filesystem read failure is non-fatal.
}
// 3d. PGLite data-dir diagnosis (WAL-repair wave). Only meaningful when the
// connect already FAILED on a PGLite brain (engine === null): the connect
// error was swallowed by the fs-only fallback, so this check re-derives the
// dir state from disk and names the repair ladder. Skipped under --fast
// (connect wasn't attempted, so "engine === null" proves nothing there).
if (!fastMode && !engine) {
try {
const cfg = loadConfig();
if (cfg?.engine === 'pglite') {
// 3d. PGLite data-dir diagnosis (WAL-repair wave) + scratch-store probe
// (#2674). The data-dir check re-derives the failure state from DISK (the
// connect error was swallowed by the fs-only fallback); the probe adds the
// RUNTIME dimension (a throwaway store that opens fine proves the WASM
// runtime is healthy). Both only fire when the connect already FAILED on a
// PGLite brain (engine === null, not --fast — under --fast connect wasn't
// attempted, so "engine === null" proves nothing there).
//
// Probe cost gate (a PGLite cold start is 5–20s): auto-runs ONLY when init
// failed AND the disk diagnosis didn't already fully explain it — a live
// lock or a missing dir needs no runtime probe (and 'locked' was exactly
// the reviewed false-positive: blaming the store while `gbrain serve` held
// it). Explicit --probe-pglite always runs it. A routine healthy
// `gbrain doctor` never pays it.
{
const probeRequested = args.includes('--probe-pglite');
let cfgForProbe: ReturnType<typeof loadConfig> = null;
try { cfgForProbe = loadConfig(); } catch { /* no config — nothing to diagnose */ }
const pgliteInitFailed = !engine && !fastMode && cfgForProbe?.engine === 'pglite';
let dirVerdict: import('../core/pglite-repair.ts').PgliteDirDiagnosis['verdict'] | undefined;
if (pgliteInitFailed) {
try {
const { inspectPgliteDataDir } = await import('../core/pglite-repair.ts');
const { resolve } = await import('node:path');
// Absolutize: a RELATIVE database_path would make the sidecar/backup
// lookups resolve against doctor's cwd instead of the engine's.
const pgliteDataDir = resolve(cfg.database_path || gbrainPath('brain.pglite'));
checks.push(computePgliteDataDirCheck(pgliteDataDir, inspectPgliteDataDir(pgliteDataDir)));
const pgliteDataDir = resolve(cfgForProbe!.database_path || gbrainPath('brain.pglite'));
const diagnosis = inspectPgliteDataDir(pgliteDataDir);
dirVerdict = diagnosis.verdict;
checks.push(computePgliteDataDirCheck(pgliteDataDir, diagnosis));
} catch {
// Best-effort: an unreadable config or fs failure must not stop doctor.
}
}
const dirExplainsFailure = dirVerdict === 'locked' || dirVerdict === 'missing';
if (probeRequested || (pgliteInitFailed && !dirExplainsFailure)) {
progress.start('doctor.pglite_probe');
const stopHb = startHeartbeat(progress, 'pglite scratch-store probe (cold start, can take 5–20s)…');
try {
checks.push(
await checkPgliteScratchProbe({
// A lock/missing dir explains the failure without the store being
// damaged — an explicit --probe-pglite there still reports on the
// runtime, but must not treat the store as the convicted party.
realInitFailed: pgliteInitFailed && !dirExplainsFailure,
storeDamageEvidence:
dirVerdict === 'wal-corruption-likely' || dirVerdict === 'unsupported-layout',
realStorePath: cfgForProbe?.database_path,
}),
);
} finally {
stopHb();
progress.finish();
}
} catch {
// Best-effort: an unreadable config or fs failure must not stop doctor.
}
}
@@ -8336,6 +8653,11 @@ export async function buildChecks(
// v0.36.0.0 (A5): ZE embedding key health + schema/config width consistency.
progress.heartbeat('ze_embedding_health');
checks.push(await checkZeEmbeddingHealth(engine));
// provider_sunset — brain pinned to a provider with an announced
// hosted-API shutdown; paste-ready migration hint with the actual
// column width. Warn before the date, fail after.
progress.heartbeat('provider_sunset');
checks.push(await checkProviderSunset(engine));
progress.heartbeat('embedding_width_consistency');
checks.push(await checkEmbeddingWidthConsistency(engine));
// v0.41.15.0 (T6, codex #19/#20) — facts.embedding column drift
@@ -8409,9 +8731,12 @@ export async function bootstrapDoctorChecks(engine: BrainEngine | null): Promise
return [];
}
const receipt = readReceipt(home);
const pushStatusFile = join(home, 'bootstrap', 'push-status.json');
// One reader for every push-status surface [D8]; per-root files [D13].
const { readPushStatuses, pushStatusFilesExist } = await import('../core/workspace-push.ts');
const pushStatuses = readPushStatuses();
const statusFilesOnDisk = pushStatusFilesExist();
const heartbeatFile = join(home, 'integrations', 'hooks', 'heartbeat.jsonl');
const hasBootstrapState = receipt !== null || existsSync(pushStatusFile) || existsSync(heartbeatFile);
const hasBootstrapState = receipt !== null || statusFilesOnDisk || existsSync(heartbeatFile);
if (!hasBootstrapState) return [];
const ws = receipt?.workspace_dir ?? null;
@@ -8452,43 +8777,124 @@ export async function bootstrapDoctorChecks(engine: BrainEngine | null): Promise
}
// 2. Push staleness [B4]: fail when the last successful push is >48h old
// AND the workspace tree is dirty (recent work provably unpushed).
// AND the workspace tree is dirty (recent work provably unpushed). Per-root
// status files [D13]: the WORST entry decides, so one workspace's success
// can never mask another's failure.
try {
if (existsSync(pushStatusFile)) {
if (pushStatuses.length > 0) {
const { PUSH_STALE_MS } = await import('./hook.ts'); // hook.ts owns the threshold (single source)
const s = JSON.parse(readFileSync(pushStatusFile, 'utf8')) as { ts?: string; ok?: boolean; reason?: string };
const t = s.ts ? Date.parse(s.ts) : NaN;
const stale = Number.isFinite(t) && Date.now() - t > PUSH_STALE_MS;
let dirty = false;
if (ws) {
try {
dirty = execFileSync('git', ['-C', ws, 'status', '--porcelain'], {
stdio: ['ignore', 'pipe', 'ignore'], timeout: 10_000,
}).toString().trim() !== '';
} catch { dirty = false; }
}
if (s.ok === false) {
const failing = pushStatuses.filter((s) => s.ok === false);
if (failing.length > 0) {
const s = failing[0]!;
const target = s.repoRoot ?? ws ?? undefined;
const rest = failing.length > 1 ? ` [+${failing.length - 1} more workspace(s)]` : '';
checks.push({
name: 'bootstrap_push_health',
status: 'warn',
message: `last workspace push FAILED (${s.ts ?? 'unknown'}): ${s.reason ?? 'unknown'} — run \`gbrain sources push${ws ? ` --path ${ws}` : ''}\``,
message: `last workspace push FAILED${target ? ` for ${target}` : ''} (${s.ts ?? 'unknown'}): ${s.reason ?? 'unknown'}${rest} — run \`gbrain sources push${target ? ` --path ${target}` : ''}\``,
});
} else if (stale && dirty) {
checks.push({
name: 'bootstrap_push_health',
status: 'fail',
message: `last successful push ${s.ts} (>48h) with a DIRTY workspace tree — recent agent memory is unpushed [B4]. Run \`gbrain sources push --path ${ws}\`.`,
});
} else if (stale) {
checks.push({ name: 'bootstrap_push_health', status: 'warn', message: `last successful push ${s.ts} (>48h ago); tree clean — likely just idle` });
} else {
checks.push({ name: 'bootstrap_push_health', status: 'ok', message: `last push ok (${s.ts ?? 'unknown'})` });
const stamps = pushStatuses.map((s) => Date.parse(s.ts ?? '')).filter((t) => Number.isFinite(t));
const stalest = stamps.length > 0 ? Math.min(...stamps) : NaN;
const staleIso = Number.isFinite(stalest) ? new Date(stalest).toISOString() : 'unknown';
const stale = Number.isFinite(stalest) && Date.now() - stalest > PUSH_STALE_MS;
let dirty = false;
if (ws) {
try {
dirty = execFileSync('git', ['-C', ws, 'status', '--porcelain'], {
stdio: ['ignore', 'pipe', 'ignore'], timeout: 10_000,
}).toString().trim() !== '';
} catch { dirty = false; }
}
if (stale && dirty) {
checks.push({
name: 'bootstrap_push_health',
status: 'fail',
message: `last successful push ${staleIso} (>48h) with a DIRTY workspace tree — recent agent memory is unpushed [B4]. Run \`gbrain sources push --path ${ws}\`.`,
});
} else if (stale) {
checks.push({ name: 'bootstrap_push_health', status: 'warn', message: `last successful push ${staleIso} (>48h ago); tree clean — likely just idle` });
} else {
checks.push({ name: 'bootstrap_push_health', status: 'ok', message: `last push ok (${staleIso})` });
}
}
} else if (statusFilesOnDisk) {
// Files exist but none parsed — the tolerant reader skips corrupt
// records; doctor must not let that read as "no news is good news".
checks.push({ name: 'bootstrap_push_health', status: 'warn', message: 'push status unreadable' });
}
} catch {
checks.push({ name: 'bootstrap_push_health', status: 'warn', message: 'push-status.json unreadable' });
checks.push({ name: 'bootstrap_push_health', status: 'warn', message: 'push status unreadable' });
}
// 2b. Durability job [B7/D7]: presence + LIVENESS. A presence-only check
// certifies dead jobs as healthy (the autopilot-status failure mode), so
// this warns on plist-present-but-unloaded and stale pull logs. Only warns
// when the user actually consented to the job; containers/cloud sandboxes
// are expected to have none.
try {
if (ws !== null && receipt !== null) {
const { detectExecutionEnvironment } = await import('../core/execution-env.ts');
const envKind = detectExecutionEnvironment();
if (envKind !== 'local') {
// Answered BEFORE the subprocess probes — cloud/container doctor
// runs must not pay launchctl/crontab spawns for an answer that is
// discarded (no scheduler exists there by design).
checks.push({
name: 'bootstrap_durability_job',
status: 'ok',
message: `no scheduler in this environment (${envKind}) — expected; per-turn and session-end pushes cover persistence`,
});
} else {
const { durabilityJobStatus } = await import('../core/brain-repo-durability.ts');
const { readInterviewState } = await import('../core/bootstrap/interview.ts');
const sourceId = receipt.source_id ?? 'workspace';
const js = durabilityJobStatus(sourceId);
let consented = false;
try {
const iv = readInterviewState(ws);
consented = iv.ok && (iv.state.answers['PERSIST_CRON']?.value ?? '').toLowerCase() === 'yes';
} catch { consented = false; }
if (!consented) {
if (js.kind !== 'none') {
checks.push({ name: 'bootstrap_durability_job', status: 'ok', message: `${js.kind} pull job present (not required by consent — fine)` });
}
// no consent + no job → nothing to check; stay silent
} else if (js.kind === 'none') {
checks.push({
name: 'bootstrap_durability_job',
status: 'warn',
message: `background persistence was consented (PERSIST_CRON=yes) but no scheduled job exists — run \`gbrain sources harden ${sourceId}\``,
});
} else if (js.live === false) {
checks.push({
name: 'bootstrap_durability_job',
status: 'warn',
message: `${js.kind} job is on disk but NOT loaded — a dead job looks healthy to presence checks. Re-run \`gbrain sources harden ${sourceId}\` to reload it.`,
});
} else if (!js.wrapperPresent) {
checks.push({
name: 'bootstrap_durability_job',
status: 'warn',
message: `${js.kind} job exists but its wrapper script is missing — re-run \`gbrain sources harden ${sourceId}\``,
});
} else if (js.logFresh === false) {
checks.push({
name: 'bootstrap_durability_job',
status: 'warn',
message: `${js.kind} job present but the pull log is stale (no run within 2× the interval) — the job may be dead; re-run \`gbrain sources harden ${sourceId}\``,
});
} else if (js.kind === 'crontab' && js.logFresh === undefined) {
// The crontab LINE existing proves installation, not that the cron
// daemon runs it — with no pull log yet we can't claim liveness.
checks.push({ name: 'bootstrap_durability_job', status: 'ok', message: 'crontab pull job installed (no run logged yet — liveness confirmed once it first fires)' });
} else {
checks.push({ name: 'bootstrap_durability_job', status: 'ok', message: `${js.kind} pull job present and live` });
}
}
}
} catch { /* best-effort — durability probing never fails doctor */ }
// 3. One-live-serve / lock collision note. A live serve is the healthy
// shape (it provides hook IPC); the note names the v1 contract.
try {
src/commands/embed.ts
+265 -4
View File
@@ -198,6 +198,22 @@ export interface EmbedResult {
failure_samples: string[];
/** True if this run was a dry-run. */
dryRun: boolean;
/**
* Chunkless-page safety net (`--stale` only): pages with non-empty
* content but zero `content_chunks` rows that this run chunked (or, in
* dryRun, would chunk) so their new NULL-embedding chunks fold into the
* SAME pass. 0 on a healthy brain. Additive field — see
* `ChunklessPageRow` for the detection rationale.
*/
chunkless_pages_healed: number;
/**
* Set when a single-flight run did NO work because another backfill holds
* the per-source embed lock. A hard-killed (SIGKILL/crash) run leaves its
* lock behind for up to EMBED_BACKFILL_LOCK_TTL_MIN — callers that promise
* "re-run to resume" (migrate embeddings) use this to say so instead of
* misreporting embed failures.
*/
lock_skipped?: boolean;
/**
* E1 (paced-backfill): end-of-run pacing telemetry. Present ONLY when pacing
* was active (enabled bundle). The number the operator could not get from an
@@ -317,6 +333,7 @@ export async function runEmbedCore(engine: BrainEngine, opts: EmbedOpts): Promis
failures: 0,
failure_samples: [],
dryRun: !!opts.dryRun,
chunkless_pages_healed: 0,
};
if (opts.slugs && opts.slugs.length > 0) {
@@ -375,6 +392,7 @@ export async function runEmbedCore(engine: BrainEngine, opts: EmbedOpts): Promis
try { await h.release(); } catch { /* best-effort */ }
}
serr(` [embed] another backfill is already running for source "${sid}"; skipping (single-flight).`);
result.lock_skipped = true;
return result;
}
sfLocks.push(lock);
@@ -529,6 +547,7 @@ export async function runEmbed(engine: BrainEngine, args: string[]): Promise<Emb
return {
embedded: 0, skipped: 0, would_embed: 0, total_chunks: 0,
pages_processed: 0, failures: 0, failure_samples: [], dryRun: false,
chunkless_pages_healed: 0,
};
}
@@ -988,6 +1007,201 @@ async function embedAll(
}
}
/**
* Chunkless-page safety net for `embed --stale`. `listStaleChunks` /
* `countStaleChunks` only ever look at `content_chunks` rows where
* `embedding IS NULL` — a page written directly via `putPage` that never
* went through chunking (e.g. an enrichment-generated entity stub) has NO
* chunk row at all, so it is invisible to that scan forever, even after
* unlimited `embed --stale` runs.
*
* This sweep finds pages with non-empty content (`compiled_truth` and/or
* `timeline` — both are chunked independently, mirroring `embedPage`'s
* chunkless branch) and zero `content_chunks` rows
* (`engine.listChunklessPagesWithContent`, which already excludes
* quarantined + embed_skip pages — both intentionally chunkless). The new
* chunk rows land with `embedding = NULL`, so they flow into the SAME
* `embed --stale` pass via the existing cursor below — no separate embed
* step needed here.
*
* dryRun chunks locally (a pure, in-memory operation) to report an
* accurate count without writing anything, matching embedPage's dry-run
* contract (including `pages_processed`, which embedPage's own dry-run
* branch increments for exactly this "examined, didn't write" case).
*
* Race note (review catch, three rounds — ACCEPTED RESIDUAL RISK, not
* fully closed): between listing a page and writing its chunks, a
* concurrent writer (sync, another `put_page`) could change or chunk the
* SAME page. Two mitigations, both bounded — full atomicity (a
* transaction/version-guarded conditional write inside `upsertChunks`)
* would need a new engine primitive shared by every `upsertChunks` caller,
* which is out of scope for a chunkless-page safety net:
* 1. Immediately before writing, re-fetch the LIVE page via `getPage`
* and build `inputs` from ITS CURRENT content, not the batch-list
* snapshot — closes the "content changed but still chunkless"
* sub-case, not just the "chunks appeared" one.
* 2. Re-check `getChunks` right after that same fetch — skip (don't
* overwrite) if chunks now exist AT THE TIME OF THE CHECK.
* What this does NOT close: a writer that inserts chunks in the gap
* BETWEEN step 2's check and the `upsertChunks` call immediately below it
* (no intervening `await` other than that one call, but `upsertChunks`
* itself is not conditioned on the check — this is still check-then-write,
* not compare-and-swap) can still have its chunks overwritten — HONESTLY:
* `upsertChunks` treats its input as the full desired chunk set for that
* page and deletes any existing chunk_index absent from it, so a
* concurrent writer's chunks landing in that exact gap CAN be replaced
* with this sweep's stale-content chunks (embedding NULL). This is the
* SAME check-then-write window `embedPage`'s existing single-page
* chunkless branch already ships with today (that branch doesn't even
* have step 2's re-check) — no new race CLASS is introduced, and the
* window here is a single sequential getPage+getChunks+upsertChunks
* instead of spanning a whole batch. The blast radius is bounded: the
* page is NOT deleted or corrupted, just re-chunked from a stale
* snapshot, and the NEXT write to that page (sync, another edit) that
* actually chunks it restores correct content — this sweep's own
* predicate is idempotent and doesn't compound the drift. Closing this
* fully (true atomicity) is tracked as a follow-up, not blocking this
* safety net.
*
* Per-page failure isolation (review catch): one malformed/oversized
* chunkless page must not abort the sweep and, with it, the entire
* `--stale` run before the normal NULL-embedding pass even starts — that
* would make the safety net WORSE than the bug it fixes. Each page's
* work is try/caught; a failure is recorded (`EmbedResult.failures` +
* `failure_samples`, same convention as every other embed failure path)
* and the sweep moves on.
*
* Bounded, keyset-paginated (like listStalePagesForExtraction) — a safety
* net for a rare drift case, not the primary bulk-chunking path. `BATCH_SIZE`
* is deliberately small (unlike the 2000-chunk-row default elsewhere in
* this file): each row here carries a FULL page body (`compiled_truth` +
* `timeline`), so a large batch of large pages is a real memory/latency
* concern the metadata-only `listStaleChunks` rows never had (review
* catch). It still respects the caller's pacer (no-op when pacing is off)
* and a soft wall-clock cap (`GBRAIN_EMBED_TIME_BUDGET_MS`) so a
* pathologically large damaged brain can't run this sweep unbounded — it
* heals what it can and reports the rest for the next `embed --stale` run
* (the SQL predicate is idempotent; nothing here requires finishing in one
* pass). `startedAt` is shared with the caller's overall run clock (review
* catch) — healing and the main stale loop draw from ONE combined budget
* window, not two independent 30-minute ones. `catchUp` mirrors the main
* loop's own `--catch-up` handling: removes the cap entirely (the keyset
* cursor still terminates on its own; `signal` remains the abort path).
*/
async function healChunklessPages(
engine: BrainEngine,
sourceId: string | undefined,
dryRun: boolean,
result: EmbedResult,
quiet: boolean | undefined,
signal: AbortSignal | undefined,
pacer: DbPacer | undefined,
startedAt: number,
catchUp: boolean,
): Promise<void> {
const BATCH_SIZE = 50;
const BUDGET_MS: number | null = catchUp
? null
: parseInt(process.env.GBRAIN_EMBED_TIME_BUDGET_MS || `${30 * 60 * 1000}`, 10);
const activePacer = pacer ?? createNoopPacer();
let afterPageId: number | undefined;
let pagesHealed = 0;
let budgetExceeded = false;
const buildInputs = (compiledTruth: string, timeline: string): ChunkInput[] => {
const inputs: ChunkInput[] = [];
if (compiledTruth.trim()) {
for (const c of chunkText(compiledTruth)) {
inputs.push({ chunk_index: inputs.length, chunk_text: c.text, chunk_source: 'compiled_truth' });
}
}
if (timeline.trim()) {
for (const c of chunkText(timeline)) {
inputs.push({ chunk_index: inputs.length, chunk_text: c.text, chunk_source: 'timeline' });
}
}
return inputs;
};
// BUDGET_MS === null means catch-up: no wall-clock cap on this sweep,
// mirroring the main stale loop's own --catch-up handling below.
const overBudget = (): boolean => BUDGET_MS != null && Date.now() - startedAt > BUDGET_MS;
// eslint-disable-next-line no-constant-condition
while (true) {
if (isAborted(signal)) break;
if (overBudget()) { budgetExceeded = true; break; }
const batch = await observed(activePacer, () => engine.listChunklessPagesWithContent({
batchSize: BATCH_SIZE,
...(afterPageId != null && { afterPageId }),
...(sourceId && { sourceId }),
}));
if (batch.length === 0) break;
afterPageId = batch[batch.length - 1].id;
for (const page of batch) {
if (isAborted(signal)) break;
if (overBudget()) { budgetExceeded = true; break; }
try {
if (dryRun) {
// dryRun never writes, so there's no live-refetch race to close —
// chunk the listed snapshot directly (matches embedPage's own
// dry-run, which chunks whatever getPage returned at call time).
const inputs = buildInputs(page.compiled_truth, page.timeline);
// Whitespace-only content (SQL prefilter is `<> ''`, not
// trim-aware) chunks to nothing — matches embedPage's contract.
if (inputs.length === 0) continue;
result.total_chunks += inputs.length;
result.would_embed += inputs.length;
result.pages_processed++;
pagesHealed++;
continue;
}
// Re-fetch the LIVE page + re-check chunks immediately before
// writing (see race note above): chunk CURRENT content, and skip
// rather than clobber if a concurrent writer already chunked this
// page since we listed it.
const [livePage, stillChunkless] = await Promise.all([
observed(activePacer, () => engine.getPage(page.slug, { sourceId: page.source_id })),
observed(activePacer, () => engine.getChunks(page.slug, { sourceId: page.source_id })),
]);
if (!livePage || stillChunkless.length > 0) continue;
const inputs = buildInputs(livePage.compiled_truth, livePage.timeline);
if (inputs.length === 0) continue;
await observed(activePacer, () =>
engine.upsertChunks(page.slug, inputs, { sourceId: page.source_id }),
);
pagesHealed++;
try {
await activePacer.pace(signal);
} catch (e) {
if (!(e instanceof AbortError)) throw e;
}
} catch (e) {
if (isAborted(signal)) break;
recordFailure(result, 1, page.slug, e);
serr(`\n [embed] chunkless-page heal failed for ${page.slug}: ${e instanceof Error ? e.message : e}`);
}
}
if (budgetExceeded || batch.length < BATCH_SIZE) break;
}
result.chunkless_pages_healed = pagesHealed;
if (pagesHealed > 0 && !quiet) {
if (dryRun) {
serr(`[embed] [dry-run] would chunk ${pagesHealed} page(s) with non-empty content but zero content_chunks rows`);
} else {
serr(`[embed] chunked ${pagesHealed} page(s) that had non-empty content but zero content_chunks rows (embedding them in this pass)`);
}
}
if (budgetExceeded && !quiet) {
serr(`[embed] chunkless-page sweep hit its time budget (${BUDGET_MS}ms) with more pages left; re-run embed --stale to continue healing them`);
}
}
/**
* SQL-side stale path: replaces the listPages + per-page getChunks
* walk with a count + slug-grouped SELECT. Preserves the existing
@@ -1028,11 +1242,38 @@ async function embedAllStale(
signature?: string,
externalSignal?: AbortSignal,
) {
// Shared wall-clock anchor (review catch): the healing sweep below and the
// main stale loop's own budget timer (further down) both measure against
// this SAME start time, so a run's total wall-clock spend stays capped at
// ONE `GBRAIN_EMBED_TIME_BUDGET_MS` window instead of summing two
// independent 30-minute budgets.
const overallStartedAt = Date.now();
// D7: thread sourceId so source-scoped runs only count + visit
// that source's NULL embeddings.
const sourceOpt = sourceId ? { sourceId } : undefined;
const includeNullSig = !!staleOpts?.includeNullSignature;
// Chunkless-page safety net: pre-flight count mirrors the countStaleChunks
// short-circuit just below — a healthy brain pays one extra SELECT
// count(*) and does no further work. Only when pages are actually found
// do we pay for the keyset-paginated chunk sweep. Chunking here (before
// countStaleChunks) means any newly-written NULL-embedding chunks flow
// into the SAME pass via the existing cursor.
const chunklessCount = await engine.countChunklessPagesWithContent(sourceOpt);
if (chunklessCount > 0) {
await healChunklessPages(
engine, sourceId, dryRun, result, staleOpts?.quiet, externalSignal, staleOpts?.pacer,
overallStartedAt, !!staleOpts?.catchUp,
);
}
// Review catch: an abort during healing must stop the run HERE, before
// falling through into invalidateStaleSignatureEmbeddings below (which —
// pre-existing, unchanged by this PR — does not itself check
// externalSignal). Without this, a caller-cancelled run could still NULL
// out signature-drifted embeddings and exit, leaving retrieval degraded.
if (isAborted(externalSignal)) return;
// v0.41.31: re-embed pages whose embedding_signature drifted (model/dims
// swap). dry-run must NOT mutate, so it counts signature-stale via the
// widened predicate; a live run NULLs them first so the existing
@@ -1085,7 +1326,15 @@ async function embedAllStale(
if (staleCount === 0) {
if (!staleOpts?.quiet) {
if (dryRun) {
slog('[dry-run] Would embed 0 chunks (0 stale found)');
// dryRun never writes, so a healed-but-hypothetical chunkless page's
// chunks never land in content_chunks and staleCount can't see them
// — report result.would_embed (already includes them) instead of a
// bare "0 chunks" that would contradict the returned EmbedResult.
if (result.would_embed > 0) {
slog(`[dry-run] Would embed ${result.would_embed} chunks (0 stale found; ${result.chunkless_pages_healed} chunkless page(s) would be chunked)`);
} else {
slog('[dry-run] Would embed 0 chunks (0 stale found)');
}
} else {
slog('Embedded 0 chunks (0 stale found)');
}
@@ -1101,7 +1350,16 @@ async function embedAllStale(
// made `embed.pages` claim total:1 next to a summary naming a much larger
// stale count. docs/progress-events.md allows omitting `total` when it is
// not known up front; it does not allow asserting a wrong one.
if (!staleOpts?.quiet) slog(`[dry-run] Would embed ${staleCount} stale chunks`);
//
// Log result.would_embed (staleCount + any chunkless-page-healing
// contribution from above), not the bare staleCount — otherwise this
// line understates the total whenever chunkless pages were also found.
if (!staleOpts?.quiet) {
const chunklessNote = result.chunkless_pages_healed > 0
? `, including ${result.chunkless_pages_healed} chunkless page(s)`
: '';
slog(`[dry-run] Would embed ${result.would_embed} stale chunks${chunklessNote}`);
}
return;
}
@@ -1132,9 +1390,12 @@ async function embedAllStale(
? null
: parseInt(process.env.GBRAIN_EMBED_TIME_BUDGET_MS || `${30 * 60 * 1000}`, 10);
const budgetController = new AbortController();
const budgetStart = Date.now();
// Shares overallStartedAt with the chunkless-page healing sweep above
// (review catch) so the two phases draw from ONE combined budget window
// instead of each getting a fresh 30 minutes.
const budgetStart = overallStartedAt;
let budgetTimer = BUDGET_MS != null
? setTimeout(() => budgetController.abort(), BUDGET_MS)
? setTimeout(() => budgetController.abort(), Math.max(0, budgetStart + BUDGET_MS - Date.now()))
: undefined;
// E-4 (paced-backfill): the budget measures WORK, not waiting. After each
// batch, re-arm the timer to fire at start + BUDGET + total-paced-sleep, so a
src/commands/export.ts
+12 -3
View File
@@ -108,7 +108,11 @@ export async function runExport(engine: BrainEngine, args: string[]) {
let exported = 0;
for (const page of pages) {
const tags = await engine.getTags(page.slug);
// Slugs are unique per source, not brain-wide, so both sidecar reads are
// pinned to the page's own source. Unscoped, `getTags` falls back to
// `source_id = 'default'` and stamps the default source's tags onto a
// same-slug page from another source (dropping its real ones).
const tags = await engine.getTags(page.slug, { sourceId: page.source_id });
const md = serializeMarkdown(
page.frontmatter,
page.compiled_truth,
@@ -120,8 +124,13 @@ export async function runExport(engine: BrainEngine, args: string[]) {
mkdirSync(dirname(filePath), { recursive: true });
writeFileSync(filePath, md);
// Export raw data as sidecar JSON
const rawData = await engine.getRawData(page.slug);
// Export raw data as sidecar JSON. Unscoped, this matches the slug in
// EVERY source and the loop below merges the rows into one sidecar keyed
// by `rd.source`, so another source's raw data silently overwrites this
// page's own on a key collision.
const rawData = await engine.getRawData(page.slug, undefined, {
sourceId: page.source_id,
});
if (rawData.length > 0) {
const slugParts = page.slug.split('/');
const rawDir = join(outDir, ...slugParts.slice(0, -1), '.raw');
src/commands/extract.ts
+44 -4
View File
@@ -81,8 +81,10 @@ const BATCH_SIZE = 100;
const STALE_BATCH_SIZE = Math.max(1, Number(process.env.GBRAIN_EXTRACT_STALE_BATCH) || 25);
// v0.42.7: wall-clock budget for one `extract --stale` invocation (default
// 30 min). `--catch-up` removes the cap (loops until 0 stale). Mirrors
// embedAllStale's time-budget shape.
const STALE_TIME_BUDGET_MS = Math.max(1000, Number(process.env.GBRAIN_EXTRACT_TIME_BUDGET_MS) || 30 * 60 * 1000);
// embedAllStale's time-budget shape. Exported so the #2849 deferred-sweep
// submitters (sync's size-gate defer branch + the jobs continuation chain)
// derive their job timeout_ms from the SAME budget instead of hardcoding.
export const STALE_TIME_BUDGET_MS = Math.max(1000, Number(process.env.GBRAIN_EXTRACT_TIME_BUDGET_MS) || 30 * 60 * 1000);
/**
* v0.42.7 (#1696): best-effort extraction stamp for the source-correct write
@@ -488,15 +490,53 @@ export async function extractLinksFromFile(
// --- Timeline extraction ---
/**
* Index of the first dash (—, –, -) that can serve as the Source — Summary
* delimiter: it must have whitespace on both sides and sit outside every
* markdown-link span. Hyphens inside link targets
* (`../people/alice-example.md`) and dashes inside link labels
* (`[Deals — Q1 Review](...)`) are content, not delimiters — splitting on
* them shatters one entry into two fragments whose halves re-insert on
* every sync (the (page_id, date, summary, source) uniqueness sees each
* fragment shape as a new row). Returns -1 when the line has no delimiter.
*/
function findDelimiterOutsideLinks(text: string): number {
let depth = 0;
for (let i = 0; i < text.length; i++) {
const c = text[i];
if (c === '[' || c === '(') depth++;
else if (c === ']' || c === ')') { if (depth > 0) depth--; }
else if (
depth === 0 &&
(c === '—' || c === '–' || c === '-') &&
i > 0 && /\s/.test(text[i - 1]) &&
i + 1 < text.length && /\s/.test(text[i + 1])
) {
return i;
}
}
return -1;
}
/** Extract timeline entries from markdown content */
export function extractTimelineFromContent(content: string, slug: string): ExtractedTimelineEntry[] {
const entries: ExtractedTimelineEntry[] = [];
// Format 1: Bullet — - **YYYY-MM-DD** | Source — Summary
const bulletPattern = /^-\s+\*\*(\d{4}-\d{2}-\d{2})\*\*\s*\|\s*(.+?)\s*[—–-]\s*(.+)$/gm;
// The delimiter search is link-aware (see findDelimiterOutsideLinks); a
// bullet with no delimiter (e.g. an auto-generated backlink line
// `- **date** | Referenced in [X](y.md)`) is kept whole as the summary
// rather than dropped or fragmented.
const bulletPattern = /^-\s+\*\*(\d{4}-\d{2}-\d{2})\*\*\s*\|\s*(.+)$/gm;
let match;
while ((match = bulletPattern.exec(content)) !== null) {
entries.push({ slug, date: match[1], source: match[2].trim(), summary: match[3].trim() });
const rest = match[2].trim();
const at = findDelimiterOutsideLinks(rest);
if (at >= 0) {
entries.push({ slug, date: match[1], source: rest.slice(0, at).trim(), summary: rest.slice(at + 1).trim() });
} else {
entries.push({ slug, date: match[1], source: 'markdown', summary: rest });
}
}
// Format 2: Header — ### YYYY-MM-DD — Title
src/commands/hook.ts
+276 -34
View File
@@ -67,6 +67,15 @@ import {
} from '../core/transcripts/claude-code-jsonl.ts';
import { CLAUDE_HOOK_OUTPUT_CAP_CHARS } from '../core/bootstrap/host-specs.ts';
import { readManifest, readReceipt, type InstallReceipt } from '../core/bootstrap/format.ts';
import { githubOwnerRepoString } from '../core/repo-visibility.ts';
import { detectExecutionEnvironment } from '../core/execution-env.ts';
import {
readPushStatuses,
readPushStatusForRoot,
sanitizePushReason,
summarizePushStatuses,
workspaceRootHash,
} from '../core/workspace-push.ts';
import { realpathOrResolve } from '../core/path-confine.ts';
// ── Tunables ────────────────────────────────────────────────────────────────
@@ -112,6 +121,17 @@ const USER_PROMPT_WINDOW_TURNS = 4;
export const PRIOR_CONTEXT_MAX_BYTES = 32 * 1024;
/** user-prompt transcript parse budget (tail bytes — the window only needs the newest turns). */
const USER_PROMPT_TRANSCRIPT_MAX_BYTES = 2 * 1024 * 1024;
/** stop-hook push [D3]: hard budget for the debounce decision + detached spawn
* (the spawn itself is instant; the budget bounds the two 1s git probes). */
const STOP_PUSH_DEADLINE_MS = 3000;
/** stop-hook push debounce default (minutes) for local + ephemeral-container
* environments; cloud-sandbox defaults to 0 (every turn) — a reclaimed VM's
* tail loss is permanent, everywhere else SessionStart recovery covers it [D17]. */
export const STOP_PUSH_DEBOUNCE_MIN_DEFAULT = 5;
/** failure banner [D19]: re-announce floor while the same failure persists. */
export const PUSH_ANNOUNCE_REFIRE_MS = 30 * 60 * 1000;
/** failure banner budget (well under ENG-1's whole-payload cap). */
const PUSH_BANNER_MAX_CHARS = 300;
// ── Test seam ───────────────────────────────────────────────────────────────
@@ -591,16 +611,19 @@ async function lastSessionLine(): Promise<string | null> {
async function pushStatusNote(): Promise<string | null> {
try {
const home = await resolveHome();
const p = join(home, 'bootstrap', 'push-status.json');
if (!existsSync(p)) return null;
const s = JSON.parse(readFileSync(p, 'utf8')) as { ts?: string; ok?: boolean; reason?: string };
if (s.ok === false) {
return `Workspace push is FAILING (since ${s.ts ?? 'unknown'}): ${s.reason ?? 'unknown reason'} — run gbrain doctor`;
// One reader + one aggregation for every status surface [D8]; per-root
// files [D13] so one workspace's success can't mask another's failure.
const entries = readPushStatuses();
if (entries.length === 0) return null;
const { failing, stalestTs } = summarizePushStatuses(entries);
if (failing.length > 0) {
const e = failing[0]!;
const which = e.repoRoot ? ` for ${e.repoRoot}` : '';
const rest = failing.length > 1 ? ` [+${failing.length - 1} more workspace(s)]` : '';
return `Workspace push${which} is FAILING (since ${e.ts ?? 'unknown'}): ${sanitizePushReason(e.reason)}${rest} — run gbrain doctor`;
}
const t = s.ts ? Date.parse(s.ts) : NaN;
if (Number.isFinite(t) && Date.now() - t > PUSH_STALE_MS) {
return `Workspace push: last success ${s.ts} (>48h ago) — recent work may be unpushed [B4]`;
if (stalestTs !== null && Date.now() - stalestTs > PUSH_STALE_MS) {
return `Workspace push: last success ${new Date(stalestTs).toISOString()} (>48h ago) — recent work may be unpushed [B4]`;
}
return null;
} catch {
@@ -677,13 +700,10 @@ async function resolveBootstrapWorkspaceRoot(ws: string): Promise<string | null>
return root;
}
/** owner/name from a github https/ssh remote URL, or null. Local mirror of
* repo.ts's parser (kept here so the engine-free hook doesn't import repo.ts). */
/** owner/name from a github https/ssh remote URL, or null. Canonical parser
* (repo-visibility.ts is engine-free, so the hook contract holds). */
function githubOwnerName(url: string): string | null {
const m =
/^https:\/\/github\.com\/([^/]+)\/([^/]+?)(?:\.git)?\/?$/.exec(url.trim()) ??
/^git@github\.com:([^/]+)\/([^/]+?)(?:\.git)?$/.exec(url.trim());
return m ? `${m[1]}/${m[2]}` : null;
return githubOwnerRepoString(url);
}
/**
@@ -706,14 +726,22 @@ async function repoPhaseComplete(root: string): Promise<boolean> {
if (!receipt || typeof receipt.repo_url !== 'string' || receipt.repo_url.length === 0) return false;
if (realpathOrResolve(receipt.workspace_dir) !== realpathOrResolve(root)) return false;
const want = githubOwnerName(receipt.repo_url);
if (!want) return false;
const fetchUrl = await tryExecAsync('git', ['-C', root, 'remote', 'get-url', 'origin']);
if (githubOwnerName(fetchUrl ?? '') !== want) return false;
// Push URL (remote.origin.pushurl) via the config key directly (no dash-flag):
// unset → `git push` uses the fetch URL (already matched). Only a configured
// push URL that points elsewhere blocks the push.
const pushUrl = await tryExecAsync('git', ['-C', root, 'config', 'remote.origin.pushurl']);
return !pushUrl || githubOwnerName(pushUrl) === want;
if (want) {
if (githubOwnerName(fetchUrl ?? '') !== want) return false;
return !pushUrl || githubOwnerName(pushUrl) === want;
}
// Non-github repo_url (self-hosted / explicitly-trusted transports): bind
// by EXACT URL equality — the recorded url is what the repo phase (or the
// operator) verified, and a later remote redirect must still block the
// push. Without this branch, every non-github install's no-daemon push
// deferred forever (post-#4024 regression).
if ((fetchUrl ?? '').trim() !== receipt.repo_url) return false;
return !pushUrl || pushUrl.trim() === receipt.repo_url;
} catch {
return false;
}
@@ -749,13 +777,7 @@ async function dirtyTreePush(
try {
const root = await resolveBootstrapWorkspaceRoot(ws);
if (!root) return null;
const [status, aheadRaw] = await Promise.all([
tryExecAsync('git', ['-C', root, 'status', '--porcelain']),
tryExecAsync('git', ['-C', root, 'rev-list', '--count', '@{u}..HEAD']),
]);
const dirty = (status ?? '') !== '';
const ahead = aheadRaw !== null ? parseInt(aheadRaw, 10) || 0 : 0;
if (!dirty && ahead === 0) return null; // clean + up to date → nothing to recover
if (!(await treeNeedsPush(root))) return null; // clean + up to date → nothing to recover
// There IS unpushed work. Defer until the repo phase verified privacy +
// recorded repo_url — never recover-push to an unverified origin
// (create-repo-first race). Only fires when work actually exists (P2-1).
@@ -783,6 +805,181 @@ async function dirtyTreePush(
}
}
/** True when the workspace has uncommitted changes or commits ahead of
* upstream — shared by the SessionStart recovery push and the stop-hook
* per-turn push. Two 1s-capped git probes; never throws. */
async function treeNeedsPush(root: string): Promise<boolean> {
// Dirty tree → always needs a push. For "ahead", measure against the SAME
// ref workspacePush targets (origin/<default-branch>), NOT @{u}: a branch
// with no upstream makes `@{u}..HEAD` error → 0, which would report a clean
// + committed-but-unpushed tree as push_clean and silently strand it (the
// exact tail-loss the per-turn push exists to prevent). When the origin ref
// doesn't resolve yet (never pushed), any commit past the empty tree counts
// as needs-push.
const status = await tryExecAsync('git', ['-C', root, 'status', '--porcelain']);
if ((status ?? '') !== '') return true;
const branch = await tryExecAsync('git', ['-C', root, 'branch', '--show-current']);
const b = (branch ?? '').trim();
if (b) {
const ahead = await tryExecAsync('git', ['-C', root, 'rev-list', '--count', `origin/${b}..HEAD`]);
if (ahead !== null) return (parseInt(ahead, 10) || 0) > 0;
// origin/<b> doesn't exist (never pushed) → any local commit needs pushing.
const have = await tryExecAsync('git', ['-C', root, 'rev-list', '--count', 'HEAD']);
return (parseInt(have ?? '0', 10) || 0) > 0;
}
// Detached HEAD / no branch name — fall back to the upstream measure.
const ahead = await tryExecAsync('git', ['-C', root, 'rev-list', '--count', '@{u}..HEAD']);
return ahead !== null && (parseInt(ahead, 10) || 0) > 0;
}
// ── stop-hook per-turn push [D3/D17/D20] ────────────────────────────────────
//
// SessionEnd never fires on /exit (upstream: closed not-planned), can't fire
// on crash, and a cloud sandbox VM may simply be reclaimed between turns —
// so the Stop boundary (fires after EVERY assistant turn) is the only cadence
// that always runs while work exists. Debounced per workspace root, detached
// spawn (instant), fail-open everywhere.
function stopPushStatePath(root: string): string {
return join(resolveGbrainHome(), 'bootstrap', `stop-push-${workspaceRootHash(root)}.json`);
}
/** Debounce resolution: env GBRAIN_STOP_PUSH_DEBOUNCE_MIN (minutes; 0 = every
* turn) → file-plane config hooks.stop_push_debounce_min → environment-kind
* default (cloud-sandbox: 0, everything else: 5). */
function stopPushDebounceMs(): number {
const env = process.env.GBRAIN_STOP_PUSH_DEBOUNCE_MIN;
if (env !== undefined) {
const n = Number.parseInt(env, 10);
if (Number.isFinite(n) && n >= 0) return n * 60_000;
}
try {
const cfg = loadConfig();
const v = cfg?.hooks?.stop_push_debounce_min;
const n = typeof v === 'number' ? v : typeof v === 'string' ? Number.parseInt(v, 10) : NaN;
if (Number.isFinite(n) && n >= 0) return n * 60_000;
} catch {
/* tolerant read — fall through to the default */
}
return detectExecutionEnvironment() === 'cloud-sandbox' ? 0 : STOP_PUSH_DEBOUNCE_MIN_DEFAULT * 60_000;
}
/** Floor for the [D20] failing-status retry cadence: a stuck push (e.g. gh
* unauthenticated for a day) must not re-run the full network ladder on every
* single turn — one retry a minute keeps recovery fast without the storm. */
export const STOP_PUSH_FAILING_RETRY_FLOOR_MS = 60_000;
/** Decide + (maybe) spawn the per-turn push. Returns the heartbeat reason.
* Ordered cheapest-first: the debounce (two file reads) answers the common
* case before any git subprocess runs — repoPhaseComplete's git probes only
* execute on turns that might actually spawn a push. */
async function stopPushIfDue(ws: string, io: HookIo): Promise<string> {
if (process.env.GBRAIN_STOP_PUSH === '0') return 'push_disabled';
const root = await resolveBootstrapWorkspaceRoot(ws);
if (!root) return 'push_skipped_not_bootstrap';
const stateP = stopPushStatePath(root);
let lastTs: number | null = null;
try {
const s = JSON.parse(readFileSync(stateP, 'utf8')) as { ts?: string };
const t = Date.parse(s.ts ?? '');
if (Number.isFinite(t)) lastTs = t;
} catch {
/* missing/corrupt state → due (fail-open) */
}
// [D20] a failing push bypasses the normal debounce so recovery is fast —
// but with a 60s floor so a persistently failing push can't re-run the
// network verification ladder on every turn (the push lock bounds
// concurrency, not cadence; the banner is already showing the failure).
const failing = readPushStatusForRoot(root)?.ok === false;
const now = Date.now();
// Healthy: the normal debounce (0 = every turn in cloud). Failing: a fixed
// 60s retry floor — faster than a long local debounce so a transient failure
// recovers within a turn or two, but NEVER every-turn (a Math.min against the
// cloud debounce of 0 was a re-run-the-ladder-every-turn storm; adversarial
// review caught it).
const windowMs = failing ? STOP_PUSH_FAILING_RETRY_FLOOR_MS : stopPushDebounceMs();
if (lastTs !== null && now - lastTs < windowMs) return 'push_debounced';
// Same privacy gate as SessionEnd: never push before the repo phase has
// verified the origin and recorded repo_url (create-repo-first race).
if (!(await repoPhaseComplete(root))) return 'push_deferred_repo_pending';
if (!(await treeNeedsPush(root))) return 'push_clean';
try {
// Written BEFORE the spawn so repeated fail-fast children stay debounced
// on the healthy path; the [D20] failing-status bypass handles retries.
mkdirSync(join(resolveGbrainHome(), 'bootstrap'), { recursive: true, mode: 0o700 });
const tmp = `${stateP}.tmp-${process.pid}`;
writeFileSync(tmp, JSON.stringify({ ts: new Date(now).toISOString(), root }) + '\n', { mode: 0o600 });
renameSync(tmp, stateP);
} catch {
/* state-write failure must not block the push itself */
}
try {
(io.spawnPush ?? spawnDetachedPush)(root);
return 'push_spawned';
} catch {
return 'push_unavailable';
}
}
// ── push-failure banner [D5/D13/D19] ────────────────────────────────────────
interface PushAnnounceState {
announced_ts?: string;
last_announce_at?: string;
}
/**
* The pending ≤300-char failure banner, or null. `record()` marks the due
* failures announced and is called ONLY after the banner actually reached
* stdout — a deadline-suppressed banner must re-fire next turn. Announce
* state is a sidecar next to each per-root status file (`<file>.announced`):
* each new failure `ts` announces once, then re-announces at most every
* PUSH_ANNOUNCE_REFIRE_MS while the failure persists [D19].
*/
function pendingPushFailureBanner(): { text: string; record: () => void } | null {
try {
const failing = readPushStatuses().filter((e) => e.ok === false);
if (failing.length === 0) return null;
const now = Date.now();
const due = failing.filter((e) => {
try {
const s = JSON.parse(readFileSync(`${e.file}.announced`, 'utf8')) as PushAnnounceState;
if (s.announced_ts !== e.ts) return true;
const last = Date.parse(s.last_announce_at ?? '');
return !Number.isFinite(last) || now - last > PUSH_ANNOUNCE_REFIRE_MS;
} catch {
return true; // never announced (or unreadable state) → due
}
});
if (due.length === 0) return null;
const first = due[0]!;
const which = first.repoRoot ?? 'the workspace';
const more = due.length > 1 ? ` (+${due.length - 1} more workspace(s))` : '';
const text = (
`NOTICE: the background workspace push for ${which} is FAILING ` +
`(${sanitizePushReason(first.reason)})${more} — work is committed locally ` +
'but NOT on GitHub. Run gbrain doctor.'
).slice(0, PUSH_BANNER_MAX_CHARS);
const record = () => {
for (const e of due) {
try {
writeFileSync(
`${e.file}.announced`,
JSON.stringify({ announced_ts: e.ts, last_announce_at: new Date(now).toISOString() }) + '\n',
{ mode: 0o600 },
);
} catch {
/* fail-open — worst case the banner re-fires */
}
}
};
return { text, record };
} catch {
return null;
}
}
// ── user-prompt [ENG-1, S3#8, A9] ───────────────────────────────────────────
interface UserPromptOutcome {
@@ -798,7 +995,18 @@ async function hookUserPrompt(io: HookIo): Promise<number> {
if (!expired) write(io, s);
};
// [D5/D19] Same-session failure surfacing: a refused/failed background push
// becomes visible on the NEXT turn — to the model via additionalContext AND
// to the human via systemMessage ("never silent" must not depend on the
// model choosing to relay its own tooling's failure). Embedded in the main
// payload when one is written; emitted alone on every degraded path.
// Computed INSIDE the deadline-raced closure: its sync file reads must be
// budgeted by the 800ms deadline, not free-ride before the race starts.
let banner: ReturnType<typeof pendingPushFailureBanner> = null;
let wrotePayload = false;
const work = (async (): Promise<UserPromptOutcome> => {
banner = pendingPushFailureBanner();
const j = await readStdinJson(io, 300);
if (!j) return { outcome: 'degraded', reason: 'no_stdin' };
@@ -890,19 +1098,27 @@ async function hookUserPrompt(io: HookIo): Promise<number> {
// [ENG-1] The 10000-char harness cap applies to the WHOLE stdout payload;
// the block is budgeted ≤8KB server-side, but JSON escaping inflates, so
// trim defensively rather than letting the harness divert-and-drop.
// trim defensively rather than letting the harness divert-and-drop. The
// banner (≤300 chars, fixed) rides inside the same payload [D5] — only
// blockText is trimmed, so the failure notice survives the cap loop.
const bannerPrefix = banner ? `${banner.text}\n\n` : '';
const buildPayload = (block: string) =>
JSON.stringify({
hookSpecificOutput: { hookEventName: 'UserPromptSubmit', additionalContext: bannerPrefix + block },
...(banner ? { systemMessage: banner.text } : {}),
});
let blockText = text;
let payload = JSON.stringify({
hookSpecificOutput: { hookEventName: 'UserPromptSubmit', additionalContext: blockText },
});
let payload = buildPayload(blockText);
while (payload.length > CLAUDE_HOOK_OUTPUT_CAP_CHARS && blockText.length > 0) {
blockText = blockText.slice(0, Math.max(0, blockText.length - (payload.length - CLAUDE_HOOK_OUTPUT_CAP_CHARS) - 16));
payload = JSON.stringify({
hookSpecificOutput: { hookEventName: 'UserPromptSubmit', additionalContext: blockText },
});
payload = buildPayload(blockText);
}
if (blockText.length === 0) return { outcome: 'degraded', reason: 'over_cap', turns: turns.length };
guardedWrite(payload + '\n');
if (!expired) {
wrotePayload = true;
banner?.record();
}
// Partial trim is delivery-count drift: the serve already logged the FULL
// post-budget set at the response write, but pages cut from the tail here
// were never injected. Record it so the doctor's heartbeat reconciliation
@@ -927,6 +1143,21 @@ async function hookUserPrompt(io: HookIo): Promise<number> {
expired = true;
result = { outcome: 'error', reason: errorCode(e) };
}
// Banner-only emission [D5]: every path that did NOT write the main payload
// (no_serve, ipc_unavailable, no_pglite_path, empty windows, transcript
// aborts, …) still surfaces the push failure — unless the deadline expired,
// in which case record() was never called and the banner re-fires next turn.
// (Local copy: TS cannot track the closure-side assignment of `banner`.)
const pendingBanner = banner as { text: string; record: () => void } | null;
if (pendingBanner && !wrotePayload && !expired) {
guardedWrite(
JSON.stringify({
hookSpecificOutput: { hookEventName: 'UserPromptSubmit', additionalContext: pendingBanner.text },
systemMessage: pendingBanner.text,
}) + '\n',
);
pendingBanner.record();
}
await writeHeartbeat({
ts: new Date().toISOString(),
event: 'user-prompt',
@@ -1031,8 +1262,9 @@ async function hookStop(io: HookIo): Promise<number> {
const t0 = Date.now();
let outcome: HookHeartbeatEntry['outcome'] = 'ok';
let reason: string | undefined;
let j: Record<string, unknown> | null = null;
try {
const j = await readStdinJson(io, 300);
j = await readStdinJson(io, 300);
const sessionId = sanitizeSessionId(j?.session_id);
const dir = await liveBufferDir();
const exchange = firstString(j, ['last_assistant_message', 'lastAssistantMessage', 'prompt']);
@@ -1047,11 +1279,21 @@ async function hookStop(io: HookIo): Promise<number> {
outcome = 'error';
reason = errorCode(e);
}
// Per-turn durability push [D3/D17/D20] — its own try/deadline so the
// buffer append above and the heartbeat below are never at risk.
let pushReason: string | undefined;
try {
const ws = io.cwd ?? (typeof j?.cwd === 'string' ? (j.cwd as string) : process.cwd());
const raced = await withDeadline(STOP_PUSH_DEADLINE_MS, stopPushIfDue(ws, io));
pushReason = raced === DEADLINE ? 'push_unavailable' : raced;
} catch {
pushReason = 'push_unavailable';
}
await writeHeartbeat({
ts: new Date().toISOString(),
event: 'stop',
outcome,
...(reason ? { reason } : {}),
...((reason ?? pushReason) ? { reason: reason ?? pushReason } : {}),
duration_ms: Date.now() - t0,
});
return 0;
src/commands/import.ts
+35 -7
View File
@@ -25,6 +25,34 @@ import {
resumeFilter,
} from '../core/import-checkpoint.ts';
/**
* Records one failed file against the run's error-grouping state and
* returns the running count for its group plus an unredacted sample
* message for display.
*
* `key` groups structurally-identical errors (e.g. the same failure
* across many files) so a single noisy failure mode doesn't produce
* thousands of near-duplicate warning lines — quoted substrings (typically
* a per-file slug or path) are blanked for the GROUPING key only. The
* printed `sample` is always a real, unredacted occurrence of the error
* (the first one seen for that key), so identifying details that are
* constant across the whole group — a Postgres table or constraint name,
* for instance — survive into what actually gets shown to the user.
* Pre-fix, the redacted key itself was printed, so e.g. a `pages_source_id_fkey`
* foreign-key violation surfaced as `table "" violates foreign key constraint ""`.
*/
export function recordImportFailure(
errorCounts: Record<string, number>,
errorSamples: Record<string, string>,
msg: string,
): { key: string; count: number; sample: string } {
const key = msg.replace(/"[^"]*"/g, '""');
const count = (errorCounts[key] ?? 0) + 1;
errorCounts[key] = count;
if (!(key in errorSamples)) errorSamples[key] = msg;
return { key, count, sample: errorSamples[key] };
}
function defaultWorkers(): number {
const cpuCount = cpus().length;
const memGB = totalmem() / (1024 ** 3);
@@ -288,6 +316,7 @@ export async function runImport(
let chunksCreated = 0;
const importedSlugs: string[] = [];
const errorCounts: Record<string, number> = {};
const errorSamples: Record<string, string> = {};
const failures: Array<{ path: string; error: string }> = []; // Bug 9
// #3839: paths that succeeded (imported OR unchanged) this run, keyed the
// same way as `failures` above (importRelPath) so a path that failed on a
@@ -351,12 +380,11 @@ export async function runImport(
}
} catch (e: unknown) {
const msg = e instanceof Error ? e.message : String(e);
const errorKey = msg.replace(/"[^"]*"/g, '""');
errorCounts[errorKey] = (errorCounts[errorKey] || 0) + 1;
if (errorCounts[errorKey] <= 5) {
const { count, sample } = recordImportFailure(errorCounts, errorSamples, msg);
if (count <= 5) {
console.error(` Warning: skipped ${relativePath}: ${msg}`);
} else if (errorCounts[errorKey] === 6) {
console.error(` (suppressing further "${errorKey.slice(0, 60)}..." errors)`);
} else if (count === 6) {
console.error(` (suppressing further "${sample.slice(0, 60)}..." errors)`);
}
errors++;
skipped++;
@@ -457,9 +485,9 @@ export async function runImport(
progress.finish();
// Error summary
for (const [err, count] of Object.entries(errorCounts)) {
for (const [key, count] of Object.entries(errorCounts)) {
if (count > 5) {
console.error(` ${count} files failed: ${err.slice(0, 100)}`);
console.error(` ${count} files failed: ${errorSamples[key].slice(0, 100)}`);
}
}
src/commands/jobs.ts
+38 -1
View File
@@ -1708,7 +1708,44 @@ export async function registerBuiltinHandlers(
});
worker.register('extract', async (job) => {
const { runExtractCore } = await import('./extract.ts');
const { runExtractCore, extractStaleFromDB, STALE_TIME_BUDGET_MS } = await import('./extract.ts');
// #2849: stale mode — the durable follow-up for extraction deferred by
// performSync's size gate (totalChanges > 100). Runs the same DB-source
// watermark sweep as `gbrain extract --stale`, scoped to the source the
// sync that deferred it was scoped to (job.data.sourceId; absent =
// unscoped, matching what the CLI hint tells a default-brain operator
// to run). The sweep is checkout-less + idempotent, so retries and
// overlapping submissions converge.
if (job.data.stale === true) {
const sourceIdFilter = typeof job.data.sourceId === 'string' ? job.data.sourceId : undefined;
const r = await extractStaleFromDB(engine, {
dryRun: !!job.data.dryRun,
jsonMode: false,
includeFrontmatter: false,
sourceIdFilter,
catchUp: false,
});
// Internal 30-min budget hit with work remaining → chain a
// continuation job so a very large deferred backlog converges without
// waiting for the next sync. Forward-progress guard (pagesProcessed >
// 0) prevents an infinite chain if the sweep can't advance.
if (!job.data.dryRun && r.staleRemaining > 0 && r.pagesProcessed > 0) {
try {
const queue = new MinionQueue(engine);
// NO maxWaiting: with an unscoped (NULL-sourceId) payload the
// coalesce filter matches ANY waiting 'extract' job and would
// swallow the continuation. Each completed sweep chains at most
// one continuation and the sweep is an idempotent watermark scan,
// so there is no pile-up to guard against.
await queue.add(
'extract',
{ ...job.data, continuation_of: job.id },
{ timeout_ms: STALE_TIME_BUDGET_MS + 5 * 60 * 1000 },
);
} catch { /* best-effort: next sync/manual sweep picks up the rest */ }
}
return { stale: true, source_id: sourceIdFilter ?? null, ...r };
}
const mode = (typeof job.data.mode === 'string' && ['links', 'timeline', 'all'].includes(job.data.mode))
? (job.data.mode as 'links' | 'timeline' | 'all')
: 'all';
src/commands/migrate-embeddings.ts
+14 -1
View File
@@ -389,7 +389,20 @@ export async function runMigrateEmbeddings(
exit(0);
} else {
if (flags.json) {
console.log(JSON.stringify({ status: 'incomplete', plan, embedded: embedResult.embedded, remaining }, null, 2));
console.log(JSON.stringify({
status: 'incomplete', plan, embedded: embedResult.embedded, remaining,
...(embedResult.lock_skipped && { lock_skipped: true }),
}, null, 2));
} else if (embedResult.lock_skipped) {
// E2E-observed failure mode: a hard-killed (SIGKILL/crash) migration
// leaves its single-flight embed lock behind, and every immediate
// re-run "resumes" without embedding anything. Say so — "re-run to
// resume" would be a lie until the lock expires.
const { EMBED_BACKFILL_LOCK_TTL_MIN } = await import('../core/embed-backfill-lock.ts');
serr(`Migration paused: ${remaining} chunk(s) still stale, and the re-embed was SKIPPED because`);
serr('another embed backfill holds the per-source lock. If that is a live run (check');
serr('`gbrain jobs list`), let it finish. If a previous migration was killed hard, its lock');
serr(`expires after at most ${EMBED_BACKFILL_LOCK_TTL_MIN} minutes — re-run the same command then.`);
} else {
serr(`Migration incomplete: ${remaining} chunk(s) still stale (embed failures or an interrupted run).`);
serr('Re-run the same command to resume — completed chunks are never re-embedded.');
src/commands/migrate-engine.ts
+24 -13
View File
@@ -80,6 +80,10 @@ export function manifestMatchesTarget(manifest: MigrateManifest, targetId: strin
return manifest.schema_version === 2 && manifest.target_id === targetId;
}
function makeManifestKey(sourceId: string, slug: string): string {
return sourceId === 'default' ? slug : `${sourceId}::${slug}`;
}
function loadManifest(): MigrateManifest | null {
const path = getManifestPath();
if (!existsSync(path)) return null;
@@ -151,6 +155,25 @@ export async function copyMigrationSources(source: BrainEngine, target: BrainEng
}
}
export async function copyPageLinksToTarget(
source: BrainEngine,
target: BrainEngine,
page: Page,
failedKeys: ReadonlySet<string> = new Set(),
): Promise<void> {
const links = await source.getLinks(page.slug, { sourceId: page.source_id });
for (const link of links) {
const toSourceId = link.to_source_id ?? page.source_id;
if (failedKeys.has(makeManifestKey(toSourceId, link.to_slug))) continue;
await target.addLink(
link.from_slug, link.to_slug,
link.context, link.link_type,
undefined, undefined, undefined,
{ fromSourceId: page.source_id, toSourceId },
);
}
}
/**
* postgres.js's UNDEFINED_VALUE guard rejects any bound parameter that is JS
* `undefined` — unlike PGLite, it will not silently treat it as SQL NULL.
@@ -564,8 +587,6 @@ export async function runMigrateEngine(sourceEngine: BrainEngine, args: string[]
// entries were bare slugs; we keep treating those as default-source for
// back-compat resume.
const completedSet = new Set(manifest?.completed_slugs || []);
const makeManifestKey = (sourceId: string, slug: string): string =>
sourceId === 'default' ? slug : `${sourceId}::${slug}`;
if (!manifest) {
manifest = {
completed_slugs: [],
@@ -680,17 +701,7 @@ export async function runMigrateEngine(sourceEngine: BrainEngine, args: string[]
progress.tick(1);
continue;
}
const sourceOpts = { sourceId: page.source_id };
const links = await sourceEngine.getLinks(page.slug, sourceOpts);
for (const link of links) {
if (failedKeys.has(makeManifestKey(page.source_id, link.to_slug))) continue;
await targetEngine.addLink(
link.from_slug, link.to_slug,
link.context, link.link_type,
undefined, undefined, undefined,
{ fromSourceId: page.source_id, toSourceId: page.source_id },
);
}
await copyPageLinksToTarget(sourceEngine, targetEngine, page, failedKeys);
progress.tick(1);
}
progress.finish();
src/commands/serve-http.ts
+50 -1
View File
@@ -27,7 +27,12 @@ import { OAuthTokenRevocationRequestSchema } from '@modelcontextprotocol/sdk/sha
import type { BrainEngine } from '../core/engine.ts';
import { operations, OperationError } from '../core/operations.ts';
import type { OperationContext, AuthInfo } from '../core/operations.ts';
import { GBrainOAuthProvider, validateTokenEndpointAuthMethod } from '../core/oauth-provider.ts';
import {
GBrainOAuthProvider,
validateTokenEndpointAuthMethod,
dcrRegistrationContext,
DEFAULT_DCR_TTL_MIN_SECONDS,
} from '../core/oauth-provider.ts';
import type { SqlQuery } from '../core/oauth-provider.ts';
import { hasScope, ALLOWED_SCOPES_LIST, normalizeScopesInput } from '../core/scope.ts';
import { normalizeSourceInput, normalizeFederatedReadInput } from '../core/source-id.ts';
@@ -702,11 +707,41 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption
// constructor option instead of monkey-patching `_clientsStore` after
// construction. Same outcome (no /register endpoint when --enable-dcr
// is not passed); cleaner shape for tests and future maintainers.
// #2179: admin-configured clamp window for DCR-requested token TTLs.
// DB-plane config keys (`gbrain config set oauth.dcr_ttl_min_seconds ...`).
// FAIL-CLOSED defaults: an unset/invalid max is bounded by the operator's
// own --token-ttl (never a fixed permissive ceiling), and an inverted
// window collapses to the min bound — the same direction clampDcrTokenTtl
// itself resolves. A bad config narrows the window; it never widens it.
const parseDcrTtlBound = (raw: unknown, fallback: number): number => {
const n = Number(raw);
return raw != null && Number.isFinite(n) && n >= 1 ? Math.floor(n) : fallback;
};
let dcrTtlMinSeconds = DEFAULT_DCR_TTL_MIN_SECONDS;
let dcrTtlMaxSeconds = Math.max(tokenTtl, dcrTtlMinSeconds);
try {
dcrTtlMinSeconds = parseDcrTtlBound(await engine.getConfig('oauth.dcr_ttl_min_seconds'), DEFAULT_DCR_TTL_MIN_SECONDS);
dcrTtlMaxSeconds = parseDcrTtlBound(await engine.getConfig('oauth.dcr_ttl_max_seconds'), Math.max(tokenTtl, dcrTtlMinSeconds));
} catch {
// Config read is best-effort; the fail-closed defaults stand.
dcrTtlMaxSeconds = Math.max(tokenTtl, dcrTtlMinSeconds);
}
if (dcrTtlMinSeconds > dcrTtlMaxSeconds) {
console.error(
`[serve-http] WARNING: oauth.dcr_ttl_min_seconds (${dcrTtlMinSeconds}) exceeds ` +
`oauth.dcr_ttl_max_seconds (${dcrTtlMaxSeconds}); collapsing the window to ` +
`the min bound (${dcrTtlMinSeconds}).`,
);
dcrTtlMaxSeconds = dcrTtlMinSeconds;
}
const oauthProvider = new GBrainOAuthProvider({
sql,
tokenTtl,
dcrDisabled: !enableDcr,
allowClientCredentialsDcr: enableDcrInsecure === true,
dcrTtlMinSeconds,
dcrTtlMaxSeconds,
});
// #1353: loud stderr security WARN when DCR is enabled. DCR is an
@@ -820,6 +855,20 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption
app.use('/register', cors(corsOAuthOptions));
app.use('/revoke', cors(corsOAuthOptions));
// #2179: capture the optional `token_ttl_seconds` DCR extension field
// BEFORE the SDK's /register handler runs — its request schema strips
// unknown body members, so the value would never reach registerClient.
// The rest of the chain runs inside dcrRegistrationContext; the clients
// store clamps + persists it. Malformed values are ignored (fail-safe:
// absent → server default; out-of-range → clamped downstream; a TTL hint
// never rejects a registration). express.json() here is idempotent with
// the SDK router's own body parser.
app.use('/register', express.json(), (req: Request, _res: Response, next: NextFunction) => {
const raw = (req.body as Record<string, unknown> | null | undefined)?.token_ttl_seconds;
const tokenTtlSeconds = typeof raw === 'number' && Number.isFinite(raw) ? raw : undefined;
dcrRegistrationContext.run({ tokenTtlSeconds }, next);
});
// ---------------------------------------------------------------------------
// Custom client_credentials handler (before mcpAuthRouter)
// SDK's token handler only supports authorization_code and refresh_token
src/commands/sources.ts
+21 -6
View File
@@ -57,6 +57,7 @@ import {
parseSourceConfig,
normalizeSourceConfig,
isSourceFederated,
sourceFederationState,
type SourceRow as LoadedSourceRow,
} from '../core/sources-load.ts';
@@ -470,8 +471,14 @@ async function runList(engine: BrainEngine, args: string[]): Promise<void> {
// Human-readable table.
console.log('SOURCES');
console.log('───────');
for (const e of entries) {
const fedMark = e.federated ? 'federated' : (e as any).archived ? '⚠ archived' : 'isolated';
for (let i = 0; i < entries.length; i++) {
const e = entries[i];
// Explicit `federated: false` (`sources unfederate`) fully isolates a
// source's reads in both directions; an absent key ('unset') only keeps
// it out of OTHER anchors' reads — its own unqualified reads still widen
// outward (see sourceFederationState). Collapsing both to "isolated"
// overstates what an unset flag does.
const fedMark = (e as any).archived ? '⚠ archived' : sourceFederationState(rows[i].config);
const pathStr = e.local_path ?? '(no local path)';
const sync = e.last_sync_at ? `last sync ${e.last_sync_at}` : 'never synced';
console.log(` ${e.id.padEnd(20)} ${fedMark.padEnd(12)} ${String(e.page_count).padStart(6)} pages ${sync}`);
@@ -1556,10 +1563,18 @@ Subcommands:
per finding via .gbrain-scan-allow) and on
tracked deny-list files (*.pglite, .env*,
*.pem, *.key, .gbrain/**). Refuses remotes not
verifiably private via gh; single-flight (a
concurrent push exits 0 as "skipped"); pushes
even on a clean tree. Writes
~/.gbrain/bootstrap/push-status.json.
verifiably private — verified via REST, falling
back to pure git protocol where gh is blocked
(cloud proxies); private verdicts cached 1h.
Unverified-remote overrides (self-hosted git
you trust; every use warns loudly): the flag
above, GBRAIN_ALLOW_UNVERIFIED_REMOTE=1, or
"gbrain config set push.allow_unverified_remote
true" (file-plane — reaches detached hook
children). Single-flight (a concurrent push
exits 0 as "skipped"); pushes even on a clean
tree. Writes per-root status under
~/.gbrain/bootstrap/.
unharden <id> Remove durability cron/hook/credential wiring.
Source id: [a-z0-9-]{1,32}. Immutable citation key.
src/commands/status.ts
+12 -14
View File
@@ -15,7 +15,7 @@
* - Workers — supervisor health from the audit JSONL
* - Queue — live minion_jobs counts BY status (NO time window —
* old stuck jobs are exactly what status surfaces)
* - Autopilot — daemon PID liveness via kill -0 probe
* - Autopilot — daemon PID liveness plus gbrain-autopilot identity probe
*
* Exit codes (kubectl-style):
* 0 snapshot produced successfully (even if it carries warnings)
@@ -40,6 +40,10 @@ import { existsSync, readFileSync } from 'node:fs';
import { gbrainPath, loadConfig, isThinClient } from '../core/config.ts';
import { callRemoteTool, unpackToolResult } from '../core/mcp-client.ts';
import { VERSION } from '../version.ts';
import {
classifyAutopilotLockHolder,
type AutopilotLockProbeDeps,
} from '../core/autopilot-lock.ts';
import {
buildSyncStatusReport,
type SyncStatusReport,
@@ -297,8 +301,10 @@ function buildWorkerSummary(): WorkerSummary {
return { crashes_24h, clean_exits_24h, by_cause, last_event_ts };
}
function buildAutopilotStatus(): AutopilotStatus {
const lockPath = gbrainPath('autopilot.lock');
export function buildAutopilotStatus(
lockPath: string = gbrainPath('autopilot.lock'),
deps: AutopilotLockProbeDeps = {},
): AutopilotStatus {
const lockfile_present = existsSync(lockPath);
let pid: number | null = null;
let running = false;
@@ -308,16 +314,8 @@ function buildAutopilotStatus(): AutopilotStatus {
const parsed = parseInt(raw, 10);
if (Number.isFinite(parsed) && parsed > 0) {
pid = parsed;
try {
// kill -0 probes liveness without sending a real signal. Throws ESRCH
// if the PID is gone, EPERM if alive but owned by another user (which
// still tells us "something with that PID exists").
process.kill(parsed, 0);
running = true;
} catch (err) {
const code = (err as NodeJS.ErrnoException).code;
running = code === 'EPERM';
}
const holder = classifyAutopilotLockHolder(parsed, process.pid, deps);
running = holder.state === 'alive-autopilot' || holder.state === 'alive-unknown';
}
} catch {
/* unreadable lockfile, leave pid=null/running=false */
@@ -592,7 +590,7 @@ function renderHuman(report: StatusReport): string {
if (a.running) {
lines.push(` running (PID ${a.pid})`);
} else if (a.lockfile_present) {
lines.push(` stale lockfile (PID ${a.pid ?? '?'} not alive). Run \`gbrain autopilot --install\` to restart.`);
lines.push(` stale lockfile (PID ${a.pid ?? '?'} is not a live autopilot process). Run \`gbrain autopilot --install\` to restart.`);
} else {
lines.push(' not running. Install with `gbrain autopilot --install`.');
}
src/commands/sync.ts
+61 -3
View File
@@ -3647,10 +3647,68 @@ async function performSyncInner(engine: BrainEngine, opts: SyncOpts): Promise<Sy
// covered regardless.
const extractOpts = opts.sourceId ? { sourceId: opts.sourceId } : undefined;
if (!opts.noExtract && totalChanges > 100 && pagesAffected.length > 0) {
// #2849: above the size gate the deferred extraction must be DURABLY
// QUEUED, not just hinted. The autopilot cycle's extract phase is
// slug-scoped (an up_to_date follow-up sync hands it an empty
// pagesAffected), so a webhook-driven large sync left
// `links_extracted_at` unstamped FOREVER unless an operator ran
// `gbrain extract --stale` by hand. Submit a source-scoped stale-sweep
// job bound to the consumed commit (idempotency key) so repeated
// webhook deliveries / sync retries of the same commit coalesce onto
// one job. The sweep itself is the watermark scan — it picks up the
// pages this run imported AND any banked across resumed runs.
// Best-effort: queue submission failure falls back to the hint-only
// behavior (the pages stay stale + visible to doctor, never mis-stamped).
let queuedJobId: number | string | null = null;
try {
const { MinionQueue } = await import('../core/minions/queue.ts');
const { STALE_TIME_BUDGET_MS } = await import('./extract.ts');
const queue = new MinionQueue(engine);
const payload = {
stale: true,
...(opts.sourceId ? { sourceId: opts.sourceId } : {}),
reason: 'sync_size_gate',
// Bound to the PIN this run drained to (== headCommit unless resuming
// a stored target), not live HEAD — the sweep covers what we imported.
deferred_commit: pin,
};
// The stale sweep has its own internal wall-clock budget
// (GBRAIN_EXTRACT_TIME_BUDGET_MS-derived); without an explicit
// timeout_ms the job would inherit the tight null-default and get
// wall-clock-killed mid-sweep (#1737 class). 5-min headroom.
const timeoutMs = STALE_TIME_BUDGET_MS + 5 * 60 * 1000;
// NO maxWaiting here: with an unscoped (NULL-sourceId) payload the
// queue's coalesce filter matches ANY waiting 'extract' job (e.g. a
// remediation-submitted {mode:'links'} row) and returns THAT job —
// silently dropping the sweep while we log "queued". The idempotency
// key alone is the dedup for repeat submissions toward the same pin.
const key = `extract-stale:${opts.sourceId ?? 'default'}:${pin}`;
const isLiveSweep = (j: { status: string; data: Record<string, unknown> }): boolean =>
j.data?.stale === true && ['waiting', 'delayed', 'active'].includes(j.status);
let job = await queue.add('extract', payload, { idempotency_key: key, timeout_ms: timeoutMs });
if (!isLiveSweep(job)) {
// The key slot holds a FINISHED row: a prior sweep toward this pin
// that completed BEFORE this run's pages landed (checkpoint-resume /
// blocked-advance re-sync of the same target). Those pages went
// stale after that sweep's watermark pass, so coalescing onto the
// finished row would strand them — queue a fresh sweep under a
// run-unique key. (An 'active' sweep is safe to coalesce onto: its
// end-of-run staleRemaining re-count chains a continuation.)
job = await queue.add('extract', payload, {
idempotency_key: `${key}:${Date.now()}`,
timeout_ms: timeoutMs,
});
}
// Only claim "queued" once we verified the returned row IS a live
// stale sweep — never trust queue.add's row blind.
if (isLiveSweep(job)) queuedJobId = job.id;
} catch { /* best-effort — hint below still tells the operator */ }
slog(
` Large sync: deferring link/timeline extraction. ` +
`Run 'gbrain extract --stale${opts.sourceId ? ` --source-id ${opts.sourceId}` : ''}' ` +
`(or let the autopilot cycle's extract phase sweep it).`,
` Large sync: deferring link/timeline extraction` +
(queuedJobId != null
? ` — queued stale-sweep job #${queuedJobId} (source: ${opts.sourceId ?? 'default'}); a running jobs worker will consume it.`
: `.`) +
` Run 'gbrain extract --stale${opts.sourceId ? ` --source-id ${opts.sourceId}` : ''}' to extract now.`,
);
}
if (!opts.noExtract && totalChanges <= 100 && pagesAffected.length > 0) {
src/commands/takes.ts
+11 -2
View File
@@ -624,12 +624,13 @@ async function cmdExtract(engine: BrainEngine, rest: string[]): Promise<void> {
const sub = rest[0];
if (sub !== '--from-pages') {
process.stderr.write(
'Usage: gbrain takes extract --from-pages [--yes] [--dry-run] [--source-id <id>] [--max-pages N (clamped to 1000)] [--include-covered] [--holder <name>]\n' +
'Usage: gbrain takes extract --from-pages [--yes] [--dry-run] [--json] [--source-id <id>] [--max-pages N (clamped to 1000)] [--include-covered] [--holder <name>]\n' +
'Runs progress: pages that already hold takes are skipped, so repeat runs sweep a large corpus in slices. --include-covered rescans everything (refresh).\n',
);
process.exit(1);
}
const dryRun = rest.includes('--dry-run');
const json = rest.includes('--json');
const skipConfirm = rest.includes('--yes');
const sourceIdx = rest.indexOf('--source-id');
const sourceIdFilter = sourceIdx >= 0 ? rest[sourceIdx + 1] : undefined;
@@ -667,9 +668,17 @@ async function cmdExtract(engine: BrainEngine, rest: string[]): Promise<void> {
holder,
});
if (result.llm_unavailable) {
process.stderr.write(`[takes extract] chat gateway unavailable (no API key configured).\n`);
if (json) {
process.stdout.write(`${JSON.stringify(result, null, 2)}\n`);
} else {
process.stderr.write(`[takes extract] chat gateway unavailable (no API key configured).\n`);
}
process.exit(2);
}
if (json) {
process.stdout.write(`${JSON.stringify(result, null, 2)}\n`);
return;
}
process.stdout.write(
`takes extract --from-pages: ${result.claims_extracted} claim(s) from ${result.pages_scanned} page(s)` +
(dryRun ? ' (dry-run)' : '') + '\n',
src/commands/upgrade.ts
+42 -11
View File
@@ -473,19 +473,39 @@ export async function runPostUpgrade(args: string[] = []): Promise<void> {
// `ze_sunset_notice_shown` (same pattern as the search-mode banner).
try {
const shown = await engine.getConfig('ze_sunset_notice_shown');
const { DEFAULT_EMBEDDING_MODEL } = await import('../core/ai/defaults.ts');
const { DEFAULT_EMBEDDING_MODEL, ZEROENTROPY_SUNSET_DATE } = await import('../core/ai/defaults.ts');
const effectiveModel = cfgSchema.embedding_model ?? DEFAULT_EMBEDDING_MODEL;
const rerankerModel = await engine.getConfig('search.reranker.model');
// Effective reranker via the plane search actually reranks with
// (mode bundle + search.reranker.* overrides) — the bare config
// key is unset by default while balanced/tokenmax rerank with the
// bundle's zeroentropyai model. Same resolution as the
// provider_sunset doctor check.
let rerankerModel: string | undefined;
try {
const { loadSearchModeConfig, resolveSearchMode } = await import('../core/search/mode.ts');
const knobs = resolveSearchMode(await loadSearchModeConfig(engine));
if (knobs.reranker_enabled) rerankerModel = knobs.reranker_model;
} catch { /* no reranker-exposure claim */ }
const onZeEmbedding = effectiveModel.startsWith('zeroentropyai:');
const onZeReranker = !!rerankerModel?.startsWith('zeroentropyai:');
if (shown !== 'true' && (onZeEmbedding || onZeReranker)) {
// Paste-ready --dim from the ACTUAL column width (config can
// drift): keeping the current width avoids a needless dimension
// transition + index rebuild when the target supports it.
let colDims: number | null = null;
try {
const { readContentChunksEmbeddingDim } = await import('../core/embedding-dim-check.ts');
colDims = (await readContentChunksEmbeddingDim(engine)).dims;
} catch { /* fresh brain — omit --dim */ }
const dimFlag = colDims ? ` --dim ${colDims}` : '';
console.log('');
console.log('═══════════════════════════════════════════════════════════════');
console.log('[gbrain] ACTION REQUIRED: ZeroEntropy hosted API sunsets 2026-09-04.');
console.log(`[gbrain] ACTION REQUIRED: ZeroEntropy hosted API sunsets ${ZEROENTROPY_SUNSET_DATE}.`);
if (onZeEmbedding) {
console.log(`[gbrain] This brain embeds with ${effectiveModel}. After the sunset,`);
console.log('[gbrain] semantic retrieval STOPS WORKING (queries can no longer be');
console.log('[gbrain] embedded against your existing vectors).');
console.log('[gbrain] semantic retrieval STOPS WORKING entirely — your EXISTING');
console.log('[gbrain] vectors become unqueryable (queries embed through the same');
console.log('[gbrain] endpoint), not just new content.');
}
if (onZeReranker) {
console.log(`[gbrain] The reranker (${rerankerModel}) also sunsets; search falls`);
@@ -493,17 +513,28 @@ export async function runPostUpgrade(args: string[] = []): Promise<void> {
}
console.log('═══════════════════════════════════════════════════════════════');
console.log('');
console.log('Migrate before the sunset (resumable; preview cost first):');
console.log(' gbrain migrate embeddings --to <provider:model> --dry-run');
console.log(' gbrain migrate embeddings --to <provider:model>');
console.log('Two fixes, either works:');
console.log('');
console.log('Self-hosting zembed-1 (weights are Apache-2.0) via llama-server /');
console.log('ollama also works and preserves your existing vectors — point');
console.log('embedding at the local endpoint instead of migrating.');
console.log('[1] Self-host the same model — zembed-1 weights are Apache-2.0. Serve');
console.log(' them via llama-server or Ollama and point the config at the local');
console.log(' endpoint. Keeps every existing vector; NO re-embed at all. See');
console.log(' docs/guides/embedding-migration.md ("Self-hosting instead of migrating").');
console.log('');
console.log('[2] Migrate to another provider (resumable; preview cost first):');
console.log(` gbrain migrate embeddings --to <provider:model>${dimFlag} --dry-run`);
console.log(` gbrain migrate embeddings --to <provider:model>${dimFlag}`);
if (colDims) {
console.log(` (--dim ${colDims} is this brain's current index width — keep it to`);
console.log(' avoid a needless schema rebuild when the target supports it.)');
}
if (onZeReranker) {
console.log('');
console.log('Reranker: gbrain config set search.reranker.enabled false (or pick another).');
}
console.log('');
console.log(`\`gbrain doctor\` will keep flagging this until the brain is off the`);
console.log('provider (check name: provider_sunset).');
console.log('');
await engine.setConfig('ze_sunset_notice_shown', 'true');
}
} catch {
src/core/ai/defaults.ts
+11
View File
@@ -19,3 +19,14 @@
// steps: {2560, 1280, 640, 320, 160, 80, 40} — see ai/dims.ts.
export const DEFAULT_EMBEDDING_MODEL = 'zeroentropyai:zembed-1';
export const DEFAULT_EMBEDDING_DIMENSIONS = 1280;
/**
* ZeroEntropy announced (2026-07-24) that its hosted API — including
* /models/embed and /models/rerank — shuts down on this date. Query
* embedding uses the same endpoint as ingestion, so a brain still on a
* `zeroentropyai:*` embedding model loses semantic retrieval ENTIRELY on
* that date (existing vectors become unqueryable, not just new content).
* Single source of truth for the upgrade banner + the `provider_sunset`
* doctor check. Self-hosting the Apache-2.0 zembed-1 weights is unaffected.
*/
export const ZEROENTROPY_SUNSET_DATE = '2026-09-04';
src/core/ai/recipes/claude-cli.ts
+4
View File
@@ -35,7 +35,11 @@ export const claudeCli: Recipe = {
// No embedding or expansion touchpoints — chat-only.
chat: {
models: [
'claude-fable-5',
'claude-opus-5',
'claude-opus-4-8',
'claude-opus-4-7',
'claude-sonnet-5',
'claude-sonnet-4-6',
'claude-haiku-4-5-20251001',
],
src/core/autopilot-lock.ts
+64
View File
@@ -0,0 +1,64 @@
import { execFileSync } from 'node:child_process';
export type AutopilotLockHolder =
| { state: 'dead' }
| { state: 'self' }
| { state: 'alive-autopilot' }
| { state: 'alive-foreign' }
| { state: 'alive-unknown' };
export interface AutopilotLockProbeDeps {
isPidAlive?: (pid: number) => boolean;
readProcessCommand?: (pid: number) => string | null;
}
export function isPidAlive(pid: number): boolean {
if (!Number.isFinite(pid) || pid <= 0) return false;
try {
process.kill(pid, 0);
return true;
} catch (error: unknown) {
return (error as NodeJS.ErrnoException).code === 'EPERM';
}
}
export function readProcessCommand(pid: number): string | null {
if (!Number.isFinite(pid) || pid <= 0) return null;
try {
const out = execFileSync('ps', ['-p', String(pid), '-o', 'args='], {
encoding: 'utf8',
stdio: ['ignore', 'pipe', 'ignore'],
timeout: 1000,
}).trim();
return out.length > 0 ? out : null;
} catch {
return null;
}
}
export function looksLikeGbrainAutopilotCommand(command: string): boolean {
const normalized = command.replace(/\\/g, '/').trim();
if (!/(^|\s)autopilot(\s|$)/i.test(normalized)) return false;
if (/(^|[\/\s])gbrain(?:\.exe)?(\s|$)/i.test(normalized)) return true;
return /(^|\s)(?:\S+\/)?(?:\.{1,2}\/)?(?:src\/)?cli\.(?:ts|js|mjs)(\s|$)/i.test(normalized)
|| /(^|\s)\S*\/src\/cli\.(?:ts|js|mjs)(\s|$)/i.test(normalized);
}
export function classifyAutopilotLockHolder(
pid: number,
currentPid: number = process.pid,
deps: AutopilotLockProbeDeps = {},
): AutopilotLockHolder {
if (!Number.isFinite(pid) || pid <= 0) return { state: 'dead' };
if (pid === currentPid) return { state: 'self' };
const probeAlive = deps.isPidAlive ?? isPidAlive;
if (!probeAlive(pid)) return { state: 'dead' };
const probeCommand = deps.readProcessCommand ?? readProcessCommand;
const command = probeCommand(pid);
if (command === null) return { state: 'alive-unknown' };
return looksLikeGbrainAutopilotCommand(command)
? { state: 'alive-autopilot' }
: { state: 'alive-foreign' };
}
src/core/bootstrap/assets.ts
+10
View File
@@ -36,6 +36,8 @@ import T_ACCESS from '../../../templates/bootstrap/ACCESS_POLICY.md.template' wi
import T_MEMORY_README from '../../../templates/bootstrap/memory-README.md.template' with { type: 'file' };
// @ts-ignore
import T_GITIGNORE from '../../../templates/bootstrap/gitignore.template' with { type: 'file' };
// @ts-ignore
import T_CLOUD_SETUP from '../../../templates/bootstrap/cloud-setup-script.sh' with { type: 'file' };
/** Where each rendered file lands, relative to the workspace root. */
export interface BootstrapTemplate {
@@ -47,6 +49,14 @@ export interface BootstrapTemplate {
group: 'identity' | 'contract' | 'scaffold';
}
/** The cloud environment setup script [D16] — NOT a rendered template (it is
* pasted into the cloud env config, never written into the workspace). Lives
* as a template file deliberately: inline script strings in src modules would
* bleed their dashed tokens into the CLI flag registry. */
export function loadCloudSetupScript(): string {
return readFileSync(T_CLOUD_SETUP as unknown as string, 'utf8');
}
export const BOOTSTRAP_TEMPLATES: BootstrapTemplate[] = [
{ assetPath: T_AGENTS as unknown as string, dest: 'AGENTS.md', group: 'contract' },
{ assetPath: T_CLAUDE as unknown as string, dest: 'CLAUDE.md', group: 'contract' },
src/core/bootstrap/attach.ts
+20 -1
View File
@@ -36,6 +36,8 @@ import {
type AgentManifest,
type InstallReceipt,
} from './format.ts';
import { gitOriginUrl } from './status.ts';
import { type RepoReceipt } from './repo.ts';
import { BootstrapError } from './lock.ts';
export type AttachStepKind = 'register_source' | 'hooks_repair' | 'mcp_add' | 'verify';
@@ -99,7 +101,23 @@ export function attachWorkspace(workspaceDir: string, opts: AttachWorkspaceOptio
const existing = readReceipt(gbrainHomeDir);
const sameWorkspace = existing !== null && realpathOrResolve(existing.workspace_dir) === resolvedWs;
const receipt: InstallReceipt = {
// Record repo_url from the adopted origin so the no-daemon push gate
// (repoPhaseComplete) recognizes the repo phase as done on this machine —
// otherwise the per-turn/session-end pushes defer FOREVER after an attach
// (the ONLY install path in a cloud sandbox, where `bootstrap repo` is
// refused). This does NOT bypass the privacy gate: workspacePush still runs
// the visibility ladder at push time and fails closed on a public/
// unverifiable origin. Preserve an existing repo_url on a same-workspace
// re-attach; else adopt the current github origin.
const existingRepoUrl = sameWorkspace ? (existing as RepoReceipt).repo_url : undefined;
const originUrl = gitOriginUrl(resolvedWs);
// Record the origin as repo_url whenever one exists — github or self-hosted.
// repoPhaseComplete binds by owner/name for github and by exact URL otherwise;
// either way workspacePush still runs the visibility ladder at push time, so a
// non-verifiable origin stays fail-closed (refused unless the operator sets the
// escape hatch). Recording it only clears the "repo phase ran" gate.
const adoptedRepoUrl = existingRepoUrl ?? (originUrl ?? undefined);
const receipt: RepoReceipt = {
receipt_version: 1,
workspace_dir: resolvedWs,
source_id: manifest.source_id,
@@ -111,6 +129,7 @@ export function attachWorkspace(workspaceDir: string, opts: AttachWorkspaceOptio
brain_created_by_bootstrap: sameWorkspace ? existing.brain_created_by_bootstrap : false,
created_paths: sameWorkspace ? existing.created_paths : [],
registrations: sameWorkspace ? existing.registrations : [],
...(adoptedRepoUrl ? { repo_url: adoptedRepoUrl } : {}),
};
// writeReceipt assumes the bootstrap/ subdir exists; attach runs on a fresh
// machine where nothing has created it yet.
src/core/bootstrap/hooks.ts
+181 -2
View File
@@ -33,6 +33,7 @@ import {
} from 'node:fs';
import { dirname, isAbsolute, join } from 'node:path';
import {
CLAUDE_COMMITTED_SETTINGS_FILE_RELPATH,
CLAUDE_HOOK_DEFAULT_TIMEOUT_SECS,
CLAUDE_HOOK_EVENTS,
CLAUDE_HOOK_SUBCOMMAND,
@@ -125,6 +126,74 @@ export function buildClaudeHookCommand(
return parts.map(shellQuote).join(' ');
}
export function claudeCommittedSettingsPath(workspaceDir: string): string {
return join(workspaceDir, CLAUDE_COMMITTED_SETTINGS_FILE_RELPATH);
}
/**
* The COMMITTED carrier's command [D12]: PATH-resolved and fail-open. No
* absolute binary path — the committed file travels between machines and
* cloud sessions; wherever gbrain is not installed the hook exits 0 silently
* instead of erroring every turn.
*/
export function buildPortableClaudeHookCommand(event: ClaudeHookEvent, env: ClaudeHookEnv): string {
const assignments: string[] = [`GBRAIN_SOURCE=${env.GBRAIN_SOURCE}`];
if (env.GBRAIN_HOME) assignments.push(`GBRAIN_HOME=${env.GBRAIN_HOME}`);
const invoke = ['env', ...assignments, 'gbrain', 'hook', CLAUDE_HOOK_SUBCOMMAND[event]]
.map(shellQuote)
.join(' ');
return `command -v gbrain >/dev/null 2>&1 && ${invoke} || exit 0`;
}
/** Events the COMMITTED settings file already carries with our marker [D12]
* — the local writer skips these so one event never fires from both files. */
/** Pull the GBRAIN_SOURCE value out of a rendered portable hook command so the
* exact-match check is agnostic to the (operator-chosen) source id. Returns
* null when the command isn't shaped like ours. */
function extractHookSource(command: string): string | null {
const m = /command -v gbrain >\/dev\/null 2>&1 && env GBRAIN_SOURCE=('[^']*'|[^ ]+) gbrain hook /.exec(command);
if (!m) return null;
const raw = m[1]!;
return raw.startsWith("'") ? raw.slice(1, -1).replace(/'\\''/g, "'") : raw;
}
export function committedHookEvents(workspaceDir: string): Set<ClaudeHookEvent> {
const carried = new Set<ClaudeHookEvent>();
try {
const raw = readFileSync(claudeCommittedSettingsPath(workspaceDir), 'utf8');
const parsed = JSON.parse(raw) as { hooks?: Record<string, unknown> };
const hooks = parsed?.hooks;
if (typeof hooks !== 'object' || hooks === null) return carried;
for (const event of CLAUDE_HOOK_EVENTS) {
const groups = (hooks as Record<string, unknown>)[event];
if (!Array.isArray(groups)) continue;
const ours = groups.some(
(g) =>
typeof g === 'object' && g !== null &&
Array.isArray((g as HookMatcherGroup).hooks) &&
((g as HookMatcherGroup).hooks as unknown[]).some(
(h) =>
isOurs(h) &&
// A committed file is repo-contributor-writable: a marker + a
// bare `includes('gbrain hook')` substring is spoofable
// (`evil; # gbrain hook` suppresses the real local install AND
// runs attacker code). Require the EXACT portable-command shape
// this event would render — the anchored `command -v gbrain …`
// guard + `|| exit 0` structure a foreign command can't fake.
typeof (h as HookCommandEntry).command === 'string' &&
(h as HookCommandEntry).command === buildPortableClaudeHookCommand(event, {
GBRAIN_SOURCE: extractHookSource((h as HookCommandEntry).command as string) ?? '',
}),
),
);
if (ours) carried.add(event);
}
} catch {
/* absent/corrupt committed file → nothing carried */
}
return carried;
}
function isOurs(entry: unknown): boolean {
return (
typeof entry === 'object' &&
@@ -253,6 +322,11 @@ export function writeClaudeHooks(
}
const events = opts.events ?? [...CLAUDE_HOOK_EVENTS];
// [D12] Dedupe invariant: an event carried by the COMMITTED settings file
// never also fires from the local file. The local writer still strips its
// own prior entries for carried events (removing stale local copies), but
// re-adds nothing for them.
const carried = committedHookEvents(workspaceDir);
let removedPrior = 0;
const installed: Array<{ event: ClaudeHookEvent; command: string }> = [];
@@ -269,6 +343,13 @@ export function writeClaudeHooks(
const { kept, removed } = stripOurEntries(groups as unknown[]);
removedPrior += removed;
if (carried.has(event)) {
notes.push(`${event}: carried by the committed .claude/settings.json — local entry skipped [D12]`);
if (kept.length === 0) delete hooks[event];
else hooks[event] = kept;
continue;
}
const command = buildClaudeHookCommand(opts.gbrainBin, event, opts.env);
const timeout = opts.timeoutSecs?.[event] ?? CLAUDE_HOOK_DEFAULT_TIMEOUT_SECS[event];
const entry: HookCommandEntry = {
@@ -294,14 +375,112 @@ export function writeClaudeHooks(
return { settingsPath, installed, removedPrior, backupPath, brokenBackupPath, notes };
}
/**
* Write hooks into the COMMITTED `.claude/settings.json` [D12] — the only
* carrier that survives into fresh cloud clones (hooks are snapshotted at
* session start; the gitignored local file never exists there). Commands are
* PATH-resolved and fail-open (buildPortableClaudeHookCommand). After the
* committed write, the same events are stripped from the LOCAL file so an
* event never fires from both carriers.
*/
export function writeCommittedClaudeHooks(
workspaceDir: string,
opts: { env: ClaudeHookEnv; events?: ClaudeHookEvent[]; timeoutSecs?: Partial<Record<ClaudeHookEvent, number>> },
): WriteClaudeHooksResult {
if (opts.env.GBRAIN_HOME) {
throw new Error(
'GBRAIN_HOME is machine-specific and must not be embedded in the COMMITTED hook carrier ' +
'(the file travels between machines) — isolated installs stay on the local carrier',
);
}
for (const [k, v] of Object.entries(opts.env)) {
if (typeof v === 'string' && /[\n\r\0]/.test(v)) {
throw new Error(`env ${k} contains control characters — refusing to embed in a hook command`);
}
}
const settingsPath = claudeCommittedSettingsPath(workspaceDir);
const { settings, existed, brokenBackupPath, notes } = loadSettings(settingsPath);
let hooks = settings.hooks as Record<string, unknown> | undefined;
if (typeof hooks !== 'object' || hooks === null || Array.isArray(hooks)) {
if (hooks !== undefined) {
notes.push(
`WARNING: existing "hooks" key was not an object (${JSON.stringify(hooks).slice(0, 80)}); ` +
`replaced — the original file is in the .bak backup.`,
);
}
hooks = {};
}
const events = opts.events ?? [...CLAUDE_HOOK_EVENTS];
let removedPrior = 0;
const installed: Array<{ event: ClaudeHookEvent; command: string }> = [];
for (const event of events) {
let groups = hooks[event];
if (!Array.isArray(groups)) {
if (groups !== undefined) {
notes.push(`WARNING: existing hooks.${event} was not an array; replaced — original in the .bak backup.`);
}
groups = [];
}
const { kept, removed } = stripOurEntries(groups as unknown[]);
removedPrior += removed;
const command = buildPortableClaudeHookCommand(event, opts.env);
const timeout = opts.timeoutSecs?.[event] ?? CLAUDE_HOOK_DEFAULT_TIMEOUT_SECS[event];
const entry: HookCommandEntry = {
type: 'command',
command,
timeout,
[GBRAIN_HOOK_MARKER_KEY]: GBRAIN_HOOK_MARKER_VALUE,
};
kept.push({ hooks: [entry] });
hooks[event] = kept;
installed.push({ event, command });
}
settings.hooks = hooks;
let backupPath: string | null = null;
if (existed && brokenBackupPath === null) {
backupPath = `${settingsPath}.bak`;
copyFileSync(settingsPath, backupPath);
}
atomicWriteJson(settingsPath, settings);
// [D12] dedupe: the committed carrier now owns these events — remove any
// local copies so nothing double-fires on this machine.
const localCleanup = removeHooksFromFile(claudeSettingsPath(workspaceDir));
if (localCleanup.removed > 0) {
notes.push(
`removed ${localCleanup.removed} local settings.local.json entr${localCleanup.removed === 1 ? 'y' : 'ies'} — the committed carrier owns the events now [D12]`,
);
}
return { settingsPath, installed, removedPrior, backupPath, brokenBackupPath, notes };
}
/**
* Remove ONLY marker-carrying entries [G5]. A parse-broken file is left
* untouched (removal must never destroy what it cannot read) — the note says
* so. Event arrays we emptied lose their key; an emptied hooks object loses
* its key; foreign structure survives.
* its key; foreign structure survives. Cleans BOTH carriers (local +
* committed [D12]); the returned settingsPath/backup describe the local one,
* with committed-file actions reported via notes.
*/
export function removeClaudeHooks(workspaceDir: string): RemoveClaudeHooksResult {
const settingsPath = claudeSettingsPath(workspaceDir);
const local = removeHooksFromFile(claudeSettingsPath(workspaceDir));
const committed = removeHooksFromFile(claudeCommittedSettingsPath(workspaceDir));
const notes = [...local.notes];
if (committed.removed > 0) {
notes.push(`also removed ${committed.removed} entr${committed.removed === 1 ? 'y' : 'ies'} from the committed ${committed.settingsPath} [D12]`);
} else {
notes.push(...committed.notes.map((n) => `(committed carrier) ${n}`));
}
return {
settingsPath: local.settingsPath,
removed: local.removed + committed.removed,
backupPath: local.backupPath,
notes,
};
}
function removeHooksFromFile(settingsPath: string): RemoveClaudeHooksResult {
const notes: string[] = [];
if (!existsSync(settingsPath)) {
return { settingsPath, removed: 0, backupPath: null, notes: ['no settings file — nothing to remove'] };
src/core/bootstrap/host-specs.ts
+10
View File
@@ -99,6 +99,16 @@ export const TARGETS: Record<string, HostSpecTarget> = {
/** Settings file the hook writer targets, relative to the workspace root. */
export const CLAUDE_SETTINGS_FILE_RELPATH = join('.claude', 'settings.local.json');
/** The COMMITTED hook carrier [D12]. Cloud sessions clone the repo fresh and
* snapshot hook config at session start — the gitignored settings.local.json
* never exists there, and hooks written mid-session never activate. Cloud and
* attach installs therefore write hooks into the repo-committed
* `.claude/settings.json` with a PATH-resolved fail-open command (no absolute
* paths — the file travels between machines). Local installs keep
* settings.local.json; the writers enforce that one event never fires from
* both files. */
export const CLAUDE_COMMITTED_SETTINGS_FILE_RELPATH = join('.claude', 'settings.json');
/** Hook events bootstrap wires (plan D5 + hook events table).
* v0.45.7 ambient recall adds PreCompact: it BANKS the window's standing
* entities into session_context_state so the post-compaction SessionStart
src/core/bootstrap/lock.ts
+4
View File
@@ -57,6 +57,10 @@ export type BootstrapErrorCode =
/** Privacy verify could not complete (rate limit / 5xx) — refuse and name
* the reason, never fail-open; the fix is re-running, not panic [G8]. */
| 'VERIFY_UNAVAILABLE'
/** Cloud sandbox: a repo created mid-session is never attached to the
* session's GitHub proxy scope (REST 403, push denied) — creation must
* happen outside; the session is opened ON the repo, then attach [D-cloud]. */
| 'CLOUD_SANDBOX_REPO'
/** No agent.json — not an agent workspace. */
| 'NOT_A_WORKSPACE'
/** agent.json says `initialized: false` — an unrendered template clone [CX2-1]. */
src/core/bootstrap/repo.ts
+79 -32
View File
@@ -35,6 +35,8 @@ import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
import { configDir } from '../config.ts';
import { realpathOrResolve } from '../path-confine.ts';
import { defaultRunner, isProxyBlocked403, parseGithubOwnerRepo, type ExecRunner } from '../repo-visibility.ts';
import { detectExecutionEnvironment } from '../execution-env.ts';
import { loadWorkspaceAllowlist, scanFiles, SCAN_ALLOW_FILENAME } from '../secret-scan.ts';
import { GITHUB_URL_PLACEHOLDER } from './assets.ts';
import {
@@ -52,30 +54,10 @@ import { BootstrapError } from './lock.ts';
// Exec seam (shared by uninstall.ts; the dispatcher passes the real runner)
// ---------------------------------------------------------------------------
export interface ExecResult {
code: number;
stdout: string;
stderr: string;
}
/** Injectable subprocess seam. argv[0] is the binary; never a shell string. */
export type ExecRunner = (argv: string[]) => Promise<ExecResult>;
/** Default runner: Bun.spawn, both streams piped, spawn failure → code 127. */
export const defaultRunner: ExecRunner = async (argv: string[]): Promise<ExecResult> => {
try {
const proc = Bun.spawn(argv, { stdout: 'pipe', stderr: 'pipe', stdin: 'ignore' });
const [stdout, stderr, code] = await Promise.all([
new Response(proc.stdout).text(),
new Response(proc.stderr).text(),
proc.exited,
]);
return { code, stdout, stderr };
} catch (e) {
// Binary not on PATH (or unspawnable) — the conventional not-found code.
return { code: 127, stdout: '', stderr: (e as Error).message };
}
};
// Canonical definitions moved to repo-visibility.ts (the visibility ladder
// needs the same seam and must stay a leaf module); re-exported here so every
// existing consumer (uninstall.ts, bootstrap.ts, tests) keeps its import path.
export { defaultRunner, type ExecResult, type ExecRunner } from '../repo-visibility.ts';
// ---------------------------------------------------------------------------
// Receipt extension: the created repo URL [CX2-12 idempotency key]
@@ -97,6 +79,11 @@ export interface RepoReceipt extends InstallReceipt {
// Helpers
// ---------------------------------------------------------------------------
/** The one copy of the cloud repo-adoption instruction (two error sites). */
export const CLOUD_ATTACH_FLOW_HINT =
'Create the private repo from a normal machine (or github.com), open a cloud session ON that repo, ' +
'then run `gbrain bootstrap attach`.';
/** GitHub repo-name slug: lowercase, alnum runs joined by '-'. */
export function slugifyRepoName(name: string): string {
const slug = name
@@ -106,12 +93,11 @@ export function slugifyRepoName(name: string): string {
return slug || 'agent';
}
/** Parse owner/name out of an https or ssh GitHub remote URL. */
/** Parse owner/name out of an https or ssh GitHub remote URL. Thin adapter
* over the canonical repo-visibility parser (one grammar, three consumers). */
export function parseGithubRemote(url: string): { owner: string; name: string } | null {
const m =
/^https:\/\/github\.com\/([^/]+)\/([^/]+?)(?:\.git)?\/?$/.exec(url.trim()) ??
/^git@github\.com:([^/]+)\/([^/]+?)(?:\.git)?$/.exec(url.trim());
return m ? { owner: m[1], name: m[2] } : null;
const p = parseGithubOwnerRepo(url);
return p ? { owner: p.owner, name: p.repo } : null;
}
/** Re-exported for existing consumers; the single definition lives in
@@ -147,6 +133,28 @@ function isRateLimitOr5xx(stderr: string): boolean {
return /HTTP 5\d\d|HTTP 429|rate limit/i.test(stderr);
}
/** `gh auth status`'s `--active` flag (added in cli/cli v2.57.0, 2024-09-11)
* scopes the check to only the active account instead of aggregating every
* registered account. On an older `gh`, passing an unrecognized flag makes
* the WHOLE command fail — so Gate 2 must detect support before using it. */
const GH_ACTIVE_FLAG_MIN_VERSION = [2, 57, 0] as const;
/** Parses the `X.Y.Z` out of `gh --version`'s first line (`gh version X.Y.Z (DATE)`).
* Returns null on any unrecognized format — callers treat that as "unknown,
* don't assume support". */
function parseGhVersion(versionOutput: string): readonly [number, number, number] | null {
const m = /\bgh version (\d+)\.(\d+)\.(\d+)/.exec(versionOutput);
if (!m) return null;
return [Number(m[1]), Number(m[2]), Number(m[3])];
}
function ghVersionAtLeast(v: readonly [number, number, number], min: readonly [number, number, number]): boolean {
for (let i = 0; i < 3; i++) {
if (v[i] !== min[i]) return v[i]! > min[i]!;
}
return true;
}
/** The authenticated gh login, or null when it cannot be read/parsed. */
async function fetchAuthedLogin(runner: ExecRunner): Promise<string | null> {
const res = await runner(['gh', 'api', 'user']);
@@ -277,11 +285,20 @@ async function verifyRepoPrivate(runner: ExecRunner, owner: string, name: string
}
const res = await runner(['gh', 'api', `repos/${owner}/${name}`, '--jq', '.private']);
if (res.code !== 0) {
const reason = isRateLimitOr5xx(res.stderr) ? 'GitHub API rate limit / server error' : `gh api failed: ${res.stderr.trim() || `exit ${res.code}`}`;
// Classify the failure so the operator gets the REAL fix: a sandbox
// egress proxy blocking REST for a repo not attached to the session is a
// different problem from a token/rate-limit failure [D14 messaging].
const proxyBlocked = isProxyBlocked403(res.stderr);
const reason = proxyBlocked
? "this sandbox's egress proxy blocks GitHub REST for repos not attached to the session"
: isRateLimitOr5xx(res.stderr)
? 'GitHub API rate limit / server error'
: `gh api failed: ${res.stderr.trim() || `exit ${res.code}`}`;
const nextStep = proxyBlocked ? CLOUD_ATTACH_FLOW_HINT : 'The repo may be fine — re-run `gbrain bootstrap repo` to verify.';
throw new BootstrapError(
'VERIFY_UNAVAILABLE',
`could not verify that ${owner}/${name} is private (${reason}). ` +
'The repo may be fine — re-run `gbrain bootstrap repo` to verify. Nothing is pushed to a repo whose privacy is unverified.',
`${nextStep} Nothing is pushed to a repo whose privacy is unverified.`,
{ details: { owner, name, stderr: res.stderr } },
);
}
@@ -599,7 +616,22 @@ export async function createPrivateRepo(
}
// Gate 2: authenticated. Exit-code-2 — the human runs `gh auth login`.
const ghAuth = await runner(['gh', 'auth', 'status']);
// `--hostname github.com` scopes the check to the host this flow actually
// targets (every downstream call — parseGithubOwnerRepo, the repo-create
// URL fallback, etc. — is github.com-only), so an unrelated broken account
// on some other configured host (e.g. a GitHub Enterprise instance) can't
// false-block it either. `--active` (only when the installed `gh` supports
// it) further restricts that host's check to the active account. Bare
// `gh auth status` aggregates EVERY registered account on EVERY host and
// exits 1 if any one of them is invalid — even an unused, long-expired
// account — which false-blocks this gate while the active account (what
// `gh`/`git` actually use) is perfectly healthy.
const ghVersionTuple = parseGhVersion(ghVersion.stdout);
const ghSupportsActiveFlag = ghVersionTuple !== null && ghVersionAtLeast(ghVersionTuple, GH_ACTIVE_FLAG_MIN_VERSION);
const ghAuthArgv = ghSupportsActiveFlag
? ['gh', 'auth', 'status', '--active', '--hostname', 'github.com']
: ['gh', 'auth', 'status', '--hostname', 'github.com'];
const ghAuth = await runner(ghAuthArgv);
if (ghAuth.code !== 0) {
throw new BootstrapError(
'GH_AUTH',
@@ -684,6 +716,21 @@ export async function createPrivateRepo(
return { url: originUrl, name: parsed.name, disposition: viaUrlMatch ? 'reused' : 'adopted', reused: true };
}
// Cloud-sandbox guard: `gh repo create` inside a proxied cloud session
// makes a repo the session is NOT attached to — REST verification 403s and
// the proxy denies every push to it, so creation there is a dead end by
// construction. Fail fast with the flow that works instead. (Adoption of an
// EXISTING attached origin above is untouched — that is the sanctioned path.)
if (detectExecutionEnvironment() === 'cloud-sandbox') {
throw new BootstrapError(
'CLOUD_SANDBOX_REPO',
'this is a cloud sandbox session — a repo created from inside it would not be attached to the ' +
"session's GitHub scope (verification and pushes are blocked by the proxy). " +
CLOUD_ATTACH_FLOW_HINT,
{ exitCode: 2 },
);
}
// Ensure a git repo exists (main branch on fresh init).
const gitDir = await runner(['git', '-C', workspaceDir, 'rev-parse', '--git-dir']);
if (gitDir.code !== 0) {
src/core/bootstrap/status.ts
+64 -22
View File
@@ -31,7 +31,10 @@ import { join } from 'node:path';
import { VERSION } from '../../version.ts';
import { loadConfigFileOnly } from '../config.ts';
import { binaryOnPath, detectExecutionEnvironment, type ExecutionEnvironment } from '../execution-env.ts';
import { resolveGbrainHome } from '../gbrain-home.ts';
import { githubOwnerRepoString, isProxyBlocked403 } from '../repo-visibility.ts';
import { BOOTSTRAP_TEMPLATES } from './assets.ts';
import {
readManifest,
@@ -40,7 +43,12 @@ import {
type ManifestState,
} from './format.ts';
import { interviewStatePath, status as interviewStatus } from './interview.ts';
import { CLAUDE_SETTINGS_FILE_RELPATH, GBRAIN_HOOK_MARKER_KEY, GBRAIN_HOOK_MARKER_VALUE } from './host-specs.ts';
import {
CLAUDE_COMMITTED_SETTINGS_FILE_RELPATH,
CLAUDE_SETTINGS_FILE_RELPATH,
GBRAIN_HOOK_MARKER_KEY,
GBRAIN_HOOK_MARKER_VALUE,
} from './host-specs.ts';
// ---------------------------------------------------------------------------
// The phase list [D5] — single TS source of truth
@@ -86,13 +94,6 @@ interface PhaseSpec {
detect: (ws: string, ctx: DetectCtx) => { state: PhaseState; detail?: string };
}
function binaryOnPath(name: string): boolean {
try {
return Bun.which(name) !== null;
} catch {
return false;
}
}
/** The workspace's `origin` remote URL, or null (not a repo / no origin).
* Shared with verify.ts's origin-probe sites — one 5s-timeout idiom. */
@@ -119,15 +120,25 @@ export type OriginVisibility =
| { verdict: 'public'; detail: string }
| { verdict: 'unknown'; detail: string };
/** Probe an origin's visibility via `gh repo view --json isPrivate` (the same
* 5s timeout idiom as gitOriginUrl). gh missing / offline / non-GitHub →
* 'unknown', never an invented answer. */
/** Probe an origin's visibility via REST — `gh api repos/{owner}/{repo}` —
* NEVER `gh repo view` (GraphQL under the hood; cloud sandbox proxies pin
* GraphQL to a fixed operation set and 403 it even with a user token). Sync
* by contract: this runs inside the sync phase-detect chain, so the full
* async ladder lives in verify/push; status keeps the cheap REST answer.
* gh missing / offline / non-GitHub / 403 → 'unknown', never an invented
* answer; a proxy-shaped 403 names the real fix. */
export function probeOriginVisibility(origin: string): OriginVisibility {
const ownerRepo = githubOwnerRepoString(origin);
if (ownerRepo === null) {
// Never send a non-github origin string into a REST path — self-hosted
// hostnames (or URL-embedded credentials) must not reach api.github.com.
return { verdict: 'unknown', detail: 'origin is not a github.com URL — cannot verify via REST' };
}
try {
// env: process.env — Bun's execFileSync otherwise resolves the binary
// against the STARTUP env snapshot, making PATH-shimmed test fakes (and
// any runtime PATH change) invisible (the workspace-push.ts precedent).
const out = execFileSync('gh', ['repo', 'view', origin, '--json', 'isPrivate', '--jq', '.isPrivate'], {
const out = execFileSync('gh', ['api', `repos/${ownerRepo}`, '--jq', '.private'], {
stdio: ['ignore', 'pipe', 'pipe'],
timeout: 5_000,
env: process.env,
@@ -138,21 +149,36 @@ export function probeOriginVisibility(origin: string): OriginVisibility {
if (out === 'false') return { verdict: 'public', detail: `${origin} is PUBLIC` };
return { verdict: 'unknown', detail: `gh returned unexpected output: ${out.slice(0, 80)}` };
} catch (e) {
const err = e as NodeJS.ErrnoException;
const err = e as NodeJS.ErrnoException & { stderr?: Buffer | string };
if (err?.code === 'ENOENT') {
return { verdict: 'unknown', detail: 'gh CLI not installed — cannot verify origin visibility' };
}
return { verdict: 'unknown', detail: `gh repo view failed (${(err?.message ?? 'unknown').slice(0, 120)})` };
const stderr = (err?.stderr ?? '').toString();
if (isProxyBlocked403(stderr)) {
return {
verdict: 'unknown',
detail: 'GitHub REST blocked by this sandbox\'s egress proxy (repo not attached to the session) — push-time verification falls back to git protocol',
};
}
return { verdict: 'unknown', detail: `gh api failed (${(err?.message ?? 'unknown').slice(0, 120)})` };
}
}
/** True when `<ws>/.claude/settings.local.json` carries a gbrain hook marker. */
/** True when either hook carrier — the gitignored `settings.local.json` OR
* the committed `.claude/settings.json` [D12] — carries a gbrain marker. */
export function hooksInstalled(ws: string): boolean {
try {
const raw = readFileSync(join(ws, CLAUDE_SETTINGS_FILE_RELPATH), 'utf8');
// Structural probe without a full merge: the marker key/value pair is the
// idempotency contract host-specs.ts pins, so a substring check is honest.
return raw.includes(`"${GBRAIN_HOOK_MARKER_KEY}"`) && raw.includes(`"${GBRAIN_HOOK_MARKER_VALUE}"`);
const probe = (relpath: string): boolean => {
try {
const raw = readFileSync(join(ws, relpath), 'utf8');
// Structural probe without a full merge: the marker key/value pair is
// the idempotency contract host-specs.ts pins — substring is honest.
return raw.includes(`"${GBRAIN_HOOK_MARKER_KEY}"`) && raw.includes(`"${GBRAIN_HOOK_MARKER_VALUE}"`);
} catch {
return false;
}
};
return probe(CLAUDE_SETTINGS_FILE_RELPATH) || probe(CLAUDE_COMMITTED_SETTINGS_FILE_RELPATH);
} catch {
return false;
}
@@ -416,6 +442,11 @@ export interface StatusReport {
phases: PhaseStatus[];
/** Resume hint of the first non-done phase; null when everything is done. */
next: string | null;
/** WHERE this install is running [D-cloud]: 'local' | 'cloud-sandbox' |
* 'ephemeral-container'. Installing agents branch on this — cron is skipped
* in containers, repo creation is redirected in cloud sandboxes, and the
* runbook's cloud section keys off it. */
execution_environment: ExecutionEnvironment;
runbookSkew?: RunbookSkew;
/** Template-door privacy gate: set only for an UNINITIALIZED template clone
* whose origin exists and is not verifiably private. 'public' is a hard
@@ -491,11 +522,21 @@ export async function statusReport(ws: string, opts: StatusReportOpts = {}): Pro
}
}
// Support blob [B5].
// Support blob [B5]. Push status via the shared per-root reader [D8/D13]:
// a failing workspace wins over another's success; else the newest record.
let lastPush: StatusSupport['last_push'] = null;
try {
const p = join(gbrainHomeDir, 'bootstrap', 'push-status.json');
if (existsSync(p)) lastPush = JSON.parse(readFileSync(p, 'utf8')) as StatusSupport['last_push'];
const { readPushStatuses, summarizePushStatuses } = await import('../workspace-push.ts');
const entries = readPushStatuses();
const { failing } = summarizePushStatuses(entries);
const pick = failing[0] ?? entries.sort((a, b) => Date.parse(b.ts ?? '') - Date.parse(a.ts ?? ''))[0];
if (pick) {
lastPush = {
...(pick.ts !== undefined ? { ts: pick.ts } : {}),
...(pick.ok !== undefined ? { ok: pick.ok } : {}),
...(pick.reason !== undefined ? { reason: pick.reason } : {}),
};
}
} catch {
lastPush = null;
}
@@ -533,6 +574,7 @@ export async function statusReport(ws: string, opts: StatusReportOpts = {}): Pro
workspace: ws,
phases,
next,
execution_environment: detectExecutionEnvironment(),
...(runbookSkew ? { runbookSkew } : {}),
...(privacyGate ? { privacy_gate: privacyGate } : {}),
support,
src/core/bootstrap/verify.ts
+130 -16
View File
@@ -35,12 +35,13 @@ import { join } from 'node:path';
import type { BrainEngine } from '../engine.ts';
import { operations, type Operation, type OperationContext } from '../operations.ts';
import { loadConfigFileOnly, type GBrainConfig } from '../config.ts';
import { detectExecutionEnvironment } from '../execution-env.ts';
import { resolveGbrainHome } from '../gbrain-home.ts';
import { realpathOrResolve } from '../path-confine.ts';
import { runMaintenanceSweep } from '../sweep.ts';
import { detectCapabilities, renderCapabilityReport, type CapabilityReport } from '../capability.ts';
import { loadWorkspaceAllowlist, matchesGlob, scanFiles, type SecretFinding } from '../secret-scan.ts';
import { PUSH_DENY_GLOBS, verifyRemotePrivacy, pushStatusPath } from '../workspace-push.ts';
import { PUSH_DENY_GLOBS, verifyRemotePrivacy, readPushStatuses, summarizePushStatuses } from '../workspace-push.ts';
import { FACTS_DEFAULT_VISIBILITY_KEY } from '../facts/visibility.ts';
import { byteFloors } from './render.ts';
import { BOOTSTRAP_TEMPLATES, loadQuestionBank } from './assets.ts';
@@ -317,14 +318,69 @@ function checkDenyGlobs(ws: string): VerifyCheck {
}
}
function checkRepoPrivacy(ws: string): VerifyCheck {
/** Self-repair channel for existing installs [B8]: a git-TRACKED .mcp.json
* carries an absolute machine-specific binary path into the private repo.
* Fresh renders gitignore it; this warn heals pre-fix installs. Never gates. */
function checkMcpJsonHygiene(ws: string): VerifyCheck {
const id = 'mcp_json_hygiene';
try {
if (!existsSync(join(ws, '.mcp.json'))) return { id, ok: true, detail: 'no .mcp.json in the workspace' };
try {
execFileSync('git', ['-C', ws, 'ls-files', '--error-unmatch', '.mcp.json'], {
stdio: 'ignore', timeout: 5_000,
});
} catch {
return { id, ok: true, detail: '.mcp.json present but untracked (gitignored) — correct' };
}
return {
id,
ok: true, // warn-only: informational, never gating
detail:
'WARN: machine-specific .mcp.json is COMMITTED to the repo — run `git rm --cached .mcp.json` ' +
'(bootstrap now gitignores it; it regenerates via `claude mcp add` / `bootstrap hooks --repair`)',
};
} catch (e) {
return { id, ok: true, detail: `mcp.json hygiene probe failed (${(e as Error).message})` };
}
}
/** [D12 upgrade-path guard]: an event carried by BOTH hook files double-fires
* every turn (possible when the committed carrier arrives via git pull onto a
* machine whose local file predates the dedupe-aware writers). Warn-only. */
function checkHookCarrierOverlap(ws: string): VerifyCheck {
const id = 'hook_carrier_overlap';
try {
const events = (['SessionStart', 'UserPromptSubmit', 'Stop', 'SessionEnd'] as const).filter((event) => {
const has = (rel: string): boolean => {
try {
const parsed = JSON.parse(readFileSync(join(ws, rel), 'utf8')) as { hooks?: Record<string, unknown> };
const groups = parsed?.hooks?.[event];
return Array.isArray(groups) && JSON.stringify(groups).includes('"_gbrain"');
} catch {
return false;
}
};
return has(join('.claude', 'settings.json')) && has(join('.claude', 'settings.local.json'));
});
if (events.length === 0) return { id, ok: true, detail: 'no event fires from both hook carriers' };
return {
id,
ok: true, // warn-only self-repair channel
detail: `WARN: ${events.join(', ')} fire from BOTH .claude/settings.json and settings.local.json (double-fire) — run \`gbrain bootstrap hooks --repair\` to dedupe`,
};
} catch (e) {
return { id, ok: true, detail: `carrier overlap probe failed (${(e as Error).message})` };
}
}
async function checkRepoPrivacy(ws: string): Promise<VerifyCheck> {
const id = 'repo_privacy';
try {
const origin = gitOriginUrl(ws);
if (!origin) {
return { id, ok: true, detail: 'local-only (no origin remote) — run `gbrain bootstrap repo` any time to add the private body' };
}
const verdict = verifyRemotePrivacy(ws);
const verdict = await verifyRemotePrivacy(ws);
if (verdict.verdict === 'private') return { id, ok: true, detail: `origin verified private (${origin})` };
if (verdict.verdict === 'not_private') return { id, ok: false, detail: `origin is NOT private: ${verdict.detail} — make it private before pushing workspace contents` };
return { id, ok: false, detail: `origin visibility unverifiable (${verdict.detail}) — refusing to bless an unverified remote [G8]; re-run once gh works` };
@@ -333,6 +389,37 @@ function checkRepoPrivacy(ws: string): VerifyCheck {
}
}
/** Informational, NEVER gating [D-cloud]: name the detected execution
* environment and its expected degradations so an installing agent (and the
* pasted verify report) states them as facts instead of rediscovering them
* as mystery failures. */
function checkExecutionEnvironment(): VerifyCheck {
const id = 'execution_env';
try {
const env = detectExecutionEnvironment();
if (env === 'cloud-sandbox') {
return {
id,
ok: true,
detail:
'cloud sandbox detected — expected degradations: no crontab (scheduled pull skipped; per-turn/session-end pushes cover it); ' +
'GitHub GraphQL always blocked and REST scoped to session-attached repos (privacy verification falls back to git protocol); ' +
'pushes restricted to the session\'s working branch; only repo-committed files carry into the next session',
};
}
if (env === 'ephemeral-container') {
return {
id,
ok: true,
detail: 'container detected — no reliable scheduler (scheduled pull skipped); event-driven pushes cover persistence',
};
}
return { id, ok: true, detail: 'local machine — full persistence surface available' };
} catch (e) {
return { id, ok: true, detail: `environment detection failed (${(e as Error).message}) — treated as local` };
}
}
/** [CX-P1.1] Single-principal posture: facts written without an explicit
* visibility must be recallable by the owner's own sessions (the harness
* reads at visibility='world'). Set-IF-UNSET only, through the engine config
@@ -390,16 +477,33 @@ function checkMcpSurface(): VerifyCheck {
}
}
/**
* Pure, engine-free derivation of the collision-fallback source_id for a
* workspace — a deterministic hash of the workspace's real path, no DB
* lookup involved. `resolveSourceIdCollision` (below) is the only thing that
* decides WHETHER this id is actually needed (that half requires the engine,
* since the sources registry lives only in the DB) — but the id itself is
* safe to preview from an engine-free phase. `bootstrap hooks` does exactly
* that, so a human has the fallback id in hand before they ever hand-register
* a source, instead of discovering it only after an FK error + a corrective
* `verify` run.
*/
export function deriveWorkspaceSourceId(ws: string): string {
const hash = createHash('sha256').update(realpathOrResolve(ws)).digest('hex').slice(0, 8);
return `workspace-${hash}`;
}
/**
* source_id collision resolution [engine seam]. Render is ENGINE-FREE and the
* sources registry lives ONLY in the DB (no registry file exists), so verify —
* the one bootstrap subcommand holding an engine — is where a manifest
* source_id already registered to a DIFFERENT checkout is detected. On
* collision it derives a stable `workspace-<8char-path-hash>` id, persists it
* to agent.json (render preserves it on re-render), and names the re-register
* steps; every consumer (hooks GBRAIN_SOURCE env, verify, status hints,
* attach, repo persistence) reads manifest.source_id, so the derived id
* propagates. Returns a sourceId ONLY when it derived one.
* collision it derives a stable `workspace-<8char-path-hash>` id (via
* `deriveWorkspaceSourceId`), persists it to agent.json (render preserves it
* on re-render), and names the re-register steps; every consumer (hooks
* GBRAIN_SOURCE env, verify, status hints, attach, repo persistence) reads
* manifest.source_id, so the derived id propagates. Returns a sourceId ONLY
* when it derived one.
*/
async function resolveSourceIdCollision(
engine: BrainEngine,
@@ -419,8 +523,7 @@ async function resolveSourceIdCollision(
if (realpathOrResolve(registered) === realpathOrResolve(brainDir)) {
return { sourceId: null, check: null }; // same checkout — no collision
}
const hash = createHash('sha256').update(realpathOrResolve(ws)).digest('hex').slice(0, 8);
const derived = `workspace-${hash}`;
const derived = deriveWorkspaceSourceId(ws);
writeManifest(ws, { ...state.manifest, source_id: derived });
return {
sourceId: derived,
@@ -692,11 +795,19 @@ async function checkHooksSmoke(engine: BrainEngine, ws: string, sourceId: string
function checkPushProbe(ws: string): VerifyCheck {
const id = 'push_probe';
try {
const p = pushStatusPath();
if (existsSync(p)) {
const s = JSON.parse(readFileSync(p, 'utf8')) as { ts?: string; ok?: boolean; reason?: string };
if (s.ok === true) return { id, ok: true, detail: `last workspace push succeeded (${s.ts ?? 'unknown time'})` };
return { id, ok: true, warn: true, detail: `last workspace push FAILED (${s.ts ?? 'unknown'}): ${s.reason ?? 'unknown'} — run \`gbrain sources push --path ${ws}\`` };
// Read through the shared per-root reader [D8/D13] — a v0.45.8+ push
// writes push-status-<roothash>.json, not the legacy single file, so the
// old direct read reported "no push recorded" on every fresh install.
const entries = readPushStatuses();
if (entries.length > 0) {
const { failing } = summarizePushStatuses(entries);
if (failing.length > 0) {
const s = failing[0]!;
const rest = failing.length > 1 ? ` [+${failing.length - 1} more]` : '';
return { id, ok: true, warn: true, detail: `last workspace push FAILED (${s.ts ?? 'unknown'}): ${s.reason ?? 'unknown'}${rest} — run \`gbrain sources push --path ${s.repoRoot ?? ws}\`` };
}
const ok = entries.find((e) => e.ok === true);
return { id, ok: true, detail: `last workspace push succeeded (${ok?.ts ?? 'unknown time'})` };
}
const origin = gitOriginUrl(ws);
if (origin) return { id, ok: true, warn: true, detail: 'origin exists but no push recorded yet — run `gbrain sources push` once to prove the persistence path' };
@@ -813,7 +924,10 @@ export async function verifyWorkspace(
checks.push(checkByteFloors(ws));
checks.push(checkSecretScan(ws));
checks.push(checkDenyGlobs(ws));
checks.push(checkRepoPrivacy(ws));
checks.push(await checkRepoPrivacy(ws));
checks.push(checkExecutionEnvironment());
checks.push(checkMcpJsonHygiene(ws));
checks.push(checkHookCarrierOverlap(ws));
// Round-trip family only makes sense on a reachable engine.
if (engineHealthy) {
src/core/brain-repo-durability.ts
+188 -19
View File
@@ -39,8 +39,10 @@ import {
import { findResolverFile, RESOLVER_FILENAMES } from './resolver-filenames.ts';
import { redactSecretsInText } from './minions/handlers/shell-redact.ts';
import { ensureGbrainHome, resolveGbrainHome } from './gbrain-home.ts';
// Static import → bundled into the --compile binary so the taxonomy never drifts
// and needs no runtime skills/ directory.
import { binaryOnPath } from './execution-env.ts';
import { loadFilingRules, type FilingRulesDoc } from './filing-audit.ts';
// Bundled into the --compile binary as the fallback taxonomy for repos that
// don't ship their own — see resolveFilingRules().
import filingRulesDoc from '../../skills/_brain-filing-rules.json';
// ── Types ───────────────────────────────────────────────────────────────────
@@ -244,10 +246,26 @@ exit 4
// ── Managed AGENTS/RESOLVER block (taxonomy from filing rules; no drift) ─────
function renderTaxonomyLines(): string {
/**
* Resolve the filing-rules taxonomy for `repoPath`: prefer the repo's own
* `skills/_brain-filing-rules.json`, then `_brain-filing-rules.json` at the
* repo root, else the bundled default. Fails open — a missing or malformed
* repo file must never break `sources harden`.
*/
function resolveFilingRules(repoPath: string): FilingRulesDoc {
for (const dir of [join(repoPath, 'skills'), repoPath]) {
try {
const rules = loadFilingRules(dir);
if (rules) return rules;
} catch { /* malformed — fall through to the bundled default */ }
}
return filingRulesDoc as FilingRulesDoc;
}
function renderTaxonomyLines(rules: FilingRulesDoc): string {
const seen = new Set<string>();
const lines: string[] = [];
for (const r of (filingRulesDoc as any).rules ?? []) {
for (const r of rules.rules ?? []) {
const dir = String(r.directory || '').trim();
if (!dir || seen.has(dir)) continue;
seen.add(dir);
@@ -256,7 +274,8 @@ function renderTaxonomyLines(): string {
return lines.join('\n');
}
function renderManagedBlock(): string {
function renderManagedBlock(repoPath: string): string {
const rules = resolveFilingRules(repoPath);
return `${AGENTS_BEGIN}
<!-- gbrain durability rules. This block is regenerated by \`gbrain sources harden\`.
Do not index as user knowledge; do not edit between the markers. -->
@@ -264,7 +283,7 @@ function renderManagedBlock(): string {
1. **Deterministic filing — never use /tmp as storage.** Every persistent output
goes to its taxonomy path (canonical, from \`skills/_brain-filing-rules.json\`):
${renderTaxonomyLines()}
${renderTaxonomyLines(rules)}
Writing to /tmp, scratch dirs, or outside the repo is forbidden for anything
meant to persist.
@@ -284,7 +303,7 @@ ${AGENTS_END}`;
function patchResolverFile(repoPath: string, dryRun: boolean): { status: StepStatus; detail: string } {
const existing = findResolverFile(repoPath);
const target = existing ?? join(repoPath, RESOLVER_FILENAMES[1]); // default AGENTS.md
const block = renderManagedBlock();
const block = renderManagedBlock(repoPath);
const name = relative(repoPath, target) || target;
let current = '';
@@ -312,6 +331,19 @@ function patchResolverFile(repoPath: string, dryRun: boolean): { status: StepSta
// ── Local untracked post-commit hook (D9) ───────────────────────────────────
/** Resolve the active hooks dir (honors a pre-existing core.hooksPath). */
/** Worktree-safe path inside the git dir [D6]: `.git` is a FILE in worktrees
* and submodules, so any path under it must come from git itself (rev-parse
* with the git-path query), never string-joined onto `<repo>/.git/`. */
function gitDirPath(repoPath: string, rel: string): string {
try {
const p = execFileSync('git', ['-C', repoPath, 'rev-parse', '--git-path', rel], {
stdio: ['ignore', 'pipe', 'ignore'], timeout: 10_000, env: { ...process.env, ...GIT_ENV },
}).toString().trim();
if (p) return isAbsolute(p) ? p : join(repoPath, p);
} catch { /* fall through to the classic layout */ }
return join(repoPath, '.git', rel);
}
function resolveHooksDir(repoPath: string): { dir: string; tracked: boolean } {
let hooksPath = '';
try {
@@ -325,12 +357,12 @@ function resolveHooksDir(repoPath: string): { dir: string; tracked: boolean } {
const tracked = !dir.includes(`${join('.git', '')}`) && !dir.endsWith('.git/hooks');
return { dir, tracked };
}
return { dir: join(repoPath, '.git', 'hooks'), tracked: false };
return { dir: gitDirPath(repoPath, 'hooks'), tracked: false };
}
/** Ensure a repo-relative path is in .git/info/exclude so our hook stays untracked. */
/** Ensure a repo-relative path is in the git exclude file so our hook stays untracked. */
function ensureExcluded(repoPath: string, relPath: string): void {
const exclude = join(repoPath, '.git', 'info', 'exclude');
const exclude = gitDirPath(repoPath, 'info/exclude');
try {
mkdirSync(dirname(exclude), { recursive: true });
let body = existsSync(exclude) ? readFileSync(exclude, 'utf-8') : '';
@@ -418,6 +450,58 @@ export function commitWriteThroughFile(repoPath: string, absPath: string, slug:
}
}
// ── Push-state query (D14) ───────────────────────────────────────────────────
export type PushLogStatus = 'ok' | 'needs_attention' | 'unknown';
export interface PushLogOutcome {
status: PushLogStatus;
detail: string;
/** UTC timestamp parsed from the log line, when found. */
at?: string;
}
const PUSH_LOG_OK = /^(\S+) \[push\] (?:ok|ok-after-rebase) (\S+)\b/;
const PUSH_LOG_LOCAL_ONLY = /^(\S+) \[push\] LOCAL-ONLY, NEEDS ATTENTION: (\S+) /;
const PUSH_LOG_LOCK_TIMEOUT = /^(\S+) \[push\] lock-timeout (\S+)\b/;
/**
* Best-effort read of the most recently logged push outcome for `branch`,
* from the shared hook log ($GBRAIN_HOME/brain-push.log). The push itself
* runs detached in the background (see `renderPostCommitHook`), so nothing
* synchronous ever learns whether a given commit's own push landed — this is
* the queryable substitute: "as of the last thing the hook logged for this
* branch, were pushes landing?"
*
* The log is host-wide and keyed only by branch name, not repo path, so two
* different hardened repos sharing a branch name (e.g. both on `main`) share
* this signal. That's an acceptable approximation for a liveness check, not
* a per-repo guarantee.
*/
export function getLastPushOutcome(branch: string): PushLogOutcome {
const log = pushLogPath();
if (!existsSync(log)) return { status: 'unknown', detail: 'no push attempts logged yet' };
let lines: string[];
try {
lines = readFileSync(log, 'utf-8').split('\n');
} catch (e) {
return { status: 'unknown', detail: `push log unreadable: ${(e as Error).message}` };
}
for (let i = lines.length - 1; i >= 0; i--) {
const line = lines[i];
if (!line) continue;
let m = line.match(PUSH_LOG_OK);
if (m && m[2] === branch) return { status: 'ok', detail: line.trim(), at: m[1] };
m = line.match(PUSH_LOG_LOCAL_ONLY);
if (m && m[2] === branch) return { status: 'needs_attention', detail: line.trim(), at: m[1] };
m = line.match(PUSH_LOG_LOCK_TIMEOUT);
if (m && m[2] === branch) return { status: 'needs_attention', detail: line.trim(), at: m[1] };
}
return { status: 'unknown', detail: `no push attempt logged yet for branch '${branch}'` };
}
// ── Committed helper ────────────────────────────────────────────────────────
function installHelper(repoPath: string, dryRun: boolean): { status: StepStatus; detail: string } {
@@ -525,6 +609,65 @@ function launchdPlistPath(sourceId: string): string {
return join(process.env.HOME || '', 'Library', 'LaunchAgents', `${cronLabel(sourceId)}.plist`);
}
/** Default scheduled-pull interval — ONE definition (harden + doctor probe). */
export const DEFAULT_PULL_INTERVAL_SEC = 1800;
export interface DurabilityJobStatus {
kind: 'launchd' | 'crontab' | 'none';
wrapperPresent: boolean;
/** Liveness rung [D7]: darwin = launchctl reports the label loaded; linux =
* the crontab line exists. undefined = probe unavailable on this host. */
live?: boolean;
/** Pull log fresher than 2× the interval. undefined = no log yet (fresh
* install / never fired) — absence is not evidence of death. */
logFresh?: boolean;
}
/**
* Presence + LIVENESS of the scheduled-pull job [D7]. Presence-only checks
* (existsSync on the plist) certify dead jobs as healthy — the documented
* autopilot-status failure mode — so this probes whether the job is actually
* loaded/registered and whether its log shows recent life.
*/
export function durabilityJobStatus(
sourceId: string,
intervalSec = DEFAULT_PULL_INTERVAL_SEC,
platform: NodeJS.Platform = process.platform,
): DurabilityJobStatus {
const wrapperPresent = existsSync(cronWrapperPath(sourceId));
let kind: DurabilityJobStatus['kind'] = 'none';
let live: boolean | undefined;
if (platform === 'darwin') {
if (existsSync(launchdPlistPath(sourceId))) {
kind = 'launchd';
try {
execFileSync('launchctl', ['list', cronLabel(sourceId)], {
stdio: 'ignore', timeout: 5_000, env: process.env,
});
live = true;
} catch {
live = false; // plist on disk but not loaded — the dead-job shape
}
}
} else if (binaryOnPath('crontab')) {
try {
const tab = execSync('crontab -l 2>/dev/null', { encoding: 'utf-8', env: process.env });
if (tab.includes(cronLabel(sourceId))) {
kind = 'crontab';
live = true; // the line exists; cron itself is the OS's liveness domain
}
} catch { /* no crontab for this user */ }
}
let logFresh: boolean | undefined;
try {
const log = join(process.env.HOME || '', '.gbrain', 'brain-pull.log');
if (existsSync(log)) {
logFresh = Date.now() - statSync(log).mtimeMs <= 2 * intervalSec * 1000;
}
} catch { /* leave undefined */ }
return { kind, wrapperPresent, ...(live !== undefined ? { live } : {}), ...(logFresh !== undefined ? { logFresh } : {}) };
}
/** Pure cron-wrapper renderer (DB-free pull; secret-free — sources the shell
* profile rather than baking keys in). Exported for tests. */
export function renderCronWrapper(sourceId: string, repoPath: string, branch: string, cli: string, logPath: string): string {
@@ -534,9 +677,12 @@ export function renderCronWrapper(sourceId: string, repoPath: string, branch: st
# Sources the shell profile for secrets, then runs the hardened, DB-free pull.
[ -f ~/.zshenv ] && source ~/.zshenv 2>/dev/null
source ~/.zshrc 2>/dev/null || source ~/.bashrc 2>/dev/null || true
# Self-disable if the captured checkout is gone (rename/relocation).
if [ ! -d '${q(repoPath)}/.git' ]; then
echo "$(date -u +%FT%TZ) [cron] path gone, skipping: ${q(repoPath)}" >> "${q(logPath)}" 2>/dev/null || true
# Self-disable when the captured checkout is no longer a git working tree:
# gone, OR its path reused by a non-git directory. git rev-parse recognizes
# both the classic .git-dir layout AND worktrees/submodules (where the git
# marker is a FILE), so a bare dir test would wrongly disable a live worktree.
if ! git -C '${q(repoPath)}' rev-parse --is-inside-work-tree >/dev/null 2>&1; then
echo "$(date -u +%FT%TZ) [cron] not a git work tree, skipping: ${q(repoPath)}" >> "${q(logPath)}" 2>/dev/null || true
exit 0
fi
exec '${q(cli)}' sources pull --path '${q(repoPath)}' --branch '${q(branch)}'
@@ -566,10 +712,30 @@ export function generateBrainPullPlist(label: string, wrapperPath: string, home:
</plist>`;
}
function installDurabilityCron(sourceId: string, repoPath: string, branch: string, intervalSec: number, dryRun: boolean): { status: StepStatus; detail: string } {
export function installDurabilityCron(
sourceId: string,
repoPath: string,
branch: string,
intervalSec: number,
dryRun: boolean,
platform: NodeJS.Platform = process.platform,
): { status: StepStatus; detail: string } {
// Probe FIRST [D-cloud/B2]: containers and cloud sandboxes ship without
// crontab, and that is EXPECTED there — the honest answer is a skip that
// names what still covers persistence, not a needs_attention that reads
// like a bug (and no wrapper file is written for a job that can't exist).
if (platform !== 'darwin' && !binaryOnPath('crontab')) {
return {
status: 'skipped',
detail:
'no crontab on this host (container/cloud sandbox) — scheduled pull skipped; ' +
'the post-commit auto-push and per-turn/session-end pushes cover persistence here. ' +
'Run `gbrain sources harden <id>` on a persistent machine to add the scheduled pull.',
};
}
const wrapper = dryRun ? cronWrapperPath(sourceId) : writeCronWrapper(sourceId, repoPath, branch);
const home = process.env.HOME || '';
if (process.platform === 'darwin') {
if (platform === 'darwin') {
const plistPath = launchdPlistPath(sourceId);
if (dryRun) return { status: 'fixed', detail: `would install launchd ${cronLabel(sourceId)} every ${intervalSec}s (dry-run)` };
mkdirSync(dirname(plistPath), { recursive: true });
@@ -583,12 +749,15 @@ function installDurabilityCron(sourceId: string, repoPath: string, branch: strin
const marker = `# ${cronLabel(sourceId)}`;
const cronLine = `*/${minutes} * * * * ${wrapper} ${marker}`;
if (dryRun) return { status: 'fixed', detail: `would install crontab (every ${minutes}m) (dry-run)` };
// env: process.env on both calls — Bun otherwise resolves `crontab` against
// the STARTUP env snapshot, making runtime PATH changes (and PATH-shimmed
// test fakes) invisible (the workspace-push.ts / status.ts precedent).
let existingCron = '';
try { existingCron = execSync('crontab -l 2>/dev/null', { encoding: 'utf-8' }); } catch { /* none */ }
try { existingCron = execSync('crontab -l 2>/dev/null', { encoding: 'utf-8', env: process.env }); } catch { /* none */ }
const kept = existingCron.split('\n').filter(l => l && !l.includes(marker));
const next = [...kept, cronLine, ''].join('\n');
try {
execSync('crontab -', { input: next, stdio: ['pipe', 'ignore', 'ignore'] });
execSync('crontab -', { input: next, stdio: ['pipe', 'ignore', 'ignore'], env: process.env });
return { status: 'fixed', detail: `crontab every ${minutes}m` };
} catch (e) {
return { status: 'needs_attention', detail: `crontab install failed: ${(e as Error).message.slice(0, 120)}` };
@@ -667,7 +836,7 @@ function resolveRepoRoot(path: string): string {
}
}
function currentBranch(repoPath: string): string {
export function currentBranch(repoPath: string): string {
try {
return execFileSync('git', ['-C', repoPath, 'rev-parse', '--abbrev-ref', 'HEAD'], {
stdio: ['ignore', 'pipe', 'ignore'], timeout: 10_000, env: { ...process.env, ...GIT_ENV },
@@ -701,7 +870,7 @@ export async function hardenBrainRepo(opts: HardenOpts): Promise<DurabilityRepor
const dryRun = !!opts.dryRun;
const installCron = opts.installCron !== false;
const verify = opts.verify !== false;
const intervalSec = opts.intervalSec ?? 1800;
const intervalSec = opts.intervalSec ?? DEFAULT_PULL_INTERVAL_SEC;
const redact = opts.pat ? (s: string) => redactSecretsInText(s, new Map([['github_pat', opts.pat!]])) : (s: string) => s;
const log = (l: string) => opts.logger?.(redact(l));
src/core/budget/budget-tracker.ts
+1 -1
View File
@@ -332,7 +332,7 @@ export class BudgetTracker {
// pricing we can't enforce the cap, and silently ignoring it would
// void the contract.
const msg = `${this.opts.label}: no pricing entry for model "${estimate.modelId}" (kind=${estimate.kind}). ` +
`Add it to src/core/${estimate.kind === 'embed' ? 'embedding-pricing.ts' : 'anthropic-pricing.ts'} or drop --max-cost.`;
`Add it to src/core/${estimate.kind === 'embed' || estimate.kind === 'rerank' ? 'embedding-pricing.ts' : 'anthropic-pricing.ts'} or drop --max-cost.`;
this.fireExhausted();
throw new BudgetExhausted(msg, {
reason: 'no_pricing',
src/core/check-resolvable.ts
+5 -17
View File
@@ -26,6 +26,7 @@ import {
findPrimaryResolverPath,
loadSkillTriggerIndex,
} from './skill-trigger-index.ts';
import { parseSkillFrontmatter } from './skill-frontmatter.ts';
// ---------------------------------------------------------------------------
// Types
@@ -218,26 +219,13 @@ export function parseResolverEntries(resolverContent: string): ResolverEntry[] {
// needed for AGENTS.md-only OpenClaw deployments. See D-CX-12 / F-ENG-1.
/**
* Simple YAML frontmatter parser — extracts triggers array if present.
* Extract the triggers array through the shared SKILL.md frontmatter parser.
*
* Normalizes CRLF → LF before parsing so Windows checkouts (where
* `core.autocrlf=true` is the default) parse correctly. Without this,
* the `^---\n` and `^triggers:\s*\n` regexes never match because the
* file content is `---\r\n` / `triggers:\r\n`, and every skill on
* Windows is reported as `mece_gap` regardless of its actual content.
* CI runs on Ubuntu-only so the bug only surfaces in user environments.
* Keeping this compatibility export routed through `parseSkillFrontmatter`
* prevents doctor gap detection from drifting from the trigger index.
*/
export function extractTriggers(skillContent: string): string[] {
const content = skillContent.replace(/\r\n/g, '\n');
const fmMatch = content.match(/^---\n([\s\S]*?)\n---/);
if (!fmMatch) return [];
const fm = fmMatch[1];
const triggersMatch = fm.match(/^triggers:\s*\n((?:\s+-\s+.+\n?)*)/m);
if (!triggersMatch) return [];
return triggersMatch[1]
.split('\n')
.map(l => l.replace(/^\s+-\s+/, '').replace(/^["']|["']$/g, '').trim())
.filter(Boolean);
return parseSkillFrontmatter(skillContent)?.triggers ?? [];
}
/**
src/core/cli-flag-registry.generated.ts
+58 -58
View File
@@ -8,102 +8,102 @@
// (help-text mentions count): accepting an ignored flag is the pre-#2185
// status quo; missing a real one breaks working invocations.
export const CLI_FLAG_REGISTRY: Record<string, readonly string[]> = {
'advisor': ['--aliases', '--all', '--apply', '--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--skills-dir', '--source', '--stale', '--supersessions', '--surface', '--thin', '--verbose', '--workspace', '--yes'],
'agent': ['--aliases', '--all', '--brain', '--detach', '--fanout-manifest', '--flag', '--flags', '--follow', '--help', '--include-null-signature', '--json', '--max-turns', '--model', '--no-extract', '--no-follow', '--note', '--pattern', '--pending', '--reset', '--resolve', '--since', '--source', '--stale', '--subagent-def', '--supersessions', '--thin', '--timeout-ms', '--tools', '--word'],
'anomalies': ['--aliases', '--all', '--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--lookback-days', '--mcp-only', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--sigma', '--since', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout'],
'apply-migrations': ['--ab', '--all', '--auto-update', '--brain', '--break-lock', '--build-index', '--by-mention', '--compile', '--days', '--dry-run', '--exclusive', '--fast', '--force', '--force-all', '--force-orchestrator', '--force-retry', '--force-schema', '--from-meetings', '--from-pages', '--help', '--history', '--host-dir', '--http', '--json', '--lang', '--list', '--locks', '--markdown', '--max-age', '--migrate-only', '--migration', '--mode', '--multimodal', '--no-autopilot-install', '--no-embedding', '--no-extract', '--non-interactive', '--phase', '--priority', '--refresh-unqualified', '--remediate', '--rollback', '--skip-verify', '--source', '--stale', '--surface', '--undo-wave', '--use-captured-snapshot', '--with-calibration', '--yes'],
'auth': ['--aliases', '--all', '--bound-brain', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--enable-dcr', '--enable-dcr-insecure', '--fast', '--federated-read', '--force', '--from-pages', '--grant-types', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--pattern', '--pending', '--redirect-uri', '--reset', '--resolve', '--scopes', '--source', '--stale', '--supersessions', '--surface', '--takes-holders', '--thin', '--token', '--token-endpoint-auth-method', '--yes'],
'autopilot': ['--aliases', '--all', '--auto-fix', '--batch', '--brain', '--break-lock', '--by-type', '--check', '--dimensions', '--dir', '--drain', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--ff-only', '--fix', '--force', '--force-break-lock', '--force-retry', '--from-pages', '--help', '--http', '--include-null-signature', '--inject-bootstrap', '--inline', '--input', '--install', '--interval', '--json', '--markdown', '--max-age', '--max-rss', '--max-usd', '--migrate-only', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--no-extract', '--no-inject', '--no-mutate', '--no-worker', '--non-interactive', '--now', '--once', '--output', '--path', '--pattern', '--pending', '--phase', '--refresh-cache', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--source', '--stale', '--status', '--supersessions', '--surface', '--swap-only', '--target', '--target-score', '--thin', '--timeout', '--to', '--uninstall', '--unsafe-bypass-dream-guard', '--user', '--version', '--yes'],
'advisor': ['--aliases', '--all', '--apply', '--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--skills-dir', '--source', '--stale', '--supersessions', '--surface', '--thin', '--token-ttl', '--verbose', '--workspace', '--yes'],
'agent': ['--aliases', '--all', '--brain', '--detach', '--fanout-manifest', '--federated', '--flag', '--flags', '--follow', '--help', '--include-null-signature', '--json', '--max-turns', '--model', '--no-extract', '--no-federated', '--no-follow', '--note', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--since', '--source', '--stale', '--subagent-def', '--supersessions', '--thin', '--timeout-ms', '--tools', '--word'],
'anomalies': ['--aliases', '--all', '--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--lookback-days', '--mcp-only', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--sigma', '--since', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl'],
'apply-migrations': ['--ab', '--all', '--auto-update', '--brain', '--break-lock', '--build-index', '--by-mention', '--compile', '--days', '--dry-run', '--exclusive', '--fast', '--force', '--force-all', '--force-orchestrator', '--force-retry', '--force-schema', '--from-meetings', '--from-pages', '--help', '--history', '--host-dir', '--http', '--json', '--lang', '--list', '--locks', '--markdown', '--max-age', '--migrate-only', '--migration', '--mode', '--multimodal', '--no-autopilot-install', '--no-embedding', '--no-extract', '--non-interactive', '--phase', '--priority', '--refresh-unqualified', '--remediate', '--rollback', '--skip-verify', '--source', '--stale', '--surface', '--token-ttl', '--undo-wave', '--use-captured-snapshot', '--with-calibration', '--yes'],
'auth': ['--aliases', '--all', '--bound-brain', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--enable-dcr', '--enable-dcr-insecure', '--fast', '--federated-read', '--force', '--from-pages', '--grant-types', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--pattern', '--pending', '--redirect-uri', '--reset', '--resolve', '--scopes', '--source', '--stale', '--supersessions', '--surface', '--takes-holders', '--thin', '--token', '--token-endpoint-auth-method', '--token-ttl', '--yes'],
'autopilot': ['--aliases', '--all', '--auto-fix', '--batch', '--brain', '--break-lock', '--by-type', '--check', '--dim', '--dimensions', '--dir', '--drain', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--ff-only', '--fix', '--force', '--force-break-lock', '--force-retry', '--from-pages', '--help', '--http', '--include-null-signature', '--inject-bootstrap', '--inline', '--input', '--install', '--interval', '--json', '--markdown', '--max-age', '--max-rss', '--max-usd', '--migrate-only', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-inject', '--no-mutate', '--no-worker', '--non-interactive', '--now', '--once', '--output', '--path', '--pattern', '--pending', '--phase', '--refresh-cache', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--source', '--stale', '--status', '--supersessions', '--surface', '--swap-only', '--target', '--target-score', '--thin', '--timeout', '--to', '--token-ttl', '--uninstall', '--unsafe-bypass-dream-guard', '--user', '--version', '--yes'],
'backfill': ['--aliases', '--all', '--batch-size', '--brain', '--concurrency', '--dry-run', '--fresh', '--help', '--include-null-signature', '--json', '--keep-index', '--list', '--max-errors', '--max-rows', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--resume', '--source', '--stale', '--supersessions', '--thin'],
'bench': ['--baseline', '--brain', '--explain', '--force', '--from', '--help', '--json', '--label', '--lang', '--limit', '--markdown', '--multimodal', '--near-symbol', '--restore-only', '--source', '--stale', '--symbol-kind', '--thin', '--threshold-jaccard', '--threshold-latency-multiplier', '--threshold-top1', '--to', '--tool'],
'book-mirror': ['--aliases', '--all', '--allow-empty', '--apply', '--asof', '--author', '--auto', '--background', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--brain-wide-max-cost-usd', '--budget-usd-per-day', '--by-mention', '--chapters-dir', '--content', '--context-file', '--date', '--days', '--dry-run', '--entities', '--explain', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--help', '--http', '--image', '--include-null-signature', '--json', '--kind', '--limit', '--max-turns', '--max-usd', '--mode', '--model', '--multimodal', '--no-confirm', '--no-embedding', '--no-extract', '--no-follow', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--save', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--timeout', '--timeout-ms', '--title', '--trusted-extraction', '--url', '--with-db', '--yes'],
'bootstrap': ['--abbrev-ref', '--abort', '--accept-visibility-change-consequences', '--all', '--brain', '--branch', '--cached', '--compile', '--confirm', '--delete-brain', '--env', '--exclude-standard', '--fast', '--file', '--flag', '--force', '--from-pages', '--full', '--gbrain-bin', '--get', '--git-dir', '--harness', '--heads', '--help', '--home', '--http', '--init', '--isolated', '--jq', '--json', '--local', '--minimal', '--name-only', '--no-cron', '--no-embedding', '--no-hooks', '--no-verify', '--once', '--only', '--others', '--pat-file', '--path', '--pglite', '--porcelain', '--private', '--push', '--push-only', '--quiet', '--rebase', '--repair', '--scope', '--set', '--short', '--show', '--show-toplevel', '--skip', '--source', '--status', '--surface', '--unset-all', '--verify', '--version', '--visibility', '--workspace', '--yes'],
'brainstorm': ['--aliases', '--all', '--brain', '--chunker-debug', '--code', '--compile', '--fast', '--fix', '--force', '--force-rechunk', '--force-resume', '--from-pages', '--full', '--help', '--http', '--include-null-signature', '--json', '--judge-model', '--lang', '--limit', '--list-runs', '--markdown', '--max-cost', '--max-far-set', '--max-ideas-per-judge-call', '--model', '--no-embed', '--no-embedding', '--no-extract', '--no-save', '--pattern', '--pending', '--reset', '--resolve', '--resume', '--retry-failed', '--retry-judge', '--save', '--source', '--stale', '--strict-budget', '--supersessions', '--surface', '--thin', '--timeout', '--yes'],
'cache': ['--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--no-embedding', '--source', '--surface', '--yes'],
'calibration': ['--ab', '--aliases', '--all', '--allow-empty', '--apply', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--dry-run', '--entities', '--explain', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--help', '--holder', '--http', '--image', '--include-null-signature', '--json', '--key-prefix', '--kind', '--lang', '--limit', '--markdown', '--max-usd', '--mode', '--multimodal', '--near-symbol', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--phase', '--progress-interval', '--progress-json', '--quiet', '--regenerate', '--repo', '--reset', '--resolve', '--restore-only', '--save', '--scrub-gstack', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--symbol-kind', '--thin', '--trusted-extraction', '--undo-wave', '--url', '--with-calibration', '--with-db', '--yes'],
'call': ['--aliases', '--all', '--all-sources', '--as-context', '--auto-fix', '--background', '--brain', '--by-mention', '--catch-up', '--concurrency', '--confirm-destructive', '--content', '--cost-estimate', '--count', '--days', '--depth', '--dim', '--dir', '--direction', '--enable-dcr', '--enable-dcr-insecure', '--explain', '--fast', '--federated', '--file', '--fix', '--follow', '--force', '--from', '--from-meetings', '--grant-types', '--grep', '--hard-deadline', '--help', '--http', '--image', '--include-expired', '--include-frontmatter', '--include-null-signature', '--infer-dates', '--install', '--interval', '--json', '--key', '--kind', '--lang', '--limit', '--link-source', '--link-type', '--llm', '--migrate-only', '--missing-path', '--multimodal', '--ner', '--no-embed', '--no-expand', '--no-extract', '--no-federated', '--no-hard-deadline', '--no-retry-connect', '--no-save', '--older-than', '--page', '--param', '--params', '--password', '--path', '--pattern', '--pending', '--pglite', '--port', '--progress-interval', '--progress-json', '--public-url', '--queue', '--quiet', '--reenrich-after', '--refresh-cache', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--restore-only', '--save', '--scopes', '--session', '--sigma', '--since', '--slug', '--slug-prefix', '--source', '--source-id', '--stale', '--status', '--stdin', '--strategy', '--suites', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--synthesize', '--tag', '--thin', '--timeout', '--to', '--today', '--token', '--token-ttl', '--tools-json', '--type', '--uninstall', '--url', '--version', '--watch', '--with-calibration', '--workers', '--yes'],
'capture': ['--aliases', '--all', '--allow-empty', '--apply', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--depth', '--entities', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--help', '--http', '--image', '--include-null-signature', '--json', '--kind', '--limit', '--max-usd', '--mcp-only', '--mode', '--multimodal', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--repo', '--reset', '--resolve', '--restore-only', '--save', '--scopes', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--stdin', '--supersessions', '--surface', '--thin', '--timeout', '--trusted-extraction', '--type', '--url', '--what', '--where', '--who', '--with-db', '--yes'],
'book-mirror': ['--aliases', '--all', '--allow-empty', '--apply', '--asof', '--author', '--auto', '--background', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--brain-wide-max-cost-usd', '--budget-usd-per-day', '--by-mention', '--chapters-dir', '--content', '--context-file', '--date', '--days', '--dry-run', '--entities', '--explain', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--help', '--http', '--image', '--include-null-signature', '--json', '--kind', '--limit', '--max-turns', '--max-usd', '--mode', '--model', '--multimodal', '--no-confirm', '--no-embedding', '--no-extract', '--no-follow', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--save', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--timeout', '--timeout-ms', '--title', '--token-ttl', '--trusted-extraction', '--url', '--with-db', '--yes'],
'bootstrap': ['--abbrev-ref', '--abort', '--accept-visibility-change-consequences', '--active', '--all', '--allow-unverified-remote', '--brain', '--branch', '--cached', '--compile', '--confirm', '--count', '--delete-brain', '--diff-filter', '--env', '--error-unmatch', '--exclude-standard', '--fast', '--file', '--flag', '--force', '--from-pages', '--full', '--gbrain-bin', '--get', '--git-dir', '--git-path', '--harness', '--heads', '--help', '--home', '--hostname', '--http', '--init', '--is-inside-work-tree', '--isolated', '--jq', '--json', '--local', '--minimal', '--name-only', '--no-cron', '--no-embedding', '--no-hooks', '--no-verify', '--once', '--only', '--others', '--pat-file', '--path', '--pglite', '--porcelain', '--private', '--push', '--push-only', '--quiet', '--rebase', '--repair', '--scope', '--set', '--short', '--show', '--show-toplevel', '--skip', '--source', '--status', '--surface', '--token-ttl', '--unset-all', '--verify', '--version', '--visibility', '--workspace', '--yes'],
'brainstorm': ['--aliases', '--all', '--brain', '--chunker-debug', '--code', '--compile', '--fast', '--file', '--fix', '--force', '--force-rechunk', '--force-resume', '--from-pages', '--full', '--help', '--http', '--include-null-signature', '--json', '--judge-model', '--lang', '--limit', '--list-runs', '--markdown', '--max-cost', '--max-far-set', '--max-ideas-per-judge-call', '--model', '--no-embed', '--no-embedding', '--no-extract', '--no-save', '--pattern', '--pending', '--reset', '--resolve', '--resume', '--retry-failed', '--retry-judge', '--save', '--source', '--stale', '--strict-budget', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl', '--yes'],
'cache': ['--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--no-embedding', '--source', '--surface', '--token-ttl', '--yes'],
'calibration': ['--ab', '--aliases', '--all', '--allow-empty', '--apply', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--dry-run', '--entities', '--explain', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--help', '--holder', '--http', '--image', '--include-null-signature', '--json', '--key-prefix', '--kind', '--lang', '--limit', '--markdown', '--max-usd', '--mode', '--multimodal', '--near-symbol', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--phase', '--progress-interval', '--progress-json', '--quiet', '--regenerate', '--repo', '--reset', '--resolve', '--restore-only', '--save', '--scrub-gstack', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--symbol-kind', '--thin', '--token-ttl', '--trusted-extraction', '--undo-wave', '--url', '--with-calibration', '--with-db', '--yes'],
'call': ['--aliases', '--all', '--all-sources', '--as-context', '--auto-fix', '--background', '--brain', '--by-mention', '--catch-up', '--concurrency', '--confirm-destructive', '--content', '--cost-estimate', '--count', '--days', '--depth', '--dim', '--dir', '--direction', '--enable-dcr', '--enable-dcr-insecure', '--explain', '--fast', '--federated', '--file', '--fix', '--follow', '--force', '--from', '--from-meetings', '--grant-types', '--grep', '--hard-deadline', '--help', '--http', '--image', '--include-expired', '--include-frontmatter', '--include-null-signature', '--infer-dates', '--install', '--interval', '--json', '--key', '--kind', '--lang', '--limit', '--link-source', '--link-type', '--llm', '--migrate-only', '--missing-path', '--multimodal', '--ner', '--no-embed', '--no-expand', '--no-extract', '--no-federated', '--no-hard-deadline', '--no-retry-connect', '--no-save', '--older-than', '--page', '--param', '--params', '--password', '--path', '--pattern', '--pending', '--pglite', '--port', '--probe-pglite', '--progress-interval', '--progress-json', '--public-url', '--queue', '--quiet', '--reenrich-after', '--refresh-cache', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--restore-only', '--save', '--scopes', '--session', '--sigma', '--since', '--slug', '--slug-prefix', '--source', '--source-id', '--stale', '--status', '--stdin', '--strategy', '--suites', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--synthesize', '--tag', '--thin', '--timeout', '--to', '--today', '--token', '--token-ttl', '--tools-json', '--type', '--uninstall', '--url', '--version', '--watch', '--with-calibration', '--workers', '--yes'],
'capture': ['--aliases', '--all', '--allow-empty', '--apply', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--depth', '--entities', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--help', '--http', '--image', '--include-null-signature', '--json', '--kind', '--limit', '--max-usd', '--mcp-only', '--mode', '--multimodal', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--repo', '--reset', '--resolve', '--restore-only', '--save', '--scopes', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--stdin', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl', '--trusted-extraction', '--type', '--url', '--what', '--where', '--who', '--with-db', '--yes'],
'check-backlinks': ['--background', '--brain', '--brain-wide-max-cost-usd', '--dir', '--dry-run', '--explain', '--follow', '--help', '--include-frontmatter', '--json', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--source', '--stale', '--timeout', '--type'],
'check-resolvable': ['--brain', '--dry-run', '--fix', '--help', '--json', '--skills-dir', '--source', '--strict', '--verbose'],
'check-update': ['--all', '--brain', '--check', '--ff-only', '--help', '--json', '--markdown', '--migrate-only', '--non-interactive', '--refresh-cache', '--source', '--swap-only', '--to', '--version', '--yes'],
'check-update': ['--all', '--brain', '--check', '--dim', '--ff-only', '--help', '--json', '--markdown', '--migrate-only', '--non-interactive', '--refresh-cache', '--source', '--swap-only', '--to', '--version', '--yes'],
'claw-test': ['--ab', '--agent', '--all', '--auto-update', '--brain', '--break-lock', '--build-index', '--by-mention', '--compile', '--days', '--dir', '--exclusive', '--force-retry', '--force-schema', '--from-meetings', '--help', '--history', '--http', '--json', '--keep-tempdir', '--lang', '--list-agents', '--live', '--local', '--locks', '--markdown', '--max-age', '--message', '--multimodal', '--no-embed', '--no-embedding', '--no-extract', '--path', '--pglite', '--phase', '--priority', '--progress-json', '--prompt-file', '--refresh-unqualified', '--remediate', '--rollback', '--run-id', '--scenario', '--skip-verify', '--source', '--stale', '--transcripts', '--undo-wave', '--use-captured-snapshot', '--with-calibration', '--yes'],
'code-callees': ['--aliases', '--all', '--all-sources', '--brain', '--chunker-debug', '--clone-dir', '--confirm-destructive', '--federated', '--force', '--help', '--include-null-signature', '--json', '--limit', '--no-extract', '--no-federated', '--no-json', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--thin', '--url', '--url-managed', '--yes'],
'code-callers': ['--aliases', '--all', '--all-sources', '--brain', '--chunker-debug', '--clone-dir', '--confirm-destructive', '--federated', '--force', '--help', '--include-null-signature', '--json', '--limit', '--no-extract', '--no-federated', '--no-json', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--thin', '--url', '--url-managed', '--yes'],
'code-def': ['--aliases', '--all', '--brain', '--chunker-debug', '--help', '--include-null-signature', '--json', '--lang', '--limit', '--no-extract', '--no-json', '--pattern', '--pending', '--pretty', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--yes'],
'code-refs': ['--aliases', '--all', '--brain', '--chunker-debug', '--help', '--include-null-signature', '--json', '--lang', '--limit', '--no-extract', '--no-json', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--yes'],
'config': ['--aliases', '--all', '--brain', '--column', '--coverage-override', '--embedding-dimensions', '--embedding-model', '--fast', '--federated-read', '--follow', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--markdown', '--model', '--multimodal', '--no-embedding', '--no-extract', '--pattern', '--pending', '--pglite', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--surface', '--thin', '--yes'],
'config': ['--aliases', '--all', '--brain', '--column', '--coverage-override', '--detail', '--embedding-dimensions', '--embedding-model', '--fast', '--federated-read', '--follow', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--markdown', '--model', '--multimodal', '--no-embedding', '--no-extract', '--no-federated', '--pattern', '--pending', '--pglite', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--surface', '--thin', '--token-ttl', '--yes'],
'connect': ['--agent', '--bearer-token-env-var', '--bind', '--brain', '--client-id', '--client-secret', '--force', '--grant-types', '--help', '--http', '--install', '--json', '--name', '--oauth', '--public-url', '--register', '--scopes', '--show-token', '--source', '--timeout-ms', '--token', '--token-endpoint-auth-method', '--url', '--version', '--yes'],
'conversation-parser': ['--aliases', '--all', '--brain', '--help', '--include-null-signature', '--json', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin'],
'doctor': ['--ab', '--abi', '--aliases', '--all', '--allow-shell-jobs', '--auto', '--auto-fix', '--auto-update', '--background', '--batch', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--build-index', '--by-mention', '--by-type', '--check', '--column', '--compile', '--concurrency', '--confidence', '--confirm', '--content-audit', '--count', '--days', '--delete-brain', '--detach', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--exclusive', '--explain', '--fast', '--fix', '--follow', '--force', '--force-break-lock', '--force-retry', '--force-schema', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--grant-types', '--harness', '--health-interval', '--help', '--history', '--home', '--http', '--include-flagged', '--include-frontmatter', '--include-null-signature', '--include-pseudo', '--index-audit', '--init', '--input', '--jq', '--json', '--lang', '--limit', '--locks', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-crashes', '--max-jobs', '--max-rss', '--max-usd', '--mcp-only', '--migrate-only', '--model', '--multimodal', '--name-only', '--name-status', '--near-symbol', '--nice', '--no', '--no-embed', '--no-embedding', '--no-extract', '--no-mutate', '--oauth-client-secret', '--older-than', '--once', '--overwrite', '--parallel', '--params', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--priority', '--progress-interval', '--progress-json', '--query', '--queue', '--quiet', '--rebuild-rollup', '--refresh', '--refresh-unqualified', '--regenerate', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--restore-only', '--resume', '--review-lower', '--rollback', '--scope', '--scopes', '--set', '--show-toplevel', '--since', '--skills-dir', '--skip-bare-tweet', '--skip-failed', '--skip-urls', '--skip-verify', '--slugs', '--source', '--source-id', '--stale', '--stats', '--status', '--strategy', '--strict', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--target', '--target-score', '--thin', '--timeout', '--to', '--top-k', '--type', '--undo-wave', '--unsafe-bypass-dream-guard', '--untracked-files', '--url', '--use-captured-snapshot', '--verbose', '--verify', '--version', '--window', '--with-calibration', '--workers', '--yes'],
'dream': ['--against', '--aliases', '--all', '--allow-regression', '--anchor', '--asof', '--background', '--batch', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--budget-usd', '--budget-usd-answer', '--budget-usd-retrieval', '--by-type', '--by-type-floor', '--code', '--committed-baseline', '--compare', '--compile', '--concurrent', '--ctx-size', '--cycles', '--date', '--dimensions', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--embeddings', '--expansion', '--explain', '--fast', '--federated', '--fix', '--fixtures', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--force-retry', '--format', '--from', '--from-db', '--from-pages', '--gold', '--harness', '--help', '--http', '--include-holdout', '--include-null-signature', '--input', '--install', '--json', '--judge-model', '--justification', '--keyword-only', '--lang', '--limit', '--llm', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--max-tokens', '--max-usd', '--mcp-only', '--min-recall', '--mode', '--model', '--models', '--modes', '--multimodal', '--name', '--name-only', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-llm', '--no-mutate', '--no-trajectory', '--once', '--out', '--output', '--output-dir', '--parallel', '--path', '--pattern', '--pending', '--pglite', '--phase', '--priority', '--progress-interval', '--progress-json', '--pull', '--quiet', '--receipt-dir', '--remediate', '--repo', '--reranking', '--reset', '--resolve', '--restore-only', '--resume-from', '--retrieval-only', '--rounds', '--rubric-version', '--save', '--seed', '--short', '--show-toplevel', '--since', '--skip-replay', '--slot-a-model', '--slot-b-model', '--slot-c-model', '--slug', '--slug-prefix', '--source', '--source-id', '--stale', '--suite', '--suites', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--take', '--task', '--thin', '--threshold', '--timeout', '--to', '--top-k', '--undo', '--unsafe-bypass-dream-guard', '--update-baseline', '--verify', '--version', '--window', '--yes'],
'doctor': ['--ab', '--abbrev-ref', '--abi', '--abort', '--aliases', '--all', '--allow-shell-jobs', '--allow-unverified-remote', '--auto', '--auto-fix', '--auto-update', '--background', '--batch', '--brain', '--brain-wide-max-cost-usd', '--branch', '--break-lock', '--build-index', '--by-mention', '--by-type', '--cached', '--check', '--column', '--compile', '--concurrency', '--confidence', '--confirm', '--content-audit', '--count', '--days', '--delete-brain', '--detach', '--detail', '--diff-filter', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--exclude-standard', '--exclusive', '--explain', '--fast', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--force-retry', '--force-schema', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--get', '--git-dir', '--git-path', '--grant-types', '--harness', '--health-interval', '--help', '--history', '--home', '--http', '--include-flagged', '--include-frontmatter', '--include-null-signature', '--include-pseudo', '--index-audit', '--init', '--input', '--is-inside-work-tree', '--jq', '--json', '--lang', '--limit', '--local', '--locks', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-crashes', '--max-jobs', '--max-rss', '--max-usd', '--mcp-only', '--migrate-only', '--model', '--multimodal', '--name-only', '--name-status', '--near-symbol', '--nice', '--no', '--no-cron', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-mutate', '--no-verify', '--oauth-client-secret', '--older-than', '--once', '--others', '--overwrite', '--parallel', '--params', '--pat-file', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--priority', '--probe-pglite', '--progress-interval', '--progress-json', '--push-only', '--query', '--queue', '--quiet', '--rebase', '--rebuild-rollup', '--refresh', '--refresh-unqualified', '--regenerate', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--restore-only', '--resume', '--review-lower', '--rollback', '--scope', '--scopes', '--set', '--short', '--show-current', '--show-toplevel', '--since', '--skills-dir', '--skip-bare-tweet', '--skip-failed', '--skip-urls', '--skip-verify', '--slugs', '--source', '--source-id', '--stale', '--stats', '--status', '--strategy', '--strict', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--target', '--target-score', '--thin', '--timeout', '--to', '--token-ttl', '--top-k', '--type', '--undo-wave', '--unsafe-bypass-dream-guard', '--unset-all', '--untracked-files', '--url', '--use-captured-snapshot', '--verbose', '--verify', '--version', '--window', '--with-calibration', '--workers', '--yes'],
'dream': ['--against', '--aliases', '--all', '--allow-regression', '--anchor', '--asof', '--background', '--batch', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--budget-usd', '--budget-usd-answer', '--budget-usd-retrieval', '--by-type', '--by-type-floor', '--code', '--committed-baseline', '--compare', '--compile', '--concurrent', '--ctx-size', '--cycles', '--date', '--detail', '--dimensions', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--embeddings', '--expansion', '--explain', '--fast', '--federated', '--fix', '--fixtures', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--force-retry', '--format', '--from', '--from-db', '--from-pages', '--gold', '--harness', '--help', '--http', '--include-holdout', '--include-null-signature', '--input', '--install', '--json', '--judge-model', '--justification', '--keyword-only', '--lang', '--limit', '--llm', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--max-tokens', '--max-usd', '--mcp-only', '--min-recall', '--mode', '--model', '--models', '--modes', '--multimodal', '--name', '--name-only', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-llm', '--no-mutate', '--no-trajectory', '--once', '--out', '--output', '--output-dir', '--parallel', '--path', '--pattern', '--pending', '--pglite', '--phase', '--priority', '--progress-interval', '--progress-json', '--pull', '--quiet', '--receipt-dir', '--remediate', '--repo', '--reranking', '--reset', '--resolve', '--restore-only', '--resume-from', '--retrieval-only', '--rounds', '--rubric-version', '--save', '--seed', '--short', '--show-toplevel', '--since', '--skip-replay', '--slot-a-model', '--slot-b-model', '--slot-c-model', '--slug', '--slug-prefix', '--source', '--source-id', '--stale', '--suite', '--suites', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--take', '--task', '--thin', '--threshold', '--timeout', '--to', '--token-ttl', '--top-k', '--undo', '--unsafe-bypass-dream-guard', '--update-baseline', '--verify', '--version', '--window', '--yes'],
'edges-backfill': ['--aliases', '--all', '--all-sources', '--brain', '--concurrency', '--federated', '--help', '--include-null-signature', '--json', '--max-age', '--max-chunks', '--max-cost-usd', '--no-extract', '--no-federated', '--older-than', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--thin', '--timeout', '--workers'],
'embed': ['--aliases', '--all', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--catch-up', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--fix', '--follow', '--force', '--force-break-lock', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--lang', '--markdown', '--max-age', '--max-cost-usd', '--model', '--multimodal', '--name', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--pace', '--pace-max-concurrency', '--parallel', '--path', '--pattern', '--pending', '--pglite', '--prefix', '--priority', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--restore-only', '--serial', '--slugs', '--source', '--stale', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--thin', '--timeout', '--version'],
'enrich': ['--aliases', '--all', '--all-sources', '--allow-empty', '--apply', '--asof', '--auto', '--background', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--budget-usd-per-day', '--by-mention', '--clone-dir', '--code', '--concurrency', '--confirm-destructive', '--content', '--date', '--days', '--dry-run', '--embedding-dimensions', '--embedding-model', '--entities', '--explain', '--fast', '--federated', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--from-pages', '--help', '--http', '--image', '--include-null-signature', '--json', '--judge-model', '--kind', '--lang', '--limit', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--max-usd', '--min-context', '--mode', '--model', '--multimodal', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--offset', '--older-than', '--order', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--reenrich-after', '--remediate', '--reset', '--resolve', '--restore-only', '--resume', '--save', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--source-id', '--stale', '--stats', '--supersessions', '--surface', '--symbol-kind', '--thin', '--thin-threshold', '--timeout', '--trusted-extraction', '--types', '--url', '--url-managed', '--version', '--with-db', '--workers', '--yes'],
'eval': ['--ab-relational', '--against', '--aliases', '--all', '--allow-regression', '--background', '--baseline', '--batch', '--brain', '--brain-wide-max-cost-usd', '--budget-usd', '--budget-usd-answer', '--budget-usd-retrieval', '--committed-baseline', '--compare', '--compare-limit', '--concurrent', '--config-a', '--config-b', '--corpus', '--cycles', '--days', '--dedup-cosine', '--dedup-max-per-page', '--dedup-type-ratio', '--dimensions', '--distance-min', '--embedding-dimensions', '--embedding-model', '--expand', '--explain', '--fast', '--fixtures', '--follow', '--force', '--from-capture', '--from-db', '--from-pages', '--gold', '--grounding-min', '--harness', '--help', '--http', '--include-holdout', '--include-null-signature', '--input', '--json', '--judge', '--justification', '--k', '--limit', '--llm', '--max-pair-chars', '--max-tokens', '--max-usd', '--md', '--metric', '--min-recall', '--mode', '--model', '--models', '--modes', '--multimodal', '--name', '--no', '--no-cache', '--no-embed', '--no-embedding', '--no-expand', '--no-extract', '--no-llm', '--older-than', '--out', '--output', '--output-dir', '--parallel', '--pattern', '--pending', '--progress-interval', '--progress-json', '--qrels', '--queries-file', '--query', '--questions', '--quiet', '--receipt-dir', '--refresh-cache', '--remediate', '--reset', '--resolve', '--rrf-k', '--rubric-version', '--runs', '--sampling', '--save', '--seed', '--severity', '--short', '--show-toplevel', '--since', '--skip-replay', '--slot-a-model', '--slot-b-model', '--slot-c-model', '--slug', '--slug-prefix', '--source', '--stale', '--strategy', '--strict', '--suite', '--suites', '--supersessions', '--surface', '--task', '--thin', '--threshold', '--threshold-expected-top1', '--threshold-first-relevant-hit', '--threshold-jaccard', '--threshold-latency-multiplier', '--threshold-recall-at-k', '--threshold-top1', '--timeout', '--tool', '--top-k', '--top-regressions', '--until', '--update-baseline', '--usefulness-min', '--verbose', '--version', '--with-code-intel', '--yes'],
'embed': ['--aliases', '--all', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--catch-up', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--fix', '--follow', '--force', '--force-break-lock', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--lang', '--markdown', '--max-age', '--max-cost-usd', '--model', '--multimodal', '--name', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--pace', '--pace-max-concurrency', '--parallel', '--path', '--pattern', '--pending', '--pglite', '--prefix', '--priority', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--restore-only', '--serial', '--slugs', '--source', '--stale', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--thin', '--timeout', '--token-ttl', '--version'],
'enrich': ['--aliases', '--all', '--all-sources', '--allow-empty', '--apply', '--asof', '--auto', '--background', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--budget-usd-per-day', '--by-mention', '--clone-dir', '--code', '--concurrency', '--confirm-destructive', '--content', '--date', '--days', '--detail', '--dry-run', '--embedding-dimensions', '--embedding-model', '--entities', '--explain', '--fast', '--federated', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--from-pages', '--help', '--http', '--image', '--include-null-signature', '--json', '--judge-model', '--kind', '--lang', '--limit', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--max-usd', '--min-context', '--mode', '--model', '--multimodal', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--offset', '--older-than', '--order', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--reenrich-after', '--remediate', '--reset', '--resolve', '--restore-only', '--resume', '--save', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--source-id', '--stale', '--stats', '--supersessions', '--surface', '--symbol-kind', '--thin', '--thin-threshold', '--timeout', '--token-ttl', '--trusted-extraction', '--types', '--url', '--url-managed', '--version', '--with-db', '--workers', '--yes'],
'eval': ['--ab-relational', '--against', '--aliases', '--all', '--allow-regression', '--background', '--baseline', '--batch', '--brain', '--brain-wide-max-cost-usd', '--budget-usd', '--budget-usd-answer', '--budget-usd-retrieval', '--committed-baseline', '--compare', '--compare-limit', '--concurrent', '--config-a', '--config-b', '--corpus', '--cycles', '--days', '--dedup-cosine', '--dedup-max-per-page', '--dedup-type-ratio', '--dimensions', '--distance-min', '--embedding-dimensions', '--embedding-model', '--expand', '--explain', '--fast', '--fixtures', '--follow', '--force', '--from-capture', '--from-db', '--from-pages', '--gold', '--grounding-min', '--harness', '--help', '--http', '--include-holdout', '--include-null-signature', '--input', '--json', '--judge', '--justification', '--k', '--limit', '--llm', '--max-pair-chars', '--max-tokens', '--max-usd', '--md', '--metric', '--min-recall', '--mode', '--model', '--models', '--modes', '--multimodal', '--name', '--no', '--no-cache', '--no-embed', '--no-embedding', '--no-expand', '--no-extract', '--no-llm', '--older-than', '--out', '--output', '--output-dir', '--parallel', '--pattern', '--pending', '--progress-interval', '--progress-json', '--qrels', '--queries-file', '--query', '--questions', '--quiet', '--receipt-dir', '--refresh-cache', '--remediate', '--reset', '--resolve', '--rrf-k', '--rubric-version', '--runs', '--sampling', '--save', '--seed', '--severity', '--short', '--show-toplevel', '--since', '--skip-replay', '--slot-a-model', '--slot-b-model', '--slot-c-model', '--slug', '--slug-prefix', '--source', '--stale', '--strategy', '--strict', '--suite', '--suites', '--supersessions', '--surface', '--task', '--thin', '--threshold', '--threshold-expected-top1', '--threshold-first-relevant-hit', '--threshold-jaccard', '--threshold-latency-multiplier', '--threshold-recall-at-k', '--threshold-top1', '--timeout', '--token-ttl', '--tool', '--top-k', '--top-regressions', '--until', '--update-baseline', '--usefulness-min', '--verbose', '--version', '--with-code-intel', '--yes'],
'export': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dir', '--explain', '--federated', '--fix', '--follow', '--help', '--include-null-signature', '--json', '--lang', '--markdown', '--multimodal', '--near-symbol', '--no-extract', '--no-federated', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--repo', '--reset', '--resolve', '--restore-only', '--slug-prefix', '--source', '--stale', '--supersessions', '--symbol-kind', '--thin', '--timeout', '--type'],
'extract': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--by-mention', '--catch-up', '--code', '--concurrency', '--dir', '--dry-run', '--explain', '--federated', '--follow', '--from-meetings', '--help', '--include-frontmatter', '--include-null-signature', '--infer-dates', '--json', '--kind', '--lang', '--markdown', '--max-age', '--max-cost-usd', '--multimodal', '--name-status', '--near-symbol', '--ner', '--no-extract', '--no-federated', '--older-than', '--pack', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--repo', '--reset', '--resolve', '--restore-only', '--run-id', '--since', '--slug', '--source', '--source-id', '--stale', '--strategy', '--supersessions', '--symbol-kind', '--thin', '--timeout', '--type', '--verbose', '--workers', '--yes'],
'extract-conversation-facts': ['--aliases', '--all', '--all-sources', '--background', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--by-mention', '--clone-dir', '--code', '--concurrency', '--confirm-destructive', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fix', '--follow', '--force', '--force-break-lock', '--help', '--include-null-signature', '--json', '--judge-model', '--lang', '--limit', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--model', '--multimodal', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--older-than', '--override-disabled', '--path', '--pattern', '--pending', '--pglite', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--restore-only', '--segment-limit', '--session', '--since', '--sleep', '--slug', '--source', '--source-id', '--stale', '--supabase', '--supersessions', '--symbol-kind', '--thin', '--timeout', '--types', '--url', '--url-managed', '--version', '--workers', '--yes'],
'features': ['--aliases', '--all', '--auto-fix', '--background', '--batch-size', '--brain', '--by-mention', '--catch-up', '--concurrency', '--dir', '--explain', '--from-meetings', '--help', '--include-frontmatter', '--include-null-signature', '--infer-dates', '--json', '--kind', '--ner', '--no-extract', '--pace', '--pace-max-concurrency', '--pack', '--path', '--pattern', '--pending', '--priority', '--progress-json', '--quiet', '--repo', '--reset', '--resolve', '--run-id', '--since', '--slugs', '--source', '--source-id', '--stale', '--supersessions', '--thin', '--type', '--verbose', '--workers'],
'files': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--explain', '--fast', '--follow', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--no-pointer', '--page', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--retry-failed', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout', '--type', '--yes'],
'forget': ['--aliases', '--all', '--allow-empty', '--apply', '--as-context', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-tokens', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--entities', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--grep', '--help', '--http', '--image', '--include-expired', '--include-null-signature', '--json', '--kind', '--limit', '--max-usd', '--mcp-only', '--mode', '--multimodal', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--query', '--quiet', '--reason', '--repo', '--reset', '--resolve', '--restore-only', '--rollup', '--save', '--session', '--session-id', '--since', '--since-last-run', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--timeout', '--today', '--trusted-extraction', '--url', '--watch', '--with-db', '--yes'],
'founder': ['--aliases', '--all', '--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--mcp-only', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--since', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout', '--until'],
'files': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--explain', '--fast', '--follow', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--no-pointer', '--page', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--retry-failed', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl', '--type', '--yes'],
'forget': ['--aliases', '--all', '--allow-empty', '--apply', '--as-context', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-tokens', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--entities', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--grep', '--help', '--http', '--image', '--include-expired', '--include-null-signature', '--json', '--kind', '--limit', '--max-usd', '--mcp-only', '--mode', '--multimodal', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--query', '--quiet', '--reason', '--repo', '--reset', '--resolve', '--restore-only', '--rollup', '--save', '--session', '--session-id', '--since', '--since-last-run', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--timeout', '--today', '--token-ttl', '--trusted-extraction', '--url', '--watch', '--with-db', '--yes'],
'founder': ['--aliases', '--all', '--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--mcp-only', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--since', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl', '--until'],
'friction': ['--agent', '--base', '--brain', '--compare', '--help', '--hint', '--json', '--kind', '--message', '--no-redact', '--phase', '--redact', '--run-id', '--severity', '--source', '--transcript-path', '--transcripts'],
'frontmatter': ['--aliases', '--all', '--allow-catch-all', '--brain', '--cached', '--diff-filter', '--dry-run', '--exclude-standard', '--fast', '--fix', '--force', '--from-pages', '--get', '--help', '--http', '--include-catch-all', '--include-null-signature', '--json', '--name-only', '--name-status', '--no-embedding', '--no-extract', '--no-verify', '--others', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--strategy', '--supersessions', '--surface', '--thin', '--timeout', '--uninstall', '--write-back'],
'graph-query': ['--aliases', '--all', '--brain', '--depth', '--direction', '--explain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-foreign', '--include-null-signature', '--json', '--lang', '--markdown', '--mcp-only', '--multimodal', '--near-symbol', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--surface', '--symbol-kind', '--thin', '--timeout', '--type'],
'hook': ['--aliases', '--all', '--batch-limit', '--brain', '--budget-ms', '--count', '--delete-brain', '--detach', '--env', '--fast', '--force', '--from-pages', '--harness', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--once', '--path', '--pattern', '--pending', '--porcelain', '--reset', '--resolve', '--show-toplevel', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--timeout'],
'import': ['--aliases', '--all', '--asof', '--background', '--brain', '--brain-wide-max-cost-usd', '--by-mention', '--cached', '--code', '--compile', '--concurrency', '--embedding-dimensions', '--embedding-model', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--fix', '--follow', '--force', '--force-rechunk', '--fresh', '--from-pages', '--full', '--help', '--http', '--include-gitignored', '--include-null-signature', '--json', '--lang', '--markdown', '--max-age', '--multimodal', '--name-status', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--older-than', '--others', '--path', '--pattern', '--pending', '--pglite', '--priority', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--since', '--skip-failed', '--source', '--source-id', '--stale', '--strategy', '--supabase', '--supersessions', '--surface', '--thin', '--timeout', '--url', '--workers'],
'init': ['--all', '--brain', '--chat-model', '--check', '--ctx-size', '--embedding-dimensions', '--embedding-model', '--entity', '--expansion-model', '--fast', '--force', '--from-pages', '--grant-types', '--help', '--http', '--issuer-url', '--json', '--judge-model', '--key', '--mcp-only', '--mcp-url', '--migrate-only', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--non-interactive', '--oauth-client-id', '--oauth-client-secret', '--path', '--pglite', '--provenance', '--schema-pack', '--scopes', '--skip-embed-check', '--source', '--stale', '--supabase', '--surface', '--target', '--to', '--touchpoint', '--url', '--version'],
'integrations': ['--auto', '--brain', '--dry-run', '--embeddings', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--no-embedding', '--overwrite', '--refresh', '--reranking', '--source', '--surface', '--target'],
'integrity': ['--aliases', '--all', '--auto', '--backend', '--background', '--brain', '--brain-wide-max-cost-usd', '--check', '--confidence', '--cost', '--dry-run', '--explain', '--fast', '--follow', '--force', '--fresh', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--limit', '--no-embedding', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--review-lower', '--skip-bare-tweet', '--skip-urls', '--source', '--stale', '--supabase', '--supersessions', '--surface', '--thin', '--timeout', '--type', '--url'],
'jobs': ['--abbrev-ref', '--aliases', '--all', '--allow-empty', '--allow-protected', '--allow-shell-jobs', '--apply', '--asof', '--auto', '--auto-fix', '--auto-with-prompt', '--background', '--backoff-delay', '--backoff-jitter', '--backoff-type', '--batch', '--batch-size', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--break-lock', '--budget-usd', '--budget-usd-per-day', '--by-mention', '--by-type', '--cached', '--catch-up', '--check', '--cli-path', '--cluster', '--cluster-errors', '--code', '--concurrency', '--confidence', '--confirm-destructive', '--content', '--date', '--days', '--delay', '--detach', '--diff-filter', '--dimensions', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--empty', '--entities', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--federated-read', '--ff-only', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--force-retry', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--hard-deadline', '--health-interval', '--held-out', '--help', '--http', '--idempotency-key', '--image', '--include-frontmatter', '--include-gitignored', '--include-null-signature', '--infer-dates', '--inject-bootstrap', '--inline', '--input', '--install', '--interval', '--is-ancestor', '--json', '--kind', '--lang', '--limit', '--lock', '--markdown', '--max-age', '--max-attempts', '--max-cost-usd', '--max-crashes', '--max-rss', '--max-runtime-min', '--max-sources', '--max-stalled', '--max-usd', '--max-waiting', '--mcp-only', '--migrate-only', '--min-context', '--missing-path', '--mode', '--model', '--multimodal', '--name-only', '--name-status', '--near-symbol', '--ner', '--nice', '--no', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-federate', '--no-gpg-sign', '--no-hard-deadline', '--no-inject', '--no-mutate', '--no-pull', '--no-renames', '--no-schema-pack', '--no-verify', '--no-worker', '--non-interactive', '--now', '--offset', '--older-than', '--once', '--order', '--orphan', '--others', '--output', '--override-disabled', '--pace', '--pace-max-concurrency', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--phase', '--pid-file', '--priority', '--progress-interval', '--progress-json', '--queue', '--quiet', '--redact-secrets', '--reenrich-after', '--refresh-cache', '--refresh-ms', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--resume', '--retry-failed', '--review-lower', '--run-id', '--save', '--segment-limit', '--serial', '--session', '--session-id', '--short', '--show-toplevel', '--sigkill-rescue', '--since', '--skip-bare-tweet', '--skip-failed', '--skip-urls', '--sleep', '--slug', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--stats', '--status', '--strategy', '--supersessions', '--surface', '--swap-only', '--symbol-kind', '--target', '--target-score', '--thin', '--thin-threshold', '--timeout', '--timeout-ms', '--to', '--trusted-extraction', '--type', '--types', '--uninstall', '--unsafe-bypass-dream-guard', '--url', '--user', '--verbose', '--verify', '--version', '--watch', '--wedge-rescue', '--with-db', '--workers', '--yes'],
'lint': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--exclude', '--explain', '--fast', '--fix', '--follow', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout'],
'frontmatter': ['--aliases', '--all', '--allow-catch-all', '--brain', '--cached', '--diff-filter', '--dry-run', '--exclude-standard', '--fast', '--fix', '--force', '--from-pages', '--get', '--help', '--http', '--include-catch-all', '--include-null-signature', '--json', '--name-only', '--name-status', '--no-embedding', '--no-extract', '--no-verify', '--others', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--strategy', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl', '--uninstall', '--write-back'],
'graph-query': ['--aliases', '--all', '--brain', '--depth', '--direction', '--explain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-foreign', '--include-null-signature', '--json', '--lang', '--markdown', '--mcp-only', '--multimodal', '--near-symbol', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--surface', '--symbol-kind', '--thin', '--timeout', '--token-ttl', '--type'],
'hook': ['--aliases', '--all', '--allow-unverified-remote', '--batch-limit', '--brain', '--budget-ms', '--cached', '--count', '--delete-brain', '--detach', '--diff-filter', '--end-of-options', '--env', '--exclude-standard', '--fast', '--force', '--from-pages', '--get', '--harness', '--help', '--http', '--include-null-signature', '--jq', '--json', '--name-only', '--no-embedding', '--no-extract', '--once', '--others', '--path', '--pattern', '--pending', '--porcelain', '--quiet', '--reset', '--resolve', '--show-current', '--show-toplevel', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl'],
'import': ['--aliases', '--all', '--asof', '--background', '--brain', '--brain-wide-max-cost-usd', '--by-mention', '--cached', '--code', '--compile', '--concurrency', '--embedding-dimensions', '--embedding-model', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--fix', '--follow', '--force', '--force-rechunk', '--fresh', '--from-pages', '--full', '--help', '--http', '--include-gitignored', '--include-null-signature', '--json', '--lang', '--markdown', '--max-age', '--multimodal', '--name-status', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--older-than', '--others', '--path', '--pattern', '--pending', '--pglite', '--priority', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--since', '--skip-failed', '--source', '--source-id', '--stale', '--strategy', '--supabase', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl', '--url', '--workers'],
'init': ['--all', '--brain', '--chat-model', '--check', '--ctx-size', '--embedding-dimensions', '--embedding-model', '--entity', '--expansion-model', '--fast', '--force', '--from-pages', '--grant-types', '--help', '--http', '--issuer-url', '--json', '--judge-model', '--key', '--mcp-only', '--mcp-url', '--migrate-only', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--non-interactive', '--oauth-client-id', '--oauth-client-secret', '--path', '--pglite', '--provenance', '--schema-pack', '--scopes', '--skip-embed-check', '--source', '--stale', '--supabase', '--surface', '--target', '--to', '--token-ttl', '--touchpoint', '--url', '--version'],
'integrations': ['--auto', '--brain', '--dry-run', '--embeddings', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--no-embedding', '--overwrite', '--refresh', '--reranking', '--source', '--surface', '--target', '--token-ttl'],
'integrity': ['--aliases', '--all', '--auto', '--backend', '--background', '--brain', '--brain-wide-max-cost-usd', '--check', '--confidence', '--cost', '--dry-run', '--explain', '--fast', '--follow', '--force', '--fresh', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--limit', '--no-embedding', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--review-lower', '--skip-bare-tweet', '--skip-urls', '--source', '--stale', '--supabase', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl', '--type', '--url'],
'jobs': ['--abbrev-ref', '--aliases', '--all', '--allow-empty', '--allow-protected', '--allow-shell-jobs', '--apply', '--asof', '--auto', '--auto-fix', '--auto-with-prompt', '--background', '--backoff-delay', '--backoff-jitter', '--backoff-type', '--batch', '--batch-size', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--break-lock', '--budget-usd', '--budget-usd-per-day', '--by-mention', '--by-type', '--cached', '--catch-up', '--check', '--cli-path', '--cluster', '--cluster-errors', '--code', '--concurrency', '--confidence', '--confirm-destructive', '--content', '--date', '--days', '--delay', '--detach', '--diff-filter', '--dim', '--dimensions', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--empty', '--entities', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--federated-read', '--ff-only', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--force-retry', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--hard-deadline', '--health-interval', '--held-out', '--help', '--http', '--idempotency-key', '--image', '--include-frontmatter', '--include-gitignored', '--include-null-signature', '--infer-dates', '--inject-bootstrap', '--inline', '--input', '--install', '--interval', '--is-ancestor', '--json', '--kind', '--lang', '--limit', '--lock', '--markdown', '--max-age', '--max-attempts', '--max-cost-usd', '--max-crashes', '--max-rss', '--max-runtime-min', '--max-sources', '--max-stalled', '--max-usd', '--max-waiting', '--mcp-only', '--migrate-only', '--min-context', '--missing-path', '--mode', '--model', '--multimodal', '--name-only', '--name-status', '--near-symbol', '--ner', '--nice', '--no', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-federate', '--no-federated', '--no-gpg-sign', '--no-hard-deadline', '--no-inject', '--no-mutate', '--no-pull', '--no-renames', '--no-schema-pack', '--no-verify', '--no-worker', '--non-interactive', '--now', '--offset', '--older-than', '--once', '--order', '--orphan', '--others', '--output', '--override-disabled', '--pace', '--pace-max-concurrency', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--phase', '--pid-file', '--priority', '--progress-interval', '--progress-json', '--queue', '--quiet', '--redact-secrets', '--reenrich-after', '--refresh-cache', '--refresh-ms', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--resume', '--retry-failed', '--review-lower', '--run-id', '--save', '--segment-limit', '--serial', '--session', '--session-id', '--short', '--show-toplevel', '--sigkill-rescue', '--since', '--skip-bare-tweet', '--skip-failed', '--skip-urls', '--sleep', '--slug', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--stats', '--status', '--strategy', '--supersessions', '--surface', '--swap-only', '--symbol-kind', '--target', '--target-score', '--thin', '--thin-threshold', '--timeout', '--timeout-ms', '--to', '--token-ttl', '--trusted-extraction', '--type', '--types', '--uninstall', '--unsafe-bypass-dream-guard', '--url', '--user', '--verbose', '--verify', '--version', '--watch', '--wedge-rescue', '--with-db', '--workers', '--yes'],
'lint': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--exclude', '--explain', '--fast', '--fix', '--follow', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl'],
'lsd': ['--brain', '--force-resume', '--help', '--json', '--judge-model', '--limit', '--list-runs', '--max-cost', '--max-far-set', '--max-ideas-per-judge-call', '--no-save', '--resume', '--retry-judge', '--save', '--source', '--strict-budget', '--yes'],
'maintain': ['--aliases', '--all', '--background', '--brain', '--break-lock', '--by-mention', '--catch-up', '--column', '--concurrency', '--content-audit', '--count', '--detach', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--fix', '--force', '--force-retry', '--force-schema', '--from-meetings', '--full', '--help', '--include-flagged', '--include-frontmatter', '--include-null-signature', '--index-audit', '--infer-dates', '--input', '--json', '--kind', '--lang', '--locks', '--markdown', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-usd', '--migrate-only', '--multimodal', '--near-symbol', '--ner', '--nice', '--no-extract', '--no-mutate', '--older-than', '--once', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--progress-json', '--query', '--queue', '--quiet', '--rebuild-rollup', '--regenerate', '--remediate', '--remediation-plan', '--reset', '--resolve', '--restore-only', '--resume', '--run-id', '--safe', '--scope', '--since', '--skills-dir', '--skip-failed', '--slugs', '--source', '--source-id', '--stale', '--status', '--supabase', '--supersessions', '--symbol-kind', '--target', '--target-score', '--thin', '--to', '--top-k', '--type', '--unsafe-bypass-dream-guard', '--url', '--verbose', '--window', '--workers', '--yes'],
'migrate': ['--ab', '--aliases', '--all', '--auto-update', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--build-index', '--by-mention', '--catch-up', '--compile', '--days', '--dim', '--dry-run', '--embedding-dimensions', '--embedding-model', '--embeddings', '--exclusive', '--explain', '--fast', '--fix', '--follow', '--force', '--force-break-lock', '--force-retry', '--force-schema', '--from-meetings', '--from-pages', '--help', '--history', '--http', '--ignore-env-override', '--ignore-missing-key', '--include-null-signature', '--json', '--lang', '--locks', '--markdown', '--max-age', '--model', '--multimodal', '--name', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--non-interactive', '--pace', '--pace-max-concurrency', '--parallel', '--path', '--pattern', '--pending', '--phase', '--prefix', '--priority', '--progress-interval', '--progress-json', '--quiet', '--refresh-unqualified', '--remediate', '--reranking', '--reset', '--resolve', '--restore-only', '--resume', '--rollback', '--skip-verify', '--slugs', '--source', '--stale', '--supersessions', '--surface', '--symbol-kind', '--thin', '--timeout', '--to', '--undo', '--undo-wave', '--url', '--use-captured-snapshot', '--version', '--with-calibration', '--yes'],
'models': ['--aliases', '--all', '--brain', '--ctx-size', '--embedding-dimensions', '--embedding-model', '--embeddings', '--help', '--include-null-signature', '--json', '--judge-model', '--model', '--multimodal', '--no', '--no-embed', '--no-extract', '--pattern', '--pending', '--reranking', '--reset', '--resolve', '--skip', '--source', '--stale', '--supersessions', '--thin', '--undo', '--version'],
'maintain': ['--aliases', '--all', '--background', '--brain', '--break-lock', '--by-mention', '--catch-up', '--column', '--concurrency', '--content-audit', '--count', '--detach', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--fix', '--force', '--force-retry', '--force-schema', '--from-meetings', '--full', '--help', '--include-flagged', '--include-frontmatter', '--include-null-signature', '--index-audit', '--infer-dates', '--input', '--json', '--kind', '--lang', '--locks', '--markdown', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-usd', '--migrate-only', '--multimodal', '--near-symbol', '--ner', '--nice', '--no-extract', '--no-mutate', '--older-than', '--once', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--probe-pglite', '--progress-json', '--query', '--queue', '--quiet', '--rebuild-rollup', '--regenerate', '--remediate', '--remediation-plan', '--reset', '--resolve', '--restore-only', '--resume', '--run-id', '--safe', '--scope', '--since', '--skills-dir', '--skip-failed', '--slugs', '--source', '--source-id', '--stale', '--status', '--supabase', '--supersessions', '--symbol-kind', '--target', '--target-score', '--thin', '--to', '--top-k', '--type', '--unsafe-bypass-dream-guard', '--url', '--verbose', '--window', '--workers', '--yes'],
'migrate': ['--ab', '--aliases', '--all', '--auto-update', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--build-index', '--by-mention', '--catch-up', '--compile', '--days', '--dim', '--dry-run', '--embedding-dimensions', '--embedding-model', '--embeddings', '--exclusive', '--explain', '--fast', '--fix', '--follow', '--force', '--force-break-lock', '--force-retry', '--force-schema', '--from-meetings', '--from-pages', '--help', '--history', '--http', '--ignore-env-override', '--ignore-missing-key', '--include-null-signature', '--json', '--lang', '--locks', '--markdown', '--max-age', '--model', '--multimodal', '--name', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--non-interactive', '--pace', '--pace-max-concurrency', '--parallel', '--path', '--pattern', '--pending', '--phase', '--prefix', '--priority', '--progress-interval', '--progress-json', '--quiet', '--refresh-unqualified', '--remediate', '--reranking', '--reset', '--resolve', '--restore-only', '--resume', '--rollback', '--skip-verify', '--slugs', '--source', '--stale', '--supersessions', '--surface', '--symbol-kind', '--thin', '--timeout', '--to', '--token-ttl', '--undo', '--undo-wave', '--url', '--use-captured-snapshot', '--version', '--with-calibration', '--yes'],
'models': ['--aliases', '--all', '--brain', '--ctx-size', '--detail', '--embedding-dimensions', '--embedding-model', '--embeddings', '--help', '--include-null-signature', '--json', '--judge-model', '--model', '--multimodal', '--no', '--no-embed', '--no-extract', '--pattern', '--pending', '--reranking', '--reset', '--resolve', '--skip', '--source', '--stale', '--supersessions', '--thin', '--undo', '--version'],
'mounts': ['--alias', '--brain', '--cache', '--database-path', '--database-url', '--db-path', '--db-url', '--engine', '--explain', '--help', '--id', '--json', '--lang', '--lock', '--markdown', '--mcp-url', '--multimodal', '--near-symbol', '--path', '--restore-only', '--skills-dir', '--source', '--stale', '--symbol-kind', '--thin', '--verbose'],
'notability-eval': ['--aliases', '--all', '--brain', '--embedding-dimensions', '--embedding-model', '--help', '--in', '--include-null-signature', '--json', '--model', '--multimodal', '--no', '--no-embed', '--no-extract', '--out', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--skip-llm', '--source', '--stale', '--supersessions', '--target-high', '--target-low', '--target-medium', '--thin', '--version'],
'onboard': ['--aliases', '--all', '--allow-empty', '--allow-protected', '--apply', '--asof', '--auto', '--auto-with-prompt', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--check', '--content', '--date', '--days', '--entities', '--explain', '--federated', '--file', '--follow', '--from-pages', '--help', '--history', '--http', '--image', '--include-null-signature', '--json', '--kind', '--limit', '--max-usd', '--mode', '--multimodal', '--no-extract', '--offset', '--params', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--remediation-plan', '--reset', '--resolve', '--resume', '--save', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--target-score', '--thin', '--trusted-extraction', '--url', '--with-db', '--yes'],
'orphans': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--count', '--explain', '--follow', '--help', '--include-null-signature', '--include-pseudo', '--json', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--timeout'],
'pages': ['--aliases', '--all', '--brain', '--dry-run', '--help', '--include-null-signature', '--json', '--no-extract', '--older-than', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin'],
'pglite-repair': ['--brain', '--break-lock', '--dry-rnu', '--dry-run', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--no-embedding', '--path', '--quiet', '--source', '--surface', '--yes'],
'post-upgrade': ['--all', '--apply-clean-hunks', '--brain', '--check', '--code', '--compile', '--concurrency', '--embedding-dimensions', '--embedding-model', '--fast', '--ff-only', '--force', '--force-all', '--force-orchestrator', '--force-retry', '--force-schema', '--format', '--from-pages', '--help', '--host-dir', '--http', '--inject-bootstrap', '--inline', '--install', '--interval', '--json', '--limit', '--list', '--markdown', '--max-rss', '--migrate-only', '--migration', '--mode', '--model', '--multimodal', '--name-only', '--no', '--no-autopilot-install', '--no-embed', '--no-embedding', '--no-inject', '--no-worker', '--non-interactive', '--now', '--quiet', '--repo', '--since', '--skills-dir', '--skip-verify', '--source', '--stale', '--status', '--surface', '--swap-only', '--target', '--to', '--uninstall', '--user', '--verbose', '--verify', '--version', '--workers', '--yes'],
'protocol': ['--all', '--allow-empty', '--apply', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--entities', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--help', '--http', '--image', '--json', '--kind', '--limit', '--max-usd', '--mode', '--multimodal', '--no-embedding', '--offset', '--path', '--progress-interval', '--progress-json', '--quiet', '--save', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--stats', '--surface', '--synthesize', '--target', '--timeout', '--token', '--trusted-extraction', '--url', '--with-db', '--yes'],
'providers': ['--brain', '--ctx-size', '--embedding-dimensions', '--embedding-model', '--embeddings', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--reranking', '--source', '--surface', '--touchpoint', '--version'],
'pglite-repair': ['--brain', '--break-lock', '--dry-rnu', '--dry-run', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--no-embedding', '--path', '--quiet', '--source', '--surface', '--token-ttl', '--yes'],
'post-upgrade': ['--all', '--apply-clean-hunks', '--brain', '--check', '--code', '--compile', '--concurrency', '--detail', '--dim', '--embedding-dimensions', '--embedding-model', '--fast', '--ff-only', '--force', '--force-all', '--force-orchestrator', '--force-retry', '--force-schema', '--format', '--from-pages', '--help', '--host-dir', '--http', '--inject-bootstrap', '--inline', '--install', '--interval', '--json', '--limit', '--list', '--markdown', '--max-rss', '--migrate-only', '--migration', '--mode', '--model', '--multimodal', '--name-only', '--no', '--no-autopilot-install', '--no-embed', '--no-embedding', '--no-inject', '--no-worker', '--non-interactive', '--now', '--path', '--pglite', '--quiet', '--repo', '--reset', '--since', '--skills-dir', '--skip-verify', '--source', '--stale', '--status', '--supabase', '--surface', '--swap-only', '--target', '--to', '--token-ttl', '--uninstall', '--user', '--verbose', '--verify', '--version', '--workers', '--yes'],
'protocol': ['--all', '--allow-empty', '--apply', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--entities', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--help', '--http', '--image', '--json', '--kind', '--limit', '--max-usd', '--mode', '--multimodal', '--no-embedding', '--offset', '--path', '--progress-interval', '--progress-json', '--quiet', '--save', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--stats', '--surface', '--synthesize', '--target', '--timeout', '--token', '--token-ttl', '--trusted-extraction', '--url', '--with-db', '--yes'],
'providers': ['--brain', '--ctx-size', '--embedding-dimensions', '--embedding-model', '--embeddings', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--reranking', '--source', '--surface', '--token-ttl', '--touchpoint', '--version'],
'publish': ['--accent', '--bg', '--border', '--brain', '--card-bg', '--code-bg', '--error', '--fg', '--help', '--json', '--link', '--muted', '--out', '--password', '--source', '--title'],
'quarantine': ['--aliases', '--all', '--apply', '--brain', '--code', '--compile', '--explain', '--fast', '--fix', '--force', '--force-rechunk', '--from-pages', '--help', '--http', '--include-flagged', '--include-null-signature', '--json', '--lang', '--limit', '--markdown', '--multimodal', '--near-symbol', '--no-embed', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--surface', '--symbol-kind', '--thin'],
'recall': ['--aliases', '--all', '--allow-empty', '--apply', '--as-context', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-tokens', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--entities', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--grep', '--help', '--http', '--image', '--include-expired', '--include-null-signature', '--json', '--kind', '--limit', '--max-usd', '--mcp-only', '--mode', '--multimodal', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--query', '--quiet', '--reason', '--repo', '--reset', '--resolve', '--restore-only', '--rollup', '--save', '--session', '--session-id', '--since', '--since-last-run', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--timeout', '--today', '--trusted-extraction', '--url', '--watch', '--with-db', '--yes'],
'quarantine': ['--aliases', '--all', '--apply', '--brain', '--code', '--compile', '--explain', '--fast', '--fix', '--force', '--force-rechunk', '--from-pages', '--help', '--http', '--include-flagged', '--include-null-signature', '--json', '--lang', '--limit', '--markdown', '--multimodal', '--near-symbol', '--no-embed', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--surface', '--symbol-kind', '--thin', '--token-ttl'],
'recall': ['--aliases', '--all', '--allow-empty', '--apply', '--as-context', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-tokens', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--entities', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--grep', '--help', '--http', '--image', '--include-expired', '--include-null-signature', '--json', '--kind', '--limit', '--max-usd', '--mcp-only', '--mode', '--multimodal', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--query', '--quiet', '--reason', '--repo', '--reset', '--resolve', '--restore-only', '--rollup', '--save', '--session', '--session-id', '--since', '--since-last-run', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--timeout', '--today', '--token-ttl', '--trusted-extraction', '--url', '--watch', '--with-db', '--yes'],
'reconcile-links': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--explain', '--follow', '--help', '--include-frontmatter', '--include-null-signature', '--json', '--name-status', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--source', '--stale', '--strategy', '--supersessions', '--thin', '--timeout', '--type'],
'reindex': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--code', '--compile', '--concurrency', '--cost-estimate', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--fix', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--lang', '--limit', '--markdown', '--max-age', '--max-cost-usd', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--no-extract', '--older-than', '--path', '--pattern', '--pending', '--pglite', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--repo', '--reset', '--resolve', '--source', '--stale', '--supabase', '--supersessions', '--surface', '--thin', '--timeout', '--version', '--workers', '--yes'],
'reindex': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--code', '--compile', '--concurrency', '--cost-estimate', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--fix', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--lang', '--limit', '--markdown', '--max-age', '--max-cost-usd', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--no-extract', '--older-than', '--path', '--pattern', '--pending', '--pglite', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--repo', '--reset', '--resolve', '--source', '--stale', '--supabase', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl', '--version', '--workers', '--yes'],
'reindex-code': ['--abi', '--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--chunker-debug', '--code', '--compile', '--concurrency', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fix', '--follow', '--force', '--force-rechunk', '--help', '--include-null-signature', '--json', '--judge-model', '--lang', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--model', '--multimodal', '--no', '--no-embed', '--no-extract', '--older-than', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--serial', '--source', '--stale', '--supersessions', '--thin', '--timeout', '--version', '--workers', '--yes'],
'reindex-frontmatter': ['--aliases', '--all', '--brain', '--concurrency', '--dry-run', '--force', '--help', '--include-null-signature', '--json', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--slug-prefix', '--source', '--stale', '--supersessions', '--thin', '--workers', '--yes'],
'reindex-search-vector': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--explain', '--follow', '--help', '--include-null-signature', '--json', '--migrate-only', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--timeout', '--yes'],
'reinit-pglite': ['--abbrev-ref', '--all', '--allow-empty', '--brain', '--break-lock', '--chat-model', '--concurrency', '--confirm-destructive', '--diff-filter', '--dir', '--embedding-dimensions', '--embedding-model', '--empty', '--entity', '--exclude', '--exclude-standard', '--expansion-model', '--fast', '--force', '--force-break-lock', '--format', '--from-pages', '--full', '--grant-types', '--hard-deadline', '--help', '--http', '--include-gitignored', '--interval', '--is-ancestor', '--issuer-url', '--json', '--key', '--lock', '--max-age', '--max-sources', '--mcp-only', '--mcp-url', '--migrate-only', '--missing-path', '--model', '--name-only', '--name-status', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-gpg-sign', '--no-hard-deadline', '--no-pull', '--no-renames', '--no-schema-pack', '--no-sync', '--no-verify', '--non-interactive', '--oauth-client-id', '--oauth-client-secret', '--orphan', '--parallel', '--path', '--pglite', '--priority', '--provenance', '--quiet', '--repo', '--retry-failed', '--schema-pack', '--scopes', '--serial', '--short', '--show-toplevel', '--skip-embed-check', '--skip-failed', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--strategy', '--supabase', '--surface', '--target', '--timeout', '--to', '--url', '--verify', '--version', '--watch', '--workers', '--yes'],
'remote': ['--all', '--background', '--brain', '--break-lock', '--by-mention', '--column', '--concurrency', '--content-audit', '--count', '--detach', '--dim', '--dir', '--drain', '--embedding-dimensions', '--embedding-model', '--fast', '--fix', '--force', '--force-retry', '--force-schema', '--from-pages', '--full', '--help', '--http', '--include-flagged', '--index-audit', '--json', '--locks', '--markdown', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-usd', '--mcp-only', '--migrate-only', '--multimodal', '--nice', '--no-embedding', '--older-than', '--parallel', '--params', '--path', '--pglite', '--phase', '--pid-file', '--porcelain', '--progress-json', '--query', '--queue', '--rebuild-rollup', '--regenerate', '--remediate', '--remediation-plan', '--reset', '--resume', '--scope', '--scopes', '--since', '--skills-dir', '--skip-failed', '--slugs', '--source', '--source-id', '--stale', '--status', '--supabase', '--surface', '--target', '--target-score', '--timeout', '--to', '--top-k', '--url', '--window', '--workers', '--yes'],
'repair-jsonb': ['--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--explain', '--fast', '--follow', '--force', '--from-pages', '--help', '--http', '--json', '--lang', '--markdown', '--multimodal', '--near-symbol', '--no-embedding', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--restore-only', '--source', '--stale', '--supabase', '--surface', '--symbol-kind', '--thin', '--timeout', '--url'],
'reinit-pglite': ['--abbrev-ref', '--all', '--allow-empty', '--brain', '--break-lock', '--chat-model', '--concurrency', '--confirm-destructive', '--diff-filter', '--dir', '--embedding-dimensions', '--embedding-model', '--empty', '--entity', '--exclude', '--exclude-standard', '--expansion-model', '--fast', '--force', '--force-break-lock', '--format', '--from-pages', '--full', '--grant-types', '--hard-deadline', '--help', '--http', '--include-gitignored', '--interval', '--is-ancestor', '--issuer-url', '--json', '--key', '--lock', '--max-age', '--max-sources', '--mcp-only', '--mcp-url', '--migrate-only', '--missing-path', '--model', '--name-only', '--name-status', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-gpg-sign', '--no-hard-deadline', '--no-pull', '--no-renames', '--no-schema-pack', '--no-sync', '--no-verify', '--non-interactive', '--oauth-client-id', '--oauth-client-secret', '--orphan', '--parallel', '--path', '--pglite', '--priority', '--provenance', '--quiet', '--repo', '--retry-failed', '--schema-pack', '--scopes', '--serial', '--short', '--show-toplevel', '--skip-embed-check', '--skip-failed', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--strategy', '--supabase', '--surface', '--target', '--timeout', '--to', '--token-ttl', '--url', '--verify', '--version', '--watch', '--workers', '--yes'],
'remote': ['--all', '--background', '--brain', '--break-lock', '--by-mention', '--column', '--concurrency', '--content-audit', '--count', '--detach', '--dim', '--dir', '--drain', '--embedding-dimensions', '--embedding-model', '--fast', '--fix', '--force', '--force-retry', '--force-schema', '--from-pages', '--full', '--help', '--http', '--include-flagged', '--index-audit', '--json', '--locks', '--markdown', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-usd', '--mcp-only', '--migrate-only', '--multimodal', '--nice', '--no-embedding', '--older-than', '--parallel', '--params', '--path', '--pglite', '--phase', '--pid-file', '--porcelain', '--probe-pglite', '--progress-json', '--query', '--queue', '--rebuild-rollup', '--regenerate', '--remediate', '--remediation-plan', '--reset', '--resume', '--scope', '--scopes', '--since', '--skills-dir', '--skip-failed', '--slugs', '--source', '--source-id', '--stale', '--status', '--supabase', '--surface', '--target', '--target-score', '--timeout', '--to', '--token-ttl', '--top-k', '--url', '--window', '--workers', '--yes'],
'repair-jsonb': ['--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--explain', '--fast', '--follow', '--force', '--from-pages', '--help', '--http', '--json', '--lang', '--markdown', '--multimodal', '--near-symbol', '--no-embedding', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--restore-only', '--source', '--stale', '--supabase', '--surface', '--symbol-kind', '--thin', '--timeout', '--token-ttl', '--url'],
'report': ['--brain', '--content', '--dir', '--help', '--json', '--source', '--title', '--type'],
'repos': ['--abbrev-ref', '--abort', '--aliases', '--all', '--all-sources', '--allow-unverified-remote', '--brain', '--branch', '--break-lock', '--cached', '--clone-dir', '--compile', '--confirm-destructive', '--count', '--detect', '--diff-filter', '--dry-run', '--exclude-standard', '--explain', '--fast', '--federated', '--file', '--fix', '--force', '--force-break-lock', '--format', '--from-pages', '--full', '--get', '--git-dir', '--github-repo', '--help', '--http', '--include-null-signature', '--include-warns', '--jq', '--json', '--keep-storage', '--lang', '--local', '--markdown', '--max-age', '--max-cost-usd', '--message', '--multimodal', '--name', '--name-only', '--name-status', '--near-symbol', '--no-cron', '--no-embedding', '--no-extract', '--no-federate', '--no-federated', '--no-harden', '--no-verify', '--others', '--params', '--pat-file', '--path', '--pattern', '--pending', '--porcelain', '--push-only', '--quiet', '--rebase', '--repo', '--reset', '--resolve', '--restore-only', '--secret', '--set', '--short', '--show-toplevel', '--source', '--source-id', '--stale', '--status', '--strategy', '--supersessions', '--surface', '--symbol-kind', '--thin', '--unset-all', '--url', '--url-managed', '--yes'],
'repos': ['--abbrev-ref', '--abort', '--aliases', '--all', '--all-sources', '--allow-unverified-remote', '--brain', '--branch', '--break-lock', '--cached', '--clone-dir', '--compile', '--confirm-destructive', '--count', '--detect', '--diff-filter', '--dry-run', '--exclude-standard', '--explain', '--fast', '--federated', '--file', '--fix', '--force', '--force-break-lock', '--format', '--from-pages', '--full', '--get', '--git-dir', '--git-path', '--github-repo', '--help', '--http', '--include-null-signature', '--include-warns', '--is-inside-work-tree', '--json', '--keep-storage', '--lang', '--local', '--markdown', '--max-age', '--max-cost-usd', '--message', '--multimodal', '--name', '--name-only', '--name-status', '--near-symbol', '--no-cron', '--no-embedding', '--no-extract', '--no-federate', '--no-federated', '--no-harden', '--no-verify', '--others', '--params', '--pat-file', '--path', '--pattern', '--pending', '--porcelain', '--push-only', '--quiet', '--rebase', '--repo', '--reset', '--resolve', '--restore-only', '--secret', '--set', '--short', '--show-toplevel', '--source', '--source-id', '--stale', '--status', '--strategy', '--supersessions', '--surface', '--symbol-kind', '--thin', '--token-ttl', '--unset-all', '--url', '--url-managed', '--yes'],
'resolvers': ['--auto', '--backend', '--brain', '--cost', '--help', '--json', '--source'],
'retrieval-upgrade': ['--aliases', '--all', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--catch-up', '--dim', '--dry-run', '--embedding-dimensions', '--embedding-model', '--embeddings', '--explain', '--fast', '--follow', '--force', '--from-pages', '--help', '--http', '--ignore-env-override', '--ignore-missing-key', '--include-null-signature', '--json', '--markdown', '--model', '--multimodal', '--name', '--no', '--no-embed', '--no-embedding', '--no-extract', '--non-interactive', '--pace', '--pace-max-concurrency', '--parallel', '--pattern', '--pending', '--prefix', '--priority', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reranking', '--reset', '--resolve', '--resume', '--slugs', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout', '--to', '--undo', '--version', '--yes'],
'retrieval-upgrade': ['--aliases', '--all', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--catch-up', '--dim', '--dry-run', '--embedding-dimensions', '--embedding-model', '--embeddings', '--explain', '--fast', '--follow', '--force', '--from-pages', '--help', '--http', '--ignore-env-override', '--ignore-missing-key', '--include-null-signature', '--json', '--markdown', '--model', '--multimodal', '--name', '--no', '--no-embed', '--no-embedding', '--no-extract', '--non-interactive', '--pace', '--pace-max-concurrency', '--parallel', '--pattern', '--pending', '--prefix', '--priority', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reranking', '--reset', '--resolve', '--resume', '--slugs', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout', '--to', '--token-ttl', '--undo', '--version', '--yes'],
'routing-eval': ['--brain', '--fix', '--help', '--json', '--llm', '--skills-dir', '--source', '--strict', '--verbose'],
'salience': ['--aliases', '--all', '--brain', '--days', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--kind', '--limit', '--mcp-only', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--slug-prefix', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout'],
'schema': ['--alias', '--aliases', '--all', '--apply', '--as-filing-rules', '--brain', '--dims', '--expert', '--expert-routing', '--extractable', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--inverse', '--json', '--kind', '--no-embedding', '--no-extract', '--pack', '--page-type', '--pattern', '--pending', '--prefix', '--primitive', '--reset', '--resolve', '--schema-pack', '--since', '--source', '--source-id', '--stale', '--supersessions', '--surface', '--target-type', '--thin', '--to', '--with-db'],
'self-upgrade': ['--all', '--brain', '--check', '--check-only', '--ff-only', '--force', '--help', '--json', '--markdown', '--migrate-only', '--non-interactive', '--refresh-cache', '--source', '--swap-only', '--to', '--version', '--yes'],
'salience': ['--aliases', '--all', '--brain', '--days', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--kind', '--limit', '--mcp-only', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--slug-prefix', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl'],
'schema': ['--alias', '--aliases', '--all', '--apply', '--as-filing-rules', '--brain', '--dims', '--expert', '--expert-routing', '--extractable', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--inverse', '--json', '--kind', '--no-embedding', '--no-extract', '--pack', '--page-type', '--pattern', '--pending', '--prefix', '--primitive', '--reset', '--resolve', '--schema-pack', '--since', '--source', '--source-id', '--stale', '--supersessions', '--surface', '--target-type', '--thin', '--to', '--token-ttl', '--with-db'],
'self-upgrade': ['--all', '--brain', '--check', '--check-only', '--dim', '--ff-only', '--force', '--help', '--json', '--markdown', '--migrate-only', '--non-interactive', '--refresh-cache', '--source', '--swap-only', '--to', '--version', '--yes'],
'serve': ['--aliases', '--all', '--bind', '--bound-slug-prefixes', '--brain', '--enable-dcr', '--enable-dcr-insecure', '--fast', '--federated-read', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--log-full-params', '--name', '--no-embedding', '--no-extract', '--once', '--parallel', '--pattern', '--pending', '--port', '--prefix', '--print-admin-token', '--public-url', '--reset', '--resolve', '--source', '--stale', '--stdio-idle-timeout', '--supersessions', '--suppress', '--suppress-bootstrap-token', '--surface', '--thin', '--token-ttl', '--yes'],
'skillify': ['--brain', '--description', '--dry-run', '--force', '--help', '--json', '--mutating', '--recent', '--skills-dir', '--source', '--strict', '--triggers', '--verbose', '--writes-pages', '--writes-to'],
'skillopt': ['--aliases', '--all', '--allow-mutate-bundled', '--background', '--batch-size', '--benchmark', '--bootstrap-from-routing', '--bootstrap-from-skill', '--bootstrap-reviewed', '--bootstrap-tasks', '--brain', '--brain-wide-max-cost-usd', '--chunker-debug', '--dry-run', '--epochs', '--follow', '--force', '--held-out', '--help', '--include-null-signature', '--json', '--judge-model', '--lr', '--lr-schedule', '--max-cost-usd', '--max-runtime-min', '--model', '--no-extract', '--no-mutate', '--optimizer-model', '--patch', '--pattern', '--pending', '--reset', '--resolve', '--resume', '--rewrite', '--skills-dir', '--source', '--split', '--stale', '--supersessions', '--target-model', '--target-models', '--thin', '--verbose', '--yes'],
'skillpack': ['--all', '--apply-clean-hunks', '--author', '--brain', '--dry-run', '--exit-code', '--fast', '--fix', '--force', '--force-unlock', '--format', '--from', '--from-pages', '--frontmatter', '--full', '--help', '--homepage', '--http', '--json', '--license', '--list', '--minimal', '--name-only', '--no-cache', '--no-embedding', '--no-lint', '--note', '--out', '--overwrite-local', '--push', '--quick', '--quiet', '--refresh', '--repo', '--schema-pack', '--short', '--since', '--skills-dir', '--skip-doctor', '--source', '--strict', '--surface', '--target', '--tier', '--trust', '--url', '--verbose', '--verify', '--workspace', '--yes'],
'skillpack': ['--all', '--apply-clean-hunks', '--author', '--brain', '--dry-run', '--exit-code', '--fast', '--fix', '--force', '--force-unlock', '--format', '--from', '--from-pages', '--frontmatter', '--full', '--help', '--homepage', '--http', '--json', '--license', '--list', '--minimal', '--name-only', '--no-cache', '--no-embedding', '--no-lint', '--note', '--out', '--overwrite-local', '--push', '--quick', '--quiet', '--refresh', '--repo', '--schema-pack', '--short', '--since', '--skills-dir', '--skip-doctor', '--source', '--strict', '--surface', '--target', '--tier', '--token-ttl', '--trust', '--url', '--verbose', '--verify', '--workspace', '--yes'],
'skillpack-check': ['--background', '--brain', '--brain-wide-max-cost-usd', '--explain', '--fast', '--follow', '--help', '--json', '--list', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--source', '--stale', '--strict', '--timeout', '--yes'],
'smoke-test': ['--brain', '--help', '--json', '--source'],
'sources': ['--abbrev-ref', '--abort', '--aliases', '--all', '--all-sources', '--allow-unverified-remote', '--brain', '--branch', '--break-lock', '--cached', '--clone-dir', '--compile', '--confirm-destructive', '--count', '--detect', '--diff-filter', '--dry-run', '--exclude-standard', '--explain', '--fast', '--federated', '--file', '--fix', '--force', '--force-break-lock', '--format', '--from-pages', '--full', '--get', '--git-dir', '--github-repo', '--help', '--http', '--include-null-signature', '--include-warns', '--jq', '--json', '--keep-storage', '--lang', '--local', '--markdown', '--max-age', '--max-cost-usd', '--message', '--multimodal', '--name', '--name-only', '--name-status', '--near-symbol', '--no-cron', '--no-embedding', '--no-extract', '--no-federate', '--no-federated', '--no-harden', '--no-verify', '--others', '--params', '--pat-file', '--path', '--pattern', '--pending', '--porcelain', '--push-only', '--quiet', '--rebase', '--repo', '--reset', '--resolve', '--restore-only', '--secret', '--set', '--short', '--show-toplevel', '--source', '--source-id', '--stale', '--status', '--strategy', '--supersessions', '--surface', '--symbol-kind', '--thin', '--unset-all', '--url', '--url-managed', '--yes'],
'status': ['--abbrev-ref', '--abi', '--abort', '--aliases', '--all', '--all-sources', '--allow-empty', '--apply', '--asof', '--auto', '--background', '--batch-size', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--brain-wide-max-cost-usd', '--branch', '--break-lock', '--budget-usd-per-day', '--by-mention', '--cached', '--catch-up', '--clone-dir', '--code', '--column', '--compile', '--concurrency', '--confirm-destructive', '--content', '--content-audit', '--count', '--date', '--days', '--deadline-ms', '--depth', '--detach', '--diff-filter', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--empty', '--entities', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--ff-only', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--force-retry', '--force-schema', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--git-path', '--hard-deadline', '--help', '--http', '--image', '--include-flagged', '--include-frontmatter', '--include-gitignored', '--include-null-signature', '--index-audit', '--infer-dates', '--install', '--interval', '--is-ancestor', '--json', '--judge-model', '--kind', '--lang', '--limit', '--lock', '--locks', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-runtime', '--max-sources', '--max-usd', '--mcp-only', '--migrate-only', '--min-context', '--missing-path', '--mode', '--model', '--multimodal', '--name', '--name-only', '--name-status', '--near-symbol', '--ner', '--nice', '--no', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-gpg-sign', '--no-hard-deadline', '--no-pull', '--no-recurse-submodules', '--no-renames', '--no-schema-pack', '--no-verify', '--object-format', '--offset', '--older-than', '--order', '--orphan', '--others', '--overwrite', '--pace', '--pace-max-concurrency', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--prefix', '--priority', '--progress-interval', '--progress-json', '--query', '--queue', '--quiet', '--rebase', '--rebuild-rollup', '--reenrich-after', '--refresh', '--regenerate', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--resume', '--retry-failed', '--run-id', '--save', '--scope', '--scopes', '--section', '--serial', '--session', '--session-id', '--short', '--show-toplevel', '--since', '--skills-dir', '--skip-failed', '--slug', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--stats', '--status', '--stdin', '--strategy', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--target', '--target-score', '--thin', '--thin-threshold', '--timeout', '--to', '--top-k', '--trusted-extraction', '--type', '--types', '--url', '--url-managed', '--verbose', '--verify', '--version', '--watch', '--what', '--where', '--who', '--window', '--with-db', '--workers', '--yes'],
'sources': ['--abbrev-ref', '--abort', '--aliases', '--all', '--all-sources', '--allow-unverified-remote', '--brain', '--branch', '--break-lock', '--cached', '--clone-dir', '--compile', '--confirm-destructive', '--count', '--detect', '--diff-filter', '--dry-run', '--exclude-standard', '--explain', '--fast', '--federated', '--file', '--fix', '--force', '--force-break-lock', '--format', '--from-pages', '--full', '--get', '--git-dir', '--git-path', '--github-repo', '--help', '--http', '--include-null-signature', '--include-warns', '--is-inside-work-tree', '--json', '--keep-storage', '--lang', '--local', '--markdown', '--max-age', '--max-cost-usd', '--message', '--multimodal', '--name', '--name-only', '--name-status', '--near-symbol', '--no-cron', '--no-embedding', '--no-extract', '--no-federate', '--no-federated', '--no-harden', '--no-verify', '--others', '--params', '--pat-file', '--path', '--pattern', '--pending', '--porcelain', '--push-only', '--quiet', '--rebase', '--repo', '--reset', '--resolve', '--restore-only', '--secret', '--set', '--short', '--show-toplevel', '--source', '--source-id', '--stale', '--status', '--strategy', '--supersessions', '--surface', '--symbol-kind', '--thin', '--token-ttl', '--unset-all', '--url', '--url-managed', '--yes'],
'status': ['--abbrev-ref', '--abi', '--abort', '--aliases', '--all', '--all-sources', '--allow-empty', '--apply', '--asof', '--auto', '--background', '--batch-size', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--brain-wide-max-cost-usd', '--branch', '--break-lock', '--budget-usd-per-day', '--by-mention', '--cached', '--catch-up', '--clone-dir', '--code', '--column', '--compile', '--concurrency', '--confirm-destructive', '--content', '--content-audit', '--count', '--date', '--days', '--deadline-ms', '--depth', '--detach', '--detail', '--diff-filter', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--empty', '--entities', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--ff-only', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--force-retry', '--force-schema', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--git-path', '--hard-deadline', '--help', '--http', '--image', '--include-flagged', '--include-frontmatter', '--include-gitignored', '--include-null-signature', '--index-audit', '--infer-dates', '--install', '--interval', '--is-ancestor', '--json', '--judge-model', '--kind', '--lang', '--limit', '--lock', '--locks', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-runtime', '--max-sources', '--max-usd', '--mcp-only', '--migrate-only', '--min-context', '--missing-path', '--mode', '--model', '--multimodal', '--name', '--name-only', '--name-status', '--near-symbol', '--ner', '--nice', '--no', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-gpg-sign', '--no-hard-deadline', '--no-pull', '--no-recurse-submodules', '--no-renames', '--no-schema-pack', '--no-verify', '--object-format', '--offset', '--older-than', '--order', '--orphan', '--others', '--overwrite', '--pace', '--pace-max-concurrency', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--prefix', '--priority', '--probe-pglite', '--progress-interval', '--progress-json', '--query', '--queue', '--quiet', '--rebase', '--rebuild-rollup', '--reenrich-after', '--refresh', '--regenerate', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--resume', '--retry-failed', '--run-id', '--save', '--scope', '--scopes', '--section', '--serial', '--session', '--session-id', '--short', '--show-toplevel', '--since', '--skills-dir', '--skip-failed', '--slug', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--stats', '--status', '--stdin', '--strategy', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--target', '--target-score', '--thin', '--thin-threshold', '--timeout', '--to', '--token-ttl', '--top-k', '--trusted-extraction', '--type', '--types', '--url', '--url-managed', '--verbose', '--verify', '--version', '--watch', '--what', '--where', '--who', '--window', '--with-db', '--workers', '--yes'],
'storage': ['--aliases', '--all', '--brain', '--federated', '--fix', '--help', '--include-null-signature', '--json', '--no-extract', '--no-federated', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--thin', '--to'],
'sweep': ['--aliases', '--all', '--batch-limit', '--brain', '--budget-ms', '--help', '--include-null-signature', '--json', '--no-extract', '--once', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--timeout'],
'sync': ['--abbrev-ref', '--abi', '--abort', '--aliases', '--all', '--all-sources', '--allow-empty', '--asof', '--auto', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--branch', '--break-lock', '--by-mention', '--cached', '--catch-up', '--clone-dir', '--code', '--column', '--compile', '--concurrency', '--confirm-destructive', '--content-audit', '--count', '--depth', '--detach', '--diff-filter', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--empty', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--ff-only', '--fix', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--force-retry', '--force-schema', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--git-path', '--hard-deadline', '--help', '--http', '--include-flagged', '--include-frontmatter', '--include-gitignored', '--include-null-signature', '--index-audit', '--infer-dates', '--interval', '--is-ancestor', '--json', '--kind', '--lang', '--lock', '--locks', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-sources', '--max-usd', '--migrate-only', '--missing-path', '--model', '--multimodal', '--name', '--name-only', '--name-status', '--ner', '--nice', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-gpg-sign', '--no-hard-deadline', '--no-pull', '--no-recurse-submodules', '--no-renames', '--no-schema-pack', '--no-verify', '--object-format', '--older-than', '--orphan', '--others', '--overwrite', '--pace', '--pace-max-concurrency', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--prefix', '--priority', '--progress-interval', '--progress-json', '--query', '--queue', '--quiet', '--rebase', '--rebuild-rollup', '--refresh', '--regenerate', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--resume', '--retry-failed', '--run-id', '--save', '--scope', '--serial', '--short', '--show-toplevel', '--since', '--skills-dir', '--skip-failed', '--slug', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--status', '--stdin', '--strategy', '--supabase', '--supersessions', '--surface', '--target', '--target-score', '--thin', '--timeout', '--to', '--top-k', '--type', '--url', '--url-managed', '--verbose', '--verify', '--watch', '--window', '--workers', '--yes'],
'takes': ['--aliases', '--all', '--brain', '--bucket-size', '--by', '--claim', '--dir', '--domain', '--dry-run', '--evidence', '--expired', '--fast', '--federated', '--force', '--from-pages', '--help', '--holder', '--http', '--include-covered', '--include-null-signature', '--json', '--kind', '--limit', '--max-pages', '--no-embedding', '--no-extract', '--no-federated', '--outcome', '--path', '--pattern', '--pending', '--quality', '--refresh', '--repo', '--reset', '--resolve', '--restore-only', '--row', '--since', '--slugs', '--sort', '--source', '--source-id', '--stale', '--supersessions', '--surface', '--thin', '--unit', '--until', '--value', '--weight', '--who', '--yes'],
'think': ['--aliases', '--all', '--anchor', '--brain', '--calibration-holder', '--explain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--max-usd', '--mcp-only', '--model', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--rounds', '--save', '--since', '--source', '--stale', '--supersessions', '--surface', '--take', '--thin', '--timeout', '--until', '--with-calibration'],
'sync': ['--abbrev-ref', '--abi', '--abort', '--aliases', '--all', '--all-sources', '--allow-empty', '--asof', '--auto', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--branch', '--break-lock', '--by-mention', '--cached', '--catch-up', '--clone-dir', '--code', '--column', '--compile', '--concurrency', '--confirm-destructive', '--content-audit', '--count', '--depth', '--detach', '--diff-filter', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--empty', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--ff-only', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--force-retry', '--force-schema', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--git-path', '--hard-deadline', '--help', '--http', '--include-flagged', '--include-frontmatter', '--include-gitignored', '--include-null-signature', '--index-audit', '--infer-dates', '--interval', '--is-ancestor', '--json', '--kind', '--lang', '--lock', '--locks', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-sources', '--max-usd', '--migrate-only', '--missing-path', '--model', '--multimodal', '--name', '--name-only', '--name-status', '--ner', '--nice', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-gpg-sign', '--no-hard-deadline', '--no-pull', '--no-recurse-submodules', '--no-renames', '--no-schema-pack', '--no-verify', '--object-format', '--older-than', '--orphan', '--others', '--overwrite', '--pace', '--pace-max-concurrency', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--prefix', '--priority', '--probe-pglite', '--progress-interval', '--progress-json', '--query', '--queue', '--quiet', '--rebase', '--rebuild-rollup', '--refresh', '--regenerate', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--resume', '--retry-failed', '--run-id', '--save', '--scope', '--serial', '--short', '--show-toplevel', '--since', '--skills-dir', '--skip-failed', '--slug', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--status', '--stdin', '--strategy', '--supabase', '--supersessions', '--surface', '--target', '--target-score', '--thin', '--timeout', '--to', '--token-ttl', '--top-k', '--type', '--url', '--url-managed', '--verbose', '--verify', '--watch', '--window', '--workers', '--yes'],
'takes': ['--aliases', '--all', '--brain', '--bucket-size', '--by', '--claim', '--dir', '--domain', '--dry-run', '--evidence', '--expired', '--fast', '--federated', '--force', '--from-pages', '--help', '--holder', '--http', '--include-covered', '--include-null-signature', '--json', '--kind', '--limit', '--max-pages', '--no-embedding', '--no-extract', '--no-federated', '--outcome', '--path', '--pattern', '--pending', '--quality', '--refresh', '--repo', '--reset', '--resolve', '--restore-only', '--row', '--since', '--slugs', '--sort', '--source', '--source-id', '--stale', '--supersessions', '--surface', '--thin', '--token-ttl', '--unit', '--until', '--value', '--weight', '--who', '--yes'],
'think': ['--aliases', '--all', '--anchor', '--brain', '--calibration-holder', '--explain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--max-usd', '--mcp-only', '--model', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--rounds', '--save', '--since', '--source', '--stale', '--supersessions', '--surface', '--take', '--thin', '--timeout', '--token-ttl', '--until', '--with-calibration'],
'transcripts': ['--aliases', '--all', '--brain', '--days', '--full', '--help', '--include-null-signature', '--json', '--limit', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--timeout'],
'upgrade': ['--all', '--apply-clean-hunks', '--brain', '--check', '--code', '--compile', '--concurrency', '--embedding-dimensions', '--embedding-model', '--fast', '--ff-only', '--force', '--force-all', '--force-orchestrator', '--force-retry', '--force-schema', '--format', '--from-pages', '--help', '--host-dir', '--http', '--inject-bootstrap', '--inline', '--install', '--interval', '--json', '--limit', '--list', '--markdown', '--max-rss', '--migrate-only', '--migration', '--mode', '--model', '--multimodal', '--name-only', '--no', '--no-autopilot-install', '--no-embed', '--no-embedding', '--no-inject', '--no-worker', '--non-interactive', '--now', '--quiet', '--repo', '--since', '--skills-dir', '--skip-verify', '--source', '--stale', '--status', '--surface', '--swap-only', '--target', '--to', '--uninstall', '--user', '--verbose', '--verify', '--version', '--workers', '--yes'],
'watch': ['--aliases', '--all', '--brain', '--fast', '--federated', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--max-pages', '--min-confidence', '--no-embedding', '--no-extract', '--no-federated', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--window-turns'],
'upgrade': ['--all', '--apply-clean-hunks', '--brain', '--check', '--code', '--compile', '--concurrency', '--detail', '--dim', '--embedding-dimensions', '--embedding-model', '--fast', '--ff-only', '--force', '--force-all', '--force-orchestrator', '--force-retry', '--force-schema', '--format', '--from-pages', '--help', '--host-dir', '--http', '--inject-bootstrap', '--inline', '--install', '--interval', '--json', '--limit', '--list', '--markdown', '--max-rss', '--migrate-only', '--migration', '--mode', '--model', '--multimodal', '--name-only', '--no', '--no-autopilot-install', '--no-embed', '--no-embedding', '--no-inject', '--no-worker', '--non-interactive', '--now', '--path', '--pglite', '--quiet', '--repo', '--reset', '--since', '--skills-dir', '--skip-verify', '--source', '--stale', '--status', '--supabase', '--surface', '--swap-only', '--target', '--to', '--token-ttl', '--uninstall', '--user', '--verbose', '--verify', '--version', '--workers', '--yes'],
'watch': ['--aliases', '--all', '--brain', '--fast', '--federated', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--max-pages', '--min-confidence', '--no-embedding', '--no-extract', '--no-federated', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--token-ttl', '--window-turns'],
'ze-switch': ['--aliases', '--all', '--brain', '--confirm-reembed', '--dry-run', '--force', '--help', '--ignore-env-override', '--ignore-missing-key', '--include-null-signature', '--json', '--markdown', '--no-extract', '--non-interactive', '--pattern', '--pending', '--reset', '--resolve', '--resume', '--source', '--stale', '--supersessions', '--thin', '--undo', '--yes'],
};
src/core/config.ts
+13
View File
@@ -27,6 +27,10 @@ function getConfigPath() { return configPath(); }
export interface GBrainConfig {
engine: 'postgres' | 'pglite';
/** File-plane hook-lane keys (read by engine-free hook/push children).
* `gbrain config set` routes these two dotted keys here, not to the DB. */
push?: { allow_unverified_remote?: boolean };
hooks?: { stop_push_debounce_min?: number | string };
database_url?: string;
database_path?: string;
openai_api_key?: string;
@@ -984,6 +988,10 @@ export const KNOWN_CONFIG_KEYS: readonly string[] = [
'agent.use_gateway_loop',
// #2778: per-turn output-token cap for the subagent loop (default 8192).
'agent.max_output_tokens',
// File-plane bootstrap hook-lane keys (routed to ~/.gbrain/config.json by
// `config set` — engine-free hook/push children read loadConfigFileOnly).
'push.allow_unverified_remote',
'hooks.stop_push_debounce_min',
// DB-plane (v0.32.3 search modes + related)
'search.mode',
'search.cache.enabled',
@@ -1115,6 +1123,11 @@ export const KNOWN_CONFIG_KEYS: readonly string[] = [
'orphans.exclude_slugs',
'sync.cost_gate_min_usd',
'sync.federated_v2',
// #2179: clamp window for DCR-requested per-client token TTLs. Read by
// `gbrain serve --http` at startup; unset min defaults to 300s, unset max
// defaults fail-closed to max(--token-ttl, min).
'oauth.dcr_ttl_min_seconds',
'oauth.dcr_ttl_max_seconds',
'embed.backfill_cooldown_min',
'embed.backfill_max_usd_per_source_24h',
'embed.backfill_max_usd',
src/core/conversation-parser/builtins.ts
+42 -2
View File
@@ -1,7 +1,7 @@
/**
* v0.41.16.0 — Built-in conversation parser pattern registry.
*
* Seventeen hand-vetted patterns covering the chat-export formats this
* Eighteen hand-vetted patterns covering the chat-export formats this
* codebase is most likely to encounter. Each pattern's regex was
* derived from a public format reference (source_doc field) so future
* maintainers can verify against the wild shape.
@@ -50,7 +50,7 @@ export function cleanSpeaker(raw: string, override?: RegExp): string {
return stripped || raw.trim();
}
/** The 17 hand-vetted built-in patterns. */
/** The 18 hand-vetted built-in patterns. */
export const BUILTIN_PATTERNS: readonly PatternEntry[] = [
// -------------------------------------------------------------------
// INLINE-DATE patterns (date in every line; less ambiguous; tried first).
@@ -670,6 +670,46 @@ export const BUILTIN_PATTERNS: readonly PatternEntry[] = [
test_negative: ['<alice> classic irc, no time', '[18:37] @alice: matrix'],
source_doc: 'weechat default logger.format `%H:%M %p\\t%m`',
},
{
id: 'markdown-heading-turn',
origin: 'builtin',
// gbrain transcript-ingest shape: a heading-only line ('## User' /
// '## Assistant' / '### Human') opens a turn; the message text is
// the continuation lines below the heading (D5), not anything on
// the heading line itself. No per-line timestamps — date comes
// from frontmatter / effective_date. The speaker set is closed
// (User/Assistant/Human/System only) so ordinary section headings
// like '## Summary' never match, and a heading with trailing prose
// ('## User said hello') is rejected rather than mis-captured.
regex: /^#{2,3}\s+(User|Assistant|Human|System)\s*:?\s*()$/,
captures: {
speaker_group: 1,
text_group: 2,
},
date_source: 'frontmatter',
time_format: '24h',
timezone_policy: 'utc_assumed_with_warn',
multi_line: true,
score_continuations_as_body: true,
// Narrowed to a role-prefix superset (NOT bare `/^#{2,3}\s/`): a body
// that pastes unrelated markdown headings (e.g. a document with many
// '## Section' headings) would otherwise inflate the D18 scorer's
// anchor-candidate denominator without inflating the anchored count,
// starving the pattern's score toward 0 on otherwise-valid transcripts.
// Still a strict superset of `regex` per validatePatternEntry's
// invariant (every test_positive sample passes both).
quick_reject: /^#{2,3}\s+(?:User|Assistant|Human|System)\b/,
test_positive: ['## User', '## Assistant', '### Human', '## System', '## User:'],
test_negative: [
'## Summary',
'#### User',
'User: plain no heading',
'## User said hello',
],
source_doc:
'gbrain nightly transcript ingest: compiled_truth bodies use markdown headings per turn',
},
];
/**
src/core/conversation-parser/parse.ts
+1 -1
View File
@@ -392,7 +392,7 @@ function getNonBlankLines(body: string, headCap?: number): string[] {
* window) and `scorePatternFull` (whole body) delegate here so the
* quick_reject + regex loop lives in one place. Reused by
* `parseConversation`'s fallback path which pre-splits ONCE and
* passes the array to all 17 candidates (saves 16 redundant body
* passes the array to all 18 candidates (saves 17 redundant body
* splits per fallback pass).
*/
function scoreFromLines(
src/core/cycle.ts
+7
View File
@@ -2063,6 +2063,13 @@ export async function runCycle(
yieldDuringPhase: opts.yieldDuringPhase,
once: opts.onceForPhase === 'patterns',
deadlineAtMs: opts.deadlineAtMs ?? null,
// #1586: scope pattern writes to the cycle's resolved source, same as
// synthesize above. Without it the child's put_page rows land in
// 'default' while the reverse-write drops the file into the named
// source's checkout — the row and the file disagree about which
// source owns the page, which is what doctor reports as
// multi_source_drift.
sourceId: cycleSourceId,
}));
result.duration_ms = duration_ms;
phaseResults.push(result);
src/core/cycle/extract-atoms.ts
+37 -27
View File
@@ -6,14 +6,16 @@
// pages already extracted by content hash — see "Idempotency" below).
// 2. Dedup by content_hash; transcripts win on collision.
// 3. Per work-item, ask Haiku for 1-3 atoms.
// 4. Write each atom via engine.putPage(slug, page, {sourceId})
// with sourceId threaded so federated brains route correctly.
// 4. Write each atom via importFromContent(slug, markdown, {sourceId})
// with sourceId threaded so federated brains route correctly. The
// canonical import path (not engine.putPage) is what chunks and embeds
// the page — see the write site below and #2163.
//
// Idempotency (per-atom, via deterministic slug):
// Each atom's slug is `atoms/<source-date>/<stem>-<title-hash>` — built from
// the SOURCE date (the transcript's own date / the page slug), NOT the run
// date, plus a 6-char hash of the title. Re-extracting the same atom resolves
// to the SAME slug, so engine.putPage upserts in place instead of minting a
// to the SAME slug, so the import upserts in place instead of minting a
// duplicate. This closes three bugs in one scheme:
// - PR #1414's page-side re-extraction.
// - The cross-day transcript duplicate: append-only transcripts grow daily,
@@ -51,7 +53,9 @@ import type { BrainEngine } from '../engine.ts';
import type { PhaseResult } from '../cycle.ts';
import type { GBrainConfig } from '../config.ts';
import type { ProgressReporter } from '../progress.ts';
import { chat as gatewayChat, withBudgetTracker } from '../ai/gateway.ts';
import { chat as gatewayChat, withBudgetTracker, isAvailable } from '../ai/gateway.ts';
import { importFromContent } from '../import-file.ts';
import { serializeMarkdown } from '../markdown.ts';
import { BudgetExhausted, BudgetTracker, isModelPriceable } from '../budget/budget-tracker.ts';
import { writeReceipt } from '../extract/receipt-writer.ts';
import { upsertExtractRollup } from '../extract/rollup-writer.ts';
@@ -681,32 +685,38 @@ export async function runPhaseExtractAtoms(
item.kind === 'transcript'
? { source_path: item.filePath }
: { source_slug: item.slug };
// v0.41.2.1 D9 #1 — thread sourceId through every putPage so
// atoms land in the source we discovered them from. Pre-fix
// the third arg was missing and atoms always wrote to 'default'.
await engine.putPage(
slug,
// Serialize to markdown and import via the canonical pipeline so
// the atom is chunked (+ embedded when a provider is configured).
// engine.putPage is a bare page-row upsert that never chunks, so
// atoms written through it never reached content_chunks and were
// invisible to search — the same defect #2163 fixed for concept
// pages in synthesize-concepts.ts, which was never applied here.
//
// `type: 'atom'` rides in frontmatter, which parseMarkdown honours
// as an explicit override ahead of path inference, so the page type
// survives the round-trip. sourceId stays threaded (v0.41.2.1 D9 #1)
// so atoms still land in the source they were discovered from.
const md = serializeMarkdown(
{
title: atom.title,
type: 'atom',
compiled_truth: atom.body,
frontmatter: {
type: 'atom',
atom_type: atom.atom_type,
...originFrontmatter,
source_hash: item.contentHash.slice(0, 16),
...(atom.source_quote && { source_quote: atom.source_quote }),
...(atom.lesson && { lesson: atom.lesson }),
...(atom.concepts && atom.concepts.length > 0 && { concepts: atom.concepts }),
...(atom.virality_score !== undefined && { virality_score: atom.virality_score }),
...(atom.emotional_register && { emotional_register: atom.emotional_register }),
extracted_at: new Date().toISOString(),
extracted_by: 'extract_atoms-v0.41.2.1',
},
timeline: '',
atom_type: atom.atom_type,
...originFrontmatter,
source_hash: item.contentHash.slice(0, 16),
...(atom.source_quote && { source_quote: atom.source_quote }),
...(atom.lesson && { lesson: atom.lesson }),
...(atom.concepts && atom.concepts.length > 0 && { concepts: atom.concepts }),
...(atom.virality_score !== undefined && { virality_score: atom.virality_score }),
...(atom.emotional_register && { emotional_register: atom.emotional_register }),
extracted_at: new Date().toISOString(),
extracted_by: 'extract_atoms-v0.41.2.1',
},
{ sourceId },
atom.body,
'',
{ type: 'atom', title: atom.title, tags: [] },
);
await importFromContent(engine, slug, md, {
sourceId,
noEmbed: !isAvailable('embedding'),
});
totalAtomsExtracted++;
}
} else {
src/core/cycle/patterns.ts
+55 -33
View File
@@ -30,14 +30,13 @@ import { serializeMarkdown } from '../markdown.ts';
import type { Page, PageType } from '../types.ts';
// #2415: allow-list + output-root resolution shared with the synthesize
// phase — both phases must agree on the configured namespace.
// runPgliteSubagentsInline is shared too: PGLite has no separate Minions
// worker process (the embedded data-dir holds an exclusive file lock), so a
// job submitted via queue.add() sits in 'waiting' forever unless something
// drives the claim -> run -> complete loop inline. synthesize.ts already
// does this for its own children; patterns.ts previously submitted and
// waited without ever draining, so every real (non-dry-run) invocation on a
// PGLite brain hung until subagentWaitTimeoutMs (default 35 min).
import { loadAllowedSlugPrefixes, loadOutputRoot, runPgliteSubagentsInline } from './synthesize.ts';
// runSubagentsInline is shared too: a job submitted via queue.add() sits in
// 'waiting' forever unless something drives the claim -> run -> complete
// loop — on PGLite because no separate worker can open the embedded
// data-dir, on Postgres because the parent phase itself occupies a worker
// slot and can deadlock a fully-occupied worker (#2050). synthesize.ts
// drains its own children the same way.
import { loadAllowedSlugPrefixes, loadOutputRoot, runSubagentsInline } from './synthesize.ts';
import { probeChatModel } from '../ai/gateway.ts';
import { normalizeModelId } from '../model-id.ts';
@@ -60,6 +59,13 @@ export interface PatternsPhaseOpts {
* mid-phase and starves every tail phase (#2781).
*/
deadlineAtMs?: number | null;
/**
* #1586: the cycle's resolved source. Stamped onto every subagent child as
* `source_id` so put_page writes land in this source's rows, and passed to
* reverseWriteRefs so getPage/getTags read the correct (source_id, slug)
* row. Unset → legacy 'default'. Mirrors synthesize.ts's `sourceId`.
*/
sourceId?: string;
}
/**
@@ -185,18 +191,20 @@ export async function runPhasePatterns(
}
const queue = new MinionQueue(engine);
// PGLite children drain inline (no separate worker can open the embedded
// data-dir), so give this job a private per-run queue: the inline drain
// must never claim unrelated 'default'-queue jobs a Postgres worker owns.
// Mirrors synthesize.ts's childQueueName derivation exactly.
const childQueueName = engine.kind === 'pglite'
? `dream-inline-${Date.now()}-${randomUUID().slice(0, 8)}`
: 'default';
// #2050: children drain inline on BOTH engines (see runSubagentsInline),
// so give this job a private per-run queue: the inline drain must never
// claim unrelated 'default'-queue jobs, and a 'default'-queue worker must
// never claim a child this parent is about to run itself. Mirrors
// synthesize.ts's childQueueName derivation exactly.
const childQueueName = `dream-inline-${Date.now()}-${randomUUID().slice(0, 8)}`;
const data: SubagentHandlerData = {
prompt: buildPatternsPrompt(reflections, config.minEvidence, config.sourceSlugPrefix, config.outputSlugPrefix),
model: config.model,
max_turns: 30,
allowed_slug_prefixes: allowedSlugPrefixes,
// #1586: scope every child tool call to the cycle's resolved source so
// put_page writes land there instead of the hardcoded 'default'.
...(opts.sourceId ? { source_id: opts.sourceId } : {}),
};
const submitOpts: Partial<MinionJobInput> = {
max_stalled: 3,
@@ -207,12 +215,11 @@ export async function runPhasePatterns(
allowProtectedSubmit: true,
});
// PGLite cannot run a separate Minions worker because the embedded DB
// holds an exclusive file lock. Drain this phase's private child queue
// inline so the parent observes the terminal state instead of polling
// waitForCompletion until subagentWaitTimeoutMs expires. No-op on
// Postgres (a real worker process claims the job there).
await runPgliteSubagentsInline(engine, queue, childQueueName, opts.yieldDuringPhase);
// Drain this phase's private child queue inline so the parent observes
// the terminal state instead of polling waitForCompletion until
// subagentWaitTimeoutMs expires. Runs on BOTH engines — on Postgres the
// parent job otherwise deadlocks a fully-occupied worker (#2050).
await runSubagentsInline(engine, queue, childQueueName, opts.yieldDuringPhase);
let outcome: string;
try {
@@ -243,10 +250,14 @@ export async function runPhasePatterns(
// Collect refs the subagent wrote (codex finding #2 — query tool exec rows).
// v0.32.8: refs carry source_id so reverseWriteRefs targets the right
// (source, slug) row instead of the first DB match.
const writtenRefs = await collectChildPutPageSlugs(engine, [job.id]);
// #1586: refs carry the cycle's resolved source (children wrote there via
// SubagentHandlerData.source_id), so getPage/getTags read the same row the
// child wrote, and the reverse-write treats it as the native source.
const cycleSourceId = opts.sourceId ?? 'default';
const writtenRefs = await collectChildPutPageSlugs(engine, [job.id], cycleSourceId);
// Reverse-write to fs.
const reverseWriteCount = await reverseWriteRefs(engine, opts.brainDir, writtenRefs);
const reverseWriteCount = await reverseWriteRefs(engine, opts.brainDir, writtenRefs, cycleSourceId);
const details = {
reflections_considered: reflections.length,
@@ -454,13 +465,14 @@ When done, briefly list the pattern slugs you wrote/updated in your final messag
async function collectChildPutPageSlugs(
engine: BrainEngine,
childIds: number[],
sourceId = 'default',
): Promise<Array<{ slug: string; source_id: string }>> {
if (childIds.length === 0) return [];
// v0.32.8: subagent put_page tool schema doesn't expose source_id (subagents
// are scoped to a single source). Default to 'default' here; multi-source
// dream cycles are a v0.33 follow-up. The point of threading source_id is
// so reverseWriteRefs can pass it through getPage and pick the correct
// (source_id, slug) row instead of whatever the DB happens to return.
// are scoped to a single source). #1586: stamp the cycle's resolved source —
// children write there via SubagentHandlerData.source_id — so reverseWriteRefs
// can pass it through getPage and pick the correct (source_id, slug) row
// instead of whatever the DB happens to return. Unset → legacy 'default'.
const rows = await engine.executeRaw<{ slug: string }>(
`SELECT DISTINCT
COALESCE(input->>'slug', (input #>> '{}')::jsonb->>'slug') AS slug
@@ -474,7 +486,7 @@ async function collectChildPutPageSlugs(
return rows
.map(r => r.slug)
.filter((s): s is string => typeof s === 'string' && s.length > 0)
.map(slug => ({ slug, source_id: 'default' }));
.map(slug => ({ slug, source_id: sourceId }));
}
// ── Reverse-write ────────────────────────────────────────────────────
@@ -485,6 +497,7 @@ async function reverseWriteRefs(
engine: BrainEngine,
brainDir: string,
refs: Array<{ slug: string; source_id: string }>,
nativeSourceId = 'default',
): Promise<number> {
let count = 0;
for (const { slug, source_id } of refs) {
@@ -496,11 +509,12 @@ async function reverseWriteRefs(
const tags = await engine.getTags(slug, { sourceId: source_id });
try {
const md = renderPageToMarkdown(page, tags);
// v0.32.8 F6: non-default sources land under brainDir/.sources/<id>/<slug>.md
// so same-slug-different-source pages don't collide on disk. Default-source
// pages stay at brainDir/<slug>.md so single-source brains see no change.
// `.sources/` is a reserved prefix; walkBrainRepo skips dot-dirs.
const filePath = source_id === 'default'
// v0.32.8 F6: foreign-source pages land under brainDir/.sources/<id>/<slug>.md
// so same-slug-different-source pages don't collide on disk. Pages belonging
// to the cycle's own source (#1586: brainDir IS that source's checkout —
// legacy 'default' when unscoped) stay at brainDir/<slug>.md so single-source
// brains see no change. `.sources/` is a reserved prefix; walkBrainRepo skips dot-dirs.
const filePath = source_id === nativeSourceId
? join(brainDir, `${slug}.md`)
: join(brainDir, '.sources', source_id, `${slug}.md`);
mkdirSync(dirname(filePath), { recursive: true });
@@ -558,3 +572,11 @@ function failed(error: PhaseError): PhaseResult {
function makeError(cls: string, code: string, message: string, hint?: string): PhaseError {
return hint ? { class: cls, code, message, hint } : { class: cls, code, message };
}
// `__testing` re-exports otherwise-private helpers so unit tests can pin the
// source-scoping contract (#1586) without driving a whole dream cycle.
// Mirrors synthesize.ts's `__testing` block.
export const __testing = {
collectChildPutPageSlugs,
reverseWriteRefs,
};
src/core/cycle/synthesize.ts
+148 -37
View File
@@ -37,6 +37,8 @@ import { basename, join, dirname, isAbsolute, resolve } from 'node:path';
import type { BrainEngine } from '../engine.ts';
import type { PhaseResult, PhaseError } from '../cycle.ts';
import { MinionQueue } from '../minions/queue.ts';
import { reconnectAfterConnectionError } from '../minions/reconnect.ts';
import { isRetryableConnError } from '../retry-matcher.ts';
import { waitForCompletion, TimeoutError } from '../minions/wait-for-completion.ts';
import { makeSubagentHandler } from '../minions/handlers/subagent.ts';
import type { MinionJobInput, MinionJobContext, MinionHandler, SubagentHandlerData } from '../minions/types.ts';
@@ -276,39 +278,101 @@ export interface SynthesizePhaseOpts {
once?: boolean;
}
const INLINE_PGLITE_LOCK_MS = 30_000;
const INLINE_LOCK_MS = 30_000;
/**
* PGLite cannot be served by a separate Minions worker process: the embedded
* data-dir holds an exclusive file lock, so subagent children enqueued by the
* synth parent would sit in 'waiting' until waitForCompletion times out.
* Drive the same claim → run → complete/fail loop a worker would perform,
* inline, against this phase's private child queue.
* Drain this phase's private child queue inline: drive the same claim → run →
* complete/fail loop a worker would perform, from the parent's own slot.
*
* Why inline on BOTH engines:
* - PGLite: no separate Minions worker can run at all (the embedded
* data-dir holds an exclusive file lock), so children would sit in
* 'waiting' until waitForCompletion times out.
* - Postgres (#2050): the parent phase itself runs as a job inside a
* `jobs work` process. A worker whose slots are all occupied by such
* parents (autopilot spawns its drain worker at the default
* concurrency=1) can never claim the child the parent is blocking on —
* a structural self-deadlock. Running children inline means a child
* never needs a worker slot, so the deadlock is impossible at ANY
* concurrency, and no extra DB-pool pressure is added: the child's work
* replaces the parent's idle waitForCompletion polling in the slot the
* parent already holds.
*
* `yieldDuringPhase` is ticked on a 60s interval while a child runs so the
* 5-min cycle lock TTL keeps refreshing during long (up to 30-min) children.
* The child's own claim lock is heartbeated at lockMs/3 (worker cadence
* parity) — on Postgres a concurrent worker sweeps handleStalled() across
* ALL queues, so without renewal any child running longer than lockMs would
* be requeued mid-run and stall-churned to dead.
*/
export async function runPgliteSubagentsInline(
export async function runSubagentsInline(
engine: BrainEngine,
queue: MinionQueue,
queueName: string,
yieldDuringPhase?: () => Promise<void>,
handler: MinionHandler = makeSubagentHandler({ engine }),
lockMs: number = INLINE_LOCK_MS,
): Promise<void> {
if (engine.kind !== 'pglite') return;
// #3555 interaction: the drain's queue ops used to be bare awaits, so a
// transient pooler reap mid-drain threw out of the loop and stranded the
// remaining children in this per-run private queue — which no worker will
// ever claim. Mirror the worker's recovery: on a retryable connection
// error, rebuild the pool (shared reconnectAfterConnectionError) and retry
// the loop; non-retryable errors still propagate (real bug → phase fails).
const MAX_CONN_ERROR_STREAK = 5;
let connErrorStreak = 0;
let sawConnError = false;
const recoverOrThrow = async (site: string, e: unknown): Promise<void> => {
if (!isRetryableConnError(e) || ++connErrorStreak > MAX_CONN_ERROR_STREAK) throw e;
sawConnError = true;
const msg = e instanceof Error ? e.message : String(e);
process.stderr.write(`[dream] inline drain ${site} hit a connection error; reconnecting and retrying: ${msg}\n`);
await reconnectAfterConnectionError(engine, `inline-${site}`, e);
// Small cooperative backoff (setTimeout keeps the cycle-lock keepalive
// and any concurrent timers firing) before the loop retries.
await new Promise((r) => setTimeout(r, Math.min(1000, Math.max(50, Math.floor(lockMs / 3)))));
};
while (true) {
// Housekeeping a worker would normally perform, so child rows can reach
// terminal states (delayed retries promoted, timeouts dead-lettered)
// before the synth parent enters waitForCompletion polling.
await queue.promoteDelayed();
await queue.handleStalled();
await queue.handleTimeouts();
await queue.handleWallClockTimeouts(INLINE_PGLITE_LOCK_MS);
const lockToken = randomUUID();
const job = await queue.claim(lockToken, INLINE_PGLITE_LOCK_MS, queueName, ['subagent']);
if (!job) return;
let job: Awaited<ReturnType<MinionQueue['claim']>>;
try {
// Housekeeping a worker would normally perform, so child rows can reach
// terminal states (delayed retries promoted, timeouts dead-lettered)
// before the synth parent enters waitForCompletion polling.
await queue.promoteDelayed();
await queue.handleStalled();
await queue.handleTimeouts();
await queue.handleWallClockTimeouts(lockMs);
job = await queue.claim(lockToken, lockMs, queueName, ['subagent']);
} catch (e) {
await recoverOrThrow('queue-ops', e);
continue;
}
connErrorStreak = 0;
if (!job) {
if (!sawConnError) return;
// A connection-error window may have left a child 'active' under a
// lock nobody renews (a claim that committed but whose row never
// reached us, or a lost outcome write below). handleStalled() at the
// loop top requeues it once the lock expires (≤ lockMs), so only exit
// once the queue is actually quiet.
let active = 0;
try {
const rows = await engine.executeRaw<{ n: number }>(
`SELECT count(*)::int AS n FROM minion_jobs WHERE queue = $1 AND status = 'active'`,
[queueName],
);
active = rows[0]?.n ?? 0;
} catch (e) {
await recoverOrThrow('active-check', e);
continue;
}
if (active === 0) return;
await new Promise((r) => setTimeout(r, 1000));
continue;
}
const abort = new AbortController();
const shutdown = new AbortController();
@@ -364,18 +428,45 @@ export async function runPgliteSubagentsInline(
const keepalive = yieldDuringPhase
? setInterval(() => { yieldDuringPhase().catch(() => { /* best-effort */ }); }, 60_000)
: null;
// #2050: heartbeat the child's claim lock while the handler runs so a
// concurrent Postgres worker's handleStalled() sweep (all queues, not
// just its own) can't requeue a live child. A false return means the row
// was cancelled or reclaimed — abort the handler. Errors are swallowed
// (best-effort; the next tick retries), never an unhandledRejection.
const renewTimer = setInterval(() => {
queue.renewLock(job.id, lockToken, lockMs)
.then((ok) => {
if (!ok && !abort.signal.aborted) abort.abort(new Error('lock-renewal-failed'));
})
.catch(() => { /* best-effort; next tick retries */ });
}, Math.max(50, Math.floor(lockMs / 3)));
// Run, then record — separated so a completeJob connection error can't
// masquerade as a handler failure, and a failJob connection error can't
// escape the drain and strand the remaining children (worker.ts #1720
// parity: reconnect + retry the recording once; if it still fails, leave
// the row for the loop's own handleStalled to requeue after lock expiry).
let result: unknown;
let handlerErr: unknown;
let handlerRan = false;
try {
const result = await handler(context);
await queue.completeJob(
job.id,
lockToken,
result != null ? (typeof result === 'object' ? result as Record<string, unknown> : { value: result }) : undefined,
);
result = await handler(context);
handlerRan = true;
} catch (e) {
handlerErr = e;
}
const record = async (): Promise<void> => {
if (handlerRan) {
await queue.completeJob(
job.id,
lockToken,
result != null ? (typeof result === 'object' ? result as Record<string, unknown> : { value: result }) : undefined,
);
return;
}
// Timeout is terminal (handleTimeouts parity: stall → retry,
// timeout → dead), never a delayed retry.
const timedOut = abort.signal.aborted;
const errorText = timedOut ? 'timeout exceeded' : (e instanceof Error ? e.message : String(e));
const errorText = timedOut ? 'timeout exceeded' : (handlerErr instanceof Error ? handlerErr.message : String(handlerErr));
const attemptsExhausted = job.attempts_made + 1 >= job.max_attempts;
await queue.failJob(
job.id,
@@ -384,9 +475,30 @@ export async function runPgliteSubagentsInline(
timedOut || attemptsExhausted ? 'dead' : 'delayed',
0,
);
};
try {
try {
await record();
} catch (recordErr) {
if (!isRetryableConnError(recordErr)) throw recordErr;
sawConnError = true;
const msg = recordErr instanceof Error ? recordErr.message : String(recordErr);
process.stderr.write(`[dream] inline drain: recording job ${job.id} outcome hit a connection error; reconnecting and retrying once: ${msg}\n`);
await reconnectAfterConnectionError(engine, 'inline-record', recordErr);
try {
await record();
} catch (retryErr) {
// Leave the row to the loop's own handleStalled: the claim lock
// stops renewing (finally clears renewTimer), expires within
// lockMs, and the next iteration requeues it on a live pool.
const retryMsg = retryErr instanceof Error ? retryErr.message : String(retryErr);
process.stderr.write(`[dream] inline drain: outcome recording retry for job ${job.id} also failed (${retryMsg}); leaving the row for stall requeue\n`);
}
}
} finally {
if (timeoutTimer) clearTimeout(timeoutTimer);
if (keepalive) clearInterval(keepalive);
clearInterval(renewTimer);
}
}
}
@@ -572,12 +684,11 @@ export async function runPhaseSynthesize(
}
const queue = new MinionQueue(engine);
// PGLite children drain inline (no separate worker can open the embedded
// data-dir), so give them a private per-run queue: the inline drain must
// never claim unrelated 'default'-queue jobs a Postgres worker owns.
const childQueueName = engine.kind === 'pglite'
? `dream-inline-${Date.now()}-${randomUUID().slice(0, 8)}`
: 'default';
// #2050: children drain inline on BOTH engines (see runSubagentsInline),
// so give them a private per-run queue: the inline drain must never claim
// unrelated 'default'-queue jobs, and a 'default'-queue worker must never
// claim a child this parent is about to run itself.
const childQueueName = `dream-inline-${Date.now()}-${randomUUID().slice(0, 8)}`;
const childIds: number[] = [];
/** Map child job_id → chunk metadata for D6 orchestrator-side slug rewrite. */
const chunkInfo = new Map<number, { idx: number; hash6: string }>();
@@ -685,11 +796,11 @@ export async function runPhaseSynthesize(
}
}
// PGLite cannot run a separate Minions worker because the embedded DB
// holds an exclusive file lock. Drain this phase's private child queue
// inline so the parent observes terminal child states instead of polling
// waiters until subagentWaitTimeoutMs expires. No-op on Postgres.
await runPgliteSubagentsInline(engine, queue, childQueueName, opts.yieldDuringPhase);
// Drain this phase's private child queue inline so the parent observes
// terminal child states instead of polling waiters until
// subagentWaitTimeoutMs expires. Runs on BOTH engines — on Postgres the
// parent job otherwise deadlocks a fully-occupied worker (#2050).
await runSubagentsInline(engine, queue, childQueueName, opts.yieldDuringPhase);
// Wait for every child to reach a terminal state. Tick yieldDuringPhase
// every 5 min so the cycle lock TTL refreshes.
@@ -1709,6 +1820,6 @@ export const __testing = {
buildSynthesisPrompt,
stampDreamProvenance,
reverseWriteRefs,
runPgliteSubagentsInline,
runSubagentsInline,
loadSynthConfig,
};
src/core/doctor-categories.ts
+3
View File
@@ -99,6 +99,7 @@ export const BRAIN_CHECK_NAMES: ReadonlySet<string> = new Set([
'ocr_health',
'orphan_ratio',
'oversized_pages',
'pglite_scratch_probe',
'quarantined_pages',
'raw_provenance',
'flagged_pages',
@@ -152,6 +153,7 @@ export const OPS_CHECK_NAMES: ReadonlySet<string> = new Set([
'bootstrap_hooks_heartbeat',
'bootstrap_last_verify',
'bootstrap_push_health',
'bootstrap_durability_job',
'bootstrap_runbook_skew',
'bootstrap_serve_lock',
'batch_retry_health',
@@ -170,6 +172,7 @@ export const OPS_CHECK_NAMES: ReadonlySet<string> = new Set([
'pgvector',
'pool_budget',
'progressive_batch_audit_health',
'provider_sunset',
'queue_health',
'reranker_health',
'rls',
src/core/engine.ts
+30 -1
View File
@@ -1,6 +1,6 @@
import type {
Page, PageInput, PageFilters, GetPageOpts,
Chunk, ChunkInput, StaleChunkRow, StalePageRow,
Chunk, ChunkInput, StaleChunkRow, StalePageRow, ChunklessPageRow,
SearchResult, SearchOpts,
Link, GraphNode, GraphPath, RelationalFanoutRow, RelationalFanoutOpts,
TimelineEntry, TimelineInput, TimelineOpts,
@@ -1087,6 +1087,35 @@ export interface BrainEngine {
// common denominator on the wire).
afterUpdatedAt?: string | null;
}): Promise<StaleChunkRow[]>;
/**
* Pre-flight count for the chunkless-page safety net: pages with
* non-empty `compiled_truth` AND/OR non-empty `timeline` — both are
* chunked independently by the healer — and ZERO `content_chunks` rows.
* `embed --stale` only scans `content_chunks` (embedding IS NULL) — a
* page written directly via `putPage` that never got chunked has no
* chunk row to find, so it stays invisible to that scan forever.
* `opts.sourceId` scopes the count to a single source, matching
* `countStaleChunks`. Quarantined and `embed_skip` pages are excluded —
* both are intentionally chunkless by design, not drift needing repair.
* See `ChunklessPageRow` for the full rationale.
*/
countChunklessPagesWithContent(opts?: { sourceId?: string }): Promise<number>;
/**
* List pages with non-empty `compiled_truth` and/or `timeline` and zero
* `content_chunks` rows (sibling of `countChunklessPagesWithContent`;
* same predicate). Keyset-paginated on `id` (mirrors
* `listStalePagesForExtraction`) — pass the last row's `id` as
* `afterPageId` for the next page. Default `batchSize` 50 — deliberately
* small (unlike the 2000-row default on chunk-metadata-only cursors
* elsewhere): each row here carries a FULL page body, so a large batch
* of large pages is a real memory concern this is a safety-net sweep for
* a rare drift case, not the primary bulk-import chunking path.
*/
listChunklessPagesWithContent(opts?: {
batchSize?: number;
afterPageId?: number;
sourceId?: string;
}): Promise<ChunklessPageRow[]>;
/**
* Delete every chunk for a page. Internal page-id lookup is sourceId-scoped
* when `opts.sourceId` is given; otherwise the bare-slug subquery returns
src/core/execution-env.ts
+93
View File
@@ -0,0 +1,93 @@
/**
* Execution-environment detection — the third bootstrap axis.
*
* `detectHarness` (bootstrap.ts) answers WHICH AGENT HOST (claude-code vs
* codex). This module answers WHERE THAT HOST IS RUNNING:
*
* - `cloud-sandbox` → a hosted, reclaimed-after-inactivity VM behind a
* credential-injecting egress proxy (Claude Code on
* the web and lookalikes). No crontab, no surviving
* background processes; GitHub REST is scoped to
* session-attached repos and GraphQL is pinned to a
* fixed operation set.
* - `ephemeral-container` → Docker/Render/Railway/Fly-class containers.
* Long-lived enough for normal hook cadence, but
* schedulers (crontab/launchd) are unreliable or
* absent (wiped on deploy).
* - `local` → a normal machine. Everything works.
*
* Consumers branch persistence strategy (cron install vs event-driven pushes),
* push-cadence defaults, and error messaging on this value. Detection is pure
* and signal-injected so tests never depend on the machine running them.
*/
import { existsSync } from 'node:fs';
export type ExecutionEnvironment = 'local' | 'cloud-sandbox' | 'ephemeral-container';
/** Injectable probe signals (production defaults: process.env + existsSync). */
export interface EnvProbeSignals {
env?: Record<string, string | undefined>;
fileExists?: (p: string) => boolean;
}
/**
* True when outbound credentials are substituted by a proxy rather than held
* locally — the Claude Code cloud signature. Signals (any one suffices):
*
* - GH_TOKEN / GITHUB_TOKEN carry the documented literal `proxy-injected`
* placeholder (the proxy attaches real credentials on the wire).
* - The https proxy URL carries an anthropic-egress-control JWT.
*
* Load-bearing caveat for verification code: inside such an environment an
* "anonymous" HTTP probe may be silently authenticated by the proxy, so
* anonymous-probe results must be treated as ambiguous (see repo-visibility).
*/
export function isCredentialInjectingProxy(
env: Record<string, string | undefined> = process.env,
): boolean {
if (env.GH_TOKEN === 'proxy-injected' || env.GITHUB_TOKEN === 'proxy-injected') return true;
const proxy = env.https_proxy ?? env.HTTPS_PROXY ?? '';
return /anthropic-egress/i.test(proxy);
}
/**
* Detect where this process is running. Order matters: the cloud sandbox is
* ALSO a container, so its signals are checked first.
*
* 1. `CLAUDE_CODE_REMOTE === 'true'` — official, never true locally.
* 2. `CLAUDE_CODE_REMOTE_SESSION_ID` with the documented `cse_` prefix.
* 3. A credential-injecting proxy signature (see above).
* 4. Container platforms: RENDER / RAILWAY_ENVIRONMENT / FLY_APP_NAME env,
* or the /.dockerenv marker file (same signal set autopilot's
* detectInstallTarget has used for its ephemeral branch).
* 5. Otherwise: local.
*/
export function detectExecutionEnvironment(signals: EnvProbeSignals = {}): ExecutionEnvironment {
const env = signals.env ?? process.env;
const fileExists = signals.fileExists ?? existsSync;
if (env.CLAUDE_CODE_REMOTE === 'true') return 'cloud-sandbox';
if ((env.CLAUDE_CODE_REMOTE_SESSION_ID ?? '').startsWith('cse_')) return 'cloud-sandbox';
if (isCredentialInjectingProxy(env)) return 'cloud-sandbox';
if (
env.RENDER ||
env.RAILWAY_ENVIRONMENT ||
env.FLY_APP_NAME ||
fileExists('/.dockerenv')
) {
return 'ephemeral-container';
}
return 'local';
}
/** Whether `name` resolves on PATH. Moved here from bootstrap/status.ts so
* environment-aware code (cron install, preflight) shares one probe. The
* LIVE process.env.PATH is passed explicitly: Bun otherwise resolves against
* the startup env snapshot, making runtime PATH changes (and PATH-shimmed
* test fakes) invisible — the workspace-push.ts / status.ts precedent. */
export function binaryOnPath(name: string): boolean {
try {
return Bun.which(name, { PATH: process.env.PATH ?? '' }) !== null;
} catch {
return false;
}
}
src/core/git-remote.ts
+1 -1
View File
@@ -393,7 +393,7 @@ export function hasTrackedContent(path: string): boolean {
* transport. Default stays `never`. These ops act on an ALREADY-validated origin
* (set + checked at clone time); `http.followRedirects=false` is the live guard.
*/
function durableSsrfFlags(): string[] {
export function durableSsrfFlags(): string[] {
const fileAllow = process.env.GBRAIN_GIT_ALLOW_FILE_TRANSPORT === '1' ? 'always' : 'never';
return [
'-c', 'http.followRedirects=false',
src/core/minions/reconnect.ts
+24
View File
@@ -0,0 +1,24 @@
/**
* Shared "rebuild the DB pool after a retryable connection failure" helper.
*
* PostgresEngine exposes reconnect(); PGLite and test doubles may not. Absence
* is a no-op so non-Postgres callers preserve their legacy behavior.
*
* Extracted from MinionWorker's private method (#1491/#3555) so the inline
* child drain (#2050, cycle/synthesize.ts runSubagentsInline) recovers from
* the same transient pooler reaps instead of throwing out of the drain and
* stranding children in a per-run queue no worker will ever claim.
*/
export async function reconnectAfterConnectionError(
engine: unknown,
site: string,
error: unknown,
): Promise<void> {
const reconnect = (engine as { reconnect?: (ctx?: { error?: unknown }) => Promise<void> }).reconnect;
if (!reconnect) return;
try {
await reconnect.call(engine, { error });
} catch (re) {
console.error(`[minions] reconnect after ${site} error failed: ${re instanceof Error ? re.message : String(re)}`);
}
}
src/core/minions/supervisor.ts
+8 -2
View File
@@ -42,7 +42,7 @@ import {
unlinkSync,
writeSync,
} from 'fs';
import { dirname } from 'path';
import { dirname, resolve } from 'path';
import type { BrainEngine } from '../engine.ts';
import { tryAcquireDbLock, type DbLockHandle } from '../db-lock.ts';
import { currentBrainId } from './worker-registry.ts';
@@ -574,7 +574,13 @@ export class MinionSupervisor {
// 5. Announce start.
this.emit('started', {
supervisor_pid: process.pid,
pid_file: this.opts.pidFile,
// Resolved to absolute at emit time (relative to THIS process's cwd,
// the only context in which a relative --pid-file was meaningful) so a
// later reader (e.g. `gbrain doctor`, possibly running from a
// different cwd) doesn't misresolve it. `this.opts.pidFile` itself
// stays as-given for this process's own reads/writes below, which are
// already correctly relative to this same cwd.
pid_file: resolve(this.opts.pidFile),
concurrency: this.opts.concurrency,
queue: this.opts.queue,
max_crashes: this.opts.maxCrashes,
src/core/minions/worker.ts
+3 -10
View File
@@ -31,6 +31,7 @@ import {
} from './lock-renewal-tick.ts';
import { lockRenewalAudit } from '../audit/lock-renewal-audit.ts';
import { isRetryableConnError } from '../retry-matcher.ts';
import { reconnectAfterConnectionError as reconnectEngineAfterConnError } from './reconnect.ts';
/**
* Abort reasons that signal infrastructure failure (PgBouncer outage,
@@ -735,18 +736,10 @@ export class MinionWorker extends EventEmitter {
/**
* Rebuild the worker-owned DB pool after a retryable connection failure.
*
* PostgresEngine exposes reconnect(); PGLite and test doubles may not. Absence
* is a no-op so non-Postgres workers preserve their legacy behavior.
* Shared with the inline child drain (#2050) via minions/reconnect.ts.
*/
private async reconnectAfterConnectionError(site: string, error: unknown): Promise<void> {
const reconnect = (this.engine as { reconnect?: (ctx?: { error?: unknown }) => Promise<void> }).reconnect;
if (!reconnect) return;
try {
await reconnect.call(this.engine, { error });
} catch (re) {
console.error(`[worker] reconnect after ${site} error failed: ${re instanceof Error ? re.message : String(re)}`);
}
await reconnectEngineAfterConnError(this.engine, site, error);
}
/** RSS watchdog. Called from the per-job finally and the periodic timer.
src/core/oauth-provider.ts
+123 -17
View File
@@ -13,6 +13,7 @@
* - Legacy access_tokens fallback for backward compat
*/
import { AsyncLocalStorage } from 'node:async_hooks';
import type { Response } from 'express';
import type {
OAuthClientInformationFull,
@@ -235,14 +236,75 @@ interface GBrainOAuthProviderOptions {
* (operator-trusted, registers grants directly).
*/
allowClientCredentialsDcr?: boolean;
/**
* #2179: lower bound (seconds) for DCR-requested per-client token TTLs.
* Requests below it clamp up. Default DEFAULT_DCR_TTL_MIN_SECONDS (300).
*/
dcrTtlMinSeconds?: number;
/**
* #2179: upper bound (seconds) for DCR-requested per-client token TTLs.
* Requests above it clamp down. Unset defaults FAIL-CLOSED to
* max(tokenTtl, dcrTtlMinSeconds): an anonymous DCR registrant can never
* elect a longer-lived token than the operator's own --token-ttl unless
* the admin explicitly widened the window.
*/
dcrTtlMaxSeconds?: number;
}
// ---------------------------------------------------------------------------
// DCR token TTL (#2179)
// ---------------------------------------------------------------------------
/**
* Default lower clamp bound for DCR-requested token TTLs (#2179). Admins
* override via the `oauth.dcr_ttl_min_seconds` / `oauth.dcr_ttl_max_seconds`
* config keys, read once by `gbrain serve --http` at startup. There is
* deliberately NO fixed default max: an unset max derives fail-closed from
* the operator's --token-ttl (`max(tokenTtl, min)`), so a self-registering
* client can never out-live the server default without explicit admin opt-in.
*/
export const DEFAULT_DCR_TTL_MIN_SECONDS = 300; // 5 minutes
/**
* Clamp a DCR-requested token TTL into the admin-configured [min, max]
* window. Bounds are REQUIRED — callers resolve them (fail-closed) first.
* Never rejects (#2179): out-of-range values clamp to the nearest bound.
* Non-integer requests floor; an inverted window collapses to the min bound.
*/
export function clampDcrTokenTtl(
requested: number,
min: number,
max: number,
): number {
const lo = Math.max(1, Math.floor(min));
const hi = Math.max(lo, Math.floor(max));
return Math.min(hi, Math.max(lo, Math.floor(requested)));
}
/**
* Request-scoped carrier for the `token_ttl_seconds` DCR extension field
* (#2179). The MCP SDK's /register handler validates the request body against
* a strict schema and STRIPS unknown members before they reach
* `clientsStore.registerClient`, so serve-http's /register middleware parses
* the raw body and runs the SDK chain inside this AsyncLocalStorage context;
* the store reads it back out at registration time. No context (CLI, admin
* API, programmatic registration) means "no TTL request" — default behavior.
*/
export const dcrRegistrationContext = new AsyncLocalStorage<{ tokenTtlSeconds?: number }>();
// ---------------------------------------------------------------------------
// Clients Store
// ---------------------------------------------------------------------------
class GBrainClientsStore implements OAuthRegisteredClientsStore {
constructor(private sql: SqlQuery, private allowClientCredentialsDcr = false) {}
// #2179: DCR TTL bounds are required — the provider resolves fail-closed
// defaults (max bounded by tokenTtl); no permissive fallback lives here.
constructor(
private sql: SqlQuery,
private allowClientCredentialsDcr: boolean,
private dcrTtlMin: number,
private dcrTtlMax: number,
) {}
async getClient(clientId: string): Promise<OAuthClientInformationFull | undefined> {
const rows = await this.sql`
@@ -392,6 +454,27 @@ class GBrainClientsStore implements OAuthRegisteredClientsStore {
}
}
// #2179: optional `token_ttl_seconds` hint from the DCR request body,
// carried via dcrRegistrationContext (the SDK strips unknown body
// members). Fail-safe posture: absent or malformed → server default TTL;
// out-of-range → clamped into [dcrTtlMin, dcrTtlMax]; never rejected.
// Persist into oauth_clients.token_ttl (the same per-client override the
// admin API writes) and echo the EFFECTIVE value in the registration
// response so the caller can show the user what it actually got.
let effectiveTtl: number | undefined;
const requestedTtl = dcrRegistrationContext.getStore()?.tokenTtlSeconds;
if (typeof requestedTtl === 'number' && Number.isFinite(requestedTtl)) {
const clamped = clampDcrTokenTtl(requestedTtl, this.dcrTtlMin, this.dcrTtlMax);
try {
await this.sql`UPDATE oauth_clients SET token_ttl = ${clamped} WHERE client_id = ${clientId}`;
effectiveTtl = clamped;
} catch (e) {
// Pre-migration schema without the token_ttl column: keep the
// registration, but do NOT echo a TTL that wasn't persisted.
if (!isUndefinedColumnError(e, 'token_ttl')) throw e;
}
}
// Public clients: omit `client_secret` entirely from the response so
// the wire payload matches RFC 7591 §3.2.1 ("if the client is a
// public client, the authorization server MUST NOT issue a client
@@ -403,6 +486,9 @@ class GBrainClientsStore implements OAuthRegisteredClientsStore {
client_id_issued_at: now,
};
if (clientSecret) response.client_secret = clientSecret;
if (effectiveTtl !== undefined) {
(response as Record<string, unknown>).token_ttl_seconds = effectiveTtl;
}
return response;
}
}
@@ -420,10 +506,21 @@ export class GBrainOAuthProvider implements OAuthServerProvider {
constructor(options: GBrainOAuthProviderOptions) {
this.sql = options.sql;
this._clientsStore = new GBrainClientsStore(this.sql, options.allowClientCredentialsDcr === true);
this.dcrDisabled = options.dcrDisabled === true;
this.tokenTtl = options.tokenTtl || 3600;
this.refreshTtl = options.refreshTtl || 30 * 24 * 3600;
// #2179 fail-closed: an unset DCR max is bounded by the operator's own
// token TTL — never a fixed permissive ceiling — so a self-registering
// client cannot elect a longer-lived token than the server default
// unless the admin explicitly configured a wider window.
const dcrTtlMin = options.dcrTtlMinSeconds ?? DEFAULT_DCR_TTL_MIN_SECONDS;
const dcrTtlMax = options.dcrTtlMaxSeconds ?? Math.max(this.tokenTtl, dcrTtlMin);
this._clientsStore = new GBrainClientsStore(
this.sql,
options.allowClientCredentialsDcr === true,
dcrTtlMin,
dcrTtlMax,
);
}
get clientsStore(): OAuthRegisteredClientsStore {
@@ -931,20 +1028,10 @@ export class GBrainOAuthProvider implements OAuthServerProvider {
const requestedScopes = requestedScope ? parseScopeString(requestedScope) : allowedScopes;
const grantedScopes = requestedScopes.filter(s => hasScope(allowedScopes, s));
// Per-client TTL override (stored in oauth_clients.token_ttl)
// Column may not exist on PGLite/older schemas — graceful fallback
let clientTtl: number | undefined;
try {
const ttlRows = await this.sql`SELECT token_ttl FROM oauth_clients WHERE client_id = ${clientId}`;
if (ttlRows.length > 0 && ttlRows[0].token_ttl) clientTtl = Number(ttlRows[0].token_ttl);
} catch (e) {
// F5 hardening: same posture as the deleted_at probe above. Only the
// "column doesn't exist" path is a non-fatal fall-through.
if (!isUndefinedColumnError(e, 'token_ttl')) throw e;
}
// Client credentials: access token only, NO refresh token (RFC 6749 4.4.3)
return this.issueTokens(clientId, grantedScopes, undefined, false, clientTtl);
// Per-client TTL (oauth_clients.token_ttl) is applied inside issueTokens
// so all three grant paths honor it (#2179).
return this.issueTokens(clientId, grantedScopes, undefined, false);
}
// -------------------------------------------------------------------------
@@ -1204,17 +1291,36 @@ export class GBrainOAuthProvider implements OAuthServerProvider {
// Internal: Issue access + optional refresh tokens
// -------------------------------------------------------------------------
/**
* Per-client TTL override lookup (oauth_clients.token_ttl). Set by the
* admin API, the CLI, or a DCR `token_ttl_seconds` request (#2179).
* Column may not exist on older schemas — graceful fallback to undefined.
*/
private async lookupClientTokenTtl(clientId: string): Promise<number | undefined> {
try {
const ttlRows = await this.sql`SELECT token_ttl FROM oauth_clients WHERE client_id = ${clientId}`;
if (ttlRows.length > 0 && ttlRows[0].token_ttl) return Number(ttlRows[0].token_ttl);
} catch (e) {
// F5 hardening posture: only the "column doesn't exist" path is a
// non-fatal fall-through.
if (!isUndefinedColumnError(e, 'token_ttl')) throw e;
}
return undefined;
}
private async issueTokens(
clientId: string,
scopes: string[],
resource: URL | undefined,
includeRefresh: boolean,
ttlOverride?: number,
): Promise<OAuthTokens> {
const accessToken = generateToken('gbrain_at_');
const accessHash = hashToken(accessToken);
const now = Math.floor(Date.now() / 1000);
const effectiveTtl = ttlOverride || this.tokenTtl;
// #2179: the per-client override lives here (not in individual grant
// handlers) so client_credentials, authorization_code AND refresh
// issuance all honor oauth_clients.token_ttl consistently.
const effectiveTtl = (await this.lookupClientTokenTtl(clientId)) || this.tokenTtl;
const accessExpiry = now + effectiveTtl;
await this.sql`
src/core/operations.ts
+5 -2
View File
@@ -10,7 +10,7 @@ import { clampSearchLimit } from './engine.ts';
import type { GBrainConfig } from './config.ts';
import type { PageType } from './types.ts';
import { importFromContent } from './import-file.ts';
import { writePageThrough } from './write-through.ts';
import { writePageThrough, type WriteThroughResult } from './write-through.ts';
import { hybridSearch, hybridSearchCached, stampContentFlags, stampUnverifiedExtractions } from './search/hybrid.ts';
import { expandQuery } from './search/expansion.ts';
import { dedupResults } from './search/dedup.ts';
@@ -1323,7 +1323,10 @@ const put_page: Operation = {
// Trust gating:
// - Subagent sandbox (viaSubagent without allowedSlugPrefixes) → DB-only.
// - All other writes → write-through.
let writeThrough: { written: boolean; path?: string; skipped?: string; error?: string } | undefined;
// put_page's own trust-gating produces two skip reasons ('subagent_sandbox',
// 'dry_run') that never come out of writePageThrough itself — widen the
// field rather than losing the commit/pushed/lastPushStatus typing.
let writeThrough: (Omit<WriteThroughResult, 'skipped'> & { skipped?: WriteThroughResult['skipped'] | 'subagent_sandbox' | 'dry_run' }) | undefined;
const isSandboxSubagent = ctx.viaSubagent === true
&& !(Array.isArray(ctx.allowedSlugPrefixes) && ctx.allowedSlugPrefixes.length > 0);
if (!ctx.dryRun && result.status !== 'error' && !isSandboxSubagent) {
src/core/pglite-engine.ts
+164 -1
View File
@@ -1,5 +1,10 @@
import { PGlite } from '@electric-sql/pglite';
import type { Transaction } from '@electric-sql/pglite';
// Engine-live path: static top-level imports (scratch probe, #2674) — the
// engine-dynamic-import guard forbids lazy `import()` here.
import { mkdtemp, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join as joinPath, resolve as resolvePath, sep as pathSep } from 'node:path';
// Engine-live path: static top-level import (no lazy `import()`). Supplies
// PGLite's WASM/fsBundle/extension assets embedded via `with { type: 'file' }`
// so a `bun build --compile` binary can serve a PGLite brain (Bun vfs #1340).
@@ -55,7 +60,7 @@ import { attemptWalRepairAndRetry, closeRepairEpisodeIfOpen, type WalRepairRecei
import { getFtsLanguage } from './fts-language.ts';
import type {
Page, PageInput, PageFilters, PageType,
Chunk, ChunkInput, StaleChunkRow, StalePageRow,
Chunk, ChunkInput, StaleChunkRow, StalePageRow, ChunklessPageRow,
SearchResult, SearchOpts,
Link, GraphNode, GraphPath,
TimelineEntry, TimelineInput, TimelineOpts,
@@ -87,6 +92,8 @@ import { buildSourceFactorCase, buildHardExcludeClause, buildVisibilityClause, b
import { unverifiedExtractionFragment } from './extraction-review.ts';
import { shouldExcludeFromOrphanReporting, loadOrphanPolicyOverrides } from './orphan-policy.ts';
import { LINK_EXTRACTOR_VERSION_TS } from './link-extraction.ts';
import { EMBED_SKIP_FILTER_FRAGMENT } from './embed-skip.ts';
import { QUARANTINE_FILTER_FRAGMENT } from './quarantine.ts';
import {
normalizeEngineColumn,
buildVectorCastFragment,
@@ -420,6 +427,95 @@ async function preservingProcessExitCode<T>(fn: () => Promise<T>): Promise<T> {
}
}
/**
* #2674 — the scratch-store probe, the diagnostic half of the issue.
*
* PGLite reports only `Aborted()` to JS and prints the real PANIC (e.g.
* `could not locate a valid checkpoint record`) to its own stderr, so from
* the JS-visible error alone a damaged store is indistinguishable from a
* broken WASM runtime. The one thing that CAN tell them apart is opening a
* throwaway store on the same machine:
*
* - scratch store works → the runtime is healthy; the REAL store is damaged.
* - scratch store fails too → the runtime cannot start here at all.
*
* Stderr capture: PGLite 0.4.3 exposes no print/printErr hook on
* `PGliteOptions` (checked: only `debug`, which still writes to the
* process's own stderr), so we deliberately do NOT try to intercept the
* PANIC text — monkey-patching process.stderr.write around an async WASM
* init is exactly the hack the classifier comments warn against. The
* probe's ok/fail outcome carries the diagnosis instead; `verdict` is
* populated from the JS-visible error for callers that want it.
*
* Runs the SAME code path as the real engine (PGlite.create with the
* embedded WASM/extension assets) but deliberately NOT PGLiteEngine.connect():
* connect wraps failures in buildPgliteInitErrorMessage, whose hint text
* would then pollute re-classification of the probe error.
*
* Safety: the scratch dir comes from mkdtemp under os.tmpdir() and is
* additionally checked against `realStorePath` (refuses any overlap in
* either direction) — a bug here must never touch the brain being
* diagnosed. The dir is removed in a finally, success or failure.
*/
export interface PgliteScratchProbeResult {
ok: boolean;
duration_ms: number;
/** JS-visible error when ok=false (the PANIC itself lands on stderr, not here). */
error?: string;
verdict?: PgliteInitFailure;
}
export async function probePgliteScratchStore(
realStorePath?: string,
): Promise<PgliteScratchProbeResult> {
const scratchDir = await mkdtemp(joinPath(tmpdir(), 'gbrain-pglite-probe-'));
if (realStorePath) {
const real = resolvePath(realStorePath);
const scratch = resolvePath(scratchDir);
if (scratch === real || scratch.startsWith(real + pathSep) || real.startsWith(scratch + pathSep)) {
await rm(scratchDir, { recursive: true, force: true }).catch(() => {});
throw new Error(
`refusing to probe: scratch dir ${scratch} overlaps the real store ${real}`,
);
}
}
const started = Date.now();
let db: PGlite | null = null;
try {
// Same assets as the real engine's connect(): the embedded WASM/fsBundle/
// extension options (Bun vfs #1340) — a compiled binary's probe must
// exercise the same runtime path the real store open uses.
const embedded = await getEmbeddedPgliteOptions();
db = await preservingProcessExitCode(() =>
PGlite.create({
dataDir: joinPath(scratchDir, 'store'),
...embedded,
}),
);
await db.query(`CREATE TABLE scratch_probe (id int PRIMARY KEY, note text)`);
await db.query(`INSERT INTO scratch_probe VALUES (1, 'ok')`);
const res = await db.query<{ note: string }>(`SELECT note FROM scratch_probe WHERE id = 1`);
if (res.rows[0]?.note !== 'ok') {
throw new Error(`scratch store read-back mismatch: ${JSON.stringify(res.rows)}`);
}
return { ok: true, duration_ms: Date.now() - started };
} catch (err) {
const message = stringifyPgliteInitError(err);
return {
ok: false,
duration_ms: Date.now() - started,
error: message,
verdict: classifyPgliteInitError(message),
};
} finally {
if (db) {
try { await db.close(); } catch { /* probe store — nothing to save */ }
}
await rm(scratchDir, { recursive: true, force: true }).catch(() => {});
}
}
export class PGLiteEngine implements BrainEngine {
readonly kind = 'pglite' as const;
private _db: PGLiteDB | null = null;
@@ -2945,6 +3041,73 @@ export class PGLiteEngine implements BrainEngine {
return rows as unknown as StaleChunkRow[];
}
/**
* Shared chunkless-page-with-content predicate (mirrors PostgresEngine).
* Excludes quarantined + embed_skip pages — both are intentionally
* chunkless by design, not drift the safety net should repair.
*/
private buildChunklessPagesWhere(opts?: { sourceId?: string }): { where: string; params: unknown[] } {
const conds: string[] = [
'p.deleted_at IS NULL',
// healChunklessPages chunks BOTH compiled_truth and timeline (mirrors
// embedPage) — a timeline-only page (rare but schema-legal) has
// something to heal even with compiled_truth = ''.
`(p.compiled_truth <> '' OR p.timeline <> '')`,
EMBED_SKIP_FILTER_FRAGMENT,
QUARANTINE_FILTER_FRAGMENT,
'NOT EXISTS (SELECT 1 FROM content_chunks cc WHERE cc.page_id = p.id)',
];
const params: unknown[] = [];
if (opts?.sourceId) {
params.push(opts.sourceId);
conds.push(`p.source_id = $${params.length}`);
}
return { where: conds.join(' AND '), params };
}
async countChunklessPagesWithContent(opts?: { sourceId?: string }): Promise<number> {
const { where, params } = this.buildChunklessPagesWhere(opts);
const { rows } = await this.db.query(
`SELECT count(*)::int AS count FROM pages p WHERE ${where}`,
params,
);
const count = (rows[0] as { count: number } | undefined)?.count ?? 0;
return Number(count);
}
async listChunklessPagesWithContent(opts?: {
batchSize?: number;
afterPageId?: number;
sourceId?: string;
}): Promise<ChunklessPageRow[]> {
const { where, params } = this.buildChunklessPagesWhere(opts);
let afterClause = '';
if (opts?.afterPageId != null) {
params.push(opts.afterPageId);
afterClause = ` AND p.id > $${params.length}`;
}
// Small default (unlike the 2000-row chunk-metadata cursors elsewhere):
// each row here carries a FULL page body. See engine.ts docstring.
const limit = opts?.batchSize ?? 50;
params.push(limit);
const limitIdx = params.length;
const { rows } = await this.db.query(
`SELECT p.id, p.slug, p.source_id, p.compiled_truth, p.timeline
FROM pages p
WHERE ${where}${afterClause}
ORDER BY p.id
LIMIT $${limitIdx}`,
params,
);
return (rows as Record<string, unknown>[]).map(r => ({
id: r.id as number,
slug: r.slug as string,
source_id: (r.source_id as string | undefined) ?? 'default',
compiled_truth: (r.compiled_truth as string | null) ?? '',
timeline: (r.timeline as string | null) ?? '',
}));
}
async deleteChunks(slug: string, opts?: { sourceId?: string }): Promise<void> {
const sourceId = opts?.sourceId ?? 'default';
// Source-qualify the page-id subquery; slugs are only unique per source.
src/core/postgres-engine.ts
+75 -1
View File
@@ -60,7 +60,7 @@ import { getFtsLanguage, applyFtsLanguagePolicy } from './fts-language.ts';
import { MARKDOWN_CHUNKER_VERSION } from './chunkers/recursive.ts';
import type {
Page, PageInput, PageFilters, PageType,
Chunk, ChunkInput, StaleChunkRow, StalePageRow,
Chunk, ChunkInput, StaleChunkRow, StalePageRow, ChunklessPageRow,
SearchResult, SearchOpts,
Link, GraphNode, GraphPath,
TimelineEntry, TimelineInput, TimelineOpts,
@@ -93,6 +93,8 @@ import { DELETE_BATCH_SIZE } from './engine-constants.ts';
import { SOURCE_CONFIG_OBJECT_SQL } from './source-config-sql.ts';
import { shouldExcludeFromOrphanReporting, loadOrphanPolicyOverrides } from './orphan-policy.ts';
import { LINK_EXTRACTOR_VERSION_TS } from './link-extraction.ts';
import { EMBED_SKIP_FILTER_FRAGMENT } from './embed-skip.ts';
import { QUARANTINE_FILTER_FRAGMENT } from './quarantine.ts';
function escapeSqlStringLiteral(value: string): string {
return value.replace(/'/g, "''");
@@ -2878,6 +2880,78 @@ export class PostgresEngine implements BrainEngine {
});
}
/**
* Shared chunkless-page-with-content predicate (mirrors PGLiteEngine).
* Excludes quarantined + embed_skip pages — both are intentionally
* chunkless by design, not drift the safety net should repair.
*/
private buildChunklessPagesWhere(opts?: { sourceId?: string }): { where: string; params: unknown[] } {
const conds: string[] = [
'p.deleted_at IS NULL',
// healChunklessPages chunks BOTH compiled_truth and timeline (mirrors
// embedPage) — a timeline-only page (rare but schema-legal) has
// something to heal even with compiled_truth = ''.
`(p.compiled_truth <> '' OR p.timeline <> '')`,
EMBED_SKIP_FILTER_FRAGMENT,
QUARANTINE_FILTER_FRAGMENT,
'NOT EXISTS (SELECT 1 FROM content_chunks cc WHERE cc.page_id = p.id)',
];
const params: unknown[] = [];
if (opts?.sourceId) {
params.push(opts.sourceId);
conds.push(`p.source_id = $${params.length}`);
}
return { where: conds.join(' AND '), params };
}
async countChunklessPagesWithContent(opts?: { sourceId?: string }): Promise<number> {
const { where, params } = this.buildChunklessPagesWhere(opts);
// RLS scope binding (opt-in via GBRAIN_RLS_SCOPE_BINDING).
return await this.withScopedReadTransaction(undefined, opts?.sourceId, async (tx) => {
const rows = await tx.unsafe(
`SELECT count(*)::int AS count FROM pages p WHERE ${where}`,
params as Parameters<typeof tx.unsafe>[1],
);
return Number((rows[0] as { count?: number } | undefined)?.count ?? 0);
});
}
async listChunklessPagesWithContent(opts?: {
batchSize?: number;
afterPageId?: number;
sourceId?: string;
}): Promise<ChunklessPageRow[]> {
const { where, params } = this.buildChunklessPagesWhere(opts);
let afterClause = '';
if (opts?.afterPageId != null) {
params.push(opts.afterPageId);
afterClause = ` AND p.id > $${params.length}`;
}
// Small default (unlike the 2000-row chunk-metadata cursors elsewhere):
// each row here carries a FULL page body. See engine.ts docstring.
const limit = opts?.batchSize ?? 50;
params.push(limit);
const limitIdx = params.length;
// RLS scope binding (opt-in via GBRAIN_RLS_SCOPE_BINDING).
return await this.withScopedReadTransaction(undefined, opts?.sourceId, async (tx) => {
const rows = await tx.unsafe(
`SELECT p.id, p.slug, p.source_id, p.compiled_truth, p.timeline
FROM pages p
WHERE ${where}${afterClause}
ORDER BY p.id
LIMIT $${limitIdx}`,
params as Parameters<typeof tx.unsafe>[1],
);
return (rows as Record<string, unknown>[]).map(r => ({
id: r.id as number,
slug: r.slug as string,
source_id: (r.source_id as string | undefined) ?? 'default',
compiled_truth: (r.compiled_truth as string | null) ?? '',
timeline: (r.timeline as string | null) ?? '',
}));
});
}
async deleteChunks(slug: string, opts?: { sourceId?: string }): Promise<void> {
const sql = this.sql;
const sourceId = opts?.sourceId ?? 'default';
src/core/repo-visibility.ts
+449
View File
@@ -0,0 +1,449 @@
/**
* repo-visibility.ts — ONE repo-visibility verdict for every consumer
* (workspace-push gate [G8], bootstrap repo/verify/status probes), replacing
* three drifted probes (TODOS ~5097). Engine-free; every IO seam injectable.
*
* Why a ladder and not one gh call: the previous probe (`gh repo view`, a
* GraphQL call) is structurally unavailable in cloud sandboxes — their GitHub
* proxy pins GraphQL to a fixed operation set and 403s everything else, even
* with a user-supplied token — which silently converted every hook push into
* `refused_visibility`. REST is the sanctioned surface, and pure git protocol
* works anywhere `git push` works.
*
* rung 1 REST `gh api repos/{owner}/{repo}` → .private (github.com)
* rung 2 authed `git ls-remote <url> HEAD` → exists + readable
* rung 3 anon GET <repo>/info/refs?service=git-upload-pack, no auth
*
* Verdict matrix (fail-closed BOTH directions — see [D4]/[D14] below):
* rest true → private/rest
* rest false → public/rest (refuse)
* rung2 ok + rung3 attributed 401 → private/git-protocol
* rung2 ok + rung3 proven 200 → public/git-protocol (refuse) — unless a
* credential-injecting proxy makes the
* "anonymous" probe ambiguous → unverifiable
* anything else → unverifiable (refuse, with rung log)
*
* [D4] A 200 counts as PUBLIC only with advertisement proof (content-type
* `application/x-git-upload-pack-advertisement` or the pkt-line body
* prefix). SSO-fronted self-hosted servers answer 200 with an HTML
* login page — that must read `unverifiable`, never a false "PUBLIC".
* [D14] A 401/404 counts as PRIVATE-signal only with origin attribution
* (`www-authenticate` challenge; github.com additionally recognized by
* realm/request-id headers). The private verdict AUTHORIZES pushing, so
* an auth-demanding middlebox that 401s all anonymous traffic must not
* launder a public repo into "private".
*/
import { mkdirSync, readFileSync, renameSync, writeFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { ensureGbrainHome } from './gbrain-home.ts';
import { durableSsrfFlags } from './git-remote.ts';
import { isCredentialInjectingProxy } from './execution-env.ts';
// ── Subprocess seam (canonical home; bootstrap/repo.ts re-exports) ─────────
export interface ExecResult {
code: number;
stdout: string;
stderr: string;
}
/** Injectable subprocess seam. argv[0] is the binary; never a shell string. */
export type ExecRunner = (argv: string[]) => Promise<ExecResult>;
/** Default runner: Bun.spawn, both streams piped, spawn failure → code 127.
* GIT_TERMINAL_PROMPT=0 so an unauthenticated git can never hang on a prompt. */
export const defaultRunner: ExecRunner = async (argv: string[]): Promise<ExecResult> => {
try {
const proc = Bun.spawn(argv, {
stdout: 'pipe',
stderr: 'pipe',
stdin: 'ignore',
env: { ...process.env, GIT_TERMINAL_PROMPT: '0' },
});
const [stdout, stderr, code] = await Promise.all([
new Response(proc.stdout).text(),
new Response(proc.stderr).text(),
proc.exited,
]);
return { code, stdout, stderr };
} catch (e) {
return { code: 127, stdout: '', stderr: (e as Error).message };
}
};
/** Race a runner call against a wall-clock cap. On timeout the rung reports
* code 124 and the ladder degrades — a hung network probe must never hang a
* push child. (The raced process is left to die on its own; every rung's
* default runner disables interactive prompts, so hangs are network-bound.) */
async function runWithTimeout(runner: ExecRunner, argv: string[], ms: number): Promise<ExecResult> {
let timer: ReturnType<typeof setTimeout> | undefined;
const timeout = new Promise<ExecResult>((resolve) => {
timer = setTimeout(() => resolve({ code: 124, stdout: '', stderr: `timeout after ${ms}ms` }), ms);
});
try {
return await Promise.race([runner(argv), timeout]);
} finally {
clearTimeout(timer);
}
}
// ── Canonical remote-URL parser (union of the three prior grammars) ────────
/**
* Parse `{owner, repo}` from a github.com remote URL. Accepts the https form
* (with or without `.git` / trailing slash), the scp-like form
* (`git@github.com:o/r`), and `ssh://git@github.com/o/r`.
*/
export function parseGithubOwnerRepo(url: string): { owner: string; repo: string } | null {
const s = url.trim();
let m = /^https:\/\/github\.com\/([^/]+)\/([^/]+?)(?:\.git)?\/?$/.exec(s);
if (!m) m = /^(?:ssh:\/\/)?git@github\.com[:/]([^/]+)\/([^/]+?)(?:\.git)?\/?$/.exec(s);
return m ? { owner: m[1]!, repo: m[2]! } : null;
}
/** Convenience: the combined `owner/repo` string, or null. */
export function githubOwnerRepoString(url: string): string | null {
const p = parseGithubOwnerRepo(url);
return p ? `${p.owner}/${p.repo}` : null;
}
/**
* The anonymous smart-HTTP probe URL for an https origin, or null when the
* origin has no https probe surface (scp/ssh non-github, file paths).
* github.com ssh/scp origins are converted to their https equivalent.
* URL userinfo is STRIPPED: an origin with embedded credentials would make
* the "anonymous" probe authenticated, misreading a private repo as public.
*/
export function anonProbeUrl(originUrl: string): string | null {
const gh = parseGithubOwnerRepo(originUrl);
if (gh) return `https://github.com/${gh.owner}/${gh.repo}.git/info/refs?service=git-upload-pack`;
const s = originUrl.trim().replace(/\/+$/, '');
if (!/^https:\/\//.test(s)) return null;
try {
const u = new URL(s);
u.username = '';
u.password = '';
return `${u.toString().replace(/\/+$/, '')}/info/refs?service=git-upload-pack`;
} catch {
return null; // unparseable https-looking string — no probe surface
}
}
/** True when a gh stderr/body is a 403 authored by a sandbox egress proxy
* (session scoping), as opposed to a real GitHub API 403. One predicate for
* the three call sites that must keep the HTTP-403 pre-check and the
* classification paired. */
export function isProxyBlocked403(text: string): boolean {
return /HTTP 403/i.test(text) && classifyGh403(text) === 'proxy';
}
// ── gh 403 classification ───────────────────────────────────────────────────
export type Gh403Class = 'github' | 'proxy' | 'unknown';
/**
* Classify a 403 body/stderr: a real GitHub API error is JSON carrying
* `message` + `documentation_url`; sandbox egress proxies answer with
* text/plain or non-GitHub JSON ("not enabled for this session",
* x-deny-reason). The class steers the user message — "attach the repo to
* this session" vs "check your token" — and MUST NOT gate the verdict.
*/
export function classifyGh403(body: string): Gh403Class {
const jsonStart = body.indexOf('{');
if (jsonStart >= 0) {
try {
const parsed = JSON.parse(body.slice(jsonStart)) as Record<string, unknown>;
if (typeof parsed.message === 'string') {
if ('documentation_url' in parsed) return 'github';
return 'proxy'; // JSON error without GitHub's shape — proxy-authored
}
} catch {
/* not JSON → fall through */
}
}
if (/not enabled for this session|x-deny-reason|host_not_allowed|access denied by the .*proxy/i.test(body)) {
return 'proxy';
}
return 'unknown';
}
// ── The ladder ──────────────────────────────────────────────────────────────
export type VisibilityVia = 'rest' | 'git-protocol';
export interface RungResult {
rung: 'rest' | 'authed-ls-remote' | 'anon-refs';
outcome: string;
}
export type RepoVisibilityVerdict =
| { verdict: 'private'; via: VisibilityVia; detail: string; rungs: RungResult[] }
| { verdict: 'public'; via: VisibilityVia; detail: string; rungs: RungResult[] }
| { verdict: 'unverifiable'; detail: string; rungs: RungResult[] };
export interface VerifyRepoVisibilityOpts {
originUrl: string;
/** Run git with this repo's config (credential helpers). Recommended. */
repoDir?: string;
runner?: ExecRunner;
fetchImpl?: typeof fetch;
/** Environment for the proxy-signature check (default process.env). */
env?: Record<string, string | undefined>;
/** Per-rung wall-clock cap (default 15s). */
timeoutMs?: number;
}
const ADVERTISEMENT_CONTENT_TYPE = 'application/x-git-upload-pack-advertisement';
/** pkt-line advertisement prefix: 4 hex length digits then the service line. */
const ADVERTISEMENT_BODY_RE = /^[0-9a-f]{4}# service=git-upload-pack/;
function rungLog(rungs: RungResult[]): string {
return rungs.map((r) => `${r.rung}: ${r.outcome}`).join('; ');
}
/** The actionable line for proxy-ambiguous / unverifiable outcomes. Spelled
* once so push, verify, and status degrade with the same instruction. */
export const UNVERIFIED_REMOTE_HINT =
'confirm the repo is private in the GitHub UI, then set GBRAIN_ALLOW_UNVERIFIED_REMOTE=1 ' +
'(or `gbrain config set push.allow_unverified_remote true`) to push anyway';
export async function verifyRepoVisibility(opts: VerifyRepoVisibilityOpts): Promise<RepoVisibilityVerdict> {
const runner = opts.runner ?? defaultRunner;
const fetchImpl = opts.fetchImpl ?? fetch;
const env = opts.env ?? process.env;
const timeoutMs = opts.timeoutMs ?? 15_000;
const rungs: RungResult[] = [];
const url = opts.originUrl.trim();
const gh = parseGithubOwnerRepo(url);
// Rung 1 — REST (github.com origins only). NEVER GraphQL: `gh repo view`
// rides GraphQL, which cloud proxies pin to a fixed operation set.
if (gh) {
const res = await runWithTimeout(runner, ['gh', 'api', `repos/${gh.owner}/${gh.repo}`, '--jq', '.private'], timeoutMs);
const out = res.stdout.trim();
if (res.code === 0 && out === 'true') {
rungs.push({ rung: 'rest', outcome: 'private' });
return { verdict: 'private', via: 'rest', detail: `${gh.owner}/${gh.repo} verified private via REST`, rungs };
}
if (res.code === 0 && out === 'false') {
rungs.push({ rung: 'rest', outcome: 'public' });
return { verdict: 'public', via: 'rest', detail: `${gh.owner}/${gh.repo} is PUBLIC (REST)`, rungs };
}
if (res.code === 127) {
rungs.push({ rung: 'rest', outcome: 'gh not installed' });
} else if (/HTTP 403/i.test(res.stderr)) {
rungs.push({
rung: 'rest',
outcome: isProxyBlocked403(res.stderr)
? 'blocked by an egress proxy (repo not attached to this session) — falling back to git protocol'
: `403 (${classifyGh403(res.stderr)}) — falling back to git protocol`,
});
} else {
rungs.push({ rung: 'rest', outcome: `failed (${(res.stderr.trim() || `exit ${res.code}`).slice(0, 120)})` });
}
} else {
rungs.push({ rung: 'rest', outcome: 'not a github.com origin — skipped' });
}
// Rung 2 — authed existence: the repo's own credential config answers
// "does this origin exist and can OUR credentials read it". Hardened like
// every other remote-touching git call: SSRF config flags (no ext helpers,
// no redirect-follow, file transport only behind the test/self-hosted env
// escape) and end-of-options so a dash-prefixed origin can never be parsed
// as an option (the upload-pack command-execution class).
const lsArgv = [
'git',
...(opts.repoDir ? ['-C', opts.repoDir] : []),
...durableSsrfFlags(),
'ls-remote',
'--end-of-options',
url,
'HEAD',
];
const ls = await runWithTimeout(runner, lsArgv, timeoutMs);
if (ls.code !== 0) {
rungs.push({ rung: 'authed-ls-remote', outcome: `failed (${(ls.stderr.trim() || `exit ${ls.code}`).slice(0, 120)})` });
return {
verdict: 'unverifiable',
detail: `origin not readable with current credentials — a push would fail anyway (${rungLog(rungs)})`,
rungs,
};
}
rungs.push({ rung: 'authed-ls-remote', outcome: 'ok (exists + readable)' });
// Rung 3 — anonymous probe.
const probeUrl = anonProbeUrl(url);
if (!probeUrl) {
rungs.push({ rung: 'anon-refs', outcome: 'no https probe surface for this origin' });
return { verdict: 'unverifiable', detail: `origin readable but privacy unprovable (${rungLog(rungs)}) — ${UNVERIFIED_REMOTE_HINT}`, rungs };
}
let status = 0;
let contentType = '';
let wwwAuthenticate = '';
let githubAttributed = false;
let bodyPrefix = '';
try {
const ctl = new AbortController();
const timer = setTimeout(() => ctl.abort(), timeoutMs);
try {
// redirect: manual — a redirected probe proves nothing about THIS origin
// (and following it would extend the request surface); 3xx falls through
// to the unexpected-status arm → unverifiable, fail-closed.
const res = await fetchImpl(probeUrl, { redirect: 'manual', signal: ctl.signal });
status = res.status;
contentType = res.headers.get('content-type') ?? '';
wwwAuthenticate = res.headers.get('www-authenticate') ?? '';
githubAttributed = res.headers.has('x-github-request-id');
bodyPrefix = (await res.text()).slice(0, 64);
} finally {
clearTimeout(timer);
}
} catch (e) {
rungs.push({ rung: 'anon-refs', outcome: `network error (${(e as Error).message.slice(0, 80)})` });
return { verdict: 'unverifiable', detail: `anonymous probe unreachable (${rungLog(rungs)}) — ${UNVERIFIED_REMOTE_HINT}`, rungs };
}
if (status === 401 || status === 404) {
// [D14] attribution required before the push-authorizing PRIVATE verdict.
// RFC 7235 makes www-authenticate mandatory on EVERY 401 — a middlebox's
// included — so the challenge alone is NEVER proof. github.com origins
// require the GitHub request-id header. For NON-github origins there is no
// trustable attribution signal at all (a spoofing/inspecting middlebox
// 401s identically to a real private server), so a 401 there is
// `unverifiable` — the operator confirms via the escape hatch rather than
// us laundering a possibly-public repo into a push authorization. Both
// adversarial reviewers flagged the prior "challenge ⇒ private" as a
// public-repo-exfil path; fail closed.
const attributed = gh !== null && githubAttributed;
if (attributed) {
rungs.push({ rung: 'anon-refs', outcome: `${status} with GitHub-attributed auth challenge (not anonymously readable)` });
return {
verdict: 'private',
via: 'git-protocol',
detail: `origin exists, reads with credentials, and refuses anonymous access (${rungLog(rungs)})`,
rungs,
};
}
const why = gh !== null
? `${status} without GitHub attribution (x-github-request-id) — possible middlebox`
: `${status} on a non-github origin — no trustable attribution signal`;
rungs.push({ rung: 'anon-refs', outcome: `${why}, not trusted as a privacy signal` });
return { verdict: 'unverifiable', detail: `un-attributed ${status} on the anonymous probe (${rungLog(rungs)}) — ${UNVERIFIED_REMOTE_HINT}`, rungs };
}
if (status === 200) {
// [D4] advertisement proof required before the "public" verdict.
const proven =
contentType.toLowerCase().includes(ADVERTISEMENT_CONTENT_TYPE) || ADVERTISEMENT_BODY_RE.test(bodyPrefix);
if (!proven) {
rungs.push({ rung: 'anon-refs', outcome: '200 without a git advertisement (SSO/login page?) — not trusted as a public signal' });
return { verdict: 'unverifiable', detail: `anonymous 200 without advertisement proof (${rungLog(rungs)}) — ${UNVERIFIED_REMOTE_HINT}`, rungs };
}
if (isCredentialInjectingProxy(env)) {
// The "anonymous" request may have been silently authenticated by the
// sandbox proxy — a private repo would ALSO read 200 here. Ambiguous.
rungs.push({ rung: 'anon-refs', outcome: '200 advertisement behind a credential-injecting proxy — ambiguous' });
return {
verdict: 'unverifiable',
detail: `cannot prove privacy from inside this sandbox's authenticated proxy (${rungLog(rungs)}) — ${UNVERIFIED_REMOTE_HINT}`,
rungs,
};
}
rungs.push({ rung: 'anon-refs', outcome: '200 with a git advertisement — anonymously clonable' });
return { verdict: 'public', via: 'git-protocol', detail: `origin is anonymously readable (${rungLog(rungs)})`, rungs };
}
rungs.push({ rung: 'anon-refs', outcome: `unexpected HTTP ${status}` });
return { verdict: 'unverifiable', detail: `anonymous probe answered HTTP ${status} (${rungLog(rungs)}) — ${UNVERIFIED_REMOTE_HINT}`, rungs };
}
// ── Verdict cache [D11] ─────────────────────────────────────────────────────
//
// PRIVATE verdicts only, 1h TTL, keyed on the exact origin URL. `public` and
// `unverifiable` are NEVER cached: failures must re-verify every push so a
// fixed repo unblocks on the next turn, while a private→public flip is
// honored at most one TTL late (deliberate owner action; the secret-scan gate
// still runs on every push). Kills the per-turn network cost of the
// cloud-sandbox debounce-0 push cadence.
export const VISIBILITY_CACHE_TTL_MS = 60 * 60 * 1000;
/** Cache key = origin URL with any userinfo (`user:pat@`) stripped — a PAT in
* an https remote must never be persisted into the on-disk cache file. */
function cacheKey(originUrl: string): string {
try {
const u = new URL(originUrl);
u.username = '';
u.password = '';
return u.toString();
} catch {
return originUrl; // scp/ssh/file forms carry no URL userinfo
}
}
interface VisibilityCacheEntry {
verdict: 'private';
via: VisibilityVia;
verified_at: string;
}
export function visibilityCachePath(): string {
return join(ensureGbrainHome(), 'bootstrap', 'visibility-cache.json');
}
/** A fresh cached PRIVATE verdict for this origin, or null. */
export function readCachedPrivateVerdict(
originUrl: string,
opts: { now?: number; path?: string } = {},
): RepoVisibilityVerdict | null {
const now = opts.now ?? Date.now();
try {
const raw = readFileSync(opts.path ?? visibilityCachePath(), 'utf8');
const map = JSON.parse(raw) as Record<string, VisibilityCacheEntry>;
const entry = map[cacheKey(originUrl)];
if (!entry || entry.verdict !== 'private') return null;
const at = Date.parse(entry.verified_at);
if (!Number.isFinite(at) || now - at > VISIBILITY_CACHE_TTL_MS) return null;
return {
verdict: 'private',
via: entry.via,
detail: `verified private ${entry.via === 'rest' ? 'via REST' : 'via git protocol'} at ${entry.verified_at} (cached)`,
rungs: [],
};
} catch {
return null; // missing/corrupt cache is a miss, never an error
}
}
/** Record a PRIVATE verdict (no-op for any other verdict). Prunes expired
* entries; tolerant of a corrupt existing file (starts fresh). */
export function writeVisibilityCache(
originUrl: string,
verdict: RepoVisibilityVerdict,
opts: { now?: number; path?: string } = {},
): void {
if (verdict.verdict !== 'private') return;
const now = opts.now ?? Date.now();
const path = opts.path ?? visibilityCachePath();
let map: Record<string, VisibilityCacheEntry> = {};
try {
const parsed = JSON.parse(readFileSync(path, 'utf8')) as Record<string, VisibilityCacheEntry>;
for (const [k, v] of Object.entries(parsed)) {
const at = Date.parse(v?.verified_at ?? '');
if (v?.verdict === 'private' && Number.isFinite(at) && now - at <= VISIBILITY_CACHE_TTL_MS) map[k] = v;
}
} catch {
map = {};
}
map[cacheKey(originUrl)] = { verdict: 'private', via: verdict.via, verified_at: new Date(now).toISOString() };
try {
mkdirSync(dirname(path), { recursive: true });
const tmp = `${path}.tmp-${process.pid}`;
writeFileSync(tmp, JSON.stringify(map, null, 2) + '\n', { mode: 0o600 });
renameSync(tmp, path);
} catch {
/* cache write failure is never an error — next push just re-verifies */
}
}
src/core/rerank-audit.ts
+2 -1
View File
@@ -27,12 +27,13 @@
import { createAuditWriter, computeIsoWeekFilename } from './audit/audit-writer.ts';
/** Stable error-classification union; matches RerankError.reason. */
/** Stable error-classification union for reranker fail-open audit rows. */
export type RerankFailureReason =
| 'auth'
| 'rate_limit'
| 'network'
| 'timeout'
| 'budget'
| 'payload_too_large'
| 'unknown';
src/core/search/hybrid.ts
+7
View File
@@ -1841,6 +1841,13 @@ export async function hybridSearchCached(
// resolves) into the cache key so a row written under one exclude
// policy can't be served to a lookup under another.
hardExcludes: resolveHardExcludes(opts?.exclude_slug_prefixes, opts?.include_slug_prefixes),
// #3515 — fold the EFFECTIVE detail level into the cache key. detail
// gates dedup, chunk-source filtering, and the compiled_truth boost, so
// a `--detail low` write (compiled-truth-only result set) must never be
// served to a default `medium` lookup. Resolve auto-detect the same way
// bare hybridSearch does (opts.detail ?? autoDetectDetail(query)) so an
// auto-detected `high` query keys like an explicit `high` one.
detail: opts?.detail ?? autoDetectDetail(query),
});
// Cache decision: opts.useCache (explicit) wins over global config; global
src/core/search/mode.ts
+28 -1
View File
@@ -779,7 +779,18 @@ export function attributeKnob<K extends keyof ModeBundle>(
// to cache.ttl_seconds, with no warning and no way for an operator to tell.
// Same one-time global cold-miss pattern as the bumps above; refills within
// cache.ttl_seconds (3600s default).
export const KNOBS_HASH_VERSION = 15;
//
// bump 15→16 (#3515): `detail` folds into the key via ctx.detail (det=).
// detail is result-affecting by design — it gates dedup, chunk-source
// filtering, and the compiled_truth boost — but was absent from the key, so
// a `--detail low` write (compiled-truth-only result set) was served to a
// default `medium` lookup for the whole TTL. Same contamination class as
// [CDX-4], floor_ratio (v=3), and relationalRetrieval (v=10). v=14 was
// claimed by #3514 (compiled_truth boost scope, #3430) and v=15 by the
// `fts=` fold (#3677), so this lands as v=16 per the D8 sequencing
// convention (see the v=4/v=5 note above). Same one-time global cold-miss
// pattern as the bumps above.
export const KNOBS_HASH_VERSION = 16;
/**
* v0.36 (D8 / CDX-2) — second-arg context for the cache key. The
@@ -818,6 +829,17 @@ export interface KnobsHashContext {
* 'none' for legacy callers that don't thread excludes.
*/
hardExcludes?: string[];
/**
* v=16 (#3515): the EFFECTIVE detail level for this call — per-call
* SearchOpts.detail, or the auto-detected level when the caller didn't
* specify (hybridSearchCached threads `opts.detail ?? autoDetectDetail(query)`,
* matching what bare hybridSearch resolves). detail gates dedup,
* chunk-source filtering, and the compiled_truth boost, so a detail=low
* write must never be served to a detail=medium lookup. Lives in ctx (not
* ResolvedSearchKnobs) because it's per-call, not a mode knob — same path
* as col=/prov=. Undefined falls back to 'medium' (the documented default).
*/
detail?: 'low' | 'medium' | 'high';
}
export function knobsHash(
@@ -921,6 +943,11 @@ export function knobsHash(
// memoizes and validates against /^[a-z][a-z0-9_]*$/, so this stays a
// cheap, bounded string.
`fts=${getFtsLanguage()}`,
// v=16 addition (#3515, append-only): effective detail level. detail
// gates dedup, chunk-source filtering, and the compiled_truth boost, so
// a low write (compiled-truth-only set) must never be served to a
// medium/high lookup. Undefined falls back to 'medium' (the default).
`det=${ctx?.detail ?? 'medium'}`,
];
const h = createHash('sha256');
h.update(parts.join('|'));
src/core/search/rerank.ts
+13 -2
View File
@@ -20,6 +20,7 @@
import { createHash } from 'crypto';
import type { SearchResult } from '../types.ts';
import { rerank as gatewayRerank, RerankError, type RerankInput, type RerankResult } from '../ai/gateway.ts';
import { BudgetExhausted } from '../budget/budget-tracker.ts';
import { logRerankFailure, type RerankFailureReason } from '../rerank-audit.ts';
export interface RerankerOpts {
@@ -44,6 +45,17 @@ function hashQuery(query: string): string {
return createHash('sha256').update(query, 'utf8').digest('hex').slice(0, 8);
}
function classifyRerankFailure(err: unknown): RerankFailureReason {
if (err instanceof RerankError) return err.reason;
if (
err instanceof BudgetExhausted ||
(err && typeof err === 'object' && (err as { tag?: unknown }).tag === 'BUDGET_EXHAUSTED')
) {
return 'budget';
}
return 'unknown';
}
/**
* Reorder the top `topNIn` results by reranker relevance score. The
* un-reranked tail (any rows past topNIn) preserves its original RRF
@@ -83,8 +95,7 @@ export async function applyReranker(
...(opts.model ? { model: opts.model } : {}),
});
} catch (err) {
const reason: RerankFailureReason =
err instanceof RerankError ? err.reason : 'unknown';
const reason = classifyRerankFailure(err);
const errorSummary = err instanceof Error ? err.message : String(err);
try {
logRerankFailure({
src/core/skill-frontmatter.ts
+18 -1
View File
@@ -27,6 +27,8 @@
* not a silent failure to exempt.
*/
import { FAILSAFE_SCHEMA, safeLoad as yamlSafeLoad } from 'js-yaml';
// ---------------------------------------------------------------------------
// Types
// ---------------------------------------------------------------------------
@@ -87,7 +89,8 @@ export interface ParsedFrontmatter {
* `readFileSync(path, 'utf-8')` at the boundary.
*/
export function parseSkillFrontmatter(content: string): ParsedFrontmatter | null {
const fmMatch = content.match(/^---\n([\s\S]*?)\n---/);
const normalized = content.replace(/\r\n/g, '\n');
const fmMatch = normalized.match(/^---\n([\s\S]*?)\n---/);
if (!fmMatch) return null;
const raw = fmMatch[1];
const out: ParsedFrontmatter = { raw };
@@ -133,6 +136,20 @@ export function parseSkillFrontmatter(content: string): ParsedFrontmatter | null
* top-level fields below it. Stops at the first non-indented line.
*/
function parseArrayField(raw: string, field: string): string[] | undefined {
try {
const parsed = yamlSafeLoad(raw, { schema: FAILSAFE_SCHEMA });
if (parsed && typeof parsed === 'object' && Object.hasOwn(parsed, field)) {
const value = (parsed as Record<string, unknown>)[field];
if (!Array.isArray(value)) return undefined;
return value
.filter((item): item is string => typeof item === 'string')
.map(item => item.trim())
.filter(Boolean);
}
} catch {
// Preserve the tolerant legacy behavior for partially malformed YAML.
}
// Inline form: `field: [a, b, c]` or `field: []`
const inlineRe = new RegExp(`^${field}:\\s*\\[([^\\]]*)\\]\\s*$`, 'm');
const inlineMatch = raw.match(inlineRe);
src/core/sources-load.ts
+22
View File
@@ -160,6 +160,28 @@ export function isSourceFederated(config: unknown): boolean {
return parsed.federated === true;
}
/**
* Three-way federation state for display (CLI `sources list`, etc.).
*
* `isSourceFederated` collapses to a boolean for the inclusion check (does
* this source show up in OTHER anchors' unqualified reads?), which is
* correctly strict — 'unset' behaves like 'isolated' there. But 'unset' and
* 'isolated' are NOT interchangeable for display: only an explicit
* `federated: false` (`sources unfederate` / `--no-federated`) opts a source
* out of cross-source read mixing in both directions. A source that has
* simply never set the flag still widens its OWN unqualified reads to
* include the federated set (the #1434 sole-source convenience, pinned
* behavior — see test/local-federated-search-scope.test.ts and
* test/unfederate-read-scope-2928.test.ts). Labeling it "isolated" overstates
* what the flag actually does.
*/
export function sourceFederationState(config: unknown): 'federated' | 'isolated' | 'unset' {
const raw = parseSourceConfig(config).federated;
if (raw === true) return 'federated';
if (raw === false) return 'isolated';
return 'unset';
}
/**
* Enumerate every source. Order: 'default' first, then alphabetical by id.
*
src/core/sync-failure-ledger.ts
+5 -5
View File
@@ -515,9 +515,9 @@ export function clearFailures(sourceId: string, paths: string[]): void {
}
/**
* Acknowledge OPEN file failures (human `--skip-failed`). Scoped to one
* source when `sourceId` is given (never acks another source — #1939 Codex
* #2). Sentinels (`<head>`) are NEVER acknowledged this way.
* Acknowledge OPEN or AUTO_SKIPPED file failures (human `--skip-failed`).
* Scoped to one source when `sourceId` is given (never acks another source
* — #1939 Codex #2). Sentinels (`<head>`) are NEVER acknowledged this way.
*/
export function acknowledgeFailures(sourceId?: string): AcknowledgeResult {
return withLedgerLock(() => {
@@ -526,7 +526,7 @@ export function acknowledgeFailures(sourceId?: string): AcknowledgeResult {
let changed = 0;
const acked: SyncFailure[] = [];
for (const e of entries) {
if (e.state !== 'open') continue;
if (e.state !== 'open' && e.state !== 'auto_skipped') continue;
if (sourceId !== undefined && e.source_id !== sourceId) continue;
if (!isSkippablePath(e.path)) continue;
e.state = 'acknowledged';
@@ -543,7 +543,7 @@ export function acknowledgeFailures(sourceId?: string): AcknowledgeResult {
/**
* Mark the given chronic file paths `auto_skipped` (valve fired). Only OPEN,
* non-sentinel rows transition. Auto-skipped rows stay UNRESOLVED so doctor
* keeps warning until the file imports cleanly.
* keeps warning until the file imports cleanly or a human acknowledges them.
*/
export function autoSkipFailures(sourceId: string, paths: string[]): AcknowledgeResult {
if (paths.length === 0) return { count: 0, summary: [] };
src/core/types.ts
+23
View File
@@ -623,6 +623,29 @@ export interface StaleChunkRow {
page_id: number;
}
/**
* A page with non-empty `compiled_truth` and/or `timeline` (both are
* chunked independently by the healer) but ZERO `content_chunks` rows,
* returned by `listChunklessPagesWithContent`. `embed --stale` scans
* `content_chunks` (embedding IS NULL) — a page written directly via
* `putPage` that never went through the chunking step (e.g. an
* enrichment-generated entity stub) has no chunk row to go stale, so it is
* invisible to that scan forever. This is the safety-net detection: find
* such pages so `embed --stale` can chunk them and fold the resulting
* NULL-embedding chunks into the same run.
*
* Quarantined and `embed_skip` pages are excluded by the underlying query
* (`src/core/quarantine.ts` / `src/core/embed-skip.ts`) — both are
* INTENTIONALLY chunkless by design (content-quality gate), not drift.
*/
export interface ChunklessPageRow {
id: number;
slug: string;
source_id: string;
compiled_truth: string;
timeline: string;
}
/**
* v0.42.7 (#1696) — a page that needs link/timeline extraction, returned by
* `listStalePagesForExtraction`. Carries the page CONTENT (compiled_truth +
src/core/workspace-push.ts
+198 -55
View File
@@ -38,21 +38,33 @@
* ahead?} on success AND failure [B4] (doctor + SessionStart digest
* read it).
*
* Remote-privacy gate [G8]: refuses a remote that is not VERIFIABLY private
* — `gh repo view --json isPrivate` must answer `true`. Unverifiable
* visibility (gh missing/unauthed, non-GitHub host, path remote) is a
* refusal with a named reason, never fail-open. Escape hatch for self-hosted
* git: `allowUnverifiedRemote` (CLI `--allow-unverified-remote`), logged.
* Remote-privacy gate [G8]: refuses a remote that is not VERIFIABLY private.
* Verification routes through the repo-visibility LADDER (repo-visibility.ts):
* REST first (`gh api repos/...` — never GraphQL, which cloud proxies pin),
* then pure git protocol (authed ls-remote + attributed anonymous probe), so
* the gate keeps working where gh is broken/blocked. Unverifiable visibility
* is a refusal with the per-rung reason, never fail-open. Escape hatches for
* self-hosted git you trust, loudly logged, most-portable last: CLI
* `allowUnverifiedRemote` > env GBRAIN_ALLOW_UNVERIFIED_REMOTE=1 > file-plane
* config `push.allow_unverified_remote` (readable by detached hook children
* that can never see a shell export or the DB plane).
*/
import {
existsSync, mkdirSync, readFileSync, renameSync, rmSync, statSync, writeFileSync,
existsSync, mkdirSync, readdirSync, readFileSync, renameSync, rmSync, statSync, writeFileSync,
} from 'fs';
import { dirname, join } from 'path';
import { createHash, randomBytes } from 'crypto';
import { execFileSync } from 'child_process';
import { GIT_ENV, GIT_ENV_AUTH, GIT_SSRF_SUBCOMMAND_FLAGS, detectDefaultBranch, divergenceSafePull } from './git-remote.ts';
import { loadConfigFileOnly } from './config.ts';
import { ensureGbrainHome } from './gbrain-home.ts';
import {
githubOwnerRepoString,
readCachedPrivateVerdict,
verifyRepoVisibility,
writeVisibilityCache,
} from './repo-visibility.ts';
import { isProcessAlive } from './pglite-lock.ts';
import {
loadWorkspaceAllowlist, matchesGlob, pathAllowlisted, scanText,
@@ -307,62 +319,170 @@ export type RemotePrivacyVerdict =
| { verdict: 'not_private'; detail: string }
| { verdict: 'unverifiable'; detail: string };
/** Parse `owner/repo` out of a github.com remote URL (https or scp-like). */
/** Parse `owner/repo` out of a github.com remote URL (https, scp-like, or
* ssh://). Back-compat adapter over the canonical repo-visibility parser. */
export function parseGithubOwnerRepo(url: string): string | null {
let m = /^https:\/\/github\.com\/([^/]+)\/([^/]+?)(?:\.git)?\/?$/.exec(url);
if (m) return `${m[1]}/${m[2]}`;
m = /^(?:ssh:\/\/)?git@github\.com[:/]([^/]+)\/([^/]+?)(?:\.git)?\/?$/.exec(url);
if (m) return `${m[1]}/${m[2]}`;
return null;
return githubOwnerRepoString(url);
}
/**
* Verify the origin remote is a PRIVATE repo via `gh repo view --json
* isPrivate`. Anything short of an affirmative `true` is either
* `not_private` (affirmative public) or `unverifiable` (gh missing/unauthed,
* non-GitHub host, path/file remote) — the caller refuses both unless the
* escape hatch is set [G8: never fail-open].
* Verify the origin remote is a PRIVATE repo via the repo-visibility ladder
* (REST first, git-protocol fallback — see repo-visibility.ts for the full
* verdict matrix). Anything short of a proven `private` is either
* `not_private` (proven public) or `unverifiable` — the caller refuses both
* unless an escape hatch is set [G8: never fail-open]. Fresh `private`
* verdicts are cached (1h TTL, private-only) so the per-turn push cadence
* doesn't re-pay the network probes every turn [D11].
*/
export function verifyRemotePrivacy(root: string): RemotePrivacyVerdict {
export async function verifyRemotePrivacy(root: string): Promise<RemotePrivacyVerdict> {
const url = tryGit(root, ['remote', 'get-url', 'origin'], { timeoutMs: 10_000 });
if (!url) return { verdict: 'unverifiable', detail: 'no origin remote configured' };
const ownerRepo = parseGithubOwnerRepo(url);
if (!ownerRepo) {
return {
verdict: 'unverifiable',
detail: `origin is not a github.com URL — cannot verify visibility via gh`,
};
}
if (readCachedPrivateVerdict(url) !== null) return { verdict: 'private' };
const v = await verifyRepoVisibility({ originUrl: url, repoDir: root });
writeVisibilityCache(url, v);
if (v.verdict === 'private') return { verdict: 'private' };
if (v.verdict === 'public') return { verdict: 'not_private', detail: v.detail };
return { verdict: 'unverifiable', detail: v.detail };
}
/** File-plane escape hatch [D18]: `gbrain config set push.allow_unverified_remote true`.
* Read tolerantly from ~/.gbrain/config.json — detached hook children never see
* a shell export, and the DB plane is unreadable while a serve holds the
* single-writer lock, so the file plane is the only channel that always works. */
export function configAllowsUnverifiedRemote(): boolean {
try {
const out = execFileSync(
'gh',
['repo', 'view', ownerRepo, '--json', 'isPrivate', '--jq', '.isPrivate'],
{ stdio: ['ignore', 'pipe', 'pipe'], timeout: 20_000, env: process.env },
).toString().trim();
if (out === 'true') return { verdict: 'private' };
if (out === 'false') return { verdict: 'not_private', detail: `${ownerRepo} is PUBLIC` };
return { verdict: 'unverifiable', detail: `gh returned unexpected output: ${out.slice(0, 80)}` };
} catch (e) {
const err = e as NodeJS.ErrnoException;
if (err?.code === 'ENOENT') {
return { verdict: 'unverifiable', detail: 'gh CLI not installed — cannot verify repo visibility' };
}
return {
verdict: 'unverifiable',
detail: `gh repo view failed (${(err?.message ?? 'unknown').slice(0, 120)})`,
};
const cfg = loadConfigFileOnly();
const v = cfg?.push?.allow_unverified_remote as unknown;
return v === true || v === 'true' || v === '1';
} catch {
return false;
}
}
// ── push-status.json [B4] ───────────────────────────────────────────────────
/** Legacy single-file location (pre-per-root). Read-only compatibility: the
* reader consults it only when NO per-root files exist yet (fresh upgrade). */
export function pushStatusPath(): string {
return join(ensureGbrainHome(), 'bootstrap', 'push-status.json');
}
function writePushStatus(status: { ts: string; ok: boolean; reason?: string; ahead?: number }): void {
/** Stable short key for per-root state files [D3/D13]: one gbrain home serves
* many bootstrap workspaces, so every push/debounce/announce state file is
* keyed by workspace root — a single shared file is last-writer-wins, which
* either masks one workspace's failure or defeats another's debounce. */
export function workspaceRootHash(root: string): string {
return createHash('sha256').update(root).digest('hex').slice(0, 12);
}
export function pushStatusPathForRoot(root: string): string {
return join(ensureGbrainHome(), 'bootstrap', `push-status-${workspaceRootHash(root)}.json`);
}
/** One parsed push-status record [D8: one reader, three formatters]. */
export interface PushStatusEntry {
ts?: string;
ok?: boolean;
reason?: string;
ahead?: number;
repoRoot?: string;
/** Absolute path of the status file (per-root announce-state keying). */
file: string;
}
const PUSH_STATUS_FILE_RE = /^push-status-[0-9a-f]{12}\.json$/;
/** Tolerant reader over every push-status file (banner, SessionStart note,
* and doctor all consume THIS — one parse, one schema owner). Per-root files
* win; the legacy global file is consulted only when none exist, so a stale
* pre-upgrade record can't shadow live per-root state. */
export function readPushStatuses(): PushStatusEntry[] {
const dir = join(ensureGbrainHome(), 'bootstrap');
const parse = (file: string): PushStatusEntry | null => {
try {
const s = JSON.parse(readFileSync(file, 'utf8')) as Omit<PushStatusEntry, 'file'>;
return { ...s, file };
} catch {
return null; // torn/corrupt/missing → skip, never throw
}
};
let names: string[] = [];
try {
const p = pushStatusPath();
names = readdirSync(dir).filter((n) => PUSH_STATUS_FILE_RE.test(n));
} catch {
return [];
}
// The legacy file is consulted only when NO per-root FILES exist on disk —
// if per-root files exist but are all corrupt, an empty result is the honest
// answer (doctor pairs it with pushStatusFilesExist for the loud warn); a
// stale pre-upgrade record must never shadow live-but-unreadable state.
if (names.length === 0) {
const legacy = parse(pushStatusPath());
return legacy ? [legacy] : [];
}
return names
.map((n) => parse(join(dir, n)))
.filter((e): e is PushStatusEntry => e !== null)
// A record whose workspace no longer exists on disk is a ghost: it can
// never be cleared by a re-push (the root is gone), so it must not feed
// the failure banner / staleness note forever (deleted Conductor
// workspaces are routine). Entries without repoRoot (legacy shape in a
// per-root file) are kept — absence of evidence is not a ghost.
.filter((e) => e.repoRoot === undefined || existsSync(e.repoRoot));
}
/** True when any push-status file exists on disk (parseable or not). The
* reader skips corrupt files silently; doctor pairs this with an empty read
* to say "status present but unreadable" instead of saying nothing. */
export function pushStatusFilesExist(): boolean {
const dir = join(ensureGbrainHome(), 'bootstrap');
try {
if (readdirSync(dir).some((n) => PUSH_STATUS_FILE_RE.test(n))) return true;
} catch {
/* fall through */
}
return existsSync(pushStatusPath());
}
/** The per-root status for one workspace, or null. Direct keyed read first —
* the scan is only a fallback for legacy records carrying a repoRoot field. */
export function readPushStatusForRoot(root: string): PushStatusEntry | null {
const keyedPath = pushStatusPathForRoot(root);
try {
const s = JSON.parse(readFileSync(keyedPath, 'utf8')) as Omit<PushStatusEntry, 'file'>;
return { ...s, file: keyedPath };
} catch {
/* fall through to the scan */
}
return readPushStatuses().find((e) => e.repoRoot === root) ?? null;
}
/** One aggregation for every status surface (SessionStart note, doctor,
* banner counting): the failing entries and the stalest success timestamp. */
/** Failure reasons carry remote-influenced text (git stderr → rung log →
* push-status.reason). Clamp to a safe printable charset + length before ANY
* model- or human-visible surface embeds it (banner, doctor, status blob) so
* it can't become an injection or formatting vector — the free-form
* counterpart to hook.ts's reasonCode() for typed codes. */
export function sanitizePushReason(reason: string | undefined): string {
if (!reason) return 'unknown reason';
return reason.replace(/[^\x20-\x7E]/g, ' ').replace(/[`$\\]/g, "'").slice(0, 140);
}
export function summarizePushStatuses(entries: PushStatusEntry[]): {
failing: PushStatusEntry[];
stalestTs: number | null;
} {
const failing = entries.filter((e) => e.ok === false);
const stamps = entries.map((e) => Date.parse(e.ts ?? '')).filter((t) => Number.isFinite(t));
return { failing, stalestTs: stamps.length > 0 ? Math.min(...stamps) : null };
}
function writePushStatus(
status: { ts: string; ok: boolean; reason?: string; ahead?: number; repoRoot: string },
): void {
try {
const p = pushStatusPathForRoot(status.repoRoot);
mkdirSync(dirname(p), { recursive: true, mode: 0o700 });
// tmp+rename (the writeReceipt pattern): a concurrent reader never sees a
// torn half-written status file.
@@ -409,12 +529,14 @@ export async function workspacePush(opts: WorkspacePushOpts): Promise<WorkspaceP
const finish = (r: WorkspacePushResult): WorkspacePushResult => {
// B4: written on success AND failure — only by the lock WINNER (a skip
// must not clobber the in-flight run's eventual status).
// must not clobber the in-flight run's eventual status). Keyed per root
// [D13] so one workspace's success can never mask another's failure.
writePushStatus({
ts: new Date().toISOString(),
ok: r.ok,
...(r.reason !== undefined ? { reason: r.reason } : {}),
...(r.ahead !== undefined ? { ahead: r.ahead } : {}),
repoRoot: root,
});
return r;
};
@@ -586,21 +708,42 @@ export async function workspacePush(opts: WorkspacePushOpts): Promise<WorkspaceP
reason: 'no origin remote configured — nothing to push to',
});
}
if (opts.allowUnverifiedRemote) {
log('WARN: --allow-unverified-remote set — skipping repo-visibility verification');
} else {
const privacy = verifyRemotePrivacy(root);
if (privacy.verdict !== 'private') {
const reason =
privacy.verdict === 'not_private'
? `origin is not private: ${privacy.detail} — refusing to push workspace contents`
: `cannot verify origin visibility (${privacy.detail}) — refusing to push. ` +
`Pass --allow-unverified-remote for self-hosted git you trust.`;
// Escape-hatch resolution [D18], most explicit first: CLI flag > env var >
// file-plane config. Every path warns loudly. The hatches downgrade ONLY
// the `unverifiable` verdict — the ladder still runs and an affirmatively
// PUBLIC origin refuses regardless, matching every piece of user-facing
// copy ("for self-hosted git you trust", never "push to public repos").
const unverifiedVia = opts.allowUnverifiedRemote
? 'the allow-unverified-remote flag'
: process.env.GBRAIN_ALLOW_UNVERIFIED_REMOTE === '1'
? 'GBRAIN_ALLOW_UNVERIFIED_REMOTE=1'
: configAllowsUnverifiedRemote()
? 'config push.allow_unverified_remote (sticky — unset it once verification works)'
: null;
{
const privacy = await verifyRemotePrivacy(root);
if (privacy.verdict === 'not_private') {
const reason = `origin is not private: ${privacy.detail} — refusing to push workspace contents` +
(unverifiedVia !== null ? ` (the unverified-remote override via ${unverifiedVia} does NOT cover proven-public origins)` : '');
log(`PUSH REFUSED: ${reason}`);
return finish({
ok: false, status: 'refused_visibility', repoRoot: root, branch, committed, reason,
});
}
if (privacy.verdict === 'unverifiable') {
if (unverifiedVia !== null) {
log(`WARN: origin visibility unverifiable — pushing anyway via ${unverifiedVia}; you are trusting this remote`);
} else {
const reason =
`cannot verify origin visibility (${privacy.detail}) — refusing to push. ` +
`Pass --allow-unverified-remote (or set GBRAIN_ALLOW_UNVERIFIED_REMOTE=1, or ` +
'`gbrain config set push.allow_unverified_remote true`) for self-hosted git you trust.';
log(`PUSH REFUSED: ${reason}`);
return finish({
ok: false, status: 'refused_visibility', repoRoot: root, branch, committed, reason,
});
}
}
}
// 6. pull AFTER commit [CX2-7] — a dirty tree is impossible here short of
src/core/write-through.ts
+118 -8
View File
@@ -22,12 +22,15 @@
*/
import { existsSync, statSync, mkdirSync, writeFileSync, renameSync, unlinkSync, readdirSync } from 'fs';
import { basename, dirname, join } from 'path';
import { basename, dirname, isAbsolute, join, relative, resolve } from 'path';
import { randomBytes } from 'crypto';
import type { BrainEngine } from './engine.ts';
import { serializePageToMarkdown, resolvePageFilePath } from './markdown.ts';
import { isWriteTargetContained } from './path-confine.ts';
import { isDurabilityHardened, commitWriteThroughFile } from './brain-repo-durability.ts';
import {
isDurabilityHardened, commitWriteThroughFile, currentBranch, getLastPushOutcome,
type PushLogOutcome,
} from './brain-repo-durability.ts';
/** Minimal logger surface — structurally compatible with operations.ts `Logger`. */
export interface WriteThroughLogger {
@@ -39,11 +42,28 @@ export interface WriteThroughResult {
path?: string;
/**
* True when the write was also committed to git (#2426). Only attempted on
* repos hardened via `gbrain sources harden` (durability hook installed);
* the hook then background-pushes the commit. Best-effort — a false/absent
* value never blocks the write.
* repos hardened via `gbrain sources harden` (durability hook installed).
* Commit-only — this says nothing about whether the commit ever reached the
* remote. Best-effort — a false/absent value never blocks the write.
*/
committed?: boolean;
/**
* Set alongside `committed: true`. The actual push runs detached in the
* post-commit hook (see brain-repo-durability.ts), so at the moment this
* result is returned the outcome for THIS commit is genuinely unknown —
* 'pending' is the only honest value. Check `lastPushStatus` (or
* $GBRAIN_HOME/brain-push.log directly) afterward to see whether pushes for
* this branch are landing.
*/
pushed?: 'pending';
/**
* Best-effort snapshot of the most recently logged push outcome for this
* branch (read from the hook's shared log), taken right after the commit
* above. It reflects push history UP TO that point — not the push this
* write just queued — so callers and health tooling can tell "pushes for
* this branch have been failing" apart from "this write committed fine".
*/
lastPushStatus?: PushLogOutcome;
/**
* Non-error reasons the file was not written:
* - no_repo_configured: the resolved target (source `local_path` or, for a
@@ -73,6 +93,58 @@ export interface WritePageThroughOpts {
logger?: WriteThroughLogger;
}
/**
* Vet a `pages.source_path` before it is trusted as a write target.
*
* The column is populated from the scanner's relative path at import time, so
* the normal value is a clean repo-relative `.md` path. This rejects the shapes
* that would make `join(root, value)` unsafe or nonsensical — absolute paths,
* `..` traversal, NUL bytes, non-markdown artifacts, and blanks. Containment is
* still re-checked by `isWriteTargetContained` after the join; this is the
* cheap structural filter in front of it.
*/
function sanitizeRecordedSourcePath(raw: string | null | undefined): string | null {
if (!raw) return null;
const value = raw.trim();
if (!value || value.includes('\0')) return null;
if (!value.toLowerCase().endsWith('.md')) return null;
if (isAbsolute(value)) return null;
if (value.split(/[\\/]/).some((segment) => segment === '..')) return null;
return value;
}
/**
* Recover a page-root-relative write target from a `file://` `source_uri`.
*
* `gbrain capture --file` records the absolute input path as `source_uri` but
* does NOT set `source_path` (that column is the file-scanner's). So a file the
* user authored INSIDE the brain repo and then captured has no file of record,
* and the slug-derived fallback would mint a twin beside the very file that was
* just read. When the recorded URI points at a path under `pageRoot`, that path
* IS the file of record — use it.
*
* Returns null for anything not a contained `.md` file so the caller falls back
* to the slug path.
*/
function recordedPathFromFileUri(sourceUri: string | null | undefined, pageRoot: string): string | null {
if (!sourceUri || !sourceUri.startsWith('file://')) return null;
let abs = sourceUri.slice('file://'.length);
if (!abs) return null;
// Percent-decode only when it looks encoded — the CLI stores raw paths, so a
// literal '%' in a filename must not be mangled.
if (/%[0-9A-Fa-f]{2}/.test(abs)) {
try {
abs = decodeURIComponent(abs);
} catch {
return null;
}
}
if (abs.includes('\0') || !abs.toLowerCase().endsWith('.md')) return null;
const rel = relative(resolve(pageRoot), resolve(abs));
if (!rel || isAbsolute(rel) || rel.split(/[\\/]/).some((segment) => segment === '..')) return null;
return rel;
}
/**
* Render the DB row for `slug` to markdown and atomically write it under
* `sync.repo_path`. Never throws — failures are reported via the result's
@@ -104,11 +176,37 @@ export async function writePageThrough(
[sourceId],
);
const sourceLocalPath = srcRows[0]?.local_path ?? null;
// Prefer the page's recorded `source_path` — the ACTUAL file this row was
// imported from — over a slug-derived name. Deriving `<slug>.md` mints a
// SECOND file beside the original whenever the on-disk name isn't the slug,
// which is the common case for a human-authored vault: `Library/People/
// Steve Jobs.md` has slug `library/people/steve-jobs`, so a later put_page
// dropped a lowercase `steve-jobs.md` twin next to it. Two artifacts, one
// row, and the newer content in whichever the caller didn't expect.
//
// It also desyncs `gbrain sync`, which keys delete-reconcile on
// `source_path` (see collectMissingSourcePaths): the twin is invisible to
// reconcile, so deleting the ORIGINAL file deletes the page even though a
// file for it is still on disk.
//
// A NULL `source_path` means the page was born via put/capture and has no
// file of record yet — the slug-derived path stays correct for those.
const pathRows = await engine.executeRaw<{ source_path: string | null; source_uri: string | null }>(
`SELECT source_path, source_uri FROM pages WHERE source_id = $1 AND slug = $2 AND deleted_at IS NULL LIMIT 1`,
[sourceId, slug],
);
const recordedPath = sanitizeRecordedSourcePath(pathRows[0]?.source_path);
const recordedUri = pathRows[0]?.source_uri ?? null;
if (sourceLocalPath) {
if (!existsSync(sourceLocalPath) || !statSync(sourceLocalPath).isDirectory()) {
return { written: false, skipped: 'repo_not_found' };
}
filePath = join(sourceLocalPath, `${slug}.md`);
filePath = join(
sourceLocalPath,
recordedPath ?? recordedPathFromFileUri(recordedUri, sourceLocalPath) ?? `${slug}.md`,
);
writeRoot = sourceLocalPath;
} else {
const repoPath = await engine.getConfig('sync.repo_path');
@@ -127,7 +225,9 @@ export async function writePageThrough(
if (collide.length > 0) {
return { written: false, skipped: 'source_repo_belongs_to_other_source' };
}
filePath = resolvePageFilePath(repoPath, slug, sourceId);
const pageRoot = sourceId === 'default' ? repoPath : join(repoPath, '.sources', sourceId);
const knownPath = recordedPath ?? recordedPathFromFileUri(recordedUri, pageRoot);
filePath = knownPath ? join(pageRoot, knownPath) : resolvePageFilePath(repoPath, slug, sourceId);
writeRoot = repoPath;
}
@@ -204,13 +304,23 @@ export async function writePageThrough(
// post-commit hook background-pushes the commit. Best-effort: a commit
// failure never fails the write (the DB row + file are the durable sinks).
let committed = false;
let pushed: 'pending' | undefined;
let lastPushStatus: PushLogOutcome | undefined;
try {
if (isDurabilityHardened(writeRoot)) {
committed = commitWriteThroughFile(writeRoot, filePath, slug);
if (committed) {
pushed = 'pending';
lastPushStatus = getLastPushOutcome(currentBranch(writeRoot));
}
}
} catch { /* best-effort */ }
return { written: true, path: filePath, ...(committed ? { committed } : {}) };
return {
written: true,
path: filePath,
...(committed ? { committed, pushed, lastPushStatus } : {}),
};
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
opts.logger?.warn(`[write-through] failed for ${slug}: ${msg}`);
templates/bootstrap/AGENTS.md.template
+5 -1
View File
@@ -72,7 +72,11 @@ during long work reads as broken.
**Gate 2 — Recover missed context.** Scan the conversation for earlier messages that
never got processed. Before sending the final reply, rescan for anything that
arrived mid-turn.
arrived mid-turn. On a harness WITHOUT hooks (Codex — pull protocol), also run
`gbrain bootstrap status` once at the start of a conversation: it surfaces a
failing or stale workspace push that hook-carrying harnesses would have shown
automatically. If it reports the push FAILING, tell {{PRINCIPAL_NAME}} plainly —
their memory is landing locally but not in the durable repo.
**Gate 3 — Entity lookup (brain first).** For each real person, project, company, or
commitment named in the message, search the brain before answering: `recall` for hot
templates/bootstrap/CLAUDE.md.template
+4
View File
@@ -20,3 +20,7 @@ reach is a separate knob — see ACCESS_POLICY.md).
over file greps for anything about people, projects, or the past.
- Follow the per-message gates in AGENTS.md — especially Gate 3 (brain first) and
Gate 7 (write-back, same turn).
- Cloud sandbox sessions (fresh clones): if the gbrain MCP tools or hooks are
missing, the binary installs via the environment setup script (print it with
`gbrain bootstrap cloud-setup-script`), then run `gbrain bootstrap hooks
--repair` — committed hooks go live on the NEXT session (startup snapshot).
templates/bootstrap/GITHUB.md.template
+9 -4
View File
@@ -4,12 +4,17 @@ This workspace is durably backed up to **{{GITHUB_REPO_URL}}**. The repository m
remain private.
- **Tracked:** identity files, memory, `brain/`, `skills/`, `state/interview.json`,
`state/mcp.json`, schedules.
- **Ignored:** credentials, local databases, caches, hook state, transcripts, and
schedules.
- **Ignored:** credentials, local databases, caches, hook state, transcripts,
machine-specific harness wiring (`.mcp.json`, `.claude/settings.local.json`), and
anything matching the deny list in `.gitignore`.
- **How it syncs:** `gbrain sources push` — a secret-scan-gated commit + push that
refuses public remotes. It runs at session end automatically; a 15-minute
background job does the same if enabled. Run it by hand after meaningful changes.
refuses public remotes. On Claude Code it runs automatically per turn
(debounced) and at session end via hooks; on Codex (no hook system) run it at
natural stopping points (the AGENTS.md gate reminds you). If background
persistence is enabled, a git post-commit hook auto-pushes each commit and a
30-minute pull job keeps multi-machine checkouts fresh. Run it by hand after
meaningful changes on any harness.
- **If a push is blocked:** the scan names the file and pattern out loud. Fix or
allowlist deliberately — never force past it silently.
- **Honest forget semantics:** git history is append-only. Deleting a line from a
templates/bootstrap/cloud-setup-script.sh
+41
View File
@@ -0,0 +1,41 @@
#!/bin/bash
# gbrain — cloud environment setup script.
# Paste this into your cloud environment's setup script (it runs as root
# before the session starts; what it writes to disk is snapshot-cached and
# reused by later sessions). Printed by: gbrain bootstrap cloud-setup-script
set -eu
# 1. Bun runtime (gbrain runs on bun). Installed VIA npm — bun's own package
# fetching is proxy-incompatible in cloud sandboxes; npm's is not.
command -v bun >/dev/null 2>&1 || npm install -g bun
# 2. gbrain from the canonical GitHub source. NEVER `npm install -g gbrain`:
# the npm registry package with that name is unrelated squatter code.
GBRAIN_DIR=/opt/gbrain
# Pinned to latest-stable — the SAME ref the canonical local install uses.
if [ ! -d "$GBRAIN_DIR/.git" ]; then
git clone --depth 1 --branch latest-stable https://github.com/garrytan/gbrain "$GBRAIN_DIR"
else
# Fail loud (set -e) on a broken update — never npm-install + run stale code
# as root against a half-updated checkout.
git -C "$GBRAIN_DIR" fetch --depth 1 origin latest-stable
git -C "$GBRAIN_DIR" checkout -q FETCH_HEAD
fi
cd "$GBRAIN_DIR"
# npm (not bun) for dependency fetching — same proxy constraint as above.
npm install --no-audit --no-fund
# 3. PATH-resolved launcher: the repo-committed hook commands and MCP
# registration expect `gbrain` on PATH (they are fail-open where it isn't).
cat > /usr/local/bin/gbrain <<'LAUNCHER'
#!/bin/sh
exec bun /opt/gbrain/src/cli.ts "$@"
LAUNCHER
chmod +x /usr/local/bin/gbrain
gbrain --version
# After the session starts, finish wiring INSIDE the session:
# gbrain bootstrap status --json # confirms execution_environment: cloud-sandbox
# gbrain bootstrap attach # adopt the brain repo this session is opened on
# gbrain bootstrap hooks --harness claude-code # committed-carrier hooks (next session picks them up)
templates/bootstrap/gitignore.template
+7
View File
@@ -27,6 +27,13 @@ state/heartbeat-state.local.json
# Hook + harness wiring (machine-specific; regenerated by `bootstrap hooks --repair`)
.claude/settings.local.json
# Backups the hook writers drop next to the committed settings.json — git IS
# the backup for the committed carrier, so these must never be committed.
.claude/*.bak
.claude/*.broken-*
# Written by `claude mcp add` at project scope — carries an absolute,
# machine-specific binary path that must not land in the repo.
.mcp.json
# Caches, logs, scratch
logs/
templates/bootstrap/questions.json
+1 -1
View File
@@ -126,7 +126,7 @@
"PERSIST_CRON": {
"consent": true,
"phase": "repo",
"question": "Enable background persistence? This installs a 15-minute job that commits and pushes this workspace to your private repo (secret-scan-gated). Declining still persists at session end.",
"question": "Enable background persistence? This installs a git post-commit hook that auto-pushes each commit in the background, plus a 30-minute pull job that keeps multi-machine checkouts fresh (on machines with a scheduler). Declining still persists via the per-turn and session-end pushes.",
"default": "no",
"allowed": ["yes", "no"],
"maxLength": 4
templates/bootstrap/template-repo/.gitignore
+7
View File
@@ -27,6 +27,13 @@ state/heartbeat-state.local.json
# Hook + harness wiring (machine-specific; regenerated by `bootstrap hooks --repair`)
.claude/settings.local.json
# Backups the hook writers drop next to the committed settings.json — git IS
# the backup for the committed carrier, so these must never be committed.
.claude/*.bak
.claude/*.broken-*
# Written by `claude mcp add` at project scope — carries an absolute,
# machine-specific binary path that must not land in the repo.
.mcp.json
# Caches, logs, scratch
logs/
templates/bootstrap/template-repo/AGENTS.md
+5 -1
View File
@@ -76,7 +76,11 @@ during long work reads as broken.
**Gate 2 — Recover missed context.** Scan the conversation for earlier messages that
never got processed. Before sending the final reply, rescan for anything that
arrived mid-turn.
arrived mid-turn. On a harness WITHOUT hooks (Codex — pull protocol), also run
`gbrain bootstrap status` once at the start of a conversation: it surfaces a
failing or stale workspace push that hook-carrying harnesses would have shown
automatically. If it reports the push FAILING, tell {{PRINCIPAL_NAME}} plainly —
their memory is landing locally but not in the durable repo.
**Gate 3 — Entity lookup (brain first).** For each real person, project, company, or
commitment named in the message, search the brain before answering: `recall` for hot
templates/bootstrap/template-repo/CLAUDE.md
+4
View File
@@ -20,3 +20,7 @@ reach is a separate knob — see ACCESS_POLICY.md).
over file greps for anything about people, projects, or the past.
- Follow the per-message gates in AGENTS.md — especially Gate 3 (brain first) and
Gate 7 (write-back, same turn).
- Cloud sandbox sessions (fresh clones): if the gbrain MCP tools or hooks are
missing, the binary installs via the environment setup script (print it with
`gbrain bootstrap cloud-setup-script`), then run `gbrain bootstrap hooks
--repair` — committed hooks go live on the NEXT session (startup snapshot).
templates/bootstrap/template-repo/GITHUB.md
+9 -4
View File
@@ -4,12 +4,17 @@ This workspace is durably backed up to **(not yet created — bootstrap repo set
remain private.
- **Tracked:** identity files, memory, `brain/`, `skills/`, `state/interview.json`,
`state/mcp.json`, schedules.
- **Ignored:** credentials, local databases, caches, hook state, transcripts, and
schedules.
- **Ignored:** credentials, local databases, caches, hook state, transcripts,
machine-specific harness wiring (`.mcp.json`, `.claude/settings.local.json`), and
anything matching the deny list in `.gitignore`.
- **How it syncs:** `gbrain sources push` — a secret-scan-gated commit + push that
refuses public remotes. It runs at session end automatically; a 15-minute
background job does the same if enabled. Run it by hand after meaningful changes.
refuses public remotes. On Claude Code it runs automatically per turn
(debounced) and at session end via hooks; on Codex (no hook system) run it at
natural stopping points (the AGENTS.md gate reminds you). If background
persistence is enabled, a git post-commit hook auto-pushes each commit and a
30-minute pull job keeps multi-machine checkouts fresh. Run it by hand after
meaningful changes on any harness.
- **If a push is blocked:** the scan names the file and pattern out loud. Fix or
allowlist deliberately — never force past it silently.
- **Honest forget semantics:** git history is append-only. Deleting a line from a
templates/bootstrap/template-repo/README.md
+1 -1
View File
@@ -1,6 +1,6 @@
# gbrain agent workspace — template
<!-- gbrain-template-stamp: 0.45.10.0 -->
<!-- gbrain-template-stamp: 0.45.11.0 -->
This repository is the **"Use this template"** distribution artifact for a
[gbrain](https://github.com/garrytan/gbrain) personal-agent workspace — the same
test/agent-cli.test.ts
+196 -2
View File
@@ -8,13 +8,14 @@
* glue.
*/
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
import { describe, test, expect, beforeAll, afterAll, beforeEach, spyOn } from 'bun:test';
import * as fs from 'node:fs';
import * as path from 'node:path';
import * as os from 'node:os';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { MinionQueue } from '../src/core/minions/queue.ts';
import { __testing as agentTesting } from '../src/commands/agent.ts';
import { __testing as agentTesting, runAgentRun } from '../src/commands/agent.ts';
import { withEnv } from './helpers/with-env.ts';
import { parseSince } from '../src/commands/agent-logs.ts';
import { isProtectedJobName, PROTECTED_JOB_NAMES } from '../src/core/minions/protected-names.ts';
@@ -137,6 +138,17 @@ describe('parseRunFlags', () => {
const { flags } = agentTesting.parseRunFlags(['--fanout-manifest', '/tmp/m.json']);
expect(flags.fanoutManifest).toBe('/tmp/m.json');
});
test('#2922: --source parsed as a leading value-flag', () => {
const { flags, rest } = agentTesting.parseRunFlags(['--source', 'corporate', 'do', 'x']);
expect(flags.source).toBe('corporate');
expect(rest).toEqual(['do', 'x']);
});
test('#2922: --source missing its value throws a usage error', () => {
expect(() => agentTesting.parseRunFlags(['--source'])).toThrow(/requires a value/);
expect(() => agentTesting.parseRunFlags(['--source', '--detach', 'x'])).toThrow(/requires a value/);
});
});
describe('parseSince', () => {
@@ -266,6 +278,188 @@ describe('queue.add trusted-submit gate for subagent', () => {
});
});
describe('#2922: submit-time source resolution', () => {
beforeEach(async () => {
await engine.executeRaw(`DELETE FROM sources WHERE id != 'default'`);
await engine.unsetConfig('sources.default');
await engine.executeRaw(
`INSERT INTO sources (id, name) VALUES ('corporate', 'Corporate') ON CONFLICT (id) DO NOTHING`,
);
});
async function jobData(jobId: number): Promise<Record<string, unknown>> {
const rows = await engine.executeRaw<{ data: unknown }>(
`SELECT data FROM minion_jobs WHERE id = $1`, [jobId],
);
return typeof rows[0]!.data === 'string'
? JSON.parse(rows[0]!.data as string)
: rows[0]!.data as Record<string, unknown>;
}
async function onlyJobData(): Promise<Record<string, unknown>> {
const rows = await engine.executeRaw<{ id: number }>(
`SELECT id FROM minion_jobs WHERE name = 'subagent' ORDER BY id`,
);
expect(rows.length).toBe(1);
return jobData(rows[0]!.id);
}
test('explicit --source lands on SubagentHandlerData.source_id', async () => {
await withEnv({ GBRAIN_SOURCE: undefined }, async () => {
await runAgentRun(engine, ['--detach', '--source', 'corporate', 'write', 'a', 'page']);
const data = await onlyJobData();
expect(data.source_id).toBe('corporate');
expect(data.prompt).toBe('write a page');
});
});
test('no --source: sources.default (tier 5) is honored instead of the seed default', async () => {
await withEnv({ GBRAIN_SOURCE: undefined }, async () => {
await engine.setConfig('sources.default', 'corporate');
await runAgentRun(engine, ['--detach', 'write', 'a', 'page']);
const data = await onlyJobData();
expect(data.source_id).toBe('corporate');
});
});
test('GBRAIN_SOURCE env (tier 2) is honored', async () => {
await withEnv({ GBRAIN_SOURCE: 'corporate' }, async () => {
await runAgentRun(engine, ['--detach', 'write', 'a', 'page']);
const data = await onlyJobData();
expect(data.source_id).toBe('corporate');
});
});
test('no signal at all: resolves to the seed default (legacy behavior preserved)', async () => {
await withEnv({ GBRAIN_SOURCE: undefined }, async () => {
await runAgentRun(engine, ['--detach', 'write', 'a', 'page']);
const data = await onlyJobData();
expect(data.source_id).toBe('default');
});
});
test('fan-out children all carry the resolved source_id', async () => {
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'fanout-source-'));
try {
await withEnv({ GBRAIN_SOURCE: undefined }, async () => {
const manifestPath = path.join(tmp, 'm.json');
fs.writeFileSync(manifestPath, JSON.stringify([
{ prompt: 'chunk 1' }, { prompt: 'chunk 2' },
]));
await runAgentRun(engine, [
'--source', 'corporate', '--fanout-manifest', manifestPath, '--detach',
]);
const rows = await engine.executeRaw<{ id: number }>(
`SELECT id FROM minion_jobs WHERE name = 'subagent' ORDER BY id`,
);
expect(rows.length).toBe(2);
for (const r of rows) {
const data = await jobData(r.id);
expect(data.source_id).toBe('corporate');
}
});
} finally {
fs.rmSync(tmp, { recursive: true, force: true });
}
});
test('--source "" (empty explicit value) exits 2 without silently falling back', async () => {
const spy = spyOn(process, 'exit').mockImplementation(() => { throw new Error('EXIT'); });
const errSpy = spyOn(console, 'error').mockImplementation(() => {});
try {
await withEnv({ GBRAIN_SOURCE: undefined }, async () => {
try {
await runAgentRun(engine, ['--detach', '--source', '', 'write', 'a', 'page']);
throw new Error('expected runAgentRun to exit');
} catch (e: any) {
expect(e.message).toBe('EXIT');
}
});
expect(spy).toHaveBeenCalledWith(2);
const rows = await engine.executeRaw<{ id: number }>(
`SELECT id FROM minion_jobs WHERE name = 'subagent'`,
);
expect(rows.length).toBe(0);
} finally {
spy.mockRestore();
errSpy.mockRestore();
}
});
test('--source __all__ is rejected (subagent writes must target exactly one source)', async () => {
const spy = spyOn(process, 'exit').mockImplementation(() => { throw new Error('EXIT'); });
const errSpy = spyOn(console, 'error').mockImplementation(() => {});
try {
await withEnv({ GBRAIN_SOURCE: undefined }, async () => {
try {
await runAgentRun(engine, ['--detach', '--source', '__all__', 'write', 'a', 'page']);
throw new Error('expected runAgentRun to exit');
} catch (e: any) {
expect(e.message).toBe('EXIT');
}
});
expect(spy).toHaveBeenCalledWith(2);
const rows = await engine.executeRaw<{ id: number }>(
`SELECT id FROM minion_jobs WHERE name = 'subagent'`,
);
expect(rows.length).toBe(0);
} finally {
spy.mockRestore();
errSpy.mockRestore();
}
});
test('--source pointing at a nonexistent id surfaces a clean error, not a stack trace', async () => {
const spy = spyOn(process, 'exit').mockImplementation(() => { throw new Error('EXIT'); });
const errSpy = spyOn(console, 'error').mockImplementation(() => {});
try {
await withEnv({ GBRAIN_SOURCE: undefined }, async () => {
try {
await runAgentRun(engine, ['--detach', '--source', 'does-not-exist', 'write', 'a', 'page']);
throw new Error('expected runAgentRun to exit');
} catch (e: any) {
expect(e.message).toBe('EXIT');
}
});
expect(spy).toHaveBeenCalledWith(1);
expect(errSpy.mock.calls.some(call => String(call[0]).includes('not found'))).toBe(true);
const rows = await engine.executeRaw<{ id: number }>(
`SELECT id FROM minion_jobs WHERE name = 'subagent'`,
);
expect(rows.length).toBe(0);
} finally {
spy.mockRestore();
errSpy.mockRestore();
}
});
test('--source pointing at an archived source is rejected with a restore hint', async () => {
const spy = spyOn(process, 'exit').mockImplementation(() => { throw new Error('EXIT'); });
const errSpy = spyOn(console, 'error').mockImplementation(() => {});
try {
await engine.executeRaw(`UPDATE sources SET archived = true WHERE id = 'corporate'`);
await withEnv({ GBRAIN_SOURCE: undefined }, async () => {
try {
await runAgentRun(engine, ['--detach', '--source', 'corporate', 'write', 'a', 'page']);
throw new Error('expected runAgentRun to exit');
} catch (e: any) {
expect(e.message).toBe('EXIT');
}
});
expect(spy).toHaveBeenCalledWith(1);
expect(errSpy.mock.calls.some(call => String(call[0]).includes('archived'))).toBe(true);
const rows = await engine.executeRaw<{ id: number }>(
`SELECT id FROM minion_jobs WHERE name = 'subagent'`,
);
expect(rows.length).toBe(0);
} finally {
await engine.executeRaw(`UPDATE sources SET archived = false WHERE id = 'corporate'`);
spy.mockRestore();
errSpy.mockRestore();
}
});
});
describe('fan-out manifest shape (integration)', () => {
test('fanout-manifest with 3 entries creates 3 subagent children + 1 aggregator', async () => {
// Manually replicate what runAgentRun does for --fanout-manifest > 1.
test/ambient-recall.test.ts
+12
View File
@@ -16,6 +16,7 @@
*/
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { configureGateway } from '../src/core/ai/gateway.ts';
import { operations } from '../src/core/operations.ts';
import { MEMORY_VERBS_VERSION, VERB_NAMES } from '../src/core/verbs.ts';
import {
@@ -60,6 +61,17 @@ async function call(
}
beforeAll(async () => {
// Hermetic embedding: pin the gateway to a KEYLESS config (empty env) so
// `remember`'s fact-embed degrades gracefully (degraded_dedup) instead of
// firing a real OpenAI call. On CI the process carries a dummy
// OPENAI_API_KEY (sk-test-*) that a shard-neighbor can leak into the
// gateway singleton via a captured env (the bunfig preload configures with
// `env: {...process.env}`); a present-but-invalid key turns the keyless
// degrade into a hard 401. The delta/context_pack tests exercise
// cursor/budget logic, not embedding quality, so keyless is correct and
// makes them independent of shard bin-packing. Dimensions stay 1536 to
// match the preload's schema.
configureGateway({ embedding_model: 'openai:text-embedding-3-large', embedding_dimensions: 1536, env: {} });
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
test/autopilot-drain-deadlock-2050.test.ts
+237
View File
@@ -0,0 +1,237 @@
/**
* #2050 — a patterns/synthesize parent job that blocks on its own subagent
* child must not deadlock a fully-occupied Postgres drain worker.
*
* Pre-fix, the inline child drain was PGLite-only (runPgliteSubagentsInline
* returned early unless engine.kind === 'pglite') and Postgres children were
* submitted to the 'default' queue. Autopilot spawns its drain worker with no
* --concurrency (resolves to 1), so the parent occupied the only slot, the
* child was never claimed, and the phase burned the whole
* subagent_wait_timeout_ms window before cancelling its own child — every
* cycle, forever.
*
* These tests run the REAL phase/drain code against a PGLite engine masked as
* kind='postgres' (only the `kind` property is intercepted; every method call
* hits the real engine), which is exactly the branch the production Postgres
* environment takes. All assertions are behavioral: on unmodified master they
* fail with outcome 'timeout' / a never-claimed 'waiting' child on the
* 'default' queue.
*/
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
import { mkdtempSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import type { BrainEngine } from '../src/core/engine.ts';
import { resetPgliteState } from './helpers/reset-pglite.ts';
import { runPhasePatterns } from '../src/core/cycle/patterns.ts';
import { MinionQueue } from '../src/core/minions/queue.ts';
import { withEnv } from './helpers/with-env.ts';
let engine: PGLiteEngine;
let schemaVersion: string;
/** Mask a PGLite engine as Postgres: only `kind` is intercepted, every
* method executes on the real engine (bound to the target so internal
* state is untouched). */
function maskAsPostgres(target: PGLiteEngine): BrainEngine {
return new Proxy(target, {
get(t, prop) {
if (prop === 'kind') return 'postgres';
const v = Reflect.get(t, prop);
return typeof v === 'function' ? v.bind(t) : v;
},
}) as unknown as BrainEngine;
}
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({ database_url: '' });
await engine.initSchema();
schemaVersion = (await engine.getConfig('version')) ?? '7';
}, 60_000);
afterAll(async () => {
await engine.disconnect();
});
beforeEach(async () => {
await resetPgliteState(engine);
await engine.setConfig('version', schemaVersion);
});
async function seedReflections(): Promise<void> {
for (let i = 0; i < 3; i++) {
await engine.executeRaw(
`INSERT INTO pages (slug, type, title, compiled_truth)
VALUES ($1, 'note', $2, $3)`,
[
`wiki/personal/reflections/2026-07-0${i + 1}-reflection`,
`Reflection ${i + 1}`,
`Recurring theme fixture number ${i + 1}.`,
],
);
}
}
describe('#2050 — patterns parent must not deadlock on its own child (postgres path)', () => {
test('child is drained inline on a private queue instead of waiting out the parent', async () => {
const brainDir = mkdtempSync(join(tmpdir(), 'gbrain-2050-patterns-'));
try {
await seedReflections();
// Small wait window: on master the child is never claimed (no worker in
// this process, parent would hold the only slot in production), so the
// phase burns this whole window and reports outcome 'timeout'.
await engine.setConfig('dream.patterns.subagent_wait_timeout_ms', '2000');
const pgAlike = maskAsPostgres(engine);
const result = await withEnv({ ANTHROPIC_API_KEY: 'sk-ant-test' }, () =>
runPhasePatterns(pgAlike, { brainDir, dryRun: false }),
);
// Master: child_outcome 'timeout' + PATTERNS_CHILD_TIMEOUT (self-deadlock,
// then the phase cancels its own child). Fixed: the inline drain actually
// ran the child (the fake API key fails fast → 'dead'), no deadlock.
expect(result.details.child_outcome).toBe('dead');
expect(result.error?.code).toBe('PATTERNS_CHILD_DEAD');
const jobs = await engine.executeRaw<{ queue: string; status: string }>(
`SELECT queue, status FROM minion_jobs WHERE name = 'subagent' ORDER BY id DESC LIMIT 1`,
);
expect(jobs).toHaveLength(1);
// Master: queue 'default' (claimable by — and deadlocked behind — the
// same worker running the parent), status 'cancelled'.
expect(jobs[0].queue).toStartWith('dream-inline-');
expect(jobs[0].status).toBe('dead');
} finally {
rmSync(brainDir, { recursive: true, force: true });
}
}, 60_000);
test('inline drain runs children on a non-pglite engine and heartbeats their lock', async () => {
const { __testing } = await import('../src/core/cycle/synthesize.ts');
const drain = (__testing as Record<string, unknown>).runSubagentsInline
?? (__testing as Record<string, unknown>).runPgliteSubagentsInline;
const pgAlike = maskAsPostgres(engine);
const queue = new MinionQueue(pgAlike);
const job = await queue.add(
'subagent',
{ prompt: 'noop', model: 'anthropic:claude-sonnet-4-5', max_turns: 1 },
{ queue: 'inline-2050-test', max_stalled: 3 },
{ allowProtectedSubmit: true },
);
// Stub handler outliving the (shortened) claim lock: sleeps past lockMs,
// then runs the same handleStalled() sweep a concurrent Postgres worker
// fires on a timer. With the heartbeat the lock is fresh and the sweep
// must NOT touch the running child; without it the child would be
// requeued mid-run (stall churn → dead after max_stalled).
let sweptWhileRunning = 0;
const stub = async () => {
await new Promise((r) => setTimeout(r, 1400));
const { requeued, dead } = await queue.handleStalled();
sweptWhileRunning = [...requeued, ...dead].filter((j) => j.id === job.id).length;
return { ok: true };
};
// Master behavioral failure: runPgliteSubagentsInline no-ops on a
// kind='postgres' engine, the stub never runs, the job stays 'waiting'.
await (drain as (
e: BrainEngine, q: MinionQueue, name: string,
y?: () => Promise<void>, h?: unknown, lockMs?: number,
) => Promise<void>)(pgAlike, queue, 'inline-2050-test', undefined, stub, 1000);
const after = await queue.getJob(job.id);
expect(after?.status).toBe('completed');
expect(sweptWhileRunning).toBe(0);
}, 30_000);
// #3555 interaction: the drain's queue ops must survive a transient pooler
// reap (reconnect + retry) instead of throwing out of the loop and
// stranding children in the per-run private queue no worker will claim.
/** Mask as Postgres AND expose a counting reconnect() for the drain's
* recovery path (the real PGLite engine has no reconnect method). */
function maskAsPostgresWithReconnect(target: PGLiteEngine, onReconnect: () => void): BrainEngine {
return new Proxy(target, {
get(t, prop) {
if (prop === 'kind') return 'postgres';
if (prop === 'reconnect') return async () => { onReconnect(); };
const v = Reflect.get(t, prop);
return typeof v === 'function' ? v.bind(t) : v;
},
}) as unknown as BrainEngine;
}
const connError = () =>
Object.assign(new Error('write CONNECTION_ENDED supavisor reaped the socket'), { code: 'CONNECTION_ENDED' });
test('a transient connection error on claim reconnects and the child still completes', async () => {
const { __testing } = await import('../src/core/cycle/synthesize.ts');
const drain = (__testing as Record<string, unknown>).runSubagentsInline;
let reconnects = 0;
const pgAlike = maskAsPostgresWithReconnect(engine, () => { reconnects++; });
const queue = new MinionQueue(pgAlike);
const job = await queue.add(
'subagent',
{ prompt: 'noop', model: 'anthropic:claude-sonnet-4-5', max_turns: 1 },
{ queue: 'inline-2050-conn-claim', max_stalled: 3 },
{ allowProtectedSubmit: true },
);
let failuresLeft = 1;
const realClaim = queue.claim.bind(queue);
queue.claim = (async (...args: Parameters<MinionQueue['claim']>) => {
if (failuresLeft-- > 0) throw connError();
return realClaim(...args);
}) as MinionQueue['claim'];
// Pre-fix: the bare-await claim throws out of the drain, the child
// strands 'waiting' in the private queue, and this await rejects.
await (drain as (
e: BrainEngine, q: MinionQueue, name: string,
y?: () => Promise<void>, h?: unknown, lockMs?: number,
) => Promise<void>)(pgAlike, queue, 'inline-2050-conn-claim', undefined, async () => ({ ok: true }), 1000);
const after = await queue.getJob(job.id);
expect(after?.status).toBe('completed');
expect(reconnects).toBe(1);
}, 30_000);
test('a transient connection error recording the outcome retries once and the child still completes', async () => {
const { __testing } = await import('../src/core/cycle/synthesize.ts');
const drain = (__testing as Record<string, unknown>).runSubagentsInline;
let reconnects = 0;
const pgAlike = maskAsPostgresWithReconnect(engine, () => { reconnects++; });
const queue = new MinionQueue(pgAlike);
const job = await queue.add(
'subagent',
{ prompt: 'noop', model: 'anthropic:claude-sonnet-4-5', max_turns: 1 },
{ queue: 'inline-2050-conn-record', max_stalled: 3 },
{ allowProtectedSubmit: true },
);
let failuresLeft = 1;
const realComplete = queue.completeJob.bind(queue);
queue.completeJob = (async (...args: Parameters<MinionQueue['completeJob']>) => {
if (failuresLeft-- > 0) throw connError();
return realComplete(...args);
}) as MinionQueue['completeJob'];
// Pre-fix: completeJob's rejection landed in the drain's catch, which
// called failJob for a handler that SUCCEEDED (misrecorded outcome) — or
// escaped the drain entirely if failJob also hit the dead pool.
await (drain as (
e: BrainEngine, q: MinionQueue, name: string,
y?: () => Promise<void>, h?: unknown, lockMs?: number,
) => Promise<void>)(pgAlike, queue, 'inline-2050-conn-record', undefined, async () => ({ ok: true }), 1000);
const after = await queue.getJob(job.id);
expect(after?.status).toBe('completed');
expect(reconnects).toBe(1);
}, 30_000);
});

Some files were not shown because too many files have changed in this diff Show More