Compare commits

...
39 Commits
Author SHA1 Message Date
Garry Tan 04bd97ab75 Merge remote-tracking branch 'origin/master' into garrytan/nouakchott
# Conflicts:
#	BOOTSTRAP_FOR_AGENTS.md
#	CHANGELOG.md
#	VERSION
#	openclaw.plugin.json
#	package.json
#	src/core/cli-flag-registry.generated.ts
#	templates/bootstrap/template-repo/README.md
2026-08-13 15:25:13 -07:00
testandClaude Fable 5 ac402f55f8 v0.45.10.0 fix: community fix-wave 2 — 21 fixes from 9 contributors + maintainer (search completeness, sync safety, doctor honesty)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 12:18:13 -07:00
test cf3527a40f test: conform the #3762 rider to test-isolation rules R3/R4 (engine in beforeAll, afterAll disconnect) 2026-08-13 12:18:13 -07:00
test e2b2819e63 chore: regenerate CLI flag registry after wave-2 assembly 2026-08-13 12:18:13 -07:00
testandGarry Tan 2ae5d60b98 fix(doctor): distinguish a damaged PGLite store from a broken runtime (#2674) (#3569)
Wave-assembled from PR #3569 by @time-attack.

Co-Authored-By: Garry Tan <garrytan@gmail.com>
2026-08-13 12:18:13 -07:00
testandGarry Tan bd4c976a85 fix(sync,webhook): consume deferred link extraction above the size gate (#2849) (#3561)
Wave-assembled from PR #3561 by @time-attack.

Co-Authored-By: Garry Tan <garrytan@gmail.com>
2026-08-13 12:18:13 -07:00
testandGarry Tan 9b9bd8b241 fix(autopilot): stop the drain-worker self-deadlocking at concurrency=1 (#2050) (#3559)
Wave-assembled from PR #3559 by @time-attack.

Co-Authored-By: Garry Tan <garrytan@gmail.com>
2026-08-13 12:18:13 -07:00
testandGarry Tan fd9bb12b42 fix(doctor,upgrade): warn brains pinned to a provider shutting down 2026-09-04 (#3548)
Wave-assembled from PR #3548 by @time-attack. Composition conflicts vs wave
item #3993 resolved: EmbedResult keeps both chunkless_pages_healed and the
PR's lock_skipped; KEY_FILES.md keeps the #3993 embed entry and takes this
PR's migrate-embeddings entry.

Co-Authored-By: Garry Tan <garrytan@gmail.com>
2026-08-13 12:18:13 -07:00
testandTime Attakc 5bd2c51053 fix(search): fold detail into the query-cache key (#3515) (#3544)
Wave-assembled from PR #3544 by @time-attack.

Co-Authored-By: Time Attakc <89218912+time-attack@users.noreply.github.com>
2026-08-13 12:18:13 -07:00
testandTime Attakc 189bf856ee feat(oauth): accept token_ttl_seconds at DCR, clamped to admin policy (#2179) (#3456)
Wave-assembled from PR #3456 by @time-attack.

Co-Authored-By: Time Attakc <89218912+time-attack@users.noreply.github.com>
2026-08-13 12:18:13 -07:00
testandGarry Tan 735dec83b7 fix(extract,cli,pricing): link-aware timeline delimiter, dispatchable backfill, zerank-2 rerank pricing (nwf2) (#3280)
Wave-assembled from PR #3280 by @time-attack.

Co-Authored-By: Garry Tan <garrytan@gmail.com>
2026-08-13 12:18:13 -07:00
testandRerankerGuo fd0e371d5b fix(sync): acknowledge auto-skipped failures (#3829) (#3891)
Wave-assembled from PR #3891 by @RerankerGuo. Conflict resolution: KEY_FILES.md
kept master's newer migrate-engine/import-file/sync entries; the PR's 3-state
machine wording re-applied onto master's sync-failure-ledger entry.

Co-Authored-By: RerankerGuo <121015044+RerankerGuo@users.noreply.github.com>
2026-08-13 12:18:13 -07:00
testandRerankerGuo 136fc109c1 fix(doctor): parse flow-style skill triggers (#3887) (#3888)
Wave-assembled from PR #3888 by @RerankerGuo. Conflict resolution: KEY_FILES.md
check-resolvable/skill-frontmatter/skill-trigger-index entries taken as the
PR's current-state prose; the PR's stale pre-fold 'doctor.ts extension' entry
dropped (master already folded it into the main doctor entry).

Co-Authored-By: RerankerGuo <121015044+RerankerGuo@users.noreply.github.com>
2026-08-13 12:18:13 -07:00
testandRerankerGuo 1243a7d3bb fix(migrate): preserve cross-source link targets (#3859) (#3876)
Wave-assembled from PR #3876 by @RerankerGuo. Conflict resolution: re-applied
the to_source_id fix (copyPageLinksToTarget) onto master's rewritten
migrate-engine.ts link-copy loop; KEY_FILES.md kept master's newer
import-file/sync entries and merged the PR's to_source_id prose + test pin
into the migrate-engine entry.

Co-Authored-By: RerankerGuo <121015044+RerankerGuo@users.noreply.github.com>
2026-08-13 12:18:13 -07:00
testandDov Stern 44eea64084 put_page: report push state honestly instead of implying it via committed (#3936)
Wave-assembled from PR #3936 by @dovstern. Conflict resolution: kept master's
resolveRepoRoot() block AND the PR's exported currentBranch in
src/core/brain-repo-durability.ts. Adaptation: the new serial test now writes
the simulated push log under $GBRAIN_HOME/.gbrain (CX2-8 parent-dir semantics
landed on master after the PR's base).

Co-Authored-By: Dov Stern <dovstern@users.noreply.github.com>
2026-08-13 12:18:13 -07:00
testandDov Stern 45bd04ff9f sources harden: resolve _brain-filing-rules.json per repo instead of always using the bundled copy (#4018)
Wave-assembled from PR #4018 by @dovstern. Conflict resolution: import-block
union in src/core/brain-repo-durability.ts (master's gbrain-home/execution-env
imports kept, PR's loadFilingRules import + resolveFilingRules threading
re-applied onto master's shape). Rider: test renamed to
filing-rules-resolution.serial.test.ts (mutates HOME).

Co-Authored-By: Dov Stern <dovstern@users.noreply.github.com>
2026-08-13 12:18:13 -07:00
testandmasashiono0611 8a626999f0 fix(embed): detect chunkless pages with non-empty content in embed --stale (#3993)
Wave-assembled from PR #3993 by @Masashi-Ono0611. Conflict resolution:
KEY_FILES.md embed.ts entry unioned (PR's chunkless prose + master's newer
keyless-brains tail). Semantic fix: master's keyless clean-refusal return in
runEmbed gained the PR's required chunkless_pages_healed field.

Co-Authored-By: masashiono0611 <masashi.ono.0611@gmail.com>
2026-08-13 12:18:13 -07:00
testandClément Barberousse 033029f25d feat(claude-cli): list the Claude 5 models the CLI already serves (#3976)
Wave-assembled from PR #3976 by @clement0909472. Rider: four toContain pins
(claude-fable-5, claude-opus-5, claude-opus-4-8, claude-sonnet-5) in
test/claude-cli-recipe.test.ts.

Co-Authored-By: Clément Barberousse <clement.barberousse.pro@gmail.com>
2026-08-13 12:18:13 -07:00
testandDov Stern a729ca8a8c sources: list shows unset federation distinctly from explicit isolated (#3940)
Wave-assembled from PR #3940 by @dovstern.

Co-Authored-By: Dov Stern <dovstern@users.noreply.github.com>
2026-08-13 12:18:13 -07:00
testandGBrain Contributor 5087507de0 fix(autopilot): ignore foreign PIDs in stale locks (#2503) (#3860)
Wave-assembled from PR #3860 by @javieraldape.

Co-Authored-By: GBrain Contributor <contributor@example.com>
2026-08-13 12:18:13 -07:00
testandBo ed6e4e3219 fix(import): preserve table/constraint names in error-summary grouping (#3841)
Wave-assembled from PR #3841 by @bo-developing.

Co-Authored-By: Bo <bo.developing@gmail.com>
2026-08-13 12:18:13 -07:00
testandJon McCutchen f8b0ececcb fix(write-through): honor the page's recorded source_path instead of re-deriving <slug>.md (#3782)
Wave-assembled from PR #3782 by @JonMcCutchen. Rider-check: verified the
non-default pageRoot join (repoPath/.sources/<sourceId>) matches how
pages.source_path is recorded (source-root-relative via importFile's
relative(dir, filePath)); no mismatch, no change needed.

Co-Authored-By: Jon McCutchen <jmmccutchen1@gmail.com>
2026-08-13 12:18:13 -07:00
testandAustin Wilhite ca260baaaa fix(cycle): extract_atoms writes atoms outside the chunk/embed pipeline, making every atom invisible to search (#3762)
Wave-assembled from PR #3762 by @awilhite. Rider: PGLite round-trip chunk-count
assertion in test/extract-atoms-chunk-embed.test.ts (verified fails without the
src change).

Co-Authored-By: Austin Wilhite <austinw80@gmail.com>
2026-08-13 12:18:13 -07:00
testandAlexey 0c485415a5 fix(export): scope tag + raw-data sidecar reads to the page's own source (#3755)
Wave-assembled from PR #3755 by @alexey-metaengage.

Co-Authored-By: Alexey <alexey@metaengage.ai>
2026-08-13 12:18:13 -07:00
testandmasashiono0611 0cfedd026d fix(agent): resolve the brain source at submit time instead of hardcoding the seed default (#3647)
Wave-assembled from PR #3647 by @Masashi-Ono0611.

Co-Authored-By: masashiono0611 <masashi.ono.0611@gmail.com>
2026-08-13 12:18:13 -07:00
Garry TanandClaude Fable 5 2817dd47e4 test(ci): stop memory-verbs-conformance leaking a fake-keyed gateway into shard-mates
The deterministic-embedder helper configures the MODULE-GLOBAL gateway with a
fake OpenAI key; the file's afterAll never reset it. The bunfig preload's
per-test restore only fires when the gateway is UNCONFIGURED, so the fake-keyed
config persisted for every later file in the shard process — turn-context's
corpus writes then embedded against real OpenAI and 401'd (CI shard-8 failure;
shard re-binning from this branch's new test files exposed it).

Fix both sides: conformance's afterAll now resetGateway()s back to the preload
baseline and nulls both test transports; turn-context's beforeAll does the same
defensively so it stays hermetic regardless of shard composition.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 10:09:19 -07:00
Garry TanandClaude Fable 5 a34935a312 Merge remote-tracking branch 'origin/master' into garrytan/nouakchott + v0.45.11.0
Master's 12-commit fix batch (#4064 hooks-phase source_id + eager brain/,
#4065 --help mutation guards, #4063 Gate-2 active-account check, plus search/
doctor/dream/autopilot/skills fixes) merges cleanly except one import-union
conflict in the dispatcher test. Release re-bumped 0.45.10.0 → 0.45.11.0 per
the release queue (a sibling wave holds an in-flight claim): VERSION,
package.json, openclaw.plugin.json, runbook stamp, template tree, CHANGELOG
header + self-references all move together. Derived files (template tree,
llms bundles, skills lock, flag registry) regenerated from merged source.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 09:53:38 -07:00
Javier AldapeandSofía González b92cc967df fix(search): bucket rerank budget failures (#3628) (#4002)
Co-authored-by: Sofía González <sofiagonzalez@Sofias-MacBook-Air.local>
2026-08-13 07:01:12 -07:00
MasaandClaude Sonnet 5 3eccd4ccd6 v0.45.9.0 fix(bootstrap): preview source_id + create brain/ eagerly in hooks phase (#4064)
* fix(bootstrap): preview source_id + create brain/ eagerly in hooks phase

`bootstrap render`/`hooks` never told a human what source_id the
workspace expects until `verify` (the only engine-holding phase) ran.
A human who hand-registered a source before that point would guess an
"intuitive" name, hit an FK error on the first `verify` roundtrip (the
guessed id has no `sources` row), then hit `overlapping_path` on the
retry (their first guess still claims the same brain/ dir) — three
round trips to land the right id.

`hooks` is the last ENGINE-FREE phase before `verify`, and already
knows both the manifest's current source_id and the workspace path, so
it now:
  - creates `<ws>/brain` eagerly (idempotent mkdir), removing the
    manual-mkdir step before `git init && sources add`
  - prints the exact `gbrain sources add <source_id> --path <brain>`
    command
  - previews the collision-fallback id verify would derive
    (`workspace-<hash>`) — a pure function of the workspace's real
    path, so it needs no DB lookup and is safe to preview engine-free

The collision-fallback derivation itself is unchanged; it is now
factored into an exported `deriveWorkspaceSourceId()` in verify.ts so
both call sites (the new hooks preview and the existing
`resolveSourceIdCollision`) share one formula instead of two copies
drifting apart.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016JNV9ABwb32DZEwvo7P4ay

* fix(bootstrap): --force the preview command; quote paths; fix runbook

Codex review round 2 caught three issues with the source_id preview
added in the prior commit:

- The printed `gbrain sources add <id> --path <brain>` command failed
  immediately on a pristine bootstrap: the brain/ dir this phase just
  created is empty (no git history), so `sources add --path` fail-fasts
  as `not_a_git_repo` (#2707). Fixed by appending `--force` — the same
  sanctioned opt-in `test/bootstrap-verify.serial.test.ts` already uses
  to register a brand-new brain/ before any content exists
  (`addSource(engine, { id: 'workspace', localPath: ..., force: true })`).
  Safe here specifically because brainDir is the fixed
  `<workspace>/brain` path this phase just created, not an arbitrary
  user path.
- brainDir was interpolated unquoted; a workspace path containing a
  space broke the printed command. Added a local
  `shellQuoteForDisplay()` (mirroring the existing private `shellQuote`
  already duplicated in hooks.ts / sources-ops.ts / connect.ts).
- The dispatcher test only pattern-matched the collision-fallback id's
  shape (`workspace-[0-9a-f]{8}`) instead of pinning exact equality
  with `deriveWorkspaceSourceId()`, so preview/verify drift could pass
  silently. Now asserts exact equality, plus a new test for the space-
  quoting fix.

Also corrects BOOTSTRAP_FOR_AGENTS.md's runbook step 5, which claimed
skill scaffolding "registers `brain/` as the workspace source" — no
code path does this automatically (confirmed by grep); the step now
points at the `hooks` phase's actual preview + --force command instead
of telling the installing agent there is "nothing to judge" on a step
that silently never ran.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016JNV9ABwb32DZEwvo7P4ay

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-13 06:50:06 -07:00
Masa 154814b095 fix(bootstrap): guard mutating subcommands against --help side effects (#4065)
`gbrain bootstrap <subcommand> --help` (a help token AFTER the subcommand
name, e.g. `gbrain bootstrap uninstall --help`) fell through into the
subcommand's own arg parsing instead of printing help, since none of the
mutating handlers (repo/hooks/verify/attach/uninstall/render/interview)
checked for --help/-h/help themselves. `uninstall --help` ran a real
uninstall; `repo --help` created a real private GitHub repo; etc.

Add a SUBCOMMAND_HELP usage map plus a pre-dispatch hasHelpToken() guard in
runBootstrap so a help token anywhere in the subcommand's args short-circuits
before any lock/runner/engine/handler call. Bare `help` (no dashes) is also
recognized, except for `interview` (its --set KEY value free-text answers
could legitimately be the literal word "help").

New test/bootstrap-subcommand-help.serial.test.ts arms fixtures so the real
operation would reach its side effect if the guard were removed (an
already-rendered workspace for render/hooks/attach, an operational verify
config, an isolated uninstall home with a real receipt-tracked file, a fresh
interview workspace) and asserts nothing mutates.
2026-08-13 06:19:20 -07:00
MasaandClaude Fable 5 9b720b04af fix(doctor): honor the recorded pid_file in supervisor_singleton check (#4006)
* fix(doctor): honor the recorded pid_file in supervisor_singleton check

`gbrain doctor`'s `supervisor_singleton` check (#1849) compares the local
pidfile holder against the queue-scoped DB lock holder. It read
`readSupervisorPid(DEFAULT_PID_FILE)` unconditionally, even though the
supervisor's own 'started' audit event already records the pid-file path
actually in use (`this.opts.pidFile`). A supervisor launched with a custom
`--pid-file` (e.g. a launchd-managed deployment) would then get a false
"singleton mismatch" warning against its own healthy, single instance,
because the pidfile doctor read was never the one the supervisor wrote.

- doctor.ts now prefers `lastStarted.pid_file` when present, falling back
  to `DEFAULT_PID_FILE` for events that predate the field.
- supervisor.ts resolves `pid_file` to an absolute path at emit time (the
  only cwd context in which a relative `--pid-file` is meaningful), so a
  later reader running from a different cwd doesn't misresolve it. The
  process's own internal pidfile guard/read/write paths are untouched.

The DB lock (`gbrain_cycle_locks`) remains the sole singleton authority per
#1849 — this only corrects which pidfile the diagnostic display reads.

Added test/doctor-supervisor-singleton-pidfile.test.ts covering the fixed
path, the still-mismatching absent-pidfile case, and a source-grep pin for
the compatibility fallback.

* docs(comments): correct pid_file fallback rationale and drop unsupported claims

The 'started' audit event has carried pid_file since the supervisor's
introduction, so the fallback comment no longer claims a predate case;
DEFAULT_PID_FILE is env-overridable (GBRAIN_SUPERVISOR_PID_FILE), so the
comments stop calling it HOME-derived; and the custom --pid-file scenario
is now described as an example rather than asserted as common.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0126D3zLWL5RE3CVxnPANiiU

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 06:02:23 -07:00
Masa e8785c1ab1 fix(dream): scope the patterns phase to the cycle's resolved source (#1586) (#4034)
#1586 threaded the cycle's resolved source through the synthesize phase so
subagent-written pages land in that source's (source_id, slug) rows. The
patterns phase was not updated and kept the pre-#1586 shape: it stamped a
literal 'default' on every collected ref and compared the reverse-write path
against a literal 'default'.

On a per-source cycle that files the page against the wrong source. The row is
created under 'default' (the child had no source_id to scope its put_page
calls) while the reverse-write drops the file into the named source's checkout,
because source_id === 'default' selects the brainDir/<slug>.md branch and
brainDir IS that source's checkout. Row and file then disagree about which
source owns the page — what doctor reports as multi_source_drift.

Applies the same threading synthesize.ts already uses: PatternsPhaseOpts gains
sourceId, cycle.ts passes cycleSourceId at the patterns call site the way it
already does for synthesize, the child carries SubagentHandlerData.source_id,
and reverseWriteRefs takes the cycle source as its native source. Unset stays
'default', so unscoped callers are unchanged.
2026-08-13 05:51:48 -07:00
arisgysel-designandarisgysel-design 638dd0d247 fix(autopilot): reopen targeted remediation slots (#4046) (#4067)
Co-authored-by: arisgysel-design <arisgysel-design@users.noreply.github.com>
2026-08-13 05:44:16 -07:00
Masa 697016f69d fix(bootstrap): Gate 2 checks only the active gh account, not every registered one (#4063)
* fix(bootstrap): Gate 2 checks only the active gh account, not every registered one

createPrivateRepo's Gate 2 ran bare `gh auth status` and treated any
non-zero exit as "not authenticated". That command aggregates every
registered account across every host and exits 1 if even one of them
has auth issues — so a stale, unused, expired account (or one on an
unrelated GitHub Enterprise host) false-blocks `gbrain bootstrap repo`
even while the actual active account works fine.

Gate 2 now scopes the check with `--hostname github.com` (this flow is
already github.com-only end to end: parseGithubOwnerRepo, the
repo-create URL fallback, etc.) and, when the installed `gh` supports
it, `--active` as well (added in cli/cli v2.57.0 — confirmed present at
v2.57.0 and absent at v2.56.0 by diffing status.go across tags on
cli/cli). Support is detected from the `gh --version` output Gate 1
already captures, so an older `gh` falls back to the host-scoped bare
form instead of hard-failing on an unrecognized flag.

Verified `gh auth status --active`'s semantics directly against
cli/cli's pkg/cmd/auth/status/status.go: passing --active skips the
per-host loop over non-active accounts entirely (`if opts.Active {
continue }`), so only the active account's entry can affect the exit
code.

* fix(bootstrap): regenerate flag registry for the new gh --active/--hostname literals

The prior commit's Gate 2 change added the string literals `--active` and
`--hostname` inside src/core/bootstrap/repo.ts (both in the gh argv and in
comments). src/commands/bootstrap.ts statically imports repo.ts, and
scripts/generate-flag-registry.ts scans one level of relative imports from
each CLI_ONLY case block to build the committed, freshness-pinned
src/core/cli-flag-registry.generated.ts (#2185) — so the 'bootstrap' entry
was stale relative to a fresh `bun run build:flag-registry` run, failing
test/cli-flag-validation.test.ts's freshness guard in CI.

Regenerated via `bun run build:flag-registry`; the only change is 'bootstrap'
gaining '--active' and '--hostname' alongside its existing ~70 entries. This
is the generator's documented, deliberately over-inclusive behavior (accepting
an unused flag is the pre-#2185 status quo) — neither flag is
security-sensitive or read from user input; both are hardcoded in the `gh`
subprocess invocation, not accepted from `gbrain bootstrap`'s own CLI args.

Confirmed via `git fetch upstream && git log HEAD..upstream/master --oneline`
(0 commits) that this is not upstream drift — purely caused by this PR's own
diff.
2026-08-13 05:12:01 -07:00
Ziyang Guo 758a2d4293 fix(takes): emit JSON for page extraction (#4004)
Fixes #3962

Return the structured extraction result for --json callers while preserving the existing human summary. Add a behavior-level regression test that proves stdout is parseable JSON.
2026-08-13 05:01:29 -07:00
MasaandClaude Opus 5 52389dbe5b fix(conversation-parser): add markdown-heading turn pattern (## User / ## Assistant) (#4005)
* fix(facts): make transcript pages facts-extraction eligible

`gbrain extract-conversation-facts`'s ALLOWED_TYPES allowlist omitted the
`transcript` page type, so gbrain's own nightly transcript-ingest pages
were silently skipped by both the CLI `--types` validation and the
`cycle.conversation_facts_backfill.types` config filter. Even with the
type allowed, the built-in conversation-parser had no pattern for the
`## User` / `## Assistant` markdown-heading turn shape that transcript
ingest writes into `compiled_truth`, so parsing would still yield 0
segments.

This PR makes an explicit decision: transcript pages ARE now
facts-extraction eligible. That is a real behavioral change (a new,
potentially large corpus starts flowing through the extraction +
segment-cost path), not a no-op bugfix — flagging it plainly rather than
padding out the change as narrower than it is.

Changes:
- `src/commands/extract-conversation-facts.ts`: add `'transcript'` to
  `ALLOWED_TYPES` / `ALLOWED_TYPE_ALIASES` (the single source of truth
  for this allowlist).
- `src/core/conversation-parser/builtins.ts`: add the `markdown-heading-turn`
  builtin pattern recognizing heading-only `## User` / `## Assistant` /
  `## Human` / `## System` lines as turn openers, with D5 continuation-line
  body absorption. `quick_reject` is deliberately scoped to the role-prefix
  (not a bare `#{2,3}` heading check) so a message body that happens to
  paste unrelated markdown headings doesn't starve the D18 scorer's
  anchor-candidate ratio.
- `src/commands/jobs.ts`, `src/commands/doctor.ts` (x2 checks),
  `src/commands/sources.ts`: these each carried their own hand-copied
  literal of the same allowed-types list (background-job type filter,
  `conversation_facts_backlog` doctor check, `conversation_format_coverage`
  doctor check, `facts_backfill_estimate`). Switched each to import
  `ALLOWED_TYPES` from the command module instead of re-listing it, so this
  class of drift (a type added in one place, silently excluded everywhere
  else) can't recur.
- `docs/architecture/KEY_FILES.md`: updated the two stale mentions (pattern
  count 17→18, allowlist list) to current-state per this repo's own
  reference-doc convention.

Known limitation (not fixed here, scope-bounded intentionally): parsing is
context-free, same as every other multi-line builtin in this registry — a
message body that contains a literal `## User` line (e.g. someone pasting
a markdown transcript excerpt into their own message) would be read as a
turn boundary. This is a pre-existing property of the whole parser
(`applyPattern`'s per-line scan has no fence-awareness), not something
this PR introduces or could fix without a much larger, separate change to
the shared orchestrator affecting all 18 patterns. Flagging it here rather
than silently shipping the same limitation as the other 17 builtins.

Tests: 4 new tests (2 in test/extract-conversation-facts.test.ts, 2 in
test/conversation-parser/parse.test.ts) covering the allowlist, the new
pattern's positive match + continuation absorption, and that ordinary
`## Summary`-style headings are correctly rejected. Full targeted suite
(conversation-parser + facts-extraction + doctor backlog + build-llms
freshness): 263 pass / 0 fail. typecheck clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0126D3zLWL5RE3CVxnPANiiU

* fix(facts): read the type allowlist from core, not the command module

CI caught this: the known-flags registry drifted for doctor, sources, and
repos. The obvious remedy the guard prints -- regenerate and commit -- would
have been a regression, so this takes the other route.

The generator walks one level of a command module's relative imports and
harvests every flag-shaped string it finds, help text included, and is
deliberately over-inclusive. Importing extract-conversation-facts.ts just to
read ALLOWED_TYPES therefore spliced that command's entire flag vocabulary
(--types, --sleep, --slug, --segment-limit, --override-disabled, ...) into
the allowlists of three commands that implement none of it: `gbrain doctor
--types foo` would have passed validation and been silently ignored. That is
the exact defect class #2185 exists to close.

(jobs.ts is unaffected: it already imported the command module on one line
for runExtractConversationFactsCore, so those flags were already in its
registry entry before this branch.)

ALLOWED_TYPES + ALLOWED_TYPE_ALIASES now live in
src/core/conversation-facts-types.ts, a constants-only module with no CLI
text to harvest. extract-conversation-facts.ts re-exports both so its
existing importers are unchanged.

Verified: registry regenerates to zero drift (was doctor/repos/sources),
cli-flag-validation 24 pass, typecheck clean, 287 pass across the touched
areas. Confirmed against a clean upstream/master worktree that the drift was
introduced by this branch and is not pre-existing.

* fix(conversation-parser): reduce to the parser pattern only

Withdraws the `transcript` allowlist half of this branch. The premise was
wrong: `transcript` is not an upstream page type. `ALL_PAGE_TYPES` does not
contain it, `gbrain-base.yaml` declares `conversation` for "long-running
chat/transcript pages" and marks it `extractable: true` precisely so
extract-conversation-facts walks it, and `gbrain-base-v2.yaml` lists
`transcript` as an alias of `source` (a media primitive). Pages typed
`transcript` are a convention of my own ingest pipeline, not something
upstream produces — the fix for that belongs on my side, by emitting
`conversation`.

That takes the four call-site de-duplications with it (they existed only to
keep the allowlist in sync), and with them the flag-registry drift: no
imports are added, so the registry regenerates to zero drift with no
constants module needed.

What remains is the half that stands on its own: a `conversation` page whose
body uses `## User` / `## Assistant` headings matches none of the 17 builtins
and parses to 0 segments. `markdown-heading-turn` is an 18th pattern in the
same shape as the iMessage/Circleback additions before it.

Verified: typecheck clean, 181 pass / 0 fail across the parser, extraction,
flag-registry and llms-freshness suites, registry drift zero.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 04:51:32 -07:00
Sean GearinandClaude Fable 5 fb141969f5 fix(skills): company-brainify — close three sanitization-defeating defects (#4048)
* fix(skills): company-brainify — close three sanitization-defeating defects

1. Scope truncation: the first structural grep used '>' and overwrote the
   retrieval-discovered scope list; Phase 1 now writes retrieval paths to
   /tmp/brainify-scope.txt explicitly and both greps append.
2. Facts reconciliation: the skill claimed 'gbrain sync' makes the DB stop
   serving deleted Facts-fence rows. Sync's convergence contract covers page
   import only — fact extraction is explicitly decoupled (src/commands/sync.ts
   CONVERGENCE CONTRACT); the reconcile lives in the extract-facts sweep
   (src/core/cycle/extract-facts.ts). The procedure now triggers the sweep and
   verifies removal with 'gbrain recall --grep' before certifying.
3. Backup retention glob: the backup is created as
   shared-brain-history-backup-<ts>.git but cleanup documented
   brain-history-backup-<date>.git — a pattern that matches nothing, silently
   retaining the pre-sanitization history mirror forever. Globs now agree.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(skills): regenerate skills.lock.json after master fix-wave

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 04:37:15 -07:00
Sean GearinandClaude Fable 5 fa5ec8399f fix(skills): citation-graph-ingest — check-backlinks requires a subcommand (#4049)
* fix(skills): citation-graph-ingest — check-backlinks requires a subcommand

Bare 'gbrain check-backlinks' exits with a usage error; the CLI requires
'check' or 'fix' (src/commands/backlinks.ts runBacklinks). The hygiene step
now invokes 'check-backlinks check', matching every other invocation in the
skill pack.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(skills): regenerate skills.lock.json after master fix-wave

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 04:27:44 -07:00
paul-0320andClaude Fable 5 3ce296e315 test(sweep): reset console-prefix's stdout redirect in the shared afterEach — the ENG-5 serve harness leaks it across files (#4061)
runServe()'s stdio path flips console-prefix's module-global
stdout→stderr redirect (#3844). bun test runs every file in one
process, so after sweep.test.ts's serve-wiring tests the flag stays
on and any later file pinning slog's stdout routing fails
(test/sync-all-parallel.test.ts, test/console-prefix.test.ts) —
shard-composition dependent, so it surfaces as a flake. Same reset
the donor harness (test/serve-stdio-lifecycle.test.ts) already
carries.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 04:13:40 -07:00
121 changed files with 6800 additions and 459 deletions
+10 -4
View File
@@ -1,4 +1,4 @@
<!-- gbrain-runbook-stamp: 0.45.10.0 -->
<!-- gbrain-runbook-stamp: 0.45.11.0 -->
<!-- This stamp must equal the VERSION file at every release; CI enforces it
(scripts/check-bootstrap-tag.sh). `gbrain bootstrap status` compares it to
the installed binary and warns on skew. -->
@@ -112,9 +112,15 @@ you needed; report the count at the end (it feeds the install-time measurement).
4. **Render.** `gbrain bootstrap render` — identity files appear. Show the human
SOUL.md. Existing files are never overwritten (re-runs are safe; `--force`
backs up first).
5. **Skills + brain wiring.** The CLI scaffolds the skill set and registers
`brain/` as the workspace source. Nothing to judge here; relay the output.
6. **Wire the harness.** `gbrain bootstrap hooks --harness <detected>`:
5. **Skills.** `gbrain skillpack scaffold --all` — the CLI scaffolds the skill
set. Nothing to judge here; relay the output.
6. **Wire the harness + register the brain source.** `gbrain bootstrap hooks
--harness <detected>` creates `<workspace>/brain` and prints the exact
`gbrain sources add <source_id> --path <brain> --force` command for THIS
workspace — run it verbatim (don't guess a different id; a guessed id
only surfaces as an FK error at `verify` time, by which point a wrong
guess also blocks the correct id with an `overlapping_path` error). It
also:
- Claude Code: installs per-turn hooks ON by default — do NOT ask; loading the
brain every turn is the whole point of installing gbrain for your agent. Tell
the human it is on and how to turn it off (`GBRAIN_HOOKS=0`, or re-run with
+74 -3
View File
@@ -2,13 +2,13 @@
All notable changes to GBrain will be documented in this file.
## [0.45.10.0] - 2026-08-12
## [0.45.11.0] - 2026-08-12
**The install now ends by telling you the two things that matter: you own the brain, and here's the first skill to run.** A working install used to finish on a health report and three tour prompts — technically complete, but a new user walked away without the two facts that make gbrain worth trusting and worth using. Now `gbrain bootstrap verify` ends with a hand-off: **what you own** (every memory is a markdown file in YOUR private GitHub repo — read it, take it to a second machine, delete it and the brain is gone; or the local-only variant with the one command that gives it a durable home) and **what to do next** (run the `cold-start` skill — say "fill my brain" and your agent imports your Gmail, calendar, and contacts through ClawVisor, an OAuth vault so the agent never holds raw tokens, or offline archives like Google Takeout, one consented phase at a time).
The structural fix underneath: `cold-start` — the skill designed exactly for "I just installed this, now what?" — was excluded from the downstream skill bundle, so the paste-in install audience it was written for could never scaffold it. It's now bundled, it's the #1 recommended skill (ahead of the book-mirror flagship, because every flagship skill only becomes magical once the brain holds your real life), and a new drift guard fails CI if any recommended skill ever becomes unscaffoldable again.
To take advantage of v0.45.10.0: existing installs can run `gbrain skillpack scaffold cold-start` and say "fill my brain"; fresh installs get the full hand-off automatically.
To take advantage of v0.45.11.0: existing installs can run `gbrain skillpack scaffold cold-start` and say "fill my brain"; fresh installs get the full hand-off automatically.
### Added
- **The verify hand-off block.** On PASS, `gbrain bootstrap verify` prints (and returns in `--json` as `handoff`) the ownership statement — with the actual repo URL, or the local-only variant pointing at `gbrain bootstrap repo` — followed by the cold-start next action. The runbook's Hand off section now instructs the installing agent to make both land ("say them plainly, confirm they landed") and to OFFER running cold-start on the spot.
@@ -24,7 +24,7 @@ To take advantage of v0.45.10.0: existing installs can run `gbrain skillpack sca
Under the hood: the upgrade nudge now compares the version you're actually running (a stale or foreign cache can't tell you to upgrade to something you already have), a broken settings file makes the installer stop and tell you rather than quietly replace it, and `gbrain init --supabase` fails loudly in a script instead of pretending it worked. Every fix landed with a test, and a two-model adversarial review pass (Claude + Codex) caught a cluster of follow-on issues in the fixes themselves — a keyless upgrade command that pointed at a rejected path, a compiled-binary detection that broke for renamed binaries — which are fixed here too.
To take advantage of v0.45.10.0: nothing to do — `gbrain self-upgrade` (or your next `gbrain` invocation's upgrade nudge) brings you current, and the improvements are all in the install/first-run path a new brain hits automatically.
To take advantage of v0.45.11.0: nothing to do — `gbrain self-upgrade` (or your next `gbrain` invocation's upgrade nudge) brings you current, and the improvements are all in the install/first-run path a new brain hits automatically.
### Added
- **A real-PTY DX exploration harness** (`test/helpers/tty-harness.ts` + `scripts/dx-explore.ts`). It spawns any CLI — gbrain, `claude`, `codex` — under a true pseudo-terminal, timestamps every output burst, and turns silence windows into a measurable stall report, so "the user stared at a frozen screen for nine seconds" is an artifact, not a hunch. A `drive` mode lets an agent steer a live TUI across separate tool calls. Developer instrument only; transcripts are gitignored and nothing in the shipped product depends on it.
@@ -43,6 +43,77 @@ To take advantage of v0.45.10.0: nothing to do — `gbrain self-upgrade` (or you
- **`gbrain bootstrap hooks` with a missing harness CLI** now still installs per-turn hooks and reports the phase as partial (so a resuming agent re-runs it once the CLI is on PATH) instead of leaving a false "wire complete".
- **`gbrain bootstrap interview --set/--skip` after a confirmation** warns that it voided the read-back instead of failing silently later at render.
- Review-pass self-fixes: the keyless upgrade hint now names the re-init command that actually works (not the schema-sizing field `config set` rejects); compiled-binary detection for the detached update refresh no longer breaks for a renamed/official-named binary; the DX harness scrubs copied credentials even on interrupt and reaps the child's whole process tree.
## [0.45.10.0] - 2026-08-13
**21 more community and maintainer bug fixes. Search answers get more complete, sync gets safer, and doctor learns to warn you before a provider dies.**
This wave continues the v0.45.8.0 cleanup: no new product surface, just fixes. The
standouts: pages created by the idea-extraction cycle were invisible to search (they
were written without search chunks) and now show up like everything else, with a repair
path for existing brains. Query caching now keys on your detail setting, so a compact
answer is never served to a full-detail request. And doctor now warns you loudly if your
brain is pinned to an embedding provider that has announced a shutdown, weeks before it
happens instead of after.
Also riding: the rerank budget fix that landed directly this week. Contributed by @javieraldape.
## To take advantage of v0.45.10.0
`gbrain upgrade` is enough. No schema migration.
1. **Upgrade and check:**
```bash
gbrain upgrade
gbrain doctor
```
2. **If doctor now warns about your embedding provider,** that is the new sunset check
doing its job. It names the provider, the date, and the migration command.
3. **Heal previously-invisible atom pages:**
```bash
gbrain embed --stale
```
4. **Things to watch:** the query cache key version moved, so the first re-ask of a
cached question is a one-time cache miss. If anything else looks wrong, file an issue
with `gbrain doctor` output: https://github.com/garrytan/gbrain/issues
### Itemized changes
**Search and recall**
- Atom pages produced by the extraction cycle are chunked and embedded like every other page, so they appear in search results. Contributed by @awilhite.
- `embed --stale` detects and heals pages that have content but no chunks. Contributed by @Masashi-Ono0611.
- The query cache folds the detail knob into its key, so compact and full-detail answers never cross. Contributed by @time-attack.
- Rerank budget failures are bucketed under their real cause instead of "unknown". Contributed by @javieraldape.
**Sync, import, and write-through**
- Deferred link extraction above the size gate is consumed instead of dropped. Contributed by @time-attack.
- Import error summaries name the failing table and constraint. Contributed by @bo-developing.
- Write-through honors the page's recorded source path instead of recomputing it. Contributed by @JonMcCutchen.
- The managed filing-rules block renders each repo's own taxonomy, not the bundled default. Contributed by @dovstern.
- Timeline extraction no longer splits on bare hyphens inside link labels. Contributed by @time-attack.
- Export scopes tag and raw-data sidecar reads to the page's source. Contributed by @alexey-metaengage.
- Cross-source link targets survive an engine migration. Contributed by @RerankerGuo.
**Doctor and diagnostics**
- A damaged PGLite store is reported as store damage, with runtime problems kept separate, and the verdict requires positive evidence. Contributed by @time-attack.
- New check: brains pinned to an embedding provider with an announced shutdown get a loud warning with the migration path. Contributed by @time-attack.
- Source listing distinguishes unset federation from explicit false. Contributed by @dovstern.
- `put_page` reports push state honestly instead of implying success. Contributed by @dovstern.
- Flow-style skill triggers parse correctly in skill health checks. Contributed by @RerankerGuo.
- Sync-failure records auto-skipped as chronic stay visible to doctor until a human resolves them. Contributed by @RerankerGuo.
**Autopilot and agents**
- The drain worker no longer self-deadlocks at concurrency=1, and its DB reconnect logic is shared with queue operations. Contributed by @time-attack.
- Stale-lock reaping ignores foreign PIDs it did not create. Contributed by @javieraldape.
- Agent jobs resolve their brain source at submit time, not execution time. Contributed by @Masashi-Ono0611.
**OAuth**
- Dynamic client registration accepts `token_ttl_seconds`, clamped to admin policy, and an unset TTL cap now derives from `--token-ttl` instead of a permissive default. Contributed by @time-attack.
**Models**
- The claude-cli recipe lists the Claude 5 family ids the CLI already serves, with pins. Contributed by @clement0909472.
**For contributors**
- The CLI flag registry, one wave rider test, and the bootstrap version stamps were refreshed as part of assembly.
## [0.45.9.0] - 2026-08-12
+1 -1
View File
@@ -506,7 +506,7 @@ four numeric segments are required first. Historical 3-segment versions
| `CHANGELOG.md` | Top entry header `## [0.31.4.1] - YYYY-MM-DD` plus the "To take advantage of v0.31.4.1" block. | Standard Keep-a-Changelog header. |
| `TODOS.md` | Any TODO entries that mention "follow-up from vX.Y.Z.W" use the version of the release that filed them. Update only when filing NEW follow-up TODOs. | Inline `vX.Y.Z.W` references in TODO bodies. |
| `CLAUDE.md` | The Key Files section's per-file annotations carry `vX.Y.Z.W (#NNN)` tags noting which release introduced a behavior. Update whenever a wave's annotations get folded in. | Inline `vX.Y.Z.W (#NNN, contributed by @user)` references. |
| `openclaw.plugin.json` | OpenClaw plugin manifest (v0.45.6.0, #4033). Hand-maintained; `test/openclaw-plugin-manifest.test.ts` fails the suite if it drifts from `package.json`. Merges from master auto-resolve it to master's version — re-bump it with the trio. | `"version": "0.45.8.0"` |
| `openclaw.plugin.json` | OpenClaw plugin manifest (v0.45.6.0, #4033). Hand-maintained; `test/openclaw-plugin-manifest.test.ts` fails the suite if it drifts from `package.json`. Merges from master auto-resolve it to master's version — re-bump it with the trio. | `"version": "0.45.10.0"` |
| `BOOTSTRAP_FOR_AGENTS.md` | Runbook stamp on line 1. `scripts/check-bootstrap-tag.sh` (in `bun run verify` + CI) fails when it drifts from `VERSION`; refresh it in the same commit as the bump. | `<!-- gbrain-runbook-stamp: X.Y.Z.W -->` |
| `templates/bootstrap/template-repo/` | Vendored template tree with an embedded version stamp. Auto-derived, but NOT by `bun install`: run `bun run scripts/generate-template-repo.ts --out templates/bootstrap/template-repo` after the bump; `scripts/check-bootstrap-templates.sh` fails CI on drift. | `<!-- gbrain-template-stamp: X.Y.Z.W -->` in generated files. |
+1 -1
View File
@@ -1 +1 @@
0.45.10.0
0.45.11.0
+1
View File
@@ -176,6 +176,7 @@ Unit tests and what they cover:
- `test/watch-command.test.ts``gbrain watch` push transport (#2095): streaming loop, rolling window, session dedupe, `--json` JSONL shape, `channel: 'watch'` event logging, clean EOF return. Hermetic PGLite + injected line/write deps (no subprocess, no real stdin).
- `test/watch-sigint.serial.test.ts``gbrain watch` SIGINT lifecycle against a real spawned CLI subprocess with a tmpdir brain. SERIAL: parallel unit shards flake on concurrent subprocess spawns (same rationale as `apply-migrations-pglite-spawn.serial.test.ts`).
- `test/autopilot-launchd-lifecycle.serial.test.ts` — autopilot lifecycle behavior, not generated-string assertions: the full install → self-disable → status → reinstall → uninstall arc with `launchctl` replaced by an argv recorder and the generated wrapper executed by a REAL bash against a genuinely deleted repo (every platform), plus a darwin-only fail-SKIP describe against the real launchd under a per-run unique label (`GBRAIN_AUTOPILOT_LABEL`) so it can never collide with — or tear down — a real install on the host. Serial: spawns subprocesses and pins HOME/GBRAIN_HOME for the whole file.
- `test/autopilot-fanout.test.ts` — Autopilot fan-out and #4046 policy regression: targeted idempotency keys reopen per dispatch interval while stable doctor/remediate keys remain unchanged; the 60-minute full-cycle floor wins with a remaining small plan, and an all-fresh restart check advances the process-local clock without masking failed stale-source submissions.
- `test/agent-scheduler-contract.serial.test.ts` — the documented external agent-scheduler shell chain (`gbrain sync --repo X && gbrain embed --stale`, live-sync.md / INSTALL_FOR_AGENTS.md Step 7) driven end-to-end through a real `/bin/sh` against a keyless PGLite brain: the `&&` short-circuit IS the contract (argv arrays can't exercise it), the keyless bare stale embed exits 0, and the pull-failure case that must break the chain does. Anti-vacuity: the fixture commits a real page and every read-back asserts pages >= 1. Serial: real spawned CLI + tmpdir HOME.
- `test/cli-format-volunteer.test.ts``formatResult`'s `volunteer_context` human rendering: pointer lines with confidence/arm/rationale, the empty-result message, the approximate stats summary.
- `test/config.test.ts` — config redaction.
+18
View File
@@ -1,5 +1,23 @@
# ZeroEntropy — zembed-1 + zerank-2
> **Hosted API shutdown: 2026-09-04.** ZeroEntropy announced (2026-07-24)
> that its hosted endpoints — `/models/embed` and `/models/rerank` — shut
> down on that date. A brain still embedding through the hosted API loses
> semantic retrieval entirely on that date: query embedding uses the same
> endpoint, so **existing vectors become unqueryable**, not just new
> content. Two fixes, either works:
>
> 1. **Self-host the same model** — zembed-1 weights are Apache-2.0. Serve
> them via `llama-server` or Ollama and point the config at the local
> endpoint. Keeps every existing vector; no re-embed at all.
> 2. **Migrate to another provider** — `gbrain migrate embeddings --to
> <provider:model> --dim <N> --dry-run` (resumable; see
> [the migration guide](../guides/embedding-migration.md)). `gbrain
> doctor` (check `provider_sunset`) prints this command with your
> brain's actual `--dim` filled in.
>
> The hosted setup below remains accurate until the shutdown date.
[ZeroEntropy](https://zeroentropy.dev) ships two specialized small models
for retrieval pipelines:
File diff suppressed because one or more lines are too long
+37
View File
@@ -27,6 +27,37 @@ gbrain migrate embeddings --to voyage:voyage-3-large --yes
declared width and is required for recipes that don't declare one (litellm,
llama-server, and other bring-your-own-model providers).
**Pick `--dim` = your brain's current column width when the target supports
it.** A different width triggers the destructive schema transition (column +
index rebuild across all three dim-pinned tables); the same width skips it
entirely. `gbrain doctor` (check `provider_sunset`, for providers with an
announced shutdown) prints the paste-ready command with your actual width
already filled in — it reads the real `vector(N)` column, not the config
value, which can drift.
## How affected brains find out (provider sunsets)
Two surfaces flag a brain whose embedding model (or reranker) is on a
provider with an announced hosted-API shutdown, such as ZeroEntropy
(2026-09-04):
- **`gbrain doctor`** — the `provider_sunset` check warns on every run until
the brain is off the provider. After the shutdown date it escalates to
`fail` only when embedded vectors actually exist on the dead provider
(retrieval is genuinely down); a zero-vector brain whose config merely
resolves to the dead default stays `warn`, so doctor-as-CI-gate setups
don't start exiting 1 on the date. The reranker side resolves through the
same plane search actually reranks with (the mode bundle +
`search.reranker.*` overrides). The message carries the paste-ready
migration command with the brain's actual `--dim`. Accepted the risk?
`gbrain config set doctor.suppress_provider_sunset true` silences it.
- **`gbrain upgrade`** — a one-shot banner (gated by
`ze_sunset_notice_shown`) with the same two fixes.
Both state the full consequence: after the shutdown, **existing vectors
become unqueryable** — query embedding uses the same endpoint as ingestion —
not just new content.
## What it does, in order
1. **Plan.** Counts every chunk not already in the target embedding space —
@@ -88,6 +119,12 @@ continues where it stopped. An in-flight marker (`embedding_migration.state`
in DB config) records the target; it is cleared only when the backlog drains
to zero.
One caveat after a HARD kill (SIGKILL, crash, power loss — not Ctrl-C): the
run's per-source single-flight embed lock is left behind, and an immediate
re-run skips the re-embed and reports the migration as paused. The command
says so explicitly (`lock_skipped` in `--json`); the lock expires on its own
after at most 60 minutes, then the same re-run resumes normally.
A page whose chunks straddle two stale batches is embedded correctly but not
stamped by the embed loop (which only stamps all-or-nothing per batch), so the
migration runs one reconcile pass after the drain that stamps every
+3 -1
View File
@@ -23,7 +23,7 @@ The resolved provider + dimensions get persisted to `~/.gbrain/config.json` atom
| Provider | env vars | default dims | cost ($/1M tokens) | local? | multimodal? |
|---|---|---|---|---|---|
| `zeroentropyai` | `ZEROENTROPY_API_KEY` | 2560 (Matryoshka to 1280/640/320/...) | 0.05 | no | no |
| `zeroentropyai` (hosted API **shuts down 2026-09-04** — see note below) | `ZEROENTROPY_API_KEY` | 2560 (Matryoshka to 1280/640/320/...) | 0.05 | no | no |
| `openai` | `OPENAI_API_KEY` | 1536 | 0.13 | no | no |
| `openrouter` | `OPENROUTER_API_KEY` | 1536 | 0.02 | no | model-dependent |
| `voyage` | `VOYAGE_API_KEY` | 1024 | 0.18 | no | yes (`voyage-multimodal-3`) |
@@ -42,6 +42,8 @@ The resolved provider + dimensions get persisted to `~/.gbrain/config.json` atom
**Note on local providers.** Ollama and llama-server have no required API key, so they don't show up in env-detection auto-pick. Pick them explicitly with `--embedding-model ollama:<model>` to avoid silently routing to a daemon that may not be running.
**Note on the ZeroEntropy hosted API.** ZeroEntropy announced (2026-07-24) that its hosted endpoints shut down on **2026-09-04**. A brain still embedding through the hosted API loses semantic retrieval entirely on that date — query embedding uses the same endpoint, so existing vectors become unqueryable, not just new content. Either self-host the Apache-2.0 zembed-1 weights via llama-server/Ollama (keeps every existing vector, no re-embed), or migrate with `gbrain migrate embeddings` — see [the migration guide](../guides/embedding-migration.md). `gbrain doctor` (check `provider_sunset`) flags affected brains and prints the paste-ready command with the brain's actual `--dim` filled in.
## If first import fails
If `gbrain import` fails with `expected N dimensions, not M`, run `gbrain doctor`. The output will print the exact `gbrain config set ...` or `gbrain retrieval-upgrade` command to repair the mismatch. **You should not need to delete `~/.gbrain`.** The bug-class that historically forced `rm -rf` recoveries is closed as of v0.37.
+16
View File
@@ -163,6 +163,22 @@ await oauthProvider.registerClientManual(
For self-service client registration (Dynamic Client Registration, RFC 7591),
start the server with `--enable-dcr`. DCR is off by default.
DCR requests may include an optional `token_ttl_seconds` field (integer,
seconds) to request a per-client access-token lifetime. The server clamps the
request into an admin-configured window — never rejects over it — persists the
effective value as the client's TTL override, and echoes it back as
`token_ttl_seconds` in the registration response. Subsequent `/token` responses
for that client carry the matching `expires_in`. Clients that omit the field
keep the server default (`--token-ttl`). The window defaults fail-closed: min
300 seconds, max bounded by your `--token-ttl` — a self-registering client
cannot request a longer-lived token than the server default unless you
explicitly widen the window:
```bash
gbrain config set oauth.dcr_ttl_min_seconds 600
gbrain config set oauth.dcr_ttl_max_seconds 86400
```
### 3. Expose the server
**Bind explicitly.** `gbrain serve --http` defaults to `127.0.0.1`.
+4
View File
@@ -135,6 +135,10 @@ sync that calls import emits `sync.import.<file>`, not `import.<file>`.
Stable phase names shipped in v0.15.2:
- `doctor.db_checks` (umbrella for all DB-side doctor checks)
- `doctor.pglite_probe` (the #2674 scratch-store probe; only when PGLite init
failed with an unexplained/damage-class disk state or `--probe-pglite` was
passed — a cold start can take 520s, so the heartbeat is the only sign of
life)
- `orphans.scan`
- `embed.pages`
- `extract.links_fs`, `extract.timeline_fs`, `extract.links_db`, `extract.timeline_db`
+17 -1
View File
@@ -661,7 +661,7 @@ four numeric segments are required first. Historical 3-segment versions
| `CHANGELOG.md` | Top entry header `## [0.31.4.1] - YYYY-MM-DD` plus the "To take advantage of v0.31.4.1" block. | Standard Keep-a-Changelog header. |
| `TODOS.md` | Any TODO entries that mention "follow-up from vX.Y.Z.W" use the version of the release that filed them. Update only when filing NEW follow-up TODOs. | Inline `vX.Y.Z.W` references in TODO bodies. |
| `CLAUDE.md` | The Key Files section's per-file annotations carry `vX.Y.Z.W (#NNN)` tags noting which release introduced a behavior. Update whenever a wave's annotations get folded in. | Inline `vX.Y.Z.W (#NNN, contributed by @user)` references. |
| `openclaw.plugin.json` | OpenClaw plugin manifest (v0.45.6.0, #4033). Hand-maintained; `test/openclaw-plugin-manifest.test.ts` fails the suite if it drifts from `package.json`. Merges from master auto-resolve it to master's version — re-bump it with the trio. | `"version": "0.45.8.0"` |
| `openclaw.plugin.json` | OpenClaw plugin manifest (v0.45.6.0, #4033). Hand-maintained; `test/openclaw-plugin-manifest.test.ts` fails the suite if it drifts from `package.json`. Merges from master auto-resolve it to master's version — re-bump it with the trio. | `"version": "0.45.10.0"` |
| `BOOTSTRAP_FOR_AGENTS.md` | Runbook stamp on line 1. `scripts/check-bootstrap-tag.sh` (in `bun run verify` + CI) fails when it drifts from `VERSION`; refresh it in the same commit as the bump. | `<!-- gbrain-runbook-stamp: X.Y.Z.W -->` |
| `templates/bootstrap/template-repo/` | Vendored template tree with an embedded version stamp. Auto-derived, but NOT by `bun install`: run `bun run scripts/generate-template-repo.ts --out templates/bootstrap/template-repo` after the bump; `scripts/check-bootstrap-templates.sh` fails CI on drift. | `<!-- gbrain-template-stamp: X.Y.Z.W -->` in generated files. |
@@ -4128,6 +4128,22 @@ await oauthProvider.registerClientManual(
For self-service client registration (Dynamic Client Registration, RFC 7591),
start the server with `--enable-dcr`. DCR is off by default.
DCR requests may include an optional `token_ttl_seconds` field (integer,
seconds) to request a per-client access-token lifetime. The server clamps the
request into an admin-configured window — never rejects over it — persists the
effective value as the client's TTL override, and echoes it back as
`token_ttl_seconds` in the registration response. Subsequent `/token` responses
for that client carry the matching `expires_in`. Clients that omit the field
keep the server default (`--token-ttl`). The window defaults fail-closed: min
300 seconds, max bounded by your `--token-ttl` — a self-registering client
cannot request a longer-lived token than the server default unless you
explicitly widen the window:
```bash
gbrain config set oauth.dcr_ttl_min_seconds 600
gbrain config set oauth.dcr_ttl_max_seconds 86400
```
### 3. Expose the server
**Bind explicitly.** `gbrain serve --http` defaults to `127.0.0.1`.
+1 -1
View File
@@ -1,7 +1,7 @@
{
"id": "gbrain-context-engine",
"name": "gbrain",
"version": "0.45.10.0",
"version": "0.45.11.0",
"description": "Personal knowledge brain with Postgres + pgvector hybrid search",
"family": "bundle-plugin",
"configSchema": {
+1 -1
View File
@@ -154,7 +154,7 @@
"bun": ">=1.3.10"
},
"license": "MIT",
"version": "0.45.10.0",
"version": "0.45.11.0",
"overrides": {
"@hono/node-server": "^2.0.5",
"fast-uri": "^3.1.5",
+1 -1
View File
@@ -159,7 +159,7 @@ mismatch, typo'd `--type`) before reporting anything.
```bash
gbrain link-sources # citation-graph should appear with the expected count
gbrain check-backlinks # confirm no orphaned references
gbrain check-backlinks check # confirm no orphaned references
```
## Run it (worked example, synthetic fixture)
+28 -13
View File
@@ -139,13 +139,17 @@ cd "$BRAIN"
gbrain recall --grep "salary"
```
Collect every returned slug into the scope list.
Resolve every returned slug to its repo-relative file path and write the
paths into `/tmp/brainify-scope.txt` (one per line). This file is the
scope list; the structural pass below APPENDS to it — nothing later in
the procedure may truncate it, or the retrieval-discovered pages
silently drop out of scope.
2. Structural discovery — people files that belong to the company, plus
keyword hits across the wider scan scope:
```bash
grep -rli 'company: *"acme-example"' people/ --include="*.md" | sort > /tmp/brainify-scope.txt
grep -rli 'company: *"acme-example"' people/ --include="*.md" | sort >> /tmp/brainify-scope.txt
grep -rli -E 'salary|equity|carry|retention|underperform|performance review|hard conversation' \
meetings/ daily/ companies/ projects/ analysis/ --include="*.md" 2>/dev/null >> /tmp/brainify-scope.txt
sort -u -o /tmp/brainify-scope.txt /tmp/brainify-scope.txt
@@ -254,16 +258,22 @@ For sanitization, sensitive fact rows must be ACTUALLY REMOVED: find them
(`gbrain recall --grep`), then delete the row from the page's Facts fence
(step 5), exactly like a sensitive take. On an in-place shared brain, the
page edit must then be re-synced (`gbrain sync` re-imports the edited page)
so the shared database no longer serves the row — an edited page over an
un-synced DB still leaks through retrieval. `forget` alone can never certify
a brain clean.
AND the facts index reconciled — sync's convergence contract covers page
import only; downstream fact extraction is explicitly decoupled
(`src/commands/sync.ts`, "CONVERGENCE CONTRACT"), so the DB keeps serving
the deleted row until the extract-facts reconcile runs. Trigger it
(`gbrain sweep`, or wait for the serve-resident sweep), then confirm with
`gbrain recall --grep` that the row is actually gone. An edited page over
an un-reconciled facts index still leaks through retrieval. `forget` alone
can never certify a brain clean.
After edits: on the **staging-copy** path the fact rows are removed by editing
the copied markdown directly (there is no live DB to re-sync yet — the team DB
is built fresh when Phase 5 Step 0 turns the export into a source). On the
**in-place shared-brain** path, `gbrain sync` re-imports the changed pages so
the DB matches the markdown. Either way, run `gbrain check-backlinks check` to
catch pages still pointing at removed content.
**in-place shared-brain** path, run `gbrain sync` so the page content matches
the markdown, then reconcile and verify the facts index as above. Either way,
run `gbrain check-backlinks check` to catch pages still pointing at removed
content.
### Phase 4: Verify
@@ -502,7 +512,10 @@ recovery line.
mirror-clone backup in `~/.gbrain/backups/` for a retention window
(~30 days is a sane default), then delete it — it contains the
pre-sanitization history and should not accumulate indefinitely:
`rm -rf ~/.gbrain/backups/brain-history-backup-<date>.git`
`rm -rf ~/.gbrain/backups/shared-brain-history-backup-<date>.git`
(the glob must match the `shared-brain-history-backup-*` name the backup
step created — a mismatched pattern deletes nothing and silently retains
the pre-sanitization history forever)
- If the repo carries push hooks or auto-hardening wiring, re-verify remotes
and hooks survived the rewrite before handing the repo to the team
@@ -592,9 +605,11 @@ This skill guarantees:
covered by the sanitization scan; everything else is excluded by default,
and the Phase 4 verification greps run against the exported tree before
the first push.
- Sensitive fact rows are deleted from the page's Facts fence and re-synced,
never merely expired — `gbrain forget` retains the row (struck through,
served via `--include-expired`) and can never certify clean.
- Sensitive fact rows are deleted from the page's Facts fence, re-synced,
and the facts index reconciled (extract-facts sweep) with the removal
verified via `gbrain recall --grep`, never merely expired — `gbrain
forget` retains the row (struck through, served via `--include-expired`)
and can never certify clean.
- The history-purge filter list and its restore manifest both derive from
the COMPLETE set of sanitized paths, never a subset.
- Every strip decision is a per-file model judgment grounded in a full read;
@@ -623,7 +638,7 @@ Three artifacts:
- Scope: [N files scanned across people/, meetings/, daily/, ...]
- Flagged: [M files with hits] (triage list attached)
- Edited: [K files sanitized; T takes removed; F fact rows removed + re-synced]
- Edited: [K files sanitized; T takes removed; F fact rows removed + re-synced + facts index reconciled]
- Verification: [grep residuals: 0 confirmed-sensitive; retrieval checks: clean]
- History: [not purged | fresh-export | purged after confirmed gate — backup at <path>]
- Next re-audit: [date / cron slot]
+2 -2
View File
@@ -33,10 +33,10 @@
"capture/SKILL.md": "98568ac96331f57397ea072749641d9748b1ce31e8b09d512b8db25c8fcda65f",
"citation-fixer/SKILL.md": "abdadbf0740a529b9c4f86f05bba416417624503fdcbc6054402d5546afd08b4",
"citation-fixer/routing-eval.jsonl": "52b23b71e66fdc18aee67d0576099b0c83997d648cf4ecf8fe7753b91b6c9c53",
"citation-graph-ingest/SKILL.md": "6510856cc14a653dcade510702890343bc0527de14bc2f1ed0d2524f248c798c",
"citation-graph-ingest/SKILL.md": "849b0cdc64b7ff14d0e6771bde15f0edc3c2fc29af08be015753a5f88a03205f",
"citation-graph-ingest/routing-eval.jsonl": "a1ba605d35e736b741b9e8aac1e7d50b61a7cbcada893d67099b55bf5a0d2635",
"cold-start/SKILL.md": "20be3d1b637621fd9fbd268072f6647533a23f596e30cb593523b051708aaddd",
"company-brainify/SKILL.md": "2c058b39f5364b8ceb5c53b4525cce8645734f16cc3c229a490b005d58a78311",
"company-brainify/SKILL.md": "ae48372512645f532820e43faaf18a8fa768a691b2144973dfc89465f84d84c6",
"company-brainify/routing-eval.jsonl": "6f27f835eda9ae77a2b694534c78a043a871349820e8c638c3d8bbba6d3aa17b",
"concept-synthesis/SKILL.md": "ed02d2e385143b16a1e69ee5934288fb4d0b755f68c4312faff663e6b2d7c4ed",
"concept-synthesis/routing-eval.jsonl": "96dbd7d9c1b606e9e06262d0c06282399741e2bccb8eeb7b9ca88c20f44cda0e",
+1 -1
View File
@@ -3056,7 +3056,7 @@ SETUP
migrate embeddings --to <p:model> Re-embed onto another embedding provider
upgrade Self-update
check-update [--json] Check for new versions
doctor [--json] [--fast] Health check (resolver, skills, pgvector, RLS, embeddings)
doctor [--json] [--fast] [--probe-pglite] Health check (resolver, skills, pgvector, RLS, embeddings; --probe-pglite runs the scratch-store probe)
integrations [subcommand] Manage integration recipes (senses + reflexes)
PAGES
+82 -3
View File
@@ -18,6 +18,8 @@ import type { BrainEngine } from '../core/engine.ts';
import { MinionQueue } from '../core/minions/queue.ts';
import { waitForCompletion, TimeoutError } from '../core/minions/wait-for-completion.ts';
import type { MinionJobInput, SubagentHandlerData, AggregatorHandlerData } from '../core/minions/types.ts';
import { resolveSourceId, ALL_SOURCES } from '../core/source-resolver.ts';
import { fetchSource } from '../core/sources-load.ts';
import { runAgentLogs } from './agent-logs.ts';
// ── arg parsing helpers ────────────────────────────────────
@@ -72,6 +74,10 @@ SUBMITTING
--max-turns <n> Max assistant turns (default 20)
--tools a,b,c Subset of registered tool names (comma list)
--timeout-ms <n> Per-job wall-clock timeout
--source <id> Brain source the subagent's writes are scoped to.
Default: the standard resolution chain (GBRAIN_SOURCE,
.gbrain-source, sources.default, ...) see
\`gbrain sources current\`
--fanout-manifest <path> JSON array of {prompt, input_vars?} one child each
--follow Tail status until terminal (default on TTY)
--detach Submit + print job id, exit immediately
@@ -116,6 +122,7 @@ interface RunFlags {
maxTurns?: number;
tools?: string[];
timeoutMs?: number;
source?: string;
fanoutManifest?: string;
follow: boolean;
detach: boolean;
@@ -181,6 +188,7 @@ function parseRunFlags(args: string[]): { flags: RunFlags; rest: string[] } {
case '--max-turns': flags.maxTurns = parseIntFlagValue(requireFlagValue(args, ++i, a), a); break;
case '--tools': flags.tools = requireFlagValue(args, ++i, a).split(',').map(s => s.trim()).filter(Boolean); break;
case '--timeout-ms': flags.timeoutMs = parseIntFlagValue(requireFlagValue(args, ++i, a), a); break;
case '--source': flags.source = requireFlagValue(args, ++i, a); break;
case '--fanout-manifest': flags.fanoutManifest = requireFlagValue(args, ++i, a); break;
case '--follow': flags.follow = true; break;
case '--no-follow': flags.follow = false; break;
@@ -203,17 +211,86 @@ function parseRunFlags(args: string[]): { flags: RunFlags; rest: string[] } {
return { flags, rest };
}
/**
* Predicate: is this error one of the source resolver's user-facing throws
* we want to surface as a clean stderr line + exit 1? Mirrors
* dream.ts:isResolverUserError anything else (connection failures,
* genuine bugs) propagates with a stack trace.
*/
function isResolverUserError(e: unknown): boolean {
if (!(e instanceof Error)) return false;
const m = e.message;
return (m.startsWith('Source "') && m.includes(' not found.'))
|| m.startsWith('Invalid --source value')
|| m.startsWith('Invalid GBRAIN_SOURCE value');
}
/**
* #2922: resolve the brain source for a subagent submission via the
* canonical chain (explicit --source GBRAIN_SOURCE .gbrain-source
* local_path match sources.default sole non-default 'default').
* Pre-fix, `gbrain agent run` never resolved a source, so every page an
* agent job wrote landed in the seed 'default' source even on brains with
* `gbrain sources default <id>` configured.
*
* The `__all__` sentinel is rejected here: subagent writes must target
* exactly one source (and `validateSourceId` at tool-registry build time
* would reject it anyway better to fail at submit than at claim).
*/
async function resolveAgentSource(engine: BrainEngine, explicit: string | undefined): Promise<string> {
// An empty `--source ""` must fail loudly, not silently degrade to the
// env/dotfile/default tiers (resolveSourceId's `if (explicit)` treats a
// falsy value as omitted — explicit-but-empty would slip through).
if (explicit !== undefined && explicit.trim() === '') {
console.error('gbrain agent run: --source requires a non-empty value. Run `gbrain agent run --help`.');
process.exit(2);
}
let resolved: string;
try {
resolved = await resolveSourceId(engine, explicit ?? null);
} catch (e) {
if (isResolverUserError(e)) {
console.error(`gbrain agent run: ${(e as Error).message}`);
process.exit(1);
}
throw e;
}
if (resolved === ALL_SOURCES) {
console.error(
`gbrain agent run: --source ${ALL_SOURCES} is not supported — ` +
`subagent writes must target exactly one source. Pass a concrete --source <id>.`,
);
process.exit(2);
}
// Archived-source guard, mirroring dream.ts: writing subagent pages into
// an archived (normally invisible) source would mask them until restore.
const src = await fetchSource(engine, resolved);
if (src?.archived === true) {
console.error(
`gbrain agent run: source ${resolved} is archived; restore with ` +
`\`gbrain sources restore ${resolved}\` before submitting agent jobs`,
);
process.exit(1);
}
return resolved;
}
export async function runAgentRun(engine: BrainEngine, args: string[]): Promise<void> {
const { flags, rest } = parseRunFlags(args);
const queue = new MinionQueue(engine);
// #2922: resolve once at submit time; both the single-job and fan-out
// paths stamp it on SubagentHandlerData.source_id so buildOpContext
// scopes every tool call to it instead of the legacy 'default'.
const sourceId = await resolveAgentSource(engine, flags.source);
// Fan-out path: --fanout-manifest supplies explicit child inputs. The
// aggregator submits first (so its id is available as parent for each
// child); children submit with on_child_fail='continue' so mixed
// outcomes don't cascade; aggregator waits in waiting-children until
// Lane 1B's terminal-set check unblocks it.
if (flags.fanoutManifest) {
await runFanout(engine, queue, flags, rest.join(' '));
await runFanout(engine, queue, flags, rest.join(' '), sourceId);
return;
}
@@ -223,7 +300,7 @@ export async function runAgentRun(engine: BrainEngine, args: string[]): Promise<
process.exit(2);
}
const data: SubagentHandlerData = { prompt };
const data: SubagentHandlerData = { prompt, source_id: sourceId };
if (flags.subagentDef) data.subagent_def = flags.subagentDef;
if (flags.model) data.model = flags.model;
if (flags.maxTurns) data.max_turns = flags.maxTurns;
@@ -248,7 +325,7 @@ export async function runAgentRun(engine: BrainEngine, args: string[]): Promise<
// ── fan-out ───────────────────────────────────────────────
async function runFanout(engine: BrainEngine, queue: MinionQueue, flags: RunFlags, promptTemplate: string): Promise<void> {
async function runFanout(engine: BrainEngine, queue: MinionQueue, flags: RunFlags, promptTemplate: string, sourceId: string): Promise<void> {
const manifestPath = flags.fanoutManifest!;
let manifest: Array<{ prompt?: string; input_vars?: Record<string, unknown> }>;
try {
@@ -272,6 +349,7 @@ async function runFanout(engine: BrainEngine, queue: MinionQueue, flags: RunFlag
const entry = manifest[0]!;
const data: SubagentHandlerData = {
prompt: entry.prompt ?? promptTemplate,
source_id: sourceId,
...(entry.input_vars ? { input_vars: entry.input_vars } : {}),
...(flags.subagentDef ? { subagent_def: flags.subagentDef } : {}),
...(flags.model ? { model: flags.model } : {}),
@@ -303,6 +381,7 @@ async function runFanout(engine: BrainEngine, queue: MinionQueue, flags: RunFlag
for (const entry of manifest) {
const data: SubagentHandlerData = {
prompt: entry.prompt ?? promptTemplate,
source_id: sourceId,
...(entry.input_vars ? { input_vars: entry.input_vars } : {}),
...(flags.subagentDef ? { subagent_def: flags.subagentDef } : {}),
...(flags.model ? { model: flags.model } : {}),
+26 -6
View File
@@ -34,8 +34,7 @@ import type { BrainEngine, SourceRow } from '../core/engine.ts';
import type { MinionQueue } from '../core/minions/queue.ts';
import { NON_GLOBAL_PHASES, GLOBAL_PHASES, LAST_GLOBAL_AT_KEY } from '../core/cycle.ts';
import { sourceConfigHasRemoteUrl } from '../core/sources-load.ts';
const FULL_CYCLE_FLOOR_MIN = 60;
import { AUTOPILOT_FULL_CYCLE_FLOOR_MINUTES } from './autopilot-remediation-policy.ts';
// #2194 fix #2: failure cooldown. A source whose autopilot-cycle keeps
// failing/timing-out re-dispatches every tick today (only SUCCESS gates
@@ -81,6 +80,8 @@ export interface FanoutResult {
/** True when this tick fell back to the legacy single-job path
* (no sources rows / engine empty). */
legacy_fallback: boolean;
/** True when every enumerated source is inside the freshness window. */
all_sources_fresh: boolean;
}
/**
@@ -180,7 +181,11 @@ export function readLastFullCycleAt(src: SourceRow): Date | null {
* a brain may have fresh sync but stale extract/embed. The 60-min floor on
* full-cycle is the canonical freshness signal for autopilot dispatch.
*/
export function isSourceStale(src: SourceRow, now = Date.now(), floorMin = FULL_CYCLE_FLOOR_MIN): boolean {
export function isSourceStale(
src: SourceRow,
now = Date.now(),
floorMin = AUTOPILOT_FULL_CYCLE_FLOOR_MINUTES,
): boolean {
const last = readLastFullCycleAt(src);
if (last === null) return true;
const ageMin = (now - last.getTime()) / 60_000;
@@ -328,7 +333,7 @@ export function selectSourcesForDispatch(
sources: SourceRow[],
fanoutMax: number,
now = Date.now(),
floorMin = FULL_CYCLE_FLOOR_MIN,
floorMin = AUTOPILOT_FULL_CYCLE_FLOOR_MINUTES,
recentFailures: Map<string, SourceFailure> = new Map(),
cooldownOpts: CooldownOpts = { baseMin: FAILURE_COOLDOWN_BASE_MIN, capMin: FAILURE_COOLDOWN_CAP_MIN },
): { dispatch: SourceRow[]; skippedFresh: SourceRow[]; skippedCap: SourceRow[]; skippedCooldown: SourceRow[] } {
@@ -406,7 +411,14 @@ export async function dispatchPerSource(
} else {
log(`[dispatch] job #${job.id} autopilot-cycle (legacy single-source)`);
}
return { dispatched: [], skipped_fresh: [], skipped_cap: [], skipped_cooldown: [], legacy_fallback: true };
return {
dispatched: [],
skipped_fresh: [],
skipped_cap: [],
skipped_cooldown: [],
legacy_fallback: true,
all_sources_fresh: false,
};
}
// #2194 fix #2: load recent per-source failures + cooldown knobs so a
@@ -426,7 +438,14 @@ export async function dispatchPerSource(
}
const { dispatch, skippedFresh, skippedCap, skippedCooldown } =
selectSourcesForDispatch(sources, opts.fanoutMax, Date.now(), FULL_CYCLE_FLOOR_MIN, recentFailures, cooldownOpts);
selectSourcesForDispatch(
sources,
opts.fanoutMax,
Date.now(),
AUTOPILOT_FULL_CYCLE_FLOOR_MINUTES,
recentFailures,
cooldownOpts,
);
const dispatched: string[] = [];
for (const src of dispatch) {
@@ -509,6 +528,7 @@ export async function dispatchPerSource(
skipped_cap: skippedCap.map(s => s.id),
skipped_cooldown: skippedCooldown.map(s => s.id),
legacy_fallback: false,
all_sources_fresh: skippedFresh.length === sources.length,
};
}
@@ -0,0 +1,46 @@
export const AUTOPILOT_FULL_CYCLE_FLOOR_MINUTES = 60;
export interface AutopilotRemediationPlanShape {
score: number;
planLength: number;
estimatedSeconds: number;
minutesSinceLastFull: number;
}
/**
* Keep recommendation keys stable for doctor/remediate checkpoints while
* giving Autopilot a fresh single-flight slot on every dispatch interval.
*/
export function autopilotRemediationIdempotencyKey(
recommendationKey: string,
dispatchSlot: string,
): string {
return `${recommendationKey}:autopilot:${dispatchSlot}`;
}
/**
* A full cycle is a freshness invariant, independent of the current score or
* targeted plan. Large/slow/severely degraded plans retain the existing
* hammer behavior before the freshness floor is reached.
*/
export function shouldRunAutopilotFullCycle({
score,
planLength,
estimatedSeconds,
minutesSinceLastFull,
}: AutopilotRemediationPlanShape): boolean {
return minutesSinceLastFull >= AUTOPILOT_FULL_CYCLE_FLOOR_MINUTES
|| planLength > 3
|| estimatedSeconds >= 300
|| score < 70;
}
export function shouldSleepHealthyAutopilot(
score: number,
planLength: number,
minutesSinceLastFull: number,
): boolean {
return score >= 95
&& planLength === 0
&& minutesSinceLastFull < AUTOPILOT_FULL_CYCLE_FLOOR_MINUTES;
}
+53 -23
View File
@@ -25,6 +25,11 @@ import { execSync } from 'child_process';
import type { BrainEngine } from '../core/engine.ts';
import { loadPreferences } from '../core/preferences.ts';
import { loadConfig, loadConfigFileOnly, saveConfig, gbrainPath as gbrainHomePath } from '../core/config.ts';
import {
classifyAutopilotLockHolder,
type AutopilotLockProbeDeps,
isPidAlive,
} from '../core/autopilot-lock.ts';
import { ChildWorkerSupervisor } from '../core/minions/child-worker-supervisor.ts';
import { VERSION } from '../version.ts';
import {
@@ -41,6 +46,11 @@ import { evaluateQuietHours } from '../core/minions/quiet-hours.ts';
import { inspectLock } from '../core/db-lock.ts';
import { registerCleanup } from '../core/process-cleanup.ts';
import { resolveAutopilotDispatchTimeoutMs } from './autopilot-timeout.ts';
import {
autopilotRemediationIdempotencyKey,
shouldRunAutopilotFullCycle,
shouldSleepHealthyAutopilot,
} from './autopilot-remediation-policy.ts';
// Path helpers live in a LEAF core module so other commands (gbrain migrate)
// can read the daemon's state files without importing this one — a dynamic
// import of a command module drags its whole flag surface into the importer's
@@ -244,19 +254,22 @@ export function shouldSpawnAutopilotWorker(args: string[]): boolean {
return !args.includes('--no-worker');
}
export function isPidAlive(pid: number): boolean {
if (!Number.isFinite(pid) || pid <= 0) return false;
export { isPidAlive };
export const AUTOPILOT_FOREIGN_PID_TAKEOVER_GRACE_MS = 10 * 60 * 1000;
function autopilotLockAgeMs(lockPath: string): number | null {
try {
process.kill(pid, 0);
return true;
} catch (error: unknown) {
return (error as NodeJS.ErrnoException).code === 'EPERM';
return Date.now() - statSync(lockPath).mtimeMs;
} catch {
return null;
}
}
export function decideLockAcquisition(
lockPath: string,
currentPid: number,
deps: AutopilotLockProbeDeps = {},
): { action: 'acquire' } | { action: 'exit'; holderPid: number } | { action: 'takeover'; reason: string } {
if (!existsSync(lockPath)) return { action: 'acquire' };
@@ -268,10 +281,21 @@ export function decideLockAcquisition(
}
const holderPid = Number.parseInt(raw, 10);
const sameProcess = Number.isFinite(holderPid) && holderPid === currentPid;
const alive = !sameProcess && isPidAlive(holderPid);
const holder = classifyAutopilotLockHolder(holderPid, currentPid, deps);
if (alive) return { action: 'exit', holderPid };
if (holder.state === 'alive-autopilot' || holder.state === 'alive-unknown') {
return { action: 'exit', holderPid };
}
if (holder.state === 'alive-foreign') {
const lockAgeMs = autopilotLockAgeMs(lockPath);
if (lockAgeMs !== null && lockAgeMs >= AUTOPILOT_FOREIGN_PID_TAKEOVER_GRACE_MS) {
return { action: 'takeover', reason: `foreign pid ${raw || '<empty>'} with stale lock` };
}
return { action: 'exit', holderPid };
}
if (holder.state === 'self') {
return { action: 'takeover', reason: `own pid ${raw || '<empty>'}` };
}
return { action: 'takeover', reason: `dead pid ${raw || '<empty>'}` };
}
@@ -875,8 +899,8 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
//
// New logic: compute the remediation plan (cheap; no full doctor
// walk), then route to the right level of intervention:
// - Score >= 95 + empty plan: full cycle every 60min (phase-
// coupling exercise), otherwise sleep.
// - Full cycle every 60min regardless of score/plan (phase-
// coupling + freshness invariant); healthy brains sleep before it.
// - Small plan (<=3 steps, <5min): submit individual handlers.
// - Large plan or low score: full autopilot-cycle (the hammer).
//
@@ -1121,16 +1145,16 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
const estTotal = plan.reduce((s, r) => s + r.est_seconds, 0);
// Track time since last full cycle for the 60-min floor.
const FULL_CYCLE_FLOOR_MIN = 60;
const minutesSinceLastFull = (Date.now() - lastFullCycleAt) / 60000;
const shouldFullCycle =
(score >= 95 && plan.length === 0 && minutesSinceLastFull >= FULL_CYCLE_FLOOR_MIN) ||
plan.length > 3 ||
estTotal >= 300 ||
score < 70;
const shouldFullCycle = shouldRunAutopilotFullCycle({
score,
planLength: plan.length,
estimatedSeconds: estTotal,
minutesSinceLastFull,
});
const shouldSleep = score >= 95 && plan.length === 0 && minutesSinceLastFull < FULL_CYCLE_FLOOR_MIN;
const shouldSleep = shouldSleepHealthyAutopilot(score, plan.length, minutesSinceLastFull);
if (shouldSleep) {
if (jsonMode) {
@@ -1181,7 +1205,11 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
if (jsonMode) process.stderr.write(JSON.stringify({ event: 'global_maintenance_dispatch_failed', error: e instanceof Error ? e.message : String(e) }) + '\n');
}
}
if (result.dispatched.length > 0 || result.legacy_fallback) {
// On restart the process-local clock starts overdue. If persisted
// source timestamps say every source is fresh, advance the local
// clock too; otherwise a non-empty targeted plan would be skipped
// on every tick until the persisted 60-minute window elapsed.
if (result.dispatched.length > 0 || result.legacy_fallback || result.all_sources_fresh) {
lastFullCycleAt = Date.now();
}
if (jsonMode) {
@@ -1205,15 +1233,17 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
}
} else {
// Small targeted plan — submit individual handlers per step.
// D9 content-hash idempotency keys (from computeRecommendations).
// maxWaiting:1 per submit per codex #17 (closes the backpressure
// gap the prior implementation had for targeted submits).
// Recommendation keys stay stable for doctor/remediate checkpoints;
// Autopilot adds the dispatch interval so completed rows cannot hold
// the remediation slot forever (#4046).
// maxWaiting:1 per submit per codex #17 bounds the cross-window
// backlog if a targeted handler runs longer than one interval.
for (const step of plan) {
try {
const isProtected = !!step.protected;
const submitOpts = {
queue: 'default',
idempotency_key: step.idempotency_key,
idempotency_key: autopilotRemediationIdempotencyKey(step.idempotency_key, slot),
max_attempts: 2,
timeout_ms: timeoutMs,
maxWaiting: 1,
+113 -1
View File
@@ -74,7 +74,7 @@ import {
statusReport,
type StatusReport,
} from '../core/bootstrap/status.ts';
import { verifyWorkspace } from '../core/bootstrap/verify.ts';
import { verifyWorkspace, deriveWorkspaceSourceId } from '../core/bootstrap/verify.ts';
export const BOOTSTRAP_HELP = `gbrain bootstrap — paste-in agent install (Claude Code / Codex)
@@ -116,6 +116,60 @@ Env: GBRAIN_BOOTSTRAP_ABORT_AFTER=<phase> (test seam — abort after that phase'
const SUPPORT_HINT =
'If you are stuck: run `gbrain bootstrap status --json` and relay the "support" block verbatim.';
/**
* Per-subcommand `--help`/`-h`/`help` usage text for the subcommands that
* MUTATE state (create a repo, register MCP/hooks, run the verify contract,
* adopt a workspace, remove receipt-tracked paths, record an interview
* answer). `runBootstrap`'s dispatch checks `args[0]` for top-level help
* (`--help`/`-h`/`help`/no args), but a help token AFTER the subcommand name
* (e.g. `gbrain bootstrap repo --help`, `gbrain bootstrap uninstall help`)
* previously fell straight into the subcommand's own arg parsing, which had
* no help handling of its own so it ran the real mutation instead of
* printing help. `status`/`cloud-setup-script` are pure reads, so they don't
* need a guard.
*/
const SUBCOMMAND_HELP: Record<string, string> = {
render:
'gbrain bootstrap render [--force] [--only F] [--minimal]\n' +
' Render identity files from the confirmed interview answers. Never clobbers; --force backs up first.',
repo:
'gbrain bootstrap repo\n' +
' Create the dedicated PRIVATE GitHub repo (or adopt an EMPTY private repo you created\n' +
' under your own account), verify the privacy bit via the API, push.',
hooks:
'gbrain bootstrap hooks [--harness claude-code|codex] [--repair] [--no-hooks] [--gbrain-bin <path>]\n' +
' Register MCP (+ per-turn hooks on Claude Code, ON by default; --no-hooks opts out).',
verify:
'gbrain bootstrap verify [--json]\n' +
' The whole install contract (round-trip, graph floor, magic moment, scans, hooks smoke). Exit 0 or not done.',
attach:
'gbrain bootstrap attach [--harness H]\n' +
' Machine two: adopt a cloned agent workspace.',
uninstall:
'gbrain bootstrap uninstall [--delete-brain] [--home <dir>] [--yes]\n' +
' Receipt-keyed removal. The repo stays yours.',
interview:
'gbrain bootstrap interview --init | --set KEY "value" | --skip KEY | --status | --show | --confirm <hash>\n' +
' Create/record/read interview state. See `gbrain bootstrap --help` for the per-flag description.',
};
/**
* `--help`/`-h` are always recognized. The bare word `help` (no dashes) is
* ALSO recognized for every subcommand above EXCEPT `interview` mirroring
* the top-level `sub === 'help'` handling for a user who tries the same
* spelling after a subcommand name. `interview` is excluded from the
* bare-word form because its `--set KEY "value"` free-text answers could
* legitimately BE the literal word "help" (e.g. a one-word answer); none of
* the other subcommands' flags take arbitrary prose, only booleans, enums,
* or paths, so the bare-word collision risk there is negligible (matches the
* already-accepted low-impact risk of `-h` colliding with a literal path
* value like `--home -h`).
*/
function hasHelpToken(args: string[], allowBareWord: boolean): boolean {
if (args.includes('--help') || args.includes('-h')) return true;
return allowBareWord && args.includes('help');
}
/** Thrown by the A7 abort seam; mapped to exit 130 (simulated kill). */
export class BootstrapAbortInjected extends Error {
constructor(phase: string) {
@@ -154,6 +208,19 @@ function resolveWorkspace(args: string[]): string {
return ws ? resolve(ws) : process.cwd();
}
/**
* POSIX single-quote anything not already shell-safe, for commands printed
* as copy/paste guidance (mirror of the private `shellQuote` in
* core/bootstrap/hooks.ts, core/sources-ops.ts, and commands/connect.ts
* same contract: `$()`/backticks in a value are inert literals once quoted).
* A workspace path containing a space or shell metacharacter must not turn
* "the exact command to run" into a broken (or, pasted blind, dangerous) one.
*/
function shellQuoteForDisplay(arg: string): string {
if (/^[A-Za-z0-9_.:/@=-]+$/.test(arg)) return arg;
return `'${arg.replace(/'/g, "'\\''")}'`;
}
// ── Shared plumbing ─────────────────────────────────────────────────────────
type Harness = 'claude-code' | 'codex';
@@ -746,6 +813,41 @@ async function runHooks(ws: string, rest: string[], home: string, runner: ExecRu
const gbrainHome = process.env.GBRAIN_HOME?.trim() || undefined;
return withLock(ws, async () => {
// 0. source_id visibility seam: `hooks` is the last ENGINE-FREE phase
// before `verify` (which alone can detect a source_id collision — the
// sources registry lives only in the DB). Without this, a human who
// hand-registers a source before verify has no way to know the exact id
// the workspace expects, guesses an "intuitive" name instead, and only
// discovers the mismatch via a `verify` roundtrip FK error — then, after
// switching to the manifest's id, an `overlapping_path` error from the
// earlier guess still claiming the same brain/ dir. Printing the current
// id (and the collision-fallback id verify would derive, a pure path
// hash that needs no engine) up front — plus creating brain/ so
// registration can happen immediately — collapses that multi-round-trip
// loop to one command.
const brainDir = join(ws, 'brain');
mkdirSync(brainDir, { recursive: true });
// --force: brain/ was just created empty — `sources add` fail-fasts on a
// --path that exists but isn't a git repo with committed, tracked
// content (#2707), and gbrain deliberately never auto-git-inits a --path
// source itself (a --path source is the user's own directory — the
// consent boundary #2967 established for sync-time self-heal applies
// here too). --force is the sanctioned opt-in for exactly this "register
// before git-init exists" case (see sources-ops.ts's own not_a_git_repo
// message), and it is safe here because brainDir is not an arbitrary
// user path — it is the fixed `<workspace>/brain` subdir this phase just
// created. Without --force, the printed command below would itself throw
// not_a_git_repo the instant it's pasted.
const quoted = shellQuoteForDisplay(brainDir);
console.log(
`brain source: register this workspace's brain/ now if you haven't — ` +
`\`gbrain sources add ${sourceId} --path ${quoted} --force\` (brain/ is freshly created and empty; ` +
`--force is the documented opt-in for registering before git-init exists). If '${sourceId}' is ` +
`already claimed by a different checkout on this brain, \`gbrain bootstrap verify\` will detect the ` +
`collision and switch this workspace to '${deriveWorkspaceSourceId(ws)}' — re-run the same command ` +
`with that id instead.`,
);
// 1. MCP registration — argv built by the host-format module, executed
// through the runner seam, recorded on the receipt.
// A missing host binary (exit 127) skips MCP registration but NOT the
@@ -1126,6 +1228,16 @@ export async function runBootstrap(args: string[], opts: RunBootstrapOpts = {}):
return 2;
}
// Subcommand-level help: BEFORE any subcommand body runs, so a help token
// after a mutating subcommand (repo/hooks/verify/attach/uninstall/render/
// interview) never falls through into the real operation, regardless of
// what other flags/values precede it in `rest`. No install-log entry
// either — this isn't a phase run.
if (SUBCOMMAND_HELP[sub] && hasHelpToken(rest, sub !== 'interview')) {
console.log(SUBCOMMAND_HELP[sub]);
return 0;
}
// The install log records the PHASE name, and the hooks subcommand is the
// 'wire' phase (status.ts phase list) — one mapping, used at every log site.
const logPhaseName = sub === 'hooks' ? 'wire' : sub;
+336 -14
View File
@@ -782,6 +782,124 @@ export async function checkSourceConfigShape(engine: BrainEngine): Promise<Check
}
}
/**
* #2674 pglite_scratch_probe: distinguish a damaged PGLite store from a
* broken WASM runtime.
*
* PGLite reports only `Aborted()` to JS (the PANIC goes to its own stderr),
* so when init fails, the error string cannot say WHICH of the two it is.
* The probe initializes a throwaway store in a temp dir, round-trips a row,
* and reads the outcome:
*
* - scratch works, real init failed the runtime is fine; the failure is
* specific to YOUR store. The store-damage verdict is only ASSERTED when
* the caller supplies positive evidence (`storeDamageEvidence`: a
* damage-class disk diagnosis from `inspectPgliteDataDir`, or a
* wasm-abort/corrupt classification of the real init error). engine=null
* alone also covers locks and config refusals blaming the store for
* those was the original false-positive defect; without evidence the
* message hedges and points at the `pglite_data_dir` diagnosis instead.
* - scratch fails too the runtime cannot start on this machine; report
* OS + Bun versions on #223.
*
* COST GATE: a PGLite cold start is 520s on loaded machines, so this never
* runs on a routine `gbrain doctor`. It runs only when (a) the real PGLite
* engine actually failed to open (engine=null, not --fast, configured engine
* is pglite) AND the disk diagnosis didn't already fully explain the failure
* (a live lock / missing dir needs no runtime probe), or (b) the operator
* asks with `--probe-pglite`.
*
* `probeFn` is a test seam so message routing can be pinned without paying
* real cold starts.
*/
export async function checkPgliteScratchProbe(opts: {
realInitFailed: boolean;
/**
* Positive evidence the REAL store is damaged: `inspectPgliteDataDir`
* verdict wal-corruption-likely/unsupported-layout (buildChecks path) or a
* wasm-abort/corrupt classification of the actual connect error (remote
* path). Without it the scratch-ok arm hedges instead of asserting damage.
*/
storeDamageEvidence?: boolean;
realStorePath?: string;
probeFn?: () => Promise<import('../core/pglite-engine.ts').PgliteScratchProbeResult>;
}): Promise<Check> {
const name = 'pglite_scratch_probe';
try {
const probe =
opts.probeFn ??
(async () => {
const { probePgliteScratchStore } = await import('../core/pglite-engine.ts');
return probePgliteScratchStore(opts.realStorePath);
});
const r = await probe();
const secs = (r.duration_ms / 1000).toFixed(1);
if (r.ok) {
if (opts.realInitFailed && opts.storeDamageEvidence) {
return {
name,
status: 'fail',
message:
`A scratch PGLite store initialized, wrote and read back fine on this machine (${secs}s), ` +
`so the runtime is healthy and YOUR STORE is damaged — not the WASM runtime. ` +
`Your markdown is unaffected: the DB holds derived data (chunks, embeddings, links, facts) that a re-sync rebuilds. ` +
`Recover: \`gbrain pglite-repair --dry-run\` to diagnose, \`gbrain pglite-repair --yes\` for in-place WAL repair (data preserved); ` +
`if that can't fix it, restore a backup of the store directory or run \`gbrain reinit-pglite\` (wipes + re-inits + re-syncs; ` +
`defaults embedding flags from your config file).`,
details: { scratch_ok: true, duration_ms: r.duration_ms },
};
}
if (opts.realInitFailed) {
// Runtime proven healthy, but no independent evidence of store DAMAGE
// — engine=null also covers locks, config refusals, and transient
// failures. Hedge rather than convict the store (#2674 review).
return {
name,
status: 'warn',
message:
`A scratch PGLite store initialized, wrote and read back fine on this machine (${secs}s), ` +
`so the WASM runtime is healthy — the failure opening your brain is specific to your store, ` +
`its lock, or its configuration. See the \`pglite_data_dir\` check for the on-disk diagnosis; ` +
`\`gbrain pglite-repair --dry-run\` diagnoses without mutating anything.`,
details: { scratch_ok: true, duration_ms: r.duration_ms },
};
}
return {
name,
status: 'ok',
message: `PGLite runtime healthy: scratch store round-trip in ${secs}s.`,
details: { scratch_ok: true, duration_ms: r.duration_ms },
};
}
const errLine = (r.error ?? 'unknown error').split('\n')[0];
if (opts.realInitFailed) {
return {
name,
status: 'fail',
message:
`A fresh scratch PGLite store ALSO failed to start (${secs}s), so the WASM runtime cannot run ` +
`on this machine — your store is not necessarily damaged. Report your OS and Bun versions on ` +
`https://github.com/garrytan/gbrain/issues/223. Scratch error: ${errLine}`,
details: { scratch_ok: false, duration_ms: r.duration_ms, error: r.error, verdict: r.verdict },
};
}
return {
name,
status: 'warn',
message:
`Your real store opened, but a fresh scratch PGLite store failed to initialize (${secs}s) — ` +
`new stores can't be created on this machine. Report your OS and Bun versions on ` +
`https://github.com/garrytan/gbrain/issues/223. Scratch error: ${errLine}`,
details: { scratch_ok: false, duration_ms: r.duration_ms, error: r.error, verdict: r.verdict },
};
} catch (e) {
// Includes the never-touch-the-real-store guard refusal. The probe not
// running is a diagnostic gap, not a diagnosis — warn, don't fail.
const msg = e instanceof Error ? e.message : String(e);
return { name, status: 'warn', message: `scratch probe could not run: ${msg}` };
}
}
export async function doctorReportRemote(
engine: BrainEngine,
opts: { sourceIds?: string[] } = {},
@@ -804,6 +922,23 @@ export async function doctorReportRemote(
status: 'fail',
message: e instanceof Error ? e.message : String(e),
});
// #2674: on PGLite, a dead connection is exactly the ambiguous case the
// scratch probe exists for — pay its cold start only on this failure path.
// Unlike buildChecks (where the connect error was swallowed upstream), the
// real error IS in hand here: classify it, and only let the probe assert
// store damage on a damage-class verdict (wasm-abort/corrupt) — a lock or
// config refusal classifies 'unknown' and gets the hedged message.
if (engine.kind === 'pglite') {
let realStorePath: string | undefined;
try { realStorePath = loadConfig()?.database_path; } catch { /* no config */ }
let storeDamageEvidence = false;
try {
const { classifyPgliteInitError, stringifyPgliteInitError } = await import('../core/pglite-engine.ts');
const verdict = classifyPgliteInitError(stringifyPgliteInitError(e));
storeDamageEvidence = verdict === 'wasm-abort' || verdict === 'corrupt';
} catch { /* classifier unavailable — stay hedged (fail-closed) */ }
checks.push(await checkPgliteScratchProbe({ realInitFailed: true, storeDamageEvidence, realStorePath }));
}
// Without a connection, every other check is meaningless — short-circuit.
return computeDoctorReport(checks);
}
@@ -1747,6 +1882,8 @@ export async function checkVoiceGateHealth(engine: BrainEngine): Promise<Check>
* Below that they're noise; reranker fails open anyway.
* 5) Payload-too-large failures: warn at >=1 (indicates a workload
* mismatch that the operator should know about).
* 6) Budget/pricing failures: warn at >=1 with the rerank pricing surface
* and --max-cost escape hatch.
*
* Engine-agnostic (file-based + one config-key read).
*/
@@ -1785,6 +1922,15 @@ export async function checkRerankerHealth(engine: BrainEngine): Promise<Check> {
};
}
const budgetFails = failures.filter((f) => f.reason === 'budget');
if (budgetFails.length > 0) {
return {
name: 'reranker_health',
status: 'warn',
message: `${budgetFails.length} reranker budget/pricing failure(s) in last 7 days. Fix: add rerank pricing to src/core/embedding-pricing.ts or drop --max-cost.`,
};
}
const transientFails = failures.filter(
(f) => f.reason === 'network' || f.reason === 'timeout' || f.reason === 'rate_limit',
);
@@ -2502,6 +2648,131 @@ export async function checkZeEmbeddingHealth(engine: BrainEngine): Promise<Check
}
}
/**
* provider_sunset doctor check (#3390 follow-up).
*
* Detects a brain whose EFFECTIVE embedding model (gateway-resolved, which is
* how default-config brains land on the shipped default) is on a provider
* with an announced hosted-API shutdown, and prints a paste-ready migration
* command with the brain's ACTUAL `content_chunks.embedding` column width
* filled in not the config value, which can drift. Keeping the current
* width avoids a needless dimension transition + index rebuild when the
* target supports it.
*
* Unlike the one-shot upgrade banner (`ze_sunset_notice_shown`), this fires
* on every `gbrain doctor` run until the brain is off the provider
* warn before the shutdown date; fail after it ONLY when the brain is
* actually exposed (embedded vectors exist in the affected column, so
* retrieval is genuinely down). A zero-vector brain whose config merely
* RESOLVES to the dead default stays warn otherwise every stock fresh
* install (and every doctor-as-CI-gate) starts exiting 1 on the date with
* no code change. Suppress entirely (accepted-risk installs) via
* `gbrain config set doctor.suppress_provider_sunset true`.
* No network call; one catalog query for the column width.
*
* `now` is injectable so tests can pin BOTH sides of the date without
* waiting for the calendar (the date itself is a compile-time constant).
*/
export async function checkProviderSunset(engine: BrainEngine, now: number = Date.now()): Promise<Check> {
const name = 'provider_sunset';
try {
const suppressed = await engine.getConfig('doctor.suppress_provider_sunset').catch(() => null);
if (suppressed === 'true' || suppressed === '1') {
return {
name,
status: 'ok',
message: 'Check suppressed via doctor.suppress_provider_sunset (unset it to re-enable).',
};
}
const { DEFAULT_EMBEDDING_MODEL, ZEROENTROPY_SUNSET_DATE } = await import('../core/ai/defaults.ts');
// Effective model: gateway when configured (file/env plane, the runtime
// truth); the shipped default otherwise — an unset-config brain resolves
// to the default at runtime, so it is just as affected.
let model = DEFAULT_EMBEDDING_MODEL;
try {
const { getEmbeddingModel } = await import('../core/ai/gateway.ts');
model = getEmbeddingModel();
} catch {
// Gateway unconfigured — runtime resolves the shipped default.
}
// Effective reranker: resolve through the SAME plane search actually
// reranks with — resolveSearchMode (mode bundle + search.reranker.*
// config overrides; hybrid.ts passes `resolvedMode.reranker_model`).
// The gateway plane is unset by default while balanced/tokenmax rerank
// with the bundle's zeroentropyai model — reading the gateway here
// would false-ok the exact brains this check exists to protect.
let reranker: string | undefined;
try {
const { loadSearchModeConfig, resolveSearchMode } = await import('../core/search/mode.ts');
const knobs = resolveSearchMode(await loadSearchModeConfig(engine));
if (knobs.reranker_enabled) reranker = knobs.reranker_model;
} catch {
// Mode resolution failed — make no reranker-exposure claim.
}
const onSunsetEmbedding = model.startsWith('zeroentropyai:');
const onSunsetReranker = !!reranker?.startsWith('zeroentropyai:');
if (!onSunsetEmbedding && !onSunsetReranker) {
return {
name,
status: 'ok',
message: `No configured provider has an announced shutdown (embedding: ${model}).`,
};
}
const past = now >= Date.parse(`${ZEROENTROPY_SUNSET_DATE}T00:00:00Z`);
const parts: string[] = [];
let hasVectors = false;
if (onSunsetEmbedding) {
let dims: number | null = null;
try {
const { readContentChunksEmbeddingDim } = await import('../core/embedding-dim-check.ts');
dims = (await readContentChunksEmbeddingDim(engine)).dims;
} catch {
// Column probe failed (fresh/odd brain) — omit --dim from the hint.
}
try {
const rows = await engine.executeRaw(
`SELECT 1 AS one FROM content_chunks WHERE embedding IS NOT NULL LIMIT 1`,
);
hasVectors = rows.length > 0;
} catch {
// Probe failed (fresh/odd brain) — no exposure claim, warn-only.
}
const dimFlag = dims ? ` --dim ${dims}` : '';
parts.push(
past
? hasVectors
? `embedding_model="${model}": the hosted API shut down on ${ZEROENTROPY_SUNSET_DATE} — semantic retrieval is offline (queries can no longer be embedded against your existing vectors).`
: `embedding_model="${model}": the hosted API shut down on ${ZEROENTROPY_SUNSET_DATE}. No embedded vectors exist yet, so retrieval is not impacted — but embedding will fail until the config points elsewhere.`
: `embedding_model="${model}": the hosted API shuts down on ${ZEROENTROPY_SUNSET_DATE}. On that date semantic retrieval stops entirely — existing vectors become unqueryable (query embedding uses the same endpoint), not just new content.`,
);
parts.push(
`Two fixes, either works: ` +
`[1] self-host the same model — zembed-1 weights are Apache-2.0; serve them via llama-server or Ollama and point the config at the local endpoint. Keeps every existing vector, no re-embed (docs/guides/embedding-migration.md, "Self-hosting instead of migrating"). ` +
`[2] migrate to another provider (resumable; preview cost first): ` +
`gbrain migrate embeddings --to <provider:model>${dimFlag} --dry-run` +
(dims ? ` — keep --dim ${dims} (this brain's actual index width) to avoid a needless schema rebuild when the target supports it.` : ''),
);
}
if (onSunsetReranker) {
parts.push(
`The reranker (${reranker}) is on the same provider; after the shutdown search falls back to unreranked ordering. ` +
`Fix: gbrain config set search.reranker.enabled false, or point search.reranker.model at another provider.`,
);
}
if (onSunsetEmbedding || onSunsetReranker) {
parts.push('Accepted the risk? Silence this check: gbrain config set doctor.suppress_provider_sunset true');
}
// fail = retrieval is ACTUALLY down (past the date AND embedded vectors
// exist on the dead provider). Reranker-only exposure stays warn — search
// fails open to unreranked ordering (degraded, not down).
const failNow = past && onSunsetEmbedding && hasVectors;
return { name, status: failNow ? 'fail' : 'warn', message: parts.join(' ') };
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
return { name, status: 'warn', message: `Could not check provider sunset status: ${msg}` };
}
}
/**
* v0.36.0.0 (A5): embedding_width_consistency doctor check.
*
@@ -5606,7 +5877,15 @@ export async function buildChecks(
if (lastStarted && engine) {
const queue = typeof lastStarted.queue === 'string' ? lastStarted.queue : 'default';
const effectiveMaxRss = typeof lastStarted.max_rss_mb === 'number' ? lastStarted.max_rss_mb : null;
const localPid = readSupervisorPid(DEFAULT_PID_FILE).pid;
// The 'started' event already records the pid-file path actually in use
// (this.opts.pidFile, which reflects a custom --pid-file). Prefer that
// over re-deriving DEFAULT_PID_FILE locally so a custom --pid-file
// deployment doesn't false-positive a singleton mismatch against itself.
// Falls back to DEFAULT_PID_FILE when the event carries no usable value.
const pidFilePath = typeof lastStarted.pid_file === 'string' && lastStarted.pid_file.length > 0
? lastStarted.pid_file
: DEFAULT_PID_FILE;
const localPid = readSupervisorPid(pidFilePath).pid;
const localHost = hostname();
// Read the DB singleton lock holder for this queue.
@@ -6314,24 +6593,62 @@ export async function buildChecks(
// Filesystem read failure is non-fatal.
}
// 3d. PGLite data-dir diagnosis (WAL-repair wave). Only meaningful when the
// connect already FAILED on a PGLite brain (engine === null): the connect
// error was swallowed by the fs-only fallback, so this check re-derives the
// dir state from disk and names the repair ladder. Skipped under --fast
// (connect wasn't attempted, so "engine === null" proves nothing there).
if (!fastMode && !engine) {
try {
const cfg = loadConfig();
if (cfg?.engine === 'pglite') {
// 3d. PGLite data-dir diagnosis (WAL-repair wave) + scratch-store probe
// (#2674). The data-dir check re-derives the failure state from DISK (the
// connect error was swallowed by the fs-only fallback); the probe adds the
// RUNTIME dimension (a throwaway store that opens fine proves the WASM
// runtime is healthy). Both only fire when the connect already FAILED on a
// PGLite brain (engine === null, not --fast — under --fast connect wasn't
// attempted, so "engine === null" proves nothing there).
//
// Probe cost gate (a PGLite cold start is 520s): auto-runs ONLY when init
// failed AND the disk diagnosis didn't already fully explain it — a live
// lock or a missing dir needs no runtime probe (and 'locked' was exactly
// the reviewed false-positive: blaming the store while `gbrain serve` held
// it). Explicit --probe-pglite always runs it. A routine healthy
// `gbrain doctor` never pays it.
{
const probeRequested = args.includes('--probe-pglite');
let cfgForProbe: ReturnType<typeof loadConfig> = null;
try { cfgForProbe = loadConfig(); } catch { /* no config — nothing to diagnose */ }
const pgliteInitFailed = !engine && !fastMode && cfgForProbe?.engine === 'pglite';
let dirVerdict: import('../core/pglite-repair.ts').PgliteDirDiagnosis['verdict'] | undefined;
if (pgliteInitFailed) {
try {
const { inspectPgliteDataDir } = await import('../core/pglite-repair.ts');
const { resolve } = await import('node:path');
// Absolutize: a RELATIVE database_path would make the sidecar/backup
// lookups resolve against doctor's cwd instead of the engine's.
const pgliteDataDir = resolve(cfg.database_path || gbrainPath('brain.pglite'));
checks.push(computePgliteDataDirCheck(pgliteDataDir, inspectPgliteDataDir(pgliteDataDir)));
const pgliteDataDir = resolve(cfgForProbe!.database_path || gbrainPath('brain.pglite'));
const diagnosis = inspectPgliteDataDir(pgliteDataDir);
dirVerdict = diagnosis.verdict;
checks.push(computePgliteDataDirCheck(pgliteDataDir, diagnosis));
} catch {
// Best-effort: an unreadable config or fs failure must not stop doctor.
}
}
const dirExplainsFailure = dirVerdict === 'locked' || dirVerdict === 'missing';
if (probeRequested || (pgliteInitFailed && !dirExplainsFailure)) {
progress.start('doctor.pglite_probe');
const stopHb = startHeartbeat(progress, 'pglite scratch-store probe (cold start, can take 520s)…');
try {
checks.push(
await checkPgliteScratchProbe({
// A lock/missing dir explains the failure without the store being
// damaged — an explicit --probe-pglite there still reports on the
// runtime, but must not treat the store as the convicted party.
realInitFailed: pgliteInitFailed && !dirExplainsFailure,
storeDamageEvidence:
dirVerdict === 'wal-corruption-likely' || dirVerdict === 'unsupported-layout',
realStorePath: cfgForProbe?.database_path,
}),
);
} finally {
stopHb();
progress.finish();
}
} catch {
// Best-effort: an unreadable config or fs failure must not stop doctor.
}
}
@@ -8337,6 +8654,11 @@ export async function buildChecks(
// v0.36.0.0 (A5): ZE embedding key health + schema/config width consistency.
progress.heartbeat('ze_embedding_health');
checks.push(await checkZeEmbeddingHealth(engine));
// provider_sunset — brain pinned to a provider with an announced
// hosted-API shutdown; paste-ready migration hint with the actual
// column width. Warn before the date, fail after.
progress.heartbeat('provider_sunset');
checks.push(await checkProviderSunset(engine));
progress.heartbeat('embedding_width_consistency');
checks.push(await checkEmbeddingWidthConsistency(engine));
// v0.41.15.0 (T6, codex #19/#20) — facts.embedding column drift
+265 -4
View File
@@ -198,6 +198,22 @@ export interface EmbedResult {
failure_samples: string[];
/** True if this run was a dry-run. */
dryRun: boolean;
/**
* Chunkless-page safety net (`--stale` only): pages with non-empty
* content but zero `content_chunks` rows that this run chunked (or, in
* dryRun, would chunk) so their new NULL-embedding chunks fold into the
* SAME pass. 0 on a healthy brain. Additive field see
* `ChunklessPageRow` for the detection rationale.
*/
chunkless_pages_healed: number;
/**
* Set when a single-flight run did NO work because another backfill holds
* the per-source embed lock. A hard-killed (SIGKILL/crash) run leaves its
* lock behind for up to EMBED_BACKFILL_LOCK_TTL_MIN callers that promise
* "re-run to resume" (migrate embeddings) use this to say so instead of
* misreporting embed failures.
*/
lock_skipped?: boolean;
/**
* E1 (paced-backfill): end-of-run pacing telemetry. Present ONLY when pacing
* was active (enabled bundle). The number the operator could not get from an
@@ -317,6 +333,7 @@ export async function runEmbedCore(engine: BrainEngine, opts: EmbedOpts): Promis
failures: 0,
failure_samples: [],
dryRun: !!opts.dryRun,
chunkless_pages_healed: 0,
};
if (opts.slugs && opts.slugs.length > 0) {
@@ -375,6 +392,7 @@ export async function runEmbedCore(engine: BrainEngine, opts: EmbedOpts): Promis
try { await h.release(); } catch { /* best-effort */ }
}
serr(` [embed] another backfill is already running for source "${sid}"; skipping (single-flight).`);
result.lock_skipped = true;
return result;
}
sfLocks.push(lock);
@@ -529,6 +547,7 @@ export async function runEmbed(engine: BrainEngine, args: string[]): Promise<Emb
return {
embedded: 0, skipped: 0, would_embed: 0, total_chunks: 0,
pages_processed: 0, failures: 0, failure_samples: [], dryRun: false,
chunkless_pages_healed: 0,
};
}
@@ -988,6 +1007,201 @@ async function embedAll(
}
}
/**
* Chunkless-page safety net for `embed --stale`. `listStaleChunks` /
* `countStaleChunks` only ever look at `content_chunks` rows where
* `embedding IS NULL` a page written directly via `putPage` that never
* went through chunking (e.g. an enrichment-generated entity stub) has NO
* chunk row at all, so it is invisible to that scan forever, even after
* unlimited `embed --stale` runs.
*
* This sweep finds pages with non-empty content (`compiled_truth` and/or
* `timeline` both are chunked independently, mirroring `embedPage`'s
* chunkless branch) and zero `content_chunks` rows
* (`engine.listChunklessPagesWithContent`, which already excludes
* quarantined + embed_skip pages both intentionally chunkless). The new
* chunk rows land with `embedding = NULL`, so they flow into the SAME
* `embed --stale` pass via the existing cursor below no separate embed
* step needed here.
*
* dryRun chunks locally (a pure, in-memory operation) to report an
* accurate count without writing anything, matching embedPage's dry-run
* contract (including `pages_processed`, which embedPage's own dry-run
* branch increments for exactly this "examined, didn't write" case).
*
* Race note (review catch, three rounds ACCEPTED RESIDUAL RISK, not
* fully closed): between listing a page and writing its chunks, a
* concurrent writer (sync, another `put_page`) could change or chunk the
* SAME page. Two mitigations, both bounded full atomicity (a
* transaction/version-guarded conditional write inside `upsertChunks`)
* would need a new engine primitive shared by every `upsertChunks` caller,
* which is out of scope for a chunkless-page safety net:
* 1. Immediately before writing, re-fetch the LIVE page via `getPage`
* and build `inputs` from ITS CURRENT content, not the batch-list
* snapshot closes the "content changed but still chunkless"
* sub-case, not just the "chunks appeared" one.
* 2. Re-check `getChunks` right after that same fetch skip (don't
* overwrite) if chunks now exist AT THE TIME OF THE CHECK.
* What this does NOT close: a writer that inserts chunks in the gap
* BETWEEN step 2's check and the `upsertChunks` call immediately below it
* (no intervening `await` other than that one call, but `upsertChunks`
* itself is not conditioned on the check this is still check-then-write,
* not compare-and-swap) can still have its chunks overwritten HONESTLY:
* `upsertChunks` treats its input as the full desired chunk set for that
* page and deletes any existing chunk_index absent from it, so a
* concurrent writer's chunks landing in that exact gap CAN be replaced
* with this sweep's stale-content chunks (embedding NULL). This is the
* SAME check-then-write window `embedPage`'s existing single-page
* chunkless branch already ships with today (that branch doesn't even
* have step 2's re-check) no new race CLASS is introduced, and the
* window here is a single sequential getPage+getChunks+upsertChunks
* instead of spanning a whole batch. The blast radius is bounded: the
* page is NOT deleted or corrupted, just re-chunked from a stale
* snapshot, and the NEXT write to that page (sync, another edit) that
* actually chunks it restores correct content this sweep's own
* predicate is idempotent and doesn't compound the drift. Closing this
* fully (true atomicity) is tracked as a follow-up, not blocking this
* safety net.
*
* Per-page failure isolation (review catch): one malformed/oversized
* chunkless page must not abort the sweep and, with it, the entire
* `--stale` run before the normal NULL-embedding pass even starts that
* would make the safety net WORSE than the bug it fixes. Each page's
* work is try/caught; a failure is recorded (`EmbedResult.failures` +
* `failure_samples`, same convention as every other embed failure path)
* and the sweep moves on.
*
* Bounded, keyset-paginated (like listStalePagesForExtraction) a safety
* net for a rare drift case, not the primary bulk-chunking path. `BATCH_SIZE`
* is deliberately small (unlike the 2000-chunk-row default elsewhere in
* this file): each row here carries a FULL page body (`compiled_truth` +
* `timeline`), so a large batch of large pages is a real memory/latency
* concern the metadata-only `listStaleChunks` rows never had (review
* catch). It still respects the caller's pacer (no-op when pacing is off)
* and a soft wall-clock cap (`GBRAIN_EMBED_TIME_BUDGET_MS`) so a
* pathologically large damaged brain can't run this sweep unbounded it
* heals what it can and reports the rest for the next `embed --stale` run
* (the SQL predicate is idempotent; nothing here requires finishing in one
* pass). `startedAt` is shared with the caller's overall run clock (review
* catch) healing and the main stale loop draw from ONE combined budget
* window, not two independent 30-minute ones. `catchUp` mirrors the main
* loop's own `--catch-up` handling: removes the cap entirely (the keyset
* cursor still terminates on its own; `signal` remains the abort path).
*/
async function healChunklessPages(
engine: BrainEngine,
sourceId: string | undefined,
dryRun: boolean,
result: EmbedResult,
quiet: boolean | undefined,
signal: AbortSignal | undefined,
pacer: DbPacer | undefined,
startedAt: number,
catchUp: boolean,
): Promise<void> {
const BATCH_SIZE = 50;
const BUDGET_MS: number | null = catchUp
? null
: parseInt(process.env.GBRAIN_EMBED_TIME_BUDGET_MS || `${30 * 60 * 1000}`, 10);
const activePacer = pacer ?? createNoopPacer();
let afterPageId: number | undefined;
let pagesHealed = 0;
let budgetExceeded = false;
const buildInputs = (compiledTruth: string, timeline: string): ChunkInput[] => {
const inputs: ChunkInput[] = [];
if (compiledTruth.trim()) {
for (const c of chunkText(compiledTruth)) {
inputs.push({ chunk_index: inputs.length, chunk_text: c.text, chunk_source: 'compiled_truth' });
}
}
if (timeline.trim()) {
for (const c of chunkText(timeline)) {
inputs.push({ chunk_index: inputs.length, chunk_text: c.text, chunk_source: 'timeline' });
}
}
return inputs;
};
// BUDGET_MS === null means catch-up: no wall-clock cap on this sweep,
// mirroring the main stale loop's own --catch-up handling below.
const overBudget = (): boolean => BUDGET_MS != null && Date.now() - startedAt > BUDGET_MS;
// eslint-disable-next-line no-constant-condition
while (true) {
if (isAborted(signal)) break;
if (overBudget()) { budgetExceeded = true; break; }
const batch = await observed(activePacer, () => engine.listChunklessPagesWithContent({
batchSize: BATCH_SIZE,
...(afterPageId != null && { afterPageId }),
...(sourceId && { sourceId }),
}));
if (batch.length === 0) break;
afterPageId = batch[batch.length - 1].id;
for (const page of batch) {
if (isAborted(signal)) break;
if (overBudget()) { budgetExceeded = true; break; }
try {
if (dryRun) {
// dryRun never writes, so there's no live-refetch race to close —
// chunk the listed snapshot directly (matches embedPage's own
// dry-run, which chunks whatever getPage returned at call time).
const inputs = buildInputs(page.compiled_truth, page.timeline);
// Whitespace-only content (SQL prefilter is `<> ''`, not
// trim-aware) chunks to nothing — matches embedPage's contract.
if (inputs.length === 0) continue;
result.total_chunks += inputs.length;
result.would_embed += inputs.length;
result.pages_processed++;
pagesHealed++;
continue;
}
// Re-fetch the LIVE page + re-check chunks immediately before
// writing (see race note above): chunk CURRENT content, and skip
// rather than clobber if a concurrent writer already chunked this
// page since we listed it.
const [livePage, stillChunkless] = await Promise.all([
observed(activePacer, () => engine.getPage(page.slug, { sourceId: page.source_id })),
observed(activePacer, () => engine.getChunks(page.slug, { sourceId: page.source_id })),
]);
if (!livePage || stillChunkless.length > 0) continue;
const inputs = buildInputs(livePage.compiled_truth, livePage.timeline);
if (inputs.length === 0) continue;
await observed(activePacer, () =>
engine.upsertChunks(page.slug, inputs, { sourceId: page.source_id }),
);
pagesHealed++;
try {
await activePacer.pace(signal);
} catch (e) {
if (!(e instanceof AbortError)) throw e;
}
} catch (e) {
if (isAborted(signal)) break;
recordFailure(result, 1, page.slug, e);
serr(`\n [embed] chunkless-page heal failed for ${page.slug}: ${e instanceof Error ? e.message : e}`);
}
}
if (budgetExceeded || batch.length < BATCH_SIZE) break;
}
result.chunkless_pages_healed = pagesHealed;
if (pagesHealed > 0 && !quiet) {
if (dryRun) {
serr(`[embed] [dry-run] would chunk ${pagesHealed} page(s) with non-empty content but zero content_chunks rows`);
} else {
serr(`[embed] chunked ${pagesHealed} page(s) that had non-empty content but zero content_chunks rows (embedding them in this pass)`);
}
}
if (budgetExceeded && !quiet) {
serr(`[embed] chunkless-page sweep hit its time budget (${BUDGET_MS}ms) with more pages left; re-run embed --stale to continue healing them`);
}
}
/**
* SQL-side stale path: replaces the listPages + per-page getChunks
* walk with a count + slug-grouped SELECT. Preserves the existing
@@ -1028,11 +1242,38 @@ async function embedAllStale(
signature?: string,
externalSignal?: AbortSignal,
) {
// Shared wall-clock anchor (review catch): the healing sweep below and the
// main stale loop's own budget timer (further down) both measure against
// this SAME start time, so a run's total wall-clock spend stays capped at
// ONE `GBRAIN_EMBED_TIME_BUDGET_MS` window instead of summing two
// independent 30-minute budgets.
const overallStartedAt = Date.now();
// D7: thread sourceId so source-scoped runs only count + visit
// that source's NULL embeddings.
const sourceOpt = sourceId ? { sourceId } : undefined;
const includeNullSig = !!staleOpts?.includeNullSignature;
// Chunkless-page safety net: pre-flight count mirrors the countStaleChunks
// short-circuit just below — a healthy brain pays one extra SELECT
// count(*) and does no further work. Only when pages are actually found
// do we pay for the keyset-paginated chunk sweep. Chunking here (before
// countStaleChunks) means any newly-written NULL-embedding chunks flow
// into the SAME pass via the existing cursor.
const chunklessCount = await engine.countChunklessPagesWithContent(sourceOpt);
if (chunklessCount > 0) {
await healChunklessPages(
engine, sourceId, dryRun, result, staleOpts?.quiet, externalSignal, staleOpts?.pacer,
overallStartedAt, !!staleOpts?.catchUp,
);
}
// Review catch: an abort during healing must stop the run HERE, before
// falling through into invalidateStaleSignatureEmbeddings below (which —
// pre-existing, unchanged by this PR — does not itself check
// externalSignal). Without this, a caller-cancelled run could still NULL
// out signature-drifted embeddings and exit, leaving retrieval degraded.
if (isAborted(externalSignal)) return;
// v0.41.31: re-embed pages whose embedding_signature drifted (model/dims
// swap). dry-run must NOT mutate, so it counts signature-stale via the
// widened predicate; a live run NULLs them first so the existing
@@ -1085,7 +1326,15 @@ async function embedAllStale(
if (staleCount === 0) {
if (!staleOpts?.quiet) {
if (dryRun) {
slog('[dry-run] Would embed 0 chunks (0 stale found)');
// dryRun never writes, so a healed-but-hypothetical chunkless page's
// chunks never land in content_chunks and staleCount can't see them
// — report result.would_embed (already includes them) instead of a
// bare "0 chunks" that would contradict the returned EmbedResult.
if (result.would_embed > 0) {
slog(`[dry-run] Would embed ${result.would_embed} chunks (0 stale found; ${result.chunkless_pages_healed} chunkless page(s) would be chunked)`);
} else {
slog('[dry-run] Would embed 0 chunks (0 stale found)');
}
} else {
slog('Embedded 0 chunks (0 stale found)');
}
@@ -1101,7 +1350,16 @@ async function embedAllStale(
// made `embed.pages` claim total:1 next to a summary naming a much larger
// stale count. docs/progress-events.md allows omitting `total` when it is
// not known up front; it does not allow asserting a wrong one.
if (!staleOpts?.quiet) slog(`[dry-run] Would embed ${staleCount} stale chunks`);
//
// Log result.would_embed (staleCount + any chunkless-page-healing
// contribution from above), not the bare staleCount — otherwise this
// line understates the total whenever chunkless pages were also found.
if (!staleOpts?.quiet) {
const chunklessNote = result.chunkless_pages_healed > 0
? `, including ${result.chunkless_pages_healed} chunkless page(s)`
: '';
slog(`[dry-run] Would embed ${result.would_embed} stale chunks${chunklessNote}`);
}
return;
}
@@ -1132,9 +1390,12 @@ async function embedAllStale(
? null
: parseInt(process.env.GBRAIN_EMBED_TIME_BUDGET_MS || `${30 * 60 * 1000}`, 10);
const budgetController = new AbortController();
const budgetStart = Date.now();
// Shares overallStartedAt with the chunkless-page healing sweep above
// (review catch) so the two phases draw from ONE combined budget window
// instead of each getting a fresh 30 minutes.
const budgetStart = overallStartedAt;
let budgetTimer = BUDGET_MS != null
? setTimeout(() => budgetController.abort(), BUDGET_MS)
? setTimeout(() => budgetController.abort(), Math.max(0, budgetStart + BUDGET_MS - Date.now()))
: undefined;
// E-4 (paced-backfill): the budget measures WORK, not waiting. After each
// batch, re-arm the timer to fire at start + BUDGET + total-paced-sleep, so a
+12 -3
View File
@@ -108,7 +108,11 @@ export async function runExport(engine: BrainEngine, args: string[]) {
let exported = 0;
for (const page of pages) {
const tags = await engine.getTags(page.slug);
// Slugs are unique per source, not brain-wide, so both sidecar reads are
// pinned to the page's own source. Unscoped, `getTags` falls back to
// `source_id = 'default'` and stamps the default source's tags onto a
// same-slug page from another source (dropping its real ones).
const tags = await engine.getTags(page.slug, { sourceId: page.source_id });
const md = serializeMarkdown(
page.frontmatter,
page.compiled_truth,
@@ -120,8 +124,13 @@ export async function runExport(engine: BrainEngine, args: string[]) {
mkdirSync(dirname(filePath), { recursive: true });
writeFileSync(filePath, md);
// Export raw data as sidecar JSON
const rawData = await engine.getRawData(page.slug);
// Export raw data as sidecar JSON. Unscoped, this matches the slug in
// EVERY source and the loop below merges the rows into one sidecar keyed
// by `rd.source`, so another source's raw data silently overwrites this
// page's own on a key collision.
const rawData = await engine.getRawData(page.slug, undefined, {
sourceId: page.source_id,
});
if (rawData.length > 0) {
const slugParts = page.slug.split('/');
const rawDir = join(outDir, ...slugParts.slice(0, -1), '.raw');
+44 -4
View File
@@ -81,8 +81,10 @@ const BATCH_SIZE = 100;
const STALE_BATCH_SIZE = Math.max(1, Number(process.env.GBRAIN_EXTRACT_STALE_BATCH) || 25);
// v0.42.7: wall-clock budget for one `extract --stale` invocation (default
// 30 min). `--catch-up` removes the cap (loops until 0 stale). Mirrors
// embedAllStale's time-budget shape.
const STALE_TIME_BUDGET_MS = Math.max(1000, Number(process.env.GBRAIN_EXTRACT_TIME_BUDGET_MS) || 30 * 60 * 1000);
// embedAllStale's time-budget shape. Exported so the #2849 deferred-sweep
// submitters (sync's size-gate defer branch + the jobs continuation chain)
// derive their job timeout_ms from the SAME budget instead of hardcoding.
export const STALE_TIME_BUDGET_MS = Math.max(1000, Number(process.env.GBRAIN_EXTRACT_TIME_BUDGET_MS) || 30 * 60 * 1000);
/**
* v0.42.7 (#1696): best-effort extraction stamp for the source-correct write
@@ -488,15 +490,53 @@ export async function extractLinksFromFile(
// --- Timeline extraction ---
/**
* Index of the first dash (, , -) that can serve as the Source Summary
* delimiter: it must have whitespace on both sides and sit outside every
* markdown-link span. Hyphens inside link targets
* (`../people/alice-example.md`) and dashes inside link labels
* (`[Deals — Q1 Review](...)`) are content, not delimiters splitting on
* them shatters one entry into two fragments whose halves re-insert on
* every sync (the (page_id, date, summary, source) uniqueness sees each
* fragment shape as a new row). Returns -1 when the line has no delimiter.
*/
function findDelimiterOutsideLinks(text: string): number {
let depth = 0;
for (let i = 0; i < text.length; i++) {
const c = text[i];
if (c === '[' || c === '(') depth++;
else if (c === ']' || c === ')') { if (depth > 0) depth--; }
else if (
depth === 0 &&
(c === '—' || c === '' || c === '-') &&
i > 0 && /\s/.test(text[i - 1]) &&
i + 1 < text.length && /\s/.test(text[i + 1])
) {
return i;
}
}
return -1;
}
/** Extract timeline entries from markdown content */
export function extractTimelineFromContent(content: string, slug: string): ExtractedTimelineEntry[] {
const entries: ExtractedTimelineEntry[] = [];
// Format 1: Bullet — - **YYYY-MM-DD** | Source — Summary
const bulletPattern = /^-\s+\*\*(\d{4}-\d{2}-\d{2})\*\*\s*\|\s*(.+?)\s*[—–-]\s*(.+)$/gm;
// The delimiter search is link-aware (see findDelimiterOutsideLinks); a
// bullet with no delimiter (e.g. an auto-generated backlink line
// `- **date** | Referenced in [X](y.md)`) is kept whole as the summary
// rather than dropped or fragmented.
const bulletPattern = /^-\s+\*\*(\d{4}-\d{2}-\d{2})\*\*\s*\|\s*(.+)$/gm;
let match;
while ((match = bulletPattern.exec(content)) !== null) {
entries.push({ slug, date: match[1], source: match[2].trim(), summary: match[3].trim() });
const rest = match[2].trim();
const at = findDelimiterOutsideLinks(rest);
if (at >= 0) {
entries.push({ slug, date: match[1], source: rest.slice(0, at).trim(), summary: rest.slice(at + 1).trim() });
} else {
entries.push({ slug, date: match[1], source: 'markdown', summary: rest });
}
}
// Format 2: Header — ### YYYY-MM-DD — Title
+35 -7
View File
@@ -25,6 +25,34 @@ import {
resumeFilter,
} from '../core/import-checkpoint.ts';
/**
* Records one failed file against the run's error-grouping state and
* returns the running count for its group plus an unredacted sample
* message for display.
*
* `key` groups structurally-identical errors (e.g. the same failure
* across many files) so a single noisy failure mode doesn't produce
* thousands of near-duplicate warning lines quoted substrings (typically
* a per-file slug or path) are blanked for the GROUPING key only. The
* printed `sample` is always a real, unredacted occurrence of the error
* (the first one seen for that key), so identifying details that are
* constant across the whole group a Postgres table or constraint name,
* for instance survive into what actually gets shown to the user.
* Pre-fix, the redacted key itself was printed, so e.g. a `pages_source_id_fkey`
* foreign-key violation surfaced as `table "" violates foreign key constraint ""`.
*/
export function recordImportFailure(
errorCounts: Record<string, number>,
errorSamples: Record<string, string>,
msg: string,
): { key: string; count: number; sample: string } {
const key = msg.replace(/"[^"]*"/g, '""');
const count = (errorCounts[key] ?? 0) + 1;
errorCounts[key] = count;
if (!(key in errorSamples)) errorSamples[key] = msg;
return { key, count, sample: errorSamples[key] };
}
function defaultWorkers(): number {
const cpuCount = cpus().length;
const memGB = totalmem() / (1024 ** 3);
@@ -288,6 +316,7 @@ export async function runImport(
let chunksCreated = 0;
const importedSlugs: string[] = [];
const errorCounts: Record<string, number> = {};
const errorSamples: Record<string, string> = {};
const failures: Array<{ path: string; error: string }> = []; // Bug 9
// #3839: paths that succeeded (imported OR unchanged) this run, keyed the
// same way as `failures` above (importRelPath) so a path that failed on a
@@ -351,12 +380,11 @@ export async function runImport(
}
} catch (e: unknown) {
const msg = e instanceof Error ? e.message : String(e);
const errorKey = msg.replace(/"[^"]*"/g, '""');
errorCounts[errorKey] = (errorCounts[errorKey] || 0) + 1;
if (errorCounts[errorKey] <= 5) {
const { count, sample } = recordImportFailure(errorCounts, errorSamples, msg);
if (count <= 5) {
console.error(` Warning: skipped ${relativePath}: ${msg}`);
} else if (errorCounts[errorKey] === 6) {
console.error(` (suppressing further "${errorKey.slice(0, 60)}..." errors)`);
} else if (count === 6) {
console.error(` (suppressing further "${sample.slice(0, 60)}..." errors)`);
}
errors++;
skipped++;
@@ -457,9 +485,9 @@ export async function runImport(
progress.finish();
// Error summary
for (const [err, count] of Object.entries(errorCounts)) {
for (const [key, count] of Object.entries(errorCounts)) {
if (count > 5) {
console.error(` ${count} files failed: ${err.slice(0, 100)}`);
console.error(` ${count} files failed: ${errorSamples[key].slice(0, 100)}`);
}
}
+38 -1
View File
@@ -1708,7 +1708,44 @@ export async function registerBuiltinHandlers(
});
worker.register('extract', async (job) => {
const { runExtractCore } = await import('./extract.ts');
const { runExtractCore, extractStaleFromDB, STALE_TIME_BUDGET_MS } = await import('./extract.ts');
// #2849: stale mode — the durable follow-up for extraction deferred by
// performSync's size gate (totalChanges > 100). Runs the same DB-source
// watermark sweep as `gbrain extract --stale`, scoped to the source the
// sync that deferred it was scoped to (job.data.sourceId; absent =
// unscoped, matching what the CLI hint tells a default-brain operator
// to run). The sweep is checkout-less + idempotent, so retries and
// overlapping submissions converge.
if (job.data.stale === true) {
const sourceIdFilter = typeof job.data.sourceId === 'string' ? job.data.sourceId : undefined;
const r = await extractStaleFromDB(engine, {
dryRun: !!job.data.dryRun,
jsonMode: false,
includeFrontmatter: false,
sourceIdFilter,
catchUp: false,
});
// Internal 30-min budget hit with work remaining → chain a
// continuation job so a very large deferred backlog converges without
// waiting for the next sync. Forward-progress guard (pagesProcessed >
// 0) prevents an infinite chain if the sweep can't advance.
if (!job.data.dryRun && r.staleRemaining > 0 && r.pagesProcessed > 0) {
try {
const queue = new MinionQueue(engine);
// NO maxWaiting: with an unscoped (NULL-sourceId) payload the
// coalesce filter matches ANY waiting 'extract' job and would
// swallow the continuation. Each completed sweep chains at most
// one continuation and the sweep is an idempotent watermark scan,
// so there is no pile-up to guard against.
await queue.add(
'extract',
{ ...job.data, continuation_of: job.id },
{ timeout_ms: STALE_TIME_BUDGET_MS + 5 * 60 * 1000 },
);
} catch { /* best-effort: next sync/manual sweep picks up the rest */ }
}
return { stale: true, source_id: sourceIdFilter ?? null, ...r };
}
const mode = (typeof job.data.mode === 'string' && ['links', 'timeline', 'all'].includes(job.data.mode))
? (job.data.mode as 'links' | 'timeline' | 'all')
: 'all';
+14 -1
View File
@@ -389,7 +389,20 @@ export async function runMigrateEmbeddings(
exit(0);
} else {
if (flags.json) {
console.log(JSON.stringify({ status: 'incomplete', plan, embedded: embedResult.embedded, remaining }, null, 2));
console.log(JSON.stringify({
status: 'incomplete', plan, embedded: embedResult.embedded, remaining,
...(embedResult.lock_skipped && { lock_skipped: true }),
}, null, 2));
} else if (embedResult.lock_skipped) {
// E2E-observed failure mode: a hard-killed (SIGKILL/crash) migration
// leaves its single-flight embed lock behind, and every immediate
// re-run "resumes" without embedding anything. Say so — "re-run to
// resume" would be a lie until the lock expires.
const { EMBED_BACKFILL_LOCK_TTL_MIN } = await import('../core/embed-backfill-lock.ts');
serr(`Migration paused: ${remaining} chunk(s) still stale, and the re-embed was SKIPPED because`);
serr('another embed backfill holds the per-source lock. If that is a live run (check');
serr('`gbrain jobs list`), let it finish. If a previous migration was killed hard, its lock');
serr(`expires after at most ${EMBED_BACKFILL_LOCK_TTL_MIN} minutes — re-run the same command then.`);
} else {
serr(`Migration incomplete: ${remaining} chunk(s) still stale (embed failures or an interrupted run).`);
serr('Re-run the same command to resume — completed chunks are never re-embedded.');
+24 -13
View File
@@ -80,6 +80,10 @@ export function manifestMatchesTarget(manifest: MigrateManifest, targetId: strin
return manifest.schema_version === 2 && manifest.target_id === targetId;
}
function makeManifestKey(sourceId: string, slug: string): string {
return sourceId === 'default' ? slug : `${sourceId}::${slug}`;
}
function loadManifest(): MigrateManifest | null {
const path = getManifestPath();
if (!existsSync(path)) return null;
@@ -151,6 +155,25 @@ export async function copyMigrationSources(source: BrainEngine, target: BrainEng
}
}
export async function copyPageLinksToTarget(
source: BrainEngine,
target: BrainEngine,
page: Page,
failedKeys: ReadonlySet<string> = new Set(),
): Promise<void> {
const links = await source.getLinks(page.slug, { sourceId: page.source_id });
for (const link of links) {
const toSourceId = link.to_source_id ?? page.source_id;
if (failedKeys.has(makeManifestKey(toSourceId, link.to_slug))) continue;
await target.addLink(
link.from_slug, link.to_slug,
link.context, link.link_type,
undefined, undefined, undefined,
{ fromSourceId: page.source_id, toSourceId },
);
}
}
/**
* postgres.js's UNDEFINED_VALUE guard rejects any bound parameter that is JS
* `undefined` unlike PGLite, it will not silently treat it as SQL NULL.
@@ -564,8 +587,6 @@ export async function runMigrateEngine(sourceEngine: BrainEngine, args: string[]
// entries were bare slugs; we keep treating those as default-source for
// back-compat resume.
const completedSet = new Set(manifest?.completed_slugs || []);
const makeManifestKey = (sourceId: string, slug: string): string =>
sourceId === 'default' ? slug : `${sourceId}::${slug}`;
if (!manifest) {
manifest = {
completed_slugs: [],
@@ -680,17 +701,7 @@ export async function runMigrateEngine(sourceEngine: BrainEngine, args: string[]
progress.tick(1);
continue;
}
const sourceOpts = { sourceId: page.source_id };
const links = await sourceEngine.getLinks(page.slug, sourceOpts);
for (const link of links) {
if (failedKeys.has(makeManifestKey(page.source_id, link.to_slug))) continue;
await targetEngine.addLink(
link.from_slug, link.to_slug,
link.context, link.link_type,
undefined, undefined, undefined,
{ fromSourceId: page.source_id, toSourceId: page.source_id },
);
}
await copyPageLinksToTarget(sourceEngine, targetEngine, page, failedKeys);
progress.tick(1);
}
progress.finish();
+50 -1
View File
@@ -27,7 +27,12 @@ import { OAuthTokenRevocationRequestSchema } from '@modelcontextprotocol/sdk/sha
import type { BrainEngine } from '../core/engine.ts';
import { operations, OperationError } from '../core/operations.ts';
import type { OperationContext, AuthInfo } from '../core/operations.ts';
import { GBrainOAuthProvider, validateTokenEndpointAuthMethod } from '../core/oauth-provider.ts';
import {
GBrainOAuthProvider,
validateTokenEndpointAuthMethod,
dcrRegistrationContext,
DEFAULT_DCR_TTL_MIN_SECONDS,
} from '../core/oauth-provider.ts';
import type { SqlQuery } from '../core/oauth-provider.ts';
import { hasScope, ALLOWED_SCOPES_LIST, normalizeScopesInput } from '../core/scope.ts';
import { normalizeSourceInput, normalizeFederatedReadInput } from '../core/source-id.ts';
@@ -702,11 +707,41 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption
// constructor option instead of monkey-patching `_clientsStore` after
// construction. Same outcome (no /register endpoint when --enable-dcr
// is not passed); cleaner shape for tests and future maintainers.
// #2179: admin-configured clamp window for DCR-requested token TTLs.
// DB-plane config keys (`gbrain config set oauth.dcr_ttl_min_seconds ...`).
// FAIL-CLOSED defaults: an unset/invalid max is bounded by the operator's
// own --token-ttl (never a fixed permissive ceiling), and an inverted
// window collapses to the min bound — the same direction clampDcrTokenTtl
// itself resolves. A bad config narrows the window; it never widens it.
const parseDcrTtlBound = (raw: unknown, fallback: number): number => {
const n = Number(raw);
return raw != null && Number.isFinite(n) && n >= 1 ? Math.floor(n) : fallback;
};
let dcrTtlMinSeconds = DEFAULT_DCR_TTL_MIN_SECONDS;
let dcrTtlMaxSeconds = Math.max(tokenTtl, dcrTtlMinSeconds);
try {
dcrTtlMinSeconds = parseDcrTtlBound(await engine.getConfig('oauth.dcr_ttl_min_seconds'), DEFAULT_DCR_TTL_MIN_SECONDS);
dcrTtlMaxSeconds = parseDcrTtlBound(await engine.getConfig('oauth.dcr_ttl_max_seconds'), Math.max(tokenTtl, dcrTtlMinSeconds));
} catch {
// Config read is best-effort; the fail-closed defaults stand.
dcrTtlMaxSeconds = Math.max(tokenTtl, dcrTtlMinSeconds);
}
if (dcrTtlMinSeconds > dcrTtlMaxSeconds) {
console.error(
`[serve-http] WARNING: oauth.dcr_ttl_min_seconds (${dcrTtlMinSeconds}) exceeds ` +
`oauth.dcr_ttl_max_seconds (${dcrTtlMaxSeconds}); collapsing the window to ` +
`the min bound (${dcrTtlMinSeconds}).`,
);
dcrTtlMaxSeconds = dcrTtlMinSeconds;
}
const oauthProvider = new GBrainOAuthProvider({
sql,
tokenTtl,
dcrDisabled: !enableDcr,
allowClientCredentialsDcr: enableDcrInsecure === true,
dcrTtlMinSeconds,
dcrTtlMaxSeconds,
});
// #1353: loud stderr security WARN when DCR is enabled. DCR is an
@@ -820,6 +855,20 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption
app.use('/register', cors(corsOAuthOptions));
app.use('/revoke', cors(corsOAuthOptions));
// #2179: capture the optional `token_ttl_seconds` DCR extension field
// BEFORE the SDK's /register handler runs — its request schema strips
// unknown body members, so the value would never reach registerClient.
// The rest of the chain runs inside dcrRegistrationContext; the clients
// store clamps + persists it. Malformed values are ignored (fail-safe:
// absent → server default; out-of-range → clamped downstream; a TTL hint
// never rejects a registration). express.json() here is idempotent with
// the SDK router's own body parser.
app.use('/register', express.json(), (req: Request, _res: Response, next: NextFunction) => {
const raw = (req.body as Record<string, unknown> | null | undefined)?.token_ttl_seconds;
const tokenTtlSeconds = typeof raw === 'number' && Number.isFinite(raw) ? raw : undefined;
dcrRegistrationContext.run({ tokenTtlSeconds }, next);
});
// ---------------------------------------------------------------------------
// Custom client_credentials handler (before mcpAuthRouter)
// SDK's token handler only supports authorization_code and refresh_token
+9 -2
View File
@@ -57,6 +57,7 @@ import {
parseSourceConfig,
normalizeSourceConfig,
isSourceFederated,
sourceFederationState,
type SourceRow as LoadedSourceRow,
} from '../core/sources-load.ts';
@@ -470,8 +471,14 @@ async function runList(engine: BrainEngine, args: string[]): Promise<void> {
// Human-readable table.
console.log('SOURCES');
console.log('───────');
for (const e of entries) {
const fedMark = e.federated ? 'federated' : (e as any).archived ? '⚠ archived' : 'isolated';
for (let i = 0; i < entries.length; i++) {
const e = entries[i];
// Explicit `federated: false` (`sources unfederate`) fully isolates a
// source's reads in both directions; an absent key ('unset') only keeps
// it out of OTHER anchors' reads — its own unqualified reads still widen
// outward (see sourceFederationState). Collapsing both to "isolated"
// overstates what an unset flag does.
const fedMark = (e as any).archived ? '⚠ archived' : sourceFederationState(rows[i].config);
const pathStr = e.local_path ?? '(no local path)';
const sync = e.last_sync_at ? `last sync ${e.last_sync_at}` : 'never synced';
console.log(` ${e.id.padEnd(20)} ${fedMark.padEnd(12)} ${String(e.page_count).padStart(6)} pages ${sync}`);
+12 -14
View File
@@ -15,7 +15,7 @@
* - Workers supervisor health from the audit JSONL
* - Queue live minion_jobs counts BY status (NO time window
* old stuck jobs are exactly what status surfaces)
* - Autopilot daemon PID liveness via kill -0 probe
* - Autopilot daemon PID liveness plus gbrain-autopilot identity probe
*
* Exit codes (kubectl-style):
* 0 snapshot produced successfully (even if it carries warnings)
@@ -40,6 +40,10 @@ import { existsSync, readFileSync } from 'node:fs';
import { gbrainPath, loadConfig, isThinClient } from '../core/config.ts';
import { callRemoteTool, unpackToolResult } from '../core/mcp-client.ts';
import { VERSION } from '../version.ts';
import {
classifyAutopilotLockHolder,
type AutopilotLockProbeDeps,
} from '../core/autopilot-lock.ts';
import {
buildSyncStatusReport,
type SyncStatusReport,
@@ -297,8 +301,10 @@ function buildWorkerSummary(): WorkerSummary {
return { crashes_24h, clean_exits_24h, by_cause, last_event_ts };
}
function buildAutopilotStatus(): AutopilotStatus {
const lockPath = gbrainPath('autopilot.lock');
export function buildAutopilotStatus(
lockPath: string = gbrainPath('autopilot.lock'),
deps: AutopilotLockProbeDeps = {},
): AutopilotStatus {
const lockfile_present = existsSync(lockPath);
let pid: number | null = null;
let running = false;
@@ -308,16 +314,8 @@ function buildAutopilotStatus(): AutopilotStatus {
const parsed = parseInt(raw, 10);
if (Number.isFinite(parsed) && parsed > 0) {
pid = parsed;
try {
// kill -0 probes liveness without sending a real signal. Throws ESRCH
// if the PID is gone, EPERM if alive but owned by another user (which
// still tells us "something with that PID exists").
process.kill(parsed, 0);
running = true;
} catch (err) {
const code = (err as NodeJS.ErrnoException).code;
running = code === 'EPERM';
}
const holder = classifyAutopilotLockHolder(parsed, process.pid, deps);
running = holder.state === 'alive-autopilot' || holder.state === 'alive-unknown';
}
} catch {
/* unreadable lockfile, leave pid=null/running=false */
@@ -592,7 +590,7 @@ function renderHuman(report: StatusReport): string {
if (a.running) {
lines.push(` running (PID ${a.pid})`);
} else if (a.lockfile_present) {
lines.push(` stale lockfile (PID ${a.pid ?? '?'} not alive). Run \`gbrain autopilot --install\` to restart.`);
lines.push(` stale lockfile (PID ${a.pid ?? '?'} is not a live autopilot process). Run \`gbrain autopilot --install\` to restart.`);
} else {
lines.push(' not running. Install with `gbrain autopilot --install`.');
}
+61 -3
View File
@@ -3647,10 +3647,68 @@ async function performSyncInner(engine: BrainEngine, opts: SyncOpts): Promise<Sy
// covered regardless.
const extractOpts = opts.sourceId ? { sourceId: opts.sourceId } : undefined;
if (!opts.noExtract && totalChanges > 100 && pagesAffected.length > 0) {
// #2849: above the size gate the deferred extraction must be DURABLY
// QUEUED, not just hinted. The autopilot cycle's extract phase is
// slug-scoped (an up_to_date follow-up sync hands it an empty
// pagesAffected), so a webhook-driven large sync left
// `links_extracted_at` unstamped FOREVER unless an operator ran
// `gbrain extract --stale` by hand. Submit a source-scoped stale-sweep
// job bound to the consumed commit (idempotency key) so repeated
// webhook deliveries / sync retries of the same commit coalesce onto
// one job. The sweep itself is the watermark scan — it picks up the
// pages this run imported AND any banked across resumed runs.
// Best-effort: queue submission failure falls back to the hint-only
// behavior (the pages stay stale + visible to doctor, never mis-stamped).
let queuedJobId: number | string | null = null;
try {
const { MinionQueue } = await import('../core/minions/queue.ts');
const { STALE_TIME_BUDGET_MS } = await import('./extract.ts');
const queue = new MinionQueue(engine);
const payload = {
stale: true,
...(opts.sourceId ? { sourceId: opts.sourceId } : {}),
reason: 'sync_size_gate',
// Bound to the PIN this run drained to (== headCommit unless resuming
// a stored target), not live HEAD — the sweep covers what we imported.
deferred_commit: pin,
};
// The stale sweep has its own internal wall-clock budget
// (GBRAIN_EXTRACT_TIME_BUDGET_MS-derived); without an explicit
// timeout_ms the job would inherit the tight null-default and get
// wall-clock-killed mid-sweep (#1737 class). 5-min headroom.
const timeoutMs = STALE_TIME_BUDGET_MS + 5 * 60 * 1000;
// NO maxWaiting here: with an unscoped (NULL-sourceId) payload the
// queue's coalesce filter matches ANY waiting 'extract' job (e.g. a
// remediation-submitted {mode:'links'} row) and returns THAT job —
// silently dropping the sweep while we log "queued". The idempotency
// key alone is the dedup for repeat submissions toward the same pin.
const key = `extract-stale:${opts.sourceId ?? 'default'}:${pin}`;
const isLiveSweep = (j: { status: string; data: Record<string, unknown> }): boolean =>
j.data?.stale === true && ['waiting', 'delayed', 'active'].includes(j.status);
let job = await queue.add('extract', payload, { idempotency_key: key, timeout_ms: timeoutMs });
if (!isLiveSweep(job)) {
// The key slot holds a FINISHED row: a prior sweep toward this pin
// that completed BEFORE this run's pages landed (checkpoint-resume /
// blocked-advance re-sync of the same target). Those pages went
// stale after that sweep's watermark pass, so coalescing onto the
// finished row would strand them — queue a fresh sweep under a
// run-unique key. (An 'active' sweep is safe to coalesce onto: its
// end-of-run staleRemaining re-count chains a continuation.)
job = await queue.add('extract', payload, {
idempotency_key: `${key}:${Date.now()}`,
timeout_ms: timeoutMs,
});
}
// Only claim "queued" once we verified the returned row IS a live
// stale sweep — never trust queue.add's row blind.
if (isLiveSweep(job)) queuedJobId = job.id;
} catch { /* best-effort — hint below still tells the operator */ }
slog(
` Large sync: deferring link/timeline extraction. ` +
`Run 'gbrain extract --stale${opts.sourceId ? ` --source-id ${opts.sourceId}` : ''}' ` +
`(or let the autopilot cycle's extract phase sweep it).`,
` Large sync: deferring link/timeline extraction` +
(queuedJobId != null
? ` — queued stale-sweep job #${queuedJobId} (source: ${opts.sourceId ?? 'default'}); a running jobs worker will consume it.`
: `.`) +
` Run 'gbrain extract --stale${opts.sourceId ? ` --source-id ${opts.sourceId}` : ''}' to extract now.`,
);
}
if (!opts.noExtract && totalChanges <= 100 && pagesAffected.length > 0) {
+11 -2
View File
@@ -624,12 +624,13 @@ async function cmdExtract(engine: BrainEngine, rest: string[]): Promise<void> {
const sub = rest[0];
if (sub !== '--from-pages') {
process.stderr.write(
'Usage: gbrain takes extract --from-pages [--yes] [--dry-run] [--source-id <id>] [--max-pages N (clamped to 1000)] [--include-covered] [--holder <name>]\n' +
'Usage: gbrain takes extract --from-pages [--yes] [--dry-run] [--json] [--source-id <id>] [--max-pages N (clamped to 1000)] [--include-covered] [--holder <name>]\n' +
'Runs progress: pages that already hold takes are skipped, so repeat runs sweep a large corpus in slices. --include-covered rescans everything (refresh).\n',
);
process.exit(1);
}
const dryRun = rest.includes('--dry-run');
const json = rest.includes('--json');
const skipConfirm = rest.includes('--yes');
const sourceIdx = rest.indexOf('--source-id');
const sourceIdFilter = sourceIdx >= 0 ? rest[sourceIdx + 1] : undefined;
@@ -667,9 +668,17 @@ async function cmdExtract(engine: BrainEngine, rest: string[]): Promise<void> {
holder,
});
if (result.llm_unavailable) {
process.stderr.write(`[takes extract] chat gateway unavailable (no API key configured).\n`);
if (json) {
process.stdout.write(`${JSON.stringify(result, null, 2)}\n`);
} else {
process.stderr.write(`[takes extract] chat gateway unavailable (no API key configured).\n`);
}
process.exit(2);
}
if (json) {
process.stdout.write(`${JSON.stringify(result, null, 2)}\n`);
return;
}
process.stdout.write(
`takes extract --from-pages: ${result.claims_extracted} claim(s) from ${result.pages_scanned} page(s)` +
(dryRun ? ' (dry-run)' : '') + '\n',
+42 -11
View File
@@ -473,19 +473,39 @@ export async function runPostUpgrade(args: string[] = []): Promise<void> {
// `ze_sunset_notice_shown` (same pattern as the search-mode banner).
try {
const shown = await engine.getConfig('ze_sunset_notice_shown');
const { DEFAULT_EMBEDDING_MODEL } = await import('../core/ai/defaults.ts');
const { DEFAULT_EMBEDDING_MODEL, ZEROENTROPY_SUNSET_DATE } = await import('../core/ai/defaults.ts');
const effectiveModel = cfgSchema.embedding_model ?? DEFAULT_EMBEDDING_MODEL;
const rerankerModel = await engine.getConfig('search.reranker.model');
// Effective reranker via the plane search actually reranks with
// (mode bundle + search.reranker.* overrides) — the bare config
// key is unset by default while balanced/tokenmax rerank with the
// bundle's zeroentropyai model. Same resolution as the
// provider_sunset doctor check.
let rerankerModel: string | undefined;
try {
const { loadSearchModeConfig, resolveSearchMode } = await import('../core/search/mode.ts');
const knobs = resolveSearchMode(await loadSearchModeConfig(engine));
if (knobs.reranker_enabled) rerankerModel = knobs.reranker_model;
} catch { /* no reranker-exposure claim */ }
const onZeEmbedding = effectiveModel.startsWith('zeroentropyai:');
const onZeReranker = !!rerankerModel?.startsWith('zeroentropyai:');
if (shown !== 'true' && (onZeEmbedding || onZeReranker)) {
// Paste-ready --dim from the ACTUAL column width (config can
// drift): keeping the current width avoids a needless dimension
// transition + index rebuild when the target supports it.
let colDims: number | null = null;
try {
const { readContentChunksEmbeddingDim } = await import('../core/embedding-dim-check.ts');
colDims = (await readContentChunksEmbeddingDim(engine)).dims;
} catch { /* fresh brain — omit --dim */ }
const dimFlag = colDims ? ` --dim ${colDims}` : '';
console.log('');
console.log('═══════════════════════════════════════════════════════════════');
console.log('[gbrain] ACTION REQUIRED: ZeroEntropy hosted API sunsets 2026-09-04.');
console.log(`[gbrain] ACTION REQUIRED: ZeroEntropy hosted API sunsets ${ZEROENTROPY_SUNSET_DATE}.`);
if (onZeEmbedding) {
console.log(`[gbrain] This brain embeds with ${effectiveModel}. After the sunset,`);
console.log('[gbrain] semantic retrieval STOPS WORKING (queries can no longer be');
console.log('[gbrain] embedded against your existing vectors).');
console.log('[gbrain] semantic retrieval STOPS WORKING entirely — your EXISTING');
console.log('[gbrain] vectors become unqueryable (queries embed through the same');
console.log('[gbrain] endpoint), not just new content.');
}
if (onZeReranker) {
console.log(`[gbrain] The reranker (${rerankerModel}) also sunsets; search falls`);
@@ -493,17 +513,28 @@ export async function runPostUpgrade(args: string[] = []): Promise<void> {
}
console.log('═══════════════════════════════════════════════════════════════');
console.log('');
console.log('Migrate before the sunset (resumable; preview cost first):');
console.log(' gbrain migrate embeddings --to <provider:model> --dry-run');
console.log(' gbrain migrate embeddings --to <provider:model>');
console.log('Two fixes, either works:');
console.log('');
console.log('Self-hosting zembed-1 (weights are Apache-2.0) via llama-server /');
console.log('ollama also works and preserves your existing vectors — point');
console.log('embedding at the local endpoint instead of migrating.');
console.log('[1] Self-host the same model — zembed-1 weights are Apache-2.0. Serve');
console.log(' them via llama-server or Ollama and point the config at the local');
console.log(' endpoint. Keeps every existing vector; NO re-embed at all. See');
console.log(' docs/guides/embedding-migration.md ("Self-hosting instead of migrating").');
console.log('');
console.log('[2] Migrate to another provider (resumable; preview cost first):');
console.log(` gbrain migrate embeddings --to <provider:model>${dimFlag} --dry-run`);
console.log(` gbrain migrate embeddings --to <provider:model>${dimFlag}`);
if (colDims) {
console.log(` (--dim ${colDims} is this brain's current index width — keep it to`);
console.log(' avoid a needless schema rebuild when the target supports it.)');
}
if (onZeReranker) {
console.log('');
console.log('Reranker: gbrain config set search.reranker.enabled false (or pick another).');
}
console.log('');
console.log(`\`gbrain doctor\` will keep flagging this until the brain is off the`);
console.log('provider (check name: provider_sunset).');
console.log('');
await engine.setConfig('ze_sunset_notice_shown', 'true');
}
} catch {
+11
View File
@@ -19,3 +19,14 @@
// steps: {2560, 1280, 640, 320, 160, 80, 40} — see ai/dims.ts.
export const DEFAULT_EMBEDDING_MODEL = 'zeroentropyai:zembed-1';
export const DEFAULT_EMBEDDING_DIMENSIONS = 1280;
/**
* ZeroEntropy announced (2026-07-24) that its hosted API including
* /models/embed and /models/rerank shuts down on this date. Query
* embedding uses the same endpoint as ingestion, so a brain still on a
* `zeroentropyai:*` embedding model loses semantic retrieval ENTIRELY on
* that date (existing vectors become unqueryable, not just new content).
* Single source of truth for the upgrade banner + the `provider_sunset`
* doctor check. Self-hosting the Apache-2.0 zembed-1 weights is unaffected.
*/
export const ZEROENTROPY_SUNSET_DATE = '2026-09-04';
+4
View File
@@ -35,7 +35,11 @@ export const claudeCli: Recipe = {
// No embedding or expansion touchpoints — chat-only.
chat: {
models: [
'claude-fable-5',
'claude-opus-5',
'claude-opus-4-8',
'claude-opus-4-7',
'claude-sonnet-5',
'claude-sonnet-4-6',
'claude-haiku-4-5-20251001',
],
+64
View File
@@ -0,0 +1,64 @@
import { execFileSync } from 'node:child_process';
export type AutopilotLockHolder =
| { state: 'dead' }
| { state: 'self' }
| { state: 'alive-autopilot' }
| { state: 'alive-foreign' }
| { state: 'alive-unknown' };
export interface AutopilotLockProbeDeps {
isPidAlive?: (pid: number) => boolean;
readProcessCommand?: (pid: number) => string | null;
}
export function isPidAlive(pid: number): boolean {
if (!Number.isFinite(pid) || pid <= 0) return false;
try {
process.kill(pid, 0);
return true;
} catch (error: unknown) {
return (error as NodeJS.ErrnoException).code === 'EPERM';
}
}
export function readProcessCommand(pid: number): string | null {
if (!Number.isFinite(pid) || pid <= 0) return null;
try {
const out = execFileSync('ps', ['-p', String(pid), '-o', 'args='], {
encoding: 'utf8',
stdio: ['ignore', 'pipe', 'ignore'],
timeout: 1000,
}).trim();
return out.length > 0 ? out : null;
} catch {
return null;
}
}
export function looksLikeGbrainAutopilotCommand(command: string): boolean {
const normalized = command.replace(/\\/g, '/').trim();
if (!/(^|\s)autopilot(\s|$)/i.test(normalized)) return false;
if (/(^|[\/\s])gbrain(?:\.exe)?(\s|$)/i.test(normalized)) return true;
return /(^|\s)(?:\S+\/)?(?:\.{1,2}\/)?(?:src\/)?cli\.(?:ts|js|mjs)(\s|$)/i.test(normalized)
|| /(^|\s)\S*\/src\/cli\.(?:ts|js|mjs)(\s|$)/i.test(normalized);
}
export function classifyAutopilotLockHolder(
pid: number,
currentPid: number = process.pid,
deps: AutopilotLockProbeDeps = {},
): AutopilotLockHolder {
if (!Number.isFinite(pid) || pid <= 0) return { state: 'dead' };
if (pid === currentPid) return { state: 'self' };
const probeAlive = deps.isPidAlive ?? isPidAlive;
if (!probeAlive(pid)) return { state: 'dead' };
const probeCommand = deps.readProcessCommand ?? readProcessCommand;
const command = probeCommand(pid);
if (command === null) return { state: 'alive-unknown' };
return looksLikeGbrainAutopilotCommand(command)
? { state: 'alive-autopilot' }
: { state: 'alive-foreign' };
}
+38 -1
View File
@@ -133,6 +133,28 @@ function isRateLimitOr5xx(stderr: string): boolean {
return /HTTP 5\d\d|HTTP 429|rate limit/i.test(stderr);
}
/** `gh auth status`'s `--active` flag (added in cli/cli v2.57.0, 2024-09-11)
* scopes the check to only the active account instead of aggregating every
* registered account. On an older `gh`, passing an unrecognized flag makes
* the WHOLE command fail so Gate 2 must detect support before using it. */
const GH_ACTIVE_FLAG_MIN_VERSION = [2, 57, 0] as const;
/** Parses the `X.Y.Z` out of `gh --version`'s first line (`gh version X.Y.Z (DATE)`).
* Returns null on any unrecognized format callers treat that as "unknown,
* don't assume support". */
function parseGhVersion(versionOutput: string): readonly [number, number, number] | null {
const m = /\bgh version (\d+)\.(\d+)\.(\d+)/.exec(versionOutput);
if (!m) return null;
return [Number(m[1]), Number(m[2]), Number(m[3])];
}
function ghVersionAtLeast(v: readonly [number, number, number], min: readonly [number, number, number]): boolean {
for (let i = 0; i < 3; i++) {
if (v[i] !== min[i]) return v[i]! > min[i]!;
}
return true;
}
/** The authenticated gh login, or null when it cannot be read/parsed. */
async function fetchAuthedLogin(runner: ExecRunner): Promise<string | null> {
const res = await runner(['gh', 'api', 'user']);
@@ -594,7 +616,22 @@ export async function createPrivateRepo(
}
// Gate 2: authenticated. Exit-code-2 — the human runs `gh auth login`.
const ghAuth = await runner(['gh', 'auth', 'status']);
// `--hostname github.com` scopes the check to the host this flow actually
// targets (every downstream call — parseGithubOwnerRepo, the repo-create
// URL fallback, etc. — is github.com-only), so an unrelated broken account
// on some other configured host (e.g. a GitHub Enterprise instance) can't
// false-block it either. `--active` (only when the installed `gh` supports
// it) further restricts that host's check to the active account. Bare
// `gh auth status` aggregates EVERY registered account on EVERY host and
// exits 1 if any one of them is invalid — even an unused, long-expired
// account — which false-blocks this gate while the active account (what
// `gh`/`git` actually use) is perfectly healthy.
const ghVersionTuple = parseGhVersion(ghVersion.stdout);
const ghSupportsActiveFlag = ghVersionTuple !== null && ghVersionAtLeast(ghVersionTuple, GH_ACTIVE_FLAG_MIN_VERSION);
const ghAuthArgv = ghSupportsActiveFlag
? ['gh', 'auth', 'status', '--active', '--hostname', 'github.com']
: ['gh', 'auth', 'status', '--hostname', 'github.com'];
const ghAuth = await runner(ghAuthArgv);
if (ghAuth.code !== 0) {
throw new BootstrapError(
'GH_AUTH',
+23 -7
View File
@@ -484,16 +484,33 @@ function checkMcpSurface(): VerifyCheck {
}
}
/**
* Pure, engine-free derivation of the collision-fallback source_id for a
* workspace a deterministic hash of the workspace's real path, no DB
* lookup involved. `resolveSourceIdCollision` (below) is the only thing that
* decides WHETHER this id is actually needed (that half requires the engine,
* since the sources registry lives only in the DB) but the id itself is
* safe to preview from an engine-free phase. `bootstrap hooks` does exactly
* that, so a human has the fallback id in hand before they ever hand-register
* a source, instead of discovering it only after an FK error + a corrective
* `verify` run.
*/
export function deriveWorkspaceSourceId(ws: string): string {
const hash = createHash('sha256').update(realpathOrResolve(ws)).digest('hex').slice(0, 8);
return `workspace-${hash}`;
}
/**
* source_id collision resolution [engine seam]. Render is ENGINE-FREE and the
* sources registry lives ONLY in the DB (no registry file exists), so verify
* the one bootstrap subcommand holding an engine is where a manifest
* source_id already registered to a DIFFERENT checkout is detected. On
* collision it derives a stable `workspace-<8char-path-hash>` id, persists it
* to agent.json (render preserves it on re-render), and names the re-register
* steps; every consumer (hooks GBRAIN_SOURCE env, verify, status hints,
* attach, repo persistence) reads manifest.source_id, so the derived id
* propagates. Returns a sourceId ONLY when it derived one.
* collision it derives a stable `workspace-<8char-path-hash>` id (via
* `deriveWorkspaceSourceId`), persists it to agent.json (render preserves it
* on re-render), and names the re-register steps; every consumer (hooks
* GBRAIN_SOURCE env, verify, status hints, attach, repo persistence) reads
* manifest.source_id, so the derived id propagates. Returns a sourceId ONLY
* when it derived one.
*/
async function resolveSourceIdCollision(
engine: BrainEngine,
@@ -513,8 +530,7 @@ async function resolveSourceIdCollision(
if (realpathOrResolve(registered) === realpathOrResolve(brainDir)) {
return { sourceId: null, check: null }; // same checkout — no collision
}
const hash = createHash('sha256').update(realpathOrResolve(ws)).digest('hex').slice(0, 8);
const derived = `workspace-${hash}`;
const derived = deriveWorkspaceSourceId(ws);
writeManifest(ws, { ...state.manifest, source_id: derived });
return {
sourceId: derived,
+78 -8
View File
@@ -40,8 +40,9 @@ import { findResolverFile, RESOLVER_FILENAMES } from './resolver-filenames.ts';
import { redactSecretsInText } from './minions/handlers/shell-redact.ts';
import { ensureGbrainHome, resolveGbrainHome } from './gbrain-home.ts';
import { binaryOnPath } from './execution-env.ts';
// Static import → bundled into the --compile binary so the taxonomy never drifts
// and needs no runtime skills/ directory.
import { loadFilingRules, type FilingRulesDoc } from './filing-audit.ts';
// Bundled into the --compile binary as the fallback taxonomy for repos that
// don't ship their own — see resolveFilingRules().
import filingRulesDoc from '../../skills/_brain-filing-rules.json';
// ── Types ───────────────────────────────────────────────────────────────────
@@ -245,10 +246,26 @@ exit 4
// ── Managed AGENTS/RESOLVER block (taxonomy from filing rules; no drift) ─────
function renderTaxonomyLines(): string {
/**
* Resolve the filing-rules taxonomy for `repoPath`: prefer the repo's own
* `skills/_brain-filing-rules.json`, then `_brain-filing-rules.json` at the
* repo root, else the bundled default. Fails open a missing or malformed
* repo file must never break `sources harden`.
*/
function resolveFilingRules(repoPath: string): FilingRulesDoc {
for (const dir of [join(repoPath, 'skills'), repoPath]) {
try {
const rules = loadFilingRules(dir);
if (rules) return rules;
} catch { /* malformed — fall through to the bundled default */ }
}
return filingRulesDoc as FilingRulesDoc;
}
function renderTaxonomyLines(rules: FilingRulesDoc): string {
const seen = new Set<string>();
const lines: string[] = [];
for (const r of (filingRulesDoc as any).rules ?? []) {
for (const r of rules.rules ?? []) {
const dir = String(r.directory || '').trim();
if (!dir || seen.has(dir)) continue;
seen.add(dir);
@@ -257,7 +274,8 @@ function renderTaxonomyLines(): string {
return lines.join('\n');
}
function renderManagedBlock(): string {
function renderManagedBlock(repoPath: string): string {
const rules = resolveFilingRules(repoPath);
return `${AGENTS_BEGIN}
<!-- gbrain durability rules. This block is regenerated by \`gbrain sources harden\`.
Do not index as user knowledge; do not edit between the markers. -->
@@ -265,7 +283,7 @@ function renderManagedBlock(): string {
1. **Deterministic filing never use /tmp as storage.** Every persistent output
goes to its taxonomy path (canonical, from \`skills/_brain-filing-rules.json\`):
${renderTaxonomyLines()}
${renderTaxonomyLines(rules)}
Writing to /tmp, scratch dirs, or outside the repo is forbidden for anything
meant to persist.
@@ -285,7 +303,7 @@ ${AGENTS_END}`;
function patchResolverFile(repoPath: string, dryRun: boolean): { status: StepStatus; detail: string } {
const existing = findResolverFile(repoPath);
const target = existing ?? join(repoPath, RESOLVER_FILENAMES[1]); // default AGENTS.md
const block = renderManagedBlock();
const block = renderManagedBlock(repoPath);
const name = relative(repoPath, target) || target;
let current = '';
@@ -432,6 +450,58 @@ export function commitWriteThroughFile(repoPath: string, absPath: string, slug:
}
}
// ── Push-state query (D14) ───────────────────────────────────────────────────
export type PushLogStatus = 'ok' | 'needs_attention' | 'unknown';
export interface PushLogOutcome {
status: PushLogStatus;
detail: string;
/** UTC timestamp parsed from the log line, when found. */
at?: string;
}
const PUSH_LOG_OK = /^(\S+) \[push\] (?:ok|ok-after-rebase) (\S+)\b/;
const PUSH_LOG_LOCAL_ONLY = /^(\S+) \[push\] LOCAL-ONLY, NEEDS ATTENTION: (\S+) /;
const PUSH_LOG_LOCK_TIMEOUT = /^(\S+) \[push\] lock-timeout (\S+)\b/;
/**
* Best-effort read of the most recently logged push outcome for `branch`,
* from the shared hook log ($GBRAIN_HOME/brain-push.log). The push itself
* runs detached in the background (see `renderPostCommitHook`), so nothing
* synchronous ever learns whether a given commit's own push landed this is
* the queryable substitute: "as of the last thing the hook logged for this
* branch, were pushes landing?"
*
* The log is host-wide and keyed only by branch name, not repo path, so two
* different hardened repos sharing a branch name (e.g. both on `main`) share
* this signal. That's an acceptable approximation for a liveness check, not
* a per-repo guarantee.
*/
export function getLastPushOutcome(branch: string): PushLogOutcome {
const log = pushLogPath();
if (!existsSync(log)) return { status: 'unknown', detail: 'no push attempts logged yet' };
let lines: string[];
try {
lines = readFileSync(log, 'utf-8').split('\n');
} catch (e) {
return { status: 'unknown', detail: `push log unreadable: ${(e as Error).message}` };
}
for (let i = lines.length - 1; i >= 0; i--) {
const line = lines[i];
if (!line) continue;
let m = line.match(PUSH_LOG_OK);
if (m && m[2] === branch) return { status: 'ok', detail: line.trim(), at: m[1] };
m = line.match(PUSH_LOG_LOCAL_ONLY);
if (m && m[2] === branch) return { status: 'needs_attention', detail: line.trim(), at: m[1] };
m = line.match(PUSH_LOG_LOCK_TIMEOUT);
if (m && m[2] === branch) return { status: 'needs_attention', detail: line.trim(), at: m[1] };
}
return { status: 'unknown', detail: `no push attempt logged yet for branch '${branch}'` };
}
// ── Committed helper ────────────────────────────────────────────────────────
function installHelper(repoPath: string, dryRun: boolean): { status: StepStatus; detail: string } {
@@ -766,7 +836,7 @@ function resolveRepoRoot(path: string): string {
}
}
function currentBranch(repoPath: string): string {
export function currentBranch(repoPath: string): string {
try {
return execFileSync('git', ['-C', repoPath, 'rev-parse', '--abbrev-ref', 'HEAD'], {
stdio: ['ignore', 'pipe', 'ignore'], timeout: 10_000, env: { ...process.env, ...GIT_ENV },
+1 -1
View File
@@ -332,7 +332,7 @@ export class BudgetTracker {
// pricing we can't enforce the cap, and silently ignoring it would
// void the contract.
const msg = `${this.opts.label}: no pricing entry for model "${estimate.modelId}" (kind=${estimate.kind}). ` +
`Add it to src/core/${estimate.kind === 'embed' ? 'embedding-pricing.ts' : 'anthropic-pricing.ts'} or drop --max-cost.`;
`Add it to src/core/${estimate.kind === 'embed' || estimate.kind === 'rerank' ? 'embedding-pricing.ts' : 'anthropic-pricing.ts'} or drop --max-cost.`;
this.fireExhausted();
throw new BudgetExhausted(msg, {
reason: 'no_pricing',
+5 -17
View File
@@ -26,6 +26,7 @@ import {
findPrimaryResolverPath,
loadSkillTriggerIndex,
} from './skill-trigger-index.ts';
import { parseSkillFrontmatter } from './skill-frontmatter.ts';
// ---------------------------------------------------------------------------
// Types
@@ -218,26 +219,13 @@ export function parseResolverEntries(resolverContent: string): ResolverEntry[] {
// needed for AGENTS.md-only OpenClaw deployments. See D-CX-12 / F-ENG-1.
/**
* Simple YAML frontmatter parser extracts triggers array if present.
* Extract the triggers array through the shared SKILL.md frontmatter parser.
*
* Normalizes CRLF LF before parsing so Windows checkouts (where
* `core.autocrlf=true` is the default) parse correctly. Without this,
* the `^---\n` and `^triggers:\s*\n` regexes never match because the
* file content is `---\r\n` / `triggers:\r\n`, and every skill on
* Windows is reported as `mece_gap` regardless of its actual content.
* CI runs on Ubuntu-only so the bug only surfaces in user environments.
* Keeping this compatibility export routed through `parseSkillFrontmatter`
* prevents doctor gap detection from drifting from the trigger index.
*/
export function extractTriggers(skillContent: string): string[] {
const content = skillContent.replace(/\r\n/g, '\n');
const fmMatch = content.match(/^---\n([\s\S]*?)\n---/);
if (!fmMatch) return [];
const fm = fmMatch[1];
const triggersMatch = fm.match(/^triggers:\s*\n((?:\s+-\s+.+\n?)*)/m);
if (!triggersMatch) return [];
return triggersMatch[1]
.split('\n')
.map(l => l.replace(/^\s+-\s+/, '').replace(/^["']|["']$/g, '').trim())
.filter(Boolean);
return parseSkillFrontmatter(skillContent)?.triggers ?? [];
}
/**
+58 -58
View File
@@ -8,102 +8,102 @@
// (help-text mentions count): accepting an ignored flag is the pre-#2185
// status quo; missing a real one breaks working invocations.
export const CLI_FLAG_REGISTRY: Record<string, readonly string[]> = {
'advisor': ['--aliases', '--all', '--apply', '--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--skills-dir', '--source', '--stale', '--supersessions', '--surface', '--thin', '--verbose', '--workspace', '--yes'],
'agent': ['--aliases', '--all', '--brain', '--detach', '--fanout-manifest', '--flag', '--flags', '--follow', '--help', '--include-null-signature', '--json', '--max-turns', '--model', '--no-extract', '--no-follow', '--note', '--pattern', '--pending', '--reset', '--resolve', '--since', '--source', '--stale', '--subagent-def', '--supersessions', '--thin', '--timeout-ms', '--tools', '--word'],
'anomalies': ['--aliases', '--all', '--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--lookback-days', '--mcp-only', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--sigma', '--since', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout'],
'apply-migrations': ['--ab', '--all', '--auto-update', '--brain', '--break-lock', '--build-index', '--by-mention', '--compile', '--days', '--dry-run', '--exclusive', '--fast', '--force', '--force-all', '--force-orchestrator', '--force-retry', '--force-schema', '--from-meetings', '--from-pages', '--help', '--history', '--host-dir', '--http', '--json', '--lang', '--list', '--locks', '--markdown', '--max-age', '--migrate-only', '--migration', '--mode', '--multimodal', '--no-autopilot-install', '--no-embedding', '--no-extract', '--non-interactive', '--phase', '--priority', '--refresh-unqualified', '--remediate', '--rollback', '--skip-verify', '--source', '--stale', '--surface', '--undo-wave', '--use-captured-snapshot', '--with-calibration', '--yes'],
'auth': ['--aliases', '--all', '--bound-brain', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--enable-dcr', '--enable-dcr-insecure', '--fast', '--federated-read', '--force', '--from-pages', '--grant-types', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--pattern', '--pending', '--redirect-uri', '--reset', '--resolve', '--scopes', '--source', '--stale', '--supersessions', '--surface', '--takes-holders', '--thin', '--token', '--token-endpoint-auth-method', '--yes'],
'autopilot': ['--aliases', '--all', '--auto-fix', '--batch', '--brain', '--break-lock', '--by-type', '--check', '--dimensions', '--dir', '--drain', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--ff-only', '--fix', '--force', '--force-break-lock', '--force-retry', '--from-pages', '--help', '--http', '--include-null-signature', '--inject-bootstrap', '--inline', '--input', '--install', '--interval', '--json', '--markdown', '--max-age', '--max-rss', '--max-usd', '--migrate-only', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--no-extract', '--no-inject', '--no-mutate', '--no-worker', '--non-interactive', '--now', '--once', '--output', '--path', '--pattern', '--pending', '--phase', '--refresh-cache', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--source', '--stale', '--status', '--supersessions', '--surface', '--swap-only', '--target', '--target-score', '--thin', '--timeout', '--to', '--uninstall', '--unsafe-bypass-dream-guard', '--user', '--version', '--yes'],
'advisor': ['--aliases', '--all', '--apply', '--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--skills-dir', '--source', '--stale', '--supersessions', '--surface', '--thin', '--token-ttl', '--verbose', '--workspace', '--yes'],
'agent': ['--aliases', '--all', '--brain', '--detach', '--fanout-manifest', '--federated', '--flag', '--flags', '--follow', '--help', '--include-null-signature', '--json', '--max-turns', '--model', '--no-extract', '--no-federated', '--no-follow', '--note', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--since', '--source', '--stale', '--subagent-def', '--supersessions', '--thin', '--timeout-ms', '--tools', '--word'],
'anomalies': ['--aliases', '--all', '--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--lookback-days', '--mcp-only', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--sigma', '--since', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl'],
'apply-migrations': ['--ab', '--all', '--auto-update', '--brain', '--break-lock', '--build-index', '--by-mention', '--compile', '--days', '--dry-run', '--exclusive', '--fast', '--force', '--force-all', '--force-orchestrator', '--force-retry', '--force-schema', '--from-meetings', '--from-pages', '--help', '--history', '--host-dir', '--http', '--json', '--lang', '--list', '--locks', '--markdown', '--max-age', '--migrate-only', '--migration', '--mode', '--multimodal', '--no-autopilot-install', '--no-embedding', '--no-extract', '--non-interactive', '--phase', '--priority', '--refresh-unqualified', '--remediate', '--rollback', '--skip-verify', '--source', '--stale', '--surface', '--token-ttl', '--undo-wave', '--use-captured-snapshot', '--with-calibration', '--yes'],
'auth': ['--aliases', '--all', '--bound-brain', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--enable-dcr', '--enable-dcr-insecure', '--fast', '--federated-read', '--force', '--from-pages', '--grant-types', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--pattern', '--pending', '--redirect-uri', '--reset', '--resolve', '--scopes', '--source', '--stale', '--supersessions', '--surface', '--takes-holders', '--thin', '--token', '--token-endpoint-auth-method', '--token-ttl', '--yes'],
'autopilot': ['--aliases', '--all', '--auto-fix', '--batch', '--brain', '--break-lock', '--by-type', '--check', '--dim', '--dimensions', '--dir', '--drain', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--ff-only', '--fix', '--force', '--force-break-lock', '--force-retry', '--from-pages', '--help', '--http', '--include-null-signature', '--inject-bootstrap', '--inline', '--input', '--install', '--interval', '--json', '--markdown', '--max-age', '--max-rss', '--max-usd', '--migrate-only', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-inject', '--no-mutate', '--no-worker', '--non-interactive', '--now', '--once', '--output', '--path', '--pattern', '--pending', '--phase', '--refresh-cache', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--source', '--stale', '--status', '--supersessions', '--surface', '--swap-only', '--target', '--target-score', '--thin', '--timeout', '--to', '--token-ttl', '--uninstall', '--unsafe-bypass-dream-guard', '--user', '--version', '--yes'],
'backfill': ['--aliases', '--all', '--batch-size', '--brain', '--concurrency', '--dry-run', '--fresh', '--help', '--include-null-signature', '--json', '--keep-index', '--list', '--max-errors', '--max-rows', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--resume', '--source', '--stale', '--supersessions', '--thin'],
'bench': ['--baseline', '--brain', '--explain', '--force', '--from', '--help', '--json', '--label', '--lang', '--limit', '--markdown', '--multimodal', '--near-symbol', '--restore-only', '--source', '--stale', '--symbol-kind', '--thin', '--threshold-jaccard', '--threshold-latency-multiplier', '--threshold-top1', '--to', '--tool'],
'book-mirror': ['--aliases', '--all', '--allow-empty', '--apply', '--asof', '--author', '--auto', '--background', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--brain-wide-max-cost-usd', '--budget-usd-per-day', '--by-mention', '--chapters-dir', '--content', '--context-file', '--date', '--days', '--dry-run', '--entities', '--explain', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--help', '--http', '--image', '--include-null-signature', '--json', '--kind', '--limit', '--max-turns', '--max-usd', '--mode', '--model', '--multimodal', '--no-confirm', '--no-embedding', '--no-extract', '--no-follow', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--save', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--timeout', '--timeout-ms', '--title', '--trusted-extraction', '--url', '--with-db', '--yes'],
'bootstrap': ['--abbrev-ref', '--abort', '--accept-visibility-change-consequences', '--all', '--allow-unverified-remote', '--brain', '--branch', '--cached', '--compile', '--confirm', '--count', '--delete-brain', '--diff-filter', '--env', '--error-unmatch', '--exclude-standard', '--fast', '--file', '--flag', '--force', '--from-pages', '--full', '--gbrain-bin', '--get', '--git-dir', '--git-path', '--harness', '--heads', '--help', '--home', '--http', '--init', '--is-inside-work-tree', '--isolated', '--jq', '--json', '--local', '--minimal', '--name-only', '--no-cron', '--no-embedding', '--no-hooks', '--no-verify', '--once', '--only', '--others', '--pat-file', '--path', '--pglite', '--porcelain', '--private', '--push', '--push-only', '--quiet', '--rebase', '--repair', '--scope', '--set', '--short', '--show', '--show-toplevel', '--skip', '--source', '--status', '--surface', '--unset-all', '--verify', '--version', '--visibility', '--workspace', '--yes'],
'brainstorm': ['--aliases', '--all', '--brain', '--chunker-debug', '--code', '--compile', '--fast', '--fix', '--force', '--force-rechunk', '--force-resume', '--from-pages', '--full', '--help', '--http', '--include-null-signature', '--json', '--judge-model', '--lang', '--limit', '--list-runs', '--markdown', '--max-cost', '--max-far-set', '--max-ideas-per-judge-call', '--model', '--no-embed', '--no-embedding', '--no-extract', '--no-save', '--pattern', '--pending', '--reset', '--resolve', '--resume', '--retry-failed', '--retry-judge', '--save', '--source', '--stale', '--strict-budget', '--supersessions', '--surface', '--thin', '--timeout', '--yes'],
'cache': ['--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--no-embedding', '--source', '--surface', '--yes'],
'calibration': ['--ab', '--aliases', '--all', '--allow-empty', '--apply', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--dry-run', '--entities', '--explain', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--help', '--holder', '--http', '--image', '--include-null-signature', '--json', '--key-prefix', '--kind', '--lang', '--limit', '--markdown', '--max-usd', '--mode', '--multimodal', '--near-symbol', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--phase', '--progress-interval', '--progress-json', '--quiet', '--regenerate', '--repo', '--reset', '--resolve', '--restore-only', '--save', '--scrub-gstack', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--symbol-kind', '--thin', '--trusted-extraction', '--undo-wave', '--url', '--with-calibration', '--with-db', '--yes'],
'call': ['--aliases', '--all', '--all-sources', '--as-context', '--auto-fix', '--background', '--brain', '--by-mention', '--catch-up', '--concurrency', '--confirm-destructive', '--content', '--cost-estimate', '--count', '--days', '--depth', '--dim', '--dir', '--direction', '--enable-dcr', '--enable-dcr-insecure', '--explain', '--fast', '--federated', '--file', '--fix', '--follow', '--force', '--from', '--from-meetings', '--grant-types', '--grep', '--hard-deadline', '--help', '--http', '--image', '--include-expired', '--include-frontmatter', '--include-null-signature', '--infer-dates', '--install', '--interval', '--json', '--key', '--kind', '--lang', '--limit', '--link-source', '--link-type', '--llm', '--migrate-only', '--missing-path', '--multimodal', '--ner', '--no-embed', '--no-expand', '--no-extract', '--no-federated', '--no-hard-deadline', '--no-retry-connect', '--no-save', '--older-than', '--page', '--param', '--params', '--password', '--path', '--pattern', '--pending', '--pglite', '--port', '--progress-interval', '--progress-json', '--public-url', '--queue', '--quiet', '--reenrich-after', '--refresh-cache', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--restore-only', '--save', '--scopes', '--session', '--sigma', '--since', '--slug', '--slug-prefix', '--source', '--source-id', '--stale', '--status', '--stdin', '--strategy', '--suites', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--synthesize', '--tag', '--thin', '--timeout', '--to', '--today', '--token', '--token-ttl', '--tools-json', '--type', '--uninstall', '--url', '--version', '--watch', '--with-calibration', '--workers', '--yes'],
'capture': ['--aliases', '--all', '--allow-empty', '--apply', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--depth', '--entities', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--help', '--http', '--image', '--include-null-signature', '--json', '--kind', '--limit', '--max-usd', '--mcp-only', '--mode', '--multimodal', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--repo', '--reset', '--resolve', '--restore-only', '--save', '--scopes', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--stdin', '--supersessions', '--surface', '--thin', '--timeout', '--trusted-extraction', '--type', '--url', '--what', '--where', '--who', '--with-db', '--yes'],
'book-mirror': ['--aliases', '--all', '--allow-empty', '--apply', '--asof', '--author', '--auto', '--background', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--brain-wide-max-cost-usd', '--budget-usd-per-day', '--by-mention', '--chapters-dir', '--content', '--context-file', '--date', '--days', '--dry-run', '--entities', '--explain', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--help', '--http', '--image', '--include-null-signature', '--json', '--kind', '--limit', '--max-turns', '--max-usd', '--mode', '--model', '--multimodal', '--no-confirm', '--no-embedding', '--no-extract', '--no-follow', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--save', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--timeout', '--timeout-ms', '--title', '--token-ttl', '--trusted-extraction', '--url', '--with-db', '--yes'],
'bootstrap': ['--abbrev-ref', '--abort', '--accept-visibility-change-consequences', '--active', '--all', '--allow-unverified-remote', '--brain', '--branch', '--cached', '--compile', '--confirm', '--count', '--delete-brain', '--diff-filter', '--env', '--error-unmatch', '--exclude-standard', '--fast', '--file', '--flag', '--force', '--from-pages', '--full', '--gbrain-bin', '--get', '--git-dir', '--git-path', '--harness', '--heads', '--help', '--home', '--hostname', '--http', '--init', '--is-inside-work-tree', '--isolated', '--jq', '--json', '--local', '--minimal', '--name-only', '--no-cron', '--no-embedding', '--no-hooks', '--no-verify', '--once', '--only', '--others', '--pat-file', '--path', '--pglite', '--porcelain', '--private', '--push', '--push-only', '--quiet', '--rebase', '--repair', '--scope', '--set', '--short', '--show', '--show-toplevel', '--skip', '--source', '--status', '--surface', '--token-ttl', '--unset-all', '--verify', '--version', '--visibility', '--workspace', '--yes'],
'brainstorm': ['--aliases', '--all', '--brain', '--chunker-debug', '--code', '--compile', '--fast', '--file', '--fix', '--force', '--force-rechunk', '--force-resume', '--from-pages', '--full', '--help', '--http', '--include-null-signature', '--json', '--judge-model', '--lang', '--limit', '--list-runs', '--markdown', '--max-cost', '--max-far-set', '--max-ideas-per-judge-call', '--model', '--no-embed', '--no-embedding', '--no-extract', '--no-save', '--pattern', '--pending', '--reset', '--resolve', '--resume', '--retry-failed', '--retry-judge', '--save', '--source', '--stale', '--strict-budget', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl', '--yes'],
'cache': ['--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--no-embedding', '--source', '--surface', '--token-ttl', '--yes'],
'calibration': ['--ab', '--aliases', '--all', '--allow-empty', '--apply', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--dry-run', '--entities', '--explain', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--help', '--holder', '--http', '--image', '--include-null-signature', '--json', '--key-prefix', '--kind', '--lang', '--limit', '--markdown', '--max-usd', '--mode', '--multimodal', '--near-symbol', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--phase', '--progress-interval', '--progress-json', '--quiet', '--regenerate', '--repo', '--reset', '--resolve', '--restore-only', '--save', '--scrub-gstack', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--symbol-kind', '--thin', '--token-ttl', '--trusted-extraction', '--undo-wave', '--url', '--with-calibration', '--with-db', '--yes'],
'call': ['--aliases', '--all', '--all-sources', '--as-context', '--auto-fix', '--background', '--brain', '--by-mention', '--catch-up', '--concurrency', '--confirm-destructive', '--content', '--cost-estimate', '--count', '--days', '--depth', '--dim', '--dir', '--direction', '--enable-dcr', '--enable-dcr-insecure', '--explain', '--fast', '--federated', '--file', '--fix', '--follow', '--force', '--from', '--from-meetings', '--grant-types', '--grep', '--hard-deadline', '--help', '--http', '--image', '--include-expired', '--include-frontmatter', '--include-null-signature', '--infer-dates', '--install', '--interval', '--json', '--key', '--kind', '--lang', '--limit', '--link-source', '--link-type', '--llm', '--migrate-only', '--missing-path', '--multimodal', '--ner', '--no-embed', '--no-expand', '--no-extract', '--no-federated', '--no-hard-deadline', '--no-retry-connect', '--no-save', '--older-than', '--page', '--param', '--params', '--password', '--path', '--pattern', '--pending', '--pglite', '--port', '--probe-pglite', '--progress-interval', '--progress-json', '--public-url', '--queue', '--quiet', '--reenrich-after', '--refresh-cache', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--restore-only', '--save', '--scopes', '--session', '--sigma', '--since', '--slug', '--slug-prefix', '--source', '--source-id', '--stale', '--status', '--stdin', '--strategy', '--suites', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--synthesize', '--tag', '--thin', '--timeout', '--to', '--today', '--token', '--token-ttl', '--tools-json', '--type', '--uninstall', '--url', '--version', '--watch', '--with-calibration', '--workers', '--yes'],
'capture': ['--aliases', '--all', '--allow-empty', '--apply', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--depth', '--entities', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--help', '--http', '--image', '--include-null-signature', '--json', '--kind', '--limit', '--max-usd', '--mcp-only', '--mode', '--multimodal', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--repo', '--reset', '--resolve', '--restore-only', '--save', '--scopes', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--stdin', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl', '--trusted-extraction', '--type', '--url', '--what', '--where', '--who', '--with-db', '--yes'],
'check-backlinks': ['--background', '--brain', '--brain-wide-max-cost-usd', '--dir', '--dry-run', '--explain', '--follow', '--help', '--include-frontmatter', '--json', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--source', '--stale', '--timeout', '--type'],
'check-resolvable': ['--brain', '--dry-run', '--fix', '--help', '--json', '--skills-dir', '--source', '--strict', '--verbose'],
'check-update': ['--all', '--brain', '--check', '--ff-only', '--help', '--json', '--markdown', '--migrate-only', '--non-interactive', '--refresh-cache', '--source', '--swap-only', '--to', '--version', '--yes'],
'check-update': ['--all', '--brain', '--check', '--dim', '--ff-only', '--help', '--json', '--markdown', '--migrate-only', '--non-interactive', '--refresh-cache', '--source', '--swap-only', '--to', '--version', '--yes'],
'claw-test': ['--agent', '--brain', '--dir', '--help', '--json', '--keep-tempdir', '--list-agents', '--live', '--local', '--message', '--no-embed', '--no-embedding', '--path', '--pglite', '--progress-json', '--prompt-file', '--run-id', '--scenario', '--source', '--transcripts'],
'code-callees': ['--aliases', '--all', '--all-sources', '--brain', '--chunker-debug', '--clone-dir', '--confirm-destructive', '--federated', '--force', '--help', '--include-null-signature', '--json', '--limit', '--no-extract', '--no-federated', '--no-json', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--thin', '--url', '--url-managed', '--yes'],
'code-callers': ['--aliases', '--all', '--all-sources', '--brain', '--chunker-debug', '--clone-dir', '--confirm-destructive', '--federated', '--force', '--help', '--include-null-signature', '--json', '--limit', '--no-extract', '--no-federated', '--no-json', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--thin', '--url', '--url-managed', '--yes'],
'code-def': ['--aliases', '--all', '--brain', '--chunker-debug', '--help', '--include-null-signature', '--json', '--lang', '--limit', '--no-extract', '--no-json', '--pattern', '--pending', '--pretty', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--yes'],
'code-refs': ['--aliases', '--all', '--brain', '--chunker-debug', '--help', '--include-null-signature', '--json', '--lang', '--limit', '--no-extract', '--no-json', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--yes'],
'config': ['--aliases', '--all', '--brain', '--column', '--coverage-override', '--embedding-dimensions', '--embedding-model', '--fast', '--federated-read', '--follow', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--markdown', '--model', '--multimodal', '--no-embedding', '--no-extract', '--pattern', '--pending', '--pglite', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--surface', '--thin', '--yes'],
'config': ['--aliases', '--all', '--brain', '--column', '--coverage-override', '--detail', '--embedding-dimensions', '--embedding-model', '--fast', '--federated-read', '--follow', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--markdown', '--model', '--multimodal', '--no-embedding', '--no-extract', '--no-federated', '--pattern', '--pending', '--pglite', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--surface', '--thin', '--token-ttl', '--yes'],
'connect': ['--agent', '--bearer-token-env-var', '--bind', '--brain', '--client-id', '--client-secret', '--force', '--grant-types', '--help', '--http', '--install', '--json', '--name', '--oauth', '--public-url', '--register', '--scopes', '--show-token', '--source', '--timeout-ms', '--token', '--token-endpoint-auth-method', '--url', '--version', '--yes'],
'conversation-parser': ['--aliases', '--all', '--brain', '--help', '--include-null-signature', '--json', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin'],
'doctor': ['--ab', '--abbrev-ref', '--abi', '--abort', '--aliases', '--all', '--allow-shell-jobs', '--allow-unverified-remote', '--auto', '--auto-fix', '--auto-update', '--background', '--batch', '--brain', '--brain-wide-max-cost-usd', '--branch', '--break-lock', '--build-index', '--by-mention', '--by-type', '--cached', '--check', '--column', '--compile', '--concurrency', '--confidence', '--confirm', '--content-audit', '--count', '--days', '--delete-brain', '--detach', '--diff-filter', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--exclude-standard', '--exclusive', '--explain', '--fast', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--force-retry', '--force-schema', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--get', '--git-dir', '--git-path', '--grant-types', '--harness', '--health-interval', '--help', '--history', '--home', '--http', '--include-flagged', '--include-frontmatter', '--include-null-signature', '--include-pseudo', '--index-audit', '--init', '--input', '--is-inside-work-tree', '--jq', '--json', '--lang', '--limit', '--local', '--locks', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-crashes', '--max-jobs', '--max-rss', '--max-usd', '--mcp-only', '--migrate-only', '--model', '--multimodal', '--name-only', '--name-status', '--near-symbol', '--nice', '--no', '--no-cron', '--no-embed', '--no-embedding', '--no-extract', '--no-mutate', '--no-verify', '--oauth-client-secret', '--older-than', '--once', '--others', '--overwrite', '--parallel', '--params', '--pat-file', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--priority', '--progress-interval', '--progress-json', '--push-only', '--query', '--queue', '--quiet', '--rebase', '--rebuild-rollup', '--refresh', '--refresh-unqualified', '--regenerate', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--restore-only', '--resume', '--review-lower', '--rollback', '--scope', '--scopes', '--set', '--short', '--show-current', '--show-toplevel', '--since', '--skills-dir', '--skip-bare-tweet', '--skip-failed', '--skip-urls', '--skip-verify', '--slugs', '--source', '--source-id', '--stale', '--stats', '--status', '--strategy', '--strict', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--target', '--target-score', '--thin', '--timeout', '--to', '--top-k', '--type', '--undo-wave', '--unsafe-bypass-dream-guard', '--unset-all', '--untracked-files', '--url', '--use-captured-snapshot', '--verbose', '--verify', '--version', '--window', '--with-calibration', '--workers', '--yes'],
'dream': ['--against', '--aliases', '--all', '--allow-regression', '--anchor', '--asof', '--background', '--batch', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--budget-usd', '--budget-usd-answer', '--budget-usd-retrieval', '--by-type', '--by-type-floor', '--code', '--committed-baseline', '--compare', '--compile', '--concurrent', '--ctx-size', '--cycles', '--date', '--dimensions', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--embeddings', '--expansion', '--explain', '--fast', '--federated', '--fix', '--fixtures', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--force-retry', '--format', '--from', '--from-db', '--from-pages', '--gold', '--harness', '--help', '--http', '--include-holdout', '--include-null-signature', '--input', '--install', '--json', '--judge-model', '--justification', '--keyword-only', '--lang', '--limit', '--llm', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--max-tokens', '--max-usd', '--mcp-only', '--min-recall', '--mode', '--model', '--models', '--modes', '--multimodal', '--name', '--name-only', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-llm', '--no-mutate', '--no-trajectory', '--once', '--out', '--output', '--output-dir', '--parallel', '--path', '--pattern', '--pending', '--pglite', '--phase', '--priority', '--progress-interval', '--progress-json', '--pull', '--quiet', '--receipt-dir', '--remediate', '--repo', '--reranking', '--reset', '--resolve', '--restore-only', '--resume-from', '--retrieval-only', '--rounds', '--rubric-version', '--save', '--seed', '--short', '--show-toplevel', '--since', '--skip-replay', '--slot-a-model', '--slot-b-model', '--slot-c-model', '--slug', '--slug-prefix', '--source', '--source-id', '--stale', '--suite', '--suites', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--take', '--task', '--thin', '--threshold', '--timeout', '--to', '--top-k', '--undo', '--unsafe-bypass-dream-guard', '--update-baseline', '--verify', '--version', '--window', '--yes'],
'doctor': ['--ab', '--abbrev-ref', '--abi', '--abort', '--aliases', '--all', '--allow-shell-jobs', '--allow-unverified-remote', '--auto', '--auto-fix', '--auto-update', '--background', '--batch', '--brain', '--brain-wide-max-cost-usd', '--branch', '--break-lock', '--build-index', '--by-mention', '--by-type', '--cached', '--check', '--column', '--compile', '--concurrency', '--confidence', '--confirm', '--content-audit', '--count', '--days', '--delete-brain', '--detach', '--detail', '--diff-filter', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--exclude-standard', '--exclusive', '--explain', '--fast', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--force-retry', '--force-schema', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--get', '--git-dir', '--git-path', '--grant-types', '--harness', '--health-interval', '--help', '--history', '--home', '--http', '--include-flagged', '--include-frontmatter', '--include-null-signature', '--include-pseudo', '--index-audit', '--init', '--input', '--is-inside-work-tree', '--jq', '--json', '--lang', '--limit', '--local', '--locks', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-crashes', '--max-jobs', '--max-rss', '--max-usd', '--mcp-only', '--migrate-only', '--model', '--multimodal', '--name-only', '--name-status', '--near-symbol', '--nice', '--no', '--no-cron', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-mutate', '--no-verify', '--oauth-client-secret', '--older-than', '--once', '--others', '--overwrite', '--parallel', '--params', '--pat-file', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--priority', '--probe-pglite', '--progress-interval', '--progress-json', '--push-only', '--query', '--queue', '--quiet', '--rebase', '--rebuild-rollup', '--refresh', '--refresh-unqualified', '--regenerate', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--restore-only', '--resume', '--review-lower', '--rollback', '--scope', '--scopes', '--set', '--short', '--show-current', '--show-toplevel', '--since', '--skills-dir', '--skip-bare-tweet', '--skip-failed', '--skip-urls', '--skip-verify', '--slugs', '--source', '--source-id', '--stale', '--stats', '--status', '--strategy', '--strict', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--target', '--target-score', '--thin', '--timeout', '--to', '--token-ttl', '--top-k', '--type', '--undo-wave', '--unsafe-bypass-dream-guard', '--unset-all', '--untracked-files', '--url', '--use-captured-snapshot', '--verbose', '--verify', '--version', '--window', '--with-calibration', '--workers', '--yes'],
'dream': ['--against', '--aliases', '--all', '--allow-regression', '--anchor', '--asof', '--background', '--batch', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--budget-usd', '--budget-usd-answer', '--budget-usd-retrieval', '--by-type', '--by-type-floor', '--code', '--committed-baseline', '--compare', '--compile', '--concurrent', '--ctx-size', '--cycles', '--date', '--detail', '--dimensions', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--embeddings', '--expansion', '--explain', '--fast', '--federated', '--fix', '--fixtures', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--force-retry', '--format', '--from', '--from-db', '--from-pages', '--gold', '--harness', '--help', '--http', '--include-holdout', '--include-null-signature', '--input', '--install', '--json', '--judge-model', '--justification', '--keyword-only', '--lang', '--limit', '--llm', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--max-tokens', '--max-usd', '--mcp-only', '--min-recall', '--mode', '--model', '--models', '--modes', '--multimodal', '--name', '--name-only', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-llm', '--no-mutate', '--no-trajectory', '--once', '--out', '--output', '--output-dir', '--parallel', '--path', '--pattern', '--pending', '--pglite', '--phase', '--priority', '--progress-interval', '--progress-json', '--pull', '--quiet', '--receipt-dir', '--remediate', '--repo', '--reranking', '--reset', '--resolve', '--restore-only', '--resume-from', '--retrieval-only', '--rounds', '--rubric-version', '--save', '--seed', '--short', '--show-toplevel', '--since', '--skip-replay', '--slot-a-model', '--slot-b-model', '--slot-c-model', '--slug', '--slug-prefix', '--source', '--source-id', '--stale', '--suite', '--suites', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--take', '--task', '--thin', '--threshold', '--timeout', '--to', '--token-ttl', '--top-k', '--undo', '--unsafe-bypass-dream-guard', '--update-baseline', '--verify', '--version', '--window', '--yes'],
'edges-backfill': ['--aliases', '--all', '--all-sources', '--brain', '--concurrency', '--federated', '--help', '--include-null-signature', '--json', '--max-age', '--max-chunks', '--max-cost-usd', '--no-extract', '--no-federated', '--older-than', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--thin', '--timeout', '--workers'],
'embed': ['--aliases', '--all', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--catch-up', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--fix', '--follow', '--force', '--force-break-lock', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--lang', '--markdown', '--max-age', '--max-cost-usd', '--model', '--multimodal', '--name', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--pace', '--pace-max-concurrency', '--parallel', '--path', '--pattern', '--pending', '--pglite', '--prefix', '--priority', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--restore-only', '--serial', '--slugs', '--source', '--stale', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--thin', '--timeout', '--version'],
'enrich': ['--aliases', '--all', '--all-sources', '--allow-empty', '--apply', '--asof', '--auto', '--background', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--budget-usd-per-day', '--by-mention', '--clone-dir', '--code', '--concurrency', '--confirm-destructive', '--content', '--date', '--days', '--dry-run', '--embedding-dimensions', '--embedding-model', '--entities', '--explain', '--fast', '--federated', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--from-pages', '--help', '--http', '--image', '--include-null-signature', '--json', '--judge-model', '--kind', '--lang', '--limit', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--max-usd', '--min-context', '--mode', '--model', '--multimodal', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--offset', '--older-than', '--order', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--reenrich-after', '--remediate', '--reset', '--resolve', '--restore-only', '--resume', '--save', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--source-id', '--stale', '--stats', '--supersessions', '--surface', '--symbol-kind', '--thin', '--thin-threshold', '--timeout', '--trusted-extraction', '--types', '--url', '--url-managed', '--version', '--with-db', '--workers', '--yes'],
'eval': ['--ab-relational', '--against', '--aliases', '--all', '--allow-regression', '--background', '--baseline', '--batch', '--brain', '--brain-wide-max-cost-usd', '--budget-usd', '--budget-usd-answer', '--budget-usd-retrieval', '--committed-baseline', '--compare', '--compare-limit', '--concurrent', '--config-a', '--config-b', '--corpus', '--cycles', '--days', '--dedup-cosine', '--dedup-max-per-page', '--dedup-type-ratio', '--dimensions', '--distance-min', '--embedding-dimensions', '--embedding-model', '--expand', '--explain', '--fast', '--fixtures', '--follow', '--force', '--from-capture', '--from-db', '--from-pages', '--gold', '--grounding-min', '--harness', '--help', '--http', '--include-holdout', '--include-null-signature', '--input', '--json', '--judge', '--justification', '--k', '--limit', '--llm', '--max-pair-chars', '--max-tokens', '--max-usd', '--md', '--metric', '--min-recall', '--mode', '--model', '--models', '--modes', '--multimodal', '--name', '--no', '--no-cache', '--no-embed', '--no-embedding', '--no-expand', '--no-extract', '--no-llm', '--older-than', '--out', '--output', '--output-dir', '--parallel', '--pattern', '--pending', '--progress-interval', '--progress-json', '--qrels', '--queries-file', '--query', '--questions', '--quiet', '--receipt-dir', '--refresh-cache', '--remediate', '--reset', '--resolve', '--rrf-k', '--rubric-version', '--runs', '--sampling', '--save', '--seed', '--severity', '--short', '--show-toplevel', '--since', '--skip-replay', '--slot-a-model', '--slot-b-model', '--slot-c-model', '--slug', '--slug-prefix', '--source', '--stale', '--strategy', '--strict', '--suite', '--suites', '--supersessions', '--surface', '--task', '--thin', '--threshold', '--threshold-expected-top1', '--threshold-first-relevant-hit', '--threshold-jaccard', '--threshold-latency-multiplier', '--threshold-recall-at-k', '--threshold-top1', '--timeout', '--tool', '--top-k', '--top-regressions', '--until', '--update-baseline', '--usefulness-min', '--verbose', '--version', '--with-code-intel', '--yes'],
'embed': ['--aliases', '--all', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--catch-up', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--fix', '--follow', '--force', '--force-break-lock', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--lang', '--markdown', '--max-age', '--max-cost-usd', '--model', '--multimodal', '--name', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--pace', '--pace-max-concurrency', '--parallel', '--path', '--pattern', '--pending', '--pglite', '--prefix', '--priority', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--restore-only', '--serial', '--slugs', '--source', '--stale', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--thin', '--timeout', '--token-ttl', '--version'],
'enrich': ['--aliases', '--all', '--all-sources', '--allow-empty', '--apply', '--asof', '--auto', '--background', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--budget-usd-per-day', '--by-mention', '--clone-dir', '--code', '--concurrency', '--confirm-destructive', '--content', '--date', '--days', '--detail', '--dry-run', '--embedding-dimensions', '--embedding-model', '--entities', '--explain', '--fast', '--federated', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--from-pages', '--help', '--http', '--image', '--include-null-signature', '--json', '--judge-model', '--kind', '--lang', '--limit', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--max-usd', '--min-context', '--mode', '--model', '--multimodal', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--offset', '--older-than', '--order', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--reenrich-after', '--remediate', '--reset', '--resolve', '--restore-only', '--resume', '--save', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--source-id', '--stale', '--stats', '--supersessions', '--surface', '--symbol-kind', '--thin', '--thin-threshold', '--timeout', '--token-ttl', '--trusted-extraction', '--types', '--url', '--url-managed', '--version', '--with-db', '--workers', '--yes'],
'eval': ['--ab-relational', '--against', '--aliases', '--all', '--allow-regression', '--background', '--baseline', '--batch', '--brain', '--brain-wide-max-cost-usd', '--budget-usd', '--budget-usd-answer', '--budget-usd-retrieval', '--committed-baseline', '--compare', '--compare-limit', '--concurrent', '--config-a', '--config-b', '--corpus', '--cycles', '--days', '--dedup-cosine', '--dedup-max-per-page', '--dedup-type-ratio', '--dimensions', '--distance-min', '--embedding-dimensions', '--embedding-model', '--expand', '--explain', '--fast', '--fixtures', '--follow', '--force', '--from-capture', '--from-db', '--from-pages', '--gold', '--grounding-min', '--harness', '--help', '--http', '--include-holdout', '--include-null-signature', '--input', '--json', '--judge', '--justification', '--k', '--limit', '--llm', '--max-pair-chars', '--max-tokens', '--max-usd', '--md', '--metric', '--min-recall', '--mode', '--model', '--models', '--modes', '--multimodal', '--name', '--no', '--no-cache', '--no-embed', '--no-embedding', '--no-expand', '--no-extract', '--no-llm', '--older-than', '--out', '--output', '--output-dir', '--parallel', '--pattern', '--pending', '--progress-interval', '--progress-json', '--qrels', '--queries-file', '--query', '--questions', '--quiet', '--receipt-dir', '--refresh-cache', '--remediate', '--reset', '--resolve', '--rrf-k', '--rubric-version', '--runs', '--sampling', '--save', '--seed', '--severity', '--short', '--show-toplevel', '--since', '--skip-replay', '--slot-a-model', '--slot-b-model', '--slot-c-model', '--slug', '--slug-prefix', '--source', '--stale', '--strategy', '--strict', '--suite', '--suites', '--supersessions', '--surface', '--task', '--thin', '--threshold', '--threshold-expected-top1', '--threshold-first-relevant-hit', '--threshold-jaccard', '--threshold-latency-multiplier', '--threshold-recall-at-k', '--threshold-top1', '--timeout', '--token-ttl', '--tool', '--top-k', '--top-regressions', '--until', '--update-baseline', '--usefulness-min', '--verbose', '--version', '--with-code-intel', '--yes'],
'export': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dir', '--explain', '--federated', '--fix', '--follow', '--help', '--include-null-signature', '--json', '--lang', '--markdown', '--multimodal', '--near-symbol', '--no-extract', '--no-federated', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--repo', '--reset', '--resolve', '--restore-only', '--slug-prefix', '--source', '--stale', '--supersessions', '--symbol-kind', '--thin', '--timeout', '--type'],
'extract': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--by-mention', '--catch-up', '--code', '--concurrency', '--dir', '--dry-run', '--explain', '--federated', '--follow', '--from-meetings', '--help', '--include-frontmatter', '--include-null-signature', '--infer-dates', '--json', '--kind', '--lang', '--markdown', '--max-age', '--max-cost-usd', '--multimodal', '--name-status', '--near-symbol', '--ner', '--no-extract', '--no-federated', '--older-than', '--pack', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--repo', '--reset', '--resolve', '--restore-only', '--run-id', '--since', '--slug', '--source', '--source-id', '--stale', '--strategy', '--supersessions', '--symbol-kind', '--thin', '--timeout', '--type', '--verbose', '--workers', '--yes'],
'extract-conversation-facts': ['--aliases', '--all', '--all-sources', '--background', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--by-mention', '--clone-dir', '--code', '--concurrency', '--confirm-destructive', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fix', '--follow', '--force', '--force-break-lock', '--help', '--include-null-signature', '--json', '--judge-model', '--lang', '--limit', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--model', '--multimodal', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--older-than', '--override-disabled', '--path', '--pattern', '--pending', '--pglite', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--restore-only', '--segment-limit', '--session', '--since', '--sleep', '--slug', '--source', '--source-id', '--stale', '--supabase', '--supersessions', '--symbol-kind', '--thin', '--timeout', '--types', '--url', '--url-managed', '--version', '--workers', '--yes'],
'features': ['--aliases', '--all', '--auto-fix', '--background', '--batch-size', '--brain', '--by-mention', '--catch-up', '--concurrency', '--dir', '--explain', '--from-meetings', '--help', '--include-frontmatter', '--include-null-signature', '--infer-dates', '--json', '--kind', '--ner', '--no-extract', '--pace', '--pace-max-concurrency', '--pack', '--path', '--pattern', '--pending', '--priority', '--progress-json', '--quiet', '--repo', '--reset', '--resolve', '--run-id', '--since', '--slugs', '--source', '--source-id', '--stale', '--supersessions', '--thin', '--type', '--verbose', '--workers'],
'files': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--explain', '--fast', '--follow', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--no-pointer', '--page', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--retry-failed', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout', '--type', '--yes'],
'forget': ['--aliases', '--all', '--allow-empty', '--apply', '--as-context', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-tokens', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--entities', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--grep', '--help', '--http', '--image', '--include-expired', '--include-null-signature', '--json', '--kind', '--limit', '--max-usd', '--mcp-only', '--mode', '--multimodal', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--query', '--quiet', '--reason', '--repo', '--reset', '--resolve', '--restore-only', '--rollup', '--save', '--session', '--session-id', '--since', '--since-last-run', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--timeout', '--today', '--trusted-extraction', '--url', '--watch', '--with-db', '--yes'],
'founder': ['--aliases', '--all', '--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--mcp-only', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--since', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout', '--until'],
'files': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--explain', '--fast', '--follow', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--no-pointer', '--page', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--retry-failed', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl', '--type', '--yes'],
'forget': ['--aliases', '--all', '--allow-empty', '--apply', '--as-context', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-tokens', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--entities', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--grep', '--help', '--http', '--image', '--include-expired', '--include-null-signature', '--json', '--kind', '--limit', '--max-usd', '--mcp-only', '--mode', '--multimodal', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--query', '--quiet', '--reason', '--repo', '--reset', '--resolve', '--restore-only', '--rollup', '--save', '--session', '--session-id', '--since', '--since-last-run', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--timeout', '--today', '--token-ttl', '--trusted-extraction', '--url', '--watch', '--with-db', '--yes'],
'founder': ['--aliases', '--all', '--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--mcp-only', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--since', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl', '--until'],
'friction': ['--agent', '--brain', '--help', '--hint', '--json', '--kind', '--message', '--no-redact', '--phase', '--redact', '--run-id', '--severity', '--source', '--transcript-path', '--transcripts'],
'frontmatter': ['--aliases', '--all', '--allow-catch-all', '--brain', '--cached', '--diff-filter', '--dry-run', '--exclude-standard', '--fast', '--fix', '--force', '--from-pages', '--get', '--help', '--http', '--include-catch-all', '--include-null-signature', '--json', '--name-only', '--name-status', '--no-embedding', '--no-extract', '--no-verify', '--others', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--strategy', '--supersessions', '--surface', '--thin', '--timeout', '--uninstall', '--write-back'],
'graph-query': ['--aliases', '--all', '--brain', '--depth', '--direction', '--explain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-foreign', '--include-null-signature', '--json', '--lang', '--markdown', '--mcp-only', '--multimodal', '--near-symbol', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--surface', '--symbol-kind', '--thin', '--timeout', '--type'],
'hook': ['--aliases', '--all', '--allow-unverified-remote', '--batch-limit', '--brain', '--budget-ms', '--cached', '--count', '--delete-brain', '--detach', '--diff-filter', '--end-of-options', '--env', '--exclude-standard', '--fast', '--force', '--from-pages', '--get', '--harness', '--help', '--http', '--include-null-signature', '--jq', '--json', '--name-only', '--no-embedding', '--no-extract', '--once', '--others', '--path', '--pattern', '--pending', '--porcelain', '--quiet', '--reset', '--resolve', '--show-current', '--show-toplevel', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--timeout'],
'import': ['--aliases', '--all', '--asof', '--background', '--brain', '--brain-wide-max-cost-usd', '--by-mention', '--cached', '--code', '--compile', '--concurrency', '--embedding-dimensions', '--embedding-model', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--fix', '--follow', '--force', '--force-rechunk', '--fresh', '--from-pages', '--full', '--help', '--http', '--include-gitignored', '--include-null-signature', '--json', '--lang', '--markdown', '--max-age', '--multimodal', '--name-status', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--older-than', '--others', '--path', '--pattern', '--pending', '--pglite', '--priority', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--since', '--skip-failed', '--source', '--source-id', '--stale', '--strategy', '--supabase', '--supersessions', '--surface', '--thin', '--timeout', '--url', '--workers'],
'init': ['--all', '--brain', '--chat-model', '--check', '--ctx-size', '--embedding-dimensions', '--embedding-model', '--entity', '--expansion-model', '--fast', '--flag', '--force', '--from-pages', '--grant-types', '--help', '--http', '--issuer-url', '--json', '--judge-model', '--key', '--mcp-only', '--mcp-url', '--migrate-only', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--non-interactive', '--oauth-client-id', '--oauth-client-secret', '--path', '--pglite', '--provenance', '--schema-pack', '--scopes', '--skip-embed-check', '--source', '--stale', '--supabase', '--surface', '--to', '--touchpoint', '--url', '--version'],
'integrations': ['--auto', '--brain', '--dry-run', '--embeddings', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--no-embedding', '--overwrite', '--refresh', '--reranking', '--source', '--surface', '--target'],
'integrity': ['--aliases', '--all', '--auto', '--backend', '--background', '--brain', '--brain-wide-max-cost-usd', '--check', '--confidence', '--cost', '--dry-run', '--explain', '--fast', '--follow', '--force', '--fresh', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--limit', '--no-embedding', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--review-lower', '--skip-bare-tweet', '--skip-urls', '--source', '--stale', '--supabase', '--supersessions', '--surface', '--thin', '--timeout', '--type', '--url'],
'jobs': ['--abbrev-ref', '--aliases', '--all', '--allow-empty', '--allow-protected', '--allow-shell-jobs', '--apply', '--asof', '--auto', '--auto-fix', '--auto-with-prompt', '--background', '--backoff-delay', '--backoff-jitter', '--backoff-type', '--batch', '--batch-size', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--break-lock', '--budget-usd', '--budget-usd-per-day', '--by-mention', '--by-type', '--cached', '--catch-up', '--check', '--cli-path', '--cluster', '--cluster-errors', '--code', '--concurrency', '--confidence', '--confirm-destructive', '--content', '--date', '--days', '--delay', '--detach', '--diff-filter', '--dimensions', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--empty', '--entities', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--federated-read', '--ff-only', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--force-retry', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--hard-deadline', '--health-interval', '--held-out', '--help', '--http', '--idempotency-key', '--image', '--include-frontmatter', '--include-gitignored', '--include-null-signature', '--infer-dates', '--inject-bootstrap', '--inline', '--input', '--install', '--interval', '--is-ancestor', '--json', '--kind', '--lang', '--limit', '--lock', '--markdown', '--max-age', '--max-attempts', '--max-cost-usd', '--max-crashes', '--max-rss', '--max-runtime-min', '--max-sources', '--max-stalled', '--max-usd', '--max-waiting', '--mcp-only', '--migrate-only', '--min-context', '--missing-path', '--mode', '--model', '--multimodal', '--name-only', '--name-status', '--near-symbol', '--ner', '--nice', '--no', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-federate', '--no-gpg-sign', '--no-hard-deadline', '--no-inject', '--no-mutate', '--no-pull', '--no-renames', '--no-schema-pack', '--no-verify', '--no-worker', '--non-interactive', '--now', '--offset', '--older-than', '--once', '--order', '--orphan', '--others', '--output', '--override-disabled', '--pace', '--pace-max-concurrency', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--phase', '--pid-file', '--priority', '--progress-interval', '--progress-json', '--queue', '--quiet', '--redact-secrets', '--reenrich-after', '--refresh-cache', '--refresh-ms', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--resume', '--retry-failed', '--review-lower', '--run-id', '--save', '--segment-limit', '--serial', '--session', '--session-id', '--short', '--show-toplevel', '--sigkill-rescue', '--since', '--skip-bare-tweet', '--skip-failed', '--skip-urls', '--sleep', '--slug', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--stats', '--status', '--strategy', '--supersessions', '--surface', '--swap-only', '--symbol-kind', '--target', '--target-score', '--thin', '--thin-threshold', '--timeout', '--timeout-ms', '--to', '--trusted-extraction', '--type', '--types', '--uninstall', '--unsafe-bypass-dream-guard', '--url', '--user', '--verbose', '--verify', '--version', '--watch', '--wedge-rescue', '--with-db', '--workers', '--yes'],
'lint': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--exclude', '--explain', '--fast', '--fix', '--follow', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout'],
'frontmatter': ['--aliases', '--all', '--allow-catch-all', '--brain', '--cached', '--diff-filter', '--dry-run', '--exclude-standard', '--fast', '--fix', '--force', '--from-pages', '--get', '--help', '--http', '--include-catch-all', '--include-null-signature', '--json', '--name-only', '--name-status', '--no-embedding', '--no-extract', '--no-verify', '--others', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--strategy', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl', '--uninstall', '--write-back'],
'graph-query': ['--aliases', '--all', '--brain', '--depth', '--direction', '--explain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-foreign', '--include-null-signature', '--json', '--lang', '--markdown', '--mcp-only', '--multimodal', '--near-symbol', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--surface', '--symbol-kind', '--thin', '--timeout', '--token-ttl', '--type'],
'hook': ['--aliases', '--all', '--allow-unverified-remote', '--batch-limit', '--brain', '--budget-ms', '--cached', '--count', '--delete-brain', '--detach', '--diff-filter', '--end-of-options', '--env', '--exclude-standard', '--fast', '--force', '--from-pages', '--get', '--harness', '--help', '--http', '--include-null-signature', '--jq', '--json', '--name-only', '--no-embedding', '--no-extract', '--once', '--others', '--path', '--pattern', '--pending', '--porcelain', '--quiet', '--reset', '--resolve', '--show-current', '--show-toplevel', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl'],
'import': ['--aliases', '--all', '--asof', '--background', '--brain', '--brain-wide-max-cost-usd', '--by-mention', '--cached', '--code', '--compile', '--concurrency', '--embedding-dimensions', '--embedding-model', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--fix', '--follow', '--force', '--force-rechunk', '--fresh', '--from-pages', '--full', '--help', '--http', '--include-gitignored', '--include-null-signature', '--json', '--lang', '--markdown', '--max-age', '--multimodal', '--name-status', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--older-than', '--others', '--path', '--pattern', '--pending', '--pglite', '--priority', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--since', '--skip-failed', '--source', '--source-id', '--stale', '--strategy', '--supabase', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl', '--url', '--workers'],
'init': ['--all', '--brain', '--chat-model', '--check', '--ctx-size', '--embedding-dimensions', '--embedding-model', '--entity', '--expansion-model', '--fast', '--flag', '--force', '--from-pages', '--grant-types', '--help', '--http', '--issuer-url', '--json', '--judge-model', '--key', '--mcp-only', '--mcp-url', '--migrate-only', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--non-interactive', '--oauth-client-id', '--oauth-client-secret', '--path', '--pglite', '--provenance', '--schema-pack', '--scopes', '--skip-embed-check', '--source', '--stale', '--supabase', '--surface', '--to', '--token-ttl', '--touchpoint', '--url', '--version'],
'integrations': ['--auto', '--brain', '--dry-run', '--embeddings', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--no-embedding', '--overwrite', '--refresh', '--reranking', '--source', '--surface', '--target', '--token-ttl'],
'integrity': ['--aliases', '--all', '--auto', '--backend', '--background', '--brain', '--brain-wide-max-cost-usd', '--check', '--confidence', '--cost', '--dry-run', '--explain', '--fast', '--follow', '--force', '--fresh', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--limit', '--no-embedding', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--review-lower', '--skip-bare-tweet', '--skip-urls', '--source', '--stale', '--supabase', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl', '--type', '--url'],
'jobs': ['--abbrev-ref', '--aliases', '--all', '--allow-empty', '--allow-protected', '--allow-shell-jobs', '--apply', '--asof', '--auto', '--auto-fix', '--auto-with-prompt', '--background', '--backoff-delay', '--backoff-jitter', '--backoff-type', '--batch', '--batch-size', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--break-lock', '--budget-usd', '--budget-usd-per-day', '--by-mention', '--by-type', '--cached', '--catch-up', '--check', '--cli-path', '--cluster', '--cluster-errors', '--code', '--concurrency', '--confidence', '--confirm-destructive', '--content', '--date', '--days', '--delay', '--detach', '--diff-filter', '--dim', '--dimensions', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--empty', '--entities', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--federated-read', '--ff-only', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--force-retry', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--hard-deadline', '--health-interval', '--held-out', '--help', '--http', '--idempotency-key', '--image', '--include-frontmatter', '--include-gitignored', '--include-null-signature', '--infer-dates', '--inject-bootstrap', '--inline', '--input', '--install', '--interval', '--is-ancestor', '--json', '--kind', '--lang', '--limit', '--lock', '--markdown', '--max-age', '--max-attempts', '--max-cost-usd', '--max-crashes', '--max-rss', '--max-runtime-min', '--max-sources', '--max-stalled', '--max-usd', '--max-waiting', '--mcp-only', '--migrate-only', '--min-context', '--missing-path', '--mode', '--model', '--multimodal', '--name-only', '--name-status', '--near-symbol', '--ner', '--nice', '--no', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-federate', '--no-federated', '--no-gpg-sign', '--no-hard-deadline', '--no-inject', '--no-mutate', '--no-pull', '--no-renames', '--no-schema-pack', '--no-verify', '--no-worker', '--non-interactive', '--now', '--offset', '--older-than', '--once', '--order', '--orphan', '--others', '--output', '--override-disabled', '--pace', '--pace-max-concurrency', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--phase', '--pid-file', '--priority', '--progress-interval', '--progress-json', '--queue', '--quiet', '--redact-secrets', '--reenrich-after', '--refresh-cache', '--refresh-ms', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--resume', '--retry-failed', '--review-lower', '--run-id', '--save', '--segment-limit', '--serial', '--session', '--session-id', '--short', '--show-toplevel', '--sigkill-rescue', '--since', '--skip-bare-tweet', '--skip-failed', '--skip-urls', '--sleep', '--slug', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--stats', '--status', '--strategy', '--supersessions', '--surface', '--swap-only', '--symbol-kind', '--target', '--target-score', '--thin', '--thin-threshold', '--timeout', '--timeout-ms', '--to', '--token-ttl', '--trusted-extraction', '--type', '--types', '--uninstall', '--unsafe-bypass-dream-guard', '--url', '--user', '--verbose', '--verify', '--version', '--watch', '--wedge-rescue', '--with-db', '--workers', '--yes'],
'lint': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--exclude', '--explain', '--fast', '--fix', '--follow', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl'],
'lsd': ['--brain', '--force-resume', '--help', '--json', '--judge-model', '--limit', '--list-runs', '--max-cost', '--max-far-set', '--max-ideas-per-judge-call', '--no-save', '--resume', '--retry-judge', '--save', '--source', '--strict-budget', '--yes'],
'maintain': ['--aliases', '--all', '--background', '--brain', '--break-lock', '--by-mention', '--catch-up', '--column', '--concurrency', '--content-audit', '--count', '--detach', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--fix', '--force', '--force-retry', '--force-schema', '--from-meetings', '--full', '--help', '--include-flagged', '--include-frontmatter', '--include-null-signature', '--index-audit', '--infer-dates', '--input', '--json', '--kind', '--lang', '--locks', '--markdown', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-usd', '--migrate-only', '--multimodal', '--near-symbol', '--ner', '--nice', '--no-extract', '--no-mutate', '--older-than', '--once', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--progress-json', '--query', '--queue', '--quiet', '--rebuild-rollup', '--regenerate', '--remediate', '--remediation-plan', '--reset', '--resolve', '--restore-only', '--resume', '--run-id', '--safe', '--scope', '--since', '--skills-dir', '--skip-failed', '--slugs', '--source', '--source-id', '--stale', '--status', '--supabase', '--supersessions', '--symbol-kind', '--target', '--target-score', '--thin', '--to', '--top-k', '--type', '--unsafe-bypass-dream-guard', '--url', '--verbose', '--window', '--workers', '--yes'],
'migrate': ['--ab', '--aliases', '--all', '--auto-update', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--build-index', '--by-mention', '--catch-up', '--compile', '--days', '--dim', '--dry-run', '--embedding-dimensions', '--embedding-model', '--embeddings', '--exclusive', '--explain', '--fast', '--fix', '--follow', '--force', '--force-break-lock', '--force-retry', '--force-schema', '--from-meetings', '--from-pages', '--help', '--history', '--http', '--ignore-env-override', '--ignore-missing-key', '--include-null-signature', '--json', '--lang', '--locks', '--markdown', '--max-age', '--model', '--multimodal', '--name', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--non-interactive', '--pace', '--pace-max-concurrency', '--parallel', '--path', '--pattern', '--pending', '--phase', '--prefix', '--priority', '--progress-interval', '--progress-json', '--quiet', '--refresh-unqualified', '--remediate', '--reranking', '--reset', '--resolve', '--restore-only', '--resume', '--rollback', '--skip-verify', '--slugs', '--source', '--stale', '--supersessions', '--surface', '--symbol-kind', '--thin', '--timeout', '--to', '--undo', '--undo-wave', '--url', '--use-captured-snapshot', '--version', '--with-calibration', '--yes'],
'models': ['--aliases', '--all', '--brain', '--ctx-size', '--embedding-dimensions', '--embedding-model', '--embeddings', '--help', '--include-null-signature', '--json', '--judge-model', '--model', '--multimodal', '--no', '--no-embed', '--no-extract', '--pattern', '--pending', '--reranking', '--reset', '--resolve', '--skip', '--source', '--stale', '--supersessions', '--thin', '--undo', '--version'],
'maintain': ['--aliases', '--all', '--background', '--brain', '--break-lock', '--by-mention', '--catch-up', '--column', '--concurrency', '--content-audit', '--count', '--detach', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--fix', '--force', '--force-retry', '--force-schema', '--from-meetings', '--full', '--help', '--include-flagged', '--include-frontmatter', '--include-null-signature', '--index-audit', '--infer-dates', '--input', '--json', '--kind', '--lang', '--locks', '--markdown', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-usd', '--migrate-only', '--multimodal', '--near-symbol', '--ner', '--nice', '--no-extract', '--no-mutate', '--older-than', '--once', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--probe-pglite', '--progress-json', '--query', '--queue', '--quiet', '--rebuild-rollup', '--regenerate', '--remediate', '--remediation-plan', '--reset', '--resolve', '--restore-only', '--resume', '--run-id', '--safe', '--scope', '--since', '--skills-dir', '--skip-failed', '--slugs', '--source', '--source-id', '--stale', '--status', '--supabase', '--supersessions', '--symbol-kind', '--target', '--target-score', '--thin', '--to', '--top-k', '--type', '--unsafe-bypass-dream-guard', '--url', '--verbose', '--window', '--workers', '--yes'],
'migrate': ['--ab', '--aliases', '--all', '--auto-update', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--build-index', '--by-mention', '--catch-up', '--compile', '--days', '--dim', '--dry-run', '--embedding-dimensions', '--embedding-model', '--embeddings', '--exclusive', '--explain', '--fast', '--fix', '--follow', '--force', '--force-break-lock', '--force-retry', '--force-schema', '--from-meetings', '--from-pages', '--help', '--history', '--http', '--ignore-env-override', '--ignore-missing-key', '--include-null-signature', '--json', '--lang', '--locks', '--markdown', '--max-age', '--model', '--multimodal', '--name', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--non-interactive', '--pace', '--pace-max-concurrency', '--parallel', '--path', '--pattern', '--pending', '--phase', '--prefix', '--priority', '--progress-interval', '--progress-json', '--quiet', '--refresh-unqualified', '--remediate', '--reranking', '--reset', '--resolve', '--restore-only', '--resume', '--rollback', '--skip-verify', '--slugs', '--source', '--stale', '--supersessions', '--surface', '--symbol-kind', '--thin', '--timeout', '--to', '--token-ttl', '--undo', '--undo-wave', '--url', '--use-captured-snapshot', '--version', '--with-calibration', '--yes'],
'models': ['--aliases', '--all', '--brain', '--ctx-size', '--detail', '--embedding-dimensions', '--embedding-model', '--embeddings', '--help', '--include-null-signature', '--json', '--judge-model', '--model', '--multimodal', '--no', '--no-embed', '--no-extract', '--pattern', '--pending', '--reranking', '--reset', '--resolve', '--skip', '--source', '--stale', '--supersessions', '--thin', '--undo', '--version'],
'mounts': ['--alias', '--brain', '--cache', '--database-path', '--database-url', '--db-path', '--db-url', '--engine', '--explain', '--help', '--id', '--json', '--lang', '--lock', '--markdown', '--mcp-url', '--multimodal', '--near-symbol', '--path', '--restore-only', '--skills-dir', '--source', '--stale', '--symbol-kind', '--thin', '--verbose'],
'notability-eval': ['--aliases', '--all', '--brain', '--embedding-dimensions', '--embedding-model', '--help', '--in', '--include-null-signature', '--json', '--model', '--multimodal', '--no', '--no-embed', '--no-extract', '--out', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--skip-llm', '--source', '--stale', '--supersessions', '--target-high', '--target-low', '--target-medium', '--thin', '--version'],
'onboard': ['--aliases', '--all', '--allow-empty', '--allow-protected', '--apply', '--asof', '--auto', '--auto-with-prompt', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--check', '--content', '--date', '--days', '--entities', '--explain', '--federated', '--file', '--follow', '--from-pages', '--help', '--history', '--http', '--image', '--include-null-signature', '--json', '--kind', '--limit', '--max-usd', '--mode', '--multimodal', '--no-extract', '--offset', '--params', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--remediation-plan', '--reset', '--resolve', '--resume', '--save', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--target-score', '--thin', '--trusted-extraction', '--url', '--with-db', '--yes'],
'orphans': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--count', '--explain', '--follow', '--help', '--include-null-signature', '--include-pseudo', '--json', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--timeout'],
'pages': ['--aliases', '--all', '--brain', '--dry-run', '--help', '--include-null-signature', '--json', '--no-extract', '--older-than', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin'],
'pglite-repair': ['--brain', '--break-lock', '--dry-rnu', '--dry-run', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--no-embedding', '--path', '--quiet', '--source', '--surface', '--yes'],
'post-upgrade': ['--all', '--apply-clean-hunks', '--brain', '--check', '--code', '--compile', '--concurrency', '--embedding-dimensions', '--embedding-model', '--fast', '--ff-only', '--flag', '--force', '--force-all', '--force-orchestrator', '--force-retry', '--force-schema', '--format', '--from-pages', '--help', '--host-dir', '--http', '--inject-bootstrap', '--inline', '--install', '--interval', '--json', '--limit', '--list', '--markdown', '--max-rss', '--migrate-only', '--migration', '--mode', '--model', '--multimodal', '--name-only', '--no', '--no-autopilot-install', '--no-embed', '--no-embedding', '--no-inject', '--no-worker', '--non-interactive', '--now', '--quiet', '--repo', '--since', '--skills-dir', '--skip-verify', '--source', '--stale', '--status', '--surface', '--swap-only', '--target', '--to', '--uninstall', '--user', '--verbose', '--verify', '--version', '--workers', '--yes'],
'protocol': ['--all', '--allow-empty', '--apply', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--entities', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--help', '--http', '--image', '--json', '--kind', '--limit', '--max-usd', '--mode', '--multimodal', '--no-embedding', '--offset', '--path', '--progress-interval', '--progress-json', '--quiet', '--save', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--stats', '--surface', '--synthesize', '--target', '--timeout', '--token', '--trusted-extraction', '--url', '--with-db', '--yes'],
'providers': ['--brain', '--ctx-size', '--embedding-dimensions', '--embedding-model', '--embeddings', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--reranking', '--source', '--surface', '--touchpoint', '--version'],
'pglite-repair': ['--brain', '--break-lock', '--dry-rnu', '--dry-run', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--no-embedding', '--path', '--quiet', '--source', '--surface', '--token-ttl', '--yes'],
'post-upgrade': ['--all', '--apply-clean-hunks', '--brain', '--check', '--code', '--compile', '--concurrency', '--detail', '--dim', '--embedding-dimensions', '--embedding-model', '--fast', '--ff-only', '--flag', '--force', '--force-all', '--force-orchestrator', '--force-retry', '--force-schema', '--format', '--from-pages', '--help', '--host-dir', '--http', '--inject-bootstrap', '--inline', '--install', '--interval', '--json', '--limit', '--list', '--markdown', '--max-rss', '--migrate-only', '--migration', '--mode', '--model', '--multimodal', '--name-only', '--no', '--no-autopilot-install', '--no-embed', '--no-embedding', '--no-inject', '--no-worker', '--non-interactive', '--now', '--path', '--pglite', '--quiet', '--repo', '--reset', '--since', '--skills-dir', '--skip-verify', '--source', '--stale', '--status', '--supabase', '--surface', '--swap-only', '--target', '--to', '--token-ttl', '--uninstall', '--user', '--verbose', '--verify', '--version', '--workers', '--yes'],
'protocol': ['--all', '--allow-empty', '--apply', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--entities', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--help', '--http', '--image', '--json', '--kind', '--limit', '--max-usd', '--mode', '--multimodal', '--no-embedding', '--offset', '--path', '--progress-interval', '--progress-json', '--quiet', '--save', '--session', '--session-id', '--since', '--slug', '--slugs', '--source', '--stats', '--surface', '--synthesize', '--target', '--timeout', '--token', '--token-ttl', '--trusted-extraction', '--url', '--with-db', '--yes'],
'providers': ['--brain', '--ctx-size', '--embedding-dimensions', '--embedding-model', '--embeddings', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--reranking', '--source', '--surface', '--token-ttl', '--touchpoint', '--version'],
'publish': ['--accent', '--bg', '--border', '--brain', '--card-bg', '--code-bg', '--error', '--fg', '--help', '--json', '--link', '--muted', '--out', '--password', '--source', '--title'],
'quarantine': ['--aliases', '--all', '--apply', '--brain', '--code', '--compile', '--explain', '--fast', '--fix', '--force', '--force-rechunk', '--from-pages', '--help', '--http', '--include-flagged', '--include-null-signature', '--json', '--lang', '--limit', '--markdown', '--multimodal', '--near-symbol', '--no-embed', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--surface', '--symbol-kind', '--thin'],
'recall': ['--aliases', '--all', '--allow-empty', '--apply', '--as-context', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-tokens', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--entities', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--grep', '--help', '--http', '--image', '--include-expired', '--include-null-signature', '--json', '--kind', '--limit', '--max-usd', '--mcp-only', '--mode', '--multimodal', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--query', '--quiet', '--reason', '--repo', '--reset', '--resolve', '--restore-only', '--rollup', '--save', '--session', '--session-id', '--since', '--since-last-run', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--timeout', '--today', '--trusted-extraction', '--url', '--watch', '--with-db', '--yes'],
'quarantine': ['--aliases', '--all', '--apply', '--brain', '--code', '--compile', '--explain', '--fast', '--fix', '--force', '--force-rechunk', '--from-pages', '--help', '--http', '--include-flagged', '--include-null-signature', '--json', '--lang', '--limit', '--markdown', '--multimodal', '--near-symbol', '--no-embed', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--surface', '--symbol-kind', '--thin', '--token-ttl'],
'recall': ['--aliases', '--all', '--allow-empty', '--apply', '--as-context', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-tokens', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--entities', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--grep', '--help', '--http', '--image', '--include-expired', '--include-null-signature', '--json', '--kind', '--limit', '--max-usd', '--mcp-only', '--mode', '--multimodal', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--query', '--quiet', '--reason', '--repo', '--reset', '--resolve', '--restore-only', '--rollup', '--save', '--session', '--session-id', '--since', '--since-last-run', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--timeout', '--today', '--token-ttl', '--trusted-extraction', '--url', '--watch', '--with-db', '--yes'],
'reconcile-links': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--explain', '--follow', '--help', '--include-frontmatter', '--include-null-signature', '--json', '--name-status', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--source', '--stale', '--strategy', '--supersessions', '--thin', '--timeout', '--type'],
'reindex': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--code', '--compile', '--concurrency', '--cost-estimate', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--fix', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--lang', '--limit', '--markdown', '--max-age', '--max-cost-usd', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--no-extract', '--older-than', '--path', '--pattern', '--pending', '--pglite', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--repo', '--reset', '--resolve', '--source', '--stale', '--supabase', '--supersessions', '--surface', '--thin', '--timeout', '--version', '--workers', '--yes'],
'reindex': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--code', '--compile', '--concurrency', '--cost-estimate', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--fix', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--lang', '--limit', '--markdown', '--max-age', '--max-cost-usd', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--no-extract', '--older-than', '--path', '--pattern', '--pending', '--pglite', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--repo', '--reset', '--resolve', '--source', '--stale', '--supabase', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl', '--version', '--workers', '--yes'],
'reindex-code': ['--abi', '--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--chunker-debug', '--code', '--compile', '--concurrency', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fix', '--follow', '--force', '--force-rechunk', '--help', '--include-null-signature', '--json', '--judge-model', '--lang', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--model', '--multimodal', '--no', '--no-embed', '--no-extract', '--older-than', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--serial', '--source', '--stale', '--supersessions', '--thin', '--timeout', '--version', '--workers', '--yes'],
'reindex-frontmatter': ['--aliases', '--all', '--brain', '--concurrency', '--dry-run', '--force', '--help', '--include-null-signature', '--json', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--slug-prefix', '--source', '--stale', '--supersessions', '--thin', '--workers', '--yes'],
'reindex-search-vector': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--explain', '--follow', '--help', '--include-null-signature', '--json', '--migrate-only', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--timeout', '--yes'],
'reinit-pglite': ['--abbrev-ref', '--all', '--allow-empty', '--brain', '--break-lock', '--chat-model', '--concurrency', '--confirm-destructive', '--diff-filter', '--dir', '--embedding-dimensions', '--embedding-model', '--empty', '--entity', '--exclude', '--exclude-standard', '--expansion-model', '--fast', '--force', '--force-break-lock', '--format', '--from-pages', '--full', '--grant-types', '--hard-deadline', '--help', '--http', '--include-gitignored', '--interval', '--is-ancestor', '--issuer-url', '--json', '--key', '--lock', '--max-age', '--max-sources', '--mcp-only', '--mcp-url', '--migrate-only', '--missing-path', '--model', '--name-only', '--name-status', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-gpg-sign', '--no-hard-deadline', '--no-pull', '--no-renames', '--no-schema-pack', '--no-sync', '--no-verify', '--non-interactive', '--oauth-client-id', '--oauth-client-secret', '--orphan', '--parallel', '--path', '--pglite', '--priority', '--provenance', '--quiet', '--repo', '--retry-failed', '--schema-pack', '--scopes', '--serial', '--short', '--show-toplevel', '--skip-embed-check', '--skip-failed', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--strategy', '--supabase', '--surface', '--timeout', '--to', '--url', '--verify', '--version', '--watch', '--workers', '--yes'],
'remote': ['--all', '--background', '--brain', '--break-lock', '--by-mention', '--column', '--concurrency', '--content-audit', '--count', '--detach', '--dim', '--dir', '--drain', '--embedding-dimensions', '--embedding-model', '--fast', '--fix', '--force', '--force-retry', '--force-schema', '--from-pages', '--full', '--help', '--http', '--include-flagged', '--index-audit', '--json', '--locks', '--markdown', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-usd', '--mcp-only', '--migrate-only', '--multimodal', '--nice', '--no-embedding', '--older-than', '--parallel', '--params', '--path', '--pglite', '--phase', '--pid-file', '--porcelain', '--progress-json', '--query', '--queue', '--rebuild-rollup', '--regenerate', '--remediate', '--remediation-plan', '--reset', '--resume', '--scope', '--scopes', '--since', '--skills-dir', '--skip-failed', '--slugs', '--source', '--source-id', '--stale', '--status', '--supabase', '--surface', '--target', '--target-score', '--timeout', '--to', '--top-k', '--url', '--window', '--workers', '--yes'],
'repair-jsonb': ['--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--explain', '--fast', '--follow', '--force', '--from-pages', '--help', '--http', '--json', '--lang', '--markdown', '--multimodal', '--near-symbol', '--no-embedding', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--restore-only', '--source', '--stale', '--supabase', '--surface', '--symbol-kind', '--thin', '--timeout', '--url'],
'reinit-pglite': ['--abbrev-ref', '--all', '--allow-empty', '--brain', '--break-lock', '--chat-model', '--concurrency', '--confirm-destructive', '--diff-filter', '--dir', '--embedding-dimensions', '--embedding-model', '--empty', '--entity', '--exclude', '--exclude-standard', '--expansion-model', '--fast', '--force', '--force-break-lock', '--format', '--from-pages', '--full', '--grant-types', '--hard-deadline', '--help', '--http', '--include-gitignored', '--interval', '--is-ancestor', '--issuer-url', '--json', '--key', '--lock', '--max-age', '--max-sources', '--mcp-only', '--mcp-url', '--migrate-only', '--missing-path', '--model', '--name-only', '--name-status', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-gpg-sign', '--no-hard-deadline', '--no-pull', '--no-renames', '--no-schema-pack', '--no-sync', '--no-verify', '--non-interactive', '--oauth-client-id', '--oauth-client-secret', '--orphan', '--parallel', '--path', '--pglite', '--priority', '--provenance', '--quiet', '--repo', '--retry-failed', '--schema-pack', '--scopes', '--serial', '--short', '--show-toplevel', '--skip-embed-check', '--skip-failed', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--strategy', '--supabase', '--surface', '--timeout', '--to', '--token-ttl', '--url', '--verify', '--version', '--watch', '--workers', '--yes'],
'remote': ['--all', '--background', '--brain', '--break-lock', '--by-mention', '--column', '--concurrency', '--content-audit', '--count', '--detach', '--dim', '--dir', '--drain', '--embedding-dimensions', '--embedding-model', '--fast', '--fix', '--force', '--force-retry', '--force-schema', '--from-pages', '--full', '--help', '--http', '--include-flagged', '--index-audit', '--json', '--locks', '--markdown', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-usd', '--mcp-only', '--migrate-only', '--multimodal', '--nice', '--no-embedding', '--older-than', '--parallel', '--params', '--path', '--pglite', '--phase', '--pid-file', '--porcelain', '--probe-pglite', '--progress-json', '--query', '--queue', '--rebuild-rollup', '--regenerate', '--remediate', '--remediation-plan', '--reset', '--resume', '--scope', '--scopes', '--since', '--skills-dir', '--skip-failed', '--slugs', '--source', '--source-id', '--stale', '--status', '--supabase', '--surface', '--target', '--target-score', '--timeout', '--to', '--token-ttl', '--top-k', '--url', '--window', '--workers', '--yes'],
'repair-jsonb': ['--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--explain', '--fast', '--follow', '--force', '--from-pages', '--help', '--http', '--json', '--lang', '--markdown', '--multimodal', '--near-symbol', '--no-embedding', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--restore-only', '--source', '--stale', '--supabase', '--surface', '--symbol-kind', '--thin', '--timeout', '--token-ttl', '--url'],
'report': ['--brain', '--content', '--dir', '--help', '--json', '--source', '--title', '--type'],
'repos': ['--abbrev-ref', '--abort', '--aliases', '--all', '--all-sources', '--allow-unverified-remote', '--brain', '--branch', '--break-lock', '--cached', '--clone-dir', '--compile', '--confirm-destructive', '--count', '--detect', '--diff-filter', '--dry-run', '--exclude-standard', '--explain', '--fast', '--federated', '--file', '--fix', '--force', '--force-break-lock', '--format', '--from-pages', '--full', '--get', '--git-dir', '--git-path', '--github-repo', '--help', '--http', '--include-null-signature', '--include-warns', '--is-inside-work-tree', '--json', '--keep-storage', '--lang', '--local', '--markdown', '--max-age', '--max-cost-usd', '--message', '--multimodal', '--name', '--name-only', '--name-status', '--near-symbol', '--no-cron', '--no-embedding', '--no-extract', '--no-federate', '--no-federated', '--no-harden', '--no-verify', '--others', '--params', '--pat-file', '--path', '--pattern', '--pending', '--porcelain', '--push-only', '--quiet', '--rebase', '--repo', '--reset', '--resolve', '--restore-only', '--secret', '--set', '--short', '--show-toplevel', '--source', '--source-id', '--stale', '--status', '--strategy', '--supersessions', '--surface', '--symbol-kind', '--thin', '--unset-all', '--url', '--url-managed', '--yes'],
'repos': ['--abbrev-ref', '--abort', '--aliases', '--all', '--all-sources', '--allow-unverified-remote', '--brain', '--branch', '--break-lock', '--cached', '--clone-dir', '--compile', '--confirm-destructive', '--count', '--detect', '--diff-filter', '--dry-run', '--exclude-standard', '--explain', '--fast', '--federated', '--file', '--fix', '--force', '--force-break-lock', '--format', '--from-pages', '--full', '--get', '--git-dir', '--git-path', '--github-repo', '--help', '--http', '--include-null-signature', '--include-warns', '--is-inside-work-tree', '--json', '--keep-storage', '--lang', '--local', '--markdown', '--max-age', '--max-cost-usd', '--message', '--multimodal', '--name', '--name-only', '--name-status', '--near-symbol', '--no-cron', '--no-embedding', '--no-extract', '--no-federate', '--no-federated', '--no-harden', '--no-verify', '--others', '--params', '--pat-file', '--path', '--pattern', '--pending', '--porcelain', '--push-only', '--quiet', '--rebase', '--repo', '--reset', '--resolve', '--restore-only', '--secret', '--set', '--short', '--show-toplevel', '--source', '--source-id', '--stale', '--status', '--strategy', '--supersessions', '--surface', '--symbol-kind', '--thin', '--token-ttl', '--unset-all', '--url', '--url-managed', '--yes'],
'resolvers': ['--auto', '--backend', '--brain', '--cost', '--help', '--json', '--source'],
'retrieval-upgrade': ['--aliases', '--all', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--catch-up', '--dim', '--dry-run', '--embedding-dimensions', '--embedding-model', '--embeddings', '--explain', '--fast', '--follow', '--force', '--from-pages', '--help', '--http', '--ignore-env-override', '--ignore-missing-key', '--include-null-signature', '--json', '--markdown', '--model', '--multimodal', '--name', '--no', '--no-embed', '--no-embedding', '--no-extract', '--non-interactive', '--pace', '--pace-max-concurrency', '--parallel', '--pattern', '--pending', '--prefix', '--priority', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reranking', '--reset', '--resolve', '--resume', '--slugs', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout', '--to', '--undo', '--version', '--yes'],
'retrieval-upgrade': ['--aliases', '--all', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--catch-up', '--dim', '--dry-run', '--embedding-dimensions', '--embedding-model', '--embeddings', '--explain', '--fast', '--follow', '--force', '--from-pages', '--help', '--http', '--ignore-env-override', '--ignore-missing-key', '--include-null-signature', '--json', '--markdown', '--model', '--multimodal', '--name', '--no', '--no-embed', '--no-embedding', '--no-extract', '--non-interactive', '--pace', '--pace-max-concurrency', '--parallel', '--pattern', '--pending', '--prefix', '--priority', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reranking', '--reset', '--resolve', '--resume', '--slugs', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout', '--to', '--token-ttl', '--undo', '--version', '--yes'],
'routing-eval': ['--brain', '--fix', '--help', '--json', '--llm', '--skills-dir', '--source', '--strict', '--verbose'],
'salience': ['--aliases', '--all', '--brain', '--days', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--kind', '--limit', '--mcp-only', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--slug-prefix', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout'],
'schema': ['--alias', '--aliases', '--all', '--apply', '--as-filing-rules', '--brain', '--dims', '--expert', '--expert-routing', '--extractable', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--inverse', '--json', '--kind', '--no-embedding', '--no-extract', '--pack', '--page-type', '--pattern', '--pending', '--prefix', '--primitive', '--reset', '--resolve', '--schema-pack', '--since', '--source', '--source-id', '--stale', '--supersessions', '--surface', '--target-type', '--thin', '--to', '--with-db'],
'self-upgrade': ['--all', '--brain', '--check', '--check-only', '--ff-only', '--force', '--help', '--json', '--markdown', '--migrate-only', '--non-interactive', '--refresh-cache', '--source', '--swap-only', '--to', '--version', '--yes'],
'salience': ['--aliases', '--all', '--brain', '--days', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--kind', '--limit', '--mcp-only', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--slug-prefix', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout', '--token-ttl'],
'schema': ['--alias', '--aliases', '--all', '--apply', '--as-filing-rules', '--brain', '--dims', '--expert', '--expert-routing', '--extractable', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--inverse', '--json', '--kind', '--no-embedding', '--no-extract', '--pack', '--page-type', '--pattern', '--pending', '--prefix', '--primitive', '--reset', '--resolve', '--schema-pack', '--since', '--source', '--source-id', '--stale', '--supersessions', '--surface', '--target-type', '--thin', '--to', '--token-ttl', '--with-db'],
'self-upgrade': ['--all', '--brain', '--check', '--check-only', '--dim', '--ff-only', '--force', '--help', '--json', '--markdown', '--migrate-only', '--non-interactive', '--refresh-cache', '--source', '--swap-only', '--to', '--version', '--yes'],
'serve': ['--aliases', '--all', '--bind', '--bound-slug-prefixes', '--brain', '--enable-dcr', '--enable-dcr-insecure', '--fast', '--federated-read', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--log-full-params', '--name', '--no-embedding', '--no-extract', '--once', '--parallel', '--pattern', '--pending', '--port', '--prefix', '--print-admin-token', '--public-url', '--reset', '--resolve', '--source', '--stale', '--stdio-idle-timeout', '--supersessions', '--suppress', '--suppress-bootstrap-token', '--surface', '--thin', '--token-ttl', '--yes'],
'skillify': ['--brain', '--description', '--dry-run', '--force', '--help', '--json', '--mutating', '--recent', '--skills-dir', '--source', '--strict', '--triggers', '--verbose', '--writes-pages', '--writes-to'],
'skillopt': ['--aliases', '--all', '--allow-mutate-bundled', '--background', '--batch-size', '--benchmark', '--bootstrap-from-routing', '--bootstrap-from-skill', '--bootstrap-reviewed', '--bootstrap-tasks', '--brain', '--brain-wide-max-cost-usd', '--chunker-debug', '--dry-run', '--epochs', '--follow', '--force', '--held-out', '--help', '--include-null-signature', '--json', '--judge-model', '--lr', '--lr-schedule', '--max-cost-usd', '--max-runtime-min', '--model', '--no-extract', '--no-mutate', '--optimizer-model', '--patch', '--pattern', '--pending', '--reset', '--resolve', '--resume', '--rewrite', '--skills-dir', '--source', '--split', '--stale', '--supersessions', '--target-model', '--target-models', '--thin', '--verbose', '--yes'],
'skillpack': ['--all', '--apply-clean-hunks', '--author', '--brain', '--dry-run', '--exit-code', '--fast', '--fix', '--force', '--force-unlock', '--format', '--from', '--from-pages', '--frontmatter', '--full', '--help', '--homepage', '--http', '--json', '--license', '--list', '--minimal', '--name-only', '--no-cache', '--no-embedding', '--no-lint', '--note', '--out', '--overwrite-local', '--push', '--quick', '--quiet', '--refresh', '--repo', '--schema-pack', '--short', '--since', '--skills-dir', '--skip-doctor', '--source', '--strict', '--surface', '--target', '--tier', '--trust', '--url', '--verbose', '--verify', '--workspace', '--yes'],
'skillpack': ['--all', '--apply-clean-hunks', '--author', '--brain', '--dry-run', '--exit-code', '--fast', '--fix', '--force', '--force-unlock', '--format', '--from', '--from-pages', '--frontmatter', '--full', '--help', '--homepage', '--http', '--json', '--license', '--list', '--minimal', '--name-only', '--no-cache', '--no-embedding', '--no-lint', '--note', '--out', '--overwrite-local', '--push', '--quick', '--quiet', '--refresh', '--repo', '--schema-pack', '--short', '--since', '--skills-dir', '--skip-doctor', '--source', '--strict', '--surface', '--target', '--tier', '--token-ttl', '--trust', '--url', '--verbose', '--verify', '--workspace', '--yes'],
'skillpack-check': ['--background', '--brain', '--brain-wide-max-cost-usd', '--explain', '--fast', '--follow', '--help', '--json', '--list', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--source', '--stale', '--strict', '--timeout', '--yes'],
'smoke-test': ['--brain', '--help', '--json', '--source'],
'sources': ['--abbrev-ref', '--abort', '--aliases', '--all', '--all-sources', '--allow-unverified-remote', '--brain', '--branch', '--break-lock', '--cached', '--clone-dir', '--compile', '--confirm-destructive', '--count', '--detect', '--diff-filter', '--dry-run', '--exclude-standard', '--explain', '--fast', '--federated', '--file', '--fix', '--force', '--force-break-lock', '--format', '--from-pages', '--full', '--get', '--git-dir', '--git-path', '--github-repo', '--help', '--http', '--include-null-signature', '--include-warns', '--is-inside-work-tree', '--json', '--keep-storage', '--lang', '--local', '--markdown', '--max-age', '--max-cost-usd', '--message', '--multimodal', '--name', '--name-only', '--name-status', '--near-symbol', '--no-cron', '--no-embedding', '--no-extract', '--no-federate', '--no-federated', '--no-harden', '--no-verify', '--others', '--params', '--pat-file', '--path', '--pattern', '--pending', '--porcelain', '--push-only', '--quiet', '--rebase', '--repo', '--reset', '--resolve', '--restore-only', '--secret', '--set', '--short', '--show-toplevel', '--source', '--source-id', '--stale', '--status', '--strategy', '--supersessions', '--surface', '--symbol-kind', '--thin', '--unset-all', '--url', '--url-managed', '--yes'],
'status': ['--abbrev-ref', '--abi', '--abort', '--aliases', '--all', '--all-sources', '--allow-empty', '--apply', '--asof', '--auto', '--background', '--batch-size', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--brain-wide-max-cost-usd', '--branch', '--break-lock', '--budget-usd-per-day', '--by-mention', '--cached', '--catch-up', '--clone-dir', '--code', '--column', '--compile', '--concurrency', '--confirm-destructive', '--content', '--content-audit', '--count', '--date', '--days', '--deadline-ms', '--depth', '--detach', '--diff-filter', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--empty', '--entities', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--ff-only', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--force-retry', '--force-schema', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--git-path', '--hard-deadline', '--help', '--http', '--image', '--include-flagged', '--include-frontmatter', '--include-gitignored', '--include-null-signature', '--index-audit', '--infer-dates', '--install', '--interval', '--is-ancestor', '--json', '--judge-model', '--kind', '--lang', '--limit', '--lock', '--locks', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-runtime', '--max-sources', '--max-usd', '--mcp-only', '--migrate-only', '--min-context', '--missing-path', '--mode', '--model', '--multimodal', '--name', '--name-only', '--name-status', '--near-symbol', '--ner', '--nice', '--no', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-gpg-sign', '--no-hard-deadline', '--no-pull', '--no-recurse-submodules', '--no-renames', '--no-schema-pack', '--no-verify', '--object-format', '--offset', '--older-than', '--order', '--orphan', '--others', '--overwrite', '--pace', '--pace-max-concurrency', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--prefix', '--priority', '--progress-interval', '--progress-json', '--query', '--queue', '--quiet', '--rebase', '--rebuild-rollup', '--reenrich-after', '--refresh', '--regenerate', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--resume', '--retry-failed', '--run-id', '--save', '--scope', '--scopes', '--section', '--serial', '--session', '--session-id', '--short', '--show-toplevel', '--since', '--skills-dir', '--skip-failed', '--slug', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--stats', '--status', '--stdin', '--strategy', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--target', '--target-score', '--thin', '--thin-threshold', '--timeout', '--to', '--top-k', '--trusted-extraction', '--type', '--types', '--url', '--url-managed', '--verbose', '--verify', '--version', '--watch', '--what', '--where', '--who', '--window', '--with-db', '--workers', '--yes'],
'sources': ['--abbrev-ref', '--abort', '--aliases', '--all', '--all-sources', '--allow-unverified-remote', '--brain', '--branch', '--break-lock', '--cached', '--clone-dir', '--compile', '--confirm-destructive', '--count', '--detect', '--diff-filter', '--dry-run', '--exclude-standard', '--explain', '--fast', '--federated', '--file', '--fix', '--force', '--force-break-lock', '--format', '--from-pages', '--full', '--get', '--git-dir', '--git-path', '--github-repo', '--help', '--http', '--include-null-signature', '--include-warns', '--is-inside-work-tree', '--json', '--keep-storage', '--lang', '--local', '--markdown', '--max-age', '--max-cost-usd', '--message', '--multimodal', '--name', '--name-only', '--name-status', '--near-symbol', '--no-cron', '--no-embedding', '--no-extract', '--no-federate', '--no-federated', '--no-harden', '--no-verify', '--others', '--params', '--pat-file', '--path', '--pattern', '--pending', '--porcelain', '--push-only', '--quiet', '--rebase', '--repo', '--reset', '--resolve', '--restore-only', '--secret', '--set', '--short', '--show-toplevel', '--source', '--source-id', '--stale', '--status', '--strategy', '--supersessions', '--surface', '--symbol-kind', '--thin', '--token-ttl', '--unset-all', '--url', '--url-managed', '--yes'],
'status': ['--abbrev-ref', '--abi', '--abort', '--aliases', '--all', '--all-sources', '--allow-empty', '--apply', '--asof', '--auto', '--background', '--batch-size', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--brain-wide-max-cost-usd', '--branch', '--break-lock', '--budget-usd-per-day', '--by-mention', '--cached', '--catch-up', '--clone-dir', '--code', '--column', '--compile', '--concurrency', '--confirm-destructive', '--content', '--content-audit', '--count', '--date', '--days', '--deadline-ms', '--depth', '--detach', '--detail', '--diff-filter', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--empty', '--entities', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--ff-only', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--force-retry', '--force-schema', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--git-path', '--hard-deadline', '--help', '--http', '--image', '--include-flagged', '--include-frontmatter', '--include-gitignored', '--include-null-signature', '--index-audit', '--infer-dates', '--install', '--interval', '--is-ancestor', '--json', '--judge-model', '--kind', '--lang', '--limit', '--lock', '--locks', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-runtime', '--max-sources', '--max-usd', '--mcp-only', '--migrate-only', '--min-context', '--missing-path', '--mode', '--model', '--multimodal', '--name', '--name-only', '--name-status', '--near-symbol', '--ner', '--nice', '--no', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-gpg-sign', '--no-hard-deadline', '--no-pull', '--no-recurse-submodules', '--no-renames', '--no-schema-pack', '--no-verify', '--object-format', '--offset', '--older-than', '--order', '--orphan', '--others', '--overwrite', '--pace', '--pace-max-concurrency', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--prefix', '--priority', '--probe-pglite', '--progress-interval', '--progress-json', '--query', '--queue', '--quiet', '--rebase', '--rebuild-rollup', '--reenrich-after', '--refresh', '--regenerate', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--resume', '--retry-failed', '--run-id', '--save', '--scope', '--scopes', '--section', '--serial', '--session', '--session-id', '--short', '--show-toplevel', '--since', '--skills-dir', '--skip-failed', '--slug', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--stats', '--status', '--stdin', '--strategy', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--target', '--target-score', '--thin', '--thin-threshold', '--timeout', '--to', '--token-ttl', '--top-k', '--trusted-extraction', '--type', '--types', '--url', '--url-managed', '--verbose', '--verify', '--version', '--watch', '--what', '--where', '--who', '--window', '--with-db', '--workers', '--yes'],
'storage': ['--aliases', '--all', '--brain', '--federated', '--fix', '--help', '--include-null-signature', '--json', '--no-extract', '--no-federated', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--thin', '--to'],
'sweep': ['--aliases', '--all', '--batch-limit', '--brain', '--budget-ms', '--help', '--include-null-signature', '--json', '--no-extract', '--once', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--timeout'],
'sync': ['--abbrev-ref', '--abi', '--abort', '--aliases', '--all', '--all-sources', '--allow-empty', '--asof', '--auto', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--branch', '--break-lock', '--by-mention', '--cached', '--catch-up', '--clone-dir', '--code', '--column', '--compile', '--concurrency', '--confirm-destructive', '--content-audit', '--count', '--depth', '--detach', '--diff-filter', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--empty', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--ff-only', '--fix', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--force-retry', '--force-schema', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--git-path', '--hard-deadline', '--help', '--http', '--include-flagged', '--include-frontmatter', '--include-gitignored', '--include-null-signature', '--index-audit', '--infer-dates', '--interval', '--is-ancestor', '--json', '--kind', '--lang', '--lock', '--locks', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-sources', '--max-usd', '--migrate-only', '--missing-path', '--model', '--multimodal', '--name', '--name-only', '--name-status', '--ner', '--nice', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-gpg-sign', '--no-hard-deadline', '--no-pull', '--no-recurse-submodules', '--no-renames', '--no-schema-pack', '--no-verify', '--object-format', '--older-than', '--orphan', '--others', '--overwrite', '--pace', '--pace-max-concurrency', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--prefix', '--priority', '--progress-interval', '--progress-json', '--query', '--queue', '--quiet', '--rebase', '--rebuild-rollup', '--refresh', '--regenerate', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--resume', '--retry-failed', '--run-id', '--save', '--scope', '--serial', '--short', '--show-toplevel', '--since', '--skills-dir', '--skip-failed', '--slug', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--status', '--stdin', '--strategy', '--supabase', '--supersessions', '--surface', '--target', '--target-score', '--thin', '--timeout', '--to', '--top-k', '--type', '--url', '--url-managed', '--verbose', '--verify', '--watch', '--window', '--workers', '--yes'],
'takes': ['--aliases', '--all', '--brain', '--bucket-size', '--by', '--claim', '--dir', '--domain', '--dry-run', '--evidence', '--expired', '--fast', '--federated', '--force', '--from-pages', '--help', '--holder', '--http', '--include-covered', '--include-null-signature', '--json', '--kind', '--limit', '--max-pages', '--no-embedding', '--no-extract', '--no-federated', '--outcome', '--path', '--pattern', '--pending', '--quality', '--refresh', '--repo', '--reset', '--resolve', '--restore-only', '--row', '--since', '--slugs', '--sort', '--source', '--source-id', '--stale', '--supersessions', '--surface', '--thin', '--unit', '--until', '--value', '--weight', '--who', '--yes'],
'think': ['--aliases', '--all', '--anchor', '--brain', '--calibration-holder', '--explain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--max-usd', '--mcp-only', '--model', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--rounds', '--save', '--since', '--source', '--stale', '--supersessions', '--surface', '--take', '--thin', '--timeout', '--until', '--with-calibration'],
'sync': ['--abbrev-ref', '--abi', '--abort', '--aliases', '--all', '--all-sources', '--allow-empty', '--asof', '--auto', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--branch', '--break-lock', '--by-mention', '--cached', '--catch-up', '--clone-dir', '--code', '--column', '--compile', '--concurrency', '--confirm-destructive', '--content-audit', '--count', '--depth', '--detach', '--diff-filter', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--empty', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--ff-only', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--force-retry', '--force-schema', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--git-path', '--hard-deadline', '--help', '--http', '--include-flagged', '--include-frontmatter', '--include-gitignored', '--include-null-signature', '--index-audit', '--infer-dates', '--interval', '--is-ancestor', '--json', '--kind', '--lang', '--lock', '--locks', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-sources', '--max-usd', '--migrate-only', '--missing-path', '--model', '--multimodal', '--name', '--name-only', '--name-status', '--ner', '--nice', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-gpg-sign', '--no-hard-deadline', '--no-pull', '--no-recurse-submodules', '--no-renames', '--no-schema-pack', '--no-verify', '--object-format', '--older-than', '--orphan', '--others', '--overwrite', '--pace', '--pace-max-concurrency', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--prefix', '--priority', '--probe-pglite', '--progress-interval', '--progress-json', '--query', '--queue', '--quiet', '--rebase', '--rebuild-rollup', '--refresh', '--regenerate', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--resume', '--retry-failed', '--run-id', '--save', '--scope', '--serial', '--short', '--show-toplevel', '--since', '--skills-dir', '--skip-failed', '--slug', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--status', '--stdin', '--strategy', '--supabase', '--supersessions', '--surface', '--target', '--target-score', '--thin', '--timeout', '--to', '--token-ttl', '--top-k', '--type', '--url', '--url-managed', '--verbose', '--verify', '--watch', '--window', '--workers', '--yes'],
'takes': ['--aliases', '--all', '--brain', '--bucket-size', '--by', '--claim', '--dir', '--domain', '--dry-run', '--evidence', '--expired', '--fast', '--federated', '--force', '--from-pages', '--help', '--holder', '--http', '--include-covered', '--include-null-signature', '--json', '--kind', '--limit', '--max-pages', '--no-embedding', '--no-extract', '--no-federated', '--outcome', '--path', '--pattern', '--pending', '--quality', '--refresh', '--repo', '--reset', '--resolve', '--restore-only', '--row', '--since', '--slugs', '--sort', '--source', '--source-id', '--stale', '--supersessions', '--surface', '--thin', '--token-ttl', '--unit', '--until', '--value', '--weight', '--who', '--yes'],
'think': ['--aliases', '--all', '--anchor', '--brain', '--calibration-holder', '--explain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--max-usd', '--mcp-only', '--model', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--rounds', '--save', '--since', '--source', '--stale', '--supersessions', '--surface', '--take', '--thin', '--timeout', '--token-ttl', '--until', '--with-calibration'],
'transcripts': ['--aliases', '--all', '--brain', '--days', '--full', '--help', '--include-null-signature', '--json', '--limit', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--timeout'],
'upgrade': ['--all', '--apply-clean-hunks', '--brain', '--check', '--code', '--compile', '--concurrency', '--embedding-dimensions', '--embedding-model', '--fast', '--ff-only', '--flag', '--force', '--force-all', '--force-orchestrator', '--force-retry', '--force-schema', '--format', '--from-pages', '--help', '--host-dir', '--http', '--inject-bootstrap', '--inline', '--install', '--interval', '--json', '--limit', '--list', '--markdown', '--max-rss', '--migrate-only', '--migration', '--mode', '--model', '--multimodal', '--name-only', '--no', '--no-autopilot-install', '--no-embed', '--no-embedding', '--no-inject', '--no-worker', '--non-interactive', '--now', '--quiet', '--repo', '--since', '--skills-dir', '--skip-verify', '--source', '--stale', '--status', '--surface', '--swap-only', '--target', '--to', '--uninstall', '--user', '--verbose', '--verify', '--version', '--workers', '--yes'],
'watch': ['--aliases', '--all', '--brain', '--fast', '--federated', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--max-pages', '--min-confidence', '--no-embedding', '--no-extract', '--no-federated', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--window-turns'],
'upgrade': ['--all', '--apply-clean-hunks', '--brain', '--check', '--code', '--compile', '--concurrency', '--detail', '--dim', '--embedding-dimensions', '--embedding-model', '--fast', '--ff-only', '--flag', '--force', '--force-all', '--force-orchestrator', '--force-retry', '--force-schema', '--format', '--from-pages', '--help', '--host-dir', '--http', '--inject-bootstrap', '--inline', '--install', '--interval', '--json', '--limit', '--list', '--markdown', '--max-rss', '--migrate-only', '--migration', '--mode', '--model', '--multimodal', '--name-only', '--no', '--no-autopilot-install', '--no-embed', '--no-embedding', '--no-inject', '--no-worker', '--non-interactive', '--now', '--path', '--pglite', '--quiet', '--repo', '--reset', '--since', '--skills-dir', '--skip-verify', '--source', '--stale', '--status', '--supabase', '--surface', '--swap-only', '--target', '--to', '--token-ttl', '--uninstall', '--user', '--verbose', '--verify', '--version', '--workers', '--yes'],
'watch': ['--aliases', '--all', '--brain', '--fast', '--federated', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--max-pages', '--min-confidence', '--no-embedding', '--no-extract', '--no-federated', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--token-ttl', '--window-turns'],
'ze-switch': ['--aliases', '--all', '--brain', '--confirm-reembed', '--dry-run', '--force', '--help', '--ignore-env-override', '--ignore-missing-key', '--include-null-signature', '--json', '--markdown', '--no-extract', '--non-interactive', '--pattern', '--pending', '--reset', '--resolve', '--resume', '--source', '--stale', '--supersessions', '--thin', '--undo', '--yes'],
};
+5
View File
@@ -1123,6 +1123,11 @@ export const KNOWN_CONFIG_KEYS: readonly string[] = [
'orphans.exclude_slugs',
'sync.cost_gate_min_usd',
'sync.federated_v2',
// #2179: clamp window for DCR-requested per-client token TTLs. Read by
// `gbrain serve --http` at startup; unset min defaults to 300s, unset max
// defaults fail-closed to max(--token-ttl, min).
'oauth.dcr_ttl_min_seconds',
'oauth.dcr_ttl_max_seconds',
'embed.backfill_cooldown_min',
'embed.backfill_max_usd_per_source_24h',
'embed.backfill_max_usd',
+42 -2
View File
@@ -1,7 +1,7 @@
/**
* v0.41.16.0 Built-in conversation parser pattern registry.
*
* Seventeen hand-vetted patterns covering the chat-export formats this
* Eighteen hand-vetted patterns covering the chat-export formats this
* codebase is most likely to encounter. Each pattern's regex was
* derived from a public format reference (source_doc field) so future
* maintainers can verify against the wild shape.
@@ -50,7 +50,7 @@ export function cleanSpeaker(raw: string, override?: RegExp): string {
return stripped || raw.trim();
}
/** The 17 hand-vetted built-in patterns. */
/** The 18 hand-vetted built-in patterns. */
export const BUILTIN_PATTERNS: readonly PatternEntry[] = [
// -------------------------------------------------------------------
// INLINE-DATE patterns (date in every line; less ambiguous; tried first).
@@ -670,6 +670,46 @@ export const BUILTIN_PATTERNS: readonly PatternEntry[] = [
test_negative: ['<alice> classic irc, no time', '[18:37] @alice: matrix'],
source_doc: 'weechat default logger.format `%H:%M %p\\t%m`',
},
{
id: 'markdown-heading-turn',
origin: 'builtin',
// gbrain transcript-ingest shape: a heading-only line ('## User' /
// '## Assistant' / '### Human') opens a turn; the message text is
// the continuation lines below the heading (D5), not anything on
// the heading line itself. No per-line timestamps — date comes
// from frontmatter / effective_date. The speaker set is closed
// (User/Assistant/Human/System only) so ordinary section headings
// like '## Summary' never match, and a heading with trailing prose
// ('## User said hello') is rejected rather than mis-captured.
regex: /^#{2,3}\s+(User|Assistant|Human|System)\s*:?\s*()$/,
captures: {
speaker_group: 1,
text_group: 2,
},
date_source: 'frontmatter',
time_format: '24h',
timezone_policy: 'utc_assumed_with_warn',
multi_line: true,
score_continuations_as_body: true,
// Narrowed to a role-prefix superset (NOT bare `/^#{2,3}\s/`): a body
// that pastes unrelated markdown headings (e.g. a document with many
// '## Section' headings) would otherwise inflate the D18 scorer's
// anchor-candidate denominator without inflating the anchored count,
// starving the pattern's score toward 0 on otherwise-valid transcripts.
// Still a strict superset of `regex` per validatePatternEntry's
// invariant (every test_positive sample passes both).
quick_reject: /^#{2,3}\s+(?:User|Assistant|Human|System)\b/,
test_positive: ['## User', '## Assistant', '### Human', '## System', '## User:'],
test_negative: [
'## Summary',
'#### User',
'User: plain no heading',
'## User said hello',
],
source_doc:
'gbrain nightly transcript ingest: compiled_truth bodies use markdown headings per turn',
},
];
/**
+1 -1
View File
@@ -392,7 +392,7 @@ function getNonBlankLines(body: string, headCap?: number): string[] {
* window) and `scorePatternFull` (whole body) delegate here so the
* quick_reject + regex loop lives in one place. Reused by
* `parseConversation`'s fallback path which pre-splits ONCE and
* passes the array to all 17 candidates (saves 16 redundant body
* passes the array to all 18 candidates (saves 17 redundant body
* splits per fallback pass).
*/
function scoreFromLines(
+7
View File
@@ -2063,6 +2063,13 @@ export async function runCycle(
yieldDuringPhase: opts.yieldDuringPhase,
once: opts.onceForPhase === 'patterns',
deadlineAtMs: opts.deadlineAtMs ?? null,
// #1586: scope pattern writes to the cycle's resolved source, same as
// synthesize above. Without it the child's put_page rows land in
// 'default' while the reverse-write drops the file into the named
// source's checkout — the row and the file disagree about which
// source owns the page, which is what doctor reports as
// multi_source_drift.
sourceId: cycleSourceId,
}));
result.duration_ms = duration_ms;
phaseResults.push(result);
+37 -27
View File
@@ -6,14 +6,16 @@
// pages already extracted by content hash — see "Idempotency" below).
// 2. Dedup by content_hash; transcripts win on collision.
// 3. Per work-item, ask Haiku for 1-3 atoms.
// 4. Write each atom via engine.putPage(slug, page, {sourceId})
// with sourceId threaded so federated brains route correctly.
// 4. Write each atom via importFromContent(slug, markdown, {sourceId})
// with sourceId threaded so federated brains route correctly. The
// canonical import path (not engine.putPage) is what chunks and embeds
// the page — see the write site below and #2163.
//
// Idempotency (per-atom, via deterministic slug):
// Each atom's slug is `atoms/<source-date>/<stem>-<title-hash>` — built from
// the SOURCE date (the transcript's own date / the page slug), NOT the run
// date, plus a 6-char hash of the title. Re-extracting the same atom resolves
// to the SAME slug, so engine.putPage upserts in place instead of minting a
// to the SAME slug, so the import upserts in place instead of minting a
// duplicate. This closes three bugs in one scheme:
// - PR #1414's page-side re-extraction.
// - The cross-day transcript duplicate: append-only transcripts grow daily,
@@ -51,7 +53,9 @@ import type { BrainEngine } from '../engine.ts';
import type { PhaseResult } from '../cycle.ts';
import type { GBrainConfig } from '../config.ts';
import type { ProgressReporter } from '../progress.ts';
import { chat as gatewayChat, withBudgetTracker } from '../ai/gateway.ts';
import { chat as gatewayChat, withBudgetTracker, isAvailable } from '../ai/gateway.ts';
import { importFromContent } from '../import-file.ts';
import { serializeMarkdown } from '../markdown.ts';
import { BudgetExhausted, BudgetTracker, isModelPriceable } from '../budget/budget-tracker.ts';
import { writeReceipt } from '../extract/receipt-writer.ts';
import { upsertExtractRollup } from '../extract/rollup-writer.ts';
@@ -681,32 +685,38 @@ export async function runPhaseExtractAtoms(
item.kind === 'transcript'
? { source_path: item.filePath }
: { source_slug: item.slug };
// v0.41.2.1 D9 #1 — thread sourceId through every putPage so
// atoms land in the source we discovered them from. Pre-fix
// the third arg was missing and atoms always wrote to 'default'.
await engine.putPage(
slug,
// Serialize to markdown and import via the canonical pipeline so
// the atom is chunked (+ embedded when a provider is configured).
// engine.putPage is a bare page-row upsert that never chunks, so
// atoms written through it never reached content_chunks and were
// invisible to search — the same defect #2163 fixed for concept
// pages in synthesize-concepts.ts, which was never applied here.
//
// `type: 'atom'` rides in frontmatter, which parseMarkdown honours
// as an explicit override ahead of path inference, so the page type
// survives the round-trip. sourceId stays threaded (v0.41.2.1 D9 #1)
// so atoms still land in the source they were discovered from.
const md = serializeMarkdown(
{
title: atom.title,
type: 'atom',
compiled_truth: atom.body,
frontmatter: {
type: 'atom',
atom_type: atom.atom_type,
...originFrontmatter,
source_hash: item.contentHash.slice(0, 16),
...(atom.source_quote && { source_quote: atom.source_quote }),
...(atom.lesson && { lesson: atom.lesson }),
...(atom.concepts && atom.concepts.length > 0 && { concepts: atom.concepts }),
...(atom.virality_score !== undefined && { virality_score: atom.virality_score }),
...(atom.emotional_register && { emotional_register: atom.emotional_register }),
extracted_at: new Date().toISOString(),
extracted_by: 'extract_atoms-v0.41.2.1',
},
timeline: '',
atom_type: atom.atom_type,
...originFrontmatter,
source_hash: item.contentHash.slice(0, 16),
...(atom.source_quote && { source_quote: atom.source_quote }),
...(atom.lesson && { lesson: atom.lesson }),
...(atom.concepts && atom.concepts.length > 0 && { concepts: atom.concepts }),
...(atom.virality_score !== undefined && { virality_score: atom.virality_score }),
...(atom.emotional_register && { emotional_register: atom.emotional_register }),
extracted_at: new Date().toISOString(),
extracted_by: 'extract_atoms-v0.41.2.1',
},
{ sourceId },
atom.body,
'',
{ type: 'atom', title: atom.title, tags: [] },
);
await importFromContent(engine, slug, md, {
sourceId,
noEmbed: !isAvailable('embedding'),
});
totalAtomsExtracted++;
}
} else {
+55 -33
View File
@@ -30,14 +30,13 @@ import { serializeMarkdown } from '../markdown.ts';
import type { Page, PageType } from '../types.ts';
// #2415: allow-list + output-root resolution shared with the synthesize
// phase — both phases must agree on the configured namespace.
// runPgliteSubagentsInline is shared too: PGLite has no separate Minions
// worker process (the embedded data-dir holds an exclusive file lock), so a
// job submitted via queue.add() sits in 'waiting' forever unless something
// drives the claim -> run -> complete loop inline. synthesize.ts already
// does this for its own children; patterns.ts previously submitted and
// waited without ever draining, so every real (non-dry-run) invocation on a
// PGLite brain hung until subagentWaitTimeoutMs (default 35 min).
import { loadAllowedSlugPrefixes, loadOutputRoot, runPgliteSubagentsInline } from './synthesize.ts';
// runSubagentsInline is shared too: a job submitted via queue.add() sits in
// 'waiting' forever unless something drives the claim -> run -> complete
// loop — on PGLite because no separate worker can open the embedded
// data-dir, on Postgres because the parent phase itself occupies a worker
// slot and can deadlock a fully-occupied worker (#2050). synthesize.ts
// drains its own children the same way.
import { loadAllowedSlugPrefixes, loadOutputRoot, runSubagentsInline } from './synthesize.ts';
import { probeChatModel } from '../ai/gateway.ts';
import { normalizeModelId } from '../model-id.ts';
@@ -60,6 +59,13 @@ export interface PatternsPhaseOpts {
* mid-phase and starves every tail phase (#2781).
*/
deadlineAtMs?: number | null;
/**
* #1586: the cycle's resolved source. Stamped onto every subagent child as
* `source_id` so put_page writes land in this source's rows, and passed to
* reverseWriteRefs so getPage/getTags read the correct (source_id, slug)
* row. Unset legacy 'default'. Mirrors synthesize.ts's `sourceId`.
*/
sourceId?: string;
}
/**
@@ -185,18 +191,20 @@ export async function runPhasePatterns(
}
const queue = new MinionQueue(engine);
// PGLite children drain inline (no separate worker can open the embedded
// data-dir), so give this job a private per-run queue: the inline drain
// must never claim unrelated 'default'-queue jobs a Postgres worker owns.
// Mirrors synthesize.ts's childQueueName derivation exactly.
const childQueueName = engine.kind === 'pglite'
? `dream-inline-${Date.now()}-${randomUUID().slice(0, 8)}`
: 'default';
// #2050: children drain inline on BOTH engines (see runSubagentsInline),
// so give this job a private per-run queue: the inline drain must never
// claim unrelated 'default'-queue jobs, and a 'default'-queue worker must
// never claim a child this parent is about to run itself. Mirrors
// synthesize.ts's childQueueName derivation exactly.
const childQueueName = `dream-inline-${Date.now()}-${randomUUID().slice(0, 8)}`;
const data: SubagentHandlerData = {
prompt: buildPatternsPrompt(reflections, config.minEvidence, config.sourceSlugPrefix, config.outputSlugPrefix),
model: config.model,
max_turns: 30,
allowed_slug_prefixes: allowedSlugPrefixes,
// #1586: scope every child tool call to the cycle's resolved source so
// put_page writes land there instead of the hardcoded 'default'.
...(opts.sourceId ? { source_id: opts.sourceId } : {}),
};
const submitOpts: Partial<MinionJobInput> = {
max_stalled: 3,
@@ -207,12 +215,11 @@ export async function runPhasePatterns(
allowProtectedSubmit: true,
});
// PGLite cannot run a separate Minions worker because the embedded DB
// holds an exclusive file lock. Drain this phase's private child queue
// inline so the parent observes the terminal state instead of polling
// waitForCompletion until subagentWaitTimeoutMs expires. No-op on
// Postgres (a real worker process claims the job there).
await runPgliteSubagentsInline(engine, queue, childQueueName, opts.yieldDuringPhase);
// Drain this phase's private child queue inline so the parent observes
// the terminal state instead of polling waitForCompletion until
// subagentWaitTimeoutMs expires. Runs on BOTH engines — on Postgres the
// parent job otherwise deadlocks a fully-occupied worker (#2050).
await runSubagentsInline(engine, queue, childQueueName, opts.yieldDuringPhase);
let outcome: string;
try {
@@ -243,10 +250,14 @@ export async function runPhasePatterns(
// Collect refs the subagent wrote (codex finding #2 — query tool exec rows).
// v0.32.8: refs carry source_id so reverseWriteRefs targets the right
// (source, slug) row instead of the first DB match.
const writtenRefs = await collectChildPutPageSlugs(engine, [job.id]);
// #1586: refs carry the cycle's resolved source (children wrote there via
// SubagentHandlerData.source_id), so getPage/getTags read the same row the
// child wrote, and the reverse-write treats it as the native source.
const cycleSourceId = opts.sourceId ?? 'default';
const writtenRefs = await collectChildPutPageSlugs(engine, [job.id], cycleSourceId);
// Reverse-write to fs.
const reverseWriteCount = await reverseWriteRefs(engine, opts.brainDir, writtenRefs);
const reverseWriteCount = await reverseWriteRefs(engine, opts.brainDir, writtenRefs, cycleSourceId);
const details = {
reflections_considered: reflections.length,
@@ -454,13 +465,14 @@ When done, briefly list the pattern slugs you wrote/updated in your final messag
async function collectChildPutPageSlugs(
engine: BrainEngine,
childIds: number[],
sourceId = 'default',
): Promise<Array<{ slug: string; source_id: string }>> {
if (childIds.length === 0) return [];
// v0.32.8: subagent put_page tool schema doesn't expose source_id (subagents
// are scoped to a single source). Default to 'default' here; multi-source
// dream cycles are a v0.33 follow-up. The point of threading source_id is
// so reverseWriteRefs can pass it through getPage and pick the correct
// (source_id, slug) row instead of whatever the DB happens to return.
// are scoped to a single source). #1586: stamp the cycle's resolved source
// children write there via SubagentHandlerData.source_id — so reverseWriteRefs
// can pass it through getPage and pick the correct (source_id, slug) row
// instead of whatever the DB happens to return. Unset → legacy 'default'.
const rows = await engine.executeRaw<{ slug: string }>(
`SELECT DISTINCT
COALESCE(input->>'slug', (input #>> '{}')::jsonb->>'slug') AS slug
@@ -474,7 +486,7 @@ async function collectChildPutPageSlugs(
return rows
.map(r => r.slug)
.filter((s): s is string => typeof s === 'string' && s.length > 0)
.map(slug => ({ slug, source_id: 'default' }));
.map(slug => ({ slug, source_id: sourceId }));
}
// ── Reverse-write ────────────────────────────────────────────────────
@@ -485,6 +497,7 @@ async function reverseWriteRefs(
engine: BrainEngine,
brainDir: string,
refs: Array<{ slug: string; source_id: string }>,
nativeSourceId = 'default',
): Promise<number> {
let count = 0;
for (const { slug, source_id } of refs) {
@@ -496,11 +509,12 @@ async function reverseWriteRefs(
const tags = await engine.getTags(slug, { sourceId: source_id });
try {
const md = renderPageToMarkdown(page, tags);
// v0.32.8 F6: non-default sources land under brainDir/.sources/<id>/<slug>.md
// so same-slug-different-source pages don't collide on disk. Default-source
// pages stay at brainDir/<slug>.md so single-source brains see no change.
// `.sources/` is a reserved prefix; walkBrainRepo skips dot-dirs.
const filePath = source_id === 'default'
// v0.32.8 F6: foreign-source pages land under brainDir/.sources/<id>/<slug>.md
// so same-slug-different-source pages don't collide on disk. Pages belonging
// to the cycle's own source (#1586: brainDir IS that source's checkout —
// legacy 'default' when unscoped) stay at brainDir/<slug>.md so single-source
// brains see no change. `.sources/` is a reserved prefix; walkBrainRepo skips dot-dirs.
const filePath = source_id === nativeSourceId
? join(brainDir, `${slug}.md`)
: join(brainDir, '.sources', source_id, `${slug}.md`);
mkdirSync(dirname(filePath), { recursive: true });
@@ -558,3 +572,11 @@ function failed(error: PhaseError): PhaseResult {
function makeError(cls: string, code: string, message: string, hint?: string): PhaseError {
return hint ? { class: cls, code, message, hint } : { class: cls, code, message };
}
// `__testing` re-exports otherwise-private helpers so unit tests can pin the
// source-scoping contract (#1586) without driving a whole dream cycle.
// Mirrors synthesize.ts's `__testing` block.
export const __testing = {
collectChildPutPageSlugs,
reverseWriteRefs,
};
+148 -37
View File
@@ -37,6 +37,8 @@ import { basename, join, dirname, isAbsolute, resolve } from 'node:path';
import type { BrainEngine } from '../engine.ts';
import type { PhaseResult, PhaseError } from '../cycle.ts';
import { MinionQueue } from '../minions/queue.ts';
import { reconnectAfterConnectionError } from '../minions/reconnect.ts';
import { isRetryableConnError } from '../retry-matcher.ts';
import { waitForCompletion, TimeoutError } from '../minions/wait-for-completion.ts';
import { makeSubagentHandler } from '../minions/handlers/subagent.ts';
import type { MinionJobInput, MinionJobContext, MinionHandler, SubagentHandlerData } from '../minions/types.ts';
@@ -276,39 +278,101 @@ export interface SynthesizePhaseOpts {
once?: boolean;
}
const INLINE_PGLITE_LOCK_MS = 30_000;
const INLINE_LOCK_MS = 30_000;
/**
* PGLite cannot be served by a separate Minions worker process: the embedded
* data-dir holds an exclusive file lock, so subagent children enqueued by the
* synth parent would sit in 'waiting' until waitForCompletion times out.
* Drive the same claim run complete/fail loop a worker would perform,
* inline, against this phase's private child queue.
* Drain this phase's private child queue inline: drive the same claim run
* complete/fail loop a worker would perform, from the parent's own slot.
*
* Why inline on BOTH engines:
* - PGLite: no separate Minions worker can run at all (the embedded
* data-dir holds an exclusive file lock), so children would sit in
* 'waiting' until waitForCompletion times out.
* - Postgres (#2050): the parent phase itself runs as a job inside a
* `jobs work` process. A worker whose slots are all occupied by such
* parents (autopilot spawns its drain worker at the default
* concurrency=1) can never claim the child the parent is blocking on
* a structural self-deadlock. Running children inline means a child
* never needs a worker slot, so the deadlock is impossible at ANY
* concurrency, and no extra DB-pool pressure is added: the child's work
* replaces the parent's idle waitForCompletion polling in the slot the
* parent already holds.
*
* `yieldDuringPhase` is ticked on a 60s interval while a child runs so the
* 5-min cycle lock TTL keeps refreshing during long (up to 30-min) children.
* The child's own claim lock is heartbeated at lockMs/3 (worker cadence
* parity) on Postgres a concurrent worker sweeps handleStalled() across
* ALL queues, so without renewal any child running longer than lockMs would
* be requeued mid-run and stall-churned to dead.
*/
export async function runPgliteSubagentsInline(
export async function runSubagentsInline(
engine: BrainEngine,
queue: MinionQueue,
queueName: string,
yieldDuringPhase?: () => Promise<void>,
handler: MinionHandler = makeSubagentHandler({ engine }),
lockMs: number = INLINE_LOCK_MS,
): Promise<void> {
if (engine.kind !== 'pglite') return;
// #3555 interaction: the drain's queue ops used to be bare awaits, so a
// transient pooler reap mid-drain threw out of the loop and stranded the
// remaining children in this per-run private queue — which no worker will
// ever claim. Mirror the worker's recovery: on a retryable connection
// error, rebuild the pool (shared reconnectAfterConnectionError) and retry
// the loop; non-retryable errors still propagate (real bug → phase fails).
const MAX_CONN_ERROR_STREAK = 5;
let connErrorStreak = 0;
let sawConnError = false;
const recoverOrThrow = async (site: string, e: unknown): Promise<void> => {
if (!isRetryableConnError(e) || ++connErrorStreak > MAX_CONN_ERROR_STREAK) throw e;
sawConnError = true;
const msg = e instanceof Error ? e.message : String(e);
process.stderr.write(`[dream] inline drain ${site} hit a connection error; reconnecting and retrying: ${msg}\n`);
await reconnectAfterConnectionError(engine, `inline-${site}`, e);
// Small cooperative backoff (setTimeout keeps the cycle-lock keepalive
// and any concurrent timers firing) before the loop retries.
await new Promise((r) => setTimeout(r, Math.min(1000, Math.max(50, Math.floor(lockMs / 3)))));
};
while (true) {
// Housekeeping a worker would normally perform, so child rows can reach
// terminal states (delayed retries promoted, timeouts dead-lettered)
// before the synth parent enters waitForCompletion polling.
await queue.promoteDelayed();
await queue.handleStalled();
await queue.handleTimeouts();
await queue.handleWallClockTimeouts(INLINE_PGLITE_LOCK_MS);
const lockToken = randomUUID();
const job = await queue.claim(lockToken, INLINE_PGLITE_LOCK_MS, queueName, ['subagent']);
if (!job) return;
let job: Awaited<ReturnType<MinionQueue['claim']>>;
try {
// Housekeeping a worker would normally perform, so child rows can reach
// terminal states (delayed retries promoted, timeouts dead-lettered)
// before the synth parent enters waitForCompletion polling.
await queue.promoteDelayed();
await queue.handleStalled();
await queue.handleTimeouts();
await queue.handleWallClockTimeouts(lockMs);
job = await queue.claim(lockToken, lockMs, queueName, ['subagent']);
} catch (e) {
await recoverOrThrow('queue-ops', e);
continue;
}
connErrorStreak = 0;
if (!job) {
if (!sawConnError) return;
// A connection-error window may have left a child 'active' under a
// lock nobody renews (a claim that committed but whose row never
// reached us, or a lost outcome write below). handleStalled() at the
// loop top requeues it once the lock expires (≤ lockMs), so only exit
// once the queue is actually quiet.
let active = 0;
try {
const rows = await engine.executeRaw<{ n: number }>(
`SELECT count(*)::int AS n FROM minion_jobs WHERE queue = $1 AND status = 'active'`,
[queueName],
);
active = rows[0]?.n ?? 0;
} catch (e) {
await recoverOrThrow('active-check', e);
continue;
}
if (active === 0) return;
await new Promise((r) => setTimeout(r, 1000));
continue;
}
const abort = new AbortController();
const shutdown = new AbortController();
@@ -364,18 +428,45 @@ export async function runPgliteSubagentsInline(
const keepalive = yieldDuringPhase
? setInterval(() => { yieldDuringPhase().catch(() => { /* best-effort */ }); }, 60_000)
: null;
// #2050: heartbeat the child's claim lock while the handler runs so a
// concurrent Postgres worker's handleStalled() sweep (all queues, not
// just its own) can't requeue a live child. A false return means the row
// was cancelled or reclaimed — abort the handler. Errors are swallowed
// (best-effort; the next tick retries), never an unhandledRejection.
const renewTimer = setInterval(() => {
queue.renewLock(job.id, lockToken, lockMs)
.then((ok) => {
if (!ok && !abort.signal.aborted) abort.abort(new Error('lock-renewal-failed'));
})
.catch(() => { /* best-effort; next tick retries */ });
}, Math.max(50, Math.floor(lockMs / 3)));
// Run, then record — separated so a completeJob connection error can't
// masquerade as a handler failure, and a failJob connection error can't
// escape the drain and strand the remaining children (worker.ts #1720
// parity: reconnect + retry the recording once; if it still fails, leave
// the row for the loop's own handleStalled to requeue after lock expiry).
let result: unknown;
let handlerErr: unknown;
let handlerRan = false;
try {
const result = await handler(context);
await queue.completeJob(
job.id,
lockToken,
result != null ? (typeof result === 'object' ? result as Record<string, unknown> : { value: result }) : undefined,
);
result = await handler(context);
handlerRan = true;
} catch (e) {
handlerErr = e;
}
const record = async (): Promise<void> => {
if (handlerRan) {
await queue.completeJob(
job.id,
lockToken,
result != null ? (typeof result === 'object' ? result as Record<string, unknown> : { value: result }) : undefined,
);
return;
}
// Timeout is terminal (handleTimeouts parity: stall → retry,
// timeout → dead), never a delayed retry.
const timedOut = abort.signal.aborted;
const errorText = timedOut ? 'timeout exceeded' : (e instanceof Error ? e.message : String(e));
const errorText = timedOut ? 'timeout exceeded' : (handlerErr instanceof Error ? handlerErr.message : String(handlerErr));
const attemptsExhausted = job.attempts_made + 1 >= job.max_attempts;
await queue.failJob(
job.id,
@@ -384,9 +475,30 @@ export async function runPgliteSubagentsInline(
timedOut || attemptsExhausted ? 'dead' : 'delayed',
0,
);
};
try {
try {
await record();
} catch (recordErr) {
if (!isRetryableConnError(recordErr)) throw recordErr;
sawConnError = true;
const msg = recordErr instanceof Error ? recordErr.message : String(recordErr);
process.stderr.write(`[dream] inline drain: recording job ${job.id} outcome hit a connection error; reconnecting and retrying once: ${msg}\n`);
await reconnectAfterConnectionError(engine, 'inline-record', recordErr);
try {
await record();
} catch (retryErr) {
// Leave the row to the loop's own handleStalled: the claim lock
// stops renewing (finally clears renewTimer), expires within
// lockMs, and the next iteration requeues it on a live pool.
const retryMsg = retryErr instanceof Error ? retryErr.message : String(retryErr);
process.stderr.write(`[dream] inline drain: outcome recording retry for job ${job.id} also failed (${retryMsg}); leaving the row for stall requeue\n`);
}
}
} finally {
if (timeoutTimer) clearTimeout(timeoutTimer);
if (keepalive) clearInterval(keepalive);
clearInterval(renewTimer);
}
}
}
@@ -572,12 +684,11 @@ export async function runPhaseSynthesize(
}
const queue = new MinionQueue(engine);
// PGLite children drain inline (no separate worker can open the embedded
// data-dir), so give them a private per-run queue: the inline drain must
// never claim unrelated 'default'-queue jobs a Postgres worker owns.
const childQueueName = engine.kind === 'pglite'
? `dream-inline-${Date.now()}-${randomUUID().slice(0, 8)}`
: 'default';
// #2050: children drain inline on BOTH engines (see runSubagentsInline),
// so give them a private per-run queue: the inline drain must never claim
// unrelated 'default'-queue jobs, and a 'default'-queue worker must never
// claim a child this parent is about to run itself.
const childQueueName = `dream-inline-${Date.now()}-${randomUUID().slice(0, 8)}`;
const childIds: number[] = [];
/** Map child job_id → chunk metadata for D6 orchestrator-side slug rewrite. */
const chunkInfo = new Map<number, { idx: number; hash6: string }>();
@@ -685,11 +796,11 @@ export async function runPhaseSynthesize(
}
}
// PGLite cannot run a separate Minions worker because the embedded DB
// holds an exclusive file lock. Drain this phase's private child queue
// inline so the parent observes terminal child states instead of polling
// waiters until subagentWaitTimeoutMs expires. No-op on Postgres.
await runPgliteSubagentsInline(engine, queue, childQueueName, opts.yieldDuringPhase);
// Drain this phase's private child queue inline so the parent observes
// terminal child states instead of polling waiters until
// subagentWaitTimeoutMs expires. Runs on BOTH engines — on Postgres the
// parent job otherwise deadlocks a fully-occupied worker (#2050).
await runSubagentsInline(engine, queue, childQueueName, opts.yieldDuringPhase);
// Wait for every child to reach a terminal state. Tick yieldDuringPhase
// every 5 min so the cycle lock TTL refreshes.
@@ -1709,6 +1820,6 @@ export const __testing = {
buildSynthesisPrompt,
stampDreamProvenance,
reverseWriteRefs,
runPgliteSubagentsInline,
runSubagentsInline,
loadSynthConfig,
};
+2
View File
@@ -99,6 +99,7 @@ export const BRAIN_CHECK_NAMES: ReadonlySet<string> = new Set([
'ocr_health',
'orphan_ratio',
'oversized_pages',
'pglite_scratch_probe',
'quarantined_pages',
'raw_provenance',
'flagged_pages',
@@ -171,6 +172,7 @@ export const OPS_CHECK_NAMES: ReadonlySet<string> = new Set([
'pgvector',
'pool_budget',
'progressive_batch_audit_health',
'provider_sunset',
'queue_health',
'reranker_health',
'rls',
+30 -1
View File
@@ -1,6 +1,6 @@
import type {
Page, PageInput, PageFilters, GetPageOpts,
Chunk, ChunkInput, StaleChunkRow, StalePageRow,
Chunk, ChunkInput, StaleChunkRow, StalePageRow, ChunklessPageRow,
SearchResult, SearchOpts,
Link, GraphNode, GraphPath, RelationalFanoutRow, RelationalFanoutOpts,
TimelineEntry, TimelineInput, TimelineOpts,
@@ -1087,6 +1087,35 @@ export interface BrainEngine {
// common denominator on the wire).
afterUpdatedAt?: string | null;
}): Promise<StaleChunkRow[]>;
/**
* Pre-flight count for the chunkless-page safety net: pages with
* non-empty `compiled_truth` AND/OR non-empty `timeline` both are
* chunked independently by the healer and ZERO `content_chunks` rows.
* `embed --stale` only scans `content_chunks` (embedding IS NULL) a
* page written directly via `putPage` that never got chunked has no
* chunk row to find, so it stays invisible to that scan forever.
* `opts.sourceId` scopes the count to a single source, matching
* `countStaleChunks`. Quarantined and `embed_skip` pages are excluded
* both are intentionally chunkless by design, not drift needing repair.
* See `ChunklessPageRow` for the full rationale.
*/
countChunklessPagesWithContent(opts?: { sourceId?: string }): Promise<number>;
/**
* List pages with non-empty `compiled_truth` and/or `timeline` and zero
* `content_chunks` rows (sibling of `countChunklessPagesWithContent`;
* same predicate). Keyset-paginated on `id` (mirrors
* `listStalePagesForExtraction`) pass the last row's `id` as
* `afterPageId` for the next page. Default `batchSize` 50 deliberately
* small (unlike the 2000-row default on chunk-metadata-only cursors
* elsewhere): each row here carries a FULL page body, so a large batch
* of large pages is a real memory concern this is a safety-net sweep for
* a rare drift case, not the primary bulk-import chunking path.
*/
listChunklessPagesWithContent(opts?: {
batchSize?: number;
afterPageId?: number;
sourceId?: string;
}): Promise<ChunklessPageRow[]>;
/**
* Delete every chunk for a page. Internal page-id lookup is sourceId-scoped
* when `opts.sourceId` is given; otherwise the bare-slug subquery returns
+24
View File
@@ -0,0 +1,24 @@
/**
* Shared "rebuild the DB pool after a retryable connection failure" helper.
*
* PostgresEngine exposes reconnect(); PGLite and test doubles may not. Absence
* is a no-op so non-Postgres callers preserve their legacy behavior.
*
* Extracted from MinionWorker's private method (#1491/#3555) so the inline
* child drain (#2050, cycle/synthesize.ts runSubagentsInline) recovers from
* the same transient pooler reaps instead of throwing out of the drain and
* stranding children in a per-run queue no worker will ever claim.
*/
export async function reconnectAfterConnectionError(
engine: unknown,
site: string,
error: unknown,
): Promise<void> {
const reconnect = (engine as { reconnect?: (ctx?: { error?: unknown }) => Promise<void> }).reconnect;
if (!reconnect) return;
try {
await reconnect.call(engine, { error });
} catch (re) {
console.error(`[minions] reconnect after ${site} error failed: ${re instanceof Error ? re.message : String(re)}`);
}
}
+8 -2
View File
@@ -42,7 +42,7 @@ import {
unlinkSync,
writeSync,
} from 'fs';
import { dirname } from 'path';
import { dirname, resolve } from 'path';
import type { BrainEngine } from '../engine.ts';
import { tryAcquireDbLock, type DbLockHandle } from '../db-lock.ts';
import { currentBrainId } from './worker-registry.ts';
@@ -574,7 +574,13 @@ export class MinionSupervisor {
// 5. Announce start.
this.emit('started', {
supervisor_pid: process.pid,
pid_file: this.opts.pidFile,
// Resolved to absolute at emit time (relative to THIS process's cwd,
// the only context in which a relative --pid-file was meaningful) so a
// later reader (e.g. `gbrain doctor`, possibly running from a
// different cwd) doesn't misresolve it. `this.opts.pidFile` itself
// stays as-given for this process's own reads/writes below, which are
// already correctly relative to this same cwd.
pid_file: resolve(this.opts.pidFile),
concurrency: this.opts.concurrency,
queue: this.opts.queue,
max_crashes: this.opts.maxCrashes,
+3 -10
View File
@@ -31,6 +31,7 @@ import {
} from './lock-renewal-tick.ts';
import { lockRenewalAudit } from '../audit/lock-renewal-audit.ts';
import { isRetryableConnError } from '../retry-matcher.ts';
import { reconnectAfterConnectionError as reconnectEngineAfterConnError } from './reconnect.ts';
/**
* Abort reasons that signal infrastructure failure (PgBouncer outage,
@@ -735,18 +736,10 @@ export class MinionWorker extends EventEmitter {
/**
* Rebuild the worker-owned DB pool after a retryable connection failure.
*
* PostgresEngine exposes reconnect(); PGLite and test doubles may not. Absence
* is a no-op so non-Postgres workers preserve their legacy behavior.
* Shared with the inline child drain (#2050) via minions/reconnect.ts.
*/
private async reconnectAfterConnectionError(site: string, error: unknown): Promise<void> {
const reconnect = (this.engine as { reconnect?: (ctx?: { error?: unknown }) => Promise<void> }).reconnect;
if (!reconnect) return;
try {
await reconnect.call(this.engine, { error });
} catch (re) {
console.error(`[worker] reconnect after ${site} error failed: ${re instanceof Error ? re.message : String(re)}`);
}
await reconnectEngineAfterConnError(this.engine, site, error);
}
/** RSS watchdog. Called from the per-job finally and the periodic timer.
+123 -17
View File
@@ -13,6 +13,7 @@
* - Legacy access_tokens fallback for backward compat
*/
import { AsyncLocalStorage } from 'node:async_hooks';
import type { Response } from 'express';
import type {
OAuthClientInformationFull,
@@ -235,14 +236,75 @@ interface GBrainOAuthProviderOptions {
* (operator-trusted, registers grants directly).
*/
allowClientCredentialsDcr?: boolean;
/**
* #2179: lower bound (seconds) for DCR-requested per-client token TTLs.
* Requests below it clamp up. Default DEFAULT_DCR_TTL_MIN_SECONDS (300).
*/
dcrTtlMinSeconds?: number;
/**
* #2179: upper bound (seconds) for DCR-requested per-client token TTLs.
* Requests above it clamp down. Unset defaults FAIL-CLOSED to
* max(tokenTtl, dcrTtlMinSeconds): an anonymous DCR registrant can never
* elect a longer-lived token than the operator's own --token-ttl unless
* the admin explicitly widened the window.
*/
dcrTtlMaxSeconds?: number;
}
// ---------------------------------------------------------------------------
// DCR token TTL (#2179)
// ---------------------------------------------------------------------------
/**
* Default lower clamp bound for DCR-requested token TTLs (#2179). Admins
* override via the `oauth.dcr_ttl_min_seconds` / `oauth.dcr_ttl_max_seconds`
* config keys, read once by `gbrain serve --http` at startup. There is
* deliberately NO fixed default max: an unset max derives fail-closed from
* the operator's --token-ttl (`max(tokenTtl, min)`), so a self-registering
* client can never out-live the server default without explicit admin opt-in.
*/
export const DEFAULT_DCR_TTL_MIN_SECONDS = 300; // 5 minutes
/**
* Clamp a DCR-requested token TTL into the admin-configured [min, max]
* window. Bounds are REQUIRED callers resolve them (fail-closed) first.
* Never rejects (#2179): out-of-range values clamp to the nearest bound.
* Non-integer requests floor; an inverted window collapses to the min bound.
*/
export function clampDcrTokenTtl(
requested: number,
min: number,
max: number,
): number {
const lo = Math.max(1, Math.floor(min));
const hi = Math.max(lo, Math.floor(max));
return Math.min(hi, Math.max(lo, Math.floor(requested)));
}
/**
* Request-scoped carrier for the `token_ttl_seconds` DCR extension field
* (#2179). The MCP SDK's /register handler validates the request body against
* a strict schema and STRIPS unknown members before they reach
* `clientsStore.registerClient`, so serve-http's /register middleware parses
* the raw body and runs the SDK chain inside this AsyncLocalStorage context;
* the store reads it back out at registration time. No context (CLI, admin
* API, programmatic registration) means "no TTL request" default behavior.
*/
export const dcrRegistrationContext = new AsyncLocalStorage<{ tokenTtlSeconds?: number }>();
// ---------------------------------------------------------------------------
// Clients Store
// ---------------------------------------------------------------------------
class GBrainClientsStore implements OAuthRegisteredClientsStore {
constructor(private sql: SqlQuery, private allowClientCredentialsDcr = false) {}
// #2179: DCR TTL bounds are required — the provider resolves fail-closed
// defaults (max bounded by tokenTtl); no permissive fallback lives here.
constructor(
private sql: SqlQuery,
private allowClientCredentialsDcr: boolean,
private dcrTtlMin: number,
private dcrTtlMax: number,
) {}
async getClient(clientId: string): Promise<OAuthClientInformationFull | undefined> {
const rows = await this.sql`
@@ -392,6 +454,27 @@ class GBrainClientsStore implements OAuthRegisteredClientsStore {
}
}
// #2179: optional `token_ttl_seconds` hint from the DCR request body,
// carried via dcrRegistrationContext (the SDK strips unknown body
// members). Fail-safe posture: absent or malformed → server default TTL;
// out-of-range → clamped into [dcrTtlMin, dcrTtlMax]; never rejected.
// Persist into oauth_clients.token_ttl (the same per-client override the
// admin API writes) and echo the EFFECTIVE value in the registration
// response so the caller can show the user what it actually got.
let effectiveTtl: number | undefined;
const requestedTtl = dcrRegistrationContext.getStore()?.tokenTtlSeconds;
if (typeof requestedTtl === 'number' && Number.isFinite(requestedTtl)) {
const clamped = clampDcrTokenTtl(requestedTtl, this.dcrTtlMin, this.dcrTtlMax);
try {
await this.sql`UPDATE oauth_clients SET token_ttl = ${clamped} WHERE client_id = ${clientId}`;
effectiveTtl = clamped;
} catch (e) {
// Pre-migration schema without the token_ttl column: keep the
// registration, but do NOT echo a TTL that wasn't persisted.
if (!isUndefinedColumnError(e, 'token_ttl')) throw e;
}
}
// Public clients: omit `client_secret` entirely from the response so
// the wire payload matches RFC 7591 §3.2.1 ("if the client is a
// public client, the authorization server MUST NOT issue a client
@@ -403,6 +486,9 @@ class GBrainClientsStore implements OAuthRegisteredClientsStore {
client_id_issued_at: now,
};
if (clientSecret) response.client_secret = clientSecret;
if (effectiveTtl !== undefined) {
(response as Record<string, unknown>).token_ttl_seconds = effectiveTtl;
}
return response;
}
}
@@ -420,10 +506,21 @@ export class GBrainOAuthProvider implements OAuthServerProvider {
constructor(options: GBrainOAuthProviderOptions) {
this.sql = options.sql;
this._clientsStore = new GBrainClientsStore(this.sql, options.allowClientCredentialsDcr === true);
this.dcrDisabled = options.dcrDisabled === true;
this.tokenTtl = options.tokenTtl || 3600;
this.refreshTtl = options.refreshTtl || 30 * 24 * 3600;
// #2179 fail-closed: an unset DCR max is bounded by the operator's own
// token TTL — never a fixed permissive ceiling — so a self-registering
// client cannot elect a longer-lived token than the server default
// unless the admin explicitly configured a wider window.
const dcrTtlMin = options.dcrTtlMinSeconds ?? DEFAULT_DCR_TTL_MIN_SECONDS;
const dcrTtlMax = options.dcrTtlMaxSeconds ?? Math.max(this.tokenTtl, dcrTtlMin);
this._clientsStore = new GBrainClientsStore(
this.sql,
options.allowClientCredentialsDcr === true,
dcrTtlMin,
dcrTtlMax,
);
}
get clientsStore(): OAuthRegisteredClientsStore {
@@ -931,20 +1028,10 @@ export class GBrainOAuthProvider implements OAuthServerProvider {
const requestedScopes = requestedScope ? parseScopeString(requestedScope) : allowedScopes;
const grantedScopes = requestedScopes.filter(s => hasScope(allowedScopes, s));
// Per-client TTL override (stored in oauth_clients.token_ttl)
// Column may not exist on PGLite/older schemas — graceful fallback
let clientTtl: number | undefined;
try {
const ttlRows = await this.sql`SELECT token_ttl FROM oauth_clients WHERE client_id = ${clientId}`;
if (ttlRows.length > 0 && ttlRows[0].token_ttl) clientTtl = Number(ttlRows[0].token_ttl);
} catch (e) {
// F5 hardening: same posture as the deleted_at probe above. Only the
// "column doesn't exist" path is a non-fatal fall-through.
if (!isUndefinedColumnError(e, 'token_ttl')) throw e;
}
// Client credentials: access token only, NO refresh token (RFC 6749 4.4.3)
return this.issueTokens(clientId, grantedScopes, undefined, false, clientTtl);
// Per-client TTL (oauth_clients.token_ttl) is applied inside issueTokens
// so all three grant paths honor it (#2179).
return this.issueTokens(clientId, grantedScopes, undefined, false);
}
// -------------------------------------------------------------------------
@@ -1204,17 +1291,36 @@ export class GBrainOAuthProvider implements OAuthServerProvider {
// Internal: Issue access + optional refresh tokens
// -------------------------------------------------------------------------
/**
* Per-client TTL override lookup (oauth_clients.token_ttl). Set by the
* admin API, the CLI, or a DCR `token_ttl_seconds` request (#2179).
* Column may not exist on older schemas graceful fallback to undefined.
*/
private async lookupClientTokenTtl(clientId: string): Promise<number | undefined> {
try {
const ttlRows = await this.sql`SELECT token_ttl FROM oauth_clients WHERE client_id = ${clientId}`;
if (ttlRows.length > 0 && ttlRows[0].token_ttl) return Number(ttlRows[0].token_ttl);
} catch (e) {
// F5 hardening posture: only the "column doesn't exist" path is a
// non-fatal fall-through.
if (!isUndefinedColumnError(e, 'token_ttl')) throw e;
}
return undefined;
}
private async issueTokens(
clientId: string,
scopes: string[],
resource: URL | undefined,
includeRefresh: boolean,
ttlOverride?: number,
): Promise<OAuthTokens> {
const accessToken = generateToken('gbrain_at_');
const accessHash = hashToken(accessToken);
const now = Math.floor(Date.now() / 1000);
const effectiveTtl = ttlOverride || this.tokenTtl;
// #2179: the per-client override lives here (not in individual grant
// handlers) so client_credentials, authorization_code AND refresh
// issuance all honor oauth_clients.token_ttl consistently.
const effectiveTtl = (await this.lookupClientTokenTtl(clientId)) || this.tokenTtl;
const accessExpiry = now + effectiveTtl;
await this.sql`
+5 -2
View File
@@ -10,7 +10,7 @@ import { clampSearchLimit } from './engine.ts';
import type { GBrainConfig } from './config.ts';
import type { PageType } from './types.ts';
import { importFromContent } from './import-file.ts';
import { writePageThrough } from './write-through.ts';
import { writePageThrough, type WriteThroughResult } from './write-through.ts';
import { hybridSearch, hybridSearchCached, stampContentFlags, stampUnverifiedExtractions } from './search/hybrid.ts';
import { expandQuery } from './search/expansion.ts';
import { dedupResults } from './search/dedup.ts';
@@ -1323,7 +1323,10 @@ const put_page: Operation = {
// Trust gating:
// - Subagent sandbox (viaSubagent without allowedSlugPrefixes) → DB-only.
// - All other writes → write-through.
let writeThrough: { written: boolean; path?: string; skipped?: string; error?: string } | undefined;
// put_page's own trust-gating produces two skip reasons ('subagent_sandbox',
// 'dry_run') that never come out of writePageThrough itself — widen the
// field rather than losing the commit/pushed/lastPushStatus typing.
let writeThrough: (Omit<WriteThroughResult, 'skipped'> & { skipped?: WriteThroughResult['skipped'] | 'subagent_sandbox' | 'dry_run' }) | undefined;
const isSandboxSubagent = ctx.viaSubagent === true
&& !(Array.isArray(ctx.allowedSlugPrefixes) && ctx.allowedSlugPrefixes.length > 0);
if (!ctx.dryRun && result.status !== 'error' && !isSandboxSubagent) {
+164 -1
View File
@@ -1,5 +1,10 @@
import { PGlite } from '@electric-sql/pglite';
import type { Transaction } from '@electric-sql/pglite';
// Engine-live path: static top-level imports (scratch probe, #2674) — the
// engine-dynamic-import guard forbids lazy `import()` here.
import { mkdtemp, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join as joinPath, resolve as resolvePath, sep as pathSep } from 'node:path';
// Engine-live path: static top-level import (no lazy `import()`). Supplies
// PGLite's WASM/fsBundle/extension assets embedded via `with { type: 'file' }`
// so a `bun build --compile` binary can serve a PGLite brain (Bun vfs #1340).
@@ -55,7 +60,7 @@ import { attemptWalRepairAndRetry, closeRepairEpisodeIfOpen, type WalRepairRecei
import { getFtsLanguage } from './fts-language.ts';
import type {
Page, PageInput, PageFilters, PageType,
Chunk, ChunkInput, StaleChunkRow, StalePageRow,
Chunk, ChunkInput, StaleChunkRow, StalePageRow, ChunklessPageRow,
SearchResult, SearchOpts,
Link, GraphNode, GraphPath,
TimelineEntry, TimelineInput, TimelineOpts,
@@ -87,6 +92,8 @@ import { buildSourceFactorCase, buildHardExcludeClause, buildVisibilityClause, b
import { unverifiedExtractionFragment } from './extraction-review.ts';
import { shouldExcludeFromOrphanReporting, loadOrphanPolicyOverrides } from './orphan-policy.ts';
import { LINK_EXTRACTOR_VERSION_TS } from './link-extraction.ts';
import { EMBED_SKIP_FILTER_FRAGMENT } from './embed-skip.ts';
import { QUARANTINE_FILTER_FRAGMENT } from './quarantine.ts';
import {
normalizeEngineColumn,
buildVectorCastFragment,
@@ -420,6 +427,95 @@ async function preservingProcessExitCode<T>(fn: () => Promise<T>): Promise<T> {
}
}
/**
* #2674 the scratch-store probe, the diagnostic half of the issue.
*
* PGLite reports only `Aborted()` to JS and prints the real PANIC (e.g.
* `could not locate a valid checkpoint record`) to its own stderr, so from
* the JS-visible error alone a damaged store is indistinguishable from a
* broken WASM runtime. The one thing that CAN tell them apart is opening a
* throwaway store on the same machine:
*
* - scratch store works the runtime is healthy; the REAL store is damaged.
* - scratch store fails too the runtime cannot start here at all.
*
* Stderr capture: PGLite 0.4.3 exposes no print/printErr hook on
* `PGliteOptions` (checked: only `debug`, which still writes to the
* process's own stderr), so we deliberately do NOT try to intercept the
* PANIC text monkey-patching process.stderr.write around an async WASM
* init is exactly the hack the classifier comments warn against. The
* probe's ok/fail outcome carries the diagnosis instead; `verdict` is
* populated from the JS-visible error for callers that want it.
*
* Runs the SAME code path as the real engine (PGlite.create with the
* embedded WASM/extension assets) but deliberately NOT PGLiteEngine.connect():
* connect wraps failures in buildPgliteInitErrorMessage, whose hint text
* would then pollute re-classification of the probe error.
*
* Safety: the scratch dir comes from mkdtemp under os.tmpdir() and is
* additionally checked against `realStorePath` (refuses any overlap in
* either direction) a bug here must never touch the brain being
* diagnosed. The dir is removed in a finally, success or failure.
*/
export interface PgliteScratchProbeResult {
ok: boolean;
duration_ms: number;
/** JS-visible error when ok=false (the PANIC itself lands on stderr, not here). */
error?: string;
verdict?: PgliteInitFailure;
}
export async function probePgliteScratchStore(
realStorePath?: string,
): Promise<PgliteScratchProbeResult> {
const scratchDir = await mkdtemp(joinPath(tmpdir(), 'gbrain-pglite-probe-'));
if (realStorePath) {
const real = resolvePath(realStorePath);
const scratch = resolvePath(scratchDir);
if (scratch === real || scratch.startsWith(real + pathSep) || real.startsWith(scratch + pathSep)) {
await rm(scratchDir, { recursive: true, force: true }).catch(() => {});
throw new Error(
`refusing to probe: scratch dir ${scratch} overlaps the real store ${real}`,
);
}
}
const started = Date.now();
let db: PGlite | null = null;
try {
// Same assets as the real engine's connect(): the embedded WASM/fsBundle/
// extension options (Bun vfs #1340) — a compiled binary's probe must
// exercise the same runtime path the real store open uses.
const embedded = await getEmbeddedPgliteOptions();
db = await preservingProcessExitCode(() =>
PGlite.create({
dataDir: joinPath(scratchDir, 'store'),
...embedded,
}),
);
await db.query(`CREATE TABLE scratch_probe (id int PRIMARY KEY, note text)`);
await db.query(`INSERT INTO scratch_probe VALUES (1, 'ok')`);
const res = await db.query<{ note: string }>(`SELECT note FROM scratch_probe WHERE id = 1`);
if (res.rows[0]?.note !== 'ok') {
throw new Error(`scratch store read-back mismatch: ${JSON.stringify(res.rows)}`);
}
return { ok: true, duration_ms: Date.now() - started };
} catch (err) {
const message = stringifyPgliteInitError(err);
return {
ok: false,
duration_ms: Date.now() - started,
error: message,
verdict: classifyPgliteInitError(message),
};
} finally {
if (db) {
try { await db.close(); } catch { /* probe store — nothing to save */ }
}
await rm(scratchDir, { recursive: true, force: true }).catch(() => {});
}
}
export class PGLiteEngine implements BrainEngine {
readonly kind = 'pglite' as const;
private _db: PGLiteDB | null = null;
@@ -2945,6 +3041,73 @@ export class PGLiteEngine implements BrainEngine {
return rows as unknown as StaleChunkRow[];
}
/**
* Shared chunkless-page-with-content predicate (mirrors PostgresEngine).
* Excludes quarantined + embed_skip pages both are intentionally
* chunkless by design, not drift the safety net should repair.
*/
private buildChunklessPagesWhere(opts?: { sourceId?: string }): { where: string; params: unknown[] } {
const conds: string[] = [
'p.deleted_at IS NULL',
// healChunklessPages chunks BOTH compiled_truth and timeline (mirrors
// embedPage) — a timeline-only page (rare but schema-legal) has
// something to heal even with compiled_truth = ''.
`(p.compiled_truth <> '' OR p.timeline <> '')`,
EMBED_SKIP_FILTER_FRAGMENT,
QUARANTINE_FILTER_FRAGMENT,
'NOT EXISTS (SELECT 1 FROM content_chunks cc WHERE cc.page_id = p.id)',
];
const params: unknown[] = [];
if (opts?.sourceId) {
params.push(opts.sourceId);
conds.push(`p.source_id = $${params.length}`);
}
return { where: conds.join(' AND '), params };
}
async countChunklessPagesWithContent(opts?: { sourceId?: string }): Promise<number> {
const { where, params } = this.buildChunklessPagesWhere(opts);
const { rows } = await this.db.query(
`SELECT count(*)::int AS count FROM pages p WHERE ${where}`,
params,
);
const count = (rows[0] as { count: number } | undefined)?.count ?? 0;
return Number(count);
}
async listChunklessPagesWithContent(opts?: {
batchSize?: number;
afterPageId?: number;
sourceId?: string;
}): Promise<ChunklessPageRow[]> {
const { where, params } = this.buildChunklessPagesWhere(opts);
let afterClause = '';
if (opts?.afterPageId != null) {
params.push(opts.afterPageId);
afterClause = ` AND p.id > $${params.length}`;
}
// Small default (unlike the 2000-row chunk-metadata cursors elsewhere):
// each row here carries a FULL page body. See engine.ts docstring.
const limit = opts?.batchSize ?? 50;
params.push(limit);
const limitIdx = params.length;
const { rows } = await this.db.query(
`SELECT p.id, p.slug, p.source_id, p.compiled_truth, p.timeline
FROM pages p
WHERE ${where}${afterClause}
ORDER BY p.id
LIMIT $${limitIdx}`,
params,
);
return (rows as Record<string, unknown>[]).map(r => ({
id: r.id as number,
slug: r.slug as string,
source_id: (r.source_id as string | undefined) ?? 'default',
compiled_truth: (r.compiled_truth as string | null) ?? '',
timeline: (r.timeline as string | null) ?? '',
}));
}
async deleteChunks(slug: string, opts?: { sourceId?: string }): Promise<void> {
const sourceId = opts?.sourceId ?? 'default';
// Source-qualify the page-id subquery; slugs are only unique per source.
+75 -1
View File
@@ -60,7 +60,7 @@ import { getFtsLanguage, applyFtsLanguagePolicy } from './fts-language.ts';
import { MARKDOWN_CHUNKER_VERSION } from './chunkers/recursive.ts';
import type {
Page, PageInput, PageFilters, PageType,
Chunk, ChunkInput, StaleChunkRow, StalePageRow,
Chunk, ChunkInput, StaleChunkRow, StalePageRow, ChunklessPageRow,
SearchResult, SearchOpts,
Link, GraphNode, GraphPath,
TimelineEntry, TimelineInput, TimelineOpts,
@@ -93,6 +93,8 @@ import { DELETE_BATCH_SIZE } from './engine-constants.ts';
import { SOURCE_CONFIG_OBJECT_SQL } from './source-config-sql.ts';
import { shouldExcludeFromOrphanReporting, loadOrphanPolicyOverrides } from './orphan-policy.ts';
import { LINK_EXTRACTOR_VERSION_TS } from './link-extraction.ts';
import { EMBED_SKIP_FILTER_FRAGMENT } from './embed-skip.ts';
import { QUARANTINE_FILTER_FRAGMENT } from './quarantine.ts';
function escapeSqlStringLiteral(value: string): string {
return value.replace(/'/g, "''");
@@ -2878,6 +2880,78 @@ export class PostgresEngine implements BrainEngine {
});
}
/**
* Shared chunkless-page-with-content predicate (mirrors PGLiteEngine).
* Excludes quarantined + embed_skip pages both are intentionally
* chunkless by design, not drift the safety net should repair.
*/
private buildChunklessPagesWhere(opts?: { sourceId?: string }): { where: string; params: unknown[] } {
const conds: string[] = [
'p.deleted_at IS NULL',
// healChunklessPages chunks BOTH compiled_truth and timeline (mirrors
// embedPage) — a timeline-only page (rare but schema-legal) has
// something to heal even with compiled_truth = ''.
`(p.compiled_truth <> '' OR p.timeline <> '')`,
EMBED_SKIP_FILTER_FRAGMENT,
QUARANTINE_FILTER_FRAGMENT,
'NOT EXISTS (SELECT 1 FROM content_chunks cc WHERE cc.page_id = p.id)',
];
const params: unknown[] = [];
if (opts?.sourceId) {
params.push(opts.sourceId);
conds.push(`p.source_id = $${params.length}`);
}
return { where: conds.join(' AND '), params };
}
async countChunklessPagesWithContent(opts?: { sourceId?: string }): Promise<number> {
const { where, params } = this.buildChunklessPagesWhere(opts);
// RLS scope binding (opt-in via GBRAIN_RLS_SCOPE_BINDING).
return await this.withScopedReadTransaction(undefined, opts?.sourceId, async (tx) => {
const rows = await tx.unsafe(
`SELECT count(*)::int AS count FROM pages p WHERE ${where}`,
params as Parameters<typeof tx.unsafe>[1],
);
return Number((rows[0] as { count?: number } | undefined)?.count ?? 0);
});
}
async listChunklessPagesWithContent(opts?: {
batchSize?: number;
afterPageId?: number;
sourceId?: string;
}): Promise<ChunklessPageRow[]> {
const { where, params } = this.buildChunklessPagesWhere(opts);
let afterClause = '';
if (opts?.afterPageId != null) {
params.push(opts.afterPageId);
afterClause = ` AND p.id > $${params.length}`;
}
// Small default (unlike the 2000-row chunk-metadata cursors elsewhere):
// each row here carries a FULL page body. See engine.ts docstring.
const limit = opts?.batchSize ?? 50;
params.push(limit);
const limitIdx = params.length;
// RLS scope binding (opt-in via GBRAIN_RLS_SCOPE_BINDING).
return await this.withScopedReadTransaction(undefined, opts?.sourceId, async (tx) => {
const rows = await tx.unsafe(
`SELECT p.id, p.slug, p.source_id, p.compiled_truth, p.timeline
FROM pages p
WHERE ${where}${afterClause}
ORDER BY p.id
LIMIT $${limitIdx}`,
params as Parameters<typeof tx.unsafe>[1],
);
return (rows as Record<string, unknown>[]).map(r => ({
id: r.id as number,
slug: r.slug as string,
source_id: (r.source_id as string | undefined) ?? 'default',
compiled_truth: (r.compiled_truth as string | null) ?? '',
timeline: (r.timeline as string | null) ?? '',
}));
});
}
async deleteChunks(slug: string, opts?: { sourceId?: string }): Promise<void> {
const sql = this.sql;
const sourceId = opts?.sourceId ?? 'default';
+2 -1
View File
@@ -27,12 +27,13 @@
import { createAuditWriter, computeIsoWeekFilename } from './audit/audit-writer.ts';
/** Stable error-classification union; matches RerankError.reason. */
/** Stable error-classification union for reranker fail-open audit rows. */
export type RerankFailureReason =
| 'auth'
| 'rate_limit'
| 'network'
| 'timeout'
| 'budget'
| 'payload_too_large'
| 'unknown';
+7
View File
@@ -1841,6 +1841,13 @@ export async function hybridSearchCached(
// resolves) into the cache key so a row written under one exclude
// policy can't be served to a lookup under another.
hardExcludes: resolveHardExcludes(opts?.exclude_slug_prefixes, opts?.include_slug_prefixes),
// #3515 — fold the EFFECTIVE detail level into the cache key. detail
// gates dedup, chunk-source filtering, and the compiled_truth boost, so
// a `--detail low` write (compiled-truth-only result set) must never be
// served to a default `medium` lookup. Resolve auto-detect the same way
// bare hybridSearch does (opts.detail ?? autoDetectDetail(query)) so an
// auto-detected `high` query keys like an explicit `high` one.
detail: opts?.detail ?? autoDetectDetail(query),
});
// Cache decision: opts.useCache (explicit) wins over global config; global
+28 -1
View File
@@ -779,7 +779,18 @@ export function attributeKnob<K extends keyof ModeBundle>(
// to cache.ttl_seconds, with no warning and no way for an operator to tell.
// Same one-time global cold-miss pattern as the bumps above; refills within
// cache.ttl_seconds (3600s default).
export const KNOBS_HASH_VERSION = 15;
//
// bump 15→16 (#3515): `detail` folds into the key via ctx.detail (det=).
// detail is result-affecting by design — it gates dedup, chunk-source
// filtering, and the compiled_truth boost — but was absent from the key, so
// a `--detail low` write (compiled-truth-only result set) was served to a
// default `medium` lookup for the whole TTL. Same contamination class as
// [CDX-4], floor_ratio (v=3), and relationalRetrieval (v=10). v=14 was
// claimed by #3514 (compiled_truth boost scope, #3430) and v=15 by the
// `fts=` fold (#3677), so this lands as v=16 per the D8 sequencing
// convention (see the v=4/v=5 note above). Same one-time global cold-miss
// pattern as the bumps above.
export const KNOBS_HASH_VERSION = 16;
/**
* v0.36 (D8 / CDX-2) second-arg context for the cache key. The
@@ -818,6 +829,17 @@ export interface KnobsHashContext {
* 'none' for legacy callers that don't thread excludes.
*/
hardExcludes?: string[];
/**
* v=16 (#3515): the EFFECTIVE detail level for this call per-call
* SearchOpts.detail, or the auto-detected level when the caller didn't
* specify (hybridSearchCached threads `opts.detail ?? autoDetectDetail(query)`,
* matching what bare hybridSearch resolves). detail gates dedup,
* chunk-source filtering, and the compiled_truth boost, so a detail=low
* write must never be served to a detail=medium lookup. Lives in ctx (not
* ResolvedSearchKnobs) because it's per-call, not a mode knob same path
* as col=/prov=. Undefined falls back to 'medium' (the documented default).
*/
detail?: 'low' | 'medium' | 'high';
}
export function knobsHash(
@@ -921,6 +943,11 @@ export function knobsHash(
// memoizes and validates against /^[a-z][a-z0-9_]*$/, so this stays a
// cheap, bounded string.
`fts=${getFtsLanguage()}`,
// v=16 addition (#3515, append-only): effective detail level. detail
// gates dedup, chunk-source filtering, and the compiled_truth boost, so
// a low write (compiled-truth-only set) must never be served to a
// medium/high lookup. Undefined falls back to 'medium' (the default).
`det=${ctx?.detail ?? 'medium'}`,
];
const h = createHash('sha256');
h.update(parts.join('|'));
+13 -2
View File
@@ -20,6 +20,7 @@
import { createHash } from 'crypto';
import type { SearchResult } from '../types.ts';
import { rerank as gatewayRerank, RerankError, type RerankInput, type RerankResult } from '../ai/gateway.ts';
import { BudgetExhausted } from '../budget/budget-tracker.ts';
import { logRerankFailure, type RerankFailureReason } from '../rerank-audit.ts';
export interface RerankerOpts {
@@ -44,6 +45,17 @@ function hashQuery(query: string): string {
return createHash('sha256').update(query, 'utf8').digest('hex').slice(0, 8);
}
function classifyRerankFailure(err: unknown): RerankFailureReason {
if (err instanceof RerankError) return err.reason;
if (
err instanceof BudgetExhausted ||
(err && typeof err === 'object' && (err as { tag?: unknown }).tag === 'BUDGET_EXHAUSTED')
) {
return 'budget';
}
return 'unknown';
}
/**
* Reorder the top `topNIn` results by reranker relevance score. The
* un-reranked tail (any rows past topNIn) preserves its original RRF
@@ -83,8 +95,7 @@ export async function applyReranker(
...(opts.model ? { model: opts.model } : {}),
});
} catch (err) {
const reason: RerankFailureReason =
err instanceof RerankError ? err.reason : 'unknown';
const reason = classifyRerankFailure(err);
const errorSummary = err instanceof Error ? err.message : String(err);
try {
logRerankFailure({
+18 -1
View File
@@ -27,6 +27,8 @@
* not a silent failure to exempt.
*/
import { FAILSAFE_SCHEMA, safeLoad as yamlSafeLoad } from 'js-yaml';
// ---------------------------------------------------------------------------
// Types
// ---------------------------------------------------------------------------
@@ -87,7 +89,8 @@ export interface ParsedFrontmatter {
* `readFileSync(path, 'utf-8')` at the boundary.
*/
export function parseSkillFrontmatter(content: string): ParsedFrontmatter | null {
const fmMatch = content.match(/^---\n([\s\S]*?)\n---/);
const normalized = content.replace(/\r\n/g, '\n');
const fmMatch = normalized.match(/^---\n([\s\S]*?)\n---/);
if (!fmMatch) return null;
const raw = fmMatch[1];
const out: ParsedFrontmatter = { raw };
@@ -133,6 +136,20 @@ export function parseSkillFrontmatter(content: string): ParsedFrontmatter | null
* top-level fields below it. Stops at the first non-indented line.
*/
function parseArrayField(raw: string, field: string): string[] | undefined {
try {
const parsed = yamlSafeLoad(raw, { schema: FAILSAFE_SCHEMA });
if (parsed && typeof parsed === 'object' && Object.hasOwn(parsed, field)) {
const value = (parsed as Record<string, unknown>)[field];
if (!Array.isArray(value)) return undefined;
return value
.filter((item): item is string => typeof item === 'string')
.map(item => item.trim())
.filter(Boolean);
}
} catch {
// Preserve the tolerant legacy behavior for partially malformed YAML.
}
// Inline form: `field: [a, b, c]` or `field: []`
const inlineRe = new RegExp(`^${field}:\\s*\\[([^\\]]*)\\]\\s*$`, 'm');
const inlineMatch = raw.match(inlineRe);
+22
View File
@@ -160,6 +160,28 @@ export function isSourceFederated(config: unknown): boolean {
return parsed.federated === true;
}
/**
* Three-way federation state for display (CLI `sources list`, etc.).
*
* `isSourceFederated` collapses to a boolean for the inclusion check (does
* this source show up in OTHER anchors' unqualified reads?), which is
* correctly strict 'unset' behaves like 'isolated' there. But 'unset' and
* 'isolated' are NOT interchangeable for display: only an explicit
* `federated: false` (`sources unfederate` / `--no-federated`) opts a source
* out of cross-source read mixing in both directions. A source that has
* simply never set the flag still widens its OWN unqualified reads to
* include the federated set (the #1434 sole-source convenience, pinned
* behavior see test/local-federated-search-scope.test.ts and
* test/unfederate-read-scope-2928.test.ts). Labeling it "isolated" overstates
* what the flag actually does.
*/
export function sourceFederationState(config: unknown): 'federated' | 'isolated' | 'unset' {
const raw = parseSourceConfig(config).federated;
if (raw === true) return 'federated';
if (raw === false) return 'isolated';
return 'unset';
}
/**
* Enumerate every source. Order: 'default' first, then alphabetical by id.
*
+5 -5
View File
@@ -515,9 +515,9 @@ export function clearFailures(sourceId: string, paths: string[]): void {
}
/**
* Acknowledge OPEN file failures (human `--skip-failed`). Scoped to one
* source when `sourceId` is given (never acks another source #1939 Codex
* #2). Sentinels (`<head>`) are NEVER acknowledged this way.
* Acknowledge OPEN or AUTO_SKIPPED file failures (human `--skip-failed`).
* Scoped to one source when `sourceId` is given (never acks another source
* #1939 Codex #2). Sentinels (`<head>`) are NEVER acknowledged this way.
*/
export function acknowledgeFailures(sourceId?: string): AcknowledgeResult {
return withLedgerLock(() => {
@@ -526,7 +526,7 @@ export function acknowledgeFailures(sourceId?: string): AcknowledgeResult {
let changed = 0;
const acked: SyncFailure[] = [];
for (const e of entries) {
if (e.state !== 'open') continue;
if (e.state !== 'open' && e.state !== 'auto_skipped') continue;
if (sourceId !== undefined && e.source_id !== sourceId) continue;
if (!isSkippablePath(e.path)) continue;
e.state = 'acknowledged';
@@ -543,7 +543,7 @@ export function acknowledgeFailures(sourceId?: string): AcknowledgeResult {
/**
* Mark the given chronic file paths `auto_skipped` (valve fired). Only OPEN,
* non-sentinel rows transition. Auto-skipped rows stay UNRESOLVED so doctor
* keeps warning until the file imports cleanly.
* keeps warning until the file imports cleanly or a human acknowledges them.
*/
export function autoSkipFailures(sourceId: string, paths: string[]): AcknowledgeResult {
if (paths.length === 0) return { count: 0, summary: [] };
+23
View File
@@ -623,6 +623,29 @@ export interface StaleChunkRow {
page_id: number;
}
/**
* A page with non-empty `compiled_truth` and/or `timeline` (both are
* chunked independently by the healer) but ZERO `content_chunks` rows,
* returned by `listChunklessPagesWithContent`. `embed --stale` scans
* `content_chunks` (embedding IS NULL) a page written directly via
* `putPage` that never went through the chunking step (e.g. an
* enrichment-generated entity stub) has no chunk row to go stale, so it is
* invisible to that scan forever. This is the safety-net detection: find
* such pages so `embed --stale` can chunk them and fold the resulting
* NULL-embedding chunks into the same run.
*
* Quarantined and `embed_skip` pages are excluded by the underlying query
* (`src/core/quarantine.ts` / `src/core/embed-skip.ts`) both are
* INTENTIONALLY chunkless by design (content-quality gate), not drift.
*/
export interface ChunklessPageRow {
id: number;
slug: string;
source_id: string;
compiled_truth: string;
timeline: string;
}
/**
* v0.42.7 (#1696) a page that needs link/timeline extraction, returned by
* `listStalePagesForExtraction`. Carries the page CONTENT (compiled_truth +
+118 -8
View File
@@ -22,12 +22,15 @@
*/
import { existsSync, statSync, mkdirSync, writeFileSync, renameSync, unlinkSync, readdirSync } from 'fs';
import { basename, dirname, join } from 'path';
import { basename, dirname, isAbsolute, join, relative, resolve } from 'path';
import { randomBytes } from 'crypto';
import type { BrainEngine } from './engine.ts';
import { serializePageToMarkdown, resolvePageFilePath } from './markdown.ts';
import { isWriteTargetContained } from './path-confine.ts';
import { isDurabilityHardened, commitWriteThroughFile } from './brain-repo-durability.ts';
import {
isDurabilityHardened, commitWriteThroughFile, currentBranch, getLastPushOutcome,
type PushLogOutcome,
} from './brain-repo-durability.ts';
/** Minimal logger surface — structurally compatible with operations.ts `Logger`. */
export interface WriteThroughLogger {
@@ -39,11 +42,28 @@ export interface WriteThroughResult {
path?: string;
/**
* True when the write was also committed to git (#2426). Only attempted on
* repos hardened via `gbrain sources harden` (durability hook installed);
* the hook then background-pushes the commit. Best-effort a false/absent
* value never blocks the write.
* repos hardened via `gbrain sources harden` (durability hook installed).
* Commit-only this says nothing about whether the commit ever reached the
* remote. Best-effort a false/absent value never blocks the write.
*/
committed?: boolean;
/**
* Set alongside `committed: true`. The actual push runs detached in the
* post-commit hook (see brain-repo-durability.ts), so at the moment this
* result is returned the outcome for THIS commit is genuinely unknown
* 'pending' is the only honest value. Check `lastPushStatus` (or
* $GBRAIN_HOME/brain-push.log directly) afterward to see whether pushes for
* this branch are landing.
*/
pushed?: 'pending';
/**
* Best-effort snapshot of the most recently logged push outcome for this
* branch (read from the hook's shared log), taken right after the commit
* above. It reflects push history UP TO that point not the push this
* write just queued so callers and health tooling can tell "pushes for
* this branch have been failing" apart from "this write committed fine".
*/
lastPushStatus?: PushLogOutcome;
/**
* Non-error reasons the file was not written:
* - no_repo_configured: the resolved target (source `local_path` or, for a
@@ -73,6 +93,58 @@ export interface WritePageThroughOpts {
logger?: WriteThroughLogger;
}
/**
* Vet a `pages.source_path` before it is trusted as a write target.
*
* The column is populated from the scanner's relative path at import time, so
* the normal value is a clean repo-relative `.md` path. This rejects the shapes
* that would make `join(root, value)` unsafe or nonsensical absolute paths,
* `..` traversal, NUL bytes, non-markdown artifacts, and blanks. Containment is
* still re-checked by `isWriteTargetContained` after the join; this is the
* cheap structural filter in front of it.
*/
function sanitizeRecordedSourcePath(raw: string | null | undefined): string | null {
if (!raw) return null;
const value = raw.trim();
if (!value || value.includes('\0')) return null;
if (!value.toLowerCase().endsWith('.md')) return null;
if (isAbsolute(value)) return null;
if (value.split(/[\\/]/).some((segment) => segment === '..')) return null;
return value;
}
/**
* Recover a page-root-relative write target from a `file://` `source_uri`.
*
* `gbrain capture --file` records the absolute input path as `source_uri` but
* does NOT set `source_path` (that column is the file-scanner's). So a file the
* user authored INSIDE the brain repo and then captured has no file of record,
* and the slug-derived fallback would mint a twin beside the very file that was
* just read. When the recorded URI points at a path under `pageRoot`, that path
* IS the file of record use it.
*
* Returns null for anything not a contained `.md` file so the caller falls back
* to the slug path.
*/
function recordedPathFromFileUri(sourceUri: string | null | undefined, pageRoot: string): string | null {
if (!sourceUri || !sourceUri.startsWith('file://')) return null;
let abs = sourceUri.slice('file://'.length);
if (!abs) return null;
// Percent-decode only when it looks encoded — the CLI stores raw paths, so a
// literal '%' in a filename must not be mangled.
if (/%[0-9A-Fa-f]{2}/.test(abs)) {
try {
abs = decodeURIComponent(abs);
} catch {
return null;
}
}
if (abs.includes('\0') || !abs.toLowerCase().endsWith('.md')) return null;
const rel = relative(resolve(pageRoot), resolve(abs));
if (!rel || isAbsolute(rel) || rel.split(/[\\/]/).some((segment) => segment === '..')) return null;
return rel;
}
/**
* Render the DB row for `slug` to markdown and atomically write it under
* `sync.repo_path`. Never throws failures are reported via the result's
@@ -104,11 +176,37 @@ export async function writePageThrough(
[sourceId],
);
const sourceLocalPath = srcRows[0]?.local_path ?? null;
// Prefer the page's recorded `source_path` — the ACTUAL file this row was
// imported from — over a slug-derived name. Deriving `<slug>.md` mints a
// SECOND file beside the original whenever the on-disk name isn't the slug,
// which is the common case for a human-authored vault: `Library/People/
// Steve Jobs.md` has slug `library/people/steve-jobs`, so a later put_page
// dropped a lowercase `steve-jobs.md` twin next to it. Two artifacts, one
// row, and the newer content in whichever the caller didn't expect.
//
// It also desyncs `gbrain sync`, which keys delete-reconcile on
// `source_path` (see collectMissingSourcePaths): the twin is invisible to
// reconcile, so deleting the ORIGINAL file deletes the page even though a
// file for it is still on disk.
//
// A NULL `source_path` means the page was born via put/capture and has no
// file of record yet — the slug-derived path stays correct for those.
const pathRows = await engine.executeRaw<{ source_path: string | null; source_uri: string | null }>(
`SELECT source_path, source_uri FROM pages WHERE source_id = $1 AND slug = $2 AND deleted_at IS NULL LIMIT 1`,
[sourceId, slug],
);
const recordedPath = sanitizeRecordedSourcePath(pathRows[0]?.source_path);
const recordedUri = pathRows[0]?.source_uri ?? null;
if (sourceLocalPath) {
if (!existsSync(sourceLocalPath) || !statSync(sourceLocalPath).isDirectory()) {
return { written: false, skipped: 'repo_not_found' };
}
filePath = join(sourceLocalPath, `${slug}.md`);
filePath = join(
sourceLocalPath,
recordedPath ?? recordedPathFromFileUri(recordedUri, sourceLocalPath) ?? `${slug}.md`,
);
writeRoot = sourceLocalPath;
} else {
const repoPath = await engine.getConfig('sync.repo_path');
@@ -127,7 +225,9 @@ export async function writePageThrough(
if (collide.length > 0) {
return { written: false, skipped: 'source_repo_belongs_to_other_source' };
}
filePath = resolvePageFilePath(repoPath, slug, sourceId);
const pageRoot = sourceId === 'default' ? repoPath : join(repoPath, '.sources', sourceId);
const knownPath = recordedPath ?? recordedPathFromFileUri(recordedUri, pageRoot);
filePath = knownPath ? join(pageRoot, knownPath) : resolvePageFilePath(repoPath, slug, sourceId);
writeRoot = repoPath;
}
@@ -204,13 +304,23 @@ export async function writePageThrough(
// post-commit hook background-pushes the commit. Best-effort: a commit
// failure never fails the write (the DB row + file are the durable sinks).
let committed = false;
let pushed: 'pending' | undefined;
let lastPushStatus: PushLogOutcome | undefined;
try {
if (isDurabilityHardened(writeRoot)) {
committed = commitWriteThroughFile(writeRoot, filePath, slug);
if (committed) {
pushed = 'pending';
lastPushStatus = getLastPushOutcome(currentBranch(writeRoot));
}
}
} catch { /* best-effort */ }
return { written: true, path: filePath, ...(committed ? { committed } : {}) };
return {
written: true,
path: filePath,
...(committed ? { committed, pushed, lastPushStatus } : {}),
};
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
opts.logger?.warn(`[write-through] failed for ${slug}: ${msg}`);
+1 -1
View File
@@ -1,6 +1,6 @@
# gbrain agent workspace — template
<!-- gbrain-template-stamp: 0.45.10.0 -->
<!-- gbrain-template-stamp: 0.45.11.0 -->
This repository is the **"Use this template"** distribution artifact for a
[gbrain](https://github.com/garrytan/gbrain) personal-agent workspace — the same
+196 -2
View File
@@ -8,13 +8,14 @@
* glue.
*/
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
import { describe, test, expect, beforeAll, afterAll, beforeEach, spyOn } from 'bun:test';
import * as fs from 'node:fs';
import * as path from 'node:path';
import * as os from 'node:os';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { MinionQueue } from '../src/core/minions/queue.ts';
import { __testing as agentTesting } from '../src/commands/agent.ts';
import { __testing as agentTesting, runAgentRun } from '../src/commands/agent.ts';
import { withEnv } from './helpers/with-env.ts';
import { parseSince } from '../src/commands/agent-logs.ts';
import { isProtectedJobName, PROTECTED_JOB_NAMES } from '../src/core/minions/protected-names.ts';
@@ -137,6 +138,17 @@ describe('parseRunFlags', () => {
const { flags } = agentTesting.parseRunFlags(['--fanout-manifest', '/tmp/m.json']);
expect(flags.fanoutManifest).toBe('/tmp/m.json');
});
test('#2922: --source parsed as a leading value-flag', () => {
const { flags, rest } = agentTesting.parseRunFlags(['--source', 'corporate', 'do', 'x']);
expect(flags.source).toBe('corporate');
expect(rest).toEqual(['do', 'x']);
});
test('#2922: --source missing its value throws a usage error', () => {
expect(() => agentTesting.parseRunFlags(['--source'])).toThrow(/requires a value/);
expect(() => agentTesting.parseRunFlags(['--source', '--detach', 'x'])).toThrow(/requires a value/);
});
});
describe('parseSince', () => {
@@ -266,6 +278,188 @@ describe('queue.add trusted-submit gate for subagent', () => {
});
});
describe('#2922: submit-time source resolution', () => {
beforeEach(async () => {
await engine.executeRaw(`DELETE FROM sources WHERE id != 'default'`);
await engine.unsetConfig('sources.default');
await engine.executeRaw(
`INSERT INTO sources (id, name) VALUES ('corporate', 'Corporate') ON CONFLICT (id) DO NOTHING`,
);
});
async function jobData(jobId: number): Promise<Record<string, unknown>> {
const rows = await engine.executeRaw<{ data: unknown }>(
`SELECT data FROM minion_jobs WHERE id = $1`, [jobId],
);
return typeof rows[0]!.data === 'string'
? JSON.parse(rows[0]!.data as string)
: rows[0]!.data as Record<string, unknown>;
}
async function onlyJobData(): Promise<Record<string, unknown>> {
const rows = await engine.executeRaw<{ id: number }>(
`SELECT id FROM minion_jobs WHERE name = 'subagent' ORDER BY id`,
);
expect(rows.length).toBe(1);
return jobData(rows[0]!.id);
}
test('explicit --source lands on SubagentHandlerData.source_id', async () => {
await withEnv({ GBRAIN_SOURCE: undefined }, async () => {
await runAgentRun(engine, ['--detach', '--source', 'corporate', 'write', 'a', 'page']);
const data = await onlyJobData();
expect(data.source_id).toBe('corporate');
expect(data.prompt).toBe('write a page');
});
});
test('no --source: sources.default (tier 5) is honored instead of the seed default', async () => {
await withEnv({ GBRAIN_SOURCE: undefined }, async () => {
await engine.setConfig('sources.default', 'corporate');
await runAgentRun(engine, ['--detach', 'write', 'a', 'page']);
const data = await onlyJobData();
expect(data.source_id).toBe('corporate');
});
});
test('GBRAIN_SOURCE env (tier 2) is honored', async () => {
await withEnv({ GBRAIN_SOURCE: 'corporate' }, async () => {
await runAgentRun(engine, ['--detach', 'write', 'a', 'page']);
const data = await onlyJobData();
expect(data.source_id).toBe('corporate');
});
});
test('no signal at all: resolves to the seed default (legacy behavior preserved)', async () => {
await withEnv({ GBRAIN_SOURCE: undefined }, async () => {
await runAgentRun(engine, ['--detach', 'write', 'a', 'page']);
const data = await onlyJobData();
expect(data.source_id).toBe('default');
});
});
test('fan-out children all carry the resolved source_id', async () => {
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'fanout-source-'));
try {
await withEnv({ GBRAIN_SOURCE: undefined }, async () => {
const manifestPath = path.join(tmp, 'm.json');
fs.writeFileSync(manifestPath, JSON.stringify([
{ prompt: 'chunk 1' }, { prompt: 'chunk 2' },
]));
await runAgentRun(engine, [
'--source', 'corporate', '--fanout-manifest', manifestPath, '--detach',
]);
const rows = await engine.executeRaw<{ id: number }>(
`SELECT id FROM minion_jobs WHERE name = 'subagent' ORDER BY id`,
);
expect(rows.length).toBe(2);
for (const r of rows) {
const data = await jobData(r.id);
expect(data.source_id).toBe('corporate');
}
});
} finally {
fs.rmSync(tmp, { recursive: true, force: true });
}
});
test('--source "" (empty explicit value) exits 2 without silently falling back', async () => {
const spy = spyOn(process, 'exit').mockImplementation(() => { throw new Error('EXIT'); });
const errSpy = spyOn(console, 'error').mockImplementation(() => {});
try {
await withEnv({ GBRAIN_SOURCE: undefined }, async () => {
try {
await runAgentRun(engine, ['--detach', '--source', '', 'write', 'a', 'page']);
throw new Error('expected runAgentRun to exit');
} catch (e: any) {
expect(e.message).toBe('EXIT');
}
});
expect(spy).toHaveBeenCalledWith(2);
const rows = await engine.executeRaw<{ id: number }>(
`SELECT id FROM minion_jobs WHERE name = 'subagent'`,
);
expect(rows.length).toBe(0);
} finally {
spy.mockRestore();
errSpy.mockRestore();
}
});
test('--source __all__ is rejected (subagent writes must target exactly one source)', async () => {
const spy = spyOn(process, 'exit').mockImplementation(() => { throw new Error('EXIT'); });
const errSpy = spyOn(console, 'error').mockImplementation(() => {});
try {
await withEnv({ GBRAIN_SOURCE: undefined }, async () => {
try {
await runAgentRun(engine, ['--detach', '--source', '__all__', 'write', 'a', 'page']);
throw new Error('expected runAgentRun to exit');
} catch (e: any) {
expect(e.message).toBe('EXIT');
}
});
expect(spy).toHaveBeenCalledWith(2);
const rows = await engine.executeRaw<{ id: number }>(
`SELECT id FROM minion_jobs WHERE name = 'subagent'`,
);
expect(rows.length).toBe(0);
} finally {
spy.mockRestore();
errSpy.mockRestore();
}
});
test('--source pointing at a nonexistent id surfaces a clean error, not a stack trace', async () => {
const spy = spyOn(process, 'exit').mockImplementation(() => { throw new Error('EXIT'); });
const errSpy = spyOn(console, 'error').mockImplementation(() => {});
try {
await withEnv({ GBRAIN_SOURCE: undefined }, async () => {
try {
await runAgentRun(engine, ['--detach', '--source', 'does-not-exist', 'write', 'a', 'page']);
throw new Error('expected runAgentRun to exit');
} catch (e: any) {
expect(e.message).toBe('EXIT');
}
});
expect(spy).toHaveBeenCalledWith(1);
expect(errSpy.mock.calls.some(call => String(call[0]).includes('not found'))).toBe(true);
const rows = await engine.executeRaw<{ id: number }>(
`SELECT id FROM minion_jobs WHERE name = 'subagent'`,
);
expect(rows.length).toBe(0);
} finally {
spy.mockRestore();
errSpy.mockRestore();
}
});
test('--source pointing at an archived source is rejected with a restore hint', async () => {
const spy = spyOn(process, 'exit').mockImplementation(() => { throw new Error('EXIT'); });
const errSpy = spyOn(console, 'error').mockImplementation(() => {});
try {
await engine.executeRaw(`UPDATE sources SET archived = true WHERE id = 'corporate'`);
await withEnv({ GBRAIN_SOURCE: undefined }, async () => {
try {
await runAgentRun(engine, ['--detach', '--source', 'corporate', 'write', 'a', 'page']);
throw new Error('expected runAgentRun to exit');
} catch (e: any) {
expect(e.message).toBe('EXIT');
}
});
expect(spy).toHaveBeenCalledWith(1);
expect(errSpy.mock.calls.some(call => String(call[0]).includes('archived'))).toBe(true);
const rows = await engine.executeRaw<{ id: number }>(
`SELECT id FROM minion_jobs WHERE name = 'subagent'`,
);
expect(rows.length).toBe(0);
} finally {
await engine.executeRaw(`UPDATE sources SET archived = false WHERE id = 'corporate'`);
spy.mockRestore();
errSpy.mockRestore();
}
});
});
describe('fan-out manifest shape (integration)', () => {
test('fanout-manifest with 3 entries creates 3 subagent children + 1 aggregator', async () => {
// Manually replicate what runAgentRun does for --fanout-manifest > 1.
+237
View File
@@ -0,0 +1,237 @@
/**
* #2050 a patterns/synthesize parent job that blocks on its own subagent
* child must not deadlock a fully-occupied Postgres drain worker.
*
* Pre-fix, the inline child drain was PGLite-only (runPgliteSubagentsInline
* returned early unless engine.kind === 'pglite') and Postgres children were
* submitted to the 'default' queue. Autopilot spawns its drain worker with no
* --concurrency (resolves to 1), so the parent occupied the only slot, the
* child was never claimed, and the phase burned the whole
* subagent_wait_timeout_ms window before cancelling its own child every
* cycle, forever.
*
* These tests run the REAL phase/drain code against a PGLite engine masked as
* kind='postgres' (only the `kind` property is intercepted; every method call
* hits the real engine), which is exactly the branch the production Postgres
* environment takes. All assertions are behavioral: on unmodified master they
* fail with outcome 'timeout' / a never-claimed 'waiting' child on the
* 'default' queue.
*/
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
import { mkdtempSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import type { BrainEngine } from '../src/core/engine.ts';
import { resetPgliteState } from './helpers/reset-pglite.ts';
import { runPhasePatterns } from '../src/core/cycle/patterns.ts';
import { MinionQueue } from '../src/core/minions/queue.ts';
import { withEnv } from './helpers/with-env.ts';
let engine: PGLiteEngine;
let schemaVersion: string;
/** Mask a PGLite engine as Postgres: only `kind` is intercepted, every
* method executes on the real engine (bound to the target so internal
* state is untouched). */
function maskAsPostgres(target: PGLiteEngine): BrainEngine {
return new Proxy(target, {
get(t, prop) {
if (prop === 'kind') return 'postgres';
const v = Reflect.get(t, prop);
return typeof v === 'function' ? v.bind(t) : v;
},
}) as unknown as BrainEngine;
}
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({ database_url: '' });
await engine.initSchema();
schemaVersion = (await engine.getConfig('version')) ?? '7';
}, 60_000);
afterAll(async () => {
await engine.disconnect();
});
beforeEach(async () => {
await resetPgliteState(engine);
await engine.setConfig('version', schemaVersion);
});
async function seedReflections(): Promise<void> {
for (let i = 0; i < 3; i++) {
await engine.executeRaw(
`INSERT INTO pages (slug, type, title, compiled_truth)
VALUES ($1, 'note', $2, $3)`,
[
`wiki/personal/reflections/2026-07-0${i + 1}-reflection`,
`Reflection ${i + 1}`,
`Recurring theme fixture number ${i + 1}.`,
],
);
}
}
describe('#2050 — patterns parent must not deadlock on its own child (postgres path)', () => {
test('child is drained inline on a private queue instead of waiting out the parent', async () => {
const brainDir = mkdtempSync(join(tmpdir(), 'gbrain-2050-patterns-'));
try {
await seedReflections();
// Small wait window: on master the child is never claimed (no worker in
// this process, parent would hold the only slot in production), so the
// phase burns this whole window and reports outcome 'timeout'.
await engine.setConfig('dream.patterns.subagent_wait_timeout_ms', '2000');
const pgAlike = maskAsPostgres(engine);
const result = await withEnv({ ANTHROPIC_API_KEY: 'sk-ant-test' }, () =>
runPhasePatterns(pgAlike, { brainDir, dryRun: false }),
);
// Master: child_outcome 'timeout' + PATTERNS_CHILD_TIMEOUT (self-deadlock,
// then the phase cancels its own child). Fixed: the inline drain actually
// ran the child (the fake API key fails fast → 'dead'), no deadlock.
expect(result.details.child_outcome).toBe('dead');
expect(result.error?.code).toBe('PATTERNS_CHILD_DEAD');
const jobs = await engine.executeRaw<{ queue: string; status: string }>(
`SELECT queue, status FROM minion_jobs WHERE name = 'subagent' ORDER BY id DESC LIMIT 1`,
);
expect(jobs).toHaveLength(1);
// Master: queue 'default' (claimable by — and deadlocked behind — the
// same worker running the parent), status 'cancelled'.
expect(jobs[0].queue).toStartWith('dream-inline-');
expect(jobs[0].status).toBe('dead');
} finally {
rmSync(brainDir, { recursive: true, force: true });
}
}, 60_000);
test('inline drain runs children on a non-pglite engine and heartbeats their lock', async () => {
const { __testing } = await import('../src/core/cycle/synthesize.ts');
const drain = (__testing as Record<string, unknown>).runSubagentsInline
?? (__testing as Record<string, unknown>).runPgliteSubagentsInline;
const pgAlike = maskAsPostgres(engine);
const queue = new MinionQueue(pgAlike);
const job = await queue.add(
'subagent',
{ prompt: 'noop', model: 'anthropic:claude-sonnet-4-5', max_turns: 1 },
{ queue: 'inline-2050-test', max_stalled: 3 },
{ allowProtectedSubmit: true },
);
// Stub handler outliving the (shortened) claim lock: sleeps past lockMs,
// then runs the same handleStalled() sweep a concurrent Postgres worker
// fires on a timer. With the heartbeat the lock is fresh and the sweep
// must NOT touch the running child; without it the child would be
// requeued mid-run (stall churn → dead after max_stalled).
let sweptWhileRunning = 0;
const stub = async () => {
await new Promise((r) => setTimeout(r, 1400));
const { requeued, dead } = await queue.handleStalled();
sweptWhileRunning = [...requeued, ...dead].filter((j) => j.id === job.id).length;
return { ok: true };
};
// Master behavioral failure: runPgliteSubagentsInline no-ops on a
// kind='postgres' engine, the stub never runs, the job stays 'waiting'.
await (drain as (
e: BrainEngine, q: MinionQueue, name: string,
y?: () => Promise<void>, h?: unknown, lockMs?: number,
) => Promise<void>)(pgAlike, queue, 'inline-2050-test', undefined, stub, 1000);
const after = await queue.getJob(job.id);
expect(after?.status).toBe('completed');
expect(sweptWhileRunning).toBe(0);
}, 30_000);
// #3555 interaction: the drain's queue ops must survive a transient pooler
// reap (reconnect + retry) instead of throwing out of the loop and
// stranding children in the per-run private queue no worker will claim.
/** Mask as Postgres AND expose a counting reconnect() for the drain's
* recovery path (the real PGLite engine has no reconnect method). */
function maskAsPostgresWithReconnect(target: PGLiteEngine, onReconnect: () => void): BrainEngine {
return new Proxy(target, {
get(t, prop) {
if (prop === 'kind') return 'postgres';
if (prop === 'reconnect') return async () => { onReconnect(); };
const v = Reflect.get(t, prop);
return typeof v === 'function' ? v.bind(t) : v;
},
}) as unknown as BrainEngine;
}
const connError = () =>
Object.assign(new Error('write CONNECTION_ENDED supavisor reaped the socket'), { code: 'CONNECTION_ENDED' });
test('a transient connection error on claim reconnects and the child still completes', async () => {
const { __testing } = await import('../src/core/cycle/synthesize.ts');
const drain = (__testing as Record<string, unknown>).runSubagentsInline;
let reconnects = 0;
const pgAlike = maskAsPostgresWithReconnect(engine, () => { reconnects++; });
const queue = new MinionQueue(pgAlike);
const job = await queue.add(
'subagent',
{ prompt: 'noop', model: 'anthropic:claude-sonnet-4-5', max_turns: 1 },
{ queue: 'inline-2050-conn-claim', max_stalled: 3 },
{ allowProtectedSubmit: true },
);
let failuresLeft = 1;
const realClaim = queue.claim.bind(queue);
queue.claim = (async (...args: Parameters<MinionQueue['claim']>) => {
if (failuresLeft-- > 0) throw connError();
return realClaim(...args);
}) as MinionQueue['claim'];
// Pre-fix: the bare-await claim throws out of the drain, the child
// strands 'waiting' in the private queue, and this await rejects.
await (drain as (
e: BrainEngine, q: MinionQueue, name: string,
y?: () => Promise<void>, h?: unknown, lockMs?: number,
) => Promise<void>)(pgAlike, queue, 'inline-2050-conn-claim', undefined, async () => ({ ok: true }), 1000);
const after = await queue.getJob(job.id);
expect(after?.status).toBe('completed');
expect(reconnects).toBe(1);
}, 30_000);
test('a transient connection error recording the outcome retries once and the child still completes', async () => {
const { __testing } = await import('../src/core/cycle/synthesize.ts');
const drain = (__testing as Record<string, unknown>).runSubagentsInline;
let reconnects = 0;
const pgAlike = maskAsPostgresWithReconnect(engine, () => { reconnects++; });
const queue = new MinionQueue(pgAlike);
const job = await queue.add(
'subagent',
{ prompt: 'noop', model: 'anthropic:claude-sonnet-4-5', max_turns: 1 },
{ queue: 'inline-2050-conn-record', max_stalled: 3 },
{ allowProtectedSubmit: true },
);
let failuresLeft = 1;
const realComplete = queue.completeJob.bind(queue);
queue.completeJob = (async (...args: Parameters<MinionQueue['completeJob']>) => {
if (failuresLeft-- > 0) throw connError();
return realComplete(...args);
}) as MinionQueue['completeJob'];
// Pre-fix: completeJob's rejection landed in the drain's catch, which
// called failJob for a handler that SUCCEEDED (misrecorded outcome) — or
// escaped the drain entirely if failJob also hit the dead pool.
await (drain as (
e: BrainEngine, q: MinionQueue, name: string,
y?: () => Promise<void>, h?: unknown, lockMs?: number,
) => Promise<void>)(pgAlike, queue, 'inline-2050-conn-record', undefined, async () => ({ ok: true }), 1000);
const after = await queue.getJob(job.id);
expect(after?.status).toBe('completed');
expect(reconnects).toBe(1);
}, 30_000);
});
+11 -1
View File
@@ -80,6 +80,13 @@ describe('autopilot.ts ↔ dispatchPerSource wiring', () => {
expect(freshnessBlock).toContain('pull: sourceConfigHasRemoteUrl(src.config)');
});
test('#4046: targeted dispatch scopes stable recommendation keys to the interval', () => {
expect(AUTOPILOT_SRC).toContain(
'idempotency_key: autopilotRemediationIdempotencyKey(step.idempotency_key, slot)',
);
expect(AUTOPILOT_SRC).not.toContain('idempotency_key: step.idempotency_key,');
});
test('#2781: dispatchGlobalMaintenance gets the full-cycle floor, not the outer (non-full-cycle) timeoutMs', () => {
// Live #2781 regression, found in review: dispatchGlobalMaintenance's
// call used the object-shorthand `timeoutMs`, which resolved to the
@@ -101,9 +108,12 @@ describe('autopilot.ts ↔ dispatchPerSource wiring', () => {
expect(dispatchGlobalCall).not.toMatch(/\{\s*repoPath,\s*slot,\s*timeoutMs,/);
});
test('updates lastFullCycleAt on dispatch (so the 60-min floor is honored)', () => {
test('updates lastFullCycleAt after dispatch or an all-fresh restart check', () => {
// After the dispatchPerSource call, the lastFullCycleAt module var
// must update so the next tick doesn't immediately re-fan-out.
expect(AUTOPILOT_SRC).toMatch(
/result\.dispatched\.length > 0 \|\| result\.legacy_fallback \|\| result\.all_sources_fresh/,
);
expect(AUTOPILOT_SRC).toMatch(/lastFullCycleAt\s*=\s*Date\.now\(\)/);
});
+78
View File
@@ -18,6 +18,11 @@ import {
resolveFanoutMax,
dispatchPerSource,
} from '../src/commands/autopilot-fanout.ts';
import {
autopilotRemediationIdempotencyKey,
shouldRunAutopilotFullCycle,
shouldSleepHealthyAutopilot,
} from '../src/commands/autopilot-remediation-policy.ts';
import type { SourceRow, BrainEngine } from '../src/core/engine.ts';
function src(id: string, last_full_cycle_at?: string | null, extra: Record<string, unknown> = {}): SourceRow {
@@ -74,6 +79,66 @@ describe('isSourceStale', () => {
});
});
describe('Autopilot remediation policy (#4046)', () => {
test('targeted remediation keys reopen in each dispatch interval', () => {
const recommendationKey = 'default:sync:deadbeef';
const firstSlot = '2026-08-13T06:00:00.000Z';
const nextSlot = '2026-08-13T06:05:00.000Z';
expect(autopilotRemediationIdempotencyKey(recommendationKey, firstSlot)).toBe(
'default:sync:deadbeef:autopilot:2026-08-13T06:00:00.000Z',
);
expect(autopilotRemediationIdempotencyKey(recommendationKey, nextSlot)).not.toBe(
autopilotRemediationIdempotencyKey(recommendationKey, firstSlot),
);
});
test('an overdue full cycle wins even while a small remediation plan exists', () => {
expect(shouldRunAutopilotFullCycle({
score: 94,
planLength: 2,
estimatedSeconds: 30,
minutesSinceLastFull: 61,
})).toBe(true);
});
test('a fresh small remediation plan stays targeted', () => {
expect(shouldRunAutopilotFullCycle({
score: 94,
planLength: 2,
estimatedSeconds: 30,
minutesSinceLastFull: 10,
})).toBe(false);
});
test('only a fresh healthy brain with no plan sleeps', () => {
expect(shouldSleepHealthyAutopilot(95, 0, 59)).toBe(true);
expect(shouldSleepHealthyAutopilot(95, 0, 60)).toBe(false);
expect(shouldSleepHealthyAutopilot(95, 1, 10)).toBe(false);
});
test('large, slow, or severely degraded plans still use the full cycle', () => {
expect(shouldRunAutopilotFullCycle({
score: 90,
planLength: 4,
estimatedSeconds: 30,
minutesSinceLastFull: 10,
})).toBe(true);
expect(shouldRunAutopilotFullCycle({
score: 90,
planLength: 2,
estimatedSeconds: 300,
minutesSinceLastFull: 10,
})).toBe(true);
expect(shouldRunAutopilotFullCycle({
score: 69,
planLength: 1,
estimatedSeconds: 30,
minutesSinceLastFull: 10,
})).toBe(true);
});
});
describe('selectSourcesForDispatch', () => {
const NOW = Date.parse('2026-05-22T12:00:00.000Z');
const fresh = (id: string, agoMin: number) =>
@@ -313,6 +378,19 @@ describe('dispatchPerSource — integration with stubbed engine + queue', () =>
const result = await dispatchPerSource(engine, queue, fanoutOpts);
expect(result.dispatched.length).toBe(0);
expect(result.skipped_fresh.length).toBe(2);
expect(result.all_sources_fresh).toBe(true);
expect(added.length).toBe(0);
});
test('a failed stale-source submission is not misclassified as all fresh', async () => {
const { engine, fanoutOpts } = makeStubs([src('stale')]);
const queue = {
add: async () => { throw new Error('queue unavailable'); },
} as unknown as Parameters<typeof dispatchPerSource>[1];
const result = await dispatchPerSource(engine, queue, fanoutOpts);
expect(result.dispatched).toEqual([]);
expect(result.all_sources_fresh).toBe(false);
});
});
+62 -6
View File
@@ -1,8 +1,13 @@
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
import { mkdtempSync, rmSync, writeFileSync } from 'fs';
import { mkdtempSync, rmSync, utimesSync, writeFileSync } from 'fs';
import { join } from 'path';
import { tmpdir } from 'os';
import { decideLockAcquisition, isPidAlive } from '../src/commands/autopilot.ts';
import {
AUTOPILOT_FOREIGN_PID_TAKEOVER_GRACE_MS,
decideLockAcquisition,
isPidAlive,
} from '../src/commands/autopilot.ts';
import { looksLikeGbrainAutopilotCommand } from '../src/core/autopilot-lock.ts';
let tmp: string;
let lockPath: string;
@@ -41,11 +46,49 @@ describe('decideLockAcquisition', () => {
});
});
test('keeps a lock whose holder is alive regardless of age', () => {
writeFileSync(lockPath, String(process.pid));
expect(decideLockAcquisition(lockPath, process.pid + 100_000)).toEqual({
test('keeps a lock whose holder is a live gbrain autopilot process', () => {
writeFileSync(lockPath, '1234');
expect(decideLockAcquisition(lockPath, process.pid, {
isPidAlive: (pid) => pid === 1234,
readProcessCommand: () => 'gbrain autopilot --repo repo',
})).toEqual({
action: 'exit',
holderPid: process.pid,
holderPid: 1234,
});
});
test('keeps a fresh lock when the live PID command is unrecognized', () => {
writeFileSync(lockPath, '1234');
expect(decideLockAcquisition(lockPath, process.pid, {
isPidAlive: (pid) => pid === 1234,
readProcessCommand: () => '/sbin/launchd',
})).toEqual({
action: 'exit',
holderPid: 1234,
});
});
test('takes over a stale lock when the PID was reused by a foreign process', () => {
writeFileSync(lockPath, '1234');
const stale = new Date(Date.now() - AUTOPILOT_FOREIGN_PID_TAKEOVER_GRACE_MS - 1000);
utimesSync(lockPath, stale, stale);
expect(decideLockAcquisition(lockPath, process.pid, {
isPidAlive: (pid) => pid === 1234,
readProcessCommand: () => '/sbin/launchd',
})).toEqual({
action: 'takeover',
reason: 'foreign pid 1234 with stale lock',
});
});
test('keeps a live lock when process identity cannot be inspected', () => {
writeFileSync(lockPath, '1234');
expect(decideLockAcquisition(lockPath, process.pid, {
isPidAlive: (pid) => pid === 1234,
readProcessCommand: () => null,
})).toEqual({
action: 'exit',
holderPid: 1234,
});
});
@@ -56,3 +99,16 @@ describe('decideLockAcquisition', () => {
expect(decideLockAcquisition(lockPath, process.pid).action).toBe('takeover');
});
});
describe('looksLikeGbrainAutopilotCommand', () => {
test('matches packaged and source-tree autopilot invocations', () => {
expect(looksLikeGbrainAutopilotCommand('gbrain autopilot --repo repo')).toBe(true);
expect(looksLikeGbrainAutopilotCommand('./gbrain/src/cli.ts autopilot')).toBe(true);
expect(looksLikeGbrainAutopilotCommand('bun src/cli.ts autopilot --repo repo')).toBe(true);
});
test('rejects unrelated live processes', () => {
expect(looksLikeGbrainAutopilotCommand('/sbin/launchd')).toBe(false);
expect(looksLikeGbrainAutopilotCommand('/usr/bin/python worker.py')).toBe(false);
});
});
+142
View File
@@ -23,6 +23,7 @@ import type { ExecRunner } from '../src/core/bootstrap/repo.ts';
import { attachWorkspace } from '../src/core/bootstrap/attach.ts';
import { readReceipt, receiptPath, writeManifest, type InstallReceipt } from '../src/core/bootstrap/format.ts';
import { GBRAIN_HOOK_MARKER_KEY, GBRAIN_HOOK_MARKER_VALUE } from '../src/core/bootstrap/host-specs.ts';
import { deriveWorkspaceSourceId } from '../src/core/bootstrap/verify.ts';
import { initState, setAnswer, skipAnswer, confirm, readBackHash } from '../src/core/bootstrap/interview.ts';
let tmpParent: string; // GBRAIN_HOME parent (configDir appends .gbrain)
@@ -122,6 +123,147 @@ describe('consent-skip is a decline (HOOKS_CONSENT)', () => {
}, 30_000);
});
describe('hooks previews the source_id + creates brain/ eagerly [defect: multi-round-trip source registration]', () => {
// Self-contained fixtures (same pattern as the flip block below): the
// file-level `ws` is shared by other describes, so this needs its own.
const scratch: string[] = [];
function freshWorkspace(): { fws: string; fhome: string; fparent: string } {
const fparent = mkdtempSync(join(tmpdir(), 'gb-srcid-'));
const fhome = join(fparent, '.gbrain');
mkdirSync(fhome, { recursive: true });
const fws = mkdtempSync(join(tmpdir(), 'gb-srcid-ws-'));
scratch.push(fparent, fws);
const prev = process.env.GBRAIN_HOME;
process.env.GBRAIN_HOME = fparent;
try {
expect(initState(fws).ok).toBe(true);
for (const [key, value] of Object.entries(REQUIRED_ANSWERS)) {
const r = setAnswer(fws, key, value);
if (!r.ok) throw new Error(r.message);
}
expect(setAnswer(fws, 'MCP_SCOPE', 'project').ok).toBe(true);
const h = readBackHash(fws);
if (!h.ok) throw new Error(h.message);
expect(confirm(fws, h.hash).ok).toBe(true);
} finally {
if (prev === undefined) delete process.env.GBRAIN_HOME;
else process.env.GBRAIN_HOME = prev;
}
return { fws, fhome, fparent };
}
afterAll(() => {
for (const d of scratch) rmSync(d, { recursive: true, force: true });
});
async function withHome<T>(parent: string, fn: () => Promise<T>): Promise<T> {
const prev = process.env.GBRAIN_HOME;
process.env.GBRAIN_HOME = parent;
try {
return await fn();
} finally {
if (prev === undefined) delete process.env.GBRAIN_HOME;
else process.env.GBRAIN_HOME = prev;
}
}
test('hooks prints the exact `sources add` command up front and creates brain/ before verify ever runs', async () => {
const { fws, fparent } = freshWorkspace();
const brainDir = join(fws, 'brain');
const out = await withHome(fparent, async () => {
expect((await capture(() => runBootstrap(['render', '--workspace', fws]))).result).toBe(0);
// render is engine-free and never touches brain/ — the gap this fix closes.
expect(existsSync(brainDir)).toBe(false);
const { runner } = makeRunner();
return capture(() =>
runBootstrap(['hooks', '--workspace', fws, '--harness', 'claude-code', '--gbrain-bin', process.execPath], {
runner,
}),
);
});
expect(out.result).toBe(0);
// brain/ now exists — ready for `sources add` without a manual mkdir
// round trip.
expect(existsSync(brainDir)).toBe(true);
// The manifest's actual source_id (default: 'workspace' for a first
// render) is spelled out verbatim — no guessing an "intuitive" name that
// would only surface as an FK error three steps later at verify time.
// --force is required and printed: brain/ is freshly created and has no
// git history yet, so `sources add` without --force would fail-fast on
// `not_a_git_repo` (#2707) the instant the printed command is pasted —
// the exact same `{ force: true }` the engine-backed verify fixtures use
// to register a brand new brain/ (test/bootstrap-verify.serial.test.ts).
expect(out.out).toContain(`gbrain sources add workspace --path ${brainDir} --force`);
// The collision-fallback id is previewed too, pinned to the SAME formula
// verify.ts would derive (not just the id's shape) — so preview/verify
// drift, or a change to the derivation formula, fails this test.
expect(out.out).toContain(`'${deriveWorkspaceSourceId(fws)}'`);
expect(deriveWorkspaceSourceId(fws)).toMatch(/^workspace-[0-9a-f]{8}$/);
}, 30_000);
test('a --repair re-run is idempotent: brain/ survives, the same preview reprints', async () => {
const { fws, fparent } = freshWorkspace();
const brainDir = join(fws, 'brain');
await withHome(fparent, async () => {
expect((await capture(() => runBootstrap(['render', '--workspace', fws]))).result).toBe(0);
expect((await capture(() => runBootstrap(['hooks', '--workspace', fws, '--harness', 'claude-code', '--gbrain-bin', process.execPath], { runner: makeRunner().runner }))).result).toBe(0);
// Simulate the human having already registered + committed into brain/.
writeFileSync(join(brainDir, 'probe.md'), '# probe\n');
const repaired = await capture(() =>
runBootstrap(
['hooks', '--workspace', fws, '--harness', 'claude-code', '--repair', '--gbrain-bin', process.execPath],
{ runner: makeRunner().runner },
),
);
expect(repaired.result).toBe(0);
expect(repaired.out).toContain(`gbrain sources add workspace --path ${brainDir} --force`);
// Idempotent mkdir never clobbers what the human already committed.
expect(existsSync(join(brainDir, 'probe.md'))).toBe(true);
});
}, 30_000);
test('a workspace path containing a space is shell-quoted in the printed command', async () => {
const fparent = mkdtempSync(join(tmpdir(), 'gb-srcid-space-'));
scratch.push(fparent);
const fhome = join(fparent, '.gbrain');
mkdirSync(fhome, { recursive: true });
const fwsRoot = mkdtempSync(join(tmpdir(), 'gb-srcid-space-root-'));
scratch.push(fwsRoot);
const fws = join(fwsRoot, 'has space');
mkdirSync(fws, { recursive: true });
const prev = process.env.GBRAIN_HOME;
process.env.GBRAIN_HOME = fparent;
try {
expect(initState(fws).ok).toBe(true);
for (const [key, value] of Object.entries(REQUIRED_ANSWERS)) {
const r = setAnswer(fws, key, value);
if (!r.ok) throw new Error(r.message);
}
expect(setAnswer(fws, 'MCP_SCOPE', 'project').ok).toBe(true);
const h = readBackHash(fws);
if (!h.ok) throw new Error(h.message);
expect(confirm(fws, h.hash).ok).toBe(true);
} finally {
if (prev === undefined) delete process.env.GBRAIN_HOME;
else process.env.GBRAIN_HOME = prev;
}
const brainDir = join(fws, 'brain');
const out = await withHome(fparent, async () => {
expect((await capture(() => runBootstrap(['render', '--workspace', fws]))).result).toBe(0);
const { runner } = makeRunner();
return capture(() =>
runBootstrap(['hooks', '--workspace', fws, '--harness', 'claude-code', '--gbrain-bin', process.execPath], {
runner,
}),
);
});
expect(out.result).toBe(0);
// A bare, unquoted path with a space would split into two shell words —
// the printed command must single-quote it so copy/paste actually works.
expect(out.out).toContain(`--path '${brainDir}' --force`);
expect(out.out).not.toContain(`--path ${brainDir} --force`);
}, 30_000);
});
describe('per-turn hooks are ON by default (v0.45 flip); --no-hooks opts out', () => {
// Self-contained fixtures: the file-level `ws` deliberately SKIPS
// HOOKS_CONSENT (for the decline test), so the default-on path needs a
+58 -1
View File
@@ -88,7 +88,9 @@ afterEach(() => {
function happyRules(overrides: Rule[] = []): Rule[] {
return [
...overrides,
{ key: 'gh --version', code: 0 },
// Realistic modern `gh --version` output — Gate 2 parses this to decide
// whether `--active` (added in cli/cli v2.57.0) is safe to pass.
{ key: 'gh --version', code: 0, stdout: 'gh version 2.97.0 (2026-07-31)\nhttps://github.com/cli/cli/releases/tag/v2.97.0\n' },
{ key: 'auth status', code: 0 },
// First origin probe: no remote yet. After create, git reports the URL.
{ key: 'remote get-url origin', times: 1, code: 2, stderr: 'error: No such remote' },
@@ -393,6 +395,61 @@ describe('createPrivateRepo', () => {
expect(err.message).toContain('gh auth login');
});
test('Gate 2 on a modern gh checks only the active account on github.com — a stale, invalid non-active/other-host account must not false-block bootstrap', async () => {
// `gh auth status` (no flags) aggregates every registered account on
// every host and exits 1 if ANY one of them is invalid, even if the
// active github.com account (what `gh`/`git` actually use for this
// flow) is perfectly healthy — confirmed against cli/cli's status.go
// (statusRun: the non-active-account loop at `if opts.Active { continue }`
// is skipped entirely under `--active`, so only the active account's
// entry can contribute to the exit code; `--hostname` narrows Hosts()
// to the given host before that loop even starts). Gate 2 must pass
// both flags on a `gh` that supports them — this test pins the exact
// argv so a regression back to the unscoped bare form is caught.
const { runner, calls } = makeRunner(happyRules());
await createPrivateRepo(ws, { runner, gbrainHomeDir: home });
const authCall = calls.find((c) => c[0] === 'gh' && c[1] === 'auth' && c[2] === 'status');
expect(authCall).toEqual(['gh', 'auth', 'status', '--active', '--hostname', 'github.com']);
});
test('Gate 2 on a pre-2.57.0 gh (no `--active` support) falls back to the host-scoped bare form instead of hard-failing on an unknown flag', async () => {
// `--active` was added in cli/cli v2.57.0 (confirmed: present in the
// v2.57.0 status.go, absent in v2.56.0 and earlier). Passing it to an
// older `gh` would make the WHOLE command fail on an unrecognized flag —
// a false GH_AUTH for a perfectly authenticated user. Gate 2 must detect
// this from the `gh --version` output already captured in Gate 1 and
// drop only `--active` (keeping `--hostname`, which has existed since
// gh's early versions). Pinned right at the boundary (2.56.0, one minor
// below the flag's introduction) rather than an arbitrarily old version,
// so an off-by-one in the `>=` comparison would fail this test.
const { runner, calls } = makeRunner(
happyRules([
{ key: 'gh --version', code: 0, stdout: 'gh version 2.56.0 (2024-08-01)\n' },
]),
);
await createPrivateRepo(ws, { runner, gbrainHomeDir: home });
const authCall = calls.find((c) => c[0] === 'gh' && c[1] === 'auth' && c[2] === 'status');
expect(authCall).toEqual(['gh', 'auth', 'status', '--hostname', 'github.com']);
});
test('Gate 2 on exactly gh 2.57.0 (the version that introduced `--active`) uses the enhanced form', async () => {
const { runner, calls } = makeRunner(
happyRules([{ key: 'gh --version', code: 0, stdout: 'gh version 2.57.0 (2024-09-11)\n' }]),
);
await createPrivateRepo(ws, { runner, gbrainHomeDir: home });
const authCall = calls.find((c) => c[0] === 'gh' && c[1] === 'auth' && c[2] === 'status');
expect(authCall).toEqual(['gh', 'auth', 'status', '--active', '--hostname', 'github.com']);
});
test('Gate 2 with an unparseable `gh --version` falls back to the host-scoped bare form (fail conservative, not assume `--active` support)', async () => {
const { runner, calls } = makeRunner(
happyRules([{ key: 'gh --version', code: 0, stdout: 'not a recognizable version string\n' }]),
);
await createPrivateRepo(ws, { runner, gbrainHomeDir: home });
const authCall = calls.find((c) => c[0] === 'gh' && c[1] === 'auth' && c[2] === 'status');
expect(authCall).toEqual(['gh', 'auth', 'status', '--hostname', 'github.com']);
});
// ── create-repo-first adoption (the human made the repo, opened it in Claude
// Code / Codex, then ran bootstrap) + hardening of the adoption path ──────
@@ -0,0 +1,340 @@
/**
* bootstrap subcommand `--help`/`-h`/`help` guard (src/commands/bootstrap.ts).
*
* Regression test for a safety defect: `gbrain bootstrap <subcommand> --help`
* (a help token AFTER the subcommand name, e.g. `gbrain bootstrap repo
* --help`) fell straight through into the subcommand's own arg parsing, and
* none of the mutating handlers (repo/hooks/verify/attach/uninstall/render/
* interview) checked for a help token themselves. So `--help` did not print
* help it ran the real operation (private repo creation, MCP/hook
* registration, the verify contract, workspace adoption, receipt-keyed file
* removal, or recording an interview answer).
*
* Each fixture below is deliberately built so the real (non-`--help`)
* operation WOULD reach its dangerous side effect if the guard were removed
* an operational verify config, an already-initialized attach workspace,
* an isolated uninstall home with a real receipt + a bootstrap-created file
* to survive, a fresh interview workspace to prove `--init` never runs. This
* proves the guard intercepts BEFORE any write/exec call, not merely that
* the operation would have failed anyway on an unprepared fixture.
*
* Serial: mutates GBRAIN_HOME.
*/
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, statSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { runBootstrap } from '../src/commands/bootstrap.ts';
import type { ExecRunner } from '../src/core/bootstrap/repo.ts';
import { interviewStatePath } from '../src/core/bootstrap/interview.ts';
import { receiptPath, writeReceipt, type InstallReceipt } from '../src/core/bootstrap/format.ts';
import { readInstallLog } from '../src/core/bootstrap/status.ts';
import { initState, setAnswer, skipAnswer, confirm, readBackHash } from '../src/core/bootstrap/interview.ts';
let tmpParent: string; // GBRAIN_HOME parent (configDir appends .gbrain)
let home: string;
let ws: string;
let prevHome: string | undefined;
const REQUIRED_ANSWERS: Record<string, string> = {
AGENT_NAME: 'HelpGuard',
PRINCIPAL_NAME: 'Pat Example',
AGENT_PURPOSE: 'Maintain the research corpus and draft the weekly memo without re-briefing.',
AGENT_TOP_JOBS: '- corpus upkeep\n- weekly memo\n- meeting prep',
PRINCIPAL_CONTEXT: 'Runs a small research group; values signal over noise.',
VOICE_REGISTER: 'Direct: three options, the second one wins.',
};
/** Recording fake runner never spawns anything; asserts zero calls when
* the --help guard is doing its job. */
function makeRunner(): { runner: ExecRunner; calls: string[][] } {
const calls: string[][] = [];
const runner: ExecRunner = async (argv: string[]) => {
calls.push(argv);
return { code: 0, stdout: '', stderr: '' };
};
return { runner, calls };
}
/** Capture console.log + console.error around an async call. */
async function capture<T>(fn: () => Promise<T>): Promise<{ result: T; out: string; err: string }> {
const origLog = console.log;
const origErr = console.error;
let out = '';
let err = '';
console.log = (...args: unknown[]) => {
out += args.map(String).join(' ') + '\n';
};
console.error = (...args: unknown[]) => {
err += args.map(String).join(' ') + '\n';
};
try {
const result = await fn();
return { result, out, err };
} finally {
console.log = origLog;
console.error = origErr;
}
}
beforeAll(async () => {
tmpParent = mkdtempSync(join(tmpdir(), 'gb-subhelp-'));
home = join(tmpParent, '.gbrain');
mkdirSync(home, { recursive: true });
ws = mkdtempSync(join(tmpdir(), 'gb-subhelp-ws-'));
prevHome = process.env.GBRAIN_HOME;
process.env.GBRAIN_HOME = tmpParent;
// Interview: all required answers, confirmed — so a real (non---help)
// render/hooks/repo/attach call on this workspace would succeed, giving
// the --help guard something real to intercept.
expect(initState(ws).ok).toBe(true);
for (const [key, value] of Object.entries(REQUIRED_ANSWERS)) {
const r = setAnswer(ws, key, value);
if (!r.ok) throw new Error(r.message);
}
expect(setAnswer(ws, 'MCP_SCOPE', 'project').ok).toBe(true);
expect(skipAnswer(ws, 'HOOKS_CONSENT').ok).toBe(true);
const h = readBackHash(ws);
if (!h.ok) throw new Error(h.message);
expect(confirm(ws, h.hash).ok).toBe(true);
// Materialize a real render + receipt so render/hooks/attach --help have
// real, pre-existing state they could (but must not) mutate. `ws` now also
// satisfies attach's precondition (an `initialized: true` agent.json).
const render = await capture(() => runBootstrap(['render', '--workspace', ws]));
expect(render.result).toBe(0);
// An "operational" verify config: loadConfig() resolves, so a real
// (non---help) verify would proceed past the "no brain configured" early
// exit into createEngine()/connect() — which would create files under
// `verifyDataDir`. It's never touched if the --help guard works.
const verifyDataDir = join(home, 'verify-brain.pglite');
writeFileSync(join(home, 'config.json'), JSON.stringify({ engine: 'pglite', database_path: verifyDataDir }), 'utf8');
});
afterAll(() => {
if (prevHome === undefined) delete process.env.GBRAIN_HOME;
else process.env.GBRAIN_HOME = prevHome;
rmSync(tmpParent, { recursive: true, force: true });
rmSync(ws, { recursive: true, force: true });
});
describe('bootstrap <subcommand> --help/-h/help never runs the real operation', () => {
test('render --help: usage text, exit 0, receipt + install log untouched', async () => {
const receiptBefore = readFileSync(receiptPath(home), 'utf8');
const mtimeBefore = statSync(receiptPath(home)).mtimeMs;
const logCountBefore = readInstallLog(home, 1000).length;
const r = await capture(() => runBootstrap(['render', '--workspace', ws, '--help']));
expect(r.result).toBe(0);
expect(r.out).toContain('Render identity files');
expect(readFileSync(receiptPath(home), 'utf8')).toBe(receiptBefore);
expect(statSync(receiptPath(home)).mtimeMs).toBe(mtimeBefore);
expect(readInstallLog(home, 1000).length).toBe(logCountBefore);
});
test('repo --help: usage text, exit 0, zero exec calls', async () => {
const { runner, calls } = makeRunner();
const r = await capture(() => runBootstrap(['repo', '--workspace', ws, '--help'], { runner }));
expect(r.result).toBe(0);
expect(r.out).toContain('PRIVATE GitHub repo');
expect(calls.length).toBe(0);
});
test('repo help (bare word, no dashes): usage text, exit 0, zero exec calls', async () => {
const { runner, calls } = makeRunner();
const r = await capture(() => runBootstrap(['repo', '--workspace', ws, 'help'], { runner }));
expect(r.result).toBe(0);
expect(r.out).toContain('PRIVATE GitHub repo');
expect(calls.length).toBe(0);
});
test('hooks --help: usage text, exit 0, zero exec calls, receipt untouched', async () => {
const receiptBefore = readFileSync(receiptPath(home), 'utf8');
const mtimeBefore = statSync(receiptPath(home)).mtimeMs;
const { runner, calls } = makeRunner();
const r = await capture(() =>
runBootstrap(
['hooks', '--workspace', ws, '--harness', 'claude-code', '--gbrain-bin', process.execPath, '--help'],
{ runner },
),
);
expect(r.result).toBe(0);
expect(r.out).toContain('Register MCP');
expect(calls.length).toBe(0);
expect(readFileSync(receiptPath(home), 'utf8')).toBe(receiptBefore);
expect(statSync(receiptPath(home)).mtimeMs).toBe(mtimeBefore);
});
test('verify --help: usage text, exit 0, engine never connects (data dir never created)', async () => {
const verifyDataDir = join(home, 'verify-brain.pglite');
expect(existsSync(verifyDataDir)).toBe(false); // sanity: config points at a path that doesn't exist yet
const r = await capture(() => runBootstrap(['verify', '--workspace', ws, '--help']));
expect(r.result).toBe(0);
expect(r.out).toContain('install contract');
// A real (unguarded) verify would have called engine.connect(), which
// acquires a file lock under the data dir — creating it. It must still
// be absent.
expect(existsSync(verifyDataDir)).toBe(false);
});
test('attach --help: usage text, exit 0, receipt untouched — a real attach here WOULD rewrite it', async () => {
// `ws` already has an `initialized: true` agent.json (from the render
// above) and an existing receipt — attach's real precondition, and the
// exact case its own docstring says it does NOT refuse (it merges into
// the existing receipt instead). So an unguarded attach --help here
// would visibly touch the receipt.
const receiptBefore = readFileSync(receiptPath(home), 'utf8');
const mtimeBefore = statSync(receiptPath(home)).mtimeMs;
const r = await capture(() => runBootstrap(['attach', '--workspace', ws, '--help']));
expect(r.result).toBe(0);
expect(r.out).toContain('adopt a cloned agent workspace');
expect(readFileSync(receiptPath(home), 'utf8')).toBe(receiptBefore);
expect(statSync(receiptPath(home)).mtimeMs).toBe(mtimeBefore);
});
describe('uninstall --help (isolated home + a real receipt-tracked file that must survive)', () => {
function seedUninstallFixture(): { uninstallWs: string; isolatedHome: string; dummyCreatedPath: string } {
const uninstallWs = mkdtempSync(join(tmpdir(), 'gb-subhelp-uninstall-ws-'));
const isolatedHome = join(uninstallWs, '.gbrain');
mkdirSync(join(isolatedHome, 'brain.pglite'), { recursive: true });
mkdirSync(join(isolatedHome, 'bootstrap'), { recursive: true });
writeFileSync(join(isolatedHome, 'config.json'), '{"engine":"pglite"}', 'utf8');
const dummyCreatedPath = join(uninstallWs, 'DUMMY_CREATED.md');
writeFileSync(dummyCreatedPath, 'created-by-bootstrap — must survive --help', 'utf8');
const receipt: InstallReceipt = {
receipt_version: 1,
workspace_dir: uninstallWs,
source_id: 'workspace',
agent_name: 'Uninstall Help Test',
created_at: '2026-01-01T00:00:00.000Z',
created_by: '0.0.0-test',
brain_created_by_bootstrap: false,
created_paths: [dummyCreatedPath],
registrations: [],
};
writeReceipt(isolatedHome, receipt);
return { uninstallWs, isolatedHome, dummyCreatedPath };
}
test('--help: usage text, exit 0, zero exec calls, receipt-tracked file + receipt survive', async () => {
const { uninstallWs, isolatedHome, dummyCreatedPath } = seedUninstallFixture();
try {
const receiptBefore = readFileSync(receiptPath(isolatedHome), 'utf8');
const { runner, calls } = makeRunner();
const r = await capture(() =>
runBootstrap(['uninstall', '--workspace', uninstallWs, '--home', isolatedHome, '--help'], { runner }),
);
expect(r.result).toBe(0);
expect(r.out).toContain('Receipt-keyed removal');
expect(calls.length).toBe(0);
expect(existsSync(dummyCreatedPath)).toBe(true);
expect(readFileSync(receiptPath(isolatedHome), 'utf8')).toBe(receiptBefore);
} finally {
rmSync(uninstallWs, { recursive: true, force: true });
}
});
test('-h: same interception (the highest-risk subcommand, short flag spelling)', async () => {
const { uninstallWs, isolatedHome, dummyCreatedPath } = seedUninstallFixture();
try {
const { runner, calls } = makeRunner();
const r = await capture(() =>
runBootstrap(['uninstall', '--workspace', uninstallWs, '--home', isolatedHome, '-h'], { runner }),
);
expect(r.result).toBe(0);
expect(r.out).toContain('Receipt-keyed removal');
expect(calls.length).toBe(0);
expect(existsSync(dummyCreatedPath)).toBe(true);
} finally {
rmSync(uninstallWs, { recursive: true, force: true });
}
});
test('help (bare word): same interception', async () => {
const { uninstallWs, isolatedHome, dummyCreatedPath } = seedUninstallFixture();
try {
const { runner, calls } = makeRunner();
const r = await capture(() =>
runBootstrap(['uninstall', '--workspace', uninstallWs, '--home', isolatedHome, 'help'], { runner }),
);
expect(r.result).toBe(0);
expect(r.out).toContain('Receipt-keyed removal');
expect(calls.length).toBe(0);
expect(existsSync(dummyCreatedPath)).toBe(true);
} finally {
rmSync(uninstallWs, { recursive: true, force: true });
}
});
});
describe('interview + help (Warning: help combined with a real action flag)', () => {
test('interview --init --help: usage text, exit 0, state/interview.json never created', async () => {
const freshWs = mkdtempSync(join(tmpdir(), 'gb-subhelp-interview-'));
try {
expect(existsSync(interviewStatePath(freshWs))).toBe(false);
const r = await capture(() => runBootstrap(['interview', '--workspace', freshWs, '--init', '--help']));
expect(r.result).toBe(0);
expect(r.out).toContain('Create/record/read interview state');
expect(existsSync(interviewStatePath(freshWs))).toBe(false);
} finally {
rmSync(freshWs, { recursive: true, force: true });
}
});
test('interview --set KEY value --help: usage text, exit 0, no answer recorded', async () => {
const freshWs = mkdtempSync(join(tmpdir(), 'gb-subhelp-interview-'));
try {
expect(initState(freshWs).ok).toBe(true);
const r = await capture(() =>
runBootstrap(['interview', '--workspace', freshWs, '--set', 'AGENT_NAME', 'ShouldNotRecord', '--help']),
);
expect(r.result).toBe(0);
expect(r.out).toContain('Create/record/read interview state');
const raw = JSON.parse(readFileSync(interviewStatePath(freshWs), 'utf8')) as { answers: Record<string, unknown> };
expect(raw.answers['AGENT_NAME']).toBeUndefined();
} finally {
rmSync(freshWs, { recursive: true, force: true });
}
});
test('interview --set VOICE_REGISTER "help" (bare word as a legitimate answer VALUE, no --help flag): still records — proves the bare-word exclusion for interview', async () => {
const freshWs = mkdtempSync(join(tmpdir(), 'gb-subhelp-interview-'));
try {
expect(initState(freshWs).ok).toBe(true);
const r = await capture(() => runBootstrap(['interview', '--workspace', freshWs, '--set', 'VOICE_REGISTER', 'help']));
expect(r.result).toBe(0);
expect(r.out).toContain('recorded');
const raw = JSON.parse(readFileSync(interviewStatePath(freshWs), 'utf8')) as { answers: Record<string, { value?: string }> };
expect(raw.answers['VOICE_REGISTER']?.value).toBe('help');
} finally {
rmSync(freshWs, { recursive: true, force: true });
}
});
});
});
+32 -1
View File
@@ -214,6 +214,38 @@ describe("extractTriggers", () => {
expect(triggers).toEqual(["what do we know", "tell me about"]);
});
test("parses triggers from a single-line YAML flow sequence", () => {
const fm = [
"---",
"name: transcript-insights",
'triggers: ["analyze this transcript", podcast takeaways]',
"---",
"",
].join("\n");
expect(extractTriggers(fm)).toEqual([
"analyze this transcript",
"podcast takeaways",
]);
});
test("parses triggers from a wrapped YAML flow sequence (#3887)", () => {
const fm = [
"---",
"name: transcript-insights",
"triggers: [analyze this transcript, podcast takeaways, video transcript insights, interview",
" takeaways, policy mechanisms from transcript]",
"---",
"",
].join("\n");
expect(extractTriggers(fm)).toEqual([
"analyze this transcript",
"podcast takeaways",
"video transcript insights",
"interview takeaways",
"policy mechanisms from transcript",
]);
});
test("returns [] when frontmatter is missing", () => {
expect(extractTriggers("# Just a body, no frontmatter\n")).toEqual([]);
});
@@ -450,4 +482,3 @@ function afterEachCleanup(fn: () => void) {
const { afterEach } = require("bun:test");
afterEach(fn);
}
+5
View File
@@ -88,6 +88,11 @@ describe('claude-cli recipe registration', () => {
expect(recipe!.touchpoints.chat!.supports_tools).toBe(true);
expect(recipe!.touchpoints.chat!.supports_subagent_loop).toBe(true);
expect(recipe!.touchpoints.chat!.models).toContain('claude-sonnet-4-6');
// Wave rider for #3976: pin the Claude 5 family the CLI already serves.
expect(recipe!.touchpoints.chat!.models).toContain('claude-fable-5');
expect(recipe!.touchpoints.chat!.models).toContain('claude-opus-5');
expect(recipe!.touchpoints.chat!.models).toContain('claude-opus-4-8');
expect(recipe!.touchpoints.chat!.models).toContain('claude-sonnet-5');
expect(recipe!.touchpoints.embedding).toBeUndefined();
expect(recipe!.touchpoints.expansion).toBeUndefined();
});
+27
View File
@@ -276,6 +276,33 @@ describe('parseConversation — disabledBuiltinIds', () => {
// Multi-line continuation (D5)
// ---------------------------------------------------------------------------
describe('parseConversation — markdown-heading-turn (gbrain transcript ingest)', () => {
test('parses ## User / ## Assistant heading-only turns with continuation body', () => {
const body = [
'## User',
'What is the capital of France?',
'## Assistant',
'The capital of France is Paris.',
'It is also its largest city.',
].join('\n');
const r = parseConversation(body, { fallbackDate: '2026-08-11' });
expect(r.matched_pattern_id).toBe('markdown-heading-turn');
expect(r.messages).toHaveLength(2);
expect(r.messages[0].speaker).toBe('User');
expect(r.messages[0].text).toBe('What is the capital of France?');
expect(r.messages[1].speaker).toBe('Assistant');
expect(r.messages[1].text).toBe(
'The capital of France is Paris.\nIt is also its largest city.',
);
});
test('does not mistake an ordinary ## Summary heading for a turn', () => {
const body = ['## Summary', 'This is not a speaker turn.'].join('\n');
const r = parseConversation(body, { fallbackDate: '2026-08-11' });
expect(r.matched_pattern_id).not.toBe('markdown-heading-turn');
});
});
describe('parseConversation — multi-line continuation (D5)', () => {
test('iMessage continuation absorbs orphan lines', () => {
const body = [
+19
View File
@@ -305,6 +305,25 @@ describe('BudgetTracker.reserve', () => {
expect((caught as BudgetExhausted).reason).toBe('no_pricing');
});
test('#3628: unknown hosted rerank provider points no-pricing guidance at embedding-pricing', () => {
const t = new BudgetTracker({ maxCostUsd: 1.0, label: 'test', auditPath });
let caught: unknown = null;
try {
t.reserve({
modelId: 'acmecorp:unpriced-reranker-v9',
estimatedInputTokens: 100,
maxOutputTokens: 0,
kind: 'rerank',
});
} catch (err) {
caught = err;
}
expect(caught).toBeInstanceOf(BudgetExhausted);
expect((caught as BudgetExhausted).reason).toBe('no_pricing');
expect((caught as Error).message).toContain('embedding-pricing.ts');
expect((caught as Error).message).not.toContain('anthropic-pricing.ts');
});
test('v0.40.x REGRESSION: known hosted embed (openai) still real-priced (trips a tiny cap)', () => {
// 3-small is $0.02/1M tokens. 1M tokens projects $0.02 > $0.0001 cap, so a
// real (nonzero) price trips the cost gate — proving it's NOT on the $0
+3 -2
View File
@@ -136,7 +136,7 @@ describe('D2 — knobsHash differs across cross-modal knob values', () => {
return resolveSearchMode({ mode: 'balanced' });
}
test('KNOBS_HASH_VERSION is 15 (cross-modal still appended; 14→15 FTS language fold)', () => {
test('KNOBS_HASH_VERSION is 16 (cross-modal still appended; 15→16 detail fold #3515)', () => {
// v0.35 ladder: 1→2 reranker, 2→3 floor_ratio. v0.36 piggybacks on v=3
// with 7 cross-modal knobs + column/provider context. v0.40.4 (salem) +
// v0.39 T21 (master) bump to v=4 for graph_signals + schema-pack fields.
@@ -149,7 +149,8 @@ describe('D2 — knobsHash differs across cross-modal knob values', () => {
// #3430: 13→14 compiled_truth boost no longer applies at detail=medium.
// 14→15: the resolved FTS configuration name (fts=) — a language switch
// plus `reindex-search-vector` must not keep serving pre-switch rows.
expect(KNOBS_HASH_VERSION).toBe(15);
// #3515: 15→16 detail fold (det=).
expect(KNOBS_HASH_VERSION).toBe(16);
});
test('flipping unified_multimodal changes the hash', () => {
+1 -1
View File
@@ -5,7 +5,7 @@
* zero pattern pages written (e.g. when no subagent-capable worker slot was
* free for the whole wait window) a silent no-op for days.
*
* A later fix added runPgliteSubagentsInline to this phase (patterns.ts
* A later fix added the shared inline drain (now runSubagentsInline) to this phase (patterns.ts
* previously submitted a job and waited without anything ever claiming it on
* PGLite synthesize.ts already had this inline drain, patterns.ts didn't).
* So a fake ANTHROPIC_API_KEY here now gets claimed and actually attempted;
+114
View File
@@ -0,0 +1,114 @@
/**
* Regression for #1586's unfixed half: the patterns phase.
*
* #1586 threaded the cycle's resolved source through `synthesize.ts` so dream
* output lands in the named source's `(source_id, slug)` rows. `patterns.ts`
* was left on the pre-#1586 shape: `collectChildPutPageSlugs` stamped a literal
* `'default'` and `reverseWriteRefs` compared against a literal `'default'`.
*
* The result on a per-source cycle was a page filed against the wrong source:
* the ROW is created under `default` (the child had no `source_id` to scope
* its put_page calls), while the reverse-write drops the FILE into the named
* source's checkout — `source_id === 'default'` selected the
* `brainDir/<slug>.md` branch, and on a per-source cycle `brainDir` IS the
* named source's checkout. Row and file then disagree about which source owns
* the page, which is what `doctor` reports as `multi_source_drift`.
*
* These tests pin both halves of the fix, plus the unscoped legacy behavior.
*/
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { mkdtempSync, rmSync, existsSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { __testing } from '../src/core/cycle/patterns.ts';
const { collectChildPutPageSlugs, reverseWriteRefs } = __testing;
const SOURCE_ID = 'coast';
const SLUG = 'wiki/personal/patterns/a-pattern';
let engine: PGLiteEngine;
let brainDir: string;
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
brainDir = mkdtempSync(join(tmpdir(), 'gbrain-patterns-scope-'));
const db = (engine as any).db;
await db.exec(`
INSERT INTO sources (id, name, local_path)
VALUES ('${SOURCE_ID}', 'coast', '/tmp/coast-brain')
ON CONFLICT (id) DO NOTHING;
`);
await db.exec(`
INSERT INTO minion_jobs (id, queue, name, data, status)
VALUES (2001, 'default', 'subagent', '{}'::jsonb, 'completed')
ON CONFLICT (id) DO NOTHING;
`);
await db.query(
`INSERT INTO subagent_tool_executions (job_id, message_idx, tool_use_id, tool_name, status, input)
VALUES (2001, 0, 'tool_p1', 'brain_put_page', 'complete', $1::jsonb)`,
[JSON.stringify({ slug: SLUG, body: 'a pattern' })],
);
// The page the child wrote lives in the cycle's source, not in 'default'.
await engine.putPage(SLUG, {
type: 'note',
title: 'A pattern',
compiled_truth: 'a pattern',
} as any, { sourceId: SOURCE_ID });
});
afterAll(async () => {
await engine.disconnect();
rmSync(brainDir, { recursive: true, force: true });
});
describe('#1586: the patterns phase scopes its writes to the cycle source', () => {
test('collected refs carry the cycle source, not a hardcoded default', async () => {
const refs = await collectChildPutPageSlugs(engine as any, [2001], SOURCE_ID);
expect(refs).toHaveLength(1);
expect(refs[0]!.slug).toBe(SLUG);
// Pre-fix this was the literal 'default' no matter which source the cycle
// resolved, which is what filed the page against the wrong source.
expect(refs[0]!.source_id).toBe(SOURCE_ID);
});
test('unscoped callers keep the legacy default source', async () => {
const refs = await collectChildPutPageSlugs(engine as any, [2001]);
expect(refs[0]!.source_id).toBe('default');
});
test('reverse-write resolves the row the child actually wrote', async () => {
const refs = await collectChildPutPageSlugs(engine as any, [2001], SOURCE_ID);
const count = await reverseWriteRefs(engine as any, brainDir, refs, SOURCE_ID);
// The lookup is keyed on the ref's source_id, so a ref carrying the cycle
// source resolves the row the child wrote there.
expect(count).toBe(1);
});
test('the cycle source is native — its pages stay at brainDir/<slug>.md', async () => {
const refs = await collectChildPutPageSlugs(engine as any, [2001], SOURCE_ID);
await reverseWriteRefs(engine as any, brainDir, refs, SOURCE_ID);
expect(existsSync(join(brainDir, `${SLUG}.md`))).toBe(true);
expect(existsSync(join(brainDir, '.sources', SOURCE_ID, `${SLUG}.md`))).toBe(false);
});
test('a foreign source still lands under .sources/<id>/', async () => {
const foreignDir = mkdtempSync(join(tmpdir(), 'gbrain-patterns-foreign-'));
try {
const refs = await collectChildPutPageSlugs(engine as any, [2001], SOURCE_ID);
// brainDir belongs to 'default' here, so 'coast' is foreign to it.
await reverseWriteRefs(engine as any, foreignDir, refs, 'default');
expect(existsSync(join(foreignDir, '.sources', SOURCE_ID, `${SLUG}.md`))).toBe(true);
expect(existsSync(join(foreignDir, `${SLUG}.md`))).toBe(false);
} finally {
rmSync(foreignDir, { recursive: true, force: true });
}
});
});
+299
View File
@@ -0,0 +1,299 @@
// #2674 — pglite_scratch_probe: distinguish a damaged PGLite store from a
// broken WASM runtime.
//
// The load-bearing fact: PGLite reports only `Aborted()` to JS and prints the
// real PANIC (`could not locate a valid checkpoint record …`) to its own
// stderr, so the init-error classifier can never discriminate the two cases
// from the default path. The scratch-store probe is the only thing that can:
// a throwaway store that opens fine proves the runtime is healthy.
//
// Pinned contracts:
// - COST GATE: the probe never runs on a routine healthy doctor, and never
// when the on-disk diagnosis already fully explains the failure (a live
// lock or a missing dir). It runs when PGLite init actually failed
// (engine=null + config engine is pglite + not --fast) with an
// unexplained/damage-class dir state, or on explicit `--probe-pglite`.
// - EVIDENCE-GATED verdict (the reviewed false-positive fix): "YOUR STORE
// is damaged" is asserted ONLY with positive evidence — a damage-class
// disk diagnosis (wal-corruption-likely / unsupported-layout) or a
// wasm-abort/corrupt classification of the real connect error. Without
// evidence the scratch-ok arm hedges (warn) instead of convicting the
// store: engine=null also covers locks and config refusals.
// - Discrimination messaging: scratch-ok + evidence names the STORE
// (recovery ladder: pglite-repair first, then reinit-pglite / restore
// backup; markdown unaffected); scratch-fail names the RUNTIME (report
// OS + Bun on #223).
// - Cleanup: the scratch temp dir is removed on success AND failure.
// - Never-touch-the-real-store guard: overlap with the real store path is
// refused before any PGLite work.
// - Cross-surface parity: BOTH buildChecks() and doctorReportRemote() emit
// the check on the PGLite connection-failure path.
//
// NOTE: the buildChecks / doctorReportRemote tests import only symbols that
// exist on master, so on an unmodified master checkout they fail
// BEHAVIORALLY (the check is absent from the emitted list), not via a
// missing export.
import { describe, test, expect } from 'bun:test';
import { mkdtempSync, writeFileSync, mkdirSync, readdirSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import type { BrainEngine } from '../src/core/engine.ts';
import { buildChecks, doctorReportRemote, type Check } from '../src/commands/doctor.ts';
import { withEnv } from './helpers/with-env.ts';
const PROBE_TIMEOUT = 90_000; // PGLite cold start is 520s on loaded machines
function findCheck(checks: Check[], name: string): Check | undefined {
return checks.find((c) => c.name === name);
}
function scratchDirsInTmp(): Set<string> {
return new Set(readdirSync(tmpdir()).filter((d) => d.startsWith('gbrain-pglite-probe-')));
}
/** GBRAIN_HOME parent dir whose .gbrain/config.json declares a pglite engine. */
function makePgliteHome(): { home: string; storePath: string } {
const home = mkdtempSync(join(tmpdir(), 'gbrain-probe-home-'));
const dotGbrain = join(home, '.gbrain');
mkdirSync(dotGbrain, { recursive: true });
const storePath = join(dotGbrain, 'brain.pglite');
writeFileSync(
join(dotGbrain, 'config.json'),
JSON.stringify({ engine: 'pglite', database_path: storePath }),
);
return { home, storePath };
}
/**
* Fabricate a PG17-pglite-shaped data dir that `inspectPgliteDataDir` reads
* as `wal-corruption-likely`: valid layout (PG_VERSION 17, base/, 8192-byte
* pg_control, pg_wal/) plus a STALE postmaster.pid (dead PID) the
* unclean-shutdown marker. Same fixture shape as doctor-pglite-datadir.test.ts.
*/
function makeDamagedStore(storePath: string): void {
mkdirSync(join(storePath, 'global'), { recursive: true });
mkdirSync(join(storePath, 'base'), { recursive: true });
mkdirSync(join(storePath, 'pg_wal'), { recursive: true });
writeFileSync(join(storePath, 'PG_VERSION'), '17\n');
writeFileSync(join(storePath, 'global', 'pg_control'), Buffer.alloc(8192));
writeFileSync(join(storePath, 'postmaster.pid'), '99999999\n'); // dead PID → stale marker
}
const NO_DB_ENV = { GBRAIN_DATABASE_URL: undefined, DATABASE_URL: undefined };
describe('pglite_scratch_probe — buildChecks gating (#2674)', () => {
test(
'damage-class disk diagnosis + healthy runtime → probe runs and blames THE STORE',
async () => {
const { home, storePath } = makePgliteHome();
makeDamagedStore(storePath);
await withEnv({ ...NO_DB_ENV, GBRAIN_HOME: home }, async () => {
const before = scratchDirsInTmp();
const checks = await buildChecks(null, ['--scope=brain'], 'config-file-path');
// The disk diagnosis fires alongside the probe (shared inspection).
const dirCheck = findCheck(checks, 'pglite_data_dir');
expect(dirCheck).toBeDefined();
expect(dirCheck!.status).toBe('fail');
const check = findCheck(checks, 'pglite_scratch_probe');
expect(check).toBeDefined();
// On this (healthy) machine the scratch store works, and the disk
// shows unclean-shutdown damage, so the verdict is evidence-backed:
// runtime fine, YOUR STORE is damaged.
expect(check!.status).toBe('fail');
expect(check!.message).toContain('YOUR STORE is damaged');
expect(check!.message).toContain('pglite-repair');
expect(check!.message).toContain('reinit-pglite');
expect(check!.message).toContain('markdown is unaffected');
expect(check!.message).not.toContain('cannot run');
// Cleanup: no leaked scratch dirs.
const after = scratchDirsInTmp();
for (const d of after) expect(before.has(d)).toBe(true);
});
},
PROBE_TIMEOUT,
);
test('cost gate: a MISSING store dir explains the failure — no probe, no store conviction', async () => {
const { home } = makePgliteHome(); // config exists, store dir never created
await withEnv({ ...NO_DB_ENV, GBRAIN_HOME: home }, async () => {
const checks = await buildChecks(null, ['--scope=brain'], 'config-file-path');
// The disk diagnosis names the actual state (missing → init pointer)…
const dirCheck = findCheck(checks, 'pglite_data_dir');
expect(dirCheck).toBeDefined();
expect(dirCheck!.status).toBe('warn');
// …and the probe does NOT run: nothing ambiguous to discriminate, and
// pre-fix this exact shape produced the "YOUR STORE is damaged" false
// positive.
expect(findCheck(checks, 'pglite_scratch_probe')).toBeUndefined();
});
});
test('cost gate: --fast never pays the probe, even with a failed pglite engine', async () => {
const { home } = makePgliteHome();
await withEnv({ ...NO_DB_ENV, GBRAIN_HOME: home }, async () => {
const checks = await buildChecks(null, ['--fast', '--scope=brain'], 'config-file-path');
expect(findCheck(checks, 'pglite_scratch_probe')).toBeUndefined();
});
});
test('cost gate: engine=null with a NON-pglite config does not probe', async () => {
const home = mkdtempSync(join(tmpdir(), 'gbrain-probe-home-'));
mkdirSync(join(home, '.gbrain'), { recursive: true });
writeFileSync(
join(home, '.gbrain', 'config.json'),
JSON.stringify({ engine: 'postgres', database_url: 'postgres://localhost/nope' }),
);
await withEnv({ ...NO_DB_ENV, GBRAIN_HOME: home }, async () => {
const checks = await buildChecks(null, ['--scope=brain'], 'config-file-path');
expect(findCheck(checks, 'pglite_scratch_probe')).toBeUndefined();
});
});
test(
'on demand: --probe-pglite with no init failure reports the runtime as healthy (ok)',
async () => {
const { home } = makePgliteHome();
await withEnv({ ...NO_DB_ENV, GBRAIN_HOME: home }, async () => {
// --fast keeps the walk cheap; the explicit flag still wins.
const checks = await buildChecks(null, ['--fast', '--probe-pglite', '--scope=brain'], 'config-file-path');
const check = findCheck(checks, 'pglite_scratch_probe');
expect(check).toBeDefined();
expect(check!.status).toBe('ok');
expect(check!.message).toContain('runtime healthy');
});
},
PROBE_TIMEOUT,
);
});
describe('pglite_scratch_probe — doctorReportRemote parity (#2674)', () => {
test(
'a dead PGLite connection with an abort-class error triggers the probe and blames the store',
async () => {
const home = mkdtempSync(join(tmpdir(), 'gbrain-probe-home-'));
await withEnv({ ...NO_DB_ENV, GBRAIN_HOME: home }, async () => {
const stub = {
kind: 'pglite',
getStats: async () => {
throw new Error('Aborted(). Build with -sASSERTIONS for more info.');
},
} as unknown as BrainEngine;
const report = await doctorReportRemote(stub);
const check = findCheck(report.checks, 'pglite_scratch_probe');
expect(check).toBeDefined();
// wasm-abort classification of the REAL error = damage evidence.
expect(check!.status).toBe('fail');
expect(check!.message).toContain('YOUR STORE is damaged');
});
},
PROBE_TIMEOUT,
);
test(
'a LOCK error on the remote surface must NOT convict the store (the reviewed false positive)',
async () => {
const home = mkdtempSync(join(tmpdir(), 'gbrain-probe-home-'));
await withEnv({ ...NO_DB_ENV, GBRAIN_HOME: home }, async () => {
const stub = {
kind: 'pglite',
getStats: async () => {
throw new Error('Could not acquire PGLite lock. Another gbrain process is using the database.');
},
} as unknown as BrainEngine;
const report = await doctorReportRemote(stub);
const check = findCheck(report.checks, 'pglite_scratch_probe');
expect(check).toBeDefined();
// Lock errors classify 'unknown' → no damage evidence → hedged warn.
expect(check!.status).toBe('warn');
expect(check!.message).not.toContain('YOUR STORE is damaged');
});
},
PROBE_TIMEOUT,
);
test('a dead POSTGRES connection on the remote surface does not probe', async () => {
const stub = {
kind: 'postgres',
getStats: async () => {
throw new Error('connection refused');
},
} as unknown as BrainEngine;
const report = await doctorReportRemote(stub);
expect(findCheck(report.checks, 'pglite_scratch_probe')).toBeUndefined();
});
});
describe('probePgliteScratchStore — probe internals (#2674)', () => {
test('never-touch-the-real-store guard refuses overlap and leaks nothing', async () => {
const { probePgliteScratchStore } = await import('../src/core/pglite-engine.ts');
const before = scratchDirsInTmp();
// The scratch dir lives under tmpdir(), so a "real store" AT tmpdir()
// must trip the overlap guard before any PGLite work happens.
await expect(probePgliteScratchStore(tmpdir())).rejects.toThrow(/refusing to probe/);
const after = scratchDirsInTmp();
for (const d of after) expect(before.has(d)).toBe(true);
});
test('verdict routing: probe outcomes route to the right message via the probeFn seam', async () => {
// Message routing via the probeFn test seam — no cold start needed.
const { checkPgliteScratchProbe } = await import('../src/commands/doctor.ts');
// scratch ok + real init failed + DAMAGE EVIDENCE → the store is convicted.
const storeDamaged = await checkPgliteScratchProbe({
realInitFailed: true,
storeDamageEvidence: true,
probeFn: async () => ({ ok: true, duration_ms: 1234 }),
});
expect(storeDamaged.status).toBe('fail');
expect(storeDamaged.message).toContain('YOUR STORE is damaged');
expect(storeDamaged.message).toContain('pglite-repair');
expect(storeDamaged.message).toContain('reinit-pglite');
expect(storeDamaged.message).toContain('markdown is unaffected');
// scratch ok + real init failed + NO evidence → hedged warn, never a
// store conviction (engine=null also covers locks + config refusals).
const hedged = await checkPgliteScratchProbe({
realInitFailed: true,
probeFn: async () => ({ ok: true, duration_ms: 1234 }),
});
expect(hedged.status).toBe('warn');
expect(hedged.message).not.toContain('YOUR STORE is damaged');
expect(hedged.message).toContain('runtime is healthy');
expect(hedged.message).toContain('pglite_data_dir');
// real init failed + scratch failed too → runtime is broken, ask for OS/Bun.
const runtimeBroken = await checkPgliteScratchProbe({
realInitFailed: true,
probeFn: async () => ({
ok: false,
duration_ms: 1234,
error: 'Aborted(). Build with -sASSERTIONS for more info.',
verdict: 'unknown' as const,
}),
});
expect(runtimeBroken.status).toBe('fail');
expect(runtimeBroken.message).toContain('runtime cannot run');
expect(runtimeBroken.message).toContain('issues/223');
expect(runtimeBroken.message).toContain('OS and Bun versions');
expect(runtimeBroken.message).not.toContain('YOUR STORE is damaged');
// healthy real store + scratch failed → warn, still runtime-facing.
const scratchOnly = await checkPgliteScratchProbe({
realInitFailed: false,
probeFn: async () => ({ ok: false, duration_ms: 10, error: 'boom' }),
});
expect(scratchOnly.status).toBe('warn');
expect(scratchOnly.message).toContain('real store opened');
// probeFn throwing (e.g. the overlap guard) → warn, not a diagnosis.
const guardTrip = await checkPgliteScratchProbe({
realInitFailed: true,
probeFn: async () => {
throw new Error('refusing to probe: overlap');
},
});
expect(guardTrip.status).toBe('warn');
expect(guardTrip.message).toContain('could not run');
});
});
@@ -0,0 +1,136 @@
/**
* Regression test: `gbrain doctor`'s `supervisor_singleton` check should read
* the pid-file path that was used at supervisor start (recorded on the
* 'started' audit event as `pid_file` `this.opts.pidFile` in
* `MinionSupervisor.start()`), not blindly re-derive `DEFAULT_PID_FILE`
* locally.
*
* Pre-fix, `doctor.ts` read `readSupervisorPid(DEFAULT_PID_FILE).pid` even
* though the 'started' event already carried the real path. A supervisor
* launched with a custom `--pid-file` (e.g. a launchd/systemd unit passing an
* explicit pidfile path) could then produce a false "singleton mismatch" warn against its
* own live DB lock, because the pidfile doctor.ts checked was never the one
* the supervisor actually wrote.
*
* #1849 established the queue-scoped DB lock as the real singleton
* authority `supervisor_singleton` is a diagnostic display derived from
* that lock, not a second enforcement mechanism. This fix only corrects
* which pidfile the display reads; it does not touch the DB lock itself.
*/
import { describe, test, expect, beforeAll, afterAll, beforeEach, afterEach } from 'bun:test';
import * as fs from 'fs';
import * as os from 'os';
import * as path from 'path';
import { withEnv } from './helpers/with-env.ts';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { resetPgliteState } from './helpers/reset-pglite.ts';
import { buildChecks } from '../src/commands/doctor.ts';
import { supervisorLockId } from '../src/core/minions/supervisor.ts';
import { computeSupervisorAuditFilename } from '../src/core/minions/handlers/supervisor-audit.ts';
let engine: PGLiteEngine;
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
});
afterAll(async () => {
await engine.disconnect();
});
let auditDir: string;
let pidFileDir: string;
beforeEach(async () => {
await resetPgliteState(engine);
await engine.executeRaw(`DELETE FROM gbrain_cycle_locks`);
auditDir = fs.mkdtempSync(path.join(os.tmpdir(), 'doctor-supervisor-singleton-audit-'));
pidFileDir = fs.mkdtempSync(path.join(os.tmpdir(), 'doctor-supervisor-singleton-pidfile-'));
});
afterEach(() => {
try { fs.rmSync(auditDir, { recursive: true, force: true }); } catch { /* best-effort */ }
try { fs.rmSync(pidFileDir, { recursive: true, force: true }); } catch { /* best-effort */ }
});
function writeStartedEvent(fields: Record<string, unknown>): void {
const file = path.join(auditDir, computeSupervisorAuditFilename());
const row = { event: 'started', ts: new Date().toISOString(), supervisor_pid: process.pid, ...fields };
fs.writeFileSync(file, `${JSON.stringify(row)}\n`, 'utf8');
}
async function holdLiveLock(queue: string, holderPid: number, holderHost: string): Promise<void> {
await engine.executeRaw(
`INSERT INTO gbrain_cycle_locks (id, holder_pid, holder_host, acquired_at, ttl_expires_at, last_refreshed_at)
VALUES ($1, $2, $3, now(), now() + interval '30 minutes', now())`,
[supervisorLockId(queue), holderPid, holderHost],
);
}
async function findSingletonCheck() {
const checks = await buildChecks(engine, []);
return checks.find((c) => c.name === 'supervisor_singleton');
}
describe('doctor supervisor_singleton — honors the recorded pid_file (custom --pid-file)', () => {
test('custom --pid-file matching the live DB lock holder → single, not mismatch', async () => {
await withEnv({ GBRAIN_AUDIT_DIR: auditDir }, async () => {
// A pid-file at a path OTHER than DEFAULT_PID_FILE, holding the pid of
// this very test process (guaranteed alive via process.kill(pid, 0)).
const customPidFile = path.join(pidFileDir, 'custom-supervisor.pid');
fs.writeFileSync(customPidFile, String(process.pid), 'utf8');
// The 'started' event records that custom path, matching what
// MinionSupervisor.start() actually emits (this.opts.pidFile).
writeStartedEvent({ pid_file: customPidFile, queue: 'default', max_rss_mb: 512 });
// The DB lock (the real singleton authority per #1849) is held by
// this same (host, pid) — a single, healthy supervisor.
await holdLiveLock('default', process.pid, os.hostname());
const check = await findSingletonCheck();
expect(check).toBeDefined();
expect(check?.status).toBe('ok');
expect(check?.message).toContain('Single supervisor');
// Pre-fix this would have read DEFAULT_PID_FILE (which this test never
// writes to), gotten pid=null, and warned 'mismatch' instead.
expect(check?.message).not.toContain('mismatch');
});
});
test('pid_file that does not resolve to the lock holder → mismatch (an absent/wrong local pidfile still warns)', async () => {
await withEnv({ GBRAIN_AUDIT_DIR: auditDir }, async () => {
// pid_file recorded, but nothing is written at that path — readSupervisorPid
// resolves to pid=null, so the DB lock holder can't be locally confirmed.
const customPidFile = path.join(pidFileDir, 'never-written-supervisor.pid');
writeStartedEvent({ pid_file: customPidFile, queue: 'default', max_rss_mb: 512 });
await holdLiveLock('default', process.pid, os.hostname());
const check = await findSingletonCheck();
expect(check).toBeDefined();
expect(check?.status).toBe('warn');
expect(check?.message).toContain('singleton lock is held by');
});
});
});
// Compatibility fallback: a 'started' event from before this fix (or any
// caller that never recorded `pid_file`) has no such field. doctor.ts falls
// back to DEFAULT_PID_FILE for those — behavior-preserving relative to
// pre-fix. Pinned as a source-grep (rather than a behavioral run against the
// real DEFAULT_PID_FILE) because DEFAULT_PID_FILE is a module-load-time
// constant derived from $HOME; asserting against it directly would make the
// test's outcome depend on whatever real supervisor state happens to exist
// on the machine running the suite.
describe('doctor supervisor_singleton — pid_file fallback (source-grep)', () => {
test('falls back to DEFAULT_PID_FILE when the started event has no pid_file', async () => {
const source = await Bun.file(new URL('../src/commands/doctor.ts', import.meta.url)).text();
expect(source).toMatch(
/typeof lastStarted\.pid_file === 'string' && lastStarted\.pid_file\.length > 0\s*\n\s*\? lastStarted\.pid_file\s*\n\s*: DEFAULT_PID_FILE/,
);
expect(source).toContain('readSupervisorPid(pidFilePath)');
});
});
+27
View File
@@ -127,6 +127,33 @@ describe('doctor command', () => {
}
});
test('#3628: reranker_health surfaces budget failures with pricing guidance', async () => {
const { checkRerankerHealth } = await import('../src/commands/doctor.ts');
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gbrain-rerank-budget-doctor-'));
try {
await withEnv({ GBRAIN_AUDIT_DIR: tmpDir }, async () => {
logRerankFailure({
model: 'acmecorp:unpriced-reranker-v9',
reason: 'budget',
query_hash: 'budget01',
doc_count: 30,
error_summary: 'no pricing entry for model "acmecorp:unpriced-reranker-v9" (kind=rerank)',
});
const check = await checkRerankerHealth({
async getConfig(key: string): Promise<string | null> {
return key === 'search.reranker.enabled' ? 'true' : null;
},
} as any);
expect(check.status).toBe('warn');
expect(check.message).toContain('budget/pricing');
expect(check.message).toContain('embedding-pricing.ts');
expect(check.message).toContain('--max-cost');
});
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true });
}
});
test('runDoctor accepts null engine for filesystem-only mode', async () => {
const { runDoctor } = await import('../src/commands/doctor.ts');
// runDoctor should accept null engine — it runs filesystem checks only.
+7 -6
View File
@@ -25,6 +25,7 @@ if (skip) {
}
const CLI = join(import.meta.dir, '..', '..', 'src', 'cli.ts');
const DOCTOR_PROGRESS_TIMEOUT_MS = 60_000;
describeE2E('gbrain doctor --progress-json (E2E)', () => {
beforeAll(async () => {
@@ -41,7 +42,7 @@ describeE2E('gbrain doctor --progress-json (E2E)', () => {
const res = spawnSync('bun', [CLI, '--progress-json', 'doctor', '--json'], {
encoding: 'utf-8',
env: { ...process.env, NO_COLOR: '1' },
timeout: 30_000,
timeout: DOCTOR_PROGRESS_TIMEOUT_MS,
});
// Even if some checks warn, doctor runs to completion. Failures would
@@ -85,13 +86,13 @@ describeE2E('gbrain doctor --progress-json (E2E)', () => {
// progress-line pollution on stdout.
const parsed = JSON.parse(res.stdout);
expect(Array.isArray(parsed.checks) || Array.isArray(parsed)).toBe(true);
});
}, DOCTOR_PROGRESS_TIMEOUT_MS + 5_000);
test('default (no --progress-json) writes human-plain progress to stderr only', () => {
const res = spawnSync('bun', [CLI, 'doctor'], {
encoding: 'utf-8',
env: { ...process.env, NO_COLOR: '1' },
timeout: 30_000,
timeout: DOCTOR_PROGRESS_TIMEOUT_MS,
});
// Stdout may contain the check summary (human-readable) but should NOT
@@ -103,13 +104,13 @@ describeE2E('gbrain doctor --progress-json (E2E)', () => {
if (res.stderr.length > 0) {
expect(res.stderr).toContain('doctor.db_checks');
}
});
}, DOCTOR_PROGRESS_TIMEOUT_MS + 5_000);
test('--quiet suppresses progress entirely', () => {
const res = spawnSync('bun', [CLI, '--quiet', 'doctor'], {
encoding: 'utf-8',
env: { ...process.env, NO_COLOR: '1' },
timeout: 30_000,
timeout: DOCTOR_PROGRESS_TIMEOUT_MS,
});
// With --quiet the reporter emits no start/finish/tick lines on stderr.
@@ -117,5 +118,5 @@ describeE2E('gbrain doctor --progress-json (E2E)', () => {
// just no progress phases.
expect(res.stderr).not.toContain('[doctor.db_checks]');
expect(res.stderr).not.toContain('"event":"start"');
});
}, DOCTOR_PROGRESS_TIMEOUT_MS + 5_000);
});
+3 -3
View File
@@ -644,7 +644,7 @@ describe('E2E synthesize — PGLite inline subagent drain (takeover of #2699)',
);
let ticks = 0;
await synthTesting.runPgliteSubagentsInline(
await synthTesting.runSubagentsInline(
rig.engine,
queue,
queueName,
@@ -685,7 +685,7 @@ describe('E2E synthesize — PGLite inline subagent drain (takeover of #2699)',
{ allowProtectedSubmit: true },
);
await synthTesting.runPgliteSubagentsInline(
await synthTesting.runSubagentsInline(
rig.engine,
queue,
queueName,
@@ -719,7 +719,7 @@ describe('E2E synthesize — PGLite inline subagent drain (takeover of #2699)',
// Handler only ends when ctx.signal fires — like the real subagent
// handler mid-LLM-call. Without the inline timeout timer this awaits
// forever and the drain (and the whole cycle) wedges.
await synthTesting.runPgliteSubagentsInline(
await synthTesting.runSubagentsInline(
rig.engine,
queue,
queueName,
+50
View File
@@ -720,6 +720,56 @@ describeBoth('Engine parity — Postgres vs PGLite', () => {
expect(await pgliteEngine.countStalePagesForExtraction({ sourceId: SRC })).toBe(2);
});
// Chunkless-page safety net (embed --stale detection gap): a page with
// non-empty content but zero content_chunks rows (e.g. a putPage-only
// write) must be found on BOTH engines identically, and quarantined /
// embed_skip pages (intentionally chunkless by design) must be excluded
// identically on both. Isolated under a dedicated source.
test('chunkless-page-with-content detection: Postgres ↔ PGLite parity', async () => {
const SRC = 'chunkless-parity';
for (const eng of [pgEngine, pgliteEngine]) {
await eng.executeRaw(`INSERT INTO sources (id, name, config) VALUES ($1, 'Chunkless Parity', '{}'::jsonb) ON CONFLICT DO NOTHING`, [SRC]);
// cp/stub: non-empty content, never chunked — THE bug this fix targets.
await eng.putPage('cp/stub', { type: 'person', title: 'Stub', compiled_truth: 'stub content, never chunked' }, { sourceId: SRC });
// cp/chunked: same shape, but chunked — must be excluded.
await eng.putPage('cp/chunked', { type: 'note', title: 'Chunked', compiled_truth: 'chunked content' }, { sourceId: SRC });
await eng.upsertChunks('cp/chunked', [
{ chunk_index: 0, chunk_text: 'chunked content', chunk_source: 'compiled_truth' },
], { sourceId: SRC });
// cp/empty: no content — must be excluded (the #2822 empty-put class, not this bug).
await eng.putPage('cp/empty', { type: 'note', title: 'Empty', compiled_truth: '' }, { sourceId: SRC });
// cp/quarantined: chunkless BY DESIGN — must be excluded.
await eng.putPage('cp/quarantined', {
type: 'note', title: 'Quarantined', compiled_truth: 'junk content',
frontmatter: { quarantine: { reason: 'junk_pattern', detail: 'parity fixture', assessed_at: new Date().toISOString() } },
}, { sourceId: SRC });
// cp/skipped: chunkless BY DESIGN — must be excluded.
await eng.putPage('cp/skipped', {
type: 'note', title: 'Skipped', compiled_truth: 'x'.repeat(500),
frontmatter: { embed_skip: { reason: 'oversized', bytes: 500, assessed_at: new Date().toISOString() } },
}, { sourceId: SRC });
}
expect(await pgEngine.countChunklessPagesWithContent({ sourceId: SRC })).toBe(1);
expect(await pgliteEngine.countChunklessPagesWithContent({ sourceId: SRC })).toBe(1);
const pgRows = await pgEngine.listChunklessPagesWithContent({ sourceId: SRC });
const pgliteRows = await pgliteEngine.listChunklessPagesWithContent({ sourceId: SRC });
expect(pgRows.map(r => r.slug)).toEqual(['cp/stub']);
expect(pgliteRows.map(r => r.slug)).toEqual(['cp/stub']);
expect(pgRows[0].compiled_truth).toBe('stub content, never chunked');
expect(pgliteRows[0].compiled_truth).toBe(pgRows[0].compiled_truth);
// Unscoped count/list is >= the scoped count on both engines (other
// tests' fixtures may also be chunkless — this only asserts the SRC
// subset is reachable without scoping, not an exact global count).
const pgAllSlugs = (await pgEngine.listChunklessPagesWithContent({ batchSize: 10000 })).map(r => r.slug);
const pgliteAllSlugs = (await pgliteEngine.listChunklessPagesWithContent({ batchSize: 10000 })).map(r => r.slug);
expect(pgAllSlugs).toContain('cp/stub');
expect(pgliteAllSlugs).toContain('cp/stub');
});
test('v0.41.39 listEnrichCandidates parity (thin filter + source-aware inbound + order)', async () => {
const stub = 'Stub page.';
const pageSql = `
+66 -1
View File
@@ -33,7 +33,10 @@ import { PGLiteEngine } from '../../src/core/pglite-engine.ts';
import { resetPgliteState } from '../helpers/reset-pglite.ts';
import { validateSourceId } from '../../src/core/utils.ts';
import { extractTakesFromDb } from '../../src/core/cycle/extract-takes.ts';
import { copyMigrationSources } from '../../src/commands/migrate-engine.ts';
import {
copyMigrationSources,
copyPageLinksToTarget,
} from '../../src/commands/migrate-engine.ts';
let engine: PGLiteEngine;
let migrationTarget: PGLiteEngine;
@@ -124,6 +127,68 @@ describe('multi-source bug class', () => {
expect(await migrationTarget.getPage('people/alice', { sourceId: 'media-corpus' })).not.toBeNull();
});
test('migration preserves a cross-source link target source (#3859)', async () => {
await copyMigrationSources(engine, migrationTarget);
const pages = await engine.listPages({ limit: 100 });
for (const page of pages) {
await migrationTarget.putPage(page.slug, {
type: page.type,
title: page.title,
compiled_truth: page.compiled_truth,
timeline: page.timeline,
frontmatter: page.frontmatter,
}, { sourceId: page.source_id });
}
await engine.addLink(
'media/x/post-123',
'concepts/widget',
'cross-source migration',
'references',
undefined,
undefined,
undefined,
{ fromSourceId: 'media-corpus', toSourceId: 'default' },
);
const sourcePage = await engine.getPage(
'media/x/post-123',
{ sourceId: 'media-corpus' },
);
await copyPageLinksToTarget(engine, migrationTarget, sourcePage!);
const copied = await migrationTarget.getLinks(
'media/x/post-123',
{ sourceId: 'media-corpus' },
);
expect(copied).toHaveLength(1);
expect(copied[0]!.to_source_id).toBe('default');
await engine.addLink(
'people/alice',
'concepts/widget',
'failed cross-source target',
'references',
undefined,
undefined,
undefined,
{ fromSourceId: 'media-corpus', toSourceId: 'default' },
);
const filteredSourcePage = await engine.getPage(
'people/alice',
{ sourceId: 'media-corpus' },
);
await copyPageLinksToTarget(
engine,
migrationTarget,
filteredSourcePage!,
new Set(['concepts/widget']),
);
expect(await migrationTarget.getLinks(
'people/alice',
{ sourceId: 'media-corpus' },
)).toHaveLength(0);
});
test('listAllPageRefs returns one row per (slug, source_id), ordered (F11)', async () => {
const refs = await engine.listAllPageRefs();
// 4 rows: alice@default, alice@media-corpus, widget@default, post-123@media-corpus
+52
View File
@@ -675,6 +675,58 @@ describeE2E('serve-http OAuth 2.1 E2E (v0.26.1 + v0.26.2 + v0.26.3)', () => {
}
}, 15_000);
// =========================================================================
// #2179: DCR token_ttl_seconds — wire-level clamp + echo
// =========================================================================
//
// The unit tests in test/oauth-dcr-ttl.test.ts prove the store-level clamp;
// this is the HTTP seam: the MCP SDK's /register handler STRIPS unknown
// body members, so the field only works if serve-http's middleware carries
// it through dcrRegistrationContext. With the clamp window unset, the max
// derives fail-closed from the server's --token-ttl (default 3600) — a
// huge request must come back clamped to that, not rejected — and the
// minted token must match.
test('DCR /register accepts token_ttl_seconds, clamps to policy, echoes effective value (#2179)', async () => {
const res = await fetch(`${BASE}/register`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
client_name: 'e2e-dcr-ttl',
redirect_uris: ['https://example.com/cb'],
grant_types: ['authorization_code'],
token_endpoint_auth_method: 'client_secret_basic',
scope: 'read',
token_ttl_seconds: 365 * 24 * 3600, // way above any sane max
}),
});
expect(res.ok).toBe(true);
const body = await res.json() as any;
if (body.client_id) dcrClientIds.push(body.client_id);
// Echoed effective value = clamped fail-closed to the server's
// --token-ttl (3600, the default — the e2e server sets no flag and no
// oauth.dcr_ttl_max_seconds config).
expect(body.token_ttl_seconds).toBe(3600);
// And a client that omits the field gets no echo (backward compatible).
const res2 = await fetch(`${BASE}/register`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
client_name: 'e2e-dcr-no-ttl',
redirect_uris: ['https://example.com/cb'],
grant_types: ['authorization_code'],
token_endpoint_auth_method: 'client_secret_basic',
scope: 'read',
}),
});
expect(res2.ok).toBe(true);
const body2 = await res2.json() as any;
if (body2.client_id) dcrClientIds.push(body2.client_id);
expect(body2.token_ttl_seconds).toBeUndefined();
}, 15_000);
// =========================================================================
// v0.26.2: revoke-client CLI subprocess test
// =========================================================================
@@ -0,0 +1,308 @@
/**
* Chunkless-page safety net for `embed --stale`.
*
* `embedAllStale` (and the underlying `listStaleChunks`/`countStaleChunks`)
* only ever scan `content_chunks` rows where `embedding IS NULL`. A page
* written directly via `putPage` that never went through chunking (e.g. an
* enrichment-generated entity stub the dogfooding case this fix targets)
* has NO `content_chunks` row at all, so it is invisible to that scan
* forever: there is no row to go stale, no matter how many times
* `embed --stale` runs.
*
* Two layers tested here:
* 1. The detection primitive itself (`countChunklessPagesWithContent` /
* `listChunklessPagesWithContent`) including that quarantined and
* `embed_skip` pages are excluded, since BOTH are intentionally
* chunkless by design (content-quality gate), not drift to repair.
* 2. The end-to-end `embed --stale` wiring a chunkless page gets
* chunked AND embedded in the SAME pass, while pre-existing stale
* chunks and intentionally-chunkless pages are unaffected.
*
* Named `.serial.test.ts` (mirrors v0_37_gap_fill.serial.test.ts Lane D.2):
* configures the AI gateway + a fake embed transport for its whole
* lifecycle, which withEnv() can't wrap.
*/
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { resetPgliteState } from './helpers/reset-pglite.ts';
import { configureGateway, resetGateway, __setEmbedTransportForTests } from '../src/core/ai/gateway.ts';
import { runEmbedCore } from '../src/commands/embed.ts';
import { EMBED_SKIP_KEY, buildEmbedSkipMarker } from '../src/core/embed-skip.ts';
import { QUARANTINE_KEY, buildQuarantineMarker } from '../src/core/quarantine.ts';
import type { BrainEngine } from '../src/core/engine.ts';
const DIMS = 1536;
let engine: PGLiteEngine;
beforeAll(async () => {
configureGateway({
embedding_model: 'openai:text-embedding-3-large',
embedding_dimensions: DIMS,
env: { ...process.env, OPENAI_API_KEY: 'sk-test-fake' },
});
__setEmbedTransportForTests(async ({ values }: { values: string[] }) => ({
embeddings: values.map(() => new Array(DIMS).fill(0.001)),
usage: { tokens: values.length * 4 },
} as never));
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
}, 30000);
afterAll(async () => {
__setEmbedTransportForTests(null);
resetGateway();
await engine.disconnect();
});
beforeEach(async () => {
await resetPgliteState(engine);
});
describe('countChunklessPagesWithContent / listChunklessPagesWithContent', () => {
test('detects a page written via putPage that was never chunked', async () => {
await engine.putPage('stub/entity-a', {
type: 'person',
title: 'Entity A',
compiled_truth: 'Entity A is a stub written directly via putPage.',
});
expect(await engine.countChunklessPagesWithContent()).toBe(1);
const rows = await engine.listChunklessPagesWithContent();
expect(rows).toHaveLength(1);
expect(rows[0].slug).toBe('stub/entity-a');
expect(rows[0].compiled_truth).toContain('stub written directly');
});
test('excludes pages that already have chunk rows', async () => {
await engine.putPage('normal/page', { type: 'note', title: 'Normal', compiled_truth: 'hello world' });
await engine.upsertChunks('normal/page', [
{ chunk_index: 0, chunk_text: 'hello world', chunk_source: 'compiled_truth' },
]);
expect(await engine.countChunklessPagesWithContent()).toBe(0);
expect(await engine.listChunklessPagesWithContent()).toEqual([]);
});
test('excludes pages with empty content', async () => {
// Matches pages.compiled_truth's schema DEFAULT '' — the #2822 empty-put
// case. Empty content has nothing to chunk; it is not this bug class.
await engine.putPage('empty/page', { type: 'note', title: 'Empty', compiled_truth: '' });
expect(await engine.countChunklessPagesWithContent()).toBe(0);
});
test('detects a timeline-only page (empty compiled_truth, non-empty timeline)', async () => {
// healChunklessPages chunks compiled_truth AND timeline independently
// (mirrors embedPage) — the SQL predicate must not require
// compiled_truth alone or this class of page is never even detected.
await engine.putPage('timeline-only/page', {
type: 'note',
title: 'Timeline Only',
compiled_truth: '',
timeline: '2026-01-01: something happened',
});
expect(await engine.countChunklessPagesWithContent()).toBe(1);
const rows = await engine.listChunklessPagesWithContent();
expect(rows.map(r => r.slug)).toEqual(['timeline-only/page']);
});
test('excludes quarantined pages (intentionally chunkless by design)', async () => {
await engine.putPage('junk/page', {
type: 'note',
title: 'Junk',
compiled_truth: 'Cloudflare interstitial junk content',
frontmatter: { [QUARANTINE_KEY]: buildQuarantineMarker('junk_pattern', 'test fixture') },
});
expect(await engine.countChunklessPagesWithContent()).toBe(0);
});
test('excludes embed_skip pages (intentionally chunkless by design)', async () => {
await engine.putPage('oversized/page', {
type: 'note',
title: 'Oversized',
compiled_truth: 'x'.repeat(1000),
frontmatter: { [EMBED_SKIP_KEY]: buildEmbedSkipMarker(1000) },
});
expect(await engine.countChunklessPagesWithContent()).toBe(0);
});
});
describe('embed --stale chunkless-page safety net (end-to-end)', () => {
test('a putPage-only page gets chunked AND embedded in the same --stale pass', async () => {
await engine.putPage('stub/heal-me', {
type: 'person',
title: 'Heal Me',
compiled_truth: 'This entity stub was written directly via putPage and never chunked.',
});
expect(await engine.getChunks('stub/heal-me')).toEqual([]);
const result = await runEmbedCore(engine, { stale: true, quiet: true });
expect(result.chunkless_pages_healed).toBe(1);
const chunks = await engine.getChunks('stub/heal-me');
expect(chunks.length).toBeGreaterThan(0);
for (const c of chunks) {
expect(c.embedded_at).not.toBeNull();
}
});
test('pre-existing NULL-embedding chunks on other pages still get embedded (no regression)', async () => {
await engine.putPage('normal/pre-chunked', { type: 'note', title: 'Pre-chunked', compiled_truth: 'hello world' });
await engine.upsertChunks('normal/pre-chunked', [
{ chunk_index: 0, chunk_text: 'hello world', chunk_source: 'compiled_truth' },
]);
await engine.putPage('stub/heal-me-2', {
type: 'person',
title: 'Heal Me Two',
compiled_truth: 'Another chunkless stub.',
});
const result = await runEmbedCore(engine, { stale: true, quiet: true });
expect(result.chunkless_pages_healed).toBe(1);
expect(result.embedded).toBeGreaterThanOrEqual(2); // 1 pre-existing + >=1 healed
const preChunked = await engine.getChunks('normal/pre-chunked');
expect(preChunked[0]?.embedded_at).not.toBeNull();
});
test('quarantined and embed_skip pages stay chunkless — the safety net does not touch them', async () => {
await engine.putPage('junk/quarantined', {
type: 'note',
title: 'Junk',
compiled_truth: 'junk content',
frontmatter: { [QUARANTINE_KEY]: buildQuarantineMarker('junk_pattern', 'test fixture') },
});
await engine.putPage('oversized/skipped', {
type: 'note',
title: 'Oversized',
compiled_truth: 'x'.repeat(1000),
frontmatter: { [EMBED_SKIP_KEY]: buildEmbedSkipMarker(1000) },
});
const result = await runEmbedCore(engine, { stale: true, quiet: true });
expect(result.chunkless_pages_healed).toBe(0);
expect(await engine.getChunks('junk/quarantined')).toEqual([]);
expect(await engine.getChunks('oversized/skipped')).toEqual([]);
});
test('dry-run reports the would-be-healed count without writing any chunks', async () => {
await engine.putPage('stub/dry-run-only', {
type: 'person',
title: 'Dry Run Only',
compiled_truth: 'This stub must NOT be chunked by a dry run.',
});
const result = await runEmbedCore(engine, { stale: true, dryRun: true, quiet: true });
expect(result.chunkless_pages_healed).toBe(1);
expect(result.would_embed).toBeGreaterThan(0);
expect(await engine.getChunks('stub/dry-run-only')).toEqual([]); // no mutation
});
test('healthy brain (no chunkless pages) pays no extra cost and behaves exactly as before', async () => {
await engine.putPage('normal/only-page', { type: 'note', title: 'Only', compiled_truth: 'hello' });
await engine.upsertChunks('normal/only-page', [
{ chunk_index: 0, chunk_text: 'hello', chunk_source: 'compiled_truth' },
]);
const result = await runEmbedCore(engine, { stale: true, quiet: true });
expect(result.chunkless_pages_healed).toBe(0);
expect(result.embedded).toBe(1);
});
test('race mitigation: a page chunked by a concurrent writer between list and write is not clobbered', async () => {
// Review catch: healChunklessPages lists chunkless pages, chunks them
// in memory, then writes. If a concurrent writer (sync, another
// put_page/embed) chunks the SAME page in between, a naive write would
// overwrite the concurrent writer's (newer) chunks with this sweep's
// stale-content snapshot. Simulate that race by injecting a write
// immediately after listChunklessPagesWithContent returns — i.e. AFTER
// the sweep has decided to heal this page but BEFORE its own write.
await engine.putPage('stub/raced', {
type: 'person',
title: 'Raced',
compiled_truth: 'Original content the sweep read.',
});
let injected = false;
const realList = engine.listChunklessPagesWithContent.bind(engine);
const raceEngine = new Proxy(engine, {
get(target, prop, receiver) {
if (prop === 'listChunklessPagesWithContent') {
return async (opts?: Parameters<typeof realList>[0]) => {
const rows = await realList(opts);
if (!injected && rows.some(r => r.slug === 'stub/raced')) {
injected = true;
// Simulate the concurrent writer: chunks the page with DIFFERENT
// content than what the sweep just read.
await engine.upsertChunks('stub/raced', [
{ chunk_index: 0, chunk_text: 'concurrently-written chunk', chunk_source: 'compiled_truth' },
]);
}
return rows;
};
}
const value = Reflect.get(target, prop, receiver);
return typeof value === 'function' ? value.bind(target) : value;
},
}) as unknown as BrainEngine;
const result = await runEmbedCore(raceEngine, { stale: true, quiet: true });
// The race was detected and the page was skipped this pass (not
// counted as healed by this sweep) rather than clobbered.
expect(result.chunkless_pages_healed).toBe(0);
const chunks = await engine.getChunks('stub/raced');
expect(chunks).toHaveLength(1);
expect(chunks[0].chunk_text).toBe('concurrently-written chunk');
});
test('one broken chunkless page does not abort the whole --stale run (per-page failure isolation)', async () => {
// Review catch: healChunklessPages must try/catch per page. Before the
// fix, an exception from getPage/getChunks/upsertChunks for ONE
// chunkless page propagated out of healChunklessPages entirely,
// aborting embedAllStale before it even reached the normal
// NULL-embedding pass — making the safety net worse than the bug.
await engine.putPage('stub/broken', {
type: 'person',
title: 'Broken',
compiled_truth: 'This page will fail to heal.',
});
await engine.putPage('normal/unrelated', { type: 'note', title: 'Unrelated', compiled_truth: 'fine' });
await engine.upsertChunks('normal/unrelated', [
{ chunk_index: 0, chunk_text: 'fine', chunk_source: 'compiled_truth' },
]);
const brokenEngine = new Proxy(engine, {
get(target, prop, receiver) {
if (prop === 'getPage') {
return async (slug: string, opts?: unknown) => {
if (slug === 'stub/broken') throw new Error('simulated getPage failure');
return (engine.getPage as (s: string, o?: unknown) => unknown)(slug, opts);
};
}
const value = Reflect.get(target, prop, receiver);
return typeof value === 'function' ? value.bind(target) : value;
},
}) as unknown as BrainEngine;
const result = await runEmbedCore(brokenEngine, { stale: true, quiet: true });
// The broken page's failure is recorded, not swallowed silently...
expect(result.failures).toBeGreaterThanOrEqual(1);
expect(result.failure_samples.some(s => s.includes('stub/broken'))).toBe(true);
// ...but did NOT abort the run: the unrelated pre-existing stale chunk
// still got embedded in the SAME pass.
expect(result.embedded).toBeGreaterThanOrEqual(1);
const unrelatedChunks = await engine.getChunks('normal/unrelated');
expect(unrelatedChunks[0]?.embedded_at).not.toBeNull();
});
});
+25
View File
@@ -177,6 +177,31 @@ describe('runEmbed --all (parallel)', () => {
expect(result.embedded).toBe(0);
});
test('#1737 review catch: signal aborted during chunkless-page healing stops before invalidateStaleSignatureEmbeddings', async () => {
// Round-3 review finding: embedAllStale must check the abort signal
// immediately after the chunkless-page healing sweep, before falling
// through into invalidateStaleSignatureEmbeddings (a write path) —
// otherwise a caller-cancelled run could still NULL out
// signature-drifted embeddings and exit, leaving retrieval degraded.
const ac = new AbortController();
let invalidateCalled = false;
const engine = mockEngine({
countChunklessPagesWithContent: async () => 1,
listChunklessPagesWithContent: async () => {
// Simulate the caller's cancellation firing WHILE the healing sweep
// is mid-flight (e.g. worker timeout / lock loss / SIGTERM).
ac.abort(new Error('lock-lost'));
return [];
},
invalidateStaleSignatureEmbeddings: async () => { invalidateCalled = true; return 0; },
countStaleChunks: async () => 0,
});
await runEmbedCore(engine, { stale: true, signal: ac.signal });
expect(invalidateCalled).toBe(false);
});
test('respects GBRAIN_EMBED_CONCURRENCY=1 (serial)', async () => {
const pages = Array.from({ length: 5 }, (_, i) => ({ slug: `page-${i}` }));
const chunksBySlug = new Map(
+193
View File
@@ -0,0 +1,193 @@
/**
* Regression: extract_atoms wrote atoms with `engine.putPage`, a bare
* `INSERT INTO pages ... ON CONFLICT` that never chunks or embeds. Atom pages
* therefore never reached `content_chunks` and were invisible to search the
* same defect #2163 fixed for concept pages in synthesize-concepts.ts, which
* was never applied one layer down.
*
* The fix routes atoms through `serializeMarkdown` -> `importFromContent`,
* which means atom content now makes a round-trip through YAML frontmatter
* that the raw row-write bypassed. These tests lock the invariants that trip
* on: page type, the grounding frontmatter propose_takes depends on, and
* LLM-authored text that is hostile to YAML/frontmatter parsing.
*
* Mostly pure functions; the final describe block is a PGLite round-trip
* that pins the actual defect (atoms produce content_chunks rows). No model
* calls anywhere the chat gateway is stubbed.
*/
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { serializeMarkdown, parseMarkdown } from '../src/core/markdown.ts';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { runPhaseExtractAtoms } from '../src/core/cycle/extract-atoms.ts';
import type { ChatResult, ChatOpts } from '../src/core/ai/gateway.ts';
/** Mirrors the write site in extract-atoms.ts. */
function roundTrip(atom: {
title: string;
body: string;
atom_type?: string;
source_quote?: string;
lesson?: string;
concepts?: string[];
source_slug?: string;
}) {
const md = serializeMarkdown(
{
atom_type: atom.atom_type ?? "insight",
...(atom.source_slug && { source_slug: atom.source_slug }),
source_hash: 'abc123def4567890',
...(atom.source_quote && { source_quote: atom.source_quote }),
...(atom.lesson && { lesson: atom.lesson }),
...(atom.concepts && atom.concepts.length > 0 && { concepts: atom.concepts }),
extracted_at: '2026-08-02T00:00:00.000Z',
extracted_by: 'extract_atoms-v0.41.2.1',
},
atom.body,
'',
{ type: 'atom', title: atom.title, tags: [] },
);
// filePath is deliberately omitted: the atom slug is not a real file, so
// nothing but the explicit frontmatter `type:` can carry the page type.
return { md, parsed: parseMarkdown(md) };
}
describe('atom markdown round-trip — page type', () => {
test("explicit frontmatter type: atom survives with no filePath to infer from", () => {
const { parsed } = roundTrip({ title: 'Pilots de-risk procurement', body: 'A short claim.' });
expect(parsed.type).toBe('atom');
});
test('explicit type wins over a filePath that would infer something else', () => {
const { md } = roundTrip({ title: 'Pilots de-risk procurement', body: 'A short claim.' });
// meetings/ is a seeded path prefix; the frontmatter must still win, or
// re-importing an atom would silently retype it and break `type='atom'`.
expect(parseMarkdown(md, 'meetings/2026-08-02-standup.md').type).toBe('atom');
});
});
describe('atom markdown round-trip — grounding frontmatter', () => {
test('source_quote, lesson, concepts and source_slug all survive', () => {
const { parsed } = roundTrip({
title: 'Benchmarks decide agent selection',
body: 'The buyer picked on measured latency, not brand.',
atom_type: 'insight',
source_quote: 'We ran both and took the faster one.',
lesson: 'Measured evidence beats vendor claims.',
concepts: ['agent-evaluation', 'procurement'],
source_slug: '2026-06-02-polaris-stefan',
});
expect(parsed.frontmatter.source_quote).toBe('We ran both and took the faster one.');
expect(parsed.frontmatter.lesson).toBe('Measured evidence beats vendor claims.');
expect(parsed.frontmatter.concepts).toEqual(['agent-evaluation', 'procurement']);
expect(parsed.frontmatter.source_slug).toBe('2026-06-02-polaris-stefan');
expect(parsed.frontmatter.atom_type).toBe('insight');
});
test('body is preserved as compiled_truth', () => {
const body = 'Procurement stalls when the pilot has no exit criteria.';
expect(roundTrip({ title: 'Exit criteria', body }).parsed.compiled_truth.trim()).toBe(body);
});
});
describe('atom markdown round-trip — LLM text hostile to YAML', () => {
// Atom titles/quotes are model-authored free text. The raw row-write never
// serialized them; the import path does, so these must be quoted correctly.
test('a title containing a colon does not corrupt the frontmatter', () => {
const { parsed } = roundTrip({
title: 'Discovery: the part buyers actually pay for',
body: 'A claim.',
});
expect(parsed.type).toBe('atom');
expect(parsed.title).toBe('Discovery: the part buyers actually pay for');
});
test('a quote containing quotes and a hash survives', () => {
const q = 'He said "no", then asked about #pricing.';
expect(roundTrip({ title: 'Objection', body: 'A claim.', source_quote: q })
.parsed.frontmatter.source_quote).toBe(q);
});
test('a body containing a --- rule does not truncate the atom or leak frontmatter', () => {
const body = 'First point.\n\n---\n\nSecond point.';
const { parsed } = roundTrip({ title: 'Two points', body });
expect(parsed.type).toBe('atom');
expect(parsed.compiled_truth).toContain('First point.');
expect(parsed.frontmatter.extracted_by).toBe('extract_atoms-v0.41.2.1');
});
test('a title that is purely numeric stays a string', () => {
expect(roundTrip({ title: '2026', body: 'A claim.' }).parsed.title).toBe('2026');
});
});
describe("atom frontmatter must not contain undefined", () => {
// js-yaml throws on undefined, so every OPTIONAL atom field at the write
// site has to use a conditional spread rather than a bare assignment. The
// old engine.putPage row-write silently tolerated undefined; the import
// path does not. atom_type is safe to assign unconditionally only because
// parseAtomsResponse drops any atom missing it — see the !atomType guard.
test("serializeMarkdown throws if an optional field is passed as undefined", () => {
expect(() =>
serializeMarkdown({ lesson: undefined }, "body", "", {
type: "atom",
title: "t",
tags: [],
}),
).toThrow();
});
test("a conditional spread omits the key instead, which is safe", () => {
const lesson: string | undefined = undefined;
const md = serializeMarkdown({ ...(lesson ? { lesson } : {}) }, "body", "", {
type: "atom",
title: "t",
tags: [],
});
expect(parseMarkdown(md).frontmatter.lesson).toBeUndefined();
expect(parseMarkdown(md).type).toBe("atom");
});
});
describe('extracted atoms reach content_chunks (PGLite round-trip)', () => {
// Wave rider for #3762: the whole point of the fix is that atoms become
// searchable. Pre-fix, extract_atoms wrote atom pages via engine.putPage
// (a bare row upsert), so this chunk-count assertion FAILS without the
// src/core/cycle/extract-atoms.ts change — verified by reverting only the
// src file and re-running.
let engine: PGLiteEngine;
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
});
afterAll(async () => {
await engine.disconnect();
});
test('runPhaseExtractAtoms writes content_chunks rows for atom pages', async () => {
{
const chat = async (_o: ChatOpts): Promise<ChatResult> => ({
text: `[{"title":"Chunked atom","atom_type":"insight","body":"Enterprise buyers want tangible prototypes, not renders."}]`,
blocks: [{ type: 'text', text: '' }],
stopReason: 'end',
usage: { input_tokens: 500, output_tokens: 200, cache_read_tokens: 0, cache_creation_tokens: 0 },
model: 'anthropic:claude-haiku-4-5',
providerId: 'anthropic',
});
const result = await runPhaseExtractAtoms(engine, {
_transcripts: [{ filePath: '/fake/meeting.txt', content: 'transcript content', contentHash: 'abc123def4567890' }],
_pages: [],
_chat: chat,
});
expect(result.status).toBe('ok');
expect(result.details?.atoms_extracted).toBe(1);
const rows = await engine.executeRaw<{ count: number }>(
`SELECT COUNT(*)::int AS count
FROM content_chunks cc
JOIN pages p ON p.id = cc.page_id
WHERE p.type = 'atom'`,
);
expect(rows[0].count).toBeGreaterThan(0);
}
}, 60000);
});
+18
View File
@@ -146,6 +146,24 @@ test('conversation-facts allowlist includes native iMessage page types (#2756)',
expect(ALLOWED_TYPES).toContain('imessage-daily');
});
test('parses a markdown-heading turn body (## User / ## Assistant)', () => {
const body = [
'## User',
'What is the capital of France?',
'## Assistant',
'The capital of France is Paris.',
'It is also its largest city.',
].join('\n');
const msgs = parseConversationMessages(body, { fallbackDate: '2026-08-11' });
expect(msgs).toHaveLength(2);
expect(msgs[0].speaker).toBe('User');
expect(msgs[0].text).toBe('What is the capital of France?');
expect(msgs[1].speaker).toBe('Assistant');
expect(msgs[1].text).toBe(
'The capital of France is Paris.\nIt is also its largest city.',
);
});
// ---------------------------------------------------------------------------
// splitIntoSegments — PR's 5 cases verbatim plus tuning regression.
// ---------------------------------------------------------------------------
+46 -14
View File
@@ -8,11 +8,11 @@ import {
describe('extractMarkdownLinks', () => {
it('extracts relative markdown links', () => {
const content = 'Check [Pedro](../people/pedro-franceschi.md) and [Brex](../../companies/brex.md).';
const content = 'Check [Alice](../people/alice-example.md) and [Acme](../../companies/acme-example.md).';
const links = extractMarkdownLinks(content);
expect(links).toHaveLength(2);
expect(links[0].name).toBe('Pedro');
expect(links[0].relTarget).toBe('../people/pedro-franceschi.md');
expect(links[0].name).toBe('Alice');
expect(links[0].relTarget).toBe('../people/alice-example.md');
});
it('skips external URLs ending in .md', () => {
@@ -34,12 +34,12 @@ describe('extractMarkdownLinks', () => {
describe('extractLinksFromFile', () => {
it('resolves relative paths to slugs', async () => {
const content = '---\ntitle: Test\n---\nSee [Pedro](../people/pedro.md).';
const allSlugs = new Set(['people/pedro', 'deals/test-deal']);
const content = '---\ntitle: Test\n---\nSee [Alice](../people/alice.md).';
const allSlugs = new Set(['people/alice', 'deals/test-deal']);
const links = await extractLinksFromFile(content, 'deals/test-deal.md', allSlugs);
expect(links.length).toBeGreaterThanOrEqual(1);
expect(links[0].from_slug).toBe('deals/test-deal');
expect(links[0].to_slug).toBe('people/pedro');
expect(links[0].to_slug).toBe('people/alice');
});
it('skips links to non-existent pages', async () => {
@@ -50,15 +50,15 @@ describe('extractLinksFromFile', () => {
});
it('extracts frontmatter company links (v0.13, includeFrontmatter opt-in)', async () => {
const content = '---\ncompany: brex\ntype: person\n---\nContent.';
const content = '---\ncompany: acme-example\ntype: person\n---\nContent.';
// v0.13 canonical: person page with company: X → person → company works_at (outgoing).
// Resolver needs companies/brex to exist in allSlugs to emit the edge.
const allSlugs = new Set(['people/test', 'companies/brex']);
// Resolver needs companies/acme-example to exist in allSlugs to emit the edge.
const allSlugs = new Set(['people/test', 'companies/acme-example']);
const links = await extractLinksFromFile(content, 'people/test.md', allSlugs, { includeFrontmatter: true });
const companyLinks = links.filter(l => l.link_type === 'works_at');
expect(companyLinks.length).toBeGreaterThanOrEqual(1);
expect(companyLinks[0].from_slug).toBe('people/test');
expect(companyLinks[0].to_slug).toBe('companies/brex');
expect(companyLinks[0].to_slug).toBe('companies/acme-example');
});
it('extracts frontmatter investors array (v0.13: incoming direction)', async () => {
@@ -91,8 +91,8 @@ describe('extractLinksFromFile', () => {
it('frontmatter extraction is default OFF (back-compat)', async () => {
// Without includeFrontmatter, fs-source no longer auto-extracts frontmatter.
// Matches db-source behavior. User opts in with --include-frontmatter flag.
const content = '---\ncompany: brex\ntype: person\n---\nContent.';
const allSlugs = new Set(['people/test', 'companies/brex']);
const content = '---\ncompany: acme-example\ntype: person\n---\nContent.';
const allSlugs = new Set(['people/test', 'companies/acme-example']);
const links = await extractLinksFromFile(content, 'people/test.md', allSlugs);
expect(links).toEqual([]);
});
@@ -118,8 +118,8 @@ describe('extractLinksFromFile', () => {
});
it('infers deal_for type for deals -> companies', async () => {
const content = 'See [Brex](../companies/brex.md).';
const allSlugs = new Set(['deals/seed', 'companies/brex']);
const content = 'See [Acme](../companies/acme-example.md).';
const allSlugs = new Set(['deals/seed', 'companies/acme-example']);
const links = await extractLinksFromFile(content, 'deals/seed.md', allSlugs);
expect(links[0].link_type).toBe('deal_for');
});
@@ -161,6 +161,38 @@ describe('extractTimelineFromContent', () => {
expect(entries).toHaveLength(1);
});
it('does not split on hyphens inside markdown link targets', () => {
const content = `- **2025-03-18** | Referenced in [Alice](../people/alice-example.md)`;
const entries = extractTimelineFromContent(content, 'companies/acme-example');
expect(entries).toHaveLength(1);
expect(entries[0].source).toBe('markdown');
expect(entries[0].summary).toBe('Referenced in [Alice](../people/alice-example.md)');
});
it('does not split on spaced dashes inside link labels', () => {
const content = `- **2025-03-18** | Referenced in [Deals — Q1 Review](../deals/q1-review.md)`;
const entries = extractTimelineFromContent(content, 'companies/acme-example');
expect(entries).toHaveLength(1);
expect(entries[0].source).toBe('markdown');
expect(entries[0].summary).toBe('Referenced in [Deals — Q1 Review](../deals/q1-review.md)');
});
it('splits on the first spaced dash outside links', () => {
const content = `- **2025-03-18** | [Board notes](../meetings/2025-03-18-board.md) — Approved the hire`;
const entries = extractTimelineFromContent(content, 'test');
expect(entries).toHaveLength(1);
expect(entries[0].source).toBe('[Board notes](../meetings/2025-03-18-board.md)');
expect(entries[0].summary).toBe('Approved the hire');
});
it('keeps delimiterless bullet lines whole instead of dropping them', () => {
const content = `- **2025-03-18** | Imported from legacy tracker`;
const entries = extractTimelineFromContent(content, 'test');
expect(entries).toHaveLength(1);
expect(entries[0].source).toBe('markdown');
expect(entries[0].summary).toBe('Imported from legacy tracker');
});
it('extracts inline citation format entries', () => {
const content = `Closed the seed round with fund-a leading. [Source: board meeting notes, 2025-04-02]`;
const entries = extractTimelineFromContent(content, 'deals/acme-seed');

Some files were not shown because too many files have changed in this diff Show More