Compare commits

...
Author SHA1 Message Date
Garry TanandClaude Fable 5 9e678f467d Merge origin/master (v0.46.2.0) into garrytan/opencode-support
Trio kept at 0.46.4.0 (wave version, highest); CHANGELOG carries both
entries with 0.46.4.0 topmost; TODOS union-resolved (master's re-filed
pin-privacy candidate dropped — this branch ships it as DONE);
cli-flag-registry + template-repo regenerated; llms rebuilt.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 15:41:54 -07:00
Garry TanandClaude Fable 5 0c78213d92 v0.46.2.0 feat(dream): two-stage triage cascade — scored triage gates frontier synthesis (#4152) (#4164)
* feat(dream): scored triage-v1 cascade gates frontier synthesis (#4152)

Migration v129 widens dream_verdicts with score/content_type/segments/
entities/model/triage_version (legacy boolean rows = cache miss, re-judged
once). judgeSignificance emits an ordinal 0-1 salience score with
non-overlapping bands, three-window head/middle/tail sampling, and a
never-clamp out-of-range rule; degenerate verdicts are never cached.
runTriagePass (exported, shared with retriage) runs a bounded pool under a
dream.triage.max_ms wall-clock miss budget with (model, TRIAGE_VERSION)
cache validity; the gate score >= dream.triage.threshold is applied at read
time so retuning costs zero re-judging. Passing files carry a
verbatim-verified TRIAGE MAP block into the synthesis prompt;
dream.synthesize.max_turns defaults to 16 (config-restorable, pinned by a
regression test); an opt-in per-source daily cap fails open on count-query
errors and never stamps the cooldown when nothing was submitted; stranded
dream-inline-* rows self-heal behind a 1h liveness grace.

* feat(dream): retriage command — spend-gated re-score + backlog reconciliation (#4152)

gbrain dream retriage re-scores the corpus through the shared runTriagePass
and reconciles the queued synth-v2 backlog: below-threshold jobs cancel,
above-threshold jobs stranded in provably-dead dream-inline-* queues (older
than the 1h liveness grace) convert for resubmission (cancel releases the
idempotency slot), possibly-live queues are never touched, and legacy
dream:synth: keys are excluded at the SQL filter. Guardrails: upfront cost
estimate with a >$5 confirmation (--yes skips), --max-usd counts every paid
attempt including unreliable responses and spans --audit-rejects (frontier
second opinion on stride-sampled rejects), --cancel-unmatched refuses
truncated (--limit) or empty corpus scans, key-source vs payload source_id
mismatches are skipped, and statuses re-check immediately before each
cancel. dream/dream-retriage --help answer engine-free through the real CLI
(CLI_ONLY_SELF_HELP routing).

* chore: bump version and changelog (v0.46.2.0)

Docs: KEY_FILES synthesize/dream-retriage entries rewritten to current
state, cron-schedule triage-cascade section (threshold dial, retriage
recipe, mid-tier pairing guidance), six follow-up TODOs filed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(dream): close the structured-review P1/P2s — budget, liveness, preview honesty (#4152)

P1: --max-usd with --audit-rejects now refuses an unpriced synthesis model
(the audit's spend was un-estimable and silently un-metered), and an
un-estimable audit always triggers the confirmation gate. P2s: the daily
cap bounds NEW spend only — files whose idempotency keys already exist
coalesce/self-heal instead of stranding for the 24h window; a live
gbrain_cycle_locks row marks every dream-inline queue possibly-live
regardless of age (slow sequential children can outlive the 1h grace);
delayed retries in provably-dead queues convert for resubmit alongside
waiting ones; and --dry-run --cancel-unmatched counts its would-cancels
instead of understating the destructive preview.

* fix(dream): round-2 structured-review edges — audit-dollar gate, per-source lock scope, coalescible-key cap (#4152)

The spend confirmation now gates on the KNOWN estimate (a priced audit
confirms on its own dollars even when the triage model is unpriced); live
cycle locks suppress inline-queue conversions per-source (only the legacy
bare gbrain-cycle lock is global), so a busy source never indefinitely
blocks another source's cleanup; and the daily cap's existing-key credit
counts only coalescible rows (cancelled/dead keys get cleared on re-add and
would have minted fresh paid jobs past the cap).

* chore: regenerate flag registry (kill the --limit-truncated phantom from an error-string scan)

* fix(dream,models): round-3 structured-review — fail-loud stray retriage flags, dashboard shows the real triage route (#4152)

`gbrain dream --reconcile-queue` (retriage flag without the subcommand)
now exits 2 with a did-you-mean instead of silently running the full paid
maintenance cycle (the flag registry unions retriage flags into `dream`,
so the pre-dispatch validator alone can't catch it). `gbrain models` gains
the overrideKey seam so the triage row reports `models.dream.triage` as the
effective spending route (with the legacy verdict-model chain as fallback),
matching loadSynthConfig's actual resolution.

* docs: describe --max-usd honestly as an estimate-based soft stop (codex r4 P2)

* docs: update project documentation for v0.46.2.0

Sweep the remaining doc surface for #4152 triage-cascade drift:

- skills/maintain/SKILL.md: synthesize phase now describes the two-stage
  cascade (scored triage gate, read-time threshold, triage map, max_turns
  16) and points at `gbrain dream retriage` for re-scoring + backlog drain
- docs/architecture/system-of-record.md: dream_verdicts row is a scored
  triage cache rebuildable via `dream retriage --force`, not a boolean
  verdict cache
- skills/conventions/model-routing.md: utility-tier example is the dream
  triage judge (prefers models.dream.triage)
- docs/operations/spend-controls.md: name `dream retriage --max-usd` as an
  LLM-cost cap outside the embedding-spend posture scope
- skills/RESOLVER.md: add retriage trigger phrases to the dream-cycle row
- regenerate skills.lock.json + llms bundles

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: apply cross-model doc-review fixes for v0.46.2.0

Verified findings from the post-ship documentation review:

- docs/guides/cron-schedule.md: document the three missing triage knobs
  (max_chars 24000/floor 1000, max_tokens 2048/floor 256, concurrency 4
  clamped 1-16); qualify "every file scored" with the max_ms deferral;
  audit-rejects uses the synthesis model, not "frontier"
- docs/architecture/KEY_FILES.md: add migration v129 to the migrate.ts
  inventory; disambiguate the cap-hit dream_verdicts sentence (triage
  verdict stays cached, cap site writes nothing new); add the degraded
  field to details.triage
- skills/maintain/SKILL.md: --dry-run describes the scored triage (not a
  Haiku boolean filter) and points at retriage --dry-run for zero-call
  previews; qualify triage coverage with the max_ms budget; drop the
  stale "8-phase" count (ALL_PHASES outgrew it)
- skills/conventions/model-routing.md: document the models.dream.triage
  pre-read exception to the resolution chain
- docs/operations/spend-controls.md: retriage --max-usd is an
  estimate-based soft stop, not a hard cap
- regenerate skills.lock.json + llms bundles

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(ci): route dream exit codes through setCliExitVerdict; declare retriage triggers in maintain skill

Shard 4: raw process.exitCode writes in dream.ts/dream-retriage.ts are
zeroed by the flush-exit owned-verdict channel — route all nine sites
through setCliExitVerdict (doctor.ts pattern).

Shard 3: RESOLVER.md routes "retriage the backlog" / "re-score the
triage" to skills/maintain — declare both in the skill's frontmatter
triggers (round-trip pin) + regenerate skills.lock.json.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 15:36:25 -07:00
Garry TanandClaude Fable 5 e38e4a845b v0.46.4.0 chore(release): re-bump 0.46.2.0 → 0.46.4.0 (version slots claimed by in-flight PRs)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 15:20:13 -07:00
Garry TanandClaude Fable 5 526595ce01 docs: cross-model doc-review fixes — opencode roster + probe/remote accuracy
Findings from the standard post-ship Codex doc review, verified against
the shipped code: the bootstrap guide's intro, install table, and door
inventory still described a two-client product (opencode added to all
three); INSTALL_FOR_AGENTS' grok section said the personal-agent path is
Claude Code/Codex only; OPENCODE.md called its recipe the bootstrap
"manual equivalent" (bootstrap additionally pins GBRAIN_SOURCE + full
surface), lacked the mcp-list trust caution the pin doc carries, and
never documented the connect --install / --force remote lane; the pin
doc's provisioning bullet now names the pack-verify-install posture the
CI job actually runs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 13:22:41 -07:00
Garry TanandClaude Fable 5 2d8c772171 docs: fold the post-review opencode fix-wave behaviors into KEY_FILES
Three current-state completions the fix agents didn't carry into the
per-file index: connect --agent opencode --install's --force semantics
(maps to the writer's allowReplaceOtherSource — ours-at-old-url
replaceable, foreign still refuses), removeOpencodeMcpEntry's
skipOtherSource option, and bootstrap uninstall's expectation-keyed
opencode sweep (both merged global filenames under the config-dir lock
plus the project file; other-workspace entries skipped with a note).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 13:14:33 -07:00
Garry TanandClaude Fable 5 bdef2dada5 v0.46.2.0 feat(opencode): full-parity client support — bootstrap, harness, connect, claw-test, e2e door
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 12:54:44 -07:00
Garry TanandClaude Fable 5 cc902834d5 chore: regen cli-flag-registry after review-fix waves
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 12:52:40 -07:00
Garry TanandClaude Fable 5 eaca6a94d8 fix: adversarial-review fix wave — cross-model (Claude + Codex) findings
P0: the registration probe no longer executes from the invoking cwd
(mkdtemp cwd for user scope; project scope skips the live probe —
parse-back is authoritative), so a cloned repo's committed opencode.json
can't gain code execution during bootstrap. Probe timeouts now kill the
child (SIGTERM→SIGKILL, bounded drain) instead of abandoning it over the
PGLite lock. Global writes reconcile mcp.<name> across BOTH merged
global filenames. Stale-target cleanup takes the target config-dir lock.
Backups are unique per operation; rollback is content-guarded and
remove-path backups tighten to 0600 when token-bearing. connect --force
now works on the opencode lane with url-appropriate refusal copy.
atomic-write cleans tmp litter and survives the exists/realpath race.
Bun-lane fingerprint is fail-closed on gbrain-less args. Scope answers
trim. bounded() clears its drain-cap timer. CI installs opencode from
byte-verified tarballs. Consent-semantics + hermetic-live-runner
follow-ups filed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 12:51:29 -07:00
Garry TanandClaude Fable 5 7df02405bc fix: pre-landing review fixes — review-army + red-team wave
Security: opencode error-path snippets render <paste-token-here> instead
of the live bearer; the harness opencode catch redacts like the claude
lane; test-harness child envs unconditionally drop GITHUB_TOKEN/ACTIONS_*.
Correctness: bootstrap-lock coverage for every shared-config writer
(hooks/connect/uninstall); uninstall + step-aside gate sweep BOTH global
opencode filenames (merged namespace); uninstall passes a sourceId
expectation and skips other-workspace entries; cross-kind fingerprint
matches classify ours-other-source instead of silently replacing;
dangling-symlink writes preserve the link; failed-smoke rollback restores
atomically; registration probe pins OPENCODE_DISABLE_AUTOUPDATE, a 20s
cap, ANSI-stripped exact-name matching. Guard: check-opencode-pin now
cross-checks per-platform integrities + every OPENCODE_VERSION copy.
Plus deny-path/uninstall/rollback/truth-table/symlink test coverage,
help-prose cosmetics, downgrade doc note, 3 P3 TODOs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 12:02:50 -07:00
Garry TanandClaude Fable 5 bf0e8795f8 fix(bootstrap): close two plan-audit gaps — ACCESS_POLICY opencode scope paragraph + doctor host:opencode pin
Plan-completion audit (ship Step 8) flagged both as PARTIAL: the
ACCESS_POLICY template's MCP-scope section didn't state opencode's
inverted default (user-global; project spawns with NO trust prompt),
and bootstrap_harness_health had no named pin proving an opencode
receipt flows through the host-generic filter. Template-repo regenerated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 11:13:26 -07:00
Garry TanandClaude Fable 5 a534240d2c test: coverage pins for the opencode channel widenings
Ship-audit additions: hook.ts --harness opencode flag-parse attribution
end-to-end, and 'opencode' membership in VOLUNTEER_CHANNELS +
isHarnessChannel (a regression here silently rebadges opencode deliveries
as claude-code).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 10:59:54 -07:00
Garry Tan 6db8dc8a34 Merge remote-tracking branch 'origin/master' into garrytan/opencode-support
# Conflicts:
#	TODOS.md
2026-08-15 10:45:27 -07:00
Garry TanandClaude Fable 5 750275533f chore(todos): file opencode-wave follow-ups + retire fired triggers by title
Door-adapter extraction (test-side) and cadence policy: DONE — the 4th-door
trigger fired. CI-tail composite action re-filed with the sharpened trigger
(first GREEN grok-door AND opencode-door dispatches). hermesChildEnv
GITHUB_* backport: DONE via the shared factory. PIN-doc privacy guard:
DONE (check-pin-doc-privacy.sh in verify). New follow-ups: first-dispatch
watch, plugin/event-system wiring (ambient-recall lane), BrainBench
adapter (with hermes+grok), connect --oauth authorization-code lane,
OPENCODE_CONFIG* re-observation on bumps, opencode-install PTY promotion.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 10:26:44 -07:00
Garry TanandClaude Fable 5 17e36adb57 dx(explore): opencode-install TTY scenario — bootstrap paste block under the real interactive TUI
Unlike grok's brain-only prompt, opencode gets the FULL bootstrap paste
block (it is a bootstrap-supported harness) under a hermetic HOME + both
XDG dirs, the double autoupdate kill (config seed + env), BROWSER=false so
a first-run can never bounce the operator's browser, and auth.json
pre-registered for the secret scrub. Keyless posture INVERTS the grok
scenario: the anonymous free tier means a --keyless run should COMPLETE
the flow — a sign-in wall here is itself a pin-refresh signal, and the
generic early-stop in runInstallSession records it as friction if it ever
appears.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 10:25:52 -07:00
Garry TanandClaude Fable 5 519c3748da docs: opencode across the install/testing/architecture surface
README client roster + remote-connect bullet (with the not-OpenClaw
disambiguation and the bootstrap-supported banner), INSTALL_FOR_AGENTS
'If you are opencode' block (routes bootstrap-capable readers to the
runbook; brain-only registration otherwise), docs/INSTALL per-client list,
MEMORY_VERBS register snippet, bootstrap guide (degradation-matrix row
naming the INVERTED scope default + rationale; harness-mode opencode
bullet; dx-explore scenario line), ambient-recall/push-context harness
mentions, and KEY_FILES current-state entries (opencode-json.ts,
atomic-write.ts, connect/harness/hooks/claw-test entry refreshes).
llms bundles regenerated (build:llms chaser).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 10:24:35 -07:00
Garry TanandClaude Fable 5 fcbf547841 ci(heavy): opencode-door job (day-one full posture, keyless SMOKE) + canary leg + pin guards + hermes installer re-pin
opencode-door takes hermes-door's triggers (nightly + labels + dispatch;
cadence policy: nightly for the NEWEST door agent) with grok-door's
internals — keyless-first ordering, secretless npm provisioning with
wrapper AND per-platform integrity pre-checks, pass-count + paid sentinels,
mid-job version-drift tripwire, evidence scrub RE-KEYED to
ANTHROPIC_API_KEY + auth.json (not XAI/mcp_credentials), unconditional
credential removal. No dedicated dispatch input (any workflow_dispatch
already passes the non-PR arm — an input would be dead yaml). The keyless
tier includes the nonce SMOKE (free tier), so the core door needs NO
secret; the paid anthropic leg rides the secret hermes-door already
consumes. opencode-door-canary lands IN-WAVE (schedule-only,
continue-on-error, unpinned latest): opencode ships near-continuously — a
red canary is a pin-refresh signal, never a gate. real-agent-e2e adds the
opencode door file + env pins.

Guards: check-opencode-pin.sh (stamp↔workflow parity, job-block anchored so
the UNPINNED canary leg cannot satisfy it; fail-closed when the door exists
without the pin doc) and check-pin-doc-privacy.sh (placeholder discipline
for ALL docs/mcp/*-CLI-PIN.md — no operator home paths, no key-shaped
material outside sha512 pins, no non-example emails), both in bun run
verify + guards-manifest, both with fixture-tree bun tests.

Maintenance: hermes-door installer pin refreshed (upstream install.sh
drifted past the prior digest — last two nightlies red; reviewed: the
--commit payload-pin path is intact and the payload pins are unchanged).
docs/TESTING.md gains the opencode door entry + the door cadence policy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 10:21:12 -07:00
Garry TanandClaude Fable 5 422438bfe7 test(e2e): opencode door — split-gated real-binary e2e on the extracted family (keyless SMOKE included)
The door goes a step beyond grok's split gating: opencode's anonymous free
tier drives MCP tool calls with zero credentials (observed, load-bearing),
so even the nonce SMOKE runs keyless. Tiers: T1 bare-semver version pin
(the SST-vs-claimant discriminator), T2 INSTALL via the documented
'mcp add … -- gbrain serve --surface verbs' shape + the honest 'mcp list'
discriminator (spawns servers; ✓/✗ text asserted — exit code is 0 even on
failure), T2b spawn-gate CANARY (a project-config decoy is spawn-attempted
with no trust prompt — if this ever gates, the bootstrap user-global
default rationale changed: re-observe), T3 writer parity (gbrain's
opencode-json.ts output handshakes through the real binary; cross-tool
preservation both ways incl. the autoupdate seed), T4 keyless SMOKE
(per-run nonce + STRUCTURAL gbrain_* tool_use proof via parseOpencodeJsonl,
list preflight before any turn, 2 attempts), and the paid T5 anthropic leg
(hasOpencodeAuth-gated; self-validating models-gate pins the model id
BEFORE any spend). Hermeticity: HOME + both XDG dirs per child, tmp cwds,
config/credential tripwire over the operator's real opencode state,
checkout guard, --pure on every probe (mcp list autoloads plugins), --pure
placed BEFORE the '--' separator (a trailing append lands inside the server
command — caught live). run-e2e.sh scrubs the OPENCODE_ prefix.

Verified live: 6/6 pass in 35.8s (keyless tier + paid anthropic leg)
against the hermetically pinned opencode-ai@1.18.18.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 10:13:21 -07:00
Garry TanandClaude Fable 5 ccc9f171c0 refactor(test): door-family extraction in agent-harness — shared resolver/childEnv/spawn core; grok+hermes ported; opencode first consumer
The 'P3 — Door-adapter extraction + CI-tail composite action' TODO armed
this at 'the NEXT door agent (4th)' — opencode is the 4th. Test-side only
(the composite CI-tail action stays deferred until the first green
grok-door AND opencode-door dispatches; workflow yaml can't be proven
locally).

- makeBinaryResolver: one shape (fail-closed $*_BIN > which > landing
  spots + nvm/PATH sweeps); claude/codex/hermes/grok resolvers become
  factory products with identical candidate lists.
- makeAgentChildEnv + GITHUB_STEP_META_KEYS: hermeticChildEnv + per-agent
  overrides + key deletion + the step-metadata scrub + binDir prepend.
  hermesChildEnv GAINS the GITHUB_* deletion via the factory (the filed P2
  backport; truth-table extended).
- runOneShotSpawn: shared timeout/kill/kill-9-escalation/bounded-drain core;
  hermesOneShotTurn gains the escalation + bounded drain (strictly safer,
  nothing pinned the old unbounded wait); grokOneShotTurn is now a thin argv
  builder over it.
- 5a-opencode family (first consumer): resolveOpencodeBinary (fail-closed
  OPENCODE_BIN), hasOpencodeAuth (PAID-leg-only gate — the keyless free
  tier carries the core SMOKE), opencodeChildEnv (HOME + BOTH XDG dirs,
  anthropic re-admission, other-provider + OPENCODE_CONFIG* shadow-trio
  deletes), seedOpencodeConfig (config half of the double autoupdate kill),
  opencodeOneShotTurn, and parseOpencodeJsonl (event shapes pinned from the
  live v1.18.18 observation — {type,part} with part.text / part.tool).

EV1 gate (local keyless grok-door): 4 pass / paid-skip in 20.5s BEFORE and
AFTER the port, against a hermetically npm-pinned @xai-official/grok@1.0.4.
The hermes door self-skips without a binary — its port is pinned by the
unit truth-tables (stated honestly, per the plan).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 10:08:18 -07:00
Garry TanandClaude Fable 5 a90547e6fe v0.46.1.0 feat(minions,db): per-job process isolation + pool-starvation fixes (garrytan-agents#5, #6) (#4151)
* fix(minions): cancel abandoned probe + lock-renewal queries instead of orphaning pool slots (#6)

Three hot paths raced a live query against a timer and abandoned the loser,
leaving the query holding a checked-out pool slot for its full server-side
duration. Under a saturated transaction-mode pooler those orphaned slots
starve lock renewal ('lock-renewal-failed' cascades) and the health probe.

- Health probe: pass the deadline AbortController's signal into
  executeRaw('SELECT 1') so a hung probe is cancelled via postgres.js
  .cancel() (runUnsafe already wires signal -> pending.cancel()).
- Minion lock renewal: LockRenewalDeps.renewLock widened with optional
  { signal }; runLockRenewalTick aborts a per-call controller when the
  timeout wins the race; MinionQueue.renewLock forwards the signal to
  executeRawDirect. Optional-param widening keeps the 14 existing hermetic
  tests compiling untouched.
- Cycle drain renewal (synthesize.ts): the inline best-effort tick had no
  per-call timeout and no re-entrancy guard, so a hung renewLock stacked a
  fresh checked-out slot per interval firing. Extracted as exported
  runDrainRenewalTick (per-call signal + timeout + swallow) behind a
  tick-in-flight guard.

Tests: 2 new signal paths in worker-lock-renewal.test.ts, probe-signal
assertion in worker-supervised-db-probe.test.ts, new hermetic
minion-queue-renewlock-signal.test.ts + cycle-drain-renewal.test.ts.
scripts/check-worker-lock-renewal-shape.sh stays green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(minions,db): cancel timed-out wedge probes + db-lock refreshes (#6)

Same abandoned-racer class as the previous commit, in two more spots:

- probeQueueState raced probeQueueStateInner against its 1500ms budget but
  the losing wedge/age queries kept running on the pool after the race
  resolved — under pool exhaustion (the exact regime the probe exists to
  detect) the orphaned query held a slot and made the exhaustion worse. The
  timeout now aborts a per-probe signal threaded through queryWedgeSignals
  and the oldest-waiting age query. Closes the filed TODOS entry.
- withRefreshingLock raced handle.refresh() against heartbeatTimeoutMs the
  same way; DbLockHandle.refresh now accepts { signal } (Postgres forwards
  to executeRawDirect; PGLite ignores it — no pool to starve), the timeout
  aborts it, and a re-entrancy guard stops overlapping ticks (15s min
  cadence vs 30s default timeout could stack two).

Tests: new hermetic queue-probe-cancellation.test.ts (signal threading,
timeout-aborts, fast-path-not-aborted, fail-open contract).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(db): explicit jittered max_lifetime on all four client pools (#6)

Makes the pool connection lifetime explicit at every postgres() call site
(db.ts module singleton, engine instance pool, ConnectionManager read +
direct pools) and adds GBRAIN_POOL_MAX_LIFETIME_S as an incident escape
hatch (N seconds; 0 disables recycling).

NOT a behavior change at default: postgres.js (verified against the pinned
3.4.9) already defaults max_lifetime to 60*(30+rand*30) — 30-60 min,
jittered per pool — and max_lifetime only recycles connections as they
return to the pool; it cannot reclaim a leaked checkout. Framed accordingly:
explicitness + operator knob, not a fix for the starvation class (that is
the cancellation work in the two prior commits).

Tests: hermetic resolver suite (env forms, 0-disables, jitter bounds,
warn-once on invalid values, per-call jitter variance).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(minions): pool-state health-probe diagnostics — pool starved vs server unreachable (#6)

"[health] DB unreachable" sent operators chasing database capacity while the
real fault was client-side: the server sat at ~10% of max_connections. The
probe now names the failing layer:

- New src/core/minions/db-probe.ts (hermetic, injected-deps — the
  lock-renewal-tick pattern): on read-pool probe failure, a 3s direct-lane
  SELECT 1 disambiguates. Direct OK -> verdict 'pool_starved' ("server IS
  reachable; the fault is in the transaction-pooler path — client pool
  exhaustion or a pooler-layer fault", deliberately an honest disjunction).
  Both fail -> 'server_unreachable'. No direct lane -> 'unknown'. Both
  probes carry AbortSignals — a hung probe is cancelled, never abandoned.
- New src/core/pool-gauge.ts: approximate in-flight counters at the engine's
  raw/direct/reserved/transaction seams, surfaced via a duck-typed
  PostgresEngine.getPoolDiagnostics() (no BrainEngine churn, no PGLite
  stub). Explicitly labeled a tracked SUBSET — template-path traffic is
  untracked and no waiter/available figures are derived (that would be
  invented telemetry). Counters use try/finally (runUnsafe throws
  synchronously on a pre-aborted signal) and clamp at zero.
- worker.ts probe adapter emits the verdict in every failure line and on the
  final unhealthy payload; exit semantics UNCHANGED (exiting on a starved
  pool is correct recovery — it frees all client-held slots).
- jobs.ts: verdict-aware fatal text, plus a startup warning when a
  Supabase-shaped engine is running single-pool (kill-switch collapse used
  to be silent — renewal + probes + workload all sharing one pool is the
  precondition for this incident class).
- Runbook: verdict interpretation table in queue-operations-runbook.md.

Tests: pool-gauge.test.ts (pure + engine seams incl. rejected-query and
sync-throw leak guards), db-probe.test.ts (full verdict matrix, signal
cancellation, fail-open diagnostics, no-waiter-wording pin).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(db): route withReservedConnection to the direct session pool when dual-pool is active (#6)

Long-hold reserved work — CREATE INDEX CONCURRENTLY (vector-index), non-
transactional migration DDL, and backfill BEGIN..COMMIT batches (the observed
353s COMMIT session) — previously reserved from the worker's shared READ
pool, pinning slots under the 5-min pooler statement_timeout. It now reserves
from the DIRECT session lane, whose 30-min statement_timeout and
maintenance_work_mem GUCs are the right fit, and stops competing with handler
workload.

Heartbeat protection: concurrent direct reserves are capped at
directPoolSize - 1 (default 2 of 3) via a per-process semaphore so
claim/renewLock always keep >= 1 direct slot; overflow falls back to the
read pool — exactly the pre-change behavior, so this commit is strictly
never-worse than master. (Deliberate rejection of queue-for-a-permit: that
would block migrations behind multi-minute index builds. Per-process is the
correct scope: each process owns its own direct pool, so a CLI migration
cannot starve a worker's heartbeats.) Never rerouted inside an open
transaction (same guard shape as executeRawDirect); kill-switch collapse
degrades to status quo. Callers unchanged.

Tests: postgres-engine-reserved-routing.test.ts — direct when active, read
when kill-switched/in-tx, semaphore cap + overflow + permit release on fn
throw and on reserve() failure.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(minions): job-isolation protocol, child exit codes, shared job-context builder (#5)

Foundation for per-job process isolation (no behavior change yet):

- job-isolation.ts — the parent<->child protocol: atomic outcome-file codec
  (tmp+rename; 32MiB decode cap that throws UnrecoverableError so oversize
  results die LOUDLY on attempt 1 instead of retrying identically or being
  silently truncated; decode errors report byte counts, never file content),
  handler-error encode/reconstruct preserving the two instanceof branches
  executeJob dispatches on (UnrecoverableError, RateLeaseUnavailableError),
  child argv/env contract, child-CLI resolution (env override -> compiled
  binary -> bun-dev fallback -> null for fail-fast), and killProcessGroup —
  children run detached in their own process group because SIGKILL on a tini
  pid alone kills tini and orphans the handler grandchild (tini cannot
  forward SIGKILL), and Bun rejects negative pids in process.kill()
  (oven-sh/bun#15791) so group signaling falls back to POSIX /bin/kill.
- worker-exit-codes.ts — reserved run-child codes 13 (usage/PGLite),
  14 (not claimed / token mismatch), 15 (result-write failed). Result-file
  presence, not the exit code, classifies the normal path: a reported
  handler FAILURE is still exit 0.
- job-context.ts — MinionJobContext builder extracted verbatim from
  executeJob so the child wires the exact same token-fenced DB callbacks;
  worker.ts now calls it (behavioral no-op, full minions suite green).

Tests: job-isolation-protocol.test.ts — codec round-trip + all decode
failure paths, instanceof reconstruction, invocation resolution, and REAL
detached-process group-kill tests incl. the grandchild-death guarantee
(runs under bun test, so the Bun negative-pid fallback is exercised for
real, not mocked).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(minions): hidden 'jobs run-child' single-job entrypoint (#5)

The child side of process isolation. `gbrain jobs run-child --job-id N`
(internal; spawned by the worker, absent from user help):

- re-reads the job row and validates status='active' + lock-token match
  before running anything — a reclaimed/cancelled job exits 14 with the
  handler never invoked (the DB stays ground truth; no payload
  serialization across the boundary);
- registers the same handler surface as the worker via
  registerBuiltinHandlers({quiet}) — which includes plugin discovery, so
  plugin subagent jobs isolate identically — resolved through the new
  MinionWorker.getHandler() accessor;
- builds the shared token-fenced MinionJobContext against the CHILD's own
  engine, runs the handler, and writes ONE atomic outcome file: handler
  failure is an encoded error outcome with exit 0 (a reported failure is a
  successful report); only write-failure exits 15;
- runs NO worker machinery (no probe/stall/lock timers — the parent owns
  liveness). Installs a SIGTERM handler (fires ctx.signal + shutdownSignal
  so handlers get the drain window to finish and report) and a
  parent-liveness watchdog polling process.kill(parentPid, 0) — a ppid
  check is dead code under tini — that aborts the handler and hard-exits
  after a grace so orphaned LLM-bound work stops burning spend;
- CLI layer owns engine.disconnect() + process.exit() (engine-ownership
  invariant); PGLite exits 13 (isolation is Postgres-only, like jobs work).

Flag registry regenerated for the internal job-id flag.

Tests: run-child-entry.test.ts against real in-memory PGLite with a REAL
claim-minted token — success (incl. a fenced updateProgress landing),
handler-failure outcome, token-mismatch never-runs, missing job, missing
handler, and the parent-death watchdog aborting a live handler.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(minions): process isolation — run claimed jobs in SIGKILL-able children (#5)

The parent-side seam. executeJob swaps ONE line — handler(context) vs
runJobInChild(...) — and every existing reporting branch (completeJob,
failJob dead/delayed, lease release, infra-abort no-burn) is reused verbatim
on the child's reconstructed outcome. Blast radius of a stuck or crashing
handler drops from N in-flight jobs to exactly one.

child-job-runner.ts:
- detached spawn (own process group) + tini wrap when available; stdio
  ['ignore','inherit','inherit'] so handler logs stream to the operator;
  per-job lifecycle log lines (spawned / exited code+signal);
- per-job abort -> group SIGTERM now, group SIGKILL at +25s (inside the 30s
  force-evict window, which stays as an untouched backstop) — force-eviction
  is now a real kill, not an abandonment;
- worker shutdown -> same SIGTERM so the child's handlers get the drain
  window to finish AND report; a child that reported before the kill
  completes normally; one that couldn't throws ChildWorkerShutdownError,
  which the worker RELEASES with no attempt burned — routine deploys must
  not burn attempts (codex-2 #7);
- pre-exec spawn failure -> ChildSpawnInfraError, also released with no
  attempt burned (one bad CLI path must not dead-letter a queue);
- child env contract: fenced lock token, outcome path, parent pid for the
  orphan watchdog, GBRAIN_POOL_SIZE=3 + GBRAIN_DIRECT_POOL_SIZE=1 bounds
  (children run no heartbeats; sockets die with the process — the point).

worker.ts: MinionWorkerOpts gains jobIsolation / childCliInvocation /
childTiniPath (defaults preserve inline behavior exactly); when isolated the
parent-side MinionJobContext is not built at all (the child builds its own).

Tests: child-job-runner.test.ts (real .mjs children: success + env contract,
error/lease outcome reconstruction, crash, SIGTERM-ignorer -> group SIGKILL,
pre-aborted, spawn ENOENT, both shutdown semantics);
worker-job-isolation.test.ts (real PGLite worker end-to-end: claim -> child
-> fenced completeJob with the REAL claim token, failJob on error outcome,
crash burns attempt, spawn failure releases with zero attempts burned).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(cli): --job-isolation flag, supervisor pass-through, e2e lane (#5)

The user-facing surface for per-job process isolation:

- jobs work --job-isolation <inline|process> (space or = form; env fallback
  GBRAIN_JOB_ISOLATION; default inline — fully opt-in). With 'process' the
  worker resolves the child CLI ONCE at startup (GBRAIN_JOB_CHILD_CLI ->
  compiled binary -> bun-dev fallback) and REFUSES to start on an
  unresolvable/nonexistent path — a bad path discovered per-job would stall
  the queue one released claim at a time. detectTini() wraps children when
  available. Startup banner names the mode + child CLI; combining with
  --max-rss prints a note that the watchdog now covers the worker only.
- jobs supervisor --job-isolation passes through via buildWorkerArgs as a
  CONDITIONAL push — inline/omitted keeps existing deployments' worker argv
  byte-identical (pinned arrays in supervisor-build-worker-args.test.ts are
  untouched; two new cases added).
- pool_starved fatal text now names the flag as a remedy (handler
  connections die with each job's child).
- help text for work + supervisor + the jobs index; flag registry
  regenerated.
- NEW test/e2e/job-isolation.test.ts, wired into e2e.yml tier1 EXPLICITLY —
  the workflow runs only named files (no glob), so an unwired e2e file would
  be silent coverage loss. Legs: concurrency-3 isolated drain through real
  children against real Postgres (the child-pool topology), and the REAL
  `jobs run-child` CLI entrypoint end-to-end (engine bootstrap, quiet
  handler registry, token validation, outcome protocol). Follows the #4128
  ambient-URL-guard conventions (explicit env in the e2e lane).
- serialization parity (codex-2 #8): a non-JSONB-serializable result fails
  loudly in BOTH modes (inline completeJob serialization vs child exit 15) —
  isolation never falsely completes a job inline mode would have failed.

Tests: jobs-isolation-flag.test.ts (parser matrix), extended
supervisor-build-worker-args + worker-job-isolation, cli-flag-validation
green via regen, jobs-subcommand-help.serial green (engine-free help path).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(minions,db): pool-starvation diagnostics + job isolation reference; file the follow-ups (#5, #6)

- KEY_FILES.md: entries for the six new modules (job-isolation,
  child-job-runner, run-child, job-context, db-probe, pool-gauge) and
  current-state updates for worker/queue/supervisor/jobs/db/db-lock/
  lock-renewal-tick/synthesize.
- minions-deployment.md: a --job-isolation section modeled on --nice — how
  the parent/child split works, preserved error semantics, orphan story, and
  the sizing notes (pooler CLIENT connection math: concurrency 15 ~ 73;
  --max-rss covers the worker only; spawn cost guidance; the lock token is a
  fencing token, not a secret).
- TESTING.md: inventory entries for the 12 new unit files + the e2e lane
  (which is wired EXPLICITLY into e2e.yml tier1 — no glob exists).
- TODOS.md: filed the 10 follow-ups, headlined by the P1-companion
  nested-checkout audit (the strongest remaining #6 root-cause candidate —
  this wave mitigates the starvation class and fixes the diagnostic; it does
  not claim to close every leak path), plus per-handler isolation policy,
  per-child RSS caps, the connection-budget clamp, autopilot pass-through,
  connection-audit release events, the doctor connection_routing check, and
  Sql-proxy checkout instrumentation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: repoint the deadlineAtMs structural pin at the extracted job-context builder

The deadline-plumbing structural test grepped worker.ts for the literal
deadlineAtMs derivation, which moved verbatim into job-context.ts (the
builder shared by inline mode and 'jobs run-child'). The pin now checks the
derivation in job-context.ts AND that worker.ts calls buildJobContext — the
same contract, at its new home.

Full-suite triage note: an isolated A/B of the 22 files that failed in the
parallel full-suite run shows IDENTICAL results on this branch and on the
master base (290 pass / 5 fail — doctor-minions-check + unified-multimodal,
both env-dependent) — zero regression delta from this wave.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(minions,db): adversarial-review hardening — shutdown/attempt semantics, TOCTOU semaphore, gauge + parity gaps (#5, #6)

A 23-agent adversarial review (5 dimension finders + per-finding refuters)
over the wave's diff confirmed 12 defects; all fixed here:

- [P2] run-child conflated worker SIGTERM with the per-job abort: cooperative
  handlers bailed mid-deploy, reported an error outcome, and the parent
  BURNED an attempt per routine deploy — while signal-ignoring handlers got
  the no-burn release (the exact inversion of the shutdown guarantee).
  SIGTERM now fires ONLY shutdownSignal (inline signal-separation parity —
  handlers finish + report inside the drain window), parent death still
  aborts both, and the parent classifies an ERROR outcome that arrives
  during shutdown as ChildWorkerShutdownError (released, not burned; a
  genuinely-failing job coinciding with a deploy gets one free retry).
- [P2] the reserved-direct semaphore was a check-then-increment spanning
  `await ddl()` — same-tick concurrent reserves could overshoot the cap and
  starve the heartbeat slot it exists to protect. The permit is now taken in
  the same synchronous frame as the check.
- [P3] RSS-watchdog drain (gracefulShutdown aborts BOTH signals, reason
  'watchdog') was classified as a per-job abort and burned attempts on
  innocent isolated jobs. Shutdown classification now wins unless the
  per-job reason is job-targeted (timeout/cancel/lock-*).
- [P3] force-evict's failJob('dead') could race executeJob's own recording
  in isolation mode (group SIGKILL at 25s + slow decode > 30s window) and
  dead-letter a job with attempts remaining — skipped when isolated (the
  inFlight eviction, which is what unblocks the worker, stays).
- [P3] child bootstrap exits were burned as handler crashes: exit 13 →
  ChildSpawnInfraError (release), exit 14 → new ChildNotClaimedError
  (release; the claim is provably owned elsewhere).
- [P3] missing handler in the child was 'generic' (retried to max_attempts)
  vs inline's immediate dead-letter — now 'unrecoverable' (parity).
- [P3] result-shape parity: the {value: x} wrap now happens CHILD-side,
  before JSON serialization, so Date/toJSON results can't flip the wrap
  decision across the boundary.
- [P3] child env no longer raises a stricter user GBRAIN_POOL_SIZE (pooler
  MaxClients tuning respected; explicit GBRAIN_JOB_CHILD_POOL_SIZE wins;
  invalid values fall back instead of flowing to the 10-conn fallback).
- [P3] transaction() gauge used a chained .finally that a synchronous
  begin() throw (nested tx on a clone) would skip — now try/finally.
- [P3 vacuity x3] new pins: db-lock heartbeat cancellation wiring +
  re-entrancy, the synthesize drain-loop guard + tick call (the shape guard
  only covers worker.ts), and GBRAIN_POOL_MAX_LIFETIME_S reaching a REAL
  constructed pool (postgres() is lazy — no I/O).

New tests: error-outcome-during-shutdown, watchdog double-abort,
timeout-beats-shutdown precedence, bootstrap exit codes, pool-size env
matrix, SIGTERM-only-aborts-shutdown (in-process emit), child-side wrap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(minions,db): pre-landing review fixes — specialist + red-team findings (#5, #6)

Ship's review army (5 specialists + red team over the full diff; 0 critical
from specialists, 3 confirmed critical from red team) — all findings fixed:

Red team (critical):
- reserved-direct cap: removed the Math.max(1, size-1) floor — at
  direct_pool_size=1 it let a multi-minute reserve consume the ONLY direct
  session and starve claim/renewLock heartbeats (the #6 class reintroduced).
  cap = size - 1, direct routing only when cap >= 1; size<=1 uses the read
  pool (true status quo). Pinned by a size=1 routing test.
- silent group-kill failure: the SIGKILL escalation now logs loudly when
  delivery fails (distroless hosts without /bin/kill would otherwise void
  the kill guarantee with zero diagnostics while the job duplicated
  elsewhere), and skips the redundant signal when the child already exited.
- spawn-failure circuit breaker: a deterministically broken child CLI looped
  claim/release forever, invisible to the stall detector (every settle
  refreshes the progress clock). After 3 consecutive spawn/bootstrap
  failures the worker emits unhealthy(child_spawn_failing) for a
  process-manager restart; counter resets on any spawn that runs. Plus the
  predicate-mismatch guard: jobIsolation 'process' without childCliInvocation
  now throws at construction (it silently ran handlers inline while the
  evict path believed it was isolated).

Specialists (informational, all applied):
- performance: parent-side outcome decode is async (a 32MiB-capped file must
  not block the event loop running renewal ticks); /bin/kill by absolute
  path (also the security finding).
- security: lease payloads are shape-validated before reconstruction
  (corrupt outcome files degrade to generic); the child-CLI override is
  canonicalized to an absolute path so the fail-fast check validates the
  binary that actually spawns.
- data-migration: max_lifetime default is now a per-CONNECTION jitter
  FUNCTION (matching the postgres.js built-in shape — a pre-evaluated number
  synchronized every connection in a pool onto one recycle deadline);
  reserved.release() throws no longer leak the gauge or the direct permit.
- testing: child harnesses use a readiness handshake instead of fixed 400ms
  sleeps (CI-load flake); the orphan-watchdog test uses a real reaped pid
  (a magic high pid is allocatable under Linux pid_max); new pins for the
  dual-pool probe gating (probeDirect wired ONLY when isDualPoolActive),
  the executeRawDirect/transaction gauge seams incl. the sync begin()-throw
  leak guard, the ddl()-throw read-pool fallback, and the --job-isolation
  help text.
- maintainability: abort-reason literals shared via types.ts (dead
  'cancel'/'cancelled' entries dropped), DEFAULT_DIRECT_POOL_SIZE and
  CHILD_READ_POOL_MAX named, redundant dynamic imports removed, unrefTimer
  helper, getConnectionRouting shared accessor, docstring + fixture-header
  corrections.

Deferred with TODOS entries: raceWithAbortTimeout DRY helper (5 sites), lazy
handler resolution in run-child, e2e-lane negative tests for the run-child
bootstrap guards + operator-flow messages, behavioral withRefreshingLock test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: bump version and changelog (v0.46.1.0)

Issues #5 + #6 wave: pool-starvation cancellation + diagnostics, and opt-in
per-job process isolation. Version locations: VERSION, package.json,
CHANGELOG.md, openclaw.plugin.json, BOOTSTRAP_FOR_AGENTS.md stamp, and the
regenerated bootstrap template tree.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: passwordless fixture URLs in the pool-wiring tests

The pre-push credential guard (correctly) blocks any URL-with-password shape
in a pushed diff, including fake placeholders. The never-connected fixture
URLs don't need a password at construction time — drop it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: README Minions capability + KEY_FILES reserved-routing entry for v0.46.1.0

document-release sweep: the wave's docs covered the guides, TESTING, and the
new module entries but missed two spots — the README Job queue capability
paragraph (now names --job-isolation process and the probe verdicts, linking
both guides) and the KEY_FILES postgres-engine.ts entry (now carries the
withReservedConnection direct-lane routing invariants + getPoolDiagnostics
seam, pinned by test/postgres-engine-reserved-routing.test.ts). llms-full.txt
regenerated for the README edit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: cross-model doc-review precision fixes for v0.46.1.0

Codex review of the shipped docs vs the diff, each finding verified against
the code before applying:

- minions-deployment: group-SIGKILL platform caveat (Bun /bin/kill fallback),
  lock-token fencing scoped to queue writes (handler side effects bounded by
  the watchdog, not the token), connection math relabeled (pooler-lane vs
  direct session-lane split), no-per-child-RSS-cap note, GBRAIN_JOB_CHILD_CLI
  + the 3-consecutive-spawn-failure breaker documented.
- queue-operations-runbook: verdict rides the TERMINAL probe line (not every
  N/3 line), server_unreachable hedged (both-lanes-failed is the evidence),
  pooler-layer fault added to the 0-in-flight reading, jobs cancel described
  as cooperative inline vs real kill under isolation.
- KEY_FILES: run-child SIGTERM fires shutdownSignal ONLY (both only on
  parent death); third no-burn child class (ChildNotClaimedError).
- TESTING: e2e concurrency leg uses the fixture (no child DB pools); only
  the run-child leg boots real child pools.
- CHANGELOG: one wording precision fix (reserved holds leave a heartbeat
  slot, not "always keep a free slot").

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 10:06:04 -07:00
Garry TanandClaude Fable 5 56a272f18d feat(claw-test): opencode runner — detection, pinned one-shot invoke, multi-provider env allowlist
OpencodeRunner (4th AgentRunner): detectBinary('OPENCODE_BIN','opencode');
argv pinned to 'run <brief> --format default' (explicit format so an
upstream default flip cannot silently change the transcript shape; NO
--auto — MCP tool calls fire in run mode without it, verified). Env
allowlist = BASE + an EXPLICIT multi-provider delta (XAI / Google / Gemini
/ OpenRouter keys — BASE carries only Anthropic+OpenAI, and a live-lane
operator on other providers would otherwise see a misleading auth failure)
+ XDG dirs + OPENCODE_CONFIG(_DIR) + OPENCODE_DISABLE_AUTOUPDATE;
OPENCODE_CONFIG_CONTENT (the inline config-shadow channel) deliberately
absent. Bare-semver version preamble (the SST-vs-claimant discriminator)
+ global-config mcp.gbrain contamination tripwire (JSONC-tolerant, checks
BOTH merged filenames). --list-agents pin moves to all four runners with
the openclaw<opencode ordering note.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 10:01:21 -07:00
Garry TanandClaude Fable 5 40c5774fa8 feat(connect): --agent opencode — env-interpolated bearer, direct-writer --install
buildOpencodeMcpAddArgv pins the validated one-liner: the --header value
carries opencode's {env:GBRAIN_REMOTE_TOKEN} interpolation LITERALLY, so the
token never enters argv, the config file, or --json output. The print block
mirrors codexBlock (export line + one-liner + restart note). --install goes
through a new ConnectDeps.writeOpencodeRemoteEntry member (the existing
injectable seam, connect.ts:ConnectDeps) wrapping the JSONC writer in env
token mode — no opencode binary required, idempotent re-runs, foreign
same-name entries refuse with the writer's message (token-redacted), and
the D4 probe smoke-tests the credential end to end.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 09:58:16 -07:00
Garry TanandClaude Fable 5 71fb120514 feat(bootstrap): opencode harness-mode target — managed remote entry with inline bearer, remove/status/rollback
HarnessSelector gains 'opencode' (forced-wire like codex: the JSONC writer
needs no opencode CLI). Wiring is one managed mcp.<name> remote entry with
the inline Authorization bearer in the user-global opencode config, 0600,
under the [X11] lock ordering (config-dir → opencode-dir). Ownership [C8]:
idempotent re-runs match on the serve url; rotation across a url change
recognizes the old entry via the PRIOR receipt's url; anything else under
the name refuses inside the writer. Failed-smoke rollback restores the .bak
or removes a fresh entry, and the fresh mint is revoked (impostor-guard
economics hold). --remove classifies against the receipt url and skips
not-ours entries with a note; --status recovers the bearer from the entry
(url-matched — a foreign entry's credential is never transmitted). Consent
copy: per-host numbered item, joined-list reach statement (a fourth harness
can no longer silently mislabel the ternary tree), opencode off-ramp.

Fixture hygiene: the harness serial fixture now injects opencodeConfig +
detectOpencode — the default path resolution reaches the operator's REAL
~/.config/opencode (the claudeUserSettingsPath lesson, caught live when the
registrar-mode test wrote a fixture token there; cleaned up).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 09:53:45 -07:00
Garry TanandClaude Fable 5 3577432456 feat(bootstrap): opencode workspace lane — hooks --harness opencode, scope-aware direct-writer registration, channels, templates
The Harness union widening is typecheck-SILENT at every existing
'claude-code ? A : B' ternary, so the hot sites now dispatch exhaustively
(HARNESSES satisfies anchor; exec-lane bin map returns null for opencode —
its registrations go through the JSONC writer whose fingerprint IS the
[FIX7] check, never through <host> mcp get).

Scope INVERSION for opencode: default user-global (opencode spawns
project-config-defined MCP servers with NO trust prompt — verified; a
committed project entry would auto-execute on every collaborator machine).
MCP_SCOPE=project is an explicit opt-in that writes the workspace
opencode.json with a PATH-resolved command (committed-candidate file: no
absolute machine paths, no fail-open analog exists) and prints the sharing
warning + enabled:false opt-out. detectHarness probes OPENCODE/OPENCODE_PID
(observed 1.18.18). Ownership: a remote-type mcp.gbrain in the global
config makes the stdio lane step aside (codexBlockOwnsName analog); foreign
entries refuse. Verification: writer post-render parse-back is
authoritative; best-effort 'opencode mcp list --pure' probe (skipped on
plugin-bearing configs — mcp list is a code-execution surface).

Atomic with this commit (each-commit-green): questions.json MCP_SCOPE +
SURFACE_PRIMARY copy, AGENTS/GITHUB template pull-protocol generalization,
BOOTSTRAP_FOR_AGENTS.md scope guidance + opencode wiring bullet,
status.ts interview/wire resume hints, check-bootstrap-templates.sh §(e)
pins (now 'Claude Code and opencode' + the 'NO trust prompt' spawn-gate
rationale pin), guard-test fixtures, the status-test hint pin, the vendored
template-repo regen, the offline docker opencode leg, and uninstall's
receipt-keyed opencode removal. Channels: 'opencode' joins
VOLUNTEER_CHANNELS + HARNESS_CHANNELS (reserved attribution slot, codex
precedent).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 09:45:48 -07:00
Garry TanandClaude Fable 5 39687cb7fe feat(bootstrap): opencode-json managed config writer + opencode-2026-08 host spec
- src/core/bootstrap/opencode-json.ts: comment-preserving JSONC writer
  (jsonc-parser surgical edits — opencode's own mcp add preserves comments,
  the writer matches that bar). Ownership is a 4-state structural
  fingerprint (ours-same-source | ours-other-source | foreign | absent)
  keyed on GBRAIN_SOURCE EQUALITY ([FIX7] parity), never a marker key.
  Distinct read-failure classes (ENOENT create / empty-as-{} / unreadable
  refuse); foreign refusal on write AND remove; post-render validation
  (our entry round-trips, every other key survives) keeps the original on
  failure; 0600 + .bak-0600 only for inline-bearer entries; bearer
  recovery helper for harness --status.
- host-specs.ts: TARGETS['opencode-2026-08'] (verified 2026-08-15 against
  a hermetic opencode-ai@1.18.18) + opencodeConfigDir/GlobalConfigPath/
  ProjectConfigPath (XDG-only — OPENCODE_CONFIG* observed INERT in
  1.18.18, honoring them would be a silent no-op install) +
  OPENCODE_HAS_HOOKS=false.
- atomic-write.ts: rule-of-three extraction of the symlink-resolving,
  mode-inheriting atomic writer; codex-toml.ts + hooks.ts ported onto it
  (behavior pinned by their existing suites).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 09:31:45 -07:00
Garry TanandClaude Fable 5 6d25dd0711 docs(mcp): pin observed opencode CLI behavior (OPENCODE-CLI-PIN.md) + registration guide
Phase-0 hermetic observation of opencode-ai@1.18.18 (npm wrapper + platform
payload integrities pinned). Load-bearing observations: keyless anonymous
free tier answers headless runs AND drives MCP tool calls without --auto
(nonce SMOKE proven end-to-end against a real gbrain serve --surface verbs);
mcp list is the honest discriminator (spawns servers, exit 0 regardless —
parse the text); mcp add takes '-- command' (undocumented in --help) but
always writes user-global opencode.jsonc; project-defined local servers
spawn with NO trust gate (drives the user-global bootstrap default); JSONC
parses in .json-named files and both filenames merge; OPENCODE_CONFIG* env
vars observed inert (docs-contradiction, called out); OPENCODE=1 set in bash
children (detectHarness probe); AGENTS.md loads, CLAUDE.md not double-loaded.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 09:25:00 -07:00
Garry TanandClaude Fable 5 4922905fb9 v0.46.0.0 feat(transcripts): cross-harness session import — ingest, status, six format adapters (cathedral 4) (#4130)
* feat(transcripts): adapter seam — session contract, detection registry, claude lane with timestamps

Cathedral-4 commit 1: the TranscriptAdapter seam at src/core/transcripts/.
types.ts carries the session-granular AsyncGenerator contract (return value =
per-file diagnostics so a zero-yield file explains itself), format-specific
byte caps, and the ONE buildTranscriptSlug helper (per-provider dirs, id8
collision suffix). detect.ts owns the adapter registry, head-sample sniffing
(explicit format wins, symlinks lstat-rejected), and the injectable
HARNESS_ROOTS discovery surface. claude-code.ts wraps the SHIPPED parser;
claude-code-jsonl.ts gains the ADDITIVE parseClaudeSessionFile (full-file,
reject-over-cap, real per-message timestamps) — hook-lane parseTranscript
output is pinned byte-identical by the new regression test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(transcripts): codex, openclaw, and hermes adapters — verified shapes, drift alarms, copy-then-read

Cathedral-4 commit 2. codex.ts: turn selection is STRUCTURAL — user turns
from event_msg user_message, assistant turns from response_item output_text;
response_item user/developer rows are injected preambles and never leak
(fixture-pinned). openclaw.ts: session header + message lines, real
timestamps, model_change/custom/compaction skipped, .checkpoint.*.jsonl
siblings rejected at detect. hermes.ts: copy-then-read (DB + wal/shm
sidecars to a temp dir) because readonly WAL opens need -shm write access
and lock against a live writer; schema verified against the installed
hermes-agent v0.20.0 SCHEMA_SQL, SPEC_TARGET provisional, multi-session
cardinality with tool-only sessions skipped. Detection matrix pins all four
formats. Codex + OpenClaw shapes verified against live local files
2026-08-14.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(transcripts): chatgpt + claude.ai export adapters — mapping-tree walk, extracted-JSON v1

Cathedral-4 commit 3 (CP1). chatgpt-export.ts walks the mapping TREE via
current_node parent pointers (regenerated branches dropped by design;
orphaned parents terminate quietly; latest-leaf fallback when current_node
is absent) — the branched/orphaned/fallback cases are fixture-pinned.
claude-export.ts is the flat sibling (human maps to user, empty rows
skipped). Both take the EXTRACTED conversations.json only (unzip-first
errors; zip wrapper is a filed TODO), reject-not-truncate over the export
cap, and carry provisional SPEC_TARGETs pending a fresh real export sample.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(transcripts): render pipeline — shared anchor pattern, anchor-escape, fail-closed redaction, part splitting

Cathedral-4 commit 4. render.ts renders sessions in the conversation-parser
imessage-slack builtin (regex IMPORTED, never re-declared — round-trip
pinned through parseConversation), with real UTC timestamps (missing ones
carry forward, zero-timestamp sessions REFUSED — provenance is never
fabricated). Anchor-shaped BODY lines are backslash-escaped so hostile
message content cannot forge speakers or timestamps on re-parse (P0).
Redaction is fail-closed for the page lane: secret-scan + user pattern file
(harvest-private-patterns convention; the slack-channel default is excluded
because it eats issue refs) + agent-imperative COUNTING stamped into
hash-covered transcript_import frontmatter (never content_flag). Long
sessions split at message boundaries (~300KB parts, 2-message overlap)
under the embed_skip threshold; part 1 keeps the base slug, ids are unique
per part.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(facts): batch slugs selector + the transcripts-ingest facts lane

Cathedral-4 commit 5. runExtractConversationFactsCore gains a slugs[] batch
selector (serial, same per-page advisory lock + durable-outcome gates as
enumeration) so a caller with a known page set invokes the core ONCE —
per-slug invocations multiply config resolution, checkpoint IO, and receipt
writes by page count. ingest-facts.ts wraps that single invocation in ONE
withBudgetTracker (opts.budgetTracker alone is not accounting — the gateway
reads AsyncLocalStorage) and pre-checks facts.extraction_enabled with a
notice instead of the core's throw.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(transcripts): gbrain transcripts ingest — session-atomic import CLI, embed-OFF default, clean-scan watermark

Cathedral-4 commit 6. ingest.ts is the engine-facing core: detect → parse
(per-session) → since/limit filters → fail-closed redaction → render/split →
importFromContent per part (noEmbed unless the embed flag opts in) →
putRawData → stale-part reconciliation (deletes part>of leftovers).
Atomicity is the SESSION: failed sessions count and skip, integrity
failures (duplicate-lookup, read-back, raw-data miss) abort the whole run.
The command layer resolves ONE source id through the 6-tier chain, threads
activePack once, streams progress (phase transcripts.ingest, stderr), and
advances the since-last op-checkpoint watermark ONLY after a clean,
untruncated, non-dry scan (fingerprint binds source + pathspec + format +
adapter version). transcripts joins CLI_ONLY_SELF_HELP and
SELF_HELP_WITHOUT_ENGINE (engine-free help); flag registry regenerated.
Facts flag targets every touched slug including hash-skipped pages.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(transcripts): discovery mode, --all, and the status gap table

Cathedral-4 commit 7 (CP0 + CP2). No-arg ingest runs confined discovery
over the harness roots and shows what WOULD be imported (safe default);
the all flag imports the discovered set. The status subcommand derives its
imported side from ONE paginated pages walk (client-side transcript_import
filtering, distinct session ids) — durable truth that catches late-arriving
sessions no watermark can — and matches JSONL files by
session-id-in-basename; the hermes store reports at session granularity.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(transcripts): e2e PGLite suite + putRawData zero-row parity fix

Cathedral-4 commit 8. The e2e suite (R3/R4: engine in beforeAll, disconnect
in afterAll) pins: cross-harness round-trip (codex + openclaw into one
source, frontmatter + raw-data assertions), dry-run zero-writes, idempotent
re-runs with hash-skipped slugs still visible to the facts lane,
redaction-before-write, part splitting under the embed-skip threshold with
unique per-part ids, the dangerous split-then-shrink transition (stale
higher parts deleted), since/limit clean-scan semantics (limit truncation
freezes the watermark; the follow-up run converges), per-file error
taxonomy, and the drift signal.

PGLite putRawData now RETURNING-checks and throws on a missing page,
matching the Postgres engine — the run-level integrity abort was previously
false on the e2e backend (eng outside-voice finding 17).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(eval): write-back fidelity through the adapter path (in-repo pin)

Cathedral-4 commit 9. The BrainBench write-back suite renders normalized
turns directly and never exercises raw parsing/detection/redaction/import —
this deterministic e2e closes the bypass in-repo: raw codex + openclaw
fixture FILES enter via runTranscriptsIngest, the shipped extractor core
runs with the injected gold extractor (decision-15 seam, zero LLM), and the
planted facts are probed with provenance pointing at imported conversation
pages. Cross-harness continuity pinned: one source holds facts grounded in
both harnesses' sessions. Re-extraction dedup pinned via the
durable-outcome gate. The full BrainBench raw-fixture sidecar schema (+
corpus-hash coverage + baseline re-cut) lives in the sibling gbrain-evals
repo and is filed as a follow-up TODO.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(transcripts): conversation-archive native-importer update, KEY_FILES seam entry, progress phase, 8 follow-up TODOs

Cathedral-4 commit 10. conversation-archive now points at the native
importer for the six covered formats and states the native-vs-manual PII
delta (secrets + user patterns native; broad PII detection stays the human
pass — filed as a TODO). check-fixture-privacy scans the new
test/fixtures/transcripts dir with the same banned-token contract.
KEY_FILES gains the src/core/transcripts/ seam entry and the updated
transcripts-command entry; progress-events documents the transcripts.ingest
phase. TODOS: 8 follow-ups (OpenClaw/Codex go-forward capture, scheduled
re-import consent design, PII pass, more adapters, zip unwrapping,
BrainBench raw-fixture schema in the sibling repo, hermes verification) +
the TODOS flip-contract-adapters entry notes the codex parser unblock.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(transcripts): review-army + red-team + cross-model fixes — identity hashing, watermark safety, redacted raw, healing re-runs

Cathedral-4 commit 11: 30+ findings from 5 specialists, a red team, and two
Codex passes (adversarial pass REPRODUCED the identity P0 against PGLite),
all folded.

Identity (P0): slug + dedup ids are now sha256 hashes (12-hex slug, 16-hex
harness-namespaced frontmatter id) — prefix identity let same-prefix session
ids silently overwrite a same-day page or dedup-skip a different-day one,
and every export fallback id collided.

Watermark safety: drift files, malformed lines, and page-import error
statuses all freeze the clean-scan watermark; unparseable timestamps are
skipped (never admitted to the compare); explicit --since values are
validated + Z-normalized and never advance the watermark (only full-coverage
runs attest); the --all fingerprint binds the resolved user-stated spec, not
the expanded file list; --limit counts NEW WORK only (hash-skipped re-scans
are free, so batched backfill converges instead of looping the imported
prefix).

Redaction: putRawData persists the REDACTED metadata copy (was the original
— the redacted copy was built and discarded); raw flatness is enforced
(nested values dropped); speaker labels are cleaned + anchor-stripped;
patterns compile once per run.

Healing re-runs: all-skipped sessions verify-and-heal raw_data instead of
assuming it; stale-part reconciliation is SQL-enumerated (walks past crash
holes) and runs on every pass. hermes.ts is text again (escaped NUL); the
sidecar-inclusive byte cap bounds the copy; codex detect is structural
(JSON.parse, not substring); claude-export detect gets the symmetric
mapping guard; directory expansion filters to importable extensions;
per-session heartbeats cover multi-session stores; status reads ONE
frontmatter-only query; empty slugs selector is a no-op, never full-corpus
enumeration; export-loader deduplicated (export-json.ts).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: bump version and changelog (v0.46.0.0)

Cathedral 4 takes the MINOR per lineage (0.43/0.44/0.45 were cathedrals 1-3).
All six version locations move together: VERSION, package.json, CHANGELOG,
openclaw.plugin.json, the bootstrap runbook stamp, and the regenerated
template tree + llms bundles + lockfile.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(transcripts): verification-pass residuals — raw refresh on skipped re-runs, resolved-slug follow, scoped all-lane watermark, content-derived fallback ids

Cathedral-4 commit 13: the Codex verification pass confirmed the review-wave
fixes hold and found four residuals in the new code, all folded. Skipped
re-runs now COMPARE the stored raw-data row instead of assuming existence
means freshness (a private pattern added after first import refreshes the
stored copy; healthy re-runs stay write-free). Raw-data writes and stale-part
reconciliation follow the slug importFromContent actually RESOLVED (identity
dedup can land part 1 on an existing page under a different slug — the old
code aborted every re-run on the nonexistent rendered slug). The all-lane
watermark fingerprint carries host + harness roots (DB-backed checkpoints are
shared across machines on one brain; a bare literal let machine B inherit
machine A's watermark). Export fallback session ids are content-derived,
never a bare per-file ordinal (two files' first id-less conversations
collided).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(transcripts): build the planted secret token at runtime

The redaction tests plant an AWS-shaped token to assert it never reaches a
page; as a committed literal it (correctly) trips the pre-push credential
guard, which scans the diff with the same pattern the runtime scanner uses.
Constructing it at test runtime keeps the regression coverage and keeps the
committed bytes credential-free.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: update project documentation for v0.46.0.0

document-release pass over the cathedral-4 transcripts-import ship, verified
against the final diff (three code commits landed after the branch's docs
commit) plus a cross-model doc review:

- README: transcripts importer added to "How to get data in" (discovery /
  all / status examples), with the redaction claim scoped to what the code
  scrubs (bodies, titles, speakers, session metadata)
- KEY_FILES: current-state corrections — sha256 hash12/hash16 ids (stale
  id8 claim), host-scoped all-lane watermark fingerprint, shared
  export-json.ts loader + content-derived fallback ids, healed redacted
  raw metadata on skipped re-runs, status = one executeRaw frontmatter
  query, JSONL cap clarified (50MB import; 10MB is the hook tail reader)
- CHANGELOG (wording only): tool/thinking claim made precise (one-line
  placeholders do land), facts backfill gated on the cycle phase being
  enabled, format flag added to the flag list
- progress-events: per-session heartbeats documented alongside per-file
  ticks
- conversation-archive skill: ~4K per-message body cap + placeholder
  delta disclosed; IMPORT half covers both native and manual paths
- TODOS: "Native AI-chat export importer" marked Completed v0.46.0.0;
  Perplexity cross-reference fixed
- cli.ts: top-level help now advertises the transcripts family, not just
  recent (no dashed flags; registry regen = no diff)
- llms-full.txt + skills.lock.json regenerated

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(transcripts): full-pipeline e2e for all six formats

Closes the coverage gap the ship left: codex and openclaw were the only
formats traveling parse -> redact -> render -> import -> page in e2e; the
other four stopped at adapter-level unit tests. Now every format lands as
real pages against PGLite: claude-code (placeholders + real anchor
timestamps from the shipped fixture), hermes (ONE store file -> MANY pages —
the multi-session ingest path, per-session raw_data, plus limit-truncation
convergence on a multi-session file), chatgpt export (per-thread pages under
the chatgpt directory with title slugs; abandoned branches never land), and
claude.ai export (title-slugged pages under the claude directory). Titles
are asserted on the page column, where import promotes them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 08:07:35 -07:00
Garry TanandClaude Fable 5 3ebda1fc87 v0.45.20.0 feat(claw-test): grok e2e install harness — runner, door, CI job, TTY scenario (#4137)
* docs(mcp): pin observed grok CLI behavior (GROK-CLI-PIN.md)

Phase-0 observation transcript against a real Grok Build v1.0.4 install
(pinned npm @xai-official/grok). Keyless scope complete: GROK_HOME seam,
lazy exit-0 mcp add, honest mcp doctor discriminator (7 verbs discovered),
saved TOML schema verbatim, trust-gated vendor fallback, volatile-path
inventory for the tripwire, keyless auth error. Paid probes marked
pending auth per plan D0. Machine-stable stamp block feeds
scripts/check-grok-pin.sh.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(claw-test): extract detectBinary/filterAllowlistEnv into agent-runner

Byte-identical detect()/env-filter bodies moved out of runners/hermes.ts and
runners/openclaw.ts (rule-of-three: the grok runner lands next). Behavior-
preserving: same reason strings, same ordering; existing runner tests green.
Adds direct unit pins for override precedence (PATH-shim depends on it), the
leak barrier, and the non-executable stat branch. openclaw.ts's prompt-file
comment respelled dash-free (flag-registry prose-bleed class).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(claw-test): grok runner — detection, pinned one-shot invoke, env allowlist

GrokRunner (xAI Grok Build CLI) as the third registered agent. Pinned argv
from docs/mcp/GROK-CLI-PIN.md observations: single-shot flag + plain output
format; permission flags deliberately absent pending the authed observation.
Env delta: GROK_HOME + XAI_API_KEY. Version preamble recorded as a stdout
transcript event (mis-bound community binary diagnosable from transcript);
loud warning when the operator's ~/.claude.json registers gbrain (the
trust-gated vendor-config contamination channel). Tests: detection contract
incl. the first through-runner shell-metacharacter pin, shim argv/env leak
barrier, three-way alphabetical list-agents pin. Flag registry regenerated
(argv-literal bleed is accepted over-inclusion; no SAFETY_FLAGS collision).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(e2e): grok door — split-gated real-binary e2e + harness helper family

install-real-grok.serial.test.ts: keyless tier (version-shape pin, documented
registration via a PATH-staged bin dir, saved-TOML asserts through
Bun.TOML.parse, mcp doctor handshake proving the seven-verb surface keyless,
vendor-fallback provenance guard, direct-TOML surface + config-preservation
pin) gated on opt-in + binary only; paid SMOKE additionally on XAI_API_KEY,
asserting a per-run nonce fact with web search disabled. mcp add is lazy
(exit-0-always, observed) — doctor is the honest discriminator. Bounded
tripwire over the operator's real ~/.grok config/credential files (volatile
paths excluded) + a checkout guard. Helpers: resolveGrokBinary (GROK_BIN
override honored), hasGrokAuth, grokChildEnv (explicit key re-admission +
GITHUB_ENV/PATH/OUTPUT/STATE scrub), seedGrokConfig (auto_update kill-switch),
stageGbrainBinDir (compiled copy, bun-run wrapper fallback), grokOneShotTurn;
seedBrainForAgent gains a nonce-fact override (hermes path unchanged).
run-e2e.sh scrubs GROK_* so the door structurally cannot fire under test:e2e.
Verified live: keyless tier 4 pass / 0 fail in 28.9s against Grok Build v1.0.4.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci(heavy): grok-door job (keyless-first, npm-pinned) + check-grok-pin guard

grok-door provisions the pinned npm package (registry integrity pre-checked
against the GROK-CLI-PIN.md stamp — a re-published version becomes a loud
re-pin decision) in a secretless step, runs the KEYLESS door tier before the
secret precondition (missing XAI_API_KEY fails loudly but only after the free
compat coverage is banked), then a named bad-key preflight, the full paid run
with a paid-sentinel (a skipping paid tier can never read green while the key
is present), and a mid-job version-drift tripwire. Pre-secret gating posture:
real-agent-e2e label or the run_grok_door dispatch input ONLY — no schedule,
no generic heavy-tests label — so an absent secret cannot paint nightly runs
red; the secret-enable follow-up re-adds schedule + heavy-tests + a
latest-version canary leg. real-agent-e2e job gains the grok door file +
opt-in var. Backports to hermes-door in the same commit: unconditional
door.txt evidence copy (the zero-pass failure class now leaves a trace) and
persist-credentials: false on checkouts. New scripts/check-grok-pin.sh
(distribution_kind-aware, grok-door-block-anchored, SKIP-graceful) wired into
verify + check:all, with guard tests covering ok/skip/drift/exclusivity/dupes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(mcp): Grok Build install guide + surface wiring

docs/mcp/GROK.md (HERMES.md skeleton): surface-verbs register form matching
current CLAUDE_CODE.md guidance, direct-TOML block with the startup-timeout
gotcha, the trust-gated vendor-config fallback with honest precedence and the
doctor source field, doctor-as-the-real-probe verify (7 tools discovered),
headless auth + model pin + auto-update seed, cron pairing, troubleshooting
incl. wrong-grok-on-PATH (community CLI collision), grok/groq/ngrok
disambiguation, and the skills-placement note. Honest classification
everywhere: brain-only install; bootstrap does not support Grok yet.
Version-bounded phrasing on all three user surfaces (README bullet,
INSTALL_FOR_AGENTS block, GROK.md footer). MEMORY_VERBS registration
one-liner added. llms bundles regenerated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(dx): structural transcript redaction + PTY hot-loop bounds + settle audit trail

Behavior wave preceding the runInstallSession extraction (kept separate so
the extraction stays pure motion): saveTranscript gains an explicit redact
seam applied to EVERY artifact as one pass over the serialized string (a
secret split across frame boundaries can't survive frames.jsonl); dx-explore
builds the redaction map from PROVIDER_KEY_NAMES, redacts the live screen
mirror at every tick (the mirror outlives interrupted runs), redacts
events.jsonl, and hard-fails via an independent post-save grep that deletes
any leaking file (structural redaction is primary, the grep is the check).
mirrorSession strips a bounded raw tail instead of the full buffer
(quadratic on 25-minute sessions); waitForAny matches only output after the
paste (the pasted prompt contains verify-adjacent copy); settle notes a
quiet-but-dialog-shaped tail (note-only) and stops early at grok's observed
sign-in copy; main() guards ptySupported() so an unsupported Bun fails in
1s, not 25 minutes. Unit tests: redact bundle + purity/short-value skip.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(dx): extract the shared install-session tail (runInstallSession + stageBinDir)

The claude/codex install scenarios were ~65-line copy-paste twins; the grok
scenario would have been the third. The duplicated tail (launch → mirror →
settle → paste → race verify-copy vs exit → trailing quiet → save) moves into
runInstallSession(ctx, {argv, cwd, env, extraAllow, dropEnv, prompt, timeoutMs,
meta}); per-agent preparation (claude TUI seed, codex auth copy + git init)
stays bespoke in each scenario. Carries two behavior deltas that belong to the
preceding fix commit and are stated here honestly: the twins' mirror now
passes the redaction map, and the verify race is scoped to output after the
paste (since-mark).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* dx: grok-install TTY scenario — brain-only GROK.md prompt, sign-in-wall early-stop

scenarioGrokInstall drives REAL interactive grok through the GROK.md
brain-only install (deliberately NOT the bootstrap paste block — the docs
classify grok as brain-only) with its own success patterns (the doctor
handshake banner). Keyless posture verified live: intro animation → sign-in
screen at ~6s ('Approve in your browser to finish signing in' + device code,
copy pinned in GROK-CLI-PIN.md) → skip-splash Enter → early-stop at 16s with
the friction recorded and the full transcript bundle written, instead of
pasting into the sign-in wall for the 25-minute race. Hardening found by the
same run: settle + early-stop strip bounded raw tails (the post-paste spinner
made full-buffer ANSI stripping the hot loop); the textless-splash heuristic
counts 3-plus-letter word runs (the animation is U+2800 braille — glyph
enumeration misses it); XAI_API_KEY joins PROVIDER_KEY_NAMES so --keyless is
honest and the redaction map covers it; BROWSER kill-switch so a keyless run
never bounces the operator's browser; grok credential path pre-registered for
the scrub.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: conform detectBinary/filterAllowlistEnv pins to isolation rule R1 (withEnv)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(todos): file grok-wave follow-ups (secret-enable lane, connect, backports, registry unification)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: review-army wave — P1 paid-turn PATH seam + security/redaction/CI hardening

Findings from the ship pre-landing review (5 specialists + Claude adversarial
+ 2 Codex passes), all applied:

P1 (Codex structured review): grokOneShotTurn rebuilt its child env without
the staged bin dir, so the documented bare-gbrain MCP registration that
doctor had just validated could not resolve during the actual paid turn on a
clean runner — grokChildEnv gains a binDir PATH-prepend and the door threads
it through the SMOKE.

Security: the version preamble now runs execFileSync with the FILTERED env
(it ran a shell one-liner with the full parent env — ambient secrets exposed
to a possibly mis-bound binary, plus a quoting seam on which-resolved paths);
detectBinary's which goes through execFileSync (the extraction had introduced
shell interpolation of binName); the grok-door auth-preflight scrubs the
writable GITHUB_* step files and disables web search; grokChildEnv also
deletes GITHUB_STEP_SUMMARY/GITHUB_ACTION_PATH; the live-lane runner no
longer forwards ANTHROPIC/OPENAI keys to grok (foreign-provider filter, with
shim-test barrier); stageGbrainBinDir rejects shell-active repo paths;
per-platform npm payload integrities pinned (wrapper integrity covers only
the wrapper tarball) with the version-immutability assumption stated; docs
quote the env flag value for spaced homes.

Redaction: a secret straddling PTY frame records survived frames.jsonl as
joinable halves (each frame is its own JSON record — the contiguous value
never existed in the serialized string, and the covering test passed
vacuously; verified empirically in review). saveTranscript now coalesces
straddling frames before redaction, the test asserts on the JOINED data
stream, assertNoSecrets gains stripped-ANSI + joined-jsonl passes and never
deletes files that predate the run (a --dir at repo root could have deleted
a pre-existing .env), and the init/drive scenarios pass the redact map to
the live mirror.

Correctness/perf: resolveGrokBinary fails CLOSED on an invalid GROK_BIN
(fall-through could bind the colliding community binary despite the pin);
--keyless now actually drops provider keys in all three install scenarios;
waitForAny strips a bounded window (the verify race had re-introduced the
quadratic full-slice strip); raw-tail windows widened 32K→128K for SGR-dense
repaints; grok-door compiles gbrain ONCE via a GBRAIN_COMPILED_BIN
short-circuit (two bun test processes each paid the compile); bounded stream
drain + SIGKILL escalation on turn timeout; CI cleanup also removes the
preflight home and tmp door homes; the pin guard fails closed once the door
job exists, strips single-quoted env values, and asserts npm_version ==
grok_version; sign-in-wall early path defers cleanup to finally.

Tests: vendor-tripwire fire/silent pins, preamble-failure resilience,
fail-closed GROK_BIN truth table, coalesce unit + joined-frames assertion,
SAFETY_FLAGS collision guard, awaited withEnv call sites, foreign-key
barrier, guard fail-closed/quote/equality cases. All affected suites green;
keyless door 4-pass live re-verified; verify 44/44.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: bump version and changelog (v0.45.17.0)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: update project documentation for v0.45.17.0

docs/INSTALL.md: per-client MCP guide list now covers every client guide
in docs/mcp/ (adds HERMES.md, OPENCLAW.md, CLAUDE_COWORK.md alongside the
new GROK.md line — README's client roster and this list now agree).
docs/guides/bootstrap.md: tty-harness example CLI list includes grok,
which the grok-install DX scenario now drives.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 07:43:54 -07:00
Garry TanandClaude Fable 5 1a06fc100a v0.45.19.0 chore(test): consolidate the PTY layer — delete dead harness, real-PTY init picker test (#4139)
* chore(test): delete dead cli-pty-runner PTY harness + self-test (T1)

launchPty had zero callers since v0.25.1; its documented consumer
(test/e2e/skill-smoke-openclaw.test.ts) was never written. The numbered-menu
parsers match Claude-style cursor menus, not gbrain's typed-number picker,
so nothing is folded forward. git history preserves the file.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(init): real-PTY serial test for the interactive init pickers (T2)

Drives bun run src/cli.ts init under a true pseudo-terminal via launchTty:
keyless provider choice, then a NON-default search mode (tokenmax vs the
keyless-env conservative recommendation) so the assertions cannot pass via
the pickers' 60s default fallback. Hermetic HOME+GBRAIN_HOME temp root,
Anthropic key dropped, prompt-liveness bounds, close() in finally, CI
fail-loud PTY guard. Serial lane so it runs in required CI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: fix three false coverage comments pointing at deleted or wrong harnesses (T3)

Two files deferred interactive-picker coverage to each other in a circle via
the deleted cli-pty-runner; a third called a piped-stdin e2e file PTY-based.
All three now point at the real coverage: test/init-picker-pty.serial.test.ts
for TTY branches, init-fresh-pglite for non-TTY branches.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: truth-up TTY-testing docs after harness deletion (T4)

KEY_FILES entry now describes the surviving tty-harness + dx-explore layer
(deleted runner's entry removed); TESTING.md gains the four-tier TTY-testing
decision table incl. the serial-lane CI rule and the non-default-value
assertion rule; tty-harness header stops calling its unit suite
zero-subprocess (the live block spawns sh).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(test): drop stale shard-weight entry for deleted cli-pty-runner test (T5)

No re-mine: pre-deletion CI logs would resurrect the deleted key, and
neither serial nor e2e files receive sharded weights. tty-harness.test.ts
keeps the median fallback until the next routine mining run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(todos): PTY transcript capture via Bun terminal option; file e2e CI-lane gap (T6)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(todos): e2e CI-lane gap entry — match current e2e.yml named-file list

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: pre-landing review fixes — EOF picker case, dropEnv hardening, wording truth-ups

Testing specialist: add the Ctrl-D EOF case at the provider prompt (the shared
readLineSafe branch the deleted harness's comments falsely claimed to cover).
Maintainability: drop ANTHROPIC_AUTH_TOKEN alongside the API key; derive the
liveness bound from a named READLINE_FALLBACK_MS; reword three named-file CI
claims to the glob-free phrasing; un-future the tty-harness consumer list.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: adversarial-review fixes — ASCII-safe PTY menu match, live TESTING.md exemplar

Red team: a PTY chunk boundary inside the menu line's multibyte em-dash would
permanently corrupt the match buffer (per-chunk utf-8 decode); match the pure
ASCII prefix instead. Adversarial: TESTING.md's piped-stdin row cited a file
that runs in no CI lane; cite the fast-loop example and annotate the manual one.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: codex adversarial fixes — fixed 20s liveness bound; file the EOF 60s init stall

The liveness budget is now a fixed interaction bound independent of the
fallback constant, so dead input cannot pass even if the production fallback
shortens. The confirmed post-EOF stall (mode picker burns its full 60s after
Ctrl-D because stdin never yields another line) is a pre-existing product
bug — filed in TODOS with the probe numbers; the EOF test's early close is
now documented as deliberate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: bump version and changelog (v0.45.19.0)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: update project documentation for v0.45.19.0

- docs/TESTING.md: add per-file inventory entries for the new real-PTY
  init-picker serial test and the tty-harness pure-helper suite
- docs/guides/bootstrap.md: note the tty-harness now also backs a
  required-CI test (init pickers), while the DX layer stays an instrument

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 07:22:29 -07:00
MasaandClaude Fable 5 180c648ba0 docs(ai-providers): document the claude-cli recipe (#4113)
Neither README.md nor docs/ mentions the claude-cli recipe (shipped in
v0.42.66.0, #3310) — the only description of how it routes gateway.chat()/
toolLoop() through the local `claude` CLI, what it strips from the
subprocess env, and how gbrain models doctor's fixed 5s probe timeout
interacts with a cold subprocess start lived in source comments. Adds
docs/ai-providers/claude-cli.md following the existing zeroentropy.md /
llama-server-reranker.md format. Documentation of existing shipped
behavior only — no README/CLAUDE.md/code changes.


Claude-Session: https://claude.ai/code/session_01SMJA4RCTXLgsXjPM4o1qcP

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 04:54:18 -07:00
Masa 181fc8c1a2 docs(cli): tell contributors how to resolve a flag-registry conflict (#4138)
`src/core/cli-flag-registry.generated.ts` is a committed generated artifact
that upstream regenerates on most waves, so any branch that also regenerates
it conflicts on the whole body — repeatedly, since rebasing only resets the
clock until the next wave.

The header already says how to regenerate but not what to do when the file
conflicts, which is the moment a contributor is actually looking at it. Adds
that: take the base branch's copy wholesale, re-run the generator, and let the
freshness test catch a regeneration done against the wrong base.

Comment-only. No flag entries change; the regenerated artifact differs from
its committed form solely by the new header lines.
2026-08-15 04:31:47 -07:00
Garry TanandClaude Fable 5 4deee227be v0.45.18.0 fix(serve-http,pglite): UTC-instant spend day boundary + snapshot timezone parity (#4131)
* fix(serve-http,pglite): UTC-instant spend day boundary + snapshot timezone parity

The admin spend query compared created_at against a NAIVE date_trunc result,
reinterpreted in each session's timezone — any non-UTC session shifted the day
boundary by its offset and underreported today's spend every evening. The
boundary is now a timestamptz instant (double AT TIME ZONE), pinned by a
session-timezone-adversarial regression test (Etc/GMT+12 / Etc/GMT-12 / UTC)
that is red on the old query at any wall-clock hour.

Root cause of the local-red/CI-green suite: dumpDataDir bakes the BUILD
process's TimeZone into the snapshot tar, so snapshot-restored engines ran
sessions in the build machine's zone while cold-init engines follow the
runtime (bun test pins TZ=UTC). Restored engines now re-pin the session to
the runtime zone (heals existing tarballs with no rebuild), the builder pins
TZ=UTC before any PGLite work, and a serial parity test asserts cold and
snapshot engines agree on their session UTC offset.

* chore: bump version and changelog (v0.45.18.0)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 21:59:24 -07:00
Garry TanandClaude Fable 5 52140808fd v0.45.17.0 fix(test): refuse test runs with ambient database URLs — four-layer #3485 guard (re-land of #4126) (#4128)
* fix(test): name-floor guard for every destructive-SQL test + repo-wide coverage gate (#3485)

Moves assertSafeE2eDatabaseUrl to leaf module test/helpers/db-guard.ts
(re-exported from test/e2e/helpers.ts for existing call sites) and calls it
before connect() in all ten files that run destructive SQL against the
ambient URL — the eight from #3485, one newer offender
(bootstrap-keyed-postgres.serial), and the raw-postgres()-client OAuth suite
the original audit could not see.

test/db-guard-coverage.test.ts is the static gate that keeps the class
closed: walks every test file bun collects repo-wide (all naming patterns,
fixtures included), detects ambient-URL reads at the assignment site (any
binding name, both env vars, bracket notation), recognizes four connect
idioms, treats env-var deletes as scrubs not reads, refuses comment-only
guard mentions, and pins its own classifiers with positive controls so it
can never pass vacuously.

Patch for the ten files adopted from #3485 by @cheRoma (fork access blocked
a PR) — thank you.

* feat(test): refuse to start a test run while a database URL is ambient (#3485)

A bunfig [test] preload (registered first) hard-fails any bun test invocation
while DATABASE_URL or GBRAIN_DATABASE_URL is set, unless
GBRAIN_TEST_ALLOW_DATABASE_URL=1 — refusing with instructions, never silently
unsetting (a silent unset would turn DB-gated e2e tests into green skips).

Boundaries: run-e2e.sh and the e2e/heavy workflows opt in at their own
subprocess boundary (run-e2e.sh also keeps the opt-in vars past its hermetic
GBRAIN_* scrub and drops GBRAIN_DATABASE_URL, which has no name floor on
spawned-CLI paths); the unit/slow wrappers strip both vars instead — unit
tests need no database — which keeps `bun run test:full` with a DB URL
exported reaching its e2e leg. The phantom-redirect parity file rides the
e2e lane and CI's jsonb-parity job so its Postgres arm stays reachable.

Six subprocess tests spawn real bun test children against the actual
bunfig registration: refuses each var, refuses both, strict override value,
override allows, empty-string treated as unset, clean run.

* fix(tests-heavy): shared database name floor for the heavy shell lane (#3485)

The heavy lane runs schema drops, source-registry rewrites, migration
replays, and parallel syncs against whatever the environment names — outside
bun, where the preload guard cannot fire. tests/heavy/_db_floor.sh mirrors
test/helpers/db-guard.ts: sourced by run-heavy.sh and by every script
documented for direct invocation, it floors BOTH DATABASE_URL and
GBRAIN_DATABASE_URL (the CLI these scripts shell out to prefers the latter)
and strips query strings before extracting the name, so a
?host=/tmp/test-sockets parameter cannot smuggle a test-shaped segment past
the check.

* chore: bump version and changelog (v0.45.15.0)

TESTING.md documents the four guard layers and the cwd caveat; TODOS.md
files the disclosure-policy follow-up (P2).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: update project documentation for v0.45.15.0

Cross-reference pass after the #3485 test-safety wave (Wave -1):

- docs/TESTING.md: complete the guard layers (heavy shell floor
  tests/heavy/_db_floor.sh, schema-drift's accepted inline floor), note
  the phantom-redirect Postgres arm riding the e2e lane in the file
  taxonomy + E2E inventory.
- docs/architecture/KEY_FILES.md: scripts/run-e2e.sh entry updated to
  current behavior (no-args list carries phantom-redirect parity; #3485
  opt-in boundary, GBRAIN_DATABASE_URL drop, GBRAIN_E2E_ALLOW_DB
  preserved through the env scrub).
- CONTRIBUTING.md: heads-up that bare `bun test` refuses to start with a
  database URL ambient + the name floor for own-Postgres/Supabase e2e.
- tests/heavy/README.md: database name floor section (which scripts
  source it, PGLite scripts unset instead, new-script rule).
- .env.testing.example: Supabase's default "postgres" database name
  fails the floor — dedicated test DB or one-shot GBRAIN_E2E_ALLOW_DB.
- CHANGELOG.md v0.45.15.0: three accuracy-of-wording touches (headline
  "silently", lane boundary phrasing, note the one accepted inline
  floor) — no entries removed or regenerated.

Codex cross-model doc review ran; concrete gaps applied above. llms
bundles regenerated (no byte changes — touched docs are link-only).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(test): keep the coverage gate's own scrub-pattern out of the R1 isolation lint's sight

The gate detects 'delete process.env.X' as a scrub-not-read; the R1 lint greps
the same token textually and flagged the gate's comment and classifier fixture
as env mutations. Comment reworded; fixture built by concatenation so the
classifier still receives the contiguous statement.

* chore: re-slot as v0.45.17.0 (re-land of reverted #4126)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 21:44:07 -07:00
Garry TanandClaude Fable 5 83a4a94c38 v0.45.16.0 fix: W0 verified-bug hotfix wave — cycle-lock fencing, queue reaper integrity, 10x test suite (code-smell series) (#4127)
* fix(cycle,locks): fenced lock identity + steal-abort — the cycle lock is actually refreshed in production (W0 Tier-1 #1)

The 2026-08-14 audit (CONFIRMED by adversarial verification) found the cycle
DB lock was effectively never refreshed: lock.refresh() was reachable only
through buildYieldDuringPhase, three of five pass sites handed phases the raw
caller hook, and NO production caller (jobs.ts, autopilot.ts) sets
yieldDuringPhase at all — so with the 5-minute TTL against 35-minute subagent
waits, every long cycle lost its lock mid-run and a second cycle could start
against the same source.

Fixes, per the fix-wave plan (D5.10/D5.11/D5.6):

- db-lock: refresh() and release() predicates now require the acquisition
  fence (id, holder_pid, acquired_at::text) captured at acquire time, so a
  PID-reuse impostor or a stolen handle can never refresh or delete a
  successor's row. refresh() returns true only while owned; a fenced miss is
  distinguished from transient DB errors (which still throw and retry).
- cycle: runCycle owns a SERIALIZED background refresher (6x per TTL window,
  GBRAIN_CYCLE_LOCK_REFRESH_MS escape hatch) for the cycle lock only — Minion
  job-lock renewal stays on the phase-boundary hooks per the cycle.ts:618
  decision. A detected steal aborts an internal controller; the combined
  signal reaches every existing checkAborted() boundary, and the five long
  phases (synthesize, extract_atoms, patterns, synthesize_concepts,
  consolidate) race their awaits against it since their opts cannot carry a
  signal yet. The three raw yieldDuringPhase pass sites are now wrapped.
- A steal returns a structured partial report (reason 'lock_stolen') instead
  of throwing; completed phases' writes are durable, the freshness stamp is
  skipped, and the fenced release leaves the successor's row intact.
- supervisor: a fenced refresh returning false is CERTAIN lock loss, not a
  blip — exit LOCK_LOST immediately instead of resetting the failure counter.
- withRefreshingLock: stops its heartbeat and reports loudly when the fenced
  refresh proves the lock gone.

Closes TODO-OPS-2 (refresh had no rows-affected check, so lock loss was
undetectable).

Tests: db-lock-fencing (fence round-trip, steal → refresh false, fenced
release no-op, refresher abort/serialization/transient-vs-steal, yield hook
steal reporting), cycle-lock-steal.serial (end-to-end mid-run steal →
partial/lock_stolen report, no further phases, successor row intact +
steal-free regression guard). All pre-existing lock suites green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(minions): reset started_at on every automatic re-run path (W0 Tier-1 #7)

handleWallClockTimeouts anchors on now() - started_at, but only the manual
`jobs retry` path cleared started_at — its own docstring documented the bug.
The four automatic paths (failJob's delayed branch, handleStalled's requeue,
promoteDelayed, and releaseLeaseFullJob — the fourth site surfaced by
adversarial verification) preserved the FIRST claim's timestamp, so an
exponential-backoff job burned its wall-clock budget while parked in
'delayed' and could be dead-lettered before executing a single line of its
retry attempt.

All four paths now clear started_at; claim()'s COALESCE re-stamps per
attempt. Terminal failures (failed/dead) keep started_at for duration
accounting. Pinned end-to-end: a job whose first attempt ran an hour
survives the sweep on its fresh attempt, and the negative control proves the
sweep still kills genuinely overrunning attempts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(minions): shared killJobs tail — stall-death notifies parents; reapers use parents-first lock order (W0 Tier-1 #4)

handleStalled's dead-letter branch set status='dead' and emitted NOTHING: no
child_done inbox row, no aggregator unblock. A child that died via max-stall
stranded its parent in 'waiting-children' forever — the exact hang the v0.15
comment says was fixed for timeouts (resolveParent has no periodic caller;
the worker only logs counts). Meanwhile handleTimeouts and
handleWallClockTimeouts carried two verbatim copies of the ~45-line
notify-and-unblock block.

- One private killJobs(tx, rows, outcome, errorText) now owns the child_done
  insert + waiting-children unblock; all three reapers route through it.
  handleStalled's dead branch emits outcome 'dead' / 'max stalled count
  exceeded' (distinct from 'timeout' so consumers can tell stall-death from
  overrun).
- Deadlock safety (Codex eng-review D5.12): failJob locks the parent BEFORE
  touching the child, while the reapers previously updated children first —
  opposite lock order. All three reapers now discover candidates with a plain
  read, lock parents in ascending-id order via lockParentsOrdered(), then
  transition children under a re-checked FOR UPDATE SKIP LOCKED subselect in
  the same transaction.

Pinned: stall-exhausted child → child_done(dead) + parent flips to waiting;
budget-remaining stall requeues without touching the parent; all three
reapers' outcome/error strings asserted through the shared tail (D5.5).
Full minions e2e suite green (187 tests).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(embed): carry modality through every re-embed path — one shared field list (W0 Tier-1 #3)

CONFIRMED in the audit + adversarial verification: preserveCodeMetadata
(commands/embed.ts) rebuilt ChunkInputs without `modality`, and upsertChunks
overwrites that column from EXCLUDED — so every CLI re-embed path (embedPage,
embed --all, embed --stale, including the autopilot-reachable stale loop)
flipped image chunks to modality='text'. The image search arm filters
cc.modality='image', so image retrieval silently went to zero while keyword
search started returning raw OCR text. The minion twin in core/embed-stale.ts
carried modality correctly and its comment documented this exact hazard —
the two hand-copied field lists had diverged.

carryChunkMetadata (core/embed-stale.ts) is now the single carry list;
preserveCodeMetadata delegates to it, killing the divergence class at the
root (the full loop merge lands in W6). embedding_image stays deliberately
un-carried (COALESCEd by the upsert; getChunks returns pgvector strings).

Pinned: the carry preserves modality + all 8 code-metadata fields; an image
chunk round-trips the stale-merge intact; and the write-side contract test
documents WHY the carry is load-bearing (omission demonstrably resets to
'text').

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(import): throw typed ImportAbortError instead of process.exit — MCP server survives failed preflights (W0 Tier-1 #5)

runImport called process.exit(1) at five preflight/argv sites (deferred-setup
sentinel, missing embedding credentials, invalid --workers, missing dir,
unreadable dir). Correct for the CLI — but runImport is invoked IN-PROCESS by
the sync_brain MCP op (via performFullSync), the autopilot daemon, and the
minion sync handler, so a first/forced sync against a brain with unusable
embedding credentials terminated the stdio MCP server mid-tool-call with no
error envelope (verified reachable in adversarial review; daemon/worker paths
are partially shielded by noEmbed defaults, the MCP path was not).

The five sites now throw ImportAbortError (exitCode, alreadyReported) AFTER
printing their user-facing messages exactly as before; the CLI dispatch case
maps the error to process.exit(exitCode) — byte-identical CLI behavior. The
in-process callers get a normal error: the MCP op returns an error envelope,
the job handler fails the job, the daemon logs and continues.

Pinned: three abort classes throw typed (not exit), and the calling process
demonstrably survives.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(lint): single scan for --fix — true fixed count, half the work (W0 Tier-1 #14)

runLint ran its own full read+lint+fix loop for human output, then called
runLintCore a second time for the summary line. Every page was linted twice,
and because the first pass had already written the fixes, the second pass's
total_fixed counted against already-fixed content — `gbrain lint --fix`
printed "0 auto-fixed." after fixing N issues.

runLintCore now exposes per-page hooks (onPageScanned for the progress bar,
onPageIssues with the applied fix count); the CLI streams its human detail
from the same single pass that produces the canonical counts. Pinned: two
pages scan as exactly two ticks, total_fixed matches the page-level fix
count, the fix lands on disk, and a second run reports 0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(cli): confirm prompts resolve on EOF and refuse non-TTY in-prompt (W0 Tier-1 #15)

Port-ledger note: since the audit, both destructive-command callers
(pglite-repair, reinit-pglite) gained caller-side non-TTY guards
('Non-TTY environment requires --yes'), so the original always-hangs case is
already blocked upstream. The residual: a TTY session whose stdin hits EOF
mid-prompt still parked forever — pglite-repair's readline had no 'close'
handler and reinit-pglite's raw data-listener had no 'end' path (and its
prompt wrote to stdout, polluting --json output).

Both prompts now: refuse non-TTY in-prompt (defense-in-depth, safe default
false), resolve(false) on EOF/close, prompt on stderr, and clean up their
listeners. Decline paths and --yes/-y escape hatches unchanged. No new test:
exercising EOF-mid-TTY needs a PTY harness — the W5 prompt canonicalization
(core/prompt.ts) picks that up when all seven prompt copies converge.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(ci): guard self-test harness — a guard that cannot fail is not coverage (W0 Tier-1 #11)

The audit proved scripts/check-no-double-retry.sh had been PERMANENTLY GREEN
since it shipped: its `[^)]*` regex could not cross the `)` in `() =>`, so
the canonical banned shape `withRetry(() => engine.addLinksBatch(...))` was
invisible, and its multi-line fallback was gated on pcregrep — installed
neither locally nor in CI. check-jsonb-pattern.sh carried the same
nested-paren hole. Two more structural findings: package.json's `check:all`
was a second, stale, hand-synced guard registry (the exact disease this
fix-wave exists to cure), and three guards were reachable ONLY from it —
i.e. never run anywhere.

- Both regexes fixed; the no-double-retry multi-line pass now uses perl
  (always present) instead of pcregrep (never present). Real tree verified
  clean under the fixed patterns.
- scripts/guards-manifest.tsv is THE single guard registry: all 45 guards
  classified (scanner / buildfresh / repostate, per Codex D5.14 — build and
  freshness guards are exempt-with-reason, not fixture-tested).
- scripts/guard-self-test.sh runs every selftest=yes scanner against
  known-bad (must fail) and known-good (must pass) fixture trees via the
  GBRAIN_GUARD_ROOT seam, enforces manifest completeness for new guards, and
  carries a runtime budget (D4.5) so guard sprawl surfaces here first.
  Wired into `bun run verify`; adding a self-test = flip a manifest flag +
  two fixture files.
- `check:all` deleted; its three orphaned guards (newlines, exports-count,
  no-legacy-getconnection) verified green and wired into the real registry.

The bad fixtures are the exact shapes the old regexes missed — the harness
fails loudly on the pre-fix scripts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(test-infra): PGLite snapshot default-on for `bun run test` — idempotent, shard-safe, handler-aware (W0 Tier-1 #16)

500+ test files each cold-boot PGLite and replay all 126 migrations, but the
snapshot fixture that skips that was enabled ONLY inside scripts/ci-local.sh
— the everyday `bun run test` loop paid full cold-init on every file
(measured: 1.63s → 0.91s per PGLite-booting file with the fixture).

- run-unit-parallel.sh (the `bun run test` entrypoint) builds + exports the
  snapshot BEFORE its shard fan-out. Opt out: GBRAIN_NO_SNAPSHOT=1.
- build-pglite-snapshot.ts is now idempotent: hash short-circuit exits in
  ~40ms when fresh, and REBUILDS stale snapshots — the old build-if-missing
  guard left a stale-but-present snapshot permanently on the warn+slow path.
  ci-local.sh now calls it unconditionally.
- Shard/workspace concurrency safety (Codex D5.8): atomic mkdir lock with
  takeover-on-stale; tar written first, version file last, so a crash can
  never leave a fresh-looking torn fixture.
- Hash soundness (Codex D5.13 / #4): 19+ migrations carry executable
  `handler` code with empty sql — invisible to the sql-only hash, so editing
  a handler reused a stale snapshot. The handler SOURCE now folds into the
  hash via Function.prototype.toString.

Migration-replay coverage is unchanged: the replay canary tests clear
GBRAIN_PGLITE_SNAPSHOT themselves and migrate.test.ts exercises
runMigrations directly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(test-infra): snapshot bakes the pinned test embedding shape; loader refuses shape mismatches (W0 follow-through)

Turning the snapshot default-on exposed a latent poisoning class: the build
script ran with an UNCONFIGURED gateway, so initSchema fell back to the
shipped default (1280-d zembed columns) — while bunfig's preload pins every
`bun test` file to the legacy OpenAI 1536-d shape. The moment tests loaded
the fixture, every embedding write failed with "expected 1280 dimensions,
not 1536" (115 suite failures from one root cause).

- The pinned shape now lives ONCE in test/helpers/legacy-embedding-config.ts;
  both the bunfig preload and the snapshot build script consume it (no
  hand-copied twins — the exact disease this wave cures). The build also
  isolates GBRAIN_HOME so ambient machine config can't leak in.
- The version file records dims= and model= alongside the schema hash; the
  loader resolves its own would-be shape through the same gateway-or-default
  fallback initSchema uses and REFUSES a shape-mismatched snapshot (falls
  back to cold init with a rebuild hint). Pre-W0 hash-only version files
  read as stale. A test that reconfigures the gateway to a different shape
  now correctly bypasses the fixture instead of writing into wrong columns.
- The build's freshness short-circuit checks all three lines.
- Rephrased a guard comment that spelled a batch-call token literally —
  check-system-of-record scans scripts/ comments (the prose-bleed class,
  third occurrence this month).

put-page-provenance: 9 fail → 0 under the fixture, still 3x faster than
cold init.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(eval): fix-wave baseline metrics — measure the 10x claim (W0, D4.13)

Records the series' starting numbers: god-file line counts (the registry
waves' targets), guard census (47 guards / 3 self-tested / single registry),
and the measured snapshot speedup (1.63s → 0.91s per PGLite test file).
Each wave PR appends its row; the deltas are the receipt. The retrieval-
quality canary (eval gate on a non-production brain) is documented as the
mandatory pre-W1 step — W0 touches no search paths and the production brain
is single-writer-held by the live serve.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(cycle,ci): duck-type-tolerant signal combining + contract-test updates (W0 follow-through)

- anyAbortSignal no longer uses AbortSignal.any: CycleOpts.signal has always
  been duck-typed in practice (test stubs pass { aborted: false } and flip
  the flag; pre-W0 the raw object flowed straight into checkAborted).
  AbortSignal.any threw ERR_INVALID_ARG_TYPE and broke the autopilot-cycle
  handler suite. Manual fan-in: real signals propagate via listener,
  listener-less stubs are polled at 50ms, and the RETURNED signal is a
  genuine AbortSignal so phases can hand it to fetch/timers.
- cycle-abort.test.ts source-contract tests updated to the cycleSignal truth
  (boundaries now check the combined external+steal signal) and additionally
  pin that the combine folds BOTH sources.
- Restored the `typecheck` entry an errant edit dropped from
  run-verify-parallel's CHECKS array (caught by its own contract test —
  the registry pinning working as designed).
- De-flaked the refresher steal test: poll to a 5s deadline instead of a
  fixed 120ms sleep (shard-load timer starvation).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: pre-landing review fixes — 5 specialists + coverage audit findings (W0 ship pass)

Specialist review (testing/maintainability/security/performance/data-migration,
5 parallel fresh-context reviewers + ship coverage audit at 92%) on the W0
diff. Every accepted finding fixed in-line:

- db-lock: fence rendered as extract(epoch from acquired_at)::text — THREE
  specialists independently flagged timestamptz::text as GUC-fragile (the
  fence is captured on the acquire pool but compared on the direct pool; a
  TimeZone/DateStyle divergence would turn every refresh into a false steal
  and loop the supervisor through LOCK_LOST). Epoch text is session-invariant.
- cycle: anyAbortSignal returns {signal, dispose}; runCycle disposes in its
  finally — the forward listener lives on the CALLER's signal and the
  autopilot daemon reuses one shutdown signal across every tick, so
  undisposed combines accumulated listeners + captured controllers for the
  daemon's lifetime (MaxListenersExceededWarning within ~10 ticks). Stub
  poll timers clear on dispose too. Helper moved out of the import block and
  behaviorally tested (5 cases incl. the daemon-leak class).
- queue: retroactive stranded-parent sweep on every handleStalled tick — the
  per-kill unblock was forward-only, so parents stranded by PRE-upgrade
  stall-deaths (children already 'dead') never healed. Idempotent NOT-EXISTS
  UPDATE; pinned with stranded-heals + live-child-stays tests.
- build-pglite-snapshot: the stale-lock takeover could NEVER acquire
  (mkdirSync on an existing dir always throws), so one crashed builder left
  every future rebuild waiting the full deadline then proceeding UNLOCKED
  forever. Takeover now removes the stale dir first; lock timeout is
  env-tunable; hermetic setup moved into main() (ESM hoisting made the
  module-scope placement illusory) and the temp home is cleaned up.
- check-no-double-retry.sh: the perl multi-line pass exited 1 from clean
  batches — under pipefail, xargs's 123 would override grep's verdict the
  moment src/ outgrows one batch (a future silent miss of the exact class
  this guard just got cured of; repro'd by the reviewer). Output-presence now
  decides; multi-line bad fixture added so the pass self-tests.
- check-jsonb-pattern.sh: the widened greedy pattern false-positived a SAFE
  ::text::jsonb line followed by a paren-bearing ${expr()}::jsonb on the same
  line (proven by repro); bracket-bounded [^}]* pattern can't span
  interpolations — good fixture now pins the multi-interpolation shape.
- check-engine-dynamic-import.ts: also matches require() calls (the new
  snapshot-loader require was invisible to the guard, its marker decorative);
  4 pre-existing lazy requires in tryLoadSnapshot marked with their existing
  justification.
- Coverage gaps closed: supervisor fenced-false → immediate LOCK_LOST test;
  snapshot shape/hash guard tests (pre-W0 version files refused, dims/model
  mismatch refused, handler-edit changes the hash); anyAbortSignal behavior
  suite; steal-test window widened 200ms → 1.5s (shard-load starvation).
- lint: tree walked once (onPagesCollected sizes the progress bar; the CLI's
  extra collectPages walk removed); stale docstrings corrected (hook fires
  AFTER the fix attempt; carry list includes modality).
- Suite hygiene: 3 fresh-brain-premise tests opt out of the default-on
  snapshot; the check:all contract test now pins the single CHECKS registry.
- TODOS.md: 6 fix-wave deferrals filed (each individually decided in review);
  TODO-OPS-2 marked CLOSED by this wave.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: red-team findings — unfenced file-lock half, 5th started_at path, sync-phase steal coverage (W0 ship pass 2)

The post-specialist red team found what five specialists and the coverage
audit all missed — two of them critical:

- cycle (CRITICAL): the PGLite composite lock's FILE half was rewritten
  unconditionally even when the fenced DB refresh reported a steal — the
  losing holder clobbered the successor's file lock with its own pid on the
  very tick it detected the loss, after which its pid-checked file release
  DELETED the successor's only host-local protection mid-run (single-writer
  violation). The file half now rewrites only while the DB fence says owned.
- queue (CRITICAL): fifth path of the started_at class — every
  waiting-children→waiting parent unblock (killJobs, completeJob resolve,
  failJob remove_dep/ignore, cancelJob, resolveParent, the new retroactive
  sweep: 7 sites) preserved the parent's attempt-1 anchor, so an aggregator
  whose children ran >5 minutes was wall-clock dead-lettered on re-claim —
  orphaning the exact child_done results the W0 parent-unblock fix just
  delivered. All 7 unblock sites now clear started_at; pinned by a
  parked-parent-survives-the-sweep test.
- cycle: the sync phase — production's LONGEST await (resumable imports can
  run hours) — was the one long phase outside steal coverage. Now raced like
  the other five (sync checkpoints, holds its own per-source lock, and its
  stall watchdog bounds the dangling import).
- build-pglite-snapshot: takeover verifies lock-dir mtime staleness before
  rmdir (two exhausted waiters could steal each other's LIVE lock);
  hermetic temp home created only past the short-circuit (was leaking one
  dir per `bun run test`).
- Stale docs: jobs.ts import-handler comment claimed a process.exit that no
  longer exists; CLAUDE.md's engine-dynamic-import exception list now names
  the snapshot loader's require() cluster (build:llms regenerated in this
  commit).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: bump version and changelog (v0.45.15.0)

W0 verified-bug hotfix wave of the code-smell fix-wave series. All six
version locations synced (VERSION, package.json, openclaw.plugin.json,
runbook stamp, template stamp, lockfile).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: update project documentation for v0.45.15.0

- docs/TESTING.md: tiers table now documents the default-on PGLite schema
  snapshot for `bun run test` (GBRAIN_NO_SNAPSHOT opt-out); new "PGLite
  schema snapshot" + "Guard registry and self-test" sections (build/loader
  contract, GBRAIN_SNAPSHOT_LOCK_TIMEOUT_MS, guards-manifest.tsv,
  GBRAIN_GUARD_ROOT); removed the deleted `check:all` tier; added the nine
  new W0 test suites to the unit-test inventory.
- docs/architecture/KEY_FILES.md: current-state refresh for db-lock.ts
  (fenced handles, boolean refresh, LockStolenError), cycle.ts (dedicated
  serialized lock refresher, GBRAIN_CYCLE_LOCK_REFRESH_MS, steal-abort with
  reason lock_stolen, composed DB+file lock semantics; dropped the closed
  TODO-OPS-2 residual), minions/queue.ts (shared killJobs tail,
  lockParentsOrdered, stranded-parent sweep, started_at resets),
  supervisor.ts (fenced miss exits LOCK_LOST immediately), embed.ts
  (carryChunkMetadata shared field list), import.ts (typed
  ImportAbortError), lint.ts (single-pass --fix), pglite-repair.ts (EOF-safe
  stderr confirm prompts), check-no-double-retry.sh (arrow-paren-crossing
  pattern, perl fallback); new entries for guards-manifest.tsv +
  guard-self-test.sh and build-pglite-snapshot.ts; swept stale check:all
  references.
- CONTRIBUTING.md: verify check count refreshed; check:all replaced with the
  guard-registry + self-test workflow.

llms bundles verified fresh (bun run build:llms — no byte changes;
test/build-llms.test.ts green). CHANGELOG/TODOS/VERSION already current from
the ship pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: cross-model doc-review fixes for v0.45.15.0 (Codex pass)

- CHANGELOG 0.45.15.0: the image-chunk recovery command is `gbrain backfill
  modality` (flipped chunks are not stale, so an embed --stale re-run cannot
  restore them — doctor names the same fix); upgrade note now also covers
  jobs supervisor/worker restarts; the prompt-hang fix names its two commands
  instead of implying all destructive prompts; guard self-test claim scoped
  to self-tested scanners.
- KEY_FILES: cycle entry counts all 23 ALL_PHASES (was 9); raced-wait nuance
  for the 5 long phases (in-flight work runs to its bounded timeout);
  snapshot-lock last-resort unlocked path + version-file-not-tar gate scope;
  guards manifest registers/classifies but does not schedule (CHECKS array
  stays the execution list).
- TESTING.md: same snapshot-lock last-resort honesty.
- CONTRIBUTING.md: self-test scope (selftest=yes rows), stale ~85s inner-loop
  figure and 19+ check count refreshed.
- FIX_WAVE_BASELINES.md: two W0 line counts refreshed per the doc's own
  method (post-ship-pass HEAD).

llms bundle rebuilt (no byte changes); guards + build-llms test green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: re-bump to v0.45.16.0 (version queue collision with #4125)

The sibling jobs fix wave (PR #4125, open) claims v0.45.15.0; per the
user's call this PR advances past it. All six version locations re-synced.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: withEnv() for snapshot opt-out in embedding-dim fresh-brain case (test-isolation guard)

The W0 ship-pass fix used a manual save/delete/restore of
GBRAIN_PGLITE_SNAPSHOT, which check-test-isolation rule R1 flags on CI
(the local ship verify ran before this file gained the mutation).
withEnv() scopes the opt-out to the connect() call with identical
behavior.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 21:10:03 -07:00
Garry TanandClaude Fable 5 0f03a0f929 v0.45.15.0 fix(minions,cli): duplicate-cycle single-flight, claim-time timeout backfill, real jobs --help (#2/#3/#4) (#4125)
* fix(cli): real help for jobs and its subcommands — never start a daemon on --help

`jobs` was in CLI_ONLY but not CLI_ONLY_SELF_HELP, so `gbrain jobs --help`
printed the generic one-line stub and the real help block (with the worker
entry point) was unreachable. Naively registering it would have been worse:
a help token after the subcommand fell through into the subcommand body, so
`jobs work --help` would have started a real worker daemon.

- Hoist the help block to JOBS_HELP; document watch, stats --cluster-errors,
  smoke rescue flags; precise footer naming exactly the five subcommands
  with dedicated help.
- JOBS_SUBCOMMAND_HELP (bootstrap.ts pattern) for work/supervisor/submit/
  watch/prune, guarded at the top of runJobs BEFORE the thin-client refusal
  and the switch. Only --help/-h; bare 'help' can be a job name.
- cli.ts: add jobs to CLI_ONLY_SELF_HELP + SELF_HELP_WITHOUT_ENGINE (help
  answers engine-free); top-level JOBS section gains supervisor + watch.
- Regenerate the CLI flag registry (help text is harvested for flags).
- Tests: jobs-subcommand-help.serial (spawned CLI, engine-free env hygiene,
  anti-stub + fast-exit-proves-no-daemon); jobs added to HELP_WITHOUT_BRAIN.

* fix(minions): claim-time timeout fallback + v128 backfill/duplicate-cleanup + jobs get budget surface

Pre-existing queued rows with timeout_ms = NULL fell to the minutes-scale
null-default wall-clock sweep (2 x lock-duration x max_stalled ~= 5 min at
defaults), so long handlers queued before submit-time stamping were
dead-lettered mid-progress — identical work succeeded or died purely on
insertion time, and the queue was effectively undrainable.

Three layers now apply the handler budget (explicit timeout_ms always wins):
- claim(): COALESCE timeout_ms from HANDLER_DEFAULT_TIMEOUT_MS (raw-object
  jsonb bind; executeRawDirect preserved), deriving timeout_at from the
  coalesced value. Durable invariant; also revives the worker abort timer,
  deadlineAtMs budget clamping, inline-drain abort, and the handleTimeouts
  first-killer for legacy rows. Names outside the map stay NULL (fail-open).
- migration v128 statement 1: one-shot backfill for non-terminal rows of the
  8 long-lane handlers (values snapshotted at authoring time; never sync with
  the live map). No timeout_at stamp for active rows — the 2x wall-clock
  bound is the gentler sufficient repair.
- migration v128 statement 2: cancel all-but-newest ticker-keyed duplicate
  waiting cycles per (name, queue, source) — prefix-guarded so manually
  submitted cycles are never touched; rows preserved as cancelled for audit.

jobs get now prints the effective budget (1x deadline when claimed; 2x
wall-clock backstop) with a defensive Date|string deadline render;
timeout_at joins JOB_DATE_FIELDS for thin-client rehydration.

Tests: migrations-v128 (backfill matrix, cleanup scopes, manual/parented
exclusions, ledger + SQL-level rerun idempotency, empty-table no-op),
claim-fallback block in minions.test.ts, formatJobDetail render states,
rehydration field. PGLite snapshot rebuilt for the new migration.

* fix(autopilot,minions): maxPending single-flight dispatch guard + honest coalesce surfaces

The autopilot dispatch guards accumulated unbounded byte-identical cycles
once a job stalled in 'active': the slot idempotency key rotates every
baseInterval so it never dedups across ticks, and maxWaiting counts only
waiting rows. One observed brain held ~111 queued duplicates with zero
completions.

New INTERNAL submit option maxPending (single-flight):
- counts waiting rows PLUS live-lock active rows (lock_until > now()); an
  expired-lock active belongs to a dead/blocked worker and never suppresses
  dispatch, so fresh waiting rows keep feeding the waitingClaimable>0 wedge
  detectors instead of starving them
- EXACT source scope via COALESCE(data->>'sourceId', data->>'source_id')
  compared with IS NOT DISTINCT FROM (NULL matches only NULL) — a legacy
  no-source dispatch can never coalesce into a per-source row; maxWaiting
  keeps its intentional NULL-as-wildcard scope, now two-spelling aware
- same advisory-lock namespace as maxWaiting so both guards serialize;
  maxPending checked first when both are supplied
- adopted at all three autopilot dispatch sites (legacy fallback,
  per-source fan-out — safe there precisely because of the exact scope —
  and global maintenance), replacing maxWaiting where present

Honest coalesce surfaces: all three add() coalesce paths (idempotency
fast-path, cap-hit, ON CONFLICT race fallback) stamp non-persisted
coalesced metadata; fanout splits FanoutResult.dispatched vs .coalesced,
emits dispatch_coalesced events, and the summary reports both. The
backpressure audit gains pending_count/max_pending, and jobs stats prints
a 24h Backpressure line (current + previous ISO-week audit files, queue
filtered) plus a suppressed-by hint naming the in-flight job — so
suppression is never silent even while waiting sits at 0.

Tests: maxPending block (live-lock vs expired-lock, exact NULL scope, both
spellings, both-guards interaction, race smoke), coalesce-metadata pins,
fanout opts + coalesce-event guards, audit reader (week boundary, queue
filter, malformed lines), and DB-gated e2e: issue reproduction (stalled
active suppresses cross-slot re-dispatch), full recovery loop with a real
claim + real sweeps, fan-out preservation under maxPending, and a real-PG
concurrent same-scope race pinning the advisory-lock guarantee.

* fix(minions,jobs): adversarial-review fixes — honest coalesce contract + hardened surfaces

Codex structured review (ship gate, P2s) + Codex adversarial challenge:
- dispatchGlobalMaintenance returns dispatched: false when the submission
  coalesced — same honest-dispatch contract as dispatchPerSource.
- The jobs stats suppression hint is driven by the audit's latest
  returned_job_id per name and scoped to THAT job's source, so on
  multi-source brains source A's waiting row can't mask source B's wedge.
  readRecentCoalesceCounts now returns {count, last_returned_job_id}.
- Backpressure audit writes are deferred to after the submission
  transaction commits — filesystem I/O no longer runs while holding the
  advisory lock + a pool connection (a hung audit volume degraded one
  submission, not the whole scope's queue).
- The audit reader caps per-file reads at the last 4MB (tail slice,
  partial first line dropped) so a caller-grown audit file can't OOM the
  diagnostic that reads it.
- v128's duplicate cleanup additionally requires data.sourceId IS NULL:
  the ticker only writes snake_case source_id, so camelCase rows are
  by definition not ticker-provenance and are never swept.
- Supervisor help states the real --max-crashes semantics (soft degraded
  threshold; hard stop at 10x N via GBRAIN_SUPERVISOR_HARD_STOP_CRASHES)
  and the brain-scoped PID file default.

* chore: bump version and changelog (v0.45.15.0)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: brain-scoped supervisor pidfile default in minions-deployment example

The jobs.ts help text was corrected this wave to name the brain-scoped
default (~/.gbrain/supervisor-<brain-id>.pid); the deployment guide's
example output still showed the old un-scoped path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: accuracy pass from cross-model doc review (v0.45.15.0)

- CHANGELOG: dead-worker (not wedged-worker) dispatch claim; scope the
  truthful-dispatch bullet to autopilot cycle dispatch; jobs submit has no
  --json flag (JSON is the default non-follow output); v128 manual-cycle
  carve-out names the ticker-key heuristic honestly.
- KEY_FILES: handler-timeouts map is 8 handlers across 30/10/60-min tiers;
  queue.ts subagent gate is capability-based (classifyCapabilities), not the
  retired Anthropic pin; drop stale jobs.ts line-range refs; jobs submit
  flag list is not the "full" MinionJobInput surface (maxPending internal).
- jobs.ts help: prune --older-than is days-only (no Nh forms); supervisor
  exit code 4 (DB queue lock lost) documented.
- types.ts: coalesced JSDoc names the real JSON output path.
- minions-deployment: detach payload example matches the real fields.
- TODOS: bank handler-catalog + dispatch-event-schema doc gap (P3).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(cli,autopilot,jobs): red-team review fixes — prototype-safe help lookup + honest targeted dispatch + hint precision

Red-team pass on the final diff (4 informational findings, all fixed):
- Object.hasOwn guards on JOBS_SUBCOMMAND_HELP and bootstrap's
  SUBCOMMAND_HELP: `jobs constructor --help` (or toString/valueOf/…)
  printed Object.prototype functions instead of the full help.
- The targeted-plan dispatch loop now splits on job.coalesced and emits
  dispatch_coalesced — the honest-dispatch contract this wave applies to
  every other dispatch surface in the same file.
- jobs stats: the hint slice reuses the count-sorted entries (insertion
  order could crowd out the highest-volume names past the cap), and the
  target CTE re-checks name+queue so a shared cross-brain audit dir can
  never name an unrelated job as the suppressor.

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 20:42:18 -07:00
Garry TanandClaude Fable 5 2b8c200b6e Revert #4126 (v0.45.15.0 test-safety wave) — operator-requested unmerge
Reverts the six rebase-merged commits c439fad23..418dc1543:
DATABASE_URL preload guard, name floors, coverage gate, heavy-lane floor,
version bump, and docs. Restores master to v0.45.14.0 (dd99e40c2 state).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 20:34:33 -07:00
Garry Tan 418dc15437 fix(test): keep the coverage gate's own scrub-pattern out of the R1 isolation lint's sight
The gate detects 'delete process.env.X' as a scrub-not-read; the R1 lint greps
the same token textually and flagged the gate's comment and classifier fixture
as env mutations. Comment reworded; fixture built by concatenation so the
classifier still receives the contiguous statement.
2026-08-14 20:20:49 -07:00
Garry TanandClaude Fable 5 bf0a49bf97 docs: update project documentation for v0.45.15.0
Cross-reference pass after the #3485 test-safety wave (Wave -1):

- docs/TESTING.md: complete the guard layers (heavy shell floor
  tests/heavy/_db_floor.sh, schema-drift's accepted inline floor), note
  the phantom-redirect Postgres arm riding the e2e lane in the file
  taxonomy + E2E inventory.
- docs/architecture/KEY_FILES.md: scripts/run-e2e.sh entry updated to
  current behavior (no-args list carries phantom-redirect parity; #3485
  opt-in boundary, GBRAIN_DATABASE_URL drop, GBRAIN_E2E_ALLOW_DB
  preserved through the env scrub).
- CONTRIBUTING.md: heads-up that bare `bun test` refuses to start with a
  database URL ambient + the name floor for own-Postgres/Supabase e2e.
- tests/heavy/README.md: database name floor section (which scripts
  source it, PGLite scripts unset instead, new-script rule).
- .env.testing.example: Supabase's default "postgres" database name
  fails the floor — dedicated test DB or one-shot GBRAIN_E2E_ALLOW_DB.
- CHANGELOG.md v0.45.15.0: three accuracy-of-wording touches (headline
  "silently", lane boundary phrasing, note the one accepted inline
  floor) — no entries removed or regenerated.

Codex cross-model doc review ran; concrete gaps applied above. llms
bundles regenerated (no byte changes — touched docs are link-only).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 20:20:49 -07:00
Garry TanandClaude Fable 5 f7b8890b8d chore: bump version and changelog (v0.45.15.0)
TESTING.md documents the four guard layers and the cwd caveat; TODOS.md
files the disclosure-policy follow-up (P2).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 20:20:49 -07:00
Garry Tan 9f598e4b20 fix(tests-heavy): shared database name floor for the heavy shell lane (#3485)
The heavy lane runs schema drops, source-registry rewrites, migration
replays, and parallel syncs against whatever the environment names — outside
bun, where the preload guard cannot fire. tests/heavy/_db_floor.sh mirrors
test/helpers/db-guard.ts: sourced by run-heavy.sh and by every script
documented for direct invocation, it floors BOTH DATABASE_URL and
GBRAIN_DATABASE_URL (the CLI these scripts shell out to prefers the latter)
and strips query strings before extracting the name, so a
?host=/tmp/test-sockets parameter cannot smuggle a test-shaped segment past
the check.
2026-08-14 20:20:49 -07:00
Garry Tan 296222eff0 feat(test): refuse to start a test run while a database URL is ambient (#3485)
A bunfig [test] preload (registered first) hard-fails any bun test invocation
while DATABASE_URL or GBRAIN_DATABASE_URL is set, unless
GBRAIN_TEST_ALLOW_DATABASE_URL=1 — refusing with instructions, never silently
unsetting (a silent unset would turn DB-gated e2e tests into green skips).

Boundaries: run-e2e.sh and the e2e/heavy workflows opt in at their own
subprocess boundary (run-e2e.sh also keeps the opt-in vars past its hermetic
GBRAIN_* scrub and drops GBRAIN_DATABASE_URL, which has no name floor on
spawned-CLI paths); the unit/slow wrappers strip both vars instead — unit
tests need no database — which keeps `bun run test:full` with a DB URL
exported reaching its e2e leg. The phantom-redirect parity file rides the
e2e lane and CI's jsonb-parity job so its Postgres arm stays reachable.

Six subprocess tests spawn real bun test children against the actual
bunfig registration: refuses each var, refuses both, strict override value,
override allows, empty-string treated as unset, clean run.
2026-08-14 20:20:49 -07:00
Garry Tan c439fad23e fix(test): name-floor guard for every destructive-SQL test + repo-wide coverage gate (#3485)
Moves assertSafeE2eDatabaseUrl to leaf module test/helpers/db-guard.ts
(re-exported from test/e2e/helpers.ts for existing call sites) and calls it
before connect() in all ten files that run destructive SQL against the
ambient URL — the eight from #3485, one newer offender
(bootstrap-keyed-postgres.serial), and the raw-postgres()-client OAuth suite
the original audit could not see.

test/db-guard-coverage.test.ts is the static gate that keeps the class
closed: walks every test file bun collects repo-wide (all naming patterns,
fixtures included), detects ambient-URL reads at the assignment site (any
binding name, both env vars, bracket notation), recognizes four connect
idioms, treats env-var deletes as scrubs not reads, refuses comment-only
guard mentions, and pins its own classifiers with positive controls so it
can never pass vacuously.

Patch for the ten files adopted from #3485 by @cheRoma (fork access blocked
a PR) — thank you.
2026-08-14 20:20:49 -07:00
Garry TanandClaude Fable 5 dd99e40c2a v0.45.14.0 feat(bootstrap): harness mode — wire framework-spawned agents to a running serve (#4043) (#4106)
* feat(bootstrap): harness-lane settings writers — marker/path params, permissions.allow, CODEX_HOME (#4043 step 1)

writeClaudeHooksAt/removeClaudeHooksAt with marker VALUE parameterization
(bootstrap-v1 and bootstrap-harness-v1 coexist; each removal strips only its
own), onBrokenJson relocate|abort policy (user-scope files must never be
relocated over a stray comment), refuseOnForeignGbrainMarker double-fire
guard, and addPermissionsAllowEntry/removePermissionsAllowEntry (set
semantics, no marker, foreign entries preserved). Atomic writes hardened:
realpath-resolved targets (dotfile symlinks survive), mode preservation,
random tmp suffix, timestamped backup strategy. codexConfigPath now honors
CODEX_HOME (config dir itself — pinned by the real-codex e2e convention).
Legacy wrappers keep byte-identical behavior; existing writer suite untouched
and green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(bootstrap): codex-toml managed block writer — the fired CX2-17 revisit (#4043 step 2)

One [mcp_servers.<name>] table with inline bearer_token between full-line
markers; everything outside survives byte-for-byte. Foreign-server detection
parses the config (Bun.TOML.parse, no new dependency) with our block
stripped, so inline-table/dotted/quoted spellings can't false-negative into
a codex-bricking duplicate table. Rewrites re-anchor at EOF; renders are
parse-validated with an ours-keys-exactly assert before rename; damaged
markers refuse. Secrets hygiene: 0600 tmp/target/.bak, group-readable
configs tightened with a note. CRLF preserved, missing trailing newline
repaired. TARGETS['codex-2026-08'] flipped to verified (codex-cli 0.147.0:
serde field scan; codex hooks existence recorded; CODEX_HOME resolution).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(auth): least-privilege legacy tokens — adopt the scopes TEXT[] column (#4043 step 3)

The dormant original-schema access_tokens.scopes column becomes THE scope
store: verifyAccessToken's legacy branch honors it (NULL = grandfathered
full access, so every existing token is byte-identical; a filtered-empty
array is deny, so typos fail closed), and a column is structurally immune
to the permissions-object-replacement wipe class. That class gets fixed at
its known site too: auth permissions set-takes-holders now MERGES into the
permissions JSONB instead of replacing it (a routine visibility edit would
have silently deleted the source_id federation grant and re-escalated).

New surface: gbrain auth create --scopes read,write (comma/whitespace,
mint-time validation); auth list shows id + scopes columns (grandfathered
rendered honestly); auth revoke --id <uuid> for precise revocation (names
are not unique — bulk revoke-by-name now says when it hit several). New
src/core/token-mint.ts (mintLegacyToken with federation source grant +
RETURNING id; revokeLegacyTokenById never touches same-name siblings) for
the harness rotation contract. The admin dashboard's agents endpoint stops
hardcoding full access for every legacy key and reads the real grant.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(bootstrap): harness receipt + mcp-registration core extraction (#4043 step 4)

HarnessReceipt is a machine-level sibling of receipt.json (the install
receipt is workspace-keyed; a harness-only box has no workspace) with the
same CX2-12 discipline: typed read states, newer-format refusal, broken-file
backup-aside, atomic 0600 writes. Write-ahead contract: targets persist as
pending at mint time and confirm as wiring lands, and token.previous_id
carries the prior token through the mint-first rotation, so a crash at any
step leaves a receipt --remove can consume.

The pure MCP-registration helpers (normalizeMcpUrl, argv builders,
redactToken, validateToken, shellQuote/cmdString) move from
src/commands/connect.ts to src/core/mcp-registration.ts — the harness lane
lives in core and core must not import from commands. connect.ts re-exports
(surface + tests unchanged). buildClaudeMcpAddArgv gains an optional scope
param (claude's default is local; harness must pass user); loopback helper
exported for the harness --url guard.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(bootstrap): `gbrain bootstrap harness` — wire framework-spawned sessions to a running serve (#4043 step 5)

The orchestrator (src/core/bootstrap/harness.ts): consent block in the
#4029 honesty register (reach stated as fact, transcript capture its own
numbered item, off-ramps in the same breath; non-TTY requires --yes),
/health probe with a loopback guard (remote brains are gbrain connect's
charter), mint-first rotation (previous token revoked BY ID only after
every target confirms and the smoke passes), write-ahead harness receipt
(crash at any step leaves consumable state), registration ownership checks
(--force to replace a foreign-url server; --remove skips what it no longer
owns), user-XOR-project hook scopes with the double-fire refusal, the
GBRAIN_HOOK_LANE=harness runtime defer guard in `gbrain hook` (workspace
bootstrap installs win), --no-capture context-only wiring, Postgres
degradation + version-skew honesty lines, --status with host-config token
recovery and honest degrades, and engine-free-first --remove that defers
the revoke under a live PGLite serve.

Dispatcher wiring: `bootstrap harness` subcommand (home-dir lock, own
install-log phase), uninstall runs harness removal FIRST (revoke needs the
DB alive; --delete-brain would destroy harness.json) and treats
NO_RECEIPT/HOME_GUARD/RECEIPT_MISMATCH as "no workspace install" once
harness wiring is cleared; runHooks' codex stdio lane defers to a
harness-managed server name (one owner per name); the stale "Codex has no
hook system" line now states the truth. Flag registry regenerated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(bootstrap): harness doctor check + lifecycle e2e; CLAUDE_CONFIG_DIR-safe user-settings path (#4043 step 6)

Doctor gains bootstrap_harness_health inside the existing bootstrap check
group (no new status phase): skip when not a harness box / warn when the
serve is down (a normal transient) or the receipt is unreadable / fail when
targets are failed-or-pending or a rotation never converged — and a
harness-only box now opens the bootstrap check gate at all (it previously
got ZERO checks).

E2E lifecycle against a real `serve --http` on a hermetic PGLite brain:
pre-minted scoped token (the documented PGLite escape), real /health +
bearer smoke, both harness lanes wired, --status with token recovery from
the codex block, --remove leaving the codex config byte-identical, the
mint-under-live-serve refusal, and a live insufficient_scope refusal of an
admin op — least privilege proven end to end.

Root-cause fix the e2e caught: Bun's homedir() reads the password database
and ignores a remapped HOME, so claudeUserSettingsPath now resolves via
CLAUDE_CONFIG_DIR (Claude Code's own override) then $HOME explicitly —
without it, sandboxed runs write into the operator's REAL settings file
(the write-ahead receipt's remove path self-healed the one incident).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(bootstrap): v0.45.9.0 — local harness mode docs, scoped-token honesty pins, TODOS follow-ups (#4043 step 7)

docs/guides/bootstrap.md gains the "Local harness mode" section + the
missing Postgres row in the degradation matrix; DEPLOY.md/CODEX.md stop
claiming the token grandfather is unconditional and distinguish the connect
lane (token in env) from the harness lane (inline, 0600, consented);
KEY_FILES.md bootstrap cluster describes the parameterized writers,
codex-toml.ts, harness.ts, token-mint.ts, and mcp-registration.ts in
current-state voice; RESOLVER.md routes "wire this box's coding agents"
to bootstrap harness; setup skill points at it. Seven follow-ups filed in
TODOS.md (serve port record, http-transport scope asymmetry, unique token
names, codex hook lane, PGLite admin-lane minting, OpenClaw setup hook —
self-demoted: plugin installs run with lifecycle scripts disabled and the
manifest has no setup field — and federated-drift visibility). Doctor's
harness messages spell flags without leading dashes (the flag-registry
prose-bleed class; registry stays fresh). VERSION/package.json/CHANGELOG/
openclaw.plugin.json → 0.45.9.0 (0.45.8.0 is claimed by an open PR);
template stamp + runbook stamp refreshed; llms bundles rebuilt.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(doctor): categorize bootstrap_harness_health + de-flag harness message prose (#4043 triage)

The full-suite triage against a pristine-master baseline surfaced the two
in-branch failures: the doctor-categories drift guard (new check name not
in OPS_CHECK_NAMES) and the flag-registry freshness guard (doctor's harness
messages carried bare id/http flag tokens, which the generator harvests
into every importing command's allowlist — the known prose-bleed class;
messages now spell flags without leading dashes). Every other failing file
(18) fails identically on master with this environment — pre-existing,
not this wave's.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(hook): pin the harness-lane yield guard + honor the io.cwd seam (#4043 eng review E5)

The GBRAIN_HOOK_LANE=harness defer guard (workspace bootstrap installs win
over user-scope harness wiring — the C6 double-fire defense) was the one
new branch with no direct test. Two serial cases now pin it: lane +
bootstrap-v1 markers in the cwd → every event yields silently (exit 0, no
output, no heartbeat); lane without markers — including a harness-marker-
only settings file — runs normally and heartbeats. The guard now resolves
the cwd through the same io.cwd test seam the handlers use.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(bootstrap): harness convergence + rotation hardening — eng-review outside-voice X-batch (#4043)

Fifteen codex outside-voice findings, thirteen accepted and fixed, one
accepted as an advisory line, one kept as the settled PGLite deferral:

- X1: explicit --harness codex FORCES wiring (the TOML writer needs no
  codex CLI — the exact no-CLI box the issue filed); detection heuristics
  gate only the `all` default.
- X2: --source now reaches the mint as a scalar write-floor grant; it was
  written to the receipt and hook env but never scoped the token.
- X3: re-runs converge — writeClaudeHooksAt strips our marker across ALL
  events before wiring the requested subset (--no-capture now unwires
  Stop/SessionEnd), and apply unwires prior-receipt targets the new plan
  drops (changed --project sets no longer strand live wiring).
- X4: token.previous_ids is an array — a failed rotation accumulates every
  unrevoked id and the next converge (or --remove) revokes them ALL; the
  --token lane carries them too.
- X5: real rollback — the previous claude registration (url + bearer from
  mcp get) is restored on add-failure or failed smoke, and the codex .bak
  is restored on failed smoke, so "old clients keep working" is true in
  the registration sense, not just the token sense.
- X6: the receipt guard + write-ahead write now precede the mint — a crash
  or newer-format refusal can no longer strand an unrecorded live token.
- X7: consent copy tells the truth — supplied tokens are "written only
  into the host registrations", and the reach paragraph matches the actual
  harness/hook/capture selection.
- X8: a pre-existing permissions.allow entry is recorded as pre-existing
  and never deleted by remove.
- X9: codex wiring prints the experimental_use_rmcp_client advisory.
- X10: an unknown-tool tool_error counts as verified (auth + dispatch
  succeeded) — a --surface verbs serve is no longer declared broken by
  smoke or --status.
- X11: user-scope writes run under a config-dir lock and fresh files are
  created 0600.
- X12: --status is genuinely read-only (no home mkdir, no lock).
- X13: registrar mode (non-loopback --url + --token) wires MCP only —
  hooks talk to the LOCAL brain and would split-brain the box; the http-
  bearer warning is no longer discarded.
- X14: flag parsing fails closed (missing values, --url+--port,
  --status+--remove all error instead of resolving by precedence).

12 new serial cases pin the batch; 463 wave tests green incl. the live e2e.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(harness): ship-review hardening — pre-landing review army + coverage/plan audits (#4043)

Review fixes (6 specialists + coverage + plan-completion audits at ship):
- Smoke-fail rollback symmetry: a FRESH claude registration is removed on a
  failed smoke (previously only replacements were restored); an unrecoverable
  replacement fails the target honestly instead of staying green.
- [X14] --project with a missing/flag-like value errors instead of silently
  widening hook wiring to user scope; auth create --scopes/--takes-holders
  missing values error instead of minting a grandfathered full-access token.
- normalizeTokenScopes fails CLOSED on representation drift: only never-written
  NULL grandfathers; undecoded '{a,b}' array-literal strings parse; any other
  non-null shape denies.
- set-takes-holders merge guards the left operand with jsonb_typeof so
  historically damaged (scalar/array) permissions rows repair on edit instead
  of compounding into a jsonb array; e2e updates pin the REAL auth.ts SQL
  shape + the source_id-survives-merge regression.
- codexBlockOwnsName scopes the name check to INSIDE the managed block and
  shares the writer's marker constants (parseCodexBlockBearer too).
- Refuse-rather-than-guess on unverifiable URLs: --remove and stale-target
  cleanup skip claude registrations whose URL cannot be parsed.
- [X11] parity: codex config.toml writes/removes serialize under a lock on
  the config's own dir; runUninstall takes the HOME lock around harness
  removal (same key as runHarness).
- [D12] the harness-lane hook yield guard also honors the committed
  .claude/settings.json carrier — checking only settings.local.json would
  double-fire events owned by the committed carrier.
- token-mint uses isUndefinedColumnError (message-shaped variants included);
  TOKEN_ID_RE shared with the auth revoke --id CLI gate.
- Stale 'Codex has no hook system' copy in the real-codex e2e + bootstrap
  guide updated to the honest 'gbrain does not wire Codex hooks yet'.
- New tests: runUninstall harness-first composition (harness-only box +
  abort-before-teardown), DATABASE_URL-gated mintLegacyToken Postgres parity,
  renderTokenScopes, isServeOlderThanScopes matrix, codexBlockOwnsName,
  fresh/unrecoverable smoke-fail rollback, committed-carrier yield.
- TODOS: smoke identity-verification hardening, lock.ts message polish,
  auth-create/doctor dedupe follow-ups; docs: binary-downgrade scoping note.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(harness): adversarial-review hardening — codex + red-team convergence at ship (#4043)

Cross-model adversarial passes (Codex exec + red-team subagent) on top of the
review-army batch; three-reviewer convergence upgraded the loopback-impostor
class from TODO to fixed:

- CANARY before the smoke: a random same-format bearer must FAIL auth before
  the real token is verified — an impostor squatting the loopback port cannot
  distinguish the canary from the real token, so it is caught whichever way
  it answers; on ANY failed smoke the fresh mint is revoked immediately
  (nothing live is ever left with an unverified endpoint).
- Pre-approval integrity: the permissions.allow entry is gated on the MCP
  registration actually landing (a failed/ownership-refused registration must
  not bless a foreign server) and is rolled back with a failed smoke.
- --status recovers a bearer ONLY from a registration whose URL matches the
  receipt ([C8] everywhere) — never transmits another install's credential.
- Half-removed receipts (zero targets, minted token awaiting deferred revoke)
  FAIL doctor + exit 1 from --status instead of reading vacuously green;
  --status also exits 1 on failed/pending targets and unconverged rotations.
- [X3] stale-target cleanup deferred until AFTER the smoke passes (mint-first
  applies to removals too: a mint/lock failure no longer strands a box that
  had working wiring); stale-remove exit codes checked, not assumed.
- oauth-provider's pre-v38 fallback SELECT keeps the ORIGINAL-schema scopes
  column — a failed permissions projection no longer grandfathers scoped
  tokens to full admin.
- Hook yield guard PARSES settings and requires a live bootstrap-v1 entry for
  THIS event (both carriers) — a repo committing marker-lookalike strings can
  no longer disable the machine-wide capture lane, and unwired events run.
- permissions writers fail closed on policy shapes they don't understand;
  auth list renders through the SAME normalizer the verify path uses;
  isServeOlderThanScopes pins the first scope-aware release (no cry-wolf on
  the next CLI bump); harness receipts shape-validate before consumers
  dereference; rollback bearers are validated before re-registration.
- [X11] lock parity on every remaining path: removeHarness host-removals,
  stale cleanup, codex rollback; runUninstall holds the HOME lock across the
  whole teardown (no mint window between harness removal and rm of
  <home>/bootstrap); --project hook writes carry the [D12] committed-carrier
  events; cross-home user-scope-vs-project double-fire refused where knowable.
- applyHarness --json emits ONLY the JSON document on stdout (prose → stderr).
- Registry prose-bleed swept (a comment's literal typo'd flag would have made
  the typo VALID); TODOS updated: canary+fresh-revoke landed, identity
  comparison + orphan-mint reconciliation filed as residuals.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(release): sync harness docs with the shipped adversarial-review behavior (v0.45.14.0)

/document-release drift pass against the final #4043 diff:
- KEY_FILES.md: drop the stale duplicate src/commands/hook.ts entry left by
  the mid-wave append; the surviving entry now describes the per-event yield
  guard that parses BOTH workspace settings carriers. harness.ts entry gains
  the canary-gated smoke + symmetric rollback (fresh mint revoked on any
  failed smoke), the registration-gated pre-approval, post-smoke stale
  cleanup, the --status exit contract (incl. half-removed receipts), the
  URL-matched bearer recovery, the SCOPES_MIN_SERVE_VERSION pin, apply
  --json stdout-only discipline, and codex config-dir lock parity. hooks.ts
  entry notes the permissions writers fail closed on policy shapes they
  don't understand.
- docs/guides/bootstrap.md: the mint-first bullet states the failed-smoke
  rollback + fresh-mint retirement guarantee; the --status bullet spells out
  the cron exit contract and the install-level --json contract.
- README.md: the bootstrap guide link mentions local harness mode.
- CHANGELOG.md: restore the blank line before the 0.45.12.0 header
  (formatting only; no entry content touched).
- llms bundles regenerated (bun run build:llms; freshness test green).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(release): cross-model doc-review fixes — honest scoping + two filed residuals (v0.45.14.0)

Independent doc-review pass (Claude subagent; codex session init failed on an
unrelated local MCP timeout) against the shipped #4043 diff. Verified 10
findings against the code; applied 8, filed 2:

- bootstrap.md + KEY_FILES.md: scope the --status bearer-recovery URL-match
  claim to the Claude Code lane (the codex managed block is read at the
  receipt-recorded path; its url key is not yet compared — filed in TODOS);
  add the no-install exit-0/plain vs exit-2/--json distinction and the
  unreachable-serve / failed-verify exit-1 trips; document the wider flag
  surface + registrar mode (--url/--port/--force/--name/--no-hooks); state
  that a supplied --token is never revoked by --remove or rotation.
- KEY_FILES.md: permissions-writer clause corrected — the add path fails
  closed on alien policy shapes, removal leaves what it can't read untouched;
  token-mint.ts entry gains the required takesHolders option and TOKEN_ID_RE.
- DEPLOY.md: revoke-by-name hits every same-name token; auth revoke --id +
  the id/scopes columns in auth list are the precise path.
- RESOLVER.md: restore an "install gbrain into this agent workspace" trigger.
- CHANGELOG (factual drift only): damaged permissions rows are reset to a
  clean object, not recovered; scope-display claim narrowed to the verify +
  CLI display paths.
- TODOS.md: filed the codex-lane [C8] URL-match residual and the admin
  dashboard scope-display normalizer residual.
- llms bundles regenerated; guards green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(harness): close the two doc-review residuals — codex-lane bearer ownership + dashboard scope honesty (#4043)

- parseCodexBlockBearer takes an expectedUrl: --status only recovers the
  managed block's bearer when the block's url matches the receipt ([C8]
  parity with the claude lane — two GBRAIN_HOMEs sharing the one user-global
  codex config could otherwise hand install A install B's credential).
- The admin dashboard renders legacy-token scopes through the SAME
  normalizeTokenScopes the verify path uses (NULL = grandfathered full
  access; damaged/deny rows show what the serve actually enforces) instead
  of raw array_to_string.
- Both TODOS residual entries removed (fixed, not filed).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(skills): regenerate skills.lock.json — RESOLVER.md + setup/SKILL.md edits from the #4043 wave

The wave's skill-routing updates (harness routing row in RESOLVER.md, the
harness pointer in setup/SKILL.md) landed without the manifest-lock chaser;
CI's check:skills-manifest caught the stale hashes. Full verify suite green
locally (39/39).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 11:51:01 -07:00
300 changed files with 33921 additions and 2461 deletions
+5
View File
@@ -6,6 +6,11 @@
DATABASE_URL=postgresql://postgres:postgres@localhost:5433/gbrain_test
# Option B: Real Supabase instance (tests the actual production path)
# DATABASE_URL=postgresql://postgres.[project-ref]:[password]@aws-0-us-east-1.pooler.supabase.com:6543/postgres
# NOTE (#3485): destructive tests enforce a database-name floor — the name must
# carry "test" as a word segment (gbrain_test passes; Supabase's default
# "postgres" does not). Use a dedicated test project/database, or opt the exact
# name in one-shot: GBRAIN_E2E_ALLOW_DB=postgres bun run test:e2e
# (never a shell-profile export — that would permanently disarm the floor).
# Tier 2 (required for skill tests, optional for mechanical tests)
OPENAI_API_KEY=sk-...
+13 -2
View File
@@ -61,10 +61,15 @@ jobs:
- name: Run JSONB double-encode parity tests on real Postgres
env:
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/gbrain_test
# #3485 preload guard: this job intentionally tests against a DB.
GBRAIN_TEST_ALLOW_DATABASE_URL: '1'
# --timeout also raises bun's 5s default hook budget (beforeAll/afterAll
# do NOT inherit a test's third-arg timeout; verified on bun 1.3.x).
# Every runner script in scripts/ passes it; bare invocations must too.
run: bun test --timeout=60000 test/e2e/op-checkpoint-jsonb-parity.test.ts test/e2e/jsonb-roundtrip.test.ts
# phantom-redirect rides this job: its Postgres arm is the other
# engine-parity backstop and no other CI lane carries DATABASE_URL to it
# (the unit wrappers strip the URL per #3485).
run: bun test --timeout=60000 test/e2e/op-checkpoint-jsonb-parity.test.ts test/e2e/jsonb-roundtrip.test.ts test/phantom-redirect-engine-parity.test.ts
tier1:
name: Tier 1 (Mechanical)
@@ -91,9 +96,13 @@ jobs:
bun-version: 1.3.13
- run: bun install
- name: Run Tier 1 E2E tests
run: bun test --timeout=60000 test/e2e/mechanical.test.ts test/e2e/mcp.test.ts
# job-isolation rides tier1 deliberately: e2e.yml runs only explicitly
# NAMED files (no glob) — an unwired e2e file is silent coverage loss.
run: bun test --timeout=60000 test/e2e/mechanical.test.ts test/e2e/mcp.test.ts test/e2e/job-isolation.test.ts
env:
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/gbrain_test
# #3485 preload guard: this job intentionally tests against a DB.
GBRAIN_TEST_ALLOW_DATABASE_URL: '1'
tier2:
name: Tier 2 (LLM Skills)
@@ -161,6 +170,8 @@ jobs:
run: bun test --timeout=60000 test/e2e/skills.test.ts test/e2e/zeroentropy-live.test.ts
env:
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/gbrain_test
# #3485 preload guard: this job intentionally tests against a DB.
GBRAIN_TEST_ALLOW_DATABASE_URL: '1'
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
# v0.33.3.0: ZE live API tests skip gracefully when this is unset,
+541 -10
View File
@@ -18,6 +18,11 @@ on:
# label so we don't fan out on unrelated label changes.
types: [labeled, synchronize, reopened]
workflow_dispatch:
inputs:
run_grok_door:
description: 'Run the grok-door job (pre-secret posture: label or this input only)'
type: boolean
default: false
permissions:
contents: read
@@ -64,6 +69,8 @@ jobs:
- name: Run heavy tests
env:
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/gbrain_test
# #3485 preload guard: this job intentionally tests against a DB.
GBRAIN_TEST_ALLOW_DATABASE_URL: '1'
run: bun run test:heavy
# Agent-bootstrap offline Docker e2e [A7] — placeholder registration.
@@ -102,8 +109,9 @@ jobs:
retention-days: 14
if-no-files-found: ignore
# Real-agent door e2e: drives the ACTUAL `claude` + `codex` + `hermes`
# binaries (no PATH shims) against a real gbrain over MCP. These pay real API
# Real-agent door e2e: drives the ACTUAL `claude` + `codex` + `hermes` +
# `grok` + `opencode` binaries (no PATH shims) against a real gbrain over
# MCP. These pay real API
# cost and need the binaries installed + authed, which a stock GitHub runner
# does NOT have — so the tests self-SKIP (describe.skipIf on binary/auth) and
# the job is a clean no-op here. It exists so a self-hosted /
@@ -120,11 +128,23 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 20
env:
# Open the hermes opt-in door here so binary/auth absence — not the
# opt-in var — is what skips (same posture as the claude/codex doors).
# Open the hermes/grok opt-in doors here so binary/auth absence — not
# the opt-in var — is what skips (same posture as the claude/codex
# doors). The grok door's keyless tier additionally self-skips without
# a grok binary, which a stock runner does not have.
GBRAIN_REAL_HERMES_E2E: '1'
GBRAIN_REAL_GROK_E2E: '1'
GBRAIN_REAL_OPENCODE_E2E: '1'
# Pin so a provisioned runner's grok version-shape test asserts against
# the supported version (and a colliding community `grok` binary fails
# loud instead of running the keyless tier confusingly).
GROK_VERSION: "1.0.4"
# Same posture for opencode: a provisioned runner's version pin.
OPENCODE_VERSION: "1.18.18"
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
persist-credentials: false
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.3.13
@@ -139,7 +159,9 @@ jobs:
for f in \
test/e2e/bootstrap-real-claude.serial.test.ts \
test/e2e/bootstrap-real-codex.serial.test.ts \
test/e2e/install-real-hermes.serial.test.ts; do
test/e2e/install-real-hermes.serial.test.ts \
test/e2e/install-real-grok.serial.test.ts \
test/e2e/install-real-opencode.serial.test.ts; do
[ -f "$f" ] && files+=("$f")
done
if [ "${#files[@]}" -eq 0 ]; then
@@ -178,10 +200,12 @@ jobs:
HERMES_VERSION: "0.20.0"
HERMES_GIT_TAG: "v2026.8.3"
HERMES_GIT_COMMIT: "3c27eb6234bf91b8ceee9e9071591b31e9b148cb"
HERMES_INSTALL_SHA256: "c118ff31618dc70339049ce71061b8f1351a1c70d9c2a236ed50d8a2550c550d"
HERMES_INSTALL_SHA256: "868ed3a91e0fabbff6d7418b3ede82bf4833652ec4e77196a42852fb35a9e5b9"
GBRAIN_REAL_HERMES_E2E: '1'
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
persist-credentials: false
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.3.13
@@ -275,11 +299,13 @@ jobs:
EXIT=0
bun test --timeout=600000 test/e2e/install-real-hermes.serial.test.ts > door.txt 2>&1 || EXIT=$?
tail -40 door.txt
# Preserve the FULL bun output UNCONDITIONALLY (upload stays
# failure-gated): the zero-pass failure class below exits with the
# summary as its only trace, and bun prints failure details before
# the summary, so the 40-line tail can drop exactly the lines a
# paid-CI triage needs.
cp door.txt "$GBRAIN_E2E_EVIDENCE_DIR/" 2>/dev/null || true
if [ "$EXIT" -ne 0 ]; then
# Preserve the FULL bun output for the failure artifact — bun
# prints failure details before the summary, so the 40-line tail
# above can drop exactly the lines a paid-CI triage needs.
cp door.txt "$GBRAIN_E2E_EVIDENCE_DIR/" 2>/dev/null || true
exit "$EXIT"
fi
# This job provisions the binary + auth above, so the door must
@@ -334,3 +360,508 @@ jobs:
- name: Remove hermes credentials (unconditional)
if: always()
run: rm -f ~/.hermes/.env
# Grok door e2e (xAI Grok Build): PROVISIONS the real grok binary via the
# pinned npm package (registry integrity verified — stronger than the
# curl-installer path; both pins live in docs/mcp/GROK-CLI-PIN.md, enforced
# against this file by scripts/check-grok-pin.sh in `bun run verify`).
# KEYLESS-FIRST ordering (deliberate divergence from hermes-door): grok's
# mcp add/list/doctor run keyless, so the compat tier runs and banks its
# coverage BEFORE the secret precondition — a missing XAI_API_KEY still
# fails this job loudly, but only after the free tier proved the install
# surface, so pre-secret runs are diagnostic instead of pure red.
#
# PRE-SECRET GATING POSTURE: `real-agent-e2e` label or the run_grok_door
# dispatch input ONLY — deliberately NOT `schedule` and NOT the generic
# `heavy-tests` label, so an absent XAI_API_KEY secret cannot paint nightly
# heavy runs (or unrelated heavy-labeled PRs) red. The commit that lands
# AFTER an admin creates the XAI_API_KEY secret (an external prerequisite,
# not a code change) re-adds: the schedule leg, the heavy-tests label leg,
# a default-on dispatch, and a latest-version canary matrix leg
# (continue-on-error, schedule-scoped, own timeout) so the pinned lane
# stays deterministic while the canary tracks what users actually run.
grok-door:
name: Grok door e2e (real binary, keyless-first)
if: |
(github.event_name == 'pull_request' &&
contains(github.event.pull_request.labels.*.name, 'real-agent-e2e')) ||
(github.event_name == 'workflow_dispatch' &&
inputs.run_grok_door == true)
runs-on: ubuntu-latest
# Measured local door wall-time: keyless tier ~29s + one-time compiled
# gbrain build (~2-4 min) + npm install (~10s); paid SMOKE turn budget
# 2 x 240s. 20 min = measured + >50% headroom (GROK-CLI-PIN.md).
timeout-minutes: 20
env:
# Pin values documented in docs/mcp/GROK-CLI-PIN.md — update them
# together, deliberately, after reviewing upstream changes
# (scripts/check-grok-pin.sh fails `bun run verify` on drift).
GROK_VERSION: "1.0.4"
GROK_NPM_PACKAGE: "@xai-official/grok"
GROK_NPM_INTEGRITY: "sha512-Nu3SFXTqwvCQr/LQFwrQYgngJhUQwX2h9ZSgzW4HowidjbPBWtMVO0xI88d2z6/zlDSNaT5YP/uk+2DthKQMsg=="
# Per-platform payload pins: the wrapper's integrity covers only the
# wrapper tarball; the binary that EXECUTES is the platform sub-package.
GROK_NPM_LINUX_X64_INTEGRITY: "sha512-Dan2LfKcFBiabuDGHaGgMT8Ndzibo2ljvSjh4MlpV5117JL+S/0KMbdyYpk+13d7t+4znniW1cm+rRwUGSAvtw=="
GROK_NPM_LINUX_ARM64_INTEGRITY: "sha512-zGK42Eq3ZmIa7cSVnl6CiJ4cxTCMsNLQCmCoLJhy5eZXfAvZ1DA3K3HXmKCj4OScX8SalYlp7mx8HWl9Y6gytw=="
GBRAIN_REAL_GROK_E2E: '1'
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
persist-credentials: false
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.3.13
- run: bun install
- name: Prepare evidence dir
run: |
echo "GBRAIN_E2E_EVIDENCE_DIR=$RUNNER_TEMP/grok-door-evidence" >> "$GITHUB_ENV"
mkdir -p "$RUNNER_TEMP/grok-door-evidence"
# Compile gbrain ONCE for both bun test invocations below —
# ensureCompiledGbrain's cache is per-process, so without this the
# keyless and paid runs each pay the 2-4 min compile.
- name: Build gbrain (compile once for both door runs)
run: |
bun build --compile --outfile "$RUNNER_TEMP/gbrain-door-bin" src/cli.ts
echo "GBRAIN_COMPILED_BIN=$RUNNER_TEMP/gbrain-door-bin" >> "$GITHUB_ENV"
# SECRETLESS provisioning: the npm registry verifies the per-platform
# payload against its integrity metadata; the pre-check pins that the
# registry still serves the SAME integrity we observed (a re-published
# 1.0.4 becomes a loud re-pin decision, not silently different code
# running next to secrets in later steps). Version assert lives here
# too — before any secret-bearing step.
- name: Install grok (pinned npm package)
timeout-minutes: 10
run: |
served=$(npm view "$GROK_NPM_PACKAGE@$GROK_VERSION" dist.integrity 2>/dev/null || true)
if [ "$served" != "$GROK_NPM_INTEGRITY" ]; then
echo "::error::grok npm integrity drift for $GROK_NPM_PACKAGE@$GROK_VERSION — registry serves '$served', pinned '$GROK_NPM_INTEGRITY'. Re-pin deliberately: update the stamps in docs/mcp/GROK-CLI-PIN.md + this workflow after reviewing upstream (see the pin doc's re-observation checklist)." >&2
exit 1
fi
# The platform sub-package is the binary that actually runs — pin it
# too (per-arch; ubuntu-latest is x64 today, arm64 pinned for a
# future runner switch).
arch=$(uname -m)
case "$arch" in
x86_64) plat_pkg="$GROK_NPM_PACKAGE-linux-x64"; plat_pin="$GROK_NPM_LINUX_X64_INTEGRITY" ;;
aarch64|arm64) plat_pkg="$GROK_NPM_PACKAGE-linux-arm64"; plat_pin="$GROK_NPM_LINUX_ARM64_INTEGRITY" ;;
*) echo "::error::unsupported runner arch for the grok payload pin: $arch" >&2; exit 1 ;;
esac
plat_served=$(npm view "$plat_pkg@$GROK_VERSION" dist.integrity 2>/dev/null || true)
if [ "$plat_served" != "$plat_pin" ]; then
echo "::error::grok platform payload integrity drift for $plat_pkg@$GROK_VERSION — registry serves '$plat_served', pinned '$plat_pin'. Re-pin deliberately (GROK-CLI-PIN.md stamps + this workflow)." >&2
exit 1
fi
npm install -g "$GROK_NPM_PACKAGE@$GROK_VERSION"
if ! command -v grok >/dev/null 2>&1; then
echo "::error::grok did not resolve on PATH after npm install" >&2
exit 1
fi
version_output=$(grok --version)
echo "$version_output"
# Observed shape: `grok 1.0.4 (buildhash)` (GROK-CLI-PIN.md).
if ! printf '%s' "$version_output" | grep -qF "grok $GROK_VERSION"; then
echo "::error::grok version drift — expected 'grok $GROK_VERSION' in: $version_output (see docs/mcp/GROK-CLI-PIN.md triage table)" >&2
exit 1
fi
# KEYLESS TIER FIRST — banks the free compat coverage (documented-shape
# registration, TOML schema, doctor handshake proving the seven-verb
# surface) before anything needs the secret. XAI_API_KEY is absent from
# this step by construction, so the paid describe self-skips.
- name: Run grok door tests (keyless tier)
run: |
EXIT=0
bun test --timeout=600000 test/e2e/install-real-grok.serial.test.ts > door-keyless.txt 2>&1 || EXIT=$?
tail -40 door-keyless.txt
cp door-keyless.txt "$GBRAIN_E2E_EVIDENCE_DIR/" 2>/dev/null || true
if [ "$EXIT" -ne 0 ]; then
exit "$EXIT"
fi
# Exact expected shape for this tier: 4 keyless tests pass, the
# 1 paid test skips. Zero-pass or partial-pass refuses green.
pass_count=$(grep -Eo '[0-9]+ pass' door-keyless.txt | tail -1 | grep -Eo '^[0-9]+' || true)
if [ -z "$pass_count" ] || [ "$pass_count" -lt 4 ]; then
echo "::error::grok door keyless tier expected 4 passing tests, summary shows '${pass_count:-none}' — refusing to go green (see docs/mcp/GROK-CLI-PIN.md triage table)" >&2
exit 1
fi
- name: Preconditions (secret present)
env:
XAI_API_KEY: ${{ secrets.XAI_API_KEY }}
run: |
if [ -z "$XAI_API_KEY" ]; then
echo "::error::XAI_API_KEY secret is empty — the keyless tier above already ran (its coverage is banked); the paid SMOKE needs the secret. Admin: create the XAI_API_KEY repo/environment secret (console.x.ai), then re-run. Fork PRs get no secrets from GitHub." >&2
exit 1
fi
# Named bad-key preflight: key-rot fails HERE, at a step named for it,
# instead of surfacing as a confusing SMOKE failure (GROK-CLI-PIN.md
# triage table). One minimal paid probe.
- name: Auth preflight (bad-key tripwire)
env:
XAI_API_KEY: ${{ secrets.XAI_API_KEY }}
run: |
export GROK_HOME="$RUNNER_TEMP/grok-preflight-home"
mkdir -p "$GROK_HOME"
# Verbatim kill-switch homes (update together): seedGrokConfig in
# test/helpers/agent-harness.ts and scenarioGrokInstall in
# scripts/dx-explore.ts.
printf '[cli]\nauto_update = false\n' > "$GROK_HOME/config.toml"
# This step runs the third-party agent binary directly: never hand
# it the WRITABLE step-metadata files (appending to GITHUB_ENV/PATH
# poisons the later secret-bearing steps — the same channel
# grokChildEnv scrubs for test-spawned children), and kill its web
# tools like the door SMOKE does.
out=$(env -u GITHUB_ENV -u GITHUB_PATH -u GITHUB_OUTPUT -u GITHUB_STATE -u GITHUB_STEP_SUMMARY \
grok -p "reply with exactly: PREFLIGHT-OK" --output-format plain --disable-web-search 2>&1) || {
echo "::error::grok auth preflight failed — the XAI_API_KEY secret is present but rejected (rotate it at console.x.ai; see GROK-CLI-PIN.md triage table). Output: ${out:0:300}" >&2
exit 1
}
echo "auth preflight ok"
- name: Run grok door tests (full — paid SMOKE included)
env:
XAI_API_KEY: ${{ secrets.XAI_API_KEY }}
run: |
EXIT=0
bun test --timeout=600000 test/e2e/install-real-grok.serial.test.ts > door.txt 2>&1 || EXIT=$?
tail -40 door.txt
cp door.txt "$GBRAIN_E2E_EVIDENCE_DIR/" 2>/dev/null || true
if [ "$EXIT" -ne 0 ]; then
exit "$EXIT"
fi
# PAID-SENTINEL: with the key present, a skipping paid tier must
# never read as green (the split-gating false-green class). The
# grep target is the suite's literal skip log — mirrored in
# test/e2e/install-real-grok.serial.test.ts (change together).
if grep -q 'SKIP paid tier' door.txt; then
echo "::error::grok door paid tier skipped despite a present XAI_API_KEY — hasGrokAuth() gate drift; refusing to go green" >&2
exit 1
fi
pass_count=$(grep -Eo '[0-9]+ pass' door.txt | tail -1 | grep -Eo '^[0-9]+' || true)
if [ -z "$pass_count" ] || [ "$pass_count" -lt 5 ]; then
echo "::error::grok door full run expected 5 passing tests (6 once the JSON tool-call test lands), summary shows '${pass_count:-none}'" >&2
exit 1
fi
# Auto-update tripwire: the seeded [cli] auto_update=false is the whole
# kill-switch (no env form observed) — a version that MOVED mid-job
# means it failed and the pins above are no longer what just ran.
- name: Version re-check (mid-job drift tripwire)
if: always()
run: |
if command -v grok >/dev/null 2>&1; then
version_output=$(grok --version || true)
if ! printf '%s' "$version_output" | grep -qF "grok $GROK_VERSION"; then
echo "::error::grok version moved mid-job — auto-update kill-switch failed (expected 'grok $GROK_VERSION', got: $version_output)" >&2
exit 1
fi
fi
- name: Scrub credentials from evidence (defensive)
if: failure() && env.GBRAIN_E2E_EVIDENCE_DIR != ''
env:
XAI_API_KEY: ${{ secrets.XAI_API_KEY }}
run: |
# Same triple as hermes-door: filenames, symlinks, content. Auth is
# env-only here (the job never writes the key to disk — stronger
# than the hermes .env posture), so the content grep is the layer
# that matters for grok-written logs on the failure path.
find "$GBRAIN_E2E_EVIDENCE_DIR" -type f \( -name '.env' -o -name '*.env' -o -name 'mcp_credentials.json' \) -exec rm -f {} + 2>/dev/null || true
find "$GBRAIN_E2E_EVIDENCE_DIR" -type l -delete 2>/dev/null || true
if [ -n "$XAI_API_KEY" ]; then
grep -rlF "$XAI_API_KEY" "$GBRAIN_E2E_EVIDENCE_DIR" 2>/dev/null | while IFS= read -r f; do
echo "::warning::removing evidence file containing the API key: ${f#"$GBRAIN_E2E_EVIDENCE_DIR"/}" >&2
rm -f "$f"
done
fi
- name: Upload grok door evidence
if: failure() && env.GBRAIN_E2E_EVIDENCE_DIR != ''
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: grok-door-evidence
path: ${{ env.GBRAIN_E2E_EVIDENCE_DIR }}
retention-days: 14
if-no-files-found: ignore
# Auth travels env-only, but grok MAY persist derived credentials after
# an authed turn (the inventory is pending — GROK-CLI-PIN.md); remove
# the known candidate unconditionally so nothing outlives the job even
# on a future self-hosted runner.
- name: Remove grok credentials (unconditional)
if: always()
run: |
rm -f ~/.grok/mcp_credentials.json
rm -rf "$RUNNER_TEMP/grok-preflight-home"
# Cancellation/timeout bypasses the suite's afterAll cleanup — the
# hermetic homes carry no key file (env-only auth) but may hold
# grok-derived credentials once the authed inventory lands.
rm -rf /tmp/gb-grok-* 2>/dev/null || true
# opencode door e2e (SST opencode): PROVISIONS the real opencode binary via
# the pinned npm package (wrapper + per-platform payload integrities
# verified — both pins live in docs/mcp/OPENCODE-CLI-PIN.md, enforced
# against this file by scripts/check-opencode-pin.sh in `bun run verify`).
#
# DAY-ONE FULL POSTURE (a step past grok's pre-secret gating, deliberate):
# opencode's anonymous free tier drives MCP tool calls keyless (observed,
# load-bearing — OPENCODE-CLI-PIN.md §One-shot), so the ENTIRE core door —
# including the nonce SMOKE — runs with no secret; and the paid anthropic
# leg rides the ANTHROPIC_API_KEY secret that already exists (hermes-door
# consumes it). So this job takes the hermes-door triggers (nightly +
# labels + dispatch, cadence policy: nightly for the NEWEST door agent)
# with grok-door's internals (keyless-first ordering, secretless pinned
# provisioning, sentinels, scrub triple, unconditional credential removal).
# No dedicated dispatch input: any workflow_dispatch already passes the
# non-PR arm, so an input would be dead yaml.
opencode-door:
name: opencode door e2e (real binary, keyless SMOKE)
if: |
github.event_name != 'pull_request' ||
contains(github.event.pull_request.labels.*.name, 'real-agent-e2e') ||
contains(github.event.pull_request.labels.*.name, 'heavy-tests')
runs-on: ubuntu-latest
# Measured local door wall-time: full 6-test run 35.8s + one-time
# compiled gbrain build (~2-4 min) + npm install (~15s); free-tier +
# paid turn budgets 2 x 240s each. 20 min = measured + >50% headroom.
timeout-minutes: 20
env:
# Pin values documented in docs/mcp/OPENCODE-CLI-PIN.md — update them
# together, deliberately, after reviewing upstream changes
# (scripts/check-opencode-pin.sh fails `bun run verify` on drift).
OPENCODE_VERSION: "1.18.18"
OPENCODE_NPM_PACKAGE: "opencode-ai"
OPENCODE_NPM_INTEGRITY: "sha512-J+5HFq8tf+wPBBpBpMPSNjSytF2/EkNWYfFZh4si1d9auFbQriqDyqZv+vFUsLWERfdMU32Eajwuiq3rKBvZLQ=="
# Per-platform payload pins: the wrapper's integrity covers only the
# wrapper tarball; the binary that EXECUTES is the platform sub-package.
OPENCODE_NPM_LINUX_X64_INTEGRITY: "sha512-WmeUnhljYJ252wywKTiW4bNDzsas2njpjPUEh0jM6HKNI4vFxJtREtzaWViY4AKEAcOkLWT8Ll17ixvcHz3AnA=="
OPENCODE_NPM_LINUX_ARM64_INTEGRITY: "sha512-e8D3g0qJEIzawEg2+ygW3vkZjAYL2ssyAx4GbihjwXwZFvlZZy5zRWWzdz5KLBoHSTl0FB73vNtnNeXONyHpVQ=="
GBRAIN_REAL_OPENCODE_E2E: '1'
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
persist-credentials: false
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.3.13
- run: bun install
- name: Prepare evidence dir
run: |
echo "GBRAIN_E2E_EVIDENCE_DIR=$RUNNER_TEMP/opencode-door-evidence" >> "$GITHUB_ENV"
mkdir -p "$RUNNER_TEMP/opencode-door-evidence"
# Compile gbrain ONCE for both bun test invocations below.
- name: Build gbrain (compile once for both door runs)
run: |
bun build --compile --outfile "$RUNNER_TEMP/gbrain-door-bin" src/cli.ts
echo "GBRAIN_COMPILED_BIN=$RUNNER_TEMP/gbrain-door-bin" >> "$GITHUB_ENV"
# SECRETLESS provisioning, pack-verify-install: `npm pack` DOWNLOADS
# each artifact and reports the integrity of the BYTES it wrote, so the
# asserts below cover the tarballs actually held — closing the
# view-then-install TOCTOU (two registry round-trips a payload-swapping
# registry could split). The wrapper then installs FROM the verified
# local tarball, not a fresh registry resolve of the name. Payload
# resolution, honestly: that install still fetches the platform
# sub-package (opencode-linux-*) over the network; after the pack step
# byte-confirms the registry's payload artifact matches its pin, npm
# validates the install-time fetch against the same packument
# integrity. No --ignore-scripts: opencode-ai's postinstall places the
# platform binary (verified locally — with the flag the CLI refuses to
# run). Version assert lives here too — before any secret-bearing step.
- name: Install opencode (pinned npm package, pack-verify-install)
timeout-minutes: 10
run: |
packdir=$(mktemp -d)
read_integrity() {
node -e 'let d;try{d=JSON.parse(require("fs").readFileSync(0,"utf8"))}catch{d=[]}process.stdout.write((Array.isArray(d)&&d[0]&&d[0].integrity)||"")'
}
pushd "$packdir" >/dev/null
served=$(npm pack "$OPENCODE_NPM_PACKAGE@$OPENCODE_VERSION" --json 2>/dev/null | read_integrity || true)
if [ "$served" != "$OPENCODE_NPM_INTEGRITY" ]; then
echo "::error::opencode npm integrity drift for $OPENCODE_NPM_PACKAGE@$OPENCODE_VERSION — packed tarball integrity '$served', pinned '$OPENCODE_NPM_INTEGRITY'. Re-pin deliberately: update the stamps in docs/mcp/OPENCODE-CLI-PIN.md + this workflow after reviewing upstream (see the pin doc's re-observation checklist)." >&2
exit 1
fi
arch=$(uname -m)
case "$arch" in
x86_64) plat_pkg="opencode-linux-x64"; plat_pin="$OPENCODE_NPM_LINUX_X64_INTEGRITY" ;;
aarch64|arm64) plat_pkg="opencode-linux-arm64"; plat_pin="$OPENCODE_NPM_LINUX_ARM64_INTEGRITY" ;;
*) echo "::error::unsupported runner arch for the opencode payload pin: $arch" >&2; exit 1 ;;
esac
plat_served=$(npm pack "$plat_pkg@$OPENCODE_VERSION" --json 2>/dev/null | read_integrity || true)
if [ "$plat_served" != "$plat_pin" ]; then
echo "::error::opencode platform payload integrity drift for $plat_pkg@$OPENCODE_VERSION — packed tarball integrity '$plat_served', pinned '$plat_pin'. Re-pin deliberately (OPENCODE-CLI-PIN.md stamps + this workflow)." >&2
exit 1
fi
npm install -g ./opencode-ai-*.tgz
popd >/dev/null
rm -rf "$packdir"
if ! command -v opencode >/dev/null 2>&1; then
echo "::error::opencode did not resolve on PATH after npm install" >&2
exit 1
fi
version_output=$(opencode --version)
echo "$version_output"
# Observed shape: BARE semver (`1.18.18` — no name, no hash); the
# SST-vs-claimant discriminator (OPENCODE-CLI-PIN.md §Pin).
if [ "$(printf '%s' "$version_output" | tr -d '[:space:]')" != "$OPENCODE_VERSION" ]; then
echo "::error::opencode version drift — expected bare '$OPENCODE_VERSION', got: $version_output (see docs/mcp/OPENCODE-CLI-PIN.md triage table)" >&2
exit 1
fi
# KEYLESS TIER FIRST — and on opencode that includes the nonce SMOKE
# (free tier). ANTHROPIC_API_KEY is absent from this step by
# construction, so the paid describe self-skips.
- name: Run opencode door tests (keyless tier — SMOKE included)
run: |
EXIT=0
bun test --timeout=600000 test/e2e/install-real-opencode.serial.test.ts > door-keyless.txt 2>&1 || EXIT=$?
tail -40 door-keyless.txt
cp door-keyless.txt "$GBRAIN_E2E_EVIDENCE_DIR/" 2>/dev/null || true
if [ "$EXIT" -ne 0 ]; then
exit "$EXIT"
fi
# Exact expected shape for this tier: 5 keyless tests pass (T1, T2,
# T2b, T3, T4-SMOKE), the 1 paid test skips. Zero/partial-pass
# refuses green.
pass_count=$(grep -Eo '[0-9]+ pass' door-keyless.txt | tail -1 | grep -Eo '^[0-9]+' || true)
if [ -z "$pass_count" ] || [ "$pass_count" -lt 5 ]; then
echo "::error::opencode door keyless tier expected 5 passing tests, summary shows '${pass_count:-none}' — refusing to go green (see docs/mcp/OPENCODE-CLI-PIN.md triage table)" >&2
exit 1
fi
- name: Preconditions (secret present)
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
run: |
if [ -z "$ANTHROPIC_API_KEY" ]; then
echo "::error::ANTHROPIC_API_KEY secret is empty — the keyless tier above already ran (its coverage, including the SMOKE, is banked); the paid anthropic leg needs the secret hermes-door already consumes. Fork PRs get no secrets from GitHub." >&2
exit 1
fi
# Full run (paid anthropic leg included). The T5 models-gate inside the
# suite is the named bad-pin tripwire: it validates the pinned model id
# against the AUTHED `opencode models` list BEFORE any spend.
- name: Run opencode door tests (full — paid anthropic leg included)
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
run: |
EXIT=0
bun test --timeout=600000 test/e2e/install-real-opencode.serial.test.ts > door.txt 2>&1 || EXIT=$?
tail -40 door.txt
cp door.txt "$GBRAIN_E2E_EVIDENCE_DIR/" 2>/dev/null || true
if [ "$EXIT" -ne 0 ]; then
exit "$EXIT"
fi
# PAID-SENTINEL: with the key present, a skipping paid tier must
# never read as green (the split-gating false-green class). The
# grep target is the suite's literal skip log — mirrored in
# test/e2e/install-real-opencode.serial.test.ts (change together).
if grep -q 'SKIP paid tier' door.txt; then
echo "::error::opencode door paid tier skipped despite a present ANTHROPIC_API_KEY — hasOpencodeAuth() gate drift; refusing to go green" >&2
exit 1
fi
pass_count=$(grep -Eo '[0-9]+ pass' door.txt | tail -1 | grep -Eo '^[0-9]+' || true)
if [ -z "$pass_count" ] || [ "$pass_count" -lt 6 ]; then
echo "::error::opencode door full run expected 6 passing tests, summary shows '${pass_count:-none}'" >&2
exit 1
fi
# Auto-update tripwire: the DOUBLE kill (config seed + env var) is the
# whole defense — a version that MOVED mid-job means it failed and the
# pins above are no longer what just ran.
- name: Version re-check (mid-job drift tripwire)
if: always()
run: |
if command -v opencode >/dev/null 2>&1; then
version_output=$(opencode --version || true)
if [ "$(printf '%s' "$version_output" | tr -d '[:space:]')" != "$OPENCODE_VERSION" ]; then
echo "::error::opencode version moved mid-job — auto-update kill failed (expected '$OPENCODE_VERSION', got: $version_output)" >&2
exit 1
fi
fi
- name: Scrub credentials from evidence (defensive)
if: failure() && env.GBRAIN_E2E_EVIDENCE_DIR != ''
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
run: |
# Same triple as the sibling doors, RE-KEYED for this lane: the
# credential file candidate is opencode's auth.json and the content
# grep sweeps ANTHROPIC_API_KEY (not XAI). Auth is env-only here —
# the content grep is the layer that matters for opencode-written
# logs on the failure path.
find "$GBRAIN_E2E_EVIDENCE_DIR" -type f \( -name '.env' -o -name '*.env' -o -name 'auth.json' \) -exec rm -f {} + 2>/dev/null || true
find "$GBRAIN_E2E_EVIDENCE_DIR" -type l -delete 2>/dev/null || true
if [ -n "$ANTHROPIC_API_KEY" ]; then
grep -rlF "$ANTHROPIC_API_KEY" "$GBRAIN_E2E_EVIDENCE_DIR" 2>/dev/null | while IFS= read -r f; do
echo "::warning::removing evidence file containing the API key: ${f#"$GBRAIN_E2E_EVIDENCE_DIR"/}" >&2
rm -f "$f"
done
fi
- name: Upload opencode door evidence
if: failure() && env.GBRAIN_E2E_EVIDENCE_DIR != ''
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: opencode-door-evidence
path: ${{ env.GBRAIN_E2E_EVIDENCE_DIR }}
retention-days: 14
if-no-files-found: ignore
# Auth travels env-only, but a future login flow would persist
# auth.json — remove the known candidate unconditionally so nothing
# outlives the job even on a future self-hosted runner.
- name: Remove opencode credentials (unconditional)
if: always()
run: |
rm -f ~/.local/share/opencode/auth.json
rm -rf /tmp/gb-opencode-* 2>/dev/null || true
# opencode canary: latest-version leg (schedule-scoped, continue-on-error,
# own timeout — landed IN-WAVE, reversing the grok-style deferral, because
# opencode ships near-continuously and a frozen pin goes stale in weeks;
# the pinned lane above stays the deterministic gate while this tracks
# what users actually run). Keyless tier only (incl. the free-tier SMOKE);
# no secret ever reaches this job. A red here is a PIN-REFRESH SIGNAL
# (OPENCODE-CLI-PIN.md §Pin-refresh cadence), never a gate.
opencode-door-canary:
name: opencode door canary (latest, keyless, non-gating)
if: github.event_name == 'schedule'
runs-on: ubuntu-latest
timeout-minutes: 20
continue-on-error: true
env:
GBRAIN_REAL_OPENCODE_E2E: '1'
# Deliberately NO OPENCODE_VERSION pin: T1 asserts the bare-semver
# SHAPE only, and the suite runs against whatever `latest` is today.
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
persist-credentials: false
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.3.13
- run: bun install
- name: Build gbrain
run: |
bun build --compile --outfile "$RUNNER_TEMP/gbrain-door-bin" src/cli.ts
echo "GBRAIN_COMPILED_BIN=$RUNNER_TEMP/gbrain-door-bin" >> "$GITHUB_ENV"
- name: Install opencode@latest (unpinned — the whole point)
timeout-minutes: 10
run: |
npm install -g opencode-ai@latest
command -v opencode >/dev/null 2>&1
echo "canary version: $(opencode --version)"
- name: Run opencode door tests (keyless tier against latest)
run: |
EXIT=0
bun test --timeout=600000 test/e2e/install-real-opencode.serial.test.ts > door-canary.txt 2>&1 || EXIT=$?
tail -40 door-canary.txt
if [ "$EXIT" -ne 0 ]; then
echo "::warning::opencode canary red against latest — pin-refresh signal (OPENCODE-CLI-PIN.md §Pin-refresh cadence); the pinned lane is the gate."
exit "$EXIT"
fi
+25 -12
View File
@@ -1,16 +1,17 @@
<!-- gbrain-runbook-stamp: 0.45.13.0 -->
<!-- gbrain-runbook-stamp: 0.46.4.0 -->
<!-- This stamp must equal the VERSION file at every release; CI enforces it
(scripts/check-bootstrap-tag.sh). `gbrain bootstrap status` compares it to
the installed binary and warns on skew. -->
# BOOTSTRAP_FOR_AGENTS.md — become your human's persistent agent
You are an AI agent (Claude Code or Codex) reading this because a human pasted a
block asking you to set yourself up as their persistent personal agent, with gbrain
as your memory. This runbook is the judgment layer; the deterministic work lives in
`gbrain bootstrap` subcommands with exit codes. Follow it top to bottom.
You are an AI agent (Claude Code, Codex, or opencode) reading this because a human
pasted a block asking you to set yourself up as their persistent personal agent, with
gbrain as your memory. This runbook is the judgment layer; the deterministic work lives
in `gbrain bootstrap` subcommands with exit codes. Follow it top to bottom.
**Scope note:** this path is for Claude Code and Codex (desktop apps or CLIs).
**Scope note:** this path is for Claude Code, Codex, and opencode (desktop apps or
CLIs; opencode = the SST terminal agent, opencode.ai — not OpenClaw).
Running OpenClaw or Hermes? Use `INSTALL_FOR_AGENTS.md` instead.
**End state:** this folder is your workspace — identity files rendered from your
@@ -96,12 +97,16 @@ you needed; report the count at the end (it feeds the install-time measurement).
3. **Interview.** `gbrain bootstrap interview --init`, then ask the questions from
the bank (the CLI prints them) in three batches, recording each answer verbatim
with `--set KEY "value"`. Push once on vague answers to the required questions.
Claude Code only: with the final batch, also ask the ONE operational consent —
MCP scope. It is not one of the 12 interview questions; consents ride alongside
the bank. The choice: project (recommended — any other repo you open cannot
read your brain) vs user (your agent everywhere, but any repo you open can
reach it — read and write — and two open sessions contend for the database).
Record it with
Claude Code and opencode: with the final batch, also ask the ONE operational
consent — MCP scope. It is not one of the 12 interview questions; consents ride
alongside the bank. On Claude Code the choice: project (recommended — any other
repo you open cannot read your brain) vs user (your agent everywhere, but any
repo you open can reach it — read and write — and two open sessions contend for
the database). On opencode the recommendation INVERTS: user-global is the
default and the sharing-safe choice (opencode spawns project-config-defined
servers with NO trust prompt, so a committed project entry executes on every
collaborator's machine) — offer project only as a deliberate opt-in and state
that consequence. Record it with
`gbrain bootstrap interview --set MCP_SCOPE <project|user>` BEFORE the
read-back, so the confirmation covers it. On Codex, skip this question
entirely — the wiring step states the Codex reality instead.
@@ -133,6 +138,14 @@ you needed; report the count at the end (it feeds the install-time measurement).
on this machine can reach the brain (read and write) through its MCP
tools; the off-ramps are `codex mcp remove gbrain` (registration only) or
`gbrain bootstrap uninstall` (full teardown).
- opencode: writes the MCP entry directly into opencode's JSONC config (no
CLI exec needed) and relies on the AGENTS.md protocol, which opencode loads
natively — say plainly that opencode gets pull-based context, not per-turn
push. Scope follows the recorded MCP_SCOPE answer (user-global default; a
project answer writes the committed-candidate `opencode.json` and the CLI
prints the sharing warning). Restart opencode after wiring — it reads config
at session start. Off-ramps: the entry's `"enabled": false`, or
`gbrain bootstrap uninstall`.
7. **Private repo.** `gbrain bootstrap repo` — creates a PRIVATE GitHub repo from
the workspace, verifies the privacy bit through the API, pushes. If the human
started from a repo they created themselves (create-repo-first: an EMPTY private
+423
View File
@@ -2,6 +2,429 @@
All notable changes to GBrain will be documented in this file.
## [0.46.4.0] - 2026-08-15
**opencode joins the supported-client roster — at full parity from day one.**
(opencode is opencode.ai, SST's terminal agent — not OpenClaw.) Unlike earlier
clients that started with a manual recipe, opencode lands with every install
lane gbrain has: the paste-in workspace bootstrap, machine-level harness
wiring, `gbrain connect`, a claw-test runner, and a real-binary e2e door in
CI. Every asserted flag, config shape, and quirk was observed against a
pinned install (opencode 1.18.18), recorded in a machine-checked pin
document, and exercised against the real binary — including the part that
makes opencode special: its keyless anonymous free tier drives MCP tool
calls, so the end-to-end proof needs zero secrets.
### Added
- **`gbrain bootstrap hooks --harness opencode`** — workspace-lane MCP
registration via direct, comment-preserving JSONC writes (never a CLI
exec, works offline). MCP scope is honored with a deliberately INVERTED
default: user-global, because opencode spawns project-config servers with
no trust prompt; project scope is an explicit opt-in that prints a sharing
warning. A structural ownership fingerprint refuses to touch entries
gbrain didn't write.
- **`gbrain bootstrap harness --harness opencode`** — machine-level remote
MCP wiring with an inline bearer written 0600, token rotation across URL
changes, content-guarded rollback on failed smoke, `--status` and
`--remove`.
- **`gbrain connect --agent opencode [--install]`** — env-interpolated
bearer (`{env:GBRAIN_REMOTE_TOKEN}`): the token never enters the config
file. `--force` replaces a registration whose endpoint moved.
- **`gbrain claw-test --agent opencode`** and a split-gated real-binary e2e
door in CI: keyless tier (version pin, install + `mcp list` handshake,
spawn-gate canary, writer parity, MCP SMOKE on the free tier) plus a paid
Anthropic leg that model-gates before spending; npm supply-chain
provisioning verifies the actual downloaded tarball bytes against pinned
integrities; a schedule-only canary tracks the latest upstream release.
- **Docs:** `docs/mcp/OPENCODE.md` install guide,
`docs/mcp/OPENCODE-CLI-PIN.md` observation pin (with a verify-time drift
guard and a pin-refresh cadence), roster updates across README / INSTALL /
bootstrap guides. opencode reads the rendered AGENTS.md pull-protocol
contract natively.
### Changed
- The bootstrap config writers (Claude hooks JSON, Codex TOML, opencode
JSONC) now share one atomic-write helper; symlinked configs — including
dangling dotfile-manager links — survive writes as links.
- The door-test family (binary resolution, hermetic child envs, one-shot
spawns) extracted into shared factories; the hermes and grok runners were
ported onto them, hermes child envs gained the GitHub step-metadata scrub,
and the hermes installer pin was refreshed (its nightly door had gone red
on upstream installer drift).
- A new pin-doc privacy guard asserts every agent pin document ships with
placeholder paths and no key material.
### Fixed
- Security and robustness hardening from the pre-landing cross-model review
pass: registration verification probes run isolated and time-bounded, and
a hung probe is killed instead of abandoned; global config writes
reconcile both opencode global filenames under the bootstrap lock; config
backups are unique per operation with content-guarded restore; error
paths never echo credentials; test-harness child processes drop CI
credentials before spawning third-party binaries.
### To take advantage of v0.46.4.0
opencode users: run `gbrain bootstrap hooks --harness opencode` in your
brain workspace (or paste the standard bootstrap block into an opencode
session). The keyless free tier is enough to verify the wiring end to end —
`opencode mcp list` should show `✓ gbrain connected`. Existing installs:
nothing changes; this release adds a client, it doesn't modify brain
behavior.
## [0.46.2.0] - 2026-08-15
**Dream synthesis now triages before it spends.**
([#4152](https://github.com/garrytan/gbrain/issues/4152)) The synthesize
phase used to point its most expensive model at every transcript that
cleared a yes/no check — on a busy brain that meant an unbounded queue of
long frontier-model jobs grinding through logistics and small talk. It is
now a two-stage cascade: a cheap scored triage reads every file first, and
only what scores above your threshold reaches the synthesis model, which
starts from a map of the noteworthy passages instead of hunting through raw
transcript.
### Added
- **Scored triage gate.** Every transcript gets a 01 salience score,
content type, candidate quotes, and entity candidates from the utility-tier
model (one call per new file, cached in `dream_verdicts` with the judging
model + prompt version — migration v129). The gate
(`dream.triage.threshold`, default 0.5) is applied at read time: retune it
any time and re-gating costs **zero** new LLM calls. Provider hiccups
(truncation, refusal, unparseable output) are never cached as rejections —
those files are re-judged next cycle, and an outage reports as "triage
degraded", never as "everything scored low".
- **`gbrain dream retriage`** — re-score the corpus and reconcile the queued
synthesis backlog. `--dry-run` previews from cached scores with zero LLM
calls; `--reconcile-queue` cancels queued jobs that score below the gate
AND converts jobs stranded in dead per-run queues so the next cycle
actually re-submits them; `--audit-rejects <n>` gets a frontier-model
second opinion on a sample of rejects (the threshold-calibration loop).
Every sweep prints an upfront cost estimate and asks before spending more
than a few dollars (`--yes` to skip, `--max-usd` for an estimate-based
budget stop that counts every paid call, including unreliable ones — it
can overshoot by up to the configured triage concurrency). Guardrails: queues
younger than an hour are treated as possibly-live and never touched;
`--cancel-unmatched` refuses to run off a truncated or empty corpus scan.
- **Triage map in the synthesis prompt.** Passing files hand the synthesis
subagent their pre-extracted quotes and entities (verbatim-verified against
the chunk text) so it works from signal instead of re-scanning sludge.
- **Cost knobs.** `dream.synthesize.max_turns` (default now 16, was a
hardcoded 30 — set it back via config if your written-page counts drop;
`details.synthesis.avg_turns` shows cap pressure),
`dream.triage.max_ms` (per-cycle triage time budget, default 5 min — a big
cold corpus triages across a few cycles, with deferred files labeled "not
yet triaged", never silently rejected), and an opt-in per-source daily
synthesis cap (`dream.synthesize.max_submissions_per_source_per_day`,
default off; 200/day is a sane value for busy deployments). The intended
pairing is the shipped mid-tier synthesis default — frontier-model
overrides are unnecessary with triage doing the reading.
### Fixed
- A run whose submissions were all skipped (cap, already-synthesized) no
longer starts the 12-hour cooldown, so the skipped files retry on the next
cycle instead of waiting half a day.
- Synthesis jobs stranded in a dead per-run queue by a killed cycle are
self-healed on the next run (cancelled and re-submitted into the live
queue) instead of stalling the phase for the full 35-minute wait.
- `gbrain dream retriage --help` (and richer `gbrain dream --help`) now
print real usage instead of the generic one-line stub, with no brain
configured.
To take advantage of v0.46.2.0: upgrade and run `gbrain dream` as usual —
existing verdicts are re-scored automatically on the next cycle (cheap,
utility-tier). If you have a queued synthesis backlog, run
`gbrain dream retriage --dry-run` to preview, then
`gbrain dream retriage --reconcile-queue` to drain it for pennies. Tune
`dream.triage.threshold` freely; re-gating is free.
## [0.46.1.0] - 2026-08-15
**A stuck job can no longer take down your whole worker.** Field reports from
a production deployment ([#5](https://github.com/garrytan-agents/gbrain/issues/5),
[#6](https://github.com/garrytan-agents/gbrain/issues/6)) showed two
compounding failure modes: a handler that ignored its abort signal could only
be "force-evicted" (abandoned but still running, still holding connections),
and abandoned probe/renewal queries starved the connection pool until the
worker killed itself with a misleading "DB unreachable" — while the database
sat at a fraction of capacity. This release fixes the starvation class and
adds real per-job blast-radius control.
### Added
- **`gbrain jobs work --job-isolation process`** (also
`gbrain jobs supervisor --job-isolation process`, env
`GBRAIN_JOB_ISOLATION`): each claimed job runs in its own child process.
A stuck handler is group-SIGKILLed for real instead of abandoned, a crash
or memory blowup takes one job instead of all N, and the OS reclaims every
leaked resource when the child dies. The worker keeps claiming, renewing,
and recording; handler-error semantics (unrecoverable → dead, rate-lease →
no attempt burned, backoff otherwise) are preserved across the boundary.
Worker shutdown gives children the drain window to finish and report — a
routine deploy never burns a job attempt. Recommended for long-running
LLM-bound handlers; see the new section in `docs/guides/minions-deployment.md`.
- **Health-probe verdicts that name the failing layer.** When the worker's DB
probe fails, it now disambiguates via the direct session lane and says
`pool_starved` ("server IS reachable; the fault is in the
transaction-pooler path") or `server_unreachable` — instead of the blanket
"DB unreachable" that historically sent operators debugging database
capacity while the real fault was client-side. A startup warning also makes
single-pool mode (direct-lane kill switch) loud instead of silent, and
`docs/guides/queue-operations-runbook.md` gains a verdict-interpretation
table.
- `GBRAIN_POOL_MAX_LIFETIME_S`: explicit client-pool connection max-lifetime
knob (0 disables; default stays the per-connection 3060min jitter).
### Fixed
- **Timed-out DB probes and lock renewals are now cancelled, not abandoned.**
Every place that raced a query against a timer (health probe, minion lock
renewal, cycle-drain renewal, submit-time queue probes, DB-lock refresh)
previously let the losing query keep running on a checked-out connection —
under pool exhaustion each abandoned racer held a slot and made the
exhaustion worse, starving the lock heartbeat first. All five sites now
abort the query via its cancellation signal so the slot is released.
- Long-running maintenance holds (index rebuilds, non-transactional
migrations, backfill write batches) now reserve from the direct session
lane instead of pinning the worker's shared pool — capped so reserved
holds always leave a direct-lane slot for the claim/renewal heartbeats,
and falling back to the previous behavior when the direct lane is
unavailable.
Full operational detail: `docs/guides/minions-deployment.md` (isolation
sizing: connections, memory, spawn cost) and
`docs/guides/queue-operations-runbook.md` (probe verdicts).
## [0.46.0.0] - 2026-08-14
**Your other agents' sessions become brain knowledge.** Until now only Claude
Code sessions flowed into the brain automatically; every Codex rollout,
OpenClaw session, and Hermes conversation on your disk — often years of
decisions — was invisible. `gbrain transcripts ingest` imports them all as
readable conversation pages with provenance back to the exact session file,
and the facts pipeline makes them answer "what did I decide about X, in
whichever agent I said it" as one query. Consumer chat exports (ChatGPT and
Claude.ai `conversations.json`) import through the same door.
- **One command, six formats.** `gbrain transcripts ingest <path-or-glob>`
auto-detects Claude Code JSONL, Codex rollouts, OpenClaw sessions, the
Hermes SQLite store (read from a lock-safe copy), and extracted
ChatGPT/Claude.ai exports. No arguments shows what it WOULD import across
your harness directories; `--all` imports the discovered set;
`gbrain transcripts status` shows the found-vs-imported gap per harness.
- **Safe by default.** Secrets are redacted before anything is written
(bodies, titles, speaker labels, and session metadata; plus your
`harvest-private-patterns.txt` rules), message content that mimics
conversation formatting cannot forge speakers or timestamps, and imports
are a readable text-turn archive by design — tool payloads and thinking
blocks never land in pages (one-line placeholders mark where they
happened). Embedding is off by default for bulk backfills
(opt in with the embed flag, or run the embed backfill later).
- **Free to re-run.** Unchanged sessions skip on content hash; long sessions
split into searchable parts that reconcile themselves when a session
shrinks; interrupted runs converge on the next pass, healing any half-done
writes. `--since last` resumes from the previous complete run and never
advances past files it could not fully read.
- **Facts on demand.** `--facts` extracts through the shipped
conversation-facts pipeline under a budget cap; imported pages also flow
into the existing scheduled backfill when that cycle phase is enabled.
### Added
- `gbrain transcripts ingest` and `gbrain transcripts status` subcommands
(engine-free `--help`), with discovery mode, `--all`, `--dry-run`,
`--format`, `--limit`, `--since <iso|last>`, `--source-id`, `--facts`,
`--max-cost-usd`, `--embed`, `--json`, `--quiet`.
- Transcript-adapter seam at `src/core/transcripts/` (session-granular
contract with per-file diagnostics and drift alarms; dated spec targets per
host format) and adapters for Codex, OpenClaw, Hermes, ChatGPT export, and
Claude.ai export; the shipped Claude Code parser gains an additive
timestamp-preserving mode, regression-pinned for the hook lane.
- Batch `slugs` selector on the conversation-facts extraction core (one
invocation per import run; an empty list is a no-op, never a full-corpus
walk).
- Write-back fidelity e2e through the raw adapter path (gold-extractor
seam), pinning cross-harness continuity in one source.
### Changed
- `skills/conversation-archive` now routes the covered formats to the native
importer and states the native-vs-manual privacy delta.
- The fixture-privacy gate also scans the new transcript fixture corpus.
### Fixed
- PGLite `putRawData` now detects a missing page like the Postgres engine
(integrity failures abort instead of silently no-opping).
### To take advantage of v0.46.0.0
Upgrade, then run `gbrain transcripts ingest` with no arguments to see every
importable session log on the machine, and `gbrain transcripts ingest --all`
to import them. Unzip consumer exports first and pass the extracted
`conversations.json`. On PGLite, stop `gbrain serve` for the import (the
single-writer lock error names the PID if you forget). Run
`gbrain transcripts status` any time to see what's still waiting.
## [0.45.20.0] - 2026-08-14
**Grok Build joins the supported-client roster.** xAI's `grok` CLI can now wire a gbrain brain in one command, and — like Hermes before it — the install path is proven against the real binary, not written from docs: every asserted flag, config shape, and exit-code quirk was observed against a pinned Grok Build install, recorded in a machine-checked pin document, and exercised by a real-binary e2e door that CI can run.
### Added
- **Grok Build install support.** `grok mcp add gbrain -- gbrain serve --surface verbs` wires the seven-verb memory surface into xAI's coding agent; [docs/mcp/GROK.md](docs/mcp/GROK.md) carries the full guide — registration, direct TOML config, the trust-gated vendor-config fallback (an existing Claude Code registration may already work), verification via `grok mcp doctor` (the honest probe: the add itself is lazy and always exits 0), headless auth, model pinning, auto-update pinning for reproducible environments, cron pairing, and troubleshooting (including the colliding community `grok` binary and where Grok actually discovers skills). `INSTALL_FOR_AGENTS.md` gains the matching "If you are Grok Build" block; the guide is honest that this is the brain-only install — the `gbrain bootstrap` personal-agent path doesn't support Grok yet.
- **`gbrain claw-test --live --agent grok`.** Grok is the third registered agent runner, so guide-following friction runs and `gbrain friction diff --base <hermes-run> --compare <grok-run>` work out of the box. The runner records a version preamble in every transcript (a mis-bound community binary is diagnosable after the fact) and warns loudly when the operator's `~/.claude.json` registers gbrain — Grok reads vendor MCP configs for trusted folders, a contamination channel no other supported agent has.
- **A real-binary "door" e2e for Grok** (`test/e2e/install-real-grok.serial.test.ts`), split-gated so the free tier needs no API key: version pin, the documented registration shape end-to-end, the seven-verb handshake proven keyless, a vendor-fallback provenance guard, and the direct-TOML surface all run with just the binary; the paid recall smoke (a per-run nonce fact, web search disabled) additionally needs `XAI_API_KEY`. A label-gated `grok-door` CI job provisions the pinned npm package (wrapper AND per-platform payload integrities pre-checked against the pin doc), banks the keyless coverage before any secret is required, and refuses green on a silently-skipped paid tier.
- **`docs/mcp/GROK-CLI-PIN.md`** — the observed-behavior pin (config schema verbatim, lazy-add semantics, honest doctor discriminator, keyless TUI sign-in behavior, volatile-path inventory, supported-version policy) with machine-readable stamps enforced against the CI workflow by a new `check-grok-pin` verify guard, which fails closed if the pin doc ever disappears out from under the door job.
- **A `grok-install` DX scenario** (`scripts/dx-explore.ts`) drives the REAL interactive Grok TUI through the brain-only install under a PTY, with a sign-in-wall early-stop so keyless runs record the friction in seconds instead of pasting into a login screen for the full wall clock.
### Changed
- **Agent runners share their detection/env plumbing.** The byte-identical binary-resolution and env-filtering bodies moved out of the per-agent runners into `agent-runner.ts` (`detectBinary`/`filterAllowlistEnv`), shrinking every future runner; binary resolution never passes through a shell anymore. The live-lane Grok runner forwards only Grok's own credentials — the operator's Anthropic/OpenAI keys never reach a third-party binary.
- **PTY transcripts are structurally redacted at every write site.** Provider-key values are replaced in every artifact — including the live screen mirror that outlives interrupted runs — with a hard-failing independent check behind the redaction; a secret split across output bursts can no longer be reassembled from the frame log. `--keyless` now genuinely drops provider keys in all install scenarios, the leak check never touches files that predate the run, and the PTY hot loops strip bounded windows instead of the whole buffer (repaint-heavy TUIs were making every poll quadratic).
- **The hermes-door CI job got the same hardening sweep:** checkout token persistence off, and the door's full test output is preserved as evidence even for the failure class that previously left no trace.
To take advantage of v0.45.20.0: nothing changes for existing installs — this release adds a client, it doesn't modify brain behavior. Grok Build users: follow [docs/mcp/GROK.md](docs/mcp/GROK.md) (two commands: register, then `grok mcp doctor gbrain` to verify the seven-verb handshake). Maintainers enabling the paid CI lane: create the `XAI_API_KEY` repo secret, then land the follow-up that adds the schedule and canary legs.
## [0.45.19.0] - 2026-08-15
**The interactive `gbrain init` pickers are now tested on a real terminal — and the repo carries one PTY layer instead of two.** The first thing every new user touches (the embedding provider picker and the search-mode picker) previously had no true-TTY coverage; the comments that claimed otherwise pointed at a harness nothing ever called.
### Added
- A real-PTY test for the interactive init flow (`test/init-picker-pty.serial.test.ts`). It drives both pickers under a true pseudo-terminal, proves typed input actually lands (non-default selections plus bounded response times, so dead input can never pass through the pickers' silent defaults), covers the Ctrl-D/EOF fallback, and runs in the required serial CI lane. Fully hermetic: temp-root home, no provider keys visible, child process reaped even on failed assertions.
### Removed
- The unused second PTY harness (`test/helpers/cli-pty-runner.ts` plus its self-test, ~657 lines). Its spawn path never gained a caller across ~20 minor versions. The real-terminal layer is consolidated on `test/helpers/tty-harness.ts` (Bun's built-in `terminal:` spawn — no native modules).
### Fixed
- Three test-file comments that claimed interactive-picker coverage existed where it didn't — two deferred to each other in a circle, one described a piped-stdin test as PTY-based. All three now point at the real coverage.
### Changed
- `docs/TESTING.md` gains a four-tier decision table for TTY and interactive-CLI testing (injected `isTTY` → piped stdin → real-PTY serial test → DX-exploration instrument), including the rule that CI-required interactive tests live in the serial lane. `docs/architecture/KEY_FILES.md` now describes the surviving harness accurately, and related project docs and the stale shard-weight entry were trued up to match.
## [0.45.18.0] - 2026-08-15
**Today's agent spend now reads correctly at every hour, in every timezone.** The admin spend endpoint computed "today" against a naive timestamp that each database session reinterpreted in its own timezone — on any non-UTC session (a PGLite brain following the host clock, a timezone-configured Postgres role), the day boundary shifted by the offset and every evening's spend silently underreported as 0. The boundary is now a UTC instant, independent of session timezone, pinned by a regression test that exercises sessions 12 hours either side of UTC at any wall-clock hour.
The same class also made the new test-suite snapshot fixture time-of-day flaky: the snapshot bakes the build machine's timezone into the restored cluster, so snapshot-restored engines ran sessions in the builder's zone while cold-init engines followed the running process. Restored engines now re-pin their session to the runtime zone (existing tarballs heal without a rebuild), the snapshot builder pins UTC so tarballs are deterministic across hosts, and a parity test asserts cold and snapshot engines agree on their UTC offset.
### Fixed
- `/admin/api/agents/spend`: `spent_cents_today` no longer underreports on non-UTC sessions (UTC-instant day boundary).
- Snapshot-restored PGLite engines behave identically to cold-init engines regardless of the machine that built the tarball.
## [0.45.17.0] - 2026-08-15
**A test run can no longer silently touch a real brain.** `gbrain init` writes your
database URL into `~/.gbrain/.env`; anyone who had that sourced and ran a bare
`bun test` in the repo was one destructive fixture away from their own data
(#3485 — it has happened). Four independent layers now stand in the way, and
each one fails loudly instead of silently skipping:
- **The run refuses to start.** A test preload (registered first in
`bunfig.toml`) hard-fails any `bun test` invocation while `DATABASE_URL` or
`GBRAIN_DATABASE_URL` is ambient, with instructions — it never silently
unsets, because a silent unset would turn database-gated e2e tests into
green skips. The e2e and heavy lanes opt in at their own boundary;
the unit and slow lanes strip the variables at theirs, so
`bun run test:full` with a database URL exported still reaches its e2e leg.
- **Destructive tests check the database name.** Every test that runs
destructive SQL against the ambient URL now calls a shared name floor
(moved to a leaf module so unit-directory tests can use it too): the
database name must carry "test" as a word segment, or be opted in
explicitly, one-shot. (One suite keeps its own equivalent inline floor,
pinned by the coverage gate.) This adopts the patch contributed in #3485 by
@cheRoma — thank you — extended to two newer files the original audit
predates and one raw-client suite it couldn't see.
- **Shell lanes get the same floor.** The heavy-test scripts (schema drops,
parallel syncs, migration replays) share a floor that checks BOTH database
URL variables and strips query strings before extracting the name, so a
`?host=/tmp/test-sockets` parameter can't smuggle a test-shaped segment
past it.
- **A repo-wide static gate keeps it that way.** A scanner walks every test
file bun would collect (all naming patterns, fixtures included), flags any
file that reads an ambient database URL, opens a connection, and runs
destructive SQL without a guard — and its own classifiers are pinned by
positive controls so the gate can never rot into passing vacuously.
### Added
- Test-run guard preload (`test/helpers/database-url-guard-preload.ts`) with
subprocess tests covering every branch: both variables, both-set, override,
strict override value, empty-string, and clean runs.
- Shared destructive-SQL name floor `test/helpers/db-guard.ts` (re-exported
from `test/e2e/helpers.ts` for existing call sites) and shell twin
`tests/heavy/_db_floor.sh`.
- Repo-wide destructive-SQL coverage gate `test/db-guard-coverage.test.ts`
with classifier self-tests and positive controls.
### Fixed
- Ten destructive test files now verify the database name before connecting
(#3485; patch by @cheRoma, extended).
- The heavy lane's fixture builder, sync-lock, upgrade-matrix, and wallclock
scripts refuse non-test-shaped database names instead of operating on
whatever the environment points at.
- The phantom-redirect engine-parity test's Postgres arm is now carried by
the e2e lane and CI's parity job — previously no lane could reach it.
### To take advantage of v0.45.17.0
Nothing to configure. If a bare `bun test` now refuses to start, the message
tells you exactly why and what to do — usually just unset the database URL
(unit tests need no database) or use `bun run test:e2e`, which opts in at its
own boundary. If your e2e database has a non-test-shaped name, opt in one-shot
with `GBRAIN_E2E_ALLOW_DB=<name>` rather than exporting it in your shell
profile — a permanent export would disarm the guard for exactly the database
it protects.
## [0.45.16.0] - 2026-08-14
**Fix wave W0: the verified-bug hotfix pass of the code-smell series.** A 10-auditor sweep of the codebase produced 122 findings; the top claims were adversarially verified, and this release fixes every verified live bug — the ones that survived the skeptic pass. Long-running brains get the biggest wins: background cycles can no longer silently run twice, dead background jobs no longer strand their parents, and image search no longer silently degrades after re-embedding. Developers get a test suite that runs 10x faster.
### Fixed
- **The background-cycle lock is now actually refreshed while a cycle runs.** Long phases (synthesis, pattern extraction, consolidation — up to 35-minute waits) previously outlived the 5-minute lock TTL with no heartbeat, so a second cycle could start against the same brain and both would write concurrently — duplicated LLM spend and racy writes on Postgres/Supabase brains. A dedicated refresher now heartbeats the lock, every refresh and release is fenced to the exact acquisition (a recycled PID or a superseded run can never touch a successor's lock — including the PGLite file lock, which is no longer rewritten after a detected steal), and a run that loses its lock stops at the next phase boundary with a structured `lock_stolen` report instead of compounding. The job supervisor treats a fenced miss as certain loss and exits for a clean restart.
- **Background jobs that die from repeated stalls now notify and unblock their waiting parents.** Previously an aggregator parent whose child was dead-lettered by the stall sweep waited forever; a self-healing sweep also releases parents stranded before the upgrade.
- **Retried jobs no longer burn their wall-clock budget while waiting in backoff.** Every automatic re-run path — and every parent-unblock path — resets the per-attempt clock, so exponential-backoff retries and long-waiting aggregators aren't dead-lettered before executing a line.
- **Re-embedding no longer flips image chunks to text.** `gbrain embed --stale` (including the autopilot path) preserved every chunk field except `modality`, silently zeroing image retrieval until the next full import. One shared carry list now serves every re-embed path.
- **A failed first sync no longer kills the MCP server.** Import preflight failures (missing embedding credentials, unreadable target) now surface as normal tool errors instead of terminating the serving process mid-call.
- **`gbrain lint --fix` reports the true fix count** (it previously scanned everything twice and reported "0 auto-fixed" after fixing issues) and walks the tree once.
- **The PGLite repair and re-init confirmation prompts can no longer hang forever** on closed or piped stdin: EOF declines safely, and prompts write to stderr so `--json` output stays clean.
### Changed
- **`bun run test` is ~10x faster** (measured: a full parallel suite run dropped from ~82 to ~8 minutes). The PGLite schema snapshot is now default-on for the everyday test loop, rebuilt automatically when migrations or the pinned embedding shape change, concurrency-safe across parallel shards and workspaces, and refused on any shape mismatch so a wrong fixture can never poison the suite.
- **CI guards now prove they can fail.** A guard registry classifies all 45 check scripts; self-tested scanner guards run against known-bad fixtures on every verify (the registry tracks fixture coverage for the rest), so a guard whose pattern rots into a permanently-green no-op fails the build instead of masquerading as coverage. Two such rotted patterns were found and fixed in the process, along with three guards that were wired into a registry nobody ran.
To take advantage of v0.45.16.0: upgrade and restart any long-running `gbrain serve`, autopilot, or jobs supervisor/worker daemon so the fenced lock refresh and job-reaper fixes take effect. If you run image search, run `gbrain backfill modality` once after upgrading to restore any image chunks a prior re-embed flipped to text (`gbrain doctor` surfaces the affected count and the exact command). No schema migration and no config changes are required.
## [0.45.15.0] - 2026-08-14
**The queue that drains itself: three background-jobs fixes reported from a downstream agent deployment (upstream issues #2, #3, #4).** A brain whose autopilot cycle stalled mid-run could accumulate byte-identical queued cycles forever while every long job queued before an upgrade died minutes in — and the operator diagnosing it couldn't even find the worker entry point, because `gbrain jobs --help` printed a one-line stub. All three failure modes are closed, and the queue now tells you when it's holding work back.
### Fixed
- **Stalled cycles no longer breed duplicates.** Autopilot dispatch now uses a single-flight guard (`maxPending`) that counts waiting jobs AND actively-running jobs with a live lock, scoped per source — so a cycle stuck in `active` suppresses re-dispatch instead of minting a new duplicate every tick. A job whose worker died stops counting the moment its lock lapses, so a dead worker can never silently freeze dispatch: the fresh waiting row keeps feeding the existing wedge detectors. Applies to per-source cycles, the legacy single-source path, and brain-wide maintenance.
- **Long jobs queued before an upgrade get their real budget.** Handler wall-clock budgets now also resolve at claim time (not just at submit), so rows inserted with no budget — including anything queued on an older version — run with their documented allowance instead of being dead-lettered by the minutes-scale default. Migration v128 backfills budgets for everything still in flight and cancels the duplicate cycle backlog (newest per source survives; manually submitted cycles without a ticker-style idempotency key are never touched; cancelled rows are kept for audit).
- **`gbrain jobs --help` prints the real surface.** The full subcommand list plus dedicated help for `work`, `supervisor`, `submit`, `watch`, and `prune` — engine-free, and a help flag after a subcommand can no longer fall through and start a real worker daemon.
### Added
- **`gbrain jobs stats` shows suppressed dispatch.** A `Backpressure (24h)` line reports submissions coalesced onto in-flight jobs, plus a hint naming the specific in-flight job holding a queue-empty name back — the visibility that was missing when "nothing queued, nothing completing" was the only symptom.
- **`gbrain jobs get <id>` shows the effective wall-clock budget** — the stamped timeout and deadline, or which default applies and when it kicks in.
- Autopilot cycle dispatch tells the truth: a submission that coalesced onto an existing job reports `dispatch_coalesced` (and `coalesced: true` in `jobs submit`'s JSON output) instead of claiming a dispatch that never inserted a row.
To take advantage of v0.45.15.0: upgrade and run any gbrain command — migration v128 applies automatically, backfilling budgets for queued long jobs and clearing any duplicate cycle backlog. If a queue looked wedged before, `gbrain jobs stats` now names the in-flight job to inspect and `gbrain jobs work --help` documents the worker daemon flags end to end.
## [0.45.14.0] - 2026-08-14
**The box that already has a brain: framework-spawned coding agents get brain access by default.** The bootstrap door built in v0.45.0.0 was for a human at a laptop. A growing share of Claude Code and Codex sessions are spawned by an agent framework — your OpenClaw, or anything that shells out to headless sessions — on a machine that already hosts a brain and a running `gbrain serve --http`. Until now those sessions got nothing unless someone hand-replicated settings writers across every project directory. One command fixes that:
gbrain bootstrap harness --yes
### Added
- **`gbrain bootstrap harness`** — machine-level wiring, no agent workspace or interview required. Mints a least-privilege bearer token, registers a user-scope HTTP MCP server for Claude Code, pre-approves its tools for headless runs (the `permissions.allow` gate that otherwise blocks `claude -p`), wires the five lifecycle hooks (user scope by default, or exactly the dirs you pass with repeatable `--project`), and writes Codex's config block directly — with the token inline, because framework-spawned codex inherits no shell profile for an env var to live in. Everything is stated before it happens (reach, plainly: read AND write, every session on the machine; transcript capture is its own consent line with `--no-capture` as its off-ramp), non-interactive runs require `--yes`, re-runs are idempotent, and `--remove` tears down exactly what the machine-level receipt records.
- **Scoped bearer tokens.** `gbrain auth create --scopes read,write` narrows a token to exactly those operations; tokens created without scopes keep their historical full access, byte for byte. The harness token uses this by default, and its reads span the brain's federated sources — the same reach a local session gets. `gbrain auth list` now shows each token's id and honest scope; `gbrain auth revoke --id <uuid>` revokes precisely one token (names were never unique). The admin dashboard shows real grants instead of assuming full access.
- **Safe rotation by construction.** Re-running harness wiring mints the new token first, wires and verifies everything, and only then revokes the previous token by id — a failed re-wire leaves the old credential fully working. A wiring crash at any step leaves a consumable receipt: `--remove` and `gbrain bootstrap uninstall` clean up partial states instead of stranding them.
- **`gbrain bootstrap harness --status`** probes the live truth — serve health, token validity (recovered from the host's own registration, redacted), per-target states, and honest degrades — with cron-friendly exit codes. `gbrain doctor` gains a `bootstrap_harness_health` check that distinguishes "serve is down (normal transient)" from "wiring incomplete" from "rotation never converged".
- **Honesty on Postgres brains.** Per-turn hook injection is PGLite-only today; harness mode says so plainly at install time, wires the hooks anyway (they light up when the engine-uniform listener lands), and names MCP as the active seam.
### Fixed
- A routine `gbrain auth permissions set-takes-holders` edit silently deleted a token's other stored grants (whole-object replace); it now merges — and resets rows whose stored grants were damaged by a historical encoding bug to a clean object instead of compounding them.
- Registration ownership on multi-brain machines: harness wiring refuses to replace an MCP registration that points at a different brain's serve without `--force`, and removal skips registrations it no longer owns — or whose ownership it cannot verify — instead of deleting another install's wiring.
- User-scope settings writers resolve Claude Code's config location the way Claude Code does (`CLAUDE_CONFIG_DIR`, then `$HOME`) — sandboxed environments previously risked writing to the operator's real settings file.
- Ship-review hardening (three adversarial passes at ship): the post-wiring verification now sends a deliberately invalid credential first — an endpoint that accepts it is not a real serve, wiring rolls back, and the fresh token is retired immediately on ANY failed verification; a failed verification also rolls a fresh Claude Code registration (and its headless pre-approval) back to the pre-run state; the pre-approval never lands when the registration itself failed; prior wiring is only cleaned up after the replacement verifies; `--status` and `gbrain doctor` report honestly on partially-applied or partially-removed installs instead of reading vacuously green, and `--status` only recovers a bearer from a registration it can verify as its own; token-scope reads fail closed on damaged rows across the verify and CLI display paths (`auth list` shows exactly what the serve enforces); a value-less `--project` or `--scopes` flag errors loudly instead of silently widening scope or minting a full-access token; settings writers refuse to rewrite permission policy shapes they don't understand; config writes serialize under cross-install locks on every path (apply, remove, cleanup, rollback); and per-turn hooks defer per-event to workspaces that carry their hook wiring in committed settings.
To take advantage of v0.45.14.0: upgrade, then on any agent-framework box run `gbrain bootstrap harness --yes` against your running `gbrain serve --http`. On PGLite brains, pre-mint with `gbrain auth create bootstrap-harness --scopes read,write` while the serve is stopped and pass `--token`. Restart your serve after upgrading so token scoping is enforced by the new verify path — the install says this too, exactly when it applies. See the "Local harness mode" section of docs/guides/bootstrap.md.
## [0.45.13.0] - 2026-08-13
**The Truthful Surface wave: your agent's MCP catalog now tells the truth. What's listed is callable, empty answers explain themselves, and new clients start with a focused ~26-tool surface they can widen on demand.**
+5 -1
View File
@@ -70,7 +70,11 @@ Per-file detail is in `docs/architecture/KEY_FILES.md`.
- **Engine-live paths avoid runtime dynamic `import()` for helper dependencies.** In
`src/core/pglite-engine.ts`, `src/core/postgres-engine.ts`, and
`src/core/migrate.ts`, dependencies previously reached through runtime dynamic
imports use static top-level imports. The only current dynamic-`import()` exceptions
imports use static top-level imports. Besides the snapshot loader's lazy
`require()` cluster in `pglite-engine.ts:tryLoadSnapshot` (fs/crypto/
migrate/pglite-schema + one gateway shape lookup — lazy so production
builds without the test-fixture path don't eager-load; the guard now
matches `require()` calls too), the only dynamic-`import()` exceptions
are the four `ai/gateway.ts` lookups in both engines'
`initSchema()` and `_upsertChunksOnce()` methods; each remains lazy inside a
local `try/catch` because the gateway has a large provider/config closure and,
+25 -5
View File
@@ -92,11 +92,11 @@ The canonical reference for test tiers, isolation rules, timing, and the E2E
lifecycle is [`docs/TESTING.md`](docs/TESTING.md). The short version:
```bash
# Inner edit loop (~85s on a Mac dev box)
bun run test # parallel 4-shard fan-out (memory-adaptive) + serial post-pass
# Inner edit loop (~8min full suite on a Mac dev box; single files in seconds)
bun run test # parallel 4-shard fan-out (memory-adaptive) + serial post-pass; PGLite snapshot default-on
bun test test/markdown.test.ts # specific unit test
# Pre-push gate (19+ parallel checks + typecheck)
# Pre-push gate (40+ parallel checks + typecheck)
bun run verify
# Pre-merge sanity (everything CI runs)
@@ -115,7 +115,21 @@ DATABASE_URL=postgresql://postgres:postgres@localhost:5434/gbrain_test bun run t
DATABASE_URL=postgresql://... bun run test:e2e
```
Heads-up: a bare `bun test` refuses to start while `DATABASE_URL` or
`GBRAIN_DATABASE_URL` is set in your environment — some tests run destructive
SQL against whatever those URLs point at. Unset the variable for unit runs
(they need no database) or use the wrappers: the unit/slow runners strip the
variables at their boundary, and `bun run test:e2e` opts in at its own. The
refusal message walks you through it; details in
[`docs/TESTING.md`](docs/TESTING.md) ("Database-URL run guard"). If you point
`bun run test:e2e` at your own Postgres or Supabase, a second floor applies:
the database name must carry "test" as a word segment (like `gbrain_test`
above) or destructive tests refuse to run — opt a differently-named database
in one-shot with `GBRAIN_E2E_ALLOW_DB=<name>`.
Use `bun run verify` before pushing. It runs 19+ guard checks in parallel
Use `bun run verify` before pushing. It runs 40+ guard checks in parallel
(`scripts/run-verify-parallel.sh`), including: banned fork-name leaks
(`scripts/check-privacy.sh`), `JSON.stringify(x)::jsonb` interpolation
patterns (`scripts/check-jsonb-pattern.sh`), `\r` progress bleed to stdout
@@ -124,8 +138,14 @@ patterns (`scripts/check-jsonb-pattern.sh`), `\r` progress bleed to stdout
loop" below), silent fallback to recursive chunking in the compiled binary
(`scripts/check-wasm-embedded.sh`), stale admin-dashboard build artifacts
(`scripts/check-admin-build.sh`), resolver drift on bundled skills
(`bun run check:resolver`), and typecheck. `bun run check:all` runs the full
historical sweep including the trailing-newline and exports-count checks.
(`bun run check:resolver`), and typecheck. The guard REGISTRY is
`scripts/guards-manifest.tsv`, and `scripts/guard-self-test.sh` (also in
`verify`) proves each self-tested scanner guard (`selftest=yes` in the
manifest; coverage ratchets up from the `todo` rows) can actually fail by
running it against known-bad fixtures — a new `scripts/check-*` guard must be
registered in the manifest or the build fails. There is no `check:all` script; the
trailing-newline, exports-count, and no-legacy-getconnection checks run in
`verify` with everything else.
### Writing tests that survive the parallel loop
+28
View File
@@ -227,6 +227,34 @@ Keep `--args` last (everything after it becomes server argv) and verify with
`hermes mcp test gbrain` — the add exits 0 even on failure. Full reference:
[docs/mcp/HERMES.md](docs/mcp/HERMES.md).
**If you are Grok Build** (xAI's `grok` CLI): register gbrain as your MCP server:
```bash
grok mcp add gbrain -e "GBRAIN_HOME=$HOME" -- gbrain serve --surface verbs
```
The add is lazy (exit 0 without connecting) — verify with
`grok mcp doctor gbrain`, which spawns the server and must report
`7 tools discovered`. This is the brain-only install; the `gbrain bootstrap`
personal-agent path does not support Grok yet (Claude Code, Codex, and opencode only).
Verified against Grok Build v1.0.4. Full reference:
[docs/mcp/GROK.md](docs/mcp/GROK.md).
**If you are opencode** (the SST terminal agent, opencode.ai — not OpenClaw):
you are a bootstrap-supported harness — for the full persistent-personal-agent
install, follow `BOOTSTRAP_FOR_AGENTS.md` instead of this page. For the
brain-only MCP registration:
```bash
opencode mcp add gbrain --env GBRAIN_HOME=$HOME -- gbrain serve --surface verbs
```
The add is lazy (exit 0 without connecting) — verify with `opencode mcp list`,
which spawns the server and must show `✓ gbrain connected` (the exit code is 0
even on failure; read the output). Restart opencode afterwards — it reads
config at session start. Verified against opencode v1.18.18. Full reference:
[docs/mcp/OPENCODE.md](docs/mcp/OPENCODE.md).
Whether you scaffolded or not, read `skills/RESOLVER.md` (in your workspace, or the
bundled copy at `~/gbrain/skills/RESOLVER.md` when running from the cloned repo). It's
the skill dispatcher — tells you which skill to read for any task. Save this to your
+19 -2
View File
@@ -173,6 +173,8 @@ GBrain exposes nearly all of its 100+ operations as MCP tools (stdio and HTTP; a
- **[Codex](docs/mcp/CODEX.md)** — `gbrain connect https://your-host/mcp --token gbrain_xxx --agent codex` (or `--install`). Codex reads the bearer from `$GBRAIN_REMOTE_TOKEN` at runtime, so the token never lands in Codex config.
- **[Cursor / Windsurf / any stdio MCP client](docs/mcp/CLAUDE_CODE.md)** — same shape, add `{"command": "gbrain", "args": ["serve"]}` to your MCP config.
- **[Hermes](docs/mcp/HERMES.md)** — `printf 'Y\n' | hermes mcp add gbrain --env GBRAIN_HOME=$HOME --connect-timeout 60 --command $(which gbrain) --args serve`. Keep `--args` last, and verify with `hermes mcp test gbrain` (the add exits 0 even on failure).
- **[Grok Build](docs/mcp/GROK.md)** — `grok mcp add gbrain -e "GBRAIN_HOME=$HOME" -- gbrain serve --surface verbs`. The add is lazy (exit 0 without connecting) — verify with `grok mcp doctor gbrain`, which spawns the server and reports `7 tools discovered`. Verified against Grok Build v1.0.4.
- **[opencode](docs/mcp/OPENCODE.md)** (opencode.ai / SST — not OpenClaw) — `opencode mcp add gbrain --env GBRAIN_HOME=$HOME -- gbrain serve --surface verbs`, or let `gbrain bootstrap hooks --harness opencode` write the config for you (opencode is a bootstrap-supported harness — it reads AGENTS.md natively). The add is lazy — verify with `opencode mcp list`, which spawns the server (`✓ gbrain connected`). Remote: `gbrain connect https://your-host/mcp --token gbrain_xxx --agent opencode [--install]` — the config stores only the `{env:GBRAIN_REMOTE_TOKEN}` interpolation. Verified against opencode v1.18.18.
- **[OpenClaw](docs/mcp/OPENCLAW.md)** — the ClawHub bundle plugin registers gbrain automatically (`openclaw.plugin.json` ships in this repo), or add `{"command": "gbrain", "args": ["serve"]}` to `~/.openclaw/config.json`'s `mcpServers`.
- **[Claude Desktop (Cowork)](docs/mcp/CLAUDE_DESKTOP.md)** — Settings → Integrations → add the URL of your HTTP server. Remote only; the local `claude_desktop_config.json` does not work for remote servers.
- **[Claude Cowork (team plan)](docs/mcp/CLAUDE_COWORK.md)** — org Owner adds the connector under Organization Settings → Connectors.
@@ -234,6 +236,21 @@ curl -X POST https://your-brain/ingest \
For mobile capture, the inbox folder source picks up anything dropped into
`~/.gbrain/inbox/` from iOS Shortcuts / AirDrop / Drafts / Finder.
Your other agents' histories import in one command. `gbrain transcripts ingest`
parses agent session logs (Claude Code, Codex, OpenClaw, Hermes) and extracted
consumer chat exports (ChatGPT / Claude.ai `conversations.json`) into readable
conversation pages with provenance back to the exact session file. Secrets are
scrubbed from message bodies, titles, speakers, and session metadata before
anything is written, embedding is off by default for bulk backfills, and
re-runs are free — unchanged sessions skip on content hash:
```bash
gbrain transcripts ingest # discover importable session logs
gbrain transcripts ingest --all # import everything discovered
gbrain transcripts ingest ~/Downloads/conversations.json # consumer export (unzip first)
gbrain transcripts status # found vs imported, per harness
```
Third-party skillpacks can ship custom ingestion sources (Granola, Linear,
voice, OCR) against the versioned `IngestionSource` contract at
`gbrain/ingestion`. See [`docs/skillpack-anatomy.md`](docs/skillpack-anatomy.md).
@@ -295,7 +312,7 @@ The whole loop is described in [`docs/architecture/topologies.md`](docs/architec
**Self-wiring knowledge graph.** Every `put_page` extracts entity refs from markdown/wikilinks/typed-link syntax and writes edges with zero LLM calls. Typed edges (`attended`, `works_at`, `invested_in`, `founded`, `advises`, `mentions`, …). Multi-hop traversal via `gbrain graph-query`. The graph is what produces the +31.4 P@5 lift over vector-only RAG. **Obsidian-style vaults:** bare `[[note-name]]` wikilinks that point across folders — you wrote `[[struktura]]` but the page lives at `projects/struktura.md` — resolve by basename once you opt in with `gbrain config set link_resolution.global_basename true`. Off by default; `gbrain doctor` tells you how many edges you'd gain before you flip it. See [migrating an Obsidian vault](INSTALL_FOR_AGENTS.md#step-45-wire-the-knowledge-graph).
**Job queue (Minions).** BullMQ-shaped, Postgres-native job queue. Durable subagents (LLM tool loops that survive crashes via two-phase pending→done persistence), shell jobs with audit, child jobs with cascading timeouts, rate leases for outbound providers, attachments via S3/Supabase storage. Replaces "spawn subagent as fire-and-forget Promise" with something that recovers from anything.
**Job queue (Minions).** BullMQ-shaped, Postgres-native job queue. Durable subagents (LLM tool loops that survive crashes via two-phase pending→done persistence), shell jobs with audit, child jobs with cascading timeouts, rate leases for outbound providers, attachments via S3/Supabase storage. Opt-in per-job process isolation (`gbrain jobs work --job-isolation process`) runs each claimed job in its own SIGKILL-able child process, so a stuck handler dies for real and a crash takes one job instead of the whole worker; when the worker's DB health probe fails, it names the failing layer (`pool_starved` vs `server_unreachable`) instead of a blanket "DB unreachable". Sizing and rollout guidance in [`docs/guides/minions-deployment.md`](docs/guides/minions-deployment.md); probe-verdict triage in [`docs/guides/queue-operations-runbook.md`](docs/guides/queue-operations-runbook.md). Replaces "spawn subagent as fire-and-forget Promise" with something that recovers from anything.
**Non-English brains (FTS language config).** The Postgres full-text search tokenizer is configurable via `GBRAIN_FTS_LANGUAGE`. Defaults to `english`. Set it to any text-search configuration that exists in your Postgres instance:
@@ -466,7 +483,7 @@ the page PK, soft-delete-filtered, source-safe) and completes in seconds.
## Docs
- [`docs/INSTALL.md`](docs/INSTALL.md) — every install path, end to end
- [`docs/guides/bootstrap.md`](docs/guides/bootstrap.md) — the persistent-personal-agent bootstrap contract (interview, identity files, hooks, private repo, security posture, uninstall)
- [`docs/guides/bootstrap.md`](docs/guides/bootstrap.md) — the persistent-personal-agent bootstrap contract (interview, identity files, hooks, private repo, security posture, uninstall), plus local harness mode (`gbrain bootstrap harness`) for wiring framework-spawned Claude Code/Codex sessions to a running serve
- [`docs/what-schemas-unlock.md`](docs/what-schemas-unlock.md) — why schemas matter: 7 killer use cases, the structural argument for typed page kinds, the agent-co-curates pattern (v0.40.7.0)
- [`docs/schema-author-tutorial.md`](docs/schema-author-tutorial.md) — 5-minute walkthrough: fork the bundled pack, add a custom type, backfill existing pages, prove the wiring via `gbrain whoknows`
- [`docs/architecture/`](docs/architecture/) — system design, topologies, retrieval theory
+555 -29
View File
@@ -1,5 +1,162 @@
# TODOS
## Issues #5+#6 follow-ups (pool starvation + process isolation; plan: ~/.claude/plans/system-instruction-you-are-working-witty-moore.md)
- [ ] **P1-companion — nested-checkout audit + dev-mode detection.** **What:**
`transaction()` callers that invoke parent-engine methods (or module helpers
taking `engine` not `tx`) take a SECOND read-pool slot while holding the tx
slot — e.g. the `operations.ts` advisory-lock loop around `tx.addLink`. Under
a saturated pool this is a client-side self-deadlock class. Audit call sites;
add a dev-mode warning (e.g. a tx-depth counter consulted by `runUnsafe`).
**Why:** the #6 incident's exact 240s-idle sessions were never reproduced
under a debugger; this is the strongest remaining candidate — the shipped
wave mitigates the starvation class but does not close this path. **Effort:**
M. **Priority:** P1-companion.
- [ ] **P2 — per-handler isolation policy.** **What:** a per-handler-name set
(e.g. long-running LLM-bound handlers isolate, sub-second `lint`/`backlinks`
stay inline) instead of the all-or-nothing `--job-isolation process`.
**Why:** spawn cost (~0.31s) is noise for 644s subagent jobs, meaningful
for sub-second handlers; one worker should be able to mix. **Context:**
`worker.ts` executeJob's `isolated` gate is the seam. **Effort:** M.
**Priority:** P2.
- [ ] **P2 — per-child --max-rss caps.** **What:** RSS watchdog for isolation
children (the worker-level watchdog covers the worker only in process mode;
a startup note ships today). **Context:** child-job-runner.ts owns the child
lifecycle; a poll of the child's RSS + group-kill on breach mirrors the
worker watchdog. **Effort:** M. **Priority:** P2.
- [ ] **P2 — jobs-side connection-budget clamp for isolated workers.** **What:**
warn/clamp concurrency when `concurrency × (child pool + 1) + parent pools`
exceeds a configured budget (GBRAIN_MAX_CONNECTIONS-style; precedent
`sync-concurrency.ts:clampWorkersForConnectionBudget`). **Why:** isolation
multiplies pooler CLIENT connections (~73 at concurrency 15); today the
budget lives only in docs math. **Effort:** S. **Priority:** P2.
- [ ] **P3 — --job-isolation pass-through for the autopilot's embedded
supervisor.** **What:** `autopilot.ts` builds its own worker args; add the
conditional flag there (jobs supervisor already passes through). **Effort:**
S. **Priority:** P3.
- [ ] **P3 — runLockRenewalTick adoption in the cycle drain.** **What:**
`synthesize.ts` now uses the minimal `runDrainRenewalTick` (per-call signal +
guard); adopting the full tick would add the audit channel + bounded
reconnect. **Effort:** S. **Priority:** P3.
- [ ] **P3 — streaming child progress.** **What:** isolation children report
progress via their own token-fenced DB writes today (identical to inline);
an IPC stream would only add parent-side visibility (e.g. lifecycle events
in `jobs watch`). **Effort:** M. **Priority:** P3.
- [ ] **P3 — connection-audit release events + plain-idle visibility.**
**What:** `logConnectionEvent` never emits `release`, so the JSONL cannot
answer "who holds a slot"; and `getIdleBlockers` filters
`state='idle in transaction'` only — the #6 incident's plain-`idle` sessions
were invisible to it. **Effort:** M. **Priority:** P3.
- [ ] **P3 — doctor connection_routing check.** **What:** wire
`ConnectionManager.describeMode()` + `healthCheck()` (both currently
zero-caller outside tests) into a doctor check naming the routing mode,
kill-switch state, and per-pool probe latency. Comments in four files
already reference this check as if it existed. **Effort:** S.
**Priority:** P3.
- [ ] **P3 — isolation test-gap follow-ups (pre-landing review).** **What:**
(a) spawned-CLI negative tests for `jobs run-child` bootstrap guards (PGLite
→ exit 13; missing job-id/env → exit 13) and for `jobs work` with
isolation on + an unresolvable child CLI (fail-fast exit 1) — both need a
real engine bootstrap so they live in the e2e lane; (b) a behavioral (not
structural) test driving `withRefreshingLock` with a hung injected
`handle.refresh` (signal aborted at timeout, no overlapping ticks); (c) a
force-evict-skip test for isolation mode (needs the 30s evict window made
injectable); (d) operator-flow message tests (verdict-tailored FATAL text,
single-pool startup banner). **Why:** the ship coverage audit scored the
wave 82% — these are the surviving gaps. **Effort:** M. **Priority:** P3.
- [ ] **P3 — raceWithAbortTimeout shared helper.** **What:** the
"Promise.race a query vs a setTimeout that aborts an AbortController,
clearTimeout in finally" pattern now exists at five sites (db-probe
withDeadline, synthesize runDrainRenewalTick, lock-renewal-tick callAbort,
db-lock tickAbort, supervisor probeAbort), each re-deriving the same
invariants. Extract one helper and adopt it. **Effort:** S. **Priority:** P3.
- [ ] **P3 — lazy handler resolution in run-child.** **What:** every isolation
child runs full registerBuiltinHandlers (incl. plugin discovery) to resolve
ONE handler; the job name is known from the row — a resolve-by-name path
would skip discovery for builtins. Matters only if isolation is ever used
for short jobs (documented as not the target). **Effort:** S. **Priority:** P3.
- [ ] **P3 — full checkout instrumentation via a Sql proxy.** **What:** the
CheckoutGauge covers raw/direct/reserved/tx seams only; tagged-template
traffic (most engine load) is untracked. A proxy around the postgres.js Sql
callable could count real checkouts — investigate cost/fragility before
building. **Why:** would turn the probe's "tracked subset" caveat into full
coverage. **Effort:** M. **Priority:** P3.
## Security-process follow-ups (filed with Wave 1 of the fix-wave campaign, 2026-08-14)
- [ ] **P2 — Vulnerability disclosure policy.** **What:** a written disclosure
process: advisory ownership, severity ladder, reporter acknowledgment SLA,
embargo windows, private patch review, supported-version/backport policy,
release timing, post-release rotation guidance. **Why:** private vulnerability
reporting is now enabled (#579) and a reporter has a channel, but a channel
without a process leaves triage decisions ad-hoc; a public PR diff can still
broadcast attack surface mid-embargo. **Context:** filed from the fix-wave
campaign's Codex review (CX-11); the campaign deliberately shipped only the
toggle + reporter acknowledgment. Start from the responsible-disclosure rules
already in CLAUDE.md and docs/RELEASING.md. **Effort:** M. **Priority:** P2.
## Code-smell fix-wave deferrals (filed at W0; plan: ~/.claude/plans/system-instruction-you-are-working-encapsulated-eclipse.md)
Each was individually decided as a deferral in the CEO/eng reviews of the
fix-wave plan; the wave series (W0.5W9, 3.4, 3.6) tracks its own scope there.
- [ ] **Full engine staged merge** (~10 domains onto shared query modules +
Dialect record). **Priority: P2.** Gated on the W9 two-slice pilot criteria
(structure+params+results parity on chronicle AND the searchKeyword/CJK
hard seam; ≥40% domain LOC cut; Dialect ≤~6 fields; query-builder extension
≤~150 lines). The terminal fix for the engine-divergence/JSONB class —
blast radius is the production hot path, hence pilot-gated. Blocked by: W9.
- [ ] **gateway.ts file split** behind a re-export facade (~121 import sites
unmoved). **Priority: P3.** After W8's behavior changes so the split is
pure motion; needs the CLAUDE.md engine-dynamic-import exemption-path
chasers + check-engine-dynamic-import.sh + build:llms.
- [ ] **BrainEngine 149-method interface → domain repos** (65 methods have
0-1 callers; 3 already deleted in W3). **Priority: P3.** Shape informed by
the W9 pilot's query-module seam.
- [ ] **Legacy Anthropic-SDK subagent loop deletion.** **Priority: P2.** One
release after W8 flips `agent.use_gateway_loop` default ON (flag stays as
the revert path for that release).
- [ ] **Deeper test-suite speedup** beyond the W0 snapshot default-on (which
already cut the full parallel suite ~4,900s → ~490s). **Priority: P3.**
Revisit with post-W0 timing data; diminishing returns until measured.
- [ ] **PGLite schema build-time derivation** from SCHEMA_SQL via a named
transform list. **Priority: P3.** Only if W3's schema drift TEST proves
annoying in practice — the test alone kills the drift bug class (Codex
D4.8/D5.23: fresh-schema equivalence ≠ upgrade correctness; old-shape
bootstrap fixtures + replay coverage stay regardless).
## Jobs fix-wave follow-ups (filed v0.45.15.0 — upstream issues #2/#3/#4)
- [ ] **P2 — `jobs submit --max-pending` public flag.** maxPending stays an
internal submit option this wave (Codex C4): its semantics exclude
delayed/paused/waiting-children rows, and identity is (name, queue, source)
so distinct payloads collapse. Decide the public contract (include delayed?
explicit scope key?) after the primitive soaks in autopilot, then mirror
parseMaxWaitingFlag (clamp [1,100]) + help + flag-registry regen + optional
submit_job MCP param. Where: src/commands/jobs.ts, src/core/operations.ts.
- [ ] **P2 — maxPending at the other single-flight dispatch sites.** The
freshness sync submit (src/commands/autopilot.ts freshness loop) and the
targeted remediation steps (autopilot.ts targeted-submit loop) still use
maxWaiting: 1; widening to maxPending changes behavior of those lanes
(suppression while a long run is active) and needs its own review. Where:
src/commands/autopilot.ts.
- [ ] **P2 — Help-stub sweep for the other CLI_ONLY commands.** The `jobs`
defect class exists elsewhere: `gbrain search modes --help` connects an
engine before help routing, and the search subcommands have no help guards
(jobs/bootstrap/skillpack now carry the guard pattern to copy). Audit every
CLI_ONLY member missing from CLI_ONLY_SELF_HELP; the top-level help promises
per-command help for all of them. Where: src/cli.ts, src/commands/search.ts.
- [ ] **P3 — jobs stats: fuller backpressure/audit surfacing.** The 24h
Backpressure line + suppressed-by hint shipped; per-decision breakdowns,
longer windows, and doctor integration remain (the audit file header's B4
follow-up). Where: src/commands/jobs.ts, src/core/minions/backpressure-audit.ts.
- [ ] **P3 — jobs watch: timeout/deadline column.** `jobs get` shows the
effective budget; the live dashboard doesn't. Where: src/commands/jobs-watch.ts.
- [ ] **P3 — jobs help + operator docs: handler catalog and dispatch-event
schema.** `gbrain jobs --help`'s HANDLER TYPES section lists 8 of the ~40
registered handlers, and the autopilot dispatch JSON events (`dispatched`,
`dispatch_coalesced`, `fanout_summary` with its `coalesced` array) have no
schema documentation outside the CHANGELOG. Where: src/commands/jobs.ts
(JOBS_HELP), docs/guides/queue-operations-runbook.md.
## Truthful-surface wave follow-ups (filed with T14, amendment 35 + D14.5)
Deferred from the MCP consumer-feedback wave (plan at
@@ -179,14 +336,6 @@ fix-now findings landed on the branch; these four are the review-deferred tail.
`requestToolsPersistLimiter`; the surface_change audit rows already give
a DB-side count to enforce against if needed. **Effort:** medium.
**Priority:** P3.
- [ ] **P3 — cancel (not just abandon) timed-out submit-time queue probes.**
**What:** the WP5 wedge/pause probes time-bound via Promise.race, but the
losing query keeps running on the pool after the race resolves. Wire
AbortSignal / statement_timeout so a slow probe releases its slot. **Why:**
under pool exhaustion (the exact regime the probes exist to detect) an
abandoned probe query holds a pooler slot and makes the exhaustion worse.
**Context:** `src/core/minion/supervisor.ts` queryWedgeSignals callers in
`src/core/operations.ts` submit paths. **Effort:** small. **Priority:** P3.
- [ ] **P3 — document the status --json snapshot union under schema_version.**
**What:** a short protocol note (docs/progress-events.md sibling) pinning
the `get_status_snapshot` v2 shape as a discriminated union on
@@ -286,7 +435,7 @@ Deferred from the BrainBench wave (eng-reviewed; plan + GSTACK REVIEW REPORT at
- [ ] **`--live` agent-in-the-loop know-to-ask.** Replay fixtures with a real model deciding whether to issue retrieval calls; grade the agent, not just the deterministic reflex. Pre-registered in `docs/eval/BRAINBENCH.md` (the v1 metric grades the injection decision, which IS the shipped mechanism). Needs: seeded N-repeat methodology for model stochasticity + budget rails. Priority: P2.
- [ ] **Intrusion-budget gating calibration.** `avg_injected_tokens` is reported, non-gating (decision 18) — a wrong threshold is worse than none. After a few weeks of scoreboard data across PRs, pick calibrated per-seam thresholds and promote it to a gated metric. Priority: P2.
- [ ] **Flip contract adapters to production — claude-code half now unblocked.** `adapters/claude-code.ts` exports the UserPromptSubmit hook wire types; the real hook (`gbrain hook user-prompt`, shipped with the bootstrap lane and extended with cross-turn dedupe + the channel feedback loop in the cathedral-3 convergence) swaps the in-process transport for an exec of the hook script and flips `seam: 'contract'``'production'` with continuous bench numbers. Note the production hook also exercises transcript-based dedupe, which the memoryless contract row deliberately doesn't. Same for codex fragments when that integration lands. Priority: P1 (the claude-code integration has landed; this is now standalone-actionable).
- [ ] **Flip contract adapters to production — claude-code half now unblocked.** `adapters/claude-code.ts` exports the UserPromptSubmit hook wire types; the real hook (`gbrain hook user-prompt`, shipped with the bootstrap lane and extended with cross-turn dedupe + the channel feedback loop in the cathedral-3 convergence) swaps the in-process transport for an exec of the hook script and flips `seam: 'contract'``'production'` with continuous bench numbers. Note the production hook also exercises transcript-based dedupe, which the memoryless contract row deliberately doesn't. For the codex half: the cathedral-4 transcripts lane shipped a verified codex rollout PARSER (`src/core/transcripts/codex.ts`, structural turn selection pinned against a live sample) — a codex contract adapter can now consume it instead of waiting for a hook integration. Priority: P1 (the claude-code integration has landed; codex parsing has landed; this is now standalone-actionable).
- [ ] **Cathedral 1 conformance-kit fixture import.** The memory-verbs conformance scenarios convert to BrainBench fixtures via the published `evals/brainbench/schema/fixture.schema.json` once `garrytan/cathedral-1` merges ("conformance tests double as BrainBench seed fixtures", decision log 2026-06-12). Free corpus growth from already-reviewed scenarios. Blocked by: cathedral-1 on master. Priority: P2.
- [ ] **Live-embeddings fidelity mode (`--embeddings`).** Hermetic CI grades the keyword/alias arms only (disclosed); an opt-in mode seeding real embeddings would grade write-back/continuity retrieval through the vector path. Same budget rails as `--llm`. Priority: P3.
- [ ] **Community fixture intake + competitor adapters.** The TD1 remainder after the generated corpus absorbed in-PR growth: an `external-authors/`-style intake path for contributed fixtures (validator + privacy guard already gate them) and adapters for non-gbrain memory systems against the published schemas, enabling true head-to-head rows in the gbrain-evals scorecard. Priority: P3.
@@ -694,6 +843,16 @@ job) and sync. See CLAUDE.md "Pace Mode".
supervisor-detection downgrade. Today these inherit config/env pacing only when
they call `runEmbedCore`.
- [ ] **P1-companion — Supervisor concurrency 3→2 + job-kind slot fairness (E7).**
**v0.45.15.0 annotation (jobs fix wave):** make the whole wedge-detector FAMILY
suppression-aware while here — the supervisor watchdog (supervisor.ts wedge
predicate) and doctor's `wedged_queue` check both require waiting > 0, and
`maxPending` single-flight keeps waiting at 0 while a job is in flight.
Mitigations already shipped: maxPending counts only LIVE-LOCK actives (a
dead/blocked worker's expired-lock row never suppresses, so fresh waiting rows
re-feed the detectors) and `jobs stats` prints a Backpressure line + a
suppressed-by hint. Remaining: teach watchdog/doctor to treat
recent-coalesces + stale live active as wedge signal; also note the worker
in-flight stall-check hole (worker.ts stall check skips when inFlight > 0).
The daemon-side root cause the external wrapper's probe was blind to:
`embed-backfill`/`autopilot-cycle` jobs can occupy all supervisor slots
(`:215` below). Pacing makes backfills safe; this fixes the residual death rate.
@@ -800,7 +959,10 @@ events at the IPC delivery point and dedupes via the transcript's
0700 dir) and (b) a secret-file home for `turn_context` auth (same hash-keyed run dir).
The cathedral-3 branch prototyped (a) as `resolveSocketPathForConfig` (see branch
history at commit 2350294c) before the convergence dropped it pending the secret
design. **Trigger:** a Postgres-brain user asking why hooks stay silent. **Start:**
design. **Trigger:** a Postgres-brain user asking why hooks stay silent — and as of
#4043, every `gbrain bootstrap harness` install on a Postgres brain: harness mode
pre-wires all five hooks and states the degradation plainly, so this listener is what
lights them up. **Start:**
`src/core/context/resolve-ipc.ts` socket-path helpers + `src/mcp/server.ts` listener gate
+ `src/commands/hook.ts:no_pglite_path` branch.
- [ ] **P3 — thin-client remote push route.** Thin-client installs (remote_mcp) have no
@@ -1244,11 +1406,14 @@ but were deliberately scoped OUT — neither is a #1784 regression.
deserves its own deliberate change. Fix: mirror the extracted
`buildCostRefusal({json, ...})` helper (`reindex-code.ts`). The guardrail
(exit 2, no spend) stays; only the FORMAT splits on `--json`.
- [ ] **P3 — `gbrain jobs --help` has no subcommand list.** jobs.ts dispatches
- [x] **P3 — `gbrain jobs --help` has no subcommand list.** jobs.ts dispatches
on a bare subcommand string with no HELP const, so `watch` (and every other
jobs subcommand) is undocumented in `--help`. The new `watch` `--json` /
`--follow` flags are documented only in the file JSDoc. Add a HELP table to the
`jobs` command listing every subcommand + its flags.
**Completed:** v0.45.15.0 (2026-08-14) — JOBS_HELP + JOBS_SUBCOMMAND_HELP with a
guard above the thin-client refusal; `jobs`/`jobs work` etc. `--help` print real
usage engine-free and can never start a daemon.
## v0.42.12.0 self-upgrade follow-ups (v0.43+)
@@ -1420,7 +1585,11 @@ and tested; these are documented tradeoffs and stronger-but-bigger versions.
Deferred from the v0.41.38.0 wave (code-callers/callees pin + dream-on-postgres).
Documented tradeoffs, not blockers — the shipped bug fixes are complete and tested.
- [ ] **P1 — Per-source autopilot fan-out passes the global repoPath.**
- [x] **P1 — Per-source autopilot fan-out passes the global repoPath.**
**Completed (verified already fixed):** v0.45.15.0 audit (2026-08-14) — the
handler binds FS phases to the source's `local_path` and never falls through
to the global repoPath (`effectiveBrainDir = sourceId ? sourceLocalPath :
repoPath` in src/commands/jobs.ts, with per-source null → skip FS phases).
`src/commands/autopilot-fanout.ts:~206` submits every per-source `autopilot-cycle`
job with `repoPath: opts.repoPath` (the global checkout), not `src.local_path`.
With v0.41.38.0's `cycleSourceId = opts.sourceId ?? resolveSourceForDir(...)`,
@@ -1786,18 +1955,14 @@ single canonical `src/core/model-pricing.ts` with `canonicalLookup`.
operator pipes directly into `crontab -e` instead of copy-paste-massage.
~80 LOC. Mirrors `gbrain sync --break-lock` argv shape.
- **TODO-OPS-2 (P2)**: Lock-loss detection — extend `DbLockHandle.refresh()`
to throw `LockLostError` on 0 rows affected. Codex caught during the
v0.41.19.0 plan review: `refresh()` runs `UPDATE ... WHERE holder_pid = pid`
with no rows-affected check (`db-lock.ts:108-114`, `:151-156`). If the
TTL expired and another worker took over, the original keeps writing
silently. v0.41.19.0 ships TTL=5min + active in-phase refresh via
`buildYieldDuringPhase` which makes the race window much narrower, but
an `await chat()` call that exceeds the 5min wallclock window can still
hit it. Fix: `RETURNING id` on the UPDATE + check `rows.length === 0`
throw tagged `LockLostError`. Phases catch + abort cleanly (write partial
progress, return `status: 'fail'` with reason `'lock_lost'`). Behavioral
contract change with phase-abort fallout; needs its own design pass.
- [x] **TODO-OPS-2 (P2)**: Lock-loss detection — CLOSED by the W0 fix-wave
(code-smell series). `refresh()` now runs a FENCED update (id + holder_pid +
epoch-rendered `acquired_at`) with `RETURNING id`, returns `false` on 0
rows, and runCycle's steal controller aborts the run at the next boundary
with a structured `reason: 'lock_stolen'` partial report (LockStolenError;
raced awaits cover the 5 long phases). The supervisor exits LOCK_LOST
immediately on a fenced miss. Pinned by `test/db-lock-fencing.test.ts` +
`test/cycle-lock-steal.serial.test.ts`.
## v0.41.20.0 status + doctor-categories wave follow-ups (v0.42+)
@@ -4176,7 +4341,7 @@ purpose; needs baseline-governance care per the BrainBench gate rules.
### PTY-mode transcript capture
**Priority:** P3
**What:** `transcript-capture.ts` currently uses plain `child_process.spawn` pipes. Some agents only emit ANSI colors / progress UI on a TTY. v1.1 adds a PTY mode (likely via `node-pty`) so live-mode transcripts capture the full agent UX.
**What:** `transcript-capture.ts` currently uses plain `child_process.spawn` pipes. Some agents only emit ANSI colors / progress UI on a TTY. v1.1 adds a PTY mode so live-mode transcripts capture the full agent UX. Do NOT add node-pty for this: Bun's built-in `terminal:` spawn option (Bun 1.3.10+, already pinned in engines) is the dependency-free path, proven by `test/helpers/tty-harness.ts` — reuse `launchTty` or its spawn shape.
**Why:** Faithful transcripts make the friction → reasoning link more useful. v1 accepts that some agent UI is lost.
@@ -4184,6 +4349,28 @@ purpose; needs baseline-governance care per the BrainBench gate rules.
---
### Non-tier-1 e2e files run in no required CI lane
**Priority:** P2
**What:** Unit shards exclude `test/e2e/*` (`scripts/test-shard.sh`), and `.github/workflows/e2e.yml` runs only explicitly named files (a handful across its jobs — e.g. `test/e2e/mechanical.test.ts`, `test/e2e/mcp.test.ts`, the jsonb-parity pair); there is no glob. Every other `test/e2e/*.test.ts` — including PGLite-only files that need no `DATABASE_URL`, like `init-fresh-pglite.test.ts` — executes only when someone runs `bun run test:e2e` by hand. Decide per file: wire into a required workflow, re-home PGLite-only files to the serial lane (the pattern `test/init-picker-pty.serial.test.ts` uses), or explicitly document them as manual-only.
**Why:** Tests that never run in required CI are silent coverage loss — they rot without failing. Surfaced by the TTY-harness cleanup review when the new PTY picker test almost landed in the same dead lane.
**Effort:** S-M (CC ~30-60m for the audit + re-homing; workflow wiring adds CI-minutes cost per file).
---
### Ctrl-D during `gbrain init` stalls 60s at the next prompt (readLineSafe does not latch EOF)
**Priority:** P2
**What:** Pressing Ctrl-D at the interactive provider picker is detected immediately (keyless fallback in ~200ms), but Bun's stdin never yields another line after EOF while `isTTY` stays true — so the SUBSEQUENT search-mode picker sits its full 60s `readLineSafe` fallback before init completes (probed under a real PTY: keyless notice at 0.2s, mode prompt rendered at 1.2s, exit at 61.1s). Fix: `readLineSafe` (src/commands/init.ts) should latch EOF — once stdin has ended, later calls return their default immediately instead of waiting out the timer. Regression test: extend the EOF case in `test/init-picker-pty.serial.test.ts` to run init to completion and assert exit well under the fallback window (the case currently closes early on purpose to keep the 60s stall out of required CI — see the comment there).
**Why:** A user who hits Ctrl-D at the first prompt stares at a frozen screen for a full minute before init finishes. Cross-model adversarial review finding (Codex), confirmed by a real-PTY probe.
**Effort:** S (CC ~20m: EOF latch + regression-test extension).
---
### Read-side host-isolation (`$GBRAIN_HOST_HOME`)
**Priority:** P3
@@ -5408,6 +5595,91 @@ respective shapes. Small, mechanical; pinned by `test/init-embed-check.test.ts`
**Depends on:** nothing.
## Harness-mode follow-ups (#4043, filed at build time)
- [ ] **P2 — serve-side port/pid record for discovery.** `gbrain bootstrap harness`
and its `--status` probe `/health` at 127.0.0.1:3131 (or an explicit `--url`/`--port`);
a serve on a non-default port is invisible without flags. Write a record (port, pid,
started_at) from `runServeHttp`'s `app.listen` callback into `~/.gbrain/run/`,
mtime-as-heartbeat like `src/core/autopilot-paths.ts` — the stale-record semantics
(crashed serve, multi-serve boxes) are why this deferred; a wrong record misdirecting
probes is worse than no record. **Trigger:** a harness box running serve on a custom
port asking why discovery misses it. **Start:** `src/commands/serve-http.ts` listen
callback + `src/core/bootstrap/harness.ts` url resolution.
- [ ] **P3 — legacy HTTP transport scope asymmetry.** `src/mcp/http-transport.ts` is
test-only (no production caller; `serve --http` uses serve-http.ts) and hardcodes
`scopes: []` with no per-op scope gate — if it is ever revived, a scoped legacy token
is fully UNSCOPED there. Mirror the `scopes TEXT[]` honor + `hasScope` dispatch gate
before any revival. **Trigger:** any production caller of `startHttpTransport`.
- [ ] **P3 — partial unique index on active `access_tokens.name`.** Names are not
unique; `auth revoke <name>` clears every active row and the 23505 handler in
`auth create` is dead code for name collisions. Harness mode sidesteps this with
revoke-by-id + receipt-carried ids, but a
`CREATE UNIQUE INDEX ... ON access_tokens (name) WHERE revoked_at IS NULL` would
make names honest for humans too. Needs a dedup pass first on brains that already
carry twins. **Start:** `src/core/migrate.ts` (CONCURRENTLY + `transaction: false`).
- [ ] **P2 — codex hook lane.** codex-cli 0.147.0 ships a real hook system (hooks.json;
PreToolUse…SessionEnd — recorded on `TARGETS['codex-2026-08']` in
`src/core/bootstrap/host-specs.ts`), falsifying the old "codex has no hooks" premise.
Wiring SessionEnd transcript capture (+ SessionStart context) would give codex
sessions the same memory loop Claude Code gets, and supersedes the FF2 notify-sweeper
idea. Needs its own dated spec-target verification (payload shapes, deny-unknown-fields
config) + e2e before any writer lands. **Trigger:** first user asking why codex
sessions don't persist; **Start:** `host-specs.ts` TARGETS + a codex sibling of
`writeClaudeHooksAt`.
- [ ] **P3 — PGLite admin-lane scoped minting.** `gbrain bootstrap harness` refuses to
mint under a live PGLite serve (single-writer) and points at pre-mint + `--token`.
Auto-driving `POST /admin/login` + `POST /admin/api/api-keys` (when
GBRAIN_ADMIN_BOOTSTRAP_TOKEN is present) would erase that friction — BLOCKED ON
extending that admin route to carry a scopes/permissions payload (today it inserts
only id/name/token_hash, so it can only mint full-access tokens, defeating the
harness lane's least-privilege default). **Start:** `src/commands/serve-http.ts`
api-keys route + `src/core/bootstrap/harness.ts` mint seam.
- [ ] **P3 — OpenClaw plugin setup hook (self-demoted from the #4043 wave).** The
issue's closing ask is "frameworks call `gbrain bootstrap harness` at setup time".
The in-repo `openclaw.plugin.json` cannot express it: OpenClaw installs plugins with
lifecycle scripts disabled (`--ignore-scripts`) and the manifest schema has no
setup/command field (verified against the OpenClaw plugin docs, 2026-08-12). When the
plugin API grows a setup surface, add `gbrain bootstrap harness --yes` AND remove the
manifest's static stdio `mcpServers.gbrain` entry in the same commit (one owner per
server name). **Trigger:** OpenClaw plugin-API setup/command support shipping.
- [ ] **P3 — harness federated-drift visibility.** The harness token's
`permissions.source_id` federation array is a mint-time snapshot of the
`federated=true` sources; sources added later are invisible to wired sessions until
a re-run rotates the token. `--status` could diff the snapshot against the live
config and suggest a re-run — needs either an engine open (breaks status's
engine-free posture under a live PGLite serve) or a sources probe over MCP with the
recovered token. **Start:** `src/core/bootstrap/harness.ts:statusHarness`.
- [ ] **P3 — harness smoke: add BRAIN-IDENTITY comparison on top of the canary
(ship-review residual).** The ship-review batch landed the two cheap layers: an
apply-time CANARY (a random same-format bearer must fail auth before the real smoke —
an impostor cannot tell the canary from the real token, so it is caught whichever way
it answers) and immediate revocation of the fresh mint on any failed smoke. The
remaining hardening is comparing the smoke's returned identity against the local
brain's (the default mint path already opens the engine and could capture it);
registrar mode (`--token` + remote url) has no engine and would state the weaker
guarantee honestly. **Start:** `src/core/bootstrap/harness.ts` steps 5+8.
- [ ] **P3 — harness orphan-mint reconciliation (red-team finding).** A hard crash in
the window between the mint INSERT committing and the `receipt.token.id` save leaves
an ACTIVE token no receipt records — `--remove` cannot revoke it and doctor never
flags it. On apply, when the prior receipt has `minted: true` but no id, list active
`access_tokens` rows matching `token.name` created after `receipt.created_at` and
fold them into `previous_ids` (or surface them loudly). **Start:**
`src/core/bootstrap/harness.ts` step 5 + `src/core/token-mint.ts`.
- [ ] **P3 — bootstrap lock.ts error-path polish (plan micro-item, deferred at ship).**
Non-EEXIST mkdir errors (EACCES/EROFS) misreport as BOOTSTRAP_IN_PROGRESS, and the
missing-dir message says "workspace directory" even when the lock target is the
gbrain HOME (harness lane) or a host config dir. Add an accurate message path.
**Start:** `src/core/bootstrap/lock.ts:acquireBootstrapLock`.
- [ ] **P3 — dedupe `auth create` against `mintLegacyToken`.** `src/commands/auth.ts`
create() re-implements the INSERT + `{a,b}` text[]-literal trick that token-mint.ts
owns (the extraction note says so); routing create() through `mintLegacyToken` (the
engine is in scope inside `withConfiguredSql`) would leave one canonical mint. Same
for the doctor's inline `/health` probe vs `probeServeHealth`, which also wants an
injectable fetch seam so `bootstrap_harness_health` tests stop making real TEST-NET
calls (3s each). **Start:** `src/commands/auth.ts:create`, `src/commands/doctor.ts`
bootstrap_harness_health.
## Agent-bootstrap wave follow-ups (filed at build time)
- [ ] **P2 — repoPhaseComplete is single-workspace (one global receipt).** The
@@ -5654,10 +5926,14 @@ respective shapes. Small, mechanical; pinned by `test/init-embed-check.test.ts`
- [ ] **P2 — `gbrain ingest feed`: native feed adapter.** blog-ingest ships the
agent-procedure layer; the durable path is a deterministic RSS/Atom adapter
(discovery, pagination, canonical-URL dedup, 429 backoff) behind one command.
- [ ] **P2 — Native AI-chat export importer.** conversation-archive converts
ChatGPT/Claude/Perplexity exports via agent procedure; a native importer
(export JSON → conversations/ pages) makes it deterministic. Pairs with the
existing conversation-parser surface.
- [x] **P2 — Native AI-chat export importer.** **Completed:** v0.46.0.0 (2026-08-14).
`gbrain transcripts ingest` imports extracted ChatGPT and Claude.ai
`conversations.json` exports natively (adapters at
`src/core/transcripts/{chatgpt-export,claude-export}.ts`, rendering on the
conversation-parser surface). Perplexity has no adapter yet — a candidate
leaf module on the same `TranscriptAdapter` seam (the pattern the
cathedral-4 "More harness adapters" follow-up below documents); the
conversation-archive skill keeps the manual procedure for it meanwhile.
- [ ] **P2 — Entity-guard as a native op.** phonetic-name-guard's own changelog
proves prose-only failed: ASR-variant entity collisions need a native check
(registry + alias table consulted at put/import time). The wave shipped the
@@ -5716,3 +5992,253 @@ respective shapes. Small, mechanical; pinned by `test/init-embed-check.test.ts`
subsystem that deserves its own eng + CEO review, not a rider. The currency work
(`skillpack status`/`sync`, doctor `skill_currency`) already keeps the brain's skill
set current on upgrade; this item is purely about semantic retrieval of skills.
## opencode wave follow-ups (filed at build time)
- [ ] **P2 — Watch the first opencode-door + canary dispatches.** The job is
day-one full posture (nightly + labels; keyless SMOKE + paid anthropic leg
on the existing secret) — after the wave merges, confirm the first nightly
run goes green end-to-end and the canary leg's latest-version result, then
update OPENCODE-CLI-PIN.md §Pending auth with anything the authed CI run
observes (exact `opencode models` output, per-turn cost note). Effort: S.
- [ ] **P3 — Wire opencode's plugin/event system** (the ambient-recall lane).
opencode ships a JS plugin system with lifecycle events; `OPENCODE_HAS_HOOKS
= false` in host-specs.ts marks the gap. Needs its own observation pass
(plugin API shapes, event timing, context-injection surface) before design —
would upgrade opencode from pull-protocol to per-turn push, above codex.
Effort: M/L.
- [ ] **P3 — BrainBench opencode adapter.** `src/eval/brainbench/adapters/` +
`ALL_HARNESSES` entry — build together with the already-filed hermes + grok
adapters (three pending; one eval wave). Effort: M.
- [ ] **P3 — connect `--agent opencode --oauth`.** opencode's `mcp auth` is an
authorization-code OAuth flow (not client-credentials) — a connect lane for
it needs the interactive-grant plumbing the current `--oauth`
(perplexity/generic client-credentials) path does not model. Effort: M.
- [ ] **P3 — Re-observe the OPENCODE_CONFIG* env trio on version bumps.**
Observed INERT in 1.18.18 (docs-contradiction pinned in OPENCODE-CLI-PIN.md
§Path seams); host-specs resolves via XDG only. If a future release
activates them, `opencodeConfigDir()` and the hermetic child-env deletes
must move together. The pin doc's re-observation checklist carries the
probe. Effort: S.
- [ ] **P3 — opencode-install PTY promotion.** Same criterion as grok-install:
2 consecutive stable dx-scenario runs ≥1 month apart with unchanged
boot/first-run copy → promote to a PTY assertion test. opencode's keyless
free tier means the scenario should COMPLETE the bootstrap, making it a
stronger promotion candidate than grok's sign-in-wall early-stop. Effort: M.
## Transcripts-import follow-ups (filed from cathedral-4, `gbrain transcripts ingest`)
Scoped OUT of the cathedral-4 PR by the CEO review's cherry-pick ceremony and the
eng review — each carries a named design, none is a bug. Context: the import lane
(adapters at `src/core/transcripts/`, session-atomic pipeline, embed-OFF default)
covers DEAD logs; go-forward capture beyond Claude Code is deliberately absent.
- [ ] **OpenClaw go-forward capture.** Blocked upstream: the OpenClaw PluginApi exposes only `registerContextEngine` — no end-of-turn/agent-end capability. When the host grows one, the plugin (`src/openclaw-context-engine.ts`) subscribes and emits the session into the corpus lane (`~/.gbrain/transcripts/corpus` sidecar protocol) the way `gbrain hook session-end` does for Claude Code; the openclaw session PARSER already ships. Consent must ride a capture line like the bootstrap harness `--no-capture` model. Priority: P2.
- [ ] **Codex go-forward capture (notify sweeper).** `docs/designs/AGENT_BOOTSTRAP_PLAN.md` FF2 names the design (notify sweeper over `~/.codex/sessions`); the rollout parser now ships in `src/core/transcripts/codex.ts`, so the sweeper is pure wiring: on codex notify, run `gbrain transcripts ingest <rollout> --quiet`. Needs the same consent posture as capture. Priority: P2.
- [ ] **Scheduled re-import cycle phase.** `transcripts ingest --since last --all` as an opt-in cycle phase so dead-log import self-refreshes. REQUIRES its own consent-line design first: reading harness dirs on a schedule is capture-adjacent (the "Autonomous transcript watchers" decision above rules the spirit); the clean-scan watermark + status gap table already make manual re-runs cheap. Priority: P3.
- [ ] **PII auto-detection redaction pass for imports.** The native lane redacts secrets (secret-scan) + user patterns (`harvest-private-patterns.txt`, emails included) and counts imperatives; broad PII detection (names, phones, addresses) is its own subsystem — the conversation-archive skill keeps the human scrub step for sensitive corpora meanwhile. Priority: P2.
- [ ] **More harness adapters: Cursor / Gemini CLI / Copilot CLI.** Leaf modules on the `TranscriptAdapter` seam (~1h each with an agent): dated SPEC_TARGET + scrubbed fixture + drift alarm, per the shipped six. Formats unverified locally — verify a real sample first (the hermes gate pattern). Priority: P3.
- [ ] **ChatGPT/Claude.ai export zip unwrapping.** v1 requires the EXTRACTED `conversations.json` ("unzip first" is documented + error-hinted). Add zip handling without a heavy dependency (Bun has no built-in zip; evaluate a minimal vendored inflate or shelling to `unzip` with confinement). Priority: P3.
- [ ] **BrainBench raw-format fixture schema (sibling repo).** The in-repo pin (`test/e2e/transcripts-writeback-fidelity.test.ts`) grades raw files through the adapters with the gold extractor, but the BrainBench corpus schema (gbrain-evals) still rejects unknown keys and its corpus hash doesn't cover raw sidecars. Needs: versioned raw-fixture sidecar type + loader + hash coverage + baseline re-cut in gbrain-evals, then a `write_back_fidelity_raw` suite row here. Priority: P2.
- [ ] **Hermes SPEC_TARGET verification against a populated store.** The schema came from the installed hermes-agent v0.20.0 source (`SCHEMA_SQL`), but no populated `state.db` existed on the dev machine — the fixture is synthetic-by-declaration. Verify against a real store after some Hermes sessions accrue, then flip `status: 'provisional'``'verified'` and pin the `active`/`compacted` semantics the adapter currently ignores. Priority: P3.
## Grok Build wave follow-ups (filed at build time)
- [ ] **P1 — Enable the grok-door paid lane once XAI_API_KEY exists.** Admin
creates the `XAI_API_KEY` repo/environment secret (console.x.ai; prefer a
protected GitHub Environment scoped to door jobs), then one commit re-adds
to `grok-door` in heavy-tests.yml: the `schedule` leg, the `heavy-tests`
label leg, default-on dispatch, and a latest-version CANARY matrix leg
(`continue-on-error`, schedule-scoped, own timeout) so the pinned lane stays
deterministic while the canary tracks what users run. Same session: run the
pending-auth Phase-0 observations (paid one-shot smoke, authed model list +
measured per-turn cost pins, credential-file inventory after login → door
evidence exclusions + TTY secretPaths, authed first-run TUI copy) into
`docs/mcp/GROK-CLI-PIN.md`, and pin `parseGrokJson` + the separate
non-retried JSON toolCall door test (one extra paid turn) once the
streaming-json event shape is observed. Effort: S (CC ~30min + admin).
- [ ] **P2 — `gbrain connect --agent grok`.** One-command install UX:
`AgentId`/`AGENT_SPECS`/`AGENT_IDS` in `src/commands/connect.ts`, a
`buildGrokMcpAddArgv` in `src/core/mcp-registration.ts` (shape already
pinned in GROK-CLI-PIN.md), connect tests ("all four agents" pin moves to
five), KEY_FILES entry. Deferred from the grok wave to avoid a second
observation pass; the pin doc now exists, so this is mechanical. Effort: S.
- [ ] **P2 — HERMES.md surface refresh.** The hermes register command predates
the truthful-surface wave and wires the full 100+-op catalog;
CLAUDE_CODE.md + GROK.md now recommend `--surface verbs`. Update the
register one-liner + Direct config block (+ INSTALL_FOR_AGENTS hermes
block) and re-verify against the pinned hermes. Effort: S.
- [x] **P2 — Backport the GITHUB_ENV/GITHUB_PATH/GITHUB_OUTPUT/GITHUB_STATE
deletion from `grokChildEnv` to `hermesChildEnv`** (and consider narrowing
the `GITHUB_` ALLOW_PREFIX to the read-only metadata names) — the prefix
rule forwards writable CI step-metadata files to untrusted agent children.
Unit truth-table exists for the grok side to clone. Effort: S.
DONE (opencode-support wave): `hermesChildEnv` now rides `makeAgentChildEnv`,
which scrubs the GITHUB_* step-metadata files for every door agent; truth-table
extended in `test/helpers/agent-harness.unit.test.ts`.
- [ ] **P3 — Grok bootstrap-harness target.** `gbrain bootstrap` personal-agent
support for Grok Build: `HarnessSelector` + `parseHarnessArgs`, a dated
`TARGETS` spec in `host-specs.ts`, a `wireGrok` branch + TOML writer (grok
config schema pinned; `codex-toml.ts` is the precedent), receipt/rollback/
status handling, and the INSTALL_FOR_AGENTS honest-classification flip.
Docs currently state "bootstrap does not support Grok yet". Effort: M.
- [x] **P3 — Door-adapter extraction (test-side) + door cadence policy.**
Trigger FIRED at the 4th door agent (opencode, the opencode-support wave):
`makeBinaryResolver`/`makeAgentChildEnv`/`runOneShotSpawn` extracted in
`test/helpers/agent-harness.ts`, grok+hermes ported (hermes gained the
GITHUB_* scrub + bounded drain), opencode landed as first consumer; the
cadence policy is adopted in `docs/TESTING.md` (nightly for the newest
agent, label-only after 2 stable monthly cycles).
- [ ] **P3 — Door CI-tail composite action.** Trigger: the FIRST GREEN
grok-door AND opencode-door dispatches (workflow yaml cannot be proven
locally, and refactoring never-run jobs compounds risk — grok-door has
never dispatched: its XAI_API_KEY secret does not exist yet). Hoist the
shared workflow tail (evidence prep / scrub triple / upload / pass-count +
paid sentinels / version re-check / cred cleanup) from
hermes-door/grok-door/opencode-door into a composite action; port
opencode-door as first consumer (it is the freshest copy). Until then the
three doors' scrub blocks carry cross-reference comments. Effort: M.
- [ ] **P3 — Promote grok-install to a PTY assertion test.** Criterion: 2
consecutive stable runs ≥1 month apart of the dx scenario (pre-ship ritual
on grok-touching waves) with unchanged boot/sign-in copy. Would be the
repo's first gbrain-driving PTY assertion test — keep it an instrument
until the copy proves stable. Effort: M.
- [ ] **P3 — Nightly cross-agent friction-diff artifact.** After door runs,
`gbrain friction diff --base <hermes-run> --compare <grok-run>` rendered
into a CI artifact so guide-following friction regressions surface without
a dev-box session. Effort: S/M.
- [ ] **P3 — `xai:` provider block in model-pricing.ts.** Grok models
(grok-4.6/4.5 observed) for cost views once xAI pricing is sourced;
separate concern from the harness wave (CANONICAL_PRICING discipline).
Effort: S.
- [ ] **P3 — BrainBench grok adapter.** `src/eval/brainbench/adapters/` +
`ALL_HARNESSES` entry — same seam as the already-filed hermes adapter
(TODOS "BrainBench hermes adapter"); build both together. Effort: M.
- [ ] **P3 — Client-registry unification (Approach C).** The repo carries 7
hardcoded client lists (connect AGENT_SPECS, bootstrap Harness,
HarnessSelector, host-specs TARGETS, claw-test registry, brainbench
ALL_HARNESSES, volunteer HARNESS_CHANNELS); grok proved the claw-test
registry shape generalizes. Unify into one data-driven table AFTER the
door-adapter extraction lands (earn it — don't freeze hermes-isms in).
Effort: L.
- [x] **P3 — PIN-doc privacy guard.** DONE (opencode-support wave):
`scripts/check-pin-doc-privacy.sh` (in `bun run verify` + guards-manifest,
fixture-tested) asserts every `docs/mcp/*-CLI-PIN.md` uses placeholder paths
and carries no key-shaped material or non-example emails.
- [x] **P3 — opencode-door npm view-vs-install TOCTOU.** DONE (adversarial-review
fix wave): the door job's install step is now pack-verify-install — `npm pack
<pkg>@<ver> --json` downloads the artifact and reports the integrity of the
BYTES written; both the wrapper and the platform payload are asserted against
their pins before `npm install -g ./opencode-ai-*.tgz` installs from the
verified local tarball (no fresh registry resolve of the name; the payload's
install-time fetch is npm-validated against the same byte-confirmed packument).
Verified locally on darwin-arm64 (wrapper integrity == pin; `--ignore-scripts`
breaks opencode's postinstall binary placement, so it is deliberately absent).
- [x] **P3 — `opencode mcp list` probe spawns project-config servers.** DONE
(adversarial-review fix wave): the user-scope probe spawns from a fresh EMPTY
mkdtemp cwd (no project config can load), project scope SKIPS the live probe
entirely with a printed note (parse-back is authoritative), and the probe now
holds the real process handle so the 20s timeout actually kills the child
(SIGTERM → SIGKILL) instead of abandoning it.
- [ ] **P3 — dedupe the opencode read→parse→classify dance.** The
read-config → parseOpencodeConfig → opencodeEntryKind sequence is spelled
three times (bootstrap.ts runHooks pre-check, harness.ts apply expectUrl
fallback, harness.ts remove ownership check); extract a
`classifyOpencodeEntryAt(path, name, expect)` helper and drop the
double-printed other-source warning (the caller AND the writer note it).
Effort: S.
## opencode adversarial-review fix-wave follow-ups (filed at fix time)
- [ ] **P2 — per-harness MCP-scope consent key.** An interview MCP_SCOPE answer
recorded for Claude Code (where 'project' is the privacy-SAFE default)
currently authorizes opencode's INVERTED-risk scopes without fresh
confirmation ('project' on opencode = committed file that auto-spawns on
every collaborator machine, no trust gate), and an ABSENT answer defaults
opencode to user-global exposure (any repo on the machine reaches the
brain). Design a harness-specific consent confirm — either per-harness
answer keys (MCP_SCOPE_OPENCODE) or a one-time "your recorded scope means
something riskier here — confirm" gate on the opencode lane. Relates to the
agent-bootstrap A8 consent-semantics TODO. Effort: M.
- [ ] **P3 — opencodeEntryKind remote ownership: normalize the url compare.**
Ownership uses exact string equality on the entry url vs the receipt/expect
url — trailing-slash and host-case variants misclassify in BOTH directions
(ours read as foreign → orphaned entry; a variant-url foreign endpoint
never matches, fine, but the asymmetry is accidental). Consider URL
normalization (scheme/host case-fold, trailing-slash) plus an
Authorization-shape check before comparing. Effort: S.
- [ ] **P2 — claw-test --live runners inherit real HOME/XDG.** The grok /
hermes / opencode --live runners run against the operator's real
HOME/XDG config surface and only WARN on a pre-existing global gbrain
entry; a scripted run can mutate or exercise the operator's live wiring.
Consider a fail-closed flag (refuse when a global gbrain registration
exists unless --allow-live-config) or hermetic-by-default across the
runner family. Effort: M.
- [ ] **P3 — fixed-name `.bak` parity: codex-toml.ts + hooks.ts writers.**
opencode-json.ts now takes UNIQUE `.bak-<hex>` backups per operation
(overlapping runs can't clobber each other's snapshot; harness restores
from the returned path and unlinks on success). The codex TOML writer and
the hooks settings writers still use fixed-name backups with the same
theoretical overlap window — port the unique-backup pattern (and the
restore-guard compare) for parity. Effort: S/M.
## Dream triage cascade follow-ups (#4152, filed at implementation)
- [ ] **P2 — Incremental submit-drain + deadline threading in synthesize
fan-out.** What: restructure the fan-out to submit bounded batches and
drain each before submitting more, stopping against the parent job's
`deadlineAtMs`. Why: today the phase bulk-submits every accepted child
then drains sequentially inside `autopilot-cycle`'s 30-min wall clock
(`handler-timeouts.ts:44`); a timeout mid-drain strands the remainder in
the run's private queue (the C1 self-heal + retriage conversion now
recover them, but not creating strands beats recovering them). Blocked
by: `runCycle` does not thread deadline/abort into phases (verified
absent at the synthesize call site, cycle.ts ~2030). Context: outside
voice C2 on the #4152 eng review; the triage `max_ms` budget bounds the
cheap half, this bounds the expensive half. Effort: M/L.
- [ ] **P2 — Scheduled reject sample-audit with spend-posture
integration.** What: automate `dream retriage --audit-rejects N` on a
cadence (weekly cron or post-cycle sampling) writing disagreement-rate
telemetry, gated by `spend.posture`. Why: the threshold is an
intuition-set 0.5 until real false-negative data exists; the cascade
literature is unanimous that unaudited gates drift (eng-review search
check). The manual flag ships with #4152; this files the loop that runs
without an operator remembering. Depends on: a few weeks of production
score distributions. Effort: M.
- [ ] **P3 — Borderline-band routing (0.300.49 → mid-tier model or batch
digest).** What: a second lane where near-threshold files get a cheaper
treatment instead of the binary keep/drop. Why: the issue marked it
optional; it adds a third model lane + a second threshold pair, which
should be tuned from `details.triage` score distributions rather than
guessed. Blocked by: production calibration data (see the audit TODO
above). Effort: M.
- [ ] **P3 — Source×corpus multiplier: per-source corpus mapping or
explicit fan-out consent.** What: `dream.synthesize.session_corpus_dir`
is GLOBAL config while synth idempotency keys are SOURCE-namespaced, so
N registered sources each re-fan the same corpus (a live deployment saw
3 × ~1,250 jobs/day of the same files). Triage verdicts are
source-agnostic (judged once) and the cascade cuts each source's fanout
by the pass rate, but total synthesis is still N× the corpus. Why
deferred: pages land per-source, so per-source synthesis may be intended
semantics for some operators — needs its own issue + design (per-source
corpus config keys vs an explicit multi-source consent flag). Diagnostic:
`dream retriage --reconcile-queue --json` reports `queue.by_source`.
Context: outside voice C3 argued root-cause-first; scoped out twice
during the #4152 review. Comment on #4152 after ship. Effort: M.
- [ ] **P3 — Dream triage perf follow-ups (from the #4152 ship review).**
What: (a) batch the per-file `getDreamVerdict` PK probes in `runTriagePass`
into one prefetch (unnest join on (file_path, content_hash)) and reuse it
for retriage's spend-estimate loop (currently 2×N sequential roundtrips on
the operator sweep); (b) a partial index for `countRecentSynthSubmissions`
(`(created_at) WHERE name='subagent' AND idempotency_key LIKE
'dream:synth-v2:%'`) so the opt-in daily cap's count is index-served on
busy brains; (c) a shared `seedTriageVerdict` test helper to collapse the
five hand-rolled triage-v1 seed blocks. Why: all flagged by the ship
review's performance/maintainability specialists; none block — cache
probes are ~0.1% of adjacent LLM latency and the cap is default-off.
Effort: M.
- [ ] **P3 — Per-file single-flight for triage cache misses.** What:
concurrent passes (retriage while a cycle runs) can double-judge the same
uncached file (~1¢/file, last-write-wins converges — benign but untidy);
a per-(file,hash) advisory claim would dedupe. Why deferred: real locks
are heavy machinery for a benign-cost race; the retriage help documents
the behavior. Context: outside-voice CX5 on the #4152 ship review.
Effort: M.
+1 -1
View File
@@ -1 +1 @@
0.45.13.0
0.46.4.0
+3
View File
@@ -27,6 +27,7 @@
"gray-matter": "^4.0.3",
"heic-decode": "^2.1.0",
"js-yaml": "^3.15.1",
"jsonc-parser": "^3.3.1",
"marked": "^18.0.2",
"openai": "^4.0.0",
"pgvector": "^0.2.0",
@@ -469,6 +470,8 @@
"json-schema-typed": ["json-schema-typed@8.0.2", "", {}, "sha512-fQhoXdcvc3V28x7C7BMs4P5+kNlgUURe2jmUT1T//oBRMDrqy1QPelJimwZGo7Hg9VPV3EQV5Bnq4hbFy2vetA=="],
"jsonc-parser": ["jsonc-parser@3.3.1", "", {}, "sha512-HUgH65KyejrUFPvHFPbqOY0rsFip3Bo5wb4ngvdi1EpCYWUQDC5V+Y7mZws+DLkr4M//zQJoanu1SP+87Dv1oQ=="],
"kind-of": ["kind-of@6.0.3", "", {}, "sha512-dcS1ul+9tmeD95T+x28/ehLgd9mENa3LsvDTtzm3vyBEO7RPptvAD+t44WVXaUjTBRcrpFeFlC8WCruUR456hw=="],
"libheif-js": ["libheif-js@1.19.8", "", {}, "sha512-vQJWusIxO7wavpON1dusciL8Go9jsIQ+EUrckauFYAiSTjcmLAsuJh3SszLpvkwPci3JcL41ek2n+LUZGFpPIQ=="],
+5 -1
View File
@@ -23,4 +23,8 @@ timeout = 60_000
# were appending rows into the operator's real ~/.gbrain/sync-failures.jsonl,
# which `gbrain doctor` reads and warns on. See
# test/helpers/sync-failures-preload.ts.
preload = ["./test/helpers/legacy-embedding-preload.ts", "./test/helpers/audit-dir-preload.ts", "./test/helpers/sync-failures-preload.ts"]
# #3485: database-url-guard-preload runs FIRST — it refuses to start the run
# while DATABASE_URL/GBRAIN_DATABASE_URL is ambient without the explicit
# GBRAIN_TEST_ALLOW_DATABASE_URL=1 opt-in that the e2e wrappers set at their
# own subprocess boundary. See test/helpers/database-url-guard-preload.ts.
preload = ["./test/helpers/database-url-guard-preload.ts", "./test/helpers/legacy-embedding-preload.ts", "./test/helpers/audit-dir-preload.ts", "./test/helpers/sync-failures-preload.ts"]
+5
View File
@@ -99,6 +99,11 @@ Per-client setup guides live in [`docs/mcp/`](mcp/):
- [`docs/mcp/CLAUDE_DESKTOP.md`](mcp/CLAUDE_DESKTOP.md)
- [`docs/mcp/CHATGPT.md`](mcp/CHATGPT.md)
- [`docs/mcp/PERPLEXITY.md`](mcp/PERPLEXITY.md)
- [`docs/mcp/HERMES.md`](mcp/HERMES.md) — Hermes (Nous Research CLI)
- [`docs/mcp/GROK.md`](mcp/GROK.md) — Grok Build (xAI CLI)
- [`docs/mcp/OPENCODE.md`](mcp/OPENCODE.md) — opencode (opencode.ai / SST terminal agent)
- [`docs/mcp/OPENCLAW.md`](mcp/OPENCLAW.md) — OpenClaw (bundle plugin or stdio)
- [`docs/mcp/CLAUDE_COWORK.md`](mcp/CLAUDE_COWORK.md) — Claude Cowork (team plan)
- [`docs/mcp/DEPLOY.md`](mcp/DEPLOY.md) — production deploy patterns
The HTTP server ships with an admin SPA at `/admin`, an SSE activity feed at `/admin/events`, DCR-style client registration, scope-gated `read`/`write`/`admin` access, and rate limiting.
+145 -6
View File
@@ -7,17 +7,76 @@ only.
### Test command tiers
Seven test command tiers, each with a clear scope:
Six test command tiers, each with a clear scope:
| Command | What it runs | Wallclock | When to use |
|---|---|---|---|
| `bun run test` | Parallel unit-test fast loop. Sharded fan-out via `scripts/run-unit-parallel.sh` (default 4 shards — CPU-detected, clamped to a max of 8; 4 matches CI's fan-out and avoids PGLite WASM-init contention), then a serial pass over `*.serial.test.ts`. Excludes `*.slow.test.ts` and `test/e2e/*`. No pre-checks, no typecheck. Memory-safe by default: total concurrency (shards × intra-shard files) is capped to available memory at `GBRAIN_TEST_MEM_PER_FILE_MB` (default 1536 — a PGLite WASM instance) per concurrent file, and two phantom-failure classes are automatically re-run serially (the rescue pass): failures carrying the WASM out-of-memory signature, and shards killed externally (SIGTERM/SIGKILL well before the shard timeout — sibling workspaces' process cleanup, memory jetsam). Phantoms pass serially and the run goes green with an `oom_rescued` note; real failures fail again serially and stay red. Knobs: `GBRAIN_TEST_NO_MEM_ADAPT=1`, `GBRAIN_TEST_NO_OOM_FALLBACK=1`, `GBRAIN_TEST_MAX_CONCURRENCY` (intra-shard, default 4), `GBRAIN_TEST_SHARD_TIMEOUT` / `GBRAIN_TEST_SHARD_KILL_AFTER`, plus `--shards N` / `--max-concurrency N` / `--dry-run` script args. | a few minutes on a Mac dev box | Inner edit loop. Default. |
| `bun run test` | Parallel unit-test fast loop. Sharded fan-out via `scripts/run-unit-parallel.sh` (default 4 shards — CPU-detected, clamped to a max of 8; 4 matches CI's fan-out and avoids PGLite WASM-init contention), then a serial pass over `*.serial.test.ts`. Excludes `*.slow.test.ts` and `test/e2e/*`. No pre-checks, no typecheck. Builds/refreshes the PGLite schema snapshot BEFORE the shard fan-out and exports `GBRAIN_PGLITE_SNAPSHOT` so PGLite-booting files restore a baked schema instead of replaying every migration (~10x wallclock on a full run; see "PGLite schema snapshot" below). Opt out: `GBRAIN_NO_SNAPSHOT=1`. Memory-safe by default: total concurrency (shards × intra-shard files) is capped to available memory at `GBRAIN_TEST_MEM_PER_FILE_MB` (default 1536 — a PGLite WASM instance) per concurrent file, and two phantom-failure classes are automatically re-run serially (the rescue pass): failures carrying the WASM out-of-memory signature, and shards killed externally (SIGTERM/SIGKILL well before the shard timeout — sibling workspaces' process cleanup, memory jetsam). Phantoms pass serially and the run goes green with an `oom_rescued` note; real failures fail again serially and stay red. Knobs: `GBRAIN_TEST_NO_MEM_ADAPT=1`, `GBRAIN_TEST_NO_OOM_FALLBACK=1`, `GBRAIN_TEST_MAX_CONCURRENCY` (intra-shard, default 4), `GBRAIN_TEST_SHARD_TIMEOUT` / `GBRAIN_TEST_SHARD_KILL_AFTER`, plus `--shards N` / `--max-concurrency N` / `--dry-run` script args. | a few minutes on a Mac dev box | Inner edit loop. Default. |
| `bun run verify` | CI's authoritative pre-test gate set, fanned out in parallel by `scripts/run-verify-parallel.sh`: the full `check:*` battery (privacy, jsonb, progress, source-id, test-isolation, wasm, …) plus `bun run typecheck`. The `CHECKS` array in that script is the single source of truth — CI literally calls `bun run verify` in a dedicated job. | ~16s (parallel; typecheck dominates) | Before pushing; before `/ship`. |
| `bun run test:full` | `verify && bun run test && bun run test:slow && [smart e2e]`. The local equivalent of "everything CI runs." Smart e2e: runs e2e only when `DATABASE_URL` is set; else loud skip notice to stderr. | ~3-5min depending on slow + e2e | Pre-merge sanity, before opening a PR. |
| `bun run test:slow` | Just the `*.slow.test.ts` set (intentional cold-path correctness checks). | seconds-to-minutes | When touching slow-path code. |
| `bun run test:serial` | Just the `*.serial.test.ts` set (cross-file-contention quarantine; one bun process per file for true module-registry isolation). | ~1s per quarantined file | Debugging a specific quarantined file. |
| `bun run test:e2e` | Real Postgres E2E. Requires Docker + `DATABASE_URL`. Sequential. | ~5-10min | Pre-ship; nightly. |
| `bun run check:all` | The historical pre-check scripts (chained sequentially in package.json). Overlaps `verify` heavily but is NOT a superset — `verify`'s `CHECKS` array in `scripts/run-verify-parallel.sh` is the authoritative gate; `check:all` keeps a few local-only extras (trailing-newline, exports-count, no-legacy-getconnection). | ~10s | Local-only sweep for the extras. |
There is no `check:all` script anymore — it was a second, hand-synced guard
registry that drifted from `verify` (three checks were reachable ONLY from it,
i.e. never ran anywhere). The `CHECKS` array in `scripts/run-verify-parallel.sh`
is the single execution list, and it now includes the former `check:all`-only
extras (`check:newlines`, `check:exports-count`, `check:no-legacy-getconnection`).
The guard REGISTRY is `scripts/guards-manifest.tsv` (see "Guard registry and
self-test" below).
### PGLite schema snapshot (default-on)
`scripts/build-pglite-snapshot.ts` (`bun run build:pglite-snapshot`) bakes a
post-`initSchema()` PGLite data dir into `test/fixtures/pglite-snapshot.tar`
plus a version file; `PGLiteEngine.initSchema()` restores the tar instead of
replaying the embedded schema + all migrations when the env var
`GBRAIN_PGLITE_SNAPSHOT` points at it. Both `bun run test`
(`scripts/run-unit-parallel.sh`, before the shard fan-out) and
`scripts/ci-local.sh` call the builder unconditionally and export the env var.
Measured effect: a full parallel suite run drops ~10x (PGLite-booting files go
~1.63s → ~0.91s each). Properties:
- **Idempotent.** A hash short-circuit exits in ~40ms when the snapshot is
fresh, and REBUILDS a stale one. The hash covers `PGLITE_SCHEMA_SQL`, every
migration's `sql` + `sqlFor.pglite`, AND each migration `handler`'s function
source (`Function.prototype.toString`) — 19+ migrations carry executable
handler code with empty `sql` that a sql-only hash cannot see.
- **Concurrency-safe.** Parallel shard runners / sibling workspaces serialize
on an atomic `mkdir` lock (`test/fixtures/.pglite-snapshot.lock`) with
staleness-verified takeover of a crashed builder; the tar is written first
and the version file last, so a crash can never leave a fresh-looking torn
fixture. `GBRAIN_SNAPSHOT_LOCK_TIMEOUT_MS` (default 120000) bounds the
waiter; an exhausted waiter facing a still-live lock proceeds unlocked as a
last resort (the loader gate below validates the version file, not the tar
bytes).
- **Never authoritative.** The loader (`tryLoadSnapshot` in
`src/core/pglite-engine.ts`) verifies the schema hash AND the embedding
shape the snapshot was baked with (`dims=` / `model=` lines in the version
file) against what this process would create; any mismatch — including a
version file without shape lines — warns once and falls through to normal
cold init. A wrong fixture can never poison the suite.
- **Opt out.** `GBRAIN_NO_SNAPSHOT=1` skips the build + env export for a run;
the migration-replay canary tests clear the env themselves regardless.
Pinned by `test/snapshot-shape-guard.test.ts` (hash + shape refusal matrix,
handler-source hash sensitivity).
### Guard registry and self-test
`scripts/guards-manifest.tsv` is THE single registry of `scripts/check-*`
guards (currently 45), each classified `scanner` (greps/parses repo sources —
must eventually carry fixtures), `buildfresh`, or `repostate` (build/freshness
guards are exempt-with-reason, not fixture-tested).
`scripts/guard-self-test.sh` (`bun run check:guard-self-test`, wired into
`bun run verify`) proves every `selftest=yes` scanner CAN fail: it runs each
one against known-bad (must exit non-zero) and known-good (must pass) fixture
trees under `test/fixtures/guards/<guard>/{bad,good}/` via the
`GBRAIN_GUARD_ROOT` env seam, and enforces manifest completeness — a new
`scripts/check-*` script that isn't registered in the manifest fails the
build. A guard whose pattern rots into a permanently-green no-op now fails CI
instead of masquerading as coverage.
### Shell dispatch and Windows
@@ -73,10 +132,32 @@ Triage rule: a `warn-pass` EXIT-HANG line in `.context/test-summary.txt` is NOT
- `*.test.ts` → fast loop (parallel up-to-4-shard fan-out, memory-adaptive).
- `*.slow.test.ts` → run via `bun run test:slow` only (intentional cold-path tests; would dominate the fast loop's wallclock).
- `*.serial.test.ts` → run via `bun run test:serial` after the parallel pass completes; one bun process per file (`--max-concurrency=1` within a shared process is not enough — the module registry still leaks `mock.module`). Quarantine for tests that share file-wide state and race when run alongside other files in the same `bun test` process. Several dozen files, discovered by the `*.serial.test.ts` glob — no list to maintain. Typical residents: `mock.module(...)` users (top-level mocks leak across files in a shard process, e.g. `test/embed.serial.test.ts`), env-coupled files (e.g. `test/brain-registry.serial.test.ts`), and process-lifecycle suites that assert on `process.exitCode` (e.g. `test/pglite-engine-disconnect.serial.test.ts`). **Do not put the parallelism back on a serial file unless you've fixed the contention root cause** (it just re-introduces the flake).
- `test/e2e/*.test.ts` → real-Postgres E2E. Skipped when `DATABASE_URL` is unset.
- `test/e2e/*.test.ts` → real-Postgres E2E. Skipped when `DATABASE_URL` is unset. One out-of-directory file rides this lane: `test/phantom-redirect-engine-parity.test.ts` (lives in `test/` for its PGLite arm, but its Postgres arm is only reachable through a DATABASE_URL-bearing lane — the unit wrappers strip the URL per #3485, so `run-e2e.sh`'s no-args list and CI's parity job carry it).
- `tests/heavy/*.sh` → ops-shape shell scripts. Cost minutes per run; NOT in default `bun test`. Run via `bun run test:heavy` or scheduled nightly via `.github/workflows/heavy-tests.yml`. Examples: pg_upgrade matrix (boot legacy brain → walk to head), RSS budget gate (measure peak worker RSS vs committed baseline), read-latency-under-sync (p50/p95/p99 under concurrent writer load), sync lock regression (N concurrent syncs assert 1 winner + N-1 lock-busy + zero leaked `gbrain_cycle_locks` rows). See `tests/heavy/README.md` for when to add a script here vs `*.slow.test.ts`. Files prefixed with `_` (e.g. `tests/heavy/_build_legacy_fixtures.sh`) are helpers/libs invoked by sibling tests — the runner skips them.
- `test/fuzz/*.test.ts` → property-based fuzz harness. Pure-validator targets in `pure-validators.test.ts` are guarded by `scripts/check-fuzz-purity.sh` (in `bun run verify`), which `bun build --target=bun` bundles each target and greps the resulting bundle for banned transitive imports (`node:fs`, `node:child_process`, engine modules). Anything that fails the guard moves to `mixed-validators.test.ts` (still property-tested, but no purity guarantee) or `filesystem-validators.test.ts` (fs-backed, uses temp dirs). Fuzz tests run in the default `bun test` loop because they're fast (~3s for ~12 properties × 1000 runs each).
### TTY and interactive-CLI testing
Four escalating tools; reach for the cheapest one that answers the question:
| Question | Tool | Example |
|---|---|---|
| Does the TTY/non-TTY branch logic pick right? | Inject `isTTY` into the pure function — no subprocess | `test/init-provider-picker.test.ts`, `test/jobs-watch-mode.test.ts` |
| Does the real CLI behave right when stdin is NOT a terminal? | Spawn the CLI with piped/ignored stdio | `test/cli-stdin-hang.test.ts` (fast loop); `test/e2e/init-fresh-pglite.test.ts` (manual `test:e2e` lane — see the TODOS e2e CI-lane entry) |
| Does the real CLI render menus and read typed input under a REAL terminal? | `launchTty` from `test/helpers/tty-harness.ts` in a `*.serial.test.ts` file | `test/init-picker-pty.serial.test.ts` |
| How does the install FEEL (stalls, copy, silence windows)? | `scripts/dx-explore.ts` — instrument, not a test; nothing asserts | transcripts under `.context/dx-runs/` (see `docs/guides/bootstrap.md`) |
Real-PTY test rules: put the file in the serial lane (`*.serial.test.ts` — that
lane runs in required CI; a new `test/e2e/*` file does NOT, since unit shards
exclude the directory and the e2e workflow runs only explicitly named files,
no glob);
assert NON-default picker values (bare Enter and each prompt's 60s
`readLineSafe` timeout both resolve to the default, so a defaults-asserting
test passes with dead input); always `await session.close()` in a `finally`
(only `close()` clears the harness wall timer); and point `HOME` plus
`GBRAIN_HOME` at a temp root with pass-through auth keys stripped via
`dropEnv` so picker state is machine-independent.
### Skills-manifest freshness guard
`skills/skills.lock.json` is a committed sha256 inventory of every bundled file under
@@ -90,7 +171,7 @@ Any change under `skills/` must regenerate it: `bun run scripts/generate-skills-
**This section is the canonical home of the test-isolation discipline** — CONTRIBUTING.md and other docs link here rather than restating the rules.
The cross-file flake class is enforced statically by `scripts/check-test-isolation.sh`, wired into `bun run verify` and `bun run check:all`. Rules (non-serial unit files only; `*.serial.test.ts` and `test/e2e/*` are skipped):
The cross-file flake class is enforced statically by `scripts/check-test-isolation.sh`, wired into `bun run verify`. Rules (non-serial unit files only; `*.serial.test.ts` and `test/e2e/*` are skipped):
| Rule | What it bans | Fix |
|---|---|---|
@@ -161,6 +242,35 @@ The quarantine has grown to dozens of files — treat it as debt: every addition
`bun test` runs all tests without a database. E2E tests skip gracefully when `DATABASE_URL` is not set.
**Database-URL run guard (#3485).** A `bun test` invocation REFUSES to start while
`DATABASE_URL` or `GBRAIN_DATABASE_URL` is ambient in the environment, because some
tests run destructive SQL against whatever those URLs point at (a bare `bun test`
with `~/.gbrain/.env` sourced has wiped a real brain). The guard is a bunfig
`[test]` preload (`test/helpers/database-url-guard-preload.ts`); it hard-fails with
instructions rather than silently unsetting (a silent unset would turn
DATABASE_URL-gated e2e tests into green skips). The e2e wrappers
(`scripts/run-e2e.sh`, the e2e/heavy workflows) opt in at their own boundary via
`GBRAIN_TEST_ALLOW_DATABASE_URL=1`; the unit/slow wrappers instead strip both
URL vars at their boundary (unit tests need no database), which keeps
`bun run test:full` working with DATABASE_URL exported. Caveat: bun loads
`bunfig.toml` from the invocation cwd, so the preload layer only applies to
runs started at the repo root — the per-file name floor below is the layer
that doesn't care about cwd. Two more layers apply after the opt-in: every
test that runs destructive SQL on the ambient URL must call
`assertSafeE2eDatabaseUrl()` (`test/helpers/db-guard.ts` — name floor: the database
name must contain "test" as a segment, or be opted in via `GBRAIN_E2E_ALLOW_DB`)
or carry an inline name floor the coverage gate recognizes
(`test/e2e/schema-drift.test.ts` keeps its own `looksLikeTestDb`, deliberately
different because it also accepts `*_e2e`), and `test/db-guard-coverage.test.ts`
statically scans the suite and fails when a file connects to `DATABASE_URL` and
runs destructive SQL unguarded. The heavy shell lane gets the same floor outside
bun: `tests/heavy/_db_floor.sh` (sourced by `scripts/run-heavy.sh` for the whole
lane, and by each database-touching heavy script itself, since scripts are
documented for direct invocation — the PGLite-based heavy scripts unset the URL
instead) checks BOTH URL variables and strips query strings before extracting
the database name, so a `?host=/tmp/test-sockets` parameter can't smuggle a
test-shaped segment past it.
Unit tests and what they cover:
- `test/markdown.test.ts` — frontmatter parsing; `splitBody` sentinel precedence, horizontal-rule preservation, `inferType` wiki subtypes.
@@ -175,6 +285,8 @@ Unit tests and what they cover:
- `test/volunteer-context.test.ts` — push-based context core (#2095), hermetic in-memory PGLite: `parseWindow` lenient `user:`/`assistant:` parsing, multi-turn window extraction, confidence-gated volunteering (arm confidences, multi-turn/newest-turn boosts, `min_confidence` gate, max-pages cap), slug-only suppression, privacy (rationales are deterministic templates; synopses pass the takes/facts fence), and the approximate usage-stats join.
- `test/watch-command.test.ts``gbrain watch` push transport (#2095): streaming loop, rolling window, session dedupe, `--json` JSONL shape, `channel: 'watch'` event logging, clean EOF return. Hermetic PGLite + injected line/write deps (no subprocess, no real stdin).
- `test/watch-sigint.serial.test.ts``gbrain watch` SIGINT lifecycle against a real spawned CLI subprocess with a tmpdir brain. SERIAL: parallel unit shards flake on concurrent subprocess spawns (same rationale as `apply-migrations-pglite-spawn.serial.test.ts`).
- `test/init-picker-pty.serial.test.ts` — the interactive `gbrain init` pickers (embedding-provider + search-mode) driven under a REAL pseudo-terminal via `launchTty`: typed input lands (a NON-default mode choice verified by a follow-up non-TTY config read — bare Enter and the `readLineSafe` timeout both resolve to defaults, so a defaults-asserting test would pass with dead input), prompt-to-acknowledgement gaps bounded well under the fallback window, plus the Ctrl-D/EOF keyless fallback. On CI, missing PTY support fails loud instead of skipping. Hermetic: HOME + GBRAIN_HOME at a temp root, pass-through auth keys stripped via `dropEnv`; `session.close()` in `finally`. Serial: PTY spawn + full PGLite bootstrap, and the serial lane is what runs in required CI.
- `test/tty-harness.test.ts` — the real-PTY harness's pure helpers (`stripAnsi`, `computeStalls`, `renderStallsReport`, `parseDriveCommand`, `buildClaudeTuiSeed`) with zero subprocesses; the file's live-PTY smokes are `describe.skipIf(!ptySupported())`-gated.
- `test/autopilot-launchd-lifecycle.serial.test.ts` — autopilot lifecycle behavior, not generated-string assertions: the full install → self-disable → status → reinstall → uninstall arc with `launchctl` replaced by an argv recorder and the generated wrapper executed by a REAL bash against a genuinely deleted repo (every platform), plus a darwin-only fail-SKIP describe against the real launchd under a per-run unique label (`GBRAIN_AUTOPILOT_LABEL`) so it can never collide with — or tear down — a real install on the host. Serial: spawns subprocesses and pins HOME/GBRAIN_HOME for the whole file.
- `test/autopilot-fanout.test.ts` — Autopilot fan-out and #4046 policy regression: targeted idempotency keys reopen per dispatch interval while stable doctor/remediate keys remain unchanged; the 60-minute full-cycle floor wins with a remaining small plan, and an all-fresh restart check advances the process-local clock without masking failed stale-source submissions.
- `test/agent-scheduler-contract.serial.test.ts` — the documented external agent-scheduler shell chain (`gbrain sync --repo X && gbrain embed --stale`, live-sync.md / INSTALL_FOR_AGENTS.md Step 7) driven end-to-end through a real `/bin/sh` against a keyless PGLite brain: the `&&` short-circuit IS the contract (argv arrays can't exercise it), the keyless bare stale embed exits 0, and the pull-failure case that must break the chain does. Anti-vacuity: the fixture commits a real page and every read-back asserts pages >= 1. Serial: real spawned CLI + tmpdir HOME.
@@ -222,6 +334,18 @@ Unit tests and what they cover:
- `test/enrichment-service.test.ts` — entity slugification, extraction, tier escalation.
- `test/data-research.test.ts` — recipe validation, MRR/ARR extraction, dedup, tracker parsing, HTML stripping.
- `test/minions.test.ts` — Minions job queue: CRUD, state machine, backoff, stall detection, dependencies, worker lifecycle, lock management, claim mechanics, depth/child-cap, timeouts, cascade kill, idempotency, `child_done` inbox, attachments, removeOnComplete/Fail, `max_stalled` clamp/default/plumbing coverage.
- `test/minion-queue-renewlock-signal.test.ts``renewLock` forwards its optional AbortSignal to `executeRawDirect` (stub-engine capture); legacy 3-arg calls unchanged; token-fence miss returns false.
- `test/cycle-drain-renewal.test.ts``runDrainRenewalTick` (cycle drain): per-call signal aborted on timeout (slot released), onLost once on a lost fence, throws swallowed, hung renewal resolves at the deadline.
- `test/queue-probe-cancellation.test.ts``probeQueueState`/`queryWedgeSignals` signal threading: the 1500ms budget CANCELS the losing probe query; fast-path signals never abort; throw still collapses to `{probe_failed: true}`.
- `test/db-pool-max-lifetime.test.ts``resolveMaxLifetimeSeconds`: env forms, 0-disables, 3060min jitter bounds, warn-once on invalid, per-call jitter variance.
- `test/pool-gauge.test.ts``CheckoutGauge` pure semantics + the PostgresEngine seams with fake pools: counted while in flight, released on resolve, on REJECTED queries, and on the SYNCHRONOUS pre-aborted-signal throw (leak guards); `getPoolDiagnostics` fail-open.
- `test/db-probe.test.ts``runDbProbe` verdict matrix (pool_starved / server_unreachable / unknown), honest-disjunction + no-waiter-arithmetic wording pins, hung probes cancelled via their signals, diagnostics absent/throwing fail open.
- `test/postgres-engine-reserved-routing.test.ts``withReservedConnection` routing: direct pool when dual-pool active, read pool when kill-switched/in-tx, semaphore cap (directPoolSize1) with read-pool overflow, permit released on fn throw and reserve failure.
- `test/job-isolation-protocol.test.ts` — outcome-file codec round-trip + every decode failure path (missing/malformed/oversize→UnrecoverableError; byte counts, never content), handler-error instanceof reconstruction, child-CLI invocation resolution, and REAL detached-process `killProcessGroup` tests incl. the grandchild-death guarantee (exercises the Bun negative-pid `/bin/kill` fallback for real under `bun test`).
- `test/run-child-entry.test.ts``runChildJobEntry` on real in-memory PGLite with a REAL claim-minted token: success (fenced updateProgress lands), handler-failure outcome (exit 0), token-mismatch never runs the handler (exit 14), missing job/handler, parent-death watchdog aborts a live handler.
- `test/child-job-runner.test.ts``runJobInChild` against real .mjs children: success + full env contract (incl. `GBRAIN_DIRECT_POOL_SIZE=1`), error/lease outcome reconstruction, crash, SIGTERM-ignorer → group SIGKILL at the injected grace, pre-aborted signal, spawn ENOENT → `ChildSpawnInfraError`, worker-shutdown drain (report-during-drain completes; non-reporting kill → `ChildWorkerShutdownError`).
- `test/worker-job-isolation.test.ts` — full parent path on PGLite with the `fake-run-child.mjs` fixture: claim → child → fenced completeJob (real token over env), error outcome → failJob, crash burns the attempt, spawn failure RELEASES with zero attempts burned, and the codex-2 #8 serialization-parity pin (unreportable results fail in BOTH modes, never falsely complete).
- `test/jobs-isolation-flag.test.ts``parseJobIsolationFlag`: space/= forms, env fallback + flag-wins, empty-env default, other flags untouched.
- `test/extract.test.ts` — link extraction, timeline extraction, frontmatter parsing, directory type inference.
- `test/extract-db.test.ts``gbrain extract --source db`: typed link inference, idempotency, `--type` filter, `--dry-run` JSON output.
- `test/extract-fs.test.ts``gbrain extract --source fs`: first-run inserts + second-run reports zero, dry-run dedups candidates across files, second-run perf regression guard for the N+1 dedup bug.
@@ -264,10 +388,19 @@ Unit tests and what they cover:
- `test/longmemeval-sanitize.test.ts` — sanitization parity pinning that `INJECTION_PATTERNS` from `src/core/think/sanitize.ts` is the single source of truth (adding a pattern there must cover both `<take>` framing and `<chat_session>` framing, no per-surface regex drift).
- `test/openai-compat-multimodal.test.ts` — gateway's openai-compatible multimodal path: happy-path single + multi-input embedding, unauthenticated proxy mode, dimension-mismatch guard (throws `AIConfigError` with model id + observed + expected pre-storage), default-dim fallback when recipe declares `default_dims`, HTTP 401 / 400 / malformed-JSON / non-array error paths, regression that the existing Voyage `/multimodalembeddings` recipe still routes through its dedicated path. Hermetic via the `__setEmbedTransportForTests` seam.
- `test/serve-stdio-lifecycle.test.ts``MCP_STDIO=1` env guard: stdin EOF does NOT trigger shutdown when the env is set, SIGTERM still does (guard scope is correct), unset env preserves the CLI lifecycle. Exercises the `ServeOptions.mcpStdio?: boolean` test seam directly so tests don't mutate `process.env`.
- `test/db-lock-fencing.test.ts` — fenced lock identity: a `DbLockHandle` carries its acquisition fence, `refresh()` returns true while owned and false after a steal (0-row fenced UPDATE), a stolen-from handle's `release()` is a fenced no-op that leaves the successor's row intact, and `startCycleLockRefresher` aborts its controller with `LockStolenError` on a fenced miss while serializing ticks (a slow refresh never overlaps the next).
- `test/cycle-lock-steal.serial.test.ts` — runCycle steal-abort arc end-to-end: a mid-run steal produces a structured partial report (`reason: 'lock_stolen'`), runs no further phases, and never touches the successor's lock row; a steal-free cycle completes and releases normally.
- `test/cycle-any-abort-signal.test.ts``anyAbortSignal` combining: pre-aborted inputs, late aborts propagating their reason, duck-typed signal stubs (no `addEventListener`) observed via poll, and `dispose()` detaching the caller-signal listener + clearing the poll timer (the daemon leak class).
- `test/queue-stall-parent-unblock.test.ts` — the shared `killJobs` tail: a stall-exhausted child lands `child_done(dead)` in its parent's inbox and unblocks the parent, a requeued child doesn't touch the parent, all three reapers route through the tail with their own outcome, and the idempotent stranded-parent sweep self-heals parents whose children were already dead (without unblocking parents that still have a live child).
- `test/queue-started-at-retry.test.ts` — every automatic re-run path clears `started_at` (failJob delayed branch, stall requeue, lease release, promoteDelayed, parent re-claim) so a retried job's wall-clock budget measures execution, not backoff wait; end-to-end survival of the wall-clock sweep on a fresh attempt.
- `test/embed-modality-preserved.test.ts``carryChunkMetadata` carries modality + all code-metadata fields through re-embed merges (an image chunk stays image), plus the write-side contract that omitting modality resets it to text (why the shared list is load-bearing).
- `test/import-abort-error.test.ts``runImport` preflight/argv failures throw typed `ImportAbortError` instead of exiting the process; the calling process survives the abort.
- `test/lint-fix-single-pass.test.ts``gbrain lint --fix` walks the tree once and `total_fixed` reports the fixes THIS run applied.
- `test/snapshot-shape-guard.test.ts` — PGLite snapshot loader refusal matrix: shape-less version files, dims/model mismatches, and stale schema hashes are all refused; matching hash + shape loads; a migration-handler edit changes the hash.
### E2E test inventory
E2E tests live in `test/e2e/` and run against real Postgres+pgvector (require `DATABASE_URL`), except where noted as PGLite in-memory (no `DATABASE_URL` needed).
E2E tests live in `test/e2e/` and run against real Postgres+pgvector (require `DATABASE_URL`), except where noted as PGLite in-memory (no `DATABASE_URL` needed). One file outside the directory also rides the e2e lane: `test/phantom-redirect-engine-parity.test.ts` (Postgres arm; see the file taxonomy above).
- `bun run test:e2e` runs Tier 1 (mechanical, all operations, no API keys). Includes dedicated cases for the postgres-engine `addLinksBatch` / `addTimelineEntriesBatch` bind path — postgres-js's JSONB bind (`jsonb_to_recordset(($1::jsonb)->'rows')`) differs from PGLite's and gets its own coverage.
- `test/e2e/search-quality.test.ts` — search quality against PGLite (no API keys, in-memory).
@@ -279,12 +412,18 @@ E2E tests live in `test/e2e/` and run against real Postgres+pgvector (require `D
- `test/e2e/sync.test.ts``--skip-failed` failure-loop test alongside happy-path tests: broken file → `performSync` returns `blocked_by_failures` with grouped breakdown → `performSync({skipFailed: true})` advances bookmark and returns `AcknowledgeResult` with code summary → second broken file → second cycle. Saves and restores the user's real `~/.gbrain/sync-failures.jsonl` so the test is hermetic. Asserts bookmark gating, JSONL state, dedup across paths, summary aggregation, and the literal doctor-rendering string format.
- `test/e2e/upgrade.test.ts` — check-update against real GitHub API (network required).
- `test/e2e/minions-shell-pglite.test.ts` — PGLite `--follow` inline shell-job path (in-memory, no `DATABASE_URL` required) — the path the minion-orchestrator skill documents for dev use.
- `test/e2e/job-isolation.test.ts` — process isolation on real Postgres (DATABASE_URL-gated, wired EXPLICITLY into `.github/workflows/e2e.yml` tier1 — the workflow runs only named files): a concurrency-3 isolated drain through real child processes (the `fake-run-child.mjs` fixture — real spawns, no child DB pools), and the REAL `jobs run-child` CLI entrypoint end-to-end (engine bootstrap incl. the child's own pools, quiet handler registry, token validation, outcome protocol).
- `test/e2e/pglite-cli-exit.serial.test.ts` — real spawned-CLI exit behavior on PGLite (in-memory, no `DATABASE_URL`): read commands (`search`/`get`/`query`) exit 0 promptly; CLI_ONLY `capture` exits clean and frees the single-writer lock; the `#2084` describes pin every swept disconnect site — a failed op exits 1 with the error on stderr, and the dashboard, read-only-timeout, doctor, and `dream --dry-run` paths all exit with no force-exit banner.
- `test/e2e/pgbouncer-teardown.test.ts` — PgBouncer TRANSACTION-mode teardown (#2084 / the #1972#2015#2084 class). Pins the bug CLASS, not timings: a CLI op against a txn-mode pooled URL exits 0 with intact stdout and does NOT ride the 10s hard-deadline backstop (the `engine.disconnect() did not return` banner is the smoking gun — pre-#2084 it printed on 100% of query-shaped ops). Gated by `GBRAIN_PGBOUNCER_URL` + `GBRAIN_PGBOUNCER_DIRECT_URL` (NOT `DATABASE_URL`) — set automatically by `bun run ci:local`'s `pgbouncer` compose service; skips gracefully elsewhere. Uses a DEDICATED `gbrain_pgbouncer` database so it never races the `gbrain_test` TRUNCATE fixtures.
- `test/e2e/volunteer-context-postgres.test.ts``volunteer_context` on REAL Postgres (#2095; engine parity beyond the hermetic PGLite unit suite): resolution arms through the actual op handler, the fire-and-forget volunteer-event sink landing rows, the stats join, and the RLS pin that `context_volunteer_events` has ROW LEVEL SECURITY enabled (keeps the v35 auto-RLS event trigger honest for migration-created tables). `DATABASE_URL`-gated.
- `test/e2e/openclaw-reference-compat.test.ts``check-resolvable` + skillpack install-model against a minimal AGENTS.md workspace fixture (`test/fixtures/openclaw-reference-minimal/`), regression guard for the OpenClaw deployment shape.
- `test/e2e/workspace-generic-compat.test.ts` — always-on (PGLite, no binary): pins the INSTALL_FOR_AGENTS.md "any repo with a workspace" contract against `test/fixtures/generic-agents-workspace/` (Hermes is the motivating consumer): `cwd_walk_up` detection, the `GBRAIN_SKILLS_DIR` override, `check-resolvable` on a root AGENTS.md, and scaffold additivity + refuse-overwrite. The real Hermes-behavior proof is the door suite below.
- `test/e2e/install-real-hermes.serial.test.ts` — the hermes "door": real `hermes` binary + real `hermes mcp add` handshake (full-catalog tool discovery; the count tracks the op catalog, so the test asserts discovery happened, not a number) + a paid `hermes -z` recall turn against a seeded brain. Triple-gated: `GBRAIN_REAL_HERMES_E2E=1` (explicit opt-in — run-e2e.sh scrubs GBRAIN_*, so it can never fire under `bun run test:e2e`) + resolvable binary + non-empty ANTHROPIC key (anthropic-pinned on purpose: a second provider key flips hermes provider-auto into a mis-routed 401). Hermetic HOME + HERMES_HOME with a tripwire on the operator's real config; evidence copies to `GBRAIN_E2E_EVIDENCE_DIR` for CI upload. Venue: heavy-tests.yml (`real-agent-e2e` + `hermes-door` jobs).
- `test/e2e/install-real-grok.serial.test.ts` — the grok "door" (xAI Grok Build; every asserted shape observed against the pin in `docs/mcp/GROK-CLI-PIN.md`). SPLIT-GATED, a deliberate divergence from the hermes door: grok's `mcp add/list/doctor` run keyless, so the compat tier (version-shape pin, documented-shape `grok mcp add gbrain -- gbrain serve --surface verbs` via a PATH-staged bin dir, saved-TOML asserts via `Bun.TOML.parse`, `mcp doctor` handshake proving the seven-verb surface, vendor-fallback provenance guard, direct-TOML surface) needs only `GBRAIN_REAL_GROK_E2E=1` + a resolvable binary; the paid SMOKE additionally needs a non-empty `XAI_API_KEY` and asserts a PER-RUN NONCE fact (grok has fs/shell tools — the committed fact is greppable, so recall of it proves nothing) with web search disabled. `mcp add` is lazy (exit 0 always) — `mcp doctor <name> --json` is the honest discriminator (exit 0/1 observed). Hermetic HOME + GROK_HOME + tmp cwd on every spawn (grok reads vendor MCP configs for trusted folders and loads `.envrc` from cwd); bounded tripwire over the operator's real `~/.grok` config/credential files (volatile paths excluded — grok rewrites logs/sessions/bin/docs every run) + a checkout guard that no `.grok/`/`.mcp.json` appeared in the repo root. Venue: heavy-tests.yml (`real-agent-e2e` + `grok-door` jobs); run directly via `GBRAIN_REAL_GROK_E2E=1 bun test test/e2e/install-real-grok.serial.test.ts`.
- `test/e2e/install-real-opencode.serial.test.ts` — the opencode "door" (SST opencode; every asserted shape observed against the pin in `docs/mcp/OPENCODE-CLI-PIN.md`). SPLIT-GATED a step past the grok door: opencode's anonymous FREE TIER drives MCP tool calls keyless, so even the nonce SMOKE runs in the keyless tier — T1 bare-semver version pin (the SST-vs-claimant discriminator), T2 documented-shape `opencode mcp add gbrain --env … -- gbrain serve --surface verbs` + the honest `opencode mcp list` discriminator (it SPAWNS every server; `✓/✗` text is the assertion surface — exit code is 0 even on failure, and `mcp debug` is OAuth-only), T2b spawn-gate CANARY (a project-config decoy is spawn-attempted with NO trust prompt — if this ever gates, the bootstrap user-global scope default's rationale changed: re-observe), T3 writer parity (gbrain's `opencode-json.ts` output handshakes through the real binary; cross-tool preservation both ways), T4 keyless SMOKE (per-run nonce + STRUCTURAL `gbrain_*` tool_use proof via `parseOpencodeJsonl`, `--format json`). The paid T5 anthropic leg additionally needs a non-empty `ANTHROPIC_API_KEY` and self-validates the pinned model id against the authed `opencode models` list BEFORE any spend. Hermetic HOME + both XDG dirs + tmp cwd on every spawn; `--pure` on every probe (`mcp list` autoloads plugins — a code-execution surface); bounded tripwire over the operator's real opencode configs/auth.json + a repo-root checkout guard. Venue: heavy-tests.yml (`real-agent-e2e` + `opencode-door` jobs, plus the schedule-only `opencode-door-canary` latest-version leg — continue-on-error, a pin-refresh signal, never a gate); run directly via `GBRAIN_REAL_OPENCODE_E2E=1 bun test test/e2e/install-real-opencode.serial.test.ts`.
**Door cadence policy** (adopted with the 4th door agent): the NEWEST door agent runs at nightly/schedule cadence (currently opencode, whose canary leg also tracks `latest`); a door drops to label-only (`real-agent-e2e`) after 2 stable monthly cycles with unchanged pins. Rationale: churn concentrates in the newest integration; steady-state doors pay for themselves on demand, not nightly.
- `test/helpers/tty-harness.ts` + `test/tty-harness.test.ts` — the DX real-PTY harness (`Bun.spawn({terminal:})`): pure text/timing helpers unit-tested with zero subprocesses, plus three live PTY smokes against `sh` guarded by `describe.skipIf(!ptySupported())`. The harness itself is a dev instrument surface — its consumer `scripts/dx-explore.ts` never runs in CI (transcripts land in gitignored `.context/dx-runs/`); see `docs/guides/bootstrap.md` for the scenario runbook.
- `test/e2e/search-swamp.test.ts` — reproduces the source-swamp case. Seeds a curated `originals/talks/article-outline-fat-code` page against two `<fork>/chat/` pages stuffed with the same multi-word phrase. Asserts the article wins keyword AND vector ranking, that `detail=high` lets the chat swamp re-surface, and that `source_id` passes through the two-stage CTE intact. PGLite in-memory.
- `test/e2e/search-exclude.test.ts``test/` + `archive/` pages hidden by default, `include_slug_prefixes` opts back in, caller-supplied `exclude_slug_prefixes` adds to defaults. Both keyword and vector search paths.
- `test/e2e/engine-parity.test.ts` — Postgres ↔ PGLite top-result and result-set parity for `searchKeyword` + `searchVector` (Postgres ranks pages then picks best chunk while PGLite returns chunks directly, so the source-boost behavior needs parity coverage). Skips without `DATABASE_URL`.
+150
View File
@@ -0,0 +1,150 @@
# claude-cli — the `claude-cli` recipe (routes chat/toolLoop through the local `claude` CLI)
This page documents the `claude-cli` recipe as it already ships (added in
v0.42.66.0, PR #3310 — see `CHANGELOG.md`) — it is not proposing new
behavior. The implementation lives at `src/core/ai/recipes/claude-cli.ts`
and `src/core/ai/providers/claude-cli-language-model.ts`; neither
`README.md` nor `docs/` mentioned this recipe before this page, so the only
description of how it behaves lived in those source comments.
`claude-cli` routes `gateway.chat()` and `gateway.toolLoop()` through the
`claude` CLI binary as a subprocess (`claude --print ...`) instead of through
the Anthropic SDK. It sits alongside the existing `anthropic` recipe as a
second `Recipe` entry with the same touchpoint shape; which one a given
model string resolves to is a per-call choice: `anthropic:claude-sonnet-5`
resolves to the `native-anthropic` implementation (SDK + `ANTHROPIC_API_KEY`),
`claude-cli:claude-sonnet-5` resolves to `ClaudeCliLanguageModel` (subprocess,
CLI-managed auth).
**Chat-only — no embedding.** `gateway.embed()` throws immediately for
`claude-cli` models (`claude-cli has no embedding model. Use openai or google
for embeddings.`). Claude has no first-party embedding model regardless of
transport; pair this recipe with `openai`, `google`, or `voyage` for
embeddings the same way the `anthropic` recipe's docs already recommend.
## Setup
1. Install Claude Code (the `claude` CLI) and run `claude` once to log in.
If the binary is not on `PATH`, point the gateway at it explicitly:
```bash
export GBRAIN_CLAUDE_CLI_BIN=/path/to/claude
```
2. Point a model tier (or any per-call model string) at `claude-cli:`:
```bash
gbrain config set models.tier.subagent claude-cli:claude-sonnet-5
```
Any of the models the recipe declares work the same way:
`claude-cli:claude-opus-5`, `claude-cli:claude-haiku-4-5-20251001`, etc.
Short aliases (`claude-cli:sonnet`, `claude-cli:haiku`, `claude-cli:opus`)
resolve the same way the `anthropic` recipe's aliases do.
The recipe declares `auth_env: { required: [] }`, and neither the recipe
nor the adapter code reads or passes any API-key-shaped config value to the
subprocess — whatever the `claude` binary does for its own auth (see below)
is between it and its own login state, not something gbrain's config layer
participates in. There is also no `provider_base_urls` entry for this
recipe — it has no base URL, only a subprocess binary path
(`GBRAIN_CLAUDE_CLI_BIN`).
## What actually happens on a call
Each `doGenerate` call spawns `claude --print --output-format json --model
<id> --disable-slash-commands --tools '' --strict-mcp-config` as a
subprocess, with `cwd` set to a per-process directory under the OS tmpdir
(`join(tmpdir(), 'gbrain-claude-cli-cwd-' + process.pid)`, created via
`mkdirSync(..., { recursive: true })` if missing — code doesn't otherwise
touch or inspect its contents), and pipes the rendered prompt to it on
stdin:
- `--tools ''` disables every built-in tool (Bash/Read/WebSearch/…) — the
subprocess must behave like a raw LLM, not a full agent.
- `--strict-mcp-config` skips loading the user's MCP servers. Without it,
every call would boot the user's configured MCP servers — including
gbrain's own MCP, which would recurse and contend for the PGLite
single-writer lock.
- The subprocess env is a copy of gbrain's own process env with exactly
three keys deleted before spawn: `ANTHROPIC_API_KEY`, `ANTHROPIC_AUTH_TOKEN`,
`ANTHROPIC_BASE_URL`. Everything else in gbrain's environment is inherited
as-is. The recipe's source comment states the intent (stop an
`ANTHROPIC_API_KEY` present in gbrain's own env from being picked up by the
subprocess), scoped to those three variables specifically — the doc does
not claim this rules out every other way `claude` could end up billing
through a non-subscription path (e.g. other env-based auth switches the CLI
itself may support); that is between the installed `claude` binary and its
own configuration, not something this recipe's code inspects.
- Beyond that env-scrub, auth resolution is entirely up to the installed
`claude` binary — the recipe does not manage or forward credentials
itself. Whatever `claude` is already logged in / authenticated with on
this machine is what it authenticates with here too (see the `claude` CLI's
own docs for how it stores and resolves that).
`--bare` (which would skip loading the user-level `~/.claude/CLAUDE.md`
entirely) is not among the flags passed, because it also forces
`ANTHROPIC_API_KEY` auth (per the recipe's source comment). One effect of
not passing it: the user-level `~/.claude/CLAUDE.md` still loads and gets
cached tokens on every call.
The adapter does not use `claude`'s own agentic tool-calling — it injects a
fenced instruction block into the system prompt teaching the model a
`<use_tools>[{id,name,input}, ...]</use_tools>` JSON emission format
(`buildToolUseInstructions`), then parses that block back out of the plain
text response into ai-sdk tool-call parts (`extractToolCalls`). This
protocol-over-text approach is what lets `supports_subagent_loop: true`
work through the `--print`, no-built-in-tools subprocess shape described
above.
## Constraints
| Area | Behavior |
|---|---|
| Embedding | Not supported — `gateway.embed()` throws for `claude-cli` models. Pair with another provider for embeddings. |
| Streaming | Not implemented. `doStream()` throws. `gateway.toolLoop()` (the main caller) is non-streaming already, so this is not a practical limitation for subagent dispatch, but any caller that expects a streaming chat surface cannot use `claude-cli`. |
| Tool use | JSON emission via a system-prompt-injected protocol, not the CLI's native tool-call mechanism. Parallel tool calls in one turn round-trip correctly. |
| Multimodal | Not supported over the subprocess path. File/image message parts are rendered as a `[file <mediaType>]` text stub, not sent as actual content. |
| Prompt caching | The recipe declares `supports_prompt_cache: false`. The CLI manages its own caching internally but does not expose it through gbrain's `cache_control` control plane, so from the gateway's point of view this model does not support prompt caching. |
| Usage / token counts | Reported `usage.input_tokens` / `usage.output_tokens` are read straight from the CLI's `--output-format json` envelope (`result.usage?.input_tokens` / `output_tokens`); gbrain does not independently count tokens for this path. |
| Cost figures | The recipe declares `cost_per_1m_input_usd: 3.0` / `cost_per_1m_output_usd: 15.0` — the same Sonnet-class figures the `anthropic` recipe declares (`price_last_verified: 2026-06-17`) — purely so gbrain's budget ledger has a number to attribute per call. Neither the recipe nor the adapter code checks what you're actually billed; treat these as the ledger's nominal per-call number, not a verified charge. |
| User-level CLAUDE.md | `~/.claude/CLAUDE.md` still loads on every call (see above) — only the working directory changes (see "What actually happens on a call" for exactly what that directory is and isn't). |
## Known doctor caveat: cold-start subprocess vs the fixed 5s probe timeout
`gbrain models doctor`'s chat reachability probe (`probeModel` in
`src/commands/models.ts`) wraps every chat call in a fixed 5-second
`AbortController` timeout, independent of any per-recipe timeout the recipe
itself declares (`claude-cli` does not declare a `default_timeout_ms`).
Spawning the `claude` binary and letting it start up is generally fast, but
is not instantaneous — a slow first invocation (cold process cache, slow
disk, contended machine) can outrun that 5-second window.
When that happens, the probe's `AbortController` fires, the subprocess is
killed (`child.kill('SIGTERM')`), and the adapter's abort handler rejects
with a fixed message (`claude-cli adapter aborted`). `classifyError` in
`src/commands/models.ts` only maps a message to `status: network` if it
matches `/timeout|network|econn|fetch failed|enotfound/`; `claude-cli
adapter aborted` matches none of those, so it falls through to
`status: unknown` — the classifier's catch-all — instead of `status:
network`, which is what a plain slow/unreachable HTTP provider would map
to on the same probe timeout. So a `status: unknown` result on a
`claude-cli:` model is not necessarily a broken configuration on its own;
a cold subprocess start outrunning the fixed 5s window is one thing that
can produce it (the same class of first-call cold-start the embedding
reachability probe's own code comment already calls out for local
embedders), and re-running the probe is a reasonable first thing to try.
`status: unknown` on its own doesn't distinguish that from any other
unclassified failure, so if a re-run keeps producing it, treat it as an
unclassified error worth investigating rather than assuming cold-start.
## Troubleshooting
| Symptom | Where it comes from | Try |
|---|---|---|
| `claude-cli spawn failed: ...` / stdin write failure | `spawn()`'s `error` event or a failed `stdin.write` — commonly means the `claude` binary was not found on `PATH` | Install Claude Code, or set `GBRAIN_CLAUDE_CLI_BIN` to the binary's path |
| `claude-cli exited <code>: ...` | Non-zero exit from the `claude` subprocess itself; the message is whatever the CLI wrote to stderr/stdout | Run `claude` interactively with the same model to see the underlying CLI error directly (e.g. not logged in, model unavailable) |
| `claude-cli output not JSON: ...` | `JSON.parse(stdout)` threw (stdout wasn't valid JSON at all) | Confirm the installed `claude` CLI version still supports `--print --output-format json`; this adapter's JSON handling was verified against CLI 2.1.145 |
| `claude-cli JSON event array had no "result" event` | stdout parsed as a JSON array (the `"verbose": true` event-stream shape in `~/.claude/settings.json`) but none of the events had `type: "result"` | Check `~/.claude/settings.json` for `"verbose": true`; the adapter tolerates the array shape but still needs a `result` event in it |
| `gbrain models doctor` reports `chat` as `status: unknown` for a `claude-cli:` model | See "Known doctor caveat" above — `classifyError` falls through to `unknown` for the adapter's abort message | Re-run the probe; if it persists, treat it as an unclassified failure and investigate directly (e.g. run the same model via `gbrain models doctor --json` or call `claude` by hand) |
| A call bills through the Anthropic API instead of the local session | The adapter deletes `ANTHROPIC_API_KEY` / `ANTHROPIC_AUTH_TOKEN` / `ANTHROPIC_BASE_URL` from the subprocess env — this covers gbrain's own env leaking into the call. It does not inspect any other auth/billing switch the installed `claude` CLI itself may support | If billing looks wrong, check the `claude` CLI's own auth/billing configuration on this machine, not just gbrain's env |
File diff suppressed because one or more lines are too long
+1 -1
View File
@@ -86,7 +86,7 @@ the repo. The architectural rule still holds — these aren't
| `mcp_request_log` | Audit trail. Volatile by design. |
| `minion_jobs` / `minion_inbox` / `minion_attachments` | Job queue. Restarts re-enqueue or drop. |
| `eval_candidates` / `eval_capture_failures` | Contributor-mode dev loop; opt-in capture. |
| `dream_verdicts` | Cheap verdict cache. Rebuildable by re-running Haiku. |
| `dream_verdicts` | Scored triage cache (salience score, quotes, entities, judging model + prompt version). Rebuildable via `gbrain dream retriage --force`. |
| `gbrain_cycle_locks` / migration ledger | Infrastructure. |
| `op_checkpoint_paths` | Sync-resume checkpoint. Append-only progress banking; a completed sync makes it irrelevant. |
| `config` (some keys) | Site-local routing config (e.g. `sync.repo_path`). |
+61
View File
@@ -0,0 +1,61 @@
# Fix-wave series baselines (W0 → W9)
Recorded per wave so the series' "10x better for 2x effort" claim is measured,
not vibed (fix-wave plan D4.13). Update this file in each wave's PR; keep the
prior rows — the deltas ARE the receipt.
## How to refresh
```bash
wc -l src/commands/doctor.ts src/core/pglite-engine.ts src/core/postgres-engine.ts \
src/core/operations.ts src/core/migrate.ts src/commands/sync.ts \
src/core/ai/gateway.ts src/cli.ts src/core/engine.ts \
src/core/search/hybrid.ts src/core/search/mode.ts src/core/cycle.ts
ls scripts/check-* | wc -l # guard count
bash scripts/guard-self-test.sh # self-tested count + harness runtime
bun run test > /tmp/suite.txt 2>&1; echo $? # wall-clock from the run banner
```
Retrieval-quality canary (MANDATORY before W1, and after W1/W3/W9): run
`gbrain eval gate` against a NON-PRODUCTION brain (the production PGLite brain
is single-writer and usually held by a live `gbrain serve`; eval runs never
touch `~/.gbrain` per the eval discipline — results land in
`<repo>/.gbrain-evals/eval-results.jsonl`). Record the gate verdict + headline
metrics here per run.
## W0 (2026-08-14, branch garrytan/code-smell-fix-wave @ post-hotfix)
God-file line counts (the audit's structural targets, BEFORE the registry waves):
| File | Lines |
|---|---|
| src/commands/doctor.ts | 10,057 |
| src/core/operations.ts | 7,459 |
| src/core/pglite-engine.ts | 6,874 |
| src/core/postgres-engine.ts | 6,847 |
| src/core/migrate.ts | 6,201 |
| src/commands/sync.ts | 5,991 |
| src/core/ai/gateway.ts | 4,049 |
| src/cli.ts | 3,301 |
| src/core/cycle.ts | 2,933 |
| src/core/search/hybrid.ts | 2,453 |
| src/core/engine.ts | 2,320 |
| src/core/search/mode.ts | 1,232 |
Guards: 47 scripts/check-* files; 3 self-tested (harness <1s, budget 30s);
single registry established (guards-manifest.tsv; `check:all` deleted; 3
previously-unreachable guards wired into verify).
Test infra: PGLite snapshot default-on for `bun run test`. Per-PGLite-file:
1.63s cold → 0.91s snapshotted (measured on test/db-lock-fencing.test.ts).
Full-suite wall-clock (post-snapshot): recorded in the W0 ship notes — see
the run banner of the W0 PR's `bun run test` evidence.
Retrieval canary: NOT RUN at W0 (production brain locked by live serve; W0
touches no search paths). REQUIRED before W1 lands.
Verified-bug status at W0 ship: cycle-lock refresh + fencing (TODO-OPS-2
closed), stall-death parent unblock, started_at ×4, modality carry, import
typed aborts, lint single-pass, prompt EOF safety, guard self-test harness,
snapshot default-on. W0a superseded by master's WP1/D7 (port-ledger in the
plan file).
+1 -1
View File
@@ -32,7 +32,7 @@ pure win. See the per-verb latency table in
calls `context_pack` / `delta` over MCP (they are on `--surface verbs`) or the
CLI (`gbrain context-pack`, `gbrain delta`) at the boundary and injects the
returned `text` (or renders the structured arms). This is the portable path —
no hooks required. It is the primary path for Codex (which has no hooks) and
no hooks required. It is the primary path for Codex and opencode (no wired hooks) and
for Postgres brains (which have no local IPC socket).
- **Push (PGLite + Claude Code):** the bundled hook framework fires
automatically at `SessionStart` (injects a warm pack — including the
+101 -6
View File
@@ -1,7 +1,7 @@
# GBrain Bootstrap — your harness as your agent
`gbrain bootstrap` turns a Claude Code or Codex session into a persistent personal
agent: identity files rendered from your own answers, a local PGLite brain,
`gbrain bootstrap` turns a Claude Code, Codex, or opencode session into a
persistent personal agent: identity files rendered from your own answers, a local PGLite brain,
per-turn context, session-triggered schedules, and a private GitHub repo as the
agent's durable, portable body. This guide is the full contract — what gets
installed, what runs when, what it can and cannot do, and how to undo all of it.
@@ -19,7 +19,7 @@ follows is `BOOTSTRAP_FOR_AGENTS.md` at the repo root, fetched at the
| Identity files (SOUL/USER/MEMORY/AGENTS/CLAUDE/HEARTBEAT/ACCESS_POLICY/GITHUB) | your workspace folder | loaded at session start |
| `agent.json` manifest + `brain/`, `memory/`, `skills/`, `state/` | workspace | — |
| Local brain (PGLite) | `~/.gbrain/` (never in the repo) | while a session's MCP serve is open |
| MCP registration (`gbrain serve`) | Claude Code: project scope by default; Codex: user-global (no scope flag) | spawned by your harness per session |
| MCP registration (`gbrain serve`) | Claude Code: project scope by default; Codex: user-global (no scope flag); opencode: user-global by default (project scope is an explicit opt-in — see the degradation matrix) | spawned by your harness per session |
| Hooks (Claude Code, ON by default) | local installs: `.claude/settings.local.json` (gitignored); cloud sandboxes: the COMMITTED `.claude/settings.json` (PATH-resolved, fail-open commands) | each prompt; fail-open; `--no-hooks` opts out at install, `GBRAIN_HOOKS=0` disables at runtime |
| Per-turn persistence | Stop hook → debounced, detached scan-gated push (per workspace; 5 min default, every turn in cloud sandboxes) | after each assistant turn; `GBRAIN_STOP_PUSH=0` disables; `GBRAIN_STOP_PUSH_DEBOUNCE_MIN` / config `hooks.stop_push_debounce_min` tune it |
| Session persistence | SessionEnd hook → scan-gated commit+push | at session end (note: the harness never fires SessionEnd on `/exit` — the per-turn push is what covers that) |
@@ -154,8 +154,92 @@ you'd apply to any journal: write what you'd be comfortable persisting.
| API keys | everything (keyless mode) | semantic search, auto-extraction |
| GitHub / `gh` | full local agent | off-machine durability (repo re-runnable later) |
| Hooks (Claude Code) | pull protocol via AGENTS.md gates | automatic per-turn context + session-end persistence |
| Codex (no hook system, no MCP scope flag) | pull protocol + MCP tools | per-turn push (stated plainly; not oversold) + the ability to confine MCP reach to one folder (`codex mcp add` is always user-global) |
| Codex (no wired hooks, no MCP scope flag) | pull protocol + MCP tools | per-turn push (stated plainly; not oversold — codex 0.147+ ships a hook system, but gbrain does not wire it yet) + the ability to confine MCP reach to one folder (`codex mcp add` is always user-global) |
| opencode (no wired hooks; scope INVERTED: user-global by default) | pull protocol (opencode reads AGENTS.md natively) + MCP tools; project scope available as an explicit opt-in | per-turn push (opencode ships a plugin/event system, but gbrain does not wire it yet). The project-scope default is deliberately NOT offered: opencode spawns project-config servers with no trust prompt, so a committed entry would auto-execute on every collaborator machine |
| Second simultaneous session | first session unaffected | second session's brain tools fail politely (one live serve per brain — v1 contract) |
| Postgres brain (incl. harness mode) | MCP tools every session + pull protocol | per-turn hook injection (`no_pglite_path`: the hook IPC socket is PGLite-only today; hooks stay pre-wired and light up when the engine-uniform listener lands) |
## Local harness mode (`gbrain bootstrap harness`, #4043)
The workspace install above is built for a human's laptop. A box run by an
agent framework (your OpenClaw, or anything that shells out to `claude -p` /
codex exec) already hosts a brain and a running `gbrain serve --http` — and
those framework-spawned sessions get zero brain access by default. Harness
mode wires them in one command, with no `agent.json` and no interview:
gbrain bootstrap harness --yes
- Mints a **least-privilege** bearer token (scopes `read+write`, stored in the
`access_tokens.scopes` column; reads span the brain's federated sources).
Re-runs rotate mint-first: the previous token is revoked by id only after
the new one is wired and smoke-tested, so clients are never dead mid-swap.
The smoke sends a deliberately invalid credential first — an endpoint that
accepts anything is not this brain's serve — and a failed smoke rolls the
wiring back (fresh registrations removed, replaced ones restored, the
headless pre-approval stripped) and retires the fresh mint immediately, so
nothing live is ever left pointed at an unverified endpoint. Prior wiring
is only cleaned up after the replacement verifies.
- Claude Code: user-scope HTTP MCP registration, `mcp__gbrain` pre-approved in
user-scope `permissions.allow` (headless `claude -p` blocks MCP tools
without it), and the five lifecycle hooks — user scope by default, or
exactly the dirs you pass with repeatable `--project` (never both; the two
would double-fire every event). `--no-capture` wires context injection only
and skips the transcript-capture events.
- Codex: one managed `[mcp_servers.gbrain]` block with the bearer token
INLINE in the codex config (0600) — framework-spawned codex inherits no
shell profile, so the env-var lane the `connect` path uses would never
reach it.
- opencode: one managed `mcp.gbrain` remote entry with the bearer header
INLINE in the user-global JSONC config (0600), written by the same
comment-preserving editor the workspace lane uses — the `{env:…}`
interpolation the `connect` path prefers would resolve empty under a
framework-spawned opencode for the same no-shell-profile reason.
Note: downgrading gbrain below the release that introduced opencode support
after wiring it leaves the opencode entry in place for manual removal —
edit the opencode config by hand, or re-upgrade and run
`gbrain bootstrap harness --remove`.
- Honesty on Postgres brains: per-turn injection is degraded (the matrix row
above); MCP is the active seam and the summary says so.
- `--status [--json]` probes the live truth (serve health, token validity via
host-config recovery — the Claude Code lane only recovers a bearer from a
registration whose URL matches the receipt; the codex managed block is read
from the exact path the receipt records — and per-target states) with a
cron-honest exit contract: 0 only when the serve, token, and every target
verify and the rotation has converged (honest degrades count as OK); 1 on
an unreachable serve, a failed token verify, failed or pending targets, an
unconverged rotation, or a half-removed install whose token still awaits
revocation. With no install at all it says so and exits 0 (2 under
`--json`, so machine callers can tell absence apart). `gbrain doctor`
carries a matching `bootstrap_harness_health` check. `--json` on the
install itself emits a single machine-readable document on stdout (prose
goes to stderr).
- The full flag surface lives in `gbrain bootstrap --help`: `--url`/`--port`
point at a non-default serve (a non-loopback `--url` is refused unless you
also pass `--token`, which flips into registrar mode — MCP wiring only, no
hooks, nothing minted), `--force` replaces a foreign same-name MCP
registration, `--name` renames the server, `--harness` picks the hosts,
and `--no-hooks` skips hook wiring entirely.
- `--remove` tears down exactly what the machine-level receipt
(`<home>/bootstrap/harness.json`) records — host removals are engine-free
and run even while a serve is live; the token revoke defers with exact
instructions if a live PGLite serve holds the brain. `gbrain bootstrap
uninstall` removes harness wiring first, automatically.
- Everything is stated before it happens; non-interactive runs require
`--yes`. Close active Claude Code sessions for the cleanest user-scope
settings writes (the host also writes that file).
PGLite note: minting needs the single-writer lock, so on a PGLite brain
either pre-mint (`gbrain auth create bootstrap-harness --scopes read,write`
while the serve is stopped) and pass `--token`, or stop/re-run/restart.
Postgres brains mint fine while the serve runs. A token you supply is never
revoked by `--remove` or rotation (it is not the harness's to revoke) —
retire it yourself with `gbrain auth revoke` when you're done with it.
Binary-downgrade note: token scoping is data-only (no migration), so a gbrain
binary OLDER than the release that shipped it verifies every scoped token as
FULL-ACCESS — the old verify path never reads the scopes column. If you
downgrade after a harness install, revoke the scoped tokens first
(`gbrain auth revoke` with the id flag) and re-mint once you upgrade again.
## Multi-device
@@ -195,11 +279,16 @@ that changed shape, a harness that stopped calling our MCP server):
keyless-`init` → interview → render → `gbrain bootstrap hooks --harness codex`
path (executing the real `codex mcp add` into a hermetic `~/.codex/config.toml`),
asserts the rendered `AGENTS.md` carries the Gate-3 brain-first pull protocol
(Codex has no hook system, so the pull protocol is its per-turn seam), then
(gbrain does not wire Codex hooks yet, so the pull protocol is its per-turn seam), then
spends one live `codex exec` turn to prove real codex → gbrain MCP → brain →
a seeded, brain-only fact (falling back to a shell `gbrain query` if headless
stdio-MCP is unavailable).
opencode's real-binary door lives in
`test/e2e/install-real-opencode.serial.test.ts` (its writer-parity leg
handshakes gbrain's direct JSONC registration through the actual binary);
`docs/TESTING.md` carries the full door inventory and cadence policy.
These pay real API cost and take 30s2min per turn, so they are NOT in the PR
shard. Everything is hermetic (temp `HOME` / `CODEX_HOME` / `CLAUDE_CONFIG_DIR` /
`GBRAIN_HOME` per test — the operator's real `~/.claude`, `~/.gbrain`, `~/.codex`
@@ -220,12 +309,16 @@ bun test test/e2e/bootstrap-real-codex.serial.test.ts
## DX exploration harness (developer instrument, not a test)
The door tests prove the install WORKS; they say nothing about how it FEELS.
`test/helpers/tty-harness.ts` spawns any CLI (gbrain, `claude`, `codex`) under a
`test/helpers/tty-harness.ts` spawns any CLI (gbrain, `claude`, `codex`, `grok`, `opencode`) under a
real pseudo-terminal (Bun's `terminal:` spawn option) and records every output
burst with a millisecond timestamp, so unnecessary pauses become a measurable
artifact (`computeStalls``stalls.md`) instead of a vibe. Same hermetic env as
`agent-harness.ts`; pure helpers are unit-tested in `test/tty-harness.test.ts`
(zero subprocesses, PTY smokes self-skip where `terminal:` is unavailable).
The harness itself also backs one required-CI test: `test/init-picker-pty.serial.test.ts`
asserts the interactive `gbrain init` pickers under a real PTY (see the
TTY decision table in `docs/TESTING.md`). The DX-exploration layer below stays
an instrument — nothing in it asserts.
`scripts/dx-explore.ts` drives it to capture the fresh-user funnel as timestamped
transcripts under `.context/dx-runs/` (gitignored — nothing asserts, no CI):
@@ -235,6 +328,8 @@ bun run scripts/dx-explore.ts help # comprehension surfaces (no key
bun run scripts/dx-explore.ts init [--keyless] # interactive init, naive-user autopilot
bun run scripts/dx-explore.ts claude-install # REAL claude running the paste-in bootstrap
bun run scripts/dx-explore.ts codex-install # REAL codex, same
bun run scripts/dx-explore.ts opencode-install # REAL opencode running the paste-in bootstrap
bun run scripts/dx-explore.ts grok-install # REAL grok, brain-only GROK.md install (no bootstrap path)
bun run scripts/dx-explore.ts drive -- gbrain init # manual: steer a live TUI via a file channel
```
+43
View File
@@ -115,6 +115,49 @@ it nightly and Phase 4 below (plus most of Phase 2's hygiene checks) is
covered. The pseudocode that follows is the harness-side variant for agents
that also do LLM-driven entity sweeps and memory consolidation on top.
### Synthesis cost control: the triage cascade
The synthesize phase is a two-stage cascade: a cheap scored triage
(utility-tier model, one call per new transcript) gates the expensive
per-transcript synthesis subagents. The dials:
- `dream.triage.threshold` (default 0.5) — the gate. Scores are cached, so
retuning it re-gates instantly with **zero** new LLM calls. Raise it if too
much routine content synthesizes; lower it if real signal is being skipped.
- `models.dream.triage` — the triage model (default: utility tier / Haiku).
- `dream.triage.max_chars` (default 24000, floor 1000) — per-transcript
sample window (head/middle/tail) sent to the judge. Not part of cache
validity — after changing it, `gbrain dream retriage --force` re-judges
under the new sampling.
- `dream.triage.max_tokens` (default 2048, floor 256) — judge output budget.
- `dream.triage.concurrency` (default 4, clamped 116) — concurrent judge
calls.
- `dream.synthesize.max_turns` (default 16) — synthesis turn budget. The
triage map hands the subagent pre-extracted segments, so the mid-tier
default model (`models.dream.synthesize`, tier `reasoning`) with a 16-turn
budget is the intended pairing — frontier-model overrides are unnecessary
and slow the queue. Completeness comes from triage coverage (every file
scored, minus files deferred under the `max_ms` budget below) plus
segment-guided prompts, not model size. If written-page counts
drop after upgrading, set it back to 30 and check
`details.synthesis.avg_turns` for cap pressure.
- `dream.triage.max_ms` (default 5 min) — per-cycle wall-clock budget for
judging NEW files; a big cold corpus triages across a few cycles (cached
files are free). Deferred files are labeled "not yet triaged", never
silently rejected.
- `dream.synthesize.max_submissions_per_source_per_day` (default 0 = off) —
opt-in backstop cap on synthesis jobs per source; 200/day is a sane value
for busy deployments.
Maintenance recipe — after changing the threshold, upgrading through a
`TRIAGE_VERSION` bump, or to drain a queued synthesis backlog:
```bash
gbrain dream retriage --dry-run # what would change (zero LLM calls)
gbrain dream retriage --reconcile-queue # re-score + cancel below-threshold queued jobs
gbrain dream retriage --audit-rejects 20 # synthesis-model second opinion on 20 rejects
```
### What It Does
```
+64 -1
View File
@@ -87,6 +87,69 @@ check warns if what you asked for isn't what's actually running (e.g. a
negative value denied without privilege, or an OS `RLIMIT_NICE` clamp). This
is distinct from the concurrency / inflight cap and composes with it.
### Per-job process isolation (`--job-isolation process`)
By default all concurrency slots execute inside one worker process. A
handler that ignores its abort signal can only be force-evicted — the
promise is abandoned, still running, still holding connections and memory —
and any worker exit destroys every in-flight job at once. With isolation on,
each claimed job runs in its own child process: a stuck handler is
group-SIGKILLed for real (group signaling under Bun falls back to POSIX
`/bin/kill`; if that's unavailable the worker logs that isolation is
degraded), a crash or OOM in a child takes that one job instead of all N,
and the OS reclaims every leaked resource when the child dies:
```bash
# Recommended for long-running LLM-bound handlers (subagent):
gbrain jobs supervisor --concurrency 4 --job-isolation process
# Bare worker, or durably via env:
GBRAIN_JOB_ISOLATION=process gbrain jobs work --concurrency 4
```
How it works: the worker keeps claim, lock renewal, and all result
recording; the child (an internal `run-child` entrypoint of the same gbrain
binary) re-validates the claim, runs the handler with its own small engine
pool, and reports one atomic outcome file. Handler-error semantics are
preserved across the boundary (unrecoverable → dead, rate-lease → no attempt
burned, everything else → normal backoff). On worker shutdown children get
the drain window to finish and report; a child killed before reporting is
released with no attempt burned. If the worker dies hard, the orphaned child
self-terminates via a parent-liveness watchdog and the stall sweeper
requeues the job after lock expiry — the lock token fences the orphan's
queue writes (result recording, progress, state transitions) into no-ops.
The handler's own side effects (page writes through its engine) can still
land until the watchdog stops the child; that window is the watchdog's
poll + grace, not unbounded.
Sizing notes:
- **Connections:** each child opens its own small pools (read 3 by default,
override via `GBRAIN_JOB_CHILD_POOL_SIZE`; direct 1). Worked example at
concurrency 15: 15×(3+1) + the worker's 10+3 ≈ **73 client connections**
total — 55 ride the transaction-pooler lane (multiplexed, no extra server
backends) and 18 are lazy direct session-lane connections, each holding a
real server backend while open. Budget the pooler-lane count against your
pooler's client limit and the session-lane count against
`max_connections`.
- **Memory:** `--max-rss` covers the WORKER process only in this mode
(handler memory lives in the children; the worker prints a note when both
are set). There is no per-child RSS cap yet — a runaway child is contained
only by host/container limits. Size host memory for concurrency × handler
footprint.
- **Spawn cost:** ~0.31s per job (engine connect included) — noise for
long-running handlers, meaningful for sub-second ones (`lint`,
`backlinks`). Keep those inline or on a separate inline worker.
- **Security note:** the child receives the job's lock token via env. It is
a *fencing* token (split-brain protection), not a secret — same-user env
already contains the database URL.
- **Child CLI resolution:** the worker fail-fast validates the child CLI at
startup (compiled `gbrain` binary, bun-dev fallback, or the
`GBRAIN_JOB_CHILD_CLI` env override — the ops/test escape hatch). Three
consecutive child spawn/bootstrap failures self-exit the worker as
unhealthy (a deterministically broken child CLI) for process-manager
restart instead of burning attempts across the queue.
### Which supervisor when?
The supervisor solves in-process crash recovery. Platform-level
@@ -140,7 +203,7 @@ Three-command pattern an agent can drive without shell archaeology:
```bash
# Start (returns PIDs + pid_file on stdout as JSON, then detaches)
gbrain jobs supervisor start --detach --json
# → {"event":"started","supervisor_pid":1234,"worker_pid":1235,"pid_file":"/Users/you/.gbrain/supervisor.pid"}
# → {"event":"started","supervisor_pid":1234,"pid_file":"/Users/you/.gbrain/supervisor-<brain-id>.pid","detached":true}
# Check health (machine-parseable JSON, no log scraping)
gbrain jobs supervisor status --json
+2 -2
View File
@@ -12,7 +12,7 @@ The push channels share one zero-LLM core (`src/core/context/volunteer.ts`):
| `reflex` | automatic, inside the context engine | default-on for plugin hosts; nothing to call |
| `op` | `gbrain volunteer-context` / MCP `volunteer_context` | agents without the plugin; one call per turn |
| `watch` | `gbrain watch` | stream a transcript in, volunteered pages stream out |
| `claude-code` / `codex` | `gbrain hook user-prompt` (registered by `gbrain bootstrap`) | per-prompt injection inside a harness; see "Harness hooks" below |
| `claude-code` / `codex` / `opencode` | `gbrain hook user-prompt` (registered by `gbrain bootstrap`) | per-prompt injection inside a harness; see "Harness hooks" below |
## How it decides
@@ -74,7 +74,7 @@ this channel production-grade rather than spammy-and-invisible:
- **The feedback loop.** The serve logs each DELIVERED block's volunteered
pages and pointers to `context_volunteer_events` under the hook's channel
(`claude-code` by default; a codex hook registration passes
`--harness codex`). `gbrain volunteer-context --stats` then shows
`--harness codex` / `--harness opencode`). `gbrain volunteer-context --stats` then shows
per-harness precision, and `gbrain doctor`'s `volunteer_channels` check
shows which channels actually fire, with guidance for the two quiet cases:
"hook installed but never registered (restart the session)" and "registered
+41 -3
View File
@@ -71,7 +71,10 @@ gbrain jobs get <id>
## Rescue actions (in order of escalation)
```bash
# Force-kill a single stuck job:
# Cancel a single stuck job (inline mode: cooperative — the handler must
# observe its abort signal, and after 30s it is force-evicted from tracking
# but the promise keeps running; with --job-isolation process the child is
# actually SIGTERM→SIGKILLed once cancellation is detected):
gbrain jobs cancel <id>
# Clear a specific job entirely (last resort):
@@ -85,8 +88,20 @@ gbrain jobs smoke --wedge-rescue
- **stalled-forever** — A worker claimed a job, started executing, and has
held the row for over an hour. The wall-clock sweep evicts jobs past
2× `timeout_ms`; if one's still active, either no `timeout_ms` was set
or the sweep is newly deployed and this job predates it. Cancel it.
2× `timeout_ms`. Long-lane handlers (subagent, autopilot-cycle,
embed-backfill, …) always have a budget now: it stamps at submit, is
COALESCEd from `HANDLER_DEFAULT_TIMEOUT_MS` at claim for legacy NULL rows,
and migration v128 backfilled rows that predate both. `gbrain jobs get <id>`
prints the effective budget and which kill path applies. If a short-lane
job is still active with no budget, the null-default sweep
(2 × lock-duration × max_stalled) evicts it within minutes. Cancel it if
you can't wait.
- **duplicate cycles** — Historic brains could accumulate byte-identical
waiting `autopilot-cycle` rows when a job stalled in `active`. v128
cancelled that backlog (newest ticker-keyed row per source survives), and
the `maxPending` dispatch guard prevents new accumulation. Suppressed
dispatches are visible in `jobs stats` (Backpressure line) and the
backpressure audit JSONL.
- **waiting-depth** — Submitters are piling up jobs faster than workers
drain them. Set `--max-waiting N` on the submission or on the programmatic
`queue.add()` call. If you want a taller pile, raise the threshold via
@@ -107,6 +122,29 @@ claiming. Start one:
GBRAIN_ALLOW_SHELL_JOBS=1 gbrain jobs work --concurrency 4
```
## Reading the DB-probe verdicts (pool starved vs server unreachable)
When the worker's health probe fails repeatedly, the terminal
`[health] DB probe failed N consecutive times (verdict: ...)` line — and the
`unhealthy` payload the supervisor sees — carries a verdict that names the
failing LAYER (the intermediate `(N/3)` lines log only the failure detail).
Read it before touching anything — the historical failure mode here was
hours spent evaluating a database instance upgrade while the server sat at
10% of max_connections.
| Verdict | What it means | What to do |
|---|---|---|
| `pool_starved` | The read-pool probe failed but the DIRECT-lane probe succeeded — the database server is reachable; the fault is in the transaction-pooler path (client pool exhaustion or a pooler-layer fault; the probe deliberately does not distinguish the two). | Look at client-side load: long-running handler queries holding slots, `GBRAIN_POOL_SIZE` too small for the workload, or a pooler-layer incident. Do NOT resize the database. The worker exit is correct recovery — it frees every client-held slot. |
| `server_unreachable` | Both the pooler lane and the direct lane failed. | Check connectivity/capacity first: network, DNS, the database itself. Both-lanes-failed is the evidence — credential/config errors or a saturated direct lane can also land here, so glance at the probe detail text before concluding the server is down. |
| `unknown` | The read probe failed and no direct lane exists to disambiguate (single-pool mode: non-Supabase, kill switch active, or no derivable direct URL). | Check the startup log for the single-pool warning; consider `GBRAIN_DIRECT_DATABASE_URL` so future incidents self-diagnose. |
The `gbrain-tracked in flight` counts in the message are a tracked SUBSET
(raw/direct/reserved/transaction seams only) — most template-path queries are
untracked, so `0 in flight` next to a `pool_starved` verdict means the
saturation lives in that untracked traffic or at the pooler layer itself,
not that the pool is idle. The verdict, not the counts, is the
authoritative signal.
## Related
- [Minions worker deployment](minions-deployment.md) — supervisor lifecycle,
+6 -1
View File
@@ -11,7 +11,12 @@
Recent versions of the Codex CLI (`@openai/codex`) support remote
streamable-HTTP MCP servers with a bearer token read from an environment
variable. The token lives in your shell env, not in Codex's config file.
variable. On THIS page's `gbrain connect` path the token lives in your shell
env, not in Codex's config file. The exception is `gbrain bootstrap harness`
(local agent-framework boxes): framework-spawned codex inherits no shell
profile, so that lane writes the token INLINE into a managed, 0600
`[mcp_servers.gbrain]` block in the codex config — stated in its consent
block, removable with `gbrain bootstrap harness --remove`.
## Fastest path: `gbrain connect`
+15 -8
View File
@@ -4,9 +4,11 @@
> PKCE, refresh rotation, optional DCR), an embedded React admin dashboard at
> `/admin`, scoped operations, and a live SSE activity feed. Legacy bearer
> tokens still work — `verifyAccessToken` falls back to the `access_tokens`
> table and grandfathers tokens to `read+write+admin`. Both the legacy fallback
> and the OAuth tables work on PGLite and Postgres (both engine schemas carry
> `access_tokens`). See [SECURITY.md](../../SECURITY.md) for env vars and
> table; tokens with no `scopes` grant are grandfathered to `read+write+admin`,
> while tokens minted with `gbrain auth create --scopes …` (or by
> `gbrain bootstrap harness`) are honored at exactly their granted scopes.
> Both the legacy fallback and the OAuth tables work on PGLite and Postgres
> (both engine schemas carry `access_tokens`). See [SECURITY.md](../../SECURITY.md) for env vars and
> tunable defaults.
Access your brain from any device, any AI client. GBrain ships two transports:
@@ -68,8 +70,9 @@ This requires:
2. A public tunnel (ngrok, Tailscale, or cloud host)
3. A bearer token created via `gbrain auth create <name>`
Existing bearer tokens are grandfathered as `read+write+admin` scopes on the
OAuth-capable HTTP server, so no migration is required.
Existing bearer tokens (no `scopes` grant) are grandfathered as
`read+write+admin` on the OAuth-capable HTTP server, so no migration is
required; `gbrain auth create --scopes read,write` mints narrowed tokens.
## OAuth 2.1 Setup
@@ -223,7 +226,8 @@ Write ops can additionally be fenced per client with `--bound-slug-prefixes`
## Legacy Bearer Token Setup
Bearer tokens are the simple path when you don't need per-client scoping.
They grandfather to `read+write+admin` scopes on the HTTP server.
Without a `--scopes` grant they grandfather to `read+write+admin` on the
HTTP server; pass `--scopes read,write` at creation to narrow one.
### 1. Set up the tunnel
@@ -249,8 +253,11 @@ gbrain auth list
gbrain auth revoke "claude-desktop"
```
Tokens are per-client. Create one for each device/app. Revoke individually
if compromised. Tokens are stored SHA-256 hashed in your database.
Tokens are per-client. Create one for each device/app. Names are not
unique: `gbrain auth revoke "<name>"` revokes EVERY active token carrying
that name — use `gbrain auth list` (shows each token's id and scopes) and
`gbrain auth revoke --id <uuid>` to revoke exactly one. Tokens are stored
SHA-256 hashed in your database.
### 3. Connect your AI client
+190
View File
@@ -0,0 +1,190 @@
# Grok Build CLI pin — observed behavior notes (v1.0.4)
Dev-facing companion to [GROK.md](GROK.md): every fact below was OBSERVED against a
real install (2026-08-14), not researched from docs. The claw-test GrokRunner, the
install door e2e, and the heavy-tests grok-door CI job assert exactly these shapes —
when Grok Build releases change them, update this file, the workflow pins, and the
affected assertions together (`scripts/check-grok-pin.sh` in `bun run verify` enforces
the workflow-side match).
Naming note: **grok** (xAI Grok Build CLI, `XAI_API_KEY`) is not **groq** (Groq Inc.
inference, `src/core/ai/recipes/groq.ts`, `GROQ_API_KEY`) and not **ngrok** (tunnels).
<!-- grok-pin: distribution_kind=npm -->
<!-- grok-pin: npm_package=@xai-official/grok -->
<!-- grok-pin: npm_version=1.0.4 -->
<!-- grok-pin: npm_integrity=sha512-Nu3SFXTqwvCQr/LQFwrQYgngJhUQwX2h9ZSgzW4HowidjbPBWtMVO0xI88d2z6/zlDSNaT5YP/uk+2DthKQMsg== -->
<!-- grok-pin: npm_linux_x64_integrity=sha512-Dan2LfKcFBiabuDGHaGgMT8Ndzibo2ljvSjh4MlpV5117JL+S/0KMbdyYpk+13d7t+4znniW1cm+rRwUGSAvtw== -->
<!-- grok-pin: npm_linux_arm64_integrity=sha512-zGK42Eq3ZmIa7cSVnl6CiJ4cxTCMsNLQCmCoLJhy5eZXfAvZ1DA3K3HXmKCj4OScX8SalYlp7mx8HWl9Y6gytw== -->
<!-- grok-pin: grok_version=1.0.4 -->
<!-- grok-pin: installer_sha256=43d0943123edade1383a476a4f778674877acee7c1f98a00f094c4a0f7349321 -->
<!-- grok-pin: observed_date=2026-08-14 -->
## Pin
- **Grok Build v1.0.4**, `grok --version` output shape: `grok 1.0.4 (d846eb93d94d)`
(version + build hash; the door's shape assert is `/^grok \d+\.\d+\.\d+ \([0-9a-f]+\)$/`).
- **Provisioning (CI + local): pinned npm install**`@xai-official/grok@1.0.4`,
registry integrity `sha512-Nu3SFX…`. The package fans out to
`@xai-official/grok-{darwin,linux,win32}-{arm64,x64}` optional deps at the same
version; the CI job pins the LINUX payload integrities too (stamps above) because
the wrapper's integrity covers only the wrapper tarball — the platform sub-package
is the binary that executes. Load-bearing assumption, stated explicitly: npm
version-immutability (a published version cannot be replaced on npmjs; only a new
version or an unpublish, both of which fail the pinned install loudly).
- Installer path (fallback only): `https://x.ai/cli/install.sh`, sha256
`43d0943123edade1383a476a4f778674877acee7c1f98a00f094c4a0f7349321` (17,686 bytes).
It SUPPORTS version pinning (`bash -s <X.Y.Z>`) and downloads versioned artifacts
`grok-<version>-<os>-<arch>` from `https://x.ai/cli` (fallback GCS bucket
`grok-build-public-artifacts`), self-checks `--version` post-download. Platform
string from `uname -s`/`uname -m` with a Rosetta correction on Apple Silicon.
- Verified against macOS arm64; npm `os`/`cpu` matrix covers linux x64/arm64 for CI.
## GROK_HOME — HONORED (verified)
`GROK_HOME=<tmp> HOME=<tmp> grok mcp list|add|doctor` read+write `<tmp>/config.toml`
and do NOT touch `~/.grok`. Belt-and-suspenders (HOME + GROK_HOME both to tmp) stays
in the door anyway. NOTE what grok writes into `$GROK_HOME` on EVERY run (tripwire
exclusions — these are VOLATILE): `active_sessions.lock`, `active_sessions.json`,
`bin/grok-<version>` (it copies its own binary in), `logs/unified.jsonl`,
`docs/user-guide/*.md` (it ships its user guide into the home), `leader.sock` (a
leader daemon socket; `--leader-socket <PATH>` overrides). The tripwire hashes ONLY
`config.toml` + credential-class files, never the volatile set.
## One-shot (`-p`)
- `grok -p "<prompt>"` (`-p, --single`) prints the response to stdout and exits.
- `--output-format plain|json|streaming-json|streaming-messages-json` (default plain;
`streaming-json` = NDJSON of native ACP session updates; `streaming-messages-json` =
Anthropic Messages wire format; `--include-partial-messages` adds deltas).
- **Keyless one-shot: exit 1**, message (verbatim, both stdout and stderr):
`Not signed in. To authenticate without a browser, run:\n grok login --device-code\n\nAlternatively, set the XAI_API_KEY environment variable or run `grok login` on a machine with a browser.`
`hasGrokAuth()` = non-empty `XAI_API_KEY`; the TTY scenario's keyless early-stop
matcher is `Not signed in`.
- Cost/toolset flags that EXIST (observed in --help): `--always-approve`,
`--permission-mode default|acceptEdits|auto|dontAsk|bypassPermissions|plan`,
`--tools <LIST>`, `--disallowed-tools <LIST>`, `--allow/--deny <RULE>`,
`--disable-web-search` (dedicated kill for web search + fetch — the door SMOKE uses
THIS, not a tools list), `--max-turns <N>`, `-m/--model`, `--reasoning-effort`
(alias `--effort`), `--cwd <PATH>`, `--rules`, `--prompt-file`, `--prompt-json`,
`--json-schema` (implies json output), `--verbatim`, `--sandbox <PROFILE>`
(env `GROK_SANDBOX`), `--no-memory`, `--no-plan`, `--no-subagents`.
- There is NO auto-update CLI flag. Auto-update is config: `[cli] auto_update = true`
is the DEFAULT — hermetic homes MUST seed `[cli] auto_update = false`. Manual
updater: `grok update [--check --json --version <V> --force-reinstall --alpha]`.
- `.envrc` gotcha: `load_envrc = true` by default — grok loads `.envrc` from the
working directory. Door/live spawns pin `cwd` to tmp workspaces partly for this.
## Auth + model pin (non-interactive)
- Keyless error pinned above; `grok login --device-code` exists for headless
interactive auth; `XAI_API_KEY` env is the documented headless path (its end-to-end
smoke is **pending auth** — no key was available at observation time; the door's
paid tier stays skip-gated until then, per plan D0).
- `grok models` works KEYLESS (exit 0): prints `You are not authenticated.`, then
`Default model: grok-4.6` and the visible list (`grok-4.6 (default)`, `grok-4.5`).
Authenticated list may be larger; per-turn cost pins are **pending auth**.
- Model pin mechanism: per-call `-m <model>` (authoritative in tests — immune to
config rewrites) and `[models] default = "<model>"` in config.toml.
## `grok mcp add` — THE big observed facts
- Shape: `grok mcp add <name> [-e KEY=value]... [-s user|project] [-t stdio|http|sse] -- <command> [args...]`
— everything after `--` is the server argv. **`-e/--env` is REPEATABLE, one
KEY=value per flag** (their docs pin this as a breaking change from earlier
releases: `use -e A=1 -e B=2, not --env A=1 B=2` — the hermes replace-bug class is
fixed upstream). Server names: letters, numbers, hyphens, underscores only.
- **Add is LAZY: exit 0 always, NO handshake at add time, no interactive prompt**
(`Added stdio MCP server 'gbrain' … to user config` / `File modified:
$GROK_HOME/config.toml`). Adding a NONEXISTENT command also exits 0. Never assert
add's exit code; never treat `enabled = true` in the saved TOML as a handshake
proof (it is written unconditionally).
- Scope: `-s user` (default) → `~/.grok/config.toml`; `-s project`
`./.grok/config.toml` (committable; reference secrets as `${VAR}`).
- **Bare command names resolve via the CALLER'S PATH** (verified): registering
`-- gbrain serve --surface verbs` with a PATH-prefixed bin dir works — doctor
resolved bare `gbrain` to the staged wrapper and completed the handshake. The
bun-run wrapper shim (`#!/bin/sh\nexec bun run <abs>/src/cli.ts "$@"`) works as the
staged binary (the fallback lane when a compiled binary is unavailable).
- Startup timeout: per-server `startup_timeout_sec` (default 30) or global env
`GROK_MCP_STARTUP_TIMEOUT_SECS` (seconds) / `MCP_TIMEOUT` (ms, Claude-compatible).
The bun-run wrapper cold-transpiles slowly — the door sets 60+.
## Saved config schema (verbatim, from a real add)
```toml
[mcp_servers.gbrain]
command = "/tmp/<staged-bin>/gbrain"
args = [
"serve",
"--surface",
"verbs",
]
enabled = true
[mcp_servers.gbrain.env]
GBRAIN_SOURCE = "workspace"
GBRAIN_HOME = "/tmp/<brain-home>"
```
Full schema keys (from grok's own shipped user guide, `$GROK_HOME/docs/user-guide/`):
`command`, `args`, `env`, `enabled` (default true), `startup_timeout_sec` (default
30), `tool_timeout_sec` (default 6000), `tool_timeouts`.
## Probes — the HONEST discriminator exists
- **`grok mcp doctor <name> --json`**: SPAWNS the server for real. Good server →
**exit 0** with checks `command found` / `server started` / `handshake OK`
(`"detail": "protocol 2025-11-25"`) / **`7 tools discovered`** (the verbs surface's
seven verbs, proven keyless end-to-end). Broken server (nonexistent command) →
**exit 1**, check `command not found`, `passed: false`, plus a `hint`. THE door's
hard discriminator; the T4 doctor pre-flight gates the paid loop (plan M6 resolves
to the honest branch).
- Doctor `--json` also enumerates config **sources** with per-source status —
`~/.grok/config.toml`, `~/.claude.json`, `.mcp.json` — and each server carries a
`"source"` field (`"config"`, `".mcp.json"`, …): the T2b provenance assertion reads
this directly.
- `grok mcp list --json` → exit 0, array of `{command, args, env, enabled, name,
scope}`.
- `grok inspect` (keyless, exit 0) shows version, CWD, `Project trusted: yes/no`,
instructions, permissions, skills, agents — the config-discovery audit surface.
## Vendor-config fallback — TRUST-GATED (verified)
A project `.mcp.json` in the cwd is SEEN by doctor (source `found`, server listed
with `source: ".mcp.json"`) but the server check reports **`folder untrusted`** and
`mcp list` shows nothing until the folder is trusted (first-run trust flow). So:
fresh tmp HOME + fresh cwd ⇒ vendor entries structurally cannot activate (door
provenance guarantee), and on an operator's machine the fallback only engages for
folders they already trusted — the live-lane warning (operator `~/.claude.json`
carrying `mcpServers.gbrain`) still applies for trusted folders.
## When the door goes red (triage)
| Failure class | Signature | Remediation |
|---|---|---|
| npm pin drift | install step: version/integrity mismatch | Re-pin deliberately: bump `npm_version`+`npm_integrity` stamps here, re-run the re-observation checklist below, update workflow env pins (check-grok-pin.sh enforces the pair) |
| installer digest drift (fallback path) | `sha256sum -c` fails on install.sh | Diff the new installer, re-pin `installer_sha256` after review |
| version drift mid-run | `grok --version` re-check ≠ pinned | Auto-update engaged — verify `[cli] auto_update = false` seeding; re-pin if a deliberate bump |
| blank XAI_API_KEY secret | named precondition/paid-sentinel failure | Admin adds/rotates the repo Actions secret (console.x.ai origin); keyless tier still ran |
| invalid/expired key | bad-key preflight fails (pin its message after first authed run) | Rotate the secret; no code change |
| tripwire fired | manifest mismatch on config/credential files only | True isolation breach — stop, inspect which file changed; volatile-path drift alone must NOT fire (bug in exclusions if it does) |
| real door regression | doctor checks or recall assert fail with pins intact | Bisect against the pinned version; file upstream if grok-side |
Re-observation checklist on a version bump: re-run the npm/installer pin captures
(§Pin), the help-surface diff (`--help`, `mcp --help`, `mcp add --help`), and the
mcp add → saved-TOML → doctor sequence (§add/§probes). The one-shot/auth/model
sections only need re-observation if their assertions start failing.
## Keyless TUI behavior (observed via the dx-explore PTY instrument)
Under a real PTY with no credentials, interactive `grok` plays a Braille-
pattern intro animation (U+2800-range glyphs) for a few seconds, then settles
(~6s) onto a SIGN-IN screen: "Approve in your browser to finish signing in"
plus a device code (and a ctrl+c hint). There is no unattended path past it.
Two hazards for PTY automation, both observed: the animation frames carry
zero word-like text (3+-letter runs) — a text-presence heuristic must count
letter runs, not enumerate glyphs; and pasting into the sign-in screen leaves
a persistent full-screen spinner redrawing at ~5 frames/sec, which starves
quiet-based settling and makes full-buffer ANSI stripping the hot loop
(strip bounded raw tails instead). Headless keyless is the clean
`Not signed in` error above. The `grok-install` dx scenario early-stops at
the sign-in copy (or a persistently textless screen) with the friction
recorded — that IS the keyless measurement.
## Supported-version policy
gbrain's grok integration is verified against **Grok Build v1.0.4** (this pin). The
canary CI leg (enabled with the secret) tracks latest and is continue-on-error; the
pinned lane is the deterministic gate. **Pending auth** (requires `XAI_API_KEY`):
paid one-shot smoke, authed model list + per-turn cost pins, credential-file
inventory after login (feeds evidence exclusions + TTY secretPaths), AUTHED
first-run TUI dialog copy (the keyless TUI + headless copies are pinned above).
+155
View File
@@ -0,0 +1,155 @@
# Connect GBrain to Grok Build
> This page is the MCP-registration reference for **Grok Build** — xAI's
> official `grok` CLI (early beta, subscriber-gated; not the community
> `superagent-ai/grok-cli`, which ships a colliding `grok` binary — see
> Troubleshooting). For the full brain install — CLI, engine, skills, dream
> cycle — follow [INSTALL_FOR_AGENTS.md](../../INSTALL_FOR_AGENTS.md) first;
> this page wires the finished brain into Grok Build over stdio MCP.
> The `gbrain bootstrap` persistent-personal-agent path is **not yet
> supported for Grok** (Claude Code and Codex only today) — brain-only
> install is what this page delivers.
Grok Build spawns `gbrain serve` as a local stdio subprocess. No server, no
tunnel, no token needed. Works with both PGLite and Supabase engines.
## Register (recommended)
```bash
grok mcp add gbrain -e "GBRAIN_HOME=$HOME" -- gbrain serve --surface verbs
```
`--surface verbs` exposes the seven-verb memory protocol (`recall`,
`remember`, `entity`, `synthesize`, `forget`, `context_pack`, `delta`
[MEMORY_VERBS v1](../protocol/MEMORY_VERBS_v1.md)) instead of the full
100+-op catalog — the recommended starting surface for coding agents.
Three facts about `grok mcp add`, all observed:
- **The env flag is repeatable, one `KEY=value` per flag** (`-e A=1 -e B=2`).
Server argv goes after `--`.
- **Registration is lazy.** The add writes config and exits 0 without
connecting — even for a nonexistent command. Verify with `grok mcp doctor`
(below), never with the add's exit code.
- **Scope:** the default writes to `~/.grok/config.toml`; add `-s project`
to write a committable `./.grok/config.toml` instead (reference secrets as
`${VAR}` in project scope — values are stored verbatim).
## Direct config (equally supported)
The add command writes an `[mcp_servers.gbrain]` block into
`~/.grok/config.toml` (or `./.grok/config.toml` with project scope; the
`GROK_HOME` env var relocates the user config dir). You can write it
yourself instead:
```toml
[mcp_servers.gbrain]
command = "gbrain"
args = ["serve", "--surface", "verbs"]
startup_timeout_sec = 60
enabled = true
[mcp_servers.gbrain.env]
GBRAIN_HOME = "/home/alice-example"
```
`startup_timeout_sec` defaults to 30; raise it (or export
`GROK_MCP_STARTUP_TIMEOUT_SECS`) if gbrain runs from source via `bun run`,
which cold-transpiles on first spawn. To remove gbrain, delete the block (or
set `enabled = false` to disable without losing the config).
## Zero-config vendor fallback
Grok Build also reads MCP registrations from `~/.claude.json`, `.cursor/mcp.json`,
and a project `.mcp.json` — at lower priority than its own config, and **only
for folders you have trusted** in Grok (fresh folders report
`folder untrusted` until you accept the trust prompt). If you already
registered gbrain for Claude Code, Grok may pick it up with zero
configuration. `grok mcp doctor --json` reports every source it consulted
and which one each server came from — check the `source` field to see which
config won before assuming the native one did.
## Verify
```bash
grok mcp list --json # entry: {"name":"gbrain","enabled":true,...}
grok mcp doctor gbrain # THE real probe: spawns the server
```
`grok mcp doctor gbrain` performs the actual handshake — expect the checks
`command found`, `server started`, `handshake OK`, and `7 tools discovered`
(the seven verbs), exit 0. A broken registration exits 1 with a failing
check and a hint. Then one real round-trip:
```bash
grok -p "use the gbrain recall tool to answer: what did I import most recently?"
```
`grok -p` (single-turn headless) prints the final answer on stdout.
## Headless auth + model pin
For cron jobs, CI, or any non-TTY run:
- **Auth:** export `XAI_API_KEY` (from console.x.ai). Keyless headless runs
exit 1 with `Not signed in`; `grok login --device-code` is the
interactive-terminal alternative, `grok login` the browser one.
- **Model pin:** pass `-m <model>` per call, or set it in config:
```toml
[models]
default = "grok-4.5"
```
- **Updates:** Grok self-updates by default. For pinned/reproducible
environments, seed:
```toml
[cli]
auto_update = false
```
## Pair with cron
Grok Build has no built-in cron; schedule headless one-shots with your
system scheduler:
```bash
# crontab: brain maintenance every 4 hours
0 */4 * * * XAI_API_KEY=... grok -p "Run gbrain sync and report anything unusual" --output-format plain
```
See [docs/guides/cron-schedule.md](../guides/cron-schedule.md) for the full
brain maintenance protocol (sync, embed, dream cycle).
## Troubleshooting
- **Wrong `grok` on PATH** — the community `superagent-ai/grok-cli` also
installs a `grok` binary. The official CLI answers `grok --version` with
`grok X.Y.Z (buildhash)`; anything else is the other tool. Install the
official one via `npm install -g @xai-official/grok` or
`curl -fsSL https://x.ai/cli/install.sh | bash`.
- **grok ≠ groq ≠ ngrok** — Grok Build (xAI, `XAI_API_KEY`) is not Groq
(the inference provider, `GROQ_API_KEY`) and not ngrok (tunnels). A
mis-set key produces auth errors against the wrong service.
- **`Not signed in` (exit 1)** — no auth in a headless run. Export
`XAI_API_KEY` or run `grok login --device-code`.
- **Doctor says `folder untrusted`** — the registration came from a vendor
config (`.mcp.json` / `~/.claude.json`) in a folder Grok hasn't been told
to trust. Trust the folder in an interactive session, or register
natively with `grok mcp add`.
- **Doctor times out on `server started`** — raise `startup_timeout_sec`
(or `GROK_MCP_STARTUP_TIMEOUT_SECS=90`) if gbrain runs via `bun run`.
- **Skills note:** `gbrain skillpack scaffold` writes `skills/<name>/SKILL.md`
into your workspace, which Grok does **not** auto-discover as Grok skills
(it reads `.grok/skills`, `~/.grok/skills`, `~/.agents/skills`, plugins).
gbrain's skills still work as reference documents the agent reads;
`grok inspect` shows what Grok actually discovered.
- **`grok inspect`** — the config-discovery audit: version, cwd trust,
instructions, permissions, skills, agents, MCP sources.
---
Verified against **Grok Build v1.0.4** (early beta — expect churn; the pin
is enforced in CI). Dev-facing observed-behavior notes (exact flag
semantics, exit-code caveats, config schema, CI pin values) live in
[GROK-CLI-PIN.md](GROK-CLI-PIN.md).
+5 -2
View File
@@ -12,7 +12,10 @@ file, the workflow pins, and the affected assertions together.
version stamp; CI installs the RELEASE TAG `v2026.8.3` = commit `3c27eb62` — the two
differ by post-release main commits, same declared version. If a CI door run ever
diverges from these notes, re-observe against the tag checkout.)
- Installer sha256: `c118ff31618dc70339049ce71061b8f1351a1c70d9c2a236ed50d8a2550c550d`
- Installer sha256: `868ed3a91e0fabbff6d7418b3ede82bf4833652ec4e77196a42852fb35a9e5b9`
(refreshed 2026-08-15: upstream installer drifted past the prior pin —
reviewed; the `--commit` payload-pin path the door depends on is intact,
and the payload pins (tag+commit) are unchanged)
(download https://hermes-agent.nousresearch.com/install.sh to a file first; verify; then run)
- Installer flags used: `--skip-setup --non-interactive`; binary lands at `~/.local/bin/hermes`
- Python 3.11.15 via uv
@@ -93,7 +96,7 @@ non-interactive. `hermes cron tick` = run due jobs once and exit. `hermes cron l
`git -C ~/.hermes/hermes-agent rev-parse HEAD` and loud-fails on any mismatch, so an
installer that silently ignores unknown flags (or a moved checkout layout) can never
run unpinned upstream code on a runner that later holds secrets.
- `HERMES_INSTALL_SHA256: "c118ff31618dc70339049ce71061b8f1351a1c70d9c2a236ed50d8a2550c550d"`
- `HERMES_INSTALL_SHA256: "868ed3a91e0fabbff6d7418b3ede82bf4833652ec4e77196a42852fb35a9e5b9"`
- Door test asserts `hermes --version` output contains `v$HERMES_VERSION` when the env var is set.
- `hermes --version` output shape: `Hermes Agent v0.20.0 (2026.8.3)` + install dir + python lines.
+252
View File
@@ -0,0 +1,252 @@
# opencode CLI pin — observed behavior notes (v1.18.18)
Dev-facing companion to [OPENCODE.md](OPENCODE.md): every fact below was OBSERVED
against a real hermetic install (2026-08-15, macOS arm64), not researched from docs.
The claw-test OpencodeRunner, the install door e2e, and the heavy-tests
opencode-door CI job assert exactly these shapes — when opencode releases change
them, update this file, the workflow pins, and the affected assertions together
(`scripts/check-opencode-pin.sh` in `bun run verify` enforces the workflow-side
match). Where an observation CONTRADICTS opencode's docs, the observation wins and
the contradiction is called out inline.
Naming note: **opencode** (SST, opencode.ai, npm `opencode-ai`) is not **OpenClaw**
(the agent platform gbrain ships a runner for) and not the original `opencode` CLI
that was renamed Crush — see Troubleshooting in OPENCODE.md for the binary-name
collision.
<!-- opencode-pin: distribution_kind=npm -->
<!-- opencode-pin: npm_package=opencode-ai -->
<!-- opencode-pin: npm_version=1.18.18 -->
<!-- opencode-pin: npm_integrity=sha512-J+5HFq8tf+wPBBpBpMPSNjSytF2/EkNWYfFZh4si1d9auFbQriqDyqZv+vFUsLWERfdMU32Eajwuiq3rKBvZLQ== -->
<!-- opencode-pin: npm_linux_x64_integrity=sha512-WmeUnhljYJ252wywKTiW4bNDzsas2njpjPUEh0jM6HKNI4vFxJtREtzaWViY4AKEAcOkLWT8Ll17ixvcHz3AnA== -->
<!-- opencode-pin: npm_linux_arm64_integrity=sha512-e8D3g0qJEIzawEg2+ygW3vkZjAYL2ssyAx4GbihjwXwZFvlZZy5zRWWzdz5KLBoHSTl0FB73vNtnNeXONyHpVQ== -->
<!-- opencode-pin: opencode_version=1.18.18 -->
<!-- opencode-pin: observed_date=2026-08-15 -->
## Pin
- **opencode v1.18.18**, `opencode --version` output shape: bare `1.18.18`
version only, NO binary-name prefix, NO build hash (unlike grok's
`grok 1.0.4 (hash)`). The door's T1 shape assert is `/^\d+\.\d+\.\d+$/` on the
trimmed output; SST identity is discriminated by the `mcp`+`debug` subcommands
existing (`opencode debug paths` exits 0 and prints the path table below —
the renamed-to-Crush ancestor and other claimants have neither).
- **Provisioning (CI + local): pinned npm, pack-verify-install**
`opencode-ai@1.18.18`, registry integrity `sha512-J+5HFq…`. The CI job
`npm pack`s the wrapper AND the runner's platform payload first (pack
reports the integrity of the bytes it actually downloaded — closing the
view-then-install TOCTOU), asserts both against the stamps above, then
installs FROM the verified local wrapper tarball; the install-time platform
sub-package fetch is validated by npm against the same packument integrity
the pack step just byte-confirmed. The wrapper fans out to per-platform
payloads (`opencode-{darwin,linux,windows}-{arm64,x64}[-baseline|-musl]`) as
optionalDependencies at the same version; the LINUX payload integrities are
pinned separately because the wrapper's integrity covers only the wrapper
tarball. Darwin arm64 payload observed at
`sha512-VkG+bz8u8Xqg9NzPK+2/71nEd4DKKlo2NLZurQ1eLAzDnmb1CMYZif/o6Shl8YFuTuYU/30k6yufl4Zr0Ij64g==`
(informational — the CI runners are linux). Same npm version-immutability
assumption as the grok pin, stated explicitly.
- A curl installer (`https://opencode.ai/install`) exists but is NOT the pinned
lane; npm is.
## Pin-refresh cadence (this CLI ships near-continuously)
opencode releases far faster than grok (patch releases near-daily). The pinned
lane is the deterministic gate; the **canary leg** in `opencode-door` (schedule-
scoped, `continue-on-error`, installs `opencode-ai@latest`) exists to surface
drift BEFORE it strands the pin. Policy: when the canary leg reds or the pin is
>6 weeks old, run the re-observation checklist (bottom) against latest, bump the
stamps + workflow env pins together, and note behavior deltas in this file.
Do not chase every patch release; refresh on canary signal or the 6-week clock.
## Path seams — XDG honored; OPENCODE_CONFIG* env vars are INERT (verified)
`opencode debug paths` is the authoritative dump. Observed under
`HOME=<tmp> XDG_CONFIG_HOME=<tmp>/.config XDG_DATA_HOME=<tmp>/.local/share`:
```
config <XDG_CONFIG_HOME>/opencode (opencode.json + opencode.jsonc)
data <XDG_DATA_HOME>/opencode (auth.json, opencode.db*, log/, repos/)
state <tmp>/.local/state/opencode (locks/)
cache <tmp>/.cache/opencode (bin/)
tmp /tmp/opencode
```
- **HOME + XDG_CONFIG_HOME/XDG_DATA_HOME redirection works fully on macOS**
(nothing was written outside the hermetic home across the whole observation
run). The door uses HOME + both XDG vars, belt-and-suspenders.
- **DOCS-CONTRADICTION: `OPENCODE_CONFIG`, `OPENCODE_CONFIG_DIR`, and
`OPENCODE_CONFIG_CONTENT` had NO observable effect on config resolution in
1.18.18** — probes registered via each were absent from `mcp list`, while the
XDG-resolved global config was still read. gbrain's path helpers therefore
resolve via XDG only and deliberately do NOT honor `OPENCODE_CONFIG*`;
re-observe on version bump (if a future release activates them, the helpers
and this section change together). Hermetic child envs still DELETE all three
(defense against a future release activating them).
- Volatile paths (tripwire exclusions): `opencode.db`, `opencode.db-shm`,
`opencode.db-wal`, `log/`, `repos/` under data; `locks/` under state; `bin/`
under cache. The tripwire hashes only `opencode.json(c)` + `auth.json`.
- Vendor quirk: opencode writes a `.gitignore` (node_modules, package.json, …)
into the CONFIG dir on first touch.
## Config format — JSONC everywhere, both filenames merge (verified)
- `~/.config/opencode/opencode.jsonc` AND `~/.config/opencode/opencode.json`
are BOTH read when both exist (servers from each appeared simultaneously in
`mcp list`) — merge, not first-wins. opencode's own `mcp add` writes the
`.jsonc` name.
- **Comments parse in `.json`-named files too** (a `// comment` inside project
`opencode.json` did not break resolution). JSONC is the effective grammar for
every config file regardless of extension → gbrain's writer treats all
opencode configs as JSONC (jsonc-parser surgical edits; comments survive).
- Project config: `opencode.json` in the project root is read (lookup traverses
up); a project-scope entry appears alongside global entries.
- Unknown keys inside an `mcp.<name>` entry are TOLERATED in 1.18.18 (an
`_gbrain` probe key neither errored nor hid the server). gbrain still does
NOT write marker keys — ownership is judged by structural fingerprint — so a
future strict-schema flip cannot brick a user's opencode.
- `opencode debug config` prints the resolved merge (rendering has a doubled-
line quirk; treat it as a debug view, not a parse surface).
## `opencode mcp add` — observed facts
- Shape: `opencode mcp add <name> [--env KEY=VALUE]... -- <command> [args...]`
(local) or `opencode mcp add <name> --url <URL> [--header KEY=VALUE]...`
(remote). The `-- command` form is real but UNDOCUMENTED in `--help` (the
help lists only `--url/--env/--header`; the error copy for a bare add says
`Provide either --url <url> or a command after --`).
- **Always writes the GLOBAL `opencode.jsonc`** — even when a project
`opencode.json` with an `mcp` table exists in the cwd. There is NO scope
flag. Project-scope registration requires writing the file directly (gbrain's
writer does).
- **Add is lazy**: exit 0, no spawn, no prompt — for unreachable URLs and
nonexistent commands alike. Never treat add's exit code as a handshake.
- **Rewrites preserve comments and foreign keys** (a seeded `// comment` and a
`theme` key survived a subsequent add) — opencode uses a JSONC-preserving
editor internally; gbrain's writer matches that bar.
- `--header` values are stored verbatim, including `{env:VAR}` interpolation
syntax (`Authorization=Bearer {env:GBRAIN_REMOTE_TOKEN}` round-trips).
## Saved config schema (verbatim, from real adds)
```jsonc
{
"$schema": "https://opencode.ai/config.json",
"mcp": {
"gbrain": {
"type": "local",
"command": ["gbrain", "serve", "--surface", "verbs"],
"environment": { "GBRAIN_SOURCE": "workspace", "GBRAIN_HOME": "/tmp/<brain-home>" }
},
"gbrain-remote": {
"type": "remote",
"url": "https://brain.example/mcp",
"headers": { "Authorization": "Bearer {env:GBRAIN_REMOTE_TOKEN}" }
}
}
}
```
`enabled` is optional (absent = enabled). `oauth` was not written by the CLI and
is omitted by gbrain's writer (no OAuth interference with bearer headers was
observed). Local commands: an absolute `command[0]` works; PATH-resolved bare
`gbrain` resolves via the SPAWNING process's PATH (the door verifies the staged
bin-dir prepend).
## Probes — `mcp list` is the honest discriminator; `mcp debug` is NOT
- **`opencode mcp list` SPAWNS every configured local server and connects every
remote one**, then prints per-server status: `✓ <name> connected` or
`✗ <name> failed` with a reason line (`Executable not found in $PATH:
"gbrain"`, `SSE error: …`). THE door's keyless handshake proof. Caveats:
**exit code is 0 even when servers fail** (parse the text, assert
`✓ gbrain connected`), output is clack-style UI with ANSI codes, and there is
no `--json`.
- **`mcp list` is also a code-execution surface**: it spawned a PROJECT-defined
`type:local` command from a fresh checkout with NO prompt and NO trust gate
(verified with a touch-file probe). Two consequences: (1) gbrain's
bootstrap default scope for opencode is USER-GLOBAL — a committed project
entry would auto-spawn on every collaborator's machine; (2) any gbrain-run
probe uses `--pure` (kills external plugin autoload) + `OPENCODE_DISABLE_AUTOUPDATE=1`.
- `opencode mcp debug <name>` is OAUTH debugging only — on a local server it
prints `MCP server <name> is not a remote server` and exits 0. Not a
discriminator.
- No tool-count line exists in `mcp list` (grok's `7 tools discovered` has no
analog); tool discovery is proven by the SMOKE turn's `tool_use` events
instead.
## One-shot (`opencode run`) — KEYLESS WORKS (anonymous free tier)
- `opencode run "<msg>"` prints the ANSWER TEXT ALONE on stdout; the session
banner (`> build · <model>`) and UI go to stderr. Exit 0 on success; exit 1
with a structured JSON error (`"ref": "err_…"`) on failure (e.g. bogus
model).
- **Keyless runs WORK**: with zero credentials and no auth.json, `run` answers
via opencode's anonymous free tier (default model observed:
`opencode/big-pickle`; `opencode models` lists 8 keyless `opencode/*` models,
most `-free` suffixed; `opencode stats` reports $0.00). There is no
`Not signed in` wall in headless run mode.
- **MCP tools fire in keyless run mode WITHOUT `--auto`** (verified: the free
model called `gbrain_recall` and returned a seeded per-run nonce with
`--auto` absent). `--auto` exists (`auto-approve permissions that are not
explicitly denied (dangerous!)`) but the door does not need or use it.
- MCP tool naming: `<server>_<tool>` (observed `gbrain_recall`).
- `--format json` emits NDJSON events, every event
`{type, timestamp, sessionID, part}`; types observed: `step_start`,
`tool_use`, `text`, `step_finish`. Tool events carry
`part: {type:"tool", tool:"gbrain_recall", callID, state:{status:"completed",
input:{…}, output:"<stringified JSON>"}}` — `parseOpencodeJsonl` pins this.
- Model flag: `-m/--model <provider/model>` (`opencode/big-pickle` confirmed;
paid ids follow models.dev convention — see Pending auth).
- Keyless SMOKE end-to-end (proven 2026-08-15): pinned opencode + free model +
real `gbrain serve --surface verbs` (7 verbs banner) recalled a per-run nonce
through MCP with zero credentials, keyless PGLite brain.
## Environment — detectHarness + child-env facts (verified)
- Inside `run`'s bash tool, opencode sets **`OPENCODE=1`** and `OPENCODE_PID`
in child processes → `gbrain bootstrap`'s `detectHarness()` probes
`OPENCODE`.
- Auto-update kill: `OPENCODE_DISABLE_AUTOUPDATE=1` env + `"autoupdate": false`
config — the door seeds BOTH; version stayed pinned across every observed
run. `opencode upgrade` is the manual updater.
- Rules files: project `AGENTS.md` is loaded; a sibling `CLAUDE.md` is NOT
double-loaded (nonce test: only the AGENTS.md nonce surfaced) — AGENTS.md
wins per level, exactly as documented. gbrain's rendered pull-protocol
contract works unchanged.
- `.well-known/opencode` remote config: never observed to fire in any CLI run
(docs list it atop the lookup order). No kill needed today; re-observe on
version bump.
## Auth (only needed for PAID providers)
- Anonymous free tier needs nothing on disk; `auth.json` is only created by
`opencode auth login` at `<XDG_DATA_HOME>/opencode/auth.json`
(`opencode providers`, alias `auth`, prints the path).
- The optional paid door leg gates on `ANTHROPIC_API_KEY` (env-only) and
self-validates the model id against the authed `opencode models` output
before spending.
## When the door goes red (triage)
| Failure class | Signature | Remediation |
|---|---|---|
| npm pin drift | install step: version/integrity mismatch | Re-pin deliberately: bump `npm_version`+`npm_integrity` (+ platform stamps), run the re-observation checklist, update workflow env pins (check-opencode-pin.sh enforces the pair) |
| canary leg red, pinned leg green | latest-version leg fails install/asserts | Upstream changed shape — schedule a pin refresh; pinned lane still gates |
| version drift mid-run | `opencode --version` re-check ≠ pinned | Auto-update engaged — verify BOTH kills (env + config seed); re-pin if deliberate |
| `✗ gbrain failed` in `mcp list` | `Executable not found in $PATH` / spawn error | Staged bin dir missing from PATH, or abs path wrong — registration bug, not opencode drift |
| free-tier drift | keyless SMOKE stops answering / new auth wall | Re-observe keyless posture; if the free tier is gated, flip the SMOKE to the ANTHROPIC leg and re-pin this section |
| paid leg: model id unknown | models-gate assert fails before any spend | Update the pinned anthropic model id from the authed `opencode models` output |
| tripwire fired | manifest mismatch on `opencode.json(c)`/`auth.json` only | True isolation breach — stop and inspect; volatile-path drift alone must NOT fire |
| real door regression | handshake or nonce assert fails, pins intact | Bisect against the pinned version; file upstream if opencode-side |
Re-observation checklist on a version bump: npm pin captures (§Pin), help-surface
diff (`--help`, `run --help`, `mcp --help`, `mcp add --help`), the
add → saved-config → `mcp list` sequence (§add/§Probes), the keyless `run`
posture (§One-shot — free tier presence, stdout purity, MCP-without---auto),
`debug paths`, and the `OPENCODE_CONFIG*` inertness probe (§Path seams). The
spawn-gate probe (§Probes) re-runs whenever release notes mention MCP trust or
permissions.
## Pending auth (requires ANTHROPIC_API_KEY; the core door does NOT)
Authed `opencode models` list + exact `anthropic/<model>` id confirmation,
one paid one-shot smoke + per-turn cost note, `auth.json` verbatim shape after
`opencode auth login` (feeds evidence exclusions + TTY secretPaths), and
whether the authed TUI first-run differs from the keyless one pinned in the
dx scenario. The opencode-door paid leg self-validates the model id before
spending, so these pins harden the door but do not block it.
## Supported-version policy
gbrain's opencode integration is verified against **opencode v1.18.18** (this
pin). The canary CI leg tracks latest (continue-on-error); the pinned lane is
the deterministic gate. Keyless free-tier behavior is a LOAD-BEARING
observation (the SMOKE rides it) — treat free-tier changes as pin-refresh
triggers, not flakes.
+175
View File
@@ -0,0 +1,175 @@
# Connect GBrain to opencode
> This page is the MCP-registration reference for **opencode** — the SST
> terminal coding agent (opencode.ai, npm `opencode-ai`; not OpenClaw, and not
> the original `opencode` CLI that was renamed Crush — see Troubleshooting).
> For the full brain install — CLI, engine, skills, dream cycle — follow
> [INSTALL_FOR_AGENTS.md](../../INSTALL_FOR_AGENTS.md) first; this page wires
> the finished brain into opencode over stdio MCP. opencode is a
> **bootstrap-supported harness**: `gbrain bootstrap hooks --harness opencode`
> registers the brain for you (and `gbrain connect --agent opencode` handles
> remote brains — see below) — the commands on this page are the standalone
> manual recipe. Bootstrap's own registration additionally pins the workspace
> source (`GBRAIN_SOURCE`) and the full op surface, so the two are not
> byte-identical.
opencode spawns `gbrain serve` as a local stdio subprocess. No server, no
tunnel, no token needed. Works with both PGLite and Supabase engines — and
because opencode natively reads `AGENTS.md`, a gbrain workspace's rendered
brain contract loads with zero extra configuration.
## Register (recommended)
```bash
opencode mcp add gbrain --env GBRAIN_HOME=$HOME -- gbrain serve --surface verbs
```
`--surface verbs` exposes the seven-verb memory protocol (`recall`,
`remember`, `entity`, `synthesize`, `forget`, `context_pack`, `delta`
[MEMORY_VERBS v1](../protocol/MEMORY_VERBS_v1.md)) instead of the full
100+-op catalog — the recommended starting surface for coding agents.
Three facts about `opencode mcp add`, all observed:
- **The local-command form is `-- <command> [args...]` after the flags**
it's real but missing from `--help` (which shows only `--url/--env/--header`).
`--env` is repeatable, one `KEY=VALUE` per flag.
- **Registration is lazy.** The add writes config and exits 0 without
connecting — even for a nonexistent command. Verify with `opencode mcp list`
(below), never with the add's exit code.
- **It always writes the USER-GLOBAL config**
(`~/.config/opencode/opencode.jsonc`) — there is no scope flag. For a
project-scoped entry, write the project `opencode.json` directly (next
section) — but read the sharing warning first.
## Direct config (equally supported)
Global (`~/.config/opencode/opencode.jsonc`) or project (`opencode.json` in
the repo root — opencode's lookup traverses up to the git root):
```jsonc
{
"$schema": "https://opencode.ai/config.json",
"mcp": {
"gbrain": {
"type": "local",
"command": ["gbrain", "serve", "--surface", "verbs"],
"environment": { "GBRAIN_HOME": "/home/alice-example" },
"enabled": true
}
}
}
```
Comments are fine — opencode parses JSONC in both `.json` and `.jsonc` files,
and both filenames are read (merged) when both exist. To remove gbrain,
delete the entry, or set `"enabled": false` to disable without losing it.
**Sharing warning for project config:** opencode spawns project-defined local
MCP servers with **no trust prompt** — a committed `opencode.json` carrying a
gbrain entry executes on every collaborator's machine. Teammates without
gbrain get a failing spawn each session; teammates WITH gbrain attach their
own `host` brain to your repo's context. Prefer the user-global config (the
gbrain bootstrap default); if you do commit a project entry, use the
PATH-resolved `"gbrain"` command form (never an absolute path) and tell
collaborators `"enabled": false` is the opt-out.
## Verify
```bash
opencode mcp list # the real probe: SPAWNS the server
```
`opencode mcp list` performs the actual spawn + handshake for every
configured server — expect `✓ gbrain connected`. A broken registration shows
`✗ gbrain failed` with the reason (e.g. `Executable not found in $PATH`).
Because it spawns everything — including any project `opencode.json` entries
in your cwd, with no trust prompt — run it from a directory you trust
(gbrain's own bootstrap verification probe runs from an empty temp directory
for exactly this reason, and skips the live probe entirely for project-scoped
registrations).
Two caveats: the exit code is 0 even when servers fail (read the output, not
`$?`), and `opencode mcp debug` is OAuth-only diagnostics — it is NOT a
handshake probe for local servers. Then one real round-trip:
```bash
opencode run "use the gbrain recall tool to answer: what did I import most recently?"
```
`opencode run` (headless one-shot) prints the final answer alone on stdout
(UI goes to stderr). MCP tools work in run mode without any permission flags.
## Remote brains (`gbrain connect`)
For a brain served over HTTP on another machine:
```bash
gbrain connect https://your-host/mcp --token gbrain_xxx --agent opencode [--install]
```
Without `--install` it prints the config block to add; with `--install` it
writes the entry directly into the user-global config (no opencode binary
required — the JSONC write IS the registration) and smoke-tests the token.
Either way the config stores only the `{env:GBRAIN_REMOTE_TOKEN}`
interpolation — opencode resolves the env var at read time, so the token
never lands in the file. Export `GBRAIN_REMOTE_TOKEN` in your shell profile.
`--force` replaces a gbrain-managed entry whose endpoint moved (a rotated
serve); an entry gbrain didn't write is never replaced — pick another
`--name`. (Framework-spawned opencode inherits no shell profile;
`gbrain bootstrap harness --harness opencode` covers that case with an
inline-bearer entry written 0600.)
## Auth + model pin
- **Keyless works.** opencode ships an anonymous free tier (default model
`opencode/big-pickle` at observation time) — headless runs and MCP tool
calls work with zero credentials. For paid providers, export the provider
key (e.g. `ANTHROPIC_API_KEY`) or run `opencode auth login` (credentials
land in `~/.local/share/opencode/auth.json`).
- **Model pin:** pass `-m <provider/model>` per call, or set `"model"` in the
config. `opencode models` lists what your credentials can reach.
- **Updates:** opencode self-updates by default. For pinned/reproducible
environments, set BOTH `"autoupdate": false` in config AND
`OPENCODE_DISABLE_AUTOUPDATE=1` in the environment.
## Pair with cron
opencode has no built-in cron; schedule headless one-shots with your system
scheduler:
```bash
# crontab: brain maintenance every 4 hours
0 */4 * * * opencode run "Run gbrain sync and report anything unusual"
```
See [docs/guides/cron-schedule.md](../guides/cron-schedule.md) for the full
brain maintenance protocol (sync, embed, dream cycle).
## Troubleshooting
- **Wrong `opencode` on PATH** — the name has prior claimants (the original
`opencode` project was renamed Crush). The SST CLI answers
`opencode --version` with a bare semver (`1.18.18`) and has `opencode mcp`
+ `opencode debug paths` subcommands. Install it via
`npm install -g opencode-ai` or `curl -fsSL https://opencode.ai/install | bash`.
- **opencode ≠ OpenClaw** — opencode (opencode.ai / SST) is the terminal
agent this page covers; OpenClaw is the agent platform with its own gbrain
runner and docs ([OPENCLAW.md](OPENCLAW.md)).
- **`✗ gbrain failed — Executable not found in $PATH`** — the registered
command was the bare `"gbrain"` name and opencode's PATH doesn't carry it.
Use the absolute binary path in the user-global config, or fix PATH.
- **Registered but nothing changed mid-session** — opencode reads config at
session start; restart opencode (or start a new session) after registering.
- **`OPENCODE_CONFIG` seems ignored** — observed inert in v1.18.18: only
`HOME`/`XDG_CONFIG_HOME` move the config location. Don't rely on it.
- **Which config won?**`opencode debug config` prints the resolved merge;
`opencode debug paths` prints every directory opencode uses.
- **Rules files** — opencode loads the project `AGENTS.md` (a sibling
`CLAUDE.md` is NOT double-loaded; AGENTS.md wins). gbrain's rendered
workspace contract rides this natively.
---
Verified against **opencode v1.18.18** (fast-moving project — the pin is
enforced in CI, with a latest-version canary leg watching for drift).
Dev-facing observed-behavior notes (exact flag semantics, exit-code caveats,
config schema, CI pin values) live in [OPENCODE-CLI-PIN.md](OPENCODE-CLI-PIN.md).
+1 -1
View File
@@ -23,7 +23,7 @@ gbrain config set spend.posture gated # default — gates enforce
| Value | Effect |
|-------|--------|
| `gated` (default) | Every cost gate enforces its limit as documented below. |
| `tokenmax` | Every embedding-spend gate in the table below prints its estimate and **proceeds** — informational only. Spend is still recorded to the ledger; posture removes the *ceiling*, not the *accounting*. (Commands with their own LLM cost caps outside this doc's embedding scope — e.g. `extract-conversation-facts --max-cost-usd` — don't resolve posture; their per-call flags govern.) |
| `tokenmax` | Every embedding-spend gate in the table below prints its estimate and **proceeds** — informational only. Spend is still recorded to the ledger; posture removes the *ceiling*, not the *accounting*. (Commands with their own LLM cost caps outside this doc's embedding scope — e.g. `extract-conversation-facts --max-cost-usd`, `dream retriage --max-usd` (an estimate-based soft stop) — don't resolve posture; their per-call flags govern.) |
`spend.posture` is deliberately separate from `search.mode=tokenmax` (which governs
retrieval payload size, not embedding spend). When a gate fires and
+4
View File
@@ -28,6 +28,7 @@ Any of these commands stream events when `--progress-json` is set:
- `gbrain eval`
- `gbrain eval brainbench`
- `gbrain apply-migrations` (the orchestrator + every child command)
- `gbrain transcripts ingest` (per-file ticks + a per-session heartbeat over the import set)
Non-bulk commands (`stats`, `graph-query`, `get`, `put`, etc.) don't emit
events — they return in under a second.
@@ -158,6 +159,9 @@ Stable phase names shipped in v0.15.2:
fixture count and a percentage would lie
- `export.pages`
- `files.sync`
- `transcripts.ingest` (one tick per session-log file; sessions inside a
multi-session file — the hermes store, consumer exports — don't get their
own ticks, so total = file count; each session emits a heartbeat instead)
Sub-phases exposed via `child()`:
+10
View File
@@ -64,6 +64,16 @@ If `claude` is not found: install Claude Code first, or use a block below.
codex mcp add gbrain -- gbrain serve --surface verbs
```
**Grok Build** (verify with `grok mcp doctor gbrain` — the add is lazy)
```bash
grok mcp add gbrain -e "GBRAIN_HOME=$HOME" -- gbrain serve --surface verbs
```
**opencode** (verify with `opencode mcp list` — the add is lazy, and list SPAWNS the server)
```bash
opencode mcp add gbrain --env GBRAIN_HOME=$HOME -- gbrain serve --surface verbs
```
**OpenClaw / any stdio MCP host** — register the server command
`gbrain serve --surface verbs`. Remote brains: `gbrain serve --http` on the
host, then `gbrain connect https://host/mcp --token gbrain_xxx --install` on
+125 -15
View File
@@ -225,7 +225,11 @@ Per-file detail is in `docs/architecture/KEY_FILES.md`.
- **Engine-live paths avoid runtime dynamic `import()` for helper dependencies.** In
`src/core/pglite-engine.ts`, `src/core/postgres-engine.ts`, and
`src/core/migrate.ts`, dependencies previously reached through runtime dynamic
imports use static top-level imports. The only current dynamic-`import()` exceptions
imports use static top-level imports. Besides the snapshot loader's lazy
`require()` cluster in `pglite-engine.ts:tryLoadSnapshot` (fs/crypto/
migrate/pglite-schema + one gateway shape lookup — lazy so production
builds without the test-fixture path don't eager-load; the guard now
matches `require()` calls too), the only dynamic-`import()` exceptions
are the four `ai/gateway.ts` lookups in both engines'
`initSchema()` and `_upsertChunksOnce()` methods; each remains lazy inside a
local `try/catch` because the gateway has a large provider/config closure and,
@@ -1243,6 +1247,34 @@ Keep `--args` last (everything after it becomes server argv) and verify with
`hermes mcp test gbrain` — the add exits 0 even on failure. Full reference:
[docs/mcp/HERMES.md](docs/mcp/HERMES.md).
**If you are Grok Build** (xAI's `grok` CLI): register gbrain as your MCP server:
```bash
grok mcp add gbrain -e "GBRAIN_HOME=$HOME" -- gbrain serve --surface verbs
```
The add is lazy (exit 0 without connecting) — verify with
`grok mcp doctor gbrain`, which spawns the server and must report
`7 tools discovered`. This is the brain-only install; the `gbrain bootstrap`
personal-agent path does not support Grok yet (Claude Code, Codex, and opencode only).
Verified against Grok Build v1.0.4. Full reference:
[docs/mcp/GROK.md](docs/mcp/GROK.md).
**If you are opencode** (the SST terminal agent, opencode.ai — not OpenClaw):
you are a bootstrap-supported harness — for the full persistent-personal-agent
install, follow `BOOTSTRAP_FOR_AGENTS.md` instead of this page. For the
brain-only MCP registration:
```bash
opencode mcp add gbrain --env GBRAIN_HOME=$HOME -- gbrain serve --surface verbs
```
The add is lazy (exit 0 without connecting) — verify with `opencode mcp list`,
which spawns the server and must show `✓ gbrain connected` (the exit code is 0
even on failure; read the output). Restart opencode afterwards — it reads
config at session start. Verified against opencode v1.18.18. Full reference:
[docs/mcp/OPENCODE.md](docs/mcp/OPENCODE.md).
Whether you scaffolded or not, read `skills/RESOLVER.md` (in your workspace, or the
bundled copy at `~/gbrain/skills/RESOLVER.md` when running from the cloned repo). It's
the skill dispatcher — tells you which skill to read for any task. Save this to your
@@ -1521,11 +1553,12 @@ wins; fix the row.
|---------|-------|
| "Set up GBrain", first boot | `skills/setup/SKILL.md` |
| "Now what?", "fill my brain", "cold start", "bootstrap my data", "import my data", "what should I import first" | `skills/cold-start/SKILL.md` |
| "Install gbrain into this agent/harness", "agent workspace bootstrap", "gbrain bootstrap", "wire gbrain hooks", "set up the maintenance sweep" | Run `gbrain bootstrap` (paste-in harness install: hooks + sweep + config). See `docs/guides/bootstrap.md` |
| "agent workspace bootstrap", "install gbrain into this agent workspace", "gbrain bootstrap", "paste-in install", "set up the maintenance sweep" | Run `gbrain bootstrap` (paste-in workspace install: interview + identity files + hooks + sweep). See `docs/guides/bootstrap.md` |
| "wire this box's coding agents to the brain", "framework-spawned sessions need brain access", "wire gbrain hooks without a workspace", "hook Claude Code/Codex to the running serve" | Run `gbrain bootstrap harness --yes` (machine-level wiring to a running `serve --http`: scoped token + user-scope MCP + headless pre-approval + hooks; no agent.json). See the "Local harness mode" section of `docs/guides/bootstrap.md` |
| "Migrate from Obsidian/Notion/Logseq" | `skills/migrate/SKILL.md` |
| Brain health check, maintenance run | `skills/maintain/SKILL.md` |
| "Extract links", "build link graph", "populate timeline" | `skills/maintain/SKILL.md` (extraction sections) |
| "Run dream", "process today's session", "synthesize my conversations", "consolidate yesterday's conversations", "what patterns did you see", "did the dream cycle run" | `skills/maintain/SKILL.md` (dream cycle section) |
| "Run dream", "process today's session", "synthesize my conversations", "consolidate yesterday's conversations", "what patterns did you see", "did the dream cycle run", "retriage the backlog", "re-score the triage" | `skills/maintain/SKILL.md` (dream cycle section) |
| "Brain health", "what features am I missing", "brain score" | Run `gbrain features --json` |
| "Set up autopilot", "run brain maintenance", "keep brain updated" | Run `gbrain autopilot --install --repo ~/brain` |
| "Upgrade gbrain", "update gbrain", "gbrain update available", `UPGRADE_AVAILABLE`, "is gbrain up to date" | `skills/gbrain-upgrade/SKILL.md` |
@@ -1765,6 +1798,8 @@ GBrain exposes nearly all of its 100+ operations as MCP tools (stdio and HTTP; a
- **[Codex](docs/mcp/CODEX.md)** — `gbrain connect https://your-host/mcp --token gbrain_xxx --agent codex` (or `--install`). Codex reads the bearer from `$GBRAIN_REMOTE_TOKEN` at runtime, so the token never lands in Codex config.
- **[Cursor / Windsurf / any stdio MCP client](docs/mcp/CLAUDE_CODE.md)** — same shape, add `{"command": "gbrain", "args": ["serve"]}` to your MCP config.
- **[Hermes](docs/mcp/HERMES.md)** — `printf 'Y\n' | hermes mcp add gbrain --env GBRAIN_HOME=$HOME --connect-timeout 60 --command $(which gbrain) --args serve`. Keep `--args` last, and verify with `hermes mcp test gbrain` (the add exits 0 even on failure).
- **[Grok Build](docs/mcp/GROK.md)** — `grok mcp add gbrain -e "GBRAIN_HOME=$HOME" -- gbrain serve --surface verbs`. The add is lazy (exit 0 without connecting) — verify with `grok mcp doctor gbrain`, which spawns the server and reports `7 tools discovered`. Verified against Grok Build v1.0.4.
- **[opencode](docs/mcp/OPENCODE.md)** (opencode.ai / SST — not OpenClaw) — `opencode mcp add gbrain --env GBRAIN_HOME=$HOME -- gbrain serve --surface verbs`, or let `gbrain bootstrap hooks --harness opencode` write the config for you (opencode is a bootstrap-supported harness — it reads AGENTS.md natively). The add is lazy — verify with `opencode mcp list`, which spawns the server (`✓ gbrain connected`). Remote: `gbrain connect https://your-host/mcp --token gbrain_xxx --agent opencode [--install]` — the config stores only the `{env:GBRAIN_REMOTE_TOKEN}` interpolation. Verified against opencode v1.18.18.
- **[OpenClaw](docs/mcp/OPENCLAW.md)** — the ClawHub bundle plugin registers gbrain automatically (`openclaw.plugin.json` ships in this repo), or add `{"command": "gbrain", "args": ["serve"]}` to `~/.openclaw/config.json`'s `mcpServers`.
- **[Claude Desktop (Cowork)](docs/mcp/CLAUDE_DESKTOP.md)** — Settings → Integrations → add the URL of your HTTP server. Remote only; the local `claude_desktop_config.json` does not work for remote servers.
- **[Claude Cowork (team plan)](docs/mcp/CLAUDE_COWORK.md)** — org Owner adds the connector under Organization Settings → Connectors.
@@ -1826,6 +1861,21 @@ curl -X POST https://your-brain/ingest \
For mobile capture, the inbox folder source picks up anything dropped into
`~/.gbrain/inbox/` from iOS Shortcuts / AirDrop / Drafts / Finder.
Your other agents' histories import in one command. `gbrain transcripts ingest`
parses agent session logs (Claude Code, Codex, OpenClaw, Hermes) and extracted
consumer chat exports (ChatGPT / Claude.ai `conversations.json`) into readable
conversation pages with provenance back to the exact session file. Secrets are
scrubbed from message bodies, titles, speakers, and session metadata before
anything is written, embedding is off by default for bulk backfills, and
re-runs are free — unchanged sessions skip on content hash:
```bash
gbrain transcripts ingest # discover importable session logs
gbrain transcripts ingest --all # import everything discovered
gbrain transcripts ingest ~/Downloads/conversations.json # consumer export (unzip first)
gbrain transcripts status # found vs imported, per harness
```
Third-party skillpacks can ship custom ingestion sources (Granola, Linear,
voice, OCR) against the versioned `IngestionSource` contract at
`gbrain/ingestion`. See [`docs/skillpack-anatomy.md`](docs/skillpack-anatomy.md).
@@ -1887,7 +1937,7 @@ The whole loop is described in [`docs/architecture/topologies.md`](docs/architec
**Self-wiring knowledge graph.** Every `put_page` extracts entity refs from markdown/wikilinks/typed-link syntax and writes edges with zero LLM calls. Typed edges (`attended`, `works_at`, `invested_in`, `founded`, `advises`, `mentions`, …). Multi-hop traversal via `gbrain graph-query`. The graph is what produces the +31.4 P@5 lift over vector-only RAG. **Obsidian-style vaults:** bare `[[note-name]]` wikilinks that point across folders — you wrote `[[struktura]]` but the page lives at `projects/struktura.md` — resolve by basename once you opt in with `gbrain config set link_resolution.global_basename true`. Off by default; `gbrain doctor` tells you how many edges you'd gain before you flip it. See [migrating an Obsidian vault](INSTALL_FOR_AGENTS.md#step-45-wire-the-knowledge-graph).
**Job queue (Minions).** BullMQ-shaped, Postgres-native job queue. Durable subagents (LLM tool loops that survive crashes via two-phase pending→done persistence), shell jobs with audit, child jobs with cascading timeouts, rate leases for outbound providers, attachments via S3/Supabase storage. Replaces "spawn subagent as fire-and-forget Promise" with something that recovers from anything.
**Job queue (Minions).** BullMQ-shaped, Postgres-native job queue. Durable subagents (LLM tool loops that survive crashes via two-phase pending→done persistence), shell jobs with audit, child jobs with cascading timeouts, rate leases for outbound providers, attachments via S3/Supabase storage. Opt-in per-job process isolation (`gbrain jobs work --job-isolation process`) runs each claimed job in its own SIGKILL-able child process, so a stuck handler dies for real and a crash takes one job instead of the whole worker; when the worker's DB health probe fails, it names the failing layer (`pool_starved` vs `server_unreachable`) instead of a blanket "DB unreachable". Sizing and rollout guidance in [`docs/guides/minions-deployment.md`](docs/guides/minions-deployment.md); probe-verdict triage in [`docs/guides/queue-operations-runbook.md`](docs/guides/queue-operations-runbook.md). Replaces "spawn subagent as fire-and-forget Promise" with something that recovers from anything.
**Non-English brains (FTS language config).** The Postgres full-text search tokenizer is configurable via `GBRAIN_FTS_LANGUAGE`. Defaults to `english`. Set it to any text-search configuration that exists in your Postgres instance:
@@ -2058,7 +2108,7 @@ the page PK, soft-delete-filtered, source-safe) and completes in seconds.
## Docs
- [`docs/INSTALL.md`](docs/INSTALL.md) — every install path, end to end
- [`docs/guides/bootstrap.md`](docs/guides/bootstrap.md) — the persistent-personal-agent bootstrap contract (interview, identity files, hooks, private repo, security posture, uninstall)
- [`docs/guides/bootstrap.md`](docs/guides/bootstrap.md) — the persistent-personal-agent bootstrap contract (interview, identity files, hooks, private repo, security posture, uninstall), plus local harness mode (`gbrain bootstrap harness`) for wiring framework-spawned Claude Code/Codex sessions to a running serve
- [`docs/what-schemas-unlock.md`](docs/what-schemas-unlock.md) — why schemas matter: 7 killer use cases, the structural argument for typed page kinds, the agent-co-curates pattern (v0.40.7.0)
- [`docs/schema-author-tutorial.md`](docs/schema-author-tutorial.md) — 5-minute walkthrough: fork the bundled pack, add a custom type, backfill existing pages, prove the wiring via `gbrain whoknows`
- [`docs/architecture/`](docs/architecture/) — system design, topologies, retrieval theory
@@ -3200,6 +3250,49 @@ it nightly and Phase 4 below (plus most of Phase 2's hygiene checks) is
covered. The pseudocode that follows is the harness-side variant for agents
that also do LLM-driven entity sweeps and memory consolidation on top.
### Synthesis cost control: the triage cascade
The synthesize phase is a two-stage cascade: a cheap scored triage
(utility-tier model, one call per new transcript) gates the expensive
per-transcript synthesis subagents. The dials:
- `dream.triage.threshold` (default 0.5) — the gate. Scores are cached, so
retuning it re-gates instantly with **zero** new LLM calls. Raise it if too
much routine content synthesizes; lower it if real signal is being skipped.
- `models.dream.triage` — the triage model (default: utility tier / Haiku).
- `dream.triage.max_chars` (default 24000, floor 1000) — per-transcript
sample window (head/middle/tail) sent to the judge. Not part of cache
validity — after changing it, `gbrain dream retriage --force` re-judges
under the new sampling.
- `dream.triage.max_tokens` (default 2048, floor 256) — judge output budget.
- `dream.triage.concurrency` (default 4, clamped 116) — concurrent judge
calls.
- `dream.synthesize.max_turns` (default 16) — synthesis turn budget. The
triage map hands the subagent pre-extracted segments, so the mid-tier
default model (`models.dream.synthesize`, tier `reasoning`) with a 16-turn
budget is the intended pairing — frontier-model overrides are unnecessary
and slow the queue. Completeness comes from triage coverage (every file
scored, minus files deferred under the `max_ms` budget below) plus
segment-guided prompts, not model size. If written-page counts
drop after upgrading, set it back to 30 and check
`details.synthesis.avg_turns` for cap pressure.
- `dream.triage.max_ms` (default 5 min) — per-cycle wall-clock budget for
judging NEW files; a big cold corpus triages across a few cycles (cached
files are free). Deferred files are labeled "not yet triaged", never
silently rejected.
- `dream.synthesize.max_submissions_per_source_per_day` (default 0 = off) —
opt-in backstop cap on synthesis jobs per source; 200/day is a sane value
for busy deployments.
Maintenance recipe — after changing the threshold, upgrading through a
`TRIAGE_VERSION` bump, or to drain a queued synthesis backlog:
```bash
gbrain dream retriage --dry-run # what would change (zero LLM calls)
gbrain dream retriage --reconcile-queue # re-score + cancel below-threshold queued jobs
gbrain dream retriage --audit-rejects 20 # synthesis-model second opinion on 20 rejects
```
### What It Does
```
@@ -3866,7 +3959,7 @@ The push channels share one zero-LLM core (`src/core/context/volunteer.ts`):
| `reflex` | automatic, inside the context engine | default-on for plugin hosts; nothing to call |
| `op` | `gbrain volunteer-context` / MCP `volunteer_context` | agents without the plugin; one call per turn |
| `watch` | `gbrain watch` | stream a transcript in, volunteered pages stream out |
| `claude-code` / `codex` | `gbrain hook user-prompt` (registered by `gbrain bootstrap`) | per-prompt injection inside a harness; see "Harness hooks" below |
| `claude-code` / `codex` / `opencode` | `gbrain hook user-prompt` (registered by `gbrain bootstrap`) | per-prompt injection inside a harness; see "Harness hooks" below |
## How it decides
@@ -3928,7 +4021,7 @@ this channel production-grade rather than spammy-and-invisible:
- **The feedback loop.** The serve logs each DELIVERED block's volunteered
pages and pointers to `context_volunteer_events` under the hook's channel
(`claude-code` by default; a codex hook registration passes
`--harness codex`). `gbrain volunteer-context --stats` then shows
`--harness codex` / `--harness opencode`). `gbrain volunteer-context --stats` then shows
per-harness precision, and `gbrain doctor`'s `volunteer_channels` check
shows which channels actually fire, with guidance for the two quiet cases:
"hook installed but never registered (restart the session)" and "registered
@@ -3981,9 +4074,11 @@ Source: https://raw.githubusercontent.com/garrytan/gbrain/master/docs/mcp/DEPLOY
> PKCE, refresh rotation, optional DCR), an embedded React admin dashboard at
> `/admin`, scoped operations, and a live SSE activity feed. Legacy bearer
> tokens still work — `verifyAccessToken` falls back to the `access_tokens`
> table and grandfathers tokens to `read+write+admin`. Both the legacy fallback
> and the OAuth tables work on PGLite and Postgres (both engine schemas carry
> `access_tokens`). See [SECURITY.md](../../SECURITY.md) for env vars and
> table; tokens with no `scopes` grant are grandfathered to `read+write+admin`,
> while tokens minted with `gbrain auth create --scopes …` (or by
> `gbrain bootstrap harness`) are honored at exactly their granted scopes.
> Both the legacy fallback and the OAuth tables work on PGLite and Postgres
> (both engine schemas carry `access_tokens`). See [SECURITY.md](../../SECURITY.md) for env vars and
> tunable defaults.
Access your brain from any device, any AI client. GBrain ships two transports:
@@ -4045,8 +4140,9 @@ This requires:
2. A public tunnel (ngrok, Tailscale, or cloud host)
3. A bearer token created via `gbrain auth create <name>`
Existing bearer tokens are grandfathered as `read+write+admin` scopes on the
OAuth-capable HTTP server, so no migration is required.
Existing bearer tokens (no `scopes` grant) are grandfathered as
`read+write+admin` on the OAuth-capable HTTP server, so no migration is
required; `gbrain auth create --scopes read,write` mints narrowed tokens.
## OAuth 2.1 Setup
@@ -4200,7 +4296,8 @@ Write ops can additionally be fenced per client with `--bound-slug-prefixes`
## Legacy Bearer Token Setup
Bearer tokens are the simple path when you don't need per-client scoping.
They grandfather to `read+write+admin` scopes on the HTTP server.
Without a `--scopes` grant they grandfather to `read+write+admin` on the
HTTP server; pass `--scopes read,write` at creation to narrow one.
### 1. Set up the tunnel
@@ -4226,8 +4323,11 @@ gbrain auth list
gbrain auth revoke "claude-desktop"
```
Tokens are per-client. Create one for each device/app. Revoke individually
if compromised. Tokens are stored SHA-256 hashed in your database.
Tokens are per-client. Create one for each device/app. Names are not
unique: `gbrain auth revoke "<name>"` revokes EVERY active token carrying
that name — use `gbrain auth list` (shows each token's id and scopes) and
`gbrain auth revoke --id <uuid>` to revoke exactly one. Tokens are stored
SHA-256 hashed in your database.
### 3. Connect your AI client
@@ -4408,6 +4508,16 @@ If `claude` is not found: install Claude Code first, or use a block below.
codex mcp add gbrain -- gbrain serve --surface verbs
```
**Grok Build** (verify with `grok mcp doctor gbrain` — the add is lazy)
```bash
grok mcp add gbrain -e "GBRAIN_HOME=$HOME" -- gbrain serve --surface verbs
```
**opencode** (verify with `opencode mcp list` — the add is lazy, and list SPAWNS the server)
```bash
opencode mcp add gbrain --env GBRAIN_HOME=$HOME -- gbrain serve --surface verbs
```
**OpenClaw / any stdio MCP host** — register the server command
`gbrain serve --surface verbs`. Remote brains: `gbrain serve --http` on the
host, then `gbrain connect https://host/mcp --token gbrain_xxx --install` on
+1 -1
View File
@@ -1,7 +1,7 @@
{
"id": "gbrain-context-engine",
"name": "gbrain",
"version": "0.45.13.0",
"version": "0.46.4.0",
"description": "Personal knowledge brain with Postgres + pgvector hybrid search",
"family": "bundle-plugin",
"configSchema": {
+8 -3
View File
@@ -50,7 +50,9 @@
"check:admin-scope-drift": "bash scripts/check-admin-scope-drift.sh",
"check:cli-exec": "bash scripts/check-cli-executable.sh",
"check:engine-dynamic-import": "bash scripts/check-engine-dynamic-import.sh",
"check:all": "bash scripts/check-privacy.sh && bash scripts/check-proposal-pii.sh && bash scripts/check-test-real-names.sh && bash scripts/check-jsonb-pattern.sh && bash scripts/check-source-id-projection.sh && bash scripts/check-source-config-leak.sh && bash scripts/check-progress-to-stdout.sh && bash scripts/check-no-tracked-symlinks.sh && bash scripts/check-no-legacy-getconnection.sh && bash scripts/check-test-isolation.sh && bash scripts/check-trailing-newline.sh && bash scripts/check-wasm-embedded.sh && bash scripts/check-pglite-embedded.sh && bash scripts/check-exports-count.sh && bash scripts/check-admin-build.sh && bash scripts/check-admin-scope-drift.sh && bash scripts/check-cli-executable.sh && bash scripts/check-skill-brain-first.sh && bash scripts/check-operations-filter-bypass.sh && bash scripts/check-gateway-routed-no-direct-anthropic.sh && bash scripts/check-worker-pool-atomicity.sh && bash scripts/check-key-files-current-state.sh && bash scripts/check-no-double-retry.sh && bash scripts/check-batch-audit-site.sh && bash scripts/check-engine-dynamic-import.sh",
"check:grok-pin": "bash scripts/check-grok-pin.sh",
"check:opencode-pin": "bash scripts/check-opencode-pin.sh",
"check:pin-doc-privacy": "bash scripts/check-pin-doc-privacy.sh",
"check:gateway-routed": "bash scripts/check-gateway-routed-no-direct-anthropic.sh",
"check:worker-pool-atomicity": "bash scripts/check-worker-pool-atomicity.sh",
"check:doc-history": "bash scripts/check-key-files-current-state.sh",
@@ -97,7 +99,9 @@
"prepublish:clawhub": "bun run build:all",
"publish:clawhub": "clawhub package publish . --family bundle-plugin",
"check:skill-refs": "bun scripts/check-skill-refs.mjs",
"gate:skills": "bash scripts/skills-commit-gate.sh"
"gate:skills": "bash scripts/skills-commit-gate.sh",
"check:guard-self-test": "bash scripts/guard-self-test.sh",
"check:no-legacy-getconnection": "bash scripts/check-no-legacy-getconnection.sh"
},
"openclaw": {
"compat": {
@@ -130,6 +134,7 @@
"gray-matter": "^4.0.3",
"heic-decode": "^2.1.0",
"js-yaml": "^3.15.1",
"jsonc-parser": "^3.3.1",
"marked": "^18.0.2",
"openai": "^4.0.0",
"pgvector": "^0.2.0",
@@ -155,7 +160,7 @@
"bun": ">=1.3.10"
},
"license": "MIT",
"version": "0.45.13.0",
"version": "0.46.4.0",
"overrides": {
"@hono/node-server": "^2.0.5",
"fast-uri": "^3.1.5",
+98 -3
View File
@@ -1,6 +1,13 @@
#!/usr/bin/env bun
// scripts/build-pglite-snapshot.ts
//
// TZ pinned to UTC BEFORE any PGLite work: dumpDataDir bakes this process's
// TimeZone into the tar's cluster defaults. Building under the host zone made
// restored engines run sessions in the build machine's zone (the engine also
// re-pins at restore — this is the belt to that suspender, and it keeps any
// OTHER zone-derived state baked into the tar deterministic across hosts).
process.env.TZ = 'UTC';
//
// Tier 3 fast-restore: boot a fresh PGLite, run the full initSchema (forward
// bootstrap + PGLITE_SCHEMA_SQL + every migration), dump the post-init state
// to a tar fixture. Test files that read GBRAIN_PGLITE_SNAPSHOT can skip the
@@ -18,10 +25,14 @@
//
// Re-run whenever you touch src/core/migrate.ts or src/schema.sql.
import { writeFileSync, mkdirSync } from "node:fs";
import { dirname } from "node:path";
import { writeFileSync, mkdirSync, existsSync, readFileSync, rmdirSync, rmSync, mkdtempSync, statSync } from "node:fs";
import { dirname, join } from "node:path";
import { tmpdir } from "node:os";
import * as crypto from "node:crypto";
import { configureGateway, getEmbeddingDimensions, getEmbeddingModel } from "../src/core/ai/gateway.ts";
import { LEGACY_EMBEDDING_CONFIG } from "../test/helpers/legacy-embedding-config.ts";
import { PGLiteEngine, computeSnapshotSchemaHash } from "../src/core/pglite-engine.ts";
import { MIGRATIONS } from "../src/core/migrate.ts";
import { PGLITE_SCHEMA_SQL } from "../src/core/pglite-schema.ts";
@@ -33,9 +44,84 @@ function computeSchemaHash(): string {
async function main() {
const fixturePath = "test/fixtures/pglite-snapshot.tar";
const versionPath = "test/fixtures/pglite-snapshot.version";
const lockPath = "test/fixtures/.pglite-snapshot.lock";
mkdirSync(dirname(fixturePath), { recursive: true });
// W0 fix-wave: build under the EXACT embedding shape the test suite pins.
// bunfig.toml preloads test/helpers/legacy-embedding-preload.ts, which
// configures the gateway to the shared LEGACY_EMBEDDING_CONFIG (OpenAI
// 1536-d) for every `bun test` file — so the snapshot's baked vector(dims)
// columns MUST match that shape, not the builder machine's ambient config
// (nor the shipped 1280-d default an unconfigured gateway falls back to).
// Set in main(), not module scope: ESM hoists imports, so module-scope
// placement implied an ordering it never had — config reads are lazy.
configureGateway({ ...LEGACY_EMBEDDING_CONFIG, env: { ...process.env } });
const schemaHash = computeSchemaHash();
// W0 fix-wave (Tier-1 #16): idempotent short-circuit. Runners now call this
// script UNCONDITIONALLY (build-if-missing left stale-but-present snapshots
// permanently on the warn+slow path); a fresh snapshot exits in ~ms.
const isFresh = () => {
if (!existsSync(fixturePath) || !existsSync(versionPath)) return false;
const lines = readFileSync(versionPath, "utf-8").trim().split("\n");
return lines[0] === schemaHash
&& lines[1] === `dims=${getEmbeddingDimensions()}`
&& lines[2] === `model=${getEmbeddingModel()}`;
};
if (isFresh()) {
console.log(`[build-pglite-snapshot] up to date (hash ${schemaHash.slice(0, 16)}...) — nothing to do`);
return;
}
// GBRAIN_HOME isolation is only needed once we actually BUILD (the engine
// boot reads config). Red-team catch: creating it before the isFresh()
// short-circuit leaked one temp dir per invocation on the COMMON path
// (this script runs on every `bun run test`).
const hermeticHome = mkdtempSync(join(tmpdir(), "gbrain-snapshot-hermetic-"));
process.env.GBRAIN_HOME = hermeticHome;
// W0 fix-wave (D5.8): concurrency lock. Parallel shard runners / concurrent
// Conductor workspaces invoking this simultaneously must not tear the tar.
// mkdir is atomic; the loser polls until the winner finishes, then
// re-checks freshness and exits.
let ownLock = false;
try {
mkdirSync(lockPath);
ownLock = true;
} catch {
console.log(`[build-pglite-snapshot] another builder holds ${lockPath}; waiting...`);
const timeoutMs = Number(process.env.GBRAIN_SNAPSHOT_LOCK_TIMEOUT_MS) || 120_000;
const deadline = Date.now() + timeoutMs;
while (existsSync(lockPath) && Date.now() < deadline) {
await new Promise(r => setTimeout(r, 250));
}
if (isFresh()) {
console.log(`[build-pglite-snapshot] concurrent builder finished; snapshot fresh`);
return;
}
// Stale lock (crashed builder) or still-stale snapshot: TAKE OVER.
// W0 ship-review catch: mkdirSync on a still-existing dir always throws
// EEXIST — the original retry could never acquire, so a single crashed
// builder left every future rebuild waiting the full deadline and then
// proceeding UNLOCKED forever (the stale dir was never removed).
// Red-team refinement: verify STALENESS (lock dir mtime older than the
// full wait window) before the rmdir — two exhausted waiters would
// otherwise each rmdir+mkdir and the second would steal the first's
// just-created LIVE lock, re-opening the torn-tar window.
try {
if (existsSync(lockPath)) {
const ageMs = Date.now() - statSync(lockPath).mtimeMs;
if (ageMs > timeoutMs) {
console.log(`[build-pglite-snapshot] stale lock (age ${Math.round(ageMs / 1000)}s > ${Math.round(timeoutMs / 1000)}s) — taking over`);
rmdirSync(lockPath);
}
}
mkdirSync(lockPath);
ownLock = true;
} catch { /* lock is LIVE (fresh mtime) or takeover raced; proceed unlocked as last resort */ }
}
try {
console.log(`[build-pglite-snapshot] schema hash: ${schemaHash.slice(0, 16)}...`);
console.log(`[build-pglite-snapshot] booting PGLite (in-memory)...`);
const engine = new PGLiteEngine();
@@ -53,12 +139,21 @@ async function main() {
const dump = await engine.db.dumpDataDir("none");
const buffer = Buffer.from(await dump.arrayBuffer());
// Write tar first, version LAST — the version file is the commit point, so
// a crash between the writes leaves a stale-hash (ignored) snapshot, never
// a fresh-looking torn one. Lines 2-3 record the embedding shape the
// snapshot was baked with; the loader refuses a shape-mismatched snapshot
// (the W0 1280-vs-1536 incident class).
writeFileSync(fixturePath, buffer);
writeFileSync(versionPath, schemaHash + "\n");
writeFileSync(versionPath, `${schemaHash}\ndims=${getEmbeddingDimensions()}\nmodel=${getEmbeddingModel()}\n`);
await engine.disconnect();
console.log(`[build-pglite-snapshot] wrote ${fixturePath} (${buffer.length} bytes)`);
console.log(`[build-pglite-snapshot] wrote ${versionPath}`);
} finally {
if (ownLock) { try { rmdirSync(lockPath); } catch { /* best effort */ } }
try { rmSync(hermeticHome, { recursive: true, force: true }); } catch { /* best effort */ }
}
}
await main();
+1 -1
View File
@@ -4,7 +4,7 @@
# scopes to admin/src/ so we can't import the source list directly; instead
# this script extracts both lists and diffs them.
#
# Wired into `bun run verify` and `bun run check:all`.
# Wired into `bun run verify` (single guard registry: scripts/guards-manifest.tsv).
#
# Exits 0 on match, 1 on drift, 2 on internal error (file missing, parse fail).
#
+26 -15
View File
@@ -25,14 +25,16 @@
# (d) Phase-list check [D5]: every `Phase: <name>` in BOOTSTRAP_FOR_AGENTS.md
# must appear in src/core/bootstrap/status.ts (the TS phase list is the
# single source; the runbook defers to it). Skips while either is absent.
# (e) Harness-scoping counter-signal pins: the MCP-scope consent is Claude
# Code only (Codex has no scope flag — `codex mcp add` is user-global).
# Tripwires against accidental deletion of the load-bearing prose, not
# proofs of placement: the runbook must carry the Codex bullet's
# "Do NOT offer an MCP scope choice" and the phase-3 "Claude Code only"
# scoping; questions.json's MCP_SCOPE.question must START WITH
# "(Claude Code only". Intentional rewording updates these pins in the
# same commit. Skips while the runbook/bank are absent.
# (e) Harness-scoping counter-signal pins: the MCP-scope consent applies on
# Claude Code and opencode (Codex has no scope flag — `codex mcp add` is
# user-global; opencode DEFAULTS to user-global — no trust gate on
# project-config servers). Tripwires against accidental deletion of the
# load-bearing prose, not proofs of placement: the runbook must carry the
# Codex bullet's "Do NOT offer an MCP scope choice" and the phase-3
# "Claude Code and opencode" scoping; questions.json's MCP_SCOPE.question
# must START WITH "(Claude Code and opencode". Intentional rewording
# updates these pins in the same commit. Skips while the runbook/bank are
# absent.
#
# BSD/GNU grep portable (no \t escapes). Uses `bun` for JSON parsing — the
# check runs via `bun run verify`, so bun is always present.
@@ -195,7 +197,7 @@ else
echo "SKIP: phase-list check (runbook or src/core/bootstrap/status.ts absent)"
fi
# ── (e) harness-scoping counter-signal pins (MCP scope is Claude Code only) ─
# ── (e) harness-scoping counter-signal pins (scope = Claude Code + opencode) ─
if [ -f "$RUNBOOK" ]; then
if ! grep -qF 'Do NOT offer an MCP scope choice' "$RUNBOOK"; then
fail=1
@@ -204,11 +206,20 @@ if [ -f "$RUNBOOK" ]; then
echo " without this line, Codex-door agents re-ask a dead question." >&2
echo " Rewording intentionally? Update this pin in the same commit." >&2
fi
if ! grep -qF 'Claude Code only' "$RUNBOOK"; then
if ! grep -qF 'Claude Code and opencode' "$RUNBOOK"; then
fail=1
echo "FAIL: BOOTSTRAP_FOR_AGENTS.md lost the 'Claude Code only' scoping on the" >&2
echo " MCP-scope consent (phase 3). Without it the consent reads as" >&2
echo " harness-blind and Codex-door agents ask it." >&2
echo "FAIL: BOOTSTRAP_FOR_AGENTS.md lost the 'Claude Code and opencode' scoping" >&2
echo " on the MCP-scope consent (phase 3). Without it the consent reads as" >&2
echo " harness-blind: Codex-door agents ask a dead question and opencode" >&2
echo " agents miss the inverted (user-global) default." >&2
echo " Rewording intentionally? Update this pin in the same commit." >&2
fi
if ! grep -qF 'NO trust prompt' "$RUNBOOK"; then
fail=1
echo "FAIL: BOOTSTRAP_FOR_AGENTS.md lost the opencode spawn-gate rationale" >&2
echo " ('NO trust prompt'). Without it agents recommend the Claude-style" >&2
echo " project default on opencode — where a committed project entry" >&2
echo " auto-executes on every collaborator machine." >&2
echo " Rewording intentionally? Update this pin in the same commit." >&2
fi
else
@@ -216,9 +227,9 @@ else
fi
if [ -f "$QUESTIONS" ] && command -v bun >/dev/null 2>&1; then
if ! GBRAIN_QJSON="$QUESTIONS" bun -e \
'const fs=require("fs");let b;try{b=JSON.parse(fs.readFileSync(process.env.GBRAIN_QJSON,"utf8"));}catch(e){process.exit(1);}if(!b.questions){process.exit(1);}const e=b.questions.MCP_SCOPE;const q=(e&&e.question)||"";process.exit(q.startsWith("(Claude Code only")&&e.phase==="interview"?0:1);'; then
'const fs=require("fs");let b;try{b=JSON.parse(fs.readFileSync(process.env.GBRAIN_QJSON,"utf8"));}catch(e){process.exit(1);}if(!b.questions){process.exit(1);}const e=b.questions.MCP_SCOPE;const q=(e&&e.question)||"";process.exit(q.startsWith("(Claude Code and opencode")&&e.phase==="interview"?0:1);'; then
fail=1
echo "FAIL: questions.json MCP_SCOPE.question must start with '(Claude Code only'" >&2
echo "FAIL: questions.json MCP_SCOPE.question must start with '(Claude Code and opencode'" >&2
echo " AND MCP_SCOPE.phase must be 'interview' (the consent is recorded" >&2
echo " pre-confirm during the interview; a 'wire' phase re-creates the" >&2
echo " bank-vs-runbook contradiction). Also fails when the questions" >&2
+10 -1
View File
@@ -49,7 +49,16 @@ for (const file of files) {
}
function visit(node: ts.Node): void {
if (ts.isCallExpression(node) && node.expression.kind === ts.SyntaxKind.ImportKeyword) {
// W0 ship-review catch: match BOTH lazy-loading forms. The guard
// previously matched only `import(...)` call expressions, so a
// `require(...)` on an engine-live path passed silently and its
// engine-dynamic-import-ok marker was decorative.
const isDynamicImport = ts.isCallExpression(node)
&& node.expression.kind === ts.SyntaxKind.ImportKeyword;
const isRequireCall = ts.isCallExpression(node)
&& ts.isIdentifier(node.expression)
&& node.expression.text === 'require';
if (isDynamicImport || isRequireCall) {
const { line } = sourceFile.getLineAndCharacterOfPosition(node.expression.getStart(sourceFile));
const sourceLine = lines[line] ?? '';
if (!markerLines.has(line)) {
+11 -5
View File
@@ -21,10 +21,16 @@
set -euo pipefail
FIXTURE_DIR="test/fixtures/conversation-formats"
# cathedral-4: the transcripts-import fixtures (raw harness/export shapes)
# carry the same placeholder-names-only contract as conversation-formats.
FIXTURE_DIRS=("test/fixtures/conversation-formats" "test/fixtures/transcripts")
if [ ! -d "$FIXTURE_DIR" ]; then
echo "[check-fixture-privacy] $FIXTURE_DIR does not exist; nothing to check"
EXISTING_DIRS=()
for d in "${FIXTURE_DIRS[@]}"; do
[ -d "$d" ] && EXISTING_DIRS+=("$d")
done
if [ ${#EXISTING_DIRS[@]} -eq 0 ]; then
echo "[check-fixture-privacy] no fixture dirs exist; nothing to check"
exit 0
fi
@@ -45,7 +51,7 @@ BANNED_TOKENS=(
errors=0
for token in "${BANNED_TOKENS[@]}"; do
matches=$(grep -ril "$token" "$FIXTURE_DIR" 2>/dev/null || true)
matches=$(grep -ril "$token" "${EXISTING_DIRS[@]}" 2>/dev/null || true)
if [ -n "$matches" ]; then
echo "[check-fixture-privacy] BANNED token '$token' found in:"
echo "$matches" | sed 's/^/ - /'
@@ -61,4 +67,4 @@ if [ "$errors" -gt 0 ]; then
exit 1
fi
echo "[check-fixture-privacy] OK: no banned tokens found in $FIXTURE_DIR"
echo "[check-fixture-privacy] OK: no banned tokens found in ${EXISTING_DIRS[*]}"
+125
View File
@@ -0,0 +1,125 @@
#!/usr/bin/env bash
# scripts/check-grok-pin.sh — grok pin consistency guard.
#
# GROK-CLI-PIN.md is the single observed-behavior source for the grok
# integration; its pins fan out to the heavy-tests grok-door job env, the
# GrokRunner argv, and the door e2e assertions. The prose rule is "update
# together" — this guard turns the workflow half of that rule into CI:
#
# 1. docs/mcp/GROK-CLI-PIN.md carries a machine-stable stamp block
# (`<!-- grok-pin: key=value -->`, one per line) including
# distribution_kind (npm | installer).
# 2. The grok-door job env in .github/workflows/heavy-tests.yml must carry
# EXACTLY the pin set for the chosen distribution_kind:
# npm: GROK_VERSION==grok_version, GROK_NPM_PACKAGE==npm_package,
# GROK_NPM_INTEGRITY==npm_integrity; no GROK_INSTALL_SHA256.
# installer: GROK_VERSION==grok_version,
# GROK_INSTALL_SHA256==installer_sha256; no GROK_NPM_INTEGRITY.
# (The pin DOC may document both — the fallback path stays written down;
# exclusivity is about which pins the WORKFLOW actually enforces.)
#
# Greps are anchored to the grok-door job block so a future canary matrix leg
# (or a second door job) cannot satisfy the check by accident.
#
# SKIP-GRACEFUL: missing pin doc, missing workflow, or no grok-door job yet →
# SKIP (exit 0), matching scripts/check-bootstrap-tag.sh. Test override:
# GBRAIN_GROK_PIN_GUARD_ROOT points file resolution at a fixture tree.
# BSD/GNU portable (no \t escapes, no GNU-only flags).
set -euo pipefail
ROOT="${GBRAIN_GROK_PIN_GUARD_ROOT:-$(cd "$(dirname "$0")/.." && pwd)}"
PIN_FILE="$ROOT/docs/mcp/GROK-CLI-PIN.md"
WORKFLOW="$ROOT/.github/workflows/heavy-tests.yml"
if [ ! -f "$WORKFLOW" ]; then
echo "check-grok-pin: SKIP (no $WORKFLOW)"
exit 0
fi
if ! grep -q '^ grok-door:' "$WORKFLOW"; then
echo "check-grok-pin: SKIP (no grok-door job in heavy-tests.yml yet)"
exit 0
fi
# Once the grok-door job EXISTS, a missing pin doc is a FAILURE, not a skip —
# deleting/renaming the doc must not silently disable the supply-chain gate.
if [ ! -f "$PIN_FILE" ]; then
echo "check-grok-pin: FAIL — grok-door job exists but $PIN_FILE is missing (the pin doc is the gate's source of truth)" >&2
exit 1
fi
fail() {
echo "check-grok-pin: FAIL — $1" >&2
exit 1
}
# --- 1. Parse the stamp block ------------------------------------------------
stamp() {
# First occurrence wins; a missing stamp yields the empty string (callers
# decide whether that is a failure) — the `|| true` keeps set -e/pipefail
# from treating grep's no-match exit as a script error.
{ grep -E "^<!-- grok-pin: $1=" "$PIN_FILE" || true; } | head -1 \
| sed -e 's/^<!-- grok-pin: [a-z0-9_]*=//' -e 's/ -->$//'
}
# Duplicate stamps are drift bait (two values, which one is real?).
dupes=$({ grep -E '^<!-- grok-pin: ' "$PIN_FILE" || true; } | sed -e 's/^<!-- grok-pin: //' -e 's/=.*$//' | sort | uniq -d)
[ -n "$dupes" ] && fail "duplicate grok-pin stamp(s) in GROK-CLI-PIN.md: $dupes"
DIST_KIND=$(stamp distribution_kind)
GROK_VERSION_PIN=$(stamp grok_version)
[ -n "$DIST_KIND" ] || fail "GROK-CLI-PIN.md is missing the distribution_kind stamp"
[ -n "$GROK_VERSION_PIN" ] || fail "GROK-CLI-PIN.md is missing the grok_version stamp"
case "$DIST_KIND" in
npm|installer) ;;
*) fail "distribution_kind stamp must be npm or installer; got '$DIST_KIND'" ;;
esac
# --- 2. Extract the grok-door job block --------------------------------------
# Jobs sit at 2-space indent; the block ends at the next 2-space-indented key.
job_block=$(awk '
/^ grok-door:/ { f = 1; print; next }
f && /^ [A-Za-z0-9_-]+:/ { exit }
f { print }
' "$WORKFLOW")
[ -n "$job_block" ] || fail "could not extract the grok-door job block"
wf_env() {
# Strip either quote style: a YAML-formatter pass flipping double to single
# quotes must not read as pin drift.
{ printf '%s\n' "$job_block" | grep -E "^ $1:" || true; } | head -1 \
| sed -e "s/^ $1:[[:space:]]*//" -e 's/^"//' -e 's/"$//' -e "s/^'//" -e "s/'\$//"
}
WF_VERSION=$(wf_env GROK_VERSION)
WF_NPM_PACKAGE=$(wf_env GROK_NPM_PACKAGE)
WF_NPM_INTEGRITY=$(wf_env GROK_NPM_INTEGRITY)
WF_INSTALL_SHA=$(wf_env GROK_INSTALL_SHA256)
[ -n "$WF_VERSION" ] || fail "grok-door job env is missing GROK_VERSION"
[ "$WF_VERSION" = "$GROK_VERSION_PIN" ] || fail "GROK_VERSION drift — workflow '$WF_VERSION' vs pin-doc stamp '$GROK_VERSION_PIN' (update together; see the pin doc's re-observation checklist)"
if [ "$DIST_KIND" = "npm" ]; then
NPM_PACKAGE_PIN=$(stamp npm_package)
NPM_INTEGRITY_PIN=$(stamp npm_integrity)
[ -n "$NPM_PACKAGE_PIN" ] || fail "distribution_kind=npm but GROK-CLI-PIN.md is missing the npm_package stamp"
[ -n "$NPM_INTEGRITY_PIN" ] || fail "distribution_kind=npm but GROK-CLI-PIN.md is missing the npm_integrity stamp"
[ -n "$WF_NPM_PACKAGE" ] || fail "distribution_kind=npm but the grok-door job env is missing GROK_NPM_PACKAGE"
[ -n "$WF_NPM_INTEGRITY" ] || fail "distribution_kind=npm but the grok-door job env is missing GROK_NPM_INTEGRITY"
[ "$WF_NPM_PACKAGE" = "$NPM_PACKAGE_PIN" ] || fail "GROK_NPM_PACKAGE drift — workflow '$WF_NPM_PACKAGE' vs stamp '$NPM_PACKAGE_PIN'"
[ "$WF_NPM_INTEGRITY" = "$NPM_INTEGRITY_PIN" ] || fail "GROK_NPM_INTEGRITY drift — workflow vs stamp mismatch"
# npm_version is a documented near-duplicate of grok_version — assert they
# agree so bumping one alone can never pass green.
NPM_VERSION_PIN=$(stamp npm_version)
if [ -n "$NPM_VERSION_PIN" ] && [ "$NPM_VERSION_PIN" != "$GROK_VERSION_PIN" ]; then
fail "npm_version stamp ($NPM_VERSION_PIN) disagrees with grok_version stamp ($GROK_VERSION_PIN) — update together"
fi
[ -z "$WF_INSTALL_SHA" ] || fail "distribution_kind=npm but the grok-door job also pins GROK_INSTALL_SHA256 — one provisioning mode only (mode exclusivity)"
else
INSTALL_SHA_PIN=$(stamp installer_sha256)
[ -n "$INSTALL_SHA_PIN" ] || fail "distribution_kind=installer but GROK-CLI-PIN.md is missing the installer_sha256 stamp"
[ -n "$WF_INSTALL_SHA" ] || fail "distribution_kind=installer but the grok-door job env is missing GROK_INSTALL_SHA256"
[ "$WF_INSTALL_SHA" = "$INSTALL_SHA_PIN" ] || fail "GROK_INSTALL_SHA256 drift — workflow vs stamp mismatch"
[ -z "$WF_NPM_INTEGRITY" ] || fail "distribution_kind=installer but the grok-door job also pins GROK_NPM_INTEGRITY — one provisioning mode only (mode exclusivity)"
fi
echo "check-grok-pin: ok ($DIST_KIND mode, grok $GROK_VERSION_PIN)"
+27 -9
View File
@@ -17,26 +17,43 @@ set -euo pipefail
ROOT="$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
cd "$ROOT"
# Match the interpolated form: ${JSON.stringify(...)}::jsonb
# Using grep -P for Perl-compatible regex (lookahead-free pattern is enough here).
PATTERN='\$\{JSON\.stringify\([^)]*\)\}::jsonb'
# W0 fix-wave (Tier-1 #11): self-test seam — the guard harness points this at
# a known-bad fixture tree and asserts exit 1.
SCAN_ROOT="${GBRAIN_GUARD_ROOT:-src/}"
if grep -rEn "$PATTERN" src/ 2>/dev/null; then
# Match the interpolated form: ${JSON.stringify(...)}::jsonb
#
# W0 fix-wave (Tier-1 #11): the previous `\([^)]*\)` argument matcher could
# not cross a nested `)` — `${JSON.stringify(obj.get())}::jsonb` was
# invisible (the same regex-hole class that made check-no-double-retry a
# permanently-green no-op). `[^}]*` spans nested parens but CANNOT cross the
# interpolation's closing `}`, so a safe `${JSON.stringify(x)}::text::jsonb`
# followed by a separate `${expr()}::jsonb` on the same line is not spanned
# into a false positive (ship-review catch — the greedy `.*` variant was).
PATTERN='\$\{JSON\.stringify\([^}]*\)\}::jsonb'
if grep -rEn "$PATTERN" "$SCAN_ROOT" 2>/dev/null; then
echo
echo "ERROR: Found JSON.stringify(...)::jsonb pattern in src/."
echo "ERROR: Found JSON.stringify(...)::jsonb pattern in $SCAN_ROOT."
echo " postgres.js v3 stringifies again, producing JSONB string literals."
echo " Use sql.json(x) instead. See feedback_postgres_jsonb_double_encode.md."
exit 1
fi
echo "OK: no JSON.stringify(x)::jsonb interpolation pattern in src/"
echo "OK: no JSON.stringify(x)::jsonb interpolation pattern in $SCAN_ROOT"
# v0.13.1 #219: guard against max_stalled DEFAULT 1 regressing in any schema
# source file. DEFAULT 1 dead-lettered any SIGKILL'd job on first stall, making
# the "10/10 rescued" claim false for out-of-the-box users. Default is 5 now.
MAX_STALLED_PATTERN='max_stalled\s+INTEGER\s+NOT\s+NULL\s+DEFAULT\s+1\b'
if grep -rEn "$MAX_STALLED_PATTERN" src/schema.sql src/core/migrate.ts src/core/pglite-schema.ts src/core/schema-embedded.ts 2>/dev/null; then
# Schema files are fixed paths; under a fixture root (self-test) they don't
# exist — skip rather than fail on the missing-file grep.
SCHEMA_FILES=()
for f in src/schema.sql src/core/migrate.ts src/core/pglite-schema.ts src/core/schema-embedded.ts; do
[ -f "$f" ] && SCHEMA_FILES+=("$f")
done
if [ "${#SCHEMA_FILES[@]}" -gt 0 ] && grep -rEn "$MAX_STALLED_PATTERN" "${SCHEMA_FILES[@]}" 2>/dev/null; then
echo
echo "ERROR: max_stalled DEFAULT 1 reintroduced in schema."
echo " Must be DEFAULT 5 to preserve SIGKILL-rescue guarantee. See #219."
@@ -51,10 +68,11 @@ echo "OK: max_stalled defaults are 5 in all schema sources"
# [JSON.stringify(x)]) — which is the exact shape that double-encoded the
# op_checkpoints pin and aborted every sync in #2339. The AST-lite scanner below
# catches it. `set -e` propagates its non-zero exit.
# Under a fixture root, scan that root; the AST-lite scanner takes roots as argv.
if command -v node >/dev/null 2>&1; then
node scripts/check-jsonb-params.mjs
node scripts/check-jsonb-params.mjs ${GBRAIN_GUARD_ROOT:+"$GBRAIN_GUARD_ROOT"}
elif command -v bun >/dev/null 2>&1; then
bun scripts/check-jsonb-params.mjs
bun scripts/check-jsonb-params.mjs ${GBRAIN_GUARD_ROOT:+"$GBRAIN_GUARD_ROOT"}
else
echo "WARN: neither node nor bun on PATH; skipping check-jsonb-params.mjs" >&2
fi
+36 -19
View File
@@ -19,16 +19,24 @@ set -euo pipefail
ROOT="$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
cd "$ROOT"
# W0 fix-wave (Tier-1 #11): self-test seam. The guard harness points this at
# a known-bad fixture tree and asserts exit 1 — the guard can no longer rot
# into a permanently-green no-op unnoticed.
SCAN_ROOT="${GBRAIN_GUARD_ROOT:-src/}"
# Match: withRetry(...) wrapping any of the 3 engine batch methods.
# The greedy `.*` between `withRetry(` and `engine.` covers both the
# arrow-fn form and any direct invocation. (gbrain-allow-direct-insert: doc comment)
# Multi-line wraps are caught by `grep -E` per file (line-wise) for the
# common single-line case; multi-line wraps still get caught by a separate
# multi-line pass below.
PATTERN='withRetry\([^)]*engine\.(addLinksBatch|addTimelineEntriesBatch|upsertChunks)'
#
# W0 fix-wave (Tier-1 #11): the previous pattern used `[^)]*` between
# `withRetry(` and `engine.`, which can never cross the `)` in `() =>` — so
# the CANONICAL banned shape (an arrow function wrapping the engine batch
# call) was invisible and the guard had been permanently green since it
# shipped. `.*` (line-bounded by grep) covers the arrow form, async arrows,
# and any argument shape. (Spelled without the literal call token here —
# check-system-of-record scans scripts/ comments too: the prose-bleed class.)
PATTERN='withRetry\(.*engine\.(addLinksBatch|addTimelineEntriesBatch|upsertChunks)'
# Single-line scan (covers ~95% of real cases).
if grep -rEn "$PATTERN" src/ --include='*.ts' 2>/dev/null; then
if grep -rEn "$PATTERN" "$SCAN_ROOT" --include='*.ts' 2>/dev/null; then
echo
echo "ERROR: Found withRetry(...engine.{addLinksBatch|addTimelineEntriesBatch|upsertChunks})"
echo " pattern in src/."
@@ -47,17 +55,26 @@ if grep -rEn "$PATTERN" src/ --include='*.ts' 2>/dev/null; then
exit 1
fi
# Multi-line scan: a withRetry( on one line and the engine call on the next
# few. Bounded to 3-line window so we don't flag distant unrelated calls.
# Uses pcregrep if available, else falls back to a simple awk window.
if command -v pcregrep >/dev/null 2>&1; then
if pcregrep -r -M -n --include='\.ts$' \
'withRetry\([^)]*\n\s*\(?[^)]*=>\s*engine\.(addLinksBatch|addTimelineEntriesBatch|upsertChunks)' \
src/ 2>/dev/null; then
echo
echo "ERROR: Multi-line withRetry(...engine.batch...) wrap found in src/. See above."
exit 1
fi
# Multi-line scan: a withRetry( on one line and the engine call within the
# next 3 lines. W0 fix-wave (Tier-1 #11): the previous pass was gated on
# pcregrep, which is not installed on dev machines OR CI — it never ran.
# perl is always available; same 3-line window, always on.
#
# Ship-review catch: perl must ALWAYS exit 0 and let OUTPUT PRESENCE decide.
# An exit-1-from-clean-batches design breaks under `set -o pipefail` the
# moment src/ outgrows one xargs batch (xargs exits 123, overriding grep's
# verdict) — a silently missed violation, the same permanently-green class
# this guard was just cured of.
MULTILINE_MATCHES=$(find "$SCAN_ROOT" -name '*.ts' -type f -print0 2>/dev/null | xargs -0 perl -0777 -ne '
if (/withRetry\([^\n]*\n(?:[^\n]*\n){0,2}?[^\n]*engine\.(?:addLinksBatch|addTimelineEntriesBatch|upsertChunks)/) {
print "$ARGV: multi-line withRetry wrap around an engine batch call\n";
}
' 2>/dev/null || true)
if [ -n "$MULTILINE_MATCHES" ]; then
echo "$MULTILINE_MATCHES"
echo
echo "ERROR: Multi-line withRetry(...engine.batch...) wrap found in $SCAN_ROOT. See above."
exit 1
fi
echo "OK: no withRetry(...engine.batch...) double-retry patterns in src/"
echo "OK: no withRetry(...engine.batch...) double-retry patterns in $SCAN_ROOT"
+149
View File
@@ -0,0 +1,149 @@
#!/usr/bin/env bash
# scripts/check-opencode-pin.sh — opencode pin consistency guard.
#
# OPENCODE-CLI-PIN.md is the single observed-behavior source for the opencode
# integration; its pins fan out to the heavy-tests opencode-door job env, the
# OpencodeRunner argv, and the door e2e assertions. The prose rule is "update
# together" — this guard turns the workflow half of that rule into CI:
#
# 1. docs/mcp/OPENCODE-CLI-PIN.md carries a machine-stable stamp block
# (`<!-- opencode-pin: key=value -->`, one per line) including
# distribution_kind (npm | installer).
# 2. The opencode-door job env in .github/workflows/heavy-tests.yml must carry
# EXACTLY the pin set for the chosen distribution_kind:
# npm: OPENCODE_VERSION==opencode_version, OPENCODE_NPM_PACKAGE==npm_package,
# OPENCODE_NPM_INTEGRITY==npm_integrity; no OPENCODE_INSTALL_SHA256.
# installer: OPENCODE_VERSION==opencode_version,
# OPENCODE_INSTALL_SHA256==installer_sha256; no OPENCODE_NPM_INTEGRITY.
# (The pin DOC may document both — the fallback path stays written down;
# exclusivity is about which pins the WORKFLOW actually enforces.)
#
# Greps are anchored to the opencode-door job block so a future canary matrix leg
# (or a second door job) cannot satisfy the check by accident.
#
# SKIP-GRACEFUL: missing pin doc, missing workflow, or no opencode-door job yet →
# SKIP (exit 0), matching scripts/check-bootstrap-tag.sh. Test override:
# GBRAIN_OPENCODE_PIN_GUARD_ROOT points file resolution at a fixture tree.
# BSD/GNU portable (no \t escapes, no GNU-only flags).
set -euo pipefail
ROOT="${GBRAIN_OPENCODE_PIN_GUARD_ROOT:-$(cd "$(dirname "$0")/.." && pwd)}"
PIN_FILE="$ROOT/docs/mcp/OPENCODE-CLI-PIN.md"
WORKFLOW="$ROOT/.github/workflows/heavy-tests.yml"
if [ ! -f "$WORKFLOW" ]; then
echo "check-opencode-pin: SKIP (no $WORKFLOW)"
exit 0
fi
if ! grep -q '^ opencode-door:' "$WORKFLOW"; then
echo "check-opencode-pin: SKIP (no opencode-door job in heavy-tests.yml yet)"
exit 0
fi
# Once the opencode-door job EXISTS, a missing pin doc is a FAILURE, not a skip —
# deleting/renaming the doc must not silently disable the supply-chain gate.
if [ ! -f "$PIN_FILE" ]; then
echo "check-opencode-pin: FAIL — opencode-door job exists but $PIN_FILE is missing (the pin doc is the gate's source of truth)" >&2
exit 1
fi
fail() {
echo "check-opencode-pin: FAIL — $1" >&2
exit 1
}
# --- 1. Parse the stamp block ------------------------------------------------
stamp() {
# First occurrence wins; a missing stamp yields the empty string (callers
# decide whether that is a failure) — the `|| true` keeps set -e/pipefail
# from treating grep's no-match exit as a script error.
{ grep -E "^<!-- opencode-pin: $1=" "$PIN_FILE" || true; } | head -1 \
| sed -e 's/^<!-- opencode-pin: [a-z0-9_]*=//' -e 's/ -->$//'
}
# Duplicate stamps are drift bait (two values, which one is real?).
dupes=$({ grep -E '^<!-- opencode-pin: ' "$PIN_FILE" || true; } | sed -e 's/^<!-- opencode-pin: //' -e 's/=.*$//' | sort | uniq -d)
[ -n "$dupes" ] && fail "duplicate opencode-pin stamp(s) in OPENCODE-CLI-PIN.md: $dupes"
DIST_KIND=$(stamp distribution_kind)
OPENCODE_VERSION_PIN=$(stamp opencode_version)
[ -n "$DIST_KIND" ] || fail "OPENCODE-CLI-PIN.md is missing the distribution_kind stamp"
[ -n "$OPENCODE_VERSION_PIN" ] || fail "OPENCODE-CLI-PIN.md is missing the opencode_version stamp"
case "$DIST_KIND" in
npm|installer) ;;
*) fail "distribution_kind stamp must be npm or installer; got '$DIST_KIND'" ;;
esac
# --- 2. Extract the opencode-door job block --------------------------------------
# Jobs sit at 2-space indent; the block ends at the next 2-space-indented key.
job_block=$(awk '
/^ opencode-door:/ { f = 1; print; next }
f && /^ [A-Za-z0-9_-]+:/ { exit }
f { print }
' "$WORKFLOW")
[ -n "$job_block" ] || fail "could not extract the opencode-door job block"
wf_env() {
# Strip either quote style: a YAML-formatter pass flipping double to single
# quotes must not read as pin drift.
{ printf '%s\n' "$job_block" | grep -E "^ $1:" || true; } | head -1 \
| sed -e "s/^ $1:[[:space:]]*//" -e 's/^"//' -e 's/"$//' -e "s/^'//" -e "s/'\$//"
}
WF_VERSION=$(wf_env OPENCODE_VERSION)
WF_NPM_PACKAGE=$(wf_env OPENCODE_NPM_PACKAGE)
WF_NPM_INTEGRITY=$(wf_env OPENCODE_NPM_INTEGRITY)
WF_INSTALL_SHA=$(wf_env OPENCODE_INSTALL_SHA256)
[ -n "$WF_VERSION" ] || fail "opencode-door job env is missing OPENCODE_VERSION"
[ "$WF_VERSION" = "$OPENCODE_VERSION_PIN" ] || fail "OPENCODE_VERSION drift — workflow '$WF_VERSION' vs pin-doc stamp '$OPENCODE_VERSION_PIN' (update together; see the pin doc's re-observation checklist)"
# EVERY OPENCODE_VERSION: env line in the WHOLE workflow (the real-agent-e2e
# door job carries a second copy) must equal the stamp — bumping the door job
# alone must never pass green. Env keys sit at line start after indentation,
# so comments mentioning the name never match.
all_wf_versions=$({ grep -E '^[[:space:]]*OPENCODE_VERSION:' "$WORKFLOW" || true; } \
| sed -e 's/^[[:space:]]*OPENCODE_VERSION:[[:space:]]*//' -e 's/^"//' -e 's/"$//' -e "s/^'//" -e "s/'\$//")
for v in $all_wf_versions; do
[ "$v" = "$OPENCODE_VERSION_PIN" ] || fail "an OPENCODE_VERSION occurrence elsewhere in heavy-tests.yml ('$v') disagrees with the pin-doc stamp '$OPENCODE_VERSION_PIN' — every copy in the workflow moves with the stamp"
done
if [ "$DIST_KIND" = "npm" ]; then
NPM_PACKAGE_PIN=$(stamp npm_package)
NPM_INTEGRITY_PIN=$(stamp npm_integrity)
[ -n "$NPM_PACKAGE_PIN" ] || fail "distribution_kind=npm but OPENCODE-CLI-PIN.md is missing the npm_package stamp"
[ -n "$NPM_INTEGRITY_PIN" ] || fail "distribution_kind=npm but OPENCODE-CLI-PIN.md is missing the npm_integrity stamp"
[ -n "$WF_NPM_PACKAGE" ] || fail "distribution_kind=npm but the opencode-door job env is missing OPENCODE_NPM_PACKAGE"
[ -n "$WF_NPM_INTEGRITY" ] || fail "distribution_kind=npm but the opencode-door job env is missing OPENCODE_NPM_INTEGRITY"
[ "$WF_NPM_PACKAGE" = "$NPM_PACKAGE_PIN" ] || fail "OPENCODE_NPM_PACKAGE drift — workflow '$WF_NPM_PACKAGE' vs stamp '$NPM_PACKAGE_PIN'"
[ "$WF_NPM_INTEGRITY" = "$NPM_INTEGRITY_PIN" ] || fail "OPENCODE_NPM_INTEGRITY drift — workflow vs stamp mismatch"
# npm_version is a documented near-duplicate of opencode_version — assert they
# agree so bumping one alone can never pass green.
NPM_VERSION_PIN=$(stamp npm_version)
if [ -n "$NPM_VERSION_PIN" ] && [ "$NPM_VERSION_PIN" != "$OPENCODE_VERSION_PIN" ]; then
fail "npm_version stamp ($NPM_VERSION_PIN) disagrees with opencode_version stamp ($OPENCODE_VERSION_PIN) — update together"
fi
# Platform-payload integrity stamps (the door job byte-pins the linux
# sub-packages too): when the pin doc carries them, the job env must match.
X64_PIN=$(stamp npm_linux_x64_integrity)
if [ -n "$X64_PIN" ]; then
WF_X64=$(wf_env OPENCODE_NPM_LINUX_X64_INTEGRITY)
[ -n "$WF_X64" ] || fail "pin doc stamps npm_linux_x64_integrity but the opencode-door job env is missing OPENCODE_NPM_LINUX_X64_INTEGRITY"
[ "$WF_X64" = "$X64_PIN" ] || fail "OPENCODE_NPM_LINUX_X64_INTEGRITY drift — workflow vs stamp mismatch"
fi
ARM64_PIN=$(stamp npm_linux_arm64_integrity)
if [ -n "$ARM64_PIN" ]; then
WF_ARM64=$(wf_env OPENCODE_NPM_LINUX_ARM64_INTEGRITY)
[ -n "$WF_ARM64" ] || fail "pin doc stamps npm_linux_arm64_integrity but the opencode-door job env is missing OPENCODE_NPM_LINUX_ARM64_INTEGRITY"
[ "$WF_ARM64" = "$ARM64_PIN" ] || fail "OPENCODE_NPM_LINUX_ARM64_INTEGRITY drift — workflow vs stamp mismatch"
fi
[ -z "$WF_INSTALL_SHA" ] || fail "distribution_kind=npm but the opencode-door job also pins OPENCODE_INSTALL_SHA256 — one provisioning mode only (mode exclusivity)"
else
INSTALL_SHA_PIN=$(stamp installer_sha256)
[ -n "$INSTALL_SHA_PIN" ] || fail "distribution_kind=installer but OPENCODE-CLI-PIN.md is missing the installer_sha256 stamp"
[ -n "$WF_INSTALL_SHA" ] || fail "distribution_kind=installer but the opencode-door job env is missing OPENCODE_INSTALL_SHA256"
[ "$WF_INSTALL_SHA" = "$INSTALL_SHA_PIN" ] || fail "OPENCODE_INSTALL_SHA256 drift — workflow vs stamp mismatch"
[ -z "$WF_NPM_INTEGRITY" ] || fail "distribution_kind=installer but the opencode-door job also pins OPENCODE_NPM_INTEGRITY — one provisioning mode only (mode exclusivity)"
fi
echo "check-opencode-pin: ok ($DIST_KIND mode, opencode $OPENCODE_VERSION_PIN)"
+1 -1
View File
@@ -10,7 +10,7 @@
# - new logging APIs that may show up later (the regex matches the URL,
# not the consumer; any leak will trip)
#
# Wired into bun run check:all and bun run verify.
# Wired into bun run verify (single guard registry: scripts/guards-manifest.tsv).
#
# Exit codes: 0 = clean, 1 = found at least one suspect line.
set -euo pipefail
+70
View File
@@ -0,0 +1,70 @@
#!/usr/bin/env bash
# scripts/check-pin-doc-privacy.sh — PIN-doc privacy guard.
#
# The docs/mcp/*-CLI-PIN.md files carry VERBATIM observation transcripts from
# real installs (help output, saved configs, error copy). That verbatim
# discipline is the point — but it is also exactly how an operator path
# (/Users/<name>/…), a key fragment, or an account id ends up committed and
# shipped with every release. This guard asserts the placeholder discipline:
#
# 1. No operator home paths: /Users/<name>/ or /home/<name>/ must appear as
# placeholders (<tmp>, $HOME, ~/) — never as a real username path.
# Bare `~/.grok`-style spellings are fine (that IS the placeholder).
# 2. No key material: long high-entropy tokens with known prefixes
# (sk-…, xai-…, gbrain_<64+hex-ish>, ANTHROPIC/OPENAI/XAI key shapes).
# npm `sha512-…` integrity pins are EXPECTED content — excluded.
# 3. No obvious account ids: emails outside example.com/invalid domains.
#
# SKIP-GRACEFUL: no pin docs yet → SKIP (exit 0). Test override:
# GBRAIN_PIN_PRIVACY_GUARD_ROOT points file resolution at a fixture tree.
# BSD/GNU portable.
set -uo pipefail
ROOT="${GBRAIN_PIN_PRIVACY_GUARD_ROOT:-$(cd "$(dirname "$0")/.." && pwd)}"
shopt -s nullglob
PIN_DOCS=("$ROOT"/docs/mcp/*-CLI-PIN.md)
shopt -u nullglob
if [ "${#PIN_DOCS[@]}" -eq 0 ]; then
echo "check-pin-doc-privacy: SKIP (no docs/mcp/*-CLI-PIN.md yet)"
exit 0
fi
fail=0
for doc in "${PIN_DOCS[@]}"; do
rel="${doc#"$ROOT"/}"
# 1. Operator home paths (a real username after /Users/ or /home/).
hits=$(grep -nE '(/Users|/home)/[A-Za-z][A-Za-z0-9._-]+/' "$doc" || true)
if [ -n "$hits" ]; then
fail=1
echo "FAIL: $rel carries operator home path(s) — replace with <tmp>/\$HOME/~ placeholders:" >&2
printf '%s\n' "$hits" | sed 's/^/ /' >&2
fi
# 2. Key material. sha512- npm integrity pins are expected; exclude lines
# carrying them before scanning for long secret-shaped runs.
hits=$(grep -v 'sha512-' "$doc" | grep -nE '(sk-[A-Za-z0-9_-]{20,}|xai-[A-Za-z0-9_-]{20,}|gbrain_[A-Za-z0-9]{32,}|AKIA[0-9A-Z]{16})' || true)
if [ -n "$hits" ]; then
fail=1
echo "FAIL: $rel carries key-shaped material — redact before committing:" >&2
printf '%s\n' "$hits" | sed 's/^/ /' >&2
fi
# 3. Emails outside the documentation-safe domains.
hits=$(grep -nE '[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}' "$doc" \
| grep -vE '@(example\.(com|org|net)|[A-Za-z0-9.-]*invalid)' || true)
if [ -n "$hits" ]; then
fail=1
echo "FAIL: $rel carries a non-placeholder email address:" >&2
printf '%s\n' "$hits" | sed 's/^/ /' >&2
fi
done
if [ "$fail" -ne 0 ]; then
echo "check-pin-doc-privacy: FAIL (pin docs ship with every release — placeholder discipline is the privacy IRON RULE)" >&2
exit 1
fi
echo "check-pin-doc-privacy: ok (${#PIN_DOCS[@]} pin doc(s))"
+6 -6
View File
@@ -236,12 +236,12 @@ bash scripts/check-progress-to-stdout.sh
bash scripts/check-trailing-newline.sh
bash scripts/check-wasm-embedded.sh
bun run typecheck
echo \"[runner] Tier 3: building PGLite snapshot fixture (cached across reruns)\"
if [ ! -f test/fixtures/pglite-snapshot.tar ] || [ ! -f test/fixtures/pglite-snapshot.version ]; then
bun run build:pglite-snapshot
else
echo \"[runner] snapshot fixture exists; engine will validate hash at load time\"
fi
echo \"[runner] Tier 3: PGLite snapshot fixture (idempotent; rebuilds on hash drift)\"
# W0 fix-wave (Tier-1 #16): unconditional call — the build script self-
# short-circuits on a fresh hash and rebuilds STALE snapshots (the old
# if-missing guard left a stale-but-present snapshot permanently on the
# warn+slow path). Concurrency-safe via the script's mkdir lock (D5.8).
bun run build:pglite-snapshot
export GBRAIN_PGLITE_SNAPSHOT=test/fixtures/pglite-snapshot.tar
echo \"[runner] resolving E2E file selection (--diff aware)\"
${DIFF_E2E_PREP}
+377 -98
View File
@@ -28,6 +28,8 @@
* so the interview completes unattended. Pays real API cost;
* takes 10-25 min. Run in background and watch session/screen.txt.
* codex-install Same for REAL `codex` (interactive TUI).
* opencode-install Same for REAL `opencode` (bootstrap-supported; the keyless
* run rides the anonymous free tier and should COMPLETE).
* drive -- <cmd> Manual mode: spawn ANY command under the PTY and steer it
* across separate shell calls via a file control channel:
* watch: cat <dir>/session/screen.txt
@@ -45,6 +47,7 @@
* bun run scripts/dx-explore.ts init
* bun run scripts/dx-explore.ts claude-install
* bun run scripts/dx-explore.ts codex-install
* bun run scripts/dx-explore.ts opencode-install [--keyless]
* bun run scripts/dx-explore.ts drive [--no-hermetic-home] -- gbrain init
* Options: --dir <out> transcript dir (default .context/dx-runs/<scenario>-<ts>)
* --gbrain <bin> use an existing gbrain binary (default: compile+cache)
@@ -69,6 +72,10 @@ import {
saveTranscript,
seedClaudeTuiConfig,
parseDriveCommand,
ptySupported,
redactSecrets,
stripAnsi,
MIN_REDACT_SECRET_LEN,
type TtySession,
} from '../test/helpers/tty-harness.ts';
@@ -118,11 +125,14 @@ interface CliArgs {
driveArgv: string[];
}
/** Provider keys the hermetic base allows through; --keyless drops them. */
/** Provider keys the hermetic base allows through; --keyless drops them.
* Also the redaction-map source: every non-empty value here is scrubbed
* from every written artifact. */
const PROVIDER_KEY_NAMES = [
'ANTHROPIC_API_KEY',
'ANTHROPIC_AUTH_TOKEN',
'OPENAI_API_KEY',
'XAI_API_KEY',
'GSTACK_ANTHROPIC_API_KEY',
'GSTACK_OPENAI_API_KEY',
];
@@ -202,10 +212,25 @@ interface ScenarioCtx {
* suffixes). ALWAYS deleted at cleanup --keep keeps transcripts and
* hermetic dirs for forensics, never credentials. */
secretPaths: string[];
/** Secret VALUES (name value) redacted from every written artifact
* transcripts, the live screen mirror, events.jsonl. Structural, not
* checklist: writes go through redactSecrets at the write site. */
redact: Record<string, string>;
events: Array<{ tMs: number; kind: 'input' | 'note' | 'screen'; data: string }>;
t0: number;
}
/** Non-empty provider-key VALUES currently in the environment the redaction
* map for every artifact write. */
function buildRedactMap(): Record<string, string> {
const out: Record<string, string> = {};
for (const name of PROVIDER_KEY_NAMES) {
const v = process.env[name];
if (v && v.trim().length >= MIN_REDACT_SECRET_LEN) out[name] = v;
}
return out;
}
function newCtx(args: CliArgs, needsGbrain: boolean): ScenarioCtx {
const outDir = path.resolve(
args.dir ?? path.join(REPO_ROOT, '.context', 'dx-runs', `${args.scenario}-${nowStamp()}`),
@@ -217,6 +242,7 @@ function newCtx(args: CliArgs, needsGbrain: boolean): ScenarioCtx {
keep: args.keep,
cleanups: [],
secretPaths: [],
redact: buildRedactMap(),
events: [],
t0: Date.now(),
};
@@ -266,7 +292,10 @@ function finishCtx(ctx: ScenarioCtx): void {
scrubSecrets(ctx);
fs.writeFileSync(
path.join(ctx.outDir, 'events.jsonl'),
ctx.events.map((e) => JSON.stringify(e)).join('\n') + (ctx.events.length ? '\n' : ''),
redactSecrets(
ctx.events.map((e) => JSON.stringify(e)).join('\n') + (ctx.events.length ? '\n' : ''),
ctx.redact,
),
);
if (ctx.keep && ctx.secretPaths.length > 0) {
log(`--keep: retained hermetic dirs, but scrubbed ${ctx.secretPaths.length} credential file(s)`);
@@ -290,13 +319,20 @@ function finishCtx(ctx: ScenarioCtx): void {
}
/** Live session mirror so a watcher (or a Conductor agent) can follow along:
* session/screen.txt (latest visible tail) + session/status.json. */
function mirrorSession(dir: string, session: TtySession): () => void {
* session/screen.txt (latest visible tail) + session/status.json. Each tick
* strips only a bounded RAW tail (a full-buffer stripAnsi every 500ms is
* quadratic on a 25-minute session) and redacts before writing the mirror
* is a live artifact that outlives an interrupted run, so it must never
* carry a raw key even transiently. */
function mirrorSession(dir: string, session: TtySession, redact?: Record<string, string>): () => void {
const sessDir = path.join(dir, 'session');
fs.mkdirSync(sessDir, { recursive: true });
const timer = setInterval(() => {
try {
fs.writeFileSync(path.join(sessDir, 'screen.txt'), session.visible().slice(-8000));
fs.writeFileSync(
path.join(sessDir, 'screen.txt'),
redactSecrets(stripAnsi(session.raw().slice(-131_072)).slice(-8000), redact),
);
fs.writeFileSync(
path.join(sessDir, 'status.json'),
JSON.stringify(
@@ -322,6 +358,7 @@ function saveSession(ctx: ScenarioCtx, name: string, session: TtySession, extraM
saveTranscript(dir, {
frames: session.frames(),
raw: session.raw(),
redact: ctx.redact,
meta: {
scenario: name || path.basename(ctx.outDir),
argv: session.argv,
@@ -331,6 +368,53 @@ function saveSession(ctx: ScenarioCtx, name: string, session: TtySession, extraM
...extraMeta,
},
});
assertNoSecrets(dir, ctx.redact, ctx.t0);
}
/** Independent double-check of the structural redaction above: grep every
* written artifact for each raw secret value and HARD-FAIL on a hit (delete
* the leaking file, mark the run failed). A leak here means redactSecrets
* missed a rendering (e.g. ANSI-interleaved) that is a bug to fix, never
* a warning to scroll past. */
function assertNoSecrets(dir: string, redact: Record<string, string>, sinceMs: number): void {
const values = Object.entries(redact).filter(([, v]) => v && v.length >= MIN_REDACT_SECRET_LEN);
if (values.length === 0) return;
const walk = (d: string): string[] =>
fs.readdirSync(d, { withFileTypes: true }).flatMap((e) => {
const p = path.join(d, e.name);
return e.isDirectory() ? walk(p) : e.isFile() ? [p] : [];
});
let leaked = false;
for (const file of walk(dir)) {
// NEVER touch files that predate this run: --dir can point anywhere
// (repo root, even $HOME) and deleting a pre-existing .env that happens
// to contain the key would be data loss, not leak containment.
try { if (fs.statSync(file).mtimeMs < sinceMs - 1000) continue; } catch { continue; }
let body: string;
try { body = fs.readFileSync(file, 'utf8'); } catch { continue; }
// Joined-data pass for frame records: a secret split across JSONL
// records never appears contiguously in the file body.
let joinedData = '';
if (file.endsWith('.jsonl')) {
for (const line of body.split('\n')) {
try { joinedData += String(JSON.parse(line)?.data ?? ''); } catch { /* not a data record */ }
}
}
for (const [name, value] of values) {
const hit =
body.includes(value) ||
stripAnsi(body).includes(value) || // ANSI-interleaved rendering
(joinedData !== '' && (joinedData.includes(value) || stripAnsi(joinedData).includes(value)));
if (hit) {
leaked = true;
fs.rmSync(file, { force: true });
log(`SECRET LEAK: ${file} contained ${name} despite structural redaction — file deleted; fix redactSecrets coverage before trusting transcripts`);
}
}
}
if (leaked) {
process.exitCode = 1;
}
}
// ── scenario: help ───────────────────────────────────────────────────────────
@@ -376,7 +460,7 @@ async function scenarioInit(ctx: ScenarioCtx, args: CliArgs): Promise<void> {
dropEnv: args.keyless ? PROVIDER_KEY_NAMES : undefined,
timeoutMs: 600_000,
});
const stopMirror = mirrorSession(ctx.outDir, session);
const stopMirror = mirrorSession(ctx.outDir, session, ctx.redact);
const steps: string[] = [];
let lastMarkPos = 0;
@@ -429,7 +513,10 @@ async function settlePastBootDialogs(
while (Date.now() < deadline) {
await session.waitForQuiet({ quietMs: 2000, timeoutMs: 30_000 });
if (session.exited()) return;
const tail = session.visible().slice(-2500);
// Bounded strip: a repaint-heavy TUI (observed: grok's splash animation)
// grows the raw buffer by MBs — stripping the FULL buffer every
// iteration is the quadratic hot loop; strip a raw tail instead.
const tail = stripAnsi(session.raw().slice(-131_072)).slice(-2500);
if (!handled.has('trust') && /trust this ?folder/i.test(tail.replace(/\s+/g, ' '))) {
handled.add('trust');
event(ctx, 'note', 'boot dialog: workspace trust — accepted (option 1)');
@@ -456,10 +543,120 @@ async function settlePastBootDialogs(
session.sendKey('Enter');
continue;
}
// Grok Build sign-in screen (observed keyless copy: "Not signed in").
// No unattended path exists past it — record the friction and stop
// settling; the caller's wall clock must not burn on a login dialog.
if (/not signed in/i.test(tail)) {
event(ctx, 'note', 'boot dialog: grok sign-in required — no unattended path; stopping settle');
return;
}
// Quiet with no KNOWN dialog: if the tail still LOOKS like a prompt
// (numbered options / y-n / picker glyph), note it — a silently
// mis-settled boot is otherwise invisible in the audit trail.
if (/(?:^|\n)\s*(?:\d+\.\s|[❯›]\s)|\((?:y\/n|Y\/n)\)/m.test(tail.slice(-400))) {
event(ctx, 'note', 'settle: quiet with an unmatched dialog-shaped tail — proceeding (note-only; check the transcript if the paste lands oddly)');
}
return; // quiet + no dialog = at the input prompt
}
}
/** Stage the compiled gbrain into a fresh bin dir (PATH-prepend target) so
* the agent's bare `gbrain` runs this checkout. */
function stageBinDir(ctx: ScenarioCtx): string {
const binDir = tmp(ctx, 'gb-dx-bin-');
fs.copyFileSync(ctx.gbrainBin, path.join(binDir, 'gbrain'));
fs.chmodSync(path.join(binDir, 'gbrain'), 0o755);
return binDir;
}
interface InstallSessionOpts {
argv: string[];
cwd: string;
env: Record<string, string | undefined>;
extraAllow?: string[];
dropEnv?: string[];
/** The pasted prompt. Per-agent: the bootstrap runbook block for
* claude/codex, a brain-only GROK.md-driven block for grok. */
prompt: string;
/** Success copy to race against (default: the bootstrap verify patterns).
* Brain-only scenarios pass their own grok's is the doctor banner. */
successPatterns?: Array<RegExp | string>;
timeoutMs?: number;
meta: Record<string, unknown>;
}
/** The shared install-session tail the claude/codex/grok scenarios all run:
* launch mirror settle boot dialogs paste the prompt race
* verify-success copy vs exit let trailing output land save. Per-agent
* PREPARATION (TUI seeds, auth copies, git init) deliberately stays in each
* scenario the loop is what was duplicated, the prep genuinely differs. */
async function runInstallSession(ctx: ScenarioCtx, opts: InstallSessionOpts): Promise<void> {
const timeoutMs = opts.timeoutMs ?? 1_800_000;
let earlyTerminal: string | undefined;
log(`watch live: cat ${path.join(ctx.outDir, 'session', 'screen.txt')}`);
const session = launchTty(opts.argv, {
cwd: opts.cwd,
env: opts.env,
extraAllow: opts.extraAllow,
dropEnv: opts.dropEnv,
timeoutMs,
});
const stopMirror = mirrorSession(ctx.outDir, session, ctx.redact);
try {
await settlePastBootDialogs(ctx, session);
// Sign-in wall / textless splash: no unattended path exists past either.
// Observed (grok v1.0.4 keyless): headless prints "Not signed in"; the
// TUI loops a full-screen glyph animation with NO textual prompt for 8+
// minutes. Try one Enter (the common skip-splash gesture), then if the
// screen still carries no meaningful text — or shows the sign-in copy —
// record the friction (that IS the keyless measurement) and end early
// instead of pasting into a wall for the full wall clock.
const tailText = () => stripAnsi(session.raw().slice(-131_072)).slice(-2500);
// Sign-in copy, both surfaces observed (GROK-CLI-PIN.md): headless prints
// "Not signed in"; the TUI settles (~6s, after an intro animation) onto
// "Approve in your browser to finish signing in" + a device code.
const signInWall = () =>
/not signed in|approve in your browser|finish signing in|sign in with/i.test(tailText());
// Word-like text only: splash/spinner screens render Braille-pattern
// glyphs (U+2800 range, observed) with scattered digits/SGR residue but
// ZERO 3+-letter runs, while any real prompt/sign-in screen carries
// words. Counting letter runs beats enumerating glyph exceptions.
const meaningfulLen = (s: string) => (s.match(/[A-Za-z]{3,}/g) ?? []).join('').length;
if (!session.exited() && (signInWall() || meaningfulLen(tailText()) < 40)) {
event(ctx, 'note', 'sign-in wall or textless splash — sending one Enter (skip-splash attempt)');
session.sendKey('Enter');
await Bun.sleep(3000);
if (!session.exited() && (signInWall() || meaningfulLen(tailText()) < 40)) {
event(ctx, 'note', 'sign-in wall / no textual prompt persists — ending session early (keyless friction recorded)');
earlyTerminal = 'sign-in-wall-or-splash';
return; // finally owns cleanup; save happens after it
}
}
event(ctx, 'input', 'paste install prompt');
// Scope the verify match to output AFTER the paste: the pasted prompt
// itself contains verify-adjacent copy, and matching the full buffer
// re-scans a growing transcript every poll.
const pasteMark = session.mark();
session.send(opts.prompt);
await Bun.sleep(1500);
session.sendKey('Enter');
const raceBudget = Math.max(timeoutMs - 300_000, Math.floor(timeoutMs * 0.8));
const done = await Promise.race([
session
.waitForAny(opts.successPatterns ?? VERIFY_SUCCESS_PATTERNS, { timeoutMs: raceBudget, since: pasteMark })
.then(() => 'verify-signal')
.catch(() => 'no-signal'),
session.waitForExit(raceBudget).then(() => 'exited'),
]);
event(ctx, 'note', `terminal condition: ${done}`);
await session.waitForQuiet({ quietMs: 5000, timeoutMs: 60_000 });
} finally {
stopMirror();
await session.close();
}
saveSession(ctx, '', session, earlyTerminal ? { ...opts.meta, terminal: earlyTerminal } : opts.meta);
}
/** The README paste block, pointed at THIS repo's runbook, plus a persona
* appendix so the interview completes unattended. The appendix is the ONLY
* deviation from the shipped block flagged in meta so the audit discounts it. */
@@ -481,14 +678,12 @@ function installPrompt(): string {
);
}
async function scenarioClaudeInstall(ctx: ScenarioCtx): Promise<void> {
async function scenarioClaudeInstall(ctx: ScenarioCtx, args: CliArgs): Promise<void> {
const home = tmp(ctx, 'gb-dx-home-');
const cfg = tmp(ctx, 'gb-dx-ccfg-');
const gbHome = tmp(ctx, 'gb-dx-gbhome-');
const ws = tmp(ctx, 'gb-dx-ws-');
const binDir = tmp(ctx, 'gb-dx-bin-');
fs.copyFileSync(ctx.gbrainBin, path.join(binDir, 'gbrain'));
fs.chmodSync(path.join(binDir, 'gbrain'), 0o755);
const binDir = stageBinDir(ctx);
seedClaudeTuiConfig(cfg, {
apiKey: process.env.ANTHROPIC_API_KEY ?? process.env.GSTACK_ANTHROPIC_API_KEY,
@@ -501,61 +696,33 @@ async function scenarioClaudeInstall(ctx: ScenarioCtx): Promise<void> {
ctx.secretPaths.push(path.join(cfg, '.claude.json'));
log('REAL interactive claude running the paste-in bootstrap (10-25 min, real API cost)');
log(`watch live: cat ${path.join(ctx.outDir, 'session', 'screen.txt')}`);
const session = launchTty(
// --dangerously-skip-permissions: v1 measures flow + copy + stalls without
// permission-dialog babysitting. Permission-prompt COUNT is a separate
// drive-mode pass (the dialogs are Claude Code's chrome, not gbrain copy).
['claude', '--dangerously-skip-permissions'],
{
cwd: ws,
env: {
HOME: home,
CLAUDE_CONFIG_DIR: cfg,
GBRAIN_HOME: gbHome,
PATH: `${binDir}:${process.env.PATH ?? ''}`,
},
timeoutMs: 1_800_000,
await runInstallSession(ctx, {
// dangerously-skip-permissions (spelled dash-free here): v1 measures flow
// + copy + stalls without permission-dialog babysitting. Permission-prompt
// COUNT is a separate drive-mode pass (the dialogs are Claude Code's
// chrome, not gbrain copy).
argv: ['claude', '--dangerously-skip-permissions'],
cwd: ws,
env: {
HOME: home,
CLAUDE_CONFIG_DIR: cfg,
GBRAIN_HOME: gbHome,
PATH: `${binDir}:${process.env.PATH ?? ''}`,
},
dropEnv: args.keyless ? PROVIDER_KEY_NAMES : undefined,
prompt: installPrompt(),
meta: {
promptDeviation: 'unattended persona appendix + local runbook path + preinstalled binary',
runbook: 'BOOTSTRAP_FOR_AGENTS.md (local)',
},
);
const stopMirror = mirrorSession(ctx.outDir, session);
try {
// Get past first-run chrome (trust dialog, bypass warning), then paste.
await settlePastBootDialogs(ctx, session);
event(ctx, 'input', 'paste install prompt');
session.send(installPrompt());
await Bun.sleep(1500);
session.sendKey('Enter');
// Run until verify-success copy or exit or wall clock.
const done = await Promise.race([
session
.waitForAny(VERIFY_SUCCESS_PATTERNS, {
timeoutMs: 1_500_000,
})
.then(() => 'verify-signal')
.catch(() => 'no-signal'),
session.waitForExit(1_500_000).then(() => 'exited'),
]);
event(ctx, 'note', `terminal condition: ${done}`);
// Let trailing output land.
await session.waitForQuiet({ quietMs: 5000, timeoutMs: 60_000 });
} finally {
stopMirror();
await session.close();
}
saveSession(ctx, '', session, {
promptDeviation: 'unattended persona appendix + local runbook path + preinstalled binary',
runbook: 'BOOTSTRAP_FOR_AGENTS.md (local)',
});
}
async function scenarioCodexInstall(ctx: ScenarioCtx): Promise<void> {
async function scenarioCodexInstall(ctx: ScenarioCtx, args: CliArgs): Promise<void> {
const home = tmp(ctx, 'gb-dx-home-');
const gbHome = tmp(ctx, 'gb-dx-gbhome-');
const ws = tmp(ctx, 'gb-dx-ws-');
const binDir = tmp(ctx, 'gb-dx-bin-');
fs.copyFileSync(ctx.gbrainBin, path.join(binDir, 'gbrain'));
fs.chmodSync(path.join(binDir, 'gbrain'), 0o755);
const binDir = stageBinDir(ctx);
// Hermetic ~/.codex with ONLY the operator's auth (same posture as the
// codex door test). codex refuses untrusted cwds — a git repo satisfies it.
@@ -573,46 +740,152 @@ async function scenarioCodexInstall(ctx: ScenarioCtx): Promise<void> {
spawnSync('git', ['-C', ws, 'config', 'user.name', 'DX Explore']);
log('REAL interactive codex running the paste-in bootstrap (10-25 min, real API cost)');
log(`watch live: cat ${path.join(ctx.outDir, 'session', 'screen.txt')}`);
const session = launchTty(
['codex', '--sandbox', 'workspace-write', '--ask-for-approval', 'never'],
{
cwd: ws,
env: {
HOME: home,
CODEX_HOME: codexHome,
GBRAIN_HOME: gbHome,
PATH: `${binDir}:${process.env.PATH ?? ''}`,
},
extraAllow: ['OPENAI_API_KEY', 'CODEX_*'],
timeoutMs: 1_800_000,
await runInstallSession(ctx, {
argv: ['codex', '--sandbox', 'workspace-write', '--ask-for-approval', 'never'],
cwd: ws,
env: {
HOME: home,
CODEX_HOME: codexHome,
GBRAIN_HOME: gbHome,
PATH: `${binDir}:${process.env.PATH ?? ''}`,
},
extraAllow: ['OPENAI_API_KEY', 'CODEX_*'],
dropEnv: args.keyless ? PROVIDER_KEY_NAMES : undefined,
prompt: installPrompt(),
meta: {
promptDeviation: 'unattended persona appendix + local runbook path + preinstalled binary',
runbook: 'BOOTSTRAP_FOR_AGENTS.md (local)',
},
});
}
// ── scenario: grok-install ───────────────────────────────────────────────────
/** Brain-only success copy for the grok scenario: the doctor banner proves
* the registration handshook (observed, docs/mcp/GROK-CLI-PIN.md) grok has
* NO bootstrap path, so the bootstrap verify patterns must not be its bar. */
const GROK_INSTALL_SUCCESS_PATTERNS: Array<RegExp | string> = [
/7 tools discovered/i,
/handshake OK/i,
];
/** GROK.md-driven brain-only prompt deliberately NOT the bootstrap paste
* block: the docs classify grok as brain-only install (no `gbrain bootstrap`
* support), so the scenario must not test an unsupported flow. */
function grokInstallPrompt(): string {
const guide = path.join(REPO_ROOT, 'docs', 'mcp', 'GROK.md');
return (
`Read and follow: ${guide}\n` +
`Goal: wire the gbrain memory brain into you (Grok Build) over stdio MCP — ` +
`brain-only install, no bootstrap. Steps: ` +
`1) run \`gbrain init --pglite --no-embedding --non-interactive\`; ` +
`2) register gbrain exactly as the guide's Register section shows; ` +
`3) verify with \`grok mcp doctor gbrain\` — you are not done until it reports ` +
`the tools-discovered check passing; ` +
`4) one recall round-trip: use the gbrain remember tool to store ` +
`"${PERSONA.AGENT_NAME} prefers ${PERSONA.VOICE_REGISTER}" and then recall it.\n\n` +
`[Unattended-run appendix — I am stepping away; do not wait for my input. ` +
`gbrain is already installed and on PATH. If a step needs auth that is ` +
`unavailable, note the exact error and stop.]`
);
const stopMirror = mirrorSession(ctx.outDir, session);
try {
await settlePastBootDialogs(ctx, session);
event(ctx, 'input', 'paste install prompt');
session.send(installPrompt());
await Bun.sleep(1500);
session.sendKey('Enter');
const done = await Promise.race([
session
.waitForAny(VERIFY_SUCCESS_PATTERNS, {
timeoutMs: 1_500_000,
})
.then(() => 'verify-signal')
.catch(() => 'no-signal'),
session.waitForExit(1_500_000).then(() => 'exited'),
]);
event(ctx, 'note', `terminal condition: ${done}`);
await session.waitForQuiet({ quietMs: 5000, timeoutMs: 60_000 });
} finally {
stopMirror();
await session.close();
}
saveSession(ctx, '', session, {
promptDeviation: 'unattended persona appendix + local runbook path + preinstalled binary',
runbook: 'BOOTSTRAP_FOR_AGENTS.md (local)',
}
async function scenarioGrokInstall(ctx: ScenarioCtx, args: CliArgs): Promise<void> {
const home = tmp(ctx, 'gb-dx-home-');
const gbHome = tmp(ctx, 'gb-dx-gbhome-');
const ws = tmp(ctx, 'gb-dx-ws-');
const binDir = stageBinDir(ctx);
// Hermetic ~/.grok seeded with the auto-update kill-switch (config-only
// mechanism, default ON — observed v1.0.4). Auth travels via XAI_API_KEY
// env only; grok MAY persist derived credentials after an authed session,
// so the known candidate is pre-registered for the scrub (rm of a file
// that never appears is a no-op).
const grokHome = path.join(home, '.grok');
fs.mkdirSync(grokHome, { recursive: true });
// Verbatim kill-switch homes (update together): seedGrokConfig in
// test/helpers/agent-harness.ts and the grok-door auth-preflight printf
// in .github/workflows/heavy-tests.yml.
fs.writeFileSync(path.join(grokHome, 'config.toml'), '[cli]\nauto_update = false\n');
ctx.secretPaths.push(path.join(grokHome, 'mcp_credentials.json'));
log('REAL interactive grok running the GROK.md brain-only install (real API cost when authed)');
log('keyless runs stop at the observed sign-in screen — that friction IS the measurement');
await runInstallSession(ctx, {
argv: ['grok'],
cwd: ws,
env: {
HOME: home,
GROK_HOME: grokHome,
GBRAIN_HOME: gbHome,
PATH: `${binDir}:${process.env.PATH ?? ''}`,
// Never let a keyless first-run bounce the OPERATOR's browser for
// sign-in; the settle loop stops at the observed "Not signed in" copy.
BROWSER: '/usr/bin/false',
},
extraAllow: ['XAI_API_KEY'],
// --keyless drops provider keys AFTER extraAllow re-admission — a keyless
// grok-install must measure the sign-in wall, not silently run authed.
dropEnv: args.keyless ? PROVIDER_KEY_NAMES : undefined,
prompt: grokInstallPrompt(),
successPatterns: GROK_INSTALL_SUCCESS_PATTERNS,
meta: {
promptDeviation: 'unattended appendix + local GROK.md path + preinstalled binary',
runbook: 'docs/mcp/GROK.md (local, brain-only — grok has no bootstrap path)',
},
});
}
// ── scenario: opencode-install ───────────────────────────────────────────────
async function scenarioOpencodeInstall(ctx: ScenarioCtx, args: CliArgs): Promise<void> {
const home = tmp(ctx, 'gb-dx-home-');
const gbHome = tmp(ctx, 'gb-dx-gbhome-');
const ws = tmp(ctx, 'gb-dx-ws-');
const binDir = stageBinDir(ctx);
// Hermetic HOME + BOTH XDG dirs (config/auth/data all move — observed
// v1.18.18, OPENCODE-CLI-PIN.md §Path seams), seeded with the config half
// of the double autoupdate kill; the env half rides the session env below.
const xdgConfig = path.join(home, '.config');
const ocCfgDir = path.join(xdgConfig, 'opencode');
fs.mkdirSync(ocCfgDir, { recursive: true });
fs.writeFileSync(
path.join(ocCfgDir, 'opencode.json'),
JSON.stringify({ $schema: 'https://opencode.ai/config.json', autoupdate: false }, null, 2) + '\n',
);
// Auth travels env-only for the anthropic leg; a login flow would persist
// auth.json — pre-register the known candidate for the scrub (rm of a file
// that never appears is a no-op).
ctx.secretPaths.push(path.join(home, '.local', 'share', 'opencode', 'auth.json'));
spawnSync('git', ['init', '-q', ws]);
spawnSync('git', ['-C', ws, 'config', 'user.email', 'dx@example.com']);
spawnSync('git', ['-C', ws, 'config', 'user.name', 'DX Explore']);
log('REAL interactive opencode running the paste-in bootstrap (opencode is a bootstrap-supported harness)');
log('keyless runs ride the anonymous free tier (observed) — the flow should COMPLETE keyless; a sign-in wall here is itself a pin-refresh signal');
await runInstallSession(ctx, {
argv: ['opencode'],
cwd: ws,
env: {
HOME: home,
XDG_CONFIG_HOME: xdgConfig,
XDG_DATA_HOME: path.join(home, '.local', 'share'),
OPENCODE_DISABLE_AUTOUPDATE: '1',
GBRAIN_HOME: gbHome,
PATH: `${binDir}:${process.env.PATH ?? ''}`,
// Never let a first-run bounce the OPERATOR's browser for sign-in.
BROWSER: '/usr/bin/false',
},
extraAllow: ['ANTHROPIC_API_KEY'],
// --keyless drops provider keys AFTER extraAllow re-admission — on
// opencode that measures the FREE-TIER path, not a wall (observed).
dropEnv: args.keyless ? PROVIDER_KEY_NAMES : undefined,
prompt: installPrompt(),
meta: {
promptDeviation: 'unattended persona appendix + local runbook path + preinstalled binary',
runbook: 'BOOTSTRAP_FOR_AGENTS.md (local — opencode is bootstrap-supported)',
},
});
}
@@ -650,7 +923,7 @@ async function scenarioDrive(ctx: ScenarioCtx, args: CliArgs): Promise<void> {
env,
timeoutMs: 3_600_000,
});
const stopMirror = mirrorSession(ctx.outDir, session);
const stopMirror = mirrorSession(ctx.outDir, session, ctx.redact);
let offset = 0;
let stopping = false;
@@ -701,10 +974,16 @@ const SCENARIOS: Record<string, { needsGbrain: boolean; run: (ctx: ScenarioCtx,
init: { needsGbrain: true, run: scenarioInit },
'claude-install': { needsGbrain: true, run: scenarioClaudeInstall },
'codex-install': { needsGbrain: true, run: scenarioCodexInstall },
'grok-install': { needsGbrain: true, run: scenarioGrokInstall },
'opencode-install': { needsGbrain: true, run: scenarioOpencodeInstall },
drive: { needsGbrain: true, run: scenarioDrive },
};
async function main(): Promise<void> {
if (!ptySupported()) {
log('this Bun lacks PTY (terminal:) support — upgrade Bun (engines.bun in package.json) before running dx scenarios');
process.exit(2);
}
const args = parseArgs(process.argv.slice(2));
const scenario = SCENARIOS[args.scenario];
if (!scenario) {
+6
View File
@@ -156,6 +156,12 @@ export function renderRegistryModule(registry: Record<string, string[]>): string
// Regenerate: bun run build:flag-registry
// Freshness + drift pinned by test/cli-flag-validation.test.ts (#2185).
//
// Merge conflict here? Do not hand-merge it. This file is regenerated on most
// upstream waves, so a branch that also regenerates it conflicts on the whole
// body. Take the base branch's copy wholesale, then re-run the command above —
// the freshness test named above fails loudly if that regeneration was done
// against the wrong base.
//
// Per-command legal flags for CLI_ONLY commands, derived from each command's
// source (case block + imported modules + one level of relative imports +
// scripts/generate-flag-registry.ts EXTRA_FLAGS). Deliberately over-inclusive
+94
View File
@@ -0,0 +1,94 @@
#!/usr/bin/env bash
# W0 fix-wave (Tier-1 #11 / D5.14): guard self-test harness.
#
# The audit found scripts/check-no-double-retry.sh had been PERMANENTLY GREEN
# since it shipped: its regex could not match the canonical banned shape, and
# its multi-line fallback was gated on pcregrep, which is installed nowhere.
# A guard that cannot fail is worse than no guard — it reads as coverage.
#
# This harness makes that class structurally impossible for scanner guards:
# every guard marked `selftest yes` in scripts/guards-manifest.tsv is run
# against test/fixtures/guards/<guard>/bad (MUST exit non-zero) and
# .../good (MUST exit 0), via the GBRAIN_GUARD_ROOT override each guard
# honors. Adding a self-test to a `todo` scanner = flip the manifest flag +
# drop two fixture files.
#
# Also prints total harness wall-clock (guard-runtime budget line, D4.5):
# fails if the self-test pass exceeds the budget, so guard sprawl shows up
# here before it shows up as slow `bun run verify`.
#
# Usage: scripts/guard-self-test.sh
# Exit: 0 = every self-tested guard fails on bad + passes on good.
set -uo pipefail
ROOT="$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
cd "$ROOT"
MANIFEST="scripts/guards-manifest.tsv"
FIXTURES="test/fixtures/guards"
BUDGET_SECONDS=30
START=$(date +%s)
failures=0
tested=0
if [ ! -f "$MANIFEST" ]; then
echo "ERROR: $MANIFEST missing — the guard registry is load-bearing."
exit 1
fi
run_guard() {
local guard="$1" fixture_root="$2"
case "$guard" in
*.mjs) GBRAIN_GUARD_ROOT="$fixture_root" node "scripts/$guard" "$fixture_root" >/dev/null 2>&1 ;;
*) GBRAIN_GUARD_ROOT="$fixture_root" bash "scripts/$guard" >/dev/null 2>&1 ;;
esac
}
while IFS=$'\t' read -r guard klass selftest _notes; do
case "$guard" in ''|'#'*) continue ;; esac
[ "$selftest" = "yes" ] || continue
tested=$((tested + 1))
bad="$FIXTURES/$guard/bad"
good="$FIXTURES/$guard/good"
if [ ! -d "$bad" ] || [ ! -d "$good" ]; then
echo "FAIL $guard: manifest says selftest=yes but fixtures missing under $FIXTURES/$guard/{bad,good}"
failures=$((failures + 1))
continue
fi
if run_guard "$guard" "$bad"; then
echo "FAIL $guard: did NOT flag the known-bad fixture — the guard is a no-op (the check-no-double-retry class)"
failures=$((failures + 1))
elif ! run_guard "$guard" "$good"; then
echo "FAIL $guard: flagged the known-good fixture — false positive"
failures=$((failures + 1))
else
echo "ok $guard (bad→fail, good→pass)"
fi
done < "$MANIFEST"
# Manifest completeness: every scripts/check-* guard must have a manifest row
# (new guards can't silently skip classification).
for f in scripts/check-*.sh scripts/check-*.mjs; do
base="$(basename "$f")"
# The .ts companion of check-engine-dynamic-import is an implementation file.
if ! grep -q "^${base} " "$MANIFEST"; then
echo "FAIL $base: no row in $MANIFEST — classify it (scanner|buildfresh|repostate)"
failures=$((failures + 1))
fi
done
ELAPSED=$(( $(date +%s) - START ))
echo "guard self-test: $tested guard(s) self-tested, ${ELAPSED}s (budget ${BUDGET_SECONDS}s)"
if [ "$ELAPSED" -gt "$BUDGET_SECONDS" ]; then
echo "FAIL guard self-test exceeded the ${BUDGET_SECONDS}s runtime budget — trim fixtures or parallelize before adding more"
failures=$((failures + 1))
fi
if [ "$failures" -gt 0 ]; then
echo "ERROR: $failures guard self-test failure(s)."
exit 1
fi
echo "OK: all self-tested guards catch their bad fixtures and pass their good ones"
+65
View File
@@ -0,0 +1,65 @@
# CI guard registry (W0 fix-wave, Tier-1 #11 / D5.14).
# THE single registry of scripts/check-* guards. package.json's `check:all`
# (a second, stale, hand-synced copy) was deleted; scripts/run-verify-parallel.sh
# executes guards, and scripts/guard-self-test.sh consumes THIS file to
# self-test every scanner guard against known-bad/known-good fixtures under
# test/fixtures/guards/<guard>/{bad,good}/ (env: GBRAIN_GUARD_ROOT).
#
# class: scanner = greps/parses repo sources; MUST eventually carry fixtures
# (selftest yes|todo). A scanner guard with selftest=todo is
# tracked debt — the class that produced two permanently-
# green guards (check-no-double-retry, pcregrep-gated pass).
# buildfresh = runs builds/regenerators and diffs outputs; self-tests
# don't apply (the build IS the test). exempt.
# repostate = checks repo/file state (modes, symlinks, VERSION stamps);
# exempt with reason.
#
# guard class selftest notes
check-no-double-retry.sh scanner yes regex hole fixed in W0 (could not match `() =>`); perl multi-line pass replaces never-installed pcregrep
check-jsonb-pattern.sh scanner yes nested-paren hole fixed in W0; safe ::text::jsonb spelling stays unflagged
check-jsonb-params.mjs scanner yes positional $N::jsonb AST-lite scanner; argv/env root override
check-batch-audit-site.sh scanner todo
check-bun-test-timeout.sh scanner todo
check-fixture-privacy.sh scanner todo
check-no-legacy-getconnection.sh scanner todo was reachable from neither verify nor CI pre-W0 (check:all only)
check-no-pii-in-agent-voice.sh scanner todo
check-operations-filter-bypass.sh scanner todo
check-pagetype-exhaustive.sh scanner todo
check-pg-url-redaction.sh scanner todo
check-privacy.sh scanner todo
check-progress-to-stdout.sh scanner todo
check-proposal-pii.sh scanner todo
check-search-path.sh scanner todo
check-skill-brain-first.sh scanner todo
check-skill-refs.mjs scanner todo
check-source-config-leak.sh scanner todo
check-source-id-projection.sh scanner todo
check-source-scope-onboard.sh scanner todo
check-synthetic-corpus-privacy.sh scanner todo
check-system-of-record.sh scanner todo
check-test-real-names.sh scanner todo
check-worker-lock-renewal-shape.sh scanner todo
check-worker-pool-atomicity.sh scanner todo
check-gateway-routed-no-direct-anthropic.sh scanner todo
check-engine-dynamic-import.sh scanner todo .ts companion is its implementation, not a separate guard
check-key-files-current-state.sh scanner todo
check-exports-count.sh scanner todo was reachable from neither verify nor CI pre-W0 (check:all only)
check-trailing-newline.sh scanner todo was reachable from neither verify nor CI pre-W0 (check:all only)
check-test-isolation.sh scanner todo allowlist data file: check-test-isolation.allowlist
check-admin-build.sh buildfresh exempt runs the admin build; the build is the test
check-admin-embedded.sh buildfresh exempt embed freshness diff
check-admin-scope-drift.sh buildfresh exempt regenerates + diffs
check-bootstrap-templates.sh buildfresh exempt regenerates template tree + diffs
check-eval-glossary-fresh.sh buildfresh exempt regenerates + diffs
check-fuzz-purity.sh buildfresh exempt executes fuzz corpus
check-image-decoders-embedded.sh buildfresh exempt binary embed check
check-pglite-embedded.sh buildfresh exempt binary embed check
check-skills-manifest-fresh.sh buildfresh exempt regenerates + diffs
check-tool-catalog-fresh.sh buildfresh exempt regenerates + diffs
check-wasm-embedded.sh buildfresh exempt binary embed check
check-bootstrap-tag.sh repostate exempt VERSION stamp drift check
check-cli-executable.sh repostate exempt file-mode check
check-no-tracked-symlinks.sh repostate exempt git index state check
check-grok-pin.sh repostate exempt pin-stamp drift check (GROK-CLI-PIN.md stamps vs heavy-tests grok-door env); own bun guard tests in test/check-bootstrap-guards.test.ts
check-opencode-pin.sh repostate exempt pin-stamp drift check (OPENCODE-CLI-PIN.md stamps vs heavy-tests opencode-door env); own bun guard tests in test/check-bootstrap-guards.test.ts
check-pin-doc-privacy.sh repostate exempt PIN-doc placeholder discipline (no operator paths/key material/emails in docs/mcp/*-CLI-PIN.md); own bun guard tests in test/check-bootstrap-guards.test.ts
1 # CI guard registry (W0 fix-wave, Tier-1 #11 / D5.14).
2 # THE single registry of scripts/check-* guards. package.json's `check:all`
3 # (a second, stale, hand-synced copy) was deleted; scripts/run-verify-parallel.sh
4 # executes guards, and scripts/guard-self-test.sh consumes THIS file to
5 # self-test every scanner guard against known-bad/known-good fixtures under
6 # test/fixtures/guards/<guard>/{bad,good}/ (env: GBRAIN_GUARD_ROOT).
7 #
8 # class: scanner = greps/parses repo sources; MUST eventually carry fixtures
9 # (selftest yes|todo). A scanner guard with selftest=todo is
10 # tracked debt — the class that produced two permanently-
11 # green guards (check-no-double-retry, pcregrep-gated pass).
12 # buildfresh = runs builds/regenerators and diffs outputs; self-tests
13 # don't apply (the build IS the test). exempt.
14 # repostate = checks repo/file state (modes, symlinks, VERSION stamps);
15 # exempt with reason.
16 #
17 # guard class selftest notes
18 check-no-double-retry.sh scanner yes regex hole fixed in W0 (could not match `() =>`); perl multi-line pass replaces never-installed pcregrep
19 check-jsonb-pattern.sh scanner yes nested-paren hole fixed in W0; safe ::text::jsonb spelling stays unflagged
20 check-jsonb-params.mjs scanner yes positional $N::jsonb AST-lite scanner; argv/env root override
21 check-batch-audit-site.sh scanner todo
22 check-bun-test-timeout.sh scanner todo
23 check-fixture-privacy.sh scanner todo
24 check-no-legacy-getconnection.sh scanner todo was reachable from neither verify nor CI pre-W0 (check:all only)
25 check-no-pii-in-agent-voice.sh scanner todo
26 check-operations-filter-bypass.sh scanner todo
27 check-pagetype-exhaustive.sh scanner todo
28 check-pg-url-redaction.sh scanner todo
29 check-privacy.sh scanner todo
30 check-progress-to-stdout.sh scanner todo
31 check-proposal-pii.sh scanner todo
32 check-search-path.sh scanner todo
33 check-skill-brain-first.sh scanner todo
34 check-skill-refs.mjs scanner todo
35 check-source-config-leak.sh scanner todo
36 check-source-id-projection.sh scanner todo
37 check-source-scope-onboard.sh scanner todo
38 check-synthetic-corpus-privacy.sh scanner todo
39 check-system-of-record.sh scanner todo
40 check-test-real-names.sh scanner todo
41 check-worker-lock-renewal-shape.sh scanner todo
42 check-worker-pool-atomicity.sh scanner todo
43 check-gateway-routed-no-direct-anthropic.sh scanner todo
44 check-engine-dynamic-import.sh scanner todo .ts companion is its implementation, not a separate guard
45 check-key-files-current-state.sh scanner todo
46 check-exports-count.sh scanner todo was reachable from neither verify nor CI pre-W0 (check:all only)
47 check-trailing-newline.sh scanner todo was reachable from neither verify nor CI pre-W0 (check:all only)
48 check-test-isolation.sh scanner todo allowlist data file: check-test-isolation.allowlist
49 check-admin-build.sh buildfresh exempt runs the admin build; the build is the test
50 check-admin-embedded.sh buildfresh exempt embed freshness diff
51 check-admin-scope-drift.sh buildfresh exempt regenerates + diffs
52 check-bootstrap-templates.sh buildfresh exempt regenerates template tree + diffs
53 check-eval-glossary-fresh.sh buildfresh exempt regenerates + diffs
54 check-fuzz-purity.sh buildfresh exempt executes fuzz corpus
55 check-image-decoders-embedded.sh buildfresh exempt binary embed check
56 check-pglite-embedded.sh buildfresh exempt binary embed check
57 check-skills-manifest-fresh.sh buildfresh exempt regenerates + diffs
58 check-tool-catalog-fresh.sh buildfresh exempt regenerates + diffs
59 check-wasm-embedded.sh buildfresh exempt binary embed check
60 check-bootstrap-tag.sh repostate exempt VERSION stamp drift check
61 check-cli-executable.sh repostate exempt file-mode check
62 check-no-tracked-symlinks.sh repostate exempt git index state check
63 check-grok-pin.sh repostate exempt pin-stamp drift check (GROK-CLI-PIN.md stamps vs heavy-tests grok-door env); own bun guard tests in test/check-bootstrap-guards.test.ts
64 check-opencode-pin.sh repostate exempt pin-stamp drift check (OPENCODE-CLI-PIN.md stamps vs heavy-tests opencode-door env); own bun guard tests in test/check-bootstrap-guards.test.ts
65 check-pin-doc-privacy.sh repostate exempt PIN-doc placeholder discipline (no operator paths/key material/emails in docs/mcp/*-CLI-PIN.md); own bun guard tests in test/check-bootstrap-guards.test.ts
+26 -6
View File
@@ -36,6 +36,16 @@ set -euo pipefail
cd "$(dirname "$0")/.."
# #3485: this wrapper IS the e2e boundary — opt in to running with a database
# URL present. The bunfig test preload (database-url-guard-preload.ts) refuses
# bare `bun test` runs while DATABASE_URL/GBRAIN_DATABASE_URL is ambient; the
# per-file name floor (test/helpers/db-guard.ts) still applies after this.
export GBRAIN_TEST_ALLOW_DATABASE_URL=1
# The e2e suite runs on DATABASE_URL only; an ambient GBRAIN_DATABASE_URL
# would pass the opt-in yet reach CLI-subprocess paths with no name floor —
# drop it here so only the floored variable crosses the boundary.
unset GBRAIN_DATABASE_URL
# --- HOME isolation: snapshot real user config before switching ---
# Tolerate unset HOME (minimal containers, exotic CI shells) without tripping set -u.
REAL_HOME="${HOME:-/tmp}"
@@ -76,13 +86,20 @@ mkdir -p "$E2E_TMP_HOME/.gbrain"
# (not an allowlist rebuild), so PATH, HOME, TMPDIR, CI, DATABASE_URL, and bun
# internals survive untouched. We keep GBRAIN_HOME (just set above for HOME
# isolation); everything else GBRAIN_* is an operator override the suite must
# not inherit — which also scrubs GBRAIN_REAL_HERMES_E2E, so the paid hermes
# door suite structurally cannot fire under this runner (its venue is
# heavy-tests.yml's direct bun test). Adapts GStack's buildHermeticEnv()
# allowlist to gbrain's shell E2E runner.
for _e2e_var in $(env | grep -oE '^(CONDUCTOR_|MCP_|OPENCLAW_|HERMES_|GBRAIN_)[A-Za-z0-9_]*' | sort -u); do
# not inherit — which also scrubs GBRAIN_REAL_HERMES_E2E and
# GBRAIN_REAL_GROK_E2E / GBRAIN_REAL_OPENCODE_E2E, so the real-agent door
# suites structurally
# cannot fire under this runner (their venue is heavy-tests.yml's direct bun
# test). GROK_ also drops an operator's GROK_BIN/GROK_HOME; OPENCODE_ drops
# OPENCODE_BIN and the OPENCODE_CONFIG* trio. Adapts GStack's
# buildHermeticEnv() allowlist to gbrain's shell E2E runner.
for _e2e_var in $(env | grep -oE '^(CONDUCTOR_|MCP_|OPENCLAW_|HERMES_|GROK_|OPENCODE_|GBRAIN_)[A-Za-z0-9_]*' | sort -u); do
case "$_e2e_var" in
GBRAIN_HOME) ;; # required for HOME isolation (set above) — keep
GBRAIN_TEST_ALLOW_DATABASE_URL) ;; # #3485 preload opt-in (set above) — keep
GBRAIN_E2E_ALLOW_DB) ;; # #3485 name-floor opt-in — the guard's own error
# message tells operators to set it; stripping it
# here would make that escape hatch a dead end
*) unset "$_e2e_var" || true ;;
esac
done
@@ -99,7 +116,10 @@ fi
if [ "$#" -gt 0 ]; then
files=("$@")
else
files=(test/e2e/*.test.ts)
# phantom-redirect lives in test/ (its PGLite arm runs in the unit suite) but
# its Postgres arm is only reachable through a DATABASE_URL-bearing lane —
# the unit wrappers strip the URL (#3485), so this lane must carry it.
files=(test/e2e/*.test.ts test/phantom-redirect-engine-parity.test.ts)
fi
# SHARD env (e.g. SHARD=1/4) keeps every M-th file starting at index N (1-indexed).
+4
View File
@@ -16,6 +16,10 @@
set -euo pipefail
cd "$(dirname "$0")/.."
# #3485: the heavy lane runs destructive shell scripts against DATABASE_URL —
# apply the shared name floor once here for every script it dispatches.
source tests/heavy/_db_floor.sh
PATTERN="${1:-}"
heavy_files=()
+5
View File
@@ -4,6 +4,11 @@
# CI runs both; bun run ci:local skips slow tests via run-unit-shard.sh.
set -euo pipefail
# #3485: unit/slow tests need no database — strip ambient DB URLs at this
# wrapper boundary so the bunfig preload guard passes and nothing can reach a
# real brain. The e2e wrapper (run-e2e.sh) is the only lane that keeps them.
unset DATABASE_URL GBRAIN_DATABASE_URL
cd "$(dirname "$0")/.."
slow_files=()
+24
View File
@@ -44,8 +44,32 @@
set -uo pipefail
# #3485: unit tests need no database — strip ambient DB URLs at this wrapper
# boundary so the bunfig preload guard passes and nothing can reach a real
# brain. The e2e wrapper (run-e2e.sh) is the only lane that keeps them.
unset DATABASE_URL GBRAIN_DATABASE_URL
cd "$(dirname "$0")/.."
# ──────────────────────────────────────────────────────────────────────────
# W0 fix-wave (Tier-1 #16): PGLite schema snapshot, DEFAULT-ON for the plain
# `bun run test` loop. 500+ test files each cold-boot PGLite + replay 126
# migrations without it; the fixture was previously enabled ONLY inside
# scripts/ci-local.sh, so the everyday loop paid the full cost. The build
# script is idempotent (hash short-circuit) and concurrency-safe (mkdir
# lock, D5.8), and its hash folds handler-migration source (D5.13), so an
# unconditional call here is cheap and always current. Runs BEFORE the shard
# fan-out — shards inherit a finished fixture. Opt out: GBRAIN_NO_SNAPSHOT=1
# (the migration-replay canary tests clear the env themselves regardless).
# ──────────────────────────────────────────────────────────────────────────
if [ "${GBRAIN_NO_SNAPSHOT:-0}" != "1" ]; then
if bun run build:pglite-snapshot >/dev/null 2>&1; then
export GBRAIN_PGLITE_SNAPSHOT=test/fixtures/pglite-snapshot.tar
else
echo "[run-unit-parallel] snapshot build failed (non-fatal) — tests run with cold init" >&2
fi
fi
# ──────────────────────────────────────────────────────────────────────────
# CPU detection: Apple Silicon perf cores → Mac total physical → nproc → 4.
# Returns a single positive integer.
+5
View File
@@ -14,6 +14,11 @@
set -euo pipefail
# #3485: unit/slow tests need no database — strip ambient DB URLs at this
# wrapper boundary so the bunfig preload guard passes and nothing can reach a
# real brain. The e2e wrapper (run-e2e.sh) is the only lane that keeps them.
unset DATABASE_URL GBRAIN_DATABASE_URL
cd "$(dirname "$0")/.."
# --max-concurrency=N is forwarded to `bun test`. v0.26.4: invoked by
+11
View File
@@ -68,10 +68,21 @@ CHECKS=(
"check:no-double-retry"
"check:batch-audit-site"
"check:engine-dynamic-import"
"check:grok-pin"
"check:opencode-pin"
"check:pin-doc-privacy"
"check:worker-lock-renewal-shape"
"check:bootstrap-tag"
"check:bootstrap-templates"
"check:skill-refs"
# W0 fix-wave (Tier-1 #11): guard self-tests — every scanner guard proves it
# can fail (bad fixture → exit 1) before it counts as coverage. Registry:
# scripts/guards-manifest.tsv (package.json's stale `check:all` copy deleted).
"check:guard-self-test"
# Previously reachable ONLY from the deleted check:all (i.e. never run):
"check:newlines"
"check:exports-count"
"check:no-legacy-getconnection"
"typecheck"
)
-1
View File
@@ -103,7 +103,6 @@
"test/cli-help-discoverability.test.ts": 1670,
"test/cli-multimodal-integration.test.ts": 26677,
"test/cli-options.test.ts": 1658,
"test/cli-pty-runner.test.ts": 16,
"test/cli-query-image.test.ts": 56,
"test/cli.test.ts": 2633,
"test/code-callers-cli.test.ts": 2,
+3 -2
View File
@@ -102,11 +102,12 @@ wins; fix the row.
|---------|-------|
| "Set up GBrain", first boot | `skills/setup/SKILL.md` |
| "Now what?", "fill my brain", "cold start", "bootstrap my data", "import my data", "what should I import first" | `skills/cold-start/SKILL.md` |
| "Install gbrain into this agent/harness", "agent workspace bootstrap", "gbrain bootstrap", "wire gbrain hooks", "set up the maintenance sweep" | Run `gbrain bootstrap` (paste-in harness install: hooks + sweep + config). See `docs/guides/bootstrap.md` |
| "agent workspace bootstrap", "install gbrain into this agent workspace", "gbrain bootstrap", "paste-in install", "set up the maintenance sweep" | Run `gbrain bootstrap` (paste-in workspace install: interview + identity files + hooks + sweep). See `docs/guides/bootstrap.md` |
| "wire this box's coding agents to the brain", "framework-spawned sessions need brain access", "wire gbrain hooks without a workspace", "hook Claude Code/Codex to the running serve" | Run `gbrain bootstrap harness --yes` (machine-level wiring to a running `serve --http`: scoped token + user-scope MCP + headless pre-approval + hooks; no agent.json). See the "Local harness mode" section of `docs/guides/bootstrap.md` |
| "Migrate from Obsidian/Notion/Logseq" | `skills/migrate/SKILL.md` |
| Brain health check, maintenance run | `skills/maintain/SKILL.md` |
| "Extract links", "build link graph", "populate timeline" | `skills/maintain/SKILL.md` (extraction sections) |
| "Run dream", "process today's session", "synthesize my conversations", "consolidate yesterday's conversations", "what patterns did you see", "did the dream cycle run" | `skills/maintain/SKILL.md` (dream cycle section) |
| "Run dream", "process today's session", "synthesize my conversations", "consolidate yesterday's conversations", "what patterns did you see", "did the dream cycle run", "retriage the backlog", "re-score the triage" | `skills/maintain/SKILL.md` (dream cycle section) |
| "Brain health", "what features am I missing", "brain score" | Run `gbrain features --json` |
| "Set up autopilot", "run brain maintenance", "keep brain updated" | Run `gbrain autopilot --install --repo ~/brain` |
| "Upgrade gbrain", "update gbrain", "gbrain update available", `UPGRADE_AVAILABLE`, "is gbrain up to date" | `skills/gbrain-upgrade/SKILL.md` |
+5 -1
View File
@@ -12,7 +12,7 @@ Four tiers:
| Tier | Purpose | Default | Examples |
|---|---|---|---|
| `utility` | fast classification, expansion, verdict, dedup | `claude-haiku-4-5-20251001` | query expansion, facts contradiction classifier, dream synthesize verdict |
| `utility` | fast classification, expansion, verdict, dedup | `claude-haiku-4-5-20251001` | query expansion, facts contradiction classifier, dream triage judge (prefers `models.dream.triage`) |
| `reasoning` | default chat, synthesis, generation | `claude-sonnet-4-6` | gateway chat, dream synthesize, patterns, facts extraction |
| `deep` | slow, expensive reasoning | `claude-opus-4-7` | `gbrain think`, auto-think, cross-modal eval slot B |
| `subagent` | Anthropic-only multi-turn tool loop | `claude-sonnet-4-6` | `gbrain agent run` |
@@ -28,6 +28,10 @@ Override priority (highest first):
7. Tier default (the table above)
8. Hardcoded caller fallback
One exception: the dream triage judge pre-reads `models.dream.triage` first —
when that key is set, it wins over this entire chain (`gbrain models` reports
it as the effective route).
Power-user recipes:
```bash
+28 -8
View File
@@ -49,9 +49,11 @@ upstream: conversation-history+transcript-save@fc834ee
Two halves of one loop:
1. **IMPORT** — raw export or session log → one dated markdown page per
conversation under `conversations/``gbrain import`/`gbrain sync`
parser validation → fact extraction → gap check.
1. **IMPORT** — raw export or session log → dated markdown pages under
`conversations/` (the native importer writes them directly and splits
long sessions into parts; the manual path converts one page per
conversation, then `gbrain import`/`gbrain sync`) → parser validation →
fact extraction → gap check.
2. **RETRIEVE** — search the archive, pull threads, build timelines, and
answer "when did I first discuss X".
@@ -59,11 +61,29 @@ Years of AI-assistant history is one of the largest personal corpora most
users own. This skill makes it first-class brain content instead of a JSON
blob in a downloads folder.
**No native raw-export importer exists.** `gbrain import <dir>` ingests
markdown directories; nothing in the CLI parses a provider's raw
`conversations.json` directly. The conversion step below is agent work.
(A native `gbrain import --format chatgpt|claude` is a filed follow-up; until
it lands, this procedure is the supported path.)
**A native importer now exists: `gbrain transcripts ingest`.** It parses
agent session logs (Claude Code, Codex, OpenClaw, Hermes) AND extracted
consumer exports (ChatGPT `conversations.json`, Claude.ai export) directly:
detection, secret redaction, imessage-slack rendering, long-session
splitting, and idempotent re-runs are all native. Prefer it over the manual
procedure whenever the source is one of those six formats:
```
gbrain transcripts ingest ~/Downloads/conversations.json # unzip first
gbrain transcripts ingest # discover harness logs
gbrain transcripts status # found vs imported gaps
```
Native-vs-manual delta to know: the native lane redacts SECRETS (key
patterns) plus your `~/.gbrain/harvest-private-patterns.txt` regexes and
counts agent-directed imperatives into frontmatter, but broad PII detection
(names, phones, addresses) remains YOUR review pass — the manual procedure's
human scrub step still applies to sensitive corpora. Two more deltas: the
native lane caps each message at ~4K characters in the page body (readable
archive, not verbatim — the session file named in `source_uri` stays the
verbatim record), and tool/thinking traffic appears only as one-line
placeholders. Providers without a native adapter (e.g. Perplexity) keep
using the manual conversion below.
## Where Conversations Live
+28 -12
View File
@@ -18,6 +18,8 @@ triggers:
- "populate links"
- "backfill graph"
- "extract timeline entries"
- "retriage the backlog"
- "re-score the triage"
- "run dream"
- "process today's session"
- "process yesterday's transcripts"
@@ -116,7 +118,8 @@ gbrain extract timeline --dir ~/brain
### Dream cycle (v0.23): synthesize + patterns
`gbrain dream` runs the full 8-phase maintenance cycle:
`gbrain dream` runs the full maintenance cycle (core phases shown; opt-in
phases like atoms/concepts/drift slot in between):
```
lint -> backlinks -> sync -> synthesize -> extract -> patterns -> embed -> orphans
@@ -124,14 +127,25 @@ lint -> backlinks -> sync -> synthesize -> extract -> patterns -> embed -> orpha
The two new phases consolidate yesterday's conversations into long-term memory:
**Synthesize phase:** reads transcripts from `dream.synthesize.session_corpus_dir`,
runs a cheap Haiku verdict (cached in `dream_verdicts`) to filter routine
ops sessions, then fans out one Sonnet subagent per worth-processing
transcript. Each subagent writes reflections (`wiki/personal/reflections/...`),
originals (`wiki/originals/ideas/...`), and people timeline entries. The
orchestrator collects the slugs from `subagent_tool_executions` (NOT
`pages.updated_at` — that would pick up unrelated writes) and reverse-renders
each new page from DB → markdown on disk.
**Synthesize phase (two-stage cascade):** reads transcripts from
`dream.synthesize.session_corpus_dir`, then triages before it spends: a cheap
utility-tier judge (`models.dream.triage`) scores every new file 01 for
salience and pre-extracts candidate quotes + entities, cached in
`dream_verdicts` with the judging model + prompt version (bounded per cycle
by `dream.triage.max_ms`, default 5 min — deferred files retry next cycle,
never silently rejected). Only files scoring
at or above `dream.triage.threshold` (default 0.5 — applied at read time, so
retuning the threshold re-gates with zero new LLM calls) fan out one synthesis
subagent per transcript chunk, each primed with the triage map and capped at
`dream.synthesize.max_turns` (default 16). Each subagent writes reflections
(`wiki/personal/reflections/...`), originals (`wiki/originals/ideas/...`), and
people timeline entries. The orchestrator collects the slugs from
`subagent_tool_executions` (NOT `pages.updated_at` — that would pick up
unrelated writes) and reverse-renders each new page from DB → markdown on
disk. To re-apply the gate after retuning the threshold or drain a queued
backlog, run `gbrain dream retriage --dry-run` (zero LLM calls, cached
scores only) then `gbrain dream retriage --reconcile-queue`; `--force`
re-judges everything from scratch.
**Patterns phase:** runs after `extract` (so the graph state is fresh).
Reads recent reflections within `dream.patterns.lookback_days` (default 30),
@@ -164,15 +178,17 @@ timestamp is stored in `dream.synthesize.last_completion_ts` and is written
ONLY on successful runs (not on skipped/failed). Explicit `--input` /
`--date` / `--from` / `--to` invocations bypass cooldown.
**`--dry-run` semantics:** runs the cheap Haiku significance filter (caches
verdicts) but skips the Sonnet synthesis pass. NOT zero LLM calls.
**`--dry-run` semantics:** runs the scored triage pass (judges + caches
verdicts for new files) but skips the synthesis subagents. NOT zero LLM
calls — for a zero-call preview from cached scores use
`gbrain dream retriage --dry-run` instead.
**Configure synthesize on a fresh brain:**
```bash
gbrain config set dream.synthesize.session_corpus_dir /path/to/transcripts
gbrain config set dream.synthesize.enabled true
gbrain dream --phase synthesize --dry-run --json # preview
gbrain dream # full 8-phase cycle
gbrain dream # full cycle
```
**Invocation patterns:**
+4 -1
View File
@@ -19,7 +19,10 @@ Set up GBrain from scratch. Target: working brain in under 5 minutes.
> **Installing into an agent harness?** (Claude Code, Codex, OpenClaw, etc.)
> `gbrain bootstrap` is the paste-in install path — it wires hooks, the
> maintenance sweep, and harness config in one command. See
> maintenance sweep, and harness config in one command. On a box that already
> hosts a brain + a running `gbrain serve --http` (agent-framework boxes),
> `gbrain bootstrap harness --yes` wires framework-spawned Claude Code/Codex
> sessions instead — no agent workspace needed. See
> `docs/guides/bootstrap.md`. This skill covers the brain-side setup
> (database, sync, first import); the two are complementary.
+5 -5
View File
@@ -1,5 +1,5 @@
{
"RESOLVER.md": "3d32ec5cd8c15d83b18469277a5efc269bb6b60e67f4542de5a28a76760c36cd",
"RESOLVER.md": "36b43c65a41e6fce894b9559db2bce0a53f06a99450410e498323c12e12e92bb",
"_AGENT_README.md": "62613f7f1e061576b6c1b18844f59bd35f2df96ca5c45c8c41fae0772b9ce4d3",
"_brain-filing-rules.json": "cf850df6a7425464c6d63b3ace71991cc93497fa0cc8cd21acd31883e17939c6",
"_brain-filing-rules.md": "2d2d75b7c76081c56f41b2c0a5a978c355ce957300f9b0a5575dc4079ef1f877",
@@ -48,7 +48,7 @@
"conventions/cron-via-minions.md": "badb1cd6cd825d6f1ac0b6b28cc47e5d80facc783a3e59a14146ae901ee0f933",
"conventions/cross-modal.yaml": "c012c3d72614a87b1ee698173dce2a0fb0d057a54df7aab87993c4b07fff6280",
"conventions/exec-output.md": "2bf371ac3ec4987eff7cc13cd3ea8cc97c46bd43f588eec024ff27f3171bc58f",
"conventions/model-routing.md": "fb7ae8746a578500d6789b68ff40049037aa4d337b65b42f7c1745ae7080c2db",
"conventions/model-routing.md": "8b28aa706436e7b68493ec481e12be6309b0af1a553930e8b1029d675fa4b3ad",
"conventions/path-discipline.md": "8af5415721bd115e6979c96688bcf542a32675809ada9aaedbb6706a40926954",
"conventions/quality.md": "8aa681001114689d34268ccadaf0e2ff07b8f68aa5987c093a8c4a7a744f12a6",
"conventions/regex-discipline.md": "d96a9baa6f27184e165889a9c655607366a739d851684d9f41cdec294f99edac",
@@ -58,7 +58,7 @@
"conventions/subagent-routing.md": "8b8830b815a9a8581a12b489f966c0b0a39eb9b5f66e905a691a03653eef348d",
"conventions/test-before-bulk.md": "6b2c52cda9e2cd5f04c15152b3d92aeb7187ab193a15082be0f8a3991a6a5725",
"conventions/untrusted-content.md": "259384d490892cd0e1e8e054decf752d7354f516c83aee57b332c1a96aac6a6e",
"conversation-archive/SKILL.md": "867d3a202ce500027ed2ab85edd9d3359d677aa7a180105f2b7db12ad3492701",
"conversation-archive/SKILL.md": "4e1dea00f5e1e16e749a42f295fdccf556199d4400a2ba1b891aa91839e37214",
"conversation-archive/routing-eval.jsonl": "ae087a84b1fd5b108b7cdab8d035a09b3ccecd8aad53ba5f71e463059108cfca",
"correction-pipeline/SKILL.md": "caf1264b7afec46569d30f6d92b07f37ae375e3f4e6aeddd58866aec327053de",
"correction-pipeline/routing-eval.jsonl": "7f8d96606a8d7bed3d79fdcee6904764c8abb9fa0b506adb414b5c4805b69d0b",
@@ -88,7 +88,7 @@
"idea-lineage/routing-eval.jsonl": "ee2e00704b9accb7dd58bb8f126a3bc04a2c40be499180fa505dbf6d5061cd41",
"ingest/SKILL.md": "dc40ecc0072806fb8c7bb6ab9cf1f103842e05653eb55d67632d7e3ffc4dd7d2",
"install/SKILL.md": "881bd0a422f34c6df4642aae66c51e2a4cc18ad5ca6d0b52d44b4de93512a3c4",
"maintain/SKILL.md": "59da3f0227a2b41ed9c3beb334322f1ef1dbd80af733a587c0a4687a707b9815",
"maintain/SKILL.md": "33e48e31baf89b6b257ad863cdb9de444777bc1272f5ed8c2b28be3a54cbaa14",
"manifest.json": "03471868cce05fa38af6f793da54e2fc11f77ef778271a596d75bc29f9ec4c73",
"measure-before-you-fix/SKILL.md": "1fd3b40ab65cbd08f50dea16107701859165469be3c85c57d779c7b4bbf92db8",
"measure-before-you-fix/routing-eval.jsonl": "0661df9974a9cfe31216d574b1db0ef341945c2eb844ebf4ab6920fcbbc90d6c",
@@ -150,7 +150,7 @@
"resolve-before-asking/routing-eval.jsonl": "bac1bcf30337f5255ef4ce1a2a8a2b38d58ebcd576503c483190c79ec6e69489",
"schema-author/SKILL.md": "09d69ee45970191bb2592a764685f67196cabf350db40b4c0c5ffc19ea9e2df3",
"schema-unify/SKILL.md": "14ddc0f8bc7d8b11eb03dc4eb35621d140fba71bd35779835d85e297acfb0177",
"setup/SKILL.md": "68dde0de48bb4c93b13d9e19c3a20ff82155985ad09f509669aff8ece0a6f4fb",
"setup/SKILL.md": "322faf7099afd71f9add0f6dbe4d2bf9686df54c521a29e018f4ab25608e3dc0",
"signal-detector/SKILL.md": "64e4547f5a8624c53d875001b423d240ec73ee9fd026a96c7b799d287c5fb6e4",
"skill-autobench/SKILL.md": "144572ec76f3784a97645dfde587ab13d77e804f50b00dc7fbe678204de6ff21",
"skill-autobench/routing-eval.jsonl": "8d961ed6403b7e2f690948e4c18529d40f26f6f21064befc56d465966b1a9ec0",
+37 -5
View File
@@ -154,6 +154,19 @@ const CLI_ONLY_SELF_HELP = new Set([
// would leave that help dead code behind the generic stub (the init.ts:117
// trap ENG-2 names).
'bootstrap', 'hook', 'sweep',
// cathedral-4: transcripts ships its own HELP (the ingest import lane +
// the v0.29 recent reader). Without this the generic stub hides both.
'transcripts',
// jobs ships JOBS_HELP + a per-subcommand record (JOBS_SUBCOMMAND_HELP) in
// jobs.ts, guarded BEFORE the thin-client refusal and the subcommand switch
// so `jobs work --help` prints help instead of starting a worker daemon.
// Without this entry the generic stub hid the worker entry point entirely.
'jobs',
// #4152: dream ships its own printHelp AND the `dream retriage --help`
// subverb help (dispatched engine-free before parseArgs). The generic stub
// would hide both — `gbrain dream retriage --help` printed the one-line
// dream stub instead of the retriage contract (outside-voice CX9).
'dream',
]);
/**
@@ -172,6 +185,13 @@ const SELF_HELP_WITHOUT_ENGINE: Record<string, () => Promise<(engine: never, arg
maintain: async () => (await import('./commands/maintain.ts')).runMaintain as never,
'extract-conversation-facts': async () =>
(await import('./commands/extract-conversation-facts.ts')).runExtractConversationFacts as never,
transcripts: async () => (await import('./commands/transcripts.ts')).runTranscripts as never,
// runJobs accepts BrainEngine | null and its help guard returns before any
// engine (or subcommand body) is touched.
jobs: async () => (await import('./commands/jobs.ts')).runJobs as never,
// runDream accepts BrainEngine | null; --help (and `retriage --help`) is
// answered before any engine-bearing work per the dream.ts IRON RULE.
dream: async () => (await import('./commands/dream.ts')).runDream as never,
};
/** Returns true when the command's own help was printed. */
@@ -2344,16 +2364,26 @@ async function handleCliOnly(command: string, args: string[]) {
try {
switch (command) {
case 'import': {
const { runImport } = await import('./commands/import.ts');
const { runImport, ImportAbortError } = await import('./commands/import.ts');
// v0.41 (Codex r2 #3 fix): honor errors counter for exit code.
// runImport's per-file catch already records failures, but the
// CLI was discarding the result so the process exited 0 even
// when files failed (e.g. content-sanity hard-block throws,
// size-cap throws, parse errors). Surface non-zero on errors > 0
// so wrappers (sync, CI scripts, `&& gbrain doctor`) propagate.
const importResult = await runImport(engine, args);
if (importResult.errors > 0) {
setCliExitVerdict(1);
try {
const importResult = await runImport(engine, args);
if (importResult.errors > 0) {
setCliExitVerdict(1);
}
} catch (e) {
// W0 (Tier-1 #5): runImport throws typed aborts instead of
// process.exit(1) so in-process callers (sync_brain MCP op,
// autopilot, minion handler) survive a preflight failure. The CLI
// keeps the exact pre-fix behavior: message already printed at the
// throw site, exit non-zero here.
if (e instanceof ImportAbortError) process.exit(e.exitCode);
throw e;
}
break;
}
@@ -3177,7 +3207,7 @@ TOOLS
orphans [--json] [--count] Find pages with no inbound wikilinks
salience [--days N] [--kind P] v0.29: pages ranked by emotional + activity salience
anomalies [--since D] [--sigma N] v0.29: cohort-based statistical anomalies (tag, type)
transcripts recent [--days N] v0.29: recent raw .txt transcripts (local-only)
transcripts <ingest|status|recent> v0.46: import agent session logs + chat exports (local-only)
dream [--dry-run] [--json] Run the overnight maintenance cycle once (cron-friendly).
See also: autopilot --install (continuous daemon).
check-resolvable [--json] [--fix] Validate skill tree (reachability/MECE/DRY)
@@ -3229,7 +3259,9 @@ JOBS (Minions)
jobs retry <id> Re-queue failed/dead job
jobs prune [--older-than 30d] Clean old jobs
jobs stats Job health dashboard
jobs watch [--follow] Live queue dashboard
jobs work [--queue Q] Start worker daemon (Postgres only)
jobs supervisor [start|status|stop] Auto-restarting worker wrapper
ADMIN
stats Brain statistics
+142 -31
View File
@@ -23,6 +23,9 @@ import { createHash, randomBytes } from 'crypto';
import { loadConfig, toEngineConfig } from '../core/config.ts';
import { createEngine } from '../core/engine-factory.ts';
import type { BrainEngine } from '../core/engine.ts';
import { assertAllowedScopes } from '../core/scope.ts';
import { TOKEN_ID_RE } from '../core/token-mint.ts';
import { normalizeTokenScopes } from '../core/legacy-token-scope.ts';
import { sqlQueryForEngine, executeRawJsonb, type SqlQuery } from '../core/sql-query.ts';
function hashToken(token: string): string {
@@ -66,8 +69,20 @@ async function withConfiguredSql<T>(
}
}
async function create(name: string, opts: { takesHolders?: string[] } = {}) {
if (!name) { console.error('Usage: auth create <name> [--takes-holders world,garry]'); process.exit(1); }
async function create(name: string, opts: { takesHolders?: string[]; scopes?: string[] } = {}) {
if (!name) { console.error('Usage: auth create <name> [--takes-holders world,garry] [--scopes read,write]'); process.exit(1); }
// #4043 least-privilege: validate scopes at mint time — the verify path
// treats a filtered-empty scopes array as DENY, so a typo must fail loudly
// here, never silently brick (or widen) the token.
if (opts.scopes !== undefined) {
try {
if (opts.scopes.length === 0) throw new Error('at least one scope is required');
assertAllowedScopes(opts.scopes);
} catch (e: any) {
console.error(`Invalid --scopes: ${e.message}`);
process.exit(1);
}
}
const token = generateToken();
const hash = hashToken(token);
@@ -84,17 +99,35 @@ async function create(name: string, opts: { takesHolders?: string[] } = {}) {
// through the wire-protocol type oid without the v0.12.0 double-encode
// bug class (verified by test/e2e/auth-permissions.test.ts:67 on
// Postgres and test/sql-query.test.ts on PGLite).
await executeRawJsonb(
engine,
`INSERT INTO access_tokens (name, token_hash, permissions)
VALUES ($1, $2, $3::jsonb)`,
[name, hash],
[permissions],
);
console.log(`Token created for "${name}" (takes_holders=${JSON.stringify(takesHolders)}):\n`);
//
// Scopes (when given) land in the original-schema scopes TEXT[] column
// via an array literal through a TEXT param — values are allowlisted,
// so the literal needs no quoting and runs identically on both engines.
// Omitted → NULL → the historical grandfathered full-access grant.
if (opts.scopes !== undefined) {
await executeRawJsonb(
engine,
`INSERT INTO access_tokens (name, token_hash, permissions, scopes)
VALUES ($1, $2, $4::jsonb, $3::text[])`,
[name, hash, `{${opts.scopes.join(',')}}`],
[permissions],
);
} else {
await executeRawJsonb(
engine,
`INSERT INTO access_tokens (name, token_hash, permissions)
VALUES ($1, $2, $3::jsonb)`,
[name, hash],
[permissions],
);
}
const scopeLine = opts.scopes !== undefined
? `scopes=${JSON.stringify(opts.scopes)}`
: 'scopes=full access (grandfathered — pass --scopes read,write to narrow)';
console.log(`Token created for "${name}" (takes_holders=${JSON.stringify(takesHolders)}, ${scopeLine}):\n`);
console.log(` ${token}\n`);
console.log('Save this token — it will not be shown again.');
console.log(`Revoke with: gbrain auth revoke "${name}"`);
console.log(`Revoke with: gbrain auth revoke "${name}" (or gbrain auth revoke --id <id> from auth list)`);
console.log(`Update visibility: gbrain auth permissions "${name}" set-takes-holders world,garry`);
});
} catch (e: any) {
@@ -121,10 +154,19 @@ async function permissions(name: string, action: string, value: string | undefin
}
const perms = { takes_holders: list };
// JSONB UPDATE via executeRawJsonb — same pattern as create() above.
// MERGE, never whole-object replace: `SET permissions = $2::jsonb`
// would silently DELETE every other grant key (source_id federation,
// and any future key) on a routine takes-holders edit — the grant-wipe
// class the #4043 review caught.
// The jsonb_typeof guard repairs rows carrying historical double-encode
// damage (a jsonb string/array scalar): `scalar || object` would produce
// a jsonb ARRAY and silently strand every grant, so a damaged left
// operand is reset to '{}' on edit — the old whole-replace semantics for
// damaged rows, merge semantics for healthy object rows.
const result = await executeRawJsonb(
engine,
`UPDATE access_tokens
SET permissions = $2::jsonb
SET permissions = (CASE WHEN jsonb_typeof(permissions) = 'object' THEN permissions ELSE '{}'::jsonb END) || $2::jsonb
WHERE name = $1
RETURNING id`,
[name],
@@ -142,10 +184,20 @@ async function permissions(name: string, action: string, value: string | undefin
}
}
/** Render a token row's scope grant honestly (#4043: NULL = grandfathered).
* Routes through the SAME normalizer the verify path uses the ops surface
* must never claim admin on a row the serve actually scopes or denies. */
export function renderTokenScopes(scopes: unknown): string {
const normalized = normalizeTokenScopes(scopes);
if (normalized === undefined) return 'admin (grandfathered)';
if (normalized.length === 0) return '(deny-all)';
return normalized.join(',');
}
async function list() {
await withConfiguredSql(async (sql) => {
const rows = await sql`
SELECT name, created_at, last_used_at, revoked_at
SELECT id, name, scopes, created_at, last_used_at, revoked_at
FROM access_tokens
ORDER BY created_at DESC
`;
@@ -153,20 +205,22 @@ async function list() {
console.log('No tokens found. Create one: gbrain auth create "my-client"');
return;
}
console.log('Name Created Last Used Status');
console.log('─'.repeat(80));
console.log('ID Name Scopes Created Last Used Status');
console.log('─'.repeat(126));
for (const r of rows) {
const id = String(r.id).padEnd(36);
const name = (r.name as string).padEnd(20);
const scopes = renderTokenScopes(r.scopes).padEnd(21);
const created = new Date(r.created_at as string).toISOString().slice(0, 19);
const lastUsed = r.last_used_at ? new Date(r.last_used_at as string).toISOString().slice(0, 19) : 'never'.padEnd(19);
const status = r.revoked_at ? 'REVOKED' : 'active';
console.log(`${name} ${created} ${lastUsed} ${status}`);
console.log(`${id} ${name} ${scopes} ${created} ${lastUsed} ${status}`);
}
});
}
async function revoke(name: string) {
if (!name) { console.error('Usage: auth revoke <name>'); process.exit(1); }
if (!name) { console.error('Usage: auth revoke <name> | auth revoke --id <uuid>'); process.exit(1); }
await withConfiguredSql(async (sql) => {
const rows = await sql`
UPDATE access_tokens SET revoked_at = now()
@@ -177,10 +231,36 @@ async function revoke(name: string) {
console.error(`No active token found with name "${name}".`);
process.exit(1);
}
if (rows.length > 1) {
console.log(`Note: ${rows.length} active tokens carried the name "${name}" — all revoked. Use revoke --id for precision.`);
}
console.log(`Token "${name}" revoked.`);
});
}
/** #4043: names are not unique revoke-by-id is the precise path. The
* revocation semantics are canonical in src/core/token-mint.ts
* (revokeLegacyTokenById); this CLI wrapper keeps its own UPDATE only to
* RETURN the name for the confirmation line keep the two in lockstep. */
async function revokeById(id: string) {
if (!id || !TOKEN_ID_RE.test(id)) {
console.error('Usage: auth revoke --id <uuid> (ids are shown by `gbrain auth list`)');
process.exit(1);
}
await withConfiguredSql(async (sql) => {
const rows = await sql`
UPDATE access_tokens SET revoked_at = now()
WHERE id = ${id}::uuid AND revoked_at IS NULL
RETURNING name
`;
if (rows.length === 0) {
console.error(`No active token found with id "${id}".`);
process.exit(1);
}
console.log(`Token "${rows[0].name}" (${id}) revoked.`);
});
}
async function test(url: string, token: string) {
if (!url || !token) {
console.error('Usage: auth test <url> --token <token>');
@@ -781,22 +861,40 @@ async function clientsCmd(args: string[]) {
* still works.
*/
/**
* Parse `auth create` args into `{ name, takesHolders }`.
* Parse `auth create` args into `{ name, takesHolders, scopes }`.
*
* Exported + pure so the positional-vs-flag logic is unit-testable. Only
* excludes the --takes-holders VALUE from the positional search when the flag
* is present the pre-v0.41 inline version used `rest[takesIdx + 1]` which
* excludes flag VALUES from the positional search when their flag is
* present the pre-v0.41 inline version used `rest[takesIdx + 1]` which
* resolved to `rest[0]` when `takesIdx === -1`, silently dropping the name on
* the bare `gbrain auth create <name>` form.
*
* --scopes accepts comma- and/or whitespace-separated input (the
* register-client #3990 normalization precedent). Validation against the
* allowed scope set happens in create() so the error path exits cleanly.
*/
export function parseAuthCreateArgs(rest: string[]): { name: string; takesHolders?: string[] } {
export function parseAuthCreateArgs(rest: string[]): { name: string; takesHolders?: string[]; scopes?: string[]; error?: string } {
const takesIdx = rest.indexOf('--takes-holders');
const takesHolders = takesIdx >= 0 && rest[takesIdx + 1]
? rest[takesIdx + 1].split(',').map(s => s.trim()).filter(Boolean)
: undefined;
const takesValue = takesIdx >= 0 ? rest[takesIdx + 1] : undefined;
const positional = rest.find(a => !a.startsWith('--') && a !== takesValue);
return { name: positional || '', takesHolders };
// Fail closed on a missing/flag-like value: `--scopes` as the last arg
// silently minting a grandfathered FULL-ACCESS token is the exact
// fail-open-by-silent-precedence class the harness parser rejects [X14].
if (takesIdx >= 0 && (takesValue === undefined || takesValue.startsWith('--'))) {
return { name: '', error: 'the takes-holders flag requires a value (e.g. world,garry)' };
}
const takesHolders = takesValue !== undefined
? takesValue.split(',').map(s => s.trim()).filter(Boolean)
: undefined;
const scopesIdx = rest.indexOf('--scopes');
const scopesValue = scopesIdx >= 0 ? rest[scopesIdx + 1] : undefined;
if (scopesIdx >= 0 && (scopesValue === undefined || scopesValue.startsWith('--'))) {
return { name: '', error: 'the scopes flag requires a value (e.g. read,write) — omitting it would mint a full-access token' };
}
const scopes = scopesValue !== undefined
? scopesValue.split(/[\s,]+/).map(s => s.trim()).filter(Boolean)
: undefined;
const positional = rest.find(a => !a.startsWith('--') && a !== takesValue && a !== scopesValue);
return { name: positional || '', takesHolders, ...(scopes !== undefined ? { scopes } : {}) };
}
export async function runAuth(args: string[]): Promise<void> {
@@ -804,12 +902,21 @@ export async function runAuth(args: string[]): Promise<void> {
switch (cmd) {
case 'create': {
// v0.28: optional --takes-holders world,garry,brain (default: world only)
// #4043: optional --scopes read,write (default: full access, grandfathered)
const parsed = parseAuthCreateArgs(rest);
await create(parsed.name, { takesHolders: parsed.takesHolders });
if (parsed.error) {
console.error(`Error: ${parsed.error}`);
process.exit(1);
}
await create(parsed.name, { takesHolders: parsed.takesHolders, scopes: parsed.scopes });
return;
}
case 'list': await list(); return;
case 'revoke': await revoke(rest[0]); return;
case 'revoke': {
if (rest[0] === '--id') { await revokeById(rest[1] || ''); return; }
await revoke(rest[0]);
return;
}
case 'permissions': {
// gbrain auth permissions <name> set-takes-holders world,garry
await permissions(rest[0] || '', rest[1] || '', rest[2]);
@@ -830,13 +937,17 @@ export async function runAuth(args: string[]): Promise<void> {
console.log(`GBrain Token Management
Usage:
gbrain auth create <name> [--takes-holders world,garry,brain]
gbrain auth create <name> [--takes-holders world,garry,brain] [--scopes read,write]
Create a legacy bearer token. v0.28: --takes-holders
sets the per-token allow-list for the takes.holder
field (default: ["world"]). MCP-bound calls to
takes_list / takes_search / query filter by this.
gbrain auth list List all tokens
gbrain auth revoke <name> Revoke a legacy token
--scopes narrows the token to the listed op scopes
(comma or space separated; omit = full access,
grandfathered).
gbrain auth list List all tokens (id, scopes, usage)
gbrain auth revoke <name> Revoke a legacy token (ALL active rows with that name)
gbrain auth revoke --id <uuid> Revoke exactly one token by id (names are not unique)
gbrain auth permissions <name> set-takes-holders <h1,h2,h3>
Update visibility for an existing token
gbrain auth register-client <name> [options] Register an OAuth 2.1 client (v0.26+)
+79 -28
View File
@@ -69,8 +69,13 @@ export interface FanoutOpts {
}
export interface FanoutResult {
/** Source ids dispatched this tick. */
/** Source ids whose submission INSERTED a fresh job this tick. */
dispatched: string[];
/** Source ids whose submission coalesced onto an existing pending job
* (maxPending single-flight or same-slot idempotency) work is in
* flight, but no new row was created. Kept separate so no surface
* claims a dispatch that didn't insert. */
coalesced: string[];
/** Source ids skipped because their last_full_cycle_at is still fresh. */
skipped_fresh: string[];
/** Source ids beyond the fanoutMax cap (will retry next tick). */
@@ -400,19 +405,32 @@ export async function dispatchPerSource(
{ repoPath: opts.repoPath },
{
queue: 'default',
// Slot key dedups repeats within one slot; maxPending: 1 is the
// cross-slot guard — an in-flight (waiting or live-lock active)
// cycle suppresses re-dispatch even after the slot rotates. This
// closes the unbounded-duplicate loop: slot rotation used to mint
// a fresh key every baseInterval while maxWaiting ignored the
// active row, growing the queue forever when a cycle stalled.
idempotency_key: `autopilot-cycle:${opts.slot}`,
max_attempts: 2,
timeout_ms: opts.timeoutMs,
maxWaiting: 1,
maxPending: 1,
},
);
if (opts.jsonMode) {
if (job.coalesced) {
if (opts.jsonMode) {
emit(JSON.stringify({ event: 'dispatch_coalesced', job_id: job.id, mode: 'legacy', slot: opts.slot }));
} else {
log(`[dispatch] coalesced onto job #${job.id} autopilot-cycle (legacy single-source; already in flight)`);
}
} else if (opts.jsonMode) {
emit(JSON.stringify({ event: 'dispatched', job_id: job.id, mode: 'legacy', slot: opts.slot }));
} else {
log(`[dispatch] job #${job.id} autopilot-cycle (legacy single-source)`);
}
return {
dispatched: [],
coalesced: [],
skipped_fresh: [],
skipped_cap: [],
skipped_cooldown: [],
@@ -448,6 +466,7 @@ export async function dispatchPerSource(
);
const dispatched: string[] = [];
const coalesced: string[] = [];
for (const src of dispatch) {
try {
const shouldPull = sourceConfigHasRemoteUrl(src.config);
@@ -470,26 +489,43 @@ export async function dispatchPerSource(
idempotency_key: `autopilot-cycle:${src.id}:${opts.slot}`,
max_attempts: 2,
timeout_ms: opts.timeoutMs,
// DELIBERATELY no maxWaiting: 1 here. maxWaiting is per
// (name, queue), so it would coalesce all N per-source jobs
// sharing name='autopilot-cycle' down to ONE waiting job —
// killing the fan-out. The per-source idempotency_key
// already provides the right dedup granularity (one job per
// source per slot, regardless of how many ticks try).
// Still DELIBERATELY no maxWaiting here (its NULL-as-wildcard
// source scope would coalesce N per-source jobs down to one).
// maxPending is safe: its scope is EXACT on
// COALESCE(data.sourceId, data.source_id), so each source keeps
// an independent single-flight cap — and unlike the slot key, it
// suppresses cross-slot re-dispatch while THIS source's cycle is
// still in flight (waiting or live-lock active).
maxPending: 1,
},
);
dispatched.push(src.id);
if (opts.jsonMode) {
emit(JSON.stringify({
event: 'dispatched',
job_id: job.id,
mode: 'per_source',
source_id: src.id,
pull: shouldPull,
slot: opts.slot,
}));
if (job.coalesced) {
coalesced.push(src.id);
if (opts.jsonMode) {
emit(JSON.stringify({
event: 'dispatch_coalesced',
job_id: job.id,
mode: 'per_source',
source_id: src.id,
slot: opts.slot,
}));
} else {
log(`[dispatch] coalesced onto job #${job.id} autopilot-cycle source=${src.id} (already in flight)`);
}
} else {
log(`[dispatch] job #${job.id} autopilot-cycle source=${src.id}${shouldPull ? ' pull=yes' : ''}`);
dispatched.push(src.id);
if (opts.jsonMode) {
emit(JSON.stringify({
event: 'dispatched',
job_id: job.id,
mode: 'per_source',
source_id: src.id,
pull: shouldPull,
slot: opts.slot,
}));
} else {
log(`[dispatch] job #${job.id} autopilot-cycle source=${src.id}${shouldPull ? ' pull=yes' : ''}`);
}
}
} catch (e) {
// Per-source submit failure does NOT abort the tick (codex E1 F1
@@ -524,6 +560,7 @@ export async function dispatchPerSource(
return {
dispatched,
coalesced,
skipped_fresh: skippedFresh.map(s => s.id),
skipped_cap: skippedCap.map(s => s.id),
skipped_cooldown: skippedCooldown.map(s => s.id),
@@ -546,16 +583,18 @@ export function isGlobalMaintenanceStale(lastGlobalAtIso: string | null, now = D
* #2194 fix #3 / #2227 bug #3 dispatch the single brain-wide maintenance job
* that runs the `global` cycle phases (embed, orphans, purge, ) ONCE per
* window, instead of N per-source cycles each running them concurrently (the
* RSS blowout). Single-flight is structural: one `idempotency_key` +
* `maxWaiting:1`, so a slow run never stacks. Gated on `autopilot.last_global_at`
* (stamped by the handler on success). Postgres-only fan-out concern; on PGLite
* the file lock already serializes, but the job is still correct there.
* RSS blowout). Single-flight is structural: one `idempotency_key` per slot +
* `maxPending:1` (an in-flight waiting/live-lock-active run suppresses
* re-dispatch even across slot rotation), so a slow run never stacks. Gated on
* `autopilot.last_global_at` (stamped by the handler on success). Postgres-only
* fan-out concern; on PGLite the file lock already serializes, but the job is
* still correct there.
*/
export async function dispatchGlobalMaintenance(
engine: BrainEngine,
queue: MinionQueue,
opts: { repoPath: string; slot: string; timeoutMs: number; jsonMode: boolean; emit?: (l: string) => void; log?: (l: string) => void },
): Promise<{ dispatched: boolean; reason: 'stale' | 'fresh' }> {
): Promise<{ dispatched: boolean; coalesced?: boolean; reason: 'stale' | 'fresh' }> {
const emit = opts.emit ?? ((line) => process.stderr.write(line + '\n'));
const log = opts.log ?? ((line) => console.log(line));
@@ -575,14 +614,26 @@ export async function dispatchGlobalMaintenance(
{ repoPath: opts.repoPath, phases: GLOBAL_PHASES },
{
queue: 'default',
// Structural single-flight: one global job per slot; maxWaiting:1 coalesces
// any surplus so a slow brain-wide pass never stacks duplicates.
// Structural single-flight: one global job per slot; maxPending:1
// coalesces any surplus — including across slot rotation while a slow
// brain-wide pass is still in flight — so duplicates never stack.
idempotency_key: `autopilot-global:${opts.slot}`,
max_attempts: 2,
timeout_ms: opts.timeoutMs,
maxWaiting: 1,
maxPending: 1,
},
);
if (job.coalesced) {
if (opts.jsonMode) {
emit(JSON.stringify({ event: 'dispatch_coalesced', job_id: job.id, mode: 'global_maintenance', slot: opts.slot }));
} else {
log(`[dispatch] coalesced onto job #${job.id} autopilot-global-maintenance (already in flight)`);
}
// dispatched: false — no row was inserted (same honest-dispatch contract
// as dispatchPerSource, where coalesced sources are excluded from
// `dispatched`). The coalesced flag says work is already in flight.
return { dispatched: false, coalesced: true, reason: 'stale' };
}
if (opts.jsonMode) {
emit(JSON.stringify({ event: 'dispatched', job_id: job.id, mode: 'global_maintenance', slot: opts.slot }));
} else {
+19 -3
View File
@@ -1209,13 +1209,19 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
// source timestamps say every source is fresh, advance the local
// clock too; otherwise a non-empty targeted plan would be skipped
// on every tick until the persisted 60-minute window elapsed.
if (result.dispatched.length > 0 || result.legacy_fallback || result.all_sources_fresh) {
// Coalesced counts as work-in-flight: before dispatched/coalesced
// split, a coalesced submission advanced this clock via dispatched —
// keep that behavior, or an all-coalesced tick (single-flight
// suppression) would retake the full-cycle branch every tick and
// starve the targeted-plan path for the whole in-flight window.
if (result.dispatched.length > 0 || result.coalesced.length > 0 || result.legacy_fallback || result.all_sources_fresh) {
lastFullCycleAt = Date.now();
}
if (jsonMode) {
process.stderr.write(JSON.stringify({
event: 'fanout_summary',
dispatched: result.dispatched,
coalesced: result.coalesced,
skipped_fresh: result.skipped_fresh,
skipped_cap: result.skipped_cap,
skipped_cooldown: result.skipped_cooldown,
@@ -1225,7 +1231,8 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
}) + '\n');
} else if (!result.legacy_fallback) {
console.log(
`[dispatch] fanout: ${result.dispatched.length} dispatched, ` +
`[dispatch] fanout: ${result.dispatched.length} dispatched` +
`${result.coalesced.length > 0 ? ` (${result.coalesced.length} coalesced onto in-flight)` : ''}, ` +
`${result.skipped_fresh.length} fresh, ${result.skipped_cap.length} capped, ` +
`${result.skipped_cooldown.length} cooldown ` +
`(score=${score}, max=${fanoutMax})`,
@@ -1254,7 +1261,16 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
submitOpts,
isProtected ? { allowProtectedSubmit: true } : undefined,
);
if (jsonMode) {
// Honest-dispatch contract (same as the fanout paths): a
// coalesced submission never claims a dispatch that didn't
// insert a row.
if (job.coalesced) {
if (jsonMode) {
process.stderr.write(JSON.stringify({ event: 'dispatch_coalesced', job_id: job.id, mode: 'targeted', step: step.id, score, plan_size: plan.length }) + '\n');
} else {
console.log(`[dispatch] coalesced onto job #${job.id} ${step.job} (targeted: ${step.id}; already in flight)`);
}
} else if (jsonMode) {
process.stderr.write(JSON.stringify({ event: 'dispatched', job_id: job.id, mode: 'targeted', step: step.id, score, plan_size: plan.length }) + '\n');
} else {
console.log(`[dispatch] job #${job.id} ${step.job} (targeted: ${step.id}; score=${score})`);
+592 -41
View File
@@ -27,8 +27,9 @@
* B5 relay instruction), never a stack trace.
*/
import { mkdirSync, readdirSync } from 'node:fs';
import { basename, isAbsolute, join, resolve } from 'node:path';
import { existsSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { basename, dirname, isAbsolute, join, resolve } from 'node:path';
import { VERSION } from '../version.ts';
import { loadConfig, loadConfigFileOnly, toEngineConfig } from '../core/config.ts';
@@ -62,11 +63,32 @@ import {
} from '../core/bootstrap/hooks.ts';
import {
guardReceiptOverwrite,
readHarnessReceiptState,
readManifest,
readReceipt,
writeReceipt,
type InstallReceipt,
} from '../core/bootstrap/format.ts';
import {
applyHarness,
codexBlockOwnsName,
ensureHarnessHome,
parseHarnessArgs,
removeHarness,
statusHarness,
type HarnessDeps,
} from '../core/bootstrap/harness.ts';
import { codexConfigPath, opencodeConfigDir, opencodeGlobalConfigPath, opencodeProjectConfigPath } from '../core/bootstrap/host-specs.ts';
import {
opencodeEntryKind,
opencodeEntrySnippet,
opencodeRemoteEntryExists,
parseOpencodeConfig,
reconcileOpencodeSiblingGlobal,
removeOpencodeMcpEntry,
writeOpencodeMcpEntry,
} from '../core/bootstrap/opencode-json.ts';
import { promptLine } from '../core/cli-util.ts';
import {
appendInstallLog,
gitOriginUrl,
@@ -76,7 +98,7 @@ import {
} from '../core/bootstrap/status.ts';
import { verifyWorkspace, deriveWorkspaceSourceId } from '../core/bootstrap/verify.ts';
export const BOOTSTRAP_HELP = `gbrain bootstrap — paste-in agent install (Claude Code / Codex)
export const BOOTSTRAP_HELP = `gbrain bootstrap — paste-in agent install (Claude Code / Codex / opencode)
Usage: gbrain bootstrap <subcommand> [flags]
@@ -92,16 +114,29 @@ Subcommands (run \`gbrain bootstrap status\` first — it is the resume entrypoi
render [--force] [--only F] [--minimal]
Render identity files from the confirmed answers.
Never clobbers; --force backs up first.
hooks [--harness claude-code|codex] [--repair] [--no-hooks] [--gbrain-bin <path>]
hooks [--harness claude-code|codex|opencode] [--repair] [--no-hooks] [--gbrain-bin <path>]
Register MCP (+ per-turn hooks on Claude Code,
ON by default; --no-hooks opts out, GBRAIN_HOOKS=0
disables at runtime).
disables at runtime). opencode registrations are
written directly into its JSONC config (user-global
by default; MCP_SCOPE=project is an explicit opt-in
with a sharing warning).
repo Create the dedicated PRIVATE GitHub repo (or adopt
an EMPTY private repo you created under your own
account), verify the privacy bit via the API, push.
verify [--json] The whole install contract (round-trip, graph floor,
magic moment, scans, hooks smoke). Exit 0 or not done.
attach [--harness H] Machine two: adopt a cloned agent workspace.
harness [--harness claude-code|codex|opencode|all] [--url U | --port N] [--source ID]
[--token-name NAME | --token TOK] [--name MCPNAME] [--project DIR]...
[--no-hooks] [--no-capture] [--force] [--status] [--remove] [--yes] [--json]
Wire framework-spawned Claude Code / Codex / opencode
sessions to a RUNNING \`gbrain serve --http\` on this box
(#4043): scoped bearer token, user-scope MCP + headless
pre-approval, lifecycle hooks (user scope, or per --project
dir), codex config block, opencode config entry. No
agent.json needed. Idempotent; --remove tears it down.
(--local is an accepted no-op alias.)
cloud-setup-script Print the paste-ready cloud environment setup
script (installs the gbrain binary into the
environment snapshot; npm-based bun fetching
@@ -137,7 +172,7 @@ const SUBCOMMAND_HELP: Record<string, string> = {
' Create the dedicated PRIVATE GitHub repo (or adopt an EMPTY private repo you created\n' +
' under your own account), verify the privacy bit via the API, push.',
hooks:
'gbrain bootstrap hooks [--harness claude-code|codex] [--repair] [--no-hooks] [--gbrain-bin <path>]\n' +
'gbrain bootstrap hooks [--harness claude-code|codex|opencode] [--repair] [--no-hooks] [--gbrain-bin <path>]\n' +
' Register MCP (+ per-turn hooks on Claude Code, ON by default; --no-hooks opts out).',
verify:
'gbrain bootstrap verify [--json]\n' +
@@ -223,12 +258,26 @@ function shellQuoteForDisplay(arg: string): string {
// ── Shared plumbing ─────────────────────────────────────────────────────────
type Harness = 'claude-code' | 'codex';
type Harness = 'claude-code' | 'codex' | 'opencode';
/** Best-effort harness auto-detect; the --harness flag always wins. */
/** Every workspace-lane harness exhaustive-switch anchors key off this so
* a future member is a COMPILE error at each dispatch site, not a silent
* fall-through into another harness's branch (the union-widening trap: a
* `harness === 'claude-code' ? A : B` ternary routes every new member down
* B). */
const HARNESSES = ['claude-code', 'codex', 'opencode'] as const satisfies readonly Harness[];
function isHarness(v: string | undefined): v is Harness {
return (HARNESSES as readonly string[]).includes(v ?? '');
}
/** Best-effort harness auto-detect; the --harness flag always wins.
* opencode sets OPENCODE=1 (+OPENCODE_PID) in its bash-tool children
* verified against opencode 1.18.18 (OPENCODE-CLI-PIN.md §Environment). */
export function detectHarness(env: Record<string, string | undefined> = process.env): Harness | null {
if (env.CLAUDECODE || env.CLAUDE_CODE_ENTRYPOINT) return 'claude-code';
if (env.CODEX_HOME || env.CODEX_SANDBOX || env.CODEX_CI) return 'codex';
if (env.OPENCODE || env.OPENCODE_PID) return 'opencode';
return null;
}
@@ -264,7 +313,16 @@ async function verifyMcpTargetsWorkspace(
gbrainBin: string,
sourceId: string,
): Promise<'match' | 'mismatch' | 'unknown'> {
const bin = harness === 'claude-code' ? 'claude' : 'codex';
// Exec-lane harnesses only. opencode registrations go through the direct
// JSONC writer whose 4-state fingerprint IS the [FIX7] check (structural,
// no exec) — it never routes here; 'unknown' keeps a stray call honest.
const EXEC_HARNESS_BIN = {
'claude-code': 'claude',
codex: 'codex',
opencode: null,
} as const satisfies Record<Harness, string | null>;
const bin = EXEC_HARNESS_BIN[harness];
if (bin === null) return 'unknown';
let res;
try {
res = await runner([bin, 'mcp', 'get', name]);
@@ -279,6 +337,121 @@ async function verifyMcpTargetsWorkspace(
return hasBin && hasSource ? 'match' : 'mismatch';
}
/** Wall-clock cap on the best-effort `opencode mcp list` probe: `mcp list`
* SPAWNS every configured server, and a hung spawn must not hang the install
* on timeout the probe child is actually TERMINATED (SIGTERM, then SIGKILL
* ~2s later) and the result degrades to the could-not-confirm branch (code
* 124, repo-visibility's raced-runner convention). */
const OPENCODE_PROBE_TIMEOUT_MS = 20_000;
/** Injectable probe-spawn seam (the door serial tests capture argv + cwd +
* env and fake the child). The default holds the REAL process handle via
* Bun.spawn a Promise.race that merely abandons a hung `opencode mcp list`
* leaves its spawned MCP servers running (including the just-registered
* `gbrain serve`, which then squats the PGLite single-writer lock) and keeps
* the CLI's event loop alive past flushThenExit. */
export interface OpencodeProbeHandle {
exited: Promise<number>;
kill(force?: boolean): void;
stdout: Promise<string>;
stderr: Promise<string>;
/** Detach the child + its pipes from the event loop (called when the probe
* gives up on a hung child/grandchild so the CLI can still exit). */
unref?: () => void;
}
export type OpencodeProbeSpawn = (
argv: string[],
opts: { cwd: string; env: Record<string, string | undefined> },
) => OpencodeProbeHandle;
function defaultOpencodeProbeSpawn(
argv: string[],
opts: { cwd: string; env: Record<string, string | undefined> },
): OpencodeProbeHandle {
const proc = Bun.spawn(argv, {
cwd: opts.cwd,
env: opts.env as Record<string, string>,
stdin: 'ignore',
stdout: 'pipe',
stderr: 'pipe',
});
return {
exited: proc.exited,
kill: (force?: boolean) => {
try {
proc.kill(force ? 9 : undefined);
} catch {
/* already dead */
}
},
stdout: new Response(proc.stdout).text().catch(() => ''),
stderr: new Response(proc.stderr).text().catch(() => ''),
unref: () => {
try {
proc.unref();
} catch {
/* best-effort */
}
},
};
}
/** Run the opencode registration probe with OPENCODE_DISABLE_AUTOUPDATE=1 on
* the spawn env (OPENCODE-CLI-PIN.md §Probes: the auto-updater must never
* fire mid-probe) from an explicit `cwd` callers pass a fresh EMPTY temp
* dir, never the invoking cwd, because opencode merges a project
* opencode.json from cwd and spawns its local servers with NO trust prompt
* (a cloned malicious repo must not get code execution out of an install
* probe). On timeout the child is killed (SIGTERM SIGKILL) and the pipes
* are drained BOUNDED (a spawned MCP-server grandchild can inherit the pipe
* fds and hold them open past the direct child's death). Exported for the
* timeout-kill unit test. */
export async function runOpencodeProbe(
argv: string[],
opts: { cwd: string; spawn?: OpencodeProbeSpawn; timeoutMs?: number },
): Promise<{ code: number; stdout: string; stderr: string }> {
const spawnFn = opts.spawn ?? defaultOpencodeProbeSpawn;
const timeoutMs = opts.timeoutMs ?? OPENCODE_PROBE_TIMEOUT_MS;
const env: Record<string, string | undefined> = { ...process.env, OPENCODE_DISABLE_AUTOUPDATE: '1' };
let handle: OpencodeProbeHandle;
try {
handle = spawnFn(argv, { cwd: opts.cwd, env });
} catch (e) {
// Bun.spawn throws synchronously when the binary is absent — map to the
// shell's 127 convention so the caller's not-on-PATH branch fires.
return { code: 127, stdout: '', stderr: e instanceof Error ? e.message : String(e) };
}
// Bounded race helper that never leaves a live timer holding the loop.
const raceMs = async <T>(p: Promise<T>, ms: number, fallback: T): Promise<T> => {
let timer: ReturnType<typeof setTimeout> | undefined;
try {
return await Promise.race([p, new Promise<T>((res) => { timer = setTimeout(() => res(fallback), ms); })]);
} finally {
clearTimeout(timer);
}
};
let code = await raceMs<number | null>(handle.exited, timeoutMs, null);
const timedOut = code === null;
if (code === null) {
handle.kill(); // graceful first — opencode tears its servers down on TERM
code = await raceMs<number | null>(handle.exited, 2_000, null);
if (code === null) {
handle.kill(true); // SIGKILL is not refusable; the wait below is paranoia-bounded
code = await raceMs<number | null>(handle.exited, 2_000, null);
}
}
const drainCap = timedOut ? 2_000 : 5_000;
const [stdout, stderr] = await Promise.all([
raceMs(handle.stdout, drainCap, ''),
raceMs(handle.stderr, drainCap, ''),
]);
if (timedOut || code === null) {
handle.unref?.(); // a grandchild may still hold the pipes — never hold the CLI's exit
return { code: 124, stdout, stderr: stderr || `timeout after ${timeoutMs}ms` };
}
return { code, stdout, stderr };
}
async function withLock<T>(ws: string, fn: () => Promise<T>): Promise<T> {
const handle = await acquireBootstrapLock(ws);
try {
@@ -747,11 +920,21 @@ async function runRepo(ws: string, rest: string[], home: string, runner: ExecRun
});
}
async function runHooks(ws: string, rest: string[], home: string, runner: ExecRunner): Promise<number> {
const harnessFlag = flagValue(rest, '--harness') as Harness | undefined;
const harness = harnessFlag ?? detectHarness();
if (!harness || (harness !== 'claude-code' && harness !== 'codex')) {
console.error('cannot auto-detect the harness — pass --harness claude-code or --harness codex');
async function runHooks(
ws: string,
rest: string[],
home: string,
runner: ExecRunner,
probeSpawn?: OpencodeProbeSpawn,
): Promise<number> {
const harnessFlag = flagValue(rest, '--harness');
const harness = isHarness(harnessFlag) ? harnessFlag : harnessFlag ? null : detectHarness();
if (!harness) {
console.error(
harnessFlag
? `unknown --harness '${harnessFlag}' — pass --harness claude-code, codex, or opencode`
: 'cannot auto-detect the harness — pass --harness claude-code, codex, or opencode',
);
return 2;
}
// --repair is an idempotent-run alias: the same registration/write path as a
@@ -783,18 +966,44 @@ async function runHooks(ws: string, rest: string[], home: string, runner: ExecRu
return 2;
}
const mcpScope = ((consentAnswer(ws, 'MCP_SCOPE') ?? 'project').toLowerCase() === 'user' ? 'user' : 'project') as 'project' | 'user';
// Raw (unbanked) MCP_SCOPE answer — several harness branches need to know
// whether a human EXPLICITLY chose a scope vs the bank default filling in.
// typeof guard: readInterviewState validates `answers` is an object but not
// per-answer shapes — a hand-edited value of 3 must not throw.
const rawScopeAnswer = (() => {
const read = readInterviewState(ws);
const raw = read.ok ? read.state.answers['MCP_SCOPE'] : undefined;
// .trim(): a hand-edited or sloppily-recorded ' project' must not
// silently resolve to the user-global default (scope answers are
// security-relevant on opencode).
return raw?.skipped !== true && typeof raw?.value === 'string' ? raw.value.trim().toLowerCase() : undefined;
})();
// Scope resolution is per-harness (exhaustive switch — see HARNESSES):
// - claude-code: consent answer, bank default 'project' (the privacy-safe
// default: any other repo you open cannot read the brain).
// - codex: no scope flag exists; the value is ignored (note below).
// - opencode: default 'user' — OPPOSITE of claude-code, because opencode
// spawns project-config-defined servers with NO trust gate (verified,
// OPENCODE-CLI-PIN.md §Probes): a committed project entry would auto-spawn
// on every collaborator's machine. 'project' only via an EXPLICIT answer
// (the sharing warning prints at write time).
const mcpScope = ((): 'project' | 'user' => {
switch (harness) {
case 'claude-code':
return (consentAnswer(ws, 'MCP_SCOPE') ?? 'project').toLowerCase() === 'user' ? 'user' : 'project';
case 'codex':
return 'project'; // ignored — codex registrations are user-global (no scope flag)
case 'opencode':
return rawScopeAnswer === 'project' ? 'project' : 'user';
}
})();
// A persisted 'project' answer is meaningless on Codex (`codex mcp add` has no
// scope flag) — reachable via attach from a Claude Code machine or a pre-fix
// install. Fires on each hooks/repair run while the stale answer persists.
// Raw read, NOT consentAnswer: the bank default is 'project', so the resolved
// value would fire this note on every Codex install where no one was asked.
if (harness === 'codex') {
const read = readInterviewState(ws);
const raw = read.ok ? read.state.answers['MCP_SCOPE'] : undefined;
// typeof guard: readInterviewState validates `answers` is an object but not
// per-answer shapes — a hand-edited value of 3 must not throw.
if (raw?.skipped !== true && typeof raw?.value === 'string' && raw.value.toLowerCase() === 'project') {
if (rawScopeAnswer === 'project') {
console.error(
"note: the recorded MCP_SCOPE answer 'project' has no effect on Codex — " +
'`codex mcp add` has no scope flag; the registration is user-global (any repo ' +
@@ -812,6 +1021,40 @@ async function runHooks(ws: string, rest: string[], home: string, runner: ExecRu
const hooksConsent = !noHooks && (consentAnswer(ws, 'HOOKS_CONSENT') ?? 'yes').toLowerCase() === 'yes';
const gbrainHome = process.env.GBRAIN_HOME?.trim() || undefined;
// One owner per codex server name: if the harness lane's managed TOML block
// owns [mcp_servers.gbrain], this stdio registration must not fight it —
// the FIX7 mismatch path would `codex mcp remove` the harness's server and
// strand orphan marker comments (#4043 ownership rule).
if (harness === 'codex' && codexBlockOwnsName(codexConfigPath(), 'gbrain')) {
console.log(
"the 'gbrain' codex MCP server is managed by `gbrain bootstrap harness` (marker block in the codex " +
'config) — skipping the stdio registration. Run `gbrain bootstrap harness --remove` first if you ' +
'want this workspace-lane stdio registration instead.',
);
return 0;
}
// Same ownership rule, opencode spelling: a REMOTE-type mcp.gbrain in the
// user-global config is either the harness lane's (inline bearer) or
// foreign — the stdio lane must not fight it in either case. BOTH global
// filenames are checked: opencode merges opencode.json AND opencode.jsonc
// when both exist, so a remote entry in EITHER file owns the name even
// when the path resolver would pick the other for writing.
if (
harness === 'opencode' &&
mcpScope === 'user' &&
[join(opencodeConfigDir(), 'opencode.jsonc'), join(opencodeConfigDir(), 'opencode.json')].some((p) =>
opencodeRemoteEntryExists(p, 'gbrain'),
)
) {
console.log(
"the 'gbrain' opencode MCP entry in the user-global config is a remote server (managed by " +
'`gbrain bootstrap harness`, or foreign) — skipping the stdio registration. Run ' +
'`gbrain bootstrap harness --remove` first (or remove the entry) if you want this ' +
'workspace-lane stdio registration instead.',
);
return 0;
}
return withLock(ws, async () => {
// 0. source_id visibility seam: `hooks` is the last ENGINE-FREE phase
// before `verify` (which alone can detect a source_id collision — the
@@ -855,6 +1098,157 @@ async function runHooks(ws: string, rest: string[], home: string, runner: ExecRu
// binary. The old early-return silently dropped hooks while the copy said
// only "MCP registration skipped".
let mcpSkipped = false;
if (harness === 'opencode') {
// Direct-writer lane (no exec): registrations land via the JSONC
// writer whose 4-state fingerprint is the [FIX7] check. Scope resolves
// to a FILE here — user → global config (absolute binary path),
// project → committed-candidate opencode.json (PATH-resolved command;
// no absolute machine paths in a file that travels, and no fail-open
// analog exists — the sharing warning below is the mitigation).
const configPath = mcpScope === 'project' ? opencodeProjectConfigPath(ws) : opencodeGlobalConfigPath();
const command =
mcpScope === 'project'
? ['gbrain', 'serve', '--surface', 'full']
: [gbrainBin, 'serve', '--surface', 'full'];
const entry = {
kind: 'local' as const,
name: 'gbrain',
command,
environment: { GBRAIN_SOURCE: sourceId, ...(gbrainHome ? { GBRAIN_HOME: gbrainHome } : {}) },
};
try {
// [X11] config-dir lock parity with the harness lane: the user-global
// config is shared across workspaces AND homes, so gbrain writers
// serialize on ITS directory. The project-scope file lives in the
// workspace root, which withLock(ws) already holds — the lock is
// non-reentrant, so the same-dir case skips the nested acquire.
const ocCfgDir = dirname(configPath);
let ocLock: { release(): void } | null = null;
if (resolve(ocCfgDir) !== resolve(ws)) {
mkdirSync(ocCfgDir, { recursive: true }); // the lock needs the dir; the writer mkdirs later anyway
ocLock = await acquireBootstrapLock(ocCfgDir);
}
let w: ReturnType<typeof writeOpencodeMcpEntry>;
try {
// [FIX7] parity: an existing entry pointing at a DIFFERENT workspace
// is warned about and replaced (same behavior as the exec lanes'
// mismatch path); a FOREIGN entry refuses inside the writer. The
// pre-check parse carries the same paste-by-hand snippet the writer
// uses so a corrupt config never strands the user.
const existingText = existsSync(configPath) ? readFileSync(configPath, 'utf8') : '';
const existingKind = opencodeEntryKind(
parseOpencodeConfig(existingText, configPath, opencodeEntrySnippet(entry)),
'gbrain',
{ sourceId },
);
if (existingKind === 'ours-other-source') {
console.error(`existing 'gbrain' opencode entry targets a DIFFERENT workspace — replacing it.`);
}
// Two-filename merge blind spot: opencode merges BOTH user-global
// filenames, so a same-name gbrain entry in the SIBLING file would
// survive this write as a shadow registration. Reconcile it under
// the same config-dir lock (ours → removed with a note; foreign →
// refuse loudly naming both files). User scope only — the project
// file has no observed sibling semantics.
if (mcpScope === 'user') {
const sib = reconcileOpencodeSiblingGlobal(configPath, 'gbrain', { sourceId });
for (const note of sib.notes) console.error(note);
}
w = writeOpencodeMcpEntry(configPath, entry, {
expect: { sourceId },
allowReplaceOtherSource: true,
});
} finally {
ocLock?.release();
}
console.log(
`MCP registered with opencode (scope: ${mcpScope === 'project' ? 'project (explicit opt-in)' : 'user-global'}) — ` +
`wrote ${w.configPath}${w.replacedPrior ? ' (replaced prior gbrain entry)' : ''}; ` +
'restart opencode (config is read at session start).',
);
for (const note of w.notes) console.error(note);
if (mcpScope === 'project') {
console.error(
'SHARING WARNING: opencode spawns project-config-defined MCP servers with NO trust prompt — ' +
'if this opencode.json is committed, every collaborator machine will spawn gbrain (teammates ' +
'without gbrain see a failing spawn each session; teammates WITH gbrain attach THEIR host ' +
'brain to this repo). The command is PATH-resolved ("gbrain" — requires gbrain on PATH); ' +
'the teammate opt-out is `"enabled": false` on the entry. The user-global default avoids all of this.' +
(gbrainHome
? ` Also: the entry embeds this machine's GBRAIN_HOME path (${gbrainHome}) — it won't be portable to other machines.`
: ''),
);
} else if (rawScopeAnswer === undefined) {
console.log(
"scope defaulted to user-global — opencode spawns project-defined servers with no trust gate, " +
'so the committed-file scope is explicit-opt-in only (record MCP_SCOPE=project to choose it).',
);
}
} catch (e) {
console.error((e as Error).message);
return 1;
}
// Registration smoke: the writer's post-render validation already
// proved the config parses and carries exactly our entry (that is the
// authoritative check). Best-effort live probe when the binary is on
// PATH: `opencode mcp list` SPAWNS servers (the honest discriminator)
// — run it with --pure (no external plugin autoload; `mcp list` is a
// code-execution surface otherwise) and skip it entirely when a
// plugin-bearing config is present (OPENCODE-CLI-PIN.md §Probes).
try {
const parsedCfg = parseOpencodeConfig(
existsSync(configPath) ? readFileSync(configPath, 'utf8') : '',
configPath,
);
if (mcpScope === 'project') {
// SECURITY: opencode merges the project opencode.json from the
// probe's cwd and spawns its local servers with NO trust prompt —
// running `mcp list` inside this workspace would execute whatever
// the (possibly just-cloned) repo's config names. Parse-back stays
// the authoritative check; the human runs the live probe.
console.log(
'live `opencode mcp list` probe skipped for project scope — config parse-back is authoritative; ' +
'run `opencode mcp list` yourself in this workspace to confirm.',
);
} else if (parsedCfg.plugin !== undefined) {
console.log('live `opencode mcp list` probe skipped (plugin-bearing config) — config parse-back is the verification.');
} else {
// SECURITY: the probe spawns from a fresh EMPTY temp dir, never the
// invoking cwd — no project opencode.json can load there (the same
// no-trust-prompt spawn surface as the project-scope skip above).
const probeCwd = mkdtempSync(join(tmpdir(), 'gbrain-opencode-probe-'));
let probe: { code: number; stdout: string; stderr: string };
try {
probe = await runOpencodeProbe(['opencode', 'mcp', 'list', '--pure'], {
cwd: probeCwd,
...(probeSpawn ? { spawn: probeSpawn } : {}),
});
} finally {
rmSync(probeCwd, { recursive: true, force: true });
}
// `mcp list` colorizes when a TTY-ish env leaks through — strip ANSI
// escapes before matching, and anchor the name on whitespace/EOL so
// a `gbrain-remote` entry can never satisfy a bare \bgbrain\b (\b
// matches before the hyphen).
const plain = probe.stdout.replace(/\u001b\[[0-9;]*m/g, '');
if (probe.code === 127) {
console.log('`opencode` is not on PATH — registration written; the config activates when opencode next starts here.');
} else if (probe.code === 0 && /✓\s+gbrain(\s|$)/.test(plain)) {
console.log('`opencode mcp list` handshake: ✓ gbrain connected.');
} else if (probe.code === 0 && /✗\s+gbrain(\s|$)/.test(plain)) {
console.error(
'WARNING: `opencode mcp list` reports ✗ gbrain failed — the spawn did not handshake ' +
'(is the gbrain binary path valid on this machine?). The exit code of `mcp list` is 0 even ' +
'on failure; this warning is from parsing its output.',
);
} else {
console.log('MCP registration written; could not confirm via `opencode mcp list` (best-effort probe).');
}
}
} catch {
/* smoke is best-effort */
}
} else {
const argvs =
harness === 'claude-code'
? registerClaudeMcp({ gbrainBin, scope: mcpScope, sourceId, ...(gbrainHome ? { gbrainHome } : {}) })
@@ -963,6 +1357,7 @@ async function runHooks(ws: string, rest: string[], home: string, runner: ExecRu
} catch {
/* smoke is best-effort */
}
} // end exec-lane registration (claude-code / codex)
// 3. Hooks (Claude Code only, consent-gated).
let hooksWritten = false;
@@ -1010,16 +1405,31 @@ async function runHooks(ws: string, rest: string[], home: string, runner: ExecRu
: 'hooks declined (HOOKS_CONSENT set to no) — the AGENTS.md pull protocol covers per-turn context instead; re-enable with `gbrain bootstrap hooks --harness claude-code`.',
);
}
} else if (harness === 'codex') {
console.log('gbrain does not wire Codex hooks yet — per-turn context is the AGENTS.md pull protocol (stated plainly; the codex hook lane is a filed follow-up).');
} else {
console.log('Codex has no hook system — per-turn context is the AGENTS.md pull protocol (stated plainly, not a bug).');
console.log(
'gbrain does not wire opencode\'s plugin/event system yet — per-turn context is the AGENTS.md ' +
'pull protocol, which opencode loads natively (the opencode plugin lane is a filed follow-up).',
);
}
// 4. Receipt registration record [CX2-12]. Detail records what actually
// landed; nothing landed at all (127 + no hooks) → no receipt entry.
if (!mcpSkipped || hooksWritten) {
const receiptScope = ((): string => {
switch (harness) {
case 'claude-code':
return mcpScope;
case 'codex':
return 'user'; // codex registrations are always user-global
case 'opencode':
return mcpScope; // user default; project only via explicit opt-in
}
})();
appendReceiptRegistration(home, ws, {
host: harness,
scope: harness === 'claude-code' ? mcpScope : 'user',
scope: receiptScope,
detail: hooksWritten ? (mcpSkipped ? 'hooks' : 'mcp+hooks') : 'mcp',
});
}
@@ -1111,11 +1521,74 @@ export function workspaceBrainStats(ws: string): { sources: string[]; pages: num
return { sources, pages };
}
/** `gbrain bootstrap harness` (#4043) machine-level, no workspace, no
* agent.json. Locks on the gbrain HOME (there is no workspace to lock). */
async function runHarness(rest: string[], home: string, runner: ExecRunner): Promise<number> {
const flags = parseHarnessArgs(rest);
if (flags.error) {
console.error(flags.error);
return 2;
}
const deps: HarnessDeps = {
runner,
gbrainHome: home,
// Fallback only — the flag itself is parsed (and error-checked) once, by
// parseHarnessArgs; flags.gbrainBin wins inside applyHarness.
gbrainBin: resolveGbrainBin(),
isTTY: process.stdout.isTTY === true,
prompt: promptLine,
};
// [X12] --status is READ-ONLY: no home mkdir, no lock — it must work (and
// stay side-effect-free) even while an apply/remove holds the mutex.
if (flags.status) {
return statusHarness(flags, deps);
}
ensureHarnessHome(home);
return withLock(home, async () => {
if (flags.remove) {
const code = await removeHarness(flags, deps);
abortIfInjected('harness');
return code;
}
const code = await applyHarness(flags, deps);
abortIfInjected('harness');
return code;
});
}
async function runUninstall(ws: string, rest: string[], home: string, runner: ExecRunner): Promise<number> {
const deleteBrain = rest.includes('--delete-brain');
const yes = rest.includes('--yes');
const homeFlag = flagValue(rest, '--home');
const effectiveHome = homeFlag ? resolve(homeFlag) : home;
return withLock(ws, async () => {
// The HOME lock (runHarness's mutex) is held across the ENTIRE uninstall
// body — not just the harness-removal step — so a concurrent
// `bootstrap harness` apply can never mint+wire in the window between
// harness removal and the workspace teardown's rm of <home>/bootstrap
// (which would strand a fresh receipt + live wiring). Consistent order
// (ws → home), distinct dirs, so no deadlock; same-dir configs skip the
// nested acquire (the lock is non-reentrant).
const body = async (): Promise<number> => {
// Harness wiring is removed FIRST (#4043 ordering, load-bearing twice
// over: the token revoke needs the DB alive, and --delete-brain rmSyncs
// <home>/bootstrap — which would destroy harness.json unconsumed).
const harnessState = readHarnessReceiptState(effectiveHome);
let harnessRemoved = false;
if (harnessState.state !== 'absent') {
console.log('harness wiring detected — removing it first (token revoke needs the brain alive).');
const flags = parseHarnessArgs(['--remove', ...(yes ? ['--yes'] : [])]);
const code = await removeHarness(flags, { runner, gbrainHome: effectiveHome });
if (code !== 0) {
console.error(
'harness removal did not fully converge — stopping BEFORE workspace teardown so the harness ' +
'receipt is never stranded. Fix the reported issue (or stop the live serve) and re-run.',
);
return 1;
}
harnessRemoved = true;
}
if (deleteBrain) {
// Facts-export offer BEFORE any deletion can run — facts are user
// knowledge, not derived state; after the rm there is nothing to export.
@@ -1127,25 +1600,91 @@ async function runUninstall(ws: string, rest: string[], home: string, runner: Ex
// the durability teardown below needs it and the manifest may not survive.
const preState = readManifest(ws);
const durabilitySourceId = preState.state === 'initialized' ? preState.manifest.source_id : 'workspace';
const result = await uninstallWorkspace(ws, {
deleteBrain,
...(yes ? { confirm: async () => true } : {}),
gbrainHomeDir: homeFlag ? resolve(homeFlag) : home,
homeExplicit: homeFlag !== undefined,
brainStats: async () => workspaceBrainStats(ws),
});
let result;
try {
result = await uninstallWorkspace(ws, {
deleteBrain,
...(yes ? { confirm: async () => true } : {}),
gbrainHomeDir: effectiveHome,
homeExplicit: homeFlag !== undefined,
brainStats: async () => workspaceBrainStats(ws),
});
} catch (e) {
// A harness-only box has machine-level wiring but no workspace install:
// the pre-teardown refusals that mean "this workspace isn't the
// bootstrapped one" end the run as success once harness removal ran.
// LIVE_SERVE and everything else stay hard refusals.
if (
harnessRemoved &&
e instanceof BootstrapError &&
(e.code === 'NO_RECEIPT' || e.code === 'HOME_GUARD' || e.code === 'RECEIPT_MISMATCH')
) {
console.log(`no workspace install on this machine (naming the refusal: ${e.code}); harness wiring removed.`);
abortIfInjected('uninstall');
return 0;
}
throw e;
}
// Execute the structured host-registration removals the module returned.
for (const reg of result.registration_removals) {
if (reg.host === 'claude-code') {
const r = removeClaudeHooks(ws);
if (r.removed > 0) console.log(`removed ${r.removed} gbrain hook entr${r.removed === 1 ? 'y' : 'ies'} from ${r.settingsPath}`);
for (const note of r.notes) console.error(note);
const rm = await runner(['claude', 'mcp', 'remove', 'gbrain']);
if (rm.code !== 0) console.error('note: `claude mcp remove gbrain` did not succeed — remove it by hand if it lingers.');
} else {
const rm = await runner(['codex', 'mcp', 'remove', 'gbrain']);
if (rm.code !== 0) console.error('note: `codex mcp remove gbrain` did not succeed — remove it by hand if it lingers.');
switch (reg.host) {
case 'claude-code': {
const r = removeClaudeHooks(ws);
if (r.removed > 0) console.log(`removed ${r.removed} gbrain hook entr${r.removed === 1 ? 'y' : 'ies'} from ${r.settingsPath}`);
for (const note of r.notes) console.error(note);
const rm = await runner(['claude', 'mcp', 'remove', 'gbrain']);
if (rm.code !== 0) console.error('note: `claude mcp remove gbrain` did not succeed — remove it by hand if it lingers.');
break;
}
case 'codex': {
const rm = await runner(['codex', 'mcp', 'remove', 'gbrain']);
if (rm.code !== 0) console.error('note: `codex mcp remove gbrain` did not succeed — remove it by hand if it lingers.');
break;
}
case 'opencode': {
// Direct-writer removal (fingerprint-keyed; foreign entries refuse
// inside the module). Every candidate file best-effort — the
// receipt's scope names where the registration landed, but a stale
// entry in another file costs nothing to sweep. BOTH global
// filenames are swept: opencode merges opencode.json AND
// opencode.jsonc when both exist, so sweeping only the resolver's
// pick would strand a gbrain entry in the other file. The removal
// is expectation-keyed on THIS workspace's source id — a gbrain
// entry from a DIFFERENT workspace is skipped with a note, never
// silently deleted (it is not this uninstall's to remove).
const sweep = (p: string): void => {
try {
const r = removeOpencodeMcpEntry(p, 'gbrain', { sourceId: durabilitySourceId }, { skipOtherSource: true });
if (r.removed) console.log(`removed the gbrain opencode MCP entry from ${p}`);
for (const note of r.notes) console.error(note);
} catch (e) {
console.error(`note: could not remove the gbrain opencode entry from ${p}: ${(e as Error).message}`);
}
};
// Global files run under the config-dir bootstrap lock (the writer
// contract; harness.ts [X11] parity). Only when the dir exists — no
// dir means no config, and uninstall must not create one just to
// lock it.
const ocDir = opencodeConfigDir();
const globals = [join(ocDir, 'opencode.jsonc'), join(ocDir, 'opencode.json')].filter((p) => existsSync(p));
if (globals.length > 0) {
try {
const ocLock = await acquireBootstrapLock(ocDir);
try {
for (const p of globals) sweep(p);
} finally {
ocLock.release();
}
} catch (e) {
console.error(`note: could not lock the opencode config dir (${(e as Error).message}) — entries left for a re-run.`);
}
}
// The project file's dir IS the workspace, which withLock(ws)
// already holds — the lock is non-reentrant, so no nested acquire.
sweep(opencodeProjectConfigPath(ws));
break;
}
}
}
@@ -1196,6 +1735,8 @@ async function runUninstall(ws: string, rest: string[], home: string, runner: Ex
console.log('The workspace repo and its files remain yours — the body is portable by design.');
abortIfInjected('uninstall');
return 0;
};
return resolve(effectiveHome) === resolve(ws) ? body() : withLock(effectiveHome, body);
});
}
@@ -1204,6 +1745,9 @@ async function runUninstall(ws: string, rest: string[], home: string, runner: Ex
export interface RunBootstrapOpts {
/** Exec seam for gh/claude/codex subprocesses (tests inject a recorder). */
runner?: ExecRunner;
/** Spawn seam for the opencode `mcp list` probe (tests capture argv, cwd,
* and env; the default holds a real Bun.spawn handle so timeouts kill). */
probeSpawn?: OpencodeProbeSpawn;
}
/** Dispatch. Returns the process exit code (cli.ts passes it to setCliExitVerdict). */
@@ -1221,7 +1765,7 @@ export async function runBootstrap(args: string[], opts: RunBootstrapOpts = {}):
const logCtx: LogCtx = { home, ws, ...(harnessForLog ? { harness: harnessForLog } : {}) };
const t0 = Date.now();
const KNOWN = new Set(['status', 'interview', 'render', 'repo', 'hooks', 'verify', 'attach', 'uninstall', 'cloud-setup-script']);
const KNOWN = new Set(['status', 'interview', 'render', 'repo', 'hooks', 'verify', 'attach', 'uninstall', 'harness', 'cloud-setup-script']);
if (!KNOWN.has(sub)) {
console.error(`unknown subcommand: ${sub}`);
console.error(BOOTSTRAP_HELP);
@@ -1233,13 +1777,17 @@ export async function runBootstrap(args: string[], opts: RunBootstrapOpts = {}):
// interview) never falls through into the real operation, regardless of
// what other flags/values precede it in `rest`. No install-log entry
// either — this isn't a phase run.
if (SUBCOMMAND_HELP[sub] && hasHelpToken(rest, sub !== 'interview')) {
if (Object.hasOwn(SUBCOMMAND_HELP, sub) && hasHelpToken(rest, sub !== 'interview')) {
// Object.hasOwn: a plain-object lookup resolves inherited keys, so
// `bootstrap constructor --help` would print Object.prototype.constructor.
console.log(SUBCOMMAND_HELP[sub]);
return 0;
}
// The install log records the PHASE name, and the hooks subcommand is the
// 'wire' phase (status.ts phase list) — one mapping, used at every log site.
// 'harness' is its own log phase (NOT a status.ts phase — that list is
// CI-pinned; install.jsonl phase names are free-form telemetry).
const logPhaseName = sub === 'hooks' ? 'wire' : sub;
try {
@@ -1265,7 +1813,7 @@ export async function runBootstrap(args: string[], opts: RunBootstrapOpts = {}):
code = await runRepo(ws, rest, home, runner);
break;
case 'hooks':
code = await runHooks(ws, rest, home, runner);
code = await runHooks(ws, rest, home, runner, opts.probeSpawn);
break;
case 'verify':
code = await runVerify(ws, rest, home);
@@ -1276,6 +1824,9 @@ export async function runBootstrap(args: string[], opts: RunBootstrapOpts = {}):
case 'uninstall':
code = await runUninstall(ws, rest, home, runner);
break;
case 'harness':
code = await runHarness(rest, home, runner);
break;
default:
return 2; // unreachable
}
+13 -1
View File
@@ -28,11 +28,15 @@ import { parseProgressEvents, verifyExpectedPhases } from '../core/claw-test/pro
import { resolveAgentRunner, listRegisteredAgents, registerAgentRunner, validateBinPathEnv } from '../core/claw-test/agent-runner.ts';
import { OpenClawRunner } from '../core/claw-test/runners/openclaw.ts';
import { HermesRunner } from '../core/claw-test/runners/hermes.ts';
import { GrokRunner } from '../core/claw-test/runners/grok.ts';
import { OpencodeRunner } from '../core/claw-test/runners/opencode.ts';
import { createTranscriptSink } from '../core/claw-test/transcript-capture.ts';
// Ensure built-in runners are registered.
registerAgentRunner('openclaw', () => new OpenClawRunner());
registerAgentRunner('hermes', () => new HermesRunner());
registerAgentRunner('grok', () => new GrokRunner());
registerAgentRunner('opencode', () => new OpencodeRunner());
interface HarnessOpts {
scenario: string;
@@ -412,6 +416,12 @@ async function runScripted(
const AGENT_INSTALL_HINTS: Record<string, string> = {
openclaw: 'install openclaw or set OPENCLAW_BIN',
hermes: 'install hermes (https://hermes-agent.nousresearch.com) or set HERMES_BIN',
// Official xAI CLI only — the community superagent-ai grok-cli ships a
// colliding `grok` binary (docs/mcp/GROK-CLI-PIN.md).
grok: 'install grok (npm: @xai-official/grok, or https://x.ai/cli/install.sh) or set GROK_BIN',
// SST terminal agent — not OpenClaw, and not the renamed-to-Crush ancestor
// that shares the binary name (docs/mcp/OPENCODE-CLI-PIN.md).
opencode: 'install opencode (npm: opencode-ai, or https://opencode.ai/install) or set OPENCODE_BIN',
};
/**
@@ -998,5 +1008,7 @@ Examples:
gbrain claw-test --scenario fresh-install
gbrain claw-test --scenario upgrade-from-v0.18 --keep-tempdir
gbrain claw-test --live --agent openclaw
gbrain claw-test --live --agent hermes`);
gbrain claw-test --live --agent opencode
gbrain claw-test --live --agent hermes
gbrain claw-test --live --agent grok`);
}
+176 -148
View File
@@ -9,7 +9,7 @@
* needed for the connection.
*
* gbrain connect <mcp-url> [--token <bearer>] [--name gbrain]
* [--agent claude-code|codex|perplexity|generic]
* [--agent claude-code|codex|opencode|perplexity|generic]
* [--oauth [--register | --client-id ID --client-secret SECRET] [--scopes "read write"]]
* [--install] [--yes] [--json] [--show-token] [--force]
* [--timeout-ms N]
@@ -27,32 +27,79 @@
* only; --install runs it).
* - codex: `codex mcp add <name> --url <url> --bearer-token-env-var
* GBRAIN_REMOTE_TOKEN` (bearer via env var; --install runs it).
* - opencode: `opencode mcp add <name> --url <url> --header
* "Authorization=Bearer {env:GBRAIN_REMOTE_TOKEN}"` (the interpolation is
* stored literally; --install writes the entry directly via
* opencode-json.ts no binary needed).
* - perplexity: GUI connector (Settings Connectors). Supports bearer or
* OAuth; no --install.
* - generic: prints the connector fields for any other MCP client.
*/
import { execFileSync } from 'child_process';
import { mkdirSync } from 'node:fs';
import { dirname } from 'node:path';
import type { ConnectProbeResult } from '../core/connect-probe.ts';
import { probeBrainIdentity, DEFAULT_PROBE_TIMEOUT_MS } from '../core/connect-probe.ts';
import { opencodeGlobalConfigPath } from '../core/bootstrap/host-specs.ts';
import { acquireBootstrapLock } from '../core/bootstrap/lock.ts';
import {
GBRAIN_REMOTE_TOKEN_ENV,
reconcileOpencodeSiblingGlobal,
writeOpencodeMcpEntry,
} from '../core/bootstrap/opencode-json.ts';
import { promptLine } from '../core/cli-util.ts';
import {
NAME_RE,
REDACTED,
buildClaudeMcpAddArgv,
buildCodexMcpAddArgv,
buildOpencodeMcpAddArgv,
cmdString,
isValidName,
issuerFromMcpUrl,
normalizeMcpUrl,
redactToken,
shellQuote,
validateToken,
} from '../core/mcp-registration.ts';
export const ENV_VAR = 'GBRAIN_REMOTE_TOKEN';
// The pure registration helpers moved to src/core/mcp-registration.ts for
// #4043 (the bootstrap harness lane consumes them; core must not import from
// commands). Re-exported so this module's public surface — and every test
// that imports from it — is unchanged.
export {
REDACTED,
buildClaudeMcpAddArgv,
buildCodexMcpAddArgv,
buildOpencodeMcpAddArgv,
cmdString,
isLinkLocalOrMetadata,
issuerFromMcpUrl,
isValidName,
normalizeMcpUrl,
redactToken,
validateToken,
type TokenValidation,
type UrlResult,
} from '../core/mcp-registration.ts';
// Defined from the writer's exported constant so the printed interpolation and
// the ownership fingerprint literal ({env:GBRAIN_REMOTE_TOKEN}) cannot drift.
export const ENV_VAR = GBRAIN_REMOTE_TOKEN_ENV;
export const PLACEHOLDER_TOKEN = '<paste-your-token>';
export const PLACEHOLDER_SECRET = '<paste-your-client-secret>';
export const REDACTED = '***';
export const DEFAULT_NAME = 'gbrain';
export const DEFAULT_SCOPES = 'read write';
const NAME_RE = /^[a-z0-9][a-z0-9_-]*$/;
// Single source of truth shared with the probe (was a duplicated 15_000 literal).
const DEFAULT_TIMEOUT_MS = DEFAULT_PROBE_TIMEOUT_MS;
export type AgentId = 'claude-code' | 'codex' | 'perplexity' | 'generic';
export type AgentId = 'claude-code' | 'codex' | 'opencode' | 'perplexity' | 'generic';
interface AgentSpec {
id: AgentId;
label: string; // human label for messages
binary?: string; // CLI binary backing --install ('claude' | 'codex')
binary?: string; // CLI binary backing --install ('claude' | 'codex'; opencode installs via the direct JSONC writer)
installable: boolean;
supportsOAuth: boolean; // accepts OAuth client-credentials connector fields
}
@@ -60,11 +107,14 @@ interface AgentSpec {
export const AGENT_SPECS: Record<AgentId, AgentSpec> = {
'claude-code': { id: 'claude-code', label: 'Claude Code', binary: 'claude', installable: true, supportsOAuth: false },
codex: { id: 'codex', label: 'Codex', binary: 'codex', installable: true, supportsOAuth: false },
// No `binary`: the opencode --install lane never execs a CLI (direct JSONC
// write), and it branches before the exec lane's `spec.binary` read.
opencode: { id: 'opencode', label: 'opencode', installable: true, supportsOAuth: false },
perplexity: { id: 'perplexity', label: 'Perplexity Computer', installable: false, supportsOAuth: true },
generic: { id: 'generic', label: 'your agent', installable: false, supportsOAuth: true },
};
export const AGENT_IDS: AgentId[] = ['claude-code', 'codex', 'perplexity', 'generic'];
export const AGENT_IDS: AgentId[] = ['claude-code', 'codex', 'opencode', 'perplexity', 'generic'];
// The named tools MUST be real MCP-exposed ops (verified by the round-trip
// E2E). `capture` is intentionally absent: it's a CLI-only convenience wrapper,
@@ -97,7 +147,7 @@ Usage:
gbrain connect <mcp-url> [--token <bearer>] [flags]
Prints a copy-paste setup block for your agent, or wires it up directly with
--install (claude-code + codex only). The MCP URL is your remote
--install (claude-code, codex + opencode). The MCP URL is your remote
'gbrain serve --http' endpoint; a bare host is rejected pass an explicit
https:// URL.
@@ -110,14 +160,15 @@ Auth:
Flags:
--token <bearer> Bearer token (else $${ENV_VAR}; from 'gbrain auth create')
--name <id> MCP server name in the agent (default: ${DEFAULT_NAME})
--agent <kind> claude-code (default) | codex | perplexity | generic
--agent <kind> claude-code (default) | codex | opencode | perplexity | generic
--oauth Use OAuth client credentials instead of a bearer token
--register With --oauth: mint a client on the host (gbrain auth register-client)
--client-id <id> With --oauth: use an existing OAuth client id
--client-secret <s> With --oauth: use an existing OAuth client secret
--scopes "<s>" With --oauth --register: client scopes (default: "${DEFAULT_SCOPES}")
--install Run the agent's MCP-add command, then smoke-test the token
(claude-code + codex only)
(claude-code + codex + opencode; opencode installs via a direct
config write no binary needed, token stays out of the file)
--yes Skip the install confirmation prompt
--force On --install, replace an existing server of the same name
--json Emit machine-readable JSON (secret redacted)
@@ -128,114 +179,17 @@ Examples:
gbrain connect https://brain.example.com/mcp --token gbrain_xxx
gbrain connect https://brain.example.com:3131 --install --yes
gbrain connect https://brain.example.com/mcp --token gbrain_xxx --agent codex
gbrain connect https://brain.example.com/mcp --token gbrain_xxx --agent opencode --install
gbrain connect https://brain.example.com/mcp --agent perplexity --oauth --register
gbrain connect https://brain.example.com/mcp --agent perplexity --oauth \\
--client-id gbrain_cl_xxx --client-secret gbrain_cs_xxx
`;
// ---------------------------------------------------------------------------
// Pure helpers (unit-tested in test/connect.test.ts)
// Pure helpers (unit-tested in test/connect.test.ts; registration helpers
// live in src/core/mcp-registration.ts and are re-exported above)
// ---------------------------------------------------------------------------
export type UrlResult =
| { ok: true; url: string; warning?: string }
| { ok: false; error: string };
/**
* Block link-local / cloud-metadata addresses the one class of host that is
* never a legitimate brain endpoint but IS a token-exfil target (e.g. the AWS/
* GCP metadata service at 169.254.169.254). Deliberately does NOT block
* localhost or RFC1918/LAN ranges: self-hosted brains on a private network are
* a documented, supported topology (`gbrain serve --http --bind`).
*/
export function isLinkLocalOrMetadata(hostname: string): boolean {
const h = hostname.toLowerCase().replace(/^\[|\]$/g, '');
if (/^169\.254\.\d{1,3}\.\d{1,3}$/.test(h)) return true; // IPv4 link-local incl. cloud metadata
if (h.startsWith('fe80:')) return true; // IPv6 link-local
if (h === 'fd00:ec2::254') return true; // AWS IMDSv2 over IPv6
// IPv4-mapped IPv6 (e.g. ::ffff:169.254.169.254 dotted, or ::ffff:a9fe:xxxx
// hex where a9fe == 169.254) must not slip past the dotted-IPv4 check.
const mapped = h.match(/^::ffff:(.+)$/);
if (mapped) {
if (/^169\.254\.\d{1,3}\.\d{1,3}$/.test(mapped[1])) return true;
if (mapped[1].startsWith('a9fe:')) return true;
}
return false;
}
/**
* Normalize an MCP URL to a canonical `<scheme>//<host><path>` ending in /mcp.
* Explicit spec (not best-effort) see plan D-codex findings.
*/
export function normalizeMcpUrl(input: string): UrlResult {
const raw = (input ?? '').trim();
if (!raw) {
return { ok: false, error: 'Missing MCP URL. Usage: gbrain connect <https://host/mcp> --token <bearer>' };
}
// Require an explicit scheme. A bare `host:3131` parses as scheme `host:`
// under WHATWG URL, so reject anything without `://`.
if (!/^[a-z][a-z0-9+.-]*:\/\//i.test(raw)) {
const guess = raw.replace(/^\/+/, '');
return { ok: false, error: `Add an explicit scheme, e.g. https://${guess} (a bare host:port is ambiguous).` };
}
let u: URL;
try {
u = new URL(raw);
} catch {
return { ok: false, error: `Invalid URL: ${raw}` };
}
const scheme = u.protocol.toLowerCase();
if (scheme !== 'http:' && scheme !== 'https:') {
return { ok: false, error: `Only http(s) URLs are supported (got ${u.protocol}).` };
}
if (u.username || u.password) {
return { ok: false, error: 'Remove credentials from the URL (user:pass@host is not supported); pass the token via --token.' };
}
if (u.search) {
return { ok: false, error: 'Remove the query string from the MCP URL.' };
}
if (isLinkLocalOrMetadata(u.hostname)) {
return { ok: false, error: `Refusing to target a link-local / cloud-metadata address (${u.hostname}). Point the MCP URL at the brain host's real address.` };
}
const host = u.host; // host:port; hostname already lowercased by URL
const path = u.pathname;
const trimmed = path.replace(/\/+$/, '');
const lower = trimmed.toLowerCase();
let finalPath: string;
if (path === '' || path === '/') {
finalPath = '/mcp';
} else if (lower === '/mcp') {
finalPath = '/mcp';
} else {
return {
ok: false,
error: `Unexpected path '${path}'. Pass the full /mcp URL, e.g. ${scheme}//${host}${trimmed}/mcp`,
};
}
const url = `${scheme}//${host}${finalPath}`;
const hn = u.hostname.toLowerCase();
const isLocal = hn === 'localhost' || hn === '127.0.0.1' || hn === '::1' || hn === '[::1]';
if (scheme === 'http:' && !isLocal) {
return { ok: true, url, warning: 'Warning: http:// sends your bearer token unencrypted. Use https:// unless this is localhost.' };
}
return { ok: true, url };
}
/** The OAuth issuer is the server base — the /mcp endpoint's URL minus /mcp. */
export function issuerFromMcpUrl(url: string): string {
return url.replace(/\/mcp$/, '');
}
export type TokenValidation = { ok: true } | { ok: false; error: string };
/** Reject empty/whitespace/control-char tokens (a newline is a header-injection vector). */
export function validateToken(token: string): TokenValidation {
if (!token || !token.trim()) return { ok: false, error: 'Token is empty.' };
if (/\s/.test(token)) return { ok: false, error: 'Token contains whitespace (space/tab/newline) — refusing (header-injection risk).' };
if (/[\x00-\x1f\x7f]/.test(token)) return { ok: false, error: 'Token contains control characters — refusing (header-injection risk).' };
return { ok: true };
}
export type TokenResolution =
| { kind: 'literal'; token: string }
| { kind: 'placeholder' }
@@ -255,43 +209,6 @@ export function resolveToken(opts: { tokenFlag?: string | null; env?: string | n
};
}
export function isValidName(name: string): boolean {
return NAME_RE.test(name);
}
export function buildClaudeMcpAddArgv(p: { name: string; url: string; headerToken: string }): string[] {
return ['mcp', 'add', p.name, '-t', 'http', p.url, '-H', `Authorization: Bearer ${p.headerToken}`];
}
/** Codex reads the bearer from an env var at runtime — the token is NOT in argv. */
export function buildCodexMcpAddArgv(p: { name: string; url: string; envVar: string }): string[] {
return ['mcp', 'add', p.name, '--url', p.url, '--bearer-token-env-var', p.envVar];
}
/**
* POSIX single-quote any arg that isn't already shell-safe, so `$()`, backticks,
* etc. in a token are inert literals when the block is pasted into a shell
* (double-quoting would still allow command substitution).
*/
function shellQuote(arg: string): string {
if (/^[A-Za-z0-9_.:/@-]+$/.test(arg)) return arg;
return `'${arg.replace(/'/g, "'\\''")}'`;
}
/** Render `<binary> <argv...>` as a copy-pasteable, shell-safe command string. */
export function cmdString(binary: string, argv: string[]): string {
return `${binary} ${argv.map(shellQuote).join(' ')}`;
}
export function redactToken(s: string, token: string | null): string {
// Exact-substring scrub of the known token, plus a defense-in-depth pass over
// any `Bearer <value>` shape the SDK/CLI might echo in a transformed form the
// exact match would miss. Both run on the --install error paths only.
let out = token ? s.split(token).join(REDACTED) : s;
out = out.replace(/Bearer\s+\S+/gi, `Bearer ${REDACTED}`);
return out;
}
export interface OAuthCreds {
issuer: string;
clientId: string;
@@ -330,6 +247,31 @@ function codexBlock(p: { name: string; url: string; token: string | null }): str
return lines.join('\n');
}
function opencodeBlock(p: { name: string; url: string; token: string | null }): string {
const tokenValue = p.token ?? PLACEHOLDER_TOKEN;
const cmd = cmdString('opencode', buildOpencodeMcpAddArgv({ name: p.name, url: p.url, envVar: ENV_VAR }));
const lines = [
'# Paste into opencode:',
'',
'Connect my knowledge brain, then learn what it can do:',
'',
` export ${ENV_VAR}=${shellQuote(tokenValue)}`,
` ${cmd}`,
'',
];
if (!p.token) lines.push(`Replace ${PLACEHOLDER_TOKEN} with a token from \`gbrain auth create "opencode"\` on the host.`, '');
lines.push(
`The config stores the literal \`{env:${ENV_VAR}}\` interpolation — opencode resolves it at read time, ` +
`so keep that variable exported in your shell profile; the token never lands in the config file. ` +
`Restart opencode after registering (config is read at session start).`,
'',
LEARN_INSTRUCTION,
'',
SECRET_NOTE,
);
return lines.join('\n');
}
function perplexityBearerBlock(p: { url: string; token: string | null }): string {
const tokenValue = p.token ?? PLACEHOLDER_TOKEN;
return [
@@ -401,6 +343,7 @@ export function buildConnectBlock(p: { agent: AgentId; name: string; url: string
switch (p.agent) {
case 'claude-code': return claudeBlock(p);
case 'codex': return codexBlock(p);
case 'opencode': return opencodeBlock(p);
case 'perplexity': return perplexityBearerBlock(p);
case 'generic': return genericBearerBlock(p);
}
@@ -435,6 +378,10 @@ export function buildJson(p: { url: string; name: string; agent: AgentId; token:
// Codex command carries no token (env-var name only), so it's safe verbatim.
command_argv = buildCodexMcpAddArgv({ name: p.name, url: p.url, envVar: ENV_VAR });
command = cmdString('codex', command_argv);
} else if (p.agent === 'opencode') {
// The literal {env:VAR} interpolation, not a token — safe verbatim.
command_argv = buildOpencodeMcpAddArgv({ name: p.name, url: p.url, envVar: ENV_VAR });
command = cmdString('opencode', command_argv);
}
return {
schema_version: 1,
@@ -468,6 +415,18 @@ export interface ConnectDeps {
probe(url: string, token: string, timeoutMs: number): Promise<ConnectProbeResult>;
env(name: string): string | undefined;
registerOAuthClient(name: string, scopes: string): RegisterResult;
/** opencode --install lane: direct JSONC write of a remote entry carrying
* the literal `{env:GBRAIN_REMOTE_TOKEN}` interpolation (no binary execed,
* no token on disk). Throws on a foreign same-name entry; an OURS entry at
* a different url refuses unless `allowReplaceOtherSource` (connect maps
* --force onto it). May be async: the default impl serializes on the
* config-dir bootstrap lock (the writer contract); sync test fakes remain
* assignable. */
writeOpencodeRemoteEntry(
name: string,
url: string,
opts?: { allowReplaceOtherSource?: boolean },
): { configPath: string; replacedPrior: boolean } | Promise<{ configPath: string; replacedPrior: boolean }>;
}
async function defaultPromptYesNo(question: string): Promise<boolean> {
@@ -525,6 +484,31 @@ const defaultDeps: ConnectDeps = {
probe: (url, token, timeoutMs) => probeBrainIdentity(url, token, { timeoutMs }),
env: (name) => process.env[name],
registerOAuthClient: defaultRegisterOAuthClient,
writeOpencodeRemoteEntry: async (name, url, opts) => {
// The writer's contract: callers hold acquireBootstrapLock on the config
// dir (harness.ts [X11] parity) — the user-global file is shared across
// workspaces and homes, so concurrent gbrain writers serialize here.
const configPath = opencodeGlobalConfigPath();
const cfgDir = dirname(configPath);
mkdirSync(cfgDir, { recursive: true }); // the lock needs the dir; the writer mkdirs later anyway
const lock = await acquireBootstrapLock(cfgDir);
try {
// Two-filename merge blind spot: opencode merges BOTH user-global
// filenames, so a same-name gbrain entry in the SIBLING file would
// survive this write as a shadow registration (ours → removed with a
// note; foreign → refuse loudly naming both files).
const sib = reconcileOpencodeSiblingGlobal(configPath, name, { url });
for (const note of sib.notes) console.error(note);
const r = writeOpencodeMcpEntry(
configPath,
{ kind: 'remote', name, url, tokenMode: 'env' },
{ expect: { url }, ...(opts?.allowReplaceOtherSource ? { allowReplaceOtherSource: true } : {}) },
);
return { configPath: r.configPath, replacedPrior: r.replacedPrior };
} finally {
lock.release();
}
},
};
// ---------------------------------------------------------------------------
@@ -700,8 +684,52 @@ export async function runConnect(args: string[], deps: ConnectDeps = defaultDeps
// --install path. token is guaranteed literal here (install mode resolveToken).
const realToken = token as string;
if (!spec.installable) {
fail(`--install supports claude-code and codex. ${spec.label} is set up through its own UI — drop --install to print the setup steps.`);
fail(`--install supports claude-code, codex, and opencode. ${spec.label} is set up through its own UI — drop --install to print the setup steps.`);
}
if (f.agent === 'opencode') {
// Direct-writer lane: no opencode binary required (the JSONC write IS the
// registration), and the config carries only the {env:VAR} interpolation
// — the writer's fingerprint handles idempotent re-runs and refuses a
// foreign same-name entry (--force cannot override THAT; pick --name).
// --force maps to the writer's allowReplaceOtherSource so an OURS entry
// at an old url (a rotated serve) is replaceable, mirroring the exec
// lanes' documented --force semantics.
if (!f.yes) {
if (!deps.isTTY()) {
fail('--install in a non-interactive shell requires --yes (refusing to register a credential-bearing MCP server without confirmation).');
}
const ok = await deps.promptYesNo(`Add MCP entry '${f.name}' -> ${url} to the opencode user-global config?`);
if (!ok) fail('Aborted.');
}
let w: { configPath: string; replacedPrior: boolean };
try {
w = await deps.writeOpencodeRemoteEntry(f.name, url, { allowReplaceOtherSource: f.force });
} catch (e) {
fail(redactToken((e as Error).message, realToken));
}
console.error(
`Added MCP entry '${f.name}' -> ${url} in ${w.configPath}` +
`${w.replacedPrior ? ' (replaced the prior gbrain entry)' : ''}. Restart opencode (config is read at session start).`,
);
if (deps.env(ENV_VAR) !== realToken) {
console.error(`opencode resolves {env:${ENV_VAR}} at read time. Add this to your shell profile so sessions can reach the brain:`);
console.error(` export ${ENV_VAR}=<your-token>`);
}
const ocProbe = await deps.probe(url, realToken, f.timeoutMs);
if (ocProbe.ok) {
console.error(`Verified: ${ocProbe.identity || 'brain reachable'}`);
console.error('');
console.error(LEARN_INSTRUCTION);
return;
}
console.error(
`Warning: registered '${f.name}', but the smoke-test did not verify (${ocProbe.reason}): ${redactToken(ocProbe.message, realToken)}`,
);
console.error('The agent will likely hit 401/errors until the token or URL is fixed.');
process.exit(1);
}
const binary = spec.binary as string; // 'claude' | 'codex'
if (!deps.hasBinary(binary)) {
fail(`${spec.label} CLI ('${binary}') not found on PATH. Install ${spec.label}, or drop --install to print the command to run manually.`);
+75 -2
View File
@@ -72,7 +72,7 @@ import { escapeLikePattern, buildVisibilityClause } from '../core/search/sql-ran
import { unverifiedExtractionFragment } from '../core/extraction-review.ts';
import { hnswIndexExpected, hnswMaxDimsForType } from '../core/vector-index.ts';
// Agent-bootstrap doctor group (plan B2/B4/ENG-4 + one-live-serve note).
import { readReceipt } from '../core/bootstrap/format.ts';
import { readHarnessReceiptState, readReceipt } from '../core/bootstrap/format.ts';
import { probeLivePgliteHolder, resolveBrainDataDir } from '../core/bootstrap/uninstall.ts';
import { readRunbookStamp, hooksInstalled, listVerifyRuns } from '../core/bootstrap/status.ts';
import { resolveGbrainHome } from '../core/gbrain-home.ts';
@@ -8774,11 +8774,84 @@ export async function bootstrapDoctorChecks(engine: BrainEngine | null): Promise
const pushStatuses = readPushStatuses();
const statusFilesOnDisk = pushStatusFilesExist();
const heartbeatFile = join(home, 'integrations', 'hooks', 'heartbeat.jsonl');
const hasBootstrapState = receipt !== null || statusFilesOnDisk || existsSync(heartbeatFile);
// #4043: a harness-only box (bootstrap harness, no workspace install) is
// bootstrap state too — without this, such a machine gets ZERO checks.
const harnessState = readHarnessReceiptState(home);
const hasBootstrapState =
receipt !== null || statusFilesOnDisk || existsSync(heartbeatFile) || harnessState.state !== 'absent';
if (!hasBootstrapState) return [];
const ws = receipt?.workspace_dir ?? null;
// 0. Harness registration health (#4043): three states so it neither cries
// wolf nor goes silent — skip (not a harness box) / warn (serve unreachable,
// a normal transient; or receipt unreadable) / fail (a target failed, or a
// prior rotation never converged). Token liveness needs the bearer (only
// recoverable from host config) — that's `gbrain bootstrap harness
// --status`'s job; doctor stays offline-cheap.
if (harnessState.state === 'ok') {
const hr = harnessState.receipt;
const failed = hr.targets.filter((t) => t.state === 'failed');
const pending = hr.targets.filter((t) => t.state === 'pending');
if (failed.length > 0 || pending.length > 0) {
checks.push({
name: 'bootstrap_harness_health',
status: 'fail',
message:
`harness wiring incomplete: ${failed.length} failed / ${pending.length} pending target(s)` +
` — re-run \`gbrain bootstrap harness\` to converge (details: gbrain bootstrap harness --status).`,
});
} else if (hr.token.previous_ids && hr.token.previous_ids.length > 0) {
checks.push({
name: 'bootstrap_harness_health',
status: 'fail',
message: `${hr.token.previous_ids.length} previous harness token(s) were never revoked (ids ${hr.token.previous_ids.join(', ')}) — re-run \`gbrain bootstrap harness\`, or run \`gbrain auth revoke\` with the id flag per id.`,
});
} else if (hr.targets.length === 0 && hr.token.minted && hr.token.id !== undefined) {
// Half-removed state: a remove under a live PGLite serve strips every
// host target but defers the revoke — the wiring is gone yet the minted
// token stays ACTIVE. A vacuous all-confirmed must not read green.
// (Flag names spelled without dashes here: the flag-registry generator
// harvests bare flag tokens from comments one import level deep.)
checks.push({
name: 'bootstrap_harness_health',
status: 'fail',
message: `harness removal pending: host wiring removed but the minted token (id ${hr.token.id}) is not yet revoked — stop the serve and re-run \`gbrain bootstrap harness\` with the remove flag, or run \`gbrain auth revoke\` with the id flag.`,
});
} else {
try {
const base = hr.url.replace(/\/mcp$/, '');
const res = await fetch(`${base}/health`, { signal: AbortSignal.timeout(3000) });
const body = res.ok ? ((await res.json()) as { status?: string }) : null;
if (body?.status === 'ok') {
checks.push({
name: 'bootstrap_harness_health',
status: 'ok',
message: `harness wired to ${hr.url} (serve healthy; token check: gbrain bootstrap harness --status)`,
});
} else {
checks.push({
name: 'bootstrap_harness_health',
status: 'warn',
message: `harness wired to ${hr.url} but the serve is not answering /health — start \`gbrain serve\` in http mode (a down serve is a normal transient, sessions just lose brain access until it returns).`,
});
}
} catch {
checks.push({
name: 'bootstrap_harness_health',
status: 'warn',
message: `harness wired to ${hr.url} but the serve is unreachable — start \`gbrain serve\` in http mode.`,
});
}
}
} else if (harnessState.state !== 'absent') {
checks.push({
name: 'bootstrap_harness_health',
status: 'warn',
message: `the harness receipt is unreadable (${harnessState.state}) — see \`gbrain bootstrap harness --status\`.`,
});
}
// 1. Hook heartbeat failure rate [B3 read side]. Hard errors only —
// degraded entries are DESIGNED fallbacks (pull-mode, no serve).
let hooksSeen = false;
+13
View File
@@ -0,0 +1,13 @@
/**
* Spend-gate constants for `gbrain dream retriage` (#4152, outside-voice C12).
* Split from dream-retriage.ts so tests can pin them without importing the
* command's engine-bearing module graph. The charstokens ratio lives in
* synthesize.ts (`CHARS_PER_TOKEN`, exported) the command imports it from
* there so the two estimates can't drift.
*/
/** Estimated sweeps above this ask for confirmation unless --yes. */
export const SPEND_CONFIRM_USD = 5;
/** When the model has no CANONICAL_PRICING entry, gate on file count instead. */
export const UNPRICED_CONFIRM_FILES = 500;
+697
View File
@@ -0,0 +1,697 @@
/**
* `gbrain dream retriage` (#4152) re-score the corpus and reconcile the
* synth-v2 job backlog against the triage gate.
*
* Two halves, both optional:
* 1. Re-judge: sweep the discovered corpus through runTriagePass
* (`--force` ignores the cache; `--since` treats older verdicts as
* stale; `--dry-run` performs ZERO judge calls and reads cached scores
* only). Spend-gated: prints an upfront estimate and asks for
* confirmation above ~$5 (`--yes` skips; `--max-usd` soft-stops).
* 2. `--reconcile-queue` (opt-in cancels queued work): parse every
* waiting/delayed/paused `dream:synth-v2:*` job across ALL queues (the
* live backlog largely sits in dead per-run `dream-inline-*` queues no
* worker will ever drain), match (basename, hash16) to discovered
* transcripts, then:
* - matched below threshold cancel (frontier job not worth it)
* - matched above, stale queue cancel as `converted_for_resubmit`
* (cancelled rows release their idempotency slot, so the next cycle
* re-adds them into ITS live private drain this is what actually
* migrates the backlog, outside-voice C1)
* - matched above, live queue keep
* - matched but unscored keep (never cancel on no data)
* - unmatched keep unless `--cancel-unmatched`
* - key-source data.source_id skip + count (C9 hardening)
* Cancellation is best-effort: status is re-checked immediately before
* each cancel and rows that turned `active` are skipped; the residual
* claim-vs-cancel race matches cancelJob's own BullMQ-style contract.
*
* `--audit-rejects <n>` (C6): re-judges N stride-sampled (deterministic) below-threshold files with
* the SYNTHESIS model and reports the disagreement rate the operator-run
* calibration loop for `dream.triage.threshold`.
*
* Exit codes: 0 success (even when nothing cancelled), 1 missing corpus
* config / engine, 2 usage error.
*/
import { basename } from 'node:path';
import { createInterface } from 'node:readline';
import type { BrainEngine, DreamVerdict } from '../core/engine.ts';
import {
loadSynthConfig,
runTriagePass,
parseSynthV2Key,
makeJudgeClient,
judgeSignificance,
isTriageCacheValid,
dreamInlineQueueAgeMs,
DREAM_INLINE_LIVE_GRACE_MS,
CHARS_PER_TOKEN,
type TriageFileReport,
} from '../core/cycle/synthesize.ts';
import { discoverTranscripts } from '../core/cycle/transcript-discovery.ts';
import { setCliExitVerdict } from '../core/cli-force-exit.ts';
import { MinionQueue } from '../core/minions/queue.ts';
import { canonicalLookup } from '../core/model-pricing.ts';
import { SPEND_CONFIRM_USD, UNPRICED_CONFIRM_FILES } from './dream-retriage-constants.ts';
interface RetriageArgs {
help: boolean;
threshold: number | null;
since: Date | null;
force: boolean;
reconcileQueue: boolean;
cancelUnmatched: boolean;
dryRun: boolean;
limit: number | null;
source: string | null;
json: boolean;
yes: boolean;
maxUsd: number | null;
auditRejects: number | null;
}
class UsageError extends Error {}
function parseRetriageArgs(args: string[]): RetriageArgs {
const out: RetriageArgs = {
help: false,
threshold: null,
since: null,
force: false,
reconcileQueue: false,
cancelUnmatched: false,
dryRun: false,
limit: null,
source: null,
json: false,
yes: false,
maxUsd: null,
auditRejects: null,
};
const takeValue = (flag: string, i: number): string => {
const v = args[i + 1];
if (v === undefined || v.startsWith('--')) throw new UsageError(`${flag} requires a value`);
return v;
};
for (let i = 0; i < args.length; i++) {
const a = args[i];
switch (a) {
case '--help': case '-h': out.help = true; break;
case '--force': out.force = true; break;
case '--reconcile-queue': out.reconcileQueue = true; break;
case '--cancel-unmatched': out.cancelUnmatched = true; break;
case '--dry-run': out.dryRun = true; break;
case '--json': out.json = true; break;
case '--yes': out.yes = true; break;
case '--threshold': {
const v = Number(takeValue(a, i)); i++;
if (!Number.isFinite(v) || v < 0 || v > 1) throw new UsageError('--threshold must be a number in [0,1]');
out.threshold = v;
break;
}
case '--since': {
const raw = takeValue(a, i); i++;
const ms = Date.parse(raw);
if (Number.isNaN(ms)) throw new UsageError(`--since could not parse date: ${raw}`);
out.since = new Date(ms);
break;
}
case '--limit': {
const v = parseInt(takeValue(a, i), 10); i++;
if (!Number.isFinite(v) || v < 1) throw new UsageError('--limit must be a positive integer');
out.limit = v;
break;
}
case '--source': case '--source-id': {
out.source = takeValue(a, i); i++;
break;
}
case '--max-usd': {
const v = Number(takeValue(a, i)); i++;
if (!Number.isFinite(v) || v <= 0) throw new UsageError('--max-usd must be a positive number');
out.maxUsd = v;
break;
}
case '--audit-rejects': {
const v = parseInt(takeValue(a, i), 10); i++;
if (!Number.isFinite(v) || v < 1) throw new UsageError('--audit-rejects must be a positive integer');
out.auditRejects = v;
break;
}
default:
throw new UsageError(`unknown flag for dream retriage: ${a}`);
}
}
if (out.cancelUnmatched && !out.reconcileQueue) {
throw new UsageError('--cancel-unmatched requires --reconcile-queue');
}
// CX2: a corpus scan truncated by --limit would misclassify every file outside
// the slice as "unmatched" — combining it with --cancel-unmatched would
// mass-cancel valid backlog.
if (out.cancelUnmatched && out.limit !== null) {
throw new UsageError('--cancel-unmatched cannot combine with --limit: a truncated scan misclassifies files as unmatched');
}
return out;
}
function printRetriageHelp(): void {
console.log(`gbrain dream retriage — re-score the corpus, reconcile the synth backlog
USAGE
gbrain dream retriage [flags]
FLAGS
--threshold <0..1> Gate override for this sweep (default: dream.triage.threshold)
--since <date> Treat verdicts judged before <date> as stale (re-judge)
--force Re-judge every discovered file regardless of cache
--limit <n> Only consider the first n discovered transcripts
--dry-run Zero judge calls, zero cancels; report from cached scores
--reconcile-queue Cancel waiting synth jobs per the gate (opt-in; see below)
--cancel-unmatched With --reconcile-queue: also cancel jobs whose file no
longer matches any discovered transcript
--source <id> Scope queue reconciliation to one source's jobs
--yes Skip the spend confirmation
--max-usd <n> Soft-stop judging when the ESTIMATED spend crosses n
(estimate-based; every judge attempt counts, including
unreliable responses; may overshoot by up to the
configured concurrency; requires a priced model; also
bounds --audit-rejects)
--audit-rejects <n> Re-judge n stride-sampled (deterministic) below-threshold
files with the SYNTHESIS model; report the disagreement
rate. Skipped under --dry-run. Counted in the spend gate.
--json Machine-readable output
--help This text
NOTES
--cancel-unmatched cannot combine with --limit (a truncated corpus scan
would misclassify everything outside the slice as unmatched), and refuses
to run when discovery finds zero transcripts (a corpus-mount outage must
not erase the queued retry frontier). dream-inline-* queues younger than
1h are treated as possibly LIVE (a running cycle's drain) and are never
cancelled they count as kept_live_queue. Running retriage while a cycle
is active may double-judge some cache misses (benign: last write wins).
RECONCILE SEMANTICS
matched + score < threshold cancel
matched + score >= threshold, dead dream-inline-* queue (older than 1h)
cancel (converted_for_resubmit
next cycle re-adds it into a live drain)
matched + score >= threshold, live queue keep
matched, no reliable score keep
unmatched / unparseable key keep (cancel with --cancel-unmatched)
legacy dream:synth: (v1) keys never touched
Cancelled rows release their idempotency slot, so lowering the threshold
later cleanly re-submits the work.`);
}
interface QueueCandidate {
id: number;
queue: string;
status: string;
idempotency_key: string;
source_id: string;
}
interface ReconcileStats {
candidates: number;
cancelled: number;
converted_for_resubmit: number;
kept_above_threshold: number;
kept_unscored: number;
/** Rows in a dream-inline-* queue younger than the liveness grace — possibly a running cycle's; never cancelled (CX1). */
kept_live_queue: number;
unmatched: number;
unmatched_cancelled: number;
source_mismatch: number;
other_source: number;
already_terminal: number;
by_source: Record<string, number>;
by_queue_kind: { dream_inline: number; other: number };
}
/** Per-file cost estimate in USD for one triage judge call; null when the model is unpriced. */
function estimatePerFileUsd(model: string, maxChars: number, maxTokens: number): number | null {
const pricing = canonicalLookup(model);
if (!pricing) return null;
const inputTokens = maxChars / CHARS_PER_TOKEN;
return (inputTokens / 1_000_000) * pricing.input + (maxTokens / 1_000_000) * pricing.output;
}
async function confirmOnTty(prompt: string): Promise<boolean> {
if (!process.stdin.isTTY) return false;
const rl = createInterface({ input: process.stdin, output: process.stderr });
const answer = await new Promise<string>(resolve => rl.question(`${prompt} [y/N] `, resolve));
rl.close();
return /^y(es)?$/i.test(answer.trim());
}
export async function runDreamRetriage(engine: BrainEngine | null, args: string[]): Promise<void> {
let parsed: RetriageArgs;
try {
parsed = parseRetriageArgs(args);
} catch (e) {
if (e instanceof UsageError) {
console.error(`dream retriage: ${e.message} (see: gbrain dream retriage --help)`);
setCliExitVerdict(2);
return;
}
throw e;
}
// IRON RULE: --help short-circuits before any engine-bearing work.
if (parsed.help) {
printRetriageHelp();
return;
}
if (engine === null) {
console.error('gbrain dream retriage requires a connected brain; run `gbrain init` first');
setCliExitVerdict(1);
return;
}
const config = await loadSynthConfig(engine);
if (!config.corpusDir) {
console.error('dream retriage: dream.synthesize.session_corpus_dir is unset — nothing to retriage');
setCliExitVerdict(1);
return;
}
const threshold = parsed.threshold ?? config.triage.threshold;
let transcripts = discoverTranscripts({
corpusDir: config.corpusDir,
meetingTranscriptsDir: config.meetingTranscriptsDir ?? undefined,
minChars: config.minChars,
excludePatterns: config.excludePatterns,
});
if (parsed.limit !== null) transcripts = transcripts.slice(0, parsed.limit);
// ── Half 1: score the corpus (cached reads in --dry-run; judged otherwise) ──
let reports: TriageFileReport[];
const byPath = new Map<string, DreamVerdict>();
let passStats = { judged: 0, cacheHits: 0, unreliable: 0, deferred: 0 };
// Estimated spend accumulated across the triage sweep AND the reject audit —
// one budget spans both halves (CX3 + security review).
let estimatedSpendUsd = 0;
if (parsed.dryRun) {
// Zero judge calls: read cached verdicts only. Files without a valid
// triage-v1 score report as needs_triage.
reports = [];
for (const t of transcripts) {
const cached = await engine.getDreamVerdict(t.filePath, t.contentHash);
const valid = cached !== null && !parsed.force
&& isTriageCacheValid(cached, config.triage.model, parsed.since ?? undefined);
if (cached && valid) {
passStats.cacheHits++;
byPath.set(t.filePath, cached);
reports.push({
filePath: t.filePath,
worth: cached.score !== null && cached.score >= threshold,
score: cached.score,
content_type: cached.content_type,
reasons: cached.reasons,
cached: true,
});
} else {
reports.push({
filePath: t.filePath,
worth: false,
score: null,
content_type: null,
reasons: ['needs_triage (dry-run performs no judge calls)'],
cached: false,
deferred: true,
});
passStats.deferred++;
}
}
} else {
// Spend gate (outside-voice C12): estimate the miss count upfront and
// confirm above SPEND_CONFIRM_USD unless --yes.
let missCount = 0;
for (const t of transcripts) {
if (parsed.force) { missCount++; continue; }
const cached = await engine.getDreamVerdict(t.filePath, t.contentHash);
const valid = cached !== null
&& isTriageCacheValid(cached, config.triage.model, parsed.since ?? undefined);
if (!valid) missCount++;
}
const perFileUsd = estimatePerFileUsd(config.triage.model, config.triage.maxChars, config.triage.maxTokens);
// CX3: --max-usd is estimate-based; an unpriced model would silently
// disable the budget the operator explicitly asked for — refuse instead.
if (parsed.maxUsd !== null && perFileUsd === null) {
console.error(
`dream retriage: --max-usd requires a priced model; "${config.triage.model}" has no CANONICAL_PRICING entry`,
);
setCliExitVerdict(2);
return;
}
// The frontier audit spends too (security review): fold its worst case
// into the gated estimate so --audit-rejects can't ride around the gate.
const auditPerFileUsd = parsed.auditRejects !== null
? estimatePerFileUsd(config.model, config.triage.maxChars, config.triage.maxTokens)
: null;
// Codex structured review P1: an unpriced SYNTHESIS model would zero out
// the audit's share of the estimate AND silently disable --max-usd inside
// the audit loop — refuse the budget flag, and always confirm when the
// audit spend cannot be estimated.
const auditUnpriced = parsed.auditRejects !== null && auditPerFileUsd === null;
if (parsed.maxUsd !== null && auditUnpriced) {
console.error(
`dream retriage: --max-usd with --audit-rejects requires a priced synthesis model; ` +
`"${config.model}" has no CANONICAL_PRICING entry`,
);
setCliExitVerdict(2);
return;
}
const auditEstimateUsd = parsed.auditRejects !== null && auditPerFileUsd !== null
? parsed.auditRejects * auditPerFileUsd
: 0;
// Structured-review round 2 P1: the KNOWN portion of the estimate gates
// independently of whether the triage model is priced — an unpriced
// triage model with a large PRICED audit must still confirm on the audit
// dollars, not slide through the file-count gate on cached rejects.
const knownEstimateUsd = (perFileUsd ?? 0) * missCount + auditEstimateUsd;
const estimateUsd = perFileUsd === null ? null : knownEstimateUsd;
const gateTriggered = knownEstimateUsd > SPEND_CONFIRM_USD
|| (perFileUsd === null && missCount > UNPRICED_CONFIRM_FILES)
|| auditUnpriced; // un-estimable audit spend always confirms
const auditSuffix = auditUnpriced
? ` (audit model "${config.model}" unpriced — audit spend cannot be estimated)`
: auditEstimateUsd > 0 ? ` (incl. ≤ $${auditEstimateUsd.toFixed(2)} audit)` : '';
const estimateLine = estimateUsd !== null
? `[retriage] ${missCount} file(s) to judge with ${config.triage.model} — estimated ≤ $${estimateUsd.toFixed(2)}${auditSuffix}`
: `[retriage] ${missCount} file(s) to judge with ${config.triage.model} — no pricing entry for this model (cannot estimate; the ${UNPRICED_CONFIRM_FILES}-file confirmation gate applies)${auditSuffix}`;
process.stderr.write(estimateLine + '\n');
if (gateTriggered && !parsed.yes) {
if (parsed.json || !process.stdin.isTTY) {
console.error('dream retriage: spend estimate exceeds the confirmation gate; re-run with --yes (non-interactive)');
setCliExitVerdict(2);
return;
}
const ok = await confirmOnTty(`Proceed with ~$${estimateUsd?.toFixed(2) ?? '?'} of triage spend?`);
if (!ok) {
console.error('dream retriage: aborted at spend confirmation');
setCliExitVerdict(2);
return;
}
}
// --max-usd soft-stop: estimate-based (usage isn't threaded through the
// judge seam); stops pulling new misses once attempts × per-file estimate
// crosses the budget. runTriagePass ticks shouldStop on EVERY judge
// attempt (CX3 — unreliable responses are paid calls too); may overshoot
// by up to the configured concurrency. Remaining files report as deferred.
const shouldStop = parsed.maxUsd !== null && perFileUsd !== null
? (): boolean => {
estimatedSpendUsd += perFileUsd;
return estimatedSpendUsd >= parsed.maxUsd!;
}
: undefined;
const pass = await runTriagePass(engine, transcripts, {
model: config.triage.model,
maxChars: config.triage.maxChars,
maxTokens: config.triage.maxTokens,
threshold,
concurrency: config.triage.concurrency,
maxMs: 0, // operator sweep runs to completion; --limit / --max-usd bound it
force: parsed.force,
staleBefore: parsed.since ?? undefined,
shouldStop,
});
reports = pass.reports;
for (const [k, v] of pass.byPath) byPath.set(k, v);
passStats = { judged: pass.judged, cacheHits: pass.cacheHits, unreliable: pass.unreliable, deferred: pass.deferred };
}
// ── Half 2: queue reconciliation (opt-in) ──
let reconcile: ReconcileStats | null = null;
if (parsed.reconcileQueue) {
const queue = new MinionQueue(engine);
const rows = await engine.executeRaw<QueueCandidate>(
`SELECT id, queue, status, idempotency_key,
COALESCE(NULLIF(data->>'source_id', ''), 'default') AS source_id
FROM minion_jobs
WHERE name = 'subagent'
AND status IN ('waiting', 'delayed', 'paused')
AND idempotency_key LIKE 'dream:synth-v2:%'`,
);
// Two lookups: membership in the discovered corpus (matched at all?) vs a
// usable scored verdict. A discovered file with no reliable score is
// "matched but unscored" — kept, never cancelled on missing data. The
// '|' join is unambiguous even for basenames containing '|': hash16 is
// fixed-width hex after the final separator.
const discoveredKeys = new Set<string>();
const verdictByKey = new Map<string, DreamVerdict>();
for (const t of transcripts) {
const k = `${basename(t.filePath)}|${t.contentHash.slice(0, 16)}`;
discoveredKeys.add(k);
const v = byPath.get(t.filePath);
if (v) verdictByKey.set(k, v);
}
// CX2: an empty discovery result alongside a non-empty backlog means the
// corpus is unreachable (mount outage, permissions, wrong dir) far more
// often than it means every file was deleted. Refuse to cancel-unmatched
// in that state — a transient outage must not erase the retry frontier.
if (parsed.cancelUnmatched && transcripts.length === 0 && rows.length > 0) {
console.error(
`dream retriage: discovery found 0 transcripts but ${rows.length} queued job(s) exist; ` +
'refusing --cancel-unmatched (corpus may be unreachable). Fix discovery or drop the flag.',
);
setCliExitVerdict(2);
return;
}
reconcile = {
candidates: rows.length,
cancelled: 0,
converted_for_resubmit: 0,
kept_above_threshold: 0,
kept_unscored: 0,
kept_live_queue: 0,
unmatched: 0,
unmatched_cancelled: 0,
source_mismatch: 0,
other_source: 0,
already_terminal: 0,
by_source: {},
by_queue_kind: { dream_inline: 0, other: 0 },
};
// Codex structured review P2: queue age alone is not liveness — a cycle
// with several slow sequential children can legitimately exceed the 1h
// grace. Consult the REAL signal: a live (unexpired) cycle lock means a
// cycle is running right now, so no dream-inline queue is provably dead.
const liveLocks = await engine.executeRaw<{ id: string }>(
`SELECT id FROM gbrain_cycle_locks WHERE ttl_expires_at > NOW() AND id LIKE 'gbrain-cycle%'`,
);
// Structured-review round 2 P2: cycle locks are per-source
// (`gbrain-cycle:<source>`) — a cycle running for source A must not
// suppress conversions for source B indefinitely. Only the legacy bare
// `gbrain-cycle` lock is global.
const globalLockLive = liveLocks.some(l => l.id === 'gbrain-cycle');
const liveLockSources = new Set(
liveLocks
.map(l => (l.id.startsWith('gbrain-cycle:') ? l.id.slice('gbrain-cycle:'.length) : null))
.filter((s): s is string => s !== null),
);
if (liveLocks.length > 0) {
process.stderr.write(
`[retriage] live cycle lock(s) detected (${liveLocks.map(l => l.id).join(', ')}); ` +
`dream-inline queues for those sources are treated as possibly-live — conversions skipped\n`,
);
}
const cancelRow = async (id: number): Promise<'cancelled' | 'already_terminal'> => {
// Pre-cancel status re-check (C9): a candidate claimed by a live worker
// between the snapshot SELECT and now is skipped, not killed.
const fresh = await engine.executeRaw<{ status: string }>(
`SELECT status FROM minion_jobs WHERE id = $1`, [id],
);
const status = fresh[0]?.status;
if (status !== 'waiting' && status !== 'delayed' && status !== 'paused') return 'already_terminal';
const r = await queue.cancelJob(id);
return r ? 'cancelled' : 'already_terminal';
};
for (const row of rows) {
reconcile.by_source[row.source_id] = (reconcile.by_source[row.source_id] ?? 0) + 1;
const inlineQueueAge = dreamInlineQueueAgeMs(row.queue);
const isInlineQueue = inlineQueueAge !== null || row.queue.startsWith('dream-inline-');
if (isInlineQueue) reconcile.by_queue_kind.dream_inline++;
else reconcile.by_queue_kind.other++;
// CX1: a dream-inline-* queue younger than the liveness grace may belong
// to a cycle that is RUNNING right now — its inline drain will claim
// these rows. Never cancel anything in a possibly-live private queue
// (unparseable-timestamp names count as possibly-live, fail-safe), and
// a live cycle lock FOR THIS ROW'S SOURCE marks its inline queues
// possibly-live regardless of age (structured-review P2: slow
// sequential children can outlive the grace; round 2: per-source, so a
// busy source A never suppresses source B's cleanup indefinitely).
const lockLiveForRow = globalLockLive || liveLockSources.has(row.source_id);
const possiblyLiveQueue = isInlineQueue
&& (lockLiveForRow || inlineQueueAge === null || inlineQueueAge <= DREAM_INLINE_LIVE_GRACE_MS);
if (possiblyLiveQueue) {
reconcile.kept_live_queue++;
continue;
}
if (parsed.source !== null && row.source_id !== parsed.source) {
reconcile.other_source++;
continue;
}
const key = parseSynthV2Key(row.idempotency_key);
if (!key) {
reconcile.unmatched++;
continue;
}
// C9 hardening: the key's encoded source must agree with the payload's
// source_id — never cancel on a disagreement.
if ((key.source || 'default') !== row.source_id) {
reconcile.source_mismatch++;
continue;
}
const matchKey = `${key.basename}|${key.hash16}`;
const verdict = verdictByKey.get(matchKey);
if (!verdict || verdict.score === null) {
// Unmatched file OR matched-but-unscored (deferred/degraded): only the
// truly-unmatched are cancellable, and only behind --cancel-unmatched.
const isMatchedUnscored = discoveredKeys.has(matchKey);
if (isMatchedUnscored) {
reconcile.kept_unscored++;
} else if (parsed.cancelUnmatched) {
// Structured-review P2: the dry-run preview must count would-cancel
// unmatched rows the same way the below-threshold branch does — a
// destructive preview that understates its impact is worse than none.
if (parsed.dryRun) {
reconcile.unmatched_cancelled++; // dry-run: would cancel
} else {
const outcome = await cancelRow(row.id);
if (outcome === 'cancelled') reconcile.unmatched_cancelled++;
else reconcile.already_terminal++;
}
} else {
reconcile.unmatched++;
}
continue;
}
if (verdict.score < threshold) {
if (!parsed.dryRun) {
const outcome = await cancelRow(row.id);
if (outcome === 'cancelled') reconcile.cancelled++;
else reconcile.already_terminal++;
} else {
reconcile.cancelled++; // dry-run: would cancel
}
continue;
}
// Above threshold. C1: a row stranded in a provably-dead per-run
// dream-inline-* queue (older than the liveness grace, no live cycle
// lock — the possibly-live case was already kept above) will never be
// claimed — cancel it so the next cycle's queue.add re-creates it in a
// live drain (the cancelled row releases its idempotency slot).
// `delayed` counts too (structured-review P2): a transient-failure
// retry parked in a dead queue has no worker to promote or drain it.
if (isInlineQueue && (row.status === 'waiting' || row.status === 'delayed')) {
if (!parsed.dryRun) {
const outcome = await cancelRow(row.id);
if (outcome === 'cancelled') reconcile.converted_for_resubmit++;
else reconcile.already_terminal++;
} else {
reconcile.converted_for_resubmit++; // dry-run: would convert
}
continue;
}
reconcile.kept_above_threshold++;
}
}
// ── --audit-rejects (C6): frontier second opinion on N stride-sampled rejects ──
let audit: { sampled: number; disagreements: number; disagreement_rate: number | null } | null = null;
if (parsed.auditRejects !== null && parsed.dryRun) {
// Loud no-op instead of a silently-null audit field (maintainability review).
process.stderr.write('[retriage] --audit-rejects skipped under --dry-run (the audit spends frontier-model calls)\n');
}
if (parsed.auditRejects !== null && !parsed.dryRun) {
const rejects = reports.filter(r => r.score !== null && r.score < threshold);
// Deterministic stride-sample over the rejects in discovery order — no
// randomness, so repeated audits compare like with like.
const sample: TriageFileReport[] = [];
const stride = Math.max(1, Math.floor(rejects.length / parsed.auditRejects));
for (let i = 0; i < rejects.length && sample.length < parsed.auditRejects; i += stride) sample.push(rejects[i]);
const frontier = makeJudgeClient(config.model);
if (!frontier) {
process.stderr.write(`[retriage] --audit-rejects: no reachable provider for ${config.model}; skipping audit\n`);
} else {
const byFilePath = new Map(transcripts.map(t => [t.filePath, t]));
const auditPerFileUsd = estimatePerFileUsd(config.model, config.triage.maxChars, config.triage.maxTokens);
let disagreements = 0;
let judged = 0;
for (const r of sample) {
const t = byFilePath.get(r.filePath);
if (!t) continue;
// --max-usd spans the audit too (CX3): stop before the next frontier
// call would cross the budget.
if (parsed.maxUsd !== null && auditPerFileUsd !== null
&& estimatedSpendUsd + auditPerFileUsd > parsed.maxUsd) {
process.stderr.write(`[retriage] --audit-rejects stopped at --max-usd $${parsed.maxUsd.toFixed(2)} (audited ${judged})\n`);
break;
}
try {
const second = await judgeSignificance(frontier, t, config.model, {
maxChars: config.triage.maxChars,
maxTokens: config.triage.maxTokens,
});
estimatedSpendUsd += auditPerFileUsd ?? 0;
if (second.unreliable) continue;
judged++;
if (second.score >= threshold) disagreements++;
} catch {
// A failed call is still an attempt — count its estimated cost.
estimatedSpendUsd += auditPerFileUsd ?? 0;
// Audit is best-effort; a failed second opinion is skipped.
}
}
audit = {
sampled: judged,
disagreements,
disagreement_rate: judged > 0 ? Math.round((disagreements / judged) * 1000) / 1000 : null,
};
}
}
const passCount = reports.filter(r => r.worth).length;
const summary = {
discovered: transcripts.length,
threshold,
pass: passCount,
below_threshold: reports.filter(r => r.score !== null && !r.worth).length,
needs_triage: reports.filter(r => r.deferred).length,
retriaged: passStats.judged,
cache_hits: passStats.cacheHits,
unreliable: passStats.unreliable,
deferred: passStats.deferred,
dry_run: parsed.dryRun,
queue: reconcile,
audit,
};
if (parsed.json) {
console.log(JSON.stringify(summary, null, 2));
} else {
const would = parsed.dryRun ? ' (dry-run: no cancels performed)' : '';
console.log(`[retriage] ${summary.discovered} discovered | ${summary.pass} pass @ threshold ${threshold} | ` +
`${summary.below_threshold} below | ${summary.needs_triage} need triage | ` +
`${summary.retriaged} judged, ${summary.cache_hits} cached, ${summary.unreliable} unreliable`);
if (reconcile) {
console.log(`[retriage] queue: ${reconcile.candidates} candidates | ${reconcile.cancelled} cancelled | ` +
`${reconcile.converted_for_resubmit} converted for resubmit | ${reconcile.kept_above_threshold} kept | ` +
`${reconcile.kept_unscored} unscored kept | ${reconcile.kept_live_queue} live-queue kept | ${reconcile.unmatched} unmatched | ` +
`${reconcile.source_mismatch} source-mismatch skipped | ${reconcile.already_terminal} already terminal${would}`);
const bySource = Object.entries(reconcile.by_source).map(([s, n]) => `${s}=${n}`).join(', ');
if (bySource) console.log(`[retriage] queue by source: ${bySource} | stale dream-inline queues: ${reconcile.by_queue_kind.dream_inline}`);
}
if (audit) {
console.log(`[retriage] reject audit: ${audit.sampled} re-judged by ${config.model}, ` +
`${audit.disagreements} disagreements (rate ${audit.disagreement_rate ?? 'n/a'})`);
}
}
}
+38 -2
View File
@@ -32,6 +32,7 @@ import {
type CycleReport,
} from '../core/cycle.ts';
import { resolveSourceId } from '../core/source-resolver.ts';
import { setCliExitVerdict } from '../core/cli-force-exit.ts';
import { fetchSource } from '../core/sources-load.ts';
import { existsSync } from 'fs';
import { resolve } from 'node:path';
@@ -346,6 +347,7 @@ async function resolveBrainDir(
function printHelp() {
console.log(`Usage: gbrain dream [options]
gbrain dream retriage [flags] (see: gbrain dream retriage --help)
Run one brain maintenance cycle. Eight phases:
lint -> backlinks -> sync -> synthesize -> extract -> patterns -> embed -> orphans
@@ -354,10 +356,17 @@ The synthesize + patterns phases (v0.21) consolidate yesterday's
conversation transcripts into reflections, originals, and cross-session
pattern pages. Designed for cron (exits when done).
The synthesize phase (#4152) runs a two-stage cascade: a cheap scored triage
(model: models.dream.triage, gate: dream.triage.threshold, default 0.5) gates
the expensive per-transcript synthesis subagents (turn budget:
dream.synthesize.max_turns, default 16). Retune the threshold any time
scores are cached, so re-gating costs zero new LLM calls. \`dream retriage\`
re-scores the corpus and reconciles the queued synthesis backlog.
Options:
--dry-run Preview all fixes without writing. Note: synthesize
runs the cheap Haiku significance filter (caches
verdicts), but skips the Sonnet synthesis pass.
runs the cheap scored triage pass (caches verdicts),
but skips the synthesis subagents.
"--dry-run" does NOT mean "zero LLM calls."
--json Emit the CycleReport as JSON (agent-readable)
--phase <name> Run a single phase: ${ALL_PHASES.join(' | ')}
@@ -569,6 +578,33 @@ async function runDrain(
}
export async function runDream(engine: BrainEngine | null, args: string[]): Promise<CycleReport | void> {
// ─── `dream retriage` subverb (#4152) — dispatched BEFORE parseArgs so its
// flag set never collides with the cycle flags. `dream --help` never reaches
// here (args[0] is '--help'); `dream retriage --help` prints subcommand help
// inside runDreamRetriage without touching the engine (same IRON RULE).
if (args[0] === 'retriage') {
const { runDreamRetriage } = await import('./dream-retriage.ts');
await runDreamRetriage(engine, args.slice(1));
return;
}
// Fail-loud guard (structured-review r3 P1): the CLI flag registry unions
// retriage's flags into `dream`, so the pre-dispatch validator accepts
// `gbrain dream --reconcile-queue` — but without the `retriage` positional,
// parseArgs would ignore the flag and silently run the full (paid, writing)
// maintenance cycle instead of the reconciliation the user asked for.
{
const RETRIAGE_ONLY_FLAGS = ['--reconcile-queue', '--cancel-unmatched', '--audit-rejects'];
const stray = args.find(a => RETRIAGE_ONLY_FLAGS.includes(a));
if (stray) {
console.error(
`gbrain dream: ${stray} belongs to the 'retriage' subcommand — ` +
`did you mean: gbrain dream retriage ${args.join(' ')}`,
);
setCliExitVerdict(2);
return;
}
}
const opts = parseArgs(args);
// ─── IRON RULE: --help short-circuits BEFORE any engine-bearing work ─
+13 -14
View File
@@ -1,6 +1,7 @@
import type { BrainEngine } from '../core/engine.ts';
import { embedBatch, currentEmbeddingSignature } from '../core/embedding.ts';
import type { ChunkInput } from '../core/types.ts';
import { carryChunkMetadata } from '../core/embed-stale.ts';
import { chunkText } from '../core/chunkers/recursive.ts';
import { createProgress, type ProgressReporter } from '../core/progress.ts';
import { getCliOptions, cliOptsToProgressOptions } from '../core/cli-options.ts';
@@ -784,9 +785,11 @@ async function embedPage(
}
/**
* Carry code-chunk metadata (language, symbol_name, symbol_type, line range,
* parent scope, doc comment, qualified name) from a loaded Chunk back into a
* ChunkInput destined for upsertChunks.
* Carry per-chunk metadata modality (the W0 fix: its omission flipped
* image chunks to text) plus the code fields (language, symbol_name,
* symbol_type, line range, parent scope, doc comment, qualified name) from
* a loaded Chunk back into a ChunkInput destined for upsertChunks. The
* shared carryChunkMetadata list (core/embed-stale.ts) is authoritative.
*
* Issue #769: every re-embed used to strip these fields, and upsertChunks
* overwrites (does not COALESCE) the metadata columns from EXCLUDED, so
@@ -795,17 +798,13 @@ async function embedPage(
* (embedPage, embedAll non-stale, embedAllStale) in lock-step.
*/
function preserveCodeMetadata(loaded: any, base: ChunkInput): ChunkInput {
return {
...base,
language: loaded.language ?? undefined,
symbol_name: loaded.symbol_name ?? undefined,
symbol_type: loaded.symbol_type ?? undefined,
start_line: loaded.start_line ?? undefined,
end_line: loaded.end_line ?? undefined,
parent_symbol_path: loaded.parent_symbol_path ?? undefined,
doc_comment: loaded.doc_comment ?? undefined,
symbol_name_qualified: loaded.symbol_name_qualified ?? undefined,
};
// W0 fix-wave (Tier-1 #3, CONFIRMED): delegate to the single shared carry
// list. This local copy was missing `modality`, so every CLI re-embed path
// (embedPage, embedAll, embedAllStale) flipped image chunks to
// modality='text' — upsertChunks overwrites from EXCLUDED — silently
// zeroing image retrieval until the next full import. The minion twin in
// core/embed-stale.ts carried it correctly; one list now serves both.
return carryChunkMetadata(loaded, base);
}
async function embedAll(
+27 -2
View File
@@ -249,9 +249,17 @@ export interface ExtractConversationFactsCoreOpts {
types?: AllowedType[];
/** Process a single page; otherwise iterate all matching pages in the source. */
slug?: string;
/**
* cathedral-4 batch selector: process exactly these pages (serial, with
* the same per-page advisory lock + durable-outcome gates as enumeration).
* ONE core invocation per caller run per-slug invocations multiply
* config resolution, checkpoint IO, and receipt writes by page count.
* Takes precedence over `slug`.
*/
slugs?: string[];
/** Show would-do counts without writing facts or advancing checkpoint. */
dryRun?: boolean;
/** Cap pages processed in this invocation. */
/** Cap pages processed in this invocation (enumeration path only; ignored when `slugs` is set). */
limit?: number;
/** ISO watermark; messages older than this are filtered out. */
sinceIso?: string;
@@ -1336,7 +1344,24 @@ export async function runExtractConversationFactsCore(
// types are not silently skipped (see ALLOWED_TYPE_ALIASES).
const concreteTypes = pageTypesForAllowed(types);
if (opts.slug) {
if (opts.slugs !== undefined) {
// Batch mode is selected by the PRESENCE of the selector: an empty
// list means "process exactly these zero pages" (a no-op), never a
// fall-through to full-corpus enumeration and its LLM spend.
for (const slug of opts.slugs) {
if (signal?.aborted) throw new Error('aborted');
const page = await engine.getPage(slug, { sourceId });
if (!page) {
result.pages_skipped_disappeared++;
continue;
}
if (!concreteTypes.includes(page.type)) {
result.pages_skipped++;
continue;
}
await processPageWithLock(page);
}
} else if (opts.slug) {
const page = await engine.getPage(opts.slug, { sourceId });
if (!page) {
result.pages_skipped_disappeared++;
+54 -9
View File
@@ -159,11 +159,11 @@ export interface HookIo {
/** TEST SEAM: user-prompt deadline override (wall-clock flake control). */
userPromptDeadlineMs?: number;
/**
* Feedback-loop attribution channel (`--harness <claude-code|codex>`).
* Feedback-loop attribution channel (`--harness <claude-code|codex|opencode>`).
* Default 'claude-code' the only harness bootstrap registers hooks for
* today; a codex hook registration passes the flag explicitly.
* today; a codex/opencode hook registration passes the flag explicitly.
*/
harness?: 'claude-code' | 'codex';
harness?: 'claude-code' | 'codex' | 'opencode';
}
// ── Entry point ─────────────────────────────────────────────────────────────
@@ -175,8 +175,8 @@ Events (wired into .claude/settings.local.json by gbrain bootstrap):
push status, hook health) to stdout
user-prompt read hook JSON on stdin, request per-turn context from a
running 'gbrain serve' over IPC, print additionalContext JSON
(--harness <claude-code|codex> sets the feedback-loop channel;
default claude-code, unknown values fall back to the default)
(--harness <claude-code|codex|opencode> sets the feedback-loop
channel; default claude-code, unknown values fall back to the default)
stop append to the per-session live buffer
session-end ingest the session transcript into the dream corpus
(secret-scanned), prune old corpus files, push the workspace
@@ -195,13 +195,13 @@ export async function runHook(args: string[], io: HookIo = {}): Promise<number>
write(io, USAGE + '\n');
return 0;
}
// `--harness <claude-code|codex>` — feedback-loop channel attribution for
// user-prompt. Unknown values fall back to the default (fail-open: a bad
// registration must never break the hook contract).
// `--harness <claude-code|codex|opencode>` — feedback-loop channel
// attribution for user-prompt. Unknown values fall back to the default
// (fail-open: a bad registration must never break the hook contract).
const harnessIdx = args.indexOf('--harness');
if (harnessIdx >= 0 && !io.harness) {
const v = args[harnessIdx + 1];
if (v === 'claude-code' || v === 'codex') io = { ...io, harness: v };
if (v === 'claude-code' || v === 'codex' || v === 'opencode') io = { ...io, harness: v };
}
if (!event || !['session-start', 'user-prompt', 'stop', 'session-end', 'compact'].includes(event)) {
process.stderr.write(USAGE + '\n');
@@ -211,6 +211,51 @@ export async function runHook(args: string[], io: HookIo = {}): Promise<number>
// for silence, and a disabled hook writing telemetry would be a lie).
if (process.env.GBRAIN_HOOKS === '0') return 0;
// #4043 harness-lane defer guard: Claude Code MERGES user- and
// project-scope hook settings, so a machine wired by `bootstrap harness`
// (user scope) plus a real workspace bootstrap install (settings.local.json,
// bootstrap-v1 marker) would fire the same event twice. The workspace
// install wins; the harness lane yields silently (exit 0, no output, no
// heartbeat). Same cwd resolution as the handlers (io.cwd is the test
// seam; the harness runs hooks in the session's working dir). Fail-open:
// any read hiccup means run normally.
if (process.env.GBRAIN_HOOK_LANE === 'harness') {
try {
// BOTH workspace carriers count: settings.local.json (local installs)
// and the committed .claude/settings.json ([D12] — an event owned by
// the committed carrier is stripped from local, so checking only local
// would double-fire it against the user-scope harness wiring). The
// check PARSES the settings and requires a live bootstrap-v1 hook entry
// wiring THIS event — a raw substring match would let any repo disable
// the machine-wide capture lane by committing the two marker strings in
// an unrelated field (ship-review P1), and would over-yield events the
// workspace does not actually wire.
const eventKey = {
'session-start': 'SessionStart',
'user-prompt': 'UserPromptSubmit',
stop: 'Stop',
'session-end': 'SessionEnd',
compact: 'PreCompact',
}[event];
const dotClaude = join(io.cwd ?? process.cwd(), '.claude');
for (const file of ['settings.local.json', 'settings.json']) {
const p = join(dotClaude, file);
if (!existsSync(p)) continue;
const settings = JSON.parse(readFileSync(p, 'utf8')) as {
hooks?: Record<string, Array<{ hooks?: Array<Record<string, unknown>> }>>;
};
const groups = settings.hooks?.[eventKey ?? ''];
if (!Array.isArray(groups)) continue;
for (const g of groups) {
if (!Array.isArray(g?.hooks)) continue;
if (g.hooks.some((e) => e?._gbrain === 'bootstrap-v1')) return 0;
}
}
} catch {
/* fail-open */
}
}
switch (event) {
case 'session-start':
return hookSessionStart(io);
+26 -5
View File
@@ -64,6 +64,27 @@ function defaultWorkers(): number {
return Math.min(byPool, byCpu, byMem);
}
/**
* W0 fix-wave (Tier-1 #5): typed abort for runImport's preflight/argv
* failures. Pre-fix these five sites called process.exit(1) directly
* correct for the CLI, but runImport is ALSO invoked in-process by the
* sync_brain MCP op (via performFullSync), the autopilot daemon, and the
* minion sync handler, so a first sync with unconfigured embedding
* credentials TERMINATED the MCP server / daemon / worker mid-call. The
* user-facing messages are printed BEFORE the throw (byte-identical CLI
* output); the CLI dispatch site maps this error back to exit(exitCode).
*/
export class ImportAbortError extends Error {
readonly exitCode: number;
/** True: the user-facing message was already printed at the throw site. */
readonly alreadyReported = true;
constructor(reason: string, exitCode = 1) {
super(`import aborted: ${reason}`);
this.name = 'ImportAbortError';
this.exitCode = exitCode;
}
}
/** Bug 9 — surface per-file failures so callers (performFullSync) can gate state advances. */
export interface RunImportResult {
imported: number;
@@ -129,7 +150,7 @@ export async function runImport(
} catch (e) {
console.error(`\n${e instanceof Error ? e.message : e}`);
console.error('Tip: run `gbrain import <dir> --no-embed` to import without embedding now.');
process.exit(1);
throw new ImportAbortError('embedding disabled (deferred-setup sentinel)');
}
// v0.41.6.0 D1: preflight embedding credentials. Closes the bug class
@@ -147,7 +168,7 @@ export async function runImport(
console.error(e.userMessage);
console.error('');
}
process.exit(1);
throw new ImportAbortError('embedding credentials missing');
}
throw e;
}
@@ -221,7 +242,7 @@ export async function runImport(
workerCount = parseWorkers(workersArg ?? undefined) ?? 1;
} catch (e) {
console.error(e instanceof Error ? e.message : String(e));
process.exit(1);
throw new ImportAbortError('invalid --workers value');
}
// Find dir: first non-flag arg that isn't a value for --workers
const flagValues = new Set<number>();
@@ -231,7 +252,7 @@ export async function runImport(
if (!dirArg) {
console.error('Usage: gbrain import <dir> [--no-embed] [--workers N] [--fresh] [--source-id <id>] [--include-gitignored] [--json]');
process.exit(1);
throw new ImportAbortError('no import directory given');
}
// #1728: capture the import target ONCE as an absolute real path. Every
// downstream consumer of `dir` (collection, checkpoint load/save, resume
@@ -244,7 +265,7 @@ export async function runImport(
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
console.error(`Import target is not readable: ${dirArg} (${msg})`);
process.exit(1);
throw new ImportAbortError(`import target not readable: ${dirArg}`);
}
// v0.31.2: collect under the right strategy. Pre-fix this called
+472 -33
View File
@@ -6,12 +6,18 @@
import type { BrainEngine } from '../core/engine.ts';
import { MinionQueue } from '../core/minions/queue.ts';
import { MinionWorker } from '../core/minions/worker.ts';
import { WORKER_EXIT_RSS_WATCHDOG } from '../core/minions/worker-exit-codes.ts';
import {
WORKER_EXIT_RSS_WATCHDOG,
JOB_CHILD_EXIT_USAGE,
} from '../core/minions/worker-exit-codes.ts';
import { CHILD_ENV, resolveChildCliInvocation } from '../core/minions/job-isolation.ts';
import { runChildJobEntry } from '../core/minions/run-child.ts';
import type { MinionHandler, MinionJob, MinionJobStatus } from '../core/minions/types.ts';
import type { PaceKeyOverrides } from '../core/pace-mode.ts';
import { loadConfig, isThinClient } from '../core/config.ts';
import { callRemoteTool, unpackToolResult } from '../core/mcp-client.ts';
import { parseNiceValue, applyNiceness, getEffectiveNiceness, formatNice } from '../core/minions/niceness.ts';
import { defaultTimeoutMsFor } from '../core/minions/handler-timeouts.ts';
function parseFlag(args: string[], flag: string): string | undefined {
const idx = args.indexOf(flag);
@@ -153,6 +159,32 @@ export function resolveWorkerConcurrency(args: string[], env: NodeJS.ProcessEnv
return parsed;
}
export type JobIsolationMode = 'inline' | 'process';
/**
* issue #5: `--job-isolation <inline|process>` (space or `=` form), env
* fallback GBRAIN_JOB_ISOLATION, default inline. `process` runs each claimed
* job in a SIGKILL-able child process blast radius 1 job instead of N.
* Env injected as a param so tests never mutate process.env (rule R1).
* Invalid values fail fast (parseMaxRssFlag convention).
*/
export function parseJobIsolationFlag(
args: string[],
env: NodeJS.ProcessEnv = process.env,
): JobIsolationMode {
let raw: string | undefined;
const eqForm = args.find((a) => a.startsWith('--job-isolation='));
if (eqForm !== undefined) raw = eqForm.slice('--job-isolation='.length);
if (raw === undefined) raw = parseFlag(args, '--job-isolation');
if (raw === undefined || raw === '') raw = env.GBRAIN_JOB_ISOLATION;
if (raw === undefined || raw === '') return 'inline';
if (raw === 'inline' || raw === 'process') return raw;
console.error(
`Error: invalid job isolation mode ${JSON.stringify(raw)}. Valid: inline, process.`,
);
process.exit(1);
}
/**
* #3026: the thin-client `list`/`get` branches receive jobs as parsed JSON
* off the MCP wire, where every timestamp is an ISO string but formatJob /
@@ -163,6 +195,7 @@ export function resolveWorkerConcurrency(args: string[], env: NodeJS.ProcessEnv
*/
const JOB_DATE_FIELDS = [
'created_at', 'updated_at', 'started_at', 'finished_at', 'lock_until', 'delay_until',
'timeout_at',
] as const;
export function rehydrateJobDates<T>(job: T): T {
@@ -187,12 +220,41 @@ function formatJob(job: MinionJob): string {
return ` ${String(job.id).padEnd(6)} ${job.name.padEnd(14)} ${(job.status + stalled).padEnd(20)} ${job.queue.padEnd(10)} ${dur.padEnd(8)} ${job.created_at.toISOString().slice(0, 19)}`;
}
function formatJobDetail(job: MinionJob): string {
/** Render a timestamp that is a Date locally but may arrive as an ISO string
* on the thin-client path against an OLDER server (rehydrateJobDates only
* converts fields it knows about; a field the peer predates stays a string).
* Never call .toISOString() unguarded on wire-shaped job fields. */
function formatWhen(v: Date | string | null | undefined): string {
if (v instanceof Date) return v.toISOString();
return String(v ?? '');
}
/** The effective wall-clock budget line for `jobs get`. Wording matters: the
* 1x deadline (handleTimeouts, stamped at claim) is the NORMAL kill; the 2x
* wall-clock sweep is the lock-state-agnostic backstop. */
function formatTimeoutLines(job: MinionJob): string[] {
const lines: string[] = [];
if (job.timeout_ms != null) {
lines.push(` Timeout: ${job.timeout_ms}ms (deadline kill at 1x when claimed; wall-clock backstop at 2x)`);
if (job.timeout_at) lines.push(` Deadline: ${formatWhen(job.timeout_at)}`);
} else {
const d = defaultTimeoutMsFor(job.name);
if (d != null) {
lines.push(` Timeout: (unset) — handler default ${d}ms stamps at claim`);
} else {
lines.push(` Timeout: (unset) — null-default wall-clock sweep applies (2 x lock-duration x max_stalled, ~5m at defaults)`);
}
}
return lines;
}
export function formatJobDetail(job: MinionJob): string {
const lines = [
`Job #${job.id}: ${job.name} (${job.status.toUpperCase()}${job.status === 'dead' ? ` after ${job.attempts_made} attempts` : ''})`,
` Queue: ${job.queue} | Priority: ${job.priority}`,
` Attempts: ${job.attempts_made}/${job.max_attempts} (started: ${job.attempts_started}, stalled: ${job.stalled_counter}/${job.max_stalled})`,
` Backoff: ${job.backoff_type} ${job.backoff_delay}ms (jitter: ${job.backoff_jitter})`,
...formatTimeoutLines(job),
];
if (job.started_at) lines.push(` Started: ${job.started_at.toISOString()}`);
if (job.finished_at) lines.push(` Finished: ${job.finished_at.toISOString()}`);
@@ -210,25 +272,13 @@ function formatJobDetail(job: MinionJob): string {
return lines.join('\n');
}
export async function runJobs(engineOrNull: BrainEngine | null, args: string[]): Promise<void> {
const sub = args[0];
// Thin-client dispatch (cli.ts) passes engine=null for the subcommands
// with remote MCP routing (`list`, `get`) so no scratch local engine is
// ever built. Any other subcommand arriving with a null engine is a
// routing bug upstream of this function — refuse instead of crashing
// inside MinionQueue.
if (!engineOrNull && sub !== 'list' && sub !== 'get') {
console.error(`\`gbrain jobs ${sub ?? ''}\` needs a local engine and cannot run on a thin client.`);
process.exit(1);
}
// Null only ever reaches the MCP-routed `list`/`get` branches, which
// never touch the engine — narrowed once here so the host-only cases
// below typecheck unchanged.
const engine = engineOrNull as BrainEngine;
if (!sub || sub === '--help' || sub === '-h') {
console.log(`gbrain jobs — Minions job queue
/**
* The full jobs help block. Hoisted to a constant so `gbrain jobs --help`
* (routed engine-free via cli.ts SELF_HELP_WITHOUT_ENGINE) and bare
* `gbrain jobs` print the same text. Issue: jobs --help used to print the
* generic CLI stub because 'jobs' was missing from CLI_ONLY_SELF_HELP.
*/
const JOBS_HELP = `gbrain jobs — Minions job queue
USAGE
gbrain jobs submit <name> [--params JSON] [--follow] [--priority N]
@@ -245,15 +295,18 @@ USAGE
gbrain jobs retry <id>
gbrain jobs prune [--older-than 30d] [--dry-run]
gbrain jobs delete <id>
gbrain jobs stats
gbrain jobs smoke
gbrain jobs stats [--queue Q] [--cluster-errors]
gbrain jobs smoke [--sigkill-rescue] [--wedge-rescue]
gbrain jobs watch [--json] [--follow] [--refresh-ms=N]
gbrain jobs work [--queue Q] [--concurrency N] [--max-rss MB]
[--health-interval MS] [--nice N]
[--job-isolation inline|process]
gbrain jobs supervisor [start] [--detach] [--json]
[--concurrency N] [--queue Q] [--pid-file PATH]
[--max-crashes N] [--health-interval N]
[--allow-shell-jobs] [--cli-path PATH]
[--max-rss MB] [--nice N]
[--job-isolation inline|process]
--nice N OS scheduling priority, -20 (highest) to 19 (nicest). Lowers CPU
priority without cutting concurrency full throughput when the
@@ -266,8 +319,9 @@ USAGE
Auto-restarting wrapper around 'gbrain jobs work'. Spawns the worker
as a child process and restarts on crash with exponential backoff
(1s -> 60s cap). Writes a PID file to ~/.gbrain/supervisor.pid by
default (override via --pid-file or GBRAIN_SUPERVISOR_PID_FILE env).
(1s -> 60s cap). Writes a brain-scoped PID file to
~/.gbrain/supervisor-<brain-id>.pid by default (override via
--pid-file or GBRAIN_SUPERVISOR_PID_FILE env).
Lifecycle events are appended to
\${GBRAIN_AUDIT_DIR:-~/.gbrain/audit}/supervisor-YYYY-Www.jsonl
@@ -305,9 +359,180 @@ HANDLER TYPES (built in)
shell Run a command or argv. Requires GBRAIN_ALLOW_SHELL_JOBS=1
on the worker. Params: {cmd?, argv?, cwd, env?}.
See: docs/guides/minions-shell-jobs.md
`);
Detailed help: gbrain jobs {work|supervisor|submit|watch|prune} --help
Other subcommands are fully described above.
`;
/**
* Per-subcommand help for the flag-heavy / side-effectful subcommands.
* Pattern from bootstrap.ts SUBCOMMAND_HELP: the guard below prints these
* BEFORE the switch, so \`jobs work --help\` can never start a worker
* daemon (the defect class this record exists to prevent). Subcommands
* without an entry fall back to JOBS_HELP, which documents them fully.
*/
const JOBS_SUBCOMMAND_HELP: Record<string, string> = {
work: `gbrain jobs work — start a worker daemon (Postgres only)
USAGE
gbrain jobs work [--queue Q] [--concurrency N] [--max-rss MB]
[--health-interval MS] [--nice N]
[--job-isolation inline|process]
OPTIONS
--queue Q Queue to claim from (default: default)
--job-isolation M inline (default): handlers run in the worker process.
process: each claimed job runs in its own child
process a stuck handler is group-SIGKILLed instead
of abandoned, and a crash takes one job, not all N.
Env fallback: GBRAIN_JOB_ISOLATION. Recommended for
long-running LLM-bound handlers (subagent). Note:
--max-rss then covers the worker only, and each child
adds ~4 pooler client connections.
--concurrency N Max jobs in flight. Resolution: flag, then
GBRAIN_WORKER_CONCURRENCY env, then 1. Values < 1
are clamped to 1 with a loud stderr note.
--max-rss MB RSS watchdog. Absent: auto-sized to 50% of
min(cgroup limit, host RAM), capped at 16384 MB,
raised to a 4096 MB floor when the basis allows.
0 disables the watchdog. Values 1-255 are rejected
(megabytes, not gigabytes unit-confusion guard).
--health-interval MS Health probe cadence (default 60000). 0 disables.
Values 1-999 are rejected as unit confusion.
Under GBRAIN_SUPERVISED=1 stall detection is off;
the DB probe stays.
--nice N OS scheduling priority, -20 (highest) to 19
(nicest). Env fallback: GBRAIN_NICE; flag wins.
Negative values need root.
NOTES
Requires the Postgres engine PGLite's exclusive file lock cannot host
a long-lived daemon. For crash-resilient operation prefer:
gbrain jobs supervisor start --detach --json
`,
supervisor: `gbrain jobs supervisor — auto-restarting wrapper around 'gbrain jobs work'
USAGE
gbrain jobs supervisor [start] [--detach] [--json]
[--concurrency N] [--queue Q] [--pid-file PATH]
[--max-crashes N] [--health-interval N]
[--allow-shell-jobs] [--cli-path PATH]
[--max-rss MB] [--nice N]
[--job-isolation inline|process]
gbrain jobs supervisor status [--json] [--pid-file PATH]
gbrain jobs supervisor stop [--json] [--pid-file PATH]
OPTIONS (start)
--detach Fork and print {event, supervisor_pid, pid_file} JSON
--json JSONL lifecycle events on stdout
--concurrency N Worker concurrency (default 2)
--queue Q Queue to claim from (default: default)
--pid-file PATH PID file (default: brain-scoped
~/.gbrain/supervisor-<brain-id>.pid;
env GBRAIN_SUPERVISOR_PID_FILE)
--max-crashes N Soft crash threshold (default 10): past N crashes in
24h the supervisor reports degraded and keeps backing
off. It only STOPS permanently at the hard ceiling
default 10 x N; override or disable (0 = never) via
GBRAIN_SUPERVISOR_HARD_STOP_CRASHES.
--health-interval N Worker health probe cadence in ms
--allow-shell-jobs Enable the shell handler on the spawned worker
--cli-path PATH Explicit gbrain binary for the worker child
--max-rss MB RSS watchdog for the worker (same rules as jobs work)
--nice N OS priority for supervisor + worker children
--job-isolation M Passed through to the worker (see jobs work --help)
EXIT CODES (start)
0 clean shutdown 1 max crashes exceeded
2 another supervisor holds the PID lock 3 PID file unwritable
4 DB queue lock lost (repeated refresh failures; restart re-acquires)
`,
submit: `gbrain jobs submit — enqueue a background job
USAGE
gbrain jobs submit <name> [--params JSON] [--follow] [--priority N]
[--delay Nms] [--max-attempts N] [--max-stalled N]
[--max-waiting N]
[--backoff-type fixed|exponential] [--backoff-delay Nms]
[--backoff-jitter 0..1] [--timeout-ms Nms]
[--idempotency-key K] [--queue Q] [--dry-run]
[--redact-secrets]
OPTIONS
--params JSON Job payload (handler-specific; see HANDLER TYPES in
'gbrain jobs --help')
--follow Run inline and stream progress (constructs a real
worker; works on both engines)
--priority N Lower runs first (default 0)
--delay Nms Delay before the job becomes claimable (default 0)
--max-attempts N Retry budget (default 3)
--max-stalled N Stall-requeue budget before dead-letter (default 5)
--max-waiting N Backpressure: cap waiting jobs with this name/queue/
source before coalescing new submissions ([1,100])
--timeout-ms Nms Per-job wall-clock budget. Long-lane handlers get a
default from HANDLER_DEFAULT_TIMEOUT_MS when omitted.
--idempotency-key K At-most-one row per key (dead/cancelled free the key)
--queue Q Target queue (default: default)
--dry-run Print what would be submitted, submit nothing
--redact-secrets (shell jobs) scrub inherited env values from output
`,
watch: `gbrain jobs watch — live queue dashboard
USAGE
gbrain jobs watch [--json] [--follow] [--refresh-ms=N]
OPTIONS
--json JSON snapshots instead of the human dashboard
--follow Keep refreshing (default: on for TTY, off otherwise)
--refresh-ms=N Refresh cadence in ms (default 1000). Equals form only
'watch' does not accept a space-separated value.
`,
prune: `gbrain jobs prune — delete old terminal jobs
USAGE
gbrain jobs prune [--older-than 30d] [--dry-run]
OPTIONS
--older-than AGE Delete completed/failed/dead/cancelled jobs older than
AGE in days (default 30d; bare N or Nd hour forms
are not supported)
--dry-run Report what would be deleted without deleting
`,
};
export async function runJobs(engineOrNull: BrainEngine | null, args: string[]): Promise<void> {
const sub = args[0];
// Help guards run BEFORE the thin-client refusal below: cli.ts routes
// `jobs … --help` here engine-free (SELF_HELP_WITHOUT_ENGINE), and help
// must never require an engine — or worse, fall through to a subcommand
// body and start a real daemon. Only --help/-h are recognized; the bare
// word 'help' is NOT (e.g. `jobs submit help` is a legitimate job name).
if (!sub || sub === '--help' || sub === '-h') {
console.log(JOBS_HELP);
return;
}
if (args.slice(1).includes('--help') || args.slice(1).includes('-h')) {
// Object.hasOwn: a plain-object lookup resolves inherited keys, so
// `jobs constructor --help` (toString/valueOf/…) would print the
// Object.prototype function instead of falling back to the full help.
console.log(Object.hasOwn(JOBS_SUBCOMMAND_HELP, sub) ? JOBS_SUBCOMMAND_HELP[sub] : JOBS_HELP);
return;
}
// Thin-client dispatch (cli.ts) passes engine=null for the subcommands
// with remote MCP routing (`list`, `get`) so no scratch local engine is
// ever built. Any other subcommand arriving with a null engine is a
// routing bug upstream of this function — refuse instead of crashing
// inside MinionQueue.
if (!engineOrNull && sub !== 'list' && sub !== 'get') {
console.error(`\`gbrain jobs ${sub ?? ''}\` needs a local engine and cannot run on a thin client.`);
process.exit(1);
}
// Null only ever reaches the MCP-routed `list`/`get` branches, which
// never touch the engine — narrowed once here so the host-only cases
// below typecheck unchanged.
const engine = engineOrNull as BrainEngine;
// The constructor just stores the reference; on the null (thin-client
// list/get) paths no queue method is ever reached.
@@ -720,6 +945,67 @@ HANDLER TYPES (built in)
` gbrain jobs retry <id> # for dead-lettered jobs`,
);
}
// Backpressure visibility: maxPending suppression keeps `waiting` at 0
// while a job is in flight, which silences the waiting>0 wedge line
// above — the exact operator-confusion cost of the duplicate-cycle
// incident. Surface the last 24h of coalesce events (per name, this
// queue) from the backpressure audit JSONL, plus a hint naming the
// in-flight job when a name shows suppression with zero waiting rows
// and a stale live-lock active. Best-effort: unreadable audit files
// simply omit the line.
try {
const { readRecentCoalesceCounts } = await import('../core/minions/backpressure-audit.ts');
const coalesceCounts = readRecentCoalesceCounts({ queue: statsQueue, windowMs: 24 * 3600_000 });
if (coalesceCounts.size > 0) {
// Sort once, reuse for the summary AND the hint slice — slicing
// insertion order would let low-volume early-in-file names crowd
// out the highest-volume (most likely wedged) ones the summary
// line just highlighted.
const sortedCoalesces = [...coalesceCounts.entries()]
.sort((a, b) => b[1].count - a[1].count);
const parts = sortedCoalesces.map(([name, s]) => `${name}: ${s.count}`);
console.log(`\n Backpressure (24h): submissions coalesced onto in-flight jobs — ${parts.join(', ')}`);
// Hint loop is bounded: names come from the 24h audit window
// (normally a handful), capped defensively — this is an
// operator-invoked diagnostic, not a hot path. Each hint is
// driven by the LATEST coalesce target for the name (the audit's
// returned_job_id), scoped to that job's source — a name-wide
// aggregate would let source A's waiting row mask source B's
// wedge, or name A's job for B's coalesce (multi-source brains).
const hints = sortedCoalesces.slice(0, 10);
for (const [name, summary] of hints) {
if (summary.last_returned_job_id == null) continue;
// The target CTE re-checks name+queue: the audit dir is shared
// across brains in one GBRAIN_HOME, so an id from another
// brain's audit trail must fail the match here rather than
// name an unrelated job as the suppressor.
const rows = await engine.executeRaw<{ waiting: string; live_id: string | null; age_min: string | null }>(
`WITH target AS (
SELECT id, started_at, status, lock_until,
COALESCE(data->>'sourceId', data->>'source_id') AS scope
FROM minion_jobs WHERE id = $3 AND name = $1 AND queue = $2
)
SELECT (SELECT count(*)::text FROM minion_jobs m, target t
WHERE m.name = $1 AND m.queue = $2 AND m.status = 'waiting'
AND COALESCE(m.data->>'sourceId', m.data->>'source_id') IS NOT DISTINCT FROM t.scope) AS waiting,
(SELECT id::text FROM target WHERE status = 'active' AND lock_until > now()) AS live_id,
(SELECT floor(EXTRACT(EPOCH FROM (now() - started_at)) / 60)::text FROM target
WHERE status = 'active' AND lock_until > now()) AS age_min`,
[name, statsQueue, summary.last_returned_job_id],
);
const r = rows[0];
const ageMin = r?.age_min != null ? parseInt(r.age_min, 10) : null;
if (r && parseInt(r.waiting ?? '0', 10) === 0 && r.live_id != null && ageMin != null && ageMin > wedgeMins) {
console.log(
` ${name}: dispatch suppressed by in-flight job #${r.live_id} (age ${ageMin}m) — check \`gbrain jobs get ${r.live_id}\``,
);
}
}
}
} catch {
// Audit read is advisory; never break stats.
}
}
// v0.41 Bug 2 / Eng D8 — surface lease pressure to the operator.
@@ -943,6 +1229,59 @@ HANDLER TYPES (built in)
process.exit(0);
}
case 'run-child': {
// INTERNAL (issue #5 process isolation): spawned by `jobs work` with
// process isolation enabled. One job, one process: validate the claim,
// run the handler with the child's own engine, write ONE outcome file,
// exit. Deliberately absent from user-facing help. The CLI layer owns
// engine.disconnect() + process.exit() (engine-ownership invariant).
{
const config = loadConfig();
if (config?.engine === 'pglite') {
console.error('[run-child] process isolation requires the Postgres engine.');
await engine.disconnect();
process.exit(JOB_CHILD_EXIT_USAGE);
}
const jobIdRaw = parseFlag(args, '--job-id');
const jobId = jobIdRaw != null ? parseInt(jobIdRaw, 10) : NaN;
const lockToken = process.env[CHILD_ENV.lockToken];
const resultPath = process.env[CHILD_ENV.resultPath];
const parentPidRaw = parseInt(process.env[CHILD_ENV.parentPid] ?? '0', 10);
if (!Number.isInteger(jobId) || jobId <= 0 || !lockToken || !resultPath) {
console.error(
'[run-child] internal command spawned by the jobs worker; requires ' +
`a numeric job id plus ${CHILD_ENV.lockToken} and ${CHILD_ENV.resultPath} in env.`,
);
await engine.disconnect();
process.exit(JOB_CHILD_EXIT_USAGE);
}
// Same handler surface as the worker: registerBuiltinHandlers also
// performs plugin discovery, so plugin subagent jobs isolate too.
const throwaway = new MinionWorker(engine, { queue: 'default', concurrency: 1 });
await registerBuiltinHandlers(throwaway, engine, { quiet: true });
let code: number;
try {
code = await runChildJobEntry(
engine,
{
jobId,
lockToken,
resultPath,
parentPid: Number.isInteger(parentPidRaw) && parentPidRaw > 0 ? parentPidRaw : 0,
},
{ resolveHandler: (name) => throwaway.getHandler(name) },
);
} catch (e) {
console.error(`[run-child] fatal: ${e instanceof Error ? e.message : String(e)}`);
code = 1;
}
await engine.disconnect();
process.exit(code);
}
}
// eslint-disable-next-line no-fallthrough -- unreachable: the case above always exits
case 'work': {
// Check if PGLite
const config = (await import('../core/config.ts')).loadConfig();
@@ -1003,11 +1342,78 @@ HANDLER TYPES (built in)
}
}
// issue #5: per-job process isolation. Resolve + validate the child CLI
// invocation ONCE at startup and refuse to start on failure — a bad
// path discovered per-job would release every claim as infra failures
// (never dead-lettering, but never progressing either).
const jobIsolation = parseJobIsolationFlag(args);
let childCliInvocation: { cmd: string; argsPrefix: string[] } | null = null;
let childTiniPath = '';
if (jobIsolation === 'process') {
const { resolveGbrainCliPath } = await import('./autopilot.ts');
const inv = resolveChildCliInvocation(
process.env,
process.execPath,
process.argv[1],
() => resolveGbrainCliPath(),
);
if (!inv) {
console.error(
'Error: process isolation needs a resolvable gbrain CLI for job children ' +
'(compiled binary on PATH, or GBRAIN_JOB_CHILD_CLI override).',
);
process.exit(1);
}
// Canonicalize BEFORE validating: existsSync on a relative name checks
// cwd while spawn() resolves via PATH — the validated file and the
// executed binary could differ (security review). Resolving to an
// absolute path makes the fail-fast check and the spawn agree.
const { existsSync: childCliExists } = await import('node:fs');
const { resolve: resolveCliPath } = await import('node:path');
inv.cmd = resolveCliPath(inv.cmd);
if (!childCliExists(inv.cmd)) {
console.error(
`Error: resolved child CLI does not exist: ${inv.cmd} ` +
'(set GBRAIN_JOB_CHILD_CLI to a valid gbrain binary).',
);
process.exit(1);
}
childCliInvocation = inv;
const { detectTini } = await import('../core/minions/spawn-helpers.ts');
childTiniPath = detectTini();
if (maxRssMb > 0) {
console.error(
'[gbrain jobs] note: with process isolation on, the --max-rss watchdog covers the ' +
'WORKER process only — handler memory now lives in job children. Per-child caps are ' +
'a filed follow-up; size host memory for concurrency x handler footprint.',
);
}
}
try { await queue.ensureSchema(); }
catch (e) { console.error(e instanceof Error ? e.message : String(e)); process.exit(1); }
// issue #6: the direct-pool kill switch collapses lock renewal, health
// probes, and handler workload onto ONE shared pool — silently. Make
// the collapse loud at startup so a later 'pool_starved' incident has
// an obvious prior warning instead of a mystery.
{
const { getConnectionRouting } = await import('../core/minions/db-probe.ts');
const cm = getConnectionRouting(engine);
if (cm?.isDualPoolActive && !cm.isDualPoolActive()) {
const killSwitched = cm.describeMode?.().kill_switch_active === true;
console.error(
`[gbrain jobs] single-pool mode: lock renewal, health probes and handler workload share ` +
`one connection pool${killSwitched ? ' (direct-lane kill switch is active)' : ''}. ` +
`Under heavy handler load this pool can starve the lock heartbeat. For Supabase brains, ` +
`ensure the direct (5432) host is reachable or set GBRAIN_DIRECT_DATABASE_URL.`,
);
}
}
const worker = new MinionWorker(engine, {
queue: queueName, concurrency, maxRssMb, healthCheckInterval,
jobIsolation, childCliInvocation, childTiniPath,
});
await registerBuiltinHandlers(worker, engine);
@@ -1017,9 +1423,37 @@ HANDLER TYPES (built in)
// the external PM (systemd, Docker, cron watchdog) restart cleanly.
worker.on('unhealthy', (info) => {
if (info.reason === 'db_dead') {
// issue #6: name the failing LAYER, not just "DB unreachable" —
// that message sent operators chasing database capacity while the
// real fault was client-side pool exhaustion. Exiting is still
// correct recovery either way (it frees every client-held slot).
if (info.verdict === 'pool_starved') {
console.error(
`[health] FATAL: connection-pool path saturated after ${info.consecutiveFailures} probes — ` +
`the database server itself is reachable. (${info.message}) ` +
`Likely causes: long-running handler queries holding pool slots, or too-small GBRAIN_POOL_SIZE ` +
`for this workload. Consider --job-isolation process for long-running handlers ` +
`(handler connections then die with each job's child process). ` +
`Exiting for process-manager restart (frees all client-held slots).`,
);
} else if (info.verdict === 'server_unreachable') {
console.error(
`[health] FATAL: database server unreachable after ${info.consecutiveFailures} probes ` +
`(both pooler and direct lanes failed). (${info.message}) ` +
`Exiting for process-manager restart.`,
);
} else {
console.error(
`[health] FATAL: DB probe failed ${info.consecutiveFailures} consecutive times (${info.message}). ` +
`Exiting for process-manager restart.`,
);
}
} else if (info.reason === 'child_spawn_failing') {
console.error(
`[health] FATAL: DB unreachable after ${info.consecutiveFailures} probes (${info.message}). ` +
`Exiting for process-manager restart.`,
`[health] FATAL: ${info.consecutiveFailures} consecutive job-child spawn/bootstrap ` +
`failures (${info.message}). The child CLI is deterministically broken — fix the ` +
`worker's child CLI configuration (or GBRAIN_JOB_CHILD_CLI). Exiting for ` +
`process-manager restart.`,
);
} else {
console.error(
@@ -1048,7 +1482,10 @@ HANDLER TYPES (built in)
: `, health-check: ${Math.round(healthCheckInterval / 1000)}s`)
: '';
const niceNote = niceResult ? `, nice: ${formatNice(niceResult.effective ?? niceVal!)}` : '';
console.log(`Minion worker started (queue: ${queueName}, concurrency: ${concurrency}${watchdogNote}${healthNote}${niceNote})`);
const isolationNote = jobIsolation === 'process'
? `, isolation: process (child cli: ${childCliInvocation?.cmd}${childTiniPath ? ', tini' : ''})`
: '';
console.log(`Minion worker started (queue: ${queueName}, concurrency: ${concurrency}${watchdogNote}${healthNote}${niceNote}${isolationNote})`);
console.log(`Registered handlers: ${worker.registeredNames.join(', ')}`);
// Register in the live worker registry (issue #1815) so jobs stats / doctor
@@ -1367,6 +1804,7 @@ HANDLER TYPES (built in)
allowShellJobs,
json: jsonMode,
maxRssMb,
jobIsolation: parseJobIsolationFlag(args),
...(supNice !== undefined ? { nice_requested: supNice } : {}),
...(supNiceResult?.effective != null ? { nice_effective: supNiceResult.effective } : {}),
...(supNiceResult?.error ? { nice_error: supNiceResult.error } : {}),
@@ -1695,10 +2133,11 @@ export async function registerBuiltinHandlers(
});
worker.register('import', async (job) => {
// import.ts Core extraction deferred to v0.12.0 (import has parallel
// workers + checkpointing). Keep the CLI wrapper call but note the
// worker-kill risk is bounded: import's only process.exit fires on
// a missing dir arg, which this handler always passes.
// import.ts Core extraction deferred (import has parallel workers +
// checkpointing; the typed-API split lands in W7 of the fix-wave).
// W0 (Tier-1 #5): runImport no longer contains ANY process.exit — all
// five preflight sites throw typed ImportAbortError, which this
// handler's catch converts to a normal failJob. No worker-kill risk.
const { runImport } = await import('./import.ts');
const importArgs: string[] = [];
if (job.data.dir) importArgs.push(String(job.data.dir));
+52 -45
View File
@@ -441,6 +441,24 @@ export interface LintOpts {
* single-file targets.
*/
exclude?: string[];
/**
* W0 fix-wave (Tier-1 #14): per-page hook fired for every page WITH
* issues, after this run's fix attempt for that page fixedCount is the
* number of fixes just applied (0 when --fix is off or nothing was
* fixable). The CLI passes a printer so human detail and the aggregate
* counts come from ONE scan pre-fix, runLint ran its own full
* read+lint+fix loop and THEN called runLintCore for the summary, linting
* every page twice and reporting "0 auto-fixed" because the second pass
* saw already-fixed files.
*/
onPageIssues?: (relPath: string, issues: LintIssue[], fixedCount: number) => void;
/** Companion to onPageIssues: per-page progress tick (CLI progress bar). */
onPageScanned?: () => void;
/**
* Fired once with the collected page count before scanning starts, so the
* CLI can size its progress bar without walking the tree a second time.
*/
onPagesCollected?: (count: number) => void;
}
export interface LintResult {
@@ -468,6 +486,7 @@ export async function runLintCore(opts: LintOpts): Promise<LintResult> {
const isSingleFile = statSync(opts.target).isFile();
const pages = isSingleFile ? [opts.target] : collectPages(opts.target, opts.exclude ?? []);
opts.onPagesCollected?.(pages.length);
// Resolve content-sanity config once for this lint run (D1: lift DB
// config when reachable). Caller can pre-pass via opts.contentSanity
@@ -490,21 +509,25 @@ export async function runLintCore(opts: LintOpts): Promise<LintResult> {
await new Promise<void>((resolve) => setImmediate(resolve));
}
const content = readFileSync(page, 'utf-8');
const issues = lintContent(content, isSingleFile ? page : relative(opts.target, page), lintOpts);
const relPath = isSingleFile ? page : relative(opts.target, page);
const issues = lintContent(content, relPath, lintOpts);
opts.onPageScanned?.();
if (issues.length === 0) continue;
pagesWithIssues++;
totalIssues += issues.length;
let fixCount = 0;
if (opts.fix && issues.some(i => i.fixable)) {
const fixed = fixContent(content);
if (fixed !== content) {
const fixCount = issues.filter(i => i.fixable).length;
fixCount = issues.filter(i => i.fixable).length;
totalFixed += fixCount;
if (!opts.dryRun) {
writeFileSync(page, fixed);
}
}
}
opts.onPageIssues?.(relPath, issues, fixCount);
}
return {
@@ -547,57 +570,41 @@ export async function runLint(args: string[]) {
process.exit(1);
}
// Single file or directory — print human detail as we go, then rely on
// Core for the aggregate numbers at the end.
const isSingleFile = statSync(target).isFile();
const pages = isSingleFile ? [target] : collectPages(target, extraExcludes);
// W0 fix-wave (Tier-1 #14): ONE scan. Pre-fix this function ran its own
// full read+lint+fix loop for human output and THEN called runLintCore for
// the summary — every page linted twice, and with --fix the second pass
// saw already-fixed files so the summary reported "0 auto-fixed" after
// fixing N. Human detail now streams from runLintCore's per-page hooks
// and the counts come from the same single pass.
// Progress on stderr. Stdout keeps the per-issue human output it always had.
// Ship-review perf catch: the tree is walked ONCE — runLintCore reports the
// collected count via onPagesCollected (pre-fix the CLI ran its own
// collectPages just to size the progress bar, a second full readdir/stat
// walk on every directory lint).
const { createProgress } = await import('../core/progress.ts');
const { getCliOptions, cliOptsToProgressOptions } = await import('../core/cli-options.ts');
const progress = createProgress(cliOptsToProgressOptions(getCliOptions()));
progress.start('lint.pages', pages.length);
// v0.41 (D1): resolve content-sanity config once for this lint run.
// Mirrors runLintCore. The two paths must agree because runLint
// prints human details inline; runLintCore at end computes the
// aggregate. Sharing the resolved opts keeps both surfaces seeing
// the same rule firings.
const contentSanity = await resolveLintContentSanity();
const lintContentOpts: LintContentOpts = { contentSanity };
for (const page of pages) {
const content = readFileSync(page, 'utf-8');
const relPath = isSingleFile ? page : relative(target, page);
const issues = lintContent(content, relPath, lintContentOpts);
progress.tick(1);
if (issues.length === 0) continue;
console.log(`\n${relPath}:`);
for (const issue of issues) {
const fixLabel = issue.fixable ? ' [fixable]' : '';
console.log(` L${issue.line} ${issue.rule}: ${issue.message}${fixLabel}`);
}
if (doFix && issues.some(i => i.fixable)) {
const fixed = fixContent(content);
if (fixed !== content) {
const fixCount = issues.filter(i => i.fixable).length;
if (!dryRun) {
writeFileSync(page, fixed);
}
console.log(` ${dryRun ? '(dry run) ' : ''}Fixed ${fixCount} issue(s)`);
const result = await runLintCore({
target,
fix: doFix,
dryRun,
exclude: extraExcludes,
onPagesCollected: (count) => progress.start('lint.pages', count),
onPageScanned: () => progress.tick(1),
onPageIssues: (relPath, issues, fixedCount) => {
console.log(`\n${relPath}:`);
for (const issue of issues) {
const fixLabel = issue.fixable ? ' [fixable]' : '';
console.log(` L${issue.line} ${issue.rule}: ${issue.message}${fixLabel}`);
}
}
}
if (fixedCount > 0) {
console.log(` ${dryRun ? '(dry run) ' : ''}Fixed ${fixedCount} issue(s)`);
}
},
});
progress.finish();
// Re-run core for the aggregate counts (cheap; re-parses contents but
// produces canonical numbers for the summary line).
// Pass contentSanity through so runLintCore skips its own resolve
// (we already resolved once for the human-detail loop above).
const result = await runLintCore({ target, fix: doFix, dryRun, contentSanity, exclude: extraExcludes });
console.log(`\n${result.pages_scanned} pages scanned. ${result.total_issues} issue(s) in ${result.pages_with_issues} page(s).`);
if (doFix) {
console.log(`${dryRun ? '(dry run) ' : ''}${result.total_fixed} auto-fixed.`);
+23 -2
View File
@@ -32,6 +32,7 @@ import {
DEFAULT_ALIASES,
TIER_DEFAULTS,
resolveModel,
resolveAlias,
type ModelTier,
} from '../core/model-config.ts';
import { maybeAttachVersionSuffixHint } from '../core/ai/base-url-probe.ts';
@@ -45,11 +46,23 @@ interface PerTaskModelRoute {
description: string;
deprecatedConfigKey?: string;
envVar?: string;
/**
* #4152 (2A): an explicit pre-read key that wins over the whole
* resolveModel chain when set mirrors loadSynthConfig's triage-model
* resolution so the dashboard reports the ACTUAL spending route.
*/
overrideKey?: string;
}
const PER_TASK_KEYS: PerTaskModelRoute[] = [
{ key: 'models.dream.synthesize', tier: 'reasoning', description: 'Dream synthesis (conversation → brain pages)' },
{ key: 'models.dream.synthesize_verdict', tier: 'utility', description: 'Dream synthesis verdict (Haiku judge)' },
{
key: 'models.dream.synthesize_verdict',
tier: 'utility',
description: 'Dream triage judge (scored gate; models.dream.triage preferred)',
deprecatedConfigKey: 'dream.synthesize.verdict_model',
overrideKey: 'models.dream.triage',
},
{ key: 'models.dream.patterns', tier: 'reasoning', description: 'Pattern discovery (cross-take themes)' },
{ key: 'models.drift', tier: 'reasoning', description: 'Drift LLM judge (v0.29 scaffold)' },
{ key: 'models.auto_think', tier: 'deep', description: 'Auto-think question answering' },
@@ -128,7 +141,15 @@ async function buildReport(engine: BrainEngine): Promise<ModelsReport> {
const per_task: ModelsReport['per_task'] = [];
for (const route of PER_TASK_KEYS) {
const { key, tier, description, deprecatedConfigKey, envVar } = route;
const { key, tier, description, deprecatedConfigKey, envVar, overrideKey } = route;
// Explicit pre-read override (loadSynthConfig 2A parity): when set, it IS
// the effective spending route and must be reported as such.
const overrideValue = overrideKey ? await engine.getConfig(overrideKey) : null;
if (overrideKey && overrideValue?.trim()) {
const resolved = await resolveAlias(engine, overrideValue.trim());
per_task.push({ key, tier, resolved, source: `config: ${overrideKey}`, description });
continue;
}
const resolved = await resolveModel(engine, {
configKey: key,
deprecatedConfigKey,
+6
View File
@@ -101,9 +101,15 @@ function emitError(jsonOutput: boolean, code: string, message: string): void {
}
async function promptYesNo(question: string): Promise<boolean> {
// W0 fix-wave (Tier-1 #15): non-interactive stdin (CI, pipes, spawned
// agents) must resolve to the safe default instead of hanging forever —
// this prompt had no TTY guard and no close/EOF handler, so a piped or
// closed stdin parked the process permanently.
if (!process.stdin.isTTY) return false;
// Prompt on stderr: stdout stays clean for --json payloads.
const rl = createInterface({ input: process.stdin, output: process.stderr });
return new Promise((resolve) => {
rl.on('close', () => resolve(false)); // EOF (^D) = decline, never hang
rl.question(`${question} [y/N] `, (answer) => {
rl.close();
resolve(/^y(es)?$/i.test(answer.trim()));
+15 -4
View File
@@ -331,18 +331,29 @@ function fail(jsonOutput: boolean, reason: string, message: string): never {
}
async function promptYesNo(question: string): Promise<boolean> {
// Minimal TTY prompt — no external deps. Bun's process.stdin reads
// a single line synchronously via the async iterator.
process.stdout.write(`${question} (y/N): `);
// W0 fix-wave (Tier-1 #15): non-interactive stdin (CI, pipes, spawned
// agents) resolves to the safe default instead of hanging — this prompt
// had no TTY guard and no end/EOF path, so a closed stdin parked the
// process permanently. This command wipes the store; decline-by-default
// is the only safe non-interactive answer (--yes stays the escape hatch).
if (!process.stdin.isTTY) return false;
// Prompt on stderr so stdout stays clean for --json payloads.
process.stderr.write(`${question} (y/N): `);
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const stdin = process.stdin as any;
stdin.setEncoding?.('utf8');
return new Promise<boolean>((resolve) => {
const cleanup = () => {
stdin.off?.('data', onData);
stdin.off?.('end', onEnd);
};
const onEnd = () => { cleanup(); resolve(false); }; // EOF = decline
const onData = (chunk: string) => {
const answer = chunk.trim().toLowerCase();
stdin.off?.('data', onData);
cleanup();
resolve(answer === 'y' || answer === 'yes');
};
stdin.on?.('data', onData);
stdin.on?.('end', onEnd);
});
}
+20 -3
View File
@@ -36,6 +36,7 @@ import {
} from '../core/oauth-provider.ts';
import type { SqlQuery } from '../core/oauth-provider.ts';
import { hasScope, ALLOWED_SCOPES_LIST, normalizeScopesInput } from '../core/scope.ts';
import { normalizeTokenScopes } from '../core/legacy-token-scope.ts';
import { normalizeSourceInput, normalizeFederatedReadInput } from '../core/source-id.ts';
import { summarizeMcpParams, dispatchToolCall, requestLogStatusForResult } from '../mcp/dispatch.ts';
import { resolveStrictParamsMode } from '../mcp/validate-params.ts';
@@ -550,7 +551,13 @@ export async function queryAgentClientSpend(engine: BrainEngine): Promise<AgentC
SELECT SUM(spend_cents)::text
FROM mcp_spend_log
WHERE client_id = c.client_id
AND created_at >= date_trunc('day', now() AT TIME ZONE 'UTC')
-- Double AT TIME ZONE: the inner one yields NAIVE UTC-midnight;
-- the outer one converts it back to a timestamptz INSTANT. Without
-- it, the naive value is reinterpreted in the SESSION timezone, so
-- any non-UTC session (host-tz PGLite, a tz-configured Postgres
-- role) shifts the day boundary by the offset and today's spend
-- underreports every evening.
AND created_at >= date_trunc('day', now() AT TIME ZONE 'UTC') AT TIME ZONE 'UTC'
), '0') AS spent_cents_today,
COALESCE((
SELECT SUM(estimated_cents)::text
@@ -1365,7 +1372,9 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption
`;
const legacyKeys = await sql`
SELECT a.id, a.name, 'api_key' as auth_type,
'{"bearer"}' as grant_types, 'read write admin' as scope, a.created_at, null as token_ttl,
'{"bearer"}' as grant_types,
a.scopes,
a.created_at, null as token_ttl,
CASE WHEN a.revoked_at IS NOT NULL THEN 'revoked' ELSE 'active' END as status,
a.last_used_at,
(SELECT count(*)::int FROM mcp_request_log WHERE token_name = a.name) as total_requests,
@@ -1374,7 +1383,15 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption
`;
res.json([
...oauthClients,
...legacyKeys.map((key) => ({ ...key, source_id: null, federated_read: [] })),
...legacyKeys.map(({ scopes, ...key }) => ({
...key,
// The SAME normalizer the verify path uses — the dashboard must
// never display a grant the serve doesn't enforce (NULL =
// grandfathered full access; damaged/deny rows show empty).
scope: normalizeTokenScopes(scopes)?.join(' ') ?? 'read write admin',
source_id: null,
federated_read: [],
})),
]);
} catch (e) {
res.status(503).json({ error: 'service_unavailable' });
+449 -25
View File
@@ -1,30 +1,35 @@
/**
* gbrain transcripts Recent raw conversation transcripts.
* gbrain transcripts session transcripts: recent corpus reads and the
* cathedral-4 import lane.
*
* Local-only: this command reads `.txt` files from the dream-cycle corpus
* directories. It exists as a CLI surface so humans can trigger the same
* read path the v0.29 `get_recent_transcripts` MCP op uses (which is itself
* gated on remote=false; subagents and MCP/HTTP callers cannot reach it).
* gbrain transcripts recent dream-corpus .txt reader (v0.29 surface).
* gbrain transcripts ingest import dead session logs (Claude Code,
* Codex, OpenClaw, Hermes) and consumer chat
* exports (ChatGPT, Claude.ai) into
* conversation pages. Local-only, explicit
* paths are trusted CLI input; embedding is
* OFF by default (bulk imports defer to the
* embed backfill lane).
*
* Usage:
* gbrain transcripts recent # last 7 days, summaries
* gbrain transcripts recent --days 14
* gbrain transcripts recent --full # full content (capped at 100KB/file)
* gbrain transcripts recent --json
* PGLite note: like every engine-opening command, ingest cannot run while
* `gbrain serve` holds the single-writer lock the lock error names the PID.
*/
import type { BrainEngine } from '../core/engine.ts';
import { setCliExitVerdict } from '../core/cli-force-exit.ts';
import type { TranscriptFormat } from '../core/transcripts/types.ts';
import { runTranscriptsIngest, type TranscriptsIngestResult } from '../core/transcripts/ingest.ts';
import { isOpenclawCheckpointFile } from '../core/transcripts/openclaw.ts';
interface RunOpts {
interface RecentOpts {
days?: number;
full?: boolean;
limit?: number;
json?: boolean;
}
function parseArgs(args: string[]): RunOpts | { help: true } {
const opts: RunOpts = {};
function parseRecentArgs(args: string[]): RecentOpts | { help: true } {
const opts: RecentOpts = {};
for (let i = 0; i < args.length; i++) {
const a = args[i];
if (a === '--help' || a === '-h') return { help: true };
@@ -44,31 +49,450 @@ function parseArgs(args: string[]): RunOpts | { help: true } {
return opts;
}
const HELP = `Usage: gbrain transcripts recent [options]
const FORMATS: readonly TranscriptFormat[] = [
'claude-code',
'codex',
'openclaw',
'hermes',
'chatgpt',
'claude-export',
];
Recent raw conversation transcripts (NOT polished reflections). Reads from
the dream-cycle corpus dirs (dream.synthesize.session_corpus_dir and
dream.synthesize.meeting_transcripts_dir).
interface IngestCliOpts {
paths: string[];
format?: TranscriptFormat;
dryRun?: boolean;
limit?: number;
since?: string;
source?: string;
facts?: boolean;
maxCostUsd?: number;
embed?: boolean;
all?: boolean;
json?: boolean;
quiet?: boolean;
}
Options:
--days N Window in days (default 7)
--limit N Max transcripts (default 50)
--full Return full content (default: ~300-char summary). Capped 100KB/file.
--json JSON output for agents
--help, -h Show this help
function parseIngestArgs(args: string[]): IngestCliOpts | { help: true } | { error: string } {
const opts: IngestCliOpts = { paths: [] };
for (let i = 0; i < args.length; i++) {
const a = args[i];
if (a === '--help' || a === '-h') return { help: true };
if (a === '--json') { opts.json = true; continue; }
if (a === '--quiet') { opts.quiet = true; continue; }
if (a === '--dry-run') { opts.dryRun = true; continue; }
if (a === '--embed') { opts.embed = true; continue; }
if (a === '--facts') { opts.facts = true; continue; }
if (a === '--all') { opts.all = true; continue; }
if (a === '--format') {
const v = args[++i] as TranscriptFormat | undefined;
if (!v || !FORMATS.includes(v)) {
return { error: `unknown format '${v ?? ''}' (expected one of: ${FORMATS.join(', ')})` };
}
opts.format = v;
continue;
}
if (a === '--limit') {
const n = parseInt(args[++i] ?? '', 10);
if (!Number.isFinite(n) || n <= 0) return { error: 'limit must be a positive integer' };
opts.limit = n;
continue;
}
if (a === '--since') {
const v = args[++i];
if (!v) return { error: 'since needs an ISO timestamp or the word last' };
if (v !== 'last') {
// Validate + Z-normalize: the filter compares lexicographically
// against Z-form ISO, so an offset-form or garbage value would
// silently mis-filter (and a filtered-everything run would still
// look clean).
const d = new Date(v);
if (Number.isNaN(d.getTime())) {
return { error: `since needs a parseable ISO timestamp or the word last (got '${v}')` };
}
opts.since = d.toISOString();
continue;
}
opts.since = v;
continue;
}
if (a === '--source-id' || a === '--source') {
const v = args[++i];
if (!v) return { error: 'source-id needs a value' };
opts.source = v;
continue;
}
if (a === '--max-cost-usd') {
const n = parseFloat(args[++i] ?? '');
if (!Number.isFinite(n) || n <= 0) return { error: 'max-cost-usd must be a positive number' };
opts.maxCostUsd = n;
continue;
}
if (a.startsWith('-')) return { error: `unknown flag ${a}` };
opts.paths.push(a);
}
return opts;
}
Note: dream-generated outputs (frontmatter dream_generated: true) are skipped.
const HELP = `Usage:
gbrain transcripts ingest <path-or-glob>... [options]
gbrain transcripts ingest # discovery: show found session logs
gbrain transcripts ingest --all # import everything discovered
gbrain transcripts status # found vs imported gap table
gbrain transcripts recent [options]
ingest import dead session logs and chat exports as conversation pages
(readable text-turn archive: user/assistant text only, secrets redacted,
long sessions split into searchable parts). Re-runs are free (content-hash
skip). Embedding is OFF by default; run the embed backfill later or opt in.
--all Import every session log discovered under the harness
roots (claude/codex/openclaw projects + the hermes store)
--format F claude-code | codex | openclaw | hermes | chatgpt |
claude-export (auto-detected when omitted)
--dry-run Parse + redact + report; writes nothing
--limit N Max sessions this run
--since T Only sessions newer than ISO time T; the word "last"
resumes from the previous clean run
--source-id S Target source (default: the canonical 6-tier resolution)
--embed Embed pages at import (default: defer to embed backfill)
--facts Extract facts from imported pages (budget-capped)
--max-cost-usd F Facts budget cap (default 5)
--json Machine-readable result
--quiet Suppress the human summary
recent read recent raw dream-corpus transcripts (.txt), newest first:
--days N Window in days (default 7)
--limit N Max transcripts (default 50)
--full Full content, capped 100KB/file (default: short summary)
--json JSON output for agents
Dream-generated outputs (frontmatter dream_generated: true) are skipped.
Notes: consumer exports must be unzipped first (pass conversations.json).
On PGLite, stop gbrain serve first (single-writer lock).
`;
/** Extensions the importer understands; directory expansion filters to these. */
const IMPORTABLE_EXTENSIONS = ['.jsonl', '.db', '.json'];
/**
* Expand path-or-glob args. Directory specs filter to importable extensions
* without the filter, every stray file in a real directory (macOS Finder
* metadata, editor backups, READMEs) becomes a permanent per-file error that
* breaks cleanScan on every run, silently killing the since-last resume for
* directory scopes. Checkpoint snapshots are never imported.
*/
async function expandPaths(specs: string[]): Promise<string[]> {
const { statSync } = await import('node:fs');
const out: string[] = [];
for (const spec of specs) {
let matched = false;
try {
if (statSync(spec).isFile()) {
out.push(spec);
continue;
}
if (statSync(spec).isDirectory()) {
const glob = new Bun.Glob('**/*');
for (const p of glob.scanSync({ cwd: spec, absolute: true, onlyFiles: true })) {
if (IMPORTABLE_EXTENSIONS.some((ext) => p.endsWith(ext))) out.push(p);
}
continue;
}
} catch {
// Not a literal path — try as a glob below.
}
const glob = new Bun.Glob(spec);
for (const p of glob.scanSync({ cwd: process.cwd(), absolute: true, onlyFiles: true })) {
out.push(p);
matched = true;
}
if (!matched && !out.includes(spec)) {
// Keep the unmatched spec so the per-file error names it.
out.push(spec);
}
}
return [...new Set(out)].filter((p) => !isOpenclawCheckpointFile(p));
}
function fmtSummary(r: TranscriptsIngestResult): string {
const byHarness = new Map<string, number>();
for (const f of r.files) {
for (const s of f.sessions) {
if (!s.error) byHarness.set(s.harness, (byHarness.get(s.harness) ?? 0) + 1);
}
}
const lines: string[] = [];
const counts = [...byHarness.entries()].map(([h, n]) => `${h}: ${n}`).join(', ');
lines.push(
`sessions: ${r.sessionsImported} imported (${counts || 'none'}), ` +
`${r.sessionsFiltered} filtered, ${r.sessionsErrored} errored, ${r.sessionsSeen} seen`,
);
lines.push(
`pages: ${r.pages.imported} imported, ${r.pages.skipped} unchanged` +
(r.pages.errored ? `, ${r.pages.errored} ERRORED` : '') +
(r.pages.planned ? `, ${r.pages.planned} planned (dry run)` : '') +
(r.partsDeleted ? `, ${r.partsDeleted} stale parts deleted` : ''),
);
if (r.redactions > 0) lines.push(`redactions: ${r.redactions} secrets/patterns redacted before write`);
if (r.imperatives > 0) lines.push(`flagged: ${r.imperatives} agent-directed imperative(s) noted in frontmatter`);
if (r.driftFiles > 0) {
lines.push(
`DRIFT WARNING: ${r.driftFiles} file(s) parsed to zero sessions — the host ` +
`format may have changed; see the adapter SPEC_TARGET runbook`,
);
}
for (const f of r.files) {
if (f.error) lines.push(`error: ${f.path}: ${f.error}`);
for (const s of f.sessions) {
if (s.error) lines.push(`error: ${f.path} session ${s.sessionId}: ${s.error}`);
}
}
return lines.join('\n');
}
async function runIngest(engine: BrainEngine, args: string[]): Promise<void> {
const parsed = parseIngestArgs(args);
if ('help' in parsed) {
console.log(HELP);
return;
}
if ('error' in parsed) {
console.error(`gbrain transcripts ingest: ${parsed.error}`);
setCliExitVerdict(2);
return;
}
// The watermark fingerprint binds the USER-STATED spec, captured BEFORE
// discovery expands it — binding expanded file lists would mint a new
// fingerprint every time a harness writes a new session, so the all-lane
// since-last would never resume. Specs are RESOLVED first: the same
// relative spec from two different cwds names different scopes (must not
// share a watermark), and equivalent spellings of one dir must not
// fragment into separate watermarks.
const { resolve } = await import('node:path');
const { hostname } = await import('node:os');
// The all-lane scope is THIS machine's harness roots, so the fingerprint
// carries host + roots: checkpoints are DB-backed and shared across every
// machine on the brain — a bare literal would let machine B inherit
// machine A's watermark and silently skip local sessions it never scanned.
const { harnessRoots } = await import('../core/transcripts/detect.ts');
const checkpointSpec =
parsed.paths.length === 0
? ['--all-discovery', hostname(), ...harnessRoots().map((r) => r.root).sort()]
: [...parsed.paths].map((p) => resolve(p)).sort();
// No paths: discovery. Without the all flag, show what WOULD be imported
// and stop (a safe default for a command that can touch four harness
// histories); with it, import the discovered set.
if (parsed.paths.length === 0) {
const { discoverTranscriptFiles } = await import('../core/transcripts/discover.ts');
const discovered = discoverTranscriptFiles();
if (discovered.length === 0) {
console.log('discovery: no session logs found under the harness roots');
return;
}
if (!parsed.all) {
const byFormat = new Map<string, { n: number; bytes: number }>();
for (const d of discovered) {
const cur = byFormat.get(d.format) ?? { n: 0, bytes: 0 };
cur.n++;
cur.bytes += d.bytes;
byFormat.set(d.format, cur);
}
console.log('discovery (nothing imported yet — add the all flag to import):');
for (const [format, { n, bytes }] of byFormat) {
console.log(` ${format.padEnd(12)} ${String(n).padStart(5)} file(s) ${(bytes / 1024 / 1024).toFixed(1)} MB`);
}
console.log(' tip: `gbrain transcripts status` shows found vs imported per harness');
return;
}
parsed.paths = discovered.map((d) => d.path);
}
// Source: the canonical 6-tier chain (capture.ts pattern) — one resolved
// id threads import + raw-data + reconciliation + checkpoint fingerprint.
let sourceId = 'default';
try {
const { resolveSourceWithTier } = await import('../core/source-resolver.ts');
const r = await resolveSourceWithTier(engine, parsed.source ?? null);
sourceId = r.source_id;
} catch (e) {
console.error(`gbrain transcripts ingest: ${e instanceof Error ? e.message : String(e)}`);
setCliExitVerdict(1);
return;
}
// Active pack ONCE per command (never per file).
let activePack: { page_types: ReadonlyArray<{ name: string; path_prefixes: ReadonlyArray<string> }> } | undefined;
try {
const { loadActivePack } = await import('../core/schema-pack/load-active.ts');
const { loadConfig } = await import('../core/config.ts');
const resolved = await loadActivePack({ cfg: loadConfig(), remote: false, sourceId });
activePack = { page_types: resolved.manifest.page_types };
} catch {
activePack = undefined;
}
const paths = await expandPaths(parsed.paths);
if (paths.length === 0) {
console.error('gbrain transcripts ingest: 0 files matched');
return;
}
// --since last → op-checkpoint watermark (speed convenience only; the
// status gap table is the correctness surface). Fingerprint binds
// source + pathspec + format + adapter version so a second source or a
// different root never inherits this watermark.
const { fingerprint, loadOpCheckpoint, recordCompleted } = await import('../core/op-checkpoint.ts');
const { TRANSCRIPT_IMPORT_VERSION } = await import('../core/transcripts/render.ts');
const checkpointKey = {
op: 'transcripts-ingest',
fingerprint: fingerprint({
sourceId,
pathspec: checkpointSpec,
format: parsed.format ?? 'auto',
version: TRANSCRIPT_IMPORT_VERSION,
}),
};
let sinceIso = parsed.since;
if (parsed.since === 'last') {
sinceIso = undefined;
const keys = await loadOpCheckpoint(engine, checkpointKey);
for (const k of keys) {
if (k.startsWith('since:')) {
const v = k.slice('since:'.length);
if (!sinceIso || v > sinceIso) sinceIso = v;
}
}
if (!sinceIso && !parsed.quiet) {
console.error('transcripts ingest: no previous clean run for this scope — full scan');
}
}
const { createProgress } = await import('../core/progress.ts');
const { cliOptsToProgressOptions, getCliOptions } = await import('../core/cli-options.ts');
const reporter = createProgress(cliOptsToProgressOptions(getCliOptions()));
reporter.start('transcripts.ingest', paths.length);
let result: TranscriptsIngestResult;
try {
result = await runTranscriptsIngest(engine, {
paths,
format: parsed.format,
dryRun: parsed.dryRun,
limit: parsed.limit,
sinceIso,
sourceId,
embed: parsed.embed,
activePack,
onFileDone: () => reporter.tick(),
// Multi-session stores (one hermes state.db = thousands of sessions)
// need liveness BETWEEN file ticks.
onSession: (sessionId) => reporter.heartbeat(`session ${sessionId.slice(0, 12)}`),
});
} finally {
reporter.finish();
}
if (!parsed.embed && !parsed.dryRun && result.pages.imported > 0 && !parsed.quiet) {
console.error(
'note: pages imported without embeddings (default) — run the embed backfill ' +
'or re-run with the embed flag to make them vector-searchable now',
);
}
// Watermark: advance ONLY on a clean, untruncated, non-dry scan — and only
// when the run ATTESTED full coverage (no since bound, or since=last). An
// explicit since run never scanned below its cutoff and must not vouch for
// sessions there.
const attestsCoverage = parsed.since === undefined || parsed.since === 'last';
if (result.cleanScan && result.maxSessionTs && attestsCoverage) {
await recordCompleted(engine, checkpointKey, [`since:${result.maxSessionTs}`]);
}
// --facts: ONE extractor invocation over every touched slug (including
// hash-skipped pages — the extractor's version-token gate dedupes work).
let factsSummary: { pages: number; spentUsd?: number } | undefined;
if (parsed.facts && !parsed.dryRun && result.slugsTouched.length > 0) {
const { runIngestFacts } = await import('../core/transcripts/ingest-facts.ts');
factsSummary = await runIngestFacts(engine, {
sourceId,
slugs: [...new Set(result.slugsTouched)],
maxCostUsd: parsed.maxCostUsd,
quiet: parsed.quiet,
});
}
if (parsed.json) {
console.log(JSON.stringify({ ...result, facts: factsSummary ?? null, source_id: sourceId }, null, 2));
} else if (!parsed.quiet) {
console.log(fmtSummary(result));
if (factsSummary) {
console.log(
`facts: extracted over ${factsSummary.pages} page(s)` +
(factsSummary.spentUsd !== undefined ? `, ~$${factsSummary.spentUsd.toFixed(2)} spent` : ''),
);
}
const firstImported = result.files.flatMap((f) => f.sessions).find((s) => !s.error && s.baseSlug);
if (firstImported && !parsed.dryRun) {
console.log(`try it: gbrain query "${firstImported.baseSlug.split('/').pop()}"`);
}
}
const allFailed =
result.files.length > 0 &&
result.files.every((f) => f.error !== undefined || (f.drift && f.sessions.length === 0));
if (allFailed) setCliExitVerdict(1);
}
async function runStatus(engine: BrainEngine, args: string[]): Promise<void> {
const json = args.includes('--json');
let sourceId = 'default';
try {
const { resolveSourceWithTier } = await import('../core/source-resolver.ts');
sourceId = (await resolveSourceWithTier(engine, null)).source_id;
} catch {
// Fall through with default — status is read-only.
}
const { buildStatusRows, discoverTranscriptFiles, indexImportedSessions } = await import(
'../core/transcripts/discover.ts'
);
const rows = buildStatusRows(discoverTranscriptFiles(), await indexImportedSessions(engine, sourceId));
if (json) {
console.log(JSON.stringify({ source_id: sourceId, rows }, null, 2));
return;
}
console.log(`transcripts status (source: ${sourceId})`);
console.log(' harness found imported-sessions not-yet-imported');
for (const r of rows) {
const gap = r.gapFiles === null ? '(store-level; run ingest to see)' : String(r.gapFiles);
console.log(
` ${r.format.padEnd(12)} ${String(r.found).padStart(6)} ${String(r.importedSessions).padStart(12)} ${gap}`,
);
}
const totalGap = rows.reduce((n, r) => n + (r.gapFiles ?? 0), 0);
if (totalGap > 0) {
console.log(` backfill: gbrain transcripts ingest --all (${totalGap} file(s) waiting)`);
}
}
export async function runTranscripts(engine: BrainEngine, args: string[]): Promise<void> {
const sub = args[0];
if (sub === 'ingest') {
await runIngest(engine, args.slice(1));
return;
}
if (sub === 'status') {
await runStatus(engine, args.slice(1));
return;
}
if (sub !== 'recent') {
console.log(HELP);
if (sub && sub !== '--help' && sub !== '-h') setCliExitVerdict(2);
return;
}
const parsed = parseArgs(args.slice(1));
const parsed = parseRecentArgs(args.slice(1));
if ('help' in parsed) {
console.log(HELP);
return;
+103
View File
@@ -0,0 +1,103 @@
/**
* atomic-write.ts the ONE atomic config-file writer for bootstrap host
* surfaces (rule-of-three extraction: hooks.ts settings JSON, codex-toml.ts
* TOML text, opencode-json.ts JSONC text all swap through here).
*
* Semantics, hardened for shared user-scope targets [C10 / X11]:
* - The SYMLINK TARGET is resolved first so a dotfile-manager-linked config
* survives as a link (a bare rename would replace the link with a regular
* file). DANGLING links are resolved too (readlink, hop by hop): the write
* creates the missing target and the link survives.
* - tmp file uses a random suffix and inherits the EXISTING file's mode; a
* fresh file takes `freshMode` (caller's convention secret-bearing
* targets pass 0o600). `forceMode` overrides both (codex-toml forces 0600
* because the file carries a bearer token regardless of its prior mode).
* - chmod after write because writeFileSync's mode applies only on create.
*
* EOL and serialization stay caller-side: hooks.ts stringifies JSON,
* codex-toml converts to CRLF when the original was CRLF, opencode-json
* preserves EOLs naturally via jsonc-parser text splicing.
*/
import { randomBytes } from 'node:crypto';
import {
chmodSync,
existsSync,
lstatSync,
mkdirSync,
readlinkSync,
realpathSync,
renameSync,
statSync,
unlinkSync,
writeFileSync,
} from 'node:fs';
import { dirname, isAbsolute, resolve } from 'node:path';
/** Resolve the write target through symlinks, INCLUDING dangling ones.
* existsSync follows symlinks, so a DANGLING link reads "absent" and a bare
* rename would replace the link itself with a regular file instead the
* link text is resolved hop by hop (relative to each link's dir, bounded
* against loops) and the write lands at the final target, preserving the
* link the same way the live-symlink realpath branch does. */
function resolveWriteTarget(path: string): string {
if (existsSync(path)) {
// TOCTOU guard: the file can vanish between existsSync and realpathSync
// (a concurrent unlink), which would throw a raw ENOENT out of a writer
// that is perfectly able to proceed — fall through and treat the path as
// fresh/dangling instead.
try {
return realpathSync(path); // live file / live symlink chain
} catch {
/* raced away — resolve below */
}
}
let target = path;
for (let hops = 0; hops < 40; hops++) {
let st;
try {
st = lstatSync(target);
} catch {
return target; // truly absent — fresh-file target
}
if (!st.isSymbolicLink()) return target;
const linkText = readlinkSync(target);
target = isAbsolute(linkText) ? linkText : resolve(dirname(target), linkText);
}
return target; // pathological loop — bounded, last hop wins
}
export function atomicWriteTextFile(
path: string,
text: string,
opts?: { freshMode?: number; forceMode?: number },
): void {
const target = resolveWriteTarget(path);
mkdirSync(dirname(target), { recursive: true });
let mode: number | undefined;
if (opts?.forceMode !== undefined) {
mode = opts.forceMode;
} else {
try {
mode = statSync(target).mode & 0o777;
} catch {
mode = opts?.freshMode;
}
}
const tmp = `${target}.tmp-${randomBytes(6).toString('hex')}`;
// Failure hygiene: a throwing write/chmod/rename (ENOSPC, EACCES, target
// turned into a directory, …) must not leak the tmp file next to the
// user's config — unlink it best-effort and rethrow the original error.
try {
writeFileSync(tmp, text, { encoding: 'utf8', ...(mode !== undefined ? { mode } : {}) });
if (mode !== undefined) chmodSync(tmp, mode);
renameSync(tmp, target);
} catch (e) {
try {
unlinkSync(tmp);
} catch {
/* best-effort — the original error is the one that matters */
}
throw e;
}
}
+1 -1
View File
@@ -51,7 +51,7 @@ export interface AttachWorkspaceOptions {
/** The gbrain home receiving the install receipt (default: configDir()). */
gbrainHomeDir?: string;
/** Target harness for the hooks/MCP steps' descriptions. */
harness?: 'claude-code' | 'codex';
harness?: 'claude-code' | 'codex' | 'opencode';
/** Recorded as the receipt's created_by (the attaching binary's version). */
createdBy?: string;
}
+272
View File
@@ -0,0 +1,272 @@
/**
* codex-toml.ts managed marker-block writer for Codex's config.toml
* (#4043, the fired CX2-17 revisit; see TARGETS['codex-2026-08'] in
* host-specs.ts for the verified format assumptions).
*
* Why a direct writer exists at all: `codex mcp add` cannot express an
* inline `bearer_token` (only `--bearer-token-env-var`, which reintroduces
* the shell-profile dependency framework-spawned codex lacks), so the
* harness lane owns exactly one `[mcp_servers.<name>]` table between two
* full-line comment markers. Everything outside the markers survives
* byte-for-byte.
*
* Safety invariants [C10 + adversarial-verify corrections]:
* - Foreign-server detection parses the file (Bun.TOML.parse) with our block
* stripped a header-only regex false-negatives inline tables, dotted
* keys, and quoted headers, and a duplicate table definition is a HARD
* TOML parse error that bricks codex outright.
* - Rewrites REMOVE the old block and RE-ANCHOR at EOF (refusing on trailing
* content would let any later legitimate codex write permanently block
* update/removal; at EOF our table can never absorb someone else's keys).
* - The rendered output is parse-validated BEFORE rename, and our table's
* keys are asserted to be exactly what we wrote; on any failure the
* original file is kept.
* - Exactly one begin/end pair, in order, full-line match anything else is
* a hand-edit; refuse rather than guess.
* - Secrets hygiene: tmp file created 0600 with a random suffix, the target
* is chmod'd 0600 (it carries a bearer token), and the .bak is chmod'd
* 0600 (on re-runs it carries the PREVIOUS token).
* - CRLF configs are scanned with `\r` stripped and re-emitted with their
* dominant EOL; a missing trailing newline is repaired before append so a
* marker can never glue onto the last line (the run-2 duplicate-append
* brick).
*/
import { chmodSync, copyFileSync, existsSync, readFileSync, statSync } from 'node:fs';
import { atomicWriteTextFile } from './atomic-write.ts';
import { CODEX_TOML_BLOCK_BEGIN, CODEX_TOML_BLOCK_END } from './host-specs.ts';
export interface CodexHttpServerBlock {
/** MCP server name — bare-key charset only (goes into a table header). */
name: string;
/** Streamable-HTTP MCP endpoint url (normalized upstream). */
url: string;
/** Bearer token written inline (the whole point of the direct writer). */
bearerToken: string;
}
export interface WriteCodexBlockResult {
configPath: string;
/** True when a prior managed block was replaced (idempotent re-run). */
replacedPrior: boolean;
backupPath: string | null;
notes: string[];
}
export interface RemoveCodexBlockResult {
configPath: string;
removed: boolean;
backupPath: string | null;
notes: string[];
}
/** Escape a value for a TOML basic string. */
export function tomlString(value: string): string {
if (/[\n\r\0\t]/.test(value)) {
throw new Error('control characters are not allowed in codex config values');
}
return `"${value.replace(/\\/g, '\\\\').replace(/"/g, '\\"')}"`;
}
function assertBareKeyName(name: string): void {
if (!/^[A-Za-z0-9_-]+$/.test(name)) {
throw new Error(
`MCP server name "${name}" is not a bare TOML key ([A-Za-z0-9_-]+) — pick a simpler --name`,
);
}
}
interface BlockSpan {
/** Line index of the begin marker, or -1 when absent. */
begin: number;
/** Line index of the end marker, or -1 when absent. */
end: number;
}
/**
* Locate the managed block in \n-normalized lines. Throws on marker
* anomalies (duplicates, out of order, one without the other) those are
* hand-edits we must not guess through.
*/
function findBlock(lines: string[]): BlockSpan {
const begins: number[] = [];
const ends: number[] = [];
lines.forEach((line, i) => {
if (line === CODEX_TOML_BLOCK_BEGIN) begins.push(i);
if (line === CODEX_TOML_BLOCK_END) ends.push(i);
});
if (begins.length === 0 && ends.length === 0) return { begin: -1, end: -1 };
if (begins.length !== 1 || ends.length !== 1 || begins[0] > ends[0]) {
throw new Error(
`the gbrain-managed block markers in this config.toml are damaged ` +
`(${begins.length} begin / ${ends.length} end${begins.length === 1 && ends.length === 1 ? ', out of order' : ''}) — ` +
`the file was hand-edited inside the managed region. Fix the markers (or delete the whole block) and re-run.`,
);
}
return { begin: begins[0], end: ends[0] };
}
function stripBlock(lines: string[]): { remainder: string[]; hadBlock: boolean } {
const span = findBlock(lines);
if (span.begin === -1) return { remainder: lines, hadBlock: false };
const remainder = [...lines.slice(0, span.begin), ...lines.slice(span.end + 1)];
return { remainder, hadBlock: true };
}
function parseToml(text: string): Record<string, unknown> {
// Bun.TOML is available in this runtime (no dependency added).
const parsed = (Bun as unknown as { TOML: { parse(t: string): unknown } }).TOML.parse(text);
if (typeof parsed !== 'object' || parsed === null) {
throw new Error('config.toml did not parse to a table');
}
return parsed as Record<string, unknown>;
}
/**
* True when `[mcp_servers.<name>]` is defined OUTSIDE our managed block, in
* ANY toml spelling (header, inline table, dotted key, quoted header,
* array-of-tables). Throws when the surrounding config does not parse
* appending to a broken file could only deepen the damage.
*/
export function detectForeignCodexServer(configText: string, name: string): boolean {
const lines = configText.replace(/\r\n/g, '\n').split('\n');
const { remainder } = stripBlock(lines);
const parsed = parseToml(remainder.join('\n'));
const servers = parsed.mcp_servers;
if (typeof servers !== 'object' || servers === null) return false;
return (servers as Record<string, unknown>)[name] !== undefined;
}
function renderBlock(block: CodexHttpServerBlock): string[] {
return [
CODEX_TOML_BLOCK_BEGIN,
`[mcp_servers.${block.name}]`,
`url = ${tomlString(block.url)}`,
`bearer_token = ${tomlString(block.bearerToken)}`,
CODEX_TOML_BLOCK_END,
];
}
/** Atomic 0600 write preserving symlinks and the file's dominant EOL
* (forceMode: the file carries a bearer token regardless of prior mode). */
function atomicWriteToml(configPath: string, unixText: string, crlf: boolean): void {
const out = crlf ? unixText.replace(/\n/g, '\r\n') : unixText;
atomicWriteTextFile(configPath, out, { forceMode: 0o600 });
}
/**
* Idempotently write the managed `[mcp_servers.<name>]` block: strip any
* prior managed block, re-anchor the fresh one at EOF, validate the render,
* then swap atomically. The token lands inline, so the file is forced 0600.
*/
export function writeCodexHttpServerBlock(
configPath: string,
block: CodexHttpServerBlock,
): WriteCodexBlockResult {
assertBareKeyName(block.name);
const notes: string[] = [];
let rawText = '';
let existed = false;
if (existsSync(configPath)) {
existed = true;
rawText = readFileSync(configPath, 'utf8');
}
const crlf = rawText.includes('\r\n');
const lines = rawText.replace(/\r\n/g, '\n').split('\n');
const { remainder, hadBlock } = stripBlock(lines);
// Foreign ownership guard — a duplicate table definition is a hard TOML
// parse error that would brick codex entirely.
const remainderText = remainder.join('\n');
let parsedRemainder: Record<string, unknown>;
try {
parsedRemainder = parseToml(remainderText);
} catch (e) {
throw new Error(
`${configPath} does not parse as TOML outside the gbrain-managed block ` +
`(${(e as Error).message}) — fix the config (codex itself cannot read it either) and re-run.`,
);
}
const servers = parsedRemainder.mcp_servers;
if (typeof servers === 'object' && servers !== null && (servers as Record<string, unknown>)[block.name] !== undefined) {
throw new Error(
`[mcp_servers.${block.name}] is already defined in ${configPath} outside the gbrain-managed block — ` +
`refusing to double-define it (that is a hard TOML parse error). ` +
`Remove the existing entry (codex mcp remove ${block.name}) or pick another name (--name).`,
);
}
// Assemble: remainder (trailing newline repaired, trailing blank run
// collapsed to one separator line) + block at EOF.
const trimmed = [...remainder];
while (trimmed.length > 0 && trimmed[trimmed.length - 1].trim() === '') trimmed.pop();
const body = trimmed.length > 0 ? [...trimmed, ''] : [];
const nextLines = [...body, ...renderBlock(block), ''];
const nextText = nextLines.join('\n');
// Post-render validation: parse + assert OUR table's keys are exactly ours.
const rendered = parseToml(nextText);
const renderedServers = rendered.mcp_servers as Record<string, unknown> | undefined;
const ours = renderedServers?.[block.name];
const ourKeys = typeof ours === 'object' && ours !== null ? Object.keys(ours as object).sort() : [];
if (ourKeys.join(',') !== 'bearer_token,url') {
throw new Error(
`post-render validation failed: [mcp_servers.${block.name}] keys are [${ourKeys.join(', ')}], ` +
`expected exactly [bearer_token, url] — original file left untouched.`,
);
}
let backupPath: string | null = null;
if (existed) {
backupPath = `${configPath}.bak`;
copyFileSync(configPath, backupPath);
chmodSync(backupPath, 0o600); // on re-runs the .bak carries the previous token
const mode = statSync(configPath).mode & 0o777;
if ((mode & 0o077) !== 0) {
notes.push(
`${configPath} was group/other-readable (mode ${mode.toString(8)}); tightened to 0600 — it now carries a bearer token.`,
);
}
}
atomicWriteToml(configPath, nextText, crlf);
return { configPath, replacedPrior: hadBlock, backupPath, notes };
}
/**
* Remove the managed block (marker-keyed; content outside survives
* byte-for-byte). Absent file / absent block are calm no-ops. Damaged
* markers refuse via findBlock removal never guesses.
*/
export function removeCodexHttpServerBlock(
configPath: string,
name: string,
): RemoveCodexBlockResult {
assertBareKeyName(name);
const notes: string[] = [];
if (!existsSync(configPath)) {
return { configPath, removed: false, backupPath: null, notes: ['no config.toml — nothing to remove'] };
}
const rawText = readFileSync(configPath, 'utf8');
const crlf = rawText.includes('\r\n');
const lines = rawText.replace(/\r\n/g, '\n').split('\n');
const { remainder, hadBlock } = stripBlock(lines);
if (!hadBlock) {
return { configPath, removed: false, backupPath: null, notes: ['no gbrain-managed block — nothing to remove'] };
}
// Validate what we are about to write back — never leave codex unreadable.
const trimmed = [...remainder];
while (trimmed.length > 0 && trimmed[trimmed.length - 1].trim() === '') trimmed.pop();
const nextText = trimmed.length > 0 ? `${trimmed.join('\n')}\n` : '';
if (nextText !== '') parseToml(nextText);
const backupPath = `${configPath}.bak`;
copyFileSync(configPath, backupPath);
chmodSync(backupPath, 0o600);
atomicWriteToml(configPath, nextText, crlf);
return { configPath, removed: true, backupPath, notes };
}
+143 -3
View File
@@ -19,8 +19,8 @@
* manifest but not the receipt.
*/
import { existsSync, readFileSync, renameSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';
import { existsSync, mkdirSync, readFileSync, renameSync, rmSync, writeFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
export const FORMAT_VERSION = 1;
export const AGENT_MANIFEST_FILENAME = 'agent.json';
@@ -130,7 +130,7 @@ export interface InstallReceipt {
* corpus dir, ). Uninstall removes exactly these, nothing else. */
created_paths: string[];
/** Host registrations bootstrap performed (for marker-keyed removal). */
registrations: Array<{ host: 'claude-code' | 'codex'; scope: string; detail?: string }>;
registrations: Array<{ host: 'claude-code' | 'codex' | 'opencode'; scope: string; detail?: string }>;
}
export function receiptPath(gbrainHomeDir: string): string {
@@ -202,3 +202,143 @@ export function writeReceipt(gbrainHomeDir: string, receipt: InstallReceipt): vo
writeFileSync(tmp, `${JSON.stringify(receipt, null, 2)}\n`, 'utf8');
renameSync(tmp, path);
}
// ---------------------------------------------------------------------------
// Machine-level harness receipt (#4043 `gbrain bootstrap harness`)
//
// A SEPARATE file from receipt.json on purpose: the install receipt is
// workspace-keyed (workspace_dir/source_id/agent_name are required — a
// harness-only box has none of those), and the two lifecycles differ
// (workspace uninstall vs harness --remove). Same CX2-12 discipline: typed
// read states, newer-format refusal, broken-file backup-aside, atomic write.
//
// WRITE-AHEAD contract [F1/C7]: the apply path persists this receipt right
// after minting — every planned target starts `pending` and flips to
// `confirmed` (or records its failure) as the wiring lands, and
// `token.previous_ids` carries every not-yet-revoked prior token until rotation
// completes. A crash at any step leaves a receipt --remove can consume.
// ---------------------------------------------------------------------------
export type HarnessTargetKind = 'mcp' | 'permission' | 'hooks';
export type HarnessTargetState = 'pending' | 'confirmed' | 'failed';
export interface HarnessTarget {
host: 'claude-code' | 'codex' | 'opencode';
kind: HarnessTargetKind;
state: HarnessTargetState;
/** user scope or a --project dir (hooks); user for mcp/permission. */
scope: string;
/** Settings/config file the target writes (absent for CLI-mediated mcp). */
path?: string;
/** MCP server name (mcp targets). */
name?: string;
/** permissions.allow entry (permission targets). */
entry?: string;
/** Hook marker value (hooks targets). */
marker?: string;
/** Write mechanism note, e.g. 'toml-block' vs 'claude-cli'. */
mechanism?: string;
/** One-line failure reason when state === 'failed'. */
error?: string;
}
export interface HarnessReceipt {
harness_receipt_version: 1;
created_at: string;
created_by: string;
/** Normalized MCP endpoint the box is wired to. */
url: string;
/** Engine reported by /health at apply time (postgres → degraded per-turn). */
engine?: string;
/** Serve version at apply time — --status re-checks for skew [F7]. */
serve_version?: string;
source_id: string;
token: {
name: string;
/** Row id — the only safe revocation key. */
id?: string;
/** False when --token supplied a pre-minted token (never revoke those). */
minted: boolean;
/**
* EVERY prior minted-token id still awaiting revocation (mint-first
* rotation [C7]). An array, not a slot [X4]: a failed rotation must not
* forget the token before last, and a --token re-run must keep carrying
* ids minted by earlier runs until they are revoked.
*/
previous_ids?: string[];
};
targets: HarnessTarget[];
}
export function harnessReceiptPath(gbrainHomeDir: string): string {
return join(gbrainHomeDir, 'bootstrap', 'harness.json');
}
export type HarnessReceiptReadState =
| { state: 'absent' }
| { state: 'ok'; receipt: HarnessReceipt }
| { state: 'newer'; receiptVersion: number }
| { state: 'invalid' };
export function readHarnessReceiptState(gbrainHomeDir: string): HarnessReceiptReadState {
const path = harnessReceiptPath(gbrainHomeDir);
if (!existsSync(path)) return { state: 'absent' };
try {
const parsed = JSON.parse(readFileSync(path, 'utf8')) as HarnessReceipt;
if (parsed.harness_receipt_version === 1) {
// Version alone is not enough: every consumer dereferences targets[]
// and token.name unchecked, so a hand-damaged version-1 receipt must
// take the designed 'invalid' → backup-aside path, not a TypeError.
const shapeOk =
Array.isArray(parsed.targets) &&
typeof parsed.token === 'object' &&
parsed.token !== null &&
typeof parsed.token.name === 'string' &&
typeof parsed.url === 'string';
return shapeOk ? { state: 'ok', receipt: parsed } : { state: 'invalid' };
}
if (typeof parsed.harness_receipt_version === 'number' && parsed.harness_receipt_version > 1) {
return { state: 'newer', receiptVersion: parsed.harness_receipt_version };
}
return { state: 'invalid' };
} catch {
return { state: 'invalid' };
}
}
/**
* Pre-write guard (mirror of guardReceiptOverwrite): newer-format refuses
* with an upgrade-first error; an unreadable receipt is backed up loudly so
* the recorded targets remain recoverable.
*/
export function guardHarnessReceiptOverwrite(gbrainHomeDir: string): { brokenBackupPath?: string } {
const state = readHarnessReceiptState(gbrainHomeDir);
if (state.state === 'newer') {
throw new Error(
`the harness receipt at ${harnessReceiptPath(gbrainHomeDir)} was written by a newer gbrain ` +
`(harness_receipt_version ${state.receiptVersion}) — upgrade gbrain before re-running bootstrap harness.`,
);
}
if (state.state === 'invalid') {
const path = harnessReceiptPath(gbrainHomeDir);
const backup = `${path}.broken-${Date.now()}`;
renameSync(path, backup);
return { brokenBackupPath: backup };
}
return {};
}
/** Atomic write, 0600 (records wiring topology + token ids, not plaintext). */
export function writeHarnessReceipt(gbrainHomeDir: string, receipt: HarnessReceipt): void {
const path = harnessReceiptPath(gbrainHomeDir);
mkdirSync(dirname(path), { recursive: true });
const tmp = `${path}.tmp-${process.pid}`;
writeFileSync(tmp, `${JSON.stringify(receipt, null, 2)}\n`, { encoding: 'utf8', mode: 0o600 });
renameSync(tmp, path);
}
/** Consume (delete) the harness receipt after a fully successful --remove. */
export function deleteHarnessReceipt(gbrainHomeDir: string): void {
const path = harnessReceiptPath(gbrainHomeDir);
if (existsSync(path)) rmSync(path);
}
File diff suppressed because it is too large Load Diff
+313 -63
View File
@@ -24,15 +24,9 @@
* (and GBRAIN_HOME when isolated) ride the registration itself.
*/
import {
copyFileSync,
existsSync,
mkdirSync,
readFileSync,
renameSync,
writeFileSync,
} from 'node:fs';
import { dirname, isAbsolute, join } from 'node:path';
import { copyFileSync, existsSync, readFileSync } from 'node:fs';
import { isAbsolute, join } from 'node:path';
import { atomicWriteTextFile } from './atomic-write.ts';
import {
CLAUDE_COMMITTED_SETTINGS_FILE_RELPATH,
CLAUDE_HOOK_DEFAULT_TIMEOUT_SECS,
@@ -52,6 +46,12 @@ export interface ClaudeHookEnv {
GBRAIN_SOURCE: string;
/** Set only for --isolated installs (PARENT dir; config appends `.gbrain`). */
GBRAIN_HOME?: string;
/**
* 'harness' on #4043 harness-mode wiring: `gbrain hook` yields when the
* lane is harness AND the cwd carries a workspace bootstrap install, so
* the same event never fires twice (Claude Code merges settings scopes).
*/
GBRAIN_HOOK_LANE?: string;
}
export interface WriteClaudeHooksOpts {
@@ -60,8 +60,40 @@ export interface WriteClaudeHooksOpts {
env: ClaudeHookEnv;
/** Per-event timeout override (SECONDS — the settings-file unit). */
timeoutSecs?: Partial<Record<ClaudeHookEvent, number>>;
/** Subset of events to wire; default all four. */
/** Subset of events to wire; default every event in CLAUDE_HOOK_EVENTS. */
events?: ClaudeHookEvent[];
/**
* Marker VALUE stamped on (and stripped from) our entries. Default is the
* workspace-install marker; harness mode passes GBRAIN_HARNESS_MARKER_VALUE
* so the two installs coexist and each removal strips only its own.
*/
marker?: string;
/**
* [D12] Events already owned by the COMMITTED settings carrier the writer
* strips stale local copies of these but re-adds nothing for them, so one
* event never fires from both files. The workspace wrapper derives it from
* committedHookEvents(ws); path-parameterized callers pass their own.
*/
carriedEvents?: Set<ClaudeHookEvent>;
/**
* Backup strategy for the pre-write copy. 'fixed' (default) keeps the
* historical `.bak` sibling; 'timestamped' avoids the shared-slot problem
* when two writers touch the same file in one transaction.
*/
backupStrategy?: 'fixed' | 'timestamped';
/**
* Refuse when a gbrain entry with a DIFFERENT marker already wires one of
* our target events in this file (harness lane: a workspace install or a
* differently-scoped harness install owns it double-wiring would fire the
* same hook twice per event) [C6].
*/
refuseOnForeignGbrainMarker?: boolean;
/**
* Mode for a FRESHLY-CREATED settings file (existing files keep their mode
* via the atomic writer). Harness user-scope writes pass 0o600 to match
* Claude Code's own convention for that file [X11].
*/
freshMode?: number;
}
export interface WriteClaudeHooksResult {
@@ -125,6 +157,7 @@ export function buildClaudeHookCommand(
): string {
const assignments: string[] = [`GBRAIN_SOURCE=${env.GBRAIN_SOURCE}`];
if (env.GBRAIN_HOME) assignments.push(`GBRAIN_HOME=${env.GBRAIN_HOME}`);
if (env.GBRAIN_HOOK_LANE) assignments.push(`GBRAIN_HOOK_LANE=${env.GBRAIN_HOOK_LANE}`);
const parts = ['env', ...assignments, gbrainBin, 'hook', CLAUDE_HOOK_SUBCOMMAND[event]];
return parts.map(shellQuote).join(' ');
}
@@ -197,20 +230,30 @@ export function committedHookEvents(workspaceDir: string): Set<ClaudeHookEvent>
return carried;
}
function isOurs(entry: unknown): boolean {
function isOurs(entry: unknown, marker: string = GBRAIN_HOOK_MARKER_VALUE): boolean {
return (
typeof entry === 'object' &&
entry !== null &&
(entry as Record<string, unknown>)[GBRAIN_HOOK_MARKER_KEY] === GBRAIN_HOOK_MARKER_VALUE
(entry as Record<string, unknown>)[GBRAIN_HOOK_MARKER_KEY] === marker
);
}
/** True when the entry carries the gbrain marker KEY with any OTHER value. */
function isForeignGbrainMarked(entry: unknown, marker: string): boolean {
if (typeof entry !== 'object' || entry === null) return false;
const v = (entry as Record<string, unknown>)[GBRAIN_HOOK_MARKER_KEY];
return typeof v === 'string' && v !== marker;
}
/**
* Strip marker-carrying command entries from one event's matcher-group array.
* Groups EMPTIED by the removal are dropped; groups that were already empty
* (foreign) survive untouched. Returns the surviving groups + removal count.
* The group-drop rule is marker-independent: it fires only when THIS call's
* filter emptied a previously non-empty group, so it can never drop a group a
* different marker still owns.
*/
function stripOurEntries(groups: unknown[]): { kept: unknown[]; removed: number } {
function stripOurEntries(groups: unknown[], marker: string = GBRAIN_HOOK_MARKER_VALUE): { kept: unknown[]; removed: number } {
const kept: unknown[] = [];
let removed = 0;
for (const group of groups) {
@@ -220,7 +263,7 @@ function stripOurEntries(groups: unknown[]): { kept: unknown[]; removed: number
}
const g = group as HookMatcherGroup;
const before = g.hooks!.length;
const filtered = g.hooks!.filter((h) => !isOurs(h));
const filtered = g.hooks!.filter((h) => !isOurs(h, marker));
removed += before - filtered.length;
if (filtered.length === 0 && before > 0 && filtered.length !== before) {
continue; // we emptied it → drop the husk
@@ -234,12 +277,24 @@ function stripOurEntries(groups: unknown[]): { kept: unknown[]; removed: number
return { kept, removed };
}
/** Atomic write (tmp + rename), creating parent dirs. */
function atomicWriteJson(path: string, value: unknown): void {
mkdirSync(dirname(path), { recursive: true });
const tmp = `${path}.tmp-${process.pid}`;
writeFileSync(tmp, `${JSON.stringify(value, null, 2)}\n`, 'utf8');
renameSync(tmp, path);
/**
* Atomic write (tmp + rename), creating parent dirs. Hardened for shared
* user-scope targets [C10]: the SYMLINK TARGET is resolved first so a
* dotfile-manager-linked settings file survives as a link (a bare rename
* would replace the link with a regular file); the tmp file uses a random
* suffix and inherits the existing file's mode (a 0600 file stays 0600
* the pid-suffixed umask-default tmp was fine for gitignored workspace files
* but not for user-global config).
*/
function atomicWriteJson(path: string, value: unknown, freshMode?: number): void {
// Shared bootstrap atomic writer (symlink-resolving, mode-inheriting) —
// fresh files take the caller's convention (user-scope → 0600) [X11].
atomicWriteTextFile(path, `${JSON.stringify(value, null, 2)}\n`, { freshMode });
}
/** Pre-write backup path per strategy; timestamped avoids the shared-slot loss. */
function backupPathFor(settingsPath: string, strategy: 'fixed' | 'timestamped'): string {
return strategy === 'timestamped' ? `${settingsPath}.bak-${Date.now()}` : `${settingsPath}.bak`;
}
interface LoadedSettings {
@@ -291,13 +346,15 @@ function loadSettings(path: string): LoadedSettings {
// ── Writers [G5, CX2-17] ────────────────────────────────────────────────────
/**
* Structural-merge gbrain's hook entries into `<ws>/.claude/settings.local.json`.
* Idempotent: prior marker-carrying entries are removed before the fresh set
* is appended (run twice one entry per event). Foreign hooks, permissions,
* and every other key survive byte-for-byte at the structural level.
* Structural-merge gbrain's hook entries into an EXPLICIT settings file
* (workspace settings.local.json or user-scope ~/.claude/settings.json).
* Idempotent: prior same-marker entries are removed before the fresh set is
* appended (run twice one entry per event). Foreign hooks, other-marker
* gbrain entries, permissions, and every other key survive byte-for-byte at
* the structural level.
*/
export function writeClaudeHooks(
workspaceDir: string,
export function writeClaudeHooksAt(
settingsPath: string,
opts: WriteClaudeHooksOpts,
): WriteClaudeHooksResult {
if (!isAbsolute(opts.gbrainBin)) {
@@ -308,8 +365,9 @@ export function writeClaudeHooks(
throw new Error(`env ${k} contains control characters — refusing to embed in a hook command`);
}
}
const marker = opts.marker ?? GBRAIN_HOOK_MARKER_VALUE;
const backupStrategy = opts.backupStrategy ?? 'fixed';
const settingsPath = claudeSettingsPath(workspaceDir);
const { settings, existed, brokenBackupPath, notes } = loadSettings(settingsPath);
// hooks key: merge into an object; a structurally-foreign value is backed
@@ -326,14 +384,57 @@ export function writeClaudeHooks(
}
const events = opts.events ?? [...CLAUDE_HOOK_EVENTS];
// [C6] Same command double-fire guard: refuse when a gbrain entry carrying a
// DIFFERENT marker already wires one of our target events in this file —
// Claude Code would run both.
if (opts.refuseOnForeignGbrainMarker) {
for (const event of events) {
const groups = hooks[event];
if (!Array.isArray(groups)) continue;
for (const group of groups) {
const g = group as HookMatcherGroup;
if (!Array.isArray(g?.hooks)) continue;
for (const entry of g.hooks) {
if (isForeignGbrainMarked(entry, marker)) {
const foreign = (entry as Record<string, unknown>)[GBRAIN_HOOK_MARKER_KEY];
throw new Error(
`${settingsPath} already wires hooks.${event} under gbrain marker "${String(foreign)}" — ` +
`refusing to double-wire the same hook (both entries would fire every event). ` +
`Remove the other install first (gbrain bootstrap harness --remove, or gbrain bootstrap uninstall).`,
);
}
}
}
}
}
// [D12] Dedupe invariant: an event carried by the COMMITTED settings file
// never also fires from the local file. The local writer still strips its
// own prior entries for carried events (removing stale local copies), but
// re-adds nothing for them.
const carried = committedHookEvents(workspaceDir);
// never also fires from the local file. The caller supplies the carried set
// (the workspace wrapper reads it from committedHookEvents(ws); the harness
// --project lane does the same for its dirs) — this path-parameterized
// writer has no workspace to derive it from.
const carried = opts.carriedEvents ?? new Set<ClaudeHookEvent>();
let removedPrior = 0;
const installed: Array<{ event: ClaudeHookEvent; command: string }> = [];
// [X3] Convergence: strip OUR marker from EVERY event in the file first —
// not just the requested subset — so a re-run with fewer events (e.g.
// --no-capture dropping Stop/SessionEnd) removes the ones no longer wanted
// instead of leaving them live. Foreign and other-marker entries survive.
for (const event of Object.keys(hooks)) {
const groups = hooks[event];
if (!Array.isArray(groups)) continue; // structurally foreign — never touch
const { kept, removed } = stripOurEntries(groups, marker);
removedPrior += removed;
if (removed === 0) continue;
if (kept.length === 0) {
delete hooks[event]; // emptied by OUR removal — drop the key
} else {
hooks[event] = kept;
}
}
for (const event of events) {
let groups = hooks[event];
if (!Array.isArray(groups)) {
@@ -344,8 +445,7 @@ export function writeClaudeHooks(
}
groups = [];
}
const { kept, removed } = stripOurEntries(groups as unknown[]);
removedPrior += removed;
const kept = [...(groups as unknown[])];
if (carried.has(event)) {
notes.push(`${event}: carried by the committed .claude/settings.json — local entry skipped [D12]`);
@@ -360,7 +460,7 @@ export function writeClaudeHooks(
type: 'command',
command,
timeout,
[GBRAIN_HOOK_MARKER_KEY]: GBRAIN_HOOK_MARKER_VALUE,
[GBRAIN_HOOK_MARKER_KEY]: marker,
};
kept.push({ hooks: [entry] });
hooks[event] = kept;
@@ -370,15 +470,30 @@ export function writeClaudeHooks(
settings.hooks = hooks;
let backupPath: string | null = null;
if (existed) {
backupPath = `${settingsPath}.bak`;
if (existed && brokenBackupPath === null) {
backupPath = backupPathFor(settingsPath, backupStrategy);
copyFileSync(settingsPath, backupPath);
}
atomicWriteJson(settingsPath, settings);
atomicWriteJson(settingsPath, settings, opts.freshMode);
return { settingsPath, installed, removedPrior, backupPath, brokenBackupPath, notes };
}
/**
* Workspace-lane wrapper (historical signature: `<ws>/.claude/settings.local.json`,
* bootstrap-v1 marker, fixed `.bak`). Supplies the [D12] carried-events set so
* an event owned by the committed carrier never also fires locally.
*/
export function writeClaudeHooks(
workspaceDir: string,
opts: WriteClaudeHooksOpts,
): WriteClaudeHooksResult {
return writeClaudeHooksAt(claudeSettingsPath(workspaceDir), {
carriedEvents: committedHookEvents(workspaceDir),
...opts,
});
}
/**
* Write hooks into the COMMITTED `.claude/settings.json` [D12] the only
* carrier that survives into fresh cloud clones (hooks are snapshotted at
@@ -449,7 +564,7 @@ export function writeCommittedClaudeHooks(
// [D12] dedupe: the committed carrier now owns these events — remove any
// local copies so nothing double-fires on this machine.
const localCleanup = removeHooksFromFile(claudeSettingsPath(workspaceDir));
const localCleanup = removeClaudeHooksAt(claudeSettingsPath(workspaceDir));
if (localCleanup.removed > 0) {
notes.push(
`removed ${localCleanup.removed} local settings.local.json entr${localCleanup.removed === 1 ? 'y' : 'ies'} — the committed carrier owns the events now [D12]`,
@@ -460,31 +575,16 @@ export function writeCommittedClaudeHooks(
}
/**
* Remove ONLY marker-carrying entries [G5]. A parse-broken file is left
* untouched (removal must never destroy what it cannot read) the note says
* so. Event arrays we emptied lose their key; an emptied hooks object loses
* its key; foreign structure survives. Cleans BOTH carriers (local +
* committed [D12]); the returned settingsPath/backup describe the local one,
* with committed-file actions reported via notes.
* Remove ONLY entries carrying the given marker [G5]. A parse-broken file is
* left untouched (removal must never destroy what it cannot read) the note
* says so. Event arrays we emptied lose their key; an emptied hooks object
* loses its key; foreign structure (including other-marker gbrain entries)
* survives.
*/
export function removeClaudeHooks(workspaceDir: string): RemoveClaudeHooksResult {
const local = removeHooksFromFile(claudeSettingsPath(workspaceDir));
const committed = removeHooksFromFile(claudeCommittedSettingsPath(workspaceDir));
const notes = [...local.notes];
if (committed.removed > 0) {
notes.push(`also removed ${committed.removed} entr${committed.removed === 1 ? 'y' : 'ies'} from the committed ${committed.settingsPath} [D12]`);
} else {
notes.push(...committed.notes.map((n) => `(committed carrier) ${n}`));
}
return {
settingsPath: local.settingsPath,
removed: local.removed + committed.removed,
backupPath: local.backupPath,
notes,
};
}
function removeHooksFromFile(settingsPath: string): RemoveClaudeHooksResult {
export function removeClaudeHooksAt(
settingsPath: string,
marker: string = GBRAIN_HOOK_MARKER_VALUE,
): RemoveClaudeHooksResult {
const notes: string[] = [];
if (!existsSync(settingsPath)) {
return { settingsPath, removed: 0, backupPath: null, notes: ['no settings file — nothing to remove'] };
@@ -521,7 +621,7 @@ function removeHooksFromFile(settingsPath: string): RemoveClaudeHooksResult {
for (const event of Object.keys(hooks)) {
const groups = hooks[event];
if (!Array.isArray(groups)) continue; // structurally foreign — never touch
const { kept, removed: n } = stripOurEntries(groups);
const { kept, removed: n } = stripOurEntries(groups, marker);
removed += n;
if (n === 0) continue;
if (kept.length === 0) {
@@ -543,6 +643,156 @@ function removeHooksFromFile(settingsPath: string): RemoveClaudeHooksResult {
return { settingsPath, removed, backupPath, notes };
}
/**
* Workspace-lane removal cleans BOTH carriers (local + committed [D12]);
* the returned settingsPath/backup describe the local one, with
* committed-file actions reported via notes.
*/
export function removeClaudeHooks(workspaceDir: string): RemoveClaudeHooksResult {
const local = removeClaudeHooksAt(claudeSettingsPath(workspaceDir));
const committed = removeClaudeHooksAt(claudeCommittedSettingsPath(workspaceDir));
const notes = [...local.notes];
if (committed.removed > 0) {
notes.push(`also removed ${committed.removed} entr${committed.removed === 1 ? 'y' : 'ies'} from the committed ${committed.settingsPath} [D12]`);
} else {
notes.push(...committed.notes.map((n) => `(committed carrier) ${n}`));
}
return {
settingsPath: local.settingsPath,
removed: local.removed + committed.removed,
backupPath: local.backupPath,
notes,
};
}
// ── permissions.allow writers (harness lane, #4043) ────────────────────────
export interface PermissionsAllowResult {
settingsPath: string;
/** add: entry appended this run. remove: number of occurrences removed. */
added?: boolean;
removed?: number;
backupPath: string | null;
notes: string[];
}
/**
* Append one entry to `permissions.allow` (set semantics present means
* no-op). Stamps NO marker: permissions.allow is an array of plain strings,
* so ownership is recorded on the harness receipt (exact-string removal),
* never in the file and a marker object here would false-positive
* status.ts's whole-file `hooksInstalled` substring probe. Foreign entries
* and every other settings key survive. Broken JSON aborts (user-scope
* discipline this writer only ever targets user-scope files).
*/
export function addPermissionsAllowEntry(
settingsPath: string,
entry: string,
): PermissionsAllowResult {
const { settings, existed, notes } = loadSettings(settingsPath);
let permissions = settings.permissions as Record<string, unknown> | undefined;
if (typeof permissions !== 'object' || permissions === null || Array.isArray(permissions)) {
if (permissions !== undefined) {
// Fail CLOSED (same stance as broken JSON): "permissions" is the host's
// security policy — replacing a shape we don't understand could erase
// deny/ask rules or a future settings schema (ship-review P2).
throw new Error(
`${settingsPath}: existing "permissions" key is not an object ` +
`(${JSON.stringify(permissions).slice(0, 80)}) — refusing to rewrite security policy this writer ` +
`does not understand. Fix the file by hand, then re-run.`,
);
}
permissions = {};
}
let allow = permissions.allow as unknown[] | undefined;
if (!Array.isArray(allow)) {
if (allow !== undefined) {
throw new Error(
`${settingsPath}: existing permissions.allow is not an array — refusing to rewrite security policy ` +
`this writer does not understand. Fix the file by hand, then re-run.`,
);
}
allow = [];
}
if (allow.some((e) => e === entry)) {
return { settingsPath, added: false, backupPath: null, notes: [...notes, `${entry} already allowed — no change`] };
}
allow.push(entry);
permissions.allow = allow;
settings.permissions = permissions;
let backupPath: string | null = null;
if (existed) {
backupPath = backupPathFor(settingsPath, 'timestamped');
copyFileSync(settingsPath, backupPath);
}
atomicWriteJson(settingsPath, settings, 0o600); // user-scope-only writer [X11]
return { settingsPath, added: true, backupPath, notes };
}
/**
* Remove EXACTLY the given string from `permissions.allow`. Parse-broken file
* is left untouched (the removeClaudeHooks precedent removal never destroys
* what it cannot read). Keys are dropped only when OUR removal emptied them.
* Honest edge (stated in consent copy): if the user had independently allowed
* the same string, this removes it too set semantics carry no provenance.
*/
export function removePermissionsAllowEntry(
settingsPath: string,
entry: string,
): PermissionsAllowResult {
if (!existsSync(settingsPath)) {
return { settingsPath, removed: 0, backupPath: null, notes: ['no settings file — nothing to remove'] };
}
let loaded: LoadedSettings;
try {
loaded = loadSettings(settingsPath);
} catch (e) {
return {
settingsPath,
removed: 0,
backupPath: null,
notes: [
`WARNING: ${settingsPath} is not valid JSON (${(e as Error).message}); ` +
`left untouched — remove the "${entry}" permissions.allow entry by hand or fix the JSON and re-run.`,
],
};
}
const { settings, notes } = loaded;
const permissions = settings.permissions as Record<string, unknown> | undefined;
if (typeof permissions !== 'object' || permissions === null || Array.isArray(permissions)) {
return { settingsPath, removed: 0, backupPath: null, notes: ['no permissions object — nothing to remove'] };
}
const allow = permissions.allow as unknown[] | undefined;
if (!Array.isArray(allow)) {
return { settingsPath, removed: 0, backupPath: null, notes: ['no permissions.allow array — nothing to remove'] };
}
const kept = allow.filter((e) => e !== entry);
const removed = allow.length - kept.length;
if (removed === 0) {
return { settingsPath, removed: 0, backupPath: null, notes: [...notes, `${entry} not present — nothing to remove`] };
}
if (kept.length > 0) {
permissions.allow = kept;
} else {
delete permissions.allow; // emptied by OUR removal — drop the key
}
if (Object.keys(permissions).length === 0) {
delete settings.permissions;
}
const backupPath = backupPathFor(settingsPath, 'timestamped');
copyFileSync(settingsPath, backupPath);
atomicWriteJson(settingsPath, settings, 0o600); // user-scope-only writer [X11]
return { settingsPath, removed, backupPath, notes };
}
// ── MCP registration argv builders [G1, CX-P1.4, ENG-7] ────────────────────
export interface ClaudeMcpRegistration {
+187 -14
View File
@@ -19,8 +19,9 @@
* gbrain code.
*/
import { existsSync } from 'node:fs';
import { homedir } from 'node:os';
import { join } from 'node:path';
import { basename, dirname, join } from 'node:path';
// ── Spec-target registry [ENG-7] ────────────────────────────────────────────
@@ -38,6 +39,7 @@ export interface HostSpecTarget {
export const CLAUDE_CODE_SPEC_ID = 'claude-code-2026-08';
export const CODEX_SPEC_ID = 'codex-2026-08';
export const OPENCODE_SPEC_ID = 'opencode-2026-08';
export const TARGETS: Record<string, HostSpecTarget> = {
[CLAUDE_CODE_SPEC_ID]: {
@@ -76,21 +78,69 @@ export const TARGETS: Record<string, HostSpecTarget> = {
},
[CODEX_SPEC_ID]: {
id: CODEX_SPEC_ID,
status: 'provisional',
verifiedAt: '2026-08-08',
status: 'verified',
verifiedAt: '2026-08-12',
references: [
'docs/mcp/CODEX.md',
'https://developers.openai.com/codex/mcp',
'codex-cli 0.147.0 (binary serde field scan + live inline bearer_token wiring, issue #4043)',
],
note:
'Codex has NO hook system — the pull-protocol AGENTS.md gates are the ' +
'per-turn seam (plan D5). Local stdio MCP registration: ' +
'`codex mcp add <name> [--env K=V]... -- <command> [args...]`, which ' +
'writes [mcp_servers.<name>] into ~/.codex/config.toml. A TOML-aware ' +
'config.toml writer [CX2-17] is deliberately NOT needed in v1: ' +
'`codex mcp add` owns the config.toml write end-to-end, so gbrain never ' +
'edits the file directly. Revisit only if a v1.1 feature (notify ' +
'sweeper, FF2) must write keys `codex mcp add` cannot express.',
'Local stdio MCP registration: `codex mcp add <name> [--env K=V]... -- ' +
'<command> [args...]`, which writes [mcp_servers.<name>] into ' +
'(CODEX_HOME || ~/.codex)/config.toml — codex resolves CODEX_HOME as ' +
'the config dir itself. Streamable-HTTP servers are configured with ' +
'`url` plus `bearer_token` (inline) or `bearer_token_env_var`; the ' +
'config parser uses deny-unknown-fields, so writers must emit ONLY ' +
'verified keys and `KEY = "value"` spacing. The CX2-17 revisit trigger ' +
'FIRED (#4043): `codex mcp add` cannot express an inline bearer_token ' +
'(verified against codex-cli 0.147.0 --help), so the harness lane owns ' +
'a managed marker-delimited TOML block (codex-toml.ts) — the ONE ' +
'direct config.toml writer. One owner per server name: `codex mcp ' +
'remove` rewrites config.toml wholesale and drops comments, so the ' +
'stdio lane (runHooks) must never manage a name the harness block ' +
'owns, and vice versa. Codex 0.147.0 also ships a real hook system ' +
'(hooks.json; PreToolUse…SessionEnd) — CODEX_HAS_HOOKS=false means ' +
'"gbrain does not wire codex hooks yet" (follow-up filed), NOT "codex ' +
'has no hooks". Some codex builds gate HTTP MCP servers behind ' +
'`experimental_use_rmcp_client = true` — probe at wiring time.',
},
[OPENCODE_SPEC_ID]: {
id: OPENCODE_SPEC_ID,
status: 'verified',
verifiedAt: '2026-08-15',
references: [
'docs/mcp/OPENCODE-CLI-PIN.md',
'https://opencode.ai/docs/mcp-servers/',
'opencode-ai 1.18.18 (hermetic observation run, macOS arm64, 2026-08-15)',
],
note:
'opencode (SST, opencode.ai — not OpenClaw). Config is JSONC everywhere: ' +
'comments parse in .json-named files, and global opencode.json AND ' +
'opencode.jsonc are BOTH read (merged) when both exist; `opencode mcp ' +
'add` writes the user-global opencode.jsonc via a comment-preserving ' +
'editor, so gbrain writes match that bar (jsonc-parser surgical edits, ' +
'opencode-json.ts). MCP entries: {type:"local", command[], environment, ' +
'enabled?} / {type:"remote", url, headers} — header values keep ' +
'`{env:VAR}` interpolation verbatim; unknown keys tolerated in 1.18.18 ' +
'but gbrain writes NO marker key (ownership is a structural ' +
'fingerprint — a future strict-schema flip must not brick the host). ' +
'`mcp add` has no scope flag (always user-global); project opencode.json ' +
'is read but a project-defined LOCAL server spawns with NO trust gate ' +
'(verified) — so gbrain defaults registration to USER scope and treats ' +
'project scope as explicit opt-in with a sharing warning. `mcp list` is ' +
'the honest discriminator (spawns servers; ✓/✗ text; exit 0 regardless); ' +
'`mcp debug` is OAuth-only. Keyless anonymous free tier answers headless ' +
'`run` AND drives MCP tool calls without --auto (load-bearing for the ' +
'door SMOKE). DOCS-CONTRADICTION pinned: OPENCODE_CONFIG / _CONFIG_DIR / ' +
'_CONFIG_CONTENT observed INERT in 1.18.18 — only HOME/XDG_CONFIG_HOME ' +
'move the config; path helpers resolve via XDG only. opencode sets ' +
'OPENCODE=1 (+OPENCODE_PID) in bash-tool children — detectHarness ' +
'probes OPENCODE. AGENTS.md loads natively; CLAUDE.md is NOT ' +
'double-loaded. opencode ships a JS plugin/event system — ' +
'OPENCODE_HAS_HOOKS=false means "gbrain does not wire it yet" (follow-up ' +
'filed), NOT "opencode has no hooks"; probes run with --pure + ' +
'OPENCODE_DISABLE_AUTOUPDATE=1 because mcp list autoloads plugins.',
},
};
@@ -163,6 +213,36 @@ export const CLAUDE_HOOK_DEFAULT_TIMEOUT_SECS: Record<ClaudeHookEvent, number> =
export const GBRAIN_HOOK_MARKER_KEY = '_gbrain';
export const GBRAIN_HOOK_MARKER_VALUE = 'bootstrap-v1';
/**
* Marker VALUE for harness-mode installs (`gbrain bootstrap harness`, #4043).
* Same `_gbrain` key, distinct value: a box can carry a workspace bootstrap
* install (bootstrap-v1 in settings.local.json) AND a harness install
* (bootstrap-harness-v1 in user settings.json or a project settings.local.json)
* each removal path strips only its own entries.
*/
export const GBRAIN_HARNESS_MARKER_VALUE = 'bootstrap-harness-v1';
/**
* User-scope Claude Code settings file (harness-mode hook + permissions
* target). Resolution mirrors Claude Code itself: CLAUDE_CONFIG_DIR (its
* documented config-dir override, which the real-claude e2e harness sets)
* else $HOME/.claude checked explicitly because Bun's homedir() reads the
* password database, NOT the HOME env var, so a sandboxed test that remaps
* HOME would otherwise write into the operator's REAL settings file (this
* bit us; the write-ahead receipt's remove path self-healed it).
*/
export function claudeUserSettingsPath(): string {
const configDir = process.env.CLAUDE_CONFIG_DIR?.trim();
if (configDir) return join(configDir, 'settings.json');
const home = process.env.HOME?.trim();
return join(home || homedir(), '.claude', 'settings.json');
}
/** permissions.allow entry that pre-approves an MCP server's tools for headless runs. */
export function mcpPermissionEntry(serverName: string): string {
return `mcp__${serverName}`;
}
/** Where Claude Code stores session transcripts — the confinement root [S3#8]. */
export function claudeProjectsDir(): string {
return join(homedir(), '.claude', 'projects');
@@ -170,10 +250,103 @@ export function claudeProjectsDir(): string {
// ── Codex shapes ────────────────────────────────────────────────────────────
/** Codex CLI config file (user-global; written by `codex mcp add`, never by us). */
/**
* Codex CLI config file (user-global). Real codex resolves CODEX_HOME as the
* config DIRECTORY itself (config.toml sits directly inside it) pinned by
* test/e2e/bootstrap-real-codex.serial.test.ts and connect-bearer.test.ts,
* which set CODEX_HOME to a temp dir and assert the real binary wrote there.
* A homedir-only resolution here would make any CODEX_HOME-isolated test
* clobber the operator's real ~/.codex/config.toml.
*/
export function codexConfigPath(): string {
return join(homedir(), '.codex', 'config.toml');
const codexHome = process.env.CODEX_HOME?.trim();
return join(codexHome || join(homedir(), '.codex'), 'config.toml');
}
/** Codex has no hook system — per-turn context is pull-protocol (plan D5). */
/**
* Whether gbrain WIRES codex hooks. False = not yet: codex 0.147.0 ships a
* real hook system (hooks.json; PreToolUseSessionEnd see the TARGETS
* note), but gbrain's codex hook lane is a filed follow-up; per-turn context
* on codex remains the pull-protocol AGENTS.md gates (plan D5).
*/
export const CODEX_HAS_HOOKS = false;
/**
* Managed-block markers for the harness lane's direct config.toml writes
* (codex-toml.ts the CX2-17 revisit, fired by #4043). Full-line exact
* match at column 0; the writer requires exactly one begin/end pair.
*/
export const CODEX_TOML_BLOCK_BEGIN =
`# gbrain:${GBRAIN_HARNESS_MARKER_VALUE} begin - managed by \`gbrain bootstrap harness\`; do not edit inside`;
export const CODEX_TOML_BLOCK_END = `# gbrain:${GBRAIN_HARNESS_MARKER_VALUE} end`;
// ── opencode shapes ─────────────────────────────────────────────────────────
/**
* opencode config DIRECTORY (user-global). Resolution mirrors what the real
* binary was OBSERVED to do (OPENCODE-CLI-PIN.md §Path seams): XDG_CONFIG_HOME
* else $HOME/.config, then /opencode. The OPENCODE_CONFIG / OPENCODE_CONFIG_DIR
* / OPENCODE_CONFIG_CONTENT env vars are deliberately NOT honored here
* observed INERT in opencode 1.18.18 (probes registered through each were
* invisible to `mcp list` while the XDG-resolved config was still read), so
* honoring them would write registrations into a file opencode never reads (a
* silent no-op install). HOME is read from the env explicitly because Bun's
* homedir() reads the password database, not the HOME env var (the
* claudeUserSettingsPath lesson).
*/
export function opencodeConfigDir(): string {
const xdg = process.env.XDG_CONFIG_HOME?.trim();
if (xdg) return join(xdg, 'opencode');
const home = process.env.HOME?.trim();
return join(home || homedir(), '.config', 'opencode');
}
/**
* User-global opencode config FILE. Both `opencode.json` and `opencode.jsonc`
* are read (merged) by the host when both exist; gbrain edits the file that
* already carries content, preferring `.jsonc` (the name `opencode mcp add`
* itself writes) when both or neither exist one-owner-per-file keeps the
* merge unambiguous for `mcp.gbrain`.
*/
export function opencodeGlobalConfigPath(): string {
const dir = opencodeConfigDir();
const jsonc = join(dir, 'opencode.jsonc');
const json = join(dir, 'opencode.json');
if (existsSync(jsonc)) return jsonc;
if (existsSync(json)) return json;
return jsonc;
}
/**
* The OTHER member of the global filename pair for a given config path
* (`opencode.json` `opencode.jsonc` in the same dir), or null when the
* basename is not a pair member. opencode MERGES both files when both exist,
* so global WRITERS must reconcile `mcp.<name>` across the pair a same-name
* entry left in the sibling survives as a shadow registration whose merge
* winner is ambiguous (and a later removal of the primary "reveals" it).
* Callers apply this to the USER-GLOBAL pair only; project-scope sibling
* semantics are unobserved.
*/
export function opencodeGlobalSiblingPath(configPath: string): string | null {
const dir = dirname(configPath);
const base = basename(configPath);
if (base === 'opencode.json') return join(dir, 'opencode.jsonc');
if (base === 'opencode.jsonc') return join(dir, 'opencode.json');
return null;
}
/** Project-scope opencode config (docs-canonical name; opencode's lookup
* traverses up to the git root). Committed-file candidate the writer's
* PATH-resolved command + sharing-warning rules apply (OPENCODE.md). */
export function opencodeProjectConfigPath(workspaceDir: string): string {
return join(workspaceDir, 'opencode.json');
}
/**
* Whether gbrain WIRES opencode's hook/plugin system. False = not yet:
* opencode ships a JS plugin/event system (and `--pure` to suppress it), but
* gbrain's opencode plugin lane is a filed follow-up; per-turn context on
* opencode rides the pull-protocol AGENTS.md gates, which opencode loads
* natively (verified and CLAUDE.md is NOT double-loaded alongside it).
*/
export const OPENCODE_HAS_HOOKS = false;
+572
View File
@@ -0,0 +1,572 @@
/**
* opencode-json.ts managed `mcp.<name>` entry writer for opencode's JSONC
* configs (see TARGETS['opencode-2026-08'] in host-specs.ts and
* docs/mcp/OPENCODE-CLI-PIN.md for the verified format assumptions).
*
* Why a direct writer exists: `opencode mcp add` always targets the
* user-global opencode.jsonc (no scope flag), cannot set file modes (the
* harness lane's inline bearer needs 0600), and requires the binary on the
* box the writer covers project scope, secret hygiene, and offline/
* pre-install registration with one code path.
*
* Safety invariants (codex-toml.ts analog, adapted for JSONC):
* - ALL edits go through jsonc-parser `modify`/`applyEdits` text splicing
* that preserves comments, formatting, and EOLs byte-for-byte outside the
* edited range. opencode's own `mcp add` preserves comments (observed);
* gbrain matches that bar. JSON.parse is never used on config text.
* - Ownership is a STRUCTURAL FINGERPRINT, not a marker key (unknown keys
* are tolerated by opencode 1.18.18, but a future strict-schema flip must
* not brick the user's opencode): a local entry is ours when command[0] is
* gbrain-shaped AND environment.GBRAIN_SOURCE exists; source EQUALITY
* (not mere presence) splits `ours-same-source` from `ours-other-source`
* ([FIX7] parity with verifyMcpTargetsWorkspace) callers warn before
* overwriting another workspace's registration. A remote entry is ours
* when its url matches the caller's receipt, or when its Authorization
* header carries the `{env:GBRAIN_REMOTE_TOKEN}` interpolation (only the
* connect lane writes that). Anything else under our name is FOREIGN
* refuse, never guess.
* - Read-failure classes are distinct: ENOENT fresh file; empty/whitespace
* treated as `{}`; unreadable (EACCES etc.) refuse loudly (never
* clobber what cannot be read). A file that fails even JSONC parsing
* refuse with a paste-by-hand snippet.
* - Post-render validation before rename: the rendered text is re-parsed,
* our entry deep-asserted, and every OTHER top-level key asserted to
* survive; on any failure the original file is untouched.
* - Secrets hygiene: when the entry carries an inline bearer the target is
* forced 0600. Backups are UNIQUE per operation (`<config>.bak-<hex>`,
* returned in the result) so two overlapping runs can never clobber each
* other's snapshot, and a backup is chmod'd 0600 whenever the COPIED
* content carries an inline bearer (write AND remove paths on re-runs
* the backup carries the PREVIOUS token). Token-free entries inherit the
* file's existing mode.
* - Concurrency: callers hold acquireBootstrapLock (config-dir
* opencode-dir ordering, mirroring the codex lanes in harness.ts) the
* writer itself is lock-free like codex-toml.ts.
*/
import { randomBytes } from 'node:crypto';
import { chmodSync, copyFileSync, existsSync, readFileSync } from 'node:fs';
import { applyEdits, modify, parse as parseJsonc, printParseErrorCode, type ParseError } from 'jsonc-parser';
import { atomicWriteTextFile } from './atomic-write.ts';
import { opencodeGlobalSiblingPath } from './host-specs.ts';
export const GBRAIN_REMOTE_TOKEN_ENV = 'GBRAIN_REMOTE_TOKEN';
const ENV_INTERPOLATION = `{env:${GBRAIN_REMOTE_TOKEN_ENV}}`;
// ── Entry shapes ────────────────────────────────────────────────────────────
export interface OpencodeLocalEntry {
kind: 'local';
name: string;
/** argv command[0] is PATH-resolved "gbrain" (project scope, committed-
* file candidate) or an absolute binary path (user scope). */
command: string[];
environment: Record<string, string>;
}
export interface OpencodeRemoteEntry {
kind: 'remote';
name: string;
url: string;
/** 'inline' writes `Bearer <token>` (harness lane framework-spawned
* opencode inherits no shell profile; file forced 0600). 'env' writes the
* `{env:GBRAIN_REMOTE_TOKEN}` interpolation (connect lane token never
* enters the file). */
tokenMode: 'inline' | 'env';
bearerToken?: string;
}
export type OpencodeMcpEntry = OpencodeLocalEntry | OpencodeRemoteEntry;
export type OpencodeEntryKind =
| 'absent'
| 'ours-same-source'
| 'ours-other-source'
| 'foreign';
export interface OpencodeEntryExpectation {
/** GBRAIN_SOURCE the caller is registering (local entries). */
sourceId?: string;
/** Serve url from the caller's receipt (remote entries). */
url?: string;
}
export interface WriteOpencodeEntryResult {
configPath: string;
/** True when a prior gbrain-owned entry was replaced (idempotent re-run). */
replacedPrior: boolean;
/** Kind of the pre-existing entry (what was there before this write). */
priorKind: OpencodeEntryKind;
/** Unique per-write backup (`<config>.bak-<hex>`) of the prior file, or
* null on a fresh file. Callers that roll back restore from THIS path. */
backupPath: string | null;
/** The EXACT text this write landed rollback callers compare the current
* file content against it before restoring (a mismatch means a newer
* registration exists and a restore would clobber it). */
writtenText: string;
notes: string[];
}
export interface RemoveOpencodeEntryResult {
configPath: string;
removed: boolean;
backupPath: string | null;
notes: string[];
}
// ── Read + parse (failure classes are distinct) ─────────────────────────────
interface RawConfig {
text: string;
existed: boolean;
}
function readConfigRaw(configPath: string): RawConfig {
if (!existsSync(configPath)) return { text: '', existed: false };
let text: string;
try {
text = readFileSync(configPath, 'utf8');
} catch (e) {
throw new Error(
`${configPath} exists but cannot be read (${(e as Error).message}) — ` +
`refusing to touch a config that cannot be read back. Fix permissions and re-run.`,
);
}
return { text, existed: true };
}
/**
* Parse config text as JSONC (opencode's effective grammar for BOTH .json
* and .jsonc files OPENCODE-CLI-PIN.md §Config format). Empty/whitespace
* text parses as `{}`. Text that fails even JSONC parsing throws with a
* paste-by-hand snippet so the user is never stranded.
*/
export function parseOpencodeConfig(text: string, configPath: string, snippet?: string): Record<string, unknown> {
if (text.trim() === '') return {};
const errors: ParseError[] = [];
const parsed = parseJsonc(text, errors, { allowTrailingComma: true }) as unknown;
if (errors.length > 0) {
const first = errors[0];
throw new Error(
`${configPath} does not parse as JSONC (${printParseErrorCode(first.error)} at offset ${first.offset}) — ` +
`opencode itself cannot read it either. Fix the file, or add the entry by hand:\n${snippet ?? ''}`,
);
}
if (typeof parsed !== 'object' || parsed === null || Array.isArray(parsed)) {
throw new Error(`${configPath} is valid JSONC but not an object — fix the file and re-run.`);
}
return parsed as Record<string, unknown>;
}
// ── Ownership fingerprint ───────────────────────────────────────────────────
function isGbrainShapedCommand(command: unknown): boolean {
if (!Array.isArray(command) || command.length === 0) return false;
const head = command[0];
if (typeof head !== 'string') return false;
if (head === 'gbrain') return true; // PATH-resolved (project scope)
if (/[\\/]gbrain$/.test(head)) return true; // absolute binary path
// bun-run wrapper shim lane: `bun run <...>/gbrain/src/cli.ts` etc. The
// arg match is ANCHORED like the head-path lane: some arg must carry an
// exact `gbrain` path segment (or a hyphen-suffixed `gbrain-*` one) — a
// loose substring scan classified `bun run /opt/gbrainy-fork/src/cli.ts`
// as ours (both the `gbrain` substring and a bare `src/cli.ts$` matched).
// A gbrain-less `bun run /repo/src/cli.ts` is now NOT ours (fail-closed:
// gbrain refuses to touch what it cannot prove it owns).
if (head === 'bun' || head.endsWith('/bun')) {
return command.some((a) => typeof a === 'string' && /(?:^|[\\/])gbrain(?:[\\/-]|$)/.test(a));
}
// staged shim named gbrain-<suffix> (e.g. gbrain-shim from stageBinDir) —
// hyphen-anchored so a foreign /opt/bin/gbrainy is NOT ours.
return /[\\/]gbrain-[^\\/]*$/.test(head);
}
/**
* Classify the `mcp.<name>` entry in parsed config. The arbiter every lane
* consults before writing or removing (codexBlockOwnsName analog).
*/
export function opencodeEntryKind(
parsed: Record<string, unknown>,
name: string,
expect: OpencodeEntryExpectation = {},
): OpencodeEntryKind {
const mcp = parsed.mcp;
if (typeof mcp !== 'object' || mcp === null) return 'absent';
const entry = (mcp as Record<string, unknown>)[name];
if (entry === undefined) return 'absent';
if (typeof entry !== 'object' || entry === null) return 'foreign';
const e = entry as Record<string, unknown>;
if (e.type === 'local') {
if (!isGbrainShapedCommand(e.command)) return 'foreign';
const env = e.environment;
const src =
typeof env === 'object' && env !== null
? (env as Record<string, unknown>).GBRAIN_SOURCE
: undefined;
if (typeof src !== 'string' || src === '') return 'foreign';
// Kind mismatch (red-team): a caller expecting a REMOTE entry (harness /
// connect lanes pass expect.url) that finds a LOCAL gbrain entry is
// looking at ANOTHER lane's registration (the workspace stdio lane's) —
// never `ours-same-source`, or a silent replace (and a later --remove)
// would eat it. `ours-other-source` fires the refuse/confirm machinery.
if (expect.url !== undefined) return 'ours-other-source';
if (expect.sourceId === undefined) return 'ours-same-source';
return src === expect.sourceId ? 'ours-same-source' : 'ours-other-source';
}
if (e.type === 'remote') {
if (expect.url !== undefined && e.url === expect.url) return 'ours-same-source';
const headers = e.headers;
const auth =
typeof headers === 'object' && headers !== null
? (headers as Record<string, unknown>).Authorization
: undefined;
if (typeof auth === 'string' && auth.includes(ENV_INTERPOLATION)) {
// Only the gbrain connect lane writes the {env:GBRAIN_REMOTE_TOKEN}
// interpolation — unambiguously ours even without a receipt url. But a
// url mismatch (another serve) OR a LOCAL expectation (expect.sourceId
// — the workspace stdio lane; the kind-mismatch mirror of the local
// branch above) is another lane's wiring: ours-other-source.
return expect.url === undefined && expect.sourceId === undefined
? 'ours-same-source'
: 'ours-other-source';
}
return 'foreign';
}
return 'foreign';
}
// ── Rendering ───────────────────────────────────────────────────────────────
function entryValue(entry: OpencodeMcpEntry): Record<string, unknown> {
if (entry.kind === 'local') {
return {
type: 'local',
command: entry.command,
environment: entry.environment,
enabled: true,
};
}
const token =
entry.tokenMode === 'inline'
? `Bearer ${entry.bearerToken ?? ''}`
: `Bearer ${ENV_INTERPOLATION}`;
return {
type: 'remote',
url: entry.url,
headers: { Authorization: token },
enabled: true,
};
}
/** Copy-pasteable snippet for the refusal paths (the user is never stranded).
* SECURITY: an inline bearer is substituted with a literal placeholder the
* snippet rides thrown error messages (parse refusal, foreign refusal), and an
* error path must never embed the real secret in text that lands in logs,
* receipts, or stderr. Only the human-facing snippet changes; the write path
* still renders the real token. */
export function opencodeEntrySnippet(entry: OpencodeMcpEntry): string {
const safe: OpencodeMcpEntry =
entry.kind === 'remote' && entry.tokenMode === 'inline'
? { ...entry, bearerToken: '<paste-token-here>' }
: entry;
return JSON.stringify({ mcp: { [safe.name]: entryValue(safe) } }, null, 2);
}
function assertEntryName(name: string): void {
if (!/^[A-Za-z0-9_-]+$/.test(name)) {
throw new Error(
`MCP server name "${name}" is not a simple key ([A-Za-z0-9_-]+) — pick a simpler --name`,
);
}
}
function entryCarriesSecret(entry: OpencodeMcpEntry): boolean {
return entry.kind === 'remote' && entry.tokenMode === 'inline';
}
/** True when config text carries an INLINE bearer credential (any
* `Bearer <value>` that is not the `{env:…}` interpolation) the rule that
* decides whether a backup copy must be tightened to 0600. */
export function textCarriesInlineBearer(text: string): boolean {
return /Bearer\s+(?!\{env:)\S/.test(text);
}
/** Unique-suffix backup (`<config>.bak-<hex>`): two overlapping runs can
* never clobber each other's snapshot. The config-dir lock covers the WRITE,
* but a backup must survive until the caller's post-write verification (the
* harness network smoke) which runs AFTER the lock is released, so a fixed
* `.bak` name would let run B's writer overwrite run A's snapshot and a
* failed run A would then restore (and revoke against) run B's state.
* chmod 0600 whenever the copied content carries an inline bearer
* (copyFileSync onto a fresh path takes the source mode, but a hand-loosened
* source must not propagate a loose mode to a token-bearing backup). */
function createUniqueBackup(configPath: string, priorText: string): string {
const backupPath = `${configPath}.bak-${randomBytes(6).toString('hex')}`;
copyFileSync(configPath, backupPath);
if (textCarriesInlineBearer(priorText)) chmodSync(backupPath, 0o600);
return backupPath;
}
// No explicit eol: jsonc-parser detects and preserves the file's own EOLs
// (verified: a CRLF config keeps CRLF through modify/applyEdits).
const FORMATTING = { formattingOptions: { insertSpaces: true, tabSize: 2 } };
// ── Write ───────────────────────────────────────────────────────────────────
/**
* Idempotently write the managed `mcp.<name>` entry via a comment-preserving
* surgical edit. Refuses foreign entries; replaces ours-same-source silently;
* replaces ours-other-source only when `allowReplaceOtherSource` (callers
* warn first). Validates the render before the atomic swap.
*/
export function writeOpencodeMcpEntry(
configPath: string,
entry: OpencodeMcpEntry,
opts: { expect?: OpencodeEntryExpectation; allowReplaceOtherSource?: boolean } = {},
): WriteOpencodeEntryResult {
assertEntryName(entry.name);
if (entry.kind === 'remote' && entry.tokenMode === 'inline' && !entry.bearerToken) {
throw new Error('inline token mode requires a bearerToken');
}
const notes: string[] = [];
const snippet = opencodeEntrySnippet(entry);
const { text, existed } = readConfigRaw(configPath);
const parsed = parseOpencodeConfig(text, configPath, snippet);
const priorKind = opencodeEntryKind(parsed, entry.name, opts.expect);
if (priorKind === 'foreign') {
throw new Error(
`mcp.${entry.name} in ${configPath} is not a gbrain-managed entry — refusing to overwrite it. ` +
`Remove it (or pick another --name) and re-run.`,
);
}
if (priorKind === 'ours-other-source' && !opts.allowReplaceOtherSource) {
// Caller-appropriate refusal text: on the REMOTE path (expect.url — the
// harness/connect lanes) no GBRAIN_SOURCE is involved, and connect's
// documented escape hatch is --force; the GBRAIN_SOURCE wording belongs
// to the local/workspace lane only.
throw new Error(
opts.expect?.url !== undefined
? `mcp.${entry.name} in ${configPath} is a gbrain registration that does not match this endpoint ` +
`(${opts.expect.url}) — another install or lane owns it; pass --force to replace it, or pick another --name.`
: `mcp.${entry.name} in ${configPath} belongs to a DIFFERENT gbrain workspace ` +
`(GBRAIN_SOURCE mismatch) — re-run with the overwrite confirmation to reroute it, or pick another --name.`,
);
}
if (priorKind === 'ours-other-source') {
notes.push(
opts.expect?.url !== undefined
? `replaced a gbrain registration that did not match this endpoint (url/lane mismatch).`
: `replaced a gbrain registration that pointed at a different workspace (source mismatch).`,
);
}
const baseText = text.trim() === '' ? '{\n "$schema": "https://opencode.ai/config.json"\n}\n' : text;
const edits = modify(baseText, ['mcp', entry.name], entryValue(entry), FORMATTING);
const nextText = applyEdits(baseText, edits);
// Post-render validation: parse + deep-assert our entry + assert every
// OTHER top-level key survives. Any failure leaves the original untouched.
const rendered = parseOpencodeConfig(nextText, configPath, snippet);
const renderedMcp = rendered.mcp as Record<string, unknown> | undefined;
const ours = renderedMcp?.[entry.name];
if (JSON.stringify(ours) !== JSON.stringify(entryValue(entry))) {
throw new Error(
`post-render validation failed: mcp.${entry.name} did not round-trip — original file left untouched.`,
);
}
for (const key of Object.keys(parsed)) {
if (key === 'mcp') continue;
if (JSON.stringify(rendered[key]) !== JSON.stringify(parsed[key])) {
throw new Error(
`post-render validation failed: top-level key "${key}" changed — original file left untouched.`,
);
}
}
if (typeof parsed.mcp === 'object' && parsed.mcp !== null) {
for (const key of Object.keys(parsed.mcp as Record<string, unknown>)) {
if (key === entry.name) continue;
const before = (parsed.mcp as Record<string, unknown>)[key];
const after = renderedMcp?.[key];
if (JSON.stringify(after) !== JSON.stringify(before)) {
throw new Error(
`post-render validation failed: mcp.${key} (not ours) changed — original file left untouched.`,
);
}
}
}
const secret = entryCarriesSecret(entry);
let backupPath: string | null = null;
if (existed) {
backupPath = createUniqueBackup(configPath, text);
if (secret) chmodSync(backupPath, 0o600); // re-runs: the backup carries the previous token
}
atomicWriteTextFile(configPath, nextText, secret ? { forceMode: 0o600 } : { freshMode: 0o644 });
if (secret && existed) {
notes.push(`${configPath} tightened to 0600 — it now carries a bearer token.`);
}
return {
configPath,
replacedPrior: priorKind !== 'absent',
priorKind,
backupPath,
writtenText: nextText,
notes,
};
}
// ── Remove ──────────────────────────────────────────────────────────────────
/**
* Remove the managed entry (fingerprint-keyed; everything else survives
* byte-for-byte). Absent file / absent entry are calm no-ops. Foreign
* entries refuse removal never deletes what gbrain does not own.
* `skipOtherSource` turns an `ours-other-source` match into a calm skip-with-
* note instead of a removal (the uninstall sweep passes it: a gbrain entry
* from a DIFFERENT workspace is not this uninstall's to delete).
*/
export function removeOpencodeMcpEntry(
configPath: string,
name: string,
expect: OpencodeEntryExpectation = {},
opts: { skipOtherSource?: boolean } = {},
): RemoveOpencodeEntryResult {
assertEntryName(name);
const notes: string[] = [];
if (!existsSync(configPath)) {
return { configPath, removed: false, backupPath: null, notes: ['no opencode config — nothing to remove'] };
}
const { text } = readConfigRaw(configPath);
const parsed = parseOpencodeConfig(text, configPath);
const kind = opencodeEntryKind(parsed, name, expect);
if (kind === 'absent') {
return { configPath, removed: false, backupPath: null, notes: ['no gbrain-managed entry — nothing to remove'] };
}
if (kind === 'foreign') {
throw new Error(
`mcp.${name} in ${configPath} is not a gbrain-managed entry — refusing to remove it.`,
);
}
if (kind === 'ours-other-source' && opts.skipOtherSource) {
return {
configPath,
removed: false,
backupPath: null,
notes: [
`mcp.${name} in ${configPath} belongs to a DIFFERENT gbrain workspace (source mismatch) — left in place.`,
],
};
}
if (kind === 'ours-other-source') {
notes.push('removed a gbrain registration that pointed at a different workspace (source mismatch).');
}
const edits = modify(text, ['mcp', name], undefined, FORMATTING);
const nextText = applyEdits(text, edits);
parseOpencodeConfig(nextText, configPath); // never leave opencode unreadable
// Unique backup, 0600 when the copied content carries an inline bearer —
// the removed entry may BE the token-bearing one, and a fixed-name copy
// onto a pre-existing loose-mode backup would keep the loose mode.
const backupPath = createUniqueBackup(configPath, text);
atomicWriteTextFile(configPath, nextText);
return { configPath, removed: true, backupPath, notes };
}
// ── Sibling-global reconcile (the two-filename merge blind spot) ────────────
/**
* opencode merges the user-global `opencode.json` AND `opencode.jsonc` when
* both exist. Before writing `mcp.<name>` into one of them, reconcile the
* SIBLING file: an ours-classified entry there is removed (one owner per
* name left in place it survives as a shadow registration whose merge
* winner is ambiguous, and a later removal of the primary "reveals" it); a
* FOREIGN entry refuses loudly naming BOTH files (same refusal posture as
* the primary-file foreign case the merge winner is not ours to fight
* over). No-op when the path is not a global-pair member or the sibling is
* absent/entry-less. Callers hold the opencode config-dir bootstrap lock
* (both files share the dir one lock covers both) and call this ONLY for
* user-global writes (project-scope sibling semantics are unobserved).
*/
export function reconcileOpencodeSiblingGlobal(
configPath: string,
name: string,
expect: OpencodeEntryExpectation = {},
): { siblingPath: string | null; removed: boolean; notes: string[] } {
const siblingPath = opencodeGlobalSiblingPath(configPath);
if (!siblingPath || !existsSync(siblingPath)) return { siblingPath, removed: false, notes: [] };
const { text } = readConfigRaw(siblingPath);
const parsed = parseOpencodeConfig(text, siblingPath);
const kind = opencodeEntryKind(parsed, name, expect);
if (kind === 'absent') return { siblingPath, removed: false, notes: [] };
if (kind === 'foreign') {
throw new Error(
`mcp.${name} in ${siblingPath} is not a gbrain-managed entry — opencode merges ${siblingPath} AND ` +
`${configPath} when both exist, so writing mcp.${name} into ${configPath} would fight it with an ` +
`ambiguous merge winner. Remove it (or pick another --name) and re-run.`,
);
}
const r = removeOpencodeMcpEntry(siblingPath, name, expect);
const notes = [
`removed the gbrain mcp.${name} entry from ${siblingPath} — opencode merges both global filenames, and the ` +
`registration being written lands in ${configPath} (one owner per name).`,
...r.notes,
];
return { siblingPath, removed: r.removed, notes };
}
/**
* True when `mcp.<name>` exists as a REMOTE-type entry (regardless of
* ownership). The workspace stdio lane consults this before writing a local
* entry into the user-global config: a remote entry under our name is either
* the harness lane's (bootstrap harness) or foreign either way the stdio
* lane must not fight it (the codexBlockOwnsName analog, #4043 ownership
* rule). Best-effort: unreadable/unparseable configs return false (the write
* path re-checks with full refusal semantics).
*/
export function opencodeRemoteEntryExists(configPath: string, name: string): boolean {
try {
const { text, existed } = readConfigRaw(configPath);
if (!existed) return false;
const parsed = parseOpencodeConfig(text, configPath);
const mcp = parsed.mcp;
if (typeof mcp !== 'object' || mcp === null) return false;
const entry = (mcp as Record<string, unknown>)[name];
return typeof entry === 'object' && entry !== null && (entry as Record<string, unknown>).type === 'remote';
} catch {
return false;
}
}
// ── Status/recovery helpers ─────────────────────────────────────────────────
/**
* Recover the inline bearer from OUR remote entry (harness `--status` token
* liveness the receipt never stores the token). Returns null when the file
* or entry is absent, foreign, env-mode, or unreadable as JSONC.
*/
export function parseOpencodeEntryBearer(configPath: string, name: string, expectUrl?: string): string | null {
try {
const { text, existed } = readConfigRaw(configPath);
if (!existed) return null;
const parsed = parseOpencodeConfig(text, configPath);
const kind = opencodeEntryKind(parsed, name, { url: expectUrl });
if (kind !== 'ours-same-source') return null;
const entry = (parsed.mcp as Record<string, unknown>)[name] as Record<string, unknown>;
if (entry.type !== 'remote') return null;
const auth = (entry.headers as Record<string, unknown> | undefined)?.Authorization;
if (typeof auth !== 'string' || !auth.startsWith('Bearer ')) return null;
const token = auth.slice('Bearer '.length);
if (token.includes('{env:')) return null; // env-interpolated — no inline token to recover
return token || null;
} catch {
return null;
}
}
+5 -3
View File
@@ -209,7 +209,7 @@ export const PHASES: PhaseSpec[] = [
title: 'Identity interview (confirmed read-back)',
resume_hint:
'gbrain bootstrap interview --init, then --set each answer, then --confirm <hash>. ' +
'Claude Code only: also record the MCP scope consent (--set MCP_SCOPE <project|user>) BEFORE --confirm',
'Claude Code and opencode: also record the MCP scope consent (--set MCP_SCOPE <project|user>) BEFORE --confirm',
detect: (ws) => {
const exists = existsSync(interviewStatePath(ws));
const st = interviewStatus(ws);
@@ -265,8 +265,10 @@ export const PHASES: PhaseSpec[] = [
// outside the harness being wired). Advisory prose; the grep pins in
// scripts/check-bootstrap-templates.sh §(e) are the enforcement.
resume_hint:
'gbrain bootstrap hooks --harness <claude-code|codex> — MCP scope consent is ' +
'Claude Code only (recorded during the interview, pre-confirm); Codex registrations are always user-global (no scope flag)',
'gbrain bootstrap hooks --harness <claude-code|codex|opencode> — MCP scope consent applies on ' +
'Claude Code and opencode (recorded during the interview, pre-confirm; opencode defaults to ' +
'user-global — the sharing-safe choice, since it spawns project-config servers with no trust gate); ' +
'Codex registrations are always user-global (no scope flag)',
detect: (ws, ctx) => {
const regs = ctx.receipt?.registrations ?? [];
if (regs.length > 0) {
+1 -1
View File
@@ -98,7 +98,7 @@ export function resolveBrainDataDir(gbrainHomeDir: string): string {
// ---------------------------------------------------------------------------
export interface RegistrationRemovalRequest {
host: 'claude-code' | 'codex';
host: 'claude-code' | 'codex' | 'opencode';
scope: string;
detail?: string;
}
+68 -3
View File
@@ -1,8 +1,9 @@
/**
* AgentRunner pluggable contract for invoking external agents (openclaw,
* hermes, codex, ) inside the claw-test harness. Two implementations ship
* (openclaw, hermes); the interface stays narrow and concrete so adding
* another runner is a ~100-line file.
* hermes, grok, ) inside the claw-test harness. Three implementations ship
* (openclaw, hermes, grok); the interface stays narrow and concrete with
* the shared detect/env helpers below, adding another runner is a ~50-line
* file plus one registerAgentRunner line.
*
* The harness wraps spawn/timeout/transcript-capture; runners only have to
* answer "where's your binary?" and "how do I invoke it with this prompt?".
@@ -15,6 +16,9 @@
*
*/
import { execFileSync } from 'child_process';
import { statSync } from 'fs';
export interface AgentRunner {
/** Stable agent name used by --agent flag and friction `agent` field. */
readonly name: string;
@@ -126,6 +130,67 @@ export function validateBinPathEnv(envName: string, p: string): string | null {
return null;
}
/**
* Shared binary resolution for runners: `$<envName>` (validated) > `which
* <binName>` > unavailable, then a regular-file + executable-bit stat. The
* three runners previously carried byte-identical copies of this body; the
* extraction is behavior-preserving (same reason strings, same ordering).
*/
export function detectBinary(envName: string, binName: string): DetectResult {
const fromEnv = process.env[envName]?.trim();
let binPath: string | undefined;
if (fromEnv) {
const validation = validateBinPathEnv(envName, fromEnv);
if (validation) return { available: false, reason: validation };
binPath = fromEnv;
} else {
try {
// execFileSync, not a shell string: binName comes from callers today
// (constants), but this helper is exported — interpolating it into a
// shell would make a future metachar-bearing name become code.
const out = execFileSync('which', [binName], { encoding: 'utf-8', stdio: ['ignore', 'pipe', 'ignore'] });
const found = out.trim();
if (!found || !found.startsWith('/')) {
return { available: false, reason: `${binName} not on PATH` };
}
binPath = found;
} catch {
return { available: false, reason: `${binName} not on PATH` };
}
}
if (!binPath) return { available: false, reason: 'no binary resolved' };
try {
const s = statSync(binPath);
if (!s.isFile()) return { available: false, reason: `not a regular file: ${binPath}` };
// eslint-disable-next-line no-bitwise
if (!(s.mode & 0o111)) return { available: false, reason: `not executable: ${binPath}` };
} catch (e) {
return { available: false, reason: `stat failed: ${e instanceof Error ? e.message : String(e)}` };
}
return { available: true, binPath };
}
/**
* Build the child env for a runner spawn: process.env filtered to the
* runner's allowlist, then caller overrides merged on top (overrides win
* the live lane's PATH-shim prepend depends on this precedence).
*/
export function filterAllowlistEnv(
allowlist: readonly string[],
overrides: Record<string, string>,
): Record<string, string> {
const baseEnv: Record<string, string> = {};
for (const key of allowlist) {
const v = process.env[key];
if (typeof v === 'string') baseEnv[key] = v;
}
return { ...baseEnv, ...overrides };
}
// ---------------------------------------------------------------------------
// Registry
// ---------------------------------------------------------------------------

Some files were not shown because too many files have changed in this diff Show More