Compare commits

...
Author SHA1 Message Date
Garry Tan 771acdf752 Merge remote-tracking branch 'origin/master' into garrytan/gbrain-commit-indexing
# Conflicts:
#	BOOTSTRAP_FOR_AGENTS.md
#	CHANGELOG.md
#	VERSION
#	package.json
#	templates/bootstrap/template-repo/README.md
2026-08-12 09:16:07 -07:00
Garry TanandClaude Fable 5 2d30c3e29b chore(skills): refresh skills.lock version stamp for 0.45.5.0
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-12 08:21:59 -07:00
Garry TanandClaude Fable 5 a30add4e53 v0.45.5.0 chore(release): re-version 0.46.0.0 -> 0.45.5.0
Same release content; the version slots into the 0.45.x line per the
natural-next-off-master convention. VERSION, package.json, CHANGELOG
header, bootstrap runbook stamp, vendored template regen, lockfile, and
llms bundles all move together (trio audit green; stamp + template +
llms freshness guards re-run green).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-12 08:20:50 -07:00
Garry TanandClaude Fable 5 d351a5fa10 docs: cross-model doc-review fixes for v0.46.0.0
Round two from the post-ship documentation review, checked against the code:

- CHANGELOG.md: two shipped fixes were missing from the 0.46.0.0 entry
  (git typechange/unmerged statuses no longer dropped by sync; a wedged
  lock holder past the staleness ceiling now fails doctor with the
  break-lock remedy), and the env-resolver bullet over-claimed its
  coverage — narrowed to the doctor + staleness-threshold vars it
  actually serves. Additive + accuracy only; no entries replaced.
- docs/guides/live-sync.md: Tricky Spots item for the staleness
  wall-clock ceiling (GBRAIN_STALENESS_CEILING_HOURS, default 72h,
  ramp-not-step) and the hours_since_last_sync status field.
- docs/ENGINES.md: the quiesce paragraph over-stated marker release —
  now describes the pid-checked orphan adoption, the daemon self-heal
  poll, and the confirm-the-pid manual step for daemonless hosts.
- docs/guides/minions-fix.md: the verify block's bare autopilot --status
  now notes the exit code is the verdict (and the set -e caveat).
- llms-full.txt regenerated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-12 01:09:05 -07:00
Garry TanandClaude Fable 5 f43e626c71 docs: sync documentation for v0.46.0.0
The brain-currency wave shipped new user-facing surface the guides did not
carry: the trustworthy autopilot --status exit codes (0/1/2 + paused),
the keyless-safe sync-and-embed chain, and the migrate quiesce fence.

- docs/guides/live-sync.md: keyless clean-refusal note on the embed --stale
  primitive; new "gate on the daemon's heartbeat" verify item with the
  exit-code contract.
- docs/guides/cron-schedule.md: the recommended autopilot health-gate cron
  line (the status classifier already special-cases this monitor line so it
  never reads as an install).
- docs/ENGINES.md: migrate --to now documents the pause-marker mutex, worker
  job-pickup fence, DB-lock drain (GBRAIN_MIGRATE_QUIESCE_SECONDS, default
  300), crash-safe release, and daemon relaunch after the flip.
- docs/operations/headless-install.md: keyless mode's "refuse cleanly" claim
  made concrete — the documented chain exits 0 on a bare stale embed.
- INSTALL_FOR_AGENTS.md Step 7: health-gate bullet + keyless-safe chain note.
- skills/maintain/SKILL.md: autopilot check now teaches the exit-code
  semantics and the --json report fields.
- docs/TESTING.md: inventory entries for the two new harness-tier serial
  suites (real-launchd lifecycle, agent-scheduler shell-chain contract).
- llms-full.txt regenerated (TESTING.md is inlined in the bundle).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-12 00:54:18 -07:00
Garry TanandClaude Fable 5 d2fb2023b8 v0.46.0.0 feat(autopilot): brain currency — honest daemon health, migrate quiesce fence, keyless scheduler chain
VERSION + CHANGELOG + TODOS for the brain-currency PR-A wave, plus the
version-stamped bootstrap runbook and vendored template regen.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-12 00:44:33 -07:00
Garry TanandClaude Fable 5 375f14a06f test(autopilot): migrate the crontab wrapper-detection pin to the pure seam
The inline crontab.includes check this static pin matched became
crontabIndicatesAutopilotInstall; the pin now exercises the function
directly (wrapper line still detected) and asserts the detect path is
wired through it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-12 00:37:29 -07:00
Garry Tan 60f7710c31 Merge remote-tracking branch 'origin/master' into garrytan/gbrain-commit-indexing 2026-08-12 00:25:24 -07:00
Garry TanandClaude Fable 5 d542f8ad8d fix(migrate,minions): ownership-conditional pause release + post-claim fence re-check
Round-4 adversarial hardening on the quiesce protocol. Orphan adoption
verifies identity AFTER its atomic rename — a racer that completed the
whole adopt-and-rewrite in the read-to-rename gap gets its live marker
restored no-clobber instead of stolen. Marker release compares content
before unlinking, so a claim that changed hands can never delete the new
owner's fence mid-copy. On the worker side, a job whose claim committed
after migrate's drain probe is released back to the queue un-run
(delayed one minute, conditional on our lock token) instead of executed
into the doomed source engine. The cross-host residual — a worker on
another machine never sees a local marker file — is filed as the
DB-visible-pause follow-up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-12 00:24:59 -07:00
Garry TanandClaude Fable 5 95d3d55fbd docs: KEY_FILES current-state for the brain-currency wave; BRAIN_CURRENCY design doc
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-12 00:21:46 -07:00
Garry TanandClaude Fable 5 234c2a417f test(e2e): real-launchd lifecycle + agent-scheduler contract harness tier
Two serial-lane e2e files that assert behavior, not artifacts: a
launchd lifecycle suite (shimmed supervisor on every platform; REAL
launchd on darwin under a per-run unique label, own-label cleanup only)
driving install, self-disable via three strikes, status exit codes,
reinstall, and uninstall through the actual CLI; and an
agent-scheduler contract suite that runs the documented sync-and-embed
shell chain end-to-end against a keyless brain, including the
pull-failure case that must break the chain and the heartbeat that must
advance on quiet re-runs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-12 00:21:46 -07:00
Garry TanandClaude Fable 5 0708e6baa9 fix(embed,sync): keyless stale embed refuses cleanly; typechange statuses; test-home isolation
The documented agent-scheduler chain (sync, then a stale embed) exited 1
on every keyless brain because EmbeddingDisabledError fell through to a
blanket exit. A bare stale run now refuses cleanly via the pure
isKeylessStaleRefusal predicate (stderr hint, zero-failure result);
explicit asks — a slug, a slugs list, the all flag — still exit 1,
mirroring the dispatch precedence where a slugs list wins over stale.

buildSyncManifest handles the T/U/C git statuses instead of silently
dropping them (typechange routes to modified; the file-to-symlink
staleness residual is documented and filed). Status reports gain
hours_since_last_sync so the ceiling-ramped staleness_hours stops
doubling as the human-facing wall-clock number.

The sync-failures ledger honors GBRAIN_SYNC_FAILURES_DIR and the test
preload (bunfig) points it at a scratch dir, so test runs stop writing
failure rows into the developer's real brain home.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-12 00:21:45 -07:00
Garry TanandClaude Fable 5 0f1089e836 fix(autopilot): heartbeat-truth status, self-disable guard, migrate quiesce fence
Status becomes honest and engine-free: a pure classifier maps heartbeat
age, markers, and install detection to not_installed/disabled/paused/
never_run/stale/fresh with exit codes 0/1/2, dispatched in cli.ts before
connectEngine so a live PGLite daemon's exclusive lock or a DB outage
cannot take down the alarm. Tolerance is six intervals (the adaptive
scheduler sleeps two between ticks on healthy brains; three flapped
false alarms), NaN-proof at both parse sites, and a cron'd status
monitor line no longer reads as an install.

The generated wrapper self-disables for real when its repo vanishes —
marker first, then supervisor-specific stop (launchctl bootout /
systemctl disable) since a bare exit 0 under KeepAlive is just a quieter
respawn loop — and requires three consecutive misses so a slow-mounting
volume cannot permanently kill a healthy install. Install and uninstall
clear both markers; wrapper artifacts resolve through gbrainHomePath
with GBRAIN_HOME baked in at install time.

gbrain migrate quiesces the whole write plane for its copy window: an
unconditional atomic pause-marker claim doubles as the migration mutex
(a live holder, a foreign hold, or an unwritable marker all abort;
orphan adoption is renameSync compare-and-claim), in-flight writers
drain via the DB lock table plus active-job count (supervisor presence
locks excluded) before enumeration, minion workers stop claiming queued
jobs while the marker is parked, cleanup registers at claim time, the
daemon self-clears a dead migrate's orphan, and after a clean config
flip the daemon detects the engine identity change and relaunches onto
the new engine instead of syncing into the abandoned one indefinitely.
Malformed connection URLs classify as operator config verdicts rather
than crashes, and torn config reads cannot spuriously restart the
daemon.

Marker paths, the launchd label seam, and the pid-ownership primitives
live in the new leaf module src/core/autopilot-paths.ts so migrate and
the worker never import the autopilot command surface (flag-registry
pooling); the regenerated registry reflects the new one-level imports.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-12 00:21:25 -07:00
Garry TanandClaude Fable 5 329d99ccbe fix(source-health): wall-clock ceiling for content-relative staleness
A source whose stored content timestamp trailed its last sync reported
zero lag forever — the shape that let a dead sync daemon stay invisible
for 71 days. lagFromContentMs now ramps toward stale past a ceiling
(default 72h, GBRAIN_STALENESS_CEILING_HOURS / GBRAIN_SYNC_FRESHNESS_FAIL_HOURS)
instead of returning a flat 0; ramping rather than stepping keeps the
warn tier firing before the fail tier across sync_freshness,
federation_health, and gbrain status. The ceiling resolves once per
report (hoisted out of the per-source loops), and the numeric env
plumbing moves to a shared warn-once resolver (src/core/env-number.ts)
so doctor and source-health share one memo instead of warning twice.
Doctor's wedged-lock alarm prints minutes below one hour so a
fractional ceiling override cannot produce a "held the lock for 0h"
message.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-12 00:21:02 -07:00
Garry Tan bf063757ab Merge remote-tracking branch 'origin/master' into garrytan/gbrain-commit-indexing 2026-08-11 18:05:21 -07:00
Garry TanandClaude Fable 5 1134c27b0f fix(pglite): embed WASM + extension assets so the compiled binary can serve
A `bun build --compile` gbrain binary could not `serve` a PGLite brain: the
compile bundles JS but not PGLite's runtime payload (pglite.wasm, initdb.wasm,
pglite.data, vector/pg_trgm tarballs), so `serve` on PGLite died with a
bunfs/ENOENT. Now the assets ride inside the binary.

- src/core/pglite-embedded-assets.ts: embeds the five assets via
  `import … with { type: 'file' }` (the ENG-6 idiom) and exposes
  getEmbeddedPgliteOptions() → { pgliteWasmModule, initdbWasmModule, fsBundle,
  extensions:{vector,pg_trgm} }. WASM/fsBundle are consumed as bytes; the two
  extension tarballs are materialized to a content-addressed temp file (atomic,
  size-verified reuse) because PGLite reads them via fs.createReadStream, which
  cannot read a /$bunfs path. Unconditional (works in bun-run and compiled),
  so no fragile mode branch.
- src/core/pglite-engine.ts: static-import getEmbeddedPgliteOptions (engine
  path stays static per the engine-dynamic-import invariant); spread into both
  PGlite.create sites (initial + WAL-repair retry). The bunfs classifier stays
  as a backstop but no longer fires for a correct binary.
- scripts/check-pglite-embedded.sh (+ smoketest): compiles a focused binary and
  asserts it boots PGLite, CREATE EXTENSION vector/pg_trgm, and round-trips a
  page — wired into `bun run verify` (now 37 checks), check:all, and
  check:pglite-embedded. Fail-soft only when compile is unavailable.
- agent-harness.ts probeCompiledPglite now passes → the real-agent e2e uses the
  fast compiled MCP server. TODOS: the P2 "can't serve PGLite" item is closed.

Verified: fresh compiled binary ran `search`/`query` against a PGLite brain and
returned the seeded row (no bunfs/ENOENT); verify 37/37; pglite-engine 120/0
source-mode; typecheck clean; engine-dynamic-import + parity guards pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 15:52:19 -07:00
Garry TanandClaude Fable 5 ce781d3b95 fix(ci): real-agent-e2e job — bash array + --timeout (actionlint SC2086 + bun-test-timeout guard)
The real-agent-e2e job's file loop used an unquoted $FILES (SC2086) and ran
`bun test` without --timeout (check-bun-test-timeout guard). Switch to a bash
array and add --timeout=600000 (real-agent turns are slow; the tests self-skip
without authed binaries so it's a no-op elsewhere).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 15:34:03 -07:00
Garry TanandClaude Fable 5 2f6b817a28 test(bootstrap): real-agent e2e — drive the actual claude + codex binaries end to end
Closes the audit's biggest gap ("no real harness ever drives a turn"). Adapts
gstack's PTY/headless agent harness to prove the bootstrap install + smoke work
against the REAL binaries, not PATH shims. Test/CI/docs only — zero src changes.

- test/helpers/agent-harness.ts: hermetic clean-room child env (ported from
  gstack; drops CONDUCTOR_/CLAUDE_/GSTACK_/MCP_/GBRAIN_, promotes
  GSTACK_ANTHROPIC_API_KEY→ANTHROPIC_API_KEY), real-binary resolvers + auth
  probes, headless `claude -p --output-format stream-json` and `codex exec
  --json` turn runners, a gbrain stdio MCP-config writer, and a keyless brain
  seeder. + a fixture-parse unit test (no binary needed).
- test/e2e/bootstrap-real-claude.serial.test.ts: real `gbrain bootstrap`
  install → REAL `claude mcp add` (verified via `claude mcp get`) → verify
  exit 0 → a real `claude -p --mcp-config --strict-mcp-config` turn that
  invokes mcp__gbrain__search and answers from the brain (proven: toolCalls
  include mcp__gbrain__search, final text carries the seeded fact).
- test/e2e/bootstrap-real-codex.serial.test.ts: same install with REAL `codex
  mcp add` into a real ~/.codex/config.toml + Gate-3 pull-protocol assertion,
  then a real `codex exec --json` turn surfacing the fact (MCP or the pull-
  protocol shell path). Bounded retry absorbs codex's occasional MCP-call
  cancellation without softening the fact-requiring assertion.
- Everything hermetic (temp HOME/CLAUDE_CONFIG_DIR/CODEX_HOME/GBRAIN_HOME;
  real ~/.codex auth copied read-only) and skipIf-gated so it self-skips
  cleanly where the binaries/auth are absent.
- heavy-tests.yml: gated `real-agent-e2e` job (nightly/label, never the PR
  shard; no-op on a runner without authed binaries).
- TODOS: compiled `gbrain` binary can't serve a PGLite brain (bun compile
  omits the WASM/extension payloads); harness falls back to `bun run` serve.

Verified against live claude 4.6 + codex 0.147.0: 15 pass / 0 fail; verify
36/36; typecheck clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 14:56:52 -07:00
Garry Tan 99e15f0a10 Merge remote-tracking branch 'origin/master' into garrytan/codex-as-agent-default-install
# Conflicts:
#	CHANGELOG.md
#	VERSION
#	docs/architecture/KEY_FILES.md
#	package.json
2026-08-11 12:52:03 -07:00
Garry TanandClaude Fable 5 6fb6790b2a test(bootstrap): real end-to-end coverage — cross-session recall, per-turn content, Codex door, realistic corpus
Closes the seven e2e gaps a coverage audit surfaced: the plumbing was
well-unit-tested but the product claims ("Codex works, context shows up every
turn with real content, it remembers across restarts, machine two recovers,
Postgres works") were unproven end to end. Test-only wave — zero src changes.

- Hermetic synthetic corpus (test/fixtures/bootstrap-corpus/ + a loader helper):
  12 interlinked pages (52 edges, timelines), 12 world/private beliefs, 8 gold
  queries — curated from the gbrain-evals synthetic corpora, 100% placeholder
  names, so recall is asserted on a real multi-entity brain instead of a
  2-node self-planted probe.
- GAP1 magic moment: author a fact via the real write path, disconnect the
  engine, reopen against the same DB, recall it — a real session boundary, not
  verify.ts's same-connection SQL read-back. Plus a source-isolation assertion.
- GAP2 per-turn content: hook-under-serve Pin 1 now seeds a known fact and
  asserts its text lands in the injected block AND private beliefs never do
  (was: empty brain, empty_block accepted as a pass).
- GAP3 Codex door: assert the rendered AGENTS.md carries the Gate-3 brain-first
  pull protocol; make the fake codex shim implement `mcp get` so the [FIX7]
  target-verification can actually fail; the Docker cold-machine harness now
  exercises the hooks/MCP registration step instead of skipping it.
- GAP4 corpus recall: turn-context + verify graph-floor/qrels run on the real
  multi-entity brain with real edges.
- GAP5 attach: machine-two now re-ingests the cloned brain/ into a fresh DB and
  recalls a fact authored only on machine one — the multi-device payoff.
- GAP6 keyed + Postgres (env-gated): real embeddings prove semantic recall a
  paraphrase query can reach but keyless BM25 cannot; bootstrap verify drives a
  real Postgres engine (skipIf DATABASE_URL/keys absent).
- GAP7 persistence: session-end runs the REAL push (not the mocked seam) to a
  local bare remote and the remote receives the content; a planted secret is
  blocked at the gate; the 15-min cron installs and fires a scan-gated push.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 11:30:07 -07:00
Garry TanandClaude Fable 5 5183e9f039 feat(bootstrap): default per-turn hooks + search mode ON without a prompt
Installing gbrain for your coding agent IS the consent for the behaviors that
make it work, so stop re-litigating them with install-time questions whose
"no" defeats the product:

- Per-turn hooks (Claude Code) install ON by default — no prompt. Off-ramps:
  `--no-hooks` at install, `GBRAIN_HOOKS=0` at runtime, `bootstrap uninstall`.
  The "hooks installed" line now surfaces the kill switch so default-on is
  never silent. A persisted HOOKS_CONSENT=no (interview --skip) still declines.
- Search mode defaults to `balanced` silently (nobody knows the modes at
  install; `gbrain search modes` changes it any time).
- MCP scope stays the ONE deliberate prompt — project vs user is a real
  cross-repo privacy choice, not friction.

Marks the two consents `silent: true` in the question bank (new QuestionSpec
field), rewrites the runbook phases so the agent no longer asks them, adds the
`--no-hooks` flag (+ registry regen), and adds default-on / opt-out tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 09:48:11 -07:00
Garry TanandClaude Fable 5 f55af9da7b fix(skills): declare 'bootstrap my data' trigger in cold-start frontmatter
The doc reform added 'bootstrap my data' to cold-start's RESOLVER.md row (to
disambiguate data-bootstrap from agent-bootstrap) but not to the skill's own
frontmatter triggers, tripping the RESOLVER↔frontmatter round-trip contract
(resolver.test.ts). Declare it; regenerate skills.lock.json.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 13:53:42 -07:00
Garry TanandClaude Fable 5 56dd1438e5 fix(ci): silence SC2016 on the intentional askpass literal in release.yml
The one-shot GIT_ASKPASS script must contain literal $1 and
$TEMPLATE_REPO_PAT so they expand when /bin/sh runs it at git's credential
prompt, not when the outer shell writes the file — single quotes are correct.
Add a scoped shellcheck disable so actionlint passes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 13:46:37 -07:00
Garry TanandClaude Fable 5 d72047caff docs: sync KEY_FILES + bootstrap plan to security-fix behavior
Cross-referenced the security-fix round against the reference docs and
corrected the drift those commits introduced:

- workspace-push.ts entry: stage-FIRST-then-scan order (the TOCTOU fix),
  fail-closed blocked_unscannable, and the sources-push status -> exit-code map.
- hooks.ts entry: MCP registration pins `serve --surface full`.
- hook.ts entry: session-start/session-end pushes run in a detached child
  (non-blocking); atomic corpus write clears stale sidecars.
- bootstrap.ts entry: render hard-refuses a public origin (template door).
- verify.ts entry: source_id collision resolution (workspace-<path-hash>).
- AGENT_BOOTSTRAP_PLAN as-shipped delta note for the scan/stage reorder.

llms bundle unchanged (KEY_FILES is link-only); build:llms and
test/build-llms.test.ts green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 13:42:14 -07:00
Garry TanandClaude Fable 5 25b798d612 fix: regenerate flag registry for security-fix flags; drop fabricated gbrain capabilities doc ref
CI caught two real failures under the merged state:
- the flag registry lagged the blocked_unscannable/exit-5 flags the security
  round added, tripping the #2185 freshness guard.
- headless-install.md described the keyless capability report as a
  `gbrain capabilities` command, which the #3502 doc-command resolver
  rejects — reworded to prose (the real surface is bootstrap verify's report).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 12:09:19 -07:00
Garry TanandClaude Fable 5 7816981ac0 v0.45.0.0 feat(bootstrap): paste-in personal-agent install for Codex + Claude Code
Turns a Codex or Claude Code session into a persistent personal agent:
interview-rendered identity files, a local PGLite brain, per-turn context via
serve IPC (Claude Code hooks / Codex pull protocol), session-triggered
persistence, and a private GitHub repo as the agent's portable body. Keyless-
first (the harness model is the LLM; one optional key adds embeddings +
extraction). New `gbrain bootstrap` command family + `gbrain hook` + `gbrain
sweep`; doctor bootstrap health checks; latest-stable distribution ref +
template-repo publish job. Opt-in, additive — existing installs untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 11:04:43 -07:00
Garry TanandClaude Fable 5 2437a60784 fix(security): close adversarial-review findings — scan fails closed, whole-PEM redaction, bound repo push
Cross-model adversarial pass (Claude + Codex) on the bootstrap wave:

- secret scan fails CLOSED: an unreadable, oversized, or binary staged blob
  now blocks the push (blocked_unscannable, exit 5) instead of committing
  unscanned; only a confirmed staged deletion is skipped. This was the
  headline "block secrets before they leave the machine" property failing open.
- private-key redaction spans the whole PEM block (header+body+footer), not
  just the header line — the base64 body no longer survives into the corpus
  the sweep sends to an extraction provider.
- bootstrap repo commits the workspace (secret-scan-gated) before the first
  push and verifies the remote actually received it, so a push-fail retry
  can't adopt an empty remote as success.
- privacy verify is re-bound to origin immediately before push (a concurrent
  origin rewrite between verify and push is refused).
- session-end corpus write is atomic and clears the stale ingested/in-progress
  sidecars so a resumed session's appended transcript is re-ingested.
- public-origin refusal enforced at render (not only status); MCP "already
  registered" is verified to target this workspace, not blessed blindly;
  verify probe cleanup scopes deletes to its own slugs, not a token substring;
  allowlist fingerprint floor raised 8→16 hex.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 11:03:18 -07:00
Garry Tan 004434e886 Merge remote-tracking branch 'origin/master' into garrytan/codex-as-agent-default-install
# Conflicts:
#	docs/architecture/KEY_FILES.md
#	docs/eval-bench.md
#	src/core/cli-flag-registry.generated.ts
2026-08-10 10:28:29 -07:00
Garry TanandClaude Fable 5 7c9e9fd331 fix(bootstrap): origin-ownership gate, world visibility, collision-safe source id, consent + templates
- repo adoption requires an exact receipt repo_url match or authed-owner
  check (undefined repo_url was a wildcard); create verifies privacy BEFORE
  the first push.
- verify sets facts.default_visibility=world if unset, so agent-authored
  facts surface in per-turn context (they defaulted private before).
- source_id derives a path-hash suffix when 'workspace' is taken by another
  checkout; every consumer reads manifest.source_id.
- skipped HOOKS_CONSENT now declines (was falling through to default yes);
  --minimal refuses on an initialized manifest; tilde fences escaped.
- MCP registration pins --surface full; status hard-fails a public origin
  (template door); receipt writers guard against newer/corrupt receipts;
  uninstall only claims brain-deleted after a real rm.
- templates ship jobs disabled + provider-consent + support-relay lines;
  soul-audit re-runs over the shared interview bank. TODOS: 11 follow-ups.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 10:27:23 -07:00
Garry TanandClaude Fable 5 aeb14df35c fix(hook,sweep,ipc): non-blocking hook pushes, bounded sweep + cache, source-bound resolve
- session-start/session-end no longer run synchronous git + inline push
  inside their self-deadline; a detached child does the push and the hook
  returns immediately (blocked Claude Code startup for minutes on a dirty
  tree before).
- serve sweep drops the unbounded listAllPageRefs, resolves only candidate
  targets, claims corpus files atomically (no double-LLM-spend race), and
  caps the fence LIKE scan; heartbeat writes are O_APPEND with rare compaction.
- hot-memory cache evicts expired entries and bounds entry count (the key is
  caller-controlled via _meta.session_id).
- v1 resolve IPC honors boundSourceId like turn_context; turn-context runs
  its arms concurrently. doctor reads push/heartbeat thresholds from hook.ts.
- new tests: doctor bootstrap checks, hook push-gate + deadline, concurrent
  sweep claims, cache eviction, bound-source resolve.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 10:27:23 -07:00
Garry TanandClaude Fable 5 3cf666e288 fix(ci): regenerate flag registry for new commands, harden shard classifier + release token
- cli-flag-registry.generated.ts regenerated: bootstrap/hook/sweep and
  sources push --message/--allow-unverified-remote were missing, so the
  strict #2185 validator rejected real invocations and skipped the new
  commands entirely.
- EXIT-HANG shard classifier now requires every assigned file to have
  started before warn-passing a watchdog kill (was fail-open).
- release.yml passes TEMPLATE_REPO_PAT via http.extraheader, off the argv.
- compiled-binary e2e fails loud in CI instead of a silent permanent skip.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 10:27:03 -07:00
Garry TanandClaude Fable 5 eacc98b6a3 fix(scan,push): modern OpenAI/Voyage key patterns, scan staged blobs not disk
- secret-scan matches sk-proj-/sk-svcacct-/sk-None- and pa- Voyage keys
  (the bare sk- pattern missed every current OpenAI key format).
- workspacePush stages first, then scans the staged index blobs via
  git cat-file, closing the scan-then-stage TOCTOU where a file changed
  between snapshot and commit shipped unscanned.
- shared binary-sniff helper, memoized glob regexes, atomic push-status
  write, and tests for pull_conflict + gitignored deny-match paths.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 10:27:02 -07:00
Garry Tan c1d7a61a6d Merge remote-tracking branch 'origin/master' into garrytan/codex-as-agent-default-install
# Conflicts:
#	CLAUDE.md
#	CONTRIBUTING.md
#	README.md
#	docs/INSTALL.md
#	docs/TESTING.md
#	docs/architecture/KEY_FILES.md
#	docs/architecture/thin-client.md
#	docs/guides/search-modes.md
#	docs/mcp/CLAUDE_CODE.md
#	docs/mcp/DEPLOY.md
#	llms-full.txt
#	scripts/run-unit-parallel.sh
#	src/cli.ts
2026-08-09 17:50:53 -07:00
Garry TanandClaude Fable 5 989794701e docs(readme): refresh production-brain stats to current brain-repo counts
155,795 pages / 24,589 people / 5,340 companies, counted from the brain
repo's current HEAD; the "100K-page brain" framing moves to 150K to match.
Cron-fleet count unchanged (its store lives on the deployment host, not in
the repos available for verification).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:29:54 -07:00
Garry TanandClaude Fable 5 f94b9ef316 docs: repo-wide accuracy + MECE reform from the 9-bucket markdown audit
A code-grounded audit of every markdown file (root, architecture, guides,
mcp, tutorials, docs-root, operations/eval/designs, skills, recipes) followed
by a fix wave with per-bucket ownership. Four classes of change:

Accuracy — every documented command/flag verified against src/ before writing:
dead commands replaced with working ones (pages purge-deleted, jobs watch
--follow, gbrain restore, import-based Obsidian flow, space-separated --scopes,
real thin-client recipes, working isolation verification, real supervisor
restart procedure, curl-based ngrok health check, real minion_mode setter);
count drift fixed with rot-proof phrasing (100+ ops, 50+ bundled skills via
skills/manifest.json, 140+ engine methods, KNOBS_HASH_VERSION pointer instead
of hardcoded versions); stale claims corrected (search-mode defaults, RETRIEVAL
pipeline order incl. autocut, sentinel rules, refusal-list mechanism, engine
snapshot, shard cap 2400s + EXIT-HANG classifier in TESTING.md, latest-stable +
publish-template documented in RELEASING.md as release.yml promises).

MECE — one home per concept, pointers elsewhere: test isolation → TESTING.md;
OAuth registration + --bind/--public-url lore → DEPLOY.md; mode bundles →
guides/search-modes.md (the home the CLAUDE.md dispatcher always promised);
merge contract → schema-packs.md; WAL ladder → ENGINES.md; quiet-hours →
quiet-hours.md; capture taxonomy → entity-detection.md; person-page taxonomy →
compiled-truth.md; brain-first protocol → brain-first-lookup.md; refresh
semantics → refresh-algorithm.md; KEY_FILES.md deduplicated (58 extension
entries merged, one entry per file); infra-layer.md rewritten as a pointer page.

Privacy — placeholder sweep across guides, docs, skills, and recipes per the
iron rule; per-release narration stripped from reference docs (current-state
prose only).

Bootstrap coverage — AGENTS.md pointer, RESOLVER routing row, INSTALL.md path,
tutorial cross-links, keyless-mode sections in spend-controls/headless-install.

skills.lock.json regenerated; llms.txt/llms-full.txt rebuilt. Gates: verify
36/36, typecheck clean, doctor 96/96, skills-integrity + resolver + build-llms
+ config-set + migrations all green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:22:42 -07:00
Garry TanandClaude Fable 5 64e70c53a0 fix: doc-audit code chasers — broken recovery hints, stale op description, auto_link key
Four small code fixes surfaced by the markdown accuracy audit:
- doctor's auto-RLS recovery hint pointed at `apply-migrations --force-retry 35`,
  which cannot work (--force-retry targets the vX.Y.Z orchestrator registry, not
  the numeric schema MIGRATIONS array). Hint now points at the recreate SQL in
  docs/guides/rls-and-you.md; test pins against regression.
- v0_11_0 migration printed the same broken-mechanism class of hint
  (`config set minion_mode` writes DB config nothing reads); now names
  `apply-migrations --mode` + preferences.json, the real setter.
- submit_job's op description hardcoded a stale handler list; now points at
  registerBuiltinHandlers as the source plus the --follow discovery trick.
- `auto_link` added to KNOWN_CONFIG_KEYS: read by link-extraction, reconcile-links,
  and sweep, and documented as the off-switch in brain-ops/maintain, but the
  allowlist rejected `config set auto_link false`.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:22:24 -07:00
Garry TanandClaude Fable 5 ecc2c378d1 docs(readme): Codex as the recommended first step; OpenClaw/Hermes framed as-intended, high-cost
The install section now routes newcomers explicitly: Codex first
(subscription-priced, nothing to deploy), OpenClaw/Hermes as GBrain used the
way it was designed — always on, at real server + API cost.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 09:49:54 -07:00
Garry TanandClaude Fable 5 4627c7a047 docs(readme): per-harness install sections — Codex, Claude Code, then OpenClaw/Hermes
Each harness gets its own complete paste-block section (desktop app first,
terminal noted — Claude Code CLI is the identical harness; Codex CLI works
pull-based today); the OpenClaw/Hermes platform path keeps equal weight with
its one-click deploys and INSTALL_FOR_AGENTS block intact; memory-only and
remote-connect tiers consolidated under 'Lighter ways in'. Supersedes the
review's D5 ordering by user direction; stale heading references updated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 08:26:08 -07:00
Garry TanandClaude Fable 5 83817561e6 test(bootstrap): quarantine env-mutating suites to the serial lane
check-test-isolation R1: six new files mutate GBRAIN_HOME/env at module scope —
the serial lane (one process per file) is the guard's prescribed home for them.
All 114 tests pass post-rename.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 01:22:45 -07:00
Garry TanandClaude Fable 5 59ba03afcb fix(test): shard cap 2400s — the count-balanced heavy shard needs it under contention
Observed: the heavy shard still progressing 22s before an 1800s kill while
siblings finish at 1150-1550s (split balances file count, not weight). Filed
the load-sensitive WAL-repair flake (pre-existing, master's v0.42.75.0 wave).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 00:46:49 -07:00
Garry TanandClaude Fable 5 ce350250e2 fix(test): classify post-suite exit-hangs as warn-pass; file the leak forensics
A shard killed by the wedge watchdog with every assigned file started and zero
fail markers did all its work and leaked a handle at exit — pre-existing and
master-reproducible (P1 TODO carries the full bisect forensics). Bun's per-test
timeout turns a hung test into a (fail), so the classifier cannot mask one.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 00:11:06 -07:00
Garry TanandClaude Fable 5 12cb232470 test(ci): cache-hash policy — README + runbook edits must invalidate [C2]
The old deny-list assertion predates the paste block; README.md and
BOOTSTRAP_FOR_AGENTS.md are policy-doc re-admissions now, so their edits must
change the hash (a paste-block edit shipping under a cached green was the C2
hole).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 22:25:21 -07:00
Garry TanandClaude Fable 5 bdab11e524 fix(test): shard wedge cap tracks suite growth (1500s -> 1800s)
At ~9000 tests a healthy shard finished at 1466s and two progressing shards
were false-killed at the old cap; 1800s restores ~25% headroom over the
slowest observed healthy shard. Real hangs still hit it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 21:50:43 -07:00
Garry TanandClaude Fable 5 9f28e66bfe fix(bootstrap): register doctor check categories + system-of-record allow comments
The six bootstrap doctor checks join OPS_CHECK_NAMES; the sweep's batch link/
timeline inserts carry the explicit extract-path allow comments (the sweep IS
the extraction path for workspace pages).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 20:49:13 -07:00
Garry TanandClaude Fable 5 5374fc8519 test(bootstrap): e2e pins — hook-under-live-serve, attach, degraded modes, compiled binary, Docker harness
The permanent pins: a real serve holds the PGLite lock while the engine-free
hook completes (and a direct engine open provably throws LiveServeLockError);
stale-socket fail-open; machine-2 attach with marker-keyed hook repair;
decline-everything installs verify green with every degradation named; the
compiled binary renders bundled templates in an empty cwd. Offline Docker
harness (networkless, read-only) gated into heavy-tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 19:29:30 -07:00
Garry TanandClaude Fable 5 ffb4af81a2 docs: KEY_FILES bootstrap cluster + CLAUDE.md dispatcher row (+ build:llms)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 19:08:52 -07:00
Garry TanandClaude Fable 5 15d8107b28 feat(bootstrap): dispatcher, verify, status — the command assembled
gbrain bootstrap {status,interview,render,repo,hooks,verify,uninstall,attach}:
engine-free except verify (owns its engine, in-process sweep — no live-serve
conflict); phase list is the TS source of truth with install.jsonl telemetry
and the support blob; verify's fail-soft check suite covers the real write path
(put_page → write-through file → sweep → graph floor → recall), passes keyless,
persists snapshots, and ends with the first-run tour. cli.ts wired per the
three-touchpoint rule; doctor gains the bootstrap check group (silent on
machines with no bootstrap state). 28 new tests; 353 adjacent green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 19:06:10 -07:00
Garry TanandClaude Fable 5 6ab29f3e31 docs(bootstrap): cross-link the full-agent path from the connection docs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 18:29:24 -07:00
Garry TanandClaude Fable 5 3fb54d0d9e feat(hooks): engine-free gbrain hook command, settings writers, transcript parser
Four hook events (session-start digest + crashed-session recovery push,
user-prompt turn-context injection under an 800ms deadline and the 10KB cap,
stop buffers, session-end corpus write with redaction/retention/dedup +
best-effort push); structural JSON settings merger keyed by a _gbrain marker
(foreign hooks and permissions survive); dated host-spec registry; Claude Code
.jsonl parser as a spec-target with a scrubbed 7-shape fixture. Heartbeat is
counters-only by construction. 59 tests; zero engine modules in the import
graph.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 18:27:24 -07:00
Garry TanandClaude Fable 5 9e92c4b0c5 feat(serve): resident maintenance sweep + keyless capability probe
The lock-owning serve process now closes the persistence loop: startup (3s
post-connect, best-effort, unref'd) and idle (10-min quiet intervals through
the injectable timer seam) sweeps run facts-fence reconciliation, deterministic
link/timeline extraction over recent workspace pages, and spend-gated corpus
ingest (skipped keyless — agent-authored fences cover it). gbrain sweep --once
is the trusted CLI seam bootstrap verify uses. Capability probe renders the
honest keyless/keyed report. Full reuse of the cycle extractor + extract cores;
26 new tests, neighbors green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 18:26:59 -07:00
Garry TanandClaude Fable 5 503e58a9f5 fix(sources): harden/pull gates accept sources inside a parent git repo
The bootstrap workspace registers brain/ (a subdirectory) as the source; the
durability core already resolves the repo root, so the command gates now check
inside-a-repo rather than .git-right-here [CX2-3].

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 18:07:03 -07:00
Garry TanandClaude Fable 5 0004f280c3 feat(persistence): secret-scan, gbrain sources push, durability unification
Pattern secret scanner (own runtime allowlist, redacted previews, corpus-write
redaction mode); sources push runs the whole scan→stage→commit→pull→push
sequence under one cross-platform lock (mkdir-atomic, pid+age+token) with a
deny-glob backstop, commit-first divergence-safe pull, refuse-unverifiable
visibility, and push-status telemetry; gbrain-home choke point unifies
GBRAIN_HOME semantics with config (0700); durability is parent-repo-aware and
rotates its push log at 0600. 35 new tests; 200 existing green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 18:05:28 -07:00
Garry TanandClaude Fable 5 3f64a6d467 feat(context): IPC v2 turn_context + 8KB assembly + visibility resolver + session identity
Discriminated-union IPC with handler map, protocol echo (stale-serve detection),
shared-secret gate, server-side source binding, per-kind budgets; turn-context
assembly (reflex pointers + volunteered pages + world-only hot facts) under a
data-not-instructions envelope trimmed to the harness's 10KB hook-output cap;
facts.default_visibility resolved through one helper at all four sites (explicit
caller wins, typos fail closed); typed sessionId threads _meta.session_id into
the hot-memory cache key. 50 new tests; 180 adjacent tests confirmed green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 18:02:27 -07:00
Garry TanandClaude Fable 5 1129fa062c feat(release): latest-stable ref, template-repo publish job, bootstrap CI guards
release.yml advances the latest-stable tag only after assets publish (the paste
block's permanent ref — copies in the wild never rot) and gains a PAT-gated
publish-template job verified against the vendored tree; two skip-graceful
guards (sanctioned-ref + runbook stamp; template/token bijection + placeholder
assertion + generator byte-diff) wired into verify; README + runbook re-admitted
to the CI cache hash; vendored deterministic template tree generated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 18:00:34 -07:00
Garry TanandClaude Fable 5 75e30015a7 feat(bootstrap): private-repo lifecycle — repo create, attach, uninstall, run lock
gh-gated private repo creation with API-verified privacy (rate-limit distinct
from public), refuse-foreign-origin with attach as the sanctioned path, atomic
bootstrap mutex (pid liveness + age + token), receipt-keyed uninstall that
never wholesale-deletes the gbrain home and only offers --delete-brain for a
brain it created; read-only PGLite lock probe (never opens the engine).
54 unit tests, injectable exec seam throughout.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 17:58:40 -07:00
Garry TanandClaude Fable 5 b66bd8d155 feat(bootstrap): interview + render engines
Interview gate with read-back confirm-hash (any later answer change clears the
confirmation — the hostile single-batch case is structurally impossible),
per-answer provenance, caps + escaping at set time, config-sink routing for the
provider key; renderer with hard-fail token sweep, subordinate fencing of
principal input, never-clobber + backups, deterministic minimal mode for the
template repo, scaled byte floors. 58 unit tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 17:55:49 -07:00
Garry TanandClaude Fable 5 59a6ce091a docs(designs): spike instrument for the bootstrap wave (build order 0)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 17:46:21 -07:00
Garry TanandClaude Fable 5 1b66e911f9 feat(bootstrap): runbook, README paste block, bootstrap guide, TODOS entries
BOOTSTRAP_FOR_AGENTS.md (agent-driven install runbook: CLI phase list is the
source of truth, never-invent rules, Codex approvals preflight, keyless posture,
failure-modes table, version stamp for the skew check); README gains the
full-agent paste block pinned to latest-stable inside the Claude Code/Codex
quick start (memory-only tier stays); docs/guides/bootstrap.md carries the full
install/security/consent/degradation/uninstall contract.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 17:45:41 -07:00
Garry TanandClaude Fable 5 619d502a04 feat(bootstrap): format spec, question bank, identity templates, bundled assets
agent.json manifest (format_version 1, initialized sentinel) + machine-local
install receipt [CX2-1, CX2-12]; 12-question/6-required interview bank with
consent keys and a persist:false sink for the optional provider key [CX2-13];
ten {{TOKEN}} identity templates (generic, adapted to gbrain ops — gates call
recall/query/put_page, write-through-ops rule, keyless agent-authored facts,
silence contract); assets embedded compiled-binary-safe via file-type imports
[ENG-6].

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 17:39:42 -07:00
Garry TanandClaude Fable 5 75e708b745 docs(designs): agent-bootstrap plan + design docs (normative, review-absorbed)
The scrubbed, in-repo sources of truth for the gbrain bootstrap wave:
AGENT_BOOTSTRAP_DESIGN.md (product scope/sequencing) and
AGENT_BOOTSTRAP_PLAN.md (implementation; all review-finding IDs inlined).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 17:34:15 -07:00
47 changed files with 3582 additions and 255 deletions
+1 -1
View File
@@ -1,4 +1,4 @@
<!-- gbrain-runbook-stamp: 0.45.3.0 -->
<!-- gbrain-runbook-stamp: 0.45.5.0 -->
<!-- This stamp must equal the VERSION file at every release; CI enforces it
(scripts/check-bootstrap-tag.sh). `gbrain bootstrap status` compares it to
the installed binary and warns on skew. -->
+106
View File
@@ -2,6 +2,112 @@
All notable changes to GBrain will be documented in this file.
## [0.45.5.0] - 2026-08-12
Brain currency, part one: a brain is only useful if it's CURRENT, and until now
the machinery keeping it current could die without anyone noticing. This release
makes autopilot's health honest end-to-end — status that reads the heartbeat,
a daemon that takes itself out of rotation when its repo vanishes, migrations
that pause it instead of racing it, and staleness reporting that can no longer
say "fresh" forever.
### Fixed
- **A dead autopilot can no longer report healthy.** `gbrain autopilot --status`
now reads the daemon's heartbeat instead of checking that install artifacts
exist, and gains real exit codes for cron and CI gates: 0 fresh (or nothing
installed), 1 needs attention (stale heartbeat, never ran, or paused), 2 the
daemon took itself out of rotation. Status runs without touching the
database, so it keeps working during the exact outages it exists to
diagnose. Staleness tolerance scales with the tick interval and accounts
for the adaptive scheduler's longer healthy-brain sleeps, and a garbage
interval value can no longer silence the alarm.
- **Content-relative staleness now has a wall-clock ceiling.** A source whose
content stopped moving (or whose local clone vanished) previously reported
fresh forever off the stored content timestamp. `sync_freshness`,
`federation_health`, and `gbrain status` now ramp toward stale past a
ceiling (default 72h; `GBRAIN_STALENESS_CEILING_HOURS` to tune) — ramping,
not stepping, so the warn tier still fires before the fail tier instead of
both alarms tripping at once.
- **The documented agent-scheduler chain works on keyless brains.**
`gbrain sync --repo <path> && gbrain embed --stale` used to exit 1 on every
brain installed without an embedding key, breaking the always-current cron
for external agent schedulers. A bare stale embed now refuses cleanly
(exit 0, stderr hint); explicit embed requests (a slug, a slugs list, the
all flag) still exit 1.
- **Engine migrations and the autopilot daemon no longer race.**
`gbrain migrate --to <engine>` claims a cooperative pause marker before
touching the target — the marker doubles as a migration mutex, so a second
concurrent migrate refuses to run instead of corrupting the first one's
resume state, and a marker it cannot write refuses the migration outright
rather than running unfenced. Background job workers stop picking up new
work while the marker is parked. It then waits for in-flight
sync/embed/cycle work and running jobs to actually drain (watching the DB
lock table, capped by `GBRAIN_MIGRATE_QUIESCE_SECONDS`) instead of
sleeping a blind grace period.
The marker is released even when the migration fails or is killed: cleanup
registers the moment the claim lands, adoption of a dead run's orphan is
pid-liveness-checked (a live migrate's marker is never stolen), and the
daemon itself clears an orphan whose owning process died. After a clean
flip the daemon detects the engine change on its next tick and relaunches
onto the new engine — previously it kept syncing into the abandoned source
engine until its process happened to restart — and the migration warns if
an exported connection-string env var would override the new config.
- **Self-disable requires three consecutive misses.** A repo on an external
or cloud-synced volume that is briefly absent at login no longer
permanently takes the daemon out of rotation; one successful probe resets
the strike counter.
- **A cron'd status monitor no longer reads as an install.** Machines whose
only crontab reference is the recommended health-gate line stop reporting
"installed but never ran".
- **Malformed connection URLs stop the daemon immediately** with a clear
config verdict instead of spending the whole reconnect budget retrying a
value only the operator can fix.
- **Sync no longer silently drops git typechange and unmerged statuses.**
Replacing an indexed file's content in a way git reports as `T` or `U`
now imports as a modification instead of never reaching the index; a
copy status imports its destination path.
- **A wedged sync can no longer read as "in progress" forever.** A sync
lock holder that keeps heartbeating past the staleness ceiling without
finishing now fails `gbrain doctor`'s freshness check, naming the holder
and the exact `gbrain sync --break-lock --source <id>` remedy.
### Added
- **Autopilot self-disable guard.** The generated wrapper now stops the daemon
for real when its `--repo` path vanishes: it writes an explanatory marker,
then boots the job out of the supervisor (`launchctl bootout` on macOS,
`systemctl --user disable --now` on systemd) — a bare `exit 0` under
KeepAlive/Restart=always is just a quieter respawn loop. `--status` explains
why it stopped; a reinstall against a restored path clears the marker;
`--uninstall` clears it too.
- **`paused` status state.** A daemon parked by a migration (or by an orphaned
pause marker) now reports `paused` with exit 1 and the marker path, instead
of "running" off its still-fresh heartbeat.
- **Harness e2e tier.** A real-launchd lifecycle test on macOS (install →
load → self-disable → status, under a per-run unique label) plus a
shimmed-supervisor lifecycle that runs on every platform, and an
agent-scheduler contract test that drives the documented sync-and-embed
shell chain end-to-end against a keyless brain — including the
pull-failure case that must break the chain.
- **Honest staleness numbers in `gbrain status`.** Source rows now carry
`hours_since_last_sync` (raw wall-clock truth) alongside the
threshold-relative `staleness_hours` that drives the fresh/stale/severe
class, so the escalation ordering and the human-facing number stop being
the same field.
- **Shared numeric env resolver.** The doctor and staleness-threshold
`GBRAIN_*` numeric env vars now resolve through one warn-once helper
(`src/core/env-number.ts`), so a typo'd value falls back loudly exactly
once instead of NaN-ing a threshold silently.
### To take advantage of v0.45.5.0
- `gbrain upgrade`, then wire your scheduler's health gate to
`gbrain autopilot --status` — the exit code is now trustworthy.
- If autopilot is installed, reinstall once (`gbrain autopilot --install
--repo <path>`) so the generated wrapper picks up the self-disable guard.
- Keyless installs: your sync-and-embed cron chain now exits 0; no action
needed beyond upgrading.
## [0.45.3.0] - 2026-08-12
**Codex installs stop asking a question Codex can't honor.** The bootstrap used to offer every install a choice of MCP scope — this folder only, or the whole machine — but Codex has no per-folder registrations, so picking "this folder" led to a confusing round-trip where the agent asked permission to keep what it had already done. Now each harness gets the honest version: Claude Code records your scope choice during the interview (where it actually sticks), and Codex simply tells you the truth — its registration reaches the whole machine, read and write — along with the exact commands to remove it (just the registration, or the whole install).
+7 -1
View File
@@ -251,7 +251,13 @@ Set up using your platform's scheduler (OpenClaw cron, Railway cron, crontab), o
platform glue entirely with `gbrain autopilot --install` (built-in self-maintaining daemon):
- **Live sync** (every 15 min): `gbrain sync --repo ~/brain && gbrain embed --stale`
— or `gbrain sync --watch` for a continuous loop.
— or `gbrain sync --watch` for a continuous loop. Safe on keyless brains:
a bare `gbrain embed --stale` exits 0 with a stderr note when embeddings
are disabled, so the chain doesn't break.
- **Health gate** (daily): `gbrain autopilot --status` — exit 0 fresh (or
nothing installed), 1 needs attention (stale heartbeat, never ran, or
paused), 2 the daemon took itself out of rotation. Filesystem-only, so it
works during DB outages.
- **Auto-update** (daily): `gbrain check-update --json` (tell user, never auto-install).
- **Dream cycle** (nightly): `gbrain dream` runs the 8-phase overnight maintenance cycle.
Entity sweep, citation fixes, memory consolidation, plus (v0.23+) overnight conversation
+60
View File
@@ -1,5 +1,65 @@
# TODOS
## Brain-currency harness-e2e follow-ups (filed with the PR-A wave)
- [ ] **P1 — Extend engine-identity convergence to the other long-lived planes.**
The autopilot daemon now detects a post-migration engine flip
(`autopilotEngineIdentity` per-tick compare → clean exit for supervisor
relaunch), but `gbrain serve` (MCP) and a standalone `gbrain jobs work`
worker hold their engine handle indefinitely and keep writing into the
abandoned source engine after a flip — the same silent-divergence class,
still open on those planes (adversarial-review catch). Fix shape: the same
boot-identity compare in their main loops.
- [ ] **P2 — DB-visible pause for cross-host workers.** The pause marker now
fences local job pickup (pre-claim check + post-claim release-back in
`src/core/minions/worker.ts`), but the marker is a local file: a worker on
ANOTHER host or container pointed at the same Postgres brain never sees it
and keeps claiming jobs during a migration copy (its in-flight work IS
visible to the drain via `minion_jobs`/lock rows; new claims are the gap).
Fix shape: a row in a control table (or a pause flag in `gbrain_cycle_locks`)
that the claim query itself honors — atomic with claiming, visible
cluster-wide.
- [ ] **P2 — Route file→symlink typechanges to delete.** `buildSyncManifest`
maps git status `T` to modified, but import-file deliberately SKIPS symlinks
(the exfil guard), so replacing an indexed file with a symlink leaves the
old content indexed forever with no delete. Fix shape: when the post-change
path is a symlink, emit a delete instead of a modify.
- [ ] **P3 — Surface daemon-internal degradation in status.** A daemon stuck
in the reconnect-retry loop (crash-classified errors) keeps heartbeating,
so `--status` reads fresh while zero work happens. Fix shape: a breadcrumb
file with consecutive-failure count that showStatus reads.
- [ ] **P3 — Extract a shared `seedBrain` test helper.** The keyless-PGLite +
tmp-HOME + shimmed-PATH setup is duplicated between
`test/autopilot-launchd-lifecycle.serial.test.ts` and
`test/agent-scheduler-contract.serial.test.ts` (review-army maintainability
finding). A third harness-e2e file (the PR-B tier) should force the
extraction into `test/helpers/`; don't extract before then — two instances
is a coincidence, three is a pattern.
- [ ] **P3 — Name the quiesce protocol's magic numbers.** `migrate-engine.ts`
and `autopilot.ts` share three constants by value, not by name: the 600s
heartbeat-freshness window, the 35s default grace, and the daemon's paused
fast-poll interval. Hoist into `src/core/autopilot-paths.ts` (the shared
leaf) as named exports so the two planes can't drift.
- [ ] **P3 — Migration manifest rows don't carry content_hash.** A resume
trusts `(source_id, slug)` membership in `completed_slugs`; a page edited
BETWEEN the failed run and the resume is skipped with its stale copy left on
the target (review-army data-migration finding; pre-existing design, not a
regression). Fix shape: stamp `content_hash` per completed entry and re-copy
on mismatch during resume.
- [ ] **P2 — Keyless `gbrain dream` contract test.** The documented nightly cron
(INSTALL_FOR_AGENTS.md Step 7) runs `gbrain dream` unconditionally, and the cycle's
embed phase hits the same `EmbeddingDisabledError` class that broke the documented
sync-and-embed chain on keyless brains (fixed in `runEmbed` for the `--stale`
spelling; `test/agent-scheduler-contract.serial.test.ts` pins it). Nobody has verified that a
full keyless dream exits 0 — if any phase surfaces the disabled-embeddings error as a
phase failure, the documented nightly cron is broken identically for every
`init --no-embedding` install. **Where to start:** `src/core/cycle.ts` embed phase +
`src/commands/dream.ts` exit-code handling; test shape mirrors
`test/agent-scheduler-contract.serial.test.ts` (keyless PGLite brain, real CLI spawn, exit-code
assertion). Surfaced by the harness-e2e outside-voice review.
## BrainBench follow-ups (filed v0.44.0.0, Cathedral 2)
Deferred from the BrainBench wave (eng-reviewed; plan + GSTACK REVIEW REPORT at
+1 -1
View File
@@ -1 +1 @@
0.45.3.0
0.45.5.0
+6 -1
View File
@@ -18,4 +18,9 @@ timeout = 60_000
# runs, so audit-emitting code paths (content-sanity, shell-audit, etc.)
# can't leak fixture events into the operator's real ~/.gbrain/audit/. See
# test/helpers/audit-dir-preload.ts for the full rationale.
preload = ["./test/helpers/legacy-embedding-preload.ts", "./test/helpers/audit-dir-preload.ts"]
#
# Same treatment for the sync failure ledger: broken-fixture import/sync tests
# were appending rows into the operator's real ~/.gbrain/sync-failures.jsonl,
# which `gbrain doctor` reads and warns on. See
# test/helpers/sync-failures-preload.ts.
preload = ["./test/helpers/legacy-embedding-preload.ts", "./test/helpers/audit-dir-preload.ts", "./test/helpers/sync-failures-preload.ts"]
+19
View File
@@ -177,6 +177,25 @@ live in `test/postgres-engine-rls-scope.test.ts`.
**Migration:** `gbrain migrate --to supabase` exports everything (pages, chunks, embeddings, links, tags, timeline) and imports into Supabase. `gbrain migrate --to pglite` goes the other direction. Bidirectional, lossless.
The migration and the autopilot daemon do not race: `migrate --to` claims a
cooperative pause marker before touching the target. The marker doubles as a
migration mutex — a second concurrent migrate refuses to run, and a marker
that cannot be written refuses the migration outright. Background job workers
stop picking up new work while it is parked, and the migration waits for
in-flight sync/embed/cycle work and running jobs to actually drain (watching
the DB lock table, capped by `GBRAIN_MIGRATE_QUIESCE_SECONDS` — default 300;
`0` skips the wait). Cleanup registers the moment the claim lands, so the
marker is released on failure and on catchable signals; a marker orphaned by
an uncleanly killed run is adopted by a later migrate only after a
pid-liveness check (a live migrate's marker is never stolen), and the daemon
clears an orphan whose owning process died on its next poll. `gbrain
autopilot --status` reports `paused` (exit 1) while the marker is parked and
prints the marker path; on a host with no daemon running to self-heal,
remove an orphan by hand only after confirming the pid it names is dead.
After a clean flip the daemon detects the engine change on its next
tick and relaunches onto the new engine, and the migration warns if an
exported connection-string env var would override the new config.
### Troubleshooting: startup abort (`RuntimeError: Aborted()`)
**Symptom:** every PGLite-touching command dies at startup with
+2
View File
@@ -175,6 +175,8 @@ Unit tests and what they cover:
- `test/volunteer-context.test.ts` — push-based context core (#2095), hermetic in-memory PGLite: `parseWindow` lenient `user:`/`assistant:` parsing, multi-turn window extraction, confidence-gated volunteering (arm confidences, multi-turn/newest-turn boosts, `min_confidence` gate, max-pages cap), slug-only suppression, privacy (rationales are deterministic templates; synopses pass the takes/facts fence), and the approximate usage-stats join.
- `test/watch-command.test.ts``gbrain watch` push transport (#2095): streaming loop, rolling window, session dedupe, `--json` JSONL shape, `channel: 'watch'` event logging, clean EOF return. Hermetic PGLite + injected line/write deps (no subprocess, no real stdin).
- `test/watch-sigint.serial.test.ts``gbrain watch` SIGINT lifecycle against a real spawned CLI subprocess with a tmpdir brain. SERIAL: parallel unit shards flake on concurrent subprocess spawns (same rationale as `apply-migrations-pglite-spawn.serial.test.ts`).
- `test/autopilot-launchd-lifecycle.serial.test.ts` — autopilot lifecycle behavior, not generated-string assertions: the full install → self-disable → status → reinstall → uninstall arc with `launchctl` replaced by an argv recorder and the generated wrapper executed by a REAL bash against a genuinely deleted repo (every platform), plus a darwin-only fail-SKIP describe against the real launchd under a per-run unique label (`GBRAIN_AUTOPILOT_LABEL`) so it can never collide with — or tear down — a real install on the host. Serial: spawns subprocesses and pins HOME/GBRAIN_HOME for the whole file.
- `test/agent-scheduler-contract.serial.test.ts` — the documented external agent-scheduler shell chain (`gbrain sync --repo X && gbrain embed --stale`, live-sync.md / INSTALL_FOR_AGENTS.md Step 7) driven end-to-end through a real `/bin/sh` against a keyless PGLite brain: the `&&` short-circuit IS the contract (argv arrays can't exercise it), the keyless bare stale embed exits 0, and the pull-failure case that must break the chain does. Anti-vacuity: the fixture commits a real page and every read-back asserts pages >= 1. Serial: real spawned CLI + tmpdir HOME.
- `test/cli-format-volunteer.test.ts``formatResult`'s `volunteer_context` human rendering: pointer lines with confidence/arm/rationale, the empty-result message, the approximate stats summary.
- `test/config.test.ts` — config redaction.
- `test/files.test.ts` — MIME/hash.
File diff suppressed because one or more lines are too long
+406
View File
@@ -0,0 +1,406 @@
# Brain currency — fix the incident, then build the ladder
Generated by /plan-ceo-review on 2026-08-10
Rev 3, after two adversarial spec-review rounds (6/10 → 7/10) and an independent outside voice.
Branch: garrytan/gbrain-commit-indexing | Mode: SELECTIVE EXPANSION
Repo: garrytan/gbrain
**Citation convention:** repo-relative paths. `src/core/sync.ts` (540 lines) and
`src/commands/sync.ts` (5804 lines) are different files; both are cited.
## Origin
An investigation into "how does gbrain pick up new commits from GitHub" found it never
talks to GitHub. It diffs `git diff last_commit..HEAD` against a **local checkout**
(`src/core/sync-delta.ts:113`). Getting remote commits into that checkout is a separate,
opt-in concern.
It then found worse: on the founder's machine `gbrain autopilot` was installed, died
2026-05-31, and stayed dead **71 days** while three surfaces reported healthy.
**1. `autopilot --status` is an artifact-presence check.**
`src/commands/autopilot.ts:1775-1786` — plist `existsSync` on darwin, crontab grep
elsewhere. Never asks whether the job is loaded, the process alive, the baked `--repo`
present, or the log fresh. Always exits 0.
**2. `doctor`'s `sync_freshness` computes the 71-day number and throws it away.**
`src/core/source-health.ts:182-194`:
```ts
const wallClockSeconds = Math.floor((nowMs - lastSyncMs) / 1000); // ← the 71 days
if (wallClockSeconds < 0) return wallClockSeconds;
if (contentMs !== null && Number.isFinite(contentMs)) {
return contentMs <= lastSyncMs ? 0 : wallClockSeconds; // ← discarded
}
```
When the clone is unreachable, `src/commands/doctor.ts:4306-4344` routes the verdict here.
The function measures *drain completeness*, not *staleness*. "We caught up when we last
looked" and "we have not looked in 71 days" both return 0.
**3. `gbrain status` inherits it.** `src/commands/sync.ts:5440-5453``'fresh'` beside a
71-day-old date, exit 0. (`gbrain sources status` does report the real lag in its LAG
column, but has no warn line for it and no exit contract.)
**Root cause of the death:** `src/commands/migrate-engine.ts` (22,733 bytes) contains
**zero** autopilot references. The Supabase-to-PGLite migration rewrote
`~/.gbrain/config.json` while a daemon built on the old config kept running and died on
`config.database_url`.
## The key insight the reviews converged on
The content comparison in #2 is not a bug someone forgot. `src/commands/doctor.ts:4288-4305`
documents why it exists:
> a container restart wipes `local_path` ... **and since a no-op sync doesn't advance
> `last_sync_at`**, every QUIET source read as stale/FAIL after a restart (score-sinking
> alert storm; observed live: 16-source brain, 12 clones gone after a config-update
> restart, doctor 70→30).
**The premise in bold was invalidated after that code was written.** v0.42.52.0 added a
heartbeat at `src/commands/sync.ts:2287-2298`:
```ts
// bump last_sync_at as a heartbeat on every successful 0-changes sync...
if (opts.sourceId) {
await engine.executeRaw(`UPDATE sources SET last_sync_at = now() WHERE id = $1`, [opts.sourceId]);
}
```
A no-op sync **does** advance `last_sync_at` now. So a quiet source that is being checked
has a recent `last_sync_at` and survives a wall-clock ceiling; the 71-day case has an old
one because **no sync ran at all**. The two cases are now distinguishable, and the
fallback's justification has expired.
That is the whole incident: a wall-clock ceiling on the discard branch, in one pure
function that `doctor`, `gbrain status`, and `sources status` all call. It fixes all three
by construction, with no new table, no new command, and no migration.
It also means **the heartbeat this plan originally proposed to build already ships.** A
separate `live_ticks` table would be a fourth status surface on a fifth data source,
curing "three surfaces disagreed" by adding one more that can disagree.
## Base branch
The whole wave (PR-A, PR-B, PR-C) is based on
`garrytan/codex-as-agent-default-install`, not `master`. That branch carries the
bootstrap surface (`src/core/bootstrap/{host-specs,hooks}.ts`, `detectHarness()`) that
PR-B's harness tier needs, so **PR-B is not blocked** — an earlier revision of this doc
assumed it was.
That branch moves frequently; re-fetch before comparing anything against it. A stale
remote-tracking ref is an easy way to reach a confidently wrong conclusion here.
## Sequencing (decided)
Three PRs. Nothing is cut; the order changed.
### PR-A — close the incident (ships first)
1. **Wall-clock ceiling** in `lagFromContentMs` (`src/core/source-health.ts:189`): return
`wallClockSeconds` once it exceeds an absolute bound regardless of the content
comparison. Bound is a named env knob per repo convention
(`GBRAIN_STALENESS_CEILING_HOURS`, default 72, matching the existing
`GBRAIN_SYNC_FRESHNESS_FAIL_HOURS`).
2. **Regression test** (acceptance criterion 1 below).
3. **E3**`src/commands/migrate-engine.ts` reconciles the running daemon.
4. **Wrapper self-disable**`src/commands/autopilot.ts:1314-1359`. Adapted from
`src/core/brain-repo-durability.ts:509-512`, NOT copied: two corrections the
engineering review established.
- Predicate is `[ ! -d "$repo" ]`, not `[ ! -d "$repo/.git" ]`. `--repo` may be a
subdirectory of the checkout (sync resolves the root itself by walking up), and
`.git` is a FILE in worktrees and submodules — either shape would self-disable a
healthy install.
- `exit 0` is correct for the durability wrapper because launchd fires it on
`StartInterval` (one shot). Autopilot runs under `KeepAlive=true` +
`ThrottleInterval=60` and systemd `Restart=always`, where exiting disables nothing
and instead produces a silent respawn-every-60s loop. The wrapper must
`launchctl bootout` / `systemctl --user disable --now` itself on those targets and
drop a marker that `--status` surfaces.
5. **Reconnect classifier**`src/commands/autopilot.ts:58-78`; a JS `TypeError` must not
substring-match as a config verdict.
6. **`autopilot --status` reads the heartbeat** instead of `existsSync`, and exits nonzero
when stale.
7. **E8 hygiene** — test-run pollution of `~/.gbrain/sync-failures.jsonl`;
`buildSyncManifest` (`src/core/sync.ts:105-140`) dropping git **`T`** (typechange).
Narrowed: `src/core/sync-delta.ts:130` passes `-M` only, so `C` is unreachable without
`--find-copies` and `U` needs a conflicted worktree. `C`/`U` handled defensively.
**Not in PR-A:** the lockfile-leak fix. Removing the leaked `~/.gbrain/autopilot.lock`
deletes the signal that distinguishes *crashed* from *never installed*
(`src/commands/status.ts:595-598`) before its replacement exists. It lands in PR-C
alongside `live status`.
### PR-B — the `harness` tier alone
The harness tier is the only tier the modal gbrain user can actually run (PGLite default,
desktop harness, behind NAT), so it ships alone and early rather than buried inside the
cathedral. Reuses `src/core/bootstrap/{host-specs,hooks}.ts`, which the base branch
already provides.
### PR-C — the ladder
`live` command family, `live.mode` bundle, shape detection, `cron`/`daemon`/`webhook`
tiers, advisor collector, `init` offer, watch tier, shared `os-scheduler.ts`, and the
lockfile-leak fix. **`live_ticks` is re-examined here against the shipped
`last_sync_at` heartbeat rather than assumed** — the burden is on the new table to justify
itself.
## The constraint being satisfied (quoted so it can be checked)
`docs/designs/AGENT_BOOTSTRAP_PLAN.md` on `origin/garrytan/codex-as-agent-default-install`,
decision **D9**:
> **D9 Scheduling: almost nothing on by default.** ON: SessionEnd push (event-driven, no
> daemon). OPT-IN: 15-min harden cron. **Autopilot NOT default on PGLite** (verified: its
> sync/embed children would contend with every live serve for the single-writer lock, and
> nothing handles `LiveServeLockError` politely today) — **recommended on Postgres**; any
> future scheduled job must treat lock-held as skip-silently-and-log.
*Reconciliation:* D9 says "15-min harden cron"; the shipped default is **1800s / 30 min**
(`src/core/brain-repo-durability.ts:76`, `:659`). D9's figure is stale. This plan uses 30.
This plan's decisions are labelled **L1..L14** to avoid collision with that document.
## PR-C design (carried forward, not yet committed to a diff)
### Tiers — five active plus `off`
| tier | mechanism | expected cadence | engine gate |
|---|---|---|---|
| `off` | nothing | n/a — `live status` exits **0** | — |
| `harness` | agent hook / session boundary | event-driven, **age-exempt** | any (incl. Windows, containers) |
| `webhook` | HMAC push from GitHub | event-driven, **age-exempt**; paired keepalive `cron` supplies the age signal | any + reachable `serve --http` |
| `cron` | OS scheduler | declared `expected_cadence_seconds` | any; **PGLite floor 1800s + lock-aware skip** |
| `daemon` | resident autopilot, `runCycle` | 300s | **Postgres only** (D9) |
| `watch` | daemon + chokidar | **floor 300s for freshness purposes**, not the ~1s event latency | **Postgres only** (D9) |
Event-driven tiers are exempt from age-based failure; a webhook repo with no pushes for
three days is healthy, not failed. `watch`'s freshness cadence is decoupled from its event
latency so a GC pause is not a FAIL.
`off` is a first-class bundle member with `enabled: false`, copied from
`src/core/pace-mode.ts:65-71`.
### L1 — Shape detection predicate
| Signal | Source | Meaning |
|---|---|---|
| engine | `config.engine` | `postgres` required for `daemon`/`watch` |
| interactive desktop harness | `CLAUDECODE`, `CLAUDE_CODE_ENTRYPOINT`, `CODEX_HOME`, `CODEX_SANDBOX`, `CODEX_CI` (**env only**) | any present → cap at `harness` |
| long-lived host | `detectInstallTarget()` ∈ {`macos`, `linux-systemd`, `ephemeral-container`+injection point} | a reboot-surviving scheduler exists |
| server posture | `serve --http` configured, or `minion_mode != 'off'` | corroborating, never sufficient alone |
`macos` is in the long-lived row deliberately: `detectInstallTarget()` returns `'macos'`
unconditionally on darwin (`src/commands/autopilot.ts:1277`), and darwin is the platform
of the origin incident. Omitting it would make the incident host permanently
shape-ineligible.
**No filesystem probes for harness identity.** The `~/.claude/hooks/...` class of probe
(`src/commands/autopilot.ts:1304`) is what false-positives today. Env vars only.
Any inconclusive read falls to `harness`, never `daemon`.
### L2 — `live status` exit codes
| Condition | Exit |
|---|---|
| fresh, or `live.mode == off` | 0 |
| PGLite lock held by a live `serve` (`blocked_by_serve`) | 0 |
| tier enabled + heartbeat missing or stale | 1 |
| drifted install, or DB **connect failure** | 2 |
`live.mode == off` exiting 0 is load-bearing: otherwise every fresh install exits nonzero,
which is the `cycle_freshness` #2540 lesson (never-configured must not turn the surface
red). And lock-held is **not** an outage: `src/core/pglite-engine.ts:444` acquires the file
lock on every `connect()` and throws if it fails, so on the default engine with a resident
`serve`, treating that as exit 2 would make FAIL the steady state.
### L3 — `skipped_locked` semantics
A tick that cannot acquire the PGLite lock **does not satisfy freshness and does not
degrade it**. It is neutral: logged, not recorded as work-done, and not counted toward
staleness for a grace window of 3 consecutive skips, after which the surface reports
`blocked_by_serve` with the remediation inline. Treating it as work-done rebuilds the
71-day false-green; treating it as failure makes the default engine permanently red.
### L4 — Scheduler ownership
Ownership lives in a sidecar `~/.gbrain/live-ownership.json`, **not** in an entry comment.
On darwin both harden and autopilot install launchd **plists** (files, not comment-bearing
crontab lines), so the `# gbrain:autopilot v0.11.0` marker convention does not generalize.
The sidecar covers all install targets uniformly.
Three enumerated cases:
1. **Harden cron exists + pull opted in** → rewrite through `os-scheduler.ts`,
`ownership=live-adopted`.
2. **Harden cron exists + pull declined** → leave it entirely alone; install a separately
labelled `live` entry. **This is the default and lands first**, so PR-C's `live on`
never meets an existing harden cron without a rule.
3. **Neither exists** → install a `live` entry, `ownership=live`.
`live off` removes only entries `live` created and reverts adopted ones to harden.
Pre-existing `gbrain autopilot` installs are **migrated, not orphaned**: first `live
status` after upgrade reports `tier: daemon (legacy autopilot)` and offers one-time
adoption.
### L5 — Op scopes
| Op | scope | localOnly | remote |
|---|---|---|---|
| `live_status` | `read` | no | allowed; omits `local_path`, scheduler artifact paths, and log tail |
| `live_tick` | `write` | **yes** | reject |
| `live_on` / `live_off` | `admin` | **yes** | reject |
| `live_self_heal` | `admin` | **yes** | reject |
Self-heal walks a **DB-supplied** `local_path` and then writes a scheduler entry.
`src/commands/doctor.ts` already gates its git short-circuit on `localOnly === true`
(*"a remote-callable code path must NOT walk DB-supplied `local_path` values with
subprocess calls"*). Self-heal honors that and additionally requires a realpath match
against the anchor via `isAnchorOwnedSyncPath` (`src/commands/sync.ts:1296`).
**Bootstrap paradox, acknowledged:** if the broken thing is the scheduler entry, a
scheduled self-heal never runs. Non-scheduled triggers are the `harness` tier (PR-B) and
an explicit `gbrain live doctor`. PR-C ships self-heal with both, not with a scheduled
trigger alone.
### L6 — Revert
A code revert leaves plists, crontab lines, systemd units, and (E1) a GitHub webhook
installed and unowned. Therefore:
- **Revert requires `gbrain live off` first** on any enabled host. Stated in the PR body.
- The generated wrapper self-disables on a **marker file** written by `live on` and removed
by `live off`. Not a `gbrain live --help` probe: that adds a process spawn per tick and
assumes an exit code the CLI does not guarantee.
- The migration, if `live_ticks` survives PR-C's re-examination, is additive and uses the
**next free version at implementation time** (125 is the current max; two waves may land
first).
### L7 — E5 must not use `nag-state.ts`
`src/core/skillpack/nag-state.ts` is skillpack-scoped (schema `gbrain-skillpack-nag-v1`,
entries keyed on `pack_version`, `DEFAULT_NAG_CEILING = 3`, suppressed thereafter). Wiring
a dead-sync alarm through it means a genuinely broken brain goes silent after three
notices, which is a suppression mechanism for the exact failure mode whose defining
property was 71 days of silence.
E5 instead uses a **rate limit, not a ceiling**: at most once per session, never
suppressed permanently, escalating in terseness rather than disappearing.
### L8 — E1 webhook dependencies (previously unpriced)
Creating a GitHub webhook programmatically needs an `admin:repo_hook` token. No
acquisition, storage, scope, or rotation story existed. Therefore E1 ships in **manual
mode only**: `live on --tier webhook` generates the secret, resolves and prints the payload
URL, and the user pastes it into GitHub, matching what `gbrain sources webhook set`
(`src/commands/sources.ts:909-916`) already does. No token, no remote hook creation, no
`live off` remote deletion problem.
The "verified test ping" must originate **from GitHub**, not locally. A local ping proves
nothing through NAT and would be an artifact-presence check, the precise anti-pattern in
the Origin section.
### L9 — `live_ticks` retention
If the table survives PR-C, the sweep runs **inside `live tick`** (bounded best-effort
DELETE on a TTL), not only in the cycle's `purge` phase. `purge` is a `runCycle` phase
(`src/core/cycle.ts:1434`), and `runCycle` runs only on `daemon`/`watch` — the `cron`,
`webhook`, and `harness` tiers would accumulate forever.
## Scope decisions (all accepted; PR assignment added)
| # | Item | PR | Note |
|---|---|---|---|
| L10 | Approach C: full ladder | A/B/C | user chose the cathedral; resequenced, not cut |
| L11 | Tier default keys on deployment shape, not vendor | C | Hermes has zero detectable signal |
| E1 | Webhook tier, **manual mode** (L8) | C | |
| E2 | Self-heal with `.tmp`+rename+`.bak` rollback | C | bootstrap paradox handled per L5 |
| E3 | `migrate-engine` reconciles the daemon | **A** | the literal root cause |
| E4 | Pull cron adoption per L4, separate opt-in per L12 | C | |
| E5 | Agent-facing staleness, rate-limited not nag-ceilinged (L7) | C | |
| E6 | Windows hard error naming `--tier harness` | C | `detectInstallTarget()` has no win32 branch |
| E7 | `live_ticks`**re-examined, not assumed** | C | the shipped `last_sync_at` heartbeat may suffice |
| E8 | Hygiene, narrowed to git `T` | **A** | |
### L12 — E4's pull cron is an autonomy question
`docs/guides/upgrades-auto-update.md:41-43` states *"`auto` is deliberately NOT a default
anywhere — it's an explicit autonomy grant, because applying code from GitHub unattended
is, by design, remote code execution."* This plan does **not** flip `self_upgrade.mode`.
E4 schedules `git pull` every 30 minutes. That is content, not code, and durability keeps
gbrain's hooks local and untracked so a pulled commit cannot rewrite executable hook code.
But it is still unattended network fetch into a directory gbrain runs tooling against.
Therefore the pull cron is a **separate opt-in from the tier**, proposed and explained by
`live on`, never silently bundled.
### L13 — The directive's internal tension, stated
"OpenClaw and Hermes default to always-up-to-date" sits against L1's "shape detection
recommends, never installs" and D9's "almost nothing on by default." These are reconciled
by scope: shape detection sets the **recommended tier** and pre-selects it in the `init`
consent prompt, so a shape-matching host is one keystroke from always-on rather than
silently converted. Whether that consent is required on **upgrade** as well as fresh
install is **open decision F1** below.
## L14 — Acceptance criteria
1. **Three-surface honesty.** A source whose `local_path` is deleted, whose `last_sync_at`
is 71 days old, whose `newest_content_at` is **non-NULL**, and whose `chunker_version`
**matches** must report stale/fail from `doctor` and `gbrain status`, and must surface
the lag in `sources status`. Both fixture preconditions are required: a NULL
`newest_content_at` already falls through to wall-clock
(`src/commands/doctor.ts:4335-4342`) and a chunker mismatch already disables the
fallback (`:4318`), so a naive fixture passes against unfixed code.
*`sources status` is held to output, not exit code — it has no exit contract today and
adding one is an undeclared breaking change to a read-only dashboard.*
2. **Quiet-source non-regression.** A source with a recent `last_sync_at`, an unreachable
clone, and no new content must still report **OK**. This is the 16-source / doctor
70→30 incident; the ceiling must not re-light it.
3. **Install honesty** (PR-C). `live on --tier cron` verifies the job loaded and exits
nonzero if not; deleting the repo makes `live status` exit nonzero and name the path;
`live off` leaves nothing.
4. **Concurrency** (PR-C, **Postgres only**). Two tiers ticking produce one import and one
neutral skip record. On PGLite the second process cannot open the DB at all, so the
defined outcome is a log line and no row.
5. **Watch tier** (PR-C). E2E expects **queued-job-failure**, not synchronous rejection —
`ingest_capture` enqueues and returns.
6. **Engine parity** (PR-C, if `live_ticks` survives). DDL identical in both engines,
pinned by `test/e2e/engine-parity.test.ts`; bootstrap probe-set entry pinned by
`test/schema-bootstrap-coverage.test.ts`.
## Open decisions (unanswered — do not silently default)
- **F1.** Does shape-detected always-on apply on **upgrade** as well as fresh install?
Codebase precedent (`src/commands/upgrade.ts:513-516`, `mcp.publish_skills`) is
new-installs-only with a one-time prompt for existing. Gates PR-C only.
- **F2.** Command noun and config key: `gbrain live` + `live.mode` (requires renaming the
existing `liveSyncStatus` helper at `src/core/db-lock.ts:749` to `syncInProgress`, two
call sites) vs `gbrain sync live` + `sync.live.mode`. Gates PR-C only.
## Deferred to TODOS.md
- Full Windows `schtasks` tier — no test machine; `harness` covers it
- Per-tier cost meter for `daemon` / `watch`
- Cross-OS scheduler probing as a `live status` diagnostic (TODO-V19-D stays open; the
heartbeat makes it optional rather than load-bearing)
- Centralize the three freshness call sites onto one `freshnessVerdict()` helper
(existing filed P3, now partially satisfied by PR-A's single-function fix)
## Dream state delta
PR-A leaves brain currency *honest*. PR-B leaves it *workable for the modal user*. PR-C
leaves it *a product feature*. Remaining gap to the 12-month ideal: currency is still
something the user turns on, not something simply true of a configured brain. F1 is the
decision that closes or preserves that gap.
## Reviewer concerns (unresolved after 3 iterations)
- **Scope, from both reviewers:** PR-C remains large (command family, mode bundle, shape
detector, three tiers, advisor collector, init prompt, webhook, watch tier, scheduler
extraction). The PR-A/B/C split answers the sequencing objection but not the size of C
itself. Revisit at PR-C planning with the incident already fixed.
- **`live_ticks` necessity** is explicitly unresolved and assigned to PR-C rather than
decided here.
+6
View File
@@ -65,6 +65,12 @@ For scheduling `sync` + `embed --stale` specifically, the home doc is
# Brain health — weekly Mondays at 6 AM
0 6 * * 1 gbrain doctor --json >> /tmp/gbrain-health.log 2>&1 && gbrain embed --stale
# Autopilot health gate — daily at 7 AM. The exit code is the signal:
# 0 fresh (or nothing installed), 1 needs attention (stale heartbeat,
# never ran, or paused), 2 the daemon took itself out of rotation.
# Status is filesystem-only, so it works even during a DB outage.
0 7 * * * gbrain autopilot --status >> /tmp/gbrain-autopilot-health.log 2>&1 || your-notify "gbrain autopilot needs attention"
# Dream cycle — nightly at 2 AM
0 2 * * * /path/to/dream-cycle.sh
```
+24 -1
View File
@@ -51,6 +51,10 @@ gbrain sync --repo /path/to/brain && gbrain embed --stale
[spend controls](../operations/spend-controls.md).
- `gbrain embed --stale` -- backfill embeddings for any chunks that don't have
them. Safety net for large syncs (>100 files) or prior `--no-embed` runs.
On a keyless brain (installed with `--no-embedding`), a bare stale embed
refuses cleanly — exit 0 with a stderr note — so this chain is safe to
schedule on keyless installs; keyword search keeps working. Explicit embed
requests (a slug, `--slugs`, `--all`) still exit 1 on a keyless brain.
- `gbrain sync --watch --repo <path>` -- foreground polling loop, every 60s
(configurable with `--interval N`). Embeds inline for small changesets. Exits
after 5 consecutive failures, so run under a process manager or pair with a
@@ -151,7 +155,17 @@ vars — incident-time escape hatches, not everyday knobs.
history rewrite still hard-blocks even with `--skip-failed`. Run
`gbrain sync --skip-failed` to acknowledge a known-bad set yourself.
5. **Import checkpoints name the import target, not the caller's CWD.**
5. **Staleness can't read "fresh" forever.** A source whose content stopped
moving (or whose local clone vanished) used to report fresh indefinitely
off the stored content timestamp. Content-relative staleness now ramps
toward stale once wall-clock time since the last sync passes a ceiling
(default 72h; `GBRAIN_STALENESS_CEILING_HOURS` to tune — it tracks
`GBRAIN_SYNC_FRESHNESS_FAIL_HOURS` unless set). The ramp is gradual, so
the warn tier still fires before the fail tier. `gbrain status` source
rows carry `hours_since_last_sync` (raw wall-clock truth) alongside the
threshold-relative `staleness_hours` that drives the fresh/stale class.
6. **Import checkpoints name the import target, not the caller's CWD.**
Interrupted `gbrain import <dir>` runs may leave
`~/.gbrain/import-checkpoint.json` so the next import can resume. The
checkpoint `dir` is the absolute, resolved import target captured when
@@ -178,6 +192,15 @@ vars — incident-time escape hatches, not everyday knobs.
`gbrain embed --stale` isn't running after sync, leaving chunks invisible
to vector search.
4. **Gate on the daemon's heartbeat.** If the built-in daemon runs your sync
(`gbrain autopilot --install`), wire your scheduler's health check to
`gbrain autopilot --status`. The exit code is the signal: 0 fresh (or
nothing installed), 1 needs attention (stale heartbeat, never ran, or
paused by a migration), 2 the daemon took itself out of rotation.
`--json` emits the full report, including `heartbeat_age_seconds`. Status
reads only the filesystem — no database connection — so it keeps working
during the exact outages it exists to diagnose.
---
*Part of the [GBrain Skillpack](../GBRAIN_SKILLPACK.md).*
+3
View File
@@ -121,6 +121,9 @@ cat ~/.gbrain/preferences.json
cat ~/.gbrain/migrations/completed.jsonl
# 3. Autopilot is supervising a Minions worker child
# (v0.46+: the exit code is the verdict — 0 fresh, 1 needs attention,
# 2 self-disabled — so a nonzero exit here IS the finding, not a
# broken verify step. Under `set -e`, append `|| true` to keep going.)
gbrain autopilot --status
ps aux | grep 'jobs work'
+1 -1
View File
@@ -61,7 +61,7 @@ The runtime `gbrain init --force` re-runs the init flow against the now-populate
## Pattern 3: No key, ever (keyless mode)
`--no-embedding` isn't only a deferral — it's also the install shape for **keyless mode**, a first-class supported end state (not a broken one). With zero provider keys, gbrain runs keyword-only (BM25) search and takes memory from agent-authored `## Facts` fences and write ops; embedding and extraction paths refuse cleanly instead of failing silently.
`--no-embedding` isn't only a deferral — it's also the install shape for **keyless mode**, a first-class supported end state (not a broken one). With zero provider keys, gbrain runs keyword-only (BM25) search and takes memory from agent-authored `## Facts` fences and write ops; embedding and extraction paths refuse cleanly instead of failing silently. Concretely: the documented always-current chain (`gbrain sync --repo <path> && gbrain embed --stale`) is safe to schedule on a keyless brain — a bare stale embed exits 0 with a stderr note instead of breaking the chain, while explicit embed requests (a slug, `--slugs`, `--all`) still exit 1.
```dockerfile
FROM oven/bun:1
+59 -2
View File
@@ -1264,7 +1264,13 @@ Set up using your platform's scheduler (OpenClaw cron, Railway cron, crontab), o
platform glue entirely with `gbrain autopilot --install` (built-in self-maintaining daemon):
- **Live sync** (every 15 min): `gbrain sync --repo ~/brain && gbrain embed --stale`
— or `gbrain sync --watch` for a continuous loop.
— or `gbrain sync --watch` for a continuous loop. Safe on keyless brains:
a bare `gbrain embed --stale` exits 0 with a stderr note when embeddings
are disabled, so the chain doesn't break.
- **Health gate** (daily): `gbrain autopilot --status` — exit 0 fresh (or
nothing installed), 1 needs attention (stale heartbeat, never ran, or
paused), 2 the daemon took itself out of rotation. Filesystem-only, so it
works during DB outages.
- **Auto-update** (daily): `gbrain check-update --json` (tell user, never auto-install).
- **Dream cycle** (nightly): `gbrain dream` runs the 8-phase overnight maintenance cycle.
Entity sweep, citation fixes, memory consolidation, plus (v0.23+) overnight conversation
@@ -2230,6 +2236,25 @@ live in `test/postgres-engine-rls-scope.test.ts`.
**Migration:** `gbrain migrate --to supabase` exports everything (pages, chunks, embeddings, links, tags, timeline) and imports into Supabase. `gbrain migrate --to pglite` goes the other direction. Bidirectional, lossless.
The migration and the autopilot daemon do not race: `migrate --to` claims a
cooperative pause marker before touching the target. The marker doubles as a
migration mutex — a second concurrent migrate refuses to run, and a marker
that cannot be written refuses the migration outright. Background job workers
stop picking up new work while it is parked, and the migration waits for
in-flight sync/embed/cycle work and running jobs to actually drain (watching
the DB lock table, capped by `GBRAIN_MIGRATE_QUIESCE_SECONDS` — default 300;
`0` skips the wait). Cleanup registers the moment the claim lands, so the
marker is released on failure and on catchable signals; a marker orphaned by
an uncleanly killed run is adopted by a later migrate only after a
pid-liveness check (a live migrate's marker is never stolen), and the daemon
clears an orphan whose owning process died on its next poll. `gbrain
autopilot --status` reports `paused` (exit 1) while the marker is parked and
prints the marker path; on a host with no daemon running to self-heal,
remove an orphan by hand only after confirming the pid it names is dead.
After a clean flip the daemon detects the engine change on its next
tick and relaunches onto the new engine, and the migration warns if an
exported connection-string env var would override the new config.
### Troubleshooting: startup abort (`RuntimeError: Aborted()`)
**Symptom:** every PGLite-touching command dies at startup with
@@ -2859,6 +2884,10 @@ gbrain sync --repo /path/to/brain && gbrain embed --stale
[spend controls](../operations/spend-controls.md).
- `gbrain embed --stale` -- backfill embeddings for any chunks that don't have
them. Safety net for large syncs (>100 files) or prior `--no-embed` runs.
On a keyless brain (installed with `--no-embedding`), a bare stale embed
refuses cleanly — exit 0 with a stderr note — so this chain is safe to
schedule on keyless installs; keyword search keeps working. Explicit embed
requests (a slug, `--slugs`, `--all`) still exit 1 on a keyless brain.
- `gbrain sync --watch --repo <path>` -- foreground polling loop, every 60s
(configurable with `--interval N`). Embeds inline for small changesets. Exits
after 5 consecutive failures, so run under a process manager or pair with a
@@ -2959,7 +2988,17 @@ vars — incident-time escape hatches, not everyday knobs.
history rewrite still hard-blocks even with `--skip-failed`. Run
`gbrain sync --skip-failed` to acknowledge a known-bad set yourself.
5. **Import checkpoints name the import target, not the caller's CWD.**
5. **Staleness can't read "fresh" forever.** A source whose content stopped
moving (or whose local clone vanished) used to report fresh indefinitely
off the stored content timestamp. Content-relative staleness now ramps
toward stale once wall-clock time since the last sync passes a ceiling
(default 72h; `GBRAIN_STALENESS_CEILING_HOURS` to tune — it tracks
`GBRAIN_SYNC_FRESHNESS_FAIL_HOURS` unless set). The ramp is gradual, so
the warn tier still fires before the fail tier. `gbrain status` source
rows carry `hours_since_last_sync` (raw wall-clock truth) alongside the
threshold-relative `staleness_hours` that drives the fresh/stale class.
6. **Import checkpoints name the import target, not the caller's CWD.**
Interrupted `gbrain import <dir>` runs may leave
`~/.gbrain/import-checkpoint.json` so the next import can resume. The
checkpoint `dir` is the absolute, resolved import target captured when
@@ -2986,6 +3025,15 @@ vars — incident-time escape hatches, not everyday knobs.
`gbrain embed --stale` isn't running after sync, leaving chunks invisible
to vector search.
4. **Gate on the daemon's heartbeat.** If the built-in daemon runs your sync
(`gbrain autopilot --install`), wire your scheduler's health check to
`gbrain autopilot --status`. The exit code is the signal: 0 fresh (or
nothing installed), 1 needs attention (stale heartbeat, never ran, or
paused by a migration), 2 the daemon took itself out of rotation.
`--json` emits the full report, including `heartbeat_age_seconds`. Status
reads only the filesystem — no database connection — so it keeps working
during the exact outages it exists to diagnose.
---
*Part of the [GBrain Skillpack](../GBRAIN_SKILLPACK.md).*
@@ -3063,6 +3111,12 @@ For scheduling `sync` + `embed --stale` specifically, the home doc is
# Brain health — weekly Mondays at 6 AM
0 6 * * 1 gbrain doctor --json >> /tmp/gbrain-health.log 2>&1 && gbrain embed --stale
# Autopilot health gate — daily at 7 AM. The exit code is the signal:
# 0 fresh (or nothing installed), 1 needs attention (stale heartbeat,
# never ran, or paused), 2 the daemon took itself out of rotation.
# Status is filesystem-only, so it works even during a DB outage.
0 7 * * * gbrain autopilot --status >> /tmp/gbrain-autopilot-health.log 2>&1 || your-notify "gbrain autopilot needs attention"
# Dream cycle — nightly at 2 AM
0 2 * * * /path/to/dream-cycle.sh
```
@@ -4908,6 +4962,9 @@ cat ~/.gbrain/preferences.json
cat ~/.gbrain/migrations/completed.jsonl
# 3. Autopilot is supervising a Minions worker child
# (v0.46+: the exit code is the verdict — 0 fresh, 1 needs attention,
# 2 self-disabled — so a nonzero exit here IS the finding, not a
# broken verify step. Under `set -e`, append `|| true` to keep going.)
gbrain autopilot --status
ps aux | grep 'jobs work'
+1 -1
View File
@@ -152,7 +152,7 @@
"bun": ">=1.3.10"
},
"license": "MIT",
"version": "0.45.3.0",
"version": "0.45.5.0",
"overrides": {
"@hono/node-server": "^2.0.5",
"fast-uri": "^3.1.5",
+7
View File
@@ -195,6 +195,13 @@ Verify autopilot is running:
```bash
gbrain autopilot --status
```
The exit code is trustworthy for gating: 0 fresh (or nothing installed),
1 needs attention (stale heartbeat, never ran, or paused by a migration),
2 the daemon disabled itself (its repo path vanished). `--json` emits the
full report (`state`, `heartbeat_age_seconds`, `paused_reason`,
`disabled_reason`). Status reads only the filesystem, so it works even
when the database is down.
If not running, install it:
```bash
gbrain autopilot --install --repo ~/brain
+1 -1
View File
@@ -57,7 +57,7 @@
"idea-lineage/routing-eval.jsonl": "ee2e00704b9accb7dd58bb8f126a3bc04a2c40be499180fa505dbf6d5061cd41",
"ingest/SKILL.md": "dc40ecc0072806fb8c7bb6ab9cf1f103842e05653eb55d67632d7e3ffc4dd7d2",
"install/SKILL.md": "881bd0a422f34c6df4642aae66c51e2a4cc18ad5ca6d0b52d44b4de93512a3c4",
"maintain/SKILL.md": "e80cf5bb170c979b773a67a18e0d880c086e0b983db9a576a207074b0112c4f8",
"maintain/SKILL.md": "762e20ce7e064709a31a843704600679fd39e6734c9f14a67a98fb8c081b7513",
"manifest.json": "8fbdd5fd3c5c8f9b4ebf4c5af18f2396af6f3f638bf180e8ca0b0c35584dc6dd",
"media-ingest/SKILL.md": "33db12830ed31a4ff4a6a58c4f126bf2596ee28d54cee0027680c83fee648a20",
"meeting-ingestion/SKILL.md": "7767334c63ff3bd8e60cd4d7cd1d1b44f0d6b0a7e0ac411529a1d59cc0a7781b",
+12
View File
@@ -2251,6 +2251,18 @@ async function handleCliOnly(command: string, args: string[]) {
}
}
// Autopilot status + uninstall are filesystem-only verdicts and MUST stay
// engine-free: a running PGLite daemon holds the exclusive DB lock, so an
// engine-bound status would fail to connect in exactly the scenarios the
// exit-code contract exists to diagnose (live daemon, DB outage). Order
// mirrors runAutopilot's own flag precedence (uninstall before status).
if (command === 'autopilot' && (args.includes('--uninstall') || args.includes('--status'))) {
const { runAutopilotStatus, uninstallDaemon } = await import('./commands/autopilot.ts');
if (args.includes('--uninstall')) uninstallDaemon();
else runAutopilotStatus(args);
return;
}
// All remaining CLI-only commands need a DB connection
const engine = await connectEngine();
try {
+456 -43
View File
@@ -17,13 +17,13 @@
* gbrain autopilot --status [--json]
*/
import { existsSync, readFileSync, writeFileSync, mkdirSync, appendFileSync, utimesSync, unlinkSync, chmodSync } from 'fs';
import { existsSync, readFileSync, writeFileSync, mkdirSync, appendFileSync, utimesSync, unlinkSync, chmodSync, statSync } from 'fs';
import { setCliExitVerdict } from '../core/cli-force-exit.ts';
import { join, dirname } from 'path';
import { execSync } from 'child_process';
import type { BrainEngine } from '../core/engine.ts';
import { loadPreferences } from '../core/preferences.ts';
import { loadConfig, saveConfig, gbrainPath as gbrainHomePath } from '../core/config.ts';
import { loadConfig, loadConfigFileOnly, saveConfig, gbrainPath as gbrainHomePath } from '../core/config.ts';
import { ChildWorkerSupervisor } from '../core/minions/child-worker-supervisor.ts';
import { VERSION } from '../version.ts';
import {
@@ -40,6 +40,20 @@ import { evaluateQuietHours } from '../core/minions/quiet-hours.ts';
import { inspectLock } from '../core/db-lock.ts';
import { registerCleanup } from '../core/process-cleanup.ts';
import { resolveAutopilotDispatchTimeoutMs } from './autopilot-timeout.ts';
// Path helpers live in a LEAF core module so other commands (gbrain migrate)
// can read the daemon's state files without importing this one — a dynamic
// import of a command module drags its whole flag surface into the importer's
// CLI allowlist. Re-exported here so existing importers keep working.
import {
autopilotLockPath,
autopilotDisabledMarkerPath,
autopilotPausedMarkerPath,
autopilotDisableStrikesPath,
autopilotLaunchdLabel,
markerHolderAlive,
MIGRATE_PAUSE_MARKER_PREFIX,
} from '../core/autopilot-paths.ts';
export { autopilotLockPath, autopilotDisabledMarkerPath, autopilotPausedMarkerPath, autopilotLaunchdLabel };
/**
* v0.37.7.0 #1162 classify autopilot reconnect-loop errors.
@@ -52,11 +66,42 @@ import { resolveAutopilotDispatchTimeoutMs } from './autopilot-timeout.ts';
* config file unreadable): exit immediately so launchd's 60s
* `ThrottleInterval` backs off the relaunch instead of thrashing.
*
* Exported (string-based signature) so tests drive it without needing
* `crash` (a JS TypeError from dereferencing an undefined object): a BUG, not a
* verdict about the operator's configuration. Treated as recoverable so a code
* defect cannot permanently kill the daemon, but logged distinctly so it is not
* silently misfiled as "you forgot to set a URL."
*
* Exported (string-based signature preserved) so tests drive it without needing
* a real reconnect error.
*/
export function classifyReconnectError(err: unknown): 'recoverable' | 'unrecoverable' {
export function classifyReconnectError(err: unknown): 'recoverable' | 'unrecoverable' | 'crash' {
// Type check FIRST. Bun/V8 renders a null-deref as
// "undefined is not an object (evaluating 'config.database_url')"
// which, lowercased, contains BOTH "database_url" and "undefined" — so the
// substring rule below classified a crash as a config verdict and exited the
// daemon permanently. That is exactly how a 71-day outage started: an engine
// migration rewrote config.json, the running daemon crashed on a stale object,
// and the crash was reported as "database_url not set".
const earlyMsg = (err instanceof Error ? err.message : String(err ?? '')).toLowerCase();
// Invalid-URL errors ARE TypeErrors in JS (`new URL('garbage')`), but they
// are an operator-config verdict, not a code defect — test the message
// pattern BEFORE the blanket TypeError-means-crash rule, or a malformed
// database_url spends the whole reconnect budget before exiting.
if (earlyMsg.includes('invalid url') || earlyMsg.includes('malformed') || earlyMsg.includes('parse url')) {
return 'unrecoverable';
}
if (err instanceof Error && err.name === 'TypeError') return 'crash';
const msg = (err instanceof Error ? err.message : String(err ?? '')).toLowerCase();
// Same shape, for hosts where the error arrives as a plain string/serialized
// object and the `name` is gone.
if (
msg.includes('is not an object')
|| msg.includes('is not a function')
|| msg.includes('cannot read propert')
|| msg.includes('undefined is not')
) {
return 'crash';
}
if (msg.includes('database_url') && (msg.includes('undefined') || msg.includes('missing') || msg.includes('empty') || msg.includes('not set'))) {
return 'unrecoverable';
}
@@ -381,12 +426,15 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
return;
}
if (args.includes('--status')) {
showStatus(args.includes('--json'));
runAutopilotStatus(args);
return;
}
const repoPath = parseArg(args, '--repo') || await engine.getConfig('sync.repo_path');
const baseInterval = parseInt(parseArg(args, '--interval') || '300', 10);
// Same NaN guard as the status path: a typo'd interval would otherwise
// reach setTimeout(NaN) → 0ms and busy-loop the daemon against the DB.
const rawBaseInterval = parseInt(parseArg(args, '--interval') || '300', 10);
const baseInterval = Number.isFinite(rawBaseInterval) && rawBaseInterval > 0 ? rawBaseInterval : 300;
const jsonMode = args.includes('--json');
const forceInline = args.includes('--inline');
const noWorker = !shouldSpawnAutopilotWorker(args);
@@ -402,7 +450,7 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
// two brains sharing GBRAIN_HOME=different-paths still wrote to the
// same global lockfile and one would silently respawn the other
// forever.
const lockPath = gbrainHomePath('autopilot.lock');
const lockPath = autopilotLockPath();
try {
mkdirSync(gbrainHomePath(), { recursive: true });
const decision = decideLockAcquisition(lockPath, process.pid);
@@ -427,6 +475,15 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
const useMinionsDispatch = mode !== 'off' && engineType === 'postgres' && !forceInline;
const spawnManagedWorker = useMinionsDispatch && !noWorker;
// Engine identity at boot, re-checked every tick. A cross-engine migration
// flips config.json at the END of its copy; this long-lived process would
// otherwise keep syncing into the ABANDONED source engine indefinitely —
// the health probe keeps succeeding (the old engine stays alive as the
// preserved backup) and reconnect() deliberately restores the config
// captured at connect() (#2034), never the new file. Same silent-divergence
// class as the dead-daemon incident, moved to after the flip.
const engineIdentityAtBoot = autopilotEngineIdentity(loadConfigFileOnly());
// v0.42 self-upgrade: if a prior tick swapped the binary and exited for
// relaunch, we're now the relaunched process — reconcile the breadcrumb so a
// crash-on-launch is recorded known-bad and a success is confirmed.
@@ -592,6 +649,8 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
// runs the full cycle (phase-coupling exercise) before settling into
// targeted-submit mode.
let lastFullCycleAt = 0;
// Log the pause/resume transition once each, not every poll.
let pausedAnnounced = false;
while (!stopping) {
const cycleStart = Date.now();
@@ -601,6 +660,60 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
// declare the instance stale after 10 minutes (Codex C).
try { utimesSync(lockPath, new Date(), new Date()); } catch { /* best-effort */ }
// Post-migration convergence: if the file-plane engine identity changed
// since boot, this process is connected to the wrong engine. Exit through
// the clean shutdown path (engine close matters for PGLite WAL) so the
// supervisor relaunches on the new config; the same relaunch contract the
// self-upgrade swap relies on. Cron and one-shot targets simply pick up
// the new config on their next run.
// A torn or failed read (concurrent config write, transient EACCES) must
// not restart the daemon: skip the comparison unless the file read
// actually produced a config — a genuine migration flip never yields null.
let identityNow: string | null = null;
try {
const fileCfg = loadConfigFileOnly();
identityNow = fileCfg ? autopilotEngineIdentity(fileCfg) : null;
} catch { /* torn read mid-write; check again next tick */ }
if (identityNow !== null && identityNow !== engineIdentityAtBoot) {
console.log('[autopilot] engine config changed on disk (migration?) — exiting for relaunch on the new engine.');
await shutdown('engine-config-changed');
return;
}
// Cooperative pause (see autopilotPausedMarkerPath). Checked AFTER the
// heartbeat so a paused daemon still reads as alive, and BEFORE any DB
// work so a cross-engine migration is not racing our writes into an
// engine that is about to stop being the configured one.
if (existsSync(autopilotPausedMarkerPath())) {
// Self-heal an orphan: a migrate-owned marker whose recorded pid is dead
// was leaked by a killed migration (SIGKILL, power loss — anything its
// own cleanup could not catch). Nothing else ever deletes it, and an
// orphan parks this daemon forever. An operator's manual hold (no
// migrate signature) is never touched, and a live migrate's marker
// reads alive and is honored.
let orphaned = false;
try {
const body = readFileSync(autopilotPausedMarkerPath(), 'utf-8');
orphaned = body.startsWith(MIGRATE_PAUSE_MARKER_PREFIX) && markerHolderAlive(body) === 'dead';
} catch { /* vanished or unreadable: fall through to the normal pause */ }
if (orphaned) {
console.log('[autopilot] clearing an orphaned pause marker (its migrate process is dead); resuming.');
try { unlinkSync(autopilotPausedMarkerPath()); } catch { /* already gone */ }
} else {
if (!pausedAnnounced) {
console.log('[autopilot] paused (autopilot-paused marker present) — skipping cycles until it clears.');
pausedAnnounced = true;
}
// Poll faster than a normal tick so a migration's quiesce window is short.
await new Promise((r) => setTimeout(r, Math.min(baseInterval, 30) * 1000));
continue;
}
}
if (pausedAnnounced) {
console.log('[autopilot] resumed — pause marker cleared.');
pausedAnnounced = false;
}
// DB health check (reconnect if needed).
//
// v0.37.7.0 #1162: classify reconnect failures. Pre-fix, the
@@ -633,7 +746,17 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
logError('reconnect', e);
autopilotReconnectFails++;
const klass = classifyReconnectError(e);
if (klass === 'unrecoverable') {
if (klass === 'crash') {
// A gbrain BUG, not an operator misconfiguration. Say so plainly
// instead of blaming the config, and keep retrying: a code defect must
// not permanently disable the daemon. The consecutive-failure cap below
// still bounds it.
console.error(
`[autopilot] BUG: internal error during reconnect (${(e as Error).message ?? 'unknown'}). ` +
`This is a gbrain defect, not a configuration problem — please report it. ` +
`Retrying (${autopilotReconnectFails}/${AUTOPILOT_MAX_RECONNECT_FAILS}).`,
);
} else if (klass === 'unrecoverable') {
console.error(
`[autopilot] FATAL: unrecoverable DB error (${(e as Error).message ?? 'unknown'}). ` +
`Exiting so launchd ThrottleInterval can apply backoff.`,
@@ -1248,11 +1371,11 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
// --- Install/Uninstall ---
function plistPath(): string {
return join(process.env.HOME || '', 'Library', 'LaunchAgents', 'com.gbrain.autopilot.plist');
return join(process.env.HOME || '', 'Library', 'LaunchAgents', `${autopilotLaunchdLabel()}.plist`);
}
function systemdUnitPath(): string {
return join(process.env.HOME || '', '.config', 'systemd', 'user', 'gbrain-autopilot.service');
return join(process.env.HOME || '', '.config', 'systemd', 'user', AUTOPILOT_SYSTEMD_UNIT);
}
function ephemeralStartScriptPath(): string {
@@ -1311,9 +1434,79 @@ function detectOpenClaw(): { detected: boolean; bootstrapCandidates: string[] }
return { detected: signal, bootstrapCandidates: existing };
}
function writeWrapperScript(repoPath: string): string {
const home = process.env.HOME || '';
const gbrainDir = join(home, '.gbrain');
/** systemd unit name. The launchd label lives in `autopilotLaunchdLabel()`
* (core/autopilot-paths.ts) so installer, uninstaller, status, and the
* wrapper's self-disable can never name different jobs. */
export const AUTOPILOT_SYSTEMD_UNIT = 'gbrain-autopilot.service';
/**
* Bash block that stops the daemon for good when its captured `--repo` is gone.
*
* Two things this must get right, both learned the hard way:
*
* 1. PREDICATE. Test the repo DIRECTORY, not `$repo/.git`. `--repo` may be a
* subdirectory of the checkout (sync resolves the root itself by walking up
* with `git rev-parse` show-toplevel), and in worktrees and submodules
* `.git` is a FILE, not a directory. Testing `.git/` would self-disable a
* perfectly healthy install in both shapes.
*
* 2. MECHANISM. `exit 0` is the right answer for the sibling cron wrapper in
* `brain-repo-durability.ts` because launchd fires that one on StartInterval
* one shot, so exiting ends it. Autopilot runs under `KeepAlive=true` +
* `ThrottleInterval=60` (and systemd `Restart=always` / `RestartSec=30`),
* where exiting 0 disables NOTHING: it converts a dead install into a silent
* respawn-every-60s log-append loop that runs forever. On those two targets
* the wrapper has to actually take the job out of rotation.
*
* `linux-cron` and `ephemeral-container` are periodic/one-shot, so a plain
* exit is correct and sufficient there.
*
* Exported pure so tests can assert the emitted shape per target without
* installing a daemon.
*/
export function generateSelfDisableGuard(repoPath: string, target: InstallTarget): string {
const q = (s: string) => s.replace(/'/g, "'\\''");
const marker = autopilotDisabledMarkerPath();
const disableCmd =
target === 'macos'
? ` launchctl bootout "gui/$(id -u)/${autopilotLaunchdLabel()}" 2>/dev/null || true\n`
: target === 'linux-systemd'
? ` systemctl --user disable --now ${AUTOPILOT_SYSTEMD_UNIT} 2>/dev/null || true\n`
: '';
const strikes = autopilotDisableStrikesPath();
return `# Self-disable if the captured checkout is gone (rename / relocation / deletion).
# Tests the repo DIRECTORY: --repo may be a subdirectory of the checkout, and
# .git is a FILE in worktrees and submodules, so [ ! -d "$repo/.git" ] would
# false-positive on healthy installs.
# THREE consecutive misses before disabling: repos on external volumes, NFS,
# or cloud-synced folders are routinely absent for the first launch after
# login, and one transient miss must not permanently kill the install. Any
# successful probe resets the strike counter.
if [ ! -d '${q(repoPath)}' ]; then
_strikes=$(($(cat '${q(strikes)}' 2>/dev/null || echo 0) + 1))
echo "$_strikes" > '${q(strikes)}' 2>/dev/null || true
if [ "$_strikes" -lt 3 ]; then
echo "$(date -u +%FT%TZ) [autopilot] repo path missing (strike $_strikes of 3, disabling at 3):" '${q(repoPath)}'
exit 0
fi
echo "$(date -u +%FT%TZ) [autopilot] repo path gone, disabling:" '${q(repoPath)}'
printf '%s\\n' 'repo path gone: ${q(repoPath)}' > '${q(marker)}' 2>/dev/null || true
rm -f '${q(strikes)}' 2>/dev/null || true
${disableCmd} exit 0
fi
rm -f '${q(strikes)}' 2>/dev/null || true
`;
}
function writeWrapperScript(repoPath: string, target: InstallTarget): string {
// gbrainHomePath, not raw $HOME: the daemon writes its lock/markers through
// it and the status command reads through it, so a GBRAIN_HOME install must
// keep its wrapper (and the start-script detection that looks for it) in
// the same directory. Identical to the old behavior when GBRAIN_HOME is
// unset. The env var is also baked into the wrapper below — launchd does
// not pass the installer's environment to the spawned job.
const gbrainDir = gbrainHomePath();
mkdirSync(gbrainDir, { recursive: true });
// Wrapper sources the user's shell profile for API keys so nothing is
@@ -1352,7 +1545,8 @@ source ~/.zshrc 2>/dev/null || source ~/.bashrc 2>/dev/null || true
# fallback, keeps the wrapper self-contained regardless of where bun is installed
# or which init file the OS loaded.
export PATH=${runtimePathPrefix}"$HOME/.bun/bin:$PATH"
exec '${safeGbrainPath}' autopilot --repo '${safeRepoPath}'
${process.env.GBRAIN_HOME ? `# Baked at install: the supervisor does not pass the installer's env, and\n# without this the daemon would read/write a different home than the\n# install that configured it.\nexport GBRAIN_HOME='${(process.env.GBRAIN_HOME).replace(/'/g, "'\\''")}'\n` : ''}
${generateSelfDisableGuard(repoPath, target)}exec '${safeGbrainPath}' autopilot --repo '${safeRepoPath}'
`;
writeFileSync(wrapperPath, wrapper, { mode: 0o755 });
return wrapperPath;
@@ -1371,7 +1565,12 @@ async function installDaemon(engine: BrainEngine, args: string[]) {
const injectBootstrap = args.includes('--inject-bootstrap');
const noInject = args.includes('--no-inject');
const wrapperPath = writeWrapperScript(repoPath);
const wrapperPath = writeWrapperScript(repoPath, target);
// A fresh install clears any prior self-disable AND any leaked pause, so a
// reinstall does not report "disabled" forever or park itself from day one
// on a marker some dead migration left behind.
try { unlinkSync(autopilotDisabledMarkerPath()); } catch { /* not disabled */ }
try { unlinkSync(autopilotPausedMarkerPath()); } catch { /* not paused */ }
const home = process.env.HOME || '';
switch (target) {
@@ -1401,7 +1600,7 @@ export function generateLaunchdPlist(wrapperPath: string, home: string): string
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key><string>com.gbrain.autopilot</string>
<key>Label</key><string>${escapeXml(autopilotLaunchdLabel())}</string>
<key>ProgramArguments</key><array>
<string>${escapeXml(wrapperPath)}</string>
</array>
@@ -1438,7 +1637,7 @@ function installLaunchd(wrapperPath: string, home: string, repoPath: string) {
// normalize unconditionally.
chmodSync(plistPath(), 0o644);
execSync(`launchctl load "${plistPath()}"`, { stdio: 'pipe' });
console.log('Installed launchd service: com.gbrain.autopilot');
console.log(`Installed launchd service: ${autopilotLaunchdLabel()}`);
console.log(` Repo: ${repoPath}`);
console.log(` Log: ~/.gbrain/autopilot.log`);
console.log(' Uninstall: gbrain autopilot --uninstall');
@@ -1552,8 +1751,8 @@ function installSystemd(wrapperPath: string, repoPath: string) {
// world-writable units); mode only applies on create, so normalize.
chmodSync(unitPath, 0o644);
execSync('systemctl --user daemon-reload', { stdio: 'pipe', timeout: 10_000 });
execSync('systemctl --user enable --now gbrain-autopilot.service', { stdio: 'pipe', timeout: 15_000 });
console.log('Installed systemd user service: gbrain-autopilot.service');
execSync(`systemctl --user enable --now ${AUTOPILOT_SYSTEMD_UNIT}`, { stdio: 'pipe', timeout: 15_000 });
console.log(`Installed systemd user service: ${AUTOPILOT_SYSTEMD_UNIT}`);
console.log(` Repo: ${repoPath}`);
console.log(' Log: ~/.gbrain/autopilot.log');
console.log(' Uninstall: gbrain autopilot --uninstall');
@@ -1657,9 +1856,29 @@ function installCrontab(wrapperPath: string, home: string) {
}
}
function uninstallDaemon() {
/**
* The status verdict, engine-free. Dispatched BEFORE connectEngine in cli.ts:
* a running PGLite daemon holds the exclusive DB lock, so an engine-bound
* status could not run against a healthy live install and a DB outage would
* take down the very alarm meant to diagnose it. Everything it reads is
* filesystem (lock mtime, markers, plist/unit/crontab, log tail).
*/
export function runAutopilotStatus(args: string[]): void {
// An INSTALLED daemon always runs the default interval — the generated
// wrapper execs `autopilot --repo <path>` with no --interval. The flag is
// honored here for the manual foreground case. Garbage input must not
// become NaN: staleAfter = NaN makes every age comparison false, which
// reads a 71-day-dead daemon as 'fresh' with exit 0 — a typo'd flag would
// silently disable the very alarm this exit code exists to be.
const rawInterval = parseInt(parseArg(args, '--interval') || '300', 10);
showStatus(args.includes('--json'), Number.isFinite(rawInterval) && rawInterval > 0 ? rawInterval : 300);
}
export function uninstallDaemon() {
const home = process.env.HOME || '';
const wrapperPath = join(home, '.gbrain', 'autopilot-run.sh');
// Same resolution as writeWrapperScript — a GBRAIN_HOME install must
// uninstall the wrapper it actually wrote, not a sibling under raw $HOME.
const wrapperPath = join(gbrainHomePath(), 'autopilot-run.sh');
// Always try all four targets — the user might have run `--install` under
// one target earlier and moved hosts (e.g. macOS laptop → Linux server).
@@ -1672,7 +1891,7 @@ function uninstallDaemon() {
try {
execSync(`launchctl unload "${plistPath()}" 2>/dev/null || true`, { stdio: 'pipe' });
unlinkSync(plistPath());
console.log('Removed launchd service: com.gbrain.autopilot');
console.log(`Removed launchd service: ${autopilotLaunchdLabel()}`);
removed++;
} catch (e) {
console.error(` [warn] launchd: ${e instanceof Error ? e.message : e}`);
@@ -1682,7 +1901,7 @@ function uninstallDaemon() {
// Linux systemd user unit
if (existsSync(systemdUnitPath())) {
try {
execSync('systemctl --user disable --now gbrain-autopilot.service 2>/dev/null || true', { stdio: 'pipe', timeout: 10_000 });
execSync(`systemctl --user disable --now ${AUTOPILOT_SYSTEMD_UNIT} 2>/dev/null || true`, { stdio: 'pipe', timeout: 10_000 });
unlinkSync(systemdUnitPath());
try { execSync('systemctl --user daemon-reload', { stdio: 'pipe', timeout: 5_000 }); } catch { /* best-effort */ }
console.log('Removed systemd user service: gbrain-autopilot.service');
@@ -1761,36 +1980,230 @@ function uninstallDaemon() {
if (removed === 0) {
console.log('No autopilot install found on this host. Nothing to uninstall.');
}
// A deliberate uninstall ends the disabled/paused story: without this, a
// self-disabled install that is then uninstalled keeps reporting
// "DISABLED — repo path gone" with exit 2 forever on a machine with nothing
// installed ('disabled' outranks 'not_installed' in the classifier).
try { unlinkSync(autopilotDisabledMarkerPath()); } catch { /* not present */ }
try { unlinkSync(autopilotPausedMarkerPath()); } catch { /* not present */ }
}
function showStatus(json: boolean) {
const logFile = join(process.env.HOME || '', '.gbrain', 'autopilot.log');
let lastLine = '';
try {
const content = readFileSync(logFile, 'utf-8');
const lines = content.trim().split('\n');
lastLine = lines[lines.length - 1] || '';
} catch { /* no log */ }
/**
* The daemon's view of WHICH engine the file-plane config points at. Pure and
* deliberately narrow: only the fields an engine migration flips participate,
* so unrelated config edits (models, search knobs, spend gates) never trigger
* a restart. Compared at boot vs every tick by the daemon loop.
*/
export function autopilotEngineIdentity(
cfg: { engine?: string; database_url?: string; database_path?: string } | null,
): string {
return JSON.stringify({
engine: cfg?.engine ?? 'pglite',
url: cfg?.database_url ?? null,
path: cfg?.database_path ?? null,
});
}
let installed = false;
if (process.platform === 'darwin') {
installed = existsSync(plistPath());
} else {
export type AutopilotState = 'not_installed' | 'disabled' | 'paused' | 'never_run' | 'stale' | 'fresh';
export interface AutopilotStatusReport {
installed: boolean;
install_target: InstallTarget | null;
state: AutopilotState;
disabled_reason: string | null;
paused_reason: string | null;
heartbeat_age_seconds: number | null;
stale_after_seconds: number;
last_log: string;
}
/**
* Exit codes for `gbrain autopilot --status`, so cron and CI can gate on it.
* 0 fresh, or nothing installed (nothing claimed, nothing broken)
* 1 installed but not syncing (stale heartbeat, never ran, or parked on a
* cooperative pause marker a live migrate, or one that died without
* cleaning up; either way the brain is not being kept current)
* 2 the daemon took itself out of rotation (repo gone)
*/
export function autopilotStatusExitCode(state: AutopilotState): number {
if (state === 'disabled') return 2;
if (state === 'stale' || state === 'never_run' || state === 'paused') return 1;
return 0;
}
/**
* Pure classifier so the tri-state is testable without an installed daemon.
*
* The heartbeat is the lock mtime, which the tick loop already refreshes every
* pass (`utimesSync(lockPath, ...)`). Deliberately NOT a new artifact: a second
* one could disagree with the first, and a daemon still running a pre-upgrade
* binary would never write it, so a healthy install would report stale until it
* happened to relaunch.
*/
export function classifyAutopilotStatus(input: {
installed: boolean;
installTarget: InstallTarget | null;
disabledReason: string | null;
pausedReason?: string | null;
heartbeatAgeSeconds: number | null;
intervalSeconds: number;
lastLog: string;
}): AutopilotStatusReport {
// Tolerance = 6 intervals. The adaptive scheduler sleeps TWO intervals
// between ticks on the healthiest brains (score >= 90), and the heartbeat
// only refreshes at tick top — so a healthy gap is cycle_duration + 2x
// interval, and a 3x tolerance would flap 'stale' exit-1 alarms on exactly
// the installs doing best. 6x still catches the dead-daemon incident in
// 30 minutes at the default interval instead of 71 days. A non-finite or
// non-positive interval (a typo'd flag upstream) would make staleAfter NaN
// and every age comparison false — reading a long-dead daemon as 'fresh' —
// so the pure layer defends itself too.
const staleAfter = Number.isFinite(input.intervalSeconds) && input.intervalSeconds > 0
? input.intervalSeconds * 6
: 1800;
const pausedReason = input.pausedReason ?? null;
let state: AutopilotState;
if (input.disabledReason !== null) state = 'disabled';
else if (!input.installed) state = 'not_installed';
// Paused outranks the heartbeat states: the tick loop refreshes its
// heartbeat BEFORE honoring the pause marker, so a parked daemon looks
// 'fresh' by mtime while doing no work. Without this state a pause marker
// orphaned by a dead migrate is invisible — the daemon idles forever and
// status swears everything is fine (the 71-day incident's shape again).
else if (pausedReason !== null) state = 'paused';
else if (input.heartbeatAgeSeconds === null) state = 'never_run';
else state = input.heartbeatAgeSeconds > staleAfter ? 'stale' : 'fresh';
return {
installed: input.installed,
install_target: input.installTarget,
state,
disabled_reason: input.disabledReason,
paused_reason: pausedReason,
heartbeat_age_seconds: input.heartbeatAgeSeconds,
stale_after_seconds: staleAfter,
last_log: input.lastLog,
};
}
/**
* Which supervisor, if any, currently holds an autopilot install.
*
* Checks every target `installDaemon` can produce. The prior version grepped
* crontab ONLY on non-darwin, so systemd-user and ephemeral-container installs
* read as "not installed" cosmetic while status always exited 0, but a hard
* false failure once the exit code became load-bearing.
*/
function detectInstalledTarget(): InstallTarget | null {
if (process.platform === 'darwin' && existsSync(plistPath())) return 'macos';
if (existsSync(systemdUnitPath())) return 'linux-systemd';
if (existsSync(join(gbrainHomePath(), 'start-autopilot.sh'))) return 'ephemeral-container';
try {
const crontab = execSync('crontab -l 2>/dev/null || true', { encoding: 'utf-8' });
if (crontabIndicatesAutopilotInstall(crontab)) {
return 'linux-cron';
}
} catch { /* no crontab */ }
return null;
}
/**
* Does this crontab contain an autopilot INSTALL line? The installed line
* invokes the generated wrapper (autopilot-run.sh); older installs called
* `gbrain autopilot` directly match either. But the docs also recommend
* cron-ing `gbrain autopilot --status` as a health monitor, and counting THAT
* line as an install makes a monitor-only machine report installed/never_run
* with exit 1 forever. Comments never count. Pure and exported for tests.
*/
export function crontabIndicatesAutopilotInstall(crontab: string): boolean {
return crontab.split('\n').some((line) => {
if (line.trimStart().startsWith('#')) return false;
if (line.includes('autopilot-run.sh')) return true;
return line.includes('gbrain autopilot') && !line.includes('--status');
});
}
function showStatus(json: boolean, intervalSeconds: number) {
// gbrainHomePath, not raw HOME: the daemon writes its lock through
// gbrainHomePath() (#1226), so a GBRAIN_HOME install had status reading one
// directory while the daemon wrote another — a permanent false "stale".
const home = gbrainHomePath();
let lastLine = '';
for (const logPath of [join(home, 'autopilot.log'), join(process.env.HOME || '', '.gbrain', 'autopilot.log')]) {
try {
const crontab = execSync('crontab -l 2>/dev/null || true', { encoding: 'utf-8' });
// The installed cron line invokes the generated wrapper (…/autopilot-run.sh);
// older installs called `gbrain autopilot` directly. Match either so status
// isn't a false negative after the wrapper indirection landed.
installed = crontab.includes('autopilot-run.sh') || crontab.includes('gbrain autopilot');
} catch { /* no crontab */ }
const content = readFileSync(logPath, 'utf-8');
const lines = content.trim().split('\n');
lastLine = lines[lines.length - 1] || '';
break;
} catch { /* try the next home; supervisor log redirects bake raw $HOME */ }
}
let disabledReason: string | null = null;
try {
disabledReason = readFileSync(autopilotDisabledMarkerPath(), 'utf-8').trim() || null;
} catch { /* not self-disabled */ }
let pausedReason: string | null = null;
try {
pausedReason = readFileSync(autopilotPausedMarkerPath(), 'utf-8').trim() || 'pause marker present (no reason recorded)';
} catch { /* not paused */ }
let heartbeatAgeSeconds: number | null = null;
try {
const { mtimeMs } = statSync(autopilotLockPath());
heartbeatAgeSeconds = Math.max(0, Math.floor((Date.now() - mtimeMs) / 1000));
} catch { /* never ran, or already cleaned up */ }
const installTarget = detectInstalledTarget();
const report = classifyAutopilotStatus({
installed: installTarget !== null,
installTarget,
disabledReason,
pausedReason,
heartbeatAgeSeconds,
intervalSeconds,
lastLog: lastLine,
});
if (json) {
console.log(JSON.stringify({ installed, last_log: lastLine }));
console.log(JSON.stringify(report));
} else {
console.log(`Autopilot: ${installed ? 'installed' : 'not installed'}`);
switch (report.state) {
case 'not_installed':
console.log('Autopilot: not installed. Install with `gbrain autopilot --install`.');
break;
case 'disabled':
console.log(`Autopilot: DISABLED — ${report.disabled_reason}`);
console.log(' It stopped itself. Fix the path, then `gbrain autopilot --install --repo <path>`.');
break;
case 'paused':
console.log(`Autopilot: PAUSED — ${report.paused_reason}`);
console.log(' A pause marker is parked at ' + autopilotPausedMarkerPath() + '.');
console.log(' Normal while `gbrain migrate` runs. A marker orphaned by a dead migration');
console.log(' clears itself on the daemon\'s next poll; only remove it by hand if the');
console.log(' pid it names is dead and no daemon is running to clean it up.');
break;
case 'never_run':
console.log(`Autopilot: installed (${report.install_target}) but has NEVER run.`);
break;
case 'stale':
console.log(
`Autopilot: installed (${report.install_target}) but NOT ticking — last heartbeat ` +
`${report.heartbeat_age_seconds}s ago (stale after ${report.stale_after_seconds}s).`,
);
break;
case 'fresh':
console.log(
`Autopilot: running (${report.install_target}) — last heartbeat ` +
`${report.heartbeat_age_seconds}s ago.`,
);
break;
}
if (lastLine) console.log(`Last log: ${lastLine}`);
}
setCliExitVerdict(autopilotStatusExitCode(report.state));
}
function escapeXml(s: string): string {
+52 -35
View File
@@ -48,7 +48,8 @@ import {
import { probeSourceGitState } from '../core/git-head.ts';
// v0.41.32.0: remote staleness reads the stored newest_content_at column via
// this pure comparator (no git subprocess on the HTTP MCP doctor path).
import { lagFromContentMs } from '../core/source-health.ts';
import { lagFromContentMs, resolveStalenessCeilingSeconds } from '../core/source-health.ts';
import { resolveEnvNumber, resolveHoursEnv, warnOnceForEnv } from '../core/env-number.ts';
import { CHUNKER_VERSION } from '../core/chunkers/code.ts';
import { LINK_EXTRACTOR_VERSION_TS } from '../core/link-extraction.ts';
import { isUndefinedColumnError } from '../core/utils.ts';
@@ -3322,10 +3323,6 @@ export async function checkSubagentCapability(engine: BrainEngine): Promise<Chec
const checkSubagentProvider = checkSubagentCapability;
void checkSubagentProvider;
// Module-scoped set so each invalid-env-var warning fires once per process,
// per variable name (v0.42.7 #1696: was a single bool shared across all vars).
const _envNumberWarned = new Set<string>();
/**
* v0.42.7 (#1696): single source of truth for the extraction-lag warn
* threshold (percent). Both the `links_extraction_lag` doctor check AND the
@@ -3339,31 +3336,19 @@ export const EXTRACTION_LAG_WARN_PCT_DEFAULT = 20;
export const EXTRACTION_LAG_MIN_PAGES = 100;
/**
* v0.42.7 (#1696, C1): generic "read a positive number from an env var, warn
* once + fall back on garbage." Extracted from _resolveSyncFreshnessHours so
* the percent-threshold doctor checks don't reuse a `...Hours`-named helper.
* `opts.unit` is purely cosmetic for the warning string ('h', '%', '').
* Exported (D3) so the sync nudge resolves the threshold the same way.
* Re-exported from `src/core/env-number.ts`, which now owns the implementation
* AND the warn-once memo. `source-health.ts` needs the hours resolver for the
* staleness ceiling, and doctor already imports from source-health so the
* helper had to move to core or the import graph would cycle.
*
* The `_resolveEnvNumber` name is kept because `sync.ts:5730` dynamically
* imports it from this module.
*/
export function _resolveEnvNumber(varName: string, fallback: number, opts?: { unit?: string }): number {
const raw = process.env[varName];
if (raw === undefined || raw === '') return fallback;
const n = Number(raw);
if (!Number.isFinite(n) || n <= 0) {
if (!_envNumberWarned.has(varName)) {
_envNumberWarned.add(varName);
console.warn(
`[gbrain doctor] Ignoring invalid ${varName}=${raw}; using default ${fallback}${opts?.unit ?? ''}.`,
);
}
return fallback;
}
return n;
}
export { resolveEnvNumber as _resolveEnvNumber };
function _resolveSyncFreshnessHours(varName: string, fallback: number): number {
return _resolveEnvNumber(varName, fallback, { unit: 'h' });
}
/** Local aliases; the shared memo lives in core so it can't fork per module. */
const _resolveEnvNumber = resolveEnvNumber;
const _resolveSyncFreshnessHours = resolveHoursEnv;
/**
* Sync freshness check (v0.32.4) verify that sources with local_path have
@@ -3757,9 +3742,11 @@ export async function checkLinksExtractionLag(
const n = Number(failRaw);
if (Number.isFinite(n) && n > 0) {
failPct = n;
} else if (!_envNumberWarned.has('GBRAIN_EXTRACTION_LAG_FAIL_PCT')) {
_envNumberWarned.add('GBRAIN_EXTRACTION_LAG_FAIL_PCT');
console.warn(`[gbrain doctor] Ignoring invalid GBRAIN_EXTRACTION_LAG_FAIL_PCT=${failRaw}; hard-fail stays disabled.`);
} else {
warnOnceForEnv(
'GBRAIN_EXTRACTION_LAG_FAIL_PCT',
`[gbrain] Ignoring invalid GBRAIN_EXTRACTION_LAG_FAIL_PCT=${failRaw}; hard-fail stays disabled.`,
);
}
}
@@ -4349,7 +4336,7 @@ export async function checkSyncFreshness(
// bucket). Empty when nothing is syncing — keeps the steady-state messages
// byte-for-byte unchanged.
const inProgress: string[] = [];
let liveSyncSnap: (sourceId: string) => Promise<{ holder_pid: number; holder_host: string } | null> =
let liveSyncSnap: (sourceId: string) => Promise<{ holder_pid: number; holder_host: string; age_ms: number } | null> =
async () => null;
try {
const { inspectLock, syncLockId } = await import('../core/db-lock.ts');
@@ -4357,7 +4344,7 @@ export async function checkSyncFreshness(
try {
const snap = await inspectLock(engine, syncLockId(sourceId));
return snap && !snap.ttl_expired
? { holder_pid: snap.holder_pid, holder_host: snap.holder_host }
? { holder_pid: snap.holder_pid, holder_host: snap.holder_host, age_ms: snap.age_ms }
: null;
} catch {
return null;
@@ -4367,6 +4354,10 @@ export async function checkSyncFreshness(
/* db-lock unavailable — skip in-progress detection, staleness stands. */
}
// One ceiling for the whole report: hoisted out of the loop so every
// source is judged against the same number (and the env read + warn-once
// machinery runs once, not once per source).
const stalenessCeilingSeconds = resolveStalenessCeilingSeconds();
for (const source of sources) {
// Embed source.id in user-visible messages so `gbrain sync --source <id>`
// matches what the user copy-pastes. Show display name in parens when set.
@@ -4376,10 +4367,35 @@ export async function checkSyncFreshness(
// BUG 4: actively syncing (live lock) → healthy, count as synced_recently
// and skip the staleness checks. Keeps the 3-bucket invariant intact.
//
// ...but ONLY up to the staleness ceiling. `withRefreshingLock` bumps the
// heartbeat on its own timer regardless of whether the import is making
// forward progress (`liveSyncStatus`'s docstring is explicit: callers may
// report "running", NOT "healthy"). So a holder blocked inside a query
// keeps refreshing forever, and an uncapped in-progress verdict would
// mask that source from every staleness check indefinitely — the same
// invisible-failure class this whole pass exists to close, just reached
// through the lock table instead of the freshness column.
const liveSnap = await liveSyncSnap(source.id);
if (liveSnap) {
inProgress.push(`${display} sync in progress (pid ${liveSnap.holder_pid} on ${liveSnap.holder_host})`);
synced_recently_count++;
const ceilingMs = stalenessCeilingSeconds * 1000;
if (liveSnap.age_ms <= ceilingMs) {
inProgress.push(`${display} sync in progress (pid ${liveSnap.holder_pid} on ${liveSnap.holder_host})`);
synced_recently_count++;
continue;
}
// Sub-hour ceilings are legal (fractional env override), and an alarm
// that names a zero duration ("held the lock for 0h") reads as broken.
const heldFor = liveSnap.age_ms >= 3600_000
? `${Math.floor(liveSnap.age_ms / 3600_000)}h`
: `${Math.max(1, Math.floor(liveSnap.age_ms / 60_000))}m`;
issues.push(
`Source ${display} has held the sync lock for ${heldFor} ` +
`(pid ${liveSnap.holder_pid} on ${liveSnap.holder_host}) — heartbeating but not finishing. ` +
`Run \`gbrain sync --break-lock --source ${source.id}\` after confirming the holder is wedged.`,
);
hasFailures = true;
stale_count++;
continue;
}
@@ -4468,6 +4484,7 @@ export async function checkSyncFreshness(
contentMs !== null && Number.isFinite(contentMs) ? contentMs : null,
lastSync,
now,
stalenessCeilingSeconds,
);
thresholdAgeMs = lagSec === null ? ageMs : lagSec * 1000;
}
+39
View File
@@ -492,7 +492,46 @@ export function parsePaceArgs(
return { ...(perCallMode !== undefined && { perCallMode }), ...(perCall && { perCall }) };
}
/**
* Keyless brains: does this invocation qualify for the stale-mode CLEAN
* refusal (zero-failure result exit 0)? The documented always-current chain
* for external agent schedulers is `gbrain sync ... && gbrain embed --stale`
* (docs/guides/live-sync.md, INSTALL_FOR_AGENTS.md Step 7) a hard exit 1
* here broke that chain on every brain installed keyless (init's no-embedding
* mode), which docs/operations/headless-install.md calls a first-class
* supported end state whose embed paths "refuse cleanly". Same class as
* sync's resolveNoEmbed sentinel (sync.ts), same remedy. Deliberately NARROW:
* only the stale spelling (the chain's). An explicit slug, a slugs list, or
* the all flag is an explicit request for something impossible on a keyless
* brain and keeps exiting 1 via EmbeddingDisabledError. Note the slugs-list
* exclusion mirrors the dispatch precedence below: a slugs flag wins over
* stale, so a combined invocation is an explicit-slugs run, not a stale run.
*/
export function isKeylessStaleRefusal(args: string[], embeddingDisabled: boolean | undefined): boolean {
return args.includes('--stale')
&& !args.includes('--all')
&& !args.includes('--slugs')
&& !args.includes('--dry-run')
&& embeddingDisabled === true;
}
export async function runEmbed(engine: BrainEngine, args: string[]): Promise<EmbedResult | undefined> {
// Keyless clean refusal — see isKeylessStaleRefusal. Checked BEFORE the
// background block so we never queue a job that can only fail. stderr only;
// stdout stays empty like every other embed outcome (embed has no JSON
// result surface — do not invent one here).
if (isKeylessStaleRefusal(args, loadConfig()?.embedding_disabled)) {
process.stderr.write(
'[embed] Embeddings are disabled on this brain (keyless install). '
+ 'Nothing to backfill; keyword search keeps working. '
+ 'Enable later: set embedding_model via gbrain config, then re-run gbrain init with the force flag.\n',
);
return {
embedded: 0, skipped: 0, would_embed: 0, total_chunks: 0,
pages_processed: 0, failures: 0, failure_samples: [], dryRun: false,
};
}
// v0.36+ T7: --background submits via Minion queue, returns job_id to
// stdout, exits. Same semantics in TTY and cron (D9).
if (args.includes('--background')) {
+405 -123
View File
@@ -11,12 +11,19 @@ import { createEngine } from '../core/engine-factory.ts';
import { loadConfig, saveConfig, toEngineConfig, gbrainPath, effectiveEnvDatabaseUrl, type GBrainConfig } from '../core/config.ts';
import type { BrainEngine } from '../core/engine.ts';
import type { EngineConfig, Page } from '../core/types.ts';
import { writeFileSync, readFileSync, existsSync, unlinkSync } from 'fs';
import { writeFileSync, readFileSync, existsSync, unlinkSync, statSync, mkdirSync, renameSync } from 'fs';
import { createHash } from 'crypto';
import { resolve } from 'path';
import { resolve, dirname } from 'path';
import { createProgress } from '../core/progress.ts';
import { getCliOptions, cliOptsToProgressOptions } from '../core/cli-options.ts';
import { setCliExitVerdict } from '../core/cli-force-exit.ts';
import { registerCleanup } from '../core/process-cleanup.ts';
// LEAF module, deliberately NOT './autopilot.ts': the flag-registry generator
// follows a command's dynamic imports and would fold autopilot's entire flag
// surface into `migrate`'s accepted-flag allowlist.
import { autopilotPausedMarkerPath, autopilotLockPath, markerHolderAlive, MIGRATE_PAUSE_MARKER_PREFIX } from '../core/autopilot-paths.ts';
export { MIGRATE_PAUSE_MARKER_PREFIX };
import { listLiveLocks } from '../core/db-lock.ts';
interface MigrateOpts {
targetEngine: 'postgres' | 'pglite';
@@ -237,6 +244,225 @@ export interface MigratePageFailure {
reason: string;
}
/**
* Quiesce a running autopilot daemon for the duration of the copy.
*
* Why this brackets the copy instead of trailing it: `saveConfig` runs at the
* very END of the migration, gated on zero failures. A live daemon therefore
* keeps syncing and embedding into the SOURCE engine for the entire copy window,
* and everything it writes after page enumeration is silently lost the moment
* the config flips. `migrate-engine.ts` previously contained ZERO references to
* autopilot, which is the root cause of the observed incident: an engine
* migration rewrote `config.json` while a daemon built on the old config kept
* running, crashed on the stale object, and nothing linked the two events.
*
* Cooperative marker, not a supervisor stop see `autopilotPausedMarkerPath`.
* Returns a resume function that is safe to call twice.
*/
/**
* A twice-safe, ownership-conditional unlinker for the pause marker: it
* deletes the marker ONLY while the content is still the exact body this
* process wrote. If the marker changed hands (an adoption race resolved
* against us), deleting by path would un-pause the daemon in the middle of
* the new owner's copy window the precise failure the fence exists to
* stop. (The read-then-unlink pair has a microscopic window of its own;
* every layer here shrinks the race multiplicatively rather than claiming
* to erase it.)
*/
function markerRelease(marker: string, ownBody: string): () => void {
let released = false;
return () => {
if (released) return;
released = true;
try {
if (readFileSync(marker, 'utf8') !== ownBody) {
console.log('[migrate] pause marker changed hands; leaving it for its new owner.');
return;
}
} catch { return; /* already gone */ }
try { unlinkSync(marker); console.log('[migrate] autopilot resumed.'); }
catch { /* already gone */ }
};
}
/** Our marker body: prefix + ownership pid + timestamp. */
function markerBody(): string {
return `${MIGRATE_PAUSE_MARKER_PREFIX} (pid ${process.pid}) at ${new Date().toISOString()}\n`;
}
/**
* Drain cap in seconds. Empty/unset default; `0` disables the wait
* entirely; garbage warns and falls back (an env typo must not silently
* remove the drain).
*/
function resolveDrainCapSeconds(): number {
const raw = process.env.GBRAIN_MIGRATE_QUIESCE_SECONDS;
if (raw === undefined || raw === '') return 300;
if (raw === '0') return 0;
const n = Number(raw);
if (!Number.isFinite(n) || n < 0) {
console.warn(`[migrate] ignoring invalid GBRAIN_MIGRATE_QUIESCE_SECONDS=${JSON.stringify(raw)}; using 300.`);
return 300;
}
return n;
}
/**
* Returns a resume closure, or NULL when the pause is owned by someone else
* in which case the caller must ABORT: proceeding without owning the pause
* means the real owner's cleanup un-pauses the daemon in the middle of OUR
* copy window, and two migrations racing one manifest/target corrupt both.
*/
export async function quiesceAutopilot(engine?: BrainEngine): Promise<(() => void) | null> {
const marker = autopilotPausedMarkerPath();
// The marker is written UNCONDITIONALLY — not gated on the daemon's lock
// file existing. The lock is unlinked on every clean daemon exit (self
// upgrade swap, supervisor restart, SIGTERM) and the supervisor relaunches
// within a minute, so "no lock right now" recurs routinely on healthy
// installs; cron installs have no lock BETWEEN runs, which is most of the
// time. A migrate that skipped the marker in such a window would let the
// (re)started daemon run full cycles into the source engine for the whole
// copy — all silently lost at the config flip. A marker on a machine with
// no autopilot at all costs one tiny file, removed by the finally.
mkdirSync(dirname(marker), { recursive: true });
// Claim it atomically ('wx' = fail if it exists) — an existsSync pre-check
// would race a concurrent migrate/pauser between check and write. The pid
// in the body is the ownership handle for orphan adoption below.
let claimed = false;
const ownBody = markerBody();
try {
writeFileSync(marker, ownBody, { flag: 'wx' });
claimed = true;
} catch (e) {
if ((e as NodeJS.ErrnoException)?.code === 'EEXIST') {
// A marker already exists. Adopt it ONLY when it was written by a
// migrate whose process is provably dead — refusing to adopt a dead
// run's orphan would make it permanent (every retry defers to it and
// the daemon sits paused forever), while adopting a LIVE run's marker
// would be worse: our resume would un-pause the daemon in the middle
// of the other migrate's copy window. Content prefix alone cannot tell
// those apart; the pid can.
try {
const body = readFileSync(marker, 'utf8');
if (body.startsWith(MIGRATE_PAUSE_MARKER_PREFIX) && markerHolderAlive(body) === 'dead') {
// Take OWNERSHIP atomically. Two migrates racing the same orphan
// both read the dead pid; a plain rewrite would let BOTH claim it
// and one's cleanup would un-pause the daemon mid-copy for the
// other. rename(2) is the compare-and-claim: exactly one racer
// moves the orphan aside; the loser gets ENOENT and aborts. The
// fresh write then goes through the same exclusive 'wx' gate, so a
// third claimant sneaking into the gap loses cleanly too.
const tomb = `${marker}.reclaim.${process.pid}`;
renameSync(marker, tomb);
// Verify identity AFTER the rename: between our read above and the
// rename, a racing adopter may have completed the whole
// adopt-and-rewrite — in which case we just renamed THEIR live
// marker, not the orphan. Restore it no-clobber and bow out.
let tombBody = '';
try { tombBody = readFileSync(tomb, 'utf8'); } catch { /* vanished */ }
if (tombBody.startsWith(MIGRATE_PAUSE_MARKER_PREFIX) && markerHolderAlive(tombBody) === 'dead') {
try { unlinkSync(tomb); } catch { /* best-effort */ }
writeFileSync(marker, ownBody, { flag: 'wx' });
console.log('[migrate] adopting a pause marker left by a dead migrate run.');
claimed = true;
} else {
try { writeFileSync(marker, tombBody, { flag: 'wx' }); } catch { /* a racer claimed the gap; theirs now */ }
try { unlinkSync(tomb); } catch { /* best-effort */ }
}
}
} catch { /* lost an adoption race, or unreadable: treat as foreign */ }
if (!claimed) {
// Live migrate, operator hold, or unreadable: NOT ours. Abort rather
// than run unowned — see the null contract above.
return null;
}
} else {
// EACCES / EROFS / ENOSPC: fail CLOSED, same as the foreign-marker
// case. Proceeding unquiesced here would let a running (or supervisor
// relaunched) daemon write into the source for the entire copy — all
// silently lost at the flip, the exact incident this marker prevents.
console.error(`[migrate] cannot write the autopilot pause marker at ${marker}: ${e instanceof Error ? e.message : String(e)}`);
return null;
}
}
// Cleanup registered IMMEDIATELY after the claim — a SIGTERM during the
// drain below must not orphan the marker. (The daemon also self-clears a
// migrate-prefixed marker whose recorded pid is dead, so even a SIGKILL
// here heals within one daemon poll.)
const release = markerRelease(marker, ownBody);
const deregister = registerCleanup('migrate-autopilot-resume', async () => release());
const resume = () => { release(); deregister(); };
let heartbeatAgeMs = Infinity;
try { heartbeatAgeMs = Date.now() - statSync(autopilotLockPath()).mtimeMs; }
catch { /* no lock: no live daemon */ }
console.log(heartbeatAgeMs < 600_000
? '[migrate] autopilot detected and paused for the duration of the copy.'
: '[migrate] autopilot pause marker parked for the copy window (no live daemon detected; covers one starting mid-copy).');
// Drain in-flight writers instead of sleeping a blind grace: cycle runs,
// sync imports, and embed backfills all hold live rows in the DB lock
// table while they work, so "no live locks" is the actual quiesced signal.
// An idle daemon holds nothing and the drain returns immediately (faster
// than any fixed sleep); a mid-cycle daemon is waited on up to the cap.
// Fail-open: a probe error must never block a migration — the marker still
// parks the NEXT cycle, which was the whole pre-drain guarantee.
const capMs = resolveDrainCapSeconds() * 1000;
if (engine && capMs > 0) {
const started = Date.now();
let lastLog = 0;
for (;;) {
let live: Awaited<ReturnType<typeof listLiveLocks>>;
try {
live = (await listLiveLocks(engine))
// The supervisor lock is PRESENCE, not work: a healthy minion
// supervisor holds it continuously, so counting it would park
// every drain at the cap and cry wolf. Actual work (cycle, sync,
// embed backfill, unify, ...) holds its own rows.
.filter((l) => !l.id.startsWith('gbrain-supervisor:'));
} catch (e) {
console.warn(`[migrate] could not probe the lock table (${e instanceof Error ? e.message : String(e)}); relying on the pause marker alone.`);
break;
}
// A claimed minion job writes through the worker process; count healthy
// active jobs as writers too. Fail-open — a missing queue table (older
// schema) must not block the migration.
try {
const rows = await engine.executeRaw(
`SELECT count(*)::text AS count FROM minion_jobs WHERE status = 'active' AND lock_until > now()`,
) as Array<{ count?: string }>;
const activeJobs = Number(rows?.[0]?.count ?? 0);
if (Number.isFinite(activeJobs) && activeJobs > 0) {
live = [...live, { id: `minion-jobs-active:${activeJobs}` } as (typeof live)[number]];
}
} catch { /* no queue table: nothing to count */ }
if (live.length === 0) {
if (started !== lastLog && lastLog !== 0) console.log('[migrate] in-flight work drained; starting the copy.');
break;
}
const waited = Date.now() - started;
if (waited >= capMs) {
console.warn(
`[migrate] ${live.length} writer lock(s) still live after ${Math.round(waited / 1000)}s (${live.map((l) => l.id).join(', ')}). ` +
'Proceeding — anything they write during the copy will NOT be carried to the target. ' +
'For large migrations, stop autopilot and idle the job queue first.',
);
break;
}
if (waited - lastLog >= 30_000) {
lastLog = waited;
console.log(`[migrate] waiting for in-flight work to drain: ${live.map((l) => l.id).join(', ')} (${Math.round((capMs - waited) / 1000)}s left)...`);
}
await new Promise((r) => setTimeout(r, 3_000));
}
}
return resume;
}
export async function runMigrateEngine(sourceEngine: BrainEngine, args: string[]): Promise<void> {
const opts = parseArgs(args);
const config = loadConfig();
@@ -251,6 +477,20 @@ export async function runMigrateEngine(sourceEngine: BrainEngine, args: string[]
process.exit(1);
}
// Quiesce FIRST — before the target is even connected. The pause marker
// doubles as the migrate mutex: a second migrate aborts here instead of
// racing this one's manifest and target (a concurrent force run could
// delete pages the first run already copied). It also means everything
// below (target initSchema, manifest handling, the copy) runs with the
// daemon parked. Every process.exit in this function is above this line;
// the paths below it return through the finally instead.
const resumeAutopilot = await quiesceAutopilot(sourceEngine);
if (!resumeAutopilot) {
console.error('Could not take ownership of the autopilot pause (a concurrent migrate, an operator hold, or an unwritable marker — see above).');
console.error(`Refusing to migrate without it: a daemon this migration cannot pause keeps writing into the source engine, and those writes are lost at the flip. Inspect ${autopilotPausedMarkerPath()} if you are sure nothing is running.`);
process.exit(1);
}
// Build target config
const targetConfig: EngineConfig = { engine: opts.targetEngine };
if (opts.targetEngine === 'postgres') {
@@ -258,7 +498,11 @@ export async function runMigrateEngine(sourceEngine: BrainEngine, args: string[]
targetConfig.database_url = opts.targetUrl || effectiveEnvDatabaseUrl();
if (!targetConfig.database_url) {
console.error('Target is Supabase but no connection string provided. Use: --url <connection_string>');
process.exit(1);
// Not process.exit: that would skip the resume in the finally below and
// leave the daemon parked on the pause marker.
setCliExitVerdict(1);
resumeAutopilot();
return;
}
} else {
targetConfig.database_path = opts.targetPath || gbrainPath('brain.pglite');
@@ -307,7 +551,10 @@ export async function runMigrateEngine(sourceEngine: BrainEngine, args: string[]
console.error(`Target brain is not empty (${targetStats.page_count} pages).`);
console.error('Run with --force to overwrite, or migrate to an empty brain.');
await targetEngine.disconnect();
process.exit(1);
// Not process.exit: the resume must run (see the quiesce block above).
setCliExitVerdict(1);
resumeAutopilot();
return;
} else if (targetStats.page_count > 0 && resumingMatchingManifest) {
console.log(`Resuming previous migration: ${manifest!.completed_slugs.length} page(s) already copied.`);
}
@@ -339,133 +586,168 @@ export async function runMigrateEngine(sourceEngine: BrainEngine, args: string[]
// Pages.source_id is a foreign key. Copy the complete source catalog first,
// including archived rows and sync/routing metadata, so every page write has
// a valid parent and the target preserves source behavior.
await copyMigrationSources(sourceEngine, targetEngine);
// Get all source pages
const sourceStats = await sourceEngine.getStats();
const allPages = await sourceEngine.listPages({ limit: 100000 });
const pagesToMigrate = allPages.filter(p => !completedSet.has(makeManifestKey(p.source_id, p.slug)));
console.log(`Migrating ${pagesToMigrate.length} pages (${allPages.length} total, ${completedSet.size} already done)...`);
const progress = createProgress(cliOptsToProgressOptions(getCliOptions()));
progress.start('migrate.copy_pages', pagesToMigrate.length);
// v0.32.8 F8: thread source_id end-to-end so multi-source pages migrate
// intact. Pre-fix: putPage / getTags / getTimeline / getRawData / getLinks
// all silently defaulted to source_id='default', so non-default-source
// tags / timeline / raw / links were either dropped or attached to the
// wrong row.
//
// The pause marker must not outlive this process. `registerCleanup` covers
// signals and UNhandled rejections, but a caught-and-rethrown error (or any
// handled failure path) exits through normal unwinding — without this
// finally, that path leaks the marker and the daemon idles forever while
// the status command keeps reporting fresh. On success, finally still runs
// AFTER the config flip above, preserving the resume-after-flip ordering:
// resuming any earlier would let the daemon write into the source engine
// again, and those writes would be the exact ones the migration cannot
// carry over.
//
// Known residual: the finally fires before verifyTarget below, so a fast
// supervisor relaunch can touch the target during verification — verify is
// read-only, so the cost is at most a noisy count, not corruption.
//
// NOTE the daemon still holds the PRE-migration config in memory, and its
// health probe keeps succeeding against the old engine (the backup stays
// alive), so it would never reconnect on its own. Its tick loop compares
// the file-plane engine identity against what it booted with
// (`autopilotEngineIdentity`) and exits cleanly for supervisor relaunch on
// the new config — that check, not reconnect(), is what converges it.
let sourceStats!: Awaited<ReturnType<BrainEngine['getStats']>>;
let pagesToMigrate: Page[] = [];
let migrated = 0;
const failures: MigratePageFailure[] = [];
for (const page of pagesToMigrate) {
try {
await copyPageToTarget(sourceEngine, targetEngine, page);
// Track progress with composite key so multi-source resume is correct.
manifest!.completed_slugs.push(makeManifestKey(page.source_id, page.slug));
saveManifest(manifest!);
migrated++;
} catch (e) {
// #3194: a per-page write failure must never be swallowed into the
// success count. Leave it OUT of completed_slugs (a resume retries
// it — putPage/upsertChunks/etc. are all upserts, so re-running the
// whole page copy is safe) and surface it in the final summary below
// instead of letting "N pages copied" imply everything landed.
failures.push({
source_id: page.source_id,
slug: page.slug,
reason: e instanceof Error ? e.message : String(e),
});
}
progress.tick(1, page.slug);
}
progress.finish();
try {
await copyMigrationSources(sourceEngine, targetEngine);
if (failures.length > 0) {
console.error(`\n${failures.length} of ${pagesToMigrate.length} page(s) FAILED to copy and were NOT migrated:`);
for (const f of failures) {
const key = f.source_id === 'default' ? f.slug : `${f.source_id}::${f.slug}`;
console.error(` - ${key}: ${f.reason}`);
}
console.error('Re-run `gbrain migrate` to retry the failed pages (already-copied pages resume via the manifest).');
// Non-fatal so the run still copies links + config for everything that
// DID land, but the process must exit non-zero — a partial migration
// must never look identical to a clean one.
setCliExitVerdict(1);
}
// Get all source pages
sourceStats = await sourceEngine.getStats();
const allPages = await sourceEngine.listPages({ limit: 100000 });
pagesToMigrate = allPages.filter(p => !completedSet.has(makeManifestKey(p.source_id, p.slug)));
// Copy links (after all pages exist in target).
// v0.32.8 F8: thread source_id so cross-source links migrate correctly.
// #3194: a page that failed to copy above does NOT exist on the target,
// so any link touching it would violate the target's FK and abort this
// whole phase (the exact "addLink failed: page ... not found" crash from
// the original report). Skip links on either end of a known-failed page —
// a retry that successfully copies the page also re-copies its links.
const failedKeys = new Set(failures.map(f => makeManifestKey(f.source_id, f.slug)));
console.log('Copying links...');
progress.start('migrate.copy_links', allPages.length);
for (const page of allPages) {
if (failedKeys.has(makeManifestKey(page.source_id, page.slug))) {
console.log(`Migrating ${pagesToMigrate.length} pages (${allPages.length} total, ${completedSet.size} already done)...`);
const progress = createProgress(cliOptsToProgressOptions(getCliOptions()));
progress.start('migrate.copy_pages', pagesToMigrate.length);
// v0.32.8 F8: thread source_id end-to-end so multi-source pages migrate
// intact. Pre-fix: putPage / getTags / getTimeline / getRawData / getLinks
// all silently defaulted to source_id='default', so non-default-source
// tags / timeline / raw / links were either dropped or attached to the
// wrong row.
for (const page of pagesToMigrate) {
try {
await copyPageToTarget(sourceEngine, targetEngine, page);
// Track progress with composite key so multi-source resume is correct.
manifest!.completed_slugs.push(makeManifestKey(page.source_id, page.slug));
saveManifest(manifest!);
migrated++;
} catch (e) {
// #3194: a per-page write failure must never be swallowed into the
// success count. Leave it OUT of completed_slugs (a resume retries
// it — putPage/upsertChunks/etc. are all upserts, so re-running the
// whole page copy is safe) and surface it in the final summary below
// instead of letting "N pages copied" imply everything landed.
failures.push({
source_id: page.source_id,
slug: page.slug,
reason: e instanceof Error ? e.message : String(e),
});
}
progress.tick(1, page.slug);
}
progress.finish();
if (failures.length > 0) {
console.error(`\n${failures.length} of ${pagesToMigrate.length} page(s) FAILED to copy and were NOT migrated:`);
for (const f of failures) {
const key = f.source_id === 'default' ? f.slug : `${f.source_id}::${f.slug}`;
console.error(` - ${key}: ${f.reason}`);
}
console.error('Re-run `gbrain migrate` to retry the failed pages (already-copied pages resume via the manifest).');
// Non-fatal so the run still copies links + config for everything that
// DID land, but the process must exit non-zero — a partial migration
// must never look identical to a clean one.
setCliExitVerdict(1);
}
// Copy links (after all pages exist in target).
// v0.32.8 F8: thread source_id so cross-source links migrate correctly.
// #3194: a page that failed to copy above does NOT exist on the target,
// so any link touching it would violate the target's FK and abort this
// whole phase (the exact "addLink failed: page ... not found" crash from
// the original report). Skip links on either end of a known-failed page —
// a retry that successfully copies the page also re-copies its links.
const failedKeys = new Set(failures.map(f => makeManifestKey(f.source_id, f.slug)));
console.log('Copying links...');
progress.start('migrate.copy_links', allPages.length);
for (const page of allPages) {
if (failedKeys.has(makeManifestKey(page.source_id, page.slug))) {
progress.tick(1);
continue;
}
const sourceOpts = { sourceId: page.source_id };
const links = await sourceEngine.getLinks(page.slug, sourceOpts);
for (const link of links) {
if (failedKeys.has(makeManifestKey(page.source_id, link.to_slug))) continue;
await targetEngine.addLink(
link.from_slug, link.to_slug,
link.context, link.link_type,
undefined, undefined, undefined,
{ fromSourceId: page.source_id, toSourceId: page.source_id },
);
}
progress.tick(1);
continue;
}
const sourceOpts = { sourceId: page.source_id };
const links = await sourceEngine.getLinks(page.slug, sourceOpts);
for (const link of links) {
if (failedKeys.has(makeManifestKey(page.source_id, link.to_slug))) continue;
await targetEngine.addLink(
link.from_slug, link.to_slug,
link.context, link.link_type,
undefined, undefined, undefined,
{ fromSourceId: page.source_id, toSourceId: page.source_id },
);
progress.finish();
// Copy config (selective).
//
// v0.37 fix wave Lane C.4: these DB-plane writes are SCHEMA METADATA for
// the target engine — they record "the schema was sized using this
// embedding model + dimension." They are NOT the runtime gateway config
// (which lives in the file plane via `~/.gbrain/config.json`). When this
// function copies them, it's preserving the schema-applied state across
// the migration, not re-pointing the gateway. The newConfig below
// doesn't carry these fields because the user's existing file config
// already has them (or didn't, in which case the file plane should stay
// unset and re-read from gateway defaults).
const configKeys = ['embedding_model', 'embedding_dimensions', 'chunk_strategy'];
for (const key of configKeys) {
const val = await sourceEngine.getConfig(key);
if (val) await targetEngine.setConfig(key, val);
}
progress.tick(1);
}
progress.finish();
// Copy config (selective).
//
// v0.37 fix wave Lane C.4: these DB-plane writes are SCHEMA METADATA for
// the target engine — they record "the schema was sized using this
// embedding model + dimension." They are NOT the runtime gateway config
// (which lives in the file plane via `~/.gbrain/config.json`). When this
// function copies them, it's preserving the schema-applied state across
// the migration, not re-pointing the gateway. The newConfig below
// doesn't carry these fields because the user's existing file config
// already has them (or didn't, in which case the file plane should stay
// unset and re-read from gateway defaults).
const configKeys = ['embedding_model', 'embedding_dimensions', 'chunk_strategy'];
for (const key of configKeys) {
const val = await sourceEngine.getConfig(key);
if (val) await targetEngine.setConfig(key, val);
}
// Update local config. v0.37 fix wave: preserve existing file-plane
// embedding/expansion/chat config across the engine migration; only
// the engine + connection target should change.
//
// #3194: only flip the ACTIVE config when the migration is fully clean.
// A partial migration leaves the target's data incomplete; auto-switching
// every subsequent `gbrain` invocation onto that incomplete target would
// (a) make the failure invisible behind otherwise-normal usage and (b)
// break the natural retry — `gbrain migrate --to X` again would hit the
// "Already using X engine" guard even though the migration never actually
// finished. Leaving the file-plane config untouched keeps the source the
// active engine, so a retry (which resumes via the still-intact manifest)
// is a same-shaped command, not a special case.
if (failures.length === 0) {
const existingFile = (await import('../core/config.ts')).loadConfigFileOnly() ?? ({} as GBrainConfig);
const newConfig: GBrainConfig = {
...existingFile,
engine: opts.targetEngine,
...(opts.targetEngine === 'postgres'
? { database_url: targetConfig.database_url, database_path: undefined }
: { database_path: targetConfig.database_path, database_url: undefined }),
};
saveConfig(newConfig);
// Clean up the resume manifest — only safe once nothing is left pending.
clearManifest();
// Update local config. v0.37 fix wave: preserve existing file-plane
// embedding/expansion/chat config across the engine migration; only
// the engine + connection target should change.
//
// #3194: only flip the ACTIVE config when the migration is fully clean.
// A partial migration leaves the target's data incomplete; auto-switching
// every subsequent `gbrain` invocation onto that incomplete target would
// (a) make the failure invisible behind otherwise-normal usage and (b)
// break the natural retry — `gbrain migrate --to X` again would hit the
// "Already using X engine" guard even though the migration never actually
// finished. Leaving the file-plane config untouched keeps the source the
// active engine, so a retry (which resumes via the still-intact manifest)
// is a same-shaped command, not a special case.
if (failures.length === 0) {
const existingFile = (await import('../core/config.ts')).loadConfigFileOnly() ?? ({} as GBrainConfig);
const newConfig: GBrainConfig = {
...existingFile,
engine: opts.targetEngine,
...(opts.targetEngine === 'postgres'
? { database_url: targetConfig.database_url, database_path: undefined }
: { database_path: targetConfig.database_path, database_url: undefined }),
};
saveConfig(newConfig);
// An exported connection-string env var overrides the file config for
// every gbrain process (including the relaunched daemon), which would
// silently keep the whole brain on the OLD engine after this flip.
const envUrl = effectiveEnvDatabaseUrl();
if (envUrl && envUrl !== targetConfig.database_url) {
console.warn('WARNING: a DATABASE_URL-style env var is exported and does not match the migration target.');
console.warn('It overrides config.json for every gbrain process — unset it (shell profile, launchd plist, systemd unit) or the brain stays on the old engine.');
}
// Clean up the resume manifest — only safe once nothing is left pending.
clearManifest();
}
} finally {
resumeAutopilot();
}
if (failures.length > 0) {
+12
View File
@@ -5271,6 +5271,14 @@ export interface SyncStatusReportSource {
local_path: string | null;
sync_enabled: boolean;
last_sync_at: string | null;
/** Raw wall-clock hours since the last successful sync the honest human
* number. Distinct from staleness_hours, which is threshold-relative. */
hours_since_last_sync: number | null;
/** Threshold-relative lag driving staleness_class. For a source whose
* content is OLDER than its last sync this is the ceiling-ramped value
* (see lagFromContentMs), which deliberately under-reads raw wall-clock so
* the warn tier fires before the fail tier display hours_since_last_sync
* when a human asks "how long since we synced". */
staleness_hours: number | null;
staleness_class: 'fresh' | 'stale' | 'severe' | 'unknown';
last_commit: string | null;
@@ -5446,6 +5454,9 @@ export async function buildSyncStatusReport(
now,
);
const stalenessHours = lagSeconds === null ? null : lagSeconds / 3600;
const hoursSinceLastSync = lastSyncMs !== null && Number.isFinite(lastSyncMs)
? Math.round(((now - lastSyncMs) / 3600_000) * 10) / 10
: null;
let stalenessClass: 'fresh' | 'stale' | 'severe' | 'unknown' = 'unknown';
if (stalenessHours !== null) {
if (stalenessHours < 24) stalenessClass = 'fresh';
@@ -5464,6 +5475,7 @@ export async function buildSyncStatusReport(
local_path: src.local_path,
sync_enabled: cfgEntry.syncEnabled !== false,
last_sync_at: lastSyncIso,
hours_since_last_sync: hoursSinceLastSync,
staleness_hours: stalenessHours === null ? null : Math.round(stalenessHours * 10) / 10,
staleness_class: stalenessClass,
last_commit: row.last_commit,
+115
View File
@@ -0,0 +1,115 @@
/**
* Filesystem coordination points for the autopilot daemon.
*
* A LEAF module (imports only `path` + `config`) so other commands can read the
* daemon's state files WITHOUT importing `src/commands/autopilot.ts`.
*
* Why that matters concretely: the CLI flag-registry generator follows a
* command's dynamic `import('./x.ts')` one level deep and harvests every flag
* literal it finds. A single `await import()` of the autopilot command module inside
* `migrate-engine.ts` therefore folded autopilot's ENTIRE flag surface (install,
* uninstall, interval, no-worker, status, repo, ...) into the migrate command's
* accepted-flag allowlist, which would have made `gbrain migrate` silently
* accept and ignore any of them. Pinned by `test/cli-flag-validation.test.ts`.
*
* Flag names appear here WITHOUT leading dashes on purpose the generator
* scans comments too, so writing them literally recreates the very bug this
* comment describes.
*
* Everything here resolves through `gbrainPath` (NOT raw `process.env.HOME`)
* because that is where the daemon itself writes; a `GBRAIN_HOME` install
* otherwise has readers looking in a different directory than the writer.
*/
import { join } from 'path';
import { gbrainPath } from './config.ts';
/**
* The daemon's lock file. Its mtime IS the liveness heartbeat the tick loop
* refreshes it every pass so "is autopilot alive?" is `now - mtime` against
* the expected interval, with no scheduler probing required.
*/
export function autopilotLockPath(): string {
return gbrainPath('autopilot.lock');
}
/**
* Written by the generated wrapper when it self-disables (its captured repo path
* is gone). Read by the status command so a stopped daemon explains itself
* instead of looking merely idle.
*/
export function autopilotDisabledMarkerPath(): string {
return join(gbrainPath(), 'autopilot-disabled');
}
/**
* The launchd job label / plist basename (systemd keeps its own unit name).
*
* The env override is a TEST SEAM, not a user knob: it exists so the real-launchd
* lifecycle e2e can load a genuinely unique job on a dev Mac without colliding
* with or tearing down the machine's actual autopilot install. It is read at
* CALL time, so every invocation touching the same install (install, status,
* uninstall, the generated wrapper's self-disable) must run with the same value;
* a mismatched pair reports a false not-installed or misses the plist.
*
* The grammar check is load-bearing: the label reaches a plist FILENAME and a
* double-quoted shell line inside the generated wrapper. `escapeXml` protects
* only the XML site, so slashes, quotes, whitespace, `$`, backticks, and
* dot-dot must be rejected here.
*/
export function autopilotLaunchdLabel(): string {
const label = process.env.GBRAIN_AUTOPILOT_LABEL ?? 'com.gbrain.autopilot';
if (!/^[A-Za-z0-9._-]+$/.test(label) || label.includes('..')) {
throw new Error(`invalid GBRAIN_AUTOPILOT_LABEL: ${JSON.stringify(label)}`);
}
return label;
}
/**
* Cooperative pause. While present the tick loop keeps heartbeating but does no
* work. `gbrain migrate` uses it to quiesce the daemon for the
* duration of a cross-engine copy, so the daemon cannot keep writing into an
* engine that is about to stop being the configured one.
*
* A marker rather than stopping the supervisor: ephemeral-container installs
* have no supervisor to restart afterwards, a crash mid-migration leaves a
* removable file rather than an uninstalled daemon, and it is one code path
* across launchd / systemd / cron / container.
*/
export function autopilotPausedMarkerPath(): string {
return join(gbrainPath(), 'autopilot-paused');
}
/**
* Consecutive-miss counter for the wrapper's self-disable guard. A repo on an
* external volume, NFS, or a cloud-synced folder is routinely absent for the
* first daemon launch after login, and a single missed probe must not
* permanently take the install out of rotation the guard requires several
* consecutive strikes before disabling, and any successful probe resets this
* file.
*/
export function autopilotDisableStrikesPath(): string {
return join(gbrainPath(), 'autopilot-disable-strikes');
}
/**
* First line of every pause marker written by `gbrain migrate` the ownership
* signature that separates a migrate-owned pause (safe to adopt when its pid
* is dead) from an operator's manual hold (never touched).
*/
export const MIGRATE_PAUSE_MARKER_PREFIX = 'paused by gbrain migrate';
/**
* Is the pid recorded in a pause-marker body still a live process?
* ESRCH = provably dead. EPERM = exists but owned by another user: alive.
* No parseable pid = unknown (treat as a foreign hold; adoption needs proof).
*/
export function markerHolderAlive(body: string): 'alive' | 'dead' | 'unknown' {
const m = /\(pid (\d+)\)/.exec(body);
if (!m) return 'unknown';
try {
process.kill(Number(m[1]), 0);
return 'alive';
} catch (e) {
return (e as NodeJS.ErrnoException)?.code === 'ESRCH' ? 'dead' : 'alive';
}
}
+4 -4
View File
@@ -13,7 +13,7 @@ export const CLI_FLAG_REGISTRY: Record<string, readonly string[]> = {
'anomalies': ['--aliases', '--all', '--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--lookback-days', '--mcp-only', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--sigma', '--since', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout'],
'apply-migrations': ['--ab', '--all', '--auto-update', '--brain', '--break-lock', '--build-index', '--by-mention', '--compile', '--days', '--dry-run', '--exclusive', '--fast', '--force', '--force-all', '--force-orchestrator', '--force-retry', '--force-schema', '--from-meetings', '--from-pages', '--help', '--history', '--host-dir', '--http', '--json', '--lang', '--list', '--locks', '--markdown', '--max-age', '--migrate-only', '--migration', '--mode', '--multimodal', '--no-autopilot-install', '--no-embedding', '--no-extract', '--non-interactive', '--phase', '--priority', '--refresh-unqualified', '--remediate', '--rollback', '--skip-verify', '--source', '--stale', '--surface', '--undo-wave', '--use-captured-snapshot', '--with-calibration', '--yes'],
'auth': ['--aliases', '--all', '--bound-brain', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--enable-dcr', '--enable-dcr-insecure', '--fast', '--federated-read', '--force', '--from-pages', '--grant-types', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--pattern', '--pending', '--redirect-uri', '--reset', '--resolve', '--scopes', '--source', '--stale', '--supersessions', '--surface', '--takes-holders', '--thin', '--token', '--token-endpoint-auth-method', '--yes'],
'autopilot': ['--aliases', '--all', '--auto-fix', '--batch', '--brain', '--break-lock', '--by-type', '--check', '--dimensions', '--dir', '--drain', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--ff-only', '--fix', '--force', '--force-break-lock', '--force-retry', '--from-pages', '--full', '--help', '--http', '--include-null-signature', '--inject-bootstrap', '--inline', '--input', '--install', '--interval', '--json', '--markdown', '--max-age', '--max-rss', '--max-usd', '--migrate-only', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--no-extract', '--no-inject', '--no-mutate', '--no-worker', '--non-interactive', '--now', '--once', '--output', '--path', '--pattern', '--pending', '--phase', '--refresh-cache', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--source', '--stale', '--status', '--supersessions', '--surface', '--swap-only', '--target', '--target-score', '--thin', '--timeout', '--to', '--uninstall', '--unsafe-bypass-dream-guard', '--user', '--version', '--yes'],
'autopilot': ['--aliases', '--all', '--auto-fix', '--batch', '--brain', '--break-lock', '--by-type', '--check', '--dimensions', '--dir', '--drain', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--ff-only', '--fix', '--force', '--force-break-lock', '--force-retry', '--from-pages', '--help', '--http', '--include-null-signature', '--inject-bootstrap', '--inline', '--input', '--install', '--interval', '--json', '--markdown', '--max-age', '--max-rss', '--max-usd', '--migrate-only', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--no-extract', '--no-inject', '--no-mutate', '--no-worker', '--non-interactive', '--now', '--once', '--output', '--path', '--pattern', '--pending', '--phase', '--refresh-cache', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--source', '--stale', '--status', '--supersessions', '--surface', '--swap-only', '--target', '--target-score', '--thin', '--timeout', '--to', '--uninstall', '--unsafe-bypass-dream-guard', '--user', '--version', '--yes'],
'backfill': ['--aliases', '--all', '--batch-size', '--brain', '--concurrency', '--dry-run', '--fresh', '--help', '--include-null-signature', '--json', '--keep-index', '--list', '--max-errors', '--max-rows', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--resume', '--source', '--stale', '--supersessions', '--thin'],
'bench': ['--baseline', '--brain', '--explain', '--force', '--from', '--help', '--json', '--label', '--lang', '--limit', '--markdown', '--multimodal', '--near-symbol', '--restore-only', '--source', '--stale', '--symbol-kind', '--thin', '--threshold-jaccard', '--threshold-latency-multiplier', '--threshold-top1', '--to', '--tool'],
'book-mirror': ['--aliases', '--all', '--allow-empty', '--apply', '--asof', '--author', '--auto', '--background', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--brain-wide-max-cost-usd', '--budget-usd-per-day', '--by-mention', '--chapters-dir', '--content', '--context-file', '--date', '--days', '--dry-run', '--entities', '--explain', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--help', '--http', '--image', '--include-null-signature', '--json', '--kind', '--limit', '--max-turns', '--max-usd', '--mode', '--model', '--multimodal', '--no-confirm', '--no-embedding', '--no-extract', '--no-follow', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--save', '--session', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--thin', '--timeout', '--timeout-ms', '--title', '--trusted-extraction', '--url', '--with-db', '--yes'],
@@ -21,7 +21,7 @@ export const CLI_FLAG_REGISTRY: Record<string, readonly string[]> = {
'brainstorm': ['--aliases', '--all', '--brain', '--chunker-debug', '--code', '--compile', '--fast', '--fix', '--force', '--force-rechunk', '--force-resume', '--from-pages', '--full', '--help', '--http', '--include-null-signature', '--json', '--judge-model', '--lang', '--limit', '--list-runs', '--markdown', '--max-cost', '--max-far-set', '--max-ideas-per-judge-call', '--model', '--no-embed', '--no-embedding', '--no-extract', '--no-save', '--pattern', '--pending', '--reset', '--resolve', '--resume', '--retry-failed', '--retry-judge', '--save', '--source', '--stale', '--strict-budget', '--supersessions', '--surface', '--thin', '--timeout', '--yes'],
'cache': ['--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--no-embedding', '--source', '--surface', '--yes'],
'calibration': ['--ab', '--aliases', '--all', '--allow-empty', '--apply', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--dry-run', '--entities', '--explain', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--help', '--holder', '--http', '--image', '--include-null-signature', '--json', '--key-prefix', '--kind', '--lang', '--limit', '--markdown', '--max-usd', '--mode', '--multimodal', '--near-symbol', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--phase', '--progress-interval', '--progress-json', '--quiet', '--regenerate', '--repo', '--reset', '--resolve', '--restore-only', '--save', '--scrub-gstack', '--session', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--surface', '--symbol-kind', '--thin', '--trusted-extraction', '--undo-wave', '--url', '--with-calibration', '--with-db', '--yes'],
'call': ['--aliases', '--all', '--all-sources', '--as-context', '--auto-fix', '--background', '--brain', '--by-mention', '--catch-up', '--concurrency', '--confirm-destructive', '--content', '--cost-estimate', '--count', '--days', '--depth', '--dim', '--dir', '--direction', '--enable-dcr', '--enable-dcr-insecure', '--explain', '--fast', '--federated', '--file', '--fix', '--follow', '--force', '--from', '--from-meetings', '--grant-types', '--grep', '--hard-deadline', '--help', '--http', '--image', '--include-expired', '--include-frontmatter', '--include-null-signature', '--infer-dates', '--install', '--interval', '--json', '--key', '--kind', '--lang', '--limit', '--link-source', '--link-type', '--llm', '--migrate-only', '--missing-path', '--multimodal', '--ner', '--no-embed', '--no-expand', '--no-extract', '--no-federated', '--no-hard-deadline', '--no-retry-connect', '--no-save', '--older-than', '--page', '--param', '--params', '--password', '--path', '--pattern', '--pending', '--pglite', '--port', '--progress-interval', '--progress-json', '--public-url', '--queue', '--quiet', '--reenrich-after', '--refresh-cache', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--restore-only', '--save', '--scopes', '--session', '--sigma', '--since', '--slug', '--slug-prefix', '--source', '--source-id', '--stale', '--status', '--stdin', '--strategy', '--suites', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--synthesize', '--tag', '--thin', '--timeout', '--to', '--today', '--token', '--token-ttl', '--tools-json', '--type', '--url', '--version', '--watch', '--with-calibration', '--workers', '--yes'],
'call': ['--aliases', '--all', '--all-sources', '--as-context', '--auto-fix', '--background', '--brain', '--by-mention', '--catch-up', '--concurrency', '--confirm-destructive', '--content', '--cost-estimate', '--count', '--days', '--depth', '--dim', '--dir', '--direction', '--enable-dcr', '--enable-dcr-insecure', '--explain', '--fast', '--federated', '--file', '--fix', '--follow', '--force', '--from', '--from-meetings', '--grant-types', '--grep', '--hard-deadline', '--help', '--http', '--image', '--include-expired', '--include-frontmatter', '--include-null-signature', '--infer-dates', '--install', '--interval', '--json', '--key', '--kind', '--lang', '--limit', '--link-source', '--link-type', '--llm', '--migrate-only', '--missing-path', '--multimodal', '--ner', '--no-embed', '--no-expand', '--no-extract', '--no-federated', '--no-hard-deadline', '--no-retry-connect', '--no-save', '--older-than', '--page', '--param', '--params', '--password', '--path', '--pattern', '--pending', '--pglite', '--port', '--progress-interval', '--progress-json', '--public-url', '--queue', '--quiet', '--reenrich-after', '--refresh-cache', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--restore-only', '--save', '--scopes', '--session', '--sigma', '--since', '--slug', '--slug-prefix', '--source', '--source-id', '--stale', '--status', '--stdin', '--strategy', '--suites', '--supabase', '--supersessions', '--surface', '--symbol-kind', '--synthesize', '--tag', '--thin', '--timeout', '--to', '--today', '--token', '--token-ttl', '--tools-json', '--type', '--uninstall', '--url', '--version', '--watch', '--with-calibration', '--workers', '--yes'],
'capture': ['--aliases', '--all', '--allow-empty', '--apply', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--depth', '--entities', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--help', '--http', '--image', '--include-null-signature', '--json', '--kind', '--limit', '--max-usd', '--mcp-only', '--mode', '--multimodal', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--repo', '--reset', '--resolve', '--restore-only', '--save', '--scopes', '--session', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--stdin', '--supersessions', '--surface', '--thin', '--timeout', '--trusted-extraction', '--type', '--url', '--what', '--where', '--who', '--with-db', '--yes'],
'check-backlinks': ['--background', '--brain', '--brain-wide-max-cost-usd', '--dir', '--dry-run', '--explain', '--follow', '--help', '--include-frontmatter', '--json', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--source', '--stale', '--timeout', '--type'],
'check-resolvable': ['--brain', '--dry-run', '--fix', '--help', '--json', '--skills-dir', '--source', '--strict', '--verbose'],
@@ -55,11 +55,11 @@ export const CLI_FLAG_REGISTRY: Record<string, readonly string[]> = {
'init': ['--all', '--brain', '--chat-model', '--check', '--ctx-size', '--embedding-dimensions', '--embedding-model', '--entity', '--expansion-model', '--fast', '--force', '--from-pages', '--grant-types', '--help', '--http', '--issuer-url', '--json', '--judge-model', '--key', '--mcp-only', '--mcp-url', '--migrate-only', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--non-interactive', '--oauth-client-id', '--oauth-client-secret', '--path', '--pglite', '--provenance', '--schema-pack', '--scopes', '--skip-embed-check', '--source', '--stale', '--supabase', '--surface', '--target', '--to', '--touchpoint', '--url', '--version'],
'integrations': ['--auto', '--brain', '--dry-run', '--embeddings', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--no-embedding', '--overwrite', '--refresh', '--reranking', '--source', '--surface', '--target'],
'integrity': ['--aliases', '--all', '--auto', '--backend', '--background', '--brain', '--brain-wide-max-cost-usd', '--check', '--confidence', '--cost', '--dry-run', '--explain', '--fast', '--follow', '--force', '--fresh', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--limit', '--no-embedding', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--review-lower', '--skip-bare-tweet', '--skip-urls', '--source', '--stale', '--supabase', '--supersessions', '--surface', '--thin', '--timeout', '--type', '--url'],
'jobs': ['--abbrev-ref', '--aliases', '--all', '--allow-empty', '--allow-protected', '--allow-shell-jobs', '--apply', '--asof', '--auto', '--auto-with-prompt', '--background', '--backoff-delay', '--backoff-jitter', '--backoff-type', '--batch-size', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--break-lock', '--budget-usd', '--budget-usd-per-day', '--by-mention', '--cached', '--catch-up', '--check', '--cli-path', '--cluster', '--cluster-errors', '--code', '--concurrency', '--confidence', '--confirm-destructive', '--content', '--date', '--days', '--delay', '--detach', '--diff-filter', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--empty', '--entities', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--federated-read', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--force-retry', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--hard-deadline', '--health-interval', '--held-out', '--help', '--http', '--idempotency-key', '--image', '--include-frontmatter', '--include-gitignored', '--include-null-signature', '--infer-dates', '--inject-bootstrap', '--inline', '--input', '--install', '--interval', '--is-ancestor', '--json', '--kind', '--lang', '--limit', '--lock', '--markdown', '--max-age', '--max-attempts', '--max-cost-usd', '--max-crashes', '--max-rss', '--max-runtime-min', '--max-sources', '--max-stalled', '--max-usd', '--max-waiting', '--mcp-only', '--min-context', '--missing-path', '--mode', '--model', '--multimodal', '--name-only', '--name-status', '--near-symbol', '--ner', '--nice', '--no', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-federate', '--no-gpg-sign', '--no-hard-deadline', '--no-inject', '--no-mutate', '--no-pull', '--no-renames', '--no-schema-pack', '--no-verify', '--no-worker', '--now', '--offset', '--older-than', '--once', '--order', '--others', '--override-disabled', '--pace', '--pace-max-concurrency', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--phase', '--pid-file', '--priority', '--progress-interval', '--progress-json', '--queue', '--quiet', '--redact-secrets', '--reenrich-after', '--refresh-ms', '--remediate', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--resume', '--retry-failed', '--review-lower', '--run-id', '--save', '--segment-limit', '--serial', '--session', '--short', '--show-toplevel', '--sigkill-rescue', '--since', '--skip-bare-tweet', '--skip-failed', '--skip-urls', '--sleep', '--slug', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--stats', '--status', '--strategy', '--supersessions', '--surface', '--swap-only', '--symbol-kind', '--target', '--thin', '--thin-threshold', '--timeout', '--timeout-ms', '--trusted-extraction', '--type', '--types', '--uninstall', '--unsafe-bypass-dream-guard', '--url', '--user', '--verbose', '--version', '--watch', '--wedge-rescue', '--with-db', '--workers', '--yes'],
'jobs': ['--abbrev-ref', '--aliases', '--all', '--allow-empty', '--allow-protected', '--allow-shell-jobs', '--apply', '--asof', '--auto', '--auto-fix', '--auto-with-prompt', '--background', '--backoff-delay', '--backoff-jitter', '--backoff-type', '--batch', '--batch-size', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--break-lock', '--budget-usd', '--budget-usd-per-day', '--by-mention', '--by-type', '--cached', '--catch-up', '--check', '--cli-path', '--cluster', '--cluster-errors', '--code', '--concurrency', '--confidence', '--confirm-destructive', '--content', '--date', '--days', '--delay', '--detach', '--diff-filter', '--dimensions', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--empty', '--entities', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--federated-read', '--ff-only', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--force-retry', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--hard-deadline', '--health-interval', '--held-out', '--help', '--http', '--idempotency-key', '--image', '--include-frontmatter', '--include-gitignored', '--include-null-signature', '--infer-dates', '--inject-bootstrap', '--inline', '--input', '--install', '--interval', '--is-ancestor', '--json', '--kind', '--lang', '--limit', '--lock', '--markdown', '--max-age', '--max-attempts', '--max-cost-usd', '--max-crashes', '--max-rss', '--max-runtime-min', '--max-sources', '--max-stalled', '--max-usd', '--max-waiting', '--mcp-only', '--migrate-only', '--min-context', '--missing-path', '--mode', '--model', '--multimodal', '--name-only', '--name-status', '--near-symbol', '--ner', '--nice', '--no', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-federate', '--no-gpg-sign', '--no-hard-deadline', '--no-inject', '--no-mutate', '--no-pull', '--no-renames', '--no-schema-pack', '--no-verify', '--no-worker', '--non-interactive', '--now', '--offset', '--older-than', '--once', '--order', '--others', '--output', '--override-disabled', '--pace', '--pace-max-concurrency', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--phase', '--pid-file', '--priority', '--progress-interval', '--progress-json', '--queue', '--quiet', '--redact-secrets', '--reenrich-after', '--refresh-cache', '--refresh-ms', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--resume', '--retry-failed', '--review-lower', '--run-id', '--save', '--segment-limit', '--serial', '--session', '--short', '--show-toplevel', '--sigkill-rescue', '--since', '--skip-bare-tweet', '--skip-failed', '--skip-urls', '--sleep', '--slug', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--stats', '--status', '--strategy', '--supersessions', '--surface', '--swap-only', '--symbol-kind', '--target', '--target-score', '--thin', '--thin-threshold', '--timeout', '--timeout-ms', '--to', '--trusted-extraction', '--type', '--types', '--uninstall', '--unsafe-bypass-dream-guard', '--url', '--user', '--verbose', '--version', '--watch', '--wedge-rescue', '--with-db', '--workers', '--yes'],
'lint': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--exclude', '--explain', '--fast', '--fix', '--follow', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--surface', '--thin', '--timeout'],
'lsd': ['--brain', '--force-resume', '--help', '--json', '--judge-model', '--limit', '--list-runs', '--max-cost', '--max-far-set', '--max-ideas-per-judge-call', '--no-save', '--resume', '--retry-judge', '--save', '--source', '--strict-budget', '--yes'],
'maintain': ['--aliases', '--all', '--background', '--brain', '--break-lock', '--by-mention', '--catch-up', '--column', '--concurrency', '--content-audit', '--count', '--detach', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--fix', '--force', '--force-retry', '--force-schema', '--from-meetings', '--full', '--help', '--include-flagged', '--include-frontmatter', '--include-null-signature', '--index-audit', '--infer-dates', '--input', '--json', '--kind', '--lang', '--locks', '--markdown', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-usd', '--migrate-only', '--multimodal', '--near-symbol', '--ner', '--nice', '--no-extract', '--no-mutate', '--older-than', '--once', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--progress-json', '--query', '--queue', '--quiet', '--rebuild-rollup', '--regenerate', '--remediate', '--remediation-plan', '--reset', '--resolve', '--restore-only', '--resume', '--run-id', '--safe', '--scope', '--since', '--skills-dir', '--skip-failed', '--slugs', '--source', '--source-id', '--stale', '--status', '--supabase', '--supersessions', '--symbol-kind', '--target', '--target-score', '--thin', '--to', '--top-k', '--type', '--unsafe-bypass-dream-guard', '--url', '--verbose', '--window', '--workers', '--yes'],
'migrate': ['--ab', '--aliases', '--all', '--auto-update', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--build-index', '--by-mention', '--catch-up', '--compile', '--days', '--dim', '--dry-run', '--embedding-dimensions', '--embedding-model', '--embeddings', '--exclusive', '--explain', '--fast', '--follow', '--force', '--force-retry', '--force-schema', '--from-meetings', '--from-pages', '--help', '--history', '--http', '--ignore-env-override', '--ignore-missing-key', '--include-null-signature', '--json', '--lang', '--locks', '--markdown', '--max-age', '--model', '--multimodal', '--name', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--non-interactive', '--pace', '--pace-max-concurrency', '--parallel', '--path', '--pattern', '--pending', '--phase', '--prefix', '--priority', '--progress-interval', '--progress-json', '--quiet', '--refresh-unqualified', '--remediate', '--reranking', '--reset', '--resolve', '--restore-only', '--resume', '--rollback', '--skip-verify', '--slugs', '--source', '--stale', '--supersessions', '--surface', '--symbol-kind', '--thin', '--timeout', '--to', '--undo', '--undo-wave', '--url', '--use-captured-snapshot', '--version', '--with-calibration', '--yes'],
'migrate': ['--ab', '--aliases', '--all', '--auto-update', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--build-index', '--by-mention', '--catch-up', '--compile', '--days', '--dim', '--dry-run', '--embedding-dimensions', '--embedding-model', '--embeddings', '--exclusive', '--explain', '--fast', '--fix', '--follow', '--force', '--force-break-lock', '--force-retry', '--force-schema', '--from-meetings', '--from-pages', '--help', '--history', '--http', '--ignore-env-override', '--ignore-missing-key', '--include-null-signature', '--json', '--lang', '--locks', '--markdown', '--max-age', '--model', '--multimodal', '--name', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--non-interactive', '--pace', '--pace-max-concurrency', '--parallel', '--path', '--pattern', '--pending', '--phase', '--prefix', '--priority', '--progress-interval', '--progress-json', '--quiet', '--refresh-unqualified', '--remediate', '--reranking', '--reset', '--resolve', '--restore-only', '--resume', '--rollback', '--skip-verify', '--slugs', '--source', '--stale', '--supersessions', '--surface', '--symbol-kind', '--thin', '--timeout', '--to', '--undo', '--undo-wave', '--url', '--use-captured-snapshot', '--version', '--with-calibration', '--yes'],
'models': ['--aliases', '--all', '--brain', '--embedding-dimensions', '--embedding-model', '--embeddings', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--judge-model', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reranking', '--reset', '--resolve', '--skip', '--source', '--stale', '--supersessions', '--surface', '--thin', '--undo', '--version'],
'mounts': ['--alias', '--brain', '--cache', '--database-path', '--database-url', '--db-path', '--db-url', '--engine', '--explain', '--help', '--id', '--json', '--lang', '--lock', '--markdown', '--mcp-url', '--multimodal', '--near-symbol', '--path', '--restore-only', '--skills-dir', '--source', '--stale', '--symbol-kind', '--thin', '--verbose'],
'notability-eval': ['--aliases', '--all', '--brain', '--embedding-dimensions', '--embedding-model', '--help', '--in', '--include-null-signature', '--json', '--model', '--multimodal', '--no', '--no-embed', '--no-extract', '--out', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--skip-llm', '--source', '--stale', '--supersessions', '--target-high', '--target-low', '--target-medium', '--thin', '--version'],
+16
View File
@@ -467,6 +467,22 @@ export async function listStaleLocks(engine: BrainEngine): Promise<LockSnapshot[
return selectLockRows(engine, { staleOnly: true });
}
/**
* Every lock whose holder still looks live (TTL unexpired / holder judged
* alive by `isLockHolderLive`). This is the "is anything writing right now?"
* probe: cycle runs, sync imports, and embed backfills all hold rows in
* `gbrain_cycle_locks` while they work, so an empty result means the write
* planes this table guards are drained. Used by `gbrain migrate`'s quiesce
* to wait for in-flight work instead of sleeping a blind fixed grace.
*/
export async function listLiveLocks(
engine: BrainEngine,
ttlMinutes: number = DEFAULT_TTL_MINUTES,
): Promise<LockSnapshot[]> {
const rows = await selectLockRows(engine);
return rows.filter((snap) => isLockHolderLive(snap, ttlMinutes));
}
/**
* v0.41.6.0 D3: atomic verify-and-delete for `gbrain sync --break-lock`.
*
+75
View File
@@ -0,0 +1,75 @@
/**
* Numeric env-var resolution with a shared warn-once memo.
*
* Relocated from `src/commands/doctor.ts` (where `_resolveEnvNumber` was already
* exported but `_resolveSyncFreshnessHours` was module-private). `src/core/source-health.ts`
* needs the hours resolver for the staleness ceiling, and `doctor.ts:51` already imports
* FROM `source-health.ts` so having core reach back into commands would close an import
* cycle. Duplicating the helper instead would create a SECOND `_envNumberWarned` set, so
* the same bad env var would warn once per module and the "warn exactly once" contract
* would quietly become "warn twice".
*
* One module, one memo, imported by both planes.
*
* The warning prefix is `[gbrain]` rather than `[gbrain doctor]` because these vars are
* now read outside doctor. No test asserted the old prefix (verified before the move).
*/
/** Warn-once memo, keyed by env var name. Module-level so every caller shares it. */
const _envNumberWarned = new Set<string>();
/**
* Read a POSITIVE number from an env var; warn once and fall back on garbage.
*
* `opts.unit` is cosmetic, used only in the warning string ('h', '%', '').
* Zero and negative values are treated as garbage: every consumer of this helper wants
* a positive threshold, and `0` is far more often an unset-variable accident than an
* intentional "disable".
*/
export function resolveEnvNumber(
varName: string,
fallback: number,
opts?: { unit?: string },
): number {
const raw = process.env[varName];
if (raw === undefined || raw === '') return fallback;
const n = Number(raw);
if (!Number.isFinite(n) || n <= 0) {
if (!_envNumberWarned.has(varName)) {
_envNumberWarned.add(varName);
console.warn(
`[gbrain] Ignoring invalid ${varName}=${raw}; using default ${fallback}${opts?.unit ?? ''}.`,
);
}
return fallback;
}
return n;
}
/** `resolveEnvNumber` with the hours unit pre-applied. */
export function resolveHoursEnv(varName: string, fallback: number): number {
return resolveEnvNumber(varName, fallback, { unit: 'h' });
}
/**
* Warn once for `varName` using the SAME memo as `resolveEnvNumber`.
*
* For env vars that are disabled-unless-set, where `resolveEnvNumber`'s
* "fall back to a default" shape does not apply (e.g.
* `GBRAIN_EXTRACTION_LAG_FAIL_PCT`, whose absence means "no hard fail"), but
* whose invalid-value warning must still fire exactly once per process.
* Exposed instead of the raw Set so the memo stays encapsulated.
*/
export function warnOnceForEnv(varName: string, message: string): void {
if (_envNumberWarned.has(varName)) return;
_envNumberWarned.add(varName);
console.warn(message);
}
/**
* Test seam: clear the warn-once memo so a suite can assert the warning fires.
* Production code must never call this.
*/
export function _resetEnvNumberWarnedForTests(): void {
_envNumberWarned.clear();
}
+57 -1
View File
@@ -11,6 +11,8 @@
* await worker.start(); // polls until SIGTERM
*/
import { existsSync } from 'fs';
import { autopilotPausedMarkerPath } from '../autopilot-paths.ts';
import type { BrainEngine } from '../engine.ts';
import type {
MinionJob, MinionJobContext, MinionHandler, MinionWorkerOpts,
@@ -152,6 +154,8 @@ export class MinionWorker extends EventEmitter {
private queue: MinionQueue;
private handlers = new Map<string, MinionHandler>();
private running = false;
/** Log the pause/resume transition once each, not every poll. */
private pausedByMarkerAnnounced = false;
private inFlight = new Map<number, InFlightJob>();
private workerId = randomUUID();
@@ -535,7 +539,24 @@ export class MinionWorker extends EventEmitter {
}
}
// Claim jobs up to concurrency limit
// Claim jobs up to concurrency limit — unless the system-wide pause
// marker is parked. `gbrain migrate` quiesces writers for the copy
// window; the marker stops the autopilot dispatch loop, and gating
// the CLAIM here extends that fence to queued jobs (an already
// in-flight job finishes and is waited on by the migrate drain).
// Checked at claim time only: one existsSync per poll tick.
if (existsSync(autopilotPausedMarkerPath())) {
if (!this.pausedByMarkerAnnounced) {
console.log('[worker] pause marker present — not claiming new jobs until it clears.');
this.pausedByMarkerAnnounced = true;
}
await new Promise(resolve => setTimeout(resolve, this.opts.pollInterval));
continue;
}
if (this.pausedByMarkerAnnounced) {
console.log('[worker] pause marker cleared — resuming job claims.');
this.pausedByMarkerAnnounced = false;
}
if (this.inFlight.size < this.opts.concurrency) {
const lockToken = `${this.workerId}:${Date.now()}`;
let job: MinionJob | null;
@@ -563,6 +584,16 @@ export class MinionWorker extends EventEmitter {
}
if (job) {
// Post-claim fence re-check: the pre-claim marker check above
// races migrate's marker write — this claim may have committed
// after migrate's drain probe counted zero active jobs. A job
// claimed into that window is released back (delayed, un-run)
// instead of executed; the poll loop then parks on the marker.
if (existsSync(autopilotPausedMarkerPath())) {
console.log(`[worker] pause marker appeared after claim — releasing ${job.name} (id=${job.id}) un-run.`);
await this.releaseClaimForPause(job, lockToken);
continue;
}
// Quiet-hours gate: evaluated at claim time, not dispatch.
// Config lives on the job record (jsonb column added in
// schema migration v12). Worker releases the job back to the
@@ -629,6 +660,31 @@ export class MinionWorker extends EventEmitter {
* 'skip' status='cancelled', final_status='skipped_quiet_hours'. The
* event is dropped.
*/
/**
* Release a just-claimed job back to the queue un-run because the
* system-wide pause marker appeared between our pre-claim check and the
* claim committing. Same conditional-release SQL shape as the quiet-hours
* defer, but with a short delay: the poll loop parks on the marker, so the
* job re-enters waiting and is picked up as soon as the pause clears.
*/
private async releaseClaimForPause(job: MinionJob, lockToken: string): Promise<void> {
try {
await this.engine.executeRaw(
`UPDATE minion_jobs
SET status = 'delayed', lock_token = NULL, lock_until = NULL,
delay_until = now() + interval '1 minute',
updated_at = now()
WHERE id = $1 AND lock_token = $2`,
[job.id, lockToken],
);
} catch (e) {
// Fail-open: if the release UPDATE itself fails, the claim lock simply
// expires and the stall detector requeues the row — slower, same end
// state, and never a reason to crash the worker.
console.error(`[worker] pause release failed for job ${job.id}: ${e instanceof Error ? e.message : String(e)}`);
}
}
private async handleQuietHoursDefer(job: MinionJob, lockToken: string, verdict: 'skip' | 'defer'): Promise<void> {
try {
if (verdict === 'skip') {
+7 -2
View File
@@ -4,10 +4,15 @@
* v0.41.6.0 D5 registry + signal handlers so abnormal termination
* (SIGTERM/SIGHUP/SIGPIPE, EPIPE on stdout, uncaughtException) releases
* locks instead of leaking them for up to 30 minutes until the TTL
* expires. Pre-v0.41.6.0, `gbrain sync --full | head -20` would SIGPIPE
* gbrain, finally blocks wouldn't run, and the next sync would report
* expires. Pre-v0.41.6.0, a full `gbrain sync` piped into `head` would
* SIGPIPE gbrain, finally blocks wouldn't run, and the next sync would report
* "Another sync is in progress" because the lock row was orphaned.
*
* (The flag name is spelled out rather than written literally: the CLI
* flag-registry generator scans comments, so a bare double-dash token here
* would be inherited as an accepted flag by every command that imports this
* module. Pinned by `test/cli-flag-validation.test.ts`.)
*
* Design (per eng-review D7 + outside-voice F9-F11, 2026-05-24):
*
* - Signal scope: SIGTERM, SIGHUP, SIGPIPE, uncaughtException,
+80 -12
View File
@@ -25,6 +25,7 @@ import { execFileSync } from 'child_process';
import type { BrainEngine } from './engine.ts';
import { parseSourceConfig, type SourceRow } from './sources-load.ts';
import { isSourceUnchangedSinceSync } from './git-head.ts';
import { resolveHoursEnv } from './env-number.ts';
export interface SourceMetrics {
source_id: string;
@@ -165,17 +166,56 @@ export function commitTimeMs(localPath: string | null, sha: string | null): numb
}
}
/**
* Resolve the staleness-ceiling in seconds.
*
* `GBRAIN_STALENESS_CEILING_HOURS` overrides; otherwise it tracks
* `GBRAIN_SYNC_FRESHNESS_FAIL_HOURS` (default 72) so the ceiling and the check
* that reads it cannot drift apart by default, while still being separable
* during an incident.
*/
export function resolveStalenessCeilingSeconds(): number {
const base = resolveHoursEnv('GBRAIN_SYNC_FRESHNESS_FAIL_HOURS', 72);
return Math.floor(resolveHoursEnv('GBRAIN_STALENESS_CEILING_HOURS', base) * 3600);
}
/**
* Commit-relative lag in seconds from a STORED content timestamp (the
* `newest_content_at` column), for REMOTE consumers that cannot shell out:
* `newest_content_at` column), for consumers that cannot shell out:
* - `null` when `lastSyncMs` is unknown.
* - Negative wall-clock (future `last_sync_at`) is surfaced as-is so upstream
* clock-skew detection still fires.
* - `0` when the stored content is at or before the last sync (caught up).
* - Caught up (content at or before the last sync) `0` UNTIL wall-clock
* passes the ceiling, then a monotonic ramp (see below).
* - Wall-clock `now - lastSync` when content is newer, or when `contentMs` is
* null (no column value / pre-migration) detection never regresses.
*
* Pure. The LOCAL path does NOT use this it keys off the live commit hash via
* The core logic is pure; only the DEFAULT `ceilingSeconds` reads env. Callers
* that need determinism pass the ceiling explicitly.
*
* WHY THE RAMP (the 71-day bug):
*
* Before, "caught up" returned a hard 0 forever, so a source whose clone had
* vanished reported fresh indefinitely the wall-clock number was computed on
* the line above and thrown away. But the caught-up branch is not a mistake: it
* exists (see `checkSyncFreshness`'s clone-unavailable path) because a
* container restart wipes `local_path`, and every QUIET source would otherwise
* read stale after a restart. That justification assumed a no-op sync does not
* advance `last_sync_at`, which stopped being true in v0.42.52.0 a 0-change
* sync now heartbeats the column. So a quiet source that is genuinely being
* checked has a RECENT `last_sync_at` and stays at 0; only a source nobody has
* looked at in a very long time crosses the ceiling.
*
* It ramps rather than steps because three consumers read this one value at
* different thresholds: `federation_health` fails at 24h, `sync_freshness`
* warns at 24h / fails at 72h, and `buildSyncStatusReport` buckets at a
* hardcoded 24/72. A step to the ceiling would cross all of them in the same
* instant, firing two checks at once and skipping the warn tier entirely
* which is the alert-storm shape the caught-up branch was written to prevent.
* `max(0, wallClock - ceiling)` grows continuously, so warn still precedes
* fail and the surfaces escalate in order.
*
* The LOCAL path does NOT use this it keys off the live commit hash via
* `isSourceUnchangedSinceSync` (robust against HEAD moving to an old-dated
* commit, which a timestamp comparison would miss).
*/
@@ -183,12 +223,17 @@ export function lagFromContentMs(
contentMs: number | null,
lastSyncMs: number | null,
nowMs: number,
ceilingSeconds: number = resolveStalenessCeilingSeconds(),
): number | null {
if (lastSyncMs === null || !Number.isFinite(lastSyncMs)) return null;
const wallClockSeconds = Math.floor((nowMs - lastSyncMs) / 1000);
if (wallClockSeconds < 0) return wallClockSeconds; // clock skew passthrough
if (contentMs !== null && Number.isFinite(contentMs)) {
return contentMs <= lastSyncMs ? 0 : wallClockSeconds;
// Caught up: 0 while recently checked, then ramp once nobody has looked
// for longer than the ceiling.
return contentMs <= lastSyncMs
? Math.max(0, wallClockSeconds - ceilingSeconds)
: wallClockSeconds;
}
return wallClockSeconds; // no stored content signal — wall-clock fallback
}
@@ -220,6 +265,9 @@ export async function computeAllSourceMetrics(
// commit-hash probe; the REMOTE federation_health path leaves it off and
// reads the stored column (no subprocess on a DB-supplied local_path).
const probeContent = opts?.probeContent === true;
// One ceiling for the whole report (hoisted out of the per-source map):
// every source gets the same number, and the env read runs once.
const stalenessCeilingSeconds = resolveStalenessCeilingSeconds();
return sources.map((src) => {
const cfg = parseSourceConfig(src.config);
@@ -232,13 +280,33 @@ export async function computeAllSourceMetrics(
: Math.round((chunkStats.embedded / chunkStats.total) * 1000) / 10;
const lastMs = src.last_sync_at ? new Date(src.last_sync_at).getTime() : null;
// v0.41.32.0: commit-relative lag.
// LOCAL (probeContent): caught up iff HEAD == last_commit AND no tracked
// working-tree changes (untracked ignored) → lag 0; else wall-clock.
// Uses the live commit hash so a HEAD that moved to an old-dated commit
// is correctly NOT caught up. NULL last_commit → not caught up → wall-clock.
// REMOTE (default): read the stored newest_content_at column via
// lagFromContentMs — no git subprocess (v0.41.27.0 trust boundary).
// v0.41.32.0: commit-relative lag. TWO implementations, and the split is
// load-bearing rather than accidental duplication — see the diagram.
//
// computeAllSourceMetrics
//
// ├─ probeContent: true ──► isSourceUnchangedSinceSync (git subprocess)
// │ LOCAL only. Accurate: keys off the live
// │ commit hash, so a HEAD moved to an
// │ old-dated commit is correctly NOT caught
// │ up. NULL last_commit → wall-clock.
// │ Caller: `gbrain sources status`.
// │
// └─ probeContent: false ──► lagFromContentMs (column read)
// REMOTE-SAFE: no subprocess, because a
// remote-callable path must never shell
// out against a DB-supplied local_path
// (v0.41.27.0 trust boundary). That makes
// it structurally less accurate than the
// probe, which is exactly why it needs the
// staleness ceiling to stay honest.
// Callers: doctor `federation_health`,
// doctor `sync_freshness` (via its own
// clone-unavailable branch), and
// `buildSyncStatusReport` (gbrain status).
//
// Do NOT "unify" these behind one flag: collapsing the trust boundary into a
// boolean is how a remote caller eventually acquires a subprocess.
let lagSeconds: number | null;
if (lastMs === null) {
lagSeconds = null;
@@ -251,7 +319,7 @@ export async function computeAllSourceMetrics(
const contentMs = src.newest_content_at
? new Date(src.newest_content_at).getTime()
: null;
lagSeconds = lagFromContentMs(contentMs, lastMs, now);
lagSeconds = lagFromContentMs(contentMs, lastMs, now, stalenessCeilingSeconds);
}
return {
+17 -1
View File
@@ -206,8 +206,24 @@ export function formatCodeBreakdown(
return summary.map(s => ` ${s.code}: ${s.count}`).join('\n');
}
/**
* Where `sync-failures.jsonl` lives. Defaults to the gbrain home.
*
* `GBRAIN_SYNC_FAILURES_DIR` exists for the same reason `GBRAIN_AUDIT_DIR` does
* (#2823): the test suite exercises import/sync failure paths with deliberately
* broken fixtures, and without an override those fixture rows append into the
* operator's REAL ledger the one `gbrain doctor` reads and warns on until it
* is cleaned up. A stray `srcE / notes/bad.md SLUG_MISMATCH` row in a live
* brain came from exactly this.
*
* A dedicated var rather than leaning on `GBRAIN_HOME`: pointing GBRAIN_HOME at
* a scratch dir for the whole suite also makes `loadConfig()` return null for
* every test that reads the real config, which is a far wider blast radius than
* this problem needs (and `test/gbrain-home-isolation.test.ts` asserts the
* unset-fallback behavior directly).
*/
function _failuresDir(): string {
return _gbrainPath();
return process.env.GBRAIN_SYNC_FAILURES_DIR || _gbrainPath();
}
export function syncFailuresPath(): string {
+37 -7
View File
@@ -99,8 +99,31 @@ const CODE_EXTENSIONS = new Set<string>([
* Input format (tab-separated):
* A path/to/new-file.md
* M path/to/modified-file.md
* T path/to/retyped-file.md (file <-> symlink; content changed)
* D path/to/deleted-file.md
* R100 old/path.md new/path.md
*
* Status coverage, against the options gbrain actually passes (name-status with
* `-M` only, see `sync-delta.ts` note NO `-C` copy detection):
* A/M/D/R the everyday statuses.
* T TYPECHANGE. Reachable in normal operation: replacing a file with
* a symlink (or vice versa) emits `T`, and it was silently dropped,
* so the change never reached the index until something else
* touched the path. Treated as `modified`: the path still exists and
* its blob changed, which is exactly what re-import handles. (When a
* file becomes a symlink, import-file SKIPS symlinks by design
* the exfil guard so the old regular-file content stays indexed;
* routing the typechange to delete-then-skip is a filed follow-up.)
* C COPY. Unreachable without `-C`, handled defensively.
* U UNMERGED. Only in a conflicted worktree, which sync does not run
* against; handled defensively so a conflicted tree degrades to
* "re-import this path" rather than silently skipping it.
*
* NOTE for future editors: git option names appear here WITHOUT the leading
* double-dash on purpose. The CLI flag-registry generator string-scans module
* source including comments, so a bare double-dash token in prose registers as
* a real, accepted CLI flag on every command that imports this file. Pinned by
* `test/cli-flag-validation.test.ts`.
*/
export function buildSyncManifest(gitDiffOutput: string): SyncManifest {
const manifest: SyncManifest = {
@@ -120,20 +143,27 @@ export function buildSyncManifest(gitDiffOutput: string): SyncManifest {
if (parts.length < 2) continue;
const action = parts[0];
const path = parts[parts.length === 3 ? 2 : 1]; // For renames, new path is 3rd column
if (action === 'A') {
manifest.added.push(path);
} else if (action === 'M') {
manifest.modified.push(path);
manifest.added.push(parts[1]);
} else if (action === 'M' || action === 'T' || action === 'U') {
// T (typechange) and U (unmerged) both mean "this path exists and its
// content is not what we imported" — the same remedy as M.
manifest.modified.push(parts[1]);
} else if (action === 'D') {
manifest.deleted.push(parts[1]);
} else if (action.startsWith('R')) {
// Rename: R100\told-path\tnew-path
} else if (action.startsWith('R') || action.startsWith('C')) {
// Rename/copy: R100\told-path\tnew-path. Copy is unreachable without -C,
// but if the flags ever change, the destination must still be imported.
const oldPath = parts[1];
const newPath = parts[2];
if (oldPath && newPath) {
manifest.renamed.push({ from: oldPath, to: newPath });
if (action.startsWith('C')) {
// A copy leaves the source in place — only the destination is new.
manifest.added.push(newPath);
} else {
manifest.renamed.push({ from: oldPath, to: newPath });
}
}
}
}
+1 -1
View File
@@ -1,6 +1,6 @@
# gbrain agent workspace — template
<!-- gbrain-template-stamp: 0.45.3.0 -->
<!-- gbrain-template-stamp: 0.45.5.0 -->
This repository is the **"Use this template"** distribution artifact for a
[gbrain](https://github.com/garrytan/gbrain) personal-agent workspace — the same
@@ -0,0 +1,207 @@
/**
* Agent-scheduler contract the documented shell-chain surface, pinned.
*
* SCOPE, stated honestly: the downstream agent platforms documented at
* docs/guides/live-sync.md:62-81 (the cron-registration examples) have zero
* detection code in gbrain and no runner in this repo. Their ENTIRE
* integration is an external cron that shells out to the gbrain CLI and reads
* exit codes "/cron add ... gbrain sync --repo X && gbrain embed --stale"
* (live-sync.md) and INSTALL_FOR_AGENTS.md Step 7. This file pins that
* consumer contract through a REAL /bin/sh, because the `&&` short-circuit IS
* the contract; argv arrays cannot exercise it.
*
* (Naming note: the specific downstream platform names live in the docs, which
* carry their own scrub policy; scripts/check-test-real-names.sh bans agent
* fork names inside test fixtures, so this file uses the generic term.)
*
* Anti-vacuity: the fixture commits a real page and every read-back asserts
* pages >= 1 an empty repo would satisfy every other assertion here.
*
* Explicitly OUT of scope (same docs, separate surfaces): `gbrain
* check-update --json` (network), and the nightly keyless `gbrain dream`
* contract the cycle's embed phase has the same EmbeddingDisabledError
* class risk this file's chain case pins for embed; filed as a TODO rather
* than smoke-tested here.
*/
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { mkdtempSync, mkdirSync, rmSync, writeFileSync, existsSync, chmodSync } from 'fs';
import { tmpdir } from 'os';
import { join, resolve, dirname } from 'path';
import { spawnSync, execFileSync } from 'child_process';
const REPO = resolve(import.meta.dir, '..');
const CLI = join(REPO, 'src', 'cli.ts');
const SKIP = process.env.GBRAIN_SKIP_SUBPROCESS_TESTS === '1';
interface ShellResult { exitCode: number; stdout: string; stderr: string }
describe.skipIf(SKIP)('agent-scheduler shell-chain contract', () => {
let home = '';
let tmpbin = '';
let repoDir = '';
let cloneDir = '';
let env: Record<string, string>;
/** Run a string through a REAL POSIX shell — the && is the thing under test. */
function sh(command: string, timeoutMs: number, extraEnv: Record<string, string> = {}): ShellResult {
const res = spawnSync('/bin/sh', ['-c', command], {
cwd: REPO, // near-repo cwd keeps Bun's transpile cache warm for the shim
env: { ...env, ...extraEnv },
encoding: 'utf8',
timeout: timeoutMs,
});
return { exitCode: res.status ?? -1, stdout: res.stdout ?? '', stderr: res.stderr ?? '' };
}
function statusSyncSection(): { last_sync_at: string | null; staleness_class: string; pages: number } {
const r = sh('gbrain status --json --section sync', 90_000);
expect(r.exitCode).toBe(0);
const report = JSON.parse(r.stdout.trim().split('\n').pop()!);
const src = report.sync.sources.find((s: { source_id: string }) => s.source_id === 'default')
?? report.sync.sources[0];
expect(src).toBeDefined();
return { last_sync_at: src.last_sync_at, staleness_class: src.staleness_class, pages: src.pages };
}
function git(dir: string, args: string[]): void {
execFileSync('git', ['-C', dir, ...args], {
stdio: ['ignore', 'pipe', 'pipe'],
timeout: 30_000,
env: {
...env,
GIT_AUTHOR_DATE: '2026-08-01T00:00:00Z',
GIT_COMMITTER_DATE: '2026-08-01T00:00:00Z',
},
});
}
beforeAll(() => {
home = mkdtempSync(join(tmpdir(), 'gb-sched-'));
tmpbin = mkdtempSync(join(tmpdir(), 'gb-sched-bin-'));
repoDir = join(home, 'brain');
cloneDir = join(home, 'brain-clone');
// The shim is what makes `gbrain` a real command for /bin/sh.
writeFileSync(join(tmpbin, 'gbrain'), `#!/bin/sh\nexec bun run '${CLI}' "$@"\n`, { mode: 0o755 });
chmodSync(join(tmpbin, 'gbrain'), 0o755);
const bunDir = dirname(process.execPath || '/usr/local/bin');
env = {
PATH: `${tmpbin}:${bunDir}:/usr/bin:/bin:/usr/sbin:/sbin`,
HOME: home,
GBRAIN_HOME: home,
TMPDIR: tmpdir(),
// Repo paths cross the shell boundary via env, never string interpolation.
SCHED_REPO: repoDir,
SCHED_CLONE: cloneDir,
};
// Keyless brain — the install shape whose chain used to exit 1.
mkdirSync(join(home, '.gbrain'), { recursive: true });
writeFileSync(
join(home, '.gbrain', 'config.json'),
JSON.stringify({
engine: 'pglite',
database_path: join(home, '.gbrain', 'brain.pglite'),
embedding_disabled: true,
}) + '\n',
);
const init = spawnSync('bun', ['run', CLI, 'init', '--migrate-only'], {
cwd: REPO, env, encoding: 'utf8', timeout: 120_000,
});
if (init.status !== 0) throw new Error(`init --migrate-only failed:\n${(init.stderr ?? '').slice(-2000)}`);
// Fixture repo with ONE REAL PAGE (anti-vacuity) + repo-local identity
// (HOME=tmp has no global gitconfig — commits fail outright without it).
mkdirSync(join(repoDir, 'people'), { recursive: true });
git(repoDir, ['init', '-q', '-b', 'main']);
git(repoDir, ['config', 'user.email', 'test@example.com']);
git(repoDir, ['config', 'user.name', 'Sched Test']);
writeFileSync(join(repoDir, 'people', 'alice-example.md'), '# Alice Example\n\nA real page so nothing here passes vacuously.\n');
git(repoDir, ['add', '-A']);
git(repoDir, ['commit', '-q', '-m', 'seed page']);
}, 180_000);
afterAll(() => {
rmSync(home, { recursive: true, force: true });
rmSync(tmpbin, { recursive: true, force: true });
});
let firstSyncAt: string | null = null;
test('1. the documented chain exits 0 on a keyless brain, with the keyless hint on stderr', () => {
// Verified against unfixed embed.ts during implementation: this chain
// exited 1 (EmbeddingDisabledError -> blanket process.exit(1)), breaking
// the documented agent-scheduler cron on every keyless brain. The fix is
// load-bearing for this exact spelling.
const r = sh('gbrain sync --repo "$SCHED_REPO" --no-pull && gbrain embed --stale', 150_000);
expect(r.exitCode).toBe(0);
expect(r.stderr).toContain('Embeddings are disabled');
const s = statusSyncSection();
expect(s.pages).toBeGreaterThanOrEqual(1); // anti-vacuity: the page landed
expect(s.staleness_class).toBe('fresh');
firstSyncAt = s.last_sync_at;
expect(firstSyncAt).not.toBeNull();
}, 300_000);
test('2. quiet re-run: sync itself reports up_to_date; heartbeat strictly advances', async () => {
// Cross-second so the strict > below cannot alias on same-second stamps.
await new Promise((r) => setTimeout(r, 1_100));
const r = sh('gbrain sync --repo "$SCHED_REPO" --no-pull && gbrain embed --stale', 150_000);
expect(r.exitCode).toBe(0);
// The run's OWN verdict, not a timestamp tautology: a crashed second run
// would satisfy t2 >= t1; it cannot print "Already up to date."
expect(r.stdout).toContain('Already up to date');
const s = statusSyncSection();
expect(s.staleness_class).toBe('fresh');
// v0.42.52.0 heartbeat: a 0-change sync still bumps last_sync_at.
expect(Date.parse(s.last_sync_at!)).toBeGreaterThan(Date.parse(firstSyncAt!));
}, 300_000);
test('3. the documented weekly chain (doctor --json && embed --stale) exits 0 keyless', () => {
// Runs BEFORE the poisoned-clone case: that one appends real rows to this
// brain's sync-failures ledger, which doctor reads and may warn on.
const r = sh('gbrain doctor --json && gbrain embed --stale', 240_000);
if (r.exitCode !== 0) {
// A keyless-first-class bug in its own right — surface the failing checks.
const doc = (() => { try { return JSON.parse(r.stdout.trim().split('\n').pop()!); } catch { return null; } })();
const fails = doc?.checks?.filter((c: { status: string }) => c.status === 'fail') ?? [];
throw new Error(`keyless weekly chain exited ${r.exitCode}; failing checks: ${JSON.stringify(fails.map((f: { name: string; message: string }) => `${f.name}: ${f.message}`), null, 2)}`);
}
expect(r.exitCode).toBe(0);
}, 330_000);
test('4. pull failure breaks the chain: sync exits 1, embed never runs', () => {
// A clone whose origin is a local filesystem path: pullRepo's SSRF flag
// (protocol.file.allow=never) rejects it deterministically on every pull.
execFileSync('git', ['clone', '-q', '--no-hardlinks', repoDir, cloneDir], {
env: { ...env, GIT_ALLOW_PROTOCOL: 'file' },
stdio: ['ignore', 'pipe', 'pipe'],
timeout: 60_000,
});
git(cloneDir, ['config', 'user.email', 'test@example.com']);
git(cloneDir, ['config', 'user.name', 'Sched Test']);
// Because cases 1-2 already synced the SAME commit for source `default`,
// the clone's HEAD equals the stored anchor — so this run is ALREADY the
// quiet case: zero imports + pullFailed (protocol.file rejected) →
// partial/pull_failed → exit 1 (sync.ts:4994, the #3068 contract), and the
// && short-circuits: the keyless embed hint must NOT appear. (On a FRESH
// source the first run imports the working tree and exits 0
// warn-and-continue; it takes a second, quiet run to surface the wedge —
// both shapes converge on "a permanently failing pull cannot stay green".)
const run1 = sh('gbrain sync --repo "$SCHED_CLONE" --source default && gbrain embed --stale', 150_000);
if (run1.exitCode !== 1) {
throw new Error(`expected pull_failed exit 1, got ${run1.exitCode}\nstderr:\n${run1.stderr.slice(-2500)}\nstdout:\n${run1.stdout.slice(-800)}`);
}
expect(run1.exitCode).toBe(1);
expect(run1.stderr).not.toContain('Embeddings are disabled');
// And it stays red on repeat — a wedged pull is not a one-off blip.
const run2 = sh('gbrain sync --repo "$SCHED_CLONE" --source default && gbrain embed --stale', 150_000);
expect(run2.exitCode).toBe(1);
expect(run2.stderr).not.toContain('Embeddings are disabled');
}, 450_000);
});
+7 -1
View File
@@ -137,8 +137,14 @@ describe('autopilot wrapper script — bun PATH export (v0.42.x regression)', ()
// that installed via the wrapper indirection.
describe('autopilot showStatus — wrapper-path detection', () => {
test('status detects the autopilot-run.sh wrapper line', async () => {
// The inline crontab.includes check became the pure, unit-tested
// crontabIndicatesAutopilotInstall (a cron'd status-monitor line must not
// read as an install). Pin the behavior through the function itself and
// the detect path's wiring to it.
const { crontabIndicatesAutopilotInstall } = await import('../src/commands/autopilot.ts');
expect(crontabIndicatesAutopilotInstall("*/5 * * * * '/h/.gbrain/autopilot-run.sh' >> log 2>&1")).toBe(true);
const { readFileSync } = await import('fs');
const src = readFileSync('src/commands/autopilot.ts', 'utf8');
expect(src).toMatch(/crontab\.includes\('autopilot-run\.sh'\)/);
expect(src).toMatch(/crontabIndicatesAutopilotInstall\(crontab\)/);
});
});
@@ -0,0 +1,387 @@
/**
* Autopilot launchd lifecycle the e2e that closes the artifact-presence gap.
*
* PR-A rewrote `autopilot --status` because the old one answered "installed"
* from a bare plist existsSync while the daemon had been dead for 71 days. The
* unit tests for the fix assert the STRINGS we generate (guard text, plist
* XML) the same anti-pattern one level up. This file asserts BEHAVIOR:
*
* Describe A (every platform, incl. ubuntu CI): the full install
* self-disable status reinstall uninstall arc, with `launchctl`
* replaced by an argv recorder, and the generated wrapper executed by a
* REAL bash against a genuinely deleted repo.
*
* Describe B (darwin only, fail-SKIP): the same self-disable against the
* REAL launchd job provably loaded (RunAtLoad ran our wrapper),
* `launchctl kickstart -k` forces the guard after the repo vanishes, and
* the job is provably gone afterwards. Non-vacuous by construction: each
* link is asserted separately, so "job absent" can't be true for the
* wrong reason.
*
* Label safety: every spawn sets GBRAIN_AUTOPILOT_LABEL to a unique test
* label, so this can never collide with or tear down a real
* com.gbrain.autopilot install on the machine running the tests. Cleanup is
* OWN-LABEL ONLY: a prefix sweep would boot out a concurrent Conductor
* workspace's live test run on the same Mac.
*
* Known real-machine flake vector (describe B): the launchd-spawned wrapper
* runs with the REAL user's HOME and sources their zshenv/zshrc before the
* guard (launchd does not inherit our test env). A dotfile that execs another
* shell or stalls can eat the wrapper. The marker path and label are baked as
* absolute strings at install time so they cannot be misrouted; if the poll
* times out, the failure dump names this vector. Escape hatch:
* GBRAIN_SKIP_LAUNCHD_E2E=1.
*/
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { mkdtempSync, mkdirSync, rmSync, writeFileSync, readFileSync, existsSync, chmodSync } from 'fs';
import { tmpdir } from 'os';
import { join, resolve, dirname } from 'path';
import { spawnSync } from 'child_process';
import { withEnv } from './helpers/with-env.ts';
import { autopilotLaunchdLabel } from '../src/core/autopilot-paths.ts';
const REPO = resolve(import.meta.dir, '..');
const CLI = join(REPO, 'src', 'cli.ts');
const SKIP_SUBPROCESS = process.env.GBRAIN_SKIP_SUBPROCESS_TESTS === '1';
// ── label seam unit tests (serial file on purpose: env mutation would
// interleave with other autopilot tests in the parallel lane) ─────────────
describe('autopilotLaunchdLabel', () => {
test('default label, env unset', async () => {
await withEnv({ GBRAIN_AUTOPILOT_LABEL: undefined }, async () => {
expect(autopilotLaunchdLabel()).toBe('com.gbrain.autopilot');
});
});
test('override lands in the generated guard bootout line', async () => {
await withEnv({ GBRAIN_AUTOPILOT_LABEL: 'com.gbrain.autopilot.test.override' }, async () => {
const { generateSelfDisableGuard } = await import('../src/commands/autopilot.ts');
const guard = generateSelfDisableGuard('/data/brain', 'macos');
expect(guard).toContain('gui/$(id -u)/com.gbrain.autopilot.test.override');
});
});
test('grammar: shell/filesystem metacharacters are rejected', async () => {
// The label reaches a plist FILENAME and a double-quoted shell line;
// escapeXml protects only the XML site.
for (const evil of ['a/b', 'a b', 'a"b', "a'b", 'a$b', 'a`b', 'a..b', '']) {
await withEnv({ GBRAIN_AUTOPILOT_LABEL: evil }, async () => {
expect(() => autopilotLaunchdLabel()).toThrow(/invalid GBRAIN_AUTOPILOT_LABEL/);
});
}
});
});
// ── keyless explicit-slug contract (spawn-free; the deliberate exit-1 half
// of the embed keyless fix) ───────────────────────────────────────────────
describe('embed keyless: explicit requests still fail loudly', () => {
test('runEmbedCore throws EmbeddingDisabledError for a named slug on a keyless brain', async () => {
// The --stale path refuses cleanly (exit 0) because the documented agent
// chain depends on it. A user who NAMED a page asked for something
// impossible on a keyless brain — that must stay a structured failure.
const { assertEmbeddingEnabled, EmbeddingDisabledError } = await import('../src/core/embedding-dim-check.ts');
expect(() => assertEmbeddingEnabled({ embedding_disabled: true } as never))
.toThrow(EmbeddingDisabledError);
});
});
// ── shared subprocess helpers ────────────────────────────────────────────────
interface CliResult { exitCode: number; stdout: string; stderr: string }
function runCli(args: string[], env: Record<string, string>, timeoutMs: number): CliResult {
const res = spawnSync('bun', ['run', CLI, ...args], {
cwd: REPO, // near-repo cwd keeps Bun's transpile cache warm (5-20s/spawn otherwise)
env,
encoding: 'utf8',
timeout: timeoutMs,
});
return { exitCode: res.status ?? -1, stdout: res.stdout ?? '', stderr: res.stderr ?? '' };
}
/** Minimal PATH: tmpbin first, then the running bun's dir, then the OS basics. */
function basePath(tmpbin: string): string {
const bunDir = dirname(process.execPath || '/usr/local/bin');
return `${tmpbin}:${bunDir}:/usr/bin:/bin:/usr/sbin:/sbin`;
}
/** Write the keyless PGLite config by hand (fast path; skips init's provider dance). */
function seedBrain(home: string, env: Record<string, string>): void {
mkdirSync(join(home, '.gbrain'), { recursive: true });
writeFileSync(
join(home, '.gbrain', 'config.json'),
JSON.stringify({
engine: 'pglite',
database_path: join(home, '.gbrain', 'brain.pglite'),
embedding_disabled: true,
}) + '\n',
);
const r = runCli(['init', '--migrate-only'], env, 120_000);
if (r.exitCode !== 0) {
throw new Error(`seedBrain init --migrate-only failed (${r.exitCode}):\n${r.stderr.slice(-2000)}`);
}
}
// ── Describe A: shimmed lifecycle (all platforms) ───────────────────────────
describe.skipIf(SKIP_SUBPROCESS)('autopilot launchd lifecycle — shimmed (all platforms)', () => {
const label = `com.gbrain.autopilot.test.${process.pid}`;
let home = '';
let tmpbin = '';
let repoDir = '';
let recordFile = '';
let breadcrumbFile = '';
let env: Record<string, string>;
const plist = () => join(home, 'Library', 'LaunchAgents', `${label}.plist`);
const wrapper = () => join(home, '.gbrain', 'autopilot-run.sh');
const marker = () => join(home, '.gbrain', 'autopilot-disabled');
const recorded = () => (existsSync(recordFile) ? readFileSync(recordFile, 'utf8') : '');
beforeAll(() => {
home = mkdtempSync(join(tmpdir(), 'gb-launchd-a-'));
tmpbin = mkdtempSync(join(tmpdir(), 'gb-launchd-bin-'));
repoDir = join(home, 'brain-repo');
mkdirSync(repoDir, { recursive: true });
recordFile = join(tmpbin, 'record.log');
breadcrumbFile = join(tmpbin, 'execed.log');
// launchctl argv recorder — exit 0 so install's execSync succeeds.
writeFileSync(join(tmpbin, 'launchctl'), `#!/bin/sh\necho "launchctl $*" >> '${recordFile}'\nexit 0\n`, { mode: 0o755 });
// crontab no-op — detectInstalledTarget falls through to the REAL
// `crontab -l` on non-darwin; a dev's real gbrain cron entry would flip
// the verdict to linux-cron.
writeFileSync(join(tmpbin, 'crontab'), `#!/bin/sh\nexit 0\n`, { mode: 0o755 });
// gbrain breadcrumb — `which gbrain` at install time bakes THIS into the
// wrapper's exec line. Records context, exits 0. NOTE: asserts "a process
// execed gbrain from PATH", not "the daemon started" — assert on context.
writeFileSync(join(tmpbin, 'gbrain'), `#!/bin/sh\necho "gbrain $*" >> '${breadcrumbFile}'\nexit 0\n`, { mode: 0o755 });
for (const f of ['launchctl', 'crontab', 'gbrain']) chmodSync(join(tmpbin, f), 0o755);
env = {
PATH: basePath(tmpbin),
HOME: home,
GBRAIN_HOME: home,
GBRAIN_AUTOPILOT_LABEL: label,
TMPDIR: tmpdir(),
};
seedBrain(home, env);
}, 180_000);
afterAll(() => {
rmSync(home, { recursive: true, force: true });
rmSync(tmpbin, { recursive: true, force: true });
});
test('1. virgin status → not_installed, exit 0', () => {
const r = runCli(['autopilot', '--status', '--json'], env, 90_000);
expect(r.exitCode).toBe(0);
const report = JSON.parse(r.stdout.trim().split('\n').pop()!);
expect(report.state).toBe('not_installed');
}, 120_000);
test('2. install --target macos: plist + wrapper + recorded load', () => {
const r = runCli(['autopilot', '--install', '--target', 'macos', '--repo', repoDir], env, 90_000);
expect(r.exitCode).toBe(0);
expect(existsSync(plist())).toBe(true);
const xml = readFileSync(plist(), 'utf8');
expect(xml).toContain(`<string>${label}</string>`);
expect(recorded()).toContain(`launchctl load ${plist()}`);
const w = readFileSync(wrapper(), 'utf8');
// Guard precedes exec — the self-disable must run before the daemon could start.
expect(w.indexOf('repo path gone')).toBeGreaterThan(-1);
expect(w.indexOf('repo path gone')).toBeLessThan(w.indexOf('exec '));
expect(w).toContain(`gui/$(id -u)/${label}`);
}, 120_000);
test('3. repo deleted → three strikes, THEN marker + recorded bootout', () => {
rmSync(repoDir, { recursive: true, force: true });
// One transient miss must not kill the install (repos on external or
// cloud-synced volumes are routinely absent right after login) — the
// guard requires three consecutive misses. Runs 1 and 2 exit 0 with no
// marker and no bootout; run 3 disables for real.
for (const run of [1, 2]) {
const res = spawnSync('bash', [wrapper()], { env, encoding: 'utf8', timeout: 30_000 });
expect(res.status).toBe(0);
expect(existsSync(marker())).toBe(false);
expect(recorded()).not.toContain('launchctl bootout gui/');
expect(run).toBeGreaterThan(0);
}
const res = spawnSync('bash', [wrapper()], { env, encoding: 'utf8', timeout: 30_000 });
expect(res.status).toBe(0);
expect(existsSync(marker())).toBe(true);
expect(readFileSync(marker(), 'utf8')).toContain(repoDir);
expect(recorded()).toContain(`launchctl bootout gui/`);
expect(recorded()).toContain(label);
}, 120_000);
test('4. status after self-disable → disabled, names the path, exit 2', () => {
const r = runCli(['autopilot', '--status', '--json'], env, 90_000);
expect(r.exitCode).toBe(2);
const report = JSON.parse(r.stdout.trim().split('\n').pop()!);
expect(report.state).toBe('disabled');
expect(report.disabled_reason).toContain(repoDir);
}, 120_000);
test('5. reinstall against a RECREATED repo clears the marker', () => {
// Recreate first — reinstalling against the deleted path would only prove
// marker-clearing while leaving an immediately-doomed install.
mkdirSync(repoDir, { recursive: true });
const r = runCli(['autopilot', '--install', '--target', 'macos', '--repo', repoDir], env, 90_000);
expect(r.exitCode).toBe(0);
expect(existsSync(marker())).toBe(false);
}, 120_000);
test('6. uninstall: plist gone, recorded unload, markers cleared', () => {
// Seed a disabled marker first: without the uninstall-time clear, a
// self-disabled-then-uninstalled machine reports "DISABLED" with exit 2
// forever ('disabled' outranks 'not_installed' in the classifier).
writeFileSync(marker(), 'repo path gone: /somewhere\n');
const r = runCli(['autopilot', '--uninstall'], env, 90_000);
expect(r.exitCode).toBe(0);
expect(existsSync(plist())).toBe(false);
expect(recorded()).toContain(`launchctl unload ${plist()}`);
expect(existsSync(marker())).toBe(false);
const status = runCli(['autopilot', '--status', '--json'], env, 90_000);
expect(JSON.parse(status.stdout).state).toBe('not_installed');
expect(status.exitCode).toBe(0);
}, 240_000);
test('7. the daemon never started: breadcrumb has no autopilot exec', () => {
const crumbs = existsSync(breadcrumbFile) ? readFileSync(breadcrumbFile, 'utf8') : '';
expect(crumbs).not.toContain('autopilot --repo');
});
});
// ── Describe B: REAL launchd (darwin, fail-SKIP) ────────────────────────────
function canUseLaunchd(): boolean {
if (process.platform !== 'darwin') return false;
if (process.env.GBRAIN_SKIP_LAUNCHD_E2E === '1') return false;
if (typeof process.getuid !== 'function') return false;
const probe = spawnSync('launchctl', ['print', `gui/${process.getuid()}`], { encoding: 'utf8', timeout: 10_000 });
return probe.status === 0;
}
const LAUNCHD_OK = canUseLaunchd();
if (!LAUNCHD_OK && process.env.GBRAIN_REQUIRE_LAUNCHD === '1') {
throw new Error('[autopilot-launchd-lifecycle] GBRAIN_REQUIRE_LAUNCHD=1 but the launchd GUI domain is unavailable');
}
describe.skipIf(SKIP_SUBPROCESS || !LAUNCHD_OK)('autopilot launchd lifecycle — REAL launchd (darwin)', () => {
const uid = typeof process.getuid === 'function' ? process.getuid() : 0;
const label = `com.gbrain.autopilot.e2e.${process.pid}.${Date.now()}`;
let home = '';
let tmpbin = '';
let repoDir = '';
let breadcrumbFile = '';
let env: Record<string, string>;
const plist = () => join(home, 'Library', 'LaunchAgents', `${label}.plist`);
const marker = () => join(home, '.gbrain', 'autopilot-disabled');
const printJob = () => spawnSync('launchctl', ['print', `gui/${uid}/${label}`], { encoding: 'utf8', timeout: 10_000 });
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
beforeAll(() => {
home = mkdtempSync(join(tmpdir(), 'gb-launchd-b-'));
tmpbin = mkdtempSync(join(tmpdir(), 'gb-launchd-b-bin-'));
repoDir = join(home, 'brain-repo');
mkdirSync(repoDir, { recursive: true });
breadcrumbFile = join(tmpbin, 'execed.log');
// gbrain breadcrumb only — NO launchctl shim here: install must reach the
// real launchd. Breadcrumb path baked ABSOLUTE: launchd will not pass our
// test env to the wrapper it spawns.
writeFileSync(join(tmpbin, 'gbrain'), `#!/bin/sh\necho "gbrain $*" >> '${breadcrumbFile}'\nexit 0\n`, { mode: 0o755 });
env = {
PATH: basePath(tmpbin),
HOME: home,
GBRAIN_HOME: home,
GBRAIN_AUTOPILOT_LABEL: label,
TMPDIR: tmpdir(),
};
seedBrain(home, env);
}, 180_000);
afterAll(() => {
// Own-label only. Bootout BEFORE removing files: launchd never rescans a
// tmp-HOME LaunchAgents dir, so a leftover job dies at logout regardless,
// but booting first keeps the teardown quiet.
spawnSync('launchctl', ['bootout', `gui/${uid}/${label}`], { timeout: 10_000 });
rmSync(home, { recursive: true, force: true });
rmSync(tmpbin, { recursive: true, force: true });
});
test('1. install loads a real job and RunAtLoad runs our wrapper', async () => {
const r = runCli(['autopilot', '--install', '--target', 'macos', '--repo', repoDir], env, 90_000);
expect(r.exitCode).toBe(0);
expect(existsSync(plist())).toBe(true);
// Breadcrumb = launchd genuinely spawned the wrapper AND the wrapper ran
// end-to-end to its exec line (repo exists, guard passed). This is the
// "install verified nothing" gap actually closing.
let sawBreadcrumb = false;
for (let i = 0; i < 60; i++) {
if (existsSync(breadcrumbFile) && readFileSync(breadcrumbFile, 'utf8').includes('autopilot --repo')) {
sawBreadcrumb = true;
break;
}
await sleep(500);
}
if (!sawBreadcrumb) {
const p = printJob();
throw new Error(
`wrapper never ran under launchd within 30s.\n`
+ `Likely: a dotfile in the REAL user zshenv/zshrc exec'd another shell or stalled `
+ `(launchd runs the wrapper with your real HOME). Escape hatch: GBRAIN_SKIP_LAUNCHD_E2E=1.\n`
+ `launchctl print (exit ${p.status}):\n${(p.stdout || p.stderr).slice(0, 1500)}`,
);
}
expect(printJob().status).toBe(0); // job registered; KeepAlive keeps it so
}, 180_000);
test('2. repo deleted → guard self-disables: marker written, job gone', async () => {
rmSync(repoDir, { recursive: true, force: true });
// The guard needs three consecutive misses before disabling; under real
// launchd that is three ThrottleInterval windows (~3min). Pre-seed two
// strikes so the NEXT respawned run crosses the threshold — this still
// exercises the real counter read + threshold + bootout path while
// keeping the poll inside one throttle window.
writeFileSync(join(home, '.gbrain', 'autopilot-disable-strikes'), '2\n');
// Best-effort accelerant only. Observed live: kickstart BLOCKS (not
// errors) when the job is inside its ThrottleInterval window after the
// shim's quick exits, so we must not assert its exit code or depend on
// it at all. KeepAlive + ThrottleInterval=60 guarantees launchd respawns
// the wrapper naturally within one throttle window, and the respawned
// wrapper hits the guard. The poll therefore spans >60s.
spawnSync('launchctl', ['kickstart', '-k', `gui/${uid}/${label}`], { encoding: 'utf8', timeout: 10_000 });
let ok = false;
for (let i = 0; i < 180; i++) { // 90s: one full throttle window + slack
if (existsSync(marker()) && printJob().status !== 0) { ok = true; break; }
await sleep(500);
}
if (!ok) {
const p = printJob();
const log = join(home, '.gbrain', 'autopilot.log');
const err = join(home, '.gbrain', 'autopilot.err');
throw new Error(
`self-disable did not converge in 90s: marker=${existsSync(marker())} printExit=${p.status}\n`
+ `autopilot.log:\n${existsSync(log) ? readFileSync(log, 'utf8').slice(-1200) : '(absent)'}\n`
+ `autopilot.err:\n${existsSync(err) ? readFileSync(err, 'utf8').slice(-1200) : '(absent)'}`,
);
}
expect(readFileSync(marker(), 'utf8')).toContain(repoDir);
}, 240_000);
test('3. status reports disabled with exit 2', () => {
const r = runCli(['autopilot', '--status', '--json'], env, 90_000);
expect(r.exitCode).toBe(2);
const report = JSON.parse(r.stdout.trim().split('\n').pop()!);
expect(report.state).toBe('disabled');
}, 120_000);
});
+245
View File
@@ -0,0 +1,245 @@
/**
* PR-A / A3: the cooperative pause marker that lets `gbrain migrate --to <engine>`
* quiesce a running daemon for the duration of a cross-engine copy.
*
* The incident this closes: `migrate-engine.ts` contained ZERO references to
* autopilot, so an engine migration rewrote `config.json` while a live daemon
* kept writing into the SOURCE engine. Those writes were lost, and the daemon
* then died on a stale config object with no operator-visible link between the
* two events.
*
* EVERY test here runs against a scratch GBRAIN_HOME. The paths resolve through
* gbrainHomePath() at call time, so without the override this file would write
* and worse unconditionally DELETE markers in the developer's real
* ~/.gbrain, un-quiescing a live migration on the machine running the tests.
*/
import { describe, test, expect } from 'bun:test';
import { existsSync, writeFileSync, unlinkSync, mkdirSync, readFileSync, utimesSync } from 'fs';
import { spawnSync } from 'child_process';
import { dirname } from 'path';
import { withEnv, emptyHome } from './helpers/with-env.ts';
import {
autopilotPausedMarkerPath,
autopilotDisabledMarkerPath,
autopilotLockPath,
} from '../src/core/autopilot-paths.ts';
import { quiesceAutopilot, MIGRATE_PAUSE_MARKER_PREFIX } from '../src/commands/migrate-engine.ts';
/** Scratch home + no grace sleep: the default 35s would stall the suite. */
function quiesceEnv(extra: Record<string, string | undefined> = {}) {
return { GBRAIN_HOME: emptyHome(), GBRAIN_MIGRATE_QUIESCE_SECONDS: '0', ...extra };
}
function touchLock(mtime?: Date) {
const lock = autopilotLockPath();
mkdirSync(dirname(lock), { recursive: true });
writeFileSync(lock, `${process.pid}\n`);
if (mtime) utimesSync(lock, mtime, mtime);
}
describe('autopilot pause marker (A3 quiesce)', () => {
test('resolves inside the gbrain home, not raw $HOME', async () => {
await withEnv({ GBRAIN_HOME: emptyHome() }, () => {
// The daemon writes its lock through gbrainHomePath(); a marker resolved
// from raw process.env.HOME would land in a different directory under
// GBRAIN_HOME and the pause would silently never be observed.
const p = autopilotPausedMarkerPath();
expect(p.endsWith('autopilot-paused')).toBe(true);
expect(p).toContain(process.env.GBRAIN_HOME!);
});
});
test('pause and disabled markers are distinct files', async () => {
await withEnv({ GBRAIN_HOME: emptyHome() }, () => {
// They mean different things: paused = "a migration is running, hold";
// disabled = "your repo is gone, I stopped myself". Collapsing them would
// make a migration look like a fault in `--status`.
expect(autopilotPausedMarkerPath()).not.toBe(autopilotDisabledMarkerPath());
});
});
test('presence is the whole protocol — create then remove', async () => {
await withEnv({ GBRAIN_HOME: emptyHome() }, () => {
const p = autopilotPausedMarkerPath();
mkdirSync(dirname(p), { recursive: true });
expect(existsSync(p)).toBe(false);
writeFileSync(p, `${MIGRATE_PAUSE_MARKER_PREFIX} at 2026-08-11T00:00:00.000Z\n`);
expect(existsSync(p)).toBe(true);
unlinkSync(p);
expect(existsSync(p)).toBe(false);
});
});
test('marker content is human-readable, so an orphan explains itself', () => {
// If a migration crashes hard the marker survives. Whoever finds it must be
// able to tell what wrote it and when, rather than guessing why autopilot
// stopped doing work.
const body = `${MIGRATE_PAUSE_MARKER_PREFIX} at ${new Date(0).toISOString()}\n`;
expect(body).toContain('gbrain migrate');
expect(body).toMatch(/\d{4}-\d{2}-\d{2}T/);
});
});
describe('quiesceAutopilot (migrate-side of the protocol)', () => {
test('no daemon lock → marker STILL written (red-team: the no-lock window is routine)', async () => {
await withEnv(quiesceEnv(), async () => {
// The daemon unlinks its lock on every clean exit and the supervisor
// relaunches it within a minute; cron installs have no lock between
// runs. A migrate that skipped the marker in such a window would let a
// daemon that STARTS mid-copy write into the source engine — all lost
// at the flip. So the marker is unconditional; only the grace sleep
// keys on a live heartbeat.
const resume = await quiesceAutopilot();
expect(resume).not.toBeNull();
expect(existsSync(autopilotPausedMarkerPath())).toBe(true);
resume!();
expect(existsSync(autopilotPausedMarkerPath())).toBe(false);
});
});
test('live lock → marker written with owned prefix + pid; resume removes it, twice-safe', async () => {
await withEnv(quiesceEnv(), async () => {
touchLock();
const resume = await quiesceAutopilot();
expect(resume).not.toBeNull();
const marker = autopilotPausedMarkerPath();
expect(existsSync(marker)).toBe(true);
const body = readFileSync(marker, 'utf8');
expect(body.startsWith(MIGRATE_PAUSE_MARKER_PREFIX)).toBe(true);
expect(body).toContain(`(pid ${process.pid})`);
resume!();
expect(existsSync(marker)).toBe(false);
resume!(); // idempotent — the finally + cleanup backstop may both fire
expect(existsSync(marker)).toBe(false);
});
});
test('foreign marker → null (caller must ABORT), and the marker is untouched', async () => {
await withEnv(quiesceEnv(), async () => {
touchLock();
const marker = autopilotPausedMarkerPath();
mkdirSync(dirname(marker), { recursive: true });
writeFileSync(marker, 'paused by an operator for maintenance\n');
// Codex catch: the old contract returned a no-op resume and let the
// migration proceed UNOWNED — when the real owner finished, its cleanup
// un-paused the daemon in the middle of our copy window. Null forces
// the caller to refuse to run instead.
expect(await quiesceAutopilot()).toBeNull();
expect(existsSync(marker)).toBe(true);
expect(readFileSync(marker, 'utf8')).toContain('operator');
});
});
test('adopts a DEAD migrate\'s orphan: the marker does not become permanent', async () => {
await withEnv(quiesceEnv(), async () => {
touchLock();
const marker = autopilotPausedMarkerPath();
mkdirSync(dirname(marker), { recursive: true });
// A prior migrate wrote this and died before its finally could run.
// Refusing to adopt would park the daemon forever: every retry would
// defer to the "foreign" marker, and nothing else ever deletes it.
// A real reaped pid, so the liveness probe sees ESRCH.
const dead = spawnSync('true');
writeFileSync(marker, `${MIGRATE_PAUSE_MARKER_PREFIX} (pid ${dead.pid}) at ${new Date(0).toISOString()}\n`);
const resume = await quiesceAutopilot();
expect(resume).not.toBeNull();
expect(existsSync(marker)).toBe(true); // still paused during the copy
// Ownership is TAKEN, not shared: the body now carries OUR pid, so a
// third migrate reads a live holder and aborts instead of also adopting
// (double-adoption would un-pause the daemon mid-copy).
expect(readFileSync(marker, 'utf8')).toContain(`(pid ${process.pid})`);
resume!();
expect(existsSync(marker)).toBe(false); // and released after
});
});
test('does NOT adopt a LIVE migrate\'s marker — resume must not un-pause a concurrent copy', async () => {
await withEnv(quiesceEnv(), async () => {
touchLock();
const marker = autopilotPausedMarkerPath();
mkdirSync(dirname(marker), { recursive: true });
// Same prefix as ours, but the holder pid is alive (this very process).
// Red-team catch: prefix-only adoption let migrate B adopt migrate A's
// live marker; B's finally then deleted it mid-copy and the daemon
// resumed inside A's copy window — the exact incident this closes.
writeFileSync(marker, `${MIGRATE_PAUSE_MARKER_PREFIX} (pid ${process.pid}) at ${new Date(0).toISOString()}\n`);
expect(await quiesceAutopilot()).toBeNull();
expect(existsSync(marker)).toBe(true); // untouched — the live holder owns it
});
});
test('a prefix-matching marker WITHOUT a pid is treated as foreign, not adopted', async () => {
await withEnv(quiesceEnv(), async () => {
touchLock();
const marker = autopilotPausedMarkerPath();
mkdirSync(dirname(marker), { recursive: true });
// No ownership handle → cannot prove the holder is dead → defer.
writeFileSync(marker, `${MIGRATE_PAUSE_MARKER_PREFIX} at ${new Date(0).toISOString()}\n`);
expect(await quiesceAutopilot()).toBeNull();
expect(existsSync(marker)).toBe(true);
});
});
test('garbage grace env falls back with a warning instead of silently disabling', async () => {
await withEnv(quiesceEnv({ GBRAIN_MIGRATE_QUIESCE_SECONDS: 'garbage' }), async () => {
// Stale lock (aged past the heartbeat window) so no code path can sleep:
// the marker still gets written — quiesce keys on lock EXISTENCE — but
// the grace is skipped, letting this assert the NaN warning without a
// 35s stall. Pre-fix, Number('garbage') = NaN made `NaN > 0` false and
// the grace vanished with no signal at all.
touchLock(new Date(Date.now() - 3600_000));
const warns: string[] = [];
const origWarn = console.warn;
console.warn = (...a: unknown[]) => { warns.push(a.join(' ')); };
try {
const resume = await quiesceAutopilot();
expect(resume).not.toBeNull();
expect(existsSync(autopilotPausedMarkerPath())).toBe(true);
resume!();
} finally {
console.warn = origWarn;
}
expect(warns.join('\n')).toContain('GBRAIN_MIGRATE_QUIESCE_SECONDS');
});
});
});
describe('release is ownership-conditional', () => {
test('resume does NOT delete a marker that changed hands', async () => {
await withEnv(quiesceEnv(), async () => {
// Round-4 review catch: an adoption race resolved against us means the
// marker at this PATH now belongs to another migrate. A path-only
// unlink would un-pause the daemon in the middle of the new owner's
// copy window; release must compare content first.
const resume = await quiesceAutopilot();
expect(resume).not.toBeNull();
const marker = autopilotPausedMarkerPath();
writeFileSync(marker, `${MIGRATE_PAUSE_MARKER_PREFIX} (pid 999999) at ${new Date(0).toISOString()}\n`);
resume!();
expect(existsSync(marker)).toBe(true); // still there — not ours anymore
});
});
});
describe('pause marker fences minion job pickup', () => {
test('worker claim loop checks the marker BEFORE queue.claim (static wiring guard)', async () => {
// Third review round: the marker parked the autopilot dispatch loop but a
// minion worker kept CLAIMING queued jobs mid-copy — writes the migration
// silently lost at the flip. No in-process worker harness exists, so pin
// the wiring statically (the autopilot-supervisor-wiring pattern): the
// gate must sit in the poll loop ahead of the claim call.
const { readFileSync } = await import('fs');
const { resolve } = await import('path');
const src = readFileSync(resolve(import.meta.dir, '../src/core/minions/worker.ts'), 'utf8');
const gateIdx = src.indexOf('existsSync(autopilotPausedMarkerPath())');
const claimIdx = src.indexOf('this.queue.claim(');
expect(gateIdx).toBeGreaterThan(-1);
expect(claimIdx).toBeGreaterThan(gateIdx);
// And the POST-claim re-check (round 4): a claim can commit after
// migrate's drain probe; a job claimed into that window is released
// back un-run rather than executed.
const postGateIdx = src.indexOf('existsSync(autopilotPausedMarkerPath())', claimIdx);
expect(postGateIdx).toBeGreaterThan(claimIdx);
expect(src.indexOf('releaseClaimForPause(', postGateIdx)).toBeGreaterThan(postGateIdx);
});
});
@@ -21,6 +21,42 @@ describe('classifyReconnectError (#1162)', () => {
expect(classifyReconnectError(err)).toBe('unrecoverable');
});
describe('crash vs config verdict — the 71-day-outage class', () => {
test('real Bun null-deref → crash, NOT unrecoverable', () => {
// Verbatim Bun/V8 text. Lowercased it contains BOTH "database_url" and
// "undefined", so the substring rules below used to classify it as a
// config verdict and exit the daemon permanently. It is a gbrain bug.
const err = new TypeError("undefined is not an object (evaluating 'config.database_url')");
expect(classifyReconnectError(err)).toBe('crash');
});
test('any TypeError → crash, by error type', () => {
expect(classifyReconnectError(new TypeError('cfg.foo is not a function'))).toBe('crash');
});
test('V8-worded null-deref arriving as a plain Error → crash, by message shape', () => {
// Some hosts lose the error class across a serialization boundary.
expect(classifyReconnectError(new Error("Cannot read properties of undefined (reading 'database_url')")))
.toBe('crash');
});
test('a genuine missing-config error is still unrecoverable', () => {
// Guard against over-broadening: the crash rules must not swallow the
// real "operator never set a URL" case, which SHOULD stop the daemon.
expect(classifyReconnectError(new Error('GBRAIN_DATABASE_URL is not set'))).toBe('unrecoverable');
expect(classifyReconnectError(new Error('No brain configured. Run: gbrain init'))).toBe('unrecoverable');
});
test('a malformed URL is a config verdict even though JS throws it as a TypeError', () => {
// `new URL('garbage')` throws TypeError('Invalid URL'). The blanket
// TypeError-means-crash rule would spend the whole 30-attempt reconnect
// budget on an error only the operator can fix (adversarial review
// catch); the message pattern is tested first.
expect(classifyReconnectError(new TypeError('Invalid URL'))).toBe('unrecoverable');
expect(classifyReconnectError(new Error('failed to parse URL from postgres://'))).toBe('unrecoverable');
});
});
test('database_url empty → unrecoverable', () => {
expect(classifyReconnectError(new Error('database_url is empty'))).toBe('unrecoverable');
});
@@ -0,0 +1,229 @@
/**
* PR-A: the two surfaces that make a dead autopilot visible.
*
* generateSelfDisableGuard stops a daemon whose --repo vanished, for real,
* under a supervisor that restarts it on every exit.
* classifyAutopilotStatus tri-state verdict + exit code, so `--status` can
* no longer answer "installed" for a daemon that died 71 days ago.
*
* Both are pure, so no daemon is installed here.
*/
import { describe, test, expect } from 'bun:test';
import {
generateSelfDisableGuard,
crontabIndicatesAutopilotInstall,
classifyAutopilotStatus,
autopilotStatusExitCode,
autopilotLaunchdLabel,
autopilotEngineIdentity,
AUTOPILOT_SYSTEMD_UNIT,
} from '../src/commands/autopilot.ts';
describe('generateSelfDisableGuard', () => {
test('tests the repo DIRECTORY, never $repo/.git', () => {
const guard = generateSelfDisableGuard('/data/brain', 'macos');
expect(guard).toContain("[ ! -d '/data/brain' ]");
// .git is a FILE in worktrees and submodules, and --repo may be a
// subdirectory of the checkout — either shape would false-positive and
// disable a healthy install. Assert on the executable line only; the
// surrounding comment mentions .git precisely to explain the trap.
const testLine = guard.split('\n').find((l) => l.startsWith('if [ '))!;
expect(testLine).toBe("if [ ! -d '/data/brain' ]; then");
expect(testLine).not.toContain('.git');
});
test('macOS actually boots the job out — exit 0 alone is a respawn loop', () => {
// KeepAlive=true means launchd restarts on ANY exit, so "detect and exit"
// disables nothing; it just makes the failure quieter and more frequent.
const guard = generateSelfDisableGuard('/data/brain', 'macos');
expect(guard).toContain('launchctl bootout');
expect(guard).toContain(autopilotLaunchdLabel());
});
test('systemd disables the unit — Restart=always has the same problem', () => {
const guard = generateSelfDisableGuard('/data/brain', 'linux-systemd');
expect(guard).toContain('systemctl --user disable --now');
expect(guard).toContain(AUTOPILOT_SYSTEMD_UNIT);
});
test('cron and container just exit — they are periodic/one-shot, not KeepAlive', () => {
for (const target of ['linux-cron', 'ephemeral-container'] as const) {
const guard = generateSelfDisableGuard('/data/brain', target);
expect(guard).toContain('exit 0');
expect(guard).not.toContain('launchctl bootout');
expect(guard).not.toContain('systemctl --user disable');
}
});
test('writes a marker so --status can explain WHY it stopped', () => {
expect(generateSelfDisableGuard('/data/brain', 'macos')).toContain('autopilot-disabled');
});
test('single-quotes in the repo path cannot break out of the shell literal', () => {
const guard = generateSelfDisableGuard("/data/o'brien/brain", 'macos');
expect(guard).toContain("'/data/o'\\''brien/brain'");
});
test('one missed probe is a strike, not a disable — three consecutive misses required', () => {
// Repos on external volumes, NFS, or cloud-synced folders are routinely
// absent for the first launch after login. A single-probe guard would
// permanently bootout a healthy install on one transient miss (adversarial
// review catch); the counter makes disable require persistence.
const guard = generateSelfDisableGuard('/data/brain', 'macos');
expect(guard).toContain('autopilot-disable-strikes');
expect(guard).toContain('-lt 3');
// Below the threshold the guard exits WITHOUT writing the disabled marker
// or booting the job out: the early-exit line precedes both.
const strikeExit = guard.indexOf('exit 0');
expect(strikeExit).toBeGreaterThan(-1);
expect(strikeExit).toBeLessThan(guard.indexOf('autopilot-disabled'));
expect(strikeExit).toBeLessThan(guard.indexOf('launchctl bootout'));
// A successful probe resets the counter, or three misses spread over
// months would accumulate into a false disable.
expect(guard.trimEnd().endsWith("2>/dev/null || true")).toBe(true);
expect(guard.split('rm -f').length).toBeGreaterThanOrEqual(3);
});
});
describe('crontabIndicatesAutopilotInstall', () => {
test('wrapper line and direct daemon line count as installs', () => {
expect(crontabIndicatesAutopilotInstall("*/5 * * * * '/h/.gbrain/autopilot-run.sh' >> log 2>&1")).toBe(true);
expect(crontabIndicatesAutopilotInstall('*/5 * * * * gbrain autopilot --repo /data/brain')).toBe(true);
});
test('a status-monitor cron line is NOT an install', () => {
// docs recommend cron-ing the status command as a health gate; counting
// it as an install makes a monitor-only machine report installed and
// never_run with exit 1 forever (adversarial review catch).
expect(crontabIndicatesAutopilotInstall('*/10 * * * * gbrain autopilot --status --json >> health.log')).toBe(false);
});
test('comments and unrelated lines never count', () => {
expect(crontabIndicatesAutopilotInstall('# gbrain autopilot --repo /data/brain (disabled)')).toBe(false);
expect(crontabIndicatesAutopilotInstall('0 3 * * * /usr/local/bin/backup.sh')).toBe(false);
expect(crontabIndicatesAutopilotInstall('')).toBe(false);
});
});
describe('classifyAutopilotStatus', () => {
const base = {
installed: true,
installTarget: 'macos' as const,
disabledReason: null as string | null,
pausedReason: null as string | null,
heartbeatAgeSeconds: 10,
intervalSeconds: 300,
lastLog: '',
};
test('THE INCIDENT: installed with a 71-day-old heartbeat → stale, exit 1', () => {
// This is the case that reported "Autopilot: installed" and exited 0 for
// 71 days while the daemon was dead.
const r = classifyAutopilotStatus({ ...base, heartbeatAgeSeconds: 71 * 86400 });
expect(r.state).toBe('stale');
expect(autopilotStatusExitCode(r.state)).toBe(1);
});
test('fresh heartbeat → fresh, exit 0', () => {
const r = classifyAutopilotStatus(base);
expect(r.state).toBe('fresh');
expect(autopilotStatusExitCode(r.state)).toBe(0);
});
test('nothing installed → exit 0 (nothing claimed, nothing broken)', () => {
const r = classifyAutopilotStatus({ ...base, installed: false, installTarget: null, heartbeatAgeSeconds: null });
expect(r.state).toBe('not_installed');
expect(autopilotStatusExitCode(r.state)).toBe(0);
});
test('installed but never ticked → never_run, exit 1', () => {
const r = classifyAutopilotStatus({ ...base, heartbeatAgeSeconds: null });
expect(r.state).toBe('never_run');
expect(autopilotStatusExitCode(r.state)).toBe(1);
});
test('self-disabled wins over every other signal, exit 2', () => {
const r = classifyAutopilotStatus({ ...base, disabledReason: 'repo path gone: /data/brain' });
expect(r.state).toBe('disabled');
expect(r.disabled_reason).toContain('/data/brain');
expect(autopilotStatusExitCode(r.state)).toBe(2);
});
test('a healthy adaptive-schedule gap does not flip the verdict — a real outage does', () => {
// The adaptive scheduler sleeps TWO intervals between ticks on the
// healthiest brains and the heartbeat only refreshes at tick top, so a
// normal healthy gap is cycle_duration + 2x interval. Tolerance is 6x —
// review-army catch: at 3x, the best-behaved installs flapped stale
// exit-1 alarms in the exact cron gates this exit code ships for.
expect(classifyAutopilotStatus({ ...base, heartbeatAgeSeconds: 301 }).state).toBe('fresh');
expect(classifyAutopilotStatus({ ...base, heartbeatAgeSeconds: 1799 }).state).toBe('fresh');
expect(classifyAutopilotStatus({ ...base, heartbeatAgeSeconds: 1801 }).state).toBe('stale');
});
test('tolerance scales with the interval, not a hardcoded constant', () => {
const slow = classifyAutopilotStatus({ ...base, intervalSeconds: 3600, heartbeatAgeSeconds: 5000 });
expect(slow.stale_after_seconds).toBe(21600);
expect(slow.state).toBe('fresh');
});
test('a garbage interval cannot silence the alarm — NaN staleAfter would read dead as fresh', () => {
// Pre-fix: `--interval garbage` parsed to NaN, staleAfter = NaN, and every
// age comparison came back false — the 71-day-dead daemon reported 'fresh'
// with exit 0. The pure layer now falls back to the default tolerance.
const r = classifyAutopilotStatus({ ...base, intervalSeconds: NaN, heartbeatAgeSeconds: 71 * 86400 });
expect(r.stale_after_seconds).toBe(1800);
expect(r.state).toBe('stale');
expect(autopilotStatusExitCode(r.state)).toBe(1);
expect(classifyAutopilotStatus({ ...base, intervalSeconds: 0 }).stale_after_seconds).toBe(1800);
});
test('pause marker → paused, exit 1: a parked daemon must not report running', () => {
// The tick loop refreshes its heartbeat BEFORE honoring the pause marker,
// so a quiesced daemon looks 'fresh' by mtime while doing no work. Without
// this state, a marker orphaned by a dead migrate is invisible — the
// 71-day incident's shape again, reached through the pause path.
const r = classifyAutopilotStatus({ ...base, pausedReason: 'paused by gbrain migrate at 2026-08-11T00:00:00Z' });
expect(r.state).toBe('paused');
expect(r.paused_reason).toContain('gbrain migrate');
expect(autopilotStatusExitCode(r.state)).toBe(1);
});
test('disabled outranks paused; a leftover marker with nothing installed stays quiet', () => {
// disabled is the stronger, permanent claim (exit 2). And with no install
// at all, a stray marker pauses nothing — alarming on it would make a
// clean machine fail its cron gate forever.
const both = classifyAutopilotStatus({ ...base, disabledReason: 'repo path gone', pausedReason: 'paused' });
expect(both.state).toBe('disabled');
const stray = classifyAutopilotStatus({
...base, installed: false, installTarget: null, heartbeatAgeSeconds: null, pausedReason: 'paused',
});
expect(stray.state).toBe('not_installed');
expect(autopilotStatusExitCode(stray.state)).toBe(0);
});
});
describe('autopilotEngineIdentity (post-migration convergence)', () => {
// Red-team catch: after a clean migration flipped config.json, the daemon
// never converged — its health probe kept succeeding against the OLD engine
// (the preserved backup stays alive) and reconnect() deliberately restores
// the config captured at connect(). The tick loop now compares this
// identity against its boot value and exits for supervisor relaunch.
test('changes when the engine kind or connection target changes', () => {
const pglite = autopilotEngineIdentity({ engine: 'pglite', database_path: '/a/brain.pglite' });
expect(autopilotEngineIdentity({ engine: 'postgres', database_url: 'postgres://x/db' })).not.toBe(pglite);
expect(autopilotEngineIdentity({ engine: 'pglite', database_path: '/b/brain.pglite' })).not.toBe(pglite);
});
test('ignores everything a migration does not flip — unrelated config edits must not restart the daemon', () => {
const before = autopilotEngineIdentity({ engine: 'pglite', database_path: '/a/brain.pglite' });
const after = autopilotEngineIdentity({
engine: 'pglite', database_path: '/a/brain.pglite',
embedding_model: 'voyage-3', search: { mode: 'tokenmax' },
} as Parameters<typeof autopilotEngineIdentity>[0]);
expect(after).toBe(before);
});
test('a missing config file reads as the pglite default, not a crash', () => {
expect(autopilotEngineIdentity(null)).toBe(autopilotEngineIdentity({ engine: 'pglite' }));
});
});
+57 -2
View File
@@ -1097,7 +1097,7 @@ describe('v0.41.32.0 — commit-relative staleness', () => {
expect(result.details).toEqual({ unchanged_count: 1, synced_recently_count: 0, stale_count: 0 });
});
test('T2: REMOTE (no localOnly) reads column, quiet repo → ok, NO git subprocess', async () => {
test('T2: REMOTE (no localOnly) reads column, quiet + recently synced → ok, NO git subprocess', async () => {
const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } =
await import('../src/core/git-head.ts');
@@ -1107,7 +1107,9 @@ describe('v0.41.32.0 — commit-relative staleness', () => {
const result = await checkSyncFreshness(makeStubEngine([
{ id: 'remote', name: '', local_path: '/tmp/remote',
last_sync_at: agoMs(100 * 60 * 60 * 1000),
// Synced 1h ago: caught up AND being checked. Must stay ok — this is the
// quiet-source contract the content comparison exists to protect.
last_sync_at: agoMs(1 * 60 * 60 * 1000),
last_commit: 'x', chunker_version: currentChunkerVersion,
// Content committed BEFORE the last sync → caught up.
newest_content_at: agoMs(200 * 60 * 60 * 1000) },
@@ -1119,6 +1121,30 @@ describe('v0.41.32.0 — commit-relative staleness', () => {
expect(result.details).toEqual({ unchanged_count: 0, synced_recently_count: 1, stale_count: 0 });
});
test('T2-ceiling: REMOTE, caught up but unsynced past the ceiling → NOT ok', async () => {
// The 71-day bug at the doctor layer. This shape (caught up, synced 100h ago)
// previously reported ok forever, so a dead daemon was invisible. The ceiling
// ramps it to ~28h, which crosses the 24h warn line but NOT the 72h fail line —
// proving the escalation is ordered rather than a single jump to fail.
const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } =
await import('../src/core/git-head.ts');
let headCalls = 0;
_setGitHeadProbeForTests(() => { headCalls++; return 'x'; });
_setGitCleanProbeForTests(() => true);
const result = await checkSyncFreshness(makeStubEngine([
{ id: 'abandoned', name: '', local_path: '/tmp/abandoned',
last_sync_at: agoMs(100 * 60 * 60 * 1000),
last_commit: 'x', chunker_version: currentChunkerVersion,
newest_content_at: agoMs(200 * 60 * 60 * 1000) },
]));
expect(headCalls).toBe(0); // still no subprocess on the remote path
expect(result.status).toBe('warn'); // 28h: past warn, short of fail
expect(result.details).toEqual({ unchanged_count: 0, synced_recently_count: 0, stale_count: 1 });
});
test('T2b: REMOTE + NULL column → wall-clock fallback → stale (no git subprocess)', async () => {
const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
const { _setGitHeadProbeForTests, _setGitCleanProbeForTests } =
@@ -1621,6 +1647,35 @@ describe('BUG 4 — in-progress sync via live lock, not stale freshness', () =>
expect(result.status).toBe('fail');
});
test('a lock held PAST the staleness ceiling is wedged, not in-progress → fail', async () => {
// `withRefreshingLock` bumps the heartbeat on its own timer regardless of
// whether the import is making forward progress, so a holder blocked inside
// a query refreshes forever. An uncapped in-progress verdict would mask that
// source from every staleness check indefinitely — the same invisible-failure
// class as the freshness bug, reached through the lock table instead.
const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
await addSource('wiki', staleDate());
// Live, actively-refreshing lock (TTL 30 min in the future) — but acquired
// 100h ago, well past the 72h default ceiling.
await engine.executeRaw(
`INSERT INTO gbrain_cycle_locks (id, holder_pid, holder_host, acquired_at, ttl_expires_at, last_refreshed_at)
VALUES ($1, 4242, 'testhost', now() - interval '100 hours', now() + interval '30 minutes', now())`,
[syncLockId('wiki')],
);
const result = await checkSyncFreshness(engine);
expect(result.status).toBe('fail');
expect(result.message).toContain('held the sync lock');
expect(result.message).toContain('break-lock');
});
test('a freshly-acquired live lock is still in-progress → ok (cap does not over-fire)', async () => {
const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
await addSource('wiki', staleDate());
await holdLock('wiki', 30); // acquired_at = now(), well under the ceiling
const result = await checkSyncFreshness(engine);
expect(result.status).toBe('ok');
});
test('blocked source with banked checkpoint rows but NO live lock → still fail', async () => {
const { checkSyncFreshness } = await import('../src/commands/doctor.ts');
await addSource('wiki', staleDate());
+45
View File
@@ -0,0 +1,45 @@
/**
* The keyless clean-refusal predicate for `gbrain embed`.
*
* The documented always-current chain for external agent schedulers is
* `gbrain sync ... && gbrain embed --stale`; on a keyless brain that chain
* must exit 0 (nothing to backfill is not a failure). Every OTHER spelling is
* an explicit request for something impossible on a keyless brain and keeps
* exiting 1 via EmbeddingDisabledError the predicate's exclusions mirror
* runEmbed's dispatch precedence, where a slugs flag wins over stale.
*/
import { describe, test, expect } from 'bun:test';
import { isKeylessStaleRefusal } from '../src/commands/embed.ts';
describe('isKeylessStaleRefusal', () => {
test('the documented chain spelling refuses cleanly on a keyless brain', () => {
expect(isKeylessStaleRefusal(['--stale'], true)).toBe(true);
});
test('never fires on a brain with embeddings enabled (or unknown)', () => {
expect(isKeylessStaleRefusal(['--stale'], false)).toBe(false);
expect(isKeylessStaleRefusal(['--stale'], undefined)).toBe(false);
});
test('a slugs list keeps exiting 1 — dispatch gives slugs precedence over stale', () => {
// Review-army catch: the guard originally excluded only the all flag, so
// an explicit `embed --slugs a b --stale` on a keyless brain exited 0
// while doing nothing — contradicting the guard's own contract.
expect(isKeylessStaleRefusal(['--slugs', 'a', 'b', '--stale'], true)).toBe(false);
});
test('the all flag and dry-run are excluded — explicit asks stay loud', () => {
expect(isKeylessStaleRefusal(['--all', '--stale'], true)).toBe(false);
expect(isKeylessStaleRefusal(['--stale', '--dry-run'], true)).toBe(false);
});
test('a bare stale run with routing flags still qualifies', () => {
// Scoping flags do not change WHAT is being asked, only where.
expect(isKeylessStaleRefusal(['--stale', '--source', 'wiki'], true)).toBe(true);
});
test('no stale flag, no refusal — an explicit slug is not a stale run', () => {
expect(isKeylessStaleRefusal(['my-page'], true)).toBe(false);
expect(isKeylessStaleRefusal([], true)).toBe(false);
});
});
+107
View File
@@ -0,0 +1,107 @@
/**
* Direct coverage for the shared numeric env-var resolver.
*
* This module exists so doctor.ts and source-health.ts share ONE warn-once
* memo (a duplicate helper would warn once per module "warn exactly once"
* quietly becoming "warn twice"). It backs the staleness-ceiling resolution
* that closed the 71-day false-green, so its garbage-handling is load-bearing:
* an env typo that silently produced NaN here would resurface the incident.
*/
import { describe, test, expect, beforeEach } from 'bun:test';
import { withEnv } from './helpers/with-env.ts';
import {
resolveEnvNumber,
resolveHoursEnv,
warnOnceForEnv,
_resetEnvNumberWarnedForTests,
} from '../src/core/env-number.ts';
const VAR = 'GBRAIN_ENV_NUMBER_TEST_VAR';
function captureWarns(fn: () => void): string[] {
const warns: string[] = [];
const orig = console.warn;
console.warn = (...a: unknown[]) => { warns.push(a.join(' ')); };
try { fn(); } finally { console.warn = orig; }
return warns;
}
beforeEach(() => _resetEnvNumberWarnedForTests());
describe('resolveEnvNumber', () => {
test('unset and empty both mean fallback, with no warning', async () => {
await withEnv({ [VAR]: undefined }, () => {
const warns = captureWarns(() => {
expect(resolveEnvNumber(VAR, 72)).toBe(72);
});
expect(warns).toEqual([]);
});
await withEnv({ [VAR]: '' }, () => {
expect(resolveEnvNumber(VAR, 72)).toBe(72);
});
});
test('a valid positive number wins, including fractional', async () => {
await withEnv({ [VAR]: '36' }, () => {
expect(resolveEnvNumber(VAR, 72)).toBe(36);
});
await withEnv({ [VAR]: '0.5' }, () => {
expect(resolveEnvNumber(VAR, 72)).toBe(0.5);
});
});
test('garbage, zero, and negatives fall back — zero is an unset accident, not a disable', async () => {
for (const bad of ['garbage', 'NaN', 'Infinity', '-5', '0']) {
_resetEnvNumberWarnedForTests();
await withEnv({ [VAR]: bad }, () => {
expect(resolveEnvNumber(VAR, 72)).toBe(72);
});
}
});
test('warns exactly once per var per process, unit string included', async () => {
await withEnv({ [VAR]: 'garbage' }, () => {
const warns = captureWarns(() => {
resolveEnvNumber(VAR, 72, { unit: 'h' });
resolveEnvNumber(VAR, 72, { unit: 'h' });
resolveEnvNumber(VAR, 72, { unit: 'h' });
});
expect(warns).toHaveLength(1);
expect(warns[0]).toContain(VAR);
expect(warns[0]).toContain('72h');
});
});
});
describe('resolveHoursEnv', () => {
test('is resolveEnvNumber with the hours unit pre-applied', async () => {
await withEnv({ [VAR]: 'bogus' }, () => {
const warns = captureWarns(() => {
expect(resolveHoursEnv(VAR, 24)).toBe(24);
});
expect(warns[0]).toContain('24h');
});
});
});
describe('warnOnceForEnv', () => {
test('shares the SAME memo as resolveEnvNumber — the whole reason this module exists', async () => {
await withEnv({ [VAR]: 'garbage' }, () => {
const warns = captureWarns(() => {
resolveEnvNumber(VAR, 72); // seeds the memo for VAR
warnOnceForEnv(VAR, 'should be suppressed');
warnOnceForEnv(VAR, 'should be suppressed');
});
expect(warns).toHaveLength(1);
expect(warns[0]).not.toBe('should be suppressed');
});
});
test('fires its message once for a var resolveEnvNumber never touched', () => {
const warns = captureWarns(() => {
warnOnceForEnv('GBRAIN_ENV_NUMBER_OTHER_VAR', 'custom disabled-unless-set warning');
warnOnceForEnv('GBRAIN_ENV_NUMBER_OTHER_VAR', 'custom disabled-unless-set warning');
});
expect(warns).toEqual(['custom disabled-unless-set warning']);
});
});
+42
View File
@@ -0,0 +1,42 @@
/**
* Pre-test setup: redirect the sync failure ledger to a per-run scratch
* directory instead of the operator's real `~/.gbrain/sync-failures.jsonl`.
*
* Why this exists: the suite deliberately exercises broken-fixture import and
* sync paths (bad frontmatter, slug mismatches, unparseable YAML). Every one of
* those records a row through `src/core/sync-failure-ledger.ts`, which resolved
* to the gbrain home with no override so fixture failures appended straight
* into the operator's live ledger. That file is not inert: `gbrain doctor` reads
* it and keeps warning until each entry is fixed or acknowledged, and the rows
* carry commit attribution, so a fabricated `srcE / notes/bad.md` row looks
* exactly like a real un-indexed page.
*
* Same shape and rationale as `audit-dir-preload.ts` (#2823), and deliberately a
* DEDICATED var rather than a global `GBRAIN_HOME` redirect: pointing
* GBRAIN_HOME at a scratch dir for the whole suite would also make
* `loadConfig()` return null for every test that reads the real config a much
* wider blast radius than this problem needs, and
* `test/gbrain-home-isolation.test.ts` asserts the unset-fallback behavior
* directly.
*
* Each `scripts/run-unit-shard.sh` shard is its own `bun test` process, so each
* gets its own scratch dir no cross-shard collision, no cleanup needed (the
* OS reaps tmp, the same tradeoff `test/helpers/with-env.ts` documents).
*
* Only sets the var if it isn't already set, so a developer who exports it to
* inspect ledger output after a local run keeps their override. Tests that
* manage their own value per-test via `withEnv` are unaffected.
*
* Imported by `bunfig.toml` via `preload`.
*/
import { mkdtempSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
if (!process.env.GBRAIN_SYNC_FAILURES_DIR) {
const dir = mkdtempSync(join(tmpdir(), 'gbrain-test-syncfail-'));
process.env.GBRAIN_SYNC_FAILURES_DIR = dir;
if (process.env.GBRAIN_DEBUG_PRELOAD === '1') {
console.error(`[sync-failures-preload] GBRAIN_SYNC_FAILURES_DIR=${dir}`);
}
}
+71 -4
View File
@@ -23,10 +23,12 @@ import {
resolvePriority,
newestCommitMs,
lagFromContentMs,
resolveStalenessCeilingSeconds,
_resetPriorityWarningsForTest,
} from '../src/core/source-health.ts';
import { isSourceUnchangedSinceSync } from '../src/core/git-head.ts';
import { loadAllSources } from '../src/core/sources-load.ts';
import { withEnv } from './helpers/with-env.ts';
const HOUR = 3600_000;
@@ -161,12 +163,56 @@ describe('lagFromContentMs (pure remote/column comparator)', () => {
test('null content → wall-clock fallback', () => {
expect(lagFromContentMs(null, now - 100 * HOUR, now)).toBe(360_000); // 100h in s
});
test('content at/before last sync → caught up (0)', () => {
expect(lagFromContentMs(now - 200 * HOUR, now - 100 * HOUR, now)).toBe(0);
test('content at/before last sync, recently checked → caught up (0)', () => {
// The quiet-source contract: a 0-change sync heartbeats last_sync_at
// (sync.ts, v0.42.52.0), so a source that IS being checked stays at 0.
expect(lagFromContentMs(now - 200 * HOUR, now - HOUR, now, 72 * 3600)).toBe(0);
});
test('content at/before last sync, past the ceiling → ramps (was the 71-day bug)', () => {
// This assertion previously expected 0 at 100h and that WAS the bug: a source
// whose clone had vanished reported fresh forever, because the caught-up branch
// discarded the wall-clock number computed one line above it. 100h - 72h ceiling
// = 28h of real lag. Do not "restore" this to 0.
expect(lagFromContentMs(now - 200 * HOUR, now - 100 * HOUR, now, 72 * 3600)).toBe(28 * 3600);
});
test('ceiling boundary: just under → 0, just over → ramped', () => {
const ceiling = 72 * 3600;
expect(lagFromContentMs(now - 200 * HOUR, now - 71.9 * HOUR, now, ceiling)).toBe(0);
// Exactly at the ceiling is still 0; one second past it is one second of lag.
expect(lagFromContentMs(now - 200 * HOUR, now - 72 * HOUR, now, ceiling)).toBe(0);
expect(lagFromContentMs(now - 200 * HOUR, now - (72 * HOUR + 1000), now, ceiling)).toBe(1);
});
test('ramp is monotonic — crosses warn before fail, never steps', () => {
const ceiling = 72 * 3600;
const at = (h: number) => lagFromContentMs(now - 500 * HOUR, now - h * HOUR, now, ceiling)!;
// Strictly increasing past the ceiling, so the 1h/24h/72h consumers escalate in
// order instead of all tripping in the same instant.
expect(at(80)).toBeLessThan(at(100));
expect(at(100)).toBeLessThan(at(200));
// Enters the band from 0 rather than jumping straight to the ceiling value:
// one hour past the ceiling is one hour of lag, not 72.
expect(at(73)).toBe(3600);
});
test('content after last sync → wall-clock since sync', () => {
expect(lagFromContentMs(now - 10 * HOUR, now - 100 * HOUR, now)).toBe(360_000);
});
test('GBRAIN_STALENESS_CEILING_HOURS overrides GBRAIN_SYNC_FRESHNESS_FAIL_HOURS', async () => {
await withEnv(
{ GBRAIN_SYNC_FRESHNESS_FAIL_HOURS: '72', GBRAIN_STALENESS_CEILING_HOURS: '10' },
async () => {
// Override wins: 20h wall-clock against a 10h ceiling → 10h of lag.
expect(resolveStalenessCeilingSeconds()).toBe(10 * 3600);
expect(lagFromContentMs(now - 200 * HOUR, now - 20 * HOUR, now)).toBe(10 * 3600);
},
);
await withEnv(
{ GBRAIN_SYNC_FRESHNESS_FAIL_HOURS: '72', GBRAIN_STALENESS_CEILING_HOURS: undefined },
async () => {
// Without the override it tracks the fail threshold.
expect(resolveStalenessCeilingSeconds()).toBe(72 * 3600);
},
);
});
});
describe('isSourceUnchangedSinceSync (ignore-untracked) — local caught-up contract', () => {
@@ -338,9 +384,12 @@ describe('computeAllSourceMetrics', () => {
expect(metrics.find((m) => m.source_id === 'behind')!.lag_seconds!).toBeGreaterThan(72 * 3600);
});
test('REMOTE (default): reads newest_content_at column, NO git probe → quiet repo lag 0', async () => {
test('REMOTE (default): reads newest_content_at column, NO git probe → quiet + recently synced = lag 0', async () => {
// Criterion 2 (quiet-source non-regression) at the metrics layer: a bogus
// local_path must NOT make a source stale so long as sync is still running
// against it. This is the doctor 70->30 alert-storm case.
const contentIso = new Date(Date.now() - 200 * HOUR).toISOString(); // content predates sync
const syncIso = new Date(Date.now() - 100 * HOUR).toISOString();
const syncIso = new Date(Date.now() - 1 * HOUR).toISOString();
await engine.executeRaw(
`INSERT INTO sources (id, name, local_path, last_commit, last_sync_at, newest_content_at, config)
VALUES ('remote', 'remote', '/nonexistent/not-a-repo', 'x', $1, $2, '{"federated":true}'::jsonb)`,
@@ -352,6 +401,24 @@ describe('computeAllSourceMetrics', () => {
expect(metrics.find((m) => m.source_id === 'remote')!.lag_seconds).toBe(0);
});
test('REMOTE (default): quiet but unsynced past the ceiling → ramps, not 0', async () => {
// Criterion 1 (the 71-day bug) at the metrics layer. This previously
// asserted 0 at 100h, which is exactly how a dead daemon stayed invisible.
const contentIso = new Date(Date.now() - 200 * HOUR).toISOString();
const syncIso = new Date(Date.now() - 100 * HOUR).toISOString();
await engine.executeRaw(
`INSERT INTO sources (id, name, local_path, last_commit, last_sync_at, newest_content_at, config)
VALUES ('abandoned', 'abandoned', '/nonexistent/not-a-repo', 'x', $1, $2, '{"federated":true}'::jsonb)`,
[syncIso, contentIso],
);
const sources = await loadAllSources(engine);
const metrics = await computeAllSourceMetrics(engine, sources);
const lag = metrics.find((m) => m.source_id === 'abandoned')!.lag_seconds!;
// ~100h elapsed minus the 72h ceiling = ~28h of real lag.
expect(lag).toBeGreaterThan(27 * 3600);
expect(lag).toBeLessThan(29 * 3600);
});
test('REMOTE (default): NULL column → wall-clock fallback', async () => {
const syncIso = new Date(Date.now() - 100 * HOUR).toISOString();
await engine.executeRaw(
+22 -5
View File
@@ -202,14 +202,21 @@ describe('buildSyncStatusReport', () => {
const syncIso = new Date(now - 100 * 60 * 60 * 1000).toISOString(); // synced 100h ago
const sources = [
{ id: 'quiet', name: 'quiet', local_path: '/tmp/quiet', config: { syncEnabled: true } },
{ id: 'quiet_recent', name: 'quiet_recent', local_path: '/tmp/quiet_recent', config: { syncEnabled: true } },
{ id: 'behind', name: 'behind', local_path: '/tmp/behind', config: { syncEnabled: true } },
{ id: 'nocol', name: 'nocol', local_path: '/tmp/nocol', config: { syncEnabled: true } },
];
const engine = makeEngine({
sourceRows: [
// Newest commit 200h ago, synced 100h ago → caught up → lag 0 → fresh.
// Newest commit 200h ago, synced 100h ago → caught up, but nobody has
// synced in 100h, so the staleness ceiling ramps it to ~28h (100 - 72).
// It lands in the MIDDLE band, not 'severe' — that ordering is the point.
{ id: 'quiet', last_commit: 'a'.repeat(40), last_sync_at: syncIso,
newest_content_at: new Date(now - 200 * 60 * 60 * 1000).toISOString() },
// Same content shape but synced 1h ago → genuinely being checked → fresh.
{ id: 'quiet_recent', last_commit: 'd'.repeat(40),
last_sync_at: new Date(now - 1 * 60 * 60 * 1000).toISOString(),
newest_content_at: new Date(now - 200 * 60 * 60 * 1000).toISOString() },
// Newest commit 10h ago, synced 100h ago → behind → wall-clock → severe.
{ id: 'behind', last_commit: 'b'.repeat(40), last_sync_at: syncIso,
newest_content_at: new Date(now - 10 * 60 * 60 * 1000).toISOString() },
@@ -218,6 +225,7 @@ describe('buildSyncStatusReport', () => {
],
countRows: [
{ source_id: 'quiet', pages: 10, chunks_total: 20, chunks_unembedded: 0 },
{ source_id: 'quiet_recent', pages: 10, chunks_total: 20, chunks_unembedded: 0 },
{ source_id: 'behind', pages: 10, chunks_total: 20, chunks_unembedded: 0 },
{ source_id: 'nocol', pages: 10, chunks_total: 20, chunks_unembedded: 0 },
],
@@ -225,10 +233,19 @@ describe('buildSyncStatusReport', () => {
const report = await buildSyncStatusReport(engine, sources);
const byId = new Map(report.sources.map((s) => [s.source_id, s]));
// Legacy wall-clock would have called 'quiet' severe (100h). Content-relative
// correctly reports caught-up.
expect(byId.get('quiet')!.staleness_hours).toBe(0);
expect(byId.get('quiet')!.staleness_class).toBe('fresh');
// Three-way distinction, which is the whole point of the ramp:
// quiet_recent → 0 (caught up AND being checked)
// quiet → ~28h (caught up but abandoned 100h ago — used to report 0
// forever, which is how a dead daemon stayed invisible)
// behind → 100h (real new content unsynced)
// Naive wall-clock would flatten the first two into 'severe' alongside the third.
expect(byId.get('quiet_recent')!.staleness_hours).toBe(0);
expect(byId.get('quiet_recent')!.staleness_class).toBe('fresh');
expect(byId.get('quiet')!.staleness_hours).toBeGreaterThan(27);
expect(byId.get('quiet')!.staleness_hours).toBeLessThan(29);
// Middle band, NOT severe: the ramp crosses warn before fail so consumers at
// 24h and 72h escalate in order rather than tripping in the same instant.
expect(byId.get('quiet')!.staleness_class).toBe('stale');
expect(byId.get('behind')!.staleness_class).toBe('severe');
expect(byId.get('nocol')!.staleness_hours).toBeGreaterThan(72);
expect(byId.get('nocol')!.staleness_class).toBe('severe');
+26
View File
@@ -33,6 +33,32 @@ describe('buildSyncManifest', () => {
expect(manifest.renamed).toEqual([{ from: 'people/old.md', to: 'people/new.md' }]);
});
test('T (typechange) counts as modified — was silently dropped', () => {
// file <-> symlink. Reachable with the flags gbrain actually passes
// (`--name-status -M`). Dropping it meant the change never reached the
// index until some later commit happened to touch the same path.
const manifest = buildSyncManifest(`T\tpeople/now-a-symlink.md`);
expect(manifest.modified).toEqual(['people/now-a-symlink.md']);
expect(manifest.added).toEqual([]);
expect(manifest.deleted).toEqual([]);
});
test('U (unmerged) degrades to modified rather than vanishing', () => {
// Only reachable in a conflicted worktree, which sync does not run against.
// Defensive: re-import beats silently skipping.
expect(buildSyncManifest(`U\tpeople/conflicted.md`).modified).toEqual(['people/conflicted.md']);
});
test('C (copy) imports the destination — unreachable today, defensive', () => {
// Requires -C/--find-copies, which gbrain does not pass. If the flags ever
// change, the copy destination is a NEW path that must be imported, and the
// source is untouched — so it is an add, not a rename.
const manifest = buildSyncManifest(`C100\tpeople/src.md\tpeople/copy.md`);
expect(manifest.added).toEqual(['people/copy.md']);
expect(manifest.renamed).toEqual([]);
expect(manifest.deleted).toEqual([]);
});
test('handles empty diff', () => {
const manifest = buildSyncManifest('');
expect(manifest.added).toEqual([]);