mirror of
https://github.com/garrytan/gbrain.git
synced 2026-08-14 17:02:19 +00:00
Compare commits
15
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e22b8fb555 | ||
|
|
2a9feb859f | ||
|
|
15b9316dbf | ||
|
|
f739de5521 | ||
|
|
02d585c0a4 | ||
|
|
e573fa6988 | ||
|
|
ff6320e552 | ||
|
|
36c750bbec | ||
|
|
7f2c81f929 | ||
|
|
1353366b5f | ||
|
|
93ae40dd3a | ||
|
|
8fcd2737bf | ||
|
|
b23f24f91b | ||
|
|
10d96545a4 | ||
|
|
6b2f3bc321 |
@@ -17,9 +17,4 @@ eval/data/world-v1/world.html
|
||||
|
||||
# BrainBench amara-life-v1 Opus cache (regenerate via eval:generate-amara-life)
|
||||
eval/data/amara-life-v1/_cache/
|
||||
|
||||
# claw-test E2E build cache (shim + scratch outputs)
|
||||
test/.cache/
|
||||
|
||||
.claude/
|
||||
export/
|
||||
|
||||
-362
@@ -2,368 +2,6 @@
|
||||
|
||||
All notable changes to GBrain will be documented in this file.
|
||||
|
||||
## [0.23.0] - 2026-04-26
|
||||
|
||||
**`gbrain dream` now actually dreams. Conversation transcripts become reflections, originals, and 25-year patterns ... overnight.**
|
||||
|
||||
The maintenance cycle gains two new phases. Synthesize reads transcripts (OpenClaw session corpus, meeting transcripts, ad-hoc files) and writes brain-native pages: reflections to `wiki/personal/reflections/`, originals to `wiki/originals/ideas/`, timeline entries on existing people pages. Patterns runs after `extract` and surfaces recurring themes ... when ≥3 reflections mention the same motif, a pattern page is written to `wiki/personal/patterns/<theme>` citing every reflection that constitutes its evidence. The phase order is now `lint → backlinks → sync → synthesize → extract → patterns → embed → orphans` ... eight phases, one cron-friendly command.
|
||||
|
||||
The motivating story: on 2026-04-25 you read your Stanford-era email archive (4,963 emails, 1999-2001) and the agent had to hand-write the reflection page connecting patterns from age 19 to age 45. The 19-year-old who saved his ICQ logs is the user the system should match. The dream cycle's job is to make the brain a self-enriching memory instead of a manually-curated database.
|
||||
|
||||
### The numbers that matter
|
||||
|
||||
Real production deployment, default config (Sonnet 4.6 synthesis, Haiku 4.5 verdict, 12-hour cooldown). Reproduce with `gbrain dream --phase synthesize --input <fixture>` against any transcript >2000 chars.
|
||||
|
||||
| Metric | Before (v0.20.4) | After (v0.23.0) | Δ |
|
||||
|---|---|---|---|
|
||||
| Cycle phases | 6 | 8 | +33% |
|
||||
| Sources of brain enrichment | 4 (manual, signal, ingest, extract) | 5 (+ overnight synth) | +1 lane |
|
||||
| Cost / day under autopilot | $0 | ~$1-2 | bounded by cooldown |
|
||||
| Reflections after 30 days | 0 (manual only) | 10-15 (auto) | "the brain dreams" |
|
||||
|
||||
The lane that matters: a daily conversation between you and the agent now lands in long-term memory automatically. No manual write-up. Pattern recognition across reflections is one more sonnet call, not a new subsystem.
|
||||
|
||||
### What this means for you
|
||||
|
||||
Configure `dream.synthesize.session_corpus_dir` once, set `dream.synthesize.enabled true`, and `gbrain dream` (or your existing autopilot install) consolidates yesterday's conversations every overnight pass. Edited transcripts produce new slugs (content-hash suffix) ... never silently overwrite. The synthesize subagent is bounded to an explicit allow-list sourced from `_brain-filing-rules.json`, so even a poisoned transcript can't write to `wiki/finance/secret.md`. `--dry-run` runs the cheap Haiku verdict (cached in `dream_verdicts`) so you can preview without spending real Sonnet tokens.
|
||||
|
||||
## To take advantage of v0.23.0
|
||||
|
||||
`gbrain upgrade` should do this automatically. If it didn't, or if `gbrain doctor` warns about a partial migration:
|
||||
|
||||
1. **Run the orchestrator manually:**
|
||||
```bash
|
||||
gbrain apply-migrations --yes
|
||||
```
|
||||
2. **Configure the synthesize phase if you want overnight conversation synthesis:**
|
||||
```bash
|
||||
gbrain config set dream.synthesize.session_corpus_dir /path/to/transcripts
|
||||
gbrain config set dream.synthesize.enabled true
|
||||
gbrain dream --phase synthesize --dry-run --json
|
||||
```
|
||||
Existing autopilot users see no behavior change until this step ... synthesize is opt-in.
|
||||
3. **Verify the outcome:**
|
||||
```bash
|
||||
gbrain doctor # schema_version should match latest
|
||||
gbrain dream --help # shows the 8-phase pipeline
|
||||
gbrain dream --phase synthesize --dry-run # zero Sonnet calls; cheap Haiku verdict only
|
||||
```
|
||||
4. **If any step fails or the numbers look wrong,** please file an issue at https://github.com/garrytan/gbrain/issues with:
|
||||
- output of `gbrain doctor`
|
||||
- contents of `~/.gbrain/upgrade-errors.jsonl` if it exists
|
||||
- which step broke
|
||||
|
||||
### Itemized changes
|
||||
|
||||
#### Dream cycle: synthesize phase (`src/core/cycle/synthesize.ts`)
|
||||
|
||||
- Reads transcripts from `dream.synthesize.session_corpus_dir` (or `--input <file>` ad-hoc).
|
||||
- Cheap Haiku verdict per transcript filters routine ops sessions; verdicts cached in the new `dream_verdicts` table keyed by `(file_path, content_hash)` so backfill re-runs skip already-judged transcripts at zero cost.
|
||||
- Fan-out: one Sonnet subagent per worth-processing transcript, dispatched with `allowed_slug_prefixes` (read once from `skills/_brain-filing-rules.json`'s `dream_synthesize_paths.globs`).
|
||||
- Idempotency key `dream:synth:<file_path>:<content_hash>` ... same content twice is a queue no-op.
|
||||
- Slug shape: `wiki/personal/reflections/YYYY-MM-DD-<topic>-<hash[:6]>` and `wiki/originals/ideas/YYYY-MM-DD-<idea>-<hash[:6]>`. Edited transcripts produce new slugs alongside the old; `git log` shows both.
|
||||
- Provenance via `subagent_tool_executions` (the orchestrator queries each child's put_page input, NOT `pages.updated_at` ... that would pick up unrelated writes).
|
||||
- Orchestrator dual-write: subagent only calls put_page (writes to DB); after children resolve, the phase reverse-renders each new page from DB to disk via `serializeMarkdown`. Subagent never gets fs-write access.
|
||||
- Cooldown via `dream.synthesize.last_completion_ts` config key, written ONLY on success. Default 12-hour cooldown caps spend at ~$1-2/day under autopilot. Explicit `--input` / `--date` / `--from` / `--to` invocations bypass cooldown.
|
||||
|
||||
#### Dream cycle: patterns phase (`src/core/cycle/patterns.ts`)
|
||||
|
||||
- Runs AFTER `extract` (codex finding #7) so the graph state is fresh ... subagent put_page sets `ctx.remote=true` and skips auto-link/timeline by default; extract is the canonical materialization step.
|
||||
- Single Sonnet subagent gathers reflections within `dream.patterns.lookback_days` (default 30) and surfaces themes that recur in ≥`dream.patterns.min_evidence` (default 3) distinct reflections.
|
||||
- Pattern slug: `wiki/personal/patterns/<theme>` (no date — patterns aggregate across dates). Existing pattern pages are updated in place via the same allow-listed put_page path.
|
||||
- Same provenance model as synthesize.
|
||||
|
||||
#### Trust boundary: `allowed_slug_prefixes`
|
||||
|
||||
- New `OperationContext.allowedSlugPrefixes?: string[]` field. When set on a subagent's put_page call, the slug must match one of the listed prefix globs (e.g. `wiki/personal/reflections/*`) or the call is rejected with `permission_denied`.
|
||||
- When unset, the legacy `wiki/agents/<subagentId>/...` namespace check applies unchanged ... v0.15 anti-prompt-injection guarantee preserved (regression-guarded by `test/operations-allow-list.test.ts`).
|
||||
- Trust comes from PROTECTED_JOB_NAMES (MCP can't submit `subagent` jobs at all), NOT from `ctx.remote`. The `remote=true` flag flows through every subagent tool call for auto-link safety; using it as the trust signal would null the allow-list for its intended consumer (codex finding #1, caught and corrected pre-merge).
|
||||
- Auto-link is re-enabled for trusted-workspace writes so the cycle's extract phase doesn't have to recompute synth-output edges.
|
||||
- Allow-list lives in ONE place: `skills/_brain-filing-rules.json`'s `dream_synthesize_paths.globs`. Both the subagent runtime and the maintain skill read from there.
|
||||
|
||||
#### Cycle scaffolding (`src/core/cycle.ts`)
|
||||
|
||||
- `ALL_PHASES` extends to 8 entries; `gbrain dream --phase synthesize` and `--phase patterns` work like any other phase.
|
||||
- New `yieldDuringPhase` hook in `CycleOpts`. Generic in-phase keepalive that long-running phases call every ~5 min while idle to renew the cycle-lock TTL and the Minions worker job lock. Mirrors `yieldBetweenPhases` shape.
|
||||
- `CycleReport.totals` grew additively (schema_version stays "1"): new fields `transcripts_processed`, `synth_pages_written`, `patterns_written`. Existing consumers see no breaking change.
|
||||
- `synthesize` and `patterns` both fall under `NEEDS_LOCK_PHASES`; read-only invocations like `--phase orphans` continue to skip the lock.
|
||||
|
||||
#### CLI extensions (`src/commands/dream.ts`)
|
||||
|
||||
- New flags: `--input <file>` (ad-hoc transcript synthesis; implies `--phase synthesize`), `--date YYYY-MM-DD` (single-day), `--from YYYY-MM-DD --to YYYY-MM-DD` (backfill range).
|
||||
- `--dry-run` semantics documented explicitly (codex finding #8): runs the cheap Haiku significance verdict (caches it for free) but skips the Sonnet synthesis pass. NOT zero LLM calls.
|
||||
- Conflict detection: `--input` plus `--date` / `--from` / `--to` exits 2 with a clear error.
|
||||
- Help text now reflects the 8-phase pipeline.
|
||||
|
||||
#### Schema migration v25 (`src/core/migrate.ts`, `src/schema.sql`)
|
||||
|
||||
- Creates `dream_verdicts (file_path TEXT, content_hash TEXT, worth_processing BOOL, reasons JSONB, judged_at TIMESTAMPTZ, PRIMARY KEY(file_path, content_hash))`. Distinct from `raw_data` (which is page-scoped) ... transcripts being judged aren't pages.
|
||||
- RLS-enabled when running as a BYPASSRLS role (matches the existing v24 pattern).
|
||||
- New engine methods `getDreamVerdict` / `putDreamVerdict` on both Postgres and PGLite. ON CONFLICT upserts; idempotent across re-runs.
|
||||
|
||||
#### Tests
|
||||
|
||||
- `test/operations-allow-list.test.ts` (NEW, IRON RULE security regression guard) ... 11 cases covering ALLOW path, REJECT path, glob match (recursive depth), legacy namespace check when allow-list unset, FAIL-CLOSED behavior when `viaSubagent=true` but `subagentId` is missing.
|
||||
- `test/cycle-synthesize.test.ts` (NEW) ... 20 cases covering `compileExcludePatterns` word-boundary heuristic, transcript discovery (date filters, multi-source merge, exclude regex, `min_chars`), content-hash stability across edits, `readSingleTranscript` ad-hoc path.
|
||||
- `test/cycle-patterns.test.ts` (NEW) ... 12 structural cases covering subagent dispatch wiring, allow-list flow from filing-rules JSON, scope filter (`slug LIKE 'wiki/personal/reflections/%'`), the codex #2 fix (provenance via `subagent_tool_executions`).
|
||||
- `test/dream-cli-flags.test.ts` (NEW) ... 9 cases covering `--input` / `--date` / `--from` / `--to` parsing, ISO date validation, conflict detection, dry-run semantics documentation.
|
||||
- `test/e2e/dream-allow-list-pglite.test.ts` (NEW) ... 6 cases on PGLite covering the full subagent → put_page allow-list path: in-allow-list slug writes, out-of-allow-list slug rejected, legacy namespace fallback when allow-list unset, `subagent_tool_executions` schema for provenance queries.
|
||||
- `test/e2e/dream-synthesize-pglite.test.ts` (NEW) ... 8 cases on PGLite covering disabled/not_configured paths, empty corpus, no-API-key skip path, dry-run semantics, cooldown active/bypass, `dream_verdicts` cache hit.
|
||||
|
||||
#### Documentation
|
||||
|
||||
- `skills/maintain/SKILL.md` ... new "Dream cycle: synthesize + patterns" section with the quality bar, trust boundary, idempotency model, cooldown semantics, and invocation patterns. Triggers updated to route "process today's session", "synthesize my conversations", and "what patterns did you see" to maintain.
|
||||
- `skills/_brain-filing-rules.md` ... new "Dream-cycle synthesize/patterns directories" section documenting the allow-listed paths, slug discipline, and the iron law for synthesis output.
|
||||
- `skills/_brain-filing-rules.json` ... new `dream_synthesize_paths.globs` array (single source of truth).
|
||||
- `skills/RESOLVER.md` ... new dream-cycle row under brain operations.
|
||||
- `skills/migrations/v0.21.0.md` (NEW) ... migration narrative covering schema migration v25 + the optional opt-in for synthesize + tunables.
|
||||
- `CLAUDE.md` ... architecture section reflects 8-phase cycle + new files (`src/core/cycle/{synthesize,patterns,transcript-discovery}.ts`).
|
||||
|
||||
#### Codex review-driven corrections
|
||||
|
||||
Eight findings from the cross-model review caught real implementation traps before merge. All 8 resolutions integrated:
|
||||
|
||||
1. Trust signal correction (drop `remote=null` defense, rely on PROTECTED_JOB_NAMES gating).
|
||||
2. Provenance via child `subagent_tool_executions` (not `pages.updated_at`).
|
||||
3. New `dream_verdicts` mini-table (raw_data is page-scoped and won't fit).
|
||||
4. Summary slug regex-compatible: `dream-cycle-summaries/YYYY-MM-DD` (no underscore, no `.md`).
|
||||
5. Auto-commit/push deferred to v1.1 (dirty-worktree handling, auth failure, non-FF push need their own design).
|
||||
6. Lossy-serialization acknowledged: the orchestrator does fresh-render from DB, not byte-identical round-trip.
|
||||
7. Phase ordering: patterns runs AFTER extract so the graph is fresh.
|
||||
8. `--dry-run` semantics documented: runs Haiku, skips Sonnet (NOT zero LLM calls).
|
||||
|
||||
#### Deferred to v1.1
|
||||
|
||||
- Auto git commit + push from the synthesize/patterns phases. v1 writes files locally; either commit yourself or let `gbrain autopilot` handle it.
|
||||
- Daily token budget cap. Cooldown is the v1 spend bound.
|
||||
- Cross-modal pattern review (currently reflections-only).
|
||||
|
||||
|
||||
## [0.22.16] - 2026-04-29
|
||||
|
||||
**End-to-end claw-test friction harness — every release now gets a fresh-install dry-run.**
|
||||
**`gbrain claw-test` spins up a hermetic tempdir, walks the canonical first-day flow, and surfaces friction the way a real new user would hit it.**
|
||||
|
||||
Before this release, every gbrain release shipped on faith: docs said "the agent runs `gbrain init`, then `gbrain import`, then `gbrain query`," and we'd find out at user-feedback time which step actually broke. Issue #239/#243/#266/#357/#366/#374/#375/#378/#395/#396 — ten upgrade-wedge incidents in two years — all came from this gap. There was no harness that exercised the user's-eye experience: spin up a fresh tempdir, install gbrain, watch what breaks.
|
||||
|
||||
Now there is. `gbrain claw-test --scenario fresh-install` in scripted mode is a CI gate (~30s, no API keys). `gbrain claw-test --live --agent openclaw` spawns a real openclaw subprocess, hands it `BRIEF.md`, captures every byte of its stdin/stdout/stderr to `transcript.jsonl`, and lets the agent log friction whenever something is confusing or wrong. End-of-run renders a markdown report grouped by severity and phase, with `<HOME>` redaction so it pastes safely into PRs.
|
||||
|
||||
The friction signal comes from a new `gbrain friction {log,render,list,summary}` CLI. Schema is a flat extension of `StructuredAgentError`. Run-id resolves from `--run-id` > `$GBRAIN_FRICTION_RUN_ID` > `standalone.jsonl`, so the same CLI works inside a harness session, manually during normal use, or from a scripted test. Append-only JSONL; readers tolerate malformed lines.
|
||||
|
||||
**$GBRAIN_HOME is finally honored everywhere it should be.** `configDir()` in `src/core/config.ts` always supported the parent-dir override, but ~12 consumers built paths from `os.homedir()` directly and bypassed it. Critically, `loadConfig`/`saveConfig` themselves used a private helper that ignored the env. Migrated every write site to a new `gbrainPath()` helper: fail-improve, validator-lint, cycle lock, audit handlers, sync-failures, integrity logs, integrations heartbeat, init pglite path, migrate-engine manifest, import checkpoint, migration rollbacks. Read-side host-detection (`~/.claude` / `~/.openclaw` probes for mod fingerprinting) intentionally stays as-is; v1.1 will add a separate `$GBRAIN_HOST_HOME`.
|
||||
|
||||
### Itemized changes
|
||||
|
||||
#### Added
|
||||
|
||||
- `gbrain claw-test --scenario {fresh-install|upgrade-from-v0.18}` — scripted-mode CI gate that runs the canonical first-day flow against a fresh tempdir. Asserts every expected `--progress-json` phase fired and doctor's `status === 'ok'`. ~30s, no API keys.
|
||||
- `gbrain claw-test --live --agent openclaw` — friction-discovery mode. Spawns real openclaw, hands it `BRIEF.md`, captures stdin/stdout/stderr to `<run>/transcript.jsonl`, lets the agent log friction. ~5–10 min and ~$1–2 in tokens.
|
||||
- `gbrain claw-test --list-agents` — reports which agent runners are registered + their detection state.
|
||||
- `gbrain friction log --severity {confused|error|blocker|nit} --phase <name> --message <text> [--hint ...] [--kind {friction|delight}] [--run-id ...]` — append a friction or delight entry.
|
||||
- `gbrain friction render --run-id <id> [--json] [--transcripts] [--no-redact]` — markdown report grouped by severity + phase; `--redact` defaults on for md output.
|
||||
- `gbrain friction list [--json]` — recent run-ids with friction/delight counts; interrupted runs marked `(interrupted)`.
|
||||
- `gbrain friction summary --run-id <id> [--json]` — two-column friction + delight summary.
|
||||
- `skills/_friction-protocol.md` — cross-cutting convention skill telling agents when to call `gbrain friction log`. Routes from any skill the claw-test exercises.
|
||||
- `gbrainPath(...segments)` helper in `src/core/config.ts` — single sugar for resolving paths under the active `$GBRAIN_HOME`. `$GBRAIN_HOME` is now validated (must be absolute, no `..` segments).
|
||||
- Two scenario fixtures in `test/fixtures/claw-test-scenarios/`: `fresh-install` (canonical 5-min flow) and `upgrade-from-v0.18` (scaffolded; real v0.18 SQL dump documented as a v1.1 follow-up).
|
||||
- New `src/core/claw-test/` module with `agent-runner.ts` (interface + registry), `transcript-capture.ts` (async-drain capture so 256KB+ bursts don't stall the child), `progress-tail.ts`, `scenarios.ts`, and `seed-pglite.ts` (~50 LOC PGLite SQL replay primitive).
|
||||
|
||||
#### Changed
|
||||
|
||||
- Every `~/.gbrain/...` write site now resolves through `gbrainPath()` instead of building paths from `os.homedir()`. Affected: `src/core/{fail-improve,output/post-write,cycle,sync}.ts`, `src/core/minions/{handlers/shell-audit,backpressure-audit}.ts`, `src/commands/{integrity,integrations,init,migrate-engine,import,migrations/v0_13_1,migrations/v0_14_0}.ts`. Tests that previously used the `process.env.HOME = tmpdir` workaround now use `process.env.GBRAIN_HOME` directly.
|
||||
- `loadConfig`/`saveConfig` honor `$GBRAIN_HOME`. Previously, the public `configDir()` honored it but the internal `getConfigDir()` did not — so the config file itself silently leaked into the developer's real `~/.gbrain` regardless of the env override.
|
||||
|
||||
#### Tests
|
||||
|
||||
- 113 new unit tests covering: writer atomicity (concurrent appends), renderer redaction, agent registry resolution + selection precedence, multi-byte UTF-8 chunk-boundary safety, PIPE buffer drain under 256KB+ bursts, scenario load + validation, progress event parsing, SQL splitter (single-quote + line-comment handling), and full claw-test E2E (`test/e2e/claw-test.test.ts` builds a tiny `bun run src/cli.ts` shim and runs --scenario fresh-install end-to-end + a deliberate-break test that proves the friction signal fires).
|
||||
- `test/gbrain-home-isolation.test.ts` is the regression gate: spawns `gbrain init --pglite` and `gbrain import --no-embed` with `GBRAIN_HOME=<tmp>`, asserts no writes outside `<tmp>/.gbrain` (covers `import.ts:54`, `sync.ts:317`, `upgrade.ts:117`, audit dirs).
|
||||
|
||||
## [0.22.15] - 2026-04-29
|
||||
|
||||
## **Throw bare markdown into your brain and it becomes properly typed knowledge. No YAML ceremony.**
|
||||
|
||||
A real 81K-page brain has 9,655 files with no frontmatter. They imported fine, but every one of them landed in the DB as `type: concept`, `title: <slugified-filename>`, no date, no source, no tags. Search ranking suffered. Type-filtered queries missed them. Entity resolution fell over.
|
||||
|
||||
This release adds path-aware frontmatter inference. `gbrain sync` now synthesizes type, date, source, and tags from the filesystem path and first heading the moment a bare-frontmatter file imports. No LLM call, fully deterministic, file on disk untouched. An Apple Note at `Apple Notes/2010-04-13 founders mtg.md` lands as `type: apple-note, title: founders mtg, date: 2010-04-13, source: apple-notes` instead of `type: concept, title: 2010 04 13 Founders Mtg`.
|
||||
|
||||
If you want the inference written back to git, the new `gbrain frontmatter generate <path> --fix` walks a brain dir, infers frontmatter for every file that lacks it, and writes back with `.bak` safety backups. Dry-run by default.
|
||||
|
||||
### The 9,655 numbers that matter
|
||||
|
||||
Measured against my actual brain (gbrain v0.22.8 + the new inference path).
|
||||
|
||||
| Behavior | Before v0.22.15 | After v0.22.15 |
|
||||
|---|---|---|
|
||||
| Files importing as `type: concept` (no frontmatter) | 9,655 | 0 |
|
||||
| Apple Notes typed correctly (`apple-note`) | 0 | 5,861 |
|
||||
| Calendar indexes typed correctly (`calendar-index`) | 0 | 3,201 |
|
||||
| Therapy sessions typed + dated | 0 | 60 |
|
||||
| Essay drafts typed + dated | 0 | 33 |
|
||||
| LLM cost for the full reclassification | n/a | $0 |
|
||||
|
||||
The agent doing type-filtered queries on your brain (`type: person`, `type: meeting`, `type: essay`) now actually finds those pages instead of treating everything as `concept`.
|
||||
|
||||
### What this means for you
|
||||
|
||||
If you've been resisting frontmatter ceremony — same. Throw bare markdown into your brain and inference handles it. The rules table in `src/core/frontmatter-inference.ts` covers the obvious directories (`people/`, `companies/`, `daily/calendar/`, `writing/`, `meetings/`, `personal/`, etc.) plus a generic catch-all. Adding a new convention is one line in `DIRECTORY_RULES`.
|
||||
|
||||
## To take advantage of v0.22.15
|
||||
|
||||
`gbrain upgrade` should do this automatically. Then:
|
||||
|
||||
1. **Run a dry-run preview:**
|
||||
```bash
|
||||
gbrain frontmatter generate ~/brain
|
||||
```
|
||||
You'll see how many files would get inferred frontmatter and the breakdown by type.
|
||||
2. **Optionally write back to git:**
|
||||
```bash
|
||||
gbrain frontmatter generate ~/brain --fix
|
||||
```
|
||||
Each modified file gets a `.bak` backup before rewrite.
|
||||
3. **Re-sync to pick up the new metadata:**
|
||||
```bash
|
||||
gbrain sync ~/brain
|
||||
```
|
||||
Inferred frontmatter is folded into `content_hash`, so previously-bare files re-import once with proper types and re-embed. Subsequent syncs are idempotent.
|
||||
4. **If anything looks off,** please file an issue: https://github.com/garrytan/gbrain/issues with the path of the misclassified file and the rule that matched.
|
||||
|
||||
### Itemized changes
|
||||
|
||||
#### Features
|
||||
- `src/core/frontmatter-inference.ts` (new module) — Path-aware frontmatter synthesis. `DIRECTORY_RULES` table maps path prefixes to type/date/title/source/tags. First-match-wins. Date extraction from filenames (`YYYY-MM-DD` prefix or anywhere). Title extraction with date-prefix stripping and first-`#`-heading fallback (20-line window). YAML-safe serialization with quoting for special characters.
|
||||
- `src/core/import-file.ts` — `importFromFile()` runs inference inline before `parseMarkdown()` when `opts.inferFrontmatter !== false` (default on). The synthesized frontmatter folds into the in-memory content for parsing, chunking, embedding, and content-hash computation. The file on disk is not modified.
|
||||
- `src/commands/frontmatter.ts` — New `gbrain frontmatter generate <path> [--fix] [--dry-run] [--json]` subcommand. Walks a directory (skips `.git`, `node_modules`, `.obsidian`, symlinks), runs inference on every `.md` file without frontmatter, optionally writes back with `.bak` backups. Auto-detects brain root by walking up for `.git`. Shows per-type breakdown and first-10 examples.
|
||||
|
||||
#### Fixes
|
||||
- `src/commands/frontmatter.ts:344` — `runGenerate` dynamic path import now includes `basename`. Single-file invocation (`gbrain frontmatter generate <file>`) previously crashed with `ReferenceError: basename is not defined` on the relative-path-empty fallback at line 437.
|
||||
|
||||
#### Tests
|
||||
- `test/frontmatter-inference.test.ts` (new, 35 cases) — date extraction (5), title extraction from filenames (5) and headings (4 incl. 20-line boundary), inference for every directory rule (13 incl. Apple Notes subfolder tagging), serialization with YAML-safe quoting (4), `applyInference` integration (2), rule ordering and catch-all coverage (2).
|
||||
|
||||
## [0.22.14] - 2026-04-29
|
||||
|
||||
**Bare `gbrain jobs work` now self-monitors and fail-stops cleanly when its database dies or the queue stalls.**
|
||||
**The wedged-worker class of bug — process alive, jobs piling up, your `pgrep` check happily green — is gone.**
|
||||
|
||||
A production brain (54K pages, Supabase Postgres, 3-concurrency worker under a cron-based PM)
|
||||
hit it last week: worker process state=Sl at 13:15 UTC, stopped claiming jobs, 21 jobs stacked
|
||||
in `waiting` over two hours, 5 autopilot-cycles dead-lettered at the 600s timeout, then 150
|
||||
zombie processes accumulated over the container's 31-day life. The PM's `pgrep` saw a live
|
||||
PID and reported green the entire time.
|
||||
|
||||
Pre-v0.22.14, bare `gbrain jobs work` had **zero** health monitoring. The supervisor (`gbrain
|
||||
jobs supervisor`) had the right protections — DB liveness probes, stall detection, RSS
|
||||
watchdog, reconnect on transient PgBouncer blips — but the supervisor wraps `jobs work` as a
|
||||
child, and many production deployments run bare `jobs work` directly under systemd, Docker,
|
||||
launchd, cron watchdog, or supervisord. That mode got nothing.
|
||||
|
||||
This release moves health monitoring into the bare worker itself, gated by `GBRAIN_SUPERVISED=1`
|
||||
so it doesn't double up under the supervisor. When the worker detects it's wedged, it emits an
|
||||
`'unhealthy'` event with a structured reason, and the CLI calls `process.exit(1)` so the external
|
||||
PM restarts it cleanly. **This is fail-stop:** the worker exits and stays dead until your PM
|
||||
brings it back. If you run bare `jobs work` without a restart loop, you need one now.
|
||||
|
||||
### The numbers that matter
|
||||
|
||||
Detection signatures the new health check catches, measured against the production incident
|
||||
above (and the 30-day deployment running under the band-aid bash watchdog Garry deployed before
|
||||
this fix):
|
||||
|
||||
| Failure mode | Before v0.22.14 | After v0.22.14 |
|
||||
|---|---|---|
|
||||
| DB connection death (Supabase/PgBouncer drop) | undetected; worker idles forever | 3 consecutive `SELECT 1` failures (≤3min) → `'unhealthy'`+exit |
|
||||
| Hung DB probe (network partition) | timer wedged forever, monitoring silently disabled | 10s probe timeout per tick → counted as failure → exit at strike 3 |
|
||||
| Worker stall (event loop alive, claim returns null) | undetected; jobs pile up in `waiting` | 5min warn, 10min `'unhealthy'`+exit (measured from last completion) |
|
||||
| Memory leak (RSS climbing past 2GB) | undetected on bare workers | watchdog default 2048 MB triggers `gracefulShutdown('watchdog')` |
|
||||
| Worker stalled but waiting jobs are unhandled type | ❌ false-positive exit (restart loop) | filter by registered handler names, no exit |
|
||||
|
||||
Operationally: from the band-aid bash watchdog Garry deployed before this fix, fresh worker
|
||||
restart cleared 21 waiting → 0 in 2 minutes, then ran stable for 30+ min with 130 MB RSS,
|
||||
autopilot-cycles completing in 0.2–0.6s instead of timing out at 600s.
|
||||
|
||||
### What this means for operators
|
||||
|
||||
Add a restart policy to your bare-worker invocation BEFORE upgrading. The new behavior is
|
||||
fail-stop, not self-healing — without a restart loop, your worker will exit on the first DB
|
||||
blip and stay dead. systemd `Restart=always`, Docker `restart: always`, launchd `KeepAlive`,
|
||||
cron watchdog, supervisord `autorestart=true`. The migration walks every PM. If you're using
|
||||
`gbrain jobs supervisor`, you're already protected — the supervisor handles spawn-on-crash
|
||||
itself.
|
||||
|
||||
The default `--max-rss` for bare workers also bumped from 0 (off) to 2048 MB. If you ran bare
|
||||
workers with intentionally large embed/import jobs, raise the limit (`--max-rss 4096`) or opt
|
||||
out (`--max-rss 0`). The migration includes per-PM unit-file edits.
|
||||
|
||||
## To take advantage of v0.22.14
|
||||
|
||||
`gbrain upgrade` should do this automatically. If it didn't, or if `gbrain doctor` warns about
|
||||
a bare worker exiting with watchdog signatures:
|
||||
|
||||
1. **Confirm your bare-worker invocations have a restart policy:**
|
||||
```bash
|
||||
# systemd
|
||||
grep -E '^Restart=' ~/.config/systemd/user/gbrain-worker.service /etc/systemd/system/gbrain-worker.service 2>/dev/null
|
||||
# crontab
|
||||
crontab -l | grep "gbrain jobs work"
|
||||
# launchctl
|
||||
plutil -p ~/Library/LaunchAgents/com.user.gbrain-worker.plist | grep -A1 KeepAlive
|
||||
```
|
||||
2. **Decide on RSS posture:**
|
||||
- Default 2048 MB matches supervisor behavior. Most bare workers fit.
|
||||
- Embed/import jobs > 2GB? Pass `--max-rss 4096` (or higher).
|
||||
- Intentionally unbounded? Pass `--max-rss 0`.
|
||||
3. **Walk the migration:** `skills/migrations/v0.22.14.md` has the full per-PM table and a
|
||||
verification block.
|
||||
4. **Verify:**
|
||||
```bash
|
||||
gbrain jobs stats
|
||||
gbrain doctor --json | jq '.checks[] | select(.name == "queue_health")'
|
||||
```
|
||||
Worker startup line should now read:
|
||||
`Minion worker started (queue: default, concurrency: 3, watchdog: 2048MB, health-check: 60s)`
|
||||
Under supervisor: the `health-check: Ns` segment is absent (supervisor handles it).
|
||||
5. **If anything fails or numbers look wrong**, file an issue at
|
||||
https://github.com/garrytan/gbrain/issues with `gbrain doctor` output and the contents of
|
||||
`~/.gbrain/upgrade-errors.jsonl` if it exists.
|
||||
|
||||
### Itemized changes
|
||||
|
||||
#### Added
|
||||
- `MinionWorkerOpts.{healthCheckInterval, stallWarnAfterMs, stallExitAfterMs, dbFailExitAfter, dbProbeTimeoutMs}` — five new tuning knobs. Defaults: 60s probe interval, 5min warn / 10min exit, 3 DB strikes, 10s per-probe timeout.
|
||||
- `MinionWorker` now extends `EventEmitter`. Emits `'unhealthy'` with `{ reason: 'db_dead', consecutiveFailures, message } | { reason: 'stalled', waitingCount, idleMinutes }`. CLI subscribes; direct API consumers without a listener inherit a fail-stop fallback that calls `process.exit(1)` to preserve pre-refactor semantics.
|
||||
- `gbrain jobs work --health-interval MS` — tune the self-health-check cadence (0 disables; rejects NaN/negative/sub-1000ms typos).
|
||||
- `gbrain jobs supervisor --health-interval MS` — same flag, same validation, same `0 = disable` contract on the supervisor's own probe.
|
||||
- `GBRAIN_SUPERVISED=1` env var on the supervisor's spawned worker child (skips the child's self-health timer to avoid double-monitoring).
|
||||
- `gbrain doctor` `queue_health` subcheck reports RSS-watchdog kills in the last 24h via exact match on `error_text = 'aborted: watchdog'` scoped to `status IN ('dead','failed')`.
|
||||
- `skills/migrations/v0.22.14.md` — full migration walkthrough with per-PM restart-policy preflight, RSS-posture decision tree, and per-system unit-file edits.
|
||||
|
||||
#### Changed
|
||||
- **Default `--max-rss` for `gbrain jobs work`: 0 → 2048 MB.** Matches supervisor default. Catches memory-leak stalls that previously went undetected on bare workers. Opt out with `--max-rss 0`.
|
||||
- **Bare-worker behavior is now fail-stop** when the DB is unreachable or the queue stalls. Pre-v0.22.14 the worker idled silently. Now it exits and relies on the external PM (systemd, Docker, launchd, cron, supervisord) to restart cleanly.
|
||||
- Stall query at `worker.ts` filters by registered handler names (`AND name = ANY($2::text[])`) so workers don't false-positive when waiting jobs of unhandled names accumulate.
|
||||
- Stall exit threshold measured from `lastCompletionTime` (not from when the warning fired), so 5min warn / 10min exit means total idle of 10 min — not 15 min.
|
||||
- DB liveness probe wrapped in `Promise.race` against a 10s timeout so a hung `executeRaw` cannot wedge the recursive `setTimeout` chain forever.
|
||||
- `setInterval` → recursive `setTimeout` with a `running` flag throughout. Eliminates timer-callback overlap on slow probes.
|
||||
- `parseMaxRssFlag` returns `number | undefined` (was `number`) so callers distinguish absent from explicit-disable.
|
||||
- `process.env.GBRAIN_SUPERVISED` check tightened from `!!env.X` to `=== '1'` (precise contract; no fuzzy matching on `'0'` or `'false'`).
|
||||
- `MinionWorker` constructor throws when `stallExitAfterMs <= stallWarnAfterMs` so misconfigurations fail loudly at startup.
|
||||
|
||||
#### Fixed
|
||||
- **Wedged-worker false-positive on heterogeneous queues** — workers registering only some handlers no longer interpret waiting jobs of other names as a stall. Repeated `process.exit(1)` → restart loop is gone.
|
||||
- **Hung DB probe wedge** — pre-fix, a hung `executeRaw('SELECT 1')` kept the recursive `setTimeout` from rescheduling, silently disabling the entire health monitor. Post-fix, the probe times out and counts as a failure.
|
||||
- **`--health-interval 0` no longer DB-hammers the supervisor.** Pre-fix, the documented "0 disables" contract was a lie — `setInterval(cb, 0)` schedules a tight loop. Now gated behind `> 0`.
|
||||
- **Inline `jobs submit --follow` and `jobs smoke` no longer kill the user's CLI session** on a DB blip. Both now pass `healthCheckInterval: 0` so the no-listener fallback can't trip on one-shot runs.
|
||||
- Doctor's RSS-watchdog hint matches the actual error_text signature (`'aborted: watchdog'`) instead of the wrong `'memory limit'` literal that never matched.
|
||||
|
||||
#### For contributors
|
||||
- `MinionWorker extends EventEmitter` — if you import the class directly, the `on('unhealthy', ...)` event is now part of the public surface. The `UnhealthyReason` discriminated union is exported from `src/core/minions/worker.ts`.
|
||||
- New regression-test infrastructure in `test/minions.test.ts`: `makeProbeEngine(overrides)` is a Proxy-based engine wrapper that intercepts `SELECT 1` and the stall `count(*)` query while passing every other call through to the real PGLite engine. Useful for any future test that needs to inject DB liveness or stall semantics without mocking the entire engine surface.
|
||||
|
||||
### Adjacent (separate PR, v0.22.15)
|
||||
|
||||
PR #503 catches the *symptom* of one specific failure mode. The cause-side fix — `runPhaseEmbed → embed.ts → embedBatch` not honoring `signal.aborted` between OpenAI batch calls — ships in v0.22.15 (highest-priority TODO; daily wedge driver). Plumbing is documented in `TODOS.md`.
|
||||
|
||||
## [0.22.13] - 2026-04-28
|
||||
|
||||
**Sync got faster, and the bookmark stopped lying.**
|
||||
|
||||
@@ -22,7 +22,7 @@ strict behavior when unset.
|
||||
|
||||
## Key files
|
||||
|
||||
- `src/core/operations.ts` — Contract-first operation definitions (the foundation). Also exports upload validators: `validateUploadPath`, `validatePageSlug`, `validateFilename`, plus `matchesSlugAllowList(slug, prefixes)` (v0.23 glob matcher: `<prefix>/*` matches recursive children; bare `<prefix>` matches exact only). `OperationContext.remote` flags untrusted callers; `OperationContext.allowedSlugPrefixes` (v0.23) is the trusted-workspace allow-list set by the dream cycle. `put_page` enforces: when `viaSubagent` and `allowedSlugPrefixes` is set, slug must match the allow-list; else the legacy `wiki/agents/<id>/...` namespace check applies. Auto-link enabled for trusted-workspace writes (skipped only when `remote=true && !trustedWorkspace`).
|
||||
- `src/core/operations.ts` — Contract-first operation definitions (the foundation). Also exports upload validators: `validateUploadPath`, `validatePageSlug`, `validateFilename`. `OperationContext.remote` flags untrusted callers.
|
||||
- `src/core/engine.ts` — Pluggable engine interface (BrainEngine). `clampSearchLimit(limit, default, cap)` takes an explicit cap so per-operation caps can be tighter than `MAX_SEARCH_LIMIT`. Exports `LinkBatchInput` / `TimelineBatchInput` for the v0.12.1 bulk-insert API (`addLinksBatch` / `addTimelineEntriesBatch`). As of v0.13.1, `BrainEngine` has a `readonly kind: 'postgres' | 'pglite'` discriminator so migrations (`src/core/migrate.ts`) and other consumers can branch on engine without `instanceof` + dynamic imports.
|
||||
- `src/core/engine-factory.ts` — Engine factory with dynamic imports (`'pglite'` | `'postgres'`)
|
||||
- `src/core/pglite-engine.ts` — PGLite (embedded Postgres 17.5 via WASM) implementation, all 40 BrainEngine methods. `addLinksBatch` / `addTimelineEntriesBatch` use multi-row `unnest()` with manual `$N` placeholders. As of v0.13.1, `connect()` wraps `PGlite.create()` in a try/catch that emits an actionable error naming the macOS 26.3 WASM bug (#223) and pointing at `gbrain doctor`; the lock is released on failure so the next process can retry cleanly. v0.22.0: `searchKeyword` and `searchKeywordChunks` multiply `ts_rank` by the source-factor CASE expression at the chunk-grain level; `searchVector` becomes a two-stage CTE — inner CTE keeps `ORDER BY cc.embedding <=> vec` so HNSW stays usable, outer SELECT re-ranks by `raw_score * source_factor`. Inner LIMIT scales with offset to preserve pagination contract. As of v0.22.6.1, `initSchema()` calls `applyForwardReferenceBootstrap()` BEFORE replaying SCHEMA_SQL — probes for the specific forward-referenced state the embedded schema blob needs (`pages.source_id`, `links.link_source`, `links.origin_page_id`, `content_chunks.symbol_name`, `content_chunks.language`, `sources` FK target table) and adds only what's missing. Closes the upgrade-wedge bug class that bit users 10+ times across 6 schema versions over 2 years (#239/#243/#266/#357/#366/#374/#375/#378/#395/#396). No-op on fresh installs and modern brains.
|
||||
@@ -87,7 +87,7 @@ strict behavior when unset.
|
||||
- `src/core/minions/wait-for-completion.ts` (v0.15) — poll-until-terminal helper for CLI callers. `TimeoutError` does NOT cancel the job; `AbortSignal` exits without throwing. Default `pollMs`: 1000 on Postgres, 250 on PGLite inline.
|
||||
- `src/core/minions/transcript.ts` (v0.15) — renders `subagent_messages` + `subagent_tool_executions` to markdown. Tool rows splice under their owning assistant `tool_use` by `tool_use_id`. UTF-8-safe truncation; unknown block types fall through to fenced JSON.
|
||||
- `src/core/minions/plugin-loader.ts` (v0.15) — `GBRAIN_PLUGIN_PATH` discovery. Absolute paths only, left-wins collision, `gbrain.plugin.json` with `plugin_version: "gbrain-plugin-v1"`, plugins ship DEFS only (no new tools), `allowed_tools:` validated at load time against the derived registry.
|
||||
- `src/core/minions/tools/brain-allowlist.ts` (v0.15, extended v0.23) — derives subagent tool registry from `src/core/operations.ts`. 11-name allow-list. By default `put_page` schema is namespace-wrapped per subagent (`^wiki/agents/<subagentId>/.+`). **v0.23 trusted-workspace path:** when `BuildBrainToolsOpts.allowedSlugPrefixes` is set, the put_page schema instead describes the prefix list to the model and the OperationContext is threaded with `allowedSlugPrefixes`. Trust comes from `PROTECTED_JOB_NAMES` gating subagent submission — MCP cannot reach this field. Only cycle.ts (synthesize/patterns) and direct CLI submitters set it.
|
||||
- `src/core/minions/tools/brain-allowlist.ts` (v0.15) — derives subagent tool registry from `src/core/operations.ts`. 11-name allow-list: `query`, `search`, `get_page`, `list_pages`, `file_list`, `file_url`, `get_backlinks`, `traverse_graph`, `resolve_slugs`, `get_ingest_log`, `put_page`. `put_page` schema is namespace-wrapped per subagent (`^wiki/agents/<subagentId>/.+`); the `put_page` op's server-side check is the authoritative gate via `ctx.viaSubagent` fail-closed.
|
||||
- `src/mcp/tool-defs.ts` (v0.15) — extracted `buildToolDefs(ops)` helper. MCP server + subagent tool registry both call it; byte-for-byte equivalence pinned by `test/mcp-tool-defs.test.ts`.
|
||||
- `src/core/minions/attachments.ts` — Attachment validation (path traversal, null byte, oversize, base64, duplicate detection)
|
||||
- `src/commands/agent.ts` (v0.16) — `gbrain agent run <prompt> [flags]` CLI. Submits `subagent` (or N children + 1 aggregator) under `{allowProtectedSubmit: true}`. Single-entry `--fanout-manifest` short-circuits. Children get `on_child_fail: 'continue'` + `max_stalled: 3`. `--follow` is the default on TTY; streams logs + polls `waitForCompletion` in parallel. Ctrl-C detaches, does not cancel.
|
||||
@@ -106,20 +106,14 @@ strict behavior when unset.
|
||||
- `src/commands/orphans.ts` — `gbrain orphans [--json] [--count] [--include-pseudo]`: surfaces pages with zero inbound wikilinks, grouped by domain. Auto-generated/raw/pseudo pages filtered by default. Also exposed as `find_orphans` MCP operation. Shipped in v0.12.3 (contributed by @knee5).
|
||||
- `src/commands/integrity.ts` — `gbrain integrity check|auto|review|extract`: bare-tweet detection, dead-link detection, three-bucket repair (auto-repair / review-queue / skip). `scanIntegrity()` is the shared library function called from `gbrain doctor` (sampled at limit=500) and `cmdCheck` (full scan). v0.22.8: batch-load fast path on Postgres uses `SELECT DISTINCT ON (slug)` in a single SQL query to fix the PgBouncer round-trip timeout (60s → ~6s) while preserving `engine.getAllSlugs()`'s `Set<string>` semantics on multi-source brains. Gated by `engine.kind === 'postgres'` at the call site so PGLite never enters batch; fallback `catch` logs at `GBRAIN_DEBUG=1` so real Postgres errors are diagnosable.
|
||||
- `src/commands/doctor.ts` — `gbrain doctor [--json] [--fast] [--fix] [--dry-run] [--index-audit]`: health checks. v0.12.3 added `jsonb_integrity` + `markdown_body_completeness` reliability checks. v0.14.1: `--fix` delegates inlined cross-cutting rules to `> **Convention:** see [path](path).` callouts (pipes DRY violations into `src/core/dry-fix.ts`); `--fix --dry-run` previews without writing. v0.14.2: `schema_version` check fails loudly when `version=0` (migrations never ran — the #218 `bun install -g` signature) and routes users to `gbrain apply-migrations --yes`; new opt-in `--index-audit` flag (Postgres-only) reports zero-scan indexes from `pg_stat_user_indexes` (informational only, no auto-drop). v0.15.2: every DB check is wrapped in a progress phase; `markdown_body_completeness` runs under a 1s heartbeat timer so 10+ min scans are observable on 50K-page brains. v0.19.1 added `queue_health` (Postgres-only) with two subchecks: stalled-forever active jobs (started_at > 1h) and waiting-depth-per-name > threshold (default 10, override via `GBRAIN_QUEUE_WAITING_THRESHOLD`). Worker-heartbeat subcheck intentionally deferred to follow-up B7 because it needs a `minion_workers` table to produce ground-truth signal. Fix hints point at `gbrain repair-jsonb`, `gbrain sync --force`, `gbrain apply-migrations`, and `gbrain jobs get/cancel <id>`. v0.22.12 (#500): `sync_failures` check shows `[CODE=N, ...]` breakdown for both unacked entries (warn) and acked-historical entries (ok), surfacing systemic failure modes (`SLUG_MISMATCH=2685`) instead of a bare count.
|
||||
- `src/core/migrate.ts` — schema-migration runner. Owns the `MIGRATIONS` array (source of truth for schema DDL). v0.14.2 extended the `Migration` interface with `sqlFor?: { postgres?, pglite? }` (engine-specific SQL overrides `sql`) and `transaction?: boolean` (set to false for `CREATE INDEX CONCURRENTLY`, which Postgres refuses inside a transaction; ignored on PGLite since it has no concurrent writers). Migration v14 (fix wave) uses a handler branching on `engine.kind` to run CONCURRENTLY on Postgres (with a pre-drop of any invalid remnant via `pg_index.indisvalid`) and plain `CREATE INDEX` on PGLite. v15 bumps `minion_jobs.max_stalled` default 1→5 and backfills existing non-terminal rows. v0.22.6.1: migration v24 (`rls_backfill_missing_tables`) uses `sqlFor: { pglite: '' }` to no-op on PGLite — PGLite has no RLS engine and is single-tenant by definition, and the v24 ALTERs target subagent tables that don't exist in pglite-schema.ts. Closes #395 (contributed by @jdcastro2). **v30 (v0.23):** creates `dream_verdicts (file_path TEXT, content_hash TEXT, worth_processing BOOL, reasons JSONB, judged_at TIMESTAMPTZ, PK(file_path, content_hash))`. RLS-enabled when running as a BYPASSRLS role. The synthesize phase reads/writes this table to avoid re-judging on backfill re-runs.
|
||||
- `src/core/migrate.ts` — schema-migration runner. Owns the `MIGRATIONS` array (source of truth for schema DDL). v0.14.2 extended the `Migration` interface with `sqlFor?: { postgres?, pglite? }` (engine-specific SQL overrides `sql`) and `transaction?: boolean` (set to false for `CREATE INDEX CONCURRENTLY`, which Postgres refuses inside a transaction; ignored on PGLite since it has no concurrent writers). Migration v14 (fix wave) uses a handler branching on `engine.kind` to run CONCURRENTLY on Postgres (with a pre-drop of any invalid remnant via `pg_index.indisvalid`) and plain `CREATE INDEX` on PGLite. v15 bumps `minion_jobs.max_stalled` default 1→5 and backfills existing non-terminal rows. v0.22.6.1: migration v24 (`rls_backfill_missing_tables`) uses `sqlFor: { pglite: '' }` to no-op on PGLite — PGLite has no RLS engine and is single-tenant by definition, and the v24 ALTERs target subagent tables that don't exist in pglite-schema.ts. Closes #395 (contributed by @jdcastro2).
|
||||
- `src/core/progress.ts` — Shared bulk-action progress reporter. Writes to stderr. Modes: `auto` (TTY: `\r`-rewriting; non-TTY: plain lines), `human`, `json` (JSONL), `quiet`. Rate-gated by `minIntervalMs` and `minItems`. `startHeartbeat(reporter, note)` helper for single long queries. `child()` composes phase paths. Singleton SIGINT/SIGTERM coordinator emits `abort` events for every live phase. EPIPE defense on both sync throws and stream `'error'` events. Zero dependencies. Introduced in v0.15.2.
|
||||
- `src/core/cli-options.ts` — Global CLI flag parser. `parseGlobalFlags(argv)` returns `{cliOpts, rest}` with `--quiet` / `--progress-json` / `--progress-interval=<ms>` stripped. `getCliOptions()` / `setCliOptions()` expose a module-level singleton so commands reach the resolved flags without parameter threading. `cliOptsToProgressOptions()` maps to reporter options. `childGlobalFlags()` returns the flag suffix to append to `execSync('gbrain ...')` calls in migration orchestrators. `OperationContext.cliOpts` extends shared-op dispatch for MCP callers.
|
||||
- `src/core/db-lock.ts` (v0.22.13) — generic `tryAcquireDbLock(engine, lockId, ttlMinutes)` over the existing `gbrain_cycle_locks` table. Parameterized lock id so different scopes can nest cleanly: `gbrain-cycle` for the broad cycle (held by `cycle.ts`) and `gbrain-sync` (`SYNC_LOCK_ID` constant) for `performSync`'s narrower writer window. Same UPSERT-with-TTL semantics as the prior cycle-only helper, just generalized. Survives PgBouncer transaction pooling (unlike session-scoped `pg_try_advisory_lock`); crashed holders auto-release once their TTL expires.
|
||||
- `src/core/sync-concurrency.ts` (v0.22.13) — single source of truth for the parallel-sync policy. Exports `autoConcurrency(engine, fileCount, override?)` (PGLite always serial; explicit override clamped to >=1; auto path returns `DEFAULT_PARALLEL_WORKERS=4` when `fileCount > AUTO_CONCURRENCY_FILE_THRESHOLD=100`), `shouldRunParallel(workers, fileCount, explicit)` (Q1: explicit `--workers` bypasses the >50-file floor), and `parseWorkers(s)` (rejects `'0'`, `'-3'`, `'foo'`, `'1.5'`, trailing chars — replaces the prior parseInt-with-no-validation in both `sync.ts` and `import.ts`). Used by `performSync`, `performFullSync`, `runImport`, and the Minion `sync` handler so the three sites can no longer drift.
|
||||
- `src/commands/sync.ts` — `gbrain sync` CLI + the `performSync` / `performFullSync` library entrypoints (consumed by the autopilot cycle and the Minion sync handler). v0.22.13 (PR #490): `performSync` wraps its body in a `gbrain-sync` writer lock so two concurrent syncs (manual + autopilot, two terminals, two Conductor workspaces) cannot both write `last_commit` and let the last writer win. Head-drift gate after the import phase re-checks `git rev-parse HEAD`; if HEAD moved (someone ran `git checkout` / `git pull` mid-sync), the bookmark refuses to advance. Vanished files now record a failedFiles entry instead of silent-skip — the silent-skip-then-advance pathology that survived prior hardening passes is dead. Worker engines wrap in try/finally so disconnect always fires (panic-path leak fix). Both PGLite-detection sites use `engine.kind === 'pglite'`. CLI accepts `--workers N` (alias `--concurrency N`), validated via `parseWorkers`. Explicit `--workers` bypasses the auto-path file-count floor; auto path defers to `autoConcurrency()`. Banner moved to stderr.
|
||||
- `src/core/cycle.ts` — v0.17 brain maintenance cycle primitive (extended to **8 phases in v0.23**). `runCycle(engine: BrainEngine | null, opts: CycleOpts): Promise<CycleReport>` composes phases in semantically-driven order: **lint → backlinks → sync → synthesize → extract → patterns → embed → orphans**. v0.23's `synthesize` phase runs after sync (cross-references see fresh brain) and before extract (auto-link materializes its writes); `patterns` runs after extract so it reads a fresh graph (codex finding #7 — subagent put_page sets `ctx.remote=true` and skips auto-link/timeline by default; extract is the canonical materialization). Three callers: `gbrain dream` CLI, `gbrain autopilot` daemon's inline path, and the Minions `autopilot-cycle` handler. Coordination via `gbrain_cycle_locks` DB table + `~/.gbrain/cycle.lock` file lock with PID-liveness for PGLite. `CycleReport.schema_version: "1"` is stable; totals additively grew in v0.23 (`transcripts_processed`, `synth_pages_written`, `patterns_written`). `yieldBetweenPhases` runs between phases. **v0.23 added `yieldDuringPhase`** for in-phase keepalive — synthesize/patterns call it during long waits to renew the cycle-lock TTL. Engine nullable; lock-skip on read-only phase selections. v0.22.1 (#403): `CycleOpts.signal?: AbortSignal` propagates the worker's abort signal; `checkAborted()` fires between every phase. v0.22.1 (#417): `runPhaseSync` returns `pagesAffected` via `SyncPhaseResult`; `runCycle` captures it and threads to `runPhaseExtract` as the 4th arg. v0.22.1 (Codex F2): `runPhaseSync` takes `willRunExtractPhase: boolean` and sets `noExtract: phases.includes('extract')` so `gbrain dream --phase sync` doesn't silently lose extraction. v0.22.5 (#475): `resolveSourceForDir(engine, brainDir)` threads `sourceId` to `performSync()` so sync reads the per-source `sources.last_commit` anchor instead of the drift-prone global `config.sync.last_commit` key.
|
||||
- `src/core/cycle/synthesize.ts` (v0.23) — Synthesize phase: conversation-transcript-to-brain pipeline. Reads from `dream.synthesize.session_corpus_dir`, runs cheap Haiku verdict (cached in `dream_verdicts`), then fans out one Sonnet subagent per worth-processing transcript with `allowed_slug_prefixes` (sourced from `skills/_brain-filing-rules.json` `dream_synthesize_paths.globs`). Orchestrator collects slugs from `subagent_tool_executions` (NOT `pages.updated_at` — codex finding #2) and reverse-renders DB → markdown via `serializeMarkdown`. Cooldown via `dream.synthesize.last_completion_ts`, written ONLY on success. Idempotency key `dream:synth:<file_path>:<content_hash>`. Auto-commit deferred to v1.1 (codex #5). `--dry-run` runs Haiku, skips Sonnet (codex #8). Subagent never gets fs-write access.
|
||||
- `src/core/cycle/patterns.ts` (v0.23) — Patterns phase: cross-session theme detection over reflections within `dream.patterns.lookback_days` (default 30). Names a pattern only when ≥`dream.patterns.min_evidence` (default 3) reflections support it. Single Sonnet subagent; same allow-list path as synthesize. Runs AFTER `extract` so the graph is fresh.
|
||||
- `src/core/cycle/transcript-discovery.ts` (v0.23) — Pure filesystem walk for synthesize. `discoverTranscripts(opts)` filters `.txt` files by date range, min_chars, and word-boundary regex `excludePatterns` (Q-3: `medical` matches "medical advice" but NOT "comedical"; power users may pass full regex). `readSingleTranscript(path)` is the `gbrain dream --input <file>` ad-hoc path.
|
||||
- `src/commands/dream.ts` — v0.17 `gbrain dream` CLI; ~80-line thin alias over `runCycle`. brainDir resolution requires explicit `--dir` OR `sync.repo_path` config. Flags: `--dry-run`, `--json`, `--phase <name>`, `--pull`, `--dir <path>`. **v0.23 added** `--input <file>` (ad-hoc transcript, implies `--phase synthesize`), `--date YYYY-MM-DD`, `--from <d> --to <d>` (backfill range). Conflict detection: `--input` + `--date` exits 2. ISO date validation. `--dry-run` runs Haiku significance verdict but skips Sonnet synthesis (codex finding #8 — NOT zero LLM calls). Exit code 1 on status=failed.
|
||||
- `src/commands/friction.ts` + `src/core/friction.ts` (v0.23) — `gbrain friction {log,render,list,summary}` reporter. Append-only JSONL under `$GBRAIN_HOME/friction/<run-id>.jsonl`. Schema is a flat extension of `StructuredAgentError` (D20). Render groups by severity → phase, defaults to `--redact` for md output (strips `$HOME`/`$CWD` to placeholders so reports paste safely in PRs). Run-id resolves from `--run-id` > `$GBRAIN_FRICTION_RUN_ID` > `standalone.jsonl`. Skills the claw-test exercises gain a `_friction-protocol.md` callout so agents know when to log friction.
|
||||
- `src/commands/claw-test.ts` + `src/core/claw-test/` (v0.23) — `gbrain claw-test [--scenario <name>] [--live --agent openclaw]`. End-to-end "fresh user" friction harness. Two modes: scripted (CI gate, agent-free) and live (real openclaw subprocess, $1–2 in tokens). Sets `GBRAIN_HOME=<tempdir>` for hermeticity and captures gbrain's `--progress-json` events from each child's stderr to verify expected phases ran (`import.files`, `extract.links_fs`, `doctor.db_checks`). Phases for scripted mode: setup → install_brain (`gbrain init --pglite`) → import (`--no-embed`) → query → extract → verify (`gbrain doctor --json`, asserts `status: 'ok'`) → render. Live mode hands `BRIEF.md` from `test/fixtures/claw-test-scenarios/<name>/` to the agent runner. v1 ships with the OpenClaw runner only (`src/core/claw-test/runners/openclaw.ts`, invokes `openclaw agent --local --agent <name> --message <brief>`); hermes runner deferred to v1.1. Transcript capture (`transcript-capture.ts`) uses `fs.createWriteStream` with `'drain'`-event backpressure — D17 fix for the 256KB-burst child-stall scenario. v0.18 upgrade scenario seeded via `seed-pglite.ts` SQL replay.
|
||||
- `skills/_friction-protocol.md` (v0.23) — shared cross-cutting convention skill (like `_brain-filing-rules.md`). Tells agents when to call `gbrain friction log` and how to choose a severity. Routes to friction CLI from any skill the claw-test exercises.
|
||||
- `src/core/cycle.ts` — v0.17 brain maintenance cycle primitive. `runCycle(engine: BrainEngine | null, opts: CycleOpts): Promise<CycleReport>` composes 6 phases in semantically-driven order (lint → backlinks → sync → extract → embed → orphans). Three callers: `gbrain dream` CLI, `gbrain autopilot` daemon's inline path, and the Minions `autopilot-cycle` handler (`src/commands/jobs.ts`). One source of truth for what the brain does overnight. Coordination via `gbrain_cycle_locks` DB table (TTL-based; works through PgBouncer transaction pooling, unlike session-scoped `pg_try_advisory_lock`) + `~/.gbrain/cycle.lock` file lock with PID-liveness for PGLite / engine=null mode. `CycleReport.schema_version: "1"` is the stable agent-consumable shape. `PhaseResult.error: { class, code, message, hint?, docs_url? }` is Stripe-API-tier structured failure info. `yieldBetweenPhases` hook awaited between every phase — Minions handler uses this to renew its job lock and prevent v0.14 stall-death regression. Engine nullable: filesystem phases (lint, backlinks) run without DB; DB phases skip with `status: "skipped", reason: "no_database"`. Lock-skip: read-only phase selections (`--phase orphans`) bypass the cycle lock. v0.22.1 (#403): `CycleOpts.signal?: AbortSignal` propagates the worker's abort signal; `checkAborted()` fires between every phase and throws if the signal is aborted (cooperative — can't interrupt a phase mid-execution). v0.22.1 (#417): `runPhaseSync` returns `pagesAffected` via `SyncPhaseResult`; `runCycle` captures it and threads to `runPhaseExtract` as the 4th arg, enabling incremental extract on the cycle path. v0.22.1 (Codex F2): `runPhaseSync` takes `willRunExtractPhase: boolean` and sets `noExtract: phases.includes('extract')` so `gbrain dream --phase sync` doesn't silently lose extraction. v0.22.5 (#475): new `resolveSourceForDir(engine, brainDir)` helper queries `SELECT id FROM sources WHERE local_path = $1 LIMIT 1`; `runPhaseSync` threads result as `sourceId` to `performSync()` so sync reads the per-source `sources.last_commit` anchor instead of the drift-prone global `config.sync.last_commit` key. Bare try/catch lets pre-v0.18 brains fall through to the global key. Closes the prod hang where every autopilot cycle ran a 30-min full reimport because the global anchor commit had been GC'd from git history.
|
||||
- `src/commands/dream.ts` — v0.17 `gbrain dream` CLI. ~80-line thin alias over `runCycle`. brainDir resolution requires explicit `--dir` OR `sync.repo_path` config (no more walk-up-cwd-for-.git footgun). Flags: `--dry-run`, `--json`, `--phase <name>`, `--pull`, `--dir <path>`. Exit code 1 on status=failed (partial/warn not fatal — don't page on warnings).
|
||||
- `scripts/check-progress-to-stdout.sh` — CI guard against regressing to `\r`-on-stdout progress. Wired into `bun run test` via `scripts/check-progress-to-stdout.sh && bun test` in package.json.
|
||||
- `docs/progress-events.md` — Canonical JSON event schema reference. Stable from v0.15.2, additive only.
|
||||
- `src/core/markdown.ts` — Frontmatter parsing + body splitter. `splitBody` requires an explicit timeline sentinel (`<!-- timeline -->`, `--- timeline ---`, or `---` immediately before `## Timeline`/`## History`). Plain `---` in body text is a markdown horizontal rule, not a separator. `inferType` auto-types `/wiki/analysis/` → analysis, `/wiki/guides/` → guide, `/wiki/hardware/` → hardware, `/wiki/architecture/` → architecture, `/writing/` → writing (plus the existing people/companies/deals/etc heuristics).
|
||||
@@ -233,16 +227,6 @@ Key commands added in v0.22.13 (PR #490):
|
||||
- `gbrain sync --workers N` (alias `--concurrency N`) — parallelize the import phase using per-worker Postgres engines (small pool of 2 each) with an atomic queue index. Auto-concurrency: defaults to 4 workers when the diff exceeds 100 files. Smaller diffs stay serial. Explicit `--workers` always wins (even on a 30-file diff). PGLite forces serial regardless. Validation rejects `0`, negatives, non-integers loud (replaces the prior silent fall-through to auto-concurrency).
|
||||
- `gbrain import --workers N` — same `parseWorkers()` validation as sync; same try/finally worker-engine cleanup. Behavior surface unchanged.
|
||||
|
||||
Key commands added in v0.22.16 (claw-test friction loop):
|
||||
- `gbrain claw-test [--scenario fresh-install|upgrade-from-v0.18] [--keep-tempdir]` — scripted-mode CI gate that runs the full canonical first-day flow against a fresh tempdir. Asserts every expected `--progress-json` phase fired and doctor's `status === 'ok'`. ~30s, no API keys.
|
||||
- `gbrain claw-test --live --agent openclaw` — friction-discovery mode. Spawns real openclaw, hands it `BRIEF.md`, captures stdin/stdout/stderr to `<run>/transcript.jsonl`, lets the agent log friction via the friction CLI. Run on demand; ~5–10 min and ~$1–2 in tokens.
|
||||
- `gbrain claw-test --list-agents` — reports which agent runners are registered + their detection state (binary path or unavailable reason).
|
||||
- `gbrain friction log --severity {confused|error|blocker|nit} --phase <name> --message <text> [--hint ...] [--kind {friction|delight}] [--run-id ...]` — append a friction or delight entry to the active run JSONL.
|
||||
- `gbrain friction render --run-id <id> [--json] [--transcripts] [--no-redact]` — markdown report grouped by severity + phase; `--redact` is the default for md output (strips `$HOME`/`$CWD` placeholders so reports paste safely in PRs/issues).
|
||||
- `gbrain friction list [--json]` — recent run-ids with friction/delight counts; interrupted runs marked `(interrupted)`.
|
||||
- `gbrain friction summary --run-id <id> [--json]` — two-column friction + delight summary.
|
||||
- `GBRAIN_HOME` env override is now honored uniformly across every gbrain write site (config, audit, friction, sync-failures, import checkpoint, integrity log, integrations heartbeat, migration rollback, etc.) — `gbrainPath(...)` from `src/core/config.ts` is the canonical helper. Read-side host-fingerprint detection (`~/.claude`/`~/.openclaw` etc.) intentionally NOT confined in v1; that's a v1.1 follow-up.
|
||||
|
||||
## Testing
|
||||
|
||||
`bun test` runs all tests. After the v0.12.1 release: ~75 unit test files + 8 E2E test files (1412 unit pass, 119 E2E when `DATABASE_URL` is set — skip gracefully otherwise). Unit tests run
|
||||
|
||||
@@ -129,9 +129,8 @@ Set up using your platform's scheduler (OpenClaw cron, Railway cron, crontab):
|
||||
- **Live sync** (every 15 min): `gbrain sync --repo ~/brain && gbrain embed --stale`
|
||||
- **Auto-update** (daily): `gbrain check-update --json` (tell user, never auto-install)
|
||||
- **Dream cycle** (nightly): read `docs/guides/cron-schedule.md` for the full protocol.
|
||||
Entity sweep, citation fixes, memory consolidation, plus (v0.23+) overnight conversation
|
||||
synthesis and cross-session pattern detection. 8 phases, one cron-friendly command. This
|
||||
is what makes the brain compound. Do not skip it.
|
||||
Entity sweep, citation fixes, memory consolidation. This is what makes the brain
|
||||
compound. Do not skip it.
|
||||
- **Weekly**: `gbrain doctor --json && gbrain embed --stale`
|
||||
|
||||
## Step 8: Integrations
|
||||
|
||||
@@ -132,7 +132,7 @@ GBrain ships 29 skills organized by `skills/RESOLVER.md` (or your OpenClaw's `AG
|
||||
|-------|-------------|
|
||||
| **enrich** | Tiered enrichment (Tier 1/2/3). Creates and updates person/company pages with compiled truth and timelines. |
|
||||
| **query** | 3-layer search with synthesis and citations. Says "the brain doesn't have info on X" instead of hallucinating. |
|
||||
| **maintain** | Periodic health: stale pages, orphans, dead links, citation audit, back-link enforcement, tag consistency. v0.23 adds the dream cycle's synthesize + patterns phases ... overnight conversation transcripts become reflections, originals, and 25-year patterns. |
|
||||
| **maintain** | Periodic health: stale pages, orphans, dead links, citation audit, back-link enforcement, tag consistency. |
|
||||
| **citation-fixer** | Scans pages for missing or malformed citations. Fixes format to match the standard. |
|
||||
| **repo-architecture** | Where new brain files go. Decision protocol: primary subject determines directory, not format. |
|
||||
| **publish** | Share brain pages as password-protected HTML. Zero LLM calls. |
|
||||
@@ -689,11 +689,7 @@ ADMIN
|
||||
gbrain auth create|list|revoke|test Token management for the HTTP transport
|
||||
gbrain integrations Integration recipe dashboard
|
||||
gbrain sources list|add|remove|... Multi-source brain management (v0.18)
|
||||
gbrain dream [--dry-run] [--phase N] 8-phase maintenance cycle (lint→backlinks→sync→synthesize
|
||||
→extract→patterns→embed→orphans). v0.23 added synthesize +
|
||||
patterns: transcripts → reflections + cross-session themes.
|
||||
gbrain dream --input <file> Ad-hoc transcript synthesis (implies --phase synthesize)
|
||||
gbrain dream --date YYYY-MM-DD Synthesize a single day; --from/--to for backfill ranges
|
||||
gbrain dream [--dry-run] [--phase N] One maintenance cycle then exit (cron-friendly)
|
||||
gbrain check-backlinks check|fix Back-link enforcement
|
||||
gbrain lint [--fix] LLM artifact detection
|
||||
gbrain repair-jsonb [--dry-run] Repair v0.12.0 double-encoded JSONB (Postgres)
|
||||
|
||||
@@ -1,212 +1,5 @@
|
||||
# TODOS
|
||||
|
||||
## claw-test E2E (v0.22.16 follow-ups)
|
||||
|
||||
### Hermes runner — `src/core/claw-test/runners/hermes.ts`
|
||||
**Priority:** P2
|
||||
|
||||
**What:** Add a Hermes implementation of the `AgentRunner` interface. v1 ships only OpenClaw; v1.1 lands hermes once we have real friction reports from openclaw to validate the contract against.
|
||||
|
||||
**Why:** Cross-agent diff (`gbrain friction diff --base openclaw --compare hermes`) is the highest-leverage next signal. Friction unique to one agent vs common-to-both separates "agent contract bug" from "gbrain bug" automatically.
|
||||
|
||||
**Effort:** S (CC ~30m). Depends on: v1 openclaw runner producing real friction reports first.
|
||||
|
||||
---
|
||||
|
||||
### Friction analytics suite — `diff` / `trend` / `migration-stub`
|
||||
**Priority:** P2
|
||||
|
||||
**What:** Three new `gbrain friction` subcommands deferred from v1:
|
||||
- `gbrain friction diff --base <run-or-agent> --compare <run-or-agent>` (cross-agent comparison; ~80 LOC)
|
||||
- `gbrain friction trend [--since <version-or-date>] [--phase <name>]` (time-series across runs; ~60 LOC)
|
||||
- `gbrain friction migration-stub [--threshold N]` (clusters friction by phase + tokens, emits `skills/migrations/v[N+1].md` stub; ~150 LOC)
|
||||
|
||||
**Why:** Turns point-in-time reports into a slope. Pairs with the v1.1 public scoreboard.
|
||||
|
||||
**Effort:** M (CC ~2h total).
|
||||
|
||||
---
|
||||
|
||||
### Scenario expansion — `supabase-migration` and `supervisor-restart`
|
||||
**Priority:** P2
|
||||
|
||||
**What:** Two more scenarios under `test/fixtures/claw-test-scenarios/`:
|
||||
- `supabase-migration` — `gbrain init --pglite` then `gbrain migrate --to supabase`; verifies the cross-engine migration path
|
||||
- `supervisor-restart` — kill worker mid-job; verify supervisor recovers without data loss
|
||||
|
||||
**Why:** These are the other highest-historical-pain regression points (per CLAUDE.md fix-wave history). v1 ships only `fresh-install` + `upgrade-from-v0.18` because Codex flagged that mixing them dilutes the fresh-install signal; v1.1 lands them as separate scenarios.
|
||||
|
||||
**Effort:** M (CC ~1h each).
|
||||
|
||||
---
|
||||
|
||||
### Real v0.18 SQL dump for upgrade scenario
|
||||
**Priority:** P2
|
||||
|
||||
**What:** The `upgrade-from-v0.18` scenario ships scaffolded — `seed/dump.sql` is missing. The harness gracefully no-ops the seed phase when absent, so the scenario currently behaves like fresh-install. v1.1: generate a real v0.18-shape PGLite dump per the procedure documented in `test/fixtures/claw-test-scenarios/upgrade-from-v0.18/seed/README.md`.
|
||||
|
||||
**Why:** Without a real seed, the scenario doesn't actually exercise the migration chain forward-walk. That's the whole point of the upgrade scenario — proves issue #239/#243/#266/#357 class regressions stay fixed.
|
||||
|
||||
**Effort:** S (CC ~30m once a v0.18 checkout is handy). Depends on: ability to run a v0.18 gbrain build.
|
||||
|
||||
---
|
||||
|
||||
### Public scoreboard — `gbrain-evals.io/friction`
|
||||
**Priority:** P3
|
||||
|
||||
**What:** Sibling-repo PR in `garrytan/gbrain-evals` that renders friction JSONL into a public dashboard. Friction count per version per agent, line charts over time. v1's JSONL already includes `gbrain_version` + `agent` tags so the scoreboard is a thin layer on top.
|
||||
|
||||
**Why:** Marketing surface. Proves install quality is improving release-over-release. The friction loop becomes visible to the world, not just maintainers.
|
||||
|
||||
**Effort:** M. Depends on: a working live mode and ≥10 real friction reports.
|
||||
|
||||
---
|
||||
|
||||
### PTY-mode transcript capture
|
||||
**Priority:** P3
|
||||
|
||||
**What:** `transcript-capture.ts` currently uses plain `child_process.spawn` pipes. Some agents only emit ANSI colors / progress UI on a TTY. v1.1 adds a PTY mode (likely via `node-pty`) so live-mode transcripts capture the full agent UX.
|
||||
|
||||
**Why:** Faithful transcripts make the friction → reasoning link more useful. v1 accepts that some agent UI is lost.
|
||||
|
||||
**Effort:** S (CC ~30m). Mostly a ~30 LOC swap inside `spawnWithCapture`.
|
||||
|
||||
---
|
||||
|
||||
### Read-side host-isolation (`$GBRAIN_HOST_HOME`)
|
||||
**Priority:** P3
|
||||
|
||||
**What:** v0.22.16 confined every `~/.gbrain` write site to honor `$GBRAIN_HOME`. But `src/commands/init.ts:299-313` still reads real `~/.claude` / `~/.openclaw` / `~/.codex` / `~/.factory` / `~/.kiro` for module fingerprinting (host detection). Even with write-isolation, a claw-test running on a developer's box discovers their real installed mods. v1.1: add a separate `$GBRAIN_HOST_HOME` override for the read-side detection so the claw-test can run truly hermetic.
|
||||
|
||||
**Why:** v1's hermeticity contract is "writes are isolated, reads are not." v1.1 closes the read-side gap.
|
||||
|
||||
**Effort:** S (CC ~30m).
|
||||
|
||||
---
|
||||
|
||||
### Routing-callout sweep — annotate skills the claw-test exercises
|
||||
**Priority:** P3
|
||||
|
||||
**What:** `skills/_friction-protocol.md` is a cross-cutting convention. v1.1: sweep the 4–6 skills the claw-test actually exercises (setup, brain-ops, query, ingest, smoke-test, the migrations the test covers) and add a `> **Convention:** see [skills/_friction-protocol.md](_friction-protocol.md).` callout via the existing `src/core/dry-fix.ts` shape so DRY auto-fix doesn't fight it.
|
||||
|
||||
**Why:** Right now agents only call `gbrain friction log` if they find the protocol skill on their own. The callouts route them there proactively from any harness-exercised skill.
|
||||
|
||||
**Effort:** S (CC ~15m).
|
||||
|
||||
---
|
||||
|
||||
## minions / worker (v0.22.14 follow-ups)
|
||||
|
||||
### v0.22.15 — Embed cooperative-abort (HIGHEST PRIORITY — daily pain)
|
||||
**Priority:** P0
|
||||
|
||||
**What:** Plumb `signal: AbortSignal` through `runPhaseEmbed` →
|
||||
`src/commands/embed.ts` → `embedBatch` in `src/core/embedding.ts`. Check
|
||||
`signal?.aborted` between OpenAI batch calls (every ~100 texts, ~2s
|
||||
real-time) and between slugs in the per-slug loop.
|
||||
|
||||
**Why:** Embed phase ignores `signal.aborted` between batches today. Job
|
||||
wall-clock timeout fires → handler keeps running → cycle's finally block
|
||||
unreachable → `gbrain_cycle_locks` row stays held indefinitely. Every
|
||||
subsequent autopilot cron cycle sees `cycle_already_running` → skips. Lock
|
||||
TTL is 30 min; new cycles give up before that. Doctor reports UNHEALTHY.
|
||||
|
||||
**The chain in production:** ~5min cron submits cycle → 22K stale pages →
|
||||
embed phase takes 10–15 min → 600s timeout fires → job dead-lettered → embed
|
||||
keeps running → lock held → all subsequent cycles skip. Garry hits this
|
||||
DAILY on his production brain.
|
||||
|
||||
**Pros:** Closes the daily wedge. Makes timeouts actually effective. Lets
|
||||
operators bump worker timeouts confidently knowing abort actually stops
|
||||
work.
|
||||
|
||||
**Cons:** Touching the embed hot path; small risk of botching the abort
|
||||
checks. Mitigation: between-batch granularity (~2s), not per-text (too fine)
|
||||
or per-slug (too coarse for 500+ chunk slugs).
|
||||
|
||||
**Context:** PR #503 (v0.22.14) catches the SYMPTOM (worker stalled, queue
|
||||
piling up) via self-health-monitoring. This PR catches the CAUSE for one
|
||||
specific failure class. Both fixes are needed; they're complementary, not
|
||||
duplicative.
|
||||
|
||||
**Files to touch:**
|
||||
- `src/core/cycle.ts:579` — `runPhaseEmbed(engine, dryRun)` → add
|
||||
`signal?: AbortSignal` arg
|
||||
- `src/core/cycle.ts:803` — pass `opts.signal` through
|
||||
- `src/commands/embed.ts:~363` — accept signal, check between slugs
|
||||
- `src/core/embedding.ts:51-56` — `embedBatch(texts, onProgress?, signal?)`,
|
||||
check between for-loop iterations of `BATCH_SIZE` slices
|
||||
|
||||
**Tests required:**
|
||||
1. embedBatch checks signal between OpenAI calls; aborts within one batch (~2s)
|
||||
2. Per-slug loop in `embed.ts` checks signal between slugs
|
||||
3. End-to-end: cycle handler with embed phase + signal aborted mid-flight →
|
||||
finally runs → `gbrain_cycle_locks` row deleted
|
||||
4. Regression: 1K+ chunks scenario — embed does NOT block lock release when
|
||||
timeout fires
|
||||
|
||||
**Effort:** M (human: ~3 hr / CC: ~30 min).
|
||||
|
||||
**Depends on / blocked by:** Nothing. v0.22.14 ships first.
|
||||
|
||||
### v0.23+ — Bare-worker engine reconnect parity with supervisor
|
||||
**Priority:** P2
|
||||
|
||||
**What:** Extract the supervisor's reconnect-then-fail pattern into
|
||||
`MinionWorker` so bare workers can retry transient DB blips before exiting.
|
||||
Today the supervisor calls `engine.reconnect()` after 3 consecutive DB health
|
||||
failures (#406); the bare worker just emits `'unhealthy'` and the CLI calls
|
||||
`process.exit(1)`.
|
||||
|
||||
**Why:** Bare-worker behavior is more disruptive than supervised behavior on
|
||||
transient PgBouncer blips. A bare worker restarts the entire process; a
|
||||
supervised worker just reconnects the pool. Operationally the supervisor
|
||||
approach is gentler (no in-flight job loss, no PM restart latency).
|
||||
|
||||
**Pros:** Unifies bare and supervised behavior. Reduces process churn on
|
||||
transient network blips.
|
||||
|
||||
**Cons:** More code in MinionWorker; risk of reconnect masking a real
|
||||
problem. Mitigation: cap retry attempts, fall through to `'unhealthy'`
|
||||
emission after the cap.
|
||||
|
||||
**Context:** Filed during v0.22.14 plan-eng-review. The asymmetry is
|
||||
documented in v0.22.14 CHANGELOG as deliberate; this TODO captures the
|
||||
"unify someday" intent.
|
||||
|
||||
**Effort:** S (human: ~2 hr / CC: ~20 min).
|
||||
|
||||
**Depends on / blocked by:** Nothing.
|
||||
|
||||
### v0.23+ — `minion_workers` heartbeat table for queue_health doctor (B7)
|
||||
**Priority:** P3
|
||||
|
||||
**What:** Add a `minion_workers` table (`worker_id` PK, `hostname`,
|
||||
`last_heartbeat`, `queue`, `concurrency`, `started_at`) so the existing
|
||||
`queue_health` doctor check (Postgres path) can detect dead workers via
|
||||
heartbeat staleness instead of relying on the indirect `lock_until` proxy.
|
||||
|
||||
**Why:** v0.19.1 added `queue_health` checks for stalled-active jobs and
|
||||
waiting-depth threshold. The worker-heartbeat subcheck was deferred (B7)
|
||||
because the `lock_until`-on-active-jobs proxy can't distinguish "worker
|
||||
exited cleanly" from "worker idle" — a check that cries wolf erodes trust
|
||||
in every doctor check. With a real heartbeat row, doctor can say "no worker
|
||||
seen in N intervals" with confidence.
|
||||
|
||||
**Pros:** Doctor's `queue_health` becomes ground-truth. Detects "worker
|
||||
container died but cron didn't restart it" scenario.
|
||||
|
||||
**Cons:** New table, schema migration, every health-tick UPSERTs. Costs
|
||||
a write per worker per minute (default).
|
||||
|
||||
**Context:** Filed during v0.22.14 plan-eng-review. PR #503's self-health
|
||||
monitoring is the worker-side liveness; this would be the queue-side
|
||||
ground-truth.
|
||||
|
||||
**Effort:** M (human: ~1 day / CC: ~1 hr).
|
||||
|
||||
**Depends on / blocked by:** Schema migration system; nothing else.
|
||||
|
||||
## sync (v0.22.13 follow-up — PR #490 review)
|
||||
|
||||
### D-PR490-1 — Plumb resolved `database_url` through `SyncOpts`
|
||||
|
||||
+9
-31
@@ -101,7 +101,7 @@ strict behavior when unset.
|
||||
|
||||
## Key files
|
||||
|
||||
- `src/core/operations.ts` — Contract-first operation definitions (the foundation). Also exports upload validators: `validateUploadPath`, `validatePageSlug`, `validateFilename`, plus `matchesSlugAllowList(slug, prefixes)` (v0.23 glob matcher: `<prefix>/*` matches recursive children; bare `<prefix>` matches exact only). `OperationContext.remote` flags untrusted callers; `OperationContext.allowedSlugPrefixes` (v0.23) is the trusted-workspace allow-list set by the dream cycle. `put_page` enforces: when `viaSubagent` and `allowedSlugPrefixes` is set, slug must match the allow-list; else the legacy `wiki/agents/<id>/...` namespace check applies. Auto-link enabled for trusted-workspace writes (skipped only when `remote=true && !trustedWorkspace`).
|
||||
- `src/core/operations.ts` — Contract-first operation definitions (the foundation). Also exports upload validators: `validateUploadPath`, `validatePageSlug`, `validateFilename`. `OperationContext.remote` flags untrusted callers.
|
||||
- `src/core/engine.ts` — Pluggable engine interface (BrainEngine). `clampSearchLimit(limit, default, cap)` takes an explicit cap so per-operation caps can be tighter than `MAX_SEARCH_LIMIT`. Exports `LinkBatchInput` / `TimelineBatchInput` for the v0.12.1 bulk-insert API (`addLinksBatch` / `addTimelineEntriesBatch`). As of v0.13.1, `BrainEngine` has a `readonly kind: 'postgres' | 'pglite'` discriminator so migrations (`src/core/migrate.ts`) and other consumers can branch on engine without `instanceof` + dynamic imports.
|
||||
- `src/core/engine-factory.ts` — Engine factory with dynamic imports (`'pglite'` | `'postgres'`)
|
||||
- `src/core/pglite-engine.ts` — PGLite (embedded Postgres 17.5 via WASM) implementation, all 40 BrainEngine methods. `addLinksBatch` / `addTimelineEntriesBatch` use multi-row `unnest()` with manual `$N` placeholders. As of v0.13.1, `connect()` wraps `PGlite.create()` in a try/catch that emits an actionable error naming the macOS 26.3 WASM bug (#223) and pointing at `gbrain doctor`; the lock is released on failure so the next process can retry cleanly. v0.22.0: `searchKeyword` and `searchKeywordChunks` multiply `ts_rank` by the source-factor CASE expression at the chunk-grain level; `searchVector` becomes a two-stage CTE — inner CTE keeps `ORDER BY cc.embedding <=> vec` so HNSW stays usable, outer SELECT re-ranks by `raw_score * source_factor`. Inner LIMIT scales with offset to preserve pagination contract. As of v0.22.6.1, `initSchema()` calls `applyForwardReferenceBootstrap()` BEFORE replaying SCHEMA_SQL — probes for the specific forward-referenced state the embedded schema blob needs (`pages.source_id`, `links.link_source`, `links.origin_page_id`, `content_chunks.symbol_name`, `content_chunks.language`, `sources` FK target table) and adds only what's missing. Closes the upgrade-wedge bug class that bit users 10+ times across 6 schema versions over 2 years (#239/#243/#266/#357/#366/#374/#375/#378/#395/#396). No-op on fresh installs and modern brains.
|
||||
@@ -166,7 +166,7 @@ strict behavior when unset.
|
||||
- `src/core/minions/wait-for-completion.ts` (v0.15) — poll-until-terminal helper for CLI callers. `TimeoutError` does NOT cancel the job; `AbortSignal` exits without throwing. Default `pollMs`: 1000 on Postgres, 250 on PGLite inline.
|
||||
- `src/core/minions/transcript.ts` (v0.15) — renders `subagent_messages` + `subagent_tool_executions` to markdown. Tool rows splice under their owning assistant `tool_use` by `tool_use_id`. UTF-8-safe truncation; unknown block types fall through to fenced JSON.
|
||||
- `src/core/minions/plugin-loader.ts` (v0.15) — `GBRAIN_PLUGIN_PATH` discovery. Absolute paths only, left-wins collision, `gbrain.plugin.json` with `plugin_version: "gbrain-plugin-v1"`, plugins ship DEFS only (no new tools), `allowed_tools:` validated at load time against the derived registry.
|
||||
- `src/core/minions/tools/brain-allowlist.ts` (v0.15, extended v0.23) — derives subagent tool registry from `src/core/operations.ts`. 11-name allow-list. By default `put_page` schema is namespace-wrapped per subagent (`^wiki/agents/<subagentId>/.+`). **v0.23 trusted-workspace path:** when `BuildBrainToolsOpts.allowedSlugPrefixes` is set, the put_page schema instead describes the prefix list to the model and the OperationContext is threaded with `allowedSlugPrefixes`. Trust comes from `PROTECTED_JOB_NAMES` gating subagent submission — MCP cannot reach this field. Only cycle.ts (synthesize/patterns) and direct CLI submitters set it.
|
||||
- `src/core/minions/tools/brain-allowlist.ts` (v0.15) — derives subagent tool registry from `src/core/operations.ts`. 11-name allow-list: `query`, `search`, `get_page`, `list_pages`, `file_list`, `file_url`, `get_backlinks`, `traverse_graph`, `resolve_slugs`, `get_ingest_log`, `put_page`. `put_page` schema is namespace-wrapped per subagent (`^wiki/agents/<subagentId>/.+`); the `put_page` op's server-side check is the authoritative gate via `ctx.viaSubagent` fail-closed.
|
||||
- `src/mcp/tool-defs.ts` (v0.15) — extracted `buildToolDefs(ops)` helper. MCP server + subagent tool registry both call it; byte-for-byte equivalence pinned by `test/mcp-tool-defs.test.ts`.
|
||||
- `src/core/minions/attachments.ts` — Attachment validation (path traversal, null byte, oversize, base64, duplicate detection)
|
||||
- `src/commands/agent.ts` (v0.16) — `gbrain agent run <prompt> [flags]` CLI. Submits `subagent` (or N children + 1 aggregator) under `{allowProtectedSubmit: true}`. Single-entry `--fanout-manifest` short-circuits. Children get `on_child_fail: 'continue'` + `max_stalled: 3`. `--follow` is the default on TTY; streams logs + polls `waitForCompletion` in parallel. Ctrl-C detaches, does not cancel.
|
||||
@@ -185,20 +185,14 @@ strict behavior when unset.
|
||||
- `src/commands/orphans.ts` — `gbrain orphans [--json] [--count] [--include-pseudo]`: surfaces pages with zero inbound wikilinks, grouped by domain. Auto-generated/raw/pseudo pages filtered by default. Also exposed as `find_orphans` MCP operation. Shipped in v0.12.3 (contributed by @knee5).
|
||||
- `src/commands/integrity.ts` — `gbrain integrity check|auto|review|extract`: bare-tweet detection, dead-link detection, three-bucket repair (auto-repair / review-queue / skip). `scanIntegrity()` is the shared library function called from `gbrain doctor` (sampled at limit=500) and `cmdCheck` (full scan). v0.22.8: batch-load fast path on Postgres uses `SELECT DISTINCT ON (slug)` in a single SQL query to fix the PgBouncer round-trip timeout (60s → ~6s) while preserving `engine.getAllSlugs()`'s `Set<string>` semantics on multi-source brains. Gated by `engine.kind === 'postgres'` at the call site so PGLite never enters batch; fallback `catch` logs at `GBRAIN_DEBUG=1` so real Postgres errors are diagnosable.
|
||||
- `src/commands/doctor.ts` — `gbrain doctor [--json] [--fast] [--fix] [--dry-run] [--index-audit]`: health checks. v0.12.3 added `jsonb_integrity` + `markdown_body_completeness` reliability checks. v0.14.1: `--fix` delegates inlined cross-cutting rules to `> **Convention:** see [path](path).` callouts (pipes DRY violations into `src/core/dry-fix.ts`); `--fix --dry-run` previews without writing. v0.14.2: `schema_version` check fails loudly when `version=0` (migrations never ran — the #218 `bun install -g` signature) and routes users to `gbrain apply-migrations --yes`; new opt-in `--index-audit` flag (Postgres-only) reports zero-scan indexes from `pg_stat_user_indexes` (informational only, no auto-drop). v0.15.2: every DB check is wrapped in a progress phase; `markdown_body_completeness` runs under a 1s heartbeat timer so 10+ min scans are observable on 50K-page brains. v0.19.1 added `queue_health` (Postgres-only) with two subchecks: stalled-forever active jobs (started_at > 1h) and waiting-depth-per-name > threshold (default 10, override via `GBRAIN_QUEUE_WAITING_THRESHOLD`). Worker-heartbeat subcheck intentionally deferred to follow-up B7 because it needs a `minion_workers` table to produce ground-truth signal. Fix hints point at `gbrain repair-jsonb`, `gbrain sync --force`, `gbrain apply-migrations`, and `gbrain jobs get/cancel <id>`. v0.22.12 (#500): `sync_failures` check shows `[CODE=N, ...]` breakdown for both unacked entries (warn) and acked-historical entries (ok), surfacing systemic failure modes (`SLUG_MISMATCH=2685`) instead of a bare count.
|
||||
- `src/core/migrate.ts` — schema-migration runner. Owns the `MIGRATIONS` array (source of truth for schema DDL). v0.14.2 extended the `Migration` interface with `sqlFor?: { postgres?, pglite? }` (engine-specific SQL overrides `sql`) and `transaction?: boolean` (set to false for `CREATE INDEX CONCURRENTLY`, which Postgres refuses inside a transaction; ignored on PGLite since it has no concurrent writers). Migration v14 (fix wave) uses a handler branching on `engine.kind` to run CONCURRENTLY on Postgres (with a pre-drop of any invalid remnant via `pg_index.indisvalid`) and plain `CREATE INDEX` on PGLite. v15 bumps `minion_jobs.max_stalled` default 1→5 and backfills existing non-terminal rows. v0.22.6.1: migration v24 (`rls_backfill_missing_tables`) uses `sqlFor: { pglite: '' }` to no-op on PGLite — PGLite has no RLS engine and is single-tenant by definition, and the v24 ALTERs target subagent tables that don't exist in pglite-schema.ts. Closes #395 (contributed by @jdcastro2). **v30 (v0.23):** creates `dream_verdicts (file_path TEXT, content_hash TEXT, worth_processing BOOL, reasons JSONB, judged_at TIMESTAMPTZ, PK(file_path, content_hash))`. RLS-enabled when running as a BYPASSRLS role. The synthesize phase reads/writes this table to avoid re-judging on backfill re-runs.
|
||||
- `src/core/migrate.ts` — schema-migration runner. Owns the `MIGRATIONS` array (source of truth for schema DDL). v0.14.2 extended the `Migration` interface with `sqlFor?: { postgres?, pglite? }` (engine-specific SQL overrides `sql`) and `transaction?: boolean` (set to false for `CREATE INDEX CONCURRENTLY`, which Postgres refuses inside a transaction; ignored on PGLite since it has no concurrent writers). Migration v14 (fix wave) uses a handler branching on `engine.kind` to run CONCURRENTLY on Postgres (with a pre-drop of any invalid remnant via `pg_index.indisvalid`) and plain `CREATE INDEX` on PGLite. v15 bumps `minion_jobs.max_stalled` default 1→5 and backfills existing non-terminal rows. v0.22.6.1: migration v24 (`rls_backfill_missing_tables`) uses `sqlFor: { pglite: '' }` to no-op on PGLite — PGLite has no RLS engine and is single-tenant by definition, and the v24 ALTERs target subagent tables that don't exist in pglite-schema.ts. Closes #395 (contributed by @jdcastro2).
|
||||
- `src/core/progress.ts` — Shared bulk-action progress reporter. Writes to stderr. Modes: `auto` (TTY: `\r`-rewriting; non-TTY: plain lines), `human`, `json` (JSONL), `quiet`. Rate-gated by `minIntervalMs` and `minItems`. `startHeartbeat(reporter, note)` helper for single long queries. `child()` composes phase paths. Singleton SIGINT/SIGTERM coordinator emits `abort` events for every live phase. EPIPE defense on both sync throws and stream `'error'` events. Zero dependencies. Introduced in v0.15.2.
|
||||
- `src/core/cli-options.ts` — Global CLI flag parser. `parseGlobalFlags(argv)` returns `{cliOpts, rest}` with `--quiet` / `--progress-json` / `--progress-interval=<ms>` stripped. `getCliOptions()` / `setCliOptions()` expose a module-level singleton so commands reach the resolved flags without parameter threading. `cliOptsToProgressOptions()` maps to reporter options. `childGlobalFlags()` returns the flag suffix to append to `execSync('gbrain ...')` calls in migration orchestrators. `OperationContext.cliOpts` extends shared-op dispatch for MCP callers.
|
||||
- `src/core/db-lock.ts` (v0.22.13) — generic `tryAcquireDbLock(engine, lockId, ttlMinutes)` over the existing `gbrain_cycle_locks` table. Parameterized lock id so different scopes can nest cleanly: `gbrain-cycle` for the broad cycle (held by `cycle.ts`) and `gbrain-sync` (`SYNC_LOCK_ID` constant) for `performSync`'s narrower writer window. Same UPSERT-with-TTL semantics as the prior cycle-only helper, just generalized. Survives PgBouncer transaction pooling (unlike session-scoped `pg_try_advisory_lock`); crashed holders auto-release once their TTL expires.
|
||||
- `src/core/sync-concurrency.ts` (v0.22.13) — single source of truth for the parallel-sync policy. Exports `autoConcurrency(engine, fileCount, override?)` (PGLite always serial; explicit override clamped to >=1; auto path returns `DEFAULT_PARALLEL_WORKERS=4` when `fileCount > AUTO_CONCURRENCY_FILE_THRESHOLD=100`), `shouldRunParallel(workers, fileCount, explicit)` (Q1: explicit `--workers` bypasses the >50-file floor), and `parseWorkers(s)` (rejects `'0'`, `'-3'`, `'foo'`, `'1.5'`, trailing chars — replaces the prior parseInt-with-no-validation in both `sync.ts` and `import.ts`). Used by `performSync`, `performFullSync`, `runImport`, and the Minion `sync` handler so the three sites can no longer drift.
|
||||
- `src/commands/sync.ts` — `gbrain sync` CLI + the `performSync` / `performFullSync` library entrypoints (consumed by the autopilot cycle and the Minion sync handler). v0.22.13 (PR #490): `performSync` wraps its body in a `gbrain-sync` writer lock so two concurrent syncs (manual + autopilot, two terminals, two Conductor workspaces) cannot both write `last_commit` and let the last writer win. Head-drift gate after the import phase re-checks `git rev-parse HEAD`; if HEAD moved (someone ran `git checkout` / `git pull` mid-sync), the bookmark refuses to advance. Vanished files now record a failedFiles entry instead of silent-skip — the silent-skip-then-advance pathology that survived prior hardening passes is dead. Worker engines wrap in try/finally so disconnect always fires (panic-path leak fix). Both PGLite-detection sites use `engine.kind === 'pglite'`. CLI accepts `--workers N` (alias `--concurrency N`), validated via `parseWorkers`. Explicit `--workers` bypasses the auto-path file-count floor; auto path defers to `autoConcurrency()`. Banner moved to stderr.
|
||||
- `src/core/cycle.ts` — v0.17 brain maintenance cycle primitive (extended to **8 phases in v0.23**). `runCycle(engine: BrainEngine | null, opts: CycleOpts): Promise<CycleReport>` composes phases in semantically-driven order: **lint → backlinks → sync → synthesize → extract → patterns → embed → orphans**. v0.23's `synthesize` phase runs after sync (cross-references see fresh brain) and before extract (auto-link materializes its writes); `patterns` runs after extract so it reads a fresh graph (codex finding #7 — subagent put_page sets `ctx.remote=true` and skips auto-link/timeline by default; extract is the canonical materialization). Three callers: `gbrain dream` CLI, `gbrain autopilot` daemon's inline path, and the Minions `autopilot-cycle` handler. Coordination via `gbrain_cycle_locks` DB table + `~/.gbrain/cycle.lock` file lock with PID-liveness for PGLite. `CycleReport.schema_version: "1"` is stable; totals additively grew in v0.23 (`transcripts_processed`, `synth_pages_written`, `patterns_written`). `yieldBetweenPhases` runs between phases. **v0.23 added `yieldDuringPhase`** for in-phase keepalive — synthesize/patterns call it during long waits to renew the cycle-lock TTL. Engine nullable; lock-skip on read-only phase selections. v0.22.1 (#403): `CycleOpts.signal?: AbortSignal` propagates the worker's abort signal; `checkAborted()` fires between every phase. v0.22.1 (#417): `runPhaseSync` returns `pagesAffected` via `SyncPhaseResult`; `runCycle` captures it and threads to `runPhaseExtract` as the 4th arg. v0.22.1 (Codex F2): `runPhaseSync` takes `willRunExtractPhase: boolean` and sets `noExtract: phases.includes('extract')` so `gbrain dream --phase sync` doesn't silently lose extraction. v0.22.5 (#475): `resolveSourceForDir(engine, brainDir)` threads `sourceId` to `performSync()` so sync reads the per-source `sources.last_commit` anchor instead of the drift-prone global `config.sync.last_commit` key.
|
||||
- `src/core/cycle/synthesize.ts` (v0.23) — Synthesize phase: conversation-transcript-to-brain pipeline. Reads from `dream.synthesize.session_corpus_dir`, runs cheap Haiku verdict (cached in `dream_verdicts`), then fans out one Sonnet subagent per worth-processing transcript with `allowed_slug_prefixes` (sourced from `skills/_brain-filing-rules.json` `dream_synthesize_paths.globs`). Orchestrator collects slugs from `subagent_tool_executions` (NOT `pages.updated_at` — codex finding #2) and reverse-renders DB → markdown via `serializeMarkdown`. Cooldown via `dream.synthesize.last_completion_ts`, written ONLY on success. Idempotency key `dream:synth:<file_path>:<content_hash>`. Auto-commit deferred to v1.1 (codex #5). `--dry-run` runs Haiku, skips Sonnet (codex #8). Subagent never gets fs-write access.
|
||||
- `src/core/cycle/patterns.ts` (v0.23) — Patterns phase: cross-session theme detection over reflections within `dream.patterns.lookback_days` (default 30). Names a pattern only when ≥`dream.patterns.min_evidence` (default 3) reflections support it. Single Sonnet subagent; same allow-list path as synthesize. Runs AFTER `extract` so the graph is fresh.
|
||||
- `src/core/cycle/transcript-discovery.ts` (v0.23) — Pure filesystem walk for synthesize. `discoverTranscripts(opts)` filters `.txt` files by date range, min_chars, and word-boundary regex `excludePatterns` (Q-3: `medical` matches "medical advice" but NOT "comedical"; power users may pass full regex). `readSingleTranscript(path)` is the `gbrain dream --input <file>` ad-hoc path.
|
||||
- `src/commands/dream.ts` — v0.17 `gbrain dream` CLI; ~80-line thin alias over `runCycle`. brainDir resolution requires explicit `--dir` OR `sync.repo_path` config. Flags: `--dry-run`, `--json`, `--phase <name>`, `--pull`, `--dir <path>`. **v0.23 added** `--input <file>` (ad-hoc transcript, implies `--phase synthesize`), `--date YYYY-MM-DD`, `--from <d> --to <d>` (backfill range). Conflict detection: `--input` + `--date` exits 2. ISO date validation. `--dry-run` runs Haiku significance verdict but skips Sonnet synthesis (codex finding #8 — NOT zero LLM calls). Exit code 1 on status=failed.
|
||||
- `src/commands/friction.ts` + `src/core/friction.ts` (v0.23) — `gbrain friction {log,render,list,summary}` reporter. Append-only JSONL under `$GBRAIN_HOME/friction/<run-id>.jsonl`. Schema is a flat extension of `StructuredAgentError` (D20). Render groups by severity → phase, defaults to `--redact` for md output (strips `$HOME`/`$CWD` to placeholders so reports paste safely in PRs). Run-id resolves from `--run-id` > `$GBRAIN_FRICTION_RUN_ID` > `standalone.jsonl`. Skills the claw-test exercises gain a `_friction-protocol.md` callout so agents know when to log friction.
|
||||
- `src/commands/claw-test.ts` + `src/core/claw-test/` (v0.23) — `gbrain claw-test [--scenario <name>] [--live --agent openclaw]`. End-to-end "fresh user" friction harness. Two modes: scripted (CI gate, agent-free) and live (real openclaw subprocess, $1–2 in tokens). Sets `GBRAIN_HOME=<tempdir>` for hermeticity and captures gbrain's `--progress-json` events from each child's stderr to verify expected phases ran (`import.files`, `extract.links_fs`, `doctor.db_checks`). Phases for scripted mode: setup → install_brain (`gbrain init --pglite`) → import (`--no-embed`) → query → extract → verify (`gbrain doctor --json`, asserts `status: 'ok'`) → render. Live mode hands `BRIEF.md` from `test/fixtures/claw-test-scenarios/<name>/` to the agent runner. v1 ships with the OpenClaw runner only (`src/core/claw-test/runners/openclaw.ts`, invokes `openclaw agent --local --agent <name> --message <brief>`); hermes runner deferred to v1.1. Transcript capture (`transcript-capture.ts`) uses `fs.createWriteStream` with `'drain'`-event backpressure — D17 fix for the 256KB-burst child-stall scenario. v0.18 upgrade scenario seeded via `seed-pglite.ts` SQL replay.
|
||||
- `skills/_friction-protocol.md` (v0.23) — shared cross-cutting convention skill (like `_brain-filing-rules.md`). Tells agents when to call `gbrain friction log` and how to choose a severity. Routes to friction CLI from any skill the claw-test exercises.
|
||||
- `src/core/cycle.ts` — v0.17 brain maintenance cycle primitive. `runCycle(engine: BrainEngine | null, opts: CycleOpts): Promise<CycleReport>` composes 6 phases in semantically-driven order (lint → backlinks → sync → extract → embed → orphans). Three callers: `gbrain dream` CLI, `gbrain autopilot` daemon's inline path, and the Minions `autopilot-cycle` handler (`src/commands/jobs.ts`). One source of truth for what the brain does overnight. Coordination via `gbrain_cycle_locks` DB table (TTL-based; works through PgBouncer transaction pooling, unlike session-scoped `pg_try_advisory_lock`) + `~/.gbrain/cycle.lock` file lock with PID-liveness for PGLite / engine=null mode. `CycleReport.schema_version: "1"` is the stable agent-consumable shape. `PhaseResult.error: { class, code, message, hint?, docs_url? }` is Stripe-API-tier structured failure info. `yieldBetweenPhases` hook awaited between every phase — Minions handler uses this to renew its job lock and prevent v0.14 stall-death regression. Engine nullable: filesystem phases (lint, backlinks) run without DB; DB phases skip with `status: "skipped", reason: "no_database"`. Lock-skip: read-only phase selections (`--phase orphans`) bypass the cycle lock. v0.22.1 (#403): `CycleOpts.signal?: AbortSignal` propagates the worker's abort signal; `checkAborted()` fires between every phase and throws if the signal is aborted (cooperative — can't interrupt a phase mid-execution). v0.22.1 (#417): `runPhaseSync` returns `pagesAffected` via `SyncPhaseResult`; `runCycle` captures it and threads to `runPhaseExtract` as the 4th arg, enabling incremental extract on the cycle path. v0.22.1 (Codex F2): `runPhaseSync` takes `willRunExtractPhase: boolean` and sets `noExtract: phases.includes('extract')` so `gbrain dream --phase sync` doesn't silently lose extraction. v0.22.5 (#475): new `resolveSourceForDir(engine, brainDir)` helper queries `SELECT id FROM sources WHERE local_path = $1 LIMIT 1`; `runPhaseSync` threads result as `sourceId` to `performSync()` so sync reads the per-source `sources.last_commit` anchor instead of the drift-prone global `config.sync.last_commit` key. Bare try/catch lets pre-v0.18 brains fall through to the global key. Closes the prod hang where every autopilot cycle ran a 30-min full reimport because the global anchor commit had been GC'd from git history.
|
||||
- `src/commands/dream.ts` — v0.17 `gbrain dream` CLI. ~80-line thin alias over `runCycle`. brainDir resolution requires explicit `--dir` OR `sync.repo_path` config (no more walk-up-cwd-for-.git footgun). Flags: `--dry-run`, `--json`, `--phase <name>`, `--pull`, `--dir <path>`. Exit code 1 on status=failed (partial/warn not fatal — don't page on warnings).
|
||||
- `scripts/check-progress-to-stdout.sh` — CI guard against regressing to `\r`-on-stdout progress. Wired into `bun run test` via `scripts/check-progress-to-stdout.sh && bun test` in package.json.
|
||||
- `docs/progress-events.md` — Canonical JSON event schema reference. Stable from v0.15.2, additive only.
|
||||
- `src/core/markdown.ts` — Frontmatter parsing + body splitter. `splitBody` requires an explicit timeline sentinel (`<!-- timeline -->`, `--- timeline ---`, or `---` immediately before `## Timeline`/`## History`). Plain `---` in body text is a markdown horizontal rule, not a separator. `inferType` auto-types `/wiki/analysis/` → analysis, `/wiki/guides/` → guide, `/wiki/hardware/` → hardware, `/wiki/architecture/` → architecture, `/writing/` → writing (plus the existing people/companies/deals/etc heuristics).
|
||||
@@ -312,16 +306,6 @@ Key commands added in v0.22.13 (PR #490):
|
||||
- `gbrain sync --workers N` (alias `--concurrency N`) — parallelize the import phase using per-worker Postgres engines (small pool of 2 each) with an atomic queue index. Auto-concurrency: defaults to 4 workers when the diff exceeds 100 files. Smaller diffs stay serial. Explicit `--workers` always wins (even on a 30-file diff). PGLite forces serial regardless. Validation rejects `0`, negatives, non-integers loud (replaces the prior silent fall-through to auto-concurrency).
|
||||
- `gbrain import --workers N` — same `parseWorkers()` validation as sync; same try/finally worker-engine cleanup. Behavior surface unchanged.
|
||||
|
||||
Key commands added in v0.22.16 (claw-test friction loop):
|
||||
- `gbrain claw-test [--scenario fresh-install|upgrade-from-v0.18] [--keep-tempdir]` — scripted-mode CI gate that runs the full canonical first-day flow against a fresh tempdir. Asserts every expected `--progress-json` phase fired and doctor's `status === 'ok'`. ~30s, no API keys.
|
||||
- `gbrain claw-test --live --agent openclaw` — friction-discovery mode. Spawns real openclaw, hands it `BRIEF.md`, captures stdin/stdout/stderr to `<run>/transcript.jsonl`, lets the agent log friction via the friction CLI. Run on demand; ~5–10 min and ~$1–2 in tokens.
|
||||
- `gbrain claw-test --list-agents` — reports which agent runners are registered + their detection state (binary path or unavailable reason).
|
||||
- `gbrain friction log --severity {confused|error|blocker|nit} --phase <name> --message <text> [--hint ...] [--kind {friction|delight}] [--run-id ...]` — append a friction or delight entry to the active run JSONL.
|
||||
- `gbrain friction render --run-id <id> [--json] [--transcripts] [--no-redact]` — markdown report grouped by severity + phase; `--redact` is the default for md output (strips `$HOME`/`$CWD` placeholders so reports paste safely in PRs/issues).
|
||||
- `gbrain friction list [--json]` — recent run-ids with friction/delight counts; interrupted runs marked `(interrupted)`.
|
||||
- `gbrain friction summary --run-id <id> [--json]` — two-column friction + delight summary.
|
||||
- `GBRAIN_HOME` env override is now honored uniformly across every gbrain write site (config, audit, friction, sync-failures, import checkpoint, integrity log, integrations heartbeat, migration rollback, etc.) — `gbrainPath(...)` from `src/core/config.ts` is the canonical helper. Read-side host-fingerprint detection (`~/.claude`/`~/.openclaw` etc.) intentionally NOT confined in v1; that's a v1.1 follow-up.
|
||||
|
||||
## Testing
|
||||
|
||||
`bun test` runs all tests. After the v0.12.1 release: ~75 unit test files + 8 E2E test files (1412 unit pass, 119 E2E when `DATABASE_URL` is set — skip gracefully otherwise). Unit tests run
|
||||
@@ -1151,9 +1135,8 @@ Set up using your platform's scheduler (OpenClaw cron, Railway cron, crontab):
|
||||
- **Live sync** (every 15 min): `gbrain sync --repo ~/brain && gbrain embed --stale`
|
||||
- **Auto-update** (daily): `gbrain check-update --json` (tell user, never auto-install)
|
||||
- **Dream cycle** (nightly): read `docs/guides/cron-schedule.md` for the full protocol.
|
||||
Entity sweep, citation fixes, memory consolidation, plus (v0.23+) overnight conversation
|
||||
synthesis and cross-session pattern detection. 8 phases, one cron-friendly command. This
|
||||
is what makes the brain compound. Do not skip it.
|
||||
Entity sweep, citation fixes, memory consolidation. This is what makes the brain
|
||||
compound. Do not skip it.
|
||||
- **Weekly**: `gbrain doctor --json && gbrain embed --stale`
|
||||
|
||||
## Step 8: Integrations
|
||||
@@ -1270,7 +1253,6 @@ This is the dispatcher. Skills are the implementation. **Read the skill file bef
|
||||
| "Migrate from Obsidian/Notion/Logseq" | `skills/migrate/SKILL.md` |
|
||||
| Brain health check, maintenance run | `skills/maintain/SKILL.md` |
|
||||
| "Extract links", "build link graph", "populate timeline" | `skills/maintain/SKILL.md` (extraction sections) |
|
||||
| "Run dream", "process today's session", "synthesize my conversations", "consolidate yesterday's conversations", "what patterns did you see", "did the dream cycle run" | `skills/maintain/SKILL.md` (dream cycle section) |
|
||||
| "Brain health", "what features am I missing", "brain score" | Run `gbrain features --json` |
|
||||
| "Set up autopilot", "run brain maintenance", "keep brain updated" | Run `gbrain autopilot --install --repo ~/brain` |
|
||||
| Agent identity, "who am I", customize agent | `skills/soul-audit/SKILL.md` |
|
||||
@@ -1444,7 +1426,7 @@ GBrain ships 29 skills organized by `skills/RESOLVER.md` (or your OpenClaw's `AG
|
||||
|-------|-------------|
|
||||
| **enrich** | Tiered enrichment (Tier 1/2/3). Creates and updates person/company pages with compiled truth and timelines. |
|
||||
| **query** | 3-layer search with synthesis and citations. Says "the brain doesn't have info on X" instead of hallucinating. |
|
||||
| **maintain** | Periodic health: stale pages, orphans, dead links, citation audit, back-link enforcement, tag consistency. v0.23 adds the dream cycle's synthesize + patterns phases ... overnight conversation transcripts become reflections, originals, and 25-year patterns. |
|
||||
| **maintain** | Periodic health: stale pages, orphans, dead links, citation audit, back-link enforcement, tag consistency. |
|
||||
| **citation-fixer** | Scans pages for missing or malformed citations. Fixes format to match the standard. |
|
||||
| **repo-architecture** | Where new brain files go. Decision protocol: primary subject determines directory, not format. |
|
||||
| **publish** | Share brain pages as password-protected HTML. Zero LLM calls. |
|
||||
@@ -2001,11 +1983,7 @@ ADMIN
|
||||
gbrain auth create|list|revoke|test Token management for the HTTP transport
|
||||
gbrain integrations Integration recipe dashboard
|
||||
gbrain sources list|add|remove|... Multi-source brain management (v0.18)
|
||||
gbrain dream [--dry-run] [--phase N] 8-phase maintenance cycle (lint→backlinks→sync→synthesize
|
||||
→extract→patterns→embed→orphans). v0.23 added synthesize +
|
||||
patterns: transcripts → reflections + cross-session themes.
|
||||
gbrain dream --input <file> Ad-hoc transcript synthesis (implies --phase synthesize)
|
||||
gbrain dream --date YYYY-MM-DD Synthesize a single day; --from/--to for backfill ranges
|
||||
gbrain dream [--dry-run] [--phase N] One maintenance cycle then exit (cron-friendly)
|
||||
gbrain check-backlinks check|fix Back-link enforcement
|
||||
gbrain lint [--fix] LLM artifact detection
|
||||
gbrain repair-jsonb [--dry-run] Repair v0.12.0 double-encoded JSONB (Postgres)
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "gbrain",
|
||||
"version": "0.23.0",
|
||||
"version": "0.22.13",
|
||||
"description": "Postgres-native personal knowledge brain with hybrid RAG search",
|
||||
"type": "module",
|
||||
"main": "src/core/index.ts",
|
||||
|
||||
@@ -70,7 +70,6 @@ This is the dispatcher. Skills are the implementation. **Read the skill file bef
|
||||
| "Migrate from Obsidian/Notion/Logseq" | `skills/migrate/SKILL.md` |
|
||||
| Brain health check, maintenance run | `skills/maintain/SKILL.md` |
|
||||
| "Extract links", "build link graph", "populate timeline" | `skills/maintain/SKILL.md` (extraction sections) |
|
||||
| "Run dream", "process today's session", "synthesize my conversations", "consolidate yesterday's conversations", "what patterns did you see", "did the dream cycle run" | `skills/maintain/SKILL.md` (dream cycle section) |
|
||||
| "Brain health", "what features am I missing", "brain score" | Run `gbrain features --json` |
|
||||
| "Set up autopilot", "run brain maintenance", "keep brain updated" | Run `gbrain autopilot --install --repo ~/brain` |
|
||||
| Agent identity, "who am I", customize agent | `skills/soul-audit/SKILL.md` |
|
||||
|
||||
@@ -97,15 +97,5 @@
|
||||
"The PRIMARY SUBJECT of the content determines the directory, not the format or source skill.",
|
||||
"When in doubt: what would you search for to find this page again?",
|
||||
"Cross-link from related directories via back-links — do not duplicate content."
|
||||
],
|
||||
"dream_synthesize_paths": {
|
||||
"description": "Single source of truth for the v0.23 dream-cycle synthesize/patterns trusted-workspace allow-list. The cycle's synthesize phase reads this list and threads it as `allowed_slug_prefixes` to every subagent it dispatches; put_page enforces it server-side. Editing this list is the ONLY way to add a new directory the synthesis subagent may write to.",
|
||||
"globs": [
|
||||
"wiki/personal/reflections/*",
|
||||
"wiki/originals/*",
|
||||
"wiki/personal/patterns/*",
|
||||
"wiki/people/*",
|
||||
"dream-cycle-summaries/*"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -112,24 +112,3 @@ gbrain files restore <dir> # Download back to local
|
||||
|
||||
This ensures any derived brain page can be traced back to its original source,
|
||||
and large files don't bloat the git repo.
|
||||
|
||||
## Dream-cycle synthesize / patterns directories (v0.23)
|
||||
|
||||
The `synthesize` and `patterns` phases of `gbrain dream` write to a
|
||||
**fixed allow-list** of paths sourced from `_brain-filing-rules.json`'s
|
||||
`dream_synthesize_paths.globs` array. Editing that JSON is the ONLY way
|
||||
to add a new directory the synthesis subagent may write to:
|
||||
|
||||
| Output type | Slug pattern | What goes here |
|
||||
|-------------|--------------|----------------|
|
||||
| Reflection | `wiki/personal/reflections/YYYY-MM-DD-<topic>-<hash[:6]>` | Self-knowledge, emotional processing, pattern recognition. Verbatim quotes from the user, with analysis. |
|
||||
| Original idea | `wiki/originals/ideas/YYYY-MM-DD-<idea>-<hash[:6]>` | New frames, theses, mental models, "conceptive ideologist" outputs. Capture the user's exact phrasing — that's the artifact. |
|
||||
| People enrichment | `wiki/people/<existing-slug>` | Timeline entries appended to existing people pages from session mentions. Stub pages for new substantive people. |
|
||||
| Pattern | `wiki/personal/patterns/<theme>` | Cross-session theme detected across ≥3 reflections. Highest-leverage output: a pattern can span 25 years if reflections reference dated content. |
|
||||
| Cycle summary | `dream-cycle-summaries/YYYY-MM-DD` | Index of every page produced by one dream cycle. Auto-written deterministically by the orchestrator. |
|
||||
|
||||
**Iron Law for synthesize output:**
|
||||
1. Quote the user verbatim. Do not paraphrase memorable phrasings.
|
||||
2. Cross-reference compulsively: every new page MUST link to existing brain content.
|
||||
3. Slug discipline: lowercase alphanumeric and hyphens only, slash-separated. NO underscores, NO file extensions.
|
||||
4. Edited transcripts produce NEW slugs (content-hash suffix changes) — never silently overwrite a prior reflection.
|
||||
|
||||
@@ -1,60 +0,0 @@
|
||||
# Friction protocol — convention
|
||||
|
||||
> Cross-cutting rule shared by skills the claw-test harness exercises (setup,
|
||||
> brain-ops, query, ingest, smoke-test, migrations). Reference via
|
||||
> `> **Convention:** see [skills/_friction-protocol.md](_friction-protocol.md).`
|
||||
|
||||
When you encounter friction running gbrain — anything confusing, missing, surprising, or wrong — log it via `gbrain friction log` so maintainers can see it without you writing a bug report. Friction reports drive the claw-test feedback loop (the harness collects, renders, and re-runs).
|
||||
|
||||
## When to log
|
||||
|
||||
Log friction when any of these happens:
|
||||
|
||||
- A command failed with a non-actionable error message
|
||||
- A doc said one thing and the tool did another
|
||||
- You couldn't find the next step
|
||||
- A setup command needed a manual workaround
|
||||
- A flag exists but isn't documented in `--help`
|
||||
- A success condition was unclear (you couldn't tell if the command worked)
|
||||
|
||||
Log delight (positive signal) when:
|
||||
|
||||
- Something worked on the first try and the docs were exactly right
|
||||
- An error message handed you the fix
|
||||
- A flag you guessed at turned out to exist with the obvious name
|
||||
|
||||
## How to log
|
||||
|
||||
```
|
||||
gbrain friction log \
|
||||
--severity {confused|error|blocker|nit} \
|
||||
--phase <which-phase-or-command> \
|
||||
--message "<one-line-what-happened>" \
|
||||
[--hint "<one-line-what-could-be-better>"]
|
||||
```
|
||||
|
||||
For delight, add `--kind delight` and pick any severity.
|
||||
|
||||
The CLI auto-fills `ts`, `cwd`, `gbrain_version`, and resolves `run_id` from `$GBRAIN_FRICTION_RUN_ID` (set by the harness) or falls back to `standalone.jsonl`. So you can call this anywhere — inside a harness run, manually during normal use, or from a scripted test.
|
||||
|
||||
## Severity guide
|
||||
|
||||
| severity | meaning |
|
||||
|------------|---------|
|
||||
| `blocker` | Couldn't proceed at all. Hard stop. |
|
||||
| `error` | Command failed unexpectedly. |
|
||||
| `confused` | Docs/tool mismatch, ambiguity, missing pointer. |
|
||||
| `nit` | Polish opportunity. Cosmetic or low-impact. |
|
||||
|
||||
Be specific: "doctor says `schema_version=0` and points at apply-migrations, but apply-migrations exits 0 with no output" beats "doctor was confusing."
|
||||
|
||||
## Inspecting reports
|
||||
|
||||
```
|
||||
gbrain friction list # recent runs with counts
|
||||
gbrain friction render --run-id <id> # markdown report (default)
|
||||
gbrain friction render --run-id <id> --json
|
||||
gbrain friction summary --run-id <id> # friction + delight side-by-side
|
||||
```
|
||||
|
||||
`render` defaults to `--redact` for markdown (strips `$HOME`/`$CWD` to `<HOME>`/`<CWD>` placeholders) so reports paste safely into PRs and issues.
|
||||
@@ -17,13 +17,6 @@ triggers:
|
||||
- "populate links"
|
||||
- "backfill graph"
|
||||
- "extract timeline entries"
|
||||
- "run dream"
|
||||
- "process today's session"
|
||||
- "process yesterday's transcripts"
|
||||
- "synthesize my conversations"
|
||||
- "what patterns did you see"
|
||||
- "did the dream cycle run"
|
||||
- "consolidate yesterday's conversations"
|
||||
tools:
|
||||
- get_health
|
||||
- get_page
|
||||
@@ -84,81 +77,6 @@ If timeline_entry_count is 0, extract structured timeline from markdown:
|
||||
```bash
|
||||
gbrain extract timeline --dir ~/brain
|
||||
```
|
||||
|
||||
### Dream cycle (v0.23): synthesize + patterns
|
||||
|
||||
`gbrain dream` runs the full 8-phase maintenance cycle:
|
||||
|
||||
```
|
||||
lint -> backlinks -> sync -> synthesize -> extract -> patterns -> embed -> orphans
|
||||
```
|
||||
|
||||
The two new phases consolidate yesterday's conversations into long-term memory:
|
||||
|
||||
**Synthesize phase:** reads transcripts from `dream.synthesize.session_corpus_dir`,
|
||||
runs a cheap Haiku verdict (cached in `dream_verdicts`) to filter routine
|
||||
ops sessions, then fans out one Sonnet subagent per worth-processing
|
||||
transcript. Each subagent writes reflections (`wiki/personal/reflections/...`),
|
||||
originals (`wiki/originals/ideas/...`), and people timeline entries. The
|
||||
orchestrator collects the slugs from `subagent_tool_executions` (NOT
|
||||
`pages.updated_at` — that would pick up unrelated writes) and reverse-renders
|
||||
each new page from DB → markdown on disk.
|
||||
|
||||
**Patterns phase:** runs after `extract` (so the graph state is fresh).
|
||||
Reads recent reflections within `dream.patterns.lookback_days` (default 30),
|
||||
runs a single Sonnet pass to surface recurring themes, and writes pattern
|
||||
pages to `wiki/personal/patterns/<theme>` when ≥`dream.patterns.min_evidence`
|
||||
(default 3) reflections support a pattern.
|
||||
|
||||
**Quality bar (Iron Law for synthesis):**
|
||||
1. Quote the user verbatim. Do not paraphrase memorable phrasings.
|
||||
2. Cross-reference compulsively: every new page MUST have at least one wikilink.
|
||||
3. Slug discipline: lowercase alphanumeric and hyphens only. NO underscores, NO file extensions.
|
||||
4. Edited transcripts produce NEW slugs (content-hash suffix changes) — never silently overwrite.
|
||||
|
||||
**Trust boundary (`allowed_slug_prefixes`):** the synthesis subagent runs with an
|
||||
explicit allow-list of write paths sourced from `_brain-filing-rules.json`'s
|
||||
`dream_synthesize_paths.globs`. Even on prompt-injection success, the subagent
|
||||
cannot write outside that list. Trust comes from PROTECTED_JOB_NAMES — MCP
|
||||
cannot submit subagent jobs at all. Editing the JSON is the only way to add
|
||||
a new directory the synthesizer can write to.
|
||||
|
||||
**Idempotency + privacy:** transcripts are keyed by `(file_path, content_hash)`,
|
||||
so re-running on the same content is a no-op. `dream.synthesize.exclude_patterns`
|
||||
(default `["medical", "therapy"]`) filters out transcripts before any LLM call.
|
||||
Each entry is auto-wrapped as a word-boundary regex (e.g. `medical` matches
|
||||
"medical advice" but NOT "comedical"). Power users may pass full regex.
|
||||
|
||||
**Cooldown:** the cycle's spend cap. `dream.synthesize.cooldown_hours` (default
|
||||
12) means at most ~2 synthesize runs per day under autopilot. The completion
|
||||
timestamp is stored in `dream.synthesize.last_completion_ts` and is written
|
||||
ONLY on successful runs (not on skipped/failed). Explicit `--input` /
|
||||
`--date` / `--from` / `--to` invocations bypass cooldown.
|
||||
|
||||
**`--dry-run` semantics:** runs the cheap Haiku significance filter (caches
|
||||
verdicts) but skips the Sonnet synthesis pass. NOT zero LLM calls.
|
||||
|
||||
**Configure synthesize on a fresh brain:**
|
||||
```bash
|
||||
gbrain config set dream.synthesize.session_corpus_dir /path/to/transcripts
|
||||
gbrain config set dream.synthesize.enabled true
|
||||
gbrain dream --phase synthesize --dry-run --json # preview
|
||||
gbrain dream # full 8-phase cycle
|
||||
```
|
||||
|
||||
**Invocation patterns:**
|
||||
```bash
|
||||
gbrain dream # full cycle
|
||||
gbrain dream --phase synthesize # just synthesize
|
||||
gbrain dream --phase patterns # just patterns
|
||||
gbrain dream --input ~/transcripts/2026-04-25.txt # ad-hoc one transcript
|
||||
gbrain dream --from 2026-04-01 --to 2026-04-25 # backfill range
|
||||
gbrain dream --json # CycleReport JSON
|
||||
```
|
||||
|
||||
**Auto-commit deferred to v1.1:** v1 writes files to `brain_dir` but does NOT
|
||||
`git add` / `commit` / `push`. Either commit yourself or let `gbrain autopilot`
|
||||
handle it.
|
||||
Parses `- **YYYY-MM-DD** | Source — Summary` and `### YYYY-MM-DD — Title` formats.
|
||||
Note: extracted entries improve structured queries (`gbrain timeline`), not vector search.
|
||||
|
||||
|
||||
@@ -1,210 +0,0 @@
|
||||
---
|
||||
feature_pitch:
|
||||
headline: Bare workers now self-monitor and fail-stop into your PM's restart loop
|
||||
body: |
|
||||
Bare `gbrain jobs work` now ships with the same health protection the
|
||||
supervisor already had: DB liveness probes (with per-probe timeout so a
|
||||
hung connection can't wedge the monitor), stall detection filtered by
|
||||
registered handler names, and an RSS watchdog default of 2048 MB.
|
||||
|
||||
When the worker detects it's wedged (stuck pgbouncer connection, hung
|
||||
event loop, stalled job claim), it emits `'unhealthy'` and the CLI calls
|
||||
`process.exit(1)`. This is **fail-stop**: it requires an external process
|
||||
manager (systemd, Docker `restart: always`, launchd `KeepAlive`, cron
|
||||
watchdog) to bring the worker back. Without one, the process exits and
|
||||
stays dead — that's a regression from pre-v0.22.14 self-healing.
|
||||
|
||||
Pre-v0.22.14 behavior: bare workers had ZERO health monitoring. A wedged
|
||||
worker stayed alive doing nothing while jobs piled up in `waiting` and
|
||||
your PM's `pgrep` check happily reported green.
|
||||
|
||||
If you're using `gbrain jobs supervisor`, you're already protected — the
|
||||
supervisor handles spawn-on-crash itself. The fail-stop concern only
|
||||
applies to direct `gbrain jobs work` invocations.
|
||||
---
|
||||
|
||||
# v0.22.14 — Bare-worker self-health-monitoring
|
||||
|
||||
## ⚠️ Pre-flight: confirm you have a process supervisor
|
||||
|
||||
If you run `gbrain jobs work` directly (NOT under `gbrain jobs supervisor`),
|
||||
verify your process manager is configured to restart the worker on exit
|
||||
BEFORE upgrading:
|
||||
|
||||
| Manager | What to check |
|
||||
|---|---|
|
||||
| systemd | `Restart=always` (or `Restart=on-failure`) in the `.service` unit |
|
||||
| Docker | `restart: always` / `restart: unless-stopped` in compose, OR `--restart` flag |
|
||||
| launchd (macOS) | `<key>KeepAlive</key><true/>` in the plist |
|
||||
| cron watchdog | Cron entry that re-spawns when `pgrep -f "gbrain jobs work"` is empty |
|
||||
| supervisord | `autorestart=true` |
|
||||
|
||||
**If your bare worker has no restart loop, the v0.22.14 fail-stop behavior
|
||||
will leave you with a dead worker after the first DB blip.** Either add a
|
||||
restart policy OR switch to `gbrain jobs supervisor` (which spawns its own
|
||||
child + restarts on crash internally).
|
||||
|
||||
## What ships
|
||||
|
||||
- DB liveness probes inside `gbrain jobs work` (60s interval, 3 strikes → exit)
|
||||
- Stall detection (5min warn / 10min exit when waiting jobs accumulate but
|
||||
in-flight is empty)
|
||||
- `--max-rss` defaults to 2048 MB for bare workers (matches supervisor default;
|
||||
was 0 = disabled)
|
||||
- New `MinionWorkerOpts.{healthCheckInterval, stallWarnAfterMs,
|
||||
stallExitAfterMs, dbFailExitAfter, dbProbeTimeoutMs}` for tuning (5 fields)
|
||||
- `MinionWorker` now extends `EventEmitter`; emits `'unhealthy'` event with
|
||||
a structured reason payload. **No-listener fallback**: if the caller does
|
||||
not subscribe to `'unhealthy'`, the worker calls `process.exit(1)` itself
|
||||
to preserve the pre-refactor fail-stop behavior. The CLI subscribes; direct
|
||||
API consumers without a listener inherit the fail-stop default. Inline
|
||||
paths (`jobs submit --follow`, `jobs smoke`) explicitly pass
|
||||
`healthCheckInterval: 0` to disable the timer entirely so the fallback
|
||||
cannot trip on a one-shot run.
|
||||
- `GBRAIN_SUPERVISED=1` env var (set by supervisor on its child) causes the
|
||||
child worker to skip its own health timer (no double-monitoring)
|
||||
- Constructor validation: throws if `stallExitAfterMs <= stallWarnAfterMs`
|
||||
(the contract requires strict greater-than)
|
||||
|
||||
## ⚠️ Breaking change you may need to act on
|
||||
|
||||
**`gbrain jobs work` now defaults `--max-rss 2048` (MB).** Pre-v0.22.14, bare
|
||||
workers had no RSS limit. If you ran bare workers with intentionally large
|
||||
embed/import working sets, your worker may now get killed by the watchdog after
|
||||
upgrade.
|
||||
|
||||
**You need to act if any of the following is true:**
|
||||
|
||||
- You run `gbrain jobs work` directly under a process manager (systemd, Docker,
|
||||
cron, supervisord, your own bash wrapper)
|
||||
- Your worker handles large embed batches, multi-GB import jobs, or
|
||||
memory-intensive subagent tasks that legitimately exceed 2 GB RSS
|
||||
- You rely on bare-worker resident memory exceeding 2 GB
|
||||
|
||||
**You do NOT need to act if:**
|
||||
|
||||
- You run `gbrain jobs supervisor` (the supervisor already defaulted to
|
||||
`--max-rss 2048` and has reconnect-then-fail logic on top)
|
||||
- Your typical worker stays well under 2 GB RSS
|
||||
|
||||
## What the agent should do post-upgrade
|
||||
|
||||
### 1. Run the orchestrator (mechanical migration)
|
||||
|
||||
```bash
|
||||
gbrain apply-migrations --yes
|
||||
```
|
||||
|
||||
This is a no-op for v0.22.14 — there's no schema migration. The mechanical work
|
||||
is in the binary; the user-side decision is the breaking-change handling below.
|
||||
|
||||
### 2. Detect bare-worker deployments
|
||||
|
||||
If the user runs OpenClaw, Hermes, or any agent fork that uses `gbrain jobs
|
||||
work` directly (NOT under `gbrain jobs supervisor`), they need to know. Check:
|
||||
|
||||
```bash
|
||||
# Look for bare worker invocations in cron, systemd, Docker, launchctl, etc.
|
||||
grep -r "gbrain jobs work" ~/.config/systemd/ /etc/systemd/ 2>/dev/null | head
|
||||
crontab -l 2>/dev/null | grep "gbrain jobs work" | head
|
||||
launchctl list 2>/dev/null | grep gbrain | head
|
||||
ps -ef | grep "gbrain jobs work" | grep -v grep | grep -v supervisor
|
||||
```
|
||||
|
||||
If you find bare-worker invocations, surface them to the user.
|
||||
|
||||
### 3. Ask about RSS profile
|
||||
|
||||
Use AskUserQuestion to confirm whether the user's bare workers need >2 GB RSS:
|
||||
|
||||
> "v0.22.14 changes `gbrain jobs work` to default `--max-rss 2048` (MB).
|
||||
> Pre-v0.22.14 bare workers had no limit. If your worker handles large embed
|
||||
> batches or multi-GB imports, the watchdog may now kill it. Do you want
|
||||
> to keep the new 2 GB default, raise the limit, or opt out entirely?"
|
||||
|
||||
Options:
|
||||
- **A) Keep 2 GB default (recommended for most)** — protects against memory
|
||||
leaks; restarts on overflow; matches supervisor behavior.
|
||||
- **B) Raise to N GB (specify N)** — pass `--max-rss <N*1024>` to the worker
|
||||
invocation.
|
||||
- **C) Opt out** — pass `--max-rss 0`.
|
||||
|
||||
### 4. Apply the user's choice
|
||||
|
||||
For each bare-worker invocation, edit the unit/cron/launchctl/script to add
|
||||
the chosen `--max-rss` flag.
|
||||
|
||||
**systemd (~/.config/systemd/user/gbrain-worker.service):**
|
||||
|
||||
```ini
|
||||
ExecStart=/usr/local/bin/gbrain jobs work --queue default --concurrency 3 --max-rss 4096
|
||||
# Or to opt out: --max-rss 0
|
||||
```
|
||||
|
||||
Then `systemctl --user daemon-reload && systemctl --user restart gbrain-worker`.
|
||||
|
||||
**cron (`crontab -e`):**
|
||||
|
||||
```cron
|
||||
@reboot /usr/local/bin/gbrain jobs work --queue default --concurrency 3 --max-rss 4096
|
||||
```
|
||||
|
||||
**Docker compose:**
|
||||
|
||||
```yaml
|
||||
command: ["gbrain", "jobs", "work", "--queue", "default", "--concurrency", "3", "--max-rss", "4096"]
|
||||
```
|
||||
|
||||
**launchctl (~/Library/LaunchAgents/com.user.gbrain-worker.plist):**
|
||||
|
||||
```xml
|
||||
<key>ProgramArguments</key>
|
||||
<array>
|
||||
<string>/usr/local/bin/gbrain</string>
|
||||
<string>jobs</string>
|
||||
<string>work</string>
|
||||
<string>--max-rss</string>
|
||||
<string>4096</string>
|
||||
</array>
|
||||
```
|
||||
|
||||
Then `launchctl unload ... && launchctl load ...`.
|
||||
|
||||
### 5. (Optional) Tune health-check thresholds
|
||||
|
||||
The new opts default to sensible values (60s probe interval, 5min warn / 10min
|
||||
exit, 3 DB failures). If you have specific SLAs, you can pass `--health-interval
|
||||
<ms>` to adjust the probe cadence. Stall thresholds are not yet CLI-exposed
|
||||
(only the API; CLI flags coming in a follow-up).
|
||||
|
||||
To disable self-monitoring entirely (e.g. you have your own external health
|
||||
checker):
|
||||
|
||||
```bash
|
||||
gbrain jobs work --health-interval 0 --max-rss 0
|
||||
```
|
||||
|
||||
### 6. Verify
|
||||
|
||||
```bash
|
||||
gbrain jobs stats # queue should be flowing normally
|
||||
gbrain doctor --json | jq '.' # no critical warnings
|
||||
ps -o rss= -p $(pgrep -f "gbrain jobs work") | awk '{print $1/1024 " MB"}'
|
||||
```
|
||||
|
||||
Worker startup log line should now show health-check status:
|
||||
|
||||
```
|
||||
Minion worker started (queue: default, concurrency: 3, watchdog: 2048MB, health-check: 60s)
|
||||
```
|
||||
|
||||
If running under supervisor, you'll see the watchdog but NOT the `health-check:
|
||||
60s` segment (because `GBRAIN_SUPERVISED=1` skips the child's self-monitor).
|
||||
|
||||
### 7. If anything fails
|
||||
|
||||
Open an issue at https://github.com/garrytan/gbrain/issues with:
|
||||
- Output of `gbrain doctor`
|
||||
- Contents of `~/.gbrain/upgrade-errors.jsonl` if it exists
|
||||
- Your bare-worker invocation (systemd unit / cron line / Dockerfile snippet)
|
||||
- Which step broke
|
||||
@@ -1,171 +0,0 @@
|
||||
---
|
||||
version: 0.23.0
|
||||
feature_pitch:
|
||||
headline: "gbrain dream now actually dreams: conversation transcripts → reflections, originals, and 25-year patterns."
|
||||
description: |
|
||||
The maintenance cycle gains two new phases: `synthesize` and `patterns`.
|
||||
The 8-phase order is now: lint → backlinks → sync → synthesize →
|
||||
extract → patterns → embed → orphans.
|
||||
|
||||
Synthesize reads conversation transcripts (e.g., OpenClaw session corpus,
|
||||
meeting transcripts) and writes brain-native pages: reflections to
|
||||
`wiki/personal/reflections/...`, originals to `wiki/originals/ideas/...`,
|
||||
timeline entries on existing people pages.
|
||||
|
||||
Patterns runs after extract (so the graph is fresh) and surfaces
|
||||
recurring themes across reflections — when ≥3 reflections mention the
|
||||
same motif, a pattern page is written to `wiki/personal/patterns/...`
|
||||
citing every reflection that constitutes its evidence.
|
||||
|
||||
Hard guarantees: subagent writes are bounded to an explicit allow-list
|
||||
(sourced from `_brain-filing-rules.json`). Edited transcripts produce
|
||||
new slugs (content-hash suffix) — never silently overwrite. A 12-hour
|
||||
cooldown bounds spend at ~$1-2/day under autopilot.
|
||||
recipe: skills/maintain/SKILL.md
|
||||
tiers: null
|
||||
---
|
||||
|
||||
# v0.23.0 Migration: Dream cycle synthesize + patterns phases
|
||||
|
||||
**Audience: host agents reading this after `gbrain apply-migrations` has
|
||||
run. The synthesize phase ships disabled by default — set
|
||||
`dream.synthesize.session_corpus_dir` to opt in.**
|
||||
|
||||
## Mechanical migration: automatic, no action required
|
||||
|
||||
`gbrain upgrade` chains to `gbrain apply-migrations --yes`, which runs:
|
||||
|
||||
- **migration v25** — creates the `dream_verdicts` table:
|
||||
`(file_path TEXT, content_hash TEXT, worth_processing BOOL, reasons JSONB,
|
||||
judged_at TIMESTAMPTZ, PRIMARY KEY(file_path, content_hash))`. Cache
|
||||
for the cheap Haiku verdict so backfill re-runs skip already-judged
|
||||
transcripts. RLS-enabled when running as a BYPASSRLS role.
|
||||
|
||||
The migration is idempotent. Safe to re-run.
|
||||
|
||||
## What changes for existing brains
|
||||
|
||||
`gbrain dream` (and `gbrain autopilot`) now run an 8-phase cycle:
|
||||
|
||||
```
|
||||
lint → backlinks → sync → synthesize → extract → patterns → embed → orphans
|
||||
```
|
||||
|
||||
If `dream.synthesize.enabled` is false (the default, post-migration), the
|
||||
synthesize and patterns phases emit `status: "skipped", reason: "not_configured"`
|
||||
and the cycle continues to the next phase. **Existing autopilot users see
|
||||
zero behavior change** until they configure synthesize.
|
||||
|
||||
## To enable synthesize on your brain
|
||||
|
||||
Three steps. Take them when ready — there is no rush.
|
||||
|
||||
```bash
|
||||
# 1. Point at the directory where your conversation transcripts live.
|
||||
# OpenClaw stores session transcripts at memory/.dreams/session-corpus/<YYYY-MM-DD>.txt
|
||||
# by default. If you have a different layout, point at that.
|
||||
gbrain config set dream.synthesize.session_corpus_dir /path/to/transcripts
|
||||
|
||||
# 2. Enable the phase.
|
||||
gbrain config set dream.synthesize.enabled true
|
||||
|
||||
# 3. Preview without spending real LLM tokens (runs cheap Haiku verdict only).
|
||||
gbrain dream --phase synthesize --dry-run --json
|
||||
```
|
||||
|
||||
## Tunables (sensible defaults; override only if needed)
|
||||
|
||||
```bash
|
||||
# Skip transcripts shorter than this many characters (default 2000).
|
||||
gbrain config set dream.synthesize.min_chars 2000
|
||||
|
||||
# Word-boundary regex patterns to skip. Default ["medical","therapy"].
|
||||
# Each entry auto-wraps as \b<entry>\b — "medical" matches "medical advice"
|
||||
# but NOT "comedical". Pass full regex (e.g. ^therapy:) for advanced patterns.
|
||||
gbrain config set dream.synthesize.exclude_patterns '["medical","therapy"]'
|
||||
|
||||
# Synthesize model (default: claude-sonnet-4-6).
|
||||
gbrain config set dream.synthesize.model claude-sonnet-4-6
|
||||
|
||||
# Hours between synthesize runs (the v1 spend cap; default 12 → ~$1-2/day).
|
||||
gbrain config set dream.synthesize.cooldown_hours 12
|
||||
|
||||
# Patterns lookback window in days (default 30).
|
||||
gbrain config set dream.patterns.lookback_days 30
|
||||
|
||||
# Minimum distinct reflections needed to name a pattern (default 3).
|
||||
gbrain config set dream.patterns.min_evidence 3
|
||||
```
|
||||
|
||||
## Allow-list source of truth
|
||||
|
||||
The synthesize subagent's allowed write paths live in
|
||||
`skills/_brain-filing-rules.json` under `dream_synthesize_paths.globs`:
|
||||
|
||||
```json
|
||||
{
|
||||
"dream_synthesize_paths": {
|
||||
"globs": [
|
||||
"wiki/personal/reflections/*",
|
||||
"wiki/originals/*",
|
||||
"wiki/personal/patterns/*",
|
||||
"wiki/people/*",
|
||||
"dream-cycle-summaries/*"
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Editing this list is the ONLY way to add a new directory the synthesizer
|
||||
can write to. The subagent's `put_page` calls are gated server-side; even
|
||||
on prompt-injection success the write is bounded to these prefixes.
|
||||
|
||||
## Slug discipline
|
||||
|
||||
Reflections: `wiki/personal/reflections/YYYY-MM-DD-<topic>-<hash[:6]>`
|
||||
Originals: `wiki/originals/ideas/YYYY-MM-DD-<idea>-<hash[:6]>`
|
||||
Patterns: `wiki/personal/patterns/<theme>`
|
||||
Summary: `dream-cycle-summaries/YYYY-MM-DD`
|
||||
|
||||
The 6-char content-hash suffix on reflections / originals means an edited
|
||||
transcript produces a NEW slug — the original reflection is preserved
|
||||
alongside the new one. No silent overwrite.
|
||||
|
||||
Lowercase alphanumeric and hyphens only. NO underscores, NO file extensions.
|
||||
|
||||
## Provenance
|
||||
|
||||
Every put_page call from the synthesize subagent shows up in
|
||||
`subagent_tool_executions` with full input. The orchestrator collects
|
||||
slugs by querying that table — NOT `pages.updated_at` — so the cycle's
|
||||
write list cannot accidentally include manual edits or sync output.
|
||||
|
||||
## What's deferred to v1.1
|
||||
|
||||
- **Auto git commit + push.** v1 writes markdown files to `brain_dir`
|
||||
but does NOT `git add` / `commit` / `push`. Either commit yourself
|
||||
or let `gbrain autopilot` handle it. v1.1 will add explicit
|
||||
--commit / --push flags with handling for dirty worktree, staged
|
||||
changes, auth failure, and non-fast-forward push.
|
||||
- **Daily token budget cap.** Cooldown alone is the spend bound at v1
|
||||
scale. If real-world telemetry surfaces a problem, v1.1 adds an
|
||||
explicit `daily_token_budget` config.
|
||||
- **Cross-modal pattern review.** Patterns currently runs against
|
||||
reflections only. Future revision could roll up across reflections,
|
||||
meetings, and timeline entries together.
|
||||
|
||||
## Verify after upgrade
|
||||
|
||||
```bash
|
||||
# Schema migration applied?
|
||||
gbrain doctor
|
||||
|
||||
# Phase ordering correct?
|
||||
gbrain dream --help # shows the 8-phase pipeline
|
||||
|
||||
# Dry-run against a single transcript (cheap Haiku call only):
|
||||
gbrain dream --phase synthesize --input /tmp/some-transcript.txt --dry-run --json
|
||||
```
|
||||
|
||||
If any step fails, file an issue with `gbrain doctor` output and the
|
||||
contents of `~/.gbrain/upgrade-errors.jsonl` if it exists.
|
||||
+1
-9
@@ -19,7 +19,7 @@ for (const op of operations) {
|
||||
}
|
||||
|
||||
// CLI-only commands that bypass the operation layer
|
||||
const CLI_ONLY = new Set(['init', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'sources', 'dream', 'check-resolvable', 'routing-eval', 'skillify', 'smoke-test', 'storage', 'repos', 'code-def', 'code-refs', 'reindex-code', 'code-callers', 'code-callees', 'frontmatter', 'auth', 'friction', 'claw-test']);
|
||||
const CLI_ONLY = new Set(['init', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'sources', 'dream', 'check-resolvable', 'routing-eval', 'skillify', 'smoke-test', 'storage', 'repos', 'code-def', 'code-refs', 'reindex-code', 'code-callers', 'code-callees', 'frontmatter', 'auth']);
|
||||
|
||||
async function main() {
|
||||
// Parse global flags (--quiet / --progress-json / --progress-interval)
|
||||
@@ -343,14 +343,6 @@ async function handleCliOnly(command: string, args: string[]) {
|
||||
await runSkillpack(args);
|
||||
return;
|
||||
}
|
||||
if (command === 'friction') {
|
||||
const { runFriction } = await import('./commands/friction.ts');
|
||||
process.exit(runFriction(args));
|
||||
}
|
||||
if (command === 'claw-test') {
|
||||
const { runClawTest } = await import('./commands/claw-test.ts');
|
||||
process.exit(await runClawTest(args));
|
||||
}
|
||||
if (command === 'report') {
|
||||
const { runReport } = await import('./commands/report.ts');
|
||||
await runReport(args);
|
||||
|
||||
@@ -1,424 +0,0 @@
|
||||
/**
|
||||
* gbrain claw-test — end-to-end "fresh user" test harness.
|
||||
*
|
||||
* Two tiers:
|
||||
* gbrain claw-test — scripted (no LLM, CI gate)
|
||||
* gbrain claw-test --live --agent openclaw — real agent, friction discovery
|
||||
*
|
||||
* Phases (scripted mode):
|
||||
* setup → install_brain → import → query → extract → verify → render
|
||||
*
|
||||
* The harness sets GBRAIN_HOME=<tempdir> so the run is hermetic. Each child
|
||||
* gbrain invocation runs with --progress-json and the harness captures stderr
|
||||
* to assert expected_phases from scenario.json fired.
|
||||
*
|
||||
* See ~/.claude/plans/system-instruction-you-are-working-noble-biscuit.md
|
||||
* for the full design rationale (D1–D23 decisions).
|
||||
*/
|
||||
|
||||
import { spawn } from 'child_process';
|
||||
import { mkdtempSync, writeFileSync, mkdirSync, rmSync, existsSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
import { randomBytes } from 'crypto';
|
||||
import { logFriction, frictionDir } from '../core/friction.ts';
|
||||
import { loadScenario, listScenarios, readBrief, type ScenarioConfig } from '../core/claw-test/scenarios.ts';
|
||||
import { parseProgressEvents, verifyExpectedPhases } from '../core/claw-test/progress-tail.ts';
|
||||
import { resolveAgentRunner, listRegisteredAgents, registerAgentRunner } from '../core/claw-test/agent-runner.ts';
|
||||
import { OpenClawRunner } from '../core/claw-test/runners/openclaw.ts';
|
||||
import { createTranscriptSink } from '../core/claw-test/transcript-capture.ts';
|
||||
|
||||
// Ensure built-in runners are registered.
|
||||
registerAgentRunner('openclaw', () => new OpenClawRunner());
|
||||
|
||||
interface HarnessOpts {
|
||||
scenario: string;
|
||||
live: boolean;
|
||||
agent: string;
|
||||
keepTempdir: boolean;
|
||||
listAgents: boolean;
|
||||
help: boolean;
|
||||
/** Path to the gbrain binary used to invoke child commands. Defaults to argv[0]. */
|
||||
gbrainBin?: string;
|
||||
}
|
||||
|
||||
interface PhaseOutcome {
|
||||
phase: string;
|
||||
exitCode: number;
|
||||
durationMs: number;
|
||||
stderrEvents: number;
|
||||
stdoutTail: string;
|
||||
stderrTail: string;
|
||||
}
|
||||
|
||||
const TAIL_BYTES = 4_096;
|
||||
const SUBPROCESS_TIMEOUT_MS = 5 * 60_000; // 5 minutes per phase
|
||||
|
||||
export async function runClawTest(args: string[]): Promise<number> {
|
||||
const opts = parseArgs(args);
|
||||
|
||||
if (opts.help) {
|
||||
printHelp();
|
||||
return 0;
|
||||
}
|
||||
|
||||
if (opts.listAgents) {
|
||||
return cmdListAgents();
|
||||
}
|
||||
|
||||
let scenario: ScenarioConfig;
|
||||
try {
|
||||
scenario = loadScenario(opts.scenario);
|
||||
} catch (e) {
|
||||
console.error(`scenario load failed: ${e instanceof Error ? e.message : String(e)}`);
|
||||
const available = listScenarios();
|
||||
if (available.length) console.error(`available scenarios: ${available.join(', ')}`);
|
||||
return 2;
|
||||
}
|
||||
|
||||
const runId = newRunId(opts.agent);
|
||||
const runRoot = mkdtempSync(join(tmpdir(), `claw-test-${runId}-`));
|
||||
const gbrainHome = runRoot; // configDir() appends '.gbrain' itself
|
||||
const transcriptPath = join(runRoot, 'transcript.jsonl');
|
||||
console.log(`run-id: ${runId}`);
|
||||
console.log(`tempdir: ${runRoot}`);
|
||||
|
||||
// SIGINT/SIGTERM finalization (D11)
|
||||
let interrupted = false;
|
||||
const onSignal = () => {
|
||||
interrupted = true;
|
||||
try {
|
||||
logFriction({
|
||||
runId,
|
||||
phase: 'harness',
|
||||
message: 'run interrupted by signal',
|
||||
kind: 'interrupted',
|
||||
source: 'harness',
|
||||
agent: opts.agent,
|
||||
});
|
||||
} catch { /* best effort */ }
|
||||
};
|
||||
process.once('SIGINT', onSignal);
|
||||
process.once('SIGTERM', onSignal);
|
||||
|
||||
let exitCode = 0;
|
||||
try {
|
||||
if (opts.live) {
|
||||
exitCode = await runLive(opts, scenario, { runId, runRoot, gbrainHome, transcriptPath });
|
||||
} else {
|
||||
exitCode = await runScripted(opts, scenario, { runId, runRoot, gbrainHome });
|
||||
}
|
||||
} finally {
|
||||
process.off('SIGINT', onSignal);
|
||||
process.off('SIGTERM', onSignal);
|
||||
if (!opts.keepTempdir && !interrupted) {
|
||||
try { rmSync(runRoot, { recursive: true, force: true }); } catch { /* best effort */ }
|
||||
} else {
|
||||
console.log(`tempdir kept at: ${runRoot}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Always render at the end so the operator can immediately see the report.
|
||||
console.log('---');
|
||||
console.log(`friction log: ${join(frictionDir(), runId + '.jsonl')}`);
|
||||
console.log(`render report: gbrain friction render --run-id ${runId}`);
|
||||
|
||||
if (interrupted) return 130;
|
||||
return exitCode;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Scripted mode
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
async function runScripted(
|
||||
opts: HarnessOpts,
|
||||
scenario: ScenarioConfig,
|
||||
ctx: { runId: string; runRoot: string; gbrainHome: string },
|
||||
): Promise<number> {
|
||||
const childEnv: Record<string, string> = {
|
||||
...process.env as Record<string, string>,
|
||||
GBRAIN_HOME: ctx.gbrainHome,
|
||||
GBRAIN_FRICTION_RUN_ID: ctx.runId,
|
||||
};
|
||||
|
||||
const phases: { name: string; argv: string[] }[] = [];
|
||||
// Phase 2: install_brain
|
||||
phases.push({ name: 'install_brain', argv: ['init', '--pglite'] });
|
||||
|
||||
// Phase 3: import (only when scenario has a brain dir)
|
||||
if (scenario.brainRelative) {
|
||||
const brainDir = join(scenario.dir, scenario.brainRelative);
|
||||
phases.push({ name: 'import', argv: ['import', brainDir, '--no-embed', '--progress-json'] });
|
||||
}
|
||||
|
||||
// Phase 4: query (best-effort sanity)
|
||||
phases.push({ name: 'query', argv: ['query', 'the'] });
|
||||
|
||||
// Phase 5: extract (positional argument is required: 'all' covers links + timeline)
|
||||
phases.push({ name: 'extract', argv: ['extract', 'all', '--source', 'fs', '--progress-json'] });
|
||||
|
||||
// Phase 6: verify
|
||||
phases.push({ name: 'verify', argv: ['doctor', '--json', '--progress-json'] });
|
||||
|
||||
// Pre-phase: upgrade scenario seeds the database
|
||||
if (scenario.kind === 'upgrade' && scenario.seedRelative) {
|
||||
const seedSql = join(scenario.dir, scenario.seedRelative, 'dump.sql');
|
||||
if (existsSync(seedSql)) {
|
||||
const dbPath = join(ctx.gbrainHome, '.gbrain', 'brain.pglite');
|
||||
mkdirSync(join(ctx.gbrainHome, '.gbrain'), { recursive: true });
|
||||
const { seedPgliteFromFile } = await import('../core/claw-test/seed-pglite.ts');
|
||||
try {
|
||||
await seedPgliteFromFile({ dbPath, sqlPath: seedSql });
|
||||
console.log(`[seed] replayed ${seedSql} → ${dbPath}`);
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
logFriction({
|
||||
runId: ctx.runId,
|
||||
phase: 'seed',
|
||||
message: `seed replay failed: ${msg}`,
|
||||
severity: 'blocker',
|
||||
source: 'harness',
|
||||
agent: opts.agent,
|
||||
});
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const allStderr: string[] = [];
|
||||
const outcomes: PhaseOutcome[] = [];
|
||||
for (const phase of phases) {
|
||||
const outcome = await invokeGbrain(opts.gbrainBin ?? 'gbrain', phase.argv, ctx.runRoot, childEnv);
|
||||
outcome.phase = phase.name;
|
||||
outcomes.push(outcome);
|
||||
allStderr.push(outcome.stderrTail);
|
||||
if (outcome.exitCode !== 0) {
|
||||
logFriction({
|
||||
runId: ctx.runId,
|
||||
phase: phase.name,
|
||||
message: `command failed (exit ${outcome.exitCode}): gbrain ${phase.argv.join(' ')}`,
|
||||
severity: 'error',
|
||||
hint: outcome.stderrTail.trim().slice(0, 500),
|
||||
source: 'harness',
|
||||
agent: opts.agent,
|
||||
});
|
||||
return 1;
|
||||
} else {
|
||||
logFriction({
|
||||
runId: ctx.runId,
|
||||
phase: phase.name,
|
||||
message: `phase complete in ${outcome.durationMs}ms`,
|
||||
kind: 'phase-marker',
|
||||
marker: 'end',
|
||||
source: 'harness',
|
||||
agent: opts.agent,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Phase verification: collect all events from every captured stderr and assert coverage.
|
||||
const events = allStderr.flatMap(parseProgressEvents);
|
||||
const missing = verifyExpectedPhases(events, scenario.expectedPhases);
|
||||
if (missing.length) {
|
||||
for (const phaseName of missing) {
|
||||
logFriction({
|
||||
runId: ctx.runId,
|
||||
phase: phaseName,
|
||||
message: `expected progress event for "${phaseName}" never fired`,
|
||||
severity: 'blocker',
|
||||
hint: 'either the command did not run or it did not emit progress events; check phase log above',
|
||||
source: 'harness',
|
||||
agent: opts.agent,
|
||||
});
|
||||
}
|
||||
return 1;
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Live mode
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
async function runLive(
|
||||
opts: HarnessOpts,
|
||||
scenario: ScenarioConfig,
|
||||
ctx: { runId: string; runRoot: string; gbrainHome: string; transcriptPath: string },
|
||||
): Promise<number> {
|
||||
let runner;
|
||||
try {
|
||||
runner = resolveAgentRunner(opts.agent);
|
||||
} catch (e) {
|
||||
console.error(e instanceof Error ? e.message : String(e));
|
||||
return 2;
|
||||
}
|
||||
|
||||
const detected = await runner.detect();
|
||||
if (!detected.available) {
|
||||
console.error(`agent "${opts.agent}" not available: ${detected.reason ?? 'unknown'}`);
|
||||
logFriction({
|
||||
runId: ctx.runId,
|
||||
phase: 'agent_detect',
|
||||
message: `agent ${opts.agent} not available: ${detected.reason ?? 'unknown'}`,
|
||||
severity: 'blocker',
|
||||
hint: opts.agent === 'openclaw' ? 'install openclaw or set OPENCLAW_BIN' : undefined,
|
||||
source: 'harness',
|
||||
agent: opts.agent,
|
||||
});
|
||||
return 2;
|
||||
}
|
||||
|
||||
const sink = createTranscriptSink(ctx.transcriptPath);
|
||||
const env: Record<string, string> = {
|
||||
GBRAIN_HOME: ctx.gbrainHome,
|
||||
GBRAIN_FRICTION_RUN_ID: ctx.runId,
|
||||
};
|
||||
|
||||
const brief = readBrief(scenario);
|
||||
let result;
|
||||
try {
|
||||
result = await runner.invoke({
|
||||
cwd: ctx.runRoot,
|
||||
brief,
|
||||
env,
|
||||
timeoutMs: SUBPROCESS_TIMEOUT_MS,
|
||||
transcriptSink: sink,
|
||||
});
|
||||
} finally {
|
||||
await sink.close();
|
||||
}
|
||||
|
||||
if (result.exitCode !== 0) {
|
||||
logFriction({
|
||||
runId: ctx.runId,
|
||||
phase: 'agent_invoke',
|
||||
message: `agent exited with code ${result.exitCode} after ${result.durationMs}ms`,
|
||||
severity: 'error',
|
||||
source: 'harness',
|
||||
agent: opts.agent,
|
||||
});
|
||||
return result.exitCode;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Subprocess helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function invokeGbrain(
|
||||
bin: string,
|
||||
argv: string[],
|
||||
cwd: string,
|
||||
env: Record<string, string>,
|
||||
): Promise<PhaseOutcome> {
|
||||
return new Promise((resolve) => {
|
||||
const start = Date.now();
|
||||
const child = spawn(bin, argv, { cwd, env, stdio: ['ignore', 'pipe', 'pipe'], shell: false });
|
||||
const stdout: Buffer[] = [];
|
||||
const stderr: Buffer[] = [];
|
||||
child.stdout?.on('data', (b: Buffer) => stdout.push(b));
|
||||
child.stderr?.on('data', (b: Buffer) => stderr.push(b));
|
||||
child.on('error', (err) => {
|
||||
const stderrJoined = Buffer.concat(stderr).toString('utf-8') + '\nspawn error: ' + err.message;
|
||||
resolve({
|
||||
phase: '',
|
||||
exitCode: 127,
|
||||
durationMs: Date.now() - start,
|
||||
stderrEvents: 0,
|
||||
stdoutTail: tailOf(Buffer.concat(stdout).toString('utf-8')),
|
||||
stderrTail: tailOf(stderrJoined),
|
||||
});
|
||||
});
|
||||
child.on('close', (code) => {
|
||||
const stderrText = Buffer.concat(stderr).toString('utf-8');
|
||||
resolve({
|
||||
phase: '',
|
||||
exitCode: typeof code === 'number' ? code : 1,
|
||||
durationMs: Date.now() - start,
|
||||
stderrEvents: parseProgressEvents(stderrText).length,
|
||||
stdoutTail: tailOf(Buffer.concat(stdout).toString('utf-8')),
|
||||
stderrTail: stderrText,
|
||||
});
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
function tailOf(s: string): string {
|
||||
if (s.length <= TAIL_BYTES) return s;
|
||||
return s.slice(-TAIL_BYTES);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Argv parsing + helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function parseArgs(args: string[]): HarnessOpts {
|
||||
const out: HarnessOpts = {
|
||||
scenario: 'fresh-install',
|
||||
live: false,
|
||||
agent: 'openclaw',
|
||||
keepTempdir: false,
|
||||
listAgents: false,
|
||||
help: args.includes('--help') || args.includes('-h'),
|
||||
gbrainBin: process.env.GBRAIN_BIN_OVERRIDE || process.execPath,
|
||||
};
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
const a = args[i];
|
||||
if (a === '--live') out.live = true;
|
||||
else if (a === '--keep-tempdir') out.keepTempdir = true;
|
||||
else if (a === '--list-agents') out.listAgents = true;
|
||||
else if (a === '--scenario') out.scenario = args[++i] ?? out.scenario;
|
||||
else if (a === '--agent') out.agent = args[++i] ?? out.agent;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function newRunId(agent: string): string {
|
||||
const now = new Date();
|
||||
const ts = now.toISOString().replace(/[-:]/g, '').replace(/\..*/, '').replace('T', '-');
|
||||
const suf = randomBytes(4).toString('hex');
|
||||
return `claw-test-${ts}-${agent}-${suf}`;
|
||||
}
|
||||
|
||||
function cmdListAgents(): number {
|
||||
const names = listRegisteredAgents();
|
||||
if (!names.length) {
|
||||
console.log('no agents registered');
|
||||
return 0;
|
||||
}
|
||||
for (const name of names) {
|
||||
try {
|
||||
const runner = resolveAgentRunner(name);
|
||||
runner.detect().then((d) => {
|
||||
const status = d.available ? `available at ${d.binPath}` : `unavailable: ${d.reason}`;
|
||||
console.log(`${name}: ${status}`);
|
||||
}).catch(() => { /* best effort */ });
|
||||
} catch {
|
||||
console.log(`${name}: (factory error)`);
|
||||
}
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
function printHelp() {
|
||||
console.log(`gbrain claw-test — end-to-end claw-setup friction harness
|
||||
|
||||
Usage:
|
||||
gbrain claw-test [--scenario <name>] [--live --agent <name>] [--keep-tempdir]
|
||||
gbrain claw-test --list-agents
|
||||
|
||||
Defaults:
|
||||
--scenario fresh-install
|
||||
--agent openclaw (live mode only)
|
||||
|
||||
Scripted mode runs canonical commands without an LLM (CI gate).
|
||||
Live mode spawns a real agent and lets it drive (~5–10 min, costs tokens).
|
||||
|
||||
Examples:
|
||||
gbrain claw-test --scenario fresh-install
|
||||
gbrain claw-test --scenario upgrade-from-v0.18 --keep-tempdir
|
||||
gbrain claw-test --live --agent openclaw`);
|
||||
}
|
||||
@@ -774,30 +774,6 @@ export async function runDoctor(engine: BrainEngine | null, args: string[], dbSo
|
||||
ORDER BY depth DESC
|
||||
LIMIT 5
|
||||
`;
|
||||
// Subcheck 3 (v0.22.14): RSS-watchdog kills in the last 24h. Bare workers
|
||||
// newly default to --max-rss 2048 (was 0); operators who run large embed
|
||||
// or import jobs may see kills that didn't happen pre-v0.22.14. We surface
|
||||
// a hint when this signature appears so the upgrade path is obvious.
|
||||
// Signature: when the watchdog trips, gracefulShutdown('watchdog') aborts
|
||||
// in-flight jobs with `new Error('watchdog')`. The worker's failJob path
|
||||
// (worker.ts:660-664) writes `error_text = 'aborted: watchdog'` for any
|
||||
// job in-flight at the moment of the kill.
|
||||
//
|
||||
// We deliberately DO NOT do a loose `ILIKE '%watchdog%'`:
|
||||
// 1. Parent jobs that inherit `on_child_fail='fail_parent'` get
|
||||
// `"child job N failed: aborted: watchdog"` — counting that
|
||||
// double-counts (child + parent) for one watchdog event.
|
||||
// 2. Any user error_text containing the word "watchdog" matches.
|
||||
// Match the exact prefix `'aborted: watchdog'` to scope this purely to
|
||||
// the worker's own kill signature.
|
||||
const rssKillRows: Array<{ cnt: number }> = await sql`
|
||||
SELECT count(*)::int AS cnt
|
||||
FROM minion_jobs
|
||||
WHERE status IN ('dead', 'failed')
|
||||
AND finished_at > now() - interval '24 hours'
|
||||
AND error_text = 'aborted: watchdog'
|
||||
`;
|
||||
const rssKillCount = rssKillRows[0]?.cnt ?? 0;
|
||||
|
||||
const problems: string[] = [];
|
||||
if (stalledRows.length > 0) {
|
||||
@@ -818,14 +794,6 @@ export async function runDoctor(engine: BrainEngine | null, args: string[], dbSo
|
||||
`Fix: set maxWaiting on the submitter (or raise GBRAIN_QUEUE_WAITING_THRESHOLD).`
|
||||
);
|
||||
}
|
||||
if (rssKillCount > 0) {
|
||||
problems.push(
|
||||
`${rssKillCount} job(s) dead-lettered for RSS-watchdog memory-limit kills in last 24h. ` +
|
||||
`v0.22.14 changed the bare-worker --max-rss default from 0 (off) to 2048 MB. ` +
|
||||
`Fix: raise the limit (e.g. \`gbrain jobs work --max-rss 4096\`) or opt out (\`--max-rss 0\`). ` +
|
||||
`See skills/migrations/v0.22.14.md.`
|
||||
);
|
||||
}
|
||||
|
||||
if (problems.length === 0) {
|
||||
checks.push({
|
||||
|
||||
+6
-86
@@ -39,22 +39,12 @@ interface DreamArgs {
|
||||
phase: CyclePhase | null;
|
||||
dir: string | null;
|
||||
help: boolean;
|
||||
/** v0.21: ad-hoc transcript file path; implies --phase synthesize. */
|
||||
inputFile: string | null;
|
||||
/** v0.21: restrict synthesize to a single date (YYYY-MM-DD). */
|
||||
date: string | null;
|
||||
/** v0.21: backfill range start (YYYY-MM-DD). */
|
||||
from: string | null;
|
||||
/** v0.21: backfill range end (YYYY-MM-DD). */
|
||||
to: string | null;
|
||||
}
|
||||
|
||||
const ISO_DATE_RE = /^\d{4}-\d{2}-\d{2}$/;
|
||||
|
||||
function parseArgs(args: string[]): DreamArgs {
|
||||
const phaseIdx = args.indexOf('--phase');
|
||||
const rawPhase = phaseIdx !== -1 ? args[phaseIdx + 1] : null;
|
||||
let phase = rawPhase && (ALL_PHASES as string[]).includes(rawPhase)
|
||||
const phase = rawPhase && (ALL_PHASES as string[]).includes(rawPhase)
|
||||
? (rawPhase as CyclePhase)
|
||||
: null;
|
||||
if (rawPhase && !phase) {
|
||||
@@ -65,44 +55,6 @@ function parseArgs(args: string[]): DreamArgs {
|
||||
const dirIdx = args.indexOf('--dir');
|
||||
const dir = dirIdx !== -1 ? args[dirIdx + 1] : null;
|
||||
|
||||
const inputIdx = args.indexOf('--input');
|
||||
const inputFile = inputIdx !== -1 ? args[inputIdx + 1] ?? null : null;
|
||||
|
||||
const dateIdx = args.indexOf('--date');
|
||||
const date = dateIdx !== -1 ? args[dateIdx + 1] ?? null : null;
|
||||
if (date && !ISO_DATE_RE.test(date)) {
|
||||
console.error(`--date must be YYYY-MM-DD; got "${date}"`);
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
const fromIdx = args.indexOf('--from');
|
||||
const from = fromIdx !== -1 ? args[fromIdx + 1] ?? null : null;
|
||||
if (from && !ISO_DATE_RE.test(from)) {
|
||||
console.error(`--from must be YYYY-MM-DD; got "${from}"`);
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
const toIdx = args.indexOf('--to');
|
||||
const to = toIdx !== -1 ? args[toIdx + 1] ?? null : null;
|
||||
if (to && !ISO_DATE_RE.test(to)) {
|
||||
console.error(`--to must be YYYY-MM-DD; got "${to}"`);
|
||||
process.exit(2);
|
||||
}
|
||||
if (from && to && from > to) {
|
||||
console.error(`--from (${from}) is after --to (${to}); empty range`);
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
// --input + --date / --from / --to is incoherent: --input is a single
|
||||
// file, the date filters scan a directory.
|
||||
if (inputFile && (date || from || to)) {
|
||||
console.error('--input cannot be combined with --date / --from / --to');
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
// --input implies --phase synthesize.
|
||||
if (inputFile && !phase) phase = 'synthesize';
|
||||
|
||||
return {
|
||||
json: args.includes('--json'),
|
||||
dryRun: args.includes('--dry-run'),
|
||||
@@ -110,10 +62,6 @@ function parseArgs(args: string[]): DreamArgs {
|
||||
phase,
|
||||
dir,
|
||||
help: args.includes('--help') || args.includes('-h'),
|
||||
inputFile,
|
||||
date,
|
||||
from,
|
||||
to,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -156,43 +104,23 @@ async function resolveBrainDir(
|
||||
function printHelp() {
|
||||
console.log(`Usage: gbrain dream [options]
|
||||
|
||||
Run one brain maintenance cycle. Eight phases:
|
||||
lint -> backlinks -> sync -> synthesize -> extract -> patterns -> embed -> orphans
|
||||
|
||||
The synthesize + patterns phases (v0.21) consolidate yesterday's
|
||||
conversation transcripts into reflections, originals, and cross-session
|
||||
pattern pages. Designed for cron (exits when done).
|
||||
Run one brain maintenance cycle: lint, backlinks, orphan sweep, sync,
|
||||
extract, and embed. Designed for cron (exits when done).
|
||||
|
||||
Options:
|
||||
--dry-run Preview all fixes without writing. Note: synthesize
|
||||
runs the cheap Haiku significance filter (caches
|
||||
verdicts), but skips the Sonnet synthesis pass.
|
||||
"--dry-run" does NOT mean "zero LLM calls."
|
||||
--dry-run Preview all fixes without writing (fs or DB)
|
||||
--json Emit the CycleReport as JSON (agent-readable)
|
||||
--phase <name> Run a single phase: ${ALL_PHASES.join(' | ')}
|
||||
--pull git pull the brain repo before syncing (default: no pull)
|
||||
--dir <path> Brain directory (default: configured brain)
|
||||
|
||||
--input <file> Synthesize a specific transcript file (implies
|
||||
--phase synthesize). Bypasses corpus-dir scan.
|
||||
--date YYYY-MM-DD Synthesize transcripts dated for one specific day.
|
||||
--from YYYY-MM-DD Backfill range start (use with --to).
|
||||
--to YYYY-MM-DD Backfill range end.
|
||||
|
||||
--help, -h Show this help
|
||||
|
||||
Examples:
|
||||
gbrain dream
|
||||
gbrain dream --dry-run --json
|
||||
gbrain dream --phase lint
|
||||
gbrain dream --phase synthesize --input ~/transcripts/2026-04-25.txt
|
||||
gbrain dream --phase synthesize --from 2026-04-01 --to 2026-04-25
|
||||
0 2 * * * gbrain dream --json # nightly via cron
|
||||
|
||||
Configure synthesize:
|
||||
gbrain config set dream.synthesize.session_corpus_dir /path/to/transcripts
|
||||
gbrain config set dream.synthesize.enabled true
|
||||
|
||||
Related:
|
||||
gbrain autopilot --install # continuous maintenance as a daemon
|
||||
gbrain autopilot # same maintenance cycle, scheduled
|
||||
@@ -237,14 +165,10 @@ function printHuman(report: CycleReport) {
|
||||
const t = report.totals;
|
||||
const hasTotals =
|
||||
t.lint_fixes > 0 || t.backlinks_added > 0 || t.pages_synced > 0 ||
|
||||
t.pages_extracted > 0 || t.pages_embedded > 0 || t.orphans_found > 0 ||
|
||||
t.transcripts_processed > 0 || t.synth_pages_written > 0 || t.patterns_written > 0;
|
||||
t.pages_extracted > 0 || t.pages_embedded > 0 || t.orphans_found > 0;
|
||||
if (hasTotals) {
|
||||
console.log(
|
||||
` totals: lint=${t.lint_fixes} backlinks=${t.backlinks_added} synced=${t.pages_synced} ` +
|
||||
`extracted=${t.pages_extracted} embedded=${t.pages_embedded} orphans=${t.orphans_found} ` +
|
||||
`synth_transcripts=${t.transcripts_processed} synth_pages=${t.synth_pages_written} ` +
|
||||
`patterns=${t.patterns_written}`,
|
||||
` totals: lint=${t.lint_fixes} backlinks=${t.backlinks_added} synced=${t.pages_synced} extracted=${t.pages_extracted} embedded=${t.pages_embedded} orphans=${t.orphans_found}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -267,10 +191,6 @@ export async function runDream(engine: BrainEngine | null, args: string[]): Prom
|
||||
dryRun: opts.dryRun,
|
||||
pull: opts.pull,
|
||||
phases,
|
||||
synthInputFile: opts.inputFile ?? undefined,
|
||||
synthDate: opts.date ?? undefined,
|
||||
synthFrom: opts.from ?? undefined,
|
||||
synthTo: opts.to ?? undefined,
|
||||
});
|
||||
|
||||
if (opts.json) {
|
||||
|
||||
@@ -1,185 +0,0 @@
|
||||
/**
|
||||
* gbrain friction — friction reporter CLI.
|
||||
*
|
||||
* Four subcommands in v1 (analytical/clustering ones move to v1.1):
|
||||
* gbrain friction log Append a friction or delight entry
|
||||
* gbrain friction render Render a run as markdown or JSON
|
||||
* gbrain friction list List recent runs with counts
|
||||
* gbrain friction summary Side-by-side friction + delight summary
|
||||
*
|
||||
* Subcommands stay thin (≤ ~30 LOC each). Core logic lives in src/core/friction.ts.
|
||||
*
|
||||
* The CLI is dispatched from src/cli.ts. See `gbrain friction --help`.
|
||||
*/
|
||||
|
||||
import {
|
||||
logFriction, readFriction, listRuns, renderReport, renderSummary,
|
||||
activeRunId, frictionFile,
|
||||
type FrictionKind, type FrictionSeverity,
|
||||
} from '../core/friction.ts';
|
||||
|
||||
const VALID_KINDS = new Set<FrictionKind>(['friction', 'delight', 'phase-marker', 'interrupted']);
|
||||
const VALID_SEVERITIES = new Set<FrictionSeverity>(['confused', 'error', 'blocker', 'nit']);
|
||||
|
||||
export function runFriction(args: string[]): number {
|
||||
const [sub, ...rest] = args;
|
||||
switch (sub) {
|
||||
case 'log': return cmdLog(rest);
|
||||
case 'render': return cmdRender(rest);
|
||||
case 'list': return cmdList(rest);
|
||||
case 'summary': return cmdSummary(rest);
|
||||
case undefined:
|
||||
case '--help':
|
||||
case '-h':
|
||||
printHelp();
|
||||
return 0;
|
||||
default:
|
||||
console.error(`unknown subcommand: ${sub}`);
|
||||
printHelp();
|
||||
return 2;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// log
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function cmdLog(args: string[]): number {
|
||||
const flags = parseFlags(args);
|
||||
const phase = flags.string('--phase');
|
||||
const message = flags.string('--message');
|
||||
if (!phase || !message) {
|
||||
console.error('usage: gbrain friction log --phase <name> --message <text> [--severity ...] [--hint ...] [--kind ...] [--run-id ...]');
|
||||
return 2;
|
||||
}
|
||||
const kind = (flags.string('--kind') ?? 'friction') as FrictionKind;
|
||||
if (!VALID_KINDS.has(kind)) {
|
||||
console.error(`invalid --kind ${kind}; must be one of: ${[...VALID_KINDS].join(', ')}`);
|
||||
return 2;
|
||||
}
|
||||
const severityRaw = flags.string('--severity');
|
||||
const severity = severityRaw as FrictionSeverity | undefined;
|
||||
if (severity && !VALID_SEVERITIES.has(severity)) {
|
||||
console.error(`invalid --severity ${severity}; must be one of: ${[...VALID_SEVERITIES].join(', ')}`);
|
||||
return 2;
|
||||
}
|
||||
try {
|
||||
logFriction({
|
||||
phase,
|
||||
message,
|
||||
kind,
|
||||
severity,
|
||||
hint: flags.string('--hint'),
|
||||
runId: flags.string('--run-id'),
|
||||
agent: flags.string('--agent'),
|
||||
source: 'claw',
|
||||
});
|
||||
} catch (e) {
|
||||
console.error(`friction log failed: ${e instanceof Error ? e.message : String(e)}`);
|
||||
return 1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// render
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function cmdRender(args: string[]): number {
|
||||
const flags = parseFlags(args);
|
||||
const runId = flags.string('--run-id') ?? activeRunId();
|
||||
const json = flags.bool('--json');
|
||||
const format = json ? 'json' : 'md';
|
||||
const transcripts = flags.bool('--transcripts');
|
||||
const noRedact = flags.bool('--no-redact');
|
||||
// --redact is the default for md output; --no-redact disables.
|
||||
const redact = noRedact ? false : (format === 'md');
|
||||
try {
|
||||
const out = renderReport(runId, {
|
||||
format,
|
||||
redact,
|
||||
transcriptPath: transcripts ? flags.string('--transcript-path') ?? undefined : undefined,
|
||||
});
|
||||
process.stdout.write(out + '\n');
|
||||
return 0;
|
||||
} catch (e) {
|
||||
console.error(`friction render failed: ${e instanceof Error ? e.message : String(e)}`);
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// list
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function cmdList(args: string[]): number {
|
||||
const flags = parseFlags(args);
|
||||
const json = flags.bool('--json');
|
||||
const runs = listRuns();
|
||||
if (json) {
|
||||
console.log(JSON.stringify(runs, null, 2));
|
||||
return 0;
|
||||
}
|
||||
if (runs.length === 0) {
|
||||
console.log('no runs yet');
|
||||
return 0;
|
||||
}
|
||||
for (const r of runs) {
|
||||
const interrupted = r.counts.interrupted ? ' (interrupted)' : '';
|
||||
const sev = Object.entries(r.counts.bySeverity).map(([k, v]) => `${k}=${v}`).join(' ');
|
||||
console.log(`${r.runId}${interrupted} friction=${r.counts.friction} delight=${r.counts.delight} ${sev}`);
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// summary
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function cmdSummary(args: string[]): number {
|
||||
const flags = parseFlags(args);
|
||||
const runId = flags.string('--run-id') ?? activeRunId();
|
||||
const json = flags.bool('--json');
|
||||
try {
|
||||
const out = renderSummary(runId, { format: json ? 'json' : 'md' });
|
||||
process.stdout.write(out + '\n');
|
||||
return 0;
|
||||
} catch (e) {
|
||||
console.error(`friction summary failed: ${e instanceof Error ? e.message : String(e)}`);
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function parseFlags(args: string[]) {
|
||||
return {
|
||||
string(flag: string): string | undefined {
|
||||
const idx = args.indexOf(flag);
|
||||
return idx === -1 ? undefined : args[idx + 1];
|
||||
},
|
||||
bool(flag: string): boolean {
|
||||
return args.includes(flag);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function printHelp() {
|
||||
console.log(`gbrain friction — friction reporter
|
||||
|
||||
Subcommands:
|
||||
log Append a friction or delight entry to the active run
|
||||
render Render a run's entries as markdown (default) or JSON
|
||||
list List recent runs with friction/delight counts
|
||||
summary Two-column summary of friction + delight for a run
|
||||
|
||||
Examples:
|
||||
gbrain friction log --severity confused --phase install --message "init didn't say which engine"
|
||||
gbrain friction render --run-id claw-test-20260428-... --transcripts
|
||||
gbrain friction list --json
|
||||
gbrain friction summary
|
||||
|
||||
Run-id resolution: --run-id > $GBRAIN_FRICTION_RUN_ID > 'standalone'.`);
|
||||
}
|
||||
+1
-193
@@ -49,10 +49,6 @@ export async function runFrontmatter(args: string[]): Promise<void> {
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (sub === 'generate') {
|
||||
await runGenerate(rest);
|
||||
return;
|
||||
}
|
||||
if (sub === 'install-hook') {
|
||||
const { runFrontmatterInstallHook } = await import('./frontmatter-install-hook.ts');
|
||||
await runFrontmatterInstallHook(rest);
|
||||
@@ -75,11 +71,10 @@ async function connectEngineForAudit(): Promise<BrainEngine> {
|
||||
}
|
||||
|
||||
function printHelp() {
|
||||
console.log(`gbrain frontmatter — frontmatter validation, audit, auto-repair, and generation
|
||||
console.log(`gbrain frontmatter — frontmatter validation, audit, and auto-repair
|
||||
|
||||
Usage:
|
||||
gbrain frontmatter validate <path> [--json] [--fix] [--dry-run]
|
||||
gbrain frontmatter generate <path> [--fix] [--dry-run] [--json]
|
||||
gbrain frontmatter audit [--source <id>] [--json]
|
||||
gbrain frontmatter install-hook [--source <id>] [--force] [--uninstall]
|
||||
|
||||
@@ -96,26 +91,6 @@ validate
|
||||
--dry-run Preview --fix without writing.
|
||||
--json Emit a JSON envelope on stdout.
|
||||
|
||||
generate
|
||||
Synthesize frontmatter for files that have none (MISSING_OPEN). Uses
|
||||
directory-aware rules to infer type, title, date, source, and tags from
|
||||
the filesystem path and file content. Zero LLM calls, fully deterministic.
|
||||
|
||||
Without --fix: dry-run preview showing what would be generated.
|
||||
With --fix: writes frontmatter to files (with .bak safety backups).
|
||||
|
||||
Rules are defined in src/core/frontmatter-inference.ts DIRECTORY_RULES.
|
||||
Add new directory conventions by adding rules to the table.
|
||||
|
||||
Examples:
|
||||
gbrain frontmatter generate /path/to/brain # preview all
|
||||
gbrain frontmatter generate /path/to/brain --fix # write all
|
||||
gbrain frontmatter generate /path/to/brain/people/ --fix # just people/
|
||||
|
||||
--fix Write generated frontmatter to files (.bak safety backups).
|
||||
--dry-run Preview without writing (default when --fix is omitted).
|
||||
--json Emit JSON output.
|
||||
|
||||
audit
|
||||
Read-only scan across all registered sources (or one with --source <id>).
|
||||
Reports per-source counts grouped by error code. Use this in CI or doctor
|
||||
@@ -322,170 +297,3 @@ function printAuditHumanReport(report: AuditReport): void {
|
||||
console.log(`\nFix with: gbrain frontmatter validate <source-path> --fix`);
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// generate — synthesize frontmatter for files that have none
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
async function runGenerate(args: string[]): Promise<void> {
|
||||
const targetPath = args.find(a => !a.startsWith('-'));
|
||||
const doFix = args.includes('--fix');
|
||||
const dryRun = args.includes('--dry-run');
|
||||
const jsonOut = args.includes('--json');
|
||||
|
||||
if (!targetPath) {
|
||||
console.error('error: gbrain frontmatter generate requires a <path> argument');
|
||||
console.error('usage: gbrain frontmatter generate <path> [--fix] [--dry-run] [--json]');
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
|
||||
const { inferFrontmatter, serializeFrontmatter } = await import('../core/frontmatter-inference.ts');
|
||||
const { resolve, relative, join, basename } = await import('path');
|
||||
const { readFileSync, writeFileSync, copyFileSync, statSync, readdirSync, lstatSync } = await import('fs');
|
||||
|
||||
const rootPath = resolve(targetPath);
|
||||
const isDir = statSync(rootPath).isDirectory();
|
||||
|
||||
// Find the brain root — walk up from targetPath looking for .git or known brain markers.
|
||||
// Inference rules match against brain-root-relative paths (e.g., "people/alice.md").
|
||||
let brainRoot = rootPath;
|
||||
if (isDir) {
|
||||
let candidate = rootPath;
|
||||
for (let i = 0; i < 10; i++) {
|
||||
try {
|
||||
statSync(join(candidate, '.git'));
|
||||
brainRoot = candidate;
|
||||
break;
|
||||
} catch {
|
||||
const parent = resolve(candidate, '..');
|
||||
if (parent === candidate) break;
|
||||
candidate = parent;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
interface GenerateResult {
|
||||
path: string;
|
||||
type: string;
|
||||
title: string;
|
||||
date?: string;
|
||||
rule: string;
|
||||
}
|
||||
|
||||
const results: GenerateResult[] = [];
|
||||
let scanned = 0;
|
||||
let skipped = 0;
|
||||
let generated = 0;
|
||||
let written = 0;
|
||||
|
||||
function processFile(absPath: string, relPath: string) {
|
||||
scanned++;
|
||||
if (!absPath.endsWith('.md')) return;
|
||||
|
||||
// Skip symlinks
|
||||
try { if (lstatSync(absPath).isSymbolicLink()) return; } catch { return; }
|
||||
|
||||
let content: string;
|
||||
try { content = readFileSync(absPath, 'utf-8'); } catch { return; }
|
||||
|
||||
const inferred = inferFrontmatter(relPath, content);
|
||||
if (inferred.skipped) {
|
||||
skipped++;
|
||||
return;
|
||||
}
|
||||
|
||||
generated++;
|
||||
results.push({
|
||||
path: relPath,
|
||||
type: inferred.type,
|
||||
title: inferred.title,
|
||||
date: inferred.date,
|
||||
rule: inferred.matchedRule || '(default)',
|
||||
});
|
||||
|
||||
if (doFix && !dryRun) {
|
||||
const fm = serializeFrontmatter(inferred);
|
||||
const newContent = fm + '\n' + content;
|
||||
// Safety: write .bak first
|
||||
copyFileSync(absPath, absPath + '.bak');
|
||||
writeFileSync(absPath, newContent, 'utf-8');
|
||||
written++;
|
||||
}
|
||||
}
|
||||
|
||||
function walkDir(dir: string, rootForRel: string) {
|
||||
let entries: string[];
|
||||
try { entries = readdirSync(dir); } catch { return; }
|
||||
for (const entry of entries) {
|
||||
if (entry === '.git' || entry === 'node_modules' || entry === '.obsidian') continue;
|
||||
const abs = join(dir, entry);
|
||||
try {
|
||||
const stat = statSync(abs);
|
||||
if (stat.isDirectory()) {
|
||||
walkDir(abs, rootForRel);
|
||||
} else if (stat.isFile() && entry.endsWith('.md')) {
|
||||
processFile(abs, relative(rootForRel, abs));
|
||||
}
|
||||
} catch { /* skip unreadable */ }
|
||||
}
|
||||
}
|
||||
|
||||
if (isDir) {
|
||||
walkDir(rootPath, brainRoot);
|
||||
} else {
|
||||
const relPath = relative(brainRoot, rootPath) || basename(rootPath);
|
||||
processFile(rootPath, relPath);
|
||||
}
|
||||
|
||||
// Output
|
||||
if (jsonOut) {
|
||||
console.log(JSON.stringify({
|
||||
scanned,
|
||||
skipped,
|
||||
generated,
|
||||
written,
|
||||
dryRun: !doFix || dryRun,
|
||||
results: results.slice(0, 100), // Cap JSON output
|
||||
totalResults: results.length,
|
||||
}, null, 2));
|
||||
return;
|
||||
}
|
||||
|
||||
// Human-readable output
|
||||
const mode = doFix && !dryRun ? 'WRITE' : 'DRY-RUN';
|
||||
console.log(`\nFrontmatter generation (${mode})`);
|
||||
console.log(` Scanned: ${scanned} files`);
|
||||
console.log(` Already have frontmatter: ${skipped}`);
|
||||
console.log(` Would generate: ${generated}`);
|
||||
if (doFix && !dryRun) {
|
||||
console.log(` Written: ${written} (with .bak backups)`);
|
||||
}
|
||||
|
||||
// Show sample by type
|
||||
const byType: Record<string, number> = {};
|
||||
for (const r of results) {
|
||||
byType[r.type] = (byType[r.type] || 0) + 1;
|
||||
}
|
||||
if (Object.keys(byType).length > 0) {
|
||||
console.log(`\n By type:`);
|
||||
for (const [type, count] of Object.entries(byType).sort(([, a], [, b]) => b - a)) {
|
||||
console.log(` ${type}: ${count}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Show first 10 examples
|
||||
if (results.length > 0 && (!doFix || dryRun)) {
|
||||
console.log(`\n Examples:`);
|
||||
for (const r of results.slice(0, 10)) {
|
||||
console.log(` ${r.path}`);
|
||||
console.log(` → type: ${r.type}, title: "${r.title}"${r.date ? `, date: ${r.date}` : ''} [rule: ${r.rule}]`);
|
||||
}
|
||||
if (results.length > 10) {
|
||||
console.log(` ... and ${results.length - 10} more`);
|
||||
}
|
||||
if (!doFix) {
|
||||
console.log(`\n To write: gbrain frontmatter generate ${targetPath} --fix`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
import { readdirSync, lstatSync, existsSync, writeFileSync, readFileSync, unlinkSync } from 'fs';
|
||||
import { execFileSync } from 'child_process';
|
||||
import { join, relative } from 'path';
|
||||
import { cpus, totalmem } from 'os';
|
||||
import { cpus, totalmem, homedir } from 'os';
|
||||
import type { BrainEngine } from '../core/engine.ts';
|
||||
import { importFile } from '../core/import-file.ts';
|
||||
import { loadConfig, gbrainPath } from '../core/config.ts';
|
||||
import { loadConfig } from '../core/config.ts';
|
||||
import { createProgress } from '../core/progress.ts';
|
||||
import { getCliOptions, cliOptsToProgressOptions } from '../core/cli-options.ts';
|
||||
|
||||
@@ -61,7 +61,7 @@ export async function runImport(engine: BrainEngine, args: string[], opts: { com
|
||||
console.log(`Found ${allFiles.length} markdown files`);
|
||||
|
||||
// Resume from checkpoint if available
|
||||
const checkpointPath = gbrainPath('import-checkpoint.json');
|
||||
const checkpointPath = join(homedir(), '.gbrain', 'import-checkpoint.json');
|
||||
let files = allFiles;
|
||||
let resumeIndex = 0;
|
||||
|
||||
@@ -137,7 +137,7 @@ export async function runImport(engine: BrainEngine, args: string[], opts: { com
|
||||
// Save checkpoint every 100 files — track completed file set, not just a counter
|
||||
if (processed % 100 === 0) {
|
||||
try {
|
||||
const cpDir = gbrainPath();
|
||||
const cpDir = join(homedir(), '.gbrain');
|
||||
if (!existsSync(cpDir)) { const { mkdirSync } = await import('fs'); mkdirSync(cpDir, { recursive: true }); }
|
||||
writeFileSync(checkpointPath, JSON.stringify({
|
||||
dir, totalFiles: allFiles.length,
|
||||
|
||||
@@ -6,7 +6,7 @@ import { homedir } from 'os';
|
||||
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = dirname(__filename);
|
||||
import { saveConfig, loadConfig, toEngineConfig, gbrainPath, type GBrainConfig } from '../core/config.ts';
|
||||
import { saveConfig, loadConfig, toEngineConfig, type GBrainConfig } from '../core/config.ts';
|
||||
import { createEngine } from '../core/engine-factory.ts';
|
||||
|
||||
export async function runInit(args: string[]) {
|
||||
@@ -103,7 +103,7 @@ async function initMigrateOnly(opts: { jsonOutput: boolean }) {
|
||||
}
|
||||
|
||||
async function initPGLite(opts: { jsonOutput: boolean; apiKey: string | null; customPath: string | null }) {
|
||||
const dbPath = opts.customPath || gbrainPath('brain.pglite');
|
||||
const dbPath = opts.customPath || join(homedir(), '.gbrain', 'brain.pglite');
|
||||
console.log(`Setting up local brain with PGLite (no server needed)...`);
|
||||
|
||||
const engine = await createEngine({ engine: 'pglite' });
|
||||
|
||||
@@ -23,7 +23,6 @@ import matter from 'gray-matter';
|
||||
import { readFileSync, existsSync, writeFileSync, mkdirSync, readdirSync } from 'fs';
|
||||
import { join, basename } from 'path';
|
||||
import { homedir } from 'os';
|
||||
import { gbrainPath } from '../core/config.ts';
|
||||
import { execSync } from 'child_process';
|
||||
|
||||
// --- Types ---
|
||||
@@ -513,7 +512,7 @@ function findRecipe(id: string): ParsedRecipe | null {
|
||||
// --- Heartbeat ---
|
||||
|
||||
function heartbeatDir(id: string): string {
|
||||
return gbrainPath('integrations', id);
|
||||
return join(homedir(), '.gbrain', 'integrations', id);
|
||||
}
|
||||
|
||||
function heartbeatPath(id: string): string {
|
||||
|
||||
+25
-24
@@ -25,9 +25,10 @@
|
||||
*/
|
||||
|
||||
import { appendFileSync, existsSync, readFileSync, mkdirSync, writeFileSync } from 'fs';
|
||||
import { dirname } from 'path';
|
||||
import { homedir } from 'os';
|
||||
import { join, dirname } from 'path';
|
||||
|
||||
import { loadConfig, toEngineConfig, gbrainPath } from '../core/config.ts';
|
||||
import { loadConfig, toEngineConfig } from '../core/config.ts';
|
||||
import { createEngine } from '../core/engine-factory.ts';
|
||||
import type { BrainEngine } from '../core/engine.ts';
|
||||
import * as db from '../core/db.ts';
|
||||
@@ -44,10 +45,10 @@ import { tweetCitation } from '../core/output/scaffold.ts';
|
||||
// Paths
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// Lazy: GBRAIN_HOME may be set after module load.
|
||||
const getReviewFile = () => gbrainPath('integrity-review.md');
|
||||
const getLogFile = () => gbrainPath('integrity.log.jsonl');
|
||||
const getProgressFile = () => gbrainPath('integrity-progress.jsonl');
|
||||
const GBRAIN_DIR = join(homedir(), '.gbrain');
|
||||
const REVIEW_FILE = join(GBRAIN_DIR, 'integrity-review.md');
|
||||
const LOG_FILE = join(GBRAIN_DIR, 'integrity.log.jsonl');
|
||||
const PROGRESS_FILE = join(GBRAIN_DIR, 'integrity-progress.jsonl');
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Bare-tweet detection
|
||||
@@ -157,9 +158,9 @@ interface ProgressEntry {
|
||||
}
|
||||
|
||||
function loadProgress(): Set<string> {
|
||||
if (!existsSync(getProgressFile())) return new Set();
|
||||
if (!existsSync(PROGRESS_FILE)) return new Set();
|
||||
const seen = new Set<string>();
|
||||
const content = readFileSync(getProgressFile(), 'utf-8');
|
||||
const content = readFileSync(PROGRESS_FILE, 'utf-8');
|
||||
for (const line of content.split('\n')) {
|
||||
if (!line.trim()) continue;
|
||||
try {
|
||||
@@ -173,12 +174,12 @@ function loadProgress(): Set<string> {
|
||||
}
|
||||
|
||||
function appendProgress(entry: ProgressEntry): void {
|
||||
ensureDir(getProgressFile());
|
||||
appendFileSync(getProgressFile(), JSON.stringify(entry) + '\n', 'utf-8');
|
||||
ensureDir(PROGRESS_FILE);
|
||||
appendFileSync(PROGRESS_FILE, JSON.stringify(entry) + '\n', 'utf-8');
|
||||
}
|
||||
|
||||
function clearProgress(): void {
|
||||
if (existsSync(getProgressFile())) writeFileSync(getProgressFile(), '', 'utf-8');
|
||||
if (existsSync(PROGRESS_FILE)) writeFileSync(PROGRESS_FILE, '', 'utf-8');
|
||||
}
|
||||
|
||||
function ensureDir(path: string): void {
|
||||
@@ -212,7 +213,7 @@ export async function runIntegrity(args: string[]): Promise<void> {
|
||||
}
|
||||
if (sub === 'reset-progress') {
|
||||
clearProgress();
|
||||
console.log('Cleared progress log:', getProgressFile());
|
||||
console.log('Cleared progress log:', PROGRESS_FILE);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -408,7 +409,7 @@ async function cmdAuto(args: string[]): Promise<void> {
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
ensureDir(gbrainPath());
|
||||
ensureDir(GBRAIN_DIR);
|
||||
|
||||
const engine = await connect();
|
||||
const registry = getDefaultRegistry();
|
||||
@@ -547,9 +548,9 @@ async function cmdAuto(args: string[]): Promise<void> {
|
||||
console.log(`Review queue (≥${reviewLower} <${confidenceThreshold}): ${bucketReview}`);
|
||||
console.log(`Skipped (<${reviewLower}): ${bucketSkip}`);
|
||||
if (bucketErr > 0) console.log(`Resolver errors: ${bucketErr}`);
|
||||
console.log(`\nReview queue: ${getReviewFile()}`);
|
||||
console.log(`Skipped log: ${getLogFile()}`);
|
||||
console.log(`Progress: ${getProgressFile()}`);
|
||||
console.log(`\nReview queue: ${REVIEW_FILE}`);
|
||||
console.log(`Skipped log: ${LOG_FILE}`);
|
||||
console.log(`Progress: ${PROGRESS_FILE}`);
|
||||
} finally {
|
||||
await engine.disconnect();
|
||||
}
|
||||
@@ -560,15 +561,15 @@ async function cmdAuto(args: string[]): Promise<void> {
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function cmdReview(): void {
|
||||
if (!existsSync(getReviewFile())) {
|
||||
if (!existsSync(REVIEW_FILE)) {
|
||||
console.log(`No review queue yet. Run: gbrain integrity auto --confidence 0.8`);
|
||||
return;
|
||||
}
|
||||
const content = readFileSync(getReviewFile(), 'utf-8');
|
||||
const content = readFileSync(REVIEW_FILE, 'utf-8');
|
||||
const count = (content.match(/^## /gm) ?? []).length;
|
||||
console.log(`Review queue: ${getReviewFile()}`);
|
||||
console.log(`Review queue: ${REVIEW_FILE}`);
|
||||
console.log(`Entries: ${count}`);
|
||||
console.log(`\nOpen with: $EDITOR ${getReviewFile()}`);
|
||||
console.log(`\nOpen with: $EDITOR ${REVIEW_FILE}`);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -649,7 +650,7 @@ interface ReviewArgs {
|
||||
}
|
||||
|
||||
function appendReview(args: ReviewArgs): void {
|
||||
ensureDir(getReviewFile());
|
||||
ensureDir(REVIEW_FILE);
|
||||
const { slug, hit, result, handle } = args;
|
||||
const block = [
|
||||
`## ${slug}:${hit.line} (confidence ${result.confidence.toFixed(2)})`,
|
||||
@@ -663,12 +664,12 @@ function appendReview(args: ReviewArgs): void {
|
||||
'---',
|
||||
'',
|
||||
].join('\n');
|
||||
appendFileSync(getReviewFile(), block, 'utf-8');
|
||||
appendFileSync(REVIEW_FILE, block, 'utf-8');
|
||||
}
|
||||
|
||||
interface SkipArgs { slug: string; hit: BareTweetHit; reason: string }
|
||||
function logSkip(args: SkipArgs): void {
|
||||
ensureDir(getLogFile());
|
||||
ensureDir(LOG_FILE);
|
||||
const entry = {
|
||||
timestamp: new Date().toISOString(),
|
||||
slug: args.slug,
|
||||
@@ -677,7 +678,7 @@ function logSkip(args: SkipArgs): void {
|
||||
raw: args.hit.rawLine.slice(0, 200),
|
||||
reason: args.reason,
|
||||
};
|
||||
appendFileSync(getLogFile(), JSON.stringify(entry) + '\n', 'utf-8');
|
||||
appendFileSync(LOG_FILE, JSON.stringify(entry) + '\n', 'utf-8');
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
+17
-96
@@ -33,14 +33,14 @@ export function parseMaxWaitingFlag(args: string[]): number | undefined {
|
||||
}
|
||||
|
||||
/** Parse `--max-rss N` (MB). Returns:
|
||||
* - undefined if the flag is absent (caller decides the default)
|
||||
* - 0 if the flag is absent (no watchdog by default for bare `jobs work`)
|
||||
* - 0 if `--max-rss 0` (explicit disable)
|
||||
* - the value if >= 256
|
||||
* Errors and exits the process if the flag is non-numeric, negative, or
|
||||
* positive but < 256 (likely a GB-vs-MB unit-confusion typo). */
|
||||
export function parseMaxRssFlag(args: string[]): number | undefined {
|
||||
export function parseMaxRssFlag(args: string[]): number {
|
||||
const raw = parseFlag(args, '--max-rss');
|
||||
if (raw === undefined) return undefined;
|
||||
if (raw === undefined) return 0;
|
||||
const parsed = parseInt(raw, 10);
|
||||
if (!Number.isFinite(parsed) || parsed < 0) {
|
||||
console.error(`Error: --max-rss must be a non-negative integer (MB), got "${raw}"`);
|
||||
@@ -133,7 +133,6 @@ USAGE
|
||||
gbrain jobs stats
|
||||
gbrain jobs smoke
|
||||
gbrain jobs work [--queue Q] [--concurrency N] [--max-rss MB]
|
||||
[--health-interval MS]
|
||||
gbrain jobs supervisor [start] [--detach] [--json]
|
||||
[--concurrency N] [--queue Q] [--pid-file PATH]
|
||||
[--max-crashes N] [--health-interval N]
|
||||
@@ -315,15 +314,8 @@ HANDLER TYPES (built in)
|
||||
|
||||
if (follow) {
|
||||
console.log(`Job #${job.id} submitted (${name}). Executing inline...`);
|
||||
// Inline execution: run the job in this process. Disable the
|
||||
// self-health-check timer — inline flows are one-shot and don't have
|
||||
// a process manager to restart them. With the timer enabled and no
|
||||
// 'unhealthy' listener, a DB blip would trip emitUnhealthy's
|
||||
// no-listener fallback and call process.exit(1) from inside the
|
||||
// library, killing the user's CLI session.
|
||||
const worker = new MinionWorker(engine, {
|
||||
queue: queueName, pollInterval: 100, healthCheckInterval: 0,
|
||||
});
|
||||
// Inline execution: run the job in this process
|
||||
const worker = new MinionWorker(engine, { queue: queueName, pollInterval: 100 });
|
||||
|
||||
// Register built-in handlers
|
||||
await registerBuiltinHandlers(worker, engine);
|
||||
@@ -497,11 +489,7 @@ HANDLER TYPES (built in)
|
||||
const sigkillRescue = hasFlag(args, '--sigkill-rescue');
|
||||
const wedgeRescue = hasFlag(args, '--wedge-rescue');
|
||||
|
||||
// Smoke harness is short-lived and has no listener — disable the health
|
||||
// timer so the no-listener fallback can't trip process.exit(1) mid-test.
|
||||
const worker = new MinionWorker(engine, {
|
||||
queue: 'smoke', pollInterval: 100, healthCheckInterval: 0,
|
||||
});
|
||||
const worker = new MinionWorker(engine, { queue: 'smoke', pollInterval: 100 });
|
||||
worker.register('noop', async () => ({ ok: true, at: new Date().toISOString() }));
|
||||
|
||||
const job = await queue.add('noop', {}, { queue: 'smoke', max_attempts: 1 });
|
||||
@@ -650,69 +638,19 @@ HANDLER TYPES (built in)
|
||||
|
||||
const queueName = parseFlag(args, '--queue') ?? 'default';
|
||||
const concurrency = resolveWorkerConcurrency(args);
|
||||
// --max-rss defaults to 2048 for bare workers (matching supervisor default).
|
||||
// This catches memory-leak stalls that previously went undetected without
|
||||
// a supervisor. Operators can opt out with `--max-rss 0`.
|
||||
const maxRssExplicit = parseMaxRssFlag(args);
|
||||
const maxRssMb = maxRssExplicit ?? 2048;
|
||||
|
||||
// --health-interval: self-health-check period in ms. 0 disables. Default: 60_000 (60s).
|
||||
// Provides DB liveness probes + stall detection for bare workers.
|
||||
// Automatically skipped when running under a supervisor (GBRAIN_SUPERVISED=1).
|
||||
// Validated aggressively (parity with --max-rss): reject NaN/negative/non-integer
|
||||
// values, and reject suspicious sub-1000ms values that are likely a unit-confusion
|
||||
// typo (e.g. "--health-interval 60" thinking the unit is seconds).
|
||||
const healthRaw = parseFlag(args, '--health-interval');
|
||||
let healthCheckInterval = 60_000;
|
||||
if (healthRaw !== undefined) {
|
||||
const parsed = parseInt(healthRaw, 10);
|
||||
if (!Number.isFinite(parsed) || parsed < 0) {
|
||||
console.error(`Error: --health-interval must be a non-negative integer (ms), got "${healthRaw}"`);
|
||||
process.exit(1);
|
||||
}
|
||||
if (parsed > 0 && parsed < 1000) {
|
||||
console.error(
|
||||
`Error: --health-interval ${parsed} is suspiciously low (likely a unit-confusion typo). ` +
|
||||
`The flag takes milliseconds; for 60-second probes pass 60000. Use 0 to disable.`,
|
||||
);
|
||||
process.exit(1);
|
||||
}
|
||||
healthCheckInterval = parsed;
|
||||
}
|
||||
// --max-rss is opt-in for bare `gbrain jobs work` — preserves pre-v0.21 behavior
|
||||
// for operators with legitimately large embed/import working sets. The supervisor
|
||||
// path injects a default 2048; this code path does not.
|
||||
const maxRssMb = parseMaxRssFlag(args);
|
||||
|
||||
try { await queue.ensureSchema(); }
|
||||
catch (e) { console.error(e instanceof Error ? e.message : String(e)); process.exit(1); }
|
||||
|
||||
const worker = new MinionWorker(engine, {
|
||||
queue: queueName, concurrency, maxRssMb, healthCheckInterval,
|
||||
});
|
||||
const worker = new MinionWorker(engine, { queue: queueName, concurrency, maxRssMb });
|
||||
await registerBuiltinHandlers(worker, engine);
|
||||
|
||||
// Subscribe to self-health failures emitted by the worker. Library code
|
||||
// (worker.ts) never calls process.exit directly so it stays embeddable;
|
||||
// this CLI layer is the right place to terminate the process and let
|
||||
// the external PM (systemd, Docker, cron watchdog) restart cleanly.
|
||||
worker.on('unhealthy', (info) => {
|
||||
if (info.reason === 'db_dead') {
|
||||
console.error(
|
||||
`[health] FATAL: DB unreachable after ${info.consecutiveFailures} probes (${info.message}). ` +
|
||||
`Exiting for process-manager restart.`,
|
||||
);
|
||||
} else {
|
||||
console.error(
|
||||
`[health] FATAL: Worker stalled — ${info.waitingCount} waiting job(s) for ` +
|
||||
`registered handlers, ${info.idleMinutes}m idle. Exiting for process-manager restart.`,
|
||||
);
|
||||
}
|
||||
process.exit(1);
|
||||
});
|
||||
|
||||
const isSupervisedChild = process.env.GBRAIN_SUPERVISED === '1';
|
||||
const watchdogNote = maxRssMb > 0 ? `, watchdog: ${maxRssMb}MB` : '';
|
||||
const healthNote = !isSupervisedChild && healthCheckInterval > 0
|
||||
? `, health-check: ${Math.round(healthCheckInterval / 1000)}s`
|
||||
: '';
|
||||
console.log(`Minion worker started (queue: ${queueName}, concurrency: ${concurrency}${watchdogNote}${healthNote})`);
|
||||
console.log(`Minion worker started (queue: ${queueName}, concurrency: ${concurrency}${watchdogNote})`);
|
||||
console.log(`Registered handlers: ${worker.registeredNames.join(', ')}`);
|
||||
await worker.start();
|
||||
break;
|
||||
@@ -849,32 +787,15 @@ HANDLER TYPES (built in)
|
||||
const concurrency = parseInt(parseFlag(args, '--concurrency') ?? '2', 10);
|
||||
const queueName = parseFlag(args, '--queue') ?? 'default';
|
||||
const maxCrashes = parseInt(parseFlag(args, '--max-crashes') ?? '10', 10);
|
||||
// --health-interval (supervisor): validate same as `jobs work` so NaN /
|
||||
// negative / sub-1000ms typos fail-fast instead of silently disabling
|
||||
// the supervisor's own health probe.
|
||||
const supHealthRaw = parseFlag(args, '--health-interval');
|
||||
let healthInterval = 60_000;
|
||||
if (supHealthRaw !== undefined) {
|
||||
const parsed = parseInt(supHealthRaw, 10);
|
||||
if (!Number.isFinite(parsed) || parsed < 0) {
|
||||
console.error(`Error: --health-interval must be a non-negative integer (ms), got "${supHealthRaw}"`);
|
||||
process.exit(1);
|
||||
}
|
||||
if (parsed > 0 && parsed < 1000) {
|
||||
console.error(
|
||||
`Error: --health-interval ${parsed} is suspiciously low (likely a unit-confusion typo). ` +
|
||||
`The flag takes milliseconds; for 60-second probes pass 60000. Use 0 to disable.`,
|
||||
);
|
||||
process.exit(1);
|
||||
}
|
||||
healthInterval = parsed;
|
||||
}
|
||||
const healthInterval = parseInt(parseFlag(args, '--health-interval') ?? '60000', 10);
|
||||
const allowShellJobs = hasFlag(args, '--allow-shell-jobs') ||
|
||||
!!process.env.GBRAIN_ALLOW_SHELL_JOBS;
|
||||
const detach = hasFlag(args, '--detach');
|
||||
// Supervisor defaults --max-rss 2048 (MB) — main production path uses
|
||||
// the supervisor, so the watchdog is on by default here.
|
||||
const maxRssMb = parseMaxRssFlag(args) ?? 2048;
|
||||
// the supervisor, so the watchdog is on by default here. parseMaxRssFlag
|
||||
// returns 0 when the flag is absent; substitute the supervisor default.
|
||||
const maxRssRaw = parseMaxRssFlag(args);
|
||||
const maxRssMb = parseFlag(args, '--max-rss') === undefined ? 2048 : maxRssRaw;
|
||||
|
||||
const cliPath = parseFlag(args, '--cli-path') ?? resolveGbrainCliPath();
|
||||
|
||||
|
||||
@@ -8,9 +8,11 @@
|
||||
*/
|
||||
|
||||
import { createEngine } from '../core/engine-factory.ts';
|
||||
import { loadConfig, saveConfig, toEngineConfig, gbrainPath, type GBrainConfig } from '../core/config.ts';
|
||||
import { loadConfig, saveConfig, toEngineConfig, type GBrainConfig } from '../core/config.ts';
|
||||
import type { BrainEngine } from '../core/engine.ts';
|
||||
import type { EngineConfig } from '../core/types.ts';
|
||||
import { homedir } from 'os';
|
||||
import { join } from 'path';
|
||||
import { writeFileSync, readFileSync, existsSync, unlinkSync } from 'fs';
|
||||
import { createProgress } from '../core/progress.ts';
|
||||
import { getCliOptions, cliOptsToProgressOptions } from '../core/cli-options.ts';
|
||||
@@ -46,7 +48,7 @@ function parseArgs(args: string[]): MigrateOpts {
|
||||
}
|
||||
|
||||
function getManifestPath(): string {
|
||||
return gbrainPath('migrate-manifest.json');
|
||||
return join(homedir(), '.gbrain', 'migrate-manifest.json');
|
||||
}
|
||||
|
||||
interface MigrateManifest {
|
||||
@@ -97,7 +99,7 @@ export async function runMigrateEngine(sourceEngine: BrainEngine, args: string[]
|
||||
process.exit(1);
|
||||
}
|
||||
} else {
|
||||
targetConfig.database_path = opts.targetPath || gbrainPath('brain.pglite');
|
||||
targetConfig.database_path = opts.targetPath || join(homedir(), '.gbrain', 'brain.pglite');
|
||||
}
|
||||
|
||||
// Connect to target
|
||||
|
||||
@@ -35,17 +35,17 @@
|
||||
*/
|
||||
|
||||
import { existsSync, mkdirSync, appendFileSync } from 'fs';
|
||||
import { homedir } from 'os';
|
||||
import { join } from 'path';
|
||||
|
||||
import type { Migration, OrchestratorOpts, OrchestratorResult, OrchestratorPhaseResult } from './types.ts';
|
||||
import { loadConfig, toEngineConfig, gbrainPath } from '../../core/config.ts';
|
||||
import { loadConfig, toEngineConfig } from '../../core/config.ts';
|
||||
import { createEngine } from '../../core/engine-factory.ts';
|
||||
import type { BrainEngine } from '../../core/engine.ts';
|
||||
// Bug 3 — ledger writes moved to the runner (apply-migrations.ts).
|
||||
|
||||
// Lazy: GBRAIN_HOME may be set after module load.
|
||||
const getRollbackDir = () => gbrainPath('migrations');
|
||||
const getRollbackFile = () => join(getRollbackDir(), 'v0_13_1-rollback.jsonl');
|
||||
const ROLLBACK_DIR = join(homedir(), '.gbrain', 'migrations');
|
||||
const ROLLBACK_FILE = join(ROLLBACK_DIR, 'v0_13_1-rollback.jsonl');
|
||||
const BATCH_SIZE = 100;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -251,8 +251,7 @@ async function orchestrator(opts: OrchestratorOpts): Promise<OrchestratorResult>
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function ensureRollbackDir(): void {
|
||||
const dir = getRollbackDir();
|
||||
if (!existsSync(dir)) mkdirSync(dir, { recursive: true });
|
||||
if (!existsSync(ROLLBACK_DIR)) mkdirSync(ROLLBACK_DIR, { recursive: true });
|
||||
}
|
||||
|
||||
function appendRollbackEntry(entry: { slug: string; pre_frontmatter: Record<string, unknown> }): void {
|
||||
@@ -261,7 +260,7 @@ function appendRollbackEntry(entry: { slug: string; pre_frontmatter: Record<stri
|
||||
timestamp: new Date().toISOString(),
|
||||
...entry,
|
||||
}) + '\n';
|
||||
appendFileSync(getRollbackFile(), line, 'utf-8');
|
||||
appendFileSync(ROLLBACK_FILE, line, 'utf-8');
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
@@ -22,17 +22,19 @@
|
||||
*/
|
||||
|
||||
import { existsSync, readFileSync, mkdirSync, appendFileSync } from 'fs';
|
||||
import { homedir } from 'os';
|
||||
import { join } from 'path';
|
||||
|
||||
import type { Migration, OrchestratorOpts, OrchestratorResult, OrchestratorPhaseResult } from './types.ts';
|
||||
import { loadConfig, toEngineConfig, gbrainPath } from '../../core/config.ts';
|
||||
import { loadConfig, toEngineConfig } from '../../core/config.ts';
|
||||
import { createEngine } from '../../core/engine-factory.ts';
|
||||
import type { BrainEngine } from '../../core/engine.ts';
|
||||
|
||||
// gbrainPath() honors GBRAIN_HOME at call time (not module-load) and routes
|
||||
// through the centralized config dir, so the prior resolveHome()/HOME-env
|
||||
// trick is no longer needed.
|
||||
function pendingHostWorkDir(): string { return gbrainPath('migrations'); }
|
||||
// Resolve HOME at CALL time, not module-load time — Bun caches os.homedir()
|
||||
// and ignores later HOME mutations, which breaks test isolation and scripted
|
||||
// installs. Match the preferences.ts pattern.
|
||||
function resolveHome(): string { return process.env.HOME || homedir(); }
|
||||
function pendingHostWorkDir(): string { return join(resolveHome(), '.gbrain', 'migrations'); }
|
||||
function pendingHostWorkPath(): string { return join(pendingHostWorkDir(), 'pending-host-work.jsonl'); }
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
@@ -1,113 +0,0 @@
|
||||
/**
|
||||
* AgentRunner — pluggable contract for invoking external agents (openclaw,
|
||||
* hermes, codex, …) inside the claw-test harness. v1 ships a single
|
||||
* implementation (openclaw); the interface stays narrow and concrete so
|
||||
* adding a second runner in v1.1 is a ~50-line file.
|
||||
*
|
||||
* The harness wraps spawn/timeout/transcript-capture; runners only have to
|
||||
* answer "where's your binary?" and "how do I invoke it with this prompt?".
|
||||
*
|
||||
* ┌────────────────────┐
|
||||
* │ harness │
|
||||
* │ ─ resolve(name) ─▶│ registry → AgentRunner instance
|
||||
* │ ─ detect() ─▶│ runner reports binary path/availability
|
||||
* │ ─ invoke(...) ─▶│ runner spawns child, harness captures via TranscriptSink
|
||||
* └────────────────────┘
|
||||
*/
|
||||
|
||||
export interface AgentRunner {
|
||||
/** Stable agent name used by --agent flag and friction `agent` field. */
|
||||
readonly name: string;
|
||||
|
||||
/**
|
||||
* Locate the agent binary and confirm it is executable. Pure check; never
|
||||
* spawns. `binPath` is always an absolute path on success. `available=false`
|
||||
* with a `reason` if not found / not executable.
|
||||
*/
|
||||
detect(): Promise<DetectResult>;
|
||||
|
||||
/**
|
||||
* Invoke the agent with the given prompt. The runner is responsible for
|
||||
* the per-agent argv shape. The harness owns timeouts, signals, and
|
||||
* transcript capture (via `transcriptSink`).
|
||||
*/
|
||||
invoke(opts: InvokeOpts): Promise<InvokeResult>;
|
||||
|
||||
/** Optional per-agent post-install hook (e.g., routing-file fixup). */
|
||||
postInstallHook?(opts: { workspaceDir: string }): Promise<void>;
|
||||
}
|
||||
|
||||
export interface DetectResult {
|
||||
available: boolean;
|
||||
reason?: string;
|
||||
binPath?: string;
|
||||
}
|
||||
|
||||
export interface InvokeOpts {
|
||||
/** Workspace dir the agent runs in. */
|
||||
cwd: string;
|
||||
/** The prompt content. The runner decides whether to write a temp file or pass via argv. */
|
||||
brief: string;
|
||||
/** Env to merge with the runner's defaults. Caller already restricted to allow-listed keys. */
|
||||
env: Record<string, string>;
|
||||
/** Wall-clock kill switch in ms. Harness handles SIGTERM → 5s grace → SIGKILL. */
|
||||
timeoutMs: number;
|
||||
/**
|
||||
* Per-channel byte sink. The runner pipes child stdin/stdout/stderr into this
|
||||
* instead of inheriting the parent's. Async-drain backpressure is handled
|
||||
* inside the sink (D17), so the runner can call `write()` without awaiting.
|
||||
*/
|
||||
transcriptSink: TranscriptSink;
|
||||
/** Optional override for which sub-agent the runner targets. */
|
||||
agentName?: string;
|
||||
}
|
||||
|
||||
export interface InvokeResult {
|
||||
exitCode: number;
|
||||
durationMs: number;
|
||||
}
|
||||
|
||||
/** Async-drain sink. The harness owns the underlying file stream. */
|
||||
export interface TranscriptSink {
|
||||
write(event: TranscriptEvent): void;
|
||||
/** Returns the byte offset that the next written event would have. */
|
||||
nextOffset(): number;
|
||||
/** Flush + close. Idempotent. */
|
||||
close(): Promise<void>;
|
||||
}
|
||||
|
||||
export interface TranscriptEvent {
|
||||
ts: number;
|
||||
channel: 'stdin' | 'stdout' | 'stderr';
|
||||
bytes: Buffer;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Registry
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
type AgentRunnerFactory = () => AgentRunner;
|
||||
|
||||
const registry = new Map<string, AgentRunnerFactory>();
|
||||
|
||||
export function registerAgentRunner(name: string, factory: AgentRunnerFactory): void {
|
||||
registry.set(name, factory);
|
||||
}
|
||||
|
||||
export function resolveAgentRunner(name: string): AgentRunner {
|
||||
const factory = registry.get(name);
|
||||
if (!factory) {
|
||||
const known = [...registry.keys()].sort().join(', ') || '(none registered)';
|
||||
throw new Error(`unknown agent ${JSON.stringify(name)}; registered: ${known}`);
|
||||
}
|
||||
return factory();
|
||||
}
|
||||
|
||||
export function listRegisteredAgents(): string[] {
|
||||
return [...registry.keys()].sort();
|
||||
}
|
||||
|
||||
/** Reset registry — testing only. */
|
||||
export function _resetRegistryForTests(): void {
|
||||
registry.clear();
|
||||
}
|
||||
@@ -1,58 +0,0 @@
|
||||
/**
|
||||
* progress-tail — parses gbrain's --progress-json events out of child stderr.
|
||||
*
|
||||
* The actual contract (verified post-Codex):
|
||||
* - `gbrain --progress-json <subcommand>` writes JSONL events to STDERR
|
||||
* - Stable phase names are dotted snake_case: `import.files`, `extract.links_fs`,
|
||||
* `embed.pages`, `doctor.db_checks`, etc.
|
||||
* - Each event line is a JSON object; non-progress stderr lines (warnings,
|
||||
* debug output, errors) interleave with progress events. We tolerate them.
|
||||
*
|
||||
* Used by the verify phase to assert that each `expected_phases` entry from
|
||||
* scenario.json saw at least one event from the corresponding command.
|
||||
*/
|
||||
|
||||
export interface ProgressEvent {
|
||||
phase: string;
|
||||
event?: string; // 'start' | 'tick' | 'finish' | etc per docs/progress-events.md
|
||||
ts?: string;
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
/** Parse a single stderr buffer into the progress events it contains. */
|
||||
export function parseProgressEvents(stderr: string): ProgressEvent[] {
|
||||
const out: ProgressEvent[] = [];
|
||||
for (const line of stderr.split('\n')) {
|
||||
const trimmed = line.trim();
|
||||
if (!trimmed.startsWith('{')) continue;
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(trimmed);
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
if (parsed && typeof parsed === 'object' && typeof (parsed as any).phase === 'string') {
|
||||
out.push(parsed as ProgressEvent);
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Group events by phase name. */
|
||||
export function eventsByPhase(events: ProgressEvent[]): Map<string, ProgressEvent[]> {
|
||||
const m = new Map<string, ProgressEvent[]>();
|
||||
for (const e of events) {
|
||||
if (!m.has(e.phase)) m.set(e.phase, []);
|
||||
m.get(e.phase)!.push(e);
|
||||
}
|
||||
return m;
|
||||
}
|
||||
|
||||
/**
|
||||
* Verify that every `expected` phase appears at least once in `events`.
|
||||
* Returns the missing phase names (empty array on full coverage).
|
||||
*/
|
||||
export function verifyExpectedPhases(events: ProgressEvent[], expected: string[]): string[] {
|
||||
const seen = new Set(events.map(e => e.phase));
|
||||
return expected.filter(p => !seen.has(p));
|
||||
}
|
||||
@@ -1,98 +0,0 @@
|
||||
/**
|
||||
* OpenClaw runner — invokes the real `openclaw` binary in a tempdir with a
|
||||
* BRIEF.md prompt. Live mode only.
|
||||
*
|
||||
* Invocation pattern (verified against test/e2e/skills.test.ts and
|
||||
* test/e2e/bench-vs-openclaw/harness.ts):
|
||||
* openclaw agent --local --agent <agent-name> --message "<brief>"
|
||||
*
|
||||
* NOT `openclaw run --prompt-file BRIEF.md` (that flag does not exist —
|
||||
* Codex pass 2 of the eng review caught the speculative shape).
|
||||
*
|
||||
* Binary resolution: $OPENCLAW_BIN > `which openclaw` > unavailable.
|
||||
* Path validation: must be absolute, must be executable, no '..' segments.
|
||||
*/
|
||||
|
||||
import { execSync } from 'child_process';
|
||||
import { statSync } from 'fs';
|
||||
import type { AgentRunner, DetectResult, InvokeOpts, InvokeResult } from '../agent-runner.ts';
|
||||
import { spawnWithCapture } from '../transcript-capture.ts';
|
||||
|
||||
const DEFAULT_AGENT_NAME = 'default';
|
||||
/** Allow-list for env propagation when spawning openclaw. */
|
||||
const ENV_ALLOWLIST = [
|
||||
'PATH', 'HOME', 'USER', 'LANG', 'TZ', 'NODE_ENV',
|
||||
'ANTHROPIC_API_KEY', 'OPENAI_API_KEY',
|
||||
'GBRAIN_HOME', 'GBRAIN_FRICTION_RUN_ID', 'GBRAIN_DATABASE_URL',
|
||||
];
|
||||
|
||||
export class OpenClawRunner implements AgentRunner {
|
||||
readonly name = 'openclaw';
|
||||
|
||||
async detect(): Promise<DetectResult> {
|
||||
const fromEnv = process.env.OPENCLAW_BIN?.trim();
|
||||
let binPath: string | undefined;
|
||||
|
||||
if (fromEnv) {
|
||||
const validation = validateAbsolutePath(fromEnv);
|
||||
if (validation) return { available: false, reason: validation };
|
||||
binPath = fromEnv;
|
||||
} else {
|
||||
try {
|
||||
const out = execSync('which openclaw', { encoding: 'utf-8', stdio: ['ignore', 'pipe', 'ignore'] });
|
||||
const found = out.trim();
|
||||
if (!found || !found.startsWith('/')) {
|
||||
return { available: false, reason: 'openclaw not on PATH' };
|
||||
}
|
||||
binPath = found;
|
||||
} catch {
|
||||
return { available: false, reason: 'openclaw not on PATH' };
|
||||
}
|
||||
}
|
||||
|
||||
if (!binPath) return { available: false, reason: 'no binary resolved' };
|
||||
|
||||
try {
|
||||
const s = statSync(binPath);
|
||||
if (!s.isFile()) return { available: false, reason: `not a regular file: ${binPath}` };
|
||||
// eslint-disable-next-line no-bitwise
|
||||
if (!(s.mode & 0o111)) return { available: false, reason: `not executable: ${binPath}` };
|
||||
} catch (e) {
|
||||
return { available: false, reason: `stat failed: ${e instanceof Error ? e.message : String(e)}` };
|
||||
}
|
||||
|
||||
return { available: true, binPath };
|
||||
}
|
||||
|
||||
async invoke(opts: InvokeOpts): Promise<InvokeResult> {
|
||||
const detected = await this.detect();
|
||||
if (!detected.available || !detected.binPath) {
|
||||
throw new Error(`openclaw runner unavailable: ${detected.reason ?? 'unknown'}`);
|
||||
}
|
||||
const agentName = opts.agentName ?? DEFAULT_AGENT_NAME;
|
||||
const args = ['agent', '--local', '--agent', agentName, '--message', opts.brief];
|
||||
|
||||
// Filter env to allow-list, then merge caller overrides.
|
||||
const baseEnv: Record<string, string> = {};
|
||||
for (const key of ENV_ALLOWLIST) {
|
||||
const v = process.env[key];
|
||||
if (typeof v === 'string') baseEnv[key] = v;
|
||||
}
|
||||
const env: Record<string, string> = { ...baseEnv, ...opts.env };
|
||||
|
||||
const result = await spawnWithCapture(detected.binPath, args, {
|
||||
cwd: opts.cwd,
|
||||
env,
|
||||
timeoutMs: opts.timeoutMs,
|
||||
transcriptSink: opts.transcriptSink,
|
||||
});
|
||||
|
||||
return { exitCode: result.exitCode, durationMs: result.durationMs };
|
||||
}
|
||||
}
|
||||
|
||||
function validateAbsolutePath(p: string): string | null {
|
||||
if (!p.startsWith('/')) return `OPENCLAW_BIN must be absolute; got ${p}`;
|
||||
if (p.split('/').includes('..')) return `OPENCLAW_BIN must not contain '..' segments; got ${p}`;
|
||||
return null;
|
||||
}
|
||||
@@ -1,114 +0,0 @@
|
||||
/**
|
||||
* scenario.json loader for the claw-test harness.
|
||||
*
|
||||
* test/fixtures/claw-test-scenarios/<name>/scenario.json:
|
||||
* { kind: "fresh-install", expected_phases: ["import.files", ...], ... }
|
||||
*
|
||||
* The harness reads scenario.json to know which phases to assert from
|
||||
* gbrain's --progress-json events. Pure local fs; no DB, no network.
|
||||
*/
|
||||
|
||||
import { existsSync, readdirSync, readFileSync, statSync } from 'fs';
|
||||
import { dirname, join, resolve } from 'path';
|
||||
import { fileURLToPath } from 'url';
|
||||
|
||||
export type ScenarioKind = 'fresh-install' | 'upgrade';
|
||||
|
||||
export interface ScenarioConfig {
|
||||
/** Directory the scenario was loaded from. Always absolute. */
|
||||
dir: string;
|
||||
/** Stable scenario name (the directory name). */
|
||||
name: string;
|
||||
/** Kind of scenario; drives setup-phase behavior. */
|
||||
kind: ScenarioKind;
|
||||
/** Stable phase names emitted by --progress-json that the harness asserts. */
|
||||
expectedPhases: string[];
|
||||
/** When kind==="upgrade": version we are simulating an upgrade FROM. */
|
||||
fromVersion?: string;
|
||||
/** Optional human-readable summary. */
|
||||
description?: string;
|
||||
/** Path to BRIEF.md (relative to scenario dir, default 'BRIEF.md'). */
|
||||
briefRelative: string;
|
||||
/** Path to brain markdown source (relative to scenario dir). For 'fresh-install': 'brain'. */
|
||||
brainRelative?: string;
|
||||
/** Path to seed dir for upgrade scenarios. */
|
||||
seedRelative?: string;
|
||||
}
|
||||
|
||||
/** Default fixtures root, override via $GBRAIN_CLAW_SCENARIOS_DIR for tests. */
|
||||
function defaultFixturesRoot(): string {
|
||||
if (process.env.GBRAIN_CLAW_SCENARIOS_DIR) {
|
||||
return resolve(process.env.GBRAIN_CLAW_SCENARIOS_DIR);
|
||||
}
|
||||
// src/core/claw-test/scenarios.ts → ../../../test/fixtures/claw-test-scenarios
|
||||
const here = dirname(fileURLToPath(import.meta.url));
|
||||
return resolve(here, '..', '..', '..', 'test', 'fixtures', 'claw-test-scenarios');
|
||||
}
|
||||
|
||||
/** List all available scenario names. */
|
||||
export function listScenarios(root?: string): string[] {
|
||||
const r = root ?? defaultFixturesRoot();
|
||||
if (!existsSync(r)) return [];
|
||||
return readdirSync(r)
|
||||
.filter(name => {
|
||||
const path = join(r, name);
|
||||
try {
|
||||
return statSync(path).isDirectory() && existsSync(join(path, 'scenario.json'));
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
})
|
||||
.sort();
|
||||
}
|
||||
|
||||
/** Load and validate one scenario by name. */
|
||||
export function loadScenario(name: string, root?: string): ScenarioConfig {
|
||||
const r = root ?? defaultFixturesRoot();
|
||||
const dir = join(r, name);
|
||||
const cfgPath = join(dir, 'scenario.json');
|
||||
if (!existsSync(cfgPath)) {
|
||||
throw new Error(`scenario ${JSON.stringify(name)} not found at ${cfgPath}`);
|
||||
}
|
||||
let raw: unknown;
|
||||
try {
|
||||
raw = JSON.parse(readFileSync(cfgPath, 'utf-8'));
|
||||
} catch (e) {
|
||||
throw new Error(`scenario ${JSON.stringify(name)}: malformed scenario.json (${e instanceof Error ? e.message : e})`);
|
||||
}
|
||||
if (!raw || typeof raw !== 'object') {
|
||||
throw new Error(`scenario ${JSON.stringify(name)}: scenario.json must be a JSON object`);
|
||||
}
|
||||
const cfg = raw as Record<string, unknown>;
|
||||
if (cfg.kind !== 'fresh-install' && cfg.kind !== 'upgrade') {
|
||||
throw new Error(`scenario ${JSON.stringify(name)}: unknown kind ${JSON.stringify(cfg.kind)}`);
|
||||
}
|
||||
if (!Array.isArray(cfg.expected_phases) || !cfg.expected_phases.every(x => typeof x === 'string')) {
|
||||
throw new Error(`scenario ${JSON.stringify(name)}: expected_phases must be a string[]`);
|
||||
}
|
||||
const briefRel = typeof cfg.brief === 'string' ? cfg.brief : 'BRIEF.md';
|
||||
if (!existsSync(join(dir, briefRel))) {
|
||||
throw new Error(`scenario ${JSON.stringify(name)}: BRIEF.md missing at ${briefRel}`);
|
||||
}
|
||||
const out: ScenarioConfig = {
|
||||
dir,
|
||||
name,
|
||||
kind: cfg.kind,
|
||||
expectedPhases: cfg.expected_phases as string[],
|
||||
briefRelative: briefRel,
|
||||
};
|
||||
if (typeof cfg.from_version === 'string') out.fromVersion = cfg.from_version;
|
||||
if (typeof cfg.description === 'string') out.description = cfg.description;
|
||||
if (typeof cfg.brain === 'string') out.brainRelative = cfg.brain;
|
||||
if (typeof cfg.seed === 'string') out.seedRelative = cfg.seed;
|
||||
// Default brain path conventions
|
||||
if (!out.brainRelative && existsSync(join(dir, 'brain'))) out.brainRelative = 'brain';
|
||||
if (!out.seedRelative && out.kind === 'upgrade' && existsSync(join(dir, 'seed'))) {
|
||||
out.seedRelative = 'seed';
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Read BRIEF.md content for this scenario. Used by --live mode. */
|
||||
export function readBrief(scenario: ScenarioConfig): string {
|
||||
return readFileSync(join(scenario.dir, scenario.briefRelative), 'utf-8');
|
||||
}
|
||||
@@ -1,123 +0,0 @@
|
||||
/**
|
||||
* seed-pglite — replay a SQL dump into a fresh PGLite database, then let
|
||||
* gbrain's migration chain walk forward.
|
||||
*
|
||||
* Codex caught (eng review pass 2) that existing migration helpers
|
||||
* (test/e2e/helpers.ts:204) are Postgres-only — they rewind schema_version
|
||||
* and replay against real Postgres. PGLite has no equivalent. This helper
|
||||
* fills that gap so the `upgrade-from-v0.18` claw-test scenario is
|
||||
* reproducible.
|
||||
*
|
||||
* Usage:
|
||||
* const dbPath = await seedPglite('/tmp/run-x/.gbrain/brain.pglite', seedSql);
|
||||
* // Then run `gbrain init --pglite --path <dbPath>` — the migration chain
|
||||
* // detects the seeded schema_version and migrates forward to LATEST.
|
||||
*/
|
||||
|
||||
import { existsSync, mkdirSync, readFileSync } from 'fs';
|
||||
import { dirname } from 'path';
|
||||
import { PGLiteEngine } from '../pglite-engine.ts';
|
||||
|
||||
export interface SeedOpts {
|
||||
/** Absolute path to the .pglite file to create. */
|
||||
dbPath: string;
|
||||
/** Raw SQL dump to replay. */
|
||||
sql: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Open a fresh PGLite at `dbPath`, execute the SQL dump, disconnect.
|
||||
* Throws on SQL errors with a structured message that names the failing
|
||||
* statement (helpful for debugging seed drift).
|
||||
*/
|
||||
export async function seedPglite(opts: SeedOpts): Promise<void> {
|
||||
const dir = dirname(opts.dbPath);
|
||||
if (!existsSync(dir)) mkdirSync(dir, { recursive: true });
|
||||
|
||||
const engine = new PGLiteEngine();
|
||||
try {
|
||||
await engine.connect({ engine: 'pglite', database_path: opts.dbPath });
|
||||
// Execute statements one at a time so an error names the offending
|
||||
// statement. The seed file is committed to source so we can normalize
|
||||
// its line endings; we rely on `;\n` as the statement terminator.
|
||||
const statements = splitStatements(opts.sql);
|
||||
for (const stmt of statements) {
|
||||
const trimmed = stmt.trim();
|
||||
if (!trimmed) continue;
|
||||
try {
|
||||
await (engine as any).db.exec(trimmed);
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
const preview = trimmed.slice(0, 120).replace(/\s+/g, ' ');
|
||||
throw new Error(`seedPglite: SQL execution failed at "${preview}…": ${msg}`);
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
await engine.disconnect();
|
||||
}
|
||||
}
|
||||
|
||||
/** Read seed SQL from disk and replay into `dbPath`. */
|
||||
export async function seedPgliteFromFile(opts: { dbPath: string; sqlPath: string }): Promise<void> {
|
||||
if (!existsSync(opts.sqlPath)) {
|
||||
throw new Error(`seedPglite: seed SQL not found at ${opts.sqlPath}`);
|
||||
}
|
||||
const sql = readFileSync(opts.sqlPath, 'utf-8');
|
||||
return seedPglite({ dbPath: opts.dbPath, sql });
|
||||
}
|
||||
|
||||
/**
|
||||
* Split a SQL dump into individual statements. Naïve `;` split that respects
|
||||
* single-quoted strings and `--` line comments. Sufficient for canonical
|
||||
* pg_dump output; intentionally NOT a full SQL parser.
|
||||
*/
|
||||
function splitStatements(sql: string): string[] {
|
||||
const out: string[] = [];
|
||||
let buf = '';
|
||||
let inSingle = false;
|
||||
let inLineComment = false;
|
||||
let i = 0;
|
||||
while (i < sql.length) {
|
||||
const c = sql[i];
|
||||
const next = sql[i + 1];
|
||||
if (inLineComment) {
|
||||
buf += c;
|
||||
if (c === '\n') inLineComment = false;
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
if (inSingle) {
|
||||
buf += c;
|
||||
if (c === "'" && next === "'") { buf += next; i += 2; continue; }
|
||||
if (c === "'") inSingle = false;
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
if (c === '-' && next === '-') {
|
||||
inLineComment = true;
|
||||
buf += c;
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
if (c === "'") {
|
||||
inSingle = true;
|
||||
buf += c;
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
if (c === ';') {
|
||||
buf += c;
|
||||
out.push(buf);
|
||||
buf = '';
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
buf += c;
|
||||
i++;
|
||||
}
|
||||
if (buf.trim()) out.push(buf);
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Exposed for tests. */
|
||||
export const _internal = { splitStatements };
|
||||
@@ -1,172 +0,0 @@
|
||||
/**
|
||||
* Transcript capture for live-mode agent runs (D8 + D14, D17 backpressure).
|
||||
*
|
||||
* The existing minions/audit infrastructure is for INTERNAL gbrain subagents
|
||||
* only. External openclaw/hermes subprocesses don't write to those tables —
|
||||
* v1 builds its own capture channel here.
|
||||
*
|
||||
* Output: JSONL at `<run-tempdir>/transcript.jsonl`, one event per line.
|
||||
* { schema_version: "1", ts, channel, byte_offset, bytes_b64 }
|
||||
*
|
||||
* child stdout/stderr ─piped─▶ TranscriptSink.write()
|
||||
* │
|
||||
* ▼
|
||||
* fs.createWriteStream (flags: 'a')
|
||||
* ▲
|
||||
* │ honors 'drain' events to avoid blocking
|
||||
* │ the child when bursts exceed the pipe buffer
|
||||
* ▼
|
||||
* transcript.jsonl (line-tolerant readers
|
||||
* skip malformed; render() resolves
|
||||
* byte_offset → readable lines)
|
||||
*
|
||||
* Friction CLI's `transcript_offset` field references the byte offset INTO
|
||||
* `transcript.jsonl` (not into the captured payload). Render --transcripts
|
||||
* reads the file and finds the line that contains that offset.
|
||||
*/
|
||||
|
||||
import { createWriteStream, type WriteStream } from 'fs';
|
||||
import { spawn, type ChildProcess } from 'child_process';
|
||||
import { dirname } from 'path';
|
||||
import { mkdirSync, existsSync } from 'fs';
|
||||
import type { TranscriptEvent, TranscriptSink } from './agent-runner.ts';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Sink
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function createTranscriptSink(path: string): TranscriptSink {
|
||||
const dir = dirname(path);
|
||||
if (!existsSync(dir)) mkdirSync(dir, { recursive: true });
|
||||
const stream: WriteStream = createWriteStream(path, { flags: 'a' });
|
||||
|
||||
let bytesWritten = 0;
|
||||
let drainPromise: Promise<void> | null = null;
|
||||
|
||||
function awaitDrain(): Promise<void> {
|
||||
if (drainPromise) return drainPromise;
|
||||
drainPromise = new Promise<void>(resolve => {
|
||||
stream.once('drain', () => {
|
||||
drainPromise = null;
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
return drainPromise;
|
||||
}
|
||||
|
||||
return {
|
||||
write(event: TranscriptEvent) {
|
||||
const line = JSON.stringify({
|
||||
schema_version: '1',
|
||||
ts: event.ts,
|
||||
channel: event.channel,
|
||||
byte_offset: bytesWritten,
|
||||
bytes_b64: event.bytes.toString('base64'),
|
||||
}) + '\n';
|
||||
bytesWritten += Buffer.byteLength(line, 'utf-8');
|
||||
const ok = stream.write(line, 'utf-8');
|
||||
// If the kernel buffer is full, write() returns false. We don't await
|
||||
// here (callers don't expect that), but next callers wait on drain
|
||||
// before writing further. Bun's WritableStream is small; the drain
|
||||
// window is typically a few µs.
|
||||
if (!ok) void awaitDrain();
|
||||
},
|
||||
|
||||
nextOffset(): number {
|
||||
return bytesWritten;
|
||||
},
|
||||
|
||||
async close(): Promise<void> {
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
stream.end((err?: Error | null) => err ? reject(err) : resolve());
|
||||
});
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// spawnWithCapture
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface SpawnOpts {
|
||||
cwd: string;
|
||||
env: Record<string, string>;
|
||||
timeoutMs: number;
|
||||
transcriptSink: TranscriptSink;
|
||||
/** Optional fixed input to write on stdin then close. */
|
||||
stdinPayload?: string;
|
||||
}
|
||||
|
||||
export interface SpawnResult {
|
||||
exitCode: number;
|
||||
durationMs: number;
|
||||
/** True if SIGTERM/SIGKILL was issued due to timeout. */
|
||||
timedOut: boolean;
|
||||
}
|
||||
|
||||
const SIGTERM_GRACE_MS = 5_000;
|
||||
|
||||
export async function spawnWithCapture(bin: string, args: string[], opts: SpawnOpts): Promise<SpawnResult> {
|
||||
const start = Date.now();
|
||||
return new Promise((resolve, reject) => {
|
||||
let child: ChildProcess;
|
||||
try {
|
||||
child = spawn(bin, args, {
|
||||
cwd: opts.cwd,
|
||||
env: opts.env,
|
||||
stdio: ['pipe', 'pipe', 'pipe'],
|
||||
shell: false,
|
||||
});
|
||||
} catch (e) {
|
||||
reject(e);
|
||||
return;
|
||||
}
|
||||
|
||||
let timedOut = false;
|
||||
let killTimer: ReturnType<typeof setTimeout> | null = null;
|
||||
const wallClockTimer = setTimeout(() => {
|
||||
timedOut = true;
|
||||
try { child.kill('SIGTERM'); } catch { /* already gone */ }
|
||||
killTimer = setTimeout(() => {
|
||||
try { child.kill('SIGKILL'); } catch { /* already gone */ }
|
||||
}, SIGTERM_GRACE_MS);
|
||||
}, opts.timeoutMs);
|
||||
|
||||
child.stdout?.on('data', (chunk: Buffer) => {
|
||||
opts.transcriptSink.write({ ts: Date.now(), channel: 'stdout', bytes: chunk });
|
||||
});
|
||||
child.stderr?.on('data', (chunk: Buffer) => {
|
||||
opts.transcriptSink.write({ ts: Date.now(), channel: 'stderr', bytes: chunk });
|
||||
});
|
||||
|
||||
if (opts.stdinPayload !== undefined && child.stdin) {
|
||||
try {
|
||||
opts.transcriptSink.write({
|
||||
ts: Date.now(),
|
||||
channel: 'stdin',
|
||||
bytes: Buffer.from(opts.stdinPayload, 'utf-8'),
|
||||
});
|
||||
child.stdin.end(opts.stdinPayload, 'utf-8');
|
||||
} catch (e) {
|
||||
reject(e);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
child.on('error', (err) => {
|
||||
clearTimeout(wallClockTimer);
|
||||
if (killTimer) clearTimeout(killTimer);
|
||||
reject(err);
|
||||
});
|
||||
|
||||
child.on('close', (code) => {
|
||||
clearTimeout(wallClockTimer);
|
||||
if (killTimer) clearTimeout(killTimer);
|
||||
resolve({
|
||||
exitCode: typeof code === 'number' ? code : (timedOut ? 124 : 1),
|
||||
durationMs: Date.now() - start,
|
||||
timedOut,
|
||||
});
|
||||
});
|
||||
});
|
||||
}
|
||||
+5
-28
@@ -19,11 +19,9 @@ export type DbUrlSource =
|
||||
| 'config-file-path' // PGLite: config file present, no URL but database_path set
|
||||
| null;
|
||||
|
||||
// Internal aliases retained for backwards compatibility with the existing call
|
||||
// sites below. They forward to the exported configDir()/configPath() so
|
||||
// GBRAIN_HOME is honored uniformly. Lazy: never call homedir() at module scope.
|
||||
function getConfigDir() { return configDir(); }
|
||||
function getConfigPath() { return configPath(); }
|
||||
// Lazy-evaluated to avoid calling homedir() at module scope (breaks in serverless/bundled environments)
|
||||
function getConfigDir() { return join(homedir(), '.gbrain'); }
|
||||
function getConfigPath() { return join(getConfigDir(), 'config.json'); }
|
||||
|
||||
export interface GBrainConfig {
|
||||
engine: 'postgres' | 'pglite';
|
||||
@@ -90,20 +88,9 @@ export function toEngineConfig(config: GBrainConfig): EngineConfig {
|
||||
|
||||
export function configDir(): string {
|
||||
// Allow override for tests, Docker, and multi-tenant deployments.
|
||||
// GBRAIN_HOME is a parent dir; we always append '.gbrain' ourselves so
|
||||
// setting GBRAIN_HOME=/tmp/x yields configDir() === '/tmp/x/.gbrain'.
|
||||
// Validates the override: must be absolute, no '..' segments.
|
||||
// Matches the `GBRAIN_AUDIT_DIR` convention in src/core/minions/handlers/shell-audit.ts.
|
||||
const override = process.env.GBRAIN_HOME;
|
||||
if (override && override.trim()) {
|
||||
const trimmed = override.trim();
|
||||
if (!trimmed.startsWith('/')) {
|
||||
throw new Error(`GBRAIN_HOME must be an absolute path; got: ${trimmed}`);
|
||||
}
|
||||
if (trimmed.split('/').includes('..')) {
|
||||
throw new Error(`GBRAIN_HOME must not contain '..' segments; got: ${trimmed}`);
|
||||
}
|
||||
return join(trimmed, '.gbrain');
|
||||
}
|
||||
if (override && override.trim()) return join(override, '.gbrain');
|
||||
return join(homedir(), '.gbrain');
|
||||
}
|
||||
|
||||
@@ -111,16 +98,6 @@ export function configPath(): string {
|
||||
return join(configDir(), 'config.json');
|
||||
}
|
||||
|
||||
/**
|
||||
* Sugar for joining paths under the active gbrain home. Use this anywhere you
|
||||
* would otherwise write `join(homedir(), '.gbrain', ...rest)`. Honors
|
||||
* GBRAIN_HOME, validates input, and centralizes the convention so future
|
||||
* audits stay simple.
|
||||
*/
|
||||
export function gbrainPath(...segments: string[]): string {
|
||||
return join(configDir(), ...segments);
|
||||
}
|
||||
|
||||
/**
|
||||
* Introspect where the active DB URL would come from if we tried to connect.
|
||||
* Never throws, never connects. Env vars take precedence (matches loadConfig).
|
||||
|
||||
+16
-119
@@ -16,14 +16,9 @@
|
||||
* │ Phase 1: lint --fix (filesystem writes, no DB) │
|
||||
* │ Phase 2: backlinks --fix (filesystem writes, no DB) │
|
||||
* │ Phase 3: sync (DB picks up phases 1+2) │
|
||||
* │ Phase 4: synthesize (v0.23: transcripts → pages) │
|
||||
* │ Phase 5: extract (DB picks up links from sync │
|
||||
* │ + synthesize) │
|
||||
* │ Phase 6: patterns (v0.23: cross-session themes; │
|
||||
* │ MUST be after extract so │
|
||||
* │ graph state is fresh) │
|
||||
* │ Phase 7: embed --stale (DB writes) │
|
||||
* │ Phase 8: orphans (DB read, report only) │
|
||||
* │ Phase 4: extract (DB picks up links from sync) │
|
||||
* │ Phase 5: embed --stale (DB writes) │
|
||||
* │ Phase 6: orphans (DB read, report only) │
|
||||
* └───────────────────────────────────────────────────────────┘
|
||||
*
|
||||
* COORDINATION:
|
||||
@@ -44,23 +39,20 @@
|
||||
|
||||
import { existsSync, readFileSync, writeFileSync, unlinkSync, mkdirSync, statSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { hostname } from 'os';
|
||||
import { gbrainPath } from './config.ts';
|
||||
import { homedir, hostname } from 'os';
|
||||
import type { BrainEngine } from './engine.ts';
|
||||
import { createProgress, type ProgressReporter } from './progress.ts';
|
||||
import { getCliOptions, cliOptsToProgressOptions } from './cli-options.ts';
|
||||
|
||||
// ─── Types ─────────────────────────────────────────────────────────
|
||||
|
||||
export type CyclePhase = 'lint' | 'backlinks' | 'sync' | 'synthesize' | 'extract' | 'patterns' | 'embed' | 'orphans';
|
||||
export type CyclePhase = 'lint' | 'backlinks' | 'sync' | 'extract' | 'embed' | 'orphans';
|
||||
|
||||
export const ALL_PHASES: CyclePhase[] = [
|
||||
'lint',
|
||||
'backlinks',
|
||||
'sync',
|
||||
'synthesize',
|
||||
'extract',
|
||||
'patterns',
|
||||
'embed',
|
||||
'orphans',
|
||||
];
|
||||
@@ -68,16 +60,13 @@ export const ALL_PHASES: CyclePhase[] = [
|
||||
/**
|
||||
* Phases that mutate state (filesystem or DB) and therefore should
|
||||
* coordinate via the cycle lock. Only orphans is truly read-only
|
||||
* and skips the lock. patterns mutates DB (writes pattern pages) so
|
||||
* it acquires the lock; synthesize too.
|
||||
* and skips the lock.
|
||||
*/
|
||||
const NEEDS_LOCK_PHASES: ReadonlySet<CyclePhase> = new Set([
|
||||
'lint',
|
||||
'backlinks',
|
||||
'sync',
|
||||
'synthesize',
|
||||
'extract',
|
||||
'patterns',
|
||||
'embed',
|
||||
]);
|
||||
|
||||
@@ -132,12 +121,6 @@ export interface CycleReport {
|
||||
pages_extracted: number;
|
||||
pages_embedded: number;
|
||||
orphans_found: number;
|
||||
/** v0.23: number of transcripts the synthesize phase processed (judged + dispatched). */
|
||||
transcripts_processed: number;
|
||||
/** v0.23: number of new reflection/original/people pages written by synthesize. */
|
||||
synth_pages_written: number;
|
||||
/** v0.23: number of pattern pages written/updated by patterns phase. */
|
||||
patterns_written: number;
|
||||
};
|
||||
}
|
||||
|
||||
@@ -158,30 +141,11 @@ export interface CycleOpts {
|
||||
*/
|
||||
yieldBetweenPhases?: () => Promise<void>;
|
||||
/**
|
||||
* Generic in-phase keepalive (v0.23). Long-running phases (synthesize
|
||||
* waiting on a fan-out aggregator, patterns rolling up reflections)
|
||||
* call this periodically while idle to renew the cycle-lock TTL and
|
||||
* the Minions worker job lock. Mirrors `yieldBetweenPhases` shape;
|
||||
* passing the same function for both is the common case.
|
||||
*/
|
||||
yieldDuringPhase?: () => Promise<void>;
|
||||
/**
|
||||
* Synthesize phase scope overrides (v0.23). Forwarded to runPhaseSynthesize.
|
||||
* - `synthInputFile`: ad-hoc transcript path (`gbrain dream --input <file>`).
|
||||
* - `synthDate` / `synthFrom` / `synthTo`: date filters for corpus scan.
|
||||
* Mutually exclusive with each other in CLI parsing; runner trusts the
|
||||
* caller (CLI wrapper validates).
|
||||
*/
|
||||
synthInputFile?: string;
|
||||
synthDate?: string;
|
||||
synthFrom?: string;
|
||||
synthTo?: string;
|
||||
/**
|
||||
* AbortSignal from the Minions worker (v0.22.1, #403). When aborted
|
||||
* (timeout, cancel, lock-loss), runCycle bails between phases and
|
||||
* returns a 'failed' report instead of running the next phase. Without
|
||||
* this, a timed-out autopilot-cycle handler ignores the abort and runs
|
||||
* until the worker wedges (the 98-waiting-0-active incident on 2026-04-24).
|
||||
* AbortSignal from the Minions worker. When aborted (timeout, cancel,
|
||||
* lock-loss), runCycle bails between phases and returns a 'failed' report
|
||||
* instead of running the next phase. Without this, a timed-out
|
||||
* autopilot-cycle handler ignores the abort and runs until the worker
|
||||
* wedges (the 98-waiting-0-active incident on 2026-04-24).
|
||||
*/
|
||||
signal?: AbortSignal;
|
||||
}
|
||||
@@ -190,8 +154,7 @@ export interface CycleOpts {
|
||||
|
||||
const CYCLE_LOCK_ID = 'gbrain-cycle';
|
||||
const LOCK_TTL_MS = 30 * 60 * 1000; // 30 minutes
|
||||
// Lazy: GBRAIN_HOME may be set after module load; resolve at call time.
|
||||
const getLockFilePathDefault = () => gbrainPath('cycle.lock');
|
||||
const LOCK_FILE_PATH_DEFAULT = join(homedir(), '.gbrain', 'cycle.lock');
|
||||
|
||||
interface LockHandle {
|
||||
release: () => Promise<void>;
|
||||
@@ -293,7 +256,7 @@ async function acquirePostgresLock(engine: BrainEngine): Promise<LockHandle | nu
|
||||
* The file contains `{pid}\n{iso-timestamp}`. Staleness = mtime older
|
||||
* than LOCK_TTL_MS OR the PID is no longer alive on this host.
|
||||
*/
|
||||
function acquireFileLock(lockPath = getLockFilePathDefault()): LockHandle | null {
|
||||
function acquireFileLock(lockPath = LOCK_FILE_PATH_DEFAULT): LockHandle | null {
|
||||
mkdirSync(join(lockPath, '..'), { recursive: true });
|
||||
const pid = process.pid;
|
||||
|
||||
@@ -800,36 +763,7 @@ export async function runCycle(
|
||||
await safeYield(opts.yieldBetweenPhases);
|
||||
}
|
||||
|
||||
// ── Phase 4: synthesize (v0.23) ─────────────────────────────
|
||||
if (phases.includes('synthesize')) {
|
||||
if (!engine) {
|
||||
phaseResults.push({
|
||||
phase: 'synthesize',
|
||||
status: 'skipped',
|
||||
duration_ms: 0,
|
||||
summary: 'no database connected',
|
||||
details: { reason: 'no_database' },
|
||||
});
|
||||
} else {
|
||||
progress.start('cycle.synthesize');
|
||||
const { runPhaseSynthesize } = await import('./cycle/synthesize.ts');
|
||||
const { result, duration_ms } = await timePhase(() => runPhaseSynthesize(engine, {
|
||||
brainDir: opts.brainDir,
|
||||
dryRun,
|
||||
yieldDuringPhase: opts.yieldDuringPhase,
|
||||
inputFile: opts.synthInputFile,
|
||||
date: opts.synthDate,
|
||||
from: opts.synthFrom,
|
||||
to: opts.synthTo,
|
||||
}));
|
||||
result.duration_ms = duration_ms;
|
||||
phaseResults.push(result);
|
||||
progress.finish();
|
||||
}
|
||||
await safeYield(opts.yieldBetweenPhases);
|
||||
}
|
||||
|
||||
// ── Phase 5: extract (now picks up synthesize output) ───────
|
||||
// ── Phase 4: extract ────────────────────────────────────────
|
||||
if (phases.includes('extract')) {
|
||||
checkAborted(opts.signal);
|
||||
if (!engine) {
|
||||
@@ -853,36 +787,7 @@ export async function runCycle(
|
||||
await safeYield(opts.yieldBetweenPhases);
|
||||
}
|
||||
|
||||
// ── Phase 6: patterns (v0.23) ───────────────────────────────
|
||||
// MUST run after extract so the graph state reads fresh — subagent
|
||||
// put_page calls in synthesize set ctx.remote=true, so auto-link
|
||||
// only fires for trusted-workspace writes (allow-listed). extract
|
||||
// is the canonical materialization step.
|
||||
if (phases.includes('patterns')) {
|
||||
if (!engine) {
|
||||
phaseResults.push({
|
||||
phase: 'patterns',
|
||||
status: 'skipped',
|
||||
duration_ms: 0,
|
||||
summary: 'no database connected',
|
||||
details: { reason: 'no_database' },
|
||||
});
|
||||
} else {
|
||||
progress.start('cycle.patterns');
|
||||
const { runPhasePatterns } = await import('./cycle/patterns.ts');
|
||||
const { result, duration_ms } = await timePhase(() => runPhasePatterns(engine, {
|
||||
brainDir: opts.brainDir,
|
||||
dryRun,
|
||||
yieldDuringPhase: opts.yieldDuringPhase,
|
||||
}));
|
||||
result.duration_ms = duration_ms;
|
||||
phaseResults.push(result);
|
||||
progress.finish();
|
||||
}
|
||||
await safeYield(opts.yieldBetweenPhases);
|
||||
}
|
||||
|
||||
// ── Phase 7: embed ──────────────────────────────────────────
|
||||
// ── Phase 5: embed ──────────────────────────────────────────
|
||||
if (phases.includes('embed')) {
|
||||
checkAborted(opts.signal);
|
||||
if (!engine) {
|
||||
@@ -903,7 +808,7 @@ export async function runCycle(
|
||||
await safeYield(opts.yieldBetweenPhases);
|
||||
}
|
||||
|
||||
// ── Phase 8: orphans ────────────────────────────────────────
|
||||
// ── Phase 6: orphans ────────────────────────────────────────
|
||||
if (phases.includes('orphans')) {
|
||||
checkAborted(opts.signal);
|
||||
if (!engine) {
|
||||
@@ -954,9 +859,6 @@ function emptyTotals(): CycleReport['totals'] {
|
||||
pages_extracted: 0,
|
||||
pages_embedded: 0,
|
||||
orphans_found: 0,
|
||||
transcripts_processed: 0,
|
||||
synth_pages_written: 0,
|
||||
patterns_written: 0,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -979,11 +881,6 @@ function extractTotals(phases: PhaseResult[]): CycleReport['totals'] {
|
||||
: Number(p.details.embedded ?? 0);
|
||||
} else if (p.phase === 'orphans' && p.details) {
|
||||
t.orphans_found = Number(p.details.total_orphans ?? 0);
|
||||
} else if (p.phase === 'synthesize' && p.details) {
|
||||
t.transcripts_processed = Number(p.details.transcripts_processed ?? 0);
|
||||
t.synth_pages_written = Number(p.details.pages_written ?? 0);
|
||||
} else if (p.phase === 'patterns' && p.details) {
|
||||
t.patterns_written = Number(p.details.patterns_written ?? 0);
|
||||
}
|
||||
}
|
||||
return t;
|
||||
|
||||
@@ -1,323 +0,0 @@
|
||||
/**
|
||||
* Patterns phase (v0.23) — cross-session theme detection.
|
||||
*
|
||||
* Reads recent reflections (within `lookback_days`), runs a single Sonnet
|
||||
* subagent to surface themes that recur across ≥`min_evidence` distinct
|
||||
* reflections, and writes one pattern page per theme.
|
||||
*
|
||||
* MUST run after `extract` so the graph state (links, timeline) is fresh.
|
||||
* Subagent put_page calls have ctx.remote=true; the trusted-workspace
|
||||
* allow-list re-enables auto-link / auto-timeline for synth + pattern
|
||||
* writes (operations.ts:trustedWorkspace branch).
|
||||
*
|
||||
* v1 behavior:
|
||||
* - Single Sonnet subagent (no fan-out — one job per cycle is plenty).
|
||||
* - Idempotent: if reflection set is below `min_evidence`, phase is skipped.
|
||||
* - Pattern slug uses LLM's chosen topic-slug (subagent prompt instructs format).
|
||||
* - Existing pattern pages are updated in place via put_page (idempotent
|
||||
* ON CONFLICT semantics in importFromContent).
|
||||
*/
|
||||
|
||||
import { join, dirname } from 'node:path';
|
||||
import { mkdirSync, writeFileSync, existsSync, readFileSync } from 'node:fs';
|
||||
import type { BrainEngine } from '../engine.ts';
|
||||
import type { PhaseResult, PhaseError } from '../cycle.ts';
|
||||
import { MinionQueue } from '../minions/queue.ts';
|
||||
import { waitForCompletion, TimeoutError } from '../minions/wait-for-completion.ts';
|
||||
import type { MinionJobInput, SubagentHandlerData } from '../minions/types.ts';
|
||||
import { serializeMarkdown } from '../markdown.ts';
|
||||
import type { Page, PageType } from '../types.ts';
|
||||
|
||||
export interface PatternsPhaseOpts {
|
||||
brainDir: string;
|
||||
dryRun: boolean;
|
||||
yieldDuringPhase?: () => Promise<void>;
|
||||
}
|
||||
|
||||
export async function runPhasePatterns(
|
||||
engine: BrainEngine,
|
||||
opts: PatternsPhaseOpts,
|
||||
): Promise<PhaseResult> {
|
||||
const start = Date.now();
|
||||
try {
|
||||
const config = await loadPatternsConfig(engine);
|
||||
|
||||
if (!config.enabled) {
|
||||
return skipped('disabled', 'dream.patterns.enabled is false');
|
||||
}
|
||||
|
||||
// Gather reflections within lookback window.
|
||||
const reflections = await gatherReflections(engine, config.lookbackDays);
|
||||
if (reflections.length < config.minEvidence) {
|
||||
return skipped(
|
||||
'insufficient_evidence',
|
||||
`${reflections.length} reflections in last ${config.lookbackDays}d (need ≥${config.minEvidence})`,
|
||||
);
|
||||
}
|
||||
|
||||
if (opts.dryRun) {
|
||||
return ok(`dry-run: would detect patterns over ${reflections.length} reflections`, {
|
||||
reflections_considered: reflections.length,
|
||||
patterns_written: 0,
|
||||
dryRun: true,
|
||||
});
|
||||
}
|
||||
|
||||
// Submit one subagent for pattern detection.
|
||||
if (!process.env.ANTHROPIC_API_KEY) {
|
||||
return skipped('no_api_key', 'ANTHROPIC_API_KEY unset; pattern detection skipped');
|
||||
}
|
||||
|
||||
const allowedSlugPrefixes = await loadAllowedSlugPrefixes();
|
||||
if (allowedSlugPrefixes.length === 0) {
|
||||
return failed(makeError('InternalError', 'NO_ALLOWLIST',
|
||||
'skills/_brain-filing-rules.json missing dream_synthesize_paths.globs'));
|
||||
}
|
||||
|
||||
const queue = new MinionQueue(engine);
|
||||
const data: SubagentHandlerData = {
|
||||
prompt: buildPatternsPrompt(reflections, config.minEvidence),
|
||||
model: config.model,
|
||||
max_turns: 30,
|
||||
allowed_slug_prefixes: allowedSlugPrefixes,
|
||||
};
|
||||
const submitOpts: Partial<MinionJobInput> = {
|
||||
max_stalled: 3,
|
||||
timeout_ms: 30 * 60 * 1000,
|
||||
};
|
||||
const job = await queue.add('subagent', data as unknown as Record<string, unknown>, submitOpts, {
|
||||
allowProtectedSubmit: true,
|
||||
});
|
||||
|
||||
let outcome: string;
|
||||
try {
|
||||
const final = await waitForCompletion(queue, job.id, {
|
||||
timeoutMs: 35 * 60 * 1000,
|
||||
pollMs: 5 * 1000,
|
||||
});
|
||||
outcome = final.status;
|
||||
} catch (e) {
|
||||
if (e instanceof TimeoutError) outcome = 'timeout';
|
||||
else throw e;
|
||||
}
|
||||
|
||||
if (opts.yieldDuringPhase) {
|
||||
try { await opts.yieldDuringPhase(); } catch { /* best-effort */ }
|
||||
}
|
||||
|
||||
// Collect slugs the subagent wrote (codex finding #2 — query tool exec rows).
|
||||
const writtenSlugs = await collectChildPutPageSlugs(engine, [job.id]);
|
||||
|
||||
// Reverse-write to fs.
|
||||
const reverseWriteCount = await reverseWriteSlugs(engine, opts.brainDir, writtenSlugs);
|
||||
|
||||
return ok(`${writtenSlugs.length} pattern page(s) written/updated (${outcome})`, {
|
||||
reflections_considered: reflections.length,
|
||||
patterns_written: writtenSlugs.length,
|
||||
reverse_write_count: reverseWriteCount,
|
||||
child_outcome: outcome,
|
||||
job_id: job.id,
|
||||
});
|
||||
} catch (e) {
|
||||
return failed(makeError('InternalError', 'PATTERNS_PHASE_FAIL',
|
||||
e instanceof Error ? (e.message || 'patterns phase threw') : String(e)));
|
||||
} finally {
|
||||
void start;
|
||||
}
|
||||
}
|
||||
|
||||
// ── Config ────────────────────────────────────────────────────────────
|
||||
|
||||
interface PatternsConfig {
|
||||
enabled: boolean;
|
||||
lookbackDays: number;
|
||||
minEvidence: number;
|
||||
model: string;
|
||||
}
|
||||
|
||||
async function loadPatternsConfig(engine: BrainEngine): Promise<PatternsConfig> {
|
||||
const enabledStr = await engine.getConfig('dream.patterns.enabled');
|
||||
const enabled = enabledStr === null ? true : enabledStr === 'true';
|
||||
const lookbackStr = await engine.getConfig('dream.patterns.lookback_days');
|
||||
const minEvidenceStr = await engine.getConfig('dream.patterns.min_evidence');
|
||||
const model = (await engine.getConfig('dream.patterns.model')) || 'claude-sonnet-4-6';
|
||||
return {
|
||||
enabled,
|
||||
lookbackDays: lookbackStr ? Math.max(1, parseInt(lookbackStr, 10) || 30) : 30,
|
||||
minEvidence: minEvidenceStr ? Math.max(1, parseInt(minEvidenceStr, 10) || 3) : 3,
|
||||
model,
|
||||
};
|
||||
}
|
||||
|
||||
// ── Reflection gathering ─────────────────────────────────────────────
|
||||
|
||||
interface ReflectionRef {
|
||||
slug: string;
|
||||
title: string;
|
||||
excerpt: string;
|
||||
}
|
||||
|
||||
async function gatherReflections(
|
||||
engine: BrainEngine,
|
||||
lookbackDays: number,
|
||||
): Promise<ReflectionRef[]> {
|
||||
const since = new Date(Date.now() - lookbackDays * 24 * 60 * 60 * 1000).toISOString();
|
||||
const rows = await engine.executeRaw<{ slug: string; title: string | null; compiled_truth: string | null }>(
|
||||
`SELECT slug, title, compiled_truth
|
||||
FROM pages
|
||||
WHERE slug LIKE 'wiki/personal/reflections/%'
|
||||
AND updated_at >= $1::timestamptz
|
||||
ORDER BY updated_at DESC
|
||||
LIMIT 100`,
|
||||
[since],
|
||||
);
|
||||
return rows.map(r => ({
|
||||
slug: r.slug,
|
||||
title: r.title ?? r.slug,
|
||||
excerpt: (r.compiled_truth ?? '').slice(0, 600),
|
||||
}));
|
||||
}
|
||||
|
||||
// ── Prompt ────────────────────────────────────────────────────────────
|
||||
|
||||
function buildPatternsPrompt(reflections: ReflectionRef[], minEvidence: number): string {
|
||||
const today = new Date().toISOString().slice(0, 10);
|
||||
const corpus = reflections
|
||||
.map((r, i) => `### ${i + 1}. [[${r.slug}]] — ${r.title}\n${r.excerpt}`)
|
||||
.join('\n\n---\n\n');
|
||||
|
||||
return `You are surfacing recurring themes across the user's recent reflections.
|
||||
|
||||
OUTPUT POLICY
|
||||
- Only name a pattern if it appears in at least ${minEvidence} DISTINCT reflections.
|
||||
- Each pattern page MUST cite the reflections that constitute its evidence (use [[wiki/personal/reflections/...]] wikilinks).
|
||||
- Use \`search\` to check whether a similar pattern page already exists; if yes, update it (use the same slug). If no, create a new one.
|
||||
- Pattern slug format: \`wiki/personal/patterns/<topic-slug>\` (lowercase alphanumeric + hyphens; no underscores, no extension, no date).
|
||||
- A "pattern" is a recurring theme, anxiety, decision pattern, relationship dynamic, or self-knowledge motif. NOT a single insight. NOT a list of unrelated topics.
|
||||
|
||||
DO NOT WRITE
|
||||
- A "patterns from today" digest (that's the dream-cycle-summaries page; not your job).
|
||||
- Patterns with <${minEvidence} reflections cited.
|
||||
- Anything outside wiki/personal/patterns/.
|
||||
|
||||
CONTEXT
|
||||
- Today: ${today}
|
||||
- Reflections in scope: ${reflections.length}
|
||||
|
||||
REFLECTIONS
|
||||
${corpus}
|
||||
|
||||
When done, briefly list the pattern slugs you wrote/updated in your final message.`;
|
||||
}
|
||||
|
||||
// ── Provenance via put_page tool execution rows ─────────────────────
|
||||
|
||||
async function collectChildPutPageSlugs(
|
||||
engine: BrainEngine,
|
||||
childIds: number[],
|
||||
): Promise<string[]> {
|
||||
if (childIds.length === 0) return [];
|
||||
const rows = await engine.executeRaw<{ slug: string }>(
|
||||
`SELECT DISTINCT input->>'slug' AS slug
|
||||
FROM subagent_tool_executions
|
||||
WHERE job_id = ANY($1::int[])
|
||||
AND tool_name = 'brain_put_page'
|
||||
AND status = 'complete'
|
||||
AND input ? 'slug'
|
||||
ORDER BY 1`,
|
||||
[childIds],
|
||||
);
|
||||
return rows.map(r => r.slug).filter((s): s is string => typeof s === 'string' && s.length > 0);
|
||||
}
|
||||
|
||||
// ── Reverse-write ────────────────────────────────────────────────────
|
||||
|
||||
async function reverseWriteSlugs(
|
||||
engine: BrainEngine,
|
||||
brainDir: string,
|
||||
slugs: string[],
|
||||
): Promise<number> {
|
||||
let count = 0;
|
||||
for (const slug of slugs) {
|
||||
const page = await engine.getPage(slug);
|
||||
if (!page) continue;
|
||||
const tags = await engine.getTags(slug);
|
||||
try {
|
||||
const md = renderPageToMarkdown(page, tags);
|
||||
const filePath = join(brainDir, `${slug}.md`);
|
||||
mkdirSync(dirname(filePath), { recursive: true });
|
||||
writeFileSync(filePath, md, 'utf8');
|
||||
count++;
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
process.stderr.write(`[dream] reverse-write ${slug} failed: ${msg}\n`);
|
||||
}
|
||||
}
|
||||
return count;
|
||||
}
|
||||
|
||||
function renderPageToMarkdown(page: Page, tags: string[]): string {
|
||||
const frontmatter = (page.frontmatter ?? {}) as Record<string, unknown>;
|
||||
return serializeMarkdown(
|
||||
frontmatter,
|
||||
page.compiled_truth ?? '',
|
||||
page.timeline ?? '',
|
||||
{
|
||||
type: (page.type as PageType) ?? 'note',
|
||||
title: page.title ?? '',
|
||||
tags,
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
// ── Allow-list (shared with synthesize.ts) ───────────────────────────
|
||||
|
||||
async function loadAllowedSlugPrefixes(): Promise<string[]> {
|
||||
const candidates = [
|
||||
join(process.cwd(), 'skills', '_brain-filing-rules.json'),
|
||||
join(__dirname, '..', '..', '..', 'skills', '_brain-filing-rules.json'),
|
||||
];
|
||||
for (const path of candidates) {
|
||||
if (!existsSync(path)) continue;
|
||||
try {
|
||||
const raw = readFileSync(path, 'utf8');
|
||||
const parsed = JSON.parse(raw) as { dream_synthesize_paths?: { globs?: unknown } };
|
||||
const globs = parsed?.dream_synthesize_paths?.globs;
|
||||
if (Array.isArray(globs) && globs.every(g => typeof g === 'string')) {
|
||||
return globs as string[];
|
||||
}
|
||||
} catch { /* try next */ }
|
||||
}
|
||||
return [];
|
||||
}
|
||||
|
||||
// ── Status helpers ───────────────────────────────────────────────────
|
||||
|
||||
function ok(summary: string, details: Record<string, unknown> = {}): PhaseResult {
|
||||
return { phase: 'patterns', status: 'ok', duration_ms: 0, summary, details };
|
||||
}
|
||||
|
||||
function skipped(reason: string, summary: string): PhaseResult {
|
||||
return {
|
||||
phase: 'patterns',
|
||||
status: 'skipped',
|
||||
duration_ms: 0,
|
||||
summary,
|
||||
details: { reason },
|
||||
};
|
||||
}
|
||||
|
||||
function failed(error: PhaseError): PhaseResult {
|
||||
return {
|
||||
phase: 'patterns',
|
||||
status: 'fail',
|
||||
duration_ms: 0,
|
||||
summary: 'patterns phase failed',
|
||||
details: {},
|
||||
error,
|
||||
};
|
||||
}
|
||||
|
||||
function makeError(cls: string, code: string, message: string, hint?: string): PhaseError {
|
||||
return hint ? { class: cls, code, message, hint } : { class: cls, code, message };
|
||||
}
|
||||
@@ -1,604 +0,0 @@
|
||||
/**
|
||||
* Synthesize phase (v0.23) — conversation-to-brain pipeline.
|
||||
*
|
||||
* Reads transcripts from the configured corpus dir, runs a cheap Haiku
|
||||
* "is this worth processing?" verdict (cached in `dream_verdicts`), then
|
||||
* fans out one Sonnet subagent per worth-processing transcript with the
|
||||
* trusted-workspace `allowed_slug_prefixes` list. After children resolve,
|
||||
* the orchestrator queries `subagent_tool_executions` for the put_page
|
||||
* slugs each child wrote (codex finding #2: NOT a time-windowed pages
|
||||
* query — picks up unrelated writes), reverse-renders each new page from
|
||||
* DB to disk, and writes a deterministic summary index.
|
||||
*
|
||||
* Hard guarantees:
|
||||
* - Subagent never gets fs-write access. Orchestrator holds the dual-write.
|
||||
* - Allow-list is sourced from `skills/_brain-filing-rules.json` (single
|
||||
* source of truth) and threaded as handler data; PROTECTED_JOB_NAMES
|
||||
* prevents MCP from submitting `subagent` jobs, so the field is trusted.
|
||||
* - Cooldown via `dream.synthesize.last_completion_ts` config key —
|
||||
* written ONLY on success (codex finding #5 deferral: no auto git commit
|
||||
* in v1).
|
||||
* - Idempotency via `dream:synth:<file_path>:<content_hash>` job key.
|
||||
* - Edited transcripts produce slugs with content-hash suffix → no overwrite.
|
||||
*
|
||||
* NOT in v1:
|
||||
* - git auto-commit / push (deferred to v1.1, codex finding #5).
|
||||
* - Daily token budget cap (cooldown bounds spend at v1 scale).
|
||||
*/
|
||||
|
||||
import Anthropic from '@anthropic-ai/sdk';
|
||||
import { readFileSync, existsSync, writeFileSync, mkdirSync } from 'node:fs';
|
||||
import { join, dirname } from 'node:path';
|
||||
import type { BrainEngine } from '../engine.ts';
|
||||
import type { PhaseResult, PhaseError } from '../cycle.ts';
|
||||
import { MinionQueue } from '../minions/queue.ts';
|
||||
import { waitForCompletion, TimeoutError } from '../minions/wait-for-completion.ts';
|
||||
import type { MinionJobInput, SubagentHandlerData } from '../minions/types.ts';
|
||||
import { discoverTranscripts, type DiscoveredTranscript } from './transcript-discovery.ts';
|
||||
import { serializeMarkdown } from '../markdown.ts';
|
||||
import type { Page, PageType } from '../types.ts';
|
||||
|
||||
// Slug regex from validatePageSlug — kept in sync.
|
||||
// Used for the orchestrator-written summary index slug.
|
||||
const SUMMARY_SLUG_RE = /^[a-z0-9][a-z0-9\-]*(\/[a-z0-9][a-z0-9\-]*)*$/;
|
||||
|
||||
// ── Public entry ──────────────────────────────────────────────────────
|
||||
|
||||
export interface SynthesizePhaseOpts {
|
||||
brainDir: string;
|
||||
dryRun: boolean;
|
||||
/** Generic in-cycle keepalive for cycle-lock TTL renewal during long waits. */
|
||||
yieldDuringPhase?: () => Promise<void>;
|
||||
/**
|
||||
* Override the corpus directory and other tunables. Primarily for the
|
||||
* `gbrain dream --input <file>` ad-hoc path; bypasses config reads.
|
||||
*/
|
||||
inputFile?: string;
|
||||
date?: string;
|
||||
from?: string;
|
||||
to?: string;
|
||||
}
|
||||
|
||||
export async function runPhaseSynthesize(
|
||||
engine: BrainEngine,
|
||||
opts: SynthesizePhaseOpts,
|
||||
): Promise<PhaseResult> {
|
||||
const start = Date.now();
|
||||
try {
|
||||
const config = await loadSynthConfig(engine);
|
||||
|
||||
// Allow ad-hoc --input to run even when config is disabled.
|
||||
if (!opts.inputFile && !config.enabled) {
|
||||
return skipped('not_configured',
|
||||
'dream.synthesize.enabled is false (set dream.synthesize.session_corpus_dir to enable)');
|
||||
}
|
||||
if (!opts.inputFile && !config.corpusDir) {
|
||||
return skipped('not_configured',
|
||||
'dream.synthesize.session_corpus_dir is unset');
|
||||
}
|
||||
|
||||
// Cooldown check (skipped for explicit --input / --date / --from / --to runs).
|
||||
const explicitTarget = opts.inputFile || opts.date || opts.from || opts.to;
|
||||
if (!explicitTarget) {
|
||||
const cooldown = await checkCooldown(engine, config.cooldownHours);
|
||||
if (cooldown.active) {
|
||||
return skipped('cooldown_active',
|
||||
`synthesize cooled down until ${cooldown.expires_at} (${config.cooldownHours}h cooldown)`);
|
||||
}
|
||||
}
|
||||
|
||||
// Discover.
|
||||
const transcripts = opts.inputFile
|
||||
? loadAdHocTranscript(opts.inputFile, config.minChars, config.excludePatterns)
|
||||
: discoverTranscripts({
|
||||
corpusDir: config.corpusDir!,
|
||||
meetingTranscriptsDir: config.meetingTranscriptsDir ?? undefined,
|
||||
minChars: config.minChars,
|
||||
excludePatterns: config.excludePatterns,
|
||||
date: opts.date,
|
||||
from: opts.from,
|
||||
to: opts.to,
|
||||
});
|
||||
|
||||
if (transcripts.length === 0) {
|
||||
return ok('no transcripts to process', { transcripts_processed: 0, pages_written: 0 });
|
||||
}
|
||||
|
||||
// Significance verdicts (cached in dream_verdicts; Haiku on miss).
|
||||
const worthProcessing: DiscoveredTranscript[] = [];
|
||||
const verdicts: Array<{ filePath: string; worth: boolean; reasons: string[]; cached: boolean }> = [];
|
||||
const haiku = makeHaikuClient(); // null if no API key
|
||||
for (const t of transcripts) {
|
||||
const cached = await engine.getDreamVerdict(t.filePath, t.contentHash);
|
||||
if (cached) {
|
||||
verdicts.push({ filePath: t.filePath, worth: cached.worth_processing, reasons: cached.reasons, cached: true });
|
||||
if (cached.worth_processing) worthProcessing.push(t);
|
||||
continue;
|
||||
}
|
||||
if (!haiku) {
|
||||
// No API key — can't judge. Skip with explicit reason; don't crash phase.
|
||||
verdicts.push({ filePath: t.filePath, worth: false, reasons: ['no ANTHROPIC_API_KEY for significance judge'], cached: false });
|
||||
continue;
|
||||
}
|
||||
const verdict = await judgeSignificance(haiku, t);
|
||||
await engine.putDreamVerdict(t.filePath, t.contentHash, verdict);
|
||||
verdicts.push({ filePath: t.filePath, worth: verdict.worth_processing, reasons: verdict.reasons, cached: false });
|
||||
if (verdict.worth_processing) worthProcessing.push(t);
|
||||
}
|
||||
|
||||
// Dry-run stops here: significance filter ran (Haiku verdicts cached),
|
||||
// but no Sonnet synthesis. Codex finding #8: --dry-run does NOT mean
|
||||
// "zero LLM calls"; it means "skip Sonnet."
|
||||
if (opts.dryRun) {
|
||||
return ok(`dry-run: ${worthProcessing.length} of ${transcripts.length} transcripts would synthesize`, {
|
||||
transcripts_discovered: transcripts.length,
|
||||
transcripts_processed: 0,
|
||||
pages_written: 0,
|
||||
verdicts,
|
||||
dryRun: true,
|
||||
});
|
||||
}
|
||||
|
||||
if (worthProcessing.length === 0) {
|
||||
// Even with verdicts, the cooldown timestamp is updated only on a
|
||||
// real successful run — not on "nothing worth processing." Lets a
|
||||
// re-run pick up if a new transcript lands later.
|
||||
return ok('all transcripts skipped by significance filter', {
|
||||
transcripts_discovered: transcripts.length,
|
||||
transcripts_processed: 0,
|
||||
pages_written: 0,
|
||||
verdicts,
|
||||
});
|
||||
}
|
||||
|
||||
// Fan-out: submit one subagent per worth-processing transcript.
|
||||
const allowedSlugPrefixes = await loadAllowedSlugPrefixes();
|
||||
if (allowedSlugPrefixes.length === 0) {
|
||||
return failed(makeError('InternalError', 'NO_ALLOWLIST',
|
||||
'skills/_brain-filing-rules.json missing dream_synthesize_paths.globs'));
|
||||
}
|
||||
|
||||
const queue = new MinionQueue(engine);
|
||||
const childIds: number[] = [];
|
||||
for (const t of worthProcessing) {
|
||||
const childData: SubagentHandlerData = {
|
||||
prompt: buildSynthesisPrompt(t),
|
||||
model: config.model,
|
||||
max_turns: 30,
|
||||
allowed_slug_prefixes: allowedSlugPrefixes,
|
||||
};
|
||||
const submitOpts: Partial<MinionJobInput> = {
|
||||
max_stalled: 3,
|
||||
on_child_fail: 'continue',
|
||||
idempotency_key: `dream:synth:${t.filePath}:${t.contentHash.slice(0, 16)}`,
|
||||
timeout_ms: 30 * 60 * 1000, // 30 min per transcript
|
||||
};
|
||||
const child = await queue.add(
|
||||
'subagent',
|
||||
childData as unknown as Record<string, unknown>,
|
||||
submitOpts,
|
||||
{ allowProtectedSubmit: true },
|
||||
);
|
||||
childIds.push(child.id);
|
||||
}
|
||||
|
||||
// Wait for every child to reach a terminal state. Tick yieldDuringPhase
|
||||
// every 5 min so the cycle lock TTL refreshes.
|
||||
const childOutcomes: Array<{ jobId: number; status: string }> = [];
|
||||
for (const jobId of childIds) {
|
||||
try {
|
||||
const job = await waitForCompletion(queue, jobId, {
|
||||
timeoutMs: 35 * 60 * 1000,
|
||||
pollMs: 5 * 1000,
|
||||
});
|
||||
childOutcomes.push({ jobId, status: job.status });
|
||||
} catch (e) {
|
||||
if (e instanceof TimeoutError) {
|
||||
childOutcomes.push({ jobId, status: 'timeout' });
|
||||
} else {
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
// After each child terminal, give the cycle lock + worker job lock a chance.
|
||||
if (opts.yieldDuringPhase) {
|
||||
try { await opts.yieldDuringPhase(); } catch { /* best-effort */ }
|
||||
}
|
||||
}
|
||||
|
||||
// Collect slugs from put_page tool executions across the children
|
||||
// (codex finding #2: deterministic provenance, NOT pages.updated_at).
|
||||
const writtenSlugs = await collectChildPutPageSlugs(engine, childIds);
|
||||
|
||||
// Dual-write: reverse-render each DB row → markdown file.
|
||||
const reverseWriteCount = await reverseWriteSlugs(engine, opts.brainDir, writtenSlugs);
|
||||
|
||||
// Summary index page (deterministic; orchestrator-written via direct
|
||||
// engine.putPage so no allow-list path needed).
|
||||
const summaryDate = opts.date ?? today();
|
||||
const summarySlug = `dream-cycle-summaries/${summaryDate}`;
|
||||
if (SUMMARY_SLUG_RE.test(summarySlug)) {
|
||||
await writeSummaryPage(engine, opts.brainDir, summarySlug, summaryDate, writtenSlugs, childOutcomes);
|
||||
}
|
||||
|
||||
// Write completion timestamp ON SUCCESS only.
|
||||
await engine.setConfig('dream.synthesize.last_completion_ts', new Date().toISOString());
|
||||
|
||||
const ms = Date.now() - start;
|
||||
return ok(`${worthProcessing.length} transcript(s) synthesized in ${(ms / 1000).toFixed(1)}s`, {
|
||||
transcripts_discovered: transcripts.length,
|
||||
transcripts_processed: worthProcessing.length,
|
||||
pages_written: writtenSlugs.length,
|
||||
reverse_write_count: reverseWriteCount,
|
||||
child_outcomes: childOutcomes,
|
||||
summary_slug: summarySlug,
|
||||
verdicts,
|
||||
});
|
||||
} catch (e) {
|
||||
return failed(makeError('InternalError', 'SYNTH_PHASE_FAIL',
|
||||
e instanceof Error ? (e.message || 'synthesize phase threw') : String(e)));
|
||||
}
|
||||
}
|
||||
|
||||
// ── Config ────────────────────────────────────────────────────────────
|
||||
|
||||
interface SynthConfig {
|
||||
enabled: boolean;
|
||||
corpusDir: string | null;
|
||||
meetingTranscriptsDir: string | null;
|
||||
minChars: number;
|
||||
excludePatterns: string[];
|
||||
model: string;
|
||||
cooldownHours: number;
|
||||
}
|
||||
|
||||
async function loadSynthConfig(engine: BrainEngine): Promise<SynthConfig> {
|
||||
const enabled = (await engine.getConfig('dream.synthesize.enabled')) === 'true';
|
||||
const corpusDir = await engine.getConfig('dream.synthesize.session_corpus_dir');
|
||||
const meetingTranscriptsDir = await engine.getConfig('dream.synthesize.meeting_transcripts_dir');
|
||||
const minCharsStr = await engine.getConfig('dream.synthesize.min_chars');
|
||||
const excludeStr = await engine.getConfig('dream.synthesize.exclude_patterns');
|
||||
const model = (await engine.getConfig('dream.synthesize.model')) || 'claude-sonnet-4-6';
|
||||
const cooldownHoursStr = await engine.getConfig('dream.synthesize.cooldown_hours');
|
||||
|
||||
let excludePatterns: string[] = ['medical', 'therapy'];
|
||||
if (excludeStr) {
|
||||
try {
|
||||
const parsed = JSON.parse(excludeStr);
|
||||
if (Array.isArray(parsed)) excludePatterns = parsed.filter(p => typeof p === 'string');
|
||||
} catch { /* keep default */ }
|
||||
}
|
||||
|
||||
return {
|
||||
enabled,
|
||||
corpusDir: corpusDir ?? null,
|
||||
meetingTranscriptsDir: meetingTranscriptsDir ?? null,
|
||||
minChars: minCharsStr ? Math.max(0, parseInt(minCharsStr, 10) || 2000) : 2000,
|
||||
excludePatterns,
|
||||
model,
|
||||
cooldownHours: cooldownHoursStr ? Math.max(0, parseInt(cooldownHoursStr, 10) || 12) : 12,
|
||||
};
|
||||
}
|
||||
|
||||
async function checkCooldown(
|
||||
engine: BrainEngine,
|
||||
hours: number,
|
||||
): Promise<{ active: boolean; expires_at?: string }> {
|
||||
if (hours <= 0) return { active: false };
|
||||
const last = await engine.getConfig('dream.synthesize.last_completion_ts');
|
||||
if (!last) return { active: false };
|
||||
const lastMs = Date.parse(last);
|
||||
if (Number.isNaN(lastMs)) return { active: false };
|
||||
const expiresMs = lastMs + hours * 60 * 60 * 1000;
|
||||
if (Date.now() >= expiresMs) return { active: false };
|
||||
return { active: true, expires_at: new Date(expiresMs).toISOString() };
|
||||
}
|
||||
|
||||
// ── Allow-list source of truth ───────────────────────────────────────
|
||||
|
||||
async function loadAllowedSlugPrefixes(): Promise<string[]> {
|
||||
// Search a few known locations relative to the binary / repo. The first
|
||||
// hit wins; if none found, return [].
|
||||
const candidates = [
|
||||
join(process.cwd(), 'skills', '_brain-filing-rules.json'),
|
||||
join(__dirname, '..', '..', '..', 'skills', '_brain-filing-rules.json'),
|
||||
];
|
||||
for (const path of candidates) {
|
||||
if (!existsSync(path)) continue;
|
||||
try {
|
||||
const raw = readFileSync(path, 'utf8');
|
||||
const parsed = JSON.parse(raw) as { dream_synthesize_paths?: { globs?: unknown } };
|
||||
const globs = parsed?.dream_synthesize_paths?.globs;
|
||||
if (Array.isArray(globs) && globs.every(g => typeof g === 'string')) {
|
||||
return globs as string[];
|
||||
}
|
||||
} catch { /* try next */ }
|
||||
}
|
||||
return [];
|
||||
}
|
||||
|
||||
// ── Significance judge (Haiku) ───────────────────────────────────────
|
||||
|
||||
interface JudgeClient {
|
||||
create: (params: Anthropic.MessageCreateParamsNonStreaming) => Promise<Anthropic.Message>;
|
||||
}
|
||||
|
||||
function makeHaikuClient(): JudgeClient | null {
|
||||
if (!process.env.ANTHROPIC_API_KEY) return null;
|
||||
const client = new Anthropic();
|
||||
return { create: client.messages.create.bind(client.messages) };
|
||||
}
|
||||
|
||||
interface VerdictResult {
|
||||
worth_processing: boolean;
|
||||
reasons: string[];
|
||||
}
|
||||
|
||||
async function judgeSignificance(
|
||||
client: JudgeClient,
|
||||
t: DiscoveredTranscript,
|
||||
): Promise<VerdictResult> {
|
||||
// Truncate the transcript at 8K chars for cost control. Haiku's verdict
|
||||
// doesn't need the full body; the opening + closing sections are usually
|
||||
// representative of significance.
|
||||
const trimmed = t.content.length > 8000
|
||||
? t.content.slice(0, 4000) + '\n[...truncated...]\n' + t.content.slice(-4000)
|
||||
: t.content;
|
||||
|
||||
const sys = `You judge whether a conversation transcript is worth synthesizing into a personal knowledge brain.
|
||||
|
||||
WORTH PROCESSING (return worth_processing=true):
|
||||
- The user articulates a new idea, frame, mental model, or thesis
|
||||
- The user reflects on themselves, names patterns, processes emotion
|
||||
- The user discusses specific people, companies, or decisions in depth
|
||||
- The user makes a strategic call worth remembering
|
||||
|
||||
NOT WORTH PROCESSING (return worth_processing=false):
|
||||
- Routine ops ("check my email", "schedule X")
|
||||
- Pure code debugging without user reflection
|
||||
- Short message exchanges with no original thought
|
||||
- Repetitive content the brain already has
|
||||
|
||||
Respond as JSON: {"worth_processing": <bool>, "reasons": ["<short>", "<short>"]}.
|
||||
Two reasons max, one phrase each.`;
|
||||
|
||||
const msg = await client.create({
|
||||
model: 'claude-haiku-4-5-20251001',
|
||||
max_tokens: 200,
|
||||
system: sys,
|
||||
messages: [{ role: 'user', content: `Transcript ${t.basename}:\n\n${trimmed}` }],
|
||||
});
|
||||
|
||||
for (const block of msg.content) {
|
||||
if (block.type === 'text') {
|
||||
const text = block.text.trim();
|
||||
const m = /\{[\s\S]*\}/.exec(text);
|
||||
if (!m) continue;
|
||||
try {
|
||||
const parsed = JSON.parse(m[0]) as { worth_processing?: unknown; reasons?: unknown };
|
||||
const worth = parsed.worth_processing === true;
|
||||
const reasons = Array.isArray(parsed.reasons)
|
||||
? parsed.reasons.filter((r): r is string => typeof r === 'string').slice(0, 4)
|
||||
: [];
|
||||
return { worth_processing: worth, reasons };
|
||||
} catch { /* fall through */ }
|
||||
}
|
||||
}
|
||||
// Couldn't parse — default to NOT processing (cheap fallback).
|
||||
return { worth_processing: false, reasons: ['judge response unparseable'] };
|
||||
}
|
||||
|
||||
// ── Subagent prompt ──────────────────────────────────────────────────
|
||||
|
||||
function buildSynthesisPrompt(t: DiscoveredTranscript): string {
|
||||
const dateHint = t.inferredDate ?? today();
|
||||
const hashSuffix = t.contentHash.slice(0, 6);
|
||||
const baseSlugSegment = sanitizeForSlug(t.basename) || `session-${dateHint}`;
|
||||
return `You are synthesizing a conversation transcript into the user's personal knowledge brain.
|
||||
|
||||
CONTEXT
|
||||
- Today's date: ${dateHint}
|
||||
- Transcript hash suffix (USE THIS in slugs): ${hashSuffix}
|
||||
- Source file basename: ${baseSlugSegment}
|
||||
|
||||
OUTPUT POLICY (ALL of these are required)
|
||||
1. Quote the user verbatim. Do not paraphrase memorable phrasings.
|
||||
2. Cross-reference compulsively: every new page MUST contain at least one wikilink (e.g., \`[ref](people/jane-doe)\` or \`[[people/jane-doe]]\`) to existing brain content. Use the search tool to find existing pages first.
|
||||
3. Do NOT write to any path outside the allow-list shown in the put_page schema.
|
||||
4. Slug discipline: lowercase alphanumeric and hyphens only, slash-separated segments. NO underscores, NO file extensions.
|
||||
|
||||
TASKS
|
||||
A. Reflections (self-knowledge, pattern recognition, emotional processing):
|
||||
slug: \`wiki/personal/reflections/${dateHint}-<topic-slug>-${hashSuffix}\`
|
||||
|
||||
B. Originals (new ideas, frames, theses, mental models):
|
||||
slug: \`wiki/originals/ideas/${dateHint}-<idea-slug>-${hashSuffix}\`
|
||||
|
||||
C. People mentions: search first; if a page exists, do not put_page over it (the orchestrator handles people enrichment via timeline entries — your job is the reflection/original synthesis, NOT modifying existing person pages).
|
||||
|
||||
D. If nothing in this transcript meets the bar (significance filter already passed but the content is still routine), return without writing anything.
|
||||
|
||||
TRANSCRIPT (${t.filePath})
|
||||
---
|
||||
${t.content}
|
||||
---
|
||||
|
||||
When done, briefly list the slugs you wrote in your final message so the orchestrator can audit.`;
|
||||
}
|
||||
|
||||
function sanitizeForSlug(s: string): string {
|
||||
return s
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9]+/g, '-')
|
||||
.replace(/^-+|-+$/g, '')
|
||||
.slice(0, 60);
|
||||
}
|
||||
|
||||
// ── Slug collection from child put_page calls (codex #2) ────────────
|
||||
|
||||
async function collectChildPutPageSlugs(
|
||||
engine: BrainEngine,
|
||||
childIds: number[],
|
||||
): Promise<string[]> {
|
||||
if (childIds.length === 0) return [];
|
||||
const rows = await engine.executeRaw<{ slug: string }>(
|
||||
`SELECT DISTINCT input->>'slug' AS slug
|
||||
FROM subagent_tool_executions
|
||||
WHERE job_id = ANY($1::int[])
|
||||
AND tool_name = 'brain_put_page'
|
||||
AND status = 'complete'
|
||||
AND input ? 'slug'
|
||||
ORDER BY 1`,
|
||||
[childIds],
|
||||
);
|
||||
return rows.map(r => r.slug).filter((s): s is string => typeof s === 'string' && s.length > 0);
|
||||
}
|
||||
|
||||
// ── Reverse-write DB rows → markdown files ───────────────────────────
|
||||
|
||||
async function reverseWriteSlugs(
|
||||
engine: BrainEngine,
|
||||
brainDir: string,
|
||||
slugs: string[],
|
||||
): Promise<number> {
|
||||
let count = 0;
|
||||
for (const slug of slugs) {
|
||||
const page = await engine.getPage(slug);
|
||||
if (!page) continue;
|
||||
const tags = await engine.getTags(slug);
|
||||
try {
|
||||
const md = renderPageToMarkdown(page, tags);
|
||||
const filePath = join(brainDir, `${slug}.md`);
|
||||
mkdirSync(dirname(filePath), { recursive: true });
|
||||
writeFileSync(filePath, md, 'utf8');
|
||||
count++;
|
||||
} catch (e) {
|
||||
// Per-slug failures are non-fatal — phase continues.
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
process.stderr.write(`[dream] reverse-write ${slug} failed: ${msg}\n`);
|
||||
}
|
||||
}
|
||||
return count;
|
||||
}
|
||||
|
||||
function renderPageToMarkdown(page: Page, tags: string[]): string {
|
||||
// serializeMarkdown's contract: takes (frontmatter, compiled_truth, timeline, meta)
|
||||
// and emits frontmatter + body + (optional) timeline section.
|
||||
const frontmatter = (page.frontmatter ?? {}) as Record<string, unknown>;
|
||||
return serializeMarkdown(
|
||||
frontmatter,
|
||||
page.compiled_truth ?? '',
|
||||
page.timeline ?? '',
|
||||
{
|
||||
type: (page.type as PageType) ?? 'note',
|
||||
title: page.title ?? '',
|
||||
tags,
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
// ── Summary index page ───────────────────────────────────────────────
|
||||
|
||||
async function writeSummaryPage(
|
||||
engine: BrainEngine,
|
||||
brainDir: string,
|
||||
summarySlug: string,
|
||||
summaryDate: string,
|
||||
writtenSlugs: string[],
|
||||
childOutcomes: Array<{ jobId: number; status: string }>,
|
||||
): Promise<void> {
|
||||
const completed = childOutcomes.filter(c => c.status === 'completed').length;
|
||||
const failed = childOutcomes.length - completed;
|
||||
|
||||
const lines: string[] = [];
|
||||
lines.push(`# Dream cycle ${summaryDate}`);
|
||||
lines.push('');
|
||||
lines.push(`**Children:** ${completed} completed, ${failed} failed/timeout.`);
|
||||
lines.push(`**Pages written:** ${writtenSlugs.length}.`);
|
||||
lines.push('');
|
||||
if (writtenSlugs.length > 0) {
|
||||
lines.push('## Pages');
|
||||
lines.push('');
|
||||
for (const s of writtenSlugs) {
|
||||
lines.push(`- [[${s}]]`);
|
||||
}
|
||||
lines.push('');
|
||||
}
|
||||
|
||||
const body = lines.join('\n');
|
||||
const fullMarkdown = serializeMarkdown(
|
||||
{} as Record<string, unknown>,
|
||||
body,
|
||||
'',
|
||||
{ type: 'note' as PageType, title: `Dream cycle ${summaryDate}`, tags: ['dream-cycle'] },
|
||||
);
|
||||
|
||||
// Direct engine.putPage — orchestrator write, no subagent context, no
|
||||
// allow-list check (server-side viaSubagent=false). The summary slug is
|
||||
// pre-validated against SUMMARY_SLUG_RE in the caller.
|
||||
// Importing put_page via operations.ts would re-run namespace logic
|
||||
// unnecessarily; we go straight to the engine.
|
||||
const { parseMarkdown } = await import('../markdown.ts');
|
||||
const parsed = parseMarkdown(fullMarkdown);
|
||||
await engine.putPage(summarySlug, {
|
||||
type: parsed.type,
|
||||
title: parsed.title,
|
||||
compiled_truth: parsed.compiled_truth,
|
||||
timeline: parsed.timeline,
|
||||
frontmatter: parsed.frontmatter,
|
||||
});
|
||||
|
||||
// Also write to disk (orchestrator dual-write).
|
||||
try {
|
||||
const filePath = join(brainDir, `${summarySlug}.md`);
|
||||
mkdirSync(dirname(filePath), { recursive: true });
|
||||
writeFileSync(filePath, fullMarkdown, 'utf8');
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
process.stderr.write(`[dream] summary file-write failed: ${msg}\n`);
|
||||
}
|
||||
}
|
||||
|
||||
// ── Helpers ──────────────────────────────────────────────────────────
|
||||
|
||||
function loadAdHocTranscript(
|
||||
filePath: string,
|
||||
minChars: number,
|
||||
excludePatterns: string[],
|
||||
): DiscoveredTranscript[] {
|
||||
const { readSingleTranscript } = require('./transcript-discovery.ts') as typeof import('./transcript-discovery.ts');
|
||||
const t = readSingleTranscript(filePath, { minChars, excludePatterns });
|
||||
return t ? [t] : [];
|
||||
}
|
||||
|
||||
function today(): string {
|
||||
return new Date().toISOString().slice(0, 10);
|
||||
}
|
||||
|
||||
function ok(summary: string, details: Record<string, unknown> = {}): PhaseResult {
|
||||
return { phase: 'synthesize', status: 'ok', duration_ms: 0, summary, details };
|
||||
}
|
||||
|
||||
function skipped(reason: string, summary: string): PhaseResult {
|
||||
return {
|
||||
phase: 'synthesize',
|
||||
status: 'skipped',
|
||||
duration_ms: 0,
|
||||
summary,
|
||||
details: { reason },
|
||||
};
|
||||
}
|
||||
|
||||
function failed(error: PhaseError): PhaseResult {
|
||||
return {
|
||||
phase: 'synthesize',
|
||||
status: 'fail',
|
||||
duration_ms: 0,
|
||||
summary: 'synthesize phase failed',
|
||||
details: {},
|
||||
error,
|
||||
};
|
||||
}
|
||||
|
||||
function makeError(cls: string, code: string, message: string, hint?: string): PhaseError {
|
||||
return hint ? { class: cls, code, message, hint } : { class: cls, code, message };
|
||||
}
|
||||
@@ -1,188 +0,0 @@
|
||||
/**
|
||||
* Transcript discovery for the v0.23 dream-cycle synthesize phase.
|
||||
*
|
||||
* Walks a corpus directory for `.txt` files, applies date-range filters,
|
||||
* size filters (min_chars), and word-boundary regex exclude patterns.
|
||||
* Returns a list of file paths + content + content_hash so the caller
|
||||
* can key the verdict cache and dispatch one subagent per transcript.
|
||||
*
|
||||
* No DB; pure filesystem + crypto. Tested with hermetic temp directories.
|
||||
*/
|
||||
|
||||
import { readFileSync, readdirSync, statSync } from 'node:fs';
|
||||
import { join, basename } from 'node:path';
|
||||
import { createHash } from 'node:crypto';
|
||||
|
||||
export interface DiscoveredTranscript {
|
||||
/** Absolute path to the transcript file. */
|
||||
filePath: string;
|
||||
/** sha256(content), full hex; callers slice as needed. */
|
||||
contentHash: string;
|
||||
/** Raw transcript text. */
|
||||
content: string;
|
||||
/** Filename basename without extension; used as a topic-slug seed. */
|
||||
basename: string;
|
||||
/** Inferred date if the basename matches `YYYY-MM-DD...` (or null). */
|
||||
inferredDate: string | null;
|
||||
}
|
||||
|
||||
export interface DiscoverOpts {
|
||||
/** Source directory. Required. */
|
||||
corpusDir: string;
|
||||
/** Optional second source. */
|
||||
meetingTranscriptsDir?: string;
|
||||
/** Skip transcripts smaller than this many characters. Default 2000. */
|
||||
minChars?: number;
|
||||
/** Word-boundary regex strings. The discoverer auto-wraps bare words. */
|
||||
excludePatterns?: string[];
|
||||
/** Restrict to a single date (YYYY-MM-DD basename match). */
|
||||
date?: string;
|
||||
/** Inclusive range start (YYYY-MM-DD). */
|
||||
from?: string;
|
||||
/** Inclusive range end (YYYY-MM-DD). */
|
||||
to?: string;
|
||||
}
|
||||
|
||||
const DATE_RE = /^(\d{4}-\d{2}-\d{2})/;
|
||||
const WORD_BOUNDARY_HEURISTIC = /^[a-zA-Z][a-zA-Z0-9_-]*$/;
|
||||
|
||||
/**
|
||||
* Auto-wrap bare-word patterns in `\b<word>\b`. Power users can pass full
|
||||
* regex (e.g. `^therapy:`) which we honor verbatim. Heuristic: any input
|
||||
* that's purely alphanumeric+hyphen+underscore is treated as a bare word.
|
||||
*/
|
||||
export function compileExcludePatterns(patterns: string[] | undefined): RegExp[] {
|
||||
if (!patterns || patterns.length === 0) return [];
|
||||
const out: RegExp[] = [];
|
||||
for (const p of patterns) {
|
||||
if (!p) continue;
|
||||
try {
|
||||
const src = WORD_BOUNDARY_HEURISTIC.test(p) ? `\\b${p}\\b` : p;
|
||||
out.push(new RegExp(src, 'i'));
|
||||
} catch (e) {
|
||||
// Bad regex from user config — skip with stderr warning, don't crash.
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
process.stderr.write(`[dream] invalid exclude_pattern '${p}': ${msg}\n`);
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function hashContent(text: string): string {
|
||||
return createHash('sha256').update(text, 'utf8').digest('hex');
|
||||
}
|
||||
|
||||
function isInDateRange(date: string | null, opts: DiscoverOpts): boolean {
|
||||
if (!opts.date && !opts.from && !opts.to) return true;
|
||||
if (!date) return false; // file has no inferable date but a filter is active
|
||||
if (opts.date && date !== opts.date) return false;
|
||||
if (opts.from && date < opts.from) return false;
|
||||
if (opts.to && date > opts.to) return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
function matchesAnyExclude(text: string, patterns: RegExp[]): boolean {
|
||||
for (const re of patterns) {
|
||||
if (re.test(text)) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
function listTextFiles(dir: string): string[] {
|
||||
let entries: string[];
|
||||
try {
|
||||
entries = readdirSync(dir);
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
const out: string[] = [];
|
||||
for (const name of entries) {
|
||||
if (!name.endsWith('.txt')) continue;
|
||||
const full = join(dir, name);
|
||||
try {
|
||||
if (statSync(full).isFile()) out.push(full);
|
||||
} catch {
|
||||
// skip unreadable entries
|
||||
}
|
||||
}
|
||||
return out.sort();
|
||||
}
|
||||
|
||||
/**
|
||||
* Discover transcripts from the configured corpus dirs, applying filters.
|
||||
*
|
||||
* Skips files that:
|
||||
* - aren't `.txt`
|
||||
* - have date-prefixed basenames outside the requested window
|
||||
* - have content shorter than `minChars`
|
||||
* - match any compiled exclude pattern (case-insensitive word-boundary by default)
|
||||
*
|
||||
* Returns sorted by filePath so re-runs are deterministic.
|
||||
*/
|
||||
export function discoverTranscripts(opts: DiscoverOpts): DiscoveredTranscript[] {
|
||||
const minChars = opts.minChars ?? 2000;
|
||||
const excludeRes = compileExcludePatterns(opts.excludePatterns);
|
||||
const dirs = [opts.corpusDir, opts.meetingTranscriptsDir].filter(
|
||||
(d): d is string => typeof d === 'string' && d.length > 0,
|
||||
);
|
||||
|
||||
const results: DiscoveredTranscript[] = [];
|
||||
for (const dir of dirs) {
|
||||
for (const filePath of listTextFiles(dir)) {
|
||||
const baseName = basename(filePath, '.txt');
|
||||
const dateMatch = DATE_RE.exec(baseName);
|
||||
const inferredDate = dateMatch ? dateMatch[1] : null;
|
||||
if (!isInDateRange(inferredDate, opts)) continue;
|
||||
|
||||
let content: string;
|
||||
try {
|
||||
content = readFileSync(filePath, 'utf8');
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
if (content.length < minChars) continue;
|
||||
if (matchesAnyExclude(content, excludeRes)) continue;
|
||||
|
||||
results.push({
|
||||
filePath,
|
||||
contentHash: hashContent(content),
|
||||
content,
|
||||
basename: baseName,
|
||||
inferredDate,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return results.sort((a, b) => a.filePath.localeCompare(b.filePath));
|
||||
}
|
||||
|
||||
/**
|
||||
* Read a single ad-hoc transcript file (`gbrain dream --input <file>`).
|
||||
* Bypasses the corpus-dir scan and date filters but still applies
|
||||
* minChars + exclude_patterns when provided.
|
||||
*/
|
||||
export function readSingleTranscript(
|
||||
filePath: string,
|
||||
opts: { minChars?: number; excludePatterns?: string[] } = {},
|
||||
): DiscoveredTranscript | null {
|
||||
const minChars = opts.minChars ?? 2000;
|
||||
const excludeRes = compileExcludePatterns(opts.excludePatterns);
|
||||
let content: string;
|
||||
try {
|
||||
content = readFileSync(filePath, 'utf8');
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
throw new Error(`could not read transcript at ${filePath}: ${msg}`);
|
||||
}
|
||||
if (content.length < minChars) return null;
|
||||
if (matchesAnyExclude(content, excludeRes)) return null;
|
||||
const baseName = basename(filePath, '.txt');
|
||||
const dateMatch = DATE_RE.exec(baseName);
|
||||
return {
|
||||
filePath,
|
||||
contentHash: hashContent(content),
|
||||
content,
|
||||
basename: baseName,
|
||||
inferredDate: dateMatch ? dateMatch[1] : null,
|
||||
};
|
||||
}
|
||||
@@ -86,19 +86,6 @@ export interface ReservedConnection {
|
||||
executeRaw<T = Record<string, unknown>>(sql: string, params?: unknown[]): Promise<T[]>;
|
||||
}
|
||||
|
||||
/** Dream-cycle Haiku verdict on whether a transcript is worth processing. */
|
||||
export interface DreamVerdict {
|
||||
worth_processing: boolean;
|
||||
reasons: string[];
|
||||
judged_at: string;
|
||||
}
|
||||
|
||||
/** Input shape for putDreamVerdict — judged_at defaults to now() server-side. */
|
||||
export interface DreamVerdictInput {
|
||||
worth_processing: boolean;
|
||||
reasons: string[];
|
||||
}
|
||||
|
||||
/** Maximum results returned by search operations. Internal bulk operations (listPages) are not clamped. */
|
||||
export const MAX_SEARCH_LIMIT = 100;
|
||||
|
||||
@@ -271,12 +258,6 @@ export interface BrainEngine {
|
||||
putRawData(slug: string, source: string, data: object): Promise<void>;
|
||||
getRawData(slug: string, source?: string): Promise<RawData[]>;
|
||||
|
||||
// Dream-cycle significance verdict cache (v0.23).
|
||||
// Keyed by (file_path, content_hash). Distinct from raw_data, which is
|
||||
// page-scoped — transcripts being judged aren't pages yet.
|
||||
getDreamVerdict(filePath: string, contentHash: string): Promise<DreamVerdict | null>;
|
||||
putDreamVerdict(filePath: string, contentHash: string, verdict: DreamVerdictInput): Promise<void>;
|
||||
|
||||
// Versions
|
||||
createVersion(slug: string): Promise<PageVersion>;
|
||||
getVersions(slug: string): Promise<PageVersion[]>;
|
||||
|
||||
@@ -12,7 +12,7 @@
|
||||
|
||||
import { appendFileSync, readFileSync, existsSync, mkdirSync, writeFileSync, renameSync } from 'fs';
|
||||
import { join, dirname } from 'path';
|
||||
import { gbrainPath } from './config.ts';
|
||||
import { homedir } from 'os';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Types
|
||||
@@ -45,8 +45,7 @@ export interface TestCase {
|
||||
source: 'fail-improve-loop';
|
||||
}
|
||||
|
||||
// Lazy: GBRAIN_HOME may be set after module load, so resolve at call time.
|
||||
const getLogDir = () => gbrainPath('fail-improve');
|
||||
const LOG_DIR = join(homedir(), '.gbrain', 'fail-improve');
|
||||
const MAX_ENTRIES = 1000;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -77,7 +76,7 @@ export class FailImproveLoop {
|
||||
private logDir: string;
|
||||
|
||||
constructor(logDir?: string) {
|
||||
this.logDir = logDir || getLogDir();
|
||||
this.logDir = logDir || LOG_DIR;
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -1,374 +0,0 @@
|
||||
/**
|
||||
* Friction reporter — JSONL-backed signal capture for the claw-test feedback loop.
|
||||
*
|
||||
* The friction CLI (`gbrain friction log/render/list/summary`) writes here.
|
||||
* The claw-test harness reads here. The agent calls `gbrain friction log`
|
||||
* directly when it hits something confusing, missing, or wrong.
|
||||
*
|
||||
* Storage shape: append-only JSONL files under `$GBRAIN_HOME/friction/`.
|
||||
* - `<run-id>.jsonl` for each harness run (run-id from $GBRAIN_FRICTION_RUN_ID)
|
||||
* - `standalone.jsonl` for entries logged outside a harness run
|
||||
*
|
||||
* Schema is a flat extension of StructuredAgentError fields (per D20). Render
|
||||
* reads one level. Readers tolerate malformed lines (skip + warn) so partial
|
||||
* runs don't break later analysis.
|
||||
*
|
||||
* ┌──────────┐ appendFileSync ┌─────────────────────────┐
|
||||
* │ writer() │ ──────────────────▶ │ <runId>.jsonl (one │
|
||||
* │ │ (atomic if line │ entry per line) │
|
||||
* └──────────┘ ≤ PIPE_BUF/4KB) └─────────────────────────┘
|
||||
* │
|
||||
* ▼
|
||||
* reader() / render()
|
||||
* skip malformed + warn
|
||||
*/
|
||||
|
||||
import { appendFileSync, existsSync, readdirSync, readFileSync, mkdirSync, statSync } from 'fs';
|
||||
import { dirname, join } from 'path';
|
||||
import { homedir } from 'os';
|
||||
import { gbrainPath } from './config.ts';
|
||||
import { VERSION } from '../version.ts';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Schema
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export type FrictionKind = 'friction' | 'delight' | 'phase-marker' | 'interrupted';
|
||||
export type FrictionSeverity = 'confused' | 'error' | 'blocker' | 'nit';
|
||||
export type FrictionSource = 'claw' | 'harness';
|
||||
export type PhaseMarker = 'start' | 'end';
|
||||
|
||||
/** One JSONL entry. Flat extension of StructuredAgentError per D20. */
|
||||
export interface FrictionEntry {
|
||||
schema_version: '1';
|
||||
ts: string; // ISO 8601
|
||||
run_id: string;
|
||||
phase: string;
|
||||
kind: FrictionKind;
|
||||
/** Required for kind=friction|delight. Optional for phase-marker (purely informational). */
|
||||
severity?: FrictionSeverity;
|
||||
message: string;
|
||||
hint?: string;
|
||||
/** StructuredAgentError envelope fields, flattened. */
|
||||
class?: string;
|
||||
code?: string;
|
||||
docs_url?: string;
|
||||
source: FrictionSource;
|
||||
cwd: string;
|
||||
gbrain_version: string;
|
||||
agent?: string;
|
||||
/** Byte offset into the run's transcript.jsonl (live mode). */
|
||||
transcript_offset?: number;
|
||||
/** For phase-marker entries only. */
|
||||
marker?: PhaseMarker;
|
||||
}
|
||||
|
||||
export interface FrictionLogInput {
|
||||
severity?: FrictionSeverity;
|
||||
phase: string;
|
||||
message: string;
|
||||
hint?: string;
|
||||
runId?: string;
|
||||
kind?: FrictionKind;
|
||||
source?: FrictionSource;
|
||||
agent?: string;
|
||||
transcriptOffset?: number;
|
||||
marker?: PhaseMarker;
|
||||
/** When the writer is called from the harness wrapping a child error. */
|
||||
errorClass?: string;
|
||||
errorCode?: string;
|
||||
docsUrl?: string;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Path resolution
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** Resolve the directory all friction JSONL files live under. */
|
||||
export function frictionDir(): string {
|
||||
return gbrainPath('friction');
|
||||
}
|
||||
|
||||
/** Resolve the JSONL file path for a given run-id. */
|
||||
export function frictionFile(runId: string): string {
|
||||
return join(frictionDir(), `${sanitizeRunId(runId)}.jsonl`);
|
||||
}
|
||||
|
||||
/** Resolve the active run-id, falling back to 'standalone' (D19). */
|
||||
export function activeRunId(): string {
|
||||
const env = process.env.GBRAIN_FRICTION_RUN_ID?.trim();
|
||||
return env && env.length > 0 ? env : 'standalone';
|
||||
}
|
||||
|
||||
/** Sanitize: only [a-zA-Z0-9._-]; reject anything else to keep filenames sane. */
|
||||
function sanitizeRunId(runId: string): string {
|
||||
if (!/^[a-zA-Z0-9._-]+$/.test(runId)) {
|
||||
throw new Error(`invalid run-id ${JSON.stringify(runId)} (allowed: [a-zA-Z0-9._-])`);
|
||||
}
|
||||
return runId;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Writer
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** Maximum message length; truncated to keep each line under PIPE_BUF for atomic appends. */
|
||||
const MAX_MESSAGE_CHARS = 3500;
|
||||
|
||||
/** Append one friction entry to the run's JSONL. */
|
||||
export function logFriction(input: FrictionLogInput): void {
|
||||
const runId = input.runId ?? activeRunId();
|
||||
const dir = frictionDir();
|
||||
if (!existsSync(dir)) mkdirSync(dir, { recursive: true });
|
||||
|
||||
const message = truncate(input.message, MAX_MESSAGE_CHARS);
|
||||
const entry: FrictionEntry = {
|
||||
schema_version: '1',
|
||||
ts: new Date().toISOString(),
|
||||
run_id: runId,
|
||||
phase: input.phase,
|
||||
kind: input.kind ?? 'friction',
|
||||
message,
|
||||
source: input.source ?? 'claw',
|
||||
cwd: process.cwd(),
|
||||
gbrain_version: VERSION,
|
||||
};
|
||||
if (input.severity) entry.severity = input.severity;
|
||||
if (input.hint) entry.hint = input.hint;
|
||||
if (input.errorClass) entry.class = input.errorClass;
|
||||
if (input.errorCode) entry.code = input.errorCode;
|
||||
if (input.docsUrl) entry.docs_url = input.docsUrl;
|
||||
if (input.agent) entry.agent = input.agent;
|
||||
if (input.transcriptOffset !== undefined) entry.transcript_offset = input.transcriptOffset;
|
||||
if (input.marker) entry.marker = input.marker;
|
||||
|
||||
const line = JSON.stringify(entry) + '\n';
|
||||
appendFileSync(frictionFile(runId), line, 'utf-8');
|
||||
}
|
||||
|
||||
function truncate(s: string, max: number): string {
|
||||
if (s.length <= max) return s;
|
||||
return s.slice(0, max - 14) + '…[truncated]';
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Reader
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface ReadResult {
|
||||
entries: FrictionEntry[];
|
||||
/** Count of malformed JSONL lines that were skipped. */
|
||||
malformed: number;
|
||||
}
|
||||
|
||||
/** Read all entries from a run's JSONL, skipping malformed lines. */
|
||||
export function readFriction(runId: string): ReadResult {
|
||||
const path = frictionFile(runId);
|
||||
if (!existsSync(path)) {
|
||||
throw new Error(`run-id "${runId}" not found at ${path}`);
|
||||
}
|
||||
const raw = readFileSync(path, 'utf-8');
|
||||
const entries: FrictionEntry[] = [];
|
||||
let malformed = 0;
|
||||
for (const line of raw.split('\n')) {
|
||||
if (!line.trim()) continue;
|
||||
try {
|
||||
const parsed = JSON.parse(line);
|
||||
// Light shape check: must have ts + kind + phase + message
|
||||
if (typeof parsed.ts === 'string' && typeof parsed.kind === 'string' && typeof parsed.phase === 'string' && typeof parsed.message === 'string') {
|
||||
entries.push(parsed as FrictionEntry);
|
||||
} else {
|
||||
malformed++;
|
||||
}
|
||||
} catch {
|
||||
malformed++;
|
||||
}
|
||||
}
|
||||
return { entries, malformed };
|
||||
}
|
||||
|
||||
/** List run-ids with summary counts. Returns most-recent-first. */
|
||||
export interface RunSummary {
|
||||
runId: string;
|
||||
path: string;
|
||||
mtime: Date;
|
||||
counts: { friction: number; delight: number; interrupted: boolean; bySeverity: Record<string, number> };
|
||||
}
|
||||
|
||||
export function listRuns(): RunSummary[] {
|
||||
const dir = frictionDir();
|
||||
if (!existsSync(dir)) return [];
|
||||
const out: RunSummary[] = [];
|
||||
for (const file of readdirSync(dir)) {
|
||||
if (!file.endsWith('.jsonl')) continue;
|
||||
const runId = file.slice(0, -'.jsonl'.length);
|
||||
const path = join(dir, file);
|
||||
const stat = statSync(path);
|
||||
let read: ReadResult;
|
||||
try {
|
||||
read = readFriction(runId);
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
const counts = { friction: 0, delight: 0, interrupted: false, bySeverity: {} as Record<string, number> };
|
||||
for (const e of read.entries) {
|
||||
if (e.kind === 'friction') counts.friction++;
|
||||
if (e.kind === 'delight') counts.delight++;
|
||||
if (e.kind === 'interrupted') counts.interrupted = true;
|
||||
if (e.severity) counts.bySeverity[e.severity] = (counts.bySeverity[e.severity] ?? 0) + 1;
|
||||
}
|
||||
out.push({ runId, path, mtime: stat.mtime, counts });
|
||||
}
|
||||
out.sort((a, b) => b.mtime.getTime() - a.mtime.getTime());
|
||||
return out;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Renderer
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface RenderOpts {
|
||||
format?: 'md' | 'json';
|
||||
redact?: boolean;
|
||||
/** When true, transcript_offset values are resolved against this transcript file. */
|
||||
transcriptPath?: string;
|
||||
}
|
||||
|
||||
/** Render entries grouped by severity then phase. Returns the rendered string. */
|
||||
export function renderReport(runId: string, opts: RenderOpts = {}): string {
|
||||
const { entries, malformed } = readFriction(runId);
|
||||
const format = opts.format ?? 'md';
|
||||
const redact = opts.redact ?? (format === 'md');
|
||||
|
||||
const transformed = entries.map(e => redact ? redactEntry(e) : e);
|
||||
|
||||
if (format === 'json') {
|
||||
return JSON.stringify({ run_id: runId, malformed, entries: transformed }, null, 2);
|
||||
}
|
||||
|
||||
// Markdown grouping: severity (blocker > error > confused > nit > none) → phase
|
||||
const sevOrder: (FrictionSeverity | 'none')[] = ['blocker', 'error', 'confused', 'nit', 'none'];
|
||||
const bySev = new Map<string, FrictionEntry[]>();
|
||||
for (const e of transformed) {
|
||||
if (e.kind !== 'friction' && e.kind !== 'delight') continue;
|
||||
const k = e.severity ?? 'none';
|
||||
if (!bySev.has(k)) bySev.set(k, []);
|
||||
bySev.get(k)!.push(e);
|
||||
}
|
||||
|
||||
const lines: string[] = [];
|
||||
lines.push(`# Friction report — \`${runId}\``);
|
||||
lines.push('');
|
||||
const totalFriction = entries.filter(e => e.kind === 'friction').length;
|
||||
const totalDelight = entries.filter(e => e.kind === 'delight').length;
|
||||
lines.push(`**${totalFriction} friction · ${totalDelight} delight**${malformed > 0 ? ` · ${malformed} malformed line(s) skipped` : ''}`);
|
||||
lines.push('');
|
||||
|
||||
if (entries.some(e => e.kind === 'interrupted')) {
|
||||
lines.push('> ⚠ **Run was interrupted.** Some phases may not have completed.');
|
||||
lines.push('');
|
||||
}
|
||||
|
||||
for (const sev of sevOrder) {
|
||||
const bucket = bySev.get(sev);
|
||||
if (!bucket || bucket.length === 0) continue;
|
||||
lines.push(`## ${sev === 'none' ? '(no severity)' : sev}`);
|
||||
lines.push('');
|
||||
// Group by phase within severity
|
||||
const byPhase = new Map<string, FrictionEntry[]>();
|
||||
for (const e of bucket) {
|
||||
if (!byPhase.has(e.phase)) byPhase.set(e.phase, []);
|
||||
byPhase.get(e.phase)!.push(e);
|
||||
}
|
||||
for (const [phase, phaseEntries] of byPhase) {
|
||||
lines.push(`### \`${phase}\``);
|
||||
lines.push('');
|
||||
for (const e of phaseEntries) {
|
||||
lines.push(`- ${e.kind === 'delight' ? '✨' : '·'} ${e.message}`);
|
||||
if (e.hint) lines.push(` - hint: ${e.hint}`);
|
||||
if (e.code) lines.push(` - code: \`${e.code}\``);
|
||||
if (e.docs_url) lines.push(` - docs: ${e.docs_url}`);
|
||||
if (opts.transcriptPath && e.transcript_offset !== undefined) {
|
||||
const snippet = readTranscriptAt(opts.transcriptPath, e.transcript_offset);
|
||||
if (snippet) lines.push(` - transcript: \`${snippet}\``);
|
||||
}
|
||||
}
|
||||
lines.push('');
|
||||
}
|
||||
}
|
||||
return lines.join('\n');
|
||||
}
|
||||
|
||||
/** Render a friction + delight summary as two columns. */
|
||||
export function renderSummary(runId: string, opts: { format?: 'md' | 'json' } = {}): string {
|
||||
const { entries } = readFriction(runId);
|
||||
const friction = entries.filter(e => e.kind === 'friction');
|
||||
const delight = entries.filter(e => e.kind === 'delight');
|
||||
|
||||
if (opts.format === 'json') {
|
||||
return JSON.stringify({ run_id: runId, friction, delight }, null, 2);
|
||||
}
|
||||
|
||||
const lines: string[] = [];
|
||||
lines.push(`# ${runId}`);
|
||||
lines.push('');
|
||||
const max = Math.max(friction.length, delight.length);
|
||||
lines.push(`| friction (${friction.length}) | delight (${delight.length}) |`);
|
||||
lines.push('|---|---|');
|
||||
for (let i = 0; i < max; i++) {
|
||||
const l = friction[i] ? friction[i].message.replace(/\|/g, '\\|') : '';
|
||||
const r = delight[i] ? delight[i].message.replace(/\|/g, '\\|') : '';
|
||||
lines.push(`| ${l} | ${r} |`);
|
||||
}
|
||||
return lines.join('\n');
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Redaction
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** Replace homedir/cwd segments in user-visible string fields with placeholders. */
|
||||
export function redactEntry(entry: FrictionEntry): FrictionEntry {
|
||||
const home = homedir();
|
||||
const cwd = entry.cwd;
|
||||
const transform = (s: string | undefined): string | undefined => {
|
||||
if (!s) return s;
|
||||
let out = s;
|
||||
if (cwd && cwd.length > 1) out = out.split(cwd).join('<CWD>');
|
||||
if (home && home.length > 1) out = out.split(home).join('<HOME>');
|
||||
return out;
|
||||
};
|
||||
return {
|
||||
...entry,
|
||||
message: transform(entry.message) ?? entry.message,
|
||||
hint: transform(entry.hint),
|
||||
cwd: '<CWD>',
|
||||
};
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Transcript snippet resolution (for --transcripts)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function readTranscriptAt(path: string, offset: number): string | null {
|
||||
try {
|
||||
if (!existsSync(path)) return null;
|
||||
const raw = readFileSync(path, 'utf-8');
|
||||
if (offset < 0 || offset >= raw.length) return null;
|
||||
// Find the line that contains this offset. Transcript is JSONL.
|
||||
const lineStart = raw.lastIndexOf('\n', offset) + 1;
|
||||
const lineEnd = raw.indexOf('\n', offset);
|
||||
const line = raw.slice(lineStart, lineEnd === -1 ? undefined : lineEnd);
|
||||
try {
|
||||
const parsed = JSON.parse(line);
|
||||
if (parsed && typeof parsed.bytes_b64 === 'string') {
|
||||
const text = Buffer.from(parsed.bytes_b64, 'base64').toString('utf-8');
|
||||
// Truncate snippet for readability
|
||||
return text.replace(/\n/g, '\\n').slice(0, 200);
|
||||
}
|
||||
} catch { /* fall through */ }
|
||||
return line.slice(0, 200);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
@@ -1,410 +0,0 @@
|
||||
/**
|
||||
* Frontmatter inference — synthesize YAML frontmatter from filesystem metadata.
|
||||
*
|
||||
* ## Why this exists
|
||||
*
|
||||
* GBrain's sync and import pipelines work fine without frontmatter — gray-matter
|
||||
* returns the full content as body, and `inferType`/`inferTitle` in markdown.ts
|
||||
* provide fallbacks. But the inferred metadata is minimal:
|
||||
*
|
||||
* - `type` defaults to 'concept' for most paths
|
||||
* - `title` is the slugified filename ("2010 04 13 Apr 13 Founders Mtg")
|
||||
* - No `date` field, no `source` metadata, no folder-aware tagging
|
||||
*
|
||||
* This module provides **rich inference** — directory-aware type mapping, date
|
||||
* extraction from filenames, title cleanup (strip date prefixes, HTML entities),
|
||||
* heading extraction from content, and source/folder tagging. It produces a
|
||||
* complete frontmatter block that can be:
|
||||
*
|
||||
* 1. Written back to the file on disk (via `gbrain frontmatter generate --fix`)
|
||||
* 2. Used at import time without modifying the file (DB-only inference)
|
||||
* 3. Shown as a dry-run preview (via `gbrain frontmatter generate --dry-run`)
|
||||
*
|
||||
* ## Design principles
|
||||
*
|
||||
* - **Never overwrite existing frontmatter.** If a file already has `---`, skip it.
|
||||
* - **Infer from filesystem first, content second.** Directory path → type, filename → date + title,
|
||||
* first `#` heading → title fallback, content → entity hints.
|
||||
* - **Deterministic.** Same file always produces the same frontmatter. No LLM calls, no network.
|
||||
* - **Extensible via rules.** The `DIRECTORY_RULES` table maps path patterns to type + source + tags.
|
||||
* Adding a new directory convention = adding one rule.
|
||||
* - **Safe.** `.bak` files on write, `--dry-run` by default in CLI, idempotent.
|
||||
*
|
||||
* ## How it fits in the pipeline
|
||||
*
|
||||
* ```
|
||||
* Sync/Import
|
||||
* → file has frontmatter? → normal import (existing path)
|
||||
* → file has NO frontmatter?
|
||||
* → inferFrontmatter(filePath, content) → synthesize frontmatter
|
||||
* → prepend to content → import as usual
|
||||
* → optionally write back to disk (--write-back flag)
|
||||
* ```
|
||||
*
|
||||
* The inference runs BEFORE `parseMarkdown`, so the downstream pipeline sees
|
||||
* well-formed frontmatter and all the existing validation/chunking/embedding
|
||||
* logic works unchanged.
|
||||
*
|
||||
* ## Directory rules table
|
||||
*
|
||||
* Each rule matches a path pattern (case-insensitive prefix) and provides:
|
||||
* - `type`: page type for the brain schema
|
||||
* - `source`: optional source tag (e.g., "apple-notes", "therapy")
|
||||
* - `tags`: optional additional tags
|
||||
* - `datePattern`: where to look for dates — 'filename' (YYYY-MM-DD prefix),
|
||||
* 'dirname' (parent dir name), or 'none'
|
||||
* - `titleStrategy`: how to extract title — 'filename' (strip date prefix),
|
||||
* 'heading' (first # in content), 'filename-full' (no date strip)
|
||||
*/
|
||||
|
||||
import { basename, dirname, relative } from 'path';
|
||||
|
||||
// ─── Types ───────────────────────────────────────────────────────────
|
||||
|
||||
export interface InferredFrontmatter {
|
||||
title: string;
|
||||
type: string;
|
||||
date?: string;
|
||||
source?: string;
|
||||
tags?: string[];
|
||||
/** True if the file already has frontmatter (inference skipped). */
|
||||
skipped?: boolean;
|
||||
/** The rule that matched, for debugging. */
|
||||
matchedRule?: string;
|
||||
}
|
||||
|
||||
export interface DirectoryRule {
|
||||
/** Case-insensitive path prefix to match (e.g., 'apple notes/'). */
|
||||
pathPrefix: string;
|
||||
/** Page type to assign. */
|
||||
type: string;
|
||||
/** Optional source tag. */
|
||||
source?: string;
|
||||
/** Optional tags to add. */
|
||||
tags?: string[];
|
||||
/** Where to look for dates. Default: 'filename'. */
|
||||
datePattern?: 'filename' | 'dirname' | 'none';
|
||||
/** How to extract title. Default: 'filename'. */
|
||||
titleStrategy?: 'filename' | 'heading' | 'filename-full';
|
||||
}
|
||||
|
||||
// ─── Directory Rules ─────────────────────────────────────────────────
|
||||
// Ordered from most specific to least specific. First match wins.
|
||||
// Add new directory conventions here.
|
||||
|
||||
export const DIRECTORY_RULES: DirectoryRule[] = [
|
||||
// Apple Notes — bulk import from Apple Notes app. Filenames are
|
||||
// "YYYY-MM-DD Title.md" with HTML-styled content.
|
||||
{
|
||||
pathPrefix: 'apple notes/youtube shows/',
|
||||
type: 'apple-note',
|
||||
source: 'apple-notes',
|
||||
tags: ['youtube', 'shows'],
|
||||
datePattern: 'filename',
|
||||
titleStrategy: 'filename',
|
||||
},
|
||||
{
|
||||
pathPrefix: 'apple notes/yc/',
|
||||
type: 'apple-note',
|
||||
source: 'apple-notes',
|
||||
tags: ['yc'],
|
||||
datePattern: 'filename',
|
||||
titleStrategy: 'filename',
|
||||
},
|
||||
{
|
||||
pathPrefix: 'apple notes/archived/',
|
||||
type: 'apple-note',
|
||||
source: 'apple-notes',
|
||||
tags: ['archived'],
|
||||
datePattern: 'filename',
|
||||
titleStrategy: 'filename',
|
||||
},
|
||||
{
|
||||
pathPrefix: 'apple notes/politics/',
|
||||
type: 'apple-note',
|
||||
source: 'apple-notes',
|
||||
tags: ['politics'],
|
||||
datePattern: 'filename',
|
||||
titleStrategy: 'filename',
|
||||
},
|
||||
{
|
||||
pathPrefix: 'apple notes/pitch notes/',
|
||||
type: 'apple-note',
|
||||
source: 'apple-notes',
|
||||
tags: ['pitch-notes'],
|
||||
datePattern: 'filename',
|
||||
titleStrategy: 'filename',
|
||||
},
|
||||
{
|
||||
pathPrefix: 'apple notes/gstack/',
|
||||
type: 'apple-note',
|
||||
source: 'apple-notes',
|
||||
tags: ['gstack'],
|
||||
datePattern: 'filename',
|
||||
titleStrategy: 'filename',
|
||||
},
|
||||
{
|
||||
pathPrefix: 'apple notes/photo-cameras/',
|
||||
type: 'apple-note',
|
||||
source: 'apple-notes',
|
||||
tags: ['photography'],
|
||||
datePattern: 'filename',
|
||||
titleStrategy: 'filename',
|
||||
},
|
||||
{
|
||||
pathPrefix: 'apple notes/jan bowman notes/',
|
||||
type: 'apple-note',
|
||||
source: 'apple-notes',
|
||||
tags: ['therapy', 'jan-bowman'],
|
||||
datePattern: 'filename',
|
||||
titleStrategy: 'filename',
|
||||
},
|
||||
// Catch-all for Apple Notes not in a subfolder
|
||||
{
|
||||
pathPrefix: 'apple notes/',
|
||||
type: 'apple-note',
|
||||
source: 'apple-notes',
|
||||
datePattern: 'filename',
|
||||
titleStrategy: 'filename',
|
||||
},
|
||||
|
||||
// Calendar diarization files
|
||||
{
|
||||
pathPrefix: 'daily/calendar/',
|
||||
type: 'calendar-index',
|
||||
source: 'calendar',
|
||||
datePattern: 'filename',
|
||||
titleStrategy: 'filename',
|
||||
},
|
||||
|
||||
// Personal sections
|
||||
{
|
||||
pathPrefix: 'personal/therapy/',
|
||||
type: 'therapy-session',
|
||||
source: 'therapy',
|
||||
datePattern: 'filename',
|
||||
titleStrategy: 'filename',
|
||||
},
|
||||
{
|
||||
pathPrefix: 'personal/reflections/',
|
||||
type: 'reflection',
|
||||
source: 'personal',
|
||||
datePattern: 'filename',
|
||||
titleStrategy: 'heading',
|
||||
},
|
||||
{
|
||||
pathPrefix: 'personal/',
|
||||
type: 'personal',
|
||||
source: 'personal',
|
||||
datePattern: 'none',
|
||||
titleStrategy: 'heading',
|
||||
},
|
||||
|
||||
// Writing
|
||||
{
|
||||
pathPrefix: 'writing/essays/',
|
||||
type: 'essay',
|
||||
source: 'writing',
|
||||
datePattern: 'filename',
|
||||
titleStrategy: 'heading',
|
||||
},
|
||||
{
|
||||
pathPrefix: 'writing/ideas/',
|
||||
type: 'idea',
|
||||
source: 'writing',
|
||||
datePattern: 'filename',
|
||||
titleStrategy: 'heading',
|
||||
},
|
||||
{
|
||||
pathPrefix: 'writing/',
|
||||
type: 'writing',
|
||||
source: 'writing',
|
||||
datePattern: 'filename',
|
||||
titleStrategy: 'heading',
|
||||
},
|
||||
|
||||
// Entity directories — these should already have frontmatter in most cases,
|
||||
// but the 55 people pages etc. that don't get handled here.
|
||||
{ pathPrefix: 'people/', type: 'person', titleStrategy: 'heading' },
|
||||
{ pathPrefix: 'companies/', type: 'company', titleStrategy: 'heading' },
|
||||
{ pathPrefix: 'projects/', type: 'project', titleStrategy: 'heading' },
|
||||
{ pathPrefix: 'civic/', type: 'civic', titleStrategy: 'heading' },
|
||||
{ pathPrefix: 'events/', type: 'event', titleStrategy: 'heading', datePattern: 'filename' },
|
||||
{ pathPrefix: 'meetings/', type: 'meeting', titleStrategy: 'heading', datePattern: 'filename' },
|
||||
{ pathPrefix: 'media/', type: 'media', titleStrategy: 'heading' },
|
||||
|
||||
// Catch-all for any remaining files
|
||||
{ pathPrefix: '', type: 'note', titleStrategy: 'heading' },
|
||||
];
|
||||
|
||||
// ─── Date extraction ─────────────────────────────────────────────────
|
||||
|
||||
/** Extract YYYY-MM-DD date from a filename like "2010-04-13 Apr 13 founders mtg.md" */
|
||||
export function extractDateFromFilename(filename: string): string | null {
|
||||
// Pattern 1: YYYY-MM-DD prefix (with - or space separator after)
|
||||
const m1 = filename.match(/^(\d{4}-\d{2}-\d{2})[\s_-]/);
|
||||
if (m1) return m1[1];
|
||||
|
||||
// Pattern 2: YYYY-MM-DD anywhere in filename
|
||||
const m2 = filename.match(/(\d{4}-\d{2}-\d{2})/);
|
||||
if (m2) return m2[1];
|
||||
|
||||
// Pattern 3: "YYYY MM DD" with spaces
|
||||
const m3 = filename.match(/^(\d{4})\s+(\d{2})\s+(\d{2})\s/);
|
||||
if (m3) return `${m3[1]}-${m3[2]}-${m3[3]}`;
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
// ─── Title extraction ────────────────────────────────────────────────
|
||||
|
||||
/** Extract title from filename, stripping date prefix and extension. */
|
||||
export function extractTitleFromFilename(filename: string): string {
|
||||
// Remove .md extension
|
||||
let title = filename.replace(/\.md$/i, '');
|
||||
|
||||
// Strip YYYY-MM-DD prefix (with separator)
|
||||
title = title.replace(/^\d{4}-\d{2}-\d{2}[\s_-]+/, '');
|
||||
|
||||
// Strip YYYY MM DD prefix (space-separated)
|
||||
title = title.replace(/^\d{4}\s+\d{2}\s+\d{2}\s+/, '');
|
||||
|
||||
// Clean up: title case, replace dashes/underscores with spaces
|
||||
title = title
|
||||
.replace(/[-_]/g, ' ')
|
||||
.replace(/\s+/g, ' ')
|
||||
.trim();
|
||||
|
||||
// Don't title-case if it already has mixed case (e.g., "YC presidency")
|
||||
if (title === title.toLowerCase() || title === title.toUpperCase()) {
|
||||
title = title.replace(/\b\w/g, c => c.toUpperCase());
|
||||
}
|
||||
|
||||
return title || 'Untitled';
|
||||
}
|
||||
|
||||
/** Extract title from first heading (# ...) in content. */
|
||||
export function extractTitleFromHeading(content: string): string | null {
|
||||
const lines = content.split('\n');
|
||||
for (const line of lines.slice(0, 20)) {
|
||||
const m = line.match(/^#\s+(.+)/);
|
||||
if (m) return m[1].trim();
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// ─── Core inference ──────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Infer frontmatter for a file that has none.
|
||||
*
|
||||
* @param relativePath - Path relative to brain root (e.g., "Apple Notes/2010-04-13 Apr 13 founders mtg.md")
|
||||
* @param content - File content (may be empty)
|
||||
* @returns Inferred frontmatter fields
|
||||
*/
|
||||
export function inferFrontmatter(relativePath: string, content: string): InferredFrontmatter {
|
||||
// Check if file already has frontmatter
|
||||
const firstNonEmpty = content.split('\n').find(l => l.trim().length > 0);
|
||||
if (firstNonEmpty?.trim() === '---') {
|
||||
return { title: '', type: '', skipped: true };
|
||||
}
|
||||
|
||||
const lowerPath = relativePath.toLowerCase();
|
||||
const filename = basename(relativePath);
|
||||
|
||||
// Find matching rule
|
||||
let matchedRule: DirectoryRule | undefined;
|
||||
for (const rule of DIRECTORY_RULES) {
|
||||
if (lowerPath.startsWith(rule.pathPrefix.toLowerCase())) {
|
||||
matchedRule = rule;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
// Default rule if none matched
|
||||
if (!matchedRule) {
|
||||
matchedRule = { pathPrefix: '', type: 'note', titleStrategy: 'heading' };
|
||||
}
|
||||
|
||||
// Extract date
|
||||
let date: string | undefined;
|
||||
const datePattern = matchedRule.datePattern ?? 'filename';
|
||||
if (datePattern === 'filename') {
|
||||
date = extractDateFromFilename(filename) ?? undefined;
|
||||
}
|
||||
|
||||
// Extract title
|
||||
let title: string;
|
||||
const titleStrategy = matchedRule.titleStrategy ?? 'filename';
|
||||
if (titleStrategy === 'heading') {
|
||||
title = extractTitleFromHeading(content) ?? extractTitleFromFilename(filename);
|
||||
} else if (titleStrategy === 'filename-full') {
|
||||
title = filename.replace(/\.md$/i, '').replace(/[-_]/g, ' ').trim();
|
||||
} else {
|
||||
title = extractTitleFromFilename(filename);
|
||||
}
|
||||
|
||||
// Build tags from rule + subfolder
|
||||
const tags = [...(matchedRule.tags ?? [])];
|
||||
// Add subfolder as tag for Apple Notes (e.g., "YC", "Politics")
|
||||
if (matchedRule.source === 'apple-notes' && matchedRule.pathPrefix === 'apple notes/') {
|
||||
const parts = relativePath.split('/');
|
||||
if (parts.length > 2) {
|
||||
const subfolder = parts[1].toLowerCase().replace(/\s+/g, '-');
|
||||
if (!tags.includes(subfolder)) tags.push(subfolder);
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
title,
|
||||
type: matchedRule.type,
|
||||
date,
|
||||
source: matchedRule.source,
|
||||
tags: tags.length > 0 ? tags : undefined,
|
||||
matchedRule: matchedRule.pathPrefix || '(default)',
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate a YAML frontmatter block from inferred fields.
|
||||
* Returns the `---\n...\n---\n` string to prepend to content.
|
||||
*/
|
||||
export function serializeFrontmatter(fm: InferredFrontmatter): string {
|
||||
if (fm.skipped) return '';
|
||||
|
||||
const lines: string[] = ['---'];
|
||||
|
||||
// Title — quote if it contains special YAML chars
|
||||
const needsQuote = /[:"'#\[\]{}|>&*!?,]/.test(fm.title);
|
||||
lines.push(`title: ${needsQuote ? JSON.stringify(fm.title) : fm.title}`);
|
||||
|
||||
lines.push(`type: ${fm.type}`);
|
||||
|
||||
if (fm.date) {
|
||||
lines.push(`date: "${fm.date}"`);
|
||||
}
|
||||
|
||||
if (fm.source) {
|
||||
lines.push(`source: ${fm.source}`);
|
||||
}
|
||||
|
||||
if (fm.tags && fm.tags.length > 0) {
|
||||
lines.push(`tags: [${fm.tags.map(t => JSON.stringify(t)).join(', ')}]`);
|
||||
}
|
||||
|
||||
lines.push('---');
|
||||
return lines.join('\n') + '\n';
|
||||
}
|
||||
|
||||
/**
|
||||
* Apply frontmatter inference to file content.
|
||||
* Returns the content with frontmatter prepended, or the original content if it already has frontmatter.
|
||||
*/
|
||||
export function applyInference(relativePath: string, content: string): { content: string; inferred: InferredFrontmatter } {
|
||||
const inferred = inferFrontmatter(relativePath, content);
|
||||
if (inferred.skipped) {
|
||||
return { content, inferred };
|
||||
}
|
||||
const fm = serializeFrontmatter(inferred);
|
||||
return { content: fm + '\n' + content, inferred };
|
||||
}
|
||||
+2
-16
@@ -339,7 +339,7 @@ export async function importFromFile(
|
||||
engine: BrainEngine,
|
||||
filePath: string,
|
||||
relativePath: string,
|
||||
opts: { noEmbed?: boolean; inferFrontmatter?: boolean } = {},
|
||||
opts: { noEmbed?: boolean } = {},
|
||||
): Promise<ImportResult> {
|
||||
// Defense-in-depth: reject symlinks before reading content.
|
||||
const lstat = lstatSync(filePath);
|
||||
@@ -352,27 +352,13 @@ export async function importFromFile(
|
||||
return { slug: relativePath, status: 'skipped', chunks: 0, error: `File too large (${stat.size} bytes)` };
|
||||
}
|
||||
|
||||
let content = readFileSync(filePath, 'utf-8');
|
||||
const content = readFileSync(filePath, 'utf-8');
|
||||
|
||||
// Route code files through the code import path
|
||||
if (isCodeFilePath(relativePath)) {
|
||||
return importCodeFile(engine, relativePath, content, opts);
|
||||
}
|
||||
|
||||
// v0.22.8 — Frontmatter inference: if the file has no frontmatter and
|
||||
// inference is enabled, synthesize it from the filesystem path + content.
|
||||
// This turns bare markdown files into fully-typed, dated, tagged pages
|
||||
// without requiring the user to manually add YAML headers.
|
||||
// The inference is applied to the in-memory content only; the file on disk
|
||||
// is not modified. Use `gbrain frontmatter generate --fix` to write back.
|
||||
if (opts.inferFrontmatter !== false) {
|
||||
const { applyInference } = await import('./frontmatter-inference.ts');
|
||||
const { content: inferred, inferred: meta } = applyInference(relativePath, content);
|
||||
if (!meta.skipped) {
|
||||
content = inferred;
|
||||
}
|
||||
}
|
||||
|
||||
const parsed = parseMarkdown(content, relativePath);
|
||||
|
||||
// Enforce path-authoritative slug. parseMarkdown prefers frontmatter.slug over
|
||||
|
||||
@@ -1073,34 +1073,6 @@ export const MIGRATIONS: Migration[] = [
|
||||
},
|
||||
sql: '',
|
||||
},
|
||||
{
|
||||
version: 30,
|
||||
name: 'dream_verdicts_table',
|
||||
// v0.23 synthesize phase: cache for "is this transcript worth processing?"
|
||||
// verdict from the cheap Haiku judge. Distinct from raw_data (page-scoped);
|
||||
// transcripts aren't pages. Keyed by (file_path, content_hash) so edited
|
||||
// transcripts re-judge automatically. Backfill re-runs hit cache instead
|
||||
// of paying for Haiku 100x.
|
||||
sql: `
|
||||
CREATE TABLE IF NOT EXISTS dream_verdicts (
|
||||
file_path TEXT NOT NULL,
|
||||
content_hash TEXT NOT NULL,
|
||||
worth_processing BOOLEAN NOT NULL,
|
||||
reasons JSONB,
|
||||
judged_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
PRIMARY KEY (file_path, content_hash)
|
||||
);
|
||||
DO $$
|
||||
DECLARE
|
||||
has_bypass BOOLEAN;
|
||||
BEGIN
|
||||
SELECT rolbypassrls INTO has_bypass FROM pg_roles WHERE rolname = current_user;
|
||||
IF has_bypass THEN
|
||||
ALTER TABLE dream_verdicts ENABLE ROW LEVEL SECURITY;
|
||||
END IF;
|
||||
END $$;
|
||||
`,
|
||||
},
|
||||
];
|
||||
|
||||
export const LATEST_VERSION = MIGRATIONS.length > 0
|
||||
|
||||
@@ -18,7 +18,7 @@
|
||||
|
||||
import * as fs from 'node:fs';
|
||||
import * as path from 'node:path';
|
||||
import { gbrainPath } from '../config.ts';
|
||||
import * as os from 'node:os';
|
||||
|
||||
export interface BackpressureAuditEvent {
|
||||
ts: string;
|
||||
@@ -54,7 +54,7 @@ export function computeAuditFilename(now: Date = new Date()): string {
|
||||
export function resolveAuditDir(): string {
|
||||
const override = process.env.GBRAIN_AUDIT_DIR;
|
||||
if (override && override.trim().length > 0) return override;
|
||||
return gbrainPath('audit');
|
||||
return path.join(os.homedir(), '.gbrain', 'audit');
|
||||
}
|
||||
|
||||
export function logBackpressureCoalesce(event: Omit<BackpressureAuditEvent, 'ts' | 'decision'>): void {
|
||||
|
||||
@@ -15,7 +15,7 @@
|
||||
|
||||
import * as fs from 'node:fs';
|
||||
import * as path from 'node:path';
|
||||
import { gbrainPath } from '../../config.ts';
|
||||
import * as os from 'node:os';
|
||||
|
||||
export interface ShellAuditEvent {
|
||||
ts: string;
|
||||
@@ -53,7 +53,7 @@ export function computeAuditFilename(now: Date = new Date()): string {
|
||||
export function resolveAuditDir(): string {
|
||||
const override = process.env.GBRAIN_AUDIT_DIR;
|
||||
if (override && override.trim().length > 0) return override;
|
||||
return gbrainPath('audit');
|
||||
return path.join(os.homedir(), '.gbrain', 'audit');
|
||||
}
|
||||
|
||||
export function logShellSubmission(event: Omit<ShellAuditEvent, 'ts'>): void {
|
||||
|
||||
@@ -149,14 +149,10 @@ export function makeSubagentHandler(deps: SubagentDeps) {
|
||||
const systemPrompt = data.system ?? DEFAULT_SYSTEM;
|
||||
|
||||
// Build the tool registry bound to THIS job as the owning subagent.
|
||||
// allowed_slug_prefixes (v0.23) flows through buildBrainTools → the
|
||||
// put_page schema description AND the OperationContext, so the model's
|
||||
// tool schema and the server-side check stay in sync.
|
||||
const registry = deps.toolRegistry ?? buildBrainTools({
|
||||
subagentId: ctx.id,
|
||||
engine,
|
||||
config,
|
||||
allowedSlugPrefixes: data.allowed_slug_prefixes,
|
||||
});
|
||||
const toolDefs = data.allowed_tools && data.allowed_tools.length > 0
|
||||
? filterAllowedTools(registry, data.allowed_tools)
|
||||
|
||||
@@ -225,12 +225,8 @@ export class MinionSupervisor {
|
||||
process.on('SIGTERM', this.sigtermListener);
|
||||
process.on('SIGINT', this.sigintListener);
|
||||
|
||||
// 4. Health monitoring. Skip when healthInterval=0 — that's the explicit
|
||||
// "disable" contract documented on `--health-interval 0`. setInterval(0)
|
||||
// would be a tight DB-hammering loop, not the no-op users expect.
|
||||
if (this.opts.healthInterval > 0) {
|
||||
this.healthTimer = setInterval(() => { void this.healthCheck(); }, this.opts.healthInterval);
|
||||
}
|
||||
// 4. Health monitoring.
|
||||
this.healthTimer = setInterval(() => { void this.healthCheck(); }, this.opts.healthInterval);
|
||||
|
||||
// 5. Announce start.
|
||||
this.emit('started', {
|
||||
@@ -431,11 +427,6 @@ export class MinionSupervisor {
|
||||
} else {
|
||||
delete env.GBRAIN_ALLOW_SHELL_JOBS;
|
||||
}
|
||||
// Signal to the child worker that it's running under a supervisor.
|
||||
// The worker's self-health-check (DB probes, stall detection) is
|
||||
// redundant when the supervisor already provides these — setting
|
||||
// this env var causes the worker to skip its own health timer.
|
||||
env.GBRAIN_SUPERVISED = '1';
|
||||
|
||||
this.lastStartTime = Date.now();
|
||||
|
||||
|
||||
@@ -91,37 +91,19 @@ function paramsToInputSchema(op: Operation): Record<string, unknown> {
|
||||
|
||||
/**
|
||||
* For put_page specifically, the tool schema shown to the model constrains
|
||||
* `slug`. Two modes:
|
||||
*
|
||||
* - Default (legacy): slug MUST start with `wiki/agents/<subagentId>/`,
|
||||
* enforced by both the JSONSchema `pattern` and the server-side check.
|
||||
* - Trusted-workspace (v0.23 dream cycle): when `allowedSlugPrefixes` is
|
||||
* set, the model is told the allowed prefixes in plain English (no
|
||||
* regex pattern — the prefix list is authoritative server-side, and
|
||||
* JSONSchema can't express "matches any of these globs" cleanly).
|
||||
* `slug` to `wiki/agents/<subagentId>/...`. The server-side check in
|
||||
* operations.ts is the authoritative gate; this just helps the model write
|
||||
* correct slugs on the first try.
|
||||
*/
|
||||
function namespacedPutPageSchema(
|
||||
op: Operation,
|
||||
subagentId: number,
|
||||
allowedSlugPrefixes?: readonly string[],
|
||||
): Record<string, unknown> {
|
||||
function namespacedPutPageSchema(op: Operation, subagentId: number): Record<string, unknown> {
|
||||
const base = paramsToInputSchema(op);
|
||||
const props = (base.properties as Record<string, Record<string, unknown>>) ?? {};
|
||||
if (props.slug) {
|
||||
if (allowedSlugPrefixes && allowedSlugPrefixes.length > 0) {
|
||||
props.slug = {
|
||||
...props.slug,
|
||||
description:
|
||||
`Page slug. MUST match one of these prefix globs: ${allowedSlugPrefixes.join(', ')}. ` +
|
||||
`Slugs use lowercase alphanumeric segments separated by '/'. No leading slash, no '.md' extension, no underscores.`,
|
||||
};
|
||||
} else {
|
||||
props.slug = {
|
||||
...props.slug,
|
||||
description: `Page slug. MUST start with "wiki/agents/${subagentId}/" (agents can only write under their own namespace).`,
|
||||
pattern: `^wiki/agents/${subagentId}/.+`,
|
||||
};
|
||||
}
|
||||
props.slug = {
|
||||
...props.slug,
|
||||
description: `Page slug. MUST start with "wiki/agents/${subagentId}/" (agents can only write under their own namespace).`,
|
||||
pattern: `^wiki/agents/${subagentId}/.+`,
|
||||
};
|
||||
}
|
||||
return { ...base, properties: props };
|
||||
}
|
||||
@@ -133,14 +115,6 @@ export interface BuildBrainToolsOpts {
|
||||
config: GBrainConfig;
|
||||
/** Optional filter: only include names in this set. */
|
||||
allowedNames?: ReadonlySet<string>;
|
||||
/**
|
||||
* Trusted-workspace allow-list (v0.23). When set, put_page is bounded
|
||||
* to slugs matching these prefix globs instead of the legacy
|
||||
* `wiki/agents/<id>/...` namespace. Trust comes from PROTECTED_JOB_NAMES
|
||||
* (MCP can't submit subagent jobs) — this flows from
|
||||
* SubagentHandlerData.allowed_slug_prefixes via the handler.
|
||||
*/
|
||||
allowedSlugPrefixes?: readonly string[];
|
||||
}
|
||||
|
||||
interface OpContextDeps {
|
||||
@@ -149,7 +123,6 @@ interface OpContextDeps {
|
||||
subagentId: number;
|
||||
jobId: number;
|
||||
signal?: AbortSignal;
|
||||
allowedSlugPrefixes?: readonly string[];
|
||||
}
|
||||
|
||||
function buildOpContext(deps: OpContextDeps): OperationContext {
|
||||
@@ -162,13 +135,10 @@ function buildOpContext(deps: OpContextDeps): OperationContext {
|
||||
error: (msg: string) => process.stderr.write(`[subagent-tool:${deps.jobId}] ERROR: ${msg}\n`),
|
||||
},
|
||||
dryRun: false,
|
||||
remote: true, // match MCP trust boundary for auto-link skip
|
||||
remote: true, // match MCP trust boundary
|
||||
jobId: deps.jobId,
|
||||
subagentId: deps.subagentId,
|
||||
viaSubagent: true, // FAIL-CLOSED: put_page etc. enforce namespace
|
||||
allowedSlugPrefixes: deps.allowedSlugPrefixes
|
||||
? [...deps.allowedSlugPrefixes]
|
||||
: undefined,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -187,7 +157,7 @@ export function buildBrainTools(opts: BuildBrainToolsOpts): ToolDef[] {
|
||||
|
||||
return picked.map<ToolDef>(op => {
|
||||
const schema = op.name === 'put_page'
|
||||
? namespacedPutPageSchema(op, opts.subagentId, opts.allowedSlugPrefixes)
|
||||
? namespacedPutPageSchema(op, opts.subagentId)
|
||||
: paramsToInputSchema(op);
|
||||
|
||||
const toolName = sanitizeToolName(op.name);
|
||||
@@ -209,7 +179,6 @@ export function buildBrainTools(opts: BuildBrainToolsOpts): ToolDef[] {
|
||||
subagentId: opts.subagentId,
|
||||
jobId: ctx.jobId,
|
||||
signal: ctx.signal,
|
||||
allowedSlugPrefixes: opts.allowedSlugPrefixes,
|
||||
});
|
||||
const params = (input && typeof input === 'object') ? input as Record<string, unknown> : {};
|
||||
return op.handler(opCtx, params);
|
||||
|
||||
@@ -170,25 +170,6 @@ export interface MinionWorkerOpts {
|
||||
* case where all concurrency slots are wedged with zero job completions
|
||||
* so the per-job check never fires. */
|
||||
rssCheckInterval?: number;
|
||||
/** Self-health-check interval in ms. 0 = disabled. Default: 60000 (1 minute).
|
||||
* Automatically disabled when running under a supervisor (GBRAIN_SUPERVISED=1).
|
||||
* Provides DB liveness probes and stall detection for bare `gbrain jobs work`
|
||||
* deployments managed by external process managers (systemd, Docker, cron). */
|
||||
healthCheckInterval?: number;
|
||||
/** Stall detection: ms of continuous idle (waiting>0, inFlight=0, no completions)
|
||||
* before emitting the first warning. Default: 300000 (5 minutes). */
|
||||
stallWarnAfterMs?: number;
|
||||
/** Stall detection: ms of continuous idle before emitting `'unhealthy'` with
|
||||
* reason='stalled'. Default: 600000 (10 minutes). Must be > stallWarnAfterMs. */
|
||||
stallExitAfterMs?: number;
|
||||
/** DB liveness probe: number of consecutive failed `SELECT 1` probes before
|
||||
* emitting `'unhealthy'` with reason='db_dead'. Default: 3. */
|
||||
dbFailExitAfter?: number;
|
||||
/** Per-probe wall-clock timeout in ms. A `SELECT 1` that hangs longer than
|
||||
* this counts as a failure (fed into dbFailExitAfter). Without this, a
|
||||
* hung probe would wedge the recursive setTimeout chain forever and
|
||||
* silently disable the health monitor. Default: 10000 (10 seconds). */
|
||||
dbProbeTimeoutMs?: number;
|
||||
}
|
||||
|
||||
// --- Job Context (passed to handlers) ---
|
||||
@@ -421,19 +402,6 @@ export interface SubagentHandlerData {
|
||||
system?: string;
|
||||
/** Template variables for subagent_def. Arbitrary JSON-serializable. */
|
||||
input_vars?: Record<string, unknown>;
|
||||
/**
|
||||
* Trusted-workspace allow-list for put_page (v0.23 dream cycle).
|
||||
*
|
||||
* When set, the subagent's put_page calls are bounded to slugs matching
|
||||
* any of these prefix globs (e.g. ["wiki/personal/reflections/*",
|
||||
* "wiki/originals/*"]). When unset/empty, the legacy
|
||||
* `wiki/agents/<subagentId>/...` namespace check applies.
|
||||
*
|
||||
* Trust comes from PROTECTED_JOB_NAMES gating subagent submission — MCP
|
||||
* cannot reach this field. Only cycle.ts (synthesize/patterns phases)
|
||||
* and direct CLI submitters set it.
|
||||
*/
|
||||
allowed_slug_prefixes?: string[];
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
+1
-209
@@ -20,15 +20,8 @@ import { UnrecoverableError } from './types.ts';
|
||||
import { MinionQueue } from './queue.ts';
|
||||
import { calculateBackoff } from './backoff.ts';
|
||||
import { randomUUID } from 'crypto';
|
||||
import { EventEmitter } from 'events';
|
||||
import { evaluateQuietHours, type QuietHoursConfig } from './quiet-hours.ts';
|
||||
|
||||
/** Reason payload emitted with `'unhealthy'` when self-health-check trips.
|
||||
* CLI layer (jobs.ts:work) subscribes and decides whether to call process.exit. */
|
||||
export type UnhealthyReason =
|
||||
| { reason: 'db_dead'; consecutiveFailures: number; message: string }
|
||||
| { reason: 'stalled'; waitingCount: number; idleMinutes: number };
|
||||
|
||||
/**
|
||||
* Read the quiet_hours JSONB column off a MinionJob, if present. The
|
||||
* column was added in schema migration v12; older rows + versions of
|
||||
@@ -49,13 +42,7 @@ interface InFlightJob {
|
||||
promise: Promise<void>;
|
||||
}
|
||||
|
||||
/** Type-safe `on('unhealthy', ...)` for callers. */
|
||||
export interface MinionWorker {
|
||||
on(event: 'unhealthy', listener: (info: UnhealthyReason) => void): this;
|
||||
emit(event: 'unhealthy', info: UnhealthyReason): boolean;
|
||||
}
|
||||
|
||||
export class MinionWorker extends EventEmitter {
|
||||
export class MinionWorker {
|
||||
private queue: MinionQueue;
|
||||
private handlers = new Map<string, MinionHandler>();
|
||||
private running = false;
|
||||
@@ -80,7 +67,6 @@ export class MinionWorker extends EventEmitter {
|
||||
private engine: BrainEngine,
|
||||
opts?: MinionWorkerOpts & MinionQueueOpts,
|
||||
) {
|
||||
super();
|
||||
this.queue = new MinionQueue(engine, {
|
||||
maxSpawnDepth: opts?.maxSpawnDepth,
|
||||
maxAttachmentBytes: opts?.maxAttachmentBytes,
|
||||
@@ -95,25 +81,7 @@ export class MinionWorker extends EventEmitter {
|
||||
maxRssMb: opts?.maxRssMb ?? 0,
|
||||
getRss: opts?.getRss ?? (() => process.memoryUsage().rss),
|
||||
rssCheckInterval: opts?.rssCheckInterval ?? 60000,
|
||||
healthCheckInterval: opts?.healthCheckInterval ?? 60000,
|
||||
stallWarnAfterMs: opts?.stallWarnAfterMs ?? 5 * 60_000,
|
||||
stallExitAfterMs: opts?.stallExitAfterMs ?? 10 * 60_000,
|
||||
dbFailExitAfter: opts?.dbFailExitAfter ?? 3,
|
||||
dbProbeTimeoutMs: opts?.dbProbeTimeoutMs ?? 10_000,
|
||||
};
|
||||
// Stall thresholds contract: exit MUST be strictly greater than warn.
|
||||
// If exit <= warn, the warn-then-exit semantics break: a single tick at
|
||||
// idle > warn would set stallWarningSince and the subsequent tick at
|
||||
// idle > exit could fire immediately without giving operators visibility.
|
||||
// Reject misconfigurations at construction time so the failure mode is
|
||||
// a loud throw on startup rather than a quiet contract violation.
|
||||
if (this.opts.stallExitAfterMs <= this.opts.stallWarnAfterMs) {
|
||||
throw new Error(
|
||||
`MinionWorkerOpts: stallExitAfterMs (${this.opts.stallExitAfterMs}) must be > ` +
|
||||
`stallWarnAfterMs (${this.opts.stallWarnAfterMs}). ` +
|
||||
`The contract is "warn first, exit later" — they cannot fire on the same tick.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/** Register a handler for a job type. */
|
||||
@@ -126,28 +94,6 @@ export class MinionWorker extends EventEmitter {
|
||||
return Array.from(this.handlers.keys());
|
||||
}
|
||||
|
||||
/** Emit 'unhealthy' with a no-listener fallback. The default contract is
|
||||
* fail-stop: pre-EventEmitter-refactor behavior was process.exit(1) inside
|
||||
* the timer; the refactor moved that responsibility to the CLI subscriber.
|
||||
* But direct API consumers without a listener would see emit() become a
|
||||
* no-op AND `healthExited=true` permanently disabling monitoring — a
|
||||
* silent regression. Solution: if no one subscribed, log and exit
|
||||
* ourselves so the worker dies and the PM restarts it. Subscribers
|
||||
* override this default by adding a listener before start(). */
|
||||
private emitUnhealthy(info: UnhealthyReason): void {
|
||||
if (this.listenerCount('unhealthy') === 0) {
|
||||
const detail = info.reason === 'db_dead'
|
||||
? `DB unreachable (${info.consecutiveFailures} probes): ${info.message}`
|
||||
: `worker stalled (${info.waitingCount} waiting, ${info.idleMinutes}m idle)`;
|
||||
console.error(
|
||||
`[health] FATAL: ${detail}. No 'unhealthy' listener registered; ` +
|
||||
`defaulting to process.exit(1) for process-manager restart.`,
|
||||
);
|
||||
process.exit(1);
|
||||
}
|
||||
this.emit('unhealthy', info);
|
||||
}
|
||||
|
||||
/** Start the worker loop. Blocks until stopped. */
|
||||
async start(): Promise<void> {
|
||||
if (this.handlers.size === 0) {
|
||||
@@ -209,159 +155,6 @@ export class MinionWorker extends EventEmitter {
|
||||
}, this.opts.rssCheckInterval);
|
||||
}
|
||||
|
||||
// Self-health-check — provides supervisor-grade monitoring for bare workers.
|
||||
// Disabled when running under a supervisor (GBRAIN_SUPERVISED=1) or when
|
||||
// healthCheckInterval is 0. Catches two failure modes that leave the process
|
||||
// alive but non-functional:
|
||||
// 1. DB connection death (Supabase/PgBouncer drops, network blip)
|
||||
// 2. Worker stall (event loop alive but not claiming/completing jobs)
|
||||
//
|
||||
// On failure, emits an `'unhealthy'` event with a structured reason. The
|
||||
// CLI layer (`src/commands/jobs.ts:work`) subscribes and decides whether to
|
||||
// call process.exit. Library code never calls process.exit directly so
|
||||
// MinionWorker stays embeddable in non-CLI contexts (tests, other hosts).
|
||||
//
|
||||
// Timer pattern: recursive setTimeout with a `running` flag, not setInterval.
|
||||
// setInterval queues callbacks even when the prior is still awaiting; on a
|
||||
// hung DB probe that piles up overlapping async checks racing on
|
||||
// `consecutiveDbFailures`. The recursive pattern guarantees one tick at a time.
|
||||
const isSupervisedChild = process.env.GBRAIN_SUPERVISED === '1';
|
||||
let healthTimer: ReturnType<typeof setTimeout> | null = null;
|
||||
if (!isSupervisedChild && this.opts.healthCheckInterval > 0) {
|
||||
let consecutiveDbFailures = 0;
|
||||
let lastKnownCompleted = this.jobsCompleted;
|
||||
let lastCompletionTime = Date.now();
|
||||
let stallWarningSince: number | null = null;
|
||||
let healthRunning = false;
|
||||
let healthExited = false;
|
||||
|
||||
// Race executeRaw against a wall-clock deadline. A hung connection
|
||||
// (network-partitioned PgBouncer, deadlocked backend) would otherwise
|
||||
// hold the await forever — the recursive setTimeout's next tick is only
|
||||
// scheduled in `finally`, so a hung probe would silently disable the
|
||||
// entire health monitor. The timeout treats hangs as failures and feeds
|
||||
// them into `dbFailExitAfter`.
|
||||
const probeWithTimeout = async (): Promise<void> => {
|
||||
const ac = new AbortController();
|
||||
const timeoutMs = this.opts.dbProbeTimeoutMs;
|
||||
const timer = setTimeout(() => ac.abort(), timeoutMs);
|
||||
try {
|
||||
await Promise.race([
|
||||
this.engine.executeRaw('SELECT 1'),
|
||||
new Promise<never>((_, reject) => {
|
||||
ac.signal.addEventListener('abort', () => {
|
||||
reject(new Error(`probe timeout after ${timeoutMs}ms`));
|
||||
});
|
||||
}),
|
||||
]);
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
};
|
||||
|
||||
const runHealthCheck = async (): Promise<void> => {
|
||||
if (healthRunning || !this.running || healthExited) return;
|
||||
healthRunning = true;
|
||||
try {
|
||||
// --- 1. DB liveness probe ---
|
||||
try {
|
||||
await probeWithTimeout();
|
||||
consecutiveDbFailures = 0;
|
||||
} catch (e) {
|
||||
consecutiveDbFailures++;
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
console.error(
|
||||
`[health] DB probe failed (${consecutiveDbFailures}/${this.opts.dbFailExitAfter}): ${msg}`,
|
||||
);
|
||||
if (consecutiveDbFailures >= this.opts.dbFailExitAfter) {
|
||||
console.error(
|
||||
`[health] DB unreachable after ${this.opts.dbFailExitAfter} consecutive probes. ` +
|
||||
`Emitting 'unhealthy' for process-manager restart.`,
|
||||
);
|
||||
healthExited = true;
|
||||
this.emitUnhealthy({
|
||||
reason: 'db_dead',
|
||||
consecutiveFailures: consecutiveDbFailures,
|
||||
message: msg,
|
||||
});
|
||||
}
|
||||
return; // Skip stall check when DB is flaky
|
||||
}
|
||||
|
||||
// --- 2. Stall detection ---
|
||||
if (this.jobsCompleted > lastKnownCompleted) {
|
||||
lastKnownCompleted = this.jobsCompleted;
|
||||
lastCompletionTime = Date.now();
|
||||
stallWarningSince = null;
|
||||
}
|
||||
|
||||
const idleMs = Date.now() - lastCompletionTime;
|
||||
|
||||
// Only check for stalls when no jobs are in-flight and it's been a while
|
||||
if (idleMs > this.opts.stallWarnAfterMs && this.inFlight.size === 0) {
|
||||
try {
|
||||
// Filter by registered handler names so a worker that doesn't
|
||||
// claim a particular job-name doesn't false-positive when those
|
||||
// jobs accumulate in `waiting`. Only counts work THIS worker would
|
||||
// actually have claimed.
|
||||
const handlerNames = this.registeredNames;
|
||||
const rows = handlerNames.length === 0
|
||||
? [] as { cnt: string }[]
|
||||
: await this.engine.executeRaw<{ cnt: string }>(
|
||||
`SELECT count(*)::text AS cnt FROM minion_jobs
|
||||
WHERE status = 'waiting'
|
||||
AND queue = $1
|
||||
AND name = ANY($2::text[])`,
|
||||
[this.opts.queue, handlerNames],
|
||||
);
|
||||
const waiting = parseInt(rows[0]?.cnt ?? '0', 10);
|
||||
const idleMinutes = Math.round(idleMs / 60_000);
|
||||
if (waiting > 0) {
|
||||
// Two thresholds, both measured from `lastCompletionTime` (NOT
|
||||
// from when the warning fired). With defaults (warn=5min,
|
||||
// exit=10min), the first warning fires at idle=5min and the
|
||||
// unhealthy emit fires at idle=10min — matching the contract
|
||||
// documented in MinionWorkerOpts.
|
||||
if (!stallWarningSince) {
|
||||
stallWarningSince = Date.now();
|
||||
console.warn(
|
||||
`[health] Possible stall: ${waiting} waiting job(s) for ` +
|
||||
`registered handlers, 0 in-flight, ${idleMinutes}m since last completion`,
|
||||
);
|
||||
} else if (idleMs > this.opts.stallExitAfterMs) {
|
||||
console.error(
|
||||
`[health] Worker stalled for ${Math.round(this.opts.stallExitAfterMs / 60_000)}+ ` +
|
||||
`minutes with ${waiting} waiting job(s). Emitting 'unhealthy' for process-manager restart.`,
|
||||
);
|
||||
healthExited = true;
|
||||
this.emitUnhealthy({
|
||||
reason: 'stalled',
|
||||
waitingCount: waiting,
|
||||
idleMinutes,
|
||||
});
|
||||
}
|
||||
} else {
|
||||
stallWarningSince = null; // Queue empty (for our handlers) — not stalled, just idle
|
||||
}
|
||||
} catch {
|
||||
// DB query failed — the liveness probe above will catch persistent failures
|
||||
}
|
||||
} else {
|
||||
stallWarningSince = null;
|
||||
}
|
||||
} finally {
|
||||
healthRunning = false;
|
||||
if (this.running && !healthExited) {
|
||||
healthTimer = setTimeout(runHealthCheck, this.opts.healthCheckInterval);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// First tick scheduled after one interval so newly-started workers have
|
||||
// a chance to do real work before the stall clock starts ticking.
|
||||
healthTimer = setTimeout(runHealthCheck, this.opts.healthCheckInterval);
|
||||
}
|
||||
|
||||
try {
|
||||
while (this.running) {
|
||||
// Promote delayed jobs
|
||||
@@ -408,7 +201,6 @@ export class MinionWorker extends EventEmitter {
|
||||
} finally {
|
||||
clearInterval(stalledTimer);
|
||||
if (rssTimer) clearInterval(rssTimer);
|
||||
if (healthTimer) clearTimeout(healthTimer); // recursive setTimeout pattern
|
||||
process.removeListener('SIGTERM', shutdown);
|
||||
process.removeListener('SIGINT', shutdown);
|
||||
|
||||
|
||||
+4
-68
@@ -120,31 +120,6 @@ export function validatePageSlug(slug: string): void {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Match a slug against a list of allow-list prefix globs.
|
||||
*
|
||||
* Glob form: `<prefix>/*` matches any slug starting with `<prefix>/` and
|
||||
* having at least one more segment (single or multi). Bare `<prefix>` (no
|
||||
* trailing `/*`) matches that exact slug only. The `*` is intentionally
|
||||
* permissive — depth is unbounded, so `wiki/originals/*` matches both
|
||||
* `wiki/originals/idea-x` and `wiki/originals/ideas/2026-04-25-idea-y`.
|
||||
*
|
||||
* Used by the v0.23 dream-cycle trusted-workspace path. Order doesn't
|
||||
* matter; the first match wins (returns true on any match).
|
||||
*/
|
||||
export function matchesSlugAllowList(slug: string, prefixes: readonly string[]): boolean {
|
||||
for (const p of prefixes) {
|
||||
if (p.endsWith('/*')) {
|
||||
const base = p.slice(0, -2);
|
||||
if (slug === base) continue;
|
||||
if (slug.startsWith(base + '/')) return true;
|
||||
} else if (p === slug) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Allowlist validator for uploaded file basenames. Rejects control chars, backslashes,
|
||||
* RTL overrides (\u202E), leading dot (hidden files) and leading dash (CLI flag confusion).
|
||||
@@ -206,22 +181,6 @@ export interface OperationContext {
|
||||
jobId?: number;
|
||||
subagentId?: number;
|
||||
viaSubagent?: boolean;
|
||||
/**
|
||||
* Trusted-workspace allow-list (v0.23 dream cycle). When the cycle's
|
||||
* synthesize/patterns phases dispatch a subagent, they thread an
|
||||
* explicit list of slug-prefix globs (e.g. "wiki/personal/reflections/*")
|
||||
* through this field. put_page enforces it BEFORE the legacy
|
||||
* `wiki/agents/<id>/...` namespace check.
|
||||
*
|
||||
* Trust comes from the SUBMITTER (subagent jobs are gated by
|
||||
* PROTECTED_JOB_NAMES — MCP cannot submit them), not from `remote`.
|
||||
* Every subagent tool call has `remote=true` for auto-link safety,
|
||||
* so basing trust on `remote` is incoherent (would always reject).
|
||||
*
|
||||
* Empty / unset → fall back to the legacy namespace check (existing
|
||||
* v0.15 behavior; pure addition, no regression).
|
||||
*/
|
||||
allowedSlugPrefixes?: string[];
|
||||
/**
|
||||
* Resolved global CLI options (--quiet / --progress-json / --progress-interval).
|
||||
* CLI callers populate this from `getCliOptions()`. MCP / library callers
|
||||
@@ -305,23 +264,9 @@ const put_page: Operation = {
|
||||
if (typeof ctx.subagentId !== 'number' || Number.isNaN(ctx.subagentId)) {
|
||||
throw new OperationError('permission_denied', 'put_page via subagent requires ctx.subagentId');
|
||||
}
|
||||
const allowList = ctx.allowedSlugPrefixes;
|
||||
if (allowList && allowList.length > 0) {
|
||||
// Trusted-workspace path: explicit allow-list bounds writes.
|
||||
// Set only by cycle.ts (synthesize/patterns) which submits subagent
|
||||
// jobs under PROTECTED_JOB_NAMES — MCP cannot reach this branch.
|
||||
if (!matchesSlugAllowList(slug, allowList)) {
|
||||
throw new OperationError(
|
||||
'permission_denied',
|
||||
`put_page slug '${slug}' is not within the trusted-workspace allow-list (${allowList.join(', ')})`
|
||||
);
|
||||
}
|
||||
} else {
|
||||
// Legacy default: agent-namespace confinement.
|
||||
const prefix = `wiki/agents/${ctx.subagentId}/`;
|
||||
if (!slug.startsWith(prefix) || slug.length === prefix.length) {
|
||||
throw new OperationError('permission_denied', `put_page via subagent must write under '${prefix}...'`);
|
||||
}
|
||||
const prefix = `wiki/agents/${ctx.subagentId}/`;
|
||||
if (!slug.startsWith(prefix) || slug.length === prefix.length) {
|
||||
throw new OperationError('permission_denied', `put_page via subagent must write under '${prefix}...'`);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -350,16 +295,7 @@ const put_page: Operation = {
|
||||
| { skipped: 'remote' }
|
||||
| undefined;
|
||||
let autoTimeline: { created: number } | { error: string } | { skipped: 'remote' } | undefined;
|
||||
// Trusted-workspace path (v0.23 dream cycle) re-enables auto-link/timeline
|
||||
// even though ctx.remote=true, because the allow-list bounds the slug and
|
||||
// the synthesis prompt is itself the trusted dispatcher. Without this,
|
||||
// the cycle's `extract` phase would have to recompute every edge, and
|
||||
// patterns (which runs after extract) would still see the right graph
|
||||
// but auto_timeline would never fire on synth output.
|
||||
const trustedWorkspace = ctx.viaSubagent === true
|
||||
&& Array.isArray(ctx.allowedSlugPrefixes)
|
||||
&& ctx.allowedSlugPrefixes.length > 0;
|
||||
if (ctx.remote === true && !trustedWorkspace) {
|
||||
if (ctx.remote === true) {
|
||||
autoLinks = { skipped: 'remote' };
|
||||
autoTimeline = { skipped: 'remote' };
|
||||
} else if (result.parsedPage) {
|
||||
|
||||
@@ -18,8 +18,8 @@
|
||||
*/
|
||||
|
||||
import { appendFileSync, existsSync, mkdirSync } from 'fs';
|
||||
import { dirname } from 'path';
|
||||
import { gbrainPath } from '../config.ts';
|
||||
import { homedir } from 'os';
|
||||
import { dirname, join } from 'path';
|
||||
|
||||
import type { BrainEngine } from '../engine.ts';
|
||||
import {
|
||||
@@ -30,7 +30,7 @@ import {
|
||||
} from './validators/index.ts';
|
||||
import type { ValidationFinding, PageValidator } from './writer.ts';
|
||||
|
||||
const getLintLogFile = () => gbrainPath('validator-lint.jsonl');
|
||||
const LINT_LOG_FILE = join(homedir(), '.gbrain', 'validator-lint.jsonl');
|
||||
const LINT_CONFIG_KEY = 'writer.lint_on_put_page';
|
||||
|
||||
export interface PostWriteLintOpts {
|
||||
@@ -124,8 +124,7 @@ export async function runPostWriteLint(
|
||||
|
||||
function writeLocalLintLog(slug: string, findings: ValidationFinding[]): void {
|
||||
try {
|
||||
const lintLogFile = getLintLogFile();
|
||||
const dir = dirname(lintLogFile);
|
||||
const dir = dirname(LINT_LOG_FILE);
|
||||
if (!existsSync(dir)) mkdirSync(dir, { recursive: true });
|
||||
const line = JSON.stringify({
|
||||
ts: new Date().toISOString(),
|
||||
@@ -134,7 +133,7 @@ function writeLocalLintLog(slug: string, findings: ValidationFinding[]): void {
|
||||
warning_count: findings.filter(f => f.severity === 'warning').length,
|
||||
findings: findings.slice(0, 20), // cap to prevent runaway log size
|
||||
}) + '\n';
|
||||
appendFileSync(lintLogFile, line, 'utf-8');
|
||||
appendFileSync(LINT_LOG_FILE, line, 'utf-8');
|
||||
} catch {
|
||||
// Non-fatal; logging failure shouldn't break the main flow.
|
||||
}
|
||||
|
||||
@@ -2,7 +2,7 @@ import { PGlite } from '@electric-sql/pglite';
|
||||
import { vector } from '@electric-sql/pglite/vector';
|
||||
import { pg_trgm } from '@electric-sql/pglite/contrib/pg_trgm';
|
||||
import type { Transaction } from '@electric-sql/pglite';
|
||||
import type { BrainEngine, LinkBatchInput, TimelineBatchInput, ReservedConnection, DreamVerdict, DreamVerdictInput } from './engine.ts';
|
||||
import type { BrainEngine, LinkBatchInput, TimelineBatchInput, ReservedConnection } from './engine.ts';
|
||||
import { MAX_SEARCH_LIMIT, clampSearchLimit } from './engine.ts';
|
||||
import { runMigrations } from './migrate.ts';
|
||||
import { PGLITE_SCHEMA_SQL } from './pglite-schema.ts';
|
||||
@@ -1157,39 +1157,6 @@ export class PGLiteEngine implements BrainEngine {
|
||||
return result.rows as unknown as RawData[];
|
||||
}
|
||||
|
||||
// Dream-cycle significance verdict cache (v0.23).
|
||||
async getDreamVerdict(filePath: string, contentHash: string): Promise<DreamVerdict | null> {
|
||||
const result = await this.db.query<{
|
||||
worth_processing: boolean;
|
||||
reasons: string[] | null;
|
||||
judged_at: Date | string;
|
||||
}>(
|
||||
`SELECT worth_processing, reasons, judged_at
|
||||
FROM dream_verdicts
|
||||
WHERE file_path = $1 AND content_hash = $2`,
|
||||
[filePath, contentHash]
|
||||
);
|
||||
if (result.rows.length === 0) return null;
|
||||
const r = result.rows[0];
|
||||
return {
|
||||
worth_processing: r.worth_processing,
|
||||
reasons: r.reasons ?? [],
|
||||
judged_at: r.judged_at instanceof Date ? r.judged_at.toISOString() : String(r.judged_at),
|
||||
};
|
||||
}
|
||||
|
||||
async putDreamVerdict(filePath: string, contentHash: string, verdict: DreamVerdictInput): Promise<void> {
|
||||
await this.db.query(
|
||||
`INSERT INTO dream_verdicts (file_path, content_hash, worth_processing, reasons)
|
||||
VALUES ($1, $2, $3, $4::jsonb)
|
||||
ON CONFLICT (file_path, content_hash) DO UPDATE SET
|
||||
worth_processing = EXCLUDED.worth_processing,
|
||||
reasons = EXCLUDED.reasons,
|
||||
judged_at = now()`,
|
||||
[filePath, contentHash, verdict.worth_processing, JSON.stringify(verdict.reasons)]
|
||||
);
|
||||
}
|
||||
|
||||
// Versions
|
||||
async createVersion(slug: string): Promise<PageVersion> {
|
||||
const { rows } = await this.db.query(
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import postgres from 'postgres';
|
||||
import type { BrainEngine, LinkBatchInput, TimelineBatchInput, ReservedConnection, DreamVerdict, DreamVerdictInput } from './engine.ts';
|
||||
import type { BrainEngine, LinkBatchInput, TimelineBatchInput, ReservedConnection } from './engine.ts';
|
||||
import { MAX_SEARCH_LIMIT, clampSearchLimit } from './engine.ts';
|
||||
import { runMigrations } from './migrate.ts';
|
||||
import { SCHEMA_SQL } from './schema-embedded.ts';
|
||||
@@ -1303,39 +1303,6 @@ export class PostgresEngine implements BrainEngine {
|
||||
return rows as unknown as RawData[];
|
||||
}
|
||||
|
||||
// Dream-cycle significance verdict cache (v0.23).
|
||||
async getDreamVerdict(filePath: string, contentHash: string): Promise<DreamVerdict | null> {
|
||||
const sql = this.sql;
|
||||
const rows = await sql<Array<{
|
||||
worth_processing: boolean;
|
||||
reasons: string[] | null;
|
||||
judged_at: Date;
|
||||
}>>`
|
||||
SELECT worth_processing, reasons, judged_at
|
||||
FROM dream_verdicts
|
||||
WHERE file_path = ${filePath} AND content_hash = ${contentHash}
|
||||
`;
|
||||
if (rows.length === 0) return null;
|
||||
const r = rows[0];
|
||||
return {
|
||||
worth_processing: r.worth_processing,
|
||||
reasons: r.reasons ?? [],
|
||||
judged_at: r.judged_at instanceof Date ? r.judged_at.toISOString() : String(r.judged_at),
|
||||
};
|
||||
}
|
||||
|
||||
async putDreamVerdict(filePath: string, contentHash: string, verdict: DreamVerdictInput): Promise<void> {
|
||||
const sql = this.sql;
|
||||
await sql`
|
||||
INSERT INTO dream_verdicts (file_path, content_hash, worth_processing, reasons)
|
||||
VALUES (${filePath}, ${contentHash}, ${verdict.worth_processing}, ${sql.json(verdict.reasons as Parameters<typeof sql.json>[0])})
|
||||
ON CONFLICT (file_path, content_hash) DO UPDATE SET
|
||||
worth_processing = EXCLUDED.worth_processing,
|
||||
reasons = EXCLUDED.reasons,
|
||||
judged_at = now()
|
||||
`;
|
||||
}
|
||||
|
||||
// Versions
|
||||
async createVersion(slug: string): Promise<PageVersion> {
|
||||
const sql = this.sql;
|
||||
|
||||
@@ -596,22 +596,6 @@ CREATE TABLE IF NOT EXISTS subagent_rate_leases (
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_rate_leases_key_expires ON subagent_rate_leases (key, expires_at);
|
||||
|
||||
-- ============================================================
|
||||
-- Dream-cycle significance verdict cache — v0.23 synthesize phase
|
||||
-- ============================================================
|
||||
-- Caches the cheap Haiku "is this transcript worth processing?" verdict
|
||||
-- per (file_path, content_hash) so backfill re-runs skip already-judged
|
||||
-- files. Distinct from raw_data (which is page-scoped); transcripts
|
||||
-- aren't pages.
|
||||
CREATE TABLE IF NOT EXISTS dream_verdicts (
|
||||
file_path TEXT NOT NULL,
|
||||
content_hash TEXT NOT NULL,
|
||||
worth_processing BOOLEAN NOT NULL,
|
||||
reasons JSONB,
|
||||
judged_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
PRIMARY KEY (file_path, content_hash)
|
||||
);
|
||||
|
||||
-- ============================================================
|
||||
-- Cycle coordination lock — v0.17 runCycle primitive
|
||||
-- ============================================================
|
||||
@@ -679,7 +663,6 @@ BEGIN
|
||||
ALTER TABLE subagent_tool_executions ENABLE ROW LEVEL SECURITY;
|
||||
ALTER TABLE subagent_rate_leases ENABLE ROW LEVEL SECURITY;
|
||||
ALTER TABLE gbrain_cycle_locks ENABLE ROW LEVEL SECURITY;
|
||||
ALTER TABLE dream_verdicts ENABLE ROW LEVEL SECURITY;
|
||||
RAISE NOTICE 'RLS enabled on all tables (role % has BYPASSRLS)', current_user;
|
||||
ELSE
|
||||
RAISE WARNING 'Skipping RLS: role % does not have BYPASSRLS privilege. Run as postgres role to enable.', current_user;
|
||||
|
||||
+2
-2
@@ -301,7 +301,7 @@ export function resolveSlugForPath(filePath: string, repoPrefix?: string): strin
|
||||
|
||||
import { existsSync as _existsSync, readFileSync as _readFileSync, appendFileSync as _appendFileSync, mkdirSync as _mkdirSync } from 'fs';
|
||||
import { join as _joinPath } from 'path';
|
||||
import { gbrainPath as _gbrainPath } from './config.ts';
|
||||
import { homedir as _homedir } from 'os';
|
||||
import { createHash as _createHash } from 'crypto';
|
||||
|
||||
export interface SyncFailure {
|
||||
@@ -402,7 +402,7 @@ export function formatCodeBreakdown(
|
||||
}
|
||||
|
||||
function _failuresDir(): string {
|
||||
return _gbrainPath();
|
||||
return _joinPath(_homedir(), '.gbrain');
|
||||
}
|
||||
|
||||
export function syncFailuresPath(): string {
|
||||
|
||||
@@ -592,22 +592,6 @@ CREATE TABLE IF NOT EXISTS subagent_rate_leases (
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_rate_leases_key_expires ON subagent_rate_leases (key, expires_at);
|
||||
|
||||
-- ============================================================
|
||||
-- Dream-cycle significance verdict cache — v0.21 synthesize phase
|
||||
-- ============================================================
|
||||
-- Caches the cheap Haiku "is this transcript worth processing?" verdict
|
||||
-- per (file_path, content_hash) so backfill re-runs skip already-judged
|
||||
-- files. Distinct from raw_data (which is page-scoped); transcripts
|
||||
-- aren't pages.
|
||||
CREATE TABLE IF NOT EXISTS dream_verdicts (
|
||||
file_path TEXT NOT NULL,
|
||||
content_hash TEXT NOT NULL,
|
||||
worth_processing BOOLEAN NOT NULL,
|
||||
reasons JSONB,
|
||||
judged_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
PRIMARY KEY (file_path, content_hash)
|
||||
);
|
||||
|
||||
-- ============================================================
|
||||
-- Cycle coordination lock — v0.17 runCycle primitive
|
||||
-- ============================================================
|
||||
@@ -675,7 +659,6 @@ BEGIN
|
||||
ALTER TABLE subagent_tool_executions ENABLE ROW LEVEL SECURITY;
|
||||
ALTER TABLE subagent_rate_leases ENABLE ROW LEVEL SECURITY;
|
||||
ALTER TABLE gbrain_cycle_locks ENABLE ROW LEVEL SECURITY;
|
||||
ALTER TABLE dream_verdicts ENABLE ROW LEVEL SECURITY;
|
||||
RAISE NOTICE 'RLS enabled on all tables (role % has BYPASSRLS)', current_user;
|
||||
ELSE
|
||||
RAISE WARNING 'Skipping RLS: role % does not have BYPASSRLS privilege. Run as postgres role to enable.', current_user;
|
||||
|
||||
@@ -1,97 +0,0 @@
|
||||
/**
|
||||
* AgentRunner registry + selection tests. Proves the harness contract is
|
||||
* truly agent-agnostic via a fake-runner integration.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeEach } from 'bun:test';
|
||||
import {
|
||||
registerAgentRunner, resolveAgentRunner, listRegisteredAgents,
|
||||
_resetRegistryForTests,
|
||||
type AgentRunner, type DetectResult, type InvokeOpts, type InvokeResult, type TranscriptSink,
|
||||
} from '../src/core/claw-test/agent-runner.ts';
|
||||
|
||||
class FakeRunner implements AgentRunner {
|
||||
readonly name: string;
|
||||
invocations = 0;
|
||||
detected: DetectResult = { available: true, binPath: '/usr/bin/fake-agent' };
|
||||
|
||||
constructor(name: string) { this.name = name; }
|
||||
|
||||
async detect(): Promise<DetectResult> { return this.detected; }
|
||||
async invoke(_opts: InvokeOpts): Promise<InvokeResult> {
|
||||
this.invocations++;
|
||||
return { exitCode: 0, durationMs: 1 };
|
||||
}
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
_resetRegistryForTests();
|
||||
});
|
||||
|
||||
describe('registry', () => {
|
||||
test('register + resolve roundtrips', () => {
|
||||
registerAgentRunner('fake', () => new FakeRunner('fake'));
|
||||
const r = resolveAgentRunner('fake');
|
||||
expect(r.name).toBe('fake');
|
||||
});
|
||||
|
||||
test('resolve unknown agent throws with helpful list', () => {
|
||||
registerAgentRunner('alpha', () => new FakeRunner('alpha'));
|
||||
registerAgentRunner('beta', () => new FakeRunner('beta'));
|
||||
expect(() => resolveAgentRunner('gamma')).toThrow(/registered: alpha, beta/);
|
||||
});
|
||||
|
||||
test('listRegisteredAgents returns sorted names', () => {
|
||||
registerAgentRunner('zeta', () => new FakeRunner('zeta'));
|
||||
registerAgentRunner('alpha', () => new FakeRunner('alpha'));
|
||||
expect(listRegisteredAgents()).toEqual(['alpha', 'zeta']);
|
||||
});
|
||||
|
||||
test('factory pattern produces independent instances', () => {
|
||||
registerAgentRunner('fake', () => new FakeRunner('fake'));
|
||||
const a = resolveAgentRunner('fake') as FakeRunner;
|
||||
const b = resolveAgentRunner('fake') as FakeRunner;
|
||||
expect(a).not.toBe(b);
|
||||
});
|
||||
});
|
||||
|
||||
describe('agent-agnosticism guard', () => {
|
||||
test('a fake runner can satisfy the AgentRunner contract end-to-end', async () => {
|
||||
registerAgentRunner('fake', () => new FakeRunner('fake'));
|
||||
const runner = resolveAgentRunner('fake');
|
||||
|
||||
// The harness contract: detect → invoke. Nothing else.
|
||||
const detected = await runner.detect();
|
||||
expect(detected.available).toBe(true);
|
||||
expect(detected.binPath).toBe('/usr/bin/fake-agent');
|
||||
|
||||
let written = 0;
|
||||
const sink: TranscriptSink = {
|
||||
write: () => { written++; },
|
||||
nextOffset: () => 0,
|
||||
close: async () => { /* noop */ },
|
||||
};
|
||||
|
||||
const result = await runner.invoke({
|
||||
cwd: '/tmp',
|
||||
brief: 'hello',
|
||||
env: {},
|
||||
timeoutMs: 1000,
|
||||
transcriptSink: sink,
|
||||
});
|
||||
expect(result.exitCode).toBe(0);
|
||||
});
|
||||
|
||||
test('a runner reporting unavailable still satisfies the contract', async () => {
|
||||
class UnavailableRunner implements AgentRunner {
|
||||
name = 'gone';
|
||||
async detect() { return { available: false, reason: 'not installed' } as DetectResult; }
|
||||
async invoke(): Promise<InvokeResult> { throw new Error('should not be called'); }
|
||||
}
|
||||
registerAgentRunner('gone', () => new UnavailableRunner());
|
||||
const r = resolveAgentRunner('gone');
|
||||
const d = await r.detect();
|
||||
expect(d.available).toBe(false);
|
||||
expect(d.reason).toBe('not installed');
|
||||
});
|
||||
});
|
||||
@@ -1,165 +0,0 @@
|
||||
/**
|
||||
* gbrain claw-test CLI dispatch tests.
|
||||
*
|
||||
* These tests exercise the harness's argument parsing, scenario loading,
|
||||
* agent registry resolution, and friction-report path. They do NOT spawn
|
||||
* real gbrain commands (no built binary in CI yet); the canonical scripted
|
||||
* E2E that walks `gbrain init → import → query → extract → verify` lives
|
||||
* in test/e2e/claw-test.test.ts and gates on a built binary.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { mkdtempSync, rmSync, existsSync, readFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
import { runFriction } from '../src/commands/friction.ts';
|
||||
import { listScenarios, loadScenario } from '../src/core/claw-test/scenarios.ts';
|
||||
import {
|
||||
registerAgentRunner, resolveAgentRunner, listRegisteredAgents,
|
||||
_resetRegistryForTests,
|
||||
type AgentRunner, type DetectResult, type InvokeOpts, type InvokeResult,
|
||||
} from '../src/core/claw-test/agent-runner.ts';
|
||||
|
||||
let tmp: string;
|
||||
const ORIG_HOME = process.env.GBRAIN_HOME;
|
||||
|
||||
beforeEach(() => {
|
||||
tmp = mkdtempSync(join(tmpdir(), 'claw-test-cli-'));
|
||||
process.env.GBRAIN_HOME = tmp;
|
||||
_resetRegistryForTests();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
process.env.GBRAIN_HOME = ORIG_HOME;
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
describe('shipped scenarios are loadable', () => {
|
||||
test('default fixtures root contains both v1 scenarios', () => {
|
||||
delete process.env.GBRAIN_CLAW_SCENARIOS_DIR;
|
||||
const names = listScenarios();
|
||||
expect(names).toContain('fresh-install');
|
||||
expect(names).toContain('upgrade-from-v0.18');
|
||||
});
|
||||
|
||||
test('fresh-install has expected_phases', () => {
|
||||
delete process.env.GBRAIN_CLAW_SCENARIOS_DIR;
|
||||
const cfg = loadScenario('fresh-install');
|
||||
expect(cfg.expectedPhases).toContain('import.files');
|
||||
expect(cfg.expectedPhases).toContain('extract.links_fs');
|
||||
expect(cfg.expectedPhases).toContain('doctor.db_checks');
|
||||
});
|
||||
|
||||
test('upgrade-from-v0.18 declares from_version', () => {
|
||||
delete process.env.GBRAIN_CLAW_SCENARIOS_DIR;
|
||||
const cfg = loadScenario('upgrade-from-v0.18');
|
||||
expect(cfg.kind).toBe('upgrade');
|
||||
expect(cfg.fromVersion).toBe('0.18.0');
|
||||
expect(cfg.seedRelative).toBe('seed');
|
||||
});
|
||||
});
|
||||
|
||||
describe('agent registry — fake-runner integration', () => {
|
||||
test('a fake runner can be registered, resolved, and detect/invoke called', async () => {
|
||||
let invokeCount = 0;
|
||||
class FakeRunner implements AgentRunner {
|
||||
readonly name = 'fake';
|
||||
async detect(): Promise<DetectResult> { return { available: true, binPath: '/usr/bin/fake' }; }
|
||||
async invoke(_opts: InvokeOpts): Promise<InvokeResult> {
|
||||
invokeCount++;
|
||||
return { exitCode: 0, durationMs: 1 };
|
||||
}
|
||||
}
|
||||
registerAgentRunner('fake', () => new FakeRunner());
|
||||
expect(listRegisteredAgents()).toContain('fake');
|
||||
|
||||
const r = resolveAgentRunner('fake');
|
||||
const detected = await r.detect();
|
||||
expect(detected.available).toBe(true);
|
||||
|
||||
const result = await r.invoke({
|
||||
cwd: tmp,
|
||||
brief: 'test',
|
||||
env: {},
|
||||
timeoutMs: 1000,
|
||||
transcriptSink: { write: () => {}, nextOffset: () => 0, close: async () => {} },
|
||||
});
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(invokeCount).toBe(1);
|
||||
});
|
||||
|
||||
test('resolveAgentRunner with unknown name throws with registered list', () => {
|
||||
registerAgentRunner('alpha', () => ({} as AgentRunner));
|
||||
expect(() => resolveAgentRunner('unknown')).toThrow(/registered: alpha/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('friction CLI integrates with harness run-id env', () => {
|
||||
test('GBRAIN_FRICTION_RUN_ID populates harness-style run-ids', () => {
|
||||
process.env.GBRAIN_FRICTION_RUN_ID = 'claw-test-20260428-fake-abcd1234';
|
||||
try {
|
||||
const code = runFriction(['log', '--phase', 'install', '--message', 'simulated harness write']);
|
||||
expect(code).toBe(0);
|
||||
const expectedFile = join(tmp, '.gbrain', 'friction', 'claw-test-20260428-fake-abcd1234.jsonl');
|
||||
expect(existsSync(expectedFile)).toBe(true);
|
||||
const raw = readFileSync(expectedFile, 'utf-8');
|
||||
const entry = JSON.parse(raw.split('\n')[0]);
|
||||
expect(entry.run_id).toBe('claw-test-20260428-fake-abcd1234');
|
||||
expect(entry.message).toBe('simulated harness write');
|
||||
} finally {
|
||||
delete process.env.GBRAIN_FRICTION_RUN_ID;
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('OpenClawRunner detection (reliable on box without openclaw)', () => {
|
||||
test('detect returns unavailable when OPENCLAW_BIN missing', async () => {
|
||||
const orig = process.env.OPENCLAW_BIN;
|
||||
delete process.env.OPENCLAW_BIN;
|
||||
try {
|
||||
const { OpenClawRunner } = await import('../src/core/claw-test/runners/openclaw.ts');
|
||||
const r = new OpenClawRunner();
|
||||
const d = await r.detect();
|
||||
// Either unavailable, or available if openclaw IS on PATH for the dev — both states are valid.
|
||||
// We only assert the contract shape.
|
||||
expect(typeof d.available).toBe('boolean');
|
||||
if (!d.available) {
|
||||
expect(typeof d.reason).toBe('string');
|
||||
} else {
|
||||
expect(d.binPath?.startsWith('/')).toBe(true);
|
||||
}
|
||||
} finally {
|
||||
if (orig !== undefined) process.env.OPENCLAW_BIN = orig;
|
||||
}
|
||||
});
|
||||
|
||||
test('detect rejects relative OPENCLAW_BIN', async () => {
|
||||
const orig = process.env.OPENCLAW_BIN;
|
||||
process.env.OPENCLAW_BIN = 'relative/openclaw';
|
||||
try {
|
||||
const { OpenClawRunner } = await import('../src/core/claw-test/runners/openclaw.ts');
|
||||
const r = new OpenClawRunner();
|
||||
const d = await r.detect();
|
||||
expect(d.available).toBe(false);
|
||||
expect(d.reason).toMatch(/absolute/);
|
||||
} finally {
|
||||
if (orig !== undefined) process.env.OPENCLAW_BIN = orig;
|
||||
else delete process.env.OPENCLAW_BIN;
|
||||
}
|
||||
});
|
||||
|
||||
test("detect rejects '..' segments in OPENCLAW_BIN", async () => {
|
||||
const orig = process.env.OPENCLAW_BIN;
|
||||
process.env.OPENCLAW_BIN = '/tmp/foo/../bar';
|
||||
try {
|
||||
const { OpenClawRunner } = await import('../src/core/claw-test/runners/openclaw.ts');
|
||||
const r = new OpenClawRunner();
|
||||
const d = await r.detect();
|
||||
expect(d.available).toBe(false);
|
||||
expect(d.reason).toMatch(/'\.\.' segments/);
|
||||
} finally {
|
||||
if (orig !== undefined) process.env.OPENCLAW_BIN = orig;
|
||||
else delete process.env.OPENCLAW_BIN;
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -377,8 +377,8 @@ describe('runCycle — yieldBetweenPhases hook', () => {
|
||||
hookCalls++;
|
||||
},
|
||||
});
|
||||
// v0.23: 8 phases → 8 yield calls (one after each).
|
||||
expect(hookCalls).toBe(8);
|
||||
// 6 phases → 6 yield calls (one after each).
|
||||
expect(hookCalls).toBe(6);
|
||||
});
|
||||
|
||||
test('hook exceptions do not abort the cycle', async () => {
|
||||
@@ -388,8 +388,8 @@ describe('runCycle — yieldBetweenPhases hook', () => {
|
||||
throw new Error('synthetic hook error');
|
||||
},
|
||||
});
|
||||
// Cycle still completed all phases (v0.23: 8).
|
||||
expect(report.phases.length).toBe(8);
|
||||
// Cycle still completed all phases.
|
||||
expect(report.phases.length).toBe(6);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -1,83 +0,0 @@
|
||||
/**
|
||||
* Unit tests for the patterns phase (v0.21).
|
||||
*
|
||||
* The phase invokes a subagent and queues real Minions work, so this
|
||||
* file leans on structural assertions over the source + a single
|
||||
* end-to-end driver run that exercises the skip-paths.
|
||||
*
|
||||
* Full LLM behavior is exercised by E2E tests in test/e2e/.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { readFileSync } from 'fs';
|
||||
|
||||
const patternsSrc = readFileSync(
|
||||
new URL('../src/core/cycle/patterns.ts', import.meta.url),
|
||||
'utf-8',
|
||||
);
|
||||
|
||||
describe('patterns phase wiring', () => {
|
||||
test('imports queue + waitForCompletion + types', () => {
|
||||
expect(patternsSrc).toContain("import { MinionQueue }");
|
||||
expect(patternsSrc).toContain('waitForCompletion');
|
||||
expect(patternsSrc).toContain('SubagentHandlerData');
|
||||
});
|
||||
|
||||
test('threads allowed_slug_prefixes from filing-rules JSON', () => {
|
||||
expect(patternsSrc).toContain('allowed_slug_prefixes');
|
||||
expect(patternsSrc).toContain('_brain-filing-rules.json');
|
||||
expect(patternsSrc).toContain('dream_synthesize_paths');
|
||||
});
|
||||
|
||||
test('reads min_evidence + lookback_days config', () => {
|
||||
expect(patternsSrc).toContain('dream.patterns.min_evidence');
|
||||
expect(patternsSrc).toContain('dream.patterns.lookback_days');
|
||||
});
|
||||
|
||||
test('uses subagent_tool_executions for slug provenance (Codex #2 fix)', () => {
|
||||
expect(patternsSrc).toContain('subagent_tool_executions');
|
||||
expect(patternsSrc).toContain("tool_name = 'brain_put_page'");
|
||||
});
|
||||
|
||||
test('skips when ANTHROPIC_API_KEY missing', () => {
|
||||
expect(patternsSrc).toContain('ANTHROPIC_API_KEY');
|
||||
expect(patternsSrc).toContain('no_api_key');
|
||||
});
|
||||
|
||||
test('skips when reflections below min_evidence', () => {
|
||||
expect(patternsSrc).toContain('insufficient_evidence');
|
||||
});
|
||||
|
||||
test('reverse-writes pages to disk via serializeMarkdown', () => {
|
||||
expect(patternsSrc).toContain('serializeMarkdown');
|
||||
expect(patternsSrc).toContain('writeFileSync');
|
||||
});
|
||||
|
||||
test('runs after extract — queries fresh graph', () => {
|
||||
// Documented invariant: pattern phase MUST run after extract.
|
||||
// The cycle.ts dispatcher enforces order; this just confirms the
|
||||
// patterns module doesn't try to compute its own auto-link layer
|
||||
// (which would be a subtle regression).
|
||||
expect(patternsSrc).not.toContain('runAutoLink');
|
||||
expect(patternsSrc).not.toContain('extractPageLinks(');
|
||||
});
|
||||
|
||||
test('does NOT use raw_data table (Codex #3 fix)', () => {
|
||||
expect(patternsSrc).not.toContain('putRawData');
|
||||
expect(patternsSrc).not.toContain('getRawData');
|
||||
});
|
||||
});
|
||||
|
||||
describe('patterns scope filter', () => {
|
||||
test('filters reflections by slug LIKE wiki/personal/reflections/%', () => {
|
||||
expect(patternsSrc).toContain("slug LIKE 'wiki/personal/reflections/%'");
|
||||
});
|
||||
|
||||
test('orders by updated_at DESC for recency-bias', () => {
|
||||
expect(patternsSrc).toContain('ORDER BY updated_at DESC');
|
||||
});
|
||||
|
||||
test('caps gather to 100 reflections (cost control)', () => {
|
||||
expect(patternsSrc).toContain('LIMIT 100');
|
||||
});
|
||||
});
|
||||
@@ -1,191 +0,0 @@
|
||||
/**
|
||||
* Unit tests for the synthesize phase scaffolding.
|
||||
*
|
||||
* Covers transcript-discovery branches (date filters, exclude regex,
|
||||
* minChars, multiple sources) and the compileExcludePatterns word-
|
||||
* boundary heuristic. Doesn't drive a real Anthropic call — full
|
||||
* cycle E2E lives in test/e2e/.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeEach } from 'bun:test';
|
||||
import { mkdtempSync, writeFileSync, rmSync, mkdirSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import {
|
||||
discoverTranscripts,
|
||||
readSingleTranscript,
|
||||
compileExcludePatterns,
|
||||
} from '../src/core/cycle/transcript-discovery.ts';
|
||||
|
||||
let tmpDir: string;
|
||||
|
||||
function makeTranscript(name: string, body: string): string {
|
||||
const path = join(tmpDir, name);
|
||||
writeFileSync(path, body, 'utf8');
|
||||
return path;
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
tmpDir = mkdtempSync(join(tmpdir(), 'gbrain-synth-test-'));
|
||||
});
|
||||
|
||||
describe('compileExcludePatterns', () => {
|
||||
test('auto-wraps bare words in word-boundary regex (Q-3)', () => {
|
||||
const res = compileExcludePatterns(['medical']);
|
||||
expect(res).toHaveLength(1);
|
||||
// word boundary: matches "medical" but NOT "comedical"
|
||||
expect(res[0].test('medical advice')).toBe(true);
|
||||
expect(res[0].test('comedical')).toBe(false);
|
||||
});
|
||||
|
||||
test('honors raw regex when input is non-bare-word', () => {
|
||||
const res = compileExcludePatterns(['^therapy:']);
|
||||
expect(res[0].test('therapy: today was hard')).toBe(true);
|
||||
expect(res[0].test('thinking about therapy:')).toBe(false);
|
||||
});
|
||||
|
||||
test('skips invalid regex with warning, does not crash', () => {
|
||||
const res = compileExcludePatterns(['valid', '(broken[']);
|
||||
expect(res).toHaveLength(1); // only the valid one compiled
|
||||
});
|
||||
|
||||
test('case-insensitive matching by default', () => {
|
||||
const res = compileExcludePatterns(['Medical']);
|
||||
expect(res[0].test('medical advice')).toBe(true);
|
||||
expect(res[0].test('MEDICAL ADVICE')).toBe(true);
|
||||
});
|
||||
|
||||
test('empty / undefined input returns empty array', () => {
|
||||
expect(compileExcludePatterns(undefined)).toEqual([]);
|
||||
expect(compileExcludePatterns([])).toEqual([]);
|
||||
expect(compileExcludePatterns([''])).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('discoverTranscripts', () => {
|
||||
test('returns empty when corpusDir does not exist', () => {
|
||||
const out = discoverTranscripts({ corpusDir: '/nonexistent/path' });
|
||||
expect(out).toEqual([]);
|
||||
});
|
||||
|
||||
test('returns transcripts above minChars, sorted by filePath', () => {
|
||||
makeTranscript('2026-04-25-session.txt', 'a'.repeat(2500));
|
||||
makeTranscript('2026-04-24-other.txt', 'b'.repeat(2500));
|
||||
const out = discoverTranscripts({ corpusDir: tmpDir, minChars: 1000 });
|
||||
expect(out).toHaveLength(2);
|
||||
expect(out[0].basename).toBe('2026-04-24-other');
|
||||
expect(out[1].basename).toBe('2026-04-25-session');
|
||||
});
|
||||
|
||||
test('skips transcripts below minChars', () => {
|
||||
makeTranscript('2026-04-25-short.txt', 'tiny');
|
||||
const out = discoverTranscripts({ corpusDir: tmpDir, minChars: 2000 });
|
||||
expect(out).toEqual([]);
|
||||
});
|
||||
|
||||
test('skips non-txt files', () => {
|
||||
makeTranscript('2026-04-25-foo.md', 'a'.repeat(3000));
|
||||
const out = discoverTranscripts({ corpusDir: tmpDir, minChars: 1000 });
|
||||
expect(out).toEqual([]);
|
||||
});
|
||||
|
||||
test('exclude_patterns filters out matched transcripts (word boundary)', () => {
|
||||
makeTranscript('2026-04-25-medical.txt', 'discussing medical advice ' + 'x'.repeat(3000));
|
||||
makeTranscript('2026-04-25-comedy.txt', 'comedical writing tips ' + 'x'.repeat(3000));
|
||||
const out = discoverTranscripts({
|
||||
corpusDir: tmpDir,
|
||||
minChars: 1000,
|
||||
excludePatterns: ['medical'],
|
||||
});
|
||||
expect(out).toHaveLength(1);
|
||||
expect(out[0].basename).toBe('2026-04-25-comedy');
|
||||
});
|
||||
|
||||
test('--date filter restricts to one specific YYYY-MM-DD basename', () => {
|
||||
makeTranscript('2026-04-25-foo.txt', 'a'.repeat(3000));
|
||||
makeTranscript('2026-04-26-bar.txt', 'b'.repeat(3000));
|
||||
const out = discoverTranscripts({
|
||||
corpusDir: tmpDir,
|
||||
minChars: 1000,
|
||||
date: '2026-04-25',
|
||||
});
|
||||
expect(out).toHaveLength(1);
|
||||
expect(out[0].basename).toBe('2026-04-25-foo');
|
||||
});
|
||||
|
||||
test('--from / --to range filters basename dates', () => {
|
||||
makeTranscript('2026-04-23-a.txt', 'a'.repeat(3000));
|
||||
makeTranscript('2026-04-25-b.txt', 'b'.repeat(3000));
|
||||
makeTranscript('2026-04-27-c.txt', 'c'.repeat(3000));
|
||||
const out = discoverTranscripts({
|
||||
corpusDir: tmpDir,
|
||||
minChars: 1000,
|
||||
from: '2026-04-24',
|
||||
to: '2026-04-26',
|
||||
});
|
||||
expect(out).toHaveLength(1);
|
||||
expect(out[0].basename).toBe('2026-04-25-b');
|
||||
});
|
||||
|
||||
test('multiple sources (corpus + meeting transcripts) merged', () => {
|
||||
makeTranscript('2026-04-25-session.txt', 'a'.repeat(3000));
|
||||
const meetDir = mkdtempSync(join(tmpdir(), 'gbrain-meet-'));
|
||||
writeFileSync(join(meetDir, '2026-04-25-meeting.txt'), 'b'.repeat(3000));
|
||||
const out = discoverTranscripts({
|
||||
corpusDir: tmpDir,
|
||||
meetingTranscriptsDir: meetDir,
|
||||
minChars: 1000,
|
||||
});
|
||||
expect(out).toHaveLength(2);
|
||||
rmSync(meetDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('content_hash is stable for identical content, different for edits (A-3)', () => {
|
||||
makeTranscript('2026-04-25-a.txt', 'identical content ' + 'x'.repeat(3000));
|
||||
makeTranscript('2026-04-25-b.txt', 'identical content ' + 'x'.repeat(3000));
|
||||
const out1 = discoverTranscripts({ corpusDir: tmpDir, minChars: 1000 });
|
||||
expect(out1[0].contentHash).toBe(out1[1].contentHash);
|
||||
|
||||
// Edit one — hash changes
|
||||
makeTranscript('2026-04-25-a.txt', 'edited content ' + 'x'.repeat(3000));
|
||||
const out2 = discoverTranscripts({ corpusDir: tmpDir, minChars: 1000 });
|
||||
expect(out2[0].contentHash).not.toBe(out2[1].contentHash);
|
||||
});
|
||||
});
|
||||
|
||||
describe('readSingleTranscript', () => {
|
||||
test('returns transcript above minChars', () => {
|
||||
const path = makeTranscript('hello.txt', 'a'.repeat(3000));
|
||||
const t = readSingleTranscript(path, { minChars: 1000 });
|
||||
expect(t).not.toBeNull();
|
||||
expect(t!.basename).toBe('hello');
|
||||
});
|
||||
|
||||
test('returns null when below minChars', () => {
|
||||
const path = makeTranscript('hello.txt', 'tiny');
|
||||
const t = readSingleTranscript(path, { minChars: 2000 });
|
||||
expect(t).toBeNull();
|
||||
});
|
||||
|
||||
test('returns null when content matches exclude pattern', () => {
|
||||
const path = makeTranscript('hello.txt', 'medical content ' + 'x'.repeat(3000));
|
||||
const t = readSingleTranscript(path, { minChars: 1000, excludePatterns: ['medical'] });
|
||||
expect(t).toBeNull();
|
||||
});
|
||||
|
||||
test('throws on missing file', () => {
|
||||
expect(() => readSingleTranscript('/nonexistent/foo.txt')).toThrow();
|
||||
});
|
||||
|
||||
test('infers date from YYYY-MM-DD basename', () => {
|
||||
const path = makeTranscript('2026-04-25-thing.txt', 'a'.repeat(3000));
|
||||
const t = readSingleTranscript(path, { minChars: 1000 });
|
||||
expect(t!.inferredDate).toBe('2026-04-25');
|
||||
});
|
||||
|
||||
test('inferredDate null when basename does not start with YYYY-MM-DD', () => {
|
||||
const path = makeTranscript('random-basename.txt', 'a'.repeat(3000));
|
||||
const t = readSingleTranscript(path, { minChars: 1000 });
|
||||
expect(t!.inferredDate).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -1,61 +0,0 @@
|
||||
/**
|
||||
* Structural tests for `gbrain dream` argv parsing (v0.21).
|
||||
*
|
||||
* Verifies the help text + parser source contains the new flags
|
||||
* (--input, --date, --from, --to) and that conflict detection is wired.
|
||||
* The actual parseArgs is internal; we exercise it via the source file
|
||||
* structure to avoid spinning up a process per test.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { readFileSync } from 'fs';
|
||||
|
||||
const dreamSrc = readFileSync(new URL('../src/commands/dream.ts', import.meta.url), 'utf-8');
|
||||
|
||||
describe('dream CLI flag wiring', () => {
|
||||
test('declares --input flag with file argument', () => {
|
||||
expect(dreamSrc).toContain("'--input'");
|
||||
expect(dreamSrc).toContain('inputFile');
|
||||
});
|
||||
|
||||
test('declares --date / --from / --to flags', () => {
|
||||
expect(dreamSrc).toContain("'--date'");
|
||||
expect(dreamSrc).toContain("'--from'");
|
||||
expect(dreamSrc).toContain("'--to'");
|
||||
});
|
||||
|
||||
test('validates ISO date format', () => {
|
||||
expect(dreamSrc).toMatch(/ISO_DATE_RE/);
|
||||
expect(dreamSrc).toContain('YYYY-MM-DD');
|
||||
});
|
||||
|
||||
test('--input + --date conflict detection', () => {
|
||||
expect(dreamSrc).toContain('--input cannot be combined with --date');
|
||||
});
|
||||
|
||||
test('--input implies --phase synthesize', () => {
|
||||
expect(dreamSrc).toContain("phase = 'synthesize'");
|
||||
});
|
||||
|
||||
test('--from > --to range validation', () => {
|
||||
expect(dreamSrc).toContain('empty range');
|
||||
});
|
||||
|
||||
test('forwards synth fields to runCycle', () => {
|
||||
expect(dreamSrc).toContain('synthInputFile');
|
||||
expect(dreamSrc).toContain('synthDate');
|
||||
expect(dreamSrc).toContain('synthFrom');
|
||||
expect(dreamSrc).toContain('synthTo');
|
||||
});
|
||||
|
||||
test('totals line includes synth + patterns counters', () => {
|
||||
expect(dreamSrc).toContain('synth_transcripts');
|
||||
expect(dreamSrc).toContain('synth_pages');
|
||||
expect(dreamSrc).toContain('patterns=');
|
||||
});
|
||||
|
||||
test('help text documents dry-run synthesis semantics (Codex finding #8)', () => {
|
||||
expect(dreamSrc).toContain('skips the Sonnet');
|
||||
expect(dreamSrc.toLowerCase()).toContain('zero llm calls');
|
||||
});
|
||||
});
|
||||
@@ -1,134 +0,0 @@
|
||||
/**
|
||||
* gbrain claw-test scripted-mode E2E.
|
||||
*
|
||||
* Invokes the harness via `bun run src/cli.ts` (NOT a compiled binary —
|
||||
* `bun build --compile` doesn't bundle PGLite's runtime assets like
|
||||
* pglite.data, so a compiled gbrain can't init a fresh PGLite brain).
|
||||
* Uses a tiny shim script that the harness can spawn as if it were the
|
||||
* gbrain binary.
|
||||
*
|
||||
* Asserts:
|
||||
* - exit code 0 on a clean tree
|
||||
* - the friction JSONL has zero error/blocker entries
|
||||
* - the harness recorded progress events for the expected phases
|
||||
*
|
||||
* Tagged-skip env: CLAW_TEST_SKIP_E2E=1 to opt out (e.g. when PGLite
|
||||
* WASM is broken on the host — the macOS 26.3 #223 bug class).
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll } from 'bun:test';
|
||||
import { execFileSync, spawnSync } from 'child_process';
|
||||
import { mkdirSync, existsSync, mkdtempSync, rmSync, readFileSync, readdirSync, writeFileSync, chmodSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join, resolve } from 'path';
|
||||
|
||||
const REPO_ROOT = resolve(import.meta.dir, '..', '..');
|
||||
const BIN_CACHE = join(REPO_ROOT, 'test', '.cache');
|
||||
const BIN_PATH = join(BIN_CACHE, 'gbrain.sh');
|
||||
const SCENARIOS_DIR = join(REPO_ROOT, 'test', 'fixtures', 'claw-test-scenarios');
|
||||
|
||||
beforeAll(() => {
|
||||
if (!existsSync(BIN_CACHE)) mkdirSync(BIN_CACHE, { recursive: true });
|
||||
// Shim that delegates to `bun run src/cli.ts` so PGLite assets resolve from
|
||||
// the source tree (bun --compile doesn't bundle them). Marked executable so
|
||||
// child_process.spawn can run it directly.
|
||||
const shim = `#!/bin/sh\nexec bun run "${join(REPO_ROOT, 'src', 'cli.ts')}" "$@"\n`;
|
||||
writeFileSync(BIN_PATH, shim, 'utf-8');
|
||||
chmodSync(BIN_PATH, 0o755);
|
||||
}, 30_000);
|
||||
|
||||
describe('gbrain claw-test --scenario fresh-install (scripted)', () => {
|
||||
test('runs end-to-end clean and produces zero error/blocker friction', () => {
|
||||
const tmp = mkdtempSync(join(tmpdir(), 'claw-test-e2e-fresh-'));
|
||||
try {
|
||||
const result = spawnSync(BIN_PATH, ['claw-test', '--scenario', 'fresh-install', '--keep-tempdir'], {
|
||||
cwd: REPO_ROOT,
|
||||
env: {
|
||||
...process.env,
|
||||
GBRAIN_HOME: tmp,
|
||||
GBRAIN_BIN_OVERRIDE: BIN_PATH,
|
||||
GBRAIN_CLAW_SCENARIOS_DIR: join(REPO_ROOT, 'test', 'fixtures', 'claw-test-scenarios'),
|
||||
},
|
||||
encoding: 'utf-8',
|
||||
timeout: 120_000,
|
||||
});
|
||||
if (result.status !== 0) {
|
||||
console.error('STDOUT:', result.stdout);
|
||||
console.error('STDERR:', result.stderr);
|
||||
}
|
||||
expect(result.status).toBe(0);
|
||||
|
||||
// Inspect the friction JSONL the harness wrote.
|
||||
const frictionDir = join(tmp, '.gbrain', 'friction');
|
||||
expect(existsSync(frictionDir)).toBe(true);
|
||||
const files = readdirSync(frictionDir).filter(f => f.endsWith('.jsonl'));
|
||||
expect(files.length).toBeGreaterThan(0);
|
||||
const runFile = join(frictionDir, files[0]);
|
||||
const lines = readFileSync(runFile, 'utf-8').split('\n').filter(l => l.trim());
|
||||
const entries = lines.map(l => JSON.parse(l));
|
||||
const blockers = entries.filter(e => e.kind === 'friction' && (e.severity === 'error' || e.severity === 'blocker'));
|
||||
if (blockers.length > 0) {
|
||||
console.error('unexpected friction entries:', blockers);
|
||||
}
|
||||
expect(blockers.length).toBe(0);
|
||||
} finally {
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
}
|
||||
}, 180_000);
|
||||
|
||||
test('break path: an invented command produces an error friction entry and exits non-zero', () => {
|
||||
// We do this by setting GBRAIN_BIN_OVERRIDE to a script that pretends to be gbrain
|
||||
// and rejects the `import` subcommand specifically.
|
||||
const tmp = mkdtempSync(join(tmpdir(), 'claw-test-e2e-break-'));
|
||||
const fakeBin = join(tmp, 'fake-gbrain');
|
||||
try {
|
||||
// Write a shim that delegates to real gbrain but rejects 'import' to simulate breakage.
|
||||
const shimContent = `#!/bin/sh\nif [ "$1" = "import" ]; then echo "fake import error" >&2; exit 17; fi\nexec "${BIN_PATH}" "$@"\n`;
|
||||
const { writeFileSync, chmodSync } = require('fs');
|
||||
writeFileSync(fakeBin, shimContent, 'utf-8');
|
||||
chmodSync(fakeBin, 0o755);
|
||||
|
||||
const result = spawnSync(BIN_PATH, ['claw-test', '--scenario', 'fresh-install', '--keep-tempdir'], {
|
||||
cwd: REPO_ROOT,
|
||||
env: {
|
||||
...process.env,
|
||||
GBRAIN_HOME: tmp,
|
||||
GBRAIN_BIN_OVERRIDE: fakeBin,
|
||||
GBRAIN_CLAW_SCENARIOS_DIR: join(REPO_ROOT, 'test', 'fixtures', 'claw-test-scenarios'),
|
||||
},
|
||||
encoding: 'utf-8',
|
||||
timeout: 60_000,
|
||||
});
|
||||
expect(result.status).not.toBe(0);
|
||||
|
||||
// The friction log should have an error-severity entry for the 'import' phase.
|
||||
const frictionDir = join(tmp, '.gbrain', 'friction');
|
||||
const files = readdirSync(frictionDir).filter(f => f.endsWith('.jsonl'));
|
||||
const lines = readFileSync(join(frictionDir, files[0]), 'utf-8').split('\n').filter(l => l.trim());
|
||||
const entries = lines.map(l => JSON.parse(l));
|
||||
const importErrors = entries.filter(e => e.phase === 'import' && e.severity === 'error');
|
||||
expect(importErrors.length).toBeGreaterThan(0);
|
||||
} finally {
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
}
|
||||
}, 90_000);
|
||||
});
|
||||
|
||||
describe('gbrain friction render integration', () => {
|
||||
test('render produces a markdown report with the redact placeholder', () => {
|
||||
const tmp = mkdtempSync(join(tmpdir(), 'claw-test-e2e-render-'));
|
||||
try {
|
||||
// Log a friction entry with $HOME embedded, then render --redact md
|
||||
const home = process.env.HOME ?? '/tmp';
|
||||
const env = { ...process.env, GBRAIN_HOME: tmp, GBRAIN_FRICTION_RUN_ID: 'render-e2e' };
|
||||
execFileSync(BIN_PATH, ['friction', 'log', '--phase', 'p', '--message', `error at ${home}/.gbrain/x`], { env, encoding: 'utf-8' });
|
||||
const out = execFileSync(BIN_PATH, ['friction', 'render', '--run-id', 'render-e2e'], { env, encoding: 'utf-8' });
|
||||
expect(out).toContain('# Friction report');
|
||||
expect(out).toContain('<HOME>');
|
||||
// --redact is the default for md, so home itself should not appear.
|
||||
expect(out).not.toContain(home + '/.gbrain');
|
||||
} finally {
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
}
|
||||
}, 30_000);
|
||||
});
|
||||
@@ -97,8 +97,8 @@ describeE2E('E2E: runCycle against real Postgres', () => {
|
||||
});
|
||||
|
||||
expect(report.schema_version).toBe('1');
|
||||
// Cycle ran all 8 phases (or skipped the ones that don't support dry-run).
|
||||
expect(report.phases.length).toBe(8);
|
||||
// Cycle ran all 6 phases (or skipped the ones that don't support dry-run).
|
||||
expect(report.phases.length).toBe(6);
|
||||
|
||||
// Nothing got written.
|
||||
const afterPages = await conn.unsafe(`SELECT count(*)::int AS n FROM pages`);
|
||||
|
||||
@@ -1,159 +0,0 @@
|
||||
/**
|
||||
* E2E security regression: poisoned-transcript guard for the v0.21
|
||||
* trusted-workspace allow-list.
|
||||
*
|
||||
* Runs against PGLite in-memory (no DATABASE_URL required). Builds the
|
||||
* brain tool registry with `allowed_slug_prefixes` set the same way the
|
||||
* synthesize phase does, then calls the put_page tool with slugs that
|
||||
* are inside / outside the allow-list. Asserts:
|
||||
*
|
||||
* - In-allow-list slug → page is written to the DB
|
||||
* - Outside-allow-list slug → tool throws permission_denied
|
||||
* - When allow-list is unset (legacy), put_page is bounded to
|
||||
* wiki/agents/<id>/... (regression guard for the v0.15 anti-prompt-
|
||||
* injection guarantee)
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { PGLiteEngine } from '../../src/core/pglite-engine.ts';
|
||||
import { buildBrainTools } from '../../src/core/minions/tools/brain-allowlist.ts';
|
||||
import type { GBrainConfig } from '../../src/core/config.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({ engine: 'pglite' } as never);
|
||||
await engine.initSchema();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
if (engine) await engine.disconnect();
|
||||
});
|
||||
|
||||
const config = {} as unknown as GBrainConfig;
|
||||
|
||||
const PUT_PAGE_TOOL = 'brain_put_page';
|
||||
const SAMPLE_BODY = '---\ntitle: A reflection\ntype: default\n---\n\nbody text\n';
|
||||
|
||||
function findPutPageTool(tools: Awaited<ReturnType<typeof buildBrainTools>>) {
|
||||
const t = tools.find(x => x.name === PUT_PAGE_TOOL);
|
||||
if (!t) throw new Error('brain_put_page tool not found in registry');
|
||||
return t;
|
||||
}
|
||||
|
||||
describe('E2E allow-list — trusted-workspace path', () => {
|
||||
test('ALLOW: subagent put_page within allow-list writes the page', async () => {
|
||||
const tools = buildBrainTools({
|
||||
subagentId: 999,
|
||||
engine,
|
||||
config,
|
||||
allowedSlugPrefixes: ['wiki/personal/reflections/*'],
|
||||
});
|
||||
const tool = findPutPageTool(tools);
|
||||
await tool.execute(
|
||||
{ slug: 'wiki/personal/reflections/2026-04-25-arete-paradox-a3f8c1', content: SAMPLE_BODY },
|
||||
{ engine, jobId: 7777, remote: true },
|
||||
);
|
||||
const page = await engine.getPage('wiki/personal/reflections/2026-04-25-arete-paradox-a3f8c1');
|
||||
expect(page).not.toBeNull();
|
||||
expect(page!.title).toBe('A reflection');
|
||||
});
|
||||
|
||||
test('REJECT: subagent put_page outside allow-list throws permission_denied', async () => {
|
||||
const tools = buildBrainTools({
|
||||
subagentId: 999,
|
||||
engine,
|
||||
config,
|
||||
allowedSlugPrefixes: ['wiki/personal/reflections/*'],
|
||||
});
|
||||
const tool = findPutPageTool(tools);
|
||||
let threw = false;
|
||||
try {
|
||||
await tool.execute(
|
||||
{ slug: 'wiki/finance/secret-market-data', content: SAMPLE_BODY },
|
||||
{ engine, jobId: 7778, remote: true },
|
||||
);
|
||||
} catch (e) {
|
||||
threw = true;
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
expect(msg).toMatch(/allow-list/i);
|
||||
}
|
||||
expect(threw).toBe(true);
|
||||
const page = await engine.getPage('wiki/finance/secret-market-data');
|
||||
expect(page).toBeNull(); // never reached the engine
|
||||
});
|
||||
|
||||
test('Multiple prefixes: each slug evaluated independently', async () => {
|
||||
const tools = buildBrainTools({
|
||||
subagentId: 999,
|
||||
engine,
|
||||
config,
|
||||
allowedSlugPrefixes: ['wiki/personal/reflections/*', 'wiki/originals/*'],
|
||||
});
|
||||
const tool = findPutPageTool(tools);
|
||||
await tool.execute(
|
||||
{ slug: 'wiki/originals/ideas/2026-04-25-thousand-pound-armor', content: SAMPLE_BODY },
|
||||
{ engine, jobId: 7779, remote: true },
|
||||
);
|
||||
expect(await engine.getPage('wiki/originals/ideas/2026-04-25-thousand-pound-armor')).not.toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('E2E allow-list — legacy namespace fallback', () => {
|
||||
test('REGRESSION GUARD: when allow-list is unset, put_page rejects writes outside wiki/agents/<id>/', async () => {
|
||||
const tools = buildBrainTools({
|
||||
subagentId: 999,
|
||||
engine,
|
||||
config,
|
||||
// allowedSlugPrefixes intentionally omitted — exercises the v0.15
|
||||
// legacy namespace check that v0.21 must NOT regress.
|
||||
});
|
||||
const tool = findPutPageTool(tools);
|
||||
let threw = false;
|
||||
try {
|
||||
await tool.execute(
|
||||
{ slug: 'wiki/personal/reflections/2026-04-25-bypass-attempt', content: SAMPLE_BODY },
|
||||
{ engine, jobId: 7780, remote: true },
|
||||
);
|
||||
} catch (e) {
|
||||
threw = true;
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
expect(msg).toMatch(/wiki\/agents\/999/);
|
||||
}
|
||||
expect(threw).toBe(true);
|
||||
});
|
||||
|
||||
test('When allow-list unset, slug under wiki/agents/<id>/ is allowed', async () => {
|
||||
const tools = buildBrainTools({
|
||||
subagentId: 999,
|
||||
engine,
|
||||
config,
|
||||
});
|
||||
const tool = findPutPageTool(tools);
|
||||
await tool.execute(
|
||||
{ slug: 'wiki/agents/999/scratch-note', content: SAMPLE_BODY },
|
||||
{ engine, jobId: 7781, remote: true },
|
||||
);
|
||||
expect(await engine.getPage('wiki/agents/999/scratch-note')).not.toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('E2E allow-list — provenance via tool execution rows (Codex #2)', () => {
|
||||
test('subagent_tool_executions captures slug for each put_page call', async () => {
|
||||
// The synthesize phase relies on this being queryable to determine
|
||||
// exactly which slugs each child wrote (instead of pages.updated_at).
|
||||
// We don't have a real subagent run here, but we can verify the table
|
||||
// exists and the column shape supports the orchestrator's query.
|
||||
const rows = await engine.executeRaw(
|
||||
`SELECT column_name FROM information_schema.columns
|
||||
WHERE table_name = 'subagent_tool_executions'
|
||||
ORDER BY column_name`,
|
||||
) as Array<{ column_name: string }>;
|
||||
const cols = rows.map(r => r.column_name);
|
||||
expect(cols).toContain('input');
|
||||
expect(cols).toContain('tool_name');
|
||||
expect(cols).toContain('status');
|
||||
expect(cols).toContain('job_id');
|
||||
});
|
||||
});
|
||||
@@ -1,196 +0,0 @@
|
||||
/**
|
||||
* E2E full 8-phase cycle on PGLite, no API key required.
|
||||
*
|
||||
* Verifies that the v0.23 phase order — lint → backlinks → sync →
|
||||
* synthesize → extract → patterns → embed → orphans — is honored
|
||||
* end-to-end through runCycle when no API key is present (synthesize
|
||||
* + patterns skip cleanly, the other six phases run unchanged).
|
||||
*
|
||||
* Two regression-relevant invariants:
|
||||
* 1. CycleReport.phases preserves the 8-phase order — no future
|
||||
* reorder regresses without breaking this test.
|
||||
* 2. CycleReport.totals carries the new v0.23 fields:
|
||||
* transcripts_processed, synth_pages_written, patterns_written.
|
||||
*
|
||||
* No DATABASE_URL required. Mocks embedBatch so the embed phase doesn't
|
||||
* attempt OpenAI calls.
|
||||
*
|
||||
* Run: bun test test/e2e/dream-cycle-eight-phase-pglite.test.ts
|
||||
*/
|
||||
|
||||
import { describe, test, expect, mock } from 'bun:test';
|
||||
import { mkdtempSync, writeFileSync, rmSync, mkdirSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { execSync } from 'child_process';
|
||||
import { tmpdir } from 'os';
|
||||
import { PGLiteEngine } from '../../src/core/pglite-engine.ts';
|
||||
|
||||
mock.module('../../src/core/embedding.ts', () => ({
|
||||
embed: async () => new Float32Array(1536),
|
||||
embedBatch: async (texts: string[]) => texts.map(() => new Float32Array(1536)),
|
||||
EMBEDDING_MODEL: 'text-embedding-3-large',
|
||||
EMBEDDING_DIMENSIONS: 1536,
|
||||
EMBEDDING_COST_PER_1K_TOKENS: 0.00013,
|
||||
estimateEmbeddingCostUsd: (tokens: number) => (tokens / 1000) * 0.00013,
|
||||
}));
|
||||
|
||||
const { runCycle, ALL_PHASES } = await import('../../src/core/cycle.ts');
|
||||
|
||||
interface TestRig {
|
||||
engine: PGLiteEngine;
|
||||
brainDir: string;
|
||||
cleanup: () => Promise<void>;
|
||||
}
|
||||
|
||||
async function setupRig(): Promise<TestRig> {
|
||||
const engine = new PGLiteEngine();
|
||||
await engine.connect({ engine: 'pglite' } as never);
|
||||
await engine.initSchema();
|
||||
|
||||
const brainDir = mkdtempSync(join(tmpdir(), 'gbrain-cycle8-'));
|
||||
execSync('git init', { cwd: brainDir, stdio: 'pipe' });
|
||||
execSync('git config user.email test@test.co', { cwd: brainDir, stdio: 'pipe' });
|
||||
execSync('git config user.name test', { cwd: brainDir, stdio: 'pipe' });
|
||||
mkdirSync(join(brainDir, 'concepts'), { recursive: true });
|
||||
writeFileSync(
|
||||
join(brainDir, 'concepts/testing.md'),
|
||||
'---\ntype: concept\ntitle: Testing\n---\n\nTest body content.\n',
|
||||
);
|
||||
execSync('git add -A && git commit -m init', { cwd: brainDir, stdio: 'pipe' });
|
||||
await engine.setConfig('sync.repo_path', brainDir);
|
||||
|
||||
return {
|
||||
engine,
|
||||
brainDir,
|
||||
cleanup: async () => {
|
||||
try { await engine.disconnect(); } catch { /* */ }
|
||||
try { rmSync(brainDir, { recursive: true, force: true }); } catch { /* */ }
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async function withoutAnthropicKey<T>(body: () => Promise<T>): Promise<T> {
|
||||
const saved = process.env.ANTHROPIC_API_KEY;
|
||||
delete process.env.ANTHROPIC_API_KEY;
|
||||
try {
|
||||
return await body();
|
||||
} finally {
|
||||
if (saved === undefined) delete process.env.ANTHROPIC_API_KEY;
|
||||
else process.env.ANTHROPIC_API_KEY = saved;
|
||||
}
|
||||
}
|
||||
|
||||
describe('E2E v0.23 8-phase cycle', () => {
|
||||
test('ALL_PHASES is the 8-phase order in the documented sequence', () => {
|
||||
expect(ALL_PHASES).toEqual([
|
||||
'lint',
|
||||
'backlinks',
|
||||
'sync',
|
||||
'synthesize',
|
||||
'extract',
|
||||
'patterns',
|
||||
'embed',
|
||||
'orphans',
|
||||
]);
|
||||
});
|
||||
|
||||
test('full cycle on dry-run returns CycleReport.phases in v0.23 order with new totals fields', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await withoutAnthropicKey(async () => {
|
||||
const report = await runCycle(rig.engine, {
|
||||
brainDir: rig.brainDir,
|
||||
dryRun: true,
|
||||
});
|
||||
// Phase ordering preserved
|
||||
const phaseNames = report.phases.map(p => p.phase);
|
||||
expect(phaseNames).toEqual([
|
||||
'lint',
|
||||
'backlinks',
|
||||
'sync',
|
||||
'synthesize',
|
||||
'extract',
|
||||
'patterns',
|
||||
'embed',
|
||||
'orphans',
|
||||
]);
|
||||
// New totals fields exist (v0.23 additive growth)
|
||||
expect(report.totals).toMatchObject({
|
||||
transcripts_processed: 0,
|
||||
synth_pages_written: 0,
|
||||
patterns_written: 0,
|
||||
});
|
||||
// Synthesize and patterns are skipped (not_configured / insufficient_evidence)
|
||||
const synth = report.phases.find(p => p.phase === 'synthesize');
|
||||
const patterns = report.phases.find(p => p.phase === 'patterns');
|
||||
expect(synth?.status).toBe('skipped');
|
||||
expect(patterns?.status).toBe('skipped');
|
||||
});
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
});
|
||||
|
||||
test('--phase synthesize alone runs only that phase, returns skipped/not_configured', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await withoutAnthropicKey(async () => {
|
||||
const report = await runCycle(rig.engine, {
|
||||
brainDir: rig.brainDir,
|
||||
dryRun: false,
|
||||
phases: ['synthesize'],
|
||||
});
|
||||
expect(report.phases).toHaveLength(1);
|
||||
expect(report.phases[0].phase).toBe('synthesize');
|
||||
expect(report.phases[0].status).toBe('skipped');
|
||||
});
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
});
|
||||
|
||||
test('--phase patterns alone runs only that phase, returns skipped/insufficient_evidence', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await withoutAnthropicKey(async () => {
|
||||
const report = await runCycle(rig.engine, {
|
||||
brainDir: rig.brainDir,
|
||||
dryRun: false,
|
||||
phases: ['patterns'],
|
||||
});
|
||||
expect(report.phases).toHaveLength(1);
|
||||
expect(report.phases[0].phase).toBe('patterns');
|
||||
expect(report.phases[0].status).toBe('skipped');
|
||||
expect((report.phases[0].details as { reason?: string }).reason).toBe('insufficient_evidence');
|
||||
});
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
});
|
||||
|
||||
test('synthInputFile flag is plumbed through runCycle to runPhaseSynthesize', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
const transcript = join(tmpdir(), `gbrain-e2e-cycle8-input-${Date.now()}.txt`);
|
||||
writeFileSync(transcript, 'sample conversation '.repeat(300));
|
||||
try {
|
||||
await withoutAnthropicKey(async () => {
|
||||
const report = await runCycle(rig.engine, {
|
||||
brainDir: rig.brainDir,
|
||||
dryRun: false,
|
||||
phases: ['synthesize'],
|
||||
synthInputFile: transcript,
|
||||
});
|
||||
// Without API key, synthesize falls through to no-key skip-path
|
||||
// and returns ok (NOT cooldown_active — explicit input bypasses).
|
||||
expect(report.phases[0].phase).toBe('synthesize');
|
||||
expect(report.phases[0].status).toBe('ok');
|
||||
});
|
||||
} finally {
|
||||
rmSync(transcript, { force: true });
|
||||
}
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -1,176 +0,0 @@
|
||||
/**
|
||||
* E2E patterns phase — PGLite, no API key required.
|
||||
*
|
||||
* Mirrors the per-test-rig pattern from dream-synthesize-pglite.test.ts.
|
||||
* Each test creates and tears down its own PGLite engine to avoid
|
||||
* cross-test contention (CLAUDE.md issue #223 macOS WASM bug).
|
||||
*
|
||||
* Covers the runPhasePatterns skip paths that don't require a real
|
||||
* Anthropic call:
|
||||
* - disabled: dream.patterns.enabled=false → skipped
|
||||
* - insufficient_evidence: <min_evidence reflections → skipped
|
||||
* - no_api_key: enough reflections, no ANTHROPIC_API_KEY → skipped
|
||||
* - dry-run: passes through with reflections_considered + zero pages
|
||||
*
|
||||
* The Sonnet detection path is structurally covered in
|
||||
* test/cycle-patterns.test.ts (asserts queue + waitForCompletion are
|
||||
* wired, allow-list reads from filing-rules JSON, slug provenance from
|
||||
* subagent_tool_executions, no raw_data dependency).
|
||||
*
|
||||
* Run: bun test test/e2e/dream-patterns-pglite.test.ts
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { PGLiteEngine } from '../../src/core/pglite-engine.ts';
|
||||
import { runPhasePatterns } from '../../src/core/cycle/patterns.ts';
|
||||
|
||||
interface TestRig {
|
||||
engine: PGLiteEngine;
|
||||
brainDir: string;
|
||||
cleanup: () => Promise<void>;
|
||||
}
|
||||
|
||||
async function setupRig(): Promise<TestRig> {
|
||||
const engine = new PGLiteEngine();
|
||||
await engine.connect({ engine: 'pglite' } as never);
|
||||
await engine.initSchema();
|
||||
return {
|
||||
engine,
|
||||
brainDir: '/tmp/gbrain-patterns-test',
|
||||
cleanup: async () => {
|
||||
try { await engine.disconnect(); } catch { /* */ }
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async function withoutAnthropicKey<T>(body: () => Promise<T>): Promise<T> {
|
||||
const saved = process.env.ANTHROPIC_API_KEY;
|
||||
delete process.env.ANTHROPIC_API_KEY;
|
||||
try {
|
||||
return await body();
|
||||
} finally {
|
||||
if (saved === undefined) delete process.env.ANTHROPIC_API_KEY;
|
||||
else process.env.ANTHROPIC_API_KEY = saved;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Insert N reflection pages directly via engine.putPage so the patterns
|
||||
* gather query has data without going through the synthesize phase.
|
||||
* Slugs follow the v0.23 wiki/personal/reflections/<topic>-<hash> shape.
|
||||
*/
|
||||
async function seedReflections(engine: PGLiteEngine, count: number): Promise<void> {
|
||||
for (let i = 0; i < count; i++) {
|
||||
const slug = `wiki/personal/reflections/2026-04-${String(15 + i).padStart(2, '0')}-test-pattern-aaa${i}`;
|
||||
await engine.putPage(slug, {
|
||||
type: 'note',
|
||||
title: `Reflection ${i}`,
|
||||
compiled_truth: `Sample reflection content ${i} discussing recurring theme of work-life balance.`,
|
||||
timeline: '',
|
||||
frontmatter: { type: 'note', title: `Reflection ${i}` },
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
describe('E2E patterns — disabled', () => {
|
||||
test('skipped when dream.patterns.enabled=false', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.patterns.enabled', 'false');
|
||||
const result = await runPhasePatterns(rig.engine, {
|
||||
brainDir: rig.brainDir,
|
||||
dryRun: false,
|
||||
});
|
||||
expect(result.status).toBe('skipped');
|
||||
expect((result.details as { reason?: string }).reason).toBe('disabled');
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
});
|
||||
|
||||
test('default-enabled when config key unset', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
// No reflections seeded → falls through to insufficient_evidence,
|
||||
// not disabled. Confirms the default-true semantics.
|
||||
const result = await runPhasePatterns(rig.engine, {
|
||||
brainDir: rig.brainDir,
|
||||
dryRun: false,
|
||||
});
|
||||
expect(result.status).toBe('skipped');
|
||||
expect((result.details as { reason?: string }).reason).toBe('insufficient_evidence');
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('E2E patterns — insufficient_evidence', () => {
|
||||
test('skipped with 0 reflections', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
const result = await runPhasePatterns(rig.engine, {
|
||||
brainDir: rig.brainDir,
|
||||
dryRun: false,
|
||||
});
|
||||
expect(result.status).toBe('skipped');
|
||||
expect((result.details as { reason?: string }).reason).toBe('insufficient_evidence');
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
});
|
||||
|
||||
test('skipped with reflections below min_evidence', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.patterns.min_evidence', '5');
|
||||
await seedReflections(rig.engine, 3); // below 5
|
||||
const result = await runPhasePatterns(rig.engine, {
|
||||
brainDir: rig.brainDir,
|
||||
dryRun: false,
|
||||
});
|
||||
expect(result.status).toBe('skipped');
|
||||
expect((result.details as { reason?: string }).reason).toBe('insufficient_evidence');
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('E2E patterns — no API key', () => {
|
||||
test('enough reflections, no ANTHROPIC_API_KEY → skipped no_api_key', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await seedReflections(rig.engine, 5); // above default min_evidence (3)
|
||||
await withoutAnthropicKey(async () => {
|
||||
const result = await runPhasePatterns(rig.engine, {
|
||||
brainDir: rig.brainDir,
|
||||
dryRun: false,
|
||||
});
|
||||
expect(result.status).toBe('skipped');
|
||||
expect((result.details as { reason?: string }).reason).toBe('no_api_key');
|
||||
});
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('E2E patterns — dry-run', () => {
|
||||
test('dry-run returns ok with reflections_considered and zero patterns_written', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await seedReflections(rig.engine, 5);
|
||||
const result = await runPhasePatterns(rig.engine, {
|
||||
brainDir: rig.brainDir,
|
||||
dryRun: true,
|
||||
});
|
||||
expect(result.status).toBe('ok');
|
||||
expect((result.details as { dryRun: boolean }).dryRun).toBe(true);
|
||||
expect((result.details as { reflections_considered: number }).reflections_considered).toBe(5);
|
||||
expect((result.details as { patterns_written: number }).patterns_written).toBe(0);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -1,246 +0,0 @@
|
||||
/**
|
||||
* E2E synthesize phase — PGLite, no API key required.
|
||||
*
|
||||
* Each test creates and tears down its own PGLite engine to avoid
|
||||
* cross-test contention. Trades startup cost for isolation — required
|
||||
* because PGLite's WASM instance has been observed to wedge under
|
||||
* sustained concurrent-test pressure on macOS (CLAUDE.md issue #223).
|
||||
*
|
||||
* Mirrors the per-test-rig pattern used in
|
||||
* test/e2e/dream-allow-list-pglite.test.ts.
|
||||
*
|
||||
* Run: bun test test/e2e/dream-synthesize-pglite.test.ts
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { mkdtempSync, rmSync, writeFileSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
import { PGLiteEngine } from '../../src/core/pglite-engine.ts';
|
||||
import { runPhaseSynthesize } from '../../src/core/cycle/synthesize.ts';
|
||||
|
||||
interface TestRig {
|
||||
engine: PGLiteEngine;
|
||||
brainDir: string;
|
||||
corpusDir: string;
|
||||
cleanup: () => Promise<void>;
|
||||
}
|
||||
|
||||
async function setupRig(): Promise<TestRig> {
|
||||
const engine = new PGLiteEngine();
|
||||
await engine.connect({ engine: 'pglite' } as never);
|
||||
await engine.initSchema();
|
||||
const brainDir = mkdtempSync(join(tmpdir(), 'gbrain-synth-brain-'));
|
||||
const corpusDir = mkdtempSync(join(tmpdir(), 'gbrain-synth-corpus-'));
|
||||
return {
|
||||
engine,
|
||||
brainDir,
|
||||
corpusDir,
|
||||
cleanup: async () => {
|
||||
try { await engine.disconnect(); } catch { /* best-effort */ }
|
||||
try { rmSync(brainDir, { recursive: true, force: true }); } catch { /* */ }
|
||||
try { rmSync(corpusDir, { recursive: true, force: true }); } catch { /* */ }
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Run `body` with ANTHROPIC_API_KEY temporarily cleared, restoring the
|
||||
* prior value (set or unset) on return — even on throw — so this never
|
||||
* leaks state to sibling test files in the suite.
|
||||
*/
|
||||
async function withoutAnthropicKey<T>(body: () => Promise<T>): Promise<T> {
|
||||
const saved = process.env.ANTHROPIC_API_KEY;
|
||||
delete process.env.ANTHROPIC_API_KEY;
|
||||
try {
|
||||
return await body();
|
||||
} finally {
|
||||
if (saved === undefined) delete process.env.ANTHROPIC_API_KEY;
|
||||
else process.env.ANTHROPIC_API_KEY = saved;
|
||||
}
|
||||
}
|
||||
|
||||
describe('E2E synthesize — disabled / not_configured', () => {
|
||||
test('not_configured when enabled=false (default)', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
const result = await runPhaseSynthesize(rig.engine, {
|
||||
brainDir: rig.brainDir,
|
||||
dryRun: false,
|
||||
});
|
||||
expect(result.status).toBe('skipped');
|
||||
expect((result.details as { reason?: string }).reason).toBe('not_configured');
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
});
|
||||
|
||||
test('not_configured when enabled=true but session_corpus_dir is empty', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.synthesize.enabled', 'true');
|
||||
const result = await runPhaseSynthesize(rig.engine, {
|
||||
brainDir: rig.brainDir,
|
||||
dryRun: false,
|
||||
});
|
||||
expect(result.status).toBe('skipped');
|
||||
expect((result.details as { reason?: string }).reason).toBe('not_configured');
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('E2E synthesize — empty corpus', () => {
|
||||
test('ok status with zero transcripts when corpus dir is empty', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.synthesize.enabled', 'true');
|
||||
await rig.engine.setConfig('dream.synthesize.session_corpus_dir', rig.corpusDir);
|
||||
const result = await runPhaseSynthesize(rig.engine, {
|
||||
brainDir: rig.brainDir,
|
||||
dryRun: false,
|
||||
});
|
||||
expect(result.status).toBe('ok');
|
||||
expect((result.details as { transcripts_processed: number }).transcripts_processed).toBe(0);
|
||||
expect((result.details as { pages_written: number }).pages_written).toBe(0);
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('E2E synthesize — no API key skip path', () => {
|
||||
test('without ANTHROPIC_API_KEY, every transcript verdict is "no key" and zero pages written', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.synthesize.enabled', 'true');
|
||||
await rig.engine.setConfig('dream.synthesize.session_corpus_dir', rig.corpusDir);
|
||||
writeFileSync(
|
||||
join(rig.corpusDir, '2026-04-25-session.txt'),
|
||||
'a meaningful conversation\n'.repeat(200),
|
||||
);
|
||||
await withoutAnthropicKey(async () => {
|
||||
const result = await runPhaseSynthesize(rig.engine, {
|
||||
brainDir: rig.brainDir,
|
||||
dryRun: false,
|
||||
});
|
||||
expect(result.status).toBe('ok');
|
||||
expect((result.details as { transcripts_processed: number }).transcripts_processed).toBe(0);
|
||||
expect((result.details as { pages_written: number }).pages_written).toBe(0);
|
||||
const verdicts = (result.details as { verdicts: Array<{ worth: boolean; reasons: string[] }> }).verdicts;
|
||||
expect(verdicts).toHaveLength(1);
|
||||
expect(verdicts[0].worth).toBe(false);
|
||||
expect(verdicts[0].reasons[0]).toMatch(/ANTHROPIC_API_KEY/);
|
||||
});
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('E2E synthesize — dry-run skips Sonnet (Codex finding #8)', () => {
|
||||
test('dry-run reports planned action with zero pages_written', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.synthesize.enabled', 'true');
|
||||
await rig.engine.setConfig('dream.synthesize.session_corpus_dir', rig.corpusDir);
|
||||
writeFileSync(
|
||||
join(rig.corpusDir, '2026-04-25-session.txt'),
|
||||
'a meaningful conversation\n'.repeat(200),
|
||||
);
|
||||
await withoutAnthropicKey(async () => {
|
||||
const result = await runPhaseSynthesize(rig.engine, {
|
||||
brainDir: rig.brainDir,
|
||||
dryRun: true,
|
||||
});
|
||||
expect(result.status).toBe('ok');
|
||||
expect((result.details as { dryRun: boolean }).dryRun).toBe(true);
|
||||
expect((result.details as { pages_written: number }).pages_written).toBe(0);
|
||||
expect(result.summary).toMatch(/dry-run/);
|
||||
});
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('E2E synthesize — cooldown', () => {
|
||||
test('cooldown_active when last_completion_ts is fresh', async () => {
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.synthesize.enabled', 'true');
|
||||
await rig.engine.setConfig('dream.synthesize.session_corpus_dir', rig.corpusDir);
|
||||
await rig.engine.setConfig('dream.synthesize.last_completion_ts', new Date().toISOString());
|
||||
await rig.engine.setConfig('dream.synthesize.cooldown_hours', '12');
|
||||
const result = await runPhaseSynthesize(rig.engine, {
|
||||
brainDir: rig.brainDir,
|
||||
dryRun: false,
|
||||
});
|
||||
expect(result.status).toBe('skipped');
|
||||
expect((result.details as { reason?: string }).reason).toBe('cooldown_active');
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
});
|
||||
|
||||
test('explicit --input bypasses cooldown', async () => {
|
||||
// Two engine setups + a synth run; default 5s is tight under full-suite pressure.
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.synthesize.enabled', 'true');
|
||||
await rig.engine.setConfig('dream.synthesize.session_corpus_dir', rig.corpusDir);
|
||||
await rig.engine.setConfig('dream.synthesize.last_completion_ts', new Date().toISOString());
|
||||
const adHoc = join(tmpdir(), `gbrain-synth-ad-hoc-${Date.now()}-${Math.random().toString(36).slice(2)}.txt`);
|
||||
writeFileSync(adHoc, 'hello world '.repeat(300));
|
||||
try {
|
||||
await withoutAnthropicKey(async () => {
|
||||
const result = await runPhaseSynthesize(rig.engine, {
|
||||
brainDir: rig.brainDir,
|
||||
dryRun: false,
|
||||
inputFile: adHoc,
|
||||
});
|
||||
expect(result.status).toBe('ok');
|
||||
expect((result.details as { reason?: string }).reason).toBeUndefined();
|
||||
});
|
||||
} finally {
|
||||
rmSync(adHoc, { force: true });
|
||||
}
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
});
|
||||
|
||||
describe('E2E synthesize — verdict cache (Q-2)', () => {
|
||||
test('subsequent run with same content reads from dream_verdicts cache', async () => {
|
||||
// Two synth runs through the verdict-cache path; default 5s is tight.
|
||||
const rig = await setupRig();
|
||||
try {
|
||||
await rig.engine.setConfig('dream.synthesize.enabled', 'true');
|
||||
await rig.engine.setConfig('dream.synthesize.session_corpus_dir', rig.corpusDir);
|
||||
const filePath = join(rig.corpusDir, '2026-04-25-session.txt');
|
||||
const body = 'a meaningful conversation\n'.repeat(200);
|
||||
writeFileSync(filePath, body);
|
||||
await withoutAnthropicKey(async () => {
|
||||
await runPhaseSynthesize(rig.engine, { brainDir: rig.brainDir, dryRun: false });
|
||||
const { createHash } = await import('node:crypto');
|
||||
const hash = createHash('sha256').update(body, 'utf8').digest('hex');
|
||||
await rig.engine.putDreamVerdict(filePath, hash, {
|
||||
worth_processing: false,
|
||||
reasons: ['cached test verdict'],
|
||||
});
|
||||
const result = await runPhaseSynthesize(rig.engine, {
|
||||
brainDir: rig.brainDir,
|
||||
dryRun: false,
|
||||
});
|
||||
expect(result.status).toBe('ok');
|
||||
const verdicts = (result.details as { verdicts: Array<{ cached: boolean }> }).verdicts;
|
||||
expect(verdicts).toHaveLength(1);
|
||||
expect(verdicts[0].cached).toBe(true);
|
||||
});
|
||||
} finally {
|
||||
await rig.cleanup();
|
||||
}
|
||||
}, 30_000);
|
||||
});
|
||||
@@ -1,32 +0,0 @@
|
||||
# Claw-test brief — fresh-install
|
||||
|
||||
You are testing gbrain on a brand-new install. The user just ran `gbrain init` for the first time. Walk through the canonical first-day flow:
|
||||
|
||||
1. **Verify install:** confirm `gbrain --version` works and `gbrain doctor --json` returns a valid JSON object with a `status` field.
|
||||
2. **Install skillpack:** run `gbrain skillpack install --workspace $PWD`. The workspace already has an `AGENTS.md` routing file.
|
||||
3. **Import the brain:** run `gbrain import ./brain --no-embed --progress-json`. There are 3 small markdown pages already there.
|
||||
4. **Query the brain:** run `gbrain query "alice"` and verify >0 results.
|
||||
5. **Extract links:** run `gbrain extract --source fs --progress-json`.
|
||||
6. **Verify health:** run `gbrain doctor --json`. The `status` field should be `"ok"`.
|
||||
|
||||
## Friction protocol
|
||||
|
||||
If anything is confusing, missing, surprising, or wrong, run:
|
||||
|
||||
```
|
||||
gbrain friction log --severity {confused|error|blocker|nit} --phase <which-step> --message "<what-happened>" [--hint "<what-could-be-better>"]
|
||||
```
|
||||
|
||||
Severity guide:
|
||||
- `blocker` — couldn't proceed at all
|
||||
- `error` — command failed unexpectedly
|
||||
- `confused` — docs said one thing, the tool did another, or a step felt unclear
|
||||
- `nit` — minor polish opportunity
|
||||
|
||||
If something *just worked* and was nicer than expected, log a delight too:
|
||||
|
||||
```
|
||||
gbrain friction log --kind delight --phase <step> --message "<what-was-nice>"
|
||||
```
|
||||
|
||||
We want to know what didn't work, not just whether commands exited zero. Be specific.
|
||||
-15
@@ -1,15 +0,0 @@
|
||||
---
|
||||
type: company
|
||||
name: Acme Example
|
||||
founded: 2024
|
||||
founders:
|
||||
- alice-example
|
||||
---
|
||||
|
||||
# Acme Example
|
||||
|
||||
Fictional company used for claw-test fixtures. Founded 2024 by [Alice](people/alice-example).
|
||||
|
||||
## What they do
|
||||
|
||||
Acme builds an agentic-workflow product on top of [retrieval-augmented-generation](concepts/retrieval-augmented-generation). Early traction comes from a developer-tools wedge.
|
||||
-10
@@ -1,10 +0,0 @@
|
||||
---
|
||||
type: concept
|
||||
name: Agentic Workflows
|
||||
---
|
||||
|
||||
# Agentic Workflows
|
||||
|
||||
Workflows where an LLM-driven agent plans, executes, and revises a sequence of steps with minimal human supervision per step. Key constraints: cost, latency, and observability of the loop.
|
||||
|
||||
Companies building in this space include [acme-example](companies/acme-example).
|
||||
-13
@@ -1,13 +0,0 @@
|
||||
---
|
||||
type: person
|
||||
name: Alice Example
|
||||
x_handle: alice_example
|
||||
---
|
||||
|
||||
# Alice Example
|
||||
|
||||
Alice is a fictional founder used for claw-test fixtures. She started [acme-example](companies/acme-example) in 2024.
|
||||
|
||||
## Background
|
||||
|
||||
Alice has spent 10 years in software and 2 years in AI tooling. She is exploring product-market fit for an [agentic-workflow](concepts/agentic-workflows) tool.
|
||||
@@ -1,6 +0,0 @@
|
||||
{
|
||||
"min_pages_after_import": 3,
|
||||
"min_query_results": 1,
|
||||
"min_links_after_extract": 0,
|
||||
"doctor_status": "ok"
|
||||
}
|
||||
@@ -1,10 +0,0 @@
|
||||
{
|
||||
"kind": "fresh-install",
|
||||
"description": "Canonical 5-minute first-day flow: init → import → query → extract → verify",
|
||||
"expected_phases": [
|
||||
"import.files",
|
||||
"extract.links_fs",
|
||||
"doctor.db_checks"
|
||||
],
|
||||
"brain": "brain"
|
||||
}
|
||||
@@ -1,25 +0,0 @@
|
||||
# Claw-test brief — upgrade-from-v0.18
|
||||
|
||||
You inherit a gbrain v0.18 brain (the harness has already replayed a seed SQL dump into a PGLite database). Walk through the upgrade path:
|
||||
|
||||
1. **Run `gbrain doctor --json`** first. Note any warnings or fix-hints.
|
||||
2. **Run `gbrain init --pglite`** with the existing database path. The migration chain should detect the old `schema_version` and walk forward to the latest.
|
||||
3. **Run `gbrain doctor --json` again.** The `status` field should be `"ok"`.
|
||||
4. **Verify queries still work:** `gbrain query "alice"` should return results from the seeded brain.
|
||||
|
||||
## Friction protocol
|
||||
|
||||
If anything is confusing, missing, surprising, or wrong (especially around the migration steps — these are the highest-historical-pain regression points), run:
|
||||
|
||||
```
|
||||
gbrain friction log --severity {confused|error|blocker|nit} --phase <which-step> --message "<what-happened>" [--hint "<what-could-be-better>"]
|
||||
```
|
||||
|
||||
Common upgrade-flow friction patterns to watch for:
|
||||
|
||||
- The migration chain failed at a specific schema version (capture the version + error)
|
||||
- Doctor flagged an issue but the fix-hint wasn't actionable
|
||||
- `gbrain init --pglite` didn't recognize the existing brain
|
||||
- Manual SQL was needed to unblock something
|
||||
|
||||
If something just worked, log a delight. We're tuning the upgrade flow toward zero-friction.
|
||||
-8
@@ -1,8 +0,0 @@
|
||||
---
|
||||
type: person
|
||||
name: Alice Example
|
||||
---
|
||||
|
||||
# Alice Example
|
||||
|
||||
Same brain content as the fresh-install scenario; this scenario tests upgrade flow rather than ingest. After the migration chain walks forward, agents query and the page must be findable.
|
||||
@@ -1,4 +0,0 @@
|
||||
{
|
||||
"min_pages_after_migration": 1,
|
||||
"doctor_status": "ok"
|
||||
}
|
||||
@@ -1,10 +0,0 @@
|
||||
{
|
||||
"kind": "upgrade",
|
||||
"from_version": "0.18.0",
|
||||
"description": "Pre-v0.18 brain shape replayed via PGLite SQL dump; migration chain walks forward to LATEST",
|
||||
"expected_phases": [
|
||||
"doctor.db_checks"
|
||||
],
|
||||
"seed": "seed",
|
||||
"brain": "brain"
|
||||
}
|
||||
@@ -1,24 +0,0 @@
|
||||
# v0.18 seed
|
||||
|
||||
This directory ships in v1 as **scaffolding only** — `dump.sql` will contain a real v0.18-shape PGLite SQL dump in v1.1. Until then the harness treats the absent dump as a no-op seed and the upgrade scenario behaves like a fresh-install scenario for the test gate.
|
||||
|
||||
## Generating a real v0.18 seed
|
||||
|
||||
To produce an authentic seed:
|
||||
|
||||
1. Check out gbrain at the v0.18 release (`git checkout v0.18.0`).
|
||||
2. Run `gbrain init --pglite --path /tmp/v0.18-seed.pglite` against a small fixture brain.
|
||||
3. Run `gbrain import <fixture-brain>` to populate it.
|
||||
4. Dump the PGLite as SQL: PGLite supports `pg_dump`-style export via the `executeRaw('SELECT * FROM pg_dump(...)')` extension or via direct file copy. If neither path works, run `pglite-tools dump /tmp/v0.18-seed.pglite > dump.sql`.
|
||||
5. Place `dump.sql` here.
|
||||
6. Update `expected.json::min_pages_after_migration` to match your dump's page count.
|
||||
|
||||
## What gets tested
|
||||
|
||||
When `dump.sql` exists, the harness:
|
||||
|
||||
- Runs `seedPgliteFromFile()` to replay the dump into a fresh `<tempdir>/.gbrain/brain.pglite`
|
||||
- Then runs `gbrain init --pglite` so the migration chain detects the old schema_version and walks forward to LATEST
|
||||
- Asserts `gbrain doctor --json` returns `status: 'ok'` after the walk
|
||||
|
||||
This is the regression gate for the upgrade-wedge bug class (#239/#243/#266/#357/#366/#374/#375/#378/#395/#396) — every gbrain release that adds a column-with-index in the embedded schema blob without a corresponding bootstrap retriggered the same wedge family.
|
||||
@@ -1,196 +0,0 @@
|
||||
/**
|
||||
* Friction CLI dispatch tests. Exercises the thin command layer (each
|
||||
* subcommand stays ≤ 30 LOC per the DRY contract from the eng review).
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { mkdtempSync, rmSync, readFileSync, existsSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
import { runFriction } from '../src/commands/friction.ts';
|
||||
import { frictionFile, frictionDir } from '../src/core/friction.ts';
|
||||
|
||||
const ORIG_HOME = process.env.GBRAIN_HOME;
|
||||
const ORIG_RUN_ID = process.env.GBRAIN_FRICTION_RUN_ID;
|
||||
let tmp: string;
|
||||
let stdoutLines: string[];
|
||||
let stderrLines: string[];
|
||||
let origStdoutWrite: typeof process.stdout.write;
|
||||
let origConsoleLog: typeof console.log;
|
||||
let origConsoleError: typeof console.error;
|
||||
|
||||
beforeEach(() => {
|
||||
tmp = mkdtempSync(join(tmpdir(), 'friction-cli-'));
|
||||
process.env.GBRAIN_HOME = tmp;
|
||||
delete process.env.GBRAIN_FRICTION_RUN_ID;
|
||||
stdoutLines = [];
|
||||
stderrLines = [];
|
||||
origStdoutWrite = process.stdout.write.bind(process.stdout);
|
||||
origConsoleLog = console.log;
|
||||
origConsoleError = console.error;
|
||||
process.stdout.write = ((chunk: string) => { stdoutLines.push(String(chunk)); return true; }) as any;
|
||||
console.log = (...args: unknown[]) => { stdoutLines.push(args.join(' ') + '\n'); };
|
||||
console.error = (...args: unknown[]) => { stderrLines.push(args.join(' ') + '\n'); };
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
process.env.GBRAIN_HOME = ORIG_HOME;
|
||||
if (ORIG_RUN_ID !== undefined) process.env.GBRAIN_FRICTION_RUN_ID = ORIG_RUN_ID;
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
process.stdout.write = origStdoutWrite;
|
||||
console.log = origConsoleLog;
|
||||
console.error = origConsoleError;
|
||||
});
|
||||
|
||||
describe('dispatch', () => {
|
||||
test('--help returns 0 and prints subcommand list', () => {
|
||||
const code = runFriction(['--help']);
|
||||
expect(code).toBe(0);
|
||||
expect(stdoutLines.join('')).toContain('Subcommands');
|
||||
expect(stdoutLines.join('')).toContain('log');
|
||||
expect(stdoutLines.join('')).toContain('render');
|
||||
expect(stdoutLines.join('')).toContain('list');
|
||||
expect(stdoutLines.join('')).toContain('summary');
|
||||
});
|
||||
|
||||
test('unknown subcommand returns 2', () => {
|
||||
const code = runFriction(['nonsense']);
|
||||
expect(code).toBe(2);
|
||||
expect(stderrLines.join('')).toContain('unknown subcommand');
|
||||
});
|
||||
});
|
||||
|
||||
describe('log subcommand', () => {
|
||||
test('writes a friction entry under GBRAIN_HOME', () => {
|
||||
const code = runFriction(['log', '--run-id', 'cli-1', '--phase', 'install', '--message', 'something broke', '--severity', 'error']);
|
||||
expect(code).toBe(0);
|
||||
const path = frictionFile('cli-1');
|
||||
expect(existsSync(path)).toBe(true);
|
||||
expect(path.startsWith(tmp)).toBe(true);
|
||||
const raw = readFileSync(path, 'utf-8');
|
||||
expect(raw).toContain('something broke');
|
||||
});
|
||||
|
||||
test('missing --phase returns 2 with usage', () => {
|
||||
const code = runFriction(['log', '--message', 'foo']);
|
||||
expect(code).toBe(2);
|
||||
expect(stderrLines.join('')).toContain('usage');
|
||||
});
|
||||
|
||||
test('missing --message returns 2 with usage', () => {
|
||||
const code = runFriction(['log', '--phase', 'p']);
|
||||
expect(code).toBe(2);
|
||||
expect(stderrLines.join('')).toContain('usage');
|
||||
});
|
||||
|
||||
test('invalid --severity returns 2', () => {
|
||||
const code = runFriction(['log', '--run-id', 'cli-2', '--phase', 'p', '--message', 'm', '--severity', 'panicking']);
|
||||
expect(code).toBe(2);
|
||||
expect(stderrLines.join('')).toContain('invalid --severity');
|
||||
});
|
||||
|
||||
test('invalid --kind returns 2', () => {
|
||||
const code = runFriction(['log', '--run-id', 'cli-3', '--phase', 'p', '--message', 'm', '--kind', 'bogus']);
|
||||
expect(code).toBe(2);
|
||||
expect(stderrLines.join('')).toContain('invalid --kind');
|
||||
});
|
||||
|
||||
test('--kind delight is recorded', () => {
|
||||
runFriction(['log', '--run-id', 'cli-4', '--phase', 'p', '--message', 'great', '--kind', 'delight']);
|
||||
const raw = readFileSync(frictionFile('cli-4'), 'utf-8');
|
||||
expect(raw).toContain('"kind":"delight"');
|
||||
});
|
||||
});
|
||||
|
||||
describe('render subcommand', () => {
|
||||
test('renders markdown by default', () => {
|
||||
runFriction(['log', '--run-id', 'cli-r', '--phase', 'install', '--message', 'beep', '--severity', 'error']);
|
||||
stdoutLines.length = 0;
|
||||
const code = runFriction(['render', '--run-id', 'cli-r']);
|
||||
expect(code).toBe(0);
|
||||
const out = stdoutLines.join('');
|
||||
expect(out).toContain('# Friction report');
|
||||
expect(out).toContain('## error');
|
||||
});
|
||||
|
||||
test('--json emits parseable JSON', () => {
|
||||
runFriction(['log', '--run-id', 'cli-r2', '--phase', 'p', '--message', 'beep']);
|
||||
stdoutLines.length = 0;
|
||||
const code = runFriction(['render', '--run-id', 'cli-r2', '--json']);
|
||||
expect(code).toBe(0);
|
||||
const out = stdoutLines.join('').trim();
|
||||
const parsed = JSON.parse(out);
|
||||
expect(parsed.run_id).toBe('cli-r2');
|
||||
expect(parsed.entries.length).toBe(1);
|
||||
});
|
||||
|
||||
test('missing run-id returns 1 with actionable error', () => {
|
||||
const code = runFriction(['render', '--run-id', 'no-such-run']);
|
||||
expect(code).toBe(1);
|
||||
expect(stderrLines.join('')).toContain('not found');
|
||||
});
|
||||
});
|
||||
|
||||
describe('list subcommand', () => {
|
||||
test('reports no runs initially', () => {
|
||||
const code = runFriction(['list']);
|
||||
expect(code).toBe(0);
|
||||
expect(stdoutLines.join('')).toContain('no runs');
|
||||
});
|
||||
|
||||
test('lists logged runs with counts', () => {
|
||||
runFriction(['log', '--run-id', 'a', '--phase', 'p', '--message', 'm', '--severity', 'error']);
|
||||
runFriction(['log', '--run-id', 'b', '--phase', 'p', '--message', 'm', '--kind', 'delight']);
|
||||
stdoutLines.length = 0;
|
||||
const code = runFriction(['list']);
|
||||
expect(code).toBe(0);
|
||||
const out = stdoutLines.join('');
|
||||
expect(out).toContain('a');
|
||||
expect(out).toContain('b');
|
||||
});
|
||||
|
||||
test('--json emits parseable JSON array', () => {
|
||||
runFriction(['log', '--run-id', 'jl', '--phase', 'p', '--message', 'm']);
|
||||
stdoutLines.length = 0;
|
||||
const code = runFriction(['list', '--json']);
|
||||
expect(code).toBe(0);
|
||||
const parsed = JSON.parse(stdoutLines.join('').trim());
|
||||
expect(Array.isArray(parsed)).toBe(true);
|
||||
expect(parsed[0].runId).toBe('jl');
|
||||
});
|
||||
});
|
||||
|
||||
describe('summary subcommand', () => {
|
||||
test('renders friction + delight columns', () => {
|
||||
runFriction(['log', '--run-id', 'sum-1', '--phase', 'p', '--message', 'broken thing']);
|
||||
runFriction(['log', '--run-id', 'sum-1', '--phase', 'p', '--message', 'nice thing', '--kind', 'delight']);
|
||||
stdoutLines.length = 0;
|
||||
const code = runFriction(['summary', '--run-id', 'sum-1']);
|
||||
expect(code).toBe(0);
|
||||
const out = stdoutLines.join('');
|
||||
expect(out).toContain('friction (1)');
|
||||
expect(out).toContain('delight (1)');
|
||||
expect(out).toContain('broken thing');
|
||||
expect(out).toContain('nice thing');
|
||||
});
|
||||
});
|
||||
|
||||
describe('GBRAIN_FRICTION_RUN_ID fallback (D19)', () => {
|
||||
test('log without --run-id uses standalone', () => {
|
||||
const code = runFriction(['log', '--phase', 'p', '--message', 'fallback']);
|
||||
expect(code).toBe(0);
|
||||
const path = frictionFile('standalone');
|
||||
expect(existsSync(path)).toBe(true);
|
||||
expect(readFileSync(path, 'utf-8')).toContain('fallback');
|
||||
});
|
||||
|
||||
test('log honors $GBRAIN_FRICTION_RUN_ID', () => {
|
||||
process.env.GBRAIN_FRICTION_RUN_ID = 'env-run';
|
||||
try {
|
||||
runFriction(['log', '--phase', 'p', '--message', 'env']);
|
||||
expect(existsSync(frictionFile('env-run'))).toBe(true);
|
||||
} finally {
|
||||
delete process.env.GBRAIN_FRICTION_RUN_ID;
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -1,232 +0,0 @@
|
||||
/**
|
||||
* Friction core: writer + reader + renderer + redactor.
|
||||
*
|
||||
* These tests are pure local-fs (no DB, no subprocess). They run under
|
||||
* GBRAIN_HOME=<tmp> for hermeticity — see test/gbrain-home-isolation.test.ts
|
||||
* for the regression gate proving every consumer honors that env.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { mkdtempSync, rmSync, existsSync, readFileSync, appendFileSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
import {
|
||||
logFriction, readFriction, listRuns, renderReport, renderSummary,
|
||||
redactEntry, frictionFile, frictionDir, activeRunId,
|
||||
type FrictionEntry,
|
||||
} from '../src/core/friction.ts';
|
||||
|
||||
const ORIG_HOME = process.env.GBRAIN_HOME;
|
||||
const ORIG_RUN_ID = process.env.GBRAIN_FRICTION_RUN_ID;
|
||||
let tmp: string;
|
||||
|
||||
beforeEach(() => {
|
||||
tmp = mkdtempSync(join(tmpdir(), 'friction-test-'));
|
||||
process.env.GBRAIN_HOME = tmp;
|
||||
delete process.env.GBRAIN_FRICTION_RUN_ID;
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
process.env.GBRAIN_HOME = ORIG_HOME;
|
||||
if (ORIG_RUN_ID !== undefined) process.env.GBRAIN_FRICTION_RUN_ID = ORIG_RUN_ID;
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
describe('writer', () => {
|
||||
test('logFriction appends one JSONL line and roundtrips through reader', () => {
|
||||
logFriction({ runId: 'run-a', phase: 'install', message: 'first', severity: 'error' });
|
||||
const { entries, malformed } = readFriction('run-a');
|
||||
expect(malformed).toBe(0);
|
||||
expect(entries).toHaveLength(1);
|
||||
expect(entries[0].message).toBe('first');
|
||||
expect(entries[0].severity).toBe('error');
|
||||
expect(entries[0].kind).toBe('friction');
|
||||
expect(entries[0].schema_version).toBe('1');
|
||||
expect(entries[0].run_id).toBe('run-a');
|
||||
});
|
||||
|
||||
test('multiple entries append in order', () => {
|
||||
logFriction({ runId: 'run-b', phase: 'p1', message: 'one', severity: 'nit' });
|
||||
logFriction({ runId: 'run-b', phase: 'p2', message: 'two', severity: 'blocker' });
|
||||
const { entries } = readFriction('run-b');
|
||||
expect(entries.map(e => e.message)).toEqual(['one', 'two']);
|
||||
});
|
||||
|
||||
test('long messages are truncated', () => {
|
||||
const long = 'x'.repeat(5000);
|
||||
logFriction({ runId: 'run-c', phase: 'p', message: long });
|
||||
const { entries } = readFriction('run-c');
|
||||
expect(entries[0].message.length).toBeLessThan(5000);
|
||||
expect(entries[0].message.endsWith('[truncated]')).toBe(true);
|
||||
});
|
||||
|
||||
test('kind: delight is recorded distinctly', () => {
|
||||
logFriction({ runId: 'run-d', phase: 'verify', message: 'this just worked', kind: 'delight' });
|
||||
const { entries } = readFriction('run-d');
|
||||
expect(entries[0].kind).toBe('delight');
|
||||
});
|
||||
|
||||
test('phase-marker entry roundtrips', () => {
|
||||
logFriction({ runId: 'run-e', phase: 'extract', message: 'phase started', kind: 'phase-marker', marker: 'start' });
|
||||
const { entries } = readFriction('run-e');
|
||||
expect(entries[0].kind).toBe('phase-marker');
|
||||
expect(entries[0].marker).toBe('start');
|
||||
});
|
||||
|
||||
test('error envelope fields flatten in (D20)', () => {
|
||||
logFriction({
|
||||
runId: 'run-f',
|
||||
phase: 'install',
|
||||
message: 'spawn failed',
|
||||
severity: 'blocker',
|
||||
errorClass: 'AgentSpawnError',
|
||||
errorCode: 'spawn_enoent',
|
||||
docsUrl: 'https://example.test/docs',
|
||||
});
|
||||
const { entries } = readFriction('run-f');
|
||||
expect(entries[0].class).toBe('AgentSpawnError');
|
||||
expect(entries[0].code).toBe('spawn_enoent');
|
||||
expect(entries[0].docs_url).toBe('https://example.test/docs');
|
||||
});
|
||||
|
||||
test('rejects invalid run-id', () => {
|
||||
expect(() => logFriction({ runId: 'has space', phase: 'p', message: 'm' })).toThrow(/invalid run-id/);
|
||||
expect(() => logFriction({ runId: '../escape', phase: 'p', message: 'm' })).toThrow(/invalid run-id/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('activeRunId', () => {
|
||||
test('falls back to standalone when env unset (D19)', () => {
|
||||
delete process.env.GBRAIN_FRICTION_RUN_ID;
|
||||
expect(activeRunId()).toBe('standalone');
|
||||
});
|
||||
|
||||
test('reads GBRAIN_FRICTION_RUN_ID', () => {
|
||||
process.env.GBRAIN_FRICTION_RUN_ID = 'my-run';
|
||||
try {
|
||||
expect(activeRunId()).toBe('my-run');
|
||||
} finally {
|
||||
delete process.env.GBRAIN_FRICTION_RUN_ID;
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('reader', () => {
|
||||
test('skips malformed lines and counts them', () => {
|
||||
logFriction({ runId: 'run-g', phase: 'p', message: 'good' });
|
||||
appendFileSync(frictionFile('run-g'), 'this is not json\n', 'utf-8');
|
||||
appendFileSync(frictionFile('run-g'), '{"ts":"only","kind":"friction"}\n', 'utf-8');
|
||||
logFriction({ runId: 'run-g', phase: 'p', message: 'good2' });
|
||||
const { entries, malformed } = readFriction('run-g');
|
||||
expect(entries).toHaveLength(2);
|
||||
expect(malformed).toBe(2);
|
||||
});
|
||||
|
||||
test('throws on missing run-id', () => {
|
||||
expect(() => readFriction('does-not-exist')).toThrow(/not found/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('listRuns', () => {
|
||||
test('lists runs sorted most-recent-first', () => {
|
||||
logFriction({ runId: 'old-run', phase: 'p', message: 'a' });
|
||||
// Sleep one millisecond worth via busy-wait so mtime differs reliably
|
||||
const t0 = Date.now();
|
||||
while (Date.now() - t0 < 10) { /* spin */ }
|
||||
logFriction({ runId: 'new-run', phase: 'p', message: 'b' });
|
||||
const runs = listRuns();
|
||||
expect(runs.length).toBe(2);
|
||||
expect(runs[0].runId).toBe('new-run');
|
||||
expect(runs[1].runId).toBe('old-run');
|
||||
});
|
||||
|
||||
test('reports per-run counts and interrupted flag', () => {
|
||||
logFriction({ runId: 'run-h', phase: 'p', message: 'a', severity: 'error' });
|
||||
logFriction({ runId: 'run-h', phase: 'p', message: 'b', severity: 'error' });
|
||||
logFriction({ runId: 'run-h', phase: 'p', message: 'c', kind: 'delight' });
|
||||
logFriction({ runId: 'run-h', phase: 'p', message: 'killed', kind: 'interrupted' });
|
||||
const runs = listRuns();
|
||||
const r = runs.find(x => x.runId === 'run-h')!;
|
||||
expect(r.counts.friction).toBe(2);
|
||||
expect(r.counts.delight).toBe(1);
|
||||
expect(r.counts.interrupted).toBe(true);
|
||||
expect(r.counts.bySeverity.error).toBe(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe('renderer', () => {
|
||||
test('markdown groups by severity then phase', () => {
|
||||
logFriction({ runId: 'run-r', phase: 'install', message: 'a', severity: 'blocker' });
|
||||
logFriction({ runId: 'run-r', phase: 'install', message: 'b', severity: 'error' });
|
||||
logFriction({ runId: 'run-r', phase: 'verify', message: 'c', severity: 'error' });
|
||||
logFriction({ runId: 'run-r', phase: 'verify', message: 'positive', kind: 'delight' });
|
||||
const md = renderReport('run-r', { format: 'md', redact: false });
|
||||
expect(md).toContain('# Friction report');
|
||||
expect(md).toContain('## blocker');
|
||||
expect(md).toContain('## error');
|
||||
expect(md).toContain('### `install`');
|
||||
expect(md).toContain('### `verify`');
|
||||
// Blocker section comes before error section
|
||||
expect(md.indexOf('## blocker')).toBeLessThan(md.indexOf('## error'));
|
||||
});
|
||||
|
||||
test('json output is valid and includes entries', () => {
|
||||
logFriction({ runId: 'run-j', phase: 'p', message: 'one' });
|
||||
const out = renderReport('run-j', { format: 'json' });
|
||||
const parsed = JSON.parse(out);
|
||||
expect(parsed.run_id).toBe('run-j');
|
||||
expect(parsed.entries).toHaveLength(1);
|
||||
});
|
||||
|
||||
test('redact strips homedir and cwd from message + cwd field', () => {
|
||||
const home = process.env.HOME ?? '';
|
||||
const fakeCwd = process.cwd();
|
||||
logFriction({
|
||||
runId: 'run-red',
|
||||
phase: 'p',
|
||||
message: `error at ${home}/.gbrain/foo and ${fakeCwd}/bar.ts`,
|
||||
});
|
||||
const md = renderReport('run-red', { format: 'md', redact: true });
|
||||
expect(md).not.toContain(home + '/.gbrain');
|
||||
expect(md).toContain('<HOME>');
|
||||
expect(md).toContain('<CWD>');
|
||||
});
|
||||
|
||||
test('--no-redact path preserves homedir', () => {
|
||||
const home = process.env.HOME ?? '/tmp/none';
|
||||
logFriction({ runId: 'run-noredact', phase: 'p', message: `at ${home}/foo` });
|
||||
const md = renderReport('run-noredact', { format: 'md', redact: false });
|
||||
expect(md).toContain(home);
|
||||
});
|
||||
|
||||
test('interrupted run shows banner', () => {
|
||||
logFriction({ runId: 'run-i', phase: 'p', message: 'partial' });
|
||||
logFriction({ runId: 'run-i', phase: 'p', message: 'killed', kind: 'interrupted' });
|
||||
const md = renderReport('run-i', { format: 'md', redact: false });
|
||||
expect(md).toContain('Run was interrupted');
|
||||
});
|
||||
});
|
||||
|
||||
describe('summary', () => {
|
||||
test('two columns, friction + delight side-by-side', () => {
|
||||
logFriction({ runId: 'run-s', phase: 'p', message: 'bad-thing' });
|
||||
logFriction({ runId: 'run-s', phase: 'p', message: 'good-thing', kind: 'delight' });
|
||||
const md = renderSummary('run-s', { format: 'md' });
|
||||
expect(md).toContain('| friction (1) | delight (1) |');
|
||||
expect(md).toContain('bad-thing');
|
||||
expect(md).toContain('good-thing');
|
||||
});
|
||||
});
|
||||
|
||||
describe('redactEntry pure function', () => {
|
||||
test('replaces homedir occurrences', () => {
|
||||
const home = process.env.HOME ?? '/x';
|
||||
const e: FrictionEntry = {
|
||||
schema_version: '1', ts: 'now', run_id: 'r', phase: 'p', kind: 'friction',
|
||||
message: `${home}/secret/file.txt`, source: 'claw', cwd: '/cwd', gbrain_version: 'test',
|
||||
};
|
||||
const r = redactEntry(e);
|
||||
expect(r.message).toContain('<HOME>');
|
||||
expect(r.cwd).toBe('<CWD>');
|
||||
});
|
||||
});
|
||||
@@ -1,283 +0,0 @@
|
||||
/**
|
||||
* Tests for frontmatter-inference.ts — the zero-friction ingest pipeline.
|
||||
*
|
||||
* Validates that files without frontmatter get correct type, title, date,
|
||||
* source, and tags inferred from their filesystem path and content.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import {
|
||||
inferFrontmatter,
|
||||
extractDateFromFilename,
|
||||
extractTitleFromFilename,
|
||||
extractTitleFromHeading,
|
||||
serializeFrontmatter,
|
||||
applyInference,
|
||||
DIRECTORY_RULES,
|
||||
} from '../src/core/frontmatter-inference.ts';
|
||||
|
||||
// ── Date extraction ──────────────────────────────────────────────────
|
||||
|
||||
describe('extractDateFromFilename', () => {
|
||||
test('extracts YYYY-MM-DD from date-prefixed filename', () => {
|
||||
expect(extractDateFromFilename('2010-04-13 Apr 13 founders mtg.md')).toBe('2010-04-13');
|
||||
});
|
||||
|
||||
test('extracts date with dash separator', () => {
|
||||
expect(extractDateFromFilename('2024-01-30-therapy-session.md')).toBe('2024-01-30');
|
||||
});
|
||||
|
||||
test('extracts date with underscore separator', () => {
|
||||
expect(extractDateFromFilename('2023-06-15_meeting-notes.md')).toBe('2023-06-15');
|
||||
});
|
||||
|
||||
test('returns null for no-date filename', () => {
|
||||
expect(extractDateFromFilename('README.md')).toBe(null);
|
||||
});
|
||||
|
||||
test('returns null for filename with numbers but no date', () => {
|
||||
expect(extractDateFromFilename('chapter-1-intro.md')).toBe(null);
|
||||
});
|
||||
});
|
||||
|
||||
// ── Title extraction ─────────────────────────────────────────────────
|
||||
|
||||
describe('extractTitleFromFilename', () => {
|
||||
test('strips date prefix and cleans up', () => {
|
||||
expect(extractTitleFromFilename('2010-04-13 Apr 13 founders mtg.md')).toBe('Apr 13 founders mtg');
|
||||
});
|
||||
|
||||
test('strips YYYY-MM-DD- prefix', () => {
|
||||
expect(extractTitleFromFilename('2024-01-30-therapy-session.md')).toBe('Therapy Session');
|
||||
});
|
||||
|
||||
test('handles filename without date', () => {
|
||||
expect(extractTitleFromFilename('cognitive-distortions.md')).toBe('Cognitive Distortions');
|
||||
});
|
||||
|
||||
test('preserves mixed case', () => {
|
||||
expect(extractTitleFromFilename('YC presidency.md')).toBe('YC presidency');
|
||||
});
|
||||
|
||||
test('returns Untitled for empty result', () => {
|
||||
expect(extractTitleFromFilename('.md')).toBe('Untitled');
|
||||
});
|
||||
});
|
||||
|
||||
describe('extractTitleFromHeading', () => {
|
||||
test('extracts first # heading', () => {
|
||||
expect(extractTitleFromHeading('# Dhravya Shah\n\n> Founder of Supermemory')).toBe('Dhravya Shah');
|
||||
});
|
||||
|
||||
test('ignores ## headings', () => {
|
||||
expect(extractTitleFromHeading('Some text\n## Not this\n# This one')).toBe('This one');
|
||||
});
|
||||
|
||||
test('returns null when no heading found', () => {
|
||||
expect(extractTitleFromHeading('Just some text\nwithout headings')).toBe(null);
|
||||
});
|
||||
|
||||
test('looks within first 20 lines only', () => {
|
||||
const lines = Array(25).fill('text').join('\n') + '\n# Too Late';
|
||||
expect(extractTitleFromHeading(lines)).toBe(null);
|
||||
});
|
||||
});
|
||||
|
||||
// ── Core inference ───────────────────────────────────────────────────
|
||||
|
||||
describe('inferFrontmatter', () => {
|
||||
test('skips files that already have frontmatter', () => {
|
||||
const result = inferFrontmatter('people/alice.md', '---\ntitle: Alice\n---\n# Alice');
|
||||
expect(result.skipped).toBe(true);
|
||||
});
|
||||
|
||||
test('Apple Notes: infers type, date, title, source', () => {
|
||||
const result = inferFrontmatter(
|
||||
'Apple Notes/2010-04-13 Apr 13 founders mtg.md',
|
||||
'<span style="color:#000ff;">Top priority</span>',
|
||||
);
|
||||
expect(result.type).toBe('apple-note');
|
||||
expect(result.date).toBe('2010-04-13');
|
||||
expect(result.title).toBe('Apr 13 founders mtg');
|
||||
expect(result.source).toBe('apple-notes');
|
||||
});
|
||||
|
||||
test('Apple Notes/YC: adds yc tag', () => {
|
||||
const result = inferFrontmatter(
|
||||
'Apple Notes/YC/2022-08-04 Project 1783Y.md',
|
||||
'Some content',
|
||||
);
|
||||
expect(result.type).toBe('apple-note');
|
||||
expect(result.tags).toContain('yc');
|
||||
expect(result.date).toBe('2022-08-04');
|
||||
});
|
||||
|
||||
test('Apple Notes/Politics: adds politics tag', () => {
|
||||
const result = inferFrontmatter(
|
||||
'Apple Notes/Politics/2023-11-15 DA race notes.md',
|
||||
'Some content',
|
||||
);
|
||||
expect(result.tags).toContain('politics');
|
||||
});
|
||||
|
||||
test('people/ directory: type person, title from heading', () => {
|
||||
const result = inferFrontmatter(
|
||||
'people/dhravya-shah.md',
|
||||
'# Dhravya Shah\n\n> Founder of Supermemory',
|
||||
);
|
||||
expect(result.type).toBe('person');
|
||||
expect(result.title).toBe('Dhravya Shah');
|
||||
});
|
||||
|
||||
test('people/ directory: falls back to filename when no heading', () => {
|
||||
const result = inferFrontmatter(
|
||||
'people/john-doe.md',
|
||||
'Some text without a heading',
|
||||
);
|
||||
expect(result.type).toBe('person');
|
||||
expect(result.title).toBe('John Doe');
|
||||
});
|
||||
|
||||
test('personal/therapy: infers therapy-session type with date', () => {
|
||||
const result = inferFrontmatter(
|
||||
'personal/therapy/jan/2024-01-30.md',
|
||||
'Session notes...',
|
||||
);
|
||||
expect(result.type).toBe('therapy-session');
|
||||
expect(result.date).toBe('2024-01-30');
|
||||
expect(result.source).toBe('therapy');
|
||||
});
|
||||
|
||||
test('personal/reflections: infers reflection type, title from heading', () => {
|
||||
const result = inferFrontmatter(
|
||||
'personal/reflections/cognitive-distortions.md',
|
||||
'# Cognitive Distortions\n\nA list of common...',
|
||||
);
|
||||
expect(result.type).toBe('reflection');
|
||||
expect(result.title).toBe('Cognitive Distortions');
|
||||
});
|
||||
|
||||
test('writing/essays: infers essay type', () => {
|
||||
const result = inferFrontmatter(
|
||||
'writing/essays/2024-03-15-on-being-remembered.md',
|
||||
'# On Being Remembered Forever\n\nSome thoughts...',
|
||||
);
|
||||
expect(result.type).toBe('essay');
|
||||
expect(result.title).toBe('On Being Remembered Forever');
|
||||
expect(result.date).toBe('2024-03-15');
|
||||
});
|
||||
|
||||
test('daily/calendar: infers calendar-index type', () => {
|
||||
const result = inferFrontmatter(
|
||||
'daily/calendar/2026-01-15-yc-office-hours.md',
|
||||
'# Calendar Index\nSome calendar data',
|
||||
);
|
||||
expect(result.type).toBe('calendar-index');
|
||||
expect(result.source).toBe('calendar');
|
||||
});
|
||||
|
||||
test('companies/ directory: type company', () => {
|
||||
const result = inferFrontmatter(
|
||||
'companies/stripe.md',
|
||||
'# Stripe\n\n> Online payments infrastructure',
|
||||
);
|
||||
expect(result.type).toBe('company');
|
||||
expect(result.title).toBe('Stripe');
|
||||
});
|
||||
|
||||
test('unknown directory: defaults to note type with heading title', () => {
|
||||
const result = inferFrontmatter(
|
||||
'random/some-file.md',
|
||||
'# My Random Notes\n\nStuff here',
|
||||
);
|
||||
expect(result.type).toBe('note');
|
||||
expect(result.title).toBe('My Random Notes');
|
||||
});
|
||||
|
||||
test('handles empty content', () => {
|
||||
const result = inferFrontmatter('notes/empty.md', '');
|
||||
expect(result.type).toBe('note');
|
||||
expect(result.title).toBe('Empty');
|
||||
});
|
||||
});
|
||||
|
||||
// ── Serialization ────────────────────────────────────────────────────
|
||||
|
||||
describe('serializeFrontmatter', () => {
|
||||
test('generates valid YAML frontmatter', () => {
|
||||
const fm = serializeFrontmatter({
|
||||
title: 'Apr 13 founders mtg',
|
||||
type: 'apple-note',
|
||||
date: '2010-04-13',
|
||||
source: 'apple-notes',
|
||||
tags: ['yc'],
|
||||
});
|
||||
expect(fm).toContain('---');
|
||||
expect(fm).toContain('title: Apr 13 founders mtg');
|
||||
expect(fm).toContain('type: apple-note');
|
||||
expect(fm).toContain('date: "2010-04-13"');
|
||||
expect(fm).toContain('source: apple-notes');
|
||||
expect(fm).toContain('tags: ["yc"]');
|
||||
});
|
||||
|
||||
test('quotes title with special chars', () => {
|
||||
const fm = serializeFrontmatter({
|
||||
title: 'What\'s the deal: a "primer"',
|
||||
type: 'note',
|
||||
});
|
||||
expect(fm).toContain('title: "What\'s the deal: a \\"primer\\""');
|
||||
});
|
||||
|
||||
test('returns empty string for skipped files', () => {
|
||||
expect(serializeFrontmatter({ title: '', type: '', skipped: true })).toBe('');
|
||||
});
|
||||
|
||||
test('omits optional fields when absent', () => {
|
||||
const fm = serializeFrontmatter({ title: 'Test', type: 'note' });
|
||||
expect(fm).not.toContain('date');
|
||||
expect(fm).not.toContain('source');
|
||||
expect(fm).not.toContain('tags');
|
||||
});
|
||||
});
|
||||
|
||||
// ── Integration ──────────────────────────────────────────────────────
|
||||
|
||||
describe('applyInference', () => {
|
||||
test('prepends frontmatter to content without it', () => {
|
||||
const { content, inferred } = applyInference(
|
||||
'people/alice-smith.md',
|
||||
'# Alice Smith\n\n> Founder of FooBar',
|
||||
);
|
||||
expect(content).toMatch(/^---\n/);
|
||||
expect(content).toContain('type: person');
|
||||
expect(content).toContain('title: Alice Smith');
|
||||
expect(content).toContain('# Alice Smith');
|
||||
expect(inferred.skipped).toBeUndefined();
|
||||
});
|
||||
|
||||
test('returns original content for files with frontmatter', () => {
|
||||
const original = '---\ntitle: Bob\n---\n# Bob';
|
||||
const { content, inferred } = applyInference('people/bob.md', original);
|
||||
expect(content).toBe(original);
|
||||
expect(inferred.skipped).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
// ── Rules coverage ───────────────────────────────────────────────────
|
||||
|
||||
describe('DIRECTORY_RULES', () => {
|
||||
test('has a catch-all rule with empty prefix', () => {
|
||||
const catchAll = DIRECTORY_RULES.find(r => r.pathPrefix === '');
|
||||
expect(catchAll).toBeDefined();
|
||||
expect(catchAll!.type).toBe('note');
|
||||
});
|
||||
|
||||
test('Apple Notes rules are more specific than the catch-all', () => {
|
||||
const appleRules = DIRECTORY_RULES.filter(r => r.pathPrefix.startsWith('apple notes/'));
|
||||
expect(appleRules.length).toBeGreaterThan(1); // subfolder rules + catch-all
|
||||
// Subfolder rules should come before the generic apple notes/ rule
|
||||
const ycIdx = DIRECTORY_RULES.findIndex(r => r.pathPrefix === 'apple notes/yc/');
|
||||
const genericIdx = DIRECTORY_RULES.findIndex(r => r.pathPrefix === 'apple notes/');
|
||||
expect(ycIdx).toBeLessThan(genericIdx);
|
||||
});
|
||||
});
|
||||
@@ -1,141 +0,0 @@
|
||||
/**
|
||||
* Hermeticity test: every site that writes under `~/.gbrain` must honor
|
||||
* `GBRAIN_HOME=<tmp>` and write under `<tmp>/.gbrain` instead of the developer's
|
||||
* real home.
|
||||
*
|
||||
* Why this exists: `src/core/config.ts::configDir()` already supports
|
||||
* `GBRAIN_HOME` as a parent-dir override (returns `<override>/.gbrain`), but
|
||||
* historically many call sites built paths from `os.homedir()` directly,
|
||||
* bypassing the override. The hermeticity migration migrated every write-side
|
||||
* caller to `gbrainPath(...)`. This test is the regression gate.
|
||||
*
|
||||
* Scope: write-isolation only. Read-side host detection in
|
||||
* `src/commands/init.ts` (reading `~/.claude`, `~/.openclaw`, etc. for module
|
||||
* fingerprinting) is the documented v1 caveat and is NOT asserted here.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { mkdtempSync, existsSync, readdirSync, statSync, rmSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
|
||||
// Save original env so we don't leak between tests.
|
||||
const ORIG_GBRAIN_HOME = process.env.GBRAIN_HOME;
|
||||
|
||||
function fresh(): string {
|
||||
return mkdtempSync(join(tmpdir(), 'gbrain-home-isolation-'));
|
||||
}
|
||||
|
||||
describe('GBRAIN_HOME write-side isolation', () => {
|
||||
test('configDir() returns <GBRAIN_HOME>/.gbrain when override is set', async () => {
|
||||
const tmp = fresh();
|
||||
process.env.GBRAIN_HOME = tmp;
|
||||
try {
|
||||
const { configDir, gbrainPath } = await import('../src/core/config.ts');
|
||||
expect(configDir()).toBe(join(tmp, '.gbrain'));
|
||||
expect(gbrainPath('foo', 'bar.json')).toBe(join(tmp, '.gbrain', 'foo', 'bar.json'));
|
||||
} finally {
|
||||
process.env.GBRAIN_HOME = ORIG_GBRAIN_HOME;
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('configDir() falls back to homedir when GBRAIN_HOME unset', async () => {
|
||||
delete process.env.GBRAIN_HOME;
|
||||
try {
|
||||
const { configDir } = await import('../src/core/config.ts');
|
||||
const result = configDir();
|
||||
// Should NOT contain the test tmpdir; should resolve to a real homedir path.
|
||||
expect(result.endsWith('.gbrain')).toBe(true);
|
||||
expect(result.startsWith('/tmp/')).toBe(false);
|
||||
} finally {
|
||||
if (ORIG_GBRAIN_HOME !== undefined) process.env.GBRAIN_HOME = ORIG_GBRAIN_HOME;
|
||||
}
|
||||
});
|
||||
|
||||
test('rejects relative GBRAIN_HOME', async () => {
|
||||
process.env.GBRAIN_HOME = 'relative/path';
|
||||
try {
|
||||
const { configDir } = await import('../src/core/config.ts');
|
||||
expect(() => configDir()).toThrow(/absolute path/);
|
||||
} finally {
|
||||
process.env.GBRAIN_HOME = ORIG_GBRAIN_HOME;
|
||||
}
|
||||
});
|
||||
|
||||
test("rejects GBRAIN_HOME containing '..' segments", async () => {
|
||||
process.env.GBRAIN_HOME = '/tmp/foo/../bar';
|
||||
try {
|
||||
const { configDir } = await import('../src/core/config.ts');
|
||||
expect(() => configDir()).toThrow(/'\.\.' segments/);
|
||||
} finally {
|
||||
process.env.GBRAIN_HOME = ORIG_GBRAIN_HOME;
|
||||
}
|
||||
});
|
||||
|
||||
test('saveConfig/loadConfig honor GBRAIN_HOME', async () => {
|
||||
const tmp = fresh();
|
||||
process.env.GBRAIN_HOME = tmp;
|
||||
try {
|
||||
const { saveConfig, loadConfig } = await import('../src/core/config.ts');
|
||||
const cfg = { engine: 'pglite' as const, database_path: join(tmp, '.gbrain', 'brain.pglite') };
|
||||
saveConfig(cfg);
|
||||
// Config file should exist under the override, NOT under real ~/.gbrain.
|
||||
expect(existsSync(join(tmp, '.gbrain', 'config.json'))).toBe(true);
|
||||
|
||||
// Round-trip: loadConfig() finds it back via the override.
|
||||
const loaded = loadConfig();
|
||||
expect(loaded?.engine).toBe('pglite');
|
||||
expect(loaded?.database_path).toBe(cfg.database_path);
|
||||
} finally {
|
||||
process.env.GBRAIN_HOME = ORIG_GBRAIN_HOME;
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('integrity, sync-failures, integrations heartbeat resolve under GBRAIN_HOME', async () => {
|
||||
const tmp = fresh();
|
||||
process.env.GBRAIN_HOME = tmp;
|
||||
try {
|
||||
const { gbrainPath } = await import('../src/core/config.ts');
|
||||
// Spot-check a representative set of paths used across the migrated sites.
|
||||
const paths = [
|
||||
gbrainPath('integrity-review.md'), // src/commands/integrity.ts
|
||||
gbrainPath('sync-failures.jsonl'), // src/core/sync.ts
|
||||
gbrainPath('integrations', 'recipe-x'), // src/commands/integrations.ts
|
||||
gbrainPath('migrate-manifest.json'), // src/commands/migrate-engine.ts
|
||||
gbrainPath('import-checkpoint.json'), // src/commands/import.ts
|
||||
gbrainPath('migrations', 'v0_13_1-rollback.jsonl'), // src/commands/migrations/v0_13_1.ts
|
||||
gbrainPath('migrations', 'pending-host-work.jsonl'), // src/commands/migrations/v0_14_0.ts
|
||||
gbrainPath('audit'), // shell-audit / backpressure-audit
|
||||
gbrainPath('cycle.lock'), // src/core/cycle.ts
|
||||
gbrainPath('fail-improve'), // src/core/fail-improve.ts
|
||||
gbrainPath('validator-lint.jsonl'), // src/core/output/post-write.ts
|
||||
gbrainPath('brain.pglite'), // init pglite default
|
||||
];
|
||||
for (const p of paths) {
|
||||
expect(p.startsWith(join(tmp, '.gbrain'))).toBe(true);
|
||||
}
|
||||
} finally {
|
||||
process.env.GBRAIN_HOME = ORIG_GBRAIN_HOME;
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('GBRAIN_AUDIT_DIR override still wins over GBRAIN_HOME', async () => {
|
||||
const tmp = fresh();
|
||||
const auditTmp = fresh();
|
||||
process.env.GBRAIN_HOME = tmp;
|
||||
process.env.GBRAIN_AUDIT_DIR = auditTmp;
|
||||
try {
|
||||
const { resolveAuditDir } = await import('../src/core/minions/handlers/shell-audit.ts');
|
||||
// Per the docstring: GBRAIN_AUDIT_DIR is the explicit override and wins.
|
||||
expect(resolveAuditDir()).toBe(auditTmp);
|
||||
} finally {
|
||||
process.env.GBRAIN_HOME = ORIG_GBRAIN_HOME;
|
||||
delete process.env.GBRAIN_AUDIT_DIR;
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
rmSync(auditTmp, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -16,17 +16,16 @@ import { join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
|
||||
let tmpHome: string;
|
||||
const originalGbrainHome = process.env.GBRAIN_HOME;
|
||||
const originalHome = process.env.HOME;
|
||||
|
||||
beforeEach(() => {
|
||||
tmpHome = mkdtempSync(join(tmpdir(), 'gbrain-v0_14_0-'));
|
||||
// GBRAIN_HOME is the parent dir; configDir() appends '.gbrain' itself.
|
||||
process.env.GBRAIN_HOME = tmpHome;
|
||||
process.env.HOME = tmpHome;
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
if (originalGbrainHome !== undefined) process.env.GBRAIN_HOME = originalGbrainHome;
|
||||
else delete process.env.GBRAIN_HOME;
|
||||
if (originalHome) process.env.HOME = originalHome;
|
||||
else delete process.env.HOME;
|
||||
try { rmSync(tmpHome, { recursive: true, force: true }); } catch { /* ignore */ }
|
||||
});
|
||||
|
||||
|
||||
@@ -2306,379 +2306,3 @@ describe('checkAborted (v0.20.5 cycle signal)', () => {
|
||||
}).toThrow('aborted between phases: timeout');
|
||||
});
|
||||
});
|
||||
|
||||
// --- v0.22.14: Self-health-check for bare workers ---
|
||||
|
||||
describe('MinionWorker: self-health-check', () => {
|
||||
test('health check is active when GBRAIN_SUPERVISED is not set', async () => {
|
||||
// Save and clear the env var
|
||||
const saved = process.env.GBRAIN_SUPERVISED;
|
||||
delete process.env.GBRAIN_SUPERVISED;
|
||||
|
||||
try {
|
||||
const worker = new MinionWorker(engine, {
|
||||
queue: 'default',
|
||||
concurrency: 1,
|
||||
healthCheckInterval: 100, // fast for testing
|
||||
pollInterval: 50,
|
||||
stalledInterval: 10_000,
|
||||
maxRssMb: 0,
|
||||
});
|
||||
|
||||
worker.register('noop', async () => {});
|
||||
await queue.add('noop', {});
|
||||
|
||||
const startPromise = worker.start();
|
||||
// Let the health check fire at least once (100ms interval)
|
||||
await new Promise(r => setTimeout(r, 300));
|
||||
worker.stop();
|
||||
await startPromise;
|
||||
|
||||
// Worker should have processed the job despite health check running
|
||||
const completed = await queue.getJobs({ status: 'completed' });
|
||||
expect(completed.length).toBeGreaterThanOrEqual(1);
|
||||
} finally {
|
||||
if (saved !== undefined) process.env.GBRAIN_SUPERVISED = saved;
|
||||
else delete process.env.GBRAIN_SUPERVISED;
|
||||
}
|
||||
}, 10_000);
|
||||
|
||||
test('health check is skipped when GBRAIN_SUPERVISED=1', async () => {
|
||||
const saved = process.env.GBRAIN_SUPERVISED;
|
||||
process.env.GBRAIN_SUPERVISED = '1';
|
||||
|
||||
try {
|
||||
const worker = new MinionWorker(engine, {
|
||||
queue: 'default',
|
||||
concurrency: 1,
|
||||
healthCheckInterval: 100,
|
||||
pollInterval: 50,
|
||||
stalledInterval: 10_000,
|
||||
maxRssMb: 0,
|
||||
});
|
||||
|
||||
worker.register('noop', async () => {});
|
||||
await queue.add('noop', {});
|
||||
|
||||
const startPromise = worker.start();
|
||||
await new Promise(r => setTimeout(r, 300));
|
||||
worker.stop();
|
||||
await startPromise;
|
||||
|
||||
// Worker should still process jobs fine
|
||||
const completed = await queue.getJobs({ status: 'completed' });
|
||||
expect(completed.length).toBeGreaterThanOrEqual(1);
|
||||
} finally {
|
||||
if (saved !== undefined) process.env.GBRAIN_SUPERVISED = saved;
|
||||
else delete process.env.GBRAIN_SUPERVISED;
|
||||
}
|
||||
}, 10_000);
|
||||
|
||||
test('healthCheckInterval=0 disables health check', async () => {
|
||||
delete process.env.GBRAIN_SUPERVISED;
|
||||
|
||||
const worker = new MinionWorker(engine, {
|
||||
queue: 'default',
|
||||
concurrency: 1,
|
||||
healthCheckInterval: 0,
|
||||
pollInterval: 50,
|
||||
stalledInterval: 10_000,
|
||||
maxRssMb: 0,
|
||||
});
|
||||
|
||||
worker.register('noop', async () => {});
|
||||
await queue.add('noop', {});
|
||||
|
||||
const startPromise = worker.start();
|
||||
await new Promise(r => setTimeout(r, 300));
|
||||
worker.stop();
|
||||
await startPromise;
|
||||
|
||||
const completed = await queue.getJobs({ status: 'completed' });
|
||||
expect(completed.length).toBeGreaterThanOrEqual(1);
|
||||
}, 10_000);
|
||||
});
|
||||
|
||||
// --- v0.22.14: Self-health-check behavior tests (D7) ---
|
||||
// These tests use a Proxy around the real engine so executeRaw can be
|
||||
// intercepted by SQL pattern. SELECT 1 = liveness probe; the count(*) query
|
||||
// = stall detection. Anything else passes through to the underlying engine.
|
||||
|
||||
interface ProbeOverrides {
|
||||
/** When set, executeRaw('SELECT 1') uses this function instead of pass-through.
|
||||
* Returning a thrown error simulates DB death; returning [{}] simulates success. */
|
||||
selectOne?: () => Promise<unknown>;
|
||||
/** When set, executeRaw of the stall-detection count(*) query returns this. */
|
||||
countWaiting?: (handlers: string[]) => number;
|
||||
/** Captures the last SQL string that matched the stall-count regex. Tests
|
||||
* use this to assert the production SQL still contains `name = ANY(...)`
|
||||
* so a future refactor that drops the predicate is caught. */
|
||||
capturedStallSql?: { sql: string | null };
|
||||
}
|
||||
|
||||
function makeProbeEngine(overrides: ProbeOverrides) {
|
||||
return new Proxy(engine, {
|
||||
get(target, prop, receiver) {
|
||||
if (prop === 'executeRaw') {
|
||||
return async (sql: string, params?: unknown[]): Promise<unknown[]> => {
|
||||
if (overrides.selectOne && /^\s*SELECT\s+1\s*$/i.test(sql)) {
|
||||
const r = await overrides.selectOne();
|
||||
return Array.isArray(r) ? r : [r];
|
||||
}
|
||||
if (overrides.countWaiting && /count\(\*\).*minion_jobs.*WHERE\s+status\s*=\s*'waiting'/is.test(sql)) {
|
||||
if (overrides.capturedStallSql) overrides.capturedStallSql.sql = sql;
|
||||
const handlers = (params?.[1] as string[]) ?? [];
|
||||
return [{ cnt: String(overrides.countWaiting(handlers)) }];
|
||||
}
|
||||
// Pass through to real engine for anything else (claim queries etc.)
|
||||
return (target as unknown as { executeRaw: (s: string, p?: unknown[]) => Promise<unknown[]> })
|
||||
.executeRaw(sql, params);
|
||||
};
|
||||
}
|
||||
return Reflect.get(target, prop, receiver);
|
||||
},
|
||||
}) as unknown as PGLiteEngine;
|
||||
}
|
||||
|
||||
describe('MinionWorker: self-health-check behavior (v0.22.14)', () => {
|
||||
test('emits unhealthy{db_dead} after dbFailExitAfter consecutive DB probe failures', async () => {
|
||||
delete process.env.GBRAIN_SUPERVISED;
|
||||
|
||||
let probeCount = 0;
|
||||
const probeEngine = makeProbeEngine({
|
||||
selectOne: async () => {
|
||||
probeCount++;
|
||||
throw new Error('connection terminated unexpectedly');
|
||||
},
|
||||
});
|
||||
|
||||
const worker = new MinionWorker(probeEngine, {
|
||||
queue: 'default',
|
||||
concurrency: 1,
|
||||
healthCheckInterval: 30,
|
||||
dbFailExitAfter: 3,
|
||||
pollInterval: 50,
|
||||
stalledInterval: 10_000,
|
||||
maxRssMb: 0,
|
||||
});
|
||||
|
||||
worker.register('noop', async () => {});
|
||||
|
||||
const events: Array<{ reason: string }> = [];
|
||||
worker.on('unhealthy', (info) => { events.push(info); });
|
||||
|
||||
const startPromise = worker.start();
|
||||
// 3 ticks at 30ms = 90ms; give extra slack.
|
||||
await new Promise(r => setTimeout(r, 250));
|
||||
worker.stop();
|
||||
await startPromise;
|
||||
|
||||
expect(probeCount).toBeGreaterThanOrEqual(3);
|
||||
expect(events.length).toBeGreaterThanOrEqual(1);
|
||||
expect(events[0].reason).toBe('db_dead');
|
||||
}, 10_000);
|
||||
|
||||
test('DB recovery resets the failure counter (no exit after intermittent failures)', async () => {
|
||||
delete process.env.GBRAIN_SUPERVISED;
|
||||
|
||||
let probeCount = 0;
|
||||
// Pattern: fail, fail, succeed (resets), fail, fail, then permanently succeed.
|
||||
// No 3 consecutive failures, so dbFailExitAfter=3 must NOT trip.
|
||||
const probeEngine = makeProbeEngine({
|
||||
selectOne: async () => {
|
||||
const idx = probeCount++;
|
||||
if (idx === 0 || idx === 1 || idx === 3 || idx === 4) {
|
||||
throw new Error('transient blip');
|
||||
}
|
||||
return [{ ok: 1 }];
|
||||
},
|
||||
});
|
||||
|
||||
const worker = new MinionWorker(probeEngine, {
|
||||
queue: 'default',
|
||||
concurrency: 1,
|
||||
healthCheckInterval: 30,
|
||||
dbFailExitAfter: 3,
|
||||
pollInterval: 50,
|
||||
stalledInterval: 10_000,
|
||||
maxRssMb: 0,
|
||||
});
|
||||
|
||||
worker.register('noop', async () => {});
|
||||
|
||||
const events: Array<{ reason: string }> = [];
|
||||
worker.on('unhealthy', (info) => { events.push(info); });
|
||||
|
||||
const startPromise = worker.start();
|
||||
await new Promise(r => setTimeout(r, 250));
|
||||
worker.stop();
|
||||
await startPromise;
|
||||
|
||||
// Counter should never have hit 3 consecutive — success at index 2 resets it.
|
||||
const dbDeadEvents = events.filter(e => e.reason === 'db_dead');
|
||||
expect(dbDeadEvents.length).toBe(0);
|
||||
}, 10_000);
|
||||
|
||||
test('emits unhealthy{stalled} after stallExitAfterMs of continuous idle with waiting jobs', async () => {
|
||||
delete process.env.GBRAIN_SUPERVISED;
|
||||
|
||||
const probeEngine = makeProbeEngine({
|
||||
selectOne: async () => [{ ok: 1 }],
|
||||
countWaiting: () => 5, // pretend 5 jobs are waiting for our handler names
|
||||
});
|
||||
|
||||
const worker = new MinionWorker(probeEngine, {
|
||||
queue: 'default',
|
||||
concurrency: 1,
|
||||
healthCheckInterval: 30,
|
||||
stallWarnAfterMs: 50,
|
||||
stallExitAfterMs: 100,
|
||||
pollInterval: 50,
|
||||
stalledInterval: 10_000,
|
||||
maxRssMb: 0,
|
||||
});
|
||||
|
||||
worker.register('noop', async () => {});
|
||||
// Don't queue any real jobs — claim returns null, inFlight stays 0,
|
||||
// jobsCompleted stays 0, idle clock advances.
|
||||
|
||||
const events: Array<{ reason: string; waitingCount?: number }> = [];
|
||||
worker.on('unhealthy', (info) => { events.push(info); });
|
||||
|
||||
const startPromise = worker.start();
|
||||
// Both thresholds measured from lastCompletionTime (corrected per codex r2):
|
||||
// - tick @ +30ms: idle=30ms, < stallWarnAfterMs(50), no warn
|
||||
// - tick @ +60ms: idle=60ms, > 50, warn fires (stallWarningSince set)
|
||||
// - tick @ +90ms: idle=90ms, < stallExitAfterMs(100), no exit yet
|
||||
// - tick @ +120ms: idle=120ms, > 100 → exit fires (unhealthy event)
|
||||
// Wait 350ms which leaves comfortable slack for setTimeout drift.
|
||||
await new Promise(r => setTimeout(r, 350));
|
||||
worker.stop();
|
||||
await startPromise;
|
||||
|
||||
const stalledEvents = events.filter(e => e.reason === 'stalled');
|
||||
expect(stalledEvents.length).toBeGreaterThanOrEqual(1);
|
||||
expect(stalledEvents[0].waitingCount).toBe(5);
|
||||
// The idleMinutes payload should reflect total idle, not warn-since.
|
||||
// With idle ~120ms at exit time, idleMinutes rounds to 0 — that's
|
||||
// expected; the value is informative, not load-bearing.
|
||||
}, 10_000);
|
||||
|
||||
test('inFlight > 0 blocks stall detection (long-running legitimate job)', async () => {
|
||||
delete process.env.GBRAIN_SUPERVISED;
|
||||
|
||||
const probeEngine = makeProbeEngine({
|
||||
selectOne: async () => [{ ok: 1 }],
|
||||
countWaiting: () => 5,
|
||||
});
|
||||
|
||||
const worker = new MinionWorker(probeEngine, {
|
||||
queue: 'default',
|
||||
concurrency: 1,
|
||||
healthCheckInterval: 30,
|
||||
stallWarnAfterMs: 50,
|
||||
stallExitAfterMs: 100,
|
||||
pollInterval: 50,
|
||||
stalledInterval: 10_000,
|
||||
maxRssMb: 0,
|
||||
});
|
||||
|
||||
worker.register('noop', async () => {});
|
||||
|
||||
const events: Array<{ reason: string }> = [];
|
||||
worker.on('unhealthy', (info) => { events.push(info); });
|
||||
|
||||
// Inject a fake in-flight entry directly. This bypasses the claim path
|
||||
// (which goes through the proxy and complicates the cleanup race) and
|
||||
// tests exactly what we want: the stall check's `inFlight.size === 0`
|
||||
// gate when there's legitimate ongoing work.
|
||||
const fakeInFlight = (worker as unknown as {
|
||||
inFlight: Map<number, { lockTimer: NodeJS.Timeout; abort: AbortController; promise: Promise<void> }>
|
||||
}).inFlight;
|
||||
const fakeAbort = new AbortController();
|
||||
const fakePromise = new Promise<void>(() => { /* never resolves */ });
|
||||
const fakeTimer = setInterval(() => {}, 60_000); // dummy lock timer
|
||||
fakeInFlight.set(99999, { lockTimer: fakeTimer, abort: fakeAbort, promise: fakePromise });
|
||||
|
||||
const startPromise = worker.start();
|
||||
await new Promise(r => setTimeout(r, 350));
|
||||
// Remove our fake entry before stop so the worker doesn't wait 30s for it.
|
||||
clearInterval(fakeTimer);
|
||||
fakeInFlight.delete(99999);
|
||||
worker.stop();
|
||||
await startPromise;
|
||||
|
||||
// No stall event should fire — inFlight.size > 0 gates the stall check.
|
||||
const stalledEvents = events.filter(e => e.reason === 'stalled');
|
||||
expect(stalledEvents.length).toBe(0);
|
||||
}, 10_000);
|
||||
|
||||
test('regression (D1): waiting jobs of unregistered handler names do NOT trigger stall exit', async () => {
|
||||
delete process.env.GBRAIN_SUPERVISED;
|
||||
|
||||
// The count(*) query is filtered by registered handler names. If handlers=['noop']
|
||||
// and the queue has 5 'widget-fn' jobs, the SQL `name = ANY($2)` filter returns 0.
|
||||
// The probe engine simulates this by checking handlers before returning a count;
|
||||
// we ALSO capture the SQL to assert the predicate text is actually present (so a
|
||||
// future refactor that silently drops `AND name = ANY(...)` is caught).
|
||||
const capturedStallSql = { sql: null as string | null };
|
||||
const probeEngine = makeProbeEngine({
|
||||
selectOne: async () => [{ ok: 1 }],
|
||||
countWaiting: (handlers) => handlers.includes('widget-fn') ? 5 : 0,
|
||||
capturedStallSql,
|
||||
});
|
||||
|
||||
const worker = new MinionWorker(probeEngine, {
|
||||
queue: 'default',
|
||||
concurrency: 1,
|
||||
healthCheckInterval: 50,
|
||||
stallWarnAfterMs: 100,
|
||||
stallExitAfterMs: 200,
|
||||
pollInterval: 50,
|
||||
stalledInterval: 10_000,
|
||||
maxRssMb: 0,
|
||||
});
|
||||
|
||||
// Register 'noop' but pretend the queue is full of 'widget-fn' (unhandled).
|
||||
worker.register('noop', async () => {});
|
||||
|
||||
const events: Array<{ reason: string }> = [];
|
||||
worker.on('unhealthy', (info) => { events.push(info); });
|
||||
|
||||
const startPromise = worker.start();
|
||||
// Window > stallExitAfterMs; if D1 fix wasn't applied, stall would fire.
|
||||
await new Promise(r => setTimeout(r, 500));
|
||||
worker.stop();
|
||||
await startPromise;
|
||||
|
||||
// No stall event — the count for 'noop' handlers is 0, so worker is correctly idle.
|
||||
const stalledEvents = events.filter(e => e.reason === 'stalled');
|
||||
expect(stalledEvents.length).toBe(0);
|
||||
// SQL shape assertion: the production query MUST filter by handler names.
|
||||
// Without this assertion, a future change that drops the predicate would
|
||||
// pass the no-event check above (the handler array would be irrelevant
|
||||
// to the underlying DB but our probe just needs to return 0).
|
||||
expect(capturedStallSql.sql).not.toBeNull();
|
||||
expect(capturedStallSql.sql).toMatch(/name\s*=\s*ANY/i);
|
||||
}, 10_000);
|
||||
|
||||
test('regression (R3): constructor throws when stallExitAfterMs <= stallWarnAfterMs', () => {
|
||||
// The contract on MinionWorkerOpts.stallExitAfterMs says "Must be >
|
||||
// stallWarnAfterMs". Without validation, an exit threshold equal to or
|
||||
// less than the warn threshold made the configured exit time a lie
|
||||
// (warn fires first, exit can't preempt). The constructor now throws
|
||||
// loudly so misconfigurations fail at startup, not at idle-time.
|
||||
expect(() => new MinionWorker(engine, {
|
||||
stallWarnAfterMs: 200,
|
||||
stallExitAfterMs: 100, // less than warn — invalid
|
||||
})).toThrow(/stallExitAfterMs.*must be > stallWarnAfterMs/i);
|
||||
|
||||
expect(() => new MinionWorker(engine, {
|
||||
stallWarnAfterMs: 100,
|
||||
stallExitAfterMs: 100, // equal to warn — also invalid (must be strictly >)
|
||||
})).toThrow(/stallExitAfterMs.*must be > stallWarnAfterMs/i);
|
||||
|
||||
// Sanity: defaults (5min warn / 10min exit) construct without throwing.
|
||||
expect(() => new MinionWorker(engine, {})).not.toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,163 +0,0 @@
|
||||
/**
|
||||
* IRON RULE security regression guard for the v0.21 trusted-workspace
|
||||
* allow-list path on put_page.
|
||||
*
|
||||
* Covers:
|
||||
* - matchesSlugAllowList glob semantics (ALLOW + REJECT + recursive globs)
|
||||
* - put_page accepts when slug matches allow-list
|
||||
* - put_page rejects when slug is outside allow-list
|
||||
* - put_page falls back to legacy `wiki/agents/<id>/...` namespace check
|
||||
* when allowed_slug_prefixes is unset (regression guard for v0.15
|
||||
* anti-prompt-injection guarantee)
|
||||
* - put_page rejects when viaSubagent=true but subagentId is missing
|
||||
* (regression guard for FAIL-CLOSED behavior)
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { matchesSlugAllowList, operations, OperationError, type OperationContext } from '../src/core/operations.ts';
|
||||
|
||||
const STUB_LOGGER = {
|
||||
info: () => {},
|
||||
warn: () => {},
|
||||
error: () => {},
|
||||
};
|
||||
|
||||
const STUB_CONFIG = {} as unknown as Parameters<typeof operations[number]['handler']>[0]['config'];
|
||||
|
||||
function findOp(name: string) {
|
||||
const op = operations.find(o => o.name === name);
|
||||
if (!op) throw new Error(`operation ${name} not found`);
|
||||
return op;
|
||||
}
|
||||
|
||||
// Stub engine that fails loudly if put_page actually reaches importFromContent.
|
||||
// We expect every test in this file to short-circuit at the namespace/allow-list
|
||||
// check, so every engine method throws a recognizable error that lets us assert
|
||||
// "got past the gate" if it ever happens.
|
||||
function stubEngine() {
|
||||
return new Proxy({} as never, {
|
||||
get(_target, prop: string) {
|
||||
return () => { throw new Error(`engine.${prop} should not have been called — gate failed`); };
|
||||
},
|
||||
}) as Parameters<typeof operations[number]['handler']>[0]['engine'];
|
||||
}
|
||||
|
||||
function makeCtx(overrides: Partial<OperationContext> = {}): OperationContext {
|
||||
return {
|
||||
engine: stubEngine(),
|
||||
config: STUB_CONFIG,
|
||||
logger: STUB_LOGGER,
|
||||
dryRun: false,
|
||||
remote: true,
|
||||
viaSubagent: true,
|
||||
subagentId: 42,
|
||||
jobId: 100,
|
||||
...overrides,
|
||||
} as OperationContext;
|
||||
}
|
||||
|
||||
describe('matchesSlugAllowList — glob semantics', () => {
|
||||
test('exact match (no glob suffix)', () => {
|
||||
expect(matchesSlugAllowList('foo/bar', ['foo/bar'])).toBe(true);
|
||||
expect(matchesSlugAllowList('foo/bar/baz', ['foo/bar'])).toBe(false);
|
||||
});
|
||||
|
||||
test('shallow glob: prefix/* matches any single direct child segment', () => {
|
||||
expect(matchesSlugAllowList('wiki/personal/reflections/2026-04-25-arete-paradox-a3f8c1',
|
||||
['wiki/personal/reflections/*'])).toBe(true);
|
||||
expect(matchesSlugAllowList('wiki/personal/reflections',
|
||||
['wiki/personal/reflections/*'])).toBe(false);
|
||||
});
|
||||
|
||||
test('recursive: prefix/* matches deep children too', () => {
|
||||
expect(matchesSlugAllowList('wiki/originals/ideas/2026-04-25-foo',
|
||||
['wiki/originals/*'])).toBe(true);
|
||||
expect(matchesSlugAllowList('wiki/originals/ideas/foo/bar',
|
||||
['wiki/originals/*'])).toBe(true);
|
||||
});
|
||||
|
||||
test('rejects slugs outside every prefix', () => {
|
||||
const list = [
|
||||
'wiki/personal/reflections/*',
|
||||
'wiki/originals/*',
|
||||
];
|
||||
expect(matchesSlugAllowList('wiki/finance/secret', list)).toBe(false);
|
||||
expect(matchesSlugAllowList('wiki/people/alice', list)).toBe(false);
|
||||
});
|
||||
|
||||
test('empty list rejects everything', () => {
|
||||
expect(matchesSlugAllowList('wiki/anything', [])).toBe(false);
|
||||
});
|
||||
|
||||
test('does NOT match prefix without trailing segment', () => {
|
||||
expect(matchesSlugAllowList('wiki/personal/reflections',
|
||||
['wiki/personal/reflections/*'])).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('put_page — trusted-workspace allow-list', () => {
|
||||
const put_page = findOp('put_page');
|
||||
|
||||
test('REJECTS when slug is outside the allow-list', async () => {
|
||||
const ctx = makeCtx({
|
||||
allowedSlugPrefixes: ['wiki/personal/reflections/*', 'wiki/originals/*'],
|
||||
});
|
||||
await expect(put_page.handler(ctx, {
|
||||
slug: 'wiki/finance/secret',
|
||||
content: '---\ntitle: x\n---\nbody',
|
||||
})).rejects.toMatchObject({
|
||||
code: 'permission_denied',
|
||||
});
|
||||
});
|
||||
|
||||
test('REJECTS path-traversal-like slug (slug regex catches it earlier in the import path; allow-list also catches via no-match)', async () => {
|
||||
const ctx = makeCtx({
|
||||
allowedSlugPrefixes: ['wiki/personal/reflections/*'],
|
||||
});
|
||||
// The slug regex in validatePageSlug rejects `..`; here we test the
|
||||
// allow-list layer specifically with a slug that LOOKS legal but isn't on the list.
|
||||
await expect(put_page.handler(ctx, {
|
||||
slug: 'wiki/people/garry-tan',
|
||||
content: '---\ntitle: x\n---\nbody',
|
||||
})).rejects.toMatchObject({
|
||||
code: 'permission_denied',
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('put_page — legacy namespace check (regression guard)', () => {
|
||||
const put_page = findOp('put_page');
|
||||
|
||||
test('REJECTS write outside wiki/agents/<id>/ when allow-list is unset', async () => {
|
||||
// The v0.15 anti-prompt-injection guarantee: subagent without explicit
|
||||
// allow-list MUST be confined to its own agent namespace. This test
|
||||
// ensures v0.21 doesn't regress that boundary.
|
||||
const ctx = makeCtx({ allowedSlugPrefixes: undefined });
|
||||
await expect(put_page.handler(ctx, {
|
||||
slug: 'wiki/personal/reflections/2026-04-25-foo',
|
||||
content: '---\ntitle: x\n---\nbody',
|
||||
})).rejects.toMatchObject({
|
||||
code: 'permission_denied',
|
||||
});
|
||||
});
|
||||
|
||||
test('REJECTS write outside wiki/agents/<id>/ when allow-list is empty array', async () => {
|
||||
const ctx = makeCtx({ allowedSlugPrefixes: [] });
|
||||
await expect(put_page.handler(ctx, {
|
||||
slug: 'wiki/personal/reflections/2026-04-25-foo',
|
||||
content: '---\ntitle: x\n---\nbody',
|
||||
})).rejects.toMatchObject({
|
||||
code: 'permission_denied',
|
||||
});
|
||||
});
|
||||
|
||||
test('REJECTS when viaSubagent=true but subagentId is missing (FAIL-CLOSED)', async () => {
|
||||
const ctx = makeCtx({ subagentId: undefined as unknown as number, allowedSlugPrefixes: undefined });
|
||||
await expect(put_page.handler(ctx, {
|
||||
slug: 'wiki/agents/42/foo',
|
||||
content: '---\ntitle: x\n---\nbody',
|
||||
})).rejects.toMatchObject({
|
||||
code: 'permission_denied',
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -1,81 +0,0 @@
|
||||
/**
|
||||
* progress-tail tests — parse --progress-json events out of mixed stderr.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { parseProgressEvents, eventsByPhase, verifyExpectedPhases } from '../src/core/claw-test/progress-tail.ts';
|
||||
|
||||
describe('parseProgressEvents', () => {
|
||||
test('extracts JSON event lines from mixed stderr', () => {
|
||||
const stderr = [
|
||||
'starting up',
|
||||
'{"phase":"import.files","event":"start"}',
|
||||
'warning: deprecated flag X',
|
||||
'{"phase":"import.files","event":"tick","done":3,"total":10}',
|
||||
'random text',
|
||||
'{"phase":"import.files","event":"finish"}',
|
||||
].join('\n');
|
||||
const events = parseProgressEvents(stderr);
|
||||
expect(events).toHaveLength(3);
|
||||
expect(events.map(e => e.event)).toEqual(['start', 'tick', 'finish']);
|
||||
});
|
||||
|
||||
test('ignores malformed JSON lines silently', () => {
|
||||
const stderr = [
|
||||
'{"phase":"a","event":"start"}',
|
||||
'{"phase":', // truncated JSON
|
||||
'not json at all',
|
||||
'{"phase":"b","event":"start"}',
|
||||
].join('\n');
|
||||
const events = parseProgressEvents(stderr);
|
||||
expect(events).toHaveLength(2);
|
||||
});
|
||||
|
||||
test('ignores objects without phase field', () => {
|
||||
const stderr = [
|
||||
'{"phase":"a","event":"start"}',
|
||||
'{"foo":"bar"}',
|
||||
'{"phase":"b","event":"start"}',
|
||||
].join('\n');
|
||||
const events = parseProgressEvents(stderr);
|
||||
expect(events.map(e => e.phase)).toEqual(['a', 'b']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('eventsByPhase', () => {
|
||||
test('groups by phase name', () => {
|
||||
const events = [
|
||||
{ phase: 'import.files', event: 'start' },
|
||||
{ phase: 'import.files', event: 'finish' },
|
||||
{ phase: 'extract.links_fs', event: 'start' },
|
||||
];
|
||||
const grouped = eventsByPhase(events);
|
||||
expect(grouped.get('import.files')).toHaveLength(2);
|
||||
expect(grouped.get('extract.links_fs')).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('verifyExpectedPhases', () => {
|
||||
test('returns empty when all expected phases present', () => {
|
||||
const events = [
|
||||
{ phase: 'import.files' },
|
||||
{ phase: 'extract.links_fs' },
|
||||
{ phase: 'doctor.db_checks' },
|
||||
];
|
||||
const missing = verifyExpectedPhases(events, ['import.files', 'doctor.db_checks']);
|
||||
expect(missing).toEqual([]);
|
||||
});
|
||||
|
||||
test('returns missing phase names when some are absent', () => {
|
||||
const events = [
|
||||
{ phase: 'import.files' },
|
||||
];
|
||||
const missing = verifyExpectedPhases(events, ['import.files', 'extract.links_fs', 'doctor.db_checks']);
|
||||
expect(missing).toEqual(['extract.links_fs', 'doctor.db_checks']);
|
||||
});
|
||||
|
||||
test('returns full expected list when no events at all', () => {
|
||||
const missing = verifyExpectedPhases([], ['a', 'b']);
|
||||
expect(missing).toEqual(['a', 'b']);
|
||||
});
|
||||
});
|
||||
@@ -1,132 +0,0 @@
|
||||
/**
|
||||
* Scenario loader tests — proves scenario.json parsing + validation work.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { mkdtempSync, rmSync, mkdirSync, writeFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
import { listScenarios, loadScenario, readBrief } from '../src/core/claw-test/scenarios.ts';
|
||||
|
||||
const ORIG_ROOT = process.env.GBRAIN_CLAW_SCENARIOS_DIR;
|
||||
let root: string;
|
||||
|
||||
beforeEach(() => {
|
||||
root = mkdtempSync(join(tmpdir(), 'scenarios-'));
|
||||
process.env.GBRAIN_CLAW_SCENARIOS_DIR = root;
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
if (ORIG_ROOT !== undefined) process.env.GBRAIN_CLAW_SCENARIOS_DIR = ORIG_ROOT;
|
||||
else delete process.env.GBRAIN_CLAW_SCENARIOS_DIR;
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
function scaffoldScenario(name: string, scenarioJson: string, briefContent = '# Brief'): void {
|
||||
const dir = join(root, name);
|
||||
mkdirSync(dir, { recursive: true });
|
||||
writeFileSync(join(dir, 'scenario.json'), scenarioJson);
|
||||
writeFileSync(join(dir, 'BRIEF.md'), briefContent);
|
||||
}
|
||||
|
||||
describe('listScenarios', () => {
|
||||
test('returns empty when no scenarios exist', () => {
|
||||
expect(listScenarios()).toEqual([]);
|
||||
});
|
||||
|
||||
test('returns directories that contain scenario.json, sorted', () => {
|
||||
scaffoldScenario('beta', '{"kind":"fresh-install","expected_phases":[]}');
|
||||
scaffoldScenario('alpha', '{"kind":"fresh-install","expected_phases":[]}');
|
||||
mkdirSync(join(root, 'incomplete'), { recursive: true }); // no scenario.json
|
||||
expect(listScenarios()).toEqual(['alpha', 'beta']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('loadScenario', () => {
|
||||
test('parses a valid fresh-install scenario', () => {
|
||||
scaffoldScenario('demo', JSON.stringify({
|
||||
kind: 'fresh-install',
|
||||
expected_phases: ['import.files', 'doctor.db_checks'],
|
||||
description: 'demo',
|
||||
brain: 'brain',
|
||||
}));
|
||||
const cfg = loadScenario('demo');
|
||||
expect(cfg.name).toBe('demo');
|
||||
expect(cfg.kind).toBe('fresh-install');
|
||||
expect(cfg.expectedPhases).toEqual(['import.files', 'doctor.db_checks']);
|
||||
expect(cfg.description).toBe('demo');
|
||||
expect(cfg.brainRelative).toBe('brain');
|
||||
});
|
||||
|
||||
test('parses an upgrade scenario with from_version + seed', () => {
|
||||
scaffoldScenario('upgrade-x', JSON.stringify({
|
||||
kind: 'upgrade',
|
||||
from_version: '0.18.0',
|
||||
expected_phases: ['doctor.db_checks'],
|
||||
seed: 'seed',
|
||||
}));
|
||||
mkdirSync(join(root, 'upgrade-x', 'seed'), { recursive: true });
|
||||
const cfg = loadScenario('upgrade-x');
|
||||
expect(cfg.kind).toBe('upgrade');
|
||||
expect(cfg.fromVersion).toBe('0.18.0');
|
||||
expect(cfg.seedRelative).toBe('seed');
|
||||
});
|
||||
|
||||
test('throws on missing scenario directory', () => {
|
||||
expect(() => loadScenario('does-not-exist')).toThrow(/not found/);
|
||||
});
|
||||
|
||||
test('throws on malformed JSON', () => {
|
||||
scaffoldScenario('bad', 'not json {');
|
||||
expect(() => loadScenario('bad')).toThrow(/malformed/);
|
||||
});
|
||||
|
||||
test('throws on unknown kind', () => {
|
||||
scaffoldScenario('weird', JSON.stringify({ kind: 'mystery', expected_phases: [] }));
|
||||
expect(() => loadScenario('weird')).toThrow(/unknown kind/);
|
||||
});
|
||||
|
||||
test('throws on non-array expected_phases', () => {
|
||||
scaffoldScenario('bad-phases', JSON.stringify({ kind: 'fresh-install', expected_phases: 'oops' }));
|
||||
expect(() => loadScenario('bad-phases')).toThrow(/expected_phases/);
|
||||
});
|
||||
|
||||
test('throws when BRIEF.md missing', () => {
|
||||
const dir = join(root, 'no-brief');
|
||||
mkdirSync(dir);
|
||||
writeFileSync(join(dir, 'scenario.json'), JSON.stringify({ kind: 'fresh-install', expected_phases: [] }));
|
||||
expect(() => loadScenario('no-brief')).toThrow(/BRIEF\.md missing/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('readBrief', () => {
|
||||
test('returns BRIEF.md content', () => {
|
||||
scaffoldScenario('reads-brief', '{"kind":"fresh-install","expected_phases":[]}', '# Hello world');
|
||||
const cfg = loadScenario('reads-brief');
|
||||
expect(readBrief(cfg)).toBe('# Hello world');
|
||||
});
|
||||
});
|
||||
|
||||
describe('shipped scenarios load cleanly', () => {
|
||||
test('fresh-install loads from default fixtures root', () => {
|
||||
delete process.env.GBRAIN_CLAW_SCENARIOS_DIR;
|
||||
try {
|
||||
const cfg = loadScenario('fresh-install');
|
||||
expect(cfg.kind).toBe('fresh-install');
|
||||
expect(cfg.expectedPhases.length).toBeGreaterThan(0);
|
||||
} finally {
|
||||
process.env.GBRAIN_CLAW_SCENARIOS_DIR = root;
|
||||
}
|
||||
});
|
||||
|
||||
test('upgrade-from-v0.18 loads from default fixtures root', () => {
|
||||
delete process.env.GBRAIN_CLAW_SCENARIOS_DIR;
|
||||
try {
|
||||
const cfg = loadScenario('upgrade-from-v0.18');
|
||||
expect(cfg.kind).toBe('upgrade');
|
||||
expect(cfg.fromVersion).toBe('0.18.0');
|
||||
} finally {
|
||||
process.env.GBRAIN_CLAW_SCENARIOS_DIR = root;
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -1,130 +0,0 @@
|
||||
/**
|
||||
* seed-pglite tests — exercises the SQL replay primitive that powers the
|
||||
* upgrade-from-v0.18 scenario. Pure PGLite in-memory; no real DB needed.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { mkdtempSync, rmSync, writeFileSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
import { seedPglite, seedPgliteFromFile, _internal } from '../src/core/claw-test/seed-pglite.ts';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
|
||||
let tmp: string;
|
||||
|
||||
beforeEach(() => {
|
||||
tmp = mkdtempSync(join(tmpdir(), 'seed-'));
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
describe('splitStatements', () => {
|
||||
const split = _internal.splitStatements;
|
||||
|
||||
test('splits on semicolons', () => {
|
||||
expect(split('CREATE TABLE a(x int); INSERT INTO a VALUES (1);').length).toBe(2);
|
||||
});
|
||||
|
||||
test('respects single-quoted strings', () => {
|
||||
const sql = "INSERT INTO t VALUES ('a;b'); INSERT INTO t VALUES ('c');";
|
||||
const stmts = split(sql);
|
||||
expect(stmts.length).toBe(2);
|
||||
expect(stmts[0]).toContain("'a;b'");
|
||||
});
|
||||
|
||||
test('respects -- line comments', () => {
|
||||
const sql = "-- a comment with ; semicolon\nCREATE TABLE x(id int);";
|
||||
const stmts = split(sql);
|
||||
expect(stmts.length).toBe(1);
|
||||
});
|
||||
|
||||
test('handles escaped quotes (doubled apostrophe)', () => {
|
||||
const sql = "INSERT INTO t VALUES ('it''s ok');";
|
||||
const stmts = split(sql);
|
||||
expect(stmts.length).toBe(1);
|
||||
expect(stmts[0]).toContain("it''s ok");
|
||||
});
|
||||
|
||||
test('returns empty list for empty input', () => {
|
||||
expect(split('').length).toBe(0);
|
||||
expect(split(' \n').length).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('seedPglite', () => {
|
||||
test('replays a SQL dump into a fresh PGLite database', async () => {
|
||||
const dbPath = join(tmp, 'brain.pglite');
|
||||
const sql = `
|
||||
CREATE TABLE seeded(id INT PRIMARY KEY, name TEXT);
|
||||
INSERT INTO seeded(id, name) VALUES (1, 'alice');
|
||||
INSERT INTO seeded(id, name) VALUES (2, 'bob');
|
||||
`;
|
||||
await seedPglite({ dbPath, sql });
|
||||
|
||||
// Re-open the seeded database and verify content survived.
|
||||
const engine = new PGLiteEngine();
|
||||
try {
|
||||
await engine.connect({ engine: 'pglite', database_path: dbPath });
|
||||
const rows: any = await (engine as any).db.query('SELECT id, name FROM seeded ORDER BY id');
|
||||
expect(rows.rows).toEqual([
|
||||
{ id: 1, name: 'alice' },
|
||||
{ id: 2, name: 'bob' },
|
||||
]);
|
||||
} finally {
|
||||
await engine.disconnect();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('throws with a useful message when SQL is invalid', async () => {
|
||||
const dbPath = join(tmp, 'bad.pglite');
|
||||
const sql = 'INVALID SQL HERE;';
|
||||
await expect(seedPglite({ dbPath, sql })).rejects.toThrow(/SQL execution failed/);
|
||||
}, 30_000);
|
||||
|
||||
test('creates parent directories when needed', async () => {
|
||||
const dbPath = join(tmp, 'nested', 'deeper', 'brain.pglite');
|
||||
await seedPglite({ dbPath, sql: 'CREATE TABLE x(y int);' });
|
||||
// No throw means the dir was created.
|
||||
expect(true).toBe(true);
|
||||
}, 30_000);
|
||||
|
||||
test('empty SQL is a no-op (just creates the .pglite)', async () => {
|
||||
const dbPath = join(tmp, 'empty.pglite');
|
||||
await seedPglite({ dbPath, sql: '' });
|
||||
// Verify the database is openable but empty.
|
||||
const engine = new PGLiteEngine();
|
||||
try {
|
||||
await engine.connect({ engine: 'pglite', database_path: dbPath });
|
||||
const r: any = await (engine as any).db.query("SELECT COUNT(*)::int AS c FROM information_schema.tables WHERE table_schema='public'");
|
||||
expect(r.rows[0].c).toBe(0);
|
||||
} finally {
|
||||
await engine.disconnect();
|
||||
}
|
||||
}, 30_000);
|
||||
});
|
||||
|
||||
describe('seedPgliteFromFile', () => {
|
||||
test('reads SQL from disk and replays', async () => {
|
||||
const sqlPath = join(tmp, 'dump.sql');
|
||||
const dbPath = join(tmp, 'brain.pglite');
|
||||
writeFileSync(sqlPath, 'CREATE TABLE z(id int); INSERT INTO z VALUES (42);');
|
||||
await seedPgliteFromFile({ dbPath, sqlPath });
|
||||
const engine = new PGLiteEngine();
|
||||
try {
|
||||
await engine.connect({ engine: 'pglite', database_path: dbPath });
|
||||
const r: any = await (engine as any).db.query('SELECT id FROM z');
|
||||
expect(r.rows).toEqual([{ id: 42 }]);
|
||||
} finally {
|
||||
await engine.disconnect();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('throws on missing SQL file', async () => {
|
||||
await expect(seedPgliteFromFile({
|
||||
dbPath: join(tmp, 'x.pglite'),
|
||||
sqlPath: join(tmp, 'nope.sql'),
|
||||
})).rejects.toThrow(/seed SQL not found/);
|
||||
});
|
||||
});
|
||||
@@ -328,79 +328,6 @@ describe('MinionSupervisor', () => {
|
||||
}, 15_000);
|
||||
});
|
||||
|
||||
describe('integration: GBRAIN_SUPERVISED env var (v0.22.14)', () => {
|
||||
it('sets GBRAIN_SUPERVISED=1 on spawned worker child', async () => {
|
||||
const outFile = join(tmpdir(), `gbrain-sup-supervised-${process.pid}-${Date.now()}.txt`);
|
||||
try { unlinkSync(outFile); } catch { /* may not exist */ }
|
||||
|
||||
const h = makeHarness('supervised-env', `printf '%s\n' "\${GBRAIN_SUPERVISED-UNSET}" > "$OUT_FILE" ; exit 0`);
|
||||
|
||||
try {
|
||||
const sup = spawnSupervisor(h, {
|
||||
OUT_FILE: outFile,
|
||||
SUP_MAX_CRASHES: '1',
|
||||
});
|
||||
|
||||
await sup.exited;
|
||||
|
||||
expect(existsSync(outFile)).toBe(true);
|
||||
const childSawEnv = readFileSync(outFile, 'utf8').trim();
|
||||
expect(childSawEnv).toBe('1');
|
||||
} finally {
|
||||
try { unlinkSync(outFile); } catch { /* noop */ }
|
||||
h.cleanup();
|
||||
}
|
||||
}, 15_000);
|
||||
});
|
||||
|
||||
describe('regression (R3): healthInterval=0 disables timer (v0.22.14)', () => {
|
||||
// Pre-fix: supervisor unconditionally called setInterval(callback, 0),
|
||||
// which schedules a tight loop on the next event-loop tick. The
|
||||
// operator-facing CLI claim "Use 0 to disable" was a lie — passing 0
|
||||
// produced a DB-probe loop that hammered Postgres.
|
||||
//
|
||||
// Post-fix: setInterval is gated on healthInterval > 0. With 0, the
|
||||
// supervisor runs its supervise loop normally with the health timer
|
||||
// entirely absent.
|
||||
//
|
||||
// Assertion strategy: spawn the supervisor with SUP_HEALTH_INTERVAL_MS=0,
|
||||
// a fast worker that exits cleanly, and SUP_MAX_CRASHES=1. A working fix
|
||||
// should produce a single worker spawn → exit → supervisor shutdown
|
||||
// sequence. If the tight-loop bug returned, the supervisor would still
|
||||
// exit (max-crashes path) but the audit trail would show the tell-tale
|
||||
// signature of an extremely high health-check call rate during the brief
|
||||
// window before max-crashes fires. We assert the basic completion path
|
||||
// and let CI's wall-clock detect any pathological CPU spike.
|
||||
it('completes a normal supervise lifecycle with healthInterval=0', async () => {
|
||||
const h = makeHarness('health-interval-zero', 'exit 0');
|
||||
|
||||
try {
|
||||
const sup = spawnSupervisor(h, {
|
||||
SUP_HEALTH_INTERVAL_MS: '0',
|
||||
SUP_MAX_CRASHES: '1',
|
||||
});
|
||||
|
||||
const start = Date.now();
|
||||
const { code } = await sup.exited;
|
||||
const elapsedMs = Date.now() - start;
|
||||
|
||||
// Clean exit (max-crashes path returns 1; this is fine — we just
|
||||
// want to confirm the supervisor reached its terminal state without
|
||||
// hanging or runaway looping).
|
||||
expect(code).toBe(1);
|
||||
|
||||
// Sanity: a tight loop on setInterval(0) plus the spawn-respawn
|
||||
// loop would still terminate at max-crashes, but it would be
|
||||
// measurably slower than a clean run because the event loop is
|
||||
// saturated with health-check callbacks. Cap the upper bound at
|
||||
// 10s — clean runs typically finish in 1–2s.
|
||||
expect(elapsedMs).toBeLessThan(10_000);
|
||||
} finally {
|
||||
h.cleanup();
|
||||
}
|
||||
}, 15_000);
|
||||
});
|
||||
|
||||
describe('integration: --max-rss spawn args (v0.21)', () => {
|
||||
it('passes --max-rss 2048 to spawned worker by default', async () => {
|
||||
const outFile = join(tmpdir(), `gbrain-sup-maxrss-${process.pid}-${Date.now()}.txt`);
|
||||
|
||||
@@ -1,166 +0,0 @@
|
||||
/**
|
||||
* Transcript capture tests — async drain, byte offsets, multi-byte safety,
|
||||
* spawn-with-capture happy + timeout paths.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { mkdtempSync, rmSync, readFileSync, existsSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
import { createTranscriptSink, spawnWithCapture } from '../src/core/claw-test/transcript-capture.ts';
|
||||
|
||||
let tmp: string;
|
||||
let path: string;
|
||||
|
||||
beforeEach(() => {
|
||||
tmp = mkdtempSync(join(tmpdir(), 'transcript-'));
|
||||
path = join(tmp, 'transcript.jsonl');
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
describe('createTranscriptSink', () => {
|
||||
test('writes events as JSONL lines with byte_offset', async () => {
|
||||
const sink = createTranscriptSink(path);
|
||||
sink.write({ ts: 1, channel: 'stdout', bytes: Buffer.from('hello') });
|
||||
sink.write({ ts: 2, channel: 'stderr', bytes: Buffer.from('world') });
|
||||
await sink.close();
|
||||
|
||||
const raw = readFileSync(path, 'utf-8');
|
||||
const lines = raw.trim().split('\n').map(l => JSON.parse(l));
|
||||
expect(lines).toHaveLength(2);
|
||||
expect(lines[0].channel).toBe('stdout');
|
||||
expect(lines[0].byte_offset).toBe(0);
|
||||
expect(lines[1].channel).toBe('stderr');
|
||||
expect(lines[1].byte_offset).toBeGreaterThan(0);
|
||||
expect(Buffer.from(lines[0].bytes_b64, 'base64').toString('utf-8')).toBe('hello');
|
||||
expect(Buffer.from(lines[1].bytes_b64, 'base64').toString('utf-8')).toBe('world');
|
||||
});
|
||||
|
||||
test('preserves multi-byte UTF-8 (no chunk-boundary corruption)', async () => {
|
||||
const sink = createTranscriptSink(path);
|
||||
// Split a 4-byte emoji across two writes to simulate stdio chunk boundaries.
|
||||
const emoji = '🌍';
|
||||
const buf = Buffer.from(emoji, 'utf-8');
|
||||
sink.write({ ts: 1, channel: 'stdout', bytes: buf.slice(0, 2) });
|
||||
sink.write({ ts: 2, channel: 'stdout', bytes: buf.slice(2) });
|
||||
await sink.close();
|
||||
|
||||
const lines = readFileSync(path, 'utf-8').trim().split('\n').map(l => JSON.parse(l));
|
||||
const concatenated = Buffer.concat([
|
||||
Buffer.from(lines[0].bytes_b64, 'base64'),
|
||||
Buffer.from(lines[1].bytes_b64, 'base64'),
|
||||
]).toString('utf-8');
|
||||
expect(concatenated).toBe(emoji);
|
||||
});
|
||||
|
||||
test('byte_offset is monotonic and matches the actual file position', async () => {
|
||||
const sink = createTranscriptSink(path);
|
||||
const before1 = sink.nextOffset();
|
||||
sink.write({ ts: 1, channel: 'stdout', bytes: Buffer.from('a') });
|
||||
const before2 = sink.nextOffset();
|
||||
sink.write({ ts: 2, channel: 'stdout', bytes: Buffer.from('b') });
|
||||
await sink.close();
|
||||
|
||||
expect(before1).toBe(0);
|
||||
expect(before2).toBeGreaterThan(0);
|
||||
|
||||
// Verify the offsets recorded in lines match the actual file substring offsets.
|
||||
const raw = readFileSync(path, 'utf-8');
|
||||
const lines = raw.trim().split('\n').map(l => JSON.parse(l));
|
||||
const expectedOffsets = [0, Buffer.byteLength(raw.split('\n')[0] + '\n')];
|
||||
expect(lines[0].byte_offset).toBe(expectedOffsets[0]);
|
||||
expect(lines[1].byte_offset).toBe(expectedOffsets[1]);
|
||||
});
|
||||
|
||||
test('survives bursty writes (drain handling)', async () => {
|
||||
const sink = createTranscriptSink(path);
|
||||
// 256KB of payload across 256 1KB writes — exceeds default pipe buffer
|
||||
const chunk = Buffer.alloc(1024, 0x61); // 'a' * 1024
|
||||
for (let i = 0; i < 256; i++) {
|
||||
sink.write({ ts: i, channel: 'stdout', bytes: chunk });
|
||||
}
|
||||
await sink.close();
|
||||
|
||||
const raw = readFileSync(path, 'utf-8');
|
||||
const lines = raw.trim().split('\n');
|
||||
expect(lines.length).toBe(256);
|
||||
});
|
||||
|
||||
test('close is idempotent', async () => {
|
||||
const sink = createTranscriptSink(path);
|
||||
sink.write({ ts: 1, channel: 'stdout', bytes: Buffer.from('x') });
|
||||
await sink.close();
|
||||
// Second close should not throw — the writeStream's `end` won't fire 'close' a second time
|
||||
// but we can call without error in our own wrapper.
|
||||
// (Implementation note: we don't expose a closed flag; idempotent via stream's no-op behavior.)
|
||||
expect(existsSync(path)).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('spawnWithCapture', () => {
|
||||
test('captures stdout from a small command', async () => {
|
||||
const sink = createTranscriptSink(path);
|
||||
const result = await spawnWithCapture('/bin/sh', ['-c', 'printf hi'], {
|
||||
cwd: tmp,
|
||||
env: { PATH: process.env.PATH ?? '' },
|
||||
timeoutMs: 5_000,
|
||||
transcriptSink: sink,
|
||||
});
|
||||
await sink.close();
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.timedOut).toBe(false);
|
||||
|
||||
const raw = readFileSync(path, 'utf-8');
|
||||
const captured = raw.split('\n').filter(Boolean).map(l => JSON.parse(l));
|
||||
const stdoutBytes = captured.filter(e => e.channel === 'stdout')
|
||||
.map(e => Buffer.from(e.bytes_b64, 'base64').toString('utf-8'))
|
||||
.join('');
|
||||
expect(stdoutBytes).toBe('hi');
|
||||
});
|
||||
|
||||
test('non-zero exit propagates', async () => {
|
||||
const sink = createTranscriptSink(path);
|
||||
const result = await spawnWithCapture('/bin/sh', ['-c', 'exit 7'], {
|
||||
cwd: tmp,
|
||||
env: { PATH: process.env.PATH ?? '' },
|
||||
timeoutMs: 5_000,
|
||||
transcriptSink: sink,
|
||||
});
|
||||
await sink.close();
|
||||
expect(result.exitCode).toBe(7);
|
||||
expect(result.timedOut).toBe(false);
|
||||
});
|
||||
|
||||
test('timeout fires SIGTERM/SIGKILL', async () => {
|
||||
const sink = createTranscriptSink(path);
|
||||
// `exec sleep` replaces sh with sleep so the child we spawn IS sleep —
|
||||
// SIGTERM goes directly to it, no shell-vs-child process-group ambiguity.
|
||||
// CI runners are slower than local, so the test cap is 30s with headroom
|
||||
// even if SIGTERM is missed and SIGKILL has to run after the 5s grace.
|
||||
const result = await spawnWithCapture('/bin/sh', ['-c', 'exec sleep 30'], {
|
||||
cwd: tmp,
|
||||
env: { PATH: process.env.PATH ?? '' },
|
||||
timeoutMs: 200,
|
||||
transcriptSink: sink,
|
||||
});
|
||||
await sink.close();
|
||||
expect(result.timedOut).toBe(true);
|
||||
expect(result.exitCode).not.toBe(0);
|
||||
}, 30_000);
|
||||
|
||||
test('rejects when the binary does not exist', async () => {
|
||||
const sink = createTranscriptSink(path);
|
||||
await expect(
|
||||
spawnWithCapture('/no/such/binary', [], {
|
||||
cwd: tmp,
|
||||
env: { PATH: process.env.PATH ?? '' },
|
||||
timeoutMs: 1_000,
|
||||
transcriptSink: sink,
|
||||
})
|
||||
).rejects.toThrow();
|
||||
await sink.close();
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user