mirror of
https://github.com/garrytan/gbrain.git
synced 2026-08-14 08:53:22 +00:00
Compare commits
17
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6ee717a4b7 | ||
|
|
0d871f4ca7 | ||
|
|
9e7a6287eb | ||
|
|
ab5fac7662 | ||
|
|
65095a9331 | ||
|
|
75ca834df1 | ||
|
|
7f100a004e | ||
|
|
06f59468a4 | ||
|
|
99c0f96dbd | ||
|
|
3f0f92e670 | ||
|
|
7eb41b7b7f | ||
|
|
3cf0087142 | ||
|
|
eb579e7a82 | ||
|
|
ec239b7b28 | ||
|
|
e070079554 | ||
|
|
0aa7802d26 | ||
|
|
a66c66d64a |
+132
@@ -2,6 +2,138 @@
|
||||
|
||||
All notable changes to GBrain will be documented in this file.
|
||||
|
||||
## [0.37.7.0] - 2026-05-21
|
||||
|
||||
**Your federated brain stops silently writing to the wrong source. Your autopilot stops thrashing when you point it at a second brain. A handful of CLI surfaces that crashed on first call stop crashing.**
|
||||
|
||||
If you've ever run gbrain with more than one source (an Obsidian vault + a docs repo, a CEO with multiple team sub-brains), you've probably noticed that `gbrain import --source dept-x` silently writes to `default` instead. Or that `gbrain extract` produces zero links on a federated brain. Or that `gbrain doctor` recommends a command that can't actually fix it. This release wires every CLI surface through the same source-resolver that `gbrain serve` has used since v0.18.0, and adds a doctor check that catches the silent-collapse-to-default class.
|
||||
|
||||
Plus: autopilot lockfile finally respects `GBRAIN_HOME` so two brains can coexist. The reconnect loop that logged `config.database_url undefined` forever now exits cleanly and lets launchd back off. `gbrain reindex-frontmatter` no longer crashes before doing anything. OAuth `authorization_code` confidential clients work again. And we caught a dead-letter bug where successful subagent jobs were being marked failed because the worker tried to re-prompt past an `end_turn`.
|
||||
|
||||
### What landed
|
||||
|
||||
| Piece | What it fixes / adds | How to use it |
|
||||
|---|---|---|
|
||||
| **`gbrain import --source-id <id>`** (#1167) | `import` finally honors the source flag — pages route to the named source instead of silently collapsing to `default` | `gbrain import path/ --source-id dept-x` |
|
||||
| **`gbrain extract --source-id <id>`** (#1204) | Same fix for extract — link + timeline extraction now scopes to one source on federated brains | `gbrain extract all --source-id dept-x` |
|
||||
| **`gbrain sources current`** (#1222) | New subcommand that prints the resolved source + the tier that won (flag / env / dotfile / local_path / brain_default / seed_default) | `gbrain sources current --json` |
|
||||
| **`gbrain graph-query --include-foreign`** (#1153) | Cross-source edges no longer disappear silently. Footer shows foreign-edge count by default; `--include-foreign` walks them | `gbrain graph-query alice --depth 2 --include-foreign` |
|
||||
| **`skills/conventions/brain-routing.md`** (#1222) | Documents the canonical 6-tier source resolution chain so agents stop guessing | Read it; pointed at from CLAUDE.md |
|
||||
| **Autopilot lockfile scoped to `GBRAIN_HOME`** (#1226) | Two brains can run autopilot simultaneously without one stealing the other's lock | Set `GBRAIN_HOME=/path/to/brain-b gbrain autopilot --install` |
|
||||
| **Autopilot reconnect classifier + launchd ThrottleInterval** (#1162) | Reconnect loop that logged `config.database_url undefined` every 5s now exits cleanly. New launchd plist sets `ThrottleInterval=300` so launchd respects unrecoverable failures | `gbrain autopilot --install` regenerates the plist |
|
||||
| **OAuth confidential `authorization_code` clients** (#1166) | The MCP SDK's `clientAuth` middleware does plaintext compare; gbrain stores SHA-256 hashes, so confidential clients failed every `/token` request. Custom verifier middleware now runs before the SDK | `gbrain auth register-client --grant-types authorization_code` works again |
|
||||
| **`gbrain reindex-frontmatter`** (#1225) | Crashed on first call because it queried the engine before connecting it. Now connects via the existing command pattern | `gbrain reindex-frontmatter` |
|
||||
| **Subagent terminal-on-resume** (#1151) | Successful subagent jobs were being marked failed because the worker tried to re-prompt past an `end_turn` on resume. Short-circuits to terminal | Automatic on next worker restart |
|
||||
| **Sync walker skips git submodules** (#1169) | `pruneDir()` now detects submodules (`.git` as a file, not a directory) and stops walking into them. Stops phantom imports of submodule content | Automatic |
|
||||
| **3 new doctor checks** (T12/T13/T14) | `source_routing_health` (catches silent-collapse-to-default on federated brains), `oauth_confidential_health` (probes confidential-client `/token` reachability), `autopilot_lock_scope` (warns when lock isn't under `GBRAIN_HOME`) | `gbrain doctor` |
|
||||
|
||||
### How to verify the source-routing fix
|
||||
|
||||
```bash
|
||||
# Show which source resolved + why
|
||||
gbrain sources current
|
||||
|
||||
# JSON shape for scripting
|
||||
gbrain sources current --json
|
||||
# → {"source_id":"dept-x","tier":"dotfile","detail":".gbrain-source"}
|
||||
|
||||
# Doctor check — should be ok on a properly federated brain
|
||||
gbrain doctor --json | jq '.checks[] | select(.name=="source_routing_health")'
|
||||
```
|
||||
|
||||
The 6-tier chain documented in `skills/conventions/brain-routing.md`:
|
||||
|
||||
1. `--source <id>` flag (explicit, always wins)
|
||||
2. `GBRAIN_SOURCE` env var
|
||||
3. `.gbrain-source` dotfile walk-up
|
||||
4. Registered source whose `local_path` contains CWD
|
||||
5. Brain default (config key)
|
||||
6. Seed default (`default`)
|
||||
|
||||
### What's safe to know about
|
||||
|
||||
- **Default behavior unchanged for single-source brains.** If you only have `default`, nothing in this release changes how your commands route. The fixes only fire on federated brains (`gbrain sources list` shows more than one row).
|
||||
- **Autopilot lockfile path changed.** Old: `~/.gbrain/autopilot.lock` only. New: `$GBRAIN_HOME/autopilot.lock` (defaults to `~/.gbrain` when unset, so most users see no path change). The doctor check warns if your `GBRAIN_HOME` is set but the lock landed in the wrong place.
|
||||
- **`reindex-frontmatter`'s fix is two lines but it's load-bearing.** Pre-fix the command would `process.exit(1)` with a TypeError on first call. Post-fix it does what it always claimed to.
|
||||
- **OAuth confidential clients were dead in v0.37.0–v0.37.6.** If your agent (Hermes, custom orchestrator) uses `authorization_code` + `client_secret_post` against gbrain, you needed this. Public PKCE clients (Claude Code, Cursor) were unaffected — they don't present a secret.
|
||||
- **The 3 new doctor checks are warn-only.** None fail the doctor exit. They surface paste-ready fix hints inline.
|
||||
|
||||
### What we caught and fixed before merging
|
||||
|
||||
- **Lockfile PID-safety.** The first-pass autopilot-lock fix moved the lock path but kept the existence-only check. Codex caught that a stale lock from a crashed process would block a healthy autopilot from starting. The shipped version writes PID into the lock and checks `kill -0 <pid>` before refusing to start.
|
||||
- **OAuth confidential auth detection.** The first-pass middleware sniffed for `Authorization: Basic` headers only. That missed `client_secret_post` (form-encoded body) which is the more common shape. The shipped version handles both.
|
||||
- **`pruneDir` submodule detection.** The first-pass used `existsSync(.git)` which is true for both regular repos AND submodules. Refined to `statSync(.git).isFile()` — submodule gitfiles are FILES pointing into the parent's `.git/modules/`, regular repos have `.git` as a DIRECTORY.
|
||||
- **`resolveSourceWithTier()` is additive.** Original plan refactored `resolveSourceId()` to return the tier. Codex pointed out that breaks every existing caller. Shipped as a new function alongside the old one; the existing 6-tier chain is unchanged.
|
||||
|
||||
### Itemized changes
|
||||
|
||||
#### CLI surfaces (production code)
|
||||
|
||||
- `src/commands/import.ts` — new `--source-id <id>` flag. Resolved via `resolveSourceWithTier()` before any page write; failures surface with paste-ready `gbrain sources list` hint. Closes #1167.
|
||||
- `src/commands/extract.ts` — same `--source-id` flag, threaded through `extractLinks()` / `extractTimeline()` so SQL scopes to the named source. Closes #1204.
|
||||
- `src/commands/sources.ts` — new `gbrain sources current [--json]` subcommand. Calls `resolveSourceWithTier()` and prints `source_id`, `tier`, optional `detail`. Closes #1222.
|
||||
- `src/commands/graph-query.ts` — foreign-edge footer always present (`X foreign edges (use --include-foreign to traverse)`). `--include-foreign` flag widens the SQL filter to cross-source edges. Closes #1153.
|
||||
- `src/commands/reindex-frontmatter.ts` — wrapped query path in the standard `withEngine(...)` lifecycle so `engine.connect()` runs before the first SQL call. Closes #1225.
|
||||
- `src/commands/autopilot.ts` — `LOCK_PATH` now resolves via `gbrainPath('autopilot.lock')` (honors `GBRAIN_HOME`). New exports `classifyReconnectError(err)` (returns `'recoverable' | 'unrecoverable'`; unrecoverable causes the daemon to `process.exit(0)` and let launchd back off) and `generateLaunchdPlist(wrapperPath, home)` (pure plist string for tests). Lock file now stores PID; startup checks `kill -0 <pid>` before refusing. Closes #1162 + #1226.
|
||||
- `src/core/oauth-provider.ts` + `src/commands/serve-http.ts` — custom `/token` middleware that runs BEFORE the MCP SDK's `clientAuth`. Detects confidential auth via `Authorization: Basic` header OR `client_secret_post` form body; verifies via SHA-256 hash compare and falls through to the SDK for public PKCE clients. Closes #1166.
|
||||
- `src/core/sync.ts` — `pruneDir(name, parentDir?)` extended with optional `parentDir`; when provided, additionally rejects directories containing `.git` as a FILE (git submodule gitfile pattern). Sync + extract walkers thread `parentDir` through. Closes #1169.
|
||||
- `src/core/minions/handlers/subagent.ts` — terminal-state short-circuit on resume. When a stored message thread already ends in `stop_reason: 'end_turn'`, the handler returns `{ ok: true }` instead of issuing another `messages.create` call (which would have failed and dead-lettered a successful job). Closes #1151.
|
||||
|
||||
#### New helpers (production code)
|
||||
|
||||
- `src/core/source-resolver.ts` — additive `resolveSourceWithTier(engine, explicit, cwd)` returns `{ source_id, tier, detail? }`. New exported const `SOURCE_TIER_NAMES = ['flag', 'env', 'dotfile', 'local_path', 'brain_default', 'seed_default']` so the JSON shape is type-stable across releases. The existing `resolveSourceId()` is unchanged.
|
||||
|
||||
#### Doctor checks (production code)
|
||||
|
||||
- `src/commands/doctor.ts` — three new checks wired into both `runDoctor()` and the JSON envelope:
|
||||
- `checkSourceRoutingHealth(engine)` — scans up to 200 pages on federated brains, flags pages whose `source_id` doesn't match what `resolveSourceWithTier()` would have picked for their `source_path`. Short-circuits to `ok` for single-source brains.
|
||||
- `checkOauthConfidentialHealth(engine)` — probes registered confidential clients for `/token` reachability; warns when the v0.37.0–v0.37.6 plaintext-compare bug class would have rejected them.
|
||||
- `checkAutopilotLockScope()` — pure function check (no engine). Compares the resolved lock path to `$GBRAIN_HOME`; warns when `$GBRAIN_HOME` is set but the lock lives elsewhere, with a PID-safe hint to inspect the lock file before deleting it.
|
||||
|
||||
#### Documentation
|
||||
|
||||
- `skills/conventions/brain-routing.md` — new convention skill documenting the 6-tier source resolution chain with paste-ready agent decision table. Linked from CLAUDE.md's "Two organizational axes" section. Closes #1222.
|
||||
|
||||
#### Tests
|
||||
|
||||
- `test/source-resolver-with-tier.test.ts` — 6-tier resolution chain coverage (each tier's win condition, precedence ordering, invalid-source rejection, detail strings). Uses `withEnv()` wrapper for env-mutation isolation per the test-isolation lint.
|
||||
- `test/import-source-id.test.ts` — `--source-id` flag round-trip + error path.
|
||||
- `test/graph-query.test.ts` — foreign-edge footer + `--include-foreign` traversal.
|
||||
- `test/oauth-confidential-client.test.ts` — `client_secret_basic` + `client_secret_post` paths against the custom middleware.
|
||||
- `test/autopilot-lock-path.test.ts` — `GBRAIN_HOME` scope + PID-safe staleness detection.
|
||||
- `test/autopilot-reconnect-classifier.test.ts` — recoverable vs unrecoverable error classification + plist generator shape.
|
||||
- `test/sync-walker-submodule.test.ts` — submodule detection via gitfile-as-FILE.
|
||||
- `test/subagent-handler.test.ts` — terminal-on-resume short-circuit.
|
||||
- `test/reindex-frontmatter-connect.test.ts` — engine.connect() runs before first query.
|
||||
- `test/doctor-v0_37_7_checks.test.ts` — all 3 new doctor checks against fixture brains (single-source ok-fast-path, federated brain mismatch warn, OAuth confidential warn shape, lock-scope mismatch warn).
|
||||
|
||||
#### Closed community PRs
|
||||
|
||||
Credited contributors per the CHANGELOG attribution convention; closing comments point at the absorbed commits.
|
||||
|
||||
## To take advantage of v0.37.7.0
|
||||
|
||||
`gbrain upgrade` should do this automatically. If it didn't, or if
|
||||
`gbrain doctor` warns about anything new:
|
||||
|
||||
1. **Run the orchestrator manually:**
|
||||
```bash
|
||||
gbrain apply-migrations --yes
|
||||
```
|
||||
2. **Verify the source-routing fix on your federated brains:**
|
||||
```bash
|
||||
gbrain sources current
|
||||
gbrain doctor --json | jq '.checks[] | select(.name=="source_routing_health")'
|
||||
```
|
||||
3. **If autopilot was thrashing on a second-brain install,** check
|
||||
the new lockfile path:
|
||||
```bash
|
||||
ls -la ~/.gbrain/autopilot.lock $GBRAIN_HOME/autopilot.lock 2>/dev/null
|
||||
```
|
||||
4. **If any step fails,** file an issue at
|
||||
https://github.com/garrytan/gbrain/issues with `gbrain doctor`
|
||||
output + contents of `~/.gbrain/upgrade-errors.jsonl` if it exists.
|
||||
|
||||
## [0.37.6.0] - 2026-05-20
|
||||
|
||||
**One key, many hosted models.**
|
||||
|
||||
@@ -138,8 +138,18 @@ strict behavior when unset.
|
||||
- `src/core/enrichment-service.ts` — Global enrichment service: entity slug generation, tier auto-escalation, batch throttling
|
||||
- `src/core/data-research.ts` — Recipe validation, field extraction (MRR/ARR regex), dedup, tracker parsing, HTML stripping
|
||||
- `src/commands/embed.ts` — `gbrain embed [--stale|--all] [--slugs ...]`. v0.22.1 (#409, contributed by @atrevino47): `--stale` path now starts with `engine.countStaleChunks()` (single SELECT count(*) WHERE embedding IS NULL, ~50 bytes wire). On a fully-embedded brain that's a 1-line short-circuit — no further reads. When stale chunks exist, `engine.listStaleChunks()` returns just the chunks needing embeddings (slug + chunk_index + chunk_text + metadata, no `vector(1536)` payload). Caller groups by slug, embeds via OpenAI, re-upserts via `upsertChunks`. Replaces the prior page-walk that pulled every chunk's embedding column over the wire and discarded most.
|
||||
- `src/commands/extract.ts` — `gbrain extract links|timeline|all [--source fs|db]`: batch link/timeline extraction. fs walks markdown files, db walks pages from the engine (mutation-immune snapshot iteration; use this for live brains with no local checkout). As of v0.12.1 there is no in-memory dedup pre-load — candidates are buffered 100 at a time and flushed via `addLinksBatch` / `addTimelineEntriesBatch`; `ON CONFLICT DO NOTHING` enforces uniqueness at the DB layer, and the `created` counter returns real rows inserted (truthful on re-runs). v0.22.1 (#417): `ExtractOpts.slugs?: string[]` enables incremental extract — when set, `extractForSlugs()` reads ONLY those slugs' files (single combined links+timeline pass) instead of the full directory walk. CLI `gbrain extract` keeps full-walk behavior; the cycle path threads sync's `pagesAffected` through. `walkMarkdownFiles(brainDir)` still runs at line 455 to build `allSlugs` for link resolution — see `TODOS.md` for replacing it with `engine.getAllSlugs()`.
|
||||
- `src/commands/graph-query.ts` — `gbrain graph-query <slug> [--type T] [--depth N] [--direction in|out|both]`: typed-edge relationship traversal (renders indented tree)
|
||||
- `src/commands/extract.ts` — `gbrain extract links|timeline|all [--source fs|db] [--source-id <id>]`: batch link/timeline extraction. fs walks markdown files, db walks pages from the engine (mutation-immune snapshot iteration; use this for live brains with no local checkout). As of v0.12.1 there is no in-memory dedup pre-load — candidates are buffered 100 at a time and flushed via `addLinksBatch` / `addTimelineEntriesBatch`; `ON CONFLICT DO NOTHING` enforces uniqueness at the DB layer, and the `created` counter returns real rows inserted (truthful on re-runs). v0.22.1 (#417): `ExtractOpts.slugs?: string[]` enables incremental extract — when set, `extractForSlugs()` reads ONLY those slugs' files (single combined links+timeline pass) instead of the full directory walk. CLI `gbrain extract` keeps full-walk behavior; the cycle path threads sync's `pagesAffected` through. `walkMarkdownFiles(brainDir)` still runs at line 455 to build `allSlugs` for link resolution — see `TODOS.md` for replacing it with `engine.getAllSlugs()`. **v0.37.7.0 (#1204):** `--source-id <id>` flag scopes extraction to one brain source on federated brains. Resolved via `resolveSourceWithTier()` before any SQL runs; failures surface with a `gbrain sources list` hint. Closes the silent-collapse-to-`default` bug class for extract.
|
||||
- `src/commands/import.ts` — `gbrain import <path> [--source-id <id>]`: page import with the v0.34.2.0 path-set checkpoint described above. **v0.37.7.0 (#1167):** new `--source-id <id>` flag finally honored — pages route to the named source. Resolved via `resolveSourceWithTier()` at the boundary; the same flag is now consistent across `import`, `extract`, `graph-query`, and `sources current`. Pinned by `test/import-source-id.test.ts`.
|
||||
- `src/commands/graph-query.ts` — `gbrain graph-query <slug> [--type T] [--depth N] [--direction in|out|both] [--include-foreign]`: typed-edge relationship traversal (renders indented tree). **v0.37.7.0 (#1153):** foreign-edge footer always present (`X foreign edges (use --include-foreign to traverse)`) so cross-source edges never disappear silently; `--include-foreign` widens the SQL filter to walk them. Pinned by `test/graph-query.test.ts`.
|
||||
- `src/commands/sources.ts` — `gbrain sources {list,add,remove,archive,restore,archived,purge,current}`. **v0.37.7.0 (#1222):** new `current [--json]` subcommand calls `resolveSourceWithTier()` and prints `source_id`, `tier` (one of `flag | env | dotfile | local_path | brain_default | seed_default`), and optional `detail`. The agent-facing decision table for which tier wins lives in `skills/conventions/brain-routing.md`.
|
||||
- `src/commands/reindex-frontmatter.ts` — `gbrain reindex-frontmatter`. **v0.37.7.0 (#1225):** wrapped the query path in the standard `withEngine(...)` lifecycle so `engine.connect()` runs before the first SQL call. Pre-fix the command `process.exit(1)`'d with a TypeError on first invocation. Pinned by `test/reindex-frontmatter-connect.test.ts`.
|
||||
- `src/core/source-resolver.ts` — 6-tier source resolution. **v0.37.7.0:** new additive helper `resolveSourceWithTier(engine, explicit, cwd)` returns `{ source_id, tier: SourceTier, detail? }` alongside the existing `resolveSourceId()` (unchanged, no caller breakage). New exported const `SOURCE_TIER_NAMES = ['flag', 'env', 'dotfile', 'local_path', 'brain_default', 'seed_default']` so the JSON shape stays type-stable across releases. Order matches the 1-6 priority of `resolveSourceId()`. Consumed by `gbrain sources current`, `gbrain import --source-id`, `gbrain extract --source-id`, and the v0.37.7.0 `source_routing_health` doctor check. Pinned by `test/source-resolver-with-tier.test.ts` (uses `withEnv()` wrapper per the test-isolation lint).
|
||||
- `src/commands/autopilot.ts` extension (v0.37.7.0) — three changes for federated-brain co-existence and launchd hygiene. (1) **#1226 lockfile scope:** `LOCK_PATH` resolves via `gbrainPath('autopilot.lock')` so it honors `GBRAIN_HOME`. Two brains can run autopilot simultaneously without lock-stealing. Lock file now stores PID; startup checks `kill -0 <pid>` before refusing to start (codex CF11 PID-safety fix — stale lock from a crashed process no longer blocks a healthy autopilot). (2) **#1162 reconnect classifier:** new exported `classifyReconnectError(err)` returns `'recoverable' | 'unrecoverable'`. Unrecoverable causes the daemon to `process.exit(0)` and let launchd back off instead of the v0.37.6 loop that logged `config.database_url undefined` every 5s forever. (3) **launchd plist generator:** new exported pure function `generateLaunchdPlist(wrapperPath, home)` sets `ThrottleInterval=300` so launchd respects the exit-0 backoff. Both helpers pinned by `test/autopilot-lock-path.test.ts` + `test/autopilot-reconnect-classifier.test.ts`.
|
||||
- `src/core/oauth-provider.ts` + `src/commands/serve-http.ts` extension (v0.37.7.0, #1166) — custom `/token` middleware that runs BEFORE the MCP SDK's `clientAuth`. The SDK does plaintext compare against the request's `client_secret`; gbrain stores SHA-256 hashes only, so every confidential-client `/token` request failed in v0.37.0–v0.37.6. The new middleware detects confidential auth via `Authorization: Basic` header OR `client_secret_post` form body (both shapes per RFC 6749 §2.3.1), verifies via `verifyClient(client_id, presented_secret)` (SHA-256 hash compare), and falls through to the SDK for public PKCE clients. Public clients (Claude Code, Cursor, every other PKCE-first MCP client) are unaffected — the SDK's clientAuth path still accepts them via the v0.34.1.0 NULL-`client_secret_hash` normalization. Pinned by `test/oauth-confidential-client.test.ts` (both `client_secret_basic` and `client_secret_post` paths).
|
||||
- `src/core/sync.ts:pruneDir` extension (v0.37.7.0, #1169) — `pruneDir(name, parentDir?)` signature extended with optional `parentDir`. When provided, the helper additionally rejects directories containing `.git` as a FILE — the git submodule gitfile pattern (regular repos have `.git` as a DIRECTORY; submodules have it as a file pointing into the parent's `.git/modules/`). Sync + extract walkers thread `parentDir` through so the gitfile-as-FILE check fires per descend step. Best-effort: `statSync` failures (cross-platform permission edge) fall through and treat as a normal dir. Closes the phantom-import bug class where syncing a worktree-with-submodules silently walked into submodule trees. Pinned by `test/sync-walker-submodule.test.ts`.
|
||||
- `src/core/minions/handlers/subagent.ts` extension (v0.37.7.0, #1151) — terminal-state short-circuit on resume. When a stored message thread already ends in `stop_reason: 'end_turn'`, the handler returns `{ ok: true }` immediately instead of issuing another `messages.create` call. Pre-fix, resume tried to re-prompt past `end_turn`, the Anthropic API rejected with a 400, and the worker classified the (already-successful) job as failed and dead-lettered it. Pinned by `test/subagent-handler.test.ts`.
|
||||
- `src/commands/doctor.ts` extension (v0.37.7.0, T12+T13+T14) — three new checks wired into `runDoctor()` and the JSON envelope. (1) `checkSourceRoutingHealth(engine)` scans up to 200 pages on federated brains and flags pages whose `source_id` doesn't match what `resolveSourceWithTier()` would have picked for their `source_path`. Single-source brains short-circuit to `ok` ("no federation to check"). D5 200-page cap keeps doctor under 5s on huge brains; the cap is total across the brain, not per-source. (2) `checkOauthConfidentialHealth(engine)` probes registered confidential clients for `/token` reachability — warns when the v0.37.0–v0.37.6 plaintext-compare bug class would have rejected them. (3) `checkAutopilotLockScope()` is a pure-function check (no engine) that compares the resolved lock path to `$GBRAIN_HOME`; warns when `$GBRAIN_HOME` is set but the lock lives elsewhere, with a PID-safe inspection hint per codex CF11 (lock file holds the daemon PID; check `kill -0 <pid>` before considering deletion). All three are warn-only — they surface paste-ready fix hints without flipping the doctor exit code. Pinned by `test/doctor-v0_37_7_checks.test.ts`.
|
||||
- `skills/conventions/brain-routing.md` (v0.37.7.0, #1222) — agent-facing convention skill documenting the canonical 6-tier source resolution chain (flag → env → dotfile → local_path → brain_default → seed_default) with paste-ready decision tables. Linked from CLAUDE.md's "Two organizational axes" section and from `gbrain sources current`'s hint output.
|
||||
- `src/core/link-extraction.ts` — shared library for the v0.12.0 graph layer. extractEntityRefs (canonical, replaces backlinks.ts duplicate) matches both `[Name](people/slug)` markdown links and Obsidian `[[people/slug|Name]]` wikilinks as of v0.12.3. extractPageLinks, inferLinkType heuristics (attended/works_at/invested_in/founded/advises/source/mentions), parseTimelineEntries, isAutoLinkEnabled config helper. `DIR_PATTERN` covers `people`, `companies`, `deals`, `topics`, `concepts`, `projects`, `entities`, `tech`, `finance`, `personal`, `openclaw`. Used by extract.ts, operations.ts auto-link post-hook, and backlinks.ts.
|
||||
- `src/core/zombie-reap.ts` (v0.28.1) — idempotent `installSigchldHandler()` so JS-spawned children get reaped via Bun's internal `waitpid()`. Bun (like Node) only auto-reaps when a SIGCHLD listener is registered; without it, every child the worker spawns (shell jobs, embed batches, sub-agents) becomes a zombie on exit and holds connection slots. Called once at module load from `src/cli.ts` (with Windows platform guard — SIGCHLD doesn't exist on Windows). Cross-file leak guard via `_uninstallSigchldHandlerForTests()` for tests. Layer 1 of the three-layer zombie defense; Layer 2 is tini-as-PID-1 wrapping the worker subtree (via `src/core/minions/spawn-helpers.ts`); Layer 3 is the container's own tini for hard Bun crashes.
|
||||
- `src/core/minions/` — Minions job queue: BullMQ-inspired, Postgres-native (queue, worker, backoff, types, protected-names, quiet-hours, stagger, handlers/shell).
|
||||
|
||||
@@ -1,6 +1,15 @@
|
||||
# TODOS
|
||||
|
||||
|
||||
## v0.37.7.0 federated-brains + autopilot safety follow-ups (v0.37.x+)
|
||||
|
||||
- [ ] **.sql file indexing (#1173) — dropped from v0.37.7.0 because tree-sitter-sql.wasm is not in `src/assets/wasm/grammars/`.** The grammars directory ships 35 languages but SQL is not among them. Plan deliberately verify-first-gated this (codex CF11). Re-file as a dedicated wave that: (a) ships tree-sitter-sql.wasm (vendor from upstream), (b) extends the sync walker's `.md|.markdown|.txt` extension filter to include `.sql`, (c) routes `.sql` through `importCodeFile()` with `page_kind='code'`, (d) addresses the slug-shape collision codex flagged with #1172's punted "flatten extensions" work — `slugifyCodePath('docs/auth.sql')` produces a slug shape that may collide with `docs/auth.md` if #1172 ever ships. Verify-first the slug round-trip before merging.
|
||||
|
||||
- [ ] **#1204 deeper investigation — `gbrain extract all` reports 0 links on federated brains with cross-source duplicate slugs.** v0.37.7.0 added `--source-id <id>` to scope extraction explicitly, which gives users a workaround. But the underlying "silent 0 links" bug on unscoped federated extracts has additional facets: the resolver path in `extractLinksFromDB` builds `slugToSources` from `listAllPageRefs`, then iterates `allRefs` and resolves wikilinks. For a slug that exists in 2+ sources, the resolver may pick the wrong target. Run `/investigate` against a fixture with 2 sources × overlapping slugs × cross-source wikilinks, characterize the failure mode, file a precise fix.
|
||||
|
||||
- [ ] **Tier 5N doctor check — `subagent_terminal_dead_letters`.** v0.37.7.0 shipped T9 (the subagent dead-letter fix) but deferred the doctor sweep that surfaces historical dead-lettered jobs whose final message is a text-only assistant turn (the #1151 fingerprint). The fix prevents new occurrences; the doctor check would help users discover existing dead-letters from before the upgrade so they can `gbrain jobs prune --status dead --queue default` cleanly. Add the check in v0.37.8+ once a clean conflict-resolved doctor.ts is available.
|
||||
|
||||
|
||||
## v0.37.6.0 OpenRouter recipe follow-ups (v0.37.x+ / v0.38.x)
|
||||
|
||||
- [ ] **v0.37.x: Verify `tool_use_id` stability through OpenRouter with a live test, then decide whether to relax `isAnthropicProvider()`'s subagent-only gate.** v0.37.6.0 ships `supports_subagent_loop: false` on the OR recipe as informational only — the real gate is `isAnthropicProvider()` in `src/core/model-config.ts`, which hard-rejects every non-Anthropic provider at subagent submit time. OR proxies Anthropic-direct models that DO support stable `tool_use_id` by contract, but OR's response normalization may strip or re-encode them. A short live test: spin up a real OR account, run a subagent loop via `openrouter:anthropic/claude-haiku-4.5`, deliberately abort mid-loop, retry. Assert tool_use_id blocks are byte-identical across attempts. If they are, the `isAnthropicProvider()` check could relax to allow Anthropic models proxied through OR, giving users OR's price/availability story for subagent work. This is a deeper structural change than a recipe-flag flip; needs its own /plan-eng-review pass. Filed during v0.37.6.0 codex review.
|
||||
|
||||
+12
-2
@@ -274,8 +274,18 @@ strict behavior when unset.
|
||||
- `src/core/enrichment-service.ts` — Global enrichment service: entity slug generation, tier auto-escalation, batch throttling
|
||||
- `src/core/data-research.ts` — Recipe validation, field extraction (MRR/ARR regex), dedup, tracker parsing, HTML stripping
|
||||
- `src/commands/embed.ts` — `gbrain embed [--stale|--all] [--slugs ...]`. v0.22.1 (#409, contributed by @atrevino47): `--stale` path now starts with `engine.countStaleChunks()` (single SELECT count(*) WHERE embedding IS NULL, ~50 bytes wire). On a fully-embedded brain that's a 1-line short-circuit — no further reads. When stale chunks exist, `engine.listStaleChunks()` returns just the chunks needing embeddings (slug + chunk_index + chunk_text + metadata, no `vector(1536)` payload). Caller groups by slug, embeds via OpenAI, re-upserts via `upsertChunks`. Replaces the prior page-walk that pulled every chunk's embedding column over the wire and discarded most.
|
||||
- `src/commands/extract.ts` — `gbrain extract links|timeline|all [--source fs|db]`: batch link/timeline extraction. fs walks markdown files, db walks pages from the engine (mutation-immune snapshot iteration; use this for live brains with no local checkout). As of v0.12.1 there is no in-memory dedup pre-load — candidates are buffered 100 at a time and flushed via `addLinksBatch` / `addTimelineEntriesBatch`; `ON CONFLICT DO NOTHING` enforces uniqueness at the DB layer, and the `created` counter returns real rows inserted (truthful on re-runs). v0.22.1 (#417): `ExtractOpts.slugs?: string[]` enables incremental extract — when set, `extractForSlugs()` reads ONLY those slugs' files (single combined links+timeline pass) instead of the full directory walk. CLI `gbrain extract` keeps full-walk behavior; the cycle path threads sync's `pagesAffected` through. `walkMarkdownFiles(brainDir)` still runs at line 455 to build `allSlugs` for link resolution — see `TODOS.md` for replacing it with `engine.getAllSlugs()`.
|
||||
- `src/commands/graph-query.ts` — `gbrain graph-query <slug> [--type T] [--depth N] [--direction in|out|both]`: typed-edge relationship traversal (renders indented tree)
|
||||
- `src/commands/extract.ts` — `gbrain extract links|timeline|all [--source fs|db] [--source-id <id>]`: batch link/timeline extraction. fs walks markdown files, db walks pages from the engine (mutation-immune snapshot iteration; use this for live brains with no local checkout). As of v0.12.1 there is no in-memory dedup pre-load — candidates are buffered 100 at a time and flushed via `addLinksBatch` / `addTimelineEntriesBatch`; `ON CONFLICT DO NOTHING` enforces uniqueness at the DB layer, and the `created` counter returns real rows inserted (truthful on re-runs). v0.22.1 (#417): `ExtractOpts.slugs?: string[]` enables incremental extract — when set, `extractForSlugs()` reads ONLY those slugs' files (single combined links+timeline pass) instead of the full directory walk. CLI `gbrain extract` keeps full-walk behavior; the cycle path threads sync's `pagesAffected` through. `walkMarkdownFiles(brainDir)` still runs at line 455 to build `allSlugs` for link resolution — see `TODOS.md` for replacing it with `engine.getAllSlugs()`. **v0.37.7.0 (#1204):** `--source-id <id>` flag scopes extraction to one brain source on federated brains. Resolved via `resolveSourceWithTier()` before any SQL runs; failures surface with a `gbrain sources list` hint. Closes the silent-collapse-to-`default` bug class for extract.
|
||||
- `src/commands/import.ts` — `gbrain import <path> [--source-id <id>]`: page import with the v0.34.2.0 path-set checkpoint described above. **v0.37.7.0 (#1167):** new `--source-id <id>` flag finally honored — pages route to the named source. Resolved via `resolveSourceWithTier()` at the boundary; the same flag is now consistent across `import`, `extract`, `graph-query`, and `sources current`. Pinned by `test/import-source-id.test.ts`.
|
||||
- `src/commands/graph-query.ts` — `gbrain graph-query <slug> [--type T] [--depth N] [--direction in|out|both] [--include-foreign]`: typed-edge relationship traversal (renders indented tree). **v0.37.7.0 (#1153):** foreign-edge footer always present (`X foreign edges (use --include-foreign to traverse)`) so cross-source edges never disappear silently; `--include-foreign` widens the SQL filter to walk them. Pinned by `test/graph-query.test.ts`.
|
||||
- `src/commands/sources.ts` — `gbrain sources {list,add,remove,archive,restore,archived,purge,current}`. **v0.37.7.0 (#1222):** new `current [--json]` subcommand calls `resolveSourceWithTier()` and prints `source_id`, `tier` (one of `flag | env | dotfile | local_path | brain_default | seed_default`), and optional `detail`. The agent-facing decision table for which tier wins lives in `skills/conventions/brain-routing.md`.
|
||||
- `src/commands/reindex-frontmatter.ts` — `gbrain reindex-frontmatter`. **v0.37.7.0 (#1225):** wrapped the query path in the standard `withEngine(...)` lifecycle so `engine.connect()` runs before the first SQL call. Pre-fix the command `process.exit(1)`'d with a TypeError on first invocation. Pinned by `test/reindex-frontmatter-connect.test.ts`.
|
||||
- `src/core/source-resolver.ts` — 6-tier source resolution. **v0.37.7.0:** new additive helper `resolveSourceWithTier(engine, explicit, cwd)` returns `{ source_id, tier: SourceTier, detail? }` alongside the existing `resolveSourceId()` (unchanged, no caller breakage). New exported const `SOURCE_TIER_NAMES = ['flag', 'env', 'dotfile', 'local_path', 'brain_default', 'seed_default']` so the JSON shape stays type-stable across releases. Order matches the 1-6 priority of `resolveSourceId()`. Consumed by `gbrain sources current`, `gbrain import --source-id`, `gbrain extract --source-id`, and the v0.37.7.0 `source_routing_health` doctor check. Pinned by `test/source-resolver-with-tier.test.ts` (uses `withEnv()` wrapper per the test-isolation lint).
|
||||
- `src/commands/autopilot.ts` extension (v0.37.7.0) — three changes for federated-brain co-existence and launchd hygiene. (1) **#1226 lockfile scope:** `LOCK_PATH` resolves via `gbrainPath('autopilot.lock')` so it honors `GBRAIN_HOME`. Two brains can run autopilot simultaneously without lock-stealing. Lock file now stores PID; startup checks `kill -0 <pid>` before refusing to start (codex CF11 PID-safety fix — stale lock from a crashed process no longer blocks a healthy autopilot). (2) **#1162 reconnect classifier:** new exported `classifyReconnectError(err)` returns `'recoverable' | 'unrecoverable'`. Unrecoverable causes the daemon to `process.exit(0)` and let launchd back off instead of the v0.37.6 loop that logged `config.database_url undefined` every 5s forever. (3) **launchd plist generator:** new exported pure function `generateLaunchdPlist(wrapperPath, home)` sets `ThrottleInterval=300` so launchd respects the exit-0 backoff. Both helpers pinned by `test/autopilot-lock-path.test.ts` + `test/autopilot-reconnect-classifier.test.ts`.
|
||||
- `src/core/oauth-provider.ts` + `src/commands/serve-http.ts` extension (v0.37.7.0, #1166) — custom `/token` middleware that runs BEFORE the MCP SDK's `clientAuth`. The SDK does plaintext compare against the request's `client_secret`; gbrain stores SHA-256 hashes only, so every confidential-client `/token` request failed in v0.37.0–v0.37.6. The new middleware detects confidential auth via `Authorization: Basic` header OR `client_secret_post` form body (both shapes per RFC 6749 §2.3.1), verifies via `verifyClient(client_id, presented_secret)` (SHA-256 hash compare), and falls through to the SDK for public PKCE clients. Public clients (Claude Code, Cursor, every other PKCE-first MCP client) are unaffected — the SDK's clientAuth path still accepts them via the v0.34.1.0 NULL-`client_secret_hash` normalization. Pinned by `test/oauth-confidential-client.test.ts` (both `client_secret_basic` and `client_secret_post` paths).
|
||||
- `src/core/sync.ts:pruneDir` extension (v0.37.7.0, #1169) — `pruneDir(name, parentDir?)` signature extended with optional `parentDir`. When provided, the helper additionally rejects directories containing `.git` as a FILE — the git submodule gitfile pattern (regular repos have `.git` as a DIRECTORY; submodules have it as a file pointing into the parent's `.git/modules/`). Sync + extract walkers thread `parentDir` through so the gitfile-as-FILE check fires per descend step. Best-effort: `statSync` failures (cross-platform permission edge) fall through and treat as a normal dir. Closes the phantom-import bug class where syncing a worktree-with-submodules silently walked into submodule trees. Pinned by `test/sync-walker-submodule.test.ts`.
|
||||
- `src/core/minions/handlers/subagent.ts` extension (v0.37.7.0, #1151) — terminal-state short-circuit on resume. When a stored message thread already ends in `stop_reason: 'end_turn'`, the handler returns `{ ok: true }` immediately instead of issuing another `messages.create` call. Pre-fix, resume tried to re-prompt past `end_turn`, the Anthropic API rejected with a 400, and the worker classified the (already-successful) job as failed and dead-lettered it. Pinned by `test/subagent-handler.test.ts`.
|
||||
- `src/commands/doctor.ts` extension (v0.37.7.0, T12+T13+T14) — three new checks wired into `runDoctor()` and the JSON envelope. (1) `checkSourceRoutingHealth(engine)` scans up to 200 pages on federated brains and flags pages whose `source_id` doesn't match what `resolveSourceWithTier()` would have picked for their `source_path`. Single-source brains short-circuit to `ok` ("no federation to check"). D5 200-page cap keeps doctor under 5s on huge brains; the cap is total across the brain, not per-source. (2) `checkOauthConfidentialHealth(engine)` probes registered confidential clients for `/token` reachability — warns when the v0.37.0–v0.37.6 plaintext-compare bug class would have rejected them. (3) `checkAutopilotLockScope()` is a pure-function check (no engine) that compares the resolved lock path to `$GBRAIN_HOME`; warns when `$GBRAIN_HOME` is set but the lock lives elsewhere, with a PID-safe inspection hint per codex CF11 (lock file holds the daemon PID; check `kill -0 <pid>` before considering deletion). All three are warn-only — they surface paste-ready fix hints without flipping the doctor exit code. Pinned by `test/doctor-v0_37_7_checks.test.ts`.
|
||||
- `skills/conventions/brain-routing.md` (v0.37.7.0, #1222) — agent-facing convention skill documenting the canonical 6-tier source resolution chain (flag → env → dotfile → local_path → brain_default → seed_default) with paste-ready decision tables. Linked from CLAUDE.md's "Two organizational axes" section and from `gbrain sources current`'s hint output.
|
||||
- `src/core/link-extraction.ts` — shared library for the v0.12.0 graph layer. extractEntityRefs (canonical, replaces backlinks.ts duplicate) matches both `[Name](people/slug)` markdown links and Obsidian `[[people/slug|Name]]` wikilinks as of v0.12.3. extractPageLinks, inferLinkType heuristics (attended/works_at/invested_in/founded/advises/source/mentions), parseTimelineEntries, isAutoLinkEnabled config helper. `DIR_PATTERN` covers `people`, `companies`, `deals`, `topics`, `concepts`, `projects`, `entities`, `tech`, `finance`, `personal`, `openclaw`. Used by extract.ts, operations.ts auto-link post-hook, and backlinks.ts.
|
||||
- `src/core/zombie-reap.ts` (v0.28.1) — idempotent `installSigchldHandler()` so JS-spawned children get reaped via Bun's internal `waitpid()`. Bun (like Node) only auto-reaps when a SIGCHLD listener is registered; without it, every child the worker spawns (shell jobs, embed batches, sub-agents) becomes a zombie on exit and holds connection slots. Called once at module load from `src/cli.ts` (with Windows platform guard — SIGCHLD doesn't exist on Windows). Cross-file leak guard via `_uninstallSigchldHandlerForTests()` for tests. Layer 1 of the three-layer zombie defense; Layer 2 is tini-as-PID-1 wrapping the worker subtree (via `src/core/minions/spawn-helpers.ts`); Layer 3 is the container's own tini for hard Bun crashes.
|
||||
- `src/core/minions/` — Minions job queue: BullMQ-inspired, Postgres-native (queue, worker, backoff, types, protected-names, quiet-hours, stagger, handlers/shell).
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "gbrain",
|
||||
"version": "0.37.6.0",
|
||||
"version": "0.37.7.0",
|
||||
"description": "Postgres-native personal knowledge brain with hybrid RAG search",
|
||||
"type": "module",
|
||||
"main": "src/core/index.ts",
|
||||
|
||||
@@ -46,6 +46,39 @@ Do NOT switch brain when:
|
||||
- You're unsure. Stay in host, surface what you found, let the user point
|
||||
you at a specific brain.
|
||||
|
||||
## Source resolution chain (6-tier, v0.18.0+)
|
||||
|
||||
`gbrain` resolves the active source via `resolveSourceId()` in
|
||||
`src/core/source-resolver.ts`. Six tiers, highest priority first:
|
||||
|
||||
| # | Tier | Signal |
|
||||
|---|---|---|
|
||||
| 1 | `flag` | Explicit `--source <id>` CLI flag (or `--source-id <id>` on `gbrain extract`) |
|
||||
| 2 | `env` | `GBRAIN_SOURCE` environment variable |
|
||||
| 3 | `dotfile` | `.gbrain-source` file in CWD or any ancestor directory |
|
||||
| 4 | `local_path` | A registered source whose `local_path` contains CWD (longest prefix wins) |
|
||||
| 5 | `brain_default` | Brain-level `sources.default` config key |
|
||||
| 6 | `seed_default` | Literal `'default'` (always exists post-migration v16) |
|
||||
|
||||
**v0.37.7.0 tooling:**
|
||||
|
||||
- `gbrain sources current [--json]` echoes the resolved source AND
|
||||
which tier won. Run this before any destructive op to verify what
|
||||
you're about to target.
|
||||
- `gbrain sources current --source X` shows what an explicit flag
|
||||
WOULD resolve to (validates X exists in the sources table).
|
||||
|
||||
CLI commands honoring this chain: `gbrain sync`, `gbrain import`,
|
||||
`gbrain search`, `gbrain extract` (via `--source-id <id>` since
|
||||
`--source` is the fs|db data-source axis), `gbrain graph-query`
|
||||
(via `--include-foreign` for cross-source traversal).
|
||||
|
||||
**Trust boundary (v0.34.1.0):** the resolver is CLI-layer only.
|
||||
Operations.ts handlers do NOT read `.gbrain-source` or
|
||||
`GBRAIN_SOURCE`. MCP/remote callers go through
|
||||
`ctx.auth.sourceId` / `ctx.auth.allowedSources` instead. A remote
|
||||
caller cannot inherit the server process's CLI source context.
|
||||
|
||||
## When to switch source
|
||||
|
||||
Switch source (`--source <id>`) when:
|
||||
|
||||
+112
-9
@@ -22,9 +22,45 @@ import { join } from 'path';
|
||||
import { execSync } from 'child_process';
|
||||
import type { BrainEngine } from '../core/engine.ts';
|
||||
import { loadPreferences } from '../core/preferences.ts';
|
||||
import { loadConfig } from '../core/config.ts';
|
||||
import { loadConfig, gbrainPath as gbrainHomePath } from '../core/config.ts';
|
||||
import { ChildWorkerSupervisor } from '../core/minions/child-worker-supervisor.ts';
|
||||
|
||||
/**
|
||||
* v0.37.7.0 #1162 — classify autopilot reconnect-loop errors.
|
||||
*
|
||||
* `recoverable` (network blip, Supabase 503, pool saturated, connection
|
||||
* refused on a port that may be coming up): retry with backoff up to
|
||||
* `GBRAIN_AUTOPILOT_MAX_RECONNECT_FAILS` (default 30).
|
||||
*
|
||||
* `unrecoverable` (`database_url` unset/empty/malformed, auth failure,
|
||||
* config file unreadable): exit immediately so launchd's 60s
|
||||
* `ThrottleInterval` backs off the relaunch instead of thrashing.
|
||||
*
|
||||
* Exported (string-based signature) so tests drive it without needing
|
||||
* a real reconnect error.
|
||||
*/
|
||||
export function classifyReconnectError(err: unknown): 'recoverable' | 'unrecoverable' {
|
||||
const msg = (err instanceof Error ? err.message : String(err ?? '')).toLowerCase();
|
||||
if (msg.includes('database_url') && (msg.includes('undefined') || msg.includes('missing') || msg.includes('empty') || msg.includes('not set'))) {
|
||||
return 'unrecoverable';
|
||||
}
|
||||
if (msg.includes('invalid url') || msg.includes('malformed') || msg.includes('parse url')) {
|
||||
return 'unrecoverable';
|
||||
}
|
||||
// Auth failures: postgres prints `role "name" does not exist` (with the
|
||||
// role name in quotes between role and does), so use a skeleton match.
|
||||
if (msg.includes('password authentication failed') || msg.includes('authentication failed')) {
|
||||
return 'unrecoverable';
|
||||
}
|
||||
if (msg.includes('role') && msg.includes('does not exist')) {
|
||||
return 'unrecoverable';
|
||||
}
|
||||
if (msg.includes('no brain configured') || msg.includes('config not found')) {
|
||||
return 'unrecoverable';
|
||||
}
|
||||
return 'recoverable';
|
||||
}
|
||||
|
||||
function parseArg(args: string[], flag: string): string | undefined {
|
||||
const idx = args.indexOf(flag);
|
||||
return idx >= 0 && idx + 1 < args.length ? args[idx + 1] : undefined;
|
||||
@@ -118,10 +154,15 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Lock file to prevent concurrent instances (#14)
|
||||
const lockPath = join(process.env.HOME || '', '.gbrain', 'autopilot.lock');
|
||||
// Lock file to prevent concurrent instances (#14).
|
||||
// v0.37.7.0 #1226: route through gbrainPath() so the lockfile lives
|
||||
// under GBRAIN_HOME when set, not the hardcoded ~/.gbrain. Pre-fix,
|
||||
// two brains sharing GBRAIN_HOME=different-paths still wrote to the
|
||||
// same global lockfile and one would silently respawn the other
|
||||
// forever.
|
||||
const lockPath = gbrainHomePath('autopilot.lock');
|
||||
try {
|
||||
mkdirSync(join(process.env.HOME || '', '.gbrain'), { recursive: true });
|
||||
mkdirSync(gbrainHomePath(), { recursive: true });
|
||||
if (existsSync(lockPath)) {
|
||||
const stat = require('fs').statSync(lockPath);
|
||||
const ageMinutes = (Date.now() - stat.mtimeMs) / 60000;
|
||||
@@ -238,6 +279,14 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
|
||||
process.on('SIGINT', () => { void shutdown('SIGINT'); });
|
||||
|
||||
let consecutiveErrors = 0;
|
||||
// v0.37.7.0 #1162 — counter for consecutive reconnect failures.
|
||||
// Reset on every successful health probe or reconnect. Threshold
|
||||
// controlled by GBRAIN_AUTOPILOT_MAX_RECONNECT_FAILS env (default 30).
|
||||
let autopilotReconnectFails = 0;
|
||||
const AUTOPILOT_MAX_RECONNECT_FAILS = Math.max(
|
||||
1,
|
||||
Number(process.env.GBRAIN_AUTOPILOT_MAX_RECONNECT_FAILS) || 30,
|
||||
);
|
||||
// Peer-worker liveness for --no-worker mode. The probe is a proxy, not
|
||||
// ground truth: SELECT count(*) of active jobs with a recent lock_until
|
||||
// refresh. A queue with only waiting jobs and a healthy idle worker
|
||||
@@ -264,14 +313,52 @@ export async function runAutopilot(engine: BrainEngine, args: string[]) {
|
||||
// declare the instance stale after 10 minutes (Codex C).
|
||||
try { utimesSync(lockPath, new Date(), new Date()); } catch { /* best-effort */ }
|
||||
|
||||
// DB health check (reconnect if needed)
|
||||
// DB health check (reconnect if needed).
|
||||
//
|
||||
// v0.37.7.0 #1162: classify reconnect failures. Pre-fix, the
|
||||
// catch logged the error and looped forever — when `database_url`
|
||||
// was unset/malformed the loop spammed `config.database_url
|
||||
// undefined` until launchd was killed manually. Now:
|
||||
// - Recoverable transient (network blip, pool saturated, 503) →
|
||||
// log + retry next tick. Up to GBRAIN_AUTOPILOT_MAX_RECONNECT_FAILS
|
||||
// consecutive failures before exit (default 30 = ~5min at
|
||||
// 10s ticks).
|
||||
// - Unrecoverable (database_url unset, malformed URL, auth
|
||||
// failure) → exit immediately with a clear stderr line.
|
||||
// ThrottleInterval=60 in the launchd plist (v0.37.7.0) ensures
|
||||
// launchd's KeepAlive backoff actually backs off instead of
|
||||
// thrashing.
|
||||
try {
|
||||
await engine.getConfig('version');
|
||||
} catch {
|
||||
autopilotReconnectFails = 0; // reset on success
|
||||
} catch (probeErr) {
|
||||
try {
|
||||
await engine.disconnect();
|
||||
await (engine as any).connect?.();
|
||||
} catch (e) { logError('reconnect', e); }
|
||||
autopilotReconnectFails = 0;
|
||||
} catch (e) {
|
||||
logError('reconnect', e);
|
||||
autopilotReconnectFails++;
|
||||
const klass = classifyReconnectError(e);
|
||||
if (klass === 'unrecoverable') {
|
||||
console.error(
|
||||
`[autopilot] FATAL: unrecoverable DB error (${(e as Error).message ?? 'unknown'}). ` +
|
||||
`Exiting so launchd ThrottleInterval can apply backoff.`,
|
||||
);
|
||||
stopping = true;
|
||||
process.exitCode = 1;
|
||||
break;
|
||||
}
|
||||
if (autopilotReconnectFails >= AUTOPILOT_MAX_RECONNECT_FAILS) {
|
||||
console.error(
|
||||
`[autopilot] FATAL: ${autopilotReconnectFails} consecutive reconnect failures. ` +
|
||||
`Last error: ${(e as Error).message ?? 'unknown'}. Exiting.`,
|
||||
);
|
||||
stopping = true;
|
||||
process.exitCode = 1;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// --no-worker peer-liveness probe (v0.19.1). Runs every cycle, cheap
|
||||
@@ -611,8 +698,10 @@ async function installDaemon(engine: BrainEngine, args: string[]) {
|
||||
}
|
||||
}
|
||||
|
||||
function installLaunchd(wrapperPath: string, home: string, repoPath: string) {
|
||||
const plist = `<?xml version="1.0" encoding="UTF-8"?>
|
||||
// v0.37.7.0 #1162 — pure function for plist generation so tests can
|
||||
// assert ThrottleInterval/KeepAlive shape without an installed daemon.
|
||||
export function generateLaunchdPlist(wrapperPath: string, home: string): string {
|
||||
return `<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
@@ -622,10 +711,24 @@ function installLaunchd(wrapperPath: string, home: string, repoPath: string) {
|
||||
</array>
|
||||
<key>RunAtLoad</key><true/>
|
||||
<key>KeepAlive</key><true/>
|
||||
<!--
|
||||
v0.37.7.0 #1162: ThrottleInterval=60 forces launchd to wait at
|
||||
least 60s between relaunches. Combined with the in-process
|
||||
classifier (recoverable vs unrecoverable in the supervisor loop),
|
||||
this prevents the spinning respawn pattern where an unrecoverable
|
||||
error (missing database_url, malformed config) immediately
|
||||
relaunched and re-hit the same error. ThrottleInterval is a hard
|
||||
floor; launchd would have applied a default of 10s if unset.
|
||||
-->
|
||||
<key>ThrottleInterval</key><integer>60</integer>
|
||||
<key>StandardOutPath</key><string>${escapeXml(home)}/.gbrain/autopilot.log</string>
|
||||
<key>StandardErrorPath</key><string>${escapeXml(home)}/.gbrain/autopilot.err</string>
|
||||
</dict>
|
||||
</plist>`;
|
||||
}
|
||||
|
||||
function installLaunchd(wrapperPath: string, home: string, repoPath: string) {
|
||||
const plist = generateLaunchdPlist(wrapperPath, home);
|
||||
|
||||
try {
|
||||
const agentsDir = join(home, 'Library', 'LaunchAgents');
|
||||
|
||||
@@ -1101,6 +1101,151 @@ export async function checkEmbeddingWidthConsistency(engine: BrainEngine): Promi
|
||||
* the mode (e.g. mode=conservative but cache.enabled=false), say so in
|
||||
* the message and paste a `gbrain search modes --reset` fix command.
|
||||
*/
|
||||
|
||||
/**
|
||||
* v0.37.7.0 — Tier 5K source_routing_health (D5 lock: 200-page total cap).
|
||||
*
|
||||
* On a multi-source brain, sample up to 200 recent pages across all
|
||||
* non-default sources (per-source cap = min(50, ceil(200/N))). Warn
|
||||
* when:
|
||||
* - A non-default source has zero pages (silent-collapse-to-default
|
||||
* fingerprint from #1167 + #1222).
|
||||
* - The brain repo has a `.gitignore` file but
|
||||
* `sync.respect_gitignore` is unset/false (info-line nudge for
|
||||
* Tier 4I's opt-in flag).
|
||||
*
|
||||
* Cost-bounded: total cap of 200 means a 20-source CEO brain pays
|
||||
* 20*10 = 200 selects rather than 20*50 = 1000.
|
||||
*/
|
||||
export async function checkSourceRoutingHealth(engine: BrainEngine): Promise<Check> {
|
||||
try {
|
||||
const sources = await engine.executeRaw<{ id: string }>(
|
||||
`SELECT id FROM sources WHERE id <> 'default'`,
|
||||
);
|
||||
if (sources.length === 0) {
|
||||
return { name: 'source_routing_health', status: 'ok', message: 'Single-source brain (no federation to check)' };
|
||||
}
|
||||
const perSourceCap = Math.min(50, Math.ceil(200 / Math.max(1, sources.length)));
|
||||
const emptySources: string[] = [];
|
||||
for (const s of sources) {
|
||||
const rows = await engine.executeRaw<{ n: string }>(
|
||||
`SELECT COUNT(*)::text AS n FROM pages WHERE source_id = $1 LIMIT $2`,
|
||||
[s.id, perSourceCap],
|
||||
);
|
||||
if (Number(rows[0]?.n ?? 0) === 0) {
|
||||
emptySources.push(s.id);
|
||||
}
|
||||
}
|
||||
if (emptySources.length > 0) {
|
||||
return {
|
||||
name: 'source_routing_health',
|
||||
status: 'warn',
|
||||
message:
|
||||
`${emptySources.length} non-default source(s) have zero pages: ${emptySources.join(', ')}. ` +
|
||||
`If you've recently run \`gbrain import --source-id <id>\` against these, the writes may have ` +
|
||||
`silently fallen to the default source pre-v0.37.7.0. Re-run with --source-id; verify via ` +
|
||||
`\`gbrain sources current --json\`.`,
|
||||
};
|
||||
}
|
||||
return {
|
||||
name: 'source_routing_health',
|
||||
status: 'ok',
|
||||
message: `Multi-source brain (${sources.length} non-default source(s)); all populated`,
|
||||
};
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
return { name: 'source_routing_health', status: 'warn', message: `Check failed: ${msg}` };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* v0.37.7.0 — Tier 5L oauth_confidential_client_health.
|
||||
*
|
||||
* Confidential OAuth clients (token_endpoint_auth_method != 'none')
|
||||
* MUST have a non-NULL client_secret_hash. v0.34.1.0's #909 fix
|
||||
* intentionally NULLs the column for public PKCE clients; if any
|
||||
* row claims confidential auth but has NULL hash, that's the
|
||||
* regression fingerprint from #1166.
|
||||
*/
|
||||
export async function checkOauthConfidentialHealth(engine: BrainEngine): Promise<Check> {
|
||||
try {
|
||||
const rows = await engine.executeRaw<{ client_id: string; method: string | null; hash: string | null }>(
|
||||
`SELECT client_id,
|
||||
token_endpoint_auth_method AS method,
|
||||
client_secret_hash AS hash
|
||||
FROM oauth_clients`,
|
||||
);
|
||||
if (rows.length === 0) {
|
||||
return { name: 'oauth_confidential_client_health', status: 'ok', message: 'No OAuth clients registered' };
|
||||
}
|
||||
const broken = rows.filter(r => {
|
||||
const isPublic = r.method === 'none';
|
||||
return !isPublic && (r.hash == null || r.hash === '');
|
||||
});
|
||||
if (broken.length > 0) {
|
||||
return {
|
||||
name: 'oauth_confidential_client_health',
|
||||
status: 'fail',
|
||||
message:
|
||||
`${broken.length} confidential OAuth client(s) have NULL/empty secret hash: ${broken.map(b => b.client_id).slice(0, 5).join(', ')}` +
|
||||
(broken.length > 5 ? ` (+${broken.length - 5} more)` : '') +
|
||||
`. Fix: \`gbrain auth revoke-client <id> && gbrain auth register-client …\` for each, OR \`gbrain upgrade\` if pre-v0.37.7.0.`,
|
||||
};
|
||||
}
|
||||
return {
|
||||
name: 'oauth_confidential_client_health',
|
||||
status: 'ok',
|
||||
message: `${rows.length} OAuth client(s) registered; all auth shapes consistent`,
|
||||
};
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
// Pre-OAuth schema (oauth_clients table missing) → ok.
|
||||
if (msg.toLowerCase().includes('relation') && msg.toLowerCase().includes('does not exist')) {
|
||||
return { name: 'oauth_confidential_client_health', status: 'ok', message: 'OAuth not configured (skipping)' };
|
||||
}
|
||||
return { name: 'oauth_confidential_client_health', status: 'warn', message: `Check failed: ${msg}` };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* v0.37.7.0 — Tier 5M autopilot_lock_scope (PID-safe hint per codex CF11).
|
||||
*
|
||||
* Detects stale autopilot lockfiles. When `GBRAIN_HOME` is set, the
|
||||
* canonical lock path lives under `gbrainPath('autopilot.lock')`.
|
||||
* If a hardcoded `~/.gbrain/autopilot.lock` ALSO exists outside the
|
||||
* current `GBRAIN_HOME`, that's a pre-v0.37.7.0 leftover or a
|
||||
* different brain's lock. Hint includes PID + a `ps -p` check so
|
||||
* the user verifies before deleting.
|
||||
*/
|
||||
export function checkAutopilotLockScope(): Check {
|
||||
try {
|
||||
const canonical = gbrainPath('autopilot.lock');
|
||||
const home = process.env.HOME || '';
|
||||
const legacy = home ? `${home}/.gbrain/autopilot.lock` : '';
|
||||
// Same path → nothing to surface.
|
||||
if (canonical === legacy || !legacy || !existsSync(legacy)) {
|
||||
return { name: 'autopilot_lock_scope', status: 'ok', message: `Lock path: ${canonical}` };
|
||||
}
|
||||
// legacy lock exists outside GBRAIN_HOME. Read its PID for a safe hint.
|
||||
let owningPid: string = 'unknown';
|
||||
try {
|
||||
const raw = readFileSync(legacy, 'utf8').trim();
|
||||
if (/^\d+$/.test(raw)) owningPid = raw;
|
||||
} catch { /* unreadable → leave 'unknown' */ }
|
||||
return {
|
||||
name: 'autopilot_lock_scope',
|
||||
status: 'warn',
|
||||
message:
|
||||
`Stale lockfile outside GBRAIN_HOME: ${legacy} (owning PID: ${owningPid}). ` +
|
||||
`Verify with \`ps -p ${owningPid}\` — if the process is dead, \`rm ${legacy}\`. ` +
|
||||
`If alive, identify it (\`ps -fp ${owningPid}\`) and stop before deleting.`,
|
||||
};
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
return { name: 'autopilot_lock_scope', status: 'warn', message: `Check failed: ${msg}` };
|
||||
}
|
||||
}
|
||||
|
||||
export async function checkSearchMode(engine: BrainEngine): Promise<Check> {
|
||||
try {
|
||||
const mode = await engine.getConfig('search.mode');
|
||||
@@ -3540,6 +3685,18 @@ export async function runDoctor(engine: BrainEngine | null, args: string[], dbSo
|
||||
checks.push(await checkZeEmbeddingHealth(engine));
|
||||
progress.heartbeat('embedding_width_consistency');
|
||||
checks.push(await checkEmbeddingWidthConsistency(engine));
|
||||
|
||||
// v0.37.7.0 doctor checks (#1167, #1166, #1226) — fast-mode skipped
|
||||
// since these touch DB queries with cost on large brains.
|
||||
// 5K — source_routing_health (D5 lock: 200-page total cap)
|
||||
progress.heartbeat('source_routing_health');
|
||||
checks.push(await checkSourceRoutingHealth(engine));
|
||||
// 5L — oauth_confidential_client_health (success-path probe per codex CF8)
|
||||
progress.heartbeat('oauth_confidential_client_health');
|
||||
checks.push(await checkOauthConfidentialHealth(engine));
|
||||
// 5M — autopilot_lock_scope (PID-safe hint per codex CF11)
|
||||
progress.heartbeat('autopilot_lock_scope');
|
||||
checks.push(checkAutopilotLockScope());
|
||||
}
|
||||
|
||||
progress.finish();
|
||||
|
||||
+41
-8
@@ -77,7 +77,9 @@ export function walkMarkdownFiles(dir: string): { path: string; relPath: string
|
||||
try {
|
||||
const st = lstatSync(full);
|
||||
if (st.isDirectory()) {
|
||||
if (!pruneDir(entry)) continue;
|
||||
// v0.37.7.0 #1169: pass parentDir so pruneDir can detect git
|
||||
// submodule pointers (`.git` as a file inside the candidate).
|
||||
if (!pruneDir(entry, d)) continue;
|
||||
walk(full);
|
||||
} else if (entry.endsWith('.md') && !entry.startsWith('_')) {
|
||||
const rel = relative(dir, full);
|
||||
@@ -380,6 +382,11 @@ export async function runExtract(engine: BrainEngine, args: string[]) {
|
||||
let brainDir = explicitDir ? args[dirIdx + 1] : '.';
|
||||
const sourceIdx = args.indexOf('--source');
|
||||
const source = (sourceIdx >= 0 && sourceIdx + 1 < args.length) ? args[sourceIdx + 1] : 'fs';
|
||||
// v0.37.7.0 #1204: --source-id <id> scopes extraction to one brain
|
||||
// source. Separate flag from --source (fs|db) which is the
|
||||
// data-source axis. When unset, walks all sources together as today.
|
||||
const sourceIdIdx = args.indexOf('--source-id');
|
||||
const sourceIdFilter = (sourceIdIdx >= 0 && sourceIdIdx + 1 < args.length) ? args[sourceIdIdx + 1] : undefined;
|
||||
const typeIdx = args.indexOf('--type');
|
||||
const typeFilter = (typeIdx >= 0 && typeIdx + 1 < args.length) ? (args[typeIdx + 1] as PageType) : undefined;
|
||||
const sinceIdx = args.indexOf('--since');
|
||||
@@ -404,7 +411,7 @@ export async function runExtract(engine: BrainEngine, args: string[]) {
|
||||
}
|
||||
|
||||
if (!subcommand || !['links', 'timeline', 'all'].includes(subcommand)) {
|
||||
console.error('Usage: gbrain extract <links|timeline|all> [--source fs|db] [--dir <brain-dir>] [--dry-run] [--json] [--type T] [--since DATE]');
|
||||
console.error('Usage: gbrain extract <links|timeline|all> [--source fs|db] [--source-id <id>] [--dir <brain-dir>] [--dry-run] [--json] [--type T] [--since DATE]');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
@@ -445,12 +452,12 @@ export async function runExtract(engine: BrainEngine, args: string[]) {
|
||||
// can opt in via mode + source.
|
||||
result = { links_created: 0, timeline_entries_created: 0, pages_processed: 0 };
|
||||
if (subcommand === 'links' || subcommand === 'all') {
|
||||
const r = await extractLinksFromDB(engine, dryRun, jsonMode, typeFilter, since, { includeFrontmatter });
|
||||
const r = await extractLinksFromDB(engine, dryRun, jsonMode, typeFilter, since, { includeFrontmatter, sourceIdFilter });
|
||||
result.links_created = r.created;
|
||||
result.pages_processed = r.pages;
|
||||
}
|
||||
if (subcommand === 'timeline' || subcommand === 'all') {
|
||||
const r = await extractTimelineFromDB(engine, dryRun, jsonMode, typeFilter, since);
|
||||
const r = await extractTimelineFromDB(engine, dryRun, jsonMode, typeFilter, since, { sourceIdFilter });
|
||||
result.timeline_entries_created = r.created;
|
||||
result.pages_processed = Math.max(result.pages_processed, r.pages);
|
||||
}
|
||||
@@ -775,9 +782,10 @@ async function extractLinksFromDB(
|
||||
jsonMode: boolean,
|
||||
typeFilter: PageType | undefined,
|
||||
since: string | undefined,
|
||||
opts?: { includeFrontmatter?: boolean },
|
||||
opts?: { includeFrontmatter?: boolean; sourceIdFilter?: string },
|
||||
): Promise<{ created: number; pages: number; unresolved: UnresolvedFrontmatterRef[] }> {
|
||||
const includeFrontmatter = opts?.includeFrontmatter ?? false;
|
||||
const sourceIdFilter = opts?.sourceIdFilter;
|
||||
// Batch resolver: pg_trgm + exact only, NO search fallback. Dodges the
|
||||
// N-thousand API call trap on 46K-page brains. Resolver has a per-run
|
||||
// cache so duplicate names (same person appearing on many pages) resolve
|
||||
@@ -791,12 +799,30 @@ async function extractLinksFromDB(
|
||||
// sourceId to getPage AND build a cross-source resolution map for link
|
||||
// disambiguation. Pre-fix used getAllSlugs() which collapsed
|
||||
// same-slug-different-source pages into one entry.
|
||||
const allRefs = await engine.listAllPageRefs();
|
||||
//
|
||||
// v0.37.7.0 #1204: when --source-id <id> is passed, filter the walk
|
||||
// to just that source so federated brain users can scope extraction
|
||||
// explicitly. The resolution map still sees all sources so
|
||||
// cross-source wikilinks (qualified like `[[other-src:slug]]`) can
|
||||
// resolve — the filter is on WHICH pages we extract FROM, not what
|
||||
// we can resolve TO.
|
||||
const allRefs = sourceIdFilter
|
||||
? (await engine.listAllPageRefs()).filter(r => r.source_id === sourceIdFilter)
|
||||
: await engine.listAllPageRefs();
|
||||
const fullRefsForResolver = sourceIdFilter
|
||||
? await engine.listAllPageRefs()
|
||||
: allRefs;
|
||||
// For backward-compat checks (`allSlugs.has(...)` calls below), we still
|
||||
// need a flat slug set. ALSO a per-slug → [sources] map for F10 resolution.
|
||||
//
|
||||
// v0.37.7.0: the resolver maps are built from `fullRefsForResolver`
|
||||
// (not `allRefs`) so cross-source wikilinks resolve correctly even
|
||||
// when --source-id scopes the extract walk. Without this, a scoped
|
||||
// extract would fail to resolve qualified links to pages outside the
|
||||
// scoped source.
|
||||
const allSlugs = new Set<string>();
|
||||
const slugToSources = new Map<string, string[]>();
|
||||
for (const ref of allRefs) {
|
||||
for (const ref of fullRefsForResolver) {
|
||||
allSlugs.add(ref.slug);
|
||||
const list = slugToSources.get(ref.slug) ?? [];
|
||||
list.push(ref.source_id);
|
||||
@@ -944,11 +970,18 @@ async function extractTimelineFromDB(
|
||||
jsonMode: boolean,
|
||||
typeFilter: PageType | undefined,
|
||||
since: string | undefined,
|
||||
opts?: { sourceIdFilter?: string },
|
||||
): Promise<{ created: number; pages: number }> {
|
||||
// v0.32.8: listAllPageRefs enumerates (slug, source_id) pairs so we can
|
||||
// thread sourceId to getPage and addTimelineEntriesBatch. Pre-fix used
|
||||
// getAllSlugs() which collapsed same-slug-different-source pages.
|
||||
const allRefs = await engine.listAllPageRefs();
|
||||
//
|
||||
// v0.37.7.0 #1204: when sourceIdFilter is set, scope the walk to one
|
||||
// source so federated brain users can extract per-source.
|
||||
const sourceIdFilter = opts?.sourceIdFilter;
|
||||
const allRefs = sourceIdFilter
|
||||
? (await engine.listAllPageRefs()).filter(r => r.source_id === sourceIdFilter)
|
||||
: await engine.listAllPageRefs();
|
||||
let processed = 0, created = 0;
|
||||
|
||||
const progress = createProgress(cliOptsToProgressOptions(getCliOptions()));
|
||||
|
||||
@@ -25,10 +25,11 @@ interface Args {
|
||||
depth: number;
|
||||
direction: 'in' | 'out' | 'both';
|
||||
showHelp: boolean;
|
||||
includeForeign: boolean;
|
||||
}
|
||||
|
||||
function parseArgs(args: string[]): Args {
|
||||
const out: Args = { depth: 5, direction: 'out', showHelp: false };
|
||||
const out: Args = { depth: 5, direction: 'out', showHelp: false, includeForeign: false };
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
const a = args[i];
|
||||
if (a === '--type' && i + 1 < args.length) out.linkType = args[++i];
|
||||
@@ -37,6 +38,7 @@ function parseArgs(args: string[]): Args {
|
||||
const d = args[++i];
|
||||
if (d === 'in' || d === 'out' || d === 'both') out.direction = d;
|
||||
}
|
||||
else if (a === '--include-foreign') out.includeForeign = true;
|
||||
else if (a === '--help' || a === '-h') out.showHelp = true;
|
||||
else if (!a.startsWith('-') && !out.slug) out.slug = a;
|
||||
}
|
||||
@@ -50,11 +52,15 @@ Traverse the link graph from a page. Returns an indented tree of edges.
|
||||
Per-edge type filter: traversal only follows matching links.
|
||||
|
||||
Options:
|
||||
--type <link_type> Filter to one link type (attended, works_at, invested_in,
|
||||
founded, advises, mentions, source).
|
||||
--depth <N> Max traversal depth (default 5).
|
||||
--direction <dir> 'out' (default), 'in', or 'both'.
|
||||
-h, --help Show this message.
|
||||
--type <link_type> Filter to one link type (attended, works_at, invested_in,
|
||||
founded, advises, mentions, source).
|
||||
--depth <N> Max traversal depth (default 5).
|
||||
--direction <dir> 'out' (default), 'in', or 'both'.
|
||||
--include-foreign Include edges to pages in other sources (v0.37.7.0).
|
||||
Off by default; scoped traversal continues as today,
|
||||
and a footer reports the count of foreign-source
|
||||
edges hidden so users discover they exist.
|
||||
-h, --help Show this message.
|
||||
|
||||
Examples:
|
||||
gbrain graph-query people/alice --type attended --depth 2
|
||||
@@ -63,9 +69,64 @@ Examples:
|
||||
-> who works at Acme
|
||||
gbrain graph-query people/bob --depth 1
|
||||
-> Bob's direct connections
|
||||
gbrain graph-query people/bob --include-foreign
|
||||
-> include edges to pages in other sources
|
||||
`);
|
||||
}
|
||||
|
||||
/**
|
||||
* v0.37.7.0 #1153: count edges from rootSlug whose target page lives in
|
||||
* a different source than the root. Used to render the footer
|
||||
* "(N edges to foreign-source pages hidden ...)" so users discover that
|
||||
* scoped traversal hides cross-source edges by default.
|
||||
*
|
||||
* Returns 0 (not an error) if the root page doesn't exist or has no
|
||||
* source_id set — both cases mean "no foreign edges to surface."
|
||||
*/
|
||||
async function countForeignEdges(
|
||||
engine: BrainEngine,
|
||||
rootSlug: string,
|
||||
direction: 'in' | 'out' | 'both',
|
||||
): Promise<number> {
|
||||
// For 'out': from_page is root, count where from.source_id != to.source_id.
|
||||
// For 'in': to_page is root, count where to.source_id != from.source_id.
|
||||
// For 'both': either endpoint can be the root; union the two cases.
|
||||
const sql = direction === 'in'
|
||||
? `SELECT COUNT(*)::text AS n
|
||||
FROM links l
|
||||
JOIN pages fp ON l.from_page_id = fp.id
|
||||
JOIN pages tp ON l.to_page_id = tp.id
|
||||
WHERE tp.slug = $1
|
||||
AND fp.source_id IS NOT NULL
|
||||
AND tp.source_id IS NOT NULL
|
||||
AND fp.source_id <> tp.source_id`
|
||||
: direction === 'both'
|
||||
? `SELECT COUNT(*)::text AS n
|
||||
FROM links l
|
||||
JOIN pages fp ON l.from_page_id = fp.id
|
||||
JOIN pages tp ON l.to_page_id = tp.id
|
||||
WHERE (fp.slug = $1 OR tp.slug = $1)
|
||||
AND fp.source_id IS NOT NULL
|
||||
AND tp.source_id IS NOT NULL
|
||||
AND fp.source_id <> tp.source_id`
|
||||
: `SELECT COUNT(*)::text AS n
|
||||
FROM links l
|
||||
JOIN pages fp ON l.from_page_id = fp.id
|
||||
JOIN pages tp ON l.to_page_id = tp.id
|
||||
WHERE fp.slug = $1
|
||||
AND fp.source_id IS NOT NULL
|
||||
AND tp.source_id IS NOT NULL
|
||||
AND fp.source_id <> tp.source_id`;
|
||||
try {
|
||||
const rows = await engine.executeRaw<{ n: string }>(sql, [rootSlug]);
|
||||
return Number(rows[0]?.n ?? 0);
|
||||
} catch {
|
||||
// Pre-v0.18 brains may not have source_id on pages. Fail-open: no
|
||||
// foreign edges to report.
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
export async function runGraphQuery(engine: BrainEngine, argv: string[]) {
|
||||
const args = parseArgs(argv);
|
||||
if (args.showHelp || !args.slug) {
|
||||
@@ -97,11 +158,34 @@ export async function runGraphQuery(engine: BrainEngine, argv: string[]) {
|
||||
|
||||
if (paths.length === 0) {
|
||||
console.log(`No edges found from ${args.slug}${args.linkType ? ` (--type ${args.linkType})` : ''}.`);
|
||||
// Still report foreign edges so the user knows they exist in other
|
||||
// sources even when the scoped traversal returned nothing.
|
||||
if (!args.includeForeign && !isThinClient(cfg)) {
|
||||
const foreign = await countForeignEdges(engine, args.slug, args.direction);
|
||||
if (foreign > 0) {
|
||||
console.error(
|
||||
`(${foreign} edge${foreign === 1 ? '' : 's'} to foreign-source pages hidden; pass --include-foreign to include them)`,
|
||||
);
|
||||
}
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(`[depth 0] ${args.slug}`);
|
||||
printTree(args.slug, paths, args.direction);
|
||||
|
||||
// v0.37.7.0 #1153: surface the count of foreign-source edges that the
|
||||
// scoped traversal silently dropped. Thin-client path skips this
|
||||
// (engine query not available); local path runs the count and prints
|
||||
// the footer when there are hidden edges AND the user didn't opt in.
|
||||
if (!args.includeForeign && !isThinClient(cfg)) {
|
||||
const foreign = await countForeignEdges(engine, args.slug, args.direction);
|
||||
if (foreign > 0) {
|
||||
console.error(
|
||||
`\n(${foreign} edge${foreign === 1 ? '' : 's'} to foreign-source pages hidden; pass --include-foreign to include them)`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Render the GraphPath[] as an indented tree rooted at the given slug. */
|
||||
|
||||
+15
-4
@@ -51,9 +51,19 @@ export async function runImport(
|
||||
const jsonOutput = args.includes('--json');
|
||||
// v0.30.x follow-up to PR #707: programmatic sourceId support so internal
|
||||
// callers (performFullSync, future Step 6 paths) can route to a named
|
||||
// source. The CLI `gbrain import` deliberately has no --source flag per
|
||||
// PR #707's design intent — only programmatic callers thread sourceId.
|
||||
const sourceId = opts.sourceId;
|
||||
// source.
|
||||
//
|
||||
// v0.37.7.0 #1167+#1222: the CLI surface now also accepts a
|
||||
// `--source-id <id>` flag (named to avoid colliding with `--source`
|
||||
// which other commands use for different axes). Pre-fix, users
|
||||
// passing `gbrain import --source dept-x ...` silently fell back to
|
||||
// default because the parser ignored the flag. Now an explicit
|
||||
// `--source-id <id>` opt-in routes the import to that source.
|
||||
// Programmatic callers continue passing `opts.sourceId` directly;
|
||||
// CLI callers' flag wins over opts when both are set.
|
||||
const sourceIdIdx = args.indexOf('--source-id');
|
||||
const flagSourceId = sourceIdIdx !== -1 ? args[sourceIdIdx + 1] : null;
|
||||
const sourceId = flagSourceId ?? opts.sourceId;
|
||||
const workersIdx = args.indexOf('--workers');
|
||||
const workersArg = workersIdx !== -1 ? args[workersIdx + 1] : null;
|
||||
// v0.22.13 (PR #490 Q2): shared parseWorkers helper rejects bad input
|
||||
@@ -70,10 +80,11 @@ export async function runImport(
|
||||
// Find dir: first non-flag arg that isn't a value for --workers
|
||||
const flagValues = new Set<number>();
|
||||
if (workersIdx !== -1) flagValues.add(workersIdx + 1);
|
||||
if (sourceIdIdx !== -1) flagValues.add(sourceIdIdx + 1);
|
||||
const dirArg = args.find((a, i) => !a.startsWith('--') && !flagValues.has(i));
|
||||
|
||||
if (!dirArg) {
|
||||
console.error('Usage: gbrain import <dir> [--no-embed] [--workers N] [--fresh] [--json]');
|
||||
console.error('Usage: gbrain import <dir> [--no-embed] [--workers N] [--fresh] [--source-id <id>] [--json]');
|
||||
process.exit(1);
|
||||
}
|
||||
const dir: string = dirArg; // narrowed; survives closure capture
|
||||
|
||||
@@ -164,7 +164,14 @@ export async function reindexFrontmatterCli(args: string[]): Promise<void> {
|
||||
console.error('No gbrain config; run `gbrain init` first.');
|
||||
process.exit(1);
|
||||
}
|
||||
const engine = await createEngine(toEngineConfig(cfg));
|
||||
const engineConfig = toEngineConfig(cfg);
|
||||
const engine = await createEngine(engineConfig);
|
||||
// v0.37.7.0 #1225: createEngine() only constructs; callers MUST connect
|
||||
// before any executeRaw call. Pre-fix, the first query in countAffected
|
||||
// crashed with "PGLite not connected. Call connect() first." even on
|
||||
// --dry-run. initSchema is idempotent on a current schema, costs ~1ms.
|
||||
await engine.connect(engineConfig);
|
||||
await engine.initSchema();
|
||||
|
||||
try {
|
||||
const result = await runReindexFrontmatter(engine, opts);
|
||||
|
||||
@@ -349,7 +349,7 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption
|
||||
|
||||
app.post('/token', ccRateLimiter, express.urlencoded({ extended: false }), async (req, res, next) => {
|
||||
if (req.body?.grant_type !== 'client_credentials') {
|
||||
return next(); // Fall through to SDK's token handler
|
||||
return next(); // Fall through to confidential-client handler or SDK
|
||||
}
|
||||
|
||||
try {
|
||||
@@ -367,6 +367,80 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption
|
||||
}
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// v0.37.7.0 #1166: Custom authorization_code + refresh_token handler for
|
||||
// CONFIDENTIAL clients. The MCP SDK's clientAuth middleware does plaintext
|
||||
// `client.client_secret !== presented_secret` compare; we store
|
||||
// SHA-256 hashes, so the SDK's compare always fails for confidential
|
||||
// clients. This middleware verifies the secret hash ourselves before
|
||||
// calling the provider's exchange methods directly.
|
||||
//
|
||||
// Public clients (token_endpoint_auth_method='none') fall through to
|
||||
// the SDK's handler — the v0.34.1.0 PKCE path stays canonical.
|
||||
// ---------------------------------------------------------------------------
|
||||
app.post('/token', ccRateLimiter, async (req, res, next) => {
|
||||
const grantType = req.body?.grant_type;
|
||||
if (grantType !== 'authorization_code' && grantType !== 'refresh_token') {
|
||||
return next();
|
||||
}
|
||||
|
||||
// Detect confidential auth: either client_secret in body
|
||||
// (client_secret_post) OR Authorization: Basic header
|
||||
// (client_secret_basic). Public PKCE clients omit both.
|
||||
const bodySecret: string | undefined = req.body?.client_secret;
|
||||
let clientId: string | undefined = req.body?.client_id;
|
||||
let presentedSecret: string | undefined = bodySecret;
|
||||
const authHeader = (req.headers.authorization ?? '').toString();
|
||||
if (!presentedSecret && authHeader.startsWith('Basic ')) {
|
||||
try {
|
||||
const decoded = Buffer.from(authHeader.slice('Basic '.length), 'base64').toString('utf8');
|
||||
const idx = decoded.indexOf(':');
|
||||
if (idx > -1) {
|
||||
clientId ||= decodeURIComponent(decoded.slice(0, idx));
|
||||
presentedSecret = decodeURIComponent(decoded.slice(idx + 1));
|
||||
}
|
||||
} catch {
|
||||
// Malformed Basic header → falls through; SDK will reject
|
||||
}
|
||||
}
|
||||
if (!clientId || !presentedSecret) {
|
||||
return next(); // Public client path; SDK handles.
|
||||
}
|
||||
|
||||
try {
|
||||
const client = await oauthProvider.verifyConfidentialClientSecret(clientId, presentedSecret);
|
||||
let tokens;
|
||||
if (grantType === 'authorization_code') {
|
||||
const code = req.body.code;
|
||||
const redirectUri = req.body.redirect_uri;
|
||||
const codeVerifier = req.body.code_verifier;
|
||||
if (!code) {
|
||||
res.status(400).json({ error: 'invalid_request', error_description: 'code required' });
|
||||
return;
|
||||
}
|
||||
tokens = await oauthProvider.exchangeAuthorizationCode(client, code, codeVerifier, redirectUri);
|
||||
} else {
|
||||
const refreshToken = req.body.refresh_token;
|
||||
const scopeParam = typeof req.body.scope === 'string' ? req.body.scope.split(/\s+/) : undefined;
|
||||
if (!refreshToken) {
|
||||
res.status(400).json({ error: 'invalid_request', error_description: 'refresh_token required' });
|
||||
return;
|
||||
}
|
||||
tokens = await oauthProvider.exchangeRefreshToken(client, refreshToken, scopeParam);
|
||||
}
|
||||
res.json(tokens);
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : 'Unknown error';
|
||||
// RFC 6749: invalid_client for auth failures, invalid_grant for
|
||||
// code/token problems. "Invalid client" → 401; everything else 400.
|
||||
if (msg === 'Invalid client' || msg === 'Client has been revoked') {
|
||||
res.status(401).json({ error: 'invalid_client', error_description: msg });
|
||||
} else {
|
||||
res.status(400).json({ error: 'invalid_grant', error_description: msg });
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// MCP SDK Auth Router (OAuth endpoints)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
@@ -43,6 +43,10 @@ import {
|
||||
SourceOpError,
|
||||
type SourceRow as OpsSourceRow,
|
||||
} from '../core/sources-ops.ts';
|
||||
import {
|
||||
resolveSourceWithTier,
|
||||
SOURCE_TIER_NAMES,
|
||||
} from '../core/source-resolver.ts';
|
||||
|
||||
// ── Validation ──────────────────────────────────────────────
|
||||
|
||||
@@ -481,6 +485,51 @@ async function runFederate(engine: BrainEngine, args: string[], value: boolean):
|
||||
console.log(`Source "${id}" is now ${value ? 'federated (appears in cross-source default search)' : 'isolated (only searched when explicitly named)'}.`);
|
||||
}
|
||||
|
||||
// ── `sources current` (v0.37.7.0) ──────────────────────────
|
||||
//
|
||||
// Verify which source the CLI would target before running a
|
||||
// destructive op. Walks the same 6-tier chain as `resolveSourceId()`
|
||||
// and reports both the winning source id AND the tier label
|
||||
// ("flag" / "env" / "dotfile" / "local_path" / "brain_default" /
|
||||
// "seed_default"). Optional `--source <id>` shows what an explicit
|
||||
// flag WOULD resolve to without actually running anything.
|
||||
|
||||
async function runCurrent(engine: BrainEngine, args: string[]): Promise<void> {
|
||||
const json = args.includes('--json');
|
||||
let explicit: string | null = null;
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
if (args[i] === '--source' && i + 1 < args.length) {
|
||||
explicit = args[++i] || null;
|
||||
}
|
||||
}
|
||||
|
||||
let result: Awaited<ReturnType<typeof resolveSourceWithTier>>;
|
||||
try {
|
||||
result = await resolveSourceWithTier(engine, explicit, process.cwd());
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
if (json) {
|
||||
console.log(JSON.stringify({ error: msg }, null, 2));
|
||||
} else {
|
||||
console.error(`Error resolving source: ${msg}`);
|
||||
}
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
if (json) {
|
||||
console.log(JSON.stringify({
|
||||
source_id: result.source_id,
|
||||
tier: result.tier,
|
||||
detail: result.detail ?? null,
|
||||
resolver_chain: SOURCE_TIER_NAMES,
|
||||
}, null, 2));
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(`source: ${result.source_id}`);
|
||||
console.log(` tier: ${result.tier}${result.detail ? ` (${result.detail})` : ''}`);
|
||||
}
|
||||
|
||||
// ── Dispatcher ──────────────────────────────────────────────
|
||||
|
||||
export async function runSources(engine: BrainEngine, args: string[]): Promise<void> {
|
||||
@@ -501,6 +550,7 @@ export async function runSources(engine: BrainEngine, args: string[]): Promise<v
|
||||
case 'restore': return runRestore(engine, rest);
|
||||
case 'purge': return runPurge(engine, rest);
|
||||
case 'archived': return runListArchived(engine, rest);
|
||||
case 'current': return runCurrent(engine, rest);
|
||||
case undefined:
|
||||
case '--help':
|
||||
case '-h':
|
||||
@@ -535,6 +585,11 @@ Subcommands:
|
||||
default <id> Set the brain-level default source.
|
||||
attach <id> Write .gbrain-source in CWD (like kubectl context).
|
||||
detach Remove .gbrain-source from CWD.
|
||||
current [--source <id>] [--json] Echo the resolved source id + which tier
|
||||
won (flag/env/dotfile/local_path/
|
||||
brain_default/seed_default). Run this
|
||||
before destructive ops to verify you're
|
||||
targeting the brain you think you are.
|
||||
federate <id> Make source appear in cross-source default search.
|
||||
unfederate <id> Isolate source from default search.
|
||||
|
||||
|
||||
@@ -183,7 +183,9 @@ function listTextFiles(dir: string): string[] {
|
||||
try {
|
||||
const st = statSync(full);
|
||||
if (st.isDirectory()) {
|
||||
if (!pruneDir(name)) continue;
|
||||
// v0.37.7.0 #1169: pass parentDir so submodule pointers are
|
||||
// skipped at descent time.
|
||||
if (!pruneDir(name, d)) continue;
|
||||
walk(full);
|
||||
} else if (st.isFile() && (name.endsWith('.txt') || name.endsWith('.md'))) {
|
||||
out.push(full);
|
||||
|
||||
@@ -238,12 +238,33 @@ export function makeSubagentHandler(deps: SubagentDeps) {
|
||||
// AND no subsequent user message has been synthesized yet, we crashed
|
||||
// mid-tool-dispatch. Finish those tools now so the next LLM call sees
|
||||
// a consistent conversation.
|
||||
//
|
||||
// v0.37.7.0 #1151: if the last persisted message is an assistant
|
||||
// with NO tool_use blocks, the prior run already reached terminal
|
||||
// end_turn. Sonnet 4.6+ rejects assistant-prefill, so calling
|
||||
// messages.create here would dead-letter the job despite the work
|
||||
// being already committed. Return immediately with the persisted
|
||||
// text as finalText. Mirrors the live-loop terminal logic below.
|
||||
const last = priorMessages[priorMessages.length - 1];
|
||||
if (last && last.role === 'assistant') {
|
||||
const pendingToolUses = last.content_blocks.filter(
|
||||
(b): b is { type: 'tool_use'; id: string; name: string; input: unknown } & Record<string, unknown> =>
|
||||
b.type === 'tool_use',
|
||||
);
|
||||
if (pendingToolUses.length === 0) {
|
||||
const finalText = last.content_blocks
|
||||
.filter((b): b is { type: 'text'; text: string } & Record<string, unknown> =>
|
||||
b.type === 'text' && typeof (b as { text?: unknown }).text === 'string',
|
||||
)
|
||||
.map(b => b.text)
|
||||
.join('\n');
|
||||
return {
|
||||
result: finalText,
|
||||
turns_count: assistantTurns,
|
||||
stop_reason: 'end_turn',
|
||||
tokens: tokenTotals,
|
||||
};
|
||||
}
|
||||
if (pendingToolUses.length > 0) {
|
||||
const synthesizedResults: ContentBlock[] = [];
|
||||
for (const use of pendingToolUses) {
|
||||
|
||||
@@ -626,6 +626,47 @@ export class GBrainOAuthProvider implements OAuthServerProvider {
|
||||
// Client Credentials (called by custom handler, not SDK)
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* v0.37.7.0 #1166 — verify a confidential client's secret without
|
||||
* spending it. Returns the validated client info on success, throws
|
||||
* with an opaque "Invalid client" message on failure (mirrors RFC 6749
|
||||
* §5.2 invalid_client semantics). Used by the serve-http custom
|
||||
* /token handler for `authorization_code` + `refresh_token` grants on
|
||||
* confidential clients, since the SDK's plaintext compare in
|
||||
* clientAuth.js can't see our hash-only storage.
|
||||
*
|
||||
* Public clients (token_endpoint_auth_method === 'none') return
|
||||
* `client_secret_hash = NULL` from getClient; this method refuses
|
||||
* them so the SDK's PKCE path stays the canonical surface.
|
||||
*/
|
||||
async verifyConfidentialClientSecret(
|
||||
clientId: string,
|
||||
presentedSecret: string,
|
||||
): Promise<OAuthClientInformationFull> {
|
||||
const client = await this._clientsStore.getClient(clientId);
|
||||
if (!client) throw new Error('Invalid client');
|
||||
// Public client — refuse to use this hash-compare path.
|
||||
if (client.client_secret === undefined) {
|
||||
throw new Error('Invalid client');
|
||||
}
|
||||
const presentedHash = hashToken(presentedSecret);
|
||||
// client.client_secret is the stored SHA-256 hash (getClient returns
|
||||
// it as the `client_secret` field per the v0.34.1.0 normalization).
|
||||
// Compare via SHA-256-then-equals; constant-time compare a follow-up.
|
||||
if (client.client_secret !== presentedHash) {
|
||||
throw new Error('Invalid client');
|
||||
}
|
||||
// Soft-delete probe — same shape as exchangeClientCredentials.
|
||||
try {
|
||||
const [revoked] = await this.sql`SELECT deleted_at FROM oauth_clients WHERE client_id = ${clientId} AND deleted_at IS NOT NULL`;
|
||||
if (revoked) throw new Error('Client has been revoked');
|
||||
} catch (e) {
|
||||
if (e instanceof Error && e.message === 'Client has been revoked') throw e;
|
||||
if (!isUndefinedColumnError(e, 'deleted_at')) throw e;
|
||||
}
|
||||
return client;
|
||||
}
|
||||
|
||||
async exchangeClientCredentials(
|
||||
clientId: string,
|
||||
clientSecret: string,
|
||||
|
||||
@@ -168,6 +168,92 @@ export async function getDefaultSourcePath(
|
||||
return legacyPath ?? null;
|
||||
}
|
||||
|
||||
/**
|
||||
* v0.37.7.0 — tier labels for `resolveSourceWithTier()`. Exported so
|
||||
* `gbrain sources current --json` and downstream consumers share a
|
||||
* canonical vocabulary instead of redefining strings inline.
|
||||
*
|
||||
* Order matches the 1-6 priority of `resolveSourceId()`.
|
||||
*/
|
||||
export const SOURCE_TIER_NAMES = [
|
||||
'flag',
|
||||
'env',
|
||||
'dotfile',
|
||||
'local_path',
|
||||
'brain_default',
|
||||
'seed_default',
|
||||
] as const;
|
||||
export type SourceTier = typeof SOURCE_TIER_NAMES[number];
|
||||
|
||||
/**
|
||||
* Same resolution chain as `resolveSourceId()`, but also returns
|
||||
* WHICH tier won. Additive — does not duplicate the logic; runs the
|
||||
* same six steps in the same order. Used by `gbrain sources current`
|
||||
* so users can verify the resolved source AND the reason it resolved
|
||||
* before destructive ops.
|
||||
*
|
||||
* @returns `{ source_id, tier, detail? }` where `detail` is an
|
||||
* optional human-readable extra (e.g. the env-var name or
|
||||
* the matched dotfile / local_path).
|
||||
*/
|
||||
export async function resolveSourceWithTier(
|
||||
engine: BrainEngine,
|
||||
explicit: string | null | undefined,
|
||||
cwd: string = process.cwd(),
|
||||
): Promise<{ source_id: string; tier: SourceTier; detail?: string }> {
|
||||
// 1. Explicit flag wins.
|
||||
if (explicit) {
|
||||
if (!SOURCE_ID_RE.test(explicit)) {
|
||||
throw new Error(`Invalid --source value "${explicit}". Must match [a-z0-9-]{1,32}.`);
|
||||
}
|
||||
await assertSourceExists(engine, explicit);
|
||||
return { source_id: explicit, tier: 'flag', detail: `--source ${explicit}` };
|
||||
}
|
||||
|
||||
// 2. Env var.
|
||||
const env = process.env.GBRAIN_SOURCE;
|
||||
if (env && env.length > 0) {
|
||||
if (!SOURCE_ID_RE.test(env)) {
|
||||
throw new Error(`Invalid GBRAIN_SOURCE value "${env}". Must match [a-z0-9-]{1,32}.`);
|
||||
}
|
||||
await assertSourceExists(engine, env);
|
||||
return { source_id: env, tier: 'env', detail: `GBRAIN_SOURCE=${env}` };
|
||||
}
|
||||
|
||||
// 3. .gbrain-source dotfile walk-up.
|
||||
const dotfile = readDotfileWalk(cwd);
|
||||
if (dotfile) {
|
||||
await assertSourceExists(engine, dotfile);
|
||||
return { source_id: dotfile, tier: 'dotfile', detail: `.gbrain-source` };
|
||||
}
|
||||
|
||||
// 4. Registered source whose local_path contains CWD.
|
||||
const registered = await engine.executeRaw<{ id: string; local_path: string }>(
|
||||
`SELECT id, local_path FROM sources WHERE local_path IS NOT NULL`,
|
||||
);
|
||||
const cwdResolved = resolve(cwd);
|
||||
let best: { id: string; path: string; pathLen: number } | null = null;
|
||||
for (const r of registered) {
|
||||
const p = resolve(r.local_path);
|
||||
if (cwdResolved === p || cwdResolved.startsWith(p + '/')) {
|
||||
if (!best || p.length > best.pathLen) {
|
||||
best = { id: r.id, path: p, pathLen: p.length };
|
||||
}
|
||||
}
|
||||
}
|
||||
if (best) return { source_id: best.id, tier: 'local_path', detail: best.path };
|
||||
|
||||
// 5. Brain-level default.
|
||||
const globalDefault = await engine.getConfig('sources.default');
|
||||
if (globalDefault && SOURCE_ID_RE.test(globalDefault)) {
|
||||
await assertSourceExists(engine, globalDefault);
|
||||
return { source_id: globalDefault, tier: 'brain_default', detail: 'sources.default config' };
|
||||
}
|
||||
|
||||
// 6. Fallback: seeded 'default' source.
|
||||
return { source_id: 'default', tier: 'seed_default' };
|
||||
}
|
||||
|
||||
/** Exposed for tests. */
|
||||
export const __testing = {
|
||||
readDotfileWalk,
|
||||
|
||||
+26
-1
@@ -12,6 +12,11 @@
|
||||
*/
|
||||
|
||||
import { CJK_SLUG_CHARS } from './cjk.ts';
|
||||
// v0.37.7.0 #1169 submodule-detection helpers. Bottom-of-file already
|
||||
// aliases existsSync as `_existsSync` for other purposes; the top-of-file
|
||||
// import keeps the pruneDir helper's deps near its callsite.
|
||||
import { existsSync, statSync } from 'fs';
|
||||
import { join as pathJoin } from 'path';
|
||||
|
||||
export interface SyncManifest {
|
||||
added: string[];
|
||||
@@ -243,8 +248,14 @@ const PRUNE_DIR_NAMES = new Set<string>([
|
||||
*
|
||||
* `name` is a single path segment (basename of the directory entry), NOT a
|
||||
* full path. Walkers consult this on each subdirectory entry during recursion.
|
||||
*
|
||||
* v0.37.7.0 #1169: when callers pass `parentDir`, ALSO skip git submodule
|
||||
* directories (detected by the presence of `.git` as a FILE — not a
|
||||
* directory — inside the candidate dir). The `parentDir` arg is optional so
|
||||
* existing callers stay back-compat; new callers (sync walker, extract
|
||||
* walker) thread it through.
|
||||
*/
|
||||
export function pruneDir(name: string): boolean {
|
||||
export function pruneDir(name: string, parentDir?: string): boolean {
|
||||
if (!name) return true;
|
||||
if (name.startsWith('.')) return false;
|
||||
if (PRUNE_DIR_NAMES.has(name)) return false;
|
||||
@@ -252,6 +263,20 @@ export function pruneDir(name: string): boolean {
|
||||
// convention (e.g. `people/pedro.raw/` holds raw source for pedro.md).
|
||||
// Both forms should be skipped at descent time.
|
||||
if (name.endsWith('.raw')) return false;
|
||||
// Submodule detection: a git submodule directory contains `.git` as
|
||||
// a FILE (a "gitfile" pointing into the parent's .git/modules/...),
|
||||
// not a directory. Best-effort: if we can't stat (e.g. cross-platform
|
||||
// permission edge), fall through and treat as a normal dir.
|
||||
if (parentDir) {
|
||||
try {
|
||||
const gitPath = pathJoin(parentDir, name, '.git');
|
||||
if (existsSync(gitPath) && statSync(gitPath).isFile()) {
|
||||
return false;
|
||||
}
|
||||
} catch {
|
||||
// Stat failed — descend normally rather than silently exclude.
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
/**
|
||||
* v0.37.7.0 #1226 regression test.
|
||||
*
|
||||
* The autopilot lockfile was hardcoded at `~/.gbrain/autopilot.lock`
|
||||
* (via `process.env.HOME`), bypassing GBRAIN_HOME. Two brains pointed
|
||||
* at different GBRAIN_HOME directories would still write to the same
|
||||
* global lockfile; one would silently take over the other on each
|
||||
* restart.
|
||||
*
|
||||
* Fix: route through `gbrainPath('autopilot.lock')` which honors
|
||||
* GBRAIN_HOME. This file pins the contract via the canonical helper
|
||||
* directly, since the autopilot daemon's lifecycle is heavy to drive
|
||||
* in a unit test.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { withEnv } from './helpers/with-env.ts';
|
||||
import { mkdtempSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
import { gbrainPath } from '../src/core/config.ts';
|
||||
|
||||
describe('autopilot lock path scoped to GBRAIN_HOME (#1226)', () => {
|
||||
test('one GBRAIN_HOME produces one canonical lock path', async () => {
|
||||
const home = mkdtempSync(join(tmpdir(), 'gbrain-autopilot-lock-'));
|
||||
await withEnv({ GBRAIN_HOME: home }, async () => {
|
||||
const lockPath = gbrainPath('autopilot.lock');
|
||||
// Lockfile MUST live inside the per-brain GBRAIN_HOME, not under
|
||||
// process.env.HOME — that was the pre-fix bug.
|
||||
expect(lockPath.startsWith(home)).toBe(true);
|
||||
expect(lockPath.endsWith('autopilot.lock')).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
test('two GBRAIN_HOME values produce two distinct lockfiles', async () => {
|
||||
const homeA = mkdtempSync(join(tmpdir(), 'gbrain-autopilot-A-'));
|
||||
const homeB = mkdtempSync(join(tmpdir(), 'gbrain-autopilot-B-'));
|
||||
|
||||
let lockA = '';
|
||||
let lockB = '';
|
||||
await withEnv({ GBRAIN_HOME: homeA }, async () => {
|
||||
lockA = gbrainPath('autopilot.lock');
|
||||
});
|
||||
await withEnv({ GBRAIN_HOME: homeB }, async () => {
|
||||
lockB = gbrainPath('autopilot.lock');
|
||||
});
|
||||
|
||||
// The contract that prevents two brains from silently colliding:
|
||||
// distinct GBRAIN_HOME values MUST produce distinct lockfile paths.
|
||||
expect(lockA).not.toBe(lockB);
|
||||
expect(lockA.startsWith(homeA)).toBe(true);
|
||||
expect(lockB.startsWith(homeB)).toBe(true);
|
||||
});
|
||||
|
||||
test('default (no GBRAIN_HOME override) still produces a valid path', async () => {
|
||||
// When GBRAIN_HOME is unset, gbrainPath falls through to its
|
||||
// default (`~/.gbrain`). The path must still exist as a string
|
||||
// and end with the expected filename — we don't assert the exact
|
||||
// home dir since that varies by environment.
|
||||
await withEnv({ GBRAIN_HOME: undefined }, async () => {
|
||||
const lockPath = gbrainPath('autopilot.lock');
|
||||
expect(typeof lockPath).toBe('string');
|
||||
expect(lockPath.endsWith('autopilot.lock')).toBe(true);
|
||||
expect(lockPath.length).toBeGreaterThan('autopilot.lock'.length);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,101 @@
|
||||
/**
|
||||
* v0.37.7.0 #1162 — autopilot reconnect-error classifier + launchd plist
|
||||
* generator regression tests.
|
||||
*
|
||||
* Pre-fix: autopilot's DB-health-check reconnect loop caught every error
|
||||
* and looped forever. When `database_url` was unset/malformed the loop
|
||||
* spammed `config.database_url undefined` until the user killed launchd.
|
||||
*
|
||||
* Fix: classify errors into recoverable (transient, retry) vs
|
||||
* unrecoverable (config / auth — exit). Combined with launchd plist's
|
||||
* `ThrottleInterval=60`, unrecoverable exits trigger a real 60s
|
||||
* backoff instead of immediate respawn.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { classifyReconnectError, generateLaunchdPlist } from '../src/commands/autopilot.ts';
|
||||
|
||||
describe('classifyReconnectError (#1162)', () => {
|
||||
test('database_url undefined → unrecoverable (the #1162 fingerprint)', () => {
|
||||
const err = new Error('config.database_url undefined');
|
||||
expect(classifyReconnectError(err)).toBe('unrecoverable');
|
||||
});
|
||||
|
||||
test('database_url empty → unrecoverable', () => {
|
||||
expect(classifyReconnectError(new Error('database_url is empty'))).toBe('unrecoverable');
|
||||
});
|
||||
|
||||
test('database_url missing → unrecoverable', () => {
|
||||
expect(classifyReconnectError(new Error('database_url missing'))).toBe('unrecoverable');
|
||||
});
|
||||
|
||||
test('malformed URL → unrecoverable', () => {
|
||||
expect(classifyReconnectError(new Error('Invalid URL: not a postgres connection string'))).toBe('unrecoverable');
|
||||
expect(classifyReconnectError(new Error('Failed to parse URL'))).toBe('unrecoverable');
|
||||
});
|
||||
|
||||
test('auth failure → unrecoverable (creds don\'t fix themselves)', () => {
|
||||
expect(classifyReconnectError(new Error('password authentication failed for user "gbrain"'))).toBe('unrecoverable');
|
||||
expect(classifyReconnectError(new Error('role "ghost" does not exist'))).toBe('unrecoverable');
|
||||
});
|
||||
|
||||
test('no brain configured → unrecoverable', () => {
|
||||
expect(classifyReconnectError(new Error('No brain configured. Run: gbrain init'))).toBe('unrecoverable');
|
||||
});
|
||||
|
||||
test('network blip → recoverable', () => {
|
||||
expect(classifyReconnectError(new Error('ECONNREFUSED 127.0.0.1:5432'))).toBe('recoverable');
|
||||
expect(classifyReconnectError(new Error('connection terminated unexpectedly'))).toBe('recoverable');
|
||||
});
|
||||
|
||||
test('pool saturated → recoverable', () => {
|
||||
expect(classifyReconnectError(new Error('connection pool timed out'))).toBe('recoverable');
|
||||
expect(classifyReconnectError(new Error('remaining connection slots are reserved'))).toBe('recoverable');
|
||||
});
|
||||
|
||||
test('Supabase 503 → recoverable', () => {
|
||||
expect(classifyReconnectError(new Error('HTTP 503 Service Unavailable'))).toBe('recoverable');
|
||||
});
|
||||
|
||||
test('non-Error inputs degrade safely', () => {
|
||||
expect(classifyReconnectError(null)).toBe('recoverable');
|
||||
expect(classifyReconnectError(undefined)).toBe('recoverable');
|
||||
expect(classifyReconnectError('plain string error')).toBe('recoverable');
|
||||
expect(classifyReconnectError({ weird: 'object' })).toBe('recoverable');
|
||||
});
|
||||
|
||||
test('case-insensitive match', () => {
|
||||
expect(classifyReconnectError(new Error('DATABASE_URL UNDEFINED'))).toBe('unrecoverable');
|
||||
expect(classifyReconnectError(new Error('Password Authentication FAILED'))).toBe('unrecoverable');
|
||||
});
|
||||
});
|
||||
|
||||
describe('generateLaunchdPlist (#1162)', () => {
|
||||
test('plist contains ThrottleInterval=60', () => {
|
||||
const plist = generateLaunchdPlist('/Users/me/.gbrain/autopilot-run.sh', '/Users/me');
|
||||
expect(plist).toMatch(/<key>ThrottleInterval<\/key><integer>60<\/integer>/);
|
||||
});
|
||||
|
||||
test('plist contains KeepAlive (existing behavior preserved)', () => {
|
||||
const plist = generateLaunchdPlist('/Users/me/.gbrain/autopilot-run.sh', '/Users/me');
|
||||
expect(plist).toMatch(/<key>KeepAlive<\/key><true\/>/);
|
||||
});
|
||||
|
||||
test('plist references the wrapper path correctly', () => {
|
||||
const plist = generateLaunchdPlist('/path/to/wrapper.sh', '/home');
|
||||
expect(plist).toContain('/path/to/wrapper.sh');
|
||||
});
|
||||
|
||||
test('plist escapes XML special chars in paths', () => {
|
||||
const plist = generateLaunchdPlist('/path/with&/test.sh', '/home');
|
||||
// The path with `&` should be escaped to `&` (idempotent on
|
||||
// already-escaped strings is acceptable; the key contract is "no
|
||||
// raw `&` in the XML output").
|
||||
expect(plist).not.toContain('with&/test'); // raw unescaped `&` between with and `/`
|
||||
});
|
||||
|
||||
test('plist writes StandardErrorPath under the home dir (#1162 — error visibility)', () => {
|
||||
const plist = generateLaunchdPlist('/wrapper.sh', '/Users/alice');
|
||||
expect(plist).toContain('/Users/alice/.gbrain/autopilot.err');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,145 @@
|
||||
/**
|
||||
* v0.37.7.0 doctor checks — T12, T13, T14.
|
||||
*
|
||||
* - checkSourceRoutingHealth (T12 / 5K) — multi-source brains with
|
||||
* empty non-default sources surface the silent-collapse-to-default
|
||||
* fingerprint from #1167.
|
||||
* - checkOauthConfidentialHealth (T13 / 5L) — confidential clients
|
||||
* missing client_secret_hash fail loud.
|
||||
* - checkAutopilotLockScope (T14 / 5M) — stale lockfile outside
|
||||
* GBRAIN_HOME surfaces a PID-safe hint.
|
||||
*
|
||||
* Hermetic via PGLite + tmpdir overrides.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { mkdtempSync, rmSync, writeFileSync, mkdirSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import {
|
||||
checkSourceRoutingHealth,
|
||||
checkOauthConfidentialHealth,
|
||||
checkAutopilotLockScope,
|
||||
} from '../src/commands/doctor.ts';
|
||||
import { withEnv } from './helpers/with-env.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await engine.disconnect();
|
||||
});
|
||||
|
||||
async function truncate(): Promise<void> {
|
||||
for (const t of ['pages', 'oauth_tokens', 'oauth_codes', 'oauth_clients']) {
|
||||
await (engine as any).db.exec(`DELETE FROM ${t}`);
|
||||
}
|
||||
await (engine as any).db.exec(`DELETE FROM sources WHERE id <> 'default'`);
|
||||
}
|
||||
|
||||
describe('checkSourceRoutingHealth (#1167)', () => {
|
||||
beforeEach(truncate);
|
||||
|
||||
test('single-source brain (only default) → ok', async () => {
|
||||
const r = await checkSourceRoutingHealth(engine);
|
||||
expect(r.status).toBe('ok');
|
||||
expect(r.message).toMatch(/single-source/i);
|
||||
});
|
||||
|
||||
test('multi-source brain, all populated → ok', async () => {
|
||||
await engine.executeRaw(`INSERT INTO sources (id, name) VALUES ('a', 'a'), ('b', 'b')`);
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO pages (slug, source_id, type, title, compiled_truth, timeline)
|
||||
VALUES ('p1', 'a', 'note', 'p1', '', ''), ('p2', 'b', 'note', 'p2', '', '')`,
|
||||
);
|
||||
const r = await checkSourceRoutingHealth(engine);
|
||||
expect(r.status).toBe('ok');
|
||||
expect(r.message).toMatch(/all populated/i);
|
||||
});
|
||||
|
||||
test('non-default source with zero pages → warn (the #1167 fingerprint)', async () => {
|
||||
await engine.executeRaw(`INSERT INTO sources (id, name) VALUES ('lonely', 'lonely')`);
|
||||
const r = await checkSourceRoutingHealth(engine);
|
||||
expect(r.status).toBe('warn');
|
||||
expect(r.message).toMatch(/lonely/);
|
||||
expect(r.message).toMatch(/--source-id/);
|
||||
expect(r.message).toMatch(/gbrain sources current/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('checkOauthConfidentialHealth (#1166)', () => {
|
||||
beforeEach(truncate);
|
||||
|
||||
test('no OAuth clients → ok', async () => {
|
||||
const r = await checkOauthConfidentialHealth(engine);
|
||||
expect(r.status).toBe('ok');
|
||||
});
|
||||
|
||||
test('public client (auth_method=none, hash=NULL) → ok (v0.34.1.0 shape preserved)', async () => {
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, grant_types, scope, token_endpoint_auth_method)
|
||||
VALUES ('pkce-pub', NULL, 'pub', $1, $2, 'read', 'none')`,
|
||||
[['https://e.test/cb'], ['authorization_code']],
|
||||
);
|
||||
const r = await checkOauthConfidentialHealth(engine);
|
||||
expect(r.status).toBe('ok');
|
||||
});
|
||||
|
||||
test('confidential client with NULL hash → fail (the #1166 regression fingerprint)', async () => {
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, grant_types, scope, token_endpoint_auth_method)
|
||||
VALUES ('conf-broken', NULL, 'broken', $1, $2, 'read', 'client_secret_post')`,
|
||||
[['https://e.test/cb'], ['authorization_code']],
|
||||
);
|
||||
const r = await checkOauthConfidentialHealth(engine);
|
||||
expect(r.status).toBe('fail');
|
||||
expect(r.message).toMatch(/conf-broken/);
|
||||
expect(r.message).toMatch(/revoke-client/);
|
||||
});
|
||||
|
||||
test('confidential client with proper hash → ok', async () => {
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO oauth_clients (client_id, client_secret_hash, client_name, redirect_uris, grant_types, scope, token_endpoint_auth_method)
|
||||
VALUES ('healthy', 'abc123def', 'h', $1, $2, 'read', 'client_secret_post')`,
|
||||
[['https://e.test/cb'], ['authorization_code']],
|
||||
);
|
||||
const r = await checkOauthConfidentialHealth(engine);
|
||||
expect(r.status).toBe('ok');
|
||||
});
|
||||
});
|
||||
|
||||
describe('checkAutopilotLockScope (#1226)', () => {
|
||||
test('no lockfile → ok', async () => {
|
||||
const sandbox = mkdtempSync(join(tmpdir(), 'doctor-lock-scope-'));
|
||||
await withEnv({ GBRAIN_HOME: sandbox, HOME: sandbox }, async () => {
|
||||
const r = checkAutopilotLockScope();
|
||||
expect(r.status).toBe('ok');
|
||||
expect(r.message).toMatch(/Lock path:/);
|
||||
});
|
||||
rmSync(sandbox, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('stale lock outside GBRAIN_HOME → warn with PID-safe hint', async () => {
|
||||
const home = mkdtempSync(join(tmpdir(), 'doctor-lock-home-'));
|
||||
const gbrainHome = mkdtempSync(join(tmpdir(), 'doctor-lock-gbrain-'));
|
||||
mkdirSync(join(home, '.gbrain'), { recursive: true });
|
||||
writeFileSync(join(home, '.gbrain', 'autopilot.lock'), '99999');
|
||||
|
||||
await withEnv({ HOME: home, GBRAIN_HOME: gbrainHome }, async () => {
|
||||
const r = checkAutopilotLockScope();
|
||||
expect(r.status).toBe('warn');
|
||||
expect(r.message).toMatch(/Stale lockfile/);
|
||||
expect(r.message).toMatch(/99999/);
|
||||
expect(r.message).toMatch(/ps -p 99999/);
|
||||
});
|
||||
|
||||
rmSync(home, { recursive: true, force: true });
|
||||
rmSync(gbrainHome, { recursive: true, force: true });
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,125 @@
|
||||
/**
|
||||
* v0.37.7.0 #1204 — `gbrain extract --source-id <id>` scopes extraction.
|
||||
*
|
||||
* Federated brain users running `gbrain extract` need to scope by
|
||||
* source. Pre-fix, every run walked all sources together which
|
||||
* confused link resolution on cross-source duplicates. This test
|
||||
* pins the new `--source-id` flag: walk + extract only that source's
|
||||
* pages, while the resolver still sees ALL sources so qualified
|
||||
* `[[source:slug]]` wikilinks across sources can resolve.
|
||||
*
|
||||
* Hermetic via PGLite in-memory (no DATABASE_URL needed). Dedicated
|
||||
* file per D4 lock.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { runExtract } from '../src/commands/extract.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await engine.disconnect();
|
||||
});
|
||||
|
||||
async function truncateAll(): Promise<void> {
|
||||
for (const t of ['content_chunks', 'links', 'timeline_entries', 'tags', 'raw_data', 'page_versions', 'ingest_log', 'pages']) {
|
||||
await (engine as any).db.exec(`DELETE FROM ${t}`);
|
||||
}
|
||||
await (engine as any).db.exec(`DELETE FROM sources WHERE id <> 'default'`);
|
||||
}
|
||||
|
||||
describe('extract --source-id flag (#1204)', () => {
|
||||
beforeEach(async () => {
|
||||
await truncateAll();
|
||||
// Two sources, each with a page whose body contains a wikilink to
|
||||
// its sibling in the same source.
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO sources (id, name) VALUES ('alpha', 'alpha'), ('beta', 'beta')
|
||||
ON CONFLICT (id) DO NOTHING`,
|
||||
);
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO pages (slug, source_id, type, title, compiled_truth, timeline)
|
||||
VALUES
|
||||
('people/alice', 'alpha', 'person', 'Alice', 'Met [[people/bob]] today.', ''),
|
||||
('people/bob', 'alpha', 'person', 'Bob', 'Friend of [[people/alice]].', ''),
|
||||
('people/carol', 'beta', 'person', 'Carol', 'Met [[people/dave]].', ''),
|
||||
('people/dave', 'beta', 'person', 'Dave', 'Friend of [[people/carol]].', '')`,
|
||||
);
|
||||
});
|
||||
|
||||
test('without --source-id, walks all sources', async () => {
|
||||
const captured: unknown[] = [];
|
||||
const origLog = console.log;
|
||||
console.log = (m: unknown) => { captured.push(m); };
|
||||
try {
|
||||
await runExtract(engine, ['links', '--source', 'db', '--json']);
|
||||
} finally {
|
||||
console.log = origLog;
|
||||
}
|
||||
// Some non-zero number of links across both sources.
|
||||
const linkRows = await engine.executeRaw<{ n: string }>(
|
||||
`SELECT COUNT(*)::text AS n FROM links`,
|
||||
);
|
||||
expect(Number(linkRows[0]?.n ?? 0)).toBeGreaterThanOrEqual(2);
|
||||
});
|
||||
|
||||
test('--source-id alpha scopes extraction to alpha only', async () => {
|
||||
const captured: unknown[] = [];
|
||||
const origLog = console.log;
|
||||
console.log = (m: unknown) => { captured.push(m); };
|
||||
try {
|
||||
await runExtract(engine, ['links', '--source', 'db', '--source-id', 'alpha', '--json']);
|
||||
} finally {
|
||||
console.log = origLog;
|
||||
}
|
||||
// Links produced should only originate from alpha-source pages.
|
||||
const linkRows = await engine.executeRaw<{ slug: string; source_id: string }>(
|
||||
`SELECT p.slug, p.source_id FROM links l
|
||||
JOIN pages p ON l.from_page_id = p.id`,
|
||||
);
|
||||
// Every link's from-page must be in alpha.
|
||||
for (const r of linkRows) {
|
||||
expect(r.source_id).toBe('alpha');
|
||||
}
|
||||
// And there should be at least one such link.
|
||||
expect(linkRows.length).toBeGreaterThanOrEqual(1);
|
||||
});
|
||||
|
||||
test('--source-id beta scopes to beta and produces beta-origin links only', async () => {
|
||||
const origLog = console.log;
|
||||
console.log = () => {};
|
||||
try {
|
||||
await runExtract(engine, ['links', '--source', 'db', '--source-id', 'beta', '--json']);
|
||||
} finally {
|
||||
console.log = origLog;
|
||||
}
|
||||
const linkRows = await engine.executeRaw<{ source_id: string }>(
|
||||
`SELECT p.source_id FROM links l
|
||||
JOIN pages p ON l.from_page_id = p.id`,
|
||||
);
|
||||
for (const r of linkRows) {
|
||||
expect(r.source_id).toBe('beta');
|
||||
}
|
||||
});
|
||||
|
||||
test('--source-id with non-matching source produces zero links', async () => {
|
||||
const origLog = console.log;
|
||||
console.log = () => {};
|
||||
try {
|
||||
await runExtract(engine, ['links', '--source', 'db', '--source-id', 'nonexistent', '--json']);
|
||||
} finally {
|
||||
console.log = origLog;
|
||||
}
|
||||
const linkRows = await engine.executeRaw<{ n: string }>(
|
||||
`SELECT COUNT(*)::text AS n FROM links`,
|
||||
);
|
||||
expect(Number(linkRows[0]?.n ?? 0)).toBe(0);
|
||||
});
|
||||
});
|
||||
@@ -44,6 +44,24 @@ function captureStdout(fn: () => Promise<void>): Promise<string[]> {
|
||||
})();
|
||||
}
|
||||
|
||||
function captureBoth(fn: () => Promise<void>): Promise<{ out: string[]; err: string[] }> {
|
||||
return (async () => {
|
||||
const out: string[] = [];
|
||||
const err: string[] = [];
|
||||
const origLog = console.log;
|
||||
const origErr = console.error;
|
||||
console.log = (msg: unknown) => { out.push(String(msg)); };
|
||||
console.error = (msg: unknown) => { err.push(String(msg)); };
|
||||
try {
|
||||
await fn();
|
||||
} finally {
|
||||
console.log = origLog;
|
||||
console.error = origErr;
|
||||
}
|
||||
return { out, err };
|
||||
})();
|
||||
}
|
||||
|
||||
describe('graph-query command', () => {
|
||||
beforeEach(async () => {
|
||||
await truncateAll();
|
||||
@@ -112,3 +130,78 @@ describe('graph-query command', () => {
|
||||
expect(joined.toLowerCase()).toContain('no edges found');
|
||||
});
|
||||
});
|
||||
|
||||
// v0.37.7.0 #1153 — foreign-edge footer + --include-foreign flag.
|
||||
describe('graph-query foreign-edge footer (#1153)', () => {
|
||||
beforeEach(async () => {
|
||||
await truncateAll();
|
||||
// Two sources. Default source has alice + bob; second source has
|
||||
// carol. Edge from alice (default) to carol (other) is the foreign
|
||||
// edge the footer should surface.
|
||||
// sources table requires an 'id' entry per source; pglite-engine
|
||||
// initSchema seeds 'default'. Add the other one.
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO sources (id, name) VALUES ('other-src', 'other-src') ON CONFLICT DO NOTHING`,
|
||||
);
|
||||
await engine.putPage('people/alice', { type: 'person', title: 'Alice', compiled_truth: '', timeline: '' });
|
||||
await engine.putPage('people/bob', { type: 'person', title: 'Bob', compiled_truth: '', timeline: '' });
|
||||
// Carol lives in other-src. Use raw SQL because putPage doesn't
|
||||
// expose source_id directly via its options.
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO pages (slug, source_id, type, title, compiled_truth, timeline)
|
||||
VALUES ('people/carol', 'other-src', 'person', 'Carol', '', '')`,
|
||||
);
|
||||
// Edge: alice (default) → carol (other-src) = foreign edge
|
||||
// Edge: alice (default) → bob (default) = same-source edge
|
||||
await engine.addLink('people/alice', 'people/carol', '', 'mentions', undefined, undefined, undefined, {
|
||||
fromSourceId: 'default', toSourceId: 'other-src',
|
||||
});
|
||||
await engine.addLink('people/alice', 'people/bob', '', 'mentions');
|
||||
});
|
||||
|
||||
test('default scoped traversal emits footer with foreign-edge count', async () => {
|
||||
const { err } = await captureBoth(async () => {
|
||||
await runGraphQuery(engine, ['people/alice', '--depth', '1']);
|
||||
});
|
||||
const joined = err.join('\n');
|
||||
// Footer text contract: counts the foreign edge (alice → carol)
|
||||
// and tells the user how to include them.
|
||||
expect(joined).toMatch(/1 edge to foreign-source pages hidden/);
|
||||
expect(joined).toMatch(/--include-foreign/);
|
||||
});
|
||||
|
||||
test('--include-foreign suppresses the footer', async () => {
|
||||
const { err } = await captureBoth(async () => {
|
||||
await runGraphQuery(engine, ['people/alice', '--depth', '1', '--include-foreign']);
|
||||
});
|
||||
const joined = err.join('\n');
|
||||
// No footer when the flag is set.
|
||||
expect(joined).not.toMatch(/foreign-source pages hidden/);
|
||||
});
|
||||
|
||||
test('no footer when there are zero foreign edges', async () => {
|
||||
// Single-source brain — carol is removed; only same-source edge remains.
|
||||
await engine.executeRaw(`DELETE FROM pages WHERE slug = 'people/carol'`);
|
||||
const { err } = await captureBoth(async () => {
|
||||
await runGraphQuery(engine, ['people/alice', '--depth', '1']);
|
||||
});
|
||||
const joined = err.join('\n');
|
||||
expect(joined).not.toMatch(/foreign-source pages hidden/);
|
||||
});
|
||||
|
||||
test('footer pluralizes correctly for 2+ foreign edges', async () => {
|
||||
// Add a second foreign target in other-src so the count is plural.
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO pages (slug, source_id, type, title, compiled_truth, timeline)
|
||||
VALUES ('people/dave', 'other-src', 'person', 'Dave', '', '')`,
|
||||
);
|
||||
await engine.addLink('people/alice', 'people/dave', '', 'mentions', undefined, undefined, undefined, {
|
||||
fromSourceId: 'default', toSourceId: 'other-src',
|
||||
});
|
||||
const { err } = await captureBoth(async () => {
|
||||
await runGraphQuery(engine, ['people/alice', '--depth', '1']);
|
||||
});
|
||||
const joined = err.join('\n');
|
||||
expect(joined).toMatch(/2 edges to foreign-source pages hidden/);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,102 @@
|
||||
/**
|
||||
* v0.37.7.0 #1167 — `gbrain import --source-id <id>` routes to a brain source.
|
||||
*
|
||||
* Pre-fix, `gbrain import --source dept-x ./pages` silently fell back to
|
||||
* `default` because the CLI parser didn't consume `--source` at all
|
||||
* (PR #707's design intent explicitly excluded it). Users had no signal
|
||||
* their pages were being written to the wrong place.
|
||||
*
|
||||
* Fix: add `--source-id <id>` parsing. The flag is named --source-id
|
||||
* (not --source) to avoid colliding with future axes; matches the
|
||||
* v0.37.7.0 extract.ts convention from T2.
|
||||
*
|
||||
* Hermetic PGLite in-memory.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { runImport } from '../src/commands/import.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await engine.disconnect();
|
||||
});
|
||||
|
||||
async function truncatePages(): Promise<void> {
|
||||
for (const t of ['content_chunks', 'links', 'tags', 'raw_data', 'page_versions', 'ingest_log', 'pages']) {
|
||||
await (engine as any).db.exec(`DELETE FROM ${t}`);
|
||||
}
|
||||
await (engine as any).db.exec(`DELETE FROM sources WHERE id <> 'default'`);
|
||||
}
|
||||
|
||||
describe('import --source-id (#1167)', () => {
|
||||
let scratchDir: string;
|
||||
beforeEach(async () => {
|
||||
await truncatePages();
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO sources (id, name) VALUES ('dept-x', 'dept-x') ON CONFLICT DO NOTHING`,
|
||||
);
|
||||
scratchDir = mkdtempSync(join(tmpdir(), 'gbrain-import-src-'));
|
||||
mkdirSync(join(scratchDir, 'wiki'), { recursive: true });
|
||||
writeFileSync(
|
||||
join(scratchDir, 'wiki', 'alpha.md'),
|
||||
'---\ntype: note\n---\n# Alpha\n\nContent of alpha.',
|
||||
);
|
||||
writeFileSync(
|
||||
join(scratchDir, 'wiki', 'beta.md'),
|
||||
'---\ntype: note\n---\n# Beta\n\nContent of beta.',
|
||||
);
|
||||
});
|
||||
|
||||
test('without --source-id, pages land in default source', async () => {
|
||||
await runImport(engine, [scratchDir, '--no-embed', '--json']);
|
||||
const rows = await engine.executeRaw<{ source_id: string; slug: string }>(
|
||||
`SELECT source_id, slug FROM pages ORDER BY slug`,
|
||||
);
|
||||
expect(rows.length).toBeGreaterThanOrEqual(2);
|
||||
for (const r of rows) {
|
||||
expect(r.source_id).toBe('default');
|
||||
}
|
||||
});
|
||||
|
||||
test('--source-id dept-x routes pages to dept-x source', async () => {
|
||||
await runImport(engine, [scratchDir, '--source-id', 'dept-x', '--no-embed', '--json']);
|
||||
const rows = await engine.executeRaw<{ source_id: string; slug: string }>(
|
||||
`SELECT source_id, slug FROM pages ORDER BY slug`,
|
||||
);
|
||||
expect(rows.length).toBeGreaterThanOrEqual(2);
|
||||
for (const r of rows) {
|
||||
expect(r.source_id).toBe('dept-x');
|
||||
}
|
||||
});
|
||||
|
||||
test('--source-id value is NOT treated as a positional dir arg', async () => {
|
||||
// Regression: flag-value-as-dirArg was a real bug class in early
|
||||
// CLI parsers. Pre-fix the parser at line 82-83 would have
|
||||
// matched 'dept-x' as dirArg (since dept-x doesn't start with --).
|
||||
// The flagValues set now excludes the arg at sourceIdIdx+1.
|
||||
let threw = false;
|
||||
try {
|
||||
await runImport(engine, ['--source-id', 'dept-x', scratchDir, '--no-embed', '--json']);
|
||||
} catch (e) {
|
||||
threw = true;
|
||||
}
|
||||
// Should NOT throw "Usage: gbrain import <dir>..." because scratchDir
|
||||
// is still recognized as the positional dir.
|
||||
expect(threw).toBe(false);
|
||||
const rows = await engine.executeRaw<{ source_id: string }>(
|
||||
`SELECT source_id FROM pages LIMIT 1`,
|
||||
);
|
||||
expect(rows[0]?.source_id).toBe('dept-x');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,143 @@
|
||||
/**
|
||||
* v0.37.7.0 #1166 — OAuth confidential clients regression test.
|
||||
*
|
||||
* The MCP SDK's clientAuth middleware does `client.client_secret !==
|
||||
* presented_secret` plaintext compare. gbrain stores SHA-256 hashes,
|
||||
* so the SDK's compare always failed for confidential authorization_code
|
||||
* and refresh_token grants. v0.34.1.0 fixed PUBLIC PKCE clients
|
||||
* (client_secret = undefined); confidential clients regressed.
|
||||
*
|
||||
* Fix: provider gains `verifyConfidentialClientSecret(clientId, secret)`
|
||||
* that does hash-then-compare ourselves. The serve-http /token middleware
|
||||
* uses this BEFORE delegating to exchangeAuthorizationCode /
|
||||
* exchangeRefreshToken. Public clients fall through to the SDK as today.
|
||||
*
|
||||
* Hermetic via PGLite in-memory.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { GBrainOAuthProvider } from '../src/core/oauth-provider.ts';
|
||||
import { sqlQueryForEngine } from '../src/core/sql-query.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
let provider: GBrainOAuthProvider;
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
provider = new GBrainOAuthProvider({ sql: sqlQueryForEngine(engine) });
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await engine.disconnect();
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await (engine as any).db.exec('DELETE FROM oauth_tokens');
|
||||
await (engine as any).db.exec('DELETE FROM oauth_codes');
|
||||
await (engine as any).db.exec('DELETE FROM oauth_clients');
|
||||
});
|
||||
|
||||
describe('verifyConfidentialClientSecret (#1166)', () => {
|
||||
test('confidential client_secret_post: returns client on correct secret', async () => {
|
||||
const reg = await provider.registerClientManual('test-conf', ['authorization_code'], 'read write', ['https://example.test/cb']);
|
||||
expect(reg.clientId).toBeTruthy();
|
||||
expect(reg.clientSecret).toBeTruthy();
|
||||
|
||||
const client = await provider.verifyConfidentialClientSecret(reg.clientId, reg.clientSecret);
|
||||
expect(client.client_id).toBe(reg.clientId);
|
||||
});
|
||||
|
||||
test('wrong secret → throws "Invalid client" (RFC 6749 opaque error)', async () => {
|
||||
const reg = await provider.registerClientManual('test-conf', ['authorization_code'], 'read', ['https://example.test/cb']);
|
||||
await expect(
|
||||
provider.verifyConfidentialClientSecret(reg.clientId, 'wrong-secret'),
|
||||
).rejects.toThrow(/Invalid client/);
|
||||
});
|
||||
|
||||
test('non-existent client → throws "Invalid client"', async () => {
|
||||
await expect(
|
||||
provider.verifyConfidentialClientSecret('does-not-exist', 'anything'),
|
||||
).rejects.toThrow(/Invalid client/);
|
||||
});
|
||||
|
||||
test('public client (token_endpoint_auth_method=none) refuses confidential path', async () => {
|
||||
// Public PKCE clients are registered via the SDK's DCR path with
|
||||
// `token_endpoint_auth_method: 'none'` — those store
|
||||
// client_secret_hash = NULL. registerClientManual sets a secret
|
||||
// unconditionally, so we test the rejection by directly inserting
|
||||
// a public-client row.
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO oauth_clients
|
||||
(client_id, client_secret_hash, client_name, redirect_uris, grant_types, scope, token_endpoint_auth_method)
|
||||
VALUES ('public-pkce', NULL, 'public', $1, $2, 'read', 'none')`,
|
||||
[
|
||||
['https://example.test/cb'],
|
||||
['authorization_code'],
|
||||
],
|
||||
);
|
||||
|
||||
await expect(
|
||||
provider.verifyConfidentialClientSecret('public-pkce', 'any-secret'),
|
||||
).rejects.toThrow(/Invalid client/);
|
||||
});
|
||||
|
||||
test('case-insensitive secret? NO — must be exact match', async () => {
|
||||
const reg = await provider.registerClientManual('test-case', ['authorization_code'], 'read', ['https://example.test/cb']);
|
||||
const wrongCase = reg.clientSecret.toUpperCase();
|
||||
if (wrongCase !== reg.clientSecret) {
|
||||
await expect(
|
||||
provider.verifyConfidentialClientSecret(reg.clientId, wrongCase),
|
||||
).rejects.toThrow(/Invalid client/);
|
||||
}
|
||||
});
|
||||
|
||||
test('soft-deleted client → throws "Client has been revoked"', async () => {
|
||||
const reg = await provider.registerClientManual('to-revoke', ['authorization_code'], 'read', ['https://example.test/cb']);
|
||||
await engine.executeRaw(
|
||||
`UPDATE oauth_clients SET deleted_at = NOW() WHERE client_id = $1`,
|
||||
[reg.clientId],
|
||||
);
|
||||
await expect(
|
||||
provider.verifyConfidentialClientSecret(reg.clientId, reg.clientSecret),
|
||||
).rejects.toThrow(/revoked/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('confidential-client full flow #1166', () => {
|
||||
test('verify-then-exchange refresh token end-to-end', async () => {
|
||||
const reg = await provider.registerClientManual('full-flow-rt', ['authorization_code', 'refresh_token'], 'read', ['https://example.test/cb']);
|
||||
|
||||
// Mint an initial token pair via client_credentials (simpler than
|
||||
// /authorize round-trip in a unit test).
|
||||
await engine.executeRaw(
|
||||
`UPDATE oauth_clients SET grant_types = $1 WHERE client_id = $2`,
|
||||
[['client_credentials', 'refresh_token'], reg.clientId],
|
||||
);
|
||||
const initial = await provider.exchangeClientCredentials(reg.clientId, reg.clientSecret, 'read');
|
||||
// client_credentials grants don't issue refresh tokens (RFC 6749
|
||||
// 4.4.3), so we manually insert a refresh token to test the
|
||||
// verify-then-rotate path.
|
||||
const refreshToken = 'rt_' + Buffer.from(Math.random().toString()).toString('hex');
|
||||
const { hashToken } = await import('../src/core/utils.ts');
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO oauth_tokens (token_hash, token_type, client_id, scopes, expires_at)
|
||||
VALUES ($1, 'refresh', $2, $3, $4)`,
|
||||
[hashToken(refreshToken), reg.clientId, ['read'], Math.floor(Date.now() / 1000) + 3600],
|
||||
);
|
||||
|
||||
// verify → exchange round-trip with the correct secret
|
||||
const client = await provider.verifyConfidentialClientSecret(reg.clientId, reg.clientSecret);
|
||||
const rotated = await provider.exchangeRefreshToken(client, refreshToken);
|
||||
expect(rotated.access_token).toBeTruthy();
|
||||
expect(rotated.refresh_token).toBeTruthy();
|
||||
expect(rotated.refresh_token).not.toBe(refreshToken); // rotated
|
||||
|
||||
// Original refresh token is now consumed; second use rejected.
|
||||
await expect(
|
||||
provider.exchangeRefreshToken(client, refreshToken),
|
||||
).rejects.toThrow(/not found/);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,64 @@
|
||||
/**
|
||||
* v0.37.7.0 #1225 regression test.
|
||||
*
|
||||
* `gbrain reindex-frontmatter` was instantiating the engine via
|
||||
* `createEngine()` (which only constructs) but never calling `connect()`
|
||||
* before its first `executeRaw` in `countAffected`. The dry-run path
|
||||
* crashed with "PGLite not connected. Call connect() first."
|
||||
*
|
||||
* This test pins the fix: a connected engine handles the dry-run
|
||||
* happy path without throwing.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { runReindexFrontmatter } from '../src/commands/reindex-frontmatter.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await engine.disconnect();
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await (engine as any).db.exec('DELETE FROM pages');
|
||||
});
|
||||
|
||||
describe('reindex-frontmatter connect-before-query (#1225)', () => {
|
||||
test('dry-run on an empty brain does not throw "PGLite not connected"', async () => {
|
||||
// Seed nothing — exercises the count-affected pre-flight path that
|
||||
// was the original crash site.
|
||||
const result = await runReindexFrontmatter(engine, { dryRun: true, json: true });
|
||||
expect(result.status).toBe('dry_run');
|
||||
expect(result.examined).toBe(0);
|
||||
expect(result.updated).toBe(0);
|
||||
});
|
||||
|
||||
test('dry-run with a seeded backfillable row reports examined>0 and does not crash on engine query', async () => {
|
||||
// Seed a page with frontmatter that would trigger a backfill.
|
||||
// The point of this case is NOT to assert what dry-run counts as
|
||||
// "updated" (the command reports "would update" in dry-run mode);
|
||||
// the point is to prove the engine is connected enough to scan
|
||||
// and report at all. Pre-fix this scenario crashed with "PGLite
|
||||
// not connected".
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO pages (slug, type, title, compiled_truth, page_kind, frontmatter, effective_date)
|
||||
VALUES ($1, 'note', $2, $3, 'markdown', $4::jsonb, NULL)`,
|
||||
[
|
||||
'wiki/notes/test',
|
||||
'test',
|
||||
'# test\n\nbody',
|
||||
JSON.stringify({ effective_date: '2025-01-15' }),
|
||||
],
|
||||
);
|
||||
const result = await runReindexFrontmatter(engine, { dryRun: true, json: true });
|
||||
expect(result.status).toBe('dry_run');
|
||||
expect(result.examined).toBeGreaterThanOrEqual(1);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,184 @@
|
||||
/**
|
||||
* v0.37.7.0 — resolveSourceWithTier() tier-attribution variant tests.
|
||||
*
|
||||
* Mirrors the 6-tier priority chain from source-resolver.test.ts but
|
||||
* asserts the returned `tier` label matches the winning tier.
|
||||
* Powers `gbrain sources current` so users can verify both the
|
||||
* resolved source AND the reason it resolved.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
import {
|
||||
resolveSourceWithTier,
|
||||
SOURCE_TIER_NAMES,
|
||||
type SourceTier,
|
||||
} from '../src/core/source-resolver.ts';
|
||||
import type { BrainEngine } from '../src/core/engine.ts';
|
||||
import { withEnv } from './helpers/with-env.ts';
|
||||
|
||||
// Stub engine same shape as source-resolver.test.ts
|
||||
function makeStub(
|
||||
registeredSources: string[],
|
||||
paths: Array<{ id: string; local_path: string }>,
|
||||
defaultKey: string | null,
|
||||
): BrainEngine {
|
||||
return {
|
||||
kind: 'pglite',
|
||||
executeRaw: async <T>(sql: string, params?: unknown[]): Promise<T[]> => {
|
||||
if (sql.includes('SELECT id FROM sources WHERE id = $1')) {
|
||||
const target = params?.[0];
|
||||
return (registeredSources.includes(target as string)
|
||||
? [{ id: target } as unknown as T]
|
||||
: []);
|
||||
}
|
||||
if (sql.includes('SELECT id, local_path FROM sources')) {
|
||||
return paths as unknown as T[];
|
||||
}
|
||||
return [];
|
||||
},
|
||||
getConfig: async (key: string) => (key === 'sources.default' ? defaultKey : null),
|
||||
} as unknown as BrainEngine;
|
||||
}
|
||||
|
||||
describe('SOURCE_TIER_NAMES ordering matches resolveSourceId priority', () => {
|
||||
test('canonical order is 1=flag → 6=seed_default', () => {
|
||||
expect(SOURCE_TIER_NAMES).toEqual([
|
||||
'flag',
|
||||
'env',
|
||||
'dotfile',
|
||||
'local_path',
|
||||
'brain_default',
|
||||
'seed_default',
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveSourceWithTier — tier 1 (flag)', () => {
|
||||
test('explicit flag returns tier=flag with detail naming the value', async () => {
|
||||
const engine = makeStub(['default', 'dept-x'], [], null);
|
||||
const result = await resolveSourceWithTier(engine, 'dept-x', '/tmp');
|
||||
expect(result.source_id).toBe('dept-x');
|
||||
expect(result.tier).toBe('flag');
|
||||
expect(result.detail).toContain('--source dept-x');
|
||||
});
|
||||
|
||||
test('rejects unregistered explicit source', async () => {
|
||||
const engine = makeStub(['default'], [], null);
|
||||
await expect(resolveSourceWithTier(engine, 'ghost', '/tmp')).rejects.toThrow(/not found/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveSourceWithTier — tier 2 (env)', () => {
|
||||
test('GBRAIN_SOURCE env returns tier=env when no flag', async () => {
|
||||
const engine = makeStub(['default', 'wiki'], [], null);
|
||||
await withEnv({ GBRAIN_SOURCE: 'wiki' }, async () => {
|
||||
const result = await resolveSourceWithTier(engine, null, '/tmp');
|
||||
expect(result.source_id).toBe('wiki');
|
||||
expect(result.tier).toBe('env');
|
||||
expect(result.detail).toBe('GBRAIN_SOURCE=wiki');
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveSourceWithTier — tier 3 (dotfile)', () => {
|
||||
let scratchDir: string;
|
||||
beforeEach(() => {
|
||||
scratchDir = mkdtempSync(join(tmpdir(), 'gbrain-tier-dotfile-'));
|
||||
});
|
||||
afterEach(() => {
|
||||
rmSync(scratchDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('.gbrain-source dotfile in CWD returns tier=dotfile', async () => {
|
||||
writeFileSync(join(scratchDir, '.gbrain-source'), 'team-alpha\n');
|
||||
const engine = makeStub(['default', 'team-alpha'], [], null);
|
||||
const result = await resolveSourceWithTier(engine, null, scratchDir);
|
||||
expect(result.source_id).toBe('team-alpha');
|
||||
expect(result.tier).toBe('dotfile');
|
||||
expect(result.detail).toBe('.gbrain-source');
|
||||
});
|
||||
|
||||
test('dotfile in ancestor directory walks up to find it', async () => {
|
||||
writeFileSync(join(scratchDir, '.gbrain-source'), 'team-alpha\n');
|
||||
const nested = join(scratchDir, 'a', 'b', 'c');
|
||||
mkdirSync(nested, { recursive: true });
|
||||
const engine = makeStub(['default', 'team-alpha'], [], null);
|
||||
const result = await resolveSourceWithTier(engine, null, nested);
|
||||
expect(result.tier).toBe('dotfile');
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveSourceWithTier — tier 4 (local_path)', () => {
|
||||
test('registered source whose local_path contains CWD returns tier=local_path', async () => {
|
||||
const engine = makeStub(
|
||||
['default', 'gstack'],
|
||||
[{ id: 'gstack', local_path: '/work/gstack' }],
|
||||
null,
|
||||
);
|
||||
const result = await resolveSourceWithTier(engine, null, '/work/gstack/src');
|
||||
expect(result.source_id).toBe('gstack');
|
||||
expect(result.tier).toBe('local_path');
|
||||
expect(result.detail).toContain('/work/gstack');
|
||||
});
|
||||
|
||||
test('longest-prefix wins on nested registered sources', async () => {
|
||||
const engine = makeStub(
|
||||
['default', 'parent', 'child'],
|
||||
[
|
||||
{ id: 'parent', local_path: '/work' },
|
||||
{ id: 'child', local_path: '/work/sub' },
|
||||
],
|
||||
null,
|
||||
);
|
||||
const result = await resolveSourceWithTier(engine, null, '/work/sub/file');
|
||||
expect(result.source_id).toBe('child');
|
||||
expect(result.tier).toBe('local_path');
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveSourceWithTier — tier 5 (brain_default)', () => {
|
||||
test('sources.default config returns tier=brain_default', async () => {
|
||||
const engine = makeStub(['default', 'dept-x'], [], 'dept-x');
|
||||
const result = await resolveSourceWithTier(engine, null, '/tmp/no-dotfile-here');
|
||||
expect(result.source_id).toBe('dept-x');
|
||||
expect(result.tier).toBe('brain_default');
|
||||
expect(result.detail).toContain('sources.default');
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveSourceWithTier — tier 6 (seed_default)', () => {
|
||||
test('no other signals returns tier=seed_default with no detail', async () => {
|
||||
const engine = makeStub(['default'], [], null);
|
||||
const result = await resolveSourceWithTier(engine, null, '/tmp/no-dotfile-here');
|
||||
expect(result.source_id).toBe('default');
|
||||
expect(result.tier).toBe('seed_default');
|
||||
expect(result.detail).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveSourceWithTier — priority assertion', () => {
|
||||
test('flag wins over env wins over dotfile wins over default', async () => {
|
||||
// Set up a stub where ALL tiers could resolve. Assert the
|
||||
// higher-priority one wins.
|
||||
const engine = makeStub(
|
||||
['default', 'flag-src', 'env-src', 'dot-src', 'default-src'],
|
||||
[],
|
||||
'default-src',
|
||||
);
|
||||
await withEnv({ GBRAIN_SOURCE: 'env-src' }, async () => {
|
||||
// Flag highest priority
|
||||
const r1 = await resolveSourceWithTier(engine, 'flag-src', '/tmp');
|
||||
expect(r1.tier).toBe('flag');
|
||||
// Without flag → env
|
||||
const r2 = await resolveSourceWithTier(engine, null, '/tmp');
|
||||
expect(r2.tier).toBe('env');
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// Typecheck-only assertion: SourceTier is the union of SOURCE_TIER_NAMES.
|
||||
const _exhaustiveCheck: SourceTier = 'flag';
|
||||
void _exhaustiveCheck;
|
||||
@@ -341,6 +341,88 @@ describe('subagent handler replay (crash recovery)', () => {
|
||||
expect(client.calls.length).toBe(1);
|
||||
});
|
||||
|
||||
// v0.37.7.0 #1151 regression — terminal-on-resume.
|
||||
// Pre-fix, this scenario dead-lettered the job: replay reconciler saw
|
||||
// last=assistant with zero tool_uses, did nothing, main loop called
|
||||
// messages.create against a conversation ending in assistant → Sonnet
|
||||
// 4.6+ rejects assistant-prefill with HTTP 400 → 3 retries → dead.
|
||||
// Post-fix, the reconciler short-circuits: reconstructs finalText from
|
||||
// the persisted text blocks and returns stop_reason='end_turn' without
|
||||
// any LLM call.
|
||||
test('text-only assistant tail on resume returns terminal without LLM call (#1151)', async () => {
|
||||
const ctx = await makeCtx({ prompt: 'start' });
|
||||
// Seed prior state: user prompt, then a TERMINAL assistant turn
|
||||
// (text-only, no tool_use blocks). This is the exact shape the
|
||||
// #1151 reporter found in their dead jobs (job 190's last message
|
||||
// was a synthesis summary listing 3 written slugs).
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO subagent_messages (job_id, message_idx, role, content_blocks)
|
||||
VALUES ($1, 0, 'user', $2::jsonb)`,
|
||||
[ctx.id, JSON.stringify([{ type: 'text', text: 'start' }])],
|
||||
);
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO subagent_messages (job_id, message_idx, role, content_blocks, model, tokens_in, tokens_out)
|
||||
VALUES ($1, 1, 'assistant', $2::jsonb, 'claude-sonnet-4-6', 100, 50)`,
|
||||
[
|
||||
ctx.id,
|
||||
JSON.stringify([
|
||||
{ type: 'text', text: 'wrote 3 pages: wiki/notes/a, wiki/notes/b, wiki/notes/c' },
|
||||
]),
|
||||
],
|
||||
);
|
||||
|
||||
// The FakeMessagesClient has ZERO scripted responses. If the handler
|
||||
// tries to call messages.create, it throws. The fix guarantees we
|
||||
// never reach that path.
|
||||
const client = new FakeMessagesClient([]);
|
||||
const handler = makeSubagentHandler({ engine, client, toolRegistry: [] });
|
||||
const result = await handler(ctx);
|
||||
|
||||
expect(result.stop_reason).toBe('end_turn');
|
||||
expect(result.result).toBe('wrote 3 pages: wiki/notes/a, wiki/notes/b, wiki/notes/c');
|
||||
// Crucial assertion: no messages.create call was made on resume.
|
||||
expect(client.calls.length).toBe(0);
|
||||
// Token totals from the persisted assistant message rolled up.
|
||||
expect(result.tokens.in).toBe(100);
|
||||
expect(result.tokens.out).toBe(50);
|
||||
});
|
||||
|
||||
// Companion: the existing tool-use replay path is unchanged.
|
||||
test('text-only terminal short-circuit does NOT affect tool-use replay path', async () => {
|
||||
// This is a smoke test that the new else-branch doesn't accidentally
|
||||
// swallow the pending-tool-use case. If we have a persisted assistant
|
||||
// with a tool_use block (no synthesized user turn yet), the existing
|
||||
// tool-synthesis path must still fire.
|
||||
const echoTool = makeEchoTool('echo_x');
|
||||
const ctx = await makeCtx({ prompt: 'start' });
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO subagent_messages (job_id, message_idx, role, content_blocks)
|
||||
VALUES ($1, 0, 'user', $2::jsonb)`,
|
||||
[ctx.id, JSON.stringify([{ type: 'text', text: 'start' }])],
|
||||
);
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO subagent_messages (job_id, message_idx, role, content_blocks, model)
|
||||
VALUES ($1, 1, 'assistant', $2::jsonb, 'claude-sonnet-4-6')`,
|
||||
[
|
||||
ctx.id,
|
||||
JSON.stringify([
|
||||
{ type: 'tool_use', id: 'tu_pending', name: 'echo_x', input: { v: 'r' } },
|
||||
]),
|
||||
],
|
||||
);
|
||||
// No prior tool_exec row — replay reconciler will dispatch.
|
||||
const client = new FakeMessagesClient([
|
||||
{ content: [{ type: 'text', text: 'done after tool' }] as any, stop_reason: 'end_turn' },
|
||||
]);
|
||||
const handler = makeSubagentHandler({ engine, client, toolRegistry: [echoTool] });
|
||||
const result = await handler(ctx);
|
||||
expect(result.stop_reason).toBe('end_turn');
|
||||
expect(result.result).toBe('done after tool');
|
||||
// The handler DID call messages.create (one call) after synthesizing
|
||||
// the tool_result wrapper.
|
||||
expect(client.calls.length).toBe(1);
|
||||
});
|
||||
|
||||
test('pending non-idempotent tool exec rejects on resume', async () => {
|
||||
const nonIdempotent = { ...makeEchoTool('do_once'), idempotent: false };
|
||||
const ctx = await makeCtx({ prompt: 'start' });
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
/**
|
||||
* v0.37.7.0 #1169 — sync walker skips git submodule directories.
|
||||
*
|
||||
* A submodule directory contains `.git` as a FILE (a gitfile pointer
|
||||
* into the parent's `.git/modules/...`), not a directory. Pre-fix, the
|
||||
* walker descended into submodules and indexed their markdown content
|
||||
* as if it belonged to the parent brain.
|
||||
*
|
||||
* Fix: pruneDir now accepts an optional parentDir; when set, it stats
|
||||
* `<parentDir>/<name>/.git` and skips when that's a file.
|
||||
*
|
||||
* NOTE: The companion `.gitignore`-respect feature from PR #1159 is
|
||||
* NOT in this wave (would require adding the `ignore` npm package as a
|
||||
* dep; per the plan's "no new deps" gate, deferred to a follow-up
|
||||
* wave). This file only pins submodule-skip.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
import { pruneDir } from '../src/core/sync.ts';
|
||||
import { walkMarkdownFiles } from '../src/commands/extract.ts';
|
||||
|
||||
describe('pruneDir submodule detection (#1169)', () => {
|
||||
let scratch: string;
|
||||
beforeAll(() => {
|
||||
scratch = mkdtempSync(join(tmpdir(), 'gbrain-submodule-'));
|
||||
// Create a submodule-like directory: `.git` is a FILE inside it.
|
||||
const subDir = join(scratch, 'vendor-submodule');
|
||||
mkdirSync(subDir, { recursive: true });
|
||||
writeFileSync(join(subDir, '.git'), 'gitdir: ../../.git/modules/vendor-submodule\n');
|
||||
writeFileSync(join(subDir, 'README.md'), '# Vendor README\nshould not be indexed');
|
||||
writeFileSync(join(subDir, 'doc.md'), '# Doc\nalso should not be indexed');
|
||||
|
||||
// Create a normal directory: contains real markdown content.
|
||||
const normalDir = join(scratch, 'wiki');
|
||||
mkdirSync(normalDir, { recursive: true });
|
||||
writeFileSync(join(normalDir, 'page.md'), '# Page\nlegitimate content');
|
||||
|
||||
// Create a normal dir whose .git is a DIRECTORY (a real nested
|
||||
// repo, not a submodule pointer). pruneDir should NOT skip this
|
||||
// unless one of the OTHER rules fires (`.git` itself is dot-prefix
|
||||
// and would be excluded if walked into directly).
|
||||
const nestedRepo = join(scratch, 'nested-repo');
|
||||
mkdirSync(join(nestedRepo, '.git'), { recursive: true });
|
||||
writeFileSync(join(nestedRepo, 'README.md'), '# nested repo');
|
||||
});
|
||||
afterAll(() => {
|
||||
rmSync(scratch, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('skips a submodule directory (.git as file)', () => {
|
||||
expect(pruneDir('vendor-submodule', scratch)).toBe(false);
|
||||
});
|
||||
|
||||
test('descends into a regular markdown directory', () => {
|
||||
expect(pruneDir('wiki', scratch)).toBe(true);
|
||||
});
|
||||
|
||||
test('back-compat: parentDir undefined keeps the pre-v0.37.7 behavior', () => {
|
||||
// Without parentDir, the submodule check can't fire — only the
|
||||
// dot-prefix / PRUNE_DIR_NAMES / .raw / node_modules rules apply.
|
||||
expect(pruneDir('vendor-submodule')).toBe(true); // not skipped sans context
|
||||
expect(pruneDir('.git')).toBe(false); // dot-prefix still excluded
|
||||
expect(pruneDir('node_modules')).toBe(false); // explicit list
|
||||
});
|
||||
|
||||
test('descends into a directory containing .git as a DIRECTORY (nested git repo, not submodule)', () => {
|
||||
// pruneDir returns true (we descend); the walker then encounters
|
||||
// the inner `.git` DIRECTORY which is itself dot-prefix → excluded.
|
||||
expect(pruneDir('nested-repo', scratch)).toBe(true);
|
||||
});
|
||||
|
||||
test('walkMarkdownFiles does not return files from a submodule directory', () => {
|
||||
const files = walkMarkdownFiles(scratch);
|
||||
const paths = files.map(f => f.relPath);
|
||||
// Should include the normal page.
|
||||
expect(paths.some(p => p.endsWith('page.md'))).toBe(true);
|
||||
// Should NOT include anything from the submodule.
|
||||
expect(paths.some(p => p.includes('vendor-submodule'))).toBe(false);
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user