mirror of
https://github.com/garrytan/gbrain.git
synced 2026-08-14 00:48:18 +00:00
v0.42.76.0 fix(upgrade,security,cli): strict flag validation, bootstrap-wedge class kill, federated chunk scope (#3902)
* fix(schema): bootstrap timeline_entries.event_page_id forward reference — un-wedge pre-v121 upgrades (#2626 #2594 #2579 #2537 #2536) v0.42.56.0 (Chronicle, migration v121) added timeline_entries.event_page_id and two partial indexes in the embedded schema blobs without extending applyForwardReferenceBootstrap — any brain whose timeline_entries predates v121 wedged initSchema at blob replay ("column event_page_id does not exist") before runMigrations could apply v121, with no in-band recovery. - Add the timeline_entries.event_page_id probe + column-only ALTER to applyForwardReferenceBootstrap in BOTH engines; FK + partial indexes land via the idempotent v121 / blob replay afterwards. Stays in the always-run bootstrap (never a migration hook — those skip oddly-stamped brains). - REQUIRED_BOOTSTRAP_COVERAGE entry + strip blocks in both runtime tests. - e2e: pre-v121 rewind → full initSchema converges (indexes re-created); wedged-brain recovery — a brain that already FAILED the upgrade attempt converges on retry with full final shape (column + FK + both partial indexes) and no ledger residue. Absorbs PR #2548 (@chetan-guevara) and the e2e test from PR #2623 (@colinagent) — thank you both. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(schema): coverage guard cross-references migration-added columns — close the scan hole that shipped the v121 wedge The A2 static check treated a column as covered when the current CREATE TABLE body declared it. But a column that is BOTH in the blob's CREATE TABLE AND added by a migration is a forward reference by definition — on pre-existing tables CREATE TABLE IF NOT EXISTS no-ops and the blob's CREATE INDEX crashes initSchema before runMigrations can help. That mask is exactly how timeline_entries.event_page_id passed the guard while wedging every pre-v121 brain. - buildIndexRefCoveragePredicate: migration-added columns (from extractAddedColumnsFromMigrations over the MIGRATIONS array) require a bootstrap ALTER; CREATE TABLE presence no longer counts for them. - Unit test pins the v121 regression shape red/green with synthetic inputs; the A2 test pins the incident triple directly (migration-added + blob-indexed + bootstrap-covered). - The strengthened predicate immediately surfaced two more latent wedges of the same class: minion_jobs.timeout_at + minion_jobs.idempotency_key (migration v7, blob-indexed, unprobed). Added probes in both engines + coverage entries + runtime strip blocks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(security): serve --http hides the generated admin token from non-TTY output by default (#2624) Generated admin bootstrap tokens printed into container/log-aggregator stdout on every headless start. shouldSuppressBootstrapPrint now defaults to hidden unless stderr is an interactive TTY; env-sourced tokens are never printed; --print-admin-token is the explicit escape hatch for capturing the value on a trusted non-TTY start; --suppress-bootstrap-token still overrides everything. Unit-tested across all five postures. Absorbs PR #2625 (@irresi) — thank you. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(security): legacy bearer tokens honor permissions.takes_holders over serve --http (#2529) GBrainOAuthProvider.verifyAccessToken never returned takesHoldersAllowList, so the serve --http dispatch site always fell back to ['world'] — remote MCP callers with an operator-configured takes_holders grant saw only public takes. The legacy branch now extracts permissions.takes_holders exactly like src/mcp/http-transport.ts (fail-safe ['world'] default, non-string entries dropped, malformed permissions JSON fails closed without throwing), and AuthInfo carries the field as a typed contract. OAuth-registered clients have no takes_holders storage on oauth_clients; that lane is design work tracked in TODOS (column migration + DCR/ registration surface), not part of this hotfix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(security): get_chunks honors the federated source grant + stops shipping embedding vectors (#2555, getChunks half of #2544) The get_chunks op still used the pre-#2200 scalar pattern (ctx.sourceId ? {sourceId} : {}) and engine.getChunks had no sourceIds[] support — a federated client that could read a page via get_page got [] from get_chunks. The op now routes through sourceScopeOpts (canonical ladder: federated array > scalar floor > nothing) and both engines gain the getPage-style sourceIds[] precedence branch; the unset-opts 'default' floor is preserved for local callers (importCodeFile contract). While in the function: SELECT cc.* pulled every embedding vector over the wire per chunk only for rowToChunk to discard them — replaced with the explicit non-vector column list in both engines (the getChunks half of #2544; the per-put_page getAllSlugs half is tracked separately). getChunksWithEmbeddings stays scalar-only by design (engine-internal, zero remote-reachable callers — documented at the interface). Tests: op-level federated repro + isolation + default-floor bleed guard (PGLite), engine precedence + Chunk-shape pin, and a DATABASE_URL-gated engine-parity test covering all three scope shapes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(auth): /admin/api/register-client accepts source + federatedRead bindings (#2143 enabler) The HTTP register endpoint hardcoded source_id='default' and federated_read=undefined — only the CLI could mint a client bound to a non-default source, so HTTP-registered MCP clients wrote into 'default' regardless of intent. The endpoint now accepts optional source / federatedRead body fields, validated via assertValidSourceId with a structured 400 on bad input; omitting both preserves the historical default. The admin-UI form layer is a tracked follow-up. Absorbs PR #2016 — thank you. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(scope,calibration): think reads over MCP; source-scoped takes reads; calibration CLI reachability + model resolution (#2078-class #2451) - think op → scope:'read' for OAuth/MCP clients: the handler already forces save/take off for remote callers before persistence, so a read-scoped token can think without a write grant; local CLI persistence unchanged. Scope-annotation test carries an explicit remote-gated allowlist. - takes_list / takes_search / takes_scorecard / takes_calibration route through sourceScopeOpts (the #2200 class on the takes read lane) with engine support in BOTH engines + tests. - 'calibration' added to CLI_ONLY (the command was registered but unreachable — dispatch-gap class) and calibration_profile/voice-gate resolve models through the canonical gateway tier resolver instead of bare ids that parseModelId rejects. - BigInt-safe local-op output normalization (bigintToStringReplacer, postgres.js wire parity) — first half of the #2450 fix; the formatResult default case lands with the cli-output commit. Absorbs PR #2598 (@colinagent) and PR #2452 (@spinsirr) — thank you both. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cli): output correctness — BigInt-safe rendering, search --json, files bigint, and 5 unreachable commands (#2450 #2527 #2042 #2035-class) - normalizeLocalResult wraps the local-op output round-trip with the bigint→string replacer (postgres.js wire parity; a bare stringify THROWS on BIGSERIAL keys); formatResult's default renderer gets the same replacer so nothing upstream can crash it. - search/query --json: CLI-local formatter flag threaded through the shared formatter — stdout is a parseable result array, never human text on the --json path (the #2042 residual). - file_list normalizes size_bytes (Postgres BIGINT → Number) so MCP serialization and the CLI KB math survive; null preserved. - NEW dispatch-gap guard: every handleCliOnly top-level case label must be reachable via CLI_ONLY. It immediately caught FIVE live unreachable commands: pages, backfill, reconcile-links, notability-eval (added to CLI_ONLY), and the documented 'gbrain search modes|stats|tune' dashboards (pre-fix, 'search modes' silently keyword-searched the word "modes") — now routed via a pre-dispatch subcommand gate. 'whoknows' stays on its op-alias route (collision guard); tracked with PR #2509. Absorbs PR #2494 and PR #2531 (@javieraldape) and adapts PR #472 (@vinsew) — thank you. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(db): updateSourceConfig survives mixed-array config rows + repair/doctor cover the subagent jsonb columns (#2251) The array-coercion branch called jsonb_each(elem) bare — a mixed array (e.g. '["x", {"last_full_cycle_at": ...}]') threw 'cannot call jsonb_each on a non-object' DURING row production, permanently failing every subsequent updateSourceConfig (last_full_cycle_at could never be written again). Non-object elements are now neutralized inline via a CASE-guarded jsonb_each; the row self-heals to a flat object on the next write, object elements' keys recovered. Pinned ungated on PGLite (real Postgres semantics) and via a DATABASE_URL-gated e2e on the real engine. repair-jsonb + doctor's jsonb_integrity check extend from 5 to 8 columns (subagent_messages.content_blocks, subagent_tool_executions.input/output — historical damage rows from the pre-v0.42.53.0 positional double-encode; the write paths themselves were fixed in #2375) with a to_regclass skip for brains predating those tables. Adapts the repair/doctor extension from PR #597 (@vinsew) — thank you. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cli): dry-run honesty — strict unknown-flag rejection CLI-wide + unify-types worker defaults to dry-run (#2185 #1575) #2185: 'gbrain init --migrate-only --dry-run' applied REAL migrations — flags are read ad hoc (args.includes) so anything a handler doesn't look for was silently ignored, including intent-bearing safety flags. The CLI now validates every flag pre-dispatch and pre-engine: - Op commands validate against the operation contract (op.params + CLI-local --json/--explain), mirroring parseOpArgs so flag values that begin with '--' are never misread. parseOpArgs also gains the --key=value inline form (previously parsed as a junk key that consumed the NEXT token). - CLI_ONLY commands validate against a GENERATED per-command registry (scripts/generate-flag-registry.ts scans each command's case block + imported modules + one level of relative imports; deliberately over-inclusive so a missed flag can't break a working invocation). Committed as src/core/cli-flag-registry.generated.ts; 'bun run build:flag-registry' regenerates. - Passthrough by construction: everything after '--', plus call / config / 'jobs submit' payloads (handler-defined params are their contract). - Guards: sweep test (every command × nonsense flag → error), acceptance tests (real flags, --no- negation, = form, -- passthrough), drift guard (every CLI_ONLY member has an entry), freshness guard (committed registry == fresh generator run), subprocess smokes incl. the literal #2185 repro failing loud with zero engine work. BREAKING: scripts passing stray flags now fail loud with "Unknown flag --x for 'gbrain <cmd>'" — that is the point. #1575: the unify-types worker registration passed apply ?? true while the handler documents 'Default false (dry-run)' — the canonical operator invocation destructively retyped 25K+ pages by default. Now ?? false with a structural test; explicit --params '{"apply":true}' is the only way to mutate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(todos): file fix-wave 1 follow-ups (OAuth takes_holders design, parseFlags end-state, whoknows routing, #2544 half, #1558 UI, #2536 diagnostics) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): memory-safe unit runner — adaptive concurrency + serial OOM rescue pass A default run (4 shards × 4 intra-shard files) holds up to 16 concurrent PGLite WASM instances (~1.5GB each). With sibling Conductor workspaces running their own suites, PGLite connect failed with 'Out of memory' across every shard at once — 369 phantom test failures on a healthy branch, indistinguishable from real breakage at a glance. Two default-on layers in scripts/run-unit-parallel.sh: 1. Memory-aware sizing: total concurrency is capped to available memory (vm_stat on macOS, MemAvailable on Linux) at GBRAIN_TEST_MEM_PER_FILE_MB (default 1536) per concurrent file, shedding shards before intra-shard width. Quiet machines are unaffected (banner: mem-ok); pressured ones degrade instead of OOMing (banner: mem-adapted AxB→CxD). 2. Serial OOM rescue: failures whose shard log carries the WASM out-of-memory signature are re-run at --max-concurrency 1 after the fan-out drains. Phantoms pass serially → run goes green with an oom_rescued note and the failure blocks marked superseded; real failures fail again and stay red. Plain assertion failures never match the signature and never enter the rescue lane (existing exit-code and failure-log contract tests unchanged). Escape hatches: GBRAIN_TEST_NO_MEM_ADAPT=1, GBRAIN_TEST_NO_OOM_FALLBACK=1. Tests: OOM-once fixture rescued to exit 0; kill-switch stays red; banner advertises the sizing verdict. Documented in docs/TESTING.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test,cli): merge-seam repairs — shard timeout for the tripled suite, init flag-error contract, think scope exemption Three post-merge repairs surfaced by the first full-suite run: - Shard timeout 1500s -> 3000s: the suite roughly tripled since the cap was sized (~3900 -> 11k+ tests; PGLite inits replay 120 migrations, was 92). Two shards were killed mid-progress at 1500s. - The #2185 pre-dispatch validator now emits the same error contract as init.ts's in-handler check it preempts: lowercase 'unknown flag' on stderr + structured {status:'error', reason:'invalid_flag'} on stdout for --json callers (pinned by test/init-migrate-only.test.ts). - test/operations-trust-boundary.test.ts gets the same documented remote-gated allowlist for think's read scope (#2598) that test/oauth.test.ts already carries. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): rescue lane also covers externally-killed shards (sibling-workspace pkill / memory jetsam) Second phantom class observed on a multi-workspace Conductor machine: 3 shards SIGTERM'd + 1 SIGKILL'd at ~700s under a 3000s cap, all mid-progress — an external killer, not a wedge. The dead shards then poisoned the serial pass (lock/state residue → 18 more phantoms), and every one of the 18 passed standalone. The runner now stamps per-shard start/end epochs; a shard dying on 143/137 before 80% of SHARD_TIMEOUT is classified externally-killed and its file list joins the serial rescue queue (real wedges die AT the cap and stay red). Serial-pass failures that occur while any shard was externally killed are treated as suspect residue and rescued too. Structural tests pin the detector, threshold, and routing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(auth): keep tokenEndpointAuthMethod terminal in the register-client destructure (PKCE structural contract) The #2016 absorb appended source/federatedRead after tokenEndpointAuthMethod; test/fix-wave-structural.test.ts pins tokenEndpointAuthMethod as the final destructured field (v0.36.1.x #1077 PKCE regression contract). The added fields move into the regex's optional-middle slot. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): re-apply the #2598 remote-gated scope exemption to master's oauth scope-annotation test Taking master's v0.42.74.0 oauth.test.ts (its #2529 implementation) dropped the think read-scope allowlist that PR #2598 carries; re-applied to match test/operations-trust-boundary.test.ts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(review): pre-landing review batch — CLI-local flag acceptance, json= coherence, re-landed getChunks trim, payload-aware jsonb repair, rescue-lane tightening Ship Step 9 findings (checklist pass + 4 specialists), all verified before fixing: - CRITICAL: the #2185 validator rejected --source/--dry-run on op commands (makeContext CLI-locals consumed outside the op contract) — 'gbrain search "x" --source y' exited 1. Exempted with parser-mirroring value consumption + unit/subprocess tests incl. global-flag acceptance. - CRITICAL: --json=<v> diverged between validator (accepted) and parseOpArgs (junk-key path consumed the NEXT token, corrupting positionals). Parser now handles --json=true|false; =-forms of bare-only CLI-locals reject loud. parseOpArgs inline-= suite added (regression rule). - CRITICAL: the master merge silently restored SELECT cc.* in both engines' getChunks while docs claimed the #2544 trim. Re-landed the explicit non-vector column list + a source-level structural pin so a merge can't silently undo it again. - repair-jsonb/doctor: the subagent columns legitimately hold jsonb string scalars (persistToolExec binds pre-serialized strings) — unconditional unwrap would abort the repair run or corrupt legit values. jsonPayloadOnly predicate (JSON-container content only) on those 3 targets, mirrored in doctor + parameterized to_regclass + behavioral test (damage flagged, legit string ignored, absent table skipped). - runner rescue-lane tightening: serial failures rescue-eligible only with their own OOM signature or after an external shard kill (residue), never because a sibling shard OOM'd — flaky serial tests stay red. Rescue passes no longer double-count into TOTAL_PASS; shard timeout scales when mem-adaptation sheds shards; negative-path tests (mixed run stays red, deterministic OOM-signature failure stays red). - fail-closed remote spelling at 2 forward sites (ctx.remote !== false per the CLAUDE.md invariant); registry generator drops template-literal flag prefixes; real-PG e2es for the v121 + minion_jobs wedge classes; stale comments corrected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: regenerate flag registry after master merge (#3864 added extract help flags) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(review): red-team batch — dispatch-order validation, real --dry-run boolean, rescue-lane timeout/isolation/cap, parseable-payload repair gate Red-team pass (post-specialist) findings, all verified before fixing: - CRITICAL: validateCommandFlags checked the op lane before CLI_ONLY while dispatch runs CLI_ONLY first — dual-lane commands (think/salience/ anomalies) were validated against the WRONG contract, rejecting documented invocations ('salience --kind entity'). Lane order now mirrors dispatch. - CRITICAL: --dry-run was blessed as legal on op commands but parseOpArgs never SET it (trailing → nothing → ctx.dryRun false → the REAL destructive action ran; leading → consumed the next token). Now a CLI-local boolean exactly like --json, with --dry-run=false support and regression tests. - CRITICAL: the rescue lane ran bun test WITHOUT --timeout=60000 (bun default 5s) — PGLite phantoms re-failed on timeout and were mislabeled 'confirmed real'. Both rescue invocations now mirror the shard flags; serial files re-run one process per file (run-serial-tests.sh isolation contract); rescue wallclock capped at 2x the shard timeout. - CRITICAL: the jsonPayloadOnly probe matched container-LOOKING invalid JSON ('[INFO] fetch complete') whose repair cast would throw and abort the run mid-loop. Predicate now gates on pg_input_is_valid (PG16+, same floor as the existing IS JSON usage) + per-target catch records and continues; doctor mirrors; behavioral test covers the lookalike row. - Registry generator bounded at handleCliOnly's closing brace (the LAST case block absorbed ~100 junk flags from the rest of cli.ts, neutering strict validation for it); uppercase flag typos reject loudly in both lanes (handlers are lowercase-sensitive); --json=true spelling gets the structured invalid_flag envelope; get_chunks __all__ narrowing filed as a Wave 3 TODO. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(engines): getChunks trimmed SELECT must carry modality (codex P1-B) The #2544 egress trim replaced SELECT cc.* with an explicit column list but omitted cc.modality — every rowToChunk field except the vector must survive the trim, or the embed round-trip (getChunks -> upsertChunks) rewrites image chunks as text. Both engines; the structural pin now iterates the full rowToChunk field list instead of spot-checking. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cli): safety flags require consumption evidence in the flag registry (codex P1-A) upgrade.ts prints a help hint naming another command's --dry-run; that literal is depth-0 text for post-upgrade, so the generator allowlisted --dry-run there — recreating the exact #2185 repro this wave kills (post-upgrade --dry-run accepted, ignored, migrations run for real). Safety flags now need a tight-quoted standalone literal (an args read like has('--dry-run')) before the registry grants them; prose bleed embeds the flag inside a longer string and never qualifies. Regenerated registry drops --dry-run from post-upgrade, keeps genuine consumers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * v0.42.76.0 fix(upgrade,security,cli): strict flag validation, bootstrap-wedge class kill, federated chunk scope Version bump + CHANGELOG for the fix wave: CLI-wide unknown-flag rejection with a generated per-command registry (#2185, #1575 class), minion_jobs bootstrap probes + migration-aware coverage guard (v121 wedge class), get_chunks federated scope + egress trim (#2555, half of #2544), think read-scope over MCP (#2598), register-client source bindings (#2016, #2143 enabler), repair-jsonb/doctor subagent columns with a parse-validated damage predicate, memory-safe unit-test runner. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: update project documentation for v0.42.76.0 CONTRIBUTING.md: test-runner claims match the memory-safe 4-shard default (was 8-shard) and the CLI-only command recipe now includes the build:flag-registry regen step. KEY_FILES.md: repair-jsonb entry updated to the 8-column parse-gated current state; new entries for the strict flag-validation subsystem and src/core/source-id.ts; operations/engine/ serve-http entries updated for get_chunks scope ladder + trimmed SELECT and the register-client HTTP source bindings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: apply doc-review fixes for v0.42.76.0 Cross-model doc review caught stragglers: docs/TESTING.md still said 8-shard in the file taxonomy, carried a two-generations-stale shard timeout default (600s -> 3000s), and didn't name the new run-unit-parallel regression test or the remaining runner knobs; CONTRIBUTING.md's fast-loop file count predated the tripled suite (92+ -> 1000+); test-count claims unified at 3700+; the CLIENT_FENCED_WRITE_OPS comment in operations.ts still described think as scope write; KEY_FILES names the exported findUnknownFlag. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): rescue lane survives CI — strip ::group:: prefixes, satisfy the bun-test-timeout guard Two CI-only breaks from the master merge: (1) under GITHUB_ACTIONS the shard wraps file sections as ::group::path.test.ts, so the rescue pass extracted literal ::group:: non-paths that matched zero test files — failing_files_in_log now strips the prefix; (2) master's new check-bun-test-timeout guard greps for bare 'bun test' and tripped on run_rescue's comment text (the invocations themselves carry --timeout=60000) — comment reworded. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): shard-mechanics tests disable mem-adaptation — CI's 7GB runner collapsed explicit 2 shards to 1 The runner deliberately adapts even explicit --shards to available memory (GBRAIN_TEST_NO_MEM_ADAPT=1 is the escape hatch); on GitHub's ~7GB runners that collapsed the tests' 2-shard sandbox runs to 1 shard, breaking every 'shard 1/2:' expectation while passing locally. The tests pin shard MECHANICS with tiny synthetic files, so they now set the escape hatch; the one test that checks the mem banner overrides it back on. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
485c4773ca
commit
130d321d23
@@ -2,6 +2,23 @@
|
||||
|
||||
All notable changes to GBrain will be documented in this file.
|
||||
|
||||
## [0.42.76.0] - 2026-08-08
|
||||
|
||||
**Mistyped or unsupported flags now fail loudly instead of being silently ignored — including the ones that were supposed to make a command safe.**
|
||||
|
||||
**Strict flag validation, CLI-wide.** Every gbrain command now rejects a flag it does not understand, with a clear error naming the flag and the command, before any work runs. Before, commands read their flags ad hoc and ignored the rest — so `gbrain post-upgrade --dry-run` accepted the flag, ignored it, and applied migrations for real. That class is gone: the legal flags for every command are derived from each command's own source into a generated registry, checked before dispatch, and a command may only advertise a safety flag like `--dry-run` if its code actually reads it. On commands routed through the operations contract, a trailing `--dry-run` is now a real rehearsal switch rather than a no-op. `--json` invocations get the same error as a structured payload, so scripts fail cleanly too.
|
||||
|
||||
**A word of warning (intentional breaking change):** cron jobs or scripts that pass stray, misspelled, or long-removed flags have been running on luck — the flag did nothing. Those invocations now exit with an error naming the flag. That is the point: fix the invocation once and it means what it says forever. Everything after `--` is passthrough and remains untouched.
|
||||
|
||||
**Upgrades can't wedge on forward-referenced columns anymore — as a class.** The v0.42.56.0-era startup wedge (a schema blob referencing a column that pre-existing brains didn't have yet) had two more latent instances waiting in the jobs table. Both are now probed and healed at startup, and the schema coverage guard was rewritten to cross-reference every column referenced by the embedded schema against the set of columns any migration has ever added — so a new forward reference cannot ship without its startup probe. A recovery test walks the exact journey an affected brain takes: failed upgrade, retry on the fixed binary, converge with no leftover state blocking the way.
|
||||
|
||||
**Remote agents get more, within the same fences.** The `think` operation is now available to remote MCP callers as a read-only synthesis — the local CLI can still persist results, while remote callers are forced read-only. Chunk reads now resolve through the same source-scope rules as page reads, so a federated grant that can open a page can also read that page's chunks, and a caller without the grant cannot reach chunks outside its own floor. Chunk payloads also stop carrying raw embedding vectors over the wire — noticeably smaller responses with no behavior change, since no consumer ever read them. Two internal call sites that forward caller identity now treat anything ambiguous as untrusted, matching the fail-closed rule the rest of the codebase already follows.
|
||||
|
||||
**Source-bound clients can be minted over HTTP.** The `/admin/api/register-client` endpoint now accepts `source` and `federatedRead` bindings, mirroring the CLI's `--source` / `--federated-read` flags — so an admin UI or provisioning proxy can create a client confined to a specific brain source without shelling out to the CLI. Omitting both preserves the historical default, and invalid source ids get a structured 400.
|
||||
|
||||
**`gbrain doctor` and `repair-jsonb` see further and misfire less.** The double-encoded-JSON scan now covers the subagent execution columns, and the damage test requires the stored text to actually parse as JSON before flagging it — a legitimate string value that merely starts with `[` or `{` (a log line, a code snippet) is no longer misclassified, and a repair pass can no longer corrupt it. One damaged table no longer aborts the scan of the rest.
|
||||
|
||||
### To take advantage of v0.42.76.0
|
||||
## [0.42.75.0] - 2026-08-08
|
||||
|
||||
**The "PGLite crashes on macOS 26" era is over: gbrain now repairs a torn brain in place, automatically, with your data preserved.**
|
||||
@@ -31,6 +48,13 @@ Credit where due: @yang1996202-cpu (#2575), @AndreLYL (#223), and @roysaurav (#1
|
||||
gbrain upgrade
|
||||
```
|
||||
|
||||
Nothing to configure. If a cron job or script starts failing with `unknown flag`, that invocation was passing a flag that did nothing — remove or fix the flag and it will not regress silently again.
|
||||
|
||||
### For contributors
|
||||
|
||||
Community fixes absorbed with credit: @colinagent (#2598 think read-scope; the upgrade-rewind e2e pattern from #2623), @guim4dev (#2016 register-client source bindings), @vinsew (#597 repair-jsonb coverage extension), @javieraldape (#2494/#2531 output-correctness class — BigInt-safe local rendering and the search `--json` regression pin land here; parts of both PRs shipped earlier from master). Thank you — superseded PRs are being closed with notes.
|
||||
|
||||
The unit-test runner is now memory-safe on machines running multiple workspaces: shard concurrency adapts to actually-available memory, and a serial rescue lane re-runs files that died to OOM or external kills before calling them failures — a red suite now means real failures, not memory pressure. The flag registry regenerates via `bun run build:flag-registry` and is pinned by freshness, drift, and consumption-evidence guards.
|
||||
If your brain currently won't open, that's it — the next command repairs it. If you'd rather look first: `gbrain pglite-repair --dry-run`.
|
||||
|
||||
## [0.42.74.0] - 2026-08-07
|
||||
|
||||
+16
-4
@@ -81,7 +81,7 @@ docs/ Architecture docs
|
||||
|
||||
```bash
|
||||
# Inner edit loop (~85s on a Mac dev box, 3700+ unit tests)
|
||||
bun run test # parallel 8-shard fan-out + serial post-pass
|
||||
bun run test # parallel 4-shard fan-out (memory-adaptive) + serial post-pass
|
||||
bun test test/markdown.test.ts # specific unit test
|
||||
|
||||
# Pre-push gate (matches what CI runs on shard 1 + typecheck)
|
||||
@@ -118,9 +118,12 @@ trailing-newline and exports-count checks.
|
||||
|
||||
### Writing tests that survive the parallel loop
|
||||
|
||||
`bun run test` shards 92+ unit-test files across 8 worker processes. Files in the
|
||||
same shard share a process, so process-global state leaks between them. Four
|
||||
lint rules (`scripts/check-test-isolation.sh`, R1-R4) enforce isolation:
|
||||
`bun run test` shards 1000+ unit-test files across up to 4 worker processes,
|
||||
capping total concurrency (shards × intra-shard files) to available memory and
|
||||
re-running OOM-killed or externally-killed files serially before calling them
|
||||
failures (see `docs/TESTING.md` for the rescue-pass details and knobs). Files
|
||||
in the same shard share a process, so process-global state leaks between them.
|
||||
Four lint rules (`scripts/check-test-isolation.sh`, R1-R4) enforce isolation:
|
||||
|
||||
| Rule | What it bans | Fix |
|
||||
|---|---|---|
|
||||
@@ -217,6 +220,15 @@ automatically appears in the CLI, MCP server, and tools-json:
|
||||
For CLI-only commands (init, upgrade, import, export, files, embed, doctor, sync):
|
||||
1. Create `src/commands/mycommand.ts`
|
||||
2. Add the case to `src/cli.ts`
|
||||
3. Regenerate the flag registry: `bun run build:flag-registry`. The CLI rejects
|
||||
unknown flags before dispatch; each CLI-only command's legal flag set is
|
||||
derived from its source into `src/core/cli-flag-registry.generated.ts`.
|
||||
`test/cli-flag-validation.test.ts` pins registry freshness, drift, and
|
||||
consumption evidence (a safety flag like `--dry-run` may only be advertised
|
||||
if the command's code actually reads it), so a stale registry fails the
|
||||
build. At runtime a missing registry entry fails open — a forgotten regen
|
||||
never bricks a command. Rerun the regen whenever you add or remove a flag
|
||||
on an existing command, too.
|
||||
|
||||
Parity tests (`test/parity.test.ts`) verify CLI/MCP/tools-json stay in sync.
|
||||
|
||||
|
||||
@@ -1,5 +1,62 @@
|
||||
# TODOS
|
||||
|
||||
## Fix-wave 1 follow-ups (upgrade-wedge + trust-seam wave, 2026-08)
|
||||
|
||||
Deferred from the un-wedge-v121 hotfix wave (eng review + codex outside voice
|
||||
CLEARED; every item an explicit review decision). Waves 2–6 of the sequence are
|
||||
planned separately (provider-compat rescue is next; its original 2026-07-24
|
||||
DeepSeek-deprecation deadline has now PASSED — re-verify each cluster against
|
||||
master before starting, several fixes landed independently).
|
||||
|
||||
- [ ] **P2 — Shared strict `parseFlags` helper as the #2185 end-state (eng
|
||||
review 2B).** This wave ships the generated known-flags registry +
|
||||
pre-dispatch validator (parser and registry can drift only until the
|
||||
freshness guard fires). The structural end-state migrates commands onto one
|
||||
shared strict parser so parser == registry by construction; mechanical but
|
||||
touches 60+ command files — its own PR. Where: `src/commands/*.ts`,
|
||||
`src/cli.ts`, `scripts/generate-flag-registry.ts` (retires).
|
||||
- [ ] **P2 — `whoknows` CLI routing (surfaced by the #2035-class sweep).**
|
||||
`handleCliOnly`'s `whoknows` case (the dedicated CLI renderer with
|
||||
thin-client routing) is dead code — the command resolves via the
|
||||
`find_experts` op alias, and adding it to CLI_ONLY trips the alias-collision
|
||||
guard. Decide the intended surface alongside PR #2509 (whoknows --explain
|
||||
per-result factor breakdown) and delete whichever lane loses. Where:
|
||||
`src/cli.ts`, `src/commands/whoknows.ts`, PR #2509.
|
||||
- [ ] **P3 — #2544 second half: per-put_page `getAllSlugs` full scan.** The
|
||||
getChunks egress half shipped in this wave (explicit non-vector column
|
||||
list). The remaining Postgres-egress cost is put_page's per-call
|
||||
`getAllSlugs` table scan — needs a targeted existence probe or cached slug
|
||||
set. Where: `src/core/operations.ts` put_page path, both engines.
|
||||
- [ ] **P3 — #1558 admin-UI register form.** The `/admin/api/register-client`
|
||||
API now accepts `source` + `federatedRead` (this wave, PR #2016 absorbed);
|
||||
the admin SPA form fields + `/admin/api/sources` picker are the UI layer.
|
||||
Where: `src/commands/serve-http.ts` admin SPA blob.
|
||||
- [ ] **P3 — jsonb-integrity surfaces: batch + share (ship-review follow-up).**
|
||||
doctor's jsonbIntegrityCheck runs 2 queries per target (16 round-trips) and
|
||||
duplicates the TARGETS table with repair-jsonb (already drifted once on the
|
||||
jsonPayloadOnly predicate before being mirrored by hand). Batch the counts
|
||||
into one UNION ALL query and extract a shared targets constant
|
||||
(src/core/jsonb-integrity-targets.ts) consumed by both. Where:
|
||||
`src/commands/doctor.ts` jsonbIntegrityCheck, `src/commands/repair-jsonb.ts`.
|
||||
- [ ] **P3 — register-client HTTP-level e2e (ship-review follow-up).** The
|
||||
source/federatedRead lane is covered by unit normalizers + a structural
|
||||
route pin; a DATABASE_URL-gated serve-http e2e (register with bindings →
|
||||
assert stored client via /admin/api/agents; invalid source → 400
|
||||
invalid_source) closes the wire-level gap. Where:
|
||||
`test/e2e/serve-http-oauth.test.ts`.
|
||||
- [ ] **P3 — get_chunks `__all__` sentinel narrows to 'default' (red-team,
|
||||
Wave 3 territory).** `sourceScopeOpts` returns `{}` for a trusted local
|
||||
`--source __all__` caller (documented "spans the brain"), but both engines'
|
||||
getChunks map empty scope to the 'default' floor — the one read op where
|
||||
`{}` is reinterpreted. Fold into the Wave 3 source-federation cluster's
|
||||
`__all__` work (an explicit unscoped signal in the engine signature, or
|
||||
handler-side expansion for trusted callers). Where: `src/core/operations.ts`
|
||||
get_chunks, both engines' getChunks.
|
||||
- [ ] **P3 — #2536 wedged-migration diagnostics.** The v121 wedge aborted
|
||||
initSchema BEFORE runMigrations, so the wedged-migration diagnostics row was
|
||||
never written — operators got a bare SQL error with no remediation hint.
|
||||
Write the diagnostics row (or a stderr remediation block) from the blob-replay
|
||||
catch path too. Where: `src/core/migrate.ts`, `src/commands/apply-migrations.ts`.
|
||||
## WAL-repair wave follow-ups (#223/#1670/#2575)
|
||||
|
||||
- [ ] **P2 — gate auto-repair on an unclean-shutdown marker (adversarial F7).** The classifier
|
||||
|
||||
+4
-4
@@ -11,7 +11,7 @@ Seven test command tiers, each with a clear scope:
|
||||
|
||||
| Command | What it runs | Wallclock | When to use |
|
||||
|---|---|---|---|
|
||||
| `bun run test` | Parallel unit-test fast loop. 8-shard fan-out via `scripts/run-unit-parallel.sh`, then a serial pass over `*.serial.test.ts`. Excludes `*.slow.test.ts` and `test/e2e/*`. No pre-checks, no typecheck. | ~85s on a Mac dev box (3650+ tests) | Inner edit loop. Default. |
|
||||
| `bun run test` | Parallel unit-test fast loop. Up-to-4-shard fan-out via `scripts/run-unit-parallel.sh` (min(CPUs, 4)), then a serial pass over `*.serial.test.ts`. Excludes `*.slow.test.ts` and `test/e2e/*`. No pre-checks, no typecheck. Memory-safe by default: total concurrency (shards × intra-shard files) is capped to available memory at `GBRAIN_TEST_MEM_PER_FILE_MB` (default 1536 — a PGLite WASM instance) per concurrent file, and two phantom-failure classes are automatically re-run serially (the rescue pass): failures carrying the WASM out-of-memory signature, and shards killed externally (SIGTERM/SIGKILL well before the shard timeout — sibling workspaces' process cleanup, memory jetsam). Phantoms pass serially and the run goes green with an `oom_rescued` note; real failures fail again serially and stay red. Knobs: `GBRAIN_TEST_NO_MEM_ADAPT=1`, `GBRAIN_TEST_NO_OOM_FALLBACK=1`, `GBRAIN_TEST_MAX_CONCURRENCY` (intra-shard, default 4), `GBRAIN_TEST_SHARD_TIMEOUT` / `GBRAIN_TEST_SHARD_KILL_AFTER`, plus `--shards N` / `--max-concurrency N` / `--dry-run` script args. | ~85s on a Mac dev box (3700+ tests) | Inner edit loop. Default. |
|
||||
| `bun run verify` | CI's authoritative pre-test gate set, fanned out in parallel by `scripts/run-verify-parallel.sh`: the full `check:*` battery (~30 checks — privacy, jsonb, progress, source-id, test-isolation, wasm, …) plus `bun run typecheck`. The `CHECKS` array in that script is the single source of truth — CI literally calls `bun run verify` in a dedicated job. | ~16s (parallel; typecheck dominates) | Before pushing; before `/ship`. |
|
||||
| `bun run test:full` | `verify && bun run test && bun run test:slow && [smart e2e]`. The local equivalent of "everything CI runs." Smart e2e: runs e2e only when `DATABASE_URL` is set; else loud skip notice to stderr. | ~3-5min depending on slow + e2e | Pre-merge sanity, before opening a PR. |
|
||||
| `bun run test:slow` | Just the `*.slow.test.ts` set (intentional cold-path correctness checks). | seconds-to-minutes | When touching slow-path code. |
|
||||
@@ -50,7 +50,7 @@ there even though they pass on Linux and macOS.
|
||||
- **CI matrix** (`.github/workflows/test.yml`) runs `scripts/test-shard.sh` across 10 matrix shards partitioned by weight-aware LPT bin-packing (`scripts/sharding.ts`) and INCLUDES `*.slow.test.ts` (the two outlier slow files run as dedicated jobs alongside the matrix). CI EXCLUDES `*.serial.test.ts` from the shards and runs them in a dedicated job via `bun run test:serial`, one bun process per file — keeping serial files out of the shard processes is what preserves the `mock.module` quarantine (a top-level mock in one file leaks into every other file sharing its process). `bun run verify` gets its own job too. CI is the ground truth for "did everything pass."
|
||||
- **Local fast loop** (`scripts/run-unit-shard.sh` via the parallel wrapper) uses round-robin-by-index sharding and EXCLUDES `*.slow.test.ts` AND `*.serial.test.ts`. Local trades coverage for inner-loop speed; CI catches what local skips.
|
||||
|
||||
This divergence is intentional. Don't try to make them equal — the two scripts deliberately solve different problems. The regression test at `test/scripts/run-unit-shard.test.ts` pins what the local fast loop should and shouldn't include.
|
||||
This divergence is intentional. Don't try to make them equal — the two scripts deliberately solve different problems. The regression test at `test/scripts/run-unit-shard.test.ts` pins what the local fast loop should and shouldn't include; `test/scripts/run-unit-parallel.test.ts` pins the wrapper's memory-adaptive concurrency and the OOM/external-kill serial rescue pass.
|
||||
|
||||
### Failure-first logging
|
||||
|
||||
@@ -61,11 +61,11 @@ When `bun run test` finds any failure, the wrapper:
|
||||
3. Writes a one-line-per-shard summary to `.context/test-summary.txt` (`shard N/M: pass=X fail=Y skip=Z rc=W`).
|
||||
4. Exits non-zero. Empty failure log + non-zero exit = infrastructure problem (wedged shard, killed child); the banner says so.
|
||||
|
||||
If a shard wedges (per-shard `GBRAIN_TEST_SHARD_TIMEOUT` cap, default 600s), the wrapper writes `--- shard N: WEDGED after ${SHARD_TIMEOUT}s ---` to the failure log, includes the last 50 lines of the shard log, and proceeds with other shards' results.
|
||||
If a shard wedges (per-shard `GBRAIN_TEST_SHARD_TIMEOUT` cap, default 3000s; `GBRAIN_TEST_SHARD_KILL_AFTER` grace after TERM before KILL, default 30s), the wrapper writes `--- shard N: WEDGED after ${SHARD_TIMEOUT}s ---` to the failure log, includes the last 50 lines of the shard log, and proceeds with other shards' results.
|
||||
|
||||
### File taxonomy
|
||||
|
||||
- `*.test.ts` → fast loop (parallel 8-shard fan-out).
|
||||
- `*.test.ts` → fast loop (parallel up-to-4-shard fan-out, memory-adaptive).
|
||||
- `*.slow.test.ts` → run via `bun run test:slow` only (intentional cold-path tests; would dominate the fast loop's wallclock).
|
||||
- `*.serial.test.ts` → run via `bun run test:serial` after the parallel pass completes; one bun process per file (`--max-concurrency=1` within a shared process is not enough — the module registry still leaks `mock.module`). Quarantine for tests that share file-wide state and race when run alongside other files in the same `bun test` process. Several dozen files, discovered by the `*.serial.test.ts` glob — no list to maintain. Typical residents: `mock.module(...)` users (top-level mocks leak across files in a shard process, e.g. `test/embed.serial.test.ts`), env-coupled files (e.g. `test/brain-registry.serial.test.ts`), and process-lifecycle suites that assert on `process.exitCode` (e.g. `test/pglite-engine-disconnect.serial.test.ts`). **Do not put the parallelism back on a serial file unless you've fixed the contention root cause** (it just re-introduces the flake).
|
||||
- `test/e2e/*.test.ts` → real-Postgres E2E. Skipped when `DATABASE_URL` is unset.
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -64,7 +64,7 @@ Key files:
|
||||
thin-client routing branches. These commands bypass the operation-layer
|
||||
dispatch in cli.ts (call `engine.foo()` directly), so each gets its own
|
||||
`if (isThinClient(cfg)) { callRemoteTool(...) }` branch that maps CLI flags
|
||||
to op params. `think` is a special case: the server's `think` op
|
||||
intentionally disables `--save`/`--take` for remote callers
|
||||
(operations.ts:1103-1135 trust-boundary gate); thin-client `think` warns
|
||||
loudly when those flags are set.
|
||||
to op params. `think` is a special case: the server's `think` op is
|
||||
read-scoped for OAuth/MCP and intentionally disables `--save`/`--take` for
|
||||
remote callers in its trust-boundary gate; thin-client `think` warns loudly
|
||||
when those flags are set.
|
||||
|
||||
+2
-1
@@ -37,6 +37,7 @@
|
||||
"build:admin-embedded": "bun run scripts/build-admin-embedded.ts",
|
||||
"build:schema": "bash scripts/build-schema.sh",
|
||||
"build:llms": "bun run scripts/build-llms.ts",
|
||||
"build:flag-registry": "bun run scripts/generate-flag-registry.ts",
|
||||
"build:pglite-snapshot": "bun run scripts/build-pglite-snapshot.ts",
|
||||
"test": "bash scripts/run-unit-parallel.sh",
|
||||
"eval:autocut": "bun test test/search/autocut-eval.test.ts",
|
||||
@@ -148,7 +149,7 @@
|
||||
"bun": ">=1.3.10"
|
||||
},
|
||||
"license": "MIT",
|
||||
"version": "0.42.75.0",
|
||||
"version": "0.42.76.0",
|
||||
"overrides": {
|
||||
"@hono/node-server": "^2.0.5",
|
||||
"fast-uri": "^3.1.5",
|
||||
|
||||
@@ -0,0 +1,177 @@
|
||||
/**
|
||||
* #2185 — known-flags registry generator for CLI_ONLY commands.
|
||||
*
|
||||
* gbrain's CLI_ONLY commands read flags ad hoc (`args.includes('--force')`,
|
||||
* per-command parseFlags helpers), so there is no parser to make strict. The
|
||||
* pre-dispatch validator in src/cli.ts needs to know each command's legal
|
||||
* flags; this script derives them from the source instead of a hand-typed
|
||||
* list that would rot.
|
||||
*
|
||||
* How: parse handleCliOnly's top-level `case 'X': {` blocks out of src/cli.ts,
|
||||
* collect every `import('./commands/Y.ts')` inside each block, then scan the
|
||||
* case-block text plus each imported module (plus one level of that module's
|
||||
* ./relative same-directory imports) for `--flag` string literals — including
|
||||
* help text, which deliberately over-includes: accepting a flag the handler
|
||||
* ignores is the pre-#2185 status quo for that flag, while missing a real
|
||||
* flag would break working invocations on upgrade.
|
||||
*
|
||||
* Output: src/core/cli-flag-registry.generated.ts (committed; freshness is
|
||||
* pinned by test/cli-flag-validation.test.ts the same way build:llms pins the
|
||||
* llms bundles). Regenerate: bun run build:flag-registry
|
||||
*
|
||||
* Hand-tuning lane: EXTRA_FLAGS below, for flags that live deeper than the
|
||||
* one-level scan (add with a comment naming the deep module).
|
||||
*/
|
||||
import { readFileSync, writeFileSync, existsSync } from 'fs';
|
||||
import { dirname, resolve as resolvePath, join } from 'path';
|
||||
import { fileURLToPath } from 'url';
|
||||
|
||||
const ROOT = resolvePath(dirname(fileURLToPath(import.meta.url)), '..');
|
||||
|
||||
/** Flags that live deeper than the one-level module scan. Keep commented. */
|
||||
const EXTRA_FLAGS: Record<string, string[]> = {
|
||||
// embed's pace knobs resolve inside src/core/pace-mode.ts (two levels deep).
|
||||
embed: ['--pace', '--pace-max-concurrency'],
|
||||
// sync shares the same pace surface via env/config plus CLI passthrough.
|
||||
sync: ['--pace', '--pace-max-concurrency'],
|
||||
};
|
||||
|
||||
/** Universal helper flags every command may see (parsed or short-circuited upstream). */
|
||||
const UNIVERSAL_FLAGS = ['--help', '--json', '--brain', '--source'];
|
||||
|
||||
const FLAG_RE = /--[a-z0-9][a-z0-9-]*/g;
|
||||
|
||||
function flagsInText(text: string): Set<string> {
|
||||
const out = new Set<string>();
|
||||
for (const m of text.matchAll(FLAG_RE)) {
|
||||
// Template-literal prefixes (`--bound-${key}` scans as `--bound-`) are
|
||||
// not real flags — a trailing hyphen would make the validator accept
|
||||
// every typo sharing the prefix.
|
||||
if (!m[0].endsWith('-')) out.add(m[0]);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** One level of ./relative imports (static or dynamic) from a module's source. */
|
||||
function relativeImports(src: string, fromDir: string): string[] {
|
||||
const paths = new Set<string>();
|
||||
for (const m of src.matchAll(/from\s+'(\.\.?\/[^']+\.ts)'/g)) paths.add(m[1]);
|
||||
for (const m of src.matchAll(/import\('(\.\.?\/[^']+\.ts)'\)/g)) paths.add(m[1]);
|
||||
return [...paths]
|
||||
.map(p => resolvePath(fromDir, p))
|
||||
.filter(p => existsSync(p));
|
||||
}
|
||||
|
||||
export function buildFlagRegistry(): Record<string, string[]> {
|
||||
const cliSource = readFileSync(join(ROOT, 'src/cli.ts'), 'utf-8');
|
||||
|
||||
// CLI_ONLY membership (the single source of truth in src/cli.ts). Strip
|
||||
// line comments first — the set literal carries commentary whose quoted
|
||||
// words ('Unknown command', 'pages') must not parse as members.
|
||||
const onlyMatch = cliSource.match(/const CLI_ONLY = new Set(?:<string>)?\(\[([\s\S]*?)\]\)/);
|
||||
if (!onlyMatch) throw new Error('CLI_ONLY set not found in src/cli.ts');
|
||||
const onlyBody = onlyMatch[1].replace(/\/\/[^\n]*/g, '');
|
||||
const commands = [...onlyBody.matchAll(/'([^']+)'/g)].map(m => m[1]);
|
||||
|
||||
// handleCliOnly body — bounded at the function's closing brace (column 0).
|
||||
// Unbounded, the LAST case block absorbed every --flag literal in the rest
|
||||
// of cli.ts (printHelp's full flag surface included), handing whichever
|
||||
// command sits last in the switch a ~100-flag junk allowlist that made
|
||||
// strict validation a no-op for it.
|
||||
const fnStart = cliSource.indexOf('async function handleCliOnly');
|
||||
if (fnStart < 0) throw new Error('handleCliOnly not found in src/cli.ts');
|
||||
const fnTail = cliSource.slice(fnStart);
|
||||
const fnEndRel = fnTail.search(/\n\}\n/);
|
||||
const fnSrc = fnEndRel > 0 ? fnTail.slice(0, fnEndRel) : fnTail;
|
||||
|
||||
// handleCliOnly dispatches through TWO styles: an `if (command === 'X')`
|
||||
// chain (DB-free commands like init/auth/schema) and a switch with
|
||||
// `case 'X':` labels. Segment on BOTH marker kinds; the text between a
|
||||
// marker and the next marker belongs to that label.
|
||||
const markRe = /(?:^\s*if \(command === '([a-z0-9-]+)'\)|^ case '([a-z0-9-]+)':)/gm;
|
||||
const marks: Array<{ label: string; start: number }> = [];
|
||||
let m: RegExpExecArray | null;
|
||||
while ((m = markRe.exec(fnSrc)) !== null) {
|
||||
marks.push({ label: (m[1] ?? m[2])!, start: m.index });
|
||||
}
|
||||
|
||||
const blocks = new Map<string, string>();
|
||||
for (let i = 0; i < marks.length; i++) {
|
||||
const end = i + 1 < marks.length ? marks[i + 1].start : fnSrc.length;
|
||||
const body = fnSrc.slice(marks[i].start, end);
|
||||
// Fall-through labels share the following block.
|
||||
blocks.set(marks[i].label, (blocks.get(marks[i].label) ?? '') + body);
|
||||
}
|
||||
|
||||
// Safety flags carry destructive-bypass semantics: allowlisting one that
|
||||
// the handler never reads recreates the #2185 repro (`post-upgrade
|
||||
// --dry-run` accepted, ignored, migrations run for real). Presence isn't
|
||||
// enough — upgrade.ts prints a HINT naming another command's --dry-run,
|
||||
// which is depth-0 text for post-upgrade. These flags are only legal with
|
||||
// CONSUMPTION evidence in the command's own code: the flag as a TIGHT-QUOTED
|
||||
// standalone literal (`includes('--dry-run')`, `has('--dry-run')`,
|
||||
// `=== '--dry-run'`). Prose bleed embeds the flag inside a longer string, so
|
||||
// it never has quotes on both sides of the bare flag.
|
||||
const SAFETY_FLAGS = new Set(['--dry-run']);
|
||||
const consumes = (text: string, flag: string): boolean =>
|
||||
new RegExp(`['"\`]${flag}['"\`]`).test(text);
|
||||
|
||||
const registry: Record<string, string[]> = {};
|
||||
for (const command of commands) {
|
||||
const block = blocks.get(command) ?? '';
|
||||
const flags = new Set<string>(UNIVERSAL_FLAGS);
|
||||
const depthZero = new Set<string>();
|
||||
let depthZeroText = block;
|
||||
for (const f of flagsInText(block)) { flags.add(f); depthZero.add(f); }
|
||||
|
||||
// Modules imported inside the case block, plus one level of each module's
|
||||
// own ./relative imports.
|
||||
const commandModules = [...block.matchAll(/import\('(\.\/[^']+\.ts)'\)/g)]
|
||||
.map(mm => resolvePath(join(ROOT, 'src'), mm[1]))
|
||||
.filter(p => existsSync(p));
|
||||
for (const modPath of commandModules) {
|
||||
const modSrc = readFileSync(modPath, 'utf-8');
|
||||
depthZeroText += modSrc;
|
||||
for (const f of flagsInText(modSrc)) { flags.add(f); depthZero.add(f); }
|
||||
for (const dep of relativeImports(modSrc, dirname(modPath))) {
|
||||
for (const f of flagsInText(readFileSync(dep, 'utf-8'))) flags.add(f);
|
||||
}
|
||||
}
|
||||
|
||||
for (const f of EXTRA_FLAGS[command] ?? []) { flags.add(f); depthZero.add(f); }
|
||||
for (const f of SAFETY_FLAGS) {
|
||||
if (flags.has(f) && !consumes(depthZeroText, f)) flags.delete(f);
|
||||
}
|
||||
registry[command] = [...flags].sort();
|
||||
}
|
||||
return registry;
|
||||
}
|
||||
|
||||
export function renderRegistryModule(registry: Record<string, string[]>): string {
|
||||
const entries = Object.keys(registry)
|
||||
.sort()
|
||||
.map(cmd => ` '${cmd}': [${registry[cmd].map(f => `'${f}'`).join(', ')}],`)
|
||||
.join('\n');
|
||||
return `// AUTO-GENERATED by scripts/generate-flag-registry.ts — do not edit by hand.
|
||||
// Regenerate: bun run build:flag-registry
|
||||
// Freshness + drift pinned by test/cli-flag-validation.test.ts (#2185).
|
||||
//
|
||||
// Per-command legal flags for CLI_ONLY commands, derived from each command's
|
||||
// source (case block + imported modules + one level of relative imports +
|
||||
// scripts/generate-flag-registry.ts EXTRA_FLAGS). Deliberately over-inclusive
|
||||
// (help-text mentions count): accepting an ignored flag is the pre-#2185
|
||||
// status quo; missing a real one breaks working invocations.
|
||||
export const CLI_FLAG_REGISTRY: Record<string, readonly string[]> = {
|
||||
${entries}
|
||||
};
|
||||
`;
|
||||
}
|
||||
|
||||
if (import.meta.main) {
|
||||
const registry = buildFlagRegistry();
|
||||
const outPath = join(ROOT, 'src/core/cli-flag-registry.generated.ts');
|
||||
writeFileSync(outPath, renderRegistryModule(registry));
|
||||
const n = Object.keys(registry).length;
|
||||
const total = Object.values(registry).reduce((a, v) => a + v.length, 0);
|
||||
console.log(`wrote ${outPath} (${n} commands, ${total} flag entries)`);
|
||||
}
|
||||
@@ -13,9 +13,29 @@
|
||||
#
|
||||
# Env overrides:
|
||||
# SHARDS=N same as --shards
|
||||
# GBRAIN_TEST_SHARD_TIMEOUT per-shard wallclock cap, seconds (default 1500)
|
||||
# GBRAIN_TEST_SHARD_TIMEOUT per-shard wallclock cap, seconds (default 3000)
|
||||
# GBRAIN_TEST_SHARD_KILL_AFTER grace after TERM before KILL (default 30)
|
||||
# GBRAIN_TEST_MAX_CONCURRENCY passed through to bun test (default 4)
|
||||
# GBRAIN_TEST_MEM_PER_FILE_MB memory budget per concurrent test file used by
|
||||
# the adaptive sizing below (default 1536 — a
|
||||
# PGLite WASM instance reserves ~1-1.5GB)
|
||||
# GBRAIN_TEST_NO_MEM_ADAPT=1 disable memory-aware concurrency reduction
|
||||
# GBRAIN_TEST_NO_OOM_FALLBACK=1 disable the serial OOM-rescue pass
|
||||
#
|
||||
# Memory safety (two layers; both default-on):
|
||||
# 1. ADAPTIVE SIZING — before spawning, total concurrency (shards ×
|
||||
# intra-shard --max-concurrency) is capped to what available memory can
|
||||
# hold at GBRAIN_TEST_MEM_PER_FILE_MB per concurrent file. Concurrent
|
||||
# Conductor workspaces running their own suites shrink the budget
|
||||
# automatically instead of OOMing each other.
|
||||
# 2. SERIAL PHANTOM RESCUE — two phantom classes are re-run serially
|
||||
# (--max-concurrency 1) after the parallel pass: (a) failures whose
|
||||
# shard log carries the PGLite WASM out-of-memory signature, and
|
||||
# (b) shards killed EXTERNALLY (SIGTERM/SIGKILL well before the shard
|
||||
# timeout — sibling Conductor workspaces' process cleanup, macOS memory
|
||||
# jetsam). Phantoms pass serially and the run goes green with an
|
||||
# oom_rescued note; real failures fail again and stay red. Plain
|
||||
# assertion failures never match either signature.
|
||||
#
|
||||
# Output files (workspace-local; falls back to /tmp if .context/ unwritable):
|
||||
# .context/test-failures.log failure blocks (cleared at start)
|
||||
@@ -38,6 +58,35 @@ detect_cpus() {
|
||||
echo 4
|
||||
}
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────────
|
||||
# Available-memory detection (MB). macOS: vm_stat free + inactive +
|
||||
# speculative + purgeable pages (inactive/purgeable are reclaimable on
|
||||
# pressure, which is exactly the scenario we size for). Linux: MemAvailable.
|
||||
# Unknown platform → 0, and the caller skips adaptation entirely.
|
||||
# ──────────────────────────────────────────────────────────────────────────
|
||||
detect_available_mem_mb() {
|
||||
if command -v vm_stat >/dev/null 2>&1; then
|
||||
vm_stat 2>/dev/null | awk '
|
||||
/page size of/ { psize = $8 }
|
||||
/Pages free/ { free = $NF }
|
||||
/Pages inactive/ { inactive = $NF }
|
||||
/Pages speculative/ { spec = $NF }
|
||||
/Pages purgeable/ { purge = $NF }
|
||||
END {
|
||||
gsub(/\./, "", free); gsub(/\./, "", inactive)
|
||||
gsub(/\./, "", spec); gsub(/\./, "", purge)
|
||||
if (psize == 0) psize = 16384
|
||||
printf "%d\n", (free + inactive + spec + purge) * psize / 1048576
|
||||
}'
|
||||
return
|
||||
fi
|
||||
if [ -r /proc/meminfo ]; then
|
||||
awk '/MemAvailable/ { printf "%d\n", $2 / 1024; found = 1 } END { if (!found) print 0 }' /proc/meminfo
|
||||
return
|
||||
fi
|
||||
echo 0
|
||||
}
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────────
|
||||
# Argument parsing. --shards N override wins over $SHARDS; both are clamped.
|
||||
# ──────────────────────────────────────────────────────────────────────────
|
||||
@@ -79,12 +128,60 @@ INTRA_CONC="${MAX_CONCURRENCY_OVERRIDE:-${GBRAIN_TEST_MAX_CONCURRENCY:-4}}"
|
||||
# had completed in 968s. 1500s cap gives ~55% headroom over observed
|
||||
# 4-shard wallclock; real hangs still hit it. Override via
|
||||
# GBRAIN_TEST_SHARD_TIMEOUT=N.
|
||||
SHARD_TIMEOUT="${GBRAIN_TEST_SHARD_TIMEOUT:-1500}"
|
||||
# v0.42.74 sizing: 1500 -> 3000. The suite roughly tripled since the 1500s
|
||||
# cap was set (June: ~3900 tests, 92-migration PGLite replay; now: 11k+
|
||||
# tests, 120-migration replay per PGLite init). At 4 shards, two shards were
|
||||
# killed at 1500s while making steady per-test progress. 3000s keeps the
|
||||
# same ~55%-headroom doctrine over observed wallclock; real hangs still die.
|
||||
SHARD_TIMEOUT="${GBRAIN_TEST_SHARD_TIMEOUT:-3000}"
|
||||
SHARD_KILL_AFTER="${GBRAIN_TEST_SHARD_KILL_AFTER:-30}"
|
||||
if ! printf '%s' "$SHARD_KILL_AFTER" | grep -qE '^[0-9]+$' || [ "$SHARD_KILL_AFTER" -lt 1 ]; then
|
||||
echo "ERROR: invalid shard kill-after: $SHARD_KILL_AFTER" >&2; exit 2
|
||||
fi
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────────
|
||||
# Memory-aware concurrency (layer 1). Total concurrent test files =
|
||||
# N shards × INTRA_CONC; each concurrent file can hold a PGLite WASM
|
||||
# instance (~1-1.5GB reserved). 4×4 = 16 concurrent instances OOM'd on a
|
||||
# 128GB machine when other Conductor workspaces ran their suites at the
|
||||
# same time — every PGLite connect across every shard failed at once
|
||||
# ("Out of memory" at PGlite.create). Cap total concurrency to what's
|
||||
# actually available, keeping a 4GB reserve for the OS + bun itself.
|
||||
# Applies to explicit --shards overrides too (an operator who wants an
|
||||
# over-committed run sets GBRAIN_TEST_NO_MEM_ADAPT=1).
|
||||
# ──────────────────────────────────────────────────────────────────────────
|
||||
MEM_PER_FILE_MB="${GBRAIN_TEST_MEM_PER_FILE_MB:-1536}"
|
||||
MEM_NOTE=""
|
||||
if [ "${GBRAIN_TEST_NO_MEM_ADAPT:-0}" != "1" ]; then
|
||||
AVAIL_MB=$(detect_available_mem_mb)
|
||||
if [ "${AVAIL_MB:-0}" -gt 0 ] 2>/dev/null; then
|
||||
BUDGET_MB=$((AVAIL_MB - 4096))
|
||||
[ "$BUDGET_MB" -lt "$MEM_PER_FILE_MB" ] && BUDGET_MB="$MEM_PER_FILE_MB"
|
||||
MAX_TOTAL=$((BUDGET_MB / MEM_PER_FILE_MB))
|
||||
[ "$MAX_TOTAL" -lt 1 ] && MAX_TOTAL=1
|
||||
ORIG_N="$N"; ORIG_INTRA="$INTRA_CONC"
|
||||
# Shed shards before intra-shard concurrency: fewer bun processes frees
|
||||
# more than narrower ones (each process carries its own heap + WASM).
|
||||
while [ $((N * INTRA_CONC)) -gt "$MAX_TOTAL" ]; do
|
||||
if [ "$N" -gt 1 ]; then N=$((N - 1))
|
||||
elif [ "$INTRA_CONC" -gt 1 ]; then INTRA_CONC=$((INTRA_CONC - 1))
|
||||
else break
|
||||
fi
|
||||
done
|
||||
if [ "$N" != "$ORIG_N" ] || [ "$INTRA_CONC" != "$ORIG_INTRA" ]; then
|
||||
# Fewer shards → more files per shard → each shard legitimately runs
|
||||
# longer. Scale the per-shard cap by the shed ratio so adaptation
|
||||
# doesn't convert memory safety into false WEDGED verdicts.
|
||||
if [ "$N" -lt "$ORIG_N" ]; then
|
||||
SHARD_TIMEOUT=$((SHARD_TIMEOUT * ORIG_N / N))
|
||||
fi
|
||||
MEM_NOTE=" | mem-adapted ${ORIG_N}x${ORIG_INTRA}→${N}x${INTRA_CONC} (avail=${AVAIL_MB}MB, ${MEM_PER_FILE_MB}MB/file, timeout→${SHARD_TIMEOUT}s)"
|
||||
else
|
||||
MEM_NOTE=" | mem-ok (avail=${AVAIL_MB}MB)"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────────
|
||||
# Output directories. Prefer workspace-local .context/, fall back to /tmp.
|
||||
# ──────────────────────────────────────────────────────────────────────────
|
||||
@@ -100,7 +197,7 @@ else
|
||||
mkdir -p "$LOG_DIR" || { echo "ERROR: cannot create log dir" >&2; exit 2; }
|
||||
fi
|
||||
# Clear from prior run.
|
||||
rm -f "$LOG_DIR"/shard-*.log "$LOG_DIR"/shard-*.exit "$LOG_DIR"/shard-*.wedged 2>/dev/null
|
||||
rm -f "$LOG_DIR"/shard-*.log "$LOG_DIR"/shard-*.exit "$LOG_DIR"/shard-*.wedged "$LOG_DIR"/shard-*.start "$LOG_DIR"/shard-*.end 2>/dev/null
|
||||
: > "$FAILURES_LOG"
|
||||
: > "$SUMMARY_FILE"
|
||||
|
||||
@@ -114,7 +211,7 @@ elif command -v timeout >/dev/null 2>&1; then TIMEOUT_BIN="timeout"
|
||||
fi
|
||||
|
||||
START_TS=$(date +%s)
|
||||
echo "[unit-parallel] N=$N shards | --max-concurrency=$INTRA_CONC | timeout=${SHARD_TIMEOUT}s | kill-after=${SHARD_KILL_AFTER}s | logs=$LOG_DIR" >&2
|
||||
echo "[unit-parallel] N=$N shards | --max-concurrency=$INTRA_CONC | timeout=${SHARD_TIMEOUT}s | kill-after=${SHARD_KILL_AFTER}s | logs=$LOG_DIR${MEM_NOTE}" >&2
|
||||
|
||||
if [ "$DRY_RUN" = "1" ]; then
|
||||
echo "[unit-parallel] dry-run: would spawn $N shards with the above settings."
|
||||
@@ -133,6 +230,7 @@ SHARD_PIDS=()
|
||||
for i in $(seq 1 "$N"); do
|
||||
(
|
||||
SHARD_LOG="$LOG_DIR/shard-$i.log"
|
||||
date +%s > "$LOG_DIR/shard-$i.start"
|
||||
if [ -n "$TIMEOUT_BIN" ]; then
|
||||
"$TIMEOUT_BIN" --signal=TERM --kill-after="${SHARD_KILL_AFTER}s" "${SHARD_TIMEOUT}s" \
|
||||
env SHARD="$i/$N" \
|
||||
@@ -162,6 +260,7 @@ for i in $(seq 1 "$N"); do
|
||||
kill "$cap_pid" 2>/dev/null
|
||||
wait "$cap_pid" 2>/dev/null
|
||||
fi
|
||||
date +%s > "$LOG_DIR/shard-$i.end"
|
||||
echo "$rc" > "$LOG_DIR/shard-$i.exit"
|
||||
{ [ "$rc" = "124" ] || [ "$rc" = "137" ]; } && echo "WEDGED" > "$LOG_DIR/shard-$i.wedged"
|
||||
) &
|
||||
@@ -316,6 +415,40 @@ TOTAL_FAILURES=0
|
||||
TOTAL_PASS=0
|
||||
TOTAL_SKIP=0
|
||||
TOTAL_RC=0
|
||||
|
||||
# Layer 2 state (serial OOM rescue). A shard whose log carries the WASM
|
||||
# out-of-memory signature gets its failing files queued for a serial re-run;
|
||||
# NON_OOM_FAIL records that at least one failure exists that the rescue lane
|
||||
# must NOT absolve (plain assertion failures, wedges without the signature).
|
||||
OOM_RE='Out of memory|WebAssembly\.Memory|RuntimeError: [Aa]borted|Aborted\(\)'
|
||||
OOM_RESCUE_LIST="$LOG_DIR/oom-rescue-files.txt"
|
||||
: > "$OOM_RESCUE_LIST"
|
||||
NON_OOM_FAIL=0
|
||||
# Set when any shard was killed externally — killed-midrun shards leave lock/
|
||||
# state residue that can poison the LATER serial pass, so serial failures are
|
||||
# only rescue-eligible under this flag (or their own OOM signature). A flaky
|
||||
# serial test in an otherwise-clean run must stay red.
|
||||
EXTERNAL_KILL_ANY=0
|
||||
|
||||
# failing_files_in_log: attribute each `(fail)` block to the test file whose
|
||||
# `path.test.ts:` header most recently preceded it in bun's output. Under
|
||||
# GITHUB_ACTIONS the shard wraps each file section as `::group::path.test.ts:`
|
||||
# — strip that prefix or the rescue pass feeds bun literal `::group::...`
|
||||
# non-paths that match zero test files (CI-only; local runs have no groups).
|
||||
failing_files_in_log() {
|
||||
local file="$1"
|
||||
[ -f "$file" ] || return 0
|
||||
awk '
|
||||
/^(::group::)?[^ ].*\.test\.ts:$/ {
|
||||
current = $0
|
||||
sub(/^::group::/, "", current)
|
||||
current = substr(current, 1, length(current) - 1)
|
||||
next
|
||||
}
|
||||
/^\(fail\) / && current != "" { print current }
|
||||
' "$file" | sort -u
|
||||
}
|
||||
|
||||
for i in $(seq 1 "$N"); do
|
||||
SHARD_LOG="$LOG_DIR/shard-$i.log"
|
||||
EXIT_FILE="$LOG_DIR/shard-$i.exit"
|
||||
@@ -330,17 +463,74 @@ for i in $(seq 1 "$N"); do
|
||||
TOTAL_FAILURES=$((TOTAL_FAILURES + fail_count))
|
||||
TOTAL_SKIP=$((TOTAL_SKIP + skip_count))
|
||||
|
||||
shard_oom=0
|
||||
if [ "$rc" != "0" ] && [ "${GBRAIN_TEST_NO_OOM_FALLBACK:-0}" != "1" ] \
|
||||
&& [ -f "$SHARD_LOG" ] && grep -qE "$OOM_RE" "$SHARD_LOG"; then
|
||||
shard_oom=1
|
||||
fi
|
||||
|
||||
# External-kill detection: rc 143 (SIGTERM) / 137 (SIGKILL) with the shard
|
||||
# dying before 80% of the shard timeout means something OUTSIDE the runner
|
||||
# killed it — sibling Conductor workspaces' process cleanup and macOS
|
||||
# memory jetsam both present exactly this way (observed: 3 shards TERM'd +
|
||||
# 1 KILL'd at ~700s under a 3000s cap, all mid-progress). A REAL wedge is
|
||||
# killed BY the runner at ~SHARD_TIMEOUT and stays red. Externally-killed
|
||||
# shards are phantoms: queue for the serial rescue lane like OOM.
|
||||
shard_external_kill=0
|
||||
if [ "$shard_oom" = "0" ] && [ "${GBRAIN_TEST_NO_OOM_FALLBACK:-0}" != "1" ] \
|
||||
&& { [ "$rc" = "143" ] || [ "$rc" = "137" ]; }; then
|
||||
s_start=$(cat "$LOG_DIR/shard-$i.start" 2>/dev/null) || s_start=""
|
||||
s_end=$(cat "$LOG_DIR/shard-$i.end" 2>/dev/null) || s_end=""
|
||||
if [ -n "$s_start" ] && [ -n "$s_end" ]; then
|
||||
s_elapsed=$((s_end - s_start))
|
||||
if [ "$s_elapsed" -lt $((SHARD_TIMEOUT * 80 / 100)) ]; then
|
||||
shard_external_kill=1
|
||||
EXTERNAL_KILL_ANY=1
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -f "$WEDGED_FILE" ]; then
|
||||
TOTAL_RC=1
|
||||
if [ "$shard_external_kill" = "1" ]; then
|
||||
SHARD="$i/$N" bash scripts/run-unit-shard.sh --dry-run-list 2>/dev/null >> "$OOM_RESCUE_LIST"
|
||||
echo "shard $i/$N: KILLED externally after ${s_elapsed}s (rc=$rc, well before ${SHARD_TIMEOUT}s cap — queued for serial rescue)" >> "$SUMMARY_FILE"
|
||||
elif [ "$shard_oom" = "1" ]; then
|
||||
# Wedged UNDER memory pressure: we can't attribute failures, so queue
|
||||
# the shard's entire file list for the serial rescue pass.
|
||||
SHARD="$i/$N" bash scripts/run-unit-shard.sh --dry-run-list 2>/dev/null >> "$OOM_RESCUE_LIST"
|
||||
echo "shard $i/$N: WEDGED after ${SHARD_TIMEOUT}s (rc=$rc, OOM signature — queued for serial rescue)" >> "$SUMMARY_FILE"
|
||||
else
|
||||
NON_OOM_FAIL=1
|
||||
echo "shard $i/$N: WEDGED after ${SHARD_TIMEOUT}s (rc=$rc)" >> "$SUMMARY_FILE"
|
||||
fi
|
||||
{
|
||||
echo "--- shard $i: WEDGED after ${SHARD_TIMEOUT}s ---"
|
||||
[ -f "$SHARD_LOG" ] && tail -50 "$SHARD_LOG"
|
||||
echo ""
|
||||
} >> "$FAILURES_LOG"
|
||||
echo "shard $i/$N: WEDGED after ${SHARD_TIMEOUT}s (rc=$rc)" >> "$SUMMARY_FILE"
|
||||
continue
|
||||
fi
|
||||
|
||||
if [ "$rc" != "0" ]; then
|
||||
if [ "$shard_oom" = "1" ]; then
|
||||
# One scan, reused for both the queue append and the emptiness check.
|
||||
shard_failing_files=$(failing_files_in_log "$SHARD_LOG")
|
||||
if [ -n "$shard_failing_files" ]; then
|
||||
printf '%s\n' "$shard_failing_files" >> "$OOM_RESCUE_LIST"
|
||||
else
|
||||
# OOM signature but no attributable files (e.g. bun died before any
|
||||
# file header) → rescue the whole shard.
|
||||
SHARD="$i/$N" bash scripts/run-unit-shard.sh --dry-run-list 2>/dev/null >> "$OOM_RESCUE_LIST"
|
||||
fi
|
||||
elif [ "$shard_external_kill" = "1" ]; then
|
||||
SHARD="$i/$N" bash scripts/run-unit-shard.sh --dry-run-list 2>/dev/null >> "$OOM_RESCUE_LIST"
|
||||
echo "shard $i/$N: KILLED externally after ${s_elapsed}s (rc=$rc — queued for serial rescue)" >> "$SUMMARY_FILE"
|
||||
else
|
||||
NON_OOM_FAIL=1
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "shard $i/$N: pass=$pass_count fail=$fail_count skip=$skip_count rc=$rc" >> "$SUMMARY_FILE"
|
||||
|
||||
if [ "$rc" != "0" ]; then
|
||||
@@ -395,6 +585,17 @@ if [ "$SERIAL_FILES_COUNT" -gt 0 ]; then
|
||||
cat "$LOG_DIR/serial.log"
|
||||
if [ "$SERIAL_RC" != "0" ]; then
|
||||
TOTAL_RC=1
|
||||
if [ "${GBRAIN_TEST_NO_OOM_FALLBACK:-0}" != "1" ] \
|
||||
&& { grep -qE "$OOM_RE" "$LOG_DIR/serial.log" || [ "$EXTERNAL_KILL_ANY" = "1" ]; }; then
|
||||
# Serial failures are rescue-eligible ONLY with their own OOM signature
|
||||
# or when an externally-killed shard ran earlier in this invocation
|
||||
# (killed-midrun shards leave lock/state residue that poisons the serial
|
||||
# pass). A merely-OOM'd sibling shard is NOT grounds — a flaky serial
|
||||
# test must stay red rather than get silently absolved.
|
||||
failing_files_in_log "$LOG_DIR/serial.log" >> "$OOM_RESCUE_LIST"
|
||||
else
|
||||
NON_OOM_FAIL=1
|
||||
fi
|
||||
s_fail=$(bun_summary_count "fail" "$LOG_DIR/serial.log")
|
||||
TOTAL_FAILURES=$((TOTAL_FAILURES + s_fail))
|
||||
if [ "$s_fail" -gt 0 ]; then
|
||||
@@ -420,6 +621,92 @@ if [ "$SERIAL_FILES_COUNT" -gt 0 ]; then
|
||||
fi
|
||||
fi
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────────
|
||||
# Layer 2: serial OOM rescue. Re-run every file that failed inside an
|
||||
# OOM-signature shard, one at a time (1 shard, --max-concurrency 1), after
|
||||
# the parallel fan-out has fully drained. Phantom failures (the WASM ran out
|
||||
# of memory because 16 instances were up at once) pass here and the run goes
|
||||
# green with an oom_rescued note; real failures fail again and stay red.
|
||||
# ──────────────────────────────────────────────────────────────────────────
|
||||
OOM_RESCUED=0
|
||||
OOM_RESCUE_NOTE=""
|
||||
sort -u "$OOM_RESCUE_LIST" -o "$OOM_RESCUE_LIST" 2>/dev/null
|
||||
# grep -c exits 1 on zero matches — assign in two steps so an empty rescue
|
||||
# list yields a single "0" (the grep_count double-output bug, same class).
|
||||
RESCUE_COUNT=$(grep -c . "$OOM_RESCUE_LIST" 2>/dev/null) || RESCUE_COUNT=0
|
||||
if [ "$TOTAL_RC" != "0" ] && [ "${RESCUE_COUNT:-0}" -gt 0 ]; then
|
||||
echo "════════════ OOM rescue pass ($RESCUE_COUNT files, serial) ════════════"
|
||||
echo "[unit-parallel] OOM signature detected — re-running $RESCUE_COUNT failing file(s) at --max-concurrency 1" >&2
|
||||
RESCUE_LOG="$LOG_DIR/oom-rescue.log"
|
||||
# 60s-per-file floor with the shard cap as a minimum, and 2x the shard cap
|
||||
# as a CEILING: a wedged shard queueing its whole file list must not turn
|
||||
# `bun run test` into an unbounded multi-hour serial re-run — hitting the
|
||||
# ceiling reads as a red rescue, not silence.
|
||||
RESCUE_TIMEOUT=$((RESCUE_COUNT * 60))
|
||||
[ "$RESCUE_TIMEOUT" -lt "$SHARD_TIMEOUT" ] && RESCUE_TIMEOUT="$SHARD_TIMEOUT"
|
||||
[ "$RESCUE_TIMEOUT" -gt $((SHARD_TIMEOUT * 2)) ] && RESCUE_TIMEOUT=$((SHARD_TIMEOUT * 2))
|
||||
# Split the queue: *.serial.test.ts files require one bun PROCESS per file
|
||||
# (run-serial-tests.sh's isolation contract — top-level mock.module leaks
|
||||
# across files in a shared registry); the remainder batches in one process.
|
||||
# Both lanes mirror the shard invocation's --timeout=60000 — bun's default
|
||||
# 5s per-test timeout would re-fail PGLite phantoms (120-migration replay)
|
||||
# and mislabel them 'confirmed real'.
|
||||
grep -v '\.serial\.test\.ts$' "$OOM_RESCUE_LIST" > "$LOG_DIR/oom-rescue-batch.txt" || true
|
||||
grep '\.serial\.test\.ts$' "$OOM_RESCUE_LIST" > "$LOG_DIR/oom-rescue-serial.txt" || true
|
||||
RESCUE_RC=0
|
||||
: > "$RESCUE_LOG"
|
||||
run_rescue() { # $1 = per-invocation timeout seconds; rest = test-file args
|
||||
local t="$1"; shift
|
||||
if [ -n "$TIMEOUT_BIN" ]; then
|
||||
"$TIMEOUT_BIN" --signal=TERM --kill-after="${SHARD_KILL_AFTER}s" "${t}s" \
|
||||
bun test --max-concurrency 1 --timeout=60000 "$@" >> "$RESCUE_LOG" 2>&1
|
||||
else
|
||||
bun test --max-concurrency 1 --timeout=60000 "$@" >> "$RESCUE_LOG" 2>&1
|
||||
fi
|
||||
}
|
||||
if [ -s "$LOG_DIR/oom-rescue-batch.txt" ]; then
|
||||
# shellcheck disable=SC2046
|
||||
run_rescue "$RESCUE_TIMEOUT" $(cat "$LOG_DIR/oom-rescue-batch.txt") || RESCUE_RC=1
|
||||
fi
|
||||
if [ -s "$LOG_DIR/oom-rescue-serial.txt" ]; then
|
||||
while IFS= read -r serial_file; do
|
||||
[ -n "$serial_file" ] || continue
|
||||
run_rescue 300 "$serial_file" || RESCUE_RC=1
|
||||
done < "$LOG_DIR/oom-rescue-serial.txt"
|
||||
fi
|
||||
cat "$RESCUE_LOG"
|
||||
r_pass=$(bun_summary_count "pass" "$RESCUE_LOG")
|
||||
r_fail=$(bun_summary_count "fail" "$RESCUE_LOG")
|
||||
if [ "$RESCUE_RC" = "0" ] && [ "$NON_OOM_FAIL" = "0" ]; then
|
||||
# Every failure in the run was OOM-phantom and every rescued file passed
|
||||
# serially: the run is green. Adjust the headline numbers so they reflect
|
||||
# the rescue verdict, and mark the earlier failure blocks superseded.
|
||||
TOTAL_RC=0
|
||||
OOM_RESCUED=1
|
||||
# Do NOT fold r_pass into TOTAL_PASS — the failing shard's own summary
|
||||
# already counted the rescued files' passing tests, so folding would
|
||||
# double-count. Rescue results ride in the note instead.
|
||||
TOTAL_FAILURES=0
|
||||
OOM_RESCUE_NOTE=" | oom_rescued=${RESCUE_COUNT}files(${r_pass}p serial)"
|
||||
{
|
||||
echo "--- OOM rescue: all $RESCUE_COUNT file(s) passed serially (${r_pass} tests) ---"
|
||||
echo "--- failure blocks above were WASM out-of-memory phantoms, superseded ---"
|
||||
} >> "$FAILURES_LOG"
|
||||
echo "oom-rescue: $RESCUE_COUNT files pass=$r_pass rc=0 (phantom OOM failures superseded)" >> "$SUMMARY_FILE"
|
||||
else
|
||||
# Real failures confirmed serially (or a non-OOM failure exists anyway).
|
||||
OOM_RESCUE_NOTE=" | oom_rescue_failed=${r_fail}real"
|
||||
awk '
|
||||
/^\(fail\) / { in_block=1; print "--- oom-rescue (serial, confirmed real): " $0; next }
|
||||
in_block {
|
||||
if (/^\(pass\)/ || /^\(skip\)/ || /^[[:space:]]*$/ || /__bun_test_summary__/) { in_block=0; print ""; next }
|
||||
print $0
|
||||
}
|
||||
' "$RESCUE_LOG" >> "$FAILURES_LOG"
|
||||
echo "oom-rescue: $RESCUE_COUNT files pass=$r_pass fail=$r_fail rc=$RESCUE_RC (real failures confirmed)" >> "$SUMMARY_FILE"
|
||||
fi
|
||||
fi
|
||||
|
||||
END_TS=$(date +%s)
|
||||
ELAPSED=$((END_TS - START_TS))
|
||||
|
||||
@@ -436,10 +723,10 @@ if [ "$TOTAL_RC" != "0" ]; then
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
tail -30 "$FAILURES_LOG"
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
echo "[unit-parallel] elapsed=${ELAPSED}s | pass=$TOTAL_PASS fail=$TOTAL_FAILURES skip=$TOTAL_SKIP"
|
||||
echo "[unit-parallel] elapsed=${ELAPSED}s | pass=$TOTAL_PASS fail=$TOTAL_FAILURES skip=$TOTAL_SKIP${OOM_RESCUE_NOTE}"
|
||||
} >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "[unit-parallel] elapsed=${ELAPSED}s | pass=$TOTAL_PASS fail=$TOTAL_FAILURES skip=$TOTAL_SKIP" >&2
|
||||
echo "[unit-parallel] elapsed=${ELAPSED}s | pass=$TOTAL_PASS fail=$TOTAL_FAILURES skip=$TOTAL_SKIP${OOM_RESCUE_NOTE}" >&2
|
||||
exit 0
|
||||
|
||||
+186
-6
@@ -33,6 +33,7 @@ import { parseGlobalFlags, setCliOptions, getCliOptions } from './core/cli-optio
|
||||
import type { CliOptions } from './core/cli-options.ts';
|
||||
import { callRemoteTool, RemoteMcpError, unpackToolResult } from './core/mcp-client.ts';
|
||||
import { maybePromptForUpgrade } from './core/thin-client-upgrade-prompt.ts';
|
||||
import { CLI_FLAG_REGISTRY } from './core/cli-flag-registry.generated.ts';
|
||||
import { VERSION } from './version.ts';
|
||||
|
||||
// Build CLI name -> operation lookup
|
||||
@@ -54,8 +55,19 @@ export function bigintToStringReplacer(_key: string, value: unknown): unknown {
|
||||
return typeof value === 'bigint' ? value.toString() : value;
|
||||
}
|
||||
|
||||
// ENG-2 renderer parity: round-trip a local-engine op's return value so
|
||||
// renderers see the same shape the routed path produces. Bigint-safe via
|
||||
// bigintToStringReplacer. Exported for tests (same import-safety contract as
|
||||
// cliAliases/formatResult). (#2450)
|
||||
export function normalizeLocalResult(rawResult: unknown): unknown {
|
||||
return JSON.parse(JSON.stringify(rawResult, bigintToStringReplacer));
|
||||
}
|
||||
|
||||
// CLI-only commands that bypass the operation layer
|
||||
export const CLI_ONLY = new Set(['init', 'reinit-pglite', 'pglite-repair', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'extract-conversation-facts', 'enrich', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'maintain', 'sources', 'mounts', 'dream', 'check-resolvable', 'routing-eval', 'skillify', 'smoke-test', 'providers', 'storage', 'repos', 'code-def', 'code-refs', 'reindex', 'reindex-code', 'reindex-frontmatter', 'code-callers', 'code-callees', 'reconcile-links', 'frontmatter', 'auth', 'friction', 'claw-test', 'book-mirror', 'takes', 'think', 'salience', 'anomalies', 'calibration', 'transcripts', 'models', 'remote', 'recall', 'forget', 'edges-backfill', 'cache', 'ze-switch', 'retrieval-upgrade', 'founder', 'brainstorm', 'lsd', 'schema', 'capture', 'onboard', 'conversation-parser', 'status', 'connect', 'skillopt', 'quarantine', 'self-upgrade', 'advisor', 'watch', 'reindex-search-vector', 'pages', 'bench', 'backfill']);
|
||||
export const CLI_ONLY = new Set(['init', 'reinit-pglite', 'pglite-repair', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'extract-conversation-facts', 'enrich', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'maintain', 'sources', 'mounts', 'dream', 'check-resolvable', 'routing-eval', 'skillify', 'smoke-test', 'providers', 'storage', 'repos', 'code-def', 'code-refs', 'reindex', 'reindex-code', 'reindex-frontmatter', 'code-callers', 'code-callees', 'reconcile-links', 'frontmatter', 'auth', 'friction', 'claw-test', 'book-mirror', 'takes', 'think', 'salience', 'anomalies', 'calibration', 'transcripts', 'models', 'remote', 'recall', 'forget', 'edges-backfill', 'cache', 'ze-switch', 'retrieval-upgrade', 'founder', 'brainstorm', 'lsd', 'schema', 'capture', 'onboard', 'conversation-parser', 'status', 'connect', 'skillopt', 'quarantine', 'self-upgrade', 'advisor', 'watch', 'reindex-search-vector', 'pages', 'bench', 'backfill',
|
||||
// v0.42.58 (#2035 class, caught by the handleCliOnly reachability sweep):
|
||||
// full handler at `case 'notability-eval'` but never dispatchable.
|
||||
'notability-eval']);
|
||||
// CLI-only commands whose handlers print their own --help text. These are
|
||||
// excluded from the generic short-circuit so detailed per-command and
|
||||
// per-subcommand usage stays reachable.
|
||||
@@ -328,6 +340,30 @@ async function main() {
|
||||
}
|
||||
}
|
||||
|
||||
// #2185: strict unknown-flag validation — pre-dispatch, pre-engine. A flag
|
||||
// no handler consults (the repro: `init --migrate-only --dry-run` applying
|
||||
// REAL migrations while the user asked for a rehearsal) fails loud here
|
||||
// instead of silently doing the destructive thing. Runs after the --help
|
||||
// short-circuit so `gbrain x --help` never errors; runs before any dispatch
|
||||
// or engine connect so the error is instant and side-effect-free.
|
||||
{
|
||||
const unknown = validateCommandFlags(command, subArgs);
|
||||
if (unknown) {
|
||||
// Message contract shared with init.ts's in-handler check (which this
|
||||
// pre-dispatch validator now reaches first): lowercase 'unknown flag'
|
||||
// on stderr; --json callers get the structured error on stdout with
|
||||
// reason 'invalid_flag' (pinned by test/init-migrate-only.test.ts).
|
||||
const message = `unknown flag ${unknown} for 'gbrain ${command}'`;
|
||||
// Both --json spellings get the structured envelope (--json=false opts out).
|
||||
if (subArgs.some(a => a === '--json' || (a.startsWith('--json=') && a !== '--json=false'))) {
|
||||
process.stdout.write(JSON.stringify({ status: 'error', reason: 'invalid_flag', message }) + '\n');
|
||||
}
|
||||
console.error(`gbrain ${command}: ${message}`);
|
||||
console.error(`Run: gbrain ${command} --help`);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
// DB-free durability pull (v0.42.44 D2): the harden cron calls
|
||||
// `gbrain sources pull --path <dir>` every ~30 min. It must NOT open PGLite
|
||||
// (a live long-lived session holds the single-writer lock), so handle it
|
||||
@@ -512,8 +548,8 @@ async function main() {
|
||||
// path's return value so renderers see the same shape they'd see on the
|
||||
// routed path. Date → ISO string; bigint → string (postgres.js shape);
|
||||
// Buffer → object. Microsecond-cost; eliminates a whole drift bug class.
|
||||
const result = JSON.parse(JSON.stringify(rawResult, bigintToStringReplacer));
|
||||
const output = formatResult(op.name, result);
|
||||
const result = normalizeLocalResult(rawResult);
|
||||
const output = formatResult(op.name, result, params);
|
||||
if (output) process.stdout.write(output);
|
||||
} catch (e: unknown) {
|
||||
// v0.42.20.0 (codex D4): on error, set exitCode + return so the `finally`
|
||||
@@ -596,7 +632,7 @@ async function runThinClientRouted(
|
||||
signal: sigintController.signal,
|
||||
});
|
||||
const result = unpackToolResult(raw);
|
||||
const output = formatResult(op.name, result);
|
||||
const output = formatResult(op.name, result, params);
|
||||
if (output) process.stdout.write(output);
|
||||
} catch (e: unknown) {
|
||||
if (e instanceof RemoteMcpError) {
|
||||
@@ -814,6 +850,29 @@ export function parseOpArgs(op: Operation, args: string[]): Record<string, unkno
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
const arg = args[i];
|
||||
if (arg.startsWith('--')) {
|
||||
// #2185: `--key=value` inline form. Pre-fix this parsed as junk key
|
||||
// 'key=value' and consumed the NEXT token as its value, corrupting
|
||||
// positional parsing. Recognized here so the strict-flag validator and
|
||||
// the parser agree on the idiom.
|
||||
const eq = arg.indexOf('=');
|
||||
if (eq > 2) {
|
||||
const key = arg.slice(2, eq).replace(/-/g, '_');
|
||||
// CLI-local booleans: `--json=<v>` / `--dry-run=<v>` must parse as
|
||||
// booleans, not fall through to the junk-key path (which would
|
||||
// consume the NEXT token as a value and corrupt positional parsing).
|
||||
if (key === 'json' || key === 'dry_run') {
|
||||
params[key] = arg.slice(eq + 1) !== 'false';
|
||||
continue;
|
||||
}
|
||||
const def = op.params[key];
|
||||
if (def) {
|
||||
const raw = arg.slice(eq + 1);
|
||||
params[key] = def.type === 'boolean' ? raw !== 'false'
|
||||
: def.type === 'number' ? Number(raw)
|
||||
: raw;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
if (arg.startsWith('--no-')) {
|
||||
const positiveKey = arg.slice(5).replace(/-/g, '_');
|
||||
const positiveDef = op.params[positiveKey];
|
||||
@@ -826,6 +885,14 @@ export function parseOpArgs(op: Operation, args: string[]): Record<string, unkno
|
||||
const paramDef = op.params[key];
|
||||
if (paramDef?.type === 'boolean') {
|
||||
params[key] = true;
|
||||
} else if (key === 'json' || key === 'dry_run') {
|
||||
// CLI-local booleans, intentionally NOT on the operation contract
|
||||
// exposed over MCP/tools: json is the formatter flag; dry_run feeds
|
||||
// makeContext's ctx.dryRun. Both must never consume a value token —
|
||||
// pre-fix, `gbrain delete x --dry-run` (trailing) set NOTHING, so
|
||||
// ctx.dryRun stayed false and the REAL delete ran despite the
|
||||
// rehearsal request (the resurrected #2185 class the red team caught).
|
||||
params[key] = true;
|
||||
} else if (i + 1 < args.length) {
|
||||
params[key] = args[++i];
|
||||
if (paramDef?.type === 'number') params[key] = Number(params[key]);
|
||||
@@ -987,6 +1054,112 @@ export function applyThinClientSourceScope(
|
||||
}
|
||||
|
||||
// Exported for tests (same import-safety contract as applyThinClientSourceScope).
|
||||
// ─────────────────────────────────────────────────────────────────
|
||||
// #2185 — strict unknown-flag validation (pre-dispatch, pre-engine).
|
||||
// A flag no handler consults must fail loud instead of silently doing the
|
||||
// destructive thing (`init --migrate-only --dry-run` applied REAL migrations
|
||||
// while the user asked for a rehearsal). Two lanes:
|
||||
// - op commands: legal flags derive from the operation contract
|
||||
// (op.params) + the CLI-local formatter flags, mirroring parseOpArgs's
|
||||
// traversal so values that begin with '--' are never misread.
|
||||
// - CLI_ONLY commands: legal flags come from the generated
|
||||
// CLI_FLAG_REGISTRY (scripts/generate-flag-registry.ts scans each
|
||||
// command's source; freshness + coverage pinned by
|
||||
// test/cli-flag-validation.test.ts).
|
||||
// Everything after a literal `--` is passthrough and never validated.
|
||||
// ─────────────────────────────────────────────────────────────────
|
||||
|
||||
// Exempt by contract, not oversight:
|
||||
// - call: the generic op invoker — arbitrary --param names are its interface.
|
||||
// - config: `config set <key> <value>` values are arbitrary strings.
|
||||
// - jobs submit: job payloads carry handler-defined params (shell lane incl.).
|
||||
function flagValidationExempt(command: string, subArgs: string[]): boolean {
|
||||
return command === 'call' || command === 'config'
|
||||
|| (command === 'jobs' && subArgs[0] === 'submit');
|
||||
}
|
||||
|
||||
/** Returns the first unknown flag (e.g. '--dry-run') or null when clean. */
|
||||
export function validateCommandFlags(command: string, subArgs: string[]): string | null {
|
||||
if (flagValidationExempt(command, subArgs)) return null;
|
||||
// Lane order MUST mirror dispatch order (CLI_ONLY first): commands that are
|
||||
// BOTH an op and a CLI_ONLY member (think, salience, anomalies) dispatch to
|
||||
// handleCliOnly, whose handlers parse flags the op contract doesn't declare
|
||||
// (`salience --kind`, `think --with-calibration`) — validating those
|
||||
// against op.params rejected documented invocations.
|
||||
if (CLI_ONLY.has(command)) {
|
||||
const legal = CLI_FLAG_REGISTRY[command];
|
||||
// Registry drift fails OPEN at runtime (never brick a command); the
|
||||
// drift-guard test fails the build instead.
|
||||
if (!legal) return null;
|
||||
return findUnknownFlag(subArgs, new Set(legal));
|
||||
}
|
||||
const op = cliOps.get(command) ?? cliAliases.get(command);
|
||||
if (op) return findUnknownOpFlag(op, subArgs);
|
||||
return null; // unknown command — the dispatcher's own error handles it
|
||||
}
|
||||
|
||||
/** CLI_ONLY lane: token scan against the generated legal set. */
|
||||
export function findUnknownFlag(args: string[], legal: ReadonlySet<string>): string | null {
|
||||
for (const a of args) {
|
||||
if (a === '--') break;
|
||||
const m = /^--([a-z0-9][a-z0-9-]*)(?:=.*)?$/i.exec(a);
|
||||
if (!m) continue;
|
||||
// Casing typo = unknown flag: every handler in the repo is
|
||||
// case-sensitive-lowercase, so `--MIGRATE-ONLY` passing validation would
|
||||
// just be silently ignored downstream — the exact class this validator
|
||||
// exists to kill.
|
||||
if (/[A-Z]/.test(m[1])) return `--${m[1]}`;
|
||||
const name = `--${m[1]}`;
|
||||
if (legal.has(name)) continue;
|
||||
// --no-<flag> negation of a known flag is legal.
|
||||
if (name.startsWith('--no-') && legal.has(`--${name.slice(5)}`)) continue;
|
||||
return name;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/** Op lane: mirrors parseOpArgs so flag VALUES starting with '--' are skipped. */
|
||||
export function findUnknownOpFlag(op: Operation, args: string[]): string | null {
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
const a = args[i];
|
||||
if (a === '--') break;
|
||||
const m = /^--([a-z0-9][a-z0-9-]*)(?:=(.*))?$/i.exec(a);
|
||||
if (!m) continue;
|
||||
// Casing typo = unknown flag (see findUnknownFlag).
|
||||
if (/[A-Z]/.test(m[1])) return `--${m[1]}`;
|
||||
const rawKey = m[1];
|
||||
// CLI-local flags consumed OUTSIDE the op contract (never wire params):
|
||||
// json/explain — formatter flags; help — short-circuits pre-dispatch;
|
||||
// source — makeContext's 6-tier source resolution (deleted before wire);
|
||||
// dry-run — makeContext's ctx.dryRun projection.
|
||||
// Pre-fix, rejecting these broke documented invocations
|
||||
// (`gbrain search "x" --source y`, `gbrain put x --dry-run`).
|
||||
if (rawKey === 'json') continue;
|
||||
if ((rawKey === 'explain' || rawKey === 'help') && m[2] === undefined) continue;
|
||||
if (rawKey === 'source' || rawKey === 'dry-run') {
|
||||
// Non-boolean-style CLI-locals consume the next token as their value
|
||||
// in parseOpArgs (source does; dry-run is boolean-read) — mirror the
|
||||
// parser: source consumes a value when not inline-`=`.
|
||||
if (rawKey === 'source' && m[2] === undefined) i++;
|
||||
continue;
|
||||
}
|
||||
if (rawKey.startsWith('no-')) {
|
||||
const positive = rawKey.slice(3).replace(/-/g, '_');
|
||||
if (op.params[positive]?.type === 'boolean') continue;
|
||||
}
|
||||
const key = rawKey.replace(/-/g, '_');
|
||||
const paramDef = op.params[key];
|
||||
if (paramDef) {
|
||||
// Non-boolean flags consume the next token as their value unless
|
||||
// provided inline via `=` — exactly like parseOpArgs.
|
||||
if (paramDef.type !== 'boolean' && m[2] === undefined) i++;
|
||||
continue;
|
||||
}
|
||||
return `--${rawKey}`;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
export async function makeContext(engine: BrainEngine, params: Record<string, unknown>): Promise<OperationContext> {
|
||||
// v0.31.8 (D11): resolve sourceId via the canonical 6-tier chain. Honors
|
||||
// --source / GBRAIN_SOURCE / .gbrain-source / path-match / brain default /
|
||||
@@ -1041,7 +1214,11 @@ export async function makeContext(engine: BrainEngine, params: Record<string, un
|
||||
}
|
||||
|
||||
// Exported for tests (same import-safety contract as cliAliases/printOpHelp).
|
||||
export function formatResult(opName: string, result: unknown): string {
|
||||
export function formatResult(
|
||||
opName: string,
|
||||
result: unknown,
|
||||
params: Record<string, unknown> = {},
|
||||
): string {
|
||||
switch (opName) {
|
||||
case 'volunteer_context': {
|
||||
const r = result as any;
|
||||
@@ -1080,6 +1257,7 @@ export function formatResult(opName: string, result: unknown): string {
|
||||
case 'search':
|
||||
case 'query': {
|
||||
const results = result as any[];
|
||||
if (params.json === true) return JSON.stringify(results, null, 2) + '\n';
|
||||
if (results.length === 0) return 'No results.\n';
|
||||
// v0.40.4 — --explain switches to per-stage attribution formatter.
|
||||
// Reads CliOptions.explain via the module-level singleton.
|
||||
@@ -1166,7 +1344,9 @@ export function formatResult(opName: string, result: unknown): string {
|
||||
).join('\n') + '\n';
|
||||
}
|
||||
default:
|
||||
return JSON.stringify(result, null, 2) + '\n';
|
||||
// bigintToStringReplacer keeps this fallback renderer crash-proof even
|
||||
// if a future caller hands it a not-yet-normalized result. (#2450)
|
||||
return JSON.stringify(result, bigintToStringReplacer, 2) + '\n';
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+27
-8
@@ -376,19 +376,38 @@ export async function jsonbIntegrityCheck(
|
||||
progress?: Pick<ProgressReporter, 'heartbeat'>,
|
||||
): Promise<Check> {
|
||||
try {
|
||||
const targets: Array<{ table: string; col: string; expected: 'object' | 'array' }> = [
|
||||
{ table: 'pages', col: 'frontmatter', expected: 'object' },
|
||||
{ table: 'raw_data', col: 'data', expected: 'object' },
|
||||
{ table: 'ingest_log', col: 'pages_updated', expected: 'array' },
|
||||
{ table: 'files', col: 'metadata', expected: 'object' },
|
||||
{ table: 'page_versions', col: 'frontmatter', expected: 'object' },
|
||||
const targets: Array<{ table: string; col: string; expected: 'object' | 'array'; jsonPayloadOnly?: boolean }> = [
|
||||
{ table: 'pages', col: 'frontmatter', expected: 'object' },
|
||||
{ table: 'raw_data', col: 'data', expected: 'object' },
|
||||
{ table: 'ingest_log', col: 'pages_updated', expected: 'array' },
|
||||
{ table: 'files', col: 'metadata', expected: 'object' },
|
||||
{ table: 'page_versions', col: 'frontmatter', expected: 'object' },
|
||||
// Subagent persistence — second double-encode site (historical damage
|
||||
// rows from the pre-v0.42.53.0 positional bind; write paths fixed in
|
||||
// #2375). Mirrors repair-jsonb's targets incl. jsonPayloadOnly: these
|
||||
// columns can legitimately hold jsonb STRING scalars (persistToolExec
|
||||
// binds pre-serialized string payloads as-is), so only JSON-container
|
||||
// content counts as damage.
|
||||
{ table: 'subagent_messages', col: 'content_blocks', expected: 'array', jsonPayloadOnly: true },
|
||||
{ table: 'subagent_tool_executions', col: 'input', expected: 'object', jsonPayloadOnly: true },
|
||||
{ table: 'subagent_tool_executions', col: 'output', expected: 'object', jsonPayloadOnly: true },
|
||||
];
|
||||
let totalBad = 0;
|
||||
const breakdown: string[] = [];
|
||||
for (const { table, col } of targets) {
|
||||
for (const { table, col, jsonPayloadOnly } of targets) {
|
||||
progress?.heartbeat(`jsonb_integrity.${table}.${col}`);
|
||||
// Skip targets whose table doesn't exist on this brain (subagent_*
|
||||
// tables are v0.15+; pre-v0.15 brains naturally lack them).
|
||||
const existsRows = await engine.executeRaw<{ exists: boolean }>(
|
||||
`SELECT to_regclass($1) IS NOT NULL AS exists`,
|
||||
[table],
|
||||
);
|
||||
if (!existsRows[0]?.exists) continue;
|
||||
const damage = jsonPayloadOnly
|
||||
? `jsonb_typeof(${col}) = 'string' AND (${col} #>> '{}') ~ '^[[:space:]]*[\\[{]' AND pg_input_is_valid(${col} #>> '{}', 'jsonb')`
|
||||
: `jsonb_typeof(${col}) = 'string'`;
|
||||
const rows = await engine.executeRaw<{ n: number }>(
|
||||
`SELECT count(*)::int AS n FROM ${table} WHERE jsonb_typeof(${col}) = 'string'`,
|
||||
`SELECT count(*)::int AS n FROM ${table} WHERE ${damage}`,
|
||||
);
|
||||
const n = Number(rows[0]?.n ?? 0);
|
||||
if (n > 0) { totalBad += n; breakdown.push(`${table}.${col}=${n}`); }
|
||||
|
||||
@@ -16,16 +16,27 @@
|
||||
* (it never wrote string-typed JSONB).
|
||||
*
|
||||
* Affected columns (audit of src/schema.sql):
|
||||
* - pages.frontmatter (postgres-engine.ts:107 putPage)
|
||||
* - raw_data.data (postgres-engine.ts:668 putRawData)
|
||||
* - ingest_log.pages_updated (postgres-engine.ts:846 logIngest)
|
||||
* - files.metadata (commands/files.ts:254 file upload)
|
||||
* - page_versions.frontmatter (downstream of pages.frontmatter via
|
||||
* INSERT...SELECT FROM pages)
|
||||
* - pages.frontmatter (postgres-engine.ts:107 putPage)
|
||||
* - raw_data.data (postgres-engine.ts:668 putRawData)
|
||||
* - ingest_log.pages_updated (postgres-engine.ts:846 logIngest)
|
||||
* - files.metadata (commands/files.ts:254 file upload)
|
||||
* - page_versions.frontmatter (downstream of pages.frontmatter via
|
||||
* INSERT...SELECT FROM pages)
|
||||
* - subagent_messages.content_blocks (subagent.ts:599 persistMessage —
|
||||
* v0.16.0+, write path fixed in
|
||||
* v0.42.53.0 #2375)
|
||||
* - subagent_tool_executions.input (subagent.ts:625/660 persistToolExec
|
||||
* Pending/Failed — same wave)
|
||||
* - subagent_tool_executions.output (subagent.ts:639 persistToolExecComplete
|
||||
* — same wave)
|
||||
*
|
||||
* Other JSONB columns (minion_jobs.{data,result,progress,stacktrace},
|
||||
* minion_inbox.payload) were always written via parameterized form ($N::jsonb
|
||||
* with a string parameter, not interpolation) so they were never affected.
|
||||
* The subagent_* writes were broken via a slightly different shape than the
|
||||
* v0.12.0 wave: they used `engine.executeRaw` (postgres.js `unsafe`) with
|
||||
* `JSON.stringify(value)` + `$N::jsonb` cast. postgres.js's unsafe path
|
||||
* binds the resulting string as text, then the `::jsonb` cast wraps it as
|
||||
* a jsonb string scalar instead of parsing it. queue.ts and other
|
||||
* `executeRaw` callers were not affected because they pass raw objects
|
||||
* (postgres.js v3 auto-encodes objects to jsonb).
|
||||
*/
|
||||
|
||||
import { loadConfig, toEngineConfig } from '../core/config.ts';
|
||||
@@ -39,16 +50,40 @@ interface RepairTarget {
|
||||
column: string;
|
||||
/** Optional secondary key column for logging. */
|
||||
keyCol?: string;
|
||||
/**
|
||||
* Only unwrap string scalars whose CONTENT is a JSON container ({...} or
|
||||
* [...]). The subagent columns can legitimately hold jsonb string scalars
|
||||
* (persistToolExec binds `typeof input === 'string' ? input : stringify`,
|
||||
* so a tool's pre-serialized plain-text payload lands as a JSON string) —
|
||||
* an unconditional unwrap would cast non-JSON text and abort the entire
|
||||
* repair run, or corrupt a legitimate value on a second pass.
|
||||
*/
|
||||
jsonPayloadOnly?: boolean;
|
||||
}
|
||||
|
||||
const TARGETS: RepairTarget[] = [
|
||||
{ table: 'pages', column: 'frontmatter', keyCol: 'slug' },
|
||||
{ table: 'raw_data', column: 'data', keyCol: 'source' },
|
||||
{ table: 'ingest_log', column: 'pages_updated', keyCol: 'source_ref' },
|
||||
{ table: 'files', column: 'metadata', keyCol: 'storage_path' },
|
||||
{ table: 'page_versions', column: 'frontmatter', keyCol: 'snapshot_at' },
|
||||
{ table: 'pages', column: 'frontmatter', keyCol: 'slug' },
|
||||
{ table: 'raw_data', column: 'data', keyCol: 'source' },
|
||||
{ table: 'ingest_log', column: 'pages_updated', keyCol: 'source_ref' },
|
||||
{ table: 'files', column: 'metadata', keyCol: 'storage_path' },
|
||||
{ table: 'page_versions', column: 'frontmatter', keyCol: 'snapshot_at' },
|
||||
{ table: 'subagent_messages', column: 'content_blocks', keyCol: 'job_id', jsonPayloadOnly: true },
|
||||
{ table: 'subagent_tool_executions', column: 'input', keyCol: 'tool_use_id', jsonPayloadOnly: true },
|
||||
{ table: 'subagent_tool_executions', column: 'output', keyCol: 'tool_use_id', jsonPayloadOnly: true },
|
||||
];
|
||||
|
||||
/** The double-encode predicate for a target (see jsonPayloadOnly). */
|
||||
function damagePredicate(t: RepairTarget): string {
|
||||
const base = `jsonb_typeof(${t.column}) = 'string'`;
|
||||
// Container-looking is not enough: '[INFO] fetch complete' matches the
|
||||
// shape probe but is NOT valid JSON — the repair cast would throw and
|
||||
// abort the run. pg_input_is_valid (PG16+, same floor as the IS JSON
|
||||
// predicate updateSourceConfig already relies on) gates on parseability.
|
||||
return t.jsonPayloadOnly
|
||||
? `${base} AND (${t.column} #>> '{}') ~ '^[[:space:]]*[\\[{]' AND pg_input_is_valid(${t.column} #>> '{}', 'jsonb')`
|
||||
: base;
|
||||
}
|
||||
|
||||
export interface RepairResult {
|
||||
engine: string;
|
||||
per_target: Array<{
|
||||
@@ -113,20 +148,40 @@ export async function repairJsonb(opts: RepairOpts = { dryRun: false }): Promise
|
||||
let repaired = 0;
|
||||
|
||||
try {
|
||||
// Skip targets whose table doesn't exist yet — relevant for the
|
||||
// v0_12_2 migration on pre-v0.15 brains (subagent_* tables hadn't
|
||||
// been added yet) and any future schema additions to TARGETS.
|
||||
const existsRows = await sql.unsafe(
|
||||
`SELECT to_regclass($1) IS NOT NULL AS exists`,
|
||||
[t.table],
|
||||
) as Array<{ exists: boolean }>;
|
||||
if (!existsRows[0]?.exists) {
|
||||
progress.tick(1, `${t.table}.${t.column}=skipped(no-table)`);
|
||||
result.per_target.push({ table: t.table, column: t.column, rows_repaired: 0 });
|
||||
continue;
|
||||
}
|
||||
|
||||
if (opts.dryRun) {
|
||||
const rows = await sql.unsafe(
|
||||
`SELECT count(*)::int AS n FROM ${t.table} WHERE jsonb_typeof(${t.column}) = 'string'`,
|
||||
`SELECT count(*)::int AS n FROM ${t.table} WHERE ${damagePredicate(t)}`,
|
||||
);
|
||||
repaired = (rows[0] as unknown as { n: number }).n;
|
||||
} else {
|
||||
const rows = await sql.unsafe(
|
||||
`UPDATE ${t.table}
|
||||
SET ${t.column} = (${t.column} #>> '{}')::jsonb
|
||||
WHERE jsonb_typeof(${t.column}) = 'string'
|
||||
WHERE ${damagePredicate(t)}
|
||||
RETURNING 1`,
|
||||
);
|
||||
repaired = rows.length;
|
||||
}
|
||||
} catch (e) {
|
||||
// One target's failure (unexpected content shape, permission, etc.)
|
||||
// must not abort the remaining targets — earlier repairs are already
|
||||
// committed and the v0_12_2 migration orchestrator JSON-parses our
|
||||
// stdout. Record, report, continue.
|
||||
console.error(`[repair-jsonb] ${t.table}.${t.column} failed: ${(e as Error).message} — continuing with remaining targets`);
|
||||
repaired = 0;
|
||||
} finally {
|
||||
stopHb();
|
||||
}
|
||||
|
||||
@@ -30,6 +30,7 @@ import type { OperationContext, AuthInfo } from '../core/operations.ts';
|
||||
import { GBrainOAuthProvider, validateTokenEndpointAuthMethod } from '../core/oauth-provider.ts';
|
||||
import type { SqlQuery } from '../core/oauth-provider.ts';
|
||||
import { hasScope, ALLOWED_SCOPES_LIST, normalizeScopesInput } from '../core/scope.ts';
|
||||
import { normalizeSourceInput, normalizeFederatedReadInput } from '../core/source-id.ts';
|
||||
import { summarizeMcpParams, dispatchToolCall } from '../mcp/dispatch.ts';
|
||||
import { paramDefToSchema } from '../mcp/tool-defs.ts';
|
||||
import { getBrainHotMemoryMeta } from '../core/facts/meta-hook.ts';
|
||||
@@ -1652,7 +1653,7 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption
|
||||
// and other malformed inputs
|
||||
// normalizeScopesInput handles all four valid shapes (string, string[],
|
||||
// missing, empty) and rejects the rest with a structured 400.
|
||||
const { name, tokenTtl, grantTypes, redirectUris, tokenEndpointAuthMethod } = req.body;
|
||||
const { name, source, federatedRead, tokenTtl, grantTypes, redirectUris, tokenEndpointAuthMethod } = req.body;
|
||||
const rawScopes = (req.body as Record<string, unknown>).scopes ?? (req.body as Record<string, unknown>).scope;
|
||||
if (!name) { res.status(400).json({ error: 'Name required' }); return; }
|
||||
let scopeString: string;
|
||||
@@ -1684,8 +1685,27 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption
|
||||
});
|
||||
return;
|
||||
}
|
||||
// v0.41.x: honor optional `source` (write source_id) and `federatedRead`
|
||||
// (read source set) from the request body, mirroring the CLI's
|
||||
// `--source` / `--federated-read` flags. Omitting both preserves the
|
||||
// historical behavior (source_id='default', federated_read=[source_id]).
|
||||
// Pre-fix this endpoint hardcoded 'default'/undefined, so an admin SPA or
|
||||
// a proxy could never mint a client bound to a non-default brain source
|
||||
// over HTTP — only the CLI could. Validated here for a structured 400.
|
||||
let sourceId: string;
|
||||
let federatedReadIds: string[] | undefined;
|
||||
try {
|
||||
sourceId = normalizeSourceInput(source);
|
||||
federatedReadIds = normalizeFederatedReadInput(federatedRead);
|
||||
} catch (e) {
|
||||
res.status(400).json({
|
||||
error: 'invalid_source',
|
||||
message: e instanceof Error ? e.message : String(e),
|
||||
});
|
||||
return;
|
||||
}
|
||||
const result = await oauthProvider.registerClientManual(
|
||||
name, grants, scopeString, uris, 'default', undefined, validatedAuthMethod,
|
||||
name, grants, scopeString, uris, sourceId, federatedReadIds, validatedAuthMethod,
|
||||
);
|
||||
// Set per-client TTL if specified
|
||||
if (tokenTtl && Number(tokenTtl) > 0) {
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
// AUTO-GENERATED by scripts/generate-flag-registry.ts — do not edit by hand.
|
||||
// Regenerate: bun run build:flag-registry
|
||||
// Freshness + drift pinned by test/cli-flag-validation.test.ts (#2185).
|
||||
//
|
||||
// Per-command legal flags for CLI_ONLY commands, derived from each command's
|
||||
// source (case block + imported modules + one level of relative imports +
|
||||
// scripts/generate-flag-registry.ts EXTRA_FLAGS). Deliberately over-inclusive
|
||||
// (help-text mentions count): accepting an ignored flag is the pre-#2185
|
||||
// status quo; missing a real one breaks working invocations.
|
||||
export const CLI_FLAG_REGISTRY: Record<string, readonly string[]> = {
|
||||
'advisor': ['--aliases', '--all', '--apply', '--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--skills-dir', '--source', '--stale', '--supersessions', '--thin', '--verbose', '--workspace', '--yes'],
|
||||
'agent': ['--aliases', '--all', '--brain', '--detach', '--fanout-manifest', '--flag', '--flags', '--follow', '--help', '--include-null-signature', '--json', '--max-turns', '--model', '--no-extract', '--no-follow', '--note', '--pattern', '--pending', '--reset', '--resolve', '--since', '--source', '--stale', '--subagent-def', '--supersessions', '--thin', '--timeout-ms', '--tools', '--word'],
|
||||
'anomalies': ['--aliases', '--all', '--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--lookback-days', '--mcp-only', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--sigma', '--since', '--source', '--stale', '--supersessions', '--thin', '--timeout'],
|
||||
'apply-migrations': ['--ab', '--all', '--auto-update', '--brain', '--break-lock', '--build-index', '--by-mention', '--compile', '--days', '--dry-run', '--exclusive', '--fast', '--force', '--force-all', '--force-orchestrator', '--force-retry', '--force-schema', '--from-meetings', '--from-pages', '--help', '--history', '--host-dir', '--http', '--json', '--lang', '--list', '--locks', '--markdown', '--max-age', '--migrate-only', '--migration', '--mode', '--multimodal', '--no-autopilot-install', '--no-embedding', '--no-extract', '--non-interactive', '--phase', '--priority', '--refresh-unqualified', '--remediate', '--rollback', '--skip-verify', '--source', '--stale', '--undo-wave', '--use-captured-snapshot', '--with-calibration', '--yes'],
|
||||
'auth': ['--aliases', '--all', '--bound-brain', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--enable-dcr', '--enable-dcr-insecure', '--fast', '--federated-read', '--force', '--from-pages', '--grant-types', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--pattern', '--pending', '--redirect-uri', '--reset', '--resolve', '--scopes', '--source', '--stale', '--supersessions', '--takes-holders', '--thin', '--token', '--token-endpoint-auth-method', '--yes'],
|
||||
'autopilot': ['--aliases', '--all', '--auto-fix', '--batch', '--brain', '--break-lock', '--by-type', '--check', '--dimensions', '--dir', '--drain', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--ff-only', '--fix', '--force', '--force-break-lock', '--force-retry', '--from-pages', '--full', '--help', '--http', '--include-null-signature', '--inject-bootstrap', '--inline', '--input', '--install', '--interval', '--json', '--markdown', '--max-age', '--max-rss', '--max-usd', '--migrate-only', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--no-extract', '--no-inject', '--no-mutate', '--no-worker', '--non-interactive', '--now', '--once', '--output', '--path', '--pattern', '--pending', '--phase', '--refresh-cache', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--source', '--stale', '--status', '--supersessions', '--swap-only', '--target', '--target-score', '--thin', '--timeout', '--to', '--uninstall', '--unsafe-bypass-dream-guard', '--user', '--version', '--yes'],
|
||||
'backfill': ['--aliases', '--all', '--batch-size', '--brain', '--concurrency', '--dry-run', '--fresh', '--help', '--include-null-signature', '--json', '--keep-index', '--list', '--max-errors', '--max-rows', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--resume', '--source', '--stale', '--supersessions', '--thin'],
|
||||
'bench': ['--baseline', '--brain', '--explain', '--force', '--from', '--help', '--json', '--label', '--lang', '--limit', '--markdown', '--multimodal', '--near-symbol', '--restore-only', '--source', '--stale', '--symbol-kind', '--thin', '--threshold-jaccard', '--threshold-latency-multiplier', '--threshold-top1', '--to', '--tool'],
|
||||
'book-mirror': ['--aliases', '--all', '--allow-empty', '--apply', '--asof', '--author', '--auto', '--background', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--brain-wide-max-cost-usd', '--budget-usd-per-day', '--by-mention', '--chapters-dir', '--content', '--context-file', '--date', '--days', '--dry-run', '--entities', '--explain', '--fast', '--federated', '--file', '--follow', '--force', '--from-pages', '--help', '--http', '--image', '--include-null-signature', '--json', '--kind', '--limit', '--max-turns', '--max-usd', '--mode', '--model', '--multimodal', '--no-confirm', '--no-embedding', '--no-extract', '--no-follow', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--save', '--session', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--thin', '--timeout', '--timeout-ms', '--title', '--trusted-extraction', '--url', '--with-db', '--yes'],
|
||||
'brainstorm': ['--aliases', '--all', '--brain', '--chunker-debug', '--code', '--compile', '--fast', '--fix', '--force', '--force-rechunk', '--force-resume', '--from-pages', '--full', '--help', '--http', '--include-null-signature', '--json', '--judge-model', '--lang', '--limit', '--list-runs', '--markdown', '--max-cost', '--max-far-set', '--max-ideas-per-judge-call', '--model', '--no-embed', '--no-embedding', '--no-extract', '--no-save', '--pattern', '--pending', '--reset', '--resolve', '--resume', '--retry-failed', '--retry-judge', '--save', '--source', '--stale', '--strict-budget', '--supersessions', '--thin', '--timeout', '--yes'],
|
||||
'cache': ['--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--no-embedding', '--source', '--yes'],
|
||||
'calibration': ['--ab', '--aliases', '--all', '--allow-empty', '--apply', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--dry-run', '--entities', '--explain', '--fast', '--federated', '--file', '--force', '--from-pages', '--help', '--holder', '--http', '--image', '--include-null-signature', '--json', '--key-prefix', '--kind', '--lang', '--limit', '--markdown', '--max-usd', '--mode', '--multimodal', '--near-symbol', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--phase', '--progress-interval', '--progress-json', '--quiet', '--regenerate', '--repo', '--reset', '--resolve', '--restore-only', '--save', '--scrub-gstack', '--session', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--symbol-kind', '--thin', '--trusted-extraction', '--undo-wave', '--url', '--with-calibration', '--with-db', '--yes'],
|
||||
'call': ['--aliases', '--all', '--all-sources', '--as-context', '--auto-fix', '--background', '--brain', '--by-mention', '--catch-up', '--concurrency', '--confirm-destructive', '--content', '--cost-estimate', '--count', '--days', '--depth', '--dim', '--dir', '--direction', '--enable-dcr', '--enable-dcr-insecure', '--explain', '--fast', '--federated', '--file', '--fix', '--follow', '--force', '--from', '--from-meetings', '--grant-types', '--grep', '--hard-deadline', '--help', '--http', '--image', '--include-expired', '--include-frontmatter', '--include-null-signature', '--infer-dates', '--install', '--interval', '--json', '--key', '--kind', '--lang', '--limit', '--link-source', '--link-type', '--migrate-only', '--missing-path', '--multimodal', '--ner', '--no-embed', '--no-expand', '--no-extract', '--no-federated', '--no-hard-deadline', '--no-retry-connect', '--no-save', '--older-than', '--page', '--param', '--params', '--password', '--path', '--pattern', '--pending', '--pglite', '--port', '--progress-interval', '--progress-json', '--public-url', '--queue', '--quiet', '--reenrich-after', '--refresh-cache', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--restore-only', '--save', '--scopes', '--session', '--sigma', '--since', '--slug', '--slug-prefix', '--source', '--source-id', '--stale', '--status', '--stdin', '--strategy', '--supabase', '--supersessions', '--symbol-kind', '--tag', '--thin', '--timeout', '--to', '--today', '--token', '--token-ttl', '--tools-json', '--type', '--url', '--version', '--watch', '--with-calibration', '--workers', '--yes'],
|
||||
'capture': ['--aliases', '--all', '--allow-empty', '--apply', '--asof', '--auto', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--content', '--date', '--days', '--depth', '--entities', '--fast', '--federated', '--file', '--force', '--from-pages', '--help', '--http', '--image', '--include-null-signature', '--json', '--kind', '--limit', '--max-usd', '--mcp-only', '--mode', '--multimodal', '--no-embedding', '--no-extract', '--no-federated', '--offset', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--repo', '--reset', '--resolve', '--restore-only', '--save', '--scopes', '--session', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--stdin', '--supersessions', '--thin', '--timeout', '--trusted-extraction', '--type', '--url', '--what', '--where', '--who', '--with-db', '--yes'],
|
||||
'check-backlinks': ['--background', '--brain', '--brain-wide-max-cost-usd', '--dir', '--dry-run', '--explain', '--follow', '--help', '--include-frontmatter', '--json', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--source', '--stale', '--timeout', '--type'],
|
||||
'check-resolvable': ['--brain', '--dry-run', '--fix', '--help', '--json', '--skills-dir', '--source', '--strict', '--verbose'],
|
||||
'check-update': ['--all', '--brain', '--check', '--ff-only', '--help', '--json', '--markdown', '--migrate-only', '--non-interactive', '--refresh-cache', '--source', '--swap-only', '--to', '--version', '--yes'],
|
||||
'claw-test': ['--agent', '--brain', '--dir', '--help', '--json', '--keep-tempdir', '--list-agents', '--live', '--local', '--message', '--no-embed', '--no-embedding', '--path', '--pglite', '--progress-json', '--prompt-file', '--run-id', '--scenario', '--source', '--transcripts'],
|
||||
'code-callees': ['--aliases', '--all', '--all-sources', '--brain', '--chunker-debug', '--clone-dir', '--confirm-destructive', '--federated', '--force', '--help', '--include-null-signature', '--json', '--limit', '--no-extract', '--no-federated', '--no-json', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--thin', '--url', '--url-managed', '--yes'],
|
||||
'code-callers': ['--aliases', '--all', '--all-sources', '--brain', '--chunker-debug', '--clone-dir', '--confirm-destructive', '--federated', '--force', '--help', '--include-null-signature', '--json', '--limit', '--no-extract', '--no-federated', '--no-json', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--thin', '--url', '--url-managed', '--yes'],
|
||||
'code-def': ['--aliases', '--all', '--brain', '--chunker-debug', '--help', '--include-null-signature', '--json', '--lang', '--limit', '--no-extract', '--no-json', '--pattern', '--pending', '--pretty', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--yes'],
|
||||
'code-refs': ['--aliases', '--all', '--brain', '--chunker-debug', '--help', '--include-null-signature', '--json', '--lang', '--limit', '--no-extract', '--no-json', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--yes'],
|
||||
'config': ['--aliases', '--all', '--brain', '--column', '--coverage-override', '--embedding-dimensions', '--embedding-model', '--fast', '--follow', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--markdown', '--model', '--multimodal', '--no-embedding', '--no-extract', '--pattern', '--pending', '--pglite', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--yes'],
|
||||
'connect': ['--agent', '--bearer-token-env-var', '--bind', '--brain', '--client-id', '--client-secret', '--force', '--grant-types', '--help', '--http', '--install', '--json', '--name', '--oauth', '--public-url', '--register', '--scopes', '--show-token', '--source', '--timeout-ms', '--token', '--token-endpoint-auth-method', '--url', '--version', '--yes'],
|
||||
'conversation-parser': ['--aliases', '--all', '--brain', '--help', '--include-null-signature', '--json', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin'],
|
||||
'doctor': ['--ab', '--abi', '--aliases', '--all', '--allow-shell-jobs', '--auto', '--auto-fix', '--auto-update', '--background', '--batch', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--build-index', '--by-mention', '--by-type', '--check', '--column', '--compile', '--concurrency', '--confidence', '--content-audit', '--count', '--days', '--detach', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--exclusive', '--explain', '--fast', '--fix', '--follow', '--force', '--force-break-lock', '--force-retry', '--force-schema', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--grant-types', '--health-interval', '--help', '--history', '--http', '--include-flagged', '--include-frontmatter', '--include-null-signature', '--include-pseudo', '--index-audit', '--input', '--json', '--lang', '--limit', '--locks', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-crashes', '--max-jobs', '--max-rss', '--max-usd', '--mcp-only', '--migrate-only', '--model', '--multimodal', '--name-only', '--name-status', '--near-symbol', '--nice', '--no', '--no-embed', '--no-embedding', '--no-extract', '--no-mutate', '--oauth-client-secret', '--older-than', '--once', '--overwrite', '--parallel', '--params', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--priority', '--progress-interval', '--progress-json', '--query', '--queue', '--quiet', '--rebuild-rollup', '--refresh', '--refresh-unqualified', '--regenerate', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--restore-only', '--resume', '--review-lower', '--rollback', '--scope', '--scopes', '--since', '--skills-dir', '--skip-bare-tweet', '--skip-failed', '--skip-urls', '--skip-verify', '--slugs', '--source', '--source-id', '--stale', '--status', '--strategy', '--strict', '--supabase', '--supersessions', '--symbol-kind', '--target', '--target-score', '--thin', '--timeout', '--to', '--top-k', '--type', '--undo-wave', '--unsafe-bypass-dream-guard', '--untracked-files', '--url', '--use-captured-snapshot', '--verbose', '--version', '--window', '--with-calibration', '--workers', '--yes'],
|
||||
'dream': ['--against', '--aliases', '--all', '--anchor', '--asof', '--background', '--batch', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--budget-usd', '--by-type', '--by-type-floor', '--code', '--compile', '--concurrent', '--ctx-size', '--cycles', '--date', '--dimensions', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--embeddings', '--expansion', '--explain', '--fast', '--federated', '--fix', '--fixtures', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--force-retry', '--from', '--from-db', '--from-pages', '--help', '--http', '--include-null-signature', '--input', '--install', '--json', '--judge-model', '--keyword-only', '--lang', '--limit', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--max-tokens', '--max-usd', '--mcp-only', '--min-recall', '--mode', '--model', '--models', '--multimodal', '--name', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-llm', '--no-mutate', '--no-trajectory', '--once', '--output', '--path', '--pattern', '--pending', '--pglite', '--phase', '--priority', '--progress-interval', '--progress-json', '--pull', '--quiet', '--receipt-dir', '--remediate', '--repo', '--reranking', '--reset', '--resolve', '--restore-only', '--resume-from', '--retrieval-only', '--rounds', '--rubric-version', '--save', '--seed', '--skip-replay', '--slot-a-model', '--slot-b-model', '--slot-c-model', '--slug', '--slug-prefix', '--source', '--source-id', '--stale', '--supabase', '--supersessions', '--symbol-kind', '--take', '--task', '--thin', '--threshold', '--timeout', '--to', '--top-k', '--undo', '--unsafe-bypass-dream-guard', '--version', '--window', '--yes'],
|
||||
'edges-backfill': ['--aliases', '--all', '--all-sources', '--brain', '--concurrency', '--federated', '--help', '--include-null-signature', '--json', '--max-age', '--max-chunks', '--max-cost-usd', '--no-extract', '--no-federated', '--older-than', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--thin', '--timeout', '--workers'],
|
||||
'embed': ['--aliases', '--all', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--catch-up', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--fix', '--follow', '--force', '--force-break-lock', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--lang', '--markdown', '--max-age', '--max-cost-usd', '--model', '--multimodal', '--name', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--pace', '--pace-max-concurrency', '--parallel', '--path', '--pattern', '--pending', '--pglite', '--prefix', '--priority', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--restore-only', '--serial', '--slugs', '--source', '--stale', '--supabase', '--supersessions', '--symbol-kind', '--thin', '--timeout', '--version'],
|
||||
'enrich': ['--aliases', '--all', '--all-sources', '--allow-empty', '--apply', '--asof', '--auto', '--background', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--budget-usd-per-day', '--by-mention', '--clone-dir', '--code', '--concurrency', '--confirm-destructive', '--content', '--date', '--days', '--dry-run', '--embedding-dimensions', '--embedding-model', '--entities', '--explain', '--fast', '--federated', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--from-pages', '--help', '--http', '--image', '--include-null-signature', '--json', '--judge-model', '--kind', '--lang', '--limit', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--max-usd', '--min-context', '--mode', '--model', '--multimodal', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--offset', '--older-than', '--order', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--reenrich-after', '--remediate', '--reset', '--resolve', '--restore-only', '--resume', '--save', '--session', '--since', '--slug', '--slugs', '--source', '--source-id', '--stale', '--stats', '--supersessions', '--symbol-kind', '--thin', '--thin-threshold', '--timeout', '--trusted-extraction', '--types', '--url', '--url-managed', '--version', '--with-db', '--workers', '--yes'],
|
||||
'eval': ['--ab-relational', '--against', '--aliases', '--all', '--background', '--baseline', '--batch', '--brain', '--brain-wide-max-cost-usd', '--budget-usd', '--budget-usd-answer', '--budget-usd-retrieval', '--compare', '--compare-limit', '--concurrent', '--config-a', '--config-b', '--corpus', '--cycles', '--days', '--dedup-cosine', '--dedup-max-per-page', '--dedup-type-ratio', '--dimensions', '--distance-min', '--embedding-dimensions', '--embedding-model', '--expand', '--explain', '--fast', '--fixtures', '--follow', '--force', '--from-capture', '--from-db', '--from-pages', '--grounding-min', '--help', '--http', '--include-null-signature', '--input', '--json', '--judge', '--k', '--limit', '--max-pair-chars', '--max-tokens', '--max-usd', '--md', '--metric', '--min-recall', '--mode', '--model', '--models', '--modes', '--multimodal', '--name', '--no', '--no-cache', '--no-embed', '--no-embedding', '--no-expand', '--no-extract', '--no-llm', '--older-than', '--output', '--output-dir', '--parallel', '--pattern', '--pending', '--progress-interval', '--progress-json', '--qrels', '--queries-file', '--query', '--questions', '--quiet', '--receipt-dir', '--refresh-cache', '--remediate', '--reset', '--resolve', '--rrf-k', '--rubric-version', '--runs', '--sampling', '--save', '--seed', '--severity', '--short', '--show-toplevel', '--since', '--skip-replay', '--slot-a-model', '--slot-b-model', '--slot-c-model', '--slug', '--slug-prefix', '--source', '--stale', '--strategy', '--strict', '--suite', '--suites', '--supersessions', '--task', '--thin', '--threshold', '--threshold-expected-top1', '--threshold-first-relevant-hit', '--threshold-jaccard', '--threshold-latency-multiplier', '--threshold-recall-at-k', '--threshold-top1', '--timeout', '--tool', '--top-k', '--top-regressions', '--until', '--usefulness-min', '--verbose', '--version', '--with-code-intel', '--yes'],
|
||||
'export': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dir', '--explain', '--federated', '--fix', '--follow', '--help', '--include-null-signature', '--json', '--lang', '--markdown', '--multimodal', '--near-symbol', '--no-extract', '--no-federated', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--repo', '--reset', '--resolve', '--restore-only', '--slug-prefix', '--source', '--stale', '--supersessions', '--symbol-kind', '--thin', '--timeout', '--type'],
|
||||
'extract': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--by-mention', '--catch-up', '--code', '--concurrency', '--dir', '--dry-run', '--explain', '--federated', '--follow', '--from-meetings', '--help', '--include-frontmatter', '--include-null-signature', '--infer-dates', '--json', '--kind', '--lang', '--markdown', '--max-age', '--max-cost-usd', '--multimodal', '--name-status', '--near-symbol', '--ner', '--no-extract', '--no-federated', '--older-than', '--pack', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--repo', '--reset', '--resolve', '--restore-only', '--run-id', '--since', '--slug', '--source', '--source-id', '--stale', '--strategy', '--supersessions', '--symbol-kind', '--thin', '--timeout', '--type', '--verbose', '--workers', '--yes'],
|
||||
'extract-conversation-facts': ['--aliases', '--all', '--all-sources', '--background', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--by-mention', '--clone-dir', '--code', '--concurrency', '--confirm-destructive', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fix', '--follow', '--force', '--force-break-lock', '--help', '--include-null-signature', '--json', '--judge-model', '--lang', '--limit', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--model', '--multimodal', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--older-than', '--override-disabled', '--path', '--pattern', '--pending', '--pglite', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--restore-only', '--segment-limit', '--session', '--since', '--sleep', '--slug', '--source', '--source-id', '--stale', '--supabase', '--supersessions', '--symbol-kind', '--thin', '--timeout', '--types', '--url', '--url-managed', '--version', '--workers', '--yes'],
|
||||
'features': ['--aliases', '--all', '--auto-fix', '--background', '--batch-size', '--brain', '--by-mention', '--catch-up', '--concurrency', '--dir', '--explain', '--from-meetings', '--help', '--include-frontmatter', '--include-null-signature', '--infer-dates', '--json', '--kind', '--ner', '--no-extract', '--pace', '--pace-max-concurrency', '--pack', '--path', '--pattern', '--pending', '--priority', '--progress-json', '--quiet', '--repo', '--reset', '--resolve', '--run-id', '--since', '--slugs', '--source', '--source-id', '--stale', '--supersessions', '--thin', '--type', '--verbose', '--workers'],
|
||||
'files': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--explain', '--fast', '--follow', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--no-pointer', '--page', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--retry-failed', '--source', '--stale', '--supersessions', '--thin', '--timeout', '--type', '--yes'],
|
||||
'forget': ['--aliases', '--all', '--as-context', '--brain', '--fast', '--federated', '--force', '--from-pages', '--grep', '--help', '--http', '--include-expired', '--include-null-signature', '--json', '--limit', '--mcp-only', '--no-embedding', '--no-extract', '--no-federated', '--path', '--pattern', '--pending', '--reason', '--repo', '--reset', '--resolve', '--restore-only', '--rollup', '--session', '--session-id', '--since', '--since-last-run', '--source', '--stale', '--supersessions', '--thin', '--timeout', '--today', '--watch'],
|
||||
'founder': ['--aliases', '--all', '--brain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--mcp-only', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--since', '--source', '--stale', '--supersessions', '--thin', '--timeout', '--until'],
|
||||
'friction': ['--agent', '--brain', '--help', '--hint', '--json', '--kind', '--message', '--no-redact', '--phase', '--redact', '--run-id', '--severity', '--source', '--transcript-path', '--transcripts'],
|
||||
'frontmatter': ['--aliases', '--all', '--allow-catch-all', '--brain', '--cached', '--diff-filter', '--dry-run', '--exclude-standard', '--fast', '--fix', '--force', '--from-pages', '--get', '--help', '--http', '--include-catch-all', '--include-null-signature', '--json', '--name-only', '--name-status', '--no-embedding', '--no-extract', '--no-verify', '--others', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--strategy', '--supersessions', '--thin', '--timeout', '--uninstall', '--write-back'],
|
||||
'graph-query': ['--aliases', '--all', '--brain', '--depth', '--direction', '--explain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-foreign', '--include-null-signature', '--json', '--lang', '--markdown', '--mcp-only', '--multimodal', '--near-symbol', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--symbol-kind', '--thin', '--timeout', '--type'],
|
||||
'import': ['--aliases', '--all', '--asof', '--background', '--brain', '--brain-wide-max-cost-usd', '--by-mention', '--cached', '--code', '--compile', '--concurrency', '--embedding-dimensions', '--embedding-model', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--fix', '--follow', '--force', '--force-rechunk', '--fresh', '--from-pages', '--full', '--help', '--http', '--include-gitignored', '--include-null-signature', '--json', '--lang', '--markdown', '--max-age', '--multimodal', '--name-status', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--older-than', '--others', '--path', '--pattern', '--pending', '--pglite', '--priority', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--since', '--skip-failed', '--source', '--source-id', '--stale', '--strategy', '--supabase', '--supersessions', '--thin', '--timeout', '--url', '--workers'],
|
||||
'init': ['--all', '--brain', '--chat-model', '--check', '--ctx-size', '--embedding-dimensions', '--embedding-model', '--expansion-model', '--fast', '--force', '--from-pages', '--grant-types', '--help', '--http', '--issuer-url', '--json', '--judge-model', '--key', '--mcp-only', '--mcp-url', '--migrate-only', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--non-interactive', '--oauth-client-id', '--oauth-client-secret', '--path', '--pglite', '--schema-pack', '--scopes', '--skip-embed-check', '--source', '--stale', '--supabase', '--target', '--to', '--touchpoint', '--url', '--version'],
|
||||
'integrations': ['--auto', '--brain', '--dry-run', '--embeddings', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--no-embedding', '--overwrite', '--refresh', '--reranking', '--source', '--target'],
|
||||
'integrity': ['--aliases', '--all', '--auto', '--backend', '--background', '--brain', '--brain-wide-max-cost-usd', '--check', '--confidence', '--cost', '--dry-run', '--explain', '--fast', '--follow', '--force', '--fresh', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--limit', '--no-embedding', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--review-lower', '--skip-bare-tweet', '--skip-urls', '--source', '--stale', '--supabase', '--supersessions', '--thin', '--timeout', '--type', '--url'],
|
||||
'jobs': ['--abbrev-ref', '--aliases', '--all', '--allow-empty', '--allow-protected', '--allow-shell-jobs', '--apply', '--asof', '--auto', '--auto-with-prompt', '--background', '--backoff-delay', '--backoff-jitter', '--backoff-type', '--batch-size', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--break-lock', '--budget-usd', '--budget-usd-per-day', '--by-mention', '--cached', '--catch-up', '--check', '--cli-path', '--cluster', '--cluster-errors', '--code', '--concurrency', '--confidence', '--confirm-destructive', '--content', '--date', '--days', '--delay', '--detach', '--diff-filter', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--empty', '--entities', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--federated-read', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--force-retry', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--hard-deadline', '--health-interval', '--held-out', '--help', '--http', '--idempotency-key', '--image', '--include-frontmatter', '--include-gitignored', '--include-null-signature', '--infer-dates', '--inject-bootstrap', '--inline', '--input', '--install', '--interval', '--is-ancestor', '--json', '--kind', '--lang', '--limit', '--lock', '--markdown', '--max-age', '--max-attempts', '--max-cost-usd', '--max-crashes', '--max-rss', '--max-runtime-min', '--max-sources', '--max-stalled', '--max-usd', '--max-waiting', '--mcp-only', '--min-context', '--missing-path', '--mode', '--model', '--multimodal', '--name-only', '--name-status', '--near-symbol', '--ner', '--nice', '--no', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-federate', '--no-gpg-sign', '--no-hard-deadline', '--no-inject', '--no-mutate', '--no-pull', '--no-renames', '--no-schema-pack', '--no-verify', '--no-worker', '--now', '--offset', '--older-than', '--once', '--order', '--others', '--override-disabled', '--pace', '--pace-max-concurrency', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--phase', '--pid-file', '--priority', '--progress-interval', '--progress-json', '--queue', '--quiet', '--redact-secrets', '--reenrich-after', '--refresh-ms', '--remediate', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--resume', '--retry-failed', '--review-lower', '--run-id', '--save', '--segment-limit', '--serial', '--session', '--short', '--show-toplevel', '--sigkill-rescue', '--since', '--skip-bare-tweet', '--skip-failed', '--skip-urls', '--sleep', '--slug', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--stats', '--status', '--strategy', '--supersessions', '--swap-only', '--symbol-kind', '--target', '--thin', '--thin-threshold', '--timeout', '--timeout-ms', '--trusted-extraction', '--type', '--types', '--uninstall', '--unsafe-bypass-dream-guard', '--url', '--user', '--verbose', '--version', '--watch', '--wedge-rescue', '--with-db', '--workers', '--yes'],
|
||||
'lint': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--exclude', '--explain', '--fast', '--fix', '--follow', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--no-embedding', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--timeout'],
|
||||
'lsd': ['--brain', '--force-resume', '--help', '--json', '--judge-model', '--limit', '--list-runs', '--max-cost', '--max-far-set', '--max-ideas-per-judge-call', '--no-save', '--resume', '--retry-judge', '--save', '--source', '--strict-budget', '--yes'],
|
||||
'maintain': ['--aliases', '--all', '--background', '--brain', '--break-lock', '--by-mention', '--catch-up', '--column', '--concurrency', '--content-audit', '--count', '--detach', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--fix', '--force', '--force-retry', '--force-schema', '--from-meetings', '--full', '--help', '--include-flagged', '--include-frontmatter', '--include-null-signature', '--index-audit', '--infer-dates', '--input', '--json', '--kind', '--lang', '--locks', '--markdown', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-usd', '--migrate-only', '--multimodal', '--near-symbol', '--ner', '--nice', '--no-extract', '--no-mutate', '--older-than', '--once', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--progress-json', '--query', '--queue', '--quiet', '--rebuild-rollup', '--regenerate', '--remediate', '--remediation-plan', '--reset', '--resolve', '--restore-only', '--resume', '--run-id', '--safe', '--scope', '--since', '--skills-dir', '--skip-failed', '--slugs', '--source', '--source-id', '--stale', '--status', '--supabase', '--supersessions', '--symbol-kind', '--target', '--target-score', '--thin', '--to', '--top-k', '--type', '--unsafe-bypass-dream-guard', '--url', '--verbose', '--window', '--workers', '--yes'],
|
||||
'migrate': ['--ab', '--aliases', '--all', '--auto-update', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--build-index', '--by-mention', '--catch-up', '--compile', '--days', '--dim', '--dry-run', '--embedding-dimensions', '--embedding-model', '--embeddings', '--exclusive', '--explain', '--fast', '--follow', '--force', '--force-retry', '--force-schema', '--from-meetings', '--from-pages', '--help', '--history', '--http', '--ignore-env-override', '--ignore-missing-key', '--include-null-signature', '--json', '--lang', '--locks', '--markdown', '--max-age', '--model', '--multimodal', '--name', '--near-symbol', '--no', '--no-embed', '--no-embedding', '--no-extract', '--non-interactive', '--pace', '--pace-max-concurrency', '--parallel', '--path', '--pattern', '--pending', '--phase', '--prefix', '--priority', '--progress-interval', '--progress-json', '--quiet', '--refresh-unqualified', '--remediate', '--reranking', '--reset', '--resolve', '--restore-only', '--resume', '--rollback', '--skip-verify', '--slugs', '--source', '--stale', '--supersessions', '--symbol-kind', '--thin', '--timeout', '--to', '--undo', '--undo-wave', '--url', '--use-captured-snapshot', '--version', '--with-calibration', '--yes'],
|
||||
'models': ['--aliases', '--all', '--brain', '--embedding-dimensions', '--embedding-model', '--embeddings', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--judge-model', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reranking', '--reset', '--resolve', '--skip', '--source', '--stale', '--supersessions', '--thin', '--undo', '--version'],
|
||||
'mounts': ['--alias', '--brain', '--cache', '--database-path', '--database-url', '--db-path', '--db-url', '--engine', '--explain', '--help', '--id', '--json', '--lang', '--lock', '--markdown', '--mcp-url', '--multimodal', '--near-symbol', '--path', '--restore-only', '--skills-dir', '--source', '--stale', '--symbol-kind', '--thin', '--verbose'],
|
||||
'notability-eval': ['--aliases', '--all', '--brain', '--embedding-dimensions', '--embedding-model', '--help', '--in', '--include-null-signature', '--json', '--model', '--multimodal', '--no', '--no-embed', '--no-extract', '--out', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--skip-llm', '--source', '--stale', '--supersessions', '--target-high', '--target-low', '--target-medium', '--thin', '--version'],
|
||||
'onboard': ['--aliases', '--all', '--allow-empty', '--allow-protected', '--apply', '--asof', '--auto', '--auto-with-prompt', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--budget-usd-per-day', '--by-mention', '--check', '--content', '--date', '--days', '--entities', '--explain', '--federated', '--file', '--from-pages', '--help', '--history', '--http', '--image', '--include-null-signature', '--json', '--kind', '--limit', '--max-usd', '--mode', '--multimodal', '--no-extract', '--offset', '--params', '--path', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--remediation-plan', '--reset', '--resolve', '--resume', '--save', '--session', '--since', '--slug', '--slugs', '--source', '--stale', '--stats', '--supersessions', '--target-score', '--thin', '--trusted-extraction', '--url', '--with-db', '--yes'],
|
||||
'orphans': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--count', '--explain', '--follow', '--help', '--include-null-signature', '--include-pseudo', '--json', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--timeout'],
|
||||
'pages': ['--aliases', '--all', '--brain', '--dry-run', '--help', '--include-null-signature', '--json', '--no-extract', '--older-than', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin'],
|
||||
'pglite-repair': ['--brain', '--break-lock', '--dry-rnu', '--dry-run', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--no-embedding', '--path', '--quiet', '--source', '--yes'],
|
||||
'post-upgrade': ['--all', '--apply-clean-hunks', '--brain', '--check', '--code', '--compile', '--concurrency', '--embedding-dimensions', '--embedding-model', '--fast', '--ff-only', '--force', '--force-all', '--force-orchestrator', '--force-retry', '--force-schema', '--format', '--from-pages', '--help', '--host-dir', '--http', '--inject-bootstrap', '--inline', '--install', '--interval', '--json', '--limit', '--list', '--markdown', '--max-rss', '--migrate-only', '--migration', '--mode', '--model', '--multimodal', '--name-only', '--no', '--no-autopilot-install', '--no-embed', '--no-embedding', '--no-inject', '--no-worker', '--non-interactive', '--now', '--quiet', '--repo', '--since', '--skills-dir', '--skip-verify', '--source', '--stale', '--status', '--swap-only', '--target', '--to', '--uninstall', '--user', '--verbose', '--verify', '--version', '--workers', '--yes'],
|
||||
'providers': ['--brain', '--ctx-size', '--embedding-dimensions', '--embedding-model', '--embeddings', '--fast', '--force', '--from-pages', '--help', '--http', '--json', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--reranking', '--source', '--touchpoint', '--version'],
|
||||
'publish': ['--accent', '--bg', '--border', '--brain', '--card-bg', '--code-bg', '--error', '--fg', '--help', '--json', '--link', '--muted', '--out', '--password', '--source', '--title'],
|
||||
'quarantine': ['--aliases', '--all', '--apply', '--brain', '--code', '--compile', '--explain', '--fast', '--fix', '--force', '--force-rechunk', '--from-pages', '--help', '--http', '--include-flagged', '--include-null-signature', '--json', '--lang', '--limit', '--markdown', '--multimodal', '--near-symbol', '--no-embed', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--symbol-kind', '--thin'],
|
||||
'recall': ['--aliases', '--all', '--as-context', '--brain', '--fast', '--federated', '--force', '--from-pages', '--grep', '--help', '--http', '--include-expired', '--include-null-signature', '--json', '--limit', '--mcp-only', '--no-embedding', '--no-extract', '--no-federated', '--path', '--pattern', '--pending', '--reason', '--repo', '--reset', '--resolve', '--restore-only', '--rollup', '--session', '--session-id', '--since', '--since-last-run', '--source', '--stale', '--supersessions', '--thin', '--timeout', '--today', '--watch'],
|
||||
'reconcile-links': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--explain', '--follow', '--help', '--include-frontmatter', '--include-null-signature', '--json', '--name-status', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--source', '--stale', '--strategy', '--supersessions', '--thin', '--timeout', '--type'],
|
||||
'reindex': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--break-lock', '--code', '--compile', '--concurrency', '--cost-estimate', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fast', '--fix', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--lang', '--limit', '--markdown', '--max-age', '--max-cost-usd', '--model', '--multimodal', '--no', '--no-embed', '--no-embedding', '--no-extract', '--older-than', '--path', '--pattern', '--pending', '--pglite', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--repo', '--reset', '--resolve', '--source', '--stale', '--supabase', '--supersessions', '--thin', '--timeout', '--version', '--workers', '--yes'],
|
||||
'reindex-code': ['--abi', '--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--chunker-debug', '--code', '--compile', '--concurrency', '--dry-run', '--embedding-dimensions', '--embedding-model', '--explain', '--fix', '--follow', '--force', '--force-rechunk', '--help', '--include-null-signature', '--json', '--judge-model', '--lang', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-runtime', '--model', '--multimodal', '--no', '--no-embed', '--no-extract', '--older-than', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--serial', '--source', '--stale', '--supersessions', '--thin', '--timeout', '--version', '--workers', '--yes'],
|
||||
'reindex-frontmatter': ['--aliases', '--all', '--brain', '--concurrency', '--dry-run', '--force', '--help', '--include-null-signature', '--json', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--slug-prefix', '--source', '--stale', '--supersessions', '--thin', '--workers', '--yes'],
|
||||
'reindex-search-vector': ['--aliases', '--all', '--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--explain', '--follow', '--help', '--include-null-signature', '--json', '--no-extract', '--pattern', '--pending', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--timeout', '--yes'],
|
||||
'reinit-pglite': ['--abbrev-ref', '--all', '--allow-empty', '--brain', '--break-lock', '--chat-model', '--concurrency', '--confirm-destructive', '--diff-filter', '--embedding-dimensions', '--embedding-model', '--empty', '--exclude', '--exclude-standard', '--expansion-model', '--fast', '--force', '--force-break-lock', '--format', '--from-pages', '--full', '--grant-types', '--hard-deadline', '--help', '--http', '--include-gitignored', '--interval', '--is-ancestor', '--issuer-url', '--json', '--key', '--lock', '--max-age', '--max-sources', '--mcp-only', '--mcp-url', '--migrate-only', '--missing-path', '--model', '--name-only', '--name-status', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-gpg-sign', '--no-hard-deadline', '--no-pull', '--no-renames', '--no-schema-pack', '--no-sync', '--no-verify', '--non-interactive', '--oauth-client-id', '--oauth-client-secret', '--parallel', '--path', '--pglite', '--priority', '--quiet', '--repo', '--retry-failed', '--schema-pack', '--scopes', '--serial', '--short', '--show-toplevel', '--skip-embed-check', '--skip-failed', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--strategy', '--supabase', '--target', '--timeout', '--to', '--url', '--version', '--watch', '--workers', '--yes'],
|
||||
'remote': ['--all', '--background', '--brain', '--break-lock', '--by-mention', '--column', '--concurrency', '--content-audit', '--count', '--detach', '--dim', '--dir', '--drain', '--embedding-dimensions', '--embedding-model', '--fast', '--fix', '--force', '--force-retry', '--force-schema', '--from-pages', '--full', '--help', '--http', '--include-flagged', '--index-audit', '--json', '--locks', '--markdown', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-usd', '--mcp-only', '--migrate-only', '--multimodal', '--nice', '--no-embedding', '--older-than', '--parallel', '--params', '--pglite', '--phase', '--pid-file', '--progress-json', '--query', '--queue', '--rebuild-rollup', '--regenerate', '--remediate', '--remediation-plan', '--reset', '--resume', '--scope', '--scopes', '--since', '--skills-dir', '--skip-failed', '--slugs', '--source', '--source-id', '--stale', '--status', '--supabase', '--target', '--target-score', '--timeout', '--to', '--top-k', '--url', '--window', '--workers', '--yes'],
|
||||
'repair-jsonb': ['--background', '--brain', '--brain-wide-max-cost-usd', '--dry-run', '--explain', '--fast', '--follow', '--force', '--from-pages', '--help', '--http', '--json', '--lang', '--markdown', '--multimodal', '--near-symbol', '--no-embedding', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--restore-only', '--source', '--stale', '--supabase', '--symbol-kind', '--thin', '--timeout', '--url'],
|
||||
'report': ['--brain', '--content', '--dir', '--help', '--json', '--source', '--title', '--type'],
|
||||
'repos': ['--abbrev-ref', '--abort', '--aliases', '--all', '--all-sources', '--brain', '--branch', '--break-lock', '--cached', '--clone-dir', '--compile', '--confirm-destructive', '--detect', '--dry-run', '--explain', '--fast', '--federated', '--file', '--fix', '--force', '--force-break-lock', '--format', '--from-pages', '--full', '--get', '--git-dir', '--github-repo', '--help', '--http', '--include-null-signature', '--include-warns', '--json', '--keep-storage', '--lang', '--local', '--markdown', '--max-age', '--max-cost-usd', '--multimodal', '--name', '--name-only', '--name-status', '--near-symbol', '--no-cron', '--no-embedding', '--no-extract', '--no-federate', '--no-federated', '--no-harden', '--no-verify', '--params', '--pat-file', '--path', '--pattern', '--pending', '--porcelain', '--push-only', '--quiet', '--rebase', '--repo', '--reset', '--resolve', '--restore-only', '--secret', '--set', '--short', '--source', '--source-id', '--stale', '--status', '--strategy', '--supersessions', '--symbol-kind', '--thin', '--unset-all', '--url', '--url-managed', '--yes'],
|
||||
'resolvers': ['--auto', '--backend', '--brain', '--cost', '--help', '--json', '--source'],
|
||||
'retrieval-upgrade': ['--aliases', '--all', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--catch-up', '--dim', '--dry-run', '--embedding-dimensions', '--embedding-model', '--embeddings', '--explain', '--fast', '--follow', '--force', '--from-pages', '--help', '--http', '--ignore-env-override', '--ignore-missing-key', '--include-null-signature', '--json', '--markdown', '--model', '--multimodal', '--name', '--no', '--no-embed', '--no-embedding', '--no-extract', '--non-interactive', '--pace', '--pace-max-concurrency', '--parallel', '--pattern', '--pending', '--prefix', '--priority', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--reranking', '--reset', '--resolve', '--resume', '--slugs', '--source', '--stale', '--supersessions', '--thin', '--timeout', '--to', '--undo', '--version', '--yes'],
|
||||
'routing-eval': ['--brain', '--fix', '--help', '--json', '--llm', '--skills-dir', '--source', '--strict', '--verbose'],
|
||||
'salience': ['--aliases', '--all', '--brain', '--days', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--kind', '--limit', '--mcp-only', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--slug-prefix', '--source', '--stale', '--supersessions', '--thin', '--timeout'],
|
||||
'schema': ['--alias', '--aliases', '--all', '--apply', '--as-filing-rules', '--brain', '--dims', '--expert', '--expert-routing', '--extractable', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--inverse', '--json', '--kind', '--no-embedding', '--no-extract', '--pack', '--page-type', '--pattern', '--pending', '--prefix', '--primitive', '--reset', '--resolve', '--schema-pack', '--since', '--source', '--source-id', '--stale', '--supersessions', '--target-type', '--thin', '--to', '--with-db'],
|
||||
'self-upgrade': ['--all', '--brain', '--check', '--check-only', '--ff-only', '--force', '--help', '--json', '--markdown', '--migrate-only', '--non-interactive', '--refresh-cache', '--source', '--swap-only', '--to', '--version', '--yes'],
|
||||
'serve': ['--aliases', '--all', '--bind', '--bound-slug-prefixes', '--brain', '--enable-dcr', '--enable-dcr-insecure', '--federated-read', '--help', '--http', '--include-null-signature', '--json', '--log-full-params', '--no-extract', '--pattern', '--pending', '--port', '--print-admin-token', '--public-url', '--reset', '--resolve', '--source', '--stale', '--stdio-idle-timeout', '--supersessions', '--suppress', '--suppress-bootstrap-token', '--thin', '--token-ttl', '--yes'],
|
||||
'skillify': ['--brain', '--description', '--dry-run', '--force', '--help', '--json', '--mutating', '--recent', '--skills-dir', '--source', '--strict', '--triggers', '--verbose', '--writes-pages', '--writes-to'],
|
||||
'skillopt': ['--aliases', '--all', '--allow-mutate-bundled', '--background', '--batch-size', '--benchmark', '--bootstrap-from-routing', '--bootstrap-from-skill', '--bootstrap-reviewed', '--bootstrap-tasks', '--brain', '--brain-wide-max-cost-usd', '--chunker-debug', '--dry-run', '--epochs', '--follow', '--force', '--held-out', '--help', '--include-null-signature', '--json', '--judge-model', '--lr', '--lr-schedule', '--max-cost-usd', '--max-runtime-min', '--model', '--no-extract', '--no-mutate', '--optimizer-model', '--patch', '--pattern', '--pending', '--reset', '--resolve', '--resume', '--rewrite', '--skills-dir', '--source', '--split', '--stale', '--supersessions', '--target-model', '--target-models', '--thin', '--verbose', '--yes'],
|
||||
'skillpack': ['--all', '--apply-clean-hunks', '--author', '--brain', '--dry-run', '--exit-code', '--fast', '--fix', '--force', '--force-unlock', '--format', '--from', '--from-pages', '--frontmatter', '--full', '--help', '--homepage', '--http', '--json', '--license', '--list', '--minimal', '--name-only', '--no-cache', '--no-embedding', '--no-lint', '--note', '--out', '--overwrite-local', '--push', '--quick', '--quiet', '--refresh', '--repo', '--schema-pack', '--short', '--since', '--skills-dir', '--skip-doctor', '--source', '--strict', '--target', '--tier', '--trust', '--url', '--verbose', '--verify', '--workspace', '--yes'],
|
||||
'skillpack-check': ['--background', '--brain', '--brain-wide-max-cost-usd', '--explain', '--fast', '--follow', '--help', '--json', '--list', '--progress-interval', '--progress-json', '--quiet', '--remediate', '--source', '--stale', '--strict', '--timeout', '--yes'],
|
||||
'smoke-test': ['--brain', '--help', '--json', '--source'],
|
||||
'sources': ['--abbrev-ref', '--abort', '--aliases', '--all', '--all-sources', '--brain', '--branch', '--break-lock', '--cached', '--clone-dir', '--compile', '--confirm-destructive', '--detect', '--dry-run', '--explain', '--fast', '--federated', '--file', '--fix', '--force', '--force-break-lock', '--format', '--from-pages', '--full', '--get', '--git-dir', '--github-repo', '--help', '--http', '--include-null-signature', '--include-warns', '--json', '--keep-storage', '--lang', '--local', '--markdown', '--max-age', '--max-cost-usd', '--multimodal', '--name', '--name-only', '--name-status', '--near-symbol', '--no-cron', '--no-embedding', '--no-extract', '--no-federate', '--no-federated', '--no-harden', '--no-verify', '--params', '--pat-file', '--path', '--pattern', '--pending', '--porcelain', '--push-only', '--quiet', '--rebase', '--repo', '--reset', '--resolve', '--restore-only', '--secret', '--set', '--short', '--source', '--source-id', '--stale', '--status', '--strategy', '--supersessions', '--symbol-kind', '--thin', '--unset-all', '--url', '--url-managed', '--yes'],
|
||||
'status': ['--abbrev-ref', '--abi', '--abort', '--aliases', '--all', '--all-sources', '--allow-empty', '--apply', '--asof', '--auto', '--background', '--batch-size', '--bound-max-concurrent', '--bound-slug-prefixes', '--bound-source', '--bound-tools', '--brain', '--brain-wide-max-cost-usd', '--branch', '--break-lock', '--budget-usd-per-day', '--by-mention', '--cached', '--catch-up', '--clone-dir', '--code', '--column', '--compile', '--concurrency', '--confirm-destructive', '--content', '--content-audit', '--count', '--date', '--days', '--deadline-ms', '--depth', '--detach', '--diff-filter', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--empty', '--entities', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--ff-only', '--file', '--fix', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--force-retry', '--force-schema', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--git-path', '--hard-deadline', '--help', '--http', '--image', '--include-flagged', '--include-frontmatter', '--include-gitignored', '--include-null-signature', '--index-audit', '--infer-dates', '--install', '--interval', '--is-ancestor', '--json', '--judge-model', '--kind', '--lang', '--limit', '--lock', '--locks', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-runtime', '--max-sources', '--max-usd', '--mcp-only', '--migrate-only', '--min-context', '--missing-path', '--mode', '--model', '--multimodal', '--name', '--name-only', '--name-status', '--near-symbol', '--ner', '--nice', '--no', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-gpg-sign', '--no-hard-deadline', '--no-pull', '--no-recurse-submodules', '--no-renames', '--no-schema-pack', '--no-verify', '--object-format', '--offset', '--older-than', '--order', '--others', '--overwrite', '--pace', '--pace-max-concurrency', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--prefix', '--priority', '--progress-interval', '--progress-json', '--query', '--queue', '--quiet', '--rebase', '--rebuild-rollup', '--reenrich-after', '--refresh', '--regenerate', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--resume', '--retry-failed', '--run-id', '--save', '--scope', '--scopes', '--section', '--serial', '--session', '--short', '--show-toplevel', '--since', '--skills-dir', '--skip-failed', '--slug', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--stats', '--status', '--stdin', '--strategy', '--supabase', '--supersessions', '--symbol-kind', '--target', '--target-score', '--thin', '--thin-threshold', '--timeout', '--to', '--top-k', '--trusted-extraction', '--type', '--types', '--url', '--url-managed', '--verbose', '--verify', '--version', '--watch', '--what', '--where', '--who', '--window', '--with-db', '--workers', '--yes'],
|
||||
'storage': ['--aliases', '--all', '--brain', '--federated', '--fix', '--help', '--include-null-signature', '--json', '--no-extract', '--no-federated', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--thin', '--to'],
|
||||
'sync': ['--abbrev-ref', '--abi', '--abort', '--aliases', '--all', '--all-sources', '--allow-empty', '--asof', '--auto', '--background', '--batch-size', '--brain', '--brain-wide-max-cost-usd', '--branch', '--break-lock', '--by-mention', '--cached', '--catch-up', '--clone-dir', '--code', '--column', '--compile', '--concurrency', '--confirm-destructive', '--content-audit', '--count', '--depth', '--detach', '--diff-filter', '--dim', '--dir', '--drain', '--dry-run', '--embedding-dimensions', '--embedding-model', '--empty', '--exclude', '--exclude-standard', '--explain', '--fast', '--federated', '--ff-only', '--fix', '--follow', '--force', '--force-break-lock', '--force-rechunk', '--force-retry', '--force-schema', '--format', '--fresh', '--from-meetings', '--from-pages', '--full', '--git-path', '--hard-deadline', '--help', '--http', '--include-flagged', '--include-frontmatter', '--include-gitignored', '--include-null-signature', '--index-audit', '--infer-dates', '--interval', '--is-ancestor', '--json', '--kind', '--lang', '--lock', '--locks', '--markdown', '--max-age', '--max-cost', '--max-cost-usd', '--max-jobs', '--max-rss', '--max-sources', '--max-usd', '--migrate-only', '--missing-path', '--model', '--multimodal', '--name', '--name-only', '--name-status', '--ner', '--nice', '--no-auto-embed', '--no-embed', '--no-embedding', '--no-extract', '--no-federated', '--no-gpg-sign', '--no-hard-deadline', '--no-pull', '--no-recurse-submodules', '--no-renames', '--no-schema-pack', '--no-verify', '--object-format', '--older-than', '--others', '--overwrite', '--pace', '--pace-max-concurrency', '--pack', '--parallel', '--params', '--path', '--pattern', '--pending', '--pglite', '--phase', '--pid-file', '--porcelain', '--prefix', '--priority', '--progress-interval', '--progress-json', '--query', '--queue', '--quiet', '--rebase', '--rebuild-rollup', '--refresh', '--regenerate', '--remediate', '--remediation-plan', '--repo', '--reset', '--resolve', '--respect-gitignore', '--restore-only', '--resume', '--retry-failed', '--run-id', '--save', '--scope', '--serial', '--short', '--show-toplevel', '--since', '--skills-dir', '--skip-failed', '--slug', '--slugs', '--source', '--source-id', '--src-subpath', '--stale', '--status', '--stdin', '--strategy', '--supabase', '--supersessions', '--target', '--target-score', '--thin', '--timeout', '--to', '--top-k', '--type', '--url', '--url-managed', '--verbose', '--verify', '--watch', '--window', '--workers', '--yes'],
|
||||
'takes': ['--aliases', '--all', '--brain', '--bucket-size', '--by', '--claim', '--dir', '--domain', '--dry-run', '--evidence', '--expired', '--fast', '--federated', '--force', '--from-pages', '--help', '--holder', '--http', '--include-covered', '--include-null-signature', '--json', '--kind', '--limit', '--max-pages', '--no-embedding', '--no-extract', '--no-federated', '--outcome', '--path', '--pattern', '--pending', '--quality', '--refresh', '--repo', '--reset', '--resolve', '--restore-only', '--row', '--since', '--slugs', '--sort', '--source', '--source-id', '--stale', '--supersessions', '--thin', '--unit', '--until', '--value', '--weight', '--who', '--yes'],
|
||||
'think': ['--aliases', '--all', '--anchor', '--brain', '--calibration-holder', '--explain', '--fast', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--max-usd', '--mcp-only', '--model', '--no-embedding', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--rounds', '--save', '--since', '--source', '--stale', '--supersessions', '--take', '--thin', '--timeout', '--until', '--with-calibration'],
|
||||
'transcripts': ['--aliases', '--all', '--brain', '--days', '--full', '--help', '--include-null-signature', '--json', '--limit', '--no-extract', '--pattern', '--pending', '--reset', '--resolve', '--source', '--stale', '--supersessions', '--thin', '--timeout'],
|
||||
'upgrade': ['--all', '--apply-clean-hunks', '--brain', '--check', '--code', '--compile', '--concurrency', '--embedding-dimensions', '--embedding-model', '--fast', '--ff-only', '--force', '--force-all', '--force-orchestrator', '--force-retry', '--force-schema', '--format', '--from-pages', '--help', '--host-dir', '--http', '--inject-bootstrap', '--inline', '--install', '--interval', '--json', '--limit', '--list', '--markdown', '--max-rss', '--migrate-only', '--migration', '--mode', '--model', '--multimodal', '--name-only', '--no', '--no-autopilot-install', '--no-embed', '--no-embedding', '--no-inject', '--no-worker', '--non-interactive', '--now', '--quiet', '--repo', '--since', '--skills-dir', '--skip-verify', '--source', '--stale', '--status', '--swap-only', '--target', '--to', '--uninstall', '--user', '--verbose', '--verify', '--version', '--workers', '--yes'],
|
||||
'watch': ['--aliases', '--all', '--brain', '--fast', '--federated', '--force', '--from-pages', '--help', '--http', '--include-null-signature', '--json', '--max-pages', '--min-confidence', '--no-embedding', '--no-extract', '--no-federated', '--path', '--pattern', '--pending', '--repo', '--reset', '--resolve', '--restore-only', '--source', '--stale', '--supersessions', '--thin', '--window-turns'],
|
||||
'ze-switch': ['--aliases', '--all', '--brain', '--confirm-reembed', '--dry-run', '--force', '--help', '--ignore-env-override', '--ignore-missing-key', '--include-null-signature', '--json', '--markdown', '--no-extract', '--non-interactive', '--pattern', '--pending', '--reset', '--resolve', '--resume', '--source', '--stale', '--supersessions', '--thin', '--undo', '--yes'],
|
||||
};
|
||||
+10
-4
@@ -990,10 +990,13 @@ export interface BrainEngine {
|
||||
*/
|
||||
upsertChunks(slug: string, chunks: ChunkInput[], opts?: { sourceId?: string } & BatchOpts): Promise<void>;
|
||||
/**
|
||||
* Read every chunk for a page. `opts.sourceId` source-scopes the page
|
||||
* lookup; without it, multi-source brains return chunks from every
|
||||
* same-slug source (importCodeFile uses this for incremental embedding
|
||||
* reuse, which would then attach the wrong source's embeddings).
|
||||
* Read every chunk for a page. Scope precedence mirrors getPage (#2555):
|
||||
* a federated grant (`sourceIds[]`) wins over scalar `sourceId`; with
|
||||
* neither set, the lookup falls back to the `'default'` source (the
|
||||
* local-untyped-call default that importCodeFile's incremental embedding
|
||||
* reuse relies on). Embedding vectors are never selected — rowToChunk
|
||||
* discards them at these call sites, so pulling them was pure egress
|
||||
* (#2544).
|
||||
*/
|
||||
getChunks(slug: string, opts?: { sourceId?: string; sourceIds?: string[] }): Promise<Chunk[]>;
|
||||
/**
|
||||
@@ -2104,6 +2107,9 @@ export interface BrainEngine {
|
||||
|
||||
// Migration support
|
||||
runMigration(version: number, sql: string): Promise<void>;
|
||||
// Deliberately scalar-only (no sourceIds[] widening): engine-internal with
|
||||
// zero remote-reachable callers (verified #2555 review), so the federated
|
||||
// read-scope contract doesn't apply. Widen only if an op ever exposes it.
|
||||
getChunksWithEmbeddings(slug: string, opts?: { sourceId?: string }): Promise<Chunk[]>;
|
||||
|
||||
// Raw SQL (for Minions job queue and other internal modules)
|
||||
|
||||
+10
-4
@@ -342,7 +342,8 @@ export function normalizeSlugPrefix(prefix: string): string {
|
||||
|
||||
/**
|
||||
* Write ops a slug-bound client may call: every op that routes through
|
||||
* `enforceClientSlugFence`, plus `think` (scope `write`, but remote callers
|
||||
* `enforceClientSlugFence`, plus `think` (scope `read` for remote callers;
|
||||
* it stays on this list because it is `mutating` locally, but remote callers
|
||||
* cannot persist — `save`/`take` are forced false for `remote !== false`).
|
||||
*
|
||||
* This list is an ALLOW-list on purpose. The fence used to be enforced op
|
||||
@@ -2274,7 +2275,7 @@ const takes_calibration: Operation = {
|
||||
const think: Operation = {
|
||||
name: 'think',
|
||||
description: 'Multi-hop synthesis across pages + takes + graph. Pulls relevant evidence and produces a cited answer with conflict + gap analysis.',
|
||||
scope: 'write',
|
||||
scope: 'read',
|
||||
params: {
|
||||
question: { type: 'string', required: true, description: 'The question to think about' },
|
||||
anchor: { type: 'string', description: 'Pull the entity subgraph around this slug' },
|
||||
@@ -2285,6 +2286,8 @@ const think: Operation = {
|
||||
since: { type: 'string', description: 'Start of temporal window (YYYY-MM-DD or YYYY-MM)' },
|
||||
until: { type: 'string', description: 'End of temporal window' },
|
||||
},
|
||||
// Local CLI can persist with save/take; remote/MCP callers are forced
|
||||
// read-only below before runThink/persistSynthesis sees those flags.
|
||||
mutating: true,
|
||||
handler: async (ctx, p) => {
|
||||
const remote = ctx.remote ?? true;
|
||||
@@ -2314,7 +2317,7 @@ const think: Operation = {
|
||||
until: p.until ? String(p.until) : undefined,
|
||||
takesHoldersAllowList: ctx.takesHoldersAllowList,
|
||||
...thinkScope,
|
||||
remote: ctx.remote === true,
|
||||
remote: ctx.remote !== false, // fail-closed: anything not strictly false is untrusted (CLAUDE.md invariant)
|
||||
});
|
||||
|
||||
// Persist if --save was passed locally
|
||||
@@ -3099,6 +3102,9 @@ const get_chunks: Operation = {
|
||||
slug: { type: 'string', required: true },
|
||||
},
|
||||
handler: async (ctx, p) => {
|
||||
// #2555: route through the canonical scope ladder (federated array >
|
||||
// scalar floor > nothing) instead of the pre-#2200 scalar-only pattern —
|
||||
// a federated grant could read the page via get_page but got [] here.
|
||||
return ctx.engine.getChunks(p.slug as string, sourceScopeOpts(ctx));
|
||||
},
|
||||
scope: 'read',
|
||||
@@ -4151,7 +4157,7 @@ const find_trajectory: Operation = {
|
||||
const points = await ctx.engine.findTrajectory({
|
||||
entitySlug: p.entity_slug,
|
||||
...scope,
|
||||
remote: ctx.remote === true,
|
||||
remote: ctx.remote !== false, // fail-closed: anything not strictly false is untrusted (CLAUDE.md invariant)
|
||||
metric,
|
||||
kind,
|
||||
since,
|
||||
|
||||
@@ -751,7 +751,13 @@ export class PGLiteEngine implements BrainEngine {
|
||||
EXISTS (SELECT 1 FROM information_schema.tables
|
||||
WHERE table_schema='public' AND table_name='timeline_entries') AS timeline_entries_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema='public' AND table_name='timeline_entries' AND column_name='event_page_id') AS timeline_event_page_id_exists
|
||||
WHERE table_schema='public' AND table_name='timeline_entries' AND column_name='event_page_id') AS timeline_event_page_id_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.tables
|
||||
WHERE table_schema='public' AND table_name='minion_jobs') AS minion_jobs_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema='public' AND table_name='minion_jobs' AND column_name='timeout_at') AS minion_jobs_timeout_at_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema='public' AND table_name='minion_jobs' AND column_name='idempotency_key') AS minion_jobs_idempotency_key_exists
|
||||
`);
|
||||
const probe = rows[0] as {
|
||||
pages_exists: boolean;
|
||||
@@ -796,6 +802,9 @@ export class PGLiteEngine implements BrainEngine {
|
||||
pages_links_extracted_at_exists: boolean;
|
||||
timeline_entries_exists: boolean;
|
||||
timeline_event_page_id_exists: boolean;
|
||||
minion_jobs_exists: boolean;
|
||||
minion_jobs_timeout_at_exists: boolean;
|
||||
minion_jobs_idempotency_key_exists: boolean;
|
||||
};
|
||||
|
||||
const needsPagesBootstrap = probe.pages_exists && !probe.source_id_exists;
|
||||
@@ -874,6 +883,12 @@ export class PGLiteEngine implements BrainEngine {
|
||||
const needsPagesLinksExtractedAt = probe.pages_exists && !probe.pages_links_extracted_at_exists;
|
||||
// v121: schema-blob indexes reference event_page_id before migrations run.
|
||||
const needsTimelineEventPageId = probe.timeline_entries_exists && !probe.timeline_event_page_id_exists;
|
||||
// v7-era (#2626 class sweep): minion_jobs.timeout_at + idempotency_key are
|
||||
// migration-added AND referenced by blob indexes (idx_minion_jobs_timeout,
|
||||
// uniq_minion_jobs_idempotency) — a pre-v7 minion_jobs wedges blob replay
|
||||
// exactly like the v121 incident.
|
||||
const needsMinionJobsTimeoutAt = probe.minion_jobs_exists && !probe.minion_jobs_timeout_at_exists;
|
||||
const needsMinionJobsIdempotencyKey = probe.minion_jobs_exists && !probe.minion_jobs_idempotency_key_exists;
|
||||
|
||||
// Fresh installs (no tables yet) and modern brains both no-op.
|
||||
if (!needsPagesBootstrap && !needsLinksBootstrap && !needsChunksBootstrap
|
||||
@@ -886,7 +901,8 @@ export class PGLiteEngine implements BrainEngine {
|
||||
&& !needsContextualRetrievalColumns && !needsPagesGeneration
|
||||
&& !needsPagesEmbeddingSignature
|
||||
&& !needsPagesLinksExtractedAt
|
||||
&& !needsTimelineEventPageId) return;
|
||||
&& !needsTimelineEventPageId
|
||||
&& !needsMinionJobsTimeoutAt && !needsMinionJobsIdempotencyKey) return;
|
||||
|
||||
process.stderr.write(' Pre-v0.21 brain detected, applying forward-reference bootstrap\n');
|
||||
|
||||
@@ -1141,6 +1157,20 @@ export class PGLiteEngine implements BrainEngine {
|
||||
ALTER TABLE timeline_entries ADD COLUMN IF NOT EXISTS event_page_id INTEGER;
|
||||
`);
|
||||
}
|
||||
|
||||
if (needsMinionJobsTimeoutAt) {
|
||||
// v7: blob index idx_minion_jobs_timeout references timeout_at; a
|
||||
// pre-v7 minion_jobs wedges blob replay without it (same class as v121).
|
||||
await this.db.exec(`
|
||||
ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS timeout_at TIMESTAMPTZ;
|
||||
`);
|
||||
}
|
||||
if (needsMinionJobsIdempotencyKey) {
|
||||
// v7: blob index uniq_minion_jobs_idempotency references idempotency_key.
|
||||
await this.db.exec(`
|
||||
ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS idempotency_key TEXT;
|
||||
`);
|
||||
}
|
||||
}
|
||||
|
||||
async withReservedConnection<T>(fn: (conn: ReservedConnection) => Promise<T>): Promise<T> {
|
||||
@@ -2664,8 +2694,14 @@ export class PGLiteEngine implements BrainEngine {
|
||||
async getChunks(slug: string, opts?: { sourceId?: string; sourceIds?: string[] }): Promise<Chunk[]> {
|
||||
const sourceIds = opts?.sourceIds && opts.sourceIds.length > 0 ? opts.sourceIds : undefined;
|
||||
const source = sourceIds ?? opts?.sourceId ?? 'default';
|
||||
// #2544: explicit non-vector column list — rowToChunk discards embeddings
|
||||
// at this call site, so `cc.*` shipped every vector only to be thrown away.
|
||||
const { rows } = await this.db.query(
|
||||
`SELECT cc.* FROM content_chunks cc
|
||||
`SELECT cc.id, cc.page_id, cc.chunk_index, cc.chunk_text, cc.chunk_source,
|
||||
cc.model, cc.token_count, cc.embedded_at, cc.language,
|
||||
cc.symbol_name, cc.symbol_type, cc.start_line, cc.end_line,
|
||||
cc.parent_symbol_path, cc.doc_comment, cc.symbol_name_qualified, cc.modality
|
||||
FROM content_chunks cc
|
||||
JOIN pages p ON p.id = cc.page_id
|
||||
WHERE p.slug = $1 AND ${sourceIds ? 'p.source_id = ANY($2::text[])' : 'p.source_id = $2'}
|
||||
ORDER BY cc.chunk_index`,
|
||||
|
||||
@@ -624,7 +624,13 @@ export class PostgresEngine implements BrainEngine {
|
||||
EXISTS (SELECT 1 FROM information_schema.tables
|
||||
WHERE table_schema = current_schema() AND table_name = 'timeline_entries') AS timeline_entries_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = current_schema() AND table_name = 'timeline_entries' AND column_name = 'event_page_id') AS timeline_event_page_id_exists
|
||||
WHERE table_schema = current_schema() AND table_name = 'timeline_entries' AND column_name = 'event_page_id') AS timeline_event_page_id_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.tables
|
||||
WHERE table_schema = current_schema() AND table_name = 'minion_jobs') AS minion_jobs_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = current_schema() AND table_name = 'minion_jobs' AND column_name = 'timeout_at') AS minion_jobs_timeout_at_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = current_schema() AND table_name = 'minion_jobs' AND column_name = 'idempotency_key') AS minion_jobs_idempotency_key_exists
|
||||
`;
|
||||
const probe = probeRows[0]!;
|
||||
|
||||
@@ -703,6 +709,9 @@ export class PostgresEngine implements BrainEngine {
|
||||
pages_links_extracted_at_exists?: boolean;
|
||||
timeline_entries_exists?: boolean;
|
||||
timeline_event_page_id_exists?: boolean;
|
||||
minion_jobs_exists?: boolean;
|
||||
minion_jobs_timeout_at_exists?: boolean;
|
||||
minion_jobs_idempotency_key_exists?: boolean;
|
||||
};
|
||||
const needsContextualRetrievalColumns = (probe.pages_exists
|
||||
&& (!probeCr.pages_cr_mode_exists || !probeCr.pages_corpus_generation_exists))
|
||||
@@ -725,6 +734,14 @@ export class PostgresEngine implements BrainEngine {
|
||||
// v121: schema-blob indexes reference event_page_id before migrations run.
|
||||
const needsTimelineEventPageId = probeCr.timeline_entries_exists === true
|
||||
&& !probeCr.timeline_event_page_id_exists;
|
||||
// v7-era (#2626 class sweep): minion_jobs.timeout_at + idempotency_key are
|
||||
// migration-added AND referenced by blob indexes (idx_minion_jobs_timeout,
|
||||
// uniq_minion_jobs_idempotency) — a pre-v7 minion_jobs wedges blob replay
|
||||
// exactly like the v121 incident.
|
||||
const needsMinionJobsTimeoutAt = probeCr.minion_jobs_exists === true
|
||||
&& !probeCr.minion_jobs_timeout_at_exists;
|
||||
const needsMinionJobsIdempotencyKey = probeCr.minion_jobs_exists === true
|
||||
&& !probeCr.minion_jobs_idempotency_key_exists;
|
||||
|
||||
if (!needsPagesBootstrap && !needsLinksBootstrap && !needsChunksBootstrap
|
||||
&& !needsPagesDeletedAt && !needsMcpLogBootstrap && !needsSubagentProviderId
|
||||
@@ -736,7 +753,8 @@ export class PostgresEngine implements BrainEngine {
|
||||
&& !needsContextualRetrievalColumns && !needsPagesGeneration
|
||||
&& !needsPagesEmbeddingSignature
|
||||
&& !needsPagesLinksExtractedAt
|
||||
&& !needsTimelineEventPageId) return;
|
||||
&& !needsTimelineEventPageId
|
||||
&& !needsMinionJobsTimeoutAt && !needsMinionJobsIdempotencyKey) return;
|
||||
|
||||
process.stderr.write(' Pre-v0.21 brain detected, applying forward-reference bootstrap\n');
|
||||
|
||||
@@ -991,6 +1009,20 @@ export class PostgresEngine implements BrainEngine {
|
||||
ALTER TABLE timeline_entries ADD COLUMN IF NOT EXISTS event_page_id INTEGER;
|
||||
`);
|
||||
}
|
||||
|
||||
if (needsMinionJobsTimeoutAt) {
|
||||
// v7: blob index idx_minion_jobs_timeout references timeout_at; a
|
||||
// pre-v7 minion_jobs wedges blob replay without it (same class as v121).
|
||||
await conn.unsafe(`
|
||||
ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS timeout_at TIMESTAMPTZ;
|
||||
`);
|
||||
}
|
||||
if (needsMinionJobsIdempotencyKey) {
|
||||
// v7: blob index uniq_minion_jobs_idempotency references idempotency_key.
|
||||
await conn.unsafe(`
|
||||
ALTER TABLE minion_jobs ADD COLUMN IF NOT EXISTS idempotency_key TEXT;
|
||||
`);
|
||||
}
|
||||
}
|
||||
|
||||
async transaction<T>(fn: (engine: BrainEngine) => Promise<T>): Promise<T> {
|
||||
@@ -2613,8 +2645,15 @@ export class PostgresEngine implements BrainEngine {
|
||||
const scope = sourceIds
|
||||
? tx`p.source_id = ANY(${sourceIds}::text[])`
|
||||
: tx`p.source_id = ${scalarSourceId}`;
|
||||
// #2544: explicit non-vector column list — rowToChunk discards
|
||||
// embeddings at this call site (includeEmbedding defaults false), so
|
||||
// `cc.*` shipped every vector over the wire only to be thrown away.
|
||||
const rows = await tx`
|
||||
SELECT cc.* FROM content_chunks cc
|
||||
SELECT cc.id, cc.page_id, cc.chunk_index, cc.chunk_text, cc.chunk_source,
|
||||
cc.model, cc.token_count, cc.embedded_at, cc.language,
|
||||
cc.symbol_name, cc.symbol_type, cc.start_line, cc.end_line,
|
||||
cc.parent_symbol_path, cc.doc_comment, cc.symbol_name_qualified, cc.modality
|
||||
FROM content_chunks cc
|
||||
JOIN pages p ON p.id = cc.page_id
|
||||
WHERE p.slug = ${slug} AND ${scope}
|
||||
ORDER BY cc.chunk_index
|
||||
|
||||
@@ -63,3 +63,43 @@ export function assertValidSourceId(s: unknown): asserts s is string {
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Normalize the optional `source` field of an `/admin/api/register-client`
|
||||
* request body into a write source_id.
|
||||
*
|
||||
* Mirrors the CLI's `--source` flag. Returns the literal `'default'` when the
|
||||
* field is omitted (`undefined`/`null`) so every caller that doesn't send
|
||||
* `source` keeps landing on source_id='default' (the pre-source HTTP
|
||||
* register-client behavior). A present-but-invalid value throws (via
|
||||
* `assertValidSourceId`) so the route can surface a structured 400 instead of
|
||||
* failing at INSERT time.
|
||||
*/
|
||||
export function normalizeSourceInput(raw: unknown): string {
|
||||
if (raw === undefined || raw === null) return 'default';
|
||||
assertValidSourceId(raw);
|
||||
return raw;
|
||||
}
|
||||
|
||||
/**
|
||||
* Normalize the optional `federatedRead` field of an
|
||||
* `/admin/api/register-client` request body into a source_id array, or
|
||||
* `undefined` when omitted.
|
||||
*
|
||||
* Mirrors the CLI's `--federated-read` flag. `undefined`/`null` → `undefined`
|
||||
* so `registerClientManual` applies its own default (`[sourceId]`, a
|
||||
* non-federated client whose read scope equals its write scope). A present
|
||||
* value must be a non-empty array whose every element is a valid source_id;
|
||||
* anything else throws for a structured 400.
|
||||
*/
|
||||
export function normalizeFederatedReadInput(raw: unknown): string[] | undefined {
|
||||
if (raw === undefined || raw === null) return undefined;
|
||||
if (!Array.isArray(raw) || raw.length === 0) {
|
||||
throw new Error(
|
||||
`Invalid federatedRead: ${JSON.stringify(raw)}. ` +
|
||||
`Must be a non-empty array of source_ids.`,
|
||||
);
|
||||
}
|
||||
for (const s of raw) assertValidSourceId(s);
|
||||
return raw as string[];
|
||||
}
|
||||
|
||||
@@ -253,4 +253,67 @@ describe('PGLiteEngine#applyForwardReferenceBootstrap', () => {
|
||||
await engine.disconnect();
|
||||
}
|
||||
}, 30000);
|
||||
|
||||
test('wedged-brain recovery: a brain that already FAILED the v0.42.56 upgrade converges on retry', async () => {
|
||||
// The loudest #2626-class cohort: operators who upgraded, wedged, and are
|
||||
// retrying with a fixed binary. Simulates the failed attempt (the blob's
|
||||
// CREATE INDEX crashing on the missing column) and asserts the retry
|
||||
// converges to the FULL final shape (column + FK + both partial indexes)
|
||||
// with no residue — the failed attempt must not advance the version ledger.
|
||||
const engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
try {
|
||||
await engine.initSchema();
|
||||
const db = (engine as any).db;
|
||||
|
||||
// Rewind to the pre-v121 shape: schema AND the version counter.
|
||||
await db.exec(`
|
||||
DROP INDEX IF EXISTS idx_timeline_event_page;
|
||||
DROP INDEX IF EXISTS idx_timeline_event_dedup;
|
||||
ALTER TABLE timeline_entries DROP CONSTRAINT IF EXISTS timeline_entries_event_page_id_fkey;
|
||||
ALTER TABLE timeline_entries DROP COLUMN IF EXISTS event_page_id;
|
||||
`);
|
||||
await engine.setConfig('version', '120');
|
||||
|
||||
// The failed old-binary attempt: without the bootstrap probe, the blob's
|
||||
// CREATE INDEX was the first statement to touch the missing column.
|
||||
let wedgeError: Error | null = null;
|
||||
try {
|
||||
await db.exec(
|
||||
`CREATE INDEX IF NOT EXISTS idx_timeline_event_page
|
||||
ON timeline_entries(event_page_id) WHERE event_page_id IS NOT NULL`,
|
||||
);
|
||||
} catch (e) {
|
||||
wedgeError = e as Error;
|
||||
}
|
||||
expect(wedgeError?.message ?? '').toContain('event_page_id');
|
||||
|
||||
// The failed attempt must not have advanced the ledger.
|
||||
expect(parseInt((await engine.getConfig('version')) || '1', 10)).toBe(120);
|
||||
|
||||
// Retry with the fixed binary: full initSchema converges to LATEST with
|
||||
// the complete final shape.
|
||||
await engine.initSchema();
|
||||
expect(await engine.getConfig('version')).toBe(String(LATEST_VERSION));
|
||||
const { rows: col } = await db.query(`
|
||||
SELECT column_name FROM information_schema.columns
|
||||
WHERE table_name = 'timeline_entries' AND column_name = 'event_page_id'
|
||||
`);
|
||||
expect(col).toHaveLength(1);
|
||||
const { rows: fk } = await db.query(`
|
||||
SELECT conname FROM pg_constraint
|
||||
WHERE conname = 'timeline_entries_event_page_id_fkey'
|
||||
`);
|
||||
expect(fk).toHaveLength(1);
|
||||
const { rows: idx } = await db.query(`
|
||||
SELECT indexname FROM pg_indexes
|
||||
WHERE tablename = 'timeline_entries'
|
||||
AND indexname IN ('idx_timeline_event_page', 'idx_timeline_event_dedup')
|
||||
`);
|
||||
expect(idx).toHaveLength(2);
|
||||
} finally {
|
||||
await engine.disconnect();
|
||||
}
|
||||
}, 30000);
|
||||
|
||||
});
|
||||
|
||||
@@ -0,0 +1,273 @@
|
||||
/**
|
||||
* #2185 — strict unknown-flag validation.
|
||||
*
|
||||
* The repro that filed the issue: `gbrain init --migrate-only --dry-run`
|
||||
* applied REAL migrations — no handler consults --dry-run, and the ad-hoc
|
||||
* `args.includes()` flag style silently ignores anything it doesn't look for.
|
||||
* The pre-dispatch validator in src/cli.ts fails loud instead.
|
||||
*
|
||||
* Four guard classes:
|
||||
* 1. sweep — every CLI_ONLY command (minus documented exemptions) and every
|
||||
* op command rejects a nonsense flag via the pure validator.
|
||||
* 2. acceptance — real flags and passthrough forms stay accepted.
|
||||
* 3. drift — every CLI_ONLY member has a registry entry.
|
||||
* 4. freshness — the committed generated registry matches a fresh
|
||||
* generator run (same doctrine as the llms-bundle freshness test).
|
||||
* Plus subprocess smokes for the end-to-end error surface.
|
||||
*/
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { spawnSync } from 'child_process';
|
||||
import {
|
||||
validateCommandFlags,
|
||||
findUnknownFlag,
|
||||
findUnknownOpFlag,
|
||||
CLI_ONLY,
|
||||
} from '../src/cli.ts';
|
||||
import { CLI_FLAG_REGISTRY } from '../src/core/cli-flag-registry.generated.ts';
|
||||
import { operations, operationsByName } from '../src/core/operations.ts';
|
||||
import { buildFlagRegistry } from '../scripts/generate-flag-registry.ts';
|
||||
|
||||
const BOGUS = '--definitely-not-a-real-flag-xyz';
|
||||
const EXEMPT = new Set(['call', 'config']); // jobs is exempt only for `submit`
|
||||
|
||||
describe('#2185 sweep — every command rejects a nonsense flag', () => {
|
||||
test('every CLI_ONLY command rejects the bogus flag (validator lane)', () => {
|
||||
const accepted: string[] = [];
|
||||
for (const command of CLI_ONLY) {
|
||||
if (EXEMPT.has(command)) continue;
|
||||
const verdict = validateCommandFlags(command, [BOGUS]);
|
||||
if (verdict !== BOGUS) accepted.push(command);
|
||||
}
|
||||
expect(accepted).toEqual([]);
|
||||
});
|
||||
|
||||
test('every op command rejects the bogus flag (op lane)', () => {
|
||||
const accepted: string[] = [];
|
||||
for (const op of operations) {
|
||||
if (!op.cliHints) continue;
|
||||
const verdict = findUnknownOpFlag(op, [BOGUS]);
|
||||
if (verdict !== BOGUS) accepted.push(op.name);
|
||||
}
|
||||
expect(accepted).toEqual([]);
|
||||
});
|
||||
|
||||
test('the literal #2185 repro is rejected: init --migrate-only --dry-run', () => {
|
||||
expect(validateCommandFlags('init', ['--migrate-only', '--dry-run'])).toBe('--dry-run');
|
||||
// And --migrate-only alone stays legal.
|
||||
expect(validateCommandFlags('init', ['--migrate-only'])).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('#2185 acceptance — real usage stays legal', () => {
|
||||
test('op flags from the contract are accepted, including values starting with --', () => {
|
||||
const search = operationsByName.search;
|
||||
expect(findUnknownOpFlag(search, ['needle', '--limit', '5'])).toBeNull();
|
||||
// A VALUE that begins with -- is consumed as the value, not validated.
|
||||
expect(findUnknownOpFlag(search, ['--query', '--weird-looking-value'])).toBeNull();
|
||||
// Inline = form.
|
||||
expect(findUnknownOpFlag(search, ['needle', '--limit=5'])).toBeNull();
|
||||
// CLI-local formatter flags.
|
||||
expect(findUnknownOpFlag(search, ['needle', '--json', '--explain'])).toBeNull();
|
||||
});
|
||||
|
||||
test('--no-<flag> negation of a known boolean op param is legal', () => {
|
||||
const withBool = operations.find(o =>
|
||||
o.cliHints && Object.values(o.params).some(p => p.type === 'boolean'));
|
||||
expect(withBool).toBeDefined();
|
||||
const boolKey = Object.entries(withBool!.params).find(([, p]) => p.type === 'boolean')![0];
|
||||
const flag = `--no-${boolKey.replace(/_/g, '-')}`;
|
||||
expect(findUnknownOpFlag(withBool!, [flag])).toBeNull();
|
||||
});
|
||||
|
||||
test('everything after a literal -- is passthrough, never validated', () => {
|
||||
expect(findUnknownFlag(['--', BOGUS], new Set(['--help']))).toBeNull();
|
||||
expect(validateCommandFlags('agent', ['run', '--', BOGUS])).toBeNull();
|
||||
expect(findUnknownOpFlag(operationsByName.search, ['--', BOGUS])).toBeNull();
|
||||
});
|
||||
|
||||
test('exempt commands accept arbitrary flags by contract', () => {
|
||||
expect(validateCommandFlags('call', ['some_op', BOGUS])).toBeNull();
|
||||
expect(validateCommandFlags('config', ['set', 'k', BOGUS])).toBeNull();
|
||||
expect(validateCommandFlags('jobs', ['submit', 'shell', BOGUS])).toBeNull();
|
||||
// ...but non-submit jobs subcommands are validated.
|
||||
expect(validateCommandFlags('jobs', ['list', BOGUS])).toBe(BOGUS);
|
||||
});
|
||||
|
||||
test('registry-listed CLI_ONLY flags are accepted', () => {
|
||||
expect(validateCommandFlags('serve', ['--http', '--port', '4444'])).toBeNull();
|
||||
expect(validateCommandFlags('serve', ['--print-admin-token'])).toBeNull();
|
||||
expect(validateCommandFlags('embed', ['--stale', '--pace'])).toBeNull();
|
||||
expect(validateCommandFlags('sync', ['--full'])).toBeNull();
|
||||
});
|
||||
|
||||
// Pre-landing review regression: the validator rejected the CLI-local
|
||||
// flags makeContext consumes OUTSIDE the op contract — `gbrain search
|
||||
// "x" --source y` and `--dry-run` invocations exited 1 as unknown flags.
|
||||
test('op commands accept --source/--dry-run (makeContext CLI-locals, not wire params)', () => {
|
||||
const search = operationsByName.search;
|
||||
expect(findUnknownOpFlag(search, ['needle', '--source', 'yc-media'])).toBeNull();
|
||||
expect(findUnknownOpFlag(search, ['needle', '--source=yc-media'])).toBeNull();
|
||||
// --source's VALUE is consumed, never validated as a flag itself.
|
||||
expect(findUnknownOpFlag(search, ['--source', '--weird-value', 'needle'])).toBeNull();
|
||||
expect(findUnknownOpFlag(search, ['needle', '--dry-run'])).toBeNull();
|
||||
// Still strict right next to them.
|
||||
expect(findUnknownOpFlag(search, ['needle', '--source', 'y', BOGUS])).toBe(BOGUS);
|
||||
});
|
||||
|
||||
test('--json=<v> is coherent between validator and parser (no positional corruption)', async () => {
|
||||
const { parseOpArgs } = await import('../src/cli.ts');
|
||||
const search = operationsByName.search;
|
||||
// Validator accepts any --json form.
|
||||
expect(findUnknownOpFlag(search, ['--json=true', 'needle'])).toBeNull();
|
||||
// Parser sets the boolean and PRESERVES the positional (pre-fix the
|
||||
// junk-key path consumed 'needle' as the value of key 'json=true').
|
||||
const p = parseOpArgs(search, ['--json=true', 'needle']);
|
||||
expect(p.json).toBe(true);
|
||||
expect(p.query).toBe('needle');
|
||||
expect((p as Record<string, unknown>)['json=true']).toBeUndefined();
|
||||
expect(parseOpArgs(search, ['needle', '--json=false']).json).toBe(false);
|
||||
// = forms of bare-only CLI-locals reject loud instead of silently eating tokens.
|
||||
expect(findUnknownOpFlag(search, ['--explain=verbose'])).toBe('--explain');
|
||||
});
|
||||
});
|
||||
|
||||
describe('#2185 parseOpArgs inline = form (regression rule: changed token consumption)', () => {
|
||||
test('string and number params via =, positional untouched', async () => {
|
||||
const { parseOpArgs } = await import('../src/cli.ts');
|
||||
const search = operationsByName.search;
|
||||
const p = parseOpArgs(search, ['needle', '--limit=5']);
|
||||
expect(p.query).toBe('needle');
|
||||
expect(p.limit).toBe(5);
|
||||
});
|
||||
|
||||
test('boolean =false negates; =0 keeps the raw!==false rule (pinned semantics)', async () => {
|
||||
const { parseOpArgs } = await import('../src/cli.ts');
|
||||
const withBool = operations.find(o =>
|
||||
o.cliHints && Object.values(o.params).some(pp => pp.type === 'boolean'))!;
|
||||
const key = Object.entries(withBool.params).find(([, pp]) => pp.type === 'boolean')![0];
|
||||
const flag = `--${key.replace(/_/g, '-')}`;
|
||||
expect(parseOpArgs(withBool, [`${flag}=false`])[key]).toBe(false);
|
||||
expect(parseOpArgs(withBool, [`${flag}=0`])[key]).toBe(true);
|
||||
});
|
||||
|
||||
test('undeclared =-form key keeps the historical junk-fallthrough (validator rejects it first)', async () => {
|
||||
const { parseOpArgs } = await import('../src/cli.ts');
|
||||
const search = operationsByName.search;
|
||||
// The validator is the strict gate; the parser's legacy behavior for
|
||||
// undeclared keys is unchanged — pinned so a refactor can't silently
|
||||
// change what unvalidated callers (tests, internal) see.
|
||||
expect(findUnknownOpFlag(search, ['--not-a-param=x', 'needle'])).toBe('--not-a-param');
|
||||
const p = parseOpArgs(search, ['--not-a-param=x', 'needle']);
|
||||
expect(p.query).toBeUndefined(); // consumed by the junk key — validator prevents reaching here
|
||||
});
|
||||
});
|
||||
|
||||
describe('#2185 red-team regressions', () => {
|
||||
test('dual-lane commands (op AND CLI_ONLY) validate against the DISPATCHED lane', () => {
|
||||
// think/salience/anomalies are both ops and CLI_ONLY members; dispatch
|
||||
// runs handleCliOnly, whose handlers parse flags the op contract never
|
||||
// declares. Pre-fix the validator checked the op lane first and rejected
|
||||
// documented invocations.
|
||||
expect(validateCommandFlags('salience', ['--kind', 'entity'])).toBeNull();
|
||||
expect(validateCommandFlags('think', ['what changed?', '--with-calibration'])).toBeNull();
|
||||
expect(validateCommandFlags('salience', [BOGUS])).toBe(BOGUS);
|
||||
});
|
||||
|
||||
test('--dry-run is a real CLI-local boolean on op commands (trailing position sets it)', async () => {
|
||||
const { parseOpArgs } = await import('../src/cli.ts');
|
||||
// An op WITHOUT a declared dry_run param — pre-fix, trailing --dry-run
|
||||
// set NOTHING (ctx.dryRun stayed false → the REAL destructive action
|
||||
// ran despite the rehearsal request), and leading --dry-run consumed
|
||||
// the next token as its value.
|
||||
const search = operationsByName.search;
|
||||
expect(parseOpArgs(search, ['needle', '--dry-run']).dry_run).toBe(true);
|
||||
const leading = parseOpArgs(search, ['--dry-run', 'needle']);
|
||||
expect(leading.dry_run).toBe(true);
|
||||
expect(leading.query).toBe('needle');
|
||||
expect(parseOpArgs(search, ['needle', '--dry-run=false']).dry_run).toBe(false);
|
||||
});
|
||||
|
||||
test('uppercase flag typos reject loudly in both lanes (handlers are case-sensitive)', () => {
|
||||
expect(findUnknownFlag(['--MIGRATE-ONLY'], new Set(['--migrate-only']))).toBe('--MIGRATE-ONLY');
|
||||
expect(findUnknownOpFlag(operationsByName.search, ['--LIMIT', '5'])).toBe('--LIMIT');
|
||||
});
|
||||
|
||||
test('safety flags need consumption evidence, not prose bleed (codex P1-A)', () => {
|
||||
// upgrade.ts prints a help HINT naming another command's --dry-run; that
|
||||
// literal sits at depth 0 for post-upgrade. Pre-fix the generator
|
||||
// allowlisted it, recreating the exact #2185 repro the wave exists to
|
||||
// kill: `post-upgrade --dry-run` accepted, ignored, migrations run for
|
||||
// real. The generator now requires a tight-quoted standalone literal
|
||||
// (an args read like has('--dry-run')) before granting a safety flag.
|
||||
expect(CLI_FLAG_REGISTRY['post-upgrade']).not.toContain('--dry-run');
|
||||
// backfill genuinely consumes it (has('--dry-run') in backfill.ts) —
|
||||
// the gate must not strip real consumers.
|
||||
expect(CLI_FLAG_REGISTRY['backfill']).toContain('--dry-run');
|
||||
});
|
||||
});
|
||||
|
||||
describe('#2185 drift + freshness guards', () => {
|
||||
test('every CLI_ONLY member has a registry entry (drift guard)', () => {
|
||||
const missing = [...CLI_ONLY].filter(c => !CLI_FLAG_REGISTRY[c]);
|
||||
expect(missing).toEqual([]);
|
||||
});
|
||||
|
||||
test('committed registry matches a fresh generator run (freshness guard)', () => {
|
||||
const fresh = buildFlagRegistry();
|
||||
const freshKeys = Object.keys(fresh).sort();
|
||||
const committedKeys = Object.keys(CLI_FLAG_REGISTRY).sort();
|
||||
expect(committedKeys).toEqual(freshKeys);
|
||||
const stale = freshKeys.filter(
|
||||
key => JSON.stringify([...CLI_FLAG_REGISTRY[key]]) !== JSON.stringify(fresh[key]),
|
||||
);
|
||||
// Any listed command means: run `bun run build:flag-registry` and commit.
|
||||
expect(stale).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('#2185 subprocess smokes — end-to-end error surface', () => {
|
||||
const run = (args: string[]) =>
|
||||
spawnSync('bun', ['src/cli.ts', ...args], {
|
||||
encoding: 'utf-8',
|
||||
timeout: 30_000,
|
||||
env: { ...process.env, GBRAIN_SKIP_STARTUP_HOOKS: '1' },
|
||||
});
|
||||
|
||||
test('init --migrate-only --dry-run fails loud BEFORE any engine work', () => {
|
||||
const r = run(['init', '--migrate-only', '--dry-run']);
|
||||
expect(r.status).toBe(1);
|
||||
expect(r.stderr).toContain("unknown flag --dry-run for 'gbrain init'");
|
||||
// Pre-engine: no migration output may appear.
|
||||
expect(r.stderr).not.toContain('migration');
|
||||
});
|
||||
|
||||
test('typo on an op command fails loud with the command named', () => {
|
||||
const r = run(['search', 'needle', '--jsno']);
|
||||
expect(r.status).toBe(1);
|
||||
expect(r.stderr).toContain("unknown flag --jsno for 'gbrain search'");
|
||||
});
|
||||
|
||||
test('--help still short-circuits before validation', () => {
|
||||
const r = run(['init', '--help']);
|
||||
expect(r.status).toBe(0);
|
||||
expect(r.stderr).not.toContain('unknown flag');
|
||||
});
|
||||
|
||||
test('global flags are accepted on every command (stripped pre-dispatch)', () => {
|
||||
// --quiet is a parseGlobalFlags global: it never reaches the validator.
|
||||
// The bogus flag proves validation still ran on what remained.
|
||||
const r = run(['init', '--migrate-only', '--quiet', '--definitely-bogus-xyz']);
|
||||
expect(r.status).toBe(1);
|
||||
expect(r.stderr).toContain("unknown flag --definitely-bogus-xyz for 'gbrain init'");
|
||||
expect(r.stderr).not.toContain('--quiet');
|
||||
});
|
||||
|
||||
test('op command accepts --source end-to-end (the makeContext CLI-local regression)', () => {
|
||||
// Fast path: --help short-circuits AFTER global parse, so a bogus flag
|
||||
// alongside --source proves ordering: --source accepted, bogus rejected.
|
||||
const r = run(['search', 'needle', '--source', 'nope-source', '--definitely-bogus-xyz']);
|
||||
expect(r.status).toBe(1);
|
||||
expect(r.stderr).toContain("unknown flag --definitely-bogus-xyz for 'gbrain search'");
|
||||
expect(r.stderr).not.toContain('unknown flag --source');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,28 @@
|
||||
import { describe, expect, test } from 'bun:test';
|
||||
import { formatResult } from '../src/cli.ts';
|
||||
|
||||
describe('formatResult - search/query --json', () => {
|
||||
test('search --json renders the raw result array as parseable JSON', () => {
|
||||
const out = formatResult('search', [
|
||||
{
|
||||
slug: 'docs/example',
|
||||
score: 0.42,
|
||||
chunk_text: 'Example result text',
|
||||
},
|
||||
], { json: true });
|
||||
|
||||
expect(JSON.parse(out)).toEqual([
|
||||
{
|
||||
slug: 'docs/example',
|
||||
score: 0.42,
|
||||
chunk_text: 'Example result text',
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
test('query --json keeps empty results machine-readable', () => {
|
||||
const out = formatResult('query', [], { json: true });
|
||||
|
||||
expect(JSON.parse(out)).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -49,6 +49,91 @@ describe('CLI structure', () => {
|
||||
test('has formatResult function for CLI output', () => {
|
||||
expect(cliSource).toContain('function formatResult');
|
||||
});
|
||||
|
||||
// #2035-class dispatch-gap guard: every `case '...'` label inside
|
||||
// handleCliOnly's top-level dispatch must be a member of CLI_ONLY, else the
|
||||
// command is registered but unreachable — 'calibration' shipped exactly this
|
||||
// way. Structural, self-updating: a new case without a CLI_ONLY entry fails
|
||||
// here at PR time.
|
||||
test('every handleCliOnly top-level case label is reachable via CLI_ONLY', () => {
|
||||
const onlyMatch = cliSource.match(/const CLI_ONLY = new Set(?:<string>)?\(\[([\s\S]*?)\]\)/);
|
||||
expect(onlyMatch).not.toBeNull();
|
||||
// Strip line comments before member extraction — the set literal carries
|
||||
// commentary whose quoted words must not count as members.
|
||||
const onlyBody = onlyMatch![1].replace(/\/\/[^\n]*/g, '');
|
||||
const members = new Set([...onlyBody.matchAll(/'([^']+)'/g)].map(m => m[1]));
|
||||
|
||||
const fnStart = cliSource.indexOf('async function handleCliOnly');
|
||||
expect(fnStart).toBeGreaterThan(0);
|
||||
const fnSrc = cliSource.slice(fnStart);
|
||||
// Top-level dispatch labels sit at a fixed indent (6 spaces); nested
|
||||
// sub-switches are indented deeper and stay out of this scan.
|
||||
const caseLabels = [...fnSrc.matchAll(/^ case '([a-z0-9-]+)':/gm)].map(m => m[1]);
|
||||
expect(caseLabels.length).toBeGreaterThan(20);
|
||||
// Reachable outside CLI_ONLY, each with a documented route:
|
||||
// - 'search': pre-dispatch subcommand gate (modes|stats|tune) in main();
|
||||
// the bare command must keep routing to the `search` op for queries.
|
||||
// - 'whoknows': currently routes via the find_experts op alias; its
|
||||
// handleCliOnly case is dead (adding it to CLI_ONLY would trip the
|
||||
// alias-collision guard and silently change output). Tracked follow-up
|
||||
// alongside PR #2509 (whoknows --explain).
|
||||
const REACHABLE_VIA_OTHER_ROUTE = new Set(['search', 'whoknows']);
|
||||
const missing = caseLabels.filter(
|
||||
label => !members.has(label) && !REACHABLE_VIA_OTHER_ROUTE.has(label),
|
||||
);
|
||||
expect(missing).toEqual([]);
|
||||
// The search gate itself must exist — losing it re-deadens the dashboards.
|
||||
// (master's gate is a superset: modes|stats|tune|diagnose.)
|
||||
expect(cliSource).toMatch(/\['modes', 'stats', 'tune'(?:, 'diagnose')?\]\.includes\(subArgs\[0\] \?\? ''\)/);
|
||||
});
|
||||
});
|
||||
|
||||
// #2450 — the local-engine output normalizer used a bare JSON.stringify with
|
||||
// no replacer. A bigint anywhere in an op's return value (e.g. a BIGSERIAL
|
||||
// primary key read back by the Postgres engine) made JSON.stringify THROW
|
||||
// "Do not know how to serialize a BigInt", crashing the command before any
|
||||
// renderer ran. normalizeLocalResult stringifies via bigintToStringReplacer
|
||||
// (bigint → string, postgres.js wire shape).
|
||||
describe('BigInt-safe output normalization (#2450)', () => {
|
||||
test('bare JSON.stringify throws on a bigint (the pre-fix crash)', () => {
|
||||
expect(() => JSON.stringify({ id: 9999999999999999999n })).toThrow(
|
||||
/serialize BigInt|serialize a BigInt/i,
|
||||
);
|
||||
});
|
||||
|
||||
test('normalizeLocalResult serializes bigint → string without throwing', async () => {
|
||||
const { normalizeLocalResult } = await import('../src/cli.ts');
|
||||
const out = normalizeLocalResult({
|
||||
id: 42n,
|
||||
nested: { count: 7n },
|
||||
arr: [1n, 2n],
|
||||
str: 'unchanged',
|
||||
num: 3,
|
||||
}) as Record<string, unknown>;
|
||||
expect(out.id).toBe('42');
|
||||
expect((out.nested as Record<string, unknown>).count).toBe('7');
|
||||
expect(out.arr).toEqual(['1', '2']);
|
||||
expect(out.str).toBe('unchanged');
|
||||
expect(out.num).toBe(3);
|
||||
});
|
||||
|
||||
test('bigint past Number.MAX_SAFE_INTEGER keeps full precision as a string', async () => {
|
||||
const { normalizeLocalResult } = await import('../src/cli.ts');
|
||||
const big = 9007199254740993n; // MAX_SAFE_INTEGER + 2
|
||||
const out = normalizeLocalResult({ id: big }) as Record<string, unknown>;
|
||||
expect(out.id).toBe('9007199254740993');
|
||||
});
|
||||
|
||||
test("formatResult's default renderer is bigint-safe", async () => {
|
||||
const { formatResult } = await import('../src/cli.ts');
|
||||
expect(() => formatResult('__no_such_op__', { id: 5n })).not.toThrow();
|
||||
expect(formatResult('__no_such_op__', { id: 5n })).toContain('"5"');
|
||||
});
|
||||
|
||||
test('cli.ts no longer uses a replacer-less stringify on the normalize path', () => {
|
||||
expect(cliSource).toContain('normalizeLocalResult(rawResult)');
|
||||
expect(cliSource).not.toContain('JSON.parse(JSON.stringify(rawResult))');
|
||||
});
|
||||
});
|
||||
|
||||
describe('CLI version', () => {
|
||||
|
||||
@@ -222,6 +222,58 @@ describe('doctor command', () => {
|
||||
}
|
||||
});
|
||||
|
||||
test('jsonb_integrity: flags double-encoded subagent payloads, ignores legitimate string scalars, skips absent tables', async () => {
|
||||
const { PGLiteEngine } = await import('../src/core/pglite-engine.ts');
|
||||
const { jsonbIntegrityCheck } = await import('../src/commands/doctor.ts');
|
||||
const engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
try {
|
||||
// Seed a minion job to satisfy the FK, then two subagent rows:
|
||||
// one DOUBLE-ENCODED (string scalar whose content is a JSON array —
|
||||
// the pre-#2375 damage class) and one LEGITIMATE string scalar
|
||||
// (persistToolExec binds pre-serialized string payloads as-is).
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO minion_jobs (id, name, data, status) VALUES (990001, 'doctor-jsonb-test', '{}'::jsonb, 'completed')`,
|
||||
);
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO subagent_messages (job_id, message_idx, role, content_blocks)
|
||||
VALUES (990001, 0, 'assistant', to_jsonb('[{"type":"text"}]'::text))`,
|
||||
);
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO subagent_messages (job_id, message_idx, role, content_blocks)
|
||||
VALUES (990001, 1, 'assistant', to_jsonb('plain text payload, not JSON'::text))`,
|
||||
);
|
||||
// Container-LOOKING but invalid JSON — matches the shape probe but
|
||||
// pg_input_is_valid must exclude it (repairing it would throw).
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO subagent_messages (job_id, message_idx, role, content_blocks)
|
||||
VALUES (990001, 2, 'assistant', to_jsonb('[INFO] fetch complete'::text))`,
|
||||
);
|
||||
|
||||
const damaged = await jsonbIntegrityCheck(engine);
|
||||
expect(damaged.status).toBe('warn');
|
||||
// Exactly the double-encoded row counts — the legit string scalar doesn't.
|
||||
expect(damaged.message).toContain('subagent_messages.content_blocks=1');
|
||||
|
||||
// Cleanup, then prove the ok path again.
|
||||
await engine.executeRaw(`DELETE FROM subagent_messages WHERE job_id = 990001`);
|
||||
await engine.executeRaw(`DELETE FROM minion_jobs WHERE id = 990001`);
|
||||
expect((await jsonbIntegrityCheck(engine)).status).toBe('ok');
|
||||
|
||||
// Absent-table skip lane: rename a target table; the check must skip
|
||||
// it without throwing (pre-v0.15 brains lack subagent_* entirely).
|
||||
await engine.executeRaw(`ALTER TABLE subagent_tool_executions RENAME TO subagent_tool_executions_bak`);
|
||||
try {
|
||||
expect((await jsonbIntegrityCheck(engine)).status).toBe('ok');
|
||||
} finally {
|
||||
await engine.executeRaw(`ALTER TABLE subagent_tool_executions_bak RENAME TO subagent_tool_executions`);
|
||||
}
|
||||
} finally {
|
||||
await engine.disconnect();
|
||||
}
|
||||
});
|
||||
|
||||
test('skill conformance derives a valid host manifest when manifest.json is absent', async () => {
|
||||
const { skillConformanceCheck } = await import('../src/commands/doctor.ts');
|
||||
const skillsDir = join(tmpdir(), `gbrain-doctor-skills-${crypto.randomUUID()}`);
|
||||
|
||||
@@ -528,6 +528,38 @@ describeBoth('Engine parity — Postgres vs PGLite', () => {
|
||||
}
|
||||
});
|
||||
|
||||
test('#2555 getChunks sourceIds[] parity: federated grant + scalar floor + unset default identical on both engines', async () => {
|
||||
for (const eng of [pgEngine, pgliteEngine]) {
|
||||
await eng.executeRaw(`INSERT INTO sources (id, name, local_path) VALUES ('gcp-beta', 'gcp-beta', '/tmp/gcp-beta') ON CONFLICT (id) DO NOTHING`);
|
||||
await eng.putPage('wiki/gcp-doc', {
|
||||
type: 'note', title: 'beta doc', compiled_truth: 'beta body', timeline: '',
|
||||
}, { sourceId: 'gcp-beta' });
|
||||
await eng.upsertChunks('wiki/gcp-doc', [
|
||||
{ chunk_index: 0, chunk_text: 'gcp beta chunk', chunk_source: 'compiled_truth' },
|
||||
], { sourceId: 'gcp-beta' });
|
||||
await eng.putPage('wiki/gcp-doc', {
|
||||
type: 'note', title: 'default decoy', compiled_truth: 'decoy body', timeline: '',
|
||||
}, { sourceId: 'default' });
|
||||
await eng.upsertChunks('wiki/gcp-doc', [
|
||||
{ chunk_index: 0, chunk_text: 'gcp default decoy', chunk_source: 'compiled_truth' },
|
||||
], { sourceId: 'default' });
|
||||
}
|
||||
|
||||
for (const eng of [pgEngine, pgliteEngine]) {
|
||||
// Federated array wins over scalar and reaches the non-default source.
|
||||
const federated = await eng.getChunks('wiki/gcp-doc', { sourceId: 'default', sourceIds: ['gcp-beta'] });
|
||||
expect(federated.map(c => c.chunk_text)).toEqual(['gcp beta chunk']);
|
||||
// Out-of-grant array → empty, never a fall-through to 'default'.
|
||||
const outOfGrant = await eng.getChunks('wiki/gcp-doc', { sourceIds: ['gcp-nonexistent'] });
|
||||
expect(outOfGrant).toEqual([]);
|
||||
// Unset opts keep the historical 'default' floor.
|
||||
const unset = await eng.getChunks('wiki/gcp-doc');
|
||||
expect(unset.map(c => c.chunk_text)).toEqual(['gcp default decoy']);
|
||||
// #2544 trim keeps the Chunk shape (embedding deliberately unselected → null).
|
||||
expect(federated[0].embedding).toBeNull();
|
||||
}
|
||||
});
|
||||
|
||||
test('v114 (#1941) listLinkSources parity: same ordered provenance counts on both engines', async () => {
|
||||
const mk = async (eng: BrainEngine) => {
|
||||
for (const s of ['lsp-a', 'lsp-b', 'lsp-c']) {
|
||||
|
||||
@@ -160,4 +160,34 @@ describeIfDB('Postgres parity — updateSourceConfig', () => {
|
||||
expect(rows[0]?.typeof).toBe('object');
|
||||
expect(rows[0]?.value).toBe('2026-05-22T12:00:00.000Z');
|
||||
});
|
||||
|
||||
test('#2251: mixed-array config (non-object elements) merges instead of throwing, and self-heals to a flat object', async () => {
|
||||
await seedSource('mixed');
|
||||
// The historical bad shape that permanently blocked last_full_cycle_at
|
||||
// writes: a JSONB array holding a non-object element. The bare
|
||||
// jsonb_each(elem) threw 'cannot call jsonb_each on a non-object'
|
||||
// DURING row production, failing every subsequent updateSourceConfig.
|
||||
await engine.executeRaw(
|
||||
`UPDATE sources
|
||||
SET config = '["stray-string", {"remote_url": "https://kept"}, 42]'::jsonb
|
||||
WHERE id = 'mixed'`,
|
||||
);
|
||||
|
||||
const ok = await engine.updateSourceConfig('mixed', {
|
||||
last_full_cycle_at: '2026-07-09T00:00:00.000Z',
|
||||
});
|
||||
expect(ok).toBe(true);
|
||||
|
||||
const rows = await engine.executeRaw<{ typeof: string; cycle: string | null; kept: string | null }>(
|
||||
`SELECT jsonb_typeof(config) AS typeof,
|
||||
config->>'last_full_cycle_at' AS cycle,
|
||||
config->>'remote_url' AS kept
|
||||
FROM sources WHERE id = 'mixed'`,
|
||||
);
|
||||
// Self-healed: flat object, patch applied, object elements' keys recovered,
|
||||
// non-object stragglers dropped.
|
||||
expect(rows[0]?.typeof).toBe('object');
|
||||
expect(rows[0]?.cycle).toBe('2026-07-09T00:00:00.000Z');
|
||||
expect(rows[0]?.kept).toBe('https://kept');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -91,6 +91,70 @@ describe.skipIf(skip)('PostgresEngine forward-reference bootstrap (E2E)', () =>
|
||||
expect(await engine.getConfig('version')).toBe(String(LATEST_VERSION));
|
||||
});
|
||||
|
||||
test('pre-v121 timeline shape converges to full final shape on REAL Postgres (#2626 wedge class)', async () => {
|
||||
// The v121 wedge was Postgres-visible in production (blob CREATE INDEX
|
||||
// on a column migration v121 hadn't added yet); the PGLite twins live in
|
||||
// test/bootstrap.test.ts. Rewind schema AND the version counter to the
|
||||
// wedged cohort's true state, then assert full initSchema convergence:
|
||||
// column + FK + BOTH partial indexes, ledger at LATEST.
|
||||
await engine.initSchema();
|
||||
const conn = (engine as any).sql;
|
||||
await conn.unsafe(`
|
||||
DROP INDEX IF EXISTS idx_timeline_event_dedup;
|
||||
DROP INDEX IF EXISTS idx_timeline_event_page;
|
||||
ALTER TABLE timeline_entries DROP CONSTRAINT IF EXISTS timeline_entries_event_page_id_fkey;
|
||||
ALTER TABLE timeline_entries DROP COLUMN IF EXISTS event_page_id;
|
||||
`);
|
||||
await engine.setConfig('version', '120');
|
||||
|
||||
await engine.initSchema();
|
||||
|
||||
expect(await engine.getConfig('version')).toBe(String(LATEST_VERSION));
|
||||
const col = await conn`
|
||||
SELECT column_name FROM information_schema.columns
|
||||
WHERE table_schema = current_schema()
|
||||
AND table_name = 'timeline_entries' AND column_name = 'event_page_id'
|
||||
`;
|
||||
expect(col).toHaveLength(1);
|
||||
const fk = await conn`
|
||||
SELECT conname FROM pg_constraint WHERE conname = 'timeline_entries_event_page_id_fkey'
|
||||
`;
|
||||
expect(fk).toHaveLength(1);
|
||||
const idx = await conn`
|
||||
SELECT indexname FROM pg_indexes
|
||||
WHERE tablename = 'timeline_entries'
|
||||
AND indexname IN ('idx_timeline_event_page', 'idx_timeline_event_dedup')
|
||||
`;
|
||||
expect(idx).toHaveLength(2);
|
||||
}, 60_000);
|
||||
|
||||
test('pre-v7 minion_jobs shape (scanner-sweep wedge class) converges on REAL Postgres', async () => {
|
||||
await engine.initSchema();
|
||||
const conn = (engine as any).sql;
|
||||
await conn.unsafe(`
|
||||
DROP INDEX IF EXISTS idx_minion_jobs_timeout;
|
||||
DROP INDEX IF EXISTS uniq_minion_jobs_idempotency;
|
||||
ALTER TABLE minion_jobs DROP COLUMN IF EXISTS timeout_at;
|
||||
ALTER TABLE minion_jobs DROP COLUMN IF EXISTS idempotency_key;
|
||||
`);
|
||||
|
||||
await engine.initSchema();
|
||||
|
||||
const cols = await conn`
|
||||
SELECT column_name FROM information_schema.columns
|
||||
WHERE table_schema = current_schema()
|
||||
AND table_name = 'minion_jobs'
|
||||
AND column_name IN ('timeout_at', 'idempotency_key')
|
||||
`;
|
||||
expect(cols).toHaveLength(2);
|
||||
const idx = await conn`
|
||||
SELECT indexname FROM pg_indexes
|
||||
WHERE tablename = 'minion_jobs'
|
||||
AND indexname IN ('idx_minion_jobs_timeout', 'uniq_minion_jobs_idempotency')
|
||||
`;
|
||||
expect(idx).toHaveLength(2);
|
||||
}, 60_000);
|
||||
|
||||
// Migration v120 — schema-lint hardening (#1647 / #171). Postgres-only
|
||||
// assertions (security_invoker has no surface on embedded PGLite).
|
||||
test('v120: page_links view runs with security_invoker=on (#1647b)', async () => {
|
||||
|
||||
@@ -373,3 +373,88 @@ describe('#2200 engine secondary-fetch methods honor sourceIds[]', () => {
|
||||
expect(windowed.map(e => e.summary)).toEqual(['june event']);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// #2555 — get_chunks honors the federated source grant (same class as #1393/
|
||||
// #2200, chunk read path). Pre-fix the op used the pre-#2200 scalar pattern
|
||||
// and engine.getChunks had no sourceIds[] support: a federated client that
|
||||
// could read a page via get_page got [] from get_chunks.
|
||||
// ---------------------------------------------------------------------------
|
||||
describe('#2555 get_chunks federated scope', () => {
|
||||
const get_chunks = operations.find(o => o.name === 'get_chunks')!;
|
||||
|
||||
beforeEach(async () => {
|
||||
await engine.upsertChunks('secret/beta-doc', [
|
||||
{ chunk_index: 0, chunk_text: 'beta chunk zero', chunk_source: 'compiled_truth' },
|
||||
{ chunk_index: 1, chunk_text: 'beta chunk one', chunk_source: 'compiled_truth' },
|
||||
], { sourceId: 'beta' });
|
||||
// Same-slug decoy chunks in 'default' — the cross-source bleed guard.
|
||||
await engine.upsertChunks('secret/beta-doc', [
|
||||
{ chunk_index: 0, chunk_text: 'default decoy chunk', chunk_source: 'compiled_truth' },
|
||||
], { sourceId: 'default' });
|
||||
});
|
||||
|
||||
test('op: federated grant including the page source returns its chunks (the #2555 repro)', async () => {
|
||||
const ctx = ctxOf({ remote: true, sourceId: undefined, auth: { token: 't', clientId: 'c', scopes: [], allowedSources: ['alpha', 'beta'] } as any });
|
||||
const chunks = await get_chunks.handler(ctx, { slug: 'secret/beta-doc' }) as Array<{ chunk_text: string }>;
|
||||
expect(chunks.map(c => c.chunk_text)).toEqual(['beta chunk zero', 'beta chunk one']);
|
||||
});
|
||||
|
||||
test('op: grant excluding the page source stays empty — never falls through to default', async () => {
|
||||
const ctx = ctxOf({ remote: true, sourceId: undefined, auth: { token: 't', clientId: 'c', scopes: [], allowedSources: ['alpha'] } as any });
|
||||
const chunks = await get_chunks.handler(ctx, { slug: 'secret/beta-doc' }) as Array<{ chunk_text: string }>;
|
||||
expect(chunks).toEqual([]);
|
||||
});
|
||||
|
||||
test('op: no grant + default floor sees only the default decoy, never beta chunks', async () => {
|
||||
const ctx = ctxOf({ remote: true, sourceId: 'default', auth: undefined });
|
||||
const chunks = await get_chunks.handler(ctx, { slug: 'secret/beta-doc' }) as Array<{ chunk_text: string }>;
|
||||
expect(chunks.map(c => c.chunk_text)).toEqual(['default decoy chunk']);
|
||||
});
|
||||
|
||||
test('engine: sourceIds[] precedence over scalar; trimmed SELECT keeps the Chunk shape', async () => {
|
||||
// array beats scalar: scalar 'default' would return the decoy; array ['beta'] must win.
|
||||
const prec = await engine.getChunks('secret/beta-doc', { sourceId: 'default', sourceIds: ['beta'] });
|
||||
expect(prec.map(c => c.chunk_text)).toEqual(['beta chunk zero', 'beta chunk one']);
|
||||
// #2544 trim: embedding is deliberately not selected (rowToChunk discards
|
||||
// it here anyway) and the rest of the Chunk shape survives.
|
||||
expect(prec[0].embedding).toBeNull();
|
||||
expect(prec[0].chunk_index).toBe(0);
|
||||
expect(prec[0].chunk_source).toBe('compiled_truth');
|
||||
// Unset opts keep the historical 'default' floor (importCodeFile contract).
|
||||
const def = await engine.getChunks('secret/beta-doc');
|
||||
expect(def.map(c => c.chunk_text)).toEqual(['default decoy chunk']);
|
||||
});
|
||||
|
||||
test('#2544 structural pin: neither engine SELECTs cc.* in getChunks (the trim survives merges)', async () => {
|
||||
// The behavioral assertion above is vacuous for the trim itself —
|
||||
// rowToChunk hard-nulls embedding regardless of the SELECT. This pin
|
||||
// exists because a master merge once silently restored `SELECT cc.*`
|
||||
// while the doc comment kept claiming the trim: assert the SELECT shape
|
||||
// at the source level for BOTH engines.
|
||||
const { readFileSync } = await import('fs');
|
||||
for (const enginePath of ['src/core/postgres-engine.ts', 'src/core/pglite-engine.ts']) {
|
||||
const src = readFileSync(new URL(`../${enginePath}`, import.meta.url), 'utf-8');
|
||||
const start = src.indexOf('async getChunks(slug');
|
||||
expect(start).toBeGreaterThan(0);
|
||||
// The method's own close (`\n }` at 2-space indent) — an inline
|
||||
// `async (tx) =>` callback must not truncate the body, and the NEXT
|
||||
// method (e.g. buildStaleChunkWhere's `cc.embedding IS NULL` WHERE
|
||||
// predicate) must not leak in. Strip line comments: the pin targets
|
||||
// the SQL, not prose that may cite the anti-pattern.
|
||||
const end = src.indexOf('\n }\n', start + 10);
|
||||
const body = src.slice(start, end).replace(/\/\/[^\n]*/g, '');
|
||||
expect(body, `${enginePath} getChunks must not SELECT cc.*`).not.toContain('cc.*');
|
||||
// Every non-vector field rowToChunk reads MUST be selected — omitting
|
||||
// one silently degrades round-trips (embed.ts getChunks→upsertChunks
|
||||
// rewrote image chunks as text when cc.modality was dropped).
|
||||
for (const col of ['chunk_text', 'chunk_source', 'model', 'token_count', 'embedded_at',
|
||||
'language', 'symbol_name', 'symbol_type', 'start_line', 'end_line',
|
||||
'parent_symbol_path', 'doc_comment', 'symbol_name_qualified', 'modality']) {
|
||||
expect(body, `${enginePath} getChunks must select cc.${col}`).toContain(`cc.${col}`);
|
||||
}
|
||||
// The vector columns stay unselected.
|
||||
expect(body).not.toMatch(/cc\.embedding\b/);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
+12
-2
@@ -894,13 +894,23 @@ describe('operation scope annotations', () => {
|
||||
}
|
||||
});
|
||||
|
||||
test('mutating operations are write/admin/sources_admin/users_admin/agent scoped', () => {
|
||||
test('mutating operations are write/admin/sources_admin/users_admin/agent scoped unless remote-gated', () => {
|
||||
const { operations } = require('../src/core/operations.ts');
|
||||
// #2598, same allowlist as test/operations-trust-boundary.test.ts: think
|
||||
// is read-scoped for OAuth/MCP because its handler forces save/take off
|
||||
// for remote callers before persistence (pinned by
|
||||
// test/takes-mcp-allowlist.serial.test.ts); local CLI can still persist.
|
||||
const remoteReadOnlyMutatingOps = new Set(['think']);
|
||||
for (const op of operations) {
|
||||
if (op.mutating) {
|
||||
if (remoteReadOnlyMutatingOps.has(op.name)) {
|
||||
expect(op.scope, `${op.name} remote-gated mutating op should be read-scoped`).toBe('read');
|
||||
continue;
|
||||
}
|
||||
// v0.28: sources_admin permits sources_add / sources_remove (mutating
|
||||
// sources, not pages); read scope is the only thing too narrow for
|
||||
// any mutating op. v0.38: 'agent' is a mutating-axis scope for
|
||||
// a mutating op unless its remote path forces persistence off before
|
||||
// the handler writes. v0.38: 'agent' is a mutating-axis scope for
|
||||
// submit_agent (creates jobs, spends money, but contained by bindings).
|
||||
expect(
|
||||
['write', 'admin', 'sources_admin', 'users_admin', 'agent'],
|
||||
|
||||
@@ -86,8 +86,17 @@ describe('operations contract — every op has scope + correct mutability shape'
|
||||
'users_admin',
|
||||
'agent',
|
||||
]);
|
||||
// Remote-gated exception (#2598, same allowlist as test/oauth.test.ts):
|
||||
// `think` is read-scoped for OAuth/MCP because its handler forces
|
||||
// save/take OFF for remote callers before persistence — pinned by
|
||||
// test/takes-mcp-allowlist.serial.test.ts. Local CLI can still persist.
|
||||
const REMOTE_READ_ONLY_MUTATING_OPS = new Set(['think']);
|
||||
for (const op of operations) {
|
||||
if (op.mutating === true) {
|
||||
if (REMOTE_READ_ONLY_MUTATING_OPS.has(op.name)) {
|
||||
expect(op.scope, `remote-gated mutating op "${op.name}" should be read-scoped`).toBe('read');
|
||||
continue;
|
||||
}
|
||||
expect(
|
||||
WRITE_CLASS_SCOPES.has(op.scope ?? 'read'),
|
||||
`mutating op "${op.name}" has read-tier scope "${op.scope}"; expected one of ${[...WRITE_CLASS_SCOPES].join('/')}`,
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
/**
|
||||
* normalizeSourceInput / normalizeFederatedReadInput contract.
|
||||
*
|
||||
* The `/admin/api/register-client` HTTP endpoint historically hardcoded
|
||||
* source_id='default' (and federated_read=[source_id]) — only the CLI
|
||||
* (`--source` / `--federated-read`) could bind a client to a non-default
|
||||
* brain source. These two normalizers let the HTTP endpoint accept the same
|
||||
* inputs from the request body while preserving the historical default when
|
||||
* the fields are omitted.
|
||||
*
|
||||
* Hermetic — pure-function unit tests, no engine, no HTTP.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { normalizeSourceInput, normalizeFederatedReadInput } from '../src/core/source-id.ts';
|
||||
|
||||
describe('normalizeSourceInput', () => {
|
||||
test('undefined → "default" (backward compat)', () => {
|
||||
expect(normalizeSourceInput(undefined)).toBe('default');
|
||||
});
|
||||
|
||||
test('null → "default" (backward compat)', () => {
|
||||
expect(normalizeSourceInput(null)).toBe('default');
|
||||
});
|
||||
|
||||
test('valid source_id passes through', () => {
|
||||
expect(normalizeSourceInput('mind-agent-brain')).toBe('mind-agent-brain');
|
||||
});
|
||||
|
||||
test('single-char source_id is valid', () => {
|
||||
expect(normalizeSourceInput('a')).toBe('a');
|
||||
});
|
||||
|
||||
test('invalid source_id (underscore) throws', () => {
|
||||
expect(() => normalizeSourceInput('mind_agent_brain')).toThrow(/Invalid source_id/);
|
||||
});
|
||||
|
||||
test('invalid source_id (uppercase) throws', () => {
|
||||
expect(() => normalizeSourceInput('Mind')).toThrow(/Invalid source_id/);
|
||||
});
|
||||
|
||||
test('invalid source_id (edge hyphen) throws', () => {
|
||||
expect(() => normalizeSourceInput('-mind')).toThrow(/Invalid source_id/);
|
||||
});
|
||||
|
||||
test('non-string (number) throws', () => {
|
||||
expect(() => normalizeSourceInput(42)).toThrow(/Invalid source_id/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('register-client route wiring (structural)', () => {
|
||||
test('normalized source + federatedRead reach registerClientManual in the right positions', () => {
|
||||
// The unit tests above prove the normalizers; this pins the ROUTE —
|
||||
// a transposition of the two new positional args (or a regression to
|
||||
// the hardcoded 'default') would pass every unit test and still ship
|
||||
// clients bound to the wrong source.
|
||||
const { readFileSync } = require('fs');
|
||||
const src = readFileSync(new URL('../src/commands/serve-http.ts', import.meta.url), 'utf-8');
|
||||
expect(src).toContain('sourceId = normalizeSourceInput(source)');
|
||||
expect(src).toContain('federatedReadIds = normalizeFederatedReadInput(federatedRead)');
|
||||
expect(src).toMatch(/registerClientManual\(\s*name,\s*grants,\s*scopeString,\s*uris,\s*sourceId,\s*federatedReadIds,\s*validatedAuthMethod/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('normalizeFederatedReadInput', () => {
|
||||
test('undefined → undefined (let registerClientManual default to [sourceId])', () => {
|
||||
expect(normalizeFederatedReadInput(undefined)).toBeUndefined();
|
||||
});
|
||||
|
||||
test('null → undefined', () => {
|
||||
expect(normalizeFederatedReadInput(null)).toBeUndefined();
|
||||
});
|
||||
|
||||
test('valid single-element array passes through', () => {
|
||||
expect(normalizeFederatedReadInput(['mind-agent-brain'])).toEqual(['mind-agent-brain']);
|
||||
});
|
||||
|
||||
test('valid multi-element array passes through (read across both sources)', () => {
|
||||
expect(normalizeFederatedReadInput(['default', 'mind-agent-brain'])).toEqual([
|
||||
'default',
|
||||
'mind-agent-brain',
|
||||
]);
|
||||
});
|
||||
|
||||
test('empty array throws (ambiguous — omit the field instead)', () => {
|
||||
expect(() => normalizeFederatedReadInput([])).toThrow(/Invalid federatedRead/);
|
||||
});
|
||||
|
||||
test('non-array (string) throws', () => {
|
||||
expect(() => normalizeFederatedReadInput('default')).toThrow(/Invalid federatedRead/);
|
||||
});
|
||||
|
||||
test('array with an invalid source_id element throws', () => {
|
||||
expect(() => normalizeFederatedReadInput(['default', 'bad_id'])).toThrow(/Invalid source_id/);
|
||||
});
|
||||
});
|
||||
@@ -19,9 +19,10 @@ describe('repairJsonb — PGLite short-circuit', () => {
|
||||
});
|
||||
expect(result.engine).toBe('pglite');
|
||||
expect(result.total_repaired).toBe(0);
|
||||
// All 5 columns reported: pages.frontmatter, raw_data.data,
|
||||
// ingest_log.pages_updated, files.metadata, page_versions.frontmatter.
|
||||
expect(result.per_target.length).toBe(5);
|
||||
// All 8 columns reported: 5 from the v0.12.0 wave + 3 from the v0.16.0
|
||||
// subagent_* wave (added when persistMessage / persistToolExec* was
|
||||
// identified as a second double-encode site).
|
||||
expect(result.per_target.length).toBe(8);
|
||||
for (const t of result.per_target) {
|
||||
expect(t.rows_repaired).toBe(0);
|
||||
}
|
||||
@@ -32,6 +33,9 @@ describe('repairJsonb — PGLite short-circuit', () => {
|
||||
'page_versions.frontmatter',
|
||||
'pages.frontmatter',
|
||||
'raw_data.data',
|
||||
'subagent_messages.content_blocks',
|
||||
'subagent_tool_executions.input',
|
||||
'subagent_tool_executions.output',
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -171,6 +171,13 @@ const REQUIRED_BOOTSTRAP_COVERAGE: ForwardReference[] = [
|
||||
// v121 — referenced by the timeline event lookup and dedup indexes before
|
||||
// the numbered migration can add the column on an existing brain.
|
||||
{ kind: 'column', table: 'timeline_entries', column: 'event_page_id' },
|
||||
// v7-era — surfaced by the #2626-class scanner sweep: both columns are
|
||||
// migration-added (v7) AND referenced by blob indexes
|
||||
// (`idx_minion_jobs_timeout` partial on timeout_at, the partial UNIQUE
|
||||
// `uniq_minion_jobs_idempotency` on idempotency_key). A pre-v7 minion_jobs
|
||||
// wedges the blob replay exactly like the v121 incident.
|
||||
{ kind: 'column', table: 'minion_jobs', column: 'timeout_at' },
|
||||
{ kind: 'column', table: 'minion_jobs', column: 'idempotency_key' },
|
||||
];
|
||||
|
||||
test('applyForwardReferenceBootstrap covers every forward reference declared in REQUIRED_BOOTSTRAP_COVERAGE', async () => {
|
||||
@@ -261,6 +268,13 @@ test('applyForwardReferenceBootstrap covers every forward reference declared in
|
||||
DROP INDEX IF EXISTS idx_timeline_event_page;
|
||||
ALTER TABLE timeline_entries DROP CONSTRAINT IF EXISTS timeline_entries_event_page_id_fkey;
|
||||
ALTER TABLE timeline_entries DROP COLUMN IF EXISTS event_page_id;
|
||||
|
||||
-- v7 minion_jobs strip (#2626 class sweep): timeout_at + idempotency_key
|
||||
-- are migration-added and blob-indexed; strip so bootstrap must re-add.
|
||||
DROP INDEX IF EXISTS idx_minion_jobs_timeout;
|
||||
DROP INDEX IF EXISTS uniq_minion_jobs_idempotency;
|
||||
ALTER TABLE minion_jobs DROP COLUMN IF EXISTS timeout_at;
|
||||
ALTER TABLE minion_jobs DROP COLUMN IF EXISTS idempotency_key;
|
||||
`);
|
||||
|
||||
// Note: we don't strip sources.archived* here because they're inline in the
|
||||
@@ -346,6 +360,14 @@ test('after bootstrap, PGLITE_SCHEMA_SQL replays without crashing on missing for
|
||||
DROP INDEX IF EXISTS idx_timeline_event_page;
|
||||
ALTER TABLE timeline_entries DROP CONSTRAINT IF EXISTS timeline_entries_event_page_id_fkey;
|
||||
ALTER TABLE timeline_entries DROP COLUMN IF EXISTS event_page_id;
|
||||
|
||||
-- v7 minion_jobs strip (#2626 class sweep): the SCHEMA_SQL replay would
|
||||
-- crash on idx_minion_jobs_timeout / uniq_minion_jobs_idempotency
|
||||
-- without the bootstrap re-adding these migration-added columns.
|
||||
DROP INDEX IF EXISTS idx_minion_jobs_timeout;
|
||||
DROP INDEX IF EXISTS uniq_minion_jobs_idempotency;
|
||||
ALTER TABLE minion_jobs DROP COLUMN IF EXISTS timeout_at;
|
||||
ALTER TABLE minion_jobs DROP COLUMN IF EXISTS idempotency_key;
|
||||
`);
|
||||
|
||||
// Bootstrap, then schema replay. Either step crashing fails the test.
|
||||
@@ -599,16 +621,72 @@ function parseAlterAddColumns(sql: string): Array<{ table: string; column: strin
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* The coverage predicate for blob index-column references, extracted so the
|
||||
* v121-regression unit test below can exercise it with synthetic inputs.
|
||||
*
|
||||
* v0.42.58 (#2626 class): CREATE TABLE presence must NOT count as coverage
|
||||
* for a column that ANY migration also adds via ALTER TABLE ADD COLUMN. The
|
||||
* migration's existence proves pre-existing tables can lack the column, and
|
||||
* on those brains `CREATE TABLE IF NOT EXISTS` no-ops — so the blob's
|
||||
* CREATE INDEX crashes initSchema before runMigrations can help. For such
|
||||
* columns, only an applyForwardReferenceBootstrap ALTER counts. This is
|
||||
* exactly how `timeline_entries.event_page_id` (v121, Life Chronicle) shipped
|
||||
* a P0 upgrade wedge past the old predicate: it was in the current CREATE
|
||||
* TABLE body, so the check passed while every pre-v121 brain wedged.
|
||||
*/
|
||||
function buildIndexRefCoveragePredicate(
|
||||
tableColumns: Map<string, Set<string>>,
|
||||
bootstrapAdds: Array<{ table: string; column: string }>,
|
||||
migrationAddedKeys: Set<string>,
|
||||
): (table: string, column: string) => boolean {
|
||||
return (table: string, column: string): boolean => {
|
||||
const inBootstrap = bootstrapAdds.some(a => a.table === table && a.column === column);
|
||||
if (inBootstrap) return true;
|
||||
// Migration-added columns are forward references by definition —
|
||||
// CREATE TABLE presence is exactly the mask that hid the v121 wedge.
|
||||
if (migrationAddedKeys.has(`${table}.${column}`)) return false;
|
||||
const cols = tableColumns.get(table);
|
||||
return Boolean(cols && cols.has(column));
|
||||
};
|
||||
}
|
||||
|
||||
test('buildIndexRefCoveragePredicate: CREATE TABLE presence does not mask migration-added columns (v121 regression shape)', () => {
|
||||
const tableColumns = new Map([['timeline_entries', new Set(['id', 'event_page_id'])]]);
|
||||
const migrationAdded = new Set(['timeline_entries.event_page_id']);
|
||||
|
||||
// The exact pre-fix v121 shape: column in CREATE TABLE, added by migration,
|
||||
// NO bootstrap probe → must be UNCOVERED (old predicate said covered).
|
||||
const withoutProbe = buildIndexRefCoveragePredicate(tableColumns, [], migrationAdded);
|
||||
expect(withoutProbe('timeline_entries', 'event_page_id')).toBe(false);
|
||||
// Plain blob-native column (not migration-added) stays covered by CREATE TABLE.
|
||||
expect(withoutProbe('timeline_entries', 'id')).toBe(true);
|
||||
|
||||
// With the bootstrap probe present, the same column is covered.
|
||||
const withProbe = buildIndexRefCoveragePredicate(
|
||||
tableColumns,
|
||||
[{ table: 'timeline_entries', column: 'event_page_id' }],
|
||||
migrationAdded,
|
||||
);
|
||||
expect(withProbe('timeline_entries', 'event_page_id')).toBe(true);
|
||||
});
|
||||
|
||||
test('every CREATE INDEX column in PGLITE_SCHEMA_SQL is covered by CREATE TABLE or bootstrap (A2 static check)', async () => {
|
||||
// The structural test that closes the 11-incident wedge class. Static
|
||||
// contract: every column referenced by a CREATE INDEX in PGLITE_SCHEMA_SQL
|
||||
// must be either (a) declared in the current CREATE TABLE body, or
|
||||
// (b) added by `applyForwardReferenceBootstrap` in pglite-engine.ts.
|
||||
// must be either (a) declared in the current CREATE TABLE body AND not
|
||||
// added by any migration (see buildIndexRefCoveragePredicate — migration-
|
||||
// added columns are forward references even when the CREATE TABLE body has
|
||||
// them), or (b) added by `applyForwardReferenceBootstrap` in
|
||||
// pglite-engine.ts.
|
||||
//
|
||||
// Codex outside-voice review caught the 11th wedge: composite-index second
|
||||
// columns (`provider_id` in `(job_id, provider_id)`) are forward references
|
||||
// that earlier extractors missed. This parser walks the full column list
|
||||
// of every index — composite or not — and asserts each one is covered.
|
||||
// The 12th wedge (v121 `timeline_entries.event_page_id`, #2626 #2594 #2579
|
||||
// #2537 #2536) slipped through because CREATE TABLE presence masked the
|
||||
// forward reference; the predicate now cross-references MIGRATIONS.
|
||||
//
|
||||
// Self-updating: when a future migration adds a CREATE INDEX in
|
||||
// PGLITE_SCHEMA_SQL on a column that bootstrap doesn't yet provide, this
|
||||
@@ -616,6 +694,7 @@ test('every CREATE INDEX column in PGLITE_SCHEMA_SQL is covered by CREATE TABLE
|
||||
const { readFileSync } = await import('fs');
|
||||
const { resolve: resolvePath } = await import('path');
|
||||
const { PGLITE_SCHEMA_SQL } = await import('../src/core/pglite-schema.ts');
|
||||
const { extractAddedColumnsFromMigrations } = await import('./helpers/extract-added-columns.ts');
|
||||
|
||||
const enginePath = resolvePath(process.cwd(), 'src/core/pglite-engine.ts');
|
||||
const engineSrc = readFileSync(enginePath, 'utf-8');
|
||||
@@ -623,21 +702,23 @@ test('every CREATE INDEX column in PGLITE_SCHEMA_SQL is covered by CREATE TABLE
|
||||
const tableColumns = parseBaseTableColumns(PGLITE_SCHEMA_SQL);
|
||||
const indexRefs = parseIndexColumnReferences(PGLITE_SCHEMA_SQL);
|
||||
const bootstrapAdds = parseAlterAddColumns(engineSrc);
|
||||
const migrationAddedKeys = new Set(
|
||||
extractAddedColumnsFromMigrations().map(a => `${a.table}.${a.column}`),
|
||||
);
|
||||
|
||||
// Build the "covered" set: for each (table, column) pair, true iff it's in
|
||||
// the table's CREATE TABLE columns OR added by an ALTER TABLE in the
|
||||
// bootstrap function.
|
||||
const covered = (table: string, column: string): boolean => {
|
||||
const cols = tableColumns.get(table);
|
||||
if (cols && cols.has(column)) return true;
|
||||
return bootstrapAdds.some(a => a.table === table && a.column === column);
|
||||
};
|
||||
const covered = buildIndexRefCoveragePredicate(tableColumns, bootstrapAdds, migrationAddedKeys);
|
||||
|
||||
// Sanity checks: parser caught the codex case AND bootstrap provides it.
|
||||
expect(indexRefs).toContainEqual({ table: 'subagent_messages', column: 'provider_id' });
|
||||
expect(bootstrapAdds).toContainEqual({ table: 'subagent_messages', column: 'provider_id' });
|
||||
expect(covered('subagent_messages', 'provider_id')).toBe(true);
|
||||
|
||||
// Direct pin of the v121 incident: the column is migration-added, blob-
|
||||
// indexed, and MUST be bootstrap-covered.
|
||||
expect(migrationAddedKeys.has('timeline_entries.event_page_id')).toBe(true);
|
||||
expect(indexRefs.some(r => r.table === 'timeline_entries' && r.column === 'event_page_id')).toBe(true);
|
||||
expect(bootstrapAdds).toContainEqual({ table: 'timeline_entries', column: 'event_page_id' });
|
||||
|
||||
// The actual contract: every index column reference must be covered.
|
||||
const uncovered: Array<{ table: string; column: string }> = [];
|
||||
for (const ref of indexRefs) {
|
||||
@@ -650,10 +731,13 @@ test('every CREATE INDEX column in PGLITE_SCHEMA_SQL is covered by CREATE TABLE
|
||||
const list = uncovered.map(u => ` ${u.table}.${u.column}`).join('\n');
|
||||
throw new Error(
|
||||
`PGLITE_SCHEMA_SQL has ${uncovered.length} CREATE INDEX column reference(s) ` +
|
||||
`that are neither in the table's CREATE TABLE body nor added by ` +
|
||||
`applyForwardReferenceBootstrap:\n${list}\n\n` +
|
||||
`that are not safely covered (in the CREATE TABLE body AND not migration-added, ` +
|
||||
`or added by applyForwardReferenceBootstrap):\n${list}\n\n` +
|
||||
`Fix: extend applyForwardReferenceBootstrap in src/core/pglite-engine.ts ` +
|
||||
`(and the matching Postgres engine) with the missing ALTER TABLE ADD COLUMN.`,
|
||||
`(and the matching Postgres engine) with the missing ALTER TABLE ADD COLUMN. ` +
|
||||
`A column that is BOTH in the blob's CREATE TABLE AND added by a migration ` +
|
||||
`is a forward reference for pre-existing tables — CREATE TABLE presence ` +
|
||||
`does not cover it (that mask shipped the v121 upgrade wedge).`,
|
||||
);
|
||||
}
|
||||
}, 30000);
|
||||
|
||||
@@ -156,3 +156,21 @@ describe('runUnifyTypes', () => {
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// #1575 — the jobs worker registration must honor the handler's documented
|
||||
// dry-run default. `apply: data.apply ?? true` made the canonical operator
|
||||
// invocation (`gbrain jobs submit unify-types --allow-protected --params
|
||||
// '{"target_pack":...}'`) destructively retype pages by default while
|
||||
// UnifyTypesOpts.apply documents 'Default false (dry-run)'. Structural pin —
|
||||
// the worker source must default apply to false.
|
||||
import { readFileSync } from 'fs';
|
||||
|
||||
describe('#1575 unify-types worker dry-run default', () => {
|
||||
it('jobs.ts worker registration defaults apply to false, matching the handler contract', () => {
|
||||
const jobsSource = readFileSync(new URL('../src/commands/jobs.ts', import.meta.url), 'utf-8');
|
||||
const workerBlock = jobsSource.slice(jobsSource.indexOf("worker.register('unify-types'"));
|
||||
const registration = workerBlock.slice(0, workerBlock.indexOf('});'));
|
||||
expect(registration).toContain('apply: data.apply ?? false');
|
||||
expect(registration).not.toContain('apply: data.apply ?? true');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -72,7 +72,10 @@ function runWrapper(extraArgs: string[] = []): { code: number; stdout: string; s
|
||||
const result = spawnSync(
|
||||
'bash',
|
||||
[join(TMPROOT, 'scripts', 'run-unit-parallel.sh'), '--shards', '2', ...extraArgs],
|
||||
{ cwd: TMPROOT, encoding: 'utf-8', env: { ...process.env } },
|
||||
// Shard-mechanics tests pin explicit --shards behavior with tiny
|
||||
// synthetic files; disable mem-adaptation so a RAM-limited runner (CI's
|
||||
// ~7GB) can't collapse 2 shards -> 1 and break the shard 1/2 expectations.
|
||||
{ cwd: TMPROOT, encoding: 'utf-8', env: { ...process.env, GBRAIN_TEST_NO_MEM_ADAPT: '1' } },
|
||||
);
|
||||
return {
|
||||
code: result.status ?? -1,
|
||||
@@ -209,6 +212,8 @@ describe('passing', () => {
|
||||
HOME: process.env.HOME ?? FROOT,
|
||||
TMPDIR: process.env.TMPDIR ?? '/tmp',
|
||||
GBRAIN_TEST_SHARD_TIMEOUT: '300',
|
||||
// Same rationale as runWrapper: explicit-shard mechanics under test.
|
||||
GBRAIN_TEST_NO_MEM_ADAPT: '1',
|
||||
};
|
||||
});
|
||||
|
||||
@@ -257,3 +262,141 @@ describe('failing-on-purpose', () => {
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('run-unit-parallel.sh OOM rescue lane', () => {
|
||||
// A fixture that fails WITH the WASM out-of-memory signature on its first
|
||||
// run (no sentinel file yet) and passes once the sentinel exists — exactly
|
||||
// the phantom-failure shape: dies under parallel memory pressure, passes
|
||||
// serially. The runner must (1) detect the signature, (2) re-run the file
|
||||
// at --max-concurrency 1, (3) exit 0 with an oom_rescued note.
|
||||
let OROOT: string;
|
||||
|
||||
beforeAll(() => {
|
||||
OROOT = mkdtempSync(join(tmpdir(), 'gbrain-parallel-oom-'));
|
||||
mkdirSync(join(OROOT, 'scripts'), { recursive: true });
|
||||
mkdirSync(join(OROOT, 'test'), { recursive: true });
|
||||
for (const s of ['run-unit-parallel.sh', 'run-unit-shard.sh', 'run-serial-tests.sh']) {
|
||||
copyFileSync(resolve(REPO_ROOT, 'scripts', s), join(OROOT, 'scripts', s));
|
||||
chmodSync(join(OROOT, 'scripts', s), 0o755);
|
||||
}
|
||||
const passing = `import { describe, it, expect } from 'bun:test';
|
||||
describe('passing', () => {
|
||||
it('arithmetic works', () => { expect(1 + 1).toBe(2); });
|
||||
});`;
|
||||
const oomOnce = `import { describe, it, expect } from 'bun:test';
|
||||
import { existsSync, writeFileSync } from 'fs';
|
||||
describe('oom-once', () => {
|
||||
it('fails with the WASM OOM signature on first run, passes on retry', () => {
|
||||
const sentinel = new URL('./oom-sentinel.txt', import.meta.url).pathname;
|
||||
if (!existsSync(sentinel)) {
|
||||
writeFileSync(sentinel, 'ran-once');
|
||||
console.error('Original error: Out of memory');
|
||||
throw new Error('Out of memory (simulated PGLite WASM connect failure)');
|
||||
}
|
||||
expect(1).toBe(1);
|
||||
});
|
||||
});`;
|
||||
writeFileSync(join(OROOT, 'test', 'a-pass.test.ts'), passing);
|
||||
writeFileSync(join(OROOT, 'test', 'b-oom-once.test.ts'), oomOnce);
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
if (OROOT) rmSync(OROOT, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
function runOom(env: Record<string, string> = {}): { code: number; stdout: string; stderr: string } {
|
||||
rmSync(join(OROOT, 'test', 'oom-sentinel.txt'), { force: true });
|
||||
const result = spawnSync(
|
||||
'bash',
|
||||
[join(OROOT, 'scripts', 'run-unit-parallel.sh'), '--shards', '2'],
|
||||
{ cwd: OROOT, encoding: 'utf-8', env: { ...process.env, GBRAIN_TEST_NO_MEM_ADAPT: '1', ...env } },
|
||||
);
|
||||
return { code: result.status ?? -1, stdout: result.stdout || '', stderr: result.stderr || '' };
|
||||
}
|
||||
|
||||
it('rescues an OOM-signature failure serially and exits 0 with an oom_rescued note', () => {
|
||||
const r = runOom();
|
||||
expect(r.stdout + r.stderr).toContain('OOM rescue pass');
|
||||
expect(r.stderr).toContain('oom_rescued=');
|
||||
expect(r.code).toBe(0);
|
||||
}, 120_000);
|
||||
|
||||
it('GBRAIN_TEST_NO_OOM_FALLBACK=1 disables the rescue lane (stays red)', () => {
|
||||
const r = runOom({ GBRAIN_TEST_NO_OOM_FALLBACK: '1' });
|
||||
expect(r.code).not.toBe(0);
|
||||
expect(r.stdout + r.stderr).not.toContain('OOM rescue pass');
|
||||
}, 120_000);
|
||||
|
||||
it('memory-aware sizing is advertised in the banner (mem-ok or mem-adapted)', () => {
|
||||
// The one test that needs adaptation ON — override the harness-wide
|
||||
// NO_MEM_ADAPT base (which keeps the shard-mechanics tests deterministic
|
||||
// on RAM-limited CI runners).
|
||||
const r = runOom({ GBRAIN_TEST_NO_MEM_ADAPT: '0' });
|
||||
expect(r.stderr).toMatch(/mem-(ok|adapted)/);
|
||||
}, 120_000);
|
||||
|
||||
it('mixed run: a plain assertion failure stays red even when the OOM phantom rescues green', () => {
|
||||
// The NON_OOM_FAIL gate — the branch that stops the rescue lane from
|
||||
// absolving real failures that happened to share a run with phantoms.
|
||||
const realFail = `import { describe, it, expect } from 'bun:test';
|
||||
describe('real-failure', () => {
|
||||
it('expects 1 to equal 2', () => { expect(1).toBe(2); });
|
||||
});`;
|
||||
writeFileSync(join(OROOT, 'test', 'c-real-fail.test.ts'), realFail);
|
||||
try {
|
||||
const r = runOom();
|
||||
expect(r.code).not.toBe(0);
|
||||
} finally {
|
||||
rmSync(join(OROOT, 'test', 'c-real-fail.test.ts'), { force: true });
|
||||
}
|
||||
}, 120_000);
|
||||
|
||||
it('a deterministic failure carrying the OOM signature re-fails serially and stays red', () => {
|
||||
// The oom_rescue_failed lane: signature match queues the file, but the
|
||||
// serial re-run confirms the failure is real — run must stay red.
|
||||
const alwaysOom = `import { describe, it } from 'bun:test';
|
||||
describe('oom-always', () => {
|
||||
it('always fails with the signature', () => {
|
||||
console.error('Original error: Out of memory');
|
||||
throw new Error('Out of memory (deterministic)');
|
||||
});
|
||||
});`;
|
||||
writeFileSync(join(OROOT, 'test', 'd-oom-always.test.ts'), alwaysOom);
|
||||
try {
|
||||
const r = runOom();
|
||||
expect(r.code).not.toBe(0);
|
||||
expect(r.stderr).toContain('oom_rescue_failed=');
|
||||
expect(r.stdout + r.stderr).toContain('oom-rescue (serial, confirmed real)');
|
||||
} finally {
|
||||
rmSync(join(OROOT, 'test', 'd-oom-always.test.ts'), { force: true });
|
||||
}
|
||||
}, 120_000);
|
||||
});
|
||||
|
||||
describe('run-unit-parallel.sh external-kill rescue contract', () => {
|
||||
// An externally-killed shard (sibling workspace pkill, memory jetsam)
|
||||
// presents as rc 143/137 well before the shard timeout. Simulating a
|
||||
// mid-run external kill deterministically in a fixture is flaky, so this
|
||||
// pins the load-bearing structure instead: the early-death detector, the
|
||||
// 80%-of-timeout threshold that separates external kills from real wedges,
|
||||
// and the rescue-queue routing for both the wedged and non-wedged branches.
|
||||
it('detects early SIGTERM/SIGKILL deaths against the 80% timeout threshold', () => {
|
||||
const source = readFileSync(PARALLEL_SH_SRC, 'utf-8');
|
||||
expect(source).toContain('[ "$rc" = "143" ] || [ "$rc" = "137" ]');
|
||||
expect(source).toContain('$((SHARD_TIMEOUT * 80 / 100))');
|
||||
expect(source).toContain('shard_external_kill=1');
|
||||
});
|
||||
|
||||
it('routes externally-killed shards into the serial rescue queue, not the red path', () => {
|
||||
const source = readFileSync(PARALLEL_SH_SRC, 'utf-8');
|
||||
const killBranches = source.split('shard_external_kill" = "1"').length - 1;
|
||||
expect(killBranches).toBeGreaterThanOrEqual(2); // wedged + non-wedged branch
|
||||
expect(source).toContain('KILLED externally after ${s_elapsed}s');
|
||||
});
|
||||
|
||||
it('stamps per-shard start/end epochs so early death is measurable', () => {
|
||||
const source = readFileSync(PARALLEL_SH_SRC, 'utf-8');
|
||||
expect(source).toContain('date +%s > "$LOG_DIR/shard-$i.start"');
|
||||
expect(source).toContain('date +%s > "$LOG_DIR/shard-$i.end"');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -17,6 +17,7 @@ import { withoutAnthropicKey } from './helpers/no-anthropic-key.ts';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { dispatchToolCall } from '../src/mcp/dispatch.ts';
|
||||
import { TAKES_FENCE_BEGIN, TAKES_FENCE_END } from '../src/core/takes-fence.ts';
|
||||
import { operationsByName } from '../src/core/operations.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
let alicePageId: number;
|
||||
@@ -198,6 +199,11 @@ describe('per-token takes-holder allow-list — get_versions body channel', () =
|
||||
});
|
||||
|
||||
describe('think op — read-only on remote callers (Lane D landed)', () => {
|
||||
test('think is read-scoped for MCP while local persistence remains possible', () => {
|
||||
expect(operationsByName.think.scope).toBe('read');
|
||||
expect(operationsByName.think.mutating).toBe(true);
|
||||
});
|
||||
|
||||
test('remote save/take is forced read-only via remote_persisted_blocked flag', async () => {
|
||||
// Hermetic no-key: neutralize BOTH env var AND ~/.gbrain config key, else a
|
||||
// configured machine fires a real LLM call and the warning flips to
|
||||
|
||||
Reference in New Issue
Block a user