Compare commits

...
Author SHA1 Message Date
Garry TanandClaude Fable 5 92656a221b fix(admin): regenerate admin-embedded manifest for rebuilt SPA bundle
The Sources-tab rebuild replaced admin/dist/assets/index-CoGEje3-.js with
index-BpDk4NI4.js but src/admin-embedded.ts (generated by
scripts/build-admin-embedded.ts) still imported the deleted file, so
'gbrain serve --http' crashed on startup (Cannot find module) and all 4
admin-embed E2E serial tests failed with 'never became ready'.

Regenerated via: bun run scripts/build-admin-embedded.ts

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 11:01:15 -07:00
6ec762cbcd feat(admin): Sources tab + federation management UI
Takeover of #1601 (stacked on the #1592 takeover), rebased onto current
master. Adds /admin/api/sources (buildSyncStatusReport over the new
queryAdminSources helper — deliberately no local_path filter so push-only
brains still list sources) and four federated-read routes behind
requireAdmin that reuse the same grantReadCore/revokeReadCore/
setFederatedReadCore helpers as the CLI. Admin SPA gains a Sources page
+ per-client manage-reads UI; admin/dist rebuilt from current admin/src.
New test/admin-sources.test.ts pins the sources SQL (archived filter,
null-local_path inclusion, JSONB config shape).

Co-authored-by: bitak1 <bitak1@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 14:33:21 -07:00
4a81c017a0 feat(auth): grant-read / revoke-read / set-federated-read / list-clients (atomic SQL race-safe)
Takeover of #1592, rebased onto current master. Adds federated-read
management CLI: atomic array_append/array_remove with NOT-ANY +
deleted_at guards (closes the read-modify-write race), source-id
validation at boundaries, terminal-control sanitization for
DCR-registered client names, and 75 PGLite tests.

Co-authored-by: bitak1 <bitak1@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 14:28:25 -07:00
Masa 0612b0daa8 fix(dream): require self-contained opening summary in synthesized pages (#2770)
Synth pages written by dream synthesize currently open straight into
detail (quotes, cross-references) with no framing, so a reader who
lands on the page later — without the source transcript in front of
them — has no way to tell what it's about without reading the whole
thing.

Add OUTPUT POLICY item 5: every new page's body must open with a 2-3
sentence self-contained summary a reader unfamiliar with the source
conversation could understand on its own, before any quotes or detail.
2026-07-21 13:40:41 -07:00
maxpetrusenkoagentandmaxpetrusenkoagent <[REDACTED EMAIL]> f529eaa231 fix(jobs): refresh gateway config for queued AI work (#2125)
Long-lived minion workers can outlive DB-backed model config changes. Refresh the AI gateway before gateway-backed handlers run so queued cycle/propose_takes work does not fall back to a stale Anthropic default when the operator configured another provider.

Also record the active gateway chat model in propose_takes budget/proposal metadata instead of hardcoding claude-sonnet-4-6, and keep provider:model IDs intact for budget pricing.

Regression coverage verifies queued worker refresh, propose_takes model metadata, nested provider IDs, skipFence threading, and the updated autopilot signal source guard.

Co-authored-by: maxpetrusenkoagent <[REDACTED EMAIL]>
2026-07-21 13:21:51 -07:00
447e57ec41 fix(ai): tier-configured models reach the recipe allowlist — refresh Anthropic models, register tier resolutions, honest probe labels (#2800)
Setting `models.tier.deep anthropic:claude-opus-4-8` silently disabled
think and auto_think: the Anthropic recipe's chat allowlist stopped at
Opus 4.7, the tier-resolved model never joined the extended set that
assertTouchpoint's contract promises for config-chosen models, and the
resulting probe failure was stamped NO_ANTHROPIC_API_KEY — sending the
operator to debug env/keychain when the fix was the model id. Three
fixes, one per layer:

- recipes/anthropic.ts: add claude-fable-5, claude-opus-4-8, and
  claude-sonnet-5 to chat models; claude-sonnet-5 to expansion models.
- gateway.ts reconfigureGatewayWithEngine: resolve all four tiers and
  register the results as extended models, honoring the documented
  contract for models.default / models.tier.* (model-resolver.ts
  docstring). A tier-only model now validates like a chat/expansion one.
- think/index.ts: when the gateway client can't be built, re-probe and
  label honestly — MODEL_NOT_USABLE:<reason> for unknown_model /
  unknown_provider, NO_ANTHROPIC_API_KEY only for the actual missing-key
  case; the stub answer carries the probe detail and fix hint.

Tests: recipe-list presence pins; a new gateway-tier-extended-models
suite proving a fictional tier model validates post-reconfigure (and an
unconfigured one still doesn't); think-pipeline coverage for the honest
label (unknown_model beats missing-key even keyless); the existing
non-explicit bogus-provider test updated from the old catch-all label to
the honest one (no-throw contract unchanged).

Verified live: a brain with tier.deep=claude-opus-4-8 had think degrade
to gather-only with the misleading key warning; with this change the
probe passes and synthesis runs.

Co-authored-by: Paolo Belcastro <p3ob7o@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:06:32 -07:00
d61808d806 v0.42.64.0 fix: harden confidential OAuth token revocation (takeover of #3017) (#3032)
* fix(oauth): validate confidential revoke secrets

* fix(oauth): harden confidential token revocation

* chore: bump version and changelog (v0.42.64.0)

Co-Authored-By: OpenAI Codex <noreply@openai.com>

---------

Co-authored-by: Robin <rayme@boltdsolutions.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Garry Tan <garrytan@gmail.com>
2026-07-21 12:44:46 -07:00
60125ee626 feat(dream): --once for one-shot phase runs without toggling config gates (takeover of #2983) (#3031)
* feat(dream): --once for one-shot phase runs without toggling config gates

Fixes the "toggle enabled true, run, toggle back to false" workaround
that gbrain doctor's extract_atoms_backlog message implicitly
recommends and that #2860's reporter had to script around: with an
external orchestrator running `gbrain dream --phase patterns` on a
cadence outside the autopilot, the only way to run patterns once was
`config set dream.patterns.enabled true` -> run -> `config set ...
false`. A crash between steps left the flag stuck true, and the
autopilot (which polls the same flag) re-enqueued patterns every
cycle -- 119 LLM jobs / ~$400 over 24h before it was caught.

Root cause: `--phase X` only controls which phase FUNCTION cycle.ts
calls; it does not bypass that phase's own `dream.<phase>.enabled` /
`cycle.<phase>.enabled` config read. Each gated phase (patterns,
synthesize, conversation_facts_backfill, enrich_thin, skillopt) reads
its enabled flag internally and skips regardless of how the phase was
selected -- confirmed by reading each phase module, not assumed.
extract_atoms/synthesize_concepts are a DIFFERENT mechanism entirely
(pack-declaration via packDeclaresPhase, not a config .enabled read)
and already have a working one-shot escape hatch: `--drain`. The
existing doctor message for extract_atoms already says `--phase
extract_atoms --drain --window 120`, so no doctor text needed
updating there -- verified by reading src/commands/doctor.ts directly
rather than assuming the paraphrase in the issue was literal.

Design: `gbrain dream --phase <name> --once`. Requires an explicit
--phase (bare --once is a usage error, exit 2) so it can never
force-enable every disabled phase at once in a full/default cycle --
that would recreate the same unbounded-spend risk the flag exists to
prevent. Threaded through CycleOpts as `onceForPhase?: CyclePhase`
(the literal phase name, not a boolean) so the bypass can never leak
to a phase other than the one named, even if a future programmatic
caller passes a wider `phases` array than the CLI does. Never reads
or writes config -- the phase still evaluates its .enabled gate every
call; --once only overrides the boolean OUTCOME for that one
invocation, mirroring the existing --unsafe-bypass-dream-guard /
--input precedents (stderr warning at the bypass point, no new
config-touching code path).

Rejected alternatives (documented per task instructions):
- Making explicit --phase X always bypass .enabled: breaking change
  for existing crons that rely on the disabled flag as a cheap no-op;
  an upgrade would silently start running LLM/write phases.
- A new subcommand: adds a whole dispatch/help/arg surface that
  internally routes through the same override anyway.
- Extending --once to also bypass packDeclaresPhase for
  extract_atoms/synthesize_concepts: conflates two different gating
  mechanisms (config toggle vs. pack membership) under one flag;
  extract_atoms already has --drain, which is purpose-built for its
  batched/windowed execution model.

Design was cross-validated by an independent second-model review
(external design consultation) before implementation; its
recommendation to also update the extract_atoms doctor message to
`--once` was NOT adopted because that phase has no .enabled gate to
bypass -- doing so would be a documented no-op, contradicted by
reading src/commands/doctor.ts:3264 directly.

Tests: 9 new (structural CLI-flag wiring in dream-cli-flags.test.ts;
a real PGLite E2E test in dream-patterns-pglite.test.ts proving the
bypass fires AND that dream.patterns.enabled is never written; 4
runCycle-level tests in cycle.serial.test.ts proving onceForPhase
does not leak across phases). Verified 6 of 9 fail against the
pre-fix source (via git stash of source-only changes) to confirm
they're meaningful regressions, not tautologies.

Closes #2860

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(dream): --help short-circuits before --once usage validation

Codex review finding (P2): `gbrain dream --help --once` (no --phase)
called process.exit(2) from the new --once usage-error check inside
parseArgs before runDream's documented IRON RULE ("--help
short-circuits BEFORE any engine-bearing work") ever got a chance to
run -- parseArgs computes ALL its validations unconditionally before
runDream checks opts.help. Repo precedent for this ordering already
exists as a pinned regression test (test/dream.test.ts's "--help
--source whatever prints help and exits 0").

Fix: compute wantsHelp once in parseArgs and exempt the --once
validation when it's set, mirroring that precedent. Added the same
class of pinned tests here: bare `--once` still exits 2 with the
usage hint, `--help --once` prints help and exits 0, and a real
--phase patterns --once run against a PGLite engine proves the
bypass actually fires (falls through to insufficient_evidence
instead of disabled) without writing dream.patterns.enabled. Also
fixed the structural test in dream-cli-flags.test.ts that asserted
the exact pre-fix guard-condition source text.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(dream): --once must require an EXPLICIT --phase, not a derived one

Codex review finding (P3): the --once validation checked the derived
`phase` value, but `phase` gets defaulted implicitly by --input
(implies --phase synthesize) and --drain (implies --phase
extract_atoms) BEFORE that check ran. So `gbrain dream --input <f>
--once` and `gbrain dream --drain --once` both slipped past the
"explicit --phase required" contract silently -- and --once became a
true no-op in both cases: --drain returns from runDream before
onceForPhase is ever read (the drain path doesn't call runCycle at
all), and --input already bypasses the synthesize enabled-gate on its
own via the existing opts.inputFile check, so onceForPhase would
never even be consulted.

Fix: capture `phaseWasExplicit = phaseIdx !== -1` at the very top of
parseArgs, before the --input/--drain defaulting blocks run, and
validate --once against that instead of the derived `phase`. Updated
the usage-error message and --help text to say "an explicit --phase"
so a user hitting this understands why `--input ... --once` doesn't
count.

Tests: 2 new pins in test/dream.test.ts exercising runDream directly
(--input <file> --once exits 2; --drain --once exits 2), plus a
structural test in dream-cli-flags.test.ts pinning that
phaseWasExplicit is captured before both implicit-defaulting blocks.
Updated the two existing structural/behavioral tests whose literal
guard-condition / error-message assertions changed shape.

Verified: dream-cli-flags.test.ts 27/27, dream.test.ts 31/31,
typecheck clean.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: masashiono0611 <masashi.ono.0611@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Garry Tan <garrytan@gmail.com>
2026-07-21 12:29:35 -07:00
e320ad71b3 fix(providers): reuse buildGatewayConfig for --model test override (takeover of #2980) (#3029)
* fix(providers): reuse buildGatewayConfig for --model test override (#2863)

`gbrain providers test --model <id>` overrode the gateway with only
embedding_model/chat_model + env, dropping config.provider_base_urls
entirely. A brain configured with a custom endpoint (e.g. a China-region
DashScope base URL) would pass the bare `providers test` (which goes
through configureFromEnv() and does forward base_urls) but fail the
`--model`-scoped probe with a misleading "Incorrect API key" error, even
though the key was valid for the configured endpoint — the probe silently
fell back to the recipe's hardcoded default endpoint instead.

Root cause: two independent, drifted resolvers. The production path
(src/cli.ts#connectEngine, src/core/init-embed-check.ts) builds its
AIGatewayConfig via buildGatewayConfig(), which folds provider_base_urls,
env-sourced local-server base URLs, provider_chat_options, and file-plane
API keys. The --model override branch in runTest() hand-rolled a second,
narrower config object that only carried the overridden model + raw env.

Fix: lift `cfg` out of the existing try/catch (it was already loaded there
for the isolation-warning message) and spread `buildGatewayConfig(cfg)`
into both configureGateway() calls before overriding embedding_model/
chat_model. The isolated --model probe now resolves its endpoint exactly
the way the brain's real import/query path would; only the requested
model is overridden, so the probe still targets exactly the model the
user asked for. Falls back to bare env when no brain is configured yet
(cfg is null), matching prior first-time-install behavior.

Confirmed chat_fallback_chain (also threaded through by buildGatewayConfig)
has no runtime retry effect — it's only consumed to pre-register extended
model ids — so spreading the full production config does not mask an
isolated model's own failures behind a silent fallback.

Other diagnostic surfaces (providers list/env/explain) were checked and
are unaffected: `runProviders()` already calls configureFromEnv() (which
forwards base_urls correctly) before dispatch, and none of them accept
--model, so they never hit the broken override branch.

Adds test/providers-test-model-base-url.test.ts: drives runProviders('test',
...) end-to-end against a mocked fetch + temp GBRAIN_HOME/config.json with
provider_base_urls set for the dashscope recipe (the exact recipe named in
the bug report), asserting the outbound request hits the configured base
URL rather than the recipe default. Verified red on pre-fix code via
git stash, green after.

Closes #2863

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix: drop duplicate buildGatewayConfig import after master merge

Master's f3e78fd2 added the same import the PR carried; the textual
merge was clean but the result failed typecheck (TS2300 duplicate
identifier).

Co-authored-by: masashiono0611 <masashi.ono.0611@gmail.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: masashiono0611 <masashi.ono.0611@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Garry Tan <garrytan@gmail.com>
2026-07-21 12:17:18 -07:00
b6dd3e1121 fix(test): reset AI gateway after adaptive-embed-batch suite — cross-file config leak (#3065)
The file's final test configures the gateway with a remote provider and a
fake key, and its afterEach only clears the mock transport. With no
afterAll, the poisoned global config survives the file boundary; the next
test file in the shard that triggers an embed makes a real HTTP call and
fails. Surfaced on master when #3022's new test file reshuffled shard
composition (shard 6: synthesize-concepts-progress failed twice with a
live Google embed rejection). The legacy-embedding preload can't catch
this: it only re-applies defaults when the gateway slot is empty.

One-line root-cause fix at the leaker. A repo-wide guard for the class
(~70 files call configureGateway without a final reset) is filed as a
follow-up.

Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 12:05:31 -07:00
3cc34c92ee feat(ai): add NVIDIA NIM provider recipe (#2965) (#3022)
Adds NVIDIA NIM / API Catalog as a first-class OpenAI-compatible AI recipe:

- chat via nvidia/nemotron-3-super-120b-a12b (conservative capability
  claims: no tools, no subagent loop until proven)
- hosted embedding models incl. nvidia/llama-nemotron-embed-1b-v2 with
  Matryoshka-style dimension overrides (1024/1280/1536/2048) and fixed
  natural dims for the other catalog models
- asymmetric input_type mapping (document -> passage, query -> query) via a
  gateway compat fetch shim, since the generic openai-compatible recipe
  cannot infer that provider-specific requirement
- base URL https://integrate.api.nvidia.com/v1 verified live (OpenAI-shaped
  /v1/models, all five recipe model ids present in the catalog)

Changed from the original PR: dropped the recipe's custom resolveAuth — it
duplicated defaultResolveAuth's Authorization-Bearer behavior exactly and
violated the IRON RULE that only Azure overrides resolveAuth
(test/ai/recipes-existing-regression.test.ts). NVIDIA_API_KEY now flows
through defaultResolveAuth via auth_env.required, and the recipe test pins
resolveAuth === undefined + the default Bearer resolution + the missing-key
AIConfigError. Also scrubbed a private downstream-agent name from ported
comments per the repo privacy rule.

Takeover of #2965 by @ravehorn.

Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: SAGE Codex <codex@sage.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 00:17:26 -07:00
Anton Senkovskiy a93fcf504f fix(chronicle): bound last-seen to <= asof/today so future events do not read as "seen today" (#2993)
getLastSeen had no upper date bound, so a future-dated chronicle event (a
scheduled calendar-event, a planned milestone) became the entity's "last
seen" date; finalizeLastSeen's Math.max(0, ...) then clamped the negative
delta, reporting days_ago: 0 -- the entity reads as seen-today. Recording
future events is intended (eligibility ELIGIBLE_TYPES includes calendar-event);
the reader just needs to stop counting them as "seen".

Bound the query to te.date <= COALESCE(asof, current_date) in both engines,
mirroring getOnThisDay's existing te.date < target bound. asof now reaches
the WHERE clause (previously it only reached finalizeLastSeen), so as-of
time-travel is honored for the date filter too.

Regression test added: an entity with past events plus a future event -> last
seen returns the most recent PAST event, not the future one; and as-of after
the future date lets it through. Fails before, passes after.
2026-07-21 00:04:41 -07:00
Hanchen Qiu 84fad4738d fix(pages): default chunker_version to MARKDOWN_CHUNKER_VERSION on INSERT (#2807) (#2988)
putPage's INSERT used COALESCE(<chunkerVersion>, 1), so callers that don't
supply chunker_version (no MCP/subagent caller does — it's internal metadata)
landed new pages at version 1. Dream subagents write through putPage directly,
so their pages got v1 and doctor's contextual_retrieval_coverage check flagged
them as "older chunker_version" forever, even though they were chunked and
embedded with the current chunker.

Default the INSERT to MARKDOWN_CHUNKER_VERSION on both engines. The ON CONFLICT
UPDATE still COALESCE-preserves an explicitly supplied version. Add an
engine-level regression test.
2026-07-20 23:47:09 -07:00
Hanchen Qiu f815246eef fix(cli): register reconcile-links in CLI_ONLY so dispatch reaches its handler (#2900) (#2987)
reconcile-links is advertised in `gbrain --help` and implemented with a
`case 'reconcile-links'` block in handleCliOnly, but it was missing from the
CLI_ONLY Set. Dispatch only enters handleCliOnly when the command is in
CLI_ONLY, so every invocation fell through to the shared-operations lookup and
hit the generic "Unknown command" branch — leaving the documented doc↔impl
edge-rebuild tool silently unreachable via the CLI.

Add 'reconcile-links' to CLI_ONLY, plus a reachability regression test
mirroring the #2035 (`calibration`) guard.
2026-07-20 23:23:03 -07:00
Vishnu JandClaude Fable 5 42c4ea929f fix(test): isolate audit writes to a per-run scratch dir in the shared bootstrap (#2823) (#2966)
The content-sanity gate's audit logger (logContentSanityAssessment)
defaults, via audit-writer.ts::resolveAuditDir(), to writing
~/.gbrain/audit/content-sanity-YYYY-Www.jsonl on disk. A GBRAIN_AUDIT_DIR
env override exists, but nothing in the shared test bootstrap ever set
it, so any test that exercised an audit-emitting code path without
wrapping the call in its own withEnv() fell through to the operator's
real audit trail. test/import-file.test.ts's oversize-boundary fixture
('borderline-slug', content just under MAX_FILE_SIZE but over
DEFAULT_BYTES_BLOCK) fired a real soft_block event into the developer's
live ~/.gbrain/audit on every run — which doctor's
content_sanity_audit_recent check then reported as production signal.

Fix: add a bootstrap preload (test/helpers/audit-dir-preload.ts, wired
via bunfig.toml) that points GBRAIN_AUDIT_DIR at a fresh per-process
mkdtemp dir before any test file loads. Each run-unit-shard.sh shard is
its own bun process, so each shard gets its own scratch dir with no
cross-shard collision. This closes the leak for every audit-emitting
test, not just this fixture. It respects a developer-exported override
(only sets the var when unset), and files that manage their own
per-test GBRAIN_AUDIT_DIR via withEnv() are unaffected.

Also fix a latent isolation bug this surfaced: gbrain-home-isolation.test.ts
unconditionally deleted GBRAIN_AUDIT_DIR in a finally block instead of
restoring the prior value, which clobbered the bootstrap's scratch dir
for every test file that ran after it in the same shard process.

Adds test/audit/audit-dir-preload.test.ts to pin the behavior: it
reproduces the exact soft_block event shape and asserts it lands in the
scratch dir, never in ~/.gbrain/audit.

Reported by @paul-0320.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 23:07:58 -07:00
Benjamin D. Smith 6370ce3d7e fix(infrastructure): chmod 644 autopilot supervisor files on install (#2963)
📝 Summary:
• launchd rejects group/world-writable agent plists — when the installer
  runs under a umask-0 parent shell, `writeFileSync(plistPath(), plist)`
  produces a 0666 plist that makes `launchctl load`/`bootstrap` fail with
  the opaque `Bootstrap failed: 5: Input/output error` and the login-time
  LaunchAgents scan skip the file silently
• on an affected machine the daemon never registers while everything
  looks installed — the plist exists, launchd's disabled-table says
  enabled, and no log file is ever created

🔧 Technical Improvements:
• `installLaunchd`: write plist with `{ mode: 0o644 }` AND
  `chmodSync(0o644)` — writeFileSync mode applies only on create, so a
  reinstall over an existing 0666 plist must normalize explicitly
• `installSystemd`: same hardening on the unit file (systemd warns on
  world-writable units); symmetric with the launchd path
• Restart-policy rewrite path (`generateSystemdUnit` rewrite of an
  existing unit): chmod is load-bearing here — the file always exists,
  so the write mode never applies
• `chmodSync` added to the fs import

📊 Code Changes: 18 insertions, 4 deletions (net +14)

📦 Files Modified:
• src/commands/autopilot.ts (minor updates) — mode + chmod on the three
  supervisor-file writers; comments carry the launchd failure signature
  so the next EIO hunt greps straight to it
2026-07-20 22:48:16 -07:00
levineam c21d7b253a fix(doctor): derive host skill manifests (SUP-3488) (#2961) 2026-07-20 22:38:25 -07:00
zsimovanforgeopsandForge 4df7796061 Preserve modality and symbol metadata in embed-stale merge (#2969)
embedStaleForSource rebuilds each page's chunks as a merged ChunkInput[]
carrying only five fields (chunk_index, chunk_text, chunk_source,
embedding, token_count), while upsertChunks writes the metadata columns
as EXCLUDED.<col>. Any page containing at least one stale chunk
therefore has ALL its chunks' metadata reset on the next embed-stale
pass:

- image chunks flip modality 'image' -> 'text' and disappear from the
  cross-modal image search arm permanently (it filters
  modality='image'), while keeping their embedding_image vector — the
  data looks intact but is unreachable;
- code chunks lose language, symbol_name, symbol_type,
  symbol_name_qualified.

The read side compounds this: rowToChunk never returned modality, so a
correct merge was impossible without also extending the Chunk shape.

Fix: expose modality on Chunk/rowToChunk and carry modality, language,
and the symbol fields through the merge. embedding_image is deliberately
not carried — upsertChunks already COALESCEs it server-side.

Repair for affected brains: UPDATE content_chunks SET modality='image'
WHERE chunk_source='image_asset' AND embedding_image IS NOT NULL.

The new test seeds a mixed page (settled image chunk + stale text chunk
with symbol metadata) and asserts both survive an embedStaleForSource
pass; it fails on master.

Co-authored-by: Forge (Ron) <forge@zsimovan.dev>
2026-07-20 22:14:46 -07:00
zsimovanforgeopsandForge 6db4cea2e4 Resolve relative storage paths in doctor image_assets check (#2971)
The image_assets check statSyncs files.storage_path directly, but
sync-ingested assets store repo-relative paths. Run doctor from any
directory other than the brain repo and every image is reported
'missing from disk' — a persistent false WARN with a suggested fix
(gbrain sync --skip-failed) that does nothing.

Resolve relative paths against sync.repo_path before statting; absolute
paths are untouched. Falls back to cwd when the config key is unset,
preserving the old behavior for brains without a configured repo.

Co-authored-by: Forge (Ron) <forge@zsimovan.dev>
2026-07-20 21:59:24 -07:00
sameerbopardikarandSameer Bopardikar 11eebc3605 fix(conversation): extract iMessage facts with real timestamps (#2756) (#2958)
Co-authored-by: Sameer Bopardikar <203024074+sameerbopardikar@users.noreply.github.com>
2026-07-20 16:57:53 -07:00
Eddie AshandEddie Ash ee45653a02 fix: initialize foreground chat gateway (#2590) (#3003)
Co-authored-by: Eddie Ash <119880+cazador481@users.noreply.github.com>
2026-07-20 16:47:12 -07:00
zsimovanforgeopsandForge 706d3cea3d Scope maxWaiting backpressure by data.sourceId (#2970)
The submission-time backpressure cap counts all waiting (name, queue)
rows regardless of which source a job targets. On a multi-source brain
this makes per-source submissions with maxWaiting: 1 mutually exclusive:
while one source's sync sits waiting, every other source's freshness
sync coalesces into that row and never runs. The dispatch log shows the
starved source 'dispatched' each interval (queue.add returns the other
source's waiting row), so the starvation is invisible unless you notice
sources.last_sync_at falling behind — we found a secondary source 29
hours stale on a 5-minute freshness interval.

Fix: when the submitted data carries a string sourceId, key the advisory
lock, the waiting count, and the coalesce target on it. Submissions
without sourceId keep the existing single-scope behavior, so
single-source brains and non-sync jobs are unchanged.

The new test asserts same-source submissions still coalesce while a
different source gets its own row and its own cap; it fails on master.

Co-authored-by: Forge (Ron) <forge@zsimovan.dev>
2026-07-20 16:38:03 -07:00
MasaandClaude Sonnet 5 2934c53c1d fix(sources): validate --path is a git repo with committed content at registration (#2707) (#2975)
* fix(sources): validate --path is a git repo at registration time (#2707)

`sources add --path <dir>` accepted any existing non-git directory with
zero validation, deferring the failure to the first `gbrain sync` ("Not
inside a git repository: ..."). By the time that surfaces, the source
has been silently stale for however long nobody read the sync logs.

Add a registration-time check (git-remote.ts:isInsideGitRepo, mirroring
sync.ts's discoverGitRoot walk-up so subdir-of-git-repo sources still
pass) that rejects an existing-but-non-git --path directory with an
actionable error pointing at `git init && git add -A && git commit`.
Non-existent paths are unaffected (out of scope — different, pre-existing
failure mode) and `--force` opts out for callers who want to register
before git-init exists.

This is registration-time validation ONLY — it never auto-`git init`s
the directory, preserving the consent boundary #2967 established for
sync-time self-heal (a --path source is the user's own external
directory; gbrain must not mutate it without explicit ask).

Also documents the git requirement (docs/guides/multi-source-brains.md),
including the "files must be committed, not just present" gotcha and
that a stale/unreachable sync anchor already self-heals on plain
`gbrain sync` (verified manually against HEAD — no reset-anchor command
needed).

* fix(sources): require a committed HEAD + shell-quote remediation cmd (codex round 1)

Codex review round 1 on #2707 found two real gaps:

1. isInsideGitRepo alone accepts a `git init`ed-but-never-committed
   directory (rev-parse --show-toplevel succeeds with no HEAD), so
   registration would still pass a source that fails sync's own
   "No commits in repo ... Make at least one commit before syncing."
   Add hasGitCommits (git rev-parse HEAD) as a second required check.

2. The remediation command in the error message interpolated the raw
   path unquoted — spaces, $(), backticks, etc. would break or, worse,
   execute unintended shell syntax if pasted. POSIX single-quote it
   (mirrors src/commands/connect.ts:shellQuote; duplicated locally
   rather than imported, since commands/ depends on core/ not the
   reverse).

* fix(sources): require tracked content in HEAD, not just a resolvable HEAD (codex round 2)

Codex review round 2 P1: hasGitCommits (rev-parse HEAD) accepted a repo
with an empty commit (git commit --allow-empty) followed by untracked
files — HEAD resolves fine (to git's well-known empty-tree object), so
registration passed, but the first sync would "succeed" importing
nothing and then silently never notice the untracked files change. The
exact same gap applied to an untracked subdirectory of an otherwise-
real git repo (monorepo case).

Replace hasGitCommits with hasTrackedContent (`git ls-tree HEAD -- .`,
non-recursive — one entry is enough, no need to walk the whole
subtree). `-C path` + pathspec `.` scopes correctly to both a repo
toplevel and a subdirectory-of-a-repo source, and an empty tree lists
zero entries where a bare `rev-parse HEAD` would still succeed. Also
subsumes the "no commits at all" case hasGitCommits covered (ls-tree
on an unborn repo fails the same way), so this is one check instead
of two.

Updated the error copy and docs/guides/multi-source-brains.md to match
what's actually verified now.

* fix(sources): O(1)-output tree-emptiness probe, avoid maxBuffer overflow (codex round 3)

Codex review round 3 found the round-2 `git ls-tree HEAD -- .` listing
buffers the whole (non-recursive) tree — a real repo with ~17-20K
directly-tracked entries exceeds execFileSync's default 1 MiB
maxBuffer, throws ENOBUFS, and the catch-all incorrectly rejects a
perfectly valid registration.

Replace the listing with `git rev-parse --verify HEAD:./` (resolves
the tree object for `path` specifically, correct for both toplevel and
subdirectory sources same as before) compared against git's canonical
empty-tree SHA-1 (4b825dc6...) — a fixed ~40-byte read regardless of
how many entries the tree has, structurally immune to this class of
bug rather than just raising the threshold. Added a 300-file
regression test locking this in.

Declined a second round-3 finding (P1: reject a tree if ANY untracked
file exists anywhere under the path, not just when the tree is
entirely empty) — untracked files never being synced is standard,
existing git-source behavior throughout this codebase (identical for
--url managed clones), not a bug specific to this validation. Enforcing
zero-untracked-files at registration would reject ordinary repos with
gitignored build output, .DS_Store, editor swapfiles, etc. Out of
scope relative to what #2707 actually asks for (a directory with real,
committed content that will sync) and how every other git source in
this system already behaves.

* fix(sources): derive empty-tree OID per repo instead of hardcoding SHA-1 (codex round 4)

Codex review round 4 P2, confirmed by directly testing against a
`git init --object-format=sha256` repo: the hardcoded SHA-1 empty-tree
constant only matches SHA-1 repositories. An empty SHA-256 repo's real
empty-tree OID is a different (64-char) hash, so the SHA-1 comparison
silently mismatched and let an empty/untracked SHA-256 source through
— exactly the case this validation exists to catch.

Replace the constant with `emptyTreeOid()`: `git hash-object -t tree
--stdin < /dev/null` computed in the target repo's own context, so it
returns the correct empty-tree OID for whichever object format that
repo actually uses, without gbrain needing to know or care which one.
Added gated regression tests (git 2.29+ / --object-format=sha256,
test.skipIf on older git) for both the empty-repo-rejected and
real-content-registers-fine cases.

Converging here (4 review rounds; this is the last outstanding
finding from round 4, and round 4 raised only this one issue).

* fix(test): --force the incidental non-git second source in #1434 routing test (#2707)

CI caught a real regression from this PR's registration-time git
validation: test/sync-sole-non-default-routing.test.ts's "2+ non-default
sources" case registers a bare mkdtempSync temp dir (no git init) as a
second source purely to have 2 sources present — the directory's content
was never meant to be exercised, only its existence as a distinct
local_path. #2707's new validation correctly rejects that dir at
registration time, since nothing else in the test suite told it
otherwise.

--force is the right fix, not adding unnecessary git-init/commit
boilerplate to secondRepo: it documents that this specific registration
intentionally doesn't care about git-validity, matching what a real
caller opting into the legacy lenient behavior would do.

Verified: the specific test (3/3 pass), plus every other test file in
the repo using `sources add --path` (sources.test.ts, sources-ops.test.ts
already covered by the PR's own commits; repos-alias.test.ts,
sync-cost-gate.serial.test.ts — 11/11 pass, no similar fixture gap).
typecheck clean, verify 31/31.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-20 16:28:53 -07:00
b60656245f fix(migrate): v124 notice to stderr — the stdout-cleanliness guard caught it on master (#3035)
Same class as #3019's v123 fix; the guard test added there flagged this
within one push. Route the notice through process.stderr.write.

Co-authored-by: Garry Tan <garrytan@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 16:19:40 -07:00
90 changed files with 4649 additions and 189 deletions
+7
View File
@@ -2,6 +2,13 @@
All notable changes to GBrain will be documented in this file.
## [0.42.64.0] - 2026-07-20
### Fixed
- Confidential OAuth clients can now revoke access tokens through the standard revocation endpoint when client secrets are stored as hashes. Invalid credentials fail closed, malformed or mixed authentication is rejected, backend failures remain retryable, and discovery metadata accurately advertises supported authentication methods.
No schema migrations.
## [0.42.63.0] - 2026-07-20
**Schema commands now open the local brain you actually configured.**
+1 -1
View File
@@ -1 +1 @@
0.42.63.0
0.42.64.0
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+1 -1
View File
@@ -7,7 +7,7 @@
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600&family=JetBrains+Mono:wght@400;500&display=swap" rel="stylesheet" />
<script type="module" crossorigin src="/admin/assets/index-CoGEje3-.js"></script>
<script type="module" crossorigin src="/admin/assets/index-BpDk4NI4.js"></script>
<link rel="stylesheet" crossorigin href="/admin/assets/index-GxkWX7v3.css">
</head>
<body>
+6 -2
View File
@@ -5,13 +5,14 @@ import { AgentsPage } from './pages/Agents';
import { RequestLogPage } from './pages/RequestLog';
import { CalibrationPage } from './pages/Calibration';
import { JobsWatchPage } from './pages/JobsWatch';
import { SourcesPage } from './pages/Sources';
import { api } from './api';
type Page = 'login' | 'dashboard' | 'agents' | 'log' | 'calibration' | 'jobs';
type Page = 'login' | 'dashboard' | 'agents' | 'sources' | 'log' | 'calibration' | 'jobs';
function getPage(): Page {
const hash = window.location.hash.replace('#', '') || 'dashboard';
if (['login', 'dashboard', 'agents', 'log', 'calibration', 'jobs'].includes(hash)) return hash as Page;
if (['login', 'dashboard', 'agents', 'sources', 'log', 'calibration', 'jobs'].includes(hash)) return hash as Page;
return 'dashboard';
}
@@ -54,6 +55,8 @@ export function App() {
onClick={() => navigate('dashboard')}>Dashboard</a>
<a className={`nav-item ${page === 'agents' ? 'active' : ''}`}
onClick={() => navigate('agents')}>Agents</a>
<a className={`nav-item ${page === 'sources' ? 'active' : ''}`}
onClick={() => navigate('sources')}>Sources</a>
<a className={`nav-item ${page === 'log' ? 'active' : ''}`}
onClick={() => navigate('log')}>Request Log</a>
<a className={`nav-item ${page === 'calibration' ? 'active' : ''}`}
@@ -83,6 +86,7 @@ export function App() {
<main className="main">
{page === 'dashboard' && <DashboardPage />}
{page === 'agents' && <AgentsPage />}
{page === 'sources' && <SourcesPage />}
{page === 'log' && <RequestLogPage />}
{page === 'calibration' && <CalibrationPage />}
{page === 'jobs' && <JobsWatchPage />}
+18
View File
@@ -52,4 +52,22 @@ export const api = {
apiFetchText(`/admin/api/calibration/charts/${encodeURIComponent(type)}${holder ? `?holder=${encodeURIComponent(holder)}` : ''}`),
// v0.41 D2 — live minion-jobs dashboard snapshot.
jobsWatch: () => apiFetch('/admin/api/jobs/watch'),
// v0.41.29 Sources tab + federated-read management
sources: () => apiFetch('/admin/api/sources'),
agentsFederatedRead: () => apiFetch('/admin/api/agents/federated-read'),
grantRead: (clientId: string, sourceId: string) =>
apiFetch(`/admin/api/agents/${encodeURIComponent(clientId)}/grant-read`, {
method: 'POST',
body: JSON.stringify({ source_id: sourceId }),
}),
revokeRead: (clientId: string, sourceId: string) =>
apiFetch(`/admin/api/agents/${encodeURIComponent(clientId)}/revoke-read`, {
method: 'POST',
body: JSON.stringify({ source_id: sourceId }),
}),
setFederatedRead: (clientId: string, sourceIds: string[]) =>
apiFetch(`/admin/api/agents/${encodeURIComponent(clientId)}/set-federated-read`, {
method: 'POST',
body: JSON.stringify({ source_ids: sourceIds }),
}),
};
+196
View File
@@ -381,8 +381,16 @@ function CredentialsModal({ credentials, onClose }: {
);
}
interface FederationState {
source_id: string | null;
federated_read: string[];
}
function AgentDrawer({ agent, onClose, onRevoked }: { agent: Agent; onClose: () => void; onRevoked: () => void }) {
const [tab, setTab] = useState<'claude-code' | 'chatgpt' | 'claude-cowork' | 'perplexity' | 'cursor' | 'json'>('claude-code');
const [federation, setFederation] = useState<FederationState | null>(null);
const [allSources, setAllSources] = useState<string[]>([]);
const [showFederation, setShowFederation] = useState(false);
const copy = (text: string) => navigator.clipboard.writeText(text);
const serverUrl = window.location.origin;
@@ -390,6 +398,30 @@ function AgentDrawer({ agent, onClose, onRevoked }: { agent: Agent; onClose: ()
const isOAuth = agent.auth_type === 'oauth';
const agentName = agent.name || agent.client_name || 'unknown';
// Lazy-load federation state when the drawer opens for an OAuth client.
// The /admin/api/agents endpoint doesn't carry source_id / federated_read,
// so we fetch /admin/api/agents/federated-read separately and pair by id.
useEffect(() => {
if (!isOAuth || !cid) return;
let cancelled = false;
Promise.all([
api.agentsFederatedRead().catch(() => ({ clients: [] })),
api.sources().catch(() => ({ sources: [] })),
]).then(([feds, srcs]: any) => {
if (cancelled) return;
const me = (feds.clients || []).find((c: any) => c.client_id === cid);
setFederation(me ? { source_id: me.source_id, federated_read: me.federated_read || [] } : null);
setAllSources((srcs.sources || []).map((s: any) => s.source_id));
});
return () => { cancelled = true; };
}, [cid, isOAuth]);
const reloadFederation = async () => {
const feds: any = await api.agentsFederatedRead().catch(() => ({ clients: [] }));
const me = (feds.clients || []).find((c: any) => c.client_id === cid);
setFederation(me ? { source_id: me.source_id, federated_read: me.federated_read || [] } : null);
};
// For API keys, we can't show the actual token (it was shown once at creation).
// For OAuth, we show the client_id and tell them to use their secret.
@@ -553,6 +585,34 @@ function AgentDrawer({ agent, onClose, onRevoked }: { agent: Agent; onClose: ()
<span>{agent.token_ttl ? (agent.token_ttl >= 31536000 ? 'No expiry' : agent.token_ttl >= 86400 ? `${Math.floor(agent.token_ttl / 86400)}d` : agent.token_ttl >= 3600 ? `${Math.floor(agent.token_ttl / 3600)}h` : `${agent.token_ttl}s`) : '1h (default)'}</span>
</div>
{isOAuth && federation && (
<>
<div className="section-title" style={{ display: 'flex', alignItems: 'center', justifyContent: 'space-between' }}>
<span>Federation</span>
<button
className="btn btn-secondary"
style={{ padding: '4px 10px', fontSize: 12 }}
onClick={() => setShowFederation(true)}
>
Manage reads
</button>
</div>
<div style={{ display: 'grid', gridTemplateColumns: '120px 1fr', gap: '6px 12px', fontSize: 13 }}>
<span style={{ color: 'var(--text-secondary)' }}>Write source</span>
<span className="mono">{federation.source_id || '(none)'}</span>
<span style={{ color: 'var(--text-secondary)' }}>Federated reads</span>
<span style={{ fontSize: 12 }}>
{federation.federated_read.length === 0
? <span style={{ color: 'var(--text-muted)' }}>(empty no federated reads)</span>
: federation.federated_read.map((s) => (
<span key={s} className="badge badge-read" style={{ marginRight: 4, marginBottom: 2 }}>{s}</span>
))
}
</span>
</div>
</>
)}
{/*
Config Export visible for both auth_type=oauth AND auth_type=api_key.
Claude Code + Cursor + JSON tabs render real snippets regardless
@@ -628,6 +688,142 @@ function AgentDrawer({ agent, onClose, onRevoked }: { agent: Agent; onClose: ()
)}
</div>
</div>
{showFederation && federation && (
<FederationModal
clientId={cid}
clientName={agentName}
allSources={allSources}
currentReads={federation.federated_read}
writeSource={federation.source_id}
onClose={() => setShowFederation(false)}
onSaved={async () => {
await reloadFederation();
setShowFederation(false);
}}
/>
)}
</>
);
}
/**
* FederationModal admin counterpart of `gbrain auth set-federated-read`.
* Source checkbox list; "Save" submits the full new list via the
* race-safe atomic SQL path in setFederatedReadCore. Per the CLI's
* documented contract, this is wholesale-replace semantics concurrent
* grant/revoke from a CLI operator would be last-writer-wins against
* a Save here.
*/
function FederationModal({
clientId, clientName, allSources, currentReads, writeSource, onClose, onSaved,
}: {
clientId: string;
clientName: string;
allSources: string[];
currentReads: string[];
writeSource: string | null;
onClose: () => void;
onSaved: () => Promise<void> | void;
}) {
const [selected, setSelected] = useState<Set<string>>(new Set(currentReads));
const [saving, setSaving] = useState(false);
const [error, setError] = useState<string | null>(null);
const toggle = (id: string) => {
const next = new Set(selected);
if (next.has(id)) next.delete(id); else next.add(id);
setSelected(next);
};
const handleSave = async () => {
setSaving(true);
setError(null);
try {
await api.setFederatedRead(clientId, Array.from(selected));
await onSaved();
} catch (e: any) {
setError(e.message || 'save failed');
setSaving(false);
}
};
// Union: all known sources + any current reads not in the source list
// (e.g. orphan entries from before the source was deleted). The latter
// surface as "(missing source)" so operators can revoke them.
const allKnown = new Set([...allSources, ...currentReads]);
const ordered = Array.from(allKnown).sort();
return (
<div className="modal-overlay" onClick={onClose}>
<div className="modal" onClick={(e) => e.stopPropagation()} style={{ maxWidth: 520 }}>
<div className="modal-header">
<div style={{ fontSize: 16, fontWeight: 600 }}>Manage federated reads</div>
<div style={{ fontSize: 13, color: 'var(--text-secondary)', marginTop: 4 }}>
<strong>{clientName}</strong> pick which sources this client can read in addition to its
{writeSource ? <> write source <code className="mono">{writeSource}</code></> : <> write source</>}.
</div>
</div>
<div className="modal-body" style={{ maxHeight: '50vh', overflowY: 'auto' }}>
{ordered.length === 0 && (
<div style={{ color: 'var(--text-muted)', fontSize: 13 }}>
No sources registered. Use <code>gbrain sources add &lt;id&gt; --path &lt;dir&gt;</code> from the CLI first.
</div>
)}
{ordered.map((id) => {
const isOrphan = !allSources.includes(id);
const isWriteSource = id === writeSource;
return (
<label
key={id}
style={{
display: 'flex',
alignItems: 'center',
gap: 10,
padding: '8px 10px',
borderBottom: '1px solid var(--border)',
cursor: 'pointer',
fontSize: 13,
}}
>
<input
type="checkbox"
checked={selected.has(id)}
onChange={() => toggle(id)}
style={{ width: 16, height: 16, margin: 0, flexShrink: 0, cursor: 'pointer' }}
/>
<span className="mono" style={{ flex: 1, minWidth: 0, overflow: 'hidden', textOverflow: 'ellipsis', whiteSpace: 'nowrap' }}>{id}</span>
{isWriteSource && <span className="badge badge-write" style={{ fontSize: 10, flexShrink: 0 }}>write source</span>}
{isOrphan && <span className="badge badge-danger" style={{ fontSize: 10, flexShrink: 0 }}>missing source</span>}
</label>
);
})}
</div>
{error && (
<div style={{
background: 'rgba(239,68,68,0.08)',
border: '1px solid rgba(239,68,68,0.3)',
color: '#ef4444',
padding: '10px 12px',
borderRadius: 6,
margin: '12px 0',
fontSize: 12,
}}>
{error}
</div>
)}
<div className="modal-footer">
<button type="button" className="btn btn-secondary" onClick={onClose} disabled={saving}>Cancel</button>
<button
type="button"
className="btn btn-primary"
onClick={handleSave}
disabled={saving}
>
{saving ? 'Saving…' : `Save (${selected.size} source${selected.size === 1 ? '' : 's'})`}
</button>
</div>
</div>
</div>
);
}
+195
View File
@@ -0,0 +1,195 @@
import React, { useState, useEffect } from 'react';
import { api } from '../api';
interface SourceRow {
source_id: string;
name: string;
local_path: string | null;
sync_enabled: boolean;
last_sync_at: string | null;
staleness_hours: number | null;
staleness_class: 'fresh' | 'stale' | 'severe' | 'unknown';
last_commit: string | null;
pages: number;
chunks_total: number;
chunks_unembedded: number;
embedding_coverage_pct: number;
}
interface FederatedClient {
client_id: string;
client_name: string;
source_id: string | null;
federated_read: string[];
}
function timeAgo(iso: string | null): string {
if (!iso) return 'never';
const s = Math.floor((Date.now() - new Date(iso).getTime()) / 1000);
if (s < 0) return 'in the future?';
if (s < 60) return 'just now';
if (s < 3600) return `${Math.floor(s / 60)}m ago`;
if (s < 86400) return `${Math.floor(s / 3600)}h ago`;
return `${Math.floor(s / 86400)}d ago`;
}
function stalenessColor(cls: string): string {
switch (cls) {
case 'fresh': return '#4ade80';
case 'stale': return '#fbbf24';
case 'severe': return '#ef4444';
default: return 'var(--text-muted)';
}
}
function coverageColor(pct: number): string {
if (pct >= 99) return '#4ade80';
if (pct >= 90) return '#fbbf24';
return '#ef4444';
}
export function SourcesPage() {
const [sources, setSources] = useState<SourceRow[]>([]);
const [clients, setClients] = useState<FederatedClient[]>([]);
const [loading, setLoading] = useState(true);
const [error, setError] = useState<string | null>(null);
const load = async () => {
setLoading(true);
setError(null);
try {
const [srcReport, clientsResp] = await Promise.all([
api.sources(),
api.agentsFederatedRead(),
]);
setSources(srcReport.sources || []);
setClients(clientsResp.clients || []);
} catch (e: any) {
setError(e.message || 'load failed');
} finally {
setLoading(false);
}
};
useEffect(() => { load(); }, []);
// Reverse-lookup: for each source, which clients can read it?
const readersBySource = (sourceId: string): string[] =>
clients.filter((c) => c.federated_read.includes(sourceId)).map((c) => c.client_name);
// Reverse-lookup: which clients WRITE to this source (source_id == sourceId)?
const writersBySource = (sourceId: string): string[] =>
clients.filter((c) => c.source_id === sourceId).map((c) => c.client_name);
return (
<div style={{ padding: 24, maxWidth: 1200 }}>
<div style={{ display: 'flex', alignItems: 'center', justifyContent: 'space-between', marginBottom: 24 }}>
<h1 style={{ fontSize: 24, margin: 0 }}>Sources</h1>
<button
onClick={load}
style={{
background: 'transparent',
border: '1px solid var(--border)',
color: 'var(--text-secondary)',
padding: '6px 12px',
borderRadius: 6,
fontSize: 12,
cursor: 'pointer',
}}
>
Refresh
</button>
</div>
{loading && <div style={{ color: 'var(--text-muted)' }}>Loading</div>}
{error && (
<div style={{
background: 'rgba(239,68,68,0.08)',
border: '1px solid rgba(239,68,68,0.3)',
color: '#ef4444',
padding: 12,
borderRadius: 6,
marginBottom: 16,
fontSize: 13,
}}>
Failed to load sources: {error}
</div>
)}
{!loading && !error && sources.length === 0 && (
<div style={{ color: 'var(--text-muted)', padding: 16 }}>
No active sources with a local_path. Use{' '}
<code style={{ background: 'var(--bg-elevated)', padding: '2px 6px', borderRadius: 4 }}>
gbrain sources add &lt;id&gt; --path &lt;dir&gt;
</code>{' '}
to register one.
</div>
)}
{!loading && !error && sources.length > 0 && (
<div style={{ overflowX: 'auto' }}>
<table style={{ width: '100%', borderCollapse: 'collapse', fontSize: 13 }}>
<thead>
<tr style={{ borderBottom: '1px solid var(--border)', textAlign: 'left', color: 'var(--text-muted)' }}>
<th style={{ padding: '10px 12px' }}>ID</th>
<th style={{ padding: '10px 12px', textAlign: 'right' }}>Pages</th>
<th style={{ padding: '10px 12px', textAlign: 'right' }}>Chunks</th>
<th style={{ padding: '10px 12px', textAlign: 'right' }}>Embed%</th>
<th style={{ padding: '10px 12px' }}>Last Sync</th>
<th style={{ padding: '10px 12px' }}>Writers</th>
<th style={{ padding: '10px 12px' }}>Readers (federated)</th>
</tr>
</thead>
<tbody>
{sources.map((s) => {
const readers = readersBySource(s.source_id);
const writers = writersBySource(s.source_id);
return (
<tr key={s.source_id} style={{ borderBottom: '1px solid var(--border)' }}>
<td style={{ padding: '10px 12px', fontFamily: 'JetBrains Mono, monospace' }}>
<div>{s.source_id}</div>
{s.name !== s.source_id && (
<div style={{ fontSize: 11, color: 'var(--text-muted)', fontFamily: 'inherit' }}>{s.name}</div>
)}
</td>
<td style={{ padding: '10px 12px', textAlign: 'right', fontFamily: 'JetBrains Mono, monospace' }}>{s.pages.toLocaleString()}</td>
<td style={{ padding: '10px 12px', textAlign: 'right', fontFamily: 'JetBrains Mono, monospace' }}>{s.chunks_total.toLocaleString()}</td>
<td style={{ padding: '10px 12px', textAlign: 'right', color: coverageColor(s.embedding_coverage_pct), fontFamily: 'JetBrains Mono, monospace' }}>
{s.embedding_coverage_pct.toFixed(0)}%
</td>
<td style={{ padding: '10px 12px', color: s.local_path == null ? 'var(--text-muted)' : stalenessColor(s.staleness_class) }}>
{s.local_path == null ? 'push-only' : timeAgo(s.last_sync_at)}
</td>
<td style={{ padding: '10px 12px', fontSize: 12, color: 'var(--text-secondary)' }}>
{writers.length === 0 ? <span style={{ color: 'var(--text-muted)' }}>none</span> : writers.join(', ')}
</td>
<td style={{ padding: '10px 12px', fontSize: 12, color: 'var(--text-secondary)' }}>
{readers.length === 0 ? <span style={{ color: 'var(--text-muted)' }}>none</span> : readers.join(', ')}
</td>
</tr>
);
})}
</tbody>
</table>
</div>
)}
<div style={{
marginTop: 24,
padding: 12,
background: 'var(--bg-elevated)',
border: '1px solid var(--border)',
borderRadius: 6,
fontSize: 12,
color: 'var(--text-muted)',
lineHeight: 1.6,
}}>
<strong style={{ color: 'var(--text-secondary)' }}>Two scopes per OAuth client:</strong>{' '}
<em>Writers</em> = clients with this source as their <code>source_id</code> (write authority).{' '}
<em>Readers</em> = clients with this source in their <code>federated_read</code> list (read access via federation).
Manage federation per-client from the <a href="#agents" style={{ color: '#60a5fa' }}>Agents</a> tab using the
"Manage reads" action.
</div>
</div>
);
}
+6 -1
View File
@@ -13,4 +13,9 @@ timeout = 60_000
# fixtures still match the schema. v0.37's production default is ZE/1280;
# tests that want the new default call configureGateway() explicitly in
# their own beforeAll.
preload = ["./test/helpers/legacy-embedding-preload.ts"]
#
# #2823: redirect GBRAIN_AUDIT_DIR to a per-run scratch dir BEFORE any test
# runs, so audit-emitting code paths (content-sanity, shell-audit, etc.)
# can't leak fixture events into the operator's real ~/.gbrain/audit/. See
# test/helpers/audit-dir-preload.ts for the full rationale.
preload = ["./test/helpers/legacy-embedding-preload.ts", "./test/helpers/audit-dir-preload.ts"]
+2
View File
@@ -3,6 +3,8 @@
On-demand reference (see CLAUDE.md Reference map). Current behavior + invariants
only.
`test/e2e/serve-http-oauth.test.ts` additionally pins confidential POST/Basic revocation, public-client SDK fallthrough, malformed/mixed authentication rejection, cross-client isolation, unknown-token opacity, metadata auth methods, no-store responses, strict post-revoke `401`, and retryable backend `503` semantics.
### Test command tiers
Seven test command tiers, each with a clear scope:
File diff suppressed because one or more lines are too long
+45 -1
View File
@@ -114,8 +114,11 @@ Flip later with `gbrain sources federate <id>` / `unfederate <id>`.
Full subcommand reference:
```
gbrain sources add <id> --path <p> [--name <n>] [--federated|--no-federated]
gbrain sources add <id> --path <p> [--name <n>] [--federated|--no-federated] [--force]
Register a source. id: [a-z0-9](?:[a-z0-9-]{0,30}[a-z0-9])?
--path must be a git repo (or a subdirectory of one) — see
"The git requirement for --path sources" below. --force
skips that check to register before git-init exists.
gbrain sources list [--json] List all sources with page counts + federation state.
gbrain sources remove <id> [--yes] [--dry-run] [--keep-storage]
Cascade-delete a source (pages, chunks, timeline).
@@ -128,6 +131,47 @@ gbrain sources federate <id>
gbrain sources unfederate <id>
```
## The git requirement for --path sources
Every `--path` source must be a git repository (or live inside one — a
subdirectory of a git repo works too) with at least one committed, tracked
file under that path. `gbrain sources add` validates this at registration
time and refuses a directory that doesn't qualify — no `.git` at all, a
`git init` with no commit yet, or a commit made before `git add` — with an
actionable error instead of silently registering a source that will fail
(or worse, "succeed" while importing nothing) on its first `gbrain sync`.
Fix it with:
```bash
git -C <path> init
git -C <path> add -A
git -C <path> commit -m "initial import"
gbrain sources add <id> --path <path>
```
Two details that are easy to miss:
- **Files must actually be committed, not just present.** The sync walker
reads files through git objects, so `git init` alone — even followed by an
empty commit (`git commit --allow-empty`) — isn't enough. Registration
checks for real tracked content (`git ls-tree HEAD` scoped to the path),
not just a resolvable `HEAD`, so this footgun is caught immediately
instead of surfacing later as a sync that imports nothing.
- **`--force` registers the source anyway**, skipping the check. Use this if
you're registering a path before an automated pipeline gets around to
`git init`-ing it. GBrain never auto-`git init`s a `--path` source for
you — it's your directory, not a gbrain-managed clone (same consent
boundary as sync-time self-heal, which also never mutates a `--path`
source without an explicit ask).
**If sync ever reports a problem with the sync anchor** (`last_commit`) —
after a force-push, a history rewrite, or a from-scratch `git init` on a
directory that was synced before — you do not need to reset anything by
hand. `gbrain sync` detects an unreachable or non-ancestor anchor
automatically and recovers: either a full reimport (anchor object missing)
or a direct tree-to-tree diff against the orphaned bookmark (anchor present
but rewritten), advancing the anchor to the new HEAD when it completes.
## Citation format for agents
When agents receive multi-source results they MUST cite pages in
+1 -1
View File
@@ -144,7 +144,7 @@
"bun": ">=1.3.10"
},
"license": "MIT",
"version": "0.42.63.0",
"version": "0.42.64.0",
"overrides": {
"@hono/node-server": "^1.19.13",
"fast-uri": "^3.1.2",
+3 -3
View File
@@ -1,13 +1,13 @@
// AUTO-GENERATED — do not edit by hand.
// Run `bun run scripts/build-admin-embedded.ts` to regenerate.
// Source: admin/dist/ at 2026-05-27.
// Source: admin/dist/ at 2026-07-22.
//
// Bun resolves the file: imports to a path that works at runtime even
// inside a compiled binary (`bun build --compile`). The manifest maps
// the request path the express handler sees to (resolved-path, mime).
// @ts-ignore — type: 'file' is Bun ESM, not in lib.d.ts
import A_0_assets_index_CoGEje3__js from '../admin/dist/assets/index-CoGEje3-.js' with { type: 'file' };
import A_0_assets_index_BpDk4NI4_js from '../admin/dist/assets/index-BpDk4NI4.js' with { type: 'file' };
// @ts-ignore — type: 'file' is Bun ESM, not in lib.d.ts
import A_1_assets_index_GxkWX7v3_css from '../admin/dist/assets/index-GxkWX7v3.css' with { type: 'file' };
// @ts-ignore — type: 'file' is Bun ESM, not in lib.d.ts
@@ -19,7 +19,7 @@ export interface AdminAsset {
}
export const ADMIN_ASSETS: Record<string, AdminAsset> = {
"/admin/assets/index-CoGEje3-.js": { path: A_0_assets_index_CoGEje3__js as unknown as string, mime: "application/javascript; charset=utf-8" },
"/admin/assets/index-BpDk4NI4.js": { path: A_0_assets_index_BpDk4NI4_js as unknown as string, mime: "application/javascript; charset=utf-8" },
"/admin/assets/index-GxkWX7v3.css": { path: A_1_assets_index_GxkWX7v3_css as unknown as string, mime: "text/css; charset=utf-8" },
"/admin/index.html": { path: A_2_index_html as unknown as string, mime: "text/html; charset=utf-8" },
};
+1 -1
View File
@@ -54,7 +54,7 @@ export function bigintToStringReplacer(_key: string, value: unknown): unknown {
}
// CLI-only commands that bypass the operation layer
export const CLI_ONLY = new Set(['init', 'reinit-pglite', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'extract-conversation-facts', 'enrich', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'sources', 'mounts', 'dream', 'check-resolvable', 'routing-eval', 'skillify', 'smoke-test', 'providers', 'storage', 'repos', 'code-def', 'code-refs', 'reindex', 'reindex-code', 'reindex-frontmatter', 'code-callers', 'code-callees', 'frontmatter', 'auth', 'friction', 'claw-test', 'book-mirror', 'takes', 'think', 'salience', 'anomalies', 'calibration', 'transcripts', 'models', 'remote', 'recall', 'forget', 'edges-backfill', 'cache', 'ze-switch', 'founder', 'brainstorm', 'lsd', 'schema', 'capture', 'onboard', 'conversation-parser', 'status', 'connect', 'skillopt', 'quarantine', 'self-upgrade', 'advisor', 'watch', 'reindex-search-vector']);
export const CLI_ONLY = new Set(['init', 'reinit-pglite', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'extract-conversation-facts', 'enrich', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'sources', 'mounts', 'dream', 'check-resolvable', 'routing-eval', 'skillify', 'smoke-test', 'providers', 'storage', 'repos', 'code-def', 'code-refs', 'reindex', 'reindex-code', 'reindex-frontmatter', 'code-callers', 'code-callees', 'reconcile-links', 'frontmatter', 'auth', 'friction', 'claw-test', 'book-mirror', 'takes', 'think', 'salience', 'anomalies', 'calibration', 'transcripts', 'models', 'remote', 'recall', 'forget', 'edges-backfill', 'cache', 'ze-switch', 'founder', 'brainstorm', 'lsd', 'schema', 'capture', 'onboard', 'conversation-parser', 'status', 'connect', 'skillopt', 'quarantine', 'self-upgrade', 'advisor', 'watch', 'reindex-search-vector']);
// CLI-only commands whose handlers print their own --help text. These are
// excluded from the generic short-circuit so detailed per-command and
// per-subcommand usage stays reachable.
+586 -1
View File
@@ -24,6 +24,8 @@ import { loadConfig, toEngineConfig } from '../core/config.ts';
import { createEngine } from '../core/engine-factory.ts';
import type { BrainEngine } from '../core/engine.ts';
import { sqlQueryForEngine, executeRawJsonb, type SqlQuery } from '../core/sql-query.ts';
import { pgArray } from '../core/oauth-provider.ts';
import { assertValidSourceId } from '../core/source-id.ts';
function hashToken(token: string): string {
return createHash('sha256').update(token).digest('hex');
@@ -165,6 +167,100 @@ async function list() {
});
}
/**
* `gbrain auth list-clients [--json]` read surface for OAuth 2.1 clients.
*
* The existing `gbrain auth list` shows LEGACY bearer tokens from
* `access_tokens`; this is the parallel for v0.26+ OAuth clients. Separate
* commands rather than merged output because the two models have different
* field sets (legacy: lifecycle dates; OAuth: scopes + source_id +
* federated_read).
*
* Human output is card-style (multi-line per client) instead of a fixed-
* width table federated_read can hold many ids per client and a wide
* single-line layout truncates / wraps badly on terminals < 200 cols.
* JSON output uses a `schema_version: 1` envelope; additive only.
*/
async function listClients(args: string[]) {
const json = args.includes('--json');
const includeDeleted = args.includes('--include-deleted');
await withConfiguredSql(async (sql) => {
// Codex finding #2 (medium): default-hide soft-deleted clients so admin
// soft-deletes are honored by the CLI surface. Opt-in via flag.
const rows = includeDeleted
? await sql`
SELECT client_id, client_name, scope, source_id, federated_read,
grant_types, created_at, deleted_at
FROM oauth_clients
ORDER BY client_name
`
: await sql`
SELECT client_id, client_name, scope, source_id, federated_read,
grant_types, created_at, deleted_at
FROM oauth_clients
WHERE deleted_at IS NULL
ORDER BY client_name
`;
if (json) {
const clients = rows.map((r) => ({
client_id: String(r.client_id),
client_name: String(r.client_name),
scope: r.scope == null ? null : String(r.scope),
source_id: r.source_id == null ? null : String(r.source_id),
federated_read: Array.isArray(r.federated_read)
? (r.federated_read as string[]).map(String)
: [],
grant_types: Array.isArray(r.grant_types)
? (r.grant_types as string[]).map(String)
: [],
created_at:
r.created_at instanceof Date
? r.created_at.toISOString()
: r.created_at == null
? null
: String(r.created_at),
deleted_at:
r.deleted_at instanceof Date
? r.deleted_at.toISOString()
: r.deleted_at == null
? null
: String(r.deleted_at),
}));
process.stdout.write(JSON.stringify({ schema_version: 1, clients }, null, 2) + '\n');
return;
}
if (rows.length === 0) {
console.log(
includeDeleted
? 'No OAuth clients found (including deleted). Register one: gbrain auth register-client <name>'
: 'No active OAuth clients found. Register one: gbrain auth register-client <name>'
+ '\n(Use --include-deleted to also show soft-deleted clients.)',
);
return;
}
for (let i = 0; i < rows.length; i++) {
const r = rows[i];
const fed = Array.isArray(r.federated_read)
? (r.federated_read as string[]).map(String)
: [];
const grants = Array.isArray(r.grant_types)
? (r.grant_types as string[]).map(String)
: [];
const deletedAt = r.deleted_at;
const status = deletedAt == null
? ''
: ` [SOFT-DELETED ${deletedAt instanceof Date ? deletedAt.toISOString() : String(deletedAt)}]`;
console.log(`${sanitizeForTerminal(String(r.client_name))}${status}`);
console.log(` client_id: ${sanitizeForTerminal(String(r.client_id))}`);
console.log(` scope: ${r.scope == null ? '(none)' : sanitizeForTerminal(String(r.scope))}`);
console.log(` grant types: ${grants.length ? sanitizeForTerminal(grants.join(', ')) : '(none)'}`);
console.log(` write source: ${r.source_id == null ? '(none)' : sanitizeForTerminal(String(r.source_id))}`);
console.log(` federated: ${fed.length ? sanitizeForTerminal(fed.join(', ')) : '(empty)'}`);
if (i < rows.length - 1) console.log('');
}
});
}
async function revoke(name: string) {
if (!name) { console.error('Usage: auth revoke <name>'); process.exit(1); }
await withConfiguredSql(async (sql) => {
@@ -301,6 +397,475 @@ async function test(url: string, token: string) {
console.log(`\n🧠 Your brain is live! (${elapsed}s)`);
}
/**
* Strip ANSI escapes + C0/C1 control characters from a string before
* printing it to the operator's terminal. Defense for the
* codex-flagged terminal-control-injection class: a client_name or
* source_id registered via DCR with `\x1b[2J` (clear-screen) or
* `\x1b]0;TITLE\x07` (OSC title-change) would poison
* `gbrain auth list-clients` output otherwise.
*
* Replaces unsafe bytes with their `\xNN` hex escape so the operator
* sees that something weird is in the field, instead of silent
* mutilation. Tab and newline are preserved as-is so legitimate
* multi-line values render.
*/
export function sanitizeForTerminal(s: string): string {
// ALL C0/C1 controls + DEL get escaped. Codex re-review caught that
// preserving `\n` lets a DCR-registered client_name spoof additional
// human-output lines in list-clients (a real attack — newline in the
// name visually adds a fake row to the operator's terminal). Tab is
// also escaped for the same reason — field-separator spoofing.
// C0: 0x00-0x1F. DEL: 0x7F. C1: 0x80-0x9F.
return s.replace(/[\x00-\x1f\x7f-\x9f]/g, (ch) =>
`\\x${ch.charCodeAt(0).toString(16).padStart(2, '0')}`,
);
}
export interface ResolvedClient {
client_id: string;
client_name: string;
source_id: string | null;
federated_read: string[];
deleted_at: Date | string | null;
}
export type FederatedReadOutcome =
| { kind: 'noop'; reason: 'already-granted' | 'not-present' | 'same-list'; client: ResolvedClient; current: string[] }
| { kind: 'updated'; client: ResolvedClient; before: string[]; after: string[] };
/**
* Resolve an OAuth client by client_id (exact) or client_name (unique).
* Errors on no-match and on ambiguous client_name (>1 row). client_id
* takes precedence if a long hash is passed and matches, returns
* immediately without ever querying by name.
*
* Legacy bearer tokens in `access_tokens` are NOT searched. Federated read
* scope is an OAuth-client concept (oauth_clients.federated_read column);
* legacy bearers have no source scope.
*/
/**
* Resolve an OAuth client. Codex finding #2 (medium): default-hide
* soft-deleted clients so admin-soft-deleted rows aren't mutated by the
* CLI. The `includeDeleted` opt is reserved for future read-side surfaces;
* grant/revoke/set ALWAYS filter active rows only.
*/
export async function resolveClient(
sql: SqlQuery,
nameOrId: string,
opts: { includeDeleted?: boolean } = {},
): Promise<ResolvedClient> {
const allowDeleted = opts.includeDeleted === true;
const byId = allowDeleted
? await sql`
SELECT client_id, client_name, source_id, federated_read, deleted_at
FROM oauth_clients WHERE client_id = ${nameOrId} LIMIT 1
`
: await sql`
SELECT client_id, client_name, source_id, federated_read, deleted_at
FROM oauth_clients WHERE client_id = ${nameOrId} AND deleted_at IS NULL LIMIT 1
`;
if (byId.length === 1) return normalizeClientRow(byId[0]);
const byName = allowDeleted
? await sql`
SELECT client_id, client_name, source_id, federated_read, deleted_at
FROM oauth_clients WHERE client_name = ${nameOrId}
`
: await sql`
SELECT client_id, client_name, source_id, federated_read, deleted_at
FROM oauth_clients WHERE client_name = ${nameOrId} AND deleted_at IS NULL
`;
if (byName.length === 0) {
throw new Error(
`No active OAuth client found with name or id "${nameOrId}". ` +
`Run \`gbrain auth register-client <name>\` to create one, ` +
`or \`gbrain auth list-clients\` to see what exists. ` +
`(Soft-deleted clients are hidden by default.)`,
);
}
if (byName.length > 1) {
const ids = byName.map((r) => ` ${String(r.client_id)}`).join('\n');
throw new Error(
`Multiple active OAuth clients named "${nameOrId}". Pass the full client_id instead:\n${ids}`,
);
}
return normalizeClientRow(byName[0]);
}
function normalizeClientRow(row: Record<string, unknown>): ResolvedClient {
const fed = row.federated_read;
return {
client_id: String(row.client_id),
client_name: String(row.client_name),
source_id: row.source_id == null ? null : String(row.source_id),
federated_read: Array.isArray(fed) ? (fed as string[]).map(String) : [],
deleted_at: row.deleted_at == null
? null
: (row.deleted_at as Date | string),
};
}
/**
* Validate the source_id shape AND DB existence. Codex finding #3 (medium):
* a manually-INSERTed source row with weird chars (e.g. comma, quote)
* would otherwise land in oauth_clients.federated_read as a never-deletable
* malformed entry. Fail at the boundary before the existence query so
* malformed input gets the validator's hint, not a "does not exist" hint
* pointing at a non-creatable id.
*/
export async function assertSourceExists(sql: SqlQuery, sourceId: string): Promise<void> {
assertValidSourceId(sourceId);
const rows = await sql`SELECT id FROM sources WHERE id = ${sourceId} LIMIT 1`;
if (rows.length === 0) {
throw new Error(
`Source "${sourceId}" does not exist. Run \`gbrain sources list\` to see registered sources, ` +
`or \`gbrain sources add ${sourceId}\` to create it.`,
);
}
}
/**
* Atomic append: array_append + NOT-ANY guard so the row-lock fully
* serializes concurrent grant/revoke against the same client. Codex
* finding #1 (HIGH): the previous read-modify-write shape allowed a
* concurrent revoke to be silently UNDONE by a racing grant.
*
* Returns the post-write federated_read array, or null when no rows
* matched (already-granted, soft-deleted, or missing client). Callers
* disambiguate via prior resolveClient + includes() check.
*
* `WHERE deleted_at IS NULL` is part of the atomic guard so a client
* soft-deleted between resolveClient and the UPDATE can't be mutated.
*/
async function appendFederatedReadAtomic(
sql: SqlQuery,
clientId: string,
sourceId: string,
): Promise<string[] | null> {
const rows = await sql`
UPDATE oauth_clients
SET federated_read = array_append(federated_read, ${sourceId})
WHERE client_id = ${clientId}
AND deleted_at IS NULL
AND NOT (${sourceId} = ANY(federated_read))
RETURNING federated_read
`;
if (rows.length === 0) return null;
const fed = rows[0].federated_read;
return Array.isArray(fed) ? (fed as string[]).map(String) : [];
}
/**
* Atomic remove: array_remove + ANY guard. Same race-correctness story
* as appendFederatedReadAtomic. Returns post-write array or null.
*/
async function removeFederatedReadAtomic(
sql: SqlQuery,
clientId: string,
sourceId: string,
): Promise<string[] | null> {
const rows = await sql`
UPDATE oauth_clients
SET federated_read = array_remove(federated_read, ${sourceId})
WHERE client_id = ${clientId}
AND deleted_at IS NULL
AND ${sourceId} = ANY(federated_read)
RETURNING federated_read
`;
if (rows.length === 0) return null;
const fed = rows[0].federated_read;
return Array.isArray(fed) ? (fed as string[]).map(String) : [];
}
/**
* Wholesale array overwrite for `set-federated-read`. Honors the
* deleted_at filter. Last-writer-wins semantics under concurrent
* `set` calls is acceptable the user is asserting "this exact list"
* intent; concurrent set+set just means whichever ran second wins.
* Concurrent set+grant or set+revoke is also last-writer-wins, which
* is the documented contract for `set`.
*/
async function replaceFederatedReadAtomic(
sql: SqlQuery,
clientId: string,
next: string[],
): Promise<string[] | null> {
// TEXT[] binding via pgArray() string-literal escaping (see helper
// for the security note). Our narrow SqlQuery surface
// (src/core/sql-query.ts) doesn't bind JS arrays directly.
const literal = pgArray(next);
const rows = await sql`
UPDATE oauth_clients
SET federated_read = ${literal}
WHERE client_id = ${clientId}
AND deleted_at IS NULL
RETURNING federated_read
`;
if (rows.length === 0) return null;
const fed = rows[0].federated_read;
return Array.isArray(fed) ? (fed as string[]).map(String) : [];
}
/**
* Pure helper: dedupe a comma-separated source-id list while preserving
* insertion order. Empty input empty array. Exported so the CLI parser
* and tests share one normalizer.
*/
export function parseSourceCsv(csv: string): string[] {
const requested = csv.split(',').map((s) => s.trim()).filter(Boolean);
const seen = new Set<string>();
const out: string[] = [];
for (const s of requested) {
if (!seen.has(s)) {
seen.add(s);
out.push(s);
}
}
return out;
}
export interface FederatedReadOpts {
/** When true, compute the outcome but skip the persisting UPDATE. */
dryRun?: boolean;
}
/**
* Core: append a source to the client's federated_read.
*
* Atomicity contract (Codex finding #1, HIGH):
* The actual write goes through `appendFederatedReadAtomic` which
* serializes at the row-lock so concurrent grant/revoke against the
* same client cannot lose updates. The race vector that previously
* silently restored revoked access is closed: under two operators
* racing `revoke-read sensitive` + `grant-read harmless`, postgres
* serializes the two UPDATEs and BOTH ops apply (sensitive removed,
* harmless added), instead of one clobbering the other.
*
* The reported `before` is the snapshot at resolveClient time, which
* may be stale relative to a concurrent racer. The `after` reflects
* the post-UPDATE state from RETURNING (always fresh).
*/
export async function grantReadCore(
sql: SqlQuery,
nameOrId: string,
sourceId: string,
opts: FederatedReadOpts = {},
): Promise<FederatedReadOutcome> {
const client = await resolveClient(sql, nameOrId);
await assertSourceExists(sql, sourceId);
if (client.federated_read.includes(sourceId)) {
return { kind: 'noop', reason: 'already-granted', client, current: client.federated_read };
}
if (opts.dryRun) {
// Compute the would-be result without touching the row. Last-known
// snapshot is best-effort under concurrent writes.
const projected = [...client.federated_read, sourceId];
return { kind: 'updated', client, before: client.federated_read, after: projected };
}
const after = await appendFederatedReadAtomic(sql, client.client_id, sourceId);
if (after === null) {
// Two equivalent failure modes: (a) racing grant-read already added
// the source and the NOT-ANY guard suppressed our UPDATE, or
// (b) the client was soft-deleted between resolveClient and UPDATE.
// (a) is the more common path. Re-resolve to confirm + report.
const reresolved = await resolveClient(sql, client.client_id, { includeDeleted: true });
if (reresolved.deleted_at != null) {
throw new Error(`Client "${client.client_name}" was soft-deleted before write could land.`);
}
return { kind: 'noop', reason: 'already-granted', client: reresolved, current: reresolved.federated_read };
}
return { kind: 'updated', client, before: client.federated_read, after };
}
/**
* Core: remove a source from the client's federated_read. Atomic via
* array_remove + ANY-guard. Same race-correctness rationale as
* grantReadCore concurrent ops serialize at the row lock.
*/
export async function revokeReadCore(
sql: SqlQuery,
nameOrId: string,
sourceId: string,
opts: FederatedReadOpts = {},
): Promise<FederatedReadOutcome> {
const client = await resolveClient(sql, nameOrId);
if (!client.federated_read.includes(sourceId)) {
return { kind: 'noop', reason: 'not-present', client, current: client.federated_read };
}
if (opts.dryRun) {
const projected = client.federated_read.filter((s) => s !== sourceId);
return { kind: 'updated', client, before: client.federated_read, after: projected };
}
const after = await removeFederatedReadAtomic(sql, client.client_id, sourceId);
if (after === null) {
// Same disambiguation as grant: either a concurrent revoke already
// removed the source (most common) or the client was soft-deleted.
const reresolved = await resolveClient(sql, client.client_id, { includeDeleted: true });
if (reresolved.deleted_at != null) {
throw new Error(`Client "${client.client_name}" was soft-deleted before write could land.`);
}
return { kind: 'noop', reason: 'not-present', client: reresolved, current: reresolved.federated_read };
}
return { kind: 'updated', client, before: client.federated_read, after };
}
/**
* Core: replace the whole federated_read list. Idempotent on same list.
*
* Race semantics: wholesale-overwrite + deleted_at guard. Concurrent
* set+set is last-writer-wins (documented contract for `set` the
* operator is asserting the exact list). Concurrent set+grant or
* set+revoke is also last-writer-wins. If a strict-merge semantics is
* needed, use grant-read / revoke-read individually.
*/
export async function setFederatedReadCore(
sql: SqlQuery,
nameOrId: string,
sourceCsv: string,
opts: FederatedReadOpts = {},
): Promise<FederatedReadOutcome> {
const next = parseSourceCsv(sourceCsv);
const client = await resolveClient(sql, nameOrId);
for (const s of next) {
await assertSourceExists(sql, s);
}
const prev = client.federated_read;
const same = prev.length === next.length && prev.every((v, i) => v === next[i]);
if (same) {
return { kind: 'noop', reason: 'same-list', client, current: prev };
}
if (opts.dryRun) {
return { kind: 'updated', client, before: prev, after: next };
}
const after = await replaceFederatedReadAtomic(sql, client.client_id, next);
if (after === null) {
throw new Error(`Client "${client.client_name}" was soft-deleted before write could land.`);
}
return { kind: 'updated', client, before: prev, after };
}
function printOutcome(
verb: 'grant' | 'revoke' | 'set',
sourceArg: string,
outcome: FederatedReadOutcome,
dryRun: boolean,
): void {
// Terminal-injection defense (Codex finding #5, low): a client_name
// registered via DCR with ANSI escapes or control chars would
// otherwise poison this output. Sanitize ALL strings that round-trip
// from the DB before printing.
const s = sanitizeForTerminal;
const prefix = dryRun ? '[dry-run] ' : '';
if (outcome.kind === 'noop') {
const name = s(outcome.client.client_name);
if (outcome.reason === 'already-granted') {
console.log(`${prefix}No change: "${name}" already reads "${s(sourceArg)}".`);
} else if (outcome.reason === 'not-present') {
console.log(`${prefix}No change: "${name}" did not read "${s(sourceArg)}".`);
} else {
console.log(`${prefix}No change: "${name}" federated_read already matches.`);
}
console.log(` federated_read: ${outcome.current.map(s).join(', ') || '(empty)'}`);
return;
}
const { client, before, after } = outcome;
const name = s(client.client_name);
const wouldOrDid = dryRun ? 'Would' : 'Did';
if (verb === 'grant') {
console.log(`${prefix}${wouldOrDid} grant: "${name}" can now read "${s(sourceArg)}".`);
console.log(` federated_read: ${after.map(s).join(', ')}`);
} else if (verb === 'revoke') {
console.log(`${prefix}${wouldOrDid} revoke: "${name}" no longer reads "${s(sourceArg)}".`);
console.log(` federated_read: ${after.map(s).join(', ') || '(empty — client has no federated reads)'}`);
} else {
console.log(`${prefix}${wouldOrDid} update "${name}" federated_read:`);
console.log(` before: ${before.map(s).join(', ') || '(empty)'}`);
console.log(` after: ${after.map(s).join(', ') || '(empty)'}`);
}
if (after.length === 0) {
console.log(
'Warning: client now reads no sources via federation. Queries through this ' +
'client will only see content scoped explicitly via its write source.',
);
}
}
/**
* Strip `--dry-run` from a positional-arg list. Returns the filtered list
* plus the flag value. Kept positional-tolerant the existing
* `auth grant-read alice source` shape MUST keep working, AND
* `auth grant-read alice source --dry-run` AND `auth grant-read --dry-run alice source`.
*/
export function extractDryRun(args: string[]): { dryRun: boolean; rest: string[] } {
let dryRun = false;
const rest: string[] = [];
for (const a of args) {
if (a === '--dry-run') {
dryRun = true;
continue;
}
rest.push(a);
}
return { dryRun, rest };
}
async function grantRead(args: string[]): Promise<void> {
const { dryRun, rest } = extractDryRun(args);
const [nameOrId, sourceId] = rest;
if (!nameOrId || !sourceId) {
console.error('Usage: gbrain auth grant-read <client-name-or-id> <source-id> [--dry-run]');
process.exit(1);
}
try {
await withConfiguredSql(async (sql) => {
const outcome = await grantReadCore(sql, nameOrId, sourceId, { dryRun });
printOutcome('grant', sourceId, outcome, dryRun);
});
} catch (e: any) {
console.error('Error:', e.message);
process.exit(1);
}
}
async function revokeRead(args: string[]): Promise<void> {
const { dryRun, rest } = extractDryRun(args);
const [nameOrId, sourceId] = rest;
if (!nameOrId || !sourceId) {
console.error('Usage: gbrain auth revoke-read <client-name-or-id> <source-id> [--dry-run]');
process.exit(1);
}
try {
await withConfiguredSql(async (sql) => {
const outcome = await revokeReadCore(sql, nameOrId, sourceId, { dryRun });
printOutcome('revoke', sourceId, outcome, dryRun);
});
} catch (e: any) {
console.error('Error:', e.message);
process.exit(1);
}
}
async function setFederatedRead(args: string[]): Promise<void> {
const { dryRun, rest } = extractDryRun(args);
const [nameOrId, sourceCsv] = rest;
if (!nameOrId || sourceCsv === undefined) {
console.error(
'Usage: gbrain auth set-federated-read <client-name-or-id> <source-id1,source-id2,...> [--dry-run]',
);
console.error('Pass an empty string ("") to clear all federated reads.');
process.exit(1);
}
try {
await withConfiguredSql(async (sql) => {
const outcome = await setFederatedReadCore(sql, nameOrId, sourceCsv, { dryRun });
printOutcome('set', sourceCsv, outcome, dryRun);
});
} catch (e: any) {
console.error('Error:', e.message);
process.exit(1);
}
}
async function revokeClient(clientId: string) {
if (!clientId) {
console.error('Usage: auth revoke-client <client_id>');
@@ -319,7 +884,7 @@ async function revokeClient(clientId: string) {
console.error(`No client found with id "${clientId}"`);
process.exit(1);
}
console.log(`OAuth client revoked: "${rows[0].client_name}" (${clientId})`);
console.log(`OAuth client revoked: "${sanitizeForTerminal(String(rows[0].client_name))}" (${clientId})`);
console.log('Tokens and authorization codes purged via cascade.');
});
} catch (e: any) {
@@ -440,6 +1005,15 @@ export function parseRegisterClientArgs(args: string[]): RegisterClientArgs {
if (!grantTypesSet && out.redirectUris.length > 0) {
out.grantTypes = ['authorization_code', 'refresh_token'];
}
// Codex re-review (medium): validate source_id shape at the CLI boundary
// so register-client can't seed malformed entries into source_id /
// federated_read that subsequent grant/revoke/set commands can't manage.
assertValidSourceId(out.sourceId);
if (out.federatedRead) {
for (const s of out.federatedRead) {
assertValidSourceId(s);
}
}
return out;
}
@@ -557,6 +1131,10 @@ export async function runAuth(args: string[]): Promise<void> {
}
case 'register-client': await registerClient(rest[0], rest.slice(1)); return;
case 'revoke-client': await revokeClient(rest[0]); return;
case 'list-clients': await listClients(rest); return;
case 'grant-read': await grantRead(rest); return;
case 'revoke-read': await revokeRead(rest); return;
case 'set-federated-read': await setFederatedRead(rest); return;
case 'test': {
const tokenIdx = rest.indexOf('--token');
const url = rest.find(a => !a.startsWith('--') && a !== rest[tokenIdx + 1]);
@@ -594,6 +1172,13 @@ Usage:
--bound-max-concurrent <n> Bound submit_agent concurrency (default: 1)
--budget-usd-per-day <usd> Bound submit_agent daily spend cap
gbrain auth revoke-client <client_id> Hard-delete an OAuth 2.1 client (cascades to tokens + codes)
gbrain auth list-clients [--json] List OAuth 2.1 clients with scope + write source + federated_read.
gbrain auth grant-read <name|client_id> <source-id> [--dry-run]
Add a source to the client's federated_read list (idempotent).
gbrain auth revoke-read <name|client_id> <source-id> [--dry-run]
Remove a source from the client's federated_read list (idempotent).
gbrain auth set-federated-read <name|client_id> "<id1,id2,...>" [--dry-run]
Replace the client's whole federated_read list. Pass "" to clear.
gbrain auth test <url> --token <token> Smoke-test a remote MCP server
`);
}
+18 -4
View File
@@ -17,7 +17,7 @@
* gbrain autopilot --status [--json]
*/
import { existsSync, readFileSync, writeFileSync, mkdirSync, appendFileSync, utimesSync, unlinkSync } from 'fs';
import { existsSync, readFileSync, writeFileSync, mkdirSync, appendFileSync, utimesSync, unlinkSync, chmodSync } from 'fs';
import { setCliExitVerdict } from '../core/cli-force-exit.ts';
import { join } from 'path';
import { execSync } from 'child_process';
@@ -1263,7 +1263,14 @@ function installLaunchd(wrapperPath: string, home: string, repoPath: string) {
try {
const agentsDir = join(home, 'Library', 'LaunchAgents');
mkdirSync(agentsDir, { recursive: true });
writeFileSync(plistPath(), plist);
writeFileSync(plistPath(), plist, { mode: 0o644 });
// launchd rejects group/world-writable agent plists: bootstrap/load fails
// with the opaque "Bootstrap failed: 5: Input/output error" and the login
// scan skips the file silently. writeFileSync's mode only applies on
// create — a reinstall over an existing plist keeps the old bits (a 0666
// plist written under an umask-0 parent stays 0666 forever) — so
// normalize unconditionally.
chmodSync(plistPath(), 0o644);
execSync(`launchctl load "${plistPath()}"`, { stdio: 'pipe' });
console.log('Installed launchd service: com.gbrain.autopilot');
console.log(` Repo: ${repoPath}`);
@@ -1353,7 +1360,11 @@ export function migrateSystemdUnitToRestartAlways(): { rewritten: boolean; reaso
return { rewritten: false, reason: 'hand-edited' };
}
try {
writeFileSync(unitPath, generateSystemdUnit(execMatch![1]));
writeFileSync(unitPath, generateSystemdUnit(execMatch![1]), { mode: 0o644 });
// This path always rewrites an EXISTING unit, so writeFileSync's mode
// never applies — chmod is the only thing that normalizes a unit born
// 0666 under a umask-0 parent (systemd warns on world-writable units).
chmodSync(unitPath, 0o644);
try {
execSync('systemctl --user daemon-reload', { stdio: 'pipe', timeout: 10_000 });
} catch {
@@ -1370,7 +1381,10 @@ function installSystemd(wrapperPath: string, repoPath: string) {
try {
const unitPath = systemdUnitPath();
mkdirSync(join(process.env.HOME || '', '.config', 'systemd', 'user'), { recursive: true });
writeFileSync(unitPath, unit);
writeFileSync(unitPath, unit, { mode: 0o644 });
// Same umask-0 hardening as the launchd path (systemd warns on
// world-writable units); mode only applies on create, so normalize.
chmodSync(unitPath, 0o644);
execSync('systemctl --user daemon-reload', { stdio: 'pipe', timeout: 10_000 });
execSync('systemctl --user enable --now gbrain-autopilot.service', { stdio: 'pipe', timeout: 15_000 });
console.log('Installed systemd user service: gbrain-autopilot.service');
+22 -15
View File
@@ -3056,7 +3056,7 @@ export async function computeConversationFactsBacklogCheck(
const typesRaw = await engine.getConfig(
'cycle.conversation_facts_backfill.types',
);
let types = ['conversation', 'meeting', 'slack', 'email'];
let types = ['conversation', 'meeting', 'slack', 'email', 'imessage', 'imessage-daily'];
if (typesRaw) {
try {
const parsed = JSON.parse(typesRaw);
@@ -4345,7 +4345,7 @@ export async function buildChecks(
// 2. Skill conformance (SKILL group — gated)
if (scope === 'all' && skillsDir) {
const conformanceResult = checkSkillConformance(skillsDir);
const conformanceResult = skillConformanceCheck(skillsDir);
checks.push(conformanceResult);
}
@@ -4927,8 +4927,8 @@ export async function buildChecks(
try {
const { readConversationBodyForParsing } = await import('../core/conversation-parser/body.ts');
const { parseConversation } = await import('../core/conversation-parser/parse.ts');
const allowedTypes = ['conversation', 'meeting', 'slack', 'email'] as const;
// PageFilters supports singular `type` only; iterate the 4 types
const allowedTypes = ['conversation', 'meeting', 'slack', 'email', 'imessage', 'imessage-daily'] as const;
// PageFilters supports singular `type` only; iterate the allowed types
// and cap at ~50/each to land at ~200 total max.
const sample: import('../core/types.ts').Page[] = [];
for (const t of allowedTypes) {
@@ -7178,9 +7178,17 @@ export async function buildChecks(
let vanished = 0;
const vanishedPaths: string[] = [];
const fs = await import('node:fs');
const nodePath = await import('node:path');
// storage_path is repo-relative for sync-ingested assets. Resolving
// against cwd made this check a false-positive WARN whenever doctor
// ran outside the brain repo.
const repoRoot = (await engine.getConfig('sync.repo_path')) ?? process.cwd();
for (const r of rows) {
const abs = nodePath.isAbsolute(r.storage_path)
? r.storage_path
: nodePath.join(repoRoot, r.storage_path);
try {
fs.statSync(r.storage_path);
fs.statSync(abs);
} catch {
vanished++;
if (vanishedPaths.length < 5) vanishedPaths.push(r.storage_path);
@@ -7424,15 +7432,13 @@ function printAutoFixReport(report: AutoFixReport, dryRun: boolean, jsonOutput:
/** Quick skill conformance check — frontmatter + required sections */
function checkSkillConformance(skillsDir: string): Check {
const manifestPath = join(skillsDir, 'manifest.json');
if (!existsSync(manifestPath)) {
return { name: 'skill_conformance', status: 'warn', message: 'manifest.json not found' };
}
export function skillConformanceCheck(skillsDir: string): Check {
try {
const manifest = JSON.parse(readFileSync(manifestPath, 'utf-8'));
const skills = manifest.skills || [];
// Host workspaces are allowed to omit a gbrain-specific manifest. Keep
// conformance aligned with resolver_health and skill_brain_first by using
// the canonical fallback that derives entries from direct SKILL.md files.
const manifest = loadOrDeriveManifest(skillsDir);
const skills = manifest.skills;
let passing = 0;
const failing: string[] = [];
@@ -7452,7 +7458,8 @@ function checkSkillConformance(skillsDir: string): Check {
}
if (failing.length === 0) {
return { name: 'skill_conformance', status: 'ok', message: `${passing}/${skills.length} skills pass` };
const derivedNote = manifest.derived ? ' (derived from SKILL.md files)' : '';
return { name: 'skill_conformance', status: 'ok', message: `${passing}/${skills.length} skills pass${derivedNote}` };
}
return {
name: 'skill_conformance',
@@ -7460,7 +7467,7 @@ function checkSkillConformance(skillsDir: string): Check {
message: `${passing}/${skills.length} pass. Failing: ${failing.join(', ')}`,
};
} catch {
return { name: 'skill_conformance', status: 'warn', message: 'Could not parse manifest.json' };
return { name: 'skill_conformance', status: 'warn', message: 'Could not load or derive skills manifest' };
}
}
+65
View File
@@ -76,6 +76,18 @@ interface DreamArgs {
drain: boolean;
/** Drain wallclock budget in seconds. Default 300 (5 min). */
windowSeconds: number;
/**
* issue #2860 `--once`. One-shot bypass of the named `--phase`'s own
* `dream.<phase>.enabled` / `cycle.<phase>.enabled` config gate, for this
* invocation only. Never reads or writes config unlike the old
* "toggle enabled true, run, toggle back to false" workaround, a crash
* mid-run can't leave any global state stuck. Requires an explicit
* `--phase <name>`; bare `--once` is a usage error (there'd be no single
* phase to target). Applies only to phases with a config `.enabled` gate
* (patterns, synthesize, conversation_facts_backfill, enrich_thin,
* skillopt) a no-op for phases that always run when named directly.
*/
once: boolean;
}
const ISO_DATE_RE = /^\d{4}-\d{2}-\d{2}$/;
@@ -105,6 +117,14 @@ function collectFlagValues(args: string[], flag: string): string[] | null {
function parseArgs(args: string[]): DreamArgs {
const phaseIdx = args.indexOf('--phase');
// issue #2860 (Codex P3): captured BEFORE --input/--drain get a chance to
// implicitly default `phase` below, so --once's validation can require
// the user actually TYPED --phase, not merely that some phase ended up
// resolved. Without this, `--input <f> --once` and `--drain --once`
// slip past the "explicit --phase required" contract (the derived
// `phase` value is already non-null by the time that check runs) and
// --once becomes silently ineffective for both.
const phaseWasExplicit = phaseIdx !== -1;
const rawPhase = phaseIdx !== -1 ? args[phaseIdx + 1] : null;
let phase = rawPhase && (ALL_PHASES as string[]).includes(rawPhase)
? (rawPhase as CyclePhase)
@@ -214,6 +234,35 @@ function parseArgs(args: string[]): DreamArgs {
}
}
// issue #2860: --once requires an EXPLICIT single --phase target (typed
// by the user, not merely implied by --input/--drain — see
// `phaseWasExplicit` above). Bare `--once` (full/default cycle) has no
// single phase to bypass the gate for, and force-enabling EVERY
// currently-disabled phase at once would be exactly the kind of
// surprise-spend risk the flag exists to prevent. An implicit phase
// (from --input or --drain) is rejected too: --drain returns before
// onceForPhase is ever read, and --input already bypasses the
// synthesize gate on its own, so --once would silently do nothing in
// either case — reject loudly instead of pretending it worked (Codex
// review finding).
//
// Codex review finding: `--help` must short-circuit BEFORE this exits(2),
// matching the "IRON RULE" pinned by test/dream.test.ts's
// "--help --source whatever prints help and exits 0" case — `gbrain
// dream --help --once` (no --phase) must show help, not a usage error.
const once = args.includes('--once');
const wantsHelp = args.includes('--help') || args.includes('-h');
if (once && !phaseWasExplicit && !wantsHelp) {
console.error(
'--once requires an explicit --phase <name> (bypasses that one ' +
'phase\'s dream.<phase>.enabled / cycle.<phase>.enabled gate for ' +
'this run only; never touches config). A phase implied by --input ' +
'or --drain does not count — --once would silently do nothing for ' +
'those. Usage: gbrain dream --phase <name> --once',
);
process.exit(2);
}
return {
json: args.includes('--json'),
dryRun: args.includes('--dry-run'),
@@ -229,6 +278,7 @@ function parseArgs(args: string[]): DreamArgs {
source,
drain,
windowSeconds,
once,
};
}
@@ -310,6 +360,17 @@ Options:
"--dry-run" does NOT mean "zero LLM calls."
--json Emit the CycleReport as JSON (agent-readable)
--phase <name> Run a single phase: ${ALL_PHASES.join(' | ')}
--once With --phase <name>: run that phase once even if its
own dream.<phase>.enabled / cycle.<phase>.enabled
config gate is false. Never reads or writes config
unlike toggling the flag on/off around the run, a
crash mid-invocation can't leave it stuck. Applies to
patterns, synthesize, conversation_facts_backfill,
enrich_thin, skillopt; no-op on phases with no such
gate. Requires an EXPLICIT --phase <name> a phase
implied by --input or --drain does not count (bare
--once, or --once with --input/--drain and no
explicit --phase, is a usage error).
--pull git pull the brain repo before syncing (default: no pull)
--dir <path> Brain directory (default: configured brain). On a
postgres/remote brain with no local checkout, the
@@ -353,6 +414,7 @@ Examples:
gbrain dream
gbrain dream --dry-run --json
gbrain dream --phase lint
gbrain dream --phase patterns --once # run once, ignore dream.patterns.enabled=false
gbrain dream --phase synthesize --input ~/transcripts/2026-04-25.txt
gbrain dream --phase synthesize --from 2026-04-01 --to 2026-04-25
0 2 * * * gbrain dream --json # nightly via cron
@@ -594,6 +656,9 @@ export async function runDream(engine: BrainEngine | null, args: string[]): Prom
synthFrom: opts.from ?? undefined,
synthTo: opts.to ?? undefined,
synthBypassDreamGuard: opts.bypassDreamGuard,
// issue #2860: opts.phase is guaranteed non-null here when opts.once is
// set (parseArgs enforces --once requires --phase).
onceForPhase: opts.once ? opts.phase! : undefined,
});
if (opts.json) {
+4 -2
View File
@@ -33,7 +33,7 @@ import type { BrainEngine } from '../core/engine.ts';
import type { EnrichCandidate, PageType } from '../core/types.ts';
import { operations } from '../core/operations.ts';
import type { OperationContext } from '../core/operations.ts';
import { isAvailable, chat, getChatModel, withBudgetTracker } from '../core/ai/gateway.ts';
import { configureGatewayIfUninitialized, isAvailable, chat, getChatModel, withBudgetTracker } from '../core/ai/gateway.ts';
import { BudgetTracker, BudgetExhausted } from '../core/budget/budget-tracker.ts';
import { hybridSearch } from '../core/search/hybrid.ts';
import { serializeMarkdown } from '../core/markdown.ts';
@@ -807,7 +807,9 @@ export async function runEnrich(engine: BrainEngine, args: string[]): Promise<vo
process.exit(1);
}
// Chat gateway required for non-dry-run.
// Chat gateway is required for non-dry-run. Recover a cold singleton before
// reporting an availability error (#2590).
if (!parsed.dryRun && !isAvailable('chat')) configureGatewayIfUninitialized();
if (!parsed.dryRun && !isAvailable('chat')) {
console.error('Chat gateway unavailable. Configure a chat model (e.g. `gbrain config set chat_model anthropic:claude-haiku-4-5`), or pass --dry-run to preview candidates.');
process.exit(1);
+18 -4
View File
@@ -71,7 +71,7 @@ import {
extractFactsFromTurn,
isFactsExtractionEnabled,
} from '../core/facts/extract.ts';
import { isAvailable, withBudgetTracker } from '../core/ai/gateway.ts';
import { configureGatewayIfUninitialized, isAvailable, withBudgetTracker } from '../core/ai/gateway.ts';
import { BudgetTracker, BudgetExhausted } from '../core/budget/budget-tracker.ts';
import { listSources } from '../core/sources-ops.ts';
import {
@@ -81,7 +81,6 @@ import {
} from '../core/op-checkpoint.ts';
import { createProgress } from '../core/progress.ts';
import { getCliOptions, cliOptsToProgressOptions, maybeBackground } from '../core/cli-options.ts';
import { loadConfig } from '../core/config.ts';
import { createHash } from 'crypto';
// v0.41.15.0 (T5): worker-pool primitive + per-source-clamp wrapper +
// per-page advisory lock + delete-orphans-first replay safety. See plan
@@ -141,7 +140,14 @@ export const DEFAULT_MAX_COST_USD = 5.0;
* `--types` flag is an explicit per-run override; cycle config is
* the single source of truth.
*/
export const ALLOWED_TYPES = ['conversation', 'meeting', 'slack', 'email'] as const;
export const ALLOWED_TYPES = [
'conversation',
'meeting',
'slack',
'email',
'imessage',
'imessage-daily',
] as const;
export type AllowedType = (typeof ALLOWED_TYPES)[number];
/**
@@ -757,6 +763,12 @@ async function processPage(
source_markdown_slug: page.slug,
source: PER_SEGMENT_SOURCE_PREFIX,
source_session: sessionId,
// Preserve the conversation's valid time instead of defaulting every
// extracted fact to extraction time. Epoch-anchored parses have no
// trustworthy date, so they retain the existing now() fallback.
...(seg.startIso && !seg.startIso.startsWith('1970-')
? { valid_from: new Date(seg.startIso) }
: {}),
context:
fact.context ?? `from ${page.slug} segment ${seg.startIso}..${seg.endIso}`,
}));
@@ -1354,7 +1366,9 @@ export async function runExtractConversationFacts(
process.exit(1);
}
// Chat gateway is required for non-dry-run.
// Chat gateway is required for non-dry-run. Recover a cold singleton before
// reporting an availability error (#2590).
if (!parsed.dryRun && !isAvailable('chat')) configureGatewayIfUninitialized();
if (!parsed.dryRun && !isAvailable('chat')) {
console.error('Chat gateway unavailable. Configure an Anthropic or compatible chat model, or pass --dry-run to preview segmentation.');
process.exit(1);
+58 -15
View File
@@ -7,7 +7,7 @@ import type { BrainEngine } from '../core/engine.ts';
import { MinionQueue } from '../core/minions/queue.ts';
import { MinionWorker } from '../core/minions/worker.ts';
import { WORKER_EXIT_RSS_WATCHDOG } from '../core/minions/worker-exit-codes.ts';
import type { MinionJob, MinionJobStatus } from '../core/minions/types.ts';
import type { MinionHandler, MinionJob, MinionJobStatus } from '../core/minions/types.ts';
import type { PaceKeyOverrides } from '../core/pace-mode.ts';
import { loadConfig, isThinClient } from '../core/config.ts';
import { callRemoteTool, unpackToolResult } from '../core/mcp-client.ts';
@@ -22,6 +22,49 @@ function hasFlag(args: string[], flag: string): boolean {
return args.includes(flag);
}
/**
* Long-lived workers outlive operator config changes. Re-stamp the AI gateway
* from DB-backed model config immediately before queued jobs enter gateway-backed
* paths, so a stale process-level default cannot route new work to the wrong
* provider.
*/
async function refreshGatewayForJob(engine: BrainEngine): Promise<void> {
const { reconfigureGatewayWithEngine } = await import('../core/ai/gateway.ts');
await reconfigureGatewayWithEngine(engine);
}
const GATEWAY_REFRESH_JOB_NAMES = new Set([
'embed',
'extract-conversation-facts',
'enrich',
'contextual_reindex_per_chunk',
'autopilot-cycle',
'synthesize',
'patterns',
'consolidate',
'extract_facts',
'extract-atoms-drain',
'embed-backfill',
'extract-takes-from-pages',
'embed-catch-up',
]);
function registerBuiltinJob(
worker: MinionWorker,
engine: BrainEngine,
name: string,
handler: MinionHandler,
): void {
if (!GATEWAY_REFRESH_JOB_NAMES.has(name)) {
worker.register(name, handler);
return;
}
worker.register(name, async (job) => {
await refreshGatewayForJob(engine);
return await handler(job);
});
}
/** Parse `--max-waiting N` from CLI args. Returns undefined if absent.
* Throws on malformed input (caller should surface the error and exit).
* Clamps to [1, 100] to match the queue-layer clamp in MinionQueue.add.
@@ -1439,7 +1482,7 @@ export async function registerBuiltinHandlers(
return { ...result, embed_job_id: embedJobId, embed_skip_reason: embedSkipReason };
});
worker.register('embed', async (job) => {
registerBuiltinJob(worker, engine, 'embed', async (job) => {
const { runEmbedCore } = await import('./embed.ts');
// Primary Minion progress channel is job.updateProgress (DB-backed,
// readable via `gbrain jobs get <id>`). Stderr from the worker daemon
@@ -1486,7 +1529,7 @@ export async function registerBuiltinHandlers(
// BudgetTracker inside its own process. BudgetExhausted is caught at
// the core level and returned as `result.budget_exhausted: true` (NOT
// a job failure) so the user can resume with a higher cap.
worker.register('extract-conversation-facts', async (job) => {
registerBuiltinJob(worker, engine, 'extract-conversation-facts', async (job) => {
const { runExtractConversationFactsCore } = await import('./extract-conversation-facts.ts');
const sourceId = typeof job.data.sourceId === 'string' ? job.data.sourceId : undefined;
if (!sourceId) {
@@ -1497,7 +1540,7 @@ export async function registerBuiltinHandlers(
}
const types = Array.isArray(job.data.types)
? (job.data.types as string[]).filter((t) =>
['conversation', 'meeting', 'slack', 'email'].includes(t),
['conversation', 'meeting', 'slack', 'email', 'imessage', 'imessage-daily'].includes(t),
)
: undefined;
const result = await runExtractConversationFactsCore(engine, {
@@ -1545,7 +1588,7 @@ export async function registerBuiltinHandlers(
// at the core level and returned as result.budget_exhausted (NOT a failure).
// Strict per-source: the CLI fans out one job per source when --source is
// omitted, so a job ALWAYS carries data.sourceId.
worker.register('enrich', async (job) => {
registerBuiltinJob(worker, engine, 'enrich', async (job) => {
const { runEnrichCore } = await import('./enrich.ts');
const sourceId = typeof job.data.sourceId === 'string' ? job.data.sourceId : undefined;
if (!sourceId) {
@@ -1685,13 +1728,13 @@ export async function registerBuiltinHandlers(
const { makeContextualReindexHandler } = await import(
'../core/minions/handlers/contextual-reindex-per-chunk.ts'
);
worker.register('contextual_reindex_per_chunk', makeContextualReindexHandler({ engine }));
registerBuiltinJob(worker, engine, 'contextual_reindex_per_chunk', makeContextualReindexHandler({ engine }));
}
// derivation); the handler returns { partial, status, report } so
// `gbrain jobs get <id>` shows the full structured report. Does NOT
// throw on partial: a flaky phase must not block every future cycle.
worker.register('autopilot-cycle', async (job) => {
registerBuiltinJob(worker, engine, 'autopilot-cycle', async (job) => {
const { runCycle } = await import('../core/cycle.ts');
// v0.41.30 (T2): fall back to null (NOT cwd '.') when no repo is configured.
// The queued cycle is the same primitive `gbrain dream` uses; a checkout-less
@@ -1986,12 +2029,12 @@ export async function registerBuiltinHandlers(
};
// PROTECTED — internally spawn subagent children
worker.register('synthesize', makePhaseHandler('synthesize'));
worker.register('patterns', makePhaseHandler('patterns'));
worker.register('consolidate', makePhaseHandler('consolidate'));
registerBuiltinJob(worker, engine, 'synthesize', makePhaseHandler('synthesize'));
registerBuiltinJob(worker, engine, 'patterns', makePhaseHandler('patterns'));
registerBuiltinJob(worker, engine, 'consolidate', makePhaseHandler('consolidate'));
// Open — DB writes only, no LLM spend
worker.register('extract_facts', makePhaseHandler('extract_facts'));
registerBuiltinJob(worker, engine, 'extract_facts', makePhaseHandler('extract_facts'));
worker.register('resolve_symbol_edges', makePhaseHandler('resolve_symbol_edges'));
worker.register('recompute_emotional_weight', makePhaseHandler('recompute_emotional_weight'));
@@ -2001,7 +2044,7 @@ export async function registerBuiltinHandlers(
// window / defer behavior. On LockUnavailableError (the routine cycle holds
// the per-source lock) the job completes `{ deferred: true }` and retries
// next tick instead of failing — cooperative interleave (CODEX accepted).
worker.register('extract-atoms-drain', async (job) => {
registerBuiltinJob(worker, engine, 'extract-atoms-drain', async (job) => {
const { runExtractAtomsDrainForSource } = await import('../core/cycle/extract-atoms-drain.ts');
const { LockUnavailableError } = await import('../core/db-lock.ts');
const sourceId = typeof job.data.sourceId === 'string' ? job.data.sourceId : undefined;
@@ -2029,7 +2072,7 @@ export async function registerBuiltinHandlers(
// Cost-bounded via D6 ($10/job BudgetTracker) + D19 (source-level cooldown
// + 24h rolling cap, gated at submit time). NOT in PROTECTED_JOB_NAMES —
// embedding-only spend, no API-by-the-minute risk like subagent.
worker.register('embed-backfill', async (job) => {
registerBuiltinJob(worker, engine, 'embed-backfill', async (job) => {
const { makeEmbedBackfillHandler } = await import('../core/minions/handlers/embed-backfill.ts');
return await makeEmbedBackfillHandler(engine)(job);
});
@@ -2050,7 +2093,7 @@ export async function registerBuiltinHandlers(
// (LLM-bearing). Two-gate consent enforced at the handler boundary:
// refuses to run unless takes.bootstrap_enabled config is true, even
// when allowProtectedSubmit was set at queue.add time.
worker.register('extract-takes-from-pages', async (job) => {
registerBuiltinJob(worker, engine, 'extract-takes-from-pages', async (job) => {
const { extractTakesFromPages } = await import('../core/extract-takes-from-pages.ts');
const data = (job.data ?? {}) as { sourceId?: string; maxPages?: number };
const bootstrapCfg = await engine.getConfig('takes.bootstrap_enabled');
@@ -2077,7 +2120,7 @@ export async function registerBuiltinHandlers(
// remediation pipeline. Wraps runEmbedCore with stale + catchUp + the
// priority/batchSize the recommendation supplies. NOT in
// PROTECTED_JOB_NAMES (embedding spend only).
worker.register('embed-catch-up', async (job) => {
registerBuiltinJob(worker, engine, 'embed-catch-up', async (job) => {
const { runEmbedCore } = await import('./embed.ts');
const data = (job.data ?? {}) as {
sourceId?: string;
+24 -4
View File
@@ -7,9 +7,9 @@
import { listRecipes, getRecipe } from '../core/ai/recipes/index.ts';
import { configureGateway, embedOne, isAvailable as gwIsAvailable, chat as gwChat } from '../core/ai/gateway.ts';
import { buildGatewayConfig } from '../core/ai/build-gateway-config.ts';
import { probeOllama, probeLMStudio } from '../core/ai/probes.ts';
import { loadConfig } from '../core/config.ts';
import { buildGatewayConfig } from '../core/ai/build-gateway-config.ts';
import { AIConfigError, AITransientError } from '../core/ai/errors.ts';
import type { Recipe } from '../core/ai/types.ts';
@@ -173,8 +173,18 @@ async function runTest(args: string[]): Promise<void> {
// the divergence at the top of the test so the recovery experience
// doesn't repeat the bug-reporter's "providers test ✓ but import still
// broken" trap.
//
// #2863: `cfg` is lifted out of the try block (not just used for the
// warning) so the configureGateway calls below can reuse it. Before this
// fix, the --model override only forwarded embedding_model/chat_model +
// env, dropping config.provider_base_urls entirely — a probe against a
// custom endpoint (e.g. a regional DashScope base URL) would silently
// fall back to the recipe's hardcoded default endpoint and fail with a
// misleading "Incorrect API key" error even though the key was valid for
// the configured endpoint.
let cfg: ReturnType<typeof loadConfig> | null = null;
try {
const cfg = loadConfig();
cfg = loadConfig();
const configuredModel = tpArg === 'embedding' ? cfg?.embedding_model : cfg?.chat_model;
if (!configuredModel) {
console.error(
@@ -190,17 +200,27 @@ async function runTest(args: string[]): Promise<void> {
}
} catch { /* loadConfig throws when no brain configured — first-time install path; the no-config branch above handles it. */ }
// Reuse the SAME resolver the production path uses (buildGatewayConfig —
// also used by cli.ts#connectEngine and init-embed-check.ts) so the probe
// sees the identical base_urls / provider_chat_options / folded API keys
// that a real `gbrain import`/`gbrain query` call would. Only the
// touchpoint's model (+ embedding dims) is overridden on top, so an
// isolated `--model` probe still targets exactly the requested model —
// it just resolves that model's endpoint the way the brain actually
// would. Falls back to bare env when no brain is configured yet (cfg is
// null on first-time install, matching the old behavior for that case).
const baseGatewayConfig = cfg ? buildGatewayConfig(cfg) : { env: { ...process.env } };
if (tpArg === 'embedding') {
const dims = recipe?.touchpoints.embedding?.default_dims ?? 1536;
configureGateway({
...baseGatewayConfig,
embedding_model: modelArg,
embedding_dimensions: dims,
env: { ...process.env },
});
} else {
configureGateway({
...baseGatewayConfig,
chat_model: modelArg,
env: { ...process.env },
});
}
void modelId; // intentionally unused but preserved for readability
+240
View File
@@ -22,6 +22,7 @@ import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/
import { ListToolsRequestSchema, CallToolRequestSchema } from '@modelcontextprotocol/sdk/types.js';
import { mcpAuthRouter } from '@modelcontextprotocol/sdk/server/auth/router.js';
import { requireBearerAuth } from '@modelcontextprotocol/sdk/server/auth/middleware/bearerAuth.js';
import { OAuthTokenRevocationRequestSchema } from '@modelcontextprotocol/sdk/shared/auth.js';
import type { BrainEngine } from '../core/engine.ts';
import { operations, OperationError } from '../core/operations.ts';
import type { OperationContext, AuthInfo } from '../core/operations.ts';
@@ -37,6 +38,7 @@ import { VERSION } from '../version.ts';
import * as db from '../core/db.ts';
import { sqlQueryForEngine, executeRawJsonb } from '../core/sql-query.ts';
import { MinionQueue } from '../core/minions/queue.ts';
import { isRetryableError } from '../core/retry-matcher.ts';
import {
computeContentHash,
validateIngestionEvent,
@@ -363,6 +365,42 @@ export interface AgentClientSpend {
inflight_count: number;
}
/**
* `/admin/api/sources` source list the input rows for buildSyncStatusReport.
*
* Queries the JSONB config column directly (listSources doesn't carry it,
* but buildSyncStatusReport needs syncEnabled / strategy fields).
*
* Deliberately does NOT filter on local_path: in a push-only deployment
* (content arrives via MCP put_page / capture / ingest, not `gbrain sync`
* of a server checkout) every source has a null local_path filtering on
* it would empty both the Sources tab AND the federation source-picker.
* buildSyncStatusReport does no disk I/O, so null-local_path sources
* report fine (pages/chunks from SQL, staleness 'unknown' / never-synced).
*/
export async function queryAdminSources(engine: BrainEngine): Promise<
Array<{ id: string; name: string; local_path: string | null; config: Record<string, unknown> }>
> {
const rows = await engine.executeRaw<{
id: string;
name: string;
local_path: string | null;
config: Record<string, unknown> | string | null;
}>(
`SELECT id, name, local_path, config FROM sources
WHERE archived IS NOT TRUE
ORDER BY id`,
);
return rows.map((r) => ({
id: r.id,
name: r.name,
local_path: r.local_path,
config: typeof r.config === 'string'
? (JSON.parse(r.config) as Record<string, unknown>)
: (r.config ?? {}),
}));
}
export async function queryAgentClientSpend(engine: BrainEngine): Promise<AgentClientSpend[]> {
const sql = sqlQueryForEngine(engine);
const rows = await sql`
@@ -745,6 +783,93 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption
}
});
// The SDK's /revoke handler compares the presented secret with
// client.client_secret as plaintext. GBrain stores only a SHA-256 hash, so
// confidential clients need the same hash-aware validation used above for
// authorization_code and refresh_token exchanges. Public clients present no
// secret and continue through to the SDK's PKCE-compatible handler.
app.post('/revoke', ccRateLimiter, express.urlencoded({ extended: false }), async (req, res, next) => {
res.setHeader('Cache-Control', 'no-store');
const rawClientId: unknown = req.body?.client_id;
const rawBodySecret: unknown = req.body?.client_secret;
const authHeader = (req.headers.authorization ?? '').toString();
// RFC 6749 §2.3: one client-authentication method per request. Reject
// duplicates/arrays from express.urlencoded rather than letting them reach
// hashToken() as non-strings and become a misleading invalid_client error.
const hasBasicAuth = /^Basic\b/i.test(authHeader);
if (
(rawClientId !== undefined && typeof rawClientId !== 'string') ||
(rawBodySecret !== undefined && typeof rawBodySecret !== 'string') ||
(hasBasicAuth && (rawClientId !== undefined || rawBodySecret !== undefined))
) {
res.status(400).json({ error: 'invalid_request', error_description: 'Malformed or mixed client authentication' });
return;
}
let clientId = typeof rawClientId === 'string' ? rawClientId : undefined;
let presentedSecret = typeof rawBodySecret === 'string' && rawBodySecret.length > 0
? rawBodySecret
: undefined;
if (hasBasicAuth) {
try {
const match = authHeader.match(/^Basic\s+([^\s]+)$/i);
if (!match) throw new Error('Malformed Basic authentication');
const decoded = Buffer.from(match[1], 'base64').toString('utf8');
const idx = decoded.indexOf(':');
if (idx < 1) throw new Error('Malformed Basic authentication');
clientId = decodeURIComponent(decoded.slice(0, idx).replace(/\+/g, ' '));
presentedSecret = decodeURIComponent(decoded.slice(idx + 1).replace(/\+/g, ' '));
if (!presentedSecret) throw new Error('Malformed Basic authentication');
} catch {
res.setHeader('WWW-Authenticate', 'Basic realm="gbrain"');
res.status(401).json({ error: 'invalid_client', error_description: 'Invalid client' });
return;
}
}
if (!clientId || !presentedSecret) return next();
const parsedRequest = OAuthTokenRevocationRequestSchema.safeParse(req.body);
if (!parsedRequest.success || parsedRequest.data.token.length === 0) {
res.status(400).json({ error: 'invalid_request', error_description: 'Valid token required' });
return;
}
let client;
try {
client = await oauthProvider.verifyConfidentialClientSecret(clientId, presentedSecret);
} catch (e) {
const msg = e instanceof Error ? e.message : '';
if (msg === 'Invalid client' || msg === 'Client has been revoked') {
if (hasBasicAuth) res.setHeader('WWW-Authenticate', 'Basic realm="gbrain"');
res.status(401).json({ error: 'invalid_client', error_description: 'Invalid client' });
return;
}
console.error('[serve-http] revoke client verification failed:', msg || 'Unknown error');
const retryable = isRetryableError(e);
res.status(retryable ? 503 : 500).json({
error: retryable ? 'temporarily_unavailable' : 'server_error',
error_description: retryable ? 'Token revocation temporarily unavailable' : 'Token revocation failed',
});
return;
}
try {
await oauthProvider.revokeToken(client, parsedRequest.data);
// RFC 7009 §2.2: successful revocation, including an unknown token, is 200.
res.status(200).end();
} catch (e) {
const msg = e instanceof Error ? e.message : 'Unknown error';
console.error('[serve-http] token revocation failed:', msg);
const retryable = isRetryableError(e);
res.status(retryable ? 503 : 500).json({
error: retryable ? 'temporarily_unavailable' : 'server_error',
error_description: retryable ? 'Token revocation temporarily unavailable' : 'Token revocation failed',
});
}
});
// ---------------------------------------------------------------------------
// MCP SDK Auth Router (OAuth endpoints)
// ---------------------------------------------------------------------------
@@ -796,6 +921,16 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption
if (body?.grant_types_supported && !body.grant_types_supported.includes('client_credentials')) {
body.grant_types_supported.push('client_credentials');
}
if (body?.token_endpoint_auth_methods_supported) {
for (const method of ['client_secret_basic', 'none']) {
if (!body.token_endpoint_auth_methods_supported.includes(method)) {
body.token_endpoint_auth_methods_supported.push(method);
}
}
}
if (body?.revocation_endpoint_auth_methods_supported && !body.revocation_endpoint_auth_methods_supported.includes('client_secret_basic')) {
body.revocation_endpoint_auth_methods_supported.push('client_secret_basic');
}
return origJson(body);
};
}
@@ -1412,6 +1547,111 @@ export async function runServeHttp(engine: BrainEngine, options: ServeHttpOption
}
});
// ---------------------------------------------------------------------------
// Sources tab — read-only view of registered sources with sync + embed
// coverage stats. Drives the admin SPA's `Sources` page.
//
// Returns the same shape `gbrain sources status --json` prints, so the
// SPA stays in lockstep with the CLI surface.
// ---------------------------------------------------------------------------
app.get('/admin/api/sources', requireAdmin, async (_req: Request, res: Response) => {
try {
const { buildSyncStatusReport } = await import('./sync.ts');
const report = await buildSyncStatusReport(engine, await queryAdminSources(engine));
res.json(report);
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
res.status(503).json({ error: 'service_unavailable', detail: msg });
}
});
// ---------------------------------------------------------------------------
// Federated-read management (admin-side counterparts of the CLI commands
// `gbrain auth grant-read / revoke-read / set-federated-read`). All three
// route through the same *Core helpers as the CLI so race-safety,
// soft-delete filter, and source-id shape validation apply uniformly.
//
// The admin SPA's `Agents` page renders "Manage reads" actions per
// client backed by these endpoints.
// ---------------------------------------------------------------------------
app.get('/admin/api/agents/federated-read', requireAdmin, async (_req: Request, res: Response) => {
try {
const rows = await sql`
SELECT client_id, client_name, source_id, federated_read
FROM oauth_clients
WHERE deleted_at IS NULL
ORDER BY client_name
`;
const clients = rows.map((r) => ({
client_id: String(r.client_id),
client_name: String(r.client_name),
source_id: r.source_id == null ? null : String(r.source_id),
federated_read: Array.isArray(r.federated_read)
? (r.federated_read as string[]).map(String)
: [],
}));
res.json({ clients });
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
res.status(503).json({ error: 'service_unavailable', detail: msg });
}
});
app.post('/admin/api/agents/:clientId/grant-read', requireAdmin, express.json(), async (req: Request, res: Response) => {
const clientId = String(req.params.clientId ?? '');
const sourceId = String(req.body?.source_id ?? '').trim();
if (!clientId || !sourceId) {
res.status(400).json({ error: 'invalid_request', detail: 'clientId path param + source_id body required' });
return;
}
try {
const { grantReadCore } = await import('./auth.ts');
const outcome = await grantReadCore(sql, clientId, sourceId);
res.json({ outcome });
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
res.status(400).json({ error: 'mutation_failed', detail: msg });
}
});
app.post('/admin/api/agents/:clientId/revoke-read', requireAdmin, express.json(), async (req: Request, res: Response) => {
const clientId = String(req.params.clientId ?? '');
const sourceId = String(req.body?.source_id ?? '').trim();
if (!clientId || !sourceId) {
res.status(400).json({ error: 'invalid_request', detail: 'clientId path param + source_id body required' });
return;
}
try {
const { revokeReadCore } = await import('./auth.ts');
const outcome = await revokeReadCore(sql, clientId, sourceId);
res.json({ outcome });
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
res.status(400).json({ error: 'mutation_failed', detail: msg });
}
});
app.post('/admin/api/agents/:clientId/set-federated-read', requireAdmin, express.json(), async (req: Request, res: Response) => {
const clientId = String(req.params.clientId ?? '');
const rawIds = req.body?.source_ids;
if (!clientId || !Array.isArray(rawIds)) {
res.status(400).json({ error: 'invalid_request', detail: 'clientId path param + source_ids[] body required' });
return;
}
// Encode the array as CSV so the same setFederatedReadCore signature
// (string CSV input) the CLI uses applies here. Empty array → empty
// string → clears the list.
const csv = rawIds.map((s) => String(s).trim()).filter(Boolean).join(',');
try {
const { setFederatedReadCore } = await import('./auth.ts');
const outcome = await setFederatedReadCore(sql, clientId, csv);
res.json({ outcome });
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
res.status(400).json({ error: 'mutation_failed', detail: msg });
}
});
// ---------------------------------------------------------------------------
// SSE live activity feed
// ---------------------------------------------------------------------------
+18 -5
View File
@@ -7,7 +7,9 @@
* full story.
*
* Subcommands:
* gbrain sources add <id> --path <path> [--name <display>] [--federated|--no-federated]
* gbrain sources add <id> --path <path> [--name <display>] [--federated|--no-federated] [--force]
* --path must be a git-initialized repo (files committed,
* not just present) #2707. --force skips the check.
* gbrain sources list [--json]
* gbrain sources remove <id> [--yes] [--dry-run] [--keep-storage]
* gbrain sources rename <id> <new-name>
@@ -120,7 +122,7 @@ async function runAdd(engine: BrainEngine, args: string[]): Promise<void> {
if (!id) {
console.error(
'Usage: gbrain sources add <id> [--path <path> | --url <https-url>] ' +
'[--name <display>] [--federated|--no-federated] [--clone-dir <path>]',
'[--name <display>] [--federated|--no-federated] [--clone-dir <path>] [--force]',
);
process.exit(2);
}
@@ -132,6 +134,7 @@ async function runAdd(engine: BrainEngine, args: string[]): Promise<void> {
let cloneDir: string | undefined;
let patFile: string | undefined;
let noHarden = false;
let force = false;
for (let i = 1; i < args.length; i++) {
const a = args[i];
@@ -143,6 +146,7 @@ async function runAdd(engine: BrainEngine, args: string[]): Promise<void> {
if (a === '--clone-dir') { cloneDir = args[++i]; continue; }
if (a === '--pat-file') { patFile = args[++i]; continue; }
if (a === '--no-harden') { noHarden = true; continue; }
if (a === '--force') { force = true; continue; }
console.error(`Unknown flag: ${a}`);
process.exit(2);
}
@@ -162,6 +166,7 @@ async function runAdd(engine: BrainEngine, args: string[]): Promise<void> {
remoteUrl,
federated,
cloneDir,
force,
});
// Topology A discovery: if the just-added source carries a brain-resident
@@ -1178,7 +1183,14 @@ async function runAudit(engine: BrainEngine, args: string[]): Promise<void> {
// frontmatter.type and estimates per-page segment count from body
// bytes. Estimated per-segment Sonnet cost is a rough heuristic
// (~2000 in + 500 out tokens at $3/MTok in + $15/MTok out ≈ $0.013).
const FACTS_BACKFILL_ALLOWED = ['conversation', 'meeting', 'slack', 'email'];
const FACTS_BACKFILL_ALLOWED = [
'conversation',
'meeting',
'slack',
'email',
'imessage',
'imessage-daily',
];
const FACTS_BACKFILL_CHARS_PER_SEGMENT = 6500; // matches SEGMENT_TEXT_CHAR_LIMIT
const FACTS_BACKFILL_USD_PER_SEGMENT = 0.013;
let factsBackfillPages = 0;
@@ -1368,8 +1380,9 @@ function printHelp(): void {
console.log(`gbrain sources — manage multi-source brain configuration (v0.26.5)
Subcommands:
add <id> --path <p> [--name <n>] [--federated|--no-federated]
Register a new source.
add <id> --path <p> [--name <n>] [--federated|--no-federated] [--force]
Register a new source. --path must be a git repo
with committed files; --force skips that check.
list [--json] List registered sources with page counts.
remove <id> [--confirm-destructive] [--dry-run]
Permanently delete a source and all its data.
+43
View File
@@ -90,6 +90,38 @@ export function isValidOpenAITextEmbedding3Dim(modelId: string, dims: number): b
return Number.isInteger(dims) && dims >= 1 && dims <= max;
}
// NVIDIA NIM hosted embedding models use asymmetric input_type values. Most
// emit fixed natural dimensions, but llama-nemotron-embed-1b-v2 accepts
// Matryoshka-style dimension overrides (e.g. matching an existing 1280d
// brain column without re-embedding through another provider).
const NVIDIA_EMBEDDING_DIMS: Record<string, number> = {
'nvidia/nv-embedqa-e5-v5': 1024,
'nvidia/llama-nemotron-embed-1b-v2': 2048,
'nvidia/nv-embed-v1': 4096,
'nvidia/nv-embedcode-7b-v1': 4096,
};
const NVIDIA_EMBEDDING_DIM_OPTIONS: Record<string, number[]> = {
'nvidia/llama-nemotron-embed-1b-v2': [1024, 1280, 1536, 2048],
};
export function isNvidiaEmbeddingModel(modelId: string): boolean {
return modelId in NVIDIA_EMBEDDING_DIMS;
}
export function nvidiaEmbeddingDim(modelId: string): number | undefined {
return NVIDIA_EMBEDDING_DIMS[modelId];
}
export function nvidiaEmbeddingDimOptions(modelId: string): number[] | undefined {
return NVIDIA_EMBEDDING_DIM_OPTIONS[modelId];
}
export function supportsNvidiaEmbeddingDimension(modelId: string, dims: number): boolean {
const options = nvidiaEmbeddingDimOptions(modelId);
return !!options && options.includes(dims);
}
/**
* Build the providerOptions blob for embedMany() that pins output dimensions.
*
@@ -194,6 +226,17 @@ export function dimsProviderOptions(
},
};
}
// NVIDIA NIM hosted embeddings are OpenAI-compatible but require
// asymmetric input_type. Use passage for indexing/document-side vectors
// and query for search-side vectors. Only llama-nemotron-embed-1b-v2
// supports a dimensions override; fixed-dim models reject it.
if (isNvidiaEmbeddingModel(modelId)) {
const opts: Record<string, any> = {
input_type: inputType === 'query' ? 'query' : 'passage',
};
if (supportsNvidiaEmbeddingDimension(modelId, dims)) opts.dimensions = dims;
return { openaiCompatible: opts };
}
// OpenAI text-embedding-3 family on the openai-compatible adapter
// (Azure OpenAI hosts these via its OpenAI-compatible /embeddings
// endpoint). The provider defaults to the model's native size (3072
+55
View File
@@ -53,6 +53,8 @@ import { dimsProviderOptions } from './dims.ts';
import { hasAnthropicKey } from './anthropic-key.ts';
import { AIConfigError, AITransientError, normalizeAIError } from './errors.ts';
import { runGuardrails, hasGuardrails, type GuardrailHook } from '../guardrails.ts';
import { loadConfig } from '../config.ts';
import { buildGatewayConfig } from './build-gateway-config.ts';
// ---- Gateway-wide AI-HTTP timeout (v0.42.20.0, #1762/#1775) ----
//
@@ -117,6 +119,18 @@ const DEFAULT_RERANKER_MODEL = 'zeroentropyai:zerank-2';
let _config: AIGatewayConfig | null = null;
const _modelCache = new Map<string, any>();
/**
* Recover the process-global gateway for foreground command entrypoints that
* were reached without cli.ts's normal engine-connect initialization (#2590).
* Existing configured gateways, including their DB-resolved model overrides,
* are deliberately left unchanged.
*/
export function configureGatewayIfUninitialized(): void {
if (_config) return;
const config = loadConfig();
if (config) configureGateway(buildGatewayConfig(config));
}
/**
* v0.31.12 recipe-models merge: per-gateway-instance set of model ids the
* user opted into via config. Keyed by provider id (`anthropic`, `openai`,
@@ -507,6 +521,20 @@ export async function reconfigureGatewayWithEngine(engine: BrainEngine): Promise
const expansionFull = newExpansion.includes(':') ? newExpansion : prefixWithProviderFrom(cfg.expansion_model ?? DEFAULT_EXPANSION_MODEL, newExpansion);
const chatFull = newChat.includes(':') ? newChat : prefixWithProviderFrom(cfg.chat_model ?? DEFAULT_CHAT_MODEL, newChat);
// ALSO resolve the four tier models and register them as extended models.
// assertTouchpoint's contract (model-resolver.ts) says config-chosen models —
// `models.default` and `models.tier.*` included — bypass the native recipe
// allowlist, but pre-fix only chat/expansion/embedding/reranker were
// registered. A model reachable ONLY through a tier (e.g. `models.tier.deep`
// set to an Opus newer than the recipe list) failed `probeChatModel` at call
// time and silently degraded think/auto_think to the gather-only stub.
// Resolving per-tier also honors `models.default` (it sits above tiers in
// the resolveModel chain).
const tierModels: string[] = [];
for (const tier of ['utility', 'reasoning', 'deep', 'subagent'] as const) {
tierModels.push(await resolveModel(engine, { tier, fallback: TIER_DEFAULTS[tier] }));
}
_config = { ...cfg, expansion_model: expansionFull, chat_model: chatFull };
_modelCache.clear();
_shrinkState.clear();
@@ -518,6 +546,7 @@ export async function reconfigureGatewayWithEngine(engine: BrainEngine): Promise
_config.chat_model,
_config.reranker_model,
...(_config.chat_fallback_chain ?? []),
...tierModels,
]) {
if (m) registerExtendedModel(m);
}
@@ -1045,6 +1074,30 @@ const voyageCompatFetch = (async (input: RequestInfo | URL, init?: RequestInit)
* float[] (not base64), so the Layer 2 cap compares against the JSON
* payload size of each embedding rather than a base64 string length.
*/
/**
* NVIDIA NIM compatibility shim. NVIDIA uses the OpenAI embeddings wire
* shape but requires asymmetric input_type values: query for retrieval and
* passage for indexed documents. The generic gateway store carries
* query/document across the AI SDK boundary; map document to passage here.
*/
const nvidiaCompatFetch = (async (input: RequestInfo | URL, init?: RequestInit) => {
let baseInit: RequestInit = init ?? {};
if (baseInit.body && typeof baseInit.body === 'string') {
try {
const parsed = JSON.parse(baseInit.body);
if (parsed && typeof parsed === 'object' && parsed.input_type === undefined) {
parsed.input_type = __embedInputTypeStore.getStore() === 'query' ? 'query' : 'passage';
const headers = new Headers(baseInit.headers ?? {});
headers.delete('content-length');
baseInit = { ...baseInit, body: JSON.stringify(parsed), headers };
}
} catch {
// Preserve the provider response when the SDK body is unexpectedly non-JSON.
}
}
return fetch(input as any, baseInit);
}) as unknown as typeof fetch;
const zeroEntropyCompatFetch = (async (input: RequestInfo | URL, init?: RequestInit) => {
// OUTBOUND: normalize URL, rewrite path /embeddings → /models/embed, then
// rewrite body. fetch accepts RequestInfo (string | Request) | URL; we
@@ -1305,6 +1358,8 @@ function instantiateEmbedding(recipe: Recipe, modelId: string, cfg: AIGatewayCon
? voyageCompatFetch
: recipe.id === 'zeroentropyai'
? zeroEntropyCompatFetch
: recipe.id === 'nvidia'
? nvidiaCompatFetch
: openAICompatAsymmetricFetch);
const client = createOpenAICompatible({
name: recipe.id,
+4 -1
View File
@@ -17,13 +17,16 @@ export const anthropic: Recipe = {
touchpoints: {
// No embedding model available.
expansion: {
models: ['claude-haiku-4-5-20251001', 'claude-sonnet-4-6'],
models: ['claude-haiku-4-5-20251001', 'claude-sonnet-5', 'claude-sonnet-4-6'],
cost_per_1m_tokens_usd: 0.25,
price_last_verified: '2026-05-10',
},
chat: {
models: [
'claude-fable-5',
'claude-opus-4-8',
'claude-opus-4-7',
'claude-sonnet-5',
'claude-sonnet-4-6',
'claude-haiku-4-5-20251001',
],
+2
View File
@@ -25,6 +25,7 @@ import { zeroentropyai } from './zeroentropyai.ts';
import { llamaServerReranker } from './llama-server-reranker.ts';
import { moonshot } from './moonshot.ts';
import { mistral } from './mistral.ts';
import { nvidia } from './nvidia.ts';
const ALL: Recipe[] = [
openai,
@@ -46,6 +47,7 @@ const ALL: Recipe[] = [
zeroentropyai,
moonshot,
mistral,
nvidia,
];
/** Map from `provider:id` key to recipe. */
+71
View File
@@ -0,0 +1,71 @@
import type { Recipe } from '../types.ts';
/**
* NVIDIA NIM / API Catalog exposes OpenAI-compatible /v1/chat/completions
* and /v1/embeddings APIs.
*
* Retrieval models use asymmetric encoding. The gateway maps gbrain's
* document/query distinction to NVIDIA's wire values:
* document -> input_type: passage
* query -> input_type: query
*
* The model ids below intentionally keep NVIDIA's full catalog ids because
* the hosted endpoint expects values like `nvidia/nv-embedqa-e5-v5` in the
* request body. Short aliases are provided for CLI ergonomics.
*/
export const nvidia: Recipe = {
id: 'nvidia',
name: 'NVIDIA NIM',
tier: 'openai-compat',
implementation: 'openai-compatible',
base_url_default: 'https://integrate.api.nvidia.com/v1',
auth_env: {
required: ['NVIDIA_API_KEY'],
setup_url: 'https://build.nvidia.com',
},
aliases: {
'nv-embedqa-e5-v5': 'nvidia/nv-embedqa-e5-v5',
'llama-nemotron-embed-1b-v2': 'nvidia/llama-nemotron-embed-1b-v2',
'nemotron-3-super': 'nvidia/nemotron-3-super-120b-a12b',
'nemotron-3-super-120b-a12b': 'nvidia/nemotron-3-super-120b-a12b',
'nv-embed-v1': 'nvidia/nv-embed-v1',
'nv-embedcode-7b-v1': 'nvidia/nv-embedcode-7b-v1',
},
// No resolveAuth override: NVIDIA is plain `Authorization: Bearer <key>`,
// which defaultResolveAuth derives from auth_env.required. IRON RULE
// (test/ai/recipes-existing-regression.test.ts): only Azure overrides
// resolveAuth.
touchpoints: {
chat: {
models: [
'nvidia/nemotron-3-super-120b-a12b',
],
supports_tools: false,
supports_subagent_loop: false,
// Do not treat Nemotron as a Minions subagent driver until tool-calling
// and replay stability are proven through a separate adapter test.
max_context_tokens: 128000,
price_last_verified: '2026-05-24',
},
embedding: {
models: [
'nvidia/nv-embedqa-e5-v5',
'nvidia/llama-nemotron-embed-1b-v2',
'nvidia/nv-embed-v1',
'nvidia/nv-embedcode-7b-v1',
],
// Default to the lightest tested hosted model. Larger NVIDIA models are
// supported via explicit embedding_dimensions (2048 or 4096).
default_dims: 1024,
dims_options: [1024, 2048, 4096],
// Conservative split; hosted NVIDIA embedding endpoints require
// input_type and may reject large payloads before tokenizing.
max_batch_tokens: 8192,
chars_per_token: 4,
safety_factor: 0.75,
cost_per_1m_tokens_usd: undefined,
price_last_verified: '2026-05-24',
},
},
setup_hint: 'Get an API key at https://build.nvidia.com, then `export NVIDIA_API_KEY=...`.',
};
+37 -2
View File
@@ -1,7 +1,7 @@
/**
* v0.41.16.0 Built-in conversation parser pattern registry.
*
* Fourteen hand-vetted patterns covering the chat-export formats this
* Fifteen hand-vetted patterns covering the chat-export formats this
* codebase is most likely to encounter. Each pattern's regex was
* derived from a public format reference (source_doc field) so future
* maintainers can verify against the wild shape.
@@ -50,7 +50,7 @@ export function cleanSpeaker(raw: string, override?: RegExp): string {
return stripped || raw.trim();
}
/** The 14 hand-vetted built-in patterns. */
/** The 15 hand-vetted built-in patterns. */
export const BUILTIN_PATTERNS: readonly PatternEntry[] = [
// -------------------------------------------------------------------
// INLINE-DATE patterns (date in every line; less ambiguous; tried first).
@@ -178,6 +178,41 @@ export const BUILTIN_PATTERNS: readonly PatternEntry[] = [
'OpenClaw meeting-ingestion pipeline reformat of Circleback transcripts (see your OpenClaw skills/meeting-ingestion/SKILL.md)',
},
{
// iMessage sync's time-only 12-hour shape. AM/PM is required so this
// cannot shadow bold-paren-time's 24-hour form or imessage-slack's
// full-date form.
id: 'bold-paren-time-12h',
origin: 'builtin',
regex: /^\*\*(.+?)\*\*\s*\((\d{1,2}):(\d{2})\s*(AM|PM|am|pm)\)\s*:\s*(.*)$/,
captures: {
speaker_group: 1,
hour_group: 2,
minute_group: 3,
ampm_group: 4,
text_group: 5,
},
date_source: 'frontmatter',
time_format: '12h_ampm',
timezone_policy: 'utc_assumed_with_warn',
multi_line: false,
quick_reject: /^\*\*/,
test_positive: [
'**Me** (9:04 AM): sounds good, see you then',
'**+155****0135** (9:39 AM): Will do',
'**Alice Example** (12:00 PM): noon message',
'**Bob Example** (5:38 pm): lowercase ampm',
],
test_negative: [
'**Alice** (00:00): 24h shape',
'**Alice Example** (2024-03-15 9:00 AM): full-date iMessage shape',
'**[18:37] G T:** telegram bracket',
'Alice (9:00 AM): missing the bold',
],
source_doc:
'Time-only 12h AM/PM iMessage export shape: `**Speaker** (H:MM AM): text`',
},
{
// Fathom/phone-call raw transcripts in this workspace use a plain
// `Speaker A: ...` / `Speaker B: ...` shape with no per-line time.
+13 -2
View File
@@ -321,11 +321,22 @@ export function applyPattern(
if (!body) return [];
const out: MatchedMessage[] = [];
const lines = body.split(/\r?\n/);
// Some multi-day conversation exports use markdown date headings instead
// of repeating a date on every message. Keep the caller's context immutable
// while advancing a local date anchor as those headings are encountered.
const runningCtx: DateContext = { ...dateCtx };
const dateHeaderRe = /^#{1,4}\s+(\d{4}-\d{2}-\d{2})\s*$/;
for (let i = 0; i < lines.length; i++) {
const rawLine = lines[i];
const line = rawLine.trim();
if (!line) continue;
const dateHeader = dateHeaderRe.exec(line);
if (dateHeader) {
runningCtx.fallbackDate = dateHeader[1];
continue;
}
// Quick-reject fast path.
if (entry.quick_reject && !entry.quick_reject.test(line)) {
// Continuation handling for orphan lines.
@@ -339,7 +350,7 @@ export function applyPattern(
const m = entry.regex.exec(line);
if (m) {
const iso = buildIso(m, entry, dateCtx);
const iso = buildIso(m, entry, runningCtx);
if (iso === null) continue; // reconstruction failed; skip line
const rawSpeaker = m[entry.captures.speaker_group] ?? '';
const speaker = cleanSpeaker(rawSpeaker, entry.speaker_clean);
@@ -380,7 +391,7 @@ function getNonBlankLines(body: string, headCap?: number): string[] {
* window) and `scorePatternFull` (whole body) delegate here so the
* quick_reject + regex loop lives in one place. Reused by
* `parseConversation`'s fallback path which pre-splits ONCE and
* passes the array to all 12 candidates (saves 11 redundant body
* passes the array to all 15 candidates (saves 14 redundant body
* splits per fallback pass).
*/
function scoreFromLines(
+34 -2
View File
@@ -479,6 +479,27 @@ export interface CycleOpts {
* Validated via `assertValidSourceId` in `cycleLockIdFor` (defense-in-depth).
*/
sourceId?: string;
/**
* issue #2860 one-shot per-invocation bypass of a phase's own
* `dream.<phase>.enabled` / `cycle.<phase>.enabled` config gate. Wired
* from `gbrain dream --phase <name> --once`.
*
* Deliberately typed as the SINGLE named `CyclePhase`, not a boolean
* each gated phase's dispatch block below only honors the override when
* `onceForPhase` matches ITS OWN phase name, so the bypass can never leak
* to a different phase even if a caller passes a wider `phases` array
* than the CLI does (the CLI always restricts to `phases: [phase]`).
*
* Never reads or writes config the phase still evaluates its config
* gate every call; this only overrides the boolean OUTCOME for that one
* call. Applies to: patterns, synthesize, conversation_facts_backfill,
* enrich_thin, skillopt (the phases that gate on a `.enabled` config
* key read inside the phase's own module). Does NOT apply to
* extract_atoms / synthesize_concepts those are pack-gated via
* `packDeclaresPhase`, a different mechanism with its own existing
* one-shot escape hatch (`--drain` for extract_atoms).
*/
onceForPhase?: CyclePhase;
/**
* Absolute wall-clock deadline (epoch ms) of the enclosing minion job,
* from `MinionJobContext.deadlineAtMs` (the claim-time `timeout_at`
@@ -1695,6 +1716,7 @@ export async function runCycle(
// #1586: scope synthesized writes to the cycle's resolved source
// (explicit --source wins, else derived from the checkout dir).
sourceId: cycleSourceId,
once: opts.onceForPhase === 'synthesize',
}));
result.duration_ms = duration_ms;
phaseResults.push(result);
@@ -1898,6 +1920,7 @@ export async function runCycle(
brainDir,
dryRun,
yieldDuringPhase: opts.yieldDuringPhase,
once: opts.onceForPhase === 'patterns',
deadlineAtMs: opts.deadlineAtMs ?? null,
}));
result.duration_ms = duration_ms;
@@ -2114,7 +2137,11 @@ export async function runCycle(
progress.start('cycle.conversation_facts_backfill');
const { runPhaseConversationFactsBackfill } = await import('./cycle/conversation-facts-backfill.ts');
const { result, duration_ms } = await timePhase(() =>
runPhaseConversationFactsBackfill(engine, { dryRun, signal: opts.signal }),
runPhaseConversationFactsBackfill(engine, {
dryRun,
signal: opts.signal,
once: opts.onceForPhase === 'conversation_facts_backfill',
}),
);
result.duration_ms = duration_ms;
phaseResults.push(result);
@@ -2142,7 +2169,11 @@ export async function runCycle(
progress.start('cycle.enrich_thin');
const { runPhaseEnrichThin } = await import('./cycle/enrich-thin.ts');
const { result, duration_ms } = await timePhase(() =>
runPhaseEnrichThin(engine, { dryRun, signal: opts.signal }),
runPhaseEnrichThin(engine, {
dryRun,
signal: opts.signal,
once: opts.onceForPhase === 'enrich_thin',
}),
);
result.duration_ms = duration_ms;
phaseResults.push(result);
@@ -2173,6 +2204,7 @@ export async function runCycle(
runPhaseSkillopt({
engine,
dryRun,
once: opts.onceForPhase === 'skillopt',
...(opts.signal ? { signal: opts.signal } : {}),
}),
);
+25 -11
View File
@@ -57,6 +57,14 @@ import {
export interface ConversationFactsBackfillPhaseOpts {
dryRun?: boolean;
signal?: AbortSignal;
/**
* issue #2860 `gbrain dream --phase conversation_facts_backfill --once`.
* Bypasses the `cycle.conversation_facts_backfill.enabled` gate for THIS
* call only; never reads or writes config. Per-source + brain-wide cost/
* walltime caps still apply the override lifts the on/off switch, not
* the spend guards.
*/
once?: boolean;
}
/** Phase return shape (matches PhaseResult contract from cycle.ts). */
@@ -155,17 +163,23 @@ export async function runPhaseConversationFactsBackfill(
const cfg = await loadCfg(engine);
if (!cfg.enabled) {
return {
phase: 'conversation_facts_backfill',
status: 'skipped',
duration_ms: 0,
summary: 'cycle.conversation_facts_backfill.enabled=false (default OFF)',
details: {
reason: 'disabled',
enable_hint:
'gbrain config set cycle.conversation_facts_backfill.enabled true',
},
};
if (!opts.once) {
return {
phase: 'conversation_facts_backfill',
status: 'skipped',
duration_ms: 0,
summary: 'cycle.conversation_facts_backfill.enabled=false (default OFF)',
details: {
reason: 'disabled',
enable_hint:
'gbrain config set cycle.conversation_facts_backfill.enabled true',
},
};
}
process.stderr.write(
'[dream] --once: cycle.conversation_facts_backfill.enabled is false but ' +
'--phase conversation_facts_backfill --once forces this run (config untouched)\n',
);
}
const startedAt = Date.now();
+22 -10
View File
@@ -45,6 +45,12 @@ import {
export interface EnrichThinPhaseOpts {
dryRun?: boolean;
signal?: AbortSignal;
/**
* issue #2860 `gbrain dream --phase enrich_thin --once`. Bypasses the
* `cycle.enrich_thin.enabled` gate for THIS call only; never reads or
* writes config. Per-source + brain-wide cost/walltime caps still apply.
*/
once?: boolean;
}
export interface EnrichThinPhaseResult {
@@ -139,16 +145,22 @@ export async function runPhaseEnrichThin(
const cfg = await loadCfg(engine);
if (!cfg.enabled) {
return {
phase: 'enrich_thin',
status: 'skipped',
duration_ms: 0,
summary: 'cycle.enrich_thin.enabled=false (default OFF)',
details: {
reason: 'disabled',
enable_hint: 'gbrain config set cycle.enrich_thin.enabled true',
},
};
if (!opts.once) {
return {
phase: 'enrich_thin',
status: 'skipped',
duration_ms: 0,
summary: 'cycle.enrich_thin.enabled=false (default OFF)',
details: {
reason: 'disabled',
enable_hint: 'gbrain config set cycle.enrich_thin.enabled true',
},
};
}
process.stderr.write(
'[dream] --once: cycle.enrich_thin.enabled is false but ' +
'--phase enrich_thin --once forces this run (config untouched)\n',
);
}
const startedAt = Date.now();
+13 -1
View File
@@ -37,6 +37,12 @@ export interface PatternsPhaseOpts {
brainDir: string;
dryRun: boolean;
yieldDuringPhase?: () => Promise<void>;
/**
* issue #2860 `gbrain dream --phase patterns --once`. Bypasses the
* `dream.patterns.enabled` gate for THIS call only; never reads or
* writes config.
*/
once?: boolean;
/**
* Absolute deadline (epoch ms) of the enclosing minion job, or null for
* direct callers (`gbrain dream`). When set, the subagent's job timeout
@@ -99,7 +105,13 @@ export async function runPhasePatterns(
const config = await loadPatternsConfig(engine);
if (!config.enabled) {
return skipped('disabled', 'dream.patterns.enabled is false');
if (!opts.once) {
return skipped('disabled', 'dream.patterns.enabled is false');
}
process.stderr.write(
'[dream] --once: dream.patterns.enabled is false but ' +
'--phase patterns --once forces this run (config untouched)\n',
);
}
// Gather reflections within lookback window.
+5 -3
View File
@@ -39,7 +39,7 @@
import { randomUUID, createHash } from 'node:crypto';
import { BaseCyclePhase, type ScopedReadOpts, type BasePhaseOpts } from './base-phase.ts';
import { chat as gatewayChat } from '../ai/gateway.ts';
import { chat as gatewayChat, getChatModel } from '../ai/gateway.ts';
import { writeReceipt } from '../extract/receipt-writer.ts';
import { upsertExtractRollup } from '../extract/rollup-writer.ts';
import { GBrainError } from '../types.ts';
@@ -330,6 +330,8 @@ class ProposeTakesPhase extends BaseCyclePhase {
opts.reporter.start('propose_takes.pages' as never, pages.length);
}
const modelId = opts.model ?? getChatModel();
for (const page of pages) {
result.pages_scanned += 1;
this.tick(opts);
@@ -359,7 +361,7 @@ class ProposeTakesPhase extends BaseCyclePhase {
// Budget pre-check before the LLM call. Estimate: ~1500 input tokens + 500 output.
const budget = this.checkBudget({
modelId: opts.model ?? 'claude-sonnet-4-6',
modelId,
estimatedInputTokens: 1500,
maxOutputTokens: 500,
});
@@ -408,7 +410,7 @@ class ProposeTakesPhase extends BaseCyclePhase {
p.weight,
p.domain ?? null,
JSON.stringify(existingTakes),
opts.model ?? 'claude-sonnet-4-6',
modelId,
],
);
result.proposals_inserted += 1;
+16 -2
View File
@@ -252,6 +252,13 @@ export interface SynthesizePhaseOpts {
* correct (source_id, slug) row. Unset legacy 'default'.
*/
sourceId?: string;
/**
* issue #2860 `gbrain dream --phase synthesize --once`. Bypasses the
* `dream.synthesize.enabled` gate for THIS call only (does NOT bypass
* the `session_corpus_dir` not-configured check there's nothing to
* run without a corpus). Never reads or writes config.
*/
once?: boolean;
}
export async function runPhaseSynthesize(
@@ -285,8 +292,14 @@ export async function runPhaseSynthesize(
'dream.synthesize.session_corpus_dir is unset');
}
if (!opts.inputFile && !config.enabled) {
return skipped('not_configured',
'dream.synthesize.enabled is explicitly false');
if (!opts.once) {
return skipped('not_configured',
'dream.synthesize.enabled is explicitly false');
}
process.stderr.write(
'[dream] --once: dream.synthesize.enabled is false but ' +
'--phase synthesize --once forces this run (config untouched)\n',
);
}
// Cooldown check (skipped for explicit --input / --date / --from / --to runs).
@@ -1041,6 +1054,7 @@ OUTPUT POLICY (ALL of these are required)
2. Cross-reference compulsively: every new page MUST contain at least one wikilink (e.g., \`[ref](people/jane-doe)\` or \`[[people/jane-doe]]\`) to existing brain content. Use the search tool to find existing pages first.
3. Do NOT write to any path outside the allow-list shown in the put_page schema.
4. Slug discipline: lowercase alphanumeric and hyphens only, slash-separated segments. NO underscores, NO file extensions.
5. Self-contained opening: begin every new page's body with a 2-3 sentence summary that a reader unfamiliar with this transcript could understand on its own, before any quotes or detail. Do not assume the reader has the source conversation for context.
TASKS
A. Reflections (self-knowledge, pattern recognition, emotional processing):
+16
View File
@@ -206,6 +206,22 @@ export async function embedStaleForSource(
chunk_source: c.chunk_source,
embedding: staleIdxToEmbedding.get(c.chunk_index) ?? undefined,
token_count: c.token_count || Math.ceil(c.chunk_text.length / 4),
// Carry through per-chunk metadata. upsertChunks writes these as
// EXCLUDED.<col> (not COALESCE), so omitting them here resets image
// rows to modality='text' (breaking the image search arm's
// modality='image' filter) and wipes code-chunk symbol metadata on
// every embed-stale pass. embedding_image is deliberately NOT
// carried: the upsert COALESCEs it, and getChunks returns the
// pgvector as a string which upsertChunks would mis-serialize.
modality: c.modality ?? undefined,
language: c.language ?? undefined,
symbol_name: c.symbol_name ?? undefined,
symbol_type: c.symbol_type ?? undefined,
start_line: c.start_line ?? undefined,
end_line: c.end_line ?? undefined,
parent_symbol_path: c.parent_symbol_path ?? undefined,
doc_comment: c.doc_comment ?? undefined,
symbol_name_qualified: c.symbol_name_qualified ?? undefined,
}));
await observed(pacer, () => engine.upsertChunks(slug, merged, { sourceId: keySourceId }));
// v0.41.31: stamp provenance only when EVERY chunk was stale (fully
+23 -2
View File
@@ -29,6 +29,9 @@ import {
isOpenAITextEmbedding3Model,
isValidOpenAITextEmbedding3Dim,
maxOpenAITextEmbedding3Dim,
nvidiaEmbeddingDim,
nvidiaEmbeddingDimOptions,
supportsNvidiaEmbeddingDimension,
} from './ai/dims.ts';
/**
@@ -366,7 +369,9 @@ function validateDimAgainstTouchpoint(
dimsOptions: number[] | undefined,
requestedDims: number | undefined,
): ResolveSchemaDimResult {
const dim = requestedDims ?? defaultDims;
const nvidiaNaturalDims = recipe.id === 'nvidia' ? nvidiaEmbeddingDim(modelId) : undefined;
const effectiveDefaultDims = nvidiaNaturalDims ?? defaultDims;
const dim = requestedDims ?? effectiveDefaultDims;
if (!Number.isInteger(dim) || dim <= 0) {
return {
@@ -396,7 +401,7 @@ function validateDimAgainstTouchpoint(
dim,
model: `${recipe.id}:${modelId}`,
provider: recipe.id,
recipeDefault: defaultDims,
recipeDefault: effectiveDefaultDims,
};
}
@@ -411,6 +416,22 @@ function isCustomDimValidForProvider(
requestedDims: number,
dimsOptions: number[] | undefined,
): CustomDimCheck {
// NVIDIA models are mixed: some fixed-dim, one Matryoshka-style. Handle
// them before generic recipe dims_options so llama-nemotron can use 1280d.
if (recipe.id === 'nvidia') {
const naturalDims = nvidiaEmbeddingDim(modelId);
if (naturalDims !== undefined && requestedDims === naturalDims) return { valid: true, error: '' };
if (supportsNvidiaEmbeddingDimension(modelId, requestedDims)) return { valid: true, error: '' };
const options = nvidiaEmbeddingDimOptions(modelId);
return {
valid: false,
error:
`NVIDIA model "${modelId}" does not support dimensions ${requestedDims}. ` +
`Natural dimensions: ${naturalDims ?? 'unknown'}. ` +
(options ? `Supported overrides: ${options.join(', ')}.` : 'No dimension overrides are supported for this NVIDIA model.'),
};
}
// Tier 1: recipe-declared dims_options.
if (dimsOptions && dimsOptions.length > 0) {
if (dimsOptions.includes(requestedDims)) return { valid: true, error: '' };
+77
View File
@@ -303,6 +303,83 @@ export function validateRepoState(
return 'healthy';
}
/**
* True if `path` is itself a git repo OR a subdirectory of one, per
* `git rev-parse --show-toplevel`. Mirrors the walk-up discovery
* `sync.ts:discoverGitRoot` performs at sync time (#753/#774 subdir-of-git
* sources are valid), so a directory that passes this check is guaranteed
* not to hit sync's "Not inside a git repository" error later. Used by
* `addSource` (#2707) to validate `--path` at registration time instead of
* deferring the failure to the first sync.
*/
export function isInsideGitRepo(path: string): boolean {
try {
execFileSync('git', ['-C', path, 'rev-parse', '--show-toplevel'], {
stdio: ['ignore', 'pipe', 'pipe'],
timeout: 10_000,
env: { ...process.env, ...GIT_ENV },
});
return true;
} catch {
return false;
}
}
/**
* The empty-tree object ID for `path`'s repo, derived (not hardcoded) so
* this works for both the default SHA-1 object format and the opt-in
* `--object-format=sha256` one (git 2.29+) each has its own empty-tree
* OID. `git hash-object -t tree --stdin < /dev/null` computes the hash of
* a zero-entry tree using whatever hash algorithm `path`'s repo is
* configured for, without needing to know which one that is. #2707 codex
* round 4 (P2): an earlier version hardcoded the well-known SHA-1 constant
* (`4b825dc6...`), which silently mismatched and so let an empty
* SHA-256 repo through on a SHA-256 repo's real (different) empty-tree
* OID.
*/
function emptyTreeOid(path: string): string {
return execFileSync('git', ['-C', path, 'hash-object', '-t', 'tree', '--stdin'], {
input: '',
stdio: ['pipe', 'pipe', 'pipe'],
timeout: 10_000,
env: { ...process.env, ...GIT_ENV },
}).toString().trim();
}
/**
* True if `path`'s HEAD tree has at least one tracked entry scoped to
* `path` itself. `-C path` + the `HEAD:./` revision syntax resolves the
* tree object for `path` specifically (not the whole repo root), so this
* is correct for both a repo's toplevel AND a subdirectory-of-a-repo
* source then a single OID comparison against that repo's empty-tree
* object (see `emptyTreeOid`) tells us whether that tree is empty. #2707
* codex round 3 (P2): unlike listing (`ls-tree`), this is O(1) output no
* `maxBuffer` exposure on a repo with a very large number of entries.
*
* Subsumes "no commits at all" (`HEAD:./` on an unborn repo fails to
* resolve there's no HEAD) AND "has a HEAD commit but it's empty"
* (#2707 codex round 2): `git commit --allow-empty` followed by creating
* untracked files resolves `HEAD:./` successfully (to the empty-tree OID)
* but that tree has zero entries a directory that would pass a bare
* `rev-parse HEAD` check yet still can't sync (or worse, "succeeds"
* importing nothing and then never notices the untracked files change
* the silent-staleness class #2707 exists to prevent). A directory
* that's `git init`ed but never committed, or where this specific path
* was never `git add`ed, fails this check either way.
*/
export function hasTrackedContent(path: string): boolean {
try {
const out = execFileSync('git', ['-C', path, 'rev-parse', '--verify', 'HEAD:./'], {
stdio: ['ignore', 'pipe', 'pipe'],
timeout: 10_000,
env: { ...process.env, ...GIT_ENV },
});
return out.toString().trim() !== emptyTreeOid(path);
} catch {
return false;
}
}
// ── Durability helpers (v0.42.44) ───────────────────────────────────────────
// Used by the brain-repo durability feature (`gbrain sources harden/pull`) and
// the DB-free pull cron. These are the auth-capable, rebase-aware counterparts
+2 -1
View File
@@ -5667,7 +5667,8 @@ export const MIGRATIONS: Migration[] = [
END;
$fn$ LANGUAGE plpgsql;
`);
console.log(` v124: update_page_search_vector() no longer indexes compiled_truth (was overflowing tsvector on large pages, #2704)`);
process.stderr.write(` v124: update_page_search_vector() no longer indexes compiled_truth (was overflowing tsvector on large pages, #2704)
`);
},
},
];
+17 -5
View File
@@ -163,24 +163,36 @@ export class MinionQueue {
if (opts?.maxWaiting !== undefined) {
const maxWaiting = Math.max(1, Math.floor(opts.maxWaiting));
const backpressureQueue = opts?.queue ?? 'default';
// Multi-source scope: jobs of the same (name, queue) but different
// data.sourceId are independent workstreams (per-source sync/cycle).
// Counting them together made a waiting default-source sync swallow
// every other source's freshness sync — a secondary source sat 29h stale
// while dispatch logs showed its syncs "dispatched" (coalesced into
// the default row). Key the lock and the count on sourceId when the
// submission carries one; NULL keeps legacy single-scope behavior.
const bpSourceId = typeof (data as Record<string, unknown> | undefined)?.sourceId === 'string'
? (data as Record<string, unknown>).sourceId as string
: null;
await tx.executeRaw(
`SELECT pg_advisory_xact_lock(hashtext('minion_maxwaiting:' || $1 || ':' || $2))`,
[jobName, backpressureQueue]
`SELECT pg_advisory_xact_lock(hashtext('minion_maxwaiting:' || $1 || ':' || $2 || ':' || coalesce($3, '')))`,
[jobName, backpressureQueue, bpSourceId]
);
const waitingCountRows = await tx.executeRaw<{ count: string }>(
`SELECT count(*)::text AS count
FROM minion_jobs
WHERE name = $1 AND queue = $2 AND status = 'waiting'`,
[jobName, backpressureQueue]
WHERE name = $1 AND queue = $2 AND status = 'waiting'
AND ($3::text IS NULL OR data->>'sourceId' IS NOT DISTINCT FROM $3)`,
[jobName, backpressureQueue, bpSourceId]
);
const waitingCount = parseInt(waitingCountRows[0]?.count ?? '0', 10);
if (waitingCount >= maxWaiting) {
const existingWaiting = await tx.executeRaw<Record<string, unknown>>(
`SELECT * FROM minion_jobs
WHERE name = $1 AND queue = $2 AND status = 'waiting'
AND ($3::text IS NULL OR data->>'sourceId' IS NOT DISTINCT FROM $3)
ORDER BY created_at DESC, id DESC
LIMIT 1`,
[jobName, backpressureQueue]
[jobName, backpressureQueue, bpSourceId]
);
if (existingWaiting.length > 0) {
const coalesced = rowToMinionJob(existingWaiting[0]);
+1 -1
View File
@@ -55,7 +55,7 @@ export interface AgentClientBindings {
* `redirect_uri` containing `,`) would be parsed by Postgres as MULTIPLE
* array elements, smuggling values past validation. See CSO finding #5.
*/
function pgArray(arr: string[]): string {
export function pgArray(arr: string[]): string {
if (!arr || arr.length === 0) return '{}';
const escaped = arr.map(s => `"${s.replace(/\\/g, '\\\\').replace(/"/g, '\\"')}"`);
return `{${escaped.join(',')}}`;
+9 -1
View File
@@ -23,6 +23,7 @@ import { runMigrations } from './migrate.ts';
import { PGLITE_SCHEMA_SQL, getPGLiteSchema } from './pglite-schema.ts';
import { DEFAULT_EMBEDDING_MODEL, DEFAULT_EMBEDDING_DIMENSIONS } from './ai/defaults.ts';
import { DELETE_BATCH_SIZE } from './engine-constants.ts';
import { MARKDOWN_CHUNKER_VERSION } from './chunkers/recursive.ts';
import { acquireLock, releaseLock, type LockHandle } from './pglite-lock.ts';
import { getFtsLanguage } from './fts-language.ts';
import type {
@@ -1035,7 +1036,7 @@ export class PGLiteEngine implements BrainEngine {
const ingestedAt = (sourceKind || sourceUri || ingestedVia) ? new Date().toISOString() : null;
const { rows } = await this.db.query(
`INSERT INTO pages (source_id, slug, type, page_kind, title, compiled_truth, timeline, frontmatter, content_hash, updated_at, effective_date, effective_date_source, import_filename, chunker_version, source_path, source_kind, source_uri, ingested_via, ingested_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8::jsonb, $9, now(), $10::timestamptz, $11, $12, COALESCE($13, 1), $14, $15, $16, $17, $18::timestamptz)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8::jsonb, $9, now(), $10::timestamptz, $11, $12, COALESCE($13, ${MARKDOWN_CHUNKER_VERSION}), $14, $15, $16, $17, $18::timestamptz)
ON CONFLICT (source_id, slug) DO UPDATE SET
type = EXCLUDED.type,
page_kind = EXCLUDED.page_kind,
@@ -3674,6 +3675,13 @@ export class PGLiteEngine implements BrainEngine {
THEN ep.frontmatter->'event'->'who' ELSE '[]'::jsonb END
) AS w(name) WHERE w.name = $1 OR w.name LIKE $2)))`,
];
// "Last seen" is a PAST relation: chronicle stores future events
// (calendar-event is eligible), which must not read as "last seen".
// Bound to <= asof/today, mirroring getOnThisDay's `te.date < target`.
let seenThrough: string;
if (opts?.asof) { params.push(opts.asof); seenThrough = `$${params.length}::date`; }
else { seenThrough = `current_date`; }
where.push(`te.date <= ${seenThrough}`);
this.pushChronicleSource(where, params, opts);
const result = await this.db.query(
`SELECT te.date::text AS last_date, ep.slug AS last_event_slug
+8 -1
View File
@@ -35,6 +35,7 @@ import {
EmbeddingColumnNotRegisteredError,
} from './search/embedding-column.ts';
import { getFtsLanguage } from './fts-language.ts';
import { MARKDOWN_CHUNKER_VERSION } from './chunkers/recursive.ts';
import type {
Page, PageInput, PageFilters, PageType,
Chunk, ChunkInput, StaleChunkRow, StalePageRow,
@@ -1097,7 +1098,7 @@ export class PostgresEngine implements BrainEngine {
const ingestedAt = (sourceKind || sourceUri || ingestedVia) ? new Date() : null;
const rows = await sql`
INSERT INTO pages (source_id, slug, type, page_kind, title, compiled_truth, timeline, frontmatter, content_hash, updated_at, effective_date, effective_date_source, import_filename, chunker_version, source_path, source_kind, source_uri, ingested_via, ingested_at)
VALUES (${sourceId}, ${slug}, ${page.type}, ${pageKind}, ${page.title}, ${page.compiled_truth}, ${page.timeline || ''}, ${sql.json(frontmatter as Parameters<typeof sql.json>[0])}, ${hash}, now(), ${effectiveDate}, ${effectiveDateSource}, ${importFilename}, COALESCE(${chunkerVersion}::smallint, 1), ${sourcePath}, ${sourceKind}, ${sourceUri}, ${ingestedVia}, ${ingestedAt})
VALUES (${sourceId}, ${slug}, ${page.type}, ${pageKind}, ${page.title}, ${page.compiled_truth}, ${page.timeline || ''}, ${sql.json(frontmatter as Parameters<typeof sql.json>[0])}, ${hash}, now(), ${effectiveDate}, ${effectiveDateSource}, ${importFilename}, COALESCE(${chunkerVersion}::smallint, ${MARKDOWN_CHUNKER_VERSION}), ${sourcePath}, ${sourceKind}, ${sourceUri}, ${ingestedVia}, ${ingestedAt})
ON CONFLICT (source_id, slug) DO UPDATE SET
type = EXCLUDED.type,
page_kind = EXCLUDED.page_kind,
@@ -3825,12 +3826,18 @@ export class PostgresEngine implements BrainEngine {
const sql = this.sql;
// "Seen" = the entity's own page has a timeline row, OR an event's `who`
// array references the entity (exact slug or wikilink-substring match).
// "Last seen" is a PAST relation: the chronicle legitimately stores
// future events (calendar-event is eligibility-eligible), so bound to
// <= asof/today or a scheduled event reads as "seen today". Mirrors
// getOnThisDay's `te.date < target` bound.
const seenThrough = opts?.asof ? sql`${opts.asof}::date` : sql`current_date`;
const rows = await sql`
SELECT te.date::text AS last_date, ep.slug AS last_event_slug
FROM timeline_entries te
JOIN pages p ON p.id = te.page_id AND p.deleted_at IS NULL
LEFT JOIN pages ep ON ep.id = te.event_page_id
WHERE (te.event_page_id IS NULL OR ep.deleted_at IS NULL)
AND te.date <= ${seenThrough}
AND (
p.slug = ${entitySlug}
OR (ep.id IS NOT NULL AND EXISTS (
+19 -7
View File
@@ -29,6 +29,12 @@ export interface SkilloptPhaseOpts {
engine: BrainEngine;
dryRun?: boolean;
signal?: AbortSignal;
/**
* issue #2860 `gbrain dream --phase skillopt --once`. Bypasses the
* `cycle.skillopt.enabled` feature flag for THIS call only; never reads
* or writes config. Per-skill + brain-wide cost caps still apply.
*/
once?: boolean;
}
export interface SkilloptPhaseResult {
@@ -63,13 +69,19 @@ export async function runPhaseSkillopt(opts: SkilloptPhaseOpts): Promise<Skillop
enabled = v === 'true';
} catch { /* default OFF */ }
if (!enabled) {
return {
phase: 'skillopt',
status: 'skipped',
duration_ms: Date.now() - start,
summary: 'feature flag off (gbrain config set cycle.skillopt.enabled true to enable)',
details: { reason: 'feature_flag_off' },
};
if (!opts.once) {
return {
phase: 'skillopt',
status: 'skipped',
duration_ms: Date.now() - start,
summary: 'feature flag off (gbrain config set cycle.skillopt.enabled true to enable)',
details: { reason: 'feature_flag_off' },
};
}
process.stderr.write(
'[dream] --once: cycle.skillopt.enabled is false but ' +
'--phase skillopt --once forces this run (config untouched)\n',
);
}
// Per-skill + brain-wide cost caps.
+68 -1
View File
@@ -45,6 +45,8 @@ import {
parseRemoteUrl,
cloneRepo,
validateRepoState,
isInsideGitRepo,
hasTrackedContent,
RemoteUrlError,
GitOperationError,
type RepoState,
@@ -67,7 +69,8 @@ export type SourceOpErrorCode =
| 'protected_id'
| 'clone_dir_outside_gbrain'
| 'symlink_escape'
| 'unmanaged_path';
| 'unmanaged_path'
| 'not_a_git_repo';
export class SourceOpError extends Error {
constructor(
@@ -145,6 +148,13 @@ export interface AddSourceOpts {
* Only honored when remoteUrl is set.
*/
cloneDir?: string;
/**
* Skip the #2707 git-repo validation on `localPath`. Opt-in escape hatch
* for registering a path before it's git-initialized (e.g. an automated
* pipeline that populates + `git init`s the directory after `sources add`
* runs). Does NOT auto-`git init` anything see `addSource` docstring.
*/
force?: boolean;
}
export interface RemoveSourceOpts {
@@ -157,6 +167,20 @@ export interface RemoveSourceOpts {
// ── Helpers ─────────────────────────────────────────────────────────────────
/**
* POSIX single-quote `arg` unless it's already shell-safe. #2707 codex round
* 1: the `not_a_git_repo` remediation error prints a pasteable `git ...`
* command built from the caller-supplied path spaces, `$()`, backticks,
* etc. must be inert literals when pasted, which double-quoting would not
* guarantee (command substitution still runs inside "..."). Mirrors
* `src/commands/connect.ts:shellQuote` (not imported that file is a
* commands/ caller of core/, not the other way around).
*/
function shellQuote(arg: string): string {
if (/^[A-Za-z0-9_.:/@-]+$/.test(arg)) return arg;
return `'${arg.replace(/'/g, "'\\''")}'`;
}
/**
* Validate via the canonical regex from `source-id.ts` but rethrow as the
* sources-ops-tagged error so `gbrain sources add` keeps its user-facing
@@ -310,6 +334,18 @@ export function unownedHint(
// ── addSource ───────────────────────────────────────────────────────────────
/**
* #2707: `--path` registration used to accept any existing directory with
* zero git validation, deferring the failure to the first `gbrain sync`
* ("Not inside a git repository: ..."). By the time that surfaces the
* source has already been silently stale for however long nobody read the
* sync logs. This is registration-time, fail-fast validation ONLY it
* never auto-`git init`s the directory (that would cross the consent
* boundary #2967 established for sync-time self-heal: a `--path` source is
* the user's own external directory, and gbrain must not mutate it without
* explicit ask). Callers who want to register before git-init exists opt in
* via `force: true` (CLI: `--force`).
*/
export async function addSource(
engine: BrainEngine,
opts: AddSourceOpts,
@@ -437,6 +473,37 @@ export async function addSource(
}
} else {
// ── Path B: --path or no path (existing behavior, pre-v0.28) ─────────
// #2707: only validate when the path actually exists — a not-yet-created
// path is a different (pre-existing, out of scope) failure mode, and
// gating on existsSync keeps this a fail-fast check on the exact bug
// report ("plain directory accepted, sync fails later") rather than a
// broader "does this path exist" check nobody asked for.
//
// Both isInsideGitRepo AND hasTrackedContent must hold. isInsideGitRepo
// alone lets through a `git init`ed-but-never-committed directory (fails
// sync's "No commits in repo ..."), AND an empty-commit-then-untracked-
// files directory (git resolves HEAD fine but the tree is empty — the
// exact silent-staleness footgun #2707(c) describes: sync "succeeds"
// importing nothing, then never notices the untracked files change).
// hasTrackedContent's `ls-tree HEAD -- .` catches both (codex round 2).
if (
opts.localPath &&
!opts.force &&
existsSync(opts.localPath) &&
(!isInsideGitRepo(opts.localPath) || !hasTrackedContent(opts.localPath))
) {
const q = shellQuote(opts.localPath);
throw new SourceOpError(
'not_a_git_repo',
`"${opts.localPath}" is not a git repository with committed, tracked files ` +
`(or a subdirectory of one). GBrain sync requires every --path source to ` +
`be git-initialized, with the files actually committed — an empty commit ` +
`is not enough (the walker reads through git objects, so untracked files ` +
`stay invisible). Fix: \`git -C ${q} init && git -C ${q} add -A && ` +
`git -C ${q} commit -m "initial import"\`, then re-run this command. To ` +
`register anyway and git-init later, pass --force.`,
);
}
const config: Record<string, unknown> = {};
if (opts.federated !== null && opts.federated !== undefined) {
config.federated = opts.federated;
+21 -3
View File
@@ -454,13 +454,31 @@ export async function runThink(
// Closes #952 (think over MCP returns "no LLM available").
const client = opts.client ?? await tryBuildGatewayClient(modelUsed, { explicitModel: opts.modelExplicit });
if (!client) {
warnings.push('NO_ANTHROPIC_API_KEY');
// Label the failure honestly: a missing key and an unusable model id are
// different incidents with different fixes. Pre-fix EVERY null client was
// stamped NO_ANTHROPIC_API_KEY, which sent operators chasing env/keychain
// problems when the real cause was a model id the recipe didn't know
// (e.g. a tier-configured model newer than the recipe list). The re-probe
// is pure and cheap (no IO): same predicate tryBuildGatewayClient used.
const probe = probeChatModel(normalizeModelId(modelUsed));
const modelProblem = !probe.ok && probe.reason !== 'unavailable';
warnings.push(
modelProblem ? `MODEL_NOT_USABLE:${(probe as { reason: string }).reason}` : 'NO_ANTHROPIC_API_KEY',
);
const detail = !probe.ok ? probe.detail : '';
const fix = !probe.ok && probe.fix ? ` Fix: ${probe.fix}` : '';
// Degrade gracefully: return the gather without synthesis. Better than throwing.
return {
question: opts.question,
answer: '(no LLM available — set ANTHROPIC_API_KEY or pass `client`)',
answer: modelProblem
? `(model "${modelUsed}" not usable — ${detail}${fix})`
: '(no LLM available — set ANTHROPIC_API_KEY or pass `client`)',
citations: [],
gaps: ['no LLM available; gather succeeded but synthesis skipped'],
gaps: [
modelProblem
? `model "${modelUsed}" not usable (${(probe as { reason: string }).reason}); gather succeeded but synthesis skipped`
: 'no LLM available; gather succeeded but synthesis skipped',
],
pagesGathered: gather.pages.length,
takesGathered: gather.takes.length,
graphHits: gather.graphSlugs.length,
+7
View File
@@ -581,6 +581,13 @@ export interface Chunk {
parent_symbol_path?: string[] | null;
doc_comment?: string | null;
symbol_name_qualified?: string | null;
/**
* v0.27.1 multimodal. Read side of ChunkInput.modality must round-trip
* through getChunks embed-stale merge upsertChunks or image rows get
* reset to 'text' (EXCLUDED.modality on the upsert) and vanish from the
* image search arm.
*/
modality?: 'text' | 'image';
}
/**
+1
View File
@@ -329,6 +329,7 @@ export function rowToChunk(row: Record<string, unknown>, includeEmbedding = fals
parent_symbol_path: (row.parent_symbol_path as string[] | null | undefined) ?? null,
doc_comment: (row.doc_comment as string | null | undefined) ?? null,
symbol_name_qualified: (row.symbol_name_qualified as string | null | undefined) ?? null,
modality: (row.modality as 'text' | 'image' | undefined) ?? undefined,
};
}
+82
View File
@@ -0,0 +1,82 @@
import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { resetPgliteState } from './helpers/reset-pglite.ts';
import { queryAdminSources } from '../src/commands/serve-http.ts';
import { buildSyncStatusReport } from '../src/commands/sync.ts';
/**
* v0.41.29 Sources tab `/admin/api/sources` endpoint SQL.
*
* The endpoint is a thin Express handler over `queryAdminSources` +
* `buildSyncStatusReport`; the source-selection SQL is the load-bearing
* surface (same pattern as test/admin-agents-spend.test.ts).
*
* Pinned behaviors:
* - Excludes archived sources
* - INCLUDES sources with null local_path (push-only brains: filtering
* on local_path emptied the Sources tab + federation source-picker)
* - JSONB config surfaces as an object, defaulting to {}
* - Deterministic ORDER BY id
* - buildSyncStatusReport accepts the rows (no disk I/O on null paths)
*/
let engine: PGLiteEngine;
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
});
afterAll(async () => {
await engine.disconnect();
});
beforeEach(async () => {
await resetPgliteState(engine);
});
describe('queryAdminSources (/admin/api/sources SQL)', () => {
it('includes push-only sources with null local_path', async () => {
await engine.executeRaw(
`INSERT INTO sources (id, name, local_path, config)
VALUES ('push-only', 'push-only', NULL, '{}'::jsonb)`,
);
const sources = await queryAdminSources(engine);
const ids = sources.map((s) => s.id);
expect(ids).toContain('push-only');
expect(sources.find((s) => s.id === 'push-only')!.local_path).toBe(null);
});
it('excludes archived sources', async () => {
await engine.executeRaw(
`INSERT INTO sources (id, name, archived) VALUES ('gone', 'gone', true)`,
);
const sources = await queryAdminSources(engine);
expect(sources.map((s) => s.id)).not.toContain('gone');
});
it('surfaces JSONB config as an object and orders by id', async () => {
await engine.executeRaw(
`INSERT INTO sources (id, name, config)
VALUES ('bbb', 'bbb', '{"syncEnabled": true}'::jsonb),
('aaa', 'aaa', '{}'::jsonb)`,
);
const sources = await queryAdminSources(engine);
const ids = sources.map((s) => s.id);
expect(ids.indexOf('aaa')).toBeLessThan(ids.indexOf('bbb'));
expect(sources.find((s) => s.id === 'bbb')!.config).toEqual({ syncEnabled: true });
expect(sources.find((s) => s.id === 'aaa')!.config).toEqual({});
});
it('buildSyncStatusReport accepts the rows (null local_path does not throw)', async () => {
await engine.executeRaw(
`INSERT INTO sources (id, name, local_path, config)
VALUES ('push-only', 'push-only', NULL, '{}'::jsonb)`,
);
const report = await buildSyncStatusReport(engine, await queryAdminSources(engine));
expect(report.schema_version).toBe(1);
const row = report.sources.find((s) => s.source_id === 'push-only');
expect(row).toBeDefined();
});
});
+10 -1
View File
@@ -28,7 +28,7 @@
* (excluding the OpenAI canonical fast-path recipe).
*/
import { afterEach, beforeEach, describe, expect, mock, test } from 'bun:test';
import { afterAll, afterEach, beforeEach, describe, expect, mock, test } from 'bun:test';
import {
configureGateway,
resetGateway,
@@ -40,6 +40,15 @@ import {
} from '../../src/core/ai/gateway.ts';
import { AIConfigError, AITransientError } from '../../src/core/ai/errors.ts';
// The last test in this file leaves the gateway configured with a remote
// provider + fake key and a REAL embed transport. Without a final reset,
// that config leaks into whichever test file the shard runs next — the
// first downstream embed then makes a live HTTP call (broke master shard 6
// when #3022's new test file reshuffled shard composition). The bunfig
// legacy-embedding preload only re-applies its default when the gateway is
// UNCONFIGURED, so a configured-but-stale slot survives file boundaries.
afterAll(() => resetGateway());
// --------- Test helpers ---------
/**
+89
View File
@@ -0,0 +1,89 @@
import { describe, expect, test } from 'bun:test';
import {
dimsProviderOptions,
nvidiaEmbeddingDimOptions,
supportsNvidiaEmbeddingDimension,
} from '../../src/core/ai/dims.ts';
import { getRecipe, RECIPES } from '../../src/core/ai/recipes/index.ts';
import { nvidia } from '../../src/core/ai/recipes/nvidia.ts';
import { defaultResolveAuth } from '../../src/core/ai/gateway.ts';
import { AIConfigError } from '../../src/core/ai/errors.ts';
describe('recipe: nvidia', () => {
test('registered with OpenAI-compatible NIM endpoint', () => {
expect(RECIPES.has('nvidia')).toBe(true);
expect(getRecipe('nvidia')).toBe(nvidia);
expect(nvidia.id).toBe('nvidia');
expect(nvidia.tier).toBe('openai-compat');
expect(nvidia.implementation).toBe('openai-compatible');
expect(nvidia.base_url_default).toBe('https://integrate.api.nvidia.com/v1');
});
test('auth flows through defaultResolveAuth — NVIDIA_API_KEY as bearer token', () => {
// IRON RULE: only Azure overrides resolveAuth. NVIDIA is plain
// Authorization Bearer, so the recipe must NOT declare its own resolver;
// defaultResolveAuth derives the header from auth_env.required.
expect(nvidia.resolveAuth).toBeUndefined();
expect(nvidia.auth_env?.required).toEqual(['NVIDIA_API_KEY']);
expect(defaultResolveAuth(nvidia, { NVIDIA_API_KEY: 'fake-nvidia' }, 'embedding')).toEqual({
headerName: 'Authorization',
token: 'Bearer fake-nvidia',
});
expect(() => defaultResolveAuth(nvidia, {}, 'chat')).toThrow(AIConfigError);
});
test('chat touchpoint declares Nemotron 3 Super without subagent-loop claims', () => {
const chat = nvidia.touchpoints.chat!;
expect(chat.models).toContain('nvidia/nemotron-3-super-120b-a12b');
expect(chat.supports_tools).toBe(false);
expect(chat.supports_subagent_loop).toBe(false);
expect(chat.max_context_tokens).toBe(128000);
});
test('embedding touchpoint declares tested NVIDIA models and natural dimensions', () => {
const e = nvidia.touchpoints.embedding!;
expect(e.models).toContain('nvidia/nv-embedqa-e5-v5');
expect(e.models).toContain('nvidia/llama-nemotron-embed-1b-v2');
expect(e.models).toContain('nvidia/nv-embed-v1');
expect(e.models).toContain('nvidia/nv-embedcode-7b-v1');
expect(e.default_dims).toBe(1024);
expect(e.dims_options).toEqual([1024, 2048, 4096]);
expect(e.max_batch_tokens).toBeGreaterThan(0);
});
test('aliases allow short model names while preserving NVIDIA catalog ids', () => {
expect(nvidia.aliases?.['nv-embedqa-e5-v5']).toBe('nvidia/nv-embedqa-e5-v5');
expect(nvidia.aliases?.['llama-nemotron-embed-1b-v2']).toBe('nvidia/llama-nemotron-embed-1b-v2');
expect(nvidia.aliases?.['nemotron-3-super']).toBe('nvidia/nemotron-3-super-120b-a12b');
expect(nvidia.aliases?.['nemotron-3-super-120b-a12b']).toBe('nvidia/nemotron-3-super-120b-a12b');
});
test('dimsProviderOptions emits passage input_type by default for NVIDIA embeddings', () => {
expect(dimsProviderOptions('openai-compatible', 'nvidia/nv-embedqa-e5-v5', 1024)).toEqual({
openaiCompatible: { input_type: 'passage' },
});
});
test('dimsProviderOptions maps query/document inputType for NVIDIA embeddings', () => {
expect(dimsProviderOptions('openai-compatible', 'nvidia/nv-embedqa-e5-v5', 1024, 'query')).toEqual({
openaiCompatible: { input_type: 'query' },
});
expect(dimsProviderOptions('openai-compatible', 'nvidia/nv-embedqa-e5-v5', 1024, 'document')).toEqual({
openaiCompatible: { input_type: 'passage' },
});
});
test('llama-nemotron supports a 1280d Matryoshka dimension override', () => {
expect(nvidiaEmbeddingDimOptions('nvidia/llama-nemotron-embed-1b-v2')).toContain(1280);
expect(supportsNvidiaEmbeddingDimension('nvidia/llama-nemotron-embed-1b-v2', 1280)).toBe(true);
expect(dimsProviderOptions('openai-compatible', 'nvidia/llama-nemotron-embed-1b-v2', 1280, 'query')).toEqual({
openaiCompatible: { input_type: 'query', dimensions: 1280 },
});
});
test('fixed-dim NVIDIA models omit dimensions because they reject overrides', () => {
const opts = dimsProviderOptions('openai-compatible', 'nvidia/nv-embedqa-e5-v5', 1024, 'query');
expect(opts).toEqual({ openaiCompatible: { input_type: 'query' } });
expect(JSON.stringify(opts)).not.toContain('dimensions');
});
});
+14
View File
@@ -35,6 +35,20 @@ describe('Anthropic recipe model IDs', () => {
expect(anthropic.aliases?.['claude-sonnet-4-6-20250929']).toBe('claude-sonnet-4-6');
});
it('current-generation models are listed for chat (Fable 5 / Opus 4.8 / Sonnet 5)', () => {
// Regression guard for the tier-config incident: a brain with
// `models.tier.deep = anthropic:claude-opus-4-8` had think/auto_think
// silently degrade because the recipe list stopped at Opus 4.7.
const chatModels = anthropic.touchpoints?.chat?.models ?? [];
expect(chatModels).toContain('claude-fable-5');
expect(chatModels).toContain('claude-opus-4-8');
expect(chatModels).toContain('claude-sonnet-5');
});
it('Sonnet 5 is listed for expansion', () => {
expect(anthropic.touchpoints?.expansion?.models ?? []).toContain('claude-sonnet-5');
});
it('all listed models follow naming conventions', () => {
const allModels = [
...(anthropic.touchpoints?.chat?.models ?? []),
+79
View File
@@ -0,0 +1,79 @@
/**
* Regression gate for #2823: the shared test bootstrap
* (`test/helpers/audit-dir-preload.ts`, wired via `bunfig.toml`'s
* `preload`) must redirect `GBRAIN_AUDIT_DIR` to a per-run scratch
* directory BEFORE any test file runs, so audit-emitting code paths never
* fall through to the operator's real `~/.gbrain/audit/`.
*
* Before the fix, `test/import-file.test.ts`'s oversize-content boundary
* fixture (`'borderline-slug'`) fired a real `soft_block` content-sanity
* event straight into the developer's live audit trail on every test run.
* This file reproduces that exact event shape directly against the audit
* module (no PGLite/import-file machinery needed) and asserts it lands
* only in the scratch dir.
*/
import { describe, test, expect } from 'bun:test';
import { homedir, tmpdir } from 'os';
import { join } from 'path';
import { readFileSync, existsSync } from 'fs';
import { resolveAuditDir } from '../../src/core/audit/audit-writer.ts';
import {
logContentSanityAssessment,
readRecentContentSanityEvents,
computeContentSanityAuditFilename,
} from '../../src/core/audit/content-sanity-audit.ts';
import { assessContentSanity } from '../../src/core/content-sanity.ts';
describe('shared test-bootstrap audit isolation (#2823)', () => {
test('GBRAIN_AUDIT_DIR is set by the preload to a scratch dir, not the real ~/.gbrain/audit', () => {
const dir = process.env.GBRAIN_AUDIT_DIR;
expect(dir).toBeTruthy();
expect(dir).not.toBe(join(homedir(), '.gbrain', 'audit'));
// mkdtempSync(tmpdir(), ...) always lives directly under os.tmpdir().
expect(dir!.startsWith(tmpdir())).toBe(true);
});
test('resolveAuditDir() resolves to the preload-set scratch dir', () => {
const expected = process.env.GBRAIN_AUDIT_DIR;
expect(expected).toBeTruthy();
expect(resolveAuditDir()).toBe(expected!);
});
test('an oversize content-sanity event (the import-file.test.ts "borderline-slug" shape) never reaches the real ~/.gbrain/audit', () => {
// Unique per test-run so a stale match from a prior manual run can
// never produce a false pass.
const sentinelSlug = `borderline-slug-audit-dir-preload-${Date.now()}-${Math.random().toString(36).slice(2)}`;
const realAuditDir = join(homedir(), '.gbrain', 'audit');
const realAuditFile = join(realAuditDir, computeContentSanityAuditFilename());
// Reproduce the exact disposition the leaking fixture hits: body bytes
// over DEFAULT_BYTES_BLOCK (500_000) with no junk pattern match →
// shouldSkipEmbed=true, no shouldQuarantine → classified 'soft_block'.
const result = assessContentSanity({
compiled_truth: 'x'.repeat(600_000),
timeline: '',
title: 'Borderline',
});
expect(result.shouldSkipEmbed).toBe(true);
expect(result.shouldQuarantine).toBe(false);
logContentSanityAssessment(sentinelSlug, 'default', result);
// 1. The event IS readable back through the audit module — proves the
// write succeeded and landed in the dir resolveAuditDir() reports.
const recent = readRecentContentSanityEvents(1);
const found = recent.find((e) => e.slug === sentinelSlug);
expect(found).toBeDefined();
expect(found?.event_type).toBe('soft_block');
// 2. The real ~/.gbrain/audit content-sanity file for the current ISO
// week — if it exists at all on this machine — does NOT contain the
// sentinel slug. This is the actual regression: before the fix, this
// assertion would fail on any machine with a real ~/.gbrain.
if (existsSync(realAuditFile)) {
const contents = readFileSync(realAuditFile, 'utf8');
expect(contents).not.toContain(sentinelSlug);
}
});
});
+568
View File
@@ -0,0 +1,568 @@
/**
* Tests for `gbrain auth grant-read|revoke-read|set-federated-read`.
*
* Pure helper: parseSourceCsv (no DB).
* DB-coupled: resolveClient, assertSourceExists, *Core fns exercised
* against a real PGLite via the canonical block.
*/
import { describe, expect, test, beforeAll, afterAll, beforeEach } from 'bun:test';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { resetPgliteState } from './helpers/reset-pglite.ts';
import { sqlQueryForEngine } from '../src/core/sql-query.ts';
import { pgArray } from '../src/core/oauth-provider.ts';
import {
parseSourceCsv,
resolveClient,
assertSourceExists,
grantReadCore,
revokeReadCore,
setFederatedReadCore,
extractDryRun,
sanitizeForTerminal,
} from '../src/commands/auth.ts';
let engine: PGLiteEngine;
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
});
afterAll(async () => {
await engine.disconnect();
});
beforeEach(async () => {
await resetPgliteState(engine);
});
// ---------------------------------------------------------------------------
// pure helpers
// ---------------------------------------------------------------------------
describe('parseSourceCsv', () => {
test('splits and trims', () => {
expect(parseSourceCsv('a,b,c')).toEqual(['a', 'b', 'c']);
expect(parseSourceCsv(' a , b ')).toEqual(['a', 'b']);
});
test('drops empty segments', () => {
expect(parseSourceCsv('a,,b,')).toEqual(['a', 'b']);
expect(parseSourceCsv(',,')).toEqual([]);
expect(parseSourceCsv('')).toEqual([]);
});
test('dedupes while preserving first-seen order', () => {
expect(parseSourceCsv('a,b,a,c,b')).toEqual(['a', 'b', 'c']);
});
});
describe('extractDryRun', () => {
test('absent flag → false', () => {
expect(extractDryRun(['alice', 'proj-x'])).toEqual({
dryRun: false,
rest: ['alice', 'proj-x'],
});
});
test('flag at end', () => {
expect(extractDryRun(['alice', 'proj-x', '--dry-run'])).toEqual({
dryRun: true,
rest: ['alice', 'proj-x'],
});
});
test('flag at start', () => {
expect(extractDryRun(['--dry-run', 'alice', 'proj-x'])).toEqual({
dryRun: true,
rest: ['alice', 'proj-x'],
});
});
test('flag in middle', () => {
expect(extractDryRun(['alice', '--dry-run', 'proj-x'])).toEqual({
dryRun: true,
rest: ['alice', 'proj-x'],
});
});
test('no args', () => {
expect(extractDryRun([])).toEqual({ dryRun: false, rest: [] });
});
});
// ---------------------------------------------------------------------------
// DB-coupled
// ---------------------------------------------------------------------------
async function seedSource(id: string): Promise<void> {
const sql = sqlQueryForEngine(engine);
await sql`INSERT INTO sources (id, name) VALUES (${id}, ${id}) ON CONFLICT (id) DO NOTHING`;
}
async function seedClient(name: string, federated: string[] = []): Promise<string> {
// Ensure write source FK is satisfied — every seeded client points at 'default'.
await seedSource('default');
const sql = sqlQueryForEngine(engine);
const clientId = `gbrain_cl_test_${name}_${Date.now()}_${Math.random().toString(16).slice(2, 8)}`;
const fedLit = pgArray(federated);
await sql`
INSERT INTO oauth_clients (client_id, client_name, client_secret_hash,
redirect_uris, grant_types, scope,
client_id_issued_at, source_id, federated_read)
VALUES (${clientId}, ${name}, ${'dummy-hash'},
${pgArray([])}, ${pgArray(['client_credentials'])}, ${'read'},
${Date.now()}, ${'default'}, ${fedLit})
`;
return clientId;
}
async function readFederated(clientId: string): Promise<string[]> {
const sql = sqlQueryForEngine(engine);
const rows = await sql`SELECT federated_read FROM oauth_clients WHERE client_id = ${clientId}`;
const fed = rows[0]?.federated_read;
return Array.isArray(fed) ? (fed as string[]).map(String) : [];
}
describe('resolveClient', () => {
test('matches by client_id', async () => {
await seedSource('default');
const id = await seedClient('alice', ['default']);
const sql = sqlQueryForEngine(engine);
const c = await resolveClient(sql, id);
expect(c.client_name).toBe('alice');
expect(c.federated_read).toEqual(['default']);
});
test('matches by client_name', async () => {
await seedSource('default');
await seedClient('alice', ['default']);
const sql = sqlQueryForEngine(engine);
const c = await resolveClient(sql, 'alice');
expect(c.client_name).toBe('alice');
});
test('errors loudly on no-match', async () => {
const sql = sqlQueryForEngine(engine);
await expect(resolveClient(sql, 'nobody')).rejects.toThrow(/No active OAuth client found/);
});
test('errors loudly on ambiguous client_name', async () => {
await seedSource('default');
await seedClient('bob', ['default']);
await seedClient('bob', ['default']);
const sql = sqlQueryForEngine(engine);
await expect(resolveClient(sql, 'bob')).rejects.toThrow(/Multiple active OAuth clients named/);
});
test('null source_id is preserved as null (legacy row tolerance)', async () => {
const sql = sqlQueryForEngine(engine);
const clientId = `gbrain_cl_test_null_${Date.now()}`;
await sql`
INSERT INTO oauth_clients (client_id, client_name, client_secret_hash,
redirect_uris, grant_types, scope,
client_id_issued_at, source_id, federated_read)
VALUES (${clientId}, ${'legacy'}, ${'dummy'},
${pgArray([])}, ${pgArray(['client_credentials'])}, ${'read'},
${Date.now()}, ${null}, ${pgArray([])})
`;
const c = await resolveClient(sql, clientId);
expect(c.source_id).toBeNull();
expect(c.federated_read).toEqual([]);
});
});
describe('assertSourceExists', () => {
test('passes when present', async () => {
await seedSource('proj-x');
const sql = sqlQueryForEngine(engine);
await expect(assertSourceExists(sql, 'proj-x')).resolves.toBeUndefined();
});
test('throws with paste-ready hint when missing', async () => {
const sql = sqlQueryForEngine(engine);
await expect(assertSourceExists(sql, 'ghost')).rejects.toThrow(
/Source "ghost" does not exist.*gbrain sources add ghost/s,
);
});
});
describe('grantReadCore', () => {
test('appends when not present and persists', async () => {
await seedSource('default');
await seedSource('proj-x');
const id = await seedClient('alice', ['default']);
const sql = sqlQueryForEngine(engine);
const outcome = await grantReadCore(sql, 'alice', 'proj-x');
expect(outcome.kind).toBe('updated');
if (outcome.kind === 'updated') {
expect(outcome.before).toEqual(['default']);
expect(outcome.after).toEqual(['default', 'proj-x']);
}
expect(await readFederated(id)).toEqual(['default', 'proj-x']);
});
test('is idempotent — second call is a noop, list unchanged', async () => {
await seedSource('default');
await seedSource('proj-x');
const id = await seedClient('alice', ['default']);
const sql = sqlQueryForEngine(engine);
await grantReadCore(sql, 'alice', 'proj-x');
const outcome = await grantReadCore(sql, 'alice', 'proj-x');
expect(outcome.kind).toBe('noop');
if (outcome.kind === 'noop') {
expect(outcome.reason).toBe('already-granted');
}
expect(await readFederated(id)).toEqual(['default', 'proj-x']);
});
test('refuses unknown source (fails BEFORE mutating)', async () => {
await seedSource('default');
const id = await seedClient('alice', ['default']);
const sql = sqlQueryForEngine(engine);
await expect(grantReadCore(sql, 'alice', 'ghost')).rejects.toThrow(/does not exist/);
expect(await readFederated(id)).toEqual(['default']);
});
test('refuses unknown client', async () => {
const sql = sqlQueryForEngine(engine);
await expect(grantReadCore(sql, 'nobody', 'whatever')).rejects.toThrow(/No active OAuth client found/);
});
test('accepts client_id resolution too', async () => {
await seedSource('default');
await seedSource('proj-x');
const id = await seedClient('alice', ['default']);
const sql = sqlQueryForEngine(engine);
await grantReadCore(sql, id, 'proj-x');
expect(await readFederated(id)).toEqual(['default', 'proj-x']);
});
test('rejects malformed source_id BEFORE existence check (Codex finding #3)', async () => {
await seedSource('default');
const id = await seedClient('alice', ['default']);
const sql = sqlQueryForEngine(engine);
// Even with a row in `sources` having a weird id, the validator at the
// boundary refuses. Closes the "manual SQL plants a row, CLI lets it
// become unmanageable in federated_read" vector.
await sql`INSERT INTO sources (id, name) VALUES (${'has,"weird"-bits'}, ${'weird'})`;
await expect(grantReadCore(sql, 'alice', 'has,"weird"-bits')).rejects.toThrow(/Invalid source_id/);
// DB unchanged.
expect(await readFederated(id)).toEqual(['default']);
});
});
describe('revokeReadCore', () => {
test('removes when present', async () => {
await seedSource('default');
await seedSource('proj-x');
const id = await seedClient('alice', ['default', 'proj-x']);
const sql = sqlQueryForEngine(engine);
const outcome = await revokeReadCore(sql, 'alice', 'proj-x');
expect(outcome.kind).toBe('updated');
expect(await readFederated(id)).toEqual(['default']);
});
test('is idempotent — second call is a noop, list unchanged', async () => {
await seedSource('default');
const id = await seedClient('alice', ['default']);
const sql = sqlQueryForEngine(engine);
const outcome = await revokeReadCore(sql, 'alice', 'ghost-source');
expect(outcome.kind).toBe('noop');
if (outcome.kind === 'noop') {
expect(outcome.reason).toBe('not-present');
}
expect(await readFederated(id)).toEqual(['default']);
});
test('allows clearing the list down to empty (no implicit guard)', async () => {
await seedSource('default');
const id = await seedClient('alice', ['default']);
const sql = sqlQueryForEngine(engine);
await revokeReadCore(sql, 'alice', 'default');
expect(await readFederated(id)).toEqual([]);
});
test('does NOT validate the source exists — operator may revoke stale references', async () => {
await seedSource('default');
// federated_read carries 'proj-x' but the source row was deleted.
const id = await seedClient('alice', ['default', 'proj-x']);
const sql = sqlQueryForEngine(engine);
const outcome = await revokeReadCore(sql, 'alice', 'proj-x');
expect(outcome.kind).toBe('updated');
expect(await readFederated(id)).toEqual(['default']);
});
});
describe('setFederatedReadCore', () => {
test('replaces list wholesale', async () => {
await seedSource('a');
await seedSource('b');
await seedSource('c');
const id = await seedClient('alice', ['a']);
const sql = sqlQueryForEngine(engine);
const outcome = await setFederatedReadCore(sql, 'alice', 'b,c');
expect(outcome.kind).toBe('updated');
expect(await readFederated(id)).toEqual(['b', 'c']);
});
test('dedupes CSV input', async () => {
await seedSource('a');
await seedSource('b');
const id = await seedClient('alice', []);
const sql = sqlQueryForEngine(engine);
await setFederatedReadCore(sql, 'alice', 'a,b,a,b,a');
expect(await readFederated(id)).toEqual(['a', 'b']);
});
test('empty string clears the list', async () => {
await seedSource('a');
const id = await seedClient('alice', ['a']);
const sql = sqlQueryForEngine(engine);
await setFederatedReadCore(sql, 'alice', '');
expect(await readFederated(id)).toEqual([]);
});
test('noop when result equals current list', async () => {
await seedSource('a');
await seedSource('b');
const id = await seedClient('alice', ['a', 'b']);
const sql = sqlQueryForEngine(engine);
const outcome = await setFederatedReadCore(sql, 'alice', 'a,b');
expect(outcome.kind).toBe('noop');
if (outcome.kind === 'noop') {
expect(outcome.reason).toBe('same-list');
}
expect(await readFederated(id)).toEqual(['a', 'b']);
});
test('refuses unknown source (fails BEFORE mutating)', async () => {
await seedSource('a');
const id = await seedClient('alice', ['a']);
const sql = sqlQueryForEngine(engine);
await expect(setFederatedReadCore(sql, 'alice', 'a,ghost')).rejects.toThrow(/does not exist/);
// Original list preserved.
expect(await readFederated(id)).toEqual(['a']);
});
test('order in CSV is the order persisted', async () => {
await seedSource('a');
await seedSource('b');
await seedSource('c');
const id = await seedClient('alice', ['a']);
const sql = sqlQueryForEngine(engine);
await setFederatedReadCore(sql, 'alice', 'c,a,b');
expect(await readFederated(id)).toEqual(['c', 'a', 'b']);
});
});
// ---------------------------------------------------------------------------
// --dry-run semantics
// ---------------------------------------------------------------------------
// ---------------------------------------------------------------------------
// Codex fixes: soft-delete filter, atomic-SQL race-safety, sanitizer
// ---------------------------------------------------------------------------
describe('sanitizeForTerminal', () => {
test('preserves printable ASCII unchanged', () => {
expect(sanitizeForTerminal('alice')).toBe('alice');
expect(sanitizeForTerminal('a b-c_d.e/f@g')).toBe('a b-c_d.e/f@g');
});
test('escapes ANSI escape sequences', () => {
expect(sanitizeForTerminal('\x1b[2J')).toBe('\\x1b[2J');
expect(sanitizeForTerminal('\x1b]0;TITLE\x07')).toBe('\\x1b]0;TITLE\\x07');
});
test('escapes ALL C0 controls including tab and newline', () => {
// Codex re-review: preserving \n lets a DCR-registered name spoof
// additional rows in list-clients output. Tab spoofs field separators.
// Both are now escaped.
expect(sanitizeForTerminal('\x00\x07\x08')).toBe('\\x00\\x07\\x08');
expect(sanitizeForTerminal('line1\nline2')).toBe('line1\\x0aline2');
expect(sanitizeForTerminal('col1\tcol2')).toBe('col1\\x09col2');
});
test('escapes DEL and C1 controls', () => {
expect(sanitizeForTerminal('\x7f')).toBe('\\x7f');
expect(sanitizeForTerminal('\x9b[31m')).toBe('\\x9b[31m');
});
test('passes through unicode', () => {
expect(sanitizeForTerminal('café')).toBe('café');
expect(sanitizeForTerminal('日本語')).toBe('日本語');
});
});
describe('soft-delete filter (Codex finding #2)', () => {
async function softDeleteClient(clientId: string): Promise<void> {
const sql = sqlQueryForEngine(engine);
await sql`UPDATE oauth_clients SET deleted_at = now() WHERE client_id = ${clientId}`;
}
test('resolveClient hides soft-deleted clients by default', async () => {
await seedSource('default');
const id = await seedClient('alice', ['default']);
await softDeleteClient(id);
const sql = sqlQueryForEngine(engine);
await expect(resolveClient(sql, 'alice')).rejects.toThrow(/No active OAuth client found/);
await expect(resolveClient(sql, id)).rejects.toThrow(/No active OAuth client found/);
});
test('resolveClient with includeDeleted finds soft-deleted clients', async () => {
await seedSource('default');
const id = await seedClient('alice', ['default']);
await softDeleteClient(id);
const sql = sqlQueryForEngine(engine);
const c = await resolveClient(sql, id, { includeDeleted: true });
expect(c.client_name).toBe('alice');
expect(c.deleted_at).not.toBeNull();
});
test('grantReadCore refuses to mutate soft-deleted clients', async () => {
await seedSource('default');
await seedSource('proj-x');
const id = await seedClient('alice', ['default']);
await softDeleteClient(id);
const sql = sqlQueryForEngine(engine);
await expect(grantReadCore(sql, 'alice', 'proj-x')).rejects.toThrow(/No active OAuth client found/);
expect(await readFederated(id)).toEqual(['default']);
});
test('revokeReadCore refuses to mutate soft-deleted clients', async () => {
await seedSource('default');
const id = await seedClient('alice', ['default']);
await softDeleteClient(id);
const sql = sqlQueryForEngine(engine);
await expect(revokeReadCore(sql, 'alice', 'default')).rejects.toThrow(/No active OAuth client found/);
expect(await readFederated(id)).toEqual(['default']);
});
test('two clients with same name but only one active resolves to the active one', async () => {
await seedSource('default');
// Seed two clients with the same name; soft-delete the older one.
const sql = sqlQueryForEngine(engine);
const oldId = await seedClient('alice', ['default']);
await softDeleteClient(oldId);
const newId = await seedClient('alice', ['default']); // same name, new row
const c = await resolveClient(sql, 'alice');
expect(c.client_id).toBe(newId); // active row wins; ambiguity error suppressed
});
});
describe('atomic SQL race-safety (Codex finding #1, HIGH)', () => {
test('grant+revoke serialize at row-lock — sensitive stays revoked', async () => {
await seedSource('default');
await seedSource('sensitive');
await seedSource('harmless');
const id = await seedClient('alice', ['default', 'sensitive']);
const sql = sqlQueryForEngine(engine);
// Simulate concurrent revoke(sensitive) + grant(harmless). Real concurrency
// would race at the JS event loop boundary; here we await sequentially but
// each call goes through the ATOMIC SQL path. The contract: regardless of
// ordering, the final state has sensitive REMOVED and harmless ADDED.
await revokeReadCore(sql, 'alice', 'sensitive');
await grantReadCore(sql, 'alice', 'harmless');
const final1 = await readFederated(id);
expect(final1.sort()).toEqual(['default', 'harmless']);
// Reverse order, same final state. The pre-fix read-modify-write shape
// would have produced ['default', 'sensitive', 'harmless'] here (the
// resurrection bug Codex caught).
const id2 = await seedClient('bob', ['default', 'sensitive']);
await grantReadCore(sql, 'bob', 'harmless');
await revokeReadCore(sql, 'bob', 'sensitive');
const final2 = await readFederated(id2);
expect(final2.sort()).toEqual(['default', 'harmless']);
});
test('grant uses RETURNING to surface the post-write state', async () => {
await seedSource('default');
await seedSource('proj-x');
await seedClient('alice', ['default']);
const sql = sqlQueryForEngine(engine);
const outcome = await grantReadCore(sql, 'alice', 'proj-x');
expect(outcome.kind).toBe('updated');
if (outcome.kind === 'updated') {
// The `after` came from RETURNING, not from computing prev+sourceId
// in JS — proves the atomic path returned authoritative state.
expect(outcome.after).toEqual(['default', 'proj-x']);
}
});
test('grant noop path still survives without writing', async () => {
await seedSource('default');
const id = await seedClient('alice', ['default']);
const sql = sqlQueryForEngine(engine);
const outcome = await grantReadCore(sql, 'alice', 'default');
expect(outcome.kind).toBe('noop');
if (outcome.kind === 'noop') expect(outcome.reason).toBe('already-granted');
expect(await readFederated(id)).toEqual(['default']);
});
});
describe('dryRun mode', () => {
test('grantReadCore returns "updated" outcome but skips the write', async () => {
await seedSource('default');
await seedSource('proj-x');
const id = await seedClient('alice', ['default']);
const sql = sqlQueryForEngine(engine);
const outcome = await grantReadCore(sql, 'alice', 'proj-x', { dryRun: true });
expect(outcome.kind).toBe('updated');
if (outcome.kind === 'updated') {
expect(outcome.before).toEqual(['default']);
expect(outcome.after).toEqual(['default', 'proj-x']);
}
// Crucially: the DB row is UNCHANGED.
expect(await readFederated(id)).toEqual(['default']);
});
test('revokeReadCore returns "updated" outcome but skips the write', async () => {
await seedSource('default');
await seedSource('proj-x');
const id = await seedClient('alice', ['default', 'proj-x']);
const sql = sqlQueryForEngine(engine);
const outcome = await revokeReadCore(sql, 'alice', 'proj-x', { dryRun: true });
expect(outcome.kind).toBe('updated');
expect(await readFederated(id)).toEqual(['default', 'proj-x']);
});
test('setFederatedReadCore returns "updated" outcome but skips the write', async () => {
await seedSource('a');
await seedSource('b');
await seedSource('c');
const id = await seedClient('alice', ['a']);
const sql = sqlQueryForEngine(engine);
const outcome = await setFederatedReadCore(sql, 'alice', 'b,c', { dryRun: true });
expect(outcome.kind).toBe('updated');
expect(await readFederated(id)).toEqual(['a']);
});
test('noop outcomes are surfaced identically with or without dryRun', async () => {
await seedSource('default');
await seedSource('proj-x');
await seedClient('alice', ['default', 'proj-x']);
const sql = sqlQueryForEngine(engine);
const live = await grantReadCore(sql, 'alice', 'proj-x', { dryRun: false });
const dry = await grantReadCore(sql, 'alice', 'proj-x', { dryRun: true });
expect(live.kind).toBe('noop');
expect(dry.kind).toBe('noop');
});
test('errors still fire in dryRun (operator sees the problem before commit)', async () => {
await seedSource('default');
const id = await seedClient('alice', ['default']);
const sql = sqlQueryForEngine(engine);
await expect(
grantReadCore(sql, 'alice', 'ghost', { dryRun: true }),
).rejects.toThrow(/does not exist/);
await expect(
grantReadCore(sql, 'nobody', 'default', { dryRun: true }),
).rejects.toThrow(/No active OAuth client found/);
// DB unchanged.
expect(await readFederated(id)).toEqual(['default']);
});
});
+23
View File
@@ -174,6 +174,29 @@ describe('parseRegisterClientArgs', () => {
});
describe('error cases', () => {
test('--source with malformed id throws (validates source_id shape — codex re-review)', () => {
// Defense for the "register-client seeds an unmanageable
// federated_read entry" vector. assertValidSourceId fires before the
// function returns so DB never sees a row with bad source scope.
expect(() => parseRegisterClientArgs(['--source', 'has,weird,bits'])).toThrow(/Invalid source_id/);
expect(() => parseRegisterClientArgs(['--source', 'UPPER'])).toThrow(/Invalid source_id/);
expect(() => parseRegisterClientArgs(['--source', ''])).toThrow(/Invalid source_id|requires a value/);
});
test('--federated-read with any malformed id throws', () => {
// Single-item bad.
expect(() => parseRegisterClientArgs(['--federated-read', 'bad,source!'])).toThrow(/Invalid source_id/);
// Mixed valid + invalid — fails on the first bad one.
expect(() => parseRegisterClientArgs(['--federated-read', 'good,bad source'])).toThrow(/Invalid source_id/);
});
test('--source default + --federated-read default,team passes (regression — common case)', () => {
// Sanity: the canonical real-world invocation still parses cleanly.
const out = parseRegisterClientArgs(['--source', 'default', '--federated-read', 'default,team']);
expect(out.sourceId).toBe('default');
expect(out.federatedRead).toEqual(['default', 'team']);
});
test('--redirect-uri without value → throws', () => {
expect(() => parseRegisterClientArgs(['--redirect-uri'])).toThrow(/requires a value/);
});
+19
View File
@@ -95,6 +95,25 @@ describe('Life Chronicle timeline reads', () => {
expect(never.days_ago).toBeNull();
});
test('getLastSeen ignores future-dated events (bounds to <= asof/today)', async () => {
// Chronicle legitimately stores future events (a scheduled calendar-event,
// a planned milestone). "Last seen" must not return one, or the entity
// reads as seen-today (days_ago clamped to 0). Regression for the missing
// upper date bound in getLastSeen.
const fut = await insertPage({ slug: 'life/events/2026-08-01-fut', type: 'event', effectiveDate: '2026-08-01T10:00:00Z', frontmatter: '{"event":{"who":["people/sarah-chen"],"kind":"event"}}' });
await insertProjection(ids.meeting, fut, '2026-08-01', 'Planned Q3 launch');
// asof BEFORE the future event: last-seen is the most recent PAST event.
const seen = await engine.getLastSeen('people/sarah-chen', { asof: '2026-06-25', sourceId: 'default' });
expect(seen.last_date).toBe('2026-06-20'); // NOT 2026-08-01
expect(seen.days_ago).toBe(5); // NOT 0
// asof AFTER it: now it legitimately counts.
const later = await engine.getLastSeen('people/sarah-chen', { asof: '2026-08-02', sourceId: 'default' });
expect(later.last_date).toBe('2026-08-01');
expect(later.days_ago).toBe(1);
// Clean up so this future event doesn't leak into later shared-fixture tests.
await engine.executeRaw('UPDATE pages SET deleted_at = now() WHERE id = $1', [fut]);
});
test('source isolation: default scope excludes other-source events', async () => {
const def = await engine.getTimelineForDate('2026-06-18', { sourceId: 'default' });
expect(def.some(r => r.event_slug === 'life/events/2026-06-18-099')).toBe(false);
@@ -0,0 +1,80 @@
/**
* #2807 putPage INSERT default for chunker_version.
*
* postgres-engine / pglite-engine used `COALESCE(<chunkerVersion>, 1)` in the
* pages INSERT. Callers that don't supply chunker_version (no MCP/subagent
* caller does it's internal metadata) landed new pages at version 1, so
* doctor's contextual_retrieval_coverage check flagged dream-written pages as
* "older chunker_version" forever, even though they were chunked/embedded with
* the current chunker.
*
* Fix: default the INSERT to MARKDOWN_CHUNKER_VERSION. The ON CONFLICT UPDATE
* still COALESCE-preserves an explicitly supplied version.
*/
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { MARKDOWN_CHUNKER_VERSION } from '../src/core/chunkers/recursive.ts';
describe('#2807 — putPage chunker_version INSERT default', () => {
let engine: PGLiteEngine;
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
});
afterAll(async () => {
await engine.disconnect();
}, 30_000);
async function readChunkerVersion(slug: string): Promise<number> {
const { rows } = await (engine as any).db.query(
`SELECT chunker_version FROM pages WHERE slug = $1`,
[slug],
);
return Number((rows[0] as { chunker_version: number }).chunker_version);
}
test('page written without chunker_version defaults to MARKDOWN_CHUNKER_VERSION, not 1', async () => {
await engine.putPage('dream/no-chunker-version', {
type: 'concept',
title: 'Dream synthesized page',
compiled_truth: 'Written directly through putPage, like a dream subagent.',
timeline: '',
});
expect(await readChunkerVersion('dream/no-chunker-version')).toBe(MARKDOWN_CHUNKER_VERSION);
expect(MARKDOWN_CHUNKER_VERSION).toBeGreaterThan(1);
});
test('explicit chunker_version is honored on INSERT', async () => {
await engine.putPage('dream/explicit-chunker-version', {
type: 'concept',
title: 'Explicit version',
compiled_truth: 'Caller supplied a version.',
timeline: '',
chunker_version: 2,
});
expect(await readChunkerVersion('dream/explicit-chunker-version')).toBe(2);
});
test('re-put without chunker_version does not lower an already-current version', async () => {
await engine.putPage('dream/preserve-version', {
type: 'concept',
title: 'Preserve me',
compiled_truth: 'first write',
timeline: '',
chunker_version: MARKDOWN_CHUNKER_VERSION,
});
// Second write omits chunker_version; the UPDATE branch must not drop it
// below the current chunker version.
await engine.putPage('dream/preserve-version', {
type: 'concept',
title: 'Preserve me',
compiled_truth: 'second write',
timeline: '',
});
expect(await readChunkerVersion('dream/preserve-version')).toBe(MARKDOWN_CHUNKER_VERSION);
});
});
+1 -1
View File
@@ -77,7 +77,7 @@ describe('runConversationParser — help', () => {
});
describe('runConversationParser — list-builtins', () => {
test('human output includes all 12 pattern ids', async () => {
test('human output includes all built-in pattern ids', async () => {
const cap = captureStdio();
try {
await runConversationParser(null, ['list-builtins']);
+36 -2
View File
@@ -3,7 +3,7 @@
*
* Covers:
* - PR #1461's 6 telegram-bracket cases verbatim (REGRESSION pin)
* - All 12 built-in patterns hit their test_positive samples
* - All built-in patterns hit their test_positive samples
* - Date derivation precedence (D8)
* - Pattern priority scoring (D18) overlap resolution
* - Quick-reject fast path (D11)
@@ -116,7 +116,7 @@ describe('parseConversation — REGRESSION PR #1461 (telegram-bracket)', () => {
});
// ---------------------------------------------------------------------------
// All 12 built-ins must parse their test_positive samples
// All built-ins must parse their test_positive samples
// ---------------------------------------------------------------------------
describe('parseConversation — every built-in matches its test_positive sample', () => {
@@ -261,6 +261,40 @@ describe('parseConversation — multi-line continuation (D5)', () => {
});
});
describe('parseConversation — iMessage time-only 12h and date headings (#2756)', () => {
test('parses the time-only 12-hour iMessage shape', () => {
const r = parseConversation('**Alice Example** (9:04 PM): hello', {
fallbackDate: '2024-03-15',
});
expect(r.matched_pattern_id).toBe('bold-paren-time-12h');
expect(r.messages).toHaveLength(1);
expect(r.messages[0].timestamp).toBe('2024-03-15T21:04:00Z');
});
test('markdown date headings advance the running date without becoming message text', () => {
const body = [
'## 2024-03-15',
'**Alice Example** (9:04 AM): first day',
'## 2024-03-16',
'**Bob Example** (10:05 PM): second day',
].join('\n');
const r = parseConversation(body, { fallbackDate: '2024-03-01' });
expect(r.matched_pattern_id).toBe('bold-paren-time-12h');
expect(r.messages.map((m) => m.timestamp)).toEqual([
'2024-03-15T09:04:00Z',
'2024-03-16T22:05:00Z',
]);
expect(r.messages[0].text).toBe('first day');
});
test('date headings do not mutate the caller-provided context', () => {
const ctx = { fallbackDate: '2024-03-01', source: 'explicit' as const };
const pattern = BUILTIN_PATTERNS.find((p) => p.id === 'bold-paren-time-12h')!;
applyPattern('## 2024-03-16\n**Alice** (9:04 AM): hello', pattern, ctx);
expect(ctx.fallbackDate).toBe('2024-03-01');
});
});
// ---------------------------------------------------------------------------
// Timezone warning (D19)
// ---------------------------------------------------------------------------
+65
View File
@@ -560,3 +560,68 @@ describe('runCycle — sourceId resolution (regression #475)', () => {
expect(syncCalls.at(-1)?.sourceId).toBe('');
});
});
// ─── issue #2860: --once one-shot phase-enabled bypass (onceForPhase) ─
//
// CycleOpts.onceForPhase is deliberately typed as a single CyclePhase (not
// a boolean) so the override can never leak to a phase other than the one
// it names — even if a caller passes a wider `phases` array than the CLI
// does (dream.ts always restricts to `phases: [phase]` when --once is
// set). This exercises that boundary directly against runCycle, using the
// real (unmocked) patterns.ts module — cheap because with zero reflections
// seeded it never reaches an LLM call regardless of the enabled gate.
describe('runCycle — onceForPhase bypasses only the named phase (issue #2860)', () => {
beforeEach(async () => {
await truncateCycleLocks(sharedEngine);
await sharedEngine.setConfig('dream.patterns.enabled', 'false');
});
afterEach(async () => {
// Restore default so later describe blocks in this file (which run
// patterns as part of the full ALL_PHASES cycle) aren't affected.
await sharedEngine.setConfig('dream.patterns.enabled', 'true');
});
test('onceForPhase matching the requested phase bypasses its disabled gate', async () => {
const report = await runCycle(sharedEngine, {
brainDir: '/tmp/brain-2860-a',
phases: ['patterns'],
onceForPhase: 'patterns',
});
const patternsResult = report.phases.find(p => p.phase === 'patterns');
// Bypassed 'disabled' → falls through to the next gate (no reflections
// seeded). If the override didn't work, this would read 'disabled'.
expect(patternsResult?.status).toBe('skipped');
expect((patternsResult?.details as { reason?: string })?.reason).toBe('insufficient_evidence');
});
test('onceForPhase naming a DIFFERENT phase does not leak the bypass', async () => {
const report = await runCycle(sharedEngine, {
brainDir: '/tmp/brain-2860-b',
phases: ['patterns'],
onceForPhase: 'synthesize', // mismatched — must NOT bypass patterns' gate
});
const patternsResult = report.phases.find(p => p.phase === 'patterns');
expect(patternsResult?.status).toBe('skipped');
expect((patternsResult?.details as { reason?: string })?.reason).toBe('disabled');
});
test('no onceForPhase set → unchanged behavior (still gated)', async () => {
const report = await runCycle(sharedEngine, {
brainDir: '/tmp/brain-2860-c',
phases: ['patterns'],
});
const patternsResult = report.phases.find(p => p.phase === 'patterns');
expect(patternsResult?.status).toBe('skipped');
expect((patternsResult?.details as { reason?: string })?.reason).toBe('disabled');
});
test('config is never written by the override', async () => {
await runCycle(sharedEngine, {
brainDir: '/tmp/brain-2860-d',
phases: ['patterns'],
onceForPhase: 'patterns',
});
expect(await sharedEngine.getConfig('dream.patterns.enabled')).toBe('false');
});
});
+1 -1
View File
@@ -113,7 +113,7 @@ describe('autopilot-cycle handler contract (v0.20.5)', () => {
// the original 2000-char ceiling. The intent of the guard is unchanged:
// "the autopilot-cycle handler passes job.signal to runCycle." The
// window just needs to be wide enough to span any reasonable handler.
const handlerStart = jobsSource.indexOf("worker.register('autopilot-cycle'");
const handlerStart = jobsSource.indexOf("registerBuiltinJob(worker, engine, 'autopilot-cycle'");
expect(handlerStart).toBeGreaterThan(-1);
const handlerBlock = jobsSource.slice(handlerStart, handlerStart + 6000);
+21
View File
@@ -1,4 +1,7 @@
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
import { mkdirSync, rmSync, writeFileSync } from 'fs';
import { join } from 'path';
import { tmpdir } from 'os';
describe('doctor command', () => {
test('doctor module exports runDoctor', async () => {
@@ -120,6 +123,24 @@ describe('doctor command', () => {
}
});
test('skill conformance derives a valid host manifest when manifest.json is absent', async () => {
const { skillConformanceCheck } = await import('../src/commands/doctor.ts');
const skillsDir = join(tmpdir(), `gbrain-doctor-skills-${crypto.randomUUID()}`);
mkdirSync(join(skillsDir, 'host-only'), { recursive: true });
writeFileSync(
join(skillsDir, 'host-only', 'SKILL.md'),
'---\nname: host-only\ndescription: host-owned skill\n---\n\n# Host-only\n',
);
try {
const check = skillConformanceCheck(skillsDir);
expect(check).toMatchObject({ name: 'skill_conformance', status: 'ok' });
expect(check.message).toContain('1/1 skills pass');
expect(check.message).toContain('derived from SKILL.md files');
} finally {
rmSync(skillsDir, { recursive: true, force: true });
}
});
// v0.31.2 — facts_extraction_health check added in PR1 commit 12.
// Reads ingest_log rows with source_type='facts:absorb' (written by
// writeFactsAbsorbLog from src/core/facts/absorb-log.ts), groups by
+46
View File
@@ -135,4 +135,50 @@ describe('dream CLI flag wiring', () => {
expect(dreamSrc).toContain('cycle_already_running');
});
});
// issue #2860 — --once one-shot phase-enabled-gate bypass (structural).
// Behavioral coverage: test/e2e/dream-patterns-pglite.test.ts (bypass +
// config-untouched) and test/core/cycle.serial.test.ts (non-leak across
// phases via CycleOpts.onceForPhase).
describe('--once wiring (issue #2860)', () => {
test('declares --once flag', () => {
expect(dreamSrc).toContain("'--once'");
});
test('rejects bare --once with no --phase (exit 2)', () => {
expect(dreamSrc).toContain('--once requires an explicit --phase <name>');
// --help must short-circuit this validation (Codex review finding) —
// see the "--help --once" test in test/dream.test.ts for the
// behavioral pin of this exact ordering.
expect(dreamSrc).toContain('if (once && !phaseWasExplicit && !wantsHelp)');
});
// Codex P3 finding: the derived `phase` value gets populated by
// --input/--drain BEFORE this validation used to run, so those two
// silently slipped past an `!phase`-based check. The fix validates
// against `phaseWasExplicit` (captured at `phaseIdx !== -1`, before
// any implicit defaulting) instead. Behavioral pins live in
// test/dream.test.ts.
test('validates against phaseWasExplicit, captured before --input/--drain defaulting', () => {
expect(dreamSrc).toContain('const phaseWasExplicit = phaseIdx !== -1;');
// Must be declared before the --input-implies-synthesize and
// --drain-implies-extract_atoms defaulting blocks so it captures
// presence prior to any implicit phase assignment.
const explicitIdx = dreamSrc.indexOf('const phaseWasExplicit = phaseIdx !== -1;');
const inputImpliesIdx = dreamSrc.indexOf("phase = 'synthesize'");
const drainImpliesIdx = dreamSrc.indexOf("phase = 'extract_atoms'");
expect(explicitIdx).toBeGreaterThan(-1);
expect(explicitIdx).toBeLessThan(inputImpliesIdx);
expect(explicitIdx).toBeLessThan(drainImpliesIdx);
});
test('threads onceForPhase to runCycle, gated on opts.once', () => {
expect(dreamSrc).toMatch(/onceForPhase:\s*opts\.once\s*\?\s*opts\.phase!\s*:\s*undefined/);
});
test('documents --once in --help output', () => {
expect(dreamSrc).toContain('--once');
expect(dreamSrc).toContain('Never reads or writes config');
});
});
});
+113
View File
@@ -151,6 +151,119 @@ describe('runDream — --phase <name> restricts the cycle', () => {
});
});
// ─── --once (issue #2860) ───────────────────────────────────────────
describe('runDream — --once (issue #2860)', () => {
let repo: string;
let engine: InstanceType<typeof PGLiteEngine>;
beforeEach(async () => {
repo = makeGitRepo();
engine = await makePGLite();
}, 60_000);
afterEach(async () => {
if (engine) await engine.disconnect();
rmSync(repo, { recursive: true, force: true });
}, 60_000);
test('bare --once (no --phase) exits 2 with a usage hint', async () => {
const exitSpy = spyOn(process, 'exit').mockImplementation(() => { throw new Error('EXIT'); });
const errSpy = spyOn(console, 'error').mockImplementation(() => {});
try {
await runDream(engine, ['--dir', repo, '--once']);
throw new Error('expected runDream to exit');
} catch (e: any) {
expect(e.message).toBe('EXIT');
}
expect(exitSpy).toHaveBeenCalledWith(2);
expect(errSpy.mock.calls.flat().join(' ')).toMatch(/--once requires an explicit --phase <name>/);
exitSpy.mockRestore();
errSpy.mockRestore();
});
// Codex P3 finding: --input implicitly sets `phase = 'synthesize'` and
// --drain implicitly sets `phase = 'extract_atoms'` — an EARLIER version
// of the --once validation checked the derived `phase` value, which was
// already non-null by the time it ran, so both of these silently slipped
// past the "explicit --phase required" contract (and --once became a
// no-op: --drain returns before onceForPhase is ever read; --input
// already bypasses the synthesize gate on its own). The fix validates
// against `phaseWasExplicit` (whether the user actually typed --phase)
// instead of the derived value.
test('--input <file> --once (no explicit --phase) exits 2 — an implied phase does not count', async () => {
const exitSpy = spyOn(process, 'exit').mockImplementation(() => { throw new Error('EXIT'); });
const errSpy = spyOn(console, 'error').mockImplementation(() => {});
try {
await runDream(engine, ['--dir', repo, '--input', '/tmp/gbrain-2860-nonexistent.txt', '--once']);
throw new Error('expected runDream to exit');
} catch (e: any) {
expect(e.message).toBe('EXIT');
}
expect(exitSpy).toHaveBeenCalledWith(2);
expect(errSpy.mock.calls.flat().join(' ')).toMatch(/--once requires an explicit --phase <name>/);
exitSpy.mockRestore();
errSpy.mockRestore();
});
test('--drain --once (no explicit --phase) exits 2 — an implied phase does not count', async () => {
const exitSpy = spyOn(process, 'exit').mockImplementation(() => { throw new Error('EXIT'); });
const errSpy = spyOn(console, 'error').mockImplementation(() => {});
try {
await runDream(engine, ['--dir', repo, '--drain', '--once']);
throw new Error('expected runDream to exit');
} catch (e: any) {
expect(e.message).toBe('EXIT');
}
expect(exitSpy).toHaveBeenCalledWith(2);
expect(errSpy.mock.calls.flat().join(' ')).toMatch(/--once requires an explicit --phase <name>/);
exitSpy.mockRestore();
errSpy.mockRestore();
});
// Codex review finding: --help must short-circuit BEFORE the bare---once
// usage error, matching the same IRON RULE pinned above for --source
// ("--help --source whatever prints help and exits 0").
test('--help --once (no --phase) prints help and exits 0, not the usage error', async () => {
const exitSpy = spyOn(process, 'exit').mockImplementation(() => { throw new Error('EXIT'); });
const logSpy = spyOn(console, 'log').mockImplementation(() => {});
try {
const result = await runDream(engine, ['--help', '--once']);
expect(result).toBeUndefined();
} catch (e: any) {
throw new Error('--help with --once should NOT exit; got: ' + e.message);
}
expect(exitSpy).not.toHaveBeenCalled();
expect(logSpy.mock.calls.flat().join(' ')).toMatch(/Usage: gbrain dream/);
exitSpy.mockRestore();
logSpy.mockRestore();
});
test('--phase patterns --once bypasses dream.patterns.enabled=false without writing config', async () => {
await engine.setConfig('dream.patterns.enabled', 'false');
const report = await runDream(engine, ['--dir', repo, '--phase', 'patterns', '--once', '--json']);
expect(report).toBeTruthy();
if (report) {
expect(report.phases.length).toBe(1);
// Bypassed 'disabled' → falls through to the next gate. No
// reflections seeded, so it reports insufficient_evidence, not
// 'disabled' — proving --once actually forced the run.
expect(report.phases[0].phase).toBe('patterns');
expect((report.phases[0].details as { reason?: string }).reason).toBe('insufficient_evidence');
}
expect(await engine.getConfig('dream.patterns.enabled')).toBe('false');
});
test('--phase patterns (no --once) with dream.patterns.enabled=false still skips as disabled', async () => {
await engine.setConfig('dream.patterns.enabled', 'false');
const report = await runDream(engine, ['--dir', repo, '--phase', 'patterns', '--json']);
expect(report).toBeTruthy();
if (report) {
expect((report.phases[0].details as { reason?: string }).reason).toBe('disabled');
}
});
});
// ─── Output format ─────────────────────────────────────────────────
describe('runDream — output format', () => {
@@ -0,0 +1,77 @@
/**
* v0.42.x Life Chronicle getLastSeen, LIVE Postgres engine (#2390 follow-up).
*
* Parity coverage for the PGLite regression in
* test/chronicle-timeline-reads.test.ts: getLastSeen must bound to
* `te.date <= COALESCE(asof, current_date)` so a future-dated chronicle
* event (a scheduled calendar-event) is NOT reported as "seen today".
*
* Uses the canonical e2e harness (setupDB/teardownDB/getEngine); gated by
* DATABASE_URL via hasDatabase() and skips cleanly when unset, per the repo
* E2E lifecycle. Seeds its own fixtures via direct SQL, mirroring the PGLite
* test, then asserts the same fail-before / pass-after behavior on Postgres.
*
* Run: DATABASE_URL=... bun test test/e2e/chronicle-last-seen-postgres.test.ts
*/
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import type { PostgresEngine } from '../../src/core/postgres-engine.ts';
import { hasDatabase, setupDB, teardownDB } from './helpers.ts';
const RUN = hasDatabase();
const d = RUN ? describe : describe.skip;
let engine: PostgresEngine;
const ids: Record<string, number> = {};
async function insertPage(opts: {
slug: string; type: string; sourceId?: string;
effectiveDate?: string | null; frontmatter?: string;
}): Promise<number> {
const rows = await engine.executeRaw<{ id: number }>(
`INSERT INTO pages (source_id, slug, type, title, effective_date, frontmatter)
VALUES ($1, $2, $3, $4, $5::timestamptz, $6::text::jsonb)
RETURNING id`,
[opts.sourceId ?? 'default', opts.slug, opts.type, opts.slug,
opts.effectiveDate ?? null, opts.frontmatter ?? '{}'],
);
return rows[0].id;
}
async function insertProjection(depthId: number, eventId: number, date: string, summary: string): Promise<void> {
await engine.executeRaw(
`INSERT INTO timeline_entries (page_id, date, source, summary, detail, event_page_id)
VALUES ($1, $2::date, $3, $4, '', $5)`,
[depthId, date, `life-chronicle:event:${eventId}`, summary, eventId],
);
}
d('getLastSeen (live Postgres) bounds to <= asof/today', () => {
beforeAll(async () => {
engine = await setupDB();
ids.meeting = await insertPage({ slug: 'meetings/2026-06-18-sync', type: 'meeting' });
await insertPage({ slug: 'people/sarah-chen', type: 'person' });
// Past events for Sarah: 06-18 15:30 (commitment) and 06-20 10:00 (decision).
ids.e1 = await insertPage({ slug: 'life/events/2026-06-18-001', type: 'event', effectiveDate: '2026-06-18T15:30:00Z', frontmatter: '{"event":{"who":["people/sarah-chen"],"kind":"commitment"}}' });
ids.e3 = await insertPage({ slug: 'life/events/2026-06-20-001', type: 'event', effectiveDate: '2026-06-20T10:00:00Z', frontmatter: '{"event":{"who":["people/sarah-chen"],"kind":"decision"}}' });
await insertProjection(ids.meeting, ids.e1, '2026-06-18', 'Sarah committed to Q3');
await insertProjection(ids.meeting, ids.e3, '2026-06-20', 'Decision on launch');
// Future event: a scheduled Q3 launch on 2026-08-01.
ids.fut = await insertPage({ slug: 'life/events/2026-08-01-fut', type: 'event', effectiveDate: '2026-08-01T10:00:00Z', frontmatter: '{"event":{"who":["people/sarah-chen"],"kind":"event"}}' });
await insertProjection(ids.meeting, ids.fut, '2026-08-01', 'Planned Q3 launch');
});
afterAll(async () => { await teardownDB(); });
test('future event does not read as last-seen; asof after it lets it through', async () => {
// asof BEFORE the future event → last-seen is the most recent PAST event.
const seen = await engine.getLastSeen('people/sarah-chen', { asof: '2026-06-25', sourceId: 'default' });
expect(seen.last_date).toBe('2026-06-20'); // NOT 2026-08-01
expect(seen.days_ago).toBe(5); // NOT 0
// asof AFTER the future event → it is now in-bound and becomes last-seen.
const later = await engine.getLastSeen('people/sarah-chen', { asof: '2026-08-02', sourceId: 'default' });
expect(later.last_date).toBe('2026-08-01');
expect(later.days_ago).toBe(1);
});
});
+1 -1
View File
@@ -2,7 +2,7 @@
* v0.41.16.0 E2E test for the conversation parser cathedral against
* a real PGLite brain.
*
* For each of the 12 built-in formats: seed a page through
* For each built-in format: seed a page through
* `importFromContent`, run `parseConversation` against the body, assert
* the parser identifies the correct pattern AND produces at least one
* message AND the message timestamp lands in the expected date range.
+39
View File
@@ -99,6 +99,45 @@ describe('E2E patterns — disabled', () => {
await rig.cleanup();
}
}, 30_000);
// issue #2860 — `gbrain dream --phase patterns --once` must bypass the
// enabled gate WITHOUT touching config (the whole point: the old
// toggle-on/run/toggle-off workaround left `enabled` stuck true forever
// if the process died between steps, running patterns hourly and
// burning ~$400 in LLM spend before it was caught).
test('once:true bypasses the disabled gate for this call only', async () => {
const rig = await setupRig();
try {
await rig.engine.setConfig('dream.patterns.enabled', 'false');
// Without --once: still skipped as 'disabled' (unchanged behavior).
const gated = await runPhasePatterns(rig.engine, {
brainDir: rig.brainDir,
dryRun: false,
});
expect(gated.status).toBe('skipped');
expect((gated.details as { reason?: string }).reason).toBe('disabled');
// With --once: the disabled gate no longer short-circuits — the call
// proceeds past it to the NEXT gate (insufficient_evidence, since no
// reflections were seeded). If --once didn't work, this would still
// report 'disabled'.
const forced = await runPhasePatterns(rig.engine, {
brainDir: rig.brainDir,
dryRun: false,
once: true,
});
expect(forced.status).toBe('skipped');
expect((forced.details as { reason?: string }).reason).toBe('insufficient_evidence');
// Config was NEVER written — the override is call-scoped only, unlike
// the toggle-on/toggle-off workaround the issue exists to replace.
const stillDisabled = await rig.engine.getConfig('dream.patterns.enabled');
expect(stillDisabled).toBe('false');
} finally {
await rig.cleanup();
}
}, 30_000);
});
describe('E2E patterns — insufficient_evidence', () => {
+195
View File
@@ -13,6 +13,7 @@
*/
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { createHash } from 'crypto';
import { hasDatabase } from './helpers.ts';
const skip = !hasDatabase();
@@ -210,6 +211,12 @@ describeE2E('serve-http OAuth 2.1 E2E (v0.26.1 + v0.26.2 + v0.26.3)', () => {
expect(meta.grant_types_supported).toContain('authorization_code');
expect(meta.grant_types_supported).toContain('refresh_token');
expect(meta.grant_types_supported).toContain('client_credentials');
expect(meta.token_endpoint_auth_methods_supported).toEqual(
expect.arrayContaining(['client_secret_post', 'client_secret_basic', 'none']),
);
expect(meta.revocation_endpoint_auth_methods_supported).toEqual(
expect.arrayContaining(['client_secret_post', 'client_secret_basic']),
);
});
test('OAuth metadata issuer matches public URL', async () => {
@@ -407,6 +414,194 @@ describeE2E('serve-http OAuth 2.1 E2E (v0.26.1 + v0.26.2 + v0.26.3)', () => {
expect(data.error).toBe('invalid_grant');
});
test('confidential client can revoke its token only with its valid secret', async () => {
const { access_token } = await mintToken('read');
const wrongSecret = await fetch(`${BASE}/revoke`, {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body: `token=${encodeURIComponent(access_token)}&client_id=${clientId}&client_secret=gbrain_cs_wrong_secret`,
});
expect(wrongSecret.status).toBe(401);
expect((await wrongSecret.json() as any).error).toBe('invalid_client');
// A rejected revoke request must leave the token usable.
expect((await mcpCall(access_token, 'tools/list')).status).toBe(200);
const revoke = await fetch(`${BASE}/revoke`, {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body: `token=${encodeURIComponent(access_token)}&client_id=${clientId}&client_secret=${clientSecret}`,
});
expect(revoke.status).toBe(200);
expect(revoke.headers.get('cache-control')).toBe('no-store');
expect((await mcpCall(access_token, 'tools/list')).status).toBe(401);
}, 15_000);
test('confidential client_secret_basic revoke returns canonical auth responses', async () => {
const { access_token: wrongSecretToken } = await mintToken('read');
const wrongBasic = Buffer.from(`${encodeURIComponent(clientId!)}:${encodeURIComponent('wrong-secret')}`).toString('base64');
const rejected = await fetch(`${BASE}/revoke`, {
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
Authorization: `Basic ${wrongBasic}`,
},
body: `token=${encodeURIComponent(wrongSecretToken)}`,
});
expect(rejected.status).toBe(401);
expect(rejected.headers.get('www-authenticate')).toMatch(/^Basic /);
expect((await mcpCall(wrongSecretToken, 'tools/list')).status).toBe(200);
const { access_token } = await mintToken('read');
const validBasic = Buffer.from(`${encodeURIComponent(clientId!)}:${encodeURIComponent(clientSecret!)}`).toString('base64');
const revoked = await fetch(`${BASE}/revoke`, {
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
Authorization: `Basic ${validBasic}`,
},
body: `token=${encodeURIComponent(access_token)}`,
});
expect(revoked.status).toBe(200);
expect(revoked.headers.get('cache-control')).toBe('no-store');
expect((await mcpCall(access_token, 'tools/list')).status).toBe(401);
}, 15_000);
test('revoke validates request shape and rejects mixed client authentication', async () => {
const { access_token } = await mintToken('read');
const validBasic = Buffer.from(`${encodeURIComponent(clientId!)}:${encodeURIComponent(clientSecret!)}`).toString('base64');
const mixed = await fetch(`${BASE}/revoke`, {
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
Authorization: `Basic ${validBasic}`,
},
body: `token=${encodeURIComponent(access_token)}&client_id=${clientId}&client_secret=${clientSecret}`,
});
expect(mixed.status).toBe(400);
expect((await mixed.json() as any).error).toBe('invalid_request');
const repeatedToken = await fetch(`${BASE}/revoke`, {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body: `token=${encodeURIComponent(access_token)}&token=duplicate&client_id=${clientId}&client_secret=${clientSecret}`,
});
expect(repeatedToken.status).toBe(400);
expect((await repeatedToken.json() as any).error).toBe('invalid_request');
const missingToken = await fetch(`${BASE}/revoke`, {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body: `client_id=${clientId}&client_secret=${clientSecret}`,
});
expect(missingToken.status).toBe(400);
expect((await missingToken.json() as any).error).toBe('invalid_request');
expect((await mcpCall(access_token, 'tools/list')).status).toBe(200);
}, 15_000);
test('unknown and cross-client tokens are opaque 200 no-ops', async () => {
const unknown = await fetch(`${BASE}/revoke`, {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body: `token=unknown-token&client_id=${clientId}&client_secret=${clientSecret}`,
});
expect(unknown.status).toBe(200);
const { execSync } = await import('child_process');
const attackerRegistration = execSync(
`bun run src/cli.ts auth register-client e2e-revoke-attacker-${Date.now()} --grant-types client_credentials --scopes read`,
{ cwd: process.cwd(), encoding: 'utf8', env: { ...process.env } },
);
const attackerId = attackerRegistration.match(/Client ID:\s+(gbrain_cl_\S+)/)?.[1];
const attackerSecret = attackerRegistration.match(/Client Secret:\s+(gbrain_cs_\S+)/)?.[1];
expect(attackerId).toBeTruthy();
expect(attackerSecret).toBeTruthy();
dcrClientIds.push(attackerId!);
const { access_token: ownerToken } = await mintToken('read');
const crossClient = await fetch(`${BASE}/revoke`, {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body: `token=${encodeURIComponent(ownerToken)}&client_id=${attackerId}&client_secret=${attackerSecret}`,
});
expect(crossClient.status).toBe(200);
expect((await mcpCall(ownerToken, 'tools/list')).status).toBe(200);
}, 30_000);
test('public client revoke falls through to the SDK handler', async () => {
const { execSync } = await import('child_process');
const registration = execSync(
`bun run src/cli.ts auth register-client e2e-revoke-public-${Date.now()} --grant-types authorization_code --scopes read --token-endpoint-auth-method none`,
{ cwd: process.cwd(), encoding: 'utf8', env: { ...process.env } },
);
const publicClientId = registration.match(/Client ID:\s+(gbrain_cl_\S+)/)?.[1];
expect(publicClientId).toBeTruthy();
dcrClientIds.push(publicClientId!);
const publicToken = `gbrain_at_public_${Date.now()}`;
const tokenHash = createHash('sha256').update(publicToken).digest('hex');
const postgres = (await import('postgres')).default;
const sql = postgres(process.env.GBRAIN_DATABASE_URL || process.env.DATABASE_URL || '', { prepare: false });
try {
await sql`
INSERT INTO oauth_tokens (token_hash, token_type, client_id, scopes, expires_at)
VALUES (${tokenHash}, ${'access'}, ${publicClientId!}, ${sql.array(['read'])}, ${Math.floor(Date.now() / 1000) + 3600})
`;
} finally {
await sql.end();
}
expect((await mcpCall(publicToken, 'tools/list')).status).toBe(200);
const revoked = await fetch(`${BASE}/revoke`, {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body: `token=${encodeURIComponent(publicToken)}&client_id=${publicClientId}`,
});
expect(revoked.status).toBe(200);
expect((await mcpCall(publicToken, 'tools/list')).status).toBe(401);
}, 30_000);
test('retryable revoke backend failure returns 503 and leaves token usable', async () => {
const { access_token } = await mintToken('read');
const tokenHash = createHash('sha256').update(access_token).digest('hex');
const suffix = Date.now().toString();
const functionName = `e2e_fail_revoke_${suffix}`;
const triggerName = `e2e_fail_revoke_trigger_${suffix}`;
const postgres = (await import('postgres')).default;
const sql = postgres(process.env.GBRAIN_DATABASE_URL || process.env.DATABASE_URL || '', { prepare: false });
try {
await sql.unsafe(`
CREATE FUNCTION ${functionName}() RETURNS trigger LANGUAGE plpgsql AS $$
BEGIN
IF OLD.token_hash = '${tokenHash}' THEN
RAISE EXCEPTION 'injected retryable revoke failure' USING ERRCODE = '08006';
END IF;
RETURN OLD;
END;
$$
`);
await sql.unsafe(`
CREATE TRIGGER ${triggerName}
BEFORE DELETE ON oauth_tokens
FOR EACH ROW EXECUTE FUNCTION ${functionName}()
`);
const failed = await fetch(`${BASE}/revoke`, {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body: `token=${encodeURIComponent(access_token)}&client_id=${clientId}&client_secret=${clientSecret}`,
});
expect(failed.status).toBe(503);
expect((await failed.json() as any).error).toBe('temporarily_unavailable');
expect((await mcpCall(access_token, 'tools/list')).status).toBe(200);
} finally {
await sql.unsafe(`DROP TRIGGER IF EXISTS ${triggerName} ON oauth_tokens`);
await sql.unsafe(`DROP FUNCTION IF EXISTS ${functionName}()`);
await sql.end();
}
}, 30_000);
// =========================================================================
// v0.26.2: DCR /register response shape (RFC 7591 §3.2.1 number contract)
// =========================================================================
+49
View File
@@ -227,4 +227,53 @@ describe('embedStaleForSource', () => {
const otherStale = await engine.countStaleChunks({ sourceId: 'other' });
expect(otherStale).toBe(3);
});
test('preserves modality and code-symbol metadata across the merge round-trip', async () => {
// Regression: the merged ChunkInput[] used to rebuild rows with only 5
// fields; upsertChunks writes modality/symbol columns as EXCLUDED.<col>,
// so an image page with one stale TEXT chunk got its image row reset to
// modality='text' — permanently invisible to the image search arm.
await engine.putPage('media/mixed-page', {
type: 'image',
title: 'mixed',
compiled_truth: 'mixed modality page',
});
const imgVec = new Float32Array(1024).fill(0.03);
await engine.upsertChunks('media/mixed-page', [
{
chunk_index: 0,
chunk_text: 'field-photo.jpg',
chunk_source: 'image_asset',
modality: 'image',
embedding_image: imgVec,
// embedding intentionally present so this row is NOT stale.
embedding: new Float32Array(1536).fill(0.01),
token_count: 4,
},
{
chunk_index: 1,
chunk_text: 'ocr caption text needing embed',
chunk_source: 'compiled_truth',
language: 'python',
symbol_name: 'kept_symbol',
symbol_type: 'function',
symbol_name_qualified: 'mod::kept_symbol',
token_count: 6,
embedding: undefined, // stale — triggers the merge path
},
]);
const result = await embedStaleForSource(engine, 'default', { embedFn: fakeEmbedFn });
expect(result.embedded).toBe(1);
const after = await engine.getChunks('media/mixed-page');
const imgRow = after.find((c) => c.chunk_index === 0)!;
const txtRow = after.find((c) => c.chunk_index === 1)!;
expect(imgRow.modality).toBe('image');
expect(txtRow.language).toBe('python');
expect(txtRow.symbol_name).toBe('kept_symbol');
expect(txtRow.symbol_name_qualified).toBe('mod::kept_symbol');
// The stale text row actually got its embedding.
expect(txtRow.embedded_at).not.toBeNull();
});
});
+35
View File
@@ -36,6 +36,7 @@ import {
MAX_PAGE_BODY_BYTES,
TERMINAL_AUDIT_SOURCE,
PER_SEGMENT_SOURCE_PREFIX,
ALLOWED_TYPES,
} from '../src/commands/extract-conversation-facts.ts';
// ---------------------------------------------------------------------------
@@ -93,6 +94,11 @@ describe('parseConversationMessages', () => {
});
});
test('conversation-facts allowlist includes native iMessage page types (#2756)', () => {
expect(ALLOWED_TYPES).toContain('imessage');
expect(ALLOWED_TYPES).toContain('imessage-daily');
});
// ---------------------------------------------------------------------------
// splitIntoSegments — PR's 5 cases verbatim plus tuning regression.
// ---------------------------------------------------------------------------
@@ -319,6 +325,13 @@ describe('runExtractConversationFactsCore', () => {
timeline: '',
frontmatter: {},
});
await engine.putPage('conversations/imessage/native-example', {
type: 'imessage',
title: 'Native iMessage export',
compiled_truth: SAMPLE_BODY,
timeline: '',
frontmatter: {},
});
await engine.putPage('people/alice-example', {
type: 'person',
title: 'Alice Example',
@@ -392,6 +405,17 @@ describe('runExtractConversationFactsCore', () => {
expect(result.pages_considered).toBe(0);
});
test('native imessage page types are eligible by default', async () => {
const result = await runExtractConversationFactsCore(engine, {
sourceId: 'default',
slug: 'conversations/imessage/native-example',
dryRun: true,
sleepMs: 0,
});
expect(result.pages_considered).toBe(1);
expect(result.pages_processed).toBe(1);
});
test('sinceIso filters already-processed history', async () => {
const result = await runExtractConversationFactsCore(engine, {
sourceId: 'default',
@@ -432,6 +456,17 @@ describe('runExtractConversationFactsCore', () => {
);
expect(Number(perSegFacts[0]?.count ?? 0)).toBeGreaterThan(0);
const validTimes = await engine.executeRaw<{ valid_from: Date }>(
`SELECT valid_from FROM facts
WHERE source = $1 AND source_session = $2
ORDER BY valid_from ASC`,
[PER_SEGMENT_SOURCE_PREFIX, `${PER_SEGMENT_SOURCE_PREFIX}:conversations/imessage/alice-example`],
);
expect(validTimes.map((row) => new Date(row.valid_from).toISOString())).toEqual([
'2024-03-15T09:00:00.000Z',
'2024-03-16T08:00:00.000Z',
]);
// Terminal audit row present.
const terminalRows = await engine.executeRaw<{ count: string | number }>(
`SELECT COUNT(*) AS count FROM facts WHERE source = $1 AND source_session = $2`,
+1
View File
@@ -1,5 +1,6 @@
{"fixture_id":"imessage-001","pattern":"imessage-slack","frontmatter":{"date":"2024-03-15"},"body":"**Alice Example** (2024-03-15 9:00 AM): morning\n**Bob Example** (2024-03-15 9:01 AM): hey there\n**Alice Example** (2024-03-15 9:02 AM): how are you\n**Bob Example** (2024-03-15 9:03 AM): good thanks\n**Alice Example** (2024-03-15 9:04 AM): you?","expected_messages":5,"expected_participants":["Alice Example","Bob Example"]}
{"fixture_id":"imessage-002","pattern":"imessage-slack","frontmatter":{"date":"2024-03-15"},"body":"**Charlie Example** (2024-03-15 2:00 PM): afternoon\n**Charlie Example** (2024-03-15 2:01 PM): are you there?\n**Diana Example** (2024-03-15 2:05 PM): yes\n**Charlie Example** (2024-03-15 2:06 PM): great","expected_messages":4,"expected_participants":["Charlie Example","Diana Example"]}
{"fixture_id":"imessage-time-only-12h-001","pattern":"bold-paren-time-12h","frontmatter":{"date":"2024-03-15"},"body":"## 2024-03-15\n**Alice Example** (9:04 AM): morning\n**Bob Example** (9:05 AM): hey there\n## 2024-03-16\n**Alice Example** (10:06 PM): second day\n**Bob Example** (10:07 PM): good night","expected_messages":4,"expected_participants":["Alice Example","Bob Example"]}
{"fixture_id":"telegram-bracket-001","pattern":"telegram-bracket","frontmatter":{"date":"2026-05-24","timezone":"America/Los_Angeles"},"body":"**[18:37] 👤 Alice Example:** hello world\n**[18:38] 👤 Bob Example:** hey\n**[18:39] 👤 Alice Example:** how are you\n**[18:40] 👤 Bob Example:** good","expected_messages":4,"expected_participants":["Alice Example","Bob Example"]}
{"fixture_id":"telegram-bracket-002","pattern":"telegram-bracket","frontmatter":{"date":"2026-05-25","timezone":"America/Los_Angeles"},"body":"**[06:00] 🤖 Zion Bot:** On it.\n**[06:01] 👤 Charlie Example:** thanks\n**[06:02] 🤖 Zion Bot:** anything else?\n**[06:03] 👤 Charlie Example:** no good","expected_messages":4,"expected_participants":["Zion Bot","Charlie Example"]}
{"fixture_id":"whatsapp-iso-001","pattern":"whatsapp-iso","frontmatter":{"date":"2024-03-15"},"body":"[15/03/24, 18:37:00] Alice Example: hello\n[15/03/24, 18:37:30] Bob Example: hey\n[15/03/24, 18:38:00] Alice Example: how are you\n[15/03/24, 18:39:00] Bob Example: good","expected_messages":4,"expected_participants":["Alice Example","Bob Example"]}
@@ -0,0 +1 @@
{"fixture_id":"imessage-time-only-12h-001","pattern":"bold-paren-time-12h","frontmatter":{"date":"2024-03-15"},"body":"## 2024-03-15\n**Alice Example** (9:04 AM): morning\n**Bob Example** (9:05 AM): hey there\n## 2024-03-16\n**Alice Example** (10:06 PM): second day\n**Bob Example** (10:07 PM): good night","expected_messages":4,"expected_participants":["Alice Example","Bob Example"]}
@@ -0,0 +1,77 @@
/**
* Foreground batch commands are normally reached after cli.ts configures the
* process-global gateway. Keep the command boundary defensive as well: an
* embedding/CLI loader can leave that singleton cold while the persisted chat
* configuration and provider credentials are valid (#2590).
*/
import { afterEach, beforeEach, describe, expect, spyOn, test } from 'bun:test';
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { isAvailable, resetGateway } from '../src/core/ai/gateway.ts';
import { runExtractConversationFacts } from '../src/commands/extract-conversation-facts.ts';
import { runEnrich } from '../src/commands/enrich.ts';
let home: string;
const originalHome = process.env.GBRAIN_HOME;
const originalOpenAiKey = process.env.OPENAI_API_KEY;
beforeEach(() => {
home = mkdtempSync(join(tmpdir(), 'gbrain-foreground-gateway-'));
mkdirSync(join(home, '.gbrain'));
writeFileSync(join(home, '.gbrain', 'config.json'), JSON.stringify({
engine: 'pglite',
database_path: join(home, '.gbrain', 'brain.pglite'),
chat_model: 'openai:example-chat-model',
}));
process.env.GBRAIN_HOME = home;
process.env.OPENAI_API_KEY = 'test-key';
resetGateway();
});
afterEach(() => {
resetGateway();
if (originalHome === undefined) delete process.env.GBRAIN_HOME;
else process.env.GBRAIN_HOME = originalHome;
if (originalOpenAiKey === undefined) delete process.env.OPENAI_API_KEY;
else process.env.OPENAI_API_KEY = originalOpenAiKey;
rmSync(home, { recursive: true, force: true });
});
describe('foreground chat gateway initialization (#2590)', () => {
test('extract-conversation-facts initializes a cold gateway from persisted config before its availability gate', async () => {
const exit = spyOn(process, 'exit').mockImplementation((() => {
throw new Error('unexpected process.exit');
}) as never);
try {
await runExtractConversationFacts({
executeRaw: async () => [],
} as never, []);
} finally {
exit.mockRestore();
}
expect(exit).not.toHaveBeenCalled();
expect(isAvailable('chat')).toBe(true);
});
test('enrich initializes a cold gateway from persisted config before its availability gate', async () => {
const exit = spyOn(process, 'exit').mockImplementation((() => {
throw new Error('unexpected process.exit');
}) as never);
try {
await runEnrich({
executeRaw: async () => [],
getConfig: async () => null,
} as never, ['--yes']);
} finally {
exit.mockRestore();
}
expect(exit).not.toHaveBeenCalled();
expect(isAvailable('chat')).toBe(true);
});
});
+63
View File
@@ -0,0 +1,63 @@
/**
* reconfigureGatewayWithEngine tier-resolved models join the extended set.
*
* assertTouchpoint's extended-models contract (model-resolver.ts) says models
* the user opted into via config `models.default` and `models.tier.*`
* included bypass the native recipe allowlist. Pre-fix, only chat/expansion/
* embedding/reranker were registered, so a model reachable ONLY through a tier
* (e.g. `models.tier.deep` set to an Opus newer than the recipe list) failed
* `probeChatModel` and silently degraded think/auto_think to the gather-only
* stub mislabeled NO_ANTHROPIC_API_KEY.
*
* Uses a deliberately fictional model id so the test stays valid no matter how
* current the recipe list is.
*/
import { describe, test, expect, afterEach } from 'bun:test';
import {
configureGateway,
reconfigureGatewayWithEngine,
resetGateway,
validateModelId,
} from '../src/core/ai/gateway.ts';
import type { BrainEngine } from '../src/core/engine.ts';
function stubEngine(config: Record<string, string>): BrainEngine {
return { getConfig: async (k: string) => config[k] ?? null } as unknown as BrainEngine;
}
afterEach(() => {
resetGateway();
});
describe('reconfigureGatewayWithEngine — tier models extend the allowlist', () => {
test('a models.tier.deep model unknown to the recipe validates after reconfigure', async () => {
configureGateway({
embedding_model: 'openai:text-embedding-3-large',
embedding_dimensions: 1536,
env: { ANTHROPIC_API_KEY: 'sk-fake', OPENAI_API_KEY: 'sk-fake' },
});
// Pre-reconfigure: an id absent from the recipe allowlist is rejected.
expect(validateModelId('anthropic:claude-hypothetical-9').ok).toBe(false);
await reconfigureGatewayWithEngine(
stubEngine({ 'models.tier.deep': 'anthropic:claude-hypothetical-9' }),
);
// Post-reconfigure: the tier-configured model is in the extended set.
expect(validateModelId('anthropic:claude-hypothetical-9').ok).toBe(true);
// An id configured NOWHERE stays rejected — the allowlist still bites.
expect(validateModelId('anthropic:claude-never-configured-1').ok).toBe(false);
});
test('models.default reaches the extended set through tier resolution', async () => {
configureGateway({
embedding_model: 'openai:text-embedding-3-large',
embedding_dimensions: 1536,
env: { ANTHROPIC_API_KEY: 'sk-fake', OPENAI_API_KEY: 'sk-fake' },
});
await reconfigureGatewayWithEngine(
stubEngine({ 'models.default': 'anthropic:claude-hypothetical-10' }),
);
expect(validateModelId('anthropic:claude-hypothetical-10').ok).toBe(true);
});
});
+12 -2
View File
@@ -19,8 +19,14 @@ import { mkdtempSync, existsSync, readdirSync, statSync, rmSync } from 'fs';
import { homedir, tmpdir } from 'os';
import { join } from 'path';
// Save original env so we don't leak between tests.
// Save original env so we don't leak between tests. #2823: GBRAIN_AUDIT_DIR
// must be captured too — the shared test bootstrap (test/helpers/audit-dir-preload.ts)
// sets a process-global scratch dir before any test file runs, so "restore"
// here means "put back the preload's value," not "delete the var and let
// it fall through to the real ~/.gbrain/audit for every test file that
// runs after this one in the same shard process."
const ORIG_GBRAIN_HOME = process.env.GBRAIN_HOME;
const ORIG_GBRAIN_AUDIT_DIR = process.env.GBRAIN_AUDIT_DIR;
function fresh(): string {
return mkdtempSync(join(tmpdir(), 'gbrain-home-isolation-'));
@@ -134,7 +140,11 @@ describe('GBRAIN_HOME write-side isolation', () => {
expect(resolveAuditDir()).toBe(auditTmp);
} finally {
process.env.GBRAIN_HOME = ORIG_GBRAIN_HOME;
delete process.env.GBRAIN_AUDIT_DIR;
if (ORIG_GBRAIN_AUDIT_DIR === undefined) {
delete process.env.GBRAIN_AUDIT_DIR;
} else {
process.env.GBRAIN_AUDIT_DIR = ORIG_GBRAIN_AUDIT_DIR;
}
rmSync(tmp, { recursive: true, force: true });
rmSync(auditTmp, { recursive: true, force: true });
}
+60
View File
@@ -13,6 +13,7 @@ import { describe, test, expect, beforeAll, afterAll, mock } from 'bun:test';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { MinionWorker } from '../src/core/minions/worker.ts';
import { registerBuiltinHandlers } from '../src/commands/jobs.ts';
import { configureGateway, getChatModel, resetGateway } from '../src/core/ai/gateway.ts';
let engine: PGLiteEngine;
let worker: MinionWorker;
@@ -122,6 +123,65 @@ describe('autopilot-cycle handler — partial failure does NOT throw', () => {
});
describe('autopilot-cycle handler — phase passthrough', () => {
test('refreshes DB-backed chat model config before a queued cycle runs', async () => {
const handler = (worker as any).handlers.get('autopilot-cycle');
expect(handler).toBeDefined();
const oldModel = await engine.getConfig('models.chat');
configureGateway({
chat_model: 'anthropic:claude-sonnet-4-6',
env: { ANTHROPIC_API_KEY: 'stale-key', OPENAI_API_KEY: 'fresh-key' },
});
await engine.setConfig('models.chat', 'openai:gpt-5');
try {
const result = await handler({
data: { phases: ['orphans'], pull: false },
signal: { aborted: false } as any,
job: { id: 9, name: 'autopilot-cycle' } as any,
});
expect(result).toBeDefined();
expect(getChatModel()).toBe('openai:gpt-5');
} finally {
resetGateway();
if (oldModel === null) {
await engine.unsetConfig('models.chat');
} else {
await engine.setConfig('models.chat', oldModel);
}
}
});
test('refreshes DB-backed chat model config before gateway-backed handlers validate job data', async () => {
const handler = (worker as any).handlers.get('enrich');
expect(handler).toBeDefined();
const oldModel = await engine.getConfig('models.chat');
configureGateway({
chat_model: 'anthropic:claude-sonnet-4-6',
env: { ANTHROPIC_API_KEY: 'stale-key', OPENAI_API_KEY: 'fresh-key' },
});
await engine.setConfig('models.chat', 'openai:gpt-5');
try {
await expect(handler({
data: {},
signal: { aborted: false } as any,
job: { id: 10, name: 'enrich' } as any,
})).rejects.toThrow('enrich Minion job requires data.sourceId');
expect(getChatModel()).toBe('openai:gpt-5');
} finally {
resetGateway();
if (oldModel === null) {
await engine.unsetConfig('models.chat');
} else {
await engine.setConfig('models.chat', oldModel);
}
}
});
test('job.data.phases restricts which phases run', async () => {
const fs = await import('fs');
const { execSync } = await import('child_process');
+51
View File
@@ -0,0 +1,51 @@
/**
* Pre-test setup: redirect audit-writer output (content-sanity,
* shell-audit, supervisor-audit, slug-fallback, etc. every module built
* on `src/core/audit/audit-writer.ts`) to a per-run scratch directory
* instead of the operator's real `~/.gbrain/audit/`.
*
* Why this exists (#2823): `audit-writer.ts::resolveAuditDir()` honors a
* `GBRAIN_AUDIT_DIR` env override, but nothing in the shared test bootstrap
* ever set it. Any test that exercises an audit-emitting code path without
* wrapping the call in its own `withEnv({ GBRAIN_AUDIT_DIR: ... })` (most
* don't only the content-sanity-focused suites did) fell through to the
* real default and appended fixture rows into the operator's live audit
* trail. `test/import-file.test.ts`'s oversize-content boundary fixture
* (`'borderline-slug'`, content just under `MAX_FILE_SIZE` but over
* `DEFAULT_BYTES_BLOCK`) is the concrete offender named in the issue: it
* fires a real `soft_block` content-sanity event on every run, landing in
* `~/.gbrain/audit/content-sanity-YYYY-Www.jsonl` right alongside real
* production signal that doctor's `content_sanity_audit_recent` check
* reads.
*
* Fix: set `GBRAIN_AUDIT_DIR` once, globally, before any test file loads,
* to a fresh `mkdtemp` directory unique to THIS process. Each
* `scripts/run-unit-shard.sh` shard is its own `bun test` process, so each
* shard gets its own scratch dir automatically no cross-shard collision,
* no manual cleanup needed (short-lived test process; OS reaps tmp, same
* tradeoff `test/helpers/with-env.ts`'s `emptyHome()` documents).
*
* Individual test files that already manage their own isolated
* `GBRAIN_AUDIT_DIR` per-test via `withEnv` (e.g.
* `test/import-file-content-sanity.test.ts`, `test/audit/content-sanity-audit.test.ts`)
* are unaffected `withEnv` saves/restores around whatever this preload
* set as the process-global default, same as any other env var.
*
* Only sets the var if it isn't already set, so a developer who exports
* `GBRAIN_AUDIT_DIR` themselves (e.g. to inspect audit output after a
* local run) keeps their override.
*
* Imported by `bunfig.toml` via
* `preload = [..., "./test/helpers/audit-dir-preload.ts"]`.
*/
import { mkdtempSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
if (!process.env.GBRAIN_AUDIT_DIR) {
const dir = mkdtempSync(join(tmpdir(), 'gbrain-test-audit-'));
process.env.GBRAIN_AUDIT_DIR = dir;
if (process.env.GBRAIN_DEBUG_PRELOAD === '1') {
console.error(`[audit-dir-preload] GBRAIN_AUDIT_DIR=${dir}`);
}
}
+14
View File
@@ -1967,6 +1967,20 @@ describe('MinionQueue: v0.19.1 maxWaiting — cap correctness + race (D2/H2)', (
expect(b.queue).toBe('shell');
});
test('cross-source isolation — waiting sync for source A does NOT swallow source B (multi-source regression)', async () => {
// Regression: the cap counted all waiting (name, queue) rows regardless
// of data.sourceId, so a waiting default-source sync coalesced away every
// other source's freshness sync — a secondary source sat 29h stale while
// dispatch logs showed its syncs "dispatched".
const a = await queue.add('srcsync', { sourceId: 'default' }, { maxWaiting: 1 });
const a2 = await queue.add('srcsync', { sourceId: 'default' }, { maxWaiting: 1 });
expect(a2.id).toBe(a.id); // same source still coalesces
const b = await queue.add('srcsync', { sourceId: 'projects' }, { maxWaiting: 1 });
expect(b.id).not.toBe(a.id); // different source MUST get its own row
const b2 = await queue.add('srcsync', { sourceId: 'projects' }, { maxWaiting: 1 });
expect(b2.id).toBe(b.id); // and its own cap
});
test('unset maxWaiting — normal submit path, no coalesce, no cap', async () => {
const a = await queue.add('uncapped', {});
const b = await queue.add('uncapped', {});
+51
View File
@@ -25,6 +25,8 @@ import {
type ProposeTakesExtractor,
type ProposedTake,
} from '../src/core/cycle/propose-takes.ts';
import { configureGateway, resetGateway } from '../src/core/ai/gateway.ts';
import { BudgetMeter } from '../src/core/cycle/budget-meter.ts';
import type { OperationContext } from '../src/core/operations.ts';
import type { BrainEngine } from '../src/core/engine.ts';
import type { Page } from '../src/core/types.ts';
@@ -384,4 +386,53 @@ New prose appended here.`;
expect(typeof runIdA).toBe('string');
expect((runIdA as string).startsWith('propose-')).toBe(true);
});
test('records the configured gateway chat model when no phase model override is passed', async () => {
configureGateway({
chat_model: 'openai:gpt-5',
env: { OPENAI_API_KEY: 'test-key' },
});
try {
const pages = [buildPage({ slug: 'wiki/model-default', body: 'configured model should be recorded' })];
const { engine, captured } = buildMockEngine({ pages });
const extractor: ProposeTakesExtractor = async () => [
{ claim_text: 'configured model should be recorded', kind: 'take', holder: 'brain', weight: 0.5 },
];
await runPhaseProposeTakes(buildCtx(engine), { extractor });
const insert = captured.find(c => c.sql.includes('INSERT INTO take_proposals'));
expect(insert).toBeDefined();
expect(insert!.params[11]).toBe('openai:gpt-5');
} finally {
resetGateway();
}
});
test('keeps nested provider model ids intact for budget checks and proposal records', async () => {
configureGateway({
chat_model: 'openrouter:anthropic/claude-sonnet-4-6',
env: { OPENROUTER_API_KEY: 'test-key' },
});
try {
const pages = [buildPage({ slug: 'wiki/openrouter-model', body: 'nested provider model should stay intact' })];
const { engine, captured } = buildMockEngine({ pages });
const extractor: ProposeTakesExtractor = async () => [
{ claim_text: 'nested provider model should stay intact', kind: 'take', holder: 'brain', weight: 0.5 },
];
const result = await runPhaseProposeTakes(buildCtx(engine), {
extractor,
meter: new BudgetMeter({ budgetUsd: 0.000001, phase: 'propose_takes' }),
});
expect(result.status).toBe('ok');
expect(result.details.budget_exhausted).toBe(false);
const insert = captured.find(c => c.sql.includes('INSERT INTO take_proposals'));
expect(insert).toBeDefined();
expect(insert!.params[11]).toBe('openrouter:anthropic/claude-sonnet-4-6');
} finally {
resetGateway();
}
});
});
+107
View File
@@ -0,0 +1,107 @@
/**
* #2863 regression `gbrain providers test --model` must resolve
* `provider_base_urls` the same way the production embed/chat path does.
*
* Before the fix, the `--model` override branch in `runTest`
* (src/commands/providers.ts) forwarded only `embedding_model`/`chat_model`
* + `env` into `configureGateway`, dropping `config.provider_base_urls`
* entirely. A brain configured with a custom (e.g. China-region DashScope)
* endpoint would pass `gbrain providers test --touchpoint embedding` (no
* `--model`, uses configureFromEnv() which DOES forward base_urls) but fail
* `gbrain providers test --touchpoint embedding --model
* dashscope:text-embedding-v3` with a misleading "Incorrect API key" error
* the probe silently fell back to the recipe's hardcoded default endpoint
* (dashscope-intl.aliyuncs.com) instead of the configured one.
*
* This test drives the real `runProviders('test', ...)` CLI path end to end
* (loadConfig -> configureGateway -> gateway -> AI SDK -> fetch) and asserts
* on the actual HTTP request URL, so it fails on the pre-fix code and only
* passes once the --model override reuses buildGatewayConfig (the same
* resolver src/cli.ts#connectEngine and init-embed-check.ts use for the
* production path).
*/
import { afterEach, beforeEach, describe, expect, test } from 'bun:test';
import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { runProviders } from '../src/commands/providers.ts';
import { resetGateway } from '../src/core/ai/gateway.ts';
import { withEnv } from './helpers/with-env.ts';
const CUSTOM_BASE_URL = 'https://llm-custom.cn-beijing.maas.example.test/compatible-mode/v1';
type FetchHandler = (url: string, init: RequestInit) => Promise<Response>;
let fetchHandler: FetchHandler | null = null;
const origFetch = globalThis.fetch;
let tmpHome: string;
function okEmbeddingResponse(dims: number): Response {
const vec = Array(dims).fill(0).map((_, i) => 0.001 * i);
return new Response(
JSON.stringify({ data: [{ embedding: vec }] }),
{ status: 200, headers: { 'Content-Type': 'application/json' } },
);
}
beforeEach(() => {
fetchHandler = null;
globalThis.fetch = (async (url: string | URL | Request, init?: RequestInit) => {
if (!fetchHandler) throw new Error('fetch called but no handler installed');
return fetchHandler(typeof url === 'string' ? url : url.toString(), init ?? {});
}) as typeof fetch;
tmpHome = mkdtempSync(join(tmpdir(), 'gbrain-providers-test-base-url-'));
mkdirSync(join(tmpHome, '.gbrain'), { recursive: true });
writeFileSync(
join(tmpHome, '.gbrain', 'config.json'),
JSON.stringify({
embedding_model: 'dashscope:text-embedding-v3',
embedding_dimensions: 1024,
provider_base_urls: { dashscope: CUSTOM_BASE_URL },
}),
);
});
afterEach(() => {
globalThis.fetch = origFetch;
resetGateway();
rmSync(tmpHome, { recursive: true, force: true });
});
describe('providers test --model — provider_base_urls (#2863)', () => {
test('embedding touchpoint probe hits the configured custom base URL, not the recipe default', async () => {
let capturedUrl = '';
fetchHandler = async (url) => {
capturedUrl = url;
return okEmbeddingResponse(1024);
};
await withEnv(
{ GBRAIN_HOME: tmpHome, DASHSCOPE_API_KEY: 'test-dashscope-key' },
async () => {
await runProviders('test', ['--touchpoint', 'embedding', '--model', 'dashscope:text-embedding-v3']);
},
);
expect(capturedUrl.startsWith(CUSTOM_BASE_URL)).toBe(true);
expect(capturedUrl).not.toContain('dashscope-intl.aliyuncs.com');
});
test('bare `providers test` (no --model) already used the custom base URL (control)', async () => {
let capturedUrl = '';
fetchHandler = async (url) => {
capturedUrl = url;
return okEmbeddingResponse(1024);
};
await withEnv(
{ GBRAIN_HOME: tmpHome, DASHSCOPE_API_KEY: 'test-dashscope-key' },
async () => {
await runProviders('test', ['--touchpoint', 'embedding']);
},
);
expect(capturedUrl.startsWith(CUSTOM_BASE_URL)).toBe(true);
});
});
@@ -0,0 +1,21 @@
/**
* #2900 `reconcile-links` CLI reachability.
*
* `reconcile-links` is advertised in `gbrain --help` and fully implemented
* with a `case 'reconcile-links'` block in cli.ts's handleCliOnly switch, but
* it was missing from the CLI_ONLY Set. Dispatch only reaches handleCliOnly
* when the command is in CLI_ONLY, so every invocation fell through to the
* shared-operations lookup and hit the generic "Unknown command" branch
* leaving the documented docimpl edge-rebuild tool silently unreachable.
*
* Same class of drift as #2035 (`calibration`).
*/
import { describe, test, expect } from 'bun:test';
import { CLI_ONLY } from '../src/cli.ts';
describe('CLI_ONLY command reachability (#2900)', () => {
test('`reconcile-links` is in CLI_ONLY so dispatch reaches its handler', () => {
expect(CLI_ONLY.has('reconcile-links')).toBe(true);
});
});
+255 -3
View File
@@ -15,6 +15,7 @@ import {
} from 'fs';
import { join } from 'path';
import { tmpdir } from 'os';
import { execFileSync } from 'child_process';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import {
addSource,
@@ -358,7 +359,10 @@ describe('removeSource — clone-cleanup', () => {
const userPath = join(GBRAIN_HOME, 'user-managed-fixture');
mkdirSync(userPath, { recursive: true });
writeFileSync(join(userPath, 'file'), 'hi');
await addSource(engine, { id: 'cleanup-no', localPath: userPath });
// #2707: this fixture is intentionally not a git repo (unrelated to
// what this test covers — clone-cleanup ownership) — force past the
// registration-time git check.
await addSource(engine, { id: 'cleanup-no', localPath: userPath, force: true });
const result = await removeSource(engine, {
id: 'cleanup-no',
confirmDestructive: true,
@@ -485,8 +489,10 @@ describe('getSourceStatus', () => {
// path-only source still gets validateRepoState — but with no expected
// URL, it just probes existence + .git. Path exists with no .git → 'no-git'.
// To match contract docstring we'd want 'not-applicable' only when
// local_path is null. Test the truthful behavior:
await addSource(engine, { id: 'status-no-url', localPath: userPath });
// local_path is null. Test the truthful behavior. #2707: this fixture
// is deliberately no-git (that's what's under test for getSourceStatus)
// — force past the registration-time git check to construct it.
await addSource(engine, { id: 'status-no-url', localPath: userPath, force: true });
const s = await getSourceStatus(engine, 'status-no-url');
// local_path set but no .git: returns 'no-git'
expect(s.clone_state).toBe('no-git');
@@ -814,6 +820,252 @@ describe('addSource --url — writes ownership marker', () => {
});
});
// ---------------------------------------------------------------------------
// addSource --path — #2707 git-repo validation at registration time
//
// Deliberately does NOT run under withEnv2 (the fake-git harness above):
// writeFakeGit()'s catch-all `exit 0` would make `rev-parse --show-toplevel`
// (and therefore isInsideGitRepo) succeed unconditionally for any path,
// which defeats the point of these tests. Real system git applies here.
// ---------------------------------------------------------------------------
describe('addSource --path — #2707 git-repo validation', () => {
const SANDBOX = join(tmpdir(), `gbrain-2707-git-validate-${process.pid}`);
beforeEach(() => {
rmSync(SANDBOX, { recursive: true, force: true });
mkdirSync(SANDBOX, { recursive: true });
});
afterAll(() => {
rmSync(SANDBOX, { recursive: true, force: true });
});
test('rejects an existing non-git directory with an actionable error', async () => {
const plainDir = join(SANDBOX, 'plain');
mkdirSync(plainDir, { recursive: true });
writeFileSync(join(plainDir, 'notes.md'), 'not committed anywhere');
let threw: SourceOpError | undefined;
try {
await addSource(engine, { id: 'plain-src', localPath: plainDir });
} catch (e) {
threw = e as SourceOpError;
}
expect(threw).toBeInstanceOf(SourceOpError);
expect(threw?.code).toBe('not_a_git_repo');
expect(threw?.message).toContain(plainDir);
expect(threw?.message).toContain('--force');
expect(threw?.message).toMatch(/git .*init/);
// Source was never registered — no partial row left behind.
const rows = await engine.executeRaw<{ id: string }>(
`SELECT id FROM sources WHERE id = 'plain-src'`,
);
expect(rows.length).toBe(0);
});
test('rejects a git-initialized directory with zero commits (codex round 1)', async () => {
const unbornDir = join(SANDBOX, 'unborn');
mkdirSync(unbornDir, { recursive: true });
execFileSync('git', ['-C', unbornDir, 'init', '-q']);
// No `git add` / `git commit` — isInsideGitRepo alone would pass this.
let threw: SourceOpError | undefined;
try {
await addSource(engine, { id: 'unborn-src', localPath: unbornDir });
} catch (e) {
threw = e as SourceOpError;
}
expect(threw).toBeInstanceOf(SourceOpError);
expect(threw?.code).toBe('not_a_git_repo');
});
test('rejects an empty-commit repo whose files are untracked (codex round 2)', async () => {
// git commit --allow-empty gives a resolvable HEAD (so a bare "has a
// commit" check would wrongly pass this) but the tree is empty; files
// written afterward are untracked and invisible to the sync walker.
const emptyCommitDir = join(SANDBOX, 'empty-commit');
mkdirSync(emptyCommitDir, { recursive: true });
execFileSync('git', ['-C', emptyCommitDir, 'init', '-q']);
execFileSync('git', ['-C', emptyCommitDir, 'config', 'user.email', 'test@example.com']);
execFileSync('git', ['-C', emptyCommitDir, 'config', 'user.name', 'Test']);
execFileSync('git', ['-C', emptyCommitDir, 'commit', '--allow-empty', '-q', '-m', 'empty']);
writeFileSync(join(emptyCommitDir, 'notes.md'), 'never committed');
let threw: SourceOpError | undefined;
try {
await addSource(engine, { id: 'empty-commit-src', localPath: emptyCommitDir });
} catch (e) {
threw = e as SourceOpError;
}
expect(threw).toBeInstanceOf(SourceOpError);
expect(threw?.code).toBe('not_a_git_repo');
});
test('rejects an untracked subdirectory of an otherwise-real git repo (codex round 2)', async () => {
const parent = join(SANDBOX, 'partial-repo');
const trackedFile = join(parent, 'README.md');
const untrackedSub = join(parent, 'untracked-sub');
mkdirSync(untrackedSub, { recursive: true });
writeFileSync(trackedFile, '# fixture');
execFileSync('git', ['-C', parent, 'init', '-q']);
execFileSync('git', ['-C', parent, 'config', 'user.email', 'test@example.com']);
execFileSync('git', ['-C', parent, 'config', 'user.name', 'Test']);
execFileSync('git', ['-C', parent, 'add', 'README.md']);
execFileSync('git', ['-C', parent, 'commit', '-q', '-m', 'initial import']);
writeFileSync(join(untrackedSub, 'x.md'), 'never git add-ed');
let threw: SourceOpError | undefined;
try {
await addSource(engine, { id: 'partial-repo-src', localPath: untrackedSub });
} catch (e) {
threw = e as SourceOpError;
}
expect(threw).toBeInstanceOf(SourceOpError);
expect(threw?.code).toBe('not_a_git_repo');
});
test('registers a repo with many tracked entries without buffer-size false rejection (codex round 3)', async () => {
// Codex round 3 (P2): the earlier `git ls-tree` listing implementation
// buffered the whole tree and could exceed execFileSync's default 1 MiB
// maxBuffer on a large repo, causing an incorrect rejection. The
// rev-parse HEAD:./ + empty-tree-SHA-comparison implementation reads a
// fixed ~40-byte SHA regardless of tree size — this locks that in.
const bigDir = join(SANDBOX, 'many-entries');
mkdirSync(bigDir, { recursive: true });
for (let i = 0; i < 300; i++) {
writeFileSync(join(bigDir, `file-${i}.md`), `# entry ${i}`);
}
execFileSync('git', ['-C', bigDir, 'init', '-q']);
execFileSync('git', ['-C', bigDir, 'config', 'user.email', 'test@example.com']);
execFileSync('git', ['-C', bigDir, 'config', 'user.name', 'Test']);
execFileSync('git', ['-C', bigDir, 'add', '-A']);
execFileSync('git', ['-C', bigDir, 'commit', '-q', '-m', 'many files']);
const row = await addSource(engine, { id: 'many-entries-src', localPath: bigDir });
expect(row.local_path).toBe(bigDir);
});
// Codex round 4 (P2): the empty-tree OID is hash-algorithm-specific — a
// hardcoded SHA-1 constant silently mismatched (and so accepted) an empty
// SHA-256 repo. --object-format=sha256 needs git 2.29+; skip rather than
// hard-fail on an older CI git.
const SHA256_SUPPORTED = (() => {
try {
const probe = join(tmpdir(), `gbrain-2707-sha256-probe-${process.pid}`);
mkdirSync(probe, { recursive: true });
execFileSync('git', ['-C', probe, 'init', '-q', '--object-format=sha256'], {
stdio: ['ignore', 'pipe', 'pipe'],
});
rmSync(probe, { recursive: true, force: true });
return true;
} catch {
return false;
}
})();
test.skipIf(!SHA256_SUPPORTED)(
'rejects an empty-commit SHA-256 repo the same as a SHA-1 one (codex round 4)',
async () => {
const sha256Dir = join(SANDBOX, 'sha256-empty');
mkdirSync(sha256Dir, { recursive: true });
execFileSync('git', ['-C', sha256Dir, 'init', '-q', '--object-format=sha256']);
execFileSync('git', ['-C', sha256Dir, 'config', 'user.email', 'test@example.com']);
execFileSync('git', ['-C', sha256Dir, 'config', 'user.name', 'Test']);
execFileSync('git', ['-C', sha256Dir, 'commit', '--allow-empty', '-q', '-m', 'empty']);
writeFileSync(join(sha256Dir, 'notes.md'), 'never committed');
let threw: SourceOpError | undefined;
try {
await addSource(engine, { id: 'sha256-empty-src', localPath: sha256Dir });
} catch (e) {
threw = e as SourceOpError;
}
expect(threw).toBeInstanceOf(SourceOpError);
expect(threw?.code).toBe('not_a_git_repo');
},
);
test.skipIf(!SHA256_SUPPORTED)(
'registers a SHA-256 repo with real committed content (no regression)',
async () => {
const sha256Dir = join(SANDBOX, 'sha256-real');
mkdirSync(sha256Dir, { recursive: true });
writeFileSync(join(sha256Dir, 'README.md'), '# fixture');
execFileSync('git', ['-C', sha256Dir, 'init', '-q', '--object-format=sha256']);
execFileSync('git', ['-C', sha256Dir, 'config', 'user.email', 'test@example.com']);
execFileSync('git', ['-C', sha256Dir, 'config', 'user.name', 'Test']);
execFileSync('git', ['-C', sha256Dir, 'add', '-A']);
execFileSync('git', ['-C', sha256Dir, 'commit', '-q', '-m', 'initial import']);
const row = await addSource(engine, { id: 'sha256-real-src', localPath: sha256Dir });
expect(row.local_path).toBe(sha256Dir);
},
);
test('quotes a path with a space in the remediation command (codex round 1)', async () => {
const spacedDir = join(SANDBOX, 'has space here');
mkdirSync(spacedDir, { recursive: true });
let threw: SourceOpError | undefined;
try {
await addSource(engine, { id: 'spaced-src', localPath: spacedDir });
} catch (e) {
threw = e as SourceOpError;
}
expect(threw?.message).toContain(`'${spacedDir}'`);
});
test('--force bypasses the check and registers the plain directory as-is', async () => {
const plainDir = join(SANDBOX, 'plain-forced');
mkdirSync(plainDir, { recursive: true });
const row = await addSource(engine, {
id: 'plain-forced-src',
localPath: plainDir,
force: true,
});
expect(row.local_path).toBe(plainDir);
});
test('an already git-initialized directory registers unaffected (no regression)', async () => {
const gitDir = join(SANDBOX, 'gitrepo');
mkdirSync(gitDir, { recursive: true });
writeFileSync(join(gitDir, 'README.md'), '# fixture');
execFileSync('git', ['-C', gitDir, 'init', '-q']);
execFileSync('git', ['-C', gitDir, 'config', 'user.email', 'test@example.com']);
execFileSync('git', ['-C', gitDir, 'config', 'user.name', 'Test']);
execFileSync('git', ['-C', gitDir, 'add', '-A']);
execFileSync('git', ['-C', gitDir, 'commit', '-q', '-m', 'initial import']);
const row = await addSource(engine, { id: 'gitrepo-src', localPath: gitDir });
expect(row.local_path).toBe(gitDir);
});
test('a subdirectory of a git repo registers unaffected (#753/#774 parity with sync-time discovery)', async () => {
const gitDir = join(SANDBOX, 'gitrepo-parent');
const subDir = join(gitDir, 'sub');
mkdirSync(subDir, { recursive: true });
writeFileSync(join(subDir, 'README.md'), '# fixture');
execFileSync('git', ['-C', gitDir, 'init', '-q']);
execFileSync('git', ['-C', gitDir, 'config', 'user.email', 'test@example.com']);
execFileSync('git', ['-C', gitDir, 'config', 'user.name', 'Test']);
execFileSync('git', ['-C', gitDir, 'add', '-A']);
execFileSync('git', ['-C', gitDir, 'commit', '-q', '-m', 'initial import']);
const row = await addSource(engine, { id: 'subdir-src', localPath: subDir });
expect(row.local_path).toBe(subDir);
});
test('a not-yet-created path is unaffected (pre-existing lenient behavior, out of #2707 scope)', async () => {
const missingDir = join(SANDBOX, 'does-not-exist-yet');
expect(existsSync(missingDir)).toBe(false);
const row = await addSource(engine, { id: 'missing-src', localPath: missingDir });
expect(row.local_path).toBe(missingDir);
});
});
// ---------------------------------------------------------------------------
// isPathContained — symlink-safe confinement helper (exported for reuse)
// ---------------------------------------------------------------------------
+45 -1
View File
@@ -6,7 +6,10 @@
* shape, validation, and flag parsing.
*/
import { describe, test, expect, beforeEach } from 'bun:test';
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
import { mkdtempSync, mkdirSync, rmSync } from 'fs';
import { tmpdir } from 'os';
import { join } from 'path';
import { runSources } from '../src/commands/sources.ts';
import type { BrainEngine } from '../src/core/engine.ts';
@@ -158,6 +161,47 @@ describe('sources add', () => {
});
});
// ── add — #2707 git-repo validation (CLI wiring) ───────────────
//
// Uses a REAL on-disk temp dir (unlike the fake-path tests above) so the
// core addSource git check actually runs; the stub engine still fakes the
// DB round-trip. Confirms --force parses through to opsAddSource.
describe('sources add — #2707 --force flag wiring', () => {
let plainDir: string;
beforeEach(() => {
plainDir = mkdtempSync(join(tmpdir(), 'gbrain-sources-cli-2707-'));
});
afterEach(() => {
rmSync(plainDir, { recursive: true, force: true });
});
test('rejects a real non-git --path directory by default', async () => {
const { engine } = makeStub();
await expect(runSources(engine, ['add', 'cli-plain', '--path', plainDir]))
.rejects.toThrow(/not a git repository/);
});
test('--force registers the same directory anyway', async () => {
const { engine, calls } = makeStub({
'SELECT id, name, local_path, last_commit, last_sync_at, config, created_at': [{
id: 'cli-forced',
name: 'cli-forced',
local_path: plainDir,
last_commit: null,
last_sync_at: null,
config: '{}',
created_at: new Date(),
}],
});
await runSources(engine, ['add', 'cli-forced', '--path', plainDir, '--force']);
const insert = calls.find(c => c.sql.includes('INSERT INTO sources'));
expect(insert).toBeDefined();
expect(insert!.params[2]).toBe(plainDir);
});
});
// ── list ────────────────────────────────────────────────────
describe('sources list', () => {
+5 -1
View File
@@ -164,9 +164,13 @@ describe('#1434 — runSync auto-routes to sole_non_default source', () => {
test('2+ non-default sources: no auto-route, no nudge, falls through to default', async () => {
// Both need local_path to be counted by the sole_non_default helper.
// Pre-existing helper filters local_path IS NOT NULL.
// secondRepo is a bare temp dir (no git init) — its content is
// irrelevant to what this test verifies (that 2+ non-default sources
// disable auto-routing); --force skips #2707's registration-time git
// validation, which is orthogonal to this test's assertion.
const secondRepo = mkdtempSync(join(tmpdir(), 'gbrain-snd-routing-second-'));
await runSources(engine, ['add', 'studiovault', '--path', repoPath, '--no-federated']);
await runSources(engine, ['add', 'second-vault', '--path', secondRepo, '--no-federated']);
await runSources(engine, ['add', 'second-vault', '--path', secondRepo, '--no-federated', '--force']);
const { runSync } = await import('../src/commands/sync.ts');
const origWrite = process.stderr.write.bind(process.stderr);
+17
View File
@@ -186,6 +186,23 @@ describe('think gateway adapter — #1698 slash form + explicit-model fork', ()
});
});
describe('think gateway adapter — current-generation recipe models', () => {
test('builds clients for Opus 4.8 / Sonnet 5 / Fable 5 (recipe-list refresh)', async () => {
// Regression guard: these GA models were absent from the recipe allowlist,
// so a tier-configured deep model degraded think to the no-LLM stub.
await withEnv({ ANTHROPIC_API_KEY: 'sk-test-fake' }, async () => {
for (const id of [
'anthropic:claude-opus-4-8',
'anthropic:claude-sonnet-5',
'anthropic:claude-fable-5',
]) {
const client = await __thinkAdapter.tryBuildGatewayClient(id);
expect(client).not.toBeNull();
}
});
});
});
describe('think gateway adapter — graceful fallback shape', () => {
test('buildGracefulMessage produces a parseable Anthropic.Message-shaped object', () => {
const m = __thinkAdapter.buildGracefulMessage('anthropic:claude-opus-4-7');
+22 -1
View File
@@ -217,6 +217,24 @@ describe('runThink (with stub client)', () => {
expect(result.rounds).toBe(0);
});
test('labels an unusable CONFIGURED model honestly (MODEL_NOT_USABLE, not NO_ANTHROPIC_API_KEY)', async () => {
// Regression guard: a configured model the recipe rejects (unknown_model)
// used to be stamped NO_ANTHROPIC_API_KEY, sending operators to debug
// env/keychain when the fix was the model id. Model validity beats the key
// check in probeChatModel, so the honest label holds even keyless.
await engine.setConfig('models.think', 'anthropic:claude-bogus-9');
try {
const result = await withoutAnthropicKey(() => runThink(engine, { question: 'bad model test' }));
expect(result.warnings).toContain('MODEL_NOT_USABLE:unknown_model');
expect(result.warnings).not.toContain('NO_ANTHROPIC_API_KEY');
expect(result.answer).toContain('not usable');
expect(result.rounds).toBe(0);
expect(result.synthesisOk).toBe(false);
} finally {
await engine.unsetConfig('models.think');
}
});
test('persistSynthesis writes synthesis page + evidence rows', async () => {
const stubClient: ThinkLLMClient = {
create: async () => ({
@@ -285,8 +303,11 @@ describe('runThink — #1698 explicit-model hard error', () => {
test('NON-explicit bad model does NOT throw — graceful degrade (no modelExplicit)', async () => {
// model present but modelExplicit unset → early gate skipped; builder returns null.
// Hermetic no-key so the assertion can't be perturbed by a configured key.
// Post-honest-labeling: an unknown PROVIDER is a model problem, not a key
// problem — the warning names it instead of the old NO_ANTHROPIC_API_KEY
// catch-all. The graceful no-throw contract is unchanged.
const result = await withoutAnthropicKey(() => runThink(engine, { question: 'nonexplicit bad', model: 'bogusprovider:foo' }));
expect(result.warnings).toContain('NO_ANTHROPIC_API_KEY');
expect(result.warnings).toContain('MODEL_NOT_USABLE:unknown_provider');
expect(result.synthesisOk).toBe(false);
});
});