Compare commits

..
Author SHA1 Message Date
Garry TanandClaude Fable 5 1b168c1a7e fix(search): honor per-call source_id/all_sources in the search op — the #1484 hint's advised retry was silently ignored
The zero-hit CLI hint tells users to retry with --source-id __all__, which
works for `query` (routes through resolveRequestedScope) but the `search`
op had no source_id/all_sources params: the flag parsed into params, nothing
consumed it, and the retry re-ran the same single-source search — an
invisible false negative whose params.source_id also suppressed the hint on
the retry. search now resolves per-call scope through the same canonical
fail-closed resolver as query (local __all__ spans the brain; remote
collapses to the caller's grant; out-of-grant source_id is denied).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 11:32:34 -07:00
Garry TanandClaude Fable 5 8351f31bff test(search): pin gateway to stub OpenAI key in vector-arm warn test
The #1626 telemetry test installed a throwing embed transport but never
configured the gateway, so on keyless CI embed()'s instantiateEmbedding
threw 'OpenAI embedding requires OPENAI_API_KEY' before the transport
ran, and the swallowed-reason assertion failed. Pin the gateway with a
stub key (put-page-provenance pattern) + resetGateway in afterAll.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:49:56 -07:00
Garry TanandClaude Fable 5 16741f64bf fix(ai,search,cli): read embed-shim bodies once (#1610); zero-hit source hint (#1484); log swallowed vector-arm failures (#1626)
Three backlog fixes:

- #1610: voyageCompatFetch/zeroEntropyCompatFetch read the response body
  ONCE via text() + JSON.parse instead of resp.clone().json(). On bun <
  1.1.27 clone() truncates large bodies (oven-sh/bun#6348), the parse threw,
  and the catch fell back to the raw provider shape the AI SDK schema
  rejects — multi-chunk pages died with "Invalid JSON response". Every JSON
  return path now rebuilds the Response and strips the stale
  Content-Length/Content-Encoding headers. Shims exported as test seams;
  behavioral coverage simulates the truncating clone().

- #1484: a bare `gbrain query`/`search` that returns zero results on a
  multi-source brain now prints a stderr hint naming the source that was
  actually searched and how to widen scope (--source-id __all__). Fires only
  when the caller didn't scope explicitly; best-effort (lookup failure is
  silent). Default scope stays unchanged — widening to __all__ is a separate
  maintainer policy call.

- #1626: hybridSearch's text-vector arm no longer swallows failures dark.
  The arm only runs when the embedding provider probed available, so a throw
  (embed timeout, transient pooler error on the searchVector fan-out) now
  logs the reason via warnOncePerProcess while keeping the keyword
  fallback — a cross-source __all__ run can no longer collapse to
  "No results" with zero diagnostics.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 15:05:45 -07:00
22 changed files with 473 additions and 537 deletions
+1 -3
View File
@@ -1411,8 +1411,7 @@ This is the dispatcher. Skills are the implementation. **Read the skill file bef
| "get more out of gbrain", "is my brain set up right", "weekly brain checkup", "advise me on my brain", "gbrain advisor" | `skills/gbrain-advisor/SKILL.md` |
| Save or load reports | `skills/reports/SKILL.md` |
| "Create a skill", "improve this skill" | `skills/skill-creator/SKILL.md` |
| "save this learning to the vault", "capture this skill in Obsidian", "record this workflow in my notes", "put this setup change in the vault" | `skills/skill-vault-capture-policy/SKILL.md` |
| "Skillify this", "is this a skill?", "make this proper", "add tests and evals for this" | `skills/skillify/SKILL.md` |
| "Skillify this", "is this a skill?", "make this proper" | `skills/skillify/SKILL.md` |
| "Compress my resolver", "AGENTS.md too large", "RESOLVER.md too big", "functional area dispatcher", "shrink routing table" | `skills/functional-area-resolver/SKILL.md` |
| "Is gbrain healthy?", morning health check, skillpack-check | `skills/skillpack-check/SKILL.md` |
| "harvest this skill into gbrain", "publish this skill to gbrain", "lift this skill upstream", "share this skill with other gbrain clients", "promote my skill to gbrain" | `skills/skillpack-harvest/SKILL.md` |
@@ -1430,7 +1429,6 @@ This is the dispatcher. Skills are the implementation. **Read the skill file bef
| "Set up GBrain", first boot | `skills/setup/SKILL.md` |
| "Now what?", "fill my brain", "cold start", "bootstrap", "import my data", "what should I import first" | `skills/cold-start/SKILL.md` |
| "Migrate from Obsidian/Notion/Logseq" | `skills/migrate/SKILL.md` |
| "Connect Obsidian to gbrain", "import my vault to gbrain", "sync vault and gbrain", "embed gbrain after vault update", "is gbrain synced with my vault" | `skills/obsidian-gbrain-safe-index/SKILL.md` |
| Brain health check, maintenance run | `skills/maintain/SKILL.md` |
| "Extract links", "build link graph", "populate timeline" | `skills/maintain/SKILL.md` (extraction sections) |
| "Run dream", "process today's session", "synthesize my conversations", "consolidate yesterday's conversations", "what patterns did you see", "did the dream cycle run" | `skills/maintain/SKILL.md` (dream cycle section) |
+2 -3
View File
@@ -39,8 +39,8 @@
"skills/briefing",
"skills/citation-fixer",
"skills/concept-synthesis",
"skills/cron-scheduler",
"skills/cross-modal-review",
"skills/cron-scheduler",
"skills/daily-task-manager",
"skills/daily-task-prep",
"skills/data-research",
@@ -54,11 +54,10 @@
"skills/media-ingest",
"skills/meeting-ingestion",
"skills/minion-orchestrator",
"skills/obsidian-gbrain-safe-index",
"skills/perplexity-research",
"skills/query",
"skills/repo-architecture",
"skills/reports",
"skills/repo-architecture",
"skills/signal-detector",
"skills/skill-creator",
"skills/skillify",
+1 -3
View File
@@ -60,8 +60,7 @@ This is the dispatcher. Skills are the implementation. **Read the skill file bef
| "get more out of gbrain", "is my brain set up right", "weekly brain checkup", "advise me on my brain", "gbrain advisor" | `skills/gbrain-advisor/SKILL.md` |
| Save or load reports | `skills/reports/SKILL.md` |
| "Create a skill", "improve this skill" | `skills/skill-creator/SKILL.md` |
| "save this learning to the vault", "capture this skill in Obsidian", "record this workflow in my notes", "put this setup change in the vault" | `skills/skill-vault-capture-policy/SKILL.md` |
| "Skillify this", "is this a skill?", "make this proper", "add tests and evals for this" | `skills/skillify/SKILL.md` |
| "Skillify this", "is this a skill?", "make this proper" | `skills/skillify/SKILL.md` |
| "Compress my resolver", "AGENTS.md too large", "RESOLVER.md too big", "functional area dispatcher", "shrink routing table" | `skills/functional-area-resolver/SKILL.md` |
| "Is gbrain healthy?", morning health check, skillpack-check | `skills/skillpack-check/SKILL.md` |
| "harvest this skill into gbrain", "publish this skill to gbrain", "lift this skill upstream", "share this skill with other gbrain clients", "promote my skill to gbrain" | `skills/skillpack-harvest/SKILL.md` |
@@ -79,7 +78,6 @@ This is the dispatcher. Skills are the implementation. **Read the skill file bef
| "Set up GBrain", first boot | `skills/setup/SKILL.md` |
| "Now what?", "fill my brain", "cold start", "bootstrap", "import my data", "what should I import first" | `skills/cold-start/SKILL.md` |
| "Migrate from Obsidian/Notion/Logseq" | `skills/migrate/SKILL.md` |
| "Connect Obsidian to gbrain", "import my vault to gbrain", "sync vault and gbrain", "embed gbrain after vault update", "is gbrain synced with my vault" | `skills/obsidian-gbrain-safe-index/SKILL.md` |
| Brain health check, maintenance run | `skills/maintain/SKILL.md` |
| "Extract links", "build link graph", "populate timeline" | `skills/maintain/SKILL.md` (extraction sections) |
| "Run dream", "process today's session", "synthesize my conversations", "consolidate yesterday's conversations", "what patterns did you see", "did the dream cycle run" | `skills/maintain/SKILL.md` (dream cycle section) |
+1 -11
View File
@@ -2,7 +2,7 @@
"name": "gbrain",
"version": "0.32.3.0",
"conformance_version": "1.0.0",
"description": "Personal knowledge brain with hybrid RAG search GStack mod for agent platforms",
"description": "Personal knowledge brain with hybrid RAG search \u2014 GStack mod for agent platforms",
"skills": [
{
"name": "ingest",
@@ -34,11 +34,6 @@
"path": "migrate/SKILL.md",
"description": "Universal migration from Obsidian, Notion, Logseq, markdown, CSV, JSON, Roam"
},
{
"name": "obsidian-gbrain-safe-index",
"path": "obsidian-gbrain-safe-index/SKILL.md",
"description": "Connect and sync an Obsidian-style Markdown vault with gbrain using a cost-controlled import-first workflow, explicit embedding gates, and durable skill/workflow capture."
},
{
"name": "setup",
"path": "setup/SKILL.md",
@@ -268,11 +263,6 @@
"name": "skill-optimizer",
"path": "skill-optimizer/SKILL.md",
"description": "Self-evolving skill optimization via gbrain skillopt — SkillOpt-paper-grounded text-space optimizer with validation gating (median-of-3 + epsilon=0.05), bundled-skill safety, bootstrap review sentinel, per-skill DB lock, and atomic versioned writes."
},
{
"name": "skill-vault-capture-policy",
"path": "skill-vault-capture-policy/SKILL.md",
"description": "Capture durable operational learnings, new agent skills, and environment/setup changes into the Obsidian vault instead of transient chat memory."
}
],
"dependencies": {
-137
View File
@@ -1,137 +0,0 @@
---
name: obsidian-gbrain-safe-index
version: 1.0.0
description: |
Connect, maintain, and sync an Obsidian-style Markdown vault with gbrain while preserving a cost-controlled workflow: the vault remains the source of truth, gbrain is the searchable/embedded index, durable skills/workflows are captured into the vault, and paid embedding runs only after explicit approval.
triggers:
- "connect Obsidian to gbrain"
- "import my vault to gbrain"
- "sync vault and gbrain"
- "capture this skill in my vault"
- "embed gbrain after vault update"
- "is gbrain synced with my vault"
tools:
- terminal
- read_file
- search_files
- write_file
- patch
mutating: true
---
# Obsidian → gbrain Safe Index and Capture
## Contract
This skill guarantees:
- Treats the user's Obsidian-style Markdown vault as the source of truth before gbrain indexing.
- Keeps gbrain in conservative mode unless the user explicitly approves a more expensive mode.
- Imports vault changes with `--no-embed` first, then embeds only after explicit approval for the specific paid action.
- Captures durable new skills, workflows, and environment learnings into the vault instead of leaving them only in chat memory.
- Verifies every sync with concrete `gbrain stats`, search mode, and, when embeddings run, exact embedded chunk counts.
## Phases
1. **Resolve the vault path.**
- Prefer an existing environment variable such as `OBSIDIAN_VAULT_PATH` or `WIKI_PATH`.
- If no path is configured, search likely note directories and ask the user before writing.
- Verify the directory exists and contains markdown files or an `.obsidian` directory.
2. **Read vault operating rules before writing.**
- If the vault has `SCHEMA.md`, `index.md`, `log.md`, `AGENTS.md`, or similar operating files, read them before ingest/query/major edit.
- Respect immutable source folders such as `raw/` when the vault declares them.
- Use the vault's native link convention, usually Obsidian `[[wikilinks]]`, for durable relationships.
3. **MECE/capture decision.**
- If new knowledge belongs on an existing page, update that page.
- If it is a distinct recurring workflow or operational policy, create a small meta or concept page following the vault schema.
- Update the vault index/catalog for every new page when the vault maintains one.
- Append a log entry for meaningful vault updates when the vault maintains a log.
4. **Safe gbrain import path.**
- Pre-check source directory; do not import a nonexistent path.
- Run `gbrain config set search.mode conservative` before/after risky reinit steps.
- Run `gbrain import "$OBSIDIAN_VAULT_PATH" --no-embed`.
- Run `gbrain extract links --source fs --dir "$OBSIDIAN_VAULT_PATH"` when wikilinks changed materially.
5. **Paid embedding gate.**
- Do not run `gbrain embed --stale` unless the user explicitly asks or a prior instruction clearly approved this exact paid action.
- Before embedding, verify provider readiness with `gbrain providers test --model <provider:model>`.
- Confirm the configured embedding dimensions match the local schema.
- After embedding, verify `gbrain stats` and record exact `Pages`, `Chunks`, `Embedded`, and `Links` counts.
6. **Final verification and vault echo.**
- Run `gbrain stats` and `gbrain search modes`.
- If vault files changed, re-import with `--no-embed`; if embedding was approved, embed stale chunks afterward.
- Report what changed, what was free/local, what used API billing, and what remains pending.
## Output Format
Use a compact status table:
| Item | Status |
|---|---|
| Vault path | `/path` |
| Vault updated | yes/no + files |
| gbrain mode | conservative/balanced/tokenmax |
| Import | `--no-embed` completed / skipped / failed |
| Pages/chunks | exact counts from `gbrain stats` |
| Embeddings | exact count; note whether this run used API billing |
| Links | exact count |
| Background jobs | none / list exact jobs |
Then include:
- **Safe next step:** free/local action.
- **Paid next step:** embedding/LLM action, if any, with explicit approval requirement.
## Anti-Patterns
- Creating a duplicate skill/page when an existing Obsidian, gbrain, or vault-ingest skill already covers the workflow.
- Running `gbrain embed --stale`, `gbrain dream`, `gbrain autopilot --install`, `gbrain onboard --auto`, or `tokenmax` without explicit cost approval.
- Importing a nonexistent or wrong directory and treating a zero-page import as success.
- Forgetting to update the vault index/catalog and log after creating or materially updating vault pages.
- Recording API keys, tokens, or raw secrets in the vault or final response.
## Tools Used
- `read_file` — read vault schema/index/log and target notes.
- `search_files` — find existing vault pages and avoid duplicates.
- `write_file` / `patch` — create or update vault pages.
- `terminal` — run `gbrain`, `git`, and environment checks with secret values redacted.
## Safe Commands
```bash
export PATH="$HOME/.bun/bin:$PATH"
gbrain config set search.mode conservative
gbrain import "$OBSIDIAN_VAULT_PATH" --no-embed
gbrain extract links --source fs --dir "$OBSIDIAN_VAULT_PATH"
gbrain stats
gbrain search modes
```
## Paid / Approval-Gated Commands
```bash
gbrain providers test --model <provider:model>
gbrain embed --stale
gbrain dream
gbrain autopilot --install
gbrain onboard --auto --max-usd 5
gbrain config set search.mode tokenmax
```
## Verification Checklist
- [ ] Vault path exists and is the intended source.
- [ ] Vault operating files were read before edits when present.
- [ ] Existing pages/skills were searched to avoid duplicates.
- [ ] New/updated vault pages follow the vault schema and link convention.
- [ ] Vault index/catalog updated for new pages when present.
- [ ] Vault log appended for meaningful actions when present.
- [ ] `gbrain import ... --no-embed` completed.
- [ ] `gbrain stats` recorded pages/chunks/embeddings/links.
- [ ] `gbrain search modes` confirms conservative mode unless a different mode was explicitly approved.
- [ ] No paid/background commands ran without approval.
@@ -1,11 +0,0 @@
// Routing eval fixtures for skills/obsidian-gbrain-safe-index.
// Positive cases: intents embed a trigger phrase in natural surrounding context
// (never verbatim-identical to a trigger — the fixture linter rejects tautologies).
{"intent": "help me connect Obsidian to gbrain for my notes", "expected_skill": "obsidian-gbrain-safe-index"}
{"intent": "import my vault to gbrain but skip embeddings for now", "expected_skill": "obsidian-gbrain-safe-index"}
{"intent": "please sync vault and gbrain after I edit notes", "expected_skill": "obsidian-gbrain-safe-index"}
{"intent": "run embed gbrain after vault update tonight", "expected_skill": "obsidian-gbrain-safe-index"}
{"intent": "hey is gbrain synced with my vault right now", "expected_skill": "obsidian-gbrain-safe-index"}
// Negative cases: related but owned by other skills. Assert NO route to this skill.
{"intent": "migrate my notes from Notion to gbrain", "expected_skill": null}
{"intent": "what is on my calendar tomorrow", "expected_skill": null}
-100
View File
@@ -1,100 +0,0 @@
---
name: skill-vault-capture-policy
version: 1.0.0
description: |
Use when the user wants a durable operational learning, new agent skill, or
important environment/setup change to be captured into the Obsidian vault
instead of left only in transient chat memory. Covers the capture rule,
preferred page patterns, and index/log update obligations.
triggers:
- "save this learning to the vault"
- "capture this skill in Obsidian"
- "record this workflow in my notes"
- "put this setup change in the vault"
- "should we add this to the knowledge base"
tools:
- read_file
- search_files
- write_file
- patch
mutating: true
---
# Skill Vault Capture Policy
## Contract
This skill guarantees:
- Durable operational learnings, newly adopted skills, and important environment/setup changes are captured into the Obsidian vault rather than left only in chat memory.
- An existing page is updated when the knowledge clearly belongs there; a new page is created only when the topic is distinct and likely to recur.
- Every new vault page is added to `index.md`.
- Every meaningful create/update appends a dated entry to `log.md`.
- Small linked pages are preferred over one giant running note.
## Phases
1. **Classify the learning.**
- New gbrain operating rule, cost control, or embedding/provider change.
- New agent-fork / harness / coding-tool integration fact.
- New Obsidian vault workflow or structure decision.
- New recurring agent skill that changes how the agent should operate here.
2. **Avoid duplicates.**
- Search the vault for an existing page that already owns the topic.
- If found, update it with a new section or dated note rather than creating a near-duplicate.
3. **Create when distinct.**
- Place new pages under the vault schema: `_meta/` for operating notes, `concepts/` for workflows, `entities/` for tools/people.
- Use YAML frontmatter and at least two `[[wikilinks]]` unless it is a short seed page.
4. **Update navigation.**
- Add the page to `index.md` under the correct type heading.
- Append a `## [YYYY-MM-DD] create|update | subject` entry to `log.md`.
5. **Report the capture.**
- State which files changed and whether `index.md` / `log.md` were updated.
## Output Format
Use a short status block:
| Item | Status |
|---|---|
| Learning classified | type |
| Page created/updated | path |
| index.md updated | yes/no |
| log.md updated | yes/no |
## Anti-Patterns
- Leaving durable learnings only in chat memory.
- Creating a near-duplicate page instead of updating the existing one.
- Forgetting to update `index.md` and `log.md`.
- Writing one giant running note instead of small linked pages.
- Recording secrets, API keys, or raw credentials in the vault.
## Tools Used
- `read_file` — read `SCHEMA.md`, `index.md`, `log.md`, and target pages.
- `search_files` — find existing pages to avoid duplicates.
- `write_file` / `patch` — create or update vault pages and navigation.
## Safe Commands
```bash
# inspect vault navigation before writing
read SCHEMA.md index.md log.md
# create or update a page, then refresh catalog/log
# index.md: add [[page-slug]] under the matching type heading
# log.md: append ## [YYYY-MM-DD] create|update | subject
```
## Verification Checklist
- [ ] Vault schema/read files were checked before writing.
- [ ] Existing pages were searched to avoid duplicates.
- [ ] New/updated page has frontmatter and wikilinks.
- [ ] `index.md` updated for new pages.
- [ ] `log.md` appended for meaningful actions.
- [ ] No secrets or raw credentials were written.
@@ -1,10 +0,0 @@
// Routing eval fixtures for skills/skill-vault-capture-policy.
// Positive cases: intents embed a trigger phrase in natural surrounding context
// (never verbatim-identical to a trigger — the fixture linter rejects tautologies).
{"intent": "please save this learning to the vault so we keep it", "expected_skill": "skill-vault-capture-policy", "ambiguous_with": ["idea-ingest"]}
{"intent": "we should capture this skill in Obsidian for reuse", "expected_skill": "skill-vault-capture-policy", "ambiguous_with": ["capture"]}
{"intent": "can you record this workflow in my notes for next time", "expected_skill": "skill-vault-capture-policy"}
{"intent": "put this setup change in the vault before we forget", "expected_skill": "skill-vault-capture-policy"}
// Negative cases: related but owned by other skills or out of scope.
{"intent": "connect my Obsidian vault to gbrain", "expected_skill": null}
{"intent": "what is on my calendar tomorrow", "expected_skill": null}
+43
View File
@@ -466,6 +466,11 @@ async function main() {
const result = JSON.parse(JSON.stringify(rawResult, bigintToStringReplacer));
const output = formatResult(op.name, result);
if (output) process.stdout.write(output);
// #1484 — invisible-miss hint: a bare query/search that hit zero results
// on a multi-source brain tells the user (stderr) which source it
// actually searched and how to widen the scope.
const hint = await sourceScopeHint(op.name, params, ctx.sourceId, engine, result);
if (hint) console.error(hint);
} catch (e: unknown) {
// v0.42.20.0 (codex D4): on error, set exitCode + return so the `finally`
// STILL runs (drains every background-work sink + disconnects). A bare
@@ -837,6 +842,44 @@ async function makeContext(engine: BrainEngine, params: Record<string, unknown>)
};
}
/**
* #1484 — a bare `gbrain query`/`search` silently scopes to the resolved
* source (usually 'default'); on a multi-source brain a zero-hit run looks
* identical to "the brain doesn't know this" even when the answer lives in
* another source. Returns a stderr hint when (a) the op is query/search,
* (b) it returned zero results, (c) the caller did NOT scope explicitly
* (--source / --source-id / --all-sources), and (d) the brain has >1
* registered source. Best-effort: any lookup failure returns null.
*
* Exported for tests (same import-safety contract as formatResult).
*/
export async function sourceScopeHint(
opName: string,
params: Record<string, unknown>,
sourceId: string,
engine: BrainEngine,
result: unknown,
): Promise<string | null> {
if (opName !== 'query' && opName !== 'search') return null;
if (!Array.isArray(result) || result.length > 0) return null;
// Explicit scoping (flag tier) = user intent; don't second-guess it.
if (params.source || params.source_id || params.all_sources) return null;
if (sourceId === '__all__') return null;
try {
const rows = await engine.executeRaw<{ n: number }>(
`SELECT count(*)::int AS n FROM sources`,
);
const n = Number(rows[0]?.n ?? 0);
if (n <= 1) return null;
return (
`Hint: this brain has ${n} sources; you searched only "${sourceId}". ` +
`Retry with --source-id __all__ (all sources) or --source-id <id>.`
);
} catch {
return null; // hint is best-effort; never fail the query over it
}
}
// Exported for tests (same import-safety contract as cliAliases/printOpHelp).
export function formatResult(opName: string, result: unknown): string {
switch (opName) {
+49 -20
View File
@@ -1000,9 +1000,24 @@ const voyageCompatFetch = (async (input: RequestInfo | URL, init?: RequestInit)
// Voyage diverges from OpenAI in two places that break the parser:
// - `embedding` is a base64 string (SDK schema expects `number[]`)
// - `usage` lacks `prompt_tokens` (SDK schema requires it when usage present)
//
// #1610: read the body ONCE via text() and JSON.parse it. The pre-fix
// `await resp.clone().json()` truncated large bodies on bun < 1.1.27
// (oven-sh/bun#6348) — the parse threw, the catch fell back to the raw
// response, and multi-chunk pages died with "Invalid JSON response".
// Every JSON return path below rebuilds the Response so a stale
// Content-Length/Content-Encoding header from the original can't lie
// about the rewritten body.
const bodyText = await resp.text();
const rebuild = (body: string) => {
const headers = new Headers(resp.headers);
headers.delete('content-length');
headers.delete('content-encoding');
return new Response(body, { status: resp.status, statusText: resp.statusText, headers });
};
try {
const json: any = await resp.clone().json();
if (!json || typeof json !== 'object') return resp;
const json: any = JSON.parse(bodyText);
if (!json || typeof json !== 'object') return rebuild(bodyText);
let modified = false;
if (Array.isArray(json.data)) {
for (const item of json.data) {
@@ -1037,22 +1052,19 @@ const voyageCompatFetch = (async (input: RequestInfo | URL, init?: RequestInit)
: 0;
modified = true;
}
if (!modified) return resp;
return new Response(JSON.stringify(json), {
status: resp.status,
statusText: resp.statusText,
headers: resp.headers,
});
if (!modified) return rebuild(bodyText);
return rebuild(JSON.stringify(json));
} catch (err) {
// OOM-cap throws MUST propagate. The catch is here for "Voyage returned
// JSON I can't reshape" (parse error, unexpected schema) — falling back
// to the original response is correct in that case. Letting the
// to the original body is correct in that case. Letting the
// too-large response through here would defeat the entire purpose of
// Layer 2 (the per-embedding cap that fires when Content-Length wasn't
// available to Layer 1).
if (err instanceof VoyageResponseTooLargeError) throw err;
// If parsing/transformation fails, fall back to the original response.
return resp;
// If parsing/transformation fails, pass the original body through
// (rebuilt — resp's body stream is already consumed by text()).
return rebuild(bodyText);
}
}) as unknown as typeof fetch;
@@ -1192,9 +1204,21 @@ const zeroEntropyCompatFetch = (async (input: RequestInfo | URL, init?: RequestI
// validates. Also map usage.total_tokens → prompt_tokens (SDK requires
// prompt_tokens when `usage` is present — same divergence Voyage hit at
// gateway.ts:655).
//
// #1610: read the body ONCE via text() + JSON.parse — `resp.clone().json()`
// truncated large bodies on bun < 1.1.27 (oven-sh/bun#6348), so the parse
// threw and the catch fell back to the RAW ZE `{results: ...}` shape, which
// the AI SDK schema rejects → "Invalid JSON response" on multi-chunk pages.
const bodyText = await resp.text();
const rebuild = (body: string) => {
const headers = new Headers(resp.headers);
headers.delete('content-length');
headers.delete('content-encoding');
return new Response(body, { status: resp.status, statusText: resp.statusText, headers });
};
try {
const json: any = await resp.clone().json();
if (!json || typeof json !== 'object') return resp;
const json: any = JSON.parse(bodyText);
if (!json || typeof json !== 'object') return rebuild(bodyText);
let modified = false;
if (Array.isArray(json.results) && !Array.isArray(json.data)) {
// Layer 2 OOM cap — per-embedding size. ZE returns float[] arrays,
@@ -1228,20 +1252,25 @@ const zeroEntropyCompatFetch = (async (input: RequestInfo | URL, init?: RequestI
// SDK also expects total_tokens; ZE provides it directly.
modified = true;
}
if (!modified) return resp;
return new Response(JSON.stringify(json), {
status: resp.status,
statusText: resp.statusText,
headers: resp.headers,
});
if (!modified) return rebuild(bodyText);
return rebuild(JSON.stringify(json));
} catch (err) {
// OOM-cap throws MUST propagate. Voyage's pattern: instanceof check on
// its own tagged class. Same here — only rethrow our own cap class.
if (err instanceof ZeroEntropyResponseTooLargeError) throw err;
return resp;
return rebuild(bodyText);
}
}) as unknown as typeof fetch;
/**
* Test-only seams (#1610): the compat shims are module-private closures;
* exporting them lets tests drive the response-rewrite paths behaviorally
* (truncating clone(), stale Content-Length) without a live provider.
* Same pattern as __getShrinkStateForTests.
*/
export const __voyageCompatFetchForTests = voyageCompatFetch;
export const __zeroEntropyCompatFetchForTests = zeroEntropyCompatFetch;
/**
* Generic asymmetric-embedding shim for openai-compatible recipes that
* ship no compat fetch of their own (llama-server, litellm, ollama, ...).
+13 -2
View File
@@ -499,7 +499,7 @@ export function linkReadScopeOpts(ctx: OperationContext): { sourceId?: string; s
* FAIL-CLOSED: anything not strictly `ctx.remote === false` is untrusted.
*
* This is the SINGLE resolver for every read op that accepts a per-call
* `source_id` / `all_sources` parameter (query, code_callers, code_callees,
* `source_id` / `all_sources` parameter (query, search, code_callers, code_callees,
* get_page, search_by_image, code_blast, code_flow). Inlining the `__all__`
* branch per handler is the bug class that leaked cross-source reads (#1924,
* #1371): a remote client could pass `source_id: '__all__'` to opt out of its
@@ -1442,13 +1442,24 @@ const search: Operation = {
limit: { type: 'number', description: 'Max results (default 20)' },
offset: { type: 'number', description: 'Skip first N results (for pagination)' },
mode: { type: 'string', description: 'Search mode (conservative|balanced|tokenmax). Local callers only.' },
source_id: {
type: 'string',
description:
"Scope search to a single source. Defaults to OperationContext.sourceId. Pass '__all__' to span every source for trusted local callers; for remote callers '__all__' spans only your granted sources.",
},
all_sources: { type: 'boolean', description: "Span sources (equivalent to source_id=__all__): every source locally, your grant remotely." },
},
handler: async (ctx, p) => {
const startedAt = Date.now();
const queryText = p.query as string;
const limit = (p.limit as number) || 20;
const offset = (p.offset as number) || 0;
const scope = sourceScopeOpts(ctx);
// #1484 follow-up: route through the canonical fail-closed resolver so
// `--source-id __all__` / `all_sources` behave the same as on `query`
// (the zero-hit CLI hint advises exactly that retry). Without a per-call
// param, `search` silently ignored --source-id — the retry looked like
// a genuine miss.
const scope = resolveRequestedScope(ctx, p.source_id as string | undefined, p.all_sources === true);
// T4/D5 — per-call mode honored ONLY for trusted/local callers so a remote
// OAuth client can't escalate to the costly tokenmax bundle. Local + unknown
+12 -2
View File
@@ -1323,8 +1323,18 @@ export async function hybridSearch(
if (effectiveModality === 'both' && imageVectorList !== null) {
vectorLists = [...vectorLists, imageVectorList];
}
} catch {
// Embedding failure is non-fatal, fall back to keyword-only
} catch (err) {
// Embedding/vector failure is non-fatal fall back to keyword-only
// but say WHY (#1626): this arm only runs when the embedding provider
// probed available, so a throw here is a real failure (embed timeout,
// transient pooler error on the searchVector fan-out). Pre-fix the bare
// catch made a cross-source `--source __all__` run silently collapse to
// keyword-only/"No results" with zero diagnostics.
warnOncePerProcess(
'hybrid-vector-arm-failed',
`[gbrain] vector arm failed (fail-open, keyword-only fallback): ` +
`${err instanceof Error ? err.message : String(err)}`,
);
}
}
@@ -0,0 +1,115 @@
/**
* #1610 Voyage/ZeroEntropy compat shims must read the response body ONCE
* via text() instead of `resp.clone().json()`.
*
* On bun < 1.1.27, Response.clone() truncates large bodies (oven-sh/bun#6348):
* the clone().json() parse threw, the shim's catch fell back to the ORIGINAL
* response whose wire shape (ZE `{results: ...}`, Voyage base64 embeddings)
* the AI SDK's openai-compatible Zod schema rejects and multi-chunk pages
* failed with "Invalid JSON response".
*
* These tests simulate the truncating clone() and assert the shims still
* return the fully rewritten body. They also pin that the rewritten Response
* does NOT carry the original (now stale) Content-Length header, which lied
* about the rewritten body's size (gateway.ts previously copied
* `headers: resp.headers` verbatim).
*/
import { afterEach, describe, expect, test } from 'bun:test';
import {
__voyageCompatFetchForTests,
__zeroEntropyCompatFetchForTests,
} from '../../src/core/ai/gateway.ts';
const origFetch = globalThis.fetch;
afterEach(() => {
globalThis.fetch = origFetch;
});
/** Build a Response whose clone() truncates the body (bun < 1.1.27 behavior). */
function truncatingCloneResponse(body: string): Response {
const headers = {
'content-type': 'application/json',
// Deliberately stale after any rewrite: the original wire body's length.
'content-length': String(Buffer.byteLength(body)),
};
const resp = new Response(body, { status: 200, headers });
(resp as any).clone = () =>
new Response(body.slice(0, 32), { status: 200, headers });
return resp;
}
describe('voyageCompatFetch — single body read (#1610)', () => {
test('rewrites base64 embeddings even when clone() truncates the body', async () => {
const floats = new Float32Array([0.5, 0.25, -1]);
const b64 = Buffer.from(floats.buffer).toString('base64');
const wireBody = JSON.stringify({
object: 'list',
data: [{ object: 'embedding', embedding: b64, index: 0 }],
model: 'voyage-3',
usage: { total_tokens: 7 },
});
globalThis.fetch = (async () => truncatingCloneResponse(wireBody)) as unknown as typeof fetch;
const out = await __voyageCompatFetchForTests('https://api.voyageai.com/v1/embeddings', {
method: 'POST',
body: JSON.stringify({ input: ['hello'], model: 'voyage-3' }),
headers: { 'content-type': 'application/json' },
});
const json: any = await out.json();
expect(Array.from(json.data[0].embedding)).toEqual([0.5, 0.25, -1]);
expect(json.usage.prompt_tokens).toBe(7);
// Stale Content-Length from the wire body must not survive the rewrite.
expect(out.headers.get('content-length')).toBeNull();
expect(out.headers.get('content-encoding')).toBeNull();
});
});
describe('zeroEntropyCompatFetch — single body read (#1610)', () => {
test('rewrites {results} → {data} even when clone() truncates the body', async () => {
const wireBody = JSON.stringify({
results: [{ embedding: [0.1, 0.2] }, { embedding: [0.3, 0.4] }],
usage: { total_bytes: 42, total_tokens: 9 },
});
let fetchedUrl = '';
globalThis.fetch = (async (url: string | URL | Request) => {
fetchedUrl = String(url);
return truncatingCloneResponse(wireBody);
}) as unknown as typeof fetch;
const out = await __zeroEntropyCompatFetchForTests('https://api.zeroentropy.dev/v1/embeddings', {
method: 'POST',
body: JSON.stringify({ input: ['hello'], model: 'zembed-1' }),
headers: { 'content-type': 'application/json' },
});
expect(fetchedUrl.endsWith('/v1/models/embed')).toBe(true);
const json: any = await out.json();
// The AI SDK schema requires {data: [{embedding, index}]} — the raw ZE
// {results} fallback is exactly the pre-fix "Invalid JSON response".
expect(json.results).toBeUndefined();
expect(json.data).toHaveLength(2);
expect(json.data[0]).toEqual({ object: 'embedding', embedding: [0.1, 0.2], index: 0 });
expect(json.data[1].index).toBe(1);
expect(json.usage.prompt_tokens).toBe(9);
expect(out.headers.get('content-length')).toBeNull();
});
test('non-JSON body falls back to the original bytes (rebuilt, still readable)', async () => {
const wireBody = 'plain text, not json';
globalThis.fetch = (async () =>
new Response(wireBody, {
status: 200,
headers: { 'content-type': 'application/json' },
})) as unknown as typeof fetch;
const out = await __zeroEntropyCompatFetchForTests('https://api.zeroentropy.dev/v1/embeddings', {
method: 'POST',
body: JSON.stringify({ input: ['hello'] }),
});
// Body was consumed by the shim's single read; the fallback must
// rebuild a readable Response rather than return the drained original.
expect(await out.text()).toBe(wireBody);
});
});
+6 -4
View File
@@ -98,16 +98,18 @@ describe('zeroEntropyCompatFetch — OOM caps', () => {
expect(src).toMatch(/MAX_ZEROENTROPY_RESPONSE_BYTES\s*=\s*256\s*\*\s*1024\s*\*\s*1024/);
});
test('Layer 1: Content-Length pre-check before resp.clone().json()', async () => {
test('Layer 1: Content-Length pre-check before the body is read', async () => {
const src = await Bun.file(GATEWAY_PATH).text();
// Find the zeroEntropyCompatFetch block bounds, then assert ordering
// within it (mirroring the voyage cap test pattern).
// within it (mirroring the voyage cap test pattern). #1610 moved the
// body read from `resp.clone().json()` to a single `resp.text()` (bun
// < 1.1.27 truncates clone()d bodies, oven-sh/bun#6348).
const zeFetchStart = src.indexOf('const zeroEntropyCompatFetch');
expect(zeFetchStart).toBeGreaterThan(0);
const block = src.slice(zeFetchStart, zeFetchStart + 8000);
const block = src.slice(zeFetchStart, zeFetchStart + 9000);
const preCheckIdx = block.indexOf("resp.headers.get('content-length')");
const jsonParseIdx = block.indexOf('await resp.clone().json()');
const jsonParseIdx = block.indexOf('const bodyText = await resp.text()');
expect(preCheckIdx).toBeGreaterThan(0);
expect(jsonParseIdx).toBeGreaterThan(0);
// The pre-check MUST appear before the JSON parse — Voyage's lesson
+61
View File
@@ -0,0 +1,61 @@
/**
* #1484 invisible-miss hint. A bare `gbrain query` resolves to a single
* source (usually 'default'); on a multi-source brain a zero-hit run gave no
* signal that the answer might live in another source. sourceScopeHint
* returns the stderr hint exactly when: query/search op + zero results +
* no explicit scoping param + >1 registered source.
*/
import { describe, expect, test } from 'bun:test';
import { sourceScopeHint } from '../src/cli.ts';
import type { BrainEngine } from '../src/core/engine.ts';
function fakeEngine(sourceCount: number, fail = false): BrainEngine {
return {
executeRaw: async () => {
if (fail) throw new Error('sources table missing');
return [{ n: sourceCount }];
},
} as unknown as BrainEngine;
}
describe('sourceScopeHint (#1484)', () => {
test('fires on a bare zero-hit query against a multi-source brain', async () => {
const hint = await sourceScopeHint('query', {}, 'default', fakeEngine(3), []);
expect(hint).toContain('3 sources');
expect(hint).toContain('"default"');
expect(hint).toContain('--source-id __all__');
});
test('fires for search too', async () => {
const hint = await sourceScopeHint('search', {}, 'wiki', fakeEngine(2), []);
expect(hint).toContain('"wiki"');
});
test('silent when results were found', async () => {
expect(await sourceScopeHint('query', {}, 'default', fakeEngine(3), [{ slug: 'a' }])).toBeNull();
});
test('silent when the caller scoped explicitly', async () => {
expect(await sourceScopeHint('query', { source_id: 'wiki' }, 'wiki', fakeEngine(3), [])).toBeNull();
expect(await sourceScopeHint('query', { source: 'wiki' }, 'wiki', fakeEngine(3), [])).toBeNull();
expect(await sourceScopeHint('query', { all_sources: true }, '__all__', fakeEngine(3), [])).toBeNull();
});
test('silent when the resolved scope is already __all__', async () => {
expect(await sourceScopeHint('query', {}, '__all__', fakeEngine(3), [])).toBeNull();
});
test('silent on a single-source brain', async () => {
expect(await sourceScopeHint('query', {}, 'default', fakeEngine(1), [])).toBeNull();
});
test('silent for non-search ops and non-array results', async () => {
expect(await sourceScopeHint('get_stats', {}, 'default', fakeEngine(3), [])).toBeNull();
expect(await sourceScopeHint('query', {}, 'default', fakeEngine(3), { rows: [] })).toBeNull();
});
test('best-effort: sources lookup failure returns null, never throws', async () => {
expect(await sourceScopeHint('query', {}, 'default', fakeEngine(3, true), [])).toBeNull();
});
});
@@ -1,61 +0,0 @@
/**
* E2E smoke for skills/obsidian-gbrain-safe-index.
*
* Verifies the from-trigger-to-side-effect path that skillify requires:
* a real user trigger phrase routes to the skill, the resolver/check
* pipeline treats it as reachable, and the skill file exposes the
* gbrain commands the workflow actually runs.
*
* This stays local-only (no paid embedding, no external API): it asserts
* the documented safe/import path is present and parseable, not that it
* mutates a live brain.
*/
import { describe, expect, it } from 'bun:test';
import { existsSync, readFileSync } from 'fs';
import { join } from 'path';
const SKILLS = join(import.meta.dir, '..', '..', 'skills');
const SKILL_MD = join(SKILLS, 'obsidian-gbrain-safe-index', 'SKILL.md');
const RESOLVER = join(SKILLS, 'RESOLVER.md');
const TRIGGER_PHRASES = [
'connect my Obsidian vault to gbrain',
'import my vault to gbrain',
'sync vault and gbrain',
'capture this skill in my vault',
'embed gbrain after vault update',
'is gbrain synced with my vault',
];
describe('obsidian-gbrain-safe-index E2E', () => {
it('resolver maps real trigger phrasings to the skill', () => {
const resolver = readFileSync(RESOLVER, 'utf-8');
expect(resolver).toContain('obsidian-gbrain-safe-index/SKILL.md');
// Each representative phrase shares a token substring with a resolver row.
const rows = resolver
.split('\n')
.filter((l) => l.includes('obsidian-gbrain-safe-index/SKILL.md'))
.join('\n');
for (const phrase of TRIGGER_PHRASES) {
const hit = phrase
.toLowerCase()
.split(/\s+/)
.some((tok) => tok.length > 3 && rows.toLowerCase().includes(tok));
expect(hit, `no resolver token for: ${phrase}`).toBe(true);
}
});
it('skill documents the gbrain import-first safe path', () => {
const body = readFileSync(SKILL_MD, 'utf-8');
expect(body).toContain('gbrain import');
expect(body).toContain('--no-embed');
expect(body).toContain('gbrain config set search.mode conservative');
// Paid gate must be explicit, not a silent default.
expect(body).toContain('gbrain embed --stale');
});
it('skill is reachable from the skill tree', () => {
expect(existsSync(SKILL_MD)).toBe(true);
});
});
@@ -1,52 +0,0 @@
/**
* E2E smoke for skills/skill-vault-capture-policy.
*
* Verifies the from-trigger-to-side-effect path: a real capture request
* routes to the skill, and the skill documents the vault navigation
* obligations (index.md / log.md) that make a capture durable.
*
* Local-only: it asserts documented behavior, not live vault writes.
*/
import { describe, expect, it } from 'bun:test';
import { existsSync, readFileSync } from 'fs';
import { join } from 'path';
const SKILLS = join(import.meta.dir, '..', '..', 'skills');
const SKILL_MD = join(SKILLS, 'skill-vault-capture-policy', 'SKILL.md');
const RESOLVER = join(SKILLS, 'RESOLVER.md');
const TRIGGER_PHRASES = [
'save this learning to the vault',
'capture this skill in Obsidian',
'record this workflow in my notes',
'put this setup change in the knowledge base',
];
describe('skill-vault-capture-policy E2E', () => {
it('resolver maps real capture phrasings to the skill', () => {
const resolver = readFileSync(RESOLVER, 'utf-8');
expect(resolver).toContain('skill-vault-capture-policy/SKILL.md');
const rows = resolver
.split('\n')
.filter((l) => l.includes('skill-vault-capture-policy/SKILL.md'))
.join('\n');
for (const phrase of TRIGGER_PHRASES) {
const hit = phrase
.toLowerCase()
.split(/\s+/)
.some((tok) => tok.length > 3 && rows.toLowerCase().includes(tok));
expect(hit, `no resolver token for: ${phrase}`).toBe(true);
}
});
it('skill documents index.md and log.md update obligations', () => {
const body = readFileSync(SKILL_MD, 'utf-8');
expect(body).toContain('index.md');
expect(body).toContain('log.md');
});
it('skill is reachable from the skill tree', () => {
expect(existsSync(SKILL_MD)).toBe(true);
});
});
@@ -0,0 +1,77 @@
/**
* #1626 hybridSearch's text-vector arm must not fail DARK.
*
* The arm only runs when the embedding provider probed available, so a throw
* inside it (embed timeout, transient pooler error on searchVector) is a real
* failure. Pre-fix, a bare `catch {}` swallowed it and the run silently
* collapsed to keyword-only under `--source __all__` on a strained pooler
* that read as a non-deterministic "No results". The fix logs the swallowed
* reason via warnOncePerProcess while keeping the keyword fallback.
*/
import { afterAll, beforeAll, describe, expect, test } from 'bun:test';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { hybridSearch } from '../src/core/search/hybrid.ts';
import {
__setEmbedTransportForTests,
configureGateway,
resetGateway,
} from '../src/core/ai/gateway.ts';
import { _resetWarnOnceForTests } from '../src/core/utils.ts';
let engine: PGLiteEngine;
const origWarn = console.warn;
beforeAll(async () => {
// Pin the gateway to OpenAI with a stub key (put-page-provenance pattern):
// embed() runs instantiateEmbedding — which requires OPENAI_API_KEY — BEFORE
// the stubbed transport is reached. Without this, a keyless CI environment
// throws the config error instead of the transport's, and the assertion on
// the swallowed reason fails. The key never leaves the process.
configureGateway({
embedding_model: 'openai:text-embedding-3-large',
embedding_dimensions: 1536,
env: { ...process.env, OPENAI_API_KEY: process.env.OPENAI_API_KEY || 'sk-test-stub' },
});
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
await engine.putPage('people/alice-example', {
type: 'person',
title: 'Alice Example',
compiled_truth: 'Alice Example is a test person for the vector-arm warn test.',
});
});
afterAll(async () => {
console.warn = origWarn;
__setEmbedTransportForTests(null);
resetGateway();
await engine.disconnect();
});
describe('hybridSearch vector-arm failure telemetry (#1626)', () => {
test('embed failure logs the swallowed reason and falls back to keyword', async () => {
_resetWarnOnceForTests();
// Installing a transport makes isAvailable('embedding') true (test-seam
// fast path), so the vector arm RUNS — and then throws.
__setEmbedTransportForTests(() => {
throw new Error('pooler exploded mid-fanout');
});
const warnings: string[] = [];
console.warn = (...args: unknown[]) => {
warnings.push(args.map(String).join(' '));
};
try {
const results = await hybridSearch(engine, 'alice');
// Keyword fallback still returns results — fail-open preserved.
expect(results.some((r) => r.slug === 'people/alice-example')).toBe(true);
} finally {
console.warn = origWarn;
__setEmbedTransportForTests(null);
}
const armWarnings = warnings.filter((w) => w.includes('vector arm failed'));
expect(armWarnings).toHaveLength(1);
expect(armWarnings[0]).toContain('pooler exploded mid-fanout');
});
});
-51
View File
@@ -1,51 +0,0 @@
import { describe, expect, it } from 'bun:test';
import { readFileSync, existsSync } from 'fs';
import { join } from 'path';
const SKILL_DIR = join(import.meta.dir, '..', 'skills', 'obsidian-gbrain-safe-index');
const SKILL_MD = join(SKILL_DIR, 'SKILL.md');
const RESOLVER = join(import.meta.dir, '..', 'skills', 'RESOLVER.md');
function parseFrontmatter(raw: string): Record<string, unknown> {
const m = raw.match(/^---\n([\s\S]*?)\n---/);
if (!m) throw new Error('no frontmatter');
const out: Record<string, unknown> = {};
for (const line of m[1].split('\n')) {
const mm = line.match(/^([a-zA-Z_]+):\s*(.*)$/);
if (mm) out[mm[1]] = mm[2].trim();
}
return out;
}
describe('obsidian-gbrain-safe-index skill', () => {
it('has a SKILL.md with required frontmatter', () => {
expect(existsSync(SKILL_MD)).toBe(true);
const fm = parseFrontmatter(readFileSync(SKILL_MD, 'utf-8'));
expect(fm['name']).toBe('obsidian-gbrain-safe-index');
expect(fm['description']).toBeTruthy();
});
it('has the required conformance sections', () => {
const body = readFileSync(SKILL_MD, 'utf-8');
for (const section of ['## Contract', '## Phases', '## Output Format', '## Anti-Patterns']) {
expect(body.includes(section), `missing ${section}`).toBe(true);
}
});
it('is registered in RESOLVER.md', () => {
expect(existsSync(RESOLVER)).toBe(true);
const resolver = readFileSync(RESOLVER, 'utf-8');
expect(resolver.includes('obsidian-gbrain-safe-index/SKILL.md')).toBe(true);
});
it('has routing-eval fixtures that exercise real trigger phrasings', () => {
const evalPath = join(SKILL_DIR, 'routing-eval.jsonl');
expect(existsSync(evalPath)).toBe(true);
const lines = readFileSync(evalPath, 'utf-8')
.split('\n')
.filter((l) => l.trim() && !l.trim().startsWith('//'))
.map((l) => JSON.parse(l));
const positives = lines.filter((l) => l.expected_skill === 'obsidian-gbrain-safe-index');
expect(positives.length).toBeGreaterThanOrEqual(5);
});
});
+87
View File
@@ -0,0 +1,87 @@
/**
* #1484 follow-up the `search` op must honor per-call `source_id` /
* `all_sources` through the canonical fail-closed resolver
* (resolveRequestedScope), exactly like `query` does.
*
* Pre-fix, `search` had no source_id param at all: the zero-hit CLI hint
* advised "retry with --source-id __all__", the flag parsed into params,
* NOTHING consumed it, and the retry silently re-ran the same single-source
* search an invisible false negative (and the retry's params.source_id
* suppressed the hint, so the user got no second warning).
*/
import { describe, expect, test } from 'bun:test';
import { operationsByName } from '../src/core/operations.ts';
import type { OperationContext } from '../src/core/operations.ts';
import type { BrainEngine } from '../src/core/engine.ts';
const searchOp = operationsByName['search'];
/** Fake engine: keyword-only config so the handler's scope goes straight to
* searchKeyword, where we capture the opts it was called with. */
function makeCtx(remote: boolean, allowedSources?: string[]) {
const captured: { opts?: Record<string, unknown> } = {};
const engine = {
getConfig: async (key: string) => (key === 'search.mcp_keyword_only' ? 'true' : null),
searchKeyword: async (_q: string, opts: Record<string, unknown>) => {
captured.opts = opts;
return [];
},
} as unknown as BrainEngine;
const ctx = {
engine,
config: { engine: 'pglite' },
logger: { info: () => {}, warn: () => {}, error: () => {} },
dryRun: false,
remote,
sourceId: 'default',
...(allowedSources ? { auth: { allowedSources } } : {}),
} as unknown as OperationContext;
return { ctx, captured };
}
describe('search op per-call source scope (#1484 follow-up)', () => {
test('op declares source_id + all_sources params (the CLI hint advises them)', () => {
expect(searchOp.params.source_id).toBeDefined();
expect(searchOp.params.all_sources).toBeDefined();
});
test('default: scopes to ctx.sourceId', async () => {
const { ctx, captured } = makeCtx(false);
await searchOp.handler(ctx, { query: 'x' });
expect(captured.opts?.sourceId).toBe('default');
});
test("local + source_id '__all__' spans the whole brain (no source filter)", async () => {
const { ctx, captured } = makeCtx(false);
await searchOp.handler(ctx, { query: 'x', source_id: '__all__' });
expect(captured.opts?.sourceId).toBeUndefined();
expect(captured.opts?.sourceIds).toBeUndefined();
});
test('local + all_sources=true spans the whole brain', async () => {
const { ctx, captured } = makeCtx(false);
await searchOp.handler(ctx, { query: 'x', all_sources: true });
expect(captured.opts?.sourceId).toBeUndefined();
expect(captured.opts?.sourceIds).toBeUndefined();
});
test('explicit source_id wins over ctx.sourceId', async () => {
const { ctx, captured } = makeCtx(false);
await searchOp.handler(ctx, { query: 'x', source_id: 'wiki' });
expect(captured.opts?.sourceId).toBe('wiki');
});
test("remote + '__all__' collapses to the caller's grant (fail-closed)", async () => {
const { ctx, captured } = makeCtx(true, ['wiki', 'essays']);
await searchOp.handler(ctx, { query: 'x', source_id: '__all__' });
expect(captured.opts?.sourceIds).toEqual(['wiki', 'essays']);
});
test('remote + out-of-grant source_id is denied', async () => {
const { ctx } = makeCtx(true, ['wiki']);
await expect(searchOp.handler(ctx, { query: 'x', source_id: 'secrets' })).rejects.toThrow(
/outside your granted sources/,
);
});
});
-64
View File
@@ -1,64 +0,0 @@
import { describe, expect, it } from 'bun:test';
import { readFileSync, existsSync } from 'fs';
import { join } from 'path';
import { DEFAULT_PRIVATE_PATTERNS } from '../src/core/skillpack/harvest-lint.ts';
const SKILL_DIR = join(import.meta.dir, '..', 'skills', 'skill-vault-capture-policy');
const SKILL_MD = join(SKILL_DIR, 'SKILL.md');
const RESOLVER = join(import.meta.dir, '..', 'skills', 'RESOLVER.md');
function parseFrontmatter(raw: string): Record<string, unknown> {
const m = raw.match(/^---\n([\s\S]*?)\n---/);
if (!m) throw new Error('no frontmatter');
const out: Record<string, unknown> = {};
for (const line of m[1].split('\n')) {
const mm = line.match(/^([a-zA-Z_]+):\s*(.*)$/);
if (mm) out[mm[1]] = mm[2].trim();
}
return out;
}
describe('skill-vault-capture-policy skill', () => {
it('has a SKILL.md with required frontmatter', () => {
expect(existsSync(SKILL_MD)).toBe(true);
const fm = parseFrontmatter(readFileSync(SKILL_MD, 'utf-8'));
expect(fm['name']).toBe('skill-vault-capture-policy');
expect(fm['description']).toBeTruthy();
});
it('has the required conformance sections', () => {
const body = readFileSync(SKILL_MD, 'utf-8');
for (const section of ['## Contract', '## Phases', '## Output Format', '## Anti-Patterns']) {
expect(body.includes(section), `missing ${section}`).toBe(true);
}
});
it('is registered in RESOLVER.md', () => {
expect(existsSync(RESOLVER)).toBe(true);
expect(readFileSync(RESOLVER, 'utf-8').includes('skill-vault-capture-policy/SKILL.md')).toBe(true);
});
it('contains no private user or agent-fork names (privacy rule)', () => {
for (const file of [SKILL_MD, join(SKILL_DIR, 'routing-eval.jsonl')]) {
const body = readFileSync(file, 'utf-8');
// DEFAULT_PRIVATE_PATTERNS[0] is the banned fork-name pattern; sourced
// from harvest-lint so this file never contains the literal itself
// (scripts/check-privacy.sh would reject it).
const forkName = new RegExp(DEFAULT_PRIVATE_PATTERNS[0], 'i');
for (const name of [/\bAdam\b/, /\bHermes\b/, /\bHerdr\b/, /\bArk\b/, forkName]) {
expect(name.test(body), `private name ${name} in ${file}`).toBe(false);
}
}
});
it('has routing-eval fixtures', () => {
const evalPath = join(SKILL_DIR, 'routing-eval.jsonl');
expect(existsSync(evalPath)).toBe(true);
const positives = readFileSync(evalPath, 'utf-8')
.split('\n')
.filter((l) => l.trim() && !l.trim().startsWith('//'))
.map((l) => JSON.parse(l))
.filter((l) => l.expected_skill === 'skill-vault-capture-policy');
expect(positives.length).toBeGreaterThanOrEqual(4);
});
});
+5 -3
View File
@@ -34,7 +34,7 @@ describe('v0.31.8 — voyage Content-Length pre-check + per-item cap', () => {
expect(source).toMatch(/MAX_VOYAGE_RESPONSE_BYTES\s*=\s*256\s*\*\s*1024\s*\*\s*1024/);
});
test('Layer 1: Content-Length pre-check fires BEFORE resp.clone().json() (D10 OOM defense)', async () => {
test('Layer 1: Content-Length pre-check fires BEFORE the body is read (D10 OOM defense)', async () => {
const source = await Bun.file(new URL('../src/core/ai/gateway.ts', import.meta.url)).text();
// Anchor relative to the post-fetch handler block. The function declaration
// contains an OUTBOUND request body section earlier; we want to verify
@@ -47,8 +47,10 @@ describe('v0.31.8 — voyage Content-Length pre-check + per-item cap', () => {
// doesn't pin to comment text.
const preCheckIdx = inboundBlock.indexOf("resp.headers.get('content-length')");
// Use the full lvalue assignment so the match doesn't accidentally hit
// comment text that mentions `await resp.clone().json()` for context.
const jsonParseIdx = inboundBlock.indexOf('const json: any = await resp.clone().json()');
// comment text that mentions the body read for context. (#1610 moved the
// read from `resp.clone().json()` to a single `resp.text()` — bun <
// 1.1.27 truncates clone()d bodies, oven-sh/bun#6348.)
const jsonParseIdx = inboundBlock.indexOf('const bodyText = await resp.text()');
expect(preCheckIdx).toBeGreaterThan(0);
expect(jsonParseIdx).toBeGreaterThan(0);
// The pre-check MUST appear before the JSON parse — otherwise the OOM