Compare commits

...
Author SHA1 Message Date
bb2a3d2359 fix(models): doctor chat probe token budget — stop falsely failing reasoning models (takeover of #3250, #3221)
The models-doctor reachability probe used maxTokens: 1, which falsely
fails reasoning models (they burn output budget on internal reasoning
before emitting text). PROBE_MAX_OUTPUT_TOKENS=64, widened above any
configured extended-thinking budget (getConfiguredThinkingBudget
mirrors chat()'s recipe-alias resolution; read-only, never throws).
A length-exhausted empty-text response counts as reachable with the
limitation surfaced in the probe message.

Repair over #3250: composes with the gateway's contentless-length
guard (#3217/#3249) — when chat() itself throws the
'output budget exhausted' AIConfigError before the probe sees a
ChatResult, probeModel's catch now recognizes it and reports
reachable-with-caveat (the throw proves transport + auth + model
routing); every other AIConfigError still classifies as a failure.

Co-authored-by: Masashi-Ono0611 <Masashi-Ono0611@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 11:00:40 -07:00
f129652ab5 fix(migrate): tolerate non-superuser roles lacking BYPASSRLS (takeover of #3212)
The RLS-backfill migrations v24/v29/v31/v35 RAISE EXCEPTION when the
role lacks BYPASSRLS, permanently wedging the upgrade chain on managed
Postgres where the app role owns the tables but can never hold
BYPASSRLS (fresh installs were unaffected: schema.sql warns + skips).
Now RAISE WARNING and proceed. Still fail-closed against the original
'silent WARNING + version bump without RLS' hazard: ENABLE ROW LEVEL
SECURITY requires table ownership, so a non-owner non-bypass role
errors on the ALTER itself, aborting the DO block with schema_version
unbumped.

Repair over #3212: (1) the PR's SQL comment contained backticks inside
the TS template literal, terminating it -- the PR did not typecheck;
(2) the three pinned guards in test/migrate.test.ts asserting the old
RAISE EXCEPTION contract now pin the new one: warn-and-PROCEED (ALTERs
outside the guard branch), never warn-and-skip.

Co-authored-by: alexey-metaengage <alexey-metaengage@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 11:00:29 -07:00
9fca071458 fix(search): fold resolved source-boost map into query-cache knobs hash (takeover of #3213)
Boosts reorder results at query-build time only, so a ranking-policy
change (GBRAIN_SOURCE_BOOST tune or new defaults) kept serving rows
ranked under the old policy for up to cache.ttl_seconds. Folds
resolveBoostMap() into the cache key (KNOBS_HASH_VERSION 12 -> 13,
canonicalized sb= part, same pattern as #2825's hx=).

Repair over #3213: the PR left two KNOBS_HASH_VERSION pins at 12
(test/cross-modal-phase1.test.ts, test/search-alias-resolved-boost.test.ts),
which would have failed CI. Both updated to 13 with the bump rationale.

Co-authored-by: SVZTeam <SVZTeam@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 11:00:19 -07:00
14 changed files with 463 additions and 50 deletions
+1 -1
View File
@@ -107,7 +107,7 @@ Useful for: team mounts, brain-as-a-service deployments, dev machines without di
```bash
gbrain doctor --json # full health check
gbrain models # which AI models are configured for what
gbrain models doctor # 1-token probe per configured model
gbrain models doctor # minimal reachability probe per configured model
```
If anything's yellow, `gbrain doctor` names the fix command in the message. Most issues are missing API keys or stale schema (`gbrain upgrade --force-schema`).
+1 -1
View File
@@ -136,7 +136,7 @@ per-release `**vX.Y.Z:**` narration — CI enforces this
- `src/core/ai/model-resolver.ts` — Recipe-touchpoint validator. `assertTouchpoint(recipe, touchpoint, modelId, extendedModels?)` takes an optional 4th `extendedModels: ReadonlySet<string>`: when the modelId is in that set the native-recipe allowlist throw is bypassed (user explicitly opted in via config, so provider rejection surfaces as `model_not_found` at HTTP call time and `gbrain models doctor` catches it earlier). Default code paths with hardcoded model strings MUST NOT pass `extendedModels` — source typos still fail fast (the fail-fast contract for chat + expand + embed stays intact).
- `src/core/ai/gateway.ts` extension — module-scoped `_extendedModels: Map<providerId, Set<modelId>>` registry feeds `assertTouchpoint`'s 4th-arg path. `reconfigureGatewayWithEngine(engine)` (async, called from `cli.ts` after `engine.connect()`, before every command except `CLI_ONLY` no-DB commands) re-resolves expansion + chat defaults through `resolveModel()` so `models.tier.*` and `models.default` overrides apply to both. `DEFAULT_CHAT_MODEL` is `anthropic:claude-sonnet-4-6`. `__setChatTransportForTests` seam mirrors `__setEmbedTransportForTests` so tests drive `chat()` with a stubbed transport.
- `src/core/minions/queue.ts` extension — `MinionQueue.add()` rejects `subagent` jobs whose `data.model` resolves via `isAnthropicProvider()` to a non-Anthropic provider. Lazy-imports `model-config.ts` to avoid pulling engine types into queue's eager-load surface. Layer 1 of the three-layer subagent provider enforcement (layers 2+3: `model-config.ts:enforceSubagentAnthropic` runtime fallback + `src/commands/doctor.ts` `subagent_provider` check). Pinned by `test/agent-cli.test.ts`.
- `src/commands/models.ts``gbrain models [--json]` read-only routing dashboard: prints tier defaults (`utility`/`reasoning`/`deep`/`subagent`), the resolved value for each (re-walking the resolution chain), every per-task override (11 `PER_TASK_KEYS`: `models.dream.synthesize`, `models.dream.patterns`, `models.drift`, `models.auto_think`, `models.think`, `models.subagent`, `facts.extraction_model`, `models.eval.longmemeval`, `models.expansion`, `models.chat`, `models.dream.synthesize_verdict`), the alias map, and a source-of-truth column (`default` / `config: <key>` / `env: <VAR>`). `gbrain models doctor [--skip=<provider>] [--json]` fires a 1-token `gateway.chat()` probe against each configured chat + expansion model and classifies failures into `{model_not_found, auth, rate_limit, network, unknown}`. Wired into `cli.ts` dispatch + `CLI_ONLY` set. A zero-token `embedding_config` probe runs FIRST, before any chat/expansion probes spend money: `probeEmbeddingConfig()` reads `getEmbeddingModel()` + `getEmbeddingDimensions()` and (for Voyage flexible-dim models) checks `isValidVoyageOutputDim(dims)` against `VOYAGE_VALID_OUTPUT_DIMS`. `ProbeStatus` variant `'config'` + optional `fix?: string` on `ProbeResult` surface a paste-ready `gbrain config set ...` line in human + JSON output; touchpoint label `'embedding_config'` joins `'chat'` and `'expansion'`.
- `src/commands/models.ts``gbrain models [--json]` read-only routing dashboard: prints tier defaults (`utility`/`reasoning`/`deep`/`subagent`), the resolved value for each (re-walking the resolution chain), every per-task override (11 `PER_TASK_KEYS`: `models.dream.synthesize`, `models.dream.patterns`, `models.drift`, `models.auto_think`, `models.think`, `models.subagent`, `facts.extraction_model`, `models.eval.longmemeval`, `models.expansion`, `models.chat`, `models.dream.synthesize_verdict`), the alias map, and a source-of-truth column (`default` / `config: <key>` / `env: <VAR>`). `gbrain models doctor [--skip=<provider>] [--json]` fires a small bounded `gateway.chat()` probe (`PROBE_MAX_OUTPUT_TOKENS`, enough headroom that reasoning models which burn output budget on internal reasoning don't falsely fail; a length-exhausted empty-text response counts as reachable with the limitation surfaced in the probe message) against each configured chat + expansion model and classifies failures into `{model_not_found, auth, rate_limit, network, unknown}`. Wired into `cli.ts` dispatch + `CLI_ONLY` set. A zero-token `embedding_config` probe runs FIRST, before any chat/expansion probes spend money: `probeEmbeddingConfig()` reads `getEmbeddingModel()` + `getEmbeddingDimensions()` and (for Voyage flexible-dim models) checks `isValidVoyageOutputDim(dims)` against `VOYAGE_VALID_OUTPUT_DIMS`. `ProbeStatus` variant `'config'` + optional `fix?: string` on `ProbeResult` surface a paste-ready `gbrain config set ...` line in human + JSON output; touchpoint label `'embedding_config'` joins `'chat'` and `'expansion'`.
- `src/core/init-embed-check.ts` — embedding-key validation at `gbrain init`. `runInitEmbedCheck(opts)` runs a config-only `diagnoseEmbedding` (catches a missing key for ANY provider) plus a best-effort `liveTestEmbed` (1-token `gateway.embed(['probe'], {inputType:'query', abortSignal})`, 5s `AbortController` timeout, never throws — catches an invalid/expired key). Loud warning to stderr; init still exits 0 (`--no-embedding` is the deferred-setup escape; `--skip-embed-check` / `GBRAIN_INIT_SKIP_EMBED_CHECK=1` skip the check). Builds the effective env (`process.env` + file-plane `openai/anthropic/zeroentropy_api_key` from `loadConfigFileOnly()` + `opts.apiKey`) and configures the gateway via `buildGatewayConfig` before diagnose/probe, so the check sees the same keys AND provider base URLs runtime will (no false "missing key" for config.json-keyed users; the probe hits the right endpoint). Init-specific warning text names `--no-embedding` / `--skip-embed-check`, not the sync-flavored `--no-embed`. Wired into `initPGLite` + `initPostgres` in `src/commands/init.ts`, with the result added to the `--json` envelope as `embedding_check {ok, reason?, live_ok?}`. Pinned by `test/init-embed-check.test.ts` (hermetic via the gateway embed-transport seam + `withEnv`).
- `src/core/ai/build-gateway-config.ts``buildGatewayConfig(c: GBrainConfig): AIGatewayConfig`, extracted from `src/cli.ts` (which re-exports it for back-compat). Lets core modules (`init-embed-check.ts`) reuse it without importing the CLI entrypoint. Single owner of folding file-plane API keys (openai/anthropic/zeroentropy) into the gateway env and threading local-server `*_BASE_URL` env vars into base_urls. `process.env` wins EXCEPT empty-string / undefined values are dropped before the merge, so an injected empty `ANTHROPIC_API_KEY=''` (Claude Code neuters subprocess LLM calls this way) can't clobber a valid config-plane key; `'0'` / `'false'` are preserved. Pinned by `test/ai/build-gateway-config.test.ts`.
- `src/commands/doctor.ts` extension — `subagent_provider` check (layer 3 of 3). Warns when `models.tier.subagent` is explicitly set non-Anthropic (message names the bad value + paste-ready fix `gbrain config set models.tier.subagent anthropic:claude-sonnet-4-6`); also warns when `models.default` would sneak `subagent` into a non-Anthropic provider via tier inheritance. OK when subagent tier resolves to Anthropic. Tests in `test/doctor.test.ts`.
+1 -1
View File
@@ -47,7 +47,7 @@ Visibility:
```bash
gbrain models # print current routing table
gbrain models doctor # 1-token probe to each configured model
gbrain models doctor # minimal reachability probe to each configured model
```
**Subagent tier exists because the loop is Anthropic-only.** The handler
+71 -11
View File
@@ -9,8 +9,9 @@
* per-task overrides, alias map, and source-of-truth
* column (default / config / env).
*
* `gbrain models doctor` — opt-in probe. Fires a 1-token `gateway.chat()`
* call against each configured chat / expansion
* `gbrain models doctor` — opt-in probe. Fires a small bounded
* `gateway.chat()` call (PROBE_MAX_OUTPUT_TOKENS)
* against each configured chat / expansion
* model and reports reachability with the
* provider's error string. Catches the bug class
* that motivated v0.31.12 (the v0.31.6 chat
@@ -22,9 +23,10 @@
* --skip=<provider> — narrow `doctor` probe to skip a provider
* (e.g. cost-sensitive operators with rate limits)
*
* Per Codex F11 in plan review: no specific dollar cost claim. Probe uses
* `max_tokens: 1` against each configured model; actual cost depends on
* provider billing minimums.
* Per Codex F11 in plan review: no specific dollar cost claim. Probe caps
* output at PROBE_MAX_OUTPUT_TOKENS against each configured model (widened
* above any configured extended-thinking budget); actual cost depends on
* tokens actually generated and provider billing minimums.
*/
import type { BrainEngine } from '../core/engine.ts';
@@ -503,25 +505,78 @@ async function probeEmbeddingReachability(): Promise<ProbeResult | null> {
}
}
async function probeModel(modelStr: string, touchpoint: 'chat' | 'expansion'): Promise<ProbeResult> {
/**
* Output budget for the chat/expansion reachability probe.
*
* This was `maxTokens: 1`, which falsely fails reasoning models: they spend
* output budget on internal reasoning before emitting any text, so a 1-token
* cap is either exhausted with zero usable text (finishReason 'length') or
* rejected outright by providers that require the cap to exceed the model's
* minimum reasoning spend — and the probe then reported a config failure for
* a perfectly reachable model. 64 tokens gives every model class enough
* headroom to prove transport + auth + model routing, while staying a
* minimal-cost probe: providers bill actual tokens generated, not the cap,
* and non-reasoning models answer '.' in a handful of tokens and stop.
*
* @internal exported for tests (test/models-doctor-probe-token-budget.test.ts).
*/
export const PROBE_MAX_OUTPUT_TOKENS = 64;
/** @internal exported for tests (test/models-doctor-probe-token-budget.test.ts). */
export async function probeModel(modelStr: string, touchpoint: 'chat' | 'expansion'): Promise<ProbeResult> {
const start = Date.now();
try {
const { chat } = await import('../core/ai/gateway.ts');
const { chat, getConfiguredThinkingBudget } = await import('../core/ai/gateway.ts');
// Anthropic requires max_tokens > thinking.budgetTokens. When the user
// configured an extended-thinking budget for this model via
// provider_chat_options, a fixed small cap would be rejected outright —
// the same false-FAIL class this probe budget exists to prevent. Widen
// the cap above the configured budget; billing is still actual tokens.
const thinkingBudget = getConfiguredThinkingBudget(modelStr);
const maxTokens = (thinkingBudget ?? 0) + PROBE_MAX_OUTPUT_TOKENS;
// Use AbortController so the 5s timeout doesn't hang on a stuck network.
const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(new Error('probe timed out after 5s')), 5000);
try {
await chat({
const res = await chat({
model: modelStr,
messages: [{ role: 'user', content: '.' }],
maxTokens: 1,
maxTokens,
abortSignal: controller.signal,
});
return { model: modelStr, touchpoint, status: 'ok', message: 'reachable', elapsed_ms: Date.now() - start };
// A length-exhausted response with no text is still proof of
// reachability: the request survived transport + auth + model routing,
// and the model generated tokens — a reasoning model may spend the
// whole probe budget on internal reasoning. Report ok, but surface the
// generation limitation instead of a bare 'reachable'.
const message =
res.stopReason === 'length' && res.text.length === 0
? `reachable (spent the ${maxTokens}-token probe budget on internal reasoning without emitting text; transport verified, text generation not exercised)`
: 'reachable';
return { model: modelStr, touchpoint, status: 'ok', message, elapsed_ms: Date.now() - start };
} finally {
clearTimeout(timeoutId);
}
} catch (err) {
// Composition with the gateway's contentless-length guard (#3217): when
// chat() itself rejects a length-exhausted empty completion with an
// AIConfigError ('output budget exhausted before any content...'), the
// probe never sees a ChatResult — but the throw itself proves transport +
// auth + model routing all worked and the model generated (reasoning)
// tokens. That is exactly the reachable-with-caveat case handled above,
// not a config failure. Every other AIConfigError still classifies as a
// failure below.
const { AIConfigError } = await import('../core/ai/errors.ts');
if (err instanceof AIConfigError && /output budget exhausted/i.test(err.message)) {
return {
model: modelStr,
touchpoint,
status: 'ok',
message:
'reachable (spent the probe output budget on internal reasoning without emitting text; transport verified, text generation not exercised)',
elapsed_ms: Date.now() - start,
};
}
const { status, message } = classifyError(err);
return { model: modelStr, touchpoint, status, message, elapsed_ms: Date.now() - start };
}
@@ -555,7 +610,7 @@ export async function runModels(engine: BrainEngine, args: string[]): Promise<vo
process.stdout.write(
`Usage:
gbrain models Show routing table (read-only)
gbrain models doctor [flags] Probe each configured model (~1 token each)
gbrain models doctor [flags] Probe each configured model (one small bounded request each)
gbrain models --json Machine-readable output
Flags (doctor only):
@@ -648,6 +703,11 @@ Tiers: utility (haiku-class) | reasoning (sonnet) | deep (opus) | subagent (Anth
if (r.status !== 'ok') {
process.stdout.write(` ${r.message}\n`);
if (r.fix) process.stdout.write(` fix: ${r.fix}\n`);
} else if (r.message !== 'reachable') {
// Probe passed with a caveat (e.g. a reasoning model spent the whole
// budget on internal reasoning) — surface it in human output too, not
// only in --json.
process.stdout.write(` ${r.message}\n`);
}
}
process.stdout.write(`\nSummary: ${report.summary.ok}/${report.summary.total} reachable.\n`);
+31
View File
@@ -2891,6 +2891,37 @@ function deepMergeRecords(
return out;
}
/**
* Read the Anthropic-style extended-thinking budget configured for a model
* via `provider_chat_options` (`<provider>.thinking.budgetTokens`, provider-
* or model-scoped). The Anthropic API requires `max_tokens` to exceed
* `thinking.budgetTokens`, so callers that cap output tightly (the
* `gbrain models doctor` reachability probe) must widen their cap above this
* budget or the provider rejects the request outright — falsely failing a
* reachable model. Returns undefined when no budget is configured, the model
* string is malformed, or the gateway is unconfigured. Read-only, never throws.
*
* @internal exported for the doctor probe + tests; not part of the public gateway API.
*/
export function getConfiguredThinkingBudget(modelStr: string): number | undefined {
try {
if (!_config) return undefined;
// Mirror chat()'s resolution (resolveChatProvider → resolveRecipe): map
// recipe aliases to the canonical model id BEFORE the model-scoped
// provider_chat_options lookup, so a request made under a stale alias
// still sees the budget configured under the canonical id.
const { parsed, recipe } = resolveRecipe(modelStr);
const merged: Record<string, any> = {};
applyConfiguredChatProviderOptions(merged, _config, recipe.id, parsed.modelId);
const budget = merged[recipe.id]?.thinking?.budgetTokens;
return typeof budget === 'number' && Number.isFinite(budget) && budget > 0
? budget
: undefined;
} catch {
return undefined;
}
}
function applyConfiguredChatProviderOptions(
providerOptions: Record<string, any>,
cfg: AIGatewayConfig,
+12 -12
View File
@@ -868,13 +868,12 @@ export const MIGRATIONS: Migration[] = [
BEGIN
SELECT EXISTS (SELECT 1 FROM pg_roles pr WHERE pg_has_role(current_user, pr.oid, 'USAGE') AND (pr.rolbypassrls OR pr.rolsuper)) INTO has_bypass; -- #1385: superuser + inherited-role BYPASSRLS, not just the role's own rolbypassrls
IF NOT has_bypass THEN
-- Fail the migration loudly instead of WARNING + version-bump.
-- The runner unconditionally records schema_version on success,
-- so a silent WARNING here would permanently lock the backfill out
-- on future runs even after switching to a bypass role. Raising
-- aborts the transaction, leaves schema_version at the prior value,
-- and lets the next invocation retry after the role is fixed.
RAISE EXCEPTION 'v24 rls_backfill_missing_tables: role % does not have BYPASSRLS privilege — cannot enable RLS safely. Re-run as postgres (or another BYPASSRLS role). The migration will retry automatically on the next initSchema call.', current_user;
-- A non-superuser table OWNER (e.g. a managed-Postgres 'postgres'
-- role) can't hold BYPASSRLS, but it OWNS these tables and gbrain
-- sets no FORCE ROW LEVEL SECURITY and no policies, so the owner is
-- exempt and enabling RLS is a harmless no-op. Aborting would
-- permanently wedge the migration chain on such instances.
RAISE WARNING 'v24 rls_backfill_missing_tables: role % lacks BYPASSRLS — enabling RLS anyway (owner-exempt; safe on a single-owner DB).', current_user;
END IF;
-- These 8 are guaranteed to exist: schema.sql creates them (idempotent
@@ -1157,7 +1156,8 @@ export const MIGRATIONS: Migration[] = [
BEGIN
SELECT EXISTS (SELECT 1 FROM pg_roles pr WHERE pg_has_role(current_user, pr.oid, 'USAGE') AND (pr.rolbypassrls OR pr.rolsuper)) INTO has_bypass; -- #1385: superuser + inherited-role BYPASSRLS, not just the role's own rolbypassrls
IF NOT has_bypass THEN
RAISE EXCEPTION 'v29 cathedral_ii_code_edges_rls: role % does not have BYPASSRLS privilege — cannot enable RLS safely. Re-run as postgres (or another BYPASSRLS role). The migration will retry automatically on the next initSchema call.', current_user;
-- Owner-exempt: warn + proceed rather than abort (see v24).
RAISE WARNING 'v29 cathedral_ii_code_edges_rls: role % lacks BYPASSRLS — enabling RLS anyway (owner-exempt; safe on a single-owner DB).', current_user;
END IF;
ALTER TABLE code_edges_chunk ENABLE ROW LEVEL SECURITY;
@@ -1386,7 +1386,8 @@ export const MIGRATIONS: Migration[] = [
BEGIN
SELECT EXISTS (SELECT 1 FROM pg_roles pr WHERE pg_has_role(current_user, pr.oid, 'USAGE') AND (pr.rolbypassrls OR pr.rolsuper)) INTO has_bypass; -- #1385: superuser + inherited-role BYPASSRLS, not just the role's own rolbypassrls
IF NOT has_bypass THEN
RAISE EXCEPTION 'v31 eval_capture_tables: role % does not have BYPASSRLS privilege — cannot enable RLS safely. Re-run as postgres (or another BYPASSRLS role). The migration will retry automatically on the next initSchema call.', current_user;
-- Owner-exempt: warn + proceed rather than abort (see v24).
RAISE WARNING 'v31 eval_capture_tables: role % lacks BYPASSRLS — enabling RLS anyway (owner-exempt; safe on a single-owner DB).', current_user;
END IF;
CREATE TABLE IF NOT EXISTS eval_candidates (
@@ -1726,9 +1727,8 @@ export const MIGRATIONS: Migration[] = [
BEGIN
SELECT EXISTS (SELECT 1 FROM pg_roles pr WHERE pg_has_role(current_user, pr.oid, 'USAGE') AND (pr.rolbypassrls OR pr.rolsuper)) INTO has_bypass; -- #1385: superuser + inherited-role BYPASSRLS, not just the role's own rolbypassrls
IF NOT has_bypass THEN
-- Same posture as v24: raise to abort the migration so the runner
-- leaves config.version unbumped and retries on the next call.
RAISE EXCEPTION 'v35 auto_rls_event_trigger backfill: role % does not have BYPASSRLS — cannot enable RLS safely. Re-run as postgres (or another BYPASSRLS role).', current_user;
-- Owner-exempt: warn + proceed rather than abort (see v24).
RAISE WARNING 'v35 auto_rls_event_trigger backfill: role % lacks BYPASSRLS — enabling RLS anyway (owner-exempt; safe on a single-owner DB).', current_user;
END IF;
FOR r IN
+6 -1
View File
@@ -15,7 +15,7 @@ import type { SearchResult, SearchOpts, HybridSearchMeta } from '../types.ts';
import { embed, embedQuery } from '../embedding.ts';
import { registerBackgroundWorkDrainer } from '../background-work.ts';
import { resolveEmbeddingColumn, isCacheSafe } from './embedding-column.ts';
import { resolveHardExcludes } from './source-boost.ts';
import { resolveHardExcludes, resolveBoostMap } from './source-boost.ts';
import {
resolveAdaptiveReturn,
applyAdaptiveReturn,
@@ -1702,6 +1702,11 @@ export async function hybridSearchCached(
// resolves) into the cache key so a row written under one exclude
// policy can't be served to a lookup under another.
hardExcludes: resolveHardExcludes(opts?.exclude_slug_prefixes, opts?.include_slug_prefixes),
// v=13 — fold the resolved source-boost map (defaults merged with
// GBRAIN_SOURCE_BOOST) into the cache key. Boosts reorder results at
// query-build time only, so without this a ranking-policy change kept
// serving rows ranked under the old policy for up to cache.ttl_seconds.
sourceBoostMap: resolveBoostMap(),
});
// Cache decision: opts.useCache (explicit) wins over global config; global
+33 -1
View File
@@ -756,7 +756,16 @@ export function attributeKnob<K extends keyof ModeBundle>(
// slugs written by a process without it, and vice versa. Same one-time
// global cold-miss pattern as the bumps above; refills within
// cache.ttl_seconds (3600s default).
export const KNOBS_HASH_VERSION = 12;
//
// bump 12→13: the resolved source-boost map (DEFAULT_SOURCE_BOOSTS merged
// with GBRAIN_SOURCE_BOOST) folds into the key via ctx.sourceBoostMap. Same
// bug class as #2825's hard-excludes: boosts only applied at DB-query build
// time (cache miss), so after a ranking-policy change (env tune or upgrade
// that ships new defaults) lookups kept being served rows ranked under the
// OLD policy for up to cache.ttl_seconds — making boost tuning appear
// nondeterministic. Same one-time global cold-miss pattern; refills within
// cache.ttl_seconds (3600s default).
export const KNOBS_HASH_VERSION = 13;
/**
* v0.36 (D8 / CDX-2) second-arg context for the cache key. The
@@ -795,6 +804,17 @@ export interface KnobsHashContext {
* 'none' for legacy callers that don't thread excludes.
*/
hardExcludes?: string[];
/**
* v=13: the RESOLVED effective source-boost map the same value
* resolveBoostMap() produces at query-build time (DEFAULT_SOURCE_BOOSTS
* merged with GBRAIN_SOURCE_BOOST). Ranking priors change result ORDER,
* not membership, so they are invisible to the hard-exclude fold above
* yet a cache row ranked under an old boost policy is exactly as stale as
* one filtered under an old exclude policy. Canonicalized (entries sorted
* by prefix) so object-key insertion order is irrelevant. Undefined falls
* back to the literal 'default' for legacy callers.
*/
sourceBoostMap?: Record<string, number>;
}
export function knobsHash(
@@ -888,6 +908,18 @@ export function knobsHash(
// across processes. Sorted copy so ['a/','b/'] and ['b/','a/'] hash
// identically; undefined falls back to 'none' for legacy callers.
`hx=${ctx?.hardExcludes ? [...ctx.hardExcludes].sort().join(',') : 'none'}`,
// v=13 addition (append-only): resolved source-boost map. Boost changes
// reorder results, so a row ranked under one policy must not be served
// to a lookup under another. Entries sorted by prefix so map insertion
// order is irrelevant; undefined falls back to 'default'.
`sb=${
ctx?.sourceBoostMap
? Object.entries(ctx.sourceBoostMap)
.sort(([a], [b]) => (a < b ? -1 : a > b ? 1 : 0))
.map(([prefix, factor]) => `${prefix}:${factor}`)
.join(',')
: 'default'
}`,
];
const h = createHash('sha256');
h.update(parts.join('|'));
+4 -2
View File
@@ -136,7 +136,7 @@ describe('D2 — knobsHash differs across cross-modal knob values', () => {
return resolveSearchMode({ mode: 'balanced' });
}
test('KNOBS_HASH_VERSION is 12 (cross-modal still appended; 11→12 hard-exclude fold #2825)', () => {
test('KNOBS_HASH_VERSION is 13 (cross-modal still appended; 12→13 source-boost map fold)', () => {
// v0.35 ladder: 1→2 reranker, 2→3 floor_ratio. v0.36 piggybacks on v=3
// with 7 cross-modal knobs + column/provider context. v0.40.4 (salem) +
// v0.39 T21 (master) bump to v=4 for graph_signals + schema-pack fields.
@@ -146,7 +146,9 @@ describe('D2 — knobsHash differs across cross-modal knob values', () => {
// v0.43: 9→10 relational recall arm. #1400: 10→11 query-side input_type
// finally reaches asymmetric providers — pre-fix rows were keyed on
// document-side query vectors. #2825: 11→12 hard-exclude fold (hx=).
expect(KNOBS_HASH_VERSION).toBe(12);
// 12→13: source-boost map fold (sb=) — a ranking-policy change must not
// be served rows ranked under the previous policy.
expect(KNOBS_HASH_VERSION).toBe(13);
});
test('flipping unified_multimodal changes the hash', () => {
+28 -12
View File
@@ -442,16 +442,26 @@ describe('migration v24 — rls_backfill_missing_tables', () => {
);
});
// Codex found: if v24 RAISE WARNINGs instead of raising on non-BYPASSRLS,
// the migration runner still bumps schema_version to 24, permanently
// skipping the backfill on future runs even after the role is fixed.
// The fix is to raise loudly so the transaction aborts, version stays
// at 23, and the next initSchema call retries after role reassignment.
test('fails loudly on non-BYPASSRLS roles instead of silently bumping version', () => {
// PR #3212: managed-Postgres roles (e.g. RDS/Supabase `postgres`) can never
// hold BYPASSRLS, so RAISE EXCEPTION permanently wedged the upgrade chain
// on those instances (fresh installs were fine — schema.sql WARNs + skips).
// New contract: warn AND PROCEED. This is still fail-closed against the
// original Codex hazard ("silent WARNING + version bump without RLS"):
// ENABLE ROW LEVEL SECURITY requires table ownership, so a non-owner
// non-bypass role errors on the ALTER itself, aborting the DO block and
// leaving schema_version unbumped. The guards below pin warn-and-proceed
// (ALTERs OUTSIDE the IF branch), never warn-and-skip.
test('warns on non-BYPASSRLS roles and still enables RLS (warn-and-proceed, not skip)', () => {
const v24 = MIGRATIONS.find(m => m.version === 24);
const sql = v24!.sql || '';
expect(sql).toMatch(/RAISE EXCEPTION[^;]*BYPASSRLS/);
expect(sql).not.toMatch(/RAISE WARNING[^;]*BYPASSRLS/);
expect(sql).toMatch(/RAISE WARNING[^;]*BYPASSRLS/);
expect(sql).not.toMatch(/RAISE EXCEPTION[^;]*BYPASSRLS/);
// The ALTER statements sit AFTER the guard's END IF — the warning branch
// must not swallow the RLS enablement (warn-and-skip would silently bump
// the version without RLS, the original hazard).
expect(sql).toMatch(
/IF NOT has_bypass THEN[\s\S]*?RAISE WARNING[\s\S]*?END IF;[\s\S]*ALTER TABLE \w+ ENABLE ROW LEVEL SECURITY/,
);
});
test('LATEST_VERSION has caught up to 24', () => {
@@ -542,11 +552,12 @@ describe('migration v35 — auto_rls_event_trigger structural guards', () => {
expect(sql).toMatch(/'\^GBRAIN:RLS_EXEMPT\\s\+reason=\\S\.\{3,\}'/);
});
test('backfill is gated on rolbypassrls (matches v24 posture)', () => {
test('backfill warns on missing rolbypassrls and proceeds (matches v24 posture, PR #3212)', () => {
const v35 = MIGRATIONS.find(m => m.version === 35);
const sql = ((v35?.sqlFor as any)?.postgres ?? '') as string;
expect(sql).toMatch(/rolbypassrls/);
expect(sql).toMatch(/RAISE\s+EXCEPTION/i);
expect(sql).toMatch(/RAISE WARNING[^;]*BYPASSRLS/);
expect(sql).not.toMatch(/RAISE EXCEPTION[^;]*BYPASSRLS/);
});
});
@@ -1319,13 +1330,18 @@ describe('migration v31 — eval_capture_tables', () => {
}
});
test('Postgres variant gates RLS on BYPASSRLS and fails loudly', () => {
test('Postgres variant warns on missing BYPASSRLS and still enables RLS (PR #3212 warn-and-proceed)', () => {
const pgSql = MIGRATIONS.find(m => m.version === 31)!.sqlFor!.postgres!;
expect(pgSql).toContain('rolbypassrls');
expect(pgSql).toMatch(/IF NOT has_bypass/);
expect(pgSql).toMatch(/RAISE EXCEPTION[^;]*BYPASSRLS/);
expect(pgSql).toMatch(/RAISE WARNING[^;]*BYPASSRLS/);
expect(pgSql).not.toMatch(/RAISE EXCEPTION[^;]*BYPASSRLS/);
expect(pgSql).toContain('ALTER TABLE eval_candidates ENABLE ROW LEVEL SECURITY');
expect(pgSql).toContain('ALTER TABLE eval_capture_failures ENABLE ROW LEVEL SECURITY');
// warn-and-proceed: the ALTERs are outside the guard branch.
expect(pgSql).toMatch(
/IF NOT has_bypass THEN[\s\S]*?RAISE WARNING[\s\S]*?END IF;[\s\S]*ENABLE ROW LEVEL SECURITY/,
);
});
test('PGLite variant has no RLS / no BYPASSRLS gate', () => {
@@ -0,0 +1,231 @@
/**
* #3221 one-token doctor probes falsely fail reasoning models.
*
* The `gbrain models doctor` chat/expansion reachability probe used
* `maxTokens: 1`. Reasoning models spend output budget on internal reasoning
* before emitting any text, so a 1-token cap is either exhausted with zero
* usable text (finishReason 'length') or rejected outright by providers that
* require the cap to exceed the model's minimum reasoning spend and the
* probe then reported a failure for a perfectly reachable model.
*
* Pins (behavioral, via the gateway's `__setChatTransportForTests` seam
* no network, no gateway config needed since the transport branch returns
* before provider resolution):
*
* 1. The probe grants a sufficient diagnostic output budget (the shared
* PROBE_MAX_OUTPUT_TOKENS constant, strictly greater than 1).
* 2. A length-exhausted response with NO text still counts as reachable
* (status 'ok'), with the generation limitation surfaced in the message
* rather than a bare 'reachable'.
* 3. A normal completion keeps the plain 'reachable' message (no noise for
* the common case).
* 4. Provider errors are still classified as failures (the wider budget
* must not swallow real auth/config problems).
*/
import { describe, test, expect, afterEach } from 'bun:test';
import { readFileSync } from 'node:fs';
import { join } from 'node:path';
import {
__setChatTransportForTests,
__setGenerateTextTransportForTests,
configureGateway,
resetGateway,
type ChatOpts,
type ChatResult,
} from '../src/core/ai/gateway.ts';
import { probeModel, PROBE_MAX_OUTPUT_TOKENS } from '../src/commands/models.ts';
afterEach(() => {
__setChatTransportForTests(null);
__setGenerateTextTransportForTests(null);
resetGateway();
});
function stubResult(over: Partial<ChatResult> = {}): ChatResult {
return {
text: 'ok',
blocks: [{ type: 'text', text: 'ok' }],
stopReason: 'end',
usage: { input_tokens: 1, output_tokens: 1, cache_read_tokens: 0, cache_creation_tokens: 0 },
model: 'test:stub',
providerId: 'test',
...over,
};
}
describe('models doctor — probe token budget (#3221)', () => {
test('probe requests a sufficient output budget, not 1 token', async () => {
let seenMaxTokens: number | undefined;
__setChatTransportForTests(async (opts: ChatOpts) => {
seenMaxTokens = opts.maxTokens;
return stubResult();
});
const r = await probeModel('test:reasoner', 'chat');
expect(r.status).toBe('ok');
expect(seenMaxTokens).toBe(PROBE_MAX_OUTPUT_TOKENS);
// The bug class: a tiny budget cannot accommodate reasoning burn. Pin a
// real floor (not just > 1) so the constant can't quietly regress to a
// value that re-triggers the false-FAIL.
expect(PROBE_MAX_OUTPUT_TOKENS).toBeGreaterThanOrEqual(64);
});
test('length-exhausted empty completion is reachable, with the limitation surfaced', async () => {
// A direct reasoning model spends the whole probe budget on internal
// reasoning: valid HTTP response, finishReason 'length', empty text.
__setChatTransportForTests(async () =>
stubResult({ text: '', blocks: [], stopReason: 'length' }),
);
const r = await probeModel('test:reasoner', 'chat');
expect(r.status).toBe('ok');
expect(r.message).toContain('reachable');
// The generation limitation is reported, not silently folded into a
// bare 'reachable' (the issue's "clearly reporting the generation
// limitation" requirement).
expect(r.message).not.toBe('reachable');
expect(r.message).toContain('reasoning');
});
test('length-exhausted completion WITH text keeps the plain reachable message', async () => {
// Non-reasoning model that simply hit the cap mid-sentence: it emitted
// text, so generation itself was exercised — no caveat needed.
__setChatTransportForTests(async () =>
stubResult({ text: 'partial answ', stopReason: 'length' }),
);
const r = await probeModel('test:small-model', 'chat');
expect(r.status).toBe('ok');
expect(r.message).toBe('reachable');
});
test('normal completion reports plain reachable', async () => {
__setChatTransportForTests(async () => stubResult());
const r = await probeModel('test:small-model', 'expansion');
expect(r.status).toBe('ok');
expect(r.message).toBe('reachable');
expect(r.touchpoint).toBe('expansion');
});
test("gateway's contentless-length AIConfigError counts as reachable-with-caveat (#3217 composition)", async () => {
// When the gateway's empty-completion guard (#3217) rejects a
// length-exhausted contentless response, chat() THROWS before the probe
// ever sees a ChatResult. The throw itself proves transport + auth +
// model routing — the probe must report reachable-with-caveat, not a
// config failure.
const { AIConfigError } = await import('../src/core/ai/errors.ts');
__setChatTransportForTests(async () => {
throw new AIConfigError(
'chat(anthropic:claude-sonnet-4-6): output budget exhausted before any content was emitted (maxOutputTokens=64)',
'Raise maxTokens for this call.',
);
});
const r = await probeModel('test:reasoner', 'chat');
expect(r.status).toBe('ok');
expect(r.message).toContain('reachable');
expect(r.message).not.toBe('reachable');
expect(r.message).toContain('reasoning');
});
test('other AIConfigErrors still classify as failures', async () => {
const { AIConfigError } = await import('../src/core/ai/errors.ts');
__setChatTransportForTests(async () => {
throw new AIConfigError('chat(): no API key configured for provider anthropic');
});
const r = await probeModel('test:reasoner', 'chat');
expect(r.status).not.toBe('ok');
});
test('provider errors are still classified as failures', async () => {
__setChatTransportForTests(async () => {
throw Object.assign(new Error('401 unauthorized: bad api key'), { status: 401 });
});
const r = await probeModel('test:reasoner', 'chat');
expect(r.status).toBe('auth');
});
test('probe cap widens above a configured extended-thinking budget', async () => {
// Anthropic rejects requests where max_tokens <= thinking.budgetTokens.
// A user who configured `provider_chat_options.anthropic.thinking`
// (e.g. budgetTokens: 1024) would have the fixed 64-token probe cap
// rejected outright — the same false-FAIL class. Exercised through the
// generate-text seam so provider-option resolution stays live.
let capturedMaxOutputTokens: number | undefined;
__setGenerateTextTransportForTests(async (args: any) => {
capturedMaxOutputTokens = args.maxOutputTokens;
return {
content: [{ type: 'text', text: 'ok' }],
finishReason: 'stop',
usage: { inputTokens: 1, outputTokens: 1 },
} as any;
});
configureGateway({
embedding_model: 'openai:text-embedding-3-large',
embedding_dimensions: 1536,
chat_model: 'anthropic:claude-sonnet-4-6',
provider_chat_options: {
anthropic: { thinking: { type: 'enabled', budgetTokens: 1024 } },
},
env: { ANTHROPIC_API_KEY: 'sk-fake', OPENAI_API_KEY: 'sk-fake' },
});
const r = await probeModel('anthropic:claude-sonnet-4-6', 'chat');
expect(r.status).toBe('ok');
expect(capturedMaxOutputTokens).toBe(1024 + PROBE_MAX_OUTPUT_TOKENS);
});
test('probe cap widens for a model-scoped budget reached through a recipe alias', async () => {
// chat() resolves recipe aliases to the canonical model id before the
// model-scoped provider_chat_options lookup. The budget helper must
// mirror that: probing via the stale alias claude-sonnet-4-6-20250929
// still sees the budget configured under the canonical id.
let capturedMaxOutputTokens: number | undefined;
__setGenerateTextTransportForTests(async (args: any) => {
capturedMaxOutputTokens = args.maxOutputTokens;
return {
content: [{ type: 'text', text: 'ok' }],
finishReason: 'stop',
usage: { inputTokens: 1, outputTokens: 1 },
} as any;
});
configureGateway({
embedding_model: 'openai:text-embedding-3-large',
embedding_dimensions: 1536,
chat_model: 'anthropic:claude-sonnet-4-6',
provider_chat_options: {
'anthropic:claude-sonnet-4-6': { thinking: { type: 'enabled', budgetTokens: 2048 } },
},
env: { ANTHROPIC_API_KEY: 'sk-fake', OPENAI_API_KEY: 'sk-fake' },
});
const r = await probeModel('anthropic:claude-sonnet-4-6-20250929', 'chat');
expect(r.status).toBe('ok');
expect(capturedMaxOutputTokens).toBe(2048 + PROBE_MAX_OUTPUT_TOKENS);
});
test('human output renders ok-probe caveat messages, not only failure messages', () => {
// runModels needs a live engine, so pin the render branch structurally
// (same source-text convention as test/models-doctor-embed.test.ts): the
// non-json output path must print the message for an ok probe whose
// message deviates from the bare 'reachable' (the reasoning-burn caveat
// would otherwise be visible only under --json).
const src = readFileSync(join(__dirname, '..', 'src', 'commands', 'models.ts'), 'utf-8');
const runIdx = src.indexOf('export async function runModels');
expect(runIdx).toBeGreaterThan(0);
expect(src.slice(runIdx)).toMatch(/else if \(r\.message !== 'reachable'\)\s*\{[^}]*process\.stdout\.write\(`\s+\$\{r\.message\}\\n`\)/);
});
});
+2 -2
View File
@@ -89,7 +89,7 @@ describe('alias_resolved boost stage', () => {
});
describe('KNOBS_HASH_VERSION', () => {
it('is 12 (11→12 hard-exclude fold invalidates rows written under a different exclude policy, #2825)', () => {
expect(KNOBS_HASH_VERSION).toBe(12);
it('is 13 (12→13 source-boost map fold invalidates rows ranked under a different boost policy)', () => {
expect(KNOBS_HASH_VERSION).toBe(13);
});
});
+6 -3
View File
@@ -410,7 +410,10 @@ describe('knobsHash determinism + cross-mode separation (CDX-4)', () => {
// #2825: bumped 11→12 to fold the resolved hard-exclude prefix list
// (hx=) — cached rows leaked GBRAIN_SEARCH_EXCLUDE'd slugs across
// processes.
expect(KNOBS_HASH_VERSION).toBe(12);
// bumped 12→13 to fold the resolved source-boost map (sb=) — a
// ranking-policy change (GBRAIN_SOURCE_BOOST tune or new defaults)
// must not be served rows ranked under the previous policy.
expect(KNOBS_HASH_VERSION).toBe(13);
});
test('T1 (codex): floor_ratio set vs unset produces DIFFERENT hashes (cache contamination prevention)', () => {
@@ -575,8 +578,8 @@ describe('v0.40.4 — graph_signals knob', () => {
});
describe('v0.42.3.0 — autocut knobs', () => {
test('KNOBS_HASH_VERSION is 12 (11→12 hard-exclude fold, #2825)', () => {
expect(KNOBS_HASH_VERSION).toBe(12);
test('KNOBS_HASH_VERSION is 13 (12→13 source-boost map fold)', () => {
expect(KNOBS_HASH_VERSION).toBe(13);
});
test('bundle defaults: conservative off, balanced/tokenmax on @0.20', () => {
+36 -3
View File
@@ -27,7 +27,7 @@ import {
MODE_BUNDLES,
type ResolvedSearchKnobs,
} from '../../src/core/search/mode.ts';
import { resolveHardExcludes } from '../../src/core/search/source-boost.ts';
import { resolveHardExcludes, resolveBoostMap } from '../../src/core/search/source-boost.ts';
/** Build a baseline resolved knob set with all reranker fields filled. */
function baseKnobs(): ResolvedSearchKnobs {
@@ -44,7 +44,7 @@ function baseKnobs(): ResolvedSearchKnobs {
}
describe('KNOBS_HASH_VERSION + version invariants', () => {
test('version is 12 (…; 9→10 relational recall; 10→11 asymmetric input_type #1400; 11→12 hard-excludes #2825)', () => {
test('version is 13 (…; 10→11 asymmetric input_type #1400; 11→12 hard-excludes #2825; 12→13 source-boost map)', () => {
// v0.35.0.0: 1→2 to fold reranker fields. v0.35.6.0: 2→3 to fold
// floor_ratio. v0.36 wave: piggybacks on v=3 with 7 cross-modal knobs
// (D2) PLUS column + provider context (D8/CDX-2 cross-column isolation).
@@ -64,7 +64,9 @@ describe('KNOBS_HASH_VERSION + version invariants', () => {
// pre-fix document-side query vectors must not be served.
// #2825: 11→12 to fold the resolved hard-exclude prefix list (hx=) —
// cached rows leaked GBRAIN_SEARCH_EXCLUDE'd slugs across processes.
expect(KNOBS_HASH_VERSION).toBe(12);
// 12→13 to fold the resolved source-boost map (sb=) — a ranking-policy
// change must not be served rows ranked under the previous policy.
expect(KNOBS_HASH_VERSION).toBe(13);
});
test('hash is 16 hex chars regardless of reranker config', () => {
@@ -244,3 +246,34 @@ describe('v=12 hard-exclude participation (#2825)', () => {
);
});
});
describe('v=13 source-boost map participation', () => {
test('different boost maps → different hashes', () => {
const k = baseKnobs();
const defaults = knobsHash(k, { sourceBoostMap: resolveBoostMap(undefined) });
const tuned = knobsHash(k, { sourceBoostMap: resolveBoostMap('originals/:1.8,daily/:0.4') });
expect(defaults).not.toBe(tuned);
});
test('same map with different key insertion order → SAME hash (canonicalization)', () => {
const k = baseKnobs();
const a = knobsHash(k, { sourceBoostMap: { 'a/': 1.2, 'b/': 0.7 } });
const b = knobsHash(k, { sourceBoostMap: { 'b/': 0.7, 'a/': 1.2 } });
expect(a).toBe(b);
});
test('factor change on a single prefix changes the hash', () => {
const k = baseKnobs();
const a = knobsHash(k, { sourceBoostMap: { 'skills/': 1.2 } });
const b = knobsHash(k, { sourceBoostMap: { 'skills/': 1.3 } });
expect(a).not.toBe(b);
});
test('undefined sourceBoostMap is stable and distinct from an explicit map (legacy-caller fallback)', () => {
const k = baseKnobs();
expect(knobsHash(k)).toBe(knobsHash(k));
expect(knobsHash(k)).not.toBe(
knobsHash(k, { sourceBoostMap: resolveBoostMap(undefined) }),
);
});
});