diff --git a/CHANGELOG.md b/CHANGELOG.md index 660af18aa..c11e459c4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,106 @@ All notable changes to GBrain will be documented in this file. +## [0.35.6.0] - 2026-05-17 + +**Your search results stop letting weak pages climb to the top just because they have a lot of links pointing at them.** Off by default; turn it on with one config key. + +Here's the problem this fixes. When your agent searches your brain, gbrain ranks pages by how well they match the query, then it gives a small bonus to pages that have lots of inbound links, pages you write about often, and pages you touched recently. Those bonuses are small individually. But on a big brain indexed with a strong embedding model (the kind shipped in v0.35.0.0 with ZeroEntropy zembed-1, or anyone running OpenAI text-embedding-3-large or Voyage 3+), the strong embedder treats "topically adjacent" content as more similar than it really is. So a page that barely matches your query still lands in the candidate pool, picks up all three bonuses, and ends up ranked higher than the page that actually answers your question. + +The fix is a "floor." When you turn it on, gbrain only gives the metadata bonuses to pages near the top of the result list. Pages way down the list (far from the best match) get NO bonus, no matter how popular or important they are by other measures. So a weak match stays weak. A strong match stays on top. + +Built on a community contribution from @jayzalowitz (he runs gbrain inside [SkyTwin](https://github.com/jayzalowitz/skytwin), a twin-memory layer, and noticed the bug on his own labeled test set). His PR was #1091. We did a deep review, found a few correctness bugs, refactored the shape, and shipped the integrated version. Full credit on the commit. + +### How to turn it on + +```bash +# Try it on a single query first: +gbrain query "..." --floor-ratio 0.85 + +# Once you're happy, make it the default for every search: +gbrain config set search.floor_ratio 0.85 +``` + +`0.85` means "only boost pages that scored at least 85% as well as the best match." Good starting value if you're using a modern embedding model. Try `0.90` or `0.95` if you want the boost to apply to an even smaller head of the list. Values outside `0` to `1` are ignored, so a typo can't break your search. + +Off by default. We can't prove it helps on every brain (the original bug came from one specific test corpus), so we want you to opt in and tell us how it goes before we make it the default. + +### What you'd see in a concrete example + +Imagine you search "best meeting notes from Q1" and the brain returns: + +| Page | Match quality | Has many backlinks? | Without the floor | With the floor (0.85) | +|---|---|---|---|---| +| `meetings/q1-strategy-offsite` (the actual answer) | strong | no | bonus skipped, ranks lower | wins, ranks first | +| `people/some-popular-person` (barely matches the query) | weak (0.5x of the top match) | yes, 1000 backlinks | huge bonus, leapfrogs the meeting note | no bonus (below the floor), stays where it should | + +That second row is the bug you've maybe been hitting if your brain has a few "celebrity" pages that everyone links to. + +### What's safe to know about + +Three things to keep in mind: + +- **Your search cache will dip for a few minutes after the upgrade, then recover.** gbrain caches recent search results to make repeat queries fast. We had to change the cache key shape so it can tell "floor on" results apart from "floor off" results. While both versions are running side by side during a deploy, the cache rebuilds. Clears itself within the cache TTL (default 1 hour). +- **The gate only changes the three "metadata" bonuses (backlinks, salience, recency).** It does NOT change the exact-match bonus (when your query text literally appears in a page). Exact-match is a different kind of signal and stays on for everyone. +- **No environment variable.** If you want to set this brain-wide, use `gbrain config set search.floor_ratio 0.85`. We deliberately did not add a `GBRAIN_SEARCH_FLOOR_RATIO` env var so `gbrain search modes` doesn't end up lying to you about what's actually configured. + +### What we caught and fixed before merging + +A second-opinion review (we sent the diff to a different AI for an adversarial read) caught three real bugs the original PR shipped with: + +- **Cache could serve stale "no-floor" results to a "with-floor" caller.** Same shape as a bug we fixed in v0.32.3 for the other search knobs. Closed. +- **Pages with broken scores (NaN, comes up if an embedder version drift slipped through) would have skipped the floor and gotten boosted anyway.** Now they skip the boost entirely, which is the safer default. +- **If your search returned only negative-score results (unusual but possible), the floor would have rejected its own top match.** Now it just turns off the floor in that case. + +We also reshaped two things from the original PR: + +- **The floor is now computed once per search, not three times.** The original recomputed it at each bonus stage, which meant the order the bonuses ran in subtly changed which pages got gated out. Now it's one number, applied the same way to all three. +- **You can set it three ways instead of one.** Per-query (`--floor-ratio` flag), per-brain (config key, shown in `gbrain search modes`), or eventually per-mode bundle (the three search modes will get defaults once we have data on what works). + +### Itemized changes + +- `src/core/search/hybrid.ts` — three boost functions (`applyBacklinkBoost`, `applySalienceBoost`, `applyRecencyBoost`) gain an optional `floorThreshold?: number` parameter. Per-result loops now skip non-finite (NaN/Infinity) scores AND scores below the threshold. New exported `computeFloorThreshold(results, floorRatio)` is the single ratio→threshold converter; it returns `Number.NEGATIVE_INFINITY` (no gate) for undefined floorRatio, out-of-range values, or inputs with no positive signal. `runPostFusionStages` computes the threshold ONCE at entry and passes it uniformly to all three stages. `PostFusionOpts.floorRatio?: number` is the public-facing ratio. +- `src/core/search/mode.ts` — `ModeBundle.floor_ratio: number | undefined`. All three bundles set `floor_ratio: undefined` initially. `SearchKeyOverrides` and `SearchPerCallOpts` gain `floor_ratio?: number`. `resolveSearchMode` picks via the standard chain. `loadOverridesFromConfig` parses `search.floor_ratio` (validates 0..1 range; out-of-range silently drops). `SEARCH_MODE_CONFIG_KEYS` includes `'search.floor_ratio'`. `KNOBS_HASH_VERSION` bumped 2→3; `knobsHash()` appends a `fr=` segment at 4-decimal precision so 0.85, 0.851, and undefined all key into different rows. +- `src/core/types.ts` — `SearchOpts.floorRatio?: number`. +- `src/commands/search.ts` — `KNOB_DESCRIPTIONS` and the dashboard knob iteration include `floor_ratio`, so `gbrain search modes` and `gbrain search modes --json` surface the resolved value + source attribution alongside the other knobs. +- `test/search.test.ts` — 30+ new cases covering `computeFloorThreshold` (out-of-range, NaN, negative top, all-NaN, single result), `applyBacklinkBoost` floor gate (preservation, weak-gated, borderline-eligible, leapfrog regression, NaN skip-not-pass), `applySalienceBoost` floor gate (T6 IRON RULE parity), `applyRecencyBoost` floor gate (T6 IRON RULE regression — the modified function shipped with zero test coverage on the new param), and `runPostFusionStages` single-baseline composition (D6 pin against future single-floor refactor). +- `test/search-mode.test.ts` — `floor_ratio: undefined` added to the canonical-bundle fixtures for all three modes; `KNOBS_HASH_VERSION` pin updated to 3; new tests for `floor_ratio`-changes-hash (cache contamination prevention); `loadOverridesFromConfig` coverage for valid and out-of-range values. +- `test/search/knobs-hash-reranker.test.ts` — header comment + version assertion updated for the 2→3 bump. + +### What's NOT in scope (deferred) + +- **Default-on for any mode.** `MODE_BUNDLES.floor_ratio` stays `undefined` for conservative / balanced / tokenmax until per-corpus ablation against gbrain's own eval surfaces (`longmemeval`, `whoknows`, `suspected-contradictions`, BrainBench-Real) backs a default flip. See `TODOS.md`. +- **Per-stage floor ratios.** Single ratio applied uniformly to all three stages today. Different ratio per stage (different floor for salience vs recency) is v0.36+ if evidence shows asymmetric value. +- **Per-source floor.** Single global threshold today. Federated-read users (v0.34.1.0+) sharing a query across multiple sources get one floor across the merged result set. v0.36+ if real federated-read usage shows the suppression issue codex flagged. +- **Exact-match boost gating.** Explicit scope narrowing — exact-match is a lexical signal, different in kind from metadata boosts. +- **`GBRAIN_SEARCH_FLOOR_RATIO` env var.** Would create a hidden side door `resolveSearchMode()` can't see. Use the config key instead. + +### For contributors + +- Plan + cross-model review trail: `~/.claude/plans/swift-sniffing-nygaard.md` captures the 9-decision (D1-D9) review pass through `/plan-eng-review` and `/codex`. Three correctness bugs (cache contamination, NaN passes gate, negative-top breaks single-result) were caught by codex's outside voice; two prior eng-review recommendations (per-stage composition lock, env var) were reversed before implementation. Reading the plan is the fastest way to understand which architectural decisions are durable vs accidental. +- Community PR attribution: @jayzalowitz (PR #1091, SkyTwin twin-memory layer). The empirical motivation, the failure-mode framing, the dense-embedder targeting, and the `0.85` starting value are all from his ablation. Integration shape is gbrain-side. + +## To take advantage of v0.35.6.0 + +`gbrain upgrade` should do this automatically. If it didn't: + +1. **Run the orchestrator manually** (no-op if migrations already at HEAD; the knobsHash bump is a code-level constant, not a DB schema change): + ```bash + gbrain apply-migrations --yes + ``` +2. **Verify the new knob is wired:** + ```bash + gbrain search modes --json | grep -A 1 floor_ratio + ``` + Should show `"floor_ratio"` in the resolved knob map with `value: null` (no override set). +3. **Try the gate on a corpus you suspect of leapfrog regressions:** + ```bash + # Compare results with and without the gate; if your dense-embedder corpus + # exhibits the failure mode, gate=0.85 should restore the strong primary. + gbrain query "..." --floor-ratio 0.85 + ``` +4. **If `gbrain doctor` flags any new sync_failures or schema warnings post-upgrade,** the floor-ratio path is not the cause (no schema change in this release). File the issue at https://github.com/garrytan/gbrain/issues with `gbrain doctor` output. + ## [0.35.5.1] - 2026-05-16 **`gbrain doctor` stops counting clean supervisor exits as crashes — the "120x/24h" alarm finally reflects real crashes, with per-cause breakdown for operator triage.** diff --git a/CLAUDE.md b/CLAUDE.md index f7daedfc9..7c33c5c30 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1213,31 +1213,67 @@ know exactly what changed. ### Release-summary template -Use this structure for the top of every `## [X.Y.Z]` entry: +**Iron rule: lead ELI10, get precise after.** The first ~150 words of every entry +must be readable by someone who does NOT know gbrain's internals. No file paths, +no function names, no internal constants, no acronyms (no "RRF", no "knobsHash", +no "MODE_BUNDLES", no "CDX-4"), no jargon that requires reading the codebase to +parse. Lead with the user-visible behavior change, in everyday English, like +you're explaining it to a smart engineer who has never opened the repo. -1. **Two-line bold headline** (10-14 words total) ... should land like a verdict, not - marketing. Sound like someone who shipped today and cares whether it works. -2. **Lead paragraph** (3-5 sentences) ... what shipped, what changed for the user. - Specific, concrete, no AI vocabulary, no em dashes, no hype. -3. **A "The X numbers that matter" section** with: - - One short setup paragraph naming the source of the numbers (real production - deployment OR a reproducible benchmark ... name the file/command to run). - - A table of 3-6 key metrics with BEFORE / AFTER / Δ columns. - - A second optional table for per-category breakdown if relevant. - - 1-2 sentences interpreting the most striking number in concrete user terms. -4. **A "What this means for [audience]" closing paragraph** (2-4 sentences) tying - the metrics to a real workflow shift. End with what to do. +THEN, once the reader knows what shipped and why they'd care, drill into the +precise details: real file paths, real function names, real config keys, real +numbers. The precision part is required (the entry is also the technical record +of what changed), but it lives AFTER the plain-English lead, never before it. -Voice rules: +The shape: + +1. **One-line bold headline.** What changed for the user, in human English. No + jargon. No internal terms. Example good: "Your search stops boosting weak + pages just because they have a lot of links pointing at them." Example bad: + "PostFusionOpts gains floorRatio; KNOBS_HASH_VERSION bumped 2→3." +2. **Plain-English opener** (~3-5 sentences). Describe the problem this fixes in + everyday terms. Pretend the reader has a brain full of meeting notes and + people pages and wants to know if this release helps them. Concrete example + beats abstract description. +3. **A "How to turn it on" or "How to use it" section** with paste-ready + commands. Real flags, real config keys. This is where precision starts. +4. **A "What you'd see in a concrete example" or "The X numbers that matter" + section** with a table. Use everyday-language column headers ("Page", + "Match quality", "Has many backlinks?") even when the underlying mechanism + is technical. The table teaches what the feature does without requiring the + reader to understand how. +5. **A "What's safe to know about" or "Things to watch" section** for caveats, + side effects, cache invalidation, mid-deploy notes. Still in plain language. +6. **A "What we caught and fixed before merging" section** if the work went + through review (CEO/eng/codex/outside-voice). Translate review findings into + plain English. "We caught a stale-cache bug" beats "knobsHash() did not + include floorRatio in the v=2 hash input." +7. **`### Itemized changes`** (precision lives here). File paths, function + names, types, constants, line numbers. This section is for engineers who + need to know exactly what moved. + +Voice rules (apply throughout): - No em dashes (use commas, periods, "..."). - No AI vocabulary (delve, robust, comprehensive, nuanced, fundamental, etc.) or banned phrases ("here's the kicker", "the bottom line", etc.). -- Real numbers, real file names, real commands. Not "fast" but "~30s on 30K pages." +- Real numbers, real file names, real commands AFTER the ELI10 lead. Not "fast" + but "~30s on 30K pages." In the ELI10 lead, "fast enough that you won't + notice" or "~30 seconds even on a big brain." - Short paragraphs, mix one-sentence punches with 2-3 sentence runs. - Connect to user outcomes: "the agent does ~3x less reading" beats "improved precision." - Be direct about quality. "Well-designed" or "this is a mess." No dancing. +**The smell test:** if someone who has never opened gbrain reads the first 150 +words and walks away knowing what shipped and whether they care, the entry +passes. If they need to grep the codebase to follow along, rewrite the lead. + +**Canonical examples in this CHANGELOG:** v0.35.6.0 (floor-ratio gate, written +ELI10-lead-first), v0.34.4.0 (embed stale fix wave). Use those shapes when in +doubt. Avoid the shape of entries that lead with internal constants or release +mechanics; those exist in older history but should not be the model for new +work. + Source material to pull from: - CHANGELOG.md previous entry for prior context - Latest `gbrain-evals/docs/benchmarks/[latest].md` for headline numbers (sibling repo) diff --git a/TODOS.md b/TODOS.md index 0a12739df..9e9731cca 100644 --- a/TODOS.md +++ b/TODOS.md @@ -1,6 +1,17 @@ # TODOS +## v0.35.6.0 floor-ratio gate follow-ups (v0.36.x+) + +- [ ] **v0.36.x: Run gbrain-side floor-ratio ablation before flipping any mode-bundle default.** v0.35.6.0 ships the gate default-off (`MODE_BUNDLES[*].floor_ratio = undefined`) because the SkyTwin labeled-retrieval ablation that surfaced the regression isn't reproducible on gbrain's own eval surfaces from outside. Before any mode-bundle default flip, run the gate at `floor_ratio: undefined`, 0.85, 0.90, 0.95 across `gbrain eval longmemeval`, `gbrain eval whoknows`, `gbrain eval suspected-contradictions`, and the BrainBench-Real replay (sibling gbrain-evals repo). Quantify per-mode P@k / R@k / nDCG@k / top-1 stability deltas. Look for: regression on queries that genuinely need the long-tail boost (specific entity lookups, low-frequency topics) vs improvement on queries where weak-overlap pages were leapfrogging. The corpus-level finding determines whether tokenmax (most exposure to the failure mode) should flip first, or whether the gate stays a per-call opt-in indefinitely. Filed during v0.35.6.0 codex outside-voice review. + +- [ ] **v0.36.x: `MODE_BUNDLES.floor_ratio` integration shape — populate after ablation evidence.** v0.35.6.0 leaves `floor_ratio: undefined` in all three bundles deliberately. After the ablation TODO above, set per-mode defaults: probably `tokenmax: 0.85` first (high-context tier, broad searchLimit=50, expansion=on — most exposure to leapfrog), `balanced` second if signal holds, `conservative` only if the ablation shows the gate doesn't hurt on small candidate pools. Update the canonical-bundle tests in `test/search-mode.test.ts` (3 fixtures) when flipping. The KNOBS_HASH_VERSION does NOT need to bump for a default change — the per-bundle default is part of the hash input already. + +- [ ] **v0.36.x: Per-source floor-ratio (federated read).** v0.35.6.0 uses a single global threshold across all sources. Federated-read users (v0.34.1.0+) sharing a query across multiple sources get one floor across the merged result set, which means a high-scoring source can suppress metadata boosts for pages in another source. Codex outside-voice flagged this during v0.35.6.0 review; user explicitly chose the simpler primitive (D9=A). If a federated-read user later reports legitimate per-source winners being suppressed, the fix is a per-source threshold map computed at `runPostFusionStages` entry (one threshold per unique `source_id` in the result set). Plan reference: D9 in `~/.claude/plans/swift-sniffing-nygaard.md`. + +- [ ] **v0.36.x: Reranker top-N expansion when floor-ratio narrows the candidate pool.** Floor-ratio can suppress a legitimate candidate that would have made it to the reranker's top-N. Sanity check after the v0.36 ablation: if tokenmax with `floor_ratio: 0.85` and `reranker_top_n_in: 30` shows the reranker seeing a meaningfully different set than without the gate, consider expanding `reranker_top_n_in` when floor is set (e.g. 30 → 40) so the reranker still has 30 floor-eligible candidates to reorder. Cheap mitigation if the data supports it. Not a blocker. + + ## dreamy-thompson wave follow-ups (v0.36.x) - [ ] **v0.36.x: runThink full rewrite — drop ThinkLLMClient indirection.** v0.36's fix(think) wave landed a gateway-backed adapter at `src/core/think/index.ts:225-251` so `gbrain config set anthropic_api_key` works over MCP stdio (closed #952). The adapter routes through `gateway.chat()` but `runThink` still carries the `ThinkLLMClient` interface as the test seam — it's the last LLM-using path that doesn't use the canonical `__setChatTransportForTests` seam v0.31.12 established for chat/embed. Cleanup: drop `ThinkLLMClient`, drop the `opts.client` injection point, migrate the 12+ existing tests (`test/think-pipeline.serial.test.ts:144,181,222`, `test/think-gateway-adapter.test.ts`, plus 9+ others that stub the interface) to `__setChatTransportForTests`. Pros: codebase consistency, one fewer test-stub pattern, easier to add provider switching for think once it routes through gateway natively. Cons: 12+ test files need migration. Blocked by: v0.36 wave landing on master (so the adapter exists to lean on while migrating tests). Plan reference: D5 + D7 in `~/.claude/plans/ok-i-spun-up-dreamy-thompson.md`. diff --git a/VERSION b/VERSION index d4fe04f21..54246e16e 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.35.5.1 \ No newline at end of file +0.35.6.0 \ No newline at end of file diff --git a/llms-full.txt b/llms-full.txt index 9c59c4848..a709c4f0a 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -1321,31 +1321,67 @@ know exactly what changed. ### Release-summary template -Use this structure for the top of every `## [X.Y.Z]` entry: +**Iron rule: lead ELI10, get precise after.** The first ~150 words of every entry +must be readable by someone who does NOT know gbrain's internals. No file paths, +no function names, no internal constants, no acronyms (no "RRF", no "knobsHash", +no "MODE_BUNDLES", no "CDX-4"), no jargon that requires reading the codebase to +parse. Lead with the user-visible behavior change, in everyday English, like +you're explaining it to a smart engineer who has never opened the repo. -1. **Two-line bold headline** (10-14 words total) ... should land like a verdict, not - marketing. Sound like someone who shipped today and cares whether it works. -2. **Lead paragraph** (3-5 sentences) ... what shipped, what changed for the user. - Specific, concrete, no AI vocabulary, no em dashes, no hype. -3. **A "The X numbers that matter" section** with: - - One short setup paragraph naming the source of the numbers (real production - deployment OR a reproducible benchmark ... name the file/command to run). - - A table of 3-6 key metrics with BEFORE / AFTER / Δ columns. - - A second optional table for per-category breakdown if relevant. - - 1-2 sentences interpreting the most striking number in concrete user terms. -4. **A "What this means for [audience]" closing paragraph** (2-4 sentences) tying - the metrics to a real workflow shift. End with what to do. +THEN, once the reader knows what shipped and why they'd care, drill into the +precise details: real file paths, real function names, real config keys, real +numbers. The precision part is required (the entry is also the technical record +of what changed), but it lives AFTER the plain-English lead, never before it. -Voice rules: +The shape: + +1. **One-line bold headline.** What changed for the user, in human English. No + jargon. No internal terms. Example good: "Your search stops boosting weak + pages just because they have a lot of links pointing at them." Example bad: + "PostFusionOpts gains floorRatio; KNOBS_HASH_VERSION bumped 2→3." +2. **Plain-English opener** (~3-5 sentences). Describe the problem this fixes in + everyday terms. Pretend the reader has a brain full of meeting notes and + people pages and wants to know if this release helps them. Concrete example + beats abstract description. +3. **A "How to turn it on" or "How to use it" section** with paste-ready + commands. Real flags, real config keys. This is where precision starts. +4. **A "What you'd see in a concrete example" or "The X numbers that matter" + section** with a table. Use everyday-language column headers ("Page", + "Match quality", "Has many backlinks?") even when the underlying mechanism + is technical. The table teaches what the feature does without requiring the + reader to understand how. +5. **A "What's safe to know about" or "Things to watch" section** for caveats, + side effects, cache invalidation, mid-deploy notes. Still in plain language. +6. **A "What we caught and fixed before merging" section** if the work went + through review (CEO/eng/codex/outside-voice). Translate review findings into + plain English. "We caught a stale-cache bug" beats "knobsHash() did not + include floorRatio in the v=2 hash input." +7. **`### Itemized changes`** (precision lives here). File paths, function + names, types, constants, line numbers. This section is for engineers who + need to know exactly what moved. + +Voice rules (apply throughout): - No em dashes (use commas, periods, "..."). - No AI vocabulary (delve, robust, comprehensive, nuanced, fundamental, etc.) or banned phrases ("here's the kicker", "the bottom line", etc.). -- Real numbers, real file names, real commands. Not "fast" but "~30s on 30K pages." +- Real numbers, real file names, real commands AFTER the ELI10 lead. Not "fast" + but "~30s on 30K pages." In the ELI10 lead, "fast enough that you won't + notice" or "~30 seconds even on a big brain." - Short paragraphs, mix one-sentence punches with 2-3 sentence runs. - Connect to user outcomes: "the agent does ~3x less reading" beats "improved precision." - Be direct about quality. "Well-designed" or "this is a mess." No dancing. +**The smell test:** if someone who has never opened gbrain reads the first 150 +words and walks away knowing what shipped and whether they care, the entry +passes. If they need to grep the codebase to follow along, rewrite the lead. + +**Canonical examples in this CHANGELOG:** v0.35.6.0 (floor-ratio gate, written +ELI10-lead-first), v0.34.4.0 (embed stale fix wave). Use those shapes when in +doubt. Avoid the shape of entries that lead with internal constants or release +mechanics; those exist in older history but should not be the model for new +work. + Source material to pull from: - CHANGELOG.md previous entry for prior context - Latest `gbrain-evals/docs/benchmarks/[latest].md` for headline numbers (sibling repo) diff --git a/package.json b/package.json index 2f1907aaf..11380f93c 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gbrain", - "version": "0.35.5.1", + "version": "0.35.6.0", "description": "Postgres-native personal knowledge brain with hybrid RAG search", "type": "module", "main": "src/core/index.ts", diff --git a/src/commands/search.ts b/src/commands/search.ts index b317b2aac..29bb5ea22 100644 --- a/src/commands/search.ts +++ b/src/commands/search.ts @@ -53,6 +53,7 @@ const KNOB_DESCRIPTIONS: Record = { reranker_top_n_in: 'Candidates sent to reranker per call', reranker_top_n_out: 'Cap on reranked output (null = no truncate)', reranker_timeout_ms: 'HTTP timeout for the reranker call', + floor_ratio: 'Floor-ratio gate for metadata boosts (0..1, undefined = off)', }; interface SearchModesReport { @@ -79,6 +80,10 @@ async function buildModesReport(engine: BrainEngine): Promise 'tokenBudget', 'expansion', 'searchLimit', + // v0.35.6.0 — floor-ratio surfaced in `gbrain search modes` dashboard + // so config drift is legible. Default undefined renders as 'undefined' + // in the bundle column, 'mode' source when unset by config/per-call. + 'floor_ratio', ]; const attributions = {} as SearchModesReport['resolved']; diff --git a/src/core/search/hybrid.ts b/src/core/search/hybrid.ts index dc0cec6dd..b3073d1ac 100644 --- a/src/core/search/hybrid.ts +++ b/src/core/search/hybrid.ts @@ -47,9 +47,27 @@ const DEBUG = process.env.GBRAIN_SEARCH_DEBUG === '1'; * Apply backlink boost to a result list in place. Mutates each result's score * by (1 + BACKLINK_BOOST_COEF * log(1 + count)). Pure data transform; no DB call. * Caller fetches counts via engine.getBacklinkCounts. + * + * v0.35.6.0 — floor-ratio gate. When `floorThreshold` is provided, results + * with `r.score < floorThreshold` are SKIPPED (no boost applied). NaN scores + * are also skipped (NaN < x is false in JS, which would otherwise let NaN + * results bypass the gate). The threshold is an ABSOLUTE score, not a ratio + * — compute it once at `runPostFusionStages` entry via `computeFloorThreshold` + * so stage order doesn't change which results clear the gate. + * + * The gate is scoped to the three metadata-axis boost stages (backlink + + * salience + recency). Exact-match boost (`applyExactMatchBoost` in + * intent-weights.ts) runs independently as a lexical-relevance signal by + * design. */ -export function applyBacklinkBoost(results: SearchResult[], counts: Map): void { +export function applyBacklinkBoost( + results: SearchResult[], + counts: Map, + floorThreshold?: number, +): void { for (const r of results) { + if (!Number.isFinite(r.score)) continue; + if (floorThreshold !== undefined && r.score < floorThreshold) continue; const count = counts.get(r.slug) ?? 0; if (count > 0) { r.score *= (1.0 + BACKLINK_BOOST_COEF * Math.log(1 + count)); @@ -57,6 +75,52 @@ export function applyBacklinkBoost(results: SearchResult[], counts: Map 1 (out-of-range silently + * disables the gate; range validation lives at the config-parse layer) + * - No result has a positive, finite score (all-NaN, all-negative, or empty + * input arrays produce no positive signal — gate stays off) + * + * Otherwise returns `topScore * floorRatio`, where `topScore` is the largest + * finite score in `results`. Callers compute this ONCE before any boost stage + * runs, then pass the resulting threshold to every stage. Single-baseline + * semantic — order-independent across the three metadata-axis boosts. + * + * Why this exists: gbrain's bounded boosts (`[1.0, ~1.6]` log-compressed + * salience clip, log-scaled backlinks, half-life recency) keep any single + * boost from catastrophically flipping rankings on curated small corpora. + * On larger corpora indexed with dense embedders (text-embedding-3-large, + * Voyage 3+, ZeroEntropy zembed-1), weak-overlap candidates can land in + * top-K via baseline vector overlap and accumulate metadata boost until + * they leapfrog the legitimate primary hit. The gate restricts each + * metadata boost to the head of the candidate pool so the long tail keeps + * its unboosted relevance ranking. + * + * 0.85 is a reasonable starting value for dense-embedder corpora. Default + * stays undefined (no gate) until per-corpus ablation evidence supports a + * default flip (see `TODOS.md` floor-ratio ablation entry). + */ +export function computeFloorThreshold( + results: SearchResult[], + floorRatio: number | undefined, +): number { + if (floorRatio === undefined) return Number.NEGATIVE_INFINITY; + if (!Number.isFinite(floorRatio) || floorRatio < 0 || floorRatio > 1) { + return Number.NEGATIVE_INFINITY; + } + let top = Number.NEGATIVE_INFINITY; + for (const r of results) { + if (Number.isFinite(r.score) && r.score > top) top = r.score; + } + if (!Number.isFinite(top) || top <= 0) return Number.NEGATIVE_INFINITY; + return top * floorRatio; +} + /** * v0.29.1 — apply salience boost (emotional_weight + take_count, NO time * component). Mirror of applyBacklinkBoost. Mutate-in-place; caller re-sorts. @@ -72,9 +136,12 @@ export function applySalienceBoost( results: SearchResult[], scores: Map, strength: 'on' | 'strong', + floorThreshold?: number, ): void { const k = strength === 'strong' ? 0.30 : 0.15; for (const r of results) { + if (!Number.isFinite(r.score)) continue; + if (floorThreshold !== undefined && r.score < floorThreshold) continue; const key = `${r.source_id ?? 'default'}::${r.slug}`; const score = scores.get(key); if (!score || score <= 0) continue; @@ -101,12 +168,15 @@ export function applyRecencyBoost( decayMap: import('./recency-decay.ts').RecencyDecayMap, fallback: import('./recency-decay.ts').RecencyDecayConfig, nowMs: number = Date.now(), + floorThreshold?: number, ): void { const strengthMul = strength === 'strong' ? 1.5 : 1.0; // Sort prefixes longest-first so 'media/articles/' matches before 'media/'. const prefixes = Object.keys(decayMap).sort((a, b) => b.length - a.length); for (const r of results) { + if (!Number.isFinite(r.score)) continue; + if (floorThreshold !== undefined && r.score < floorThreshold) continue; const key = `${r.source_id ?? 'default'}::${r.slug}`; const d = dates.get(key); if (!d) continue; @@ -143,6 +213,23 @@ export interface PostFusionOpts { recency: 'off' | 'on' | 'strong'; decayMap?: import('./recency-decay.ts').RecencyDecayMap; fallback?: import('./recency-decay.ts').RecencyDecayConfig; + /** + * v0.35.6.0 — floor-ratio gate (opt-in, default off). When set, each + * metadata-axis boost stage (backlink, salience, recency) skips results + * whose score is below `floorRatio * topScore`. Threshold is computed + * ONCE at runPostFusionStages entry from the post-cosine-rescore score + * snapshot, then passed uniformly to all three stages — order-independent. + * + * Default undefined preserves prior behavior bit-for-bit. Sensible values + * for dense-embedder corpora: 0.85-0.95. See `computeFloorThreshold` for + * the empirical motivation and out-of-range handling. + * + * SCOPE: gates the three metadata stages only. Exact-match boost + * (`applyExactMatchBoost`) runs AFTER `runPostFusionStages` and is NOT + * gated — it's a lexical-relevance signal, different in kind from + * metadata boosts. + */ + floorRatio?: number; } export async function runPostFusionStages( @@ -152,12 +239,19 @@ export async function runPostFusionStages( ): Promise { if (results.length === 0) return; + // v0.35.6.0 [floor-ratio gate]: compute threshold ONCE at entry, BEFORE any + // boost mutates scores. Single-baseline semantic — the same threshold gates + // all three downstream stages. This is intentionally different from a + // per-stage recompute (which would couple stage order to gating decisions); + // see plan `swift-sniffing-nygaard.md` D6 / codex outside-voice T2. + const floorThreshold = computeFloorThreshold(results, opts.floorRatio); + // Backlink stage (existing behavior, preserved). if (opts.applyBacklinks) { try { const slugs = Array.from(new Set(results.map(r => r.slug))); const counts = await engine.getBacklinkCounts(slugs); - applyBacklinkBoost(results, counts); + applyBacklinkBoost(results, counts, floorThreshold); } catch { // Non-fatal; preserves the existing pre-v0.29.1 contract. } @@ -174,7 +268,7 @@ export async function runPostFusionStages( if (opts.salience !== 'off') { try { const scores = await engine.getSalienceScores(refs); - applySalienceBoost(results, scores, opts.salience); + applySalienceBoost(results, scores, opts.salience, floorThreshold); } catch { // Non-fatal. } @@ -191,6 +285,8 @@ export async function runPostFusionStages( opts.recency, opts.decayMap ?? DEFAULT_RECENCY_DECAY, opts.fallback ?? DEFAULT_FALLBACK, + Date.now(), + floorThreshold, ); } catch { // Non-fatal. @@ -244,6 +340,10 @@ export async function hybridSearch( tokenBudget: opts?.tokenBudget, expansion: opts?.expansion, searchLimit: opts?.limit, + // v0.35.6.0 — floor-ratio gate thread-through. Per-call value wins + // over per-key config wins over mode bundle (currently undefined for + // all 3 bundles — pending ablation evidence). + floor_ratio: opts?.floorRatio, }, }); @@ -356,10 +456,14 @@ export async function hybridSearch( ?? (suggestions.suggestedRecency !== 'off' ? suggestions.suggestedRecency : (intentRecency ?? suggestions.suggestedRecency)); - const postFusionOpts = { + const postFusionOpts: PostFusionOpts = { applyBacklinks: true, salience: salienceMode, recency: recencyMode, + // v0.35.6.0 — floor-ratio gate threaded from resolved mode. Default + // undefined for all 3 bundles → no behavior change unless caller sets + // SearchOpts.floorRatio or `search.floor_ratio` config key. + floorRatio: resolvedMode.floor_ratio, }; // Skip vector search entirely if the gateway has no embedding provider configured (Codex C3). @@ -620,6 +724,11 @@ export async function hybridSearchCached( expansion: opts?.expansion, intentWeighting: opts?.intentWeighting, searchLimit: opts?.limit, + // v0.35.6.0 — floor-ratio threaded through cache resolver too so + // knobsHash() differentiates floor-on vs floor-off cache rows. + // Without this, a no-floor write would be served to a floor-enabled + // read (ranking-correctness leak, codex T1). + floor_ratio: opts?.floorRatio, }, }); const cacheKnobsHash = knobsHash(resolvedForCache); diff --git a/src/core/search/mode.ts b/src/core/search/mode.ts index 2d04063e1..3d4694914 100644 --- a/src/core/search/mode.ts +++ b/src/core/search/mode.ts @@ -94,6 +94,23 @@ export interface ModeBundle { reranker_top_n_out: number | null; /** HTTP timeout in ms (default 5000). Threaded into gateway.rerank. */ reranker_timeout_ms: number; + /** + * v0.35.6.0 — floor-ratio gate for metadata-axis boost stages (backlink, + * salience, recency). `undefined` = no gate (default for all three modes; + * preserves prior behavior bit-for-bit). When set to a number in [0, 1], + * each gated stage skips results whose score is below + * `floorRatio * topScore`, where topScore is computed ONCE at + * runPostFusionStages entry from the post-cosine-rescore snapshot. + * + * Sensible operator override values for dense-embedder corpora: 0.85-0.95. + * Default stays undefined until per-corpus ablation evidence supports a + * mode-level default. See `TODOS.md` floor-ratio ablation entry. + * + * Scoped to the three metadata boost stages — exact-match boost + * (intent-weights.applyExactMatchBoost) runs independently as a lexical + * relevance signal and is NOT gated. + */ + floor_ratio: number | undefined; } /** @@ -119,6 +136,9 @@ export const MODE_BUNDLES: Readonly>> = reranker_top_n_in: 30, reranker_top_n_out: null, reranker_timeout_ms: 5000, + // v0.35.6.0 — undefined for all three bundles; the per-corpus ablation + // (TODOS.md) gates any default flip. + floor_ratio: undefined, }), balanced: Object.freeze({ cache_enabled: true, @@ -136,6 +156,9 @@ export const MODE_BUNDLES: Readonly>> = reranker_top_n_in: 30, reranker_top_n_out: null, reranker_timeout_ms: 5000, + // v0.35.6.0 — undefined for all three bundles; the per-corpus ablation + // (TODOS.md) gates any default flip. + floor_ratio: undefined, }), tokenmax: Object.freeze({ cache_enabled: true, @@ -155,6 +178,9 @@ export const MODE_BUNDLES: Readonly>> = reranker_top_n_in: 30, reranker_top_n_out: null, reranker_timeout_ms: 5000, + // v0.35.6.0 — undefined for all three bundles; the per-corpus ablation + // (TODOS.md) gates any default flip. + floor_ratio: undefined, }), }); @@ -186,6 +212,8 @@ export interface SearchKeyOverrides { // number | undefined. reranker_top_n_out?: number | null; reranker_timeout_ms?: number; + // v0.35.6.0 — floor-ratio gate override. + floor_ratio?: number; } /** @@ -209,6 +237,8 @@ export interface SearchPerCallOpts { reranker_top_n_in?: number; reranker_top_n_out?: number | null; reranker_timeout_ms?: number; + // v0.35.6.0 — floor-ratio per-call override. + floor_ratio?: number; } /** @@ -267,6 +297,8 @@ export function resolveSearchMode(input: ResolveSearchModeInput): ResolvedSearch reranker_top_n_in: pick('reranker_top_n_in'), reranker_top_n_out: pick('reranker_top_n_out'), reranker_timeout_ms: pick('reranker_timeout_ms'), + // v0.35.6.0 — floor-ratio resolved via the same pick chain. + floor_ratio: pick('floor_ratio'), resolved_mode, mode_valid: valid, }; @@ -317,17 +349,19 @@ export function attributeKnob( */ // v0.35.0.0+ bump 1→2: reranker fields participate in the cache key so a // tokenmax-with-reranker write can't be served to a reranker-off lookup. +// v0.35.6.0 bump 2→3: floor_ratio participates so a floor-on write can't +// be served to a floor-off lookup (cross-floor contamination, codex T1). // CDX2-F13 convention: under a version bump, additions are APPEND-ONLY at // the end of `parts[]` — reordering existing fields would silently rebuild // the hash for every existing row. // // CDX2-F12 mid-deploy duplicate-row note: because `cacheRowId()` (in -// src/core/search/query-cache.ts) includes knobsHash, a v=1 process and a -// v=2 process writing the same `(source_id, query_text)` produce DISTINCT +// src/core/search/query-cache.ts) includes knobsHash, a v=2 process and a +// v=3 process writing the same `(source_id, query_text)` produce DISTINCT // row IDs. Expect a temporary hit-rate dip + cache-row doubling for hot // queries during a rolling deploy. Clears naturally within // `cache.ttl_seconds` (default 3600s). The CHANGELOG note covers this. -export const KNOBS_HASH_VERSION = 2; +export const KNOBS_HASH_VERSION = 3; export function knobsHash(knobs: ResolvedSearchKnobs): string { // Fixed-order key list. Adding a knob here REQUIRES bumping @@ -348,6 +382,10 @@ export function knobsHash(knobs: ResolvedSearchKnobs): string { `rri=${knobs.reranker_top_n_in}`, `rro=${knobs.reranker_top_n_out ?? 'none'}`, `rrt=${knobs.reranker_timeout_ms}`, + // v=3 additions (append-only). Use 4-decimal precision so 0.85 and + // 0.851 differ in the hash; undefined uses literal 'none' so a + // floor-off write and a floor-on write key into different rows. + `fr=${knobs.floor_ratio === undefined ? 'none' : knobs.floor_ratio.toFixed(4)}`, ]; const h = createHash('sha256'); h.update(parts.join('|')); @@ -435,6 +473,16 @@ export function loadOverridesFromConfig( if (Number.isFinite(n) && n > 0) out.reranker_timeout_ms = n; } + // v0.35.6.0 — floor-ratio config key. Accepts a number in [0, 1]; values + // outside that range silently fall through (no override applied). The + // runtime computeFloorThreshold also guards against out-of-range so a + // malformed value never gates anything — defense in depth. + const fr = get('search.floor_ratio'); + if (fr !== undefined) { + const n = parseFloat(fr); + if (Number.isFinite(n) && n >= 0 && n <= 1) out.floor_ratio = n; + } + return out; } @@ -453,6 +501,8 @@ export const SEARCH_MODE_CONFIG_KEYS: ReadonlyArray = Object.freeze([ 'search.reranker.top_n_in', 'search.reranker.top_n_out', 'search.reranker.timeout_ms', + // v0.35.6.0 — floor-ratio gate + 'search.floor_ratio', ]); /** diff --git a/src/core/types.ts b/src/core/types.ts index 58dc28b33..21456b460 100644 --- a/src/core/types.ts +++ b/src/core/types.ts @@ -576,6 +576,34 @@ export interface SearchOpts { // Test seam — never set in production code. rerankerFn?: (input: { query: string; documents: string[]; topN?: number; model?: string; signal?: AbortSignal; timeoutMs?: number }) => Promise<{ index: number; relevanceScore: number }[]>; }; + /** + * v0.35.6.0 — floor-ratio gate for metadata-axis boost stages (backlink, + * salience, recency). Number in [0, 1] or undefined (default = no gate). + * + * When set, each gated stage skips results whose pre-boost score is below + * `floorRatio * topScore`, where `topScore` is computed ONCE at + * `runPostFusionStages` entry from the post-cosine-rescore snapshot. The + * same threshold gates all three stages — order-independent semantic. + * + * Resolution chain (mirrors other search-lite knobs): + * per-call `SearchOpts.floorRatio` → config `search.floor_ratio` + * → MODE_BUNDLES[mode].floor_ratio (undefined for all 3 modes today) + * → undefined fallback. + * + * SCOPE: gates ONLY the three metadata stages. Exact-match boost + * (`applyExactMatchBoost` in intent-weights.ts) runs independently as a + * lexical-relevance signal and is NOT gated by design. + * + * Sensible operator override values for dense-embedder corpora: 0.85-0.95. + * Default stays undefined pending per-corpus ablation evidence (see + * `TODOS.md` floor-ratio ablation entry). + * + * Out-of-range values (negative, > 1, NaN, Infinity) silently disable + * the gate at the runtime layer; the config-parse layer also rejects + * out-of-range values. Defense in depth — a malformed value never + * gates anything. + */ + floorRatio?: number; } /** diff --git a/test/search-mode.test.ts b/test/search-mode.test.ts index d71fba313..6883f2fcf 100644 --- a/test/search-mode.test.ts +++ b/test/search-mode.test.ts @@ -52,6 +52,9 @@ describe('SEARCH_MODES + MODE_BUNDLES canonical shape', () => { reranker_top_n_in: 30, reranker_top_n_out: null, reranker_timeout_ms: 5000, + // v0.35.6.0 — floor_ratio undefined in all three bundles; the per-corpus + // ablation TODO gates any default flip. + floor_ratio: undefined, }); }); @@ -69,6 +72,7 @@ describe('SEARCH_MODES + MODE_BUNDLES canonical shape', () => { reranker_top_n_in: 30, reranker_top_n_out: null, reranker_timeout_ms: 5000, + floor_ratio: undefined, }); }); @@ -86,6 +90,7 @@ describe('SEARCH_MODES + MODE_BUNDLES canonical shape', () => { reranker_top_n_in: 30, reranker_top_n_out: null, reranker_timeout_ms: 5000, + floor_ratio: undefined, }); }); @@ -265,9 +270,33 @@ describe('knobsHash determinism + cross-mode separation (CDX-4)', () => { test('KNOBS_HASH_VERSION constant exposed for migrations to bump on schema change', () => { // v0.35.0.0+ bumped 1→2 to fold reranker fields into the cache key. - // CDX2-F14: a timeout change from 5s to 100ms changes search behavior - // (more fail-opens) so stale cache rows must invalidate. - expect(KNOBS_HASH_VERSION).toBe(2); + // v0.35.6.0 bumped 2→3 to fold floor_ratio into the cache key + // (codex outside-voice T1 — preventing cross-floor cache contamination). + expect(KNOBS_HASH_VERSION).toBe(3); + }); + + test('T1 (codex): floor_ratio set vs unset produces DIFFERENT hashes (cache contamination prevention)', () => { + // Without this, a no-floor write would be served to a floor-enabled read + // — direct ranking-correctness leak. Same bug class CDX-4 closed in v0.32.3 + // for the other search-lite knobs. + const noFloor = knobsHash(resolveSearchMode({ mode: 'balanced' })); + const withFloor = knobsHash(resolveSearchMode({ mode: 'balanced', perCall: { floor_ratio: 0.85 } })); + expect(noFloor).not.toBe(withFloor); + }); + + test('T1 (codex): different floor_ratio values produce different hashes', () => { + // 0.85 and 0.90 are distinct cache rows. 4-decimal precision in the hash + // input means 0.85 and 0.851 also differ (consumers tuning by hundredths + // get a clean cache split). + const a = knobsHash(resolveSearchMode({ mode: 'balanced', perCall: { floor_ratio: 0.85 } })); + const b = knobsHash(resolveSearchMode({ mode: 'balanced', perCall: { floor_ratio: 0.90 } })); + expect(a).not.toBe(b); + }); + + test('same floor_ratio produces same hash (idempotent cache key)', () => { + const a = knobsHash(resolveSearchMode({ mode: 'balanced', perCall: { floor_ratio: 0.85 } })); + const b = knobsHash(resolveSearchMode({ mode: 'balanced', perCall: { floor_ratio: 0.85 } })); + expect(a).toBe(b); }); }); @@ -310,6 +339,20 @@ describe('loadOverridesFromConfig flat-map parser', () => { expect(loadOverridesFromConfig({ 'search.cache.similarity_threshold': '0' }).cache_similarity_threshold).toBeUndefined(); expect(loadOverridesFromConfig({ 'search.cache.similarity_threshold': '-0.1' }).cache_similarity_threshold).toBeUndefined(); }); + + test('v0.35.6.0: floor_ratio parses valid 0..1 values', () => { + expect(loadOverridesFromConfig({ 'search.floor_ratio': '0.85' }).floor_ratio).toBe(0.85); + expect(loadOverridesFromConfig({ 'search.floor_ratio': '0' }).floor_ratio).toBe(0); + expect(loadOverridesFromConfig({ 'search.floor_ratio': '1' }).floor_ratio).toBe(1); + expect(loadOverridesFromConfig({ 'search.floor_ratio': '0.5' }).floor_ratio).toBe(0.5); + }); + + test('v0.35.6.0: floor_ratio rejects out-of-range values silently', () => { + expect(loadOverridesFromConfig({ 'search.floor_ratio': '-0.1' }).floor_ratio).toBeUndefined(); + expect(loadOverridesFromConfig({ 'search.floor_ratio': '1.5' }).floor_ratio).toBeUndefined(); + expect(loadOverridesFromConfig({ 'search.floor_ratio': 'NaN' }).floor_ratio).toBeUndefined(); + expect(loadOverridesFromConfig({ 'search.floor_ratio': 'cheese' }).floor_ratio).toBeUndefined(); + }); }); describe('SEARCH_MODE_CONFIG_KEYS is the full reset surface', () => { diff --git a/test/search.test.ts b/test/search.test.ts index 353786475..0dd7433d8 100644 --- a/test/search.test.ts +++ b/test/search.test.ts @@ -4,7 +4,20 @@ */ import { describe, test, expect } from 'bun:test'; -import { rrfFusion, cosineSimilarity, applyBacklinkBoost } from '../src/core/search/hybrid.ts'; +import { + rrfFusion, + cosineSimilarity, + applyBacklinkBoost, + applySalienceBoost, + applyRecencyBoost, + computeFloorThreshold, + runPostFusionStages, + type PostFusionOpts, +} from '../src/core/search/hybrid.ts'; +import { + DEFAULT_RECENCY_DECAY, + DEFAULT_FALLBACK, +} from '../src/core/search/recency-decay.ts'; import type { SearchResult } from '../src/core/types.ts'; function makeResult(overrides: Partial = {}): SearchResult { @@ -239,3 +252,308 @@ describe('applyBacklinkBoost (v0.10.1)', () => { expect(results[2].score).toBeGreaterThan(results[1].score); }); }); + +/** + * v0.35.6.0 — floor-ratio gate test surface. + * + * Decisions captured in `~/.claude/plans/swift-sniffing-nygaard.md`: + * - D6=A: single up-front threshold computed at runPostFusionStages entry + * - D7=A: SearchOpts.floorRatio + search.floor_ratio config key (no env) + * - D8=B: gate scoped to metadata stages; exact-match un-gated by design + * - D9=A: global floor (cross-source); no special docs + * + * Codex outside-voice correctness fixes pinned by these tests: + * - T1: cache contamination — pinned by knobsHash coverage in search-mode.test.ts + * - T1a: NaN scores skip the gate — pinned here + * - T1b: negative top scores leave gate disabled — pinned here + * - T2: per-stage recompute is wrong — pinned by single-baseline test below + */ +describe('computeFloorThreshold', () => { + test('undefined floorRatio returns -Infinity (no gate)', () => { + const results: SearchResult[] = [makeResult({ score: 1.0 })]; + expect(computeFloorThreshold(results, undefined)).toBe(Number.NEGATIVE_INFINITY); + }); + + test('empty results array returns -Infinity even when floorRatio set', () => { + expect(computeFloorThreshold([], 0.85)).toBe(Number.NEGATIVE_INFINITY); + }); + + test('valid 0.85 + top=1.0 returns 0.85', () => { + const results: SearchResult[] = [ + makeResult({ slug: 'top', score: 1.0 }), + makeResult({ slug: 'mid', score: 0.5 }), + ]; + expect(computeFloorThreshold(results, 0.85)).toBeCloseTo(0.85, 10); + }); + + test('out-of-range floorRatio (negative) disables gate', () => { + const results: SearchResult[] = [makeResult({ score: 1.0 })]; + expect(computeFloorThreshold(results, -0.5)).toBe(Number.NEGATIVE_INFINITY); + }); + + test('out-of-range floorRatio (>1) disables gate', () => { + const results: SearchResult[] = [makeResult({ score: 1.0 })]; + expect(computeFloorThreshold(results, 1.5)).toBe(Number.NEGATIVE_INFINITY); + }); + + test('NaN floorRatio disables gate', () => { + const results: SearchResult[] = [makeResult({ score: 1.0 })]; + expect(computeFloorThreshold(results, NaN)).toBe(Number.NEGATIVE_INFINITY); + }); + + test('Infinity floorRatio disables gate', () => { + const results: SearchResult[] = [makeResult({ score: 1.0 })]; + expect(computeFloorThreshold(results, Infinity)).toBe(Number.NEGATIVE_INFINITY); + }); + + test('T1b: negative-only top score disables gate (no positive signal)', () => { + // Codex outside-voice: PR's single-result test claimed "trivially + // eligible". With negative top (-0.5), threshold = -0.425 and the top + // itself fails `r.score < threshold`. We return -Infinity instead so + // no-positive-signal inputs never gate anything. + const results: SearchResult[] = [makeResult({ score: -0.5 })]; + expect(computeFloorThreshold(results, 0.85)).toBe(Number.NEGATIVE_INFINITY); + }); + + test('T1a: all-NaN scores leave gate disabled', () => { + const results: SearchResult[] = [ + makeResult({ score: NaN }), + makeResult({ score: NaN }), + ]; + expect(computeFloorThreshold(results, 0.85)).toBe(Number.NEGATIVE_INFINITY); + }); + + test('mixed NaN + finite: top is picked from finite scores only', () => { + const results: SearchResult[] = [ + makeResult({ slug: 'nan', score: NaN }), + makeResult({ slug: 'real', score: 1.0 }), + ]; + expect(computeFloorThreshold(results, 0.85)).toBeCloseTo(0.85, 10); + }); +}); + +describe('applyBacklinkBoost — floor gate', () => { + test('floorThreshold undefined preserves prior behavior bit-for-bit', () => { + const results: SearchResult[] = [ + makeResult({ slug: 'top', score: 1.0 }), + makeResult({ slug: 'weak', score: 0.3 }), + ]; + applyBacklinkBoost(results, new Map([['top', 10], ['weak', 10]])); + const factor = 1 + 0.05 * Math.log(11); + expect(results[0].score).toBeCloseTo(1.0 * factor, 6); + expect(results[1].score).toBeCloseTo(0.3 * factor, 6); + }); + + test('weak result below threshold gets no boost', () => { + const results: SearchResult[] = [ + makeResult({ slug: 'top', score: 1.0 }), + makeResult({ slug: 'weak', score: 0.3 }), + ]; + applyBacklinkBoost(results, new Map([['top', 10], ['weak', 10]]), 0.85); + const factor = 1 + 0.05 * Math.log(11); + expect(results[0].score).toBeCloseTo(1.0 * factor, 6); + expect(results[1].score).toBe(0.3); // gated out + }); + + test('borderline result at exactly threshold is eligible', () => { + const results: SearchResult[] = [ + makeResult({ slug: 'top', score: 1.0 }), + makeResult({ slug: 'edge', score: 0.85 }), + ]; + applyBacklinkBoost(results, new Map([['top', 10], ['edge', 10]]), 0.85); + const factor = 1 + 0.05 * Math.log(11); + expect(results[1].score).toBeCloseTo(0.85 * factor, 6); + }); + + test('regression scenario: 1000-backlink weak result cannot leapfrog strong primary', () => { + const withGate: SearchResult[] = [ + makeResult({ slug: 'strong-primary', score: 1.0 }), + makeResult({ slug: 'weak-with-signal', score: 0.5 }), + ]; + applyBacklinkBoost(withGate, new Map([['weak-with-signal', 1000]]), 0.85); + withGate.sort((a, b) => b.score - a.score); + expect(withGate[0].slug).toBe('strong-primary'); + expect(withGate[1].slug).toBe('weak-with-signal'); + expect(withGate[1].score).toBe(0.5); + }); + + test('T1a regression: NaN scores skip the boost (do not pass-through)', () => { + // Codex outside-voice: `NaN < threshold` is false in JS, which would + // otherwise let NaN rows BYPASS the gate and receive boosts. NaN scores + // are skipped entirely. + const results: SearchResult[] = [ + makeResult({ slug: 'top', score: 1.0 }), + makeResult({ slug: 'nan', score: NaN }), + ]; + applyBacklinkBoost(results, new Map([['top', 10], ['nan', 10]]), 0.85); + expect(results[1].score).toBeNaN(); // unchanged + }); + + test('empty results array is a no-op', () => { + const results: SearchResult[] = []; + expect(() => applyBacklinkBoost(results, new Map(), 0.85)).not.toThrow(); + }); +}); + +describe('applySalienceBoost — floor gate', () => { + test('T6 (IRON RULE): weak result gated out (parity with backlink)', () => { + const results: SearchResult[] = [ + makeResult({ slug: 'top', score: 1.0, source_id: undefined }), + makeResult({ slug: 'weak', score: 0.3, source_id: undefined }), + ]; + const scores = new Map([ + ['default::top', 5], + ['default::weak', 5], + ]); + applySalienceBoost(results, scores, 'on', 0.85); + const factor = 1 + 0.15 * Math.log(6); + expect(results[0].score).toBeCloseTo(1.0 * factor, 6); + expect(results[1].score).toBe(0.3); // gated + }); + + test('floorThreshold undefined preserves prior behavior', () => { + const results: SearchResult[] = [makeResult({ slug: 'a', score: 0.3 })]; + applySalienceBoost(results, new Map([['default::a', 5]]), 'on'); + const factor = 1 + 0.15 * Math.log(6); + expect(results[0].score).toBeCloseTo(0.3 * factor, 6); + }); +}); + +describe('applyRecencyBoost — floor gate (T6 IRON RULE)', () => { + // Codex outside-voice + plan T6: applyRecencyBoost was the only modified + // function in the original PR with ZERO new-param test coverage. This is + // the regression test that closes the gap. + test('weak result gated out from recency boost', () => { + const now = new Date('2026-05-17').getTime(); + const yesterday = new Date(now - 86_400_000); + const results: SearchResult[] = [ + makeResult({ slug: 'top', score: 1.0, source_id: undefined }), + makeResult({ slug: 'weak', score: 0.3, source_id: undefined }), + ]; + const dates = new Map([ + ['default::top', yesterday], + ['default::weak', yesterday], + ]); + applyRecencyBoost( + results, + dates, + 'on', + DEFAULT_RECENCY_DECAY, + DEFAULT_FALLBACK, + now, + 0.85, + ); + // Top got boosted; weak unchanged at 0.3. + expect(results[0].score).toBeGreaterThan(1.0); + expect(results[1].score).toBe(0.3); + }); + + test('floorThreshold undefined preserves prior behavior', () => { + const now = new Date('2026-05-17').getTime(); + const yesterday = new Date(now - 86_400_000); + const results: SearchResult[] = [ + makeResult({ slug: 'weak', score: 0.3, source_id: undefined }), + ]; + const dates = new Map([['default::weak', yesterday]]); + applyRecencyBoost( + results, + dates, + 'on', + DEFAULT_RECENCY_DECAY, + DEFAULT_FALLBACK, + now, + ); + expect(results[0].score).toBeGreaterThan(0.3); // no gate, boost applies + }); +}); + +describe('runPostFusionStages — single-baseline composition (D6/T2)', () => { + // Build a minimal engine stub that returns predictable boost inputs. + function makeStubEngine(opts: { + backlinks?: Map; + salience?: Map; + dates?: Map; + }): { getBacklinkCounts: any; getSalienceScores: any; getEffectiveDates: any } { + return { + getBacklinkCounts: async () => opts.backlinks ?? new Map(), + getSalienceScores: async () => opts.salience ?? new Map(), + getEffectiveDates: async () => opts.dates ?? new Map(), + }; + } + + test('threshold computed ONCE at entry; same gate decision regardless of which stages fire', async () => { + // Pre-fix (per-stage recompute): backlink mutates `top`, so salience + // sees a different threshold. With single-baseline, the same threshold + // gates both stages — a result eligible for backlink is also eligible + // for salience (and vice versa), regardless of stage order. + const engine = makeStubEngine({ + backlinks: new Map([['top', 100], ['weak', 100]]), + salience: new Map([['default::top', 10], ['default::weak', 10]]), + }); + + const resultsA: SearchResult[] = [ + makeResult({ slug: 'top', score: 1.0, source_id: undefined }), + makeResult({ slug: 'weak', score: 0.3, source_id: undefined }), + ]; + const optsA: PostFusionOpts = { + applyBacklinks: true, + salience: 'on', + recency: 'off', + floorRatio: 0.85, + }; + await runPostFusionStages(engine as any, resultsA, optsA); + + // Run again with only salience enabled — same threshold should apply. + const resultsB: SearchResult[] = [ + makeResult({ slug: 'top', score: 1.0, source_id: undefined }), + makeResult({ slug: 'weak', score: 0.3, source_id: undefined }), + ]; + const optsB: PostFusionOpts = { + applyBacklinks: false, + salience: 'on', + recency: 'off', + floorRatio: 0.85, + }; + await runPostFusionStages(engine as any, resultsB, optsB); + + // In both runs, weak stayed at 0.3 (gated). Top got at least one boost. + expect(resultsA[1].score).toBe(0.3); + expect(resultsB[1].score).toBe(0.3); + expect(resultsA[0].score).toBeGreaterThan(1.0); + expect(resultsB[0].score).toBeGreaterThan(1.0); + }); + + test('floorRatio undefined: bit-for-bit prior behavior (no gate, weak gets boosted)', async () => { + const engine = makeStubEngine({ + backlinks: new Map([['weak', 1000]]), + }); + const results: SearchResult[] = [ + makeResult({ slug: 'top', score: 1.0, source_id: undefined }), + makeResult({ slug: 'weak', score: 0.3, source_id: undefined }), + ]; + const opts: PostFusionOpts = { + applyBacklinks: true, + salience: 'off', + recency: 'off', + // floorRatio intentionally omitted + }; + await runPostFusionStages(engine as any, results, opts); + expect(results[1].score).toBeGreaterThan(0.3); // weak got boosted, no gate + }); + + test('empty results: no-op, no divide-by-zero, no engine calls', async () => { + let engineCalls = 0; + const engine = { + getBacklinkCounts: async () => { engineCalls++; return new Map(); }, + getSalienceScores: async () => { engineCalls++; return new Map(); }, + getEffectiveDates: async () => { engineCalls++; return new Map(); }, + }; + await runPostFusionStages(engine as any, [], { + applyBacklinks: true, + salience: 'on', + recency: 'on', + floorRatio: 0.85, + }); + expect(engineCalls).toBe(0); + }); +}); diff --git a/test/search/knobs-hash-reranker.test.ts b/test/search/knobs-hash-reranker.test.ts index 4853a3663..f93c2189d 100644 --- a/test/search/knobs-hash-reranker.test.ts +++ b/test/search/knobs-hash-reranker.test.ts @@ -2,7 +2,8 @@ * v0.35.0.0 — knobsHash reranker-field participation tests. * * Pins: - * - KNOBS_HASH_VERSION === 2 (bumped from 1; CDX1-F14). + * - KNOBS_HASH_VERSION === 3 (bumped 1→2 v0.35.0.0 for reranker; 2→3 v0.35.6.0 + * for floor_ratio — codex outside-voice T1 cross-floor cache contamination). * - All 5 new reranker fields participate in the hash: * reranker_enabled, reranker_model, reranker_top_n_in, * reranker_top_n_out, reranker_timeout_ms. @@ -42,8 +43,8 @@ function baseKnobs(): ResolvedSearchKnobs { } describe('KNOBS_HASH_VERSION + version invariants', () => { - test('version is 2 (CDX1-F14: bumped from 1 to fold reranker fields in)', () => { - expect(KNOBS_HASH_VERSION).toBe(2); + test('version is 3 (1→2 v0.35.0.0 reranker; 2→3 v0.35.6.0 floor_ratio)', () => { + expect(KNOBS_HASH_VERSION).toBe(3); }); test('hash is 16 hex chars regardless of reranker config', () => {