mirror of
https://github.com/garrytan/gbrain.git
synced 2026-08-14 08:53:22 +00:00
Compare commits
11
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9d98f76258 | ||
|
|
6b9a1f9f9f | ||
|
|
52a61b4325 | ||
|
|
357be400f7 | ||
|
|
778c118015 | ||
|
|
0b25a8e548 | ||
|
|
17d88027d2 | ||
|
|
b38d3a071b | ||
|
|
78d2d3dc18 | ||
|
|
4dfcd75599 | ||
|
|
056de13bcd |
@@ -2,6 +2,23 @@
|
||||
|
||||
All notable changes to GBrain will be documented in this file.
|
||||
|
||||
## [0.42.34.0] - 2026-06-07
|
||||
|
||||
**Relationship questions now get relationship answers.** Ask "who invested in widget-co", "who introduced me to alice-example", or "what connects fund-a and fund-b" and gbrain resolves the named entity and walks its typed-edge graph (`invested_in`, `works_at`, `founded`, `attended`, `advises`, …) to surface the answer — even when no single page mentions both sides. Until now the graph only re-ranked results that keyword/vector search had already found; a relationship that lived purely in the edges (an investor whose page never names the company) was invisible. It now enters retrieval as a first-class candidate.
|
||||
|
||||
This is on by default in the `balanced` and `tokenmax` search modes, a pure no-op for non-relational questions and for brains with no typed edges, and off in `conservative`. On a benchmark of relationship queries whose answers are unreachable by content similarity, recall@10 goes from near-zero to over 75%. The traversal is deterministic (same query + brain → same answer), stays within a single source (it never crosses a mounted-brain boundary), excludes noisy body-text "mentions" edges by default, and is depth- and fan-out-bounded so a popular hub entity can't blow up a query.
|
||||
|
||||
### Added
|
||||
- **Typed-edge relational retrieval** (`search.relational_retrieval`, on for balanced/tokenmax). Relational questions resolve their seed entity and traverse the typed-edge graph, injecting edge-derived answers as a fourth fusion arm alongside keyword + vector. Relation vocabulary is schema-pack-extensible: a pack that defines its own link types can declare the query phrases that retrieve them. The `query` operation gains a `relational` flag (omit for the smart default; pass `false` to force lexical/vector-only). Results carry `--explain` attribution ("surfaced via invested_in from widget-co") and, for "what connects A and B", the connecting path.
|
||||
- **`relationalFanout` engine method** (PGLite + Postgres, in lockstep). Seed-array typed-edge fan-out aggregating to ranked nodes (shortest hop, edge richness, connecting path, canonical chunk), source-scoped and deterministic.
|
||||
- **`gbrain eval retrieval-quality --ab-relational`**: A/Bs the arm off vs on over a question set and reports the recall@10 lift + latency. The retrieval-quality harness gains recall@k / recall@10 metrics.
|
||||
|
||||
### Fixed
|
||||
- **Cross-source result collapse.** The search fusion/dedup key now carries `source_id`, so two pages that share a slug across mounted brains no longer merge into one result. The semantic query cache is likewise scoped per source-set, so a federated search can't be served a single-source cached result.
|
||||
|
||||
### To take advantage of v0.42.34.0
|
||||
- Just ask relationship questions in natural language — the arm is on by default in balanced/tokenmax. To turn it off: `gbrain config set search.relational_retrieval false`.
|
||||
- One-time cache note: this release advances the search-cache key version (a relational-on result must not be served to a relational-off lookup), so the first query after upgrade re-runs instead of hitting a stale cache row. No action needed; it self-heals on first use.
|
||||
## [0.42.33.0] - 2026-06-07
|
||||
|
||||
**`gbrain sync` will never delete a repo it didn't create.** If a source was registered with a `remote_url` but its `local_path` pointed at a working tree you manage yourself (not a gbrain-managed clone), a failed or degraded code sync could remove that directory and re-clone over it. Sync now re-clones **only** clones gbrain actually created — identified by an ownership marker, or by gbrain's own clone location for clones made before this release. Anything else, including your live working tree, is treated as read-only: indexed, never deleted. On an unowned path, sync aborts loudly **before touching the filesystem** and tells you how to fix the source registration. Thanks to @zaqwery for the report.
|
||||
|
||||
@@ -149,6 +149,7 @@ project resolves through `src/core/search/mode.ts`.
|
||||
| `intentWeighting` | true | true | true |
|
||||
| `tokenBudget` | **4000** | **12000** | **off** |
|
||||
| `expansion` (LLM multi-query) | false | false | **true** |
|
||||
| `relationalRetrieval` | false | **true** | **true** |
|
||||
| `searchLimit` default | 10 | 25 | 50 |
|
||||
|
||||
**Cost anchors (downstream agent input cost — gbrain itself is rounding error).**
|
||||
@@ -207,6 +208,19 @@ written against `embedding` (1536d OpenAI). Existing v=2 rows become
|
||||
unreachable on first re-query (one-time miss spike on upgrade);
|
||||
`mode.ts:KNOBS_HASH_VERSION` is the single source of truth.
|
||||
|
||||
**v0.42.34.0 knobs_hash v=9 → v=10.** Folds the `relationalRetrieval` knob +
|
||||
depth into the cache key so a relational-on result set can't be served to a
|
||||
relational-off lookup (same contamination class as graph_signals). One-time
|
||||
miss spike on upgrade.
|
||||
|
||||
**Relational retrieval (v0.42.34.0).** `relationalRetrieval` (on for
|
||||
balanced/tokenmax) adds a fourth recall arm: a relational query ("who invested
|
||||
in X", "what connects A and B") resolves its seed entity and walks the typed-edge
|
||||
graph (`src/core/search/relational-recall.ts` + `relational-intent.ts`,
|
||||
`engine.relationalFanout`), injecting edge-derived answers into RRF. Within-source,
|
||||
deterministic, mentions-excluded by default, pure no-op for non-relational queries.
|
||||
The `query` op's `relational` flag forces it on/off per call.
|
||||
|
||||
**Three CLI surfaces:**
|
||||
|
||||
gbrain search modes # what is running, with per-knob attribution
|
||||
|
||||
@@ -123,6 +123,7 @@ expansion (if enabled)
|
||||
hybrid search:
|
||||
├── vector (HNSW on chunk embeddings)
|
||||
├── keyword (BM25 via tsvector)
|
||||
├── relational (v0.42.34.0: typed-edge recall arm — relational queries only)
|
||||
├── source-aware re-rank (CASE in SQL)
|
||||
└── RRF fusion → top 30
|
||||
│
|
||||
|
||||
@@ -297,6 +297,7 @@ project resolves through `src/core/search/mode.ts`.
|
||||
| `intentWeighting` | true | true | true |
|
||||
| `tokenBudget` | **4000** | **12000** | **off** |
|
||||
| `expansion` (LLM multi-query) | false | false | **true** |
|
||||
| `relationalRetrieval` | false | **true** | **true** |
|
||||
| `searchLimit` default | 10 | 25 | 50 |
|
||||
|
||||
**Cost anchors (downstream agent input cost — gbrain itself is rounding error).**
|
||||
@@ -355,6 +356,19 @@ written against `embedding` (1536d OpenAI). Existing v=2 rows become
|
||||
unreachable on first re-query (one-time miss spike on upgrade);
|
||||
`mode.ts:KNOBS_HASH_VERSION` is the single source of truth.
|
||||
|
||||
**v0.42.34.0 knobs_hash v=9 → v=10.** Folds the `relationalRetrieval` knob +
|
||||
depth into the cache key so a relational-on result set can't be served to a
|
||||
relational-off lookup (same contamination class as graph_signals). One-time
|
||||
miss spike on upgrade.
|
||||
|
||||
**Relational retrieval (v0.42.34.0).** `relationalRetrieval` (on for
|
||||
balanced/tokenmax) adds a fourth recall arm: a relational query ("who invested
|
||||
in X", "what connects A and B") resolves its seed entity and walks the typed-edge
|
||||
graph (`src/core/search/relational-recall.ts` + `relational-intent.ts`,
|
||||
`engine.relationalFanout`), injecting edge-derived answers into RRF. Within-source,
|
||||
deterministic, mentions-excluded by default, pure no-op for non-relational queries.
|
||||
The `query` op's `relational` flag forces it on/off per call.
|
||||
|
||||
**Three CLI surfaces:**
|
||||
|
||||
gbrain search modes # what is running, with per-knob attribution
|
||||
|
||||
+1
-1
@@ -143,5 +143,5 @@
|
||||
"bun": ">=1.3.10"
|
||||
},
|
||||
"license": "MIT",
|
||||
"version": "0.42.33.0"
|
||||
"version": "0.42.34.0"
|
||||
}
|
||||
|
||||
@@ -20,12 +20,13 @@ import {
|
||||
|
||||
export async function runEvalRetrievalQuality(engine: BrainEngine, args: string[]): Promise<void> {
|
||||
const json = args.includes('--json');
|
||||
const abRelational = args.includes('--ab-relational');
|
||||
const sourceIdx = args.indexOf('--source');
|
||||
const sourceId = sourceIdx >= 0 ? args[sourceIdx + 1] : undefined;
|
||||
const fixture = args.find(a => !a.startsWith('--') && a !== sourceId);
|
||||
|
||||
if (!fixture) {
|
||||
console.error('Usage: gbrain eval retrieval-quality <fixture.jsonl> [--json] [--source <id>]');
|
||||
console.error('Usage: gbrain eval retrieval-quality <fixture.jsonl> [--json] [--source <id>] [--ab-relational]');
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
@@ -47,6 +48,48 @@ export async function runEvalRetrievalQuality(engine: BrainEngine, args: string[
|
||||
return results.map(r => r.slug);
|
||||
};
|
||||
|
||||
// v0.43 — A/B the relational recall arm (off vs on) over the same questions
|
||||
// in a fixed mode (expansion off, skipCache-equivalent via bare hybridSearch)
|
||||
// so the delta is purely the arm. Headline: graph-relationship recall@10 lift.
|
||||
if (abRelational) {
|
||||
const mk = (relationalRetrieval: boolean): SearchFn => async (q) => {
|
||||
const results = await hybridSearch(engine, q, {
|
||||
limit: 10, relationalRetrieval, expansion: false,
|
||||
...(sourceId ? { sourceId } : {}),
|
||||
});
|
||||
return results.map(r => r.slug);
|
||||
};
|
||||
const t0 = Date.now();
|
||||
const off = await runRetrievalQuality(questions, mk(false));
|
||||
const tMid = Date.now();
|
||||
const on = await runRetrievalQuality(questions, mk(true));
|
||||
const tEnd = Date.now();
|
||||
const fam = (r: typeof off, f: string) => r.families.find(x => x.family === f);
|
||||
const rel = { off: fam(off, 'graph-relationship'), on: fam(on, 'graph-relationship') };
|
||||
const payload = {
|
||||
schema_version: 1 as const,
|
||||
ab: 'relational',
|
||||
graph_relationship: {
|
||||
n: rel.on?.n ?? 0,
|
||||
recall_at_10: { off: rel.off?.recall_at_10 ?? 0, on: rel.on?.recall_at_10 ?? 0,
|
||||
delta: (rel.on?.recall_at_10 ?? 0) - (rel.off?.recall_at_10 ?? 0) },
|
||||
hit_at_3: { off: rel.off?.hit_at_3 ?? 0, on: rel.on?.hit_at_3 ?? 0 },
|
||||
},
|
||||
latency_ms: { off_total: tMid - t0, on_total: tEnd - tMid },
|
||||
families: { off: off.families, on: on.families },
|
||||
};
|
||||
if (json) {
|
||||
console.log(JSON.stringify(payload, null, 2));
|
||||
} else {
|
||||
console.log(`Relational A/B — graph-relationship n=${payload.graph_relationship.n}`);
|
||||
const g = payload.graph_relationship;
|
||||
console.log(` recall@10: off=${(g.recall_at_10.off * 100).toFixed(0)}% on=${(g.recall_at_10.on * 100).toFixed(0)}% Δ=+${(g.recall_at_10.delta * 100).toFixed(0)}pp`);
|
||||
console.log(` Hit@3: off=${(g.hit_at_3.off * 100).toFixed(0)}% on=${(g.hit_at_3.on * 100).toFixed(0)}%`);
|
||||
console.log(` latency: off=${payload.latency_ms.off_total}ms on=${payload.latency_ms.on_total}ms (arm adds ${payload.latency_ms.on_total - payload.latency_ms.off_total}ms over ${payload.graph_relationship.n} queries)`);
|
||||
}
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
const report = await runRetrievalQuality(questions, searchFn);
|
||||
const gate = evaluateGate(report);
|
||||
|
||||
|
||||
@@ -71,6 +71,9 @@ const KNOB_DESCRIPTIONS: Record<keyof ModeBundle, string> = {
|
||||
// v0.42.3.0 autocut
|
||||
autocut: 'Score-discontinuity result-sizing (cuts at the rerank-score cliff; no-op without a reranker)',
|
||||
autocut_jump: 'Autocut sensitivity: min normalized score gap that counts as a cliff (0..1, 0.20 default)',
|
||||
// v0.43 relational recall
|
||||
relationalRetrieval: 'Typed-edge relational recall arm (relational queries walk the graph; no-op otherwise)',
|
||||
relational_retrieval_depth: 'Max hops for relational traversal (1..3, 2 default)',
|
||||
};
|
||||
|
||||
interface SearchModesReport {
|
||||
|
||||
+25
-1
@@ -2,7 +2,7 @@ import type {
|
||||
Page, PageInput, PageFilters, GetPageOpts,
|
||||
Chunk, ChunkInput, StaleChunkRow, StalePageRow,
|
||||
SearchResult, SearchOpts,
|
||||
Link, GraphNode, GraphPath,
|
||||
Link, GraphNode, GraphPath, RelationalFanoutRow, RelationalFanoutOpts,
|
||||
TimelineEntry, TimelineInput, TimelineOpts,
|
||||
RawData,
|
||||
PageVersion,
|
||||
@@ -1197,6 +1197,30 @@ export interface BrainEngine {
|
||||
slug: string,
|
||||
opts?: { depth?: number; linkType?: string; direction?: 'in' | 'out' | 'both'; sourceId?: string; sourceIds?: string[] },
|
||||
): Promise<GraphPath[]>;
|
||||
/**
|
||||
* Typed-edge relational fan-out for the relational recall arm (v0.43).
|
||||
*
|
||||
* Generalizes traversePaths to a SEED ARRAY and aggregates to ranked NODES
|
||||
* (not edges): for each reached page, the shortest hop from any seed, a
|
||||
* connection-richness count, the edge types it was reached by, the shortest
|
||||
* connecting slug path, and the page's canonical (lowest-ordinal) chunk id.
|
||||
*
|
||||
* Invariants:
|
||||
* - WITHIN-SOURCE: a walk never crosses a source boundary (each branch
|
||||
* stays in its seed's own source), even when multiple sources are in
|
||||
* scope. Cross-source edge traversal is a separate, security-reviewed
|
||||
* feature.
|
||||
* - `link_source='mentions'` edges are EXCLUDED by default (noisy + the
|
||||
* densest fan-out source); opt in via `includeMentions`.
|
||||
* - `deleted_at IS NULL` at seed, every neighbor, and every returned node.
|
||||
* - Deterministic ordering: hop ASC, edge_count DESC, source_id, slug.
|
||||
* Must move in lockstep with the PGLite implementation
|
||||
* (test/e2e/engine-parity.test.ts).
|
||||
*/
|
||||
relationalFanout(
|
||||
seeds: string[],
|
||||
opts?: RelationalFanoutOpts,
|
||||
): Promise<RelationalFanoutRow[]>;
|
||||
/**
|
||||
* For a list of slugs, return how many inbound links each has.
|
||||
* Used by hybrid search backlink boost. Single SQL query, not N+1.
|
||||
|
||||
@@ -1431,6 +1431,11 @@ const query: Operation = {
|
||||
" TRUE is redundant in default mode (it's already on); it only matters to override a brain whose config turned autocut off.\n" +
|
||||
"Safe by construction: never returns empty when there are matches, only applies to the first page (omit when paginating), and is a no-op when no reranker scored the results (so it can't cut on an untrustworthy signal). Distinct from `adaptive_return`: autocut cuts on the score cliff; adaptive_return caps by question intent. Leave both unset for the smart default.",
|
||||
},
|
||||
relational: {
|
||||
type: 'boolean',
|
||||
description:
|
||||
"v0.43 — relational recall arm. SMART DEFAULT (on in balanced/tokenmax). When the question is about a RELATIONSHIP ('who invested in widget-co', 'who introduced me to alice', 'what connects fund-a and fund-b'), the brain resolves the named entity and walks its typed-edge graph (invested_in, works_at, founded, …), surfacing the answer even when no passage mentions both sides. Pure no-op for non-relational questions. Pass FALSE to force lexical/vector-only retrieval (e.g. debugging why a graph answer appeared). You almost never set this.",
|
||||
},
|
||||
},
|
||||
handler: async (ctx, p) => {
|
||||
const startedAt = Date.now();
|
||||
@@ -1528,6 +1533,8 @@ const query: Operation = {
|
||||
// v0.42.3.0 — autocut ceiling override. Omitted = smart default (ON in
|
||||
// reranked modes). `false` forces the full top-K.
|
||||
autocut: typeof p.autocut === 'boolean' ? (p.autocut as boolean) : undefined,
|
||||
// v0.43 — relational recall override. Omitted = smart default (mode bundle).
|
||||
relationalRetrieval: typeof p.relational === 'boolean' ? (p.relational as boolean) : undefined,
|
||||
});
|
||||
const latency_ms = Date.now() - startedAt;
|
||||
|
||||
|
||||
@@ -2859,6 +2859,90 @@ export class PGLiteEngine implements BrainEngine {
|
||||
return result;
|
||||
}
|
||||
|
||||
async relationalFanout(
|
||||
seeds: string[],
|
||||
opts?: import('./types.ts').RelationalFanoutOpts,
|
||||
): Promise<import('./types.ts').RelationalFanoutRow[]> {
|
||||
if (!seeds || seeds.length === 0) return [];
|
||||
const depth = Math.min(Math.max(1, opts?.depth ?? 2), 3);
|
||||
const direction = opts?.direction ?? 'both';
|
||||
const limit = Math.min(Math.max(1, opts?.limit ?? 50), 200);
|
||||
const types = opts?.linkTypes && opts.linkTypes.length > 0 ? opts.linkTypes : null;
|
||||
|
||||
// $1=seeds, $2=depth, $3=limit; optional scope/type params appended.
|
||||
const params: unknown[] = [seeds, depth, limit];
|
||||
const useSourceIds = opts?.sourceIds && opts.sourceIds.length > 0;
|
||||
let seedScope = '';
|
||||
if (useSourceIds) {
|
||||
params.push(opts!.sourceIds);
|
||||
seedScope = `AND p.source_id = ANY($${params.length}::text[])`;
|
||||
} else if (opts?.sourceId) {
|
||||
params.push(opts.sourceId);
|
||||
seedScope = `AND p.source_id = $${params.length}`;
|
||||
}
|
||||
let typeFilter = '';
|
||||
if (types) {
|
||||
params.push(types);
|
||||
typeFilter = `AND l.link_type = ANY($${params.length}::text[])`;
|
||||
}
|
||||
const mentionsFilter = opts?.includeMentions ? '' : `AND l.link_source IS DISTINCT FROM 'mentions'`;
|
||||
|
||||
const recurStep =
|
||||
direction === 'out'
|
||||
? `JOIN links l ON l.from_page_id = w.id JOIN pages p2 ON p2.id = l.to_page_id`
|
||||
: direction === 'in'
|
||||
? `JOIN links l ON l.to_page_id = w.id JOIN pages p2 ON p2.id = l.from_page_id`
|
||||
: `JOIN links l ON (l.from_page_id = w.id OR l.to_page_id = w.id)
|
||||
JOIN pages p2 ON p2.id = CASE WHEN l.from_page_id = w.id THEN l.to_page_id ELSE l.from_page_id END`;
|
||||
|
||||
const sql = `
|
||||
WITH RECURSIVE walk AS (
|
||||
SELECT p.id, p.slug, p.source_id, 0::int AS depth,
|
||||
ARRAY[p.id] AS visited, ARRAY[p.slug] AS path,
|
||||
p.source_id AS seed_source, NULL::text AS last_link_type
|
||||
FROM pages p
|
||||
WHERE p.slug = ANY($1::text[]) ${seedScope} AND p.deleted_at IS NULL
|
||||
UNION ALL
|
||||
SELECT p2.id, p2.slug, p2.source_id, w.depth + 1,
|
||||
w.visited || p2.id, w.path || p2.slug,
|
||||
w.seed_source, l.link_type
|
||||
FROM walk w
|
||||
${recurStep}
|
||||
WHERE w.depth < $2
|
||||
AND NOT (p2.id = ANY(w.visited))
|
||||
AND p2.source_id = w.seed_source
|
||||
AND p2.deleted_at IS NULL
|
||||
${mentionsFilter}
|
||||
${typeFilter}
|
||||
)
|
||||
SELECT n.source_id, n.slug,
|
||||
MIN(n.depth) AS hop,
|
||||
COUNT(DISTINCT n.last_link_type) AS edge_count,
|
||||
array_agg(DISTINCT n.last_link_type)
|
||||
FILTER (WHERE n.last_link_type IS NOT NULL) AS via_link_types,
|
||||
(array_agg(array_to_string(n.path, chr(9))
|
||||
ORDER BY n.depth ASC, array_length(n.path, 1) ASC))[1] AS path_str,
|
||||
(SELECT cc.id FROM content_chunks cc
|
||||
WHERE cc.page_id = n.id ORDER BY cc.chunk_index ASC LIMIT 1) AS canonical_chunk_id
|
||||
FROM walk n
|
||||
WHERE n.depth > 0
|
||||
GROUP BY n.source_id, n.slug, n.id
|
||||
ORDER BY hop ASC, edge_count DESC, n.source_id ASC, n.slug ASC
|
||||
LIMIT $3
|
||||
`;
|
||||
|
||||
const { rows } = await this.db.query(sql, params);
|
||||
return (rows as Record<string, unknown>[]).map(r => ({
|
||||
source_id: r.source_id as string,
|
||||
slug: r.slug as string,
|
||||
hop: Number(r.hop),
|
||||
edge_count: Number(r.edge_count),
|
||||
via_link_types: Array.isArray(r.via_link_types) ? (r.via_link_types as string[]) : [],
|
||||
path: r.path_str ? String(r.path_str).split('\t') : [],
|
||||
canonical_chunk_id: r.canonical_chunk_id == null ? null : Number(r.canonical_chunk_id),
|
||||
}));
|
||||
}
|
||||
|
||||
async getBacklinkCounts(slugs: string[]): Promise<Map<string, number>> {
|
||||
const result = new Map<string, number>();
|
||||
if (slugs.length === 0) return result;
|
||||
|
||||
@@ -2950,6 +2950,88 @@ export class PostgresEngine implements BrainEngine {
|
||||
return result;
|
||||
}
|
||||
|
||||
async relationalFanout(
|
||||
seeds: string[],
|
||||
opts?: import('./types.ts').RelationalFanoutOpts,
|
||||
): Promise<import('./types.ts').RelationalFanoutRow[]> {
|
||||
if (!seeds || seeds.length === 0) return [];
|
||||
const sql = this.sql;
|
||||
const depth = Math.min(Math.max(1, opts?.depth ?? 2), 3);
|
||||
const direction = opts?.direction ?? 'both';
|
||||
const limit = Math.min(Math.max(1, opts?.limit ?? 50), 200);
|
||||
const types = opts?.linkTypes && opts.linkTypes.length > 0 ? opts.linkTypes : null;
|
||||
|
||||
// Scope is applied to SEED selection only. Within-source traversal is
|
||||
// enforced separately by `p2.source_id = w.seed_source` in the recursive
|
||||
// step, so a walk can never cross a source boundary even when several
|
||||
// sources are in scope.
|
||||
const useSourceIds = opts?.sourceIds && opts.sourceIds.length > 0;
|
||||
const seedScope = useSourceIds
|
||||
? sql`AND p.source_id = ANY(${opts!.sourceIds!}::text[])`
|
||||
: opts?.sourceId
|
||||
? sql`AND p.source_id = ${opts.sourceId}`
|
||||
: sql``;
|
||||
const typeFilter = types ? sql`AND l.link_type = ANY(${types}::text[])` : sql``;
|
||||
const mentionsFilter = opts?.includeMentions
|
||||
? sql``
|
||||
: sql`AND l.link_source IS DISTINCT FROM 'mentions'`;
|
||||
|
||||
// Recursive step join differs by direction; everything else is shared.
|
||||
const recurStep =
|
||||
direction === 'out'
|
||||
? sql`JOIN links l ON l.from_page_id = w.id JOIN pages p2 ON p2.id = l.to_page_id`
|
||||
: direction === 'in'
|
||||
? sql`JOIN links l ON l.to_page_id = w.id JOIN pages p2 ON p2.id = l.from_page_id`
|
||||
: sql`JOIN links l ON (l.from_page_id = w.id OR l.to_page_id = w.id)
|
||||
JOIN pages p2 ON p2.id = CASE WHEN l.from_page_id = w.id THEN l.to_page_id ELSE l.from_page_id END`;
|
||||
|
||||
const rows = await sql`
|
||||
WITH RECURSIVE walk AS (
|
||||
SELECT p.id, p.slug, p.source_id, 0::int AS depth,
|
||||
ARRAY[p.id] AS visited, ARRAY[p.slug] AS path,
|
||||
p.source_id AS seed_source, NULL::text AS last_link_type
|
||||
FROM pages p
|
||||
WHERE p.slug = ANY(${seeds}::text[]) ${seedScope} AND p.deleted_at IS NULL
|
||||
UNION ALL
|
||||
SELECT p2.id, p2.slug, p2.source_id, w.depth + 1,
|
||||
w.visited || p2.id, w.path || p2.slug,
|
||||
w.seed_source, l.link_type
|
||||
FROM walk w
|
||||
${recurStep}
|
||||
WHERE w.depth < ${depth}
|
||||
AND NOT (p2.id = ANY(w.visited))
|
||||
AND p2.source_id = w.seed_source
|
||||
AND p2.deleted_at IS NULL
|
||||
${mentionsFilter}
|
||||
${typeFilter}
|
||||
)
|
||||
SELECT n.source_id, n.slug,
|
||||
MIN(n.depth) AS hop,
|
||||
COUNT(DISTINCT n.last_link_type) AS edge_count,
|
||||
array_agg(DISTINCT n.last_link_type)
|
||||
FILTER (WHERE n.last_link_type IS NOT NULL) AS via_link_types,
|
||||
(array_agg(array_to_string(n.path, chr(9))
|
||||
ORDER BY n.depth ASC, array_length(n.path, 1) ASC))[1] AS path_str,
|
||||
(SELECT cc.id FROM content_chunks cc
|
||||
WHERE cc.page_id = n.id ORDER BY cc.chunk_index ASC LIMIT 1) AS canonical_chunk_id
|
||||
FROM walk n
|
||||
WHERE n.depth > 0
|
||||
GROUP BY n.source_id, n.slug, n.id
|
||||
ORDER BY hop ASC, edge_count DESC, n.source_id ASC, n.slug ASC
|
||||
LIMIT ${limit}
|
||||
`;
|
||||
|
||||
return (rows as Record<string, unknown>[]).map(r => ({
|
||||
source_id: r.source_id as string,
|
||||
slug: r.slug as string,
|
||||
hop: Number(r.hop),
|
||||
edge_count: Number(r.edge_count),
|
||||
via_link_types: Array.isArray(r.via_link_types) ? (r.via_link_types as string[]) : [],
|
||||
path: r.path_str ? String(r.path_str).split('\t') : [],
|
||||
canonical_chunk_id: r.canonical_chunk_id == null ? null : Number(r.canonical_chunk_id),
|
||||
}));
|
||||
}
|
||||
|
||||
async getBacklinkCounts(slugs: string[]): Promise<Map<string, number>> {
|
||||
const result = new Map<string, number>();
|
||||
if (slugs.length === 0) return result;
|
||||
|
||||
+110
-12
@@ -23,6 +23,7 @@ import {
|
||||
type AdaptiveReturnDecision,
|
||||
} from './return-policy.ts';
|
||||
import { applyAutocut, type AutocutDecision } from './autocut.ts';
|
||||
import { buildRelationalArm } from './relational-recall.ts';
|
||||
import { loadConfigWithEngine } from '../config.ts';
|
||||
import { dedupResults } from './dedup.ts';
|
||||
import { applyReranker } from './rerank.ts';
|
||||
@@ -664,6 +665,9 @@ export async function applyAliasHop(
|
||||
|
||||
export interface HybridSearchOpts extends SearchOpts {
|
||||
expansion?: boolean;
|
||||
/** v0.43 — observability sink for the relational recall arm (fired/no-op,
|
||||
* kind, seeds resolved, candidates, errored). Best-effort. */
|
||||
onRelationalMeta?: (meta: import('./relational-recall.ts').RelationalArmMeta) => void;
|
||||
/**
|
||||
* T4/D5 — per-call search-mode selector (one of SEARCH_MODES). Selects the
|
||||
* whole mode bundle for this call, overriding the server-configured mode.
|
||||
@@ -806,6 +810,11 @@ export async function hybridSearch(
|
||||
// Non-boolean AutocutInput shapes (Partial) aren't a v1 per-call surface,
|
||||
// so only the boolean toggle threads here.
|
||||
autocut: typeof opts?.autocut === 'boolean' ? opts.autocut : undefined,
|
||||
// v0.43 — relational recall per-call thread-through. Per-call wins over
|
||||
// config override wins over mode bundle; without this the A/B eval gate
|
||||
// would be a no-op (both branches resolve to the same mode default).
|
||||
relationalRetrieval: opts?.relationalRetrieval,
|
||||
relational_retrieval_depth: opts?.relationalRetrievalDepth,
|
||||
},
|
||||
});
|
||||
|
||||
@@ -972,6 +981,23 @@ export async function hybridSearch(
|
||||
titleBoost: resolvedMode.title_boost,
|
||||
};
|
||||
|
||||
// v0.43 — build the relational recall arm ONCE here, before any return
|
||||
// path, so typed-edge answers contribute on ALL THREE paths: the
|
||||
// no-embedding-provider path, the embed-failed keyword fallback, and the
|
||||
// main RRF path. Parsed from the original query (deterministic); empty for
|
||||
// non-relational queries → pure no-op. (Modality gate lives on the main
|
||||
// path; the parser only matches text-shaped relational queries anyway.)
|
||||
let relationalList: SearchResult[] = [];
|
||||
if (resolvedMode.relationalRetrieval) {
|
||||
relationalList = await buildRelationalArm(engine, query, {
|
||||
sourceId: opts?.sourceId,
|
||||
sourceIds: opts?.sourceIds,
|
||||
depth: resolvedMode.relational_retrieval_depth,
|
||||
limit: opts?.limit ?? resolvedMode.searchLimit,
|
||||
onMeta: opts?.onRelationalMeta,
|
||||
});
|
||||
}
|
||||
|
||||
// Skip vector search entirely if the gateway has no embedding provider configured (Codex C3).
|
||||
// v0.36 (D10): ask "is the RESOLVED column's provider reachable?" rather
|
||||
// than "is the global default reachable?" — otherwise an unreachable
|
||||
@@ -980,13 +1006,24 @@ export async function hybridSearch(
|
||||
const { isAvailable } = await import('../ai/gateway.ts');
|
||||
const providerProbe = resolvedCol.embeddingModel || undefined;
|
||||
if (!isAvailable('embedding', providerProbe)) {
|
||||
if (keywordResults.length > 0) {
|
||||
await runPostFusionStages(engine, keywordResults, postFusionOpts);
|
||||
keywordResults.sort((a, b) => b.score - a.score);
|
||||
// v0.43 — fuse the relational arm with keyword so typed-edge answers
|
||||
// survive on the no-embedding-provider path (the relational win is most
|
||||
// valuable exactly when vector is unavailable).
|
||||
let noEmbedResults = keywordResults;
|
||||
if (relationalList.length > 0) {
|
||||
const fk = opts?.rrfK ?? RRF_K;
|
||||
noEmbedResults = rrfFusionWeighted(
|
||||
[{ list: keywordResults, k: fk }, { list: relationalList, k: fk }],
|
||||
detailResolved !== 'high',
|
||||
);
|
||||
}
|
||||
if (noEmbedResults.length > 0) {
|
||||
await runPostFusionStages(engine, noEmbedResults, postFusionOpts);
|
||||
noEmbedResults.sort((a, b) => b.score - a.score);
|
||||
}
|
||||
// T3/T4 — alias hop + evidence stamp even without an embedding provider
|
||||
// (the named-thing fix is most valuable exactly when vector is unavailable).
|
||||
const noEmbedHopped = await applyAliasHop(engine, dedupResults(keywordResults), query, {
|
||||
const noEmbedHopped = await applyAliasHop(engine, dedupResults(noEmbedResults), query, {
|
||||
sourceId: opts?.sourceId,
|
||||
sourceIds: opts?.sourceIds,
|
||||
});
|
||||
@@ -1204,11 +1241,21 @@ export async function hybridSearch(
|
||||
// v0.29.1 codex pass-2 #4: this is the third return path. Apply
|
||||
// post-fusion stages here too — without it, salience='on' silently
|
||||
// does nothing on embed failures.
|
||||
if (keywordResults.length > 0) {
|
||||
await runPostFusionStages(engine, keywordResults, postFusionOpts);
|
||||
keywordResults.sort((a, b) => b.score - a.score);
|
||||
// v0.43: fuse the relational arm with keyword via RRF so typed-edge
|
||||
// answers survive even when vector is unavailable.
|
||||
let fallbackResults = keywordResults;
|
||||
if (relationalList.length > 0) {
|
||||
const fk = opts?.rrfK ?? RRF_K;
|
||||
fallbackResults = rrfFusionWeighted(
|
||||
[{ list: keywordResults, k: fk }, { list: relationalList, k: fk }],
|
||||
detail !== 'high',
|
||||
);
|
||||
}
|
||||
const kwHopped = await applyAliasHop(engine, dedupResults(keywordResults), query, {
|
||||
if (fallbackResults.length > 0) {
|
||||
await runPostFusionStages(engine, fallbackResults, postFusionOpts);
|
||||
fallbackResults.sort((a, b) => b.score - a.score);
|
||||
}
|
||||
const kwHopped = await applyAliasHop(engine, dedupResults(fallbackResults), query, {
|
||||
sourceId: opts?.sourceId,
|
||||
sourceIds: opts?.sourceIds,
|
||||
});
|
||||
@@ -1266,6 +1313,16 @@ export async function hybridSearch(
|
||||
...vectorLists.map(list => ({ list, k: vectorK })),
|
||||
{ list: keywordResults, k: keywordK },
|
||||
];
|
||||
|
||||
// v0.43 — relational recall arm (fourth RRF arm), built above so it also
|
||||
// contributes on the keyword-only fallback path. Neutral weight (baseRrfK):
|
||||
// competes evenly with keyword/vector, not dominating. Empty for
|
||||
// non-relational queries → pure no-op. Rides every downstream stage (cosine
|
||||
// re-score, post-fusion boosts, dedup, reranker, autocut, token budget).
|
||||
if (relationalList.length > 0 && effectiveModality !== 'image') {
|
||||
allLists.push({ list: relationalList, k: baseRrfK });
|
||||
}
|
||||
|
||||
let fused = rrfFusionWeighted(allLists, detail !== 'high');
|
||||
|
||||
// Cosine re-scoring before dedup so semantically better chunks survive.
|
||||
@@ -1511,6 +1568,11 @@ export async function hybridSearchCached(
|
||||
// this, an `autocut:false` (full top-K) call could be served a trimmed
|
||||
// autocut-on cache row, or vice versa.
|
||||
autocut: typeof opts?.autocut === 'boolean' ? opts.autocut : undefined,
|
||||
// v0.43 — relational recall per-call thread-through. Per-call wins over
|
||||
// config override wins over mode bundle; without this the A/B eval gate
|
||||
// would be a no-op (both branches resolve to the same mode default).
|
||||
relationalRetrieval: opts?.relationalRetrieval,
|
||||
relational_retrieval_depth: opts?.relationalRetrievalDepth,
|
||||
},
|
||||
});
|
||||
// v0.36 (D8 / CDX-2 + codex /ship #4): resolve column for the cache
|
||||
@@ -1607,7 +1669,7 @@ export async function hybridSearchCached(
|
||||
}
|
||||
|
||||
if (!skipCache && queryEmbedding && cacheStatus !== 'disabled') {
|
||||
const hit = await cache.lookup(queryEmbedding, { sourceId: opts?.sourceId, knobsHash: cacheKnobsHash });
|
||||
const hit = await cache.lookup(queryEmbedding, { sourceId: cacheScopeKey(opts), knobsHash: cacheKnobsHash });
|
||||
if (hit.hit && hit.results) {
|
||||
cacheStatus = 'hit';
|
||||
cacheSimilarity = hit.similarity;
|
||||
@@ -1708,7 +1770,7 @@ export async function hybridSearchCached(
|
||||
) {
|
||||
trackCacheWrite(
|
||||
cache
|
||||
.store(query, queryEmbedding, results, finalMeta, { sourceId: opts?.sourceId, knobsHash: cacheKnobsHash })
|
||||
.store(query, queryEmbedding, results, finalMeta, { sourceId: cacheScopeKey(opts), knobsHash: cacheKnobsHash })
|
||||
.catch(() => { /* swallow */ }),
|
||||
);
|
||||
}
|
||||
@@ -1716,6 +1778,42 @@ export async function hybridSearchCached(
|
||||
return budgeted;
|
||||
}
|
||||
|
||||
/**
|
||||
* RRF/dedup identity for a result row, at chunk granularity.
|
||||
*
|
||||
* Includes `source_id` so two same-slug pages in different federated sources
|
||||
* don't collapse into one fusion entry (the same composite-key discipline
|
||||
* `dedup.ts:pageKey` already uses at page granularity). Pre-fix the key was
|
||||
* `slug:chunk_id`, which silently merged cross-source rows and let a
|
||||
* synthetic chunkless row (chunk_id null) key on a text prefix; the
|
||||
* `(source_id, slug, chunk_id)` shape is collision-safe for both.
|
||||
*/
|
||||
function rrfKey(r: SearchResult): string {
|
||||
const source = r.source_id ?? 'default';
|
||||
return `${source}:${r.slug}:${r.chunk_id ?? r.chunk_text.slice(0, 50)}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Canonical query-cache scope key.
|
||||
*
|
||||
* The semantic cache stores results keyed by `(scope, query, knobs_hash)`.
|
||||
* A federated search (`sourceIds`) reads a different graph than a
|
||||
* single-source one, so the two must never share a cache row. Pre-fix the
|
||||
* cache only saw scalar `sourceId`; a federated query fell through to
|
||||
* `'default'` and could cross-serve an unrelated scope.
|
||||
*
|
||||
* - federated (sourceIds set) → `__set__:` + sorted, comma-joined ids
|
||||
* (order-independent; two different source-sets get distinct keys)
|
||||
* - scalar sourceId → the id itself (single-source unchanged)
|
||||
* - unscoped → `'default'` (single-source brains unchanged)
|
||||
*/
|
||||
export function cacheScopeKey(opts?: { sourceId?: string; sourceIds?: string[] }): string {
|
||||
if (opts?.sourceIds && opts.sourceIds.length > 0) {
|
||||
return '__set__:' + [...opts.sourceIds].sort().join(',');
|
||||
}
|
||||
return opts?.sourceId ?? 'default';
|
||||
}
|
||||
|
||||
/**
|
||||
* v0.32.x search-lite — weighted RRF. Each list contributes with its own
|
||||
* effective k value, which lets intent weighting bias keyword vs vector
|
||||
@@ -1731,7 +1829,7 @@ export function rrfFusionWeighted(
|
||||
for (const { list, k } of lists) {
|
||||
for (let rank = 0; rank < list.length; rank++) {
|
||||
const r = list[rank];
|
||||
const key = `${r.slug}:${r.chunk_id ?? r.chunk_text.slice(0, 50)}`;
|
||||
const key = rrfKey(r);
|
||||
const existing = scores.get(key);
|
||||
const rrfScore = 1 / (k + rank);
|
||||
|
||||
@@ -1771,7 +1869,7 @@ export function rrfFusion(lists: SearchResult[][], k: number, applyBoost = true)
|
||||
for (const list of lists) {
|
||||
for (let rank = 0; rank < list.length; rank++) {
|
||||
const r = list[rank];
|
||||
const key = `${r.slug}:${r.chunk_id ?? r.chunk_text.slice(0, 50)}`;
|
||||
const key = rrfKey(r);
|
||||
const existing = scores.get(key);
|
||||
const rrfScore = 1 / (k + rank);
|
||||
|
||||
|
||||
+55
-1
@@ -261,6 +261,18 @@ export interface ModeBundle {
|
||||
* run. Override: `search.autocut_jump` config → mode bundle.
|
||||
*/
|
||||
autocut_jump: number;
|
||||
/**
|
||||
* v0.43 — relational recall arm. When on, a relational query ("who invested
|
||||
* in widget-co", "what connects fund-a and fund-b") resolves its seed
|
||||
* entity and walks the typed-edge graph, injecting edge-derived candidates
|
||||
* as a fourth RRF arm. Pure no-op for non-relational queries. Default OFF
|
||||
* for conservative; ON for balanced/tokenmax. Override path: per-call
|
||||
* SearchOpts.relationalRetrieval → `search.relational_retrieval` config →
|
||||
* mode bundle. See src/core/search/relational-recall.ts.
|
||||
*/
|
||||
relationalRetrieval: boolean;
|
||||
/** v0.43 — max hops for relational traversal. Default 2, hard-capped at 3. */
|
||||
relational_retrieval_depth: number;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -309,6 +321,10 @@ export const MODE_BUNDLES: Readonly<Record<SearchMode, Readonly<ModeBundle>>> =
|
||||
// v0.42.3.0 — autocut OFF: conservative has no reranker, so no trustworthy
|
||||
// cliff signal exists (autocut would no-op). Explicit for clarity.
|
||||
autocut: false,
|
||||
// v0.43 — relational recall OFF for conservative (cost-sensitive tier,
|
||||
// matches graph_signals posture). Power users opt in per-call.
|
||||
relationalRetrieval: false,
|
||||
relational_retrieval_depth: 2,
|
||||
autocut_jump: 0.2,
|
||||
}),
|
||||
balanced: Object.freeze({
|
||||
@@ -363,6 +379,10 @@ export const MODE_BUNDLES: Readonly<Record<SearchMode, Readonly<ModeBundle>>> =
|
||||
contextual_retrieval_disabled: false,
|
||||
// v0.42.3.0 — autocut ON (reranker fires; cliff signal is trustworthy).
|
||||
autocut: true,
|
||||
// v0.43 — relational recall ON (contingent on the no-regression gate;
|
||||
// ships default-false everywhere if the gate flags any regression).
|
||||
relationalRetrieval: true,
|
||||
relational_retrieval_depth: 2,
|
||||
autocut_jump: 0.2,
|
||||
}),
|
||||
tokenmax: Object.freeze({
|
||||
@@ -411,6 +431,9 @@ export const MODE_BUNDLES: Readonly<Record<SearchMode, Readonly<ModeBundle>>> =
|
||||
contextual_retrieval_disabled: false,
|
||||
// v0.42.3.0 — autocut ON.
|
||||
autocut: true,
|
||||
// v0.43 — relational recall ON for tokenmax (max-recall tier).
|
||||
relationalRetrieval: true,
|
||||
relational_retrieval_depth: 2,
|
||||
autocut_jump: 0.2,
|
||||
}),
|
||||
});
|
||||
@@ -462,6 +485,9 @@ export interface SearchKeyOverrides {
|
||||
contextual_retrieval_disabled?: boolean;
|
||||
// v0.42.3.0 — autocut overrides.
|
||||
autocut?: boolean;
|
||||
// v0.43 — relational recall overrides.
|
||||
relationalRetrieval?: boolean;
|
||||
relational_retrieval_depth?: number;
|
||||
autocut_jump?: number;
|
||||
}
|
||||
|
||||
@@ -509,6 +535,9 @@ export interface SearchPerCallOpts {
|
||||
// numeric per-call knob threaded through the bundle.
|
||||
autocut?: boolean;
|
||||
autocut_jump?: number;
|
||||
// v0.43 — relational recall per-call overrides.
|
||||
relationalRetrieval?: boolean;
|
||||
relational_retrieval_depth?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -601,6 +630,9 @@ export function resolveSearchMode(input: ResolveSearchModeInput): ResolvedSearch
|
||||
// v0.42.3.0 — autocut resolved via the same pick chain.
|
||||
autocut: pick('autocut'),
|
||||
autocut_jump: pick('autocut_jump'),
|
||||
// v0.43 — relational recall resolved via the same pick chain.
|
||||
relationalRetrieval: pick('relationalRetrieval'),
|
||||
relational_retrieval_depth: pick('relational_retrieval_depth'),
|
||||
resolved_mode,
|
||||
mode_valid: valid,
|
||||
};
|
||||
@@ -706,7 +738,7 @@ export function attributeKnob<K extends keyof ModeBundle>(
|
||||
// to take effect immediately (one-time global cache cold-miss on upgrade; refills
|
||||
// within cache.ttl_seconds). Same cache-key-contamination convention as the
|
||||
// autocut / title_boost / graph_signals bumps above.
|
||||
export const KNOBS_HASH_VERSION = 9;
|
||||
export const KNOBS_HASH_VERSION = 10;
|
||||
|
||||
/**
|
||||
* v0.36 (D8 / CDX-2) — second-arg context for the cache key. The
|
||||
@@ -814,6 +846,14 @@ export function knobsHash(
|
||||
// etc.) so a partial-knobs caller (tests passing a minimal literal) can't
|
||||
// crash the hash. Typed callers always carry the field.
|
||||
`acj=${(knobs.autocut_jump ?? 0.2).toFixed(2)}`,
|
||||
// v=10 additions (v0.43, append-only): relational recall arm. A
|
||||
// relational-on write (edge-seeded result set) must NOT be served to a
|
||||
// relational-off lookup — same contamination class as graph_signals. The
|
||||
// depth changes the candidate set too, so it folds in as well. ONE-TIME
|
||||
// cold-miss on upgrade as v=9 rows become unreachable; pinned by
|
||||
// test/model-pricing.test.ts-style drift guards and the mode tests.
|
||||
`rel=${knobs.relationalRetrieval ? 1 : 0}`,
|
||||
`reld=${knobs.relational_retrieval_depth ?? 2}`,
|
||||
];
|
||||
const h = createHash('sha256');
|
||||
h.update(parts.join('|'));
|
||||
@@ -981,6 +1021,17 @@ export function loadOverridesFromConfig(
|
||||
if (Number.isFinite(n) && n > 0 && n <= 1) out.autocut_jump = n;
|
||||
}
|
||||
|
||||
// v0.43 — relational recall arm.
|
||||
const rel = get('search.relational_retrieval');
|
||||
if (rel !== undefined) {
|
||||
out.relationalRetrieval = rel === '1' || rel.toLowerCase() === 'true';
|
||||
}
|
||||
const reld = get('search.relational_retrieval_depth');
|
||||
if (reld !== undefined) {
|
||||
const n = parseInt(reld, 10);
|
||||
if (Number.isFinite(n) && n >= 1 && n <= 3) out.relational_retrieval_depth = n;
|
||||
}
|
||||
|
||||
return out;
|
||||
}
|
||||
|
||||
@@ -1019,6 +1070,9 @@ export const SEARCH_MODE_CONFIG_KEYS: ReadonlyArray<string> = Object.freeze([
|
||||
'search.contextual_retrieval_disabled',
|
||||
// v0.42.3.0 autocut
|
||||
'search.autocut',
|
||||
// v0.43 relational recall
|
||||
'search.relational_retrieval',
|
||||
'search.relational_retrieval_depth',
|
||||
'search.autocut_jump',
|
||||
]);
|
||||
|
||||
|
||||
@@ -0,0 +1,243 @@
|
||||
/**
|
||||
* Relational-query parser (typed-edge retrieval, v0.43).
|
||||
*
|
||||
* Detects queries whose answer is a RELATIONSHIP (an edge between entities)
|
||||
* rather than a passage — "who invested in widget-co", "who at acme works on
|
||||
* payments", "who introduced me to alice", "what connects fund-a and fund-b".
|
||||
* The relational recall arm uses the parse to resolve seed entities and walk
|
||||
* the typed-edge graph.
|
||||
*
|
||||
* Pure module. No DB, no LLM, no async. Detection is regex-only and
|
||||
* deterministic, parsed from the ORIGINAL query (never the LLM-expanded
|
||||
* variant) so the recall arm stays bit-for-bit reproducible.
|
||||
*
|
||||
* Precision-first (D4): this returns a CANDIDATE. The arm fires only when a
|
||||
* resolvable seed entity is also found (seed resolution lives in
|
||||
* relational-recall.ts). Patterns require the relation phrase and the entity
|
||||
* to be adjacent, so "who invested TIME in learning Rust" does not match the
|
||||
* "who invested in <seed>" pattern.
|
||||
*
|
||||
* Vocabulary (D2): the default bank covers the common archetypes; a schema
|
||||
* pack can extend it with `extraVerbs`. Every emitted link_type is validated
|
||||
* against KNOWN_LINK_TYPES so the query side can't drift from what ingest
|
||||
* actually produces (see link-extraction.ts:inferLinkType). intro/connects
|
||||
* traverse type-agnostically (linkTypes = null) because gbrain has no
|
||||
* `introduced`/`knows` edge — any edge touching the seed is the signal.
|
||||
*
|
||||
* ReDoS: seed captures are length-bounded (`.{1,80}?`) and every pattern is
|
||||
* anchored, so there is no catastrophic-backtracking surface.
|
||||
*
|
||||
* Tested in test/relational-intent.test.ts.
|
||||
*/
|
||||
|
||||
export type RelationalKind = 'who_rel' | 'who_at' | 'connects' | 'intro';
|
||||
export type RelationDirection = 'in' | 'out' | 'both';
|
||||
|
||||
export interface RelationalQuery {
|
||||
/** Which archetype matched. */
|
||||
kind: RelationalKind;
|
||||
/** Raw entity phrases to resolve, in query order. 1 for most, 2 for connects. */
|
||||
seeds: string[];
|
||||
/** Typed edges to traverse, or null for type-agnostic traversal. */
|
||||
linkTypes: string[] | null;
|
||||
/** Traversal direction from the seed. */
|
||||
direction: RelationDirection;
|
||||
/** The matched relation phrase, for telemetry / --explain. */
|
||||
relationPhrase: string;
|
||||
}
|
||||
|
||||
/** Schema-pack vocab extension (D2=B). */
|
||||
export interface RelationVerbSpec {
|
||||
/** A regex-source alternation of phrasings, e.g. `acquired|bought`. */
|
||||
verb: string;
|
||||
/** Edges this verb maps to. MUST be a subset of KNOWN_LINK_TYPES. */
|
||||
linkTypes: string[];
|
||||
/** Direction from the seed entity named after the verb. */
|
||||
direction: RelationDirection;
|
||||
}
|
||||
|
||||
export interface RelationVocab {
|
||||
extraVerbs?: RelationVerbSpec[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Link types ingest can actually produce (link-extraction.ts + frontmatter
|
||||
* map + schema packs). The query parser may only emit a SUBSET of these, so a
|
||||
* relation phrase can never traverse an edge type that ingest never writes.
|
||||
* `validateVocab` enforces this for pack-supplied verbs.
|
||||
*/
|
||||
export const KNOWN_LINK_TYPES: ReadonlySet<string> = new Set([
|
||||
'founded',
|
||||
'invested_in',
|
||||
'advises',
|
||||
'works_at',
|
||||
'attended',
|
||||
'yc_partner',
|
||||
'led_round',
|
||||
'mentions',
|
||||
'image_of',
|
||||
'discussed_in',
|
||||
'source',
|
||||
'related_to',
|
||||
'wikilink_basename',
|
||||
]);
|
||||
|
||||
// Seeds that are pronouns / generic nouns, not entities. If a pattern's seed
|
||||
// cleans down to one of these, the parse is rejected (precision-first).
|
||||
const STOPWORD_SEEDS: ReadonlySet<string> = new Set([
|
||||
'it', 'that', 'this', 'them', 'these', 'those', 'here', 'there',
|
||||
'everyone', 'anyone', 'someone', 'anybody', 'somebody', 'people',
|
||||
'things', 'us', 'me', 'him', 'her', 'you', 'who', 'what', 'which',
|
||||
]);
|
||||
|
||||
interface CompiledPattern {
|
||||
re: RegExp;
|
||||
kind: RelationalKind;
|
||||
linkTypes: string[] | null;
|
||||
direction: RelationDirection;
|
||||
/** Number of seed capture groups (1, or 2 for connects). */
|
||||
seedGroups: 1 | 2;
|
||||
}
|
||||
|
||||
// Bounded seed capture: 1–80 chars, lazy, so the trailing anchor decides the
|
||||
// boundary without catastrophic backtracking.
|
||||
const SEED = '(.{1,80}?)';
|
||||
|
||||
// ── who_rel verb bank: "who <verb> <seed>" → traverse INTO the seed ──
|
||||
// Each entry is explicit (linkTypes inline) so there is no second lookup.
|
||||
const WHO_REL_VERBS: Array<{ verb: string; linkTypes: string[]; direction: RelationDirection }> = [
|
||||
{ verb: 'invested in|invests in|funded|backed|backs|led the round in|led the seed in|led the series [a-z] in', linkTypes: ['invested_in', 'led_round'], direction: 'in' },
|
||||
{ verb: 'founded|co-?founded|started', linkTypes: ['founded'], direction: 'in' },
|
||||
{ verb: 'advises|advised', linkTypes: ['advises'], direction: 'in' },
|
||||
{ verb: 'works at|worked at|works for', linkTypes: ['works_at'], direction: 'in' },
|
||||
{ verb: 'attended', linkTypes: ['attended'], direction: 'in' },
|
||||
];
|
||||
|
||||
function buildPatterns(vocab?: RelationVocab): CompiledPattern[] {
|
||||
const patterns: CompiledPattern[] = [];
|
||||
|
||||
// connects — two seeds, type-agnostic. Most specific, checked first.
|
||||
patterns.push({
|
||||
re: new RegExp(
|
||||
`\\b(?:what|which)\\s+(?:companies?|people|things|entities|deals?)?\\s*(?:connects?|links?|ties? together|is (?:the )?(?:connection|link|relationship) between)\\s+${SEED}\\s+(?:and|&)\\s+${SEED}\\s*\\??$`,
|
||||
'i',
|
||||
),
|
||||
kind: 'connects', linkTypes: null, direction: 'both', seedGroups: 2,
|
||||
});
|
||||
patterns.push({
|
||||
re: new RegExp(
|
||||
`\\bhow\\s+(?:are|is|do|does)\\s+${SEED}\\s+(?:and|&)\\s+${SEED}\\s+(?:connected|related|linked|associated)\\b`,
|
||||
'i',
|
||||
),
|
||||
kind: 'connects', linkTypes: null, direction: 'both', seedGroups: 2,
|
||||
});
|
||||
|
||||
// intro — type-agnostic walk around the named person (no `introduced` edge).
|
||||
patterns.push({
|
||||
re: new RegExp(
|
||||
`\\bwho\\s+(?:introduced|connected|referred)\\s+(?:me|us|him|her|them)\\s+to\\s+${SEED}\\s*\\??$`,
|
||||
'i',
|
||||
),
|
||||
kind: 'intro', linkTypes: null, direction: 'both', seedGroups: 1,
|
||||
});
|
||||
|
||||
// who_at — entity in the middle: "who at acme works on payments".
|
||||
patterns.push({
|
||||
re: new RegExp(
|
||||
`\\bwho\\s+(?:at|from|in)\\s+${SEED}\\s+(?:works? on|works?|leads?|runs?|builds?|owns?|handles?|manages?)\\b`,
|
||||
'i',
|
||||
),
|
||||
kind: 'who_at', linkTypes: ['works_at'], direction: 'in', seedGroups: 1,
|
||||
});
|
||||
|
||||
// who_rel — "who <verb> <seed>".
|
||||
for (const v of WHO_REL_VERBS) {
|
||||
patterns.push({
|
||||
re: new RegExp(`\\bwho\\s+(?:${v.verb})\\s+${SEED}\\s*\\??$`, 'i'),
|
||||
kind: 'who_rel', linkTypes: v.linkTypes, direction: v.direction, seedGroups: 1,
|
||||
});
|
||||
}
|
||||
|
||||
// outgoing variants — "what did <seed> invest in", "where does <seed> work".
|
||||
patterns.push({
|
||||
re: new RegExp(
|
||||
`\\bwhat\\s+(?:companies?|startups?|deals?)?\\s*(?:has|have|did|does)?\\s*${SEED}\\s+(?:invest(?:ed)? in)\\b`,
|
||||
'i',
|
||||
),
|
||||
kind: 'who_rel', linkTypes: ['invested_in', 'led_round'], direction: 'out', seedGroups: 1,
|
||||
});
|
||||
patterns.push({
|
||||
re: new RegExp(`\\bwhere\\s+(?:does|did|has)\\s+${SEED}\\s+work\\b`, 'i'),
|
||||
kind: 'who_rel', linkTypes: ['works_at'], direction: 'out', seedGroups: 1,
|
||||
});
|
||||
|
||||
// schema-pack extensions: "who <verb> <seed>" for each extra verb.
|
||||
for (const v of vocab?.extraVerbs ?? []) {
|
||||
patterns.push({
|
||||
re: new RegExp(`\\bwho\\s+(?:${v.verb})\\s+${SEED}\\s*\\??$`, 'i'),
|
||||
kind: 'who_rel', linkTypes: v.linkTypes, direction: v.direction, seedGroups: 1,
|
||||
});
|
||||
}
|
||||
|
||||
return patterns;
|
||||
}
|
||||
|
||||
/** Trim, drop a leading article and surrounding quotes, strip trailing `?`. */
|
||||
function cleanSeed(raw: string): string {
|
||||
return raw
|
||||
.trim()
|
||||
.replace(/\?+$/, '')
|
||||
.replace(/^["'`]|["'`]$/g, '')
|
||||
.replace(/^(?:the|a|an)\s+/i, '')
|
||||
.trim();
|
||||
}
|
||||
|
||||
function validSeed(s: string): boolean {
|
||||
if (s.length === 0 || s.length > 80) return false;
|
||||
if (STOPWORD_SEEDS.has(s.toLowerCase())) return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate that every link_type a vocab emits is one ingest can produce.
|
||||
* Throws on an unknown type so a misconfigured schema pack fails loudly at
|
||||
* load time rather than silently traversing an edge that never exists.
|
||||
*/
|
||||
export function validateVocab(vocab: RelationVocab): void {
|
||||
for (const v of vocab.extraVerbs ?? []) {
|
||||
for (const lt of v.linkTypes) {
|
||||
if (!KNOWN_LINK_TYPES.has(lt)) {
|
||||
throw new Error(
|
||||
`relational vocab: unknown link_type "${lt}" for verb /${v.verb}/ — must be one of ${[...KNOWN_LINK_TYPES].join(', ')}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse a query into a RelationalQuery, or null if it isn't relational.
|
||||
* First matching pattern wins (patterns are ordered specific → general).
|
||||
*/
|
||||
export function parseRelationalQuery(query: string, vocab?: RelationVocab): RelationalQuery | null {
|
||||
if (!query || query.length > 512) return null; // bound work; real queries are short
|
||||
const patterns = buildPatterns(vocab);
|
||||
|
||||
for (const p of patterns) {
|
||||
const m = p.re.exec(query);
|
||||
if (!m) continue;
|
||||
|
||||
if (p.seedGroups === 2) {
|
||||
const a = cleanSeed(m[1] ?? '');
|
||||
const b = cleanSeed(m[2] ?? '');
|
||||
if (!validSeed(a) || !validSeed(b)) continue;
|
||||
return { kind: p.kind, seeds: [a, b], linkTypes: p.linkTypes, direction: p.direction, relationPhrase: m[0].trim() };
|
||||
}
|
||||
|
||||
const seed = cleanSeed(m[1] ?? '');
|
||||
if (!validSeed(seed)) continue;
|
||||
return { kind: p.kind, seeds: [seed], linkTypes: p.linkTypes, direction: p.direction, relationPhrase: m[0].trim() };
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
@@ -0,0 +1,232 @@
|
||||
/**
|
||||
* Relational recall arm (typed-edge retrieval, v0.43).
|
||||
*
|
||||
* Turns a relational query into a ranked list of edge-derived candidates that
|
||||
* hybridSearch injects as a FOURTH RRF arm (alongside keyword + vector), so a
|
||||
* relationship answer competes for ranking instead of relying on lexical or
|
||||
* vector similarity to surface it.
|
||||
*
|
||||
* Flow: parseRelationalQuery → resolve seed entity(ies) (scope-aware,
|
||||
* confidence-gated) → engine.relationalFanout (within-source, deterministic)
|
||||
* → batch-hydrate to SearchResult rows (reinforcing each page's REAL canonical
|
||||
* chunk; page-level key for chunkless entity pages).
|
||||
*
|
||||
* Determinism: parses the ORIGINAL query (never an LLM-expanded variant);
|
||||
* traversal + resolution are deterministic. Fail-open: any error returns an
|
||||
* empty arm + an audit row, never breaking the search hot path.
|
||||
*
|
||||
* Federation (E2=A): resolves the seed in every in-scope source and fans out
|
||||
* from each; traversal stays WITHIN each source (no cross-boundary edges in
|
||||
* v1). Confidence gate (D3): a seed that only `fallback_slugify`-resolves is
|
||||
* dropped, so the arm never traverses from an invented slug. The tier-2
|
||||
* resolution-margin gate is a filed TODO.
|
||||
*
|
||||
* Tested in test/relational-recall.test.ts.
|
||||
*/
|
||||
|
||||
import type { BrainEngine } from '../engine.ts';
|
||||
import type { SearchResult, PageType, RelationalFanoutRow } from '../types.ts';
|
||||
import { createAuditWriter } from '../audit/audit-writer.ts';
|
||||
import { resolveEntitySlugWithSource } from '../entities/resolve.ts';
|
||||
import { parseRelationalQuery, type RelationalQuery, type RelationVocab } from './relational-intent.ts';
|
||||
|
||||
export interface RelationalArmOpts {
|
||||
sourceId?: string;
|
||||
sourceIds?: string[];
|
||||
depth?: number;
|
||||
limit?: number;
|
||||
vocab?: RelationVocab;
|
||||
onMeta?: (meta: RelationalArmMeta) => void;
|
||||
}
|
||||
|
||||
export interface RelationalArmMeta {
|
||||
fired: boolean;
|
||||
kind: RelationalQuery['kind'] | null;
|
||||
seeds_resolved: number;
|
||||
candidates: number;
|
||||
errored: boolean;
|
||||
duration_ms: number;
|
||||
}
|
||||
|
||||
interface RelationalFailureEvent {
|
||||
ts: string;
|
||||
error_summary: string;
|
||||
query_kind: string;
|
||||
}
|
||||
|
||||
const failureWriter = createAuditWriter<RelationalFailureEvent>({
|
||||
featureName: 'relational-recall-failures',
|
||||
errorLabel: 'gbrain',
|
||||
errorTrailer: '; search continues',
|
||||
});
|
||||
|
||||
/** Recent relational-arm fail-open events — consumed by doctor + search stats. */
|
||||
export function readRecentRelationalFailures(days = 7, now: Date = new Date()): RelationalFailureEvent[] {
|
||||
return failureWriter.readRecent(days, now);
|
||||
}
|
||||
|
||||
function truncate(msg: string, max = 200): string {
|
||||
return msg.length <= max ? msg : msg.slice(0, max - 1) + '…';
|
||||
}
|
||||
|
||||
/** Sources to resolve a seed against. Federated → the set; scalar → [id];
|
||||
* unscoped/__all__ → ['default'] (single-source brains; multi-source
|
||||
* enumeration under __all__ is a v1 limitation). */
|
||||
function scopeSources(opts: RelationalArmOpts): string[] {
|
||||
if (opts.sourceIds && opts.sourceIds.length > 0) return opts.sourceIds;
|
||||
if (opts.sourceId && opts.sourceId !== '__all__') return [opts.sourceId];
|
||||
return ['default'];
|
||||
}
|
||||
|
||||
/** Resolve a seed phrase to all in-scope (source_id, slug) pairs that
|
||||
* resolve to a REAL page (confidence gate D3 tier-1: drop fallback_slugify). */
|
||||
async function resolveSeedScoped(
|
||||
engine: BrainEngine,
|
||||
sources: string[],
|
||||
phrase: string,
|
||||
): Promise<Array<{ source_id: string; slug: string }>> {
|
||||
const out: Array<{ source_id: string; slug: string }> = [];
|
||||
const seen = new Set<string>();
|
||||
for (const sid of sources) {
|
||||
const r = await resolveEntitySlugWithSource(engine, sid, phrase);
|
||||
if (!r || r.source === 'fallback_slugify') continue;
|
||||
const key = `${sid}:${r.slug}`;
|
||||
if (seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
out.push({ source_id: sid, slug: r.slug });
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Batch-hydrate fanout rows into SearchResult rows in fanout (ranked) order. */
|
||||
async function hydrate(
|
||||
engine: BrainEngine,
|
||||
rows: RelationalFanoutRow[],
|
||||
seedSlug: string,
|
||||
): Promise<SearchResult[]> {
|
||||
if (rows.length === 0) return [];
|
||||
const slugs = Array.from(new Set(rows.map(r => r.slug)));
|
||||
const pageRows = await engine.executeRaw<{
|
||||
page_id: number; slug: string; source_id: string; title: string; type: string; synopsis: string | null;
|
||||
}>(
|
||||
`SELECT p.id AS page_id, p.slug, p.source_id, p.title, p.type,
|
||||
LEFT(p.compiled_truth, 240) AS synopsis
|
||||
FROM pages p
|
||||
WHERE p.slug = ANY($1::text[]) AND p.deleted_at IS NULL`,
|
||||
[slugs],
|
||||
);
|
||||
const byKey = new Map<string, typeof pageRows[number]>();
|
||||
for (const pr of pageRows) byKey.set(`${pr.source_id}:${pr.slug}`, pr);
|
||||
|
||||
const out: SearchResult[] = [];
|
||||
for (const r of rows) {
|
||||
const pr = byKey.get(`${r.source_id}:${r.slug}`);
|
||||
if (!pr) continue;
|
||||
out.push({
|
||||
slug: r.slug,
|
||||
page_id: pr.page_id,
|
||||
title: pr.title,
|
||||
type: pr.type as PageType,
|
||||
chunk_text: pr.synopsis ?? r.slug,
|
||||
chunk_source: 'compiled_truth',
|
||||
// E1: reinforce the page's REAL canonical chunk; F3: chunkless entity
|
||||
// pages key page-level (chunk_id 0 → rrfKey `source:slug:0`, stable and
|
||||
// collision-safe; SERIAL chunk ids start at 1 so 0 never aliases a real chunk).
|
||||
chunk_id: r.canonical_chunk_id ?? 0,
|
||||
chunk_index: 0,
|
||||
score: 0, // rank-based: RRF derives score from list position
|
||||
stale: false,
|
||||
source_id: r.source_id,
|
||||
relational_via_link_types: r.via_link_types,
|
||||
relational_seed: seedSlug,
|
||||
relational_hop: r.hop,
|
||||
relational_path: r.path,
|
||||
});
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the relational recall arm. Returns an empty list (pure no-op) when the
|
||||
* query isn't relational or no seed resolves. Never throws.
|
||||
*/
|
||||
export async function buildRelationalArm(
|
||||
engine: BrainEngine,
|
||||
query: string,
|
||||
opts: RelationalArmOpts = {},
|
||||
): Promise<SearchResult[]> {
|
||||
const startedAt = Date.now();
|
||||
const meta: RelationalArmMeta = {
|
||||
fired: false, kind: null, seeds_resolved: 0, candidates: 0, errored: false, duration_ms: 0,
|
||||
};
|
||||
const finish = (list: SearchResult[]) => {
|
||||
meta.candidates = list.length;
|
||||
meta.duration_ms = Date.now() - startedAt;
|
||||
opts.onMeta?.(meta);
|
||||
return list;
|
||||
};
|
||||
|
||||
const parsed = parseRelationalQuery(query, opts.vocab);
|
||||
if (!parsed) return finish([]);
|
||||
meta.kind = parsed.kind;
|
||||
|
||||
try {
|
||||
const sources = scopeSources(opts);
|
||||
const fanoutOpts = {
|
||||
linkTypes: parsed.linkTypes,
|
||||
direction: parsed.direction,
|
||||
depth: opts.depth,
|
||||
limit: opts.limit,
|
||||
};
|
||||
|
||||
if (parsed.kind === 'connects' && parsed.seeds.length === 2) {
|
||||
// Resolve both endpoints; both must resolve or the arm no-ops.
|
||||
const resA = await resolveSeedScoped(engine, sources, parsed.seeds[0]);
|
||||
const resB = await resolveSeedScoped(engine, sources, parsed.seeds[1]);
|
||||
if (resA.length === 0 || resB.length === 0) return finish([]);
|
||||
meta.seeds_resolved = resA.length + resB.length;
|
||||
|
||||
const perSource = (rs: typeof resA) => ({
|
||||
sourceId: rs.length === 1 ? rs[0].source_id : undefined,
|
||||
sourceIds: rs.length > 1 ? Array.from(new Set(rs.map(x => x.source_id))) : undefined,
|
||||
slugs: Array.from(new Set(rs.map(x => x.slug))),
|
||||
});
|
||||
const a = perSource(resA);
|
||||
const b = perSource(resB);
|
||||
const fanA = await engine.relationalFanout(a.slugs, { ...fanoutOpts, sourceId: a.sourceId, sourceIds: a.sourceIds });
|
||||
const fanB = await engine.relationalFanout(b.slugs, { ...fanoutOpts, sourceId: b.sourceId, sourceIds: b.sourceIds });
|
||||
// Shared midpoints: nodes reachable from BOTH endpoints (exclude the
|
||||
// endpoints themselves). Ordered by combined hop.
|
||||
const bByKey = new Map(fanB.map(r => [`${r.source_id}:${r.slug}`, r] as const));
|
||||
const endpointSlugs = new Set([...a.slugs, ...b.slugs]);
|
||||
const shared = fanA
|
||||
.filter(r => bByKey.has(`${r.source_id}:${r.slug}`) && !endpointSlugs.has(r.slug))
|
||||
.map(r => ({ row: r, combined: r.hop + bByKey.get(`${r.source_id}:${r.slug}`)!.hop }))
|
||||
.sort((x, y) => x.combined - y.combined || x.row.slug.localeCompare(y.row.slug))
|
||||
.map(x => x.row);
|
||||
const list = await hydrate(engine, shared, parsed.seeds.join(' ↔ '));
|
||||
meta.fired = list.length > 0;
|
||||
return finish(list);
|
||||
}
|
||||
|
||||
// who_rel / who_at / intro: single logical seed (may resolve in N sources).
|
||||
const resolved = await resolveSeedScoped(engine, sources, parsed.seeds[0]);
|
||||
if (resolved.length === 0) return finish([]);
|
||||
meta.seeds_resolved = resolved.length;
|
||||
const slugs = Array.from(new Set(resolved.map(r => r.slug)));
|
||||
const srcIds = Array.from(new Set(resolved.map(r => r.source_id)));
|
||||
const rows = await engine.relationalFanout(slugs, {
|
||||
...fanoutOpts,
|
||||
sourceId: srcIds.length === 1 ? srcIds[0] : undefined,
|
||||
sourceIds: srcIds.length > 1 ? srcIds : undefined,
|
||||
});
|
||||
const list = await hydrate(engine, rows, resolved[0].slug);
|
||||
meta.fired = list.length > 0;
|
||||
return finish(list);
|
||||
} catch (err) {
|
||||
const msg = err instanceof Error ? err.message : String(err);
|
||||
failureWriter.log({ error_summary: truncate(msg), query_kind: parsed.kind });
|
||||
meta.errored = true;
|
||||
return finish([]);
|
||||
}
|
||||
}
|
||||
@@ -719,6 +719,20 @@ export interface SearchResult {
|
||||
graph_session_demoted?: boolean;
|
||||
/** Slug prefix used for the session-diversification grouping. */
|
||||
graph_session_prefix?: string;
|
||||
/**
|
||||
* v0.43 relational recall arm — set when this result was surfaced by
|
||||
* typed-edge traversal (not lexical/vector). Drives `gbrain search
|
||||
* --explain` attribution ("surfaced via invested_in edge from widget-co").
|
||||
* Absent for organic keyword/vector results.
|
||||
*/
|
||||
/** Edge types the result was reached by (e.g. ['invested_in']). */
|
||||
relational_via_link_types?: string[];
|
||||
/** The resolved seed entity slug the traversal started from. */
|
||||
relational_seed?: string;
|
||||
/** Shortest hop distance from the seed (1 = direct neighbor). */
|
||||
relational_hop?: number;
|
||||
/** Shortest connecting slug path seed→…→result (for "how I know this"). */
|
||||
relational_path?: string[];
|
||||
/**
|
||||
* v0.40.4 full attribution (D12=A) — per-stage score deltas for the
|
||||
* `gbrain search --explain` formatter. Every boost stage stamps its
|
||||
@@ -1089,6 +1103,14 @@ export interface SearchOpts {
|
||||
* would resolve to the same mode default and the gate would be a no-op.
|
||||
*/
|
||||
graph_signals?: boolean;
|
||||
/**
|
||||
* v0.43 — relational recall arm per-call override. Per-call wins over the
|
||||
* `search.relational_retrieval` config key wins over the mode bundle. Eval
|
||||
* A/B gates need explicit per-call control or both branches resolve to the
|
||||
* same mode default. `relationalRetrievalDepth` caps traversal hops (1..3).
|
||||
*/
|
||||
relationalRetrieval?: boolean;
|
||||
relationalRetrievalDepth?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -1177,6 +1199,43 @@ export interface GraphPath {
|
||||
depth: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* One reached node from a typed-edge relational fan-out (v0.43). The recall
|
||||
* arm hydrates these into SearchResult rows and injects them as a fourth RRF
|
||||
* arm. Aggregated to the page level: `hop` is the shortest distance from any
|
||||
* seed, `edge_count` is a connection-richness proxy (distinct edge types via
|
||||
* which the node is reached), `via_link_types` names them, `path` is the
|
||||
* shortest connecting slug chain (for --explain), and `canonical_chunk_id` is
|
||||
* the page's lowest-ordinal chunk (null for frontmatter-only entity pages).
|
||||
*/
|
||||
export interface RelationalFanoutRow {
|
||||
source_id: string;
|
||||
slug: string;
|
||||
hop: number;
|
||||
edge_count: number;
|
||||
via_link_types: string[];
|
||||
path: string[];
|
||||
canonical_chunk_id: number | null;
|
||||
}
|
||||
|
||||
/** Options for BrainEngine.relationalFanout. */
|
||||
export interface RelationalFanoutOpts {
|
||||
/** Edge types to traverse; null/empty = type-agnostic. */
|
||||
linkTypes?: string[] | null;
|
||||
/** Direction from each seed. Default 'both'. */
|
||||
direction?: 'in' | 'out' | 'both';
|
||||
/** Max hops. Default 2, hard-capped at 3. */
|
||||
depth?: number;
|
||||
/** Include `link_source='mentions'` edges. Default false (typed edges only). */
|
||||
includeMentions?: boolean;
|
||||
/** Single-source scope. */
|
||||
sourceId?: string;
|
||||
/** Federated scope; traversal stays WITHIN each seed's own source. */
|
||||
sourceIds?: string[];
|
||||
/** Hard cap on returned candidate nodes. Default 50. */
|
||||
limit?: number;
|
||||
}
|
||||
|
||||
// Timeline
|
||||
export interface TimelineEntry {
|
||||
id: number;
|
||||
|
||||
@@ -33,6 +33,15 @@ export interface NamedThingQuestion {
|
||||
/** slugs that must NOT appear in top-k (hard-negative family). */
|
||||
forbidden?: string[];
|
||||
notes?: string;
|
||||
/**
|
||||
* v0.43 relational metadata (graph-relationship family). Optional, typed
|
||||
* (not buried in `notes`) so the relational parser/path/determinism tests
|
||||
* can machine-check the parse. `seed` is the entity the answer relates to;
|
||||
* `linkTypes` the edge(s) expected; `kind` the archetype.
|
||||
*/
|
||||
seed?: string;
|
||||
linkTypes?: string[];
|
||||
kind?: 'who_rel' | 'who_at' | 'connects' | 'intro';
|
||||
}
|
||||
|
||||
/** Ranked slugs for a query, best-first. */
|
||||
@@ -46,6 +55,11 @@ export interface QuestionResult {
|
||||
reciprocal_rank: number; // 0 if no relevant slug in results
|
||||
/** hard-negative only: true when NO forbidden slug appeared in top-3. */
|
||||
negative_clean?: boolean;
|
||||
/** v0.43 — fraction of relevant slugs found in top-K (K=3). 0 for hard-negative. */
|
||||
recall_at_k: number;
|
||||
/** v0.43 — fraction of relevant slugs found in top-10. The relational
|
||||
* headline metric (a relationship query often has several valid answers). */
|
||||
recall_at_10: number;
|
||||
}
|
||||
|
||||
export interface FamilyReport {
|
||||
@@ -54,6 +68,9 @@ export interface FamilyReport {
|
||||
hit_at_1: number; // rate
|
||||
hit_at_3: number; // rate
|
||||
mrr: number;
|
||||
/** v0.43 — mean recall@K / recall@10 across the family's questions. */
|
||||
recall_at_k: number;
|
||||
recall_at_10: number;
|
||||
}
|
||||
|
||||
export interface RetrievalQualityReport {
|
||||
@@ -71,14 +88,20 @@ export function scoreQuestion(q: NamedThingQuestion, ranked: string[]): Question
|
||||
const forbidden = new Set(q.forbidden ?? []);
|
||||
const topK = ranked.slice(0, K);
|
||||
const clean = !topK.some(s => forbidden.has(s));
|
||||
return { family: q.family, query: q.query, hit_at_1: clean, hit_at_3: clean, reciprocal_rank: clean ? 1 : 0, negative_clean: clean };
|
||||
return { family: q.family, query: q.query, hit_at_1: clean, hit_at_3: clean, reciprocal_rank: clean ? 1 : 0, negative_clean: clean, recall_at_k: 0, recall_at_10: 0 };
|
||||
}
|
||||
const relevant = new Set(q.relevant ?? []);
|
||||
const firstRelevantIdx = ranked.findIndex(s => relevant.has(s));
|
||||
const hit1 = firstRelevantIdx === 0;
|
||||
const hit3 = firstRelevantIdx >= 0 && firstRelevantIdx < K;
|
||||
const rr = firstRelevantIdx >= 0 ? 1 / (firstRelevantIdx + 1) : 0;
|
||||
return { family: q.family, query: q.query, hit_at_1: hit1, hit_at_3: hit3, reciprocal_rank: rr };
|
||||
const recallAt = (k: number): number => {
|
||||
if (relevant.size === 0) return 0;
|
||||
const top = ranked.slice(0, k);
|
||||
const found = top.filter(s => relevant.has(s)).length;
|
||||
return found / relevant.size;
|
||||
};
|
||||
return { family: q.family, query: q.query, hit_at_1: hit1, hit_at_3: hit3, reciprocal_rank: rr, recall_at_k: recallAt(K), recall_at_10: recallAt(10) };
|
||||
}
|
||||
|
||||
export async function runRetrievalQuality(
|
||||
@@ -106,6 +129,8 @@ export async function runRetrievalQuality(
|
||||
hit_at_1: n ? list.filter(r => r.hit_at_1).length / n : 0,
|
||||
hit_at_3: n ? list.filter(r => r.hit_at_3).length / n : 0,
|
||||
mrr: n ? list.reduce((s, r) => s + r.reciprocal_rank, 0) / n : 0,
|
||||
recall_at_k: n ? list.reduce((s, r) => s + r.recall_at_k, 0) / n : 0,
|
||||
recall_at_10: n ? list.reduce((s, r) => s + r.recall_at_10, 0) / n : 0,
|
||||
});
|
||||
}
|
||||
families.sort((a, b) => a.family.localeCompare(b.family));
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
/**
|
||||
* Query-cache scope key (federation hardening).
|
||||
*
|
||||
* A federated search reads a different graph than a single-source one, so
|
||||
* the semantic cache must key them apart. `cacheScopeKey` produces an
|
||||
* order-independent key for federated scopes and leaves single-source
|
||||
* brains on their existing key (scalar id or 'default'), so single-source
|
||||
* cache hit-rate is unchanged.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { cacheScopeKey } from '../src/core/search/hybrid.ts';
|
||||
|
||||
describe('cacheScopeKey', () => {
|
||||
test('unscoped → default (single-source unchanged)', () => {
|
||||
expect(cacheScopeKey(undefined)).toBe('default');
|
||||
expect(cacheScopeKey({})).toBe('default');
|
||||
});
|
||||
|
||||
test('scalar sourceId → itself (single-source unchanged)', () => {
|
||||
expect(cacheScopeKey({ sourceId: 'host' })).toBe('host');
|
||||
});
|
||||
|
||||
test('federated sourceIds → order-independent set key', () => {
|
||||
const k1 = cacheScopeKey({ sourceIds: ['team-b', 'team-a', 'host'] });
|
||||
const k2 = cacheScopeKey({ sourceIds: ['host', 'team-a', 'team-b'] });
|
||||
expect(k1).toBe(k2); // order does not matter
|
||||
expect(k1).toBe('__set__:host,team-a,team-b');
|
||||
});
|
||||
|
||||
test('different source-sets do NOT share a key', () => {
|
||||
const a = cacheScopeKey({ sourceIds: ['host', 'team-a'] });
|
||||
const b = cacheScopeKey({ sourceIds: ['host', 'team-b'] });
|
||||
expect(a).not.toBe(b);
|
||||
});
|
||||
|
||||
test('federated set key is distinct from any single scalar key', () => {
|
||||
const set = cacheScopeKey({ sourceIds: ['host'] });
|
||||
const scalar = cacheScopeKey({ sourceId: 'host' });
|
||||
expect(set).not.toBe(scalar); // a 1-element set still cannot serve a scalar read
|
||||
});
|
||||
});
|
||||
@@ -136,14 +136,15 @@ describe('D2 — knobsHash differs across cross-modal knob values', () => {
|
||||
return resolveSearchMode({ mode: 'balanced' });
|
||||
}
|
||||
|
||||
test('KNOBS_HASH_VERSION is 9 (cross-modal still appended; 8→9 archive-demote #1777)', () => {
|
||||
test('KNOBS_HASH_VERSION is 10 (cross-modal still appended; 9→10 relational recall)', () => {
|
||||
// v0.35 ladder: 1→2 reranker, 2→3 floor_ratio. v0.36 piggybacks on v=3
|
||||
// with 7 cross-modal knobs + column/provider context. v0.40.4 (salem) +
|
||||
// v0.39 T21 (master) bump to v=4 for graph_signals + schema-pack fields.
|
||||
// v0.40.3.0 D8 bumps to v=5 (sequenced behind salem's v=4 graph-signals).
|
||||
// v0.41.22.0 (type-unification): 5→6 for alias_resolved post-fusion boost.
|
||||
// T2: 6→7 title_boost. v0.42.3.0: 7→8 autocut. issue #1777: 8→9 archive/ demote.
|
||||
expect(KNOBS_HASH_VERSION).toBe(9);
|
||||
// v0.43: 9→10 relational recall arm.
|
||||
expect(KNOBS_HASH_VERSION).toBe(10);
|
||||
});
|
||||
|
||||
test('flipping unified_multimodal changes the hash', () => {
|
||||
|
||||
@@ -544,3 +544,71 @@ describeBoth('Engine parity — Postgres vs PGLite', () => {
|
||||
expect(pg.find((r) => r.slug === 'ep/ec-alice')!.inbound_count).toBe(2);
|
||||
});
|
||||
});
|
||||
|
||||
// ── relationalFanout parity (v0.43) ─────────────────────────────────────
|
||||
async function seedRelational(eng: BrainEngine) {
|
||||
const pages: Array<[string, 'company' | 'person']> = [
|
||||
['companies/ep-widget', 'company'],
|
||||
['companies/ep-other', 'company'],
|
||||
['people/ep-inv-a', 'person'],
|
||||
['people/ep-inv-b', 'person'],
|
||||
['people/ep-emp-c', 'person'],
|
||||
['people/ep-mentioner', 'person'],
|
||||
];
|
||||
for (const [slug, type] of pages) {
|
||||
await eng.putPage(slug, { type, title: slug, compiled_truth: `${slug} body`, timeline: '' });
|
||||
}
|
||||
await eng.upsertChunks('people/ep-inv-b', [{
|
||||
chunk_index: 0, chunk_text: 'b', chunk_source: 'compiled_truth',
|
||||
embedding: basisEmbedding(2), token_count: 1,
|
||||
}] satisfies ChunkInput[]);
|
||||
await eng.addLink('people/ep-inv-a', 'companies/ep-widget', '', 'invested_in', 'manual');
|
||||
await eng.addLink('people/ep-inv-b', 'companies/ep-widget', '', 'invested_in', 'manual');
|
||||
await eng.addLink('people/ep-emp-c', 'companies/ep-widget', '', 'works_at', 'manual');
|
||||
await eng.addLink('people/ep-mentioner', 'companies/ep-widget', '', 'mentions', 'mentions');
|
||||
await eng.addLink('people/ep-inv-a', 'companies/ep-other', '', 'invested_in', 'manual');
|
||||
}
|
||||
|
||||
describeBoth('Engine parity — relationalFanout', () => {
|
||||
let pgEngine: BrainEngine;
|
||||
let pgliteEngine: PGLiteEngine;
|
||||
|
||||
beforeAll(async () => {
|
||||
pgEngine = await setupDB();
|
||||
await seedRelational(pgEngine);
|
||||
pgliteEngine = new PGLiteEngine();
|
||||
await pgliteEngine.connect({});
|
||||
await pgliteEngine.initSchema();
|
||||
await seedRelational(pgliteEngine);
|
||||
}, 90_000);
|
||||
|
||||
afterAll(async () => {
|
||||
await pgliteEngine.disconnect();
|
||||
await teardownDB();
|
||||
}, 30_000);
|
||||
|
||||
const shape = (rows: Awaited<ReturnType<BrainEngine['relationalFanout']>>) =>
|
||||
rows.map(r => `${r.source_id}:${r.slug}:${r.hop}:${r.edge_count}:${r.via_link_types.join(',')}:${r.path.join('>')}:${r.canonical_chunk_id ?? 'null'}`);
|
||||
|
||||
test('typed-edge fan-out is identical across engines', async () => {
|
||||
const opts = { direction: 'in' as const, linkTypes: ['invested_in'] };
|
||||
const pg = await pgEngine.relationalFanout(['companies/ep-widget'], opts);
|
||||
const pglite = await pgliteEngine.relationalFanout(['companies/ep-widget'], opts);
|
||||
expect(shape(pg)).toEqual(shape(pglite));
|
||||
expect(pg.map(r => r.slug).sort()).toEqual(['people/ep-inv-a', 'people/ep-inv-b']);
|
||||
});
|
||||
|
||||
test('type-agnostic + mentions-exclusion identical across engines', async () => {
|
||||
const pg = await pgEngine.relationalFanout(['companies/ep-widget'], { direction: 'in' });
|
||||
const pglite = await pgliteEngine.relationalFanout(['companies/ep-widget'], { direction: 'in' });
|
||||
expect(shape(pg)).toEqual(shape(pglite));
|
||||
expect(pg.map(r => r.slug)).not.toContain('people/ep-mentioner');
|
||||
});
|
||||
|
||||
test('connects (multi-seed, both) identical across engines', async () => {
|
||||
const seeds = ['companies/ep-widget', 'companies/ep-other'];
|
||||
const pg = await pgEngine.relationalFanout(seeds, { direction: 'both' });
|
||||
const pglite = await pgliteEngine.relationalFanout(seeds, { direction: 'both' });
|
||||
expect(shape(pg)).toEqual(shape(pglite));
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,195 @@
|
||||
/**
|
||||
* Relational benchmark corpus (v0.43).
|
||||
*
|
||||
* A small entity graph whose relational answers are LEXICALLY UNRECOVERABLE:
|
||||
* every page body is generic and NEVER names the entity it relates to. Only
|
||||
* the typed edge connects them, so keyword + vector retrieval cannot surface
|
||||
* the answer — isolating the contribution of the typed-edge recall arm.
|
||||
*
|
||||
* Canonical source for BOTH the seed loader and the question set, so the
|
||||
* corpus and the gold answers can't drift. `relational.jsonl` is generated
|
||||
* from RELATIONAL_QUESTIONS (a drift test pins them equal).
|
||||
*/
|
||||
|
||||
import type { BrainEngine } from '../../../../src/core/engine.ts';
|
||||
import type { ChunkInput } from '../../../../src/core/types.ts';
|
||||
import type { NamedThingQuestion } from '../../../../src/eval/retrieval-quality/harness.ts';
|
||||
|
||||
// edge maps: target entity → entities related to it via that edge type.
|
||||
// invested_in / led_round point INTO the company (people/funds → company).
|
||||
const INVESTMENTS: Record<string, string[]> = {
|
||||
'companies/widget-co': ['people/alice-example', 'people/bob-example', 'funds/fund-a'],
|
||||
'companies/acme-co': ['people/carol-example', 'people/alice-example', 'funds/fund-b'],
|
||||
'companies/novapay': ['people/bob-example', 'funds/fund-a'],
|
||||
'companies/mindbridge': ['people/dave-example', 'people/carol-example'],
|
||||
'companies/helio': ['people/erin-example', 'funds/fund-b'],
|
||||
'companies/quanta': ['people/frank-example', 'people/alice-example'],
|
||||
'companies/zenith': ['people/grace-example', 'people/heidi-example'],
|
||||
'companies/orbital': ['people/ivan-example', 'people/alice-example'],
|
||||
};
|
||||
const EMPLOYMENT: Record<string, string[]> = {
|
||||
'companies/widget-co': ['people/erin-example', 'people/grace-example'],
|
||||
'companies/acme-co': ['people/dave-example', 'people/frank-example'],
|
||||
'companies/novapay': ['people/grace-example'],
|
||||
'companies/helio': ['people/heidi-example'],
|
||||
'companies/zenith': ['people/ivan-example'],
|
||||
};
|
||||
const FOUNDED: Record<string, string[]> = {
|
||||
'companies/mindbridge': ['people/carol-example'],
|
||||
'companies/quanta': ['people/frank-example'],
|
||||
'companies/widget-co': ['people/ivan-example'],
|
||||
'companies/orbital': ['people/grace-example'],
|
||||
};
|
||||
const ADVISES: Record<string, string[]> = {
|
||||
'companies/novapay': ['people/frank-example'],
|
||||
'companies/helio': ['people/alice-example'],
|
||||
};
|
||||
|
||||
// Generic, cross-mention-free bodies keyed by slug prefix.
|
||||
function bodyFor(slug: string): string {
|
||||
if (slug.startsWith('companies/')) return 'A privately held company operating in its sector. Founded some years ago; details are sparse in this note.';
|
||||
if (slug.startsWith('funds/')) return 'An early-stage venture fund. Writes first checks; portfolio is not enumerated here.';
|
||||
return 'An individual in the network. Background and current focus are not described in this note.';
|
||||
}
|
||||
|
||||
function titleFor(slug: string): string {
|
||||
const tail = slug.split('/')[1];
|
||||
return tail.replace(/-/g, ' ').replace(/\b\w/g, c => c.toUpperCase());
|
||||
}
|
||||
|
||||
/** Every distinct slug in the graph. */
|
||||
function allSlugs(): string[] {
|
||||
const set = new Set<string>();
|
||||
for (const map of [INVESTMENTS, EMPLOYMENT, FOUNDED, ADVISES]) {
|
||||
for (const [company, members] of Object.entries(map)) {
|
||||
set.add(company);
|
||||
for (const m of members) set.add(m);
|
||||
}
|
||||
}
|
||||
return [...set].sort();
|
||||
}
|
||||
|
||||
function pageType(slug: string): 'company' | 'person' {
|
||||
// funds + people render as person-ish entity pages; companies as company.
|
||||
return slug.startsWith('companies/') ? 'company' : 'person';
|
||||
}
|
||||
|
||||
// Deterministic per-slug basis embedding so pages are properly indexed
|
||||
// (chunked) like a real brain. The query has no embedding in CI (no API key),
|
||||
// so vector search can't connect entities anyway — but the pages must carry a
|
||||
// chunk to be searchable at all. Body text stays generic + cross-mention-free.
|
||||
// `dim` MUST match the schema's embedding column, which tracks the configured
|
||||
// gateway default (1280 ZE / 1536 OpenAI) and can shift with shard order —
|
||||
// so callers probe the real width via probeEmbeddingDim rather than hardcode.
|
||||
function basisEmbedding(slug: string, dim: number): Float32Array {
|
||||
let h = 0;
|
||||
for (let i = 0; i < slug.length; i++) h = (h * 31 + slug.charCodeAt(i)) >>> 0;
|
||||
const e = new Float32Array(dim);
|
||||
e[h % dim] = 1.0;
|
||||
return e;
|
||||
}
|
||||
|
||||
/**
|
||||
* Probe the actual `content_chunks.embedding` column width. pgvector stores
|
||||
* the dimension in `atttypmod` directly. Tests must size fixtures to this, not
|
||||
* a hardcoded 1536 — the column tracks the gateway default and a prior test in
|
||||
* the same shard can leave it at 1280 (ZE). Mirrors pglite-engine.test.ts.
|
||||
*/
|
||||
export async function probeEmbeddingDim(engine: BrainEngine): Promise<number> {
|
||||
const db = (engine as unknown as { db: { query: (sql: string) => Promise<{ rows: Array<{ atttypmod: number }> }> } }).db;
|
||||
const r = await db.query(
|
||||
`SELECT atttypmod FROM pg_attribute
|
||||
WHERE attrelid = 'content_chunks'::regclass AND attname = 'embedding'`,
|
||||
);
|
||||
return r.rows[0].atttypmod;
|
||||
}
|
||||
|
||||
/** Seed the corpus into a fresh brain. Bodies never name related entities. */
|
||||
export async function seedRelationalCorpus(engine: BrainEngine): Promise<void> {
|
||||
const dim = await probeEmbeddingDim(engine);
|
||||
for (const slug of allSlugs()) {
|
||||
const body = bodyFor(slug);
|
||||
await engine.putPage(slug, {
|
||||
type: pageType(slug),
|
||||
title: titleFor(slug),
|
||||
compiled_truth: body,
|
||||
timeline: '',
|
||||
});
|
||||
await engine.upsertChunks(slug, [{
|
||||
chunk_index: 0,
|
||||
chunk_text: body,
|
||||
chunk_source: 'compiled_truth',
|
||||
embedding: basisEmbedding(slug, dim),
|
||||
token_count: body.split(/\s+/).length,
|
||||
}] satisfies ChunkInput[]);
|
||||
}
|
||||
const addAll = async (map: Record<string, string[]>, linkType: string) => {
|
||||
for (const [company, members] of Object.entries(map)) {
|
||||
for (const m of members) {
|
||||
// edge points member → company (member invested_in / works_at / founded / advises company)
|
||||
await engine.addLink(m, company, '', linkType, 'manual');
|
||||
}
|
||||
}
|
||||
};
|
||||
await addAll(INVESTMENTS, 'invested_in');
|
||||
await addAll(EMPLOYMENT, 'works_at');
|
||||
await addAll(FOUNDED, 'founded');
|
||||
await addAll(ADVISES, 'advises');
|
||||
}
|
||||
|
||||
function bareName(slug: string): string {
|
||||
return slug.split('/')[1];
|
||||
}
|
||||
|
||||
/** Generate the gold question set from the same maps used to seed. */
|
||||
export function buildRelationalQuestions(): NamedThingQuestion[] {
|
||||
const qs: NamedThingQuestion[] = [];
|
||||
|
||||
for (const [company, investors] of Object.entries(INVESTMENTS)) {
|
||||
qs.push({
|
||||
family: 'graph-relationship', kind: 'who_rel', seed: company, linkTypes: ['invested_in', 'led_round'],
|
||||
query: `who invested in ${bareName(company)}`, relevant: investors,
|
||||
});
|
||||
}
|
||||
for (const [company, employees] of Object.entries(EMPLOYMENT)) {
|
||||
qs.push({
|
||||
family: 'graph-relationship', kind: 'who_rel', seed: company, linkTypes: ['works_at'],
|
||||
query: `who works at ${bareName(company)}`, relevant: employees,
|
||||
});
|
||||
}
|
||||
for (const [company, founders] of Object.entries(FOUNDED)) {
|
||||
qs.push({
|
||||
family: 'graph-relationship', kind: 'who_rel', seed: company, linkTypes: ['founded'],
|
||||
query: `who founded ${bareName(company)}`, relevant: founders,
|
||||
});
|
||||
}
|
||||
for (const [company, advisors] of Object.entries(ADVISES)) {
|
||||
qs.push({
|
||||
family: 'graph-relationship', kind: 'who_rel', seed: company, linkTypes: ['advises'],
|
||||
query: `who advises ${bareName(company)}`, relevant: advisors,
|
||||
});
|
||||
}
|
||||
|
||||
// connects: company pairs that share at least one related entity (any edge).
|
||||
const relatedTo = (company: string): Set<string> => {
|
||||
const s = new Set<string>();
|
||||
for (const map of [INVESTMENTS, EMPLOYMENT, FOUNDED, ADVISES]) for (const m of map[company] ?? []) s.add(m);
|
||||
return s;
|
||||
};
|
||||
const companies = Object.keys(INVESTMENTS);
|
||||
for (let i = 0; i < companies.length; i++) {
|
||||
for (let j = i + 1; j < companies.length; j++) {
|
||||
const a = companies[i], b = companies[j];
|
||||
const shared = [...relatedTo(a)].filter(x => relatedTo(b).has(x));
|
||||
if (shared.length === 0) continue;
|
||||
qs.push({
|
||||
family: 'graph-relationship', kind: 'connects', seed: `${a} ↔ ${b}`, linkTypes: undefined,
|
||||
query: `what connects ${bareName(a)} and ${bareName(b)}`, relevant: shared.sort(),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return qs;
|
||||
}
|
||||
|
||||
export const RELATIONAL_QUESTIONS: NamedThingQuestion[] = buildRelationalQuestions();
|
||||
@@ -0,0 +1,42 @@
|
||||
// Relational benchmark — graph-relationship family. GENERATED from
|
||||
// test/fixtures/retrieval-quality/relational/corpus.ts (canonical source).
|
||||
// Answers are lexically unrecoverable — only the typed edge connects
|
||||
// query to answer. Regenerate: bun run scripts/_gen_relational_jsonl.ts
|
||||
{"family":"graph-relationship","kind":"who_rel","seed":"companies/widget-co","linkTypes":["invested_in","led_round"],"query":"who invested in widget-co","relevant":["people/alice-example","people/bob-example","funds/fund-a"]}
|
||||
{"family":"graph-relationship","kind":"who_rel","seed":"companies/acme-co","linkTypes":["invested_in","led_round"],"query":"who invested in acme-co","relevant":["people/carol-example","people/alice-example","funds/fund-b"]}
|
||||
{"family":"graph-relationship","kind":"who_rel","seed":"companies/novapay","linkTypes":["invested_in","led_round"],"query":"who invested in novapay","relevant":["people/bob-example","funds/fund-a"]}
|
||||
{"family":"graph-relationship","kind":"who_rel","seed":"companies/mindbridge","linkTypes":["invested_in","led_round"],"query":"who invested in mindbridge","relevant":["people/dave-example","people/carol-example"]}
|
||||
{"family":"graph-relationship","kind":"who_rel","seed":"companies/helio","linkTypes":["invested_in","led_round"],"query":"who invested in helio","relevant":["people/erin-example","funds/fund-b"]}
|
||||
{"family":"graph-relationship","kind":"who_rel","seed":"companies/quanta","linkTypes":["invested_in","led_round"],"query":"who invested in quanta","relevant":["people/frank-example","people/alice-example"]}
|
||||
{"family":"graph-relationship","kind":"who_rel","seed":"companies/zenith","linkTypes":["invested_in","led_round"],"query":"who invested in zenith","relevant":["people/grace-example","people/heidi-example"]}
|
||||
{"family":"graph-relationship","kind":"who_rel","seed":"companies/orbital","linkTypes":["invested_in","led_round"],"query":"who invested in orbital","relevant":["people/ivan-example","people/alice-example"]}
|
||||
{"family":"graph-relationship","kind":"who_rel","seed":"companies/widget-co","linkTypes":["works_at"],"query":"who works at widget-co","relevant":["people/erin-example","people/grace-example"]}
|
||||
{"family":"graph-relationship","kind":"who_rel","seed":"companies/acme-co","linkTypes":["works_at"],"query":"who works at acme-co","relevant":["people/dave-example","people/frank-example"]}
|
||||
{"family":"graph-relationship","kind":"who_rel","seed":"companies/novapay","linkTypes":["works_at"],"query":"who works at novapay","relevant":["people/grace-example"]}
|
||||
{"family":"graph-relationship","kind":"who_rel","seed":"companies/helio","linkTypes":["works_at"],"query":"who works at helio","relevant":["people/heidi-example"]}
|
||||
{"family":"graph-relationship","kind":"who_rel","seed":"companies/zenith","linkTypes":["works_at"],"query":"who works at zenith","relevant":["people/ivan-example"]}
|
||||
{"family":"graph-relationship","kind":"who_rel","seed":"companies/mindbridge","linkTypes":["founded"],"query":"who founded mindbridge","relevant":["people/carol-example"]}
|
||||
{"family":"graph-relationship","kind":"who_rel","seed":"companies/quanta","linkTypes":["founded"],"query":"who founded quanta","relevant":["people/frank-example"]}
|
||||
{"family":"graph-relationship","kind":"who_rel","seed":"companies/widget-co","linkTypes":["founded"],"query":"who founded widget-co","relevant":["people/ivan-example"]}
|
||||
{"family":"graph-relationship","kind":"who_rel","seed":"companies/orbital","linkTypes":["founded"],"query":"who founded orbital","relevant":["people/grace-example"]}
|
||||
{"family":"graph-relationship","kind":"who_rel","seed":"companies/novapay","linkTypes":["advises"],"query":"who advises novapay","relevant":["people/frank-example"]}
|
||||
{"family":"graph-relationship","kind":"who_rel","seed":"companies/helio","linkTypes":["advises"],"query":"who advises helio","relevant":["people/alice-example"]}
|
||||
{"family":"graph-relationship","kind":"connects","seed":"companies/widget-co ↔ companies/acme-co","query":"what connects widget-co and acme-co","relevant":["people/alice-example"]}
|
||||
{"family":"graph-relationship","kind":"connects","seed":"companies/widget-co ↔ companies/novapay","query":"what connects widget-co and novapay","relevant":["funds/fund-a","people/bob-example","people/grace-example"]}
|
||||
{"family":"graph-relationship","kind":"connects","seed":"companies/widget-co ↔ companies/helio","query":"what connects widget-co and helio","relevant":["people/alice-example","people/erin-example"]}
|
||||
{"family":"graph-relationship","kind":"connects","seed":"companies/widget-co ↔ companies/quanta","query":"what connects widget-co and quanta","relevant":["people/alice-example"]}
|
||||
{"family":"graph-relationship","kind":"connects","seed":"companies/widget-co ↔ companies/zenith","query":"what connects widget-co and zenith","relevant":["people/grace-example","people/ivan-example"]}
|
||||
{"family":"graph-relationship","kind":"connects","seed":"companies/widget-co ↔ companies/orbital","query":"what connects widget-co and orbital","relevant":["people/alice-example","people/grace-example","people/ivan-example"]}
|
||||
{"family":"graph-relationship","kind":"connects","seed":"companies/acme-co ↔ companies/novapay","query":"what connects acme-co and novapay","relevant":["people/frank-example"]}
|
||||
{"family":"graph-relationship","kind":"connects","seed":"companies/acme-co ↔ companies/mindbridge","query":"what connects acme-co and mindbridge","relevant":["people/carol-example","people/dave-example"]}
|
||||
{"family":"graph-relationship","kind":"connects","seed":"companies/acme-co ↔ companies/helio","query":"what connects acme-co and helio","relevant":["funds/fund-b","people/alice-example"]}
|
||||
{"family":"graph-relationship","kind":"connects","seed":"companies/acme-co ↔ companies/quanta","query":"what connects acme-co and quanta","relevant":["people/alice-example","people/frank-example"]}
|
||||
{"family":"graph-relationship","kind":"connects","seed":"companies/acme-co ↔ companies/orbital","query":"what connects acme-co and orbital","relevant":["people/alice-example"]}
|
||||
{"family":"graph-relationship","kind":"connects","seed":"companies/novapay ↔ companies/quanta","query":"what connects novapay and quanta","relevant":["people/frank-example"]}
|
||||
{"family":"graph-relationship","kind":"connects","seed":"companies/novapay ↔ companies/zenith","query":"what connects novapay and zenith","relevant":["people/grace-example"]}
|
||||
{"family":"graph-relationship","kind":"connects","seed":"companies/novapay ↔ companies/orbital","query":"what connects novapay and orbital","relevant":["people/grace-example"]}
|
||||
{"family":"graph-relationship","kind":"connects","seed":"companies/helio ↔ companies/quanta","query":"what connects helio and quanta","relevant":["people/alice-example"]}
|
||||
{"family":"graph-relationship","kind":"connects","seed":"companies/helio ↔ companies/zenith","query":"what connects helio and zenith","relevant":["people/heidi-example"]}
|
||||
{"family":"graph-relationship","kind":"connects","seed":"companies/helio ↔ companies/orbital","query":"what connects helio and orbital","relevant":["people/alice-example"]}
|
||||
{"family":"graph-relationship","kind":"connects","seed":"companies/quanta ↔ companies/orbital","query":"what connects quanta and orbital","relevant":["people/alice-example"]}
|
||||
{"family":"graph-relationship","kind":"connects","seed":"companies/zenith ↔ companies/orbital","query":"what connects zenith and orbital","relevant":["people/grace-example","people/ivan-example"]}
|
||||
@@ -0,0 +1,73 @@
|
||||
/**
|
||||
* Relational A/B proof + no-regression gate (PGLite, default CI).
|
||||
*
|
||||
* Seeds the lexically-unrecoverable relational corpus, then runs the gold
|
||||
* question set twice through bare hybridSearch — relational arm OFF vs ON —
|
||||
* and asserts:
|
||||
* 1. recall@10 on the graph-relationship family jumps materially with the
|
||||
* arm on (the answers are unreachable by keyword/vector).
|
||||
* 2. a non-relational content query returns the SAME results with the arm
|
||||
* on vs off (the arm is a true no-op off-target — the regression gate).
|
||||
*
|
||||
* This is the headline proof artifact for the feature: relational queries
|
||||
* that the corpus can't surface lexically jump from ~0 to high recall.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { hybridSearch } from '../src/core/search/hybrid.ts';
|
||||
import { runRetrievalQuality, parseQuestionsJsonl, type SearchFn } from '../src/eval/retrieval-quality/harness.ts';
|
||||
import { seedRelationalCorpus, RELATIONAL_QUESTIONS } from './fixtures/retrieval-quality/relational/corpus.ts';
|
||||
import { readFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
|
||||
let eng: PGLiteEngine;
|
||||
|
||||
beforeAll(async () => {
|
||||
eng = new PGLiteEngine();
|
||||
await eng.connect({});
|
||||
await eng.initSchema();
|
||||
await seedRelationalCorpus(eng);
|
||||
}, 60_000);
|
||||
|
||||
afterAll(async () => { await eng.disconnect(); });
|
||||
|
||||
const searchFnWith = (relationalRetrieval: boolean): SearchFn => async (q) => {
|
||||
const results = await hybridSearch(eng, q, { limit: 10, relationalRetrieval, expansion: false });
|
||||
return results.map(r => r.slug);
|
||||
};
|
||||
|
||||
describe('relational A/B', () => {
|
||||
test('corpus has a meaningful question set', () => {
|
||||
expect(RELATIONAL_QUESTIONS.length).toBeGreaterThanOrEqual(30);
|
||||
});
|
||||
|
||||
test('relational.jsonl matches the corpus module (no drift)', () => {
|
||||
const path = join(import.meta.dir, 'fixtures/retrieval-quality/relational/relational.jsonl');
|
||||
const fromFile = parseQuestionsJsonl(readFileSync(path, 'utf8'));
|
||||
expect(JSON.stringify(fromFile)).toBe(JSON.stringify(RELATIONAL_QUESTIONS));
|
||||
});
|
||||
|
||||
test('recall@10 lifts materially with the relational arm ON', async () => {
|
||||
const off = await runRetrievalQuality(RELATIONAL_QUESTIONS, searchFnWith(false));
|
||||
const on = await runRetrievalQuality(RELATIONAL_QUESTIONS, searchFnWith(true));
|
||||
|
||||
const offFam = off.families.find(f => f.family === 'graph-relationship')!;
|
||||
const onFam = on.families.find(f => f.family === 'graph-relationship')!;
|
||||
|
||||
// Answers are lexically unrecoverable → baseline recall is near zero.
|
||||
expect(offFam.recall_at_10).toBeLessThan(0.25);
|
||||
// Typed-edge arm surfaces them → large lift.
|
||||
expect(onFam.recall_at_10).toBeGreaterThan(0.75);
|
||||
expect(onFam.recall_at_10 - offFam.recall_at_10).toBeGreaterThan(0.5);
|
||||
// Hit@3 should also rise sharply.
|
||||
expect(onFam.hit_at_3).toBeGreaterThan(offFam.hit_at_3);
|
||||
}, 120_000);
|
||||
|
||||
test('no-regression: non-relational query is identical arm-on vs arm-off', async () => {
|
||||
const q = 'early-stage venture fund first checks';
|
||||
const off = await searchFnWith(false)(q);
|
||||
const on = await searchFnWith(true)(q);
|
||||
expect(on).toEqual(off);
|
||||
}, 60_000);
|
||||
});
|
||||
@@ -0,0 +1,115 @@
|
||||
/**
|
||||
* relationalFanout unit tests (PGLite, default CI).
|
||||
*
|
||||
* Exercises the typed-edge fan-out SQL directly so the engine method is
|
||||
* covered even when the DATABASE_URL-gated parity test does not run. Pins:
|
||||
* typed-edge filtering, mentions-excluded-by-default, deleted_at exclusion,
|
||||
* hop/edge_count aggregation, canonical_chunk_id, multi-seed + connects,
|
||||
* and determinism.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import type { ChunkInput } from '../src/core/types.ts';
|
||||
import { probeEmbeddingDim } from './fixtures/retrieval-quality/relational/corpus.ts';
|
||||
|
||||
function emb(idx: number, dim: number): Float32Array {
|
||||
const e = new Float32Array(dim);
|
||||
e[idx % dim] = 1.0;
|
||||
return e;
|
||||
}
|
||||
|
||||
let eng: PGLiteEngine;
|
||||
let DIM = 0;
|
||||
|
||||
beforeAll(async () => {
|
||||
eng = new PGLiteEngine();
|
||||
await eng.connect({});
|
||||
await eng.initSchema();
|
||||
DIM = await probeEmbeddingDim(eng); // match the schema's column width (1280 ZE / 1536 OpenAI)
|
||||
|
||||
const pages: Array<[string, string, string]> = [
|
||||
['companies/widget-co', 'company', 'Widget Co'],
|
||||
['companies/other-co', 'company', 'Other Co'],
|
||||
['people/investor-a', 'person', 'Investor A'],
|
||||
['people/investor-b', 'person', 'Investor B'],
|
||||
['people/employee-c', 'person', 'Employee C'],
|
||||
['people/mentioner', 'person', 'Mentioner'],
|
||||
['people/deleted-investor', 'person', 'Deleted Investor'],
|
||||
];
|
||||
for (const [slug, type, title] of pages) {
|
||||
await eng.putPage(slug, { type: type as 'company' | 'person', title, compiled_truth: `${title} body`, timeline: '' });
|
||||
}
|
||||
// investor-b carries a chunk (→ canonical_chunk_id non-null); investor-a stays chunkless.
|
||||
const chunk: ChunkInput[] = [{ chunk_index: 0, chunk_text: 'b', chunk_source: 'compiled_truth', embedding: emb(2, DIM), token_count: 1 }];
|
||||
await eng.upsertChunks('people/investor-b', chunk);
|
||||
|
||||
// Edges into widget-co.
|
||||
await eng.addLink('people/investor-a', 'companies/widget-co', '', 'invested_in', 'manual');
|
||||
await eng.addLink('people/investor-b', 'companies/widget-co', '', 'invested_in', 'manual');
|
||||
await eng.addLink('people/employee-c', 'companies/widget-co', '', 'works_at', 'manual');
|
||||
await eng.addLink('people/mentioner', 'companies/widget-co', '', 'mentions', 'mentions');
|
||||
await eng.addLink('people/deleted-investor', 'companies/widget-co', '', 'invested_in', 'manual');
|
||||
// investor-a also invested in other-co → widget-co and other-co connect via investor-a.
|
||||
await eng.addLink('people/investor-a', 'companies/other-co', '', 'invested_in', 'manual');
|
||||
|
||||
// Soft-delete one investor; it must never surface.
|
||||
await eng.executeRaw(`UPDATE pages SET deleted_at = now() WHERE slug = $1`, ['people/deleted-investor']);
|
||||
}, 60_000);
|
||||
|
||||
afterAll(async () => {
|
||||
await eng.disconnect();
|
||||
});
|
||||
|
||||
describe('relationalFanout', () => {
|
||||
test('typed-edge: who invested in widget-co', async () => {
|
||||
const rows = await eng.relationalFanout(['companies/widget-co'], { direction: 'in', linkTypes: ['invested_in'] });
|
||||
const slugs = rows.map(r => r.slug).sort();
|
||||
expect(slugs).toEqual(['people/investor-a', 'people/investor-b']);
|
||||
for (const r of rows) {
|
||||
expect(r.hop).toBe(1);
|
||||
expect(r.via_link_types).toEqual(['invested_in']);
|
||||
}
|
||||
});
|
||||
|
||||
test('deleted pages are excluded', async () => {
|
||||
const rows = await eng.relationalFanout(['companies/widget-co'], { direction: 'in', linkTypes: ['invested_in'] });
|
||||
expect(rows.map(r => r.slug)).not.toContain('people/deleted-investor');
|
||||
});
|
||||
|
||||
test('mentions excluded by default, included on opt-in', async () => {
|
||||
const off = await eng.relationalFanout(['companies/widget-co'], { direction: 'in' });
|
||||
expect(off.map(r => r.slug)).not.toContain('people/mentioner');
|
||||
// type-agnostic also picks up the works_at neighbor
|
||||
expect(off.map(r => r.slug).sort()).toEqual(['people/employee-c', 'people/investor-a', 'people/investor-b']);
|
||||
|
||||
const on = await eng.relationalFanout(['companies/widget-co'], { direction: 'in', includeMentions: true });
|
||||
expect(on.map(r => r.slug)).toContain('people/mentioner');
|
||||
});
|
||||
|
||||
test('canonical_chunk_id: non-null for chunked page, null for chunkless', async () => {
|
||||
const rows = await eng.relationalFanout(['companies/widget-co'], { direction: 'in', linkTypes: ['invested_in'] });
|
||||
const a = rows.find(r => r.slug === 'people/investor-a')!;
|
||||
const b = rows.find(r => r.slug === 'people/investor-b')!;
|
||||
expect(a.canonical_chunk_id).toBeNull();
|
||||
expect(b.canonical_chunk_id).not.toBeNull();
|
||||
});
|
||||
|
||||
test('connects: shared midpoint reachable from both seeds with a path', async () => {
|
||||
const rows = await eng.relationalFanout(['companies/widget-co', 'companies/other-co'], { direction: 'both' });
|
||||
const a = rows.find(r => r.slug === 'people/investor-a');
|
||||
expect(a).toBeDefined();
|
||||
expect(a!.path.length).toBeGreaterThanOrEqual(2); // [seed, ..., node]
|
||||
expect(a!.path[a!.path.length - 1]).toBe('people/investor-a');
|
||||
});
|
||||
|
||||
test('empty seeds → []', async () => {
|
||||
expect(await eng.relationalFanout([])).toEqual([]);
|
||||
});
|
||||
|
||||
test('deterministic: same call twice is byte-identical', async () => {
|
||||
const r1 = await eng.relationalFanout(['companies/widget-co'], { direction: 'in' });
|
||||
const r2 = await eng.relationalFanout(['companies/widget-co'], { direction: 'in' });
|
||||
expect(JSON.stringify(r1)).toBe(JSON.stringify(r2));
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,136 @@
|
||||
/**
|
||||
* Relational-query parser unit tests.
|
||||
*
|
||||
* Covers the four archetypes, the no-match path, the precision-first
|
||||
* false-positive guard, schema-pack vocab extension, subset validation, and
|
||||
* the length bound (ReDoS surface).
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import {
|
||||
parseRelationalQuery,
|
||||
validateVocab,
|
||||
KNOWN_LINK_TYPES,
|
||||
type RelationVocab,
|
||||
} from '../src/core/search/relational-intent.ts';
|
||||
|
||||
describe('parseRelationalQuery — archetypes', () => {
|
||||
test('who_rel: who invested in widget-co', () => {
|
||||
const r = parseRelationalQuery('who invested in widget-co');
|
||||
expect(r).not.toBeNull();
|
||||
expect(r!.kind).toBe('who_rel');
|
||||
expect(r!.seeds).toEqual(['widget-co']);
|
||||
expect(r!.linkTypes).toEqual(['invested_in', 'led_round']);
|
||||
expect(r!.direction).toBe('in');
|
||||
});
|
||||
|
||||
test('who_rel: who founded acme (article stripped)', () => {
|
||||
const r = parseRelationalQuery('who founded the acme corporation?');
|
||||
expect(r!.kind).toBe('who_rel');
|
||||
expect(r!.seeds).toEqual(['acme corporation']);
|
||||
expect(r!.linkTypes).toEqual(['founded']);
|
||||
});
|
||||
|
||||
test('who_at: who at acme works on payments', () => {
|
||||
const r = parseRelationalQuery('who at acme works on payments');
|
||||
expect(r!.kind).toBe('who_at');
|
||||
expect(r!.seeds).toEqual(['acme']);
|
||||
expect(r!.linkTypes).toEqual(['works_at']);
|
||||
expect(r!.direction).toBe('in');
|
||||
});
|
||||
|
||||
test('intro: who introduced me to alice-example (type-agnostic)', () => {
|
||||
const r = parseRelationalQuery('who introduced me to alice-example?');
|
||||
expect(r!.kind).toBe('intro');
|
||||
expect(r!.seeds).toEqual(['alice-example']);
|
||||
expect(r!.linkTypes).toBeNull();
|
||||
expect(r!.direction).toBe('both');
|
||||
});
|
||||
|
||||
test('connects: what connects fund-a and fund-b (two seeds)', () => {
|
||||
const r = parseRelationalQuery('what connects fund-a and fund-b');
|
||||
expect(r!.kind).toBe('connects');
|
||||
expect(r!.seeds).toEqual(['fund-a', 'fund-b']);
|
||||
expect(r!.linkTypes).toBeNull();
|
||||
expect(r!.direction).toBe('both');
|
||||
});
|
||||
|
||||
test('connects: how are X and Y related', () => {
|
||||
const r = parseRelationalQuery('how are widget-co and acme related');
|
||||
expect(r!.kind).toBe('connects');
|
||||
expect(r!.seeds).toEqual(['widget-co', 'acme']);
|
||||
});
|
||||
|
||||
test('outgoing: what did alice invest in', () => {
|
||||
const r = parseRelationalQuery('what did alice invest in');
|
||||
expect(r!.kind).toBe('who_rel');
|
||||
expect(r!.seeds).toEqual(['alice']);
|
||||
expect(r!.direction).toBe('out');
|
||||
});
|
||||
});
|
||||
|
||||
describe('parseRelationalQuery — precision-first / no-match', () => {
|
||||
test('non-relational content query → null', () => {
|
||||
expect(parseRelationalQuery('what is the capital structure of a seed round')).toBeNull();
|
||||
expect(parseRelationalQuery('notes from the offsite')).toBeNull();
|
||||
expect(parseRelationalQuery('summarize the q3 board deck')).toBeNull();
|
||||
});
|
||||
|
||||
test('false-positive: "who invested TIME in learning Rust" does NOT match', () => {
|
||||
// "invested time in" is not "invested in" — adjacency guard.
|
||||
expect(parseRelationalQuery('who invested time in learning Rust')).toBeNull();
|
||||
});
|
||||
|
||||
test('pronoun / stopword seed is rejected', () => {
|
||||
expect(parseRelationalQuery('who invested in it')).toBeNull();
|
||||
expect(parseRelationalQuery('who founded them?')).toBeNull();
|
||||
});
|
||||
|
||||
test('empty / overlong input → null', () => {
|
||||
expect(parseRelationalQuery('')).toBeNull();
|
||||
expect(parseRelationalQuery('who invested in ' + 'x'.repeat(600))).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('schema-pack vocab extension (D2=B)', () => {
|
||||
const vocab: RelationVocab = {
|
||||
extraVerbs: [{ verb: 'related to|associated with', linkTypes: ['related_to'], direction: 'both' }],
|
||||
};
|
||||
|
||||
test('extra verb extends detection', () => {
|
||||
expect(parseRelationalQuery('who related to widget-co', vocab)!.linkTypes).toEqual(['related_to']);
|
||||
// same query without the pack does NOT match the extra verb
|
||||
expect(parseRelationalQuery('who related to widget-co')).toBeNull();
|
||||
});
|
||||
|
||||
test('validateVocab passes for known types', () => {
|
||||
expect(() => validateVocab(vocab)).not.toThrow();
|
||||
});
|
||||
|
||||
test('validateVocab throws on unknown link_type', () => {
|
||||
expect(() =>
|
||||
validateVocab({ extraVerbs: [{ verb: 'pwns', linkTypes: ['not_a_real_edge'], direction: 'in' }] }),
|
||||
).toThrow(/unknown link_type/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('default bank emits only known link types (no drift)', () => {
|
||||
test('every parsed linkType is a subset of KNOWN_LINK_TYPES', () => {
|
||||
const queries = [
|
||||
'who invested in widget-co',
|
||||
'who founded acme',
|
||||
'who advises bob-example',
|
||||
'who works at acme',
|
||||
'who at acme leads payments',
|
||||
'what did alice invest in',
|
||||
'where does alice work',
|
||||
];
|
||||
for (const q of queries) {
|
||||
const r = parseRelationalQuery(q);
|
||||
if (!r || r.linkTypes === null) continue;
|
||||
for (const lt of r.linkTypes) {
|
||||
expect(KNOWN_LINK_TYPES.has(lt)).toBe(true);
|
||||
}
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,72 @@
|
||||
/**
|
||||
* Relational recall arm integration tests (PGLite, default CI).
|
||||
*
|
||||
* End-to-end through buildRelationalArm: parse → resolve seed → fanout →
|
||||
* hydrate. Pins the lexically-unrecoverable win (the investor page never names
|
||||
* the company; only the invested_in edge connects them), the non-relational
|
||||
* no-op, attribution stamping, and fail-open.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { buildRelationalArm } from '../src/core/search/relational-recall.ts';
|
||||
import { probeEmbeddingDim } from './fixtures/retrieval-quality/relational/corpus.ts';
|
||||
import type { ChunkInput } from '../src/core/types.ts';
|
||||
|
||||
let eng: PGLiteEngine;
|
||||
|
||||
beforeAll(async () => {
|
||||
eng = new PGLiteEngine();
|
||||
await eng.connect({});
|
||||
await eng.initSchema();
|
||||
const dim = await probeEmbeddingDim(eng); // match schema column width (1280 ZE / 1536 OpenAI)
|
||||
|
||||
await eng.putPage('companies/widget-co', { type: 'company', title: 'Widget Co', compiled_truth: 'A payments company.', timeline: '' });
|
||||
// The investor's body deliberately NEVER mentions Widget Co — only the edge connects them.
|
||||
await eng.putPage('people/alice-example', { type: 'person', title: 'Alice Example', compiled_truth: 'Alice is a seed-stage investor based in Lisbon.', timeline: '' });
|
||||
await eng.upsertChunks('people/alice-example', [{
|
||||
chunk_index: 0, chunk_text: 'Alice is a seed-stage investor based in Lisbon.',
|
||||
chunk_source: 'compiled_truth', embedding: new Float32Array(dim), token_count: 8,
|
||||
}] satisfies ChunkInput[]);
|
||||
await eng.addLink('people/alice-example', 'companies/widget-co', '', 'invested_in', 'manual');
|
||||
}, 60_000);
|
||||
|
||||
afterAll(async () => { await eng.disconnect(); });
|
||||
|
||||
describe('buildRelationalArm', () => {
|
||||
test('surfaces the edge answer that lexical search would miss', async () => {
|
||||
const list = await buildRelationalArm(eng, 'who invested in widget-co');
|
||||
const alice = list.find(r => r.slug === 'people/alice-example');
|
||||
expect(alice).toBeDefined();
|
||||
expect(alice!.relational_via_link_types).toEqual(['invested_in']);
|
||||
expect(alice!.relational_hop).toBe(1);
|
||||
expect(alice!.relational_seed).toBe('companies/widget-co');
|
||||
// chunk-backed page → reinforces a REAL chunk id (not synthetic 0).
|
||||
expect(alice!.chunk_id).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
test('non-relational query is a pure no-op', async () => {
|
||||
const meta: { fired?: boolean } = {};
|
||||
const list = await buildRelationalArm(eng, 'summary of the payments roadmap', { onMeta: m => { meta.fired = m.fired; } });
|
||||
expect(list).toEqual([]);
|
||||
expect(meta.fired).toBe(false);
|
||||
});
|
||||
|
||||
test('unresolvable seed → no-op (never traverse from a guess)', async () => {
|
||||
const list = await buildRelationalArm(eng, 'who invested in nonexistent-phantom-xyz');
|
||||
expect(list).toEqual([]);
|
||||
});
|
||||
|
||||
test('fail-open: fanout error returns [] + errored meta, never throws', async () => {
|
||||
const original = eng.relationalFanout.bind(eng);
|
||||
let captured: { errored?: boolean } = {};
|
||||
eng.relationalFanout = async () => { throw new Error('boom'); };
|
||||
try {
|
||||
const list = await buildRelationalArm(eng, 'who invested in widget-co', { onMeta: m => { captured = m; } });
|
||||
expect(list).toEqual([]);
|
||||
expect(captured.errored).toBe(true);
|
||||
} finally {
|
||||
eng.relationalFanout = original;
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,70 @@
|
||||
/**
|
||||
* RRF fusion key is source-aware (federation hardening).
|
||||
*
|
||||
* Pre-fix the RRF/dedup key was `slug:chunk_id`, which collapsed two
|
||||
* same-slug pages living in different federated sources into one fusion
|
||||
* entry — a cross-source recall bug. The key is now
|
||||
* `(source_id, slug, chunk_id)`, matching `dedup.ts:pageKey`'s composite
|
||||
* discipline at chunk granularity. These tests pin that behavior for both
|
||||
* `rrfFusion` and `rrfFusionWeighted` and confirm single-source ranking is
|
||||
* unchanged.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { rrfFusion, rrfFusionWeighted } from '../src/core/search/hybrid.ts';
|
||||
import type { SearchResult } from '../src/core/types.ts';
|
||||
|
||||
function makeResult(overrides: Partial<SearchResult> = {}): SearchResult {
|
||||
return {
|
||||
slug: 'test-page',
|
||||
page_id: 1,
|
||||
title: 'Test',
|
||||
type: 'concept',
|
||||
chunk_text: 'unique chunk text',
|
||||
chunk_source: 'timeline', // avoid compiled_truth 2x boost noise
|
||||
chunk_id: 1,
|
||||
chunk_index: 0,
|
||||
score: 0.5,
|
||||
stale: false,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
describe('RRF key is source-aware', () => {
|
||||
test('same slug + chunk_id in DIFFERENT sources do NOT collapse', () => {
|
||||
const a = makeResult({ slug: 'people/alice', chunk_id: 1, source_id: 'team-a', chunk_text: 'x' });
|
||||
const b = makeResult({ slug: 'people/alice', chunk_id: 1, source_id: 'team-b', chunk_text: 'y' });
|
||||
const out = rrfFusion([[a, b]], 60);
|
||||
expect(out.length).toBe(2);
|
||||
expect(new Set(out.map(r => r.source_id))).toEqual(new Set(['team-a', 'team-b']));
|
||||
});
|
||||
|
||||
test('same slug + chunk_id + SAME source DO collapse (reinforce across lists)', () => {
|
||||
const a = makeResult({ slug: 'people/alice', chunk_id: 1, source_id: 'team-a', chunk_text: 'x' });
|
||||
const b = makeResult({ slug: 'people/alice', chunk_id: 1, source_id: 'team-a', chunk_text: 'x' });
|
||||
const out = rrfFusion([[a], [b]], 60);
|
||||
expect(out.length).toBe(1);
|
||||
});
|
||||
|
||||
test('single-source (no source_id → "default") ranking unchanged', () => {
|
||||
const a = makeResult({ slug: 'a', chunk_id: 1, chunk_text: 'aaa' });
|
||||
const b = makeResult({ slug: 'b', chunk_id: 2, chunk_text: 'bbb' });
|
||||
const out = rrfFusion([[a, b]], 60);
|
||||
expect(out.length).toBe(2);
|
||||
expect(out[0].slug).toBe('a'); // rank-0 wins, as before the key change
|
||||
});
|
||||
|
||||
test('chunkless rows (null chunk_id) stay distinct per source via text prefix', () => {
|
||||
const a = makeResult({ slug: 'people/alice', chunk_id: undefined as unknown as number, source_id: 'team-a', chunk_text: 'alice page' });
|
||||
const b = makeResult({ slug: 'people/alice', chunk_id: undefined as unknown as number, source_id: 'team-b', chunk_text: 'alice page' });
|
||||
const out = rrfFusion([[a, b]], 60);
|
||||
expect(out.length).toBe(2);
|
||||
});
|
||||
|
||||
test('weighted RRF is also source-aware', () => {
|
||||
const a = makeResult({ slug: 'people/alice', chunk_id: 1, source_id: 'team-a' });
|
||||
const b = makeResult({ slug: 'people/alice', chunk_id: 1, source_id: 'team-b' });
|
||||
const out = rrfFusionWeighted([{ list: [a, b], k: 60 }]);
|
||||
expect(out.length).toBe(2);
|
||||
});
|
||||
});
|
||||
@@ -89,7 +89,7 @@ describe('alias_resolved boost stage', () => {
|
||||
});
|
||||
|
||||
describe('KNOBS_HASH_VERSION', () => {
|
||||
it('is 9 (8→9 archive-demote invalidates archive-excluded cache rows, #1777)', () => {
|
||||
expect(KNOBS_HASH_VERSION).toBe(9);
|
||||
it('is 10 (9→10 relational recall arm invalidates rel-off cache rows, v0.43)', () => {
|
||||
expect(KNOBS_HASH_VERSION).toBe(10);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -77,6 +77,9 @@ describe('SEARCH_MODES + MODE_BUNDLES canonical shape', () => {
|
||||
// v0.42.3.0 — autocut OFF for conservative (no reranker).
|
||||
autocut: false,
|
||||
autocut_jump: 0.2,
|
||||
// v0.43 — relational recall OFF for conservative.
|
||||
relationalRetrieval: false,
|
||||
relational_retrieval_depth: 2,
|
||||
});
|
||||
});
|
||||
|
||||
@@ -106,6 +109,9 @@ describe('SEARCH_MODES + MODE_BUNDLES canonical shape', () => {
|
||||
// v0.42.3.0 — autocut ON.
|
||||
autocut: true,
|
||||
autocut_jump: 0.2,
|
||||
// v0.43 — relational recall ON for balanced.
|
||||
relationalRetrieval: true,
|
||||
relational_retrieval_depth: 2,
|
||||
});
|
||||
});
|
||||
|
||||
@@ -133,6 +139,9 @@ describe('SEARCH_MODES + MODE_BUNDLES canonical shape', () => {
|
||||
// v0.42.3.0 — autocut ON.
|
||||
autocut: true,
|
||||
autocut_jump: 0.2,
|
||||
// v0.43 — relational recall ON for tokenmax.
|
||||
relationalRetrieval: true,
|
||||
relational_retrieval_depth: 2,
|
||||
});
|
||||
});
|
||||
|
||||
@@ -392,7 +401,9 @@ describe('knobsHash determinism + cross-mode separation (CDX-4)', () => {
|
||||
// archive/ demote (search-exclude policy change isn't in the hash, so the
|
||||
// version bump is what invalidates archive-excluded cache rows). A query
|
||||
// must not be served from a cache row written before the policy change.
|
||||
expect(KNOBS_HASH_VERSION).toBe(9);
|
||||
// v0.43: bumped 9→10 for the relational recall arm (rel=/reld=) — a
|
||||
// relational-on write must not be served to a relational-off lookup.
|
||||
expect(KNOBS_HASH_VERSION).toBe(10);
|
||||
});
|
||||
|
||||
test('T1 (codex): floor_ratio set vs unset produces DIFFERENT hashes (cache contamination prevention)', () => {
|
||||
@@ -557,8 +568,8 @@ describe('v0.40.4 — graph_signals knob', () => {
|
||||
});
|
||||
|
||||
describe('v0.42.3.0 — autocut knobs', () => {
|
||||
test('KNOBS_HASH_VERSION is 9 (8→9 archive-demote, issue #1777)', () => {
|
||||
expect(KNOBS_HASH_VERSION).toBe(9);
|
||||
test('KNOBS_HASH_VERSION is 10 (9→10 relational recall arm, v0.43)', () => {
|
||||
expect(KNOBS_HASH_VERSION).toBe(10);
|
||||
});
|
||||
|
||||
test('bundle defaults: conservative off, balanced/tokenmax on @0.20', () => {
|
||||
@@ -621,3 +632,44 @@ describe('v0.42.3.0 — autocut knobs', () => {
|
||||
expect(attr.value).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('v0.43 — relational recall knobs', () => {
|
||||
test('bundle defaults: conservative off, balanced/tokenmax on; depth 2', () => {
|
||||
expect(MODE_BUNDLES.conservative.relationalRetrieval).toBe(false);
|
||||
expect(MODE_BUNDLES.balanced.relationalRetrieval).toBe(true);
|
||||
expect(MODE_BUNDLES.tokenmax.relationalRetrieval).toBe(true);
|
||||
for (const m of ['conservative', 'balanced', 'tokenmax'] as const) {
|
||||
expect(MODE_BUNDLES[m].relational_retrieval_depth).toBe(2);
|
||||
}
|
||||
});
|
||||
|
||||
test('per-call relationalRetrieval:false overrides bundle/config true', () => {
|
||||
// bundle default true (balanced); per-call false wins
|
||||
expect(resolveSearchMode({ mode: 'balanced', perCall: { relationalRetrieval: false } }).relationalRetrieval).toBe(false);
|
||||
// config override true on conservative (bundle false); then per-call false beats config
|
||||
expect(
|
||||
resolveSearchMode({ mode: 'conservative', overrides: { relationalRetrieval: true }, perCall: { relationalRetrieval: false } }).relationalRetrieval,
|
||||
).toBe(false);
|
||||
// config override alone flips conservative on
|
||||
expect(resolveSearchMode({ mode: 'conservative', overrides: { relationalRetrieval: true } }).relationalRetrieval).toBe(true);
|
||||
});
|
||||
|
||||
test('loadOverridesFromConfig reads search.relational_retrieval(+_depth)', () => {
|
||||
const ov = loadOverridesFromConfig({ 'search.relational_retrieval': 'true', 'search.relational_retrieval_depth': '3' });
|
||||
expect(ov.relationalRetrieval).toBe(true);
|
||||
expect(ov.relational_retrieval_depth).toBe(3);
|
||||
// out-of-range depth is ignored (falls through to bundle)
|
||||
expect(loadOverridesFromConfig({ 'search.relational_retrieval_depth': '9' }).relational_retrieval_depth).toBeUndefined();
|
||||
});
|
||||
|
||||
test('SEARCH_MODE_CONFIG_KEYS includes the relational keys', () => {
|
||||
expect(SEARCH_MODE_CONFIG_KEYS).toContain('search.relational_retrieval');
|
||||
expect(SEARCH_MODE_CONFIG_KEYS).toContain('search.relational_retrieval_depth');
|
||||
});
|
||||
|
||||
test('knobsHash: relational-on vs off differ (cache isolation)', () => {
|
||||
const on = knobsHash(resolveSearchMode({ mode: 'balanced' })); // relational true
|
||||
const off = knobsHash(resolveSearchMode({ mode: 'balanced', perCall: { relationalRetrieval: false } }));
|
||||
expect(on).not.toBe(off);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -43,7 +43,7 @@ function baseKnobs(): ResolvedSearchKnobs {
|
||||
}
|
||||
|
||||
describe('KNOBS_HASH_VERSION + version invariants', () => {
|
||||
test('version is 9 (…; 6→7 title_boost; 7→8 autocut; 8→9 archive-demote #1777)', () => {
|
||||
test('version is 10 (…; 7→8 autocut; 8→9 archive-demote #1777; 9→10 relational recall)', () => {
|
||||
// v0.35.0.0: 1→2 to fold reranker fields. v0.35.6.0: 2→3 to fold
|
||||
// floor_ratio. v0.36 wave: piggybacks on v=3 with 7 cross-modal knobs
|
||||
// (D2) PLUS column + provider context (D8/CDX-2 cross-column isolation).
|
||||
@@ -57,7 +57,8 @@ describe('KNOBS_HASH_VERSION + version invariants', () => {
|
||||
// cannot leak past the new stage. T2: 6→7 title_boost. v0.42.3.0: 7→8
|
||||
// autocut. issue #1777: 8→9 archive/ demote (search-exclude policy change
|
||||
// isn't in the hash, so the bump invalidates archive-excluded cache rows).
|
||||
expect(KNOBS_HASH_VERSION).toBe(9);
|
||||
// v0.43: 9→10 relational recall arm (rel=/reld=).
|
||||
expect(KNOBS_HASH_VERSION).toBe(10);
|
||||
});
|
||||
|
||||
test('hash is 16 hex chars regardless of reranker config', () => {
|
||||
|
||||
Reference in New Issue
Block a user