fix(validation): qualify backlink endpoint identity (#3667)

Adversarial review: survived a hostile reviewer plus two independent refuters, each told to assume the PR was broken and to default to refuting when uncertain. 19 of 62 PRs cleared that bar.

The back-link validator compared bare slugs, so a same-slug page in another source masked a genuinely missing reverse edge — silent under-reporting in exactly the multi-source setup where it matters. Now keyed on the full 4-tuple, per the `(source_id, slug)` uniqueness invariant. Verified on real Docker Postgres with 28/28 parity, which the PR itself had skipped.

Verified before merge: the PR's own tests fail when the production change is reverted; typecheck clean; MERGEABLE/CLEAN on the current base after batches 1 and 2 landed, not a stale one.

Known gap, recorded rather than hidden: remote MCP serialization of the additive Link fields is untested; the fields are additive JSON.
This commit is contained in:
daragao3
2026-08-01 03:39:12 +08:00
committed by GitHub
parent 7376c0266e
commit dba0ae7b1e
14 changed files with 740 additions and 59 deletions
@@ -0,0 +1,175 @@
# Scalar-source Backlink Validation Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Make backlink validation compare exact `(source_id, slug)` endpoint identities while preserving existing scalar, unscoped, and federated link-read semantics.
**Architecture:** Enrich every engine link-read row with the source identity of its joined from, to, and visible origin pages. Pass the validated page's scalar or federated scope into validator context; the backlink validator scopes its initial read consistently, groups targets by exact identity, and accepts only an exact reverse endpoint pair. SQL predicates remain unchanged, so trusted scalar cross-source visibility and federated all-endpoint containment remain intact.
**Tech Stack:** TypeScript, Bun test, PGLite, PostgreSQL/postgres.js.
## Global Constraints
- Use strict red-before-green TDD with duplicate slugs across sources.
- Preserve unscoped historical reads, scalar near-endpoint scoping, scalar explicit cross-source visibility, federated all-endpoint containment, and `sourceIds` precedence.
- Keep PostgreSQL and PGLite projections in parity.
- Do not change schema or conditional-write conflict semantics.
- Keep deployment, restart, migration, and push actions outside the implementation tasks; a separately authorized release workflow may perform them after verification.
- Capture full test output to files before inspecting it.
---
### Task 1: Pin the backlink false-negative in PGLite
**Files:**
- Modify: `test/writer.test.ts`
**Interfaces:**
- Consumes: `backLinkValidator.validate(PageValidationContext)` and source-qualified `putPage`/`addLink`.
- Produces: regressions for wrong-source reverse rejection, exact reverse acceptance, cross-source pair acceptance, and exact target deduplication.
- [ ] **Step 1: Add the minimal failing duplicate-slug regression**
Create `default` and `team-x` copies of the origin and target, add `(team-x, origin) -> (team-x, target)` plus the wrong reverse `(team-x, target) -> (default, origin)`, validate with `sourceId: 'team-x'`, and require one warning.
- [ ] **Step 2: Run the focused test and verify RED**
```bash
bun test test/writer.test.ts -t "wrong-source reverse" > "$TEMP/backlink-red.txt" 2>&1
```
Expected: assertion failure because current slug-only validation returns zero findings.
- [ ] **Step 3: Add the remaining behavioral regressions after the first red is recorded**
Add tests proving that the exact reverse clears the warning, a legitimate cross-source forward/reverse pair passes, and two destinations sharing one slug but differing by source are validated independently.
### Task 2: Expose exact endpoint identity from both engines
**Files:**
- Modify: `src/core/types.ts:1204-1229`
- Modify: `src/core/postgres-engine.ts:3021-3124`
- Modify: `src/core/pglite-engine.ts:2941-3037`
- Modify: `test/get-page-federated-scope.test.ts:187-246,289-306`
- Modify: `test/e2e/multi-source-bug-class.test.ts:184-205`
- Modify: `test/e2e/engine-parity.test.ts:813-875`
**Interfaces:**
- Produces: `Link.from_source_id: string`, `Link.to_source_id: string`, and `Link.origin_source_id?: string | null`.
- Preserves: `getLinks(slug, { sourceId?, sourceIds? })` and `getBacklinks(...)` filtering semantics.
- [ ] **Step 1: Add engine-contract assertions before implementation**
Assert scalar cross-source rows expose `beta -> default`, federated rows expose only in-grant endpoint IDs, `sourceIds` still beats scalar `sourceId`, and an out-of-grant origin has both `origin_slug` and `origin_source_id` null.
- [ ] **Step 2: Run the focused contract tests and verify RED**
```bash
bun test test/get-page-federated-scope.test.ts test/e2e/multi-source-bug-class.test.ts > "$TEMP/link-identity-red.txt" 2>&1
```
Expected: source-ID assertions fail because fields are absent.
- [ ] **Step 3: Extend `Link` and project IDs without changing predicates**
Use this additive contract:
```ts
export interface Link {
from_slug: string;
from_source_id: string;
to_slug: string;
to_source_id: string;
link_type: string;
context: string;
link_source?: string | null;
origin_slug?: string | null;
origin_source_id?: string | null;
origin_field?: string | null;
}
```
In all six branches per engine, project:
```sql
f.source_id AS from_source_id,
t.source_id AS to_source_id,
o.source_id AS origin_source_id
```
Keep every `WHERE` and grant-aware origin `LEFT JOIN` unchanged.
- [ ] **Step 4: Re-run contract tests and verify GREEN**
Use the same command and require all focused tests to pass.
### Task 3: Validate exact reverse identities and propagate scope
**Files:**
- Modify: `src/core/output/writer.ts:89-96,240-318`
- Modify: `src/core/output/post-write.ts:36-41,73-118`
- Modify: `src/core/output/validators/back-link.ts:24-47`
- Modify: `src/core/operations.ts:1227-1246`
- Modify: `test/post-write-lint.test.ts:67-130`
**Interfaces:**
- Produces: optional `PageValidationContext.sourceId` and `sourceIds`, with `sourceIds` taking precedence.
- `runPostWriteLint(..., opts)` accepts the same optional scope and loads the validated page through it.
- [ ] **Step 1: Add a post-write nested-read regression and verify RED**
Validate a non-default page with a wrong-source reverse via `runPostWriteLint(..., { force: true, noLog: true, sourceId: 'team-x' })`; require a backlink warning.
- [ ] **Step 2: Implement minimal scope propagation**
Add `sourceId?`/`sourceIds?` to validation context and lint options. Load pages using `sourceIds` when non-empty, otherwise scalar `sourceId`. Pass the same scope into nested validators. In the put-page success hook, call lint with the already-resolved write source ID.
- [ ] **Step 3: Implement exact backlink matching**
Initial outbound reads use the validation scope. Deduplicate rows by all four endpoint identity fields so every distinct expected origin remains represented even when targets share a source-qualified identity. Read each target using the federated grant when present, otherwise the target's exact scalar source. Accept only a row matching all four endpoint fields of the expected reverse.
- [ ] **Step 4: Run writer and post-write tests and verify GREEN**
```bash
bun test test/writer.test.ts test/post-write-lint.test.ts > "$TEMP/backlink-green.txt" 2>&1
```
Expected: all tests pass, including the recorded false-negative.
### Task 4: Verify PostgreSQL/PGLite parity and final scope
**Files:**
- Modify: `test/e2e/engine-parity.test.ts:813-875`
- Verify: all files above
**Interfaces:**
- Consumes: exact endpoint fields and unchanged filtering semantics.
- Produces: parity evidence for scalar cross-source and federated reads.
- [ ] **Step 1: Compare complete endpoint tuples across engines**
Compare sorted tuples containing `from_source_id`, `from_slug`, `to_source_id`, `to_slug`, `origin_source_id`, and `origin_slug` for scalar and federated fixtures.
- [ ] **Step 2: Run focused PGLite/source-isolation tests**
```bash
bun test test/writer.test.ts test/post-write-lint.test.ts test/get-page-federated-scope.test.ts test/e2e/multi-source-bug-class.test.ts > "$TEMP/backlink-focused.txt" 2>&1
```
Expected: exit 0.
- [ ] **Step 3: Run PostgreSQL parity when the test database is available**
```bash
bun test test/e2e/engine-parity.test.ts -t "federated sourceIds" --timeout=300000 > "$TEMP/backlink-parity.txt" 2>&1
```
Expected: exit 0; if the configured test database is unavailable, report the exact environmental blocker rather than claiming parity execution.
- [ ] **Step 4: Typecheck and inspect the final diff**
```bash
bun run typecheck > "$TEMP/backlink-typecheck.txt" 2>&1
```
Expected: exit 0. Then run `git diff --check` and confirm no version, schema, migration, deployment, or conditional-write files changed.
@@ -0,0 +1,184 @@
# Scalar-source backlink validation design
## Problem
A page identity in a multi-source brain is `(source_id, slug)`, but the back-link validator currently reasons only about `slug`.
For an outbound edge:
```text
(source-a, concepts/origin) -> (source-a, people/target)
```
the validator accepts any reverse row whose bare slugs are:
```text
people/target -> concepts/origin
```
That can incorrectly accept a row ending at `(default, concepts/origin)` instead of `(source-a, concepts/origin)`.
The bug is not that scalar `getLinks(slug, { sourceId })` permits cross-source destinations. That behavior is intentional: scalar scope qualifies the near/from endpoint while trusted local callers retain visibility into explicit cross-source edges. The gap is that a returned `Link` does not carry the source identity of either endpoint, so callers cannot distinguish same-slug pages.
## Reproduction and evidence
A deterministic PGLite reproduction creates duplicate `concepts/a` and `people/b` pages in `default` and `team-x`, then adds:
```text
(team-x, concepts/a) -> (team-x, people/b)
(team-x, people/b) -> (default, concepts/a)
```
The second edge is not a valid reverse of the first. Nevertheless:
```ts
await engine.getLinks('people/b', { sourceId: 'team-x' })
```
returns the second row, and the current validator accepts it because `to_slug === 'concepts/a'`.
Both engines implement the same scalar rule: filter `f.slug` and `f.source_id`, join the actual destination by `to_page_id`, and do not filter `t.source_id`. Federated `sourceIds` is a separate branch that constrains all visible endpoints and takes precedence over scalar scope.
## Goals
1. Validate back-links by exact source-qualified endpoint identity.
2. Preserve explicit cross-source links for trusted scalar reads.
3. Preserve federated all-endpoint containment and `sourceIds` precedence.
4. Keep PostgreSQL and PGLite behavior identical.
5. Add strict red-before-green regressions using duplicate slugs across sources.
6. Avoid schema migrations and production operational changes.
## Non-goals
- Changing scalar link reads to same-source-only reads.
- Weakening or widening federated reads.
- Changing link write identity or database schema.
- Refactoring the atomic conditional-write branch.
- Coupling deployment, restart, or migration mechanics to the backlink code change. Release operations are handled separately after verification.
## Chosen approach
Extend the engine `Link` result with endpoint source identities and use those fields in the validator.
```ts
interface Link {
from_slug: string;
from_source_id: string;
to_slug: string;
to_source_id: string;
// existing fields
origin_slug?: string | null;
origin_source_id?: string | null;
}
```
All `getLinks` and `getBacklinks` query branches in PostgreSQL and PGLite will project the source IDs from the pages already joined as `f`, `t`, and `o`. No filtering behavior changes.
This approach is preferred over a dedicated `hasExactLink` method because it keeps source identity attached to the link data everywhere, avoids duplicate engine SQL and per-edge existence queries, and matches existing source-qualified link-write and batch-row contracts.
Validator-only raw SQL is rejected because validators should consume the `BrainEngine` contract rather than bypass it with engine-specific schema knowledge.
## Engine semantics
The existing three read modes remain unchanged.
### Unscoped
`getLinks(slug)` returns rows from all same-slug from-pages across sources. Each row identifies the actual source of both endpoints.
### Scalar source
`getLinks(slug, { sourceId })` matches exactly `(sourceId, slug)` on the from side. A destination may belong to another source, and `to_source_id` reveals that exact identity.
The corresponding scalar `getBacklinks` rule continues to match the exact destination/to-page identity while allowing a cross-source referrer.
### Federated sources
`getLinks(slug, { sourceIds })` continues to constrain from and to endpoints to the grant. The origin join continues to redact an out-of-grant origin. `sourceIds` continues to take precedence over scalar `sourceId`.
Adding source IDs to returned in-grant endpoints does not disclose anything new: the existing result already discloses those pages' slugs and edges. An out-of-grant endpoint remains absent.
## Validator algorithm
The validator receives the source scope associated with the page being validated.
For every outbound edge:
```text
(from_source_id, from_slug) -> (to_source_id, to_slug)
```
it requires a reverse row:
```text
(to_source_id, to_slug) -> (from_source_id, from_slug)
```
Duplicate edge rows are deduplicated by the full endpoint pair `(from_source_id, from_slug, to_source_id, to_slug)`, not by bare target slug. This preserves separate reverse requirements when multiple same-slug origin pages point to one exact target.
For each target:
1. Read target outbound links using the target's exact scalar source when validation is scalar-scoped.
2. Under federated validation, retain the caller's `sourceIds` grant rather than converting it to scalar scope.
3. Accept only a returned row whose `from_source_id`, `from_slug`, `to_source_id`, and `to_slug` exactly match the expected reverse identity.
4. Emit the existing warning when no exact reverse exists.
This preserves legitimate cross-source pairs. For example:
```text
(source-a, concepts/origin) -> (source-b, people/target)
(source-b, people/target) -> (source-a, concepts/origin)
```
is valid.
## Validation context propagation
`PageValidationContext` must carry the relevant scalar or federated source scope. The writer and post-write lint paths must load the page with that scope and pass the same scope to nested validator reads.
This change is scoped to source routing needed by validation. It does not modify conditional-write revision or conflict semantics and must not be applied to the atomic conditional-write branch.
## Testing strategy
### PGLite strict-TDD regression
Add duplicate pages across `default` and a second source, then prove before the production fix that:
1. A forward edge in the second source plus a wrong-source reverse produces a warning.
2. Adding the exact reverse removes the warning.
3. A legitimate cross-source forward/reverse pair passes.
4. Two same-slug destination pages are not collapsed into one target identity.
The first assertion must fail against the pre-fix implementation.
### Engine contract tests
For PGLite and PostgreSQL:
1. Assert link rows expose exact from/to source IDs.
2. Assert scalar reads still return explicit cross-source destinations.
3. Assert federated reads still exclude out-of-grant endpoints.
4. Assert `sourceIds` still takes precedence over scalar `sourceId`.
5. Assert origin source identity is null when the origin is redacted by the federated branch.
### Parity and focused verification
Run:
- the focused backlink validator test;
- source-isolation and federated link tests;
- the Postgres/PGLite parity fixture with a test database;
- related writer/post-write tests;
- `bun run typecheck`.
Capture complete command output to files before inspecting summaries. Do not use production databases or restart the live service.
## Compatibility
The `Link` change is additive at runtime. Existing consumers that read only slug or provenance fields continue to work. TypeScript object literals typed as complete `Link` values may need source fields; if compatibility pressure is high, the source fields can initially be optional in the public type while engine implementations and validator tests require their presence. The preferred contract is required endpoint source IDs because every persisted link always has both pages and therefore both source IDs.
No schema migration is required because source IDs already live on the joined `pages` rows.
## Operational constraints
The implementation phase does not deploy, restart GBrain, run production migrations, or alter the atomic conditional-write branch. Release, migration, and restart operations are a separate verified workflow and do not change this design's engine or validator semantics.
+3 -1
View File
@@ -1196,7 +1196,9 @@ const put_page: Operation = {
let writerLint: { error_count: number; warning_count: number } | { skipped: string } | undefined;
try {
const { runPostWriteLint } = await import('./output/post-write.ts');
const lint = await runPostWriteLint(ctx.engine, result.slug);
const lint = await runPostWriteLint(ctx.engine, result.slug, {
sourceId: ctx.sourceId ?? 'default',
});
if (lint.ran) {
writerLint = {
error_count: lint.findings.filter(f => f.severity === 'error').length,
+12 -1
View File
@@ -38,6 +38,10 @@ export interface PostWriteLintOpts {
force?: boolean;
/** Skip file writes; used by tests. */
noLog?: boolean;
/** Exact scalar source for the page and nested validation reads. */
sourceId?: string;
/** Federated read scope; when non-empty, takes precedence over sourceId. */
sourceIds?: string[];
}
export interface PostWriteLintResult {
@@ -80,7 +84,12 @@ export async function runPostWriteLint(
return { ran: false, slug, findings: [], skippedReason: 'flag_disabled' };
}
const page = await engine.getPage(slug);
const sourceOpts = opts.sourceIds && opts.sourceIds.length > 0
? { sourceIds: opts.sourceIds }
: opts.sourceId
? { sourceId: opts.sourceId }
: undefined;
const page = await engine.getPage(slug, sourceOpts);
if (!page) {
return { ran: false, slug, findings: [], skippedReason: 'page_not_found' };
}
@@ -97,6 +106,8 @@ export async function runPostWriteLint(
timeline: page.timeline,
frontmatter: page.frontmatter ?? {},
engine,
sourceId: opts.sourceId,
sourceIds: opts.sourceIds,
};
const findings: ValidationFinding[] = [];
+31 -10
View File
@@ -23,25 +23,46 @@ export const backLinkValidator: PageValidator = {
async validate(ctx: PageValidationContext): Promise<ValidationFinding[]> {
const findings: ValidationFinding[] = [];
const federatedSourceIds = ctx.sourceIds && ctx.sourceIds.length > 0
? ctx.sourceIds
: undefined;
const outboundOpts = federatedSourceIds
? { sourceIds: federatedSourceIds }
: ctx.sourceId
? { sourceId: ctx.sourceId }
: undefined;
const outbound = await ctx.engine.getLinks(ctx.slug);
const outbound = await ctx.engine.getLinks(ctx.slug, outboundOpts);
if (outbound.length === 0) return findings;
// Iron Law: if ctx.slug → target, target must ALSO link back to ctx.slug.
// We check target's outbound links; if none of them point at ctx.slug,
// the back-link is missing.
const uniqueTargets = new Set<string>();
for (const link of outbound) uniqueTargets.add(link.to_slug);
// A federated lookup can return same-slug origins and targets from several
// sources. Deduplicate only identical endpoint pairs; every distinct origin
// still needs its own exact reverse.
const uniqueEdges = new Map<string, typeof outbound[number]>();
for (const link of outbound) {
uniqueEdges.set(
`${link.from_source_id}\0${link.from_slug}\0${link.to_source_id}\0${link.to_slug}`,
link,
);
}
for (const target of uniqueTargets) {
const targetOutbound = await ctx.engine.getLinks(target);
const hasReverse = targetOutbound.some(l => l.to_slug === ctx.slug);
for (const target of uniqueEdges.values()) {
const targetOpts = federatedSourceIds
? { sourceIds: federatedSourceIds }
: { sourceId: target.to_source_id };
const targetOutbound = await ctx.engine.getLinks(target.to_slug, targetOpts);
const hasReverse = targetOutbound.some(link =>
link.from_source_id === target.to_source_id
&& link.from_slug === target.to_slug
&& link.to_source_id === target.from_source_id
&& link.to_slug === target.from_slug
);
if (!hasReverse) {
findings.push({
slug: ctx.slug,
validator: 'back-link',
severity: 'warning',
message: `Outbound link to ${target} has no back-link (${target} does not reference ${ctx.slug}). runAutoLink should reconcile this on next put_page; flag for inspection.`,
message: `Outbound link to ${target.to_slug} has no back-link (${target.to_slug} does not reference ${ctx.slug}). runAutoLink should reconcile this on next put_page; flag for inspection.`,
});
}
}
+7 -2
View File
@@ -62,9 +62,14 @@ export const linkValidator: PageValidator = {
linkPositions.set(slug, list);
}
// Batch-check which targets exist.
// Batch-check which targets exist within the validation read scope.
const sourceOpts = ctx.sourceIds && ctx.sourceIds.length > 0
? { sourceIds: ctx.sourceIds }
: ctx.sourceId
? { sourceId: ctx.sourceId }
: undefined;
for (const slug of internalTargets) {
const page = await ctx.engine.getPage(slug);
const page = await ctx.engine.getPage(slug, sourceOpts);
if (page) continue;
const positions = linkPositions.get(slug) ?? [];
for (const pos of positions) {
+16 -2
View File
@@ -93,6 +93,10 @@ export interface PageValidationContext {
timeline: string;
frontmatter: Record<string, unknown>;
engine: BrainEngine;
/** Exact scalar source for source-qualified validation reads. */
sourceId?: string;
/** Federated read scope; when non-empty, takes precedence over sourceId. */
sourceIds?: string[];
}
// ---------------------------------------------------------------------------
@@ -249,7 +253,9 @@ export class BrainWriter {
// Validators run before the outer transaction commits.
if (strict !== 'off') {
report = await runValidators(txEngine, validators, tx.touchedSlugs);
report = await runValidators(txEngine, validators, tx.touchedSlugs, {
sourceId: 'default',
});
// `ctx.logger.info` would be nice but keep validator behavior uniform
// regardless of strict/lint mode. Caller inspects the report.
if (strict === 'strict' && report.errorCount > 0) {
@@ -281,11 +287,17 @@ async function runValidators(
engine: BrainEngine,
validators: PageValidator[],
touchedSlugs: Set<string>,
scope: { sourceId?: string; sourceIds?: string[] } = {},
): Promise<ValidationReport> {
const findings: ValidationFinding[] = [];
const sourceOpts = scope.sourceIds && scope.sourceIds.length > 0
? { sourceIds: scope.sourceIds }
: scope.sourceId
? { sourceId: scope.sourceId }
: undefined;
for (const slug of touchedSlugs) {
const page = await engine.getPage(slug);
const page = await engine.getPage(slug, sourceOpts);
if (!page) continue; // could have been deleted in this tx
// Grandfather opt-out
@@ -298,6 +310,8 @@ async function runValidators(
timeline: page.timeline,
frontmatter: page.frontmatter ?? {},
engine,
sourceId: scope.sourceId,
sourceIds: scope.sourceIds,
};
for (const v of validators) {
+24 -12
View File
@@ -2901,9 +2901,11 @@ export class PGLiteEngine implements BrainEngine {
// Remote MCP clients always land here.
if (opts?.sourceIds && opts.sourceIds.length > 0) {
const { rows } = await this.db.query(
`SELECT f.slug as from_slug, t.slug as to_slug,
`SELECT f.slug as from_slug, f.source_id as from_source_id,
t.slug as to_slug, t.source_id as to_source_id,
l.link_type, l.context, l.link_source,
o.slug as origin_slug, l.origin_field
o.slug as origin_slug, o.source_id as origin_source_id,
l.origin_field
FROM links l
JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id
@@ -2919,9 +2921,11 @@ export class PGLiteEngine implements BrainEngine {
// opts.sourceId, scope to that source (D20).
if (opts?.sourceId) {
const { rows } = await this.db.query(
`SELECT f.slug as from_slug, t.slug as to_slug,
`SELECT f.slug as from_slug, f.source_id as from_source_id,
t.slug as to_slug, t.source_id as to_source_id,
l.link_type, l.context, l.link_source,
o.slug as origin_slug, l.origin_field
o.slug as origin_slug, o.source_id as origin_source_id,
l.origin_field
FROM links l
JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id
@@ -2932,9 +2936,11 @@ export class PGLiteEngine implements BrainEngine {
return rows as unknown as Link[];
}
const { rows } = await this.db.query(
`SELECT f.slug as from_slug, t.slug as to_slug,
`SELECT f.slug as from_slug, f.source_id as from_source_id,
t.slug as to_slug, t.source_id as to_source_id,
l.link_type, l.context, l.link_source,
o.slug as origin_slug, l.origin_field
o.slug as origin_slug, o.source_id as origin_source_id,
l.origin_field
FROM links l
JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id
@@ -2951,9 +2957,11 @@ export class PGLiteEngine implements BrainEngine {
// foreign referrer nor a foreign origin slug is disclosed to the caller.
if (opts?.sourceIds && opts.sourceIds.length > 0) {
const { rows } = await this.db.query(
`SELECT f.slug as from_slug, t.slug as to_slug,
`SELECT f.slug as from_slug, f.source_id as from_source_id,
t.slug as to_slug, t.source_id as to_source_id,
l.link_type, l.context, l.link_source,
o.slug as origin_slug, l.origin_field
o.slug as origin_slug, o.source_id as origin_source_id,
l.origin_field
FROM links l
JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id
@@ -2966,9 +2974,11 @@ export class PGLiteEngine implements BrainEngine {
// v0.31.8 (D16) + #2200: federated arm above is first; two below mirror getLinks.
if (opts?.sourceId) {
const { rows } = await this.db.query(
`SELECT f.slug as from_slug, t.slug as to_slug,
`SELECT f.slug as from_slug, f.source_id as from_source_id,
t.slug as to_slug, t.source_id as to_source_id,
l.link_type, l.context, l.link_source,
o.slug as origin_slug, l.origin_field
o.slug as origin_slug, o.source_id as origin_source_id,
l.origin_field
FROM links l
JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id
@@ -2979,9 +2989,11 @@ export class PGLiteEngine implements BrainEngine {
return rows as unknown as Link[];
}
const { rows } = await this.db.query(
`SELECT f.slug as from_slug, t.slug as to_slug,
`SELECT f.slug as from_slug, f.source_id as from_source_id,
t.slug as to_slug, t.source_id as to_source_id,
l.link_type, l.context, l.link_source,
o.slug as origin_slug, l.origin_field
o.slug as origin_slug, o.source_id as origin_source_id,
l.origin_field
FROM links l
JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id
+24 -12
View File
@@ -3052,9 +3052,11 @@ export class PostgresEngine implements BrainEngine {
if (opts?.sourceIds && opts.sourceIds.length > 0) {
const ids = opts.sourceIds;
const rows = await tx`
SELECT f.slug as from_slug, t.slug as to_slug,
SELECT f.slug as from_slug, f.source_id as from_source_id,
t.slug as to_slug, t.source_id as to_source_id,
l.link_type, l.context, l.link_source,
o.slug as origin_slug, l.origin_field
o.slug as origin_slug, o.source_id as origin_source_id,
l.origin_field
FROM links l
JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id
@@ -3069,9 +3071,11 @@ export class PostgresEngine implements BrainEngine {
// opts.sourceId, scope the from-page lookup.
if (opts?.sourceId) {
const rows = await tx`
SELECT f.slug as from_slug, t.slug as to_slug,
SELECT f.slug as from_slug, f.source_id as from_source_id,
t.slug as to_slug, t.source_id as to_source_id,
l.link_type, l.context, l.link_source,
o.slug as origin_slug, l.origin_field
o.slug as origin_slug, o.source_id as origin_source_id,
l.origin_field
FROM links l
JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id
@@ -3081,9 +3085,11 @@ export class PostgresEngine implements BrainEngine {
return rows as unknown as Link[];
}
const rows = await tx`
SELECT f.slug as from_slug, t.slug as to_slug,
SELECT f.slug as from_slug, f.source_id as from_source_id,
t.slug as to_slug, t.source_id as to_source_id,
l.link_type, l.context, l.link_source,
o.slug as origin_slug, l.origin_field
o.slug as origin_slug, o.source_id as origin_source_id,
l.origin_field
FROM links l
JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id
@@ -3105,9 +3111,11 @@ export class PostgresEngine implements BrainEngine {
if (opts?.sourceIds && opts.sourceIds.length > 0) {
const ids = opts.sourceIds;
const rows = await tx`
SELECT f.slug as from_slug, t.slug as to_slug,
SELECT f.slug as from_slug, f.source_id as from_source_id,
t.slug as to_slug, t.source_id as to_source_id,
l.link_type, l.context, l.link_source,
o.slug as origin_slug, l.origin_field
o.slug as origin_slug, o.source_id as origin_source_id,
l.origin_field
FROM links l
JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id
@@ -3119,9 +3127,11 @@ export class PostgresEngine implements BrainEngine {
// v0.31.8 (D16) + #2200: federated arm above is first; two below mirror getLinks.
if (opts?.sourceId) {
const rows = await tx`
SELECT f.slug as from_slug, t.slug as to_slug,
SELECT f.slug as from_slug, f.source_id as from_source_id,
t.slug as to_slug, t.source_id as to_source_id,
l.link_type, l.context, l.link_source,
o.slug as origin_slug, l.origin_field
o.slug as origin_slug, o.source_id as origin_source_id,
l.origin_field
FROM links l
JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id
@@ -3131,9 +3141,11 @@ export class PostgresEngine implements BrainEngine {
return rows as unknown as Link[];
}
const rows = await tx`
SELECT f.slug as from_slug, t.slug as to_slug,
SELECT f.slug as from_slug, f.source_id as from_source_id,
t.slug as to_slug, t.source_id as to_source_id,
l.link_type, l.context, l.link_source,
o.slug as origin_slug, l.origin_field
o.slug as origin_slug, o.source_id as origin_source_id,
l.origin_field
FROM links l
JOIN pages f ON f.id = l.from_page_id
JOIN pages t ON t.id = l.to_page_id
+6
View File
@@ -1203,7 +1203,11 @@ export interface CodeEdgeResult {
// Links
export interface Link {
from_slug: string;
/** Exact source identity of the from-page joined by from_page_id. */
from_source_id: string;
to_slug: string;
/** Exact source identity of the to-page joined by to_page_id. */
to_source_id: string;
link_type: string;
context: string;
/**
@@ -1221,6 +1225,8 @@ export interface Link {
* multiple pages reference the same (from, to, type) tuple.
*/
origin_slug?: string | null;
/** Exact source identity of origin_slug; null when absent or grant-redacted. */
origin_source_id?: string | null;
/**
* The frontmatter field name that created this edge (e.g. 'key_people',
* 'investors'). Used for debug output and the `unresolved` response list.
+50 -12
View File
@@ -855,23 +855,61 @@ describeBoth('Engine parity — federated sourceIds[] secondary reads (#2200)',
expect(pg).toEqual(['beta-tag']); // default decoy excluded
});
function exactLinkShape(links: Awaited<ReturnType<BrainEngine['getLinks']>>): string[] {
return links.map(link => [
link.from_source_id,
link.from_slug,
link.to_source_id,
link.to_slug,
link.origin_source_id ?? null,
link.origin_slug ?? null,
link.link_type,
].join('::')).sort();
}
test('getLinks identical under sourceIds[] (all three endpoints scoped)', async () => {
const pg = (await pgEngine.getLinks('fed/doc', grant)).map(l => l.to_slug).sort();
const pglite = (await pgliteEngine.getLinks('fed/doc', grant)).map(l => l.to_slug).sort();
expect(pg).toEqual(pglite);
expect([...new Set(pg)]).toEqual(['fed/target']); // far-endpoint 'fed/outside' excluded
// F1: origin_slug nulled identically on both engines when origin is out-of-grant.
const pgOrigins = (await pgEngine.getLinks('fed/doc', grant)).map(l => l.origin_slug ?? null);
const pgliteOrigins = (await pgliteEngine.getLinks('fed/doc', grant)).map(l => l.origin_slug ?? null);
const pgLinks = await pgEngine.getLinks('fed/doc', grant);
const pgliteLinks = await pgliteEngine.getLinks('fed/doc', grant);
expect(exactLinkShape(pgLinks)).toEqual(exactLinkShape(pgliteLinks));
expect([...new Set(pgLinks.map(l => `${l.to_source_id}:${l.to_slug}`))])
.toEqual(['beta:fed/target']); // far-endpoint 'fed/outside' excluded
// F1: origin identity nulls identically when origin is out-of-grant.
const pgOrigins = pgLinks.map(l => [l.origin_source_id ?? null, l.origin_slug ?? null]);
const pgliteOrigins = pgliteLinks.map(l => [l.origin_source_id ?? null, l.origin_slug ?? null]);
expect(pgOrigins.sort()).toEqual(pgliteOrigins.sort());
expect(pgOrigins).not.toContain('fed/outside');
expect(pgOrigins).not.toContainEqual(['default', 'fed/outside']);
});
test('scalar getLinks preserves cross-source destination identity across engines', async () => {
const scalar = { sourceId: 'beta' };
const pg = await pgEngine.getLinks('fed/doc', scalar);
const pglite = await pgliteEngine.getLinks('fed/doc', scalar);
expect(exactLinkShape(pg)).toEqual(exactLinkShape(pglite));
expect(pg).toContainEqual(expect.objectContaining({
from_source_id: 'beta',
from_slug: 'fed/doc',
to_source_id: 'default',
to_slug: 'fed/outside',
}));
});
test('unscoped link reads expose exact endpoint identity across engines', async () => {
const pgLinks = await pgEngine.getLinks('fed/doc');
const pgliteLinks = await pgliteEngine.getLinks('fed/doc');
expect(exactLinkShape(pgLinks)).toEqual(exactLinkShape(pgliteLinks));
expect(pgLinks.every(link => link.from_source_id && link.to_source_id)).toBe(true);
const pgBacklinks = await pgEngine.getBacklinks('fed/doc');
const pgliteBacklinks = await pgliteEngine.getBacklinks('fed/doc');
expect(exactLinkShape(pgBacklinks)).toEqual(exactLinkShape(pgliteBacklinks));
expect(pgBacklinks.every(link => link.from_source_id && link.to_source_id)).toBe(true);
});
test('getBacklinks identical under sourceIds[] (both endpoints scoped)', async () => {
const pg = (await pgEngine.getBacklinks('fed/doc', grant)).map(l => l.from_slug).sort();
const pglite = (await pgliteEngine.getBacklinks('fed/doc', grant)).map(l => l.from_slug).sort();
expect(pg).toEqual(pglite);
expect(pg).toEqual(['fed/target']);
const pg = await pgEngine.getBacklinks('fed/doc', grant);
const pglite = await pgliteEngine.getBacklinks('fed/doc', grant);
expect(exactLinkShape(pg)).toEqual(exactLinkShape(pglite));
expect(pg.map(l => `${l.from_source_id}:${l.from_slug}`)).toEqual(['beta:fed/target']);
});
test('getTimeline identical under sourceIds[]', async () => {
+27 -7
View File
@@ -185,9 +185,16 @@ describe('#2200 get_tags honors the federated grant', () => {
});
describe('#2200 get_links honors the grant and scopes BOTH endpoints (D4A)', () => {
test('[alpha,beta] returns the in-grant beta→beta link', async () => {
test('[alpha,beta] returns the in-grant beta→beta link with exact endpoint identity', async () => {
const links = (await get_links.handler(remoteCtx(['alpha', 'beta']), { slug: 'secret/beta-doc' })) as any[];
expect(links.map(l => l.to_slug)).toContain('secret/beta-target');
const target = links.find(l => l.to_slug === 'secret/beta-target');
expect(target).toBeDefined();
expect(target).toMatchObject({
from_source_id: 'beta',
from_slug: 'secret/beta-doc',
to_source_id: 'beta',
to_slug: 'secret/beta-target',
});
});
test('[alpha,beta] does NOT leak the beta→default far-endpoint link', async () => {
@@ -205,8 +212,9 @@ describe('#2200 get_links honors the grant and scopes BOTH endpoints (D4A)', ()
const links = (await get_links.handler(remoteCtx(['alpha', 'beta']), { slug: 'secret/beta-doc' })) as any[];
const originLeakLink = links.find(l => l.link_type === 'mentions' && l.to_slug === 'secret/beta-target');
expect(originLeakLink).toBeDefined();
// origin page 'default/only-doc' is out of the [alpha,beta] grant → origin_slug nulled.
// origin page 'default/only-doc' is out of the [alpha,beta] grant → origin identity nulled.
expect(originLeakLink.origin_slug ?? null).toBeNull();
expect(originLeakLink.origin_source_id ?? null).toBeNull();
expect(links.map(l => l.origin_slug)).not.toContain('default/only-doc');
});
@@ -219,18 +227,30 @@ describe('#2200 get_links honors the grant and scopes BOTH endpoints (D4A)', ()
expect(links.map(l => l.origin_slug)).not.toContain('default/only-doc'); // origin too
});
test('D1: TRUSTED local CLI (remote=false) with a scalar scope keeps the cross-source view', async () => {
test('D1: TRUSTED local CLI scalar scope keeps cross-source view and identifies both endpoints', async () => {
// reconcileLinks / validators depend on this — local CLI sees cross-source links.
const ctx = ctxOf({ remote: false, sourceId: 'beta', auth: undefined });
const links = (await get_links.handler(ctx, { slug: 'secret/beta-doc' })) as any[];
expect(links.map(l => l.to_slug)).toContain('default/only-doc'); // cross-source visible for trusted local
const crossSource = links.find(l => l.to_slug === 'default/only-doc');
expect(crossSource).toMatchObject({
from_source_id: 'beta',
from_slug: 'secret/beta-doc',
to_source_id: 'default',
to_slug: 'default/only-doc',
});
});
});
describe('#2200 get_backlinks honors the grant and scopes BOTH endpoints (D4A)', () => {
test('[alpha,beta] returns the in-grant beta→beta backlink', async () => {
test('[alpha,beta] returns the in-grant beta→beta backlink with exact endpoint identity', async () => {
const back = (await get_backlinks.handler(remoteCtx(['alpha', 'beta']), { slug: 'secret/beta-doc' })) as any[];
expect(back.map(l => l.from_slug)).toContain('secret/beta-target');
const referrer = back.find(l => l.from_slug === 'secret/beta-target');
expect(referrer).toMatchObject({
from_source_id: 'beta',
from_slug: 'secret/beta-target',
to_source_id: 'beta',
to_slug: 'secret/beta-doc',
});
});
test('[alpha,beta] does NOT leak the default→beta far-referrer backlink', async () => {
+64
View File
@@ -127,4 +127,68 @@ describe('runPostWriteLint', () => {
expect(r.ran).toBe(true);
expect(r.findings).toEqual([]);
});
test('nested backlink reads preserve the exact non-default source', async () => {
await engine.executeRaw(
`INSERT INTO sources (id, name) VALUES ('team-x', 'team-x') ON CONFLICT (id) DO NOTHING`,
);
for (const sourceId of ['default', 'team-x']) {
await engine.putPage('notes/source-backlink', {
type: 'note', title: `Origin ${sourceId}`,
compiled_truth: '## See Also\n- [Source: X/origin, 2026-04-18](https://x.com/origin/1)',
frontmatter: {},
}, { sourceId });
await engine.putPage('people/target', {
type: 'person', title: `Target ${sourceId}`,
compiled_truth: '## See Also\n- [Source: X/target, 2026-04-18](https://x.com/target/1)',
frontmatter: {},
}, { sourceId });
}
await engine.addLink(
'notes/source-backlink', 'people/target', 'forward', 'mentions', 'manual', undefined, undefined,
{ fromSourceId: 'team-x', toSourceId: 'team-x' },
);
await engine.addLink(
'people/target', 'notes/source-backlink', 'wrong reverse', 'mentions', 'manual', undefined, undefined,
{ fromSourceId: 'team-x', toSourceId: 'default' },
);
const r = await runPostWriteLint(engine, 'notes/source-backlink', {
force: true,
noLog: true,
sourceId: 'team-x',
});
expect(r.ran).toBe(true);
expect(r.findings).toContainEqual(expect.objectContaining({
validator: 'back-link',
severity: 'warning',
}));
});
test('nested markdown-link reads do not fall through to a same-slug page in another source', async () => {
await engine.executeRaw(
`INSERT INTO sources (id, name) VALUES ('team-x', 'team-x') ON CONFLICT (id) DO NOTHING`,
);
await engine.putPage('notes/source-link', {
type: 'note', title: 'Team origin',
compiled_truth: '[Target](people/target.md)',
frontmatter: {},
}, { sourceId: 'team-x' });
await engine.putPage('people/target', {
type: 'person', title: 'Default-only target', compiled_truth: 'target', frontmatter: {},
}, { sourceId: 'default' });
const r = await runPostWriteLint(engine, 'notes/source-link', {
force: true,
noLog: true,
sourceId: 'team-x',
});
expect(r.findings).toContainEqual(expect.objectContaining({
validator: 'link',
severity: 'error',
message: expect.stringContaining('people/target'),
}));
});
});
+117
View File
@@ -653,6 +653,123 @@ describe('back-link validator', () => {
});
expect(findings).toEqual([]);
});
async function seedDuplicateBacklinkPages(): Promise<void> {
await engine.executeRaw(
`INSERT INTO sources (id, name) VALUES ('team-x', 'team-x') ON CONFLICT (id) DO NOTHING`,
);
for (const sourceId of ['default', 'team-x']) {
await engine.putPage('concepts/a', {
type: 'concept', title: `a@${sourceId}`, compiled_truth: 'a', frontmatter: {},
}, { sourceId });
await engine.putPage('people/b', {
type: 'person', title: `b@${sourceId}`, compiled_truth: 'b', frontmatter: {},
}, { sourceId });
}
}
async function validateTeamOrigin() {
return await backLinkValidator.validate({
slug: 'concepts/a',
sourceId: 'team-x',
type: 'concept',
compiledTruth: 'a',
timeline: '',
frontmatter: {},
engine,
});
}
test('wrong-source reverse does not satisfy an exact non-default backlink', async () => {
await seedDuplicateBacklinkPages();
await engine.addLink(
'concepts/a', 'people/b', 'forward', 'mentions', 'manual', undefined, undefined,
{ fromSourceId: 'team-x', toSourceId: 'team-x' },
);
await engine.addLink(
'people/b', 'concepts/a', 'wrong-source reverse', 'mentions', 'manual', undefined, undefined,
{ fromSourceId: 'team-x', toSourceId: 'default' },
);
const findings = await validateTeamOrigin();
expect(findings).toHaveLength(1);
expect(findings[0].message).toContain('people/b');
});
test('exact reverse satisfies a non-default backlink', async () => {
await seedDuplicateBacklinkPages();
await engine.addLink(
'concepts/a', 'people/b', 'forward', 'mentions', 'manual', undefined, undefined,
{ fromSourceId: 'team-x', toSourceId: 'team-x' },
);
await engine.addLink(
'people/b', 'concepts/a', 'exact reverse', 'mentions', 'manual', undefined, undefined,
{ fromSourceId: 'team-x', toSourceId: 'team-x' },
);
expect(await validateTeamOrigin()).toEqual([]);
});
test('legitimate explicit cross-source reverse pair passes', async () => {
await seedDuplicateBacklinkPages();
await engine.addLink(
'concepts/a', 'people/b', 'cross-source forward', 'mentions', 'manual', undefined, undefined,
{ fromSourceId: 'team-x', toSourceId: 'default' },
);
await engine.addLink(
'people/b', 'concepts/a', 'cross-source reverse', 'mentions', 'manual', undefined, undefined,
{ fromSourceId: 'default', toSourceId: 'team-x' },
);
expect(await validateTeamOrigin()).toEqual([]);
});
test('same-slug targets in different sources are validated independently', async () => {
await seedDuplicateBacklinkPages();
await engine.addLink(
'concepts/a', 'people/b', 'team target', 'mentions', 'manual', undefined, undefined,
{ fromSourceId: 'team-x', toSourceId: 'team-x' },
);
await engine.addLink(
'concepts/a', 'people/b', 'default target', 'mentions', 'manual', undefined, undefined,
{ fromSourceId: 'team-x', toSourceId: 'default' },
);
await engine.addLink(
'people/b', 'concepts/a', 'reverse only team target', 'mentions', 'manual', undefined, undefined,
{ fromSourceId: 'team-x', toSourceId: 'team-x' },
);
const findings = await validateTeamOrigin();
expect(findings).toHaveLength(1);
expect(findings[0].message).toContain('people/b');
});
test('federated same-slug origins retain every expected reverse identity', async () => {
await seedDuplicateBacklinkPages();
await engine.addLink(
'concepts/a', 'people/b', 'default origin', 'mentions', 'manual', undefined, undefined,
{ fromSourceId: 'default', toSourceId: 'team-x' },
);
await engine.addLink(
'concepts/a', 'people/b', 'team origin', 'mentions', 'manual', undefined, undefined,
{ fromSourceId: 'team-x', toSourceId: 'team-x' },
);
const findings = await backLinkValidator.validate({
slug: 'concepts/a',
sourceId: 'missing-scalar-must-not-win',
sourceIds: ['default', 'team-x'],
type: 'concept',
compiledTruth: 'a',
timeline: '',
frontmatter: {},
engine,
});
expect(findings).toHaveLength(2);
});
});
// ---------------------------------------------------------------------------